Evaluation study on excellent science in the European framework programmes for research and innovation : annexes phase 2 : supporting the interim evaluation of Horizon Europe.
Date: 2024-10-08
Institutions: Directorate-General for Research and Innovation
Subjects: EU research policy, innovation, new technology, open science, performance measurement, regulatory policy, research infrastructure, research programme, research results, research staff, scientific cooperation, scientific research, social impact

Evaluation study on Excellent 
Science in the European 
Framework Programmes for 
Research and Innovation 

Annexes Phase 2 – supporting the interim evaluation 
of Horizon Europe 

 
 
 
 
 
 
Evaluation  study  on  Excellent  Science  in  the  European  Framework  Programmes  for 
Research  and  Innovation  –  Annexes  Phase  2  –  supporting  the  interim  evaluation  of 
Horizon Europe 

European Commission 
Directorate-General for Research and Innovation 
Directorate G — Common Policy Centre 
Unit G2 — Common Programme Analysis & Regulatory Reform 
Contact   Daniel NEICU (DG RTD.G2) 
Email  

RTD-G2-SUPPORT@ec.europa.eu 

RTD-PUBLICATIONS@ec.europa.eu 

European Commission 
B-1049 Brussels 

Manuscript completed in February 2024 

This  document  has  been  prepared  for  the  European  Commission,  however  it  reflects  the  views  only  of  the 
authors, and the European Commission shall not be liable for any consequence stemming from the reuse. 

PDF  

ISBN 978-92-68-20838-0 

doi:10.2777/9552959 

KI-01-24-034-EN-N 

Luxembourg: Publications Office of the European Union, 2024 

Source: European Union, 2024 

The reuse policy of European Commission documents is implemented by Commission Decision 2011/833/EU 
of 12 December 2011 on the reuse of Commission documents (OJ L 330, 14.12.2011, p. 39). Unless otherwise 
noted, the reuse of this document is authorised under a Creative Commons Attribution 4.0 International (CC 
BY 4.0) licence (https://creativecommons.org/licenses/by/4.0/). This means that reuse is allowed  provided 
appropriate credit is given and any changes are indicated. 
For any use or reproduction of elements that are not owned by the European Union, permission may need to 
be sought directly from the respective rightholders. The European Union does not own the copyright in relation 
to the following elements: 
Image credits for cover page and throughout: Source: ivector # 235536634 # 249868181 # 251163013 
# 266009682, # 273480523 # 362422833 # 241215668 # 244690530 # 245719946 # 251163053 
# 252508849, 2020. Source: Stock.Adobe.com. 

 
 
 
 
EUROPEAN COMMISSION 

Evaluation study on Excellent 
Science in the European 
Framework Programmes for 
Research and Innovation 

Annexes Phase 2 – supporting the interim evaluation 
of Horizon Europe 

Edited by: 

PPMI 

IDEA Consult 

UNU-MERIT 

2024 

Directorate-General for Research and Innovation 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
CONTENTS 

Annex 1: Analysis and findings per programme part .......................... 8 

1.1. 

1.2. 

1.3. 

1.4. 

1.5. 

1.6. 

Key acronyms and definitions 

Executive summary 

European Research Council (ERC) 

Marie Skłodowska-Curie Actions (MSCA) 

Research Infrastructures (INFRA) 

Widening participation and strengthening the European Research 
Area under Horizon Europe 

1.7. 

Efficiency of all analysed programme parts 

8 

19 

38 

71 

141 

164 

228 

Annex 2: Case studies ........................................................................... 263 

Annex 2.1: Contribution of the ERC to emerging areas in the Life 

Sciences, Physical Sciences and Engineering ............................... 263 

1.1. 
1.2. 
1.3. 
1.4. 
1.5. 

Introduction 

Background information 

Case study findings 

Success factors and barriers 

Conclusions 

263 
264 
265 
274 
276 

Annex 2.2: Contribution of the ERC to emerging areas in the 

Social Sciences and Humanities ...................................................... 281 

1.1. 
1.2. 
1.3. 
1.4. 
1.5. 

Introduction 

Background information 

Case study findings 

Success factors and barriers 

Conclusions 

281 
282 
283 
292 
294 

Annex 2.3: The structuring impacts of the MSCA ............................... 300 

1.1. 
1.2. 
1.3. 
1.4. 
1.5. 

Introduction 

Background information 

Case study findings 

Reflections on evaluation criteria 

Perceived success factors 

2 

300 
300 
307 
314 
317 

 
 
1.6. 
1.7. 

Perceived barriers 

Conclusions and lessons learned across the evaluation criteria of 
interest for the case study 

318 

319 

Annex 2.4: How the MSCA contribute to more sustainable and 

greener research ............................................................................... 324 

1.1. 
1.2. 
1.3. 
1.4. 
1.5. 

Introduction 

Background information 

Case study findings 

Success factors and barriers 

Conclusions and lessons learned 

324 
324 
329 
341 
343 

Annex 2.5: Analysis of Transnational and Virtual Access support .... 347 

1.1. 
1.2. 
1.3. 
1.4. 
1.5. 

Introduction 

Background information 

Case study findings 

Success factors and barriers 

Conclusions 

347 
347 
349 
359 
360 

Annex 2.6: Analysis of the use of the challenge-oriented 

approach in the INFRA Work Programmes ..................................... 363 

1.1. 
1.2. 
1.3. 
1.4. 
1.5. 

Introduction 

Background information 

Case study findings 

Success factors and barriers 

Conclusions and lessons learned across the evaluation criteria of 
interest to this case study 

363 
363 
365 
375 

376 

Annex 2.7: Impact of the widening actions on inducing structural 

change ............................................................................................... 381 

1.1. 
1.2. 
1.3. 
1.4. 
1.5. 

Introduction 

Background information 

Case study findings 

Success factors and barriers 

Conclusions 

381 
382 
386 
393 
394 

Annex 2.8: WIDERA Actions on enhancing the European RI 

system ................................................................................................ 401 

1.1. 

Introduction 

401 

3 

 
1.2. 
1.3. 
1.4. 
1.5. 

Background information 

Key case study findings 

Success factors and barriers 

Conclusions and lessons learned 

401 
407 
420 
421 

Annex 2.9: Impact of the Framework Programmes on creating 

high-quality new knowledge ............................................................. 425 

1.1. 
1.2. 
1.3. 
1.4. 
1.5. 

Introduction 

Background information 

Case study findings 

Success factors and barriers 

Conclusions and lessons learned across the evaluation criteria of 
interest to the present case study 

425 
425 
427 
464 

468 

Annex 2.10: Strengthening Human Capital in Research and 

Innovation .......................................................................................... 476 

1.1. 
1.2. 
1.3. 
1.4. 

Introduction 

Background information 

Case study findings 

Conclusions 

476 
476 
481 
499 

Annex 2.11: Fostering the diffusion of knowledge and open 

access ................................................................................................ 505 

1.1. 
1.2. 
1.3. 
1.4. 
1.5. 

Introduction 

Background information 

Case study findings 

Success factors and barriers 

Conclusions 

505 
506 
509 
520 
524 

Annex 2.12: Gender equality in the European Framework 

Programmes for Research and Innovation ...................................... 532 

1.1. 
1.2. 
1.3. 
1.4. 
1.5. 

Introduction 

Background information 

Case study findings 

Success factors and barriers 

Conclusions 

532 
532 
536 
551 
553 

Annex 2.13: International cooperation and association ...................... 560 

4 

 
1.1. 
1.2. 
1.3. 
1.4. 
1.5. 

Introduction 

Background information 

Case study findings 

Success factors and perceived barriers 

Conclusions 

560 
560 
564 
601 
603 

Annex 2.14: Case study on the impact of the Framework 

Programmes on fostering research integrity and ethics ................ 658 

1.1. 
1.2. 
1.3. 
1.4. 

Introduction 

Background information 

Case study findings 

Conclusions 

658 
659 
662 
677 

Annex 2.15: Co-design and co-creation of R&I through citizen 

engagement ....................................................................................... 690 

1.1. 
1.2. 
1.3. 
1.4. 
1.5. 

Introduction 

Background information 

Case study findings 

Success factors and perceived barriers 

Conclusions 

690 
690 
694 
720 
721 

Annex 3: Benchmarks ........................................................................... 728 

Annex 3.1: Benchmark on the Swiss National Science 

Foundation ......................................................................................... 728 

1.1. 
1.2. 
1.3. 
1.4. 
1.5. 

1.6. 

1.7. 

1.8. 

Introduction 

Methodology and evaluation questions addressed 

Context and objectives of the benchmark initiative 

Efficiency and effectiveness 

Relevance: balance between the approaches used to fund 
research 

EU added value: the non-association of Switzerland to Horizon 
Europe 

Critical success factors and barriers, and the transferability of 
findings to the EU Framework Programme 

Conclusions 

728 
729 
729 
735 

736 

737 

739 
741 

Annex 3.2: Benchmark on the United Kingdom Research & 

Innovation (UKRI)  ............................................................................. 746 

5 

 
1.1. 
1.2. 
1.3. 
1.4. 
1.5. 

1.6. 
1.7. 
1.8. 

Introduction 

Methodology and evaluation questions addressed 

Context 

Effectiveness: results 

EU added value: association of the United Kingdom to Horizon 
Europe 

Relevance and efficiency: critical success factors and barriers 

Transferability of findings to the EU Framework Programme 

Conclusions 

746 
747 
747 
750 

753 
756 
758 
759 

Annex 3.3: Benchmark on the Austrian Science Fund ........................ 768 

1.1. 
1.2. 
1.3. 
1.4. 
1.5. 
1.6. 
1.7. 

Introduction 

Methodology and evaluation questions addressed 

Context and objectives of the benchmark initiative 

Results 

Critical success factors and barriers 

Transferability of findings to the EU Framework Programmes 

Conclusions 

768 
768 
769 
776 
784 
785 
786 

Annex 3.4: Benchmark on the Australian Research Council .............. 788 

1.1. 
1.2. 
1.3. 
1.4. 
1.5. 
1.6. 
1.7. 

Introduction 

Methodology and evaluation questions addressed 

Context and objectives of the benchmark initiative 

Results 

Critical success factors and barriers 

Transferability of findings to the EU Framework Programme 

Conclusions 

788 
788 
789 
792 
808 
809 
809 

Annex 4: Evidence table by programme part and evaluation 

criteria ................................................................................................ 811 

1.1. 
1.2. 
1.3. 
1.4. 
1.5. 

Relevance 

Coherence 

Efficiency 

Effectiveness 

EU added value 

812 
839 
869 
886 
984 

6 

 
Annex 5: Literature and information sources ...................................... 1003 

Annex 6: Quantitative research track findings .................................... 1080 

1.1. 
1.2. 
1.3. 

1.4. 

1.5. 
1.6. 

Efficiency indicators 

Network analysis 

Research Trend Analysis - Results and Interpretation of Funding 
and Topical Overlap 

Counterfactual analysis/econometric modelling – novelty and 
disruptiveness (ERC and MSCA-IF) 

SDG analysis 

Application and participation patterns 

1080 
1095 

1129 

1146 
1180 
1190 

Annex 7: Synopsis of the Survey programme ..................................... 1214 

7 

 
 
 
 
 
Annex 1: Analysis and findings per programme part 

1.1.  Key acronyms and definitions 

Abbreviations and acronyms 

AI 

Artificial intelligence 

AI4EU 

European AI on Demand Platform and Ecosystem 

ANR 

French National Research Agency 

APC 

Article processing charge 

API 

Application programming interface 

ARC 

Australian Research Council 

ASJC 

All Science Journal Classification 

BRICS 

Brazil, Russia, India, China and South Africa 

CC 

Creative Commons licence 

CERC 

Canada Excellence Research Chairs 

CET 

Clean energy technologies 

CFI 

Canada Foundation for Innovation 

CNR 

National Research Council of Italy 

CNRS 

French National Centre for Scientific Research 

COFUND 

Co-funding of Regional, National and International Programmes (MSCA) 

COST 

European Cooperation in Science & Technology 

CSA 

Coordination and Support Action 

CSIC 

Spanish National Research Council 

8 

 
Abbreviations and acronyms 

CSO 

Civil society organisation 

CWTS 
Leiden 
Ranking 

Centre for Science and Technology Studies (CWTS).  

DG 

Directorate-General 

DG RTD 

Directorate-General for Research and Innovation 

DN 

DOI 

EA 

Doctoral Networks (MSCA, Horizon Europe) 

Digital Object Identifier 

Executive Agency 

EARTO 

European Association of Research and Technology Organisations 

EC 

European Commission 

ECA 

European Court of Auditors 

EDI 

EEI 

EIC 

EIE 

European Data Infrastructure 

European Excellence Initiative 

European Innovation Council 

European Innovation Ecosystems 

EIT  

European Institute of Innovation and Technology  

EIT KIC 

European Institute of Innovation and Technology Knowledge and Innovation 
Community 

EOSC 

European Open Science Cloud 

EPSCoR 

Established Program to Stimulate Competitive Research. Programme of National 
Science Foundation (NSF), US 

EPSRC 

Engineering and Physical Sciences Research Council (UK) 

9 

 
Abbreviations and acronyms 

ERA 

European Research Area 

ERC 

European Research Council 

ERC PI 

ERC principal investigator 

ERCEA 

ERC Executive Agency 

ERDF 

European Regional Development Fund 

ERIC 

European Research Infrastructure Consortium  

ERs 

Experienced researchers 

ESF+ 

European Social Fund Plus 

ESFRI 

European Strategy Forum on Research Infrastructures 

ESI funds 

European Structural and Investment Funds 

ESIF 

European Structural and Investment Funds 

ESRs 

Early-stage researchers 

EU 

European Union 

EU-13 

EU-15 

EU-27 

EU-28 

The 13 new Member States added since 2004: Bulgaria, Croatia, Cyprus, Czechia, 
Estonia, Hungary, Latvia, Lithuania, Malta, Poland, Romania, Slovakia and 
Slovenia.  

The 15 Member States that were members prior to the accession of the new 
candidate countries on 1 May 2004. The EU-15 comprises the following 15 
countries: Austria, Belgium, Denmark, Finland, France, Germany, Greece, Ireland, 
Italy, Luxembourg, the Netherlands, Portugal, Spain, Sweden and the United 
Kingdom. 

The 27 EU Member States: Belgium, Bulgaria, Czechia, Denmark, Germany, 
Estonia, Ireland, Greece, Spain, France, Croatia, Italy, Cyprus, Latvia, Lithuania, 
Luxembourg, Hungary, Malta, the Netherlands, Austria, Poland, Portugal, 
Romania, Slovenia, Slovakia, Finland and Sweden 

The 28 Member States that constituted the membership of the until 2020, and 
included the UK: Belgium, Bulgaria, Czechia, Denmark, Germany, Estonia, 
Ireland, Greece, Spain, France, Croatia, Italy, Cyprus, Latvia, Lithuania, 

10 

 
Abbreviations and acronyms 

Luxembourg, Hungary, Malta, the Netherlands, Austria, Poland, Portugal, 
Romania, Slovenia, Slovakia, Finland, Sweden and the United Kingdom 

EUA 

European University Association 

EUCYS 

EU Contest for Young Scientists 

EUR 

Euro currency 

Euratom 

European Atomic Energy Community (EAEC) 

F2P 

Feedback-to-Policy 

FAIR  

Findability, Accessibility, Interoperability, and Reusability 

FCT 

Portuguese Foundation for Research and Technology 

FET 

Future and emerging technology 

FP 
FSTP 

Framework Programme  
Financial Support to Third Parties 

FTE 

Full-time equivalent 

FWCI 

Field Weighted Citation Index 

FWF 

Austrian Science Fund 

G7 

Group of Seven: Canada, France, Germany, Italy, Japan, the United Kingdom and 
the United States. The EU is non-enumerated member. 

GAP 

Grant Agreement Preparation 

GDP 

Gross domestic product 

GDPR 

General Data Protection Regulation 

GEP 

Gender Equality Plan 

GPT 
H2020 

General-purpose technology 
Horizon 2020 

HSE 

Higher and Secondary Education 

11 

 
Abbreviations and acronyms 

HLEG 

High Level Expert Group 

HPC 

High-Performance Computing 

ICO 

Industrial Contact Officer 

ICT 

IDT 

IF 

ILO 

Information and communications technology 

Innovative Doctoral Training 

Individual Fellowships (MSCA) 

Industrial Liaison Officer 

IMPRS 

International Max Planck Research Schools 

INFRA 

Research Infrastructures (Programme Part in Horizon 2020) 

INFRA 

Research Infrastructures (Programme Part in Horizon 2020 and Horizon Europe) 

INTERREG 

European Territorial Cooperation instrument 

IoT 

ITN 

JPI 

Internet of Things 

Innovative Training Networks; activity in the Marie Skłodowska-Curie Actions 

Joint Programming Initiative 

JRC 

Joint Research Centre 

JSTA 

Japan Science and Technology Agency 

JU 

KIP 

KPI 

Joint Undertaking 

Key Impact Pathway 

Key Performance Indicator 

LEIT 

Leadership in Enabling and Industrial Technologies 

12 

 
Abbreviations and acronyms 

LGBTI 

Lesbian, Gay, Bisexual, Transgender and Intersex 

MCAA 
MGA 

Marie Curie Alumni Association 
Model Grant Agreement 

MLE 

Mutual Learning Exercise 

MS 

Member State 

MSCA 

Marie Skłodowska-Curie Actions 

MSCA 
COFUND 

MSCA Co-funding of regional, national and international programmes 

MSCA DN 

MSCA Doctoral Networks 

MSCA IF 

MSCA Individual Fellowships 

MSCA PF 

MSCA Postdoctoral Fellowships 

MSCA SE 

MSCA Staff Exchanges 

NGO 

Non-governmental organisation 

NIH 

National Institutes of Health (US) 

NNSFC 

National Natural Science Foundation of China 

NPC 

National Contact Point 

NQIA 

National Quantum Initiative Act 

NREN 

National Research and Education Network 

NRRP 

National Recovery and Resilience Plan 

NSF 

National Science Foundation (US) 

NWO 

Dutch Organisation for Scientific Research (NL) 

OECD 

Organisation for Economic Co-operation and Development 

13 

 
 
Abbreviations and acronyms 

OPC 

Open public consultation 

OpenAIRE 

An open scholarly communication infrastructure that captures Horizon 2020 
publications, datasets and other research outputs 

ORCID 

Open Researcher and Contributor ID – a unique, free-of-charge persistent 
identifier for researchers 

ORDP 
OTH 

The Open Research Data Pilot 
Societal engagement projects and other non-traditional research organisations 

P2P 
PF 

PoC 

Public-public partnerships 
Postdoctoral Fellowship (MSCA action) 

Proof of Concept 

PPP 

Public-private partnerships 

PPTop1% 
PRC 

Top 1% most highly cited publications in a given disciplinary field 
Private (for-profit) research centre 

PSF 
PUB 

QC 

Policy Support Facility 
Public bodies 

Quantum computing 

QCN 

Quantum Community Network 

R&D  

Research and development  

R&I 

Research and innovation 

RDI  

Research, development and innovation 

REA 
REC 

Research Executive Agency  
Research organisations 

RI 

Research infrastructure  

RIA 

Research and Innovation Action 

RII 

Research Infrastructure Improvement (components of the NSF EPSCoR 
programme) 

RISE 

Research and Innovation Staff Exchanges (MSCA) 

14 

 
Abbreviations and acronyms 

RRF 

Recovery and Resilience Facility 

RRI 

Responsible research and innovation 

RTO 

Research and technology organisation 

S&T 

Science and technology  

SC 

Societal Challenge 

SDG 

Sustainable Development Goal 

SE 

Staff Exchange (MSCA action) 

SEWP 

Spreading Excellence and Widening Participation (programme part of Horizon 
2020) 

SiS 

Science in Society (Programme Part of FP7) 

SMEs 

Small and medium-sized enterprises 

SoE 

Seal of Excellence 

SRIA 

Strategic Research and Innovation Agenda 

SSAH 

Social Science, Arts and Humanities 

SSH 

Social Science and Humanities 

STEM 

Science, Technology, Engineering and Mathematics  

STI 

Science, Technology and Innovation 

SwafS 

Science with and for Society (programme part of Horizon 2020) 

TNA 

Transnational Access 

TNA/VA 

Transnational Access and Virtual Access financial instruments 

ToRR 

The European Commission’s ‘Tracking of Research Results’ tool 

15 

 
Abbreviations and acronyms 

TRL 

Technology Readiness Levels 

TTG 

Time-to-grant 

TTI 

UK 

Time-to-inform 

United Kingdom 

UKRI 

UK Research and Innovation 

URI 

Dataset with a persistent identifier and an identifier to the data file 

USD 

US dollar 

VA 

Virtual Access 

VAT 

Value-added tax 

WF 

Widening Fellowship 

WIDERA 

Widening Participation and Strengthening the European Research Area under 
Horizon Europe 

WIRE 

Yearly ‘Week of Innovative Regions in Europe’ Conferences  

Definitions 

Applicant 

Associated 
country1 

A researcher or organisation that has applied for a grant or other funding from 
the EC. An applicant may be successful or unsuccessful, i.e. they received 
funding or not. 

For the purposes of this analysis, the associated countries are taken to be 
Albania, Armenia, Bosnia and Herzegovina, the Faroe Islands, Georgia, 
Iceland, Israel, Kosovo, North Macedonia, Moldova, Montenegro, Morocco, 
New Zealand, Norway, Serbia, Tunisia, Turkey, Ukraine and the United 
Kingdom. 

Beneficiary 

A researcher or organisation that has received FP funding 

1 https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/common/guidance/list-
3rd-country participation_horizon-euratom_en.pdf 

16 

 
 
 
Definitions 

Betweenness 
centrality 

In network analysis, betweenness centrality is the ratio of the shortest paths 
that go through the node over the total shortest paths between two nodes. 

Closeness 
centrality 

In network analysis, closeness centrality averages the shortest path of a given 
node to every other node in the network. 

Degree 
centrality 

In network analysis, degree centrality indicates how many direct connections, 
or ‘edges’, a node has. 

Disruptiveness  We compute five correlated indicators of disruptiveness. These indicators are 

all based on the same rationale: a publication in focus is disruptive if a large 
number of subsequent publications cite it, but do not cite its references. In 
other words; the paper in focus, cited by many papers that do not cite its own 
reference, is a knowledge gatekeeper between older and new knowledge. 

Edge 

In network analysis, an edge is a relationship between two nodes. The exact 
nature of the relationship depends on the analysis used. For example, in social 
media, the edge could indicate that two people follow one another. In the 
context of Horizon Europe, an edge can be a project that two individuals both 
worked on. 

Emergence 
Score 

The Emergence Score looks at four S&T attributes – novelty, persistence, 
community and growth – and seeks to identify emergent terms, i.e. topical 
content that evidences these four attributes. 

High-quality 
proposal 

Node 

Novelty 

A proposal that surpasses the 70% quality threshold during evaluation. 

In network analysis, a node represents the individual of interest. For example, 
within the context of Horizon Europe, a node could be a researcher, an 
organisation or a programme part. 

The novelty indicator identifies publications that cite documents from different 
scientific communities, i.e. those which combine different topics in novel ways 
that have not previously been combined. We calculated this measure on the 
basis of combinations of journals extracted from the reference lists of the 
papers in focus. First, the algorithm produces a co-occurrence matrix of 
papers and journals by year. Then, it applies the Louvain network community 
algorithm to identify clusters of journals by year: these are scientific 
communities, i.e. groups of scientists that work on similar topics and cite each 
other. 

Participation 

The number of participations indicates the number of organisations that are 
involved in the selected project(s). One organisation participating in N projects 
is counted N times. 

Participating 
organisation 

An organisation that has participated in any project in any capacity under the 
specified Framework Programme. 

Prominence 
Percentile 

The Prominence Percentile is an indicator of the momentum or visibility of a 
particular topic. Prominence does not signify ‘importance’. Due to the nature of 
certain research fields, there are topics that will never become ‘prominent’. 

17 

 
Definitions 

However, this does not mean that such topics are not important. However, 
there is a correlation between the prominence (momentum) of a particular 
topic and the amount of funding per publication within that topic. On average, 
the higher the momentum, the more money per publication is available for 
research into that topic. 

Publications 

Publications in Annex 6 are:  
Those publications reported by beneficiaries to the EC, and matched and 
validated in Scopus 
Those publications retrieved from Scopus which acknowledge having received 
funding from either the EC or other funding bodies analysed.  

Top 1% most 
networked 

Our study team used network analysis to find the top 1% most-networked 
organisations, as defined by the organisations which had the highest 
closeness centrality score. 

Top 
Percentiles 

The Top Percentiles indicator looks at the impact of science, and measures 
excellent science by using the citations accrued to calculate the number and 
share of peer-reviewed publications resulting from projects funded by the EC 
funding programme parts, and which are core contributions to scientific fields, 
calculated as a percentage of the top-cited publications (PPTop) 

Topical Triad 

Triads, or three node subgraphs, are the structural foundation of networks. For 
use in the analysis in this report, we have constructed triads based on closely 
connected keywords, whereby these triads then represent topics. 

18 

 
 
 
1.2.  Executive summary  

European Research Council (ERC) 

As  it  was  under  Horizon  2020,  the  main  mission  of  the  ERC  under  Horizon  Europe  is  to 
‘encourage the highest quality research in Europe through competitive funding and to support 
investigator-driven  frontier  research  across  all  fields,  based  on  scientific  excellence’.  The 
ERC remains  under  Pillar 1,  ‘Excellent  Science’,  and  no  fundamental  changes  have  been 
made to it. The ERC has an indicative budget of EUR 15.027 billion for 2021-2027, which 
makes it the largest programme part under Horizon Europe. The fundamental activity of the 
ERC is ‘to provide attractive, long-term funding to support excellent investigators and their 
research teams to pursue ground-breaking, high-risk/high-gain research’2. 

Effectiveness  

The  effectiveness  of  the  ERC  programme  part  within  Horizon  Europe  is  assessed  across 
several  dimensions,  including  the  advancement  of  the  frontiers  of  knowledge,  impact  on 
European  researchers'  careers,  the  programme  part’s  impact  on  the  attractiveness  of  the 
European research system, and the extent to which it drives commercial and societal impact. 

•  Advancing frontier research as well as the scientific and technological impacts of 
new ideas in the short and long term: the ERC plays a crucial role in advancing frontier 
research.  The  bibliometric  analysis  carried  out  in  the  Evaluation  study  on  Excellent 
Science under Horizon 2020shows that ERC-funded research under Horizon 2020 has 
generally resulted in both a high number of publications and a high average number of 
citations.  Within  STEM,  ERC-funded  research  has  contributed  the  most  in  terms  of 
scientific  production  in  the  fields  of  physics,  astronomy  and  engineering  (such  as 
perovskite solar cells). Within SSH, the main fields in terms of scientific production were 
social sciences and historical studies. In terms of scientific publication citations in STEM-
fields,  the  ERC  has  contributed  most  to  the  fields  of  biology,  clinical  medicine  and 
chemistry; in SSH, its largest contribution has been to the fields of social sciences and 
economics and business. Although it is still too early to assess the scientific impacts of 
Horizon Europe projects, it is expected that the ERC under Horizon Europe will display 
similar trends to Horizon 2020 in this respect, given the continuity of the programme part 
between both FPs. In the survey of Horizon Europe ERC beneficiaries carried out for his 
study, almost all respondents stated that their ERC project has produced or is likely to 
produce  research  publications  (in  the  form  of  academic  articles,  monographs,  books, 
etc.).  The  majority  of  respondents  also  expected  to  produce  new/improved  research 
methodologies,  new  large-scale  datasets,  and  new  or  improved  tools,  methods  or 
techniques.  

• 

Improving the career prospects of the European researchers and training the new 
generation of researchers: both the survey and the interviews with ERC beneficiaries 
confirmed the strong impacts that obtaining an ERC grant had on PIs and their research 
teams.  The  main  impacts  include  strengthening  their  reputation  as  a  researcher;  the 
capacity to conduct frontier research; and the possibility to establish and consolidate their 
own  research  group.  In  addition,  according  to  the  survey,  ERC  grants  give  PIs  the 
opportunity to improve their skills, mostly in project and people management, as well as 

2 ERC Work Programme 2023. Available at: 
 https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2023/wp_horizon-erc-2023_en.pdf.  

19 

 
 
in scientific methods and/or techniques. Intersectoral mobility among ERC beneficiaries 
is very limited; however, international mobility opportunities are highly appreciated.  

•  Reforming, strengthening and unlocking the full potential and attractiveness of the 
European research system: the ERC plays an important role in making the EU research 
system more attractive to both European and non-European researchers. Evidence from 
ERC  monitoring  data  shows  the  significant  participation  of  non-EU  beneficiaries.  In 
addition, interviews with ERC beneficiaries confirmed the attractiveness of the instrument 
and its relevance for retaining excellent researchers within Europe. The main factors in 
this regard (mentioned both in the interviews and the open consultation) relate to the size 
of the funding available, the flexibility it allows to pursue ground-breaking research, and 
the length of projects. However, participation by the private for-profit sector remains very 
limited.  

•  Transforming the knowledge created into commercial and social innovations, generating 
economic  growth and  tackling  societal  challenges:  in  addition  to  scientific  publications, 
analysis  of  Horizon  2020  data  shows  that  ERC-funded  research  also  generates  a 
significant  number  of  patent  applications  (based  on  Horizon  Dashboard  data).  Other 
analysis carried out on Horizon 2020 ERC projects shows that the ERC made a significant 
contribution to the European Green Deal, Europe Fit for the Digital Age and EU4Health. 
The ERC has also contributed strongly to tackling the COVID-19 crisis3 in multiple fields 
such  as  diagnostics  and  treatments  (including  vaccines),  medical  devices,  digital  tools 
and the use of AI. The SDGs most frequently covered by the Horizon Europe ERC projects 
analysed were SDG 3 (Good health and well-being), SDG 9 (Industry and innovation) and 
SDG  7  (Affordable  and  clean  energy).  These  SDGs  reflect  the  growing  importance  of 
certain challenges in  recent years with respect to health (the COVID-19 pandemic, but 
also  the  importance  of  cancer research,  for example)  and  clean  energy  (related  to the 
climate  change  and  the  energy  crisis).  In  particular,  SDG7  appears  to  be  more  widely 
covered by the ERC under Horizon Europe than it was under Horizon 2020. 

The ERC has thus made important contributions to the creation of high-quality knowledge 
and to addressing societal challenges. Given the continuity in the programme part and first 
indications, this can be expected to continue under Horizon Europe. 

Coherence 

ERC grants, evaluated and selected on the basis of the sole criterion of scientific excellence, 
do not have a strong direct link with other parts of the Framework Programme. In the public 
consultation  on  the  past,  present,  and  future  of  the  European  Research  and  Innovation 
Framework Programmes 2014-2027, the majority of respondents (56%, or 500 out of 895) 
did not know whether they see any unexploited potential for complementarities between the 
six clusters and the ERC. However, in recent years, the ERC has deepened its collaboration 
with  the  EIC,  with  the  objective  of  exploiting  potential  synergies  and  complementarities 
between  the  two  organisations.  In  terms  of  external  coherence,  both  the  survey  and  the 
interviews indicate that, to a certain extent, additional national funding is applied for and used 
by PIs in complementarity with ERC grants. Furthermore, proposals under the ERC Proof of 
Concept scheme that have received a pass mark in relation to all three evaluation criteria but 
which could not be funded due to a lack of available budget under the call, are awarded the 
Seal  of  Excellence,  a  quality  label  that  recognises  the  value  of  project  proposals  and 

3 European Research Council, COVID-19 Frontier research in the spotlight, 2022. Available at: 
https://erc.europa.eu/sites/default/files/2022-08/COVID19-Frontier_research_in_the-
spotlight.pdf. 

20 

 
 
encourages other funding organisations to take advantage of the high-quality Horizon Europe 
evaluation process.  

EU added value 

According  to  interviews  with  beneficiaries  as  well  as  the  position  papers  submitted  to  the 
OPC4, the EU added value of ERC-funded research is significant. No other type of funding 
instrument at the European, national or regional level can match the conditions offered by 
ERC grants in terms of size and the freedom to pursue ground-breaking research. The survey 
results  show  that,  out  of  those  unsuccessful  beneficiaries  who  were  able  to  secure  other 
types of funding, the majority had to make significant changes to the scope of their original 
proposal, thus affecting its impact and outcomes, as well as the type of research that could 
be undertaken and the size of the research teams. To a more limited extent, the ERC has 
contributed  to  research  in  cross-cutting  areas  such  as  gender  equality,  and  ethics  and 
research  integrity,  and  has  encouraged  co-creation  (19.3%  of  the  ERC  beneficiaries  who 
responded  to  the  survey  indicated  that  they  plan  to  engage  citizens  and  citizen 
representatives or end-users in the co-creation of R&I content in their project5). 

Relevance 

The  ERC  programme  part  holds  significant  relevance  within  the  European  research 
landscape by playing an important role in reinforcing the EU science base in many scientific 
fields.  Evidence  from  several  sources  points  to  strong  appreciation  by  beneficiaries  of  its 
‘bottom-up’  approach.  For  example,  position  papers  submitted  to  the  OPC6,  as  well  as 
interviews  with  beneficiaries,  emphasised  the  importance  of  researchers  being  able  to 
formulate their own paths of investigation. In addition, they stressed that the ‘blue-sky’ nature 
of ERC calls, which provides more scope for pushing the boundaries of research disciplines, 
is a precondition for delivering excellent science and frontier research. As interviews and the 
survey of beneficiaries confirm, ERC grants are attractive to (top) researchers – and hence 
contribute to achieving the objectives of the ERC. Several interviewees also highlighted the 
fact that the ERC contributes to the retention of talent in Europe, as researchers have less 
incentive  to  look  for  other  funding  opportunities  outside  Europe.  The  ERC  also  produces 
highly relevant research outputs in the light of emerging needs – illustrated, for example, by 
its agility to respond to the COVID-19 crisis or its contribution to the European Green Deal. 

Marie Skłodowska-Curie Actions (MSCA) 

Effectiveness 

By July 2023 (cut-off date for the latest data received by the evaluation team), the MSCA had 
supported  projects  with  a  total  budget  of  approximately  EUR  1.4  billion.  Over  97%  of  the 
MSCA calls were focusing on transnational and international mobility of researchers funded 
through  activities  such  as  MSCA  Doctoral  Networks  (DN),  Postdoctoral  Fellowships  (PF), 

4 Public consultation on the past, present and future of the EU’s Horizon research and innovation 
programmes 2014-2027. Available at: https://research-and-innovation.ec.europa.eu/news/all-
research-and-innovation-news/results-public-consultation-future-eu-research-and-innovation-
programmes-are-now-public-2023-04-19_en. 
5  45 out of 233 respondents (ERC-BEN-038). 
6 Public consultation on the past, present and future of the EU’s Horizon research and innovation 
programmes 2014-2027. Available at: https://research-and-innovation.ec.europa.eu/news/all-
research-and-innovation-news/results-public-consultation-future-eu-research-and-innovation-
programmes-are-now-public-2023-04-19_en. 

21 

 
 
COFUND and Staff Exchanges (SE). A total of 4 212 unique organisations participated in or 
coordinated funded projects, with 14% of them being newcomers to the programme. Notably, 
64% of these newcomers came from the private sector. Additionally, MSCA demonstrated 
success  in  attracting  organisations  from  third  countries,  which  accounted  for  20%  of  all 
programme participations. 

The study revealed that the MSCA significantly enhances the skills and professional 
development of researchers. Postdoctoral fellows greatly value the freedom to pursue their 
specific  research  agendas  (91%)  and  the  opportunity  to  engage  in  fundamental  research 
(69%). Moreover, 95% of MSCA postdoctoral fellows affirm that the opportunities provided 
align with their needs for skills and competency improvement. 

The  MSCA  are  also  effective  in  providing  researchers  with  high-quality  supervision 
and  have  an  impact  on  researchers'  careers  and  employability.  Overall,  most  MSCA 
postdoctoral  fellows  are satisfied  with  the  supervision  and  agree  that  the  arrangements  in 
place  are  clearly  defined  (75%).  The  study  also  found  that  participation  in  the  MSCA 
programme  is  expected  to  lead  to  better  career  opportunities  and  prospects  for  individual 
researchers (90%). Researchers also agree that participation in the programme contributes 
to gaining a research position with better career prospects (87%). 

The  MSCA  programme  promotes  quality  and  transparency  in  recruitment  practices 
and  working  conditions  for  researchers.  MSCA  postdoctoral  fellows  report  that  host 
organisations  are  highly  committed  to  improving  the  stability  of  researchers’  jobs  (60%). 
Additionally,  host  organisations  are  perceived  to  protect  the  intellectual  property  rights  of 
researchers (64%). However, challenges related to taxation and salary disparities between 
countries were noted, as was the need for improved communication regarding fellowship / 
grant details before project commencement. 

As expected, due to it being the main objective of the programme, the study found that 
the  MSCA  are  highly  effective  in  providing  transnational  and  international  mobility 
opportunities.  An  overwhelming  majority  of  MSCA  postdoctoral  fellows  agree  that 
international mobility opportunities provided by their MSCA project to a (very) large extent 
respond to their needs. Countries with the most advanced and attractive research systems 
continue to host the largest shares of researchers. The programme also supports fellows in 
returning to their home countries, particularly in Greece, Italy, Cyprus, and Spain.   

The  MSCA  excel  in  establishing  and  reinforcing  collaborations  among  researchers 
and  institutions,  both  within  and  beyond  the  programme.  Successful  applicants  view 
MSCA  projects  as  a  powerful  means  to  create  or  enhance  collaborations  with  leading 
research organisations. Notably, 86% of MSCA beneficiary organisations believe that their 
participation  in  the  programme  significantly  strengthens  their  relationships  with  leading 
research organisations in Europe. For MSCA postdoctoral fellows, the programme provides 
ample opportunities for interdisciplinary cooperation (77% agreed with this statement). The 
study  also  shows  that  MSCA  beneficiaries  tend  to  engage  in  collaborative  activities  with 
projects  funded  under  other  parts  of  Horizon  Europe,  and  primarily  under  the  Excellent 
Science  Pillar.  Furthermore,  the  MSCA  have  succeeded  in  attracting  well-networked 
organisations,  with  the  top  1%  of  most  networked  entities  accounting  for  29.6%  of  total 
participations in the programme, surpassing the Horizon Europe average of 22.1%. 

The  MSCA  are  effective  in  increasing  the  research  and  innovation  capacity  of 
programme  participants.  The  study  indicated  that  a  large  majority  of  MSCA  beneficiary 

22 

 
organisations  agreed  that  MSCA  projects  respond  effectively  to  their  needs,  including  by 
strengthening  the  knowledge  base  and  enhancing  R&D  capacity  (76%)  and  improving 
innovation  capabilities  and  skills  (76%).  The  study  highlighted  a  strong  emphasis  on 
research-related  outputs,  with  a  high  percentage  of  MSCA  postdoctoral  fellows  reporting 
research publications as the most anticipated result of their fellowships (98%).   

Coherence  

A  significant  share  of  successful  MSCA  applicants  reported  that  their  projects  build  on 
research  activities  conducted  under  the  previous  Framework  Programme,  Horizon  2020, 
demonstrating continuity and progression in research efforts over time. 

The study also highlights external coherence between MSCA and regional / national 
programmes  in  EU  Member  States.  The  MSCA  complement  these  programmes  by 
emphasising international and intersectoral mobility for researchers. While Member States 
often focus on funding researchers tied to their home countries, the MSCA promote cross-
border  collaborations  and  a  globally  impactful  research  community.  The  MSCA  also 
contribute  to  the  long-term  viability  of  research  activities,  with  some  projects  continuing 
thanks  to  national/regional  funding.  Participants  also  sometimes  top  up  MSCA  funding 
through national/regional schemes.  

The MSCA also achieve synergies with regional / national programmes in EU Member 
States through  the  Seal  of  Excellence.  The  MSCA  Seal  of  Excellence  recognises high-
quality proposals that did not receive funding due to budget constraints. This scheme fosters 
synergies between EU and national funding programmes. Several Member States offer their 
own  support  schemes  for Seal  of  Excellence  recipients,  including  utilising cohesion policy 
funds. Additionally, new initiatives related to complementing MSCA support, funded by the 
Recovery  and  Resilience  Facility  have  emerged,  further  integrating  Horizon  Europe  with 
national  recovery  and  resilience  plans.  Finally,  some  higher  education  institutions  and 
research  organisations  in  Europe  provide  their  own  funding  to  support  Seal  of  Excellence 
recipients.  The  European  Commission  has  issued  guidance  to  Member  States  on 
establishing their own schemes to fund Seal of Excellence recipients, outlining opportunities 
for synergies with existing programmes. 

EU added value 

Under Horizon Europe, the MSCA continue to provide significant EU-added value in 
terms  of  research  excellence  and  international  mobility  opportunities  that  are  not 
available  at  national  /  regional  level.  MSCA  beneficiary  organisations  overwhelmingly 
agree  that  the  MSCA  provide  more  international  mobility  opportunities  (93%)  and  offer 
greater partner diversity, including intersectoral and interdisciplinary consortia, with a broader 
geographical  distribution  (89%).  Beneficiary  organisations  also  perceive  a  higher  level  of 
competition for research funding within Horizon Europe, indicating MSCA’s reputation as a 
competitive and prestigious programme. In terms of financial support, 70% of beneficiaries, 
who responded to the survey, acknowledge that Horizon Europe provides a higher amount 
of funding compared to national or regional schemes, reflecting its pivotal role in financially 
supporting research endeavours. Similarly, for MSCA postdoctoral fellows, the programme 
excels in providing international mobility opportunities (93%) and a higher level of competition 
for research funding (91%). Additionally, 84% believe Horizon Europe encourages them to 
pursue their careers within Europe, demonstrating its influence in retaining research talent 
within the European scientific community. Lastly, 79% agree that Horizon Europe, through 

23 

 
MSCA,  contributes  to  better  pooling  of  skills  and  expertise,  highlighting  the  programme's 
collaborative nature that fosters innovative and interdisciplinary research. 

An overwhelming majority (76%) of unsuccessful MSCA applicant organisations (DN, 
SE,  COFUND)  and  more  than  half  of  unsuccessful  MSCA  PF  individual  applicants 
(60%) were not able to implement their projects without the MSCA support. The projects 
that received support from alternative sources, after being rejected for MSCA support, were 
implemented with some or significant adaptations. The adapted projects were usually smaller 
in  scope,  less  complex  in  terms  of  research  methods,  smaller  in  the  number  of  research 
outputs and innovations to be produced, and less favourable in terms of research conditions. 
For unsuccessful applicant organisations, the adapted projects were also shorter in duration 
as well as less diverse in terms of project partners involved. The projects also had a smaller 
number  of  consortium  partners  involved,  compared  to  the  planned  MSCA  projects.  The 
MSCA also contributed to the promotion of citizen science and societal engagement – 
an integral part of the EU’s open science policy. The MSCA fully support and encourage 
participating  organisations  and  researchers  to  follow  the  principles  of  open  science. 
Approximately 38% of successful MSCA applicant organisations and 39% of selected MSCA 
postdoctoral fellows plan to engage representatives of citizens / end users in the co-creation 
of R&I content for their MSCA projects. Among those who plan to do so, a significant portion 
(55% for both organisations and applicants) intend to actively involve citizens or end users in 
co-design activities. 

The MSCA have a pronounced structuring impact on participating organisations and 
bring  added  value  by  promoting  Responsible  Research  and  Innovation  (RRI) 
principles, and gender equality. The MSCA have structuring impacts that go beyond MSCA 
projects and are spread / institutionalised in beneficiary organisations and national / regional 
systems.  This  is  particularly  evident  in  terms  of  gender  equality  policies  that  are  being 
institutionalised because of the participation in MSCA. Additionally, the majority of successful 
MSCA beneficiary organisations (52%) believe that their participation in MSCA aided their 
efforts  to  reinforce  Responsible  Research  and  Innovation  (RRI)  principles  within  their 
institutions. 

Relevance  

The MSCA is highly relevant in nurturing excellence through mobility of researchers 
across borders, sectors, and disciplines. The programme's setup and guiding principles 
actively  support 
through 
Postdoctoral Fellowships, Doctoral Networks, Staff Exchanges, and COFUND. The MSCA 
encourage researchers from any nationality, within or outside Europe, to pursue outstanding 
research and advance their skills and careers. 

interdisciplinary  mobility 

intersectoral,  and 

international, 

The MSCA play a significant role in nurturing and enhancing the skills of researchers. 
This is achieved by providing robust training opportunities, either through supporting existing 
training  programmes  or  establishing  new  ones.  MSCA  are  designed  to  equip  researchers 
with  a  wide  range  of  skills  that  are  pertinent  to  current  and  future  global  challenges. 
Additionally,  the  programme  places  a  strong  emphasis  on  transferable  skills  and 
competences,  ultimately  leading  to  improved  employability  and  career  prospects  for 
researchers.  An  overwhelming  majority  of  MSCA  postdoctoral  fellows  believe  that  the 
opportunities to enhance skills and competences in their MSCA projects meet their needs to 
a (very) large extent (95%). 

24 

 
Furthermore,  the  MSCA  play  a  crucial  role  in  addressing  the  issue  of  precarious 
employment among researchers by fostering a supportive work environment. In this 
context, the MSCA adhere to the principles outlined in the European Charter for Researchers 
and  the  Code  of  Conduct  for  the  Recruitment  of  Researchers  and  places  particular 
importance  on  effective supervision.  The study  found  that  the  opportunities  offered by  the 
MSCA projects have effectively responded to the need to gain a research position with better 
career  prospects  for  87%  of  MSCA  postdoctoral  fellows  to  a  (very)  large  or  large  extent. 
Additionally,  77%  of  MSCA  postdoctoral  fellows  found  the  projects  to  be  relevant  in 
addressing their need for attractive contractual conditions and a better salary to a (very) large 
extent.  

The  MSCA  stimulate  regional,  national,  and  international  programmes  to  foster 
excellence in research through COFUND action and the Seal of Excellence initiative. 
COFUND  provides  complimentary  funding  for  both  new  and  established  doctoral  and 
postdoctoral programmes managed by entities established in EU Member States or Horizon 
Europe  associated  countries.  The  action  also  encourages  synergies  with  Cohesion  policy 
funds  as  well  as  the  Recovery  and  Resilience  Facility. The  MSCA  also  continue  to  award 
Seal of Excellence certificates to project proposals ranked above a predefined threshold but 
that did not receive funding due to budget limitations. By awarding Seal of Excellence labels, 
the MSCA successfully promote synergies between the research and innovation framework 
programmes and the Cohesion Policy Funds, as well as the Recovery and Resilience Facility. 

The MSCA programme also considered the feedback from various stakeholders and 
aligned its focus to better reflect the European Commission’s key political priorities 
(2019-2024). Under Horizon Europe, all MSCA fellows now benefit from a newly introduced 
long-term  leave  allowance,  specific  needs  allowances  for  individuals  with  disabilities,  and 
accessible family allowances for all recipients. The programme also introduced the MSCA 
Green  Charter,  which  promotes  the sustainable implementation  of  research  activities;  and 
Guidelines  for  the  Inclusion  of  Researchers  at  Risk,  to  address  the  need  to  support 
researchers who face precarious situations or are at risk due to political, social, or human 
rights-related reasons in their home countries. 

The MSCA also showed a high degree of responsiveness and flexibility in addressing 
newly  emerging  needs  resulting  from  challenges  such  as  the  Russian  invasion  of 
Ukraine  and  the  COVID-19  crisis.  In  response  to  the  Russian  invasion  of  Ukraine,  the 
MSCA have taken proactive steps to support affected doctoral candidates and post-doctoral 
researchers from Ukraine by establishing the EUR 25 million MSCA4Ukraine scheme. The 
proposed flexibility measures taken to respond to the COVID-19 crisis were appropriate and 
effective, allowing participants to adapt to restrictions and constraints. In light of the global 
pandemic,  there  are  already  more  than  100  MSCA  projects  directly  related  to  COVID-19 
research issues alone and MSCA can support many more projects in the years to come. 

Under Horizon Europe, the MSCA continue to operate in a fully bottom-up manner. The 
stakeholders highlight the bottom-up approach to funding as being essential for the MSCA 
programme’s success in fostering excellent science. The MSCA are highly appreciated by 
researchers  for  allowing  them  to  propose  research  topics  and  ideas  based  on  their  own 
interest and without a predefined thematic focus. The survey results reflect these findings, as 
an overwhelming majority of successful MSCA PF applicants (91%) indicate that freedom to 
pursue  their  specific  research  agenda  responds  to  their  needs  to  a  very  large  and  large 
extent.  Furthermore,  although  MSCA  operate  in  a  fully  bottom-up  manner,  the  projects 
funded respond to the development of future emerging technologies making MSCA highly 
responsive to the newly emerging needs. 

25 

 
Research Infrastructures (INFRA) 

The main objective of the Research Infrastructures programme part (INFRA) within Horizon 
Europe is ‘empower Europe through world-class and accessible research infrastructures, as 
part  of  an  integrated  European  research  and  technology  infrastructures  landscape’.  The 
Horizon  Europe  Regulation  defines  ‘research  infrastructures’  (RIs)  as  facilities  that  offer 
resources and services for research communities to conduct research and innovation. INFRA 
addresses various needs in the field of RIs, such as open access to RIs, the development of 
new RIs, fostering the innovation potential of RIs, and reinforcing international cooperation. 
INFRA has an indicative budget of EUR 2.186 billion for the period 2021-2027, which makes 
it  the  smallest  programme  part under  Horizon  Europe  Pillar 1,  ‘Excellent  Science’.  INFRA 
also aims to contribute to the ERA, for which ‘strengthening Ris’ is included as a priority action 
in the new ERA policy agenda. INFRA’s ultimate goal is to enhance Europe’s research and 
innovation landscape, societal impact and global competitiveness. 

Effectiveness  

The effectiveness of the INFRA programme part within Horizon Europe is assessed across 
several dimensions. These include the consolidation and development of the RI landscape; 
the opening up and integration of RIs; the contribution made to the European Open Science 
Cloud (EOSC), as well as to international cooperation and industry collaboration, and to the 
training of researchers and the enhancement of research careers. 

•  Consolidating  and  developing  the  RI  Landscape:  INFRA  plays  a  crucial  role  in 
supporting  the  roadmap  of  the  European  Strategy  Forum  for  Research  Infrastructures 
(ESFRI),  which  outlines  the  essential  RIs  needed  in  Europe  for  the  next  10-20  years. 
INFRA  funds  various  phases  of  ESFRI  projects,  including  Design  Studies,  Preparatory 
Phases,  and  Implementation  Phases.  However,  many  interviewees  and  survey 
respondents consider the existing funding for RI development to be insufficient. 

•  Opening,  integrating  and  interconnecting  RIs:  the  programme  part  aims  to  foster 
collaboration  among  RIs  and  to  ensure  accessibility  for  researchers  across  Europe. 
Changes  made  under  Horizon  Europe  –  notably  the  discontinuation  of  Integrating 
Activities  (INFRAIA)  grants  and  the  introduction  of  the  more  challenge-oriented 
INFRASERV  grants  for  the  provision  of  access  –  have  raised  concerns  regarding  the 
complexity of project coordination, the fragmentation of funding and the sustainability of 
the services developed beyond the lifetimes of individual projects’. Interviewees indicated 
the  need  for  a  longer-term  vision  for  the  integration  of  RIs.  Further  observation  will, 
however, be needed to assess the full impact of these changes. 

•  Contribution  to  the  EOSC:  INFRA  plays  an  important  role  in  the  development  of  the 
European Open Science Cloud (EOSC). While important advances are being achieved in 
terms of FAIR data and the creation of new services, concerns are expressed about the 
lack of synchronisation between projects, potentially leading to a duplication of efforts and 
unexploited synergies. To this end, the INFRA work programme includes two topics (in 
2021  and  2024)  that  include  dedicated  activities  to  support  coordination  among 
INFRAEOSC projects. Moreover, since 2022, the European Commission has organised 
annual  ‘concertation’  meetings  of  INFRAEOSC  projects  in  order  to  further  support 
coordination  and  the  development  of  synergies.  Moreover,  interviewees  regret  the 
discontinuation of dedicated funding for the science clusters, which were considered an 
important platform for collaboration between RIs within the same scientific discipline, with 
a view to the integration of RI services and data into the EOSC. INFRA work programme 

26 

 
2023  includes  a  topic  that  aims  to  build  on  and  extend  the  level  of  cross-domain 
collaboration and EOSC alignment originally initiated with the science cluster projects. 

• 

• 

International  cooperation:  international  cooperation  between  RIs  is  considered 
essential  to  advancing  scientific  knowledge.  INFRA  facilitates  collaboration  with 
associated and third countries; however, some beneficiaries point to the need for more 
funding  for  international  collaboration  and  pre-normative  research  on  international 
standards.  

Industry  collaboration:  the  connection  between  RIs  and  the  private  sector/industry 
could be further strengthened in order to increase the impact and financial sustainability 
of RIs. Interviewees emphasise the importance of collaboration with industry , which could 
be fostered through a better linkage between research and technology infrastructures, as 
well as the increased participation of RIs in other Pillars of Horizon Europe. 

•  Training  of  researchers  and  research  careers:  INFRA  contributes  to  the  training  of 
those  researchers  using  RIs,  particularly  those  who  receive  access  through  TNA/VA. 
Training  programmes  and  career  improvement  opportunities  are  seen  as  positive 
outcomes.  

Overall,  while  INFRA  has  made  significant  contributions,  there  are  areas  in  which 
improvements and additional funding are suggested, including RI development, international 
cooperation and industry engagement. Longer-term vision and strategies are also essential 
to maximise the programme part's impact.  

Coherence 

The coherence of INFRA with other programme parts within Horizon Europe could be further 
improved. In interviews, INFRA beneficiaries indicated that the large number and diversity of 
calls make it challenging for them to identify relevant topics in programme parts with which 
they are not familiar. More guidance in navigating these calls, as well as explicit mentions of 
relevant RIs in the call texts, could increase the visibility and use of RIs in other programme 
parts. According to the network analysis performed in context of this study, INFRA is most 
connected with Cluster 4 ‘Digital, Industry and Space, Cluster 6 ‘Food, Bioeconomy, Natural 
Resources,  Agriculture  &  Environment’,  Cluster  1  ‘Health’,  and  WIDERA.  Given  the 
coherence between successive Framework Programmes, INFRA shows a very high degree 
of continuity with Horizon 2020 as well as with FP7.  

Moreover, INFRA is highly complementary to other R&I funding programmes at European, 
national  and  regional  level,  by  offering  unique  support  for  activities  that  are  typically  not 
funded  elsewhere.  At  the  same  time,  only  a  minority  of  INFRA  projects  appear  to  be 
connected to national/regional or other European research projects.  

EU added value 

The EU added value of INFRA is significant, as it enables the development of world-class 
pan-European  RIs,  reduces  fragmentation  and  duplication  in  the  RI  landscape,  fosters 
international  cooperation,  and  provides  access  to  RIs  for  researchers  within  and  beyond 
Europe.  Compared  with  national/regional  funding,  INFRA  offers  greater  access  to  RIs, 
facilitates diverse partner collaborations, and offers more international mobility opportunities 
for  researchers.  Most  INFRA  beneficiaries  view  Horizon  Europe  as  providing  substantial 
added benefits compared with national/regional support. Furthermore, among those INFRA 
applicants  who  were  unsuccessful,  the  majority  did  not  seek  alternative  funding  sources, 

27 

 
mainly  due  to  the  need  for  international  partners,  a  lack  of  alternative  funding  for  their 
research  area,  or  their  focus  on  EU-level  or  global  challenges.  Moreover,  most  of  those 
unsuccessful  applicants  that  did  seek  alternative  funding  had  to  significantly  change  the 
scope of their project activities.  

Relevance 

The  INFRA  programme  part  holds  significant  relevance  within  the  European  research 
landscape  by  playing  an  important  role  in  strengthening  RIs,  which  is  one  of  the  priority 
actions  in  the  new  ERA  Policy  Agenda.  The  ERA  underlines  the  importance  of  RIs  for 
scientific  excellence,  as  well  as  for  Europe’s  competitiveness  and  its  capacity  to  address 
societal  challenges.  INFRA  strengthens  the  RI  landscape  by  funding  the  development  of 
international 
world-class  pan-European  RIs  while  reducing 
collaboration among RIs, and facilitating access to RIs for researchers across Europe and 
beyond.  Moreover,  INFRA  has  demonstrated  its  agility  through  its  rapid  response  to  the 
COVID-19  crisis.  Its  new  challenge-oriented  approach  under  Horizon  Europe  has  further 
increased  the  relevance  of  the  programme  part  by  increasing  the  contribution  of  RIs  to 
tackling societal challenges. However, concerns are raised regarding the impact of the more 
top-down approach on the continuity of projects and the sustainability of project results in the 
longer term. Interviewees emphasise the need for longer-term planning and stability to ensure 
the longer-term impacts of projects. 

fragmentation, 

fostering 

Widening participation and strengthening the ERA under Horizon Europe 

1.1.1.  Strengthening the European Research Area  

Effectiveness 

Since  2021,  56  Horizon  Europe  grants  have  been  funded  under  the  ‘Strengthening  the 
European  Research  Area  (ERA)’  thematic  priority  of  the  ‘Widening  participation  and 
strengthening the ERA’ (WIDERA) programme part, amounting to a total net EU contribution 
of EUR 128.1 million,  or EUR 2.29 million per project. At the time of this evaluation, out of 
the  56  grants  signed,  54  were  ongoing  and  two  had  been  closed.  Out  of  total  705 
participations,  241  (34%  of  participations)  were  by  higher  or  secondary  education 
establishments, followed by 151 by research organisations (21%), 98 by public bodies (14%) 
and  84  by  private  for-profit  entities  (12%).  The  ERA  thematic  priority  has  succeeded  in 
supporting  a  wide  range  of  projects  in  the  fields  of  citizen  science,  gender  equality, 
entrepreneurship,  open  access  policies  and  practices,  RRI/research  ethics,  as  well  as 
projects to support university transformation. The progress made with regard to support for 
programme/policy-level  collaboration  between  policy-makers  has  been  more  limited,  with 
three projects in this area supported so far (in contrast, there have been 15 grants supported 
since 2021 in the field of citizen science/science education/science and society). 

Study evidence indicates the following key areas of expected impacts and outcomes resulting 
from WIDERA projects under the ‘Strengthening the ERA’ thematic priority: 

•  The immediate result of projects (particularly those focusing on acceleration services) is 
systematic evidence gathering, as well as mapping of the state-of-play and of the 
reform needs of institutions, in order to pave the way for further transformation and 
changes  in institutions  across  the  ERA.  Case  study  evidence  shows  that,  to  this  end, 
project  consortia  have  conducted  various  consultation  activities  (surveys,  workshops, 
etc.),  which  have  resulted  in  the  systematic  gathering  of  evidence  about  the  needs, 
priorities and current situations of beneficiary institutions. 

28 

 
•  The development of collaborative ties and networks of expertise across borders has 
also  been  identified  as  one  of  the  key  outcomes.  In  some  cases,  the  development  of 
clusters and groups of experts discussing possibilities and pathways for transformation 
have already resulted during the project’s inception phase. 

•  Horizon Europe support is likely to result in policy standard-setting measures (e.g. 
recommendations,  methods,  tools,  roadmaps,  educational  materials,  training 
programmes etc.) that will empower beneficiary institutions and institutions outside the 
the 
transformation  across  various 
projects 
professionalisation of research management/career development of researchers, gender 
equality, Open Science, etc.) 

to  undertake 

(e.g.,  RRI, 

fields 

•  From a longer-term perspective, institutional-level changes/improvements in standards 
and  practices  will  contribute  to  better  working  conditions,  and  to  the  career  and 
professional  development  of  staff  in  the  beneficiary  institutions  and  across  the 
ERA.  

•  Another area of impacts includes the development of the skills and competences of 

the administrative, managerial and academic staff of universities. 

•  Available evidence shows that ERA projects in the field of science communication have 
contributed to increasing the visibility and attractiveness of research careers and 
to motivating young talented people to choose a research career path. 

The  successful implementation  of  projects  and  the  realisation  of  the  above  outcomes  and 
impacts at institutional level and across the ERA will depend on a number of factors, such as 
the  ability  of  project  partners  to  ensure the  active  dissemination  of  information,  as well  as 
awareness  of  the  project’s  activities  among  the  local  research  community/wider  public, 
effective communication and coordination between project partners and their varying needs, 
and  the  commitment  of  policy  stakeholders  and  top-level  management  to  undertaking 
transformational changes. In particular, the success of ERA projects focusing on university 
transformation  depends  on  the  ability  of  projects  to  mobilise  wider groups  of  stakeholders 
whose  active  engagement  is  crucial  for  data  gathering  and  stakeholder  consultation 
processes  that  will  feed  into  development  of  policy  standards,  recommendations  and 
roadmaps.  

Coherence 

Study evidence indicates that overall, WIDERA support under the ‘Strengthening the ERA’ 
thematic priority has provided a mix of actions that have complemented and were coherent 
with other programme parts of Horizon Europe in terms of the same higher-level objectives 
being sought though different means. In particular, support provided through ERA projects 
have  complemented  the  activities  of  the  MSCA  (primarily  by  contributing  to  the  same 
overarching objective of strengthening Europe’s human capital base); INFRA (by supporting 
policy support and transformational changes in the field of open access and the management 
of research infrastructures); and the European Institute of Innovation and Technology (EIT) 
(particularly, its initiative on Innovation Capacity Building for Higher Education). Significant 
coherence between WIDERA and other Horizon Europe programme parts is also evidenced 
by  the  collaborative  efforts  made  during  project  implementation,  with  45%  of  ERA  WP 
beneficiary  organisations  indicating  that  at  least  one  planned  activity  in  their  project  is 
implemented in collaboration with projects funded under other Horizon Europe programme 
parts. 

29 

 
 
WIDERA support for programme-level collaboration between national R&I policy-makers is 
directly aimed at fostering complementarities and synergies between national and regional 
R&I programmes, while also ensuring complementarity between the activities and policies of 
projects,  and  those  of  the  Framework  Programme.  ERA  projects  focusing  on  programme-
level  collaboration  also  complement  the  European  Partnerships  identified  in  the  Strategic 
Plan for Horizon Europe by providing the possibility for Member States, associated countries 
and civil society organisations to maintain existing collaborations and establish new ones on 
priorities  of  their  choice.  Moreover,  the  available  evidence  also  indicates  that  there  are 
significant potential synergies between Horizon Europe ERA projects in the field of university 
transformation  and  the  Erasmus+  programme  –  in  particular,  the  European  University 
Alliances initiative. 

EU added value 

One  of  the  most  significant areas  of  added  value  provided  by  Horizon  Europe  support for 
ERA  actions  has  been  the  opportunity  provided  to  beneficiary  organisations  to  form 
diverse, cross-border networks of expertise that are crucial to achieving the projects’ 
results, which could not be achieved without such EU support. The EU-level scope of 
the supported projects has enabled the involvement of a wide range of stakeholders from 
different  Member  States,  which  is  the  key  source  of  knowledge  for  building  tools  and 
roadmaps that respond to the needs of (and can be adapted by) institutions across the EU. 
A number of other key sources of EU added value for the beneficiaries and for the EU as a 
whole can be seen to stem from Horizon Europe support for ERA actions (compared with 
what  would  have  been  available  at  national  level).  These  include  its  contribution  to 
excellence stemming from higher level of competition at EU level (compared to regional 
or national level); providing beneficiary organisations with funding for research topics or 
fields not covered by national or regional R&I funding programmes; pooling the skills 
and  expertise  necessary  for  research;  addressing  grand  societal  challenges;  and 
providing  a 
is  available  at 
national/regional level. 

larger  amount  of  funding  compared  with  what 

Significant additionality of Horizon Europe support is also evidenced by the large majority 
of unsuccessful applicants (81%) who discontinued their projects after being rejected 
for  Horizon  Europe  support  (i.e.  they  did  not  apply  for  additional  funding,  or  applied  for 
alternative funding but were unsuccessful). Indeed, fewer than 10% unsuccessful applicants 
received alternative funding for the activities planned in their ERA project.  

Relevance 

Study  evidence  shows  that  Horizon  Europe  projects  under  the  ‘Strengthening  the  ERA’ 
thematic  priority  have  been  highly  relevant  in  addressing  the  key  policy  priorities  and 
scientific/socioeconomic challenges faced by research institutions and by society at large. In 
particular, Horizon Europe support under this thematic priority has been closely aligned with 
the  communication  on  a  new  ERA  for  Research  and  Innovation  (2020)  and  the  Council 
Recommendation on a Pact for Research and Innovation in Europe (‘Pact for R&I’, 2021).It 
has also responded to a number of policy actions on the ERA Policy Agenda (2022-2024), 
including  promoting  gender  equality  and  fostering  inclusiveness;  enabling  Open 
Science  and  the  re-use  of  research  outputs;  the  prioritisation,  coordination  and 
direction of R&I investments and reforms; enhancing closer and deeper cross-border 
cooperation  between  HEIs;  deepening  a  truly  functioning  internal  market  for 
knowledge;  and  bringing  science  closer  to  society.  The  support  provided  by  the 
programme in the field of citizen science was also particularly relevant in the context of the 

30 

 
COVID-19  pandemic,  and  also  responded  to  the  need  to  strengthen  society’s  trust  in 
science. 

In parallel, the present study has found that ERA projects supported under Horizon Europe 
responded to some of the most pressing needs faced by beneficiary institutions, such as the 
need to  develop  their  capacities  and  the  skills  of  staff  in order  to  better  respond  to 
current  socioeconomic  needs  (e.g.  entrepreneurship  skills  and  innovativeness),  and  to 
increase competitiveness and enhance the overall knowledge base. WIDERA support 
was also highly relevant and responded to institutions’ need to increase their international 
visibility/strengthen  collaborative  ties, 
improve  the  working  conditions  of 
to 
researchers  (primarily  by  streamlining  processes  and  internal  practices)  and  to  attract 
young talents. 

1.1.2.  Widening participation and spreading excellence 

The premise underlying ‘Widening participation and spreading excellence’ is that R&I policy 
can only achieve more excellence if everyone in the EU progresses. Despite much progress 
having  been  made  in  developing  the  ERA,  disparities  remain  between  countries  that  are 
leading in terms of their R&I performance and participation in the Framework Programme, 
and those that are lagging7. The Widening actions therefore aim to strengthen collaborative 
links across Europe, and to contribute to improving research management capacities in the 
widening  countries  and spurring  institutional  and  national  reforms.  Under Horizon  Europe, 
the budget for the Widening actions has increased to EUR 2.8 billion.  

Under Horizon Europe, the Member States eligible as coordinators of Widening actions are 
Bulgaria,  Croatia,  Cyprus,  Czechia,  Estonia,  Greece,  Hungary,  Latvia,  Lithuania,  Malta, 
Poland,  Portugal,  Romania,  Slovakia  and  Slovenia.  The  eligible  associated  countries  are 
Albania,  Bosnia  &  Herzegovina,  Kosovo,  Montenegro,  North  Macedonia,  Serbia,  Turkey, 
Armenia, Georgia, Moldova, Morocco, Tunisia, Ukraine and Faroe the Islands; as well as the 
outermost  regions8.  Several  pre-existing  Widening  actions  have  been  continued  under 
Horizon Europe (e.g. Twinning, Teaming for Excellence and ERA Chairs), while several new 
actions have been introduced (e.g. Pathways to Synergies, the Hop On Facility, Excellence 
Hubs and ERA Talents).  

Effectiveness 

While  historically,  the  Widening  actions  have  been  proven  to  contribute  to  the  research 
excellence of the Union, and have appeared key to the production of excellence science 
in widening countries, to date, there is insufficient bibliometric information available to draw 
conclusions  regarding  the  effectiveness  of  Widening  actions  under  Horizon  Europe. 
Nevertheless,  almost  90%  of  the  respondents9  to  the  Horizon  Europe  Widening  survey 
indicated that their project will (probably) lead to research publications and over half indicate 
that  their  project  will  or  is  likely  to  push  the  frontiers  of  knowledge.  In  addition,  the  novel 
Widening actions introduced are intended to enhance beneficiaries’ scientific capacities and 
research excellence. These include the Hop On Facility and the Excellence Hubs, which aim 
to improve access to excellence for R&I actions in widening countries. These findings indicate 

7 European Commission Decision C(2022)2975. Horizon Europe Work Programme 2021-2022. 
11. Widening participation and strengthening the European Research Area. 
8 Article 349 of the Treaty on the Functioning of the EU (TFEU). 
9 Survey of Horizon Europe WIDERA (widening) beneficiaries, conducted May-July 2023;158 out 
of 177 respondents; WIDERA-WIDE-BEN-027. 

31 

 
 
that the Widening actions under Horizon Europe are likely to have a significant impact on the 
production of excellent science in widening countries.  

Other core expected outcomes of the Widening actions include strengthening the skills and 
capacities  of  researchers  and  institutions in  widening  countries.  So  far, Horizon Europe  is 
supporting more than 4,000 researchers via Widening actions, of which more than 40% are 
involved  in  upskilling  activities  (training,  mentoring/coaching,  mobility  and  access  to  R&I 
infrastructures). Survey evidence points to the high impact of projects under Horizon 
Europe on the improvement of skills, knowledge, and competences of researchers. In 
particular,  Twinning  was  highlighted  by  many  interviewees  and  stakeholders  representing 
academia  as  a  proven  instrument  to  bolster  capacity  building  and  improve  the  skills  of 
researchers. As a result, many voices would like to see the Twinning budget  increased. In 
addition,  under  Horizon  Europe  the  Widening  actions  have  increased  their  focus  on 
fostering  brain  circulation  across  the  ERA  through,  among  other  measures,  the  ERA 
Fellowships (piloted under Horizon 2020) for excellent researchers undertaking cross-border 
mobility  to  a  widening  country,  and  ERA  Talents,  which  aims  to  support  the  training  and 
mobility of researchers, including intersectoral mobility.  

Under  Horizon  Europe,  some  stakeholders  (e.g.  widening  beneficiaries)  still  indicate  that 
attracting international talent to widening institutions remains a challenge, especially if 
the  host  institution  is  located  in  a  rural  area  or  offers  less  attractive  renumeration. 
Nevertheless,  beneficiaries  do  note  that  the  Widening  actions  contribute  to  enhancing 
the  visibility  of  institutions  and  networks.  In  addition,  some  novelties  have  been 
introduced into ERA Chairs under Horizon Europe to facilitate the process of finding an ERA 
chairholder (e.g. the identification of the chairholder in the proposal).  

One of the key expected outcomes of the Widening actions is to induce structural changes 
into the regional or national R&I landscape or within the institutions involved, with the aim of 
fostering  R&I  performance.  Horizon  2020  survey  evidence10  shows  that  the  structural 
changes occurring at institutional level appear to be widespread, while changes are 
less  visible  at  regional  or  national  levels.  This  can  partly  be  attributed  to  the  fact  that 
changes  at  institutional  level  can  be  more  narrowly  defined  (and  are  hence  more  easily 
identified  by  multiple  stakeholders)  and  are  potentially  applicable  to  all  projects,  while 
relatively fewer projects are either expected or able to induce changes at regional or national 
level. In addition, the mechanisms by which changes at regional or national level occur vary 
significantly depending on the projects concerned. In line with prior recommendations, the 
present study underlines the need to develop a definition of ‘structural changes’ in the context 
of  the  Widening  actions,  and  to  invoke  a  consolidated  and  robust  monitoring  approach  to 
track the progress made by actions in this regard. In setting up a measuring and monitoring 
framework, it would be desirable to keep as much as possible of the existing instruments in 
order to cause as little additional burden and inefficiencies as possible.  

Based on desk research and interviews, this study proposes a definition: a structural change 
is a sustainable and/or permanent shift or change introduced as a result of or triggered by a 
Widening action under SEWP (Horizon 2020) or WIDERA (Horizon Europe) that manifests 
at  institutional,  regional,  or  national  level,  and  is  aimed  at  fostering  R&I  performance.  In 

10 Survey of Horizon 2020 beneficiaries, conducted August-September 2022 

32 

 
 
addition, structural changes are expected to impact R&I performance, and can be classified 
according to their expected impacts:  

1.  Strengthening the R&I system and its governance; 

2. 

Improving working conditions, career progression and skills; 

3.  Fostering research excellence and collaboration; and 

4.  Strengthening the research environment and infrastructure. 

The  goals  of  the  Widening  actions  under  Horizon  Europe  also  include  supporting  less 
advanced  R&I  countries  by  increasing  their  participation  in  international  research 
networks,  strengthening  academia/non-academia  cooperation,  and  reinforcing 
(regional)  ecosystems.  Several  new  Widening  actions  have  been  introduced  to  achieve 
these  goals,  such  as  the  Hop  On  Facility  and  Excellence  Hubs.  Almost  90%  of  WIDERA 
widening  beneficiaries  who  responded  to  the  survey  reported  that  their  Widening  project 
creates or strengthens collaborations with leading research organisations to a ‘large’ or ‘very 
large’ extent11.  

The  Horizon  Europe  Widening  actions  place  an  increased  focus  on  private  sector 
linkages. This is reflected in the increased share of private for-profit entities participating in 
the  Widening  actions  under  Horizon  Europe  (12.7%,  compared  with  4.7%  under  Horizon 
2020)12.  New  Widening  actions  have  been  developed  to  stimulate  industry-academia 
linkages,  as  well  as  to  scale  up  research  results  and  improve  the  sustainability  of  actions 
(e.g.  the  Dissemination  and  Exploitation  Support  Facility).  The  survey  carried  out  for  the 
present  study  indicates  that  one-quarter  of  the  WIDERA  widening  beneficiaries  who 
responded  believe  their  Horizon  Europe  project  will  lead  to  new  or  improved  products, 
services or processes13.  

Coherence  

Some interviewees and stakeholders representing academia made some remarks on 
the design and implementation of the WIDERA programme part. Some indicated that the 
Widening component and the ‘Strengthening the ERA’ component are quite distinct, and yet 
are addressed together within a single work programme. In addition, several stakeholders 
wondered  whether  some  actions  that  are  presently  categorised  under  Widening  (e.g. 
Excellence Hubs, the European Excellence Initiative, COST), would not be better positioned 
under  the  ‘Strengthening  the  ERA’  component  of  the  work  programme,  or  under  another 
programme part. In general, the Widening actions in themselves appear to form a set of 
distinct actions to support widening institutions and countries aiming to improve their 
R&I systems.  

In terms of internal coherence with Horizon Europe, the Widening actions focus mainly on 
fostering  the  participation  in  the  Framework  Programme  of  countries  lagging  in  R&I,  and 
assisting  these  countries  in  strengthening  their  R&I  systems.  Widening  actions  consist  of 
collaborative  approaches  and  Coordination  and  Support  Actions  (CSAs).  They  differ  from 
actions under Pillar 1, which focus on mono-beneficiary grants and only to a limited extent 
on  CSAs.  In  addition,  many  stakeholders  representing  academia,  as  well  as  interviewees 

11 158 out of 180 respondents; WIDERA-WIDE-BEN-025.  
12 Accessed 1 September 2023. 
13 45 out of 177 respondents; WIDERA-WIDE-BEN-027.  

33 

 
 
(e.g. NCP WIDERA representatives) raised the idea of integrating Widening elements across 
the Framework Programme as a cross-cutting issue. 

In  terms  of  external  coherence,  a  few  stakeholders  representing  academia,  as  well  as 
interviewees (e.g., NCP WIDERA representatives), posed the open question as to whether 
there  are  sufficient  differences  between  those  instruments  that  support  ecosystems, 
networks and alliances, and whether all of these offer distinct added value (e.g. Excellence 
Hubs,  the  currently  piloted  concept  of  ERA  Hubs14,  Digital  Innovation  Hubs,  Regional 
Innovation Valleys). Under WIDERA, synergies among funding sources are envisioned in the 
Teaming and Excellence Hubs to ensure the sustainability of results. A general overview of 
where  Horizon  Europe  and  other  EU  funding  sources  complement  each  other  shows  that 
there  are  15  cases  in  which  WIDERA  and  other  EU  funding  schemes  have  supported 
thematically related projects. The amount of EU funds received as complementary funding 
amounts to EUR 20.1 million15. 

EU added value 

In  general,  interviewees  and  stakeholders  acknowledge  and  appreciate  the  added 
value brought by the Widening actions. They underlined its significance in terms of access 
to research funding, building capacity, fostering (research) networks and enhancing the R&I 
critical mass and systems of host institutions and, to some extent, of regions and countries. 
Nevertheless, it is consistently emphasised that it is vital for widening countries to commit 
adequate resources to their own national R&I systems. In the survey16, only 14% of WIDERA 
widening  beneficiaries  indicated  that  the  Widening  actions  offer  no  additional  benefits 
compared  with  national  or  regional  support.  Of  the  unsuccessful  WIDERA  widening 
applicants that responded to the survey, almost 80% did not apply for alternative funding.  

In  addition,  Widening  projects  address  cross-cutting  or  transversal  issues17.  Some 
illustrations of this are provided below:  

The creation or revision of ethics guidelines in organisations or countries is regarded as a 
structural change sometimes triggered by Widening actions with the aim of fostering research 
excellence  and  collaborations.  Case  Study  7  indicates  that  slightly  fewer  than  half  of  the 
SEWP survey respondents in the Evaluation study on Excellent Science under Horizon 2020 
acknowledged that their Widening project had influenced the development or improvement 
of standards or appraisal processes in research ethics and integrity in the host institution. 

Around half of WIDERA widening beneficiaries18 indicated that they plan to engage citizens 
and citizen representatives or end-users in the co-creation of R&I content in their project. In 
addition,  novel  actions  introduced  under  Horizon  Europe  (in  particular,  Excellence  Hubs) 
engage Quadruple Helix partners, and hence, society. Such large-scale projects often cannot 
be funded at national level – hence, the EU provides added value here. 

14 Horizon Europe Programme. Information document (2022). Testing of the ERA Hub concept – 
pilot phase. Accessed on 20 October 2023 at: https://ec.europa.eu/info/funding-
tenders/opportunities/docs/2021-2027/horizon/wp-call/2022/information_horizon-widera-2022-
era-01-30_en.pdf 
15 Based on Corda data. For more information, see Main Report, Table on ‘The overview of 
complementary funding (in million EUR)’. 
16 Survey of Horizon Europe beneficiaries, conducted May-July 2023 
17 Such as responsible research and innovation (RRI), co-design and co-creation, gender 
equality, Social Sciences and Humanities, international cooperation. 
18 Survey of Horizon Europe beneficiaries, conducted May-July 2023 

34 

 
 
The  endorsement  and  application  of  EU  Charters  and  Codes  of  Conduct  and  principles 
relating  to  gender  balance  are  often  identified  as  structural  changes  that  are  triggered  by 
Widening actions either within an institution, or across a region or country. 

The  Widening  actions  spur  international  cooperation,  and  this  is  extremely  valued  by 
widening  institutions.  This  type  of  international  research  often  cannot  be  funded  through 
national  funding  sources,  so  the  widening  actions  are therefore  indispensable  to  widening 
countries  in  this  respect.  Not  only  do  such  cooperations  allow  engagement  (to  a  limited 
extent) in excellent R&I projects, but they also allow participants to significantly strengthen 
their skills and enhance their mobility, capacities, reputation, visibility and so on. 

Relevance 

Under both Horizon 2020 and Horizon Europe, stakeholders from both widening and non-
widening countries, as well as beneficiaries, acknowledge that widening measures remain 
of high strategic importance in the context of revitalising the European Research Area and 
the  New  European  Innovation  Agenda.  Stakeholders  acknowledge  the  need  for  countries 
that are less advanced in terms of R&I to upgrade their R&I systems and build R&I capacity, 
hence  allowing  the  EU  to  advance.  The  network  analysis19  indicates  that  under  Horizon 
Europe  so  far,  it  is  non-widening  countries  that  are  central  to  the  European  research 
ecosystem.  

The bottom-up approach taken in the Widening actions is highly valued by stakeholders and 
beneficiaries  as  it  allows  researchers,  research  groups  and  organisations  in  widening 
countries to strengthen their critical mass and build capacity, as well as spurring excellence 
in any specific field that is relevant to them. The Widening actions represent one of the only 
possibilities for bottom-up collaborative research under the Framework Programme.  

Widening projects align well with current technological, scientific and socioeconomic trends 
and challenges, as well as with newly emerging needs and the EU Missions. Specifically, the 
topics  that  are  addressed  most  frequently  in  WIDERA  Widening  signed  grants  are: 
ecosystems,  climate  change,  educational  sciences,  innovation  management,  oncology, 
pollution, and the sustainable economy. 

Efficiency of the analysed programme parts 

Efficiency of the selection and implementation process 

Some  parts  of  the  selection  process  are  operating  very  smoothly,  while  others  show 
room for improvement. The main indicators are summarised below: 

• 

Ineligible proposals. So far, the share of ineligible proposals in Horizon Europe 
is  3.5%,  which  is  over  three  times  higher  than  H2020  (1.1%).  Among  Pillar  1 
programmes, INFRA saw the highest rate of ineligible proposals at 11.7%, which 
represents an increase of 9.4 percentage points from H2020. 

•  Success  rates.  Oversubscription,  which  was  noted  as  a  key  challenge  in 
H2020,  may  be  seeing  a  reduction  so  far  in  HE.  The  overall  success  rate  of 
eligible proposals has increased by 4.6 percentage points and Pillar 1 has seen 
an increase of 1.6 percentage points. No part of Pillar 1 saw a reduction in its 

19 See Annex 6, Quantitative research track findings. Network Analysis. 

35 

 
 
 
success rate, even the particularly competitive programmes, ERC and MSCA. 
Within  each  programme  part,  the  average  success  rate  for  high-quality 
proposals  is  above  the  average  success  rate  of  all  proposals.  The  difference 
was most striking in ERC, where high-quality proposals are over four times more 
successful (57% compared with 14%). 

•  Pre-financing. Over 99% of Pillar 1 is financed via pre-financing, and all of the 
pre-financing payments were made within the 30-day target. ERC was the most 
efficient programme part with an average TTP of only 11.5 days – 91.9% of its 
payments were made within the 30-day target. 

•  TTG. Project selection efficiency falls short of the targets, both overall and within 
Pillar 1. The HE average TTG is 276 days, while Pillar 1’s average is 300 days. 
However, this includes  ERC, which is  understood to  have  a  higher TTG than 
other programme parts. The main issue is that when compared with H2020, HE 
TTG is 45 days longer than H2020 at its close and 27 days longer than H2020 
during the interim. 

Use of consultants 

Applicants  across  all  pillars  report  turning  to  outside  sources  to  help  to  prepare  their 
applications. Our survey revealed that within Pillar 1, help from an internal department 
dominated, with 60% of respondents saying they had used such help. Only around 10% 
of Pillar 1 applicants hired external consultants. Looking more closely, we found that 
different  programme  parts  received  support  via  different  channels.  ERC  and  MSCA 
applicants relied on dedicated departments within their organisation for help, while the 
largest share (42%) of INFRA applicants reported providing support to their consortia. 
Among WIDERA applicants, the largest share (42%) reported receiving no support from 
any source to prepare their applications. 

Estimates from the survey were combined with CORDA data to create an extrapolation 
of  the  amount  of  money  paid  to  external  consultants.  The  median  fee  reported  by 
consortium-based  applicants 
is  EUR 500.  The  median  single-beneficiary 
programmes pay less: EUR 2 000. We estimate that between EUR 10 and 13 million 
have been spent on consultancy fees within Pillar 1 so far. This represents between 
0.1 and 0.2% of the total Pillar 1 budget.  

In addition, successful applicants reported spending an average of EUR 3 500 more than 
unsuccessful applicants due to what was noted in the free responses as the ‘success 
fee’, which can range from a flat rate, conditional on success, of EUR 2 000-5 000, or a 
percentage, from 3-7%, of the project budget, once funded. 

Success rates among those who used consultants was higher than those who did not 
across  all  Pillars,  however,  the  difference  was  less  striking  in  Pillar  1  and  WIDERA, 
where  it  was  only  6  and  9  percentage  points  higher,  respectively.  The  main  reason 
given by survey respondents from all programme parts for using consultants was 
to increase the chances of success rather than to lower the burden of application 
costs. Success rates among respondents who used NCPs was also higher across all 
Pillars, suggesting that applicants may benefit from any help on their proposal. 

36 

 
 
 
Costs of preparing an HE application 

Respondents  to  the  survey  expressed  limited  satisfaction  with  the  efforts  needed  to 
prepare their HE application. Altogether, 58% of respondents from Pillar 1 and WIDERA 
agreed with the statement that the efforts needed were in proportion to the chances of 
securing  Horizon  Europe  funding,  while  54%  of  survey  respondents  agreed  that  the 
efforts needed were in proportion to the volume of funding requested. 

We  found  that  the  majority  of  the  costs  for  preparing  the  proposal  are  borne  by  the 
project’s  coordinator.  Based  on  survey  data,  a  median  coordinator  in  HE  spends 
between  36  to  45  person-days  to  prepare  an  application.  In  contrast,  the  median 
value for contributing partners is between 16 to 25 person-days. In the case of mono-
beneficiary  programmes  (ERC  and  MSCA  PF),  the  survey  data  show  that  applicants 
take  between  36  and  45  person-days  to  submit  an  application.  Mono-beneficiary 
applicant effort is comparable to that of consortium coordinators, possibly because while 
mono-beneficiary applicants do not have to manage multiple partner consortiums, they 
still have to implement most of the steps that coordinators in consortium projects do. 

Investigating  further,  we  see  that  consortia  size,  project  size,  and  application 
outcome  also  factor  into  the  burden  of  proposal  preparation.  For  coordinators, 
every additional 15 consortium partners increases the time needed by around 10 person-
days. For partners, the effort may actually decrease when the consortium is very large. 
For coordinators, longer  projects required  more  person-days to  prepare the proposal. 
For partners, the efforts did not change. Finally, successful proposals require more effort 
from  the  coordinator  than  unsuccessful  proposals,  but  the  effort  spent  by  partners  or 
single  beneficiary  applicants  was  similar  across  both  successful  and  unsuccessful 
applicants. 

When asked to compare the effort needed to prepare a HE proposal to those needed to 
submit a H2020 proposal, respondents reported no significant difference between the 
two FPs. This suggests that the simplification efforts introduced in Horizon Europe 
have  yet  to  reach  their  full  potential  to  reduce  the  application  burden  for 
applicants.    

Simplification efforts 

Lump sums 

Lump sums, which were introduced in H2020, continue to be an important simplification 
measure in HE. It is still too early to fully assess the effect of lump sums on the whole 
project lifecycle; however, some preliminary insights can already be seen. 

About half of survey respondents suggested that lump sums do not increase the time 
needed to prepare a proposal, while another half claimed that they do. However, lump 
sums were not designed to reduce the application burden but rather to simplify project 
implementation. And indeed, a large majority (over 70%) agree to a large or very large 
extent that lump sums simplify the development of the project workplan. The majority 
(over 70%) also agree to a large or very large extent that lump sums simplify reporting 
and limit the risk of errors. 

37 

 
It  was  suggested  in  the  Open  Public  Consultation  (OPC)  by  some  organisations  that 
lump sums may introduce financial risks. Our survey investigated this attitude and found 
that 50% of respondents believed lump sums increase financial risk to a very large, large, 
moderate, or small extent. There were no trends in perceived risk among different types 
of respondents (SMEs vs non-SMEs, newcomers vs repeated FP participants). It may 
be  the  case  that  financial  risk  is  perceived  mainly  because  lump  sums  are  new  and 
beneficiaries do not yet have experience with them. 

Seal of Excellence 

To date, the EC has awarded 3 818 applications with the Seal of Excellence (SoE), most 
of which went to unsuccessful MSCA PF applications. As MSCA PF was found to have 
the  least  proportionate  level  of  proposal  costs  to  EC  funding  contribution,  SoE  could 
potentially have a significant cost-savings impact. However, the study team found that 
the impact has yet to reach its full potential. Half of survey respondents answered ‘No’ 
when asked whether the SoE made it easier for them to secure funding from regional or 
national sources. 

Administrative costs 

The  beneficiaries  survey  shows  that  a  median  consortium-run  project  allocates 
around  6-10%  of  the  project  for  administrative  tasks  (such  as  project  reporting, 
project  financial  management,  and  similar).  For  mono-beneficiary  projects,  the 
estimation is the same, 6-10%. 

According  to  the  survey,  respondents  did  not  experience  a  huge  shift  in  the 
administrative  workload  from  H2020  to  HE.  About  half  indicated  that  they  considered 
costs to be proportionate to the project funding. However, about one fifth did not consider 
them  proportionate,  suggesting  that  this  is  one  area  where  improvements  would  be 
welcome. 

1.3.  European Research Council (ERC) 

1.3.1.  ERC under Horizon Europe 

The  European  Research  Council  (ERC)  was  created  during  the  Seventh  Framework 
Programme (FP7) to reinforce research excellence by supporting investigator-driven frontier 
research across all fields20. In a context in which Europe needed to produce the very best 
cutting-edge  science  in  new  and  rapidly  emerging  fields  that  are  closely  associated  with 
world-leading innovation and the long-term growth of advanced economies, the creation of 
the ERC was expected to help the EU to remain ahead in a world of growing scientific and 
technological  competition,  as  well  as  helping  to  link  science  to  technological  innovation, 
compete for talent and encourage greater investment.  

The  main  activity  of  the  ERC  ‘is  to  provide  attractive,  long-term  funding  to  support 
excellent  investigators  and  their  research  teams  to  pursue  ground-breaking,  high-

20 https://erc.europa.eu/about-erc/erc-glance.  

38 

 
 
risk/high-gain research’21. There has been a high degree of continuity in the objectives of 
the ERC over time and across framework programmes, from 2007 to Horizon Europe.  

The ERC aims to achieve this main objective via its four types of frontier research grants, 
which operate on a 'bottom-up' basis without predetermined priorities, and are awarded via 
highly competitive calls to investigators who are seeking to set up or consolidate their own 
independent  research  team  or  programme.  ERC  grants  are  also  given  to  established 
research leaders, irrespective their gender, age, and nationality, who are working or moving 
to work in Europe: 

•  Starting Grants (StG) are designed to support talented early-career scientists who have 
already produced excellent supervised work. These grants are open to researchers of any 
nationality with 2-7 years of experience since the completion of their PhD, a scientific track 
record showing great potential, and an excellent research proposal.22 Starting Grants of 
up to EUR 1.5 million may be awarded for a period of five years. 

•  Consolidator Grants (CoG) target scientists who want to consolidate their independence 
by establishing a research team and continuing to develop a successful career in Europe. 
These  grants  are open  to  researchers  of  any  nationality with  7-12  years  of  experience 
since  the  completion  of  their  PhD,  a  promising  scientific  track  record,  and  excellent 
research.23 Consolidator Grants of up to EUR 2 million may be awarded for a period of 
five years.  

•  Advanced  Grants  (AdG)  are  designed  to  support  established,  leading  principal 
investigators who want long-term funding to pursue ground-breaking, high-risk projects. 
The beneficiaries of these grants – i.e. principal investigators (PIs) – are expected to be 
active researchers who have a track record of significant research achievements during 
the last 10 years. Each PI should be an exceptional leader in terms of the originality and 
significance  of  their  research  contributions,  with  no  specific  eligibility  criteria  laid  down 
with respect to specific academic requirements. Advanced Grants of up to EUR 2.5 million 
may be awarded for a period of five years. 

•  Synergy  Grants  (SyG)  address  researchers  who  want  to  focus  on  a  very  ambitious 
research problem. A group of two or more (up to a maximum of four) PIs working together 
can apply for these grants, with one being designated as the corresponding PI (cPI). No 
specific eligibility criteria with regard to academic qualifications are laid down in relation 
to  ERC  Synergy  Grants.  Proposals  are  evaluated  on  the  sole  criterion  of  scientific 
excellence, which takes on the additional meaning of an outstanding intrinsic synergetic 
effect.  Applications  for  Synergy  Grants  must  demonstrate  that  the  proposed  research 
cannot be carried out by a single PI working alone. Synergy Grants of up to a maximum 
of EUR 10 million can be awarded for a period of six years.  

21 ERC Work Programme 2023. Available at: https://ec.europa.eu/info/funding-
tenders/opportunities/docs/2021-2027/horizon/wp-call/2023/wp_horizon-erc-2023_en.pdf.  
22 In the Work Programme 2023, the ERC has changed the reference date for calculating the 
eligibility window for Starting Grant calls (instead of the date of the actual award, the certified 
date of the successful PhD defence is now used as a reference point, to ensure fair competition 
in terms of career stage, as the term ‘award’ varies between countries and universities). 
23 As with previous grants, under the Work Programme 2023, the ERC has also changed the 
reference date for calculating the eligibility window for Consolidator Grant calls. 

39 

 
 
The sole criterion for the selection of projects to be awarded ERC frontier research grants is 
scientific excellence. To maximise the value of the excellent research performed with the help 
of these grants, the ERC also offers Proof of Concept grants: 

•  Proof of Concept grants (PoC) are designed for all PIs who are recipients of one of the 
ERC frontier research main grants (Starting, Consolidator, Advanced or Synergy) that are 
still ongoing, or have ended less than 12 months prior to 1 January 2023. The PI must be 
able to demonstrate the relationship between the idea to be taken to proof of concept and 
the ERC frontier research project in question. The Work Programme 2023 will continue to 
award Proof of Concept Grants on the basis of a lump sum of EUR 150,000 for a period 
of 18 months. The objective of PoC grants is to enable  ERC-funded ideas to progress 
along the path from ground-breaking research towards innovation. Among other aspects, 
PoC grants are expected to help with testing, experimenting, demonstrating and validating 
the idea; conducting the research required to carry out the above activities and to address 
the  weaknesses  uncovered by  them;  establishing  viability,  technical  issues  and  overall 
direction; clarifying the strategy for the protection of intellectual property rights (IPR) or 
knowledge transfer; involving industrial partners, societal or cultural organisations, policy-
makers  or  any  other  potential  stakeholders  supporting  the  translation  of  the  research 
results into innovation; and assessing potential ‘end users’ of the expected innovation. 

A Monitoring and Evaluation Strategy24 was developed by the Scientific Council of the ERC 
to continuously assess progress towards its objectives as well as the overall success of the 
initiative in  terms  of  the  four specific  objectives listed  below  and  their  various  dimensions. 
The  report  outlining  this  strategy  was  published  in  2018,  and  no  further  update  has  been 
published  by  the  ERC.  However,  given  the  high  degree  of  continuity  of  the  ERC  under 
Horizon Europe in comparison to Horizon 2020, the dimensions included in the Monitoring 
and Evaluation Strategy Report (2018) are still considered valid: 

•  The contribution of ERC funding activities to advancing the frontier of knowledge and to 
the scientific and technological impacts of new ideas in the short and long-term, especially 
in terms of: 

−  advancing knowledge: new and often unexpected scientific and technological results 
and, among these, revolutionary discoveries that trigger scientific paradigm shifts or 
become the core principles behind radical technological developments; 

−  supporting research in emerging areas: the creation and development of new scientific 
fields;  the  novelty  and  originality  of  associating  and  combining  research  areas;  the 
development of new scientific instruments and methodologies; 

−  performing  globally:  reinforcing  the  performance  and  status  of  European  research 

globally. 

•  The ERC’s contribution to improving the career prospects of European researchers, to 
training the new generation of researchers, and to attracting the world’s best scientists to 
Europe. The following dimensions of this contribution are considered: 

−  promoting  talent:  the  success  of  ERC-funded  researchers  in  their  professional 
careers, their ‘empowerment’ in controlling their research activities and, in the case of 

24 European Research Council. ERC Monitoring and Evaluation Strategy 2018. Brussels, 
October 2018. Available at: https://erc.europa.eu/sites/default/files/content/pages/pdf/erc-
monitoring-and-evaluation-2018.pdf. 

40 

 
 
young  researchers,  the  acceleration  of  their  transition  to/consolidation  of  their 
academic independence; 

−  nurturing talent: training of the next generation of researchers; 

−  attracting  talent:  attracting  talented  researchers  to  Europe  from  abroad,  including 

Europeans resident outside Europe. 

•  The ERC’s contribution, in addition to ongoing efforts at Union, national and regional level, 
to  reforming,  strengthening  and  unlocking  the  full  potential  and  attractiveness  of  the 
European research system. The following dimensions of this contribution are considered: 

−  host  institutions:  improving  the  quality  of  research  environments  in  Europe’s 
universities  and  other  research  organisations,  as  well  as  the  working  conditions 
offered to top researchers; 

− 

− 

research  policies:  inspiring  frontier  research  in  Europe,  opening  up  new  ways  of 
working within the scientific world and stimulating national research systems to reform 
their policies, priorities and fundings models; 

funding  structures:  the  influence  of  the  ERC,  its  funding  activities  and  operational 
practices  in  shaping  the  administrative  organisation  of  research  and  the  funding 
mechanisms for frontier research. 

•  The  ERC’s  contribution  to  the  knowledge created  over long  periods  of  time in  order  to 
drive  commercial  and  social  innovation  and  the  inventiveness  of  business,  generating 
economic growth, tackling global challenges and influencing policymaking. 

In addition to the above contributions, it is important to note that, under Horizon Europe, the 
objectives of openness, spreading excellence and the diffusion of knowledge, collaboration 
across disciplines and sectors, and strengthening of international collaboration, have been 
reinforced. The intervention logic including these objectives and their dimensions is depicted 
in Figure 1. This details the pathway from inputs to the (broader) impacts of the ERC. 

41 

 
Figure 1. Intervention logic of ERC – Horizon Europe. Source: compiled by the authors, based on ERC Monitoring and Evaluation Strategy 2018, interim evaluation of Horizon 2020, and 
Horizon Europe 2021-2022-2023 ERC work programmes. 

42 

 
 
 
1.3.2.  Effectiveness  

Under Horizon Europe, the ERC’s mission continues to encourage research of the highest 
quality  in  Europe  through  competitive  funding,  and  to  support  investigator-driven  frontier 
research across all fields, based on scientific excellence.  

The analysis to assess the effectiveness of the ERC has been carried out by analysing to 
what extent the ERC has achieved the following objectives: 

•  Advancing the frontier of knowledge and the scientific and technological impact of new 

ideas in the short and long term; 

• 

Improving the career prospects of European researchers, training the new generation of 
researchers, and attracting of the world’s best scientists to Europe; 

•  Reforming,  strengthening  and  unlocking  the  full  potential  and  attractiveness  of  the 

European research system; 

•  Driving  commercial  and  social  innovation  and  business  inventiveness,  generating 

economic growth, tackling societal challenges and influencing policy-making. 

1.3.2.1. 

Advancing frontier research and the scientific and technological impact of 
new ideas in the short and long term 

As  Horizon  Europe-funded  projects  are  currently  only  part-way  through  their  duration, 
representative data on scientific publications and other outputs are not yet available for this 
FP. Therefore, the analysis based on bibliometric data is carried out using Horizon 2020 data, 
as there is a high degree of continuity between ERC actions/between the two programmes. 

Scientific output and quality 

Since  2015,  the  ERCEA  has  been  conducting  a  qualitative  evaluation  of  completed  ERC 
projects.  The  latest  edition,  published  in  the  Annual  Report  on  ERC  activities  and 
achievements in 202225, uses a random sample of 450 projects and concludes that 21% of 
the  projects  led  to  a  scientific  breakthrough,  while  59%  led  to  a  major  scientific 
advance.  Only  one-fifth  of  projects  were  found  to  produce  incremental  or  no  appreciable 
scientific advance. These results have been consistent over time (see Figure 2).  

Given  the  continuity  and  stability  of  the  ERC  programme  part  between  Horizon  2020  and 
Horizon Europe, it is expected that Horizon Europe will show similar trends to Horizon 2020 
in this respect.  

25 European Research Council (2023). Annual Report on the ERC activities and achievements in 
2022. Available at: https://op.europa.eu/en/publication-detail/-/publication/3a11d526-ceab-11ed-
a05c-01aa75ed71a1. 

43 

 
 
 
Figure 2. Shares of ERC projects, by scientific outcome. Source: Annual Report on ERC activities and achievements in 
2022. 

Data from the Survey of Horizon Europe ERC beneficiaries presents preliminary findings on 
the type of project outputs expected (see Figure 3). Almost all respondents (236 out of 238) 
stated that their ERC project has produced, or is likely to produce, research publications (in 
the  form  of  academic  articles,  monographs,  books,  etc.).  A  majority  of  respondents  also 
expected  new/improved  research  methodologies,  new  large-scale  datasets,  and  new  or 
improved  tools,  methods  or  techniques.  Meanwhile,  only  1%  (2  out  of  238  respondents) 
expected their ERC project to produce large-scale product validation and market replication. 
These types of expected outcomes are in line with the types of projects funded by the ERC, 
which focuses on bottom-up fundamental research.  

99%

61%
61%

52%

Research publications

New/improved research methodologies

New large-scale datasets/ data produced

New or improved tools, methods or techniques

Educational materials

Research roadmaps

New/improved software, protocols/guidelines to…

Prototype

Patents

Testing, demonstrating and piloting

Recommendations for policy makers

Collaboration platforms

Training programmes

New or improved products, services or processes

Pre-standards/standards

Social innovation (tools, guides, strategies)

Codes of conduct

Other (please indicate):

Business plans for the development of a center of…

Large-scale product validation and market replication

34%
32%

29%

23%
22%

20%
18%

16%
14%

9%

6%
5%
4%
3%
2%
1%

Figure 3. Share of respondents by type of expected project outputs. Source: survey of Horizon Europe ERC beneficiaries 
(ERC-BEN-027), conducted May-July 2023 (n=238). 

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%

44 

 
 
 
 
Data extracted from the Horizon Dashboard26 show that the top 10 STEM fields, in terms 
of  signed  grants  under  Horizon  2020  (based  on  EuroSciVoc  concepts)  are mostly  in  the 
field of biology (e.g. proteins (583), DNA (390), genomes (328) or neurobiology (316)). Other 
top fields include oncology (429), microscopy (274) and geometry (259). In the fields of SSH, 
history (386), law (129) and revolutions (120) stand out as those with the largest numbers of 
signed  grants  in  Horizon  2020.  This  pattern  appears  to  be  similar  under  Horizon  Europe, 
albeit with certain fields such as climate change (68), machine learning (63) and governance 
(22) gaining more momentum. The increased importance of these fields aligns with societal 
changes and challenges relating to climate change and artificial intelligence, as well as to 
recent developments such as the energy crisis, COVID-19 pandemic and the Ukrainian crisis, 
which affect the EU and the rest of the world. 

However, looking at research trends based on the number of scientific publications from 
Horizon 2020 ERC projects (data for Horizon Europe are not yet available), different top fields 
emerge. The majority of scientific production occurred in Physical Sciences (62.6%), followed 
by Life Sciences (17.7%). Among the Physical Sciences, chemistry, physics and astronomy 
stand out as the subfields with the highest numbers of scientific publications.   

Figure 4. Shares of scientific publications from Horizon 2020 ERC projects, by discipline. Source: analysis of scientific 
disciplines and topics 

The two case studies developed in Annex 2 to assess the extent to which the ERC contributes 
to  trending  research  pathways  in  the  fields  of  Life  Sciences,  Physical  Sciences  and 
Engineering  on  the  one  hand,  and  Social  Sciences  and  Humanities  (SSH)  on  the  other, 
further elaborate on this analysis of Horizon 2020 scientific publications. 

With  regard  to  the  fields  of  Life  Sciences,  Physical  Sciences  and  Engineering,  the  main 
findings are as follows: 

•  Based on a ranking of the  most prominent topics27 for the ERC, ERC-funded research 
has  contributed  the  most  in  terms  of  quantity  of  outputs  (in  this  case,  scientific 
publications) in the fields of Physics, Astronomy and Engineering. These include, for 
instance  ‘Perovskite  Solar  Cells;  Solar  Cell;  Formamidine’,  ‘Exoplanets;  Kepler; 
Atmosphere’ and ‘Quantum Optics; Thermalization (Energy Absorption); Eigenvalues and 
Eigenfunctions’.  Meanwhile,  in  terms  of  impact  (measured  by  scientific  publication 

26 See Annexes 2.1 and 2.2. 
27 For more information on the methodology used, see Annex 6. 

45 

 
 
 
 
citations), the top 10 most prominent topics occur mostly in the fields of Biology, Clinical 
medicine  and  Chemistry.  These 
‘Glycolysis;  Metabolic; 
include, 
Mammalian  Target  of  Rapamycin  Complex’, 
‘Alkene;  Cross-coupling  Reactions; 
Photocatalysis’ and ‘Nivolumab; Pembrolizumab; Immunotherapy’. The field of Astronomy 
– in particular, the topic of ‘Neutron Stars; Mergers; Gravitational Waves’ – combines a 
large number of publications with a very strong impact indicator.  

instance: 

for 

•  From  an  international  standpoint,  the  analysis  of  research  trends28  for  eight  selected 
topics shows variations in the EU’s position compared with other parts of the world. 
For example, the EU-27 publications represent 52% of the total number of publications 
on the astronomy topics of ‘Star Formation; Galaxies; Galactic Evolution’ and ‘Exoplanets; 
Kepler; Atmosphere’; 51% on the topic of ‘Neutron Stars; Mergers; Gravitational Waves’; 
and 56% on the field of ‘Planet; T Tauri Star; Pre-main Sequence Stars’ – each of these 
being far beyond the shares of other countries in the analysis. For all four topics, ERC-
funded  research  represents  high  shares  of  the  overall  Horizon  2020  number  of 
publications, thus indicating that these specific astronomy topics are mainly covered by 
ERC-funded research, and less so by other parts of the Framework Programme. On the 
other hand, despite the significant ERC’s contribution on the topic of perovskite solar cells 
in Europe, China appears to have produced a larger share than Europe (as a whole) of 
global research output on this topic.  

With regard to the field of Social Sciences and Humanities, the main findings are as follows: 

•  Similarly,  based  on  a  ranking  of  the  most  prominent  topics,  ERC-funded  research  has 
contributed the most in terms of quantity of outputs (in this case scientific publications) in 
the  fields  of  Social  Sciences  and  Historical  Studies.  These  include,  for  instance 
‘Genome;  Neanderthals;  Anatomically  Modern  Humans’,  ‘Human-Robot  Interaction; 
Humanoid  Robot;  Man-Machine  Systems’  and  ‘Punishment;  Ultimatum  Game;  Guilt 
Aversion’. In terms of impact (as measured by scientific publication citations), the top 10 
most prominent topics occur mostly in the fields of Social Sciences and in Economics 
and Business. These include, for example ‘Platforms; Collaborative Consumption; Peer 
to  Peer’,  ‘Water  Footprint;  Water-Energy  Nexus;  Nexus’  and  ‘Environmental  Justice; 
Injustice;  Air  Quality’.  However,  the  topic  with  the  highest  share  of  top  1%  most-cited 
scientific publications, ‘Journalism; News; Social Media’, is in the field of Communication 
and  Textual  Studies.  Meanwhile,  the  topic  of  ‘Genome;  Neanderthals;  Anatomically 
modern humans’ in particular combines a large number of publications with a very strong 
impact indicator. 

•  From  an  international  standpoint,  the  analysis  of  research  trends29  for  seven  selected 
topics shows that the EU-27 as a whole contributes strongly to scientific production 
in the field of SSH, with most shares above those of the US and China. However, the 
analysis indicates that for all the topics selected,  the extent to which Horizon 2020 has 
contributed  to  these  fields  is  slightly  more  limited  than  is  the  case  for  STEM  topics. 
Furthermore,  within  the  contribution  made  to  SSH  by  Horizon  2020,  the  ERC  has 
contributed to a varying extent to different topics. However, it should be borne in mind 
that the analysis performed did not take into account other outputs such as books, which 
are more prominent in SSH fields. The highest share of ERC-related publications is found 
on  the  topic  of  ‘Punishment;  Ultimatum  Game;  Guilt  Aversion’  (83%),  followed  by 
‘Greenspace; Green Infrastructure; National Parks’ (40%) and ‘Water Footprint; Water-
Energy Nexus; Nexus’ (35%), where the EU outperforms all other regions in these three 
fields.  

28 For more information, see Annex 6. 
29 For more information, see Annex 6. 

46 

 
 
 
While the results above focus on the Top 200 most prominent topics that have produced the 
highest numbers of publications, a question arises as to how much EC funding went to these 
topics, and hence how ‘productive’ the ERC was in terms of the creation of  knowledge on 
these specific topics. An analysis conducted for the present study shows that the prominent 
topics that have led to the largest number of publications in each programme part are not 
necessarily  the  ones  that  are  most  productive  (i.e.  those  with  the  highest  number  of 
publications per million EUR of EC funding)30. 

Another analysis conducted during  this study  examines whether  receiving  FP  funding  has 
made  a  difference  in  terms  of  the  novelty  and  disruptiveness  of  the  research  publications 
produced later on in the researchers’ careers31. The results of this analysis do not provide 
significant evidence that researchers who received funding from the ERC under Horizon 2020 
have  gone  on  to  generate  more  disruptive  research  outputs  two  or  five  years  after  their 
application, compared with counterparts who applied and were positively evaluated, but who 
did not receive funding. This analysis focuses on funded and non-funded applicants that were 
considered  of  high  quality  by  the  ERC  evaluation  process.  Given  this  qualification, 
unsuccessful researchers are likely to have obtained funding for their  research from  other 
sources, and thus also produced impactful outputs.   

Top researchers in the ERC 

The advancement of the frontiers of knowledge has also been reflected in the many prizes 
that ERC grantees have won over time. In fact, 44.9% (106 out of 236) of the ERC PIs who 
responded to the survey of Horizon Europe ERC beneficiaries indicated that they expected 
to have their work recognised through prizes or awards as a result of their participation in the 
ERC  (ERC-BEN-032)32,  a  higher  value  than  the  34.1%  of  respondents  who  responded 
similarly to the survey of Horizon 2020 ERC beneficiaries conducted in 2022.  

• 

In total, 14 ERC grantees have been awarded the Nobel Prize, seven of which were in 
the last three years (see Table 1). Most of these Nobel Prize winners come from the field 
of physics, while one comes from SSH. Although one needs to take into account that the 
ERC was only established in 2007, it is interesting to note that the US National Science 
Foundation (NSF) has supported 258 Nobel laureates33 since its establishment in 1950. 

Nobel Prize winner 

Field 

Prof. Konstantin Novoselov 

Nobel Prize in Physics 

Prof. Serge Haroche 

Nobel Prize in Physics 

Year 

2010 

2012 

30 For each programme part, this funding analysis shows the Top 200 most prominent topics that 
produced the largest number of publications, along with the topic’s total EC funding, and the 
number of publications published per million EUR of total EC funding. More information and 
results can be found in Annex 6 ‘Quantitative research track findings’ and Annex 2 ‘Case Study 9 
on Creating high-quality new knowledge’. 
31 More information on these indicators and the analysis is found in the ‘Novelty and 
disruptiveness’ analysis in Annex 6. 
32 Survey of Horizon Europe beneficiaries, conducted May-July 2023 
33 https://www.nsf.gov/news/special_reports/nobelprizes/. 

47 

 
 
 
Nobel Prize winner 

Field 

Prof. Edvard I. Moser 

Prof. May-Britt Moser 

Prof. Jean Tirole 

Nobel Prize in Physiology or 
Medicine 

Nobel Prize in Physiology or 
Medicine 

Sveriges Riksbank Prize in 
Economic Sciences (Nobel 
Memorial Prize in Economic 
Sciences) 

Prof. Bernard Feringa 

Nobel Prize in Chemistry 

Prof. Peter J. Ratcliffe 

Nobel Prize in Physiology or 
Medicine 

Prof. Giorgio Parisi 

Nobel Prize in Physics 

Prof. Benjamin List 

Nobel Prize in Chemistry 

Prof. Svante Pääbo 

Nobel Prize in Physiology or 
Medicine 

Prof. Alain Aspect 

Nobel Prize in Physics 

Prof. Anton Zeilinger 

Nobel Prize in Physics 

Prof. Ferenc Krausz 

Nobel Prize in Physics 

Prof. Anne L’Huillier 

Nobel Prize in Physics 

Year 

2014 

2014 

2014 

2016 

2016 

2021 

2021 

2022 

2022 

2022 

2023 

2023 

Table 1. Overview of ERC grantees who have won a Nobel Prize. Source: ERC news. 

• 

In total, 11 ERC grantees were awarded the Wolf prize, six of which were in the last three 
years (see Table 2). Wolf Prizes are awarded to outstanding scientists and artists from 
around the world, for achievements in the interest of mankind and friendly relations among 
peoples. The scientific categories of the prize include Medicine, Agriculture, Mathematics, 
Chemistry  and  Physics.  Several  winners  of  the  Wolf  prize  have  also  obtained  an  ERC 
grant after being awarded the prize. 

Wolf Prize winner 

Field 

Prof. Anne L’Huillier 

Prof. Ferenc Krausz 

Physics 

Physics 

48 

Year 

2022 

2022 

 
 
Wolf Prize winner 

Field 

Prof. Giorgio Parisi 

Physics 

Prof. Caroline Dean 

Agriculture 

Prof. Simon Donaldson 

Mathematics 

Prof. Jean-François Le Gall 

Mathematics 

Prof. Leif Andersson 

Agriculture 

Prof. Peter Zoller 

Prof. Alain Aspect 

Prof. Anton Zeilinger 

Physics 

Physics 

Physics 

Year 

2021 

2020 

2020 

2019 

2014 

2013 

2010 

2010 

Table 2. Overview of ERC grantees who have won a Wolf prize. Source: ERC news. 

•  Six  ERC  grantees  have  won  a  Fields  Medal  (see  Table  3).  This  prize  recognises  and 
rewards  young  mathematicians  who  have  made  major  contributions  to  the  field  of 
mathematics and who hold the promise of future achievements.  

Fields Medal winner 

Prof. James Maynard 

Prof. Hugo Duminil-Copin 

Prof. Alessio Figalli 

Prof. Artur Avila 

Prof. Martin Hairer 

Prof. Stanislav Smirnov 

Year 

2022 

2022 

2018 

2014 

2014 

2010 

Table 3. Overview of ERC grantees who have won the Fields Medal. Source: ERC news 

49 

 
 
• 

In addition to the prizes mentioned above, numerous other important prizes have been 
awarded to ERC grantees, including in the fields of SSH. For example:  

−  The Swedish Society for Anthropology and Geography (SSAG) awards a Gold Medal 
to world-leading scholars in anthropology and human geography34. The winners of the 
prize in 2016 (Prof. Didier Fassin) and in 2022 (Prof. Thomas Hylland Eriksen) have 
both been ERC grantees.  

−  The government of Norway annually awards the Holberg Prize to outstanding scholars 
in the fields of arts, humanities, social sciences, law and theology, either within one of 
these  fields  or  through  interdisciplinary  work35.  The  winners  in  2022  (Prof.  Joan 
Martinez Alier) and in 2013 (Prof. Bruno Latour) were ERC grantees.  

−  The  International  Science  Council,  the  University  of  Bergen  and  the  European 
Consortium  for  Political  Research  award  the  Stein  Rokkan  Prize  for  Comparative 
Social  Science  Research,  in  memory  of  the  political  scientist  and  sociologist  Stein 
Rokkan.  The  prize  is  awarded  to  outstanding  scholars  who  have  made  a  ‘very 
substantial  and  original  contribution  in comparative social  science  research’36.  Prof. 
Marius Busemeyer, an ERC grantee, won the prize in 2015.  

Communication and dissemination of research results 

The ERC has a mandate to communicate to the scientific community, key stakeholders, the 
media and the public at large37. According to the External Communication Strategy 2023, the 
ERC’s  communication  tools  and  channels  are  its  website,  multimedia  (through  video 
production, infographics and podcasts), social media, its online magazine, email alerts and 
publications (with a focus on reader-friendly online formats). In addition, the ERC is frequently 
mentioned  in  the  news,  and  plans  to  explore  collaborations  with,  for  example,  television 
programmes  about  science.  Lastly,  the  National  Contact  Points,  Host  Institutions, 
Representations, or the European Parliament are also channels of communication.  

The  ERC  Annual  Activity  Reports  present  the  ERC’s  annual  communication  and 
dissemination indicators and activities. The figures presented in them show that the ERC’s 
media mentions have more than doubled in comparison to 2014; the number of subscribers 
to the ERC newsletter and news updates are 2.5 times higher than at the beginning of Horizon 
2020;  and  the  number  of  unique  website  visitors  has  increased  from  500,000  in  2014  to 
around 1 million in 2022. Social media followers have also increased significantly, especially 
users of Twitter (now known as X) and LinkedIn.  

2022 

2020 

2018 

2016 

2014 

Press announcements released 
by the ERC 

43 

35 

30 

35 

27 

34 https://en.wikipedia.org/wiki/Swedish_Society_for_Anthropology_and_Geography#Retzius_Me
dal. 
35 https://en.wikipedia.org/wiki/Holberg_Prize. 
36 https://en.wikipedia.org/wiki/Stein_Rokkan_Prize_for_Comparative_Social_Science_Research 
37 European Research Council (2023). Annual Report on the ERC activities and achievements in 
2022. Available at: https://op.europa.eu/en/publication-detail/-/publication/3a11d526-ceab-11ed-
a05c-01aa75ed71a1. 

50 

 
 
 
 
2022 

2020 

2018 

2016 

2014 

Media mentions of the ERC 

16,500 

14,200 

14,700 

7000 

Twitter followers 

141,100 

98,200 

56,000 

28,000 

11,400 

LinkedIn followers 

129,476 

75,500 

Facebook followers 

38,043 

32,700 

25,000 

14,600 

7,000 

YouTube followers 

4,700 

Views of ERC videos across 
different platforms 

600,000 

Potential reach on ERC social 
media 

1.03 
billion 

406 
million 

ERC magazine and news 
update subscribers 

70,000 

69,000 

47,000 

35,000 

27,000 

Social media mentions of the 
ERC 

126,000 

124,000 

100,000 

Website unique visitors 

1 million 

947,000 

612,000 

550,000 

500,000 

Table 4. ERC communication and dissemination indicators. Source: compiled by the study team, based on the Annual 
Activity Reports of the ERC and the Communication Strategy 2023. 

In addition to its own communication and dissemination activities, the ERC also supports 
and encourages dissemination activities by the research community. Dissemination and 
communication activities, such as participations in workshops, conferences and contact 
with the media and the research community were identified by several interviewees as 
one of the main factors contributing positively to the impact of a project, not only at the 
technical/scientifical level, but also at societal level. However, these types of activities 
often require part of the project’s budget to be spent on travel and the organisation of 
large events. 

In relation to publishing research results in open access, 26% of the respondents (59 of 
227)  indicated  that  they  had  not  experienced  any  difficulties38.  This  represents  a 
significant decline compared with the percentage reported in the survey of Horizon 2020 
ERC  beneficiaries,  conducted  August-September  2022  (in  which  the  corresponding 
figure  was  41%),  implying  that  more  researchers  now  experience  difficulties  in  this 
regard than previously. This may be explained by new requirements implemented under 
Horizon Europe – for example, the obligation to develop data management plans (DMPs) 
and  to  provide  immediate  open  access  to  their  peer-reviewed  publications  and 

38 survey of Horizon Europe ERC beneficiaries (ERC-BEN-037), conducted May-July 2023 
(n=227). 

51 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
associated metadata39. Among the ERC respondents, 29% (66 out of 227) indicated that 
they had experienced difficulties related to accessing funding to pay article processing 
fees, while 26% (59 out of 227) pointed to a lack of guidance on the implementation of 
the DMP.  

Lastly, in its 2023 work programme (WP), the ERC allocated EUR 1.5 million for the ERC 
Science Journalism Initiative, as it wishes to support an organisation or a consortium of 
organisations  to  set  up  a  funding  scheme  that  would  facilitate  stays  at  research 
institutions  of  three  to  five  months  by  journalists,  in  order  to  give  them  access  (for 
example,  via  ERC  grantees)  to  a  better  understanding  of  frontier  research,  while 
respecting journalistic independence.  

1.3.2.2. 

Improving the career prospects of European researchers, training the new 
generation of researchers, and attracting of the world’s best scientists to 
Europe 

Impact on the career prospects of European researchers 

In  the  survey  of  Horizon  Europe  ERC beneficiaries,  71%  (171  out  of  241 ERC successful 
applicants, ERC-BEN-025) stated that their ERC project gave them, to a ‘very large’ or ‘large’ 
extent,  the  opportunity  to  gain  a  research  position  with  better  career  prospects.  The 
percentage is even higher (91% or 218 out of 240 respondents) when it comes to increasing 
their  international  visibility  and  the  outreach  of  their  research.  However,  only  51%  (124 
respondents) stated that participation in the ERC had given them, to a very large or large 
extent,  attractive  contractual  conditions  or a  better  salary.  Among  unsuccessful  applicants 
(ERC-APPL-023), the shares were much lower, with only 53% (549 out of 1 370) stating that, 
Horizon Europe responds, to a very large or large extent, to their need to gain a research 
position with better career prospects; 68% (658 out of 1 365 respondents) to their need for 
international visibility; and only 38% (520 out of 1 367 respondents) to their need for better 
contractual  conditions  and  salary.  With  regard  to  aspects  relating  to  contractual  or  salary 
conditions, it should be noted that such factors are also affected by national or institutional 
employment conditions and standards.  

A majority of respondents to the survey of Horizon Europe successful applicants (ERC-BEN-
032) expected their participation in ERC to impact their career development in the following 
ways:  

•  Strengthening  their  capacity  to  conduct  frontier  research  (93%;  219  out  of  236 

respondents); 

•  Strengthening their reputation as a researcher (91%; 214 respondents); 

•  Establishing and/or consolidating their own research group (81%; 191 respondents); 

•  Obtaining a new fellowship/research grant (76%; 180 respondents); 

•  Becoming an established researcher (70%; 165 respondents); 

•  Progressing  towards  a  higher level  of  responsibility/moving  into  a  more  senior position 

(e.g. full professorship) (50%; 118 respondents). 

These aspects were confirmed during the illustrative interviews carried out with ERC PIs in 
the context of the case studies carried out for this study. Some interviewees mentioned, for 

39 For more information, please see the Case Study on Knowledge Diffusion and Open Access.  

52 

 
 
 
instance,  that  their  ERC grant had  enabled  them  to  obtain  a  permanent  position,  to  focus 
more or solely on research, and to establish their position as a top scholar.  

In  addition,  84%  (196  out  of  233  successful  ERC  applicants)  ‘strongly  agreed’  or  ‘rather 
agreed’ that, compared with the research funding available to them at national and/or regional 
level,  Horizon  Europe  provides  more  international  mobility  opportunities  for  researchers. 
Meanwhile, 72% (168 respondents) and 75% (171 respondents) strongly or rather  agreed 
that  it  is  more  likely  to  make  them  pursue  their  career  in  Europe  (as  opposed  to  seeking 
opportunities outside Europe), and that it helps them to better pool the skills and expertise 
necessary for research, respectively (ERC-BEN-044).  

In terms of gender equality, the shares of ERC applicants have improved slightly over time. 
Data on ERC calls between 2007 and 2021 show a share of 28% of female ERC applicants40, 
while preliminary and partial EC administrative and monitoring data on Horizon Europe from 
May 2023 show a 33% share of female applicants to the ERC41. In terms of ERC grantees, 
between  2007  and  2021,  the  share  of  female  grantees  was  27%,  while  preliminary  data 
provided  by  the  European  Commission  in  January  2024  show  this  has  increased  to  36% 
under  Horizon  Europe.  In  addition  to  Horizon  Europe-wide  actions  aimed  at  incentivising 
applications  from  female  researchers,  the  ERC  has  been  carrying  out  changes  to  its 
application rules – for example, with respect to maternity leave conditions. 

Support for attracting the world’s best scientists to Europe  

As explained above, the ERC contributes substantially to producing scientific excellence in 
the EU. This is reflected in its publication track record, the scientific quality of the research 
produced, the many prizes and awards received by ERC grantees, and the ground-breaking 
nature of ERC research projects. In particularly, under Horizon 2020, around 18% of ERC 
team members were non-ERA nationals, originating from almost 90 countries worldwide42. 
The  four  most  frequent  nationalities  among  non-ERA  ERC  team  members  under  Horizon 
2020 were those from India (18% of non-ERA nationalities), China (17%), the US (12%) and 
Russia (7%). The ERC Activity Report for 2020 also indicated that the largest share of non-
ERA team members was found in the category of postdoctoral researchers.  

Preliminary administrative and monitoring data on Horizon Europe received by the study team 
in May 2023 shows 109 nationalities among the total of 12,417 PIs and corresponding PIs of 
ERC applications, and 66 nationalities among the total of 2,481 PIs and corresponding PIs 
of  ERC-funded  projects.  Among  these  ERC-funded  projects,  around  10%  of  PIs  and 
corresponding PIs are of non-ERA nationality, with the most common being the United States 
(33% of non-ERA nationalities), China (15%) and India and Canada (8% each). 

Impacts on researchers’ skills and competences 

Horizon Europe strengthens the human capital engaged in R&I, as is shown by the positive 
effects it has so far had on the career prospects and reputations of researchers, as well as 
on  skills  development,  international  and  intersectoral  mobility,  and  working  conditions.  In 
particular, the ERC is supporting more than 600 researchers currently involved in ongoing 
projects.  Notably,  468  of  these  researchers  are  involved  in  upskilling  activities  (training, 

40 European Research Council, Gender statistics, 2022, available at: 
https://erc.europa.eu/sites/default/files/2022-11/Gender_statistics_June_2022.pdf.  
41 For more information, please see the Case Study on Gender Equality. 
42 European Research Council (2021). Annual report on the ERC activities and achievements in 
2020, available at: https://op.europa.eu/en/publication-detail/-/publication/09a3dcb6-9ccb-11eb-
b85c-01aa75ed71a1/language-en/format-PDF/source-283364380. 

53 

 
 
 
mentoring/coaching, mobility and access to R&I infrastructures) in projects funded under the 
Framework  Programme  (KIP243).  The  upskilling  of  researchers  is  reflected  by  those 
researchers who are in the earlier stages of their research careers (i.e. first-stage researchers 
and  recognised  researchers).  Meanwhile,  the  ERC  is  so  far  supporting  417  established 
researchers (i.e. senior researchers and top-grade researchers)44.  

The  survey  of  Horizon  Europe  ERC  beneficiaries  conducted  May-July  2023,  confirms  the 
ERC’s  impact  on  researchers’  skills  and  competences.  It  indicates  that  for  83%  of 
respondents (198 out of 233 respondents), opportunities to improve skills and competences 
were  to  a  ‘very large’  or ‘large’  extent  a  motivating  factor  to  apply  for ERC funding  (ERC-
BEN-025). However, among the unsuccessful applicants, the corresponding share was just 
47% (682 out of 1,367 respondents) (ERC-APPL-023).  

According  to  respondents  to  the  ERC  beneficiaries’  survey,  the  skills  that  are  most 
extensively  developed  through  ERC  projects  are  ‘project  and  people  management’  and 
‘scientific methods and/or techniques’, both of which were stated by 83% of respondents (192 
out of 233) to have been improved to a very large or large extent; and ‘thinking’ (78%; 196 
out  of  237)  (see  Figure  5).  The  least  well-developed  skills  were  in  the  ‘sustainable 
implementation of research activities’ and the ‘commercialisation of research outputs’ (each 
with a share of 18%, and 41 out of 230 and 43 out of 234 respondents, respectively. 

43 KIP2 or Key Impact Pathway 2 in the monitoring of Horizon Europe relates to ‘Strengthening 
human capital in research and innovation’. The figure presented here reflects the short-term 
KIP2 indicator, and is provided by the EC.  
44 Categorisation as defined in the Frascati 2015 Manual and identified as a short-term indicator 
for KIP2. 

54 

 
 
 
Figure 5. Impact of ERC projects on the development of skills. Source: survey of Horizon Europe ERC beneficiaries (ERC-
BEN-031), conducted May-July 2023 (n=233). 

Support for international and intersectoral mobility 

According to the survey of Horizon Europe ERC beneficiaries conducted May-July 2023, 61% 
of the responding beneficiaries (146 out of 241) stated that their ERC project responds to a 
‘very large’ or ‘large’ extent to their needs with regard to international mobility opportunities 

55 

 
 
 
(ERC-BEN-025). Moreover, 84% (196 out of 233) ‘strongly’ or ‘rather’ agreed that Horizon 
Europe provides more international mobility opportunities for researchers, compared with the 
research  funding  available  at  national  and/or regional  level  (ERC-BEN-044).  Furthermore, 
several  examples  were  given  during  the  interviews  of  mobility  opportunities  for  team 
members  or  the  recruitment  of  international  team  members,  which  was  also  said  to  have 
been enabled by the size and the bottom-up character of the grants, which provide a greater 
degree  of  freedom  to  define  a  relevant  research  approach  that  includes  international 
elements. 

Among non-successful applicants, similar share of respondents (53%, or 698 out of 1,366) 
stated that Horizon Europe responds to a ‘very large’ or ‘large’ extent to their needs in terms 
of international mobility opportunities (ERC-APPL-023).  

Overall,  intersectoral  (industry-academia)  mobility  remains  very  limited  among  ERC 
grantees, which is to be expected given the design and objectives of these grants. The survey 
(ERC-BEN-028) shows that only one out of 237 ERC PIs indicated that their ERC project had 
contributed  to  a  move  from  industry  to  academia  at  the  start of  the  project.  Nevertheless, 
20% of respondents to the beneficiaries’ survey considered that their ERC project responds 
to a ‘very large’ or ‘large’ extent to their needs with regard to opportunities for intersectoral 
mobility (especially industry-academia) (ERC-BEN-025)45.  

1.3.2.3. 

Reforming, strengthening and unlocking the full potential and 
attractiveness of the European research system 

Attractiveness of the European research system 

As it does for researchers within Europe, the ERC also proves attractive to researchers from 
associated and non-associated third countries. According to the Horizon Dashboard46, non-
Member States so far account for 13% (347 out of 2,727) of the total ERC participations in 
Horizon Europe, and have received 12% of the EU’s net contribution (EUR 471.6 million out 
of EUR 3.9 billion). These figures are 1 percentage point lower than those observed under 
Horizon 2020, which may be explained by the difficulties associated with the association to 
Horizon Europe of Switzerland47, which was the second largest participant in Horizon 2020, 
as well as by the fact that until very recently, the UK was not yet an associated country. In 
total, 44 countries have been hosting ERC projects under Horizon Europe, which compares 
with a total of 65 countries under Horizon 2020. Up to the present, the non-Member States 
with  the  highest  number  of  participations  in  the  ERC  programme  part  are  Israel  (187 
participations), Norway (68 participations) and the United Kingdom (31 participations).  

All PIs interviewed in the context of the case studies agree on the attractiveness of the ERC 
as  an  instrument  for  academic  researchers  performing  basic  research,  and  on  the  added 
value it brings to the EU research system (see also Section 1.3.4 on EU added value of the 
ERC). During an interview, one PI mentioned that the existence of the ERC was one of the 
incentives to stay in Europe. The attractiveness of the ERC as a funding instrument may thus 
be considered to have an influence on the attractiveness of the European research system. 
The factors that make the ERC an attractive instrument include the size of ERC grants, the 
duration  of  projects,  and  the  freedom  and  flexibility  to  pursue  basic  and  ground-breaking 

45 12.6% ‘to a moderate extent’, 12.6% ‘to a small extent’, 13% ‘not at all’, 42.3% ‘do not 
know/not applicable’. 
46 Accessed on 31 August 2023. 
47 https://www.sbfi.admin.ch/sbfi/en/home/research-and-innovation/international-cooperation-r-
and-i/eu-framework-programmes-for-research/horizon-europe.html. 

56 

 
 
 
research. These factors were raised during interviews with ERC beneficiaries, as well as by 
several position papers submitted to the open consultation48.  

Consolidating  the  structuring  effect  of  the  European  Research  Area  on  national  research 
systems in terms of changes to national and institutional policies and practices 

As found in the Evaluation study on Excellent Science under Horizon 2020, the ERC has had 
a  structuring  effect  by  increasing  the  visibility  and  reputation  of  the  host  institutions. 
Interviewees49  consider  that  the  prestige  of  ERC  grants  increased  the  attractiveness  and 
credibility of the host institutions. For both grantees and  host institutions, an ERC grant is 
considered  a  ‘standardised’  label  of  excellence  by  the  research  community.  This  effect  is 
even more important for less well-known universities and institutions. It also contributes to 
the observation by interviewees that host institutions seek to attract ERC grantees to their 
institution.  

In general, non-academic sectors play only a limited role in the ERC, and synergies between 
academia and industry are limited. For instance, data from the Horizon Dashboard shows 
that for both Horizon 2020 and Horizon Europe, private for-profit entities represent only 2% 
of  total  participation.  Although  there  are  exceptions,  for  many  projects  (with  potential  for 
commercialisation), connecting industry remains a difficult task50.  

1.3.2.4. 

Transforming knowledge into commercial and social innovations, 
generating economic growth and tackling societal challenges 

Development of intellectual property and the commercialisation of research results 

As yet, it is too early to assess IPR and the commercialisation of the results of ERC grants 
under  Horizon  Europe.  However,  in  relation  to  Horizon  2020,  data  from  the  Horizon 
Dashboard51 reports a total of 759 IPR applications among 442 ERC projects, of which 87% 
are patents. This figure corresponds to 16% of all IPR applications reported for Horizon 2020 
so far. The share compares to 38% of the total number of peer-reviewed publications, when 
one considers the ERC’s contribution to scientific production under Horizon 2020.  

These  figures  are  consistent  with  the  attitudes,  experiences  and  behaviour  towards 
commercialisation  reported  by  ERC  grantees:  the  beneficiaries’  survey  indicates  that  only 
29% – 59 out of the 241 ERC PIs who responded to the question, considered that their ERC 
project  responds  to  a  ‘very  large’  or  ‘large’  extent  to  their  needs  in  terms  of  the 
commercialisation of research results. In terms of the skills gained, as reported previously, 
only 18% of respondents (43 out of 234) indicated that they had (to a ‘very large’ or ‘large’ 
extent)  gained  skills  related  to  the  commercialisation  of  research  results  as  result  of 
participating in the ERC project, while 33% (78 out of 233) did so in relation to skills in the 
management of IPR. 

48 The public online consultation on the past, present, and future of the European Research and 
Innovation Framework programmes 2014-2027 invited stakeholders and individuals to share 
their views on the performance of the Framework Programmes. Available at: https://research-
and-innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 
49 In the context of the case study on the ‘ERC impact on creating new or pushing existing 
frontiers of science’. 
50 See the Case Study on the ‘ERC impact on creating new or pushing existing frontiers of 
science’. 
51 Accessed 1 September 2023 

57 

 
 
 
Nonetheless,  a  significant  number  of  beneficiaries  in  the  survey  (22%;  53  out  of  238) 
expected the ERC project to have/or to be likely to result in a patent, and 22% (51 out of 231) 
anticipated patent activities as part of the activities to exploit the results of the ERC project; 
15% (34 out of 231) anticipated the establishment of spin-off or start-up companies. The ERC 
website52 indicates that 400 start-ups have been identified as founded or co-founded by ERC 
PIs.  These  figures  thus  illustrate  the  potential  for  the  bottom-up  fundamental  research 
supported through the ERC to lead to IPR and commercial solutions.  

Contribution to societal challenges 

An analysis of Horizon 2020 ERC projects53 shows they have made a significant contribution 
to the European Green Deal, Europe fit for the digital age, and EU4Health. 

•  European Green Deal54: a total of 953 projects, worth more than EUR 2 billion, have been 
identified,  covering  several  fields  including  energy,  biodiversity,  conservation,  ecology 
and transport.  

•  Europe fit for the digital age55: a total of 757 projects, worth almost EUR 1.5 billion, have 
been  identified,  covering  several  fields  including  quantum  information,  AI,  biomedical 
engineering and electronics.  

•  EU4Health56:  a  total  of  2,281  projects,  worth  more  than  EUR  4.5  billion,  have  been 
including  cancer,  microbiology,  neuroscience, 

fields 

identified,  covering  several 
immunology and ageing.  

The ERC has also contributed strongly to tackling the COVID-19 crisis. An analysis carried 
out by the ERC57 identified a total of 183 projects in multiple fields such as diagnostics and 
treatments  (including  vaccines),  medical  devices,  digital  tools,  AI,  immunity,  infection  and 
pathology,  and  social  and  economic  behaviour,  wellbeing  and  crisis  management.  For 
example, EU research funding programmes provided significant support to the early stages 
of  the  mRNA  vaccine  research  of  Professor  Ugur  Sahin58,  which  was  crucial  in  the  fight 
against the COVID-19 pandemic. 

According to the SDG analysis performed for this study, the SDGs to which ERC projects 
contribute most under Horizon Europe are SDG 3 – Good health and well-being (EUR 1,692.9 

52 https://erc.europa.eu/about-erc/facts-and-figures  
53 European Research Council Mapping ERC Frontier Research Horizon 2020 Research and 
Innovation Framework Programme (2014-2020), 2022, available at: 
https://op.europa.eu/en/publication-detail/-/publication/ab16bb09-ed55-11ed-a05c-
01aa75ed71a1/language-en/format-PDF/source-292255650. 
54 European Research Council, ERC frontier research contribution to the European Green Deal, 
2022, available at: https://erc.europa.eu/sites/default/files/2022-
08/H2020_factsheet_EuropeanGreenDeal.pdf. 
55 European Research Council, frontier research contribution to a Europe fit for the digital age, 
2022, available at: https://erc.europa.eu/sites/default/files/2022-
08/H2020_factsheet_FitforDigitalAge.pdf. 
56 European Research Council, ERC frontier research contribution to EU4Health, 2022, available 
at: https://erc.europa.eu/sites/default/files/2022-08/H2020_factsheet_EU4Health.pdf. 
57 European Research Council,  COVID-19 Frontier research in the spotlight, 2022, available at: 
https://erc.europa.eu/sites/default/files/2022-08/COVID19-Frontier_research_in_the-
spotlight.pdf. 
58 https://erc.europa.eu/news-events/magazine/tackling-covid-19-%E2%80%93-role-european-
research. 

58 

 
 
 
million, or 41.2% of the total EU contribution); SDG 9 – Industry and innovation (EUR 516.7 
million, or 12.6%); and SDG 7 – Affordable and clean energy (EUR 437.4 million, or 10.6%). 
These  SDGs  reflect  the  growing  importance  of  certain  challenges  over  recent  years  with 
respect to health (the COVID-19 pandemic, but also the importance of cancer research, for 
example) and clean energy (related to climate change and the energy crisis). Compared with 
Horizon 2020 (in terms of the total number of projects), SDG7 appears to be more  widely 
covered  by  the  ERC  under  Horizon  Europe.  However,  the  documents  on  which  the  SDG 
analysis  is  based  differ  between  Horizon  2020  and  Horizon  Europe:  the  Horizon  Europe 
analysis was carried out on the basis of the proposal texts, while in Horizon 2020 the analysis 
was based on publications. This makes it difficult to compare the contribution made to SDGs 
by the two Framework Programmes. For a more in-depth analysis, please refer to Annex 6.  

Interviews with ERC beneficiaries also provide some reflections on the societal impacts of 
their ERC projects, or of the ERC as a whole. Results were mixed, with some projects having 
a  strong  societal  impact  and  others  –  although  contributing  to  the  knowledge  base  on 
societally relevant topics – having little or no direct societal impact. This also depends on the 
field  and  the  concrete  objectives  of  the  research  in  question.  In  some  cases,  PoC  grants 
proved helpful in facilitating the exploration of the commercial and social innovation potential 
of ERC-funded research. In general, these tend to be used for projects coming from the fields 
of  Life  Sciences,  Physical  Sciences  and  Engineering.  For  example,  under  the  2023  PoC 
call59, 58% of the funded projects were linked to the Physical Sciences or Engineering, 33% 
to Life Sciences, and 9% to SSH. Some interviewees mentioned that further support would 
also  be  helpful  in  transferring  the  results  of  their  research  on  towards  a  broader  public  or 
directly towards policy-makers.  

1.3.3.  Coherence 

1.3.3.1. 

Internal coherence  

The recipients of ERC grants are evaluated and selected on the basis of the sole criterion of 
scientific excellence. The ERC’s solely bottom-up, excellence-driven approach distinguishes 
it  from  other  programme  parts  of  the  Framework  Programme.  Interviews  with  ERC 
beneficiaries,  as  well  as  survey  respondents  and  position  papers  submitted  to  the  OPC60 
clearly highlight the importance of having a bottom-up funding instrument that is able to fund 
frontier research without a prescriptive logic. The distinctive features of the ERC thus respond 
to the needs of its beneficiaries, and for this reason it complements the FP’s other funding 
instruments. 

Because of these distinctive features, the ERC is also less likely to display strong synergies 
with other programme parts. In fact, in the Targeted Stakeholder Consultation61, the majority 
of respondents (56% or 500 out of 895) reported that they did not know whether they could 
see  any  unexploited  potential  for  complementarities  between  the  six  clusters  and  ERC. 

59 https://erc.europa.eu/news-events/news/erc-funds-66-grantees-unlock-commercial-and-
societal-potential-their-research.  
60 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 
61 The public online consultation on the past, present, and future of the European Research and 
Innovation Framework programmes 2014-2027 invited stakeholders and individuals to share 
their views on the performance of the Framework Programmes. Available at: https://research-
and-innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 

59 

 
 
 
However,  a  smaller  percentage  (24%;  215  out  of  895)  saw  unexploited  potential  for 
complementarities with Cluster 1 (Health). The survey results (ERC-BEN-022) also allow us 
to  examine  whether  ERC  projects  plan  to  implement  any  activities  in  collaboration  with 
projects funded under the other Horizon Europe programme parts. These results indicate that 
most respondents did not currently have any joint activities with other projects under Horizon 
Europe (71.5% or 178 out of 250); 10% (25 out of 250) planned to have joint activities with 
another ERC project; and 12.4% (31 out of 250) did not know, or indicated that the question 
was not applicable to their situation. In addition, according to the network analysis carried out 
under this study (see Annex 6), and the degree to which researchers participate in different 
programmes  (the  connectedness  of  the  programme  parts),  the  ERC is  one  of  programme 
parts with the lowest number of connections.  

A  specific  analysis  of  the  trajectory  of  ERC  researchers  from  Horizon  2020  into  Horizon 
Europe62 shows that around 8% of PIs from ERC projects under Horizon 2020 went on to 
participate in a Horizon Europe project. They have most frequently participated in Pillar 2 of 
Horizon Europe (52% of participations in Horizon Europe are by PIs that participated in the 
ERC under Horizon 2020), followed by Pillar 1 (24%), Pillar 3 (18%) and WIDERA (6%). In 
Pillar 1, the most frequent destination programme part is the MSCA (which constitutes the 
destination  for  15%  of  all  Horizon  2020  ERC  PI  participations  under  Horizon  Europe), 
followed by INFRA (5%). In Pillar 2, the most frequent destination programme part is Cluster 
4  ‘Digital,  Industry  and  space’  (20%  of  all  Horizon  2020  ERC  PI  participations  in  Horizon 
Europe), followed by Cluster 1 ‘Health’ (10%). In Pillar 3, almost all PIs went to the EIC, which 
accounts for 18% of all participations in Horizon Europe by Horizon 2020 ERC PIs. A more 
detailed  analysis  of  the  trajectory  of  ERC  PIs  is  available  in  Case  Study  9  on  Knowledge 
Creation; here, a distinction is also made by type of ERC grant.  

Over recent years, the ERC has deepened its collaboration with the EIC, with the objective 
of  exploiting  potential  synergies  and  complementarities  between  the  two  organisations. 
Examples of such collaborations include the joint workshop on Energy Storage organised in 
November  202263  and  the  joint  workshop  on  Cell  and  Gene  therapy  in  June  202164.  The 
potential  for  further  collaboration  is  great,  as  demonstrated  by  the  fact  that  25  out  of  42 
recipients  of  the  European  Innovation  Council’s  new  ‘Transition’  funding  originated  from 
research  funded  by  the  ERC65.  Another  initiative  includes  the  Next  Generation  Innovation 
Talent Schemes66, open to PhD candidates and/or postdoctoral researchers participating in 
projects funded by the ERC. 

In addition, an analysis67 performed by the study team and further developed in Annex 6, has 
sought to identify topical similarities in the research carried out under Horizon 2020 across 

62 Specifically, the analysis first identified the PIs of Horizon 2020 ERC projects, and then 
identified in which projects they have participated under Horizon Europe. In the paragraphs that 
follow, we discuss the results of this analysis. 
63 European Research Council (2023). Annual Report on the ERC activities and achievements in 
2022, available at: https://op.europa.eu/s/yqiQ.  
64 European Research Council (2022). Annual Report on the ERC activities and achievements in 
2021, available at: https://op.europa.eu/en/publication-detail/-/publication/d4a62f15-b947-11ec-
b6f4-01aa75ed71a1/language-en/format-PDF/source-283364380.  
65 https://erc.europa.eu/news/erc-funded-research-wins-most-new-eu-innovation-grants. 
66 https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/topic-
details/horizon-eic-2023-talents-01-01 
67 The research included a refinement of the topical analysis of the Horizon 2020 Programmes 
and Social Challenges (SCs) to calculate the overlap, in terms of number of overlapping 
programmes and SCs, for each of the top 200 topics. The study team aimed to pinpoint those 
topics to which EC-sponsored outputs have been distributed across multiple programmes and 
SCs. 

60 

 
 
 
the  several  programme  parts,  by  looking  at  the  scientific  publications  arising  from  the 
projects. This analysis is explained in further detail in Case Study 9 on Knowledge Creation. 
In  summary,  the  results  show  that  a  given  topic  (among  the  top  200  topics68)  on  average 
occurred in 6.3 different programmes/challenges, with a minimum of one and a maximum of 
21. Given the bottom-up character and breadth of possible research topics covered in the 
ERC and MSCA, it is not surprising to observe that the topics covered in these programme 
parts overlap, as well as being addressed in other, more specifically oriented, programme 
parts (INFRA, LEIT-ICT, FET, SC1).  

1.3.3.2. 

External coherence 

Proposals for ERC Proof of Concept grants that receive a pass mark in all three evaluation 
steps but cannot be funded due to a lack of budget available to the call are eligible for the 
Seal  of  Excellence69.  In  addition,  opportunities  also  exist  for  career  development  through 
public funding linked to ERC projects, even without receiving an ERC Grant (such as jobs in 
ERC  teams,  Visiting  Research  Fellowships,  International  Arrangement  funding  and 
mentoring). 

The survey results (ERC-BEN-021) also indicate that most respondents (74.1%, or 186 out 
of 250) did not apply for any additional funding for the research idea/activities addressed in 
their ERC project, while one-fifth of respondents (47 out of 250) applied for national/regional 
R&D  public  funding  schemes  programmes.  The  interviews  with  PIs  confirm  that  in  some 
cases they have applied for national/regional funding to complement their research activities 
under the ERC grant – for instance, to explore a new research path that became apparent 
from the research but did not fit within the scope of the ERC project.  

To  complement  these  findings,  an  analysis  of  administrative  data  based  on  the  semantic 
textual similarity method provides certain insights into complementarities between Horizon 
Europe and other EU funding programmes, particularly in terms of other EU funding sources 
that have supported those applications that were unsuccessful under Horizon Europe funding 
calls. These alternative EU funding programmes include, for example, CREA2027, DIGITAL, 
EMFAF  and  ERASMUS2027.  The  results  of  this  analysis  show  that  only  11  unsuccessful 
applications to Horizon Europe ERC (out of 13,726) received alternative funding from other 
EU  funding  programmes  (amounting  to  around  EUR  22.6  million).  Despite  this  very  small 
number of projects, an interesting observation to note is that the amount of alternative funding 
received  from  other  EU  funding  programmes  by  the  unsuccessful  ERC  applicants  was 
actually higher than the amount requested in the original ERC applications. Specifically, the 
amount  of  EU  funds  received  in  alternative  funding  (EUR  22.6  million)  was  around  26% 
higher than the total amount of ERC grants requested (EUR 17.9 million).  

One additional aspect of external coherence concerns the strength of the links between the 
ERC  and  key  EU strategic  priorities  in  terms  of  research  excellence.  The  ERC  has  taken 
steps  in  this  regard  –  for  example,  by  taking  part  in  the  annual  European  Research  and 

68 The concept of ‘prominent topics’ refers to important topics in terms of momentum and 
visibility. To calculate a topic’s prominence, the study team has combined three metrics into an 
average to constitute its prominence score: citation counts, Scopus view counts and average 
CiteScore. Based on the set of publications from Horizon 2020, the 200 topics with the highest 
prominence scores were identified by the study team. The full set of Top 200 topics can be 
retrieved online from Annex 6. 
69 https://research-and-innovation.ec.europa.eu/funding/funding-opportunities/seal-
excellence_en.  

61 

 
 
 
Innovation  Day  in  September  202270,  bringing  together  policy-makers,  researchers, 
entrepreneurs and citizens to debate and shape the future of research and innovation. The 
panel  discussed  the  importance  of  science  communication  and  public  engagement, 
particularly  among  young  citizens.  The  ERC  also  provides  additional  funding  for  ERC 
grantees  within  the  scope  of  the  Public  Engagement  with  Research  Award71,  with  the 
objective of recognising and rewarding outstanding science communicators among PIs who 
successfully engage audiences outside their domain with ERC-funded research. Lastly, the 
ERC has been able to contribute to the green and digital transitions, one of the ERA’s priority 
areas. Such contributions include projects in the fields of energy and mobility72.  

1.3.4.  EU added value of the ERC 

1.3.4.1. 

Added value compared with national/regional levels 

Available survey evidence (ERC-APPL-026) points to a small majority of unsuccessful ERC 
applicants  (53.7%,  or  676  out  of  1,258  respondents)  applying  for  alternative  sources  of 
funding  (e.g.  national/regional  funds,  grants  from  international  organisations,  etc.)  to 
implement the activities proposed in their rejected ERC application. Of those who applied for 
alternative funding (ERC-APPL-027), around half (49% or 332 out of 676) were successful, 
and a large majority of those who were successful (89%, or 294 out of 332) received their 
funding from national or regional R&D public funding schemes (ERC-APPL-029). This shows 
that there are, to some extent, funding opportunities at national or regional level that can fund 
research activities or topics similar to those funded by the ERC.  

However,  these  alternative  sources  of  funding  do  not  match  the  benefits  provided  by  the 
ERC.  For  example,  when  applying  to  alternative  sources  of  funding,  the  majority  of  the 
respondents  (84%  or  575  out  of  675)  had  to  implement  (in  some  cases,  significant) 
adaptations  of  their  project,  in  terms  of  its  scope,  research  conditions  or  project  duration 
(ERC-APPL-030). Among those who made adaptations to the applications:  

•  91.8% applied with a smaller project scope in terms of the number of research (sub)areas 

covered (501 out of 546) (ERC-APPL-031).  

•  86.8% applied with a less complex and diverse set of research methods (447 out of 515) 

(ERC-APPL-032).  

•  88.6% expected to produce a smaller amount of research outputs and innovations (465 

out of 525) (ERC-APPL-033).  

•  91.3% applied for a shorter duration of funding (464 out of 508) (ERC-APPL-034).  

•  84.6% received less favourable research conditions, such as lower salaries (423 out of 

500) (ERC-APPL-035).  

70 European Research Council (2022). Annual Report on the ERC activities and achievements in 
2022, available at: https://op.europa.eu/s/yqiQ. 
71 ERC Work Programme 2024, available at: https://ec.europa.eu/info/funding-
tenders/opportunities/docs/2021-2027/horizon/wp-call/2023/wp_horizon-erc-2023_en.pdf  
72 European Research Council, ERC Frontier Research contribution to the European Green deal, 
2022, available at: https://erc.europa.eu/sites/default/files/2022-
08/H2020_factsheet_EuropeanGreenDeal.pdf 

62 

 
 
 
These  findings  confirm  those  elements  of  ERC  funding  that  have  been  identified  as  key 
success factors contributing to the success of the ERC in funding frontier research and high-
risk/high-gain  research  projects.  Evidence  from  the  ERC  Case  Studies73,  backed  by 
qualitative  information  extracted  from  interviewees  and  position  papers  submitted  to  the 
OPC74 points to the opportunities offered by ERC funding, in comparison to other sources of 
funding, to build larger, more diverse and multidisciplinary teams and to carry out projects of 
longer duration, which helps to develop more novel research outcomes. Some interviewees 
highlighted  the  fact  that  they  would  not  have  been  able  to  fund  their  projects  via  national 
funding schemes, especially projects that are more complex or high-risk, and that getting an 
ERC grant sometimes allows researchers to better focus on research (rather than teaching 
time  or  other  activities),  which  is  not  possible  through  most  national  funding  schemes.  As 
mentioned in the ERC Case Studies, the ERC is considered the main funding instrument in 
Europe for funding bottom-up, fundamental research – and its grants are considered by the 
research community to be both more substantial and more prestigious than national funding 
opportunities. 

In  addition,  among  those  researchers  who  did  not  apply  for  additional  funding,  and  were 
therefore  unable  to  carry  out  their  projects  without  Horizon  Europe  funding  (ERC-APPL-
0316),  the  vast  majority  (81.4%,  or  447  out  of  549)  considered  the  amount  of  funding 
available from other sources to be insufficient (a ‘very important’ or ‘important’ factor) to carry 
out the project, while 78.8% (438 out of 556) said that no alternative funding was available 
for that research topic/area. This shows that, in terms of funding allocation, the ERC is able 
to fund topics/areas and research activities that are otherwise not being funded at national 
level  –  or  at  least,  not  on  the  same  scale.  This  finding  is  confirmed  in  the  survey  of  ERC 
beneficiaries (ERC-BEN-044), in which 93% of respondents (215 out of 232) strongly agreed 
or  rather  agreed  that  in  comparison  to  the  research  funding  available  to  them  at  national 
and/or regional level, the amount of funding available under Horizon Europe is higher, while 
69%  (158  out  of  230)  strongly  agreed  or  rather  agreed  that  the  ERC  provides  funding  for 
research topic(s) or field(s) not covered in national or regional R&I funding programmes. Also, 
53% strongly agreed or rather agreed that Horizon Europe funding helps to better address 
grand societal challenges compared with funding at national or regional levels (123 out of 
230), and 60% thought that it offers more flexibility to respond to technological developments 
(136  out  of  228).  The  ERC  thus  provides  important  added  value  in  relation  to  bottom-up 
research of a certain scale and complexity, as well as for specific topics that are not funded 
or less well funded at national or regional level. 

Also,  with  regard  to  research  modalities,  ERC  beneficiaries  are  positive  about  the 
opportunities provided through ERC funding compared with national or regional level funding. 
A majority of ERC PIs who responded strongly agreed or rather agreed that, compared with 
the funding available at national or regional level, Horizon Europe funding provides greater 
opportunities for international mobility for researchers (84%, or 196 out of 233); helps to better 
pool the skills and expertise necessary for research (75%, or 171 out of 229); and provides 
more access to research infrastructures, equipment and/or databases (56%, or 128 out of 
228).  

73 Analysis of Horizon 2020 also shows that ERC-funded research has contributed to the EU 
priorities of Europe fit for digital age and European Green Deal in various fields, among which 
some also remain at the forefront under Horizon Europe (e.g. quantum technologies and climate 
change).  
74 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 

63 

 
 
 
At the same time, a large majority (89%, or 207 out of 233) of the ERC PIs who responded 
strongly agreed or rather agreed that there is a higher level of competition for Horizon Europe 
funding compared with national or regional funding. Interviews also strongly confirmed this 
perspective. By adopting a peer-review evaluation approach based solely on the criterion of 
excellence, the ERC focuses on allocating funding to the best scientists and research teams 
across Europe. The highly prestigious awards given to ERC grant holders, such as Nobel 
Prizes, are an illustration of this.  

Provides higher amount of funding (compared to
the national/regional schemes)
Involves higher level of competition for research
funding

Provides more international mobility
opportunities for researchers

Helps to better pool the skills and expertise
necessary for research

Is more likely to make me pursue my career in
Europe (as opposed to seeking opportunities…

Provides funding for  research topic(s) or field(s)
not covered in national or regional R&I funding…
Gives more flexibility to respond to technological
developments

Provides more access to research
infrastructures, equipment and/or databases
Helps to better address grand societal
challenges (e.g., climate change,…
Gives more flexibility to respond to changing
socio-economic needs

Gives more attention for the human aspect to
innovation (safety, ethics, user acceptance etc.)

75 %

17 %

63 %

59 %

26 %

25 %

37 %

38 %

54 %

18 %

43 %

26 %

33 %

26 %

26 %

30 %

28 %

25 %

28 %

20 %

16 %

23 %

14 %

20 %

Provides more opportunities for
commercialization of research results   (e.g.,…
Gives more attention for the environmental
sustainability aspect to innovation

13 %

18 %

Helps to achieve economies of scale in research
(achieving the same result at lower cost)
There are no additional benefits compared to
national / regional support

10 %

16 %

4 %

6 %

Strongly agree

Rather agree

Neither agree nor disagree

Neither agree nor disagree

Strongly disagree

Do not know/Not Applicable

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%

Figure 6. Comparison between Horizon Europe funding and national and/or regional funding. Note: based on the survey 
question ‘Would you agree or disagree that, compared to the research funding available to you at national and/or regional 
level, Horizon Europe…’ (share of ERC PI respondents). Source: survey of Horizon Europe beneficiaries, conducted in May-
July 2023. 

1.3.4.2. 

Cross-cutting and transversal issues 

Given its nature, the ERC contributes to cross-cutting issues through the funding of bottom-
up  research  on  topics  on  the  one  hand,  and  through  the  development  of  individual 
researchers’ skills on the other. The first point can be illustrated using data from the Horizon 
Dashboard (accessed 10 September 2023). Based on the EuroSciVoc classification, these 

64 

 
 
 
data show two signed ERC grants75 on the topic of gender studies (versus 22 signed grants 
under Horizon 2020), and five signed ERC grants76 on the topic of ethics principles (versus 
seven signed grants under Horizon 2020). However, when consulting the ERC ERIS tool77, 
the list of ERC projects addressing gender studies is significantly longer.  

With  respect  to  skills  development,  as  shown  above,  43%  (99  out  of  233)  of  survey 
respondents  stated  that  the  ERC  had  contributed  to  a  ‘very  large’  or  ‘large’  extent  to 
developing  their  skills  in  research  ethics  and  integrity.  Around  half  (115  out  237)  of  ERC 
beneficiaries (ERC-BEN-042) reported no issues and a positive experience with the ethics 
self-assessment of their ERC project, and 21% (50 out of 237) reported only minor issues. 
Furthermore,  62%  (115  out  of  187)  of  ERC  beneficiaries  (ERC-BEN-043)  considered  the 
support (e.g. information, guidance) available for carrying out the ethics self-assessment of 
their ERC project to be ‘excellent’ or ‘good’.  

With  respect  to  co-design  and  co-creation,  only  19.3%  (45  out  of  233)  of  the  ERC 
beneficiaries that responded to the survey indicated that they plan to engage citizens and 
citizen representatives or end users in the co-creation of R&I content in their project (ERC-
BEN-038). 

1.3.5.  Relevance 

1.3.5.1. 

Delivering Excellent Science  

Bottom-up funding of Excellent Science through the ERC 

Under Horizon Europe, the ERC continues to operate on the basis of a ‘bottom-up’ approach 
without predetermined thematic priorities, with the objective of providing attractive, long-term 
funding  to  support  excellent  investigators  and  their  research  teams  to  pursue  ground-
breaking,  high-risk/high-gain  research.  Evidence  compiled  from  interviews  with  ERC 
beneficiaries and from the position papers submitted to the OPC78 shows that this approach 
is highly appreciated by beneficiaries. For example, several position papers emphasise the 
importance of researchers being able to formulate their own paths of investigation and the 
‘blue-sky’ nature of ERC calls, which gives greater scope to push the boundaries of research 
disciplines – a prelude for delivering Excellent Science and frontier research. This approach 
is also very attractive to top researchers, and hence contributes to achieving the objectives 
of the ERC.  

In  addition,  96%  (231  out  of  241  respondents)  of  the  successful  ERC  applicants  who 
responded to the beneficiaries’ survey confirmed that the ERC grant had, to a ‘very large’ or 
‘large’ extent, given them the freedom to pursue their specific research agenda, while 95% 
(229 out of 241) said that it gave them opportunities to conduct basic/fundamental research 
(ERC-BEN-025). In this regard, interviewees voiced appreciation of the fact that no detailed 
ex-ante  definition  of  expected  (societal  and  economic)  impacts  is  expected  in  an  ERC 
application, as this is very difficult to predict in detail for curiosity-driven research. This does 
not  mean  that  ERC  projects  do  not  realise  such  effects,  but  instead  it  allows  the  project 

75 The two projects are: BILQIS and F-WORD.  
76 The five projects are: MORE, NOTCOM, REASONF1RST, Abortion Figurations and 
PatentsInHumans.  
77 https://erc.europa.eu/projects-statistics/advanced-analytics-tool-eris 
78 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 

65 

 
 
 
sufficient flexibility to learn from failure, and to redirect research within the scope of the project 
along more promising pathways. 

Another element  in  the  design  of  the  ERC’s  approach  that  supports  the  excellence  of  the 
research  funded  is  the  way  in  which  the  selection  process  is  organised,  with  evaluation 
panels consisting of excellent researchers and the sole criterion of scientific excellence being 
applied.  Interviewees  mention  that  this  approach  has  supported  the  ERC’s  reputation, 
contributing to the fact that an ERC grant is considered a label of high quality. Over time, the 
ERC Scientific Council has made changes to the structure of panels under the ERC, with the 
aim of ensuring balance between panels in terms of the number of proposals assessed in 
each call; disciplinary coherence within and across panels; and the place of interdisciplinary 
research within the panel structure.  

Participation of non-EU countries 

According  to  data  from  the  Horizon  Dashboard79,  participations  from  associated  countries 
and non-associated third countries remained relatively stable throughout Horizon 2020 (the 
period considered being 2015-2021), ranging from 116 signed grants in 2021 to 225 signed 
grants in 2016. Under Horizon Europe, performance has not changed significantly, with 183 
signed grants in 2022 and 125 signed grants so far in 2023, at the time of writing. However, 
it is important to consider two elements: the current non-association to Horizon Europe of 
Switzerland,  which  was  the  second-most  frequent  partner  under  Horizon  2020,  and  the 
inclusion  of  the  United  Kingdom  under  the  group  of  non-EU countries.  So  far, the  net  EU 
contribution to associated plus non-associated third countries corresponds to 12% (i.e., EUR 
471.6 million) of the total net EU contribution to the ERC. This share is slightly lower than the 
share observed during Horizon 2020 (15%).  

The top fields in terms of participation of associated plus non-associated third countries are 
genomics, scientific computing and data processing, and systems biology.  

Tackling relevant issues  

According to data from the Horizon Dashboard80, under Horizon Europe, the field of science 
with the greatest number of ERC signed grants is natural sciences, followed by medical and 
health sciences and engineering and technology. Specifically, the fields of proteins, oncology 
and climate change are those with highest numbers of signed grants. In terms of participation 
(assessed using a keyword methodology), genomics, physical chemistry, and quantum optics 
and quantum information are the fields with greatest numbers of participations.  

As mentioned previously, the ERC adopts a ‘bottom-up’ approach with a focus on excellence. 
However, despite not being prescriptive, the ERC has been able to fund research essential 
to  tackling  emerging  needs.  For  example,  at  the  height  of  the  COVID-19  crisis,  over  180 
ongoing or completed ERC projects generated knowledge and technology that contributed to 
the response to the pandemic, including diagnostics and treatments, medical devices, and 
immunity, infection and pathology81.  

79 Accessed 6 September 2023. 
80 Accessed 6 September 2023. 
81 European Research Council, COVID-19 Frontier research in the spotlight, 2020, available at: 
https://erc.europa.eu/sites/default/files/2022-08/COVID19-Frontier_research_in_the-
spotlight.pdf.  

66 

 
 
 
In addition, the ERC has contributed to the EU priorities of a Europe fit for the digital age82 and the 
European Green Deal (EGD) 83: 

• 

• 

In the digital field, under Horizon 2020, the ERC funded over 700 projects, worth more 
than  EUR 1 billion  in  the  fields  of  artificial  intelligence,  quantum  technologies  and  the 
digital age. As mentioned above, quantum technologies continues to be one of the top 
fields under Horizon Europe.  

In  the  case  of  the  EGD,  Horizon  2020  funded  over  950  projects,  worth  more  than 
EUR 2 billion  in  the  fields  of  clean,  affordable  and  secure  energy,  farm  to  fork,  and 
biodiversity. As mentioned above, genomics and climate change continue to be among 
the top fields under Horizon Europe. 

The ERC has also been quick to react to world challenges – for example, its initiative following 
the ERC Scientific Council’s statement in support of Ukraine and its research community84. 
With ‘ERC for Ukraine’, the ERC appealed to its grantees to provide temporary employment 
to researchers and support staff from Ukraine and seeking refuge in the EU85. 

Lastly, 53% (123 out of 230) of the ERC successful applicants who responded to the survey 
(ERC-BEN-044) strongly agreed or rather agreed that, compared with the research funding 
available  at  national  and/or  regional  level,  Horizon  Europe  helps  to  better  address  grand 
societal  challenges  (e.g.  climate  change,  demographics  and  aging,  etc.);  60% (136  out  of 
228)  strongly  agreed  or  rather  agreed  that  Horizon  Europe  provides  greater  flexibility  to 
respond to technological developments; and 48% (109 out of 228) strongly agreed or rather 
agreed  that  Horizon  Europe  provides  greater  flexibility  to  respond  to  changing  socio-
economic needs.  

1.3.6.  Conclusions and lessons learned 

Conclusion 1: under Horizon Europe, as under Horizon 2020, the ERC’s main mission is to 
‘encourage the highest quality research in Europe through competitive funding and to support 
investigator-driven  frontier  research  across  all  fields,  based  on  scientific  excellence’.  The 
fundamental  activity  of  the  ERC  is  ‘to  provide  attractive,  long-term  funding  to  support 
excellent investigators and their research teams to pursue ground-breaking, high-risk/high-
gain research’. Qualitative evidence collected for this study, based on interviews with ERC 
beneficiaries and on the position papers submitted to the OPC86, show strong appreciation 
for the continuity and stability of the ERC’s structure. In addition, in view of its longer-term 
effects, this evidence also indicates that the stability in the programme part has helped the 
ERC to offer transparency to researchers and build up its strong reputation. 

82 European Research Council, ERC Frontier Research contribution to a Europe fit for the digital 
age, 2022, available at: https://erc.europa.eu/sites/default/files/2022-
08/H2020_factsheet_FitforDigitalAge.pdf.  
83 European Research Council, ERC Frontier Research contribution to the European Green deal, 
2022, available at: https://erc.europa.eu/sites/default/files/2022-
08/H2020_factsheet_EuropeanGreenDeal.pdf. 
84 Available at: https://erc.europa.eu/news/statement-erc-scientific-council-support-ukraine-and-
its-research-community 
85 See: https://erc.europa.eu/apply-grant/erc-ukraine. 
86 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 

67 

 
 
 
Survey  data  and  interviews  with  ERC  beneficiaries  demonstrate  that  the  ERC  provides 
important EU added value under the current as well as previous Framework Programmes, 
indicating  that  no  other  European,  national  or  regional  funding  instrument  matches  the 
conditions offered by ERC grants. Among others, these benefits include the scale of funding, 
the  duration  of  projects,  and  the  freedom  given  to  researchers  to  pursue  their  ideas  and 
conduct  ground-breaking  research.  The  ERC  therefore  remains  an  indispensable  funding 
instrument  within  the  European  research  landscape.  Its  excellence-driven,  bottom-up 
approach, combined with the scale of grants awarded, are important conditions that make 
the  ERC  a  successful  instrument  for  supporting  excellent  and  emerging  research,  as  is 
evidenced  by  the  bibliometric  analysis  and  analysis  of  research  topics  performed  in  the 
context of this study87.   

The  ERC  has  been  effective  in  advancing  frontier  research  and  the  scientific  and 
technological  impacts  of  new  ideas  –  evidence  of  which  can  be  seen  in  its  outstanding 
contribution to scientific production and quality, as demonstrated by the bibliometric analysis 
and  the  two  case  studies.  Accordingly,  the  ERC has  also  been  effective in  contributing  to 
addressing societal challenges: in addition to its contributions in the context of the COVID-
19 pandemic, analysis for Horizon 2020 shows that ERC-funded research in various fields 
has also contributed to the EU priorities ‘Europe fit for the digital age’88 and the European 
Green  Deal89.  Several  of  these  areas  remain  at  the  forefront  under  Horizon  Europe  (e.g. 
quantum  technologies  and  climate  change).  This  is  confirmed  in  the  survey  of  ERC 
beneficiaries, with 53% of respondents agreeing that, compared with the research funding 
available  at  national  and/or  regional  level,  Horizon  Europe  better  helps  to  address  grand 
societal  challenges  (e.g.  climate  change,  demographics  and  ageing,  etc.)90  Conversely,  in 
terms of supporting multidisciplinary and novel ideas, a few interviews with ERC beneficiaries 
indicated  that  the  allocation  of  applications  to  specific  evaluation  panels  is  not  always 
straightforward in the case of multidisciplinary or truly novel areas of research.  

Lesson learned 1.1: these conclusions regarding the stability, effectiveness and EU added 
value of the programme part call for the maintaining of the current funding modalities and 
structure of the ERC, with its sole focus on excellence and curiosity-driven research. For this 
purpose,  it  is  recommended  to  keep  the  central  aspects  of  the  ERC’s  architecture 
unchanged.  

Lesson learned 1.2: as both multidisciplinarity and novelty are important contributing factors 
implementing 
in  producing  high-quality  research,  consideration  may  be  given 
multidisciplinary evaluation panels. In addition, guidance for both applicants and evaluators 
with  respect  to  the  evaluation  of  multidisciplinary  and  novel  ideas  could  be  further 
strengthened.  

to 

Conclusion 2: having a solely bottom-up and excellence-driven approach makes the ERC a 
distinctive  programme  part  within  the  Framework  Programme.  Interviews  with  ERC 

87 See Annex 6 for more information on these analyses, as well as Case Studies 1 and 2 on the 
contribution of the ERC to emerging areas in the Life Sciences, Physical Sciences and 
Engineering and in the Social Sciences and Humanities. 
88 European Research Council (2022). ERC Frontier Research contribution to a Europe fit for the 
digital age. Available at: https://erc.europa.eu/sites/default/files/2022-
08/H2020_factsheet_FitforDigitalAge.pdf. 
89 European Research Council (2022). ERC Frontier Research contribution to the European 
Green deal. Available at: https://erc.europa.eu/sites/default/files/2022-
08/H2020_factsheet_EuropeanGreenDeal.pdf. 
90 53% (123 out of 230) of the ERC successful applicants who replied to the survey rather or 
strongly agree with this statement. 

68 

 
 
 
beneficiaries,  position  papers  submitted  to  the  OPC91,  as  well  as  survey  respondents,  all 
clearly highlight the importance of having a bottom-up funding instrument that is able to fund 
frontier research without a prescriptive logic. Its distinctive features thus continue to answer 
the needs of the beneficiaries. For this reason, it continues to complement the various other 
funding  instruments  under  Horizon  Europe.  Due  to  its  distinctive  features,  the  ERC  is 
generally  considered  complementary  to  national  and  regional  funding  (as  confirmed  by 
survey  evidence92  and  interviews  with  ERC  beneficiaries).  For  the  same  reason  of  its 
distinctiveness, the ERC is at the same time less likely to show strong synergies with other 
programme parts of Horizon Europe. 

Currently, among ERC grants, only ERC PoC proposals can be funded through the Seal of 
Excellence.  Meanwhile,  unsuccessful  ERC  applicants  can  seek  alternative  sources  for 
funding  from  national  and  regional  sources:  a  small  majority  of  ERC  unsuccessful 
applicants93applied for alternative sources of funding to implement the activities proposed in 
their  rejected  ERC  application,  of  which  around  half94  were  successful  –  a  share  slightly 
above  that  of  unsuccessful  applicants  to  the  MSCA  Postdoctoral  Fellowships  and 
organisations95 Out of the rejected ERC applicants who succeeded in obtaining alternative 
funding, 89%196obtained their funding from national or regional R&D public funding schemes. 

In  terms  of  the  valorisation  of  research  results,  synergies  with  the  EIC  are  already  being 
promoted  (e.g.  through  joint  workshops97  and  the  Next  Generation  Innovation  Talent 
Schemes98).  These  have  proven  successful,  with  25  out  of  42  recipients  of  the  European 

91 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 
92 For instance, according to ERC successful applicants and compared to the research funding 
available to them at national and/or regional level, Horizon Europe: 

• 

• 

• 
• 

• 
• 
• 

provides more international mobility opportunities for researchers (84% or 196 out of 
233 ERC successful applicants strongly or rather agree); 
helps to better pool the skills and expertise necessary for research (75% or 171 out of 
233); 
provides a higher amount of funding (93% or 215 out of 232); 
provides funding for research topic(s) or field(s) not covered in national or regional R&I 
funding programmes (69% or 158 out of 230)  
helps to better address grand societal challenges (53% or 123 out of 230); 
gives more flexibility to respond to technological developments (60% or 136 out of 228); 
provides more access to research infrastructures, equipment and/or databases (56% or 
128 out of 228). 

93 53.7% or 676 out of 1 258 respondents. 
94 49% or 332 out of 676 respondents. 
95 MSCA PFs: 46.8% or 312 out of 666 respondents; MSCA organisations: 46.6% or 54 out of 
116 respondents. 
96 294 out of 332 respondents. 
97 E.g., the joint workshop on Energy Storage organised in November 2022 (European Research 
Council (2023). Annual Report on the ERC activities and achievements in 2022. Available at: 
https://op.europa.eu/s/yqiQ) and the joint workshop on Cell and Gene therapy in June 2021 
(European Research Council (2022). Annual Report on the ERC activities and achievements in 
2021. Available at: https://op.europa.eu/en/publication-detail/-/publication/d4a62f15-b947-11ec-
b6f4-01aa75ed71a1/language-en/format-PDF/source-283364380).  
98 https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/topic-
details/horizon-eic-2023-talents-01-01. 

69 

 
 
 
Innovation Council’s new ‘Transition’ funding originating from research funded by the ERC99. 
With  regard  to  valorisation  towards  a  broader  public  or  towards  policy-makers,  some 
interviewees mention that further support for transferring their research results towards these 
target groups would also be helpful. Evidence on this last point is limited in the context of this 
study, and would need further exploration to assess the importance and the specific needs 
experienced by beneficiaries in this regard.  

Lesson learned 2.1: to increase links with national and regional-level research funding, and 
at the same time mitigate the impact of the low success rates of applications under the ERC 
(see  above),  the  potential  to  allow  other  types  of  proposals  to  be  eligible  for  the  Seal  of 
Excellence could be further explored, or national funders could be otherwise encouraged to 
provide  alternative  funding  to  ERC  applicants’  projects  that  received  a  very  high  quality 
classification but could not be funded (as is already the case in certain Member States).   

Lesson learned 2.2: greater collaboration with the EIC and actions targeting innovation and 
exploitation  in  the  research  cycle  could  be  explored  to  further  facilitate  the  exploitation  of 
research  results  and  reinforce  the  impacts  of  research  on  the  European  economy  and 
society. 

Lesson learned 2.3:  further exploration  could  be given  to  indications from  this study  with 
regard  to  the  need  to  further  support  researchers  in  transferring  their  research  results 
onwards to a broader public or to policymakers. 

Conclusion 3:  referring to overarching Conclusion 2 on relevance (presented in the main 
report of this study), several stakeholders in the open consultation pointed to a gap in terms 
of collaborative bottom-up research under the Framework Programme. Although of limited 
total size, Synergy Grants already address this gap as, by design, they support collaborative 
bottom-up research. This is also supported by some position papers submitted to the OPC100 
and an interview conducted by the study team.  

Lesson learned 3: consideration could be given to further expanding the number of Synergy 
Grants in order to further contribute to the collaborative bottom-up research dimension of the 
Framework  Programme.  While  considering  this,  expectations  should  be  aligned  with  the 
limited total size of the instrument. 

Conclusion  4:  the  ERC  has  also  been  effective  in  enhancing  the  careers  of  European 
researchers, thanks to the prestige that its grants bring.  The results of the survey of  ERC 
beneficiaries101 show that, in the longer term, researchers experience positive effects on their 

99 Open to PhD candidates and/or postdoctoral researchers participating in projects funded by 
the European Research Council (ERC). Available at: https://erc.europa.eu/news/erc-funded-
research-wins-most-new-eu-innovation-grants. 
100 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 
101 A majority of respondents to the survey to Horizon Europe successful applicants expect that 
their participation in ERC will impact their career development in the following aspects:  
Strengthening their capacity to conduct frontier research (93% or 219 out of 236 respondents); 
Strengthening their reputation as a researcher (91% or 214 respondents); 
Establishing and/or consolidating their own research group (81% or 191 respondents); 
Obtaining a new fellowship/research grant (76% or 180 respondents); 
Becoming an established researcher (70% of 165 respondents); 

70 

 
 
 
 
career,  such  as  a  strengthening  of  their  capacity  to  conduct  frontier  research  and  of  their 
reputation  as  a  researcher.  Receiving  an  ERC  grant  also  helps  researchers  to  establish 
and/or consolidate their own research group; obtain a new fellowship or research grant; or to 
progress  towards  a  higher  level  of  responsibility  or  move  into  a  more  senior  position.  In 
addition, getting an ERC grant improves the visibility of their research and opportunities for 
networking. 

At the same time, the success rates of applications for ERC grants remain relatively low under 
Horizon Europe (as confirmed in evidence provided by the ERCEA). Several interviewees 
also mentioned this as a barrier (both ERC beneficiaries and academic stakeholders). Given 
the positive impacts of the ERC on researchers and on the production of excellent research, 
several stakeholders call for an increase in the overall budget so that the ERC would be able 
to  fund,  to  a  larger  extent,  all  the  proposals  that  receive  a  very  high  quality  classification. 
Despite the current low success rates, however, interviewees confirm that the ERC remains 
a highly attractive instrument to both European and non-European researchers, thanks to the 
positive effects mentioned above. 

Another  budget-related  barrier  referred  to  by  several  stakeholders  (including  ERC 
beneficiaries  interviewed  in  the  context  of  this  study)  concerns  the  fact  that  the  EU’s  net 
contribution per project has not increased in line with inflation, thus presenting challenges for 
beneficiaries  who  face  increased  costs  and  salaries.  According  to  data  from  the  Horizon 
Dashboard,  under  both  Horizon  2020  and  (so  far)  Horizon  Europe,  the  average  net  EU 
contribution per project was EUR 1.72 million, while the average total cost per project was 
EUR  1.73  million.  Under  FP7,  both  indicators  stood  at  EUR  1.7  million.  Beneficiaries  and 
stakeholders thus call for action to be taken in order able to maintain the scope and size of 
ERC  grants,  which  is  mentioned  among  the  success  factors  of  the  programme  part  (see 
above).  

Lesson learned 4.1: an increase the overall budget of the ERC could be considered, with 
the  objective  of  funding  a  larger  share  of  the  proposals  that  receive  a  high  quality 
classification. However, consideration should be given to the potential impact on the number 
of applications – and consequently on the success rate – of an increase in the overall budget.  

Lesson  learned  4.2:  in  parallel  with  Lesson  learned  4.1,  it  is  recommended  that 
consideration should be given to an increase in the net EU contribution per project, with the 
objective  of  maintaining  the  scope  and  size  of  ERC  grants,  and  thus  the  positive  effects 
related to this. 

1.4.  Marie Skłodowska-Curie Actions (MSCA) 

1.4.1.  The MSCA under Horizon Europe 

Since  their  launch  in  1996,  the  Marie  Skłodowska-Curie  Actions  (MSCA)  have  become  a 
flagship  programme  for  researcher  mobility  and  training.  The  MSCA  provide  grants  for  all 
stages  of  researchers’  careers,  be  they  doctoral  candidates  or  highly  experienced 
researchers,  as  well  as  for  institutions  to  set  up  doctoral  programmes  and  collaborative 
projects. The MSCA foster international and intersectoral mobility of researchers, providing 
excellent and innovative research training and attractive career development and knowledge 

Progressing towards a higher level of responsibility/moving into a more senior position (e.g., full 
professorship) (50% or 118 respondents). 

71 

 
 
 
exchange opportunities. The MSCA prepare researchers to solve current and future societal 
challenges.   

1.4.1.1. 

Aims and objectives  

The  MSCA  is  particularly  aligned  with  the  Horizon  Europe  specific  objective  to  develop, 
promote  and  advance  scientific  excellence,  to  support  the  creation  and  diffusion  of  high-
quality  new  fundamental  and  applied  knowledge,  skills,  technologies  and  solutions,  to 
support training and mobility of researchers, to attract talent at all levels and contribute to the 
full  engagement  of  the  EU’s  talent  pool  in  actions  supported  under  the  Programme.  The 
regulation establishing Horizon Europe also identified the MSCA as the action contributing to 
the goals of the ‘Jobs, growth and investment’ agenda as well as the EU Global Strategy and 
to the Sustainable Development Goals102.   

Under Horizon Europe, the MSCA have an indicative budget of EUR 6.6 billion for the period 
2021-2027. The MSCA aim to fund 65 000 fellows, of which 25 000 are doctoral students103. 

Under Horizon Europe, the MSCA contribute to five main intervention areas104:   

•  Nurturing Excellence through the mobility of researchers across borders, sectors, and 

disciplines.   

•  Fostering new skills through the excellent training of researchers.  

•  Strengthening  human  capital  and  skills  development  across  the  European  Research 

Area.  

• 

Improving and facilitating synergies between R&I systems and programmes at regional, 
national, or Union level.  

•  Promoting public outreach to raise awareness of the activities supported by the MSCA 

and public recognition of researchers.  

While  the  MSCA  contribute  to  excellent  research  by  equipping  researchers  with  new 
knowledge and skills, they are also designed to have a structuring impact on higher education 
institutions and beyond academia. The actions spread excellence and set standards for high-
quality research education training in line with the European Charter for Researchers and the 
Code of Conduct for the recruitment of researchers, which promote open, merit-based and 
transparent recruitment and attractive working and employment conditions. The MSCA also 
pay  particular  attention  to  equal  opportunities  and  inclusiveness  and  encourage  funded 
projects to facilitate access by researchers at risk. 

The  MSCA  endorse  Open  Science  and  Responsible  Research  and  Innovation  through 
engaging  society  at  large,  integrating  gender  and  ethical  dimensions,  promoting  Open 

102 Council Decision (EU) 2021/764 of 10 May 2021 establishing the Specific Programme 
implementing Horizon Europe – the Framework Programme for Research and Innovation, and 
repealing Decision 2013/743/EU (Text with EEA relevance) 
103 European Commission (2021). Marie Skłodowska-Curie Actions 2021-2027. Developing 
talents, advancing research.  
104 Horizon Europe Work Programme 2023-2024. 2. Marie Skłodowska-Curie Actions. European 
Commission Decision C(2022)7550 of 6 December 2022. 

72 

 
 
 
Science practices, and ensuring open access to research outcomes. The MSCA strengthen 
the  research  capacities  of  participating  institutions  (e.g.  through  increasing  their  ability  to 
attract  funding),  contribute  to  building  new  and  sustainable  international  and  intersectoral 
partnerships  and  networks,  and  enhance  the  transfer  of  knowledge  between  sectors  and 
disciplines, improving institutions’ global reputation and visibility. 

There  are  a  number  of  changes  to  the  MSCA  under  Horizon  Europe,  compared  to  the 
previous  Horizon  2020  programme.  Four  out  of  five  actions  were  renamed:  Individual 
Fellowships (IF) became Postdoctoral Fellowships (PF), Innovative Training Networks (ITN) 
became  Doctoral  Networks  (DN),  Research  and  Innovation  Staff  Exchanges  (RISE)  were 
renamed  into  Staff  Exchanges  (SE)  and  European  Researchers’  Night  (NIGHT)  became 
MSCA and Citizens. COFUND retained its name.   

The  actions  were  also  streamlined  and  some  of  the  rules  were  harmonised  to  reduce  the 
administrative  burden  of  participating  organisations  and  researchers.  For  instance,  under 
Postdoctoral Fellowships, the architecture was simplified to include two types of fellowships 
(European and Global Fellowships) and panels (Career Restart, Reintegration, Society and 
Enterprise) were discontinued and replaced with incentives.   

The application process was also mainstreamed in Horizon Europe and a number of changes 
were introduced in the specific actions. Postdoctoral Fellowships now allow for an additional 
placement  of  up  to  six  months  in  a  non-academic  European  organisation  based in  an  EU 
Member State or Horizon Europe associated country, provided that the placement occurs at 
the  end  of  the  fellowship.  Staff  Exchanges  has  made  interdisciplinary secondments within 
Europe possible and reduced the number of person-months to up to 360 to make projects 
more  manageable.  Under  the  COFUND  action,  the  EU  contribution  now  covers  minimum 
remuneration and can be used more flexibly. Also, COFUND applications that scored at least 
85% but that did not receive funding due to insufficient budget are now eligible to receive a 
Seal of Excellence quality label.   

A  few  important  changes  were  also  introduced  in  Doctoral  Networks.  First,  in  all  Doctoral 
Networks-Industrial  Doctorates  (DN-ID),  fellows  can  now  do  periods  of  secondment  in  the 
premises  of  other  consortium  members,  which  are  powerful  tools  for  cross-fertilisation, 
transmission of good practices, and thus fostering the structural impact of the action. In the 
first Horizon Europe DN call of 2021, all DN types had the maximum duration of secondments 
limited to one third of the fellowship duration. As of the 2022 DN call, this maximum level of 
one third was lifted in the case of DN-IDs (and DN-JDs) in order to, inter alia, amplify their 
structural impact. Second, since 2023, the maximum grant size in DN was increased from 
360  person-month  to  540  person-month.  Third,  since  2023,  to  increase  the share or Joint 
Doctorates (JDs), which are a highly integrated form of collaboration and are considered the 
most strategic DN action with regards to structural impact, only for DN-JDs, the maximum 
fellowship duration was increased from three to four years and the maximum project duration 
from four to five years. 

Under Horizon Europe, all MSCA fellows are now subject to a newly introduced long-term 
leave  allowance  and  special  needs  allowance  for  individuals  with  disabilities.  Family 
allowances are now accessible to all couples forming a family.  In addition, the MSCA put a 
stronger emphasis on effective supervision, adequate mentoring, and career guidance, as 
reflected by the introduction of the MSCA Guidelines on Supervision105. In line with the goals 
of the European Green Deal, and without compromising the bottom-up nature of the MSCA, 

105 European Commission, DG EAC (2021). Marie Skłodowska-Curie Actions guidelines on 
supervision. https://data.europa.eu/doi/10.2766/508311.  

73 

 
 
 
the European Commission also introduced the MSCA Green Charter106, which promotes the 
sustainable implementation of research activities. Through the Guidelines for the Inclusion of 
researchers at risk, the MSCA continue to encourage participation of researchers at risk in 
MSCA-funded projects and beyond. 

The  structure  for  the  four  mobility-focused  actions  in  the  MSCA  is  presented  in  the  table 
below: 

DN (Doctoral 
Networks) 

PF (Postdoctoral 
Fellowships) 

SE (Staff 
Exchanges) 

COFUND (Co-
funding of 
national, 
regional, and 
international 
doctoral and 
fellowships 
programme) 

Funding 
schemes 
available 

Objectives 

Doctoral Networks 
(DN), Industrial 
Doctorates (DN-
ID) and Joint 
Doctorates (DN-
JD) 

European 
Postdoctoral 
Fellowships (EF) 
and Global 
Postdoctoral 
Fellowships (GF) 

To train creative, 
entrepreneurial, 
innovative and 
resilient doctoral 
candidates and to 
raise 
attractiveness and 
excellence of 
doctoral training in 
Europe.   

To enhance the 
creative and 
innovative 
potential of 
researchers 
holding a PhD, 
wishing to acquire 
new skills through 
advanced training, 
international, 
interdisciplinary 
and intersectoral 
mobility.   

- 

-  

To develop 
sustainable 
collaborative 
projects between 
different 
organisations from 
the academic and 
non-academic 
sectors.    

To support new 
or existing 
doctoral 
programmes 
and postdoctoral 
fellowship 
schemes at 
regional, 
national or 
international 
levels with the 
aim to spread 
the best 
practices of the 
MSCA. 

Participating 
researchers 

Early-stage 
researchers 

Experienced 
researchers 

Research and 
innovation staff 
and researchers 

Early-stage and 
experienced 
researchers  

Applicant 

Individuals apply 
with a host 
organisation 

Consortium 
(universities, 
research centres, 
businesses, SMEs 
and other non-
academic 
organisations) 

Consortium 
(universities, 
research centres, 
businesses, SMEs 
and other non-
academic 
organisations) 

A single legal 
entity 
(government 
entities, regional 
authorities, 
funding 
agencies, 
universities, 
research 
centres, 
enterprises)   

106 European Commission, DG EAC (2021). Marie Skłodowska-Curie Actions Green Charter. 
https://data.europa.eu/doi/10.2766/720690.  

74 

 
 
 
 
COFUND (Co-
funding of 
national, 
regional, and 
international 
doctoral and 
fellowships 
programme) 

COFUND 
projects should 
last for up to five 
years 

DN (Doctoral 
Networks) 

PF (Postdoctoral 
Fellowships) 

SE (Staff 
Exchanges) 

Duration 

Duration of each 
fellowship (on the 
basis of full-time 
employment) is 
between three and 
36 months, except 
in the case of joint 
doctorates, where 
maximum duration 
is 48 months.  

Staff Exchanges 
projects can last 
up to four years. 
The funding is 
provided for the 
mobility of 
seconded staff 
members from one 
month to one year.   

Standard duration 
of EF is between 
12 and 24 
months.  
GF involves an 
outgoing phase of 
12-24 months, 
followed by a 
mandatory 12-
month return 
phase.   
Both types of 
fellowships may 
include short-term 
secondments. 
Additional support 
to carry out a 
placement of up to 
6 months in a 
non-academic 
organisation, if 
applicable.   

Funding 

Research, training 
and networking 
activities as well 
as management 
and indirect costs.   

Research, training 
and networking 
activities as well 
as management 
and indirect costs.   

Research, training 
and networking 
activities as well 
as management 
and indirect costs.   

Support in the 
form of living, 
mobility 
allowance. If 
applicable, family, 
long-term leave 
and special needs 
allowances   

Support in the 
form of living, 
mobility 
allowance. If 
applicable, family, 
long-term leave 
and special needs 
allowances 

A top-up 
allowance (for 
travel, 
accommodation, 
subsistence costs) 
and a special 
needs allowance, 
if applicable.   

A fixed amount 
(COFUND 
allowance to 
support any 
costs items of 
the programme.   

A long-term 
leave allowance 
and special 
needs 
allowance, if 
applicable.   

Table 5. Structure of the MSCA actions. Source: compiled by the authors. 

Doctoral  Networks  (DN)  aim  to  implement  doctoral  programmes  with  organisations  from 
different  sectors  across  Europe  and  beyond,  to  train  creative,  entrepreneurial,  innovative, 
and  resilient  doctoral  candidates  able  to  face  current  and  future  challenges107.  DN  aim  to 
achieve the following impacts for participating organisations: 

• 

Improved quality, relevance and sustainability of doctoral training programmes and 
supervision arrangements; 

107 Horizon Europe Work Programme 2023-2024. 2. Marie Skłodowska-Curie Actions. European 
Commission Decision C(2022)7550 of 6 December 2022. 

75 

 
 
 
 
 
 
 
 
•  Enhanced cooperation and transfer of knowledge between sectors and disciplines; 

• 

Increased  integration  of  training  and  research  activities  between  participating 
organisations; 

•  Boosted R&I capacity; 

• 

Increased internationalisation and attractiveness; 

•  Regular  feedback  of  research  results  into  teaching  and  education  at  participating 

organisations. 

Postdoctoral  Fellowships  (PF)  seek  to  enhance  the  creative  and  innovative  potential  of 
postdoctoral  researchers,  helping  them  acquire  new  skills  through  advanced  training  and 
international  and  intersectoral  mobility.  To  support  this,  the  PF  have  introduced  a  non-
academic  placement  of  up  to  six  months  after  the  end  of  the  fellowship.  In  addition,  PF 
contribute to: 

•  Strengthening human capital and enhancing the quality of R&I; 

• 

Increasing the attractiveness of Europe as a destination for researchers; 

•  Facilitating knowledge transfer and fostering the culture of open science, innovation, 

and entrepreneurship. 

Staff Exchanges (SE) promote short-term international, intersectoral, and interdisciplinary 
exchanges  of  staff  members  taking  part  in  innovation  activities.  Their  main  objective  is  to 
foster  a  shared  culture  of  research  and  innovation  between  academic  and  non-academic 
organisations  through  secondments108.  Besides  researchers,  the  action  is  also  open  to 
technical, administrative, and managerial staff. SE aim to achieve, for example, the following 
impacts: 

• 

Increased  set  of  research  and  transferable  skills  and  competences,  leading  to 
improved employability and career prospects within and outside academia; 

• 

Increased international exposure leading to extended networks and opportunities; 

•  Enhanced networking and communication capacities with scientific peers, as well as 
with the general public that will increase and broaden the research and innovation 
impact; 

• 

Innovative  ways  of  cooperation  and  knowledge  transfer  between  sectors  and 
disciplines; 

•  Strengthened  and  broader 

international, 

intersectoral  and 

interdisciplinary 

collaborative networks; 

•  Boosted R&I capacity. 

108 Horizon Europe Work Programme 2023-2024. 2. Marie Skłodowska-Curie Actions. European 
Commission Decision C(2022)7550 of 6 December 2022. 

76 

 
 
 
COFUND, or the ‘Co-funding of regional, national, and international programmes’, which co-
finances  new  or  existing  doctoral  and  postdoctoral  programmes.  Its  main  objective  is  to 
spread  the  best  practices  of  the  MSCA  in  topics  such  as  international,  inter-sectoral  and 
interdisciplinary research, training, and mobility109. Participation in COFUND is expected to 
lead towards the following impacts for organisations: 

•  Enhanced quality and sustainability of research training; 

• 

Increased  global  attractiveness,  visibility,  and  reputation  of  the  participating 
organisation(s); 

•  Stronger R&I capacity and output among participating organisations; 

• 

Increased contribution of the participating organisations to the local, regional and/or 
national socio-economic ecosystems; 

•  Regular  feedback  of  research  results  into  teaching  and  education  at  participating 

organisations. 

Additionally, the MSCA and Citizens action aims to bring research and researchers closer 
to the public at large and increase awareness among the general public of the importance of 
R&I  and  young  people’s  interest  in  science  and  research  careers.  It  does  so  through  the 
organisation of the European Researchers’ Night, which is the largest science engagement 
and communication event organised in EU Member States and Horizon Europe associated 
countries110.  Since  2022,  the  action  also  promotes  Researchers  at  Schools  activities 
throughout the year. This new flagship initiative allows researchers to directly engage with 
pupils  of  all  ages,  contributing  to  a  better  understanding  of  the  challenges  faced  by 
contemporary societies and broadening access to research activities for all.  

1.4.1.2. 

Intervention logic  

The  MSCA  focus  on  excellent  research  that  is  fully  bottom-up.  The  bottom-up  approach 
allows researchers to come up with their own solutions to societal and research challenges. 
Reflecting  the  bottom-up  approach,  the  MSCA  are  open  to  all  domains  of  research  and 
innovation, from basic research up to market take-up and innovation services. The MSCA 
are comprised of five main actions, which contribute to the career development, training and 
international  mobility  of  researchers,  and  to  improving  the  science  base  of  the  European 
Union.   

Below we present the intervention logic of the MSCA, which details the pathway from inputs 
to impacts. Each of the actions of the programme creates results and impacts at three levels: 
at the level of the 1) researcher, 2) organisation, and 3) system. The MSCA create a wide 
spectrum of results and impacts. Some of the impacts are directly influenced by the MSCA. 
Through fostering researchers’ mobility, the MSCA also indirectly contribute to wider societal 
impacts such as improved R&I capacity. 

109 Horizon Europe Work Programme 2023-2024. 2. Marie Skłodowska-Curie Actions. European 
Commission Decision C(2022)7550 of 6 December 2022. 
110 Horizon Europe Work Programme 2023-2024. 2. Marie Skłodowska-Curie Actions. European 
Commission Decision C(2022)7550 of 6 December 2022. 

77 

 
 
 
Figure 7. Intervention logic of the MSCA during Horizon Europe. Source: compiled by the authors. 

78 

 
 
 
1.4.2.  Effectiveness 

Under Horizon Europe by July 2023111, the MSCA received 17 657 (99%) eligible and 223 
(1%) ineligible proposals and supported 2 633 projects across the whole programme. The 
total  budget  of  supported  projects  was  approximately  EUR  1.4  billion  with  20.8%  of  the 
budget already spent. Over 97% of the MSCA calls were funded via MSCA Transnational 
Mobility  Activities such  as  MSCA  Doctoral  Networks,  Postdoctoral  Fellowships,  COFUND, 
and  Staff  Exchanges  while Coordination  and  Support  Activities  accounted  for 2.4% of  the 
budget.  Postdoctoral  Fellowships  action  supported  the  highest  number  of  projects,  while 
Doctoral Networks accounted for the highest EU contribution across all actions.  

The  MSCA  programme  attracted  4  212  unique  organisations,  participating  either  as 
participant  organisations  or  coordinating  organisations  in  the  funded  projects.  Out  of  all 
unique organisations, 14% participated in the programme as newcomers.  

The table below provides an overview of the state of implementation of the MSCA programme 
as of July 2023.   

Type of 
action 

No. of 
projects 

EC funding, in 
EUR 

Average EC funding, in 
EUR 

No. of 
participants 

CSA 

COFUND 

64 

27 

48 620 909.44 € 

759 701.71 € 

89 459 760.00 € 

3 313 324.44 € 

428 

590 

DN 

PF 

SE 

309 

728 833 820.25 € 

2 421 374.82 € 

4 911 

2 163 

435 645 804.52 € 

201 408.14 € 

3 386 

78 

72 703 000.00 € 

932 089.74 € 

882 

Grand Total 

2 633 

1 375 263 294.21 € 

522 318.00 € 

10 197 

Table 6. Implementation state of play. Source: compiled by the study team using eCorda data. Cut-off date: July 2023. 

In the MSCA, the highest number of unique participants come from Private for-profit (PRC) 
organisations (45%), followed by Higher or Secondary Education (HEI) organisations (26%), 
and  Research  and  Technology  (REC)  organisations  (14%).  Out  of  all  unique  participants 
coming from the private sector, small and medium-sized enterprises accounted for 48%. 

111 Cut-off date for data is July 2023. 

79 

 
 
 
Type of 
organisation 

No. of unique 
participants 

Share of 
unique 
participants 

EC funding, in EUR 

Share of EC 
funding 

HEI 

OTH 

PRC 

PUB 

REC 

1 088 

26% 

976 262 114.96 € 

69% 

342 

8% 

23 218 046.44 € 

1% 

1 890 

45% 

156 409 515.64 € 

10% 

286 

606 

7% 

29 483 380.07 € 

2% 

14% 

276 131 578.49 € 

18% 

Grand Total 

4 212 

100% 

1 461 504 635.60 € 

100% 

Table 7. MSCA unique participants per type of organisation under Horizon Europe. Source: compiled by the study team 
using eCorda data. Cut-off date: July 2023. 

1.4.2.1. 

Newcomers 

Overall, 14% of unique organisations participated in the MSCA programme as newcomers. 
Participants from PRCs were the largest group of newcomers (64% of all unique newcomers). 
There  were  fewer  newcomers  from  HEIs  (6%  of  all  newcomers)  and  RECs  (6%  of  all 
newcomers). 

Type of organisation 

No. of unique newcomers 

% of unique newcomers 

Higher education institution 

Other 

Private for-profit entities 

Public bodies 

Research organisations 

Grand Total 

37 

87 

381 

50 

37 

592 

6% 

15% 

64% 

8% 

6% 

100% 

Table 8. Number of unique newcomers in MSCA programme under Horizon Europe. Source: compiled by the study team 
using eCorda data. Cut-off date: July 2023. 

The top 100 participating organisations in MSCA account for 3 128 participations so far (31% 
of all MSCA participations) and received more than EUR 666 million. 

80 

 
 
The  largest  share  of  contributions  in  MSCA  went  to  beneficiaries  from  Germany  (14.6%) 
followed by Spain (13.3%), Italy (12.4%), and France (11.9%).  

In terms of participations per type of organisations, the largest share of HEI participations in 
MSCA  were  from  Italy  (11%),  Germany  (10%),  and  Spain  (9%).  Participations  in  private 
companies (PRC) were led  by Germany (14%), France  (10%), and the Netherlands  (9%). 
Spain  (9%),  France  (9%),  and  the  United  States  (7%)  accounted  for  the  highest  share  of 
participations representing public bodies (PUB). Participations from research organisations 
(REC) were led by Spain (22%), France (20%), and Germany (15%). 

Higher 
education 
institutions 

Private for-profit 
entities 

Public bodies 

Research 
organisations 

Other 

Country 

Share 

Country 

Share 

Country 

Share 

Country 

Share 

Country 

Share 

IT 

11% 

DE 

14% 

ES 

9% 

ES 

22% 

BE 

13% 

DE 

10% 

FR 

10% 

FR 

9% 

FR 

20% 

IT 

10% 

ES 

FR 

NL 

9% 

NL 

9% 

US 

7% 

DE 

15% 

ES 

9% 

ES 

9% 

IT 

5% 

IT 

9% 

NL 

7% 

IT 

9% 

UK 

5% 

NL 

5% 

US 

9% 

8% 

8% 

Table 9. Top country participations per type of organisation in MSCA programme under Horizon Europe. Source: compiled 
by the study team using eCorda data. Cut-off date: July 2023. 

1.4.2.2. 

Business participations per type of action  

At the cut-off date of July 2023, the share of private sector participations in all MSCA actions 
was  22% of  all  participations  (this  analysis  takes into  account  calls  launched  in  2021  and 
2022).  When  broken  down  by  action  type  at  the  time  of  the  study,  Doctoral  Networks 
accounted for the highest share of all private sector participations so far in Horizon Europe 
(72%  of  all  private  sector  participations),  followed  by  COFUND112  (10%),  Postdoctoral 
Fellowships (8%), and Staff Exchanges (8%). Looking back at Horizon 2020, over the whole 
duration of the previous Framework Programme, around 20% of all MSCA participations in 
all  types  of  actions  came  from  the  private  sector.  This  means  that  a  similar  share  of 
businesses participate in Horizon Europe by now (22%, as mentioned above) compared to 
the whole duration of Horizon 2020.  

However, these numbers are directly influenced by the fact that Horizon Europe is in its early 
years and that the number of calls taken into account for this analysis varies between actions. 
For instance, the relatively low share of private sector participations in SE can very likely be 

112 That data on partners in COFUND may be affected by the lack of visibility of implementing 
partners. These partners are not recorded in part A and do not become structured data. 

81 

 
 
 
explained by the fact that only one SE call was accounted for, while there were two DN calls 
and two PF calls, which increased the absolute number of businesses participating in DN and 
PF, compared to SE. 

1.4.2.3. 

Participations by country  

Most participants in the MSCA come from EU Member States (77%) while Third Country 
participations accounted for 19%. Participations of the Associated Countries account for 
only  4%.  When  broken  by  action  type, Staff  Exchanges  (SE)  action  attracted  the  highest 
share of participations from third countries across all MSCA actions (35%).   

Associated 
countries 

EU Member 
States 

Third countries 

Total 

COFUND 

Doctoral Networks 

CSA (Coordination and 
Support actions) 

Postdoctoral Fellowships 

Staff Exchanges  

4% 

11% 

8% 

4% 

6% 

78% 

82% 

87% 

72% 

59% 

18% 

100% 

7% 

5% 

24% 

35% 

100% 

100% 

100% 

100% 

Table 10. Share of participations per action type and country group. Source: compiled by the study team using eCorda data. 
Cut-off date: July 2023. 

When  broken  down  by  country,  15  countries  account  for  83%  of  all  participations  in  the 
programme. As presented in the figure below, Germany (12%), Spain (11%), France (11%), 
and Italy (10%) top the list of countries with the highest shares of participations. 

Figure 8. Top 15 countries participating in the MSCA. Source: compiled by the study team using eCorda data. Cut-off date: 
July 2023. 

82 

 
 
 
 
When broken down by action type, Germany accounts for the highest share of organisational 
participations  in  Doctoral  Networks.  The  most  represented  country  in  terms  of  total 
participations  by  organisations  under  COFUND  and  Postdoctoral  Fellowships  actions  is 
Spain (16% and 14%, respectively). Staff Exchanges action had most of participations from 
Italy (10%). 

Figure 9. Top 5 countries participating in MSCA programme under Horizon Europe per type of action. Source: compiled by 
the study team using eCorda data. Cut-off date: July 2023. 

1.4.2.4. 

Training and continuous professional development 

By providing researchers with a comprehensive set of skills and experiences, the MSCA aim 
to enhance their employability, encourage innovation, and contribute to strengthening human 
capital in research and innovation. Among other needs, an overwhelming majority (91%) of 
MSCA  PF  fellows  stated  that  their  MSCA  project  to  a  large  or  very  large  extent  provides 
freedom to pursue their specific research agenda. MSCA PF fellows were also to a (very) 
large extent  (69%)  appreciative  of  the  opportunities  provided  by  the  MSCA  programme  to 
conduct  basic/fundamental  research  and  to  a  lesser  extent  (51%),  of  the  opportunities  to 
conduct  applied  research.  From  the  perspective  of  the  successful  MSCA  applicant 
organisations,  an  absolute majority  of  beneficiary  organisations  stated  that  participation  in 
the  MSCA  improved/is  likely  to  improve  the  skills,  knowledge,  and  competences  of 
researchers to a large or very large extent. 

Successful  MSCA  PF  applicants  also  agree  to  a  (very)  large  extent  (74%)  that  the  host 
organisation provides adequate access to research training and continuous development. 

The survey of Horizon Europe beneficiary organisations shows that  training related to new 
and/or advanced scientific methods (82%), interdisciplinary research techniques (71%), and 
proposal and report writing (64%) were provided most often to the MSCA researchers in DN, 
SE and COFUND actions. Training on the commercialisation of research outputs (13%) and 
entrepreneurship (9%) were mentioned the least. 

83 

 
 
 
Figure 10. Which type of training have you provided or plan to provide in the framework of your MSCA project? Source: 
Survey of Horizon Europe MSCA beneficiary organisations (DN, SE, COFUND), conducted in May-July, 2023 (n=1095) 

Overall,  an  overwhelming  majority  of  all  MSCA  researchers  (94%)  agree  that  the 
opportunities provided by their MSCA project responded to their need to improve their skills 
and competences. To assess the potential impact of MSCA on skills acquisition, the survey 
of MSCA researchers asked them about the extent to which, if at all, the participation in their 
Horizon Europe projects would have an impact on the development of skills in various areas. 

Among successful MSCA applicants, the table below reflects a strong belief that participation 
in  Horizon  Europe  projects  will  have  a  substantial  impact  on  the  development  of  core 
research  skills,  particularly  in  scientific  methods  and/or  techniques,  critical  and  abstract 
thinking, and communication of research results. Even though most areas have received a 
positive response, there are categories such as the commercialisation of research outputs, 
entrepreneurship,  intellectual  property  rights  management,  digital  technologies  and  their 
application, and the sustainable implementation of research activities,  where a small number 
of total respondents expect a significant impact. 

PF 
(n=878) 

COFUND 
(n=31) 

DN 
(n=559) 

SE 
(n=346) 

To a large extent / To a very large extent 

Career development 

87% 

72% 

76% 

46% 

Commercialisation of research outputs 

34% 

36% 

33% 

18% 

Communication of research results  

86% 

83% 

86% 

60% 

Digital technologies and their application 

27% 

48% 

46% 

35% 

84 

 
 
 
 
PF 
(n=878) 

COFUND 
(n=31) 

DN 
(n=559) 

SE 
(n=346) 

Entrepreneurship 

21% 

21% 

27% 

19% 

Intellectual property rights management 

42% 

48% 

45% 

21% 

Open innovation and knowledge transfer 

62% 

68% 

64% 

51% 

Open science (open access, reusability of 
research results, citizen science, etc.) 

79% 

73% 

70% 

45% 

Project and people management 

72% 

52% 

53% 

45% 

Proposal writing 

84% 

84% 

67% 

41% 

Research ethics & integrity 

61% 

73% 

76% 

41% 

Scientific methods and/or techniques  

92% 

88% 

92% 

65% 

Scientific writing 

84% 

88% 

88% 

49% 

Sustainable implementation of research 
activities  

Thinking (critical and abstract thinking, 
problem solving) 

36% 

44% 

44% 

34% 

88% 

96% 

91% 

67% 

Training and supervision of students 

66% 

29% 

44% 

35% 

Well-being, inclusion, diversity 
management 

44% 

46% 

49% 

34% 

Table 11. To what extent, if at all, will participation in your Horizon Europe project have an impact on the development of 
your skills in the following areas? Multiple options possible. Source: Surveys of Horizon Europe MSCA researchers, 
conducted in May-November 2023. 

The  findings  of  the  Survey  of  Horizon  Europe  beneficiaries  also  suggest  that  successful 
MSCA applicant organisations share a generally optimistic view regarding the potential for 
organisation-level  improvements  in  terms  of  training  and  opportunities  for  continuous 
professional  development  for  researchers.  The  successful  MSCA  applicant  organisations 
largely  anticipate  a  positive  impact,  with  the  majority  of  respondents  (74%)  expecting 
improvements  ‘to  a very  large  extent’  or ‘to  a  large extent’.  This  was  further confirmed  by 
some  interviewees,  who  considered  that  the  structured  training  provided  by  the  MSCA 
influenced or led to the development of in-house curricula in participating organisations. 

85 

 
 
 
Figure 11. To what extent, if at all, is your participation in the Horizon Europe project expected to lead to improvements at 
organisational level in the following areas?: Quality of training and opportunities for continuous professional development. 
Source: Survey of Horizon Europe MSCA beneficiary organisations (DN, SE, COFUND), conducted in May-July, 2023 
(n=479). 

1.4.2.5. 

Career advice and scientific supervision  

The  Marie  Skłodowska-Curie  Actions  (MSCA)  place  a  particularly  strong  emphasis on  the 
quality of supervision and mentorship provided to fellows, with the same standards applied 
for ERA Fellowships (funded under the WIDERA programme part). The MSCA Guidelines on 
Supervision  aim  to  ensure  that  researchers  receive  effective  support  and  mentorship. 
Effective  supervision  is  a  critical  component  of  the  MSCA  programme's  commitment  to 
nurturing the next generation of research talent113. 

According to the survey findings, more than half of successful MSCA applicants agree to a 
(very)  large  extent  that  the  host  organisation  has  a  career  development  strategy  for 
researchers  (52%  of  PF  fellows  and  50%  of  DN  fellows  agree  with  this  statement). 
Approximately the same number of MSCA applicants across each action agree that the host 
organisation provides access to career advice (55% of PF fellows and 50% of DN fellows). 
On the other hand, satisfaction with the supervision is more pronounced among successful 
applicants, with more than 70% of PF fellows and 62% of DN fellows agreeing to a (very) 
large  extent  that  the  host  organisation  provides  adequate  supervision  and  supervision 
arrangements  are  clearly  defined.  The  table  below  provides  a  detailed  breakdown  of 
researchers’  responses  in  each  type  of  action  on  services  provided  by  host  organisations 
related to career advice and scientific supervision. 

113 European Commission, Directorate-General for Education, Youth, Sport and Culture, Marie 
Skłodowska-Curie actions guidelines on supervision, Publications Office, 2021, 
https://data.europa.eu/doi/10.2766/508311  

86 

 
 
 
 
PF 
(n=878) 

COFUND 
(n=31) 

SE 
(n=346) 

DN 
(n=559) 

To a large extent / To a very large extent 

Host organisation has a career 
development strategy for researchers 

Host organisation provides access to 
career advice 

Host organisation provides adequate 
supervision and supervision arrangements 
are clearly defined 

52% 

57% 

45% 

50% 

55% 

53% 

37% 

50% 

70% 

60% 

58% 

62% 

Table 12. Based on your experience with the host organisation, to what extent do you agree with the following statements? 
Source: Surveys of Horizon Europe MSCA researchers, conducted in May-November 2023. 

The graph below outlines the perceived improvements at the organisational level in two key 
areas: the provision of career advice for researchers, and the quality of scientific supervision, 
based on the results of the Horizon Europe successful applicant organisations survey. The 
results are analysed across three different MSCA actions, namely, SE, COFUND and DN, 
along with the total percentages across all actions in each area.  

The graph highlights that there is generally a high expectation of improvement across both 
areas with slightly higher expectations for the quality of scientific supervision (76%) compared 
to the provision of career advice (64%) for researchers. 

Figure 12. To what extent, if at all, is your participation in the Horizon Europe project expected to lead to improvements at 
organisational level in the following areas?: Provision of career advice for researchers and quality of scientific supervision 
(MSCA successful applicant organisations). Source: Survey of Horizon Europe MSCA beneficiary organisations (DN, SE, 
COFUND), conducted in May-July, 2023. Note: For MSCA COFUND action, please interpret with caution due to a low 
number of responses. 

The study found that participation in the MSCA programme is anticipated to lead to 
better career opportunities and prospects for individual researchers. This is reflected in 
the  survey  findings,  according  to  which  successful  MSCA  PF  fellows  agree  that  the 
opportunities to gain a research position with better career prospects provided by the MSCA 
project respond to their needs to a (very) large extent (87%). In parallel, an absolute majority 
(90%) of successful MSCA applicant organisations to a (very) large extent believe that their 
MSCA project improved or will likely improve the career prospects of researchers. 

87 

 
 
 
 
Interviews with MSCA beneficiary organisations indicate several reasons for this finding: the 
need for researchers to seek secondments, spending part of their research abroad and in 
industry  as  part  of  their  doctoral  or  postdoctoral  education  and  increasing  the  quality  of 
training in several areas. All of these aspects were identified by interviewees as important 
factors that can improve the career progression of participating researchers.   

1.4.2.6. 

Quality and transparency of recruitment practices / working conditions 

The MSCA also play a significant role in promoting better recruitment practices and working 
conditions  for  researchers  across  Europe  and  beyond.  It  aims  to  create  an  attractive  and 
supportive  research  environment  by  encouraging  participating  organisations  to  adopt  the 
principles outlined in the European Charter for Researchers and the Code of Conduct for the 
Recruitment of Researchers114.  

The MSCA require host organisations, participating in Doctoral Networks and COFUND, to 
advertise open positions to be filled by doctoral and postdoctoral (for COFUND) fellows on 
Euraxess.  These  advertisements  must  provide  clear  eligibility  criteria  and  ensure  that  the 
selection process is fair and unbiased. Also, while MSCA-funded projects are time-bound, 
the host organisations are encouraged to provide researchers with clear and fair employment 
conditions, including fixed-term contracts that align with the duration of the research project. 
Researchers should have access to social security benefits and other employment rights in 
the host country115.  

The  graph  below  outlines  the  expectations  of  MSCA  participating  organisations  for 
organisational  improvements  in  three  areas:  quality  and  transparency  of  recruitment 
practices, working conditions of researchers, and working conditions of other employees. The 
results  of  the  Survey  of  Horizon  Europe  beneficiaries  reflect  strong  optimism  regarding 
improvements in recruitment practices (51%) and working conditions of researchers (49%). 
However,  expectations  are  more  modest  concerning  the  working  conditions  of  other 
employees, with a more even distribution across different levels of anticipated improvement. 

Figure 13. To what extent, if at all, is your participation in the Horizon Europe project expected to lead to improvements at 
organisational level in the following areas?: Quality and transparency of recruitment practices, working conditions of 
researchers; working conditions of other employees (successful applicant organisations). Source: Survey of Horizon Europe 
MSCA beneficiary organisations (DN, SE, COFUND), conducted in May-July, 2023 

114 A Council Recommendation was adopted in July 2023, which includes an annex with the 
Charter for Researchers: https://research-and-
innovation.ec.europa.eu/document/download/e53533fc-f420-4eec-9014-802b6195b837_en. 
115 Ibid.  

88 

 
 
 
 
From the MSCA researchers' perspective, more than half of all researchers agree to a (very) 
large extent (56% of PF fellows, 62% of DN fellows, 57% of SE researchers) that their host 
organisation is committed to improving researchers’ working conditions. This is in line with 
the  findings  from  the  interviews  conducted.  A  significant  number  of  respondents 
considered  that  one  of  the  most  tangible  structuring  effects  resulting  from  their 
participation  in  the  MSCA  was  the  improvement  in  the  working  conditions  of 
researchers. They pointed at measures such as offering  better job contracts or improving 
salary conditions as crucial aspects of their participation in the MSCA that resulted in better 
competitiveness of their organisations. 

A total of 60% of PF fellows and 55% of DN fellows also agree to a (very) large extent that 
their host organisation protects the intellectual property rights of researchers. However, fewer 
than  53%  of  respondents  in  each  MSCA  action  believe  that  there  are  appropriate 
complaints/appeals procedures in place in their host organisations. SE researchers are most 
pessimistic  about  both  the  extent  to  which  organisations  protect  their  intellectual  property 
rights and the complaints/appeals procedures. 

PF 
(n=878) 

COFUND 
(n=31) 

SE 
(n=346) 

DN 
(n=559) 

To a large extent / To a very large extent 

Host organisation has an appropriate 
complaints / appeals procedure 

Host organisation is committed to 
improving the stability of researchers’ 
working conditions 

44% 

53% 

42% 

45% 

56% 

70% 

57% 

62% 

Host organisation protects the intellectual 
property rights of researchers 

60% 

70% 

48% 

55% 

Table 13. Based on your experience with the host organisation, to what extent do you agree with the following statements? 
Source: Surveys of Horizon Europe MSCA researchers, conducted in May-November 2023. 

There  are  obstacles  related  to  recruitment  practices  and  working  conditions  that  can 
negatively impact researchers' fellowships during their mobility period. One of the prominent 
challenges  faced  by  MSCA  researchers  includes  taxation  and  salary  disparities  between 
different countries. This affects approximately one third of respondents in PF action and 44% 
of respondents in DN action. The second most significant challenge across all MSCA actions 
is difficulty in finding accommodation in the host country. A further notable concern is the lack 
of  information  regarding  fellowship  /  grant  details  such  as  salary,  duration,  and  other 
arrangements before the start of the project. A notable share of respondents (except for SE 
action) reported facing difficulties in obtaining visas or work / residence permits. Furthermore, 
around  9%  of  participants  in  PF  and  DN  actions  encountered  obstacles  in  securing  an 
employment  contract  with  full  social  security  coverage  in  the  host  institution  (for  SE  and 
COFUND, this issue was close to non-existent). Around 14% of DN researchers expressed 
concerns  about  a  lack  of  transparency  during  the  recruitment  process  at  the  host 
organisation, while for researchers participating in other actions this was less of an issue.  

89 

 
 
 
PF (n=878) 

COFUND 
(n=31) 

SE (n=346) 

DN (n=559) 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

30% 

44% 

15% 

65% 

N/A 

N/A 

44% 

44% 

7% 

75% 

8% 

85% 

N/A 

N/A 

14% 

77% 

27% 

43% 

17% 

63% 

24% 

51% 

43% 

36% 

8% 

66% 

0% 

83% 

1% 

42% 

9% 

68% 

10% 

51% 

13% 

63% 

3% 

62% 

14% 

57% 

3% 

71% 

0% 

80% 

6% 

71% 

6% 

72% 

7% 

58% 

7% 

53% 

2% 

60% 

10% 

46% 

5% 

65% 

3% 

63% 

1% 

64% 

10% 

50% 

12% 

74% 

0% 

73% 

3% 

73% 

14% 

65% 

23% 

61% 

10% 

70% 

3% 

65% 

30% 

49% 

Taxation and salary 
differences between different 
countries116 

Lack of transparency during 
the recruitment process at the 
host institution117 

Difficulties in finding 
accommodation in the host 
country 

Difficulties in obtaining an 
employment contract with full 
social security in the host 
institution 

Difficulties obtaining visas or 
work/residence permits 

Difficulties accessing the 
research infrastructure/ 
facilities of the host institution 

Insufficient information and 
assistance provided by 
National Contact Points or 
other institutions in your 
country 

Insufficient information and 
assistance provided by the 
Executive Agency 

Insufficient information and 
assistance provided by the 
host institution 

Lack of information regarding 
the fellowship/grant salary, 
duration and other 
arrangements prior to the 
project start 

Table 14. Have you faced the following obstacles during the participation in your MSCA project? Source: Surveys of Horizon 
Europe MSCA researchers, conducted in May-November 2023. 

Open answers to the survey of MSCA researchers provide further insight into the challenges 
faced  in  terms  of  the  quality  and  transparency  of  recruitment  practices  and  working 

90 

 
 
 
 
conditions. The open responses to the survey indicate that several researchers in DN action 
faced uncertainty regarding their final salary due to the host institution's inability to calculate 
it  before  the  contract  was  finalised.  Other researchers  in  DN and  PF  actions  encountered 
substantial deductions from their living, family, and mobility allowances, which impacted their 
financial  stability.  Additionally,  issues  in  various  MSCA  actions  arose  for  working  parents, 
particularly mothers with children, in adapting their working conditions to accommodate family 
responsibilities. Furthermore, a significant concern emerged from contracts that prohibited 
simultaneous  employment  with  other  institutions,  a  crucial  detail  that  applicants  were  not 
aware of at the application stage or after the Grant Agreement was signed. These challenges 
collectively highlight the need for even more precise communication from the host institutions, 
and potentially the MSCA programme as a whole (e.g. via dedicated brochure), about the 
rules and procedures related to salaries and working conditions. 

1.4.2.7. 

Transnational and intersectoral mobility patterns  

The  study  found  that  the  MSCA  programme  is  highly  effective  in  providing 
transnational mobility opportunities, with an overwhelming majority of successful MSCA 
researchers  (around  90%  of  respondent  in  each  type  of  action)  agreeing  that  the 
transnational mobility opportunities provided by their project to a (very) large extent respond 
to their needs.    

By  using survey  data,  the  evaluation  team  also  analysed  intersectoral  mobility  patterns  of 
MSCA  researchers.  To  interpret  the  results  of  the  survey,  a  few  caveats  must  be 
acknowledged. First, the survey asked researchers only about their main host organisation, 
while in some cases researchers may have co-host institutions. Second, the main aim of the 
survey was to compare the long-term periods (time in the main host institution) vs. shorter 
term periods (mainly secondments, placements). 

Efforts are being made to support intersectoral mobility, even if room for improvement 
remains.  For  example,  41.5%  of  MSCA  postdoctoral  fellows  indicate  that  their  Horizon 
Europe project responds to their intersectoral mobility needs. 

The participation of the non-academic sector has slightly increased in MSCA Postdoctoral 
Fellowships  between  Horizon  2020  and  Horizon  Europe.  However,  the  share  of  non-
academic beneficiaries has decreased. Only 1% of MSCA postdoctoral fellows are recruited 
in the private sector, compared with 2.6% of individual fellows under H2020118.  

From  the  perspective  of  MSCA  researchers,  however,  survey  evidence  shows  that  the 
interdisciplinary cooperation opportunities provided by their MSCA project responded 
to their needs to a (very) large extent (79% of researchers in all types of actions agreed 
with this statement). 

116 This aspect is not relevant for SE researchers, since during their secondment they keep 
working at their original / sending institution. 
117 This aspect is not relevant for SE researchers, since during their secondment they keep 
working at their original / sending institution.  
118 Under Horizon 2020, a multidisciplinary panel (Society & Enterprise) was introduced between 
2016 and 2020 with a dedicated budget allocated to fund projects with a non-academic main 
host in MSCA IF. This panel was discontinued under Horizon Europe and non-academic 
placements introduced as an incentive for more intersectoral mobility in MSCA PF. 

91 

 
 
 
As  found  in  previous  studies,  under  Horizon  Europe,  the  MSCA  programme  continues  to 
particularly  support  postdoctoral  fellows  returning  to  their  country  of  origin.  There  are 
significant  numbers  (up  to  67%  in  Greece,  56%  in  Italy,  44%  in  Cyprus)  of  postdoctoral 
fellows  who  use  the  MSCA  Postdoctoral  Fellowships  scheme  to  return  to  their  country  of 
nationality. This is particularly evident in countries such as Greece, Italy, Cyprus, and Spain.   

Figure 14. Fellows with the same nationality as the host country in Horizon Europe MSCA PF action. Source: EC monitoring 
data (cut-off date July 26th 2023). 

1.4.2.8. 

Collaborations and networks  

The  MSCA  place  significant  importance  on  establishing,  strengthening,  and  sustaining 
international and cross-sectoral collaborations among researchers and institutions. The aim 
of  such  collaboration  is  to  enable  the  smooth  movement  of  researchers  and  knowledge 
across borders and sectors, both during and after the fellowship period. The study reveals 
that overall, 86% of successful MSCA applicant organisations believe that the participation 
in the MSCA programme will strengthen or is likely to strengthen relationships with leading 
partners in Europe to a (very) large extent. Similar findings are observed when broken down 
by MSCA action type.   

To a very 
large 
extent 

To a large 
extent 

To a 
moderate 
extent 

To a small 
extent 

Not 
at all 

Number of 
responses 

COFUND  

13% 

38% 

50% 

0% 

0% 

8 

MSCA Doctoral 
Networks 

MSCA Staff 
Exchanges 

41% 

47% 

9% 

2% 

1% 

348 

44% 

40% 

10% 

2% 

0% 

126 

Table 15. To what extent, if at all, has your Horizon Europe project achieved/is likely to achieve the following 
results: Strengthening relationships with leading partners in Europe (successful MSCA applicant organisations). Source: 
Survey of Horizon Europe MSCA beneficiary organisations (DN, SE, COFUND), conducted in May-July 2023 

An  overwhelming  majority  of  successful  MSCA  applicant  organisations  believe  that  their 
MSCA  project  responds  to  their  need  to  create  or  strengthen  collaboration  with  leading 

92 

 
 
 
 
research organisations to a (very) large extent. To a lesser extent, it is believed that MSCA 
projects respond to organisations’ need to create and strengthen collaborations with leading 
industry organisations (58%).  

Interview  evidence  also  showed  that  collaboration  with  industry  partners  was  more 
challenging  than  collaboration  with  research  organisations.  Here,  interviewed  beneficiary 
organisations noted that cooperation with industry and business was highly reliant on both 
the  subject  of  their  research  and  the  size  of  the  industry  partner,  meaning  that  larger 
companies  were  seen  as  more  capable  of  collaboration  than  SMEs.  In  addition  to  this, 
cooperation with industry partners often implied significant challenges, such as complications 
about the ownership of the generated intellectual property. Despite these challenges, some 
respondents indicated that collaborating with the private sector has created tangible outputs 
related to generating impacts to end users, for example, the creation of spin-offs that bring 
research outputs to the market.   

To a very 
large 
extent 

To a 
large 
extent 

To a 
moderate 
extent 

To a 
small 
extent 

Not at 
all 

Number of 
responses 

49% 

39% 

9% 

2% 

1% 

493 

23% 

33% 

25% 

10% 

3% 

493 

To create or 
strengthen 
collaboration with 
leading research 
organisations 

To create or 
strengthen 
collaboration with 
leading industry 
organisations 

Table 16. To what extent does your Horizon Europe project respond to the following needs of your organisation? Source: 
Survey of Horizon Europe MSCA beneficiary organisations (DN, SE, COFUND), conducted in May-July 2023 

The  surveys  also revealed  that MSCA  researchers  and  MSCA organisations  tend  to 
implement  common  activities  in  collaboration  mostly  with  other  projects  funded  by 
programme parts in the Horizon Europe Excellent Science Pillar. These activities can 
encompass  mutual  conferences, 
joint 
publications. The table below indicates that most collaborations are foreseen or implemented 
in  cooperation  with  other  MSCA-funded  projects.  To  a  lesser  extent,  MSCA  organisations 
and  MSCA  researchers  tend  to  collaborate  with  the  European  Research  Council  projects. 
However,  a  significant  share  of  MSCA  organisations  (49%)  and  MSCA  researchers  (over 
60% of responding PF fellows, for example) indicated that their MSCA projects do not have 
joint activities with projects funded under other Horizon Europe programmes/clusters.  

joint  dissemination  activities,  workshops,  or 

In addition, according to survey evidence, the MSCA researchers are to a (very) large extent 
satisfied  with  the  opportunities  to  build  new  collaboration  ties,  for  example,  with  new  co-
authors, during their projects. 

93 

 
 
 
Successful MSCA beneficiary 
organisations (DN, SE, 
COFUND) 

MSCA researchers 

% of 
total 

Number of 
responses 

PF 
(n=879) 

COFUND 
(n=30) 

SE 
(n=346) 

DN 
(n=559) 

Do not know/Not Applicable 

49% 

250 

21% 

47% 

41% 

50% 

Pillar II - Cluster 1 - Health 

0% 

Pillar II - Cluster 2 - Culture, 
Creativity & Inclusive Society 

N/A 

N/A 

No, my project does not have 
joint activities with projects 
funded under other Horizon 
Europe programme/clusters 

Pillar I - European Research 
Council 

Pillar I - Marie Skłodowska-
Curie Actions 

Pillar I - Research 
Infrastructures 

Pillar II - Cluster 3 - Civil 
Security for Society 

Pillar II - Cluster 4 - Digital, 
Industry & Space 

Pillar II - Cluster 5 - Climate, 
Energy & Mobility 

Pillar II - Cluster 6 - Food, 
Bioeconomy, Natural 
Resources, Agriculture & 
Environment 

Pillar III - European 
Innovation Council 

Pillar III - European 
Innovation Ecosystems 

Pillar III - European Institute 
of Innovation and Technology 

5% 

24 

61% 

30% 

31% 

19% 

20% 

101 

8% 

0% 

1% 

3% 

1% 

1% 

7 

6 

2 

2% 

2% 

1% 

1% 

12 

10 

3 

7 

8% 

13% 

13% 

23% 

1% 

1% 

1% 

0% 

0% 

0% 

1% 

1% 

1% 

1% 

1% 

0% 

0% 

0% 

0% 

0% 

0% 

0% 

2% 

1% 

1% 

0% 

1% 

1% 

1% 

0% 

1% 

0% 

N/A 

N/A 

0% 

0% 

1% 

0% 

1% 

1% 

5 

6 

0% 

0% 

0% 

0% 

0% 

0% 

1% 

1% 

Table 17. Are there any activities planned in your project that are implemented in collaboration with projects funded under 
other Horizon Europe clusters and parts? Source: Surveys of Horizon Europe MSCA researchers and organisations, 
conducted in May-November 2023. 

94 

 
 
 
 
According  to  the  administrative  and  monitoring  data  from  the  European  Commission,  the 
MSCA  programme  facilitated  a  total  of  10  197  participations  across  various  actions.  On 
average,  each  MSCA  project  involved  partnerships  comprising  between  1.5  and  22 
collaborating organisations. 

Row Labels 

Total no. of 
participations 

Average no. of participations per 
project 

Coordination and support actions 
(CSA)119 

COFUND 

DN 

PF 

SE 

Total 

428 

590 

4 883 

3 386 

882 

10 197 

6.7 

21.9 

16.3 

1.6 

11.3 

3.9 

Table 18. Average number of participations in MSCA projects by action type. Source: compiled by the study team using 
eCorda data. Cut-off date: July 2023. 

The  study  also  found  that  the  MSCA  are  successful  in  attracting  well-networked 
organisations.120  The  participations  from  the  top  1%  of  most  networked  entities  in  MSCA 
account  for  29.6%  of  total  participations  in  the  programme,  which  is  higher  than  Horizon 
Europe average (22.1%). The network analysis also revealed that there were no newcomers 
to the list of the most central organisations within MSCA, highlighting the long-term stability 
of these organisations which have a long-standing tradition of scientific excellence (see Table 
19).  

Country 
name 

Participant legal name 

Entity 
type 

Degree 

Germany 

FRAUNHOFER GESELLSCHAFT ZUR FORDERUNG DER 
ANGEWANDTEN FORSCHUNG EV 

REC 

3354 

Spain 

France 

AGENCIA ESTATAL CONSEJO SUPERIOR DE 
INVESTIGACIONES CIENTIFICAS 

REC 

2554 

CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE 
CNRS 

REC 

2552 

119 Examples of CSA projects can include support projects for NCPs, MSCA and Citizens 
projects, projects supporting international cooperation, projects supporting the production of 
evidence needed to run the MSCA programme, among other examples. 
120 Based on network analysis, which is explained in the methodological section of the study. 

95 

 
 
 
Country 
name 

Participant legal name 

Entity 
type 

Degree 

Italy 

CONSIGLIO NAZIONALE DELLE RICERCHE 

REC 

2344 

Belgium 

KATHOLIEKE UNIVERSITEIT LEUVEN 

HES 

2076 

France 

COMMISSARIAT A L ENERGIE ATOMIQUE ET AUX 
ENERGIES ALTERNATIVES 

REC 

2040 

Netherlands 

TECHNISCHE UNIVERSITEIT DELFT 

HES 

1921 

Greece 

ETHNIKO KENTRO EREVNAS KAI TECHNOLOGIKIS 
ANAPTYXIS 

REC 

1917 

Finland 

TEKNOLOGIAN TUTKIMUSKESKUS VTT OY 

REC 

1774 

Italy 

ALMA MATER STUDIORUM - UNIVERSITA DI BOLOGNA 

HES 

1690 

Denmark 

DANMARKS TEKNISKE UNIVERSITET 

HES 

1686 

Germany 

DEUTSCHES ZENTRUM FUR LUFT - UND RAUMFAHRT 
EV 

REC 

1560 

Table 19. Most networked organisations in MSCA. Source: compiled by the study team using eCORDA data. Cut-off date: 
July 2023. 

The  MSCA  programme  also  offers  continuity  as  newcomers  to  the  Horizon  2020  MSCA 
programme  went  on  to  participate  in  the  Horizon  Europe  MSCA  programme  425  times.  A 
total  of  1 975  researchers  participated  in  MSCA  under  both  Horizon  2020  and  Horizon 
Europe.  

1.4.2.9. 

Research capacity and outputs 

The study finds that MSCA beneficiary organisations to a (very) large extent agree that their 
MSCA project respond to their needs to strengthen the knowledge base and R&D capacity 
of their organisation (76%) and to enhance the R&D and/or innovation capabilities and skills 
(76%). 

To a very 
large 
extent 

To a 
large 
extent 

To a 
moderate 
extent 

To a 
small 
extent 

Not at 
all 

Count 

30% 

46% 

15% 

5% 

1% 

493 

To strengthen the 
knowledge base and R&D 
capacity in your 
organisation  

96 

 
 
 
To a very 
large 
extent 

To a 
large 
extent 

To a 
moderate 
extent 

To a 
small 
extent 

Not at 
all 

Count 

To enhance the R&D and/or 
innovation capabilities and 
skills in your organisation  

30% 

46% 

16% 

5% 

1% 

492 

Table 20. To what extent does your Horizon Europe project respond to the following needs of your organisation? Source: 
Survey of Horizon Europe MSCA beneficiary organisations (DN, SE, COFUND), conducted in May-July, 2023 

The  study  also  found  that  participation  in  the  MSCA  is  considered  to  contribute  to  the 
participating organisations’ ability to attract funding from national and regional sources. This 
is reflected in the survey findings with 50% of successful MSCA applicant organisations to a 
(very) large extent agree that their MSCA projects resulted/are likely to result in strengthened 
capacity to attract public or private funding. The interviews showed that participation in the 
MSCA was regarded as particularly useful in making beneficiary organisations and National 
Contact  Points  more  competitive  and  better  able  to  attract  funding  from  external  sources. 
This structuring effect was particularly important for small and recently created organisations 
wanting to increase their collaborations in Europe, as well as for organisations from widening 
countries trying to make their institutions as competitive as their counterparts in non-widening 
ones. 

The  findings  from  the  survey  of  MSCA  researchers  also  highlight  the  extent  to  which  the 
expected outputs have been / are likely to be produced by MSCA researchers as a result of 
their Horizon Europe-funded projects. MSCA researchers in all types of actions reported a 
strong emphasis on research-related outputs, with the large majority of MSCA researchers 
across  all  types  of  actions  (92%  in  PF,  around  80%  in  DN  and  COFUND,  68%  in  SE) 
indicating research publications as an anticipated result. This is followed by new or improved 
research methodologies and new large-scale datasets/data. 

In contrast, the survey indicates low percentages in outputs such as business plans, large-
scale product validation, codes of conduct and pre-standards/standards, ranging from 2% to 
7% depending on the specific type of action. This reflects a lower focus on commercialisation 
in the MSCA projects. 

97 

 
 
 
Training programmes

Testing, demonstrating and piloting.

Social innovation (tools, guides, strategies)

Research roadmaps

Research publications (academic articles,…

Recommendations for policy makers/inputs to…

Prototype

Pre-standards/standards

Patents

Other (please indicate):

New/improved software, protocols/guidelines…

New/improved research methodologies

New or improved tools, methods or techniques

New or improved products, services or processes

New large-scale datasets/ data produced

Large scale product validation and market…

Educational materials

Do not know/Not Applicable

Collaboration platforms

Codes of conduct

Business plans for the development of a centre…

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

PF

DN

COFUND

SE

Figure 15. Please indicate which of the following outputs have been produced/are likely to result from your Horizon Europe 
project. Source: Surveys of Horizon Europe MSCA researchers, conducted in May-November 2023 (n=1588). 

98 

 
 
 
Similarly, when looking at the outputs that have been produced or are likely to be produced 
by beneficiary organisations participating in the MSCA, these organisations tend to prioritise 
research  publications  (94%),  training  programmes  (65%),  research  methodologies  (57%), 
new/improved  tools  and  techniques  (54%)  and  educational  materials  (54%).  The  outputs 
beneficiary organisations tend to focus the least include business plans (2%) and large-scale 
product validations and market replication (4%). 

Figure 16. Please indicate which of the following outputs have been produced/are likely to result from your Horizon Europe 
project (successful MSCA applicant organisations). Source: Survey of Horizon Europe MSCA beneficiary organisations (DN, 
SE, COFUND), conducted in May-July, 2023 (n=494). 

The  MSCA  organisations  and  the  MSCA  researchers  were  further  asked  whether  any 
exploitation activities were foreseen as a part of their project. Around half of the respondents 
(51%  of  all  organisations  and  48%  of  all  researchers  in  all  types  of  actions,  respectively) 
indicated that the project results are anticipated to be used for academic purposes. One-third 
of MSCA beneficiary organisations also indicated that the project results will be exploited to 
develop,  create,  manufacture,  or  market  a  product  or  process.  The  least  anticipated 
exploitation  activity  for both beneficiary  organisations  and  researchers  relates  to  licensing 
practices.       

1.4.2.10.  Green Transition and Sustainable Development Goals  

In  line  with  the  European  Green  Deal  and  the  provisions  guiding  the  9th  Framework 
Programme for Research and Innovation, the MSCA introduced a bespoke Green Charter 
that  lays  down  a  set  of  general  principles  and  objectives  that  promote  the  sustainable 
implementation of research activities.  

99 

 
 
 
 
The Green Charter applies to all MSCA-funded projects and participants, who are expected 
to commit to its principles on a ‘best effort’ basis; therefore, the Charter does not impact the 
bottom-up nature or the scientific excellence of the MSCA but rather encourages sustainable 
thinking  in  research  design  and  management.  Some  of  the  measures  suggested  by  the 
Charter include reducing waste, promoting green purchasing, ensuring sustainable events, 
using  low-emission  transport,  teleconferencing  when  possible,  using  renewable  energy, 
raising awareness of sustainability, and sharing best practices.   

The case study on how the MSCA contribute to more sustainable and greener research121 
observed  a  dichotomy  in  awareness  among  the  MSCA  respondents:  approximately  80% 
recognised  the  European  Green  Deal  but  lacked  substantial  awareness  about  the  MSCA 
Green  Charter  and  its  guidelines.  Despite  this  knowledge  gap,  many  MSCA-involved 
researchers  and  institutions  have  taken  proactive  steps  to  foster  sustainable  research 
practices.  

Reported challenges in implementing the Charter's guidelines include constraints related to 
eco-friendly  travel,  limited  access  to  carbon  offsetting  measures122,  and  conflicts  with 
sustainability  ideals  in  certain  research  areas.  External  factors,  such  as  geographic 
limitations  and  institutional  constraints,  further  hinder  Charter  adoption.  Nonetheless,  the 
survey  still  indicates  a  strong  inclination  toward  sustainable  practices  within  the  MSCA 
research  community,  with  a  majority  of  respondents  willing  to  align  with  environmental 
considerations  outlined  in  the  Green  Charter.  Teleconferencing  and  online  tools  are 
increasingly adopted at various research levels, reflecting a broader shift in research culture. 
Institutional  reluctance  towards  certain  measures  related  to  Green  Lab  Accreditation  and 
environmental  units  is  noted.  The  study  suggests  that  Green  Lab  Accreditation-related 
training  could  be  valuable,  though  such  resistance  at  the  institutional  level  may  impact  its 
acceptance.  As  is  further  discussed  in  Case  Study  4,  institutions  could  be  reluctant  to 
embrace environmentally sustainable practices and technologies in laboratory settings, partly 
due to apprehensions regarding the associated costs. Moreover, scepticism may exist at the 
institutional level  regarding  the  efficacy  of  accreditation programs  in  instigating  substantial 
change, alongside uncertainties regarding how accreditation criteria correlate with academic 
research  priorities  and  financial  limitations.  Feedback  highlights  the  need  for  enhanced 
communication, guidance, and specialised training areas.  

The  Charter  does  not  compromise  the  scientific  excellence  of  MSCA  but  encourages 
sustainability in research management; it has the potential to increase the EU’s visibility as a 
supporter  of  sustainable  research  and  promote  discipline-specific  sustainable  measures. 
However,  it  faces  challenges  in  inspiring  change  at  institutions  not  deeply  invested  in 
sustainability123.  The  survey  results  and  interview  responses  underscore  the  significant 
promise  of  the  MSCA  Green  Charter  in  elevating  the  EU's  profile  as  a  champion  of 
sustainable  research,  given  its  alignment  with  the  European  Commission's  stance  on 
sustainability.  Furthermore,  some  interviewees  viewed  the  current  Charter  as  a  potential 
pathway  for  enhancing  awareness  of  discipline-specific  sustainable  practices  and  utilising 

121 Annex 2. Case Study 4. on How MSCA contribute to more sustainable and greener research 
122 Options for carbon offsetting are not always made available at a researcher’s institution or by 
regional authorities, so identifying and facilitating offsetting schemes sometimes falls on 
individual researchers. 
123 For more information on the conditions, best practices, and barriers that contribute to the 
implementation of green/sustainable research activities please refer to the Annex 2. Case Study 
4. on How MSCA contribute to more sustainable and greener research. 

100 

 
 
 
established channels to address challenges encountered by certain projects within the MSCA 
program. 

The  study  further  assessed  the  MSCA’s  contribution  to  the  United  Nation’s  Sustainable 
Development  Goals  (SDGs).  The  table  below  presents  the  number  and  share  of  projects 
funded by MSCA programme contributing to a specific Sustainable Development Goal. SDGs 
that were most addressed by MSCA-funded projects are SDG3 – Good Health and Wellbeing 
(48%),  SDG4  –  Quality  Education  (30%),  and  SDG16  –  Peace,  Justice,  and  Strong 
Institutions (24%). The SDGs covered the least are SDG1 – No Poverty (3%) and SDG2 – 
Zero  Hunger  (3%).  Importantly,  the  SDGs  that  were  covered  the  most  also  received  the 
highest shares of EU contribution. 

SDG1 

SDG2 

SDG3 

SDG4 

SDG5 

SDG6 

SDG7 

SDG8 

SDG9 

SDG10 

SDG11 

SDG12 

SDG13 

SDG14 

No. of projects 

% of total 

EU contribution 
(in EUR million) 

% of total 

101 

86 

3.2% 

2.7% 

5.0 

8.3 

0.3% 

0.6% 

1 542 

48.4% 

440.9 

29.4% 

955 

579 

149 

625 

98 

545 

20 

426 

150 

426 

211 

9.5% 

5.7% 

1.6% 

7.8% 

1.0% 

5.9% 

0.1% 

3.4% 

1.8% 

3.4% 

2.1% 

29.9% 

142.1 

18.2% 

4.7% 

85.9 

24.1 

19.6% 

116.7 

15.7 

88.8 

1.4 

51.5 

26.8 

50.7 

32.1 

3.1% 

17.1% 

0.6% 

13.4% 

4.7% 

13.4% 

6.6% 

101 

 
 
 
No. of projects 

% of total 

EU contribution 
(in EUR million) 

% of total 

SDG15 

SDG16 

134 

772 

4.2% 

16.5 

24.2% 

142.9 

1.1% 

9.5% 

Table 21. Number of MSCA projects by SDG. Source: Corda and projects’ description of action data. Cut-off date: July 
2023. 

The  table  below  shows  that  under  Horizon  Europe,  the  MSCA-funded  projects  started 
focusing significantly more than in Horizon 2020 on SDG4 (quality education), SDG5 (gender 
equality),  SDG7  (affordable  and  clean  energy)  and  SDG16  (peace,  justice  and  strong 
institutions). 

102 

 
 
 
Thematic priority 

MSCA (HE) 

MSCA (H2020) 

SDG1 

SDG2 

SDG3 

SDG4 

SDG5 

SDG6 

SDG7 

SDG8 

SDG9 

SDG10 

SDG11 

SDG12 

SDG13 

SDG14 

SDG15 

SDG16 

3.2% 

2.7% 

48.4% 

29.9% 

18.2% 

4.7% 

19.6% 

3.1% 

17.1% 

0.6% 

13.4% 

4.7% 

13.4% 

6.6% 

4.2% 

24.2% 

3.0% 

0.5% 

38.4% 

7.4% 

1.3% 

3.3% 

6.5% 

3.6% 

14.7% 

0.8% 

9.4% 

2.8% 

14.2% 

4.3% 

4.3% 

7.2% 

Table 22. Comparison in the shares of MSCA projects in H2020 and HE by SDG. Source: Corda and projects’ description of 
action data. Cut-off date: July 2023. 

1.4.2.11.  Deployment of digital tools and technologies  

Science constitutes a pivotal facet of the EU's digital transformation. The European Union 
takes proactive steps to influence the digital economy through comprehensive endeavours, 
spanning  from  enhancing  investments  to  overhauling  EU  regulations.  This  encompasses 
both legislative and non-legislative measures, aimed at refining coordination among Member 

103 

 
 
States  and  facilitating  the  exchange  of  best  practices.  Within  this  context,  the  MSCA 
encourages participating institutions to create digital training modules, enabling researchers 
to  adeptly  navigate  the  expanded  realms  of  collaboration  and  knowledge  dissemination 
facilitated by emerging digital technologies. This encompasses proficiency in collaborative 
tools,  open  access  to  publications,  inclusive  data  management  following  FAIR  principles, 
and active engagement with society, including citizen science initiatives.  

The  survey  findings  indicate  that  several  factors  play  a  significant  role  in  facilitating  the 
application  of  digital  tools  in  research  projects  for  beneficiary  organisations.  Notably,  the 
encouragement  of  open  science  practices  and  data  sharing  emerges  as  a  leading  factor, 
with  53%  of  respondents  highlighting  its  importance.  Access  to  robust  infrastructure  and 
platforms for data storage, processing, and sharing, including those for AI development, is 
also deemed crucial, garnering a notable 44% agreement. Moreover, the availability of high-
quality,  standardised,  and  interoperable  data  is  considered  a  facilitator,  with  46%  of 
respondents recognising its significance. 

Financial  support  (26%)  as well  as  the  responsible  and ethical  use  of  digital  technologies 
(27%) were among the factors that were found the least important for the application of digital 
tools. 

To a very 
large 
extent 

To a 
large 
extent 

To a 
limited 
extent 

Not 
at all 

Number of 
respondents 

22% 

31% 

7% 

3% 

466 

14% 

30% 

10% 

5% 

468 

Encouragement of open science 
practices and data sharing  

Access to infrastructure and 
platforms for data storage, 
processing, and sharing (incl. 
development platforms for AI)  

Availability of high-quality, 
standardized, and interoperable data  

13% 

33% 

7% 

5% 

470 

Cross-disciplinary collaboration and 
knowledge exchange around the 
application of AI & digital tools and 
methods  

Training, education in AI & data 
analysis, programming languages, 
and software  

Clear and well-defined policies on 
data privacy, security, and ownership  

Financial support for research 
projects that incorporate digital tools 
and AI  

13% 

27% 

11% 

7% 

464 

12% 

28% 

11% 

6% 

469 

10% 

24% 

14% 

8% 

464 

9% 

17% 

12% 

13% 

461 

104 

 
 
 
To a very 
large 
extent 

To a 
large 
extent 

To a 
limited 
extent 

Not 
at all 

Number of 
respondents 

Responsible and ethical use of AI 
and digital tools in research (including 
trusted, transparent & verified models 
& algorithmic decision-making)  

8% 

19% 

14% 

11% 

459 

Table 23. To what extent do the following factors facilitate the application of digital tools in your research project? Source: 
Survey of Horizon Europe MSCA beneficiary organisations (DN, SE, COFUND), conducted in May-July, 2023. 
Respondents, who replied “do not know / cannot answer” are excluded. 

The  survey  findings  suggest  that  less  than  half  (49%)  of  successful  MSCA  applicant 
organisations believe to a (very) large extent that their participation in the MSCA project is 
expected to lead to improvements in the use of digital tools and technologies. Furthermore, 
37% of postdoctoral fellows and 46% of doctoral candidates agree that the participation in 
the MSCA will have an impact on the development of their skills related to digital technologies 
and their application. 

1.4.2.12.  Deployment of AI-based tools and methods  

The study found that MSCA researchers and organisations make use of and integrate AI-
based tools or methods in their project to a varying degree (see the table below). AI also 
serves  as  a  research  assistant  in  various  ways,  such  as  drafting  documents,  translating 
languages,  and  conducting  literature  searches  and  reviews.  However,  fewer  than  10%  of 
researchers  in  every  MSCA  action  are  using  AI  as  a  research  assistant  to  a  (very)  large 
extent  (only  2%  of  PF  respondents,  for  example).  Amongst  MSCA  successful  applicant 
organisations, only 17% use AI to communicate results. 

PF 
(n=878) 

COFUND 
(n=31) 

SE 
(n=346) 

DN 
(n=559) 

MSCA 
Organisations 

MSCA researchers 

To a large extent / To a very large extent 

2% 

3% 

6% 

9% 

N/A 

5% 

7% 

7% 

9% 

44% 

4% 

10% 

6% 

9% 

48% 

105 

Using AI as research 
assistant (drafting, translating, 
literature search and review, 
etc.)  

Using AI for data preparation 
(to clean up datasets for 
training, to generate synthetic 
data, to verify training data 
quality, etc).  

Using AI for scientific 
discovery (AI-based 

 
 
 
 
 
 
PF 
(n=878) 

COFUND 
(n=31) 

SE 
(n=346) 

DN 
(n=559) 

MSCA 
Organisations 

hypothesis development, 
combination of datasets for 
novel correlations, etc).  

Using AI to analyse data and 
outcomes, and to develop 
insights (predicting 
replicability of a study, to 
develop prediction, to explain 
certain phenomena)  

Using AI to communicate 
results (to visualise complex 
data analysis, to interact with 
outcomes, etc.)  

6% 

7% 

10% 

10% 

65% 

2% 

7% 

3% 

8% 

17% 

Table 24. To what extent do you make use/integrate AI-based tools or methods in your MSCA project? Source: Surveys of 
Horizon Europe MSCA researchers and organisations, conducted in May-November 2023. 

The survey of Horizon Europe beneficiaries provides more insight into how AI-based tools or 
methods are exploited in MSCA projects. It was observed that organisations leverage AI in 
diverse  ways,  from  research  and  development  to  practical  applications  in  fields  such  as 
health,  waste  management,  and  manufacturing.  They  also  emphasise  the  importance  of 
creating and studying new AI methods and algorithms for improved outcomes. Additionally, 
AI  is  used  extensively  in  data  processing,  decision  support,  and  behavioural  modelling, 
highlighting its versatility and impact across various research endeavours. 

1.4.2.13.  Open Access  

Open Access (OA) is a fundamental aspect of the framework programme’s commitment to 
transparent and accessible research. By encouraging researchers to make their publications 
and  data  openly  available,  the  programme  aims  to  maximise  the  impact  of  its  funded 
research.  However,  evidence  from  the  survey  and  interviews  with  MSCA  and  WIDERA 
(Widening participation) beneficiary organisations suggests that publishing project results in 
OA has been met with several pronounced challenges.  

One  of  the  difficulties  faced  in  OA  publishing  is  the  high  cost  associated  with  the  OA 
publishing  rate,  which  is  strongly  reflected  in  the  results  of  the  survey  of  Horizon  Europe 
successful applicant organisations (see the figure below), while the surveyed researchers are 
less  aware  of  this  problem.  It  acts  as  a  barrier  for  many  researchers,  especially  in  third 
countries,  who  struggle  to  afford  the  open  access  fee.  Additionally,  many  OA  journals 
possess low impact factors, leading to concerns over the visibility and prestige associated 
with publishing on these platforms. This can limit opportunities for students and early-career 
researchers aiming for maximum visibility to enhance their career prospects. The publication 
venue is the choice of the beneficiary, of Horizon Europe projects, but the publication fees 
are  reimbursable  if  the  publishing  venue  is  fully  open  access  (publication  fees  in  hybrid 
venues are not reimbursed). On the other hand, several respondents to the survey indicated 
the absence of peer review in some OA repositories, which ultimately casts uncertainty over 
the accuracy and trustworthiness of the data, as there are no formal checks or guarantees to 
ensure  the  quality  of  information  published  in  these  repositories.  These  combined  factors 

106 

 
 
 
contribute  to  a  complex  landscape  that  challenges  the  effectiveness  and  credibility  of  the 
current  OA  publishing  model.  This  probably  refers  to  low-quality  predatory  open-access 
journals. There are higher-quality OA journals, but it is the choice of the author to go for those 
journals whose main goal is profit based on charging APCs without real peer-review, rather 
than quality research and rigour. 

There were no specific difficulties or challenges
related to the openness of project results

Possible negative impact on your career
progression related to publishing in open
access
Others (please specify):.

Lack of knowledge and resources to share data
and metadata in open access

Lack of guidance on the implementation of the
data management plan

Lack of competences and knowledge on
processes for publishing in open access

Difficulties identifying relevant open
repositories or archives

Challenges to comply with data protection
regulation (GDPR)

Difficulties accessing sufficient funds to pay for
the fees required by some journals or data
repositories (non-eligible processing charges)

0% 5% 10% 15% 20% 25% 30% 35% 40%

All MSCA Organisations

COFUND

SE

DN

PF

Figure 17. Which of the following difficulties or challenges did you face when publishing project results in open access? 
Source: Surveys of Horizon Europe MSCA researchers and organisations, conducted in May-November 2023. 

1.4.3.  Coherence 

1.4.3.1. 

Internal coherence of the MSCA  

The study findings indicate that the MSCA continue to exhibit a coherent set of distinct actions 
in  place  to  support  researchers  at  various  career  stages  and  provide  opportunities  for 
different 
international).  MSCA 
Postdoctoral  Fellowships  (PF)  support  experienced  researchers,  regardless  of  their 
nationality,  in  acquiring  new  skills  and  enhancing  their  career  development  through 

forms  of  mobility  (interdisciplinary, 

intersectoral  and 

107 

 
 
 
international mobility. Doctoral Networks (DN) aim to train a new generation of researchers 
through structured doctoral training programmes created by consortia of organisations from 
academia, industry, and other sectors, promoting interdisciplinary and intersectoral mobility. 
Staff  Exchanges  (SE)  aim  to  promote  international  and  intersectoral  mobility  through  the 
exchange  of  staff  members  involved  in  research  and  innovation  activities  of  participating 
organisations.  The  COFUND  action  remains  focused  on  supporting  regional,  national,  or 
international  programmes  that  enhance  researchers’  mobility  and  training  opportunities 
through  co-funding  mechanisms.  The  action  continues  to  focus  on  and  support  new  or 
existing doctoral programmes and postdoctoral fellowship schemes. MSCA and Citizens, on 
the other hand, have a specific remit and the activities are differentiated from those of the 
other actions. MSCA and Citizens aims to bring researchers closer to the public at large, to 
increase awareness of research and innovation activities and to boost public recognition of 
science and research education. 

Specific concerns were raised by interviewees regarding other aspects related to the internal 
coherence of MSCA. Based on the findings from the interviews, the MSCA’s aim to stimulate 
research excellence by attracting and retaining excellent researchers may be undermined by 
the fact that the country correction coefficients (CCC) used for calculating the living allowance 
of  fellows  participating  in  Doctoral  Networks  and  Postdoctoral  Fellowships  fall  short  of 
accurately  capturing  the  changing  economic  landscape  within  Europe.  According  to  the 
interviewees, the current CCCs do not always accurately reflect the actual cost of living, or 
regional differences within countries. In this regard, attracting top talent poses a significant 
challenge, particularly for widening countries as they compete with non-widening EU Member 
States. However, it is important to mention that the MSCA programme is undergoing a mid-
term review of the MSCA lump sum and unit contributions. It is expected that the mid-term 
review  will  address  any  discrepancies,  to  ensure  that  the  programme  remains  sufficiently 
competitive and attractive for excellent researchers and institutions. 

On the other hand, the interviewees welcomed the change in the eligibility criteria for recruited 
fellows in the Doctoral Networks (DN) action. Under Horizon 2020, eligibility for fellows was 
limited to researchers who were in the first 4 years of their research careers (measured in 
terms  of  full-time  equivalent  research  experience)  and  had  not  been  awarded  a  doctoral 
degree.  However,  under  Horizon  Europe,  the  eligibility  criteria  have  been  expanded. 
Currently,  researchers  who  are  actively  pursuing  their  doctoral  degrees  are  eligible  to 
participate,  regardless  of  the  number  of  years  of  full-time  equivalent  research  experience 
they have. This broader scope allows individuals who might be further along in their research 
careers  but  are  still  in  the  process  of  earning  their  doctoral  degrees  to  participate  in  the 
programme, therefore making it more inclusive. 

1.4.3.2. 

Internal coherence within Horizon Europe  

The set-up and structure of the MSCA programme are particularly in line with the Horizon 
Europe specific objective: ‘to develop, promote and advance scientific excellence, to support 
the creation and diffusion of high-quality new fundamental and applied knowledge, of skills, 
technologies and solutions, to support training and mobility of researchers, to attract talent at 
all levels and contribute to the full engagement of the EU's talent pool in actions supported 
under  the  Programme’124.  The  MSCA  address  this  specific  objective  by  equipping 

124 Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 2021 
establishing Horizon Europe – the Framework Programme for Research and Innovation, laying 
down its rules for participation and dissemination, and repealing Regulations (EU) No 1290/2013 
and (EU) No 1291/2013 (Text with EEA relevance). 

108 

 
 
 
researchers with new knowledge and skills through mobility and exposure across borders, 
sectors,  and  disciplines,  enhancing  training  and  career  development  systems  while  also 
refining institutional and national recruitment processes, while upholding the principles of the 
European  Charter  for  Researchers  and  the  Code  of  Conduct  for  the  recruitment  of 
researchers. 

The study indicates that the MSCA are coherent with other Horizon Europe programme parts, 
with 16% of successful MSCA applicant organisations reporting that their MSCA project is a 
continuation  of  research  activities  carried  out  under  the  previous  Framework  Programme, 
Horizon 2020 (in terms of being based on the work carried out in the past research project). 
The figure for successful MSCA PF applicants stands at 5%.  

Coherence with the European Research Council  

The findings from the interview programme highlight the internal coherence between different 
programme  parts  in  Pillar  1  of  Horizon  Europe.  The  MSCA  were  found  to  be  particularly 
coherent with the European Research Council (ERC). Both programmes are bottom-up or 
investigator-driven  and  share  a  similar emphasis  on  the  stimulation  of  excellence  and  the 
production of high-quality research. ERC and MSCA are also seen as the main instruments 
for  the  development  and  attraction  of  talent  to  the  European  Union.  In  addition,  the 
interviewees  see  complementarities  rather  than  overlaps  between  these  two  programme 
parts. To illustrate this, the MSCA provide funding for the beneficiary’s recruited fellows while 
ERC funding supports not only the Principal Investigators (PIs) but also the researchers sub-
contracted to the project, helping PIs build their own research teams. In addition, the MSCA 
funds a broader range of target groups: both experienced and non-experienced researchers, 
as well as admin / managerial staff under Staff Exchanges. The MSCA also fund a broader 
range of TRLs compared to the ERC. While both programme parts offer funding opportunities 
for researchers at various career stages, the ERC focuses on supporting researchers who 
possess a track record of significant research achievements and aims to support pioneering 
and high-risk / high-reward research projects conducted by independent investigators. The 
interviewees also see ERC as complementary to the MSCA as researchers consider the ERC 
funding opportunities a natural next step in their careers  following their participation in the 
MSCA.  

Coherence  with  Widening  Participation  and  Strengthening  the  European  Research 
Area 

Noticeable parallels exist between MSCA Staff Exchanges and ERA Talents. Both schemes 
promote innovative collaborations in research and innovation through exchanging staff and 
sharing  knowledge  ideas  at  all  stages  of  the  innovation  chain.  However,  MSCA  Staff 
Exchanges encompass collaborations on a global scale, spanning sectors and disciplines, 
while  ERA  Talents  primarily  seeks  to  foster  inter-sectoral  collaboration  with  a  distinct 
emphasis on Widening  Countries. In contrast to MSCA Staff Exchanges, the ERA Talents 
scheme also permits intra-country exchanges under specific circumstances, provided they 
contribute to the preservation or enhancement of regional innovation ecosystems. 

The evidence from the interviews also points to the synergies that exist between  the ERA 
Fellowships and MSCA Postdoctoral Fellowships (PF). The objective of the ERA Fellowships 
is  to  incentivize  brain  circulation  towards  widening  countries.  Researchers  that  consider 
applying  to  the  ERA  Fellowships  can  do  so  by  applying  to  the  MSCA  Postdoctoral 
Fellowships  and  opting  in  for  an  automatic  resubmission  of  their  application  through  ERA 

109 

 
 
Fellowships  in  case  their  proposal  cannot  be  funded  under  the  MSCA  due  to  budget 
constraints. 

Coherence with the European Institute of Innovation and Technology (EIT) 

The  evaluation  of  Horizon  2020  considered  the  MSCA  as  coherent  with  the  European 
Institute  of  Innovation  and  Technology  (EIT).  Horizon  Europe  continues  to  encourage 
synergies between these two initiatives. Under Horizon Europe, the Commission published 
a guide on Synergies and complementarities between the MSCA and the EIT to better inform 
stakeholders  about  such  opportunities  as  well  as  about  potential  bridges  with  existing 
initiatives and projects125. The guide provides examples of synergies in projects funded under 
H2020,  including  MSCA  projects  involving  EIT  KIC  partners  in  their  research  and  training 
programmes;  it  also  reviews  different  opportunities  under  Horizon  Europe.  The  examples 
provided in the guide indicate that most synergies between the MSCA and EIT are developed 
through MSCA Doctoral Networks (previously known as Innovative Training Networks) and 
MSCA COFUND126. 

This study confirmed that instances of synergies between these two programmes  continue 
to exist under Horizon Europe. For example, project SmartOptoelectronics (MSCA-PF-2021) 
aims to develop machine learning (ML) methods to establish trends between the structural 
properties  and  the  electrochemical  performance  of  perovskite  lanthanum-based  oxides 
based  on  high-throughput  experimental  data.  The  postdoctoral  researcher’s  host 
organisation, the Institute for Energy Research in Catalonia, is a partner of EIT InnoEnergy. 
The  host  institute  offers  unique  opportunities  to  reinforce  the  technology  transfer 
competencies of the postdoctoral researcher by carrying out an industrial secondment and 
by involvement in the KIC InnoEnergy community. The same institute was already listed as 
an example of synergies between the two programmes under H2020 projects127. 

Coherence with EU Missions  

The Feedback to Policy Coordination and Support Action (CSA), which will be running 
in  parallel  with  the  MSCA  Work  Programme  2023-2025,  seeks  to  facilitate  stronger 
connections  between MSCA projects  and key EU policy priorities, thereby raising the 
visibility  of  their  contributions.  Apart  from  focusing  on  MSCA  horizontal  priorities,  this 
CSA  will  also  look  in-depth  into  the  contribution  of  MSCA  projects  to  the  five  EU 
Missions.    To  enhance  synergies  and  provide  coordinated  input  to  the  relevant  EU 
policy-making services, two MSCA Cluster events, focused on the Missions Cancer and 
Restore  our  Ocean  and  Waters,  already  took  place  since  2021.  The  MSCA  Cluster 
events aimed at showcasing the outcomes and results of MSCA projects related to the 
thematic areas and objectives of the EU Missions. 

125 European Commission, Directorate-General for Education, Youth, Sport and Culture, 
Synergies between the Marie Skłodowska-Curie actions and the European Institute of Innovation 
and Technology, Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2766/42217  
126 Ibid. 
127 European Commission, Synergies between the Marie Skłodowska-Curie Actions and the 
European Institute of Innovation and Technology. Publications Office of the European Union, 
2022. See page 15 on the 3D Press project funded under MSCA-IF 2018 Call. 

110 

 
 
 
1.4.3.3. 

External coherence with other EU objectives and instruments 

By fostering international competition among scientists and institutions in both academic and 
non-academic sectors, the MSCA contribute to the goals of the ‘Jobs, growth and investment’ 
agenda, the EU Global Strategy and the SDGs. This is achieved through the creation and 
sharing of high-quality knowledge across countries, sectors and disciplines128. 

European Research Area  

The MSCA play a significant role in contributing to and  are in line with the priorities of the 
European Research Area (ERA), a unified research and innovation space in Europe that aims 
to  enhance  the  coordination  and  collaboration  of  research  activities  across  EU  Member 
States and associated countries. The MSCA programme is particularly aligned with the ERA 
priority area on deepening a truly functioning internal market for knowledge. In this regard, 
the  MSCA  are  coherent  with  the  ERA’s  objective  to  foster international  collaboration.  The 
MSCA contribute to this by supporting researchers from around the world to work in Europe 
and European researchers to collaborate with international partners. 

The ERA also promotes gender equality and diversity in research and innovation. The MSCA 
include  measures  to  address  gender  imbalances  in  research  careers  and  encourage  the 
participation of underrepresented groups, aligning with the ERA’s commitment to inclusivity. 

Another  key  component  of  the  ERA  is  the  development  of  research  careers.  The  MSCA 
provide researchers with opportunities to develop their skills, gain interdisciplinary experience 
and  transition  between  different  career  stages.  This  aligns  with  the  ERA’s  emphasis  on 
supporting  researchers  at  all  stages  of  their  careers  and  promoting  sustainable  research 
careers. 

Also, both the ERA and MSCA emphasise the importance of open science and open access 
to  research  results.  Researchers  funded  by  MSCA  are encouraged  to  share their  findings 
openly,  contributing  to  the  dissemination  of  knowledge  and  fostering  collaboration  across 
Europe and beyond. 

Lastly, another key objective of the ERA is to promote researchers' mobility and cross-border 
collaboration. The mobility element is central to MSCA as it directly supports researchers to 
move between countries, institutions, and sectors. 

Euratom Research and Training Programme (EURATOM) 

The new Euratom Programme 2021-2025 has a single set of objectives for both direct and 
indirect actions, seeking to enhance synergies with Horizon Europe in particular in medical 
applications of radiation. A new cooperation between the MSCA and the Euratom programme 
under  Horizon  Europe  provides  new  opportunities  for  nuclear  researchers  in  the  MSCA 
Postdoctoral Fellowships. Researchers in the fields covered by the Euratom programme will 

128 Council Decision (EU) 2021/764 of 10 May 2021 establishing the Specific Programme 
implementing Horizon Europe – the Framework Programme for Research and Innovation, and 
repealing Decision 2013/743/EU (Text with EEA relevance) (OJ L 167 12.05.2021, p. 1, ELI: 
http://data.europa.eu/eli/dec/2021/764/oj)  

111 

 
 
 
be  eligible  to  apply  for  MSCA  Postdoctoral  Fellowships  under  specific  conditions129.  By 
aligning  the  objectives  and  actions,  the  Euratom  Programme  seeks  to  maximise 
collaboration, training, knowledge sharing, and innovation with the MSCA. 

Erasmus  

Both the MSCA and the Erasmus+ are initiatives of the European Union aimed at supporting 
the mobility, training and career development of individuals, including researchers. The legal 
base of these programmes encourages synergies, provided that the same activities cannot 
be funded twice. Under Horizon Europe, the Commission has published a guide on Synergies 
between  the  MSCA  and  the  Erasmus+  programmes  in  the  field  of  higher  education,130  to 
better  inform  stakeholders  from  either  programme  about  opportunities  in  the  other 
programme and potential bridges that can be developed with existing initiatives and projects.  

In  this  regard,  there  are  different  avenues  for  synergies,  for  example,  Erasmus+  Mobility 
Projects participants can integrate into MSCA research teams through traineeships, short-
term physical mobility, and staff exchanges. MSCA researchers can join Erasmus+ intensive 
programmes to share their expertise. Organisations, in particular those involved in mobility 
projects  with  third  countries,  could  use  their  networks  and  existing  partnerships  to  set  up 
collaborative research and innovation exchange projects under the MSCA Staff Exchanges.  

The connections and expertise cultivated in Erasmus Mundus programmes can serve as a 
foundation  for  establishing  doctoral  initiatives,  including  those  offering  joint  or  multiple 
degrees,  especially  within  the  MSCA  Doctoral  Networks  and  COFUND. This  collaboration 
can leverage the educational networks of Erasmus Mundus to facilitate the development of 
advanced  research-oriented  doctoral  programmes  under  the  MSCA,  fostering  a  seamless 
progression from master's-level research to doctoral studies.  

Such examples already exist in the MSCA under Horizon Europe. For example, the MSCA 
Doctoral  Network  programme  EXBRINER  is  implemented  through  an  intersectoral  and 
multidisciplinary  network  composed  of  higher  education  institutions,  research  centres  and 
industrial partners. Drawing from over a decade of collaborative involvement in the Erasmus 
Mundus Joint Doctorate/Master in Membrane Engineering, the consortium shares a common, 
globally acclaimed infrastructure for implementing highly competitive and jointly supervised 
training and research activities.  

Likewise, established European Universities, as part of the Erasmus+ European Universities 
Initiative, are encouraged to exploit the created links to develop doctoral and postdoctoral 
programmes, including joint programmes, under MSCA Doctoral Networks and COFUND, as 
well as collaborative projects under MSCA Staff Exchanges.  

129 European Commission. (2021). Work Programme of the Euratom Research and Training 
Programme 2021-2022. Retrieved from https://ec.europa.eu/info/funding-
tenders/opportunities/docs/2021-2027/euratom/wp-call/2021-2022/wp_euratom-2021-
2022_en.pdf. Also see the conditions outlined in the MSCA Work Programmes. 
130 Synergies between the Marie Skłodowska-Curie Actions and Erasmus+ in the area of higher 
education. See: https://marie-sklodowska-curie-actions.ec.europa.eu/document/synergies-
between-the-marie-sklodowska-curie-actions-and-erasmus-in-the-area-of-higher-education 

112 

 
 
 
European  University  Alliances  can  also  exploit  their  partner  networks  by  participating  in 
outreach activities under MSCA and Citizens. Such synergies are already present in Horizon 
Europe. For example, UNITA, one of the European University Alliances, is implementing an 
MSCA project supported by MSCA and Citizens action which aims to expand the outreach of 
the  European  Researchers’  Night  and  increase  the  public  recognition  of  R&D  activities  in 
rural and mountain areas.   

1.4.3.4. 

External coherence with regional/national programmes and initiatives in 
EU Member States  

Evidence from the desk research and interviews provides insight into the external coherence 
between  the  MSCA  and  regional  or  national  programmes  and  initiatives  in  EU  Member 
States.  The  MSCA  particularly  complement  national  and  regional  programmes  given  their 
emphasis on transnational, international and intersectoral mobility. Interview findings suggest 
that  while  European  Member  States  offer  a  diverse  range  of  doctoral  and  postdoctoral 
training, a common limitation is the focus on funding researchers who are geographically tied 
to their home countries. Also, enhancing and expanding national research capabilities is the 
primary  aim  of  the  support  available  within  EU  Member  States.  Consequently,  these 
limitations can hinder the potential for cross-border collaborations and knowledge exchange. 
The  MSCA  seek  to  bridge  this  gap  and  foster a  more  interconnected  research  landscape 
through  its  international  focus,  allowing  researchers  to  access  opportunities  beyond  their 
home countries, facilitating cross-border collaborations, and promoting a more integrated and 
globally  impactful  research  community.  The  survey  results  confirm  these  findings,  as  an 
overwhelming  majority  (approximately  90%)  of  both  successful  MSCA  organisations  and 
MSCA  researchers  across  all  actions  emphasised  that, compared  to  the  research  funding 
available  at  national  and/or  regional  level,  the  MSCA  provide  more  international  mobility 
opportunities for researchers131. 

The  complementarity  of  the  MSCA  to  national  and  regional  funding  opportunities  is  also 
evident in terms of ensuring the long-term viability of research activities. The findings of the 
survey  suggest  that  for  10%  of  successful  MSCA  organisations  and  13%  of  MSCA 
researchers,  their  projects  are  a  continuation  of  research  activities  carried  out  under 
national/regional R&D public funding programmes132.  

The study also found that MSCA participants tend to exploit national / regional R&D public 
funding  schemes  to  supplement  funding  for  research  activities  addressed  in  their  MSCA 
projects. In this regard, 22% of successful MSCA applicant organisations and 13% of MSCA 
researchers across all actions (25% of PF fellows  – highest share among all researchers) 
stated  that  they  applied  for  additional  funding  from  national/regional  R&D  public  funding 
sources133. However, from the perspective of successful applicant organisations participating 
in Doctoral Networks, the interview findings point to the MSCA’s misalignment with national 
doctoral programmes in many EU Member States. In this regard, participating organisations 
need  to  seek  additional  funding  to  cover  the  fourth  year  of  a  PhD,  as  opposed  to  the 

131 Survey of Horizon Europe beneficiaries, conducted in May-July, 2023 (for DN, SE, COFUND 
n=461; for PF n=809) 
132 Survey of Horizon Europe beneficiaries, conducted in May-July, 2023 (for DN, SE, COFUND 
n=50; for PF n=110) 
133 Survey of Horizon Europe beneficiaries, conducted in May-July, 2023 (for DN, SE, COFUND 
n=111; for PF n=109) 

113 

 
 
 
national/regional funding programmes where the funding is usually provided for the duration 
of 48 months. 

This  last  concern  is  a  recurrent  consideration  which  has  been  duly  acknowledged  by  the 
Commission, but that in the absence of a budget increase for the MSCA programme, cannot 
be  accommodated  at  the  moment  without  significantly  reducing  the  number  of  supported 
fellows. The European Commission approach follows the Bologna Process, which estimates 
the duration of the doctoral cycle to be three years. To accommodate for the specificities of 
Joint Doctorates, an exception was made, starting from the 2023 DN call with an extension 
to  four  years.  This  extension,  however,  was  not  accompanied  by  an  increase  in  budget. 
Doctoral  cycle  in  Europe  lasts  for  3  to  4.5  years,  with  exceptions.  Without  an  increase  in 
budget, extending the fellowship duration to 4 years for all Doctoral Networks projects would 
reduce the number of fellows funded by 25%. DN maximum grant size is 540 person/month, 
which would translate to 15 fellowships of 3 years. Extending the fellowship to 4 years would 
make  it  11  fellowships  instead  of  15.  Therefore,  if  we  hypothetically  applied  the  4-year 
fellowships  to  all  DN, the  projected  number of  fellows  for  2023-2027  would  go  down  from 
1125 to 825 fellows, i.e., 300 less. 

Seal of Excellence  

Under Horizon Europe, the MSCA continue to award Seal of Excellence certificates to project 
proposals that are ranked above a predefined quality threshold but that were not funded due 
to budgetary constraints. The Seal of Excellence scheme also facilitates synergies between 
funding programmes at the EU and national level, including the potential to seek alternative 
funding sources. Funding bodies in this regard can take advantage of the well-established 
and high-quality Horizon Europe evaluation process.  

So far, the European Commission has awarded MSCA Seal of Excellence certificates to a 
significant number of applicants for MSCA Postdoctoral Fellowships (1 956 in 2021134, 1 607 
in  2022135  and  1  737  in  2023136).  Additionally,  21  Seals  of  Excellence  were  awarded  to 
applicants who presented outstanding proposals but fell short of receiving funding from the 
COFUND action in 2021137 and 34 in 2022.   

From  Horizon  2020  to  Horizon  Europe,  the  findings  indicate  that  the  number  of  Seal  of 
Excellence  schemes  on  an  institutional,  regional,  and  national  level  is  steadily  increasing. 
There are currently 31 funding schemes in place across 16 EU Member States for MSCA 
Seal of Excellence holders who applied for Postdoctoral Fellowships and COFUND. Out of 
all  schemes,  13  of  them  are  funded  at  national  level,  five  by  regional  authorities  and  the 
remaining are operating at institutional level (mostly by universities, but also private entities). 

134 MSCA Seal of Excellence awarded to outstanding researchers. (2022). Marie Skłodowska-
Curie Actions. https://marie-sklodowska-curie-actions.ec.europa.eu/news/msca-seal-of-
excellence-awarded-to-outstanding-researchers  
135 MSCA Seal of Excellence awarded to high-quality Postdoctoral Fellowships proposals. 
(2023). Marie Skłodowska-Curie Actions. https://marie-sklodowska-curie-
actions.ec.europa.eu/news/award-of-msca-seal-of-excellence-certificates-postdoctoral-
fellowships-2022  
136 https://marie-sklodowska-curie-actions.ec.europa.eu/news/msca-seal-of-excellence-awarded-
to-outstanding-researchers-0 
137 MSCA Seal of Excellence awarded to outstanding COFUND proposals. (2023). Marie 
Skłodowska-Curie Actions. https://marie-sklodowska-curie-actions.ec.europa.eu/news/msca-
seal-of-excellence-awarded-to-outstanding-cofund-proposals  

114 

 
 
 
Of  these  schemes,  five  are  combining  national  or  regional  funding  with  cohesion  funding 
(ERDF) and six are making use of Recovery and Resilience Facility funding to support MSCA 
Seal of Excellence holders. National, regional, or institutional funding schemes are available 
in Austria, Belgium (Flanders), Bulgaria, Czechia, Cyprus, Estonia, France, Germany, Italy, 
Lithuania, Poland, Romania, Slovakia, Slovenia, Spain and Sweden. 

For example, France's Hauts-de-France region allocates EUR 9.7 million from the ERDF for 
excellent  research  and  innovation  projects,  including  those  awarded  the  MSCA  Seal  of 
Excellence  with  eligible  beneficiaries  being  higher  education 
institutions,  research 
organisations, technical centres as well as businesses138.  

More recently, new schemes have been launched and funded under National Recovery and 
Resilience Plans (NRRP) showcasing successful synergies between Horizon Europe and the 
Recovery  and  Resilience  Facility  (RRF).  For  example,  in  Italy,  the  Ministry  of  Research 
launched an initiative to support approximately 400 researchers under the National RRP. In 
Slovenia,  a  new  initiative  funded  by  the  Recovery  and  Resilience  Facility  (RRF)  provides 
financial support to successful Slovenian researchers who have been awarded the MSCA 
Seal  of  Excellence  when  applying  with  a  host  organisation  abroad  under  the  Individual  or 
Postdoctoral Fellowships calls since 2019139. 

Furthermore,  numerous  higher  education  institutions  and  research  organisations  across 
Europe offer their own initiatives to support Seal of Excellence recipients. These initiatives 
can be found in Austria, Italy, and Spain. The recipients of the COFUND Seal of Excellence, 
from  the  2021-2027  national  programme  period,  are  already  being  supported  by  Czechia 
(with a national call launched in 2022 and additional calls foreseen for consecutive years140), 
as well as Slovenia.  

To  better  guide  Member  States  on  the  available  opportunities  in  terms  of  establishing  a 
national  or  regional  level  scheme  to  fund  Seal  of  Excellence  recipients,  the  European 
Commission also issued a Commission Notice on Synergies between Horizon Europe and 
ERDF programmes. The document outlines the new opportunities available to the managing 
authorities of the cohesion policy programmes, national Horizon Europe contact points and 
Horizon Europe project promoters/proposers. The document is intended to make it easier to 
understand how the scheme can be established and operationalised taking stock of national 
or regional programmes supported by the ERDF141. 

1.4.4.  EU Added value 

The study findings indicate that the majority of unsuccessful MSCA applicant organisations 
(78%) and unsuccessful MSCA PF applicants (60%) did not  implement their projects after 
not receiving MSCA support. Additionally, out of those who secured alternative sources of 

138 Ibid. 
139 ARIS - Načrt za okrevanje in odpornost - Javni poziv za (so)financiranje projektov in 
programov za krepitev mednarodne mobilnosti slovenskih raziskovalcev in raziskovalnih 
organizacij ter za spodbujanje mednarodne vpetosti slovenskih prijaviteljev. (2023). 
http://www.arrs.si/sl/NOO/razpisi/23/poziv-mobilnost-23.asp  
140 OP JAK a MSCA. (2022). ČZU. https://www.czu.cz/cs/r-7584-projekty-a-spoluprace-s-praxi/r-
11162-visual-projekty/op-jak-a-msca.html  
141 Commission Notice Synergies between Horizon Europe and ERDF programmes 2022/C 
421/03 (OJ C, C/421, 04.11.2022, p. 7, CELEX: https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=CELEX:52022XC1104(02)  

115 

 
 
 
funding to implement the activities proposed in their MSCA application, 82% of unsuccessful 
MSCA  applicant  organisations  and  62%  of  unsuccessful  PF  applicants  implemented  their 
planned  projects  with  some  or  significant  adaptations  (see  the  figure  below  for  further 
breakdown).  

Figure 18. Compared to your original Horizon Europe project application, did the research application for alternative funding 
involve. Source: Survey of Horizon Europe MSCA unsuccessful applicant organisations, conducted in May-July, 2023. 

The evidence also reveals that national / regional R&D public funding schemes were the main 
sources  of  alternative  funding  for  both  unsuccessful  MSCA  applicant  organisations  and 
unsuccessful MSCA PF applicants. The survey results indicate that among those applicants 
who received alternative funding to implement their projects, or the activities proposed in their 
MSCA  application  (regardless  of  the  extent  to  which  adaptations  were  made) the  majority 
(78%  for  applicant  organisations,  70%  for  applicants)  received  funding  from  national  / 
regional R&D public funding schemes. Other significant sources of funding for unsuccessful 
applicant  organisations  were  private  R&D  funding  schemes  and  international  research 
funding schemes for unsuccessful applicants. 

116 

 
 
 
DN, COFUND, SE 

PF 

National/regional R&D public funding schemes  

78% 

% 

Other EU funding schemes (e.g. ERDF, ESF, 
CF, etc.)  

International research funding schemes  

Private R&D funding schemes  

My own personal funds  

Other (please specify):  

Do not know/cannot answer  

13% 

13% 

17% 

7% 

4% 

0% 

n= 

42 

7 

7 

9 

4 

2 

0 

% 

n= 

70% 

219 

11% 

16% 

7% 

2% 

9% 

1% 

34 

51 

21 

6 

29 

4 

Table 25. What were the sources of funding for your project (i.e. the project that you continued to implement after not 
receiving Horizon Europe funding)? Source: Survey of Horizon Europe MSCA unsuccessful applicant organisations (DN, 
SE, COFUND), conducted in May-July, 2023 

The study findings further underscore the adjustments that had to be made by unsuccessful 
MSCA  applicant  organisations  and  applicants  who  secured  alternative  funding.  For  the 
overwhelming  majority  of  unsuccessful  PF  applicants,  the  adapted  projects  were  usually 
smaller in scope  (63%),  less  complex in  terms  of  research  methods  (66%),  smaller in  the 
number of research outputs and innovations to be produced (61%), and less favourable in 
terms  of  research  conditions  (63%).  However,  most  unsuccessful  PF  applicants  indicated 
that the duration of the adapted projects was longer compared to the planned MSCA project 
(52%). 

117 

 
 
 
Figure 19. Compared to your Horizon Europe project application, did the research application for alternative funding involve 
(unsuccessful Horizon Europe PF applicants). Source: Survey of Horizon Europe MSCA PF unsuccessful applicants, 
conducted in May-July, 2023 

Similarly, for unsuccessful MSCA applicant organisations the adapted projects were smaller 
in the number of research (sub)areas covered (85%) and research outputs and innovations 
to be produced (81%), as well as less complex/diverse in terms of research methods (84%). 
Furthermore, the adapted projects were shorter in duration (74%) as well as less diverse in 
terms  of  the  project  partners  involved  (82%).  The  overwhelming  majority  of  unsuccessful 
MSCA applicants also indicated that the adapted projects constituted a smaller number of 
consortium partners involved (89%), compared to the planned MSCA project. 

Figure 20. Compared to your Horizon Europe project application, did the research application for alternative funding involve 
(unsuccessful Horizon Europe DN, COFUND, SE applicant organisations). Source: Survey of Horizon Europe MSCA 
unsuccessful applicant organisations (DN, SE, COFUND), conducted in May-July, 2023 

The table below presents the perceptions of unsuccessful MSCA applicant organisations and 
MSCA  PF  applicants  on  the  importance  of  various  reasons  that  hindered  them  from 
implementing projects without Horizon Europe funding.  

118 

 
 
 
 
Among  unsuccessful  MSCA  applicant  organisations,  the  most  prevalent  obstacle  was  the 
lack  of  alternative  funding  available  for  the  type  of  research  they  pursued,  with  72%  of 
respondents  considering  this  a  very  important  or  important  reason.  Additionally,  63%  of 
organisations  emphasised  that  the  amount  of  funding  available  from  other  sources  was 
insufficient.  Concerns  about  salary  and  working  conditions  also  emerged  as  another 
significant factor, with 56% of organisations indicating that the salary and working conditions 
would have been low or unsatisfactory.  

For  unsuccessful  MSCA  PF  applicants,  the  results  demonstrate  that  the  need  for  diverse 
international  partners  was  identified  as  a  critical  factor  with  an  overwhelming  89%  of 
respondents emphasizing its importance. Additionally, 79% of respondents highlighted the 
scarcity of alternative funding for the specific research domain. Lastly, 68% of respondents 
stressed  that  essential  knowledge,  expertise,  and skills were  inaccessible  without  Horizon 
Europe funding. 

119 

 
 
 
Very 
important/ 
important 

Rather 
unimportant/ 
completely 
unimportant 

Number of 
responses 

UNSUCCESSFUL MSCA APPLICANT ORGANISATIONS (DN, COFUND, SE) 

There was no alternative funding available for 
that research topic/area  

72% 

15% 

1 053 

The amount of funding available from other 
sources was insufficient  

63% 

16% 

1 033 

Salary/working conditions would have been too 
low/unsatisfactory  

56% 

25% 

1 035 

I could not access necessary knowledge, 
expertise and skills without Horizon Europe 
funding  

I could not access necessary infrastructures, 
databases and other tools without Horizon 
Europe funding  

55% 

31% 

1 045 

55% 

30% 

1 043 

My project addressed EU-level (and/or global) 
challenges  

48% 

28% 

1 045 

UNSUCCESSFUL MSCA IF APPLICANTS (PF) 

Our project required the involvement of a variety 
of international partners and could not be 
implemented without them 

89% 

9% 

555 

There was no alternative funding available for 
that research topic/area   

79% 

17% 

Necessary knowledge, expertise and skills could 
not be accessed without Horizon Europe  

68% 

26% 

Our project addressed EU-level (and/or global) 
challenges  

66% 

25% 

550 

555 

555 

Necessary research infrastructure, databases 
and other tools could not be accessed without 
Horizon Europe  

52% 

40% 

552 

Table 26. How important were the following reasons that prevented your organisation from implementing the project without 
Horizon Europe funding. Source: Survey of Horizon Europe MSCA PF unsuccessful applicants and Survey of Horizon 
Europe MSCA unsuccessful applicant organisations (DN, SE, COFUND), conducted in May-July, 2023. Respondents, who 
said "do not know / cannot answer" are excluded. 

120 

 
 
 
The  study  also  indicated  additional  reasons  that  prevented  unsuccessful  MSCA  applicant 
organisations and PF applicant researchers from implementing the projects without Horizon 
Europe  funding,  based  on  the  individual  responses  to  the  survey  of  Horizon  Europe 
unsuccessful applicants. These reasons allowed concluding as to why the MSCA funding is 
unique and cannot be entirely replaced by alternative funding, both from organisations’ and 
from postdoctoral fellows’ perspectives. 

The main types of the MSCA added value from the organisations’ perspective were 
the following:  

•  Necessity for collaboration and consortium building. Many respondents noted the value 
of  the  MSCA  in  fostering  international  and  interdisciplinary  collaborations.  The 
programme  allows  assembling  diverse  teams,  which  might  not  be  feasible  through 
smaller national or regional funding sources. 

•  Funding  scale  and  scope.  The  MSCA  funds  projects  that  are  often  larger  in  scope, 
requiring  more  substantial  funding  than  available  through  other  means.  The  financial 
volume of the MSCA and its ability to fund all partners involved is unmatched by many 
national programmes. 

•  Research  focus  and  interdisciplinarity.  The  MSCA  are  unique  in  their  approach  to 
interdisciplinary research. According to the survey respondents, other programmes were 
often too narrow or overly focused on applied research, while Horizon Europe supported 
a broad spectrum of topics. 

•  Motivational and prestige factors. Several respondents mentioned the motivational and 
prestige aspect of the MSCA. The prestige and recognition associated with the MSCA 
funding acts as a catalyst for innovation and research endeavours. 

•  High-risk, high-reward projects. The MSCA are also viewed as an avenue for ambitious 

projects that might be considered too risky by other funding bodies. 

•  Facilitates  access  to  resources.  The  MSCA  provides  access  to  world-class  networks, 
technology,  datasets,  and  ecosystems.  This  access  is  often  indispensable  for  certain 
research projects. 

• 

Impact on career and capacity building. For many young researchers, the MSCA present 
career development opportunities. It allows for capacity building and access to platforms 
that they would not otherwise have, and it helps with the recruitment of researchers for 
hosting organisations. 

The  main  types  of  the  MSCA  added  value  from  the  postdoctoral  fellows’ 
perspective were the following: 

•  Non-discrimination and social inclusion. According to researchers, in other programmes, 
instances of perceived discrimination emerged based on factors such as non-European 
citizenship, gender, and LGBTQ+ identity. This emphasises the importance of inclusive 
funding sources as the MSCA. 

121 

 
 
•  No restrictions based on supervisors' standings. Some institutions, especially academic 
ones, might prioritise or restrict access to resources based on the stature or seniority of 
a  researcher's  supervisor.  For  example,  a  junior  group  leader  or  a  newly  appointed 
faculty  member  might  not  have  the  same  pull  or  access  to  resources  as  a  tenured 
professor  or  a  senior  researcher.  This  can  limit  the  opportunities  available  to  their 
mentees or postdocs. 

•  Project specificity and suitability. Certain research projects required specific conditions 
or  collaborations  that  could  only  be  met  through  MSCA  funding,  be  it  due  to  the 
interdisciplinary nature of the work, the need for specific expertise, or the necessity for 
mobility and collaboration across borders. 

•  Value  and  utility  of  research.  A  few  responses  hinted  at  a  perceived  misalignment 
between their research objectives and the priorities of available funding bodies. Some 
felt  that  institutions  favoured  quick  technological  solutions  and  profit  over  long-term, 
sustainable solutions. 

•  Mobility  and  collaboration.  A  distinct  advantage  of  MSCA  funding  is  its  emphasis  on 
mobility and international collaboration. Without it, many researchers felt unable to move, 
collaborate, or access the necessary expertise. 

The study further indicates a strong level of agreement among successful MSCA applicant 
organisations on various aspects of Horizon Europe funding compared to their national and/or 
regional research funding. A significant 93% of respondents strongly agree or rather agree 
that  Horizon  Europe  provides  more  international  mobility  opportunities  for  researchers, 
highlighting  its  role  in  promoting  global  collaboration  and  knowledge  exchange.  Similarly, 
89% of  respondents  agree  that  Horizon  Europe  offers  greater  diversity  in  partner  profiles, 
including  multi-sectoral  and multi-disciplinary consortia, as  well  as  a  broader geographical 
distribution  of  partners.  Furthermore,  the  table  below  reveals  that  80%  of  respondents 
perceive a higher level of competition for research funding within Horizon Europe, indicating 
MSCA’s reputation as a competitive and prestigious programme. Finally, 70% of respondents 
acknowledge  that  Horizon  Europe  provides  a  higher  amount  of  funding  in  comparison  to 
national  or regional  schemes,  reflecting  its  pivotal  role  in  supporting  research  endeavours 
financially. 

Strongly agree/ 
Rather agree 

Rather disagree / 
Strongly disagree 

Number of 
responses 

SUCCESSFUL MSCA APPLICANT ORGANISATIONS (DN, COFUND, SE) 

Provides more international mobility 
opportunities for researchers  

Provides more diversity in the partners’ 
profiles (multi-sectoral, multi-disciplinary 
consortia, more geographical diversity of 
partners etc.)  

1% 

2% 

461 

460 

93% 

89% 

122 

 
 
 
Strongly agree/ 
Rather agree 

Rather disagree / 
Strongly disagree 

Number of 
responses 

Involves higher level of competition for 
research funding   

Provides higher amount of funding 
(compared to the national/regional 
schemes)  

Helps to better pool the skills and expertise 
necessary for research  

Provides funding for research topic(s) or 
field(s) not covered in national or regional 
R&I funding programmes  

Provides more access to research 
infrastructures, equipment and/or databases  

Helps to better address grand societal 
challenges (e.g., climate change, 
demographics and ageing, etc.)  

Gives more attention for the environmental 
sustainability aspect to innovation  

Gives more flexibility to respond to 
technological developments  

Provides more opportunities for 
commercialization of research results (e.g., 
applicability of results to new markets not 
present in national environment)  

Gives more attention for the human aspect 
to innovation (safety, ethics, user 
acceptance etc.)  

Gives more flexibility to respond to changing 
socio-economic needs  

Helps to achieve economies of scale in 
research (achieving the same result at lower 
cost)  

There are no additional benefits compared 
to national / regional support  

80% 

70% 

65% 

58% 

57% 

56% 

46% 

45% 

3% 

10% 

7% 

13% 

8% 

7% 

9% 

13% 

463 

460 

462 

459 

459 

459 

459 

455 

43% 

8% 

461 

43% 

39% 

36% 

17% 

9% 

11% 

15% 

50% 

459 

455 

458 

432 

Table 27. Would you agree or disagree that, compared to the research funding available to you at national and/or regional 
level, Horizon Europe (successful MSCA applicant organisations). Source: Survey of Horizon Europe MSCA beneficiary 
organisations (DN, SE, COFUND), conducted in May-July, 2023 

123 

 
 
 
Similar to unsuccessful MSCA applicant organisations, a significant majority of the surveyed 
MSCA researchers considers that the MSCA programme provides more international mobility 
opportunities  for  researchers  (93%)  and  also  involves  a  higher  level  of  competition  for 
research funding. The data also reveals that  respondents agree that Horizon Europe is 
more  likely  to  encourage  them  to  pursue  their  careers  within  Europe,  rather  than 
seeking  opportunities  outside  the  continent.  This  sentiment  highlights  the  programme's 
influence in  retaining  research  talent  within  the European  scientific community.  Regarding 
funding, a large majority of researchers also strongly or rather agree that the MSCA provide 
a higher amount of funding compared to national or regional schemes. Finally, around 3 out 
of 4 researchers in all types of actions agree that Horizon Europe contributes to better pooling 
of  skills  and  expertise  required  for  research.  This  suggests  that  the  programme's 
collaborative  nature  facilitates  the  convergence  of  diverse  knowledge  and  capabilities, 
fostering innovative and interdisciplinary research. 

COFUND 

DN 

SE 

PF 

/
e
e
r
g
a

l

y
g
n
o
r
t

S

e
e
r
g
a

r
e
h
t
a
R

/

e
e
r
g
a
s
d

i

r
e
h
t
a
R

e
e
r
g
a
s
d

i

l

y
g
n
o
r
t

S

/
e
e
r
g
a

l

y
g
n
o
r
t

S

e
e
r
g
a

r
e
h
t
a
R

/

e
e
r
g
a
s
d

i

r
e
h
t
a
R

e
e
r
g
a
s
d

i

l

y
g
n
o
r
t

S

/
e
e
r
g
a

l

y
g
n
o
r
t

S

e
e
r
g
a

r
e
h
t
a
R

/

e
e
r
g
a
s
d

i

r
e
h
t
a
R

e
e
r
g
a
s
d

i

l

y
g
n
o
r
t

S

/
e
e
r
g
a

l

y
g
n
o
r
t

S

e
e
r
g
a

r
e
h
t
a
R

/

e
e
r
g
a
s
d

i

r
e
h
t
a
R

e
e
r
g
a
s
d

i

l

y
g
n
o
r
t

S

61% 

0% 

50% 

4% 

45% 

1% 

56% 

2% 

68% 

0% 

51% 

3% 

45% 

3% 

56% 

3% 

61% 

3% 

52% 

2% 

43% 

3% 

50% 

2% 

39% 

0% 

38% 

4% 

0% 

0% 

33% 

5% 

55% 

0% 

50% 

3% 

36% 

4% 

54% 

2% 

71% 

3% 

64% 

1% 

46% 

3% 

73% 

2% 

124 

Gives more attention for the 
environmental sustainability 
aspect to innovation 

Gives more attention for the 
human aspect to innovation 
(safety, ethics, user 
acceptance etc.) 

Gives more flexibility to 
respond to technological 
developments 

Helps to achieve 
economies of scale in 
research (achieving the 
same result at lower cost) 

Helps to better address 
grand societal challenges 
(e.g., climate change, 
demographics and ageing, 
etc.) 

Helps to better pool the 
skills and expertise 
necessary for research 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Involves higher level of 
competition for research 
funding 

Is more likely to make me 
pursue my career in Europe  

Provides funding for 
research topic(s) or field(s) 
not covered in national or 
regional R&I funding 
programmes 

Provides higher amount of 
funding  

Provides more access to 
research infrastructures, 
equipment and/or 
databases 

Provides more international 
mobility opportunities for 
researchers 

Provides more opportunities 
for commercialization of 
research results   

There are no additional 
benefits compared to 
national / regional support 

COFUND 

DN 

SE 

PF 

71% 

0% 

65% 

1% 

59% 

2% 

85% 

1% 

65% 

3% 

63% 

2% 

59% 

2% 

78% 

2% 

58% 

6% 

50% 

3% 

49% 

3% 

58% 

5% 

71% 

10% 

64% 

3% 

51% 

4% 

76% 

3% 

71% 

3% 

61% 

3% 

57% 

5% 

63% 

3% 

74% 

0% 

75% 

0% 

54% 

4% 

86% 

1% 

45% 

0% 

43% 

2% 

71% 

0% 

35% 

2% 

32% 

29% 

12% 

41% 

34% 

5% 

12% 

53% 

Table 28. Would you agree or disagree that, compared to the research funding available to you at national and/or regional 
level, Horizon Europe? Source: Surveys of Horizon Europe MSCA researchers, conducted in May-November 2023. 

1.4.4.1. 

Cross-cutting issues  

Gender equality 

The findings presented below should be considered indicative, since many projects were in 
the  very  early  stages  of  implementation  at  the  time  of  the  analysis.  Due  to  the  limited 
availability of reliable data on recruited researchers, particularly in DN and COFUND projects, 
the accuracy of the information presented is limited. 

In  total,  and  with  the  above  reservations  being  made,  taking  into  account  the  MSCA 
monitoring data as of July 2023, 45% out of all researchers participating in the MSCA were 
female researchers. The recent ex post evaluation of Horizon 2020 revealed that the number 
of female researchers participating in all Horizon 2020 projects was just 23%. This means 

125 

 
 
 
that the MSCA is doing significantly better in terms of involving female researchers compared 
to other projects (using Horizon 2020 data). 

The table also suggests that there are variations in the gender balance across different types 
of MSCA actions. For instance, the MSCA Doctoral Networks is the action where genders 
are represented most equally: as of July 2023, there were 52% male doctoral candidates and 
48% female doctoral candidates.  

Male (% 
and count) 

Female (% 
and count) 

Non-binary 
(only count) 

Count 

MSCA COFUND 

62% (24) 

38% (15) 

MSCA Doctoral Networks 

52% (432) 

48% (402) 

MSCA Staff Exchanges 

57% (571) 

43% (438) 

MSCA Postdoctoral Fellowships  

57% (889) 

43% (680) 

Total  

55% (1916) 

45% (1535) 

0 

1 

0 

4 

5 

39 

835 

1009 

1573 

3456 

Table 29. Distribution of researchers participating in MSCA actions by gender. Source: Compiled by the study team using 
the eCorda dataset (cut-off date June 26 2023) 

An overwhelming majority of surveyed MSCA researchers across all actions (around 90% in 
each action) agreed to a (very) large extent that their host organisation does not discriminate 
against  researchers  based  on  gender,  age,  sexual  orientation,  disability,  religion,  or 
language. The findings indicate that host organisations appear to be successfully fostering 
an  inclusive  research  environment  where  researchers  from  different  backgrounds  and 
identities can participate and contribute. 

Responsible Research and Innovation principles 

The survey findings also point to successful MSCA applicant organisations and MSCA PF 
applicants’  experience  with  the  ethics  self-assessment  in  their  projects.  Almost  half  of 
successful MSCA applicant organisations (48%) and more than half of successful MSCA PF 
applicants (56%) reported positive experiences to a large extent, indicating that they faced 
no  issues  with  the  ethics  self-assessment.  This  high  percentage  of  positive  feedback 
suggests that the EU's process for ethics self-assessment is generally clear, accessible, and 

126 

 
 
 
manageable for most applicants, therefore enabling a successful promotion of high standards 
of ethical conduct in research and innovation. 

Figure 21. What is your experience with the ethics self-assessment in your project? Source: Survey of Horizon Europe 
MSCA beneficiary organisations (DN, SE, COFUND), conducted in May-July, 2023. 

The study further found that the majority of successful MSCA applicant organisations believe 
that their participation in MSCA to a very large and large extent (52%) aids their efforts to 
reinforce Responsible Research and Innovation (RRI) principles within their institutions. The 
table below provides a further breakdown per MSCA action. 

To a very 
large / large 
extent 

To a small 
extent / Not 
at all 

Count 

Successful MSCA applicant organisations 

Doctoral Networks  

47% 

42% 

Staff Exchanges  

COFUND  

63% 

63% 

32% 

38% 

300 

117 

8 

Table 30. To what extent does Horizon Europe project responded to the following needs of your organisation: to strengthen 
the application of Responsible Research and Innovation principles in your organisation (Successful MSCA applicant 
organisations)? Source: Survey of Horizon Europe MSCA beneficiary organisations (DN, SE, COFUND), conducted in May-
July, 2023  

1.4.4.2. 

Citizen science and societal engagement 

Horizon  Europe  aims  to  promote  citizen  science  and  societal  engagement.  Citizen 
engagement  is  an  integral  part  of  the  EU’s  open  science  policy.  Open  science  practices 

127 

 
 
 
 
include involving all relevant knowledge actors such as citizens, civil society, and end users 
in  the  co-creation  of  R&I  activities.  Citizen  science  and  societal  engagement  directly 
contribute to excellence by enlarging the scope of R&I and the quality and quantity of data 
collected, discussed, and analysed. It also increases the robustness of R&I outcomes and 
leverages collective intelligence. The MSCA actions fully support and encourage participating 
organisations and researchers to follow the principles of open science.  

the  highest  proportion)  plan 

Within  this  context,  the  study  found  that  around  one  third  of  successful  MSCA  applicant 
organisations (39%) and a much lower percentage of fellows from each MSCA action (15% 
of  PF  respondents  representing 
to  engage  citizens 
representatives / end-users in the co-creation of R&I content of their MSCA projects. Out of 
those who plan to do so, a larger share of successful applicant organisations (55%) plan to 
actively  involve  citizens  or  end-users  in  co-design  activities  than  researchers  in  all  MSCA 
actions (19% of PF respondents representing the highest proportion), that typically involve 
organising  workshops,  focus  groups,  or  similar  approaches  to  collaboratively  develop 
research and innovation agendas, roadmaps, and policies. The fact that nearly one fifth of all 
individual  PF  researchers  consider  integrating  co-design  activities  in  their  projects  can  be 
considered a very positive result, considering that the MSCA cover all scientific fields, many 
of them where co-design may not necessarily be expected. 

Successful 
MSCA 
applicant 
organisations 

COFUN
D (n=31) 

DN 
(n=559) 

SE 
(n=346) 

PF 
(n=878) 

MSCA researchers 

32% 

13% 

6% 

8% 

8% 

39% 

10% 

7% 

10% 

15% 

Co-assessment: activities, 
such as assisting in the 
monitoring, evaluation and 
feedback to governance of a 
project 

Co-creation: activities 
involving citizens and/or end-
users directly in the 
development of new 
knowledge or innovation, for 
instance through citizen 
science and user-led 
innovation.  

128 

 
 
 
 
 
Successful 
MSCA 
applicant 
organisations 

COFUN
D (n=31) 

DN 
(n=559) 

SE 
(n=346) 

PF 
(n=878) 

MSCA researchers 

55% 

10% 

11% 

11% 

19% 

Co-design: activities such as 
workshops, focus groups or 
other means to develop R&I 
agendas, roadmaps, and 
policies, including deep 
discussion on the 
implications, the ethics, the 
benefits, and the challenges 
related to R&I courses of 
action or technology 
development.  

Do not know/Not Applicable  

12% 

0% 

3% 

2% 

5% 

Table 31. Please indicate which of these activities are used to engage citizens / end-users in your MSCA project? Source: 
Survey of Horizon Europe MSCA researchers and Survey of Horizon Europe MSCA beneficiary organisations (DN, SE, 
COFUND), conducted in May-November 2023. 

1.4.5.  Relevance 

Europe must cultivate a highly skilled and adaptable workforce in research and innovation to 
address current and future challenges, including significant demographic shifts. To ensure 
excellence,  researchers  should  have  the  ability  to  move  freely,  collaborate,  and  share 
knowledge across countries, sectors, and fields, equipped with the necessary expertise to 
address societal issues and drive innovation.   

Europe is a formidable force in the scientific realm. According to Eurostat, ‘the number of full-
time equivalent researchers in the EU increased by more than 45 % between 2012 and 2022, 
from 1.43 to 2.08 million.’142 However, it is projected that the European Union will need to 
train  and  employ  at  least  one  million  new  researchers  by  2027  to  meet  the  targets  for 
increased  R&I  investment.143  This  demand  is  particularly  pressing  beyond  academia, 
extending  to  industry,  businesses  (including  small  and  medium-sized  enterprises), 
government,  civil  society  organisations,  cultural  institutions,  hospitals,  etc.  Achieving  this 
requires  cross-sector  collaboration  to  ensure  the  proper  training  and  integration  of  new 
researchers. 

The European Union must intensify its efforts to attract  more young individuals, both men 
and women, to pursue careers in research, promote inclusivity, and strike a better work-life 
balance. It should also focus on attracting researchers from non-EU countries, retaining its 
existing talent, and facilitating the return of European researchers currently working abroad. 

142 Eurostat, please see: https://ec.europa.eu/eurostat/statistics-
explained/index.php?title=R%26D_personnel.  
143 COUNCIL DECISION (EU) 2021/764 of 10 May 2021 establishing the Specific Programme 
implementing Horizon Europe – the Framework Programme for Research and Innovation, and 
repealing Decision 2013/743/EU, p. 21. 

129 

 
 
 
 
 
 
Additionally, to foster widespread excellence, it is essential to enhance the working conditions 
for researchers throughout the European Research Area (ERA). In this regard, establishing 
stronger synergies with the European Education Area as well as with the cohesion policy is 
crucial. 

In this context, the MSCA serve as the primary Union-level tool for achieving these objectives. 
Since  their  inception  in  1996,  the  MSCA  have  established  themselves  as  the  European 
Union's flagship programme for advancing doctoral education and postdoctoral training. From 
2014  to  2020,  under  Horizon  2020,  they  provided  support  to  65,000  researchers  across 
Europe  and  beyond,  encompassing  both  doctoral  candidates  and  more  experienced 
researchers. Furthermore, MSCA has also allocated funding to more than 1 000  excellent 
international doctoral networks. 

Under Horizon Europe, the MSCA aim to address the societal and global challenges identified 
above, through five areas of intervention. The relevance of each of the MSCA intervention 
areas is discussed below.  

1.4.5.1. 

Nurturing Excellence through the Mobility of Researchers across Borders, 
Sectors and Disciplines 

In an era characterised by global research endeavours, fostering international collaboration 
becomes  imperative  for  achieving  scientific  excellence.  Cultivating  excellence  through  the 
cross-border,  cross-sector,  and  interdisciplinary  mobility  of  researchers  stands  as  a 
cornerstone  of  Europe's  research  landscape.  This  strategy  not  only  promotes  innovation, 
interdisciplinary synergy, and global alliances but also serves as a magnet for attracting and 
retaining top-tier talent. By supporting mobility, Europe aspires to establish a well-equipped 
dynamic and inclusive research ecosystem to confront intricate challenges and drive societal 
advancement. 

Numerous  critical  global  issues,  including  climate  change,  public  health  crises,  and 
technological  advancements  necessitate 
interdisciplinary  solutions.  The  mobility  of 
researchers  actively  encourages  collaboration  spanning  diverse  fields,  thereby  facilitating 
more effective responses to these multifaceted challenges. 

However,  a  bibliometric  analysis  of  researcher  mobility  reveals  a  concerning  ‘brain  drain’ 
phenomenon in several European countries. Surprisingly, even within the European Union, 
a significant proportion of European researchers prefer destinations outside the EU, notably 
the  United  States.  In  response  to  this  trend,  the  European  Research  Area  (ERA)  aims  to 
create an appealing research environment that can attract talented researchers, both from 
within the EU and beyond its borders144. Empirical research demonstrates the effectiveness 
of the Marie Skłodowska-Curie Actions (MSCA) not only in attracting and retaining European 

144 European Commission, Directorate-General for Research and Innovation, Science, research 
and innovation performance of the EU 2022 – Building a sustainable future in uncertain times – 
Executive summary, Publications Office of the European Union, 2023, 
https://data.europa.eu/doi/10.2777/497405  

130 

 
 
 
talent, but also in facilitating the return of European researchers and supporting their mobility, 
particularly towards less developed regions, as evidenced by mobility flow analyses145. 

The mobility of researchers exhibits a positive correlation with the prevalence of international 
scientific  co-publications.  Robust  researcher  mobility  often  translates  into  more  extensive 
international collaboration. When researchers relocate to foreign jurisdictions, they typically 
maintain  connections  with  their  countries  of  origin,  thereby  fostering  greater  collaboration 
between home and host nations. Simultaneously, a high level of international collaboration 
tends  to  enhance  the  attractiveness  of  the  research  ecosystem,  thereby  attracting  more 
researchers and promoting mobility146. 

Furthermore, international scientific co-publications generally yield a higher citation impact 
compared to solo scientific publications. The elevated citation impact of such collaborations 
reflects the superior quality of scientific output. Several empirical studies have explored the 
impact  of  international  mobility  on  scientific  productivity,  consistently  demonstrating  that 
international mobility amplifies the volume of publications147. Additionally, research indicates 
substantial  gains  in  scientific  output  attributable  to  mobility,  with  mobility  leading  to  a 
sustained increase in a researcher's publications by 32% and citations by 63%148. 

The  MSCA  are  highly  relevant  in  nurturing  excellence  through  the  mobility  of 
researchers  across  borders,  sectors  and  disciplines.  The  setup  and  the  guiding 
principles  of  the  programme  directly  foster  transnational,  international,  intersectoral  and 
interdisciplinary mobility through its four main actions Postdoctoral Fellowships (PF), Doctoral 
Networks (DN), Staff Exchanges (SE) and COFUND. The MSCA encourage the mobility of 
researchers  within  or  outside  Europe  regardless  of  nationality  to  undertake  excellent 
research and develop both their skills and careers.  The MSCA was the most international 
part of Horizon 2020 and accounted for 50% of all third countries’ participations in Horizon 
2020. The programme also  allocated the highest total funding to newcomers compared to 
other programme parts. Also, out of all newcomers to the MSCA, more than half (64%) came 
from Private for-Profit Organisations (PRC).  

The survey findings confirm the relevance of MSCA in fostering cross-border, intersectoral 
and interdisciplinary mobility of researchers. The majority of the successful MSCA applicant 
organisations and researchers agree to a very large or large extent that the MSCA provide 
sufficient  funding  opportunities  to  conduct  interdisciplinary  research  (68%  for  beneficiary 

145 European Commission, Directorate-General for Education, Youth, Sport and Culture, Dėlkutė, 
R., Nikinmaa, J., Pupinis, M. et al., Study on mobility flows of researchers in the context of the 
Marie Sklodowska-Curie Actions – Analysis and recommendations towards a more balanced 
brain circulation across the European Research Area – Final report, Publications Office of the 
European Union, 2022, https://data.europa.eu/doi/10.2766/401134  
146 European Commission, Directorate-General for Research and Innovation, Science, research 
and innovation performance of the EU 2022 – Building a sustainable future in uncertain times – 
Executive summary, Publications Office of the European Union, 2023, 
https://data.europa.eu/doi/10.2777/497405 
147 Netz, N., Hampel, S., & Aman, V. (2020). What effects does international mobility have on 
scientists’ careers? A systematic review. Research Evaluation, 29(3), 327–351. 
https://doi.org/10.1093/reseval/rvaa007  
148 Ejermo, O., Fassio, C., & Källström, J. (2019). Does Mobility across Universities Raise 
Scientific Productivity? Oxford Bulletin of Economics and Statistics, 82(3), 603–624. 
https://doi.org/10.1111/obes.12346  

131 

 
 
 
organisations)  and  academia-industry  collaborative 
for  beneficiary 
organisations).  In  parallel,  the  majority  of  MSCA  researchers  find  international  mobility 
opportunities and interdisciplinary cooperation opportunities provided by the MSCA project 
responding to their needs to a very large or large extent. 

research 

(70% 

1.4.5.2. 

Fostering new Skills through the Excellent Training of Researchers and 
Strengthening Human Resources and Skills Development across the ERA 

The number of people obtaining doctorate-level qualifications has increased significantly in 
advanced  economies.  In  2019,  the  mean  proportion  of  individuals  aged  25  to  64  holding 
doctorates in OECD nations was approximately 1%. Projections indicate that should existing 
patterns persist, an estimated 2.3% of today's young adults will engage in doctoral studies at 
some point in their lives149. 

This  increasing  popularity  of  doctoral  education  continues  to  be  facilitated  by  government 
investment,  primarily  aimed  at  enhancing  competitiveness  and  expanding  the  talent  pool 
within  an  increasingly  globalised  landscape150.  However, empirical  evidence  indicates  that 
the  academic  sector  can  no  longer  accommodate  the  growing  influx  of  doctoral  degree 
recipients151.  Remarkably,  statistics  reveal  that  66%  of  PhD  candidates  in  the  European 
Union  express  a  desire  to  remain  in  academia,  while  fewer  than  15%  ultimately  secure 
academic research positions152. The principal reason behind this divergence relates to the 
formidable challenges associated with obtaining a tenure-track position within academia153. 
Consequently, a substantial proportion of researchers exit academia involuntarily, compelled 
to do so when funding sources become depleted, leaving them ill-prepared for a job market 
characterised by opaqueness154. 

Given the current circumstances, it is important to note that doctoral education is changing 
in  terms  of  what  it  offers.  This  includes  industrial  doctorates,  collaborative  doctoral  
programmes,  and  the  introduction  of  professional  and  career  development  opportunities. 

149 OECD. (2019b). “What are the characteristics and outcomes of doctoral graduates?”, in 
Education at a glance 2019: OECD indicators. OECD Publishing, Paris. 
https://doi.org/10.1787/8389c70e-en  
150 Sarrico, C. S. (2022). The expansion of doctoral education and the changing nature and 
purpose of the doctorate. Higher Education, 84(6), 1299–1315. https://doi.org/10.1007/s10734-
022-00946-1  
151 OECD (2021b), "Reducing the precarity of academic research careers", OECD science, 
technology and industry policy papers, No. 113, OECD Publishing, Paris. 
https://doi.org/10.1787/0f8bd468-en  
152 European Commission, Directorate-General for Research and Innovation, Almerud, M., 
Ricksten, M., O’Neill, G. et al., Knowledge ecosystems in the new ERA – Using a competence-
based approach for career development in academia and beyond, Núñez, L.(editor), De Coen, 
A.(editor), Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2777/150763  
153 Sarrico, C. S. (2022). The expansion of doctoral education and the changing nature and 
purpose of the doctorate. Higher Education, 84(6), 1299–1315. https://doi.org/10.1007/s10734-
022-00946-1  
154 European Commission, Directorate-General for Research and Innovation, Almerud, M., 
Ricksten, M., O’Neill, G. et al., Knowledge ecosystems in the new ERA – Using a competence-
based approach for career development in academia and beyond, Núñez, L.(editor), De Coen, 
A.(editor), Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2777/150763  

132 

 
 
 
Furthermore,  the  scope  of  doctoral  education  has  expanded,  with  a  focus  on  preparing 
individuals for diverse career trajectories, transcending disciplinary boundaries155. 

However,  according  to  the  European  Commission,  there  is  still  not  enough  emphasis  on 
entrepreneurship and non-academic opportunities in the training and career development of 
researchers.  The  resulting  skills  mismatches  are  a  worrying  trend  for  both  industry  and 
businesses.  It  is  having  a  detrimental  effect  on  innovation  and  productivity,  especially  in 
highly  innovative  industrial  and  service  sectors156.  Encouraging  researchers  to  pursue 
careers  beyond  academia,  through  enhanced  inter-sectoral  mobility  schemes  involving 
industry,  is  expected  to  enhance  researchers'  employability  and  foster  greater  mobility  of 
talent across the European economy and society.  

Moreover, in the competitive pursuit of global talent, it is imperative to establish favourable 
career  development  conditions  that  can  attract  and  retain  the  finest  researchers  within 
Europe. Unfortunately, precarious employment, particularly for newcomers, has not improved 
significantly  in  recent  years.  This  situation  heightens  the  likelihood  that  the  most  skilled 
researchers may opt to pursue career opportunities outside of the EU157. 

The MSCA play a significant role in nurturing and enhancing researchers’ skills. This 
is  achieved  by  providing  robust  training  opportunities,  either  through  supporting  existing 
training programmes or establishing new ones. These programmes are designed to  equip 
researchers  with  a  wide  range  of  skills  that  are  pertinent  to  current  and  future  global 
challenges. Additionally, the programme places a strong emphasis on transferable skills and 
competences,  ultimately  leading  to  improved  employability  and  career  prospects  for 
researchers.  This  emphasis  aligns  with  the  European  Commission's  endorsement  of  the 
Principles for Innovative Doctoral Training158. The survey findings confirm MSCA’s relevance 
in fostering new skills through the excellent training of researchers and strengthening skills 
development. An overwhelming majority of MSCA researchers (for example, nearly 90% of 
postdoctoral  fellows)  agree  that  opportunities  to  improve  skills  and  competencies  in  their 
MSCA project respond to their needs to a very large and large extent. Similarly, the majority 
of successful MSCA applicant organisations (76%) to a very large and large extent agree that 
MSCA projects respond to their needs to strengthen the knowledge base and R&D capacity 
in their organisation and to enhance the R&D and/or innovation capabilities and skills in their 
organisation. 

Furthermore, the MSCA play a crucial role in addressing issues related to employment 
conditions for researchers by fostering a supportive work environment. In this context, 
the MSCA adheres to the principles outlined in the European Charter for Researchers and 

155 Powell, S., & Green, H. (Eds) (2007). The doctorate worldwide. McGraw-Hill Education: 
Maidenhead. 
156 European Commission. (2020). COMMUNICATION FROM THE COMMISSION TO THE 
EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL 
COMMITTEE AND THE COMMITTEE OF THE REGIONS A new ERA for Research and 
Innovation (COM/2020/628 final). European Commission. https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=COM%3A2020%3A628%3AFIN  
157 Ibid. 
158 European Commission. (2011). Principles for Innovative Doctoral Training. 

133 

 
 
 
the Code of Conduct for the Recruitment of Researchers159. They promote open, merit-based, 
and transparent recruitment processes and strive to provide attractive working conditions and 
employment terms for researchers. Given the current state of doctoral education in Europe, 
the  MSCA  place  particular  importance  on  effective  supervision160.  MSCA  beneficiary 
organisations  are  required  to  ensure  that  researchers  receive  adequate  supervision  or 
mentoring,  along  with  appropriate  career  guidance.  This  focus  aims  to  better  prepare 
researchers  for  a  wide  range  of  career  opportunities.  The  survey  results  confirm  that  the 
opportunities offered by the MSCA project have effectively responded to the need to gain a 
research position with better career prospects. A total of 84% of all MSCA fellows reported 
that  the  project  had  a  very  large  or  large  impact  on  their  ability  to  achieve  this  goal. 
Additionally,  75% of  all  MSCA  researchers  found  that  the  MSCA  project  offered  attractive 
contractual conditions and a better salary to a very large or large extent. 

1.4.5.3. 

Improving and facilitating synergies 

The  European  Union  aims  to  improve  and  facilitate  synergies  in  research  and  innovation 
across  different  programmes  to  achieve  higher  impact,  efficiency,  collaboration,  and 
innovation  in  its  pursuit  of  addressing  complex  challenges  and  fostering  a  competitive, 
sustainable, and prosperous future for Europe. Regulation (EU) 2021/695 of the European 
Parliament and of the Council of 28 April 2021 establishing Horizon Europe, emphasises the 
special attention that must be given to coordination and complementarity between different 
EU policy priorities and objectives resulting in increased synergies between Horizon Europe 
programme  parts  and  with  other  Union  programmes161.  Within  the  context  of  MSCA,  the 
Council's Decision establishing the specific programme implementing Horizon Europe states 
that enhancing the alignment and collaboration between research and innovation systems 
and initiatives at regional, national, or European Union levels requires further attention162. 

The  MSCA  stimulate  regional,  national,  and  international  programmes  to  foster 
excellence in research through the COFUND action. COFUND provides complimentary 
funding  for both  new  and  established  doctoral  and  postdoctoral  programmes  managed  by 
entities established in EU Member States or Horizon Europe associated countries. The action 
also  encourages  synergies  with  Cohesion  policy  funds  as  well  as  the  Recovery  and 
Resilience Facility.  

The  MSCA  also  facilitate  synergies  between  funding  programmes  at  the  EU  and 
national  level  through  the  Seal  of  Excellence.  The  MSCA  continue  to  award  Seal  of 

159 European Commission, Directorate-General for Research and Innovation, The European 
Charter for Researchers – The code of conduct for the recruitment of researchers, Publications 
Office, 2006. 
160 European Commission, Directorate-General for Education, Youth, Sport and Culture, Marie 
Skłodowska-Curie actions guidelines on supervision, Publications Office, 2021, 
https://data.europa.eu/doi/10.2766/508311  
161 Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 2021 
establishing Horizon Europe – the Framework Programme for Research and Innovation, laying 
down its rules for participation and dissemination, and repealing Regulations (EU) No 1290/2013 
and (EU) No 1291/2013 (Text with EEA relevance) (OJ L 170 12.05.2021, p. 1, ELI: 
http://data.europa.eu/eli/reg/2021/695/oj)    
162 Council Decision (EU) 2021/764 of 10 May 2021 establishing the Specific Programme 
implementing Horizon Europe – the Framework Programme for Research and Innovation, and 
repealing Decision 2013/743/EU (Text with EEA relevance) (OJ L 167 12.05.2021, p. 1, ELI: 
http://data.europa.eu/eli/dec/2021/764/oj)  

134 

 
 
 
Excellence certificates to project proposals ranked above a predefined threshold but that did 
not receive funding due to budget limitations. With the label, MSCA recognises the value of 
project proposals and encourages other funding organisations to take advantage of the high-
quality evaluation process. By awarding Seal of Excellence labels, the MSCA further promote 
synergies between the research and innovation framework programmes and the European 
Regional Development Fund (ERDF), ESF and RRF163. 

1.4.5.4. 

Promoting public outreach 

The  European  Union  also  aims  to  implement  information  and  communication  activities 
related  to  Horizon  Europe  highlighting  the  results  achieved  with  the  support  of  the  Union 
funding. In this regard, efforts should be made to increase public awareness regarding the 
significance  of  research  and  innovation  activities  and  the  broader  influence  on  R&I.  To 
achieve  this,  various  means  such  as  publications,  media  relations,  events,  knowledge 
repositories, databases, multi-channel platforms, websites or a targeted use of social media 
should be exploited164.  

The  European  Union  also  aims  to  implement  information  and  communication  activities 
related to Horizon Europe highlighting the results achieved with the support of EUfunding. In 
this regard, efforts should be made to increase public awareness regarding the significance 
of  research  and  innovation  activities  and  the  broader  influence  on  R&I.  To  achieve  this, 
various  means  such  as  publications,  media  relations,  events,  knowledge  repositories, 
databases,  multi-channel  platforms,  websites  or a  targeted  use  of  social  media  should  be 
exploited.165 

The MSCA are particularly relevant in their aim to bridge the gap between research and the 
general  public,  increasing  awareness  of  research  and  innovation  activities  and  raise  the 
interest  of  young  people  in  research  and  scientific  careers.  In  this  regard,  the  MSCA  and 
Citizens  action  plays  an  important  role.  MSCA  and  Citizens  targets  a  broad  audience, 
irrespective of their scientific background, with a particular emphasis on families, students, 
and  those  who  may  face  barriers  to  engaging  with  STEAM  fields  (science,  technology, 
engineering, arts, and mathematics) or research endeavours. Inclusivity is a central principle, 
aiming to democratize access to science and research for everyone. MSCA and Citizens also 
promote gender balance and inclusiveness in Open Science and Responsible Research and 
Innovation.  

As  part  of  MSCA  and  Citizens,  the  European  Researchers’  Night  is  the  largest  research 
communication  and  promotion  event  across  EU  Member  States  and  Horizon  Europe 

163 European Commission. (2022). Commission Notice Synergies between Horizon Europe and 
ERDF programmes (2022/C 421/03) https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=CELEX%3A52022XC1104%2802%29  
164 Council Decision (EU) 2021/764 of 10 May 2021 establishing the Specific Programme 
implementing Horizon Europe – the Framework Programme for Research and Innovation, and 
repealing Decision 2013/743/EU (Text with EEA relevance) (OJ L 167 12.05.2021, p. 1, ELI: 
http://data.europa.eu/eli/dec/2021/764/oj)  
165 Council Decision (EU) 2021/764 of 10 May 2021 establishing the Specific Programme 
implementing Horizon Europe – the Framework Programme for Research and Innovation, and 
repealing Decision 2013/743/EU (Text with EEA relevance) (OJ L 167 12.05.2021, p. 1, ELI: 
http://data.europa.eu/eli/dec/2021/764/oj) 

135 

 
 
 
associated countries, with more than 1.5 million visitors each year. Linked to the European 
Researchers’ Night, the Researchers at Schools initiative brings researchers to schools and 
other pedagogical and educational centres to interact with pupils on societal challenges and 
on the key role of research to address them. Pupils will thus also learn directly about research 
projects and initiatives related to EU main priorities. Researchers at Schools have reached 
more than 323 566 pupils in 2022-2023. 

In addition, Science is Wonderful! is a major science fair funded under the MSCA, and which 
celebrates  the  value  and  impact  of  EU-funded  research,  putting  MSCA  fellows  in  the 
spotlight.  The  event  takes  place  every  year  or  every  two  years  and  gives  primary  and 
secondary school students the chance to interact with leading researchers and innovators, 
to learn more about their work in engaging formats and ask questions about scientific careers. 

1.4.5.5. 

Making MSCA actions more flexible 

The  MSCA  programme  also  welcomed  feedback  from  stakeholders,  resulting  in 
several  changes  under  Horizon  Europe,  compared  to  the  previous  Framework 
Programme.  For  example,  PFs  have  expanded  their  provisions  to  include  an  optional 
placement of up to 6 months in a non-academic organisation located in an EU Member State 
or  a  Horizon  Europe  associated  country,  with  the  placement  happening  at  the  end  of  the 
fellowship.  As  another  example  under  Horizon  Europe,  Staff  Exchanges  enable 
interdisciplinary secondments within Europe, resulting in a reduction of person-months to a 
maximum of 360 to enhance project manageability. Within the COFUND action, the MSCA 
now cover minimum remuneration which can be used more flexibly. Additionally, COFUND 
applications  scoring  at  least  85%, yet  not  securing  funding  due  to  budget  constraints,  are 
now eligible for the prestigious Seal of Excellence quality label. 

As it was explained in the introduction of this part evaluating the MSCA, important changes 
were  also  introduced  in  Doctoral  Networks.  First,  in  all  Doctoral  Networks-Industrial 
Doctorates  (DN-ID),  fellows  can  now  do  periods  of  secondment  in  the  premises  of  other 
consortium  members,  which  are powerful  tools  for cross-fertilisation,  transmission  of  good 
practices, and thus fostering structural impact. In the first Horizon Europe DN call of 2021, all 
DN types had the maximum duration of secondments limited to one third of the fellowship 
duration. As of the 2022 DN call, this maximum level of one third was lifted in the case of DN-
IDs (and DN-JDs) in order to, inter alia, amplify their structural impact. Second, since 2023, 
the maximum grant size in DN was increased from 360 person-month to 540 person-month. 
Third,  since  2023,  to  increase  the  share  or  Joint  Doctorates  (JDs),  which  are  a  highly 
integrated form of collaboration and are considered the most strategic DN action with regards 
to structural impact, only for DN-JDs, the maximum fellowship duration was increased from 
3 to 4 years and the maximum project duration from 4 to 5 years. 

1.4.5.6. 

Making the MSCA programme more socially inclusive 

Under Horizon Europe, all MSCA fellows now benefit from a newly introduced long-term leave 
allowance, specific needs allowances for individuals with disabilities, and accessible family 
allowances for all recipients. Furthermore, the MSCA place a strong emphasis on effective 
supervision, adequate mentoring, and career guidance, as underscored by the introduction 
of  the  MSCA  Guidelines  on  Supervision.  Complementing  the  objectives  of  the  European 
Green Deal and maintaining the MSCA's organic approach, the European Commission has 

136 

 
 
introduced  the  MSCA  Green  Charter,  which  promotes  the  sustainable  implementation  of 
research activities.  

1.4.5.7. 

Aligning with key political priorities of the European Commission, and 
responding to the evolving political context 

The MSCA further aligned their focus to better reflect the European Commission’s key 
political priorities for the period 2019-2024. In this regard, the MSCA issued Guidelines 
for  the  Inclusion  of  Researchers  at  Risk  in  2021.166  The  Guidelines  address  the  need  to 
support researchers who face precarious situations or are at risk due to political, social, or 
human rights-related reasons in their home countries. The guidelines aim to provide a safe 
and supportive environment for these researchers, allowing them to continue their scientific 
work and develop their careers without facing threats or constraints. The document contains 
useful background on researchers at risk, along with recommendations on improving their 
recruitment practices.  

In line with the goals of the European Green Deal and the Sustainable Development Goals 
(SDGs),  the  MSCA  also  issued  the  MSCA  Green  Charter,167  a  code  of  good  practices  for 
individuals and institutions receiving MSCA funding. The MSCA Green Charter is not binding; 
however, all MSCA projects at final reporting stage will be asked to report on the ways they 
have sought to minimise the environmental impact of their research activities and how they 
applied the principles of the Green Charter. It also allows for a better monitoring of MSCA 
beneficiaries’  practices  and  their  environmental  impact,  as  well  as  for  a  longer  term  “soft” 
approach to better support them towards greener and more sustainable practices. 

MSCA  also  showed  a  high  degree  of  responsiveness  and  flexibility  in  addressing 
newly emerging needs resulting from the challenges such as the Russian invasion of 
Ukraine  and  the  COVID-19  crisis.  In  response  to  the  Russian  invasion  of  Ukraine,  the 
MSCA have taken proactive steps to support affected doctoral candidates and post-doctoral 
researchers from Ukraine. Through the dedicated MSCA4Ukraine scheme launched in 2022, 
a funding allocation of EUR 25 million was designated to aid displaced researchers, allowing 
them  to  continue  their  scientific  pursuits  in  Europe  and  contribute  to  the  rebuilding  of 
Ukraine's research and innovation capacity when conditions permit. The scheme received 
over 600 applications from 28 countries and 111 post-doctoral and 13 doctoral fellowships 
were awarded to researchers from Ukraine. 

In  early  2021,  REA  conducted  a  survey  to  gather  information  on  the  challenges  faced  by 
participating organisations managing MSCA projects during the pandemic. The survey aimed 
to  assess  the  effectiveness of  measures  taken  by  the  European  Commission  and  REA  in 
providing  support  during  similar  future  situations.  The  findings  suggest  that  the  proposed 
flexibility measures were generally seen as appropriate and effective, allowing organisations 
to adapt to restrictions and constraints. Most respondents to the survey viewed the ability to 
fully pay MSCA researchers positively, even if they could not work in the host organisation’s 
premises. Respondents also considered the extended flexibility in project deliverables and 

166 Guidelines for Inclusion of Researchers at Risk. https://marie-sklodowska-curie-
actions.ec.europa.eu/about-msca/guidelines-for-inclusion-of-researchers-at-risk 
167 MSCA Green Charter: https://marie-sklodowska-curie-actions.ec.europa.eu/about-
msca/msca-green-charter 

137 

 
 
 
milestones as appropriate. The communication regarding flexibility measures was generally 
timely and supported by REA project officers.  

As an entirely ‘bottom-up’ and idea-driven programme, the MSCA provide great flexibility for 
research creativity in all scientific fields, leading to significant and innovative contributions to 
major societal challenges and Commission priorities, including emerging ones such as the 
EU response to COVID-19. As discussed in DG EAC’s strategic plan for 2020 – 2024, there 
are  already  more  than  100  MSCA  projects  directly  related  to  COVID-19  research  issues 
alone, and the MSCA can support many more projects in the years to come. The COVID-19 
crisis  dramatically  highlighted  the  critical  need  to  provide  researchers  with  the  right  skills, 
including the necessary resilience skills, and facilitate their outreach and networking, not only 
for addressing the immediate health challenges, but also to anticipate the broader societal, 
ecological, and economic implications of the outbreak and enhance Europe’s preparedness 
in the future. 

According to stakeholders, the MSCA is well-focused in terms of thematic areas. The 
research  and  innovation  community  is  highly  aware  of  the  supporting  tools  available  and 
thematic  areas  covered  by  the  MSCA  programme.  In  this  regard,  the  MSCA  Network  of 
National  Contact  Points  (MSCA-Net)  plays  a  crucial  role  in  assisting  researchers  and 
institutions  in  their  respective  countries  with  information,  guidance  and  support  related  to 
participating  in  MSCA  programme.  This  network  consists  of  experts  located  in  each  EU 
Member State and in associated countries. These experts serve as central points of contact 
and provide a range of services, including information and guidance about different funding 
opportunities; proposal preparation and feedback; networking and partner search; as well as 
organisation of information events and workshops. 

According to the interviewees, the support provided by the European Commission is highly 
appreciated  as  participating  organisations  and  researchers  can  take  stock  of  various 
information  channels  concerning  eligibility  criteria,  proposal  preparation  process,  training 
courses available. Interviewees also welcome the responsiveness and support received from 
the National Contact Points (NCPs). However, it was observed that the degree of support 
varies depending on the experience of the NCPs.  

Nevertheless, the organisations, researchers, and NCPs face difficulties when attempting to 
navigate the extensive pool of information that is accessible. According to the interviewees, 
this  information  is  widely  scattered.  Greater  centralisation  would  be  welcome,  especially 
given that numerous changes were made to the MSCA programme in recent years, making 
it difficult to keep track of the evolving rules and requirements.  

The MSCA bottom-up approach remains highly appreciated. Under Horizon Europe, 
the  MSCA  continue  to  operate  in  a  fully  bottom-up  manner.  The  MSCA  are  expected  to 
complement top-down collaborative research activities, notably the EU Missions. Based on 
the position papers on the past, present and the future of research and innovation framework 
programmes, stakeholders indicated that increases in the bottom-up approaches to funding 
are recommended. 

Interview  findings  highlight  the  bottom-up  approach  to  funding  as  being  essential  for  the 
MSCA  programme’s  success  in  fostering  excellent  science.  The  MSCA  are  highly 
appreciated by researchers, allowing them to propose research topics and ideas based on 
their own interest and without a predefined thematic focus. The survey results reflect these 
findings as an overwhelming majority of successful MSCA PF applicants (91%) indicate that 

138 

 
 
the freedom to pursue their specific research agenda responds to their needs to a very large 
and  large  extent.  However,  there  are  some  exceptions  to  this,  for  example,  proposed 
programmes  in  COFUND.  The  new  or  existing  doctoral  or  postdoctoral  programmes  in 
COFUND  can  cover  any  research  discipline  (bottom-up  approach)  but  can  also  focus  on 
specific  disciplines,  notably  when  they  are  based  on  national  or  regional  Research  and 
Innovation Strategies for Smart Specialisation (RIS3 strategies).   

Many  MSCA  projects  focus  on  future  and  emerging  technologies  and  the  most 
promising research areas. Although the MSCA operate in a fully bottom-up manner, the 
projects  that  are  funded  respond  to  a  large  extent  to  the  development  of  future  emerging 
technologies, as evidenced by the Future Emerging Technologies (FET) score trends. The 
FET score captures the extent to which the analysed programme parts in Pillar 1 address 
new or fast-growing research and innovation topics. With a FET score of 7.42, MSCA are 
above  the  overall  Excellent  Science  Pillar  average  (7.25).  The  top  5  corresponding  FET 
topics and the number of corresponding projects are presented in the table below.  

Programme part 

Average FET Score 

MSCA DN 

12.11 

MSCA PF 

6.80 

MSCA SE 

10.35 

MSCA COFUND 

10.79 

Overall MSCA 

7.42 

Top 5 corresponding FET 
topics (and number of 
corresponding projects) 

Open science (296) 
Scientific writing (161) 
Open data (122) 
Deep learning (92) 
Findability (81) 

Open science (2541) 
Open peer review (449) 
Scientific writing (440) 
Open data (342) 
Findability (271) 

Open science (78) 
Open data (30) 
Deep learning (26) 
Findability (25) 
Open peer review (21) 

Open science (27) 
Open data (13) 
Scientific writing (12) 
Citizen science (10) 
Responsible research and 
innovation (9) 

Open science (2970) 
Scientific writing (620) 
Open peer review (536) 
Open data (514) 
Findability (387) 

Table 32. Average FET score in MSCA with 5 corresponding FET topics. Source: HE administrative data, MAG/OpenAlex 
database. 

139 

 
 
The figure below shows the distribution of projects based on the scientific panel in the MSCA 
actions. Most of the MSCA projects fall under the Life Sciences field (23%) and Engineering 
(22%), followed by Social Sciences (19%).  

Figure 22. Projects in MSCA by scientific panel. Source: Compiled by the study team using the eCorda dataset (cut-off date 
June 26,2023) 

1.4.6.  Conclusions and lessons learned168 

Conclusion 1. Business-related skills appear to be the least commonly acquired skills that 
MSCA  fellows  gain  during  their  DN,  SE  and  COFUND  projects,  with  training  on 
entrepreneurship, the commercialisation of research output and intellectual property rights 
being the least common types of training provided. In addition, business plans and product 
validation or market replication are the least common results of the MSCA projects. Although 
patents, prototypes, new products or services are delivered more often, they remain among 
the least common outputs of the MSCA. 

Lesson learned 1.1. In the current EU context, where the number of doctoral candidates and 
postdoctoral researchers is increasing faster than the number of open academic positions, 
the MSCA programme should look for ways to encourage MSCA organisations to offer more 
training on entrepreneurship, commercialisation of research outputs and intellectual property 
rights,  among  other  types  of  training  conducive  to  successful  careers  in  non-academic 
sectors.  Another  argument  for  taking  steps  in  this  direction  is  a  significant  need  for  talent 
faced by private deep tech companies in Europe. The MSCA are particularly well-placed to 
develop  such  talent,  where  a  doctoral  and  postdoctoral  level  of  competences  is  often 
necessary. 

Conclusion 2. A major challenge faced by PF and DN fellows is to understand the precise 
financial  and  working  conditions  of  their  fellowship.  Postdoctoral  fellows  face  obstacles 
related  to  taxation  and salary  differences  between  different  countries;  as  well  as  a  lack  of 
information regarding the fellowship / grant salary, duration, and other arrangements prior to 
the project start. This challenge seems more pronounced for doctoral candidates, who are 
recruited after the MSCA consortium is formed. Issues include the unexpected taxation of 
family  and  mobility  allowance,  the  inability  of  host  organisations  to  accurately  calculate 

168 Please note that the analysis is based on the MSCA monitoring data which was received in 
July 2023. 

140 

 
 
 
 
salaries and sign work contracts without delay, the lack of communication about the fact that 
the fellows cannot be simultaneously employed by other organisations, etc.  

Lesson learned 2.1. In the context of a variety of national taxation laws and organisational 
practices, MSCA beneficiary organisations should be reminded that they are required to take 
concrete steps to communicate the financial and working conditions with fellows beforehand. 
Beneficiary organisations should also be encouraged to step up their efforts to support the 
fellows in obtaining visas or work / residence permits. At the same time, fellows should also 
take  individual  steps  to  inquire  about  national  and  organisational  rules  in  the  organisation 
they intend to join for the fellowship.    

Conclusion 3. The evaluation revealed a positive and accelerating trend for the MSCA to 
develop synergies with EU funds under shared management between the Member  States 
and  the  European  Commission,  namely  the  cohesion  policy  funds  and  the  Recovery  and 
Resilience Facility (RRF). This trend clearly seems conducive to the increased visibility and 
success  of  the  MSCA  Seal  of  Excellence;  but  also  to  the  communication  efforts  from  the 
Commission  to  encourage  Member  States  (and  especially  Widening  countries)  to  seek 
synergies between cohesion policy funds / RRF and R&I Framework Programmes in order 
to reinforce Horizon Europe impacts.   

Lesson learned 3.1. The European Commission should build on this accelerating trend by 
continuing  to  award  the  Seal  of  Excellence  and  up  the  dissemination  efforts  about  the 
examples of cohesion policy funds or RRF used to finance projects that received the Seal of 
Excellence, as well as other (still rather rare) co-funding cases between the MSCA and funds 
under  shared  management.  In  the  future,  for  example,  under  their  cohesion  policy  funds’ 
operational  programmes  Member  States  could  be  encouraged  to  foresee  co-funding 
especially for what is now covered under the MSCA institutional unit contributions under DN 
and  SE  actions  (research,  training  and  networking  costs),  as  well  as  to  finance  research, 
training  and  networking  items  under  COFUND action.  Synergies  between  cohesion  policy 
funds / RRF and MSCA may especially benefit Widening countries which tend to have lower 
resources  and  therefore  can  reap  especially  high  benefits  from  utilising  cohesion  policy 
funds / RRF.      

1.5.  Research Infrastructures (INFRA) 

1.5.1. 

INFRA under Horizon Europe 

Horizon Europe regulation defines ‘research infrastructures’ (RIs) as ‘facilities that provide 
resources  and  services  for  the  research  communities  to  conduct  research  and  foster 
innovation in their fields, including the associated human resources, major equipment or sets 
of  instruments;  knowledge-related  facilities  such  as  collections,  archives  or  scientific  data 
infrastructures; computing systems, communication networks and any other infrastructure of 
a unique nature and open to external users, essential to achieve excellence in R&I; they may, 

141 

 
 
where relevant, be used beyond research, for example for education or public services and 
they may be “single sited”, “virtual” or “distributed”’169. 

The INFRA work programme addresses several needs identified in this field, such as: 

•  The need for open access to RIs, their services and data by European researchers; 

•  The development of new RIs to fill knowledge gaps, support emerging needs and scientific 
breakthroughs, and respond to new challenges – notably in the field of health and in the 
context of the green and digital transitions; 

•  The optimisation and consolidation of existing capacities, hence avoiding fragmentation 

and unnecessary duplications; 

•  Supporting RIs at the forefront of science and technological developments, covering the 
continuum  of  needs  from  the  creation  of  fundamental  knowledge  to  the  deployment  of 
technology170. 

With  an  indicative  budget  of  EUR  2.186  million  for  the  period  2021-2027,  INFRA  is  the 
smallest  programme  part  of  Pillar  1  ‘Excellent  Science’  under  Horizon  Europe.  The  main 
objective of the Research Infrastructures programme under Horizon Europe is ‘to empower 
Europe  through  world-class  and  accessible  research  infrastructures,  as  part  of  an 
integrated European research and technology infrastructures landscape’171. 

The Regulation establishing Horizon Europe indicates four main areas of intervention for this 
programme part172: 

1. 

2. 

3. 

4. 

Consolidating  and  developing 
infrastructures; 

the 

landscape  of  European 

research 

Opening, integrating and interconnecting research infrastructures;  

Fostering the innovation potential of European research infrastructures, as well 
as activities for innovation and training; and 

Reinforcing  European 
cooperation. 

research 

infrastructure  policy  and 

international 

169 Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 2021 
establishing Horizon Europe – the Framework Programme for Research and Innovation, laying 
down its rules for participation and dissemination, and repealing Regulations (EU) No 1290/2013 
and (EU) No 1291/2013. 
170 The 2021-2022 and 2023-2024 Horizon Europe Research Infrastructures work programmes. 
171 Horizon Europe Work Programme 2023-2024, 3. Research Infrastructures, European 
Commission Decision C(2023) 2178 of 31 March 2023, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-call/2023-
2024/wp-3-research-infrastructures_horizon-2023-2024_en.pdf. 
172 Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 2021 
establishing Horizon Europe – the Framework Programme for Research and Innovation, laying 
down its rules for participation and dissemination, and repealing Regulations (EU) No 1290/2013 
and (EU) No 1291/2013. 

142 

 
 
 
By addressing the above areas, the activities carried out under INFRA are expected to foster 
the potential of research infrastructures to support scientific progress and innovation, and to 
enable  open  and  excellent  science  in  accordance  with  the  FAIR  principles,  alongside 
activities relating to Union policies and international cooperation. In addition, INFRA activities 
are expected to provide Europe with world-class sustainable research infrastructures that are 
open and accessible to the best researchers from Europe and beyond, and to encourage the 
use  of  existing  research  infrastructures,  including  those  financed  from  funds  under  the 
Union’s Cohesion Policy173.  

The  ERA  Policy  Agenda  approved  in  2022174  included  a  specific  action  directed  towards 
research infrastructures: Action 8, ‘Strengthen sustainability, accessibility and resilience of 
research infrastructures in the ERA’. This priority action was designed to respond to the new 
challenges  that  are  emerging  as  a  consequence  of  the  growing  maturity  of  the  research 
infrastructure landscape. This action refers to the need to: 

•  Ensure sustainable funding and investments in existing research infrastructures in order 

to maintain the excellence and competitiveness of the ERA in the long term; 

•  Give all researchers and innovators in Europe the same opportunities to access to the 

services provided by European infrastructures;  

• 

Increase the overall impact of the investments made in research infrastructures on the 
economy and society; and 

•  Set priorities in a more targeted way, and focus on specific scientific and political needs, 
taking  into  account  the  existing  capacities  and  services  available,  the  main  scientific 
developments, and the key societal challenges.  

This action involved several activities, among which the following can be cited: 

•  Development of the framework for the next ESFRI Roadmap; 

•  A  more  in-depth  analysis  of  the  RI  landscape  and  the  implementation  of  the  RI 

performance methodology; 

•  The preparation of the third report on the implementation of the ERIC Regulation175 and 

of a permanent monitoring framework for this Regulation; 

•  The exploration of new funding models to address the persisting challenges with regard 

to access to RIs;  

173 Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 2021 
establishing Horizon Europe – the Framework Programme for Research and Innovation, laying 
down its rules for participation and dissemination, and repealing Regulations (EU) No 1290/2013 
and (EU) No 1291/2013. 
174 European Commission, Directorate-General for Research and Innovation, European 
Research Area policy agenda : overview of actions for the period 2022-2024, Publications Office 
of the European Union, 2022, available at: https://data.europa.eu/doi/10.2777/52110. 
175 Taking into account the Expert Group report published in October 2021. 

143 

 
 
 
•  Analysis  of  the  implementation  and  updating  of  the  European  Charter  for  Access  to 

Research Infrastructures. 

These actions are to be implemented in collaboration with the Member States, in particular 
through ESFRI, as well as through engagement with a broad range of R&I stakeholders.  

INFRA  work  programmes  highlight  the  richness  of  the  EU  landscape  of  research 
infrastructures, and point to the need to direct efforts towards optimising and consolidating 
the existing capacities and avoiding unnecessary duplications and fragmentation. INFRA is 
hence expected to contribute to a more effective and well-functioning landscape of research 
infrastructures,  thereby  strengthening  the  European  Research  Area  (ERA)  and  making  it 
more attractive to researchers both within and outside of the EU. 

INFRA work programmes also stress the importance of RIs in the development of knowledge 
and innovation ecosystems or hubs. Their links to industry and business, and the alignment 
of their activities with Smart Specialisation Strategies (S3), are expected to contribute to this 
end and to have an impact on the alignment of priorities at EU level.  

Efforts to support the European Open Science Cloud (EOSC) are included in the INFRA work 
programmes.  The  EOSC  offers  ‘a  horizontal,  pan-European,  inter-operable,  federated 
ecosystem  of  standards,  technologies  and  services’  that  are  expected  to  ‘enable  and 
enhance seamless access to and reliable re-use of research outputs, i.e. data, software and 
other digital objects, included those generated or collected by other research infrastructures, 
supporting the whole research data life-cycle from discovery and mining to analysis, storage 
and management’176. The EOSC will be based on a ‘federated core, enabling access to a 
wider  ecosystem  of  data  and  services,  as  well  as  use  of  integrated  High-Performance 
Computing (HPC), Cloud, data, networks and Artificial Intelligence (AI) resources’177. Specific 
insights on this initiative are included in the EOSC Partnership report within this study. 

One of the traditional elements of the INFRA programme under the Framework Programme 
has been support for the transnational or virtual access (TNA/VA) of researchers to the best 
infrastructures  required  for  their  research.  Support  for  TNA/VA  has  predominantly  been 
science-driven,  with  a  bottom-up  disciplinary  approach.  Under  Horizon  Europe,  several 
changes  have  been  introduced,  most  notably  a  greater  focus  on  societal  challenges  and 
policy priorities – especially those relating to the transition towards a green and digital future, 
and  the  consideration  of  health  as  one  of  the  main  priorities.  Complementarities  with  the 
objectives of Horizon Europe clusters, missions and partnerships under Pillar 2 are therefore 
considered important.  

In addition, support for the development of innovative cutting-edge instrumentation, software 
and methods in collaboration with industry is continued under Horizon Europe. Strengthening 
the skills and competencies of RI staff is another of the key areas included in INFRA under 
Horizon Europe. This includes the scientific, managerial and technical skills of RI personnel, 
but also of the users of pan-EU infrastructures. 

176 Horizon Europe Work Programme 2021-2022 3. Research Infrastructures. (European 
Commission DecisionC(2022)2975 of 10 May 2022), available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-call/2021-
2022/wp-3-research-infrastructures_horizon-2021-2022_en.pdf. 
177 Ibid. 

144 

 
 
 
Synergies with other funding schemes were encouraged under the 21-22 work programme, 
particularly with the ERDF, ESF+, JTF, EMFF, EAFDR and InvestEU, as well as private funds 
and  financial  instruments.  The  ERDF  is considered  especially  relevant,  as  its  investments 
contribute to the development of R&I ecosystems, support the building of R&I capacities, and 
the development of innovative solutions originating from the Framework Programme. 

Table  33  shows  the  five  destinations  included  in  INFRA  work  programmes  under  Horizon 
Europe to date. Some of these destinations present a high degree of stability with respect to 
similar calls under Horizon 2020. This is most notable in the case of INFRADEV calls, which 
aim to support the development of pan-EU infrastructures, and INFRAEOSC, which supports 
the development of the EOSC, its services and governance structure. With respect to other 
destinations,  there  are  significant  changes  compared  with  the  previous  Framework 
Programme. The most notable concern INFRASERV calls, which aim to support the provision 
of  services  around  key  priorities  (and  hence  support  TNA),  and  INFRATECH,  which 
contributes  to  the  development  of  new  scientific  instrumentation,  tools  and  methods. 
Whereas under Horizon 2020, these were covered under one instrument (INFRAIA), which 
also included networking activities, they are now supported via other instruments. 

ACTIVITIES 

DESCRIPTION 

AIMS 

INFRADEV 
(21/22; 
23/24) 

Developing, consolidating and optimising 
the European research infrastructure 
landscape, maintaining global leadership 

INFRAEOSC 
(21/22; 
23/24) 

Enabling an operational, open and FAIR 
EOSC ecosystem 

INFRASERV 
(21/22; 
23/24) 

RI services to support health research, 
accelerate the green and digital 
transformations, and advance frontier 
knowledge  

INFRATECH 
(21/22; 
23/24) 

The next generation of scientific 
instrumentation, tools and methods and 
advanced digital solutions 

To contribute to a strong, excellent and 
impactful European Research Area by 
reinforcing RI capacities in Europe and 
their role at global level, as well as policy-
making in this field. 

To deliver a web of FAIR data and 
services for science; a trusted virtual 
environment supporting Open Science, 
based on key horizontal core functions, 
with corresponding e-infrastructures and 
service layers accessible to researchers 
across disciplines throughout Europe. 

To provide integrated RI services 
enabling R&I to address major societal 
challenges, including health challenges, 
the green and digital transformations and 
resilience to crises, as well as to support 
curiosity-driven research and the 
advancement of frontier knowledge in 
broad scientific domains. 

To enable new discoveries and maintain 
Europe’s RIs at the highest level of 
excellence, while paving the way for 
innovative solutions to societal 
challenges and new industrial 
applications, products and services. 

INFRANET 
(21/22; 
23/24) 

Enabling collaboration without 
boundaries 

To provide high-bandwidth networks and 
network services to interconnect 
researchers, data and computing 

145 

 
 
 
 
 
 
 
ACTIVITIES 

DESCRIPTION 

AIMS 

resources in a non-discriminatory way, 
regardless of the location of users and 
resources, to allow scientists to conduct 
excellent research. 

Table 33. Destinations in Horizon Europe Research Infrastructures. Source: compiled by the study team, based on the 
Horizon Europe Research Infrastructures work programmes 21-22 and 23-24. 

Through its various activities, INFRA aims achieve a series of outputs that can be analysed 
from three perspectives, namely:  

1. 

From  the  perspective  of  support  for  the  consolidation  of  the  research 
infrastructure landscape, INFRA aims to achieve: 

−  Better coordination and alignment of EU and national priorities for RIs; 

− 

Increased alignment and contribution to policy priorities and societal challenges; 

−  Synergies and complementarities between new and existing research infrastructures, 

including technology infrastructures and infrastructures financed by the ERDF. 

2. 

From the perspective of the development of and access to world-class RIs, 
INFRA aims to: 

−  Develop new pan-European research infrastructures; 

−  Provide  trans-national  and  virtual  access  to  key  research  infrastructures  (for  both 

academia and industry); 

−  Develop new innovative, customised, harmonised and efficient services and access 
opportunities  available  to  the  research  community,  helping  to  better  tackle  specific 
scientific and societal challenges and evidence-based policy making; 

−  Strengthen human capital for research infrastructures (skills, competencies, education 

and employment opportunities); 

−  Reduce  environmental  (including  climate-related)  impacts,  as  well  as  optimising 

resource and energy consumption in RIs. 

3. 

INFRA also aims to achieve a series of outputs relating to access to FAIR data, 
through the implementation of and the support for the European Open Science 
Cloud  (EOSC)  –  a  horizontal,  pan-European,  inter-operable,  federated 
ecosystem  of  standards,  technologies  and  services,  along  with  rules  of 
engagement.  In  this  regard,  INFRA  aims  to  support  the  deployment  of 
infrastructure,  tools  and  services  that  allow  researchers  to  properly  develop, 

146 

 
 
describe  with  proper  metadata,  version,  archive,  share  and  re-use  research 
software and data in line with FAIR principles. 

INFRA aims to achieve a series of  results in the medium term.  From the perspective of 
supporting the consolidation of the landscape, of particular note are its contribution to a more 
effective,  interlinked  and  well-functioning  research  infrastructure  landscape;  achieving  the 
solid  engagement  of  Member  States/associated  countries  in  pan-European  research 
infrastructures; and achieving increased visibility as well as impact and evidence for future 
policy- and decision-making on research infrastructures and ERICs. 

From  the  standpoint  of  supporting  world-class  RIs  and  ecosystems,  the  INFRA  work 
programmes stress several expected results:  

1. 

2. 

3. 

The development of innovative cutting-edge scientific instrumentation, software 
and  methods,  and  the  co-development  with  industrial  actors  of  advanced  RI 
technologies and technology transfer;  

The increased integration and collaboration of RIs with local, regional and global 
innovation and decision support systems, and the promotion of entrepreneurial 
culture;  

An increase in the technological level of industries through the co-development 
of advanced technologies and the creation of potential new markets. 

With regard to the EOSC, the work programmes expect to achieve a better management of 
and  seamless  access  to  the  continuous  flow  of  data  collected  or  produced  by  RIs  across 
different  scientific  domains  and  on  the  basis  of  FAIR  principles,  and  to  improve  trust  in 
science  through  the  increased  FAIRness,  openness  and  quality  of  scientific  research  in 
Europe. 

In the long term, INFRA is expected to contribute to the following impacts: the development 
of world-class and accessible RIs in an integrated, inter-operable and effective ecosystem of 
RIs and TIs embedded into regional, national, European and global R&I ecosystems.  

In addition, it also aims to contribute to the strengthening of: 

•  Local and regional socio-economic development; 

•  The attractiveness of the ERA to researchers and talents; 

•  Society’s long-term and consistent problem-solving capacity; 

•  The scientific performance and competitiveness of European research infrastructures; 

•  The alignment of RI capacities to address research challenges, EU policy priorities and 

industry needs; 

•  Global competitiveness and technological excellence;  

•  Global connectivity and Europe’s international cooperation policy objectives in strategic 

areas and in terms of increased capacities. 

147 

 
 
 
Figure 23. Intervention logic of Research Infrastructures – Horizon Europe. Source: compiled by the authors, based on Horizon Europe INFRA 2021-2024 work programmes.

148 

 
 
 
1.5.2.  Effectiveness  

Findings  relating  to  the  effectiveness  of  INFRA  are  discussed  below,  along  the  following 
dimensions:  

•  Consolidating and developing the RI landscape 

•  Opening, integrating and interconnecting RIs 

•  Contribution to the EOSC 

• 

International cooperation 

• 

Industry collaboration  

•  Research careers, and the training of researchers 

1.5.2.1. 

Consolidating and developing the RI landscape  

An important contribution made by INFRA to the consolidation and development of the RI 
landscape  comes  through  its  support  for  the  European  Strategy  Forum  for  Research 
Infrastructures (ESFRI). The ESFRI is responsible for developing a roadmap that outlines the 
essential research infrastructures needed in Europe over the next 10-20 years. This roadmap 
aims to help avoid over-fragmentation in the field of RIs. The ESFRI roadmap distinguishes 
between two categories: ESFRI projects and ESFRI landmarks. ESFRI projects are research 
infrastructures  in  the  preparation  phase.  These  are  expected  to  advance  to  the 
implementation  phase  within  a  10-year  period.  ESFRI  landmarks  are  RIs  that  are  at  an 
advanced stage of construction, and have a clear schedule for operation. ESFRI published 
its first roadmap in 2006, with updates published in 2008, 2010, 2016, 2018 and 2021. The 
2021 roadmap identified a total of 22 projects and 41 landmarks178.  

The  development  of  RIs  on  the  ESFRI  roadmap  is  mainly  funded  through  INFRADEV, 
particularly in the case of Design Studies, Preparatory Phase projects and Implementation 
Phase projects. Design Studies are aimed at discovering new ESFRI projects; Preparatory 
Phase projects are aimed at bringing ESFRI projects to legal, financial and technical maturity 
for implementation; and Implementation Phase projects ensure the long-term sustainability 
of  the  RI  (for  example,  funding  can  be  provided  for  enlarging  membership,  establishing 
governance structures, etc.). The table below provides an overview of the number of grants 
and net EU contribution per project type under both Horizon 2020 and Horizon Europe (as of 
August  2023).  Horizon  Europe  has  comparatively  more  Preparatory  Phase  projects  and 
fewer  Implementation  Phase  projects  than  its  predecessor.  The  average  funding  per 
Implementation Phase project is also lower under Horizon Europe (EUR 2.6 million versus 
4.2 million).  

178 Available at: https://roadmap2021.esfri.eu/projects-and-landmarks/view-the-table/. 

149 

 
 
 
 
Horizon Europe179 

Horizon 2020 

Number of 
grants 

Net EU contribution 
(in EUR million) 

Number of 
grants 

Net EU contribution 
(in EUR million) 

Design Studies180 

Preparatory Phase 
projects181 

Implementation 
Phase projects182 

TOTAL 

8 

10 

4 

22 

22 

30 

5 

57 

27 

20 

40 

87 

77 

59 

228 

364 

Table 34. Overview of contributions to the development of ESFRI projects in Horizon 2020 and Horizon Europe. Source: 
Horizon Dashboard, accessed July 2023 (for Horizon Europe data), and the Evaluation study on Excellent Science under 
Horizon 2020183, also based on Horizon Dashboard data. 

Interviewees  stress  the  importance  of  substantive  individual  support  to  ESRFI  and  other 
world-class  RIs  throughout  their  entire  lifecycle,  to  ensure  ESFRI  projects  and  landmarks 
remain at the state-of-the-art. Implementation Phase projects are important in this regard, as 
they provide funding beyond the conceptual and preparatory phases. Interviewees also point 
to  the  importance  of  the  INFRATECH  destination,  which  provides  funding  for  the  next 
generation of scientific instrumentation. This is the destination that has so far received the 
most funding within this programme part (40%, or EUR 161 million184); however, interviewees 
indicate that competition within INFRATECH is very high, with many high-quality proposals 
not being funded.  

The survey185 carried out in context of this study, as well as some interviews, indicate that 
there is a need for more funding for the development of RIs: only 49% (or 80 out of 165) of 
INFRA beneficiaries and 26% (or 24 out of 92) of unsuccessful applicants to INFRA indicated 
that funding opportunities for the development of research infrastructures were (to a ‘large’ 
or ‘very large’ extent) sufficient (see Figure 24). Interviewees also noted that the number of 
RIs has increased, yet the funding provided through INFRA has not. In fact, the share of the 
net  EU  contribution  to  INFRA  under  the  Framework  Programme  has  decreased  in 
comparison to Horizon 2020 (2.4% versus 3.8%)186. 

179 As of August 2023, this includes only projects granted under the work programme 2021-2022. 
180 HORIZON-INFRA-2022-DEV-01-01: Research infrastructure concept development. 
181 HORIZON-INFRA-2021-DEV-02-01: Preparatory phase of new ESFRI research infrastructure 
projects. 
182 HORIZON-INFRA-2021-DEV-02-02: Consolidation of the research infrastructure landscape – 
Individual support for ESFRI projects. 
183 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European framework programmes 
for research and innovation for excellent science: Horizon 2020: annexes, Luxembourg: 
Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383. 
184 Horizon Dashboard, retrieved August 2023. 
185 Survey of Horizon Europe unsuccessful and successful applicants, conducted May-July 2023. 
186 Horizon Dashboard, retrieved August 2023. 

150 

 
 
 
Figure 24. INFRA survey responses to the question ‘In your opinion, to what extent does Horizon Europe provide sufficient 
funding opportunities in the following areas?’ – Grants for the development of research infrastructures. Source: survey of 
Horizon Europe beneficiaries, conducted May-July 2023. 

Moreover, the funding provided by the INFRA programme also plays an important role in the 
development  and  operation  of  European  Research  Infrastructure  Consortia  (ERICs).  The 
ERIC regulation facilitates new European RIs with members from different countries, and is 
regarded as a ‘highly successful mechanism to initiate and integrate many RIs at European 
level,  aligning  national  investments  and  research  priorities,  pooling  resources  and 
expertise’187.  A  significant  portion  (60%)  of  the  ESFRI  landmarks  have  been  set  up  as 
ERICs188. Since the creation of the ERIC network in 2009, a total of 26 ERICs have been 
established189, and at least another seven ERIC applications are in the pipeline. On average, 
each ERIC has 14 members, with some ERICs spanning almost the entire EU. Furthermore, 
every Member State is a member of at least one ERIC, and nine  associated countries are 
also members of an ERIC.  

ERICs  are,  in  principle,  funded  through  (in  cash  or  in-kind)  contributions  their  member 
countries;  however,  the  funding  secured  by  member  countries  typically  covers  only 
coordination, and often is not sufficient to fund the joint activities set out in their statutes (e.g. 
coordination of R&I agendas). Nor does it sufficiently cover investments to keep ERICs at the 
state-of-the-art190.  The  funding  provided  by  the  INFRA  programme  part therefore plays  an 
important role in the development and operation of ERICs.  

1.5.2.2. 

Opening up, integrating and interconnecting RIs  

An important objective of the INFRA programme part is to foster collaboration between RIs 
and to ensure that RIs are accessible to researchers across Europe. Indeed, the three most 
important  results  indicated  by  INFRA  survey  respondents  are:  strengthening  relationships 
with leading partners in Europe; increasing international visibility through collaboration with 
leading  global  partners;  and  improving  access  to  physical  and  knowledge  infrastructures, 
facilities, and data resources (see Figure 25).  

187 Ibid, pp. 4. 
188 Report from the Commission to the Council and the European Parliament, 2023, Third report 
on the application of Council Regulation (EC) No 723/2009 of 25 June 2009 on the Community 
legal framework for a European Research Infrastructure Consortium (ERIC), available at: 
https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=COM:2023:488:FIN. 
189 As of August 2023. 
190 Ibid. 

151 

 
 
 
Figure 25. INFRA survey responses to the question ‘To what extent, if at all, has your Horizon Europe project achieved/is 
likely to achieve the following results?’ Source: survey of Horizon Europe beneficiaries, conducted May-July 2023. 

Under  Horizon  2020,  collaboration  between  Ris  and  the  provision  of  access  (commonly 
referred to as ‘Transnational Access/Virtual Access’ or TNA/VA) were funded mainly through 
Integrating Activities (INFRAIA) grants. These grants integrated RIs and users in a bottom-
up  manner;  typically,  consortia  were  formed  among  similar  scientific  instruments  and 
disciplines. INFRAIA grants funded a combination of 1) networking activities, 2) TNA/VA, and 
3) joint research. Under Horizon Europe, INFRAIA grants were discontinued, and the above 
functions  were  split  between  INFRASERV  which  funds  TNA/VA,  and  INFRATECH,  which 
funds joint research activities on the next generation of scientific instrumentation, tools and 
methods. In addition, a more top-down approach has been implemented, under which RIs 
are requested to create services for specific societal challenges (e.g. tackling cancer). These 
changes have had several effects on the integration of RIs and are discussed extensively in 
Case Study 6 of this study. The most important effects of these changes are discussed below.  

One key aspect that was raised many times by interviewees was the need for a longer-term 
vision regarding the integration of Ris, as well as a certain degree of continuity in order to 
achieve an impact. The new approach taken under Horizon Europe requires the formation of 
new and larger consortia. This has had positive effects on opening up towards new partners 
and most likely new TNA/VA users; however, it takes time to create sustainable networks of 
RIs and a coherent and sustainable service offering. The duration of INFRASERV projects 
(typically  3-4  years)  is  considered  by  some  interviewees  to  be  too  short  to  achieve  the 
intended impact.  

In addition to the need for continuity, concerns about the ‘fragmentation of funding’ due to the 
larger consortia for TNA/VA projects funded under INFRASERV were raised multiple times 
by interviewees and in several position papers191. These concerns indicate that this situation 
could limit the impact of projects, the level of access provided, and the ability of projects to 
achieve the ambitious objectives set out in the call texts. Some interviewees indicated that 
the  impact  of  the  programme  part  might  be  higher  if  Europe  were  to  focus  funding  more 
towards world-class pan-EU RIs, instead of distributing funding across a very broad network 
of smaller national facilities across Europe. On the other hand, national facilities point to the 
importance  of  EU  funding  to  the  continued  functioning  of  their  TNA/VA  activities  for 

191 e.g. the position papers by Helmholtz, EARTO, CNR and Masaryk University (2023) 
submitted to the ‘Public consultation on the past, present and future of the European Research & 
Innovation Framework Programmes 2014-2027’, available at: 
https ://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-Horizon-Europe-
interim-evaluation/public-consultation_en. 

152 

 
 
 
researchers  from  abroad.  A  clear  vision  in  this  regard  would  be  welcomed  by  the  RI 
community.  

The full impact of the new approach over the coming years remains to be seen. In Horizon 
2020, a total of 16,165 unique researchers received access to RIs as a result of the INFRA 
programme part192. Corresponding figures for Horizon Europe are yet to be reported by the 
projects, and are thus not available at the time of writing this report.  

In Case Study 5 of the present study (Analysis of Transnational and Virtual Access support), 
a  more  in-depth  analysis  is  presented  regarding  different  possible  models  for  funding 
TNA/VA. In addition to the INFRAIA model and the INFRASERV model, other options are 
discussed including a more centralised model in which the European Commission would be 
responsible for organising TNA/VA calls. Interviewees indicated, however, that a centralised 
call might not respond well to the broad variety of users’ needs across scientific disciplines. 
Moreover, RIs would still need to be actively involved in building user communities, creating 
new services and training users. Another hypothetical model considers a more important role 
for  ERICs  in  organising  TNA/VA,  leveraging  their  existing  coordination  capacities.  Some 
interviewees,  however,  expressed  concerns  regarding  the  impacts  of  such  a  move  on 
openness  towards  new  RIs  and  facilities.  Further,  more  distributed  support  for  TNA/VA  is 
likely to increase coherence with other programme parts, but would probably limit the number 
of  users.  Lastly,  the  option  of  a  ‘cascade  funding’  model  is  discussed  as  a  potential 
mechanism to reduce the administrative burden associated with TNA/VA; however, INFRA 
participants are generally not well acquainted with cascade funding, and would require clear 
guidelines on how to implement cascade calls.  

While some of the models above offer interesting advantages that should be further explored, 
it is important to stress that any change to the current system will come with transition costs, 
which should be weighed very carefully against the potential benefits of a new system. For a 
more in-depth analysis of the advantages and disadvantages of the different (hypothetical) 
models, please refer to Case Study 5.  

1.5.2.3. 

Contribution to the EOSC 

Furthermore, the INFRA programme part plays an important role in the development of the 
EOSC. About one-fifth of the net EU contribution under INFRA is directed towards this goal. 
Several  interviewees  stressed  the  importance  of  the  EOSC  with  regard  to  Open  Science. 
However, there are also indications that funding could be directed more effectively towards 
this  goal.  Interviewees  noted  that  there  is  a  lack  of  synchronisation  between  the  many 
different projects. This is expected to lead to duplication of efforts and unexploited synergies. 
The main issue, according to some interviewees, is a lack of time and funding to synchronise 
between projects.  

Some interviewees also expressed regret at the discontinuation of dedicated funding for the 
science clusters193, which brought together the ESFRI and other world-class RIs within the 
same scientific discipline to work towards the creation of FAIR data and eventual integration 

192 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European framework programmes 
for research and innovation for excellent science: Horizon 2020: annexes, Luxembourg: 
Publications Office of the European Union, pp. 848-858, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383. 
193 EOSC-Life, ENVRI-FAIR, PaNOSC, ESCAPE, SSHOC, funded under INFRAEOSC-04-2018, 
available at: https://ec.europa.eu/info/funding-
tenders/opportunities/portal/screen/opportunities/topic-details/infraeosc-04-2018. 

153 

 
 
into  the  EOSC.  Instead,  under Horizon  Europe,  there  will  be  ‘cluster-of-clusters’194  project 
that fosters integration between the various clusters. Interviewees considered that this may 
have  come  too  early,  and  that  more  dedicated  funding  for  each  cluster  might  have  been 
beneficial because even within a discipline, datasets may be strongly diverse, and integration 
may be challenging. The evaluation study on Excellent Science under Horizon 2020195, and 
specifically  the  Case  Study  on  the  creation  of  excellent  services,  similarly  found  that 
‘additional budget would be necessary to make all data within the clusters interoperable and 
easily  accessible’196.  The  science  clusters  were  considered  an  important  platform  for 
collaboration between RIs within similar disciplines.  

1.5.2.4. 

International cooperation 

International cooperation between RIs is essential for advancing scientific knowledge, solving 
global  challenges,  and  maximising  the  impact  of  research  efforts.  Under  Horizon  Europe, 
associated countries represent 6% of participations, and third countries represent a further 
13%. The latter share is higher than under Horizon 2020; however, it should be noted that 
85%  of  participations  by  third  countries  were  from  the  United  Kingdom  and  Switzerland  – 
which were, respectively, a Member State and an associated country under Horizon 2020. 
Other  third  countries  with  a  high  number  of  participations  are  the  United  States  (eight,  or 
0.4% of all Horizon Europe INFRA participations) and South Africa (five, or 0.3%). Among 
associated countries, the countries with the highest number of participations are Norway (67, 
or 3.8% of all Horizon Europe INFRA participations), Israel (eight, or 0.7%), Serbia (seven, 
or 0.4%) and Turkey (six, or 0.4%). 

194 HORIZON-INFRA-2023-EOSC-01-01 – Build on the science cluster approach to ensure the 
uptake of EOSC by research infrastructures and research communities. Funded under: 
https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/topic-
details/horizon-infra-2023-eosc-01-01 
195 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European framework programmes 
for research and innovation for excellent science: Horizon 2020: annexes, Luxembourg: 
Publications Office of the European Union, pp. 848-858, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383. 
196 Ibid, pp. 838 of Annexes (Annex 6.8). 

154 

 
 
Figure 26. Overview of participation under Horizon Europe and Horizon 2020, by country group. Source: Horizon 
Dashboard, retrieved August 2023. 

Interviewees and survey findings confirm the importance of international cooperation: 59.1% 
of the INFRA beneficiaries surveyed (88 out of 149) said that collaboration with international 
(non-EU) partners is important to a ‘large’ or ‘very large’ extent for the uptake of their research 
findings/innovation. On the other hand, the survey indicates that there is still a need to fund 
collaboration  with  international  partners  (non-EU  countries):  only  25.0%  of  INFRA 
beneficiaries (41 out of 164) agreed to a ‘large’ or ‘very large’ extent that there is sufficient 
funding for international collaboration. In addition, only 26.2% agreed (to a large or very large 
extent) that there is sufficient funding for pre-normative research allowing the development 
of international standards.  

The  Horizon  Europe  INFRA  work  programmes  include  specific  calls  for  international 
collaboration  in  the  field  of  infrastructures,  most  notably  those  targeting  cooperation  on 
climate  change  in  Africa  (INFRA-2021-DEV-01-02)  and  bilateral  cooperation  with  Latin 
America (INFRA-2023-DEV-01-06). Interviews confirm the relevance of this type of call, but 
question  the  long-term  sustainability  of  such  cooperation  beyond  project  lifetimes.  Some 
interviewees indicated that a longer-term strategy on international cooperation could boost 
its effectiveness.  

Another key policy instrument for boosting international cooperation is the ERIC regulation, 
introduced  above.  ERICs  have  the  potential  to  develop  globally  integrated  RI  activities  by 
facilitating third country membership in Ris; however, this needs to be further strengthened, 
as indicated in the third report on the ERIC regulation: ‘ERICs have the potential to become 
the legal instrument of choice for further developing globally integrated RI activities, provided 
that the participation of third countries in the ERICs is properly facilitated, to overcome the 
current legal and operational obstacles’197. One example of a good practice is JIV-ERIC198, 
which  facilitates  the  observation  of  black  holes  by  integrating  together  a  set  of  radio 
astronomical observatories across Africa, Europe and Asia.  

197 Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=COM:2023:488:FIN. 
198 JVI stands for Joint Institute for Very Long Baseline Interferometry, https://jive.eu/jiv-eric.  

155 

 
 
 
1.5.2.5. 

Industry collaboration 

Interviewees indicated that the connection between RIs and the private sector/industry could 
be further fostered to increase the impact of RIs and enhance their financial sustainability. 
Private for-profit entities represent only 9% of participations in INFRA (see Figure 27). This 
is  much  lower  than  across  Horizon  Europe  as  a  whole,  in  which  they  represent  30%  of 
participations. Furthermore, the share of TNA/VA users from the private sector was only 3% 
in Horizon 2020 (figures on TNA/VA users under Horizon Europe are not yet available). At 
the  same  time,  the  survey  indicates  that  funding  opportunities  for  academia-industry 
collaboration  are  sufficient  for  the  majority  of  INFRA  beneficiaries:  54%  of  INFRA 
beneficiaries  (89  out  of  165)  agreed  to  a  ‘very  large’  or ‘large’  extent  that  Horizon  Europe 
provides enough funding in this regard.  

Figure 27. Participation in INFRA by organisation type under Horizon Europe and Horizon 2020. Source: Horizon 
Dashboard, retrieved September 2023. 

One mechanism to foster RI-industry collaboration could be strengthening the participation 
of RIs in other parts of the Framework Programme beyond INFRA, in which industry is more 
active (see also the Coherence section).  

In  addition,  a  better  connection  between  research  and  technology  infrastructures  could 
increase the impact of RIs and boost industrial collaboration. The Council Conclusions on the 
ERA state that the further development of RIs is a priority action, including to ‘improve their 
connection  and  interaction  with  technology  infrastructures  and  industry  to  increase  their 
impact’.  Interviewees  also  indicated  that  an  important  aspect  in  reaching  industry  is 
collaboration with research and technology organisations (RTOs) such as CEA, Fraunhofer 
or VTT, and the technology infrastructures within which they operate, as these are known 
and  trusted  by  industry.  Several  position  papers  submitted  to  the  OPC199  call  for  a  better 
linkage  between  research  and  technology  infrastructures:  VTT  indicates  that  increased 
exchange and collaboration would ‘support the whole development chain, create spill-over 
effects  and  serve  the  industrial  ecosystems  needs  for  innovation  and  sustainability’. 
Furthermore, France Universités states that ‘the separation between research infrastructures 
and  technological  infrastructures  is  counter-productive,  insofar  as  there  is  a  continuum  of 
expertise and needs between them both. Research infrastructures should be made to meet 
the  needs  of  blue-sky  research  and  be  simultaneously  equipped  to  satisfy  the  needs  of 
specific sectors and actors further downstream in the innovation process.’ 

199 Public consultation on the past, present and future of the European Research & Innovation 
Framework Programmes 2014-2027, available at: https://ec.europa.eu/info/law/better-
regulation/have-your-say/initiatives/13460-Horizon-Europe-interim-evaluation/public-
consultation_en.  

156 

 
 
 
Lastly, interviewees pointed to the need for longer time horizons to better enable collaboration 
with industry. To this end, strategic roadmaps and a better view of future European funding 
possibilities would be helpful.  

1.5.2.6. 

Research careers and the training of researchers 

INFRA  also  contributes  to  the  training  of  those  researchers  who  use  RIs.  Notably, 
researchers who receive access to RIs via TNA/VA are trained on how to use RI services. 
One  of  the  top  five  outputs  indicated  by  the  INFRA  beneficiaries  surveyed  is  training 
programmes: 52% of the INFRA beneficiaries surveyed (86 out of 164) indicated that they 
had produced training programmes. Moreover, a large majority – 74% (118 out of 159) said 
that their project improves (or is likely to improve) the skills, knowledge and competences of 
researchers  to  a  ‘large’  or  ‘very  large’  extent.  Meanwhile,  53%  of  INFRA  beneficiaries 
surveyed  (83  out  of  159)  said  their  project  improves  (or  is  likely  to  improve)  the  career 
prospects of researchers to a ‘large’ or ‘very large’ extent.  

In addition, preliminary evidence for KIP2 ‘Strengthening human capital in R&I’ indicates that 
Horizon Europe is so far supporting more than 5,000 researchers through INFRA. Of these 
researchers, 30% are involved in upskilling activities (training, mentoring/coaching, mobility 
and  access  to  R&I  infrastructures)  in  projects  funded  by  the  programme  (KIP2).  Upskilled 
researchers are those researchers who are in the earlier stages of their research careers (i.e. 
first-stage researchers and recognised researchers).  

1.5.3.  Coherence 

1.5.3.1. 

Coherence within the Framework Programme 

According  to  interviewees,  the  participation  of  RIs  in  other  parts  of  the  Framework 
Programme could still be significantly increased. Interviewees pointed to the large number 
and diversity of calls within Horizon Europe, which makes it difficult to identify the relevant 
topics in those programme parts with which they are less well acquainted. More guidance on 
exploring relevant calls would be helpful. Several interviewees also mentioned that it would 
be beneficial for relevant RIs to be mentioned in the call texts. This could increase the visibility 
and use of RIs in other programme parts.  

One of the objectives of the new challenge-oriented approach used for certain calls200 is to 
increase  alignment  with  other  Pillars,  specifically  ‘Horizon  Europe  clusters,  missions  and 
partnerships  in  Pillar  2’201.  However,  interviewees  indicated  that  such  alignment  could  be 
further strengthened. One example of good practice in this regard relates to the EU mission 
on  cancer,  in  which  there  has  been  collaboration  between  several  projects  from  different 
programme parts, including INFRA and the Health cluster. This collaboration was enabled by 
the common participation of certain key partners in the various projects.  

According to the network analysis carried out for this study (see Annex 6),  compared with 
some parts of Horizon Europe, INFRA is relatively well connected with other programme parts 
– especially to some of the Clusters in Pillar 2. The network analysis looks at the number of 
shared researchers between programme parts and shows that INFRA is most connected with 
Cluster  4  ‘Digital,  Industry  and  Space’,  Cluster  6  ‘Food,  Bioeconomy,  Natural  Resources, 
Agriculture &  Environment’, Cluster 1 ‘Health’,  as  well  as  with  WIDERA.  These  pairs  (e.g. 

200 Notably, some topics in INFRASERV and INFRAEOSC, e.g. HORIZON-INFRA-2021-SERV-
01-01 – Research infrastructures services to support research addressing cancer. For a 
complete overview, see Case Study 6 of this study. 
201 Research Infrastructures Work Programme 2021-2022, p. 8. 

157 

 
 
INFRA/Cluster  4)  are  in  the  top  20  most-connected  pairs  of  programme  parts  in  Horizon 
Europe.  The  number  of  shared  researchers  are,  respectively,  271,  261,  138  and  135  – 
compared with a total of 2,936 researchers in INFRA (in other word, 9%, 9%, 5% and 5% of 
all  INFRA  researchers,  respectively,  also  participate  in Cluster 4,  Cluster 6,  Cluster  1  and 
WIDERA).  The  survey  also  indicates  that  beyond  the  programme  part  itself,  INFRA 
participants  collaborate  most  with  the  Clusters,  especially  Cluster  6  ‘Food,  Bioeconomy, 
Natural Resources, Agriculture and Environment’ (9.1%, or 16 out of 175, indicate this is one 
of three most important programmes for collaboration202), and Cluster 1 ‘Health’ (4.6%, or 8 
out of 175). 

According to an internal mapping of synergies conducted by DG RTD in Spring 2023, there 
are also synergies between the MSCA programme part and INFRA: the MSCA allow research 
infrastructures  to  meet  the  training  needs  of  their  researchers  and  other  staff,  and  also 
contribute to their career development. Moreover, the same mapping identified synergies with 
the Next Generation Innovation Talents Scheme, which offers internships to (among others) 
PhD candidates from the Horizon Europe INFRA programme part. 

Furthermore, an analysis of ERIC participation in Horizon Europe shows that, beyond INFRA 
(which constitutes 73% of ERIC participations), the most popular programme parts are the 
Pillar  2  Clusters;  again,  especially  Cluster  6  and  Cluster  1  (see  Table  35).  In  total,  seven 
ERICs have participated in a Cluster project.  

Pillar 

Thematic priority 

Participations 

Of which 
coordination 

Excellent Science 

Global Challenges and 
European Industrial 
Competitiveness 

Marie-Skłodowska-Curie 
Actions (MSCA) 

Research Infrastructures 
(INFRA) 

Cluster 1: Health 

Cluster 4: Digital, Industry and 
Space 

Cluster 5: Climate, Energy 
and Mobility 

Cluster 6: Food, Bioeconomy 
Natural Resources, 
Agriculture and Environment 

2 

82 

11 

1 

2 

13 

1 

6 

1 

2 

202 The full survey question is ‘Are there any activities planned in your project that are 
implemented in collaboration with projects funded under other Horizon Europe programmes or 
clusters (this could include mutual conferences, joint dissemination activities, workshops, joint 
publications, etc.)? If yes, please indicate the three most important programmes.’ 

158 

 
 
 
 
Pillar 

Thematic priority 

Participations 

Of which 
coordination 

Widening Participation and 
Strengthening the ERA 

Reforming and enhancing the 
European R&I System 

2 

Horizon Europe  

All thematic priorities  

113 

10 

Table 35. Participation of ERICs under Horizon Europe. Source: Horizon Dashboard, retrieved July 2023. 

When  considering  coherence  between  Horizon  Europe  and  the  previous  Framework 
Programmes, INFRA is found to have a very high degree of continuity. Out of a total of 2,936 
researchers  participating  in  a  Horizon  Europe  INFRA  project,  644  (22%)  had  previously 
published  in  a  Horizon  2020  INFRA  project  (see  Annex  6  for  more  detailed  results  of  the 
network analysis). Moreover, 518 researchers who published in a FP7 INFRA project went 
on  to  participate  in  a  Horizon  Europe  INFRA  project  (17%  of  all  researchers  in  Horizon 
Europe INFRA). The survey confirms this high degree of continuity between the Framework 
Programmes, 42.4% of INFRA beneficiaries (75 out of 177) indicating that their project is a 
continuation of a research project under Horizon 2020, and 17.5% 31 out of 177) indicating 
that their project is a continuation of a project under FP7. 

1.5.3.2. 

Coherence with other programmes 

According  to  interviews  and  desk  research,  INFRA  is  highly  complementary  to  other  R&I 
funding programmes at European, national and regional levels, and provides a clear added 
value (see Section 1.5.4 below). The types of activities funded by INFRA are typically not 
funded by other (European or national/regional) sources of funding. Member States tend to 
focus their contributions on national RIs and typically do not fund the access to RIs for foreign 
researchers.  Funding  at  EU  level  is  therefore  an  important  way  to  boost  collaboration 
between RIs, enable pan-EU RIs and fund transnational access to RIs.  

Whereas there is a very high degree of continuity  from one Framework Programme to the 
next,  INFRA  projects  are  somewhat  standalone  in  nature,  compared  with  other  funding 
programmes. Only 6.2% of INFRA beneficiaries (11 out of 177) indicated that their research 
project  is  a  continuation  of  a  project  funded  under  national/regional  R&D  public  funding 
programmes,  and  only  2.3%  (4  out  of  177)  indicated  it  was  a  continuation  of  another 
European research project (e.g. one funded under the ERDF, ESF, etc.). Similar results are 
found in an analysis of administrative data based on the semantic textual similarity method203. 
This analysis shows that 10 out of 81 INFRA projects under Horizon Europe (12%) received 
complementary funding, amounting to a total of EUR 44.3 million.  

203 This complementary funding analysis was performed by comparing the administrative data of 
Horizon Europe projects (participant PIC numbers and project abstracts) with administrative data 
from other EU funding programmes (particularly CREA2027, DIGITAL, EMFAF, ERASMUS2027, 
ESF, INNOFUND, ISF, LIFE2027, SMP). When PIC numbers in Horizon Europe matched those 
in the other EU funding programmes, project abstracts were compared to identify project 
similarities (using the semantic textual similarity method). Where a signed/ongoing Horizon 
Europe project was found to match a signed/ongoing project in one of the other EU funding 
programmes, these projects were flagged as receiving complementary funding from other EU 
sources. 

159 

 
 
 
1.5.4.  EU added value 

The INFRA programme part provides a significant added value, particularly by enabling the 
development of world-class pan-EU RIs, reducing the level of fragmentation and duplication 
in the RI landscape, fostering international cooperation between RIs, and enabling access to 
RIs  by  researchers  from  across  Europe  and  beyond.  INFRA  beneficiaries  indicated  that, 
compared  with  national  and/or  regional  funding,  INFRA  provides  more  access  to  RIs, 
equipment and/or databases; brings together a more diverse set of partners; and provides 
more international mobility opportunities for researchers (see Figure 28). Only 7.8% of INFRA 
beneficiaries (11 out of 141) indicated that Horizon Europe provides no additional benefits in 
comparison to national/regional support.  

Figure 28. INFRA survey responses to the question ‘Would you agree or disagree that, compared to the research funding 
available to you at national and/or regional level, Horizon Europe?’ Source: survey of Horizon Europe beneficiaries, 
conducted May-July 2023. 

A  large  majority  of  unsuccessful  applicants  who  were  surveyed  (78.9%,  or  60  out  of  76) 
indicated that they did not apply to alternative sources of funding in order to implement the 
activities proposed in their INFRA application. Of those unsuccessful applicants that did apply 
for  alternative  funding,  73.3%  (11  out  of  15)  indicated  that  these  applications  involved 
significant  adaptations  compared  with  their  original  Horizon  Europe  application.  The  main 
reasons  given  for  not  applying  for  alternative  funding  were  the  need  for  a  variety  of 
international  partners,  a  lack  of  alternative  funding  for  their  research  topic/area,  and 
addressing EU-level (and/or) global challenges (see Figure 29).  

160 

 
 
Figure 29. INFRA survey responses to the question ‘How important were the following reasons that prevented your 
organisation from implementing the project without Horizon Europe funding?’ Source: survey of Horizon Europe 
unsuccessful applicants, conducted May-July 2023. 

1.5.5.  Relevance 

The ERA, with its ‘pact for Research Innovation in Europe’204, underlines the fact that RIs are 
crucial to a well-functioning research and innovation system in Europe. One of the concrete 
actions of the new ERA Policy Agenda is therefore to strengthen RIs. Interviews and desk 
research  have  confirmed  the  relevance  of  INFRA,  as  it  is  expected  to  foster  scientific 
excellence, enhance Europe's competitiveness, and increase Europe’s capacity to address 
societal challenges. 

A  key  guiding  document  regarding  the  relevance  of  INFRA  is  the  Landscape  Analysis 
published  by  ESFRI.  The  Landscape  Analysis  provides  an  overview  of  the  European 
ecosystem  of  RIs,  including  an  analysis  of  the  needs  and  gaps  in  the  ecosystem.  In  the 
course of 2024, the most comprehensive Landscape Analysis to date will be published. This 
will  provide  the  framework  for the  next ESFRI  roadmap,  contribute  to  the  EOSC Strategic 
Research  and  Innovation  Agenda,  and  promote  the  development  of  new  RI  services.  The 
analysis will be based on a broad stakeholder consultation and will be key to ensuring the 
continuing relevance of INFRA.  

Compared  with  Horizon  2020,  the  INFRA  programme  part  under  Horizon  Europe  has  an 
increased focus on interdisciplinary research aimed at solving big societal challenges,  thus 
increasing the link between the programme part and EU policy priorities – especially those 
related to the green and digital transitions. Under Horizon Europe, around one-fifth of the net 
EU  contribution  within  INFRA  (or  EUR  100  million)  went  to  challenge-oriented  topics. 
Examples  of  such  topics  include  INFRASERV  topics  calling  for  ‘RI  services  to  support 
research  addressing  cancer’  or ‘RI  services  for a  sustainable  and  resilient  agriculture and 
agro-ecological  transitions’.  Among  the  challenge-oriented  topics  are  two  EMERGENCY 
topics calling for RI services and FAIR and open data sharing for rapid research responses 
to European preparedness for COVID-19 and other infectious diseases205. These topics were 

204 Available at: https://research-and-innovation.ec.europa.eu/strategy/strategy-2020-2024/our-
digital-future/european-research-area_en. 
205 HORIZON-INFRA-2021-EMERGENCY-01 and HORIZON-INFRA-2021-EMERGENCY-02. 

161 

 
 
 
launched in response to the COVID-19 crisis, and show how the Commission was able to 
react in a fast and flexible way to new societal evolutions and scientific needs.  

‘very 

Interviewees  were  generally  positive  towards  this  shift  to  societal  challenges  and 
acknowledge  the  need  for  RIs  to  contribute  to  solving  big  societal  challenges.  Equally, 
interviewees  still  stressed  the  importance  of  blue-sky  research  in  addressing  future 
challenges,  in  addition  to  funding  for  more  applied  research  directed  towards  specific 
challenges. Only 39.4% of INFRA beneficiaries surveyed (65 out of 165) agreed to a ‘large’ 
to  conduct 
or 
basic/fundamental research in new and emerging research areas. A slightly higher share of 
INFRA  beneficiaries  surveyed  (48.8%, or 74  out  of  164) agreed  to  a  ‘large’  or ‘very  large’ 
extent  that  there  are  sufficient  funding  opportunities  for  research  into  applications  of 
technology  in  areas  that  are  important  in  addressing  global  challenges.  Moreover,  the 
majority of INFRA beneficiaries (58.2%, or 96 out of 165) responded that, to a ‘large’ or ‘very 
large’ extent, there are enough funding opportunities for interdisciplinary research.  

funding  opportunities 

there  are  sufficient 

large’  extent 

that 

In  relation  to  this  point,  some  interviewees  expressed  concerns  about  the  increased 
dependence  of  funding  on  (top-down)  political  priorities,  and  the  instability  that  can  result 
from  this.  The  time  horizon  for  politics  is  usually  shorter  than  that  needed  for  scientific 
endeavours.  As  new  priorities  are  set,  the  continuity  of  projects  may  be  compromised, 
thereby  reducing  the  long-term  impacts  of  projects  and  the  of  the  programme.  Some 
interviewees  therefore  indicated  the  need  for  longer-term  roadmaps  (at  least  beyond  the 
typical project lifetime of 3-4 years) in order to facilitate longer-term planning and stability.  

1.5.6.  Conclusions and lessons learned 

Conclusion 1: INFRA plays an important role in opening up, integrating and interconnecting 
RIs,  contributing  to  RIs  being  accessible  to  researchers  from  across  Europe  and  beyond. 
However,  several  of  the  beneficiaries  interviewed  mentioned  that  there  is  the  need  for  a 
longer-term vision and strategy in this regard, to ensure the sustainability of the RI networks 
and services developed beyond the lifetimes of individual projects. The new challenge-driven 
approach employed under Horizon Europe for part of the transnational access provision to 
RIs  has  raised  concerns  among  interviewed  beneficiaries  regarding  the  continuity  of  RI 
networks and their ability to achieve impact. It is currently unclear how the challenge-oriented 
calls will evolve as political priorities shift, and what will happen to the new networks of RIs 
and the services they have developed after the project’s lifetime. 

Lesson learned 1.1.: longer-term roadmaps that provide RIs with a perspective beyond the 
lifetime of individual projects (3-4 years) would facilitate a necessary level of continuity in the 
integration  of  RIs.  Moreover,  it  is  important  to  achieve  a  balance  between  bottom-up  and 
challenge-driven approaches to ensure that the ERA is well prepared to address both current 
and future policy needs.  

Conclusion 2: a key contribution made by INFRA is its support for the development of RIs 
on the ESFRI roadmap, which ensures a coherent RI landscape that is high in excellence. 
To ensure the effective implementation and operation of world-class RIs, it is important that 
they receive adequate funding throughout their lifecycle.  

Lesson  learned  2.1.:  an  increase  in  funding  for  the  development  of  world-class  RIs, 
especially  beyond  the  preparatory  phases,  could  be  considered  as  part  of  a  longer-term 
vision.  Such  a  vision  should  consider  in  particular  the  balance  between  funding  the 
development of world-class RIs that are on the ESFRI Roadmap, and funding the integration 
and  opening  up  of  smaller  (national)  RIs.  Funding  for  the  integration  of  RIs  is  currently 

162 

 
 
 
fragmented across very large networks of RIs, raising concerns regarding the optimisation of 
this funding’s impact. 

Conclusion  3:  INFRA  provides  clear  added  value  and  is  highly  complementary  to  other 
sources of European and national/regional funding. At the same time, INFRA is somewhat 
disconnected  from  funding  provided  at  other  levels  (only  6.2%  of  beneficiaries  surveyed 
indicated that their project is a continuation of a project from another funding source).  

Lesson learned 3: further discussions and agreements with Member States with regard to 
the funding of RIs are important, and could alleviate pressure on the INFRA budget. 

Conclusion 4: the connection between RIs and the private sector/industry could be further 
strengthened in order to enhance the impact and financial sustainability of RIs. There is only 
limited participation by the private sector in INFRA (9% of all participations), and the use of 
RIs by the private sector also remains limited. 

Lesson learned 4: to further strengthen the connection between RIs and the private sector, 
the following suggestions could be considered:  

•  a better connection between research and technology infrastructures could help RIs in 

reaching industry, and would support the continuum of research and innovation.  

• 

the  increased  participation  of  RIs  in  other  pillars  of  Horizon  Europe  could  increase  the 
visibility and use of RIs by industry. Currently, barriers exist that prevent applicants from 
identifying relevant topics in programme parts of the FP with which they are not familiar. 
More guidance on navigating calls, as well as explicitly mentioning relevant RIs in the call 
texts, would boost the participation of RIs in other parts of the Framework Programme, 
and could foster the connection between RIs and the private sector.  

•  Lastly,  longer  time  horizons  are  often  necessary  for  co-development  with  industry; 
strategic  roadmaps  that  provide  perspectives  on  the  medium  to  longer  term  would  be 
helpful in this regard. 

Conclusion 5: INFRA contributes significantly to the development of the EOSC. However, 
support for the EOSC is currently fragmented across a large number of projects that are not 
well  synchronised.  In  addition,  the  concept  of  the  EOSC  is  still  evolving  rapidly,  which 
requires significant resources from the partners involved in order to remain up to date.  

Lesson learned 5: to further boost synergies between the projects contributing to the EOSC, 
the following measures could be considered:  

•  Increased stability and more time for coordination in projects working on the EOSC would 

increase the effectiveness of support for the EOSC.  

•  While dedicated funding for the science clusters was discontinued under Horizon Europe, 
this was considered an important platform for collaboration between RIs within the same 
scientific discipline, with a view to the integration of RI services and data into the EOSC. 
Sufficient  attention  should  still  be  paid  to  the  within-discipline  integration  of  diverse 
datasets.  

163 

 
 
 
 
 
 
 
 
 
1.6.  Widening participation and strengthening the European Research 

Area under Horizon Europe 

1.6.1.  Strengthening the ERA 

One  of  the  key  novelties  introduced  in  WIDERA  under  Horizon  Europe  was  a  strong 
alignment with the new strategic priorities for the European Research Area (ERA). Starting 
with  a  Commission  Communication  on  a  new  ERA  for Research  and  Innovation  (2020)206 
and the Council Conclusions of December 2020 on the ‘New European Research Area’, 207 
the ERA was revitalised in order to improve the European research and innovation landscape 
and support the transition to a green, digitalised and resilient Europe.  

Following Coordination and Support Actions (CSAs) under previous FPs, the ERA part of the 
WIDERA work programme has introduced actions that are aligned closely with the new policy 
framework and support the implementation of the priorities of the new ERA. ERA Destination 
3 is structured according to the four objectives set out in the ERA Communication: prioritising 
investments  and  reforms  in  research  and  innovation;  improving  access  to  excellence; 
translating R&I results into the economy; and deepening the ERA.  

The  ERA  work  programme  contributes  to  the  priority  areas  defined  in  the  Council 
Recommendation on a Pact for Research and Innovation in Europe (Pact for R&I)208: 

•  Deepening a truly functioning internal market for knowledge; 

•  Together, taking up the challenges posed by the twin green and digital transitions, 

and increasing society’s participation in the ERA; 

•  Amplifying access to excellence in R&I across the Union; 

•  Advancing concerted research and innovation investments and reforms.  

Moreover, the ERA Work Programme is increasingly shaped by and reflects the ERA Policy 
Agenda (although the ERA WP 2021-2022 predates the first ERA Policy Agenda 2022-2024). 
This is primarily achieved through the ERA work programme, which support the ERA Policy 
Agenda actions (e.g. Action 1 on Open Science, Action 4 on research careers, Action 14 on 
citizen engagement). For more details, see Section 1.5 on the relevance of the ‘Strengthening 
the  ERA’  thematic  priority.  The  ERA  work  programme  2023-2024  is  even  more  closely 
aligned with the ERA Policy Agenda, with most of its topics explicitly referring to the relevant 
ERA Policy Agenda actions209. 

206 Communication From the Commission to the European Parliament, the Council, the 
European Economic and Social Committee and the Committee of the Regions: a new ERA for 
Research and Innovation, {SWD(2020) 214 final}. 
207 Council Conclusions on the New European Research Area, available at: 
https://data.consilium.europa.eu/doc/document/ST-13567-2020-INIT/en/pdf 
208 Council Recommendation on a Pact for Research and Innovation in Europe, available at: 
https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32021H2122 
209 Internal Commission document. 

164 

 
 
According to the work programmes 2021-2022 and 2023-2024, the key expected impacts of 
the  ‘Reforming  and  enhancing  the  EU  research  and  innovation  system’  destination  of  the 
WIDERA programme are as follows: 

–  Reform and enhance the European R&I system; 
–  Accomplish the recovery and the twin transitions by prioritising of investments and 

reforms; 
Improve access to excellence; 

– 
–  Ensure high-quality scientific production and the stronger translation of R&I results 

into the economy; 
–  Deepen the ERA; 
–  Coordinate  national  and  regional  R&I  programmes  by  pooling  national  resources 
and contributing to the alignment of national research and innovation policies; 
Improve  knowledge  for  policy-making  about  the  networking  patterns  of  research 
support staff and research management; 
Identify  and  exploit  synergies  between  R&I  and  higher  education  policies  and 
programmes; 

– 

– 

–  Modernise the higher education sector, addressing higher education, research, and 

– 

innovation; 
Increase the number of interconnected knowledge ecosystems, strong in knowledge 
creation, circulation and use; 

–  Enable researchers to benefit from attractive careers; 
–  Promote inclusive gender equality in the European R&I system; 
–  Ensure a more open and inclusive research and innovation system; 
– 

Increase the capacity of the EU R&I system to conduct Open Science and set it as 
the modus operandi of modern science; 
Increase the engagement of citizens with research and innovation; 
Increase the alignment of strategic research with societal needs, expectations and 
values; 
Identify  synergies  between  secondary  and  tertiary  education,  and  between 
education and business; 
Increase  trust  in  science  and  R&I  outcomes,  and  ensure  greater  two-way 
communication between science and society; 

– 
– 

– 

– 

–  Ensure the free circulation of knowledge and a highly skilled workforce; 
– 

Improve capacities for conducting Open Science within the EU R&I system. 

To achieve the above impacts, WIDERA provides support to a number of actions (see Table 
36). 

PRIORITIZING INVESTMENT AND REFORM 

• 
• 

Programme-level collaboration between national R&I policy-makers 
Support for a strong European R&I Foresight Community to better inform R&I policy decisions 
in the European Research Area about potential futures 

TRANSLATING R&I RESULTS INTO THE ECONOMY 

Implementation of a new macro-economic modelling concept 
Standardisation Booster for fostering the exploitation of FP-funded research results 

• 
• 
•  R&I-intensive IP management: scenarios for the future 
• 
• 

Experimentation and exchange of good practices for value creation 
Testing of the ERA Hub concept – pilot phase 

165 

 
IMPROVING ACCESS TO EXCELLENCE 

• 
• 

Support for a Europe-wide training and networking scheme for research managers 
Support  for  strengthening  researchers’  skills  for  better  careers  –  leveraging  the  European 
Competence Framework for Researchers 

DEEPENING THE EUROPEAN RESEARCH AREA 

Institutional change 
• 

Acceleration  Services  in  support  of  the  institutional  transformation  of  higher  education 
institutions (HEIs) 
Protection  of  HEIs  and  research  organisations  against  conventional  and  non-conventional 
threats 

• 

Open science 
•  Capacity-building for institutional open access publishing across Europe 
•  Global cooperation on FAIR data policy and practice 
Increasing the reproducibility of scientific results 
• 
•  Modelling and quantifying the impacts of Open Science practice 
• 

Support for changes in the assessment of research and researchers to reward the practice of 
Open Science 
Supporting the development of aligned policies for open access books and monographs 
Exploitation and valorisation of results relevant for the ERA Policy Agenda 

• 
• 
•  Capacity building on the management of intellectual property (IP) to support open science  
•  Global cooperation in not-for-profit open access publishing 

Gender equality 
•  Centre of excellence on inclusive gender equality in R&I 
• 
• 

Living Lab for gender-responsive innovation 
Policy  coordination  to  advance  the  implementation  of  the  ERA  gender  equality  and 
inclusiveness objectives within Member States 
Support for the implementation of inclusive gender equality plans (supported under both work 
programme 2021-2022 and work programme 2023-2024) 
Policy support to facilitate the implementation of a zero-tolerance approach towards gender-
based violence in the ERA 
Support  for  the  implementation  of  an  EU  Manifesto  for  STE(A)M  education  and  R&I  career 
paths to tackle gender inequalities in the ERA 
Policy coordination to support all aspects of inclusive Gender Equality Plans and policies in the 
ERA 

Research ethics and integrity 
•  Developing  and  piloting  training  on  the  practice  of  open  and  responsible  research  and 

innovation 
Ensuring  reliability  and  trust  in  the  quality  of  research  ethics  expertise  in  the  context  of 
new/emerging technologies 
The challenges of research ethics and integrity in response to crisis: the coronavirus pandemic 
and beyond 
An empirical and behavioural approach to research ethics and integrity 

• 
•  Research ethics for environmental and climate technologies 
• 
•  Next-generation AI and human behaviour: promoting an ethical approach 

A review of the future of research ethics in changing research environments 

Citizen science and science education 
• 

A  capacity-building  and  brokering  network  to  make  citizen  science  an  integral  part  of  the 
European Research Area 
A European competence centre for science communication 

• 

166 

• 

• 

• 

• 

• 

• 

 
 
 
 
 
•  Developing a STE(A)M roadmap for science education under Horizon Europe 
•  Developing an effective ERA talent pipeline 
•  Open schooling for science education and a learning continuum for all 
• 
• 

Societal trust in science, research and innovation  
Stepping up institutional and territorial changes towards open and responsible research and 
innovation 
Supporting and giving recognition to citizen science in the European Research Area 
Laying the groundwork for Europe-wide citizen science campaigns 

• 
• 

Other 
• 

Building a virtual European Info Platform on Contemporary China 

Table 36. Actions supported under ‘Reforming and enhancing the EU research and innovation system’ destination of 
WIDERA programmes 2021-2022 and 2023-2024. Source: prepared by the authors, based on the WIDERA work 
programmes 2021-2022 and 2023-2024.

167 

 
 
Figure 30. Intervention logic of ‘Reforming and enhancing the EU research and innovation system’ 

168 

 
 
 
The Policy Support Facility (PSF),, introduced under Horizon 2020, has been another of 
the  key  instruments  towards  the  achievement  of  structural  changes  at  national  level  by 
supporting reforms in the R&I systems of the Member States and associated countries. Since 
2015,  Horizon  2020  has  funded  actions  through  the  PSF,  which  has  provided  participant 
countries  with  practical  support  in  identifying,  designing,  implementing  and  evaluating 
reforms  that  has  enhanced  the  quality  of  their  R&I  investments,  policies  and  systems210. 
Available evidence indicates that the PSF has had a number of benefits211 that are relevant 
to the achievement of structural reforms, such as producing evidence-based knowledge for 
national  policy-making  and  increasing  knowledge  among  national  policy-makers.  Previous 
evaluations  have  shown  that  the  PSF  contributed  to  the  development  of  operational 
recommendations 
the 
implementation of national R&I reforms such as the renewed Science Agenda of Bulgaria, 
which  paid  particular  attention  to  the  recommendations  formulated  by  the  dedicated  PSF 
Peer Review212. 

leading  experts  and  policy  practitioners 

formulated  by 

for 

The  PSF  continues  to  provide  support  under  Horizon  Europe,  with  some  key  novelties 
introduced to address the challenges and issues identified during the previous period. The 
lack of more systematic and on-demand follow-up actions to support the implementation and 
exploitation of the PSF’s results was identified as one of the key challenges of the instrument 
under Horizon 2020. Responding to this, the Horizon Europe PSF has introduced a novelty, 
PSF Open, to provide support for follow-up actions on previous PSF exercises. PSF Open 
allows  countries  that  have  already  benefitted  from  PSF  exercises  to  receive  support  in 
implementing the PSF’s recommendations. In addition to the newly introduced PSF Open, 
the PSF under Horizon Europe also supports the following activities: 

•  PSF Country (formerly PSF Peer Reviews & Specific support to countries), which 
supports in-depth assessments of a country's R&I system (or of parts thereof) carried 
out by a panel of experts and peers. These assessments lead to actionable policy 
recommendations  to  the  national  authorities  on  reforms  necessary  to  strengthen 
their R&I system. 

•  PSF Challenge, which supports the PSF Mutual Learning Exercises that focus on 
specific and operational R&I challenges of interest to several volunteering countries. 
This aims to identify and promote the exchange of good practices, lessons learned 
and success factors among participant countries. 

In total, as of October 2023, Horizon Europe PSF has supported 27 different Member States 
and associated countries through its Country, Challenge and Open services, with Romania 
(seven  PSF  projects),  Austria  and  Belgium  (five  PSF  project  each)  being  the  most  active 
participants (for more details on Horizon Europe PSF participant countries, see the  Figure 
31). 

210 https://ec.europa.eu/research-and-innovation/en/statistics/policy-support-facility 
211 https://www.scar-swg-
sbgb.eu/lw_resource/datapool/_items/item_34/bioeconomy_presentation_a4_marta_truco.pdf 
212 Interim Evaluation of Horizon 2020, available at: https://op.europa.eu/en/publication-detail/-
/publication/33dc9472-d8c9-11e8-afb3-01aa75ed71a1/language-en 

169 

 
 
7

5

5

8

6

4

2

0

a
i
r
t
s
u
A

i

a
n
a
m
o
R

i

m
u
g
e
B

l

4

4

4

4

4

3

3

3

3

2

2

2

2

2

2

2

2

2

1

1

1

1

1

1

e
c
e
e
r
G

y
a
w
r
o
N

l

a
g
u
t
r
o
P

d
n
a
n
i
F

l

i

a
n
e
v
o
l
S

a
i
r
a
g
u
B

l

a
t
l
a
M

a
i
k
a
v
o
l
S

n
e
d
e
w
S

a
v
o
d
o
M

l

a
i
t
a
o
r
C

e
c
n
a
r
F

d
n
a
e
r
I

l

y
n
a
m
r
e
G

i

a
n
a
u
h
t
i
L

d
n
a
o
P

l

a
i
v
t
a
L

i

n
a
p
S

i

a
g
r
o
e
G

e
y
i
k
r
ü
T

i

a
n
o
t
s
E

i

a
h
c
e
z
C

y
l
a
t
I

y
r
a
g
n
u
H

Figure 31. Horizon Europe PSF participant countries by number of projects (including PSF Country, PSF Challenges and 
PSF Open), as of October 2023. Source: compiled by the study team. 

A  more  detailed  analysis  by  types  of  PSF  services  shows  that  the  PSF’s  country-specific 
policy  support  for  structural  reforms  to  strengthen  R&I  systems  (PSF  Country  and  Open 
services) primarily benefitted those countries with lower R&I performance. So far, Horizon 
Europe PSF Country services supporting in-depth assessments of national R&I systems and 
PSF  Open  (follow  up  actions)  targeted  widening  countries,  including  Moldova,  Greece, 
Croatia  and  Romania  (see  Table  37).  One  of  the  key  benefits  of  the  PSF  under  Horizon 
Europe is its contribution to bridging the R&I gap between countries with higher and lower 
levels of R&I performance. 

At the same time, the PSF has also provided a framework for the exchange of good practice 
between  countries  with  differing  R&I  performance,  mainly  through  its  Mutual  Learning 
Exercises (MLEs). So far, six MLEs have been supported under PSF Challenges, and have 
featured a wide geographical coverage including both widening and non-widening countries 
taking part in the learning exercises. The most active countries in terms of participation in 
PSF Challenge (Mutual Learning Exercises) have been Romania, Austria and Belgium (each 
of  which  participated  in  five  PSF  MLEs),  followed  by  Norway,  Portugal,  Finland,  Slovenia 
(four MLEs), and Bulgaria, Malta, Slovakia and Sweden (three MLEs). So far, one MLE (on 
Citizen  Science  Initiatives  –  Policy  and  Practice)  has  been  finalised,  with  the  other  five 
exercises  still  ongoing  and  expected  to  be  finalised  in  2024.  Table  37  summarises  the 
activities funded Horizon Europe PSF as of October 2023, together with their geographical 
coverage. 

Type of 
PSF 
service 

PSF 
Country 

Exercises funded under Horizon 
Europe 

Geographical coverage 

Moldova, Greece, Croatia, Romania 

Support to Moldova on reforms in the 
public R&D sector 
Country support for policies developing 
Research Infrastructures and the R&I 
ecosystem in Greece 
PSF to support early stages of innovation 
and science-business linkages in Croatia 
PSF Country Review of the Romanian 
research and innovation (R&I) system 

PSF 
Challenge 

Mutual Learning Exercise on the EU 
Missions 

Norway, Austria, Belgium, Bulgaria, 
France, Germany, Greece, Ireland, 

170 

 
 
Type of 
PSF 
service 

Exercises funded under Horizon 
Europe 

Geographical coverage 

Mutual Learning Exercise on industrial 
decarbonisation 

Mutual Learning Exercise on Knowledge 
Valorisation – Focus on Skills, 
Intersectoral Cooperation and Incentive 
Systems 

Lithuania, Malta, Poland, Portugal, 
Romania, Slovakia, Sweden 

Georgia, Turkey, Austria, Belgium, 
Finland, Latvia, Lithuania, Portugal, 
Slovakia, Slovenia, Spain, Swede 

Moldova, Norway, Austria, Belgium, 
Bulgaria, Croatia, Estonia, Finland, 
Greece, Hungary, Ireland, Latvia, 
Malta, Poland, Romania, Slovenia, 
Spain, Sweden 

Mutual Learning Exercise on the Whole of 
Government Approach in Research and 
Innovation 

Bulgaria, Finland, Malta, Romania, 
Slovakia 

Mutual Learning Exercise (MLE) on R&I 
Foresight 

Norway, Austria, Belgium, Czechia, 
Finland, Portugal, Romania, Slovenia 

Mutual Learning Exercise on Citizen 
Science Initiatives – Policy and Practice 

Norway, Austria, Belgium, France, 
Germany, Hungary, Italy, Portugal, 
Romania, Slovenia, Sweden 

PSF Open 

Support to Greece in the implementation 
of PSF Country recommendations 
Support to Romania in the 
implementation of 10 PSF Country 
recommendations 

Greece, Romania 

Table 37. Summary of PSF activities funded under Horizon Europe, as of October 2023. Source:  compiled by the study 
team. 

1.6.1.1. 

Effectiveness 

Overview of project implementation 

According  to  the  Horizon  Dashboard213,  56  Horizon  Europe  ERA  WP  grants  have  been 
funded  since  2021,  amounting  to  a  total  cost  of  EUR 133.8  million,  and  a  total  EU  net 
contribution of EUR 128.1 million. This amounts to a net EU contribution per project of around 
EUR 2.29 million. In total, there have been 705 participations in Horizon Europe  ERA WP 
actions,  with  516  unique  participants  and  an  average  of  around  12.59  participations  per 
project. At the time of evaluation, out  of 56 signed grants, 54 were ongoing and  two were 
closed214. 

With  36%  of  proposals  being  successful,  the  ERA  thematic  priority  has  a  relatively  high 
success rate compared with other Horizon Europe programme parts. At the same time, the 

213 Data as of 28 August 2023. 
214 According to the Horizon Europe Dashboard as of 8 September 2023. 

171 

 
 
 
 
ERA thematic priority has received a share of ineligible proposals of around 13%. This share 
of ineligible/inadmissible proposals is the highest among all the programme parts analysed 
in  the  present  study,  indicating  a  potential  need  to  strengthen  the  guidance  for  potential 
applicants to the ERA actions. It is notable that all of the ineligible/inadmissible proposals 
were under 2022 calls (17.7%, or 22 out of 174 proposals). By contrast, in 2021, 48 project 
applications  were  received,  with  none  of  them  being  ineligible/inadmissible.  Similarly,  in 
2023, only two proposals were received under the ERA thematic priority (both of them being 
successful).  It  is  also  notable  that  the  absolute  majority  of  ineligible/inadmissible  ERA 
proposals in 2022 (19 out 22) were under the topic  ‘Stepping-up institutional and territorial 
changes towards open and responsible research and innovation’ (HORIZON-WIDERA-2022-
ERA-01-40). 

Out of the 705 total participations since 2021, 241 (34% of participations) were from higher 
or secondary education establishments, followed by research organisations (151, or 21%), 
public  bodies  (98,  or 14%),  Private  for-profit  entities  (84,  or 12%)  and  other  organisations 
(131, or 19%).  

In  terms  of  geographical  distribution,  the  greatest  numbers  of  participations  in  ERA  WP 
projects have come from Belgium (61), Italy (51), Germany (51), Netherlands (47), Spain (45) 
and France (38). Among non-EU countries, the greatest numbers of participations have come 
from the UK (37), Norway (19), Turkey (12) and Serbia (12) (for more details, see Figure 32). 
In terms of the EU net contribution by country, the Netherlands and Belgium came first, each 
receiving around 12% of total net EU contributions under ERA WP programme part, followed 
by  Italy  and  Germany  (11%  each),  and  Spain  (9.5%).  Among  non-EU  countries,  Norway 
received the highest share of net EU contributions under the ERA WP programme part (2.6% 
of all EU net contributions). 

Figure 32. Number of participations in ERA WP projects by country, as of 28 August 2023. Source: Horizon Dashboard. 

Analysis in terms of thematic coverage/keywords shows that WIDERA projects in the ERA 
thematic  area  cover  a  wide  variety  of  themes:  9%  (153)  of  participations  in  ERA  projects 
reported the keyword phrase ‘Responsible Research and Innovation’ (RRI), followed by ‘EU 
research policy/Research policies in the EU’ (145, or 8%), ‘Gender equality’ (107, or 6%), 
‘Inclusion’  (92,  or  5%),  Science,  technology  and  mathematics  (STEM)  (87,  or  5%  of 
participations).  Other  keywords  were  found  in  fewer  than  5%  of  participations  under  the 
Horizon Europe ERA WP (for more details, see Figure 33). 

172 

 
 
Figure 33. Percentage share of participations in WIDERA(ERA), by keyword. Source: Horizon Dashboard, as of 8 
September 2023. 

Administrative  data  and  interview  evidence  also  confirms  that  WIDERA  projects  under 
‘Strengthening  the  ERA’  supported  a  wide  variety  of  topics.  Some  of  the  most  widely 
supported topics included citizen science/science education and communication/science and 
society (15 grants supported since 2021); support for university transformation across various 
fields,  e.g.  acceleration  services  in  support  of  the  institutional  transformation  of  higher 
education  institutions  (three  grants);  Europe-wide  training  and  networking  scheme  for 
research managers (two grants); gender equality (7 grants); Open Science/reproducibility of 
research  results  (12  grants),  Responsible  research  and  Innovation/research  ethics  (four 
grants).Three grants so far have also been given under the topic ‘Support for policy makers 
– Programme-level collaboration between national R&I programmes’ (for more details, see 
Table  38).  Interview  evidence  also  confirms  that  whereas  the  ERA  WP  has  succeeded  in 
supporting a wide range of projects in the field of citizen science/science and society, as well 
as  projects  to  support  university  transformation  across  various  fields  (such  as  gender 
equality, entrepreneurship, open access policies and practices, RRIs/research ethics etc.), 
progress  with  regard  to  support  for  programme/policy  level  collaboration  between  policy-
makers has been more limited.  

173 

 
 
 
 
 
 
 
Project 
status 

Call ID 

Topic description 

Signed 
grants 

Participations 

Net EU 
contribution 

Signed 

HORIZON-

Support to the 

4 

36 

4,071,279 

WIDERA-2022-

implementation of 

ERA-01 

inclusive gender equality 

plans 

Signed 

HORIZON-

Developing a STE(A)M 

3 

35 

5,129,446 

WIDERA-2021-

roadmap for Science 

ERA-01 

Education in Horizon 

Europe 

Signed 

HORIZON-

Societal trust in science, 

3 

24 

5,334,501 

WIDERA-2021-

research and innovation 

ERA-01 

Signed 

HORIZON-

Acceleration Services in 

3 

30 

9,893,845 

WIDERA-2022-

support of the institutional 

ERA-01 

transformation of Higher 

Education Institutions 

Signed 

HORIZON-

Increasing the 

WIDERA-2022-

reproducibility of scientific 

ERA-01 

results 

Signed 

HORIZON-

Open schooling for 

WIDERA-2022-

science education and a 

ERA-01 

learning continuum for all 

3 

3 

34 

4,926,704 

52 

5,494,140 

Signed 

HORIZON-

Towards a Europe-wide 

2 

16 

2,998,309 

WIDERA-2021-

training and networking 

ERA-01 

scheme for research 

managers 

Signed 

HORIZON-

Developing an effective 

2 

26 

2,903,960 

WIDERA-2022-

ERA talent pipeline 

ERA-01 

Signed 

HORIZON-

Support for policy makers 

2 

31 

4,993,608 

WIDERA-2022-

– Programme level 

ERA-01 

collaboration between 

national R&I programmes 

174 

 
Project 
status 

Call ID 

Topic description 

Signed 
grants 

Participations 

Net EU 
contribution 

Signed 

HORIZON-

Testing of the ERA Hub 

2 

18 

2,929,862 

WIDERA-2022-

concept – pilot phase 

ERA-01 

Closed 

HORIZON-

Support the Slovenian 

1 

1 

100,000 

WIDERA-2021-

Presidency for the 

ERA-IBA 

organization of an ERA 

Conference 

Closed 

HORIZON-

Presidency event - 

1 

1 

399,027 

WIDERA-2022-

Conference on 

CONF-RI-IBA 

international cooperation 

in research and 

innovation 

Signed 

HORIZON-

Horizontal support to 

1 

15 

4,000,000 

EURATOM-2022-

European Partnerships 

ERA-LEARN-IBA 

(ERA-LEARN) 

Signed 

HORIZON-

Horizon Europe National 

1 

36 

4,000,000 

EURATOM-2022-

Contact Point (NCP) 

NCP-IBA 

Horizontal Support 

Signed 

HORIZON-

A capacity-building and 

1 

21 

3,999,043 

WIDERA-2021-

brokering network to 

ERA-01 

make citizen science an 

integral part of the 

European Research Area 

Signed 

HORIZON-

Capacity-building for 

1 

24 

2,528,181 

WIDERA-2021-

institutional open access 

ERA-01 

publishing across Europe 

Signed 

HORIZON-

Centre of excellence on 

1 

14 

4,996,758 

WIDERA-2021-

inclusive gender equality 

ERA-01 

in Research & Innovation 

Signed 

HORIZON-

Ensuring reliability and 

1 

18 

3,794,374 

WIDERA-2021-

trust in quality of 

ERA-01 

Research Ethics expertise 

175 

 
Project 
status 

Call ID 

Topic description 

Signed 
grants 

Participations 

Net EU 
contribution 

in the context of 

new/emerging 

technologies 

Signed 

HORIZON-

Global cooperation on 

1 

19 

1,883,841 

WIDERA-2021-

FAIR data policy and 

ERA-01 

practice 

Signed 

HORIZON-

Modelling and quantifying 

1 

10 

1,999,990 

WIDERA-2021-

the impacts of open 

ERA-01 

science practice 

Signed 

HORIZON-

Policy coordination to 

1 

40 

2,999,814 

WIDERA-2021-

advance the 

ERA-01 

implementation of the 

ERA gender equality and 

inclusiveness objectives 

within Member States 

Signed 

HORIZON-

Standardisation Booster 

1 

6 

2,000,000 

WIDERA-2021-

for fostering exploitation 

ERA-01 

of FP-funded research 

results 

Signed 

HORIZON-

Support for policy makers 

1 

12 

2,823,600 

WIDERA-2021-

– Programme level 

ERA-01 

collaboration between 

national R&I programmes 

Signed 

HORIZON-

Support to changes in the 

1 

18 

1,726,898 

WIDERA-2021-

assessment of research 

ERA-01 

and researchers to 

reward the practice of 

open science 

Signed 

HORIZON-

Supporting and giving 

1 

7 

4,113,312 

WIDERA-2021-

recognition to citizen 

ERA-01 

science in the European 

Research Area 

176 

 
Project 
status 

Call ID 

Topic description 

Signed 
grants 

Participations 

Net EU 
contribution 

Signed 

HORIZON-

The challenges of 

1 

16 

4,201,759 

WIDERA-2021-

research ethics and 

ERA-01 

integrity in response to 

crisis: the coronavirus 

pandemic and beyond 

Signed 

HORIZON-

The EuroScience Open 

1 

1 

2,000,000 

WIDERA-2021-

Forum (ESOF) and 

ESOF-IBA 

European City of Science 

2022 actions 

Signed 

HORIZON-

European Union Contest 

1 

WIDERA-2021-

for Young Scientists 

EUCYS-IBA 

(EUCYS) 2022 

Signed 

HORIZON-

Support to RESAVER 

1 

WIDERA-2021-

Pension Fund 

RESAVER-IBA 

1 

1 

2,000,000 

850,000 

Signed 

HORIZON-

A European competence 

1 

13 

3,103,875 

WIDERA-2022-

centre for science 

ERA-01 

communication 

Signed 

HORIZON-

An experimentation space 

1 

14 

2,889,406 

WIDERA-2022-

for the uptake and use of 

ERA-01 

R&I results for EU 

resilience and future 

preparedness 

Signed 

HORIZON-

Developing and piloting 

1 

19 

3,499,743 

WIDERA-2022-

training on the practice of 

ERA-01 

open and responsible 

research and innovation 

Signed 

HORIZON-

Living Lab for gender-

1 

17 

2,589,225 

WIDERA-2022-

responsive innovation 

ERA-01 

177 

 
Project 
status 

Call ID 

Topic description 

Signed 
grants 

Participations 

Net EU 
contribution 

Signed 

HORIZON-

Stepping-up institutional 

1 

13 

5,000,083 

WIDERA-2022-

and territorial changes 

ERA-01 

towards open and 

responsible research and 

innovation 

Signed 

HORIZON-

Supporting the 

1 

16 

1,859,813 

WIDERA-2022-

development of aligned 

ERA-01 

policies for open access 

books and monographs 

Signed 

HORIZON-

The empirical and 

1 

14 

2,541,520 

WIDERA-2022-

behavioural approach to 

ERA-01 

research ethics and 

integrity 

Signed 

HORIZON-

ERA Talent Platform 

1 

34 

3,999,996 

WIDERA-2022-

ERA-IBA 

Signed 

HORIZON-

European Union Contest 

1 

WIDERA-2023-

for Young Scientists 

EUCYS-IBA 

(EUCYS) 2023 

Signed 

HORIZON-

Support to Retirement 

1 

WIDERA-2023-

Savings Vehicle for 

RESAVER-IBA 

European Research 

Institutions and research 

performing individuals 

(RESAVER) 

1 

1 

1,000,000 

2,499,638 

Table 38. Breakdown of WIDERA (ERA thematic priority) signed grants by status, Call ID, topic description, participations 
and net EU contribution. Source: Horizon Dashboard, as of 9 September 2023. 

Project results and outputs 

Since very few ERA WP grants have been closed at the time of evaluation (just two out of 56 
grants  finalised  as  of  September  2023),  the  available  evidence  mostly  focuses  on  the 
expected or preliminary outputs and results of the grants. 

First,  the  available  evidence  shows  that  Horizon  Europe  support  under  WIDERA  (ERA 
thematic  priority) 
(e.g. 
recommendations, methods, tools, roadmaps, educational materials, training programmes, 

in  policy  standards-setting  measures 

result 

likely 

to 

is 

178 

 
etc.)  that  will  empower  beneficiary  institutions  and  institutions  outside  the  projects  to 
undertake transformation. According to the results of Horizon Europe beneficiaries’ survey, 
a large majority of ERA WP beneficiaries (82%, or 86 out of 105) indicated that their project 
had resulted in/is likely to result in recommendations for policy makers/inputs to national or 
EU  policies;  educational  materials  (63%,  or  66  out  of  105  respondents);  and  training 
programmes  (62%,  or  65  out  of  105  respondents).  Similarly,  a  significant  share  of 
respondents  indicated  that  their  project  had  resulted  in,  or  is  likely  to  result  in,  research 
roadmaps (39%, or 41 out of 105); social innovation (tools, guides, strategies) (35%, or 37 
out of 105); new or improved tools, methods or techniques (31%, or 33 out of 105); codes of 
conduct (16%,17 out of 105) (for more details, see the Figure 34). Moreover, 61% (61 out 
of 99) of ERA WP beneficiaries surveyed agreed to a ‘very large’ or ‘large’ extent that their 
projects  had  achieved/were  likely  to  achieve  the  development  of  policy-making  and 
standards-setting measures.  

Case study interviews with project stakeholders confirmed that the ongoing projects are likely 
to  result  in  policy-setting  standards,  guidelines  and  tools  that  will  enhance  university 
transformation and result in structuring effects across the ERA. For example, one of the final 
deliverables envisaged in one ERA WP project focusing on acceleration services includes 
policy recommendations to support the acceleration of entrepreneurial and innovative HEIs 
across Europe, which could be freely used by institutions and policy-makers across Europe. 
Similarly,  some  of  the  key  deliverables  planned  in  a  project  focusing  on  research 
management  professionalisation  include  standardisation  measures  with  regard  to  the 
terminology  relating  relating  to  the  research  management  profession,  and  ultimately  a 
roadmap to improve the research management (RM) system in Europe, which would be open 
for use by institutions across Europe. 

Similarly,  one  of  the  key  outputs  of  Horizon  Europe  PSF  is  country-specific  policy 
recommendations, as well as tailor-made action plans for the implementation of these PSF 
recommendations.  The  PSF  Country  Review  of  the  Romanian  Research  and  Innovation 
(R&I)  System  (2021-2022)  resulted  in  10  PSF  country  recommendations  being  made  on 
improving governance and human resources, and the internationalisation of the national R&I 
system215. In response to this, ‘Support to Romania in the implementation of 10 PSF Country 
recommendations’ under PSF Open resulted in actionable Implementation Plans216 to put the 
PSF  recommendations  into  practice.  Additionally,  between  December  2021  and  October 
2022,  the  PSF  Country  review  of  the  28  Greek  national  research  infrastructures  (NRIs) 
resulted  in  policy  recommendations  to  increase  the  contribution  of  the  national  RIs  to  the 
Greek  R&I  ecosystem217.  Following  on  from  this,  PSF  Open  ‘Support  to  Greece  in  the 

215 European Commission, PSF Country Review of the Romanian Research and Innovation 
System: Final report, 2022, available at: https://ec.europa.eu/research-and-
innovation/en/statistics/policy-support-facility/psf-country/psf-country-review-romanian-research-
and-innovation-ri-system 
216 European Commission, Implementation Plans for Recommendations from the PSF Country 
Review of the Romanian Research and Innovation System, 2023, available at: 
https://op.europa.eu/en/publication-detail/-/publication/b04c695d-1562-11ee-806b-
01aa75ed71a1/language-en/format-PDF/source-288573309 
217 European Commission, Support to Greece for policies developing research infrastructures 
and the R&I ecosystem: Final report, available at: https://op.europa.eu/en/publication-detail/-
/publication/fc6b0103-45ed-11ed-92ed-01aa75ed71a1/language-en/format-PDF/source-
270143524 

179 

 
 
implementation  of  PSF  Country  recommendations’218  will  provide  expert  support  to  define 
implementation plans for these PSF recommendations, focusing on national RIs. 

Figure 34.  WIDERA WIDERA-ERA-BEN-027: ‘Please indicate which of the following outputs have been produced/are likely 
to result from your Horizon Europe project (please select all applicable answers)’, (n=105). Source: survey of Horizon 
Europe beneficiaries, conducted May-July 2023. 

In many cases, the immediate result of projects (particularly those focusing on acceleration 
services) is systematic evidence gathering aimed at mapping out the state-of-play and 
the reform needs of institutions. This paves the way for further transformation and changes 
within institutions across the ERA. Case study evidence shows that, for this purpose, project 
consortia conduct various consultation activities (surveys, workshops, etc.), which result in 
systematic  evidence  gathering  about  the  needs,  priorities  and  the  current  situation  of 
beneficiary institutions (and in some cases, institutions outside the project consortium). An 
example of such project outcomes is provided by one of the projects covered in Case study 
8: WIDERA Actions on enhancing the European RI system, in which each partner has been 
developing  strategic  vision  statements  towards  becoming  entrepreneurial  and  innovative 
universities.  In  these  statements,  institutions  will  narrow  down  and  identify  key  areas  of 

218 https://ec.europa.eu/research-and-innovation/en/statistics/policy-support-facility/psf-
open/support-greece-implementation-psf-country-recommendations 

180 

 
 
 
institutional transformation in the area of entrepreneurship and innovativeness that they will 
aim to undertake though the project’s implementation. 

Development of collaborative ties and networks of expertise across borders was also 
identified  as  one  of  the  key  outcomes  of  WIDERA  projects  under  the  thematic  priority  of 
Strengthening  the  ERA.  Survey  results  indicate  that  networking  and  the  development  of 
cross-border ties between institutions was among the most frequently reported effects of this 
group of projects. According to the results of the Horizon Europe beneficiary survey, 78% (78 
out of 99 respondents) agreed to a ‘very large’ or ‘large’ extent that their project had resulted 
in/is likely to result in ‘strengthening relationships with leading partners in Europe’, followed 
by ‘increasing international visibility through collaboration with leading global partners’ (67%, 
67 out of 99 respondents ERA WP beneficiaries). In-depth case study and interview evidence 
confirms that nearly all the beneficiary stakeholders consider network building to be one of 
the main outcomes of Horizon Europe support. The development of clusters and groups of 
experts discussing possibilities and pathways for transformation in different thematic areas 
(e.g. entrepreneurship, the professionalisation of gender equality or research managers in 
universities)  have  already  occurred  during  the  project  inception  phase.  In  some  cases, 
projects  already  resulted  in  the  development  of  EU-wide  networks  of  experts  working  to 
gather information and develop frameworks, policies and standards that can be adopted at 
EU level. This is evidenced, for example, by the Ambassadors Network219, which has already 
been built under a WIDERA project. This network brings together 120 community builders 
and leaders from countries across Europe to co-create the future of research management 
in Europe. 

Study  evidence  also  indicates  that  ERA  WP  projects  have  contributed  or  are  likely  to 
contribute  to  development  of  the  skills  and  competences  of  the  administrative, 
managerial and academic staff of universities. According to the survey findings, 77% (77 
out of 99 respondents) of ERA WP beneficiaries agreed to a ‘very large’ or ‘large’ extent that 
their project had achieved, or is likely to achieve, improvements in the skills, knowledge and 
competences of researchers, with another 63% (63 out of 99) agreeing to a  ‘very large’ or 
‘large’ extent that their project will contribute to improving career prospects of researchers 
(see Figure 35). Likewise, some projects in the field of citizen science/science and society 
have  helped  to  equip  young  researchers  with  entrepreneurial  skills.  Case  study  evidence 
shows that in some cases, citizen science-oriented activities supported under WIDERA (e.g. 
brainstorming sessions during the competition event among young researchers-participants, 
TalentOn220)  have  resulted  in  spin-off  initiatives/start-up  companies  initiated  by  young 
researcher  beneficiaries.  Moreover,  case  study  evidence  confirms  that  citizen  science-
oriented projects – such as the EU Contest for Young Scientists (EUCYS), for example – are 
particularly effective at increasing the visibility and attractiveness of research careers 
and motivating talented young people to choose a career path in research. 

In addition to (and in parallel to) individual-level outcomes and learning effects, ERA projects 
(particularly those focusing on acceleration services/university transformation) are also likely 
to  result  in  institutional-level  changes/improvements  to  standards  and  practices, 
thereby  contributing  to  better  working  conditions,  and  the  career  and  professional 
development of staff. When asked if they expect that participation in the Horizon Europe 

219 RM Roadmap Ambassadors Network, available at: 
https://static1.squarespace.com/static/633ae0b47dc8ac471e5978a9/t/6429e8d3302b496a51e6f
569/1680468185491/RM+Roadmap+Ambassadors+Network+V1.pdf 
220 Part of the Leiden 2022 European City of Science, during the EU TalentOn, bright young 
minds (aged between 21 and 35 years) were put to the test. In this competition, they were 
arranged into teams and challenged to think about five EU Missions. 
https://leiden2022.nl/en/highlights/eu-talenton 

181 

 
 
project  would  lead  to  improvements  at  organisational  level,  around  58%  of  ERA  WP 
beneficiary respondents (57  out of 99) indicated  that they expect improvements  to a ‘very 
large’ or ‘large extent’ in the quality of training and opportunities for continuous professional 
development.  This  was  followed  by  41%  (40  out  of  97)  of  respondents  who  expected 
improvements in the provision of career advice for researchers (for more details on expected 
institutional-level improvements, see Figure 36). Similarly, case study findings221 indicate that 
WIDERA  projects  focusing  on  acceleration  services  for  universities  involve  specialised 
coaching/training  for  staff  and  the  building  of  the  knowledge  base,  which  will  equip 
universities with the necessary capacities to implement transformational changes. 

Figure 35. WIDERA-ERA-BEN-031: ‘To what extent, if at all, has your Horizon Europe project achieved/is likely to achieve 
the following results?’ Source: survey of Horizon Europe beneficiaries, conducted May-July 2023. 

221 Case study 8: WIDERA Actions on enhancing the European RI system. 

182 

 
 
Figure 36. WIDERA-ERA-BEN-033: ‘To what extent, if at all, is your participation in the Horizon Europe project expected to 
lead to improvements at organisational level in the following areas?’ Source: survey of Horizon Europe beneficiaries, 
conducted May-July 2023. 

From a longer-term perspective, the above outcomes of the ERA WP should contribute to 
structuring effects at institutional level and across the ERA, which will ultimately increase the 
overall excellence and productiveness of European institutions and research systems. 
For  example,  according  to  the  case  study  findings,  the  projects  focusing  on  the 
professionalisation  of  research  management  are  expected  to  enhance  the  efficiency  of 
national  research  and  innovation  systems,  primarily  by  freeing  up  valuable  research  time 
(allowing researchers to focus first and foremost on research tasks), and by creating a more 
supportive environment for researchers. The expected impact on the excellence of research 
systems  resulting  from  ERA  WP  projects  is  also  evidenced  by  the  survey  results:  45%  of 
survey respondents (45 out of 99) agreed to a ‘very large’ extent or ‘large’ extent that their 
project had resulted in/is likely to result in ‘Pushing the frontiers of knowledge’. Furthermore, 
40% (39 out of 97) of respondents agreed to a ‘very large’ extent or ‘large’ extent that their 
project had resulted in/is likely to result in ‘increasing research productivity (e.g. the number 
of peer reviewed publications’) (see Figure 35). 

It must be noted, however, that achieving the expected impacts will take a substantial amount 
of  time,  since  most  of  the  projects  are  still  in  their  inception  phase.  Moreover,  successful 
achievement  of  project  outputs,  results  (and  their  structural  impacts  on  the  ERA  from  a 
longer-term  perspective)  will  depend  on  a  number  of  complex  success  factors  and  the 
ability to overcome barriers: 

•  Active dissemination of information and the awareness of the project’s activities 
among the local research community and the wider public is one of the key success 
factors It is particularly important in the case of ERA project activities that are directly 
linked to activities at national/local level, as is the case with the EUCYS (EU Contest for 
Young Scientists), in which the EU-level contest gathers together the winners of national-
level contests for young scientists). Awareness of the contest among young people at 
national level varies significantly from country to country: whereas some countries and 
local authorities put a lot of effort into raising awareness of the event and attracting a lot 

183 

 
 
of interest from young people, in other countries the event is not well known among the 
youth.  

•  Project  success  depends  on  ensuring  effective  communication  and  coordination 
between different project partners and their varying needs. This was identified by 
multiple stakeholders as one of the key challenges, particularly within projects focusing 
on university transformation. This critical success factor is also confirmed by evidence 
from the survey, in which ‘Heavy research, teaching, managerial or business overloads 
of  the  researchers  from  the  participating  organisations’  was  identified  as  a  key 
challenge/potential barrier (39%. or 39 out 100 beneficiary respondents agreed to a ‘very 
large’, ‘large’ or ‘moderate’ extent that this constitutes a challenge when carrying out their 
WIDERA project) (see Figure 37). 

• 

•  According to a number of the stakeholders interviewed, project success largely depends 
on  engaging  policy  stakeholders  and  top-level  management,  and  ensuring  their 
commitment to institutional transformation.  
In the case of projects in the field of university transformation/acceleration services, one 
of the key success factors identified is the ability of project coordinators and participants 
to ensure the active involvement of wider groups of stakeholders in the project’s 
activities (e.g. data gathering, stakeholder consultation process). According to case 
study  evidence,  the  successful  development  of  a  roadmap  for  the  future  of  research 
management  (RM)  in  Europe  will  largely  depend  on  the  effective  mobilisation  and 
involvement of the 120 national ambassadors, to ensure their active participation in co-
creation and consultation activities, communication with the local research community 
and data gathering efforts. 

•  The  previous  experience  of  project  coordinators  and  their  ability  to  draw  from 
previous  good  practice  examples  was  also  found  to  be  a  key  success  factor.  For 
example,  in  the  case  of  2022  European  Capital  of  Science,  one  of  the  key  project’s 
success factors was the previous experience of coordinators with the European Capitals 
of  Culture  initiative  and  the  ability  to  use  and  amplify  this  in  the  field  of  science 
communication.  More  specifically,  this  experience  contributed  to  project  coordinators 
applying a new and innovative approach: instead of a top-down ‘science communication’ 
approach,  the  organisers  applied  a  grassroots  bottom-up  approach,  encouraging 
individuals from local communities to take the initiative in promoting science. 

184 

 
Figure 37. WIDERA-ERA-BEN-034: ‘To what extent have the following barriers constituted challenges when carrying out 
your project?’ Source: survey of Horizon Europe beneficiaries, conducted May-July 2023. 

1.6.1.2. 

Coherence 

Internal coherence 

Study evidence indicates that overall, WIDERA support under the ‘Strengthening the ERA’ 
thematic priority has provided a mix of actions that have complemented and were coherent 
with other programme parts of Horizon Europe in terms of the same higher-level objectives 
being  sought  through  different  means.  At  the  same  time,  no  evidence  of  a  duplication  of 
efforts was found. 

ERA WP actions were especially complementary to other Horizon Europe programme parts 
that  contributed  to  the  development  of  human  research  capacities,  the  development  of 
researchers’ careers and the organisational capacities of institutions. In particular, support 
for university transformation/acceleration services under WIDERA is complementary to the 
work  of  the  MSCA,  by  contributing  to  the  same  overarching  objective  of  strengthening 
Europe’s human capital base and improving the working conditions of researchers and the 
attractiveness  of  a  career  in  research.  Whereas  the  MSCA  contribute  to  this  overarching 
objective  primarily  though  supporting  dedicated  actions  in  mobility  and  the  training  of 
researchers, ERA WP support focuses on fostering structural changes in institutions, thereby 

185 

 
 
contributing  to  the  improvement  of  working  conditions  and  the  career  development  of 
researchers. Case study evidence provides examples of how WIDERA projects in the field of 
acceleration services for universities aims to boost university transformation in areas such as 
the development of research careers and talent support (including qualifications and research 
careers, research assessment, digitalisation, talent circulation and mobility, life-long learning, 
etc.).  Similarly,  the  WIDERA  project  ‘Creating  Framework  Conditions  for  Research 
Management to Strengthen the European Research Area’ aims to create a roadmap for the 
future  of  research  management  (RM)  in  Europe,  thereby  contributing  to  the  overall 
development  of  Europe’s  human  research  capacities  by  developing  European-wide 
standards and roadmaps. 

Science  communication-oriented  projects  under  WIDERA  programme  are  also 
complementary  to  the  MSCA  Researchers’  Night  initiative.  Together  (although  by 
different means), these initiatives address the overarching goal of bringing science closer to 
the  public  and  making  research  careers  more  visible  and  attractive,  especially  to  young 
people.  In  some  cases,  activities  in  the  area  of  science  communication  funded  under 
WIDERA (e.g. the European City of Science) have resulted in synergies and spill-over 
effects  with  regard  to  other  parts  of  Horizon  2020/Horizon  Europe.  For  example,  the 
MSCA Alumni Association was involved in the European City of Science and EuroScience 
Open  Forum  (ESOF  2022222)  in  Leiden.  During  the  ESOF2022  Marie  Skłodowska-Curie 
Actions Satellite Event in Leiden, over 120 MSCA fellows, supervisors, coordinators, alumni 
and  policy-makers  gathered  to  network,  debate  and  contribute  to  priority  topics  for  the 
research ecosystem223. 

Contribution to gender equality in research is also a significant area of complementarity 
between the ERA WP and other parts of Horizon Europe, particularly the MSCA. Whereas 
the  MSCA  include  measures  to  address  gender  imbalances  in  research  careers  mainly 
though support for the mobility and training of individuals, the ERA WP supports expert policy 
and knowledge support, as well as institutional transformation in the field of gender equality 
(e.g. by supporting Centres of Excellence on inclusive gender equality in R&I, the Living Lab 
for  gender-responsive  innovation,  and  support  for  the  implementation  of  inclusive  Gender 
Equality Plans). 

Significant  complementarities  also  exist  between  ‘Strengthening  the  ERA’  and  the 
Research  Infrastructures  programme  part  of  Horizon  Europe,  notably  in  the  area  of 
university  transformation  and  support  for Open  Science and  the  re-usability/exploitation  of 
research  results.  Based  on  case  study  evidence,  in  the  domain  of  university 
transformation, some ERA WP projects focusing on the professionalisation of research 
management. These projects have complemented similar initiatives supported under the 
RI,  such  as  the  RI  Train224  and  RI  Trains  Plus225  projects,  aimed  at  improving  and 
in  research 
professionalising 
infrastructures.  Both  programmes  provided 
the  professional 
development  of  research  managers  while  using  different  support  measures.  Whereas  the 
WIDERA project aimed to foster longer-term structural changes by developing a roadmap for 
the future of RM in Europe and a community to support its delivery, the support under the RI 

training  of  managerial  and 

leadership  staff 

targeted  support 

the 

for 

222 Frigge, W., Evaluation Report EuroScience Open Forum (ESOF2022), available at: 
https://leiden2022.nl/sites/default/files/2023-
06/D3.2%20ESOF2022%20Programme%20and%20evaluation%20report.pdf 
223 https://www.esof.eu/marie-sklodowska-curie-actions-msca/ 
224 http://ritrain.eu/ 
225 https://ritrainplus.eu 

186 

 
 
programmes included training and competency requirements for current and future managers 
of European RIs. 

Furthermore,  the  complementarities  between  the  ‘Strengthening  the  ERA’  part  and  RIs 
extend to their joint support for Open Science and the re-usability/exploitation of research 
results. Whereas the RI programme supports the development of European infrastructure 
that  enables  Open  Science  and  the  sharing  of  research  results  (e.g.  through  support  for 
initiatives such as the European Open Science Cloud), the ERA WP supports the systematic 
gathering  of  evidence  on  the  impacts  of  Open  Science  practice  (e.g.  ‘Modelling  and 
quantifying the impacts of Open Science practice’) and supporting the capacity building of 
institutions/institutional transformation in the field of Open Science (e.g. actions to support 
capacity-building for institutional open access publishing across Europe). 

According to the WIDERA work programme 2021-2022, projects in the area of acceleration 
services for universities should link and build on the results of the initiative of the European 
Institute of Innovation and Technology (EIT) on Innovation Capacity Building for Higher 
Education,  launched  in  2021,  which  has  a  similar  objective  to  unlock  the  full  innovation 
potential of HEIs226. 

Significant coherence between WIDERA and other Horizon Europe programme parts is also 
evidenced by the collaborative efforts taken during project implementation. According to the 
survey results, a significant share of beneficiary ERA WP organisations indicated that at least 
one planned activity in their project is implemented in collaboration with other Horizon Europe 
projects (45%, or 50 out of 111 respondents). These activities encompass joint conferences, 
dissemination activities, workshops or publications and others. Figure 38 indicates that most 
collaborations are planned or implemented in cooperation with projects under the ‘Widening 
participation and spreading excellence’ thematic priority of WIDERA, followed by the MSCA, 
and other projects under WIDERA’s ‘Reforming and enhancing the European R&I system’ 
thematic priority. 

Coherence with other FP programme parts is also evidenced by a high degree of continuity 
in WIDERA (ERA priority) projects with regard to programmes under previous FPs: around 
37% (41 out of 111) of beneficiary respondents indicated that their project is a continuation 
of  research  activities  carried  out  under  previous  Framework  Programmes  (Horizon 
2020/FP7). For more details, see Figure 39. 

226 Horizon Europe Work Programme 2021-2022: 11. Widening participation and strengthening 
the European Research Area. European Commission Decision C(2022)2975 of 10 May 2022. 

187 

 
 
 
 
Figure 38. WIDERA-ERA-BEN-022: ‘Are there any activities planned in your project that are implemented in collaboration 
with projects funded under other Horizon Europe? (multiple answer options are possible)’, n=111. Source: survey of Horizon 
Europe beneficiaries, conducted May-July 2023. 

Figure 39. WIDERA-ERA-BEN-023: ‘Is your Horizon Europe project a continuation of research activities carried out under 
previous Framework programmes/other funding schemes? (in terms of being based on the work carried out in the past 
research project). If yes, please specify which programme’. Source: survey of Horizon Europe beneficiaries, conducted May-
July 2023. 

188 

 
 
 
External coherence 

Some of the activities supported under Horizon Europe WIDERA under  the ‘Strengthening 
the  ERA’  thematic  priority  are  directly  targeted  towards  fostering  complementarities  and 
synergies  between  national  and  regional  R&I  programmes,  while  also  ensuring  the 
complementarity of project activities and policies with those of the Framework Programme 
and  relevant  European  Partnerships  and  EU  missions.  In  particular,  support  for 
programme-level collaboration between national R&I policy-makers is expected to result 
in  the  identification  of  common  research  and  innovation  priorities  agreed  among  the 
participating national and regional R&I programmes, and the implementation of multi-annual 
joint calls and other joint R&D activities between participant Member States. In particular, this 
group  of  actions  under  the  ERA  part  of  Horizon  Europe  complements  the  European 
Partnerships identified in the Strategic Plan for Horizon Europe by providing the possibility 
for Member States, associated countries and civil society organisations to maintain existing 
collaborations and establish new ones on priorities of their choosing227. However, as already 
mentioned in the project implementation overview, the progress of WIDERA in this field 
has so far been limited, with only three grants for programme-level collaboration between 
national R&I policy-makers signed since 2021. 

The  available  evidence  also  indicates  that  there  are  significant  potential  synergies 
between  Horizon  Europe  ERA  projects  and  Erasmus+  –  in  particular,  the  European 
University  Alliances  initiative.  The  relevant  case  study  found  that  a  project  consortium 
implementing activities in the field of  the professionalisation of  research management and 
aiming to develop a roadmap for the RM profession involves one of the European University 
Alliances as an associated partner. This collaboration is expected to result in mutual benefits 
and synergies: the results of this ERA project are highly relevant to the development of the 
skills and competences of research managers in leading European University Alliances, while 
the  involvement  of  an  Alliance  in  a  WIDERA  project  provides  the  project  consortium  with 
necessary  knowledge  in  the  field  of  research  infrastructure  management.  Moreover, 
WIDERA support in the field of fostering  the professionalisation of RM is also expected to 
contribute to the European University Alliances by identifying ways and measures to make 
RM  positions  compatible  between  different  countries  –  a  challenge  faced  by  multiple 
Alliances.  Synergies  between  ERA  WP  projects  focusing  on  the  professionalisation  and 
development  of  RM  and  European  University  Alliances  is  further  confirmed  by  the  recent 
progress report on University Alliance Projects supported under Horizon 2020228. This report 
emphasises the crucial role of professional research managers in the success of Alliances. 
Such  collaborations  will  ensure  that  there  are  leaders  who  have  a  full  overview  of  an 
institution's  research  capacities,  and  the  authority  to  make  decisions  regarding  resource 
allocation. 

In  some  cases, ERA  projects  have  also  been  complementary  with  equivalent  national-
level initiatives in Member States. For example, the EU Contest for Young Scientists builds 
on national events, as participants in the EU-level contest of young scientists are projects 
nominated by the national organiser in each participating country (typically, the winners of 
contests organised at national level). 

227 Horizon Europe work programme 2023-2024: 11. Widening participation and strengthening 
the European Research Area. European Commission Decision C(2023) 2178 of 31 March 2023 
228 O’Neill, G., & Acheson, H., Progress of University Alliance Projects: Projects funded under 
Horizon 2020, Luxembourg: Publications Office of the European Union, 2023, available at: 
https://op.europa.eu/en/publication-detail/-/publication/ef36cf58-e57e-11ed-a05c-
01aa75ed71a1/language-en/format-PDF/source-285015820 

189 

 
 
Coherence  between  Horizon  Europe  ERA  projects  and  other,  non-FP  funding  schemes  is 
also evidenced by complementarity in terms of funding. According to survey evidence, around 
19%  of  beneficiaries  (21  out  of  108)  applied  for  additional  funding  for  the  research 
idea/activities  addressed  in  their  Horizon  Europe  project  (including  applications  for 
national/regional  R&D  public 
funding  schemes, 
international research funding schemes, private R&D funding or other sources). Moreover, 
the majority of these organisations (57%, or 12 out of 21) were granted all, or at least part, of 
the additional funding that they applied for (For more details see Figure 40). 

funding  schemes,  other  European 

Figure 40. WIDERA-ERA-BEN-020: ‘Have you applied for any additional funding for the research idea/activities addressed 
in your Horizon Europe project? (multiple answer options possible)’, n=108. Source: survey of Horizon Europe beneficiaries, 
conducted May-July 2023. 

1.6.1.3. 

EU added value 

Quantitative  and  qualitative study  evidence  shows  that  Horizon  Europe  support  under the 
ERA  thematic  priority  provided  substantial  added  value  (both  for beneficiary  organisations 
and  for the  EU  as  a  whole),  compared  with  what  would  otherwise  have  been  available  at 
national or regional level. This EU added value mostly stemmed from supporting cross-border 
collaborative projects that could not have been implemented at Member State level. 

One the most significant areas of this added value provided by Horizon Europe support for 
ERA actions was the opportunity provided to beneficiary organisations to form diverse, 
cross-border networks of expertise that are crucial to the achievement of the projects’ 
results, and which could not be achieved without EU support. According to the results 
of the Horizon Europe beneficiaries survey, 89% of beneficiaries (84 out of 95) agreed that 
(compared  with  the  research  funding  available  to  them  at  national  and/or  regional  level) 
Horizon  Europe  ‘provides  more  diversity  in  the  partners’  profiles’  (multi-sectoral,  multi-
disciplinary  consortia,  more  geographical  diversity  of  partners,  etc.).  The  overwhelming 
majority of respondents also agreed that the programme ‘provides more international mobility 
opportunities for researchers’ (82%, 76 out of 93) (for more details, see Figure 41). This was 
largely  supported  by  case  study  evidence:  building  EU-level  networks  that  allowed  the 
development of roadmaps, frameworks and methodologies reflecting the needs of different 
EU countries, was emphasised as the main source of EU added value by multiple case study 
interviewees,  particularly  project  stakeholders  in  the  field  of  university  transformation.  A 

190 

 
 
number of case study interviewees also noted that the projects provided the opportunity to 
involve a wide range of stakeholders from a variety of regions and countries, which is the key 
source of knowledge for building tools and roadmaps that can be used at EU level. 

Contribution to excellence stemming from the higher level of competition at EU level 
(compared with regional or national levels) was also identified as one of the sources of 
the  EU  added  value  of  Horizon  Europe  ERA  projects.  The  majority  of  beneficiary 
organisations  surveyed  indicated  that,  compared  with  actions  at  national/regional  level, 
Horizon  Europe  involves  a  ‘higher  level  of  competition  for  research  funding’  (77%  of 
respondents agreed; 76 out of 94). In-depth qualitative evidence stemming from a case study 
strongly  supports  the  above  findings  regarding  the  significant  added  value  brought  by  the 
European  dimension  of  the  projects  supported,  e.g.  a  higher  level  of  excellence  in  the 
activities and outputs stemming from EU-wide competition is exemplified by EUCYS (the EU 
Contest for Young Scientists). In interviews, stakeholders confirmed that the EU-wide contest 
brought  together  the  most  talented  young  people  who  had  won  national-level  science 
contests.  This  raised  the  event  to  higher  level,  increasing  the  excellence  of  the  projects 
presented by contestants. A significant aspect of this EU added value also stemmed from the 
opportunity the event provided for contestants to interact with peers and researchers from 
various countries. 

Similarly, in the beneficiary survey, the majority of respondents agreed that Horizon Europe 
provides  beneficiary  organisations  with  funding  for  research  topic(s)  or  field(s)  not 
covered  in  national  or  regional  R&I  funding  programmes  (76%;  71  out  of  93 
respondents).  Other  significant  sources  of  the  programme’s  added  value  reported  by 
beneficiaries  (compared  with  what  would  have  been  available  at  national  level)  include 
pooling  necessary  skills  and  expertise  necessary  for  research,  addressing  grand 
societal challenges, and a higher amount of funding (for more details, see Figure 41). 

The  significant  additionality  brought  by  EU  support  is  also  confirmed  by  unsuccessful 
applicants under the ERA thematic priority. A large majority of respondents to the survey 
of  unsuccessful  applicants  (81%,  or  67  out  of  83)  indicated  that  they  had  not 
implemented their project in absence of Horizon Europe support (i.e. they did not apply 
for additional funding, or applied for alternative funding but were not successful). Less than 
16%  of  unsuccessful  applicants  (13  out  of  83)  indicated  they  had  applied  for  alternative 
funding,  with  slightly  over  half  of  them  (8  out  of  13)  reporting  that  their  application  for 
alternative  funding  (from  national/regional  or  other  alternative  funding  sources)  had  been 
successful  and  that  they  had  gone  ahead  with  the  project  or  activities  proposed  in  their 
Horizon  Europe  application.  This  shows  that  at  the  time  of  evaluation,  less  than  10%  of 
unsuccessful  applicants  (8  out  of  83)  received  alternative  funding  for  the  activities 
planned in their WIDERA project. 

191 

 
 
Figure 41, WIDERA-ERA-BEN-045: ‘Would you agree or disagree that, compared to the research funding available to you at 
national and/or regional level, Horizon Europe’. Source: survey of Horizon Europe beneficiaries, conducted May-July 2023. 

The above findings regarding the substantial EU added value provided by Horizon Europe 
ERA support (compared with what would have been available at national level) are further 
confirmed  by  the  unsuccessful  applicant  organisations’  assessments  of  the  reasons  that 
prevented  them  from  implementing  their  projects  without  Horizon  Europe  funding.  The 
majority  of  unsuccessful  applicant  respondents  (92%, or 55  out  of  60)  reported  as  a ‘very 
important’  or  ‘important’  reason  the  unavailability  of  alternative  funding  for  their  specific 
research  topic/area.  Similarly,  89%  (55  out  of  62)  indicated  that  their  project  required  the 
involvement of a variety of international partners, and could not be implemented without them. 
In general, the majority of respondents also confirmed that without Horizon Europe support, 
they could not implement projects that addressed EU-level (and/or global) challenges; they 
could  not  access  necessary  knowledge,  expertise  or  skills;  or  they  could  not  access  the 
necessary research infrastructures (for more details, see Figure 42). 

192 

 
 
Figure 42. WIDERA-ERA-APPL-029: ‘How important were the following reasons that prevented your organisation from 
implementing the project without Horizon Europe funding?’ Source: survey of Horizon Europe unsuccessful applicants, 
conducted May-July 2023 

As previously mentioned in the overview of project implementation under the WIDERA ERA 
thematic priority by keyword (see Section 5.2.1 above), the majority of ERA projects have 
focused  on  cross-cutting  and  transversal  issues  –  particularly  Open  Science  and  the  re-
usability of research results; responsible research and innovation; research ethics; gender 
equality; inclusion; citizen engagement in science; and other related topics. Bearing in mind 
the above figures regarding the additionality of Horizon Europe support, and that the majority 
of the projects could not be implemented without EU support (i.e. at national/regional level), 
the available evidence confirms the very significant EU added value of ERA WP in addressing 
cross-cutting and transversal issues. 

1.6.1.4. 

Relevance 

The  actions  supported  under  Horizon  Europe  WIDERA  ‘Strengthening  the  ERA’  thematic 
priority are closely aligned with and relevant to the ERA Policy Agenda229, which translates 
the Pact for Research and Innovation230 into 20 concrete actions. The support provided under 
ERA  work  programmes  2021-2022  and  2023-2024  addresses  the  ERA  Policy  Agenda 
actions,  although  this  alignment  is  more  focused  and  explicit  in  the  WP  2023-24231.    An 
assessment of the ‘Strengthening the ERA’ budget distribution by ERA policy action shows 
that in the ERA WP 2021-22, the largest share of the budget (around EUR 32 million) was 
dedicated to Action 14 (‘Bring science closer to citizens’)232, followed by Action 1 (‘Enable the 

229 European Research Area Policy Agenda – Research and Innovation Overview of actions for 
the period 2022-2024. European Commission, 2021. 
230 Council Recommendation on a Pact for Research and Innovation in Europe, available at: 
https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32021H2122 
231 Internal Commission document. 
232 This includes the following topics: HORIZON-WIDERA-2021-ERA-01-44: Societal trust in 
science, research and innovation; HORIZON-WIDERA-2021-ERA-01-60: A capacity-building 

193 

 
 
 
 
open  sharing  of  knowledge  and  the  re-use  of  research  outputs’)  and  Action  4  (‘Promote 
attractive  and  sustainable  research  careers,  balanced  talent  circulation  and  international, 
transdisciplinary and inter-sectoral mobility across the ERA’), each with a budget of around 
EUR 20 million. In the WP 2023-24, the highest share of the budget so far has been allocated 
to  Action  4  (EUR 17.5  million),  followed  by  Action  5  (‘Promote  gender  equality  and  foster 
inclusiveness’) and Action 10 (‘Make EU R&I missions and partnerships key contributors to 
the ERA’) – each with a budget of around EUR 12 million233. 

Non-exhaustive  mapping  of  WIDERA  working  programme  topics  against  the  ERA  Policy 
Agenda  shows  that  the  ERA  thematic  priority  has  been  highly  relevant  and  has  directly 
addressed the following policy actions:  

•  WIDERA  (ERA)  actions  in  the  field  of  gender  equality  (e.g.  support  for  the  Centre  of 
excellence  on  inclusive  gender  equality  in  Research  &  Innovation;  Support  to  the 
implementation  of  inclusive  gender  equality  plans;  Policy  coordination  to  advance  the 
implementation of the ERA gender equality and inclusiveness objectives within Member 
States) were directly relevant to the ERA policy action 5 ‘Promoting gender equality 
and  fostering  inclusiveness,  taking  note  of  the  Ljubljana  Declaration’.  A  total  of  eight 
topics under WIDERA Work Programme 2021-2022 and 2023-2024 directly supported 
ERA policy action 5. 

•  WIDERA  (ERA)  support  aimed  at  fostering  structural  changes  in  the  field  of  Open 
Science (e.g. Capacity-building for institutional open access publishing across Europe; 
Support  to  changes  in  the  assessment  of  research  and  researchers  to  reward  the 
practice  of  open  science;  support  to  Citizen  Science  and  Responsible  Research  and 
Innovation, etc.) is highly relevant to the EU’s eight ambitions in Open Science234, as well 
as to the ERA Policy Agenda’s 2022-2024 policy action 1 ‘Enable the open sharing 
of knowledge and the re-use of research outputs, including through the development 
of the European Open Science Cloud (EOSC)’; 

•  Similarly, actions to transform universities/acceleration services (including, for example, 
Support to reforms of research assessment in the European Research Area) are also 
highly  relevant  to  addressing  a  number  of  different  objectives  under  the  ERA  Policy 
Agenda strategic priority ‘Deepening a truly functioning internal market for knowledge’. 
Such  actions  include  Action  3,  Reforming  the  assessment  systems  for  research, 
researchers  and  institutions;  Action  4,  Promoting  attractive  and  sustainable 
research  careers  and  the  mobility  of  researchers;  Action  5,  Promoting  gender 
equality and  fostering  inclusiveness,  and  so  on.  Moreover, this  group  of  actions is 

and brokering network to make citizen science an integral part of the European Research Area; 
HORIZON-WIDERA-2021-ERA-01-61: Supporting and giving recognition to citizen science in the 
European Research Area; HORIZON-WIDERA-2021-ERA-01-70: Developing a STE(A)M 
roadmap for Science Education in Horizon Europe; HORIZON-WIDERA-2022-ERA-01-60: A 
European competence centre for science communication; HORIZON-WIDERA-2022-ERA-01-
70: Open schooling for science education and a learning continuum for all; HORIZON-WIDERA-
2021-ESOF-IBA: The EuroScience Open Forum (ESOF) and European City of Science 2022 
actions; HORIZON-WIDERA-2021-EUCYS-IBA: European Union Contest for Young Scientists 
(EUCYS) 2022. 
233 Ibid. 
234 https://research-and-innovation.ec.europa.eu/system/files/2019-12/ec_rtd_factsheet-open-
science_2019.pdf 

194 

 
 
highly relevant to the priorities of the European Education Area (EEA)235 in relation 
to  boosting  institutional  transformation  and  the  harmonisation  of  practices  across 
institutions (automatic recognition of qualifications and study periods abroad, a stronger 
focus on specialised education programmes in the areas relevant for addressing societal 
challenges  (digital  skills,  entrepreneurship,  green  transition,  AI,  etc.)).  Interviews  also 
confirmed that actions in field of science communication (such as the European City of 
Science or the EU Contest for Young Scientists), play a pivotal role in identifying young 
talents,  motivating  them,  helping  them  to  grow,  and  encouraging  them  to  choose 
scientific careers. 

•  FP  actions  under  WIDERA  (ERA  thematic  priority)  in  the  area  of  promoting  citizen 
science/science education and communication (e.g. Societal trust in science, research 
and  innovation;  Supporting  and  giving  recognition  to  citizen  science  in  the  European 
Research  Area;  European  competence  centre  for  science  communication;  European 
Union Contest for Young Scientists (EUCYS), etc.) are also closely aligned with bringing 
science  closer  to  society  (Action  14  of  the  European  Research  Area  Policy 
Agenda).  This  is  achieved  primarily  by  promoting  stronger  public  understanding  of, 
connection to, and engagement with science, which also brings science closer to societal 
challenges. A number of interviewees indicated that EU projects such as the European 
City  of  Science  were  of  particular  relevance  in  the  context  of  the  COVID-19 
pandemic, during which society’s trust in science and scientists was of paramount 
importance. Activities that brought science closer to the general public were particularly 
relevant in this context, and helped to nurture and maintain public’s trust in science. 

Study evidence also shows that Horizon Europe ERA WP actions were mostly relevant to 
addressing  the  needs  of  beneficiary  institutions  to  increase  their  international 
visibility and strengthen their collaborative ties with other organisations. In the survey 
of  ERA  WP  beneficiaries,  80%  (82  out  of  102)  of  respondents  agreed  to  a  ‘very  large’  or 
‘large’  extent  that  their  project  responded  to  their  organisation’s  need  to  ‘enhance  the 
international visibility (across the borders of your country)’. This was followed by the need to 
‘create or strengthen collaboration with leading research organisations’ (77%, or 79 out of 
103 respondents) (see Figure 43). A number of beneficiary stakeholders interviewed likewise 
confirmed that their project is highly relevant in the context of their institution’s need to build 
connections with peers in other countries, to build networks of expertise, and to be able to 
learn from the good practice examples of institutions in other countries.  

WIDERA  projects  under  the  ‘Strengthening  the  ERA’  thematic  priority  were  also  highly 
relevant to the institutions’ to develop their capacities and skills of staff, in order to better 
respond  to  current  socioeconomic  needs,  and  to  increase  competitiveness  and  the 
overall  knowledge  base.  Case  study  findings  suggest  that  by,  for  instance,  helping 
universities  to  build  the  competencies  needed  by  researchers  and  facilitating  the 
commercialisation  of  research  findings,  acceleration  services  make  European  HEIs  better 
aligned  to  the  needs  of  the  market  and  more  capable  of  forging  alliances  with  the  private 
sector. Likewise, 65% (66 out of 102) of ERA WP beneficiary respondents surveyed agreed 
to  a  ‘very  large’  or  ‘large  extent’  that  their  project  responds  to  their  institution’s  ‘need  to 
enhance  the  R&D  and/or  innovation  capabilities  and  skills’,  followed  by  the  need  to 
‘strengthen the knowledge base and R&D capacity in their organisation’ (63% or 65 out of 
102  respondents),  as  well  as  to  ‘enhance  their  competitiveness’  (61%,  or  62  out  of  101 
respondents) (see Figure 43). 

235 Communication from the Commission to the European Parliament, the Council, the European 
Economic and Social Committee and the Committee of the Regions on achieving the European 
Education Area by 2025. 

195 

 
 
 
 
Horizon Europe support under the ERA WP has also been highly relevant to addressing the 
need  of  institutions  (particularly  universities)  to  improve  the  working  conditions  of 
researchers  by  streamlining  processes  and  internal  practices.  According  to  the  case 
study findings, some of the projects were particularly relevant in terms of helping institutions 
and research managers to streamline human research capacity management and reduce the 
administrative  burden/burden  on  researchers  in  relation  to  non-research  activities.  This  is 
highly relevant for institutions that need to save resources or make more efficient use of their 
existing  resources.  The  relevance  of  ERA  actions  to  these  institutions’  needs  for  internal 
transformation  is  also  confirmed  by  the  survey  results: 59% (60  out  of  101) of  beneficiary 
respondents agreed that their projects respond to a ‘very large’ or ‘large’ extent to the need 
to ‘promote institutional change in their organisation, region or country’ (see Figure 43). ERA 
projects  were  also  relevant  to  addressing  a  closely  related  need  of  institutions  to  make 
research careers more attractive and to attract young talents. According to a number of 
project stakeholders who were interviewed, acceleration services supported under WIDERA 
(including  training  and  counselling  services  for  beneficiary  institutions)  helped  to  equip 
institutions  with  the  capacities  needed  to  implement  changes  (e.g.  in  the  area  of  gender 
equality for researchers, career counselling and development, etc.), making research careers 
more attractive. 

Figure 43. WIDERA-ERA-BEN-025: ‘To what extent does your Horizon Europe project respond to the following needs of 
your organisation?’ Source: survey of Horizon Europe beneficiaries, conducted May-July 2023. 

196 

 
 
 
1.6.1.5. 

Conclusions and lessons learned 

Conclusion  1:  the  actions  supported  within  the  thematic  priority  ‘Strengthening  the  ERA’ 
under  Horizon  Europe  WIDERA  directly  address  and  closely  align  with  the  European 
Research  Area  (ERA)  Policy  Agenda  2022-2024.  Specifically,  these  actions  address 
priorities such as bringing science closer to citizens, promoting Open Science and  the re-
usability of research results, fostering gender equality in research, promoting sustainable and 
attractive  research  careers,  facilitating  balanced  talent  circulation,  and  encouraging 
responsible research and innovation (RRI). The actions that fall under the thematic priority 
‘Strengthening the ERA’ have proved relevant in addressing the needs of institutions that aim 
to enhance their international visibility and establish collaborative relationships with partners 
across  borders.  These  actions  aim  to  develop  the  competencies  of  staff,  enhance 
competitiveness,  and  enable  institutions  to  better  respond  to  the  pressing  socioeconomic 
challenges  faced  by  HEIs.  In  addition,  ERA  actions  contribute  significantly  to  rendering 
research careers more attractive, primarily by supporting organisations and bolstering their 
capacities  for  institutional  reform.  These  efforts  collectively  contribute  to  creating  an 
environment  that  fosters  the  attractiveness  of  research  careers  and  promotes  positive 
changes within institutions. 

Conclusion  2:  Since  the  inception  of  the  programme,  the  thematic  priority  ‘Strengthening 
the European Research Area’ within WIDERA has exhibited a relatively high success rate of 
37%. However, compared with other programme components analysed in the study, the ERA 
thematic priority also shows the highest rate of ineligible or inadmissible proposals. This trend 
has,  however,  not  occurred  consistently  throughout  the  entire  programme  part,  as  the 
majority of ineligible or inadmissible proposals were concentrated in specific topics and call 
years. 

Conclusion 3: the ‘Strengthening the ERA’ thematic priority contributes to the development 
of  a  unified  European  Research  Area  primarily  by  supporting  cross-border  networks  of 
expertise,  sharing  good  practices,  developing  knowledge  and  skills,  and  empowering 
institutions  to  undertake  reforms.  One  of  the  most  unique  aspects  of  projects  under  this 
thematic priority is their role in setting new policy standards and providing recommendations 
for  policy-makers.  The  majority  of  beneficiaries  (82%)  indicated  that  their  projects  have 
already resulted in, or are expected to result in, recommendations for policy-makers or inputs 
into national or EU policies. The effectiveness of the ‘Strengthening the ERA’ thematic priority 
also extends to addressing cross-cutting issues (CCIs), with a significant number of projects 
focusing  on  topics  such  as  citizen  science,  gender  equality,  inclusiveness  in  research, 
responsible research and innovation (RRIs), research ethics, Open Science, and exploitation 
of research results. The contribution of ERA actions to policy/programme-level collaboration 
among policy-makers has been more limited, due to the limited interest of policy-makers and 
the funding of few grants on this topic. 

Lesson learned 3.1: to address the low level of interest among national policy-makers 
on the topic of programme-level collaboration between national R&I policy-makers, it is 
recommended  to  conduct  stakeholder  consultations  with  policy-makers  (e.g.  through 
focus group discussions or interviews). The main objective of such consultation activities 
would be to identify their priorities, challenges and areas of interest within the specific 
funding  topic  of  programme-level  collaboration.  This  exercise  would  provide  crucial 
insights  into tailoring the funding topic to  better align  with the concerns and needs of 
potential beneficiary policy-makers. 

Conclusion  4:  the  mix  of  actions  under  the  ‘Strengthening  the  ERA’  thematic  priority 
complements other parts of the Horizon Europe FP, primarily by supporting dedicated actions 

197 

 
that  contribute  to  common  overarching  strategic  goals.  Specifically,  ERA  projects 
complement  the  MSCA  by  strengthening  Europe’s  human  capital  base  and  improving  the 
working conditions of researchers, thereby enhancing the attractiveness of research careers. 
Project-level collaboration between the ‘Strengthening the ERA’ thematic priority and other 
programme  parts  is  evident,  with  almost  half  (46%)  of  ERA  projects  planning  joint 
collaborative activities such as mutual conferences, joint dissemination activities, workshops 
and joint publications with other Horizon Europe projects. Significant complementarities and 
synergies  also  exist  between  the  ‘Strengthening  the  ERA’  thematic  priority  and  non-FP 
initiatives, particularly the Erasmus+ European University Alliances initiative. Both initiatives 
contribute  to  common  strategic  goals,  including  the  professionalisation  of  research 
management  in  Europe.  In  addition,  complementarities  exist  between  WIDERA  (ERA 
thematic priority) and national/regional and other non-FP funding sources. Nearly one-fifth 
(19%)  of  ERA  project  beneficiaries  applied  for  additional  national/regional/private  or 
international  (non-FP)  funding  for  the  research  ideas/activities  addressed  in  their  Horizon 
Europe project, with over half of them receiving at least part of this additional funding. 

The  opportunity  provided  to  beneficiary  organisations  to  establish  diverse,  cross-border 
networks of expertise, which would be unattainable without EU support, constitutes one of 
the  primary  sources  of  EU  added  value  in  the  ‘Strengthening  the  ERA’  thematic  priority. 
Furthermore,  Horizon  Europe  provides  beneficiary  organisations  with  funding  for research 
topics not covered by national or regional R&I funding programmes. This provides access to 
higher amounts of funding, and facilitates the pooling of the necessary skills and expertise 
required for research, as well as the opportunity to address grand societal challenges. The 
significant additionality of EU funding for WIDERA projects under  ‘Strengthening the ERA’ 
thematic priority is also evidenced by less than 10% of unsuccessful applicants applying and 
managing  to  secure  alternative  funding  (from  national/regional,  private  or  other  non-FP 
sources) for the activities planned in their project. 

Conclusion 5: the analysis of external topics carried out in the analysis of research trends, 
clearly shows the effect of Brexit on the topics funded. The UK has been an integral part of 
the European Research Area and of EU academic environment, strengthening topics that are 
of importance to the EU-27 and to the EC, as well as relying on EU research to fill in domestic 
gaps. In addition to this, the UK funds topics that are not covered by the EC, and less so by 
the EU. It is of importance to the EU that it can profit from this expertise in order to plug any 
potential gaps in its knowledge portfolio. 

Lesson learned 5.1:  closer cooperation with the UK in terms of research activities is 
desirable. In terms of Excellent Science and based on the data from the present study, 
the re-admission of the UK to Horizon Europe and the ERA as an  associated country 
has  been  a  logical  step  for  both  sides.  A  further  strengthening  of  this  research 
cooperation calls for an extension of this arrangement beyond 2027. 

1.6.2.  Widening participation and spreading excellence 

The premise of ‘Widening Participation and Spreading Excellence’ is that R&I policy can only 
achieve  greater  excellence  if  everyone  in  the  EU  progresses.  Despite  much  progress 
having  been  made  in  developing  the  ERA,  large  disparities  remain  between  the  leading 
countries in terms of R&I performance, and those that lag236. The EU R&I system thus needs 

236 European Commission Decision C(2022)2975. Horizon Europe Work Programme 2021-2022. 
11. Widening participation and strengthening the European Research Area. 

198 

 
 
a more inclusive approach under which all EU countries can participate and benefit237. The 
widening actions aim  to foster the participation in the Framework Programme  of countries 
lagging in R&I, and to assist them in strengthening their R&I systems. Specifically, the actions 
strengthen collaborative links across Europe and open European R&I networks. They also 
contribute to improving research management capacities in widening countries, supporting 
national policy reforms, as well as exploiting the potential of the Union's talent pool through 
targeted  actions.  The  Widening  actions  have  received  3.3%  of  the  total  Horizon  Europe 
budget,  with  EUR  3.212  million  for  the  programme  part  'Widening  participation  and 
strengthening the ERA' for the period 2021-2027, of which EUR 2.842 million is for 'Widening 
participation and spreading excellence', according to the Horizon Europe Regulation.  

Under Horizon Europe238, the Member States eligible as coordinators of widening actions 
are Bulgaria, Croatia, Cyprus, Czechia, Estonia, Greece, Hungary, Latvia, Lithuania, Malta, 
Poland, Portugal, Romania, Slovakia, and Slovenia. The associated countries eligible are 
Albania,  Bosnia  &  Herzegovina,  Kosovo,  Montenegro,  North  Macedonia,  Serbia,  Turkey, 
Armenia, Georgia, Moldova, Morocco, Tunisia, Ukraine, and the Faroe Islands239. In addition, 
outermost  regions  (as  defined  in  Article  349  of  the  Treaty  on  the  Functioning  of  the  EU 
(TFEU))  are  also  fully  eligible  as  coordinators  under  this  component  –  i.e.  Guadeloupe, 
French Guiana, Martinique, Mayotte, Réunion and Saint Martin (France); the Canary Islands 
(Spain); and the Azores and Madeira (Portugal)240. All of the above countries and territories 
are referred to as ‘widening countries’.  

According  to  the  work  programme  2021-2022  and  2023-2024,  the  key  objectives  of  this 
component of the programme part ‘Widening Participation and Strengthening the ERA’ are 
as follows: 

1.  Encourage institutional reforms and transformation processes of the R&I system at 

national and regional level in widening countries in line with ERA principles; 

2.  Mobilise national investments in R&I capacity in widening countries; 

3.  Raise the bar for excellence of R&I actors in widening countries in partnership with 

outstanding European and international institutions (‘win-win situation’); 

4. 

Increase  the  number  of  participations  and  success  rates  of  widening  actors  in 
research and innovation projects in other parts of Horizon Europe (notably in Pillars 
2 and 3); 

5.  Promote  the  creation  of  new  innovation  ecosystems  and  scale  up  existing  ones 
through a set of measures that include, among others, place-based and international 
collaboration between academia and business in widening countries; 

6.  Foster  brain  circulation,  including  inter-sectoral  mobility  for  researchers  and 

innovators, and turn it into brain gain for widening countries. 

237 European Commission Decision C(2022)2975. Horizon Europe Work Programme 2021-2022. 
11. Widening participation and strengthening the European Research Area. 
238 Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 2021 
establishing Horizon Europe – the Framework Programme for Research and Innovation, laying 
down its rules for participation and dissemination, and repealing Regulations (EU) No 1290/2013 
and (EU) No 1291/2013 
239 https://rea.ec.europa.eu/horizon-europe-widening-who-should-apply_en 
240 https://eur-lex.europa.eu/EN/legal-content/glossary/outermost-regions.html  

199 

 
 
This  Widening  part  responds  to  the  second  policy  priority  of  the  ERA  communication, 
which  is  ‘improving  EU-wide  access  to  excellence’.  This  priority  is  aimed  towards  greater 
excellence and stronger R&I systems across the whole of the EU. It supports the widening 
countries to increase the performance of their R&I systems towards excellence by building 
on  dedicated  Horizon  Europe  measures  such  as 
the  Widening  actions,  and 
complementarities with the smart specialisation strategies under the Cohesion Policy.  

The  activities  of  the  Widening  actions  aim  to  strengthen  the  institutional,  scientific  and 
innovation  capacities  of  the  widening  countries  on  the  basis  of  international  as  well  as 
academic-business  partnerships.  Several  Widening  actions  are  a  continuation  of  the 
previous Framework Programme, namely: 

•  Teaming for Excellence: these actions support the modernisation of existing Centres of 
Excellence, and the creation of new ones, in widening countries on the basis of strategic 
partnerships with internationally leading excellent research institutions. The impact of this 
action  is  amplified  by  the  obligation  to  secure  complementary  investments  from  other 
sources such as Structural Funds.  

•  Twinning: these actions aim to strengthen a particular field of research in an emerging 
institution  through  knowledge  transfer and  the  exchange  of  best  practices  with leading 
institutions.  Specifically,  the  action  supports  networking  activities  between  research 
institutions  in  widening  countries,  and  at  least  two  different  top-class  leading  research 
institutions  from  two  different  EU  countries  or  associated  countries.  This  involves 
exchanges  of  researchers,  expert  visits,  workshops,  and  outreach  activities.  Some 
Twinning calls specifically address widening countries in the Western Balkans. 

•  ERA Chairs: this action brings high-level researchers to universities and other research 
organisations in widening countries with the aim of enhancing their research capacities. 
The action enables the establishment of a research team around each ERA Chair. These 
are  integrated  into  the  coordinator’s  widening  institution,  to  significantly  improve  its 
research  performance  in  a  scientific  domain  of  its  choice,  by  increasing  high-quality 
human  resources  and  implementing  the  structural  changes  necessary  to  achieve 
excellence on a sustainable basis. 

•  European  Cooperation  in  Science  &  Technology  (COST):  COST  is  an  EU-funded, 
to  set  up 
pan-European  programme 
interdisciplinary research networks in Europe and beyond. The action aims to improve the 
networking and scientific capabilities of widening countries.  

that  enables  researchers  and 

innovators 

Several new Widening actions have been introduced under Horizon Europe: 

•  European  Excellence  Initiative  (EEI):  this  action  will  engage  with  universities  and 
empower them further to be actors of change in R&I. It will raise excellence in science 
and in value creation through deeper and geographically inclusive cooperation through 
alliances of HEIs. 

•  Pathways to Synergies: the purpose of this action is to provide support for the setting-
up of interfaces between pairs of funding systems, between which barriers still occur due 
to mismatches of regional versus European approaches (e.g. the ERDF and INTERREG, 
or RRF funding and Horizon funding). The main goal of this pathway is to move formerly 
single  beneficiaries  of  regional  funding  programmes  out  of  isolation  via  cross-border 
collaboration and to prepare them for participation in Horizon Europe calls by means of a 
customised set of activities. 

200 

 
•  Dissemination and Exploitation Support Facility: this action will help beneficiaries of 
widening actions to improve the effectiveness of their dissemination and exploitation and 
unlock  new  sources  of  funding.  Improving  knowledge  diffusion,  technology  uptake  and 
achieving  spill-over  effects  is  fundamental  to  ensuring  that  researchers  and  their 
institutions build on the latest available knowledge. 

•  Hop On Facility: this facility allows research institutions from widening countries to join 
ongoing  research  and  innovation  actions  under  Horizon  Europe  Pillar  2  and  EIC 
Pathfinder. 

•  Excellence  Hubs:  these  hubs  will  strengthen  regional  innovation  excellence  through 
innovation  ecosystems  in  widening  countries  and  beyond,  by  teaming  up  and  creating 
robust linkages between academia, businesses, governments and civil society. 

•  ERA  Talents:  this  scheme  aims  to  boost  the  interoperability  of  careers  and  the 
employability of R&I talents across sectors, with a centre of gravity in widening countries. 
The  scheme  focuses  on  cross-sectoral  talent  circulation  and  academia-business 
collaboration for knowledge transfer. 

Through the above activities, the Widening actions aim to achieve a series of outputs: 

• 

Increased research excellence; 

•  Development of new R&I domains; 

• 

Improved capacity to compete for research funding; 

• 

Improved research management and administrative capacities at institutions; 

• 

Inducing structural changes in widening institutions; 

•  Attracting (international) top researchers; 

• 

Intersectoral and international mobility; 

•  Entering  into  or  establishing  new  international  and/or  interdisciplinary  networks,  or 

strengthening existing ones; 

• 

Inducing policy reforms; 

• 

Improving the quality of information, advice and training; 

•  Establishing or strengthening quadruple helix networks and ecosystems; and 

• 

Innovative R&I practices and mutual learning in regions and countries. 

In the medium term, the Widening actions first aim to achieve a couple of results relating to 
improved  access  to  excellence,  with  the  aim  of  building  up  R&I  capacities  in  a  widening 
country.  Some  of  the  desired  results  are  R&I  reforms  and  investments  at  national  and 
regional levels, as well as partnerships with leading institutions abroad. Second, the Widening 

201 

 
actions aim to achieve results related to attracting and mobilising the best  talents, such as 
developing skills for excellent science and connecting all actors across Europe. 

In the long term, the Widening actions are expected to contribute to the following impacts: 

• 

Increased science and innovation capacities for all actors in the R&I system in widening 
countries; 

•  Structural changes leading to modernised and more competitive R&I systems in eligible 

countries; 

•  Reformed R&I systems and institutions, also leading to increased attractiveness and the 

retention of research talents; 

•  Mobilisation of national and European resources for strategic investments; 

•  Stronger  linkages  and  mobility  between  academia  and  business,  and  improved  career 

permeability; 

•  Transformation  of  a  strengthened  role  for  the  higher  education  sector  in  research  and 

innovation; 

•  Greater involvement of regional actors and society in the R&I process; 

•  Better use of existing research infrastructures; 

•  Reversal of the brain drain; and 

•  Free circulation of knowledge and expertise in line with ERA priorities. 

202 

 
 
Note: ‘WID.’ refers to ‘Widening’. Source: compiled by the authors, based on the Horizon Europe WIDERA 2021-2024 work programmes and the Horizon Europe Regulation.

Figure 44 Intervention logic for ‘Widening Participation and spreading excellence under Horizon Europe’ 

203 

 
 
1.6.2.1. 

Effectiveness 

To  overcome  disparities  between  EU  Member  States  with  regard  to  participation  in  the 
Framework Programme and in terms of R&I performance, Widening measures have been 
introduced to tackle large imbalances between leading countries in R&I, and those lagging. 
This section provides preliminary insights into the Widening actions under Horizon Europe to 
the  extent  that  these  are  available,  and,  to  a  limited  extent,  focuses  on  the  long-term 
performance  and  impacts  of  Widening  actions  under  Horizon  2020.  The  main  findings  in 
relation to the effectiveness of the Widening actions are structured according to the following 
key topics: 

•  Fostering scientific excellence; 

•  Building capacity, attracting and retaining talent, and fostering mobility; 

• 

Inducing structural changes; 

•  Fostering collaboration in transnational networks and strengthening ecosystems; and 

•  Focussing on innovation, commercialisation of results and private sector involvement. 

Fostering scientific excellence 

Given  that  research  excellence  plays  a  crucial  role  in  achieving  success  within  the 
Framework Programmes, the Widening actions are dedicated to promoting the generation of 
scientific  excellence  in  Member  States  and  associated  countries  with  comparatively  lower 
levels of R&I performance. 

Historically, Widening actions have contributed to increasing the research excellence of the 
Union.  Even  though  the  Widening  actions  mainly  represent  Coordination  and  Support 
Actions, the Evaluation study on Excellent Science under Horizon 2020241 underscored the 
significance  for  widening  countries  of  Widening  in  terms  of  the  production  of  science.  In 
addition, the Widening actions are key to the production of excellent science in the widening 
countries, with 28% of the most highly cited Horizon 2020 publications by widening countries 
being  produced  because  of  Widening  actions.  Furthermore,  the  Widening  actions  have 
contributed to the improved international positioning and reputation of widening researchers 
and research groups.  

Under Horizon Europe, several novelties have been introduced that aim to enhance scientific 
capacity and research excellence. For instance, the newly introduced Hop On Facilities now 
offer the possibility for widening institutions to join ongoing collaborative R&I actions with the 
aim  of,  amongst  other  things,  improving  research  excellence  in  widening  institutions. 
Excellence Hubs, meanwhile, should improve access to excellence for R&I actors in widening 
countries (according to the WIDERA work programmes 21-22 and 23-24). 

While there is insufficient bibliometric information available yet to draw conclusions regarding 
the Widening actions under Horizon Europe, the survey results242 of the survey of Horizon 

241 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R. et al., Evaluation study of the European framework programmes 
for research and innovation for excellent science: Horizon 2020: Annexes, Luxembourg: 
Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383 
242 Survey of Horizon Europe WIDERA (widening) beneficiaries, conducted May-July 2023. 

204 

 
 
Europe  successful  participants  in  WIDERA  Widening  point  to  positive  expected  scientific 
outcomes:  

• 

89.3%  of  respondents  indicated  that  their  project  has  led  or  will  lead  to  research 
publications243, and 60.6% agreed or strongly agreed with the statement that their project 
is pushing or will push the frontiers of knowledge244.  

•  Over three-quarters of the WIDERA Widening participants who responded to the survey 
agreed  that  their  project  strengthens  the  knowledge  base  and  R&D  capacity  in  the 
organisation245  and  more  than  90%  indicated  that  their  project  enhances  the 
organisation’s international visibility to a ‘large’ or ‘very large’ extent246.  

• 

In  addition,  70.8%  agreed  that  the  project  would  probably  increase  the  research 
productivity247.  

•  A majority of unsuccessful applicants to WIDERA Widening who applied for alternative 
funding  expect  a  smaller amount  of  research  outputs  and  innovations  to  be  produced 
compared with those planned in their Horizon Europe application248.  

In this regard, beneficiaries value the fact that a small research (and innovation) component 
may be included into several Widening projects. Beneficiaries indicate that this is important 
for  the  progress  of  such  projects,  but  also  to  ensure  the  interest  of  parties249.  In  addition, 
interviewees outlined the importance of the change that at least 50% of the budget for the 
research  and  innovation  component  is  allocated  to  the  Widening  coordinator  in  Twinning 
actions (WIDERA Work Programme 2023-2024). The duration of the grant and the size of 
the budget have been underlined as being important in producing sufficient outputs. In this 
regard,  interviewees  and  many  stakeholders  representing  academia250  have  called  to 
increase  the  budgets  for  separate  Twinning  grants  (as  well  as  the  Twinning  budget  in 
general). 

Building capacity, attracting, and retaining talent, and fostering mobility 

Some  of  the  core  expected  outcomes  of  the  Widening  programme  part  are  to  strengthen 
scientific capacities and skills in widening organisations, to improve research management 
and administrative capacities at institutions, to attract top researchers to widening countries, 
and to foster (intersectoral) mobility. While it is too early yet to assess whether the Horizon 
Europe  Widening  actions  are  fully  successful  in  this  regard,  the  first  evidence  points  to 
positive results.  

In general, in the survey of Horizon Europe beneficiaries (conducted May-July 2023), almost 
90%  of  successful  WIDERA  Widening  respondents  indicated  that  the  improvement  of  the 
skills, knowledge and competences of researchers is likely or very likely to be achieved as 
an  outcome  of  their  WIDERA  project251.  In  addition,  preliminary  evidence  regarding  KIP2 

243 i.e.158 out of 177; WIDERA-WIDE-BEN-027. 
244 i.e. 103 out of 170; WIDERA-WIDE-BEN-031. 
245 76.7 %, or 138 out of 180; WIDERA-WIDE-BEN-025.  
246 162 out of 178; WIDERA-WIDE-BEN-025.  
247 WIDERA-WIDE-BEN-031. 
248 62.5 %, or 25 out of 40; WIDERA-WIDE-APPL-025.  
249 e.g. The Guild of European Research-Intensive Universities (2023). Lessons learned and way 
forward: the Guild’s recommendations for Horizon Europe.  
250 e.g. European Regions Research and Innovation Network (ERRIN). (2023). ERRIN input to 
the public consultation on the past, present and future of the European Research & Innovation 
Framework Programmes 2014-2027. 
251 152 out of 172 respondents indicated this to a ‘very large’ or ‘large’ extent; WIDERA-WIDE-
BEN-031.  

205 

 
 
‘Strengthening human capital in R&I’ indicates that Horizon Europe is so far supporting more 
than 4,000 researchers through Widening actions252. More than 40% of those researchers 
are involved in upskilling activities (training, mentoring/coaching, mobility and access to R&I 
infrastructures)  in  projects  funded  by  the  Programme  (KIP2)253.  Upskilled  researchers  are 
those  researchers  that  are  in  the  earlier  stages  of  their  research  careers  (i.e.  first-stage 
researchers and recognised researchers). This percentage is  somewhat higher than other 
programme parts under Horizon Europe.  

Twinning has been stressed by many interviewees and stakeholders representing academia 
as a proven concept for bolstering capacity building and improving the skills of researchers, 
under  both  Horizon  2020  and  Horizon  Europe.  Twinning  projects  have  proved very useful 
opportunities to exchange knowledge and skills. Twinning is also seen as an instrument that 
enables engagement in deeper relationships with leading European institutions. Therefore, 
many interviewees and stakeholders254 have called for an increased budget for Twinning, an 
increased budget per project, and the introduction of a separate budget for advanced partners 
to  incentivise  participation.  Several  Twinning  projects  were  affected  by  the  COVID-19 
pandemic, in the sense that physical exchanges, mobilities, conferences and training, etc. 
were  impossible  for  a  certain  period  in  time.  Nevertheless,  all  relevant interviewees in  the 
context of this study emphasised the added value that this action has had in terms of skills 
development and networking.  

Furthermore, Teaming and ERA Chairs were mentioned as important actions in improving 
the  skills  of  researchers,  and  preliminary  insights  from  the  coordinators  of  several  novel 
actions  (e.g.  from  ‘European  Excellence  Initiative  (EEI)  Capacity  building  to  strengthen 
networks of higher education institutions and cooperation with surrounding ecosystems’ and 
ERA Talents) point to positive results in terms of capacity building (e.g. the training of young 
researchers, scientific capacity building, experience in the coordinator’s role, developing tools 
for training, etc.).  

Under Horizon Europe, Widening actions have increased their focus on fostering brain 
circulation across the ERA through the mobility of researchers, including intersectoral 
mobility,  especially  through  two  new  actions.  First,  the  ERA  Fellowships  (piloted  under 
Horizon 2020) are assigned to excellent researchers undertaking cross-border mobility to a 
widening  country.  The  second  new  action,  ERA  Talents,  aims  to  support  the  training  and 
mobility of researchers, innovators, and other  R&I talents across sectors, with a particular 
focus  on  widening  countries.  It  encourages  inter-sectoral  collaboration  in  R&I  through  the 
cross-sectoral exchange of staff.  

In the survey mentioned above255, almost 90% of successful WIDERA widening beneficiaries 
indicated that WIDERA funding provided greater opportunities for international mobility than 
national or regional funding256. 

Attracting talent. The Evaluation study on Excellent Science under Horizon 2020257 outlined 
that the widening countries sometimes experience difficulties in attracting international talent. 

252 For 3,780 researchers supported by the Framework Programme, this category is missing. 
253 This reflects the short-term KIP2 indicator, and is provided by the EC. 
254 e.g. European Regions Research and Innovation Network (ERRIN) (2023). ERRIN input to 
the public consultation on the past, present and future of the European Research & Innovation 
Framework Programmes 2014-2027. 
255 Survey of Horizon Europe beneficiaries, conducted May-July 2023. 
256 152 out of 170 respondents. 
257 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R. et al., Evaluation study of the European framework programmes 
for research and innovation for excellent science: Horizon 2020: Annexes, Luxembourg: 
Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383. 

206 

 
 
This challenge was especially notable in the ERA Chairs action and was also underlined in 
the  Court  of  Auditors  report258.  Therefore,  certain  novelties  have  been  introduced  into  the 
ERA Chairs actions under Horizon Europe. More specifically, the ERA chairholder must be 
identified in the proposal, and there is greater flexibility regarding the appointment of the ERA 
chairholder. For instance, the chairholder may be employed part-time at the host institution 
and can opt for multiple stays (instead of permanently moving there). While these changes 
are very well appreciated, to some interviewees, this raises questions regarding the level of 
commitment and collaboration shown, and whether this will lead to the same outputs.  

In  general,  several  stakeholders  (e.g.  widening  beneficiaries)  still  indicate  that  attracting 
excellent (international) researchers is one of the major challenges for widening institutions 
– especially if the host institution is in a rural area or has less attractive salary conditions. For 
ERA  Chairs  in  particular,  a  positioning  paper259  in  the  context  of  the  OPC260  raises  the 
question as to whether it should be possible to hire an employee within the host institution, if 
the recruitment of external experts continues to be unsuccessful after several open calls. This 
paper also recommends the use of the MSCA calculation method to determine salary levels 
for ERA chairholders, rather than following national guidelines.  

Inducing structural changes 

One of the key expected outcomes of the Widening actions is to induce structural changes 
to the regional or national R&I landscape or the institutions involved, with the aim of fostering 
improved R&I performance. Nevertheless, the Evaluation study on Excellent Science under 
Horizon 2020 disclosed that there was a lack of a mutual understanding of the concept 
of ‘structural changes’ among Horizon 2020 widening beneficiaries and stakeholders. 
This can be attributed to the fact that the Horizon 2020 SEWP work programmes did not refer 
systematically to the importance of introducing structural changes at institutional, regional or 
national levels in the call texts of the actions261, and that there was no shared definition of 
structural changes, nor a common approach to monitoring and tracking structural changes 
stemming from Widening actions262.  

In line with previous recommendations263, the present study underlines the need to develop 
a  definition  of  structural  changes  in  the  context  of  the  Widening  actions,  and  to  invoke  a 
consolidated and robust monitoring approach to track the progress made by actions in this 
regard. In this study, the notion of ‘structural changes’ has been further explored by focusing 

258 European Court of Auditors. Special Report 15/2022.  
259 e.g. the Guild of European Research-Intensive Universities (2023). Lessons learned and way 
forward: the Guild’s recommendations for Horizon Europe. 
260 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 
261 The ERA Chairs calls indicated as a concrete objective, ‘Facilitating structural changes in the 
institution’. However, in the case of the Teaming calls, there was no explicit mention of structural 
changes, reforms, or similar, although it was stressed in the call texts that Teaming actions 
should contribute to improving scientific capabilities in regions or Member States, thereby 
increasing the chances of securing competitive funding and fostering a culture of research and 
innovation. Likewise, in the Twinning calls, the emphasis was put on achieving significant 
improvements in the overall scientific and innovation capacity of the coordinating institution in a 
particular field of research, without explicitly referring to structural changes. 
262 As was the case in Horizon 2020 Science with and for Society (SwafS). 
263 See, for instance, Phase 1 of this evaluation support study (European Commission, 
Directorate-General for Research and Innovation, Budraitis, M., Pranckevičius, P., Dėlkutė, R. et 
al., Evaluation study of the European framework programmes for research and innovation for 
excellent science: Horizon 2020: Annexes, Luxembourg: Publications Office of the European 
Union, 2023, available at: https://data.europa.eu/doi/10.2777/353383.) and the ‘European Court 
of Auditors. Special Report 15/2022’. 

207 

 
 
on Horizon 2020264,265, and a definition and conceptual framework have been developed on 
the basis of desk research and interviews: 

A structural change can be defined as a  sustainable and/or permanent shift or change 
introduced  because  of  or  triggered  by  a  Widening  action  under  SEWP  (Horizon  2020)  or 
WIDERA (Horizon Europe), which is manifested at institutional, regional or national level, 
and is aimed at fostering R&I performance.  

Structural changes are expected to impact R&I performance in different ways. In this sense, 
structural changes can be classified according to their expected impacts: 

1.  Strengthening the R&I system and its governance; 

2. 

Improving working conditions, career progression and skills; 

3.  Fostering research excellence and collaboration; and 

4.  Strengthening the research environment and infrastructure. 

Concrete examples of potential structural changes in these four categories of impacts 
are presented in Table 39. 

Type of structural 
change 

Strengthening the 
R&I system and its 
governance 

Improving working 
conditions, career 
progression and 
skills 

Examples 

•  Changes in governance and management* 
•  Drafting or adapting research (and innovation) agendas, strategies, 

action plans 

•  Changes in R&I monitoring or evaluation systems 
• 
Attracting/increasing R&I funding and investments* 
•  Creation of working groups, committees, platforms  
• 

Setting-up/reform of R&I funding systems, policies, instruments, 
and/or the development of/reforms in evaluation and assessment for 
funding 

•  Contributing to smart specialisation strategies (e.g. project coordinator 

being involved in the design/development of the region’s S3) 

Establishment of new research / doctoral programmes 

• 
•  Creation/revision of HR guidelines, procedures, policies, regulations* 
Introduction of changes to PhD supervision and training schemes 
• 
Endorsement and application of the EU Charter for Researchers and 
• 
the Code of Conduct for the Recruitment of Researchers or one of its 
principles: 
– 
– 
– 

transparency in hiring conditions 
international recruitment panel 
gender balance 

264 Since the introduction of structural changes can take a significant amount of time to become 
observable and the Horizon Europe Widening projects have only recently commenced, this 
phase of the evaluation study focuses solely on structural changes introduced thanks to Horizon 
2020 Widening actions. 
265 See Case Study 7, ‘Impact of the widening actions on inducing structural change’ in this 
evaluation support study. 

208 

 
 
 
 
Type of structural 
change 

Examples 

Fostering research 
excellence and 
collaboration 
(including non-
academic) 

– 
– 
– 

demonstrable career development aspects 
including mentorship and training 
stable contract conditions 

• 

Staff mobility (e.g. attracting international researchers or national 
researchers that have worked abroad) 

•  Developing new research programmes 
•  Creation/revision of other guidelines, procedures, (IPR, ethics, quality 

management systems)* 

•  Developing/adapting instruments promoting or facilitating international 
collaboration (e.g. engaging in new bilateral science agreements*), 
developing international R&I collaborations 

•  Creation of Knowledge and Technology Transfer Offices*, knowledge 

transfer infrastructure, or new local research and innovation 
partnerships 
Introducing new ways of performing science/influencing the overall 
science system of the country (e.g. Open Science practices)* 

• 

Strengthening the 
research 
environment and 
infrastructure 

Setting up and/or modernising Centres of Excellence (Teaming) 
• 
•  Creation or modernisation of laboratories, research department/centre/ 

• 

teams/units/infrastructure* 
Setting up/development/enhancement/restructuring of research 
support services* 

Table 39. Proposed categories of structural change. Note: *included in the survey of project beneficiaries launched by REA 
in 2023 as a structural change266. Source: compiled by the authors based on desk research and the Feedback-to-Policy 
survey (2022) conducted by the Research Executive Agency (REA). See Case Study 7.  

In addition, the present study provides further qualitative insights into the structural changes 
that have been introduced thanks to or triggered by Widening actions under Horizon 2020.  

First,  Horizon  2020  survey  evidence267  shows  that  the  structural  changes  occurring  at 
institutional level appear to be widespread and common across the Teaming, Twinning 
and ERA Chairs Widening projects analysed. It was, for instance, noticeable that Widening 
projects had an impact on host institutions by enhancing the quality of training, and fostering 
the  development  of  researchers'  skills,  knowledge  and  competences,  which  improved 
working conditions, career progression and skills (category 2, Table 39). Fewer Horizon 2020 
projects  had  a  direct  impact  on  the  salaries  of  researchers.  Regarding  changes  aimed  at 
fostering research excellence and collaboration (category 3), approximately three-quarters of 
the survey respondents268 believed that the widening project had been effective in enhancing 
the institution's competence in implementing Open Science practices. More information on 
the  prevalence  of  institutional  structural  changes  is  found  in  Case  Study  7  of  the  present 
study.  

Second, evidence indicates that changes occurring at regional or national levels are less 
common. This may be partly attributed to the fact that institutional changes  may be more 

266 Feedback-to-Policy survey (2022), conducted by the Research Executive Agency (REA) 
targeted at the coordinators of Horizon 2020 Widening projects. 
267 Survey of Horizon 2020 beneficiaries, conducted August-September 2022. 
268 Survey of Horizon 2020 beneficiaries, conducted August-September 2022. 

209 

 
 
 
narrowly defined and  may be more easily attributed to a project, and institutional changes 
are potentially applicable to more projects, while fewer projects can be expected or able to 
induce  changes  at  regional  or  national  levels.  Among  the  national  or  regional  structural 
changes  frequently  identified,  we  can  cite  contributions  to  the  development  of  regional  or 
national R&I strategies aimed at strengthening the R&I system and governance (category 1, 
Table  39),  as  well  as  the  implementation  of  actions  leading  to  increased  knowledge  and 
expertise in the building, use and operation of new infrastructures, facilities and laboratories 
aimed  at  strengthening  the  research  environment  and  infrastructure  (category  4)269.  The 
mechanisms through which these changes occur vary significantly and usually relate to the 
design and objectives of the project concerned and are also associated with the degree of 
political involvement and awareness, as well as alignment with national or regional policies 
and priorities.  

Limited  insights  in  relation  to  Horizon  Europe  show  that  44.6%  of  successful  WIDERA 
widening  applicants  that  responded  to  the  survey  intend  to  produce  recommendations  for 
policy-makers and inputs into national or EU policies270. 

Fostering collaboration in transnational networks and strengthening ecosystems 

The  goals  of  the  Widening  actions  under  Horizon  Europe  also  include  supporting  less 
advanced R&I countries by increasing their participation in international research networks, 
strengthening academia/non-academia cooperation, and reinforcing (regional) ecosystems. 

As  outlined  previously,  widening  beneficiaries  often  engage  in  partnerships  based  on 
previous personal connections with (some of) the researchers involved.  Widening projects 
typically  help  to  further  structure  and  institutionalise  such  collaborations.  In  addition, 
becoming  part  of  an  international  consortium  appears  to  have  a  positive  impact  on  the 
maturity and quality of the research group and on these institutions being considered reliable 
research partners. Evidence from Horizon Europe indicates that almost 90% of the WIDERA 
widening beneficiaries that responded to the survey report that their widening project creates 
or strengthens collaborations with leading research organisations to a ‘large’ or ‘very large’ 
extent271.  

Nevertheless,  qualitative  evidence  gained  through  interviews  with,  for  example,  NCP 
WIDERA representatives, indicates that some researchers in widening countries (especially 
in  institutions  with less  strong  traditions in  engaging in  European  projects)  still  experience 
difficulties  in  setting  up  and  engaging  in  collaborative  consortia.  The  network  analysis 
conducted in the context of this study272 provides more information about collaborations within 
WIDERA. The share of collaborations between widening and non-widening Member States 
accounts for 31.6% of all collaborations in Horizon Europe excluding WIDERA. Focusing on 
WIDERA  alone,  the  share  of  cross-collaborations  between  widening  and  non-widening 
Member States is higher, at 39.9%. Moreover, when examining the new connections within 
WIDERA’s  widening  calls,  it  was  found  that  there  were  4,404  collaborations  within  the 
Widening component, of which 3,631 were new – meaning that the two entities concerned 
had not previously collaborated anywhere else within Horizon Europe. The highest number 
of  new  collaborations  were  formed  between  Greek  organisations,  followed  by  Croatian 
organisations (and Greek-Bulgarian collaborations.  

To  improve  the  commercialisation  and  uptake  of  R&I  results,  the  Widening  actions  under 
Horizon Europe also aim to strengthen the innovation ecosystems for knowledge circulation 
and valorisation by establishing stronger interconnections between existing collaborative and 
supportive  structures,  engaging  a  diversity  of  stakeholders  in  multidisciplinary  and  cross-

269 For more information, see Case Study 7 in this evaluation support study.  
270 79 out of 177 respondents; WIDERA-WIDE-BEN-027.  
271 158 out of 180 respondents; WIDERA-WIDE-BEN-025.  
272 For more information and details, see Annex 6 ‘Quantitative research track findings’. 

210 

 
 
sectoral collaborations. To achieve this, the Excellence Hubs instrument has been set up to 
strengthen  regional  innovation  excellence.  These  project  consortia  should  include  at  least 
two different widening place-based R&I ecosystems, including all quadruple helix partners, 
and non-widening ecosystems or partners may also be included if they offer specific added 
value.  The  goal  is  for  these  ecosystems  to  team  up  and  create  robust  linkages  between 
academia, business, government, and society to foster a place-based innovation culture in 
line with Research and Innovation Strategies for Smart Specialisation (RIS3) and EU policy 
priorities.  Synergies  with  the  European  Innovation  Ecosystems  (EIE)  and  the  European 
Institute of Innovation and Technology (EIT) should also be sought. Projects can consist of 
multiple components such as joint R&I strategies, R&I projects, action and investment plans 
for strategy, conceptual design for pilots and demonstrators, and complementary activities 
(learning, exchanges).  

In the first Excellence Hubs call273, 13 grants were signed, including a total of 200 participants 
(189 unique participants)274. The countries with the most participations were Greece (44 out 
of 200 participants), Cyprus (22), and Bulgaria (21). The composition of the participants is 
highly diverse, with 30.0% private for-profit entities, 21.0% other organisations, 19.5% higher 
education institutions, 15.5% research organisations and 14.0% public bodies. The second 
call for proposals275 was opened in September 2023276. While the value and outcomes of the 
instrument  are yet  to  unfold,  some  preliminary  thoughts  were  shared  by  interviewees  and 
stakeholders representing academia: 

•  Excellence hubs require the involvement of various actors from the same territory, which 
entails  mutual  support  in  relation  to  access  to  good-quality  consortia,  which  is  very 
important  especially 
the  Framework 
Programmes277. 

less  experienced  with 

for  participants 

•  One stakeholder, in its position paper submitted to the OPC, recommends the inclusion 
of  cooperation  with  well-established  R&I  ecosystems  from  non-widening  countries,  as 
this could lead to the transfer of best practices and hence greater impact278. 

•  One interviewee noted that at the beginning, this instrument and its objectives (e.g. which 
TRLs are targeted) was unclear, but that presentations and the showcasing of excellent 
examples have helped to better understanding of the instrument. 

In addition, the Hop On Facilities were set up to provide the possibility for one legal entity to 
join an already-funded collaborative R&I project (i.e. a project under Pillar 2 or EIC Pathfinder 
scheme), thereby opening silos of established, closed networks. The conditions for doing so 
are as follows: 1) all present consortium partners need to agree; 2) there are no widening 
entities  already  in  the  consortium;  3)  the  widening  partner  must  add  R&I  value  to  the 
consortium; and 4) a Grant Agreement amendment needs to be drafted and a distinct work 
package  needs  to  be  planned.  Interviews  with  stakeholders  (e.g.  WIDERA  NCP 
representatives,  widening  beneficiaries  and  stakeholders  representing  academia)  offered 
some preliminary insights into the appreciation of the instrument: 

273 HORIZON-WIDERA-2022-ACCESS-04. 
274 Horizon Dashboard accessed on 3 September 2023. 
275 HORIZON-WIDERA-2023-ACCESS-07. 
276 https://rea.ec.europa.eu/funding-and-grants/horizon-europe-widening-participation-and-
spreading-excellence/excellence-hubs_en  
277 European Regions Research and Innovation Network (ERRIN) (2023). ERRIN input to the 
public consultation on the past, present and future of the European Research & Innovation 
Framework Programmes 2014-2027. 
278 The Guild of European Research-Intensive Universities, Lessons learned and way forward: 
the Guild’s recommendations for Horizon Europe, 2023. 

211 

 
 
• 

In general, the concept behind Hop On Facilities is very much appreciated by widening 
institutions and stakeholders. 

•  However,  some  stakeholders,  would  prefer  to  have  incentives  for  including  widening 

partners directly into project proposals as they are submitted279.  

•  Stakeholders  and  interviewees  point  to  difficulties  for  widening  institutions  in  finding 

consortia that can and will include them in running projects.  

–  Non-widening countries are sometimes not aware of this funding scheme. It is 
suggested  to  inform  the  coordinators  of  eligible  projects  better  about  this 
possibility.  

–  Some  stakeholders  argued  that  few  projects  are  eligible  for  Hop  On.  Some 
stakeholders  suggested  also  having  the  possibility  to  hop  on  to  consortia  in 
which  there  are  already  widening  institutions,  while  others  suggested  that  it 
might be possible to allow more than one widening country to hop on per project. 
In  addition,  one  stakeholder  recommended  expanding  the  scheme  to  include 
research infrastructure projects on aspects of innovation280.  

–  For  widening  institutions  and  researchers,  it  is  often  hard  to  find  the  right 
contacts with eligible projects and their coordinators. Keeping the list of eligible 
projects up to date was mentioned as being important in this respect.  

–  Engaging  in  the  Hop  On  Facility  might  be  considered  an  overly  bureaucratic 
burden on the coordinators, since it includes undertaking an additional approval 
process,  grant  agreement  amendments,  and  novel  work  packages.  The 
administrative burden is compensated, however, by an additional administrative 
fee to be claimed by the co-ordinator. 

–  Some stakeholders recommended the use of user-friendly networking systems 
such  as  platforms,  events,  tools  or  databases  to  ensure  that  widening 
institutions and existing consortia can find each other281.  

•  Another stakeholder mentioned that one cut off per year is not satisfactory, as it limits 

the possibility to hop on and could lead to very short collaborations282.  

Lastly, the European Excellence Initiative (EEI) calls have been developed under Widening 
to raise excellence in science and value creation through deeper and geographically inclusive 
cooperation  in  alliances  of  HEIs  (i.e.  to  drive  institutional  changes).  This  action  aims  to 
engage with universities and empower them further to become actors of change in R&I. The 
widening dimension of the EEI is open to any network or alliance of HEIs, but the centre of 

279 Masaryk University (MUNI) (2023). Contribution of Masaryk University to Public consultation 
on the past, present and future of the European Research & Innovation Framework programmes 
2014-2027. 
280 Foundation for Science and Technology Portugal (FCT), The Portuguese Foundation for 
Science and Technology’s position paper on the ex-post evaluation of Horizon 2020, and on the 
mid-term evaluation of Horizon Europe and its future, 2023. 
281 e.g. the European Association of Research and Technology Organisations (EARTO) (2023). 
EARTO Answer to the EC Consultation on the past, present and future of the European 
Research & Innovation Framework Programmes 2014-2027. 
282 Polish Chamber of Commerce for High Tech Technology (IZTECH) Working Group on EU 
Framework Programme Horizon Europe (2023). Position Paper on the past, present and future 
of the European Research & Innovation Framework programmes 2014-2027. 

212 

 
 
gravity  should  be  on  widening  countries.  The  call  under  work  programme  2021-2022283 
focused on capacity building for networks of HEIs and partners in the local ecosystem (with 
a view to preparing the networks for the full roll-out of the European Universities initiative). 
Under  this  call,  nine  grants  were  signed,  including  126  participants  (125  unique 
participants)284.  The  countries  with  the  most  participations  are  Spain  (with  14  out  of  126 
participants), Portugal (11 out of 126), Germany (nine out of 126), Romania (seven out of 
126) and Czechia (six out of 126). More than 60% of the participants are HEIs, 17.5% are 
private for-profit entities, 8.7% are other organisations, 8.7% are public bodies, and 3.2% are 
research organisations. Under the second call for proposals285, 61 proposals were submitted. 
Some initial remarks on the instrument were provided by an EEI beneficiary: 

•  The  beneficiary  indicated  that  the  application  process  was  not  as  difficult  as  the 
application  for  European  University  Initiative  (EUI).  In  addition,  there  were  fewer 
budgetary constraints compared with the EUI. 

•  The beneficiary also noted that this EEI capacity-building project is useful, since it allows 

experience to be gained in taking the coordinating role.  

Focusing  on  innovation,  the  commercialisation  of  results  and  private  sector 
involvement 

As  the  bulk  of  Widening  actions  represent  Coordination  and  Support  Actions  (CSAs),  the 
Widening  actions  have  a  rather  limited  direct  effect  on  the  commercialisation  of  research 
results.  This  was  also  stated  in  the  Evaluation  study  of  Excellent  Science  under  Horizon 
2020286. Yet, as the section above indicates, efforts were made under Horizon Europe to 
better include the private sector in projects with actions such as the Excellence Hubs, 
and to focus more on innovation. In addition, the call Dissemination and Exploitation 
Support Facility has been developed. This call is open to widening participants from Horizon 
2020  and  Horizon  Europe  and  aims  to  scale  up  research  results  and  improve  the 
sustainability of actions (e.g. translating innovations into new products, feeding results into 
policy-making). The  projects  under  the  first  call287  began  as  of  June  2023.  Whether those 
efforts  positively impact  spin-off  and  start-up  creation,  IPR activity,  knowledge  valorisation 
and market uptake will be assessed once the widening projects are further advanced. 

The Horizon Dashboard288 shows that under Horizon Europe, the percentage of private for-
profit entities in the Widening actions289 is 12.7%, while under Horizon 2020, it was 4.7%. In 
addition,  the  survey  of  successful  WIDERA  widening  respondents290  provides  information 
regarding  the  expected  innovation  and  commercialisation  activities  of  widening  projects 
under Horizon Europe: 

• 

37.3% of respondents indicated that their widening project would probably lead to testing, 
demonstrating, and piloting activities291. 

283 HORIZON-WIDERA-2021-ACCESS-05. 
284 Horizon Dashboard, accessed 3 September 2023. 
285 HORIZON-WIDERA-2023-ACCESS-03. 
286 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R. et al., Evaluation study of the European framework programmes 
for research and innovation for excellent science: Horizon 2020: Annexes, Luxembourg: 
Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383. 
287 HORIZON-WIDERA-2023-ACCESS-05. 
288 Accessed 1 September 2023. 
289 In terms of participation. 
290 Survey of Horizon Europe beneficiaries, conducted May-July 2023. 
291 66 out of 177 respondents. 

213 

 
 
• 

• 

• 

25.4% of respondents indicated that they were developing new or improved products, 
services, or processes292.  

12.4% of respondents believed their project would probably produce patents293. 

In addition, 28.8% of respondents believed their projects would, to a ‘large’ or ‘very large’ 
extent, achieve entry to new markets or (global) value chains294.  

•  More than half of successful WIDERA widening respondents strongly or rather agreed 
that,  compared  with  national  or  regional  funding,  Horizon  Europe  provides  more 
opportunities for the commercialisation of research results (e.g. the applicability of results 
to new markets not present in the national environment)295. 

In  this  regard,  stakeholders  representing  academia  made  some  recommendations  to 
enhance this even further: 

• 

Install  more  Research  and  Innovation  Actions  (RIA)  and  Innovation  Actions  (IA)  to 
complement the current Coordination and Support Actions (CSA). This would allow the 
pursuit of concrete R&I activities.  

•  A dedicated facility could be created that would allow the shadowing of project leaders 
through a shared leadership model (e.g. a Hop On Project Leader)296. This could address 
the issue that, while participation from widening countries in Horizon Europe projects is 
increasing, project leadership in RIA and AI projects is not.  

1.6.2.2. 

Coherence 

Internal coherence of WIDERA and its Widening component 

Some interviewees and stakeholders representing academia expressed concerns regarding 
the design and implementation of the WIDERA programme part and the Widening actions. 
First, some interviewees and stakeholders stressed that the WIDERA work programmes are 
somewhat  complex,  since  the  Widening  component  and  the  strengthening  the  ERA 
component are quite different yet are addressed together in a single work programme. Some 
asked whether the design could be improved by implementing Widening under a separate 
work programme297.  

Second,  in  terms  of  the  coherence  of  the  WIDERA  programme  part, several  stakeholders 
wondered whether certain actions that are categorised under Widening might not be better 
positioned under the ‘Strengthening the ERA’ component of the work programme or under 
another programme part. One stakeholder questioned whether the Excellence Hubs and the 
European Excellence Initiative are likely to achieve the priorities of the ERA across the EU if 

292 45 out of 177 respondents; WIDERA-WIDE-BEN-027.  
293 22 out of 177 respondents; WIDERA-WIDE-BEN-027.  
294 49 out of 170 respondents; WIDERA-WIDE-BEN-031.  
295 i.e., 56.8 % or 96 out of 169; WIDERA-WIDE-BEN-045.  
296 European Regions Research and Innovation Network (ERRIN). (2023). ERRIN input to the 
public consultation on the past, present and future of the European Research & Innovation 
Framework Programmes 2014-2027. 
297 e.g., Masaryk University (MUNI). (2023). Contribution of Masaryk University to Public 
consultation on the past, present and future of the European Research & Innovation Framework 
programmes 2014-2027. 

214 

 
 
they are targeted by calls within the Widening programme298. Similarly, some stakeholders 
noted  that  COST  is,  by  definition,  open  and  is  not  aimed  at  supporting  a  given  group  of 
countries and is therefore not ideally positioned under Widening299. 

In general, the Widening actions in themselves appear to be set of distinct actions to support 
widening institutions and countries aiming to improve their R&I systems. The existing actions, 
Teaming for Excellence, Twinning, ERA Chairs and COST each have specific activities and 
address  distinct  objectives.  Under  Horizon  Europe,  new  actions  have  been  introduced  to 
address  needs  in  line  with  policy  priorities  that  had  not  yet  been  sufficiently  covered. 
Specifically, actions have been introduced that foster the development and strengthening of 
(regional) ecosystems (e.g. the EEI, Excellence Hubs); support academia-business linkages 
and collaboration (e.g. the EEI, ERA Talents, Excellence Hubs); intersectoral mobility (e.g. 
ERA Talents); and empower universities to become actors of change in R&I (EEI). In addition, 
the lack of participation in Framework Programmes by widening countries has triggered the 
Hop On Facility action to allow research institutions to join ongoing R&I projects in Pillar 2 
and EIC Pathfinder. A new instrument, Pathways to Synergies, has been set up to support 
the  creation  of  interfaces  between  different  funding  systems,  to  stimulate  cross-border 
collaboration  and  stimulate  participation  in  the  Framework  Programmes.  In  addition,  to 
secure  the  sustainability  of  Widening  projects  and  stimulate  the  dissemination  and 
exploitation of results, the Dissemination and Exploitation Support Facility has been set up. 
Nevertheless, one stakeholder, in its positioning paper in the context of the open consultation, 
urges the EC to articulate more clearly how the different Widening instruments relate to one 
another, and what their ultimate objective is300.  

Lastly,  it  has  been  argued  that  the  ERA  Talents  scheme  could  complement  the  ERA 
Fellowships by focusing on training researchers and other types of R&I talents in widening 
countries, such as data stewards, research managers or knowledge valorisation officers301.  

Internal coherence with Horizon Europe  

Objectives 

The objective of the Widening actions is clear, in the sense that they focus on fostering the 
participation in the Framework Programme of countries lagging in R&I, and assisting these 
countries  in  strengthening  their  R&I  systems.  This  inclusive  approach,  whereby  all  EU 
countries  can  participate  in  and  benefit  from  the  Framework  Programme,  is  needed  to 
strengthen the ERA. In this sense, the Widening actions have a unique approach compared 
with the other three pillars of Horizon Europe.  

Instruments 

In  the  survey  of  successful  WIDERA  widening  beneficiaries302,  66.1%  of  respondents 
reported that their widening project is not a continuation of or follow-up on a previous research 

298 Masaryk University (MUNI). (2023). Contribution of Masaryk University to Public consultation 
on the past, present and future of the European Research & Innovation Framework programmes 
2014-2027. 
299 Masaryk University (MUNI). (2023). Contribution of Masaryk University to Public consultation 
on the past, present and future of the European Research & Innovation Framework programmes 
2014-2027. 
300 Guild of European Research-Intensive Universities, Lessons learned and way forward: the 
Guild’s recommendations for Horizon Europe, 2023. 
301 Guild of European Research-Intensive Universities, Lessons learned and way forward: the 
Guild’s recommendations for Horizon Europe, 2023. 
302 Survey of Horizon Europe beneficiaries, conducted May-July 2023 

215 

 
 
project303. For 12.0%, it was a continuation of a project funded under Horizon 2020304, and 
for 3.6% of an FP7 project305. In addition, the survey results enable us to examine whether a 
Widening  WIDERA  project  plans  to  implement  any  activities  in  collaboration  with  projects 
funded under the other Horizon Europe programme parts306. These results indicate that most 
respondents do not have any joint activities with projects under Horizon Europe307, or do not 
know  of  any308.  One-fifth  of  respondents309  reported  having  joint  activities  with  another 
Widening project.  

The continuity analysis310 also provides information on the continuation of research projects. 
Within Horizon Europe WIDERA, the largest knowledge inflow we can see is from Horizon 
2020’s Pillar 1 into Horizon WIDERA (512 researchers out of a total of 3,596 researchers in 
WIDERA, or 14%). Furthermore, 494 researchers from FP7 Cooperation flowed into WIDERA 
(in other words, 14% of WIDERA researchers previously participated in FP7 Cooperation).  

The following observations can be made regarding the coherence of Widening actions with 
other Horizon Europe pillars and programme parts: 

•  Taken as a whole, the Widening actions mainly focus on collaborative approaches and 
Coordination and Support Actions (CSA). This differs from Pillar 1 actions, which focus 
on mono-beneficiary grants and focus only to a limited extent on CSA. 

•  The ERA Talents scheme has been positioned within the WIDERA work programmes as 
a complement to MSCA Staff Exchanges. Many parallels exist between the two actions. 
Both  instruments  aim  to  stimulate  innovative  collaborations  in  R&I  by  facilitating  the 
exchange of personnel and the sharing of knowledge and ideas. However, ERA Talents 
places  a  particular  emphasis  on  fostering  inter-sectoral  collaboration,  especially  in 
widening  countries.  Moreover,  unlike  MSCA  Staff  Exchanges,  the  ERA  Talents 
programme allows intra-country exchanges in specific situations.  

•  The  ERA  Fellowships  and  MSCA  Postdoctoral  Fellowships  overlap  in  the  sense  that 
submissions  of  proposals  to  MSCA  Postdoctoral  Fellowships  that  are  not  funded  and 
contain a widening host institution are automatically resubmitted to the ERA Fellowships 
call.  The  objective  of  the  ERA  Fellowships  is  to  incentivise  brain  circulation  towards 
widening countries. This synergy simplifies and unburdens applicants and makes better 
use of resources (e.g. expert expenditure), as just one evaluation needs to be carried 
out under the MSCA311. 

•  The Excellence Hubs are positioned within the work programmes as a complement to 
initiatives such as Digital hubs or the EIT Regional Innovation Scheme (RIS), due to their 
strategic orientation, broader scope, and alignment with the widening eligibility criteria. 
In  addition,  since  this  action  aims  to  foster  a  real  place-based  innovation  culture  in 
widening  countries,  it  encourages  synergies  to  be  sought  with  European  Innovation 
Ecosystems (EIE), EIC pathfinder and the European Institute of Innovation & Technology 
(EIT). However, it is too early to evaluate whether any such synergies have emerged.  

303 127 out of 192 respondents; WIDERA-WIDE-BEN-023.  
304 23 out of 192 respondents; WIDERA-WIDE-BEN-023.  
305 7 out of 192 respondents; WIDERA-WIDE-BEN-023.  
306 WIDERA-WIDE-BEN-022.  
307 38.7 %, or 74 out of 191 respondents; WIDERA-WIDE-BEN-022. 
308 20.9 %, or 40 out of 191 respondents; WIDERA-WIDE-BEN-022. 
309 41 out of 191 respondents; WIDERA-WIDE-BEN-022.  
310 More information can be found in the Network Analysis in Annex 6 ‘Quantitative research 
track findings’.  
311 Internal mapping of synergies conducted by DG RTD in Spring 2023. 

216 

 
 
•  The  Targeted  Stakeholder  Consultation312  revealed  that  the  large majority  (67%313)  of 
respondents  did  not  know  whether 
for 
complementarities  between  the  six  clusters  and  Widening  and  ERA.  16%  of 
respondents314 saw unexploited potential for complementarities with Cluster 1 – Health.  

they  see  any  unexploited  potential 

• 

Importantly, several stakeholders representing academia, as well as interviewees (e.g. 
NCP  WIDERA  representatives)  raised  the  idea  of  integrating  widening  elements 
across the Framework Programme as a cross-cutting issue. Several approaches to 
achieve this were proposed.  

–  Using  part  of  the  widening  budget  to  fund  reserve-listed  proposals  (thus 
achieving  the  threshold)  submitted  by  researchers  and  organisations  from 
widening  countries  to  other  programme  parts  (e.g.  the  ERC)  could  be  highly 
beneficial. This would not compromise the criterion of scientific excellence, and 
would  facilitate  the  participation  of  widening  institutions  in  the  Framework 
Programme (and not just within Widening).  

–  Such integration could also be achieved through specific criteria – for example, 
by  ensuring  representation  from  widening  countries  in  selected  Pillar  2  calls 
through  user  engagement  (e.g.  asking  demonstrators  or  pilots  to  cover 
stakeholders from widening countries)315. 

–  Many interviewees and stakeholders would appreciate such approaches, and 
believe  they  might  serve  to  mitigate  the  ‘labelling’  and  distinction  between 
‘widening’  and  ‘non-widening’  countries,  but  they  nevertheless  underline  that 
some  dedicated  widening  budget  for  capacity  building  (CSA)  projects  is  still 
desired.  

External coherence 

A  few  stakeholders  representing  academia,  as  well  as  interviewees  (e.g.  NCP  WIDERA 
representatives),  posed  the  open  question  as  to  whether  there  are  sufficient  differences 
between the instruments supporting ecosystems, networks and alliances – and whether all 
of these provide distinct added value, or simply make the field somewhat more confusing. 
For instance, one interviewee questioned whether the ‘European Excellence Initiative (EEI) 
Capacity building to strengthen networks of higher education institutions and cooperation with 
surrounding ecosystems’ is sufficiently different from other instruments on capacity building 
among universities, such as ‘Erasmus+ Key Action 2: Cooperation among organisations and 
institutions’.  Similarly,  another  interviewee  noted  the  existence  of  Excellence  Hubs,  the 
currently piloted concept of ERA Hubs316, Digital Innovation Hubs and Regional Innovation 
Valleys,  and  argued  that  it  is  somewhat  difficult  to  see  the  difference  between  them.  This 
open  question  invites  reflection  on  both  the  coherence  and  (joint)  communication  of  the 
distinct value of the different instruments. 

312 Public Consultation on the past, present and future of the European Research & Innovation 
Framework programmes 2014-2027.  
313 595 out of 889 respondents. 
314 141 out of 889 respondents. 
315 European Regions Research and Innovation Network (ERRIN). (2023). ERRIN input to the 
public consultation on the past, present and future of the European Research & Innovation 
Framework Programmes 2014-2027. 
316 Cf. the currently piloted concept of ERA Hubs: Horizon Europe Programme Information 
document (2022). Testing of the ERA Hub concept – pilot phase. Accessed 20 October 2023 via: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2022/information_horizon-widera-2022-era-01-30_en.pdf. 

217 

 
 
Under  WIDERA,  synergies  between  funding  sources  are  envisioned  in  the  Teaming  and 
Excellence Hubs, to ensure the sustainability of the results (work programmes). A general 
overview of where Horizon Europe and other EU funding sources complement each other 
reveals  15  cases  in  which  WIDERA  and  other  EU  funding  schemes  have  supported 
thematically related projects. The amount of EU funds received as complementary funding 
amounts to EUR 20.1 million317. 

• 

In  Teaming  under  Horizon  Europe,  complementary  funding  from  national,  regional, 
European  or  private  sources  is  required  to  support  the  new  or  upgraded  Centres  of 
Excellence.  Experiences  from  Horizon  2020  have  taught  that  the  majority  of 
complementary  funding  comes  from  national  funding  or  from  the  ESIF  and  ERDF318. 
Some interviewees (e.g. national ministries and beneficiaries) indicate that this synergy 
is critically important to creating an impact, as well as to make sure that the country is 
sufficiently engaged and stands behind the Teaming project. While under Horizon 2020, 
issues were sometimes experienced by projects in relation to the securing and the timely 
arrival  of  complementary  funding319,  efforts  have  been  made  to  mitigate  these.  For 
instance,  under  Horizon  Europe,  second-stage  proposals  should  include  a  binding 
commitment on the necessary complementary funding. In addition, Teaming proposals 
undergo a joint evaluation of both parts – that funded under Horizon Europe, and under 
the  complementary  funding  source320.  In  particular,  it  is  mentioned  in  the  work 
programme that operations supported by a complementary source must comply with the 
scope of the supporting programme, and must provide a contribution to the achievement 
of  the  programme’s  specific  objectives.  Nevertheless,  stakeholders  still  regard  this 
complementary funding as potentially complicated321. Some interviewees indicated that 
a  mismatch  remains  between  Structural  Funds  and  FP  funding,  in  the  sense  that  the 
Structural Funds have to be allocated early on – often before a particular Horizon Europe 
call comes out – and that it is then difficult to renegotiate the allocations. One positioning 
paper  in  the  context  of  the  OPC322  indicates  that  under  Horizon  2020,  the  failure  to 
demonstrate access to complementary funding was one of the main shortcomings found 
in the Stage 1 evaluation process323. In addition, several operational programmes that 
could serve as a source for complementary funding only provide support for infrastructure 
and equipment when there is sufficient demand from the private sector – which is not 
always the case for research activities at lower TRLs.  

•  With  regard  to  the  Excellence  Hubs,  the  Frequently  Asked  Questions  elucidate  that a 
commitment  in  the  proposal  to  participation  from  regional  authorities  is  sufficient,  and 
that  financial  commitments  may  be  included  in  the  investment  plans  to  be  elaborated 
during the lifetime of the project.324 

317 Based on Corda data. For more information, see Main Report, Table ‘Overview of 
complementary funding (in million EUR)’. 
318 Internal mapping of synergies conducted by DG RTD in Spring 2023. 
319 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R. et al., Evaluation study of the European framework programmes 
for research and innovation for excellent science: Horizon 2020L Annexes, Luxembourg: 
Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383 
320 Internal mapping of synergies conducted by DG RTD in spring 2023. 
321 e.g. the Initiative for Science in Europe (ISE) (2023). ISE Response to the European 
Commission Public Consultation on the past, present and future of the European Research & 
Innovation Framework programmes 2014-2027. 
322 Public Consultation on the past, present and future of the European Research & Innovation 
Framework programmes 2014-2027. 
323 Guild of European Research-Intensive Universities, Lessons learned and way forward: The 
Guild’s recommendations for Horizon Europe, 2023. 
324 https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/support/faq/30740, 
accessed 29 August 2023.  

218 

 
 
A new instrument, Pathways to Synergies, has been set up to stimulate synergies between 
Horizon  Europe  and  the  Cohesion  Policy  programmes,  mainly  the  European  Regional 
Development  Fund  (ERDF),  including  INTERREG and  the  Resilience  and  Recovery  Fund 
(RRF). This pilot calls to incentivise the concrete realisation of promising sequential synergies 
via two different pathways, either upstream (focusing on human resource development and 
internationalisation), or downstream (focusing on the valorisation and upscaling of research 
results  towards  marketable  solutions).  In  March  2023,  a  call  for  proposals  was  launched. 
Projects are expected to begin in June 2024325. One interviewee noted that this is a rather 
complex instrument that requires a lot of effort from NCPs.  

Under Horizon Europe, a Seal of Excellence is awarded to Teaming applicants that exceed 
the evaluation thresholds set out in the work programme but cannot be funded due to a lack 
of budget available to the call. This Seal of Excellence offers a unique opportunity for funding 
bodies other than the European Commission to benefit from the high-quality Horizon Europe 
evaluation process. Regional or national funders can decide to set up a funding scheme to 
support Seal of Excellence recipients using institutional, regional, national or EU funding such 
as Cohesion Policy funds. At the end of 2022, the European Commission awarded 11 Seal 
of  Excellence  certificates  to  proposals  submitted  under  the  Horizon  Europe  Widening  call 
Teaming  for  Excellence326.  Interviews  with  national-level  stakeholders  from  widening 
countries reveal that some countries fund (to the extent possible) projects that have received 
a Seal of Excellence. However, one interviewee noted that, while a system is in place in their 
country to fund projects given the MSCA Seal of Excellence, it does not systematically fund 
projects given the Teaming Seal of Excellence because, according to them, the excellence 
criteria do not live up to the same standards as those of the MSCA.  

In general, it is of the utmost importance for widening countries to allocate sufficient resources 
to their R&I systems. Moreover, the full potential of all countries can only be realised when 
there are  synergies  between  the  Framework  Programme,  regional,  national  and  other EU 
support.  Consequently,  it  is  emphasised  that  the  EU  should  strive  to  further  align  and 
synchronise  the  Framework  Programme  with  the  Cohesion  Policy  funds,  the  Erasmus+ 
programme and national funding programmes.  

1.6.2.3. 

EU added value 

In  general,  interviewees  and  stakeholders  universally  acknowledged  and  appreciated  the 
added value brought about by the Widening actions. They underline the significance of the 
actions  in  terms  of  access  to  research  funding,  capacity  building,  fostering  (research) 
networks and significantly enhancing R&I critical mass and systems in host institutions, and 
to some extent, in regions and countries. At the same time, it was consistently emphasised 
that  it  is  vital  for  widening  countries  to  commit  adequate  resources  to  their  own 
national R&I systems. Such investment is essential to ensure that the effects of widening 
measures,  as  well  as  the  outcomes  of  the  Framework  Programme,  have  a  lasting  and 
meaningful impact. 

Compared  with  regional  or  national  funding,  the  WIDERA  Widening  beneficiaries  that 
responded  to  the  survey327  largely  valued  the  benefits  brought  by  participation  in  the 
Framework  Programme.  In  particular,  89%  strongly  agreed  or  rather  agreed  that  Horizon 
participation had provided more international mobility opportunities for researchers328; 88% 
strongly agreed or rather agreed that Horizon Europe provided greater diversity in partners’ 

325 https://rea.ec.europa.eu/funding-and-grants/horizon-europe-widening-participation-and-
spreading-excellence/pathways-synergies_en 
326 https://research-and-innovation.ec.europa.eu/news/all-research-and-innovation-
news/commission-awards-seal-excellence-outstanding-teaming-excellence-proposals-2022-12-
19_en 
327 Survey of Horizon Europe beneficiaries, conducted May-July 2023. 
328 152 out of 170 respondents; WIDERA-WIDE-BEN-045.  

219 

 
 
profiles329;  84%  agreed  that  it  provided  larger  amounts  of  funding  than  national/regional 
schemes330, and 77% agreed that it helps to better pool the skills and expertise necessary 
for  research331.  Only  14%  of  respondents  indicated  that  it  offers  no  additional  benefits 
compared with national or regional support332. 

Of the unsuccessful applicants to WIDERA Widening that responded to the survey, almost 
80% did  not apply for alternative funding such as national, regional or international 
funding  to  implement  the  activities  proposed  in  their  WIDERA  proposal333  and  only 
14.6% did so334. Of those that did apply, 36.3% were successful335, 10.6% were not336, and 
almost 50% were still awaiting a decision337. Of those that were already successful, 58.8% 
received national or regional funding338, while 23.5% received international funding (e.g. from 
the ERDF, ESF, CF)339. In addition, out of those that did apply for another funding source, 
almost half of the respondents made significant adaptations to their proposal340; 36.2% made 
some  adaptations341;  and  8.5%  made  minimal-to-no-adaptations342.  Those  who  did  make 
adaptations, did so in relation to: 

•  The project’s scope, with 75% of revised projects covering a smaller scope in terms of 

numbers of research areas343;  

•  Research  methods,  with  82.5% employing  a less  complex  and  diverse  set  of  research 

methods344;  

•  Research outputs, with 62.5% expecting to produce a smaller amount of research outputs 

and innovations345; 

•  Duration, with 70% planning a shorter duration346; 

•  Number of partners, with 77.5% involving fewer consortium partners347; and 

•  Partner profile, with 66.7% including less diversity in partners (in terms of internationality, 

sectors etc.)348.  

Among  those  unsuccessful  WIDERA  applicants  who  decided  not  to  apply  for  alternative 
funding, the most common reasons were that the project required the involvement of a variety 

329 150 out of 170 respondents; WIDERA-WIDE-BEN-045.  
330 142 out of 170 respondents; WIDERA-WIDE-BEN-045.  
331 128 out of 167 respondents; WIDERA-WIDE-BEN-045. 
332 21 out of 154 respondents; WIDERA-WIDE-BEN-045.  
333 257 out of 323 respondents; WIDERA-WIDE-APPL-018. 
334 47 out of 323 respondents; WIDERA-WIDE-APPL-018.  
335 17 out of 47 respondents. 
336 5 out of 47 respondents. 
337 23 out of 47 respondents. 
338 10 out of 17 respondents. 
339 3 out of 17 respondents. 
340 23 out of 47 respondents. 
341 17 out of 47 respondents. 
342 4 out of 47 respondents. 
343 30 out of 40 respondents. 
344 33 out of 40 respondents. 
345 25 out of 40 respondents. 
346 28 out of 40 respondents. 
347 31 out of 40 respondents. 
348 26 out of 39 respondents. 

220 

 
 
of international partners, and that there was simply no alternative funding available for that 
research topic349.  

This  analysis  points  to  the  added  value  of  the  EU-level  instrument  in  terms  of  a  higher 
intensity of international collaboration with more – and more diverse – partners, with a larger 
scope and degree of complexity, compared with other funding sources. 

1.6.2.4. 

Relevance 

Policy priorities within which the Widening actions are embedded 

Under both Horizon 2020 and Horizon Europe, stakeholders from both widening and non-
widening countries, as well as beneficiaries, acknowledge that widening measures remain 
of high strategic importance in the context of revitalising the European Research Area 
and the New European Innovation Agenda350. Stakeholders acknowledged the need for 
countries that are less advanced in R&I to upgrade their R&I systems, to build R&I capacity, 
thereby allowing the EU as a whole to advance. Secondary evidence shows that this R&I gap 
that exists within the EU has not yet been resolved, which is also reflected in different levels 
of participation in the Framework Programme. In addition, the network analysis conducted in 
the context of this evaluation support study indicates that, in Horizon Europe so far, mainly 
non-widening countries are central to the European research ecosystem351. Under Horizon 
Europe, the budget devoted to widening has increased significantly compared with Horizon 
2020, from EUR 935 million to EUR 2.95 billion, which is appreciated.  

The European Research Area (ERA), launched in 2000, refers to the ambition to create a 
single, borderless market for research, innovation and technology across the EU. This should 
help countries to work more effectively together, by strongly aligning their research policies 
and programmes. In 2018, a process to revitalise the ERA started352. The new ERA has four 
key  strategic  objectives,  of  which  two  are  reflected  in  the  WIDERA  Widening  work 
programmes  (2021-2022  and  2023-2024),  namely,  ‘Improved  access  to  Excellence’  and 
‘Attracting and mobilising the best talents’353:  

1. 

Improving  access  to  excellence  necessitates  strengthening  the  R&I  system, 
facilitating the rapid dissemination of best practices throughout the European Union. 
In particular, the Widening actions354 situated under this destination aim to bolster 
R&I capacities in widening countries and to stimulate national and regional R&I 
reforms  and  investments.  In  addition,  the  actions  address  capacity  building 
beyond  the  realm  of  pure  scientific  expertise,  encompassing  the  cultivation  of 
management  and  administrative  skills,  particularly  for  those  interested  in 
assuming leadership roles within consortia (for instance, in the context of Pillar 2). 

349 WIDERA-WIDE-APPL-029.  
350 COM/2022/332 final. Communication from the Commission to the European Parliament, the 
Council, the European Economic and Social Committee and the Committee of the Regions: A 
New European Innovation Agenda.  
351 For more information, see Annex 6. The geographical network analysis is performed through 
two different methods which produced the same results: one method uses the individual 
researchers’ country of origin as a basis, while another method uses organisations as a proxy for 
country of residence. 
352 European Commission, Directorate-General for Research and Innovation, The new European 
Research Area, Publications Office of the European Union, 2021, available at: 
https://data.europa.eu/doi/10.2777/2736 
353 WIDERA Work Programmes 2021-2022 and 2023-2024. 
354 These are Teaming for Excellence; Twinning; the European Excellence Initiative (EEI) 
(Strengthening capacity for excellence in higher education institutions and surrounding 
ecosystems); Pathways to synergies; Support for R&I policy making in the Western Balkans; the 
Dissemination and Exploitation Support Facility; the Hop On Facility; and Excellence Hubs.  

221 

 
 
In  addition,  this  destination  also  focuses  on  academic  and  higher  education 
systems  to  foster  reforms  in  widening  countries.  The  goal  is  to  empower  these 
countries to reach a competitive position on both the European and global stages. 
These  actions  aim  to  contribute  to  the  development  of  comprehensive  European 
ecosystems. 

2.  The  destination  ‘Attracting  and  mobilising  the  best  talents’  is  in  line  with  new 
ERA  objective  to  strengthen  the  mobility  of  researchers  and  the  free  flow  of 
knowledge  and  technology  in  an  effective  R&I  system.  The  actions355  provide 
opportunities to develop skills that researchers need for excellent science, attract, 
and maintain outstanding researchers, and to enable brain circulation for R&I 
talents across the ERA and across sectors.  

The new ERA and the Widening actions can also be seen in the context of The New European 
Innovation Agenda, adopted in 2022, and the European Strategy for Universities. The New 
European Innovation Agenda356 aims to help Europe foster the development of innovative 
technologies aimed at tackling the most pressing societal challenges, as well as facilitating 
their successful entry into the market357. The European strategy for universities358 aims to 
support and enable universities to adapt to changing conditions, to thrive, and to contribute 
to  Europe’s  resilience  and  recovery.  The  strategy  urges  Member  States  and  HEIs  across 
Europe to join forces and engage in transnational cooperation359. 

Relevance of the Widening actions and their design, given the scientific, technological, 
and socio-economic challenges 

The Widening actions under Horizon Europe are characterised by their bottom-up approach, 
which offers researchers flexibility in choosing their research topics. This approach is highly 
valued  by  stakeholders  and  beneficiaries,  as  it  allows  researchers,  research  groups  and 
organisations in widening countries to strengthen critical mass and build capacity, and to spur 
excellence in any specific field that is relevant to them. Importantly, the Evaluation support 
study  of  Excellent  Science  under  Horizon  2020360  elucidated  that  the  Widening  actions 
represent one of the only possibilities for bottom-up collaborative research under the 
Framework Programme, hence showing an imbalance in the availability of opportunities for 
this type of research. While the bottom-up approach remains a key feature, Horizon Europe 
introduces some calls related to specific themes, such as the Twinning Green Deal call361. In 
line  with  priorities,  this  will  spur  projects  in  the  following  topics:  climate  research,  green 
technologies,  renewable  energy,  sustainable  mobility,  biodiversity  research  and  the 
sustainable use of natural resources (land, water, air). 

355 These are ERA Chairs, ERA Talents, and ERA Fellowships. 
356 COM/2022/332 final. Communication from the Commission to the European Parliament, the 
Council, the European Economic and Social Committee and the Committee of the Regions: A 
New European Innovation Agenda. 
357 https://research-and-innovation.ec.europa.eu/strategy/support-policy-making/shaping-eu-
research-and-innovation-policy/new-european-innovation-agenda_en. 
358 COM(2022) 16 final. Communication from the Commission to the European Parliament, the 
Council, the European Economic and Social Committee and the Committee of the Regions on a 
European strategy for universities. 
359 https://education.ec.europa.eu/sites/default/files/2022-01/communication-european-strategy-
for-universities-graphic-version.pdf. 
360 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R. et al., Evaluation study of the European framework programmes 
for research and innovation for excellent science: Horizon 2020: Phase 1 final study report, 
Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/967813. 
361 HORIZON-WIDERA 2023-ACCESS 02-02. 

222 

 
 
Nevertheless, even though the  Widening actions do not prescribe  specific research topics 
and  areas  to  researchers  (except  for  the  Green  Deal  call),  Widening  projects  generally 
align  well  with  current  technological,  scientific,  and  socio-economic  trends  and 
challenges, as well as newly emerging needs. As an illustration, under Horizon Europe, 
the field in which by far the greatest number of projects are active is that of Natural Sciences, 
and the topics that are most frequently addressed in WIDERA widening signed grants are: 
ecosystems,  climate  change,  educational  sciences,  innovation  management,  oncology, 
pollution,  and  sustainable  economy362.  Machine learning  and  artificial  intelligence  are  also 
fairly  common  subjects  in  Horizon  Europe  widening  projects,  more  so  than  under  Horizon 
2020. Concrete examples of Horizon Europe projects are:  

•  BRAINN363  (Twinning),  developing  an  innovative  applied  neuroscience  approach  to 

characterise the effects of COVID-19 on brain health;  

•  TWINN2SET364 (Twinning), building a network to foster a sustainable energy transition 

in Greece;  

•  ACRONYM  (Twinning),  aiming  to  advance  expertise  on  migration  and  asylum 

governance in Europe.  

This  evidence  indicates  that  Horizon  Europe  widening  projects  are  largely  in  line  with  the 
latest technological, scientific and/or socio-economic developments, and contribute to the EU 
missions, such as beating cancer and addressing climate change. In the survey conducted 
in  the  context  of  this  study365,  a  large  majority  of  Horizon  Europe  WIDERA  Widening 
beneficiary respondents strongly agreed or rather agreed that the Framework Programme 
helps to  better  address  grand  societal  challenges  (e.g.  climate  change,  demographics 
and  ageing,  etc.)366,  and  that  the  FP  provides  more  flexibility  to  respond  to  changing 
socioeconomic needs367, compared with the research funding available at national and/or 
regional level.  

In  addition, in  line  with  the  new  ERA  objectives,  the  ERA  Action  Plan,  the  New  European 
Innovation  Agenda,  as  well  as  gaps  that  have  not  been  addressed  under  Horizon  2020, 
several  new  or  updated  actions  have  been  introduced  into  Horizon  Europe.  These  new 
funding  opportunities  focus  on  balanced  brain  circulation,  intersectoral  mobility,  the 
internationalisation  of  institutions,  and  the  strengthening  of  regional  ecosystems  that 
comprise higher education, innovation, research and society. For instance, the possibility to 
reinforce links between science and business, and to foster academic-industry cooperation 
and placed-based innovation ecosystems, have been addressed via instruments such as the 
Excellence Hubs, the European Excellence Initiative (EEI), and ERA Talents. The need for 
greater R&I collaboration and integration into networks of excellence is addressed through 
the Hop On Facility. Since the value of these new actions is still unfolding, most interviewees 
indicated that it was too early to comment on the needs these actions fulfil. However, some 
stakeholders and interviewees provided a few preliminary remarks regarding the relevance 
of the instruments and the issues they aim to solve: 

•  Some stakeholders have stressed that the Widening actions could benefit from actions 
supporting  R&I  activities,  such  as  more  Research  and  Innovation  Actions  (RIA)  or 

362 Source: Horizon Dashboard, accessed June 2023. 
363 https://cordis.europa.eu/project/id/101079001. 
364 https://cordis.europa.eu/project/id/101079246. 
365 Survey of Horizon Europe beneficiaries, conducted May-July 2023. 
366 70%, or 82 out of 169 strongly agreed or rather agreed; WIDERA-WIDE-BEN-045. 
367 53%, or 89 out of 168 strongly agreed or rather agreed; WIDERA-WIDE-BEN-045.  

223 

 
 
Innovation  Actions  (IA)368  in  which  there  is  no  requirement  to  include  non-widening 
partners  (since  this  would  allow  widening  countries  to  truly  drive  collaborative  R&I 
projects)369. 

•  One stakeholder noted that the ERA Talents funding scheme presents a mismatch with 
the real needs of widening countries, which should first establish an excellent science 
base,  and  that,  in  this  sense,  devoting  more  resources  to  ERA  Fellowships  is 
recommended370.  

• 

It is indicated that it is crucial to gain in-depth knowledge regarding the reasons for the 
limited participation of widening countries in some parts of the Framework Programmes. 
These  low  participation  rates  are  largely  systemic,  leading  to  a  need  to  extensively 
evaluate  the  widening  activities  and,  as  a  result,  design  future  calls  that  can  further 
encourage  widening  actors  to  apply  to  the  Framework  Programme  and  allow  them  to 
improve their R&I capabilities.  

•  Many interviewees appreciated the efforts made to target specific country groups (e.g. 
the Western Balkans). However, it was pointed out that participation mainly centred on 
one  country,  Serbia371.  Some  stakeholders  would  recommend  the  inclusion  of  equally 
relevant calls for all widening countries372.  

1.6.2.5. 

Conclusions and lessons learned 

Conclusion 1: The evidence collected in the context of this study indicates that EU funding 
to low-performing R&I countries remains rather low under Horizon Europe. Specifically, this 
report  show  that  the  geographic  diversity  of  the  Framework  Programme’s  participants  is 
comparable to that of H2020. Only 12.6% of Horizon Europe funding committed so far was 
allocated  to  participants  in  Widening  Member  States  and  10.1%  goes  to  associated 
countries373. Also, in terms of participations, Pillar 1, 2 and 3 all have most participations from 
non-Widening Member States and a higher share of funding to non-Widening Member States. 
Only  WIDERA  has  more  participations  from  Widening  Member  States  than  from  non-
Widening Member States. Moreover, organisations from Widening Member States engage 
substantially less often in the coordinator role of Framework Programme projects and grants 
compared with organisations from non-widening Member States374.  

368 e.g. the European Regions Research and Innovation Network (ERRIN). (2023). ERRIN input 
to the public consultation on the past, present and future of the European Research & Innovation 
Framework Programmes 2014-2027. 
369 e.g. the European Regions Research and Innovation Network (ERRIN). (2023). ERRIN input 
to the public consultation on the past, present and future of the European Research & Innovation 
Framework Programmes 2014-2027. 
370 Masaryk University (MUNI) (2023). Contribution of Masaryk University to Public consultation 
on the past, present and future of the European Research & Innovation Framework programmes 
2014-2027. 
371 For example, the Horizon Dashboard (accessed 2 September 2023) shows that 36.0 % of the 
total EU contribution to the call Twinning Western Balkans went to Serbia (4.3 % to Montenegro, 
5.3 % to Albania etc.). 
372 Guild of European Research-Intensive Universities, Lessons learned and way forward: the 
Guild’s recommendations for Horizon Europe, 2023. 
373 See Annex 6. 10.1 % goes to associated countries, 1.0 % goes to third countries and 12.6 % 
goes to Widening Member States.  
374 The Horizon Dashboard (accessed October 11, 2023) indicate that when focusing on the 
Member States, so far, the share of net EU contribution of widening Member States to the 
coordinator role is 11 %, while that of non-widening Member States is 89 %. In addition, the 
share of participations to the coordinator role of widening Member States is 13 %, while that of 
non-widening Member States is 87 %.  

224 

 
 
This analysis needs to be considered in the light of the overall objectives of the Framework 
Programme  (in  terms  of  facilitating  collaboration,  strengthening  impact  of  research  and 
innovation  and  creating  and  dispersing  excellent  knowledge  and  technologies)  and 
opportunities through other types of funding for regional development (for instance, ERDF or 
RFF). Yet it is relevant to note that qualitative evidence points to added value of Widening 
measures for Widening countries, and stakeholders from both Widening and non-Widening 
countries  as  well  as  beneficiaries  largely  acknowledge  that  Widening  measures  remain  of 
high strategic importance in the context of strengthening the R&I system and revitalising the 
European Research Area. As an illustration, survey evidence indicates that, compared with 
unsuccessful applicants under Pillar 1, unsuccessful applicants under WIDERA applied less 
often  for  alternative  sources  of  funding  (e.g.,  regional/national  funds),  suggesting  the 
importance of EU funding for them.  

Lesson  learned  1:  while  a  certain  degree  of  concentration  of  funding,  in  the  light  of  the 
excellence in science, is not in itself alarming, there continues to be a need for dedicated 
support for less advanced R&I systems to build their R&I capacity, hence allowing the 
EU to  advance  as  a  whole.  The  FP’s  role  in  addressing  this  issue  is  to  deliver support in 
addition  to  the  support  available  for  capacity  building  from  other  EU  funding  programmes 
(such as ERDF or RRF) by focusing on the creation and diffusion of excellent knowledge in 
the EU. Through this objective, the FP also supports less advanced R&I countries through 
enabling them to access excellent research networks, build R&I capacity, and improve their 
excellence, in particular with its widening actions.  

When focussing on the widening actions, it has been highlighted by interviewees (e.g. NCP 
Widening representatives) and in multiple positioning papers375  that it is worth exploring the 
possibility  of  further  integrating  the  widening  dimension  into  other  pillars  and  programme 
parts  to  further  support  those  countries  lagging  in  R&I376,377.  These  interviewees  and 
stakeholders believe that this could stimulate the participation of widening countries in R&I 
actions and increase their scientific and R&I leadership capabilities and R&I performance. 
Nevertheless,  this  possibility  could  be  explored  for future  Framework  Programmes.  At  the 
same time, while integrating the widening dimension into other pillars might be beneficial, it 
was often underlined that dedicated budget for Coordination and Support Actions aimed at 
capacity building is still highly beneficial for widening countries. 

Conclusion 2: as shown in Conclusion 1, there is still room to improve the participation rates 
of  widening  countries  in  the  Framework  Programme.  In  addition,  the  success  rates  of 
widening institutions are slightly lower than those of non-widening institutions378, and there is 
also asymmetry between widening countries in terms of levels of participation in the Widening 
actions379. 

Lesson  learned  2:  continued  support  for  the  development  and  advancement  of  support 
structures  for  widening  countries  at  EU,  NCP  and  institutional  levels  is  still  desired  to 
stimulate  applications  to  and  participation  in  the  Widening  actions,  as  well  as  in  the 

375 In the context of the public consultation on the past, present and future of the European 
research and innovation programmes 2014-2027. 
376 It should be noted that the legal basis in Horizon Europe does not allow this kind of further 
integration in this Framework Programme. Nevertheless, the possibility could be explored for 
future Framework Programmes. 
377 A similar approach has been taken in the ERA Fellowships, in which non-funded MSCA 
fellowship applicants that have a widening host institution are automatically resubmitted to the 
ERA Fellowships call and may hence, if successful, be funded with the widening budget. 
378 With participants from widening Member States having a success rate of 19.6%, and those 
from non-widening Member States having a success rate of 22.5%. Based on Corda data, June 
2023 release. 
379 With widening countries such as Greece, Portugal and Czechia having the most 
participations in WIDERA, as well as the largest shares of the EU contribution.   

225 

 
 
Framework Programme in general. For instance, some (less-experienced) researchers from 
(smaller) widening institutions highlighted difficulties in finding consortium partners to apply 
for Widening actions or in securing the resources and expertise to write successful proposals. 
In  this  regard,  support  remains  important  to  ensure that researchers  and  research  groups 
with  less  experience  in  EU  projects  are  motivated  and  able  to  successfully  engage  in 
research networks and consortia and to apply for Framework Programme funding. Initiatives 
such  as  the  transnational  network  of  National  Contact  Points  (NCPs)380  and  the 
NCP_WIDERA.NET381 have largely been appreciated, and several national NCP WIDERA 
representatives  expressed  their  willingness  to  engage  in  further  training  on,  for  example, 
proposal checks and different Widening instruments. Furthermore, European Cooperation in 
Science and Technology (COST) is also important and is appreciated as a relevant bottom-
up networking instrument.  

Conclusion  3:  one  of  the  key  expected  outcomes  of  the  Widening  actions  is  to  induce 
structural changes to the regional or national R&I landscape or the institutions involved, with 
the aim of fostering R&I performance. During the analysis, however, it became evident382 that 
there was  no  shared  understanding  of  the  concept  of  ‘structural  changes’  among  Horizon 
2020 widening beneficiaries and stakeholders, and that not all projects lead to such changes. 
For  instance,  the  present  study  highlights  that,  while  structural  changes  occurring  at 
institutional level seem to be fairly widespread and common (e.g. Widening projects having 
an  impact  on  the  host  institutions  by  enhancing  the  quality  of  training  and  fostering  the 
development of researchers’ skills, knowledge and competences, thereby improving working 
conditions, career progression and skills), changes occurring at regional or national levels 
are less common. 

Lesson learned 3: this programme part could benefit from a more targeted approach that 
clearly  defines  the  types  of  changes  expected  from  the  intervention,  and  introduces  a 
consolidated and robust monitoring framework to measure the progress made by actions in 
this regard. This would require the development of a definition of ‘structural changes’ in the 
context of the Widening actions, the inclusion of a list of expected structural changes in call 
texts,  and  the  introduction  of  a  consolidated  and  robust  monitoring  approach  to  track  the 
progress made by the actions in this regard. Such an approach would not only facilitate the 
monitoring of the progress of the actions in this regard, but would also help to identify barriers, 
as well as success stories that could inspire future widening applicants and beneficiaries.  

With regard to this, this study has proposed a potential definition for structural changes in the 
context of widening actions, namely: ‘a structural change can be defined as a sustainable 
and/or permanent shift or change introduced as a result of or triggered by a Widening action 
under SEWP (Horizon 2020) or WIDERA (Horizon Europe), which manifests at institutional, 
regional or national level, and is aimed at fostering R&I performance’. In addition, structural 
changes  could  be  classified according  to  their  expected  impacts  in  relation  to  R&I383.  The 
present study has grouped examples of structural changes in these distinct categories into a 
conceptual framework (see Table 39. Proposed categories of structural change).  

380 https://horizoneuropencpportal.eu/ 
381 NCP_WIDERA.NET Project is a three-year Coordination and Support Action financed by the 
European Commission and established with the aim of empowering the network of National 
Contact Points (NCP) for WIDERA under the Horizon Europe Framework Programme. 
382 See Phase 1 of this evaluation support study: European Commission, Directorate-General for 
Research and Innovation, Budraitis, M., Pranckevičius, P., Dėlkutė, R. et al., Evaluation study of 
the European framework programmes for research and innovation for excellent science: Horizon 
2020: Annexes, Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383 
383 1. Strengthening the R&I system and its governance; 2. Improving working conditions, career 
progression and skills; 3. Fostering research excellence and collaboration; 4. Strengthening the 
research environment and infrastructure. 

226 

 
 
In setting up measuring and monitoring, it would be desirable to keep as much as possible 
with existing instruments  to cause as little additional burden and inefficiencies as possible. 
Therefore, it would be worth considering the integration of the definition and the conceptual 
framework (with examples of changes) into the reporting template and integrating this (in a 
comparable  way)  into  the  periodic  REA  survey  to  follow  up  on  progress.  In  this  way,  the 
reporting template and the survey responses could provide more in-depth insights into the 
specific structural changes that are being induced during the projects (thanks to the reporting 
template and also, in the longer run, thanks to the periodic survey). Consistency in approach 
about definitions and formulation would be important in this regard. 

Conclusion  4:  the  evidence  in  this  report  has  indicated  that  the  Widening  actions  have 
triggered structural changes at institutional level and – although less visibly – at regional and 
national levels. Nevertheless, it should be borne in mind that major structural changes come 
with the support and commitment from the countries themselves, and go beyond the effects 
of specific Widening projects. As the European Court of Auditors report indicates, sustainable 
change largely depends on efforts being made by national authorities384. 

Conclusion 5: under Horizon Europe, the WIDERA work programmes consist of the thematic 
areas ‘Widening participation’ and ‘Strengthening the European Research Area’. Various new 
Widening actions have been introduced with the aim of fostering R&I systems in widening 
countries.  As  a  consequence,  several  stakeholders  (e.g.  NCP  WIDERA  representatives, 
stakeholders  representing  academia)  refer  to  WIDERA  and  the  widening  landscape  as 
somewhat  complicated,  with  many  different  –  sometimes  complex  –  instruments385.  A  few 
stakeholders  have  argued  that  some  actions  categorised  under  Widening  might  be  more 
intuitively  positioned  under  Strengthening  the  ERA386.  They  also  argue  that  some  new 
Widening  actions  are  perceived  as  rather  complex,  especially  for  less  experienced 
participants and coordinators, and that the number of Widening actions may make it harder 
for potential applicants to navigate the possibilities. In addition, the multiplicity of schemes is 
sometimes perceived as hindering the visibility and prestige of the programme part. 

Lesson  learned  5.1:  it  is  recommended  to  continue  considering  possibilities  for  the 
rationalisation and streamlining of WIDERA and the widening landscape where possible. To 
this  end,  and  given  the  multitude  of  new  widening  actions,  the  periodic  monitoring  of  the 
appreciation, results and impacts of these actions is key, and the Widening instruments and 
portfolio would need to be updated accordingly. 

Lesson learned 5.2: some interviewees (e.g. Widening beneficiaries) explained that, while 
highly  prestigious,  Teaming has  not  yet  acquired  the same  name-based  reputation as  the 
ERC or MSCA.  In  this  sense,  it  is  important  to keep  building  the visibility  of  the  Widening 
programme part and its individual actions (one idea suggested in this regard was a dedicated 
‘Widening’ website, like that of the ERC).  

384 European Court of Auditors, Measures to widen participation in Horizon 2020 were well 
designed but sustainable change will mostly depend on efforts by national authorities. Special 
report 15, 2022, Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2865/359822 
385 e.g. Initiative for Science in Europe (ISE) (2023). ISE Response to the European Commission 
Public Consultation on the past, present and future of the European Research & Innovation 
Framework programmes 2014-2027. 
386 e.g. Masaryk University (MUNI) (2023). Contribution of Masaryk University to Public 
consultation on the past, present and future of the European Research & Innovation Framework 
programmes 2014-2027. 

227 

 
 
1.7.  Efficiency of all analysed programme parts 

1.7.1.  Efficiency of Selection and Implementation Processes  

So far, the share of ineligible proposals in Horizon Europe is 3.5%, which is over three times 
higher than Horizon 2020’s 1.1%. Among Pillar 1 programmes, INFRA saw the highest 
rate of ineligible proposals at 11.7%, which represents an increase of 9.4 percentage 
points from H2020.  

Figure 45. Share of ineligible proposals. Source: Compiled by the evaluation team with CORDA data, June 2023 release. 

Oversubscription,  which  was  noted  as  a  key  challenge  in  Horizon  2020387,  may  be 
seeing a reduction so far in Horizon Europe. The overall success rate of eligible proposals 
has increased by 4.6 percentage points and Pillar 1 has seen an increase of 1.6 percentage 
points. No part of Pillar 1 saw a reduction in its success rate, even the particularly competitive 
programmes ERC and MSCA. 

Total eligible 
applications 

Success rate % 
(all proposals) 

Success rate % 
(high-quality 
proposals) 

Success rate 
H2020 % (all 
proposals 

HE388 

52 124 

Pillar 1 

34 182 

WIDERA 

1 085 

ERC 

16 328 

16.00 

15.30 

27.80 

13.98 

29.80 

29.86 

40.43 

57.43 

11.4 

13.6 

15.7389 

12.8 

387 European Commission, Directorate-General for Research and Innovation, A new horizon for 
Europe – Impact assessment of the 9th EU framework programme for research and innovation, 
Publications Office, 2018, https://data.europa.eu/doi/10.2777/194210. 
388 Excludes calculations for EIT and EIC Accelerator. 
389 SEWP was used as a comparison. 

228 

 
 
 
 
Total eligible 
applications 

Success rate % 
(all proposals) 

Success rate % 
(high-quality 
proposals) 

Success rate 
H2020 % (all 
proposals 

MSCA 
(excl. PF) 

17 657 

16.09 

21.39 

INFRA 

152 

53.40 

65.83 

14.4 

33.1 

Table 40. Overall success rates. Source: Compiled by the evaluation team with CORDA data, June 2023 release. 

The figure below provides the comparison of the success rates between the first two years 
of HE and the first two years in H2020. Generally, we see that that there is a positive turn in 
the oversubscription, as suggested by relatively higher success rates in HE. Such a change 
is likely a product of an increased HE budged (as compared to H2020).  

70%

60%

50%

40%

30%

20%

10%

0%

62%

47%

25%

24%

30%

27%

18%

18%

14%

13%

16%

13%

13%

12%

ERC

MSCA

INFRA

WIDERA

15%

10%

Specific
objective
(SEWP
&SwafS)

18%

14%

15%

13%

18%

15%

12%

11%

Pillar 1

Overall FP

HE 2021

HE 2022

H2020 2014

H2020 2015

Figure 46. Success rates comparison between HE and H2020 first two years. Source: Compiled by the evaluation team with 
CORDA data. 

Project  selection  efficiency  falls  short  of  the  targets,  both  overall  and  within  Pillar  1.  The 
average TTG is 276 days, over one month longer than the goal of 244 days. The average 
TTI is 157 days, much closer to the target of 152 days but still longer than hoped for. Across 
Pillar  1,  the  delays  are  longer  still,  with  the  average  TTG  approaching  300  days  and  the 
average TTI at 172 days. 

The lower efficiency of Pillar 1 is in large part attributed to the ERC programme, which 
has the longest TTG and TTI of any programme part in HE so far. ERC’s average TTG 
is 357 days, over 3 months past the target, and its TTI is 194 days. 

On the other hand, for both INFRA and WIDERA the average TTI and TTG are shorter than 
the respective targets (shown in blue in Table 41). 

229 

 
 
 
Average TTI390 (target 
152) and a percentage of 
proposals informed within 
the target 

Average TTS (target 91) 
and a percentage of 
grants signed within the 
target 

Average TTG (target 
244) and a percentage of 
grants signed within the 
target 

FP overall 

156.64 (67%) 

106.58 (47.7%) 

275.52 (70.4%) 

Pillar 1 

171.84 (65.7%) 

98.8 (62.7%) 

292.30 (65.1%) 

WIDERA391 

117.88 (92.9%) 

94.08 (48.8%) 

220.71 (83.3%) 

ERC 

193.69 (79.1%) 

107.20 (67.6%) 

356.70 (66.8%) 

MSCA 

152.51 (53%) 

91.20 (60.4%) 

244.40 (63.7%) 

INFRA 

96.76 (98.8%) 

130.30 (13.9%) 

226.30 (70.8%) 

Table 41. Time-to indicators (days). Source: Compiled by the evaluation team with CORDA data, June 2023 release. 
Negative values (in green) represent indicators that were reached below the legal target (indicating the difference in days). 
Positive values (in red) indicate the indicators above the legal targets. 

HE average TTG is longer than H2020 at the close and interim phases of the programme. 
On the other hand, except for Pillar 1, all HE Pillars achieved TTGs below the target of 244 
days (Table 42). When we compare HE with H2020 during the interim phase (years 2014-
2015), we see that Pillar 1’s TTG is only 6 days longer, and in fact ERC and INFRA have 
achieved TTGs which are one month shorter compared to the H2020 interim. WIDERA has 
been the most stable, with the current, H2020 interim, and H2020 close average TTG within 
4 days of each other. 

Programme 

No of 
applications 
analysed 
(HE) 

HE 
TTG 
(244 
days) 

No of 
applications 
analysed 
(H2020) 

Horizon 
2020 
TTG 

H2020 
interim 
TTG392 

Difference 
in days 
from H2020 
interim 

Overall 

51 503 

275.5 

295 885 

231.0 

249.4 

+26.1 

Pillar 1 

34 336 

292.3 

145 800 

266.7 

286.0 

+6.3 

Pillar 2393 

11 219 

237.5 

63 056 

194.1 

188.4 

+49.1 

Pillar 3 

3 770 

240.0 

76 966 

157.1 

186.5 

+53.5 

WIDERA 

1 107 

220.7 

2 943 

223.5394 

220.6 

+0.1 

390 Only proposals with information on the date of Proposal Information Letter were included in 
the TTI calculation. 
391 A Predecessor of WIDERA is considered to be Specific objective (SEWP &SwafS). 
392 Interim figure includes only projects with call closure years 2014 and 2015. 
393 Data excludes EIT and EIC Accelerator. 
394 SEWP was used as a comparison. 

230 

 
 
 
Programme 

No of 
applications 
analysed 
(HE) 

HE 
TTG 
(244 
days) 

No of 
applications 
analysed 
(H2020) 

Horizon 
2020 
TTG 

H2020 
interim 
TTG392 

Difference 
in days 
from H2020 
interim 

ERC 

16 336 

356.7 

57 801 

359.0 

386.9 

-30.2 

MSCA 

17 835 

244.4 

79 727 

208.50 

205.7 

+38.7 

INFRA 

165 

226.3 

969 

265.50 

262.3 

-36.0 

Table 42. TTG comparison with H2020. Source: Compiled by the evaluation team with CORDA data, June 2023 release and 
Excellent Science Phase 1 report. 

Over 99% of Pillar 1 is financed via pre-financing, and all of the pre-financing payments 
were made within the 30-day target. ERC was the most efficient programme part with an 
average TTP of only 11.5 days; 91.9% of its payments were made within the 30-day target. 
The speed of pre-financing is another key driver of efficient project implementation within HE. 

Progamme 
part 

Total 
number of 
projects 
analysed 

Total number 
of payments 
made395 

Average 
TTP: 
pre-
financing 

Share of pre-
financing 

Share of 
payments 
within the 
target of pre-
financing396 

ERC 

MSCA 

INFRA 

Total pillar 1 

Total 
Horizon 
Europe397 

1 128 
(47.2%) 

1 187 
(49.7%) 

74 
(3.1%) 

2 389 
(100%) 

1 185 
(48.0%) 

1 201 
(48.6%) 

83 
(3.4%) 

2 469 
(100%) 

11.5 

99.9% 

91.9% 

15.1 

99.6% 

87.6% 

19.1 

96.5% 

69.9% 

13.5 

99.6% 

89.1% 

4 564 

5 610 

17.0 

99.2% 

82.1% 

Total H2020 

33 621 

104 321 

6.7 

65% 

100% 

Table 43. Overview of payments data and TTP by programme. Source: Compiled by the evaluation team with CORDA data, 
June 2023 release. 

1.7.2.  External help available to applicants  

The  following  analysis  provides  evidence  on  different  sources  of  support  Horizon  Europe 
applicants  accessed  when  preparing  their  applications.  It  also  reports  on  the  correlation 
between  proposal  success  and  support  received  from  National  Contact  Points  and  from 

395 Negative payments and regularization are excluded.  
396 The target for pre-financing target is 30 days, for other payments 90 days.  
397 Excludes calculations for EIT and EIC Accelerator. 

231 

 
 
 
consultancies, on comparative time-to-grant values of respondents that used consultancies, and 
on consultancy fees reported by applicants.   

The main source of evidence is the survey of cluster 1, 2, and 3 beneficiaries and unsuccessful 
applicants. 

The information from the survey leads to the following Indicators:  

•  Application  support  received  from  dedicated  internal  department  in  own  organisation, 
National Contact Points, consultancy internal or external to the consortium (support may 
extend to proposal preparation, project implementation, dissemination).  

•  Comparison  of  success  rates between  respondents  drawing  on  support  from  National 
Contact Points and consultancies, compared to complementing non-user groups. 
•  Comparison  of  proposal  quality  between  respondents  drawing  on  support  from 

consultancies, compared to complementing non-user group.  

•  Comparison  of  proposal  quality  between  respondents  drawing  on  support  from 

consultancies, compared to complementing non-user group.   

•  Monetary values of fees paid to external consultancies, reported by respondents. 

The responses from the Horizon Europe Beneficiaries and Unsuccessful Applicant Surveys 
were combined with Commission eCorda data on the survey respondents). 

1.7.2.1. 

Applicants’ use of support for proposal preparation  

The subset of survey data which fed into this analysis were all valid answers to the question ‘Did 
you receive any support for your Horizon Europe project application / preparation?’ Respondents 
were able to choose multiple answers as shown in Figure 47. 

Across  the  three  pillars,  the  majority  of  survey  respondents  reported  using  a  dedicated 
department  within  their  organisation  to  help  with  their  application.  The  greatest  share  of 
applicants who reported using consultants came from Pillar 3, and the lowest share from WIDERA. 

Figure 47. Breakdown of types of application help indicated by survey respondents (overall). Source: Source: Survey of 
Horizon Europe beneficiaries, conducted in May-July, 2023 and Survey of Horizon Europe unsuccessful applicants, 
conducted in May-July, 2023. Question: ‘Did you receive any support for your Horizon Europe project 
application/preparation?’ N= 7 300.398 

398 Please note that here and other figures in efficiency analysis, we have excluded responses 
such as “do not know/cannot answer”. To see survey responses including this option please 
refer to the survey synopsis report in Annex 7. 

232 

 
 
 
Among  Pillar  I  respondents,  MSCA  PFs  were  the  least  likely  to  report  using 
consultants (5%). They also reported the highest instance of using a dedicated department 
within their organisation to help with their application. 

The rates of consultant use found by our May-June 2023 survey are slightly lower than what 
was found by the 2018 beneficiaries survey conducted by ECA, where 31% of ERC and 28% 
of MSCA respondents indicated they had used consultants. In contrast, the survey of H2020 
beneficiaries  conducted  for  the  Evaluation  study  on  Excellent  Science  in  the  European 
Framework  Programmes  for  Research  and  Innovation  Phase  1  found  that  around  17%  of 
INFRA399 used external consultants, a number that is similar to the 18% found by our team. 
However,  we  must  be  cautious  when  drawing  conclusions  from  different  surveys,  as  the 
difference  in  wording  and  sample  populations  may  cause  slight  differences  in  the 
respondents’ answers. 

Figure 48. Breakdown of types of application help indicated by survey respondents (Pillar I). Source: Source: Survey of 
Horizon Europe beneficiaries, conducted in May-July 2023 and Survey of Horizon Europe unsuccessful applicants, 
conducted in May-July 2023. Question: ‘Did you receive any support for your Horizon Europe project 
application/preparation?’ N=2 144 for consortia-based applications and 4 663 for ERC and MSCA PF. N=2 144. 

1.7.2.2. 

Overlap between applicants’ use of consultancies and of National Contact 
Points  

We looked at the overlap between those survey respondents who used NCPs and those who 
used or acted as consultants in order to test the theory that consultants use the help of NCPs 
more than regular applicants.  Based on the survey data, we found that, overall,  about 
half (53%)  of  applicants  who  used or  were  consultants  also used the  help of  NCPs, 
compared with just 19% of overall applicants. This gap is largely due to Pillars 1 and 2, 
as Pillar 3 and WIDERA showed similar rates of use in both samples populations. 

399 Only INFRA and FET applicants were asked about external consultant use for preparing 
proposals. 

233 

 
 
 
 
 
HE / Pillar 

HE overall 

Pillar 1 

Pillar 2 

Pillar 3 

WIDERA  

Total number of 
respondents 
(successful and 
unsuccessful 
applicants) 

Share of total respondents 
that use consultancies (or 
are consultancies 
themselves)400 

Share of total 
respondents that use 
NCPs 

7 300 

5 059 

1 119 

901 

221 

19% 

10% 

32% 

47% 

24% 

19% 

17% 

25% 

21% 

29% 

HE / Pillar 

Overlap: Share of total 
respondents that use 
both, consultancies (or 
are consultancies 
themselves) and NCPs 

Share of users of 
consultancies (or that are 
consultancies) that also 
use NCPs 

Share of users of NCPs 
that also use 
consultancies (or are 
consultancies 
themselves) 

HE overall 

Pillar 1 

Pillar 2 

Pillar 3 

WIDERA  

5% 

3% 

8% 

10% 

7% 

27% 

32% 

36% 

20% 

28% 

26% 

20% 

33% 

45% 

23% 

Table 44. Breakdown of overlap between consultancy and NCP use. Source: Survey of Horizon Europe beneficiaries, 
conducted in May-July 2023 and Survey of Horizon Europe unsuccessful applicants, conducted in May-July 2023. Question: 
‘Did you receive any support for your Horizon Europe project application/preparation?’ N=7 300. 

1.7.2.3. 

Correlation between applicants’ use of support for applications and 
success of application 

Among our survey respondents, successful applicants (27%) were more likely to have used 
consultants than unsuccessful ones (21%) among consortia. Survey respondents to single-
beneficiary  actions  (ERC  and  MSCA  PF)  reported  far  lower  rates  of  consultant  use  and 
similar  rates  among  both  successful  and  unsuccessful  applicants.  The  median  successful 
consortium reported paying more for the service than the median unsuccessful consortium. 
In the free response section, it was noted that consultancies may charge an additional fee if 
the proposal is successful, which could account for the difference. 

400 The figures in this column combine responses to the question “Did you receive any support 
for your Horizon Europe project application/preparation?” answer option “Yes, our organisation 
received support from a consultancy firm/expert (inside and outside the consortium) as well as 
the responsents who identified their organisation type as “External expert/consultancy firm”.  

234 

 
 
 
Total 
number of 
respondents 

Number of 
respondents 
who used 
external 
consultancies 

Share of 
respondents 
who used 
external 
consultancies 
(%) 

Median fee 
paid to 
external 
consultancies 
(EUR) 

Unsuccessful 
applicants 

1 398 

Beneficiaries 

746 

Unsuccessful 
applicants 

3 548 

Beneficiaries 

1 115 

Consortia 

Mono 
beneficiary 
calls (ERC 
and MSCA 
PF) 

290 

198 

328 

107 

20.74 

6 500 

26.54 

10 000 

9.24 

2 650 

9.60 

1 750 

Table 45. Overall consultant use among survey respondents. Source: Survey of Horizon Europe beneficiaries, conducted in 
May-July 2023 and Survey of Horizon Europe unsuccessful applicants, conducted in May-July 2023. Question: ‘Did you 
receive any support for your Horizon Europe project application/preparation?’ Answer: ‘Yes, our organisation received 
support from a consultancy firm/expert (inside or outside the consortium)’. N=2 144 for consortia and 4 663 for mono 
beneficiary. 

In  every  pillar,  the  share  of  successful  respondents  was  higher  among  survey 
respondents  who  used  external  consultancies.  The  difference  was  greatest in  Pillar 3, 
where the share of successful respondents who used the help of external consultancies was 
18 percentage points higher than those who did not. The difference was smallest in WIDERA; 
only 9 percentage points. 

Share of 
successful 
respondents 
among users of 
consultancy 
firms/experts 

Share of successful 
respondents among 
those who did not use 
consultancy 
firms/experts 

Share of successful 
respondents: All 
respondents 

HE overall 

Pillar 1 

Pillar 2 

Pillar 3 

WIDERA 

32.19 

27.81 

43.54 

29.95 

35.71 

21.84 

22.04 

28.51 

11.32 

26.24 

27.36 

24.93 

39.05 

25.08 

33.03 

Table 46. Share of successful respondents –support from consultancy/expert (inside or outside the consortium). Source: 
Survey of Horizon Europe beneficiaries, conducted in May-July. 2023 and Survey of Horizon Europe unsuccessful 
applicants, conducted in May-July 2023. Question: ‘Did you receive any support for your Horizon Europe project 
application/preparation?’ Answer: ‘Yes, our organisation received support from a consultancy firm/expert (inside or outside 
the consortium)’. N=7 300. 

We  also  analysed  the  share  of  successful  respondents  among  those  who  used  the 
help  of  NCP  and  found  it  to  be  slightly  higher,  on  average,  than  those  who  hired 
external consultancies and always higher than the average success rate across all pillars. 

235 

 
 
 
 
The  largest  difference  is  seen  in  Pillar  3  and  WIDERA,  where  the  share  of  successful 
respondents  using  NCPs  were  sensibly  higher  than  the  share  of  successful  respondents 
using external consultancies and those not using NCPs. The main takeaway seems to be 
that  applicants  do  benefit  from  help  preparing  their  applications,  whether  that  help  comes 
from external consultants or National Contact Points. 

Share of 
successful 
respondents 
among users of of 
NCP 

Share of successful 
respondents among 
those who did not use 
NCPs 

Share of successful 
respondents: All 
respondents - (%) 

HE overall 

Pillar 1 

Pillar 2 

Pillar 3 

WIDERA 

34.50 

30.35 

43.42 

36.46 

45.31 

20.75 

19.77 

28.15 

17.31 

19.91 

27.36 

24.93 

39.05 

25.08 

33.03 

Table 47. Share of successful respondents –National Contact Point support for application. Source: Survey of Horizon 
Europe beneficiaries, conducted in May-July 2023 and Survey of Horizon Europe unsuccessful applicants, conducted in 
May-July 2023. Question: ‘Did you receive any support for your Horizon Europe project application/preparation?’ Answer: 
‘Yes, our organisation received support from a consultancy firm/expert (inside or outside the consortium)’. N= 7 300. 

By  combining  CORDA  data  with  the  survey  responses,  we  were  able  to  calculate  the 
percentage of survey respondents who used consultants when preparing their high-quality 
proposals401.  Based  on  the  number  of  respondents  who  answered  the  question  “Did  you 
receive any support for your Horizon Europe project application/preparation?”, we found that 
the  overwhelming  majority  (74-80%)  of  above  the  threshold  proposals  were  written 
without the involvement of external experts. Overall, above the threshold proposals were 
more likely to have seen the involvement of consultants than below the threshold, but not by 
a wide margin. Across the programme 27% of high-quality and 25% of below the threshold 
proposals were written with the help of experts. 

Number of 
above the 
threshold 
proposals 

Share of above the 
threshold proposals 
that used experts (%) 

Share of below 
the threshold 
proposals that 
used experts (%) 

Success rate of 
above the 
threshold 
proposals (%) 

HE 
overall 

Pillar 1 

Pillar 2 

1 570 

335 

773 

27.20 

25.37 

26.00 

25.23 

8.20 

20.23 

51.46 

43.58 

56.53 

401 Proposals whose score exceeded the threshold. 

236 

 
 
 
 
Number of 
above the 
threshold 
proposals 

Share of above the 
threshold proposals 
that used experts (%) 

Share of below 
the threshold 
proposals that 
used experts (%) 

Success rate of 
above the 
threshold 
proposals (%) 

Pillar 3 

302 

WIDERA 

160 

36.09 

20.00 

36.68 

16.39 

50.33 

45.62 

Table 48. Survey respondents’ consultant use among above the threshold proposals. Source: CORDA data and Survey of 
Horizon Europe beneficiaries, conducted in May-July 2023 and Survey of Horizon Europe unsuccessful applicants, 
conducted in May-July 2023. Question: ‘Did you receive any support for your Horizon Europe project 
application/preparation?’ Answer: ‘Yes, our organisation received support from a consultancy firm/expert (inside or outside 
the consortium)’. N=2 327. 

1.7.2.4. 

Correlation between applicants’ use of support for applications and the 
quality of the project proposal 

Pillar 1 followed a substantially different trend than HE overall: some programme parts 
saw  very  different  rates  of  consultant  use  on  above  the  threshold  (so  called-high 
quality)  and  below  the  threshold  proposals.  In  Pillar  1  overall,  25%  of  high-quality 
proposals and 8% of non-high-quality proposals saw the involvement of experts. The gap is 
the  smallest  in  MSCA  PF,  which  saw  the  vast  majority  of  proposals  written  without  the 
involvement of experts, high-quality or not. MSCA PF survey respondents were also the most 
likely  to  report  using  a  dedicated  department  within  their  organisation  to  support  their 
proposal. 

Number of 
above the 
threshold 
proposals 

Share of above the 
threshold 
proposals that 
used experts (%) 

Share of below 
the threshold 
proposals that 
used experts (%) 

Success rate 
of above the 
threshold 
proposals (%) 

Programme 
part 

ERC 

MSCA (excl. 
PF) 

337 

308 

MSCA PF 

2 227 

INFRA 

27 

24.33 

14.91 

74.48 

25.97 

5.21 

18.52 

9.09 

4.42 

0.00 

42.86 

38.80 

51.85 

Table 49. Survey respondents’ external consultancies use among high-quality proposals (Pillar I only). Source: CORDA data 
and Survey of Horizon Europe beneficiaries, conducted in May-July 2023 and Survey of Horizon Europe unsuccessful 
applicants, conducted in May-July 2023. Question: ‘Did you receive any support for your Horizon Europe project 
application/preparation?’ Answer: ‘Yes, our organisation received support from a consultancy firm/expert (inside or outside 
the consortium)’. N=5 026. 

1.7.2.5. 

Correlation between successful applicants’ use of support for applications 
and the time-to-grant period 

We calculated the average TTG for the sample of survey respondents who had answered 
the  question 
for  your  Horizon  Europe  project 
application/preparation?’ We found that the average TTG was comparable among those 
who did and did not use experts. In fact, average TTG was higher among those who used 
experts in Pillars 1 and 3, but lower in Pillar 2 and WIDERA. 

receive  any  support 

‘Did  you 

237 

 
 
Pillar 

Average TTG 

Average TTG with experts 

Average TTG without 
experts 

HE 
overall 

232.0 days 

238.3 days 

228.9 days 

Pillar 1 

225.6 days 

228.9 days 

224.5 days 

Pillar 2 

234.8 days 

234.0 days 

235.1 days 

Pillar 3 

243.4 days 

252.4 days 

231.5 days 

WIDERA 

197.6 days 

196.7 days 

197.8 days 

Table 50. Average Time-To-Grant periods of proposals prepared with and without consultancies. Source: CORDA data and 
Survey of Horizon Europe beneficiaries, conducted in May-July 2023 and Survey of Horizon Europe unsuccessful 
applicants, conducted in May-July 2023. Question: ‘Did you receive any support for your Horizon Europe project 
application/preparation?’ Answer: ‘Yes, our organisation received support from a consultancy firm/expert (inside or outside 
the consortium)’. N=2 637. 

We also examined the length of time which those who use or who are consultants need to 
prepare a proposal. We found that those who used consultants required more time to 
prepare a proposal than the average sampled applicant did. The median length of time 
needed for a coordinator who used a consultant was 46-55 person-days, compared with 36-
45 person-days among the entire survey sample (more detailed about proposal preparation 
effort can be found in the section below). Among those coordinators who themselves were 
consultants, the median length of time needed was the same as in the general sample: 36-
45 person days for coordinators. The consortia partners who were themselves consultants 
were the only group to indicate less time spent on proposals: 6-15 person-days as opposed 
to 16-25 needed for the general sample. More information on the costs of preparing a Horizon 
Europe proposal can be found in section 1.7.3. 

To  estimate  the  total  fees  paid  to  consultants,  we  calculated  expected  values.  We  based 
these on the rate of consultancy use per type of funding instrument and the median costs 
reported for three applicant groups: consortia402, mono-beneficiary (ERC and MSCA PF), and 
EIC  Accelerator.  The  general  idea  is  that  the  likelihood  that  a  given  proposal  would  have 
seen  the  involvement  of  a  consultant  depends  on  the  type  of  funding  instrument  they  are 
applying  for,  and  the  value  of  the  fee  they  would  pay  depends  on  whether  they  are  a 
consortia, a single beneficiary, or an EIC Accelerator applicant. A step-by-step methodology 
follows. 

Successful applicants reported paying higher consultancy fees (holds  both for median and 
mean),  for  the  support  service.  Survey  respondents  mentioned  that  consultancies  may 
charge  an  additional  fee  on  the  condition  that  the  proposal  was  successful,  which  could 
contribute  to  the  difference  in  average  fee  values.  As  shown  in  the  Table  45,  the  median 
amount  paid  by  unsuccessful  applicants  was  around  EUR  6 500  and  the  median  paid  by 
successful applicants was around EUR 10 000.  

402 This includes all HE programmes that are organised into consortium projects, i.e. MSCA (DN, 
COFUND, SE), INFRA, WIDERA, Clusters (whole pillar 2), EIC (Pathfinder, Transition), EIE.   

238 

 
 
1.7.2.6. 

External consultancy fee estimation 

To estimate the total fees paid to  external consultancies so far under  Horizon Europe, we 
used survey responses (matched to Corda information) on the rate of consultancy use split 
by  funding  instrument  and  the  median  costs  reported  for  three  applicant  groups: 
consortia,  mono-beneficiary,  and  EIC  Accelerator.  We  based  the  calculation  of  the 
expected  value  of  fees  for the  whole  of  the  Corda  population  on  observed  patterns in  the 
data, in particular two relationships.  

1. 
 The share of proposals involving a consultancy differs between funding instruments.  
2.  The  value  of  the  fee  paid  to  the  consultancy  differs  between  applicants  types  of, 
namely between consortium, single beneficiary, or EIC Accelerator applicant. 

The  above  relationships  are  derived  from  a  close  and  sustained  observation  of  the 
data  but  ultimately  enter  the  analysis  as  assumptions.  The  significance  of  the 
associations was not statistically tested, which would have increased the robustness of the 
analysis.  The  likelihood  of  the  choice  of  involving  an  external  consultancy  depending  on 
funding  instrument  was  not  estimated  using  a  confidence  interval  formula.  To  keep  the 
exercise manageable and to remain proportionate, it is assumed that the above are the 
significant relationships and heterogeneities in the population. 

 A step-by-step methodology follows. 

Step 1 Shares of consultancy involvement by funding instrument - We extrapolated the 
survey  results  on  the  shares  of  consultancy  involvement  to  the  overall  Corda  proposal 
population split by targeted funding instruments. Based on the assumption on relationship 
number  1  and  the  survey  data,  we  calculated  for  each  funding  instrument  a  separate 
confidence interval that provides a range of upper and lower bound for the percentage 
share of proposals involving consultancies.  

The confidence intervals403 are given by the formula: 

CI = p ± 1.96*√(1/n)*p*(1-p)*(N-n/N-1) 

where  

p = share of applicants who used external consultancies for a given funding instrument 

n = survey sample size 

N = population size, given by the number of entities in CORDA data. 

The  confidence  intervals  for  each  funding  instrument  are  reported  in  the  table  below.  We 
found  that,  on  average,  between  21  and  25%  of  applicants  are  hiring  external 
consultants for the application process404. Those applying for Innovation Actions funding 
were  most  likely  to  use  consultants,  with  an  estimate  between  24  and  36%,  while  MSCA 
Postdoctoral Fellowship applicants were the least likely. Low rates of consultant use among 

403 Confidence interval in this case shows that, assuming the pupulation is homogenious, it is 
95% probablity that the true share of applicants using consultancies within the calculated 
interval.   
404 The proportion of survey respondents who used consultancy services were used as an 
estimate of the proportion in the entire population. To express the uncertainty, we calculated the 
confidence interval for the population proportion (95%). Given, that the sample of projects is 
sufficiently large (nearly 50 thousand eligble proposals in the whole HE) we used normal 
approximation (based on z-value). Hence, assuming that the population is homogenous, we can 
concider the provided intervals as likelyhood estimations.  

239 

 
 
PFs should not surprise us, as they are single beneficiary grants and the applicants have less 
resources at their disposal than do consortia. 

Funding instrument 

CI lower bound 
(% of applicants who used 
external consultants) 

CI upper bound 
(% of applicants who used 
external consultants) 

Average 

CSA 

EIC 

IA 

RIA 

MSCA 

MSCA PF 

ERC 

Other405 

21 

17 

15 

24 

20 

19 

4 

15 

21 

25 

25 

23 

36 

26 

27 

6 

18 

25 

Table 51. Estimated rates of consultancy use by funding instrument. Source: Extrapolation from Survey of Horizon Europe 
beneficiaries, conducted in May-July 2023 and Survey of Horizon Europe unsuccessful applicants, conducted in May-July 
2023. Question: ‘Did you receive any support for your Horizon Europe project application/preparation?’ Answer: ‘Yes, our 
organisation received support from a consultancy firm/expert (inside or outside the consortium)’. N=6 807. 

Step  2  Median  value  of  consultancy  fees  by  type  of  applicants  -  The  median  fees 
reported by 658406 survey respondents to the question, ‘What was the total amount that your 
organisation paid to the consultancy firm / expert for the above services in project application 
/ proposal writing?’, were EUR 7 500 for consortia, EUR 2 000 for single beneficiaries, and 
EUR 12 000 for EIC Accelerator. 

Step 3 Total of consultancy fees paid for proposals submitted under Horizon Europe 
until June 2023. To estimate the total consultancy fees paid within each programme part, 
we  multiplied,  for  each  proposal  (“proposal  x”),  the  proportion  of  proposals  involving 
consultancies of its funding instrument (from step1) with the median fee value of its applicant 
type  (step2).  This  was  then  summed  up  over  all  proposals  that  have  been  submitted.  A 
separate  sum  was  calculated  using  the  upper  and  lower  bound  value  of  the  ranges  (95% 
Confidence Interval) from step1 respectively to generate a range of the total fees paid. 

For example, the formula to calculate the upper estimate of fees paid to consultants across 
ERC would be: 

405 The category “Other” includes Cofund, PCP, FPA, and all other instruments not listed. Only 
one valid response from this group was given in the survey, therefore the likelihood estimations 
are an average of all instruments. 
406 Invalid responses were not included. 

240 

 
 
[E] = Sum (Upper CI corresponding to proposal x’s funding instrument * median fee 
corresponding to proposal x’s group) 

for all proposals submitted under ERC 

We estimate that applicants have spent between EUR 39 and 55 million on consultancy 
fees across HE so far. This represents around 0.2% of the total HE budget407. 

Pillar 1 accounted for around 23-25% of the estimated consultancy fees in HE. As the 
majority of proposals submitted to Pillar 1 came from single-beneficiary programmes, which, 
on average, paid lower fees, this result is not surprising. ERC has seen the highest volume 
of fees paid to consultants in Pillar 1, estimated to be between EUR 5 and 6 million, which is 
to be expected as it also saw the highest number of proposals. 

Programme 
part 

Lower 
estimate of 
total 
consultant 
costs (EUR 
mil) 

Upper estimate 
of total 
consultant 
costs (EUR 
mil) 

As a% of 
EC 
contribution 
(lower) 

As a% of 
EC 
contribution 
(upper) 

Number of 
proposals408 

ERC 

MSCA 
(excl. PF) 

4.99 

3.33 

MSCA PF 

1.22 

0.21 

9.90 

1.29 

INFRA 

Pillar 1 

WIDERA 

HE 
overall409 

5.99 

0.10 

0.12 

16 306 

4.73 

0.37 

0.53 

2 335 

1.83 

0.29 

0.21 

0.32 

15 271 

0.04 

0.05 

148 

13.06 

0.14 

0.19 

34 111 

1.90 

0.15 

0.22 

1 079 

39.43 

55.11 

0.16 

0.22 

53 445 

Table 52. Summary of consultancy fee estimates within Pillar 1 and WIDERA. Source: CORDA and survey data 
extrapolations by the study team. 

Survey responses to an open question shed light on the nuances of hiring consultants. We 
only received a few responses, hence these cannot be used to make generalisations about 
the whole population of respondents, but should rather be used as examples for the reasons 
to involve external help. Around five respondents noted that the consultants only reviewed 
and improved proposals, rather than writing them. The consultancy fees were in some cases 

407 According to the DG RTD website, the total HE budget is EUR 95.5 billion, refer here for 
more information: https://research-and-innovation.ec.europa.eu/funding/funding-
opportunities/funding-programmes-and-open-calls/horizon-europe_en . Given that the estimated 
costs and the overall figures of HE budget are recent (HE was launched in 2021 and the 
estimations consider 2021, 2022 and 2023 (until June), the figures in the analysis were not 
adjusted for inflation.  
408 As of June 2023. 
409 Does not include EIT. 

241 

 
 
provided for by a regional grant or by the national ministry (mentioned by five respondents), 
and were often split between the consortia partners.  

1.7.3.  Costs of preparing a Horizon Europe application 

The proposal preparation costs were calculated based on the following survey question: “In 
your  estimation,  how  many  person-days  did  your  organisation  spend  in  preparing  your 
Horizon Europe proposal?” Respondents were asked to choose one of the following options: 
Less than 5 person-days; 6 to 15 person-days; 16 to 25 person-days; 26 to 35 person-days; 
36  to  45  person-days;  46  to  55  person-days;  56  to  65  person  days;  More  than  65  person 
days. 

We complement the quantitative evidence on time cost with qualitative information from the 
survey, and results of the Public Consultation (PC). 

1.7.3.1. 

Proposal preparation effort for consortium-based projects 

The evidence of the effort to prepare a Horizon Europe proposal was collected by surveying 
the  beneficiaries  and  unsuccessful  applicants.  Given  that  all  consortium-based  project 
proposals  follow  generally  the  same  process,  their  effort  (measured in person-days 
spent to prepare the proposal) was assessed together. Such an approach allows us to 
have a maximum possible sample of responses to derive the best estimate of the proposal 
preparation cost.  

The coordinating organisation is naturally responsible for most of the proposal preparation 
effort,  particularly  the  administrative  aspects.  The  partner  organisations  play  more  of  a 
supportive  role  in  the  process.  Therefore,  this  analysis  presents  the  costs  incurred  to  the 
coordinators and partner organisations separately.  

Table 53 below presents the distribution of survey responses based on the answer option. 
The median value for the coordinators was 36-45 days, suggesting that, median consortium 
coordinators  spend  anywhere  between  36  to  45  person-days  to  prepare  a  Horizon 
Europe proposal. For partner organisations, such effort is smaller. Median partners, 
on the other hand, spend between 16 to 25 person-days to prepare a Horizon Europe 
proposal. For mono-beneficiaries, the effort is similar to coordinators: 36-45 person-days. 

The distribution of responses is presented in Table 53 and Figure 49. 

Person-days 

Coordinator (Count of 
survey responses) 

Partner (Count of 
survey responses) 

Mono-beneficiary applicants 
(Count of survey responses) 

Less than 5 
person-days 

6 to 15 
person-days 

16 to 25 
person-days 

26 to 35 
person-days 

102 

121 

200 

322 

103 

272 

694 

924 

929 

2 476 

1 936 (median) 

1 205 

242 

 
Person-days 

Coordinator (Count of 
survey responses) 

Partner (Count of 
survey responses) 

Mono-beneficiary applicants 
(Count of survey responses) 

36 to 45 
person-days 

46 to 55 
person-days 

56 to 65 
person days 

More than 65 
person days 

Total number 
of responses 

310 (median) 

203 

215 

443 

714 

295 

162 

248 

806 (median) 

517 

444 

673 

1 916 

7 965 

4 433 

Table 53. Number of person-days spent preparing Horizon Europe proposal, as reported by successful and unsuccessful 
applicants (all consortium-based programme parts in HE). Source: Survey of Horizon Europe beneficiaries, conducted in 
May-July 2023 and Survey of Horizon Europe unsuccessful applicants, conducted in May-July 2023. Question: ‘In your 
estimation, how many person-days did your organisation spend in preparing your Horizon Europe proposal?’ 
N=14 314 410, 411 

40%

30%

20%

10%

5%

10%

6%

0%

31%

24%

16%

23%

17%

11%

11%

12%

15%

9%

16%

4%

2%

3%

6%

2%

21%

18%

15%

12%

10%

Coordinator

Partner

Mono-beneficiary

Less than 5 person-days

6 to 15 person-days

16 to 25 person-days

26 to 35 person-days

36 to 45 person-days

46 to 55 person-days

56 to 65 person-days

More than 65 person-days

Figure 49. Share of responses on person-days spent preparing Horizon Europe proposal, as reported by successful and 
unsuccessful applicants (all consortium-based and mono-beneficiary programme parts in HE). Source: Survey of Horizon 
Europe beneficiaries, conducted in May-July 2023 and Survey of Horizon Europe unsuccessful applicants, conducted in 
May-July 2023. Question: ‘In your estimation, how many person-days did your organisation spend in preparing your Horizon 
Europe proposal?’ N=14 314. Median value highlighted. 

Time effort spent by coordinators 

The  proposal  preparation  effort  is  affected  by  the  factors  internal  to  the  Framework 
programme  (such  as  the  type  of  MGA  and  funding  instrument) as  well  as  the  factors  that 

410 These responses were taken from the survey responses with the following question “In your 
estimation, how many person-days did your organisation spend in preparing your Horizon 
Europe proposal?* and combined with other survey questions helping us identify the size of the 
consortia and the role of the respondents. We took the median numbers within 2 standard 
deviations. 
411 Please note that here and other figures in efficiency analysis, we have excluded responses 
such as “do not know/cannot answer”. To see survey responses including this option please 
refer to the survey synopsis report in Annex 7. 

243 

 
 
 
could be, to some extent, controlled by the consortium (such as project length and the size 
of  the  consortium).  For  example,  the  survey  results  show  that  (see  Table  51)  proposals 
associated with lump sum grants take significantly less effort to prepare. The survey shows 
that the median respondents reported that 16 to 25 person-days were enough to prepare a 
lump sum application (an in-depth analysis of the effect of lump sums is presented in the sub-
section below). In contrast, the median value for other MGA types were between 36 and 46 
person-days. 

260; 16%

373; 24%

7; 21%

7; 21%

67; 22%

6; 18%

6; 18%

56; 18%

55; 18%

41; 13%

3; 9%

3; 9%

30; 10%

28; 9%

1; 3%

16; 5%

12; 4%

252; 16%

184; 12%

153; 10%

175; 11%

25%

20%

15%

10%

102; 6%

79; 5%

5%

0%

Horizon Action Grant Budget-Based

Horizon Lump Sum Grant

Horizon Unit Grant

Less than 5 person-days

6 to 15 person-days

16 to 25 person-days

26 to 35 person-days

36 to 45 person-days

46 to 55 person-days

56 to 65 person-days

More than 65 person-days

Figure 50. Share of responses on person-days spent preparing Horizon Europe proposal, as reported by the coordinators of 
successful and unsuccessful applications, by the type of MGA. The number of responses reported for all consortium-based 
programme parts in HE. Source: Survey of Horizon Europe beneficiaries, conducted in May-July 2023 and Survey of 
Horizon Europe unsuccessful applicants, conducted in May-July 2023. Question: ‘In your estimation, how many person-days 
did your organisation spend in preparing your Horizon Europe proposal? N=1 916. Median value highlighted.412 

The funding instruments were categorised into three groups: High expense level (RIA, IA, 
PCP and COFUND), Medium expense level (CSA, MSCA COFUND/DN, SE) and a separate 
group  for  EIC  (also  called  medium  expense  level)413.  The  cross-tabulation  of  survey 
responses by the project expense level show that projects funded under RIA, IA, PCP and 
COFUND  actions  require  more  effort  than  medium-expense  level  projects.  A  median 
respondent spent  between 46 and 55 person-days  preparing and submitting a  high-
expense level proposal. A median value for the medium expense level proposals was 36-
45 days and even lower for EIC applicants at 26-35 days.  

412 Please note that here and other figures in efficiency analysis, we have excluded responses 
such as “do not know/cannot answer”. To see survey responses including this option please 
refer to the survey synopsis report in Annex 7. 
413 The funding instruments were categorised following the example of proposal writing cost 
analysis in European Commission, Directorate-General for Research and Innovation, Interim 
evaluation of Horizon 2020 – Commission staff working document, Publications Office, 
2017, https://data.europa.eu/doi/10.2777/220768 Annex 1, p. 71.  

244 

 
 
 
27%

15%

15%

11%

12%

7%

7%

5%

18%

21%

17%

12%

11%

10%

5%

5%

21%

17%

16%

15%

8%

6%

9%

8%

30%

20%

10%

0%

High expense level: RIA, IA, PCP
and COFUND
Less than 5 person-days
26 to 35 person-days

Medium expense level: CSA,
MSCA Cofund/DN/SE

Medium expense level: EIC

6 to 15 person-days
36 to 45 person-days

16 to 25 person-days
46 to 55 person-days

Figure 51. Number of person-days spent preparing Horizon Europe proposal, as reported by the coordinators of successful 
and unsuccessful applications, by project expense level. The number of responses reported414. Source: Survey of Horizon 
Europe beneficiaries, conducted in May-July 2023 and Survey of Horizon Europe unsuccessful applicants, conducted in 
May-July 2023. Question: ‘In your estimation, how many person-days did your organisation spend in preparing your Horizon 
Europe proposal?’ N=1 916. Median value highlighted. 

The survey results also show that longer projects take more effort to apply for. Figure 
52  below  illustrates  that  short  projects  (up  to  2  years)  take  a  median  of  26  to  35  days  to 
prepare an application. For projects that are planned to take between 2 and 4 years or longer, 
it takes the coordinator between 46 and 55 person-days to prepare. In addition, it is worth to 
mention, that around a third (n=49) of respondent coordinators of projects that last 4 years 
or longer reported that they spent more than 65 person-days to prepare a proposal.  

13; 18%

12; 17%

11; 15%

10; 14%

8; 11%

8; 11%

5; 7%

4; 6%

88; 11%

63; 8%

36; 4%

151; 19%

164; 20%

118; 15%

98; 12%

93; 11%

153; 17%

49; 34%

145; 16%

222; 25%

98; 11%

89; 10%

95; 11%

49; 6%

38; 4%

26; 18%

15; 10%

18; 12%

16; 11%

9; 6%

7; 5%

5; 3%

40%

30%

20%

10%

0%

Up to 2 years

2-3 years

3-4 years

4 years and longer

Less than 5 person-days
36 to 45 person-days

6 to 15 person-days
46 to 55 person-days

16 to 25 person-days
56 to 65 person days

26 to 35 person-days
More than 65 person days

Figure 52. Number of person-days spent preparing Horizon Europe proposal, as reported by the coordinators of successful 
applications, by project duration. Source: Survey of Horizon Europe beneficiaries, conducted in May-July 2023. Question: ‘In 
your estimation, how many person-days did your organisation spend in preparing your Horizon Europe proposal?’ N=703. 
Median value highlighted. 

Successful  proposals  take  longer  to  prepare  than  unsuccessful  proposals.  This  is 
particularly true for successful proposals of projects that are planned to last at least 3 years 
or longer. Such an observation potentially hints at the fact that the effort coordinators put into 
preparing the proposal makes a difference. 

Evidently,  larger  consortiums  require  more  time  from  coordinators  in  the  proposal 
preparation process. Further analysis of the comparison between the size of the consortium 
and the person-days required to submit a proposal indicated that the effort needed increases 
with an increment of approximately 15 partners. Proposals that included only one partner (i.e. 

414 Here we refer to EIC Pathfinder and EIC Transitions. 

245 

 
 
 
 
only the coordinator)415 required somewhat less time than proposals with larger consortiums. 
This further illustrates that the proposal preparation burden is very much related to the size 
of the consortium.  

The consortium size proved to correlation with the number of days required to prepare 
the  Horizon  Europe  application.  When  looking  into  a  combination  of  aspects  discussed 
above, e.g., project expense level and the size of the consortium or duration of the project 
and  the  consortium  size,  the  only  significant variation  in the  effort required  to  prepare  the 
application  was  visible  at  the  different  consortium  sizes  in  the  proposal.  Therefore,  we 
conclude that to consider the costs incurred by the proposal coordinators, the breakdown by 
the size of the consortium should be taken into account. 

50%

45%

40%

35%

30%

25%

20%

15%

10%

5%

0%

15; 44%

240; 18%

142; 11%

163; 32%

56; 11%

262; 20%

230; 17%

71; 14%

62; 12%

22; 4%

69; 14%

5; 15%

6; 18%

5; 15%

152; 11%

95; 7%

143; 11%

70; 5%

40; 8%

25; 5%

1; 3%

1; 3%

1; 3%

7; 18%

6; 15%

7; 18%

6; 15%

5; 13%

3; 8%

3; 8%

3; 8%

1 partner

2-14 partners

15-30 partners

31+ partners

Less than 5 person-days

6 to 15 person-days

16 to 25 person-days

26 to 35 person-days

36 to 45 person-days

46 to 55 person-days

56 to 65 person days

More than 65 person days

Figure 53. Number of person-days spent preparing Horizon Europe proposal, as reported by the coordinators of successful 
and unsuccessful applications, by consortium size. The number of responses reported (all consortium-based programme 
parts in HE). Source: Survey of Horizon Europe beneficiaries, conducted in May-July 2023 and Survey of Horizon Europe 
unsuccessful applicants, conducted in May-July 2023. Question: ‘In your estimation, how many person-days did your 
organisation spend in preparing your Horizon Europe proposal?’ N= 1 916. Median value highlighted. 

Effort spent by partners 

Not all partner organisations are involved in the proposal preparation effort, but those 
that do, according to our survey results’ median response, spend between 16 and 25 
person-days. Nevertheless, some variation between the type of proposal was observed. As 
shown in a figure below, applicants aiming for a lump sum grant reportedly spend less time. 
The median respondent reported needing between 6 and 15 person-days to contribute to a 
proposal writing. Nevertheless, please note that the sample size for lump sums instrument is 
very small (n=56) and with an increased sample size, after a few years we may see a change 
in such results. 

415 Please note that the HE programme parts included in this analysis to which they applied to 
are not monobeneficiary by design. Analysis of the proposal preparation effort for 
monobeneficiary proposals is discussed below. 

246 

 
 
 
60%

40%

20%

0%

2,137; 30%

27; 48%

312; 36%

224; 26%

820; 12%

1702; 24%

1,075; 15%

642; 9%

10; 18%

273; 4%

231; 3%

157; 2%

10; 18%

6; 11%

1; 2%

99; 11%

1; 2%

1; 2%

; 0%

124; 14%

71; 8%

22; 3%

16; 2%

4; 0%

Horizon Action Grant Budget-
Based

Less than 5 person-days
26 to 35 person-days

Horizon Lump Sum Grant

Horizon Unit Grant

6 to 15 person-days
36 to 45 person-days

16 to 25 person-days
46 to 55 person-days

Figure 54. Share of responses on person-days spent preparing Horizon Europe proposal, as reported by the consortium 
partners of successful and unsuccessful applications, by the type of MGA. The number of responses reported for all 
consortium-based programme parts in HE. Source: Survey of Horizon Europe beneficiaries, conducted in May-July 2023 
and Survey of Horizon Europe unsuccessful applicants, conducted in May-July 2023. Question: ‘In your estimation, how 
many person-days did your organisation spend in preparing your Horizon Europe proposal?’ N= 7965. Median value 
highlighted. 

Partners in very large consortiums also report spending less time compared to the median for all 
proposal partners. While the median partner spends between 16 and 25 days, partners involved 
in proposals with more than 31 participants report spending between 6 and 15 days. Such a trend 
suggests that proposal writing does not require the full involvement of all partners. 

1.7.3.2. 

Proposal preparation effort for mono-beneficiary projects (ERC and MSCA 
PF) 

The proposal preparation effort for the mono-beneficiary applications (such as ERC and 
MSCA  PF)  is  comparable  to  that  observed  in  the  consortium  coordinators’  sample. 
Generally,  a  median  ERC  Principal  Investigator  and  MSCA  Postdoctoral  Fellow  spend 
between 36 and 45 days preparing a Horizon Europe application. According to the beneficiary 
and unsuccessful applicant surveys, there is no significant difference between ERC and MSCA 
PF  in  terms  of  the  proposal  preparation  burden.  Hence,  further  analysis  considers  these  two 
groups together. 

924; 21%

806; 18%

319; 19%

313; 19%

318; 19%

694; 16%

673; 15%

238; 14%

205; 12%

517; 12%

444; 10%

186; 11%

611; 22%

487; 18%

456; 17%

355; 13%

312; 11%

258; 9%

272; 6%

103; 2%

75; 4%

26; 2%

197; 7%

77; 3%

25%

20%

15%

10%

5%

0%

Total

ERC

MSCA PF

Less than 5 person-days
26 to  35 person-days

6 to 15 person-days
36 to 45 person-days

16 to 25 person-days
46 to 55 person-days

Figure 55. The number of person-days spent preparing Horizon Europe proposal, as reported by successful and 
unsuccessful applicants (ERC and MSCA PF) (the number of responses and percent reported). Source: Survey of Horizon 
Europe beneficiaries, conducted in May-July 2023 and Survey of Horizon Europe unsuccessful applicants, conducted in 

247 

 
 
 
 
May-July 2023. Question: ‘In your estimation, how many person-days did you spend in preparing your Horizon Europe 
proposal?’ N=4 433. Median value highlighted.416 
The  notable  difference  in  the  proposal  preparation  effort  can  be  noticed  for  the  proposals 
following the Lump sum grant scheme. As was the case for consortium coordinators, the 
lump sum grant proposals take less time – the median time effort is between 26 and 
35 person-days. Given the importance of such a novelty in the Horizon Europe framework 
and consistent findings from the survey, a separate in-depth analysis of lump sum proposals 
is prepared in the following section. Nevertheless, please note that the sample size for lump 
sums instrument is very small (n=34) and with an increased sample size, after a few years 
we may see a change in such results. 

312; 19%

308; 19%

316; 19%

230; 14%

203; 12%

186; 11%

9; 26%

8; 24%

7; 21%

5; 15%

487; 18%

611; 22%

456; 17%

312; 11%

355; 13%

258; 9%

66; 4%

25; 2%

1; 3%

2; 6%

2; 6%

; 0%

197; 7%

77; 3%

30%

25%

20%

15%

10%

5%

0%

Horizon Action Grant Budget-
Based

Less than 5 person-days
26 to  35 person-days
56 to 65 person days

Horizon Lump Sum Grant

Horizon Unit Grant

6 to 15 person-days
36 to 45 person-days
More than 65 person days

16 to 25 person-days
46 to 55 person-days

Figure 56. The number of person-days spent preparing Horizon Europe proposal, as reported by the ERC PIs and MSCA PF 
fellows of successful and unsuccessful applications, by the type of MGA. Source: Survey of Horizon Europe beneficiaries, 
conducted in May-July 2023 and Survey of Horizon Europe unsuccessful applicants, conducted in May-July 2023. Question: 
‘In your estimation, how many person-days did you spend in preparing your Horizon Europe proposal?’ N=4 433. Median 
value highlighted. 

Unlike the trend observed for the consortium coordinators, the survey results do not indicate 
that  successful  mono-beneficiary  applicants  (ERC,  MSCA  PF)  spend  more  time  on  their 
applications than unsuccessful ones. The figure below illustrates that the median time effort 
is between 36 and 45 days. 

416 Please note that here and other figures in efficiency analysis, we have excluded responses 
such as “do not know/cannot answer”. To see survey responses including this option please 
refer to the survey synopsis report in Annex 7. 

248 

 
 
 
231; 22%

204; 20%

109; 10%

166; 16%

147; 14%

50; 5%

23; 2%

109; 10%

28; 11%

7; 3%

4; 2%

61; 25%

196; 25%

51; 21%

35; 14%

33; 13%

27; 11%

153; 19%

119; 15%

82; 10%

105; 13%

76; 10%

43; 5%

19; 2%

30%

25%

20%

15%

10%

5%

0%

Total

Less than 5 person-days
26 to  35 person-days
56 to 65 person days

ERC

6 to 15 person-days
36 to 45 person-days
More than 65 person days

MSCA PF
16 to 25 person-days
46 to 55 person-days

Figure 57. Number of person-days spent preparing Horizon Europe proposal, as reported by beneficiaries (ERC and MSCA 
PF). Source: Survey of Horizon Europe beneficiaries, conducted in May-July 2023. Question: ’In your estimation, how many 
person-days did you spend in preparing your Horizon Europe proposal?’ N=1 039. Median value highlighted. 

Finally, Table 54 shows an overview of all the findings discussed so far, with both number of 
responses and percent of total responses in each category reported. 

249 

 
 
Coordinator 

Partner 

Mono-beneficiary 
(ERC and MSCA PF) 

Person-
days 

Successful 
(% from total) 

Unsuccessful 
(% from total) 

Successful 
(% from 
total) 

Unsuccessful 
(% from total) 

Successful 
(% from total) 

Unsuccessful (% 
from total) 

< 5 

24 (3%) 

78 (6%) 

358 (11%) 

571 (13%) 

23 (2%) 

80 (2%) 

6 to 15 

34 (5%) 

87 (7%) 

991 (29%) 

1 485 (33%) 

50 (5%) 

222 (7%) 

16 to 25 

50 (7%) 

150 (12%) 

Median 
response: 
847 (25%) 

Median 
response: 1 
089 (24%) 

147 (14%) 

547 (16%) 

26 to 35 

117 (17%) 

205 (17%) 

572 (17%) 

633 (14%) 

231 (22%) 

693 (20%) 

36 to 45 

115 (16%) 

Median 
response: 195 
(16%) 

328 (10%) 

386 (8%) 

Median 
response: 
204 (20%) 

Median 
response: 602 
(18%) 

46 to 55 

Median 
response: 
90 (13%) 

113 (9%) 

138 (4%) 

157 (3%) 

109 (10%) 

408 (12%) 

56 to 65 

76 (11%) 

139 (11%) 

71 (2%) 

91 (2%) 

109 (10%) 

335 (10%) 

> 65 

197 (28%) 

246 (20%) 

100 (3%) 

148 (3%) 

166 (16%) 

507 (15%) 

703 (100%) 

1 213 (100%) 

3 405 
(100%) 

4 560 (100) 

1 039 
(100%) 

3 394 (100%) 

Table 54. Number of person-days spent preparing Horizon Europe proposal, as reported by the coordinators and applicants 
of successful and unsuccessful applications. The number of responses reported (all consortium-based programme parts in 
HE). Source: Survey of Horizon Europe beneficiaries, conducted in May-July 2023 and Survey of Horizon Europe 
unsuccessful applicants, conducted in May-July 2023. Question: ‘In your estimation, how many person-days did your 
organisation spend in preparing your Horizon Europe proposal?’ N=14 314. 

A summary of all the findings above is shown in  Table 55 below. The analysis shows that 
proposal coordinators are responsible for most of the preparation effort. The time needed to 
prepare  the  proposal  increases  with  the  size  of  the  consortium.  The  proposal  preparation 
effort needed from the partners does not seem to vary  based on the characteristics of the 
project or proposal. For ERC and MSCA PF, the proposal preparation effort is comparable to 
that of consortium proposal coordinators.   

250 

 
 
 
 
 
Group of proposals 

Median person days 

Consortium-based programme parts: MSCA (DN, SE, Cofund), INFRA, WIDERA, all Pillar 2, EIE, 
EIC (Pathfinder and Transitions) 

One beneficiary 

16 to 25 person-days 

Small consortium coordinators (2-14 partners) 

36 to 45 person-days 

Large consortium coordinators (15-30 partners) 

46 to 55 person-days 

Very large consortium coordinators (31+ 
partners) 

56 to 65 person days 

Lump sum proposal coordinators 

16 to 25 person-days 

Lump sum proposal partners 

6 to 15 person-days 

Partners in small and large consortiums (2-30 
partners), incl. EIC 

Partners in very large consortiums (31+ 
partners) 

16 to 25 person-days 

6 to 15 person-days 

EIC proposal coordinators 

26 to 35 person-days 

Mono-beneficiary programme parts 

MSCA PF and ERC 

36 to 45 person-days 

Lump sum proposal 

16 to 25 person-days 

Table 55. Summary of survey findings regarding costs of the application. Source: compiled by the study team. 

The  majority  (54%)  of  all  successful  and  unsuccessful  applicants  agree  to  a  large or very 
large extent that ‘The application costs (total time and resources needed) are proportionate 
to  the  volume  of  funding  requested  in  the  proposal’  (see  Figure  58).  Such  a  finding  is 
consistent across the programme parts related to the Excellent Science evaluation study.     

251 

 
All respondents (excl. EIC Accelerator)

2134

6709

4779

2027

748

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

ERC

384

781

MSCA (excl. PF)

97

MSCA PF

596

INFRA

23

517

102

1227

427

84

338

135

66

242

86

713

244

119

55

18

41

WIDERA

106

331

230

100

To a very large extent

To a large extent

To a moderate extent

To a small extent

Not at all

Figure 58. Perceived proportionality of the proposal writing costs (number of responses reported for each bin). Source: 
Survey of Horizon Europe beneficiaries, conducted in May-July 2023 and Survey of Horizon Europe unsuccessful 
applicants, conducted in May-July 2023. Question: ‘The application costs (total time and resources needed) are 
proportionate to the volume of funding requested in the proposal: To what extent do you agree with the following statements 
about the effort needed to prepare and submit your Horizon Europe project?’  N=16 397. 

When  looking  at  the  results  from  only  unsuccessful  applicants,  we  find  that  that  trend  is 
similar but slightly less positive: a slight minority (48%) of unsuccessful applicants agreed to 
a large or very large extent  that the application costs were proportionate to the volume of 
funding requested. 

All respondents (excl. EIC Accelerator)
ERC
MSCA (excl. PF)
MSCA PF
INFRA
WIDERA

644

284

57

362

10
64

2698

2238

998

682

298

833

36

218

273

28

124

300

167

538

204

22

63

131

395
63
63
104
5
25

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

To a very large extent

To a large extent

To a moderate extent

To a small extent

Not at all

Figure 59. Perceived proportionality of the proposal writing costs – Unsuccessful applicants. Source: Survey of Horizon 
Europe unsuccessful applicants, conducted in May-July 2023. Question: ‘The application costs (total time and resources 
needed) are proportionate to the volume of funding requested in the proposal: To what extent do you agree with the 
following statements about the effort needed to prepare and submit your Horizon Europe project?’ N=6 973.417 

When asked to compare the effort needed to apply for Horizon 2020418, the largest share of 
HE  applicants  (47%)  were  relatively  indifferent:  they  neither agreed  nor disagreed  with 
the with the statement that proposal preparation and submission in Horizon Europe is simpler 
than those in Horizon 2020. Pillar 1 and WIDERA applicants did not show any deviation from 
this trend, with the bulk of applicants in each programme part indicating neutrality. WIDERA 
was  slightly  more  positive,  with  39%  agreeing  or  strongly  agreeing  that  the  process  was 
simpler, while MSCA PF applicants were the most critical, with 32% disagreeing or strongly 
disagreeing that the proposal preparation and submission in Horizon Europe is simpler than 
in H2020. 

417 Please note that here and other figures in efficiency analysis, we have excluded responses 
such as “do not know/cannot answer”. To see survey responses including this option please 
refer to the survey synopsis report in Annex 7. 
418 Survey asked the following question: “Proposal preparation and submission in Horizon 
Europe is simpler than those in Horizon 2020: Based on your overall Framework programme 
experience, would you agree or disagree with the following statements on the project’s lifecycle 
processes in Horizon Europe, compared to Horizon 2020?” 

252 

 
 
 
 
100%

80%

60%

40%

20%

0%

47%; 4082

34%; 2927

All respondents
(excl. EIC
Accelerator)

19%; 1663

24%; 167

22%; 188

59%; 410

51%; 438

32%; 211

49%; 322

21%; 41

19%; 98

45%; 90

42%; 213

18%; 125

ERC

27%; 229

19%; 121

34%; 68

39%; 199

MSCA (excl.
PF)

MSCA PF

INFRA

WIDERA

Strongly agree and Rather agree

Neither agree nor disagree

Strongly disagree and Rather disagree

Figure 60. Perceived difference of proposal preparation effort in HE and H2020. Source: Survey of Horizon Europe 
beneficiaries, conducted in May-July 2023 and Survey of Horizon Europe unsuccessful applicants, conducted in May-July 
2023. Question: “Proposal preparation and submission in Horizon Europe is simpler than those in Horizon 2020: Based on 
your overall FP experience, would you agree or disagree with the following statements on the project’s lifecycle processes in 
Horizon Europe, compared to Horizon 2020?” N= 8 661419. 

1.7.4.  Simplification efforts 

1.7.4.1. 

Simplification effects of the lump sum model  

One of the most prominent simplification instruments introduced in the HE was the lump sum-
based project funding. While it is too early to assess the effect of such simplification on 
the whole project life-cycle (from application to audit) some first insights could already be 
drawn. Overall, 44 respondents suggested that lump sums do not increase the time needed 
to prepare a proposal, of which 17 were ERC applicants420, (see Figure 61). 

A half (36) of the respondents claimed that such an arrangement increased the time needed 
for an application. In the case of lump sum funding the workload was shifted from financial 
reporting  during  the  implementation  phase  to  detailed  budget  calculations  in  the 
proposal phase. This added another layer of complexity and additional workload to the pre-
award proposal submission process. On the other hand, the lump-sum model is not designed 
to reduce the application burden but rather simplify project implementation. 

419 Please note that here and other figures in efficiency analysis, we have excluded responses 
such as “do not know/cannot answer”. To see survey responses including this option please 
refer to the survey synopsis report in Annex 7. 
420 So far one call in ERC has been organised with the lump sum budget (ERC-2022-POC1) (as 
of June 2023).  

253 

 
 
 
 
 
Figure 61. Share of respondents agreeing that lump sum increases the time needed for proposal writing. Source: Survey of 
Horizon Europe beneficiaries, conducted in May-July 2023 Question: ‘Lump-sum increases the time needed for proposal 
writing compared to a standard proposal: To what extent do you agree with the following statements on lump-sum funding in 
your Horizon Europe project? ’N= 80.421 

One  potential  simplification  related  to  the  lump  sum  funding  is  related  to  the 
development of the project work plan and division into work packages – 60 of the survey 
respondents agree to a very large or large (see figure below).  

Figure 62. Share of respondents agreeing that lump sum simplifies the development of the project workplan. Source: Survey 
of Horizon Europe beneficiaries, conducted in May-July 2023 Question: ‘Lump-sum simplifies development of the project 
work plan and division into work packages: To what extent do you agree with the following statements on lump-sum funding 
in your Horizon Europe project?’ N= 95. 

Lump sums are clearly perceived to simplify reporting at project implementation stage. 
This finding is consistent in the survey and public consultation responses. Only two survey 
respondents  out  of  80  (3%)  disagreed  with  the  statement  that  ‘Lump-sum  simplifies  the 
reporting and limits the risks of errors (see Figure 63 below). 

Figure 63. Share of respondents agreeing that lump sum simplifies reporting and limits the risks of errors. Source: Survey of 
Horizon Europe beneficiaries, conducted in May-July 2023 Question: ‘Lump-sum simplifies the reporting and limits the risks 
of errors: To what extent do you agree with the following statements on lump-sum funding in your Horizon Europe project?’ 
N= 80. 

421 Please note that here and other figures in efficiency analysis, we have excluded responses 
such as “do not know/cannot answer”. To see survey responses including this option please 
refer to the survey synopsis report in Annex 7. 

254 

 
 
 
 
 
Half (50%; 40 in total) of beneficiary survey respondents agree that simplified funding 
comes with some financial risks, either to a very large, large, moderate or small extent 
(see Figure 63). The perceived risks may be linked to uncertainty around lump sum funding, 
which is a rather new addition to the Framework programmes with which not many project 
partners could collect experience at payment stages. The perception of risk is not correlated 
with  any  specific  characteristics  of  the  respondents.  There  were  no  differences  between 
responses from SMEs versus non-SME participants, nor between newcomers and repeated 
FP participants.   

Figure 64. Share of respondents agreeing that lump sum increases the financial risks for project participants. Source: 
Survey of Horizon Europe beneficiaries, conducted in May-July 2023 Question: ‘Lump-sum increases the financial risks for 
project participants: To what extent do you agree with the following statements on lump-sum funding in your Horizon Europe 
project?’ N= 81. 

A  small  number  of  organisations  responding  to  the  public  consultation  in  winter  2022/23 
mentioned a few risks related to lump sum funding. The nature of the consultation does not 
allow to confirm whether the respondents were recipients of the lump sum grants and their 
views were based on actual experience with the funding model. The risks mentioned include: 
high  dependency  on  consortium  partners  for  the  implementation  of  work  packages  and 
eligibility of costs. To avoid such a risk partners might start constructing their Work Packages 
in a way that they include fewer and smaller tasks, which in turn can lead to less risky and 
less  collaborative  projects.  One  public  consultation  respondent  noted  that  the  practice  of 
linking funding to delivered activities results in the artificial splitting of work packages solely 
to facilitate funding flow rather than aligning them with the logical progression of the research 
itself. Risks from inadequately low lump sum amounts in that would not account for inflation, 
were  flagged  by  another  respondent.  Since  the  public  consultation,  lump  sum  rates  have 
already  been  inflation  adjusted  for  by  the  EC.  Finally,  public  consultation  respondents 
mention the need for more guidance and clarity. The Model Grant Agreement (MGA) (in use 
for lump sum grants at the time of the public consultation) was flagged as not well-adjusted 
to lump sum financing and seen to increase risks concerning financial liability, reporting, and 
audits. Public consultation respondents asked for additional guidance on these aspects. The 
concerns  may  have  been  at  least  partially  addressed  by  a  number  of  updates  and 
improvements  that  have  taken  place  since  winter 2022/23  in  response  to  the  consultation 
responses. 

On  the  other  hand,  the  public  consultation  also  showed  strong  support  for  the  further 
application of lump sum funding mechanism to small-scale projects. Projects that are funded 
under CSA instrument (small consortia) and projects with higher TRL were also mentioned 
as good candidates to be funded under the lump sum mechanism. One public consultation 
respondent explicitly supported that lump sums make the programme simpler by removing 
the need to report actual costs, meaning easier access to the programme, especially for small 
organisations  and  newcomers.  Other  public  consultation  respondents  questioned  whether 
financial reporting imposed any additional administrative burden to start with: the usefulness 
of a simplified reporting process was thus questionable as some consortia would still have to 
collect  all  financial  information  due  to  internal  organisational  regulations  or  to  be  able  to 

255 

 
 
prepare for an audit422. In addition, given that project partners have to report the proof of the 
project results, the financial simplification was eliminated by the additional effort to collect the 
lump  sums.  All  in  all,  even  the  small  number  of  free  text  public  consultation  responses  in 
winter 2023/22 demonstrated a considerable level of uncertainty and apprehension around 
the rules and processes of lump sum funding. There are some indications that the situation 
improved to some extent until summer 2023, as the targeted beneficiary survey responses 
show that the lump sum funding mechanism is generally perceived in a more positive than 
negative  light.  The  increasing  practical  experience  with  all  stages  of  lump  sum  funding 
should, however, be used to monitor potential risk areas and address beneficiaries’ concerns. 

1.7.4.2. 

Cost-saving potential of the Seal of Excellence  

A Seal of Excellence is seen as one of the tools to address the inefficiencies arising from the 
use of resources for unsuccessful applications. To date (June 2023), the EC  has awarded 
3 818 applications with the Seal of Excellence. The absolute majority of these proposals were 
unsuccessful applications submitted to the MSCA Postdoctoral Fellowships. In an ideal case, 
around  21.7%  (3 222  in  total)  of  the  unsuccessful  projects  in  the  MSCA  could  experience 
cost  savings  if  due  to  the  SoE  (assuming  that  all  awarded  proposals  receive  funding 
somewhere else). There were also 11 WIDERA proposals (1,4% of WIDERA unsuccessful 
applications) that received SoE.  

In actuality, the real cost-saving impact of the Seal of Excellence initiative is much lower. Half 
(50%) of the respondents who provided their answer to ‘Did the Seal of Excellence received 
for  your  unsuccessful  project proposal  make  it  easier  for  you  to  secure  the  funding  from 
national/regional funding sources?’ said ‘No’ (see Figure 65). To have the EU “quality mark” 
is not enough to secure funding for the unsuccessful applicants. The final outcome of their 
unsuccessful application to a large extent depends on the availability of funds in the national 
and regional funding schemes.     

Figure 65. Perceived usefulness of SoE to secure the funding from national / regional funding sources (MSCA PF). Source: 
Survey of Horizon Europe unsuccessful applicants, conducted in May-July 2023. Question: ‘Did the Seal of Excellence 
received for your unsuccessful project proposal make it easier for you to secure the funding from national/regional funding 
sources?’ N= 66.423 

422 Assuming the respondent was familiar with the rules around lump sum funding, they may 
have referred to the beneficiary’s internal audits, as lump sum projects are not subject to audit 
under Horizon Europe. 
423 Please note that here and other figures in efficiency analysis, we have excluded responses 
such as “do not know/cannot answer”. To see survey responses including this option please 
refer to the survey synopsis report in Annex 7. 

256 

 
 
 
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%

1

1

1

3

3

9

3

1

3

1

1

4

1

1

2

1

4

2

2

1

2

2

1

1

Yes

Yes, to some extent

No

Figure 66. Perceived usefulness of SoE to secure the funding from national/regional funding sources (MSCA PF), a 
breakdown by EU country. Source: Survey of Horizon Europe unsuccessful applicants, conducted in May-July 2023. 
Question: ‘Did the Seal of Excellence received for your unsuccessful project proposal make it easier for you to secure the 
funding from national/regional funding sources?’. Presented by respondents’ country of residence. N= 51. 

1.7.5.  Administrative costs for beneficiaries  

The beneficiaries survey shows that a median consortium-run project allocates around 
6%-10% of the project budget for administrative tasks (such as project reporting, project 
financial  management,  and  similar).  The  reported  share  differs  between  coordinators  and 
partners. Coordinators’ median value falls between 11 and 15%, nevertheless mode value 
(the most frequently chosen value) is at 6%-10%, which allows us to assume 6-10% for a 
total  population  of  projects.  The  responses  are  given  visually  in  Figure  67,  followed  by  a 
numerical breakdown of responses in Table 56. Both median and mode survey responses 
suggest the same finding. In addition, there was no variation observed in terms of different 
sizes of project teams, different length of the projects or by programme part. 

31%

28%

27%

18%

18%

18%

11%

8%

17%

7%

7%

6%

6%

5%

1%

1%

2%

1%

35%

30%

25%

20%

15%

10%

5%

0%

30%
28%

23%

20%

20%

19%

20%

16%
15%

14%

18%

15%

10%

11%

10%

10%

9%

Less than 1%

1-3%

4-5%

6-10%

11-15%

16-20%

More than
20%

Pillar 1

Pillar 2

Pillar 3 WIDERA

HE

Figure 67. Distribution of responses to the survey question asking what percentage of the budget was spent on 
administrative tasks. Source: Survey of Horizon Europe unsuccessful applicants, conducted in May-July 2023. Question: 
‘Did the Seal of Excellence received for your unsuccessful project proposal make it easier for you to secure the funding from 
national/regional funding sources?’. Presented by respondents’ country of residence. N= 51.424 

424 Please note that here and other figures in efficiency analysis, we have excluded responses 
such as “do not know/cannot answer”. To see survey responses including this option please 
refer to the survey synopsis report in Annex 7. 

257 

 
 
 
 
The breakdown in the figure below reveals that the overall trend is largely mirrored across 
Pillar  1,  with  the  exception  of  INFRA,  which  saw  a  median  respondent  spending  slightly 
higher, 11-16%, of the budget on administrative costs. 

35%

30%

25%

20%

15%

10%

5%

0%

29%

30%

27%

28%

25%

23%

23%

20%

19%

15%

13%

13%

18%

15%

14%

10%

23%

24%

23%

19%

17%

18%

7%

12%

11%

8%

7%

14%

4%

2%

1%

1%

8%

4%

6%

4%

Less than 1%

1-3%

4-5%

6-10%

11-15%

16-20%

13%

11%

12%

7%

11%

10%

More than
20%

ERC

MSCA (no PF)

MSCA PF

INFRA

WIDERA

Horizon Europe

Figure 68. Distribution of responses to the survey question asking what percentage of the budget was spent on 
administrative tasks – Pillar 1 programme part. Source: Survey of Horizon Europe beneficiaries, conducted in May-July 
2023. Question: ‘In your estimation, what is the percentage share of your Horizon Europe project budget that is spent on 
administrative tasks (e.g. project reporting, project financial management, and similar)’. Number of responses: 5 183. 
Median value highlighted. 

Table 56 shows a numeric breakdown of the survey responses for coordinators and partners. 

Coordinators 

Partners 

Total of beneficiaries 

Less than 1% 

1-3% 

4-5% 

6-10% 

11-15% 

16-20% 

More than 20% 

2 

25 

107 

181 

175 

114 

88 

49 

243 

633 

51 

268 

740 

1 044 

1 225 

745 

599 

375 

920 

713 

463 

Total number of responses 

692 

3 688 

4 380 

Table 56. Share of Horizon Europe project budget spent on administrative tasks, as reported by consortium projects’ 
beneficiaries. The number of responses reported. Source: Survey of Horizon Europe beneficiaries, conducted in May-July 
2023. Question: ‘In your estimation, what is the percentage share of your Horizon Europe project budget that is spent on 
administrative tasks (e.g. project reporting, project financial management, and similar)’. Number of responses: 4 380. 

258 

 
 
 
Similarly, the share of budget that mono-beneficiary projects dedicate to administrative tasks 
is at 6% – 10%. This also corresponds to the largest percentage (mode response) (26%) of 
total responses from ERC and MSCA PF respondents.   

Beneficiaries of MSCA PF and ERC 

Less than 1% 

1-3% 

4-5% 

6-10% 

11-15% 

16-20% 

More than 20% 

Total number of responses 

50 

83 

138 

178 

97 

77 

61 

684 

Table 57. Share of Horizon Europe project budget spent on administrative tasks, as reported by beneficiaries (ERC and 
MSCA PF). The number of responses reported. Source: Survey of Horizon Europe beneficiaries, conducted in May-July 
2023. Question: ‘In your estimation, what is the percentage share of your Horizon Europe project budget that is spent on 
administrative tasks (e.g., project reporting, project financial management, and similar)’ Number of responses: 684. 

Such costs may be considered by applicants as proportionate to the project funding. Most 
(57%) of the respondents agree or strongly agree that the burden of the administrative and 
legal requirements for granting procedures were proportionate. On the other hand, around a 
fifth (19% in HE overall) of the respondents (and that varies by programme part) disagree or 
strongly  disagree  with  the  statement,  suggesting  that  this  is  one  of  the  areas  where  the 
improvements could be welcomed (see Figure 69). In addition, as discussed in the drivers 
and  barriers  section  in  Effectiveness,  the  burden  of  the  reporting  requirements  can  be 
considered as one of the less positive aspects about the participation in the HE projects.  

259 

 
 
 
 
100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%

24%; 264

15%; 892

43%; 1410

17%; 17

14%; 35

41%; 42

43%; 2577

41%; 102

28%; 29

20%; 98

39%; 140

22%; 19
8%; 63

45%; 185

45%; 374

25%; 12

21%; 38

41%; 44

39%; 191

41%; 73

14%; 804

22%; 54

23%; 189

8%; 38

All respondents

ERC

MSCA (excl. PF)

MSCA PF

7%; 12

INFRA

19%; 14

16%; 47

45%; 56

45%; 134

17%; 50

WIDERA

Strongly agree

Rather agree

Neither agree nor disagree

Rather disagree

Strongly disagree

Figure 69. The perceived proportionality of the administrative and legal requirements for granting procedures. Source: 
Survey of Horizon Europe beneficiaries, conducted in May-July 2023. Question: ‘The burden of the administrative and legal 
requirements for granting procedures was proportionate: Would you agree or disagree with the following statements about 
the administrative and management processes in your Horizon Europe project?’ Number of responses: 5 945. 

In the beneficiary survey we also asked respondents to compare the administrative effort now 
and during their H2020 projects (if they had any). Respondents were asked to compare the 
grant preparation effort, project management and implementation and financial management 
effort. The general observation is that respondents do not report a significant shift in 
the  load  of  the  administrative  burden.  In  all  of  these  three  aspects  around  a  half  of 
respondents  neither agreed  nor disagreed  that  HE  administrative  tasks  were  simpler  than 
H2020.  

When the share of respondents who agree that grant preparation effort is simpler in HE than 
in  H2020  is  compared  with  the  share  of  respondents  who  disagree  we  see  that  a  more 
positive sentiment dominates. Nevertheless, it is important to note that those who answered 
this  question  are  participating  in  EU  framework  programmes  for  at  least  a  second  time, 
hence, at least a part of the positive perception comes from them already having experience 
in  Horizon  projects.  Similar  trends  were  observed  regarding  project  management  and 
implementation as well as financial management effort.  

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%

2%; 66
9%; 257

2%; 2
9%; 8

1%; 3
9%; 22

5%; 5

15%; 14

3%; 4
10%; 12

1%; 2
12%; 21

51%; 1436

54%; 129

71%; 60

55%; 52

56%; 68

44%; 75

31%; 858

7%; 190

13%; 11
5%; 4

29%; 69

7%; 16

All respondents

ERC

MSCA (excl. PF)

21%; 20

3%; 3
MSCA PF

30%; 36

2%; 2
INFRA

34%; 57

8%; 14

WIDERA

Strongly agree

Rather agree

Neither agree nor disagree

Rather disagree

Strongly disagree

Figure 70. Perceived difference of grant preparation effort in HE and H2020. Source: Survey of Horizon Europe 
beneficiaries, conducted in May-July 2023. Question: ‘Grant preparation in Horizon Europe is simpler than in Horizon 2020: 
Based on your overall FP experience, would you agree or disagree with the following statements on the project’s lifecycle 
processes in Horizon Europe, compared to Horizon 2020?’ Number of responses: 2 806. 

260 

 
 
 
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%

2%; 62
10%; 245

1%; 1
7%; 5

2%; 4
8%; 17

7%; 5

9%; 7

6%; 7
11%; 13

2%; 3
17%; 28

53%; 1353

63%; 45

57%; 121

51%; 38

47%; 55

42%; 70

53%; 765

6%; 150

All respondents

63%; 17

4%; 3
ERC

57%; 56

7%; 15

51%; 18

9%; 7

MSCA (excl. PF)

MSCA PF

47%; 37

3%; 4
INFRA

42%; 52

7%; 12

WIDERA

Strongly agree

Rather agree

Neither agree nor disagree

Rather disagree

Strongly disagree

Figure 71. Perceived difference of project management and implementation effort in HE and H2020. Source: Survey of 
Horizon Europe beneficiaries, conducted in May-July 2023. Question: ‘Project management and implementation 
(amendments, reporting etc.) is simpler in Horizon Europe than in H2020: Based on your overall FP experience, would you 
agree or disagree with the following statements on the project’s lifecycle processes in Horizon Europe, compared to Horizon 
2020?’ Number of responses: 2 576. 

100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%

2%; 64
9%; 227

4%; 3
9%; 6

1%; 3
8%; 17

6%; 4

2%; 1

9%; 10
8%; 9

1%; 2
15%; 22

54%; 1388

62%; 43

55%; 114

68%; 45

51%; 60

53%; 77

30%; 762

5%; 142

All respondents

19%; 13

6%; 4

ERC

27%; 56

9%; 19

18%; 12

6%; 4

MSCA (excl. PF)

MSCA PF

31%; 36

2%; 2
INFRA

21%; 31

9%; 13

WIDERA

Strongly agree

Rather agree

Neither agree nor disagree

Rather disagree

Strongly disagree

Figure 72. Perceived difference of financial management effort in HE and H2020. Source: Survey of Horizon Europe 
beneficiaries, conducted in May-July 2023. Question: ‘Financial management (financial rules and reporting) is simpler in 
Horizon Europe than in Horizon 2020: Based on your overall FP experience, would you agree or disagree with the following 
statements on the project’s lifecycle processes in Horizon Europe, compared to Horizon 2020?’ Number of responses: 
2 582. 

1.7.6.  Conclusions and lessons learned 

Conclusion 1: In the first two years, the administrative performance of the evaluation and 
grant agreement preparation processes has not met the targeted time values and also does 
not reach the (very efficient) levels achieved under Horizon 2020 overall. It is worth noting 
that averages in the first few years can be disproportionately affected by a limited number of 
delayed calls. The performance improved between 2021 and 2022 and can still meet the time 
targets with respect to the average performance over the entire programme (see Annex 6).    

Conclusion 2. The attractiveness of Horizon Europe is comparable to that of H2020. A slight 
increase in the success rate between H2020 and HE was recorded (from 11.4% to 15%). 
This  positive  change  is  affected  by  an  increased  budget  for  the  HE  programme  parts, 
suggesting that relatively more excellent proposals are funded under HE. This change  will 
have a positive impact on the costs incurred by applicants as a whole, as fewer high-quality 
applications will fail to result in a productive outcome.     

Conclusion  3.  The  targeted  survey  of  unsuccessful  and  successful  applicants  of  Horizon 
Europe  suggests  that  the  costs  of  preparing  the  proposal  are  overall  not  perceived  to  be 
disproportionate.  However,  applicants  also  indicated  that  there  has  been  no  substantial 

261 

 
 
 
 
change in the costs involved in proposal preparation - nor in the administrative burden during 
project implementation - between H2020 and Horizon Europe. A median project coordinator 
spends around 36 to 45 person-days preparing the proposal, in addition  the median effort 
spent by a consortium partner, who contributes to a proposal, is between 16 to 25 person-
days.  If  the  proposal  is  successful,  the  consortium  also  has  an  obligation  to  perform 
administrative  tasks  related  to  project  reporting,  financial  management  and  similar.  Such 
tasks consume around 6%-10% of the project budget (reported as a median value of survey 
responses). 

Applicants/beneficiaries  do  not  see  much  difference  in  the  administrative  burden  between 
H2020  and  Horizon  Europe,  however,  specific  simplification  measures  introduced  under 
Horizon  Europe  are  seen  as  effective.  The  introduction  of  lump  sums  has  been  received 
positively,  suggesting  the  potential  for  simpler  procedures  (such  as  the  development  of  a 
project workplan, or reporting). Nevertheless, the scheme is very new to the FP and it is still 
too early to draw definitive conclusions. 

Lesson  learned  3.1:  While  the  number  of  lump  sum  recipients  is  small  (and  the  survey 
sample only included just 75 lump sum beneficiaries), the analysis points towards the cost-
saving potential for beneficiaries of lump sums, suggesting that this funding model should be 
continued and further monitored to understand its impact in the costs to beneficiaries and the 
EC.  This  scheme  received  particular  support  from  small-scale  projects,  those  at  a  higher 
TRL, and projects originally funded via the Coordination and Support Action.  

Conclusion 4: Support in preparing Horizon Europe applications is welcomed by applicants. 
The majority of applicants (70%) receive some type of assistance in preparing their Horizon 
Europe  proposals.  In  Pillar  1,  applicants  mostly  rely  on  internal  departments  within  their 
organisations  (50%),  receive  support  from  NCPs  (19%),  involve  external  consultancies 
(17%). However, the involvement of external consultants is particularly relevant across the 
rest of Horizon Europe. Some respondents elaborated in the comment section of the survey 
on  the  reasons  for  involving  external  consultancies,  among  the  most  mentioned  was  that 
respondents  seek  to  increases  their  chances  of  success  in  many  cases it  means  that  the 
external consultancies review the proposals prepared by the applicants. Indeed, only 59% of 
applicants agreed to a ‘very large’, ‘large’ or ‘moderate’ extent that efforts needed to prepare 
an application were in proportion with the chances of securing Horizon Europe funding. 

Lesson learned 4.1: Applicants could benefit more from the help of national contact points 
(NCPs). Around 20% of all HE respondents said they drew on support from NCPs. Among 
Pillar 1 respondents such a share was smaller, around 17%. NCPs were established as part 
of  the  H2020  Framework,  with  the  goal  of  providing  guidance,  practical  information  and 
assistance on all aspects of participation in Horizon Europe. It is worth noting that the FP 
already  allocates  a  specific  budget  for  NCPs  through  the  relevant  National  Authorities. 
Encouraging more participants to make use of these already-funded NCPs could enhance 
resource efficiency. 

262 

 
 
 
Annex 2: Case studies 

Annex 2.1: Contribution of the ERC to emerging areas in the Life 
Sciences, Physical Sciences and Engineering 

1.1. 

Introduction 

The European Research Council (ERC), set up by the European Union in 2007, is the premier 
European  funding  organisation  for  excellent  frontier  research.  The  ERC's  mission  is  to 
encourage  research  of  the  highest  quality  in  Europe  through  competitive  funding,  and  to 
fields,  based  on  scientific 
frontier  research  across  all 
support 
excellence425. Under Horizon 2020, the ERC funded 6,707 projects, with a combined budget 
of EUR 13.3 billion, and evaluated around 54,000 research proposals426.  

investigator-driven 

Under Horizon Europe, the ERC remains an important funding scheme for Pillar 1. With an 
increased  budget  of  EUR  16  billion  for  the  period  2021-2027,  financing  for  the  ERC 
represents around 17 % of the overall Horizon Europe budget. The ERC continues to offer 
four  main  grant  schemes:  Starting  Grants,  Consolidator  Grants,  Advanced  Grants  and 
Synergy Grants, using a ‘bottom-up' approach without predetermined research priorities. 

In relation to the ERC’s mission to fund frontier research, this case study examines research 
in  the  areas  of  life  sciences,  physical  sciences  and  engineering  that  has  emerged  from 
funding provided by the ERC, and which has been able to position Europe at the forefront of 
excellent science. It also investigates the impact of the ERC on the research and careers of 
European researchers, as well as its capacity to attract and retain talent in Europe.  

This case study uses a combination of methods. In particular, an analysis of research trends 
is used to identify prominent scientific fields and high-quality research based on bibliometric 
data  from  Horizon  2020.  Desk  research  and  interviews  with  relevant  beneficiaries  provide 
insights  to  enable  a  better  understanding  of  the  factors  behind  the  ERC's  contribution  to 
funding new research. In addition, EC monitoring data are used to provide complementary 
information about the performance of the ERC. 

Although this study focuses primarily on the mid-term results of Horizon Europe, the findings 
of the present case study are mostly based on quantitative and qualitative information from 
Horizon 2020 projects because bibliometric data and results and impacts of projects are not 
yet available for ongoing ERC projects under Horizon Europe. 

The case study is structured as follows. Section 1.2 provides a brief overview of the policy 
context  of  the  ERC  programme  This  is  followed  by  a  description  of  the  methodological 
approach used in this case study427. Section 1.3 presents the main findings of the case study, 
arranged according to the evaluation criteria of relevance, efficiency, effectiveness, and EU 
added value. Section 1.4 provides insights on key success factors and perceived barriers to 

425 ERC at a glance. https://erc.europa.eu/about-erc/erc-glance.    
426 European Commission, European Research Council Executive Agency, Mapping ERC 
frontier research Horizon 2020 Research and Innovation Framework Programme (2014-2020), 
Publications Office of the European Union, 2023, https://data.europa.eu/doi/10.2828/636104.  
427 This is developed in parallel with the case study report on the contribution of the ERC to 
emerging areas in the social sciences and humanities. 

263 

 
 
the progress and impact of the actions that fall within the scope of this case study. Section 
1.5 presents the conclusions and lessons learned.  

1.2.  Background information 

1.2.1.  Policy context 

Pillar I ‘Excellent Science’ of Horizon Europe aims to promote scientific excellence; attract 
the best talent to Europe; provide appropriate support to early-stage researchers; and support 
the  creation  and  diffusion  of  scientific  excellence,  as  well  as  high-quality  knowledge, 
methodologies and skills, technologies and solutions to address global social, environmental 
and economic challenges. The European Research Council (ERC), as part of this pillar, is 
required to provide attractive and flexible funding to enable talented and creative individual 
researchers (with an emphasis on early-stage researchers) and their teams, to pursue the 
most promising avenues at the frontiers of science, regardless of their nationality and country 
of  origin  and  on  the  basis  of  Union-wide  competition  based  solely  on  the  criterion  of 
excellence. 

To fulfil its mission of supporting new ways of working in the scientific world, raising the profile 
of  frontier  research  in  Europe,  as  well  as  increasing  the  visibility  of  ERC  programmes  to 
researchers  across  Europe  and  internationally,  the  ERC  offers  four  main  grant  schemes: 
Starting Grants, Consolidator Grants, Advanced Grants and Synergy Grants, using a ‘bottom-
up' approach without predetermined priorities. In addition, the ERC awards complementary 
funding for the principal investigators (PIs) of its main grants.  

1.2.2.  Overall approach and scope of the case study 

This case study examines how the ERC, through its bottom-up approach and sole evaluation 
criterion  of  excellence  in  science,  contributes  to  fostering  research  in  the  fields  of  life 
sciences, physical sciences, and engineering. 

The  main  findings  of  the  case  study  are  arranged  according  to  the  evaluation  criteria  of 
relevance, efficiency, effectiveness, and EU added value. 

This case study relies on five main methods:   

•  Bibliometric  analysis  building  on  the  data  and  findings  from  the  Evaluation  study  on 

Excellent Science under Horizon 2020; 

•  Administrative and monitoring data for the Framework Programme, which are used 

to complement the bibliometric analysis; 

•  Desk  research  on  ERC  publications  and  documents,  policy  reports,  deliverables  of 

projects and other sources relating to the ERC's performance over time; 

• 

Interviews with four ERC beneficiaries, providing additional evidence on the relevance, 
effectiveness, and added value of the ERC in funding emerging areas of life sciences, 
physical sciences and engineering; 

•  Survey  data  were  used  to  complement  information  about  the  impact  of  the  ERC  on 

beneficiaries’ careers.  

264 

 
1.3.  Case study findings 

1.3.1.  Relevance 

1.3.1.1. 

Top areas of research in the fields of life sciences, physical sciences, and 
engineering in Europe 

Data from the Science, Research and Innovation Performance report428, published in 2022 
by the European Commission, provide an overall perspective on the production of science 
and scientific productivity in the EU and in the rest of the world. Figures for 2020 (see Figure 
1), based on Scopus data, show that among the STEM429 fields, around one-quarter of all 
scientific  publications  registered  in  the  EU  were  in  the  field  of  clinical  medicine.  This  was 
followed,  by  some  margin,  by  ICT,  physics  and  astronomy,  and  enabling  and  strategic 
technologies (around 8% each). Among the main STEM fields, only the fields of biomedical 
research and enabling and  strategic technologies significantly increased their share in the 
EU between 2010 and 2020. Meanwhile, the fields of physics and astronomy, engineering 
and ICT saw their shares decreasing substantially during the same period.  

Compared  with  other  regions  of  the  world,  in  2020  the  EU  also  led  in  the  field  of  clinical 
medicine,  with  a  22%  share  of  publications  worldwide,  and  a  similar  share  (also  22%)  in 
biomedical research, but lagged behind China in the fields of ICT, physics and astronomy, 
as  well  as  in  enabling  and  strategic  technologies.  In  the  remaining  STEM  fields,  the  EU 
appeared relatively strong in the fields of mathematics and statistics (21%, first place), and 
in agriculture, fisheries and forestry (20%, second to China).  

Figure 1. EU share of publications per scientific field, 2010 and 2020. Source: Science, Research and Innovation 
Performance of the EU, 2022. 

428 European Commission, Directorate-General for Research and Innovation, Science, research 
and innovation performance of the EU 2022 : building a sustainable future in uncertain times, 
Publications Office of the European Union, 2022, https://data.europa.eu/doi/10.2777/78826.  
429 Science, Technology, Engineering and Mathematics. 

265 

 
 
 
Figures  from  the  same  European  Commission  report  on  citations  (measured  as  the 
percentage  of  EU-funded  research  that  ranks  among  the  top  10%  most-cited  scientific 
publications) indicate that in 2018, among the STEM fields, the EU only led in the field of 
biology, with 25% of EC-funded scientific publications ranking among the top 10% most-cited 
scientific publications worldwide. This means that despite the EU producing fewer scientific 
publications in the field of biology, those produced are of top quality. The EU stands out in a 
number  of  fields,  coming  in  second  with  relatively  strong  performances  in  agriculture, 
fisheries and forestry (25%, second to China); clinical medicine (23%, second to the United 
States); and mathematics and statistics (23%, second to China).  

1.3.1.2. 

Areas of research funded by the ERC in the fields of life sciences, physical 
sciences and engineering  

Data from the Horizon Dashboard on Horizon 2020 signed grants (see Table 4 in Appendix 
2) show that in the fields of life sciences, physical sciences and engineering, the majority of 
ERC grants were signed in the field of biology. Knowing that scientific publications stemming 
from ERC projects on average have a higher citation index430, this could be an initial indication 
that the ERC contributes to the production of high-quality scientific research in the field of 
biology in the EU, thus helping to position the EU at the forefront in this field. 

Data on Horizon Europe (see Table 1) points to a continuation of this trend, with most ERC 
signed grants being in the field of biology. Within this area, RNA research has become more 
important under Horizon Europe, presumably as a result of the COVID-19 pandemic. In the 
area of clinical medicine, oncology has remained an important field of research under both 
Horizon 2020 and Horizon Europe. Earth and environmental sciences and ICT appear to be 
increasing in importance, particularly in relation to research into climate change and machine 
learning.  The  increased  importance  of  these  fields  aligns  with  societal  changes  and 
challenges relating to climate change and artificial intelligence. 

Top 10 EuroSciVoc concept 

Horizon Europe ERC signed 
grants 

Scientific field(s)* 

Proteins 

Oncology 

Climatic changes 

Machine learning 

Genomes 

DNA 

Ecosystems 

79 

75 

68 

63 

60 

56 

55 

Biology 

Clinical medicine 

Earth and environmental 
sciences 

ICT 

Biology 

Biology 

Biology 

430 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European framework programmes 
for research and innovation for excellent science : Horizon 2020 : phase 1 final study report, 
Publications Office of the European Union, 2023, https://data.europa.eu/doi/10.2777/967813. 

266 

 
 
Top 10 EuroSciVoc concept 

Horizon Europe ERC signed 
grants 

Scientific field(s)* 

Spectroscopy 

RNA 

Neurobiology 

53 

50 

49 

Physics and astronomy 

Biology 

Biology 

Table 1. Top 10 EuroSciVoc concepts underlying Horizon Europe signed ERC grants Note: *scientific fields are based on 
the EuroSciVoc concepts. These concepts are not mutually exclusive, and one grant can address more than one concept.  
Source: Horizon Dashboard (accessed 6 July 2023). 

1.3.1.3. 

Most prominent topics funded by the ERC in the fields of life sciences, 
physical sciences and engineering  

An analysis of research trends carried out by the study team for Phase 2 of the present study, 
based  on  the  scientific  publications  produced  through  ERC  projects  under  Horizon  2020, 
identified several prominent topics which can be ranked according to their total number of 
publications (see Table 2), or by their impact as measured by the share of ERC-sponsored 
research output that ranks in the top 10% or top 1% most-cited publications worldwide in the 
field  (see  Table  5  and  Table  6  in  Appendix  2)431.  However,  it  should  be  noted  that  these 
figures  do  not  show  the  shares  of  funding  or  of  projects  that  these  figures  represent  (for 
example, the same number of publications could be produced by five projects worth EUR 5 
million, or 100 projects worth EUR 3 million). Nonetheless, the figures provide insights into 
which prominent topics the ERC is funding, and what impact these have had.  

Looking  at  the  ranking  based  on  total  number  of  publications,  the  top  10  prominent 
topics  mostly  focus  on  physics  and  astronomy,  with  the  field  of  engineering  relating  to 
perovskite  solar  cells  coming  top.  Project  characteristics  or  field-specific  differences  in 
publication culture might explain why this field does not correspond to the one in which most 
grants were signed (biology, see Table 4 in Appendix 2). In other words, having more grants 
signed in one particular field does not necessarily translate into a greater number of scientific 
publications. It is therefore relevant to look at publication trends (in addition to grants signed) 
in order to assess the contribution the ERC makes to the European scientific landscape. 

In particular, projects focusing on the use of perovskite in solar energy production and solar 
cells (Perovskite Solar Cells; Solar Cell; Formamidine) were the projects yielding the highest 
number of publications (418) and an equally strong impact (63% of ERC-sponsored research 
output on this topic features in the top 10% publications, and 15% of the top 1%). In addition 
to the analysis of research trends, the study team also looked at individual projects with a 
relatively  high  number  of  publications  and  a  high  average  Field  Weighted  Citation  Impact 
(FWCI)432  –  in  particular,  projects  that  addressed  the  same  prominent  topics  identified. 

431 In further detail, the concept of ‘prominent topics’ refers to important topics in terms of 
momentum and visibility. To calculate a topic’s prominence, the study team combined three 
metrics to arrive at an average constituting its prominence score. These three metrics are 
citation counts, Scopus view counts and Average CiteScore. Please see Annex 6 for a more 
detailed description of the methodology.  
432 Field Weighted Citation Impact in SciVal indicates how the number of citations received by an 
entity’s publications compares with the average number of citations received by all other similar 
publications in the data universe: how do the citations received by this entity’s publications 
compare with the world average? 
https://www.elsevier.com/__data/assets/pdf_file/0020/53327/ELSV-13013-Elsevier-Research-
Metrics-Book-r12-WEB.pdf 

267 

 
 
Accordingly,  in  the  case  of  ‘Perovskite  Solar Cells;  Solar  Cell;  Formamidine’,  examples  of 
very  successful  projects  include  PINNACLE433,  which  produced  10  scientific  publications, 
with an average FWCI 8.3% above the world average; and HYPERION434, which produced 
57 scientific publications, with an average FWCI 7.4% above the world average.   

When looking at the ranking based on citations, the top 10 prominent topics are mostly 
in  the  field  of  biology,  thus  confirming  the  earlier  indications  of  the  EU  occupying  a  top 
position in this field, as well as in the fields of clinical medicine and chemistry. Work in the 
area  of  photocatalysis  (Alkene;  Cross-coupling  Reactions;  Photocatalysis)  shows  an 
impressive 84% of ERC-funded research outputs in this area featured in the top 10% most-
cited papers, while 34% of ERC-sponsored research outputs on electrochemical reduction or 
electrocatalytic conversion of CO2 (Carbon Dioxide; Electrocatalysts; Chemical Reduction) 
ranked  among  the  top  1% of  the  world’s  most-cited  publications  on  that  topic.  In  addition, 
very  impactful  projects  such  as  MicrobioS435  (29  publications,  with  an  average  FWCI  of 
20.6%) in the field of genomes, or GelGeneCircuit436 (23 publications, with an average FWCI 
of  13.8%)  in  clinical  medicine  represent  good  examples  of  the  strong  impact  ERC-funded 
research has achieved in prominent and/or emerging fields.  

Some  fields  specifically  combine  a  large  number  of  publications  with  a  very  strong 
impact  indicator.  These  include  the  field  of  astronomy  (Neutron  Stars;  Mergers; 
Gravitational Waves), with a total of 199 publications and 71% of the research output funded 
by  the  ERC  featuring  in  the  top  10%  and  20%  of  ERC-funded  output  in  the  top  1%.  This 
includes  highly  successful  projects  such  as  BlackHoleMaps437  (with  a  total  of  24  scientific 
publications  and  an  average  FWCI  of  56.3%)  and  imbh438  (with  42  publications  and  an 
average FWCI of 10%). 

Lastly, it is important to note that although the ERC follows a bottom-up approach to funding, 
the cutting-edge research pursued by ERC grantees generates results that address a 
wide  range  of  emerging  issues  that  have  significant  societal,  economic  and  policy 
relevance,  such  as  the  COVID-19  pandemic.  In  a  report439  published  in  2020,  the  ERC 
identified 183 projects that contributed to tackling the COVID-19 crisis, in fields ranging from 
immunity,  infection  and  pathology  to  medical  devices,  digital  tools,  AI,  diagnostics  and 
treatments.  One  such  example  is  the  project  ReservoirDOCs440,  which  has  yielded  34 
publications, with an average FWCI of 11.4%.  

Number of 
publications 

Emerging fields 

Field of 
science 

Top 
1% 
share 

Top 
10% 
share 

418 

Perovskite Solar Cells; Solar Cell; 
Formamidine 

Engineering 

15.3% 

62.9% 

401 

Exoplanets; Kepler; Atmosphere 

Physics and 
astronomy 

7.5% 

48.4% 

433 https://cordis.europa.eu/project/id/759744.  
434 https://cordis.europa.eu/project/id/756962.  
435 https://cordis.europa.eu/project/id/669830.  
436 https://cordis.europa.eu/project/id/848325.  
437 https://cordis.europa.eu/project/id/647839.  
438 https://cordis.europa.eu/project/id/647208. 
439 European Research Council, 2020, COVID-19 Frontier research in the spotlight. Available at: 
https://erc.europa.eu/sites/default/files/2022-08/COVID19-Frontier_research_in_the-
spotlight.pdf.  
440 https://cordis.europa.eu/project/id/725422. 

268 

 
 
Number of 
publications 

Emerging fields 

Field of 
science 

Top 
1% 
share 

Top 
10% 
share 

Molybdenum Disulfide; Monolayer; Van 
Der Waals 

Chemistry 

8.6% 

49.6% 

Quantum Optics; Thermalization (Energy 
Absorption); Eigenvalues and 
Eigenfunctions 

Physics and 
astronomy 

4.8% 

49.8% 

Planet; T Tauri Star; Pre-main Sequence 
Stars 

Physics and 
astronomy 

5.7% 

36.2% 

Star Formation; Galaxies; Molecular 
Gases 

Physics and 
astronomy 

4.4% 

51.0% 

Neutron Stars; Mergers; Gravitational 
Waves 

Physics and 
astronomy 

19.6% 

71.4% 

Weyl; Metalloids; Paul Adrien Maurice 
Dirac 

Physics and 
astronomy 

12.6% 

50.8% 

347 

251 

229 

206 

199 

199 

195 

Topological Insulators; Topology; 
Quantum Hall Effect 

192 

Origanum; Superconductivity; Topology 

Physics and 
astronomy 

Physics and 
astronomy 

12.3% 

52.8% 

6.3% 

35.4% 

Table 2. Top 10 prominent topics from Horizon 2020 ERC-funded research, by total number of publications. Source: 
compiled by the study team, based on bibliometric analysis. 

While the results above focus on the Top 200 most prominent topics that have produced the 
greatest number of publications, the question arises as to how much EC funding these topics 
received, and hence how ‘productive’ the creation of knowledge was in these specific areas. 
An  analysis conducted in  the  context  of  this  study  shows  that  those  prominent  topics  that 
have led to the greatest number of publications in the programme parts are not necessarily 
the  ones  that  are  most  productive  (i.e.,  those  with  the  highest  number  of  publications  per 
million EUR of EC funding)441. More details on this analysis are provided in Annex 6 of this 
study. 

1.3.1.4. 

Comparison of the contribution made by Horizon 2020 to the most 
prominent topics in a global context 

Based on publications from the EU as a whole and from the main international knowledge 
producers (including a total of 32 countries) 442, the study team identified the 200 topics with 

441 This funding analysis shows per programme part, the Top 200 most prominent topics that 
produced the largest number of publications, along with the topic’s total EC funding, and the 
number of publications published per million EUR of total EC funding. More information and 
results are found in Annex 6 ‘Quantitative research track findings’ and Annex 2 ‘Case Study 9 on 
Creating high-quality new knowledge’. 
442 The detailed methodology is described in Annex 6. 

269 

 
 
the highest prominence score443. The full set of the top 200 topics can be retrieved online444. 
From  this  list  of  200  prominent  topics,  it  was  possible  to  identify  those  that  also  featured 
among  the  top  10  identified  above,  and  to  make  a  comparison  between  the  contributions 
made by the ERC and Horizon 2020 (as a whole) with those of the EU-27 as a whole and 
with  selected  third  countries,  according  to  each  prominent  topic  identified.  Table  3  below 
shows the eight prominent topics that correspond to the top topics identified for the ERC in 
the analysis above.  

The data show that the ERC represented 43% of the total number of Horizon 2020-funded 
scientific publications on the topic of ‘Perovskite Solar Cells; Solar Cell; Formamidine’ (the 
most prominent topic in terms of scientific publications (see Table 2)). Compared with certain 
other  topics,  this  is  a  relatively  low  share,  which  means  that  other  programme  parts  also 
contributed strongly to funding scientific publications on this topic. Horizon 2020 as a whole, 
however, contributed to only 9% of the EU-27 scientific publications on the topic. Moreover, 
the EU-27 overall contributed 31% of all publications – behind China (40%), but ahead of the 
United States (18%).  

Among  the  eight  selected  topics,  ERC-funded  research  represents  the  highest  share  of 
Horizon  2020-based  publications  in  two  specific  topics  in  the  field  of  astronomy:  ‘Star 
Formation; Galaxies; Galactic Evolution’ (86%) and ‘Exoplanets; Kepler; Atmosphere’ (82%). 
Also, in  the  topics  of ‘Planet;  T  Tauri  Star; Pre-main  Sequence  Stars’  and  ‘Neutron Stars; 
Mergers; Gravitational Waves’, the shares are 72% and 70%, respectively. This may indicate 
that these specific astronomy topics are mainly covered by ERC-funded research, but less 
so by other parts of the Framework Programme. Remarkably, in the first two topics mentioned 
above,  the  shares  of  Horizon  2020-funded  publications  as  a  percentage  of  total  EU-27 
publications are the lowest of the eight topics selected (5% and 7%). EU-27 publications in 
turn represent 52% of the total number of publications for these two topics: 51% for the field 
‘Neutron Stars; Mergers; Gravitational Waves’, and 56% for the field ‘Planet; T Tauri Star; 
Pre-main Sequence Stars’ – each of these far surpassing the shares of the other countries 
in the analysis. 

In addition, among the eight topics selected, the share of Horizon 2020-funded publications 
out of the total number of EU-27 publications is the highest for the topics ‘Weyl; Metalloids; 
Paul  Adrien  Maurice  Dirac’,  ‘Molybdenum  Disulfide;  Monolayer;  Van  Der  Waals’  and 
‘Topological Insulators; Topology; Quantum Hall Effect’: between 11% and 14% of all EU-27 
publications  relate  to  Horizon  2020-funded  research.  However,  the  EU-27  displays  low 
shares of between 29% and 37% in the overall scientific production in these fields, while the 
ERC has contributed with between 39% and 77%.  

Overall, the data confirm the ERC's pivotal role in contributing to positioning the EU at the 
forefront of the world’s scientific research in specific scientific fields. 

443 The Top 5 of topics with the highest prominence score are: 1) Perovskite Solar Cells; Solar 
Cell; Formamidine; 2) Nivolumab; Pembrolizumab; Immunotherapy; 3) Molybdenum Disulfide; 
Monolayer; Van Der Waals; 4) Oxygen Production; Electrocatalysts; Catalyst; 5) Cyanogen; 
Graphitic Carbon Nitride; Photocatalysts.  
444 https://doi.org/10.34894/DNWIXV. 

270 

 
 
Prominent topics 

Perovskite Solar Cells; 
Solar Cell; Formamidine 

Exoplanets; Kepler; 
Atmosphere 

Molybdenum Disulfide; 
Monolayer; Van Der 
Waals 

Planet; T Tauri Star; 
Pre-main Sequence 
Stars 

Star Formation; 
Galaxies; Galactic 
Evolution 

Neutron Stars; Mergers; 
Gravitational Waves 

Weyl; Metalloids; Paul 
Adrien Maurice Dirac 

Topological Insulators; 
Topology; Quantum Hall 
Effect 

ERC as a 
share of 
Horizon 2020-
funded 
publications 

Horizon 2020 
as a share of 
EU-27 
publications 

EU-27 as a 
share of all 
publications 

China as a 
share of all 
publications 

Japan as a 
share of all 
publications 

United 
Kingdom as a 
share of all 
publications 

United States 
as a share of 
all publications 

Total number 
of publications 
across all 
selected 
countries* 

8 

82% 

9% 

7% 

31% 

40% 

52% 

2% 

39% 

11% 

32% 

33% 

6% 

3% 

7% 

6% 

18% 

35,454 

14% 

29% 

12,739 

4% 

24% 

25,473 

72% 

9% 

56% 

3% 

4% 

12% 

25% 

6,573 

86% 

70% 

5% 

9% 

52% 

51% 

5% 

7% 

77% 

14% 

29% 

30% 

62% 

13% 

37% 

24% 

4% 

7% 

9% 

6% 

16% 

23% 

8,881 

9% 

4% 

6% 

26% 

6,357 

28% 

6,709 

27% 

6,542 

Table 3. Prominent topics under the ERC and Horizon 2020, compared with the EU-27 and selected third countries. Note: *number of publications as identified by the study team in the bibliometric analysis 
for the selected countries. Source: compiled by the study team, based on the bibliometric analysis.

271 

 
 
1.3.1.5. 

Impact of the ERC on researchers’ careers 

Overall, the ERC has very strong impacts on the careers of both principal investigators (PIs) 
and research teams as a whole.  

In the case of the PI, one of the main motivating factors for applying for an ERC grant is the 
positive  impact  it  can  have  on  the  researcher’s  career  –  especially  among  early-stage 
researchers applying for ERC Starting Grants. However, the very low success rate and the 
effort,  time  and  resources  required  to  prepare  an  excellent  proposal  may  act  as  a 
discouraging factor for early-stage researchers. In interviews, successful applicants indicated 
that an ERC Starting  Grant  had greatly increased their  chances of obtaining a permanent 
position  at  a  top  research  institution.  Some  benefits  mentioned  during  interviews  involved 
mobility from one country to another, as universities tend to compete to attract ERC grant 
holders. Others included an increase in the researcher’s independence and visibility within 
or even outside their research field. With respect to the other main ERC grants, the fact that 
these  grants  are  considered  highly  prestigious  and  a  strong  indicator  of  the  quality  of  the 
proposed research also has a positive impact on the reputation of the researcher. 

Survey data on successful ERC PIs under Horizon Europe shows that 71% (171 out of 241 
respondents, ERC-BEN-025) stated that the ERC project gave them, to a large or very large 
extent,  the  opportunity  to  gain  a  research  position  with  better  career  prospects.  The 
percentage is even higher (91% or 218 out of 240 respondents) for those respondents who 
claimed it increased the international visibility and outreach of their research.  

With regard to the PI’s research team, interviews indicate that members of the team benefit 
from the experience brought by the PI, the stability offered by a lengthy ERC-funded project 
(in comparison to other funding opportunities), and the prestige associated with being part of 
an  ERC-funded  project  –  especially  for  those  researchers  just  embarking  their  research 
careers.  In  addition,  interviewees  mentioned  that  in  comparison  to  other  funding 
opportunities, the size of the grants provided by the ERC allows larger research teams, and 
thus  offers  the  flexibility  to  hire  researchers  from  different  (sub-)disciplines.  This  further 
enriches 
involved.  Several 
interviewees  mentioned  that  after  the  project,  most  members  of  their  research  team  had 
continued to pursue a career in academia in the same field of research, and in the longer run 
several had taken up leading research positions. In the specific case of ERC Synergy Grants, 
younger researchers from different teams can benefit from greater collaboration, which was 
mentioned  as  also  increasing  their  chances  of  publishing  a  greater  number  of  scientific 
articles.  

the  early-career  stage  researchers 

the  experience 

for 

1.3.2.  Efficiency 

1.3.2.1. 

Attracting talent into the fields of life sciences, physical sciences and 
engineering  

The  ERC  has  a  strong  capacity  for  retaining  and  attracting  top  talent  in  Europe.  As 
mentioned by several interviewees, ERC grants are a very prestigious label in researchers’ 
careers,  and  receiving  a  grant  leads  almost  immediately  to  a  permanent  position  or 
promotion.  Therefore,  despite  the  perception  of low  success  rates,  the  prestige  of  holding 
such an ERC grant and the very positive impact it has on researchers’ careers are the main 
reasons for applying.  

Other elements that attract researchers include the  size and duration of the grants, and 
the relative flexibility to adapt the research to changes in context (e.g. during the COVID-19 
pandemic, travel budgets could be used differently to support the project) or findings (e.g. 
using findings in a different way from that originally envisaged in order to support new aspects 
in the research field). 

272 

 
The fact that ERC grants attract high-quality researchers and support excellent research is 
further  illustrated  by  the  fact  that,  a  total  of  13  ERC  grantees  have  so  far  received  Nobel 
Prizes445 in the fields of life sciences, physical sciences and engineering – mainly in the field 
of physics. Among the fields in which the Nobel laureates have been recognised, some match 
the topics identified above, such as quantum physics or catalysts. While account needs to be 
taken of the fact that the ERC was only established in 2007, it is interesting to note that the 
US  National  Science  Foundation  (NSF)  has  supported  258  Nobel  laureates446  since  its 
establishment in 1950. 

1.3.3.  Effectiveness 

The same elements that make ERC grants attractive to (top) researchers are also mentioned 
in relation to the effectiveness of ERC grants and the extent to which they are able to generate 
results and impacts. For instance, the PIs interviewed stressed that the size and duration of 
their grants enabled them to set up a relevant research project with a sufficiently large 
team, consequently allowing them to combine complementary expertise into the team. The 
reputation of ERC grants has also enabled them to attract and hire top-quality early career 
stage researchers to the team. These factors thus facilitate the generation of high-quality 
and novel knowledge in Europe. 

In  terms  of  support  for  emerging  fields  of  research,  an  important  factor  mentioned  by 
interviewees concerns the ERC’s ‘bottom-up’ approach and its emphasis on excellence as 
the  sole  criterion  for  ERC  grants.  This  means  that  highly  novel  and  excellent  research 
ideas  have  access  to  substantial  funding.  The  ERC  thus  contributes  to  the  very  early 
development of knowledge in new areas, and to being at the forefront of the development of 
new research fields. When such areas later emerge as important and growing fields (whether 
due to internal or external factors), a primary knowledge base is available to provide insights 
(both for the research community, as well as policy-makers, for example) and to be further 
developed. 

Bottom-up, curiosity-driven research funded by the ERC has also been able to produce 
breakthrough  innovations  and  technologies  that  have  had  a  significant  impact  on 
economic and social welfare, especially in the fields of life sciences, physical sciences and 
engineering.  This  impact  can  be  seen  from  the  fact  that  25  out  of  42  recipients  of  the 
European Innovation Council’s new ‘Transition’ funding have emerged from research funded 
by the ERC447. Another indication of this is that an analysis carried out by the ERC on projects 
carried out under FP7 and during the first years of Horizon 2020 (2014 to 2016) finds that 
more than 40% of ERC-funded projects generated research that was subsequently cited in 
patent applications448.  

1.3.4.  EU added value 

The evidence collected for the elaboration of this case study indicates that the main areas of 
EU added value provided by the ERC under the Framework Programmes, and particularly 
under Horizon Europe, are as follows:  

• 

Investing in discovery-driven research at EU level is considered essential in order to 
position Europe at the forefront of excellent science. The analysis and examples provided 
in this case study demonstrated, for example, the importance of supporting prominent 
fields at an early stage in their development – not only in order to be at the forefront of 

445 https://erc.europa.eu/news-events/events/follow-nobel-week-3-erc-grantees-awarded-2022-
nobel-prizes.  
446 https://www.nsf.gov/news/special_reports/nobelprizes/. 
447 https://erc.europa.eu/news/erc-funded-research-wins-most-new-eu-innovation-grants. 
448 https://erc.europa.eu/sites/default/files/2023-01/Assessing_the_Influence_ERC-
funded_Research_Patented_Inventions.pdf 

273 

 
 
scientific research in the world, but also to create relevant and high-quality knowledge 
and insights by the time the topic gains societal interest. Interviewees also stressed the 
importance  of  the  sole  selection  criterion  of  excellence  for  truly  bottom-up  research: 
‘high-risk/high-gain’  research  needs  to  allow  for both  unexpected  outcomes  or failure. 
The  ERC  is  considered  the  main  funding  instrument  in  Europe  for  funding  bottom-up 
fundamental research, and its grants are considered by the research community to be 
both  more  substantial  and  more  prestigious  than  national  funding  opportunities.  ERC 
grants  offer  researchers  the  opportunity  to  receive  funding  for  high-risk/high-gain 
research  projects  of  sufficient  scale  to  be  relevant  and/or  to  be  able  to  define  more 
complex research projects, combine complementary research steps or expertise into a 
single  one  project,  and  to  generate  impacts  in  research  and  beyond  without  the 
prerequisite to define these impacts in detail ex-ante. As an alternative, national funding 
opportunities  generally  do  not  offer  this  same  degree  of  excellence-driven  research 
funding; nor do they offer the same scale of funding within a single grant. Interviewees 
point to these elements as important factors in the added value of the EU-level approach. 

• 

Improving  researchers’  careers  prospects  and  specialisation,  especially  during 
early career stages. The prestige associated with an ERC grant was mentioned by all 
interviewees as having had a positive impact on their careers in terms of reputation and 
visibility.  As  one  interviewee  mentioned,  the  ERC  is  uniformly  recognised  across  the 
whole  of  Europe  for  its  excellence  and  highly  quality-driven  selection  procedure.  In 
addition, interviewees mentioned that working on an ERC application required them to 
reflect on the longer-term direction of their research in relation to the broader research 
field.  Such  reflection  helps  them  to  identify  those  pathways  with  a  high  degree  of 
expected  relevance  and  impact,  in  order  to  define  their  individual  future  research 
direction.  

•  The  funding  provided  by  the  ERC  enables  projects  with  a  longer  duration  and 
larger teams than other types of  funding.  As explained above, these two elements 
are very important for researchers to engage in ground-breaking and emerging research, 
enabling new discoveries but also allowing for failure and the redirection of elements of 
the  research,  and  providing  the  opportunity  to  develop  a  research  team  with  the 
necessary (complementary) expertise to achieve the project objectives (thus contributing 
to multidisciplinarity in the project’s approach).   

•  ERC Synergy Grants enable a well-appreciated mix of bottom-up and collaborative 
research. Synergy Grants are unique enablers for groups of PIs to come together and 
work on ambitious subjects. These grants, albeit to a limited extent, respond to the lack 
of  funding  instruments  allowing  bottom-up  collaborative  research  referred  to  by  some 
position papers submitted to the OPC449. 

1.4.  Success factors and barriers 

The following success factors have been identified:  

•  The ERC is unique in funding bottom-up and ‘blue sky’ research, which makes it 
highly appealing to researchers aiming to carry out new and groundbreaking research. 
This impression was shared both by the interviewees and  by the position papers from 
the OPC450. In addition, the design of ERC grants does not penalise research outcomes 
that  are  different  from  those  envisaged  beforehand,  thus  allowing  the  discovery  of 

449 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 
450 Ibid. 

274 

 
 
different outcomes or pathways from those originally intended. This is a necessary factor 
in encouraging truly novel research orientations. 

•  Related  to  the  above  point,  the  curiosity-driven  design  of  ERC  grants  thus  allows 
changes  to  be  accommodated  quickly  during  the  course  of  a  project,  thus  enabling 
opportunities  for  new  research  in  rapidly  evolving  fields.  As  an  illustration  of  the 
importance of this kind of flexibility, reference can be made to a project in which one work 
package out of three did not deliver in terms of publication objectives, but instead ended 
up  supporting  the  other  parts  by  informing  research  in  the  other  two  work  packages 
through high-quality data. In addition, ERC projects are not bound to organise their work 
in  the  form  of  work  packages,  thus  allowing  flexibility  regarding  research  design  and 
allowing PIs to integrate emerging aspects that were not foreseen at the beginning of the 
project. 

•  The  composition  of  teams  and  lengths  of  projects  were  also  identified  as  two 
important factors contributing to the success of the ERC in funding groundbreaking and 
emerging science. These factors were highlighted by several interviewees, who made 
reference to the relatively high degree of freedom provided by the ERC in building diverse 
and sufficiently large research teams, thus contributing to increased multidisciplinarity.  

•  The  evaluation  process  and  governance  are  also  important  success  factors 
contributing  to  the  excellence  of  the  research  funded  by  the  ERC.  Elements  that  are 
considered important in this regard are the application of scientific excellence as the sole 
selection  criterion,  as  well  as  the  use  of  a  panel-based,  peer-review  evaluation..  The 
independence of the ERC’s Scientific Council as its main governing body is considered 
essential  to  this.  These  aspects  were  mentioned  in  some  position  papers  from  the 
OPC451.  

To date, the main barriers and challenges identified include: 

•  Relatively low success rates for ERC grants continue to be mentioned as a barrier to 
applying  to  the  ERC.  As  such,  they  imply  that  a  significant  proportion  of  excellent 
research ideas may not be receiving funding. Some interviewees indicate concerns that 
this  barrier  represents  an  even  greater  risk  in  the  case  of  more  novel  and  disruptive 
ideas.  A  few  position  papers  have  therefore called  for an  increase  in  the  total  budget 
allocated to the ERC in order to fund a greater number of projects, and consequently to 
retain and fund a higher number of European researchers. Researchers who took part 
in the interview programme also pointed to this barrier, but as successful applicants, at 
the same time mentioned that the added value of an ERC grant strongly outweighed the 
costs  related  to  the  development  of  an  application  combined  with  the  relatively  low 
success rates. 

•  Related  to  the  above  point,  the  structure  of  the  ERC’s  evaluation  panels  may  be 
another  factor  hindering  the  realisation  of  its  full  potential  to  fund  highly  novel  and 
emerging  research.  A  few  interviewees  mentioned  that  when  applying  for  funding  for 
multidisciplinary or highly novel research ideas, there is not always a natural fit with one 
of the available evaluation panels. As a result, funding for projects in novel or emerging 
fields can prove more difficult to obtain. This challenge relates to the ERC’s otherwise 
well-appreciated  expert  evaluation  approach,  and  interviewees  mentioned  that  further 
guidance from the experts in the evaluation panels could help to improve the assessment 
of multidisciplinary or highly novel research proposals. 

451 Ibid. 

275 

 
 
1.5.  Conclusions  

In the fields of life sciences, physical sciences and engineering, ERC-funded research has 
contributed the most in terms of the quantity of outputs (in this case, scientific publications) 
in the fields of physics, astronomy and engineering (in particular, perovskite solar cells). 
In terms of impact (measured by scientific publication citations), its greatest contribution has 
been in the fields of biology, clinical medicine and chemistry. The field of astronomy –in 
particular,  the  topic  of  ‘Neutron  Stars;  Mergers;  Gravitational  Waves’  –  combines  a  large 
number of publications with a very strong impact indicator. Data on Horizon Europe points to 
a continuation of this trend, with most ERC grants being signed in the field of biology. Within 
this area, RNA research has become more important under Horizon Europe, presumably as 
a result of the COVID-19 pandemic. In the area of clinical medicine, oncology has remained 
an  important  field  of  research  under  both  Horizon  2020  and  Horizon  Europe.  Earth  and 
environmental sciences and ICT appear to be increasing in importance, particularly in relation 
to research into climate change and machine learning. The increased importance of these 
fields  aligns  with  societal  changes  and  challenges  relating  to  climate  change  and  artificial 
intelligence. 

In  an  international context,  the  research  trends  analysis452  on  eight  selected  topics  shows 
variations in the EU position compared with other parts of the world. For example, the 
ERC’s  contribution  on  the  topic  of  perovskite  solar  cells  is  mentioned  as  important  in 
European context, yet China appears to have a larger share than Europe (as a whole) of the 
global research output on this topic. On the other hand, EU-27 publications represent 52% of 
the total number of publications on the astronomy topic ‘Star Formation; Galaxies; Galactic 
Evolution’ and ‘Exoplanets; Kepler; Atmosphere’, 51% of all publications on the topic ‘Neutron 
Stars; Mergers; Gravitational Waves’, and 56% on the field of ‘Planet; T Tauri Star; Pre-main 
Sequence  Stars’  –  each  of  these  far  surpassing  the  shares  of  the  other  countries  in  the 
analysis.  In  all  four  topics,  ERC-funded  research  represents  large  shares  of  the  overall 
number  of  publications  produced  under  Horizon  2020,  indicating  that  these  specific 
astronomy topics are mainly covered by ERC-funded research, and less so by other parts of 
the Framework Programme. Research from the United States, meanwhile, is fairly balanced 
across all eight topics, with shares ranging between 18% and 29%. 

With  its  curiosity-driven  and  excellence-based  grants,  the  ERC  is  thus  found  to 
contribute  to  the  development  of  new  fields  of  scientific  research  in  Europe.  These 
fields relate to some extent to societal and strategic evolutions, such as the development of 
increasingly  resource-  and  climate-friendly  technologies  (e.g.  solar  cells  based  on 
Perovskite) or COVID-19 related research topics.  

The ERC’s strictly bottom-up approach and the size of its grants are key aspects that 
make the ERC a successful instrument for supporting excellent and emerging research. The 
main  elements  that  result  from  its  design,  and  which  impact  the  extent  to  which  research 
fields are supported by the ERC, are:  

1.  The  relatively  long  duration  of  projects,  which  allows  the  development  of  novel 

approaches and provides stability for research teams;  

2.  The  relatively  substantial  size  of  the  research  teams  funded,  which  allows  for the 
hiring  of  complementary  expertise  and,  where  relevant,  multidisciplinarity  in  the 
team;  

3.  The ERC’s strictly curiosity-driven approach, and the lack of a strong requirement 
for the  expected  impacts  of the  research  to  be  defined  ex-ante,  allow  flexibility  in 

452 For more information, see Annex 6. 

276 

 
 
finding unexpected outcomes and – within the boundaries of the project – to reorient 
towards more promising research paths and outcomes.  

The above elements, combined with the positive reputation associated with the ERC grants, 
make  ERC  grants  attractive  to  top  researchers  both  within  and  outside  Europe  and, 
according to interviews with PIs, allow ERC grantees to attract and hire high-quality young 
researchers to their teams, thus further positively impacting the quality of the research.  

However, further analysis conducted  in this study examined whether receiving Framework 
Programme  funding  made  a  difference  to  the  novelty  and  disruptiveness  of  the  research 
publications that researchers produce later on in their careers453. The results of this analysis 
do not provide significant evidence that those researchers who received ERC funding under 
Horizon 2020 generated more novel or disruptive research output two or five years after the 
application, compared with their counterparts who also applied but did not receive funding.  

Although the mapping of topics supported through the ERC reveals certain differences 
in  terms  of  outcomes  in  the  fields  of  life  sciences,  physical  sciences  and  engineering 
compared  with  social  sciences  and  humanities,  this  case  study  does  not  identify  any 
meaningful  differences  in  terms  of  process  or  factors  with  regard  to  the  way  the  ERC 
supports emerging fields, between each of these wide domains of research.  

Appendix 1. Literature sources 

•  Commission Implementing Decision adopting the 2023 Work Programme, in the 

framework of the Specific Programme Implementing Horizon Europe – The Framework 
Programme for Research and Innovation (2021-2027) in relation to the component 
‘European Research Council' under Pillar I ‘Excellent Science’ – C/2022/4861 final. 
https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=PI_COM:C(2022)4861. 

•  Cordis webpages of selected projects. 

•  DATAVERSE, Replication Data for the Comparison of Nationally Funded Outputs – 

EU27 and selected third countries, https://doi.org/10.34894/DNWIXV. 

•  ERC (2022). ERC Grantees Awarded Nobel Prizes. Available at: 

https://erc.europa.eu/news-events/events/follow-nobel-week-3-erc-grantees-awarded-
2022-nobel-prizes.  

•  ERC (2022). Press Release. Available at: https://erc.europa.eu/news-

events/news/2022-Nobel-physics-prize. 

•  ERC at a glance. Available at: https://erc.europa.eu/about-erc/erc-glance. 

•  ERC (2022). Assessing the Influence of ERC-funded Research on Patented Inventions, 

https://erc.europa.eu/sites/default/files/2023-01/Assessing_the_Influence_ERC-
funded_Research_Patented_Inventions.pdf. 

•  ERC (2023). How Frontier Research Spurs Patented Inventions. Available at: 

https://erc.europa.eu/news-events/news/new-study-reveals-how-frontier-research-
spurs-patented-inventions. 

•  ERC (2022). ERC Funded Research Wins New EU Innovation Grants. Available at: 

https://erc.europa.eu/news/erc-funded-research-wins-most-new-eu-innovation-grants. 

•  European Commission, Directorate-General for Research and Innovation, Science, 

research and innovation performance of the EU 2022 – Building a sustainable future in 

453 More information on these indicators and the analysis can be found in the Novelty and 
Disruptiveness analysis, Annex 6. 

277 

 
 
uncertain times, Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/78826. 

•  European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al. , Evaluation study of the European framework 
programmes for research and innovation for excellent science: Horizon 2020: phase 1 
final study report, Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/967813. 

•  European Commission, European Research Council Executive Agency, Mapping ERC 
frontier research Horizon 2020 Research and Innovation Framework Programme 
(2014-2020), Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2828/636104. 

•  European Parliament and Council (2021). Regulation (EU) 2021/695 of the European 
Parliament and of the Council of 28 April 2021 establishing Horizon Europe – the 
Framework Programme for Research and Innovation, laying down its rules for 
participation and dissemination, and repealing Regulations (EU) No 1290/2013 and 
(EU) No 1291/2013, 28 April 2021, PE/12/2021/INIT, OJ L 170, 12.5.2021, pp. 1-68. 
Available at: http://data.europa.eu/eli/reg/2021/695/oj.   

•  European Research Council (2020). COVID-19 Frontier research in the spotlight. 

Available at: https://erc.europa.eu/sites/default/files/2022-08/COVID19-
Frontier_research_in_the-spotlight.pdf.  

•  European Science Vocabulary. Available at: https://op.europa.eu/en/web/eu-

vocabularies/dataset/-
/resource?uri=http://publications.europa.eu/resource/dataset/euroscivoc. 

•  Horizon Dashboard (accessed 6 July 2023). 

•  Nobel prize winners who have received NSF funding. 

https://www.nsf.gov/news/special_reports/nobelprizes/. 

Appendix 2. Relevance evaluation criteria 

Areas of research funded by the ERC in the fields of life sciences, physical sciences 
and engineering  

Top 10 EuroSciVoc concept 

Horizon 2020 ERC signed 
grants 

Scientific field(s)* 

Proteins 

Oncology 

DNA 

Genomes 

Neurobiology 

Microscopy 

583 

429 

390 

378 

316 

274 

278 

Biology 

Clinical medicine 

Biology 

Biology 

Biology 

Physics and astronomy 

 
 
Top 10 EuroSciVoc concept 

Horizon 2020 ERC signed 
grants 

Scientific field(s)* 

Geometry 

Software 

Stem cells 

Spectroscopy 

259 

253 

250 

246 

Mathematics and statistics 

ICT 

Biomedical research 

Physics and astronomy 

Table 4. Top 10 EuroSciVoc concepts454 underlying Horizon 2020 signed ERC grants. Note: *scientific fields are based on 
the EuroSciVoc concepts. These concepts are not mutually exclusive, and one grant can address more than one concept.  
Source: Horizon Dashboard (accessed 6 July 2023). 

Most  prominent  topics  funded  by  the  ERC  in  the  fields  of  life  sciences,  physical 
sciences and engineering  

Top 1% 
share 

Emerging fields 

Field of 
science 

Number of 
publications 

Top 10% 
share 

34.3% 

Carbon Dioxide; Electrocatalysts; 
Chemical Reduction 

Chemistry 

105 

77.1% 

33.3% 

Glycolysis; Metabolic; Mammalian 
Target of Rapamycin Complex  

Biology 

32.8% 

32.0% 

Nivolumab; Pembrolizumab; 
Immunotherapy 

Clinical 
medicine 

Nasopharyngeal Swabs; Serologic 
Tests; COVID-19 

Clinical 
medicine 

30.4% 

Intrinsically Disordered Proteins; RNA-
binding Protein FUS; Liquid 

Biology 

60 

67 

50 

46 

73.3% 

62.7% 

72.0% 

76.1% 

28.2% 

Single-Cell Analysis; Small Cytoplasmic 
RNA; Single Cell RNA Seq 

Biology 

149 

65.8% 

26.7% 

Plastics; Marine Debris; Litter 

Engineeri
ng 

26.0% 

Genome; Neanderthals; Anatomically 
Modern Humans 

Biology 

75 

50 

81.3% 

70.0% 

454 European Science Vocabulary (EuroSciVoc) is the taxonomy of fields of science based on 
the OECD's 2015 Frascati Manual taxonomy. It was extended to include fields of science 
categories extracted from CORDIS content through a semi-automatic process developed using 
Natural Language Processing (NLP) techniques (https://op.europa.eu/en/web/eu-
vocabularies/dataset/-/resource?uri=http://publications.europa.eu/resource/dataset/euroscivoc). 

279 

 
 
Top 1% 
share 

Emerging fields 

Field of 
science 

Number of 
publications 

Top 10% 
share 

25.9% 

Inflammasomes; Pyroptosis; Activation 

Biology 

54 

81.5% 

25.8% 

Intestine Flora; Ruminococcaceae; 
Microorganisms 

Biology 

151 

69.5% 

Table 5. Top 10 prominent topics from Horizon 2020 ERC-funded research, by share of ERC-sponsored research outputs in 
the top 1% most-cited scientific publications. Source: compiled by the study team, based on bibliometric analysis. 

Top 10% 
Share 

Emerging fields 

Field of 
Science 

Number of 
publications 

Top 1% 
Share 

83.8% 

Alkene; Cross-coupling Reactions; 
Photocatalysis 

Chemistry 

111 

21.6% 

81.5% 

Inflammasomes; Pyroptosis; Activation 

Biology 

81.3% 

Plastics; Marine Debris; Litter 

Engineering 

54 

75 

25.9% 

26.7% 

77.1% 

Carbon Dioxide; Electrocatalysts; 
Chemical Reduction 

Chemistry 

105 

34.3% 

76.1% 

Intrinsically Disordered Proteins; RNA-
binding Protein FUS; Liquid 

Glycolysis; Metabolic; Mammalian 
Target of Rapamycin Complex 1 

Biology 

Biology 

Nasopharyngeal Swabs; Serologic 
Tests; COVID-19 

Clinical 
medicine 

73.3% 

72.0% 

71.4% 

Ligands; Cages; Self-Assembly 

Chemistry 

46 

60 

50 

70 

30.4% 

33.3% 

32.0% 

10.0% 

71.4% 

Neutron Stars; Mergers; Gravitational 
Waves 

Physics and 
astronomy 

199 

19.6% 

71.2% 

Bioprinting; Three-Dimensional 
Printing; Tissue Engineering 

Biomedical 
research 

59 

18.6% 

Table 6. Top 10 prominent topics from Horizon 2020 ERC-funded research, by share of ERC-sponsored research outputs in 
the top 10% most-cited scientific publications. Source: Study team based on bibliometric analysis. 

280 

 
 
Annex 2.2: Contribution of the ERC to emerging areas in the Social 
Sciences and Humanities 

1.1. 

Introduction 

The European Research Council (ERC), set up by the European Union in 2007, is the premier 
European  funding  organisation  for  excellent  frontier  research.  The  ERC's  mission  is  to 
encourage the research of the highest quality in Europe through competitive funding, and to 
support 
fields,  based  on  scientific 
frontier  research  across  all 
excellence455. Under Horizon 2020, the ERC funded 6,707 projects, with a combined budget 
of EUR 13.3 billion, and evaluated around 54,000 research proposals456.  

investigator-driven 

Under Horizon Europe, the ERC remains an important funding scheme for Pillar 1. With an 
increased  budget  of  EUR  16  billion  for  the  period  2021-2027,  financing  for  the  ERC 
represents around 17% of the overall Horizon Europe budget. The ERC continues to offer 
four  main  grant  schemes:  Starting  Grants,  Consolidator  Grants,  Advanced  Grants  and 
Synergy Grants, using a ‘bottom-up' approach without predetermined priorities. 

In relation to the ERC’s mission to fund frontier research, this case study examines research 
in the areas of social sciences and humanities that has emerged from funding provided by 
the ERC, and which has been able to position Europe at the forefront of excellent science. It 
also examines the impact of the ERC on the research and careers of European researchers, 
and its capacity to attract and retain talent in Europe.  

This case study uses a combination of methods. In particular, a research trend analysis is 
used  to  identify  prominent  scientific  fields  and  high-quality  research  based  on  bibliometric 
data  from  Horizon  2020.  Desk  research  and  interviews with  relevant  beneficiaries  provide 
insights  enabling  a  better  understanding  of  the  factors  behind  the  ERC’s  contribution  to 
funding new research. In addition, EC monitoring data are used to provide complementary 
information about the ERC’s performance.  

Although this study primarily focuses on the mid-term results of Horizon Europe, the findings 
of this case study are mostly based on quantitative and qualitative information from Horizon 
2020 projects, because bibliometric data and the results and impacts of projects are not yet 
available for ongoing ERC projects under Horizon Europe.  

The case study is structured as follows. Section 1.2 provides a brief overview of the policy 
context  of  the  ERC  programme.  This  is  followed  by  a  description  of  the  methodological 
approach used in this case study457. Section 1.3 presents the main findings of the case study, 
arranged according to the evaluation criteria of relevance, efficiency, effectiveness and EU 
added  value.  Section  1.4  provides  insights  on  key  success  factors,  as  well  as  perceived 
barriers to the progress and impact of actions within the scope of this case study. Section 1.5 
presents the conclusions and lessons learned. 

455 ERC at a glance, https://erc.europa.eu/about-erc/erc-glance. 
456 European Commission, European Research Council Executive Agency, Mapping ERC 
frontier research Horizon 2020 Research and Innovation Framework Programme (2014-2020), 
Publications Office of the European Union, 2023, https://data.europa.eu/doi/10.2828/636104. 
457 This is developed in parallel with the case study report on the contribution of the ERC to 
emerging areas in the life sciences, physical sciences and engineering. 

281 

 
 
1.2.  Background information 

1.2.1.  Policy context 

Pillar I 'Excellent Science'  of Horizon Europe aims to promote scientific excellence; attract 
the best talent to Europe; provide appropriate support to early-stage researchers; and support 
the  creation  and  diffusion  of  scientific  excellence,  as  well  as  high-quality  knowledge, 
methodologies and skills, technologies and solutions to address global social, environmental 
and economic challenges. The European Research Council (ERC), as part of this pillar, is 
designed to provide attractive and flexible funding to enable talented and creative individual 
researchers (with an emphasis on early-stage researchers) and their teams, to pursue the 
most promising avenues at the frontiers of science, regardless of their nationality and country 
of  origin,  and  on  the  basis  of  Union-wide  competition  based  solely  on  the  criterion  of 
excellence. 

To fulfil its mission of supporting new ways of working in the scientific world, raising the profile 
of  frontier  research  in  Europe,  as  well  as  increasing  the  visibility  of  ERC  programmes  to 
researchers  across  Europe  and  internationally,  the  ERC  offers  four  main  grant  schemes: 
Starting Grants, Consolidator Grants, Advanced Grants and Synergy Grants, using a ‘bottom-
up' approach without predetermined priorities. In addition, the ERC awards complementary 
funding for the principal investigators (PIs) of its main grants.  

1.2.2.  Overall approach and scope of the case study 

This case study examines how the ERC, through its bottom-up approach and sole evaluation 
criterion  of  excellence  in  science,  contributes  to  fostering  research  in  the  fields  of  life 
sciences, physical sciences, and engineering. 

The  main  findings  of  the  case  study  are  arranged  according  to  the  evaluation  criteria  of 
relevance, efficiency, effectiveness, and EU added value. 

This case study relies on five main methods:   

•  Bibliometric  analysis  building  on  the  data  and  findings  from  the  Evaluation  study  on 

Excellent Science under Horizon 2020; 

•  Administrative and monitoring data for the Framework Programme, which are used 

to complement the bibliometric analysis; 

•  Desk  research  on  ERC  publications  and  documents,  policy  reports,  deliverables  of 

projects and other sources relating to the ERC's performance over time; 

• 

Interviews with four ERC beneficiaries, providing additional evidence on the relevance, 
effectiveness, and added value of the ERC in funding emerging areas; 

•  Survey  data  were  used  to  complement  information  about  the  impact  of  the  ERC  on 

beneficiaries’ careers.  

282 

 
 
 
 
1.3.  Case study findings 

1.3.1.  Relevance 

1.3.1.1. 

Top areas of research in the fields of social sciences and humanities in 
Europe 

Data from the Science, Research and Innovation Performance report458, published in 2022 
by the European Commission, provide an overall perspective on the production of science 
and  scientific  productivity  in  the  EU  and  in  the  rest  of  the  world.  Figures  for  2020459  (see 
Figure  1),  based  on  Scopus  data,  show  that  among  the  fields  of  social  sciences  and 
humanities (SSH), the two fields with the highest shares of scientific publications in the EU 
are  social  sciences,  and  economics  and  business  (with  4.6%  and  3.7%,  respectively). 
Furthermore, these two fields have displayed a significant increase in their shares between 
2010 and 2020.  

Compared  with  other  regions  of  the  world, in  2020,  the EU showed  an  impressive  lead  in 
several fields of SSH, in particular in the fields of historical studies (with 36% of worldwide 
scientific  production  in  the  field  coming  from  the  EU),  communication  and  textual  studies 
(30%) and economics and business (24%). While the EU is also relatively strong in the fields 
of visual and performing arts (28%) and social sciences (23%), the leading position in these 
fields is held by the United States.  

Figures  from  the  same  European  Commission  report  on  citations  (measured  as  the 
percentage  of  EU-funded  research  that  ranks  among  the  top  10%  most-cited  scientific 
publications) indicate that in 2018, among SSH fields; the EU also led in historical studies, 
with  31.5% of  its  scientific  publications  ranking  among  the  top  10% most-cited  worldwide. 
Similarly, the EU holds a strong position in the fields of communication and textual studies, 
as well as in economics and business (both with 26%), although it ranks second behind the 
United States in both of these fields. 

458 European Commission, Directorate-General for Research and Innovation, Science, research 
and innovation performance of the EU 2022: building a sustainable future in uncertain times, 
Publications Office of the European Union, 2022, https://data.europa.eu/doi/10.2777/78826.  
459 It can be noted that the fields with the highest numbers of publications are mainly STEM-
oriented fields. It should, however, be taken into account that scientific output practices differ 
between fields, with other types of publications (such as books or publications in national 
languages) and a slower pace of publishing being more common in SSH-fields. 

283 

 
 
Figure 1. EU share of publications per scientific field, 2010 and 2020. Source: Science, Research and Innovation 
Performance of the EU, 2022. 

1.3.1.2. 

Areas of research funded by the ERC in the fields of social sciences and 
humanities  

Data from the Horizon Dashboard on Horizon 2020 signed grants (see Appendix 2) show that 
the majority of ERC grants in SSH fields were signed in the field of social sciences, even 
though the single biggest field, by a considerable margin, is that of historical studies (including 
mediaeval  history,  social  and  economic  history,  early  modern  history,  and  modern  and 
contemporary history). These results are broadly in line with those across the broader EU 
scientific production landscape and, knowing that scientific publications which stem from ERC 
projects on average have a higher citation index460, this could be an initial indication that the 
ERC contributes to the production of high-quality scientific research in the field of historical 
studies in the EU, hence helping to position the EU at the forefront in this field.  

Data on Horizon Europe (see Table 1) points to a continuation of this trend, with most ERC 
signed grants being in the field of historical studies. Most of the top 10 fields under Horizon 
2020 thus far remain in the top 10 of Horizon Europe ERC grants. However, certain topics 
such  as  governance,  democracy  and  social  inequalities  appear  to  be  increasing  in 
importance,  which  may  be  a  result  of  recent  developments  such  as  the  energy  crisis,  the 
COVID-19 pandemic or the Ukrainian crisis, which have affected the EU and the rest of the 
world. 

460 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European framework programmes 
for research and innovation for excellent science : Horizon 2020: phase 1 final study report, 
Publications Office of the European Union, 2023, https://data.europa.eu/doi/10.2777/967813. 

284 

 
 
 
Top 10 EuroSciVoc concept 

Horizon Europe ERC signed 
grants 

Scientific field(s)* 

History 

Law 

Governance 

Democracy 

Revolutions 

Human migrations 

Social inequalities 

Linguistics 

Philosophy 

Productivity 

66 

33 

22 

21 

19 

16 

16 

14 

14 

14 

Historical studies 

Social sciences 

Social sciences 

Social sciences 

Social sciences 

Social sciences 

Social sciences 

Communication and textual 
studies 

Philosophy and theology 

Economics and business 

Table 1. Top 10 EuroSciVoc concepts underlying Horizon Europe signed ERC grants. Note: *Based on the EuroSciVoc 
Concept. The concepts are not mutually exclusive, and one grant can address more than one concept. Source: Horizon 
Dashboard (accessed on the 6 July 2023). 

1.3.1.3. 

Most prominent topics funded by the ERC in the fields of social sciences 
and humanities  

An  analysis  of  research  trends  carried  out  by  the  study  team  for  this  study,  based  on  the 
scientific publications produced through ERC projects under Horizon 2020, identified several 
prominent topics which can be ranked according to their total number of publications (see 
Table 2), or by their impact as measured by the share of ERC-sponsored research output 
that ranks in the top 10% or top 1% most-cited publications worldwide in the field (see Table 
5 and Table 6 in Appendix 2)461. However, it should be noted that these figures do not show 
the  shares  of  funding  or  of  projects  that  these  figures  represent  (for  example,  the  same 
number of publications could be produced by five projects worth  EUR 5 million, or by 100 
projects worth EUR 3 million). Nonetheless, the figures provide insights into which prominent 
topics the ERC is funding, and what impact these have had.  

Looking  at  the  ranking  based  on  total  number  of  publications,  the  top  10  prominent 
topics mostly focus on social sciences, with historical studies coming top. This is a similar 
pattern to that seen above with respect to ERC signed grants, which means that there is a 

461 In further detail, the concept of ‘prominent topics’ refers to important topics in terms of 
momentum and visibility. To calculate a topic’s prominence, the study team combined three 
metrics to arrive at an average constituting its prominence score. These three metrics are 
citation counts, Scopus view counts and Average CiteScore. Please see Annex 6 for a more 
detailed description of the methodology. 

285 

 
 
correspondence  between  the  number  of  grants  signed  and  the  number  of  scientific 
publications in a given field.  

Compared with the fields of life sciences, physical sciences and engineering, it is important 
to note that fewer ERC grants are given for in SSH topics, and consequently fewer scientific 
publications  stemming  from  them.  An  additional  element  to  consider  is  the  differences  in 
scientific output practices between fields, with other types of publications (such as books or 
publications  in  national  languages)  and  slower  pace  of  publishing  being  more  common  in 
SSH fields. Another interesting feature of SSH fields is that, unlike STEM462, some prominent 
topics  do  not 
top  1%  most-cited  scientific  publications  (e.g. 
Transnationalism;  Human  Migration;  Dual  Citizenship  or  Numerosity;  Number  Line; 
Approximate). In addition, several topics, especially those ranked by the citation metric (e.g. 
Structural  Decomposition  Analysis;  Carbon  Emissions;  Material  Flow  Analysis,  or  Climate 
Change;  Dry-Bulb  Temperature;  Electricity  Demand),  show  a  strong  degree  of 
multidisciplinarity. 

feature  among 

the 

The most prominent topic (Genome; Neanderthals; Anatomically modern humans) accounts 
for 70 publications, with 47% of the ERC-sponsored research output on this topic featuring 
in the top 10% most-cited scientific publications and 10% in the top 1% most-cited scientific 
publications.  In  addition  to  its  analysis  of  research  trends,  the  study  team  also  looked  at 
projects  with  a  relatively  high  number  of  publications  and  a  high  average  Field  Weighted 
Citation Impact (FWCI)463 – in particular, projects that addressed the same prominent topics 
identified. One such example of a successful project is RESOLUTION464, which produced 21 
publications with an average FWCI of 23.1%. 

When looking at the ranking based on citations, the top 10 prominent topics are quite 
distinct, with social sciences being the most common among the top 1% most-cited scientific 
publications, while economics and business occurred the most frequently in the top 10% 
most-cited scientific publications. Nonetheless, the topic with the highest share among the 
top  1%  most-cited  scientific  publications  is  communication  and  textual  studies,  which 
includes topics of “Journalism; News; Social media”. Under this topic, PersoNews465, with 11 
scientific publications and an average FWCI of 9.6%, is an example of a successful project. 
Other relevant fields with successful projects are ‘Platforms; Collaborative consumption; Peer 
to peer’, which includes the projects iLABOUR466 (11 publications, with an average FWCI of 
11%) and PLABOUR467 (12 publications, with an average FWCI of 3.7%).   

462 Science, Technology, Engineering and Mathematics. 
463 Field Weighted Citation Impact in SciVal indicates how the number of citations received by an 
entity’s publications compares with the average number of citations received by all other similar 
publications in the data universe: how do the citations received by this entity’s publications 
compare with the world average? 
https://www.elsevier.com/__data/assets/pdf_file/0020/53327/ELSV-13013-Elsevier-Research-
Metrics-Book-r12-WEB.pdf 
464 https://cordis.europa.eu/project/id/803147.  
465 https://cordis.europa.eu/project/id/638514. 
466 https://cordis.europa.eu/project/id/639652. 
467 https://cordis.europa.eu/project/id/759776. 

286 

 
 
Number of 
publications 

Emerging fields 

Field of science 

Top 1% 
share 

Top 
10% 
share 

70 

58 

57 

54 

52 

46 

46 

45 

43 

42 

Genome; Neanderthals; Anatomically 
Modern Humans 

Historical studies 

10.0% 

47.1% 

Human-Robot Interaction; Humanoid 
Robot; Man-Machine Systems 

Social sciences 

1.7% 

25.9% 

Punishment; Ultimatum Game; Guilt 
Aversion 

Social sciences 

1.8% 

14.0% 

Civil Conflict; Civil War; Militia 

Social sciences 

7.4% 

27.8% 

Cognitive Penetration; Free Energy; 
Visual Cortex 

Psychology and 
cognitive 
sciences 

7.7% 

46.2% 

Connectome; Functional Magnetic 
Resonance Imaging; Functional 
Connectivity 

Psychology and 
cognitive 
sciences 

10.9% 

43.5% 

Transnationalism; Human Migration; 
Dual Citizenship 

Social sciences 

0.0% 

26.1% 

Numerosity; Number Line; 
Approximate 

Economics and 
business 

0.0% 

17.8% 

Brazilian Amazon; Tropical 
Deforestation; Land Cover 

Social sciences 

7.0% 

37.2% 

Language Development; Word 
Learning; Artificial Grammar Learning 

Communication 
and textual 
studies 

0.0% 

28.6% 

Table 2. Top 10 prominent topics from Horizon 2020 ERC-funded research, by total number of publications. Source: 
compiled by the study team, based on bibliometric analysis.  

While the results above focus on the Top 200 most prominent topics that have produced the 
greatest numbers of publications, the question arises as to how much EC funding these topics 
received, and hence how ‘productive’ the creation of knowledge was in these specific topics. 
An  analysis conducted  in  the  context  of  this  study  shows  that  those  prominent  topics  that 
have led to the greatest number of publications in the programme parts are not necessarily 
the  ones  that  are  most  productive  (i.e.  those  with  the  highest  number  of  publications  per 
million EUR of EC funding)468. More details on this analysis are provided in Annex 6 of this 
study.  

468 This funding analysis shows that per programme part, the Top 200 most prominent topics that 
produced the largest numbers of publications, along with the topic’s total EC funding, and the 
number of publications published per million EUR of total EC funding. More information and 
results can be found in Annex 6, ‘Quantitative research track findings’ and Annex 2, ‘Case Study 
9 on Creating high-quality new knowledge’. 

287 

 
 
1.3.1.4. 

Comparison of the contribution made by Horizon 2020 to the most 
prominent topics in a global context  

Based on publications from the EU as a whole and from the main international knowledge 
producers (including a total of 32 countries) 469, the study team identified the 200 topics with 
the highest prominence scores470. The full set of the top 200 topics can be retrieved online471. 
From  this  list  of  200  prominent  topics,  it  was  possible  to  identify  those  that  also  featured 
among the top 10 identified above, and to make comparison between the contribution of the 
ERC and  Horizon  2020  (as  a  whole) with  EU-27  as  a  whole  and  selected  third  countries, 
according to each prominent topic identified. Table 3 below shows the seven prominent topics 
that correspond to the top topics identified for the ERC in the analysis above.  

Among the seven topics selected, it appears that the EU-27 as a whole contributes strongly 
to scientific production in SSH fields, with the EU’s shares in most fields being above those 
of the US and China. This is in line with the findings presented at the beginning of the Section 
1.3.1.  However,  the  analysis  indicates  that  for  all  the  topics  selected,  Horizon  2020 
contributes to a slightly more limited extent than is the case for STEM topics, with shares 
ranging between 2% and 6% (versus 5% and 14% in the analysis of STEM topics). Within 
the  contribution  made  by  Horizon  2020,  the  ERC’s  contribution  varies  depending  on  the 
different  topics.  The  highest  share  of  ERC-related  publications  is  found  in  the  topic  of 
‘Punishment;  Ultimatum  Game;  Guilt  Aversion’  (83%),  followed  by  ‘Greenspace;  Green 
Infrastructure;  National  Parks’  (40%)  and  ‘Water  Footprint;  Water-Energy  Nexus;  Nexus’ 
(35%).  The  lowest  ERC  shares  under  Horizon  2020  are  found  in  relation  to  ‘Multifactorial 
Inheritance;  Summary  Statistic;  Single  Nucleotide  Polymorphism’  (7%)  and  ‘ARIMA; 
Mathematical  Modelling;  COVID-19’  (9%).  The  latter  indicates  that  these  topics  thus  also 
have a strong presence in other programme parts of Horizon 2020. 

469 The detailed methodology is described in Annex 6. 
470 The top five topics with the highest prominence scores are: 1) Perovskite Solar Cells; Solar 
Cell; Formamidine; 2) Nivolumab; Pembrolizumab; Immunotherapy; 3) Molybdenum Disulfide; 
Monolayer; Van Der Waals; 4) Oxygen Production; Electrocatalysts; Catalyst; 5) Cyanogen; 
Graphitic Carbon Nitride; Photocatalysts.  
471 https://doi.org/10.34894/DNWIXV 

288 

 
 
ERC as a 
share of 
Horizon 2020-
funded 
publications 

Horizon 2020 
as a share of 
EU-27 
publications 

EU-27 as a 
share of all 
publications 

China as a 
share of all 
publications 

Japan as a 
share of all 
publications 

United 
Kingdom as 
a share of 
all 
publications 

United 
States as a 
share of all 
publications 

Total 
number of 
publications 
across all 
selected 
countries* 

19% 

83% 

6% 

2% 

53% 

3% 

12% 

10% 

22% 

9,466 

54% 

4% 

2% 

11% 

27% 

6,462 

18% 

5% 

32% 

19% 

2% 

9% 

37% 

16,278 

35% 

40% 

9% 

7% 

4% 

2% 

4% 

5% 

40% 

25% 

3% 

8% 

24% 

6,149 

47% 

11% 

3% 

16% 

22% 

7,508 

40% 

12% 

4% 

13% 

30% 

9,698 

41% 

6% 

2% 

16% 

35% 

7,884 

Prominent topics 

Human-Robot Interaction; 
Humanoid Robot; Man-
Machine Systems 

Punishment; Ultimatum 
Game; Guilt Aversion 

Connectome; Functional 
Magnetic Resonance 
Imaging; Functional 
Connectivity 

Water Footprint; Water-
Energy Nexus; Nexus 

Greenspace; Green 
Infrastructure; National 
Parks 

ARIMA; Mathematical 
Modelling; COVID-19 

Multifactorial Inheritance; 
Summary Statistic; Single 
Nucleotide Polymorphism 

Table 3. Prominent topics under the ERC and Horizon 2020, compared with the EU-27 overall and selected third countries. Note: *number of publications as identified by the study team in the 
bibliometric analysis for the selected countries. Source: compiled by the study team, based on the bibliometric analysis.

289 

 
1.3.1.5. 

Impact of the ERC on researchers’ careers 

The study team did not identify differences between the impacts of the ERC on the careers 
of  researchers  in  the  fields  of  social  sciences  and  humanities,  compared  with  those  on 
researchers  in  the  fields  of  life  sciences,  physical  sciences  and  engineering.  Both  case 
studies therefore report similar findings with regard to this point. Overall, the ERC has very 
strong impacts on the careers of both the principal investigator (PI) and the research team 
as a whole.  

In the case of the PI, one of the main motivating factors for applying for an ERC grant is the 
positive  impact  it  can  have  on  the  researcher’s  career,  especially  among  early-stage 
researchers applying for ERC Starting Grants. However, the very low success rate and the 
effort,  time  and  resources  required  to  prepare  an  excellent  proposal  may  act  as  a 
discouraging factor for early-stage researchers. Interviewees indicate that an ERC Starting 
Grant had greatly increased their chances of obtaining a permanent position at a top research 
institution.  Some  benefits  mentioned  during  the  interviews  include  the  mobility  from  one 
country to another, as universities tend to compete to attract ERC grant holders, as well as 
an  increase  in  the  researcher’s  independence  and  visibility  within  and  even  outside  their 
research  field.  With  respect  to  the  other  main  ERC  grants,  the  fact  that  these  grants  are 
considered highly prestigious and a strong indicator of the quality of the proposed research 
also positively impacts the reputation of the researcher. 

Survey data on successful ERC PIs under Horizon Europe shows that 71% (171 out of 241 
respondents, ERC-BEN-025) stated that the ERC project had, to a very large or large extent, 
given  them  the  opportunity  to  gain  a  research  position  with  better  career  prospects.  The 
percentage is even higher (91% or 218 out of 240 respondents) for those respondents who 
claimed it had increased the international visibility and outreach of their research.  

With regard to the PI’s research team, interviews indicate that members of the team benefit 
from the experience brought by the PI, the stability offered by a lengthy ERC-funded project 
(in comparison to other funding opportunities), and the prestige associated with being part of 
an ERC project – especially for those researchers just embarking on their research careers. 
In  addition,  interviewees  mentioned  that  in  comparison  to  other  funding  opportunities,  the 
size  of  the  grants  provided  by  the  ERC allows  larger research  teams,  and  thus  offers  the 
flexibility  to  hire  researchers  from  different  (sub-)disciplines.  This  further  enriches  the 
experience for the early-career stage researchers involved. Several interviewees mentioned 
that after the project, most members of their research team had continued to pursue a career 
in academia in the same field of research, and in the longer run several had taken up leading 
research positions. In the specific case of ERC Synergy Grants, younger researchers from 
different teams can benefit from greater collaboration, which is mentioned as also increasing 
their chances of publishing a greater number of scientific articles. 

1.3.2.  Efficiency 

1.3.2.1. 

Attracting talent into the fields of social sciences and humanities  

The  ERC  has  a  strong  capacity  for  retaining  and  attracting  top  talent  in  Europe.  As 
several  interviewees  mentioned,  ERC grants  are a  very  prestigious  label  in  researchers’ 
careers,  and  receiving  a  grant  leads  almost  immediately  to  a  permanent  position  or 
promotion. Therefore, despite the perception of low success rates, the prestige of holding an 
ERC grant and the very positive impact it has on researchers’ careers are the main reasons 
for applying.  

290 

 
Other elements that attract researchers include the  size and duration of the grants, and 
the relative flexibility to adapt the research to changes in context (e.g. during the COVID-19 
pandemic, travel budgets could be used differently to support the project) or findings (e.g. 
using findings in a different way from that originally envisaged in order to support new aspects 
in the research field). 

In  a  similar  manner  to  that  seen  in  the  fields  of  life  sciences,  physical  sciences  and 
engineering,  the  ERC  has  funded  top  researchers  in  the  fields  of  social  sciences  and 
humanities, as indicated by several prestigious prizes being awarded to outstanding scholars 
in SSH who have at some point been awarded an ERC grant.  

1.3.3.  Effectiveness 

The same elements that make ERC grants attractive to (top) researchers are also mentioned 
in relation to the effectiveness of ERC grants and the extent to which they are able to create 
results and impacts. For instance, the PIs interviewed stressed that the size and duration of 
the  grants  enabled  them  to set  up  a relevant  research project with  a  sufficiently  large 
team, consequently allowing them to combine complementary expertise into the team. The 
reputation of ERC grants has also enabled them to attract and hire top-quality early career 
stage researchers to the team. These factors thus facilitate the generation of high-quality 
and novel knowledge in Europe. 

In  terms  of  support  for  emerging  fields  of  research,  an  important  factor  mentioned  by 
interviewees concerns the ERC’s ‘bottom-up’ approach and its emphasis on excellence as 
the sole criterion for ERC grants. This means that very novel and excellent research ideas 
have  access  to  substantial  funding.  The  ERC  thus  contributes  to  the  very  early 
development of knowledge in new areas, and to being at the forefront of the development of 
new research fields. When these areas later emerge as important and growing fields (whether 
due to internal or external factors), a primary knowledge base is available to provide insights 
(both  for  the  research  community,  and  for  policy  makers,  for  example),  and  to  be  further 
developed. 

1.3.4.  EU added value 

The evidence collected for the elaboration of this case study indicates that the main areas of 
EU added value provided by the ERC under the Framework Programmes, and particularly in 
Horizon Europe, are as follows:  

• 

Investing in discovery-driven research at EU level is considered essential in order to 
position Europe at the forefront of excellent science. The analysis and examples provided 
in  this  case  study  have  demonstrated,  for  example,  the  importance  of  supporting 
prominent fields at an early stage in their development – not only in order to be at the 
forefront of scientific research in the world, but also to create relevant and high-quality 
knowledge and insights by the time the topic gains societal interest. Interviewees also 
stressed the importance of the sole selection criterion of excellence for truly bottom-up 
research: ‘high-risk/high-gain’ research needs to allow for both unexpected outcomes or 
failure. The ERC is considered the main funding instrument in Europe for funding bottom-
up  fundamental  research,  and  ERC  grants  are  considered  both  more  substantial  and 
prestigious by the research community than national funding opportunities. ERC grants 
offer researchers the opportunity to gain funding for high-risk/high-gain research projects 
of  sufficient  scale  to  be  relevant  and/or  to  be  able  to  define  more  complex  research 
projects, combining complementary research steps or expertise into a single project, and 
generating  impacts  in  research  and  beyond,  without  the  prerequisite  to  define  these 
impacts in detail ex-ante. As an alternative, national funding opportunities generally do 

291 

 
not offer this same degree of excellence-driven research funding; nor do they offer the 
same  scale  of  funding  within  a  single  grant.  Interviewees  point  to  these  elements  as 
important factors in the added value of the EU-level approach. 

• 

Improving  researchers’  careers  prospects  and  specialisation,  especially  during 
early career stages. The prestige associated with an ERC grant was mentioned by all 
interviewees  as  having  a  positive  impact  on  their  careers  in  terms  of  reputation  and 
visibility. As one interviewee mentioned, the ERC is uniformly recognised across Europe 
for its  excellence  and  highly  quality-driven selection  procedure.  In  addition,  interviews 
mentioned that working on an ERC application requires them to reflect on the longer-
term direction of their research in relation to the broader research field. Such reflection 
helps them to identify those pathways with high expected relevance and impact, in order 
to define their individual future research direction.  

•  The  funding  provided  by  the  ERC  enables  projects  with  a  longer  duration  and 
larger teams than other  types of  funding.  As explained above, these two elements 
are very important for researchers to engage in ground-breaking and emerging research, 
enabling new discoveries but also allowing for failure and the redirection of elements of 
the  research,  and  providing  the  opportunity  to  develop  a  research  team  with  the 
necessary (complementary) expertise to achieve the project objectives (thus contributing 
to multidisciplinarity in the project’s approach). 

•  ERC Synergy Grants enable a well-appreciated mix of bottom-up and collaborative 
research. Synergy Grants are unique enablers for groups of PIs to come together and 
work on ambitious subjects. These grants, albeit to a limited extent, respond to the lack 
of  bottom-up  collaborative  research  funding  instruments  referred  to  by  some  position 
papers submitted to the OPC472.  

•  One  feature  that  appears  to  be  more  prevalent  among  SSH  projects  than  STEM 
projects relates to multidisciplinarity across STEM and SSH fields. The topic-based 
analysis identified many topics in SSH that integrate methods or expertise from STEM 
fields,  thus  showing  a  strong  degree  of  multidisciplinarity.  The  RESOLUTION  and 
iLABOUR projects mentioned above are both examples of this.  

1.4.  Success factors and barriers 

When analysing the ERC’s support for emerging fields of research, both the success factors 
and  the  main  barriers  and  challenges  identified  are  generally  applicable,  with  no  factors 
identified as being specific to STEM fields on the one hand or SSH fields on the other. As a 
result, such factors are described in similar terms in both of the ERC case studies. 

The following success factors have been identified:  

•  The ERC is unique in funding bottom-up and ‘blue-sky’ research, which makes it 
highly appealing to researchers aiming to carry out new and groundbreaking research. 
This impression was shared both by the interviewees and by the position papers from 
the OPC473. In addition, the design of ERC grants does not penalise research outcomes 

472 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 
473 Ibid. 

292 

 
 
that  are  different  from  those  envisaged  beforehand,  thus  allowing  the  discovery  of 
different outcomes or pathways from those originally intended. This is a necessary factor 
to encourage truly novel research orientations. 

•  Related  to  the  above  point,  the  curiosity-driven  design  of  ERC  grants  thus  allows 
changes  to  be  accommodated  quickly  during  the  course  of  a  project,  thus  allowing 
opportunities  for  new  research  in  rapidly  evolving  fields.  As  an  illustration  of  the 
importance of this kind of flexibility, reference can be made to a project in which one work 
package out of three did not deliver in terms of publication objectives, but instead ended 
up  supporting  the  other  parts  by  informing  research  in  the  other  two  work  packages 
through high-quality data. In addition, ERC projects are not bound to organise their work 
in  the  form  of  work  packages,  thus  allowing  flexibility  regarding  research  design  and 
allowing PIs to integrate emerging aspects that were not foreseen at the beginning of the 
project. 

•  The composition of teams and length of projects were also identified as two important 
factors contributing to the success of the ERC in funding groundbreaking and emerging 
science. These factors were highlighted by several interviewees, who made reference to 
the  relatively  high  degree  of  freedom  provided  by  the  ERC  in  building  diverse  and 
sufficiently large research teams, thus contributing to increased multidisciplinarity.  

•  The  evaluation  process  and  governance  are  also  important  success  factors 
contributing  to  the  excellence  of  research  funded  by  the  ERC.  Elements  that  are 
considered important in this regard are the application of scientific excellence as the sole 
selection  criterion,  as  well  as  the  use  of  a  panel-based,  peer-review  evaluation.  The 
independence of the ERC’s Scientific Council as its main governing body is considered 
essential  to  this.  These  aspects  were  mentioned  in  some  position  papers  from  the 
OPC474.  

To date, the main barriers and challenges identified include: 

•  Relatively low success rates for ERC grants continue to be mentioned as a barrier for 
applying  to  the  ERC.  As  such,  they  imply  that  an  important  proportion  of  excellent 
research ideas may not be receiving funding. Some interviewees indicated concerns that 
this  barrier  represents  an  even  greater  risk  in  the  case  of  more  novel  and  disruptive 
ideas.  A  few  position  papers  have  therefore  called  for an  increase  in  the  total  budget 
allocated to the ERC in order to fund more projects, and consequently to retain and fund 
a higher number of European researchers. Researchers who took part in the interview 
programme  also  pointed  to  this,  but  at  the  same  time,  as  successful  applicants, 
mentioned that the added value of an ERC grant strongly outweighed the costs related 
to the development of an application combined with the relatively low success rates. 

•  Related  to  the  above  point,  the  structure  of  the  ERC’s  evaluation  panels  may  be 
another  factor  hindering  the  realisation  of  its  full  potential  to  fund  highly  novel  and 
emerging  research.  A  few  interviewees  mentioned  that,  when  applying  for  funding 
multidisciplinary or highly novel research ideas, there is not always a natural fit with one 
of the available evaluation panels. As a result, funding for projects in novel or emerging 
fields can prove more difficult to obtain. This challenge relates to the ERC’s otherwise 

474 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 

293 

 
 
well-appreciated  expert  evaluation  approach,  and  interviewees  mentioned  that  further 
guidance from the experts in the evaluation panels could help to improve the assessment 
of multidisciplinary or highly novel research proposals. 

1.5.  Conclusions  

In  the  fields  of  social  sciences  and  humanities,  ERC-funded  research  has  contributed  the 
most in terms of the quantity of outputs (in this case, scientific publications) in the fields of 
social  sciences  and  historical  studies.  In  terms  of  impact  (measured  by  scientific 
publication citations), its greatest contribution has been in the fields of social sciences, and 
in economics and business. The topic with the highest share of top 1% most-cited scientific 
publications is communication and textual studies, with the topic of ‘Genome; Neanderthals; 
Anatomically modern humans’ in particular combining a large number of publications with a 
very strong impact indicator. Data on Horizon Europe points to a continuation of this trend, 
with most ERC grants being signed in the field of historical studies. Most of the top 10 fields 
for  ERC  grants  under  Horizon  2020  so  far  remain  in  the  top  10  under  Horizon  Europe. 
However, certain topics such as governance, democracy and social inequalities appear to be 
increasing in importance, which may be a result of recent developments such as the energy 
crisis, COVID-19 pandemic and Ukrainian crisis, which have affected the EU and the rest of 
the world. 

In an international context, the analysis of research trends475 in seven selected topics shows 
that the EU-27 as a whole contributes strongly to scientific production in the fields of 
SSH,  with  shares  in  most  fields  being  above  those  of  the  US  and  China.  However,  the 
analysis indicates that for all the topics selected, the contribution to SSH made by Horizon 
2020 is slightly more limited extent than that for STEM-topics and that. Furthermore, within 
the contribution made by Horizon 2020, the ERC contributed to a varied extent depending 
on the different topics. However, when considering the differences between SSH and STEM 
topics, it should be noted that the analysis performed did not take into account other outputs 
such as books, which are more prominent in SSH fields. The highest share of ERC-related 
publications  is  found  in  the  topic  of  ‘Punishment;  Ultimatum  Game;  Guilt  Aversion’  (83%), 
followed by ‘Greenspace; Green Infrastructure; National Parks’ (40%) and ‘Water Footprint; 
Water-Energy Nexus; Nexus’ (35%).  

With  its  curiosity-driven  and  excellence-based  grants,  the  ERC  is  thus  found  to 
contribute to the development of new fields of scientific research in Europe. To some 
extent, these fields relate to societal and strategic evolutions, such as the development of 
increasingly  resource-  and  climate-friendly  technologies  (e.g.  solar  cells  based  on 
Perovskite) or COVID-19 related research topics.  

The ERC’s strictly bottom-up approach and size of its grants are key aspects that make 
the ERC a successful instrument for supporting excellent and emerging research. The main 
elements that result from its design, and which impact the extent to which emerging fields are 
supported by the ERC, are:  

1)  The  relatively  long  duration  of  projects,  which  allows  the  development  of  novel 

approaches and provides stability for research teams;  

2)  The relatively substantial size of the research teams funded, which allows for the hiring 

of complementary expertise and, where relevant, multidisciplinarity in the team;  

475 For more information, see Annex 6. 

294 

 
 
3)  The  strictly  curiosity-driven  approach,  and  the  lack  of  a  strong  requirements  for  the 
expected  impacts  of  the  research  to  be  defined  ex-ante,  allow  flexibility  in  finding 
unexpected outcomes and – within the boundaries of the project – to reorient towards 
more promising research paths and outcomes.  

The above elements, combined with the positive reputation associated with the ERC grants, 
make  the  ERC  grants  attractive  to  top  researchers  both  within  and  outside  Europe  and, 
according to interviews with PIs, allow ERC grantees to attract and hire high-quality young 
researchers to the team, thus further positively impacting the quality of the research.  

However, further analysis conducted in Phase 2 of this study examined whether receiving 
Framework Programme funding made a difference to the novelty and disruptiveness of the 
research publications that researchers produce later on in their careers476. The results of this 
analysis  do  not  provide  significant  evidence  that  those  researchers  who  received  ERC 
funding under Horizon 2020 generated more novel or disruptive research output two or five 
years  after  the  application,  compared  with  their  counterparts  who  also  applied  but  did  not 
receive funding.  

Although the mapping of topics supported through the ERC reveals certain differences 
in  terms  of  outcomes  in  the  fields  of  life  sciences,  physical  sciences  and  engineering 
compared  with  social  sciences  and  humanities,  this  case  study  does  not  identify 
meaningful differences, in terms of the process or factors with regard to the way the ERC 
supports emerging fields, between each of these wide domains of research. However, the 
illustrative  examples  of  high-impact  projects  in  this  case  study  do  suggest  an  important 
degree of multidisciplinarity among the projects funded within SSH fields. 

Appendix 1. Literature sources 

•  Commission Implementing Decision adopting the 2023 Work Programme, in the 

framework of the Specific Programme Implementing Horizon Europe – The Framework 
Programme for Research and Innovation (2021-2027) in relation to the component 
‘European Research Council' under Pillar I ‘Excellent Science’ - C/2022/4861 final, 
available at: https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=PI_COM:C(2022)4861. 

•  Cordis webpages of selected projects. 

•  ERC at a glance. Available at: https://erc.europa.eu/about-erc/erc-glance. 

•  European Commission, Directorate-General for Research and Innovation, Science, 

research and innovation performance of the EU 2022 – Building a sustainable future in 
uncertain times, Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/78826  

•  European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European framework 
programmes for research and innovation for excellent science : Horizon 2020 : phase 1 

476 More information on these indicators and the analysis can be found in the Novelty and 
Disruptiveness analysis, Annex 6. 

295 

 
 
 
final study report, Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/967813.  

•  European Commission, European Research Council Executive Agency, Mapping ERC 
frontier research Horizon 2020 Research and Innovation Framework Programme 
(2014-2020), Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2828/636104.    

•  European Parliament and Council (2021). Regulation (EU) 2021/695 of the European 
Parliament and of the Council of 28 April 2021 establishing Horizon Europe – the 
Framework Programme for Research and Innovation, laying down its rules for 
participation and dissemination, and repealing Regulations (EU) No 1290/2013 and 
(EU) No 1291/2013, 28 April 2021, PE/12/2021/INIT, OJ L 170, 12.5.2021, pp. 1-68. 
Available at: http://data.europa.eu/eli/reg/2021/695/oj    

•  European Science Vocabulary. Available at: https://op.europa.eu/en/web/eu-

vocabularies/dataset/-
/resource?uri=http://publications.europa.eu/resource/dataset/euroscivoc. 

•  Field-Weighted Citation Impact in SciVal. Available at: 

https://www.elsevier.com/__data/assets/pdf_file/0020/53327/ELSV-13013-Elsevier-
Research-Metrics-Book-r12-WEB.pdf. 

•  Horizon Dashboard (accessed 6 July 2023). 

• 

• 

• 

https://en.wikipedia.org/wiki/Holberg_Prize. 

https://en.wikipedia.org/wiki/Stein_Rokkan_Prize_for_Comparative_Social_Science_Re
search. 

https://en.wikipedia.org/wiki/Swedish_Society_for_Anthropology_and_Geography#Retz
ius_Medal. 

Appendix 2. Relevance evaluation criteria 

Areas of research funded by the ERC in the fields of social sciences and humanities  

Top 10 EuroSciVoc 
concept 

Horizon 2020 ERC signed 
grants 

Scientific fields* 

History 

Law 

Revolutions 

Historical studies 

Social sciences 

Social sciences 

386 

129 

120 

296 

 
 
Top 10 EuroSciVoc 
concept 

Horizon 2020 ERC signed 
grants 

Scientific fields* 

Governance 

Human migrations 

Employment 

Economics 

Democracy 

Social inequalities 

Linguistics 

109 

104 

95 

88 

87 

83 

77 

Social sciences 

Social sciences 

Economics and business 

Economics and business 

Social sciences 

Social sciences 

Communication and textual 
studies 

Table 4. Top 10 EuroSciVoc concepts477 underlying Horizon 2020 signed ERC grants. Note: *scientific fields are based on 
the EuroSciVoc concepts. These concepts are not mutually exclusive, and one grant can address more than one concept. 
Source: Horizon Dashboard (accessed 6 July 2023). 

Most  prominent  topics  funded  by  the  ERC  in  the  fields  of  social  sciences  and 
humanities  

Top 1% 
Share 

Emerging fields 

Field of Science 

Number of 
publications 

Top 10% 
Share 

30.0% 

Journalism; News; Social Media 

Communication 
and textual 
studies 

10 

50.0% 

27.3% 

26.7% 

26.3% 

Structural Decomposition Analysis; 
Carbon Emissions; Material Flow 
Analysis 

Economics and 
business 

ARIMA; Mathematical Modelling; 
COVID-19 

Economics and 
business 

Platforms; Collaborative 
Consumption; Peer to Peer 

Economics and 
business 

33 

69.7% 

15 

19 

66.7% 

73.7% 

477 European Science Vocabulary (EuroSciVoc) is the taxonomy of fields of science based on 
the OECD's 2015 Frascati Manual taxonomy. It was extended to include fields of science 
categories extracted from CORDIS content through a semi-automatic process developed using 
Natural Language Processing (NLP) techniques (https://op.europa.eu/en/web/eu-
vocabularies/dataset/-/resource?uri=http://publications.europa.eu/resource/dataset/euroscivoc). 

297 

 
 
Top 1% 
Share 

Emerging fields 

Field of Science 

Number of 
publications 

Top 10% 
Share 

21.4% 

Environmental Justice; Injustice; Air 
Quality 

Social sciences 

20.0% 

20.0% 

Climate Change; Dry-Bulb 
Temperature; Electricity Demand 

Economics and 
business 

DSGE Model; Interest Rates; 
Monetary Policy 

Economics and 
business 

18.8% 

Hukou; China; Land Development 

Social sciences 

14 

10 

10 

16 

64.3% 

60.0% 

30.0% 

37.5% 

16.7% 

Multifactorial Inheritance; Summary 
Statistic; Single Nucleotide 
Polymorphism 

Psychology and 
cognitive 
sciences 

12 

58.3% 

15.4% 

One Child Policy; Family Planning 
Policy; Dual-Earner 

Social sciences 

13 

38.5% 

Table 5. Top 10 prominent topics from Horizon 2020 ERC-funded research, by share of ERC-sponsored research output in 
the top 1% most-cited scientific publications. Source: compiled by the study team, based on bibliometric analysis. 

Top 10% 
Share 

Emerging fields 

Field of Science 

Number of 
Publications 

Top 1% 
Share 

73.7% 

73.7% 

70.6% 

69.7% 

Water Footprint; Water-Energy 
Nexus; Nexus 

Economics and 
business 

Platforms; Collaborative 
Consumption; Peer to Peer 

Economics and 
business 

Material Flow Analysis; Strategic 
Materials; Metals 

Economics and 
business 

38 

19 

17 

7.9% 

26.3% 

5.9% 

Structural Decomposition Analysis; 
Carbon Emissions; Material Flow 
Analysis 

Economics and 
business 

33 

27.3% 

67.6% 

Greenspace; Green Infrastructure; 
National Parks 

Economics and 
business 

66.7% 

ARIMA; Mathematical Modelling; 
COVID-19 

Economics and 
business 

64.3% 

Environmental Justice; Injustice; Air 
Quality 

Social sciences 

34 

15 

14 

11.8% 

26.7% 

21.4% 

298 

 
Top 10% 
Share 

Emerging fields 

Field of Science 

Number of 
Publications 

Top 1% 
Share 

60.7% 

Soundscape; Tranquillity; Noise 
Control 

Psychology and 
cognitive 
sciences 

28 

7.1% 

60.0% 

Climate Change; Dry-Bulb 
Temperature; Electricity Demand 

Economics and 
business 

10 

20.0% 

58.3% 

Multifactorial Inheritance; Summary 
Statistic; Single Nucleotide 
Polymorphism 

Psychology and 
cognitive 
sciences 

12 

16.7% 

Table 6. Top 10 prominent topics from Horizon 2020 ERC-funded research by the share of ERC-sponsored research outputs 
in the top 10% most-cited scientific publications. Source: compiled by the study team, based on bibliometric analysis. 

299 

 
Annex 2.3: The structuring impacts of the MSCA 

1.1. 

Introduction 

The Marie Skłodowska-Curie Actions (MSCA) are the EU flagship programme designed to 
fund research and innovation and to equip researchers with new and relevant skills that will 
support them at every stage in their careers. The Actions achieve this objective by enabling 
the  international,  interdisciplinary  and  cross-sectoral  mobility  of  researchers,  and  by 
disseminating  and  promoting  best  practices  in  terms  of  both  the  quality  of  the  training 
provided and in the management, promotion and acquisition of human resources, including 
high-quality  working  and  employment  conditions.  This  case  study  aims  to  evaluate  the 
coherence, relevance, effectiveness, efficiency and EU added value of the MSCA in creating 
structuring impacts, as well as how such structuring impacts are generated. It also examines 
how these impacts can be transferred beyond the MSCA.  

With  respect  to  the  aims  listed  above,  the  present  case  study  focuses  on  participating 
organisations  within  Doctoral  Networks  (DN),  Staff  Exchanges  (SE)  and  COFUND.  It 
examines how their involvement in the MSCA leads – or is expected to lead – to structural 
changes. This case study takes an exploratory approach, analysing in depth the structuring 
effects  observed;  however,  this  is  subject  to  data  availability.  The  main  focus  of  the  case 
study  is  on  structural  changes  at  institutional  level,  but  regional  and  national  effects  are 
detailed where they are identified.  

1.2.  Background information 

1.2.1.  Policy context 

In  its  2030  Vision  on  the  Future  of  Universities,  the  European  Commission  laid  down  its 
strategic goals for developing higher education in Europe, with the aim of strengthening the 
role  of  universities  in  future  societies  through  actions  such  as  better  knowledge 
dissemination,  the  delivery  of  talented  graduates  and  researchers  into  the  market,  and 
promoting deeper cooperation with other sectors.478  

Various factors contributing to this goal have been included within this strategy. These include 
improving higher education to promote skills that are better suited to future needs; promoting 
diversity,  inclusiveness  and  gender  equality;  and  developing  skills,  competencies  and 
technological innovation suited for the green and digital transitions  – all of which have the 
aim  of  creating  better  graduates  that  can  contribute  to  tackling  Europe’s  societal 
challenges.479  

The MSCA have emerged as a European response to achieving these goals by promoting 
structuring impacts that embed these goals as good practices within the overall functioning 
of European universities and research institutes. The MSCA programme aims to achieve this 
by fostering collaborations, knowledge transfer, training, supervision and career guidance. 
The  MSCA  also  contribute  to  spreading  good  practices  by  supporting  the  mobility  of 

478 European Commission. Directorate General for Research and Innovation. and Centre for 
Strategy & Evaluation Services LLP (CSES)., Towards a 2030 Vision on the Future of 
Universities in Europe. (LU: Publications Office, 2020), 7, 
https://data.europa.eu/doi/10.2777/510530. 
479 European Commission, 'Commission Communication on a European Strategy for 
Universities', 2022. 

300 

 
 
researchers between countries, sectors and disciplines in order to acquire new knowledge, 
skills and competencies. Similarly, the programme promotes the principles of the European 
Charter  for Researchers  and  Code  of  Conduct  for the Recruitment  of  Researchers,  which 
facilitate transparency and promote improvements in recruitment practices and researchers' 
working  and  employment  conditions.  The  MSCA  Guidelines  on  Supervision,  which  are 
aligned with the principles of the Charter and Code, also seek to ensure that researchers are 
adequately supervised throughout the duration of their research project.480  

In  this  respect,  previous  studies  and  evaluations  have  shown  that  the  MSCA  have  strong 
structuring  impacts.  In  addition  to  promoting  the systematic implementation  of  the  Charter 
and  Code,  the  MSCA  set  standards  for  quality  (doctoral)  training,  attractive  working  and 
employment  conditions,  as  well  as  open,  transparent,  merit-based  recruitment  for  all  EU 
researchers.481  Other  evaluations  confirm  that  participation  in  the  MSCA  improves  human 
resource practices and procedures and provides researchers with fairer and more attractive 
working conditions, while at the same time improving the quality of researchers’ training and 
supervision.482 The Charter and Code have been particularly relevant to enacting structuring 
impacts, as they enable better recruitment practices and gender inclusion and, overall, the 
promotion of better policies483. 

The MSCA also contribute to harmonising and standardising the requirements and practices 
of doctoral programmes – for example, by creating joint degrees.484 The MSCA can therefore 
be  considered  to  have  a  structuring  impact  in  terms  of  improving  the  state  of  doctoral 
education  in  Europe.  Doctoral  education  in  Europe  has  remained  largely  reliant  on 
conventional education models485, even while there has been an ongoing emergence of more 
integral training models guided largely by the Seven Principles of Innovative Doctoral Training 
(IDTP)  adopted  by  the  Council  of  the  EU486.  This  shift  has led  to  the  appearance  of  more 
multi-dimensional  doctoral  programmes,  and  has  contributed  to  a  change  in  the  profile  of 
doctoral  graduates,  who  are  now  taught  a  higher  number  of  transversal  skills  during  their 
training.  Here,  the  focus  of  doctoral  programmes  has  also  undergone  some  important 
changes, shifting from a predominant focus on the training dimension of doctoral education, 
to giving more room to other aspects of doctoral education, such as mentoring and mobility, 
which are now regarded as equally important as research and training487. 

480 European Commission (2021). Marie Skłodowska-Curie Actions. Guidelines on Supervision.  
481 See, for example, European Commission (2017). FP7 ex post and H2020 interim evaluation 
of Marie Skłodowska-Curie Actions (MSCA). 
482 See, for example, European Commission. (2023). Evaluation study on Excellent Science in 
the European Framework Programmes for Research and Innovation. Phase 1. 
483 Survey of MSCA organisations (2022), conducted for the Study on mobility flows in the 
context of the MSCA.  
484 See, for example, European Commission. (2023). Evaluation study on Excellent Science in 
the European Framework Programmes for Research and Innovation. Phase 1 Annexes. Case 
Study 3: Impact of the MSCA ITN on doctoral programmes. 
485 Sabic, N. (2014). Comparative Analysis of Doctoral Education in Europe. Annals of the 
University of Bucharest / Political science series, 16(1), 129-147. https://nbn-
resolving.org/urn:nbn:de:0168-ssoar-397492 
486 ERA Steering Group for Human Resources and Mobility. (2014). Using the Principles for 
Innovative Doctoral Training as a Tool for Guiding Reforms of Doctoral Education in Europe: 
Report of the ERA Steering Group Human Resources and Mobility. ERA SGHRM. 
https://cdn5.euraxess.org/sites/default/files/principles_for_innovative_doctoral_training.pdf 
487 Hasgall, A., & Peneoasu, A.M. (2022). Survey – Doctoral education in Europe: current 
developments and trends. European University Association. 

301 

 
 
 
Similarly,  acquiring  and  disseminating  transferable  skills  has  become  an  important 
component  of  both  doctoral  education  and  researchers’  careers.  However,  this  has  been 
constrained  by  many  barriers,  such  as  the  perceived  lack  of  structured  programmes  and 
standardised  training  for  researchers  at  doctoral  level.488  Similarly,  while  mentorship  and 
mobility  have  become  more  prominent  in  researchers’  careers,  they  have  also  faced 
important  challenges,  such  as  the  (over)reliance  of  institutions  on  a  single  mentor;  a 
perceived  lack  of  institutional  diversity  and  mentor  training489;  or,  in  the  case  of  mobility, 
increasing  threats  to  its status  in  the  aftermath  of  the  COVID-19  pandemic490.  The  MSCA 
have achieved structuring impacts that address these challenges. Previous evaluations have 
shown that the training dimension of the MSCA is strong, particularly in terms of equipping 
researchers with transferable skills that are useful to early-stage researchers491, as well as in 
fostering an entrepreneurial mindset.492 

In addition, the MSCA have made an important contribution to building new and sustainable 
international and intersectoral partnerships and networks. During Horizon 2020, the MSCA 
contributed  to  the  ability  of  fellows  to  build  international  networks.493  This  trend  has  been 
similar in relation to inter-sectoral cooperation, both in terms of the development of academia-
industry  networks494  and  in  fellows’  abilities  to  build  and  maintain  international  and 
intersectoral partnerships.495 Both of these abilities are necessary to increase researchers’ 
competitiveness, according to the Commission’s own Strategy for Universities.496 

To achieve their objectives, the MSCA implement five types of actions to support excellent 
research. This case study will focus on three of these, namely:  

•  Doctoral Networks (DN), previously known as Innovative Training Networks under 
Horizon  2020,  aim  to  implement  doctoral  programmes  with  organisations  from 

https://eua.eu/downloads/publications/web_council%20on%20doctoral%20education_horizontal.
pdf. p. 22. 
488 Stehen, K., Vornhagen, J., Weinberg, Z.Y., Boulanger-Bertolus, J., Rao, A., et al. (2021) A 
structured professional development curriculum for postdoctoral fellows leads to recognized 
knowledge growth. PLOS ONE 16(11) : e0260212. https://doi.org/10.1371/journal.pone.0260212 
489 Lambert, W. Marcus, et al. /Addressing Structural Mentoring Barriers in Postdoctoral Training: 
A Qualitative Study’. bioRxiv (2022): 2022-08. https://doi.org/10.1101/2022.08.20.504665 
490 Hasgall, A. & Penegasu, A.-M. (2022). p. 23 
491 European Commission. Directorate General for Education, Youth, Sport and Culture., FP7 Ex 
Post and H2020 Interim Evaluation of Marie Skłodowska-Curie Actions (MSCA): Final Report. 
(LU: Publications Office, 2017), p. 23, https://data.europa.eu/doi/10.2766/103174. [Accessed on 
June 2022] 
492 European Commission. Directorate General for Education, Youth, Sport and Culture., Interim 
Evaluation of Horizon 2020: Commission Staff Working Document : Annex 2, 2017, p. 173, 
https://op.europa.eu/en/publication-detail/-/publication/4cedc9a0-d8cc-11e8-afb3-
01aa75ed71a1/language-en. 
493 Survey for MSCA fellows (IF, ITN, COFUND) conducted for the Study on mobility flows in the 
context of MSCA as part of the for the Evaluation study on Excellent Science in the European 
Framework Programmes for Research and Innovation. Phase 1 (2023). 
494 Survey of Horizon 2020 beneficiary organisations, conducted August-September 2022 under 
Phase 1 of the evaluation study (n=348). 
495 Evaluation for MSCA fellows three months after the end of the fellowship, conducted for the 
Evaluation study on Excellent Science in the European Framework Programmes for Research 
and Innovation. Phase 1 (2023). (Export date: 21 June 2022). RISE n=8,434, ITN n=5,347, IF 
n=1,284, COFUND n=1,544. 
496 European Commission, 'Commission Communication on a European Strategy for 
Universities,” 9. 

302 

 
 
different  sectors  across  Europe  and  beyond, 
train  creative, 
entrepreneurial,  innovative  and  resilient  doctoral  candidates  who  are  able  to  face 
current  and  future  challenges’497.  DN  aim  to  achieve  the  following  outcomes  for 
participating organisations: 

in  order 

to 

- 

Improved  quality,  relevance  and  sustainability  of  doctoral  training 
programmes and supervision arrangements; 

-  Enhanced cooperation and transfer of knowledge between sectors and 

- 

disciplines; 
Increased  integration  of  training  and  research  activities  between 
participating organisations; 

-  Boosted R&I capacity; 
- 
-  Regular  feedback  of  research  results  into  teaching  and  education  at 

Increased internationalisation and attractiveness; 

participating organisations. 

•  Staff  Exchanges  (SE),  which  promote  short-term  international,  inter-sectoral  and 
interdisciplinary exchanges of staff members partaking in innovation activities. The 
main  objective  of  these  is  to  foster  a  shared  culture  of  research  and  innovation 
between  academic  and  non-academic  organisations  through  secondments498.  In 
addition to researchers, the action is open to technical, administrative and managerial 
staff  involved  in  R&I  activities.  SE  aim  to  achieve,  for  example,  the  following 
outcomes: 

-  An increased set of research and transferable skills and competencies, 
leading to improved employability and career prospects both within and 
outside academia; 
Increased  international  exposure,  leading  to  extended  networks  and 
opportunities; 

- 

-  Enhanced  networking  and  communication  capacities  with  scientific 
peers and the general public that will increase and broaden the impact 
of research and innovation. 
Innovative ways to cooperate and transfer knowledge between sectors 
and disciplines; 

- 

-  Strengthened 

and 
interdisciplinary collaborative networks; 

broader 

international, 

inter-sectoral 

and 

-  Boosted R&I capacity. 

•  COFUND, the ‘Co-funding of regional, national and international programmes’, which 
co-finances  new  or  existing  doctoral  and  postdoctoral  programmes.  The  main 
objective  of  COFUND  is  to  disseminate  the  best  practices  of  the  MSCA  on  topics 
such  as  international,  inter-sectoral  and  interdisciplinary  research,  training  and 
mobility499. Participation in COFUND is expected to lead to the following outcomes 
for organisations: 

497 European Commission. Marie Skłodowska-Curie Actions. Work Programme 2021-2022. p. 11 
498 European Commission. Marie Skłodowska-Curie Actions. Work Programme 2021-2022. p. 
32-38 
499 European Commission. Marie Skłodowska-Curie Actions. Work Programme 2021-2022. p. 39 

303 

 
 
-  Enhanced quality and sustainability of research training; 
- 

Increased  global  attractiveness,  visibility  and  reputation 
participating organisation(s); 

for 

the 

-  Stronger R&I capacity and output among the participating organisations; 
-  An  increased  contribution  from  participating  organisations  to  local, 

regional and/or national socio-economic ecosystems; 

-  Regular  feedback  on  research  results  in  teaching  and  education  at 

participating organisations. 

1.2.2.  Overall approach and scope of the case study 

The present case study evaluates questions relating to relevance, coherence, effectiveness, 
efficiency and EU added value at the level of participating institutions within COFUND, DN 
and SE. The case study takes an exploratory approach and aims to identify, through a diverse 
mix  of  methods,  what  relevant  structuring  effects  and  best  practice  scenarios  exist.  The 
expected outcomes, as described above, are grouped into the following four key areas:  

Increasing the quality of researchers’ training and the supervision offered to them; 

• 
•  Strengthening research capacity (e.g. the ability to attract funding); 
• 

Improving human resources practices and procedures and providing fairer and more 
attractive working conditions for researchers, including through career guidance and 
development; and 

•  Building  new  and  sustainable  international  and  inter-sectoral  partnerships  and 
networks,  improving  knowledge  transfer  between  sectors  and  disciplines,  and 
enhancing their global reputation and visibility.  

The  case  study  addresses  how  structuring  impacts  are  created  and  promoted;  how 
structuring  impacts  can  be  implemented  sustainably;  how  the  various  structuring  impacts 
promoted  by  the  MSCA  align  with  the  broader  needs  of  stakeholders;  and  how  actively 
stakeholders seek to implement practices derived from the MSCA in other contexts. In doing 
so,  the  case  study  aims  to  identify  potential  barriers,  good  practices  and  lessons  to  be 
learned. In this respect – and taking into account the still-early stage of projects under Horizon 
Europe – the case study focuses on structuring impacts that have already been realised at 
the institutional, regional and national levels, and the planned impacts of the programme’s 
interventions. 

Table 1 provides a detailed overview of the questions addressed by this case study.  

Evaluation criteria 

Judgement criteria 

Relevance 

• 

The extent to which the (expected) structuring impacts of MSCA 

are relevant to the needs of stakeholders (MSCA beneficiary 

organisations, researchers, and other technical and managerial 

staff). (RV1) 

304 

 
Evaluation criteria 

Judgement criteria 

Coherence 

Effectiveness 

• 

Types of structuring impacts promoted by the programme in the 

four key areas addressed by the study: training and supervision, 

research capacity, human resources and partnerships. (RV3) 

• 

The extent to which the needs of MSCA beneficiary 

organisations, researchers and other technical and managerial 

staff are addressed in terms of available tools that would 

contribute to the programme's four key impact areas/objectives. 

(RV4) 

• 

The extent to which the structuring impacts of MSCA extend 

beyond MSCA projects and are spread/institutionalised within 

beneficiary organisations and national/regional systems. (CH1) 

• 

The existence of complementarities and synergies between the 

structuring impacts of the MSCA and national and regional 

strategies. (CH2) 

• 

The existence of complementarities and synergies between the 

structuring impacts of the MSCA and funding 

programmes/mechanisms at both the EU and national levels. 

(CH2) 

•  Coherence between the four key thematic areas/objectives of 

MSCA addressed by the study and the European Commission’s 

policy priorities (e.g. the European green and digital transitions, 

United Nations Sustainable Development Goals). (CH3) 

• 

The extent to which the MSCA projects have made an impact in 

the four key thematic areas addressed by the study, and how this 

contributes to delivering excellent science. (EFC1) 

• 

Internal and external factors that contributing to progress or 

hinder it. The key drivers and barriers that contribute to the four 

key areas addressed by the study. (EFC2) 

• 

The extent to which MSCA projects contribute to developing the 

skills of researchers and non-research staff, world-class 

infrastructures and facilities, as well as the international and 

cross-sector mobility of researchers and non-research staff. 

(EFC8) 

• 

The extent to which MSCA projects contribute to 

institutional/systemic impacts in the four key areas addressed by 

the study: training and supervision, research capacity, human 

resources, and collaborative networks/partnerships (with a 

particular focus on open and transparent selection and 

305 

 
Evaluation criteria 

Judgement criteria 

recruitment procedures; international, inter-sectoral and 

interdisciplinary cooperation; and the quality of the supervision of 

researchers). ECF9-ECF10) 

•  Qualitative information about the impact and uptake of 

inspirational frameworks and standards (C&C, Research ethics 

and integrity, Open Science, equal opportunities and 

inclusiveness, and facilitating access to at-risk researchers). 

(EFC13-EFC14) 

Efficiency 

• 

The efficiency of (post)doctoral and training programmes in 

expanding the innovative academic setting, and of secondments 

in improving the transfer of international and inter-sectoral staff 

and knowledge. (EFF1) 

•  Drivers and barriers to achieving structuring impacts with regard 

to flexibility, project management, and implementation. (EFF2-

EFF3) 

EU Added Value 

•  What would have happened if the intervention had not existed? 

(EAV1-EAV2) 

Table 1. Evaluation and judgement criteria 

To provide detailed and relevant answers to the aforementioned questions, the case study 
has made use of a wide range of methods, namely: 

•  Desk  research:  this  was  conducted  to  gain  specific  insights  from  existing 
publications on the MSCA, particularly with regard to their structuring effects. Several 
literature  sources  were  considered,  such  as  EUACDE’s  Doctoral  Education  in 
Europe:  Current  development  and  trends,  which  looks  at  the  existing  dynamics  in 
doctoral  and  postdoctoral  education  and  informs  some  of  the  structuring  effects 
originating from the MSCA; the FP7 ex-post and H2020 interim evaluation of Marie 
Skłodowska-Curie  actions  (MSCA),  and  the  Study  of  business  participation  and 
entrepreneurship in Marie Skłodowska-Curie actions (FP7 and Horizon2020), which 
provide information on the effects of the MSCA in several areas, including doctoral 
and postdoctoral training; the ERA Steering Group Human Resources and Mobility 
(ERA SGHRM)’s Report Using the Principles for Innovative Doctoral Training as a 
Tool  for  Guiding  Reforms  of  Doctoral  Education  in  Europe,  which  has  already 
established some of the pathways via which the MSCA deliver structural changes in 
doctoral  education;  and  the Case  Study  on  the  structuring  impact  of  MSCA  ITN  in 
doctoral programmes, carried out during Phase 1 of the evaluation, since this includes 
some preliminary findings on structural impacts stemming from MSCA under Horizon 
2020, to name a few examples.  

306 

 
• 

Interview  programme:  14  interviews  were  conducted  with  representatives  of 
organisations (with the aim of achieving an equal distribution of interviewees across 
the SE, DN and COFUND actions) and National Contact Points. Interviewees were 
identified on the basis of recommendations from the European Commission. Please 
refer to the Appendix 3 for a full list of the interview questions used.  

•  Text  mining  and  analysis  of  unstructured  EC  monitoring  and  administrative 
data:  this  has  been  conducted  to  identify  instances  in  which  the  participation  of 
organisations in the MSCA may have influenced policies, regulations and/or practices 
within participating organisations. To achieve this, the study investigated the relevant 
sections of such reports using a selection of keywords relevant to the scope of each 
case study, and presents the findings together with the submission of the draft final 
deliverables.  

•  Survey  Data:  data  from  surveys  of  both  successful  and  unsuccessful  applicant 
organisations  (DN,  SE,  COFUND)  was  used  to  answer  questions  relating  to  the 
following topics: synergies with regional and local funding schemes; the alignment of 
participation  with  EU-side  strategic  goals;  the  transferable  skills  of  researchers; 
career  progression  for  researchers;  development  and  access  to  physical  and 
knowledge infrastructures, facilities, and data resources; quality of training, scientific 
supervision, recruitment procedures and working conditions of researchers; changes 
to 
internationalisation. 
Comparison  of  the  situations  of  successful  and  unsuccessful  applicants  reveal  the 
impact of the MSCA programme, i.e. situations of unsuccessful applicants reveal a 
counterfactual situation – what would have happened to successful applicants if they 
had not received the MSCA funding. The analysis included in the case study is based 
on  the  following  sample  of  survey  responses  received  from  both  successful  and 
unsuccessful applicant organisations surveys:  

internal  and  external  regulatory 

infrastructures,  and 

–  472  complete  and  67  partial  responses  to  the  survey  of  successful 

applicant organisations (DN, SE, COFUND) 

–  814  complete  and  62  partial  responses  to  the  survey  of  unsuccessful 

applicant organisations (DN, SE, COFUND)     

1.3.  Case study findings 

1.3.1. 

Increasing the quality of the training and supervision offered to researchers. 

Overall, the MSCA are found to favourably increase and improve the quality of researchers’ 
training and supervision. As seen in Table 2, more than 75% of the beneficiaries surveyed 
across the actions considered that their participation in the MSCA had contributed (or was 
expected  to  contribute)  to  a  large  or  a  very  large  extent,  to  improvements  in  terms  of  the 
quality  of  training  opportunities  and  the  quality  of  scientific  supervision  within  their 
organisations. These findings were further confirmed by some interviewees, who considered 
that the structured training provided by the MSCA influenced or led to the development of 
their in-house curricula. 

In relation to this, participation in the MSCA was deemed especially relevant both in terms of 
strengthening the knowledge base and R&D capacities of their organisations, and in 

307 

 
terms of improving and developing new tools, methods or techniques, as expressed by 
a  majority  of  the  beneficiaries  surveyed:  over  75%  of  respondents  across  the  actions 
considered that their participation in the MSCA responded to those specific needs in their 
organisations, as shown in Table 3. 

Doctoral 
Networks 
(n=260) 

Staff 
Exchanges 
(n=92) 

COFUN
D (n=8) 

Quality of training and opportunities for 
continuous professional development 

Quality of scientific supervision 

75% 

75% 

76% 

100% 

79% 

88% 

Table 2. Shares of respondents who considered their Horizon Europe project is expected to lead – at a large or very large 
extent – to improvements at organisational level in the specified areas. Source: survey of Horizon Europe beneficiaries, 
conducted in May-July 2023. Not applicable/unknown and blank responses excluded. 

Doctoral 
Networks 
n=354 

Staff 
Exchange
s n=124 

COFUND 
n=8 

To strengthen the knowledge base and R&D capacity 
in your organisation 

78% 

79% 

75% 

To improve or develop new tools, methods, or 
techniques 

78% 

84% 

50% 

Table 3. Shares of respondents who considered that their Horizon Europe project responded, to a large or very large extent, 
to the specified needs of their organisation. Source: Survey of Horizon Europe beneficiaries, conducted May-July 2023. Not 
applicable/unknown and blank responses excluded. 

In a similar vein, albeit to a lesser extent, most of the beneficiaries surveyed considered that 
their participation in the MSCA had contributed, from moderate to large extent, to improving 
the  skills,  knowledge  and  competences  of  researchers.  Around  72%  of  the  surveyed 
beneficiary  organisations  in  Doctoral  Networks  and  Staff  Exchanges  agreed  with  this 
statement, as shown in Table 4.  

To a very large 
extent 

To a large 
extent 

To a moderate 
extent 

To a small 
extent 

Not at 
all 

Doctoral Networks 
(n=327) 

Staff Exchanges 
(n=115) 

13% 

41% 

31% 

10% 

6% 

15% 

36% 

36% 

10% 

4% 

COFUND (n=8) 

13% 

38% 

25% 

13% 

13% 

Table 4. To what extent, if at all, has your Horizon Europe project achieved/is likely to achieve the following results? 
Improving the skills, knowledge, and competences of researchers. Source: Survey of Horizon Europe beneficiaries, 
conducted May-July 2023. Not applicable/unknown and blank responses excluded. 

308 

 
  
 
 
1.3.2.  Strengthening research capacity (e.g. the ability to attract funding). 

Overall,  participation  in  the  MSCA  is  considered  to  contribute  to  the  participating 
organisations’ ability to attract funding from national and regional sources.  

As shown in Table 5, a large majority of beneficiaries surveyed across the actions considered 
that their participation in the MSCA had contributed to their ability to attract funding from 
national, regional and European sources. This was further confirmed by interviews with 
both  Horizon  Europe  beneficiaries  and  with  National  Contact  Points,  who  regarded 
participation in the MSCA as particularly useful for making participating organisations more 
competitive  and  able  to  attract  funding  from  external  sources.  This  structuring  effect  was 
particularly important for small and recently created organisations wanting to increase their 
collaborations in Europe, as well as for organisations from widening countries trying to make 
their institutions as competitive as their counterparts in non-widening ones.  

Doctoral 
Networks 
(n=340) 

Staff 
Exchanges 
(n=122) 

COFUND 
(n=8) 

Development of policymaking and standards 
setting measures 

86% 

86% 

63% 

Strengthened capacity to attract public or 
private funding 

97% 

94% 

100% 

Table 5. Shares of respondents who considered that their Horizon Europe project had achieved/is likely to achieve the 
specified results to a large or very large extent. Source: Survey of Horizon Europe beneficiaries, conducted May-July 2023. 
Not applicable/unknown and blank responses excluded. 

Similarly,  participation  in  the  MSCA  appears  to  be  relevant  in  terms  of  enhancing  the 
international  visibility  of  organisations.  Table  6  shows  that  more  than  70%  of  both 
beneficiaries  and  unsuccessful  applicants  across  the  actions  regarded  the  MSCA  as 
responding to their organisation’s need to increase its visibility  – which, according to most 
interviewees, represents an important factor in strengthening their research capacities. It is 
important to note that unsuccessful applicants were asked about their general beliefs related 
to the MSCA, since they have not received funding. 

To enhance international visibility 
(across the borders of your country) 

To enhance international visibility 
(across the borders of your country) 
[Unsuccessful applicants] 

Doctoral 
Networks 
(n=354) 

86% 

70% 

Staff Exchanges 
(n=125) 

COFUND 
(n=8) 

87% 

100% 

71% 

80% 

Table 6. Shares of respondents who considered that their Horizon Europe project had responded, to a large or very large 
extent, to the specified needs of their organisation. Source: Survey of Horizon Europe beneficiaries, conducted May-July 
2023; Survey of Horizon Europe unsuccessful applicants, conducted May-July 2023. Not applicable/unknown and blank 
responses excluded. 

309 

 
 
 
1.3.3. 

Improving human resources practices and procedures, and providing fairer 
and more attractive working conditions for researchers, including through 
career guidance and development 

The case study has also found that the MSCA have positive structuring effects in terms of 
improving human resources practices and improving working conditions for researchers.  

As shown in Table 7, the majority of DN and COFUND respondents believe their participation 
in  Horizon  Europe  projects  has  contributed,  to  a  large  or  very  large  extent,  in  achieving 
improvements in terms of the working conditions of researchers and the better provision 
of career advice. In terms of the quality and transparency of recruitment practices, the 
trend is similar, although slightly lower among SE participants.  

Doctoral 
Networks 
(n=229) 

Staff 
Exchanges 
(n=68) 

COFUND 
(n=8) 

Quality and transparency of recruitment practices  

57% 

48% 

100% 

Working conditions of researchers 

51% 

53% 

100% 

Provision of career advice for researchers 

65% 

60% 

88% 

Table 7. Share of respondents who considered their Horizon Europe project is expected to lead to improvements at the 
organisational level, to a large or very large extent, in the specified areas. Source: Survey of Horizon Europe beneficiaries, 
conducted May-July 2023. Not applicable/unknown and blank responses excluded. 

The above finding is in line with those from the interviews conducted. A significant number of 
respondents considered that one of the most tangible structuring effects resulting from their 
participation  in  the  MSCA  was  an  improvement  in  the  working  conditions  of  researchers. 
They pointed to measures such as the offering of job contracts or the necessity to improve 
salary  conditions  as  crucial  aspects  of  their  participation  in  the  MSCA  that  resulted  in  the 
enhanced competitiveness of their organisations. 

According  to  the  survey,  the  consensus  regarding  the  employment  conditions  of  other 
employees is rather different from those regarding other structuring effects: more than 50% 
of DN and SE respondents believed that their participation in the MSCA has had a moderate-
to-large  impact  on  the  working  conditions  of  non-researchers,  as  shown  in  Figure  1, 
compared  with  a  higher  percentage  who  considered  these  impacts  to  be  larger  for 
researchers. 

310 

 
 
All actions (n=418)

11%

22%

COFUND n=8

43%

32%

14%

19%

17%

43%

Staff Exchangess n=124

11%

28%

28%

18%

14%

Doctoral Networks n=352

10%

20%

33%

19%

17%

0%

20%

40%

60%

80%

100%

120%

To a very large extent

To a large extent

To some extent

To a limited extent

Not at all

Figure 1. To what extent has your Horizon Europe project responded to the following needs of your organisation? Improving 
working conditions of other employees. Source: Survey of Horizon Europe beneficiaries, conducted May-July 2023. Not 
applicable/unknown and blank responses excluded. Results for COFUND are influenced by the low number of responses. 

However, participation in the MSCA is regarded as a positive development in terms of career 
progression: 92% of DN respondents and 86% of SE beneficiaries surveyed consider that 
their  participation  in  Horizon  Europe  is  likely  to  improve  the  career  prospects  of 
researchers. 

To a very large 
extent 

To a large 
extent 

To a moderate 
extent 

To a small 
extent 

Not at 
all 

Doctoral Networks 
(n=343) 

Staff Exchanges 
(n=118) 

59% 

33% 

6% 

1% 

1% 

47% 

39% 

12% 

2% 

0% 

COFUND (n=8) 

100% 

0% 

0% 

0% 

0% 

Table 8. To what extent, if at all, has your Horizon Europe project achieved/is likely to achieve the following results? 
Improving the career prospects of researchers. Source: Survey of Horizon Europe beneficiaries, conducted May-July 2023. 
Not applicable/unknown and blank responses excluded. Results for COFUND are influenced by the low number of 
responses 

Interviews  with  beneficiaries  show  several  reasons  behind  this,  such  as  the  need  for 
researchers to seek secondments, spending part of their research abroad and in industry as 
part of their doctoral or postdoctoral education, and increasing the quality of training in several 
areas.  All  of  these  aspects  were  identified  by  interviewees  as  important  factors  that  can 
improve the career progression of participating researchers.  

In this respect, the MSCA are regarded as being relevant to strengthening the application of 
Responsible Research and Innovation principles within beneficiaries’ organisations: 63% of 
SE  respondents  considered  that  their  participation  in  the  MSCA  addressed  their 
organisation’s needs to strengthen its work in this area to a very large or large extent, while 
65% of DN beneficiaries surveyed considered that it did so to a moderate to large extent. 
Evidence  from  interviews  backs  up  this  trend,  albeit  to  a  moderate  extent,  since  many 
respondents  regarded  the  Responsible  Research  and  Innovation  principles  as  existing 

311 

 
 
 
standards for the overall delivery of excellence in Europe and were thus unable to identify 
the extent to which MSCA participation had influenced their adoption, although it certainly 
accelerated it.  

Doctoral Networks 
(n=300) 

Staff Exchanges 
(n=117) 

To a very 
large 
extent 

To a large 
extent 

To a 
moderate 
extent 

To a small 
extent 

Not at all 

13% 

34% 

31% 

11% 

11% 

18% 

45% 

24% 

9% 

4% 

0% 

COFUND (n=8) 

0% 

63% 

25% 

13% 

Table 9. To what extent has the Horizon Europe project responded to the following needs of your organisation? To 
strengthen the application of Responsible Research and Innovation principles in your organisation. Source: Survey of 
Horizon Europe beneficiaries, conducted May-July 2023. Not applicable/unknown and blank responses excluded. 

A  similar  trend  can  be  seen  in  terms  of  support  for  policymaking  and  the  development  of 
standard-setting  measures.  In  total,  64%  of  SE  and  55%  of  DN  beneficiaries  surveyed 
considered that their participation in the MSCA had contributed to a moderate to large extent 
to  the  establishment  of  policies  and  the  standardisation  of  procedures  within  their 
organisations. Interview evidence, on the other hand, suggests that the impact of the MSCA, 
while targeted and limited, may be tangible in terms of the policies and standards adopted by 
participating  organisations.  An  example  mentioned  by  interviewees  is  the  knowledge  and 
procedures  used  to  ease  the  entry  of  incoming  foreign  staff  into  their  institutions,  which 
entailed the creation of standardised institutional procedures to accompany and follow the 
visa applications of incoming foreign researchers.  

All actions (n=370)

11%

27%

32%

15%

14%

COFUND (n=6)

33%

17%

17%

33%

Staff Exchanges (n=106)

16%

30%

34%

13%

7%

Doctoral Networks (n=258)

9%

26%

31%

16%

17%

0%

20%

40%

60%

80%

100%

120%

To a very large extent

To a large extent

To a moderate extent

To a small extent

Not at all

Figure 2. To what extent has your Horizon Europe project responded to the following needs of your organisation? To support 
policy-making or the development of standards-setting measures. Source: Survey of Horizon Europe beneficiaries, 
conducted May-July 2023. Not applicable/unknown and blank responses excluded. 

312 

 
 
 
1.3.4.  Building new sustainable international and inter-sectoral partnerships and 
networks, better knowledge transfer between sectors and disciplines, and 
enhancing their global reputation and visibility   

Study  findings  show  that  participation  in  the  MSCA  helps  to  strengthen  collaboration  with 
leading  global  institutions,  and  has  the  positive  effect  of  enhancing  the  global  visibility  of 
participating institutions.  

As shown in Table 10, more than 88% of beneficiary organisations, regard participation in 
the MSCA as responding, to a large or very large extent, to their organisations’ capacity to 
create  and  strengthen  collaboration  with  leading  research  organisations.  Evidence 
from interviews with National Contact Points and beneficiaries backs up these findings, since 
many respondents regarded the MSCA as strengthening collaboration with leading European 
and global research institutions, thanks to its role as an indicator of excellence.  

To a very large 
extent 

To a large 
extent 

To a moderate 
extent 

To a small 
extent 

Not at 
all 

49% 

39% 

10% 

2% 

1% 

Doctoral Networks 
(n=356) 

Staff Exchanges 
(n=124) 

COFUND (n=8) 

50% 

50% 

51% 

40% 

6% 

0% 

3% 

0% 

0% 

0% 

Table 10. To what extent has your Horizon Europe project responded to the following needs of your organisation? To create 
or strengthen collaboration with leading research organisations. Source: Survey of Horizon Europe beneficiaries, conducted 
May-July 2023; Not applicable/unknown and blank responses excluded. Results for COFUND are influenced by the low 
number of responses. 

In terms of collaboration with industry partners, Table 11 shows that the trend is similar: a 
majority  of  beneficiaries  consider  that  participation  in  the  MSCA  has  helped  their 
organisations, to a large or very large extent, to  strengthen collaborations with leading 
industry partners.  

However, interview evidence also showed that collaboration with industry partners involved 
more  challenges  than  collaboration  with  research  organisations.  Here,  respondents  noted 
that  cooperation  with  industry  and  business  was  highly  reliant  on  both  the  subject  of  their 
research and the size of the industry partner, meaning that larger companies were deemed 
more  capable  of  collaborating  than  SMEs.  In  addition  to  this,  cooperation  with  industry 
partners often involved significant challenges, such as complications regarding the ownership 
of the intellectual property generated. Despite these challenges, some respondents indicated 
that  collaboration  with  the  private  sector  has  produced  tangible  outputs  in  relation  to 
generating  impacts  for  end  users,  such  as  the  creation  of  spin-offs  that  bring  research 
outputs into the market.  

313 

 
 
To a very large 
extent 

To a large 
extent 

To a moderate 
extent 

To a small 
extent 

Not at 
all 

Doctoral Networks 
(n=341) 

Staff Exchanges 
(n=113) 

26% 

35% 

26% 

11% 

3% 

19% 

35% 

27% 

12% 

6% 

COFUND (n=8) 

50% 

25% 

25% 

0% 

0% 

Table 11. To what extent has your Horizon Europe project responded to the following needs of your organisation? To create 
or strengthen collaboration with leading industry organisations. Source: Survey of Horizon Europe beneficiaries, conducted 
May-July 2023. Not applicable/unknown and blank responses excluded. Results for COFUND are influenced by the low 
number of responses 

In  terms  of  international  visibility  and  collaboration  with  global  partners,  a  majority  of  SE 
respondents considered that their participation in the MSCA contributed to increasing their 
international  visibility  through  collaboration  with  global  partners,  while  a  majority  of  DN 
respondents considered this impact to be small-to-moderate. 

To a very large 
extent 

To a large 
extent 

To a moderate 
extent 

To a small 
extent 

Not at 
all 

Doctoral Networks 
(n=266) 

Staff Exchanges 
(n=105) 

9% 

22% 

28% 

26% 

15% 

13% 

32% 

34% 

15% 

5% 

COFUND (n=6) 

0% 

0% 

17% 

33% 

50% 

Table 12. To what extent, if at all, has your Horizon Europe project achieved/is likely to achieve the following results? 
Increasing international visibility through collaboration with leading global partners. Source: Survey of Horizon Europe 
beneficiaries, conducted May-July 2023. Not applicable/unknown and blank responses are excluded. 

1.4.  Reflections on evaluation criteria 

1.4.1.  Relevance 

As demonstrated above, participation in the MSCA is regarded as crucial to increasing the 
attractiveness of research-intensive institutions in their journey to create excellent scientific 
outputs.  As shown  by  Table  4  and  Table  7,  participation  in  the  MSCA  responds  to  a  very 
large or large extent to the needs of participants in terms of strengthening the knowledge 
base and R&D capacity within their organisations; improving or developing new tools, 
methods  or  techniques;  and  enhancing  international  visibility.  Such  effects  are  also 
observed,  albeit  to  a  more  moderate  extent,  in  relation  to  strengthening  the  application  of 
Responsible  Research  and  Innovation  principles,  and  in  supporting  policy-making  or  the 
development of standards-setting measures. 

314 

 
 
 
Similarly,  Interview  evidence  shows  that  the  desired  structuring  impacts  of  the  MSCA  are 
indeed relevant to the overall delivery of Excellent Science in the EU, particularly in terms of 
international  cooperation,  cooperation  with  the  private  sector,  and  research  excellence. 
These impacts are particularly relevant in the case of widening countries, as well as small 
and  recently  created  organisations,  as  they  help  to  narrow  the  gap  between  such 
organisations and the research infrastructure of non-widening countries by levelling up the 
field for all researchers in the EU through equal and standardised rules.  

In this respect, interviewees regarded the materials, guides, and rules provided and available 
online as relevant and sufficient to inform potential applicants. However, due to their wide 
variety and heterogeneity, the documents concerned were deemed cumbersome and difficult 
to navigate, indicating the need to condense all available information into one main source 
or document to facilitate access.  

Similarly, interviews with beneficiaries revealed that the MSCA play a crucial role in signifying 
excellence, which in turn pushes institutions to refine new and upgrade existing doctoral and 
postdoctoral  programmes,  establish  industry  collaborations,  and  contribute  to  the  better 
career progression of participating researchers.  

1.4.2.  Coherence 

Evidence stemming from the interviews suggests that the MSCA have structuring impacts 
that  go  beyond  MSCA  projects  and  are  spread/institutionalised  into  beneficiary 
organisations  and  national/regional  systems.  This  is  particularly  evident  in  relation  to 
gender equality policies, which are becoming institutionalised as a result of their participation 
in MSCA actions. One interviewee, when asked ‘Can you provide an example of an aspect 
that was overlooked before MSCA and was later addressed?’, commented: ‘One important 
aspect as mentioned is gender, especially in engineering. MSCA prompted us to focus on 
this issue and provide training to understand gender differences in engineering. We bought 
and watched documentaries and discussed it together, benefitting both the students and the 
supervisors.  The  documentaries  are  now  accessible  to  the  whole  organisation,  beyond 
MSCA.  This  expanded  our  knowledge  and  made  us  explore  aspects  we  might  have 
overlooked before.’ 

Several interviewees also mentioned that certain national funding programmes/schemes had 
taken stock of the MSCA evaluation process by transposing certain criteria and high-quality 
standards when evaluating applications.  

1.4.3.  Efficiency 

As  shown  in  Figure  3,  most  survey  respondents  considered  that  their  participation  in  the 
MSCA has responded, to a moderate or very large extent, to their organisation’s needs to 
promote institutional change at an organisational, regional or national level. Evidence from 
interviews  showed  a  similar  trend,  although  with  some  important  nuances.  In  terms  of 
administrative  or  organisational  practices,  participation  in  the  MSCA  only  resulted  in 
significant  effects  where  administrative  procedures  had  not  already  been  influenced  by 
national  legislation  or  the  institutional  environment.  Such  effects  were  therefore  easily 
identifiable in widening countries or in institutions that were relatively new to participation in 
their country’s academic environment. 

315 

 
 
 
All actions (n=397)

11%

27%

28%

18%

16%

COFUND (n=8)

13%

50%

25%

13%

Staff Exchanges (n=110)

15%

36%

24%

15%

9%

Doctoral Networks (n=279)

9%

23%

30%

19%

19%

0%

20%

40%

60%

80%

100%

120%

To a very large extent

To a large extent

To a moderate extent

To a small extent

Not at all

Figure 3. To what extent has your Horizon Europe project responded to the following needs of your organisation? To 
promote institutional change in your organisation, region or country. Source: Survey of Horizon Europe beneficiaries, 
conducted May-July 2023. Not applicable/unknown and blank responses excluded. 

For well-established institutions, the MSCA did not bring about a revolutionary shift in their 
existing practices. Instead, it facilitated the implementation of anticipated positive changes, 
such as streamlining hiring procedures and adopting improved salary structures. 

1.4.4.  Effectiveness 

The  MSCA  are  very  effective  in  improving  the  skills,  knowledge  and  competences  of 
researchers;  developing  policymaking  and  standards-setting  measures;  and  strengthening 
organisations’ capacities to attract public or private funding. In addition to these, the MSCA 
were  also  found  to  be  effective  in  improving  the  quality  and  transparency  of  recruitment 
practices, the working conditions of researchers, and the provision of career advice for them.  

Evidence from interviews also backed up these findings: most interviewees considered the 
MSCA  as  promoting  an  overall  improvement  in  the  delivery  of  Excellent  Science  in  their 
institution  by  providing  necessary  tools  that  had  increased  the  competitiveness  both  of 
participating organisations and of participating individuals. In this regard, most respondents 
found several degrees of benefit depending on the country of participation, the institutional 
capacity of the participating organisations, and the type of action.  

1.4.5.  EU Added Value 

A  majority  of  unsuccessful  applicants  across  the  actions  considered  the  application  to  the 
MSCA as being useful in promoting the Responsible Research and Innovation principles in 
their organisations to a moderate-to-large extent. Interview evidence also underscores that 
participation in the MSCA is beneficial to the adoption of such principles, and they can be 
regarded as part of an important standard for the overall delivery of Excellent Science. It is 
difficult  to  conclude,  though,  whether  a  lack  of  participation  in  the  MSCA  would  make  the 
adoption of such principles difficult or impossible to achieve. 

316 

 
 
 
To a very large 
extent 

To a large 
extent 

To a moderate 
extent 

To a small 
extent 

Not at 
all 

Doctoral Networks 
(n=300) 

Staff Exchanges 
(n=117) 

COFUND (n=8) 

6% 

9% 

0% 

24% 

32% 

24% 

14% 

22% 

47% 

13% 

9% 

38% 

38% 

25% 

0% 

Table 13. To what extent has application to Horizon Europe responded to the following needs of your organisation? To 
strengthen the application of Responsible Research and Innovation principles in your organisation. Source: Survey of 
Horizon Europe unsuccessful applicants, conducted May-July 2023. Not applicable/unknown and blank responses excluded. 

A similar trend can be observed both in terms of supporting policy-making or the development 
of  standards-setting  measures,  and  in  terms  of  developing  new  tools,  methods  and 
techniques. A majority of respondents identified the MSCA as making a moderate-to-large 
contribution to these areas; however, interviewees noted that while participation in the MSCA 
can  be  regarded  as  a  facilitating  factor in  both  of  these  respects,  it  is  difficult  to  establish 
whether  or  not  the  MSCA  are  a  determining  factor,  as  both  of  these  aspects  are  actively 
pursued by participating organisations themselves.  

In  addition  to  this,  interviewees  deemed  the  MSCA  to  be  good  sources  of  added  value  in 
several  respects,  particularly  with  regard  to  collaboration  with  the  private  sector, 
improvements  in  the  career  progression  of  participating  researchers,  and  ensuring  the 
production of excellence in participating institutions. Several examples were cited to back this 
up, such as mandating the adoption of best practices in all participating institutions, which in 
turn  improved  their  competitiveness;  the  adoption  of  better  career  practices,  which  made 
institutions and countries more attractive to better researchers; and the procedural benefits 
provided by MSCA applications, which in turn made institutions better able to attract external 
funding. 

An overwhelming majority of respondents consider MSCA participation as influencing, to a 
large or very large extent, their capacity to attract funding, due to the contributions they 
make  to  developing  policy-making  and  standards-setting  measures  and  their  ability  to 
enhance the international visibility of their organisations. Both of these were identified by 
the beneficiaries interviewed as being necessary to increasing their ability to attract external 
funding. This is backed up by interview evidence: in general, participation in the MSCA was 
found  to  increase  the  participating  organisations’  abilities  to  attract  funding  from  other 
sources, especially within the EU. This effect is particularly relevant to widening countries, as 
the adoption of the MSCA rules and procedures prepares institutions for more competitive 
rounds of funding against institutions from non-widening countries. 

1.5.  Perceived success factors  

1.5.1.  Clear guidelines and support materials 

Evidence  stemming  from  the  interviews  suggests  that  the  MSCA  provide  clear guidelines, 
work programmes and an applicant’s guide tailored to each action, ensuring that researchers 
and institutions understand the programme’s expectations and adhere to its requirements. 

317 

 
 
 
1.5.2.  Strong focus on ensuring adequate supervision and mentoring 

The MSCA actions’ emphasis on the quality of supervision is considered another success 
factor in increasing the excellence of research in beneficiary organisations. The MSCA thus 
enable  a  shift  in  recognising  that  good  supervision  is  instrumental  to  ensuring  early-stage 
and  experienced  researchers  receive  the  guidance,  support  and  mentorship  they  need  to 
excel in their projects and in further employment.  

1.5.3.  Critical administrative support for the participating organisations 

Interviewees pointed out that the main drivers for achieving structuring impacts through the 
MSCA programme are the administrative offices of participating organisations. Researchers 
participating in MSCA actions may come and go, or move between projects and institutions; 
however, there is typically continuity with regard to administrative offices, thus enabling and 
facilitating the transfer of good practices that go beyond participation in the MSCA.  

1.6.  Perceived barriers 

Below the case study indicates areas, where MSCA structuring effects are weaker than they 
could potentially be due to the indicated barriers. 

1.6.1.  Transferable skills  

Findings from the interviews indicate that institutions face challenges in improving the quality 
of training for researchers, particularly in areas such as intellectual property rights, project 
management and entrepreneurship. 

1.6.2.  Human resources and working conditions  

While the MSCA promote mobility, interviewees noted that third-country researchers often 
face  challenges  with  regard to  visa  processes, cultural  adaptation  and  balancing  personal 
commitments.  This  mobility  is  juxtaposed  against  the  career  uncertainties  of  temporary 
positions  and  limited  advancement  opportunities,  particularly  in  certain  regions.  Another 
systemic challenge relates to the fact that in some countries such as Ireland, Ph.D. students 
cannot be legally granted full-time employment contracts, leading to various complications 
that can only be dealt with by enacting changes to national labour law.  

1.6.3.  Building sustainable collaborations and knowledge transfer 

Collaborations  between  the  academic  and  non-academic  sectors  present  additional 
challenges. One of the primary concerns expressed by most interviewees is the alignment of 
academic research goals with the often short-term and profit-driven objectives of the private 
sector. Intellectual property rights (IPR) represent a significant point of contention, with both 
parties needing clarity with regard to ownership, knowledge sharing and commercialisation 
rights. Small and medium-sized enterprises (SMEs) might find the complexity of the MSCA 
scheme daunting, thus deterring them from participation due to a lack of familiarity with such 
academic-industry  collaborations.  Issues  relating  to  knowledge  security  –  especially  when 
dealing  with  sensitive  technologies  –  require  strict  protocols  and  agreements,  which  can 
sometimes hinder the free flow of information and collaboration. 

318 

 
1.6.4.  Open access  

Open  Access  (OA)  publishing,  while  heralded  as  a  move  towards  the  democratisation  of 
scientific knowledge, presents several challenges that were expressed by interviewees. One 
of the foremost concerns expressed was the high costs associated with OA publishing, which 
some believe are counterintuitive to the very ethos of accessibility that OA aims to achieve. 
In addition, many OA journals suffer from a low Impact Factor (IF), a metric often used to 
gauge the significance of a publication, thereby potentially undermining the perceived value 
of a researcher's work. Some criticism was also expressed with regard to OA repositories, 
which often lack any peer-review process altogether, raising concerns about the accuracy 
and validity of data. 

1.7.  Conclusions and lessons learned across the evaluation criteria of 

interest for the case study 

The present case study finds that participation in the MSCA responds, to a very large or large 
extent, to the needs of participants in terms of strengthening the knowledge base and R&D 
capacity of their organisations; improving or developing new tools, methods or techniques; 
and  enhancing  their  international  visibility. Positive  effects  were  also  observed,  albeit  to  a 
more moderate extent, in terms of strengthening the application of the Responsible Research 
and Innovation principles, and in supporting policy-making or the development of standards-
setting measures. 

These above impacts were noted as being particularly relevant to widening countries, and to 
small and recently created organisations, since they help to narrow the gap between such 
countries and organisations and the research infrastructures of non-widening countries, by 
levelling the field for all researchers in the EU through equal and standardised rules. 

The  MSCA  display  considerable  coherence  with  other  EU  funding  initiatives,  notably  by 
enhancing organisations’ potential to secure external funding, both regionally and nationally. 
The  findings  of  the  survey  carried  out  for  the  present  study  indicate  that  the  majority  of 
respondents  recognise  the  profound  influence  of  the  MSCA  in  terms  of  bolstering  their 
capabilities to acquire funding. This finding resonates profoundly with widening countries, for 
whom the standardised practices of the MSCA level the playing field in comparison to more 
established institutions in non-widening countries.  

Findings  from  the  interviews  suggest  that  the  level  of  coherence  of  the  MSCA  varies  per 
action. For instance, Staff Exchanges appear to align with ERA Talents and, to some extent, 
Erasmus. However, alignment between ERA Talents and MSCA SE is subject to discussion 
as ERA Talents are arguably overlapping (and not really complementing) MSCA SE. MSCA 
SE  feature  the  same  pronounced  intersectoral  dimension  (a  requirement  of  the  action) as 
ERA Talents. The meaningful difference between the two, however, is that ERA Talents are 
directly targeted at widening countries. Having said this, analysis of the most recent SE (and 
previously, RISE) calls shows that the participation of widening countries is high. 

COFUND  achieves  synergies  with  cohesion  policy  funds,  which  reinforce  the  structuring 
effects of the MSCA. 27 funding schemes are currently in place across 15 EU Member States 
for  MSCA  Seal  of  Excellence  holders  who  applied  for  Postdoctoral  Fellowships  and 
COFUND.  Out  of  all  schemes,  12  of  them  are  funded  at  national  level,  six  by  regional 
authorities and nine at institutional level (mostly by universities, but also private entities). Five 
of these schemes are combining national or regional funding with cohesion funding (ERDF) 
and six are making use of Recovery and Resilience Facility funding to support MSCA Seal of 
Excellence  holders.  National,  regional,  or  institutional  funding  schemes  are  available  in 

319 

 
Austria,  Belgium  (Flanders),  Bulgaria,  Czechia,  Cyprus,  France,  Italy,  Lithuania,  Poland, 
Romania, Slovakia, Slovenia, Spain, and Sweden. 

Importantly, the MSCA has shown synergies with national/regional-level initiatives through 
the  setting  up  of  ‘Seal  of  Excellence’  schemes,  which  are  especially  valued  by  widening 
countries for their role in spawning new funding opportunities.  

Beyond  individual  projects,  the  MSCA  have  had  a  broader  structuring  impact,  influencing 
both beneficiary organisations and wider national/regional frameworks. A prime example of 
this  is  the  institutionalisation  of  gender  equality  policies  due  to  MSCA  involvement,  as 
mentioned by several interviewees. Furthermore, the rigour of the MSCA evaluation process 
has been adopted by certain national programmes, thereby setting high-quality benchmarks 
for application assessments. 

The  MSCA  were  found  to  be  very  effective  in  improving  the  skills,  knowledge  and 
competences  of  researchers,  as  well  as  developing  policy-making  and  standards-setting 
measures.  In  addition  to  this,  the  MSCA  were  also  found  to  be  effective  in  improving  the 
quality and transparency of recruitment practices, the working conditions of researchers, and 
the provision of career advice to them. 

The findings of the present study also suggest that the MSCA has been a flexible instrument, 
able  to  contribute  on  the  basis  of  specific  institutional  needs.  Specifically,  58%  of  SE 
applicants  surveyed  deemed  the  MSCA  as  having  a  moderate-to-high  value  in  spurring 
institutional change, while 54% of DN respondents saw this effect as mild-to-moderate. The 
impact  of  the  MSCA  was  more  significant  when  seen  from  the  perspective  of  beneficiary 
organisations, with the majority of participants in SE (75%) and DN (62%) actions agreeing, 
from a moderate to a very large extent, that the MSCA responded to the need to promote 
institutional change within their organisations.  

Interview findings also highlight the unique added value of the MSCA, notably in fostering 
collaboration with the private sector, advancing researcher career trajectories, and promoting 
excellence in participant institutions. This is exemplified by MSCA mandate for institutions to 
adopt  best  practices,  thus  enhancing  their  competitiveness,  as  well  as  the  introduction  of 
career development frameworks and the attraction of excellent researchers. 

However,  despite  impacts  being  visible,  participating  organisations  still  face  challenges  in 
certain areas where the MSCA structuring effects could be stronger if these challenges were 
removed. For example, institutions face difficulties in equipping researchers with transferable 
skills such as in intellectual property rights and entrepreneurship. Furthermore, third-country 
researchers, while benefitting from the focus of the MSCA on mobility, often grapple with visa 
issues,  cultural  adjustments  and  career  uncertainties.  For  instance,  in  countries  such  as 
Ireland, legal barriers exist to providing full-time contracts to Ph.D. students. Collaborations 
between the academic and non-academic sectors also give rise to challenges, especially with 
regard  to  aligning  research  goals  with  the  private  sector's  often  short-term  objectives. 
Intellectual property rights are a notable pain point, and SMEs are sometimes daunted by the 
complexity  of  the  MSCA.  Lastly,  while  Open  Access  promotes  wider  knowledge 
dissemination,  the  high  costs  and  sometimes  lower  perceived  value  of  research  in  OA 
journals are causes for concern.  

320 

 
 
Appendix 1. Literature Sources 

Case study on ‘structuring impact of MSCA ITN on doctoral programmes’ 

ERA Steering Group for Human Resources and Mobility, ‘Using the Principles for 
Innovative Doctoral Training as a Tool for Guiding Reforms of Doctoral Education in 
Europe: Report of the ERA Steering Group Human Resources and Mobility’, 2014, 
available at: 
https://euraxess.ec.europa.eu/sites/default/files/policy_library/sghrm_idtp_report_final.pdf. 

European Commission, Directorate-General for Research and Innovation, The European 
Charter for Researchers – The code of conduct for the recruitment of researchers, 
Publications Office, 2006. 

European Commission, (2023Evaluation study on Excellent Science in the European 
Framework Programmes for Research and Innovation, Annexes. (Case study on 
‘structuring impact of MSCA ITN on doctoral programmes’)  

European Commission, Commission Communication on a European Strategy for 
Universities, 2022. 

European Commission. Directorate General for Education, Youth, Sport and Culture. FP7 
Ex Post and H2020 Interim Evaluation of Marie Skłodowska-Curie Actions (MSCA): Final 
Report. Luxembourg: Publications Office of the European Union, 2017, available at: 
https://data.europa.eu/doi/10.2766/103174. 

European Commission, Directorate General for Education, Youth, Sport and Culture, 
Interim Evaluation of Horizon 2020: Commission Staff Working Document: Annex 2, 2017, 
available at: https://op.europa.eu/en/publication-detail/-/publication/4cedc9a0-d8cc-11e8-
afb3-01aa75ed71a1/language-en. 

European Commission, Directorate General for Research and Innovation. and Centre for 
Strategy & Evaluation Services LLP (CSES), Towards a 2030 Vision on the Future of 
Universities in Europe, Luxembourg: Publications Office of the European Union, 2020. 
https://data.europa.eu/doi/10.2777/510530. 

Hasgall, A., & Peneoasu, A.-M., Doctoral Education in Europe: Current Developments and 
Trends, 2022. 

Lambert, W. Marcus, Nana, N., Afonja, A., Saeed, A., Amado, A.C., & Golightly, L.M., 
Addressing Structural Mentoring Barriers in Postdoctoral Training: A Qualitative Study, 
preprint, Scientific Communication and Education, August 23, 2022, available at: 
https://doi.org/10.1101/2022.08.20.504665. 

European Commission, Directorate-General for Education, Youth, Sport and Culture, Marie 
Skłodowska-Curie actions guidelines on supervision, Luxembourg: Publications Office of 
the European Union, 2021, available at: https://data.europa.eu/doi/10.2766/508311  

Sabic, N., Comparative Analysis of Doctoral Education in Europe, Analele Universităţii Din 
Bucureşti. Seria Ştiinţe Politice, 16, no. 1 (2014): 129–47. 

Steen, K., Vornhagen, J., Weinberg, Z.Y., Boulanger-Bertolus, J., Rao, A., Evans Gardner, 
M., & Subramanian, S., A Structured Professional Development Curriculum for Postdoctoral 

321 

 
Fellows Leads to Recognized Knowledge Growth, S. Safayi. (ed.) PLOS ONE 16, no. 11 
(November 22, 2021): e0260212, https://doi.org/10.1371/journal.pone.0260212. 

Appendix 2. List of Interviews (anonymised) 

3 participants in MSCA-COFUND 

7 NCPs 

2 participants in MSCA-SE 

2 participants in MSCA-DN 

Apendix 3. List of interview questions 

Evaluation 
criteria 

Relevance 

Interview questions 

•  How relevant do you think these structuring impacts are to facilitating the 

• 

• 

• 

delivery of Excellent Science in the EU? 
To what extent does participation in the MSCA project contribute to 
developing (post)doctoral programmes in general in your institution? (DN, 
COFUND)  
To what extent does participation in the MSCA contribute to the 
development of new skills and the improvement of international and inter-
sectoral partnerships and networks? (DN, COFUND, SE) 
In your view, what structuring impact(s) is participation in the MSCA having / 
will have on (post)doctoral programmes/knowledge and staff exchange in 
your institution? 

•  Could you describe these effects in detail, compared with other practices 

• 

within your organisation? 
In your view, do you think these effects fit well with the needs of your 
organisations and partners? 

Coherence 

•  What role do you think your participation in the MSCA would play when 

applying to other (EU, national, regional) funding sources? 

•  Has your participation in the MSCA facilitated access to other funding 

sources? If so, which ones and how? 

•  What has driven the adoption of structural changes since your MSCA 

participation? Has this process involved obstacles or catalysers? If so, what 
were these? 

•  How would you assess the external coherence of SE/DN/COFUND? (with 

other institutional structures and EU funding programmes)? 

322 

 
 
 
 
Evaluation 
criteria 

Effectiveness 

Interview questions 

•  What changes have been introduced due to participating in the MSCA 

project (e.g. in relation to the training offered, career development, mobility, 
collaboration with the non-academic sector, supervision, working conditions 
and salary/stipend structure, application and selection process, etc.)? 

•  Could you describe how these changes have been introduced into your 

organisation? 

•  How effective is the participation in the MSCA project in: 

-  Raising excellence in your institution with regard to research output 

and attracting new talent/funding? 

-  Extending the academic research setting to the non-academic 

sector? 

-  Enlarging international partnerships with other organisations 
- 
Improving Open Science practices in your institution? 
- 
Increasing training in transferable competencies for non-MSCA 
fellows? 

-  Ensuring adequate supervision? 

•  Would you consider that the participation in MSCA project has led/leads to 
improvements in the management and administrative procedures practised 
within your organisation/department? If so, in what ways? 
To what extent did the participation of other institutions contribute to 
developing (post)doctoral programmes? What were the most significant 
factors contributing to this development, and what were the barriers to it?  

• 

Efficiency 

•  How would you assess the management and administrative procedures 

practised within your institution/department in ensuring that the MSCA 
project (DN, SE, COFUND) adheres to the criteria set? 

•  How would you assess other existing management and administrative 
procedures and their contributions to ensuring that the lessons learned 
and best practices from DN/SE/COFUND action are transferred to the 
structuring of doctoral programmes in general? 

•  Would you have any suggestions that could be introduced to improve the 

structuring impacts of MSCA actions (DN, SE, COFUND)? 

EU Added 
Value 

•  What benefits have you gained due to changes implemented as a result of 
your participation in MSCA projects within your institution/department? 
Would these changes have been implemented without the Horizon Europe 
project?  

•  Do you think these impacts will be long-lasting? Do you think they will have 
an effect on your organisation‘s overall competitiveness? If so, in what 
ways? 

323 

 
Annex 2.4: How the MSCA contribute to more sustainable and 
greener research 

1.1. 

Introduction 

In  line  with  the  European  Green  Deal  and  the  provisions  guiding  the  9th  Framework 
Programme  for Research  and  Innovation500,  the  Marie  Skłodowska-Curie  Actions  (MSCA) 
introduced a bespoke Green Charter that lays down a set of general principles and objectives 
that promote the sustainable implementation of project activities.  

The  MSCA  Green  Charter  constitutes  a  code  of  good  practice  for  all  recipients  of  MSCA 
funding  –  individuals,  institutions  and  consortia  –  and  seeks  to  reduce  the  environmental 
footprint  of  MSCA-funded  projects,  raise  awareness  of  environmental  sustainability,  and 
serve as a catalyst in promoting best practices in sustainable research management501. All 
MSCA participants are expected to adhere to the Green Charter on a ‘best effort’ basis, and 
to commit to as many of its provisions as possible during the implementation of their projects.  

This case study aims to assess the MSCA Green Charter’s contribution to sustainable and 
greener  research  by  examining  the  extent  to  which  MSCA  fellows  and  beneficiary 
organisations implement sustainable research activities under Horizon Europe. It looks at the 
conditions, best practices and obstacles that drive the successful uptake of such activities. 

1.2.  Background information 

1.2.1.  European Union green transition and sustainable development 

A specific objective of Horizon Europe is to advance knowledge and amplify the impact of 
Research and Innovation (R&I) in aiding the development, facilitation, and execution of Union 
policies.502 This extends to bolstering the accessibility and adoption of innovative solutions 
across  European  industries,  with  a  particular  focus  on  researcher-,  institutional-,  and 
consortium-related measures. These categories operationalise the goal of capturing broader 
society,  aimed  at  confronting  global  challenges,  including  climate  change  and  the 
Sustainable Development Goals (SDGs). 

Originating in the European Green Deal, the principle of 'Do no significant harm' applies to 
the  execution  of  the  Horizon  Europe  programme.  This  underlines  the  importance  of 
conducting  research  and  innovation  activities  in  a  manner  that  upholds  the  climate  and 
environmental priorities of the European Union, ensuring that they do not result in significant 

500 Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 2021 
establishing Horizon Europe – the Framework Programme for Research and Innovation, laying 
down its rules for participation and dissemination, and repealing Regulations (EU) No 1290/2013 
and (EU) No 1291/2013, available at: https://eur-lex.europa.eu/legal-
content/EN/ALL/?uri=CELEX%3A32021R0695  
501 Marie Skłodowska-Curie Actions (MSCA), MSCA Green Charter, available at: https://marie-
sklodowska-curie-actions.ec.europa.eu/accessible/green-charter 
502 European Commission, Directorate-General for Education, Youth, Sport and Culture, Marie 
Skłodowska-Curie Actions Green Charter, Publications Office, 2021, 
https://data.europa.eu/doi/10.2766/720690, p. 5. 

324 

 
 
 
harm  to  the  environment503.  Moreover,  the  commitment  to  monitor  and  oversee  the 
implementation  of  the  Green  Charter  for  the  Marie  Skłodowska-Curie  Actions  (MSCA)  is 
emphasised  in  the  Communication  from  the  Commission  on  a  European  Strategy  for 
Universities504. 

Also,  in  line  with  the  European  Green  Deal,  the  Council  of  the  European  Union  issued  a 
Recommendation  on  learning  for the  green  transition  and  sustainable  development on  16 
June 2022505. The Recommendation highlights the need to continue and step-up efforts to 
implement  policies  and  programmes  of  learning  to  support  the  green  transition,  and  to 
promote  learning  for  sustainable  development.  It  calls  for  whole-institution  approaches  to 
sustainability 
innovation, 
including,  among  others,  areas  such  as  research  and 
infrastructures,  facilities  and  operations506.  In  addition,  on  15  November  2023  the 
Commission  adopted  a  Proposal  for  a  Council  Recommendation  entitled  ‘Europe  on  the 
Move’  –  learning  mobility  opportunities  for  everyone”,  one  of  the  objectives  of  which  is  to 
make  learning  mobility  more  environmentally  sustainable,  including  between  higher 
education institutions (HEIs)507. 

Currently,  there  is  no  single  European  policy  framework  covering  the  sustainable 
implementation  of  research  activities  funded  from  the  EU  budget.  However,  research 
organisations and individuals are starting to take measures to integrate sustainable practices 
into  research  projects  and  research  activities.  For  example,  some  organisations  are 
evaluating the carbon footprint of their laboratories/institutions, while other organisations are 
opting  for the  use  of  teleconferencing  tools  and  avoiding  unnecessary  travel508.  Research 
institutes have also begun to develop and introduce training on environmental issues relating 
to energy, climate and biodiversity.  

1.2.2.  The Marie Skłodowska-Curie Actions (MSCA) 

The  Marie  Skłodowska-Curie  Actions  (MSCA)  are  the  European  Union’s  flagship  funding 
programme for doctoral education and postdoctoral training in the European Research Area 
(ERA) and beyond. The MSCA fund excellent research and innovation and equip researchers 
at all stages of their careers with new knowledge and skills through mobility across borders 
and exposure to different sectors and disciplines. The MSCA help to build Europe’s capacity 
for research and innovation by investing in the long-term careers of excellent researchers509. 
The MSCA also set out to enhance training and career development systems, thereby having 
a structuring impact on participating institutions. 

503 European Commission. (2023). Horizon Europe Programme Guide.  
504 European Commission. (2022). Communication from the Commission on a European 
Strategy for Universities, available at: https://education.ec.europa.eu/sites/default/files/2022-
01/communication-european-strategy-for-universities-graphic-version.pdf 
505 European Council. Recommendation of 16 June 2022 on learning for the green transition and 
sustainable development 2022/C 243/01. 
506 European Council. Recommendation of 16 June 2022 on learning for the green transition and 
sustainable development 2022/C 243/01. 
507 European Commission, Proposal for a Council Recommendation ‘“Europe on the Move” – 
learning mobility for everyone’, COM(2023) 719 final, at 8. 
508 Science Europe (n.d.). Greening Research Organisations: Reducing our Carbon Footprint, 
available at: https://www.scienceeurope.org/news/greening-research-organisations-reducing-
our-carbon-footprint/ 
509 European Commission Decision C (2021)4200 of 15 June 2021, Horizon Europe Work 
Programme 2021- 2022, 2. Marie Skłodowska-Curie Actions. 

325 

 
 
 
The programme aims to achieve its objectives through five main actions510. These include 
the four actions relevant to the present study, namely:  

•  Postdoctoral Fellowships (PF), which support the career prospects and excellence 

of postdoctoral researchers; 

•  Doctoral  Networks  (DN),  which  support  programmes  to  train  doctoral  candidates 

within academic and non-academic organisations; 

•  COFUND, which co-funds regional, national and international programmes; and 
•  Staff  Exchanges  (SE),  which  encourages  collaborations  between  organisations 

through staff exchanges. 

In line with the European Green Deal, the United Nations’ 2030 Agenda, and the Sustainable 
Development  Goals,  the  MSCA  promote  the  sustainable  implementation  of  research 
activities  through  the  newly  introduced  MSCA  Green  Charter511.  The  Charter  provides 
guidelines on how individuals, institutions and consortia who are recipients of MSCA funding 
can minimise the environmental impacts of their research activities through measures relating 
to  researchers, 
institutions,  consortia  (for  multi-beneficiary  projects),  and  outreach 
(applicable  to  MSCA  researchers  and  participating  institutions).  These  measures  are  not 
exhaustive, and institutions or researchers are invited to complement measures carried out 
under  the  Charter  with  other,  additional  initiatives512.  Although  the  implementation  of  the 
MSCA Green Charter is not mandatory, MSCA beneficiary organisations are asked in project 
proposals and at final reporting stage to report on how they will or have applied the principles 
of the Green Charter. Starting in 2024, sustainability will be considered an additional criterion 
for  the  selection  of  ex-aequo  proposals  that  are  evaluated  as  equal  in  terms  of  both 
excellence and impact.  

In addition to promoting the implementation of sustainable research activities, the MSCA also 
contribute  to  the  green  transition  and  sustainable  development  by  supporting  research 
projects  with  direct  relevance  to  the  European  Green  Deal  and  the  SDGs.  The  MSCA 
European Green Deal cluster event has already highlighted the MSCA programme’s role in 
addressing the objectives of the European Green Deal. Under the Horizon 2020 programme 
(2014-2020), the MSCA successfully supported around 500 projects that were found to be 
relevant to the European Green Deal513. The evaluation study on Excellent Science in the 
European Framework Programme for Research and Innovation also found that the methods 
employed by MSCA-funded projects contributed or related to SDGs that directly highlighted 
green practices and/or sustainability514. 

510 European Commission Decision C (2021)4200 of 15 June 2021, Horizon Europe Work 
Programme 2021- 2022, 2. Marie Skłodowska-Curie Actions. 
511 Marie Skłodowska-Curie Actions (MSCA), MSCA Green Charter, available at: https://marie-
sklodowska-curie-actions.ec.europa.eu/accessible/green-charter 
512 Marie Skłodowska-Curie Actions (MSCA). MSCA Green Charter, available at: https://marie-
sklodowska-curie-actions.ec.europa.eu/accessible/green-charter 
513 European Commission, European Research Executive Agency, MSCA cluster event on the 
European Green Deal: meeting report and key messages for policy consideration, Publications 
Office of the European Union, 2022, available at:https://data.europa.eu/doi/10.2848/870162 
514 European Commission, Evaluation study on Excellent Science in the European Framework 
Programmes for Research and Innovation. Phase 1 Annexes, 2023.  

326 

 
 
 
1.2.3.  Overall approach and scope of the case study 

The  present  case  study  evaluates  the  extent  to  which  MSCA  fellows  and  participating 
organisations  adhere  to  the  MSCA  Green  Charter,  which  was  introduced  in  2021.  The 
study  looks  at  the  implementation  of  green/sustainable  research  activities  and  the 
contribution  that  the  tailored  suggested  research  methods  make  to  this specific  area.  The 
case  study  also  aims  to  highlight  the  conditions  and  best  practices  that  contribute  to  the 
implementation of green/sustainable research activities, as well as the barriers that hinder 
them. The study builds on the findings of the survey on the MSCA Green Charter conducted 
by the Research Executive Agency (REA) at the request of the European Commission. 

The  analysis  will  provide  information  on  progress  (or  lack  thereof)  in  four  categories  of 
measures introduced under the MSCA Green Charter:  

•  Researcher-related measures; 
• 
Institution-related measures; 
•  Consortium-related measures; and 
•  Outreach-related measures.  

In  addition,  the  present  case  study  also  seeks  to  capture  the  extent  to  which  the  MSCA 
contributes  to  the  Sustainable  Development  Goals.  The  case  study  addresses  evaluation 
questions  relating  to  relevance,  coherence,  effectiveness,  efficiency  and  EU added  value. 
The table below presents the evaluation criteria relevant to the case study and its objectives.  

Evaluation 
criteria 

Relevance 

Coherence 

Effectiveness 

Judgement criteria 

• 

• 

• 

Links between the suggested intervention and the needs/problems identified 
by the participating organisations/beneficiaries  
The extent to which the operational needs of the participating 
organisations/beneficiaries are addressed in terms of the available 
suggested sustainable research practices  
The design of MSCA projects in order to achieve the Sustainable Development 
Goals (SDGs)  

•  Gaps and emerging needs identified that are not covered by the suggested 

• 

• 

• 

• 

• 

• 

intervention  
The extent to which the MSCA Green Charter maps on to the European 
Union Green Transition and Green Deal Policy  

Synergies and complementarities between the MSCA Green Charter and 
strategies/initiatives at the level of the participating organisations (CH3) 
The extent to which the MSCA Green Charter fits into the dynamic between 
the EU sustainable development / sustainable education recommendations, 
and the overarching goals of the MSCA  

The extent to which the MSCA Green Charter contributes to the sustainable 
implementation of research activities at institutional/project level 
The extent to which MSCA projects have contributed to particular Sustainable 
Development Goals (SDGs)  
Examples of good practice/success stories/challenges in implementing the 
MSCA Green Charter  

327 

 
 
Evaluation 
criteria 

Judgement criteria 

• 

Enablers and drivers versus barriers, as well as solutions to further support 
and encourage the promotion and/or uptake of sustainable research 
activities 

Efficiency 

• 

• 
• 

• 

EU added 
value 

The extent to which needs for flexibility in the implementation of sustainable 
research activities are met. Efficiency in sustainable research hinges on the 
ability to adapt to evolving conditions and requirements. Flexibility allows 
researchers to swiftly navigate challenges and seize opportunities, ensuring 
optimal resource utilization and timely achievement of objectives. 
Key drivers toward and barriers hindering progress  
Lessons learned from the experiences of applicants and participants  

Analysing the differences in environmental effects between MSCA projects  

Table 1. Evaluation Criteria 

To  address  the  above  criteria,  the  present  case  study  analyses  and  triangulates  data 
obtained from diverse sources and through the use of several different research methods: 

•  Desk research: this comprised a thorough review of existing literature and data sources 
to  gather  background  information  and  identify  key  themes  and  trends.  Several 
foundational literature sources were consulted for this part of the project, including (but 
not limited to) the ‘Green Charter and Guidelines’ and the ‘Study on the Evaluation of 
Excellent Science framework programme for research and innovation’. Other literature 
sources  were  used  to  provide  valuable  information  on  environmental  policies  and 
guidelines, such as when analysing the contribution of sustainable research practices to 
SDGs,515  as  well  as  insights  into  the  evaluation  of  scientific  research  and  innovation 
within the framework of Horizon Europe. A full list of literature sources can be found in 
Appendix  1  of  this  report.  The  desk  research  incorporates  findings  from  a  survey 
conducted by REA to assess the implementation of the MSCA Green Charter. Prior to 
analysing the survey responses, data cleaning was performed, which involved removing 
entries from respondents involved in projects under Horizon 2020 rather than Horizon 
Europe, based on the project ID. Following this cleaning, the total number of respondents 
in the analysed dataset was 513, covering all actions under the MSCA. The breakdown 
by action type includes 224 responses from participants in Doctoral Networks (DNs), 223 
responses from Postdoctoral Fellowships (PFs), 55 from Staff Exchanges (SEs), and 11 
responses  from  participants in  COFUND actions.  Geographically,  the survey  received 
responses from 36 countries, with a majority coming from Western Europe. 

• 

Interview programme: an interview programme was carried out to gather insights and 
perspectives on the topic examined by the case study. The study team aimed to ensure 
an equal distribution of respondents by action type (PF, DN, CODUND and SE). Potential 
interviewees were identified among those who responded to the survey launched by the 
Research  Executive  Agency  (REA)  on  the  implementation  of  sustainable  research 

515 European Commission, “Sustainable Development Goals,” International Partnerships, 2024, 
available at: https://international-partnerships.ec.europa.eu/policies/sustainable-development-
goals_en 

328 

 
 
 
activities (the MSCA Green Charter). In total, 11 interviews have been conducted with 
organisations participating in MSCA actions. Appendix 2 provides a full list of completed 
interviews.  

•  Survey  programme:  the  case  study  also  draws  on  data  from  a  survey  of  MSCA 
beneficiary organisations and individual fellows, carried out under the evaluation study 
on  Excellent  Science  in  the  European  Framework  Programmes  for  Research  and 
Innovation. These data provide insights into the extent of green policy measures taken 
by the beneficiary organisations and shed light on the extent of Horizon Europe’s MSCA 
contributions to the development of sustainable solutions in research and innovation.  

•  Text mining and analysis of unstructured data: the case study has used text mining 
techniques to identify markers of sustainable research activities. Researchers analysed 
large volumes of text data to detect patterns and trends relating to sustainability practices 
in research. By examining data from applications, the study aims to highlight examples 
of  how  beneficiary  organisations  within  the  MSCA  implement  (or  plan  to  implement) 
sustainable  research  activities.  Specific  keywords  were  used  to  search  for  projects 
addressing  the  subject  under  analysis  in  both  interview  and  survey  responses. 
Furthermore, the case study highlights instances of good practice, whereby beneficiary 
organisations have implemented (or intend to implement) comprehensive measures to 
ensure  sustainable  research  activities.  These  cases  are  presented  below  in  Section 
1.3.3: Examples of good practices. 

1.3.  Case study findings 

1.3.1.  Relevance and coherence 

In the Research Executive Agency (REA) survey on the implementation of the MSCA Green 
Charter, 80% of respondents reported being aware or ‘vaguely familiar’ with the European 
Green Deal and its objectives, with fairly similar levels of awareness across all action types 
(see Figure 1).516 

Figure 1. Prior to the survey, were you aware of the European Green Deal and its objectives? Source: REA survey on the 
implementation of the MSCA Green Charter (n=513). 

516 2023 Research Executive Agency Survey on the MSCA Green Charter. 

329 

36%50%38%55%45%45%35%35%31%35%18%15%26%15%20%0%10%20%30%40%50%60%70%80%90%100%COFUND (n=11)Doctoral Networks (MSCA-DN) (n=224)Postdoctoral Fellowships (MSCA-PF) (n=223)Staff Exchanges (MSCA-SE) (n=55)All actionsYesVaguely familiarNo 
 
 
 
However, when asked about the MSCA Green Charter and its Guidelines, more than half of 
respondents to the REA survey (55%) said they were unaware of the existence of the code 
of good practices for individuals and institutions who receive MSCA funding. The respondents 
least aware of the MSCA Green Charter were participants under Postdoctoral Fellows (35%) 
and COFUND (36%). 

Figure 2. Prior to this survey, were you aware of the MSCA Green Charter? Source: REA survey on the implementation of 
the MSCA Green Charter (n=513). 

A  significant  majority  of  respondents  (62%)  indicated  that  the  institutions  participating  in 
MSCA action have a dedicated strategy to promote sustainable research activities. Among 
those institutions with a strategy, 52% of respondents believed that the MSCA Green Charter 
complements  their  existing  plans  to  a  moderate  or  to  a  considerable  extent,  as  shown  in 
Figure 3. 

Figure 3. You previously indicated that your institution has a strategy in place to promote sustainable research and green 
practices. To what extent, if any, do you see the MSCA Green Charter promoting the integration of environmental 
considerations that are not foreseen by the institutional strategy? Source: REA survey on the implementation of the MSCA 
Green Charter (n=246). 

The  MSCA  Green  Charter  contributes  to  the  objectives  of  the  broader  European  Green 
Deal,517 which itself is an integral part of the European Commission’s strategy to implement 

517 Marie Skłodowska-Curie Actions, Green Charter, 2021, available at: https://marie-
sklodowska-curie-actions.ec.europa.eu/accessible/green-charter    

330 

36%55%35%49%45%64%45%65%51%55%0%10%20%30%40%50%60%70%80%90%100%COFUND (n=11)Doctoral Networks (MSCA-DN) (n=224)Postdoctoral Fellowships (MSCA-PF) (n=223)Staff Exchanges (MSCA-SE) (n=55)All actionsYesNo15%37%21%4%23%To a large extentTo a moderate extentTo a limited extentNot at allDo not know 
 
 
 
 
the  United  Nations’  2030  Agenda  and  the  resulting  Sustainable  Development  Goals 
(SDGs).518 The Green Charter directly aligns with several specific SDGs. The first of these is 
Climate Action (SDG 13), with the Green Charter specifically producing outcomes in relation 
to  targets  13.2  and  13.3.519  The  Green  Charter  helps  to  meet  SDG  13  target  13.2  by 
encouraging  researchers  to  develop  long-term  strategies  for  sustainable  research,  as 
mentioned by interviewees.  

The Charter contributes to meeting target 13.3 by mainstreaming sustainability as a relevant 
concern for researchers when designing their methodologies – an effect that is then multiplied 
by  the  magnitude  of  the  distribution  of  this  specific  research.  This  outcome  was  also 
discussed by interviewees, many of whom felt that the mainstreaming of sustainability as a 
relevant  concern  for  researchers  was  the  most  impactful  effect  of  the  Charter,  especially 
within institutions that had existing concerns or standards for sustainability.  

The Green Charter also influences the achievement of a range of Sustainable Development 
Goals (SDGs) based on the differing focus of various MSCA projects. For instance, one water 
microbiomes  researcher  highlighted  how  reducing  the  consumption  of  lab  equipment –  a 
traditionally resource-intensive aspect of research – can help to align with SDGs 12 and 17. 
Another  MSCA  researcher  emphasised  sustainable  travel  incentives,  associating  them 
directly to SDG 13. However, projects whose research focusses on sustainability outcomes 
are  challenging  to  categorise  in  terms  of  their  broader  SDG  impacts,  beyond  the 
mainstreaming effects previously noted. 

As  part  of  the  evaluation,  an  analysis  of  the  projects’  contributions  to  specific  SDGs  was 
conducted.  SDG  labels  were  assigned  to  projects  using  the  OSDG-provided  multi-label 
classification. The labels were assigned on the basis of the projects’ description data, taking 
into  account  the  relative  weight  of  the  text  segment  and  the  SDG-related  content  in  the 
corpus. For a specific SDG to be assigned to a project, at least 15% of text segments in the 
corpus had to be SDG-related, and each SDG had to account for at least 10% of all the SDG-
relevant content in the project description.  

The above analysis reveals the prominent contribution made by the MSCA to the Sustainable 
Development  Goals.  The  SDGs  most  frequently  addressed  by  MSCA-funded  projects  are 
SDG3 – Good Health and Well-Being (48%); SDG4 – Quality Education (30%); and SDG16 
– Peace, Justice and Strong Institutions (24%). The SDGs least well covered by the MSCA 
Green Charter are SDG1 – Affordable and Clean Energy (3%) and SDG2 – Decent Work and 
Economic Growth (3%). Projects contributing to those SDGs that were covered the most also 
received the highest shares of EU contribution. 

518 European Commission, European Green Deal, 2019, available at: https://eur-
lex.europa.eu/legal-content/EN/TXT/?uri=COM%3A2019%3A640%3AFIN; United Nations, The 
17 Sustainable Development Goals, 2015, available at: https://sdgs.un.org/goals 
519 United Nations, SDG Goal 13 Targets, Target 13.2: Integrate climate change measures into 
national policies, strategies and planning; Target 13.3: Improve education, awareness-raising 
and human and institutional capacity on climate change mitigation, adaptation, impact reduction 
and early warning). The MSCA Green Charter impacts two targets from this SDG. 

331 

 
 
 
 
No. of projects 

% of total 

EU contribution 
(in million EUR) 

% of total 

101 

86 

3.2% 

2.7% 

5.0 

8.3 

0.3% 

0.6% 

1,542 

48.4% 

440.9 

29.4% 

955 

579 

149 

625 

98 

545 

20 

426 

150 

426 

211 

134 

772 

29.9% 

142.1 

18.2% 

4.7% 

85.9 

24.1 

19.6% 

116.7 

3.1% 

17.1% 

0.6% 

13.4% 

4.7% 

13.4% 

6.6% 

4.2% 

15.7 

88.8 

1.4 

51.5 

26.8 

50.7 

32.1 

16.5 

24.2% 

142.9 

9.5% 

5.7% 

1.6% 

7.8% 

1.0% 

5.9% 

0.1% 

3.4% 

1.8% 

3.4% 

2.1% 

1.1% 

9.5% 

SDG1 

SDG2 

SDG3 

SDG4 

SDG5 

SDG6 

SDG7 

SDG8 

SDG9 

SDG10 

SDG11 

SDG12 

SDG13 

SDG14 

SDG15 

SDG16 

Table 2. Number of MSCA projects by SDG. Source: data on actions from Corda and project descriptions. 

The  table  below  reflects  an  overall  increase  in  the  coverage  of  SDGs  by  MSCA-funded 
projects  compared  with  the  previous  Framework  Programme,  Horizon  2020.  Under  the 
current  Framework  Programme,  Horizon  Europe,  the  MSCA  show  increased  coverage  of 
SDG4  and  SDG16,  while  the  most  important  SDGs  under  Horizon  2020  were  SDG8  and 
SDG13. 

332 

 
 
 
 
 
Thematic priority 

MSCA (HE) 

MSCA (H2020) 

SDG1 

SDG2 

SDG3 

SDG4 

SDG5 

SDG6 

SDG7 

SDG8 

SDG9 

SDG10 

SDG11 

SDG12 

SDG13 

SDG14 

SDG15 

SDG16 

3.2% 

2.7% 

48.4% 

29.9% 

18.2% 

4.7% 

19.6% 

3.1% 

17.1% 

0.6% 

13.4% 

4.7% 

13.4% 

6.6% 

4.2% 

24.2% 

3.0% 

0.5% 

38.4% 

7.4% 

1.3% 

3.3% 

6.5% 

3.6% 

14.7% 

0.8% 

9.4% 

2.8% 

14.2% 

4.3% 

4.3% 

7.2% 

Table 3. Comparison of the shares of MSCA projects under Horizon 2020 and Horizon Europe, by SDG. Source: data on 
actions from Corda and project descriptions. 

However, with regard to the aim of integrating SDG-considerations into the implementation 
of the MSCA projects, the REA survey found that institutions and researchers are confronted 
by  a  myriad  of  challenges  spanning  various  domains  and  research  areas.  One  prominent 
hurdle  is  the  stark  disparity  between  environmentally  friendly  transport  options,  such  as 
trains, and more polluting modes of transport such as planes. Differences in cost often make 
it difficult for researchers to opt for potentially greener modes of travel, especially when faced 

333 

 
 
 
with  budget  constraints  and  the  need  to  cover  long  distances  for  field  research  or 
conferences. 

Furthermore,  although  more  commonly  applied  at  regional  and  national  scales,  carbon 
offsetting  is  not  always  readily  accessible  at  the  local  level,  further  exacerbating  the 
challenges  associated  with  carbon-free  or  low-carbon  travel  arrangements.  For  example, 
carbon  offsetting  involves  compensating  for  one’s  carbon  emissions  by  investing  in 
environmental  projects  that  reduce  greenhouse  gas  emissions  or  promote  sustainability. 
However, depending on the location and availability of such projects, researchers may find it 
difficult to offset their carbon footprint effectively.  

Moreover, the scientific nature of certain research projects necessitates the use of specific 
materials and experimental protocols that may not always align with sustainability objectives. 
Researchers  often  are  required,  either  due  to  budget  constraints  or  through  the  lack  of 
available alternatives, substances that are hard to substitute with greener alternatives, such 
as the highly toxic reagents essential for their experiments. Respondents also pointed to the 
prevalence  of  single-use  plastics  in,  for  example,  molecular  biology,  which  poses  another 
notable  challenge,  as  finding  equally  effective  and  contamination-resistant  alternatives 
remains a persistent obstacle. 

Challenges in integrating sustainable practices into research projects can also be attributed 
to structural discrepancies inherent in the project topics, rather than merely stemming from a 
lack of willingness. Certain projects have unique scopes and face financing limitations that 
pose obstacles to the implementation of the guidelines outlined in the MSCA Green Charter. 
For  instance,  during  an  interview,  one  participant  described  a  project  that  involved 
geographically isolated researchers without internet access, which significantly hinders the 
feasibility of avoiding extensive travel.  

institutional  constraints  and  resistance 

Lastly, 
individuals  present  additional 
complications. Logistical constraints in administrative processes, such as purchasing goods 
and  managing  expenses,  can  impede  sustainability  initiatives.  For  instance,  policies 
governing  the  purchase  of  goods,  energy  and  other  resources  may  not  prioritise 
environmentally  friendly  options,  thus  limiting  researchers’  ability  to  make  sustainable 
choices despite their willingness to do so. 

from 

1.3.2.  Effectiveness and efficiency 

Respondents to the REA survey on the implementation of the MSCA Green Charter were 
asked  about  their  willingness,  over  the  lifetime  of  the  MSCA  project,  to  integrate 
environmental  considerations  in  line  with  the  MSCA  Green  Charter.  The  results  reveal  a 
strong commitment across all action types (72%) with only 3% of respondents indicating that 
they were unwilling. 

Respondents  who  had  earlier  indicated  their  willingness  to  take  measures  to  integrate 
environmental  considerations  into  their  work  in  line  with  the  MSCA  Green  Charter  were 
further  asked  about  what  specific  measures  they  would  be  most  willing  to  adopt  at  an 
individual, institutional and consortium level.  

334 

 
 
 
Figure 4. Will you, over the lifetime of your MSCA project, take measures to integrate environmental considerations in line 
with the MSCA Green Charter? Source: REA survey on the implementation of the MSCA Green Charter (n=513). 

At the level of individual researchers, the survey findings demonstrate a generally positive 
attitude  among  the  respondents  towards  adopting  sustainability  and  environmental 
measures.  Among  others,  measures  related  to  teleconferencing  (74%),  waste  reduction 
(66%), low-carbon transportation (63%), and minimizing resource usage (61%) were chosen 
the most (see figure below). 

In addition, the survey of Horizon Europe beneficiaries also revealed that 49% of fellows in 
PF  actions  agreed  that  participation  in  their  MSCA  project  will  have  an  impact  on  the 
development  of  their  skills  in  relation  to  the  sustainable  implementation  of  research 
activities.520 

Figure 5. Willingness to adopt sustainability measures at individual level. Source: REA survey on the implementation of the 
MSCA Green Charter (n=500). 

The REA survey on the implementation of the MSCA Green Charter further highlighted the 
institutional-level environmental measures that respondents were most willing to integrate, in 

520 Survey of Horizon Europe beneficiaries, conducted May-July 2023 (n=594). 

335 

55%78%65%75%72%36%21%31%20%26%9%1%3%5%3%0%10%20%30%40%50%60%70%80%90%100%COFUND (n=11)Doctoral Networks (MSCA-DN) (n=224)Postdoctoral Fellowships (MSCA-PF) (n=223)Staff Exchanges (MSCA-SE) (n=55)All actions (n=513)YesMaybeNo6%7%23%30%39%47%51%55%55%62%0%10%20%30%40%50%60%70%Subscribe to a green lab accreditationSet up a unit responsible for integrating environmental…Support the use of carbon offsettingPromote new forms of consumption linked to the collaborative…Promote green purchasing for project-related materialsPromote the use of renewable forms of energySupport the use of sustainable and low-carbon forms of…Provide  guidance to MSCA fellows in minimising environmental…Support MSCA researchers developing awareness on…Encourage teleconferencing and hybrid conferencing 
 
 
 
 
line with the MSCA Green Charter. Teleconferencing and hybrid conferencing emerged as 
the  most  widely  favoured  measure,  with  62%  of  respondents  expressing  willingness  to 
embrace  these  alternatives  to  reduce  the  environmental  impact  of  travel.  In  addition, 
respondents showed a keen interest in supporting MSCA researchers to develop awareness 
of  sustainability  (55%)  and  in  providing  guidance  to  MSCA  fellows  on  minimising  their 
environmental footprint (55%). Conversely, measures relating to the setting up of a dedicated 
unit responsible for integrating environmental considerations (7%) and subscription to Green 
Lab Certification (6%) were chosen least frequently by the respondents (See figure below). 

Furthermore, the survey of Horizon Europe beneficiaries found that 49% of successful MSCA 
organisations  expected  that  participation  in  the  Horizon  Europe  programme  would  lead  to 
improvements  at  organisational  level  with  regard  to  the  sustainable  implementation  of 
research activities. 521 

Figure 6. Willingness to adopt sustainability measures at institutional level. Source: REA survey on the implementation of the 
MSCA Green Charter (n=500). 

At consortium level, the survey data reveal that the most widely favoured measure, preferred 
by 60% of  respondents, is the promotion of teleconferencing and online tools. In addition, 
50% of respondents expressed a willingness to promote the sharing of best environmental 
practices  within  the  consortium.  Moreover,  48%  of  respondents  showed  keen  interest  in 
organising project-related events sustainably, emphasising a conscious effort to minimise the 
environmental impact of consortium gatherings and activities.  

521 Survey of Horizon Europe beneficiaries conducted May-July 2023 (n=594). 

336 

6%7%23%30%39%47%51%55%55%62%0%10%20%30%40%50%60%70%Subscribe to a green lab accreditationSet up a unit responsible for integrating environmental considerationsSupport the use of carbon offsettingPromote new forms of consumption linked to the collaborativeeconomyPromote green purchasing for project-related materialsPromote the use of renewable forms of energySupport the use of sustainable and low-carbon forms of transportationProvide  guidance to MSCA fellows in minimising environmental impactSupport MSCA researchers developing awareness on sustainabilityEncourage teleconferencing and hybrid conferencing 
 
 
 
Figure 7. Willingness to adopt sustainability measures at consortium level. Source: REA survey on the implementation of the 
MSCA Green Charter (n=500). 

The section that follows presents a number of good practice examples from MSCA institutions 
and  consortia  of  the  use  of  the  MSCA  Green  Charter  in  the  development  of  effective 
sustainable research practices within their organisation: 

1.3.3.  Examples of good practice 

1.3.3.1. 

Institutional measures at the University of Vienna (UvO) 

The University of Vienna (UoV) has demonstrated its commitment to sustainability through 
the  adoption  of  renewable  energy  sources  in  several  infrastructural  projects  since  2022. 
Notably,  half  of  its  buildings  use  only  certified  green  power  (UZ46).  To  reduce  carbon 
emissions resulting from travel, UoV has implemented a travel policy mandating train travel 
for  destinations  within  a  6-hour  radius,  regardless  of  flight  cost.  Moreover,  business  trips 
undertaken  by  air  incur  a  20%  surcharge  from  UoV,  with  proceeds  directed  towards  a 
sustainability  fund  overseen  by  the  university’s  rectorate.  These  measures  exemplify  the 
institution’s  proactive  approach  to  reducing  its  environmental  impact  and  promoting 
sustainable practices. 

1.3.3.2. 

Institutional Measures at a European University 

At the university of one interviewee, administrators established a carbon-offsetting budget for 
students and researchers, who were allowed governance over the allocation of that budget. 
Therefore, researchers (rather than administrators) were given authority to map the bands of 
this budget (e.g. setting amounts for intercontinental flights or local trains). For a particular 
project, after estimating the carbon cost of their research, researchers then selected an EU-
nominated carbon offsetting scheme to fund from the project’s pre-established budget. model 
proved  quite  successful  because,  in  the  interviewee’s  experience,  experienced  professors 
and administrators were much more likely to trust and follow their researchers’ judgement 
rather than the general MSCA standards. Nonetheless, adopting this specific model might 
present difficulties for the broader MSCA (there is no clear MSCA institutional funding bucket 
for this kind of endowment and, as the interviewee acknowledged, this model could be less 
popular in certain disciplines depending on administrative engagement).  

337 

7%11%37%39%48%50%60%0%10%20%30%40%50%60%70%Set up a dedicated unit responsible for integratingenvironmental considerationsEngage MSCA fellows the consortium’s efforts at implementing the Green CharterPromote training on issues of sustainable researchPromote green purchasing for project-related materialsOrganise project-related events as sustainably as possiblePromote the sharing of best environmental practices within theconsortiumPromote the use of teleconferencing and online tools 
 
 
1.3.3.3. 

Consortium-level measures in MSCA-DN project LongTrec 

As part of the Doctoral Network project LongTrec, a sustainability plan will be a part of 
the career development plan researchers will be required to monitor the carbon footprint 
of their activities. The training programmes will also include seminars where knowledge 
will be shared on the implementation of sustainable research activities. 

1.3.3.4. 

Consortium-level measures in MSCA-DN project LEMUR 

Within  the  consortium  of  the  Doctoral  Network  project  LEMUR,  the  research  team 
developed resource-efficient Machine Learning algorithms and makes research results 
available for reuse to avoid the need for re-computation. The research team also runs 
tests on small datasets to avoid unnecessary lengthy computations.  

The  survey  of  MSCA  beneficiary  organisations  also  revealed  that  more  than  half  of  the 
respondents  (60%)  were  to  a  ‘very  large’  or  ‘large’  extent  satisfied  with  the  clarity  of  the 
instructions provided in the application process in relation to the principle of ‘Do no significant 
harm’522.  However,  compared  with  the  finding  above  that  half  of  respondents  were  either 
‘aware’  of  or ‘vaguely  familiar’  with  the  European  Green  Deal  and  its  objectives,  or of  the 
MSCA Green Chart. To summarise: there are concerns regarding the level of awareness and 
understanding of key initiatives relating to sustainability and environmental goals, potentially 
highlighting the need for better communication or education in this regard. 

Figure 8. To what extent are you satisfied with the following changes in the application process introduced under Horizon 
Europe? ‘Clarity of the instructions related to the ‘Do no significant harm’ principle’. Source: survey of Horizon Europe 
beneficiaries, conducted May-July 2023 (n=425). 

From  a  financial  perspective,  only  3%  of  respondents  to  the  REA  survey  on  the 
implementation of the MSCA Green Charter  (n=367) indicated that they received financial 
support  (from  their  institution  or  external  sources)  for  the  integration  of  environmental 
considerations into their MSCA project. However, interestingly, out of those who indicated 
that they did not receive financial support, more than three-quarters (78%) did not consider 
this lack of funding to be a key factor limiting what they could achieve (Figure 9). 

522 The principle states that research and innovation activities should not support or carry out 
activities that create significant harm to any of the objectives of the European Green Deal. 

338 

17%13%16%14%33%45%50%47%17%27%22%26%17%10%9%10%17%4%3%4%0%10%20%30%40%50%60%70%80%90%100%COFUND (n=6)Doctoral Networks (DN) (n=306)Staff Exchanges (SE) (n=113)All actions (n=425)To a very large extentTo a large extentTo a moderate extentTo a small extentNot at all 
 
 
 
Figure 9. Do you consider the lack of financial support a key factor limiting what you could achieve? Source: REA survey on 
the implementation of the MSCA Green Charter (n=355). 

On the other hand, the fact that 22% of respondents did consider a lack of financial support 
to  be  a  principal  factor  limiting  the  integration  of  environmental  considerations  into  their 
projects highlighted the importance of budgetary considerations in various areas, including 
for  green  travel,  equipment  upgrades,  personnel  recruitment,  sustainable  consumables, 
outreach  and  others.  Several  overarching  themes  and  priorities  emerged  from  these 
responses: 

•  Green travel: several respondents highlighted the importance of supporting sustainable 
travel options, such as using low-carbon forms of transportation, funding for train travel 
(which can be more expensive than flying), and carbon offsetting for travel emissions.  

•  Sustainable consumables and equipment upgrades: some respondents emphasised the 
need to replace current energy-inefficient equipment with more efficient alternatives; for 
instance, reusable lab instruments or exploring alternatives to disposable plastic objects. 
When asked about the specific types of partnerships that would benefit the most from 
such  measures,  some  interviewees  suggested  that  the  MSCA  industrial  doctorates 
offered 
regarding  environmental 
considerations. 

the  most  potential 

requirements 

include 

to 

•  Outreach  and  awareness:  some  respondents  indicated  that  additional  financial 
recourses could cover outreach activities, to raise awareness and promote sustainability 
practices among relevant stakeholders.  

• 

Infrastructure  and  technology:  covering  expenses  relating  to  the  setting-up  of  digital 
infrastructures  was  mentioned  by  several  respondents.  Respondents  highlighted  the 
importance of providing collaborators with the necessary digital tools, such as modems, 
mobile data, laptops and projectors. This type of support would enable collaborators to 
participate in online events and activities.  

•  Education  and  training:  several  respondents  expressed  the  need  to  cover  additional 
costs  associated  with  the  training  courses,  workshops  and  technical  staff  needed  to 
implement changes in relation to sustainable research activities.  

Respondents  to  the  REA  survey  were  also  asked  about  areas  of  improvement  that  could 
make  the  MSCA  greener  in  the  future.  The  survey  results  highlight  the  importance  of 

339 

33%25%18%17%22%67%75%82%83%78%0%10%20%30%40%50%60%70%80%90%100%COFUND (n=6)Doctoral Networks (n=167)Postdoctoral Fellowships (n=141)Staff Exchanges (n=41)All actions (n=355)YesNo 
 
 
communication and guidance in promoting sustainable research practices within the MSCA. 
Specifically, the most frequently mentioned areas in which the MSCA programme could be 
improved  refer  to  the  provision  of  1)  more  communication  and  promotion  of  best  practice 
examples (69% of responses); 2) more guidance on integrating environmental considerations 
(56%); and 3) more communication about and promotion of the MSCA Green Charter (47%). 

Figure 10. What would you suggest to the European Commission to make the MSCA greener in the future? Source: REA 
survey on the implementation of the MSCA Green Charter (n=513). 

Those  respondents  who  considered  the  aspect  of  training  to  be  important  in  making  the 
MSCA greener in the future provided specific examples of what types of training would be 
most beneficial to them. The most frequently mentioned areas of interest and importance with 
regard  to  training  for  researchers  on  adopting  greener  practices  within  their  projects  and 
contribute to sustainability goals are as follows: 

•  Green Lab Certification: strategies and implementation 

•  Carbon footprinting for laboratory researchers 

•  Resource-efficient technologies for research goals 

•  Carbon offset strategies   

•  Alternatives to single-use plastics 

•  Environmentally conscious waste management in laboratories 

•  Sustainable Computing Solutions for Research projects 

1.3.4.  EU added value  

Environment-related  research  has  for  a  long  time  been  a  strong  feature  of  the  European 
Framework Programmes, at least since Horizon 2020’s development and implementation of 

340 

2%10%14%17%20%26%26%47%56%69%0%10%20%30%40%50%60%70%80%OtherNothing, the current approach is sufficientMore trainings by the sending and/or host institutionAsk for commitment from the leadership of your institutionInclude greening in the evaluation criteria at application stageMore trainings by the European CommissionProvide financial supportMore communication and promotion of the MSCA Green CharterMore guidance on integrating environmental considerationsMore communication and promotion of best practice examples 
 
 
 
the European environmental research and innovation policy into its policy agenda in 2014.523 
This position remains especially relevant today, given that 35% of the entire Horizon Europe 
budget shall be dedicated to climate objectives.524  

However, in relation to sustainable research practices, evidence stemming from interviews 
indicates that the Green Charter faces difficulties in inspiring change at those institutions that 
are  not  already  invested  in  sustainability.  However,  for  those  institutions  already 
implementing  sustainable  research  practices,  the  MSCA  acts  as  a  robust  catalyst  for 
enhancement.  Multiple  interviewees  indicated  that  sustainability  requirements  rarely 
appeared  as  a  priority  at  all,  much  less  as  an  important  criterion.  When  it  comes  to  the 
challenges  associated  with integrating  sustainable  practices  in  research,  because  most  of 
the  MSCA  participants  interviewed  were  able  to  make  a  strong  association  between  the 
Charter  and  EU  policy  priorities,  potential  scope  exists  for  the  Framework  Programme  to 
increase  the  extent  of  EU  added  value  generated  by  providing  funding  for  sustainable 
activities with regard to specific practices detailed in the Charter. 

1.4.  Success factors and barriers  

The present case study reveals that the extent to which sustainable research practices can 
be implemented varies depending on the scope of the research project and the research field 
in which it is situated. For example, in interviews, research coordinators managing projects 
in the social sciences generally expressed fewer sustainability concerns, due to their projects 
typically having less environmental impact overall. In contrast, resource-intensive projects in 
other  fields  present  a  higher  risk  of  environmental  or  social  harm.  Interviewees  were 
particularly concerned that researchers may be inadvertently encouraged to exaggerate their 
intended  sustainability  practices  and  the  potential  environmental  impacts  of  their  projects. 
The study also revealed other prominent barriers:  

• 

Inherent  variation  between  MSCA  actions:  interviewees  noted  that  the  impact  of 
specific sustainable practices is heavily influenced by which MSCA action they appear 
in. For instance, while MSCA Postdoctoral Fellowships, typically lasting three years, are 
a mono-beneficiary action involving individual practices, MSCA COFUND and Doctoral 
Networks, due to their broader scope and longer-term institutional commitment, tend to 
create more enduring changes within institutions. While these factors touch on more than 
just  the  MSCA  Green  Charter’s  influence,  interviewees  specifically  highlighted  their 
relevance to the Charter’s effectiveness. 

•  Sustainable  travel:  a  prevailing  challenge  for  many  participating  organisations  and 
researchers  lies  in  effectively  managing  their  journeys.  While  the  MSCA  programme 
actively promotes collaboration with international partners, it does not allocate additional 
funding  for  sustainable  international  travel.  One  significant  hurdle  stems  from  the 
discernible cost gap between environmentally friendly transport options such as trains, 
and  more  polluting  alternatives  such  as  air  travel.  This  disparity  in cost  often  poses a 
dilemma for researchers, as greener travel options, particularly over long distances, may 
be notably pricier. 

523 European Commission Knowledge Centre for Biodiversity, European Resarch and Innovation 
Programsmes, 2022, available at: https://knowledge4policy.ec.europa.eu/biodiversity/eu-
supported-projects-initiatives-biodiversity-education_en 
524 Horizon Europe, available at: https://op.europa.eu/en/publication-detail/-
/publication/1f107d76-acbe-11eb-9767-01aa75ed71a1  

341 

 
 
 
•  Carbon  offsetting  difficulties:  options  for  carbon  offsetting  are  not  always  made 
available  at  a  researcher’s  institution  or  by  regional  authorities,  so  identifying  and 
facilitating offsetting schemes sometimes falls on individual researchers. Depending on 
location  and  availability,  researchers  may  struggle  to  effectively  offset  their  carbon 
footprint.  

•  Materials  and  protocol  issues:  the  discipline-specific  nature  of  certain  research 
sometimes requires materials and procedures that do not align with sustainability goals, 
with expensive but sustainable replacements only being available in certain cases (e.g. 
plastic and glass petri dishes). Researchers and organisations find it difficult to look for 
green  alternatives  for  essential  yet  toxic  reagents.  Many  research  fields,  such  as 
molecular  biology,  rely  heavily  on  single-use  plastics,  finding  contamination-resistant 
alternatives a challenging task.  

•  Administrative constraints: policies on purchasing goods, travel and resources largely 
depend on an institution’s governing body. Therefore, financial and logistical limitations 
usually  constrain  researchers  from  opting  for  more  eco-friendly  options,  thus  limiting 
sustainable choices even for researchers who would otherwise be willing.  

The present study also reveals challenges arising from varying approaches to sustainable 
research  guidelines.  Interviewees  noted  that  the  Green Charter  faces  difficulty  in  inspiring 
change at those institutions that are not already invested in sustainability. However, for those 
institutions already investing in sustainable research practices, the MSCA acts as a robust 
catalyst for enhancement. Interviewees indicated that this outcome is an effective  driver in 
enacting further change within participating organisations. In addition, the study reveals the 
following notable drivers: 

•  Training:  training  is  useful  to  institutions,  consortia  and  individuals  who  already  are 
heading  in  a  sustainable  direction,  e.g.  training can  help  to  operationalise sustainable 
research practices when recipients have already displayed an interest in these. 

•  Existing suggestions: the detailed recommendations currently provided in the Green 
Charter are comprehensive, as indicated by the interview programme. For example, 
the  accompanying  Guidance  Materials  suggest  that  organisations  which  require 
practical  tools  to  manage  environmental  responsibilities  should  obtain  the 
International  Organization  for  Standardization  (ISO)  14000  family  Environmental 
Management, an internationally recognised benchmark525.  

•  Balance of Green Charter structure and external position: the balance between the 
structure of the Green Charter and its nature as a non-binding code of good practice is 
a  pivotal  driver  for  the  development  of  sustainable  research  guidelines.  The  Green 
Charter’s  structured  framework,  which  encompasses  specific  practices  and  general 
sustainability  themes,  allows  researchers  to  adopt  tailored,  concrete  actions  while 
adhering to overarching sustainability principles. Its position as a flexible set of guidelines 
for  the  MSCA,  rather  than  a  strict  regulation,  accommodates  diverse  research 
disciplines,  fostering  a  culture  of  adaptable,  environmentally  conscious  research 
practices that resonate with a wide range of researchers and encourage the integration 
of sustainability into their work. 

525 Marie Skłodowska-Curie Actions, Green Charter Guidance Materials, 2021, available at: 
https://marie-sklodowska-curie-actions.ec.europa.eu/document/marie-sklodowska-curie-actions-
green-charter-guidance-materials  

342 

 
 
 
Other  challenges  observed  relate  to  the  different  ways  in  which  sustainable  research 
guidelines  are  implemented.  As  interviewees  reported,  the  Green  Charter  struggles  to 
promote  change  at  those  institutions  that  have  not  yet  made  efforts  to  engage  with 
sustainable practices to any degree. However, if a given institution has already embarked 
down that path, the MSCA is a powerful catalyst to accelerate and orient sustainable change. 
Therefore, if greater engagement with sustainable research practices is the desired outcome, 
the MSCA ought to facilitate more engagement in the way the Green Charter is designed, 
allowing for the Charter to better address institutions without charters for existing sustainable 
practices.  

1.5.  Conclusions and lessons learned  

Relevance and coherence 

The  findings  of  the  REA  survey  underline  a  dichotomy  in  awareness  among  MSCA 
respondents: while the European Green Deal is generally recognised (approximately 80% of 
respondents indicating being ‘aware’ of or ‘vaguely familiar with the Green Deal), a significant 
knowledge gap exists with regard to the MSCA Green Charter and its guidelines. Despite this 
gap, many institutions involved in MSCA actions have proactively embraced strategies that 
foster sustainable research activities. 

However, the application of the Charter’s principles is not without challenges. Chief among 
these are the constraints associated with eco-friendly travel and the limited accessibility of 
carbon offsetting measures. At the same time, as shown in Figure 9, less than one-quarter 
of respondents not yet receiving financial assistance indicated that further financial aid would 
be helpful. The specificities of certain research areas further complicate the adoption of the 
Charter,  with  some  fields  requiring  the  use  of  materials  or  practices  that  do  not  align 
seamlessly with sustainability goals. 

External  factors,  such  as  the  geographical  limitations  of  certain  projects  and  political  or 
institutional constraints, also emerge as significant barriers. In addition, policies that do not 
prioritise eco-friendly purchases, for instance, restrict researchers from making sustainable 
choices, even when they are inclined to do so. 

Efficiency and effectiveness 

The REA survey on the implementation of the MSCA Green Charter provides insights into 
the  current  position  and  future  potential  of  the  research  community’s  commitment  to 
sustainability. Analysis of the survey responses reveals several clear trends and implications 
for the MSCA community. 

A  significant  majority  (72%)  of  respondents  expressed  a  willingness  to  align  with  the 
environmental considerations outlined in the Green Charter across all action types, indicating 
a strong inclination toward sustainable practices within the MSCA research community. The 
minimal opposition (3%) further underlines a collective shift towards sustainability. 

Teleconferencing and online tools have been embraced at all research levels  – individual, 
institutional and consortium. This shift does not merely indicate a transition to digital means, 
but highlights a broader change within research culture, emphasising that digital alternatives 
are  now  considered  both  practical  and  environmentally  sound.  On  an  individual  level,  a 
noticeable emphasis was found on waste reduction, low-carbon transportation, and resource 
conservation, indicating an overarching trend towards a holistic approach to sustainability. 
However,  at  the  institutional  level,  the  present  study  notes  some  reluctance  to  embrace 

343 

 
 
measures  such  as  subscribing  to  Green  Lab  Certification  or  establishing  dedicated 
environmental units. Institutions might be hesitant to do so due to concerns about the costs 
associated  with  implementing  environmentally  sustainable  practices  and  technologies  in 
laboratory settings. Additionally, at the institutional level, there may be scepticism about the 
effectiveness of accreditation programs in driving meaningful change, as well as a lack of 
clarity  on  how  accreditation  criteria  align  with  academic  research  priorities  and  funding 
constraints. In contrast, at consortium level, there is a clearer inclination towards promoting 
shared environmental practices, indicating an understanding of the collective impact of such 
measures in achieving broader sustainability goals.  

It can be argued that Green Lab Certification-related training would be considered valuable 
by beneficiaries; this aligns with the overall theme of this report, which focuses on the shift 
towards sustainability and environmental consciousness in research. Green Lab Certification, 
and  the  training  associated  with  it,  are  practical  measures  to  encourage  environmentally 
responsible practices. However, it is essential to acknowledge that the text suggests some 
resistance at institutional level with regard to measures related to Green Lab Certification. 
Such  resistance  might  influence  perceptions  regarding  the  usefulness  of  such  training 
programmes. Consequently, the above argument is in harmony with the text’s emphasis on 
sustainability, but should also take into account potential challenges to the acceptance and 
implementation of Green Lab Certification measures at various levels of research. 

At consortium level, there is a notable inclination towards promoting shared environmental 
practices. This indicates an understanding of the cumulative impact of collective measures 
in achieving wider sustainability goals. 

The REA survey study also found that 49% of successful PF fellows perceived participation 
in  MSCA  projects  as  instrumental  in  enhancing  skills  in  relation  to  sustainable  research 
practices526. On the other hand, some researchers expressed concerns about sustainability 
requirements possibly limiting the breadth of their research or compelling them to exaggerate 
the  real-world  implications  of  their  work.  This  points  to  the  necessity  of  balancing 
sustainability requirements with freedom of academic exploration. 

The most consistent feedback stemming from the survey data and interviewees is the request 
for  enhanced  communication  and  guidance.  Whether  this  concerns  showcasing  best 
practices,  or  a  call  for  greater  clarity  regarding  the  integration  of  environmental 
considerations,  the  MSCA  community  seeks  detailed  guidance  on  how  to  effectively 
implement  the  Green  Charter.  The  present  study  also  finds  that  there is  some  grassroots 
demand  for  training  in  specialised  areas,  such  as  Green  Lab  Certification  and  Carbon 
footprinting  for  laboratory  researchers,  though  there  is  also  resistance  at  the  institutional 
level,  which  implies  that  researchers  seek  concrete  skills  and  tools  to  operationalise 
sustainability in their workspaces. 

EU added value 

The  Charter  applies  to  all  MSCA-funded  projects  and  participants,  who  are  expected  to 
commit to its principles on a ‘best effort’ basis. The Charter does not impact the ‘bottom-up’ 
nature nor the scientific excellence of the MSCA, but rather encourages sustainable thinking 
across disciplines in research and project management. Some of the measures suggested 
by  the  Charter  include  reducing  waste,  promoting  green  purchasing,  ensuring  sustainable 

526 Survey of Horizon Europe beneficiaries conducted in May-July 2023 (n=594). 

344 

 
 
 
events,  using  low-emission  transport,  teleconferencing  when  possible,  using  renewable 
energy, raising awareness of sustainability, and sharing best practices.  

Evidence stemming from interviews carried out for the present study reveals that the Green 
Charter  faces  difficulty  in  inspiring  change  at  those  institutions  not  already  invested  in 
sustainability. However, among those already implementing sustainable research practices, 
the MSCA acts as a robust catalyst for enhancement. 

The  survey  findings  and  interviews  also  highlight  the  great  potential  of  the  MSCA  Green 
Charter to increase the visibility of the EU as a supporter of sustainable research, given the 
general  association  between  the  Charter  and  the  European  Commission’s  position  on 
sustainability. In addition, the current Charter was seen by some interviewees as a potential 
avenue towards increasing awareness of discipline-specific sustainable measures as well as 
existing  channels  to  overcome  the  barriers  faced  by  some  projects  under  the  MSCA 
programme. 

Appendix 1. Literature sources 

•  Council Recommendation of 16 June 2022 on learning for the green transition and 

sustainable development 2022/C 243/01  

•  Eco-Management and Audit Scheme (EMAS) (n.d.). Green Business, available at: 

https://green-business.ec.europa.eu/eco-management-and-audit-scheme-emas_en  

•  European Commission Decision C (2021)4200 of 15 June 2021, Horizon Europe Work 

Programme 2021- 2022, 2. Marie Skłodowska-Curie Actions. 

•  European Commission Decision C (2022)7550 of 6 December 2022, Horizon Europe 

Work Programme 2023- 2024, 2. Marie Skłodowska-Curie Actions. 

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 

Learning for the green transition and sustainable development: staff working document 
accompanying the proposal for a Council recommendation on learning for 
environmental sustainability, Luxembourg: Publications Office of the European Union, 
2022, available at: https://data.europa.eu/doi/10.2766/02392 

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 

Tasiopoulou, E., Billon, N., Finlayson, A., et al., Education for environmental 
sustainability: policies and approaches in European Union Member States: final report, 
Tasiopoulou, E. (editor), Billon, N. (editor), Finlayson, A. (editor), Siarova, H. (editor), 
Pribuišis, K. (editor), Gras-Velazquez, A. (editor), Mulvik, I. (editor), Bajorinaitė, M. 
(editor), Sabaliauskas, E. (editor), Fronza, V. (editor), Vežikauskaitė, J. (editor), 
Disterheft, A. (editor), Luxembourg: Publications Office of the European Union, 2022, 
available at: https://data.europa.eu/doi/10.2766/391 

•  European Commission, Directorate-General for Research and Innovation, Kadunc, M., 
Bruno, N., Keeping our eyes on the Horizon: monitoring flash series: a Horizon 2020 
monitoring report, Kadunc, M. (editor), Bruno, N. (editor), Luxembourg: Publications 
Office, 2020, available at: https://data.europa.eu/doi/10.2777/303024 

345 

 
 
•  European Commission, European Research Executive Agency, MSCA cluster event on 

the European Green Deal: meeting report and key messages for policy consideration, 
Luxembourg: Publications Office of the European Union, 2022. 

•  European Commission, Joint Research Centre, GreenComp, the European 

sustainability competence framework, Luxembourg: Publications Office of the 
European Union, 2022, available at: https://data.europa.eu/doi/10.2760/13286 

•  European Parliament, Directorate-General for Internal Policies of the Union, Schreiber, 
H., Tisch, A., Jones, M., et al., Green public procurement and the EU action plan for the 
circular economy, European Parliament, 2017, available at: 
https://data.europa.eu/doi/10.2861/3120 

•  Evaluation study on Excellent Science in the EU framework programmes for Research 

and Innovation. Phase 1 Final Report and its annexes. 

•  Marie Skłodowska-Curie Actions Green Charter – Guidance Materials (2021). 

•  Marie Skłodowska-Curie Actions Green Charter (2021). 

•  Science Europe (n.d.). Greening Research Organisations: Reducing our Carbon 

Footprint, available at: https://www.scienceeurope.org/news/greening-research-
organisations-reducing-our-carbon-footprint/  

Appendix 2. List of interviews 

The list of potential interviewees was mapped on the basis of the sample of respondents to 
the survey launched by the Research and Executive Agency (REA) on MSCA Green Charter 
practices implemented by participating organisations. In total, 12 interviews were planned for 
this  case  study.  As  of  20  March  2024,  the  study  team  had  completed  the  following  13 
interviews with the participants of funded MSCA projects. 

346 

 
 
 
Annex 2.5: Analysis of Transnational and Virtual Access support  

1.1. 

Introduction 

One  of  the  main  objectives  of  the  Research  Infrastructures  programme  part  (INFRA)  with 
regard to the various European Framework Programmes for Research and Innovation has 
always been to support researchers´ access to research infrastructures (RI) across the EU 
(i.e.  transnational  access  or  virtual  access,  TNA/VA).  Until  the  end  of  Horizon  2020, 
transnational access was funded through grants called ‘Integrating Activities’. These grants 
funded consortia of users of a particular set of RI services, and included access, networking 
and  joint  research  (i.e.  where  research  was  aimed  at  improving  the  services  of  the  Ris 
concerned). Over the years, these grants funded more than 80 different communities, some 
of which received several consecutive grants, while some received only one. 

The objective of the present case study is to shed light on the advantages and disadvantages 
of different funding models for TNA/VA. This includes an analysis of the previous system in 
place used under INFRAIA527 instruments; the systems currently in use under INFRASERV 
projects;  as  well  other  possible  alternatives  that  could  mitigate  the  limitations  of  these 
systems.  

The  case  study  is  based  on  evidence  gathered  through  desk  research,  the  analysis  of 
administrative and monitoring data, as well as from interviews with project beneficiaries of 
Horizon 2020 and Horizon Europe projects under the Research Infrastructures programme 
part. 

The  second  section  of  the  case  study  presents  background  information.  This includes  the 
policy context, the main features of TNA/VA under INFRAIA and INFRASERV calls, as well 
as  information  on  the  overall  methodological  approach  followed  for  the  elaboration  of  the 
present case study. The third section of the case study presents the findings of the analysis, 
and goes on to summarise the key success factors and barriers that should be considered in 
the design of future actions to support TNA/VA. The final section presents the conclusions of 
this case study. 

1.2.  Background information 

1.2.1.  Policy context 

The  Horizon  Europe  Regulation528  defines  research  infrastructures  (Ris)  as  ’facilities  that 
provide resources and services for the research communities to conduct research and foster 
innovation in their fields, including the associated human resources, major equipment or sets 
of  instruments;  knowledge-related  facilities  such  as  collections,  archives  or  scientific  data 
infrastructures; computing systems, communication networks and any other infrastructure of 
a unique nature and open to external users, essential to achieve excellence in R&I; they may, 
where relevant, be used beyond research, for example for education or public services and 
they may be “single sited”, “virtual” or “distributed”. 

527 INFRAIA is short for ‘integrating and opening research infrastructures of European interest’. 
528 Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 2021 
establishing Horizon Europe – the Framework Programme for Research and Innovation, laying 
down its rules for participation and dissemination, and repealing Regulations (EU) No 1290/2013 
and (EU) No 1291/2013. 

347 

 
 
With  an  indicative  budget  of  EUR  2,186  million  for  the  period  2021-2027,  INFRA  is  the 
smallest programme part of Pillar 1 ‘Excellent Science’ under Horizon Europe. The Horizon 
Europe Regulation indicates four main areas of intervention for this programme part529: 

•  Consolidating and developing the landscape of European research infrastructures; 

•  Opening, integrating and interconnecting research infrastructures;  

•  Fostering the innovation potential of European research infrastructures and activities for 

innovation and training; and 

•  Reinforcing European research infrastructure policy and international cooperation. 

By addressing these areas, INFRA will contribute, among others530, towards: 

•  Providing Europe with world-class sustainable research infrastructures that are open and 

accessible to the best researchers from Europe and beyond.  

•  Encouraging the use of existing research infrastructures, including those financed from 

funds under the Union´s Cohesion Policy531.  

Action 8 of the ERA Policy Agenda approved in 2022532 was specifically addressed towards 
research infrastructures, namely to ‘Strengthen sustainability, accessibility and resilience of 
research infrastructures in the ERA’. This action is particularly relevant to the present case 
study, as it refers to the need, among others, to ‘ensure sustainable funding and investments 
for the existing research infrastructures to maintain excellence and competitiveness of the 
ERA  in  the  long  term’  and  to  ‘give  all  researchers  and  innovators  in  Europe  the  same 
opportunities to access to the services provided by the European infrastructures’533.  

Facilitating access to infrastructures across the EU therefore remains at the core of EU policy 
action in this field, in a context in which pathways to achieve sustainable funding are still yet 
to  be  discussed.  Under  the  Framework  Programme,  transnational  and  virtual  access 
(TNA/VA) to Ris is provided free of charge to researchers or research teams (for selected 
proposals – please note that no selection is made for virtual access unless in cases of limited 
capacity). ‘Virtual access’ refers to remote access to services such as software, or to data 
archives.  

529 Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 2021 
establishing Horizon Europe – the Framework Programme for Research and Innovation, laying 
down its rules for participation and dissemination, and repealing Regulations (EU) No 1290/2013 
and (EU) No 1291/2013. 
530 Activities under INFRA are also expected to foster the potential for research infrastructures to 
support scientific advance and innovation, and to enable open and excellent science in 
accordance with the FAIR principles, alongside activities related to Union policies and 
international cooperation. 
531 Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 2021 
establishing Horizon Europe – the Framework Programme for Research and Innovation, laying 
down its rules for participation and dissemination, and repealing Regulations (EU) No 1290/2013 
and (EU) No 1291/2013. 
532 European Commission, Directorate-General for Research and Innovation, European 
Research Area policy agenda : overview of actions for the period 2022-2024, Publications Office 
of the European Union, 2022, https://data.europa.eu/doi/10.2777/52110. 
533 Other areas covered were the need to ‘Increase the overall impact of the investments made 
in research infrastructures on economy and society’, and to ‘Have a more targeted priority 
setting to focus on specific scientific and political needs, taking into account the already available 
capacities and services, main scientific developments and key societal challenges’. 

348 

 
 
Under  Horizon  2020,  TNA/VA  to  research  infrastructures  was  essentially  granted  through 
INFRAIA  calls.  These  calls  had  a  triple  focus,  targeting  TNA/VA,  joint  research  and 
networking activities. The Horizon 2020 ex-post evaluation of the Research Infrastructures 
programme  under  the  Excellent  Science  Evaluation  Study  indicated  that  in  general, 
beneficiaries positively evaluated having these three activities funded jointly by INFRA, as 
this  were  considered  important  in  helping  to  build  trust  among  partners,  to  increase  the 
visibility of the actions in a more efficient way, and to facilitate the provision of (new) services 
and  the  emergence  of  research  activities  as  a  result  of  the  calls  for  access.  In  addition, 
Horizon  2020  introduced  a  distinction  between  Starting534  and  Advanced535  Communities. 
This  change  better  addressed  the  differing  needs  of  these  two  types  of  communities,  and 
was very positively perceived by the RI community, according to the individuals interviewed 
in  the  context  of  the  Evaluation  study  on  Excellent  Science  under  Horizon  2020  and  this 
study.  

Under Horizon Europe, these activities have been split, with some calls specifically targeting 
support for TNA/VA based on a challenge-driven approach (INFRASERV) and joint research 
activities  on  the  next  generation  of  scientific  instrumentation,  tools  and  methods 
(INFRATECH). 

1.2.2.  Overall approach and scope of the case study 

The methodological steps applied in the elaboration of this case study are as follows: 

- 

First, desk research was conducted on INFRA-related documents and reports. This 
included an analysis of European Commission reports and documents; analysis of 
the  outcomes  of  the  public  consultation  on  the  past,  present  and  future  of  the 
European  Research  &  Innovation  Framework  Programmes  2014-2027536  (further 
referred to as the ‘Public consultation’), specifically the position papers published by 
stakeholders in context of this consultation; project monitoring and reporting data; 
as well as documents and reports produced by INFRA projects (e.g. ERIC Forum) 
and other relevant stakeholders (ESFRI).  

-  EC monitoring and reporting data was consulted in order to gather information on 

the patterns of participation and of users in INFRAIA and INFRASERV calls. 

- 

interviews  with  beneficiaries 

21 
INFRAIA  projects  (Horizon  2020), 
from 
INFRASERV  projects  (Horizon  Europe)  and  other  stakeholders  provided  more  in-
depth information about the perceptions of beneficiaries and the ways forward they 
suggested  to  support  TNA/VA.  These  interviews  were  complemented  with 
exchanges with European Commission officials working on INFRA. 

1.3.  Case study findings 

TNA/VA offers clear EU added value in the sense that EU national infrastructures and funding 
agencies usually do not support researchers´ access to research instruments, services and 
infrastructure located abroad. Supporting TNA/VA at EU level therefore complements what 
is  available  at  national  level,  while  at  the  same  time  helping  to  structure  and  foster 
coordination within the EU landscape of research infrastructures. The ways in which support 
to  TNA/VA  can  be  provided  at  EU  level  are,  however,  varied.  This  case  study  presents 

534 A 'Starting Community' has never been supported for the integration of its infrastructures 
under FP7 or Horizon 2020 calls, in particular within an integrating activity. 
535 'Advanced Communities' are scientific communities whose research infrastructures show an 
advanced degree of coordination and networking at present, attained, in particular, through 
Integrating Activities awarded under FP7 or previous Horizon 2020 calls. 
536 Available at: https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-
Horizon-Europe-interim-evaluation/public-consultation_en. 

349 

 
 
various possible models that have been or could be implemented to support TNA/VA under 
the  Framework  Programme.  These  models  were  identified  through  discussions  with  the 
European  Commission  and with  participants in  INFRA.  It  is  important  to  bear in  mind  that 
those  models  which  have  never  been  applied  under  INFRA  are  analytical  suggestions 
intended to help compare and contrast the advantages and disadvantages of each model. 
The possible implementation of these hypothetical models would require in-depth analyses, 
discussions with the RI community and the development of technical specifications. 

Table 1 presents the six models identified for this case study. 

Experience with 
the system 

TNA/VA model 

Main features 

Used during 
Horizon 2020 

INFRAIA model 

Applied under 
Horizon Europe 

INFRASERV model 

This model was in place under Horizon 2020 for the 
INFRA Work Programmes. It covered support for 
TNA/VA, networking and joint research activities. 
INFRAIA calls brought together consortia of similar 
types of Ris. 

The INFRASERV model was introduced under 
Horizon Europe INFRA Work Programmes. It only 
covers support for TNA/VA activities, and is mainly 
challenge-oriented. Some calls under Horizon 
Europe have also funded curiosity-driven access. 

Hypothetical 
alternative 

Centralised model 

A hypothetical model consisting of setting up an EC 
agency or body that would centralise TNA/VA calls 
across domains. 

Hypothetical 
alternative 

Cascade funding 

Hypothetical 
alternative 

ERICs as central 
points 

Hypothetical 
alternative 

Distributed TNA/VA 
support 

Cascade funding has never been applied under 
INFRA Work Programmes, but has been applied for 
quite some time under other parts of the Framework 
Programme. In a cascade funding model, a relatively 
small consortium would organise the TNA/VA calls, 
select a number of users that are eligible for TNA/VA, 
and connect them with the relevant service providers. 
The main difference with the current TNA/VA models 
is that not all service providers are beneficiaries, and 
therefore available facilities are not necessarily 
identified when a project is evaluated. 

ERICs are a specific legal form authorised by the 
European Commission that facilitates the 
establishment and operation of research 
infrastructures with European interest. This model 
would rely on the definition of ERICs as central 
points in the distribution of TNA/VA support and the 
structuring of the landscape of Ris (i.e. connecting 
researchers with services). 

Under this model, support for TNA/VA would be 
embedded into actions under other Work 
Programmes (e.g. researchers funded by the ERC 
would have access to specific funding to access 
research infrastructures abroad). TNA/VA would 
therefore not be supported under INFRA. 

Table 1. Overview of TNA/VA models identified for analysis in this case study. Source: compiled by the study team, based 
on interviews with the European Commission and INFRA beneficiaries. 

350 

 
Each  of  the  above  models  is  further  explained  below,  and  their  advantages  and 
disadvantages are listed in relation to the various evaluation criteria used in this case study 
(relevance, effectiveness, efficiency, coherence).  

1.3.1. 

INFRAIA model   

This model, used under Horizon 2020 in the INFRA Work Programmes, supported TNA/VA, 
networking  and  joint  research  activities.  The  INFRAIA  call  had  a  vertical  focus,  bringing 
together users of similar instruments and disciplines. For example, the iNEXT project537 (one 
of the top 5 INFRAIA projects in terms of number of instances of TNA/VA) provided access 
to high-end structural biology instruments and expertise for a total of 949 researchers coming 
almost exclusively from the  domain of life sciences538. On  average, each INFRAIA project 
attracted researchers from one or two activity domains.  

1.3.1.1. 

Advantages 

•  Beneficiaries of this system generally had a positive perception of this instrument, as it 
provided  communities  with  the  continuity  to  develop  and  upgrade  services  and 
instruments through successive INFRAIA grants. The INFRAIA instrument played a key 
role in bringing together similar pan-EU RIs and other national-level infrastructures and 
instruments, and fostering collaboration between them. The INFRAIA grants provided a 
stable and predictable source of funding for community building and access provision. 
The change towards INFRASERV has disrupted this continuity for the communities, as 
also indicated by EARTO in their position paper submitted to the OPC which states that 
‘the infrastructure networks that were launched under Horizon 2020 no longer have the 
means to ensure their sustainability as networks’539.  

•  The combination of TNA/VA, joint research and networking activities was perceived 
positively  by  some  beneficiaries  interviewed.  Some  specifically  highlighted  the 
complementarity of TNA/VA and joint research activities: access to facilities led to new 
ideas for joint research, and joint research, in turn, improved the quality of the TNA/VA 
services offered to researchers. Other beneficiaries interviewed, however, indicated that 
these are different activities with limited spill-over, and they therefore found it logical that 
joint research is now funded by separate grants (INFRATECH). In addition, the funding 
of networking was considered helpful for fostering collaboration within communities and 
building trust, although interviewees indicated that they still include networking activities 
within their projects’ work programmes even if they are not explicitly funded (e.g. under 
INFRASERV).  Several  position  papers  submitted  to  the  OPC  express  regret  over  the 
disappearance  of 
‘the 
disappearance of the call for integrative activities which, in addition to supporting access 
to  infrastructures  for  researchers,  also  enabled  research  activities  to  improve  the 

integrative  activities.  One, 

instance, 

indicated 

that 

for 

537 https://cordis.europa.eu/project/id/653706. 
538 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European framework programmes 
for research and innovation for excellent science: Horizon 2020: annexes, Luxembourg: 
Publications Office of the European Union, pp. 848-858, 2023, 
https://data.europa.eu/doi/10.2777/353383. 
539 EARTO (2023) Position Paper regarding the ‘Public consultation on the past, present and 
future of the European Research & Innovation Framework programmes 2014-2027’, available at: 
https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-Horizon-Europe-
interim-evaluation/public-consultation_en. 

351 

 
 
performance  of 
competitiveness’540. 

research 

infrastructures, 

is  a  weakness 

for  European 

•  Beneficiaries  also  positively  assessed  the  distinction  made  between  Starting 
Communities and Advanced Communities. This distinction in terms of the topics of 
calls was felt to better reflect the needs of communities with different levels of maturity.  

1.3.1.2. 

Disadvantages 

•  One of the main issues with INFRAIA grants was the large (and increasing) number 
of  communities  requesting  funding  under  INFRAIA.  Instead  of  integrating  the  RI 
landscape, it appeared that the landscape was becoming more fragmented and siloed 
into  groups  of  RIs  with  similar  instruments,  attracting  researchers  from  the  same 
discipline. Indeed, some of the Starting Communities also appeared to be subsections 
of  a  previous  Advanced  Community,  and  thus  not  ‘new’  in  the  sense  intended  by  the 
instrument. The high level of attractiveness and large number of communities requesting 
funding  under  INFRAIA  meant  that  not  all  communities  could  be  continuously  funded 
under  INFRAIA.  This  jeopardised  the  long-term  stability  of  the  user  communities,  as 
TNA/VA is usually not funded by national funders and organisations. 

•  The  visibility  of  TNA/VA  calls  with  regard  to  researchers  may  also  have  been 
suboptimal.  Each  INFRAIA  project  advertised  its  open  calls  through  its  own  project 
website (in total, there were 103 INFRAIA grants), resulting in the lack of an overview for 
researchers regarding the range of calls available under the various projects. This also 
led  to  a  multiplication  of  communication  and  dissemination  efforts,  as  each  project 
needed to build its own image, reputation and visibility from scratch, and its effects would 
only last for the lifetime of the project. 

•  Under INFRAIA, the administrative burden for reporting on TNA/VA costs was very 
high,  during  both  the  proposal  preparation  phase  and  the  project  execution  phase. 
Service providers and facilities under this system were required to specify each service 
they would offer, including its associated costs, in the proposal. These costs could be 
direct  (personnel  costs,  consumables)  and  indirect  (energy,  maintenance,  software 
licenses). Different methods of cost calculation existed: unit costs541; actual costs542 or a 
combination  of  both.  The  calculation  of  these  costs  was  a  very  complex  process, 
especially for Starting Communities or new participants with numerous services. There 
were also difficulties relating to the need to estimate the demand for certain services at 
proposal  phase,  which  was  especially  challenging  for  new  services.  Interviews  with 
project beneficiaries indicate, however, that the system was very time-consuming at the 
beginning, but once they were within the system, beneficiaries could continue to use it 
and it became less burdensome for them. Changes to the cost reporting system should 
therefore  be  carefully  balanced  with  the  potential  benefits.  On  the  other  hand, 
interviewees indicate that clearer guidelines and templates could have been developed.  

540 CEA, Position paper regarding the ‘Public consultation on the past, present and future of the 
European Research & Innovation Framework programmes 2014-2027’, 2023, available at: 
https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-Horizon-Europe-
interim-evaluation/public-consultation_en. 
541 Unit costs are calculated as the annual total access costs divided by the average annual total 
units of access provided. For virtual access, only the data for the last year are used, due to fast-
evolving technologies. 
542 In this method, only actual costs incurred for providing access are considered, with a 25% flat 
rate added for indirect costs. 

352 

 
 
1.3.2. 

INFRASERV challenge-driven model 

The  challenge-driven  INFRASERV  model  is  used  under  the  Horizon  Europe  Work 
Programmes. Under this model, TNA/VA support is provided to user communities working on 
the same challenge (e.g. cancer, climate risks, etc.). Instead of bringing together researchers 
from a similar discipline using similar instruments, it brings together researchers working on 
the  same  challenge,  but  gathers  together  different  types  of  instruments  and  services.  For 
example,  the  canSERV  project543  which  received  a  grant  under  INFRASERV,  provides  a 
broad  array  of  services  that  are  relevant  for  cancer  research,  most  notably  access  to 
infrastructure  and  equipment;  support  in  the  design  of  clinical  studies;  and  support  in  the 
interpretation of results. Moreover, researchers can apply for (transnational) access to these 
different services within a single platform544.  

The main advantages and disadvantages of the INFRASERV model are summarised below. 
However, for a more in-depth analysis of the impact of the challenge-based approach, please 
refer to Case Study 6, ‘Analysis of the use of the challenge-oriented approach in the INFRA 
Work Programmes’ in this study.  

1.3.2.1. 

Advantages 

•  The challenge-oriented approach contributes to the relevance of TNA/VA as it aims to 
increase  the  contribution  of  RIs  to  societal  challenges  and  establishes  a  clear 
connection  with  European  Commission  priorities  such  as  the  twin  green  and  digital 
transitions, as well as with the Horizon Europe Missions. As indicated in the 2021-2022 
Work  Programme,  ‘The  provision  of  services  at  EU  level  has  been  so  far  mainly 
organised per types of infrastructures or disciplines. The complexity and urgency of the 
socio-economic  and  environmental 
requires 
interdisciplinary  approaches  and  a  new  challenge-driven  provision  of  customised 
services  able  to  accelerate  the  pace  of  the  research  cycle  and  the  delivery  of 
solutions’545. 

that  Europe 

transition 

facing 

is 

• 

• 

INFRASERV  calls  foster  larger  interdisciplinary  consortia  working  on  the  same 
challenge. This has enabled the inclusion of new service providers and organisations, 
which  is  perceived  by  some  beneficiaries  as  a  positive  evolution  compared  with  the 
previous system. INFRASERV also fosters new collaborations between RIs and enables 
harmonisation and integration between them.  

Interviews with beneficiaries indicate that the focus on challenges has the potential to 
make services more visible to a broader set of users. Under INFRAIA, the TNA/VA 
support  offered  by  specific  user  projects  was  mostly  visible  to  users  in  the  same 
discipline, working on similar instruments. In INFRASERV, services and instruments are 
brought  together  from  a  challenge-oriented  perspective,  which  allows  researchers 
working on a specific challenge to discover new RIs and services. At present, it is too 
soon, however, to assess the real impact of challenge-oriented projects: this will need to 
be monitored once projects report on the users to whom they are providing services. 

1.3.2.2. 

Disadvantages  

• 

Interviewees tend to focus first and foremost on the negative impact of this model from 
the  perspective  of  efficiency.  The  number  and  diversity  of  instruments  and  services 

543 https://cordis.europa.eu/project/id/101058620. 
544 https://www.canserv.eu/calls/. 
545 European Commission Decision C(2022)2975 of 10 May 2022: Horizon Europe Work 
Programme 2021-2022, Research Infrastructures, pp. 61. Available: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-call/2021-
2022/wp-3-research-infrastructures_horizon-2021-2022_en.pdf. 

353 

 
 
offered for TNA/VA under a single project has increased the size of consortia, and with 
that, the complexity of projects and their related administrative burden. The cost 
reporting system, as explained under the section on INFRAIA, has remained largely the 
same.  However,  the  related  administrative  burden  has  increased  as  the  number  of 
service  providers  has  increased,  with  some  projects  offering  up  to  400  services.  The 
ERIC Forum confirms the issue voiced by many interviewees and position papers: ‘while 
integration of research services and projects that rely on services and resources from 
several  different  RIs  are  encouraged  and  would  greatly  increase  research  impact, 
administrative hurdles in TNA and project management (e.g. large number of contractual 
partners,  (lack  of)  granularity  of  service  catalogues,  budget  administration  and  cost 
accounting), lead to long project ramp-up phases and correspondingly decreased time 
for  service  delivery.  This  does  disproportionately  affect  the  completion  of  ambitious, 
highly impactful projects and cross-RI collaborations’546. 

•  Another implication of having a large number of partners within one project is the dilution 
of funding. The budgets of INFRASERV projects have not increased in comparison to 
INFRAIA, while the number of service providers has increased significantly. Interviewees 
indicate  that  the  level  of  funding  received  may  not  be  worth  the  effort  required  for 
proposal writing and coordination.  

•  Some of the beneficiaries interviewed expressed concerns about the sustainability of 
the services developed in previous and current projects. The previous INFRAIA grants 
provided communities with continuity and allowed them to upgrade previously developed 
services. Now, some interviewees fear there is too much focus on innovation and not 
enough funding to continue offering successful services developed in previous projects. 
Some interviewees pointed to the necessity of continuing cluster projects547, which foster 
the  integration  of  RIs  within  the  same  scientific  domain.  Interviewees  also  expressed 
concerns  regarding  the  sustainability  of  services  developed  in  current  INFRASERV 
projects beyond the lifetime of the project (typically three to four years).  

•  Beneficiaries who were interviewed also stressed the high transition costs incurred in 
adapting to the new INFRASERV system, notably in adapting to the new structure of the 
INFRA Work Programme and finding the relevant calls, forming new consortia, proposal 
writing, adjusting administrative processes, building trust in the consortium and devising 
efficient collaboration methods, etc. Interviewees stressed that any shift to a new system 
should  be  balanced  against  potential  benefits,  with  many  indicating  that  further  large 
changes to the structure of future INFRA Work Programme should be avoided.  

1.3.3.  Centralised model 

This  model  is  a  hypothetical  scenario,  designed  on  the  basis  of  discussions  with  the 
European Commission and the beneficiaries interviewed. Under this model, TNA/VA would 
be  offered  by  a  centralised  body  or  agency  at  the  European  Commission  through  a 
permanent or semi-permanent call (e.g. with a pre-established frequency per year) across 
disciplines and topics.  

1.3.3.1. 

Advantages 

•  The  centralisation  of  TNA/VA  support  would  increase  the  visibility  of  support  for 
researchers, as all opportunities would be provided by a central agency that acts as a 

546 ERIC Forum (2023), Position paper regarding the ‘Public consultation on the past, present 
and future of the European Research & Innovation Framework programmes 2014-2027’, 
available at: https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-
Horizon-Europe-interim-evaluation/public-consultation_en. 
547 For example funded under https://ec.europa.eu/info/funding-
tenders/opportunities/portal/screen/opportunities/topic-details/infraeosc-04-2018. 

354 

 
 
‘one-stop  shop’  for  researchers  seeking  TNA/VA  support.  This  contrasts  with  the 
situation under INFRAIA and INFRASERV, whereby each project publishes its open calls 
for  TNA/VA  on  its  own  website  and  no  central  repository  of  opportunities  exists  that 
researchers can check for their own purposes.  

•  Centralisation  is  likely  to  offer  more  flexibility  to  adapt  the  orientation  of  TNA/VA 
support in response to certain crises or events that require increased scientific attention 
(e.g.  pandemics).  When  facing  such  situations,  the  agency  would  be  able  to  direct 
TNA/VA support towards these challenges in a quicker and more agile manner. 

•  This  model  presents  a  stronger  degree  of  sustainability.  Through  the  creation  of  a 
body  or  agency  in  charge  of  providing  TNA/VA  support,  access  could  be  sustainably 
provided beyond the lifetime of individual projects.  

1.3.3.2. 

Disadvantages 

•  An important aspect of INFRAIA and INFRASERV projects is the development of new 
services through networks of RIs and facilities working together and building new user 
communities. To address the need for new and upgraded services, one can envisage 
separate grants for the development of services, which could be added to the centralised 
catalogue. 

• 

Interviewees point out that RIs would still require funding for training TNA users on how 
to  use  the  instruments  and  services.  Training  is  an  important  part  of  TNA/VA,  and 
contributes to the research careers and skills of researchers.  

•  Several interviewees pointed out that a central catalogue for TNA/VA calls  might not 
respond to the needs of users; nor might it accurately reflect the diverse characteristics 
of the RIs and installations. To counter this issue, two similar options for centralisation 
were suggested:  

–  Some interviewees pointed to the role of EOSC and the potential impact it could 
have  by  interlinking  different  portals  without  creating  new  ones.  Information 
about services and the calls for access with regard to various challenges (e.g. 
COVID-19,  cancer,  etc.)  could  be  gathered  together  and  displayed  in  a  clear 
manner.  

–  Another approach could be to have an EC ‘travel agency’ that only funds the 
travel costs of TNA users, while the platforms to which they apply for various 
RI  services  would  remain  decentralised.  Some  interviewees  saw  this  as  an 
interesting  option  because  the  instruments  themselves  are  typically  already 
funded through national funding, while travel costs for foreign researchers are 
not.  Such  an  approach  would  alleviate  the  administrative  burden  on  service 
providers  in  terms  of  cost  reporting  and  refunding.  However,  interviewees 
doubted that this would make TNA/VA easier for users, as they would have to 
apply twice (once to the RI, and once to the travel agency). Moreover, such a 
model might not be of interest to some Ris – for example, those offering services 
relating to high-performance computing, which can come with significant energy 
costs.  

•  Some fields of science tend to make greater use of resources than others. Some user 
communities are very well established, while others are only emerging. In the event of a 
centralised  call  appearing,  it  seems  likely  that  researchers  from  well-established 
communities  would  be  much  more  likely  to  apply  than  those  from  emerging 
communities.  

355 

 
•  Selecting excellent user projects would be more difficult if a centralised jury needs to 
assess proposals that come from a very broad range of scientific disciplines. Composing 
a fit-for-purpose selection board would be a challenge. One potential solution could be 
to have one selection board per scientific discipline.  

• 

Lastly, the establishment of such a body or agency in charge of managing and funding 
TNA/VA  support  would  require  a  non-negligible  amount  of  European  resources  both 
financially, logistically (IT platform) and in terms of human resources.  

1.3.4.  Cascade funding 

This model is another hypothetical scenario, and has never been applied under the INFRA 
Work Programme. However, the model has been used under other parts of the Framework 
Programme, such as under the Innovation in SMEs programme (INNOSUP). The cascade 
funding  model  could  further  be  combined  with  the  challenge-oriented  approach  of  the 
INFRASERV  model,  or with the  more  discipline-oriented  approach  of  the  INFRAIA  model. 
The main difference compared with the current implementation of INFRAIA or INFRASERV 
would be that not all service providers are beneficiaries. Instead, a relatively small consortium 
would organise the TNA/VA calls, select a number of users that are eligible for TNA/VA, and 
connect them with the relevant service providers.  

Such a system might significantly reduce the administrative burden currently perceived for 
INFRASERV projects. As indicated in a stakeholder position paper in the context of the OPC: 
‘Should  the  broad  challenge-driven  access  to  RI  be  maintained,  we  suggest  to  provide 
several  financial  options  (cascade  grants,  generous  subcontracting  options,  purchase  of 
services), so that projects can choose the best-suited option’548. 

1.3.4.1. 

Advantages 

•  Under this  model,  consortia can  be  smaller (e.g.  four to  five  partners)  which  reduces 
coordination  costs  within  the  consortium  compared  with  under  INFRAIA  and 
INFRASERV. However, there are still coordination costs associated with the coordination 
of the network of service providers. The ways in which these networks can be built are 
diverse: for example, 1) they can have a formal structure – e.g. through subcontracting 
agreements with the core partners of the consortium; 2) they can have a less formalised 
structure – for instance, based on Memorandum of Understanding (MoUs) or letters of 
intent signed during the project application phase. Other formulas can also be used, such 
as setting up accreditation procedures for service providers via which they would become 
part  of  the  network  if  they  comply  with  a  series  of  requisites  (e.g.  offering  relevant 
services in line with the scope of the action, providing adequate budgeting of the cost of 
the TNA/VA accessed, etc.). 

•  The resources required for proposal writing could be reduced. Under INFRAIA and 
INFRASERV, all services and associated costs needed to be detailed in the proposal. 
This entailed significant effort at the proposal writing stage, especially for new service 
providers.  In  a  cascading  approach,  services  would  only  be  budgeted  after  being 
requested by researchers.  

•  A cascading system would bring greater flexibility. It is difficult to know at the proposal 
stage how many users will be attracted, especially for new RIs. Cascading gives flexibility 

548 Helmholtz Association (2023). Position Paper regarding the “Public consultation on the past, 
present and future of the European Research & Innovation Framework programmes 2014-2027’, 
Available at: https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-
Horizon-Europe-interim-evaluation/public-consultation_en. 

356 

 
 
as to how to distribute money, and provides freedom to reallocate funds to those services 
that are actually requested by the users. 

1.3.4.2. 

Disadvantages 

•  Participants in the INFRA programme are largely inexperienced in the use of cascading 
grants. Clear guidelines would be required in order to implement such a system. As noted 
earlier, any change is accompanied by transition costs; thus, the potential benefits and 
drawbacks of such a system would need to be carefully studied before adopting it.  

•  The cascading approach may have negative effects for community building, as not 
all  service  providers  are  part  of  the  consortium.  The  RIs  that  are  not  part  of  the 
consortium might feel left out and, if no action were taken to encourage new participation, 
it might also be more difficult for new service providers to enter the network.  

1.3.5.  ERICs as central points 

ERICs – European Research Infrastructure Consortia – are a specific legal form authorised 
by  the  European  Commission  that  facilitates  the  establishment  and  operation  of  research 
infrastructures  of  European  interest549.  ERICs  have  the  following  characteristics:  a  legal 
capacity  recognised  in  all  EU  countries;  flexibility  to  adapt  to  the  specific  requirements  of 
each  infrastructure;  they  are  built  according  to  a  process  that  is  faster  than  creating  an 
international  organisation;  and  they  are exempt  from  VAT  and  excise  duty. By  forming  an 
ERIC, organisations and the instruments available in different countries are able to operate 
in a more coordinated manner and have the opportunity to participate jointly in calls under 
the Framework Programme as a unique participant, which reduces the costs of participation 
for the individual entities that make up the ERIC.  

The use of this model would rely on the definition of ERICs as central points in the distribution 
of  TNA/VA  support  and  structuring  the  landscape  of  RIs  (i.e.  connecting  researchers  with 
services). Under this model, INFRA calls for TNA/VA would exclusively target consolidated 
ERICs – or networks of ERICs – as beneficiaries. Individual ERICs or consortia formed by 
ERICs would be in charge of: 1) developing the network of instruments and services around 
the  topics  covered  in  the  text  calls;  and  2)  offering  and  managing  TNA/VA  support  to  the 
entities included in the network. The networks developed would include entities involved in 
the ERICs, but also other entities and instruments outside the ERIC(s) that were in line with 
the scope of the project. Under such a system, ERICs would be the key actors distributing 
TNA/VA support. A similar mechanism was proposed by the ERIC Forum in its 2020 Policy 
Brief550, namely: ‘the creation of a centralized budget managed by ERICs that could fully fund 
research  projects,  selected  by  the  ERIC  upon  scientific  merit  and  excellence,  could  be 
explored.  This  budget  could  be  composed  of  a  common  contribution  of  Member  States 
combined with European funding’. This approach would combine funds for TNA/VA as well 
as for the research project itself, which is especially relevant to some ERICs, for whom it is 
an  issue  that  the  source  of  TNA  funding  in  relation  to  multinational  research  projects  is 
different from that of funding for overall research costs.   

1.3.5.1. 

Advantages  

•  A  structure  involving  the  use  of  ERICs  could  allow  many  more  organisations  to  be 
brought  into  projects  (their  nodes)  without  all  of  these  being  formally  including  into  a 

549 https://research-and-innovation.ec.europa.eu/strategy/strategy-2020-2024/our-digital-
future/european-research-infrastructures/eric_en. 
550 ERIC Forum (2020). Policy Brief, Funding models for Access to ERIC 
Multinational/Transnational Services, p. 25, available at: https://www.eric-forum.eu/wp-
content/uploads/2020/09/ERIC-Forum_Policy-Brief.pdf. 

357 

 
 
consortium551. This would entail efficiency savings in terms of project coordination and 
cost reporting, similar to the cascade funding approach. ERICs already gather together 
very prominent instruments and organisations under a common umbrella, which attests 
to their strong coordination capacity and their ability to efficiently and effectively manage 
the addition of members to their network within the lifetime and scope of specific projects. 
However, ERICs do not yet enjoy such flexibility under the current TNA scheme, as also 
noted in the 2023 position paper552 mentioned above. Instead– like all other beneficiaries 
– they still need to specify all services and facilities during the proposal phase.  

•  The  fact  that  ERICs  are  approved  by  the  European  Commission  is  considered  by 
some interviewees to be proof of their quality and commitment; they know their domains 
well and are well connected to other types of infrastructure. 

1.3.5.2. 

Disadvantages 

•  Some interviewees expressed concerns about giving such a prominent role to ERICs. 
One such concern relates to the fact that ERICs already have an established position in 
the field of RI. From this perspective, focusing too much on ERICs might entail the risk 
of giving insufficient support to emerging communities (in both established and new 
topics).  This  is  seen  by  some  interviewees  as  something  that  might,  in  the  long  run, 
jeopardise the EU´s research capacity and competitiveness, and could be an additional 
barrier to new instruments and institutions entering into and becoming connected with 
the broader EU RI landscape. In this sense, it is argued that it would be positive to retain 
a certain level of competitiveness in the programme part.  

•  The  aim  is  for  ERICs  to  become  financially  self-sustainable,  notably  by  securing 
contributions  from  Member  States.  Providing  a  stable  source  of  EU  funding  through 
TNA/VA support would be counter-productive in this sense.  

•  The  number  of  ERICs  currently  remains  limited,  and  together  they  may  not  cover  all 
scientific  domains.  In  certain  domains,  relying  on  ERICs  as coordinators  of  TNA/VA 
support might therefore not (yet) be a realistic possibility.  

1.3.6.  Distributed TNA/VA support 

This  model  refers  to  a  significantly  different  approach:  support  for  TNA/VA  would  be 
associated  with  participation  in  other  parts  of  the  Framework  Programme.  This  could  be 
achieved, for example, by embedding support into actions under other Work Programmes 
(e.g. the ERC Work Programme could earmark some funding for TNA/VA so that researchers 
funded  by  the  ERC  would  have  access  to  specific  funding  in  order  to  access  research 
infrastructures abroad). Another option would be for the INFRA Work Programme to design 
specific instruments and to allocate funding for TNA/VA requests from participants in other 
programme parts. This approach is relatively common in national research funding schemes, 
where research funding to individual researchers, research teams or consortia of universities 
can  be  complemented  with  funding  to  allow  access  to  (usually  national)  research 
infrastructure. 

551 This is already possible in current Horizon Europe projects. 
552 ERIC Forum (2023). Position Paper regarding the ‘Public consultation on the past, present 
and future of the European Research & Innovation Framework programmes 2014-2027,. 
available at: https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-
Horizon-Europe-interim-evaluation/public-consultation_en. 

358 

 
 
 
A similar approach was also proposed by the ERIC Forum in its 2020 Policy Brief553, which 
stated:  ‘expanding  the  recognition  of  TNA  as  an  eligible  cost  across  the  European 
Commission’s actions on research and innovation, and not limiting it to programmes aiming 
at  the  implementation,  integration,  operation  and  development  of  RIs,  as  is  currently  the 
case’.  The  ERIC  Forum  perceives  this  as  a  way  to  fund  TNA  and  multinational  research 
projects through one grant, which would be beneficial for some ERICs. 

1.3.6.1. 

Advantages 

•  This  model  could  encourage  the  visibility  and  use  of  RIs  in  other  research  and 
innovation actions funded under the Framework Programme. This could also lead to a 
stronger  connection  between  research  infrastructures  and  other  actors  along  the 
innovation value chain (e.g. technology centres or industry, including SMEs). 

•  By  linking  TNA/VA  support  to  participation  in  other  actions  within  the  Framework 
Programme,  it  is  possible  that  a  better  alignment  could  be  achieved  between 
programme  parts  with  respect  to  the  work  carried  out  by  research  infrastructures  and 
their contribution to other parts of the Framework Programme. 

1.3.6.2. 

Disadvantages 

•  The  main  expected  outcome  of  this  approach  is  that,  by  only  serving  Framework 
Programme participants, the number of users is likely to be smaller than in a system 
in which access is open to all EU researchers (e.g. INFRAIA, INFRASERV).   

•  Setting  up  such  a  system  would  require  financial  and  human  resources  from  the 
European Commission as well as the adaptation of the IT instruments used to manage 
and process grants, to ensure that requests coming from participants in other programme 
parts can be processed smoothly and seamlessly. 

1.4.  Success factors and barriers 

The previous section presents various models via which support for TNA/VA can be provided 
at  EU  level,  together  with  their  associated  advantages  and  disadvantages.  The  choice 
regarding which models should be applied in the future will depend on the combination of 
needs and the political priorities attributed to each of them. Analysis of the models’ various 
advantages  and  disadvantages  points  to  the  need  to  consider  the  following  factors  when 
funding TNA/VA: 

•  Ensuring  that  TNA/VA  support  is  provided  in  a  wide  and  comprehensive  manner, 
moving away from supporting communities that are too narrow, with little margin to grow 
their user base, and ensuring openness to new service providers and organisations.  

•  Combining support for communities in alignment with political priorities and societal 
challenges, while also supporting access to instruments and services in other areas 
of science. Some of the beneficiaries interviewed stressed that science which might be 
less politically relevant at one point might become vital in the future, hence the need to 
maintain the support across all fields. 

•  Achieving  efficiency  gains  and  minimising  recurring  and  superfluous  costs  (e.g. 

excessive coordination costs with little added value). 

553 ERIC Forum (2020). Policy Brief, Funding models for Access to ERIC 
Multinational/Transnational Services, pp. 25, Available at: https://www.eric-forum.eu/wp-
content/uploads/2020/09/ERIC-Forum_Policy-Brief.pdf. 

359 

 
 
•  Ensuring  the  long-term  stability  and  sustainability  of  the  service  offer  created  in 
projects  under  INFRA.  The  creation  of  these  service  offers  takes  time  and  effort; 
according to the beneficiaries interviewed, a greater impact could be expected if TNA/VA 
and the associated services could be planned for longer time periods (compared with 
those  under 
INFRASERV),  with  corresponding  monitoring  and 
evaluation systems to check the extent to which the service offer and the mechanisms 
built to support TNA/VA are meeting the expected results. Some interviewees suggested 
the use of scientific roadmaps. These are seen as a way to: 

INFRAIA  and 

−  better structure the service offer over time and increase its sustainability: defining long-
term  horizons  for  specific  fields  could  lead  to  the  better  coordination  of  existing 
instruments and infrastructures, while giving them sufficient time to grow step by step. 

− 

increase the predictability of the calls (i.e. provide clearer information as to when calls 
that  are  key  for  specific  areas  will  be  published).  This  could  help  organisations 
participating in INFRA calls to better plan their financial and human resources in order 
to prepare for such calls. 

1.5.  Conclusions  

Supporting  TNA/VA  at  EU  level  provides  clear  EU  added  value,  as  this  is  not  usually 
supported by national infrastructures, and allows the better coordination of the service offer 
at EU level. However, various possible models could be used to support TNA/VA. This case 
study explores the strengths and weaknesses of different funding models for TNA/VA through 
a  combination  of  desk  research,  analysis  of  administrative  and  monitoring  data,  and 
interviews with project beneficiaries.  

•  The INFRAIA model was used during Horizon 2020 to effectively support TNA/VA, joint 
research  activities  and  networking  within  specific  disciplinary  domains.  It  facilitated 
collaboration  between similar  pan-EU  research  infrastructures  (RIs)  and  national-level 
infrastructures,  contributing  to  community  building  and  stability.  However,  the  system 
faced challenges in terms of the increasing fragmentation of communities. 

•  The  INFRASERV  model,  implemented  under  Horizon  Europe,  adopts  a  challenge-
oriented approach to TNA/VA support, with the aiming of addressing societal challenges 
and aligning with European Commission priorities. While fostering larger interdisciplinary 
consortia and enhancing visibility, it has led to increased complexity and administrative 
burden,  together  with  concerns  over  the  sustainability  of  results  beyond  a  project’s 
lifetime. 

•  The centralised model – a hypothetical scenario – suggests the establishment of an EC 
agency or body responsible for centralising TNA/VA calls across domains. Such a model 
could enhance visibility and long-term sustainability. However, centralised calls may not 
respond  well  to  the  needs  of  user  communities.  Moreover,  RIs  would  still  need  to  be 
actively involved in developing new services, building user communities and providing 
training.  

•  The cascade funding model, another hypothetical scenario, proposes smaller consortia 
organising TNA/VA calls and connecting users with relevant service providers. This could 
reduce coordination costs and proposal writing efforts, provided that clear guidelines and 
guidance are made available. However, such an approach might have negative effects 
on  community  building,  given  that  not  all  service  providers  would  be  part  of  the 
consortium.  

•  The  ‘ERICs  as  central  points’  model  envisages  the  use  of  ERICs  as  central  actors  in 
distributing  TNA/VA  support,  leveraging  their  coordination  capacities  and  networks. 

360 

 
While potentially efficient, concerns are raised about the dominance of ERICs, the lack 
of competition within the programme part, and the sustained financial reliance of ERICs 
on the Commission.  

•  The  distributed  TNA/VA  support  model  suggests  embedding  TNA/VA  support  within 
other parts of the Framework Programme. This approach could enhance the visibility of 
RI  and  its  alignment  with  other  Framework  Programme  actions,  but  could  limit  the 
number of users. 

Future  analysis  of  the  impact  of  the  hypothetical  models  above  would  be  required  before 
taking any decision to implement them. Such analysis would also need to take into account 
in greater detail the diversity of RIs and the TNA/VA they offer (e.g. TNA/VA  to distributed 
data  infrastructures  is  very  different  from  TNA/VA  to  single-sited  instruments).  While  the 
present case study considers such diversity, it should be noted that it relies on only a limited 
number of interviews.  

Appendix 1. Literature sources 

•  CEA (2023). Position Paper regarding the ‘Public consultation on the past, present and 
future of the European Research & Innovation Framework programmes 2014-2027’, 
available at : https ://ec.europa.eu/info/law/better-regulation/have-your-
say/initiatives/13460-Horizon-Europe-interim-evaluation/public-consultation_en. 

•  Cordis webpages of selected projects. 

•  EARTO (2023). Position Paper regarding the ‘Public consultation on the past, present 
and future of the European Research & Innovation Framework programmes 2014-
2027’, available at: https://ec.europa.eu/info/law/better-regulation/have-your-
say/initiatives/13460-Horizon-Europe-interim-evaluation/public-consultation_en. 

•  ERIC Forum (2020). Policy Brief, Funding models for Access to ERIC 

Multinational/Transnational Services, p. 25, Available at: https://www.eric-forum.eu/wp-
content/uploads/2020/09/ERIC-Forum_Policy-Brief.pdf 

•  ERIC Forum (2023). Position Paper regarding the ‘Public consultation on the past, 

present and future of the European Research & Innovation Framework programmes 
2014-2027’, available at: https://ec.europa.eu/info/law/better-regulation/have-your-
say/initiatives/13460-Horizon-Europe-interim-evaluation/public-consultation_en. 

•  European Commission Decision C(2022)2975 of 10 May 2022: Horizon Europe Work 

Programme 2021-2022, Research Infrastructures, p. 61, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2021-2022/wp-3-research-infrastructures_horizon-2021-2022_en.pdf. 

•  European Commission, Directorate-General for Research and Innovation, European 

Research Area policy agenda: overview of actions for the period 2022-2024, 
Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/52110. 

•  European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European framework 
programmes for research and innovation for excellent science : Horizon 2020 : 
annexes, Publications Office of the European Union, pp. 848-858, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383. 

361 

 
•  Helmholtz Association (2023). Position Paper regarding the ‘Public consultation on the 

past, present and future of the European Research & Innovation Framework 
programmes 2014-2027’, available at: https://ec.europa.eu/info/law/better-
regulation/have-your-say/initiatives/13460-Horizon-Europe-interim-evaluation/public-
consultation_en. 

•  Horizon 2020 Call ‘Implementing the European Open Science Cloud (H2020-
INFRAEOSC-2018-2020)’, available at: https://ec.europa.eu/info/funding-
tenders/opportunities/portal/screen/opportunities/topic-details/infraeosc-04-2018.' 

•  Horizon Dashboard (accessed July 2023). 

•  European Commission, European Research Infrastructure Consortium (ERIC), 

available at: https://research-and-innovation.ec.europa.eu/strategy/strategy-2020-
2024/our-digital-future/european-research-infrastructures/eric_en. 

• 

canSERV, Current Calls - All services provided by canSERV. canSERV - Cutting Edge 
Cancer Research Services Across Europe, available at: https://www.canserv.eu/calls/ 

•  Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 
2021 establishing Horizon Europe – the Framework Programme for Research and 
Innovation, laying down its rules for participation and dissemination, and repealing 
Regulations (EU) No 1290/2013 and (EU) No 1291/2013. 

362 

 
 
Annex 2.6: Analysis of the use of the challenge-oriented approach 
in the INFRA Work Programmes 

1.1. 

Introduction 

Horizon Europe introduced significant changes to the way in which INFRA funds access to 
research infrastructure (RI) services. For some calls, it has now moved from a vertical focus, 
bringing  together  RIs  from  similar  disciplines  and  with  similar  instruments,  to  a  more 
challenge-oriented approach that fosters interdisciplinarity. The objective of this case study 
is to analyse the impact of this new approach on the RI landscape and the user communities. 
Its  findings  show  that  the  new  approach  is  perceived  positively,  as  it  has  the  potential  to 
enlarge the user base of RIs and fosters the contribution of RIs to societal challenges. The 
shift  has,  however,  raised  important  concerns,  particularly  with  regard  to  efficiency  losses 
relating to the increased cost of coordination and the increased complexity of actions, as well 
as a perceived lack of clarity and predictability of the Work Programme. Some avenues for 
improvement  include  the  development  of  a  long-term  and  stable  vision  of  how  the  RI 
landscape should be structured, and how INFRA would support this. 

The  present  case  study  is  based  on  evidence  gathered through  desk  research;  European 
Commission  administrative  and  monitoring  data;  the  analysis  of  survey  data  and  of  the 
outcomes  of  a  public  consultation  regarding  the  past,  present  and  future  of  the  European 
Research  &  Innovation  Framework  Programmes  2014-2027;  as  well  as  interviews  with 
project beneficiaries and stakeholders related to the research infrastructure landscape.  

The  case  study  is  structured  as  follows.  The  first  section  presents  the  policy  context  and 
describes  the  main  features  of  the  INFRA  actions  under  Horizon  Europe.  In  addition,  it 
provides  an  overview  of  the  methodological  approach  used  in  elaborating  the  case study. 
The second section presents the main findings of the analysis according to the evaluation 
criteria of relevance, coherence, efficiency and effectiveness. The third section of the case 
study  summarises  the  success  factors  and  barriers  to  progress,  while  the  last  section 
presents the conclusions. 

1.2.  Background information 

1.2.1.  Policy context 

The  Horizon  Europe  Regulation554  defines  research  infrastructures  (RIs)  as  ‘facilities  that 
provide resources and services for the research communities to conduct research and foster 
innovation in their fields, including the associated human resources, major equipment or sets 
of  instruments;  knowledge-related  facilities  such  as  collections,  archives  or  scientific  data 
infrastructures; computing systems, communication networks and any other infrastructure of 
a unique nature and open to external users, essential to achieve excellence in R&I’. 

554 Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 2021 
establishing Horizon Europe – the Framework Programme for Research and Innovation, laying 
down its rules for participation and dissemination, and repealing Regulations (EU) No 1290/2013 
and (EU) No 1291/2013. 

363 

 
 
The Regulation further indicates four main areas of intervention for the INFRA programme 
part555: 

•  Consolidating and developing the landscape of European research infrastructures; 

•  Opening, integrating and interconnecting research infrastructures;  

•  Fostering  the  innovation  potential  of  European  research  infrastructures  and  their 

activities for innovation and training; and 

•  Reinforcing European research infrastructure policy and international cooperation. 

The  ERA  Policy  Agenda556,  approved  in  2022,  included  a  specific  action  directed  towards 
research infrastructures: Action 8 (‘Strengthen sustainability, accessibility and resilience of 
research infrastructures in the ERA’) was designed to respond to the new challenges that are 
emerging as a consequence of the growing maturity of the European research infrastructure 
landscape.  

The new challenge-oriented approach under Horizon Europe responds to this need to focus 
on  specific  scientific  and  political  needs.  In  the  Work  Programmes,  there  is  a  stronger 
emphasis  on  the  potential  of  European  RIs  to  ‘accelerate  the  transition  towards  a socially 
inclusive green and digital future and to support an effective and responsive health system 
as well as evidence-based policy-making’557. In line with these principles, the INFRA Work 
Programmes have adapted some of their instruments to become more challenge- or priority-
oriented than before, specifically in the case of the calls to integrate RIs, and EOSC calls.   

•  Calls  to  integrate  RIs.  Under  Horizon  2020,  INFRAIA  calls  funded  Starting558  and 
Advanced559 Communities to engage in a combination of activities: 1) the provision of 
transnational  or  virtual  access  (TNA/VA);  2)  network  activities;  and  3)  joint  research. 
Under Horizon Europe, two major changes have taken place with respect to integrating 
activities.  First,  a  distinction  has  been  made  between  INFRASERV  calls,  which  fund 
TNA/VA,  and  INFRATECH  calls,  which  fund  joint  research  on  the  next  generation  of 
scientific instrumentation, tools and methods. Second, the topics of calls have become 
challenge-oriented; for example, calling for RI services that support research on cancer.  

•  European Open Science  Cloud  (EOSC)  calls.  EOSC  calls  were  already included  in 
Horizon  2020  with  the  aim  of  supporting  the  creation  of  a  European  space  for  online 
research,  with  services  for networking  and  computing,  and  open  access  to  e-Science 
environments and global data resources. The EOSC topics called for the creation of new 
services  to  be  included  in  the  EOSC,  as  well  as  supporting  EOSC  governance, 

555 Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 2021 
establishing Horizon Europe – the Framework Programme for Research and Innovation, laying 
down its rules for participation and dissemination, and repealing Regulations (EU) No 1290/2013 
and (EU) No 1291/2013. 
556 European Commission, Directorate-General for Research and Innovation, European 
Research Area policy agenda : overview of actions for the period 2022-2024, Publications Office 
of the European Union, 2022, https://data.europa.eu/doi/10.2777/52110. 
557 INFRA Work Programme 2021-2022, p.7. 
558 A 'Starting Community' has never been supported for the integration of its infrastructures 
under FP7 or Horizon 2020 calls, in particular within an integrating activity. 
559 'Advanced Communities' are scientific communities whose research infrastructures show an 
advanced degree of coordination and networking at present, attained, in particular, through 
Integrating activities awarded under FP7 or previous Horizon 2020 calls. 

364 

 
 
increasing the FAIRness560 of data, and so on. Under Horizon Europe, some EOSC calls 
have become more thematically oriented – for example, calling for FAIR and open data 
sharing in support of cancer research.  

1.2.2.  Main features of challenge-oriented actions in Horizon Europe 

Under  Horizon  Europe,  a  total  net  EU  contribution  of  EUR  100  million  has  been  given  to 
challenge-oriented calls, divided among eight projects and 443 participations (see Table 2 in 
Appendix  2).  This  represents  around  one-fifth  of  the  net  EU  contribution  to  the  INFRA 
programme  under  Horizon  Europe  (a  total  of  EUR  520  million).  The  average  number  of 
participations in challenge-oriented projects is 63. All eight challenge-oriented projects were 
funded  under  the  Work  Programme  2021-2022.  Additional  challenge-oriented  calls  are 
present in the Work Programme 2023-2024; however, no projects have yet been granted at 
the time of writing of this case study (August 2023).  

1.2.3.  Overall approach and scope of the case study 

The methodological steps taken in the elaboration of this case study are as follows: 

•  First, desk research was conducted on INFRA-related documents and reports, including 

an analysis of European Commission reports and documents.  

•  European Commission monitoring and reporting data was used to gather information 

about the number of calls, projects, beneficiaries and EU contributions. 

•  Survey data were analysed to add additional insights, and to triangulate findings based 
on interviews and desk research. The survey was conducted in context of this study in 
July-August 2023, and covered Horizon Europe beneficiary organisations. 

• 

21 interviews were held with a selection of stakeholders and beneficiaries of the new 
challenge-oriented  projects,  with  a  focus  on  INFRASERV  grants.  Beneficiaries  of  the 
previous INFRAIA grants were also interviewed for comparison.  

1.3.  Case study findings 

The new challenge-oriented approach is generally perceived positively by interviewees, as it 
increases the contribution of RIs to societal challenges and European Commission priorities. 
However, interviewees also expressed concerns regarding efficiency as the need for larger 
and more complex consortia increases coordination costs, while funding is diluted across a 
larger number of partners. Interviewees stressed that the shift to a challenge-based approach 
entailed  transition  costs,  and  that  it  will  take  time  for the  RIs  to  adapt  to  the  new  system. 
Interviewees  also  pointed  to  the  need  for  a  strategic  roadmap  that  could  provide  a  stable 
framework for the development of RIs.  

The  sections  that  follow  discuss  the  findings  of  the  case  study  in  depth,  according  to  the 
dimensions of relevance, coherence, efficiency and effectiveness.  

560 FAIR is an acronym that stands for ‘Findable, Accessible, Interoperable, and Reusable’. 

365 

 
 
1.3.1.  Relevance 

Shifting to a challenge-based approach has created a clearer link between the topics of calls 
and the six European Commission priorities, especially the following:  

•  The European Green Deal. Several INFRASERV topics are specifically aimed at the 
green  transition,  calling  for  RI  services  for  sustainable  agriculture561;  to  respond  to 
climate  change  risks562;  to  enable  the  development  of  materials  for  the  circular 
economy563, and so on. INFRATECH topics also call for new technologies and solutions 
to make RIs more environmentally friendly564 and to ensure RIs can enable the green 
transition565.  

•  A Europe fit for the digital age. Several INFRASERV topics are specifically aimed at 
the digital transition – for example, calling for AI-based RI services for the better analysis 
of images566; digital RI services to empower neuroscience research for health and the 
brain567; services to enhance the EU’s capacity for the development of semiconductors 
or  for  innovative  applications  of  nanoscience  and  nanotechnology568.  Furthermore, 
multiple INFRATECH topics have a specific focus on the development of digital twins569.  

The  new  challenge-driven  approach  also  aligns  with  the  adoption  of  EU  Missions  under 
Horizon  Europe,  which  aim  to  ‘support  Europe’s  transformation  into  a  greener,  healthier, 
more inclusive and resilient continent’570. This Mission-oriented approach is one of the key 
changes under Horizon Europe, and aims to bring R&I closer to citizens by formulating clear 
and  inspirational  Missions.  The  challenge-oriented  topics  align  well  with  the  five  Missions 
formulated for Horizon Europe571. 

561 HORIZON-INFRA-2021-SERV-01-02: Research infrastructures services for a sustainable and 
resilient agriculture and agro-ecological transitions. 
562 HORIZON-INFRA-2023-SERV-01-01: Research infrastructure services to enable R&I 
addressing main challenges and EU priorities (Area: climate-change risks) and HORIZON-
INFRA-2021-SERV-01-03: Research infrastructures services for responding to climate-related 
risks on the environment. 
563 HORIZON-INFRA-2021-SERV-01-04: Research infrastructures services enabling the 
development of materials for a circular economy. 
564 HORIZON-INFRA-2023-TECH-01: New technologies and solutions for reducing the 
environmental and climate footprint of RIs. 
565 HORIZON-INFRA-2024-TECH-01-02: Development of tools, solutions, modules to enable 
R&I on the social aspects of the green transition. 
566 HORIZON-INFRA-2021-SERV-01-06: Enabling research infrastructure services for better use 
of imaging data to address challenges in thematic research areas. 
567 HORIZON-INFRA-2022-SERV-01-01: Implementing digital services to empower 
neuroscience research for health and brain inspired technology via EBRAINS. 
568 HORIZON-INFRA-2023-SERV-01-01: Research infrastructure services to enable R&I 
addressing main challenges and EU priorities. 
569 HORIZON-INFRA-2024-TECH-01-03 New digital twins for Destination Earth; HORIZON-
INFRA-2024-TECH-01-04 AR/VR-empowered digital twins for modelling complex phenomena in 
new RI application areas; HORIZON-INFRA-2021-TECH-01-01 Interdisciplinary digital twins for 
modelling and simulating complex phenomena at the service of research infrastructure 
communities. 
570 European Commission, accessed on 10/07/2023, EU missions in Horizon Europe, available 
at: https://research-and-innovation.ec.europa.eu/funding/funding-opportunities/funding-
programmes-and-open-calls/horizon-europe/eu-missions-horizon-europe_en. 
571 https://research-and-innovation.ec.europa.eu/funding/funding-opportunities/funding-
programmes-and-open-calls/horizon-europe/eu-missions-horizon-europe_en  

366 

 
 
Lastly, the challenge-oriented approach also responds well to the needs identified in the 2021 
Landscape Analysis of ESFRI. Each of the five thematic areas (environment; data, computing 
and  digital  research  infrastructures,  health  and  food;  physical  sciences  and  engineering; 
social and cultural innovation) stresses the need for interdisciplinary research collaboration. 
The new approach also notes that ‘none of these Missions [referring to the Horizon Europe 
Missions]  can  be  carried  out  without  interdisciplinary  research  collaboration  across  the 
traditional domains, not just within the environmental sciences but at the interface between 
Environment, Health, Food, Energy, and Humanities’572. 

interviewees  equally  emphasised 

Interviewees  generally  expressed  positive  assessments  of  this  shift  towards  societal 
challenges.  However,  several 
importance  of 
fundamental ‘blue-sky’ or ‘curiosity-driven’ research in tackling societal challenges in the long 
run.  A  balance  therefore  needs  to  be  maintained  between  fundamental  research  and 
challenge-oriented research. Some interviewees indicated this balance could be shifting too 
far towards the latter. Many position papers submitted in the context of the OPC573 also refer 
to the need for such equilibrium, while indicating that Horizon Europe (including Research 
Infrastructures) may have shifted away too much from fundamental research. For example, 
one  position  paper  notes  that  ‘notwithstanding  the  relevance  of  supporting  Research 
Infrastructures and Pillar II actions to be aligned with EU priorities and act as key contributors 
to responding to great challenges, they would also benefit from more funding opportunities 
on frontier knowledge, which thus far have been very limited’574.  

the 

1.3.2.  Coherence 

One of the objectives of the challenge-oriented approach was to increase the alignment of 
the INFRA programme part with the other Pillars of Horizon Europe, specifically the ‘Horizon 
Europe clusters, missions and partnerships in Pillar II’575.  

Interviewees  indicated  that  the  participation  of  RIs  in  other  parts  of  the  Framework 
Programme could still be improved significantly. This is also confirmed by the survey results 
(see Table 3 in Appendix 3): INFRA participants have very limited collaboration with projects 
from the Pillar II Clusters. The Cluster with the most collaborative links with INFRA appears 
to  be  Cluster  6  (Food,  Bioeconomy,  Natural  Resources,  Agriculture  &  Environment).  The 
main barrier mentioned by interviewees is the large number of calls and the lack of resources 
RIs have to screen all of these calls to see where they could contribute. Multiple interviewees 
indicated  that  it  would  be  helpful  to  receive  more  guidance  on  exploring  the  relevant  calls 
outside the INFRA programme part. Similarly, Pillar II beneficiaries and evaluators are not 
necessarily  acquainted  with  RIs.  Some  interviewees  indicated  that  the  explicit  mention  of 
relevant Ris in the texts of calls under Pillar II would boost the visibility and use of  RIs. A 
similar approach is often followed by national R&I programmes in order to boost the use of 
national RIs. 

572 ESFRI (2021), ESFRI Roadmap Part 2 – Landscape Analysis, pp. 74. Available at; 
https://roadmap2021.esfri.eu/landscape-analysis/. 
573 ‘Public consultation on the past, present and future of the European Research & Innovation 
Framework Programmes 2014-2027’, available at: https://ec.europa.eu/info/law/better-
regulation/have-your-say/initiatives/13460-Horizon-Europe-interim-evaluation/public-
consultation_en. 
574 Fundacao para a ciencia e a tecnologia (FCT) (2023). ‘Public consultation on the past, 
present and future of the European Research & Innovation Framework Programmes 2014-2027’. 
Available at: https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-
Horizon-Europe-interim-evaluation/public-consultation_en. 
575 Research Infrastructures Work Programme 2021-2022, p. 8. 

367 

 
 
Some examples of good practice in cross-pillar collaboration exist. Notably, projects working 
on the cancer mission have aligned successfully with each other; the canSERV project576 is 
closely  aligned  with  other  key  initiatives  such  as  the  UNCAN577  initiative  and  the 
EOSC4CANCER578 project. This alignment was most visible at the beginning of the projects, 
to  ensure  that  they  followed  a  similar/coordinated  direction.  The  alignment  was  achieved 
through the participation of key people in the three projects.  

1.3.3.  Efficiency 

Interviewees tended to focus first and foremost on the limitations of the new approach from 
an  efficiency  perspective.  The  new  challenge-oriented  focus  leads  to  the  need  to  bring 
together more services across more facilities and instruments. This has led to the creation of 
consortia which are, in general, larger and more complex than those funded under Horizon 
Europe. The increase in coordination costs that these large consortia entail was stressed 
by interviewees as one of the major causes of concern. Consortia applying to INFRASERV 
calls  in  particular  are  typically  much  larger  than  under  INFRAIA,  reaching  up  to  157 
beneficiaries  in  the  case  of  the  ISIDORe  project579.  Anecdotally,  on  a  few  occasions,  the 
number of partners involved in consortia has been so large that it posed difficulties for the IT 
grant management systems of the EC in terms of the management and processing of these 
grants. Similarly, multiple position papers submitted in the context of the OPC580 indicate that 
consortia  are  becoming  too  large  to  be  managed  effectively,  and  that  ‘to  relieve  the 
complexity of managing such grants, new implementation schemes should be discussed’581.  

Interviews  sometimes  associated  the  higher  coordination  costs  and  more  complex 
administrative requirements of proposals with the increased use of external consultants. 
This was stressed in particular in the case of distributed infrastructures and large consortia. 
Interviewees indicated there are now more companies offering their services to support not 
only during the application process, but also in the management of grants. The involvement 
of  consultants  in  proposals  and  projects  varies:  sometimes  they  receive  a  success  fee; 
sometimes they are part of the project as partners. The availability of these services is not 
perceived negatively by interviewees as, from their perspective, using such consultants help 
them to focus resources on research instead of writing proposals. 

The survey among Horizon Europe beneficiary organisations582, however, indicates that not 
all beneficiaries experience such an excessive administrative burden. Listed below are some 

576 https://cordis.europa.eu/project/id/101058620, funded under INFRASERV. 
577 https://uncan.eu/. The blueprint of the initiative is generated by 4.UNCAN.eu 
(https://cordis.europa.eu/project/id/101069496); a Coordination and Support Action funded under 
the Health cluster (Pillar II). 
578 This project is funded under INFRAEOSC, and aims to establish a pan European 
infrastructure that integrates cancer research-related data and makes it accessible across 
borders. For more info, visit: https://cordis.europa.eu/project/id/101058427. 
579 The ISIDORe project provided integrated services for studying infectious diseases in order to 
control the COVID-19 pandemic, and was funded under HORIZON-INFRA-2021-EMERGENCY-
02. 
580 Such as from CNRS, Helmholtz, ERIC Forum and CEA (2023) regarding the ‘Public 
consultation on the past, present and future of the European Research & Innovation Framework 
Programmes 2014-2027’, available at: https://ec.europa.eu/info/law/better-regulation/have-your-
say/initiatives/13460-Horizon-Europe-interim-evaluation/public-consultation_en. 
581 Masaryk University (2023) Position Paper regarding the ‘Public consultation on the past, 
present and future of the European Research & Innovation Framework Programmes 2014-2027’, 
available at: https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-
Horizon-Europe-interim-evaluation/public-consultation_en. 
582 Survey of Horizon Europe beneficiaries, conducted May-July 2023. 

368 

 
 
survey responses from the beneficiaries of challenge-oriented calls583 (n=32, of which one 
was a coordinator and 31 were participants):  

• 

• 

• 

• 

47%  (15  out  of  32)  of  the  beneficiaries  of  challenge-oriented  calls  (rather  or  strongly) 
agreed  that  the  burden  of  administrative  and  legal  requirements  in  relation  to  the 
granting procedures was  proportionate. This compared with 40% (13 out of 32) who 
(rather  or  strongly)  disagreed.  Beneficiaries  of  challenge-oriented  calls  are  more 
negative than those of INFRA overall: only 27% (50 out of 185) of INFRA respondents 
found that the burden relating to granting procedures is not proportionate.  

38%  (12  out  of  32)  of  the  beneficiaries  of  challenge-oriented  calls  (rather  or  strongly) 
agreed that the overall effort to prepare a Horizon Europe proposal was acceptable, 
whereas 28% (9 out of 32) did not agree or agreed only to a small extent. Again, fewer 
INFRA respondents found that the overall efforts were not acceptable (17% or 31 out of 
182).  

47%  (15  out  of  32)  of  the  beneficiaries  of  challenge-oriented  calls  indicated  that  their 
organisation spent more than 26 person-days preparing its proposal, and 9% (3 out 
of  32)  indicated  it  had  spent  more  than  65  person-days.  The  figures  for  INFRA  were 
similar.  

38%  (12  out  of  32)  of  the  beneficiaries  of  challenge-oriented  calls  indicated  that  they 
spent more than 11% of their project budget on administrative tasks compared with 
46% (86 out of 187) of INFRA respondents who indicated they spent more than 11% of 
their  budget  on  administrative  tasks.  This  indicates  that  challenge-oriented  projects 
actually entail a lower administrative burden during the course of the project.  

•  Among survey respondents who had participated in INFRA in general, as well as those 
who had participated in the challenge-oriented calls, more ‘rather’ agreed than disagreed 
that  proposal  preparation  and  submission  as  well  as  grant  preparation,  project 
management and financial management had become simpler under Horizon Europe 
compared with Horizon 2020. Nonetheless, the largest group (between 30% and 50%) 
indicated that they neither agreed nor disagreed.  

Furthermore,  interviewees  also  stressed  the  transition  costs  associated  with  adapting  to 
the new system, in terms of creating and managing new consortia, writing proposals from 
scratch, adapting administrative process flows, screening calls etc. Before implementing a 
new system, a trade-off should be considered between the benefits of a new system versus 
the associated transition costs. Beneficiaries stressed that profound changes to the structure 
of the future Work Programme and Framework Programme should be avoided.  

•  The  structure  of  the  current  Work  Programmes  has  changed  considerably,  and  the 
expectations  for  the  new  challenge-oriented  calls  were  not  always  clear.  Some 
interviewees  indicated  this  had  led  to  confusion  from  the  applicants’  side  in  terms  of 
which organisations and consortia were best placed to respond to the call requirements, 
whereas in the Work Programmes under Horizon 2020, it was clearer who the calls were 
targeting. In this regard, interviewees positively regarded the explicit mentioning of the 
RIs (or types of RIs) targeted by a certain topic. On the other hand, 82% of respondents 
to  the  survey  from  INFRA584  (152  out  of  185)  indicated  it  was  (largely  or  moderately) 

583 As listed in Table 2 in Appendix 2. 
584 Survey of Horizon Europe beneficiaries, conducted May-July 2023 (n=185). 

369 

 
 
easy to find relevant funding opportunities, while 90%585 (166 out of 184) indicated 
the  aims  and  objective  of  the  calls  were  clear  (to  a  very  large,  large  or  moderate 
extent).  Furthermore,  80%  of  INFRA  respondents586  (148  out  of  185)  indicated  it  was 
(largely or moderately) easy to identify relevant partners for the consortium.  

• 

In  addition,  interviewees  mentioned  that  the  proposal-writing  process  was  now  more 
time-consuming, given that new consortia had to be formed and proposals had to be 
rethought.  Under  Horizon  2020,  the  system  was  more  predictable  and  stable  for 
organisations  in  the  RI  ecosystem,  which  lowered  the  costs  of  proposal  writing,  as 
consortia could build on previous experiences.  

•  Transition  costs  also  occurred  during  project  implementation,  as  it  takes  time  for new 
partners to get to know each other, build trust and find an efficient and effective way of 
working.  

• 

Lastly,  many  organisations  have  Horizon  2020  and  Horizon  Europe  grants  running  in 
parallel,  each  with  different  systems  and  requirements  in  terms  of  administration:  this 
leads to confusion and inefficiencies. 

1.3.4.  Effectiveness 

Most  interviewees  indicated  that  they  held  a  positive  perception  of  the  concept  of  a 
challenge-oriented approach, as it could increase the contribution made by RIs to tackling 
societal  challenges,  as  well as  fostering  interdisciplinary collaboration  and  opening  up  the 
INFRA  programme  to  new  partners,  broadening  user  communities,  and  increasing  the 
visibility  of  the  impacts  of  the  Framework  Programme  in  the  eyes  of  the  general  public. 
However, interviewees indicated that there is a need for continuity and a long-term vision 
to enable these positive effects.  

Many  interviewees  stressed  the  fact  that  the  previous  system  under  Horizon  2020  gave 
organisations the opportunity to work with the same partners in a coherent manner across 
projects, enabling them to continue collaborating through successive calls. This is considered 
to have had a very positive impact in terms of building trust between partners, which is often 
considered crucial for the optimal implementation of projects. At the same time, it allowed RIs 
to continuously improve the services they provide to users. Interviewees perceived that in 
comparison to Horizon 2020, Horizon Europe focuses now more on innovation and less on 
the sustainability of previously achieved project results. Although seeking improvements and 
challenging  the  existing  system  may  be  considered  positive,  interviewees  stated  that 
progress  could  be  lost  if  changes  fail  to  sufficiently  consider  the  importance  of  continuity. 
Some  interviewees  mentioned  that  re-introducing  dedicated  funding  for  individual  science 
clusters could contribute to providing a collaboration platform for RIs within the same scientific 
discipline, and ensure some level of continuity (see Box 1).  

Box 1. The role of the science clusters  

Under Horizon 2020, a specific call was dedicated for science clusters: ‘INFRAEOSC-04-
2018  –  Connecting  ESFRI  infrastructures  through  Cluster  projects’.  This  call  aimed  to 
foster connections between research infrastructures identified in the ESFRI Roadmap to 
the  EOSC.  Support  was  provided  to  five  ‘clusters’  which  mainly  bring  together  ESFRI 

585 Survey of Horizon Europe beneficiaries, conducted May-July 2023 (n=184). 
586 Survey of Horizon Europe beneficiaries, conducted May-July 2023 (n=185). 

370 

 
 
projects and landmarks along five thematic areas: 1) Biomedical Science; 2) Environment 
and  Earth  Sciences;  3)  Physics  and  Analytical  Facilities;  4)  Social  Science  and 
Humanities; and 5) Astronomy. According to the Horizon 2020 evaluation study, these 
cluster projects had a positive impact as they enabled the sharing of knowledge among 
the RIs involved, in line with the principle of FAIR data. The five thematic clusters also 
had regular contact among them, which enabled them to present their interests with one 
voice.  

In  the  above  call,  the  five  science  clusters  received  a  net  EU  contribution  of  EUR  88 
million and brought together a total of 201 partners. Table 1 provides an overview of the 
five clusters.  

Project 

Scientific discipline 

Cluster members 

EOSC-Life587 

Biology and medicine 

13 research infrastructures, all of them 
included in the ESFRI roadmap 

ENVRI-FAIR588 

Environmental research 

26 research infrastructures, of which 14 
are included in the ESFRI roadmap 

PaNOSC589 

Photon neutron science 

Six research infrastructures, all of them 
included in the ESFRI roadmap 

ESCAPE590 

Astronomy- and accelerator-
based particle physics 

10 research infrastructures, of which six 
are included in the ESFRI roadmap 

SSHOC591 

Social sciences and 
humanities 

Six research infrastructures, all of them 
included in the ESFRI roadmap 

Table 1. Overview of the five cluster projects under Horizon 2020. Source: analysis from the Evaluation study on 
Excellent Science under Horizon 2020592, based on projects funded under INFRAEOSC-04-2018593 

These individual cluster projects have been discontinued under Horizon Europe. Instead, 
they  now  come  together  as  a  ‘cluster-of-clusters’  project  funded  under  the  topic 
‘HORIZON-INFRA-2023-EOSC-01-01 – Build on the science cluster approach to ensure 
the uptake of EOSC by research infrastructures and research communities’. Interviewees 
indicated that, although it is good to provide a cross-disciplinary platform for the science 
clusters,  the  clusters  would  still  have  benefited  from  dedicated  individual  support  to 
strengthen the integration within their own disciplines. Interviewees further stressed that 
they  find  science  clusters  to  be  very  important  for  the  further  development  of  EOSC 

587 https://cordis.europa.eu/project/id/824087. 
588 https://cordis.europa.eu/project/id/824068. 
589 https://cordis.europa.eu/project/id/823852. 
590 https://cordis.europa.eu/project/id/824064. 
591 https://cordis.europa.eu/project/id/823782. 
592 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European framework programmes 
for research and innovation for excellent science : Horizon 2020 : annexes, Publications Office of 
the European Union, pp. 848-858, 2023, https://data.europa.eu/doi/10.2777/353383. 
593 https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/topic-
details/infraeosc-04-2018 

371 

 
 
services: because EOSC is developed in a generic or domain-agnostic way, the clusters 
can bring the thematic and community-based orientation that would be required to make 
the services more usable and relevant for the various communities. The clusters are also 
perceived as a mechanism through which the visibility of RIs and their services can be 
strengthened and presented in a clear manner to the users and to the broader science 
and innovation community. 

One  position  paper  submitted  in  the  context  of  OPC594  further  highlights  the  positive 
impact  created  by  the  cluster  projects,  and  indicates  that  continued  support  for  the 
clusters is  required in  order  to  further  build  on  the  progress  made so  far. The  position 
paper  stresses  that  funding  is  too  often  focused  on  immediate  results,  leading  to  ‘the 
tendency of creating new (or duplicated) services and resources rather than focusing on 
operating  and  sustaining  existing  ones,  despite  their  proven  value  to  research 
communities. The situation creates uncertainties regarding the continuation of services, 
which  in  turn makes  the  user  communities  reluctant  to  take  up  and  rely  on  these  new 
EOSC services’595.  

The  shift  to  a  challenge-oriented  approach  is  perceived  by  interviewees  as  an  important 
change. To ensure that the efforts invested turn into outputs that create value in the long run, 
a certain level of stability is required. The creation of a long-term vision, including funding 
pipelines  (e.g.  through  the  development  of  roadmaps  or similar tools)  would  be  perceived 
positively,  as  this  would  allow  RIs  to  adapt  their  strategies  and  give  them  more 
confidence to invest time and resources in the creation of new services and the building 
of new user communities. Moreover, multiple interviewees indicated that a longer-term vision 
is  also  necessary  to  enable  collaborations  with  industry,  which  require  longer  time 
horizons. 

Some interviewees indicated that without prolonged support, projects with a very high level 
of complexity and a short duration would not be able to develop their full potential. Projects 
under INFRA Emergency calls were often mentioned as examples of this underdeveloped 
potential:  these  were  very  complex  projects  focusing  on  a  highly  relevant  societal  need 
(COVID-19), with a very high number of participants (i.e. up to 157 participants in the case of 
the ISIDORe project596). These projects were regarded as highly beneficial; however, the fact 
that they only ran for three years without any expectation of continuity is seen as a major 
factor  hindering  their  potential  impact,  according  to  interviewees.  For  example,  within 
ISIDORe, great effort went into harmonising access to services for research into the outbreak 
of  infectious  disease  by  creating  a  common  catalogue  of  services  and  TNA/VA  calls. 
However, full harmonisation would require more than three years. Currently, the beneficiaries 
interviewed indicate that it remains difficult for researchers to navigate the many existing RI 
services  on  this  topic.  A  longer-term  perspective  on  funding  would  enable  further 

594 EMBL (2023). Position Paper regarding the ‘Public consultation on the past, present and 
future of the European Research & Innovation Framework Programmes 2014-2027’, available at: 
https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-Horizon-Europe-
interim-evaluation/public-consultation_en. 
595 EMBL (2023). Position Paper regarding the ‘Public consultation on the past, present and 
future of the European Research & Innovation Framework Programmes 2014-2027’, available at: 
https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-Horizon-Europe-
interim-evaluation/public-consultation_en. 
596 The ISIDORe project provided integrated services for studying infectious diseases in order to 
control the COVID-19 pandemic, and was funded under HORIZON-INFRA-2021-EMERGENCY-
02. 

372 

 
 
harmonisation  of  services,  thus  benefitting  researchers.  A  more  fundamental  limitation  to 
longer-term impact is the fact that the catalogue of services will not be updated beyond the 
project lifetime, irrespective of the duration of the project.  

One position paper submitted to the OPC also points to this issue of long-term vision: ‘When 
it comes to research infrastructures, many members want to see more long-term thinking. It’s 
difficult to maintain a research infrastructure with short-term projects and funding’597. Another 
position paper explicitly points out the value of a strategic approach: ‘Individual projects are 
useful for some aspects, but real impact in this field can only be generated by following a 
long-term  strategy,  i.e.  a  new  mode  of  roadmap-based  processes  for  large  consortia  of 
national  labs.  This  would  allow  agreeing  on  significant  long-term  support  to  ensure  joint 
technology development, to strengthen European industry as a supplier and to develop new 
smart access policies for strategically particularly relevant topics’598.  

Lastly, the issue of ‘funding dilution’ or the ‘fragmentation of funding’ as a result of larger 
consortia (especially in the case of INFRASERV) was raised multiple times by interviewees 
and in several of the position papers submitted to the OPC, and is expected to limit the impact 
of the projects599. interviewees pointed out that due to funding being fragmented across many 
partners, the funding per partner may not be enough to achieve the ambitious objectives set 
out  in  the  call  texts.  The  survey  confirms  that  beneficiaries  of  challenge-oriented  calls  are 
less satisfied with the extent to which the funding foreseen in the call matches the objectives 
(see Figure 1).  

More  generally,  stakeholders  in  the  OPC600  and  interviewees  alike  regret  the  reduced 
budget share of the entire INFRA programme part in Horizon Europe compared with Horizon 
2020 (2.4% versus 3.8%)601. Several position papers refer to a perceived lack of substantial 
funding,  stating  that  this  has  ‘led  to  decreased  access  to  world-class  RI  (like  XFEL  and 
ELI602)  across  Europe,  notably  for  researchers  and  innovators  from  financially  weaker 
countries’603;  that  ‘access  to  RIs,  even  if  complemented  with  national/institutional  funding, 

597 Science Business (2023). Position Paper regarding the ‘Public consultation on the past, 
present and future of the European Research & Innovation Framework Programmes 2014-2027’, 
available at: https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-
Horizon-Europe-interim-evaluation/public-consultation_en. 
598 DESY (2023). Position Paper regarding the ‘Public consultation on the past, present and 
future of the European Research & Innovation Framework Programmes 2014-2027’, available at: 
https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-Horizon-Europe-
interim-evaluation/public-consultation_en. 
599 e.g. EARTO (2023) and CEA (2023). Position paper regarding the ‘Public consultation on the 
past, present and future of the European Research & Innovation Framework Programmes 2014-
2027’, available at: https://ec.europa.eu/info/law/better-regulation/have-your-
say/initiatives/13460-Horizon-Europe-interim-evaluation/public-consultation_en. 
600 e.g. CNR (2023). Position Paper regarding the ‘Public consultation on the past, present and 
future of the European Research & Innovation Framework Programmes 2014-2027’, available at: 
https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-Horizon-Europe-
interim-evaluation/public-consultation_en. 
601 Horizon Dashboard, retrieved in August 2023. 
602 XFEL and ELI are both advanced laser infrastructures, located in Germany and the Czech 
Republic, respectively. 
603 Helmholtz (2023). Position Paper regarding the ‘Public consultation on the past, present and 
future of the European Research & Innovation Framework Programmes 2014-2027’, available at: 
https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-Horizon-Europe-
interim-evaluation/public-consultation_en. 

373 

 
 
needs to be more strongly facilitated and funded at EU level’604; or that the cut in funding to 
TNA could have ‘unforeseen consequences’605. Some interviewees indicated that it would be 
beneficial if Europe would target its resources more towards a limited number of world-class, 
pan-EU RIs, rather than diluting funding across a large number of RIs. Others pointed to the 
continuing  need  for  funding  to  be  directed  towards  national  RIs  in  order  to  support  their 
European activities.  

Challenge-oriented calls (n=32)

1

7

16

5

2

1

INFRA (n=185)

18

66

60

30

6

5

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

To a very large extent

To a large extent

To a moderate extent

To a small extent

Not at all

Do not know/Not Applicable

Figure 1. Survey responses to the question ‘To what extent are you satisfied with the following aspects related to your 
Horizon Europe project application process: The size of the possible project funding as foreseen in the call compared to the 
objectives set in the call’. Source: Survey of Horizon Europe beneficiaries, conducted May-July 2023 

Lastly,  the  dilution  of  funding  and  the  increased  complexity  of  project  management  and 
application  in  INFRASERV  may  be  the  reason  why  the  number  of  proposals  under 
INFRASERV has remained below expectations. A total indicative budget of EUR 160.5 million 
had been foreseen for INFRASERV during the Work Programme of 2021-2022. However, 
the  actual  budget  spent  was  only  56%  of  the  indicative  budget.  For  some  topics,  no 
projects were granted. This was the case for the topic calling for RI services to respond to 
climate-related risks to the environment606, as well as the topic on empowering neuroscience 
research for health- and brain-inspired technology via EBRAINS607. Moreover, for one topic 
– ‘Research infrastructures services for sustainable and inclusive Global Value Chain and 
Europe recovery from socio-economic crises’ – not a single proposal was submitted608. 

604 CNR (2023). Position Paper regarding the ‘Public consultation on the past, present and future 
of the European Research & Innovation Framework Programmes 2014-2027’, available at: 
https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-Horizon-Europe-
interim-evaluation/public-consultation_en. 
605 Masaryk University (2023). Position Paper regarding the ‘Public consultation on the past, 
present and future of the European Research & Innovation Framework Programmes 2014-2027’, 
available at: https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-
Horizon-Europe-interim-evaluation/public-consultation_en. 
606 HORIZON-INFRA-2021-SERV-01-03. 
607 HORIZON-INFRA-2022-SERV-01-01. 
608 HORIZON-INFRA-2021-SERV-01-05. 

374 

 
 
 
1.4.  Success factors and barriers 

The success factors of the challenge-oriented approach are:  

•  Contribution  to  societal  challenges.  One  of  the  main  objectives  of  the  challenge-
oriented approach is to increase the contribution RIs make to tackling societal challenges 
such as climate change, cancer, and so on. New services that contribute to tackling these 
challenges have been developed; users of RIs have been trained to use these services; 
and new interdisciplinary collaborations between RIs have made more societally relevant 
research possible.  

•  A broader user-base of RIs. By bringing together service providers to tackle societal 
challenges, a wider diversity of RIs and services are gathered together, which is likely to 
appeal  to  a  broader  user  base.  This  allows  users  to  discover  RIs  that  they  did  not 
previously know about, and may attract new users who were not previously acquainted 
with RIs. Platforms for TNA/VA calls are now more consolidated, which increases the 
visibility of the various calls to users. Researchers working on a specific challenge can 
find all relevant services in one place whereas in the past, they would have needed to 
check  different  project  websites  to  access  different  types  of  instruments.  The  new 
approach should thus provide more complete access to RI services in their field. Multiple 
interviewees indicated that they do indeed expect to reach more users, but stressed that 
it will take time for impacts to be realised. INFRASERV projects, which provide TNA/VA 
support, only began in mid-2022, and have not yet started to report on the number of 
TNA/VA provided. 

•  Opening up to new organisations. Interviewees indicated that one of the key benefits 
of  the  new  challenge-oriented  projects  is  that  they  foster  collaboration  with  other  EU 
organisations  working  on  the  same  challenge.  For  participants,  this  is  one  of  the  key 
reasons  to  participate:  to  enlarge  their  network  and  increase  their  visibility;  24%  of 
organisations participating in INFRASERV did not participate in INFRAIA, and are thus 
new  to  TNA/VA  projects609.  However,  there  are  indications  that  some  INFRASERV 
communities are merely a combination of previously funded INFRAIA communities that 
in practice still work as distinct communities with separate calls and Work Programmes.  

The barriers to reaching the intended impacts are as follows:  

•  Large consortia result in increased complexity and funding dilution. The main issue 
with  challenge-oriented  projects  concerns  the  large  size  of  consortia,  which  results  in 
complex  project  management  and  higher  coordination  costs.  Moreover,  while  the 
number of RIs and facilities included in projects has increased considerably, the average 
budget per project has not. The consequence of this is that the budget allocated to each 
facility or instrument is reduced, while at the same time, a greater share of the budget 
goes towards coordination and administration. This reduces the interest of some RIs to 
apply for funding, and thus decreases the potential impact that can be achieved.  

•  Lack of a long-term roadmap and continuity. The analysis reveals general agreement 
that  a  strategic,  roadmap-based  approach  for  RIs  is  necessary  to  provide  a  stable 
framework  for  the  development  of  RIs.  Some  interviewees  indicated  that  the  current 
Work  Programme  focuses  too  much  on  the  creation  and  design  of  new  services  and 
innovation-oriented outcomes, and does not pay sufficient attention to the sustainability 

609 Calculated on the basis of monitoring data retreived from the Horizon Dashboard in July 
2023. 

375 

 
 
of those services and tools created previously under the Framework Programme that are 
still relevant (or could be after the implementation of upgrades). Some interviewees also 
pointed to the role that science clusters could play in creating a stable platform for RIs 
within the same scientific discipline. 

1.5.  Conclusions and lessons learned across the evaluation criteria of 

interest to this case study 

The  present  case  study  analyses  the  impact  of  the  shift  to  a  more  challenge-oriented 
approach in INFRA. In general terms, this change is perceived positively by interviewees, but 
there is one key dimension in which further improvement is called for: the need for a clear 
long-term  vision  regarding  the  expectations  of  the  INFRA  programme  for  diverse  types  of 
RIs. The creation of such a roadmap would add clarity and predictability to the RI landscape, 
and is expected to result in efficiency savings. As described above, efficiency matters are 
high on the list of applicants’ concerns. While some of these concerns can be attributed to 
the transition between one system and another and can be expected to be overcome in the 
medium term, others seem to be more structural.  

The  articulation  of  services  and  alignment  across  RIs  is  perceived  as  one  of  the  major 
benefits of INFRA. In order to achieve substantial and efficient impacts in this dimension, a 
long-term  and  stable  vision  would  be  required,  indicating  how  the  landscape  should  be 
structured. This could be achieved by, for instance, giving a more prominent and stable role 
to  the  science  clusters  and/or  developing  specific  roadmaps  for  certain  fields  or  thematic 
areas. The articulation of the landscape (including the formation of new communities) would 
involve important coordination costs; it is important that those efforts are directed towards a 
consistent and stable goal. This would help applicants and beneficiaries to adapt in a more 
efficient way to the landscape and, as a result, achieve a more structured RI landscape – to 
the benefit of EU actors in science and innovation.  

Lastly, it is important to note that the present case study draws mainly on desk research and 
a limited number of interviews. A more in-depth analysis of the needs of the RI ecosystem in 
terms of structuring EU support would help to shed light on the ways in which such support 
could be better adapted. 

Appendix 1. Literature sources 

•  CNR (2023). Position Paper regarding the ‘Public consultation on the past, present and 
future of the European Research & Innovation Framework Programmes 2014-2027’, 
available at: https://ec.europa.eu/info/law/better-regulation/have-your-
say/initiatives/13460-Horizon-Europe-interim-evaluation/public-consultation_en. 

•  Cordis webpages of selected projects. 

•  DESY (2023). Position Paper regarding the ‘Public consultation on the past, present 
and future of the European Research & Innovation Framework Programmes 2014-
2027’, available at: https://ec.europa.eu/info/law/better-regulation/have-your-
say/initiatives/13460-Horizon-Europe-interim-evaluation/public-consultation_en. 

•  EARTO (2023) and CEA (2023) Position Paper regarding the ‘Public consultation on 
the past, present and future of the European Research & Innovation Framework 
Programmes 2014-2027’, available at: https://ec.europa.eu/info/law/better-

376 

 
regulation/have-your-say/initiatives/13460-Horizon-Europe-interim-evaluation/public-
consultation_en. 

•  EMBL (2023). Position Paper regarding the ‘Public consultation on the past, present 
and future of the European Research & Innovation Framework Programmes 2014-
2027’, available at: https://ec.europa.eu/info/law/better-regulation/have-your-
say/initiatives/13460-Horizon-Europe-interim-evaluation/public-consultation_en. 

•  ESFRI (2021). ESFRI Roadmap Part 2 – Landscape Analysis, available at: 

https://roadmap2021.esfri.eu/landscape-analysis/. 

•  European Commission Directorate-General for Research and Innovation, European 

Research Area policy agenda: overview of actions for the period 2022-2024, 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/52110. 

•  European Commission Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European framework 
programmes for research and innovation for excellent science: Horizon 2020: annexes, 
Luxembourg: Publications Office of the European Union, pp. 848-858, 2023, available 
at: https://data.europa.eu/doi/10.2777/353383. 

•  European Commission, EU missions in Horizon Europe, 2023, available at: 

https://research-and-innovation.ec.europa.eu/funding/funding-opportunities/funding-
programmes-and-open-calls/horizon-europe/eu-missions-horizon-europe_en. 

•  European Commission Decision C(2022)2975 of 10 May 2022: Horizon Europe Work 

Programme 2021-2022, Research Infrastructures, p. 7, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2021-2022/wp-3-research-infrastructures_horizon-2021-2022_en.pdf. 

•  European Commission, ‘Public consultation on the past, present and future of the 

European Research & Innovation Framework Programmes 2014-2027’, available at: 
https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-Horizon-
Europe-interim-evaluation/public-consultation_en. 

•  FCT, ‘Public consultation on the past, present and future of the European Research & 

Innovation Framework Programmes 2014-2027’, 2023, available at: 
https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-Horizon-
Europe-interim-evaluation/public-consultation_en. 

•  Helmholtz (2023). Position Paper regarding the ‘Public consultation on the past, 

present and future of the European Research & Innovation Framework Programmes 
2014-2027’, available at: https://ec.europa.eu/info/law/better-regulation/have-your-
say/initiatives/13460-Horizon-Europe-interim-evaluation/public-consultation_en. 

•  Horizon 2020 call ‘Implementing the European Open Science Cloud (H2020-
INFRAEOSC-2018-2020)’, available at: https://ec.europa.eu/info/funding-
tenders/opportunities/portal/screen/opportunities/topic-details/infraeosc-04-2018. 

•  Uncan.eu, project website, available at: https://uncan.eu/. 

377 

 
•  Masaryk University (2023). Position Paper regarding the ‘Public consultation on the 

past, present and future of the European Research & Innovation Framework 
Programmes 2014-2027’, available at: https://ec.europa.eu/info/law/better-
regulation/have-your-say/initiatives/13460-Horizon-Europe-interim-evaluation/public-
consultation_en. 

•  Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 
2021 establishing Horizon Europe – the Framework Programme for Research and 
Innovation, laying down its rules for participation and dissemination, and repealing 
Regulations (EU) No 1290/2013 and (EU) No 1291/2013, available at: https://eur-
lex.europa.eu/eli/reg/2021/695/oj.  

•  Science Business (2023). Position Paper regarding the ‘Public consultation on the past, 
present and future of the European Research & Innovation Framework Programmes 
2014-2027’, available at: https://ec.europa.eu/info/law/better-regulation/have-your-
say/initiatives/13460-Horizon-Europe-interim-evaluation/public-consultation_en. 

378 

 
 
Appendix 2. Challenge-oriented actions in Horizon Europe 

Topic Code 

Description 

HORIZON-
INFRA-2021-
EOSC-01-06 

FAIR and open data 
sharing in support of 
cancer research 

HORIZON-
INFRA-2022-
EOSC-01-03 

HORIZON-
INFRA-2021-
SERV-01-01 

HORIZON-
INFRA-2021-
SERV-01-02 

HORIZON-
INFRA-2021-
SERV-01-04 

HORIZON-
INFRA-2021-
EMERGENCY-01 

FAIR and open data 
sharing in support of 
healthy oceans, seas, 
coastal and inland waters 

Research infrastructures 
services to support 
research addressing 
cancer 

Research infrastructures 
services for a sustainable 
and resilient agriculture 
and agro-ecological 
transitions 

Research infrastructures 
services enabling the 
development of materials 
for a circular economy 

FAIR and open data 
sharing in support to 
European preparedness 
for COVID-19 and other 
infectious diseases 

HORIZON-
INFRA-2021-
EMERGENCY-02 

Research infrastructure 
services for rapid research 
responses to COVID-19 
and other infectious 
disease epidemics 

TOTAL 

Signed 
Grants 

Number of 
participations 

Net EU 
Contribution 
(EUR) 

1 

2 

1 

1 

1 

1 

1 

8 

34 

7,814,549 

61 

16,447,245 

18 

14,866,441 

73 

14,252,873 

47 

13,679,983 

53 

12,000,000 

157 

20,998,624 

443 

100,059,715 

Table 2. Overview of challenge-oriented topics in Horizon Europe. Source: Horizon Dashboard, retrieved in July 2023. 

379 

 
 
Appendix 3. Coherence evaluation criteria 

Answer option 

Pillar I - European Research Council  

Pillar I - Marie Skłodowska-Curie Actions  

Pillar I - Research Infrastructures  

Pillar II - Cluster 1 - Health  

Pillar II - Cluster 2 - Culture, Creativity & Inclusive Society  

Pillar II - Cluster 3 - Civil Security for Society  

Pillar II - Cluster 4 - Digital, Industry & Space  

Pillar II - Cluster 5 - Climate, Energy & Mobility  

%610 

3.40% 

n 

6 

5.70 % 

10 

31.40% 

55 

4.60% 

1.70% 

0.60% 

2.90% 

2.90% 

8 

3 

1 

5 

5 

Pillar II - Cluster 6 - Food, Bioeconomy, Natural Resources, Agriculture & 
Environment  

9.10% 

16 

Pillar III - European Innovation Council  

Widening participation & spreading excellence  

Reforming & enhancing the European R&I system  

0.60% 

3.40% 

0.60% 

1 

6 

1 

No, my project does not have joint activities with projects funded under other 
Horizon Europe programme/clusters  

22.90% 

40 

Do not know/not applicable  

35.40% 

62 

Table 3. Survey responses of INFRA beneficiaries to the question ‘Are there any activities planned in your project that are 
implemented in collaboration with projects funded under other Horizon Europe programmes or clusters (this could include 
mutual conferences, joint dissemination activities, workshops, joint publications, etc.)? If yes, please indicate the three most 
important programmes.’ Source: Survey of Horizon Europe beneficiaries, conducted May-July 2023. 

610 The % should be interpreted as the percentage of INFRA respondents who indicated this as 
one of the three most important programmes for collaboration. 

380 

 
 
Annex 2.7: Impact of the widening actions on inducing structural 
change 

1.1. 

Introduction 

The  Spreading  Excellence  and  Widening  Participation  (SEWP)  work  programme  was 
designed with the aim of addressing disparities within the EU with regard to research and 
innovation  (R&I)  performance.  The  widening  actions  under  this  programme  part  focus  on 
involving  organisations  from  countries  that  are  less  advanced  in  R&I,  with  the  aim  of 
enhancing  their  participation  in  the  Framework  Programmes  and  increasing  their  R&I 
capacity.  One  of  the  mechanisms  through  which  this  is  expected  to  be  achieved  is  the 
introduction of structural changes in the institutions, regions or countries involved.  

This  case  study  presents  a definition  of  structural  changes  that is  tailored to  the  widening 
actions, together with a conceptual framework for categorising structural changes based on 
the  impact  they  seek  to  achieve.  The  case  study  then  provides  evidence  of  the  structural 
changes  that  have  occurred  as  a  result  of  the  Horizon  2020  widening  actions611.  The 
definition, framework and evidence presented in this case study aim to foster discussions on 
how to better monitor structural changes achieved through the widening actions. 

This case study focuses on widening projects from Horizon 2020, on three main widening 
actions,  namely Teaming,  Twinning, and ERA Chairs.  This focus is taken because the 
introduction and impact of structural changes will only be visible over time612 and it is too early 
to  assess  the  impact  of  structural  changes  in  Horizon  Europe  projects.  Nevertheless,  the 
three  widening  actions  have  been  continued  under  Horizon  Europe,  albeit  with  small 
modifications  such  as  adjustments  to  the  list  of  widening  countries613.  In  that  sense,  the 
findings relating to Horizon 2020 remain relevant in identifying pathways for future actions in 
this context. 

The case study draws on desk research and interviews to establish the conceptual framework 
on structural changes. Furthermore, it employs desk research, as well as evidence from a 
survey  and  interviews  to  provide  evidence  on  the  structural  changes  that  have  been 
introduced  as  a  result  of  the  widening  actions  (see  Section  1.2.3).  The  case  study  draws 
conclusions in view of policy actions.  

611 Under Horizon 2020, the widening countries comprised the following Member States: 
Bulgaria, Croatia, Cyprus, Czechia, Estonia, Hungary, Latvia, Lithuania, Luxembourg, Malta, 
Poland, Portugal, Romania, Slovakia and Slovenia; and the following associated countries 
(subject to valid association agreements of third countries with Horizon 2020): Albania, Armenia, 
Bosnia and Herzegovina, the Faroe Islands, North Macedonia, Georgia, Moldova, Montenegro, 
Serbia, Tunisia, Turkey and Ukraine. 
612 SWD(2017) 221 final (2017). Interim evaluation of Horizon 2020 Annex II. 
https://op.europa.eu/en/publication-detail/-/publication/4cedc9a0-d8cc-11e8-afb3-
01aa75ed71a1/language-en/format-PDF/source-288823388.  
613 Under Horizon 2020, the widening countries comprised the following Member States: 
Bulgaria, Croatia, Cyprus, Czechia, Estonia, Hungary, Latvia, Lithuania, Luxembourg, Malta, 
Poland, Portugal, Romania, Slovakia and Slovenia; and the following associated countries 
(subject to valid association agreements of third countries with Horizon 2020): Albania, Armenia, 
Bosnia and Herzegovina, the Faroe Islands, North Macedonia, Georgia, Moldova, Montenegro, 
Serbia, Tunisia, Turkey and Ukraine. 

381 

 
 
1.2.  Background information 

1.2.1.  Widening actions: from Horizon 2020 to Horizon Europe 

in  research  and 

Although  significant  efforts  have  been  made  to  strengthen  the  European  Research  Area 
(ERA),  disparities 
innovation  (R&I)  performance  persist  among 
countries614,615. As a result, the widening actions were introduced under Horizon 2020 and 
have continued under Horizon Europe. The success of some Horizon 2020 actions, such as 
twinning and teaming has led to their continuation in Horizon Europe with minor modifications 
but  with  a  same  objective:  promote  the  participation  in  the  Framework  Programme  of 
organisations  from  countries  that  are  less  advanced  in  terms  of  R&I;  and  supporting  the 
enhancement of these countries’ R&I systems616. 

Under  Horizon  2020,  the  widening  actions  were  part  of  the  work  programme  ‘Widening 
Participation  and  Spreading  Excellence’  (SEWP),  which  achieved  a  total  of  493  signed 
grants, representing an EU contribution of around EUR 1.0 billion617. Figure 1 shows that, in 
absolute  numbers,  the  action  with  a  highest  number  of  grants  was  Twinning  (211  signed 
grants),  while  Teaming  projects  accounted  for  the  largest  EU  contribution  (EUR  390.1 
million). Portugal, Cyprus and Poland were the widening countries with the highest number 
of  participations,  and  correspondingly,  received  the  highest  net  EU  contributions.  Malta, 
Luxembourg and Lithuania were the Member States with the fewest participations, receiving 
the lowest EU contributions618.  

In  Horizon  Europe  the  horizontal  SEWP  pillar  was  reshaped  and  merged  into  a  new 
horizontal pillar known as ‘Widening Participation and Strengthening the European Research 
Area’  (WIDERA).  The  first  component  in  this  pillar,  namely  ‘Widening  participation  and 
spreading excellence’ includes widening actions. 

614 European Court of Auditors, Measures to widen participation in Horizon 2020 were well 
designed but sustainable change will mostly depend on efforts by national authorities. Special 
report 15, 2022, Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2865/359822.  
615 European Commission, Directorate-General for Research and Innovation, A new horizon for 
Europe – Impact assessment of the 9th EU framework programme for research and innovation, 
Luxembourg: Publications Office of the European Union, 2018, , available at: 
https://data.europa.eu/doi/10.2777/194210.  
616 European Commission Decision. C(2022)2975 of 10 May 2022. Work Programme 2021-
2022. 11. Widening participation and strengthening the European Research Area. 
617 Horizon Dashboard R&I projects. Accessed on 29 June 2023. 
https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/horizon-
dashboard.  
618 Horizon Dashboard R&I projects. Accessed on 29 June 2023. 
https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/horizon-
dashboard. 

382 

 
 
 
R
U
E
n
i
(
n
o
i
t
u
b
i
r
t
n
o
C
U
E

)
n
o

i
l
l
i

m

500

400

300

200

100

0

225
200
175
150
125
100
75
50
25
0

s
t
n
a
r
g
d
e
n
g
i
s

f
o
r
e
b
m
u
N

Teaming

Twinning

ERA Chairs

Other

EU Contribution (in EUR million)

Number of signed grants

Figure 1. SEWP actions under Horizon 2020. Note: the following actions are included under the ‘Other’ category: 
Transnational network of national contact points (NCP); Support to JPI Urban Europe; Widening Fellowships; COST, and 
WIRE Conference. Source: Horizon Dashboard R&I projects. 

Box 1. Widening actions under Horizon 2020 (and continued under Horizon Europe) 

• 

• 

Teaming.  These  actions  support  the  creation  of  new  (or  the  modernisation  of  existing) 
Centres  of  Excellence  in  widening  countries,  based  on  strategic  partnerships  with 
internationally  leading  institutions  for  excellent  research.  The  Teaming  actions  are 
expected to serve as a stimulus for institutional reforms and/or changes in the R&I systems 
of a country or region. Under Horizon 2020, Phase I Teaming calls funded the development 
of business plans for new or updated centres of excellence. Phase II calls supported the 
implementation of these centres of excellence.  

Twinning.  These  actions  aim  to  strengthen  a  particular  field  of  research  in  a  widening 
institution by encouraging networking for excellence through knowledge transfer and the 
exchange of best practices with leading institutions. The actions link a widening university 
or research organisation with at least two internationally leading research institutions from 
two different non-widening countries. This typically involves researcher exchanges, expert 
visits, workshops and outreach activities, and is hence expected to entail some institutional 
changes.  

•  ERA  Chairs.  This  action  brings  outstanding  researchers  (ERA  Chair  holders)  to 
universities  and  other  research  organisations  in  widening  countries  that  are  striving  for 
excellence in particular scientific domains. ERA Chair holders are expected to establish a 
research  team  at  the  institution  in  order  to  improve  research  performance  and  achieve 
greater  success  in  obtaining  competitive  funding.  In  other  words,  this  action  aims  to 
implement structural  changes  in  the host  institution  in order  to  achieve  excellence  on  a 
sustainable basis. 

There has been a continuation of the three key actions – Teaming, Twinning and ERA 
Chairs (see Box 1) – under Horizon Europe. Minor changes have been made on the list of 
eligible  countries  and  a  series  of  new  actions  and  instruments  has  been  introduced  as 
presented in Box 2 in Appendix 2. Structural changes cannot yet be observed for these novel 
actions and hence they are not included in the scope of the present case study.  

383 

 
 
 
 
 
 
 
 
 
1.2.2.  Structural changes as a result of widening actions 

The  widening  actions  have  been  designed  to  strengthen  the  widening  countries’  R&I 
performance, with the introduction of structural changes being one of the mechanisms via 
which this can be achieved619. 

Within the call texts of their actions the Horizon 2020 SEWP work programmes did not refer 
systematically to the importance of introducing structural changes at institutional, regional or 
national levels. ‘Facilitating structural changes in the institution’ was mentioned as a concrete 
objective  under  the  ERA  Chairs  calls.  In  the  case  of  the  Teaming  and  Twinning  calls,  no 
explicit mention was made of structural changes. 

A review of documents and reports on the widening actions shows that there is no precise 
definition  of  structural  changes.  The  SEWP  Horizon  2020  impact  study620  states:  ‘The 
most relevant way to demonstrate the societal impact in the SEWP programme is through 
the objective of strengthening the impact of R&I. In order to strengthen the impacts of R&I, 
the SEWP instruments have been designed to generate, amongst others, structural changes 
in the widening countries. These structural changes and impacts can be triggered at three 
different levels: (1) at institutional level, (2) regional level, (3) national level’ (p.58).  

In  spite  of  the  absence  of  any  overarching  definition  of  structural  changes,  some  findings 
from  previous  studies  regarding  the  prevalence  of  structural  changes  can  already  be 
highlighted. The SEWP Horizon 2020 impact report based its findings on a survey of SEWP 
beneficiaries, which led to some interesting results:  

• 

It was stated that Teaming Phase 2 achieved its intended impact of triggering reforms 
more at the coordinating institution, than at regional and national levels.  

•  With regard to Twinning projects, the report showed that more than 50% of these projects 
triggered  institutional  reforms,  while  5-10% of  the  surveyed  projects  were  indicated to 
have led to changes in national legislation.  

•  With regard to the ERA Chair actions, the analysis indicated that objectives relating to 
R&I changes and reforms were prioritised less by beneficiaries than those that related to 
increasing scientific excellence or increasing participation in Horizon 2020 and beyond. 
In spite of this, 85% of ERA Chair respondents had targeted changes at institutional level; 
62% of respondents had targeted national or regional systemic reforms; and around 15% 
indicated that they had achieved such targets. The Special Report of the European Court 
of Auditors (2022)621 shows that around 60% of the ERA Chairs projects sampled had 
generated structural changes in the host institution. 

619 European Commission, European Research Executive Agency, Spreading excellence and 
widening participation impact report – H2020 results and outlook to Horizon Europe, 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2848/30035.  
620 European Commission, European Research Executive Agency, Spreading excellence and 
widening participation impact report : H2020 results and outlook to Horizon Europe, 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2848/30035. 
621 European Court of Auditors, Measures to widen participation in Horizon 2020 were well 
designed but sustainable change will mostly depend on efforts by national authorities. Special 
report 15, 2022, Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2865/359822.  

384 

 
 
Recent  Horizon  Europe  work  programmes  have  further  stressed  the  importance  of 
generating structural changes and reforms to address disparities between those countries 
leading  in  R&I,-  and  those  lagging  behind.  Specifically, ‘structural  changes’  and ‘reformed 
R&I systems and institutions’ were explicitly mentioned as one of the expected outcomes of 
WIDERA Destination 1, ‘Improved access to excellence’, categorised under the component 
‘Widening participation and spreading excellence’. However, these work programmes do not 
include  any  precise  definition  of  the  types  of  changes  that  would  qualify  as  ‘structural 
changes’. 

1.2.3.  Overall approach and scope of the present case study 

Based on desk research carried out on documents and reports relating to the introduction of 
structural  changes  through  the  widening  actions,  this  case  study  comprises  two  main 
methodological steps:  

1.  Definition of structural changes. The Horizon 2020 reporting template and Online Manual 
did not include any specific definition of a structural change that could guide applicants 
and  beneficiaries622.  This  case  study  therefore  presents  a  definition  of  structural 
change and a framework covering the diverse types of structural changes that may be 
induced by widening actions.  

2.  Evidence collection. This case study builds evidence on a variety of sources: 

–  Survey  evidence  has  provided  indications  of  the  numbers  and  types  of 
structural changes based on surveys during the Evaluation study on Excellent 
Science  under  Horizon  2020623  and  the  SEWP  studies  and  feedback-to-
policy624. 

– 

In-depth interviews with 12 interviewees, including 5 national stakeholders 
from widening countries and 7 coordinators of widening projects that have led 
to structural changes625.  

622 This contrasts with the ‘Science with and for Society’ (SwafS) programme part, which 
provided more detailed information in its reporting template and Online Manual regarding what 
an institutional change entailed. Template available at: 
https://ec.europa.eu/research/participants/docs/h2020-funding-guide/grants/grant-
management/reports/continuous-report_en.htm.   
623 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R.et al., Evaluation study of the European framework programmes 
for research and innovation for excellent science – Horizon 2020 : annexes, Luxembourg: 
Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383.  
624 The Research Executive Agency (REA) conducted a feedback-to-policy survey in 2023 
targeted at the coordinators of Horizon 2020 widening projects, which included questions on 
whether the project led to institutional, regional or national structural changes in R&I systems. 
The definitions of institutional changes used in the present report are aligned with those used by 
the REA in its survey. 
625 Projects leading to structural changes were identified through the feedback-to-policy survey 
carried out by REA. 

385 

 
 
1.3.  Case study findings 

1.3.1.  Structural changes – a conceptual framework 

Based  on  the  aforementioned  literature  review  and  interviews,  the  following  definition  of 
structural change is suggested for the purposes of this case study:   

A structural change can be defined as a sustainable and/or permanent shift or change 
introduced a result of or triggered by a Widening action under SEWP (Horizon 2020) or 
WIDERA (Horizon Europe), which is manifested at the institutional, regional or national 
level, and is aimed at fostering R&I performance.  

This definition hence stresses the fact that changes can occur at different levels: within the 
host  institution  (i.e.  institutional  structural  changes);  in  the  regional  R&I  environment  (i.e. 
regional-level  structural  changes);  or  at  national  level626.  It  also  requires  that  the  changes 
must  be  sustainable  or  permanent  (i.e.  ad  hoc,  temporary  changes  in  the  R&I  system  or 
institution would not be considered structural changes). 

Structural changes can be classified according to their expected impact on:  

1.  Strengthening the R&I system and its governance; 

2. 

Improving working conditions, career progression and skills; 

3.  Fostering research excellence and collaboration; 

4.  Strengthening the research environment and infrastructure. 

Concrete  examples  of  potential  structural  changes  from  each  of  these  four  categories  of 
impacts are presented in Table 1 in Appendix 2. 

1.3.2.  Structural changes introduced as a result of Horizon 2020 widening actions 

The sections that follow offer insights into the structural changes that have been introduced 
thanks to or triggered by widening actions during Horizon 2020, first at institutional level, and 
then at national and regional levels. These changes are classified according to their expected 
impacts (i.e. in line with the framework presented above).  

1.3.2.1. 

Structural changes at institutional level 

Strengthening the R&I system and its governance 

Several  Teaming,  Twinning  and  ERA  Chairs  projects  have  triggered  (or expect  to  trigger) 
changes aimed at strengthening the R&I system and governance at their host institutions. In 
the survey conducted during the Evaluation study on Excellent Science under Horizon 2020, 

626 European Commission, European Research Executive Agency, Spreading excellence and 
widening participation impact report : H2020 results and outlook to Horizon Europe, 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2848/30035.  

386 

 
 
84% of respondents indicated that their widening projects had (or were expected to have) a 
positive  impact  in  terms  of  enhancing  the  host  institution’s  capacity  to  compete  for  EU 
funding. In addition, nearly 80% of respondents reported an (expected) improved ability to 
secure regional or national funding as a direct result of the widening project. Furthermore, 
approximately  60%  of  respondents  noted  that  the  SEWP  project  had  facilitated  the 
establishment or strengthening of administrative structures, processes and standards in the 
host institution, aimed at ensuring the effective administration and management of research 
projects. 

Becoming more capable to compete for EU
funding

41%

43%

7%

5%

4%

Becoming more capable to compete for regional
or national funding

42%

37%

7%

5%

5%

Establishment/strengthening of clearly defined
administrative structures, processes or standards
that ensure the effective…

23%

35%

14%

5%

16% 7%

Strongly agree
Strongly disagree

Rather agree
Not applicable for my project

Rather disagree
Do not know/ cannot answer

0%

20%

40%

60%

80%

100%

Figure 2. Structural changes aimed at strengthening the R&I system and its governance in the host institution. Source: 
survey of Horizon 2020 beneficiary organisations, conducted August-September 2022 (n=81). Question: /Would you agree 
or disagree that your Horizon 2020 project has contributed/will contribute to the following developments in your 
organisation?’. 

Improving working conditions, career progression and skills 

Variation exists in the extent to which widening projects have improved (or are very likely to 
improve)  the  working  conditions,  skills  and  career  prospects  of  researchers  in  the  host 
institution, with some changes being widespread than others.  

The  survey  of  beneficiaries  conducted  during  the  Evaluation  study  on  Excellent  Science 
under  Horizon  2020  indicates  that  the  widening  projects  had  a  notable  impact  on  host 
institutions  by  enhancing  the  quality  of  training  and  fostering  the  development  of 
researchers'  skills,  knowledge  and  competences.  More  than  80%  of  respondents 
reported that the project had positively influenced (or would positively influence) the quality 
of training offered to researchers. In addition, more than 70% of respondents acknowledged 
the project's contribution to improving future supervision practices within the institution (as 
depicted in Figure 7 in Appendix 2). Moreover, nearly 90% of respondents believed that their 
widening  project  had  improved  (or  would  improve)  the  research  skills,  knowledge  and 
competences of researchers, as well as researchers’ transferable skills (Figure 3). Similarly, 
a  majority  of  respondents  indicated  that  the  widening  project  had  increased  (or  would 
increase) the cross-country and cross-sectoral mobility of researchers in the institution, while 
approximately three out of four respondents said that the host institution was (or would be) 
able to attract more talented researchers from abroad.  

The findings of the survey carried out for the Evaluation study on Excellent Science under 
Horizon 2020 indicated that the widening actions had fewer effects on improving working 
conditions and making procedures more transparent: 59% of respondents believed that 

387 

 
 
researchers´ career management processes had improved (or would improve), while 42% 
felt  these  processes  at  their  institution  had  become  (or  would  become)  more  transparent 
thanks to the project. Less than half of the respondents noted that their widening projects had 
led  (or  would  lead)  to  improvements  or  increased  flexibility  in  the  working  conditions  for 
researchers within their institutions, or that salaries in their institutions had been made more 
attractive.  

More  positive  results  were  found  with  regard  to  the  widening  actions’  impact  on  gender 
equality: half of the respondents indicated that their widening project had resulted (or would 
result)  in  advances  being  made  towards  achieving  gender  equal  opportunities  within  their 
institutions. 

Improving the transferable skills and
competences of researchers (project…

Improving the research skills, knowledge and
competences of researchers

73%

71%

14%

11%

16%

6%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%

To a large extent

To a moderate extent

To a small extent

Not at all

Not applicable to my project

Do not know/cannot answer

Figure 3. Results of the widening project in relation to improving working conditions, career progression and skills. Source: 
survey of Horizon 2020 beneficiary organisations, conducted August-September 2022 (n=84). Question: ‘To what extent, if 
at all, has your Horizon 2020 project achieved/is likely to achieve the following results?’. 

Fostering research excellence and collaboration (including non-academic) 

The survey627 responses from widening beneficiaries were very positive with regard to the 
(expected)  ability  of  their  widening  project  to  increase  capacities  for  networking  with 
world-class institutions (Figure 4). Furthermore, 81% of the survey respondents reported 
that  their  project  had  resulted  in  (or  would  result  in)  an  increase  in  research  productivity. 
Approximately  half  of  respondents  indicated  that  a  new  research  programme  had  been 
established with the consortium partners as an outcome of the widening project concerned. 
Slightly less than half of respondents acknowledged that their widening project had influenced 
(or would influence) the development or improvement of standards or appraisal processes 
in  research  ethics  and  integrity  in  the  host  institution.  However,  a  majority 
(approximately  three-quarters  of  respondents)  believed  that  the  widening  projects  had 
effectively  enhanced  their  institution’s  competence  in  implementing  Open  Science 
practices. 

627 Survey in the context of the Evaluation study on Excellent Science under Horizon 2020. 

388 

 
 
 
Capacities of networking with world-class
institutions were increased

46%

36%

11%4%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

To a large extent

To a moderate extent

To a small extent

Not at all

Do not know/cannot answer

Figure 4. Results of the projects in relation to increased capacity for networking with world-class institutions. Source: survey 
of Horizon 2020 beneficiary organisations, conducted August-September 2022 (n=84). Question: ‘To what extent, if at all, 
has your Horizon 2020 project achieved/is likely to achieve the following results?’. 

Becoming more competent at implementing Open
Science practices (publishing in open access journal,
citizen science, science communication, etc.)

Development/improvement of standards and/or
appraisal processes in research ethics and/or
research integrity

37%

36%

8%

5%

7%

7%

17%

28%

14%

5%

25%

11%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%

Strongly agree

Rather disagree

Rather agree

Strongly disagree

Not applicable for my project

Do not know/ cannot answer

Figure 5. Structural changes aimed at inducing new ways of performing science. Source: survey of Horizon 2020 beneficiary 
organisations, conducted August-September 2022 (row 1: n=75; row 2: n=81). Question: ‘Would you agree or disagree that 
your Horizon 2020 project has contributed/will contribute to the following developments in your organisation?’. 

Strengthening the research environment and infrastructure 

Approximately half of respondents who participated in the survey indicated that their widening 
project had played (or would play) a role in the development of  physical and knowledge 
infrastructure and facilities within their institutions. This specific structural change is thus 
not targeted as frequently as some of the other aspects covered above.  

Contribution to the development of physical
and knowledge infrastructures and facilities

26%

25%

24%

9% 15% 3%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%

Strongly agree

Rather disagree

Rather agree

Strongly disagree

Not applicable for my project

Do not know/ cannot answer

Figure 6. Results of the projects in relation to the development of physical and knowledge infrastructure. Source: survey of 
Horizon 2020 beneficiary organisations, conducted August-September 2022 (n=81). Question: ‘To what extent, if at all, has 
your Horizon 2020 project achieved/is likely to achieve the following results?’. 

389 

 
 
 
 
Added value of widening actions in producing institutional structural changes 

The interviews and survey evidence indicate that the widening actions have generated very 
positive outcomes that could not have occurred in the absence of EU funding. In general, 
interviewees  indicated  that  there  are  no  comparable  grants  in  the  widening  regions  or 
countries  that  would  allow  this  type  of  projects  or  which  would  produce  similar  outcomes, 
particularly in terms of fostering connections with leading partners or boosting the quality of 
the research produced.  

While  the  survey carried  out  for the  Evaluation  study  on  Excellent  Science  under  Horizon 
2020indicates that many Horizon 2020 projects triggered the types of changes listed above 
at  institutional  level,  changes  at  regional  or  national  level  appear  more  specific  to 
widening actions. The next section presents evidence regarding this type of changes.  

1.3.2.2. 

Structural changes at regional and/or national level  

While the section above describes structural changes occurring at institutional level, widening 
actions  have  also  shown  positive  results  in  inducing  structural  changes  and  reforms  at 
regional or national level. This section presents concrete examples of structural changes at 
the  level  of  widening  regions  or countries  that  have  been  realised  as  a  result  of  widening 
projects funded under Horizon 2020. These changes have mainly been self-reported by the 
project coordinators in a survey conducted by REA in 2023 (see footnote 624), as well as 
through interviews with national stakeholders and project beneficiaries in the context of this 
study628. Again, the changes are classified according to their expected impacts (i.e. in line 
with the framework presented in Section 1.3.1). 

Strengthening the R&I system and its governance 

Several  projects  can  be  associated  with  this  type  of  structural  change,  in  particular  with 
regard to their contribution to: 

•  The development of regional or national R&I strategies. STRENTEX629, for example, 
contributed to establishment of the regional strategy for scientific development in Serbia; 
PATSAFE630 provided input into the national patient safety strategy in Estonia; and CMMI 
– MaRITeC-X631 influenced the long-term strategy for sustainable growth in Cyprus (a 
Teaming project). A key researcher of the Serbian Twinning projects, LAMBDA632 and 

628 With a few exceptions: some changes have been deduced through interactions with EC 
project officers knowledgeable about the projects, while one was obtained from the SEWP 
Horizon 2020 impact report. Some of these structural changes have been discussed in greater 
depth during interviews with project coordinators or relevant regional or national stakeholders. 
For a of interviewees, see Annex  2 of this case study. European Commission, European 
Research Executive Agency, Spreading excellence and widening participation impact report: 
H2020 results and outlook to Horizon Europe, Luxembourg: Publications Office of the European 
Union, 2022, available at: https://data.europa.eu/doi/10.2848/30035.  
629 ERA Chair for emerging technologies and innovative research in Stretchable and Textile 
Electronics (STRENTEX); https://cordis.europa.eu/project/id/854194.  
630 Developing the University of Tartu into a well-networked PATient SAFEty research center in 
Estonia (PATSAFE); https://cordis.europa.eu/project/id/857359.  
631 Marine and Maritime Research, Innovation, Technology Centre of Excellence (CMMI – 
MaRITeC-X); https://cordis.europa.eu/project/id/857586   
632 Learning, Applying, Multiplying Big Data Analytics (LAMBDA); 
https://cordis.europa.eu/project/id/809965.  

390 

 
 
SINERGY633, was invited to participate in governmental working groups for the drafting 
of  national  guidelines  on  the  implementation  of  trustworthy  artificial  intelligence.  Even 
prior to this invite, a long-standing relationship had already been developed between the 
coordinator and the government sector. Specifically, the government sector is aware of 
the researchers’ work and follows the EU projects, and the researchers have supported 
the  government  in,  for  example,  monitoring  and  implementing  EU  strategies  and 
directives at national level.  

•  The  development  of  Smart  Specialisation  Strategies.  Similarly,  while  widening 
projects (especially Teaming projects) are typically in line with the Smart Specialisation 
Strategies  (as  requested  in  the  call  texts),  multiple  projects  have  contributed  to  the 
development of these strategies. Smart Specialisation Strategies refer to the strategic 
approach taken by regions or countries to focus on and develop their unique strengths, 
potential  and  competitive  advantages  in  specific  areas  of  research,  innovation  and 
economic activity.  

−  BrainTwin634 in Romania (a Twinning project), aimed to increase networking activities 
and strengthen collaboration in the field of neuroengineering. The project introduced 
potential future research collaboration topics into the updated Research & Innovation 
Smart Specialisation Strategy (RIS3) for the North-East Region. Meanwhile, several 
partners of the NEUROTWIN635 project (Twinning), which aimed to stimulate and step 
up scientific excellence and innovation capacity in the field of cellular and molecular 
neuroscience, contributed to the development of the Smart Specialisation Strategy for 
Kyiv, Ukraine. The VENTuRE636 project (Twinning) worked towards the creation of a 
virtual and experimental towing centre. As a result, the marine and maritime themes 
have been reinforced in Malta’s national Smart Specialisation Strategy.  

•  The  introduction  of  changes  in  research  agendas  or  priorities  in  the  regions  or 
countries.  For  example,  SIRAMM637  (Twinning),  aimed  to  create  a  network  involving 
leading Norwegian and Italian universities to create a pole of excellence on advanced 
manufacturing in Eastern Europe, triggered the inclusion of Additive Manufacturing as a 
priority in the local policy in Western Romania.  

Other  changes  pertain  to  the  strengthening  of  the  R&I  funding  systems  in  regions  or 
countries. The NEUROTWIN638 (Twinning) project contributed to development of the National 
Research Foundation in Ukraine. MaNaCa639, which focused on the structural and magnetic 
characterisation  of  magnetic  nanohybrids  and  their  application  for  cancer  therapy, 
contributed  to  the  launch  of  new  calls  of  the  Science  committee  of  Armenia  (Twinning). 

633 Capacity building in Smart and Innovative eNERGY management (SINERGY); 
https://cordis.europa.eu/project/id/952140.  
634 Development of a World-Level Neuroengineering Research Centre by European Twinning; 
https://cordis.europa.eu/project/id/952378.  
635 Pan-European twinning to re-establish world-level Neuroscience Centre in Kiev 
(NEUROTWIN); https://cordis.europa.eu/project/id/857562.  
636 a Virtual and physical ExperimeNtal Towing centre for the design of eneRgy Efficient sea-
faring vessels (VENTuRE); https://cordis.europa.eu/project/id/856887.  
637 Eastern European twinning on Structural Integrity and Reliability of Advanced Materials 
obtained through additive Manufacturing (SIRAMM); https://cordis.europa.eu/project/id/857124.  
638 Pan-European twinning to re-establish world-level Neuroscience Centre in Kiev 
(NEUROTWIN); https://cordis.europa.eu/project/id/857562.  
639 Magnetic Nanohybrids for Cancer Therapy (MaNaCa); 
https://cordis.europa.eu/project/id/857502.  

391 

 
 
Another Teaming project, KIOS CoE640, contributed to the establishment of the first Deputy 
Ministry for Research and Innovation in Cyprus (Teaming). KIOS CoE´s aim is to develop the 
KIOS Research and Innovation Centre of Excellence (KIOS CoE) to conduct multidisciplinary 
basic and applied research and innovation in the area of intelligent systems and networks, 
with  an  emphasis  on  the  monitoring,  control,  management  and  security  of  critical 
infrastructures. 

Improving working conditions, career progression and skills 

Multiple  Horizon  2020  widening  projects  have  been  identified  as  having  brought  about 
significant improvements in working conditions, career advancement and researchers' skills 
in various regions and countries. An example of this is BBCE641 (Teaming), which focused 
on  establishing  a  centre  of  excellence  and  creating  a  critical  mass  of  excellence  in 
biomaterials research for bone regeneration in the Baltics. This project had a direct impact 
on  national  regulations  on  immigration,  particularly  in  response  to  the  COVID-19 
pandemic. Towards the end of 2020, COVID-19 related restrictions on immigration from third 
countries hindered the arrival in Latvia of postdoctoral researchers recruited from India, Iran 
and Turkey. However, through successful collaboration with Latvia's Ministries of Education 
and  Science  and  of  Economics,  BBCE  influenced  the  adaptation  of  regulations,  allowing 
research  staff  to  enter  the  country  if  their  intended  stay  lasted  for  six  months  or  longer. 
Likewise,  Innorenew  CoE642  (Teaming),  which  aimed  to  improve  sustainable  building 
practices, facilitated changes in visa arrangements for incoming researchers in Slovenia. 
In  addition,  other  projects  have  also  contributed  to  changes  in  the  educational  landscape, 
typically  leading  to  an  increase  in  student  enrolment  in  these  countries.  For  example, 
MaTek643 (Twinning), a network focused on training mathematics educators, is contributing 
to  curriculum  reform  in  Slovakia.  Specifically,  the  coordinator  has  been  asked  to  share 
insights from the project to be incorporated into the reform. In addition, they are contributing 
to textbooks and materials used in classrooms. 

Fostering research excellence and collaboration (including non-academic) 

Several  widening  projects  have  led  to  regional  or  national  changes  aimed  at  fostering 
research  excellence  and  (international)  collaboration.  INTEGROMED644  (Twinning)  and 
Sano645 (Teaming) both contributed to novel  procedures  related to medical data  usage 
and sharing, which have been (or will be) taken up in a new legislative framework in Latvia, 
and  shared  among  the  institutions  in  Poland,  respectively.  For example,  researchers  from 
INTEGROMED  hosted  a  meeting  that  kickstarted  a  discussion  on  national  legislation 
regarding medical data that is in line with the GDPR. In addition, the researchers contributed 
to parts of a new biobank law in Latvia. Both pieces of legislation are still being debated.  

640 KIOS Research and Innovation Centre of Excellence; 
https://cordis.europa.eu/project/id/739551.  
641 Baltic Biomaterials Centre of Excellence (BBCE); https://cordis.europa.eu/project/id/857287.  
642 Renewable materials and healthy environments research and innovation centre of excellence 
(InnoRenew CoE); https://cordis.europa.eu/project/id/664331.  
643 Enhancement of research excellence in Mathematics Teacher Knowledge (MaTek); 
https://cordis.europa.eu/project/id/951822.  
644 Integration of knowledge and biobank resources in comprehensive translational approach for 
personalized prevention and treatment of metabolic disorders (INTEGROMED); 
https://cordis.europa.eu/project/id/857572.  
645 Centre for New Methods in Computational Diagnostics and Personalised Therapy; 
https://cordis.europa.eu/project/id/857533.  

392 

 
 
Strengthening the research environment and infrastructure 

Numerous widening projects have led to regional or national changes aimed at strengthening 
the research environment and infrastructure. In line with the call texts and objectives of these 
actions, all (Phase 2) Teaming projects have set up or modernised centres of excellence 
with the overarching objective of increasing the scientific capabilities in widening countries 
and  hence  fostering  economic  development,  improving  chances  of  gaining  competitive 
funding, and advancing national or regional R&I performance.  

In  addition,  several  Twinning  projects  have  increased  knowledge,  expertise  and 
understanding  of  how  to  build,  use  and  operate  new  infrastructures,  facilities  and 
laboratories in widening countries. Examples include the project VENTuRE646 (Twinning), in 
which Maltese researchers learned how to use, operate and build a towing tank and use it as 
a  teaching  tool;  and  the  ReNature647  project  (Twinning),  which  aimed  to  foster  scientific 
excellence and boost innovation capacity in Malta in the field of nature-based solutions for 
sustainable development. This project played a significant role in driving reforms pertaining 
to national green infrastructure. The CETOCOEN Excellence648 project (Teaming) aimed 
to  update  the  RECETOX  Centre  and  become  the  European  Centre  of  Excellence  in 
Environmental Health Sciences. The activities of this project contribute to the establishment 
of EU research infrastructure. For example, the centre is further developing a biobanking 
facility, setting  up  sample-handling  laboratories  and  making  structural  changes  to  the  way 
research infrastructure is organised.649 The centre has also formally become a national centre 
in Czechia that receives funding from the government, and has an agreement with the city of 
Brno to build a ‘living lab’ together in Brno.  

1.4.  Success factors and barriers 

In interviews, several factors were identified as playing a role in fostering the introduction of 
structural changes: 

•  The  alignment  of  the  projects  with  national  priorities,  together  with  political 
interest, appear to be factors determining the extent to which widening projects achieve 
an impact at regional and national levels. Political commitment is usually present from 
the start in the case of Teaming projects, as these projects need to secure part of their 
funding  from  other  sources,  usually  from  the  European  Regional  Development  Fund 
(ERDF), or from national and regional funding. Nevertheless, the cases analysed in the 
present case study indicate that this alignment, commitment or interest is also important 
in  the  case  of  other  widening  actions,  in  particular  in  some  of  the  Twinning  projects 
identified by this case study. 

•  Country size is mentioned by interviewees as a relevant factor facilitating the impact of 
the  widening  actions  in  inducing  structural  changes  at  regional  and  national  levels. 
Interviewees indicated that it is easier for projects to have this type of impact in smaller 
and relatively less complex R&I systems. 

646 A Virtual and physical ExperimeNtal Towing centre for the design of eneRgy Efficient sea-
faring vessels (VENTuRE); https://cordis.europa.eu/project/id/856887. 
647 Promoting Research Excellence in NAture-based soluTions for innovation, sUstainable 
economic GRowth and human wEll-being in Malta (ReNature); 
https://cordis.europa.eu/project/id/809988.  
648 CETOCOEN Excellence; https://cordis.europa.eu/project/id/857560. 
649 https://cordis.europa.eu/project/id/857560/reporting. Accessed on 14/07/2023. 

393 

 
 
• 

Interviewees  also  pointed  out  that  the  actions  taken  by  researchers  themselves  to 
reach  out  are  important.  To  give  an  example,  if  the  government  calls  for  experts  to 
engage in working groups on specific guidelines or regulations in the field of the widening 
project, it is important that researchers engage in such activities. One interviewee argued 
that  if  researchers  already  have  experience  in  working  on  government  projects  or  in 
working  groups,  this  often  means  that  policy-makers  already  know  these  experts  in 
particular fields, which may stimulate the engagement of researchers to reach out.  

Several  barriers  to  this  type  of  impact  were  also  identified  during  interviews  and  desk 
research: 

•  The absence of a specific monitoring and reporting approach to measure impacts in 
terms  of  structural  changes  mean  that  project  applicants  and  policy-makers  are  less 
attentive  to  defining  and  developing  these  types  of  impact.  In  addition,  the  fact  that 
different  policy  domains  may  be  fragmented  at  national  policy-making  level  makes  it 
more difficult for projects to have an impact beyond their specific topics. For instance, 
Smart Specialisation Strategies at national and/or regional level are usually developed 
under  ministries  of  industry  or  similar,  while  research  infrastructures  are  usually  the 
domain of ministries of science and education. This creates silos that are difficult to break 
out  of  –  not  only  for  project  beneficiaries,  but  also  for policy-makers  themselves.  The 
introduction  of  new  modes  of  governance  such  as  a  ‘whole-of-government’  approach 
could  help  to  achieve  more  directed  efforts  and  thereby  maximise  the  potential  of 
widening. 

• 

Interviewees also pointed out that limited R&I public spending at country level limits 
the potential of projects. Follow-up and complementary funding for widening projects is 
seen  by  some  beneficiaries  as  a  possible  way  to  foster  the  impact  of  these  projects, 
including their capacity to induce structural changes at institutional, regional and national 
levels. This is also highlighted in positioning papers650 in the context of the OPC651 and 
in the European Court of Auditors report, in which it was argued that efforts at national 
level with regard to widening are necessary to achieve sustainable, long-term changes.  

1.5.  Conclusions  

The  widening  actions  have  proved  effective  in  inducing  diverse  types  of  structural 
changes. These are perceived by project beneficiaries and national-level stakeholders 
interviewed as having had a positive effect on R&I performance. Moreover, the evidence 
gathered indicates that the types of impact achieved and the level at which they occur are 
diverse.The  evidence  gathered  also  shows  that  structural  changes  that  occur  at 
institutional level appear to be widespread and common across the three types of main 
widening actions analysed here,  There is also evidence pointing to a greater number of 
changes occurring at institutional level than at regional or national levels. Some of the main 
factors  explaining  this  difference  is  the  fact  that  implementing  changes  at  the  regional  or 
national level requires ‘traveling’ further compared to changes within an institution. This is 

650 e.g. European Regions Research and Innovation Network (ERRIN) (2023), Errin Input To The 
Public Consultation On The Past, Present And Future Of The European Research & Innovation 
Framework Programmes 2014-2027; and Universities of the Netherlands (UNL), Netherlands 
Federation of University Medical Centres (NFU) & The Royal Netherlands Academy of Arts and 
Sciences (KNAW) (2023)/ Position paper, Horizon Europe and Beyond.  
651 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 

394 

 
 
because the results and impact must extend over a ‘greater distance’, in contrast to changes 
within one’s own institution. Furthermore, institutional changes can be more narrowly defined 
and  applicable  to  all  projects,  and  may  be  easier  to  attribute  to  the  widening  project  than 
those occurring at national or regional level.  

Structural changes at regional and/or national level are more diverse in nature, scope 
and degree of impact. This diversity as to the types and scope of the structural changes 
observed at regional and national levels hinders a more quantitative analysis of these types 
of  effects.  Among  the  most  frequent  national  or  regional  structural  changes  identified  are 
contributions  to  the  development  of  regional  or  national  R&I  strategies,  including  Smart 
Specialisation  Strategies,  as  well  as  the  implementation  of  actions  leading  to  increased 
knowledge and expertise in the building, use and operation of new infrastructures, facilities 
and laboratories. The mechanisms through which these changes occur usually relate to how 
a  project  is  designed,  as  well  as  its  aims  and  objectives,  and  is  also  associated  with  the 
degree of political involvement and alignment with national or regional policies and priorities.  

The  absence  of  a  clear-cut  definition  of  structural  changes  in  the  context  of  the 
widening  actions  (beyond  a  mere  list  of  changes)  hinders  the  monitoring  and 
evaluation of this type of impact. Although the evidence gathered for this case study shows 
positive indications of impact at institutional, regional and national levels, a more consolidated 
and robust monitoring approach (e.g. including a definition and specific categories of impact) 
would facilitate the monitoring of the progress made by the actions in this regard. This would 
require the inclusion in their call texts and monitoring templates of a more explicit definition 
of  structural  changes,  along  with  a(n  unexhaustive)  list  of  (potential)  structural  changes 
expected in the call texts and the monitoring templates. This approach would not only help to 
monitor the progress of the actions in this regard, but would also help to identify both barriers 
and success stories that can inspire future widening applicants and beneficiaries. 

Appendix 1. Literature sources 

•  Cordis webpages of selected projects. 

•  Commission Staff Working Document 221 final (2017). Interim evaluation of Horizon 
2020 Annex II. https://op.europa.eu/en/publication-detail/-/publication/4cedc9a0-d8cc-
11e8-afb3-01aa75ed71a1/language-en/format-PDF/source-288823388.  

•  European Commission Decision C (2015)5668 of 14 August 2015, Horizon 2020 Work 

Programme 2014- 2015, 15. Spreading Excellence and Widening Participation 
(revised). 

•  European Commission Decision C(2017)2468 of 24 April 2017, Horizon 2020 Work 
Programme 2016- 2017, 15. Spreading Excellence and Widening Participation 
(revised). 

•  European Commission Decision C(2019)4575 of 2 July 2019, Horizon 2020 Work 
Programme 2018- 2020, 15. Spreading Excellence and Widening Participation 
(revised). 

•  European Commission Decision C(2022)2975 of 10 May 2022, Horizon Europe, Work 
Programme 2021-2022, 11.Widening participation and strengthening the European 
Research Area.  

395 

 
•  European Commission Decision C(2022)7550 of 6 December 2022, Horizon Europe, 
Work Programme 2023-2024, 11. Widening participation and strengthening the 
European Research Area. 

•  European Commission, Directorate-General for Research and Innovation, Horizon 
2020 indicators – Assessing the results and impact of Horizon, Luxembourg: 
Publications Office of the European Union, 2015, available at: 
https://data.europa.eu/doi/10.2777/71098.  

•  European Commission, Directorate-General for Research and Innovation, Interim 

evaluation of Twinning and ERA chairs in Horizon 2020, Luxembourg: Publications 
Office of the European Union, 2018, available at: 
https://data.europa.eu/doi/10.2777/998775. 

•  European Commission, Directorate-General for Research and Innovation, Puukka, J., 
Spreading excellence and widening participation in Horizon 2020: analysis of FP 
participation patterns and research and innovation performance of eligible countries, 
Luxembourg: Publications Office of the European Union, 2018, available at: 
https://data.europa.eu/doi/10.2777/42382. 

•  European Commission, Directorate-General for Research and Innovation, A New 

Horizon for Europe – Impact Assessment of the 9th EU Framework Programme for 
Research and Innovation, Luxembourg: Publications Office of the European Union, 
2018, available at: https://data.europa.eu/doi/10.2777/194210.  

•  European Commission, European Research Executive Agency, Spreading excellence 

and widening participation impact report: H2020 results and outlook to Horizon Europe, 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2848/30035. 

•  European Commission, European Research Executive Agency, Dobranskyte-Niskota, 

A., Fernandez-Wyttenbach, A., Salas, E., ERA chair dynamics in the widening 
programme – Report on geographical aspects (2014-2022), Luxembourg: Publications 
Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2848/719715. 

•  European Commission, Directorate-General for Research and Innovation, Budraitis, M., 

Pranckevičius, P., Dėlkutė, R.et al., Evaluation study of the European framework 
programmes for research and innovation for excellent science – Horizon 2020: 
annexes, Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383.  

•  European Commission, Horizon 2020 Online Manual, 

https://ec.europa.eu/research/participants/docs/h2020-funding-guide/grants/grant-
management/reports/continuous-report_en.htm. 

•  European Court of Auditors, Measures to widen participation in Horizon 2020 were well 

designed but sustainable change will mostly depend on efforts by national authorities. 
Special report No 15, 2022, Luxembourg: Publications Office of the European Union, 
2022, https://data.europa.eu/doi/10.2865/359822. 

396 

 
•  European Regions Research and Innovation Network (ERRIN) (2023), ERRIN Input To 
The Public Consultation On The Past, Present And Future Of The European Research 
& Innovation Framework Programmes 2014-2027. 

•  Research Executive Agency, feedback-to-policy survey, 2022, available at: 

https://rea.ec.europa.eu/funding-and-grants/horizon-europe-widening-participation-and-
spreading-excellence/project-results-and-feedback-policy_en.  

•  SWD(2017) 221 final (2017). Interim evaluation of Horizon 2020 Annex II. 

https://op.europa.eu/en/publication-detail/-/publication/4cedc9a0-d8cc-11e8-afb3-
01aa75ed71a1/language-en/format-PDF/source-288823388. 

•  Universities of the Netherlands (UNL), Netherlands Federation of University Medical 
Centres (NFU) & The Royal Netherlands Academy of Arts and Sciences (KNAW) 
(2023). Position paper, Horizon Europe and Beyond. 

Appendix 2. Additional information 

Box 2. New widening actions introduced under Horizon Europe652 

New widening actions introduced in Horizon Europe 

• 

• 

• 

• 

• 

The Hop-on Facility. This offers widening institutions the opportunity to join already-selected 
collaborative consortia. In doing so, this action promotes international networking and the 
participation of widening institutions. 

Excellence Hubs. This aims to strengthen regional innovation excellence in place-based 
innovation ecosystems through cross-border collaboration on a common strategy. This action 
allows innovation ecosystems in widening countries to team up and create better linkages 
between academia, business, government and society. 

The European Excellence Initiative (EEI) aims to transform the higher education sector and 
surrounding ecosystems. Through cooperation, the widening dimension of the European 
Excellence Initiative aims to raise the excellence in science and the valorisation of knowledge 
of Europe’s universities. 

The Pathways to synergies action aims to support the establishment of interfaces between 
two distinct funding systems, to address existing barriers resulting from disparities between 
regional and European approaches. The primary objective of the pathway is to transition 
individual beneficiaries of regional funding programmes from isolation to active engagement 
through cross-border collaboration. Its purpose is to prepare them for participation in Horizon 
Europe calls. 

The Dissemination and Exploitation Support Facility aids the beneficiaries of widening actions 
in enhancing the efficiency of their dissemination and exploitation efforts, as well as in 
accessing new funding opportunities. Enhancing the diffusion of knowledge, promoting 

652 Widening participation and spreading excellence. Accessed on 12 July 2023, available at: 
https://research-and-innovation.ec.europa.eu/funding/funding-opportunities/funding-
programmes-and-open-calls/horizon-europe/widening-participation-and-spreading-
excellence_en.  

397 

 
 
technology adoption and fostering spill-over effects are crucial in order for researchers and 
their institutions to capitalise on the most up-to-date knowledge and advances. 

• 

ERA Talents. This action aims to enhance the compatibility of career paths and employment 
opportunities for R&I talents, particularly in widening countries. Its main emphasis is on 
promoting the exchange of talents across different sectors, as well as fostering collaborations 
between academia and businesses to facilitate the transfer of knowledge. 

Examples of structural changes 

Type of structural 
change 

Strengthening the 
R&I system and its 
governance 

Improving working 
conditions, career 
progression and 
skills 

Examples 

•  Changes in governance and management* 
•  Drafting or adapting research (and innovation) agendas, strategies, 

action plans 

•  Changes in R&I monitoring or evaluation systems 
• 
Attracting/increasing R&I funding and investments* 
•  Creation of working groups, committees, platforms  
• 

Setting up/reform of R&I funding systems, policies, instruments and/or 
the development of/reforms to evaluation and assessment for funding 

•  Contributing to Smart Specialisation Strategies (e.g. the project 
coordinator was involved in design/development of the S3 of the 
region) 

• 
Establishment of new research/doctoral programmes 
•  Creation/revision of HR guidelines, procedures, policies or 

• 
• 

regulations* 
Introduction of changes to PhD supervision and training schemes 
Endorsement and application of the EU Charter for Researchers and 
the Code of Conduct for the Recruitment of Researchers, or one of its 
principles: 
– 
– 
– 
– 
– 
– 

transparency in hiring conditions; 
international recruitment panel; 
gender balance; 
demonstrable career development aspects; 
including mentorship & training; and 
stable contract conditions. 

• 

Staff mobility (e.g. attracting international researchers or national 
researchers that have worked abroad) 

Fostering research 
excellence and 
collaboration 
(including non-
academic) 

•  Developing new research programmes 
•  Creation/revision of other guidelines, procedures, (IPR, ethics, quality 

management systems)* 

•  Developing/adapting instruments that promote or facilitate 

international collaboration (e.g. engaging in new bilateral science 
agreements*), developing international R&I collaborations 

398 

 
 
 
 
Type of structural 
change 

Examples 

•  Creation of Knowledge and Technology Transfer Offices*, knowledge 

transfer infrastructure, or new local research and innovation 
partnerships 
Inducing new ways of performing science/influencing the overall 
science system of the country (e.g. open science practices)* 

• 

Strengthening the 
research 
environment and 
infrastructure 

Setting up and/or modernisation of centres of excellence (‘Teaming’) 

• 
•  Creation or modernisation of laboratories, research 
department/centre/ teams/units/infrastructure* 
Setting up/development/enhancement/restructuring of research 
support services* 

• 

Table 1. Proposed categories of structural change. Note: *included as a structural change in the survey of project 
beneficiaries launched by REA in 2023653. 

653 Feedback-to-policy survey (2022) conducted by the Research Executive Agency (REA), 
targeted at the coordinators of Horizon 2020 widening projects. 

399 

 
 
 
 
 
Improving working conditions, career progression and skills 

Increasing cross-country and cross-sector mobility
of researchers

49%

31%

4%

5%

7%

4%

Improvement of the quality of training provided to
researchers

43%

38%

6%

7%

4%

Improvement of the quality of supervision provided
to researchers

37%

36%

Attracting talented researchers to your organisation
from abroad

37%

35%

9%

3%

11%

9%

4%

11%

5%

5%

Advancement of equal opportunities through
gender mainstreaming/new gender equality policies
(e.g. Gender Equality Plans)

21%

27%

10%

7%

26%

9%

Procedures and practices for the career
management of researchers were improved

19%

40%

9%

9%

15%

9%

Procedures and practices for the recruitment of
researchers became more transparent and clear

12%

30%

12%

10%

27%

9%

Working conditions for researchers were improved
and made more flexible (esp. in terms of flexible
working hours, work-life balance, introduction of…

11%

21%

12%

11%

35%

10%

Salaries of researchers were made more financially
attractive

10%

17%

15%

19%

30%

10%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%

Strongly agree

Rather disagree

Rather agree

Strongly disagree

Not applicable for my project

Do not know/ cannot answer

Figure 7. Structural changes aimed at improving working conditions, career progression and skills. Source: survey of 
Horizon 2020 beneficiary organisations, conducted August-September 2022 (n=81). Question: ‘Would you agree or disagree 
that your Horizon 2020 project has contributed/will contribute to the following developments in your organisation?’. 

400 

 
 
 
Annex 2.8: WIDERA Actions on enhancing the European RI system 

1.1. 

Introduction 

Following on from Coordination and Support Actions (CSAs) under previous FPs, Horizon 
Europe’s WIDERA programme part, which focuses on the European Research Area (ERA), 
has introduced a number of policy measures to address the ERA policy agenda. These focus 
on four objectives: 

•  Prioritising investments and reforms in research and innovation; 
• 

Improving access to excellence, progressing towards excellence across the whole EU, 
and striving for stronger research and innovation systems;  

•  Translating R&I results into the economy to meet the objectives of the digital and green 
transitions, and to boost the resilience and competitiveness of European economies 
and societies; and 

•  Deepening the ERA, in order to further the free circulation of knowledge and ensure an 

upgraded, efficient and effective R&I system. 

To  this  end,  Horizon  Europe  WIDERA  (Destination  ‘Reforming  and  enhancing  the  EU 
research  and  innovation  system’  –  i.e.  the  ERA  thematic  priority)  supports  a  number  of 
different  actions  oriented 
institutions  and  other 
stakeholders,  with  the  overarching  goal  of  contributing  to  the  enhancement  of  a  single 
European R&I area.  

towards  policy-makers, 

research 

The  present  case  study  focuses  mainly  on  the  evaluation  of  the  relevance,  coherence, 
effectiveness  and  EU  added  value  of  the  ERA-related  actions  within  Horizon  Europe 
WIDERA  and  the  Horizon  2020  Coordination  and  Support  Actions  (CSAs).  As  previously 
agreed  with  the  members  of  the  Steering  Committee,  to  avoid  overlaps  with  other  case 
studies implemented within the framework of the present study, Case Study 8 will focus on 
two topic areas: (1) actions in the area of supporting citizen science; and (2) actions in the 
area of  university  transformation  (in  particular, acceleration  services).  This  case study  will 
help to identify  and understand the  mechanisms of influence and expected  impacts in  the 
above  areas,  including  common  standards,  methodologies  and  their  expected  structuring 
impacts across the ERA from a longer-term perspective.  

1.2.  Background information 

1.2.1.  Policy context 

The  main  areas  of  support  measures  funded  under  the  ERA  pillar  of  the  Horizon  Europe 
WIDERA programme include: 

•  Prioritising investment and reform (including support for programme-level cooperation 

• 

between national R&I programmes); 
Improving access to excellence (support for Europe-wide training and networking for 
research managers); 

•  Translating R&I results into the economy (implementation of a new macroeconomic 
modelling concept; support for standardisation in the exploitation of FP-funded research 
results; support for R&I-intensive IP management policies); 

•  Deepening  the  European  Research Area  (support  for modelling  and  quantifying  the 
impacts of Open Science practice; FAIR data policy and practice; capacity building for 

401 

 
institutional open access publishing across Europe; societal trust in science; support for 
changes in the assessment of research and researchers to reward the practice of Open 
Science;  protection  of  higher  education  institutions  (HEIs)  and  research  organisations 
against conventional and non-conventional threats; and other support measures in the 
area of gender equality, research ethics, citizen science and related areas). 

To achieve the programme’s objectives, the above activities focus on building common EU-
wide standards, policies and frameworks, enhancing cooperation and networking between 
national R&I policy-makers in order to set common policy goals, as well as measures aimed 
towards  the  professionalisation  of  research  management,  increasing  awareness  among 
research  management  staff  with  regard  to  existing  training,  networking  and  mobility 
opportunities at EU, national and regional levels. In addition, WIDERA supports other actions 
that are not the subject of calls for proposal grants to identified beneficiaries. These include, 
for  example,  initiatives  promoting  citizen  science/the  promotion  of  science,  such  as  the 
European Science in the City, the European Union Contest for Young Scientists (EUCYS) 
and others. 

The above WIDERA actions in the field of institutional transformation and citizen science are 
closely linked to some of the actions of SwafS (the Science with and for Society programme) 
under the previous Framework Programme, Horizon 2020. This link is particularly strong with 
regard  to  those  actions  supported  under  SwafS  ‘Strategic  orientation  1.  Accelerating  and 
catalysing  processes  of  institutional  change  and  Strategic  orientation  4.  Exploring  and 
supporting  citizen  science’654.  In  addition,  there  is  direct  continuity  between  SwafS  and 
WIDERA in relation to certain horizontal actions that were previously supported under SwafS 
and  are  now  funded  under  the  WIDERA  programme.  These  include  the  yearly  European 
Union Contest for Young Scientists (EUCYS) and the EuroScience Open Forum (ESOF). 

1.2.2.  Overall approach and scope of the case study 

Coordination  and  Support  Actions  (CSAs)  under  previous  FPs  and  under  the  ERA  part of 
WIDERA programme support a wide  range of actions across different thematic areas that 
relate to reforming and enhancing the European Research Area. To avoid overlap with Case 
Study 7, ‘Impact of the widening actions on inducing structural change at national level’, as 
well as possible overlaps with parallel case studies focusing on the impact of FPs on cross-
cutting  issues  (gender  equality,  fostering  the  diffusion  of  knowledge  and  open  access, 
research integrity and ethics, and citizen engagement), the present case study focuses on 
two specific intervention areas under the ERA part of WIDERA and Horizon 2020; namely: 

•  Actions in the area of supporting citizen science, including the European Science in the 

City initiative/EUCYS; and 

•  Actions in the area of support for university cooperation and transformation, including 

support for acceleration services and improving access to excellence and the 
professionalisation of research management. 

In contrast to Case Study 7, the present case study does not focus on particular countries or 
regions,  but  rather  on  European-level  frameworks,  networks  and  outputs,  primarily  in  the 
above areas of intervention. At the time of evaluation, Horizon Europe WIDERA projects in 
the above areas were not yet mature enough to assess their results and impacts – therefore, 

654 Horizon 2020 Work Programme 2018-2020: 16. Science with and for Society. (European 
Commission Decision C(2020)6320 of 17 September 2020), 
https://ec.europa.eu/research/participants/data/ref/h2020/wp/2018-2020/main/h2020-wp1820-
swfs_en.pdf 

402 

 
 
 
in  order  to  capture  longer-term  impacts  that  are  relevant  to  enhancing  the  ERA,  the  case 
study also includes Horizon 2020 projects in the above areas in its analysis. 

The key methodological steps applied in the implementation of this case study are as follows: 

- 

- 

- 

RV1 

CH1 

First, on the basis of the results of exploratory interviews and consultation with the 
members  of  the  Steering  Committee,  a  sample  of  ERA-related  projects  under 
Horizon Europe and Horizon 2020 were selected for the in-depth analysis.  

The study  team  conducted desk  research on  Horizon  Europe/Horizon  2020  ERA-
related  programme  documents  as  well  as  documents  relating  to  the  projects 
selected (including project administrative data from the CORDA database, project 
reports, project website data, and other data available online). These data provided 
contextual evidence regarding the structure of the selected ERA-related actions, the 
planned  and  implemented  activities  of  the  selected  projects,  mechanisms  of 
influence, and the outputs and expected short-term and longer-term impacts of the 
selected projects.  

Lastly, as the main source of evidence, the study team conducted semi-structured 
interviews  with  10  representatives  of  the  selected  projects.  These  interviews 
provided in-depth information concerning the structures of the projects supported, 
their policy context, relevance to the needs of the ERA, project activities, (expected) 
results/impacts,  as  well  as  success  factors  and  barriers  encountered  by  project 
beneficiaries during project implementation. 

Evaluation question 

Judgement criteria 

The extent to which ERA 
actions are relevant to the 
achievement of a single 
European research area and 
to meeting its needs. 

The extent to which ERA 
actions have contributed to 
improved coherence between 
Horizon Europe and 
national/regional R&I funding 
schemes.  

How relevant have the Framework 
Programme activities, including partnerships, 
been in delivering excellent science, given 
the stakeholders’ needs and taking into 
account the scientific, technological and/or 
socio-economic problems and issues 
identified at the time of its design and over 
time? 

How coherent has the Framework 
Programme been in delivering excellent 
science, in particular: (i) between the 
Framework Programme parts covered by 
this study; (ii) with other parts of the 
Framework Programme not covered by this 
study; (iii) with other EU programmes serving 
similar objectives – e.g. Structural Funds, the 
European Fund for Strategic Investments 
(Smart Specialisation Strategies), Erasmus+ 
(notably European Universities; and (iv) with 
relevant national, regional or international 
initiatives. 

EFC2 

What internal or external factors have 
influenced progress or the lack of progress 
of Framework Programme interventions in 

The success factors of ERA 
actions in contributing to the 

403 

 
 
Evaluation question 

Judgement criteria 

this area towards their impact? What could 
be done to address these factors in the short 
and longer term? Are there any factors that 
are more or less effective than others – and 
if so, what lessons can be drawn from them? 

EFC3 

To what extent have dissemination, 
exploitation and communication measures 
enabled the achievement of the targeted 
outcomes and impacts? What further actions 
are needed to maximise the impact of 
Framework Programme interventions in this 
area? 

EFC9 

What has been the contribution of the core 
programme parts covered in this 
procurement, particularly in relation to 
consolidating the European Research Area? 
Has the programme had a structuring effect 
on national research systems in terms of 
changes to national and institutional policies 
and practices? 

404 

university transformation and 
citizen science initiatives. 

Obstacles hindering the 
impacts of ERA actions in the 
above areas. 

The extent to which ERA 
actions have contributed to 
the development of the 
European R&I Foresight 
community, its networks, 
resources and functions, as 
well as its engagement with 
citizens. 

The contribution to building 
and mobilising joint 
resources to inform R&I 
policies at EU and national 
levels (e.g. the collection of 
manuals and guidebooks; a 
library of futures; expert 
pools; digital toolboxes or 
shared software solutions to 
support horizon scanning; 
citizen engagement projects; 
scenario building; and 
science and technology road 
mapping). 

The extent to which the ERA 
has contributed to the 
increased capacity and 
compatibility of cooperation 
and funding systems 
throughout the ERA for 
research management, as 
well as support for 
scientists/improved 
awareness of EU policy 
drivers and the specificities of 
the EU research among HEIs 
and research organisations. 

The contribution the ERA has 
made to the increased 
participation of researchers in 
standardisation activities, the 
increased development of 
new standards, or the 
revision of existing 
standards. 

 
 
 
 
 
 
 
 
Evaluation question 

Judgement criteria 

EFC10 

The long-term impact on researchers' 
careers; the institutional and systemic impact 
on doctoral and postdoctoral training 
programmes, and on working/employment 
conditions. 

Impacts/expected impacts of 
ERA actions in terms of 
universities’ transformation 
and the professionalisation of 
research managers. 

EFC14  What is the impact of Framework 

Programme actions supporting Open 
Science, citizen science, gender equality, 
international cooperation, ethics and 
integrity, and scientific input into other EU 
policies? 

EFC18 

To what extent have actions implemented 
under the Framework Programme to 
strengthen the European Research Area at 
national and institutional level (in Open 
Science, Responsible Research and 
Innovation, citizen science, gender equality, 
international cooperation, ethics and 
integrity) contributed to the attainment of key 
EU values and principles (as laid down by 
the EU Charter of Fundamental Rights and 
the EU treaty)? 

EAV1:  What is the EU added value of the 

Framework Programme interventions, 
including of the various instruments (such as 
partnerships) mobilised to deliver excellent 
science? What would have happened if the 
Framework Programme had not existed? 
Could the stakeholders have implemented 
their activities in another way, including 
through other national or regional support? 

The contribution made by 
ERA actions to the 
development of common 
research ethics and integrity 
framework/standards. 

The contribution made by ERA 
actions to the development of 
recommendations for tackling 
societal mistrust in science, 
research and innovation; 
strengthening the co-creation of 
R&I content by society; and 
spreading good practices and 
evidence of their effects. 

The extent to which ERA 
actions have contributed to 
the attainment of key EU 
values and principles, as laid 
down by the EU Charter of 
Fundamental Rights and EU 
treaty 

The extent to which ERA 
actions could have been 
implemented without the EU 
support; key areas of EU 
added value resulting from 
EU support for ERA actions. 

Table 1. Evaluation questions to be covered in the analysis in this case study 

1.2.3.  Key information about the action 

According to the Horizon Dashboard655, 56 Horizon Europe WIDERA (ERA) grants have been 
funded  since  2021,  amounting  to  a  total  net  EU  contribution  of  EUR  128.1  million.  The 

655 Data as of 28 August 2023. 

405 

 
 
 
 
average  net  EU  contribution  per  WIDERA  (ERA)  action  was  EUR  2.29  million,  while  the 
average number of participations per project was 12.59. 

WIDERA  (ERA)  has  had  705  participations,  with  the  majority  of  these  (241,  or  34%  of 
participations) being by higher or secondary education establishments, followed by research 
organisations (151, or 21%); public bodies (98, or 14%); private for-profit entities (84, or 12%); 
and other organisations (131, or 19%).  

In terms of geographical distribution, the largest numbers of participations in WIDERA (ERA) 
projects came from Belgium (61), Italy (51), Germany (51), the Netherlands (47), Spain (45) 
and France (38). Among non-EU countries, the largest numbers of participations came from 
the UK (37), Norway (19), Turkey (12) and Serbia (12) (for more details, see figure below). 
In terms of the net EU contribution by country, the Netherlands and Belgium are in joint first 
place, each with around 12% of net EU contributions under the WIDERA (ERA) programme 
part. They are followed by Italy and Germany (11% each), and Spain (9.5%). Among non-EU 
countries,  Norway  received  the  highest  share  of  net  EU  contribution  under  the  WIDERA 
(ERA) programme part (2.6% of the total net EU contribution). 

61

51 51

47 45

38 35

25 27

70

60

50

40

30

20

10

0

37

20 20 20 18 16 13 12 12 12 10 9 8 11

6 5 4 4 3

i

m
u
g
e
B

l

y
l
a
t
I

y
n
a
m
r
e
G

s
d
n
a
l
r
e
h
t
e
N

i

n
a
p
S

e
c
n
a
r
F

e
c
e
e
r
G

l

a
g
u
t
r
o
P

d
n
a
e
r
I

l

a
i
t
a
o
r
C

a
i
r
t
s
u
A

s
u
r
p
y
C

k
r
a
m
n
e
D

i

a
n
a
m
o
R

d
n
a
o
P

l

i

a
h
c
e
z
C

d
n
a
n
i
F

l

y
r
a
g
n
u
H

i

a
n
e
v
o
l
S

i

a
n
o
t
s
E

n
e
d
e
w
S

a
i
r
a
g
u
B

l

i

a
n
a
u
h
t
i
L

a
i
v
t
a
L

a
t
l
a
M

a
i
k
a
v
o
l
S

g
r
u
o
b
m
e
x
u
L

i

m
o
d
g
n
K
d
e
t
i
n
U

19

12 12

y
a
w
r
o
N

e
y
i
k
r
ü
T

i

a
b
r
e
S

Figure 1. Number of participations in WIDERA (ERA) projects by country (as of 28 August 2023). Source: Horizon 
dashboard. 

In terms of action types, a total of 606 participations (86%) involved coordination and support 
actions  (CSA),  while  the  remaining  14% of  participations  were  in  research  and  innovation 
actions (RIA). A keyword analysis of Horizon Dashboard data shows that the most prevalent 
topics covered by WIDERA (ERA) projects were Responsible Research and Innovation (RRI) 
(18  participations);  Science,  technology  and  mathematics  (STEM)  (14  participations);  EU 
research  policy  /Research  policies  in  the  EU  (13  participations);  Public  engagement  (13 
(11 
participations);  Life-long 
participations). 

(11  participations);  and  Science  education 

learning 

406 

 
 
 
1.3.  Key case study findings 

1.3.1.  Relevance 

1.3.1.1. 

Actions in the field of university transformation 

FP actions in the area of support for university cooperation and transformation and citizen 
science were relevant, and aligned closely with at least several strategic EU R&I priorities, 
particularly  in  terms  of  contributing  to  strategic  priorities  of  the  European  Research  Area 
(ERA)  and  single  European  Higher  Education  Area  (EHEA).  More  specifically,  WIDERA 
actions  to  support  university  transformation/acceleration services  address  nearly  all of  the 
strategic  objectives  of  the  Higher  Education  Transformation  Agenda  in  relation  to  R&I, 
including: 

- 
- 

- 

- 

closer and deeper cooperation between HEIs; 
pooling together the online and physical resources, courses, expertise, data and 
infrastructure of HEIs to achieve critical mass656; 
boosting institutional transformation and the harmonisation of practices between 
institutions, such as the automatic recognition of qualifications and study periods 
abroad; 
a stronger focus on specialised education programmes in areas relevant to 
addressing societal challenges including digital skills, entrepreneurship, the green 
transition and artificial intelligence (AI)657. 

Actions to support university transformation/acceleration services are also highly relevant to 
addressing some key ERA priorities. These include: 

•  Empowering the development of HEIs in line with the ERA, and in synergy with the 

European Higher Education Area (objective 13 of the European Research Area Policy 
Agenda). 

•  Similarly, actions to transform universities are also highly relevant to addressing a 
number of objectives under the ERA Policy Agenda strategic priority ‘Deepening a 
Truly Functioning Internal Market for Knowledge’. These include reforming the 
assessment systems for research, researchers and institutions; promoting attractive 
and sustainable research careers and the mobility of researchers; promoting gender 
equality, among others.658.  

Evidence from both the survey and interviews confirms that Horizon Europe WIDERA (ERA) 
projects  in  the  field  of  university  transformation/acceleration  services  and  citizen 
science  have  been  highly  relevant,  primarily  in  relation  to  the  network-building, 
internationalisation,  capacity-building  and  institutional  change-related  needs  of 
beneficiary organisations. 

656 Horizon Europe Work Programme 2021-2022: 11. Widening participation and strengthening 
the European Research Area. European Commission Decision C(2022)2975 of 10 May 2022), p. 
125. 
657 Communication from the Commission to the European Parliament, the Council, the European 
Economic and Social Committee and the Committee of the Regions on achieving the European 
Education Area by 2025. 
658 European Research Area Policy Agenda – Research and Innovation Overview of actions for 
the period 2022-2024. European Commission, 2021. 

407 

 
 
According  to  the  survey  results,  the  majority  of  Horizon  Europe  WIDERA  (ERA)  project 
beneficiaries  in  the  field  of  university  transformation/acceleration  services  and  citizen 
science659 indicated that their project primarily responded to the need of their institution ‘to 
enhance the institutional visibility (across the borders of their country’ (with 79% indicating 
that their project responded to this need to a very large or to a large extent); ‘to create or 
strengthen  collaboration  with  leading  research  organisations’  (73%);  ‘to  pursue  their 
individual  or  institutional  research  agenda’  (73%);  ‘to  strengthen  the  knowledge  base  and 
R&D capacity in their  organisation’ (64%); ‘to enhance R&D and/or innovation capabilities 
and  skills  in  their  organisation’  (64%);  ‘to  support  policy-making  or  the  development  of 
standards setting measures’ (61%); and ‘to promote institutional change in their organisation, 
region or country’ (61%). For more details, see the figure below. 

To enhance  the international visibility (across the
borders of your country)
To create or strengthen collaboration with leading
research organisations

To strengthen the application of Responsible
Research and Innovation principles in your…
To promote institutional change in your
organisation, region or country

To enhance your competitiveness

To strengthen the knowledge base and R&D
capacity in your organisation
To improve or develop new tools, methods or
techniques

To enhance the R&D and/or innovation capabilities
and skills in your organisation
To pursue your individual or institutional research
agenda
To support policy-making or the development of
standards setting measures
To improve or develop new products, services or
processes

To reduce the environmental impacts of your
products, processes or services
To strengthen existing markets, create/ receive
access to new markets and global value chains
To create or strengthen collaboration with leading
industry organisations
To develop sustainable solutions contributing to a
green transition

42%

36%

33%

33%

33%

27%

24%

24%

24%

36%

15%

6%

36%

18%

3%

30%

21%

6%

3%

27%

21%

6%

9%

18%

30%

9%

3%

39%

15%

12%

27%

27%

15%

39%

30%

3%

49%

18%

6%

21%

39%

24%

12%

18%

24%

21%

21%

9%

12%

30%

12%

9%

30%

18%

21%

15%

18%

12%

12%

27%

24%

15%

9%

9%

30%

15%

15%

To a very large extent

To a large extent

To a moderate extent

To a small extent

Not at all

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Figure 2. To what extent does your Horizon Europe project respond to the following needs of your organisation? (WIDERA 
(ERA) projects focusing on university transformation/institutional changes and citizen science), n=33. Source: survey of 
Horizon Europe beneficiaries, conducted May-July 2023. 

659 Projects involving the keywords ‘institutional change’, ‘transformation’, ‘acceleration services’, 
‘capacity development’ and ‘citizen science’. 

408 

 
 
 
Similarly,  evidence  from  interviews  confirms  that  the  projects  in  the  field  of  university 
transformation/acceleration services were highly relevant to addressing the most pertinent 
societal challenges, socio-economic developments and related needs of institutions in this 
dynamic context: 

• 

In  interviews,  stakeholders  confirmed  that  targeted  acceleration  services  supported 
under  WIDERA  (ERA)  projects  make  universities  better-equipped  in  terms  of  the 
skills and capacity they need to become more entrepreneurial and innovative. This 
will  help  students,  administrative  staff  and  academic  staff  to  better  respond  to  ever-
growing  competitiveness  and  the  need  for innovation  in  the  socioeconomic context of 
Europe today. By helping universities to build the competencies needed by researchers 
and facilitating the commercialisation of research findings, acceleration services make 
European HEIs better aligned with market needs. Similarly, acceleration services make 
universities more capable of forging alliances with the private sector, including small and 
medium-sized  enterprises,  which  is  also  crucial  for  aligning  university  research  with 
market needs. 

•  Similarly, a number of interviewees indicated that acceleration services, particularly in 
the  areas  of  Open  science,  Public  engagement  and  outreach,  and  the  sharing  of 
infrastructure and capacities, is highly relevant in terms of making institutions more 
open  to  society  and  public  engagement,  which  brings  research  closer  to  societal 
needs and pertinent societal challenges. 

•  According  to  interviewees,  acceleration  services  include  training  and  counselling 
services  for  beneficiary  institutions,  which  equip  them  with  the  capacities  needed  to 
implement  institutional  changes  (e.g.  in  the  area  of  gender  equality  for  researchers, 
career  counselling  and  development,  etc.).  This  capacity-building  effect  is  highly 
relevant,  and  addresses  the  EU  strategic  priority  of  making  research  careers  more 
attractive and attracting more young talents to European research institutions. 
•  Support  for  university  transformation/acceleration  services  is  also  highly  relevant  to 
delivering  excellence  in  science,  primarily  by  improving  the  working  conditions  of 
researchers and helping to better manage the human research capacities. For example, 
projects  in  the  area  of  creating  framework  conditions  for  research  management  help 
institutions and managers to improve the processes involved in research management 
in  such  a  way  as  to  reduce  the  administrative  burden/burden  related  to  non-research 
activities  for  researchers.  This  gives  researchers  more  time  to  conduct  research 
activities,  which  translates  into  greater  excellence  in  research.  This  need  for 
simplification  and  the  reduction  of  non-research  related  burden  on  researchers  is 
particularly relevant in the context of the growing complexity of today's research systems, 
e.g. the need to compete for EU and national-level funding grants that require significant 
administrative effort.  

•  Support for university transformation, particularly in the area of improving the framework 
for the position of research managers, is also highly relevant to the needs of institutions 
to  save  resources  and  make  more  efficient  use  of  their  resources.  According  to 
interviewees, acceleration services/support for university transformation often involve the 
training of staff, the streamlining and better definition of different processes and positions 
at  universities  (e.g.  the  definition  of  research  managers’  positions),  which  serves  the 
need of institutions to use their human resources in the most efficient way. 
Lastly,  interviewees  indicated  that  FP  support  for  projects  in  the  area  of  university 
transformation/acceleration services is highly relevant to the need of institutions to 
build connections with  peers in other countries,  as  well as to build networks  of 
expertise and, most importantly, to be able to learn from good practice examples 
of  institutions  in  other  countries.  Projects  that  focus  on  university  transformations 
usually involve workshops, seminars and other networking activities between universities 
from different countries and regions. These create a platform for building connections, 

• 

409 

 
gathering  knowledge  and  good  practice  examples  on  improving  processes  and 
institutional transformation. 

1.3.1.2. 

Actions in the field of citizen science 

The analysis carried out for this case study shows that the FP actions in the area of promoting 
citizen  science  are  highly  relevant  in  the  context  of  a  number  of  strategic  priorities  of  the 
single European Research Area, including: 

-  Bringing science closer to society (objective 14 of the European Research Area 
Policy Agenda). This includes promoting stronger public understanding, connection 
and engagement to science, which also brings science closer to societal challenges. 
A  number  of  interviewees  indicated  that  EU  projects  such  as  European  Cities  of 
Science were of particular relevance in the context of the COVID-19 pandemic, 
when society’s trust in science and scientists was of paramount importance. 
Activities that brought science closer to the general public were particularly relevant 
in this context, and helped to nurture and maintain public trust in science. 

-  Similarly, some FP projects supporting citizen science have also involved activities 
that  help  the  general  public  to  distinguish  between  scientific  evidence  and 
misinformation.  According  to  the  interviewees,  these  kinds  of  activities  have  a 
growing significance and relevance in contemporary context, when society is facing 
a growing need to tackle misinformation, disinformation and fake news. 

-  Actions in the area of citizen science are also highly relevant to addressing the 
strategic priority of promoting attractive and sustainable research careers and 
balanced talent circulation, particularly in terms of the attractiveness of research 
careers and encouraging the full recognition of the research profession – one of the 
key priority areas of ERA Policy Agenda660. Interviewees indicated that FP-funded 
projects such as European Cities of Science or the EU Contest for Young Scientists 
play a pivotal role in identifying young talents, motivating them, helping them to grow 
and encouraging them to choose scientific careers.  
Lastly, interviews carried out for this case study confirmed that FP projects in the 
area  of  citizen  science  are  also  relevant  to  promoting  gender  equality  and 
inclusiveness in European research systems and societies, thus contributing to 
the strategic priorities set out in the Ljubljana Declaration661. For example, one citizen 
science-focused  project  covered  in  the  case  study  included  initiatives  targeting 
marginalised communities, including Roma and refugees, with the aim of bringing 
these groups in society closer to science. 

- 

1.3.2.  Coherence 

Case  study  evidence  indicates  that  the  FP  actions  in  the  area  of  supporting  university 
transformation and citizen science were coherent with other parts of H2020/Horizon Europe, 
particularly in terms of complementing and building on other FP actions: 

•  According  to  the  WIDERA  Work  programme  2021-2022,  projects  in  the  area  of 
acceleration services for universities should link together and build on the results of the 
initiative of the European Institute of Innovation and Technology (EIT) on ‘Innovation 

660 Ibid. 
661 Ljubljana Declaration on Gender Equality in Research and Innovation, available at: 
https://www.gov.si/assets/ministrstva/MIZS/Dokumenti/PSEU/Ljubljana-Declaration-on-Gender-
Equality-in-Research-and-Innovation-_endorsed_final.pdf 

410 

 
 
Capacity Building for Higher Education’ launched in 2021, which has a similar objective 
to unlock the full innovation potential of HEIs662. 

improving 

research  careers.  For  example, 

•  Support  for  university  transformation/acceleration  services  under  WIDERA  also 
complements  other  parts  of  the  FP,  particularly  the  MSCA,  by  contributing  to  the 
objectives  of  strengthening  Europe's  human  capital  base  and 
the 
attractiveness  of  researchers’  careers.  While  the  MSCA  contributes  to  the  above 
strategic goals by supporting mobility and training of researchers, ERA actions focus on 
capacity building and institutional change that contribute indirectly to the attractiveness 
institutional 
of 
transformations  of  Higher  Education  Institutions  through  the  adoption  of  acceleration 
services’ aims to boost university transformation/institutional change in the specific area 
of  research  careers  and  talent  support  (qualifications  and  research  careers,  research 
assessment, digitalisation, talent circulation and mobility, life-long learning, and gender 
equality  and  inclusiveness).  Similarly,  the  WIDERA  project  ‘Creating  Framework 
Conditions for Research Management to Strengthen the European Research Area’ aims 
to  create  a  roadmap  for  the  future  of  research  management  (RM)  in  Europe.  This  is 
expected  to  contribute  to  the  professionalisation  of  the  research  management 
profession, to improve working conditions for researchers, and thereby contribute to the 
overall development of Europe’s human research capacities.  

‘Catalysation  of 

the  project 

•  Citizen science-oriented FP projects under  SwafS and the WIDERA programme are 
complementary to the MSCA Researchers’ Night initiative. Through various means, 
these  actions  together  address  the  overarching  goal  of  bringing  science  closer  to  the 
public  and  making  research  careers  more  visible  and  attractive,  especially  for  young 
people.  

•  Activities in the area of citizen science funded under WIDERA/SwafS in some cases 
resulted in synergies and spill-over effects with regard to other parts of Horizon 
2020/Horizon  Europe.  For  example,  the  MSCA  Alumni  Association  was  involved  in 
European  City  of  Science  and  the  EuroScience  Open  Forum  (ESOF  2022663),  both  in 
Leiden. During the ESOF2022 Marie Skłodowska-Curie Actions satellite event in Leiden, 
more  than  120  MSCA  fellows,  supervisors,  coordinators,  alumni  and  policy-makers 
gathered together to interact, network, debate and contribute to priority topics regarding 
the  research  ecosystem664.  Similarly,  interviewees  confirmed  that  representatives  of 
European Space Agency and European Research Council were present and contributed 
to the European City of Science by showcasing their work to wider audiences. 
In some cases, WIDERA projects in the area of university transformation also had 
complementarities with the Research Infrastructures programme by contributing 
to  the  same  higher-level  objectives.  According  to  the  stakeholders  interviewed,  the 
WIDERA  project 
for  Research 
Management’  –  RM  Roadmap  complements  the  RI  Train665  and  RI  Trains  Plus666 
projects,  which  aim  to  improve  and  professionalise  the  training  of  managerial  and 
leadership staff in research infrastructures. All three projects contribute to the objective 
of  the  professionalisation  of  research  managers  and  the  development  of  research 
management position, albeit through different means and with slightly different focuses 
in their activities. Whereas the WIDERA project aims to develop a roadmap for the future 
of  research  management  (RM) in  Europe  and  a  community  to  support  its  delivery,  RI 

‘Creating  Framework  Conditions 

focusing  on 

• 

662 Horizon Europe Work Programme 2021-2022: 11. Widening participation and strengthening 
the European Research Area. European Commission Decision C(2022)2975 of 10 May 2022. 
663 Frigge, W., Evaluation Report, EuroScience Open Forum (ESOF2022), 2022, available at: 
https://leiden2022.nl/sites/default/files/2023-
06/D3.2%20ESOF2022%20Programme%20and%20evaluation%20report.pdf 
664 https://www.esof.eu/marie-sklodowska-curie-actions-msca/ 
665 http://ritrain.eu/ 
666 https://ritrainplus.eu 

411 

 
 
Train and RI Train plus focus on training programmes and competency requirements for 
the current and future managers of European research infrastructures. 

Case  study  evidence  also  reveals  that  FP  interventions  in  the  area  of  university 
transformation and citizen science are also coherent with non-FP initiatives and schemes at 
both EU and national levels: 

•  The programme of interviews carried out for this case study found that some projects 
focusing  on  university  transformation  had  significant  synergies  with  Erasmus+, 
particularly  the  European  University  Alliances  initiative.  The  WIDERA  project  RM 
Roadmap (‘Creating Framework Conditions for Research Management’) involved Una 
Europa – one of the European University alliances – among its consortium associated 
partners. According to the stakeholders interviewed, this association was natural and is 
likely to produce tangible synergies. Each alliance is led by research managers, and the 
results of the RM Roadmap project would benefit the development of their capacities; 
meanwhile, the alliances provided the RM project with necessary expertise in the field of 
managing research infrastructures. The RM Roadmap project aims to identify ways in 
which  research  management  positions  could  become  compatible  between  different 
countries  –  a  challenge  that  is  in  practice  faced  and  addressed  by  different  research 
offices in the European University alliances. Synergies between WIDERA (ERA) projects 
focusing  on  the  professionalisation  and  development  of  research  management  and 
European  University  alliances  is  further  confirmed  by  the  recent  progress  report  on 
University  Alliance  projects supported  under  Horizon  2020667.  This  report emphasised 
the crucial role played by professional research managers in the success of alliances. 
According to the report, it is crucial to involve a greater number of professional research 
managers  and  leaders  employed  at  alliance  institutions  from  the  outset  of  project 
implementation. This would ensure that there are leaders who have a full overview of an 
institution's  research  capacities,  and  who  have  the  authority  to  make  decisions  over 
resource allocation. 
In some cases, projects were also highly complementary to equivalent national-level 
initiatives in Member States: for example, the EU Contest for Young Scientists builds 
upon events at national level. Participants in the EU-level contest for young scientists are 
projects nominated by the national organiser in each participating country (typically, they 
are the winners of contests organised at national level). 

• 

1.3.3.  Effectiveness 

1.3.3.1. 

University transformation/acceleration services 

Since  a  large  portion  of  those  projects  focusing  on  university  transformation/acceleration 
services covered in the present case study were still in their inception phase at the time of 
writing,  the  data  available  on  Horizon  Europe  mostly  allows  the  identification  of  recently 
started  and  planned  activities,  objectives  and  estimated  impacts.  However,  evidence 
stemming  from  desk  research,  the  survey  and  case  study  interviews  already  provides 
preliminary  insights  into  the  estimated  structuring  impacts  of  these  projects  on  participant 
institutions and on the ERA as a whole. 

The survey of Horizon Europe beneficiaries carried out for the present case study shows that 
university 
the 

organisations 

beneficiary 

focusing 

involved 

projects 

on 

in 

667 O’Neill, G., & Acheson, H., Progress of University Alliance Projects: Projects funded under 
Horizon 2020, Luxembourg: Publications Office of the European Union, 2023, available at: 
https://op.europa.eu/en/publication-detail/-/publication/ef36cf58-e57e-11ed-a05c-
01aa75ed71a1/language-en/format-PDF/source-285015820 

412 

 
 
transformation/institutional  change668  achieved,  or  are  likely  to  achieve,  ‘improved  skills, 
knowledge  and  competences  of  researchers’  (with  20  out  of  23  or  87%  of  organisations 
indicating this would result ‘to a very large extent’ or ‘to a large extent’), followed by ‘improving 
the  career  prospects  of  researchers’  (with  18  out  of  23  or  78%  of  surveyed  organisations 
indicating it would result ‘to a very large extent’ or ‘to a large extent’). Similarly, 16 out of 23 
or 70% of stakeholders surveyed agreed ‘to a very large extent’ or ‘to a large extent’ that their 
project has or is likely to ‘strengthen relationships with leading partners in Europe, followed 
by  ‘Increasing international visibility  through  collaboration  with  leading  global  partners’  (15 
out of 23 respondents, or around 66%). Lastly, 14 out of 23 or 61% of respondents in the 
group  indicated  that  their  project  would  result  in  the  ‘development  of  policy-making  and 
standards setting measures’ (for more details, see Figure 3 below).  

Evidence stemming from in-depth interviews and desk research carried out for the present 
case study confirms this evidence on the (expected) outcomes of the ERA projects focusing 
on university transformation/acceleration services. 

Desk research and interviews with stakeholders show that one of the most immediate/short-
term effects of those WIDERA projects focusing on institutional transformation/acceleration 
services is the mapping and identification of the state of play with regard to institutions 
and the key issues they face, which will pave the way for the development of roadmaps 
for transformation. To this end, the inception phase of these projects often includes various 
consultation activities during which systemic evidence on existing gaps, needs and expected 
areas of institutional transformation is gathered. For example, as one of its first deliverables, 
the  WIDERA  project  Accelerate  Future  HEI669  is  conducting  a  survey  of  each  partner  HEI 
about  its  current  situation,  strategic  vision  and  expected  future  changes  in  the  area  of 
entrepreneurship  and  innovation.  The  results  of  this  survey  will  feed  into  strategic  vision 
statements  towards  becoming  Entrepreneurial  and  Innovative  Universities,  which  each 
partner will develop during the project’s implementation. In these statements, institutions will 
narrow  down  and  identify  key  areas  of  institutional  transformation  in  the  areas  of 
entrepreneurship and innovativeness that they will aim to undertake in the future.  

Similarly, one of the first deliverables of the project CATALISI (‘Catalysation of institutional 
transformations  of  Higher  Education  Institutions  through  the  adoption  of  acceleration 
services’) involves exploration (through workshop and other consultation processes) of the 
state of affairs and priority areas for participant organisations with regard to the three domains 
of  action  for institutional  transformation  covered in  the  project  –  namely,  research careers 
and  talent  support;  open  science,  public  engagement  and  outreach,  and  the  sharing  of 
infrastructure  and  capacities;  sustainability  in  education  and  funding,  and  sustainability  in 
research and campus operations. This process will also result in a SWOT analysis for each 
HEI to provide a comprehensive picture and concrete areas for intervention/transformational 
pathways  with  initial  ideas  for  action  on  short,  medium,  and  long-term  goals  for  each 
institution. 

668 Projects involving the keywords ‘institutional change’, ‘transformation’, ‘acceleration services’, 
or ‘capacity development’. 
669 https://acceleratefuturehei.eu/ 

413 

 
 
Improving the career prospects of researchers

Improving the skills, knowledge and competences
of researchers

39%

35%

39%

52%

Increasing international visibility through
collaboration with leading global partners

Development of policy-making and standards
setting measures

22%

17%

44%

44%

Strengthened capacity to attract public or private
funding

13%

13%

Improving access to physical and knowledge
infrastructures, facilities and data resources

13%

17%

Strengthening relationships with leading partners
in Europe

13%

57%

Increasing research productivity (e.g., the number
of peer reviewed publications)

9%

17%

Contributing to emerging areas of science and
technology

9%

13%

Pushing the frontiers of knowledge

9%

13%

Entry to new markets and/or (global) value chains

4%

4%

Development of sustainable solutions contributing
to a green transition

0%
0%

Enhancing capacity to test, demonstrate and
prototype new technological developments

0%

5%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

To a very large extent

To a large extent

To a moderate extent

To a small extent

Not at all

Figure 3: To what extent, if at all, has your Horizon Europe project achieved/is likely to achieve the following results? 
WIDERA (ERA) projects focusing on university transformation/institutional changes). n=23. Source: survey of Horizon 
Europe beneficiaries, conducted May-July 2023. 

 Interview  evidence  also  strongly  confirms  the  survey  data  indicating  that  one  of  the  key 
results  of  the  projects  focusing  on  university  transformation/acceleration  services  is  the 
development  of  networks  of  expertise  and  international  cooperation  ties  between 
participating  institutions.  A  number  of  interviewees  confirmed  that  from  the  inception 
phase, these projects result in the development of clusters and groups of experts discussing 
the  possibilities  and  pathways  for  transformation  across  different  thematic  areas  such  as 
entrepreneurship,  gender  equality  or  the  professionalisation  of  research  managers  in 
universities.  In  some  cases,  these  projects  have  already  resulted  in  tangible  EU-wide 
networks of experts and stakeholders working to gather information and develop frameworks, 
policies and standards that can be adapted at EU level. As a result, and as a part of the RM 
Roadmap project, the Ambassadors Network has been developed, which brings together 120 

414 

 
 
community  builders  and  leaders  from  countries  across  Europe  to  co-create  the  future  of 
research management in Europe. These ambassadors – selected through a careful process 
and operating via an IT-platform – serve as liaisons between the project and their national or 
regional  communities.  The  network  of  ambassadors  serves  three  purposes:  namely,  to 
engage  communities  to  participate  in  RM  Roadmap  to  design  the  future  of  research 
management  in  order  to  gather  intelligence  and  inputs  from  national  and  local  research 
management communities, to gain their perspectives for the RM Roadmap project, and to 
function as online moderators to inform the co-creation of the future of research management 
in Europe. 

impacts  of 

key  expected 

those  FP  projects 

The desk research and case study interviews also confirmed that developing the skills and 
competences of administrative, managerial and academic staff of universities is one of 
focusing  on  university 
the 
transformation/acceleration  services.  The  stakeholders  interviewed  who  were  involved  in 
WIDERA  projects  focusing  on  acceleration  services  for  universities  indicated  that  the 
acceleration services in their projects involve specialised coaching/training for staff and  the 
building of the knowledge base. These aim to equip universities with the necessary capacities 
to  implement  transformational  changes,  rather  than  directly  fostering  the  transformation  of 
universities. For example, interviewees confirmed that one of the  key expected impacts of 
the Accelerate FutureHEI project (the Entrepreneurial & Innovative Universities Acceleration 
Programme) is to equip the staff of beneficiary institutions with skills, knowledge, evidence 
and,  more  generally,  entrepreneurship  mindsets,  that  can  empower  them  to  initiate 
institutional  change  in  the  area  of  promoting  entrepreneurship  and  innovation  (e.g. 
establishing  technology  transfer  or  partnership  offices,  or  creating  a  framework  for 
engagement with industry). 

Similarly, evidence from interviews conducted for this case study confirm that FP projects in 
the  area  of  university  transformation/acceleration  services  are  likely  to  result  in  policy 
standard-setting  measures,  tools  and  frameworks  that  can  empower  beneficiary 
institutions and institutions outside the projects to undertake transformations even after the 
project end. One of the final deliverable envisaged in the Accelerate Future HEI involves key 
learnings open to the entire European HE community, as well as policy recommendations to 
support the acceleration of entrepreneurial and innovative HEIs across Europe, that can be 
freely used by institutions and policy-makers from across Europe. Similarly, some of the key 
deliverables  planned  in  RM  Roadmap  project  include  standardization  of  research 
management profession, related terminology and ultimately the roadmap to improve the RM 
system in Europe, which would be open for use to institutions across Europe. 

All of the stakeholders involved in university transformation-focused projects also confirmed 
that  the  ultimate,  long-term  impact  they  expect  from  their  projects  is  the  increase  of 
excellence  of  research  at  institutional,  national  and  EU  level,  stemming  from 
modernisation and transformation of HEIs. For example, the ultimate aim of the Research 
Management  Roadmap  project  is  to  enhance  the  efficiency  of  national  research  and 
innovation systems, freeing up valuable research time (by allowing researchers to focus first 
and foremost on research tasks), and creating a supportive environment for researchers. By 
establishing  clear  definitions,  segmentation  and  guidance,  the  project  is  expected  lay  the 
groundwork  for  long-term  improvements  in  the  field  of  research  management,  benefitting 
both researchers and the broader R&I community. 

1.3.3.2. 

Citizen science 

Case study evidence shows that FP support in the area of citizen science – including EUCYS 
(the EU Contest for Young Scientists, European Cities of Science and others  – resulted in 
significant  outcomes  at  both  local  and  European  levels.  According  to  evidence  from 

415 

 
interviews  carried  out  for  this  case  study,  the  most  immediate  effects  of  this  support  is  to 
increase the visibility and attractiveness of research careers, and to motivate young 
talented  people  to  choose  research as  a  career  path.  Preparing  scientific  projects  and 
participating in a contest with peers, while having the opportunity to interact and communicate 
with renowned scientists, encourages young persons involved in EUCYS to pursue scientific 
careers.  According  to  the  stakeholders  interviewed,  in  addition  to  raising  awareness  and 
increasing  participants’  motivation  to  pursue  research  careers,  involvement  in  the  contest 
provides young people (in particular, the winners of the contest) with additional opportunities 
and  advantages  in  relation  to  entering  university  and  pursuing  research  careers.  For 
example,  according  to  interviewed  university  representatives,  in  Spain,  some  of  the  most 
renowned  scholarships  for  young  students  require  such  competitions  and  prizes  to  be 
considered. 

Case study evidence also shows that FP support for citizen science also results in wider 
societal outreach for science, as well as improved links between science and society, 
particularly in the local area in which the supported events took place. According to one 
interviewee, representatives of local authorities in Salamanca, which hosted EUCYS 2021, 
became  involved  in  the  event.  As  a  result,  Salamanca  has  witnessed  the  emergence  of 
numerous science-related initiatives. Indeed, part of the city’s budget was allocated to citizen 
science  and  events  to  popularise  science,  making  it  one  of  the  leading  cities  in  this area. 
Similarly, Leiden 2022 resulted in a number of spill-over, citizen science initiatives organised 
by stakeholders in the local area, and which were sustained even after the project formally 
ended: 

The  P.O.  Box  71  project  where  citizens  could  ask  questions  to 
scientists, resulted in over 1,000 questions. The university embraced 
this  and  will  continue  it.  The  ‘Knowledge  to  Our  Neighbourhoods’ 
programme, with its 365 topics of the day, will also continue thanks to 
a  local  consortium  involving  the  university,  the  University  of  Applied 
Sciences, and the municipality – stakeholders of Leiden 2022 

This case study’s in-depth analysis of Horizon 2020 projects in the field of citizen science 
funded under the SwafS programme demonstrates that in some cases, FP support resulted 
in  significant  and  high-quality  research  publication  output.  For  example,  one  of  the 
projects  covered  in  this  case  study,  ALLINTERACT  (‘Widening  and  diversifying  citizen 
engagement  in  science’),  resulted  in  seven  scientific  articles  in  top-rated  indexed  journals 
(the first or second quartiles in JCR and/or Scopus-indexed journals). Each of these articles 
has received more than 1,000 views and downloads, with some receiving more than 5,000. 
The articles were actively disseminated via social media, in many cases by citizens (more 
than 130 shares on Twitter, Reddit, Facebook, Wikipedia and digital newspapers)670. 

In some cases, FP projects in field of citizen science have resulted in significant learning 
and  career  development  effects  among  participant  researchers.  According 
to 
stakeholders in Leiden 2022 (European City of Science), this project resulted in equipping 
young academics with entrepreneurial skills, which addressed the uncertainty they face in 
the  academic  job  market.  Many  participants,  who  did  not  initially  view  themselves  as 
entrepreneurs,  left  with  a  new  mindset  and  practical  skills.  In  some  cases,  these  projects 
even resulted in spin-off initiatives. Some even initiated their own spin-off companies. As a 

670 https://allinteract.eu/impact/ 

416 

 
 
result of the associated TalentOn671 activities, two team members are now in Silicon Valley, 
seeking funding for a novel method to identify cancer – an idea that resulted from one of the 
brainstorming sessions during the competition event, which involved young researchers as 
participants.  

Policy  impacts  –  particularly  policy  measures  that  are  adapted  and  used  by  broad 
spectrum of stakeholders – are also identified as one of the key results of FP projects in 
the area of citizen science. One example is the non-populist participatory platform, ‘Sappho’, 
which was developed within the framework of the Horizon 2020 project ALLINTERACT. This 
is  currently  widely  used  by  a  number  of  different  stakeholders  (e.g.  the  Society  of  Jesus 
(Jesuits))  as  a  key  resource  for  training  to  eliminate  sexual  abuse  among  children  and 
adults672. Moreover, the project’s principal investigator and two other project members were 
commissioned  by  the  European  Commission’s  Network  of  Experts  working  on  the  Social 
dimension of Education and Training (NESET) to produce the report ‘Achieving student well-
being for all: educational contexts free of violence’673. The project’s principal investigator was 
also commissioned by the Spanish government to prepare the country’s White Book on the 
Inclusive Communication of Science674. 

1.3.3.3. 

Dissemination of outputs and results 

The available evidence indicates that dissemination and communication activities were a high 
priority  for the  projects covered  by  the  scope  of  the  present  case  study,  particularly  those 
projects in the field of acceleration services/university transformation, where wider structural 
impacts  on  the  ERA  are  highly  dependent  on  effective  dissemination  and  communication 
strategies to reach the target groups outside the project consortium.  

In  addition  to  the  traditional  dissemination  measures  and  instruments  that  projects 
used/intend  to  use  (websites,  conferences,  public  speeches,  academic  publications, 
newsletters, press releases), some projects applied innovative tools to disseminate their 
project outputs and results to wider publics. For example, in the case of RM Roadmap 
project, the project consortium involved a partner – the Crowdhelix Open Innovation platform 
– to facilitate and boost its dissemination strategy. Crowdhelix675 uses a specialised intelligent 
recommendation  software  system.  This  allows  the  project  consortium  to  match  specific 
project outcomes and dissemination items with specific target groups using natural language 
processing and machine learning. The use of this  specialised software makes it easier for 
stakeholders to access the information that is the most relevant to them, and streamlines the 
dissemination  process  for  a  project.  In  addition,  the  project  also  employs  its  national 
ambassadors’ network – a network of around 120 national experts – to facilitate consultations 
with  the  local  research  community,  gather  inputs  and  create  consensus  documents  from 

671 During the EU TalentOn event organised as part of Leiden 2022 European City of Science, , 
bright young minds (aged between 21 and 35 years old) were put to the test. Participants in the 
competition were arranged into teams and challenged to think about five EU Missions. 
https://leiden2022.nl/en/highlights/eu-talenton 
672 https://allinteract.eu/impact/ 
673 Flecha, R., Puigvert, L., & Racionero-Plaza, S., Achieving student well-being for all: 
Educational contexts free of violence, NESET report, 2023, Luxembourg: Publications Office of 
the European Union, doi: 10.2766/463854 
674 Towards Inclusive Science Communication: Reflections And Successful Actions, Fundación 
Española para la Ciencia y la Tecnología, F. S. P. (FECYT), 2021, available at: 
https://www.fecyt.es/es/system/files/publications/attachments/2022/03/comunicacion_inclusiva_c
iencia_accesible.pdf 
675 https://crowdhelix.com/ 

417 

 
 
these discussions, which could be disseminated almost immediately, especially to Member 
States. 

In some cases, the projects put significant efforts into ensuring the dissemination and 
communication  of  their outputs and results to policy-makers. According to interviews 
carried  out  for  this  case  study,  the  efforts  made  to  disseminate  the  results  of  the  Horizon 
2020 project ALLINTERACT were particularly successful, due to successful involvement of 
policy-makers (the European Parliament and the European Commission, as well as national 
policy-makers) in the dissemination activities. Project results were disseminated through a 
NESET  report  and  the  ‘White  Book  on  the  Inclusive  Communication  of  Science’ 
commissioned by the Spanish Government. According to interviewees, policy-makers who 
use the knowledge created during this project often disseminate these findings to their peers.  

Some FP projects have aimed to amplify and increase the impact of their results by organising 
dedicated  dissemination  events.  In  the  case  of  the  European  City  of  Science  (Leiden 
2022), a legacy session was organised after the project end, at which the project stakeholders 
discussed  all  of  the  significant  events  and  achievements  of  the  project.  During  this  event, 
participants discussed the project’s achievements, highlighting good practice examples and 
sharing the knowledge gained during the project. This dedicated public discussion is available 
to any interested parties on YouTube. 

1.3.4.  EU added value 

According to the results of the survey of unsuccessful applicants carried out for this study, 
EU  support  for  ERA  actions  has  in  general  constituted  a  very  significant  additionality, 
compared with  the  opportunities  provided at national  level:  the  majority  of  the  ERA 
project applications that were rejected for EU funding were not implemented. Around 
81% of unsuccessful applicants for Horizon funding under the ERA thematic priority (67 out 
of 83) indicated that they had not implemented their project without Horizon Europe support 
(i.e.  they  did  not  apply  for  alternative  funding,  or  applied  for  alternative  funding  but  were 
unsuccessful). Conversely, fewer than 10% of unsuccessful applicants (8 out of 83) received 
alternative  funding  for  the  activities  planned  in  their  proposed  WIDERA  project  (i.e.  they 
applied for alternative funding and had already received it at the time the survey was carried 
out). 

Similarly, the case study’s survey of Horizon Europe beneficiaries shows that ERA project 
beneficiaries  considered  the  cross-border  cooperation  opportunities/geographical  and 
sectoral diversity of partners in their projects as the key source of EU added value, compared 
with  what  would  be  available  to  them  at  national/regional  level.  In  total,  93%  of  ERA 
beneficiaries (27 out of 29) in the field of institutional transformation/citizen science ‘strongly 
agreed’/‘rather  agreed’  that,  compared  with  the  research  funding  available  to  them  at 
national/regional  level,  Horizon  Europe  support  ‘provides  more  diversity  in  the  partners’ 
profiles (multi-sectoral, multi-disciplinary consortia, more geographical diversity of partners, 
etc.’ This is followed by ‘more international mobility opportunities for researchers’ (22 out of 
28, or 79%) (see the figure below). This was largely supported by findings stemming from the 
interviews carried out for this case study: the building of EU-level networks, which allowed 
the development of roadmaps, frameworks and methodologies reflecting the needs of 
different EU countries was identified as the key source of EU added value by interviewees, 
particularly in the field of university transformation. A number of interviewees noted that the 
fact that the projects operated at EU level provided an opportunity to involve a wide range of 
stakeholders from a variety of regions and countries, which is a key source of intelligence 
when building tools and roadmaps that can be used at EU level.  

418 

 
Provides more diversity in the partners’ profiles (multi-sectoral, 
multi-disciplinary consortia, more geographical diversity of 
partners etc.) (N=29)

76%

17%

3%

3%

Provides more international mobility opportunities for researchers
(N=28)

68%

11%

18%

4%

Involves higher level of competition for research funding  (N=29)

55%

28%

10%

3%

3%

Provides higher amount of funding (compared to the
national/regional schemes) (N=29)

48%

14%

31%

3%

3%

Provides funding for research topic(s) or field(s) not covered in
national or regional R&I funding programmes (N=29)

45%

41%

7%

3%

3%

Gives more attention for the human aspect to innovation (safety,
ethics, user acceptance etc.) (N=29)

41%

14%

35%

3%

7%

Helps to better pool the skills and expertise necessary for research
(N=29)

38%

35%

14%

7%

7%

Gives more attention for the environmental sustainability aspect to
innovation (N=29)

35%

24%

24%

10%

3%

Helps to better address grand societal challenges (e.g., climate
change, demographics and ageing, etc.) (N=29)

28%

41%

14%

3%

10%

Gives more flexibility to respond to changing socio-economic needs
(N=29)

24%

31%

31%

3%

10%

Provides more access to research infrastructures, equipment
and/or databases (N=29)

24%

28%

24%

7%

17%

Provides more opportunities for commercialization of research
results  (e.g., applicability of results to new markets not present in
national environment) (N=28)

21%

14%

32%

4%

29%

Gives more flexibility to respond to technological developments
(N=29)

21%

17%

31%

10%

21%

Helps to achieve economies of scale in research (achieving the
same result at lower cost) (N=29)

10%

14%

38%

14%

21%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Strongly agree

Rather disagree

Rather agree

Neither agree nor disagree

Strongly disagree

Do not know/Not Applicable

Figure 4: Would you agree or disagree that, compared to the research funding available to you at national and/or regional 
level, Horizon Europe ((WIDERA (ERA) projects focusing on university transformation/institutional changes and citizen 
science). Source: survey of Horizon Europe beneficiaries, conducted May-July 2023. 

419 

 
 
 
 
Evidence gathered for the present case study indicates that the EU dimension has brought 
significant added value to the projects covered by the study. One source of such added value 
was a higher level of excellence of the activities and outputs stemming from EU-wide 
competition  (as  opposed  to  merely  local  or  national-level  competition).  The  most  vivid 
example of this added value is provided by the EUCYS (EU Contest for Young Scientists) – 
according to the stakeholders interviewed, this EU-wide contest brought together the most 
talented young people who had won science contests at national level. This European context 
raised the event to a higher level, and increased the excellence of the projects presented by 
contestants.  Moreover,  another  significant  aspect  of  EU  added  value  stems  from  the 
opportunity the contest provided for contestants to interact with peers and researchers from 
various countries. A higher level of competition as a source of added value, compared with 
national or regional-level funding sources, is also confirmed by survey evidence: 83% of ERA 
project beneficiaries (24 out of 29) in the field of institutional transformation/citizen science 
‘strongly agreed’ or ‘rather agreed’ that Horizon Europe ‘involves a higher level of competition 
for research funding’ compared with national/regional funding. 

1.4.  Success factors and barriers 

1.4.1.  Success factors  

One  of  the  key  success  factors  identified  across  different  projects  covered by  the  present 
case  study,  particularly  in  the  field  of  citizen  science,  is  the  active  dissemination  of 
information, and the awareness of the wider public regarding the project’s activities at 
local  and  national  levels.  This  is  particularly  important  in  the  case  of  projects  whose 
activities  are  directly  linked  to  activities  at  national/local  level,  for  example  in  the  case  of 
EUCYS (the EU Contest for Young Scientists), in which the EU-level contest brings together 
the  winners  of  contests  organised  by  national  organizers.  According  to  evidence  from  the 
interviews carried out for the present case study, the awareness of young people of these 
contests at national level varies significantly from country to country. While some countries 
and local authorities put great efforts into raising awareness about the event and attract a 
great deal of interest from young people, in other countries the event is not well known among 
their youth, which negatively impacts the level of involvement from the country. 

In the case of projects in the field of university transformation/acceleration services, one of 
the key success factors identified is the ability of project coordinators and participants to 
ensure the widespread involvement of wider groups of stakeholders in data gathering 
activities  and  stakeholder  consultation  processes.  The  most  vivid  example  of  this  is 
provided by the project RM Roadmap, in which the successful development of a roadmap for 
the  future  of  research  management  (RM)  in  Europe  relies  heavily  on  the  success  of 
stakeholder engagement activities. According to the stakeholders interviewed, the ultimate 
success of this project will depend on the ability and capacities of its wide network of around 
120 national ambassadors to mobilise their national research communities, in order to ensure 
their active participation in co-creation and consultation activities and in gathering necessary 
intelligence and data. 

The previous experience of project coordinators and their ability to draw from previous 
examples of good practice was also found to be a key success factor. For example, one of 
the  key  success  factors  in  the  case  of  the  2022  European  Capital  of  Science,  was  the 
experience of coordinators with the European Capitals of Culture initiative and their ability to 
use  and  amplify  this  in  the  field  of  citizen  science.  More  specifically,  this  experience 
contributed to project coordinators applying a new and innovative approach to citizen science: 
instead of a top-down ‘science communication’ approach, the organisers applied a grassroots 
‘bottom-up’  approach  in  which  individuals  from  local  communities  took  the  initiative  in 
promoting science. 

420 

 
1.4.2.  Barriers 

According  to  the  stakeholders  interviewed,  engaging  policy  stakeholders  and  top-level 
management  can  pose  challenges  in  terms  of  commitment  to  institutional  change.  Some 
projects in the field of university transformation/acceleration services encompass a number 
of different areas of transformation (e.g. gender quality; Open Science; the sustainability of 
funding,  etc.)  According  to  the  stakeholders  interviewed,  ensuring  leaders’  commitment  to 
initiating changes across such a wide scope of transformation areas may be challenging for 
universities. 

Some  project  stakeholders  indicated  that  challenges  also  resulted  from  the  Horizon  2020 
Regulation in relation to the use of project funds. More specifically, one such issue identified 
is the restriction on using project funds for publishing. According to project stakeholders, 
this  administrative  issue  should  be  addressed  as  soon  as  possible,  as  it  affects  many 
organisations and can hinder the dissemination of knowledge created during a project. 

A  number  of  stakeholders  also  identified  ensuring  effective  communication  and 
coordination between different project partners and their varying needs as one of the 
key challenges, particularly within those projects focusing on university transformation. One 
of the critical aspects among this group of projects is effectively coordinating a large number 
of  stakeholders,  some  of  whom  are  volunteers  (e.g.  the  ambassadors  programme  in  RM 
Roadmap project), and ensuring that the right information reaches the right people at the right 
time.  As  already  mentioned,  a  project’s  success  will  largely  depend  on  the  successful 
mobilisation and coordination of these wide groups of stakeholders, the involvement of whom 
will be crucial for data gathering and stakeholder consultation processes. 

Administrative burden and lengthy proposal evaluation processes (which often took around 
half  a  year)  were  in  some  cases  also  identified  as  a  source  of  difficulties,  particularly  for 
project coordinators. Some organisations also indicated that they had experienced difficulties 
relating to the new Certificate of Financial Statement (CFS): in certain cases, this new form 
was not recognised by Accountancy Europe and by the Dutch Association of Accountants. 
This presented significant challenges for beneficiary organisations, as they could not upload 
final reports without the CFS. 

COVID-19 and related restrictions were also identified as a source of challenges for some 
Horizon 2020 projects covered by the present case study: the need to switch most activities 
to  hybrid  mode  presented  additional  administrative  and  technical  challenges  for  the 
organisers of citizen science events, for example.  

1.5.  Conclusions and lessons learned 

The available evidence confirms that the FP projects covered by the  scope of the present 
case study were highly relevant in addressing scientific, technological and socio-economic 
issues, as well as to the needs of beneficiary institutions. The actions supported under these 
projects  responded  to  the  strategic  priorities  of  the  ERA  to  ensure  closer  and  deeper 
cooperation  between  HEIs,  boosting  institutional  transformation  and  the  harmonisation  of 
practices  across  institutions,  making  research  careers  more attractive  and  attracting  more 
young  talents  to  European  research  institutions.  Similarly,  projects  engaging  in  citizen 
science proved to be highly relevant to strengthening the ties between science and society 
and addressing misinformation, disinformation and fake news. Similarly, projects were highly 
relevant  to  the  needs  of institutions  to  develop  ties  of cooperation  with  partner  institutions 
abroad, as well as to build knowledge and human research capacities, and to transform their 
processes and internal practices in order to better respond to current socio-economic needs 

421 

 
and trends (e.g. the growing need for entrepreneurship and innovation skills among students, 
researchers and even administrative staff). 

Projects in the field of university transformation and citizen science were also coherent with 
other parts of Framework Programmes, mainly by contributing to the same higher-level goals 
through the use of different instruments. The available evidence indicates complementarities 
(and in some case synergies) between these projects and the MSCA (by contributing to the 
overarching goals of developing human research capacities of institutions). Similarly, projects 
in the field of university transformation/acceleration services complemented the initiative of 
the European Institute of Innovation and Technology (EIT) on Innovation Capacity Building, 
as well as the Research Infrastructures programme, by contributing to professionalising the 
training  of  managerial  and 
the  managers  of  research 
leadership  staff  (including 
infrastructures). 

The available evidence indicates that the most immediate effects of FP support for university 
transformation/acceleration  services  were  the  gathering  of  systemic  information,  the 
identification of the state of play in institutions and the key issues they face in different areas 
(e.g. the development of researchers’ careers, gender equality, entrepreneurship, research 
management,  and  so  on).  These  effects  paved  the  way  for  the  further  development  of 
transformation roadmaps. Another immediate effect of such projects is their contribution to 
the  development  of  networks  of  expertise  and  ties  of  international  cooperation  between 
participating  institutions,  thereby  strengthening  the  single  ERA.  Projects  in  the  field  of 
university  transformation  have  also  resulted  (or  are  expected  to  result,  in  the  case  of  the 
projects  that  are  still  in  their  inception  phase)  in  significant  learning  effects  –  i.e.  the 
development of the skills and competences of administrative, managerial and academic staff 
of universities. First and foremost, this includes specialised coaching/training for staff and the 
building of a knowledge base that would equip universities with the necessary capacities to 
implement 
the  area  of  university 
transformation/acceleration  services  are  also  likely  to  result  in  policy  standards-setting 
measures, tools and frameworks that would empower beneficiary institutions and institutions 
outside the projects to undertake transformation and harmonisation in their own practices. 
Examples of such practices include policy recommendations to support the acceleration of 
entrepreneurial and innovative HEIs across Europe, and the roadmap to improve research 
management systems in Europe. From a long-term perspective, all of these outcomes related 
to  the  transformation  and  modernisation  of  HEI  policies  and  practices  contributing  to 
excellence in European research systems. 

transformational  changes.  FP  projects 

in 

Projects  in  the  field  of  citizen  science  have  contributed  to  increasing  the  visibility  and 
attractiveness  of  research  careers  and  motivating  young  talented  people  to  choose  a 
research career path. FP support for citizen science also results in wider societal outreach 
for science and improved links between science and society, particularly in the local areas in 
which  supported  events  have  taken  place.  Other  outcomes  of  projects  relating  to  citizen 
science  have  included  significant  publications  outputs,  as  well  as  learning  and  career 
development effects among participant researchers. In some cases, projects have resulted 
in significant policy impacts, particularly policy measures that have been adapted and used 
by broad spectrum of stakeholders, including policy-makers and other stakeholders. 

One of the key areas of EU added value provided by the projects covered in the present case 
study involves a higher level of excellence in the activities and outputs stemming from EU-
wide  scale  of  their  activities  (as  opposed  to  what  would  have  been  available  at  local  or 
national  level):  one  example  of  this  is  the  EU  Contest  for  Young  Scientists,  which  brings 
together  the  most  talented  winners  of  national-level  competitions.  Similarly,  the  EU-level 
scale of the projects has been crucial to achieving some of the structuring impacts and results 
of  projects  in  the  field  of  university  transformation/acceleration  services,  including  the 

422 

 
development of roadmaps, frameworks and methodologies that reflect the needs of different 
EU countries and institutions. 

The  successful implementation  of  projects  and  the  realisation  of  the  above  outcomes  and 
impacts at institutional level and across the ERA will largely depend on a number of factors. 
These include the ability of project partners to ensure the active dissemination of information 
and the awareness of the local research community/wider public of the project’s activities; 
effective communication and coordination between project partners and their varying needs; 
and  the  commitment  of  policy  stakeholders  and  top-level  management  to  undertake 
transformational changes. In particular, the success of ERA projects focusing on university 
transformation  depends  on  the  ability  of  projects  to  mobilise  wider groups  of  stakeholders 
whose  active  engagement  is  crucial  for  the  data  gathering  and  stakeholder  consultation 
processes  that  will  feed  into  development  of  policy  standards,  recommendations  and 
roadmaps. 

Appendix 1. Literature sources 

•  Communication from the Commission to the European Parliament, the Council, the 

European Economic and Social Committee and the Committee of the Regions on 
achieving the European Education Area by 2025. 

•  EUCYS 2021 Salamanca: the future of science; https://eucys2021.usal.es/book/ 
•  European Commission, European Research Area Policy Agenda – Research and 

Innovation Overview of actions for the period 2022-2024, Luxembourg: 
Publications Office of the European Union, 2021. 

•  Flecha, R., Puigvert, L., & Racionero-Plaza, S. Achieving student well-being for all: 

Educational contexts free of violence, NESET report, 2023, Luxembourg: 
Publications Office of the European Union, doi: 10.2766/463854 

•  Frigge, W., Evaluation Report, EuroScience Open Forum (ESOF2022), 2022, 

available at: https://leiden2022.nl/sites/default/files/2023-
06/D3.2%20ESOF2022%20Programme%20and%20evaluation%20report.pdf 
•  Fundación Española para la Ciencia y la Tecnología, F.S.P. (FECYT), Towards 
Inclusive Science Communication: Reflections And Successful Actions, 2021, 
available at: 
https://www.fecyt.es/es/system/files/publications/attachments/2022/03/comunicacio
n_inclusiva_ciencia_accesible.pdf 

•  Horizon 2020 Work Programme 2018-2020: 16. Science with and for Society. 

(European Commission Decision C(2020)6320 of 17 September 2020) 
https://ec.europa.eu/research/participants/data/ref/h2020/wp/2018-
2020/main/h2020-wp1820-swfs_en.pdf 

•  Horizon Europe Work Programme 2021-2022: 11. Widening participation and 
strengthening the European Research Area. European Commission Decision 
C(2022)2975 of 10 May 2022. 

•  Horizon Europe Work Programme 2023-2024: 11. Widening participation and 
strengthening the European Research Area. European Commission Decision 
C(2023) 2178 of 31 March 2023 

•  Ljubljana Declaration: Gender Equality in Research and Innovation; 

https://www.gov.si/assets/ministrstva/MIZS/Dokumenti/PSEU/Ljubljana-
Declaration-on-Gender-Equality-in-Research-and-Innovation-_endorsed_final.pdf 

423 

 
•  O’Neill, G., & Acheson, H., Progress of University Alliance Projects: Projects 

funded under Horizon 2020, Luxembourg: Publications Office of the European 
Union, 2023, available at: https://op.europa.eu/en/publication-detail/-
/publication/ef36cf58-e57e-11ed-a05c-01aa75ed71a1/language-en/format-
PDF/source-285015820 

•  http://ritrain.eu/ 
•  https://acceleratefuturehei.eu/ 
•  https://allinteract.eu/impact/ 
•  https://catalisi.eu/ 
•  https://crowdhelix.com/ 
•  https://eucys2021.usal.es/ 
•  https://leiden2022.nl 
•  https://ritrainplus.eu 
•  https://www.esof.eu/marie-sklodowska-curie-actions-msca/ 
•  https://www.rmroadmap.eu/ 
•  RM Roadmap Ambassadors Network; 

https://static1.squarespace.com/static/633ae0b47dc8ac471e5978a9/t/6429e8d330
2b496a51e6f569/1680468185491/RM+Roadmap+Ambassadors+Network+V1.pdf 

•  Survey of Horizon Europe beneficiaries, conducted May-July 2023. 

424 

 
 
 
Annex 2.9: Impact of the Framework Programmes on creating high-
quality new knowledge 

1.1. 

Introduction 

One  of  the  specific  objectives  of  Horizon  Europe  is  to  develop,  promote  and  advance 
scientific  excellence  to  support  the  creation  and  diffusion  of  high-quality  new  fundamental 
and applied knowledge and of skills, technologies and solutions to support the training and 
mobility of researchers, to attract talent at all levels676.  

Against this background, the present case study aims to assess the impact of the Framework 
Programme (FP) on the production of high-quality research and its contribution to new and 
emerging,  or  prominent  topics.  Where  possible,  comparisons  are  made  with  other 
international funders or other countries. In addition, the case study discusses the balance of 
different funding opportunities within the FP and its perceived impact on research outcomes. 
Moreover, the case study sheds light on the trajectories that researchers follow within and 
across the Framework Programmes, and on the added value provided by the FP in producing 
high-quality new knowledge.  

To make its assessments, this case study builds on a range of different methods. The study 
relies  on  desk  research  carried  out  on  existing  literature  (EC  reports,  positioning  papers 
drafted by stakeholders in the context of the  OPC677) as well as data sources such as the 
Horizon Dashboard. It also relies extensively on bibliometric and research trends analyses 
conducted in the context of the Evaluation study on Excellent Science under Horizon 2020 
and this study. Furthermore, it builds on surveys conducted in the context of this evaluation 
support study, as well as EC monitoring and reporting data, and interviews with beneficiaries 
and stakeholders.  

To the extent that is possible and relevant to the context of creating high-quality knowledge, 
the present case study covers the entire Framework Programme and its programme parts. 
Given  its  wide  focus  on  the creation  of  research  outputs,  the  case  study  draws  mainly  on 
Horizon 2020 while making connections, to the extent possible, to Horizon Europe. This is 
due to the fact that the available sample of scientific publications produced under the recently 
launched Horizon Europe FP is still much smaller than that of Horizon 2020.  

1.2.  Background information 

1.2.1.  Policy context 

For decades, the Framework Programmes have aimed to promote scientific excellence and 
achieve scientific impact by creating and diffusing high-quality new knowledge. In doing so, 

676 Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 2021. 
Establishing Horizon Europe – the Framework Programme for Research and Innovation, laying 
down its rules for participation and dissemination, and repealing Regulations (EU) No 1290/2013 
and (EU) No 1291/2013. 
677 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 

425 

 
 
the FPs aim to implement the European Research Area (ERA), addressing the fragmentation, 
isolation and compartmentalisation of national research systems in Europe.  

FP7  was  particularly  effective  in  strengthening  scientific  excellence,  with  a  share  of 
publications  in  highly  ranked  journals  well  above  the  European  Union  (EU)  average,  and 
many  interdisciplinary  research  collaborations678.  The  establishing  Regulation  for  its 
successor, Horizon 2020 stated that ‘it is the Union's objective to strengthen its scientific and 
technological bases by achieving a European Research Area in which researchers, scientific 
knowledge and technology circulate freely, and by encouraging the Union to advance towards 
a knowledge society and to become a more competitive and sustainable economy in respect 
of  its  industry’679.  Moreover,  one  of  Horizon  2020’s  specific  objectives  was  to  strengthen 
Europe’s science base by improving its performance in frontier research, stimulating future 
and emerging technologies, encouraging transnational training and career development, and 
supporting research infrastructures680. The importance of creating high-quality knowledge is 
also reflected in the intervention logic of Horizon 2020 (see Figure 10 in Appendix 2).  

Under Horizon Europe, the objective of creating high-quality knowledge is further stressed 
as reflected in the intervention logic for Horizon Europe (see Figure 11 in Appendix 2)681. This 
objective is also reflected in Horizon Europe’s Key Impact Pathways (KIPs), such as KIP1 
‘Creating  high-quality  new  knowledge’  with  three  specific  indicators  to  measure  effects 
over different time frames: 

• 

• 

• 

In the short term: publications, i.e. the number of peer-reviewed scientific publications 
resulting from the Programme; 

In  the  medium  term:  citations,  i.e.  the  citation  index  of  peer-reviewed  publications 
resulting from the Programme; 

In  the  long-term:  world-class  science,  i.e.  the  number  and  share  of  peer-reviewed 
publications resulting from projects funded by the Programme that are core contributions 
to scientific fields. 

Due to its emphasis on generating novel, top-tier knowledge, Horizon Europe builds upon the 
foundation  laid  by  Horizon  2020.  Given  its  reinforced  focus  on  frontier  research,  spurring 
breakthrough innovation and focusing on political priorities682, it is anticipated that Horizon 
Europe will foster the development of a greater amount of innovative knowledge and cutting-

678 Commission presentation of its evaluation of the 7th Framework Programme for Research 
(2016), https://ec.europa.eu/commission/presscorner/detail/de/MEMO_16_146.  
679 Regulation (EU) No 1291/2013 of the European Parliament and of the Council of 11 
December 2013 establishing Horizon 2020 – the Framework Programme for Research and 
Innovation (2014-2020) and repealing Decision No 1982/2006/EC. 
680 Regulation (EU) No 1291/2013 of the European Parliament and of the Council of 11 
December 2013 establishing Horizon 2020 - the Framework Programme for Research and 
Innovation (2014-2020) and repealing Decision No 1982/2006/EC. 
681 European Commission, Directorate-General for Research and Innovation, A new horizon for 
Europe: impact assessment of the 9th EU Framework Programme for Research and Innovation, 
Luxembourg: Publications Office of the European Union, 2018, available at: 
https://data.europa.eu/doi/10.2777/194210.  
682 European Commission, Directorate-General for Research and Innovation, Horizon Europe – 
Strategic plan 2021-2024, Luxembourg: Publications Office of the European Union, 2021, 
available at: https://data.europa.eu/doi/10.2777/083753. 

426 

 
 
edge technologies, thereby enhancing scientific excellence and yielding substantial scientific 
advances. 

1.2.2.  Overall approach and scope of the case study 

The present case study relies on several main methods:  

•  Bibliometric analysis and an analysis of research trends  are used to evaluate the 
extent to which the Framework Programme’s instruments are able to foster the creation 
of high-quality new knowledge in various fields of science. It mainly builds on the findings 
from Horizon 2020 and where possible on preliminary findings from Horizon Europe.  

•  The bibliometric and research trends analyses are complemented by desk research on 
documents, policy reports, the deliverables of projects and other sources related to high-
quality  research  and  emerging  fields  of  science.  It  includes  positioning  papers  of  the 
OPC683 and the Horizon Dashboard684.  

•  Both  of  the  above  methods  are  further  enriched  by  information  gathered  through  a 
programme  of  interviews  (six  representatives  of  stakeholder  organisations  and 
networks  representing  and  supporting  researchers,  as  well  as  three  scientists  and 
coordinators of Horizon 2020 projects). 

•  Furthermore, EC monitoring and reporting data were used to provide insights into the 
extent to which projects in various programme parts of Horizon 2020 and Horizon Europe 
covered the UN Sustainable Development Goals (SDGs). In addition, EC monitoring and 
reporting data were used to track the flow of researchers within and across Framework 
Programmes. 

• 

Lastly,  survey  evidence685  is  used  to  show  the  perceptions  of  Horizon  Europe 
beneficiaries  regarding  the  Framework  Programme  and  the  funding  opportunities  it 
provides.  

1.3.  Case study findings 

In line with the rationale provided in the definition of the KIP, this chapter focuses on scientific 
publications  and  their  impacts.  The  discussion  that  follows  begins  with  a  section  on  the 
baseline situation with regard to the creation of excellent knowledge in the EU and its Member 
States, in comparison to countries and regions elsewhere in the world. Next, indicators for 
excellent  knowledge  creation  under  Horizon  2020  are  reported  in  greater  detail.  Horizon 
Europe686 is expected to generate even more new knowledge and technologies, thus 

683 Open Public Consultation on the Past, Present and Future of The European Research and 
Innovation Programmes (2014-2027). 
684 https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/horizon-
dashboard. 
685 Survey of Horizon Europe beneficiaries, conducted May-July 2023. Surveys of Horizon 
Europe MSCA researchers, conducted in May-November 2023. See Annex 7 for more 
information. 
686 Evidence on the impact of this programme in this case study is limited as Horizon Europe 
projects only started recently. However, he Horizon 2020 analysis is indicative of the mid- and 
longer-term impacts of the European Framework Programmes. 

427 

 
 
promoting  scientific  excellence  and  significant  scientific  impact687.  In  addition,  this 
section provides further information on the balance between different funding opportunities 
under  the  Framework  Programmes.  Specifically,  it  provides  information  on  the  disciplines 
and  (prominent)  topics  covered  in  the  various  programme  parts,  and  the  trajectories  that 
researchers  follow  within  the  Framework  Programme  in  terms  of  the  subsequent  funding 
sources utilised (namely, as a proxy for the flow of researchers and ideas).  

1.3.1.  Baseline: excellent knowledge creation in the world, the EU, and its Member 

States 

Recently,  it  was  demonstrated  that  the  EU  continues  to  assert  its  position  as  a 
powerhouse  in  the  realm  of  science.  Data  from  a  recent  European  Commission  (EC) 
report688 show that in 2020, the EU contributed nearly 20% of all publications documented in 
Scopus, and as such produced the second-highest share of scientific output worldwide 
after China. Among the EU Member States, larger countries such as Germany, Italy, Spain 
and  France  played  a  significant  role  in  generating  a  substantial  portion  of  scientific 
publications.  

In terms of specific fields of science, the EU is the forefront of the domains of economics 
and social sciences, as well as arts and humanities. It also exerts notable influence, albeit 
to a lesser extent, in clinical medicine and biomedical research. In contrast, the US takes 
the lead in health sciences, particularly in public health and health services, while China 
excels in applied and natural sciences as well as in chemistry. Within the EU, there has 
been a discernible rise in the proportion of publications spanning the fields of information 
and communication technologies, enabling  and strategic technologies, and physics 
and  astronomy,  while  the  majority  of  EU publications are in  the  scientific  field  of  clinical 
medicine.  

Thanks to its additional efforts in less technological fields, the EU’s standing has been less 
severely  impacted  by  the  increase  in  the  scientific  output  of  China  (which  focuses 
more heavily on technological fields). China has achieved its leading position in overall 
scientific output gradually, passing the US in 2016 and the EU in 2019, with the US facing a 
strong decline of 13 percentage points in its share of publications, while the EU’s decline was 
smaller in comparison, at 7 percentage points.  

In addition, over recent years, there has been a remarkable improvement in the  quality of 
China’s scientific output, pushing the EU into the third place in its share of top 10% and 
top  1%  most-cited  publications  worldwide.  Here,  in  a  similar  manner  to  trends  in  the 
volume of science produced, China is taking over a large part of the US share and, to a lesser 
extent, that of the EU. In terms of top 10% most-cited publications, China leads with 23.3% 
of such publications, followed by the US (23.0%) and the EU (20.6%). However, in terms of 
the  top  1% most-cited  publications,  the  US  still  dominates,  with  27.5%, followed  by China 
(23.0%)  and  the  EU (18.4%).  Among  the  EU  Member States,  Germany,  Italy,  France  and 

687 European Commission, Directorate-General for Research and Innovation, A new horizon for 
Europe – Impact assessment of the 9th EU framework programme for research and innovation, 
Luxembourg: Publications Office of the European Union, 2018, available at: 
https://data.europa.eu/doi/10.2777/194210. 
688 European Commission, Directorate-General for Research and Innovation, Science, research 
and innovation performance of the EU 2022 : building a sustainable future in uncertain times, 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/78826.  

428 

 
 
Spain are responsible for most of the top 10% most-cited publications in 2020. The shares of 
Germany and France have, however, decreased since 2000. 

Echoing the patterns of overall scientific output, among the top 10% most-cited publications 
globally,  the  EU  claims  the  largest  portions  in  the  fields  of  biology  and  historical 
studies. In contrast, the US takes the lead in the domain of health science and its associated 
scientific subfields. Meanwhile, China is the frontrunner in applied and natural sciences, with 
a  notable  emphasis  on  chemistry,  enabling  and  strategic  technologies,  engineering  and 
information and communication technologies.  

The  aforementioned  European  Commission  report  also  produced  a  specialisation  index 
indicating that the EU’s contribution to publications on the topics designated in the Horizon 
2020  Societal  Challenges689  is  about  20%,  with  the  exception  of  ‘Energy’  and  ‘Secure 
Societies’, where the shares are 17% and 15%, respectively. While in 2018, the US and the 
EU dominated publications relating to ‘Health’ (each producing 21% of all publications), 
China dominated the publications in the areas of the other five Societal Challenges. In terms 
of global shares of the top 10% most-cited scientific publications in the areas of the Societal 
Challenges,  the  US  produces  more  most-cited  publications  in  Health  than  the  EU  (30% 
versus 22%).  

1.3.2.  Excellent knowledge creation under the Framework Programmes 

The  Framework  Programmes  for  Research  &  Innovation  have  played  a  critical  role  in 
generating high-quality knowledge in the EU and beyond. In line with the rationale for the KIP 
relating to scientific impact, this section of the case study provides insights into the production 
of  excellent  research  under  Horizon  2020  across  its  distinct  pillars  and  programme 
parts690,691.  Although  Horizon  Europe  projects  only  started  recently,  it  is  expected  that 
Horizon Europe will continue with the same momentum in delivering similarly  high-
quality and highly prominent scientific output. 

Horizon  2020.  The  majority  of  publications  generated  under  Horizon  2020  originate 
from  the  Excellent  Science  pillar,  which  represents  approximately  two-thirds  of  the 
total, as shown in Figure 1. It is widely acknowledged that this pillar contributes substantially 
to  Europe’s  ability  to  prosper  as  a  knowledge  society,  to  remain  at  the  forefront  of  global 
knowledge production, and to maintain attractive conditions for talented researchers to carry 
out their research692. Within Excellent Science, the European Research Council (ERC) has 

689 These are: 1) Health, 2) Food & Bioeconomy, 3) Transport, 4) Climate & Environment, 5) 
Energy, and 6) Secure Societies. 
690 For these analyses, we build on the bibliometric analysis carried out under the Evaluation 
study on Excellent Science under Horizon 2020, European Commission, Directorate-General for 
Research and Innovation, Budraitis, M., Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of 
the European Framework Programmes for Research and Innovation for excellent science: 
Horizon 2020  annexes, Luxembourg: Publications Office of the European Union, 2023, available 
at: https://data.europa.eu/doi/10.2777/353383. 
691 After drafting this Section on Excellent knowledge creation under the Framework Programme, 
the Staff Working Document on the Horizon 2020 ex-post evaluation was published. SWD(2024) 
29 final. Evaluation Accompanying the Document Report From The Commission To The 
European Parliament And The Council Ex-post evaluation of Horizon 2020, the EU Framework 
Programme for Research and Innovation. https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=SWD%3A2024%3A29%3AFIN&qid=1706528145182  
692 e.g. Guild of European Research-Intensive Universities (2023). Lessons learned and way 
forward: the Guild’s recommendations for Horizon Europe, available at: https://www.the-
guild.eu/publications/position-papers/lessons-learned-and-way-forward-the-guild-s-
recommendations-for-horizon-europe.pdf  

429 

 
 
shown  significant  productivity  in  terms  of  generating  publications,  followed  by  the  Marie 
Skłodowska-Curie Actions (MSCA). Notably, among the components of the ERC, the ERC 
Consolidator grant was particularly successful in generating publications, as indicated by the 
Horizon Dashboard693. Both programme parts, the ERC and MSCA, also scored well in terms 
of  quality,  with  high  citation  averages  and  a  high  Field  Weighted  Citation  Index  (FWCI)694 
score,  as  presented  in  Figure  2.  Publications  produced  under  Future  and  Emerging 
Technologies (FET) receive a remarkable average number of citations and have an FWCI 
that ranks between those of the ERC and the MSCA.  

Within Pillar 2, Industrial Leadership, the ICT component in the ‘Leadership in Enabling 
and  Industrial  Technologies’  (LEITs)  programme  part  clearly  emerges  as  the  primary 
contributor to publications. Furthermore, in terms of relative funding allocations, LEIT-ICT has 
higher productivity than other LEITs programmes. Notably, however, the average number of 
citations  under  the  LEIT-ICT  programme  was  observed  to  be  lower  than  other  LEITs. 
Conversely,  when  considering  FWCI  (which  compares  the  number  of  citations  with  other 
publications in the same subject field and year), the LEIT-ICT programme already exhibits a 
relatively high score. LEIT-Biotech and LEIT-Space both combine a relatively high average 
number of citations and FWCI. 

Within  Pillar  3,  Societal  Challenges  (SC),  SC1  (which  focuses  on  Health,  Demographic 
Change,  and  Well-Being)  has  proven  the  most  successful  at  generating  publications. 
Moreover,  considering  its  budget  allocation,  SC1  has  also  performed  well  in  terms  of 
publication  productivity  compared  with  other  SCs  within  the  pillar.  In  addition,  SC1  also 
performed exceptionally well in generating citations, and has a very high FWCI, pointing to 
the very high quality of the publications produced under this programme part. In particular, 
this means that SC1 is producing publications that are cited almost three times more than 
the global average for publications in the same disciplines. SC5 (which focuses on Climate 
Action, Environment, Resource Efficiency and Raw Materials) also scores very well in terms 
of average number of citations and FWCI. SC6 (which focuses on Inclusive, Innovative and 
Reflective Societies) also achieves a very strong FWCI, yet has a lower average number of 
citations. 

The  programme  part  Spreading  Excellence  and  Widening  Participation  (SEWP)  has 
demonstrated remarkable productivity in generating publications per project, as well as taking 
into  account  its  allocated  budget.  Yet,  for  both  SEWP  and  Science  with  and  for  Society 
(SwafS),  the  figures  below  show  that  the  quality  of  publications  generated  through  these 
programme parts (assessed in terms of FWCI) is somewhat lower than those of the other 
programme  parts,  even  though  these  publications  are  still  cited  more  frequently  than  the 
global average for publications in the same disciplines (i.e. they have an FWCI above 1).  

The present analysis demonstrates that while the production  of excellent knowledge is 
thus an important feature of the Excellent Science pillar, it is not solely restricted to 
this pillar. Several other programme parts and components contribute to the creation of a 
vast set of new and high-quality knowledge in Europe. Moreover, in every programme part, 

693 https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/horizon-
dashboard, accessed August 2023. 
694 (Field-)normalised citation scores, such as FWCI, are used calculate an indication of a 
publication’s citation impact. FWCI is calculated by comparing the number of citations received 
by a publication (the mean observed citation rate) with the number of citations expected for a 
publication (the mean expected citation rate). These rates are normalised for the same 
publication year and subject field to safeguard validity and comparability.  

430 

 
 
the average impacts (in terms of FWCI) are well above the expected global averages 
for publications in the same disciplines.  

In addition, the Evaluation study on Excellent Science under Horizon 2020695 also conducted 
a benchmarking exercise to compare the impact metrics of the Horizon 2020 programme to 
those of other international funders. This benchmarking exercise showed that Horizon 2020 
had  a  higher  average  normalised  citation  score  than  any  other  funder  selected  for 
benchmarking, as well as a higher share of top 1% most-cited publications, meaning 
that the impact and quality of the publications are excellent. 

Figure 1. Share of Horizon 2020 publications, by pillar and programme part. Note: SC1: Health, Demographic Change and 
Well-Being; SC2: Food Security, Sustainable Agriculture and Forestry, Marine, Maritime, and Inland Water Research, and 
the Bioeconomy; SC3: Secure, Clean and Efficient Energy; SC4: Smart, Green and Integrated Transport; SC5: Climate 
Action, Environment, Resource Efficiency and Raw Materials; SC6: Inclusive, Innovative, and Reflective Societies; SC7: 
Secure Societies. Source: compiled by the authors, based on the evaluation study of the European Framework Programmes 
for Research and Innovation for excellent science696. 

Another  analysis,  conducted  during  this  study,  examined  whether  receiving  Framework 
Programme funding has made a difference to the novelty and disruptiveness of the research 
publications  that  researchers  go  on  to  produce  later  in  their  careers  (more  information  on 
these indicators and the analysis is found in the Novelty and Disruptiveness analysis in Annex 

695 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European Framework Programmes 
for Research and Innovation for excellent science: Horizon 2020: annexes, Luxembourg: 
Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383; Annex 6.15. 
696 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European Framework Programmes 
for Research and Innovation for excellent science: Horizon 2020: annexes, Luxembourg: 
Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383. 

431 

 
 
 
6). The results of this analysis do not provide significant evidence that those researchers who 
received funding under Horizon Europe MSCA-IF or ERC generate more disruptive research 
outputs two or five years after their application, compared with counterparts who applied but 
did not receive funding. Because the analysis focuses on funded applications as well as non-
funded  applicants  that  were  considered  to  be  of  high  quality,  it  is  likely  that  these 
unsuccessful  applicants  have  obtained  funding  for  their  research  from  other  sources,  and 
therefore also produce impactful outputs. 

30
25
20
15
10
5
0

3
2,75
2,5
2,25
2
1,75
1,5
1,25
1

T
E
F

C
R
E

I
T
F

A
C
S
M

A
R
F
N

I

o
n
a
N
-
s
T
I
E
L

i

h
c
e
t
o
B
-
s
T
I
E
L

e
c
a
p
S
-
s
T
I
E
L

s
l
a
i
r
e
t
a
M

-
s
T
I
E
L

T
C
I
-
s
T
I
E
L

s
E
M
S
n

i

n
o
i
t
a
v
o
n
n

I

g
n
i
r
u
t
c
a
f
u
n
a
M

-
s
T
I
E
L

1
C
S

5
C
S

3
C
S

2
C
S

6
C
S

7
C
S

4
C
S

P
W
E
S

S
f
a
w
S

Excellent science

Industrial leadership

Societal Challenges

SEWPSwafS

Average citations

FWCI

Figure 2. Quality and impact of publications produced under Horizon 2020, by programme part. 

Note: an FWCI of more than 1 indicates that publications from this programme part have been cited more than the global 
average of publications in the same discipline. 

Note: SC1: Health, Demographic Change and Well-Being; SC2: Food Security, Sustainable Agriculture and Forestry, 
Marine, Maritime, and Inland Water Research, and the Bioeconomy; SC3: Secure, Clean, and Efficient Energy; SC4: Smart, 
Green and Integrated Transport; SC5: Climate Action, Environment, Resource Efficiency and Raw Materials; SC6: Inclusive, 
Innovative and Reflective Societies; SC7: Secure Societies. 

Source: compiled by the authors, based on the evaluation study of the European Framework Programmes for Research and 
Innovation for excellent science697. 

Horizon Europe. The European Commission has reported on the short-term KIP1 indicator698, 
which is the number of peer-reviewed scientific publications, based on the periodic reporting699. 
Specifically, 323 projects have provided a first period report (i.e., 3.5% of projects), out of which 
49  (i.e.,  0.5%)  contain  publication  data.  Importantly,  119  publications  were  flagged  as  peer-
reviewed and had a valid DOI (out of a total of 241 publications). From these publications, 42 were 
matched and were identified as foreground publications (i.e., published after the start of 
the project)700. H2020 generated over 5 000 publications in the first three years of its operation. 
However,  comparing  the  number  of  publications  poses  a  challenge  due  to  the  disparity  in  the 

697 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European Framework Programmes 
for Research and Innovation for excellent science: Horizon 2020: annexes, Luxembourg: 
Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383. 
698 Yet notes that developments are still in progress for this indicator.  
699 On October 18, 2023 
700 Stemming from the Research Infrastructures (Pillar 1), the EIC (Pillar 3) and the Health 
Cluster (Pillar2) 

432 

 
 
 
 
 
number  of  completed  projects.  This  disparity  arises  from  the  delay  in  the  commencement  of 
Horizon Europe and the fact that most of the projects have a duration of three years and even five 
under the ERC, which makes direct comparison of publication metrics difficult.  

In addition, the JRC has produced publications that can be assigned to Horizon Europe701. 
According to data provided by the JRC702, in 2021-2022, the JRC has published almost 1,780 
scientific publications that can be attributed to Horizon Europe.  Most of these address the 
‘Green transition’ (946) and ‘Resilient Europe’ (706), and the subject generating the fewest 
publications was ‘Innovative Europe’ (31)703. The publications related to Horizon Europe were 
cited more than twice as frequently as the world average (FWCI = 2.3), pointing to their high 
quality. Publications relating to the ‘Green transition’ were cited almost three times more than 
the  world  average,  indicating  that  the  JRC  has  mainly  focused  on  producing  output 
related to the green transition which was of particularly high quality. 

Further  analysis  in  the  context  of  this study  has  elucidated  that  so  far704,  937  publications 
have  been  self-reported  by  Horizon  Europe  beneficiaries  on  the  CORDIS  Open  Data 
Portal705. From the publications that could be matched and validated706, almost 40% of these 
publications stem from the ERC, MSCA and INFRA programme parts, 32% is attributable to 
the  Clusters  in  Pillar  2,  22%  is  attributable  to  EIC,  and  5%  stem  from  WIDERA707.  These 
publications have, so far, on average, 4.1 citations.  

More generally, the stakeholder consultation708 indicates that the majority of respondents 
think  that  Horizon  Europe  is  on  track  to  develop,  promote  and  advance  scientific 
excellence (83% of respondents)709, compared with only 4% who believe this is not the case.  

1.3.3.  Balance between different funding opportunities and approaches under the 

Framework Programmes 

The European Framework Programmes for R&I provide different types of opportunities for 
funding  research,  including  top-down  and  bottom-up  research  opportunities,  collaborative 
versus  individual  research  projects,  and  opportunities  addressing  different  Technology 
Readiness  Levels  (TRLs).  This  section  presents  some  insights  regarding  the  balance 

701 The European Commission's Joint Research Centre (JRC) provides independent, evidence-
based science and knowledge, supporting EU policies to positively impact society. The major 
part of the JRC’s budget stems from the EU’s Research & Innovation programme and the 
complementary Euratom research and training programme. Consequently, several peer-
reviewed publications from the JRC have been funded thanks to, and can hence be assigned to, 
Horizon Europe. 
702 Hristova, M., & Lehto, S., JRC contribution to the interim evaluation of Horizon Europe and 
Euratom research and training programme. Analysis of peer-reviewed publications 2021-2022 
and bibliometric indicators, 2023.  
703 These publications have been manually assigned to the themes of Horizon Europe, i.e. 
Innovative Europe; Digital and industrial transition; Resilient Europe; Green transition. 
704 October 21, 2023 
705 Horizon Publications. https://data.europa.eu/data/datasets/cordis-eu-research-projects-under-
horizon-europe-2021-2027?locale=en, accessed on October 31, 2023.  
706 450 publications could be matched, validated and had administrative data available.  
707 The largest proportions are mentioned here. 
708 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027, available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 
709 30% strongly agree; 53% agree; total replies: 1,564; response rate: 56%. 

433 

 
 
between these different types of funding and the extent to which some may be preferred over 
others, depending on the specific goals of the research.  

In particular, the Framework Programmes incorporate a combination of top-down and 
bottom-up approaches to funding research. This implies that the Framework Programme 
on  the  one  hand  provides  support  for  research  that  adheres  to  predefined  thematic  or 
conceptual criteria, known as ‘top-down’ approaches. Examples of top-down approaches 
in  Horizon  2020  include  the  Pillars  of  Societal  Challenges710  and  Industrial  Leadership 
(LEITs). Under Horizon Europe, the six clusters provide top-down funding opportunities for 
projects in Health; Culture, Creativity & Inclusive Society; Civil Security for Society; Digital, 
Industry & Space; Climate, Energy & Mobility; and Food, Bioeconomy, Natural Resources, 
Agriculture & Environment. In addition, the EU Missions711, a new instrument under Horizon 
Europe712, aim to address the biggest challenges for society, and hence focus on pre-defined 
subjects. On the other hand, the Framework Programmes also allocate funding that is open 
to research projects exploring any field, known as ‘bottom-up’ approaches. These bottom-
up  approaches  include  grants  and  projects  funded  through  the  ERC,  the  MSCA,  part  of 
INFRA  (Research  Infrastructures),  SEWP  (in  Horizon  2020), and  the  Widening  part under 
WIDERA (in Horizon Europe).  

The  optimal  balance  between  bottom-up  and  top-down  funding  opportunities  within  the 
Framework  Programme  has  long  been  a  matter  of  debate.  Although  bottom-up  funding 
approaches, mainly available under Pillar 1, have yielded the most of publications relative to 
the budget,  

Figure  1  demonstrates  that both  approaches  have  generated  a  substantial  quantity  of 
publications. Furthermore, Figure 2 indicates that both approaches have the potential 
to  make  significant  impacts.  During  the  ex-post  evaluation  of  Horizon  2020713,  it  was 
underscored  that  both  the  bottom-up  and  top-down  approaches  under  Horizon  2020  were 
aligned with excellent research and societal challenges, and that the mix of approaches is 
complementary and allows better coverage of research needs and policy priorities.  

Another  debate  concerns  the  balance  between  collaborative  and  mono-beneficiary 
research opportunities. Mono-beneficiary grants, in which funds are allocated to individual 
researchers  for  specific  research  projects,  are  available  under  the  MSCA  fellowships;  the 
ERC  Starting,  Consolidator  and  Advanced  grants;  and  the  ERC  Proof  of  Concept  grant, 
among others. Collaborative research opportunities, in which diverse actors form consortia 
to work collectively towards specific R&I objectives are available in the form of (among others) 
ERC Synergy grants; Societal Challenges (under Horizon 2020) and Pillar 2 under Horizon 
Europe i.e. clusters, EIC Pathfinder (under Horizon Europe) and SEWP (e.g. Twinning and 
Teaming). Mono-beneficiary grants under the Framework Programmes are highly valued by 
beneficiaries.  In  relation  to  the  ERC  mono-beneficiary  grants,  for  instance,  interviewees 
underlined that such grants allowed them to significantly advance fields of science and build 

710 These Pillars focused on projects in Health, Food, Energy, Transport, Climate, Inclusive 
Societies, and Security. 
711 These Missions comprise: 1) Adaptation to Climate Change; 2) Cancer; 3) Oceans, Seas and 
Waters; 4) Climate-Neutral and Smart Cities; and 5) Soil Health and Food.  
712 https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-call/2021-
2022/wp-12-missions_horizon-2021-2022_en.pdf. 
713 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R. et al., Evaluation study of the European Framework Programmes 
for Research and Innovation for excellent science: Horizon 2020: phase 1 final study report, 
Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/967813.  

434 

 
 
a scientifically strong and diverse research team. In addition, many interviewees stressed the 
impact that individual grants had on the researcher’s visibility and research career (in terms 
of continuing in academia and leading to promotions or permanent positions). Collaborative 
grants are appreciated due to the fact that they allow international top researchers, scientists, 
private and public sector actors to come together in projects, each of them bringing their own 
expertise,  knowledge  and  perspectives.  It  has  been  argued  that  this  opportunity  the 
Framework Programmes offer, bringing together parties from all over the world, with different 
backgrounds,  and  in  different  disciplines,  is  unique  in  comparison  to  other  funders.  Here 
again, both approaches provide important benefits and enable important research outputs.  

Lastly, the extent to which opportunities for projects at different TRLs are covered in 
the  programme  also  needs  to  be  balanced.  Specifically,  Horizon  2020  represented  a 
substantial change compared with FP7 (2007-2013) in the sense that it covered a wider range 
of Technology Readiness Levels (including higher TRLs, above TRL4-5)714. In the evaluation 
support study of Innovative Europe, a synthesis was made of some programme parts and 
their correlation to TRL coverage. Typically, the Excellent Science pillar under Horizon 2020 
and  Horizon  Europe  focuses  on  lower  TRLs,  while  the  largest  portion  of  the  Industrial 
Leadership (Horizon 2020) and Innovative Europe (Horizon Europe) pillars focus on higher 
TRLs, although EIC Pathfinder focuses on lower TRLs (1-4)715. Under Horizon Europe, Pillar 
2 should cover all TRLs.  

During the interviews carried out for this study and in the stakeholder positioning papers in 
the context of the OPC716, several comments were made with respect to the need to balance 
different types of funding opportunities among these dimensions of Horizon Europe: 

•  Concerns were expressed by stakeholders from universities and university associations 
regarding  the  need  to  leave  sufficient  room  for  bottom-up  (or  less  prescriptive) 
research  funding  opportunities  in  various  programme  parts  of  Horizon  Europe 
other  than  the  Excellent  Science  Pillar.  One  point  made  was  that  open  calls  still 
contribute  substantially  to  addressing  political  priorities,  societal  challenges  or 
economic impact717. Stakeholders and interviewees both indicated that it is important 
to realise that ‘blue-sky’ research, as is often conducted in the ERC, is not unconnected 
with grand challenges718. A recent ERC study shows that many ERC research projects 
do indeed address political priorities and societal challenges, despite being bottom-up in 
nature719.  Another  point  made  is  that,  while  interviewees  strongly  agreed  that 
governments should be able to set priorities regarding the research they contribute to, 
and  society  and  policy  should  be  able  to  prescribe  certain  research  themes  as  being 
important, they also felt that these calls are often too prescriptive in terms of how to 
the topic should be examined, or in defining what types of solutions to investigate. Hence, 
such  calls  do  not  allow  sufficient  creativity  or  for  the  generation  of  disruptive, 

714 European Commission, Directorate-General for Research and Innovation, Naujokaitytė, R., 
Stančiauskas, V., Cakić, M., et al., Evaluation study of the European framework programmes for 
research and innovation for an innovative Europe: Final report: Phase 1, Denham, S.(editor), 
Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/144504. 
715 https://eic.ec.europa.eu/eic-frequently-asked-questions_en. 
716 In the context of the Open Public Consultation on the Past, Present and Future of The 
European Research and Innovation Programmes (2014-2027). 
717 LERU (2023), LERU key messages for the current and future European R&I Framework 
Programmes.  
718 e.g., https://www.coimbra-group.eu/europe-needs-blue-sky-research-more-than-ever/, 
accessed August 2023. 
719 https://erc.europa.eu/news-events/news/erc-reveals-mapping-its-funded-research.  

435 

 
 
breakthrough ideas. Conversely, interviewees who had participated in the Horizon 2020 
Societal  Challenges  pillar  underlined  that  they  preferred  the  top-down  approach 
because it seems more efficient. In particular, interviewees perceived the success rates 
of  bottom-up  calls  as  being  somewhat  lower than  top-down  calls.  Those  interviewees 
who participated in Societal Challenges also stressed that they did not experience the 
calls as being too prescriptive, and indicated that they had sufficient room for their own 
input.  

• 

In the Evaluation study on Excellent Science under Horizon 2020, concerns were also 
raised regarding the availability of adequate funding for collaborative fundamental 
bottom-up research under Horizon Europe (by interviewees and in positioning papers 
in the context of the OPC720). In particular, while the Excellent Science Pillar provides 
ample  opportunities  for  individual  fundamental  bottom-up  research,  besides  the  ERC 
Synergy Grants, it does not provide many opportunities for research in collaboration with 
different  (international)  partners.  However,  experts  have  stressed  the  importance  of 
balancing  mono-beneficiary  research  opportunities  with  opportunities  for  collaboration 
with  different  (international)  partners,  hence  allowing  input  from  different  viewpoints, 
experiences and disciplines, to address complex research and societal problems.  

•  Pillar  2  under  Horizon  Europe,  entitled  Global  Challenges  and 

Industrial 
Competitiveness,  focuses  on  collaborative  top-down  research  prospects.  Here, 
numerous academic stakeholders (in positioning papers submitted to the OPC721) as well 
as interviewees have argued that these mainly focus on higher TRL research, and 
do not allow sufficiently for the creation of fundamental, basic knowledge. It has 
been  argued  that  under  Horizon  2020,  opportunities  for  collaborative  fundamental 
research  had  already  diminished  (although  the  FET  programme  part  and  the  ERC 
Synergy  Grants  focused  on collaborative  fundamental  research)722.  At  the  same  time, 
other  voices  indicate  that  it  was  difficult  to  find  sufficient  support  to  take  technologies 
developed  by  consortia  to  the  higher TRL  levels.  Therefore,  stakeholders  call  for  a 
reorientation of the challenge-driven pillar to support projects at all TRL levels723. 
A  call  to  ensure balance  between  lower and  higher TRLs  in  collaborative  parts  of  the 
programme was included in the Horizon Regulation, and several stakeholders asked that 
its implementation be ensured.  

•  Along similar lines, academic and technical-oriented stakeholders724 expressed the need 
to maintain a balance between smaller and larger collaborative projects, as the size 

720 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 
721 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 
722 Science Europe (2023). Science Europe Response to the Public Consultation on Horizon 
2020 and Horizon Europe. DOI: 10.5281/zenodo.7664436. 
723 e.g. the Guild of European Research-Intensive Universities (2023). Lessons learned and way 
forward: The Guild’s recommendations for Horizon Europe. 
724 e.g. Technical Research Centre of Finland (VTT) (2023), VTT Technical Research Centre of 
Finland’s input to the European Commission consultation on the future of the EU Research and 
Innovation Framework Programmes; European Association of Research and Technology 
Organisations (EARTO) (2023), EARTO Answer to the EC Consultation on the past, present and 
future of the European Research & Innovation Framework Programmes 2014-2027. 

436 

 
 
(in terms of budget and the number of consortium members) as well as the complexity 
of  the  projects  has  increased  under  Horizon  Europe  in  comparison  to  Horizon  2020. 
These  stakeholders  point  out  that  the  trend  towards  fewer  but  larger  collaborative 
projects results in a duplication of activities among projects funded under the same 
calls (e.g. mapping exercises, gap analyses, state-of-the-art analyses) and that  large 
projects  are  extremely  resource-intensive  to  manage,  both  during  proposal 
preparation and project execution (e.g. increased administrative burdens and complex 
coordination).  This  also  may  lead  to  the  need  to  hire  consultants  to  coordinate  and 
manage  proposals,  which  can,  according  to  stakeholders,  dissuade  or  hinder  certain 
institutions  or  regions  from  applying.  Moreover,  in  combination  with  the  new  impact 
approach under Horizon Europe, in which, among other elements, detailed expectations 
regarding  impacts  are  described  in  the  call  texts,  consultants  are  often  necessary 
according  to  interviewees  in  order  to  ensure  the  research  proposal  fits  the  detailed 
requirements and specificities of the call.  

•  Some  interviewees  also  foresaw  a  specific  career-related  risk  resulting  from  this 
administrative  burden.  According  to  them,  the  coordinators  of  large  collaborative 
consortia in challenge- or mission-driven research opportunities under Pillar 2 of Horizon 
Europe  need  to  spend  considerable  time  on  administrative  and  organisational 
issues, during which they cannot focus on their research – thus making them less 
competitive as researchers in future research calls.  

Lastly,  survey  evidence725  also  provides  insights  into  extent  to  which  beneficiaries  believe 
Horizon Europe provides sufficient opportunities to conduct basic/fundamental research in 
new  and  emerging  research  areas.  In  particular,  ERC  principal  investigators  (PIs)  who 
responded to the survey agreed with this statement. This finding may, of course, be related 
to the fact these respondents currently have access to opportunities to fund basic research, 
so  they  assess  it  more  positively.  Similarly,  the  beneficiaries  of  the  EIC  Pathfinder  and 
Cluster4  indicate  that  there  are  to  a  very  large  or  large  extent  sufficient  opportunities  to 
conduct  applied  research  that  provides  the  basis  for  the  development  of  new  products, 
services or processes.  

1.3.4.  Discipline and topic orientation of the research produced under the Framework 

Programmes 

As discussed in Section 1.3.3 above, a variety of types of research is undertaken under the 
Framework  Programmes.  Among  others,  these  include  mission-  or  challenge-driven 
opportunities  versus  bottom-up  opportunities,  as  well  as  opportunities  for  basic  research 
versus  more  applied  research  with  higher  TRLs.  This  diversity  raises  the  question  as  to 
whether the Framework Programmes produce different types of research outputs across their 
programme parts in terms of disciplines or topics addressed and their prominence.  

1.3.4.1. 

To which SDGs have the different programme parts contributed?  

A first analysis provides insights into  those SDGs to which the projects in the different 
programme parts  have  contributed  most.  In  the  Evaluation  study  on  Excellent  Science 

725 Survey of Horizon Europe beneficiaries conducted May-July 2023. Surveys of Horizon 
Europe MSCA researchers, conducted in May-November 2023. See Annex 7 for more 
information. 

437 

 
 
under  Horizon  2020,  the  following  observations  were  made  with  regard  to  Horizon  2020, 
based on the assignment of SDGs to publications: 

• 

• 

In Pillar 1 Excellent Science, the most pronounced SDG was SDG 3 – Good Health and 
Well-Being;  

In  Pillar  2  Industrial  Leadership,  SDG  9  –  Industry,  Innovation  and  Infrastructure  was 
targeted most frequently in projects;  

•  For Pillar 3 Societal Challenges, the most prominent SDG was SDG 13 – Climate Action, 

closely followed by SDG 7 – Affordable and Clean Energy;  

• 

• 

In Pillar 4 Spreading Excellence and Widening Participation, SDG 3 – Good Health and 
Wellbeing was targeted most often, closely followed by SDG 4 – Quality Education. 

In Pillar 5, Science with and for Society, SDG 4 – Quality Education was targeted most 
often. 

Under Horizon Europe, it was generally found that there was an increase in the importance 
of  the  SDGs  compared  with  Horizon  2020726,727.  SDGs  have  been  assigned  to  Horizon 
Europe project proposals on the basis of the OSDG tool728. The following focus areas have 
been identified for Horizon Europe:  

• 

• 

• 

• 

In Pillar 1 Excellent Science, the most pronounced SDG was still SDG 3 – Good Health 
and Well-Being; 

In  Pillar  2  Global  Challenges  and  European  Industrial  Competitiveness,  the  most 
prominent SDG was SDG 7 – Affordable and Clean Energy  

In Pillar 3 Innovative Europe, the most prominent SDG was SDG 3 – Good Health and 
Wellbeing.  

In WIDERA, SDG 16 – Peace, Justice and Strong Institutions featured most prominently 
in terms of the number of HE projects by SDG. When it comes to the EU contribution by 
SDG, SDG 3 – Good Health and Wellbeing was the most supported.  

1.3.4.2.  Which fields of science are addressed by projects in the different 

programme parts? 

Data  analysis  reveals  which  fields  of  science  are  addressed  by  projects  across  the 
various different programme parts of the Framework Programmes729 (see Table 5 and 

726 The types of documents on which the SDG analysis is based differ between Horizon 2020 
and Horizon Europe: the Horizon Europe analysis was carried on the proposal texts, while in 
Horizon 2020 the analysis was based on publications. This makes it difficult to compare the SDG 
contribution between the two Framework Programmes. 
727 For more details, refer to Annex 6 ‘Quantitative research track findings’ of this study.  
728 For a detailed explanation of this tool, please see: 
https://arxiv.org/ftp/arxiv/papers/2211/2211.11252.pdf. 
729 Fields of science refer to their EuroSciVoc Level 1 classification, as presented in the Horizon 
Dashboard: ‘The EuroSciVoc is a multilingual taxonomy that represents all the main fields of 
science that were discovered from CORDIS content and organised through a semi-automatic 
process based on NLP techniques’ (https://op.europa.eu/en/web/eu-vocabularies/euroscivoc). 

438 

 
 
Table 6 in Appendix 3). Under Horizon 2020, the Excellent Science Pillar mainly supported 
grants in the field of ‘Natural Sciences’. In addition, many FET projects were also active in 
‘Engineering  and  Technology’.  In  Pillar  2,  many  LEIT  projects  explored  the  field  of 
‘Engineering  and  Technology’,  especially  the  LEITs  materials,  manufacturing,  nano  and 
space. The projects supported under the programme part Innovation in SMEs are very much 
scattered in terms of disciplines. In Pillar 3, projects under the various societal challenges are 
allocated to many different fields of science, in line with the types of societal challenges they 
address. Most projects under SC1 are allocated to the field of ‘Medical and Health Sciences’. 
Most projects under SC2 are allocated to the field of ‘Agricultural Sciences’, closely followed 
by  ‘Natural  Sciences’.  Projects  under  SC3,  SC4  and  SC5  are  most  active  in  the  fields  of 
Engineering and Technology; SC6 in ‘Social Sciences’; and SC7 in ‘Natural Sciences’. While 
SwafS  projects  mostly  explore  the  field  of  Social  Sciences,  SEWP  projects  are  more 
scattered across different fields.  

Similar trends can be seen under Horizon Europe. The Excellent Science Pillar still supports 
many  projects  in  the  field  of  Natural  Sciences.  The  clusters  under  Pillar  2  accommodate 
projects in various different fields of science, in line with their focus. (For instance, Cluster 1 
projects  typically  address  the  field  of  Medical  and  Health  Sciences;  Cluster  2  on  Culture, 
Creativity  and  Inclusive  Society  is  mainly  active  in  Social  Sciences.  Cluster  4  on  Digital, 
Industry  and  Space  mainly  addresses  projects  in  Natural  Sciences  and  Engineering  and 
Technology.) As in Horizon 2020, the widening part mainly focuses on Natural Sciences. 

1.3.4.3.  Which disciplines are covered by the research outputs from the different 

programme parts of Horizon 2020?  

While the previous section focused on the fields of science addressed in the projects, this 
section  provides  more  information  on  the  subject  areas  and  disciplines  addressed  in 
publications stemming from Horizon 2020. More information about this can be found in the 
Evaluation study on Excellent Science under Horizon 2020730 and Annex 6 of this study.  

When  focusing  on  the  scientific  publications  generated  in  Horizon  2020,  the  bibliometric 
analysis731 found that the greatest number of scientific publications in the Excellent Science 
pillar  were  produced  in  the  discipline  of  ‘Natural  Sciences’  (specifically,  Physics  and 
Astronomy, followed by Engineering, which is in line with the focus of the projects in terms of 
fields of science, ‘Natural Sciences’ and ‘Engineering and Technology’ (see Section 1.3.4.2). 
In Pillar 2, Industrial Leadership, the greatest number of publications were produced in the 
discipline  of  ‘Computer  Science’,  followed  by  ‘Engineering’.  Again,  this  is  in  line  with  the 
heavy focus of projects under this pillar on the field of science ‘Engineering and Technology’. 
Finally, most papers in Societal Challenges are on the scientific discipline of Medicine, which 
aligns with the focus of SC1 – Health, which produced the most publications compared with 
other  SCs.  In  addition,  ‘Engineering’  is  also  a  frequent  discipline  among  publications 

730 See Annex 3 in European Commission, Directorate-General for Research and Innovation, 
Budraitis, M., Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European Framework 
Programmes for Research and Innovation for excellent science: Horizon 2020: annexes, 
Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383. Scopus disciplines have been adopted here.  
731 See Annex 3 in European Commission, Directorate-General for Research and Innovation, 
Budraitis, M., Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European Framework 
Programmes for Research and Innovation for excellent science: Horizon 2020: annexes, 
Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383. Scopus disciplines have been adopted here. 

439 

 
 
stemming  from  Societal  Challenges,  which  is  in  line  with  the  many  projects  that  are most 
active in the fields of ‘Engineering’ and ‘Technology’ under that pillar (see Section 1.3.4.2). 

In general, the above analysis of subject areas and disciplines indicates that Horizon 2020 
has  generally  produced  many  scientific  publications  in  physical  sciences.  Nevertheless,  a 
distinction  should  be  made  between  those  programme  parts  that  cover  science  more 
generally (such as the MSCA and ERC) – which generate scientific publications in all areas 
and mostly in physical sciences – and those programme parts that are more specialised, and 
hence  generate  publications  in  these  particular  focus  areas.  For  example,  SC1  produces 
many publications in health sciences, SC2 in life sciences, and SC6 in social sciences and 
humanities.  LEIT-biotech  produces  publications 
life  sciences,  and  LEIT-ICT,  -
Manufacturing, -Materials, and -Space in physical sciences.  

in 

1.3.4.4. 

How many Horizon 2020 research outputs address new and emerging 
topics in science?  

An analysis based on the classification of research topics as new and fast growing732 shows 
that  many  of  the  research  projects  funded  under  Horizon  2020  are thematically  related  to 
new  and  fast-growing  research  topics  in  science733.  Specifically,  25.5%  of  Horizon  2020 
publications were linked to new and emerging topics and technologies734. According to 
Figure 3, approximately 34-35% of the scientific publications generated in FET (categorised 
under  the  Excellent  Science  Pillar),  the  Societal  Challenges  Pillar,  and  the  Industrial 
Leadership Pillar examine new and emerging topics and technologies. The proportion of ERC 
publications  relating  to  new  and  emerging  topics  is  lower  in  comparison;  however,  when 
considering  the  share of  publications  connected  to  new  and  emerging  topics  that  also  fall 
within the top 1% most cited and are thus highly impactful, the ERC performs better. 

732 Within the scope of the Evaluation study on Excellent Science under Horizon 2020, a 
systematic methodology was used to classify research topics as new and fast growing. More 
than 9,000 research topics were identified as new and fast growing. These included research 
topics such as gut microbiome, microplastics, and artificial intelligence. Subsequently Horizon 
2020 publications were associated with these topics. 
733 For more information on the method and results, see Annex 3 in European Commission, 
Directorate-General for Research and Innovation, Budraitis, M., Pranckevičius, P., Dėlkutė, R., 
et al., Evaluation study of the European Framework Programmes for Research and Innovation 
for excellent science: Horizon 2020: annexes, Luxembourg: Publications Office of the European 
Union, 2023, available at: https://data.europa.eu/doi/10.2777/353383. 
734 That is, 33,663 out of 131,786 publications. 

440 

 
 
35%

34%

35%

30%

23%

25%

21%

40%

35%

30%

25%

20%

15%

10%

5%

0%

2,5%

2,0%

1,5%

1,0%

0,5%

0,0%

5%

ERC

FET

MSCA

INFRA

Excellent Science

Industrial
Leadership

Societal
Challenges

SEWP

SWAFS

Share of publications linked to new and emerging technologies

Share of publications linked to new and emerging technologies that are also among the
1% cited

Figure 3. Percentage of Horizon 2020 publications addressing new and emerging topics, by programme part. Source: 
compiled by the authors, based on the bibliometric analysis carried out during the Evaluation study on Excellent Science 
under Horizon 2020735. 

1.3.4.5.  Which highly prominent topics are covered by the research outputs from 

the different programme parts of Horizon 2020?  

Evidence on research outputs under Horizon Europe is still preliminary, and the emergence 
of patterns in terms of longer-term impacts or the topics covered by scientific publications  will 
take a few years to fully materialise. As this case study aims to provide insights on the longer-
term effects of the Framework Programme, this section looks back at Horizon 2020 scientific 
publications  to  provide  more  insight  into  the  most  prominent  topics  addressed  under  the 
Framework Programme, and how these have materialised.  

The  concept  of  ‘prominent  topics’  refers  to  important  topics  in  terms  of  momentum  and 
visibility. To calculate each topic’s prominence, the study team combined three metrics into 
an average which constitutes its prominence score. These three metrics are citation counts, 
Scopus  view  counts,  and  average  CiteScore  (more  information  on  these  can  be  found  in 
Annex 6 of this evaluation study736). Based on the set of publications from Horizon 2020, the 

735 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European Framework Programmes 
for Research and Innovation for excellent science: Horizon 2020: annexes, Luxembourg: 
Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383. 
736 Annex 6, Research Trends Analysis. 

441 

 
 
 
study team identified the 200 topics with the highest prominence scores737. The full set of Top 
200 topics can be retrieved online738.  

From this set of 200 prominent topics, the five topics with the highest number of Horizon 2020 
publications are:  

1.  Perovskite Solar Cells; Solar Cell; Formamidine (969 publications); 

2.  Molybdenum Disulfide; Monolayer; Van Der Waals (895 publications); 

3.  Object Detection; Deep Learning; Intersection over Union (IOU) (814 publications); 

4.  Exoplanets; Kepler; Atmosphere (489 publications); 

5.  Nasopharyngeal Swabs; Serologic Tests; COVID-19 (377 publications). 

Given the broad scale of the Horizon 2020, it is likely that the same topic is researched 
under  different  programme  parts  of  the  same  Framework  Programme.  It  is  therefore 
interesting  to  examine  these  potential  overlaps  between  topics  to  see  whether  different 
programme parts are contributing to the same topics. In doing so, it is important to consider 
that  some  programme  parts  such  as  the  ERC  take  a  bottom-up  approach,  leaving  the 
selection of topics up to the researchers, while many other programme parts, such as Societal 
Challenges  or  LEITs,  follow  a  top-down  approach,  meaning  that  the  topics  are  largely 
predefined. Nevertheless, similar topics can be addressed via both approaches. For instance, 
within Horizon 2020 Societal Challenges 1 (Health), a topic call was specifically devoted to 
addressing  the  COVID-19  pandemic739,  yet  an  analysis  on  research  carried  out  under  the 
ERC also demonstrated a strong contribution to topics related to COVID-19 as well.  

Figure 4 shows the distribution of scientific publications on those prominent topics that led to 
the largest numbers of publications across the different programme parts. This figure shows 
that, with a few exceptions, the ERC is one of the programme parts that led to the largest 
number of publications in these prominent topics. Its contribution is particularly large in 
terms of producing publications on topics in the fields of astronomy740, quantum physics741 
and  perovskite  solar  cells742.  The  exceptions  are  topics  relating  to  COVID-19  and  ICT743, 
which  are  mostly  covered  under  SC1  and  LEIT-ICT,  respectively  –  programme  parts  that 
focus specifically on the domains of health and ICT. 

737 The Top 5 topics with the highest prominence scores are: 1) Perovskite Solar Cells; Solar 
Cell; Formamidine; 2) Nivolumab; Pembrolizumab; Immunotherapy; 3) Molybdenum Disulfide; 
Monolayer; Van Der Waals; 4) Oxygen Production; Electrocatalysts; Catalyst; 5) Intestine Flora; 
Ruminococcaceae; Microorganisms. 
738 Available at: https://doi.org/10.34894/QIJZK8. 
739 e.g. SC1-PHE-CORONAVIRUS-2020-advancing knowledge for the clinical and public heath 
response to the 2019-nCoV epidemic) 
740 Specific topics: ‘Exoplanets; Kepler; Atmosphere’ and ‘Planet; T Tauri Star; Pre-main 
Sequence Stars’. 
741 Specific topics: ‘Quantum Optics; Thermalization (Energy Absorption); Eigenvalues and 
Eigenfunctions’. 
742 Specific topics: ‘Perovskite Solar Cells; Solar Cell; Formamidine’. 
743 Specific topics: ‘Mobile Communication Systems; Slicing; Software Defined Networking’, 
‘Network Function Virtualisation; Transfer Function; Network Services’ and ‘Object Detection; 
Deep Learning; IOU’. 

442 

 
 
Perovskite Solar Cells; Solar Cell;
Formamidine

Molybdenum Disulfide; Monolayer; Van Der
Waals

39%

29%

11%

5%

4%

11%

2%

32%

17%

43%

5%

Object Detection; Deep Learning; IOU

25%

15%

5%

35%

15%

3%

Exoplanets; Kepler; Atmosphere

76%

17%

4%

Nasopharyngeal Swabs; Serologic Tests;
COVID-19

14%

10%

2%

18%

1%

51%

Mobile Communication Systems; Slicing;
Software Defined Networking

6%

89%

4%

3%

Cavity; Mechanical Oscillators; Resonators

42%

16%

40%

Quantum Optics; Thermalization (Energy
Absorption); Eigenvalues and Eigenfunctions

Network Function Virtualization; Transfer
Function; Network Services

5%

65%

15%

19%

89%

3%

Planet; T Tauri Star; Pre-main Sequence Stars

63%

34%

0%

20%

40%

60%

80%

100%

ERC

MSCA

FET

INFRA

LEITs

Societal Challenges

Other

Figure 4. Distribution of the Top 10 prominent topics that led to the greatest numbers of Horizon 2020 publications, by share 
of publications per programme part. Note: the number of unique publications is lower than the number of combined 
publications for all programmes and challenges. This is due to the double reporting of the same outputs by multiple projects 
and across multiple programmes and challenges. Source: compiled by the study team, based on Annex 6 of this Evaluation 
Study, Research Trends Analysis. 

Most of the Top 200 most-prominent topics are addressed in various programme parts. Given 
the  bottom-up  character and  breadth  of  possible  research  topics  covered in  the  ERC and 
MSCA,  it  is  not  surprising  to  observe  that  the  topics  covered  in  these  programme  parts 
overlap,  and  are  also  addressed  in  other,  more  specifically  oriented  programme  parts 
(INFRA, LEIT-ICT, FET, SC1). In addition, Table 5 in Appendix 3 shows that many projects 
in ERC, MSCA, FET and SEWP, for example, were active in the same fields of science.  

Examining  the  Top  200  most  prominent  topics  by  numbers  of  publications,  the  highest 
incidences of overlap occur in topics with very high prominence scores744. This suggests that 
topics  with  higher  prominence  or  momentum  are  also  addressed  in  different  programme 
parts. Among the Top 200 topics745, the topic ‘Object Detection; Deep Learning; IOU’ is 
present in publications in the largest number of different programme parts. This topic 

744 One notable exception is the topic ‘Nasopharyngeal Swabs; Serologic Tests; COVID-19’, for 
which there was obviously a different kind of urgency and, although of lower prominence, has 
received funding from a diverse set programmes and challenges. 
745 The Top 200 topics are available in Annex 6 of this evaluation study. 

443 

 
 
 
could  be  interpreted  as  a  general-purpose  technology  (GPT),  and  hence  might  find 
application across a large number of funding areas. At the same time, several topics only 
occur  in  publications  from  a  limited  number  of  programme  parts,  such  as  ‘Scattering 
Amplitude;  Symmetry;  Yang-Mills’  (ERC),  ‘Hemivariational  Inequality;  Contact  Problem; 
Friction’ (MSCA), and ‘Optical Networks; Traffic Grooming; Fiberoptic Cable’ (LEIT-ICT).  

Out of the top 200 topics, there are 187 for which either the ERC (a bottom-up programme 
part) or the SC (a top-down programme part) has produced publications. Of these, practically 
half of the topics are addressed in publications from both the ERC and SC, while the other 
half is only addressed in ERC publications. This means that there are no Top 200 topics 
for  which  the  SC produced  publications  and  the  ERC did  not,  which  underlines  the 
broad  nature  of  the  ERC  and,  by  extension,  the  publications  it  generates.  The  most 
prominent topics in which the ERC is highly active (in terms of producing research output) 
and the SC are not (i.e. topics for which there are no publications under Societal Challenges) 
are listed below. Of these, two are related to astronomy (topics 1 and 4) and three are related 
to physics (topics 2,3 and 5): 

1.  Exoplanets; Kepler; Atmosphere  

2.  Cavity; Mechanical Oscillators; Resonators  

3.  Quantum  Optics;  Thermalization 

(Energy  Absorption);  Eigenvalues  and 

Eigenfunctions 

4.  Planet; T Tauri Star; Pre-main Sequence Stars 

5.  Topological Insulators; Topology; Quantum Hall Effect 

Conversely, the most prominent topics for which the SC produced more publications than the 
ERC are listed below, and relate to climate, energy and health. In each case, the specific 
Societal Challenge that produced the highest number of  publications is indicated between 
brackets. 

1. 

Integrated Assessment Model; Carbon; Global Temperature Increase (SC5) 

2.  Demand Response; Demand Side Management; Energy Conservation (SC3) 

3.  Cultural Ecosystem Services; China; Natural Capital (SC5) 

4.  Nasopharyngeal Swabs; Serologic Tests; COVID-19 (SC1) 

5.  Zika Virus; Microcephaly; Flaviviridae (SC1) 

Box 1. EC funding for prominent topics 

While the results above focus on the top 200 most prominent topics that have produced 
the  most  publications,  the  question  also  arises  as  to  how  much  EC  funding  went  into 
these  topics,  and  hence  how  ‘productive’  the  creation  of  knowledge  was  in  these 
particular topics. Annex 6, ‘Quantitative research track findings’, provides details of this 
analysis for all programme parts. For the sake of this case study, tables are provided in 
this box indicating, per programme part under study: 1) the EC contribution and number 
of  publications  per  million  EUR  of  EC  funding  for  the  Top  3  prominent  topics  that 

444 

 
generated the highest number of publications (left column); and 2) the Top 3 prominent 
topics  that  were  most  productive  in  generating  publications  –  i.e.  which  generated  the 
highest number of publications per million EUR of EC funding (right column).  

As  the  table  shows,  the  prominent  topics  that  have  led  to  the  largest  number  of 
publications  in  the  programme  parts  are  not  necessarily  those  that  are  the  most 
productive  (i.e.  those  with  the  highest  number  of  publications  per  million  EUR  of  EC 
funding).  

Top 3 prominent topics that generated 
the highest number of publications 

Top 3 of prominent topics that were 
most productive in generating 
publications 

Topics 

Publications 
per million 
EUR of EC 
funding 

Topics 

Publications 
per million 
EUR of EC 
funding 

1. Perovskite Solar 
Cells; Solar Cell; 
Formamidine 

C
R
E

2. Exoplanets; Kepler; 
Atmosphere 

3. Molybdenum 
Disulfide; Monolayer; 
Van Der Waals 

1. AERONET; Aerosol; 
Aerosol Property 

2. Nasopharyngeal 
Swabs; Serologic 
Tests; COVID-19 

3. Laser-Plasma 
Interaction; Proton 
Beams; Ion 

1. Perovskite Solar 
Cells; Solar Cell; 
Formamidine 

2. Molybdenum 
Disulfide; Monolayer; 
Van Der Waals 

A
R
F
N

I

A
C
S
M

0.27 

2.67 

0.42 

0.93 

0.10 

0.07 

0.20 

0.22 

1. Nodding Syndrome; 
Onchocerciasis; Epilepsy 

18.31 

2. Thermodynamics; 
Carleman Estimate; 
Boundary Stabilization 

3. Fiberoptic Cable; 
Supercontinuum 
Generation; Four-Wave 
Mixing 

1. XMM-newton 
Telescope; Imager; Focal 
Plane 

2. Cross Section; 
Nucleons; Halos 

3. Acoustics; 
Stratosphere; Bolides 

1. Hemivariational 
Inequality; Contact 
Problem; Friction 

2. Ballast; Prestressed 
Concrete; Railroad 

6.25 

5.14 

3.61 

2.75 

2.38 

51.67 

12.55 

445 

 
 
 
 
 
3. Boson; Partons; 
Higgs Bosons 

1. Scintillation; 
Luminescence; Garnets 

2. Exosomes; 
Extracellular Vesicles; 
Microrna 

3. Multifactorial 
Inheritance; Summary 
Statistic; Single 
Nucleotide 
Polymorphism 

1. Nanoethics; Artificial 
Intelligence; Innovation 
Process 

2. Citizen Scientists; 
Monitoring; Volunteers 

3. Scientific 
Misconduct; Research 
Integrity; Responsible 
Conduct 

P
W
E
S

S
f
a
w
S

1.28 

0.61 

0.07 

0.10 

0.08 

0.14 

0.88 

3. Quantum Probability; 
Cognition; Ellsberg 
Paradox 

1. Noncommutative; 
Mathematics; Heisenberg 
Algebra 

2. Impurities; Density 
Matrix; Quantum Dots 

3. Biochemical Substance; 
Climate Models; 
Heterarchy 

1. Science 
Communication; Public 
Engagement; Research 
Personnel 

2. Dialogic; Learning 
Communities; Education 

3. Web Accessibility; 
Ubiquitous Computing; 
Learning Management 
System 

8.35 

29.57 

10.01 

7.51 

1.35 

1.20 

1.11 

446 

 
 
 
 
 
 
 
1.3.4.6.  What share do Horizon 2020 scientific publications represent in relation to 

the research output on prominent topics produced in the EU as a whole 
and by the main international knowledge producers? 

Based on the set of publications from the EU overall and the main international knowledge 
producers (including a total of 32 countries746), the 200 topics with highest prominence scores 
were  identified  by  the  study  team747.  The  full  set  of  Top  200  topics  can  be  retrieved 
online748,749.  

From this set of 200 prominent topics, the five topics with the highest number of publications 
are:  

1.  Object Detection; Deep Learning; IOU (71,093 publications) 

2.  Perovskite Solar Cells; Solar Cell; Formamidine (35,454 publications) 

3.  Nivolumab; Pembrolizumab; Immunotherapy (34,058 publications) 

4.  Boson; Partons; Higgs Bosons (27,983 publications) 

5.  Microrna;  Growth  Arrest  Specific  Transcript  5;  Small  Nucleolar  RNA  (27,251 

publications) 

Using this list of Top 200 most prominent topics, the number of FP publications that address 
these  topics  are  benchmarked  in  comparison  to  the  number  of  scientific  publications 
generated by the EU overall and by the main international producers of knowledge.  

This analysis is first approached from the perspective of those topics in which the EU is an 
important  producer of  knowledge  compared  with  other  international  knowledge  producers. 
Not only is the EU’s international position considered, but also the position of the Framework 
Programme within the scientific publication production of the EU. Next, the analysis zooms 
in on those topics in which the Framework Programme occupies the strongest position, to 
compare its contribution with that of the other international knowledge producers.  

Topics in which the EU is an important producer of knowledge 

This section focuses on those topics from the Top 200 prominent topics in which the EU is 
an important producer of knowledge. The production of publications on these topics in the 
EU-27750 is compared against the performance of other international producers of knowledge 

746 These countries include: the EU-27, the United Kingdom, China, Japan, South Korea and the 
United States. More information on this approach and methodology can be found in Annex 6. 
747 The Top 5 topics with the highest prominence score are: 1) Perovskite Solar Cells; Solar Cell; 
Formamidine; 2) Nivolumab; Pembrolizumab; Immunotherapy; 3) Molybdenum Disulfide; 
Monolayer; Van Der Waals; 4) Oxygen Production; Electrocatalysts; Catalyst; 5) Cyanogen; 
Graphitic Carbon Nitride; Photocatalysts.  
748 Top 200 topics available at: https://doi.org/10.34894/DNWIXV 
749 In addition, another international analysis has been conducted taking into consideration 180 
countries and territories to complement the analysis on the 32 main producers of knowledge. 
The results are discussed in Annex 6. Research Trend Analysis. We refer to this as the ‘global 
topic analysis’. 
750 Since the idea of this study is to focus on the long-term impact of the FP in terms of 
knowledge creation, this section focuses on EU27 instead of EU28. 

447 

 
 
(namely, the United Kingdom, China, Japan, South Korea and the United States) to examine 
the EU’s contribution.  

Looking  at  which  Top  200  prominent  topics  the  EU-27  contributes  the  most  to  in  terms  of 
number  of  scientific  publications,  Table  1  indicates  that  the  shares  of  EU-27  publications 
relating  to  ‘Boson;  Partons;  Higgs  Bosons’,  ‘Soccer;  Elites;  Football’  and  ‘Product-service 
Systems; Service Economy; Value Co-Creation’ are very high compared with the production 
stemming from the other main international knowledge producers751. Nevertheless, the share 
of FP publications as a proportion of the total number of EU-based publications in these topics 
is rather limited. Among these topics in which there is strong EU contribution, the topic for 
which the FPs’ contribution is largest is ‘Smart Cities; Big Data; Internet of Things’. The two 
topics relating to the energy transition also have a larger share of FP publications.  

Topic Names 

Boson; Partons; 
Higgs Bosons 

Soccer; Elites; 
Football 

Product-service 
Systems; Service 
Economy; Value Co-
Creation 

Energy; 
Refurbishment; 
Renovation 

Energy Transition; 
Innovation System; 
Sustainability 

Radical Right; 
Populist; Right-Wing 
Populism 

PPCP; 
Micropollutant; Drugs 

Total 
publications 
by topic*,** 

EU-27 
publications 

EU-27 publications 
as a share of total 
publications by 
topic 

FP 
publications as 
a share of EU-
27 publications 

27,983 

21,242 

76% 

1.4% 

5,904 

3,996 

68% 

0.1% 

7,485 

5,042 

67% 

1.8% 

7,317 

4,865 

66% 

3.0% 

7,696 

5,089 

66% 

3.7% 

6,665 

4,278 

64% 

1.9% 

8,447 

5,376 

64% 

1.5% 

751 The global topic analysis (based on 180 countries instead of 32) confirms that the EU27 
provides an important contribution to the knowledge on ‘Boson, partons, higgs bosons’, 
contributing to 40.8% of the world knowledge production. However, in the global topic analysis, 
‘boson, partons, higgs bosons’ is (only) at place 7 for largest EU contribution in the world. The 
largest EU contribution in the world based on the knowledge production of 180 countries is for 
the topic ‘Product-service Systems; Service Economy; Value Co-Creation’ (49.7%), followed by 
‘Energy Transition; Innovation System; Sustainability’ and ‘Energy; Refurbishment; Renovation’ 
with respectively 49.4% and 48.8%. 

448 

 
 
Topic Names 

Ionic Collisions; 
Relativistic Heavy-
ion Collisions; Heavy 
Ion 

Smart Cities; Big 
Data; Internet of 
Things 

Ibrutinib; B-Cell 
Chronic Lymphocytic 
Leukemia; 
Venetoclax 

Total 
publications 
by topic*,** 

EU-27 
publications 

EU-27 publications 
as a share of total 
publications by 
topic 

FP 
publications as 
a share of EU-
27 publications 

6,750 

4,226 

63% 

1.4% 

7,049 

4,377 

62% 

4.0% 

5,877 

3,416 

58% 

2.1% 

Table 1. Top 10 most prominent topics in which the share of EU-27 publications among the total number of publications 
produced by the main international knowledge providers752 is largest. Source: see Annex 6, Research Trends Analysis, 
Topical and funding overlap – external analysis. Based on the dataset available at: https://doi.org/10.34894/DNWIXV. 

Notes: Data are ranked on the basis of the column ‘EU-27 publications as a share of total publications by topic’ (highlighted 
in green). 

*These data, based on SciVal, constitute a ‘straight count’, assigning whole publications to countries instead of making a 
‘fractional count’. This means that this column might consider more publications per country than there are in reality. This 
column may thus be an overestimation. 

**Total number of publications for the 32 countries studied. 

For  the  Top  200  most  prominent  topics  in  the  publications  from  the  32  countries  studied, 
Figure 5 shows the division of the shares of publications the EU-27, the US and China have 
in the publication’s total. For example, for the majority of the Top 200 most prominent topics 
(i.e. 131 or 66%), the share of EU-27 publications ranges between 25% and 50% of the total 
publications.  

The  figure  below  suggests  that  the  EU-27  and  China  have  a  few  prominent  topics  for 
which they are very strong contributors in the total production of knowledge (according 
to the total number of publications produced by the 32 countries studied). Specifically, there 
are  34  out  of  the  200  most  prominent  topics  in  which  the  EU-27’s  share  of  knowledge 
production is over 50%, while China has 29 such topics.  Conversely, the findings suggest 
that the US appears to specialise in fewer specific prominent topics, but contributes 
fairly consistently to the bulk of the prominent topics with between 0% and 50% of the 
total production of publications753.  

752 i.e., The EU and the main international knowledge producers, in total including 32 countries, 
namely EU27, United Kingdom, China, Japan, South Korea, and the United States. 
753 The global topic analysis (based on 180 countries instead of 32) confirms that the US is 
specialised in fewer specific prominent topics but contribute consistently to the bulk of the 
prominent topics with 0 - 50%. 

449 

 
 
0-25%

25-50%

50-75%

75-100%

0

20

40

60

80

100

120

140

160

180

200

EU27

35

131

33

1

US

127

68

2
3

China

99

72

25

4

Figure 5. Quartiles of shares of publications by region in the total number of publications. Source: see Annex 6, Research 
Trends Analysis, Topical and funding overlap – external analysis. Based on a dataset available at: 
https://doi.org/10.34894/DNWIXV. 

Topics in which the Framework Programme (Horizon 2020) is an important producer 
of knowledge 

Table  2  shows  for  which  of  the  Top  200  prominent  topics  the  Framework  Programme 
produces the largest shares of scientific publications out of the total number of publications 
for  the  32  countries  studied.  Specifically,  the  number  of  Horizon  2020  publications  on  the 
prominent  topic  ‘Planet;  T  Tauri  Star;  Pre-main  Sequence  Stars’  amount  to  5.0%  of  all 
publications on the topic, while Horizon 2020 publications on the topic ‘Topological Insulators; 
Topology; Quantum Hall Effect’ amount to 4.9% of the total. This finding suggests that the 
FP is driving research on astronomy (specifically in these topics) in the world (for which the 
publication total for the 32 countries under study is a proxy), more than any other topics from 
the Top 200754.  

754 The global topic analysis (based on 180 countries instead of 32) confirms that Horizon 2020 
is generating a substantial share of publications in several prominent topics, such as topics in 
physics and astronomy. For instance, considering the world production (of 180 countries), the 
topic to which the FP contributes most knowledge is ‘Topological Insulators; Topology; Quantum 
Hall Effect’, with 3.8%, followed by ‘Weyl; Metalloids; Paul Adrien Maurice Dirac’ with 3.0%. 

450 

 
 
 
 
 
Topic Names 

Total 
publications 
by topics*,** 

EU-27 
publications 

EU-27 
publications 
as a share of 
all 
publications 
by topic 

FP 
publications 
as a share 
of EU-27 
publications 

FP 
publications 
as a share 
of all 
publications 
by topic 

Planet; T Tauri 
Star; Pre-main 
Sequence Stars 

Topological 
Insulators; 
Topology; 
Quantum Hall 
Effect 

Neutron Stars; 
Mergers; 
Gravitational 
Waves 

Integrated 
Assessment 
Model; Carbon; 
Global 
Temperature 
Increase 

Weyl; Metalloids; 
Paul Adrien 
Maurice Dirac 

Exoplanets; 
Kepler; 
Atmosphere 

Molybdenum 
Disulfide; 
Monolayer; Van 
Der Waals 

Single-Cell 
Analysis; Small 
Cytoplasmic RNA; 
Single Cell RNA 
Seq 

Graphite; 
Terahertz; 
Photodetectors 

6,573 

3,702 

56% 

8.9% 

5.0% 

6,542 

2,443 

37% 

13.0% 

4.9% 

6,357 

3,261 

51% 

8.7% 

4.5% 

7,796 

3,777 

48% 

8.2% 

4.0% 

6,709 

1,926 

29% 

13.8% 

4.0% 

12,739 

6,642 

52% 

7.4% 

3.8% 

25,473 

8,107 

32% 

11.0% 

3.5% 

6,899 

1,947 

28% 

12.2% 

3.4% 

8,069 

2,326 

29% 

11.8% 

3.4% 

451 

 
Topic Names 

Total 
publications 
by topics*,** 

EU-27 
publications 

EU-27 
publications 
as a share of 
all 
publications 
by topic 

FP 
publications 
as a share 
of EU-27 
publications 

FP 
publications 
as a share 
of all 
publications 
by topic 

Human-Robot 
Interaction; 
Humanoid Robot; 
Man-Machine 
Systems 

9,466 

4,999 

53% 

6.1% 

3.2% 

Table 2. Top 10 prominent topics for which the share of FP publications among the total number of publications produced by 
the main international knowledge producers755 is the largest. 

Notes: Data are ranked on the basis of the column ‘FP publications as a share of all publications by topic’ (highlighted in 
green). 

*these data, based on SciVal, constitute a ‘straight count’, assigning whole publications to countries instead of making a 
‘fractional count’. This means that this column might consider more publications per country than there are in reality. This 
column may thus be an overestimation.  

**total number of publications for the 32 countries studied. 

Source: see Annex 6, Research Trends Analysis, Topical and funding overlap – external analysis. Based on a dataset 
available at: https://doi.org/10.34894/DNWIXV. 

1.3.4.7. 

Do projects that produce high-quality research outputs on the same topic, 
but which are funded under different Horizon 2020 programme parts, 
generate different types of outcomes?  

Given the specific modalities and focus of different programme parts (see Section 1.3.3), it 
is to be expected that the types of research and outcomes produced under different 
programme parts will differ. To examine this in greater depth, an additional analysis was 
conducted to evaluate whether projects supported under different Horizon 2020 programme 
parts,  but  which  produced  high-quality  research  outcomes  on  the  same  topics,  achieved 
different outputs in terms of scientific publications or the generation of basic versus applied 
research outputs.  

To select projects and topics for more in depth, the following approach was used. First, the 
study  team  identified  those  Horizon  2020  projects  that  produced  a  high  number  of 
publications and had a high average FWCI, and noted their topic. From this list of projects, 
the study team identified several topics that appeared under several programme parts and 
which had generated important impacts, such as research on perovskite, COVID-19 and 5G. 
These topics tend to be associated with the EU’s policy priorities or with current worldwide 
challenges, as was the case with COVID-19, or with the green and digital transitions.  

Table 3 presents an illustrative selection of five impactful projects (namely, those which have 
generated  a  large  number  of  publications  with  a  high  average  FWCI)  across  different 
programme parts, all of which focus on perovskite as topic. A qualitative analysis shows that 
projects under the ERC and MSCA mostly concentrate on basic research or applied research 
with an incremental aspect, focusing on the proprieties of the mineral, while projects under 
the other programme parts have concentrated on applied research, especially with regard to 
the application of the mineral. In terms of research outputs and impacts, among this particular 

755 i.e., The EU and the main international knowledge producers, in total including 32 countries, 
namely EU27, United Kingdom, China, Japan, South Korea, and the United States. 

452 

 
 
set of projects, those projects focusing on applied research tend to have a greater number of 
research  outputs  (e.g.  in  the  form  of  more  scientific  publications)  compared  with  those 
projects focusing on basic research. This can be explained by the different types of research 
being  carried  out  and  the  impacts  that  are  to  be  expected  from  these,  as  well  as  by  the 
different levels of net EU contribution to each project, with ERC and MSCA projects receiving 
a significantly smaller contribution than other projects.  

In Table 4 below, which presents a subset of Horizon 2020 projects contributing to research 
on COVID-19, the analysis reveals a similar pattern, with the ERC project focusing more on 
basic research (particularly on the origin of the virus), and projects under Societal Challenge 
1 focusing on applied research (for instance, clinical trials of therapies to treat the virus). One 
project  under  Infrastructures  focused  on  capacity  building,  specifically  the  creation  of  a 
network of institutions that are able to address pandemics.  

453 

 
 
Project name 

Impact metrics 
calculated by 
the study team 

Pro-gramme 
part 

Type of research 

Research outputs 
according to project 
reporting 

Net EU contribution 
(EUR) 

PINNACLE 
(ended) 

10 publications 
– FWCI 8.3 

ERC 

Basic research and advances in the novel 
synthesis of perovskite nanocrystals (NCs) 

10 peer-reviewed 
publications 

1,498,187.50 

GOTSolar 
(ended) 

32 publications 
– FWCI 13.4 

FET 

Applied research focusing on disruptive 
approaches to the development of highly 
efficient, long-lasting and environmentally 
safe perovskite solar cells (PSCs). 

PeroCUBE 
(running until March 
2024) 

12 publications 
– FWCI 5.4 

LEIT 
Advanced 
Materials 

Crystal Solar 
(ended) 

10 publications 
– FWCI 18.5 

MSCA 

PERTPV 
(ended) 

44 publications 
– FWCI 5.3 

SC 3 

Applied research beyond the use of 
perovskite in solar cells, in particular 
regarding scalable production processes 
and future market entry of advanced 
products by advancing halide perovskite 
technology 

Applied and incremental research on 
producing next-generation solar cells 
exceeding 30% power conversion 
efficiency at low cost. 

30 peer-reviewed 
publications 
9 conference 
proceedings 
1 non-peer reviewed 
publication 
54 other 
Referred to as a 
success story 

17 peer-reviewed 
publications 
1 conference 
proceeding 

2,993,403.50 

5,615,731.00 

8 peer-reviewed 
publications 

251,857.80 

Applied research focusing on advancing 
perovskite thin-film PV technology to the 
next level by undertaking a ‘double-
pronged’ drive on both performance and 
the development of scalable device and 
module fabrication methodologies, 

45 peer-reviewed 
publications 
7 prototypes/ 
demonstrators/ pilots 
Considered a 
success story 

4,996,041.25 

454 

 
Project name 

Impact metrics 
calculated by 
the study team 

Pro-gramme 
part 

Type of research 

Research outputs 
according to project 
reporting 

Net EU contribution 
(EUR) 

CHEOPS 
(ended) 

13 publications 
– FWCI 7.5 

SC 3 

compatible with high-volume 
manufacturing. 

Applied research focusing on developing 
very low-cost but high-performing 
photovoltaic (PV) devices based on the 
emerging perovskite (PK) technology. 

13 peer-reviewed 
publications 
10 demonstrators/ 
prototypes/ pilots 
Considered a 
success story 

3,299,095.00 

Table 3. Illustrative selection of highly impactful Horizon 2020 projects on the topic of perovskite. Note: the second column shows the impact metrics calculated by the study team, while the fifth 
column shows the research outputs reported in Cordis. The number of publications might differ due to new updates in Cordis. Source: compiled by the study team, based on the Cordis 
webpages of the projects and the bibliometric analysis (Annex 6 to this evaluation support study)

455 

 
Project name 

Impact metrics 
calculated by 
the study team 

Pro-
gramme 
part 

Type of research 

Research 
outputs 
according to 
project 
reporting 

Net EU contribution 
(EUR) 

ReservoirDOCs 
(ended) 

34 publications 
– FWCI 11.4 

ERC 

Basic research focusing on the emergence and 
establishment of key viral pathogens. Research 
important to studying the origin of COVID-19. 

40 peer-
reviewed 
publications 

1,810,576.00 

EVA-GLOBAL 
(running until 
September 2024) 

187 publications 
– FWCI 7.7 

INFRA 

Capacity building (network) 

RECoVER 
(ended) 

57 publications 
– FWCI 12.8 

SC 1 

Applied research, building on the expertise of the 
FP7 PREPARE project to undertake clinical studies 
in primary and hospital care as well as 
epidemiological and biological investigations and 
modelling to fill knowledge gaps on SARS-CoV-2 
infectivity and transmission. 

327 peer-
reviewed 
publications 
2 other 
Considered a 
success story 

23 peer-
reviewed 
publications 
41 other 

13,902,748.75 

19,995,195.00 

Table 4. Illustrative selection of highly impactful Horizon 2020 projects on the topic of COVID-19. Note: the second column shows the impact metrics calculated by the study team, while the fifth 
column shows the research outputs reported in Cordis. The number of publications might differ due to new updates in Cordis. Source: compiled by the study team, based on the Cordis 
webpages of the projects and the bibliometric analysis (Annex 6 to this evaluation support study).

456 

 
 
 
In  summary  –  as  outlined  in  the  tables  above,  and  confirming  the  earlier findings  that  the 
production of excellent knowledge does not occur solely in the pillar of Excellent Science (see 
Section  1.3.2)  –  projects  in  various  programme  parts  appear  to  be  successful  in 
producing high-quality and impactful knowledge. In addition, these projects have mainly 
operated  in  line  with  the  overall  aims  of  the  Framework  Programme.  Specifically,  the 
Excellent Science set of activities aims to reinforce and extend the excellence of the Union's 
science  base,  while  the  Societal  Challenges  activities  cover  the  full  cycle  of  basic  and 
fundamental research, but also applied research, knowledge transfer or innovation, with a 
focus on policy priorities and the aim of stimulating the critical mass of R&I efforts required to 
achieve the Union’s policy goals. In general, participants select the programme part to 
which they wish to apply on the basis of their need or desire for mono-beneficiary or 
collaborative research; the opportunities they see for their particular research topic; 
and the programme part’s fit with their network. Indeed, from our interviews, it appears 
that  researchers  working  in  collaboration  with  other  partners  typically  seek  those 
opportunities within the Framework Programme landscape that relate to their topic of focus 
(e.g. perovskite), and do not necessarily scrutinise the landscape to first identify all potential 
topic calls to which they might contribute, before making the decision to apply. In addition, 
researchers that have a tradition of collaborating with particular partners often remain within 
the (selection of) programme part(s) they know.  

1.3.5. 

Interdisciplinary research and the integration of Social Science and Humanities 
(SSH) into research  

Given the importance and necessity of engaging in interdisciplinary research to address and 
provide  solutions  to  complex  challenges  and  to  foster  innovation756,  it  has  become  an 
ambition  of  the  Framework  Programme  to  provide  wide  support  to  interdisciplinary 
research projects.  

As a result, interdisciplinarity and cross-sectoral collaboration constituted a major component 
of Horizon 2020, and opportunities for interdisciplinary projects have been encouraged still 
further  under  Horizon  Europe,  especially  in  Pillar  2  on  Global  Challenges  and  Industrial 
Competitiveness.  

Interdisciplinary research can be defined as a mode of research by teams or individuals that 
integrates information, data, techniques, tools, perspectives, concepts and/or theories from 
two  or  more  disciplines  or  bodies  of  specialised  knowledge,  to  advance  fundamental 
understanding  or  to  solve  problems  whose  solutions  are  beyond  the  scope  of  a  single 
discipline757. 

Under Horizon 2020, interdisciplinarity was gradually streamlined in the programme across 
different sectors. It was an evaluation criterion in Horizon 2020, yet no quantitative indicator 
was set to evaluate project outcomes on the implementation of interdisciplinarity758. 

756 Duerr, F., & Herkommer, A., (2019). Why does interdisciplinary research matter? Advanced 
Optical Technologies, 8(2). https://doi.org/10.1515/aot-2019-0026. 
757 European Commission, Directorate-General for Research and Innovation, Naujokaitytė, R., 
Mačiulienė, M., Stančiauskas, V. et al., Evaluation study on the implementation of cross-cutting 
issues in Horizon 2020: study in support of the ex-post evaluation of the European Framework 
Programme for Research and Innovation Horizon 2020. Annex 3, Case study report, Dirham, 
S.(editor), Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/331671. 
758 European Commission, Directorate-General for Research and Innovation, Naujokaitytė, R., 
Mačiulienė, M., Stančiauskas, V. et al., Evaluation study on the implementation of cross-cutting 

457 

 
 
 
However,  the  bibliometric  analysis  conducted  during  the  Evaluation  study  on  Excellent 
Science under Horizon 2020759 provides insights into the share of scientific publications that 
belong  to  different  (Scopus)  disciplines  (see  also  Section  1.3.4.3  for  disciplines-per-
programme  part).  The  results  of  the  above  analysis  show  that  LEIT-Biotech,  SC7 
(Secure societies), LEIT-Manufacturing and LEIT-ICT exhibited the highest proportion 
of  interdisciplinary  publications  in  Horizon  2020,  exceeding  80%.  Conversely, 
Innovation in SMEs, FET and the ERC have comparatively lower shares of interdisciplinary 
publications  (up  to  65%).  In  addition,  the  study  indicates  that  an  increase  in  knowledge 
diversity,  measured  by  the  number  of  research  areas  covered,  has  positively  influenced 
research  productivity  in  terms  of  publication  output.  However,  the  study  also  cautions 
against excessive diversity, as this could hinder productivity.  

In general, these findings lend further support to the belief in the value and importance of 
interdisciplinary  research.  Several  stakeholders  agree  that  under  Horizon  Europe  Pillar  2, 
more  opportunities  have  been  created  for  interdisciplinary  research.  However,  some 
warnings and challenges are outlined: 

•  While interdisciplinarity may boost many research projects, it should be noted that not all 
projects will benefit equally from this approach, so a ‘one size fits all’ approach should 
be avoided760. 

• 

• 

It is argued that to facilitate such interdisciplinarity, evaluators should be better guided 
on the topic, and briefings on interdisciplinarity could be further improved761. 

Interdisciplinarity is not a synonym for cross-sectoral collaboration, meaning that simply 
including project partners from different sectors does not mean that the research is truly 
interdisciplinary. According to academic stakeholders, topics need to be built around a 
strong research component to enable interdisciplinary collaboration762. 

On  the  basis  of  a  qualitative  assessment  of  evidence,  many  stakeholders  agree  that 
interdisciplinarity should  be strongly incentivised763.  For example,  interviews  with  ERC PIs 
indicate that interdisciplinarity is appreciated, and that the sizes of grants allow them to attract 
more diverse profiles to the research team than would otherwise be possible (e.g. through 
more limited national funding). Likewise, interviewees emphasised that this type of research 

issues in Horizon 2020: study in support of the ex-post evaluation of the European Framework 
Programme for Research and Innovation Horizon 2020. Annex 3, Case study report, Dirham, 
S.(editor), Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/331671. 
759 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European Framework Programmes 
for Research and Innovation for excellent science: Horizon 2020: phase 1 final study report, 
Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/967813. 
760 e.g. Guild of European Research-Intensive Universities (2023). Lessons learned and way 
forward: the Guild’s recommendations for Horizon Europe. 
761 League of European Research Universities (LERU) (2023). LERU key messages for the 
current and future European R&I Framework Programmes. 
762 e.g. Guild of European Research-Intensive Universities (2023). Lessons learned and way 
forward: the Guild’s recommendations for Horizon Europe. 
763 e.g. League of European Research Universities (LERU) (2023). LERU key messages for the 
current and future European R&I Framework Programmes. 

458 

 
 
is not always ‘automatically’ preferred, but should depend on the topic and the relevance of 
interdisciplinarity.  

Under Horizon Europe, more than half of respondents to the survey of successful applicants 
agree that there are sufficient funding opportunities for interdisciplinary research764. 

Along  the  same  lines,  it  is  the  EC’s  ambition  to  better  integrate  Social  Science  and 
Humanities (SSH) or Social Science, Arts and Humanities (SSAH). Such integration is found 
to be important in addressing complex societal challenges, and it is important that research 
and  innovations  should  be  developed  from  the  perspective  of  societal  and  cultural 
acceptance. While several steps have been taken in this direction under Horizon Europe (e.g. 
using  a  flagging  system  for  call  topics  that  are  relevant  to  SSAH;  including  an  SSAH 
component in evaluations; and appointing SSAH experts to evaluation panels), it has been 
argued  that  there  is  still  room  for  improvement  in  the  implementation  of  these 
approaches.  

1.3.6.  Flow of research ideas and outcomes throughout the FP 

Another point of interest of the architecture of the Framework Programme is the ambition to 
‘transport’ research ideas throughout the programme parts. The underlying idea behind this 
is that distinct programme parts and pillars should not work as ‘silos’, but that research ideas 
and knowledge developed under one programme part should ideally be taken up to be further 
elaborated and applied in other programme parts.  

The survey765 conducted in the context of this study provides an indication of the share of 
Horizon Europe beneficiaries who indicated that their research project is a continuation of a 
research project funded under FP7 or Horizon 2020. The results show that INFRA and EIC 
Transition  projects  are  mainly  continuations  of  other  projects  carried  out  under  the 
Framework  Programmes.  MSCA  postdoctoral 
frequent 
continuations of prior projects.  

fellowships  are 

least 

the 

764 Survey of Horizon Europe beneficiaries, conducted May-July 2023. Surveys of Horizon 
Europe MSCA researchers, conducted in May-November 2023.  For more information, see 
Annex 7. 
765 Survey of Horizon Europe beneficiaries conducted in May-July 2023. Surveys of Horizon 
Europe MSCA researchers, conducted in May-November 2023.  For more information, see 
Annex 7.  

459 

 
 
80%

70%

60%

50%

40%

30%

20%

10%

0%

Yes, the project is a continuation of a research project funded under FP7 grants.

Yes, the project is a continuation of a research project funded under Horizon 2020.

Figure 6. Share of Horizon Europe beneficiaries who indicated that their research project is a continuation of a previous 
Framework Programme project. Source: survey of Horizon Europe beneficiaries conducted in May-July 2023. Surveys of 
Horizon Europe MSCA researchers, conducted in May-November 2023. For more information, see Annex 7. 

Note: respondents were allowed to choose multiple answers, which means that one respondent could have indicated that 
their project is a continuation of FP7 as well as of Horizon 2020. This means that these percentages cannot simply be added 
together.  

Note: MSCA organisations = MSCA organisations participating in DN, COFUND and SE actions; MSCA DN/ COFUND/SE 
res. = researchers participating in MSCA DN/COFUND/SE; WIDERA (widening) = beneficiaries of the widening actions 
under the WIDERA programme part; WIDERA (ERA) = beneficiaries of the ‘Strengthening the ERA’ component of the 
WIDERA programme part. 

While in general, it is not practically possible to follow and track research ideas using data, 
the trajectories of researchers through the Framework Programme(s) can be used as a proxy 
for this766. Thus, a series of analyses have been performed to provide preliminary insights 
into this issue. More information on the methodology used can be found in Annex  6 of this 
report.  

A first analysis767 shows in which Horizon Europe programme parts a given researcher 
tends to participate. Cluster 4768 and Cluster 5769 share the largest number of researchers 

766 While following researchers throughout the Framework Programmes provides a proxy for the 
path of research ideas, it should be emphasised that this only provides part of the story. This is 
because 1) ideas developed within a Framework Programme and are published may be taken 
up by other researchers (e.g. in other programme parts) without the active involvement of the 
'initial' researchers; and 2) fundamental researchers may simply no longer be actively involved 
once the research idea is transferred into more applied or higher TRL projects.  
767 More details and results on this analysis can be found in Annex 6.  
768 Digital, Industry and Space. 
769 Climate, Energy and Mobility. 

460 

 
 
 
out  of  all  Horizon  Europe  programme  parts  (a  total  of  1,717),  both  in  terms  of  absolute 
numbers and relative to the size of the programme parts770.  

In addition, the results of the analysis indicate that researchers under Horizon Europe 
Pillar  1  participate  more  often  in  projects  under  other  pillars  than  they  do  in  an 
additional  project  within  Pillar  1.  Specifically,  MSCA  and  INFRA  both  have  the  highest 
numbers of researchers shared with Cluster 4, in terms of absolute numbers of researchers 
(302 and 271, respectively). In relative terms, taking into account the number of participants 
in  each  programme  part,  INFRA  has  the  largest  share  of  researchers  in  common  with 
WIDERA, and MSCA still has the largest share of researchers in common with Cluster 4. In 
terms of absolute number of researchers, the ERC shares the highest number of researchers 
with Cluster 1 (although with 13 shared researchers, this number is quite low). Also relative 
to the size of the programme parts, the ERC has the biggest share of researchers in common 
with Cluster 1. 

A  second  analysis  traces  Horizon  2020  newcomers  and  their  participation  in  Horizon 
Europe over time. In particular, almost 1.7% of those researchers who were newcomers to 
the Horizon 2020 Framework Programme are (so far) already participating in Horizon Europe. 
Specifically, the following results are found with regard to the Horizon Europe programme 
parts in which they are so far participating: 

•  Horizon 2020 newcomers to the ERC most often participate in Horizon Europe MSCA, 

closely followed by Horizon Europe Cluster 4.  

•  Newcomers  to  the  MSCA  under  Horizon  2020  most  frequently  participate  in  Horizon 

Europe Cluster 4, followed by the MSCA.  

•  Horizon 2020 FET newcomers are now most active in Horizon Europe Cluster 4.  

•  Horizon 2020 INFRA newcomers most often participate in Horizon Europe INFRA. 

A third analysis, focusing on the trajectory of researchers over time across the three most 
recent  Framework  Programmes  (FP7,  Horizon  2020  and  Horizon  Europe),  reveals  that 
researchers  active  under  Pillar  1  of  Horizon  Europe  have  most  often  been  active  in  its 
thematic predecessors – Horizon 2020 Pillar 1, followed by FP7 Cooperation. More results 
are presented in Annex 6.  

Lastly, analysis of Horizon 2020 and Horizon Europe participation data provides insights into 
the trajectory of ERC researchers throughout the Framework Programmes. Specifically, this 
analysis began by identifying the PIs of Horizon 2020 ERC projects, and then identified in 
which projects they have participated under Horizon Europe. The results of this analysis are 
discussed in the paragraphs that follow.  

Out of the 7,972 ERC PIs under Horizon 2020, 609 (8%) also participated in Horizon Europe 
(all programme parts, not necessarily as coordinator). Together, they participated 739 times 
in Horizon Europe (some of them are thus participating in multiple Horizon Europe project 
teams).  

The Horizon Europe pillar that is the most popular destination for ERC PIs from Horizon 2020 
is  Pillar  2  (which  accounts  for  52%  of  all  participations  in  Horizon  Europe  by  PIs  who 

770 Here, the absolute numbers of shared researchers are divided by the sum of the total 
participants in each programme part, divided by two.  

461 

 
 
participated  in  ERC,  Horizon  2020).  This  is  followed  by  Pillar  1  (24%),  Pillar  3  (18%)  and 
WIDERA  (6%).  This  trend  is  similar  for  all  types  of  ERC  grants,  although  with  small 
differences. Notably, PIs from Proof of Concept grants (PoC) are slightly more likely to go 
towards Pillar 3 and less to Pillar 2, while PIs from Advanced Grants (AdG) show the opposite 
trend, going more towards Pillar 2 than to Pillar 3. This reflects the purposes of these different 
types of grants, with the PoC being more oriented towards valorisation and AdG focusing on 
established, leading PIs who are likely to maintain a focus on research activities. 

100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%

9%
7%

4%

18%

65%

50%

19%

AdG

27%

CoG

6%

30%

45%

19%

PoC

4%
14%

51%

30%

StG

9%

58%

30%

SyG

6%

18%

52%

24%

Total

Pillar 1

Pillar 2

Pillar 3 WIDERA

Figure 7. Share of participations per pillar of Horizon Europe by PIs from Horizon 2020 ERC projects, by type of ERC grant. 
Source: analysis of EC monitoring data. 

In Pillar 1 of Horizon Europe, the programme part that is the most frequent destination for 
ERC PIs from Horizon 2020 is MSCA (111 participations; 15% of all Horizon 2020 ERC PI 
participations  in  Horizon  Europe),  followed  by  INFRA  (34  participations;  5%).  When 
comparing the different types of ERC grants, it is notable that PIs from Synergy Grants (SyG) 
are more likely to go towards INFRA under Horizon Europe, while PIs from Starting Grants 
(StG) tend to go more often towards ERC Consolidator Grants (CoG) under Horizon Europe.  

100%

80%

60%

40%

20%

0%

8%

77%

12%

AdG

76%

21%

CoG

71%

7%

22%

PoC

46%

19%

35%

StG

44%

56%

SyG

PoC

MSCA

INFRA

CoG

AdG

Figure 8. Share of participations per programme part or type of ERC grant under Pillar 1 of Horizon Europe by PIs from 
Horizon 2020 ERC projects, by type of ERC grant. Source: analysis of EC monitoring data. 

In Pillar 2 of Horizon Europe, the programme part that is the most frequent destination for 
ERC PIs from Horizon 2020 is Cluster 4 ‘Digital, Industry and space’ (146 participations; 20% 
of all Horizon 2020 ERC PI participations in Horizon Europe), followed by Cluster 1 ‘Health’ 

462 

 
 
 
(74 participations; 10%). Similar trends can be seen when comparing the different types of 
ERC grants of the PIs under Horizon 2020. 

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%

38%

37%

21%

13%

15%

7%

AdG

19%

17%

11%

10%

CoG

44%

20%

8%

11%

15%

PoC

32%

32%

17%

20%

19%

6%

StG

19%

19%

13%

16%

SyG

38%

19%

15%

14%

10%

Total

CL4

CL1

CL5

CL6

Missions

CL2

CL3

Figure 9. Share of participations per programme part under Pillar 1 of Horizon Europe by PIs from Horizon 2020 ERC 
projects, by type of ERC grant. Note: CL1: Health; CL2: Culture, Creativity and Inclusive society; CL3: Civil security for 
society; CL4: Digital, Industry and Space; CL5: Climate, Energy and Mobility; CL6: Food, Bioeconomy, Natural Resources, 
Agriculture and Environment. Source: analysis of EC monitoring data. 

Almost all ERC PIs from Horizon 2020 who also participated in Horizon Europe under Pillar 
3 went to the European Innovation Council (EIC) (131 out of 132 participations, accounting 
for 18% of all Horizon 2020 ERC PI participations in Horizon Europe); one went to European 
Innovation Ecosystems (EIE). 

1.3.7.  EU added value 

The Framework Programmes for R&I have been greatly appreciated and are widely regarded 
as  critical  to  spurring  R&I  in  Europe.  While  the  Framework  Programmes  have  a  lot  of 
important benefits in terms of creating high-quality new knowledge, the most important factors 
providing EU added value are briefly summarised below.  

First,  it  is  widely  agreed  that  EU  Framework  Programme  funding  provides  unprecedented 
international collaboration opportunities in comparison to any other (international) funder. The 
Framework  Programmes  are  seen  as  being  among  the  only  programmes  that  allow  and 
encourage collaboration between excellent international partners from all over the world. This 
allows researchers to collaborate with the best possible partners in particular areas, enabling 
breakthroughs  in  research  that  would  not  have  been  possible  without  international 
collaboration. It is believed that this international openness should remain at the core of the 
Framework  Programme,  with  several  stakeholders  urging  for  the  Switzerland  to  be 
associated with the Framework Programme771.  

A  second  point,  related  to  the  first,  is  the  fact  that  the  EU  Framework  Programmes  allow 
networking  with  international  top  scientists  and  researchers  in  particular  fields,  hence 

771 European University Association (EUA) (2023). EUA input to the public consultation on the 
past, present and future of the European Research & Innovation Framework Programmes 2014-
2027. 

463 

 
 
 
enabling the exchange of knowledge, best practices, etc. and, as such, improving the critical 
mass for R&I in institutions in the EU.  

Third, on many occasions, interviewees argued that grants under the FPs are big enough to 
fund a sufficiently large and diverse team, thus enhancing the quality of the research. For 
instance, while national or regional funding sources can provide researchers with grants to 
fund  particular  research  projects,  Framework  Programme  funding  typically  allows  them  to 
engage larger teams, thereby providing opportunities for researchers from other disciplines 
to join and contribute to the research project.  

1.4.  Success factors and barriers 

Several factors can be identified that can either contribute to or hinder the creation of high-
quality knowledge generated under the Framework Programmes.  

The  wide  portfolio  of  instruments  available  enables  researchers  from  many  different 
organisations and fields, active in different parts of the R&I chain and at different TRLs, to be 
accommodated  by  its  various  programme  parts  or  instruments,  whether  they  work  on 
curiosity-driven  or  challenge-oriented  research.  The  findings  of  the  present  case  study 
underline the importance of offering a balance between different types of instruments 
(e.g. collaborative research versus mono-beneficiary grants), since all of these serve different 
needs  and  possess  the  ability  to  produce  high-quality  and  highly  impactful  research. 
Nevertheless, different views exist as to what the correct balance should be. For instance, 
interviewees  and  positioning  papers  submitted  to  the  OPC772  argued  that  the  Framework 
Programmes  (and  EU  research 
from  more  bottom-up 
collaborative actions.  

in  general)  could  benefit 

Some  interviewees  also  devoted  attention  to  programme  architecture  and  portfolio 
management  and  commented  on  the  ‘ideal’  flow  of  ideas  and  projects  throughout  the 
Framework Programme. In this context, one interviewee mentioned that there is a ‘short-cut’ 
from  ERC  projects  directly  to  the  EIC  for  subsequent  funding,  without  those  researchers 
passing through Pillar 2. This may sometimes be undesirable, as it is still important to work 
on basic applied research by means of low-TRL collaborative projects in Pillar 2.  

At the same time, interviewees and stakeholders have also argued that the landscape of the 
Framework  Programme  has  become  complex,  making  it  difficult  for  researchers  to 
understand the full scope of its possibilities773. At the same time, proposal and project 
administration,  especially  in  case  of  large  collaborative  projects,  often  involves 
considerable administrative burdens and inefficiencies. For instance, large collaborative 
projects  are typically  highly resource-intensive  in  terms  of  coordination  and  administration 
during the course of proposal preparation, as well as in project execution. In addition, large 
collaborative  projects  in  particular  areas  sometimes  lead  to  the  duplication  of  activities  by 
projects funded under the same calls.  

In this regard, many stakeholders call for simplification, which could be aided by the further 
harmonisation of procedures, application forms, reporting, etc. across the Framework 
Programme  However,  interviewees  and  stakeholders  also  stressed  that  making  project 

772 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 
773 Coimbra Group. (2023). Position Paper: Past, present, and future of the European Research 
& Innovation Framework Programme 2014-2027. 

464 

 
 
application  and  execution  more  standardised  than  it  currently  is  could  bring  unwanted 
negative  effects.  Criticisms  were  expressed  that  some  researchers  or  organisations  have 
become routinised in writing proposals and that they ‘know what is expected of a proposal’ 
compared with others who have less experience in writing these kinds of applications. Larger 
organisations also have access to more dedicated resources and experience to support this 
process. In this regard, it was stressed by some that it is undesirable that applicants need to 
engage (expensive) experts in order to meet the requirements and specificities of proposal 
writing and reporting. This is important, because such costs could dissuade or hinder certain 
researchers or institutions from applying in the first place.  

Some  interviewees  and  stakeholders  also  called  for  the  recognition  of  the  importance  of 
curiosity-driven  research,  in  order  for  the  EU  to  remain  at  the  forefront  of  worldwide 
research, develop emerging fields and discover novel ideas that will determine the emerging 
fields of the future. It was emphasised that scientists do not work in ‘ivory towers’, as was 
sometimes argued in the past, but that researchers largely aim to find solutions to societal 
challenges  and  thereby  achieve  societal  impacts  and  relevance.  Interviewees  and 
stakeholders  largely  understand  and  value  the  fact  that  society  and  policy-makers 
emphasise  and  prioritise  particular  societal  issues  or  themes.  Nevertheless,  it  was 
underlined that such top-down opportunities should not be too prescriptive and should still 
leave  sufficient  room  for  the  creativity  of  researchers  in  identifying  and  working  on  truly 
innovative and novel pathways to address societal issues. It must, however, be noted that 
some of the successful Pillar 2 beneficiaries interviewed in the context of this study did not 
feel that the calls were too prescriptive. In connection with a similar issue, beneficiaries who 
were interviewed also disclosed that the requirement to define specific results milestones 
upfront  for  ‘blue-sky’  research  can  hinder  the  development  of  high-risk,  breakthrough 
projects or the redirection of research when the project results are not as originally expected.  

Interdisciplinarity was often underlined as an important factor in producing high-quality and 
impactful  research.  Specifically,  it  was  frequently  argued  that  interdisciplinarity  is  the 
preferred  pathway  to  address  complex  societal  and  grand  challenges,  and  contributes  to 
impactful and high-quality knowledge creation. At the same time, beneficiaries indicated that 
interdisciplinarity  should  not  be  considered  an  absolute  must,  but  should  be  promoted  as 
much as possible to the extent that it is relevant in addressing specific research avenues. 

While this case study focuses on the impact of the full Framework Programmes in creating 
high-quality new knowledge, as well as on the drivers and barriers to this, the evidence also 
points to a number of success factors and barriers in relation to certain specific programme 
parts aimed at producing high-quality new knowledge.  

•  Pillar 1, Excellent Science  

–  Evidence gathered from interviewees and stakeholders underlines that the ERC 
is  highly  successful  in  creating  high-quality  knowledge  in  Europe,  thanks  to 
several  factors.  Specifically,  the  set-up  of  this  programme  part  builds 
entirely on its main objective of supporting excellent research in Europe. 
In doing so, the ERC succeeds in attracting excellent researchers. This fact 
in itself results in a positive impact on the extent to which novel and high-quality 
knowledge  is  created  through  this  programme  part.  The  elements  that  make 
ERC grants particularly attractive to (excellent) researchers are: 

▪ 

Its focus on excellence as the sole criterion, which has contributed 
to the strong reputation of the programme part and, consequently, to 
the  prestige  attached  to  winning  an  ERC  grant  within  the  academic 

465 

 
research community. One of the main reasons for early-career stage 
researchers  to  apply  for  an  ERC  grant  is  the  positive  effects  that 
winning  such  a  grant  is  expected  to  have  on  their  research  careers. 
According to PIs, projects funded through an ERC grant are attractive 
to other researchers, which facilitates PIs in hiring high-quality PhD 
candidates  or  post-doctoral  researchers,  thus  contributing  to  the 
quality of the research performed by the team. 

▪  The  bottom-up  approach  is  strongly  appreciated  by  academic 
researchers  who  want  to  work  on  novel  topics  or  fields  and  who 
appreciate the room this provides for the researchers’ creativity and/or 
for the redirection of research paths that have not led to the expected 
results.  This  feature  thus  stimulates  the  creation  of  novel  knowledge 
and/or the development of emerging fields. 

–  Although it is considered a basic element of the excellence-driven reputation of 
ERC grants, the peer-based evaluation process is sometimes felt to  lack the 
necessary  openness  towards  multidisciplinary  research  or  to  ideas  that 
deviate from the norm in a certain discipline, which sometimes hinders a PI in 
finding the best path for the submission of truly novel ideas. 

–  The  size  of  the  grant  is  considered  a  success  factor,  as  it  allows  for  a 
sufficiently  long  research  period  in  order  to  realise  ambitious  research 
goals, as well as the building of a sufficiently large research team. The latter 
also allows the PI to combine different types of expertise into one research 
team.  In  situations  where  only  one  researcher  can  be  hired  on  a  research 
project, that researcher is likely to be selected from within the core discipline of 
the project; in situations where more  researchers  can be hired on a research 
project  (as  with  ERC  grants),  a  combination  of  complementary  or  more 
specialised expertise can be sought out. This also contributes to both the novelty 
and broader relevance of the research. 

–  The  MSCA  contribute  significantly  to  the  mobility  of  researchers  across 
countries  and  disciplines,  which  is  indispensable  to  spurring  high-quality 
research, new collaborations, (informal) exchange, knowledge sharing, etc.  

–  University stakeholders typically indicate that the funding for the ERC and the 
MSCA should be protected, and that the total budget for the actions (as well 
as the maximum contribution per ERC grant) should be increased774 – perhaps 
raising the success rates of the actions). 

–  Several  interviewees  acknowledged  that  the  FET  programme  under  Horizon 
2020  was  highly  appreciated  by  researchers,  as  it  provided  opportunities  for 
collaborative research in basic science.  

•  Pillar 2, Global Challenges. Several success factors and barriers have been identified 
with  regard  to  Pillar  2  that  (partly)  influence  the  creation  of  impactful  and  high-quality 
knowledge.  

774 e.g. League of European Research Universities (LERU). (2023). LERU key messages for the 
current and future European R&I Framework Programme.  

466 

 
 
–  As  already  outlined  above,  many  stakeholders  and  interviewees  indicate  the 
need to rebalance Pillar 2. It has been underlined that lower TRLs are less well 
covered  in  terms  of  collaborative  applied  research  in  the  clusters775.  Many 
stakeholders call to rebalance the proportions of calls in terms of lower, medium 
and higher TRLs for the collaborative call topics within the clusters776,777. Related 
to this, other stakeholders have recommended adding research actions (RAs) 
into Pillar 2 to achieve a more balanced participation of knowledge-driven and 
solution-oriented  research,  in  addition  to  demonstration  and  innovation 
actions778.  This  would  allow  a  continuous  flow  between  advancement  of 
knowledge and technologies at different TRLs.  

–  Similarly,  as  mentioned  under  Section  1.3.7  in  relation  to  the  ERC,  cluster 
beneficiaries typically emphasised the main success factors of the projects 
as being the international  openness of this FP programme part to excellent 
partners from around the world as collaborators, as well as the amounts of the 
funding available.  

–  Several stakeholders indicated that it is so far unclear whether the EU Missions 
will provide good results, and whether they will be able to deliver added value in 
comparison  to  other  existing  instruments,  such  as  the  Clusters  or  the  EU 
partnerships779.  It  has  been  underlined  that  the  objectives  of  the  Missions 
should  be  better  conveyed780.  One  stakeholder  indicated  that  the  research 
and innovation dimension of the Missions will be limited, since the calls are very 
high up the TRL scale, while they would expect that disruptive research would 
be part of any Mission781.  

•  Pillar 3, Innovative  Leadership. The evidence collected for Pillar 3 does not provide 
many insights into the creation in new knowledge, and focuses more on innovation and 
market translation.  

– 

It  is,  however,  argued  that  there  should  be  better  links  between  the  EIC 
instruments and the RD&I carried out under Pillar 2 clusters.  

775 Among others, the European Association of Research and Technology Organisations 
(EARTO) (2023). EARTO Answer to the EC Consultation on the past, present and future of the 
European Research & Innovation Framework Programmes 2014-2027. 
776 European University Association (EUA) (2023). EUA input to the public consultation on the 
past, present and future of the European Research & Innovation Framework Programmes 2014-
2027. 
777 Guild of European Research-Intensive Universities. (2023). Lessons learned and way 
forward: the Guild’s recommendations for Horizon Europe. 
778 e.g. the Initiative for Science in Europe (ISE) (2023). ISE response to the European 
Commission public consultation on the past, present and future of the European Research & 
Innovation Framework Programmes 2014-2027. 
779 Among others, the European Association of Research and Technology Organisations 
(EARTO) (2023). EARTO Answer to the EC Consultation on the past, present and future of the 
European Research & Innovation Framework Programmes 2014-2027. 
780 Initiative for Science in Europe (ISE) (2023). ISE response to the European Commission 
public consultation on the past, present and future of the European Research & Innovation 
Framework Programmes 2014-2027. 
781 League of European Research Universities (LERU) (2023). LERU key messages for the 
current and future European R&I Framework Programme. 

467 

 
 
–  Another concern raised by one stakeholder is that universities’ interest in joining 
the activities of the EIT are decreasing considerably due to the administrative 
load and financial reporting requirements782.  

1.5.  Conclusions and lessons learned across the evaluation criteria of 

interest to the present case study 

This case study focuses on the impact of the Framework Programmes on the production of 
high-quality research. To do so, it builds on a wide variety of data sources, including desk 
research,  interviews,  survey  results,  network  analysis,  and  bibliometric  analysis.  Given  its 
focus on the creation of research outputs, the case study focuses mainly on Horizon 2020, 
but  makes  connections  to  Horizon  Europe  to  the  extent  possible  and,  where  possible, 
provides preliminary evidence in relation to this.  

The  case  study  indicates  that  the  production  of  excellent  knowledge  is  an  important 
feature  of  the  Excellent  Science  pillar,  but  that  many  other  programme  parts  and 
components  also  contribute  to  the  creation  of  a  vast  set  of  new  and  high-quality 
knowledge  in  Europe.  In  relation  to  this,  it  is  perceived  as  critically  important  to  achieve 
balance  between  the  different  types  of  instruments  in  the  Framework  Programme  (i.e. 
collaborative research versus mono-beneficiary grants, ‘bottom-up’ versus ‘top-down’ calls, 
coverage of different TRLs) since all of these serve different needs, and all possess the ability 
to produce high-quality and highly impactful research.  

The  case  study  also  provides  insights  into  the  disciplines  and  topics  covered  in  the 
Framework Programmes, for both Horizon 2020 and Horizon Europe. Compared with Horizon 
2020, Horizon Europe’s focus on projects covering SDGs has intensified. In terms of 
fields of science, the Excellent Science pillar of Horizon Europe supports many projects in 
the field of Natural Sciences, the clusters in Pillar 2 accommodate projects in those fields of 
science in line with their focus, and the widening actions mainly focus on projects in Natural 
Sciences. In addition, the bibliometric analysis on scientific publications generated by Horizon 
2020 indicated that in Pillar 1, most scientific publications were in the scientific discipline of 
Physics  and  Astronomy,  followed  by  Engineering.  In  Pillar  2,  Industrial  Leadership,  most 
publications were produced in the scientific discipline of Computer Science and Engineering. 
Lastly, most papers in Societal Challenges were in the scientific discipline of Medicine.  

Another  analysis  performed  on  the  Horizon  2020  research  outputs  indicated  the  most 
prominent  topics  in  terms  of  momentum  and  visibility, and  elucidated  the  fact  that these 
topics often occur in scientific publications generated in different programme parts. In 
addition, these projects operated mainly in line with the aims of the Framework Programme: 
the  Excellent  Science  set  of  activities  aims  to  reinforce  and  extend  the  excellence  of  the 
Union's science base, while the Societal Challenges activities cover  the full cycle including 
basic  and  fundamental  research,  but  also  applied  research,  knowledge  transfer  and 
innovation, with a focus on policy priorities and the aim of stimulating the critical mass of R&I 
efforts required to achieve the Union’s policy goals. Programme participants typically select 
to which programme part they wish to apply to on the basis of their need or desire for mono-
beneficiary  or  collaborative  research;  the  opportunities  they  see  at  first  glance  for  their 
particular research topic; and the fit of the programme part with their network.  

782 League of European Research Universities (LERU). (2023). LERU key messages for the 
current and future European R&I Framework Programme. 

468 

 
 
In a global context, the EU continues to assert its position as a powerhouse in the realm of 
science783. An analysis of the most prominent topics worldwide (using the proxy of research 
produced by international knowledge providers from 32 countries) shows that the EU-27 and 
China have a number of prominent topics in which they are very strong contributors in the 
total  production  of  knowledge.  Conversely,  the  US  appears  to  specialise  in  fewer  specific 
prominent topics, but contributes fairly consistently to the bulk of the prominent topics, with 
up to 50% of the total production of publications. The topic in  which the EU-27 makes the 
most significant contribution is ‘Boson; Partons; Higgs Bosons’ in the field of physics, with 
76% of all publications being attributed to the EU-27. However, FP publications represent a 
share of only 1.4% of all EU-27 publications on this topic. The highest FP contribution is found 
in the topic ‘Planet; T Tauri Star; Pre-main Sequence Stars’, where Horizon 2020 publications 
make  up  5.0%  of  the  total  number  of  publications  generated  by  the  main  international 
knowledge producers. 

Success factors and barriers to the creation of high-quality knowledge in the Framework 
Programme were identified, mainly on the basis of interviews and the views expressed by 
stakeholders. These include the importance of having a wide portfolio of instruments – and 
within  this  portfolio,  the  need  to  offer  a  balance  between  different  types  of  instruments 
(including  a  balance  between  curiosity-driven  and  top-down  research).  In  this  regard,  it  is 
sometimes  mentioned  that  the  Framework  Programme  and  EU  research  in  general  could 
benefit  from  more  bottom-up  collaborative  actions.  At  the  same  time,  beneficiaries  and 
stakeholders  mentioned  that  the  landscape  of  the  Framework  Programme  has  become 
complex, making it difficult for researchers to understand the full scope of the opportunities 
available. In addition, the administration of proposals and projects, particularly in case of large 
collaborative  projects,  often  entails  significant  administrative  burdens  and  inefficiencies. 
Lastly, interdisciplinarity was mentioned as an important pathway to address complex societal 
and grand challenges and to contribute to impactful and high-quality knowledge creation.  

Appendix 1. Literature sources 

•  Coimbra Group (2023). Position Paper: Past, present, and future of the European 

Research & Innovation Framework Programme 2014-2027. 

•  Commission presents its evaluation of the 7th Framework Programme for Research 
(2016). https://ec.europa.eu/commission/presscorner/detail/de/MEMO_16_146.  

•  Cordis websites of selected projects. 

•  Duerr, F., & Herkommer, A. (2019). Why does interdisciplinary research matter? 
Advanced Optical Technologies, 8(2). https://doi.org/10.1515/aot-2019-0026. 

•  ERC (2022). ERC reveals mapping of its funded research. https://erc.europa.eu/news-
events/news/erc-reveals-mapping-its-funded-research, accessed 30 August 2023.  

•  European Association of Research and Technology Organisations (EARTO) (2023). 

EARTO Answer to the EC Consultation on the past, present and future of the European 
Research & Innovation Framework Programmes 2014-2027. 

783 European Commission, Directorate-General for Research and Innovation, Science, research 
and innovation performance of the EU 2022: building a sustainable future in uncertain times, 
Publications Office of the European Union, 2022, https://data.europa.eu/doi/10.2777/78826. 

469 

 
 
•  European Commission Decision C(2022)2975. Horizon Europe. Work Programme 

2021-2022. 12. Missions. https://ec.europa.eu/info/funding-
tenders/opportunities/docs/2021-2027/horizon/wp-call/2021-2022/wp-12-
missions_horizon-2021-2022_en.pdf.  

•  European Commission, Directorate-General for Research and Innovation, A new 
horizon for Europe: impact assessment of the 9th EU Framework Programme for 
Research and Innovation, Luxembourg: Publications Office of the European Union, 
2018, available at: https://data.europa.eu/doi/10.2777/194210.  

•  European Commission, Directorate-General for Research and Innovation, Nixon, J., 
Study to support the monitoring and evaluation of the Framework Programme for 
Research and Innovation along key impact pathways – Indicator methodology and 
metadata handbook, Nixon, J.(editor), Luxembourg: Publications Office of the 
European Union, 2022, available at: https://data.europa.eu/doi/10.2777/44653.  

•  European Commission, Directorate-General for Research and Innovation, Horizon 

Europe – Strategic plan 2021-2024, Luxembourg: Publications Office of the European 
Union, 2021, available at: https://data.europa.eu/doi/10.2777/083753. 

•  European Commission, Directorate-General for Research and Innovation, Science, 

research and innovation performance of the EU 2022 : building a sustainable future in 
uncertain times, Luxembourg: Publications Office of the European Union, 2022, 
available at: https://data.europa.eu/doi/10.2777/78826. 

•  European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European Framework 
Programmes for Research and Innovation for excellent science: Horizon 2020: 
annexes, Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383. 

•  European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R. et al., Evaluation study of the European Framework 
Programmes for Research and Innovation for excellent science: Horizon 2020: phase 1 
final study report, Luxembourg: Publications Office of the European Union, 2023, 
available at: https://data.europa.eu/doi/10.2777/967813.  

•  European Commission, Directorate-General for Research and Innovation, Naujokaitytė, 
R., Stančiauskas, V., Cakić, M., et al., Evaluation study of the European framework 
programmes for research and innovation for an innovative Europe – Final report – 
Phase 1, Denham, S. (editor), Luxembourg: Publications Office of the European Union, 
2023, available at: https://data.europa.eu/doi/10.2777/144504. 

•  European Commission, Directorate-General for Research and Innovation, Naujokaitytė, 

R., Mačiulienė, M., Stančiauskas, V., et al., Evaluation study on the implementation of 
cross-cutting issues in Horizon 2020 : study in support of the ex-post evaluation of the 
European Framework Programme for Research and Innovation Horizon 2020. Annex 3, 
Case study report, Denham, S. (editor), Luxembourg: Publications Office of the 
European Union, 2023, available at: https://data.europa.eu/doi/10.2777/331671. 

•  European Science Vocabulary (EuroSciVoc), available at: 

https://op.europa.eu/en/web/eu-vocabularies/euroscivoc, accessed 30 August 2023. 

470 

 
•  European University Association (EUA) (2023). EUA input to the public consultation on 

the past, present and future of the European Research & Innovation Framework 
Programmes 2014-2027. 

•  Guild of European Research-Intensive Universities (2023). Lessons learned and way 

forward: the Guild’s recommendations for Horizon Europe. https://www.the-
guild.eu/publications/position-papers/lessons-learned-and-way-forward-the-guild-s-
recommendations-for-horizon-europe.pdf. 

• 

• 

Initiative for Science in Europe (ISE) (2023). ISE response to the European 
Commission public consultation on the past, present and future of the European 
Research & Innovation Framework Programmes 2014-2027. 

League of European Research Universities (LERU) (2023). LERU key messages for 
the current and future European R&I Framework Programme. 

•  Olsen, R. (2020). Europe needs blue sky research more than ever, available at: 
https://www.coimbra-group.eu/europe-needs-blue-sky-research-more-than-ever, 
accessed on 30 August 2023. 

•  Open Public Consultation on the Past, Present and Future of The European Research 

and Innovation Programmes (2014-2027), available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-
consultation-future-eu-research-and-innovation-programmes-are-now-public-2023-04-
19_en. 

•  Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 

2021. Establishing Horizon Europe – the Framework Programme for Research and 
Innovation, laying down its rules for participation and dissemination, and repealing 
Regulations (EU) No 1290/2013 and (EU) No 1291/2013. 

•  Regulation (EU) No 1291/2013 of the European Parliament and of the Council of 11 

December 2013 establishing Horizon 2020 – the Framework Programme for Research 
and Innovation (2014-2020) and repealing Decision No 1982/2006/EC. 

•  Science Europe (2023). Science Europe Response to the Public Consultation on 

Horizon 2020 and Horizon Europe. DOI: 10.5281/zenodo.7664436.  

•  SWD (2023)132 final. (2023). Evidence Framework on monitoring and evaluation of 

Horizon Europe. Brussels, 27.4.2023. 

•  Technical Research Centre of Finland (VTT) (2023). VTT Technical Research Centre 
of Finland’s input to the European Commission consultation on the future of the EU 
Research and Innovation Framework Programmes.  

471 

 
 
 
Appendix 2. Intervention logics of Horizon 2020 and Horizon Europe 

Figure 10. Intervention logic of Horizon 2020. Source: based on the interim evaluation of Horizon 2020, Annex II, available at: https://op.europa.eu/en/publication-detail/-/publication/4cedc9a0-
d8cc-11e8-afb3-01aa75ed71a1/language-en 

472 

 
 
 
Figure 11. Intervention logic of Horizon Europe. Source: A new horizon for Europe: impact assessment of the 9th EU Framework Programme for Research and Innovation. 

473 

 
 
 
 
Appendix 3. Fields of science in the programme parts of the Framework Programmes 

Excellent Science 

Innovative Leadership 

Societal Challenges 

SwafS  SEWP 

C
R
E

A
C
S
M

T
E
F

A
R
F
N

I

i

h
c
e
t
o
B
s
T
E
L

I

g
n
i
r
u
t
c
a
f
u
n
a
M
s
T
E
L

I

s
E
M
S
n

i

n
o
i
t
a
v
o
n
n
I

h
t
l
a
e
h

1
C
S

d
o
o
f

2
C
S

y
g
r
e
n
e

e
r
u
c
e
s

3
C
S

t
r
o
p
s
n
a
r
t

t
r
a
m
s

4
C
S

e
t
a
m

i
l

c

5
C
S

T
C

I

I

s
T
E
L

o
n
a
N
s
T
E
L

I

e
c
a
p
S
s
T
E
L

I

l

s
a
i
r
e
t
a
M
s
T
E
L

I

i

e
v
s
u
c
n

l

i

6
C
S

i

s
e
i
t
e
c
o
s
e
r
u
c
e
s

7
C
S

Natural Sciences 

77%  72%  81%  73%  75%  66%  65%  75%  61%  63%  43%  38%  64%  47%  56%  67%  45%  71% 

32% 

61% 

Medical and Health 
Sciences 
Engineering and 
Technology 

31%  29%  26%  14%  48%  12%  6%  15%  32%  3%  21%  69%  26%  2% 

3% 

7% 

5% 

8% 

7% 

30% 

26%  28%  58%  27%  52%  82%  83%  56%  80%  75%  36%  17%  51%  87%  79%  67%  17%  42% 

9% 

36% 

Social Sciences 

20%  25%  16%  39%  24%  29%  43%  48%  42%  38%  37%  26%  48%  43%  46%  53%  84%  65% 

65% 

36% 

Humanities 

11%  14%  3%  12%  1% 

3% 

1% 

7% 

3% 

1% 

2% 

2% 

1% 

2% 

2% 

2%  16%  5% 

10% 

9% 

Agricultural Sciences 

3% 

6% 

2% 

3%  24%  2% 

5% 

3% 

7%  10%  7% 

1%  65%  9% 

1%  17%  2% 

0% 

2% 

10% 

Table 5. Signed grants under Horizon 2020, by programme part and field of science. 

Note: percentages represent the share of signed grants within each programme part that are allocated to a field of science. The sum of the percentages per programme part may exceed 100% 
because in some cases, more than one field of science is allocated to a given signed grant. 

Note: these data show that the labelling of ‘field of science’ is not necessarily attributed consistently to each signed grant, hence the data should be interpreted with caution. 

Note: fields of science refer to their EuroSciVoc Level 1 classification, as presented in the Horizon Dashboard. 

Source: Horizon Dashboard784 

784 https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/horizon-dashboard, accessed August 2023. 

474 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Excellent Science 

Global Challenges and European Industrial Competitiveness  

Innovative Europe 

WIDERA 

h
t
l
a
e
H

:
1
r
e
t
s
u
C

l

,
e
r
u
t
l
u
C

:
2
r
e
t
s
u
C

l

C
R
E

A
C
S
M

A
R
F
N

I

i

e
v
s
u
c
n
I

l

d
n
a
y
t
i
v
i
t
a
e
r
C

i

y
t
e
c
o
s

Natural sciences 

54% 

37%  65% 

42% 

31% 

Engineering and technology 

17% 

16%  18% 

16% 

6% 

Social sciences 

15% 

14%  31% 

17% 

79% 

Medical and health sciences 

21% 

13%  16% 

53% 

Humanities 

Agricultural sciences 

8% 

2% 

7% 

6% 

0% 

3% 

6% 

0% 

0% 

10% 

16% 

y
t
i
r
u
c
e
s

l
i

i

v
C

:
3
r
e
t
s
u
C

l

i

y
t
e
c
o
s

r
o
f

51% 

14% 

39% 

8% 

2% 

1% 

y
r
t
s
u
d
n
I

,
l
a
t
i
g
D

i

:
4
r
e
t
s
u
C

l

e
c
a
p
S
d
n
a

66% 

63% 

35% 

6% 

2% 

6% 

,
e
r
u
t
l
u
c
i
r
g
A

,
y
m
o
n
o
c
e
o
B

i

,
d
o
o
F

:
6
r
e
t
s
u
C

l

…

54% 

24% 

50% 

8% 

1% 

32% 

n
o
i
t
a
v
o
n
n

i

n
a
e
p
o
r
u
E

s
m
e
t
s
y
s
o
c
e

C
E

I

I

T
E

i

g
n
c
n
a
h
n
e

d
n
a

i

g
n
m
r
o
f
e
R

m
e
t
s
y
s

I

&
R
n
a
e
p
o
r
u
E

i

n
o
i
t
a
p
c
i
t
r
a
p
g
n
n
e
d
W

i

i

e
c
n
e

l
l

e
c
x
e
g
n
d
a
e
r
p
s

i

d
n
a

37% 

46% 

6% 

13% 

35% 

24% 

38% 

0% 

0% 

19% 

31% 

12% 

6% 

16% 

12% 

20% 

31% 

2% 

6% 

1% 

5% 

0% 

0% 

0% 

4% 

2% 

2% 

14% 

2% 

5% 

y
t
i
l
i

b
o
M
d
n
a

y
g
r
e
n
E

,
e
t
a
m

i
l

C

:
5
r
e
t
s
u
C

l

39% 

49% 

32% 

1% 

1% 

4% 

Table 6. Signed grants under Horizon Europe, by programme part and field of science 

Note: percentages represent the share of signed grants within each programme part that are allocated to a field of science. The sum of the percentages per programme part may exceed 100% 
because in some cases, as more than one field of science may be allocated to a given signed grant. 

Note: these data show that the labelling of ‘field of science’ is not necessarily attributed consistently to each signed grant, hence the data should be interpreted with caution. 

Note: fields of science refer to their EuroSciVoc Level 1 classification, as presented in the Horizon Dashboard. 

Source: Horizon Dashboard785. 

785 https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/horizon-dashboard, accessed August 2023. 

475 

 
  
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Annex 2.10: Strengthening Human Capital in Research and 
Innovation  

1.1. 

Introduction 

Horizon  Europe,  like  its  predecessor  Framework  Programmes  (FPs),  seeks  to  strengthen 
human  capital  in  research  and  innovation  (R&I)  in  a  number  of  ways,  through  specific 
programmes.  The  funding  opportunities  provided  by  Horizon  Europe  are  expected  to 
contribute not only to the advancement of the skills, knowledge and experience possessed 
by individual researchers, but also to foster the creation of new collaborations that have a 
strong structuring effect on the European research landscape. The Framework Programme 
also aims to attract excellent researchers to Europe by offering attractive research mobility 
opportunities,  as  well  as  exposure  to  the  non-academic  sector  in  order  to  facilitate  the 
diffusion  and  circulation  of  knowledge  beyond  national  borders  and  outside  the  setting  of 
academic research training.  

The strengthening of human capital in R&I is one of the key pathway indicators to assess the 
overall  performance  of  Horizon  Europe  and  to  identify  its  short,  medium,  and  long-term 
scientific impacts. The present case study contributes to capturing and communicating the 
impact on researchers’ mobility and skills across the FP. Specific focus is given to programme 
parts  under  the  Excellent  Science  pillar,  such  as  the  European  Research  Council  (ERC), 
Marie Skłodowska-Curie Actions (MSCA), Research Infrastructures (INFRA) and widening 
participation and strengthening the European Research Area (WIDERA), without neglecting 
the  potential  impacts  of  other  framework  programme  parts.  This  case  study  addressed 
five  evaluation  criteria:  relevance,  coherence,  efficiency, 
in  relation 
questions 
effectiveness, and EU added value. 

to 

1.2.  Background information 

1.2.1.  Policy context 

To  better  capture  and  communicate  the  impacts  of  the  FP,  the  European  Commission’s 
proposal for Horizon Europe included the Key Impact Pathways (KIPs). This approach aligns 
with an enhanced ambition to increase the diversity of the impacts of EU R&I funding. KIPs 
consist of three different categories of impacts – scientific, economic and societal786. In terms 
of the scientific impacts, the European Commission aims to capture and evaluate progress 
in  achieving  the  following  Horizon  Europe  objectives  (key  scientific  impact  pathway 
indicators):  

•  Creating high-quality new knowledge (KIP1). 

•  Strengthening human capital in research and innovation (KIP2). 

•  Fostering the diffusion of knowledge and Open Science (KIP3). 

786 Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 2021 
establishing Horizon Europe – the Framework Programme for Research and Innovation, laying 
down its rules for participation and dissemination, and repealing Regulations (EU) No 1290/2013 
and (EU) No 1291/2013 (Text with EEA relevance). 

476 

 
 
Human capital is an essential part of fostering R&I, and the ability to identify and increase the 
competitive advantage of researchers’ competences is of key strategic importance in creating 
a more connected and efficient research and innovation ecosystem. In accordance with KIP2, 
Horizon Europe aims to strengthen human capital in research and innovation by improving 
the  skills,  careers  and  working  conditions  of  participating  researchers  (short,  medium  and 
long-term impacts)787. In the context of Horizon Europe, a major novelty of its KIP framework 
is  that  human  research  capacity  within  the  FP  is  monitored  at  the  level  of  individual 
researchers. Previously, the indicator of human capital in R&I was mostly analysed within the 
scope of individual research grants788. 

1.2.2.  Human capital under Horizon 2020 

The  evaluation  study  on  Excellent  Science  in  the  European  Framework  Programmes  for 
Research  and  Innovation  provides  specific  insights  into  how  Horizon  2020,  the  eighth 
Framework  Programme  for  Research  and  Innovation,  aimed  to  strengthen  human  capital. 
The  evaluation  found  that  participation  in  Horizon  2020  is  associated  with  positive 
effects  in  terms  of  career  prospects,  particularly  for  researchers  during  earlier  career 
stages, such as MSCA fellows and the recipients of ERC Starting and Consolidator grants, 
as well as the more junior members of teams supported by SEWP Programme789. ERC grant 
beneficiaries have stated that improving their career prospects was an important motivating 
factor  to  apply  for  ERC,  with  93%  of  respondents  to  a  survey  of  Horizon  2020  ERC 
beneficiaries  indicating  that  their  ERC  project  had  a  good  or  very  good  impact  on  their 
research career prospects790. Meanwhile, in a survey of MSCA fellows, 87% of respondents 
stated that their fellowship had a ‘high’ or ‘somewhat high’ impact on career progression (with 
shares  of  90% among  IF, ITN and  COFUND fellows,  respectively,  while  the  share among 
RISE fellows was 77%)791. 

Horizon  2020  also  significantly  strengthened  researchers’  skills  and  international 
mobility.  With  regard  to  international  mobility,  the  MSCA  are  the  largest  worldwide 
international researcher mobility programme, supporting approximately 49,475 fellows under 
Horizon  2020792.  The  MSCA  are  expected  to  reach  their  target  of  65,000  researchers 
supported  once  all  funded  projects  have  come  to  an  end.  The  programme  has  also 
contributed  substantially  to  the  return  mobility  of  fellows.  This  trend  is  particularly  notable 
when looking at the mobility patterns of experienced researchers participating in Individual 
Fellowships (IF) and COFUND793.  

Horizon 2020 has also contributed to fostering Europe’s capacity to attract world-class 
talent.  For  example,  the  MSCA  have  successfully  supported  researchers  from  160 

787 Ibid.  
788 European Commission, Directorate-General for Research and Innovation, Nixon, J., Study to 
support the monitoring and evaluation of the framework programme for research and innovation 
along key impact pathways: indicator methodology and metadata handbook, Nixon, J. (editor), 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/44653  
789 European Commission, Evaluation study on Excellent Science in the European Framework 
Programmes for Research and Innovation. Phase 1 Final Study Report, 2023. 
790 Survey of Horizon 2020 ERC beneficiaries, conducted August-September 2022. 
791 Survey of MSCA fellows (IF, ITN, COFUND, RISE), conducted for the Study on mobility flows 
in the context of the MSCA, 2022. 
792 Cut-off date January 2023. 
793 European Commission, Evaluation study on Excellent Science in the European Framework 
Programmes for Research and Innovation. Phase 1 Final Study Report, 2023. 

477 

 
 
nationalities (40% of all researchers involved are nationals of non-EU countries)794. The ERC 
programme has also witnessed high levels of cross-border mobility: half of Horizon 2020 ERC 
team  members  were  non-nationals  of  the  country  in  which  the  project  was  based795. 
Research  Infrastructures  (RI)  also  contributed  to  making  Europe  more  attractive  to  world-
class  researchers  from  abroad,  mainly  through  INFRASUPP  projects,  but  also  by  funding 
transnational and virtual access (TNA/VA) for researchers from abroad796.  

The  evaluation  study  also  showed  that  Horizon  2020  contributed  to  the  diversification 
and improvement of researchers’ skills and knowledge, which was particularly evident in 
the cases of the MSCA, INFRA, and SwafS (Science with and for Society). The analysis of 
the MSCA indicated a strong impact on the quality of training and supervision provided, with 
a high level of satisfaction across all actions (around 90% of IF and RISE fellows considered 
training  and  supervision  to  be  ‘good’  or ‘very  good’,  while  the  share of  COFUND and  ITN 
fellows who considered them to be ‘good’ or ‘very good’ was 80%)797. Training was also a 
frequent  component  of  INFRA  projects,  being  especially relevant  to  the  users  of  research 
infrastructures.  In  addition,  participants  in  SwafS  reported  that  researchers  improved 
research skills, knowledge and competences to a large extent (50%) or to a moderate extent 
(31%) thanks to the support they received from Horizon 2020798.  

Horizon  2020  also  aimed  to  promote  transnational  access  to  pan-EU  research 
infrastructures,  mainly  through  INFRA.  Recent  statistics  from  the  Horizon  Dashboard 
indicate  that  20,376  (above  the  target  of  20,000)  researchers  had  access  to  research 
infrastructures (RI) as a result of Horizon 2020 support, mostly stemming from INFRA. It was 
found  that  a  total  of  844  RIs  have  been  accessed,  with  an  average  of  1.6  installations 
accessed  per  RI.  The  three most  widely  used  RIs  were  all  synchrotron  radiation  sources: 
SOLEIL in France, Diamond in the UK, and MAX IV in Sweden799.  

Lastly,  Horizon  2020  strengthened  the  EU  labour  market  for  researchers  through 
EURAXESS  and  support  for  the  RESAVER  pan-European  pension  scheme.  The 
evaluation study of Excellent Science in the European Framework Programmes for Research 
and  Innovation  has  found  that  between  2014  and  2020,  a  total  of  442,752  offers  were 
published on EURAXESS, exceeding the third ERA KPI’s annual target of advertising 60,000 
research positions800.  

794 European Commission,. Evaluation study on Excellent Science in the European Framework 
Programmes for Research and Innovation. Phase 1 Annexes, 2023. 
795 Ibid. 
796 Ibid. 
797 Survey of MSCA fellows (IF, ITN, COFUND, RISE), conducted for the Study on mobility flows 
in the context of the MSCA, 2022. 
798 Survey of Horizon 2020 beneficiary organisations, conducted August-September 2022. 
799 Horizon Dashboard data, accessed 21 September 2022. Data provided by the European 
Commission indicate that 16,712 distinct researchers have so far received access to RIs through 
INFRA – under FP7, this figure reached 21,060. 
800 European Commission, Directorate-General for Research and Innovation, Naujokaitytė, R., 
Mačiulienė, M., Papageorgiou, H., et al., Evaluation study on the implementation of cross-cutting 
issues in Horizon 2020 : study in support of the ex-post evaluation of the European Framework 
Programme for Research and Innovation Horizon 2020, Dirham, S. (editor), Luxembourg: 
Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/763665  

478 

 
 
1.2.3.  Overall approach and scope of the case study 

To assess the impacts of Horizon Europe on strengthening human capital in R&I, the present 
case study addresses questions relating to relevance, coherence, effectiveness, efficiency 
and EU added value. While the case study draws on a mix of methods to answer the specific 
evaluation  questions,  it  focuses  primarily  on  the  results  of  the  survey  and  the  interview 
programmes, as well as an analysis of pre-existing EC administrative and monitoring data. 
The table below presents the evaluation criteria with specific relevance to the case study and 
its objectives. 

Evaluation criteria 

Judgement criteria 

Relevance 

Coherence 

Effectiveness 

Efficiency 

EU added value  

•  Relevance of the training opportunities offered to the needs of the 

• 

• 

• 

• 

• 

• 

• 

• 

stakeholders and target groups. 
The extent to which the needs of the target groups are addressed 
in terms of the available tools and thematic areas covered. 
Knowledge and awareness of supporting tools and thematic areas 
among the target groups. 
Evidence of support for addressing newly emerging needs 
(migration, COVID-19, the green and digital transitions). 

The positioning of the FP in relation to strengthening human 
capital in R&I, within the overall European research and innovation 
landscape and beyond.  
The extent to which the FP’s contribution to strengthening human 
capital in R&I is balanced, and the identification of any significant 
overlaps/gaps. 

Patterns of third country/associated country participation across 
different Framework Programme parts.  
The extent to which the FP helps in attracting and retaining 
European and international talent. 
The extent to which the FP contributes to the return mobility of 
researchers.  

•  Gender balance and equal representation of researchers in the 

• 

• 

• 

• 

• 

• 

• 

• 

FP. 
The FP’s contribution to improved recruitment practices and 
working conditions for researchers.  
The share of EU researchers that have been 
intersectorally/internationally mobile.  
The extent to which researchers have access to research 
infrastructures (RI) as a result of Horizon Europe support. 
The extent to which the FP contributes to the development of the 
transversal/transferable skills of researchers. 

The sufficiency of funding for research, training and networking 
activities; skills development of researchers. 
The extent to which the FP supports inter-sectoral, interdisciplinary 
cooperation. 
The contribution made by the FP to the creation of international 
collaborations. 

The situation under the FP in comparison to a situation in which 
the Framework Programme had not existed. 

479 

 
To address the criteria listed above, this case study analyses and triangulates data obtained 
from different sources and through the following research methods: 

•  Desk research: a thorough literature review has been conducted to map and synthesise 
evidence and findings from existing sources of information on human capital in R&I under 
the  EU Framework  Programmes,  including  previously  conducted  evaluations, studies, 
databases and similar materials. To provide a comparison between Horizon 2020 and 
Horizon  Europe,  special  focus  will  be  given  to  an  assessment  of  the  performance  of 
Horizon 2020 in strengthening human capital in R&I.  

•  EC  administrative  and  monitoring  data:  to  assess  the  number  and  profiles  of 
researchers supported by Horizon Europe, this case study uses EC administrative and 
monitoring data provided by participants in the FP. The data submitted by the participants 
can be easily disaggregated by career stage, Horizon pillar, cluster and component, type 
of Horizon Europe action, programme year, field of science, and gender. The analysis of 
monitoring data will also provide insights into gender balance and the participation in the 
FP  of  third  country/associated  country  nationals.  In  addition,  the  above  metrics  are 
compared with the findings from the evaluation study on Excellent Science in Horizon 
2020. 

•  Survey  programme:  to  address  the  relevant  evaluation  questions,  this  case  study 
exploits and analyses survey data collected within the scope of Phase 2 of the Evaluation 
Study on Excellent Science in the European Framework Programmes for Research and 
Innovation.  The  survey  data  mostly  contribute  to  addressing  evaluation  questions 
relating to the relevance, effectiveness and EU added value of the FP. In addition, the 
survey data provide insights into types of mobility (e.g. international, intersectoral) and 
impacts on the development of specific skills (e.g. entrepreneurship, digital technologies 
and their application, Open Science, IPR) of researchers supported by Horizon Europe. 
In this regard, the case study analyses the findings of the following surveys:  

−  A survey of beneficiary organisations; 

−  A survey of unsuccessful applicant organisations; 

−  A survey of principal investigator (PI) beneficiaries under the ERC; 

−  A survey of MSCA fellows under MSCA PF, fellows under Doctoral Networks (DN), 

and beneficiaries of COFUND and Staff Exchanges (SE) ; and 

−  A survey of unsuccessful applicants. 

• 

Interview  programme:  within  the  scope  of  this  case  study,  10  interviews  have  been 
conducted  with  the  coordinators/PIs  of  projects  supported  by  Horizon  Europe.  The 
interview programme has helped to gather insights into the relevance, scope and breadth 
of  the  training  opportunities  provided  by  Horizon  Europe.  It  also  provides  information 
about the level of coherence between different Framework Programme parts with respect 
to the formation of human capital in R&I. The sample of individual researchers selected 
for interview comprises:  

−  3 interviews with coordinators of MSCA-funded projects ;  

−  3 interviews with principal investigators of ERC-funded projects;  

480 

 
−  3 interviews with the coordinators of WIDERA (widening participation)-funded projects. 

This case study focuses in particular on the development and enhancement of the skills of 
researchers under Horizon Europe; however, due to data limitations, the case study is unable 
to provide concrete observations in relation to the career progression and working conditions 
of researchers. Many Horizon Europe projects have not yet started, and up to this point, only 
a small number of projects have been completed. As a result, this case study cannot draw 
concrete  conclusions  with  regard  to  the  indicators  evaluating  medium-  and  long-term 
progress (i.e. the careers and working conditions of researchers), as envisioned by KIP2.1 
and  KIP2.2.  However,  where  relevant,  the  study  explores  how  KIP2.1  and  KIP2.2  can  be 
further assessed, given the data constraints. 

1.3.  Case study findings 

1.3.1.  Relevance  

The  programme  parts  under  Horizon  Europe’s  Excellent  Science  pillar  exhibit  a  well-
integrated  approach  towards  the  formation  of  human  capital.  Findings  from  the  interviews 
carried  out  for  this  case  study  suggest  that  the  ERC,  MSCA  and  WIDERA  (widening 
participation)  programme  parts  together  create  a  continuum  of  funding  opportunities 
spanning  from  early-career  researchers  to  established  scientists.  These  programmes  are 
designed  to  complement  each  other,  and  collectively  contribute  to  fostering  excellence, 
mobility and the development of talent.  

to  experienced  researchers  seeking 

The MSC actions cover researchers at different career stages, ranging from early-stage PhD 
candidates 
training 
opportunities. By focusing on skills development, collaboration and mobility, the programme 
aims to contribute to the professional growth and employability of researchers across Europe 
and beyond801. 

international  mobility  and 

The ERC, on the other hand, offers funding opportunities for researchers at various career 
stages who already possess a track record of significant research achievements. ERC grants 
aim  to  support  pioneering  and  high-risk/high-gain  research  projects  conducted  by 
independent  investigators802.  Evidence  stemming  from  the  interviews  suggests  that  the 
ERC’s  ‘bottom-up’  approach  is  highly  welcomed;  however,  some  interviewees  raised 
concerns  in  connection  with  challenges  relating  to  the  composition  of  evaluation  panels, 
calling for improvements in the panels’ ability to adapt to interdisciplinary proposals.  

The WIDERA (widening participation) programme primarily targets institutions and regions in 
the widening countries in order to reduce disparities in R&I across Europe and to enhance 
the participation of countries with lower levels of R&I performance. In addition, it addresses 
the needs of researchers through dedicated funding opportunities such as ERA Talents and 
ERA Fellowships. ERA Talents focuses on collaboration and the circulation of cross-sectoral 

801 European Commission, Work Programme 2023-2024: Marie Skłodowska-Curie Actions, 
2023, available at: https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/wp-call/2023-2024/wp-2-msca-actions_horizon-2023-2024_en.pdf  
802 European Commission, Work Programme 2023: European Research Council, 2023, available 
at: https://erc.europa.eu/sites/default/files/document/file/ERC_WorkProgramme_2023.pdf  

481 

 
 
talent, while ERA Fellowships build on the MSCA Postdoctoral Fellowships scheme and allow 
host organisations in widening countries to attract excellent researchers from abroad803.  

In addition, the Research Infrastructures (RI) programme part also contributes indirectly to 
strengthening human capital by providing resources and services for researchers to conduct 
research and foster innovation. The research infrastructures involved vary from single-sided 
to  distributed  and  virtual.  RI  also  enables  access  to  digital  resources  and  massive  data 
processing power. Furthermore, RI also benefits researchers by enabling cooperation with 
leading  partners  from  industry,  providing  customised  services  and  training  on  the 
management and exploitation of advanced infrastructures804.  

Besides  direct  funding  opportunities  to  support  researchers  at  different  career  stages,  the 
Horizon Europe programme parts under Pillar 1 and WIDERA offer various mechanisms and 
tools  to  better  facilitate  coordination  and  knowledge  exchange,  as  well  as  enhancing 
interactions between beneficiaries, researchers and policy-makers.  

The  MSCA  programme  offers  additional  assistance  to  both  national  and  international 
organisations  collaborating  with  researchers  at  risk.  This  initiative  aims  to  elevate  and 
professionalise the efforts of such researchers, specifically focusing on those whose careers 
are jeopardised by political, social or economic circumstances in their home nations. With a 
dedicated budget amounting to EUR 1,299,749, the project supports support organisations, 
not researchers directly. It is mainly aimed at building capacities and enhancing networking 
among these organisations.      

In response to the Russian invasion of Ukraine, the MSCA has also proactively taken steps 
to  provide  specific  support  to  affected  doctoral  candidates  and  post-doctoral  researchers 
from Ukraine. Through the dedicated MSCA4Ukraine scheme, launched in 2022, a funding 
allocation of EUR 25 million has been designated to aid displaced researchers, allowing them 
to continue their scientific pursuits in Europe and to contribute to the rebuilding of Ukraine's 
R&I capacity when conditions permit. The scheme received  over 600 applications from 28 
countries,  and  111  post-doctoral  and  13  doctoral  fellowships  have  so  far  awarded  to 
researchers from Ukraine805. The project has received a top-up of EUR 10 million to provide 
additional fellowships. 

Lastly,  the  MSCA  Network  of  National  Contact  Points  (MSCA-NET)  plays  a  crucial  role  in 
assisting  researchers  and  institutions  in  the  respective  countries  of  the  National  Contact 
Points (NCPs) with information, guidance and support relating to participation in the MSCA 
programme. The MSCA NCP Network consists of experts located in each EU Member State 
as  well  as  in  associated  countries.  These  experts  serve  as  central  points  of  contact  and 
provide  a  range  of  services,  including  information  and  guidance  about  various  funding 

803 European Commission, Widening participation and strengthening the European Research 
Area. Horizon 2023-2024 work programme, 2023-2024, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-call/2023-
2024/wp-11-widening-participation-and-strengthening-the-european-research-area_horizon-
2023-2024_en.pdf  
804 European Commission, Work Programme 2023-2024. Research Infrastructures, 2023, 
available at: https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2023-2024/wp-3-research-infrastructures_horizon-2023-2024_en.pdf  
805 European Commission, European Commission announces support for 124 researchers who 
fled the war through #MSCA4Ukraine. Marie Skłodowska-Curie Actions, 2023, available at: 
https://marie-sklodowska-curie-actions.ec.europa.eu/news/european-commission-announces-
support-for-124-researchers-who-fled-the-war-through-msca4ukraine  

482 

 
 
opportunities, help and feedback with proposal preparation, networking and partner search, 
as well as organising information events and workshops806. 

Under Horizon Europe, the European Research Council (ERC) has launched a specific action 
designed  to  promote  the  understanding  of  frontier  research  through  journalistic  exposure. 
The  ERC  Science  Journalism  Initiative807  aims  to  bridge  the  gap  between  the  scientific 
community  and  the  public  by  facilitating  interactions  between  journalists  and  research 
institutions to promote the better understanding of frontier research. 

Meanwhile, the WIDERA programme action ‘Developing an Effective ERA Talent Pipeline’ 
aims  to  create  a  common  career  structure  for  academic  researchers,  reflecting  modern 
trajectories  and  variations  between  academic  professions,  and  drawing  on  good  practices 
from the Member States. This exercise is expected to enable the widespread recognition of 
the  ESCO  qualification  profile,  particularly  those  competences  that  PhD  trainees  and 
postdocs  at  various  stages  of  their  careers  have  obtained808.  Another  dedicated  action, 
‘Strengthening researchers’ skills for better careers – leveraging the European Competence 
Framework for Researchers’, aims to foster more open and inter-sectoral research careers, 
and better recognition for the research profession809.  

The WIDERA Work Programme 2023-2024 also includes a contribution agreement with the 
OECD  on  the  Research  Careers  Observatory.  This  aim  of  this  action  is  to  monitor  the 
implementation of the European Framework for Research Careers and to provide evidence 
on  the  effectiveness  of  measures  aiming  at  balancing  talent  circulation  and  strengthening 
research careers810.  

The WIDERA Programme is also putting additional efforts into transforming and expanding 
the existing EURAXESS network, services and portals. The EURAXESS platform is set to 
undergo a transformation into an ERA Talent Platform, which will act as a ‘one-stop shop’ 
with  improved  structure  and  governance,  taking  advantage  of  links  to  Europass  and  the 
EURES network of European public employment services811. It is expected that the newly 
transformed  platform  will  focus  in  particular  on  opening  up  new  career  trajectories  for 
researchers in academia, industry and business.  

Overall,  most  interviewees  agreed  that  strong  support  is  available  from  the  European 
Commission  to  help  with  understanding  the  Framework  Programme’s  documentation  and 
rules, including assistance from colleagues and grant offices. However, institutional support 

806 MSCA NET, MSCA-NET Project Aims at Improving, Professionalising and Harmonising 
Services of the Marie Skłodowska-Curie Actions (MSCA) NCPs, 2 May 2023, available at: 
https://msca-net.eu/  
807 European Research Council, Science Journalism Initiative, 2022, available at: 
https://erc.europa.eu/apply-grant/science-journalism-initiative  
808 European Commission, Widening participation and strengthening the European Research 
Area. Horizon 2023-2024 work programme, 2021-2022, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-call/2021-
2022/wp-11-widening-participation-and-strengthening-the-european-research-area_horizon-
2021-2022_en.pdf  
809 Ibid. 
810 European Commission, Widening participation and strengthening the European Research 
Area. Horizon 2023-2024 Work Programme, 2023-2024, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-call/2023-
2024/wp-11-widening-participation-and-strengthening-the-european-research-area_horizon-
2023-2024_en.pdf  
811 Ibid. 

483 

 
 
plays  a  vital  role,  and  promoting  stronger  research  support  positions  would  be  beneficial, 
according to interviewees. Furthermore, interviewees pointed to a good level of awareness 
with regard to the various funding opportunities available under Horizon Europe, particularly 
under Pillar 1 and the WIDERA programme part.  

1.3.2.  Coherence 

The  findings  from  the  interview  programme  highlight  coherence  between  different 
programme parts under Pillar 1 of Horizon Europe in relation to strengthening human capital 
in R&I. For example, the role and activities of the MSCA were found to be especially coherent 
with those of the ERC. Both programmes are bottom-up or investigator-driven, and share a 
similar emphasis on stimulating excellence and the production of high-quality research. The 
ERC and MSCA are also seen as the main instruments for the development and attraction of 
talent  to  the  European  Union.  Interviewees  underlined  complementarities  rather  than 
overlaps between these two programme parts. To illustrate this, the MSCA provide funding 
for the fellows recruited by the beneficiary, while ERC funding supports not only the principal 
investigators but also the researchers sub-contracted to the project, enabling PIs build their 
own research teams. Furthermore, the MSCA fund a broader range of target groups: both 
experienced and non-experienced researchers, as well as admin/managerial staff under Staff 
Exchanges. In addition, the MSCA fund a wider range of TRLs compared with the ERC. While 
both programme parts offer funding opportunities for researchers at various career stages, 
the  ERC  focuses  on  supporting  researchers  who  possess  a  track  record  of  significant 
research achievements. In doing so, he ERC aims to support pioneering and high-risk/high-
reward research projects conducted by independent investigators. Against this backdrop, the 
interviewees also regarded the ERC as complementary to the MSCA, as researchers view 
ERC funding opportunities as a possible next step in their careers after participating in the 
MSCA.  

Tangible similarities can be found between MSCA Staff Exchanges and ERA Talents. Both 
schemes  promote  innovative  collaborations  in  R&I  through  exchanging  staff  and  sharing 
knowledge ideas at all stages of the innovation chain. However, the MSCA Staff Exchanges 
encompass collaborations  on  international,  inter-sectoral  and  interdisciplinary  levels, while 
ERA  Talents  aims  to  promote  solely  inter-sectoral  cooperation,  with  a  specific  focus  on 
widening countries. Unlike MSCA Staff Exchanges, the ERA Talents scheme also enables 
same-country exchanges on condition that they serve the specific purpose of maintaining or 
reinforcing regional innovation ecosystems.  

Another  link  between  the  MSCA  and  WIDERA  programme  parts  is  provided  by  the  ERA 
Fellowships  scheme.  The  evidence  from  the  interviews  points  to  the  synergies  that  exist 
between  ERA  Fellowship  scheme  and  MSCA  Postdoctoral  Fellowship  (PF)  scheme.  The 
objective  of  the  ERA  Fellowships  is  to  incentivize  brain  circulation  towards  widening 
countries. Researchers that consider applying to the ERA Fellowships can do so by applying 
to  the  MSCA  Postdoctoral  Fellowships  and  opting  in  for  automatic  resubmission  of  an 
application  through  ERA  Fellowships  in  case  their  proposal  cannot  be  funded  under  the 
MSCA  due  to  budget  constraints.  Evidence  from  the  interviews  suggests  that  the  ERA 
Fellowship scheme successfully increases the chances of receiving funding for organisations 
located in the widening countries, thus further contributing to the narrowing of disparities in 
R&I across Europe.  

Evidence  from  the  desk  research  and  interviews  carried  out  for  this  case  study  provides 
insights into the external coherence of the programme parts analysed within the context of 
the  European  R&I  ecosystem  as  a  whole,  specifically  in  terms  of  the  formation  of  human 
capital. The findings indicate that this level of external coherence varies between widening 
and  non-widening  countries.  For  example,  in  some  non-widening  countries,  national  or 

484 

 
regional-level funding schemes exist that are similar to the ERC (for example, the Agence 
nationale de la recherche in France, the Luxembourg National Research Fund, UK Research 
and  Innovation).  Meanwhile,  in  widening  countries,  the  ERC  usually  serves  as  the  main 
opportunity to receive funding for research on such a scale. 

The overall findings of the study also point to the complementarity of Horizon Europe with 
national  and  regional  schemes  and  initiatives.  This  becomes  evident  when  considering 
Horizon  Europe’s  emphasis  on  the  international  dimension  of  research.  This  approach 
addresses the needs of researchers at different stages of their careers by providing training 
opportunities that extend beyond national borders. While European Member States do offer 
a diverse range of doctoral and postdoctoral training programmes, one common limitation of 
these is a focus on funding researchers who are geographically tied to their home countries. 
Consequently,  this  geographical  restriction  can  hinder  the  potential  for  cross-border 
collaborations  and  knowledge  exchange.  Horizon  Europe's  international  focus  seeks  to 
bridge this gap and foster a more interconnected research landscape, allowing researchers 
to access opportunities beyond their home countries, facilitating cross-border collaborations, 
and promoting a more integrated and globally impactful research community. 

The analysis conducted for this study reveals clear complementarities with the national and 
regional  R&I  funding  systems  within  EU  Member  States,  particularly  in  terms  of  securing 
additional or combined funding for projects supported by Horizon Europe. Among the MSCA 
beneficiary organisations surveyed, 22% indicated that they  had sought additional funding 
for research ideas or activities addressed in their project through national or regional R&D 
funding schemes. The findings of the MSCA survey also shed light on the effectiveness of 
such efforts, revealing that out of those organisations that applied, more than one-third (37%) 
successfully obtained additional funding. 

Similarly, with regards to successful ERC projects, 19% of the ERC beneficiary PIs surveyed 
sought  supplementary  funding  for  their  dedicated  budgets.  The  outcome  of  these  efforts 
demonstrates  the  accessibility  of  resources,  with  almost  half  (47%)  of  those  who  applied 
receiving a positive decision. 

The  figures  for  beneficiaries  of  the  WIDERA  (widening  participation)  programme  part, 
meanwhile, show that 18% applied for additional resources, with more than one-third (37%) 
of these receiving the additional funds requested. 

The above statistics underline the synergies between European and national/regional funding 
mechanisms,  and  the  potential  for  researchers  and  organisations  to  leverage  multiple 
sources  of  support.  Thus,  they  reflect  a  coordinated  ecosystem  in  which  researchers  and 
organisations  can  strategically  navigate  various  channels  to  enhance  their  research 
endeavours. 

1.3.3.  Effectiveness  

According to European Commission monitoring data, 48,637 individual researchers are so 
far  participating  in  the  Horizon  Europe  Framework  Programme,  while  11’185  individual 
researchers are participating under Pillar 1 or WIDERA programme parts. The MSCA account 
for the largest share of researchers (48%), followed by WIDERA (27%), INFRA (21%) and 
the ERC (4%). The reader should note that these metrics do not present the complete picture, 
as  some  funding  schemes  under  Pillar  1  and  the  WIDERA  programme  involve  financial 
support for participating organisations to recruit doctoral and postdoctoral researchers – for 
example, as in the case of the MSCA programme part. Due to the fact that many projects are 

485 

 
still in the early stage of implementation and the recruitment of researchers is ongoing or yet 
to start, these numbers should increase drastically in the near future.  

1.3.3.1. 

Training and continuous professional development 

By providing researchers with a comprehensive set of skills and experiences, the programme 
parts under Pillar 1 and WIDERA (widening participation) aim to enhance their employability, 
encourage  innovation,  and  contribute  to  strengthening  human  capital  in  R&I.  When 
evaluating the contributions made by the MSCA, a survey of Horizon Europe beneficiary 
organisations shows that most frequent topics of training provided to MSCA researchers 
in the DN, SE and COFUND actions related to new and/or advanced scientific methods 
(82%  of  respondents);  interdisciplinary  research  techniques  (71%);  and  proposal  and 
report writing (64%). Training on the commercialisation of research outputs (13%) and 
entrepreneurship (9%) were the topics least frequently identified by respondents. 

Figure 1. What type(s) of training have you provided, or plan to provide, in the framework of your MSCA project? Source: 
survey of Horizon Europe MSCA beneficiary organisations (DN, SE, COFUND), conducted May-July 2023 (n=1,095). 

To assess the impact of the specific programme parts under Pillar 1 and WIDERA (widening 
participation), the survey of successful Horizon Europe applicants asked MSCA PF fellows 
and ERC principal investigators to what extent, if at all, participation in their Horizon Europe 
projects would have an impact on the development of skills in various areas. The results are 
presented in the table below. 

Among MSCA Postdoctoral Fellows, there was a strong belief that participation in Horizon 
Europe projects would have a substantial impact on the development of core research skills, 
particularly in terms of scientific methods and/or techniques (93% of MSCA PF beneficiaries 
surveyed), critical and abstract thinking (89%), and the communication of research results 
(87%). Even though most areas received a positive response, there are a few categories – 
such  as 
the  commercialisation  of  research  outputs,  entrepreneurship,  and  digital 
technologies – in which fewer than 40% of respondents expected their participation to have 
a significant impact. 

ERC PIs, meanwhile, anticipated that participation in Horizon Europe projects would have a 
significant  impact  on  their  skills  in  areas  such  as  scientific  methods  (83%  of  ERC  PIs 
surveyed), project and people management (83%), and critical thinking (77%). Skills relating 
to  the  training  and  supervision  of  students  also  stand  out,  with  77%  of  PIs  expecting 
participation to have a large impact. 

486 

 
 
To a 
very 
large / 
large 
extent 

Count 
of total 

To a 
very 
large / 
large 
extent 

Count of 
total 

MSCA PFs 

ERC PIs 

93% 

824 

83% 

237 

Scientific methods and/or techniques 
(disciplinary and interdisciplinary research 
methods and techniques, use of specialised 
equipment, etc.)  

Thinking (critical and abstract thinking, problem 
solving)  

89% 

823 

77% 

237 

Communication of research results (publication 
processes, communication within the research 
community and with the broader public, ways to 
increase impact of research on policy and 
society, etc.)  

87% 

814 

67% 

231 

Career development  

87% 

823 

69% 

232 

Scientific writing  

Proposal writing  

84% 

823 

68% 

237 

84% 

820 

57% 

236 

Open Science (open access, re-usability of 
research results, citizen science, etc.)  

79% 

818 

35% 

236 

Project and people management  

72% 

823 

83% 

232 

Training and supervision of students  

66% 

818 

77% 

232 

Open innovation and knowledge transfer  

62% 

816 

35% 

234 

Research ethics and integrity  

61% 

814 

43% 

233 

Well-being, inclusion, diversity management  

44% 

804 

N/A 

N/A 

Intellectual property rights management  

42% 

806 

34% 

233 

Digital technologies and their application  

37% 

805 

32% 

230 

Sustainable implementation of research activities 
(green purchasing, awareness on environmental 
sustainability, use of renewable forms of energy)  

36% 

806 

18% 

230 

487 

 
 
To a 
very 
large / 
large 
extent 

Count 
of total 

To a 
very 
large / 
large 
extent 

Count of 
total 

MSCA PFs 

ERC PIs 

Commercialisation of research outputs  

25% 

803 

18% 

234 

Entrepreneurship  

21% 

801 

N/A 

N/A 

Figure 2. To what extent, if at all, will participation in your Horizon Europe project have an impact on the development of 
your skills in the following areas? Source: survey of Horizon Europe beneficiaries, conducted May-July 2023. 

ERC PIs were also asked about their expectations regarding the impacts of participation in 
an ERC project on their career development. The survey findings concluded that the top five 
main  expectations  were:  strengthening  their  capacity  to  conduct  frontier  research  (93%), 
strengthening their reputation as a researcher (91%), establishing and/or consolidating their 
own research group (81%), obtaining a new fellowship/research grant (76%), and becoming 
an established researcher (70%). These responses reflect a strong emphasis on enhancing 
research  capabilities,  solidifying  research  standing,  and  advancing  towards  greater 
professional independence and recognition within the academic and scientific community.  

findings,  widespread 
Evidence  stemming 
acknowledgement  of  the  ERC’s  contribution  to  the  acquisition  of  interdisciplinary  and 
transferable skills, particularly in cutting-edge fields that strengthen the capacities of PIs and 
their research teams to conduct frontier research.  

interviews  backs  up 

these 

from 

the 

100%

93%

91%

80%

60%

40%

20%

0%

81%

76%

70%

Strengthening my
capacity to conduct
frontier research

Strengthening my
reputation as a
researcher

Establishing and/or
consolidating my own
research group

Obtaining a new
fellowship/research
grant

Becoming an
established researcher

Figure 3. Do you expect that your participation in the ERC project will have any of the following impacts on you/your career 
development (successful ERC applicants)? Source: survey of Horizon Europe beneficiaries, conducted May-July 2023 
(n=236). 

1.3.3.2. 

Career advice and scientific supervision  

Among the programme parts under Pillar 1 and WIDERA (widening participation), the MSCA 
place a profound emphasis on the quality of supervision and mentorship provided to those 
researchers  who  are  funded  under  its  various  schemes.  The  MSCA  Guidelines  on 
Supervision  aim  to  ensure  that  researchers  receive  effective  support  and  mentorship. 

488 

 
 
 
Effective  supervision  is  a  critical  component  of  the  MSCA  programme’s  commitment  to 
nurturing the next generation of research talent812. 

Based on the results of the Horizon Europe successful applicant organisations survey, the 
graph below outlines perceived improvements at organisational level in two key areas: the 
provision of career advice for researchers, and the quality of scientific supervision The results 
are presented for three different MSCA actions – SE, COFUND and DN. – along with overall 
percentages across all three actions in each area.  

The graph highlights that there is generally a high expectation of improvement across both 
areas  of  career  advice  and  scientific  supervision  for  researchers,  with  expectations  being 
slightly  higher  with  regard  to  the  quality  of  scientific  supervision  (76%)  compared  with  the 
provision of career advice (64%).  

88%

60%

65%

64%

79%

88%

75%

76%

100%
80%
60%
40%
20%
0%

SE (n=111) COFUND

DN (n=327)

(n=8)

Total
(n=446)

SE (n=117) COFUND

DN (n=344)

(n=8)

Total
(n=469)

Provision of career advice for researchers

Quality of scientific supervision

Figure 4. To what extent, if at all, is your participation in the Horizon Europe project expected to lead to improvements at 
organisational level in the following areas? ‘Provision of career advice for researchers’ and ‘Quality of scientific supervision’ 
(MSCA successful applicant organisations). Source: survey of Horizon Europe beneficiaries, conducted May-July 2023. 
Note: results for the MSCA COFUND action should be interpreted with caution due to the low number of responses. 

The  WIDERA  (widening  participation)  programme  also  aims  to  enhance  career  prospects 
and  the  quality  of  scientific  supervision  for  researchers  through  the  support  provided  by 
Horizon  Europe.  The  corresponding  survey  findings  reflect  this,  with  WIDERA  (widening 
participation) beneficiary organisations showing substantial expectations for improvements 
in scientific supervision (67%) and the provision of career advice for researchers (63%). 

812 European Commission, Directorate-General for Education, Youth, Sport and Culture, Marie 
Skłodowska-Curie actions guidelines on supervision, Luxembourg: Publications Office of the 
European Union, 2021, available at: https://data.europa.eu/doi/10.2766/508311. 

489 

 
 
 
 
 
Provision of career advice for researchers
(n=161)

21%

42%

21%

10%

6%

Quality of scientific supervision (n=165)

28%

39%

24%

5%

3%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

To a very large extent

To a large extent

To some extent

To a limited extent

Not at all

Figure 5. To what extent, if at all, is your participation in the Horizon Europe project expected to lead to improvements at 
organisational level in the following areas? ‘Provision of career advice for researchers’ and ‘Quality of scientific supervision’ 
(WIDERA successful applicant organisations). Source: survey of Horizon Europe beneficiaries, conducted May-July 2023. 

1.3.3.3. 

Quality and transparency of recruitment practices/working conditions  

As  demonstrated  above,  the  WIDERA  and  the  MSCA  programmes  both  play  a  significant 
role in promoting better recruitment practices and working conditions for researchers across 
Europe and beyond. In doing so, they aim to create an attractive and supportive research 
environment by encouraging participating organisations to adopt the principles outlined in the 
European  Charter  for  Researchers  and  the  Code  of  Conduct  for  the  Recruitment  of 
Researchers813814.  

The  MSCA  require  host  organisations  to  advertise  open  positions  for  fellows  in  MSCA 
doctoral  and  postdoctoral  programmes  on  Euraxess,  provide  clear  eligibility  criteria  and 
ensure  that  the  selection  process  is  fair  and  unbiased.  In  addition,  while  MSCA-funded 
projects are time-bound, host organisations are encouraged to provide researchers with clear 
and fair employment conditions, including fixed-term contracts that align with the duration of 
the research project. Researchers should also have access to social security benefits and 
other employment rights in the host country815.  

The graph below outlines the expectations of MSCA participating organisations with regard 
to organisational improvements in three areas: the quality and transparency of recruitment 
practices,  the  working  conditions  of  researchers,  and  the  working  conditions  of  other 
employees. The results of the Survey of Horizon Europe beneficiaries reflect strong optimism 
regarding  improvements  in  recruitment  practices  (56%)  and  the  working  conditions  of 
researchers  (52%).  However,  expectations  are  more  modest  with  regard  to  the  working 
conditions  of  other  employees,  with  a  more  even  distribution  across  the  different  levels  of 
improvement anticipated. 

813 European Commission (2021). Widening participation and strengthening the European 
Research Area Work Programme 2021-2022. 
814 European Commission (2021). MSCA Work Programme 2021-2022. 
815 Ibid.  

490 

 
 
 
Working conditions of other employees…

11%

22%

32%

19%

17%

Working conditions of researchers (n=446)

17%

35%

28%

12%

8%

Quality and transparency of recruitment…

22%

34%

30%

9%

5%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

To a very large extent

To a large extent

To some extent

To a limited extent

Not at all

Figure 6. To what extent, if at all, is your participation in the Horizon Europe project expected to lead to improvements at 
organisational level in the following areas? ‘Quality and transparency of recruitment practices’, ‘Working conditions of 
researchers’ and ‘Working conditions of other employees’ (MSCA). Source: survey of Horizon Europe beneficiaries, 
conducted May-July 2023. 

In a similar vein, the results of the Survey of Horizon Europe beneficiaries show that a majority 
of WIDERA participating organisations believe that participation in the Horizon Europe project 
will lead to significant improvements in the quality and transparency of recruitment practices 
and the working conditions of researchers. The optimism is slightly less when it comes to the 
working conditions of other employees, although a considerable share of respondents (47%) 
still expect major improvements.  

Working conditions of other employees
(n=150)

15%

32%

29%

14%

9%

Working conditions of researchers (n=154)

18%

Quality and transparency of recruitment
practices (n=154)

19%

35%

33%

29%

11%

7%

29%

10%

9%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

To a very large extent

To a large extent

To some extent

To a limited extent

Not at all

Figure 7. To what extent, if at all, is your participation in the Horizon Europe project expected to lead to improvements at 
organisational level in the following areas? ‘Quality and transparency of recruitment practices’, ‘Working conditions of 
researchers’ and ‘Working conditions of other employees’ (WIDERA). Source: survey of Horizon Europe beneficiaries, 
conducted May-July 2023. 

1.3.3.4. 

Open access  

Open access (OA) is a fundamental aspect of the Framework Programme’s commitment to 
transparent and accessible research. By encouraging researchers to make their publications 
and data openly available, the programme aims to maximise the impact of  the research it 
funds816. However, evidence from the survey, as well as interviews with MSCA and WIDERA 

816 European Commission - Directorate-General for Research and Innovation. (2020). 
Background note on open access. Retrieved from https://research-and-
innovation.ec.europa.eu/system/files/2020-11/ec_rtd_background-note-open-access.pdf. 

491 

 
 
 
 
(widening  participation)  beneficiary  organisations  suggest  that  the  publication  of  project 
results in OA has been met with several pronounced challenges.  

One of the foremost difficulties faced in relation to OA publishing is the high costs associated 
with OA publishing rates – a challenge strongly reflected in the results of the survey of Horizon 
Europe successful applicant organisations (see Figure 8). These high costs act as a barrier 
for many researchers, especially those from third countries, many of whom struggle to afford 
the Open Access Fee (OAF). Under of Horizon Europe projects, the publication venue is the 
choice of the beneficiary, but the publication fees are reimbursable if the publishing venue is 
fully open access; however, publication fees for hybrid venues are not reimbursed. 

In addition, many OA journals possess low impact factors (IF), which leads to concerns over 
the  visibility  and  prestige  associated  with  publishing  via  these  platforms.  This  can  limit 
opportunities for students and early-career researchers aiming to achieve maximum visibility 
in order to enhance their career prospects. Furthermore, several respondents to the survey 
indicated  the  absence  of  peer  review  for  some  OA  repositories,  which  ultimately  creates 
uncertainty over the accuracy and trustworthiness of the data, as there are no formal checks 
or guarantees to ensure the quality of information published in such repositories.  

Taken  together,  the  factors  above  contribute  to  a  complex  landscape  that  challenges  the 
effectiveness  and  credibility  of  the  current  OA  publishing  model.  These  criticisms  refer  to 
those open-access journals perceived to be predatory and of low quality, whose main goal is 
to  make  profit  on  the  basis  of  levying  article  processing  charges  (APCs)  without  any 
meaningful peer review. However, higher-quality OA journals do exist, which prioritise quality 
research and rigour – but the choice of journal ultimately lies in the hands of the author. 

Difficulties accessing sufficient funds to pay for the fees
required by some journals or data repositories

There were no specific difficulties or challenges related to the
openness of project results

Challenges to comply with data protection regulation (GDPR)

11%

Lack of competences and knowledge on processes for
publishing in open access

Lack of knowledge and resources to share data and metadata
in open access

Lack of guidance on the implementation of the data
management plan

Difficulties identifying relevant open repositories or archives

Do not know/Not applicable

Possible negative impact on researchers´ career progression
related to publishing in open access

34%

36%

33%

19%

22%

17%

17%

17%

6%

7%

9%

8%

16%

14%

12%

14%

10%

Other

6%

4%

0% 5% 10% 15% 20% 25% 30% 35% 40%

MSCA beneficiary organisations(n=453)

WIDERA beneficiary organisations (n=168)

Figure 8. Which of the following difficulties or challenges did you face when publishing project results in open access? 
(successful applicant organisations). Source: survey of Horizon Europe beneficiaries, conducted May-July 2023. 

492 

 
 
1.3.3.5. 

Outputs  

The  findings  of  the survey  of  successful  Horizon  Europe  applicants  highlight  the  extent  to 
which the expected outputs have been/are likely to be produced by MSCA PFs and ERC PIs 
through  their  Horizon  Europe-funded  projects.  Both  groups  of  researchers  report  a  strong 
emphasis  on  research-related  outputs,  with  98%  of  MSCA  PFs  and  99%  of  ERC  PIs 
indicating research publications as a result. This is followed by new or improved research 
methodologies and new large-scale datasets/data, which are more prominent among ERC 
PIs (61%) than among MSCA PFs (50%).  

Both groups of researchers  also report a similar focus on creating new or improved  tools, 
methods or techniques (48% of MSCA PFs; 52% of ERC PIs), and on software and protocols 
(30% of MSCA PFs; 29% of ERC PIs). Educational materials (33% of MSCA PFs; 34% of 
ERC  PIs)  were  also  mentioned,  reflecting  a  commitment  to  innovation  and  education. 
Conversely, small shares of both groups indicated that they had produced (or expected to 
produce) outputs such as business plans, large-scale product validation, codes of conduct or 
pre-standards/standards, with these outputs ranging from 1% to 6% of each group. These 
findings reflect a lesser focus on commercialisation and industry standards.  

Research publications

New/improved research methodologies

New large-scale datasets/ data produced

New or improved tools, methods or techniques

Educational materials

New/improved software, protocols/guidelines to conduct research

Research roadmaps

Recommendations for policy makers/inputs to national or EU policies

Patents

Collaboration platforms

Training programmes

Testing, demonstrating and piloting.

Prototype

New or improved products, services or processes

Social innovation

Pre-standards/standards

Codes of conduct

Large scale product validation and market replication

Business plans for the development of a centre of excellence

99%
98%

61%

61%

50%

50%

52%

48%

34%
33%

29%
30%
32%

26%

18%

18%

24%
22%

16%

18%

14%

17%

20%

14%

23%

12%

9%
10%

5%

8%

6%
6%

4%
4%

1%
3%
2%
2%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%

ERC PIs (n=236)

MSCA PFs (n=829)

Figure 9. Please indicate which of the following outputs have been produced/are likely to result from your Horizon Europe 
project (successful applicants). Source: survey of Horizon Europe beneficiaries, conducted May-July 2023. 

Looking at the outputs that have been produced (or are likely to be produced) by beneficiary 
organisations participating in MSCA and WIDERA, the survey of successful Horizon Europe 
applicants revealed that both groups of organisations prioritised research publications (94% 
for MSCA; 89% for WIDERA), training programmes (65% for MSCA; 71% for WIDERA) and 
educational materials (54% for MSCA; 66% for WIDERA).  

493 

 
 
However,  compared  with  their  counterparts  in  the  MSCA,  beneficiary  organisations  in 
WIDERA  (widening  participation)  exhibit  a  more  pronounced  orientation  towards  policy 
engagement  (45%)  and  centres  of  excellence  (32%).  MSCA  beneficiary  organisations, 
meanwhile, tend to place greater emphasis on research methodologies (57%), new/improved 
tools and techniques (54%) and patents (31%).  

Research publications

Training programmes

New/improved research methodologies

New or improved tools, methods or techniques

Educational materials

New large-scale datasets/ data produced

New/improved software, protocols/guidelines to conduct research

Collaboration platforms

Testing, demonstrating and piloting.

Research roadmaps

Patents

Prototype

Recommendations for policy makers/inputs to national or EU policies

New or improved products, services or processes

Pre-standards/standards

Social innovation

Codes of conduct

Set up of center of excellence

Large scale product validation and market replication

Business plans for the development of a center of excellence

89%

94%

71%

65%

46%

57%

54%

54%

66%

46%

50%

36%

33%

36%

38%

36%
37%

33%

12%

46%

45%

31%

31%

25%

29%

26%
25%
25%

11%

14%
15%

11%
10%

7%

32%

7%

3%
4%

2%

19%

WIDERA (Widening) organisations (n=178)

MSCA organisations (n=494)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%

Figure 10. Please indicate which of the following outputs have been produced/are likely to result from your Horizon Europe 
project (successful applicant organisations). Source: survey of Horizon Europe beneficiaries, conducted May-July 2023. 

1.3.4.  Efficiency  

1.3.4.1. 

International and intersectoral mobility and collaboration 

The  present  study  finds  that  the  MSCA  programme  is  highly  effective  in  providing 
international mobility opportunities. This is reflected in the findings of the survey, in which 
an overwhelming majority of successful MSCA PF applicants agreed that the international 
mobility opportunities provided by their MSCA project respond, to a large or very large extent, 
to their needs. 

By using survey data, the evaluation team also analysed intersectoral mobility patterns of the 
MSCA  researchers.  To  interpret  the  results  of  the  survey,  a  few  caveats  must  be 
acknowledged. First, the survey asked researchers only about their main host organisation, 
while in some cases researchers may have co-host institutions. Second, the main aim of the 
survey was to compare the long-term periods (time in the main host institution) vs. shorter 
term periods (mainly secondments, placements).  

The survey showed that out of all MSCA researchers across all actions, only 3% were hosted 
long-term (as in the main host institution) in the private sector (PRC), while the highest share 
of  MSCA  researchers  were  hosted  in  higher  education  institutions,  followed  by  research 

494 

 
 
organisations. Only 8% of MSCA researchers across all actions indicated that their fellowship 
would include an additional shorter-term stay (for example, placement or secondment) in a 
non-academic sector. The majority of such short-term stays (67%) are expected to occur in 
the private sector, while 21% of short-term stays will take place in a public administration or 
government body. 

Individual researchers participating in DN, PF and COFUND actions may also opt to include 
a secondment phase, within the overall duration of their fellowship in any country worldwide. 
One -third (36%) of surveyed individuals fellows across DN, PF and COFUNDall actions will 
have  a  secondment  phase  during  their  fellowships.  The  majority  of  secondments  are 
anticipated  to  take  place  in  a  higher  education  institution  (75%)  while  private  sector 
secondments  account  for  6%  of  total  secondments.  Notably,  SE  action  is  fully  based  on 
secondments.  

From  the  perspective  of  MSCA  researchers,  however,  survey  evidence  shows  that  the 
interdisciplinary cooperation opportunities provided by their MSCA project responded 
to their needs to a (very) large extent (79% of researchers in all types of actions agreed 
with this statement).  

Combined, the countries represented in Figure 10 below have so far hosted over 80% of all 
fellows under Horizon Europe. The figure below presents the respective shares of MSCA PF 
fellows  hosted  by  the  most  popular mobility  destinations.  Countries  such  as  Spain  (16%), 
Italy (13%), France (12% and Denmark (12%) attracted the most MSCA PF fellows.  

16%

13%

12%

12%

18%

16%

14%

12%

10%

8%

6%

4%

2%

0%

8%

8%

5%

5%

4%

3%

ES

IT

FR

DE

NL

DK

BE

SE

NO

AT

Figure 11. Top 10 countries attracting the highest shares of MSCA PF fellows. Source: EC monitoring data (cut-off date 22 
July 2023). 

When PF fellowships are broken down by country group, EU non-widening countries receive 
the highest shares of MSCA PF fellows (88%), followed by EU widening countries (7%) and 
associated  countries  (5%).  The  figure  below  presents  the  mobility  patterns  of  MSCA  PF 
fellows. It shows that the majority of MSCA PF fellows from EU non-widening countries chose 
another  EU  non-widening  country  for  their  fellowships.  This  is  evident  across  all  country 
groups by nationality of fellows.  

495 

 
 
Figure 12. Nationality of postdoctoral fellows compared with countries of host institution. Source: EC monitoring data (cut-off 
date 26 July). 

As  found  in  previous  evaluation  studies,  under  Horizon  Europe,  the  MSCA  programme 
continues  to  support  fellows  in  returning  to  their  country  of  origin.  Significant  numbers  of 
fellows use the MSCA Postdoctoral Fellowship scheme as a means to return to their country 
of nationality. This is particularly evident in the case of researchers from countries such as 
Greece, Italy, Cyprus and Spain. 

67%

56%

44%

37%

70%

60%

50%

40%

30%

20%

10%

0%

23%

21%

14%

12%

10%

EL

IT

CY

ES

FR

PT

NL

IE

BE

Figure 13. Fellows with the same nationality as the host country in the Horizon Europe MSCA PF action. Source: EC 
monitoring data (cut-off date 26 July) 2023. 

1.3.4.2. 

International and intersectoral collaborations 

The  MSCA  places  significant  importance  on  establishing,  fortifying  and  sustaining 
international  and  cross-sectoral  collaborations  among  researchers  and  institutions. This  is 
carried out with the aim of enabling the smooth movement of researchers and knowledge 
across borders and sectors, both during and after the fellowship period. This study reveals 
that overall, 86% of successful MSCA applicant organisations believe that their participation 
in the MSCA programme will strengthen (or is likely to strengthen) relationships with leading 
partners in Europe to a large or very large extent.  

496 

 
 
 
Furthermore, an overwhelming majority of successful MSCA applicant organisations believe 
that  their  MSCA  project  responds  to  their  need  to  create  or  strengthen  collaboration  with 
leading research organisations to a large or very large extent. Fewer organisations believed 
that MSCA projects would respond to their need to create and strengthen collaborations with 
leading industry organisations (58%).  

Evidence  from  the  interviews  carried  out  for  this  study  also  showed  that  there  were  more 
challenges to collaboration with industry partners compared with research organisations. In 
interviews, representatives of beneficiary organisations noted that cooperation with industry 
and  business  was  highly  reliant  to  both  the  subject  of  their  research  and  the  size  of  the 
industry  partner  concerned,  with  larger  companies  being  deemed  more  capable  of 
collaboration than SMEs. In addition to this, cooperation with industry partners often  faced 
significant  complications  surrounding  the  ownership  of  the  intellectual  property  generated. 
Despite  these  challenges,  some  respondents  indicated  that  collaborating  with  the  private 
sector had produced tangible outputs related to generating impacts for end-users, such as 
the creation of spin-offs that can bring research outputs to market. 

Findings from the interview programme also provide additional insights into the challenges 
and  barriers  faced  by  ERC and  WIDERA  beneficiary  organisations  when  establishing  and 
sustaining intersectoral collaborations. ERC beneficiary organisations indicated that working 
with companies on ERC projects can involve challenges such as administrative burden and 
the need to establish a clear understanding of the project’s goals. In some cases, companies 
may already be at a different stage of research or implementation, making collaboration less 
suitable  for  ERC  projects  focusing  on  basic  research.  WIDERA  beneficiary  organisations 
indicated that they found it very difficult to collaborate with industry, especially when it comes 
to  funding.  Academics  and  industry  have  different  views  on  research,  timeframes  and 
publication.  Many  industrial collaborations  are more  like service  agreements,  whereby  the 
industrial partner orders services and pays for them, without sharing background information. 
Offering  services  and  project  opportunities  to  industry  requires  academic  institutions  to 
employ  business-oriented 
individuals  who  understand  science  and  actively  seek 
collaboration. 

1.3.5.  EU added value 

The  survey  of  unsuccessful  Horizon  Europe  applicants  highlighted  how  unsuccessful 
organisations had to adapt their projects after not receiving Horizon Europe funding. Overall, 
the findings suggest that both unsuccessful organisations under both MSCA and WIDERA 
(widening participation) had to scale down and simplify their projects substantially after not 
receiving  Horizon  Europe  funding,  with  applicants  to  MSCA  undertaking  slightly  more 
pronounced changes.  

Most  of  the  adaptations  about  which  unsuccessful  MSCA  organisations  were  asked  were 
implemented by more than 80% of those surveyed. These adaptations included significant 
changes to the project’s scope (85% of respondents), the complexity of its methods (84%), 
and reduced outputs and innovations (81%). A high percentage of respondents also reported 
a decrease in the number of consortium partners, while 82% noted less diversity among the 
project  partners.  Adaptations  were  also  made  by  a  significant  proportion  of  unsuccessful 
WIDERA  organisations,  but  these  shares  were  generally  slightly  lower  than  those  for 
unsuccessful  MSCA  organisations.  Substantial  reductions  in  project  scope  (75%  of 
respondents), complexity (83%) and duration (70%) were noted by respondents to the survey 
of unsuccessful WIDERA organisations.  

497 

 
85%

75%

84%

83%

81%

74%

70%

63%

89%

78%

82%

67%

100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%

MSCA
(n=78)

WIDERA
(n=30)

MSCA
(=n76)

WIDERA
(n=33)

MSCA
(n=74)

WIDERA
(n=25)

MSCA
(n=65)

WIDERA
(n=28)

MSCA
(n=82)

WIDERA
(n=31)

MSCA
(n=73)

WIDERA
(n=26)

A smaller project
scope in number of
research (sub)areas
covered

Less
complex/diverse
set of research
methods compared
to the ones planned
in Horizon Europe
project

Smaller amount of
research outputs
and innovations to
be produced
compared to the
ones planned in
Horizon Europe
project

A shorter duration
of the project
compared to the
planned Horizon
Europe project

Smaller number of
consortium
partners involved
compared to the
planned Horizon
Europe project

Less diversity in
project partners
involved (e.g., more
international
partners, partners
from other sectors,
etc.)

Figure 14. Compared with the activities proposed in your Horizon Europe project application, how was your project adapted? 
(unsuccessful applicant organisations). Source: survey of unsuccessful Horizon Europe applicants, conducted May-July 
2023. 

The survey of unsuccessful Horizon Europe applicants also revealed various ways in which, 
and degrees to which, unsuccessful ERC applicants’ projects had to be altered or reduced in 
terms of their scope, complexity, outputs, duration or conditions, due to the lack of Horizon 
Europe  funding.  The  main  findings  indicate  that  unsuccessful  ERC  applicants  were  more 
likely to make significant adjustments to their projects than unsuccessful MSCA applicants. 
Specifically,  unsuccessful  ERC  applicants  more  frequently  reported  reducing  the  project's 
scope in terms of research areas (92% of respondents), simplifying research methods (87%), 
producing fewer research outputs and innovations (89%), shortening the project’s duration 
(91%). Thy also reported facing less favourable research conditions, including reduced salary 
(85%). In contrast, these adaptations were reported by fewer unsuccessful MSCA applicants, 
with rates ranging from 47% for shorter project duration, to 66% for less complex research 
methods. The data suggest that not receiving Horizon Europe funding had a more profound 
impact  on  unsuccessful  ERC  applicants  compared  with  unsuccessful  applicants  to  the 
MSCA. 

92%

87%

89%

91%

63%

66%

61%

47%

85%

63%

100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%

MSCA
(n=246)

ERC
(n=501)

MSCA
(n=240)

ERC
(n=447)

MSCA
(n=225)

ERC
(n=465)

MSCA
(n=178)

ERC
(n=464)

MSCA
(n=236)

ERC
(n=423)

A smaller project scope
in number of research
(sub)areas covered

Less complex/diverse
set of research methods
compared to the ones
planned in Horizon
Europe project

Smaller amount of
research outputs and
innovations to be
produced compared to
the ones planned in
Horizon Europe project

A shorter duration of
the project compared to
the planned Horizon
Europe project

Less favourable
research conditions
(including e.g. salary)

Figure 15. Compared with the activities proposed in your Horizon Europe project application, how was the project adapted? 
(unsuccessful applicants). Source: survey of Horizon Europe unsuccessful applicants, conducted May-July 2023. 

498 

 
 
 
1.4.  Conclusions  

Relevance 

Pillar 1 – Excellent Science and the WIDERA programme parts under Horizon Europe exhibit 
a robust and integrative strategy for human capital development, with the ERC, MSCA and 
WIDERA offering a comprehensive suite of funding opportunities for researchers at varying 
career  stages.  These  programmes  synergistically  bolster  excellence,  mobility  and  talent 
development.  The  MSCA  emphasise  the  professional  growth  and  international  mobility  of 
researchers,  while  the  ERC  emphasises  high-risk/high-reward  research  led  by  pioneering 
investigators.  WIDERA  strategically  addresses  regional disparities  in  research  and  fosters 
career enhancement for researchers.  

Within the Framework Programme parts analysed, there is a distinct focus not just on direct 
funding,  but  also  on  implementing  mechanisms  and  tools  designed  for  improved 
coordination, knowledge transfer and active interaction among beneficiaries, researchers and 
policy-makers.  Evidence  from  the  interviews  carried  out  for  this  case  study  indicates  a 
positive  reception  for  the  support  provided  by  the  European  Commission,  particularly  in 
navigating  the  intricacies  of  the  Framework  Programme’s  documentation.  Furthermore, 
evidence  suggests  that  there  is  clear  awareness  of  the  available  funding  opportunities 
provided by programme parts in Pillar 1.  

Coherence 

The case study also revealed coherence and complementarities between the MSCA and the 
ERC.  As  revealed  in  interviews,  MSCA  researchers  consider  participation  in  the  ERC  a 
desired  next  step  in  a  researcher’s  career  following  their  participation  in  the  MSCA.  As 
interviewees  observed,  not  only  do  former  MSCA  researchers  apply  for ERC  funding,  but 
many  successfully  receive  it.  Therefore,  the  knowledge  and  skills  acquired  during  their 
participation in MSCA can be further advanced and applied using the funding received from 
ERC. 

Useful  coherence  also  exists  between  the  MSCA  Staff  Exchanges  and  ERA  Talents 
schemes. Both schemes promote innovative collaborations in R&I through the exchange of 
staff and sharing of knowledge and ideas at all stages of the innovation chain. However, while 
MSCA  Staff  Exchanges  encompass  collaborations  on  international,  inter-sectoral  and 
interdisciplinary  levels,  the  ERA  Talents  scheme  aims  solely  to  increase  inter-sectoral 
cooperation,  with  a  specific  focus  on  the  widening  countries.  Also,  unlike  MSCA  Staff 
Exchanges,  the  ERA  Talents  scheme  enables  same-country  exchanges  on  condition  that 
they serve the specific purpose of maintaining or reinforcing regional innovation ecosystems. 

Effectiveness  

According  to  the  European  Commission  monitoring  data,  48,637  researchers  are  so  far 
participating in  the  Framework  Programme,  with  11,185  individual  researchers  involved  in 
the  programme  parts  under  Pillar  1  and  WIDERA.  However,  the  Framework  Programme 
currently  displays  a  gender  disparity,  with  male  researchers  accounting  for  63%  of 
participants, while female participation stands at 37%. Similar trends can be seen when Pillar 
1 is broken down by programme parts, with the exception of the WIDERA programme part, 
which  shows  progress  towards  gender  balance.  Participation  rates  of  men  and  women  in 
WIDERA currently stand at 46 % and 54 %, respectively.  

499 

 
The  survey  of  Horizon  Europe  beneficiaries  carried  out  for  this  study  underlines  the 
robustness of the Pillar 1 programme parts in fortifying the core research competencies of 
their beneficiaries. A majority of MSCA Postdoctoral Fellows and ERC principal investigators 
anticipate  a  significant  enhancement  of  traditional  research  skills,  notably  in  advanced 
scientific  methods,  interdisciplinary  techniques  and  written  communication.  However,  a 
discernible  gap  exists  with  regard  to  training  associated  with  the  application  of  research, 
particularly in the commercialisation of results, entrepreneurship and digital technologies. At 
an  organisational  level,  both  MSCA  and  WIDERA  (widening  participation)  beneficiaries 
express  strong  optimism  concerning  institutional  advances  in  researcher  training  and 
sustained professional growth.  

The programme parts under Pillar 1 and WIDERA (widening participation) were also found 
to contribute to the formation of human capital in R&I by emphasising aspects such as career 
advice and the scientific supervision of researchers. This was found to be particularly evident 
in  programme  parts  such  as  WIDERA  (widening  participation)  and  MSCA.  The  survey  of 
Horizon Europe beneficiaries revealed strong expectations on the part of respondents that 
participation in Horizon Europe projects would lead to improvements in the quality of scientific 
supervision  (76%  for  MSCA  beneficiary  organisations,  67%  for  WIDERA)  and  the  quality 
provision of career advice for researchers (64% for MSCA beneficiary organisations; 63% for 
WIDERA).  

Both  the  WIDERA  (widening  participation) and  MSCA  programmes  under  Horizon  Europe 
are  strongly  geared  towards  enhancing  transparency  with  regard  to  recruitment  and 
improving  working  conditions  for  researchers  in  Europe  and  beyond.  By  endorsing  the 
principles of the European Charter for Researchers and the recruitment Code of Conduct, 
these programmes drive organisations to adopt open and unbiased hiring practices. Based 
on  the  findings  of  the  survey  of  Horizon  Europe  beneficiaries,  beneficiary  organisations 
participating  in  MSCA  actions  expect  improvements  to  a  large  or  very  large  extent  in  the 
quality and transparency of recruitment practices (56%) and the quality of working conditions 
for  researchers  (52%).  The  corresponding  shares  are  found  to  be  similar  for  beneficiary 
organisations participating in WIDERA, at 52% and 51% respectively.  

The  present  study  also  finds  that  MSCA  PF  fellows  and  ERC  principal  investigators 
emphasise  the  centrality  of  research  publications  as  the  main  anticipated  outcome 
stemming from projects funded by Horizon Europe. When looking from the perspective 
(widening 
of  participating  organisations,  besides  publications,  both  WIDERA 
participation) and MSCA participants expressed a more pronounced inclination towards 
producing outputs related to training, education and policy engagement.  

In  addition,  this  case  study  concludes  that  open  access  (OA)  plays  a  pivotal  role  in 
democratising  access  to  knowledge  and  fostering  a  culture  of  shared  learning  and 
collaboration,  but  that  participants  wishing  to  publish  in  OA  face  several  hurdles, 
according  to  feedback  from  MSCA  and  WIDERA  (widening  participation)  beneficiary 
organisations. These challenges relate to high costs and concerns over the quality and 
impact of OA journals. The perceived lack of rigorous peer review on some OA platforms 
raises further doubts about their credibility. 

EU added value   

Unsuccessful applicant organisations highlighted the challenge of accessing essential 
knowledge, expertise and infrastructures without Horizon Europe funding. Evidence also 
points to the critical role Horizon Europe plays in facilitating international collaborations. 
Among  unsuccessful  applicant  organisations,  majorities  of  those  who  had  applied  to 

500 

 
 
MSCA  (89%  of  applicants)  and  WIDERA  (widening  participation)  (86%  of  applicants) 
both  reported  being  unable  to  implement  their  project  because  the  involvement  of  a 
variety of international partners was essential to the project’s success, and it could not 
be implemented without them. 

Appendix 1. Literature sources 

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 
Marie Skłodowska-Curie actions guidelines on supervision, Luxembourg: Publications 
Office of the European Union, 2021, available at: 
https://data.europa.eu/doi/10.2766/508311   

•  European Commission, Directorate-General for Research and Innovation, Nixon, J., 
Study to support the monitoring and evaluation of the framework programme for 
research and innovation along key impact pathways: baseline and benchmark report, 
Nixon, J. (editor), Luxembourg: Publications Office of the European Union, 2022, 
available at: https://data.europa.eu/doi/10.2777/98156.  

•  European Commission, Directorate-General for Research and Innovation, Nixon, J., 
Study to support the monitoring and evaluation of the framework programme for 
research and innovation along key impact pathways: indicator methodology and 
metadata handbook, Nixon, J. (editor), Luxembourg: Publications Office of the 
European Union, 2022, available at: https://data.europa.eu/doi/10.2777/44653.  

•  European Commission, Directorate-General for Research and Innovation, Evaluation 
study on the external coherence and synergies of Horizon 2020 within the European 
research and innovation support system: case study report, Luxembourg: Publications 
Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/054469.  

•  European Commission, Directorate-General for Research and Innovation, Naujokaitytė, 
R., Mačiulienė, M., Papageorgiou, H., et al., Evaluation study on the implementation of 
cross-cutting issues in Horizon 2020: study in support of the ex-post evaluation of the 
European Framework Programme for Research and Innovation Horizon 2020, Dirham, 
S. (editor), Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/763665.  

•  European Commission, Directorate-General for Research and Innovation, Newcomers 
in EU R&I programmes : main trends in Horizon 2020, first evidence from Horizon 
Europe, Publications Office of the European Union, 2023, 
https://data.europa.eu/doi/10.2777/911220.  

•  European Commission, Directorate-General for Research and Innovation, Country 

participation : from Horizon 2020 to Horizon Europe, Luxembourg: Publications Office 
of the European Union, 2022, available at: https://data.europa.eu/doi/10.2777/210660.  

•  European Commission, Directorate-General for Research and Innovation, Naujokaitytė, 
R., Mačiulienė, M., Papageorgiou, H., et al., Evaluation study on the implementation of 
cross-cutting issues in Horizon 2020 : study in support of the ex-post evaluation of the 
European Framework Programme for Research and Innovation Horizon 2020, Dirham, 
S. (editor), Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/763665. 

501 

 
•  European Commission, Interim Evaluation of Horizon 2020. Staff Working Document 

SWD(2017), 2017. 

•  European Commission, Work Programme 2021-2022: Widening participation and 

strengthening the European Research Area, 2021-2022, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2021-2022/wp-11-widening-participation-and-strengthening-the-european-
research-area_horizon-2021-2022_en.pdf. 

•  European Commission, European Commission announces support for 124 researchers 
who fled the war through #MSCA4Ukraine. Marie Skłodowska-Curie Actions, 2023, 
available at: https://marie-sklodowska-curie-actions.ec.europa.eu/news/european-
commission-announces-support-for-124-researchers-who-fled-the-war-through-
msca4ukraine. 

•  European Commission, Evaluation study on Excellent Science in the European 

Framework Programmes for Research and Innovation. Phase 1 Final Study Report and 
its annexes, 2023. 

•  European Commission, Work Programme 2023: European Research Council, 2023, 

available at: 
https://erc.europa.eu/sites/default/files/document/file/ERC_WorkProgramme_2023.pdf. 

•  European Commission, Work Programme 2023-2024. Research Infrastructures, 

2023,available at: https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/wp-call/2023-2024/wp-3-research-infrastructures_horizon-2023-
2024_en.pdf. 

•  European Commission, Work Programme 2023-2024: Marie Skłodowska-Curie 

Actions, 2023, available at: https://ec.europa.eu/info/funding-
tenders/opportunities/docs/2021-2027/horizon/wp-call/2023-2024/wp-2-msca-
actions_horizon-2023-2024_en.pdf. 

•  European Commission, Work Programme 2021-2022: Marie Skłodowska-Curie 

Actions, 2021, available at: https://ec.europa.eu/info/funding-
tenders/opportunities/docs/2021-2027/horizon/wp-call/2021-2022/wp-2-msca-
actions_horizon-2021-2022_en.pdf. 

•  European Commission, Work Programme 2023-2024: Widening participation and 

strengthening the European Research Area, 2023-2024, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2023-2024/wp-11-widening-participation-and-strengthening-the-european-
research-area_horizon-2023-2024_en.pdf    

•  European Research Council, Science Journalism Initiative, 2022, available at: 

https://erc.europa.eu/apply-grant/science-journalism-initiative. 

•  European Commission, Research and Innovation: Gender equality in research and 

innovation, , 5 May 2023, retrieved from: https://research-and-
innovation.ec.europa.eu/strategy/strategy-2020-2024/democracy-and-rights/gender-
equality-research-and-innovation_en   

502 

 
•  Horizon Dashboard data, accessed 21 September 2022. Data provided by the 

European Commission indicate that 16,712 distinct researchers have so far received 
access to research infrastructures through INFRA; under FP7, this figure reached 
21,060. 

•  Horizon Europe Dashboard (Projects and Results).  

•  MSCA NET, MSCA-NET Project Aims at Improving, Professionalising and Harmonising 
Services of the Marie Skłodowska-Curie Actions (MSCA) NCPs, 2 May 2023, available 
at: https://msca-net.eu/.  

•  Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 
2021 establishing Horizon Europe – the Framework Programme for Research and 
Innovation, laying down its rules for participation and dissemination, and repealing 
Regulations (EU) No 1290/2013 and (EU) No 1291/2013 (Text with EEA relevance). 

•  Research project database (CORDIS). 

•  Survey of Horizon 2020 beneficiary organisations, conducted on August-September 

2022. 

•  Survey of Horizon 2020 ERC beneficiaries, conducted on August-September 2022. 

•  Survey of MSCA fellows (IF, ITN, COFUND, RISE), conducted for the study on mobility 

flows in the context of the MSCA, 2022. 

Appendix 2. List of interviews (anonymised) 

Nine interviews have been carried out with the following coordinators of the MSCA, WIDERA 
and ERC projects: 

•  Coordinator of the project in Horizon-MSCA-DN-2021 

•  Coordinator of the project in HORIZON-WIDERA-2022-TALENTS-03 

•  Coordinator of the project in HORIZON-WIDERA-2022-ERA Fellowships 

•  Coordinator of the project in HORIZON-ERC-ADG 

•  Coordinator of the project in HORIZON-ERC-COG 

•  Coordinator of the project in HORIZON-WIDERA-2022-TALENTS-03 

•  Coordinator of the project in HORIZON-WIDERA-2022-TALENTS-03 

•  Coordinator of the project in HORIZON-ERC-ADG 

•  Coordinator of the project in Horizon-MSCA-DN-2021 

503 

 
 
Appendix 3. Interview questionnaire  

Evaluation 
criteria 

Relevance 

Interview Questions 

• 

• 

• 

Please assess how well the Framework Programme responds to the 
newly emerging needs/ changes in the political, social and economic 
landscape within Europe with regard to human capital in research and 
innovation. 
Please assess how well the Framework Programme addresses the needs 
of researchers in terms of the available tools and thematic areas covered 
by Horizon Europe.  
Please assess the level of awareness and knowledge of supporting tools 
available and thematic areas covered by the Framework Programme. Is 
the research and innovation community well informed on the 
opportunities provided by Horizon Europe? What could be improved in 
this regard? 

Coherence 

•  Could you assess the coherence of the Framework Programme in its aim 

to strengthen human capital in research and innovation? Are there any 
significant overlaps or gaps within the FP? 

•  Could you assess the level of external coherence of Horizon Europe 
within the European R&I in terms of human capital formation? 

Effectiveness 

•  What are the key areas in which the programme part contributes to 

• 

• 

• 

• 

• 

• 

• 

• 

strengthening human capital? 
To what extent did the programme part contribute to developing 
structured training and career development programmes for researchers, 
and were there any noticeable gaps? 
To what extent does the programme part support the acquisition of 
interdisciplinary and transferable skills? 
Are there any major gaps in 1) the quality of training offered, or 2) the 
breadth of training offered? 
To what extent has the programme part increased access to relevant 
research infrastructure that also contributes to human capital formation? 
Are there any gaps? 
To what extent did the programme part support access to know-how in 
the form of supervision and promoting the ability to collaborate with 
colleagues at the host/beneficiary institution and between organisations 
within the consortium? 
Please assess the extent to which the programme part contributed to 
ensuring the project’s alignment with the principles of Responsible 
Research and Innovation (RRI). 

To what extent are the skills/training opportunities provided by the 
programme part sufficient? What, if any, gaps are there, and why? 
To what extent does the programme part support intersectoral and 
interdisciplinary cooperation? What are the gaps in this area? 

Efficiency 

EU added value 

•  Could you have implemented the same project activities through other 
EU, national or regional support? If yes, would your project have been 
different in terms of scope, timeframe or results in relation to excellence? 

504 

 
 
Annex 2.11: Fostering the diffusion of knowledge and open access 

1.1. 

Introduction 

Open Science plays a pivotal role in cultivating a robust scientific system, endowing Europe 
with a competitive advantage through the utilisation of open knowledge. Open Science strives 
to facilitate faster and improved access to research outcomes; to promote interdisciplinary 
collaborations; to accelerate innovation; to support informed public decision-making; and to 
actively  engage  the  public  in  the  research  process  and  in  collective  problem-solving 
endeavours. 

For many years, the European Commission has expressed the ambition for Open Science to 
become  the  norm  for  research  and  innovation  (R&I)  across  the  European  Research  Area 
(ERA).  This  is  because  Open  Science  is  said  to  have  the  potential  to  foster  the  quality, 
efficiency and impact of R&I, leading to greater responsiveness to societal challenges, and 
increasing trust of society in the science system. This ambition has been manifested in many 
strategies and initiatives taken throughout the Framework Programmes (FPs). For instance, 
under FP7, the European Commission conducted a pilot initiative, the Open Access Pilot, to 
ensure open access to peer-reviewed research articles. Under Horizon 2020, this pilot was 
extended and other novelties were introduced: the Model Grant Agreement specified rules 
regarding  the  dissemination  of  results  and  data,  and  the  programme  supported  multiple 
initiatives to maximise knowledge diffusion and Open Science (e.g., through SwafS projects, 
a Mutual Learning Exercise, etc.). Horizon Europe embraces Open Science as the  modus 
operandi for all researchers817.  

For the purposes of this case study, open access is understood, according to the definition 
used by the European Commission, as an ‘approach to the scientific process that focuses 
on  spreading  knowledge  as  soon  as  it  is  available  using  digital  and  collaborative 
technology’. This definition is in line with that used for the third Key Impact Pathway (KIP) 
under Horizon Europe, namely: ‘Fostering diffusion of knowledge and open science’, which 
focuses  on  research  outputs  and  does  not  include  other  types  of  research  tools  or  data. 
Nevertheless,  ‘Open  Science’  is  a  much  broader  concept.  Specifically,  ‘Open  Science 
consists of the sharing of knowledge, data and tools as early as possible in the Research and 
Innovation (R&I) process, in open collaboration with all relevant knowledge actors, including 
quadruple  helix  actors,  i.e.  academia,  industry,  public  authorities,  end  users,  citizens  and 
society at large’818,819. 

The present case study begins by shedding light on the progress made by the FPs in fostering 
the  production  of  open  access  outcomes.  Given  that  Horizon  Europe  commences  only 
recently, and the publication of research outputs requires a certain amount of time, this case 

817 European Commission, Directorate-General for Research and Innovation, Horizon Europe, 
open science – Early knowledge and data sharing, and open collaboration, Luxembourg: 
Publications Office of the European Union, 2021, available at: 
https://data.europa.eu/doi/10.2777/18252. 
818 This would belong to another Key Impact Pathway (6), and is currently being covered by the 
parallel Evaluation Study for an Innovative Europe under the European Commission´s 
Framework Programmes for R&I. 
819 European Commission, Directorate-General for Research and Innovation, Horizon Europe, 
open science – Early knowledge and data sharing, and open collaboration, Luxembourg: 
Publications Office of the European Union, 2021, available at: 
https://data.europa.eu/doi/10.2777/18252. 

505 

 
 
study  concentrates  on  the  outcomes  observed  under  Horizon  2020  (2014-2020),  while 
providing preliminary insights, where possible, into Horizon Europe. At the same time, the 
case  study  also  presents  figures  on  this  type  of  outcomes  at  national  level.  In  this way,  it 
provides  an  overview  of  performance  in  the  production  of  open-access  research 
outputs  across  the  Member  States  in  general,  as  well  as  within  the  EU’s  FPs,  thus 
allowing trends to be compared.  

Furthermore,  the  case  study  takes  a  broader  view  by  examining  whether  the  FPs  are 
perceived as being successful in supporting the uptake of Open Science practices at national 
level. As such, the case study sheds light on the role played by the FPs in promoting the 
adoption of Open Science practices at national level, and the challenges encountered.  

The present case study is based on desk research, and on the findings from the bibliometric 
analyses  performed  in  the  context  of  the  Evaluation  study  on  Excellent  Science  under 
Horizon 2020, as well as interview evidence from several national policy-makers and experts 
(e.g. representatives of national funding agencies and of transnational Open Science working 
groups) that are knowledgeable about Open Science practices in their respective countries. 
It  is  important  to  note,  therefore,  that  interview  evidence  is  based  on  a  limited  number  of 
interviews.  

1.2.  Background information 

1.2.1.  Policy context 

For  several  years,  the  European  Commission  has  acknowledged  Open  Science  as  a  key 
priority. As part of the Seventh Framework Programme (FP7), the European Commission 
launched  a  pilot  initiative  known  as  the  Open  Access  Pilot,  with  the  aim  of  guaranteeing 
unrestricted access to peer-reviewed research articles within specific domains. In addition, 
FP7 provided funding to set up and operate OpenAIRE, which plays a vital role in facilitating 
the implementation of Open Access principles throughout Europe. OpenAIRE operates as a 
network  that  encompasses  open  access  repositories,  archives  and  journals,  effectively 
cataloguing publications, datasets and other research outputs generated under the FP820. 

Under  Horizon  2020,  open  access  to  research  publications  became  an  obligation. 
Specifically, the Horizon 2020 Model Grant Agreement specified that each beneficiary must 
disseminate  their  results  as  soon  as  possible  by  disclosing  them  to  the  public  through 
appropriate  means  (other  than  those  that  result  from  protecting  or  exploiting  the  results), 
including  scientific  publications.  It  also  requires  that  they  ensure  open  access  (i.e.  free  of 
charge, online access by any user) to all peer-reviewed scientific publications that relate to 
its results821. Furthermore, the Open Research Data Pilot (ORDP) was introduced during the 
2014-2015  Work  Programme.  The  primary  objective  of  this  pilot  was  to  enhance  and 
maximise access to, and ensure the reusability of research data generated by projects funded 
under  Horizon  2020.  In  2017,  the  pilot  was  expanded  to  encompass  all  thematic  areas  of 
Horizon 2020 by default, adhering to the principle of being ‘as open as possible, as closed 
as necessary’. This meant that projects had the option to withhold research data from open 
access if there was a valid reason (i.e. opting out). In addition, the requirement to have a data 
management plan (DMP) was introduced. The DMP outlines the data management lifecycle 
for the data collected, processed, and/or generated within Horizon 2020 projects. 

820 https://www.openaire.eu/.  
821 https://ec.europa.eu/research/participants/data/ref/h2020/grants_manual/amga/h2020-
amga_en.pdf, accessed on 17 October 2023. 

506 

 
 
Under  Horizon  Europe,  the  principles  of  Open  Science  are  embedded  throughout  the 
programme,  and  have  evolved  to  become  it  standard  operating  procedure  or  modus 
operandi.  Specifically,  Open  Science  is  included  in  Article  14  of  the  Horizon  Europe 
Regulation, in Horizon Europe work programmes, proposal evaluations, grant agreements, 
project  execution  and  follow  up,  and  programme  evaluation822.  In  particular,  the  following 
Open  Science  practices  are  required  for  all  Horizon  Europe  beneficiaries  in  each  grant 
agreement,823: 

•  To provide immediate (without embargoes) open access to scientific peer-reviewed 

publications, by depositing them in trusted repositories824.  

•  To align the management of research data with the FAIR principles825. This involves 
creating  a  research  data  management  plan  (DMP)  as  a  deliverable  early  on  in  the 
projects (typically by month six) and regularly updating it. It also concerns open access 
to research data under the principle ‘as open as possible, as closed as necessary’. 

•  To provide comprehensive information on outputs, tools and instruments, and granting 
access  to  the  data  and  results,  unless  there  are  legitimate  interests  or  specific 
constraints,  such  as  EU  competitive  interests826  -  to  enable  the  validation  of 
publications and the validation or re-use of research data. 

Horizon  Europe  also  recommends  other  Open  Science  practices827:  the  early  and  open 
sharing of research, which can be achieved through various methods such as preregistration, 
registered  reports,  or  pre-prints;  the  participation  in  open  peer-review  processes,  e.g. 
publishing research in open peer-reviewed journals and platforms; open access to research 
outputs  other  than  publications  and  research  data  (e.g.  software,  models,  algorithms, 
workflows, etc.) by depositing them in trusted repositories. 

At the same time, Horizon Europe introduced an ambition to enhance the diversity of impacts 
that result from EU R&I funding, and hence incorporated a novel approach known as  Key 
Impact  Pathways  (KIP),  which  provide  a  framework  for  identifying  and  assessing  the 
broader scientific, societal and economic impacts that projects can generate. The third KIP, 
‘Fostering diffusion of knowledge and open science’, stresses the importance of Open 
Science in developing stronger European science systems, and aims to monitor the impact 

822 SWD(2023)132 final. (2023). Evidence Framework on monitoring and evaluation of Horizon 
Europe. Brussels, 27 April 2023. 
823 Horizon Europe (HORIZON). Programme Guide. Version 3.0, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/guidance/programme-guide_horizon_en.pdf  
824 Subject to a Creative Commons Attribution International Public Licence (CC BY) or equivalent 
licence. 
825 Findable, Accessible, Interoperable, and Reusable. 
826 In addition, in situations of public emergency, upon request by the granting authority, there is 
a requirement for immediate open access to all research outputs under open licenses such as 
CC BY, CC0, or equivalent licences. 
827 Horizon Europe (HORIZON). Programme Guide. Version 4.0, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/guidance/programme-guide_horizon_en.pdf, accessed on 14 March 2024. 

507 

 
 
of  the  Open  Science-driven  diffusion  of  knowledge  on  scientific  discovery  and  knowledge 
spillovers828. 

Box 1. KIP3, ‘Fostering diffusion of knowledge and open science’ and its indicators 

Each KIP distinguishes specific indicators in terms of its effects over different timeframes 
– i.e. in the short, medium, and long term. The figure below presents the three specific 
indicators relating to KIP3. 

Short-term: 
Shared knowledge

Medium-term: 
Knowledge diffusion

Long-term: 
New collaborations

• The share of FP 
research outputs that 
are open-access

• The share of open-
access FP research 
outputs that are 
actively used/cited

• FP beneficiaries 
having developed new 
transdisciplinary/trans-
sectoral collaborations 
with users of their 
open FP R&I outputs

The underlying principle behind the above indicators is that open access to publications, 
datasets,  software  and  other  research  outputs  facilitates  researchers  in  leveraging 
existing  work  and  accelerating  the  pace  of  scientific  advancement  in  the  short term.  It 
provides industry with access to scientific knowledge for innovation, enables evidence-
based policy-making in the public sector, and actively involves the public in the research 
process. Hence, identifying and isolating outputs associated with Open Science serves 
as  an  appropriate  measure  of  the  progress  made  in  the  short  term  towards  achieving 
these objectives.  

In the medium term, the key question arises as to whether open access research results 
are  more  cited,  compared  with  closed  or  embargoed  results.  This  indicator  thereby 
captures the influence that open access has on research.  

While research collaborations have always been key objectives of the FPs, one of the 
premises  of  adopting  Open  Science  as  their  modus  operandi  is  to  further  strengthen 
transdisciplinary  and  trans-sectoral  collaboration,  which  is  captured  in  the  long-term 
indicator.  This  indicator  captures  the  share  of  beneficiaries  who  have  developed  new 
transdisciplinary/trans-sectoral  collaborations  with  users  of  their  open  access  outputs 
produced under the FP.  

Source: compiled by the study team, based on the KIP Indicator Methodology and Metadata Handbook829. 

828 European Commission, Directorate-General for Research and Innovation, Nixon, J., Study to 
support the monitoring and evaluation of the Framework Programme for Research and 
Innovation along key impact pathways – Indicator methodology and metadata handbook, Nixon, 
J.(editor), Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/44653. 
829 European Commission, Directorate-General for Research and Innovation, Nixon, J., Study to 
support the monitoring and evaluation of the Framework Programme for Research and 
Innovation along key impact pathways – Indicator methodology and metadata handbook, Nixon, 

508 

 
 
 
 
1.2.2.  Overall approach and scope of the case study 

The present case study relies on several methodologies to achieve its objectives.  

•  Comprehensive desk research was conducted on reports and other documents in the 
field of Open Science in the EU (see Appendix 1 for an overview of the literature sources). 
Specifically, reports on the status of Open Science in the EU were consulted as well as 
documentation relating to Open Science practices in the FPs. In addition, the findings of 
the  public  consultation  on  the  past,  present  and  future  of  the  EU’s  Framework 
Programmes for R&I were also considered in the analysis. 

•  The bibliometric analysis of Horizon 2020 peer-reviewed publications conducted in the 
context  of  the  Evaluation  study  on  Excellent  Science  under  Horizon  2020  provided 
insights into the performance of the FP in generating open access outputs.  

• 

Interviews with stakeholders provided insights into the factors hindering and fostering 
the impact of the FP in supporting the adoption of Open Science practices at national 
level.  The  seven  interviewees  were  all  experts  on  the  topic  of  Open  Science  (i.e.  in 
working  groups  or  expert  groups  on  Open  Science  in  stakeholder  organisations  and 
networks).  Typically,  their  occupation  is  related  to  supporting  science  (and  Open 
Science) in a particular Member State.  

1.3.  Case study findings 

This  section  of  the  case  study  begins  by  presenting  an  overview  of  performance  across 
Member States in relation to the production of open access research outputs in general, and 
within the EU’s Framework Programmes. It then examines the extent to which Open Science 
is seen as a priority by stakeholders for the next Strategic Plan (2025-2027), as well as the 
multiple  aspects  that  would  need  to  be  considered  in  this  process,  as  identified  during 
interviews and in the desk research. 

1.3.1.  Uneven production of open access research outputs across the Member 

States 

Worldwide, the EU is a frontrunner in the production of open access publications, as is 
depicted  in  Figure  1830.  A  recent  EC  report831,832  indicates  that  in  2019,  39.2% of  the  total 
volume of EU publications is freely available through an open access publishing pathway. 
This represents an increase of 5.3 percentage points compared with 2009. Nevertheless, the 

J.(editor), Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/44653. 
830 European Commission, Directorate-General for Research and Innovation, Science, research 
and innovation performance of the EU 2022: building a sustainable future in uncertain times, 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/78826. 
831 European Commission, Directorate-General for Research and Innovation, Science, research 
and innovation performance of the EU 2022 : building a sustainable future in uncertain times, 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/78826. 
832 The data from the aforementioned EC report are used throughout Section 1.3.1 of this case 
study. It provides information on open access scientific publications (i.e. green, gold or other 
open access routes) with DOI as a percentage of total scientific publications. It relies on the 
Scopus database and 1findrdatabases, and uses full counting. The data as presented by the 
report can be found at: https://ec.europa.eu/assets/rtd/srip/2022/figure-6-1-6.xlsx  

509 

 
 
United States, Japan, Canada and South Korea follow behind closely with shares of between 
34.6% and 38.3% in 2019. While many regions, like the EU, have seen an increase in the 
shares  of  publications  that  are  open  access,  the  United States  and  Canada  actually have 
fewer  publications  in  open  access  in  2019  relative  to  2009.  A  recent  Canadian  article833 
argues that while Canada was at the forefront of open access policy a decade ago, current 
policies are dated and Canada needs to drastically revise them. In addition, there appear to 
be  large  differences  between  Canadian  funders  in  terms  of  the  number  of  open  access 
publications they generate. As the most important barrier to open access publishing, the cost 
of  publishing  in  open  access  venues  is  mentioned,  along  with  a  lack  of  support  from 
universities in encouraging open access publishing and the fact that no tangible benefits are 
associated with open access publishing in the perception of many Canadian researchers. In 
the United States, open access policy continues to gain importance as open access directly 
contributes  to  the  government’s  combined  goals  of  accelerating  scientific  discovery, 
strengthening  translation  and  policymaking,  and  lowering  barriers  to  access  for  all  of 
America834.  A  recent  memo  from  the  Executive  Office  of  the  President  directs  federal 
departments and agencies to update their public access policies by the end of 2025 in order 
to make taxpayer-funded publications and research publicly accessible, without delay or cost. 

39,2%

33,9%

43,3%

38,3%

37,6%

35,5%

38,9%

36,6%

34,6%

29,9%

23,0%

14,8%

50%

40%

30%

20%

10%

0%

EU

United States

Japan

Canada

South Korea

China

Share of OA (2009)

Share of OA (2019)

Figure 1. Share of scientific publications in open access, by country/region. Note: these shares refer to the open access 
scientific publications with digital object identifier (DOI) as a percentage of total scientific publications in each country/region 
with DOI in the year 2009 and 2019; OA = open access. Source: Science, Research and Innovation Performance of the EU 
2022835. 

Zooming in on the shares of open access publications in different EU Member States (EU-
28, including the United Kingdom), significant variations become noticeable (see Figure 2). 
The  Netherlands  leads  with  58.3%  of  its  scientific  publications  being  published  in  open 
access, very closely followed by the United Kingdom with 57.8%. In Hungary, Sweden and 
Croatia, more than half of the publications are also open access. Those countries lagging 

833 Brian Owens (2023). Catching up on open access. Accessed on 17 October 2023; available 
at: https://www.universityaffairs.ca/features/feature-article/catching-up-on-open-access/. 
834 As discussed in the Nelson Memo from the Executive Office of the President – Office of 
Science and Technology Policy, entitled ‘Ensuring Free, Immediate, and Equitable Access to 
Federally Funded Research’, 2022, available at: https://www.whitehouse.gov/wp-
content/uploads/2022/08/08-2022-OSTP-Public-Access-Memo.pdf. Accessed on 17 October 
2023. 
835 European Commission, Directorate-General for Research and Innovation, Science, research 
and innovation performance of the EU 2022: building a sustainable future in uncertain times, 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/78826. 

510 

 
 
 
behind in terms of publishing in open access are Romania (25.1%), Bulgaria (25.7%) and 
Latvia  (28.9%).  In  some  countries,  namely,  Croatia,  Malta,  Slovakia  and Bulgaria,  a slight 
downward trend can be discerned when comparing open access rates in 2009 and 2019. In 
Malta, for example, a 2020 Policy Support Facility (PSF) report836 argued that, while Malta is 
an innovative country, it is not up to speed when it comes to practicing Open Science. The 
Maltese R&I system is characterised by a fairly conservative publication culture and a lack of 
structured data sharing. It therefore concluded that improving the circulation of knowledge 
through  the  introduction  of  an  open  access  and  open  data  policy  could  improve  scientific 
research. In 2021, to promote Open Science and open access in Bulgaria, a national plan for 
the development of Open Science initiatives was approved837. This plan defined a roadmap 
for the upcoming period (2021-2025) and instruments to encourage Bulgarian researchers 
and  research  institutions  to  implement  Open  Science  principles  in  practice.  Likewise,  in 
Slovakia,  the  government  adopted  a  National  Strategy  for  Open  Science  2021-2028838.  A 
hindering factor in Slovakia was its fragmented legal system, as more than 10 laws would be 
affected  in  order  to  implement  Open  Science  principles  at  national  level.  One  of  the  key 
reasons  for  the  country  lagging  behind  in  terms  of  open  access  rates  is  that  Slovak 
researchers  often  cannot  afford  the  high  article  processing  charges  if  they  are  not 
participating in large international projects839.  

%
3
8
5

,

%
8
7
5

,

%
3
4
5

,

%
9
2
5

,

%
7
0
5

,

%
0
9
4

,

%
6
8
4

,

%
1
8
4

,

%
8
4
4

,

%
3
4
4

,

%
3
4
4

,

%
1
4
4

,

%
2
3
4

,

%
0
2
4

,

%
9
0
4

,

%
4
0
4

,

%
9
9
3

,

%
5
9
3

,

%
5
9
3

,

%
3
9
3

,

%
3
8
3

,

%
8
7
3

,

%
5
4
3

,

%
1
4
3

,

%
8
0
3

,

%
9
8
2

,

%
7
5
2

,

%
1
5
2

,

70%

60%

50%

40%

30%

20%

10%

0%

y
r
a
g
n
u
H

n
e
d
e
w
S

a
i
t
a
o
r
C

a
i
r
t
s
u
A

d
n
a
n
i
F

l

i

n
a
p
S

k
r
a
m
n
e
D

i

a
n
e
v
o
l
S

i

a
n
a
u
h
t
i
L

d
n
a
e
r
I

l

s
d
n
a
l
r
e
h
t
e
N

i

m
o
d
g
n
K
d
e
t
i
n
U

i

m
u
g
e
B

l

d
n
a
o
P

l

i

a
n
o
t
s
E

y
n
a
m
r
e
G

l

a
g
u
t
r
o
P

e
c
n
a
r
F

a
t
l
a
M

s
u
r
p
y
C

y
l
a
t
I

e
c
e
e
r
G

i

a
h
c
e
z
C

a
i
k
a
v
o
l
S

a
i
v
t
a
L

a
i
r
a
g
u
B

l

i

a
n
a
m
o
R

g
r
u
o
b
m
e
x
u
L

Share of OA (2009)

Share of OA (2019)

Figure 2. Share of scientific publications in open access, by EU-28 Member State. Note: these shares refer to the open-
access scientific publications with digital object identifier (DOI) as a percentage of the total scientific publications with DOI in 

836 European Commission, Directorate-General for Research and Innovation, Specific support to 
Malta – ‘open access – An opportunity for Malta’ – Recommendations for the development of a 
national policy for open access to publications, research data and related issues, Luxembourg: 
Publications Office of the European Union, 2020, available at: 
https://data.europa.eu/doi/10.2777/835. 
837 National plan for the development of Open Science Initiatives in Bulgaria, available at: 
https://rda.bg/en/node/47, accessed 18 October 2023. 
838 https://www.openaire.eu/blogs/slovak-national-strategy-for-open-science-2021-2028. 
839 National Strategy for Open Science 2021-2028, Slovakia, available at: 
https://www.openaire.eu/blogs/slovak-national-strategy-for-open-science-2021-2028. 

511 

 
 
 
 
each country in the year 2009 and 2019; OA = open access. Source: Science, Research and Innovation Performance of the 
EU 2022.840 

Focusing  on  publications  produced  under  Horizon  2020,  variations  in  performance 
between countries can also be observed with regard to open access publications. The United 
Kingdom,  the  Netherlands  and  Luxembourg  are  the  top  three  countries  with  the  highest 
proportions  of  publications  available  in  open  access.  Conversely,  Romania,  Greece  and 
Estonia  are  the  countries  with  the  lowest  proportion  of  Horizon  2020  publications  in  open 
access (see Figure 3). 

91%

89%

88%87%87%87%87%87%86%

85%

84%84%

83%82%82%82%82%81%81%80%

79%79%78%78%

76%

71%

69%69%

95%

90%

85%

80%

75%

70%

65%

60%

55%

50%

i

n
a
p
S

e
c
n
a
r
F

n
e
d
e
w
S

d
n
a
n
i
F

l

y
r
a
g
n
u
H

k
r
a
m
n
e
D

s
d
n
a
l
r
e
h
t
e
N

g
r
u
o
b
m
e
x
u
L

i

m
o
d
g
n
K
d
e
t
i
n
U

i

m
u
g
e
B

l

a
i
r
t
s
u
A

a
i
t
a
o
r
C

d
n
a
o
P

l

a
t
l
a
M

a
i
v
t
a
L

d
n
a
e
r
I

l

i

a
h
c
e
z
C

y
l
a
t
I

y
n
a
m
r
e
G

a
i
r
a
g
u
B

l

l

a
g
u
t
r
o
P

i

a
n
e
v
o
l
S

s
u
r
p
y
C

i

a
n
a
u
h
t
i
L

a
i
k
a
v
o
l
S

i

a
n
o
t
s
E

e
c
e
e
r
G

i

a
n
a
m
o
R

Figure 3. Share of Horizon 2020 publications published in open access, EU-28 Member State. Source: bibliometric analysis 
conducted in the Evaluation study on Excellent Science under Horizon 2020841. 

Although  comparisons  between  figures  on  open  access  publications  in  the  EU  Member 
States  in  general  (Figure  2) and  those  for Horizon  2020 publications  (Figure  3)  should  be 
approached with caution842, certain patterns can be observed. Those countries that excel at 
generating open access publications under Horizon 2020 are often the same ones that exhibit 
high levels of open access publications in general. Notable exceptions appear to be France 
and Luxembourg, both of whom are top performers in publishing open-access Horizon 2020 
publications,  but  appear to  be  mediocre  performers  in  terms  of  open-access  publishing  in 

840 European Commission, Directorate-General for Research and Innovation, Science, research 
and innovation performance of the EU 2022: building a sustainable future in uncertain times, 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/78826. 
841 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R. et al., Evaluation study of the European framework programmes 
for research and innovation for excellent science: Horizon 2020: Annexes, Luxembourg: 
Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383. 
842 This is because information is based on different methods of measuring open access shares, 
with different timeframes being considered. 

512 

 
 
 
 
general  during  the  year  2019.  Conversely,  Croatia  stands  out  as  a  top  performer  in  the 
broader  realm  of  open  access  publishing,  yet  its  performance  in  producing  open-access 
Horizon 2020 publications only places it among the average performers. Furthermore, those 
countries among the lower half of performers in terms of their share of open-access Horizon 
2020 publications also tend to achieve relatively scores for general open-access publications.  

Box 2. Horizon Europe publications and open access rate 

The European Commission reports on the short-term KIP3 indicator843, which is shared 
knowledge and is defined as the share of research outputs resulting from the Programme 
that  are  shared  through  open  knowledge  infrastructures.  Specifically,  85.2%  of  the 
reported  publications  (without  distinction  between  foreground  and  background) 
are so far open access (i.e. number of unique Horizon Europe publications that could 
be matched with OpenAIRE844). As a baseline, the European Commission calculated that, 
by January 1 2017, or three years in Horizon 2020, 77.1% of the H2020 publications were 
in open access. 

1.3.2.  Multi-faceted progress towards the uptake of open science  

The recent Open Public Consultation (OPC) launched by the European Commission on the 
past,  present  and  future  of  the  EU’s  Horizon  R&I  programmes  2014-2027845  asked 
stakeholders their opinions of the role of Open Science in the programme. The results of this 
Targeted Stakeholder Consultation show that the changes to Open Science policy introduced 
under  Horizon  Europe  are  considered  to  have  contributed  to  strengthening  the  impact  of 
research and innovation in the EU by a large majority of the respondents (71%, or 1,106 out 
of 1,564  respondents; see Figure 4). Similarly, Open Science activities aimed at the early 
sharing  of  results  (see  Figure  5)  were  considered  to  have  had  a  positive  impact  on  the 
dissemination,  exploitation  and  access  to  R&I  results  by  65%  of  respondents  (983  out  of 
1,522 respondents). While these figures are positive, it is important to note the relatively high 
shares of respondents who indicated that they held no opinion on these questions (12% in 
Figure 4 and 23% in Figure 5). This suggests that the efforts made in the area are to some 
extent still unnoticed or not widely regarded by a significant group of the stakeholders who 
participated in the Targeted Stakeholder Consultation. 

The paragraphs that follow shed light on this and analyse the ways in which the FPs have 
supported the uptake of Open Science practices, and the main challenges encountered. 

Europe  aims  to  serve  as  a  catalyst  for  open  research  practices  across  its  Member 
States and beyond846. Nevertheless, there is wide diversity in the uptake of Open Science 
across the EU (as discussed in the section above), and the journey towards adopting Open 
Science practices at Member State level often entails numerous challenges. It is therefore 

843 On October 18, 2023 
844 European Commission, Directorate-General for Research and Innovation, Nixon, J., Study to 
support the monitoring and evaluation of the framework programme for research and innovation 
along key impact pathways – Indicator methodology and metadata handbook, Nixon, J.(editor), 
Publications Office of the European Union, 2022, https://data.europa.eu/doi/10.2777/44653 
845 https://research-and-innovation.ec.europa.eu/news/all-research-and-innovation-news/results-
public-consultation-future-eu-research-and-innovation-programmes-are-now-public-2023-04-
19_en. 
846 SWD(2023)132 final. (2023). Evidence Framework on monitoring and evaluation of Horizon 
Europe. Brussels, 27 April 2023. 

513 

 
 
interesting  to  explore the  role  of  the  FP  in  driving  the  uptake  of  Open  Science  at  national 
level,  and  to  explore  how  the  FP  could  further  address  the  challenges  faced  by  these 
countries. 

To a great extent

Somewhat

A little

4,5%

Does not make a difference

9,7%

Not at all

3,3%

I do not know / No opinion

11,8%

36,0%

34,7%

0%

5%

10%

15%

20%

25%

30%

35%

40%

Figure 4. Stakeholders’ agreement regarding the contribution of Horizon Europe Open Science policy to strengthening the 
impact of EU R&I. Source: Targeted Stakeholder Consultation847. Question: ‘In your view, to what extent do the following 
changes, introduced in Horizon Europe, contribute to strengthening the impact of European research and innovation?’ 
Implementation of an open science policy. 

To a great extent

Somewhat

Not at all

A little

3,7%

8,5%

35,1%

29,5%

I do not know / No opinion

23,3%

0%

5%

10%

15%

20%

25%

30%

35%

40%

Figure 5. Stakeholders’ agreement regarding the impact of Open Science activities for the early sharing of results for the 
dissemination, exploitation and access to research and innovation results. Source: Targeted Stakeholder Consultation848. 
Question: ‘To what extent have the following initiatives helped disseminate, exploit and access research and innovation 
results?’ Open Science activities for early sharing of results. 

The European Commission supports researchers, funding organisations and Member States 
in adopting Open Science practices through several mechanisms: 

a.  Supporting the development of knowledge and tools on Open Science. 

b.  Supporting the development of infrastructures, environments and platforms enabling 

open access practices. 

847 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 
848 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 

514 

 
 
 
 
c. 

Inspiring the uptake of Open Science practices through guidelines and requirements 
for the Framework Programme and through participation in mission- and consensus-
building activities. 

Each of the above mechanisms is discussed in further detail below. 

•  Supporting the development of knowledge and tools on Open Science: the knowledge 
and tools on Open Science developed in projects funded under the FPs have been 
essential to providing researchers, practitioners and funders with tools and knowledge 
that help them apply Open Science principles in their activities.  

•  Within the ‘Science with and for Society; (SwafS) programme part under Horizon 2020, 
several calls were devoted to promoting the transparency and reproducibility of scientific 
results and to encouraging researchers at all career stages to practice Open Science. 
These  calls  mainly  focused  on  training,  skills  development  and  education  on  Open 
Science; fostering the knowledge base on Open Science; and supporting the challenge 
of  replicating  and  reproducing  research  results.  In  the  Evaluation  study  on  Excellent 
Science under Horizon 2020, a number of SwafS projects were identified that produced 
outputs  relating  to  Open  Science  (see  Table  1  in  Appendix  2).  Among  others,  these 
include:  1)  Open  Science  toolkits,  training  courses  and  materials  for  researchers  and 
policymakers;  2) novel  policy  frameworks  and  practitioner guidelines  to facilitate  open 
and FAIR research data; 3) innovative approaches and tools for disseminating research 
results;  4)  policy  recommendations  to  stimulate  sustainable  (institutional)  change 
fostering  Open  Science;  and  5)  knowledge  concerning  Open  Science  and  citizen 
science. 

•  Under  the  Horizon  Europe  WIDERA  programme  part,  even  more  calls  have  been 
devoted to fostering Open Science practices. These calls focus on capacity building for 
the application of Open Science practices as a modus operandi for modern science in 
the EU R&I system. In practice, these calls aimed to provide an evidence base on the 
impact of Open Science; to change incentives for researchers to practice Open Science; 
and to provide evidence for and support the alignment of funders and institutional policies 
on  open  access  in  the  ERA.  In  addition,  under  the  ERC  work  programme,  funding  is 
foreseen  to  execute  studies  on  the  topic  of  ‘Support  to  the  implementation  of  Open 
Science by the ERC’.  

• 

Interviews carried out for the present case study indicate that knowledge stemming from 
the outcomes of such actions as mentioned above (e.g., SwafS) could be further fostered 
and  disseminated  to  increase  awareness  among  practitioners  and  researchers. 
Interviewees stressed that the outcomes of such projects are often not continued after 
the projects come to an end, thus hindering their potential to create impacts in the long-
term. 

•  Supporting  the  development  of  infrastructures,  environments  and  platforms  enabling 
open  access  practices:  the  European  Commission  supports  the  development  of 
infrastructures,  environments  and  platforms  for  open  access  practices  that 
researchers can use for their Open Science practices. These tools and platforms support 
researchers  in,  among  other  activities,  publishing  research  output  in  open  access; 
providing repositories in which to deposit research papers, datasets, software, etc.; and 
monitoring  trends  in  open  access.  The  boxes  below  describe  the  most  important  EC-
supported  infrastructures,  environments  and  platforms:  OpenAIRE,  Zenodo,  the 
European Open Science Cloud, and Open Research Europe. 

515 

 
Box 3. Open Research Europe (ORE) 

Open  Research  Europe  (ORE)  is  the  open  access,  peer-reviewed  publishing  platform 
offered  by  the  European  Commission  as  an  optional  service  to  the  grantees  of  its 
Framework Programmes (until 2023 the service was only for beneficiaries of Horizon2020 
and Horizon Europe programmes, including EURATOM and COST actions). The platform 
enables beneficiaries of these FPs to comply with the open access terms of their funding. 
Publication in ORE does not entail any cost for the author and/or their institution(s). All 
publication  costs  are  covered  by  the  European  Commission.  The  platform  offers 
researchers a venue in which to rapidly share their results and insights, and facilitates 
open,  constructive  research  discussion849.  Indeed,  once  an  article  has  passed  pre-
publication checks, a typeset version of the article is published (pre-print) with a DOI. The 
post-publication peer review process is completely open and transparent. Once an article 
receives  two  ‘Approved’,  or  one  ‘Approved’  and  two  ‘Approved  with  Reservations’ 
statuses from peer reviewers, it will be indexed in various bibliographic databases such 
as PubMed, Scopus and Google Scholar. All articles and peer-review reports are in open 
access under open licenses (CC BY).  ORE implements Open Science practices within 
scientific  publishing,  such  as  having  no  journal  impact  factor  but  providing  metrics  at 
article  level,  which  supports  initiatives  such  as  the  Coalition  for  Advanced  Research 
Assessment  (CoARA).  The  ORE  Open  Data  Policy  aligns  with  the  requirements  for 
research data management and FAIR principles. 

Box 4. Open Access Infrastructure for Research in Europe (OpenAIRE) 

The European Commission launched OpenAIRE in 2010, under FP7, as an online portal 
and  network  of  helpdesks  and  experts850.  Its  original  objective  was  to  support  the 
implementation of European open access policies such as the Open Access Pilot under 
FP7. It helped scientists to comply with these policies by organising a ‘network of open 
repositories providing free online access to knowledge produced by scientists receiving 
grants’. Over time, the mission and services of provided by OpenAIRE have expanded. 
For example, OpenAIRE now offers a catalogue of 16 services851 that help researchers 
with publishing, building infrastructure and monitoring Open Science. It includes a bundle 
of  services  to  help  research-performing  organisations  to  transition  to  Open  Science. 
Furthermore,  OpenAIRE  created  the  repository  Zenodo  (see  Box  5  below),  and  has 
contributed to the setting up of the European Open Science Cloud (EOSC). 

Box 5. Zenodo 

Zenodo was created as ‘a catch-all repository for EC-funded research’. It was set up by 
CERN, an OpenAIRE partner, in May 2013. Zenodo gradually expanded its scope and 
now  covers  research  coming  from  all  over  the  world  and  across  all  disciplines.  It 
welcomes all material related to scholarly processes without limitations on its format, size, 
access restrictions or licence. This includes data, software and other artefacts in support 
of  publications,  materials  associated  with  conferences,  projects  or  the  institutions 
themselves. Acknowledging that the time taken to publish is often long, Zenodo offers the 
possibility to upload closed and restricted content while research is still ongoing, which is 

849 https://open-research-europe.ec.europa.eu/about/, accessed on October 18, 2023. 
850 https://openscience.eu/openaire. 
851 https://catalogue.openaire.eu/search. 

516 

 
 
expected to make the process easier for researchers as they can upload content as they 
move  forward  with  their  research.  One  major  advantage  of  Zenodo  is  that  it  makes 
research  results  easily  citable  due  to  integration  with,  for  instance,  DataCite852.  It  also 
offers  the  possibility  to  integrate  the  outcomes  into  the  reporting  templates  of  funding 
agencies (e.g. the EU FP). 

Box 6. European Open Science Cloud (EOSC) 

In  2018,  the  European  Commission  launched  the  EOSC  ‘to  provide  European 
researchers,  innovators,  companies  and  citizens  with  a  federated  and  open  multi-
disciplinary environment where they can publish, find and re-use data, tools and services 
for research, innovation and educational purposes’853.  

The  Horizon  Europe  Research 
the 
development  of  the  EOSC  through  the  INFRAEOSC  calls  for  proposals.  This  initiative 
aims  to  offer  researchers  ‘a  virtual  environment  with  open  and  seamless  services  for 
storage, management, analysis and re-use of research data, across borders and scientific 
disciplines’854,855. 

Infrastructures  Work  Programme  supports 

The INFRAEOSC projects launched under Horizon 2020 and Horizon Europe cover both 
support for the design and prototyping of new services and alignment mechanisms across 
Research  Infrastructures  (including  commercial  services  as  well  as  those  provided  by 
ESFRI infrastructures, etc.) and the development of the vision, mission and governance 
of the EOSC.  

The Horizon INFRA calls supporting Open Science through the EOSC are listed in Appendix 
2 (Table 3). 

Source: Horizon Europe INFRA Work Programmes. 

In  general,  support  for  and  the  creation  of  infrastructures,  environments  and  platforms  for 
open  access  and  Open  Science  practices  is  appreciated  by  both  interviewees  and 
stakeholders856.  Several  interviews  provided  feedback  on  these  platforms.  With  regard  to 
ORE, it was indicated that, at present time, FP beneficiaries might still be prone to publishing 
their research in traditional journals with high impact factors, as this might still be considered 

852 DataCite is a global community that aims to ensure that research outputs and resources are 
openly available and connected so their use can advance knowledge across and between 
disciplines. The community makes research more effective with metadata that connects 
research outputs and resources. https://datacite.org/what-we-do/ 
853 https://research-and-innovation.ec.europa.eu/strategy/strategy-2020-2024/our-digital-
future/open-science/european-open-science-cloud-eosc_en.  
854 https://eosc-portal.eu/.  
855 For more information on the evaluation findings with regard to the EOSC, consult the EOSC 
Partnership Report produced under this study. 
856 e.g. Science Europe, Science Europe Response to the Public Consultation on Horizon 2020 
and Horizon Europe, 2022, available at: https://www.scienceeurope.org/our-resources/science-
europe-response-to-the-public-consultation-on-horizon-2020-and-horizon-europe/. 

517 

 
 
 
beneficial  to  their  careers857.  A  positioning  paper  submitted  in  the  context  of  the  OPC858 
indicates that the development of ORE is a notable achievement, but that joint efforts at EU 
and national level are still needed to ensure its viability and attract users and contributors859. 
Some interviews also indicated that this platform could be a useful solution for those countries 
in which open access is less  well developed (in the sense of having fewer incentives and 
resources to publish in open access, such as paying for article processing charges (APCs)). 
However, such potential could only be achieved if the tool  were not restricted solely to FP 
beneficiaries. 

With regard to the EOSC, INFRAEOSC projects cover support for several activities in parallel 
(e.g. support for the design and prototyping of new services and alignment mechanisms as 
well as development of the vision, mission and governance of the EOSC). According to some 
interviewees,  this  results  in  certain  difficulties  and  inefficiencies  when  trying  to  align  and 
coordinate between different projects and strands of activities. Interviews carried out during 
the Evaluation study on Excellent Science under Horizon 2020 and this study indicate that 
the goal, vision and activities of the EOSC are at this stage not sufficiently developed and 
communicated outside the partners involved in the development of the projects. Indeed, even 
among  those  partners,  the  alignment  of  activities  and  goals  between  projects  (and  parts 
thereof)  remains  very  time-consuming  and  resource-intensive,  due  to  the  intense 
coordination efforts required (e.g. frequent meetings, communication, etc.). 

• 

Inspiring the uptake of Open Science practices through guidelines and requirements for 
the Framework Programme and through participation in Mission and consensus-building 
activities: the European Commission inspires the uptake of Open Science practices 
through  1)  developing  guidelines  and  recommendations;  2)  the  establishment  of 
requirements relating to Open Science in the application and development of projects 
under  the  Framework  Programmes;  and  3)  through  its  support  for  consensus-  and 
mission-building  activities,  e.g.  the  Coalition  for  Advancing  Research  Assessment 
(CoARA).  

•  For a  long  time,  the European  Commission  has  supported  Open  Science  through  the 
production of guidelines and recommendations. In 2012, the European Commission 
issued a recommendation regarding access to and preservation of scientific information. 
This  recommendation  encouraged  all  EU countries  to  put  all  publicly-funded  research 
results into the public domain in order to strengthen science and the knowledge-based 
economy860. This recommendation laid the basis for future actions. Good examples of 
this  approach  include  the  integration  of  Open  Science  skills  as  part  of  the  recently 
launched European Competence Framework for Researchers (ResearchComp)861 and, 
in general, the set of measures included in the European Commission’s recent proposal 

857 Initiatives such as CoARA aim, among other objectives, to reduce the dominance of metrics 
relating to the publication venue (e.g. journals with a high Journal Impact Factor (JIF)). See 
https://coara.eu/agreement/the-commitments/  
858 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. https://research-and-innovation.ec.europa.eu/news/all-
research-and-innovation-news/results-public-consultation-future-eu-research-and-innovation-
programmes-are-now-public-2023-04-19_en. 
859 e.g. Science Europe, Science Europe Response to the Public Consultation on Horizon 2020 
and Horizon Europe, 2022, available at: https://www.scienceeurope.org/our-resources/science-
europe-response-to-the-public-consultation-on-horizon-2020-and-horizon-europe/. 
860 2012/417/EU: Commission Recommendation of 17 July 2012 on access to and preservation 
of scientific information. 
861 https://research-and-innovation.ec.europa.eu/jobs-research/researchcomp-european-
competence-framework-researchers_en. 

518 

 
 
to  the  Council  on  researchers’  careers862,863  (in  line  with  the  May  2021  Council 
Conclusions on research careers864). In 2023, the Council adopted conclusions in which 
it  called  on  the  Commission  and  the  Member  States  to  support  policies  towards  a 
scholarly publishing model that is not-for-profit, open access and multi-format, with no 
costs for authors or readers. The Council encourages national open access policies and 
guidelines to make scholarly publications immediately accessible under open licences, 
and encourages support for ORE. The Pact for R&I in Europe (2021) also emphasises 
that  Member States should adopt  priority  areas  for joint action in  support  of  the  ERA. 
One such priority is to support and reward a culture of Open Science across the Union, 
including mainstreaming open access to scholarly publications and research data, and 
the  diffusion  and  uptake  of  Open  Science  principles  and  practices.  Another  priority 
relates to the development of research assessment systems for research, researchers, 
teams  and  institutions,  in  order  to  reward  quality,  impact,  Open  Science  practices, 
leadership and engagement with society.  

•  With regard to the impact of Horizon Europe processes, it is important to note that the 
introduction of the strongest requirements to apply Open Science as the modus operandi 
under Horizon Europe is perceived by interviewees as a major step towards fostering 
change at national level. As mentioned above, Open Science is included in the evaluation 
criteria for Horizon Europe proposals under the ‘Excellence’ part of the application, as 
well  as  in  the  ‘Capacity  of  the  participants  and  consortium  as  a  whole’  part,  in  which 
applicants  can  describe  their  track  record  in  Open  Science  practices.  In  addition, 
beneficiaries are now obliged to provide immediate open access to their peer-reviewed 
publications  and  associated  metadata.  They  are  also  required  to  develop  data 
management  plans  (DMP)  indicating  what  data  will  be  generated,  how  these  will  be 
exploited,  and  how  they  will  eventually  be  curated  and  preserved.  In  particular,  the 
Horizon Europe Model Grant Agreement requires that a DMP is established and regularly 
updated, and provides a template recommended for this purpose. 

• 

Interviews with representatives of ministries and funding agencies indicate that this move 
towards Open Science playing a greater role  has  triggered action at national level, 
and that many steps have been taken to align national policies and practices with the 
European Commission guidelines. Many national funders have adopted (or are planning 
to adopt) the requirement for researchers to provide and update DMPs for their research 
projects  (e.g.  in  the  Netherlands,  Estonia  and  Flanders).  These  actions  have  inspired 
several countries to include further requirements relating to Open Science practices in 
their project evaluations. For instance, some national funders (e.g. in the Netherlands) 
now  include  the  option  to  provide  narrative  CVs  in  research  proposals,  in  which 
researchers can describe their Open Science experiences.  

• 

Lastly,  the  European  Commission’s  support  for  consensus  and  mission-building 
activities was also mentioned during interviews as a key step towards fostering action 
at national level. One of the major barriers to the uptake of Open Science practices by 

862 The recommendation included a proposal for a Council Recommendation that establishes a 
new European framework for research careers; a new Charter for Researchers that replaces the 
2005 Charter and Code for Researchers with new and revised principles; and the 
abovementioned European Competence Framework for Researchers (ResearchComp), to 
support inter-sectoral mobility of researchers and diversity of research careers. 
https://ec.europa.eu/commission/presscorner/detail/en/IP_23_3807. 
863 https://euraxess.ec.europa.eu/euraxess/news/proposal-council-recommendation-european-
framework-research-careers, accessed 18 October 2023. 
864 https://www.consilium.europa.eu/en/press/press-releases/2021/05/28/improving-conditions-
for-research-careers-in-europe-council-adopts-conclusions/. 

519 

 
 
researchers  is  associated  with  the  lack  of  recognition  given  to  these  practices  in 
researchers’  career  progression.  In  this  respect,  the  support  of  the  European 
Commission  to  the  Coalition  for  Advancing  Research  Assessment  (CoARA)  is 
particularly  relevant.  Launched  in  January  2022,  this  agreement  was  reached  among 
more than 350 organisations from over 40 countries, laying down basic principles for the 
assessment of researchers. Open Science principles are given a dominant role in this 
context – while the impact of research is still important, the means by which research is 
shared and made available to others (including the broader public) is now more important 
than  ever.  In  line  with  the  priority  areas  identified  in  the  new  Pact  for  R&I  in  Europe 
(2021), the reform of assessment systems for research, researchers and institutions is 
outlined  as  a  concrete  action  in  the  ERA  policy  agenda  2022-2024865.  This  entails 
analysing barriers to the creation of modern research assessment systems at national 
and  transnational  levels,  as  well  as  creating  a  coalition  of  EU  funders  and  research 
performers  who  agree  on  a  new  approach  to  research  assessment  and  the 
implementation of the research plans.  

1.4.  Success factors and barriers 

All of the interviewees consulted in the context of this case study indicate that the European 
Commission’s actions in the field of Open Science are highly influential, and are considered 
best practice at Member State level. Desk research, interviews and the findings of the OPC866 
all point to the need to keep on developing the policies and actions on this area. With respect 
to  this,  respondents  to  the  Targeted  Stakeholder  Consultation  also  point  to  the  need  to 
develop further Open Science practices in the next Strategic Plan. This topic was deemed 
either ‘essential’ or of ‘high-priority’ by 57.5% of respondents (1,113 out of 1,932), see Figure 
6. 

Essential

High priority

Medium priority

Low priority

8,6%

Not a priority

2,2%

I do not know / No opinion

6,5%

23,8%

25,1%

33,7%

0%

5%

10%

15%

20%

25%

30%

35%

40%

Figure 6. Stakeholders’ agreement on the need to further elaborate Open Science practices in the Strategic Plan 2025-2027. 
Source: Targeted Stakeholder Consultation867. Question: ‘The Strategic Plan 2021-2024 identified specific issues to be 
considered in the implementation of Horizon Europe. Should any of these cross-cutting themes be further elaborated for the 
Strategic Plan 2025-2027?’: Open science practices. 

865 European Research Area Policy Agenda. Overview of actions for the period 2022-2024. 
https://research-and-innovation.ec.europa.eu/system/files/2021-11/ec_rtd_era-policy-agenda-
2021.pdf. 
866 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 
867 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 

520 

 
 
 
However, the interviews and desk research carried out for this case study highlight several 
important nuances and points to be considered in the development of future policies, plans 
and actions in this field: 

•  Continue fostering the application of open science, while making sure all elements 
needed for optimal deployment of open science practices are in place. Interviewees 
and stakeholder organisations (see, for example, the position papers submitted in the 
context of the OPC) are highly appreciative of the European Commission’ 360-degree 
approach to the implementation of Open Science, in which work is being carried out on 
the  policy  side;  on  the  development  of  the  tools  and  monitoring  systems;  and,  at  the 
same  time,  on  communication  and  dissemination  activities  aimed  at  ensuring 
researchers are made aware of these changes. At the  same time, some interviewees 
expressed concerns regarding the speed at which Open Science is being promoted. In 
their view, some elements (e.g. sufficient skills, resources and infrastructures) needed 
for the optimal deployment of Open Science practices are not yet fully in place. Pushing 
for the implementation of these practices could therefore, in their view, potentially lead 
to  unwanted  effects.  For  instance,  one  interviewee  stressed  that  while  the  previous 
system (under which research institutions needed to pay subscriptions in order to have 
access  to  publicly-funded  research) was  highly  disadvantageous,  there was  not  yet  a 
system  in  place  that  could  fully  ensure  similar  levels  of  trustworthiness  and  research 
quality868.  Other  interviewees  wondered  whether,  in  a  system  where  research 
assessment  is  not  yet  fully  reformed,  the  necessity  to  publish  results  in  open  access 
and/or  pay  (high)  fees  to  publish  in  excellent  journals  will  not  disadvantage  the 
competitiveness of EU research and EU researchers. Based on the evidence collected 
for this case study, it is evident that the widespread uptake of initiatives such as CoARA 
and  ORE  are  key  to  fully  reforming  the  system,  and  contain  the  elements  needed  to 
ensure high-quality and impactful research.   

•  Change  at  national  level  requires  time,  and  depends  on  the  specific  national 
context. Many countries are developing their own Open Science processes in line with 
or  inspired  by  the  European  Commission’s  Open  Science  practices.  Nevertheless, 
interviewees  stressed  that  the  ease  with  which  Open  Science  practices  can  be 
introduced  at  national  level  depends  on  the  cultural,  historical  and  institutional 
specificities of each country. The general guidelines and requirements of the FP can – 
and often do – serve as useful starting points and inspirations. However, such examples 
need to be adapted to individual national contexts, which takes time. For instance, while 
countries such as the UK869 and Netherlands (which had the highest open access rates 
in 2019; see Figure 2) have a long tradition of open access and Open Science, many 
countries do not have such a long history or are still in the process of discussing and 
adopting  institutional  and  national  policies.  For  instance,  in  Romania  (which  had  the 
lowest open access rate in 2019 according to Figure 2), a National Strategic Framework 
for  Open  Science  was  adopted  in  2022,  which  details  the  implementation  of  Open 
Science principles, challenges and actions870. This national framework also refers to the 

868 Some interviewees, for example, referred to the emergence of predatory journals that aim to 
publish articles rapidly on their platforms in return for article processing fees, without offering 
quality peer review or editorial services, and sometimes reporting false Journal Impact Factors 
(JIF). 
869 Select Committee on Science and Technology, 10th Report (2004), accessed on 18 October 
2023, available at: 
www.publications.parliament.uk/pa/cm200304/cmselect/cmsctech/399/39903.htm. 
870 More information is available in the OpenAIRE blogpost ‘Charting the Paths to Open Science: 
Romania’s Strategic Framework’ (March 2023), accessed 8 January 2024, available at: 

521 

 
 
 
European  Framework  Programmes,  and  builds  on  the  Council’s  Conclusions  on 
Research  assessment  and  the  implementation  of  Open  Science871, as  well  as  the  EC 
Recommendation on access to and preservation of scientific information872 among other 
key EU’s strategic documents. The size and the complexity of the R&I landscape at 
national level is mentioned by some interviewees as a factor that has an impact on the 
speed at which alignment with the FP can be made. Specifically, interviewees pointed 
out that smaller countries or those with less complex R&I landscapes tend to find it easier 
to  align  their  policies  and  instruments  with  the  practices  of  the  EU FP  compared  with 
larger  and/or  more  complex  R&I  systems.  For  example,  Slovakia  adopted  a  National 
Strategy  for  Open  Science  only  recently  (in  2021).  This  had  been  hindered  by  a 
fragmented legal system, under which the implementation of Open Science principles at 
national level required changes to more than 10 laws873. At the same time, aligning EU 
funding requirements and legal frameworks is critically important, as a lack of alignment 
can place researchers in difficult situations (especially those in consortia with partners 
from other countries, as mentioned in a positioning paper submitted to the OPC874).  

•  Resistance to change and a lack of awareness of the importance of Open Science 
are factors identified as hindering progress in this area. Greater awareness can be 
achieved  through  actions  that  promote  the  benefits  of  Open  Science  approaches  to 
policy-  and  decision-makers  in  government  and  in  research  funding  agencies  and 
institutions. It is also important to promote the development of skills in this area, targeting 
not only academics and researchers but also policy-makers and other stakeholders in 
the  field  of  R&I  policy.  Although  some  such  initiatives  have  been  carried  out875, 
interviewees indicate that further action could be beneficial.  

•  Another  factor  related  to  skills  is  the  scarcity  of  professionals  with  sound 
knowledge and skills in the application of Open Science principles who are able 
to support institutions and researchers in applying these principles. Interviewees 
indicated that such professionals (whose job titles depend on the individual country and 
setting, and include ‘data stewards’, ‘research engineers’, and so on) remain rare and 
that, once they achieve a certain degree of expertise, they are likely to move to positions 
in  industry.  The  main  reason  for  this  is  that  industry  is  regarded  as  providing  better 
remuneration and career development prospects, while the traditional research setting 
has not yet developed full professional career paths and recognition schemes for such 
professionals. In line with this thinking, one positioning paper submitted to the  OPC876 

https://www.openaire.eu/blogs/charting-the-path-to-open-science-romania-s-strategic-
framework#:~:text=The%20White%20Paper%20on%20the%20Transition%20to%20Open%20S
cience%202023,as%20much%20as%20possible%20open. 
871 Council Conclusions on Research assessment and implementation of Open Science (2022). 
872 EC Recommendation (EU) 2018/790 of 25 April 2018 on access to and preservation of 
scientific information. 
873 https://www.openaire.eu/blogs/slovak-national-strategy-for-open-science-2021-2028. 
874 Science Europe (2023). Science Europa response to the public consultation on Horizon 2020 
and Horizon Europe 2023. 
875 In 2017, under Horizon 2020, a mutual learning exercise on Open Science and altmetrics 
took place, allowing the sharing of experiences and mutual learning between representatives 
from 13 countries (Armenia, Austria, Belgium, Bulgaria, Croatia, France, Latvia, Lithuania, 
Moldova, Portugal, Slovenia, Sweden and Switzerland). See https://ec.europa.eu/research-and-
innovation/en/statistics/policy-support-facility/mle-open-science-altmetrics-and-rewards. 
876 Ludwig Boltzmann Gesellschaft (2023). Ludwig Boltzmann Gesellschaft position paper public 
consultation on the past, present and future of the European Research & Innovation Framework 
Programmes 2014-2027. 

522 

 
 
urges the setting up of a ‘data management helpdesk’ providing additional support to the 
FP participants, in the same way that there is an IP helpdesk.  

• 

Incentives matter. Open Science needs to be sufficiently well rewarded and recognised 
in the context of researchers’ careers. This relates to open access practices, but also to 
the  wider  definition  of  Open  Science,  including  citizen  science  and  engagement  (see 
Section  1.1  of  this  case  study).  While  initiatives  such  as  CoARA  (see  above)  were 
perceived  very  positively  by  interviewees,  they  nevertheless  indicated  that  important 
changes are still needed in terms of how national funding organisations and institutions 
apply those principles in order for such a situation to become a reality. Future actions 
addressing challenges in relation to research careers could be aligned with the actions 
set out in the ERA Policy Agenda for 2022-2024877. 

•  The  general  Open  Science  principles  applied  in  the  FP  may  also  require 
adaptations  to  individual  fields  or  disciplines,  to  better  respond  to  the  specific 
challenges faced.  

− 

Interviewees  indicated  that  it  is  important  to  maintain  a  solid  general  vision,  while 
allowing room to adapt its principles to specific settings and disciplines. This confirms 
the validity of the principle ‘as open as possible, as closed as necessary’. The term 
‘closed’  not  only  applies  to  those  cases  involving  privacy  concerns,  but  also  where 
there may be legitimate commercial interests. While this point applies to the production 
of  knowledge  as  a  whole,  it  is  particularly  relevant  to  the  case  of  research  outputs 
relating  to  mid-to-high  Technology  Readiness  Levels,  as  well  as  those  in  fields  or 
technologies that are of particular interest to the European Union. 

−  Furthermore, interviewees indicated that questions remain as to the best way to apply 
(and to urge the application of) FAIR principles as part of their FP activities, and that 
guidelines  at  EU  level  would  be  beneficial.  However,  some  divergence  of  opinion 
emerged between interviewees with regard to the ideal level of specificity/granularity 
of the desired guidance. While some interviewees believed that offering guidance at a 
general level would be sufficient, and that researchers and institutions should bear the 
responsibility for ensuring proper data management, other interviewees (for instance, 
those from national ministries and research funding organisations) indicated that more 
specific guidelines tailored to individual disciplines and to the sensitivity of data would 
be desirable. 

•  The  scope  of  actions  supporting 

the  development  of 

infrastructures, 
environments and platforms for Open Science needs to be reassessed to ensure 
its  relevance  and  effectiveness.  The  experts  consulted  in  the  context  of  this  study 
largely believe that these efforts are useful and are travelling in the right direction. At the 
same  time,  however, they  raised  some  concerns  regarding  the  scope  of  such  efforts. 
Some of the infrastructures concerned are exclusively intended for FP beneficiaries (e.g. 
Open  Research  Europe),  while  others  target  EU  researchers  (the  EOSC),  whereas 
others  are open  to  all  researchers  (Zenodo). Interviewees  often  stressed  the  fact  that 
science is global, and thus the tools that support science also need to be global in order 
to be relevant and useful for researchers. Limiting the scope of some of these tools only 
to EU researchers or to research funded by the FP was seen by some interviewees as 
an  obstacle  hindering  their  impact  and  relevance  to  the  research  community. 

877 European Commission, Directorate-General for Research and Innovation, European 
Research Area policy agenda – Overview of actions for the period 2022-2024, Luxembourg: 
Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/52110. 

523 

 
 
Interviewees also pointed to the fact that opening up these tools could have a positive 
impact on the attractiveness of Europe as a research destination in a context in which 
there is a global competition for research talent.  

1.5.  Conclusions  

The EU is a frontrunner in open access scientific output. One recent report indicate that in 
2019,  almost  40%  of  all  EU  publications  were  freely  available  through  an  open  access 
publishing  pathway,  which  was  slightly  higher  than  in  the  US,  Japan,  Canada  and  South 
Korea.  Moreover,  trends  indicate  that  Open  Science  and  open  access  are  gaining 
importance worldwide, and not just in Europe. This is illustrated by a recent memo878 from 
the  Executive  Office  of  the  US  President,  directing  federal  departments  and  agencies  to 
update their public access policies to ensure taxpayer-funded publications and research are 
made publicly accessible, without delay or cost, by the end of 2025.  

At the same time, variations exist within the EU879, in terms of the share of open access 
publications produced in different countries, with the leaders in 2019 being the Netherlands 
(58.3%  of  publications  in  open  access)  and  the  United  Kingdom  (57.8%).  The  countries 
lagging furthest behind in terms of open access publishing are Romania (25.1%), Bulgaria 
(25.7%)  and  Latvia  (28.9%).  When  comparing  performance  in  general  open  access 
publishing with the share of open access publications under Horizon 2020, the countries that 
excel in generating open-access Horizon 2020 publications are often the same as those that 
exhibit high levels of open access publication in general, with only a few exceptions.  

The above findings indicate that while Europe aims to serve as a catalyst for open research 
practices, there is still wide diversity in the uptake of Open Science across the EU. However, 
there  was  consensus  among  interviewees  and  stakeholders  that  the  European 
Commission’s  actions  in  the  field  of  Open  Science  are  highly  influential,  and  are 
considered best practice at Member State level. In exploring the role of the FP in driving 
the uptake of Open Science at national level, three mechanisms were identified via which the 
European Commission supports researchers, funding organisations and Member States:  

•  First, the knowledge and tools in relation to Open Science developed through projects 
funded under the  FPs have been essential in providing researchers, practitioners and 
funders with tools and knowledge that help them apply Open Science principles in their 
activities. For example, the SwafS and WIDERA programme parts were/are devoted to 
fostering Open Science practices.  

•  Second, 

the  European  Commission  supports 

infrastructures, 
environments and platforms for open access practices that researchers can use for their 
Open Science practices. These infrastructures, such as OpenAIRE, Zenodo, EOSC and 
ORE, are generally appreciated by beneficiaries and stakeholders. 

the  creation  of 

878 As discussed in the Nelson Memo from the Executive Office of the President – Office of 
Science and Technology Policy, entitled ‘Ensuring Free, Immediate, and Equitable Access to 
Federally Funded Research; (2022), available at: https://www.whitehouse.gov/wp-
content/uploads/2022/08/08-2022-OSTP-Public-Access-Memo.pdf, accessed on 17 October 
2023. 
879 The data presented here are for the EU-28, including the UK.  

524 

 
 
• 

Lastly, the European Commission inspires the uptake of Open Science practices through 
guidelines and the requirements of the Framework Programme, and through participation 
in mission- and consensus-building activities such as CoARA. 

In terms of EU added value, interviewees have highlighted that actions in the field of Open 
Science  within  the  context  of  the  FP  stimulate  much-needed  discussions  and  significantly 
increase awareness of the topic. The impact of such actions is greater than what could be 
achieved  if  comparable  initiatives  were  carried  out  solely  at  national  level.  The  FP’s 
contribution is stronger, as its influence is not confined solely to the research funded under 
the programme, but also contributes to the alignment of attitudes, approaches, policies and 
instruments at national and institutional levels. 

Appendix 1. Literature sources 

•  Euraxess, Proposal for a Council Recommendation on a European Framework for 

Research Careers, 2023, available at: 
https://euraxess.ec.europa.eu/euraxess/news/proposal-council-recommendation-
european-framework-research-careers, accessed 18 October 2023. 

•  European Commission, Directorate-General for Research and Innovation, Specific 

support to Malta – ‘Open Access – An Opportunity for Malta’ – Recommendations for 
the Development of a National Policy for Open Access to Publications, Research Data 
and related Issues, Luxembourg: Publications Office of the European Union, 2020, 
available at: https://data.europa.eu/doi/10.2777/835. 

•  European Commission, Directorate-General for Research and Innovation, Horizon 

Europe, open science – Early knowledge and data sharing, and open collaboration, 
Luxembourg: Publications Office of the European Union, 2021, available at: 
https://data.europa.eu/doi/10.2777/18252. 

•  European Commission, 2012/417/EU: Commission Recommendation of 17 July 2012 

on access to and preservation of scientific information. 

•  European Commission, Directorate-General for Research and Innovation, European 
Research Area policy agenda – Overview of actions for the period 2022-2024, 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/52110 

•  European Commission, Directorate-General for Research and Innovation, Johnson, R., 

Operationalising Open Research Europe as a collective publishing enterprise, 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/061886. 

•  European Commission, Directorate-General for Research and Innovation, Nixon, J., 
Study to support the monitoring and evaluation of the Framework Programme for 
Research and Innovation along key impact pathways – Indicator methodology and 
metadata handbook, Nixon, J.(editor), Luxembourg: Publications Office of the 
European Union, 2022, available at: https://data.europa.eu/doi/10.2777/44653. 

•  European Commission, Directorate-General for Research and Innovation, Science, 

research and innovation performance of the EU 2022: building a sustainable future in 

525 

 
uncertain times, Luxembourg: Publications Office of the European Union, 2022, 
available at: https://data.europa.eu/doi/10.2777/78826. 

•  European Commission, Rates of Open Access Scientific Publications, 2022, available 

at: https://ec.europa.eu/assets/rtd/srip/2022/figure-6-1-6.xlsx. 

•  European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European Framework 
Programmes for Research and Innovation for excellent science: Horizon 2020: 
annexes, Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383. 

•  European Commission, Research and Innovation Funding Opportunities, 

https://ec.europa.eu/research/participants/data/ref/h2020/grants_manual/amga/h2020-
amga_en.pdf. 

•  European Council, Improving Conditions for Research Careers in Europe: Council 

Adopts Conclusions, 2021, available at: 
https://www.consilium.europa.eu/en/press/press-releases/2021/05/28/improving-
conditions-for-research-careers-in-europe-council-adopts-conclusions/. 

•  European Research Area Policy Agenda, Overview of Actions for the Period 2022-
2024, available at: https://research-and-innovation.ec.europa.eu/system/files/2021-
11/ec_rtd_era-policy-agenda-2021.pdf 

•  Horizon Europe (HORIZON). Programme Guide. Version 3.0. 

https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/guidance/programme-guide_horizon_en.pdf. 

• 

• 

• 

https://ncpflanders.be/infosheets/open-science-under-horizon-europe, accessed on 
14/07/2023.  

Initiative for Science in Europe (ISE) (2023). ISE Response to the European 
Commission Public Consultation on the past, present and future of the European 
Research & Innovation Framework programmes 2014-2027. 

Ludwig Boltzmann Gesellschaft (2023). Ludwig Boltzmann Gesellschaft position paper 
public consultation on the past, present and future of the European Research & 
Innovation Framework Programmes 2014-2027 

•  Nelson Memo, Ensuring Free, Immediate, and Equitable Access to Federally Funded 

Research, Executive Office of the President – Office of Science and Technology Policy, 
2022, available at: https://www.whitehouse.gov/wp-content/uploads/2022/08/08-2022-
OSTP-Public-Access-Memo.pdf, accessed on 17 October 2023. 

•  Owens, B., Catching up on Open Access, 2023, available at: 

https://www.universityaffairs.ca/features/feature-article/catching-up-on-open-access/. 

•  Public consultation on the past, present and future of the EU’s Horizon research and 

innovation programmes 2014-2027, available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-

526 

 
consultation-future-eu-research-and-innovation-programmes-are-now-public-2023-04-
19_en  

•  RDA Bulgarian Node, National Plan for the Development of Open Science Initiatives in 

Bulgaria. https://rda.bg/en/node/47, accessed on 18 October 2023. 

•  ResearchComp: The European Competence Framework for Researchers, available at: 

https://research-and-innovation.ec.europa.eu/jobs-research/researchcomp-european-
competence-framework-researchers_en 

•  Science Europe, Science Europe Response to the Public Consultation on Horizon 
2020 and Horizon Europe, 2022, available at: https://www.scienceeurope.org/our-
resources/science-europe-response-to-the-public-consultation-on-horizon-2020-and-
horizon-europe/. 

•  Science Europe (2023). Science Europa response to the public consultation on Horizon 

2020 and Horizon Europe 2023. 

•  Select Committee on Science and Technology Tenth Report, 2004, 

www.publications.parliament.uk/pa/cm200304/cmselect/cmsctech/399/39903.htm, 
accessed on 18 October 2023. 

•  Slovakia, National Strategy for Open Science 2021-2028, available at: 

https://www.openaire.eu/blogs/slovak-national-strategy-for-open-science-2021-2028. 

•  SPARC Europe, An Analysis of Open Science Policies in Europe, v7, 2021, available 

at: https://sparceurope.org/what-we-do/open-data/sparc-europe-open-data-resources/, 
accessed on 18 October 2023. 

•  SWD(2023)132 final. (2023). Evidence Framework on monitoring and evaluation of 

Horizon Europe. Brussels, 27.4.2023. 

•  UNESCO Recommendation on Open Science. Draft text, available at: 

https://unesdoc.unesco.org/ark:/48223/pf0000378841. 

•  Websites on Open Science environments and platforms (Zenodo, OpenAire, Open 

Research Europe, EOSC) 

•  Work Programmes INFRA (2021-2022; 2023-2024). 

•  Work Programmes SwafS (2014-2015; 2016-2017; 2018-2020). 

•  Work Programmes WIDERA (2021-2022; 2023-2024). 

527 

 
 
 
Appendix 2. Relevant Horizon 2020 and Horizon Europe calls 

Call 

Call title 

SwafS Work Programme 2014-2015 

GARRI-3-
2014 

Scientific Information in the Digital Age: Text and Data Mining (TDM) 

GARRI-4-
2015  

Innovative approach to release and disseminate research results and measure 
their impact 

SwafS-07-
2016 

SwafS-10-
2017 

SwafS Work Programme 2016-2017 

Training on Open Science in the European Research Area 

Putting Open Science into action 

SwafS Work Programme 2018-2020 

SwafS-04-
2018 

Encouraging the re-use of research data generated by publicly funded research 
projects 

SwafS-30-
2020 

Responsible Open Science: an ethics and integrity perspective 

Table 1. Horizon 2020 SwafS calls supporting Open Science*. *Note: in line with the scope of this case study, calls 
specifically aimed at supporting citizen science, citizen engagement and RRI are not included in this table. Source: Horizon 
2020 SwafS work programmes. 

Call 

Call title 

WIDERA Work Programme 2021-2022 

Modelling and quantifying the impacts of Open Science practice 

Global cooperation on FAIR data policy and practice 

Capacity-building for institutional open publishing across Europe 

WIDERA-2021-
ERA-01-40 

WIDERA-2021-
ERA-01-41  

WIDERA-2021-
ERA-01-43  

WIDERA-2021-
ERA-01-45  

Support to changes in the assessment of research and researchers to 
reward the practice of open science 

528 

 
Call 

Call title 

WIDERA-2022-
ERA-01-40  

Stepping-up institutional and territorial changes towards open and 
responsible research and innovation 

WIDERA-2022-
ERA-01-41  

Increasing the reproducibility of scientific results 

WIDERA-2022-
ERA-01-42  

Supporting the development of aligned policies for open access books and 
monographs 

WIDERA-2022-
ERA-01-44  

Developing and piloting training on the practice of open and responsible 
research and innovation 

WIDERA Work Programme 2023-2024 

WIDERA-2023-
ERA-01-07 

Support to reforms of research assessment in the European Research Area 

WIDERA-2024-
ERA-01-07 

Capacity building on Intellectual Property (IP) management to support open 
science 

WIDERA-2024-
ERA-01-08 

Global cooperation in not-for-profit open access publishing 

WIDERA-2024-
ERA-01-09 

Support to the development and implementation of policies and practices for 
reproducibility of scientific results  

Table 2. Horizon Europe WIDERA calls supporting open science*. *Note: in line with the scope of this case study, calls 
specifically aimed at supporting citizen science, citizen engagement and RRI are not included in this table. Source: Horizon 
Europe WIDERA work programmes. 

Call 

Call title 

INFRA Work programme 2021-2022 
Destination – Enabling an operational, open and FAIR ecosystem (INFRAEOSC) 

HORIZON-INFRA-2021-EOSC-
01-01 

Supporting an EOSC-ready digitally skilled workforce 

HORIZON-INFRA-2021-EOSC-
01-02 

Supporting the development and coordination of activities of 
the EOSC Partnership  

HORIZON-INFRA-2021-EOSC-
01-03 

HORIZON-INFRA-2021-EOSC-
01-04 

Deploying EOSC-Core components for FAIR 

Innovative and customizable services for EOSC 

529 

 
Call 

Call title 

HORIZON-INFRA-2021-EOSC-
01-05 

Enabling discovery and interoperability of federated 
research objects across scientific communities 

HORIZON-INFRA-2021-EOSC-
01-06 

FAIR and open data sharing in support of cancer research 

HORIZON-INFRA-2022-EOSC-
01-01 

Services and tools to underpin a research assessment 
system that incentivises open science practices 

HORIZON-INFRA-2022-EOSC-
01-02 

Improving and coordinating technical infrastructure for 
institutional open access publishing across Europe 

HORIZON-INFRA-2022-EOSC-
01-03 

FAIR and open data sharing in support of healthy oceans, 
seas, coastal and inland waters 

HORIZON-INFRA-2022-EOSC-
01-04 

Support for initiatives helping to generate global standards, 
specifications and recommendations for open sharing of 
FAIR research data, publications and software 

INFRA Work programme 2023-2024 
Destination – Enabling an operational, open and FAIR ecosystem (INFRAEOSC) 

HORIZON-INFRA-2023-EOSC-
01-01 

Build on the science cluster approach to ensure the uptake 
of EOSC by research infrastructures and research 
communities  

HORIZON-INFRA-2023-EOSC-
01-02 

Development of community-based approaches for ensuring 
and improving the quality of scientific software and code  

HORIZON-INFRA-2023-EOSC-
01-03 

Planning, tracking, and assessing scientific knowledge 
production 

HORIZON-INFRA-2023-EOSC-
01-04 

Next-generation services for operational and sustainable 
EOSC Core Infrastructure 

HORIZON-INFRA-2023-EOSC-
01-05 

EOSC Architecture and Interoperability Framework  

HORIZON-INFRA-2023-EOSC-
01-06 

Trusted environments for sensitive data management in 
EOSC 

HORIZON-INFRA-2024-EOSC-
01-01 

FAIR and open data sharing in support of the mission 
adaptation to climate change 

HORIZON-INFRA-2024-EOSC-
01-02 

Supporting the EOSC Partnership in further consolidating 
the coordination and sustainability of the EOSC ecosystem 

HORIZON-INFRA-2024-EOSC-
01-03 

Enabling a network of EOSC federated and trustworthy 
repositories and enhancing the framework of generic and 

530 

 
Call 

Call title 

discipline specific services for data and other research 
digital objects 

HORIZON-INFRA-2024-EOSC-
01-04 

Long-term access and preservation infrastructure 
development for EOSC, including data quality aspects 

HORIZON-INFRA-2024-EOSC-
01-05 

Innovative and customizable services for EOSC Exchange  

Table 3. INFRA calls supporting Open Science through the EOSC.  

531 

 
 
Annex 2.12: Gender equality in the European Framework 
Programmes for Research and Innovation 

1.1. 

Introduction 

The  European  Commission  aims  to  promote  gender  equality  in  research  and  innovation 
(R&I). This goal is part of the European Commission Gender Equality Strategy 2020-2025, 
which sets out the Commission’s broader commitment to equality across all EU policies. In 
its  2020  Communication  ‘A  new  ERA  for  Research  and  Innovation’880,  the  European 
Commission calls for gender equality as a renewed priority for the ERA in order to strengthen 
Europe’s R&I potential. More recently, Action 5 under the ERA Policy Agenda for the period 
2022-2024 aims to promote gender equality and inclusiveness, taking note of the Ljubljana 
declaration881. 

Under  the  ninth  Framework  Programme  (FP)  for  R&I,  Horizon  Europe,  the  European 
Commission reaffirmed its commitment to gender equality in R&I as a cross-cutting priority 
and strengthened its provisions in comparison to Horizon 2020. In particular, Horizon Europe 
introduced  three  main  levels  at  which  gender  equality  is  addressed:  1)  Increasing  gender 
balance throughout the programme, with a target of 50% women in Horizon Europe-related 
boards, expert groups and evaluation committees (compared with 40% under Horizon 2020), 
and introducing gender balance among research teams as a criterion for ranking proposals 
with the same score; 2) Having a Gender Equality Plan (GEP) as an eligibility criterion for 
certain categories of legal entities; and 3) Integrating the gender dimension into R&I content 
as a requirement by default that is evaluated under the criterion of excellence882. 

The present case study sheds light on the progress achieved by the Framework Programme 
on  these  levels.  The  case  study  takes  a  Framework  Programme-wide  approach, 
including initiatives beyond Pillar 1, and aims to provide FP-wide relevant insights and 
lessons.  The  case  study  focuses  primarily  on  Horizon  Europe,  making  comparisons  with 
Horizon 2020 where possible and relevant.  

1.2.  Background information 

1.2.1.  Policy context 

1.2.1.1. 

Gender equality in research and innovation in Europe 

In  the  field  of  research  and  innovation  (R&I)  in  Europe,  gender  equality  and  gender 
mainstreaming have been a priority of the European Research Area (ERA) since the 2012 
ERA Framework Communication883, in which the European Commission laid out objectives 

880 COM (2020) A new ERA for Research and Innovation. Available at: https://eur-
lex.europa.eu/legal-content/EN/TXT/?uri=COM%3A2020%3A628%3AFIN. 
881 The Ljubljana Declaration is available at: https://data.consilium.europa.eu/doc/document/ST-
12044-2021-INIT/en/pdf.  
882 Under Horizon 2020, as of 2018, only the evaluation for RIAs/IAs under the Excellence 
criterion became mandatory. 
883 Communication from the Commission to the European Parliament, the Council, the European 
Economic and Social Committee and the Committee of the Regions. A Reinforced European 

532 

 
 
 
for fostering institutional change, namely: 1) gender balance in decision making; 2) gender 
equality in careers at all levels; and 3) a better integration of the gender dimension into R&I 
content. 

The Council Conclusions on advancing Gender Equality in the ERA884 called for cultural and 
institutional  changes  to  address  gender  imbalances  in  research  institutions  and  decision-
making  bodies.  EU  countries  were  encouraged  to  develop  ERA  national  action  plans 
addressing  the  priority  of  gender  equality  (then  Priority  4  out  of  six  ERA  priorities),  and 
research performing and funding organisations were encouraged to implement institutional 
changes, in particular through Gender Equality Plans (GEPs), which were expected to have 
very positive impacts in many research organisations. 

More recently, in its Communication of 2020, ‘A new ERA for Research and Innovation’885, 
the European Commission called for increasing gender equality to strengthen Europe’s R&I 
potential. In the Communication, the Commission highlighted that gender inequalities persist 
in  Europe’s  R&I  systems,  and  that  coordinated  action  through  education  policies  and 
research funders could promote a gender-inclusive culture. The Commission proposed, ‘as 
of 2021, in line with the Horizon Europe programme objectives, the development of inclusive 
gender equality plans with Member States and stakeholders in order to promote EU gender 
equality in R&I.’886  

On a practical level, the ERA Policy Agenda, which sets out voluntary ERA actions for the 
period 2022-2024 to contribute to the priority areas defined in the Council Recommendation 
on a Pact for Research and Innovation in Europe887 (Pact for R&I), includes a set of priority 
areas for joint action between the European Commission and the Member States – as well 
as observers participating in the ERA Forum888. One of these priority areas encourages the 
promotion  of  gender  equality,  equal  opportunities  and  inclusiveness.  As  underlined  in  the 

Research Area Partnership for Excellence and Growth. COM(2012) 392 final, available at: 
https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52012DC0392. 
884 Council conclusions of 1 December 2015  on Advancing gender equality in the European 
Research Area, available at: https://data.consilium.europa.eu/doc/document/ST-14846-2015-
INIT/en/pdf.  
885 Communication from the Commission to the European Parliament, the Council, the European 
Economic and Social Committee and the Committee of the Regions. A new ERA for Research 
and Innovation. COM(2020) 628 final, available at: https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=COM%3A2020%3A628%3AFIN 
886 In addition, with regard to the governance of the new ERA, ‘action at EU level will be guided 
by the proposed list of actions according to a timeline that will be updated by the Commission as 
implementation proceeds. Action at national level will be guided by a set of key values and 
principles, building on the experience of the past 20 years in areas such as open access, gender 
equality or career conditions for researchers and others.’ 
887 Proposal for a Council Recommendation on a Pact for Research and Innovation in Europe. 
COM/2021/407 final, available at: https://eur-lex.europa.eu/legal-
content/EN/ALL/?uri=COM:2021:407:FIN. 
888 The ERA Forum is the new body responsible for enhancing coordination towards the effective 
implementation of the ERA Policy Agenda. It is organised as a Commission expert group to 
which Member States and EFTA/EEA-associated countries send representatives, and also 
involves in its work representatives of EU-level stakeholder organisations . A dedicated 
subgroup has been created in 2023 for ERA Action 5, which plays a key role in devising policy 
frameworks and priorities with regard to inclusive gender equality in the ERA, and helping shape 
the Work Programme. Prior to the creation of the ERA Action 5 subgroup, the ERAC Standing 
Working Group on Gender in Research and Innovation (ERAC SWG GRI) played a similar policy 
support role between 2017 and 2021, following in the footsteps of the Helsinki Group on Women 
in Science, which played a core policy support role from 1999 to 2017.  

533 

 
 
Ljubljana  Declaration  on  Gender  Equality  in  R&I889,  promoting  institutional  change  in  R&I 
organisations through Gender Equality Plans is one of the most significant policy instruments 
to  achieve  long-term  and  sustainable  advancement  towards  gender  equality  in  R&I.  As 
defined  in  the  Council  Conclusions,  adopted  on  26/11/2021,  concerning  the  Future 
governance of the ERA890, Action 5 of the ERA Policy Agenda 2022-2024 – ‘Promote gender 
equality  and  foster inclusiveness,  taking  note  of  the  Ljubljana  Declaration’  –  is  associated 
with the development of four expected outcomes: 

•  Develop  a  policy  coordination  mechanism  to  support  all  aspects  of  gender  equality 
through inclusive Gender Equality Plans and policies, and a dedicated EU network on 
their implementation; 

•  Strategy  to  counteract  gender-based  violence  including  sexual  harassment  in  the 
European R&I system, and to assure gender equality in working environments through 
institutional change in any research funding or performing organisations;  

•  A policy approach to strengthen gender equality that addresses gender mainstreaming, 

to advance the new ERA; and 

•  Develop  principles  for  the  integration  and  evaluation  of  the  gender  perspective  in 
research  and  innovation  content  in  cooperation  with  national  Research  Funding 
Organisations.  

1.2.1.2. 

Gender equality under the EU´s Framework Programmes for R&I 

An  evaluation  study  on  the  implementation  of  cross-cutting  issues  under  Horizon  2020891, 
published in March 2023, found that the gender balance indicators remained relatively stable 
throughout the Horizon 2020 period, with the highest shares being achieved in the last two 
years (2020-2021), especially for women in Horizon 2020 projects and women coordinators. 
This  could  be  explained  by  the  commitment  towards  gender  equality  in  research  and 
innovation as part of the European Commission’s Strategic engagement for gender equality 
in  all  EU  policies  during  the  Horizon  2020  period.  Under  its  successor  FP,  Horizon 
Europe892,  the  European  Commission  reaffirms  its  commitment  to  gender  equality  in 

889 The Ljubljana Declaration was spearheaded by the Slovenian Presidency and co-drafted by 
the two Presidency Trios (DE, PT, SI and FR, CZ, SE), through the ERAC SWG GRI, with 
feedback from RTD. It was officially presented to EU Member States by the Slovenian Minister at 
the 28 September 2021 COMPET Council meeting, as well as at the 26 November 2021 
COMPET, eventually being endorsed by 25 EU Member States, as well as 11 other countries 
including 10 associated countries or candidates for association with Horizon Europe. The 
European Commission also endorsed it through Commissioner Mariya Gabriel. Three additional 
countries, while not officially endorsing the Declaration, have expressed explicit support for it 
(the UK, Turkey and Australia), available at: 
https://www.gov.si/assets/ministrstva/MIZS/Dokumenti/PSEU/Ljubljana-Declaration-on-Gender-
Equality-in-Research-and-Innovation-_endorsed_final.pdf. 
890 Available at: https://data.consilium.europa.eu/doc/document/ST-14308-2021-INIT/en/pdf.  
891 European Commission, Directorate-General for Research and Innovation, Naujokaitytė, R., 
Mačiulienė, M., Stančiauskas, V., et al., Evaluation study on the implementation of cross-cutting 
issues in Horizon 2020 – Study in support of the ex-post evaluation of the European Framework 
Programme for Research and Innovation Horizon 2020. Annex 3, Case study report, Dirham, S. 
(editor), Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/331671. 
892 https://research-and-innovation.ec.europa.eu/funding/funding-opportunities/funding-
programmes-and-open-calls/horizon-europe_en. 

534 

 
 
R&I.  The  goal  is  to  improve  the  European  R&I  system,  create  gender-equal  working 
environments where all talents can thrive, and better integrate the gender dimension in 
projects to improve research quality as well as the relevance to society of the knowledge, 
technologies and innovations produced893. 

In practical terms, and to a large extent confirming the objectives set out under Horizon 2020, 
gender equality is addressed in three main ways under Horizon Europe 894:  

• 

Increasing gender balance throughout the programme, with a target of 50% women 
on Horizon Europe-related boards, expert groups and evaluation committees (compared 
with 40% under Horizon 2020) and the introduction of gender balance among research 
teams as a criterion for ranking proposals with the same score.  

•  From 2022 onwards, having a Gender Equality Plan (GEP) is a criterion for eligibility in 
certain categories of legal entities from EU countries and non-EU countries associated 
with  Horizon  Europe:  public  bodies,  research  organisations  and  higher  education 
establishments.  Detailed  guidance895  published  in  2021,  as  well  as  other  forms  of 
assistance such as training sessions896 and webinars897 support organisations in meeting 
the Horizon Europe GEP eligibility criterion. 

•  The integration of a gender dimension into R&I content is a requirement by default 
and is evaluated under the criterion of excellence unless the topic description explicitly 
specifies otherwise.  

Furthermore, specific funding is dedicated towards: 1) Gender and intersectional research 
promoted under different parts of Horizon Europe, in particular under the Cluster 2 – Culture, 
Creativity and Inclusive Society work programme898; 2) Developing inclusive gender equality 
policies in support of the new ERA, through the Widening Participation and Strengthening 
the  ERA  programme  part  (WIDERA)899;  and  3)  Empowering  women  innovators,  in 
particular  through  Pillar  3  of  the  FP  –  Innovative  Europe,  and  through  the  European 
Innovation Council (EIC)900. 

893 Extracted from the Europa website: https://research-and-
innovation.ec.europa.eu/strategy/strategy-2020-2024/democracy-and-rights/gender-equality-
research-and-innovation_en#gender-equality-in-horizon-europe. 
894 See the factsheet on gender equality provision under Horizon Europe, available at: 
https://op.europa.eu/en/publication-detail/-/publication/51704c8d-ca5f-11eb-84ce-
01aa75ed71a1. 
895 Available at: https://op.europa.eu/en/publication-detail/-/publication/ffcb06c3-200a-11ec-
bd8e-01aa75ed71a1/language-en/format-PDF/source-232129669. 
896 For example, in 2021 and 2022 the Commission offered a series of online training sessions 
on Gender Equality Plans, mainly targeting organisations in Bulgaria, Greece, Hungary, Croatia, 
Italy, Bosnia and Herzegovina, Romania, Poland, Estonia, Latvia, Lithuania and Cyprus. 
897 For example, in June 2022, DG R&I organised a webinar aimed towards LEARs. 
898 For more information, see: https://research-and-innovation.ec.europa.eu/funding/funding-
opportunities/funding-programmes-and-open-calls/horizon-europe/cluster-2-culture-creativity-
and-inclusive-society_en. 
899 For more information, see: https://research-and-innovation.ec.europa.eu/funding/funding-
opportunities/funding-programmes-and-open-calls/horizon-europe/reforming-and-enhancing-
european-research-and-innovation-system_en. 
900 Several initiatives exist in this regard, for example, in 2022, in its Communication ‘A new 
European Agenda’ (COM (2022)), the European Commission proposed the launch of a Pilot 

535 

 
 
 
1.2.2.  Overall approach and scope of the case study 

The  present  case  study  examines  how  Horizon  Europe  contributes  to  fostering  gender 
equality in the R&I systems, and the progress made through the adoption and first steps in 
the implementation of the current FP. The case study takes a Framework Programme-wide 
approach,  analysing  all  pillars  and  programme  parts  of  Horizon  Europe  with  the  aims  of 
identifying FP-wide lessons.  

The  case  study  relies  on  several  methods  including  extensive  desk  research  (a  list  of 
sources  consulted  for  this  case  study  can  be  found  in  Appendix  1),  and  an  analysis  of 
administrative,  monitoring  and  reporting  data  yielded  information  on  gender  statistics 
under  the  Horizon  Europe  and  its  predecessor  FP.  A  total  of  eleven  interviews  with 
Framework Programme participants and other stakeholders were conducted to complement 
the  evidence  on  the  impact  of  the  Framework  Programmes  in  promoting  gender  equality.  
Lastly, the case study draws on the evidence collected through surveys of beneficiaries and 
unsuccessful applicants carried out under this study. In particular, these surveys examine the 
extent  to  which  applicants  consider  the  instructions  given  in  the  application  process  with 
regard to the GEP to be clear. The case study provides information on the evaluation criteria 
related to relevance, effectiveness and EU added value. 

1.3.  Case study findings  

1.3.1.  Relevance 

The latest available data on gender equality in R&I across Europe shows that there is still 
progress to be made in achieving gender equality. Table 1 presents an overview of recent 
data from the Commission’s ‘She Figures’ 2021 report901 showing that higher career stages 
display stronger differences with regard to the participation of women and men in R&I. These 
differences  are  even  more  acute  in  STEM  fields  (science,  technology,  engineering  and 
mathematics), figures for which are shown at the right-hand side of the table. These figures 
show that there is still a need for policy action, especially considering that gender equality is 
well-embedded into the EU’s principles and values and in the context of EU policy priorities. 

Women among graduates 

Women among doctoral graduates 

General 
situation 

59% 

47% 

In STEM fields 
(science, technology, 
engineering and 
mathematics) 

35% 

35% 

European innovation gender and diversity index for the first quarter of 2023. The European 
Innovation Council took up this call and included a call for proposals on this topic. For more 
information, see: https://eic.ec.europa.eu/news/statement-gender-diversity-eic-2021-06-03_en. 
More examples include the) new WomenTechEU scheme and the Women Leadership 
Programme for EIC companies. For more information, see: 
https://eic.ec.europa.eu/system/files/2022-06/EIC-work-programme-2022-06-09.pdf. 
901 European Commission (2021). She figures 2021, available at: https://research-and-
innovation.ec.europa.eu/knowledge-publications-tools-and-data/publications/all-publications/she-
figures-2021_en. 

536 

 
 
 
Women among grade B academic staff 

Women among grade A academic staff 

Women among board members (including 
leaders)  

Women among board leaders in research 
organisations 

General 
situation 

40% 

26% 

31% 

24.5% 

In STEM fields 
(science, technology, 
engineering and 
mathematics) 

28% 

19% 

N/A 

N/A 

Table 1. Gender equality in R&I (She Figures 2021). Source: Data from 2018 and 2019 for the EU-27, She Figures 2021, 
Directorate-General for Research and Innovation (European Commission). 

A  public  open  consultation  on  the  past,  present  and  future  of  the  EU  R&I  Framework 
Programmes  2014-2027902  was  held  between  December  2022  and  February  2023.  The 
findings  from  this  targeted  stakeholder  consultation  also  indicate  that  gender  equality 
continues  to  be  highly  relevant  among  stakeholders:  855  out  of  1,943  respondents  (43%) 
considered gender equality and inclusiveness an essential or high-priority theme to be further 
elaborated under the Strategic Plan 2025-2027, while 519 correspondents (27%) considered 
it to have medium priority. Conversely, 397 respondents (21%) considered gender equality 
and inclusiveness to be of low or no priority.  

Figure 1. Gender equality and inclusiveness as a priority in the 2025-2027 Strategic Plan. Source: Public consultation on the 
past, present, and future of the European Research and Innovation Framework programmes 2014-2027 (2023; n=1,943). 

902 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 

537 

 
 
 
 
1.3.2.  Effectiveness 

In general, more than half of respondents (804 out of 1,579 respondents) to the targeted 
stakeholder consultation903 agreed or strongly agreed with the statement that in the calls 
for  proposals  under  Horizon  Europe,  the  strengthened  gender  equality  provisions 
provide  the  potential  to  promote  gender  equality  across  R&I  organisations  and 
activities, compared with 199 respondents (13%) who disagreed or strongly disagreed. This 
illustrates  a  relatively  positive  perception  among  these  stakeholders  that  the  new 
requirements under Horizon Europe are effective in addressing gender equality. At the same 
time, a relatively large share of respondents indicated that they were neutral or had no opinion 
on this statement. 

1.3.2.1. 

Increasing gender balance throughout the programme  

Horizon 2020 was the first Framework Programme to adopt gender equality as a cross-cutting 
issue. Table 2 shows the progress made in relation to gender equality under Horizon 2020 
across three key indicators. Although the share of women participants in projects  stood at 
42%, the share of women coordinators under Horizon 2020 was only 23%, while the share 
of women in EC expert evaluation panels stood at 42% – above the 40% target. 

Indicators 

Horizon 2020 

Horizon 
Europe 

Percentage of women among participants in FP projects 

42.2%* 

(1) 

Percentage of women coordinators in FP projects 

23.0%** 

39.1% 

Percentage of women expert evaluators 

42%*** 

43.4% 

Table 2. Performance of cross-cutting indicators under Horizon 2020 – gender balance indicators. Source: Commission Staff 
Working Document of the Ex-post evaluation of Horizon 2020 (for the Horizon 2020 data) and data shared by the European 
Commission in January 2024 (for the Horizon Europe data). 

*This indicator is computed on the basis of self-declared data submitted by the coordinator in continuous reporting. The 
indicator measures the composition of the entire project team, including both scientific and non-scientific personnel. 

**Share of projects for which the main contact is reported as a woman, out of the total number of projects in the programme. 
Projects for which gender data was not reported are not included. 

***This indicator is based on self-declared data submitted by the expert in the Expert Portal (gender). The percentage is 
computed on the overall number of experts evaluators contracts signed for which the experts have declared their gender. 

(1) This data is not yet available in Horizon Europe. 

Horizon Europe aims to achieve a target of 50% women on Horizon Europe-related boards 
and  in  expert  groups  and  evaluation  committees  and  has  introduced  gender  balance  in 
research teams as a criterion for ranking proposals with the same score. The latest figures 
based  on  data  shared  by  the  European  Commission  on  the  percentage  of  women  in  EC 
expert evaluation panels show that a small progress had been made under Horizon Europe, 
with a share of 43% versus 42% under Horizon 2020. Regarding the percentage of women 

903 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 

538 

 
 
coordinators under Horizon  Europe, the latest figures show a share of 39%, which is also 
significantly higher than the share in Horizon 2020 (23%).  

Since the beginning of Horizon Europe, the programme has collected detailed information on 
the  researchers  involved  in  the  Horizon  projects  (in  addition  to  information  on  project 
coordinators).  These  changes  are  also  a  response  to  the  shortcomings  identified  by  the 
evaluation study on the implementation of cross-cutting issues in Horizon 2020, with regard 
to KPI 1s and 2904. The monitoring data on researchers of funded projects, provided to the 
study team as preliminary and partial data in April 2023, indicates that the share of female 
researchers  in  Horizon  Europe  project  teams  is  still  well  below  50%905.  While  this 
underrepresentation  occurs  at  all  career  stages,  the  data  also  show  that  the  imbalance  is 
more  pronounced  at  later  career  stages  (senior  researchers/associate  professors  and  full 
professors/directors of research; see Figure 2). Although these data must be considered with 
caution, due to their preliminary and partial character, this situation does mirror the general 
situation across Europe, in which the shares of women in research across different career 
stages display a similar pattern906.  

Total Researchers (n=13,8318)

37%

63%

Category D2 - Other First stage researcher (n=4,134)

Category D1 - First Stage Researcher (PhD student)
(n=2,840)

Category D – First stage researcher (n=18,109)

Category C2 - Recognised Researcher (Post Doctoral
researcher) (n=2,876)

Category C1 - Recognised Researcher (Researcher/
Assistant professor) (n=28,774)

Category B - Senior Researcher (Senior Researcher/
Associate professor) (n=53,049)

Category A - Top Grade Researcher (Full professor/Director
of research) (n=32,460)

44%

43%

56%

56%

48%

52%

44%

43%

56%

57%

35%

26%

64%

74%

0%

50%

100%

Women

Men

Figure 2. Female representation across career stages in Horizon Europe-funded project teams. Data shared by the 
European Commission (January 2024). 

Based  on  two  datasets  provided  to  the  study  in  May  2023  and  January  2024,  only  two 
programme  parts  have  a  share  of  female  researchers907  above  50%:  Reforming  and 

904 Under Horizon 2020, KPI 1 measured the percentage of women participants in Horizon 2020 
projects (total workforce) and KPI 2 measured percentage of women project coordinators in 
Horizon 2020 projects, including Marie Skłodowska-Curie Actions (MSCA) fellows, European 
Research Council (ERC) principal investigators and scientific coordinators in other Horizon 2020 
activities. 
905 While not officially a target, it is Horizon Europe’s objective to increase gender balance in 
research teams (and among those innovators funded under Pillar 3). The extent to which gender 
balance (i.e. a 50% share) is achieved in a proposal’s research team is a criterion for ranking ex 
aequo proposals – coming third in priority order among the ranking criteria. 
906 See, for instance, the data included in the ‘She Figures’ report and in the MORE studies. 
907 Data on applying teams include fellows and researchers. 

539 

 
 
 
enhancing the European R&I System, and European Innovation Ecosystems (see Table 3). 
Three Clusters (2, 1 and 6) and the Widening Participation programme part each have shares 
of between 40% and 50%, while the remaining programme parts and clusters have shares of 
female researchers substantially below 50%. Overall, the shares of female researchers in the 
teams of all project applications is around 36%, while in teams of funded projects it is around 
37%. In addition, Horizon Europe includes a novelty: the self-declaration of gender identity 
by  researchers  as  woman,  man  or  non-binary.  Preliminary  data  show  that  non-binary 
researchers so far represent around 0.1% of researchers in the teams of funded projects. 

Programme parts 

Shares of female 
researchers in 
applying teams 

Shares of female 
researchers in 
funded project 
teams 

Reforming and enhancing the European R&I 
System 

55.9% (n=4,007) 

55.4% (n=1,792) 

European Innovation Ecosystems 

52.4% (n=2,549) 

54.0% (n=895) 

Cluster 2- Culture, Creativity and Inclusive Society 

48.1% (n=30,839) 

48.6% (n=6,487) 

Cluster 1 - Health 

44.4% (n=53,959) 

47.9% (n=18,291) 

Cluster 6 - Food, Bioeconomy, Natural Resources, 
Agriculture and Environment  

43.6% (n=69,753) 

45.0% (n=21,932) 

Widening participation 

42.5% (n=17,589) 

44.7% (n=5,242) 

Marie Skłodowska-Curie Actions 

36.0% (n=80,688) 

38.0% (n=18,727) 

The European Innovation Council 

34.2% (n=53,937) 

32.8% (n=6,772) 

The European Research Council 

33.4% (n=16,965) 

35.6% (n=700) 

Research Infrastructures 

31.5% (n=9,047) 

33.7% (n=2,577) 

Cluster 5 - Climate, Energy and Mobility 

30.5% (n=87,964) 

29.8% (n=29,489) 

Cluster 3 - Civil Security for Society 

30.5% (n=16,335) 

31.8% (n=3,492) 

Cluster 4 – Digital, Industry and Space 

TOTAL 

28.2% 
(n=115,569) 

35.9% 
(n=559,256) 

27.3% (n=29,577) 

37.0% (n=138,318) 

Table 3. Share of female researchers in project teams applying to or funded by Horizon Europe, by programme part. Note: 
data on applying teams include fellows and researchers. Data on applying teams from Horizon Europe EC administrative 
and monitoring data (May 2023). Data on funded project teams shared by the European Commission (January 2024). 

540 

 
1.3.2.2. 

Gender Equality Plans as an eligibility criterion  

As a second objective relating to gender equality under the Horizon Europe, the introduction 
of the Gender Equality Plans as an eligibility criterion aims to foster gender equality in 
public bodies such as ministries, research funding organisations, municipalities, and in both 
public and private research organisations and higher education institutions (HEIs).  

The 2021 edition of the ‘She Figures’ report908 provides an overall indication that in 2020 (i.e. 
prior  to  the  introduction  of  the  GEPs  as  an  eligibility  criterion  under  Horizon  Europe),  the 
majority  of  the  research  organisations  in  many  European  countries  had  already  taken 
measures or actions towards gender equality. However, in relation to GEPs in particular, the 
information available in the Gender Equality in Academia and Research (GEAR) tool909 and 
in a report from the ERAC Standing Working Group on Gender in Research and Innovation 
(ERAC  SWG  GRI)910  shows  that  in  several  countries  the  majority  of  research  performing 
organisations  (RPOs)  did  not  yet  have  a  GEP  by  2021.  While  some  countries  (such  as 
Austria,  Spain,  Germany  and  Sweden)  already  had  legislative  requirements  for,  at  least, 
public research organisations and universities, other countries did not yet have any legislative 
framework  fostering  the  development  of  this  type  of  plan  and/or  had  paid  little  legislative 
attention to gender equality on R&I at that time (e.g. Poland). A third group of countries had 
made steady progress in the development of GEPs from 2015-2016 onwards (e.g. Portugal, 
Croatia).  

Under the previous two EU Framework Programmes for R&I (FP7 and Horizon 2020), more 
than  200  organisations  had  been  supported  in  developing  and  implementing  a  GEP 
through 30 GEP-implementing projects with a total budget of over EUR 72 million911. While 
the majority of Horizon 2020-SwafS projects supporting the development and implementation 
of  GEPs  have  ended,  several  are  still  ongoing,  such  as  ATHENA912,  CALIPER913  and 
RESET914.  

Moreover,  in  order  to  help  applicants  understand  the  requirements  of  the  GEP  eligibility 
criterion  under  Horizon  Europe,  the  European  Commission  has  taken  several  concrete 
actions, some of which have been funded the through H2020-SwafS and Horizon Europe-
WIDERA. Examples of the outcomes include: a dedicated functional mailbox and regularly 
answered questions (around 200 since April 2021), and the publication in September 2021 
of a full guidance document, detailing each of the building blocks of a GEP, and providing 
examples of good practice at national and institutional levels, as well as links to additional 
resources915. 

Data  shared  by  the  European  Commission  in  January  2024  show  that  76.6%  of  Horizon 
Europe applicants (of type Public Bodies (PUB), Higher Education Sector (HES) or Research 
Organisations (REC)only) at proposal level had a Gender Equality Plan. By EU Member State 

908 European Commission, Directorate-General for Research and Innovation, She figures 2021 – 
Gender in research and innovation : statistics and indicators, Publications Office, 2021. Available 
at: https://data.europa.eu/doi/10.2777/06090. 
909 https://eige.europa.eu/gender-mainstreaming/toolkits/gear. 
910 Available at: https://data.consilium.europa.eu/doc/document/ST-1202-2021-INIT/en/pdf. 
911 https://research-and-innovation.ec.europa.eu/strategy/strategy-2020-2024/democracy-and-
rights/gender-equality-research-and-innovation_en. 
912 https://cordis.europa.eu/project/id/101006416. 
913 https://cordis.europa.eu/project/id/873134. 
914 https://cordis.europa.eu/project/id/101006560. 
915 Available at: https://op.europa.eu/en/publication-detail/-/publication/ffcb06c3-200a-11ec-
bd8e-01aa75ed71a1. 

541 

 
 
 
(Figure  3),  Finland  (90%),  Ireland  (88%)  and  Spain  (87%)  show  the  highest  shares  of 
applying  entities  which  submitted  a  proposal  and  had  a  GEP.  On  the  other  hand,  mainly 
Eastern European countries have the lowest shares, with Bulgaria (57%), Romania (58%) 
and Slovakia (59%) at the lower end of the graph.    

%
0
9

%
8
8

%
7
8

%
3
8

%
2
8

%
2
8

%
1
8

%
1
8

%
9
7

%
6
7

%
5
7

%
4
7

%
3
7

%
1
7

%
1
7

%
1
7

%
0
7

%
9
6

%
9
6

%
7
6

%
5
6

%
4
6

%
3
6

%
2
6

%
9
5

%
8
5

%
7
5

Figure 3. Percentage of Horizon Europe applicants (of type HES, PUB and REC) which at proposal stage had a Gender 
Equality Plan, by EU Member State. Source: data shared by the European Commission (January 2024). 

Efforts to support the development of GEPs and other activities aiming to promote gender 
equality  in  the  R&I  system  have  continued  under  the  Horizon  Europe  programme  part 
‘Reforming  and  Enhancing  the  EU  R&I  System’  in  the  WIDERA  work  programme.  Since 
having  a  GEP  is  now  an  eligibility  criterion  under  Horizon  Europe,  support  for  the 
development  of  GEPs  by  R&I  organisations  has  evolved  towards  the  implementation  of 
‘inclusive’  GEP  actions,  opening  up  to  intersectionality, as  well  as  GEP  measures  that  go 
beyond the requirements of the eligibility criterion, and policy support for the inclusive gender 
equality priorities of the new ERA framework.  

Several Horizon Europe projects have already begun in the field of gender equality under the 
WIDERA programme part, showing a balanced distribution of funding between projects that 
address  the  implementation  of  more  inclusive  GEPs  (e.g.  those  that  are  intersectional  or 
address gender budgeting), and those focusing on research activities, community building 
and  networking  e.g.  BUDGET  IT,  INSPIRE,  GENDERACTIONplus.  In  addition,  under  the 
Horizon Europe work programme for 2023 and 2024916, WIDERA call topics are even more 
directly  supportive  of  the  ERA  Policy  Agenda.  They  include,  for  example,  HORIZON-
WIDERA-2024-ERA-01-10:  Policy  coordination  to  support  all  aspects  of  inclusive  Gender 
Equality  Plans  and  policies  in  the  ERA,  and  HORIZON-WIDERA-2023-ERA-01-09:  Policy 
support to facilitate the implementation of a zero-tolerance approach towards gender-based 
violence in the ERA. 

916 Available at: https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/wp-call/2023-2024/wp-11-widening-participation-and-strengthening-the-european-
research-area_horizon-2023-2024_en.pdf. 

542 

 
 
 
Interviews  with  project  coordinators  highlighted  the  importance  of  continuing  to  provide 
support and building knowledge around the creation and implementation of GEPs, especially 
in institutions in widening countries and in some specific (parts of) institutions with a focus on 
fields of science in which gender equality is still lacking and/or underperforming (such as in 
the case of AGRIGEP). Interviews also pointed to the more general need to continue previous 
work on capacity building, pooling together previously acquired knowledge and putting it to 
good use.  

At  the  same  time,  the  evidence  suggests  that  the  support  given  to  institutions  via  the 
Framework  Programme-funded  projects  on  developing  and  implementing  GEPs  has 
incentivised  progress  at  Member  State  level  to  promote  gender  equality  in  R&I,  and  that 
further  progress  has  been  stimulated  through  the  introduction  of  GEPs  as  an  eligibility 
criterion under Horizon Europe. In fact, several interviewees and stakeholders indicated that 
even though the effect of the introduction of GEPs as an eligibility criterion might be uncertain 
at  this stage,  it is  at least seen  as  a  very  positive step towards  starting  the  discussion  on 
gender equality. For example, several position papers welcomed the inclusion of GEPs as 
an eligibility criterion under Horizon Europe as a positive step towards fostering institutional 
change  and  incentivising  the  institutions  to  act.  The  same  opinion  was  shared  by  the 
interviewees, with some referring to it as a “game-changer” or the “most impactful measure” 
in terms of gender equality, as it allows discussion on this topic to be triggered at institutional 
level.  In  addition,  in  some  countries  in  which  institutions  already  had  GEPs  in  place,  the 
requirement served to improve and further develop these previous iterations, as was the case 
of Denmark, Norway and Spain917. 

In addition, in the survey of Horizon Europe beneficiaries conducted in May-July, 2023 (Table 
4), more than half of respondents (3,288 out of 6,185, or 53%) said that they were satisfied 
to  a  ‘very  large’  or  ‘large’  extent  with  the  clarity  of  the  instructions  regarding  the  GEPs 
requested under Horizon Europe. By programme part, only the ERC shows a share of less 
than 50% (100 out 251 respondents). 

Programme 
parts 

To a very 
large extent 
or large 
extent 

To a 
moderate 
extent 

To a small 
extent or 
not at all 

Do not 
know/not 
applicable 

Responses 

ERC 

100 (40%) 

64 (25%) 

29 (12%) 

58 (23%) 

251 

MSCA 

726 (52%) 

320 (23%) 

170 (12%) 

187 (13%) 

1,403 

INFRA 

109 (60%) 

41 (22%) 

15 (8%) 

18 (10%) 

WIDERA 

196 (64%) 

61 (20%) 

24 (8%) 

27 (9%) 

Cluster 1 

248 (56%) 

96 (22%) 

24 (5%) 

74 (17%) 

183 

308 

442 

917 Extract from the GEAR tool: ‘Based on the Horizon Europe requirement, all Danish 
universities have recently stepped up their efforts in respect of Gender Equality Plans (GEPs). 
For most, this task builds on previous gender diversity and inclusion initiatives, with the Horizon 
Europe requirements serving to build momentum.’ 

543 

 
 
Programme 
parts 

To a very 
large extent 
or large 
extent 

To a 
moderate 
extent 

To a small 
extent or 
not at all 

Do not 
know/not 
applicable 

Responses 

Cluster 2 

210 (55%) 

70 (18%) 

30 (8%) 

72 (19%) 

Cluster 3 

102 (60%) 

35 (21%) 

10 (6%) 

22 (13%) 

Cluster 4 

406 (55%) 

164 (22%) 

65 (9%) 

97 (13%) 

Cluster 5 

482 (51%) 

232 (25%) 

93 (10%) 

135 (14%) 

Cluster 6 

516 (52%) 

235 (24%) 

84 (8%) 

161 (16%) 

EIC* 

156 (50%) 

66 (21%) 

35 (11%) 

52 (17%) 

EIE* 

37 (54%) 

13 (19%) 

7 (10%) 

11 (16%) 

382 

169 

732 

942 

996 

309 

68 

Total 

3,288 (53%) 

1,397 (23%) 

586 (9%) 

914 (15%) 

6,185 

Table 4. Extent of satisfaction with the clarity of the instructions given regarding the requested Gender Equality Plans, per 
programme part. *Note that the majority of the respondents are private, for-profit organisations such as SMEs or start-ups, 
which are not bound by the requirement to provide a GEP. Source: survey of Horizon Europe beneficiaries, conducted May-
July 2023. 

Interviewees and some position papers submitted to the  OPC918 indicate certain concerns 
regarding  the  introduction  of  GEPs  as  an  eligibility  criterion.  First,  despite  the  general 
satisfaction with the action, some interviewees and position papers express doubts regarding 
the effectiveness of the measure in terms of promoting actual change within institutions. 
Currently, a lack of follow-up and controls over the implementation and results of the GEPs 
may  mean  that  these  plans  do  not  result  in  real  or  significant  changes.  Second,  some 
interviewees  and  position  papers  express  doubts  as  to  the  extent  to  which  institutions 
have access to sufficient support mechanisms, knowledge and expertise required to 
develop  effective  GEPs.  Third,  some  position  papers  and  survey  respondents  raised 
concerns about the increased administrative burden imposed during the proposal phase 
by the inclusion of several cross-cutting requirements, among them the GEPs. 

1.3.2.3. 

Integration of the gender dimension into R&I content 

With regard to the integration of the gender dimension into R&I content, Horizon 2020 
was the first Framework Programme to establish gender as a cross-cutting issue, with one of 

918 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. available at: 
https://research-and-innovation.ec.europa.eu/news/all-research-and-innovation-news/results-
public-consultation-future-eu-research-and-innovation-programmes-are-now-public-2023-04-
19_en. 

544 

 
 
its underpinning objectives being to integrate the gender dimension into R&I content, leading 
to an increased number of ‘gender-flagged’ topics across the FP919. 

Under Horizon 2020, at the time of grant signature, 71% of projects were categorised by the 
project officer to indicate whether or not the project integrated the gender dimension in R&I. 
The latest figures from the Horizon Dashboard (accessed in September 2023) show that 26% 
of  Horizon  2020  projects  categorised  were  flagged  as  taking  into  account  the  gender 
dimension in R&I content. However, there are considerable variations between the different 
pillars: among those projects in the SwafS work programme that had been categorised, 77% 
were  flagged  as  taking  into  account  the  gender  dimension  in  R&I  content;  meanwhile, 
Excellent Science (excluding the ERC) displays a corresponding share of 22%. An in-depth 
analysis was carried out under the evaluation study on the implementation of cross-cutting 
issues920, which provides further information on the flagging of topics and projects with regard 
to the gender dimension.  

Under Horizon Europe, the flagging of topics for the gender dimension is no longer carried 
out, since it is a requirement by default in all RIAs, IAs and Cofunds. Under this FP, the EC 
only flags (with a clear reference in the topic description) those topics that have a derogation 
on integrating the gender dimension into their content. At project level, the integration of the 
gender  dimension  into  R&I  content  is  only  checked  by  both  the  reviewer  and  the  project 
officer at project review stage. Because very few projects under Horizon Europe have so far 
gone through project review, detailed information on this is not yet available. However, EC 
administrative and monitoring data received by the study team in June 2023 sheds some first 
light on the share of projects that were exempted from the mandatory requirement to integrate 
a gender dimension into their R&I content (Table 5). So far, only a total of 441 out of 9,459 
projects  (5%)  were  exempted.  However,  significant  variations  exist  between  the  different 
programme  parts.  As  expected,  the  largest  share of  projects  exempted  are in  programme 
parts  more  related  to  industry,  infrastructure  and  technology.  For  example,  for  Research 
Infrastructures  the  share is  52%,  followed  by  Cluster  4  –  Digital,  Industry  and  Space  with 
40%, and Cluster 3 – Civil security and society, with 32%. Meanwhile, the lowest shares of 
exemptions are found in WIDERA and in Clusters 1 – Health and Cluster 5 – Climate, Energy 
and Mobility. Nevertheless, these figures should be approached with caution due to a lack of 
data regarding some programme parts.  

Share of 
projects 

Number of 
projects 

Total 
number of 
projects 

Research Infrastructures 

52% 

42 

Cluster 4 – Digital Industry and Space 

40% 

252 

81 

625 

919 Extracted from the Europa website, available at: https://research-and-
innovation.ec.europa.eu/strategy/strategy-2020-2024/democracy-and-rights/gender-equality-
research-and-innovation_en#gender-mainstreaming-through-the-integration-of-the-gender-
dimension-in-research-and-innovation-content . 
920 European Commission, Directorate-General for Research and Innovation, Naujokaitytė, R., 
Mačiulienė, M., Stančiauskas, V. et al., Evaluation study on the implementation of cross-cutting 
issues in Horizon 2020 – Study in support of the ex-post evaluation of the European Framework 
Programme for Research and Innovation Horizon 2020, Dirham, S. (editor), Luxembourg: 
Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/763665.  

545 

 
 
 
Share of 
projects 

Number of 
projects 

Total 
number of 
projects 

Cluster 3 – Civil Security for Society 

Cluster 6 – Food, Bioeconomy, Natural Resources, 
Agriculture and Environment 

Reforming European R&I and Research 
Infrastructures 

Cluster 1 – Health 

Cluster 5 – Climate, Energy and Mobility 

Cluster 2 – Culture, Creativity and Inclusive Society 

European Innovation Council 

European Innovation Ecosystems 

European Institute of Innovation and Technology 

Widening Participation 

TOTAL 

32% 

24% 

7% 

2% 

1% 

0% 

0% 

0% 

0% 

0% 

5% 

31 

98 

4 

6 

8 

97 

411 

56 

316 

617 

147 

613 

100 

17 

278 

441 

9,459 

Table 5. Number and share of Horizon Europe projects exempted from the mandatory requirement to integrate a gender 
dimension into their R&I content. Note: figures do not cover the Pillar 1 Programme parts ERC and MSCA. Source: Horizon 
Europe EC administrative and monitoring data, 26 June 2023. 

At the level of the topics from the different calls, data shared by the European Commission 
in January 2024 shows that 19.5% or 232 out of 1,192 topics do not integrate the gender 
dimension in their content.  

Like the support given to the implementation of the GEPs, the European Commission has 
also provided resources to help applicants comply with the above requirements. For example, 
in November 2020, an expert group published a report921 on Gendered Innovations to provide 
researchers  and  innovators  with  methodological  tools  for  the  analysis  of  sex,  gender  and 
intersectional  issues.  It  also  presents  concrete  case  studies,  highlighting  projects  funded 
under Horizon 2020 and addressing key research and innovation areas for Horizon Europe 
clusters, missions and partnerships. These include areas such as health, artificial intelligence 

921 European Commission, Directorate-General for Research and Innovation, Gendered 
innovations 2 – How inclusive analysis contributes to research and innovation – Policy review, 
Luxembourg: Publications Office of the European Union, 2020, available at: 
https://data.europa.eu/doi/10.2777/316197. 

546 

 
 
 
 
 
 
 
 
and  robotics,  energy,  transport,  marine  science  and  climate  change,  urban  planning, 
agriculture, fair taxation and venture funding, as well as the COVID-19 pandemic.  

1.3.2.4. 

Achievement of gender equality through additional specific funding 
elements aimed at promoting gender equality and gender research in 
various parts of Horizon Europe, developing inclusive gender equality 
policies, and empowering women innovators 

Under Clusters 2 and 4, several projects are also being funded to address gender equality 
research,  with  a  total  EU  net  contribution  of  more  than  EUR  34  million.  These  projects 
address various topics such as intersectionality, the impact of AI, and inclusion (see Appendix 
2).  

It is also important to note that, while they apply a ‘bottom-up’ approach, both the MSCA and 
the ERC contribute to advancing research and knowledge creation in the scientific field of 
gender studies. Information provided by the EC points to a total of 82 MSCA projects so far 
that have contributed to ‘women and gender studies’ or ‘gender history’, with a total of more 
than  EUR  23  million  in  EU  net  contribution  and  116  organisations  involved  (some  of  the 
projects have substantial budgets of more than EUR 2 million). With regard to ERC projects, 
Horizon  Dashboard922  data  shows  that  only  two  ERC  grants  have  been  signed  in  gender 
studies. However, using the ERC ERIS tool923, the study team has been able to identify a 
long list of ERC projects addressing gender studies (see Appendix 3 for some examples of 
gender-focused MSCA and ERC projects under Horizon Europe).  

Also  other  projects  funded  under  WIDERA,  such  as  GILL  and  INSPIRE,  contribute  to 
advancing knowledge on gender studies, beyond the GEPs. 

Under Pillar 3, the Women TechEU scheme924 offers first-class coaching and mentoring to 
female  founders,  as  well  as targeted  funding  to  help  take  their  business  to  the  next level, 
while  the  EIC  Women  Leadership  Programme925,  open  to  female  researchers  and 
entrepreneurs from the EIC Community and EIT Supernovas, offers specific training sessions 
including  interactive  exercises  and  discussions,  as  well  as  networking  events,  and 
personalised  mentorship  and  business-coaching  schemes.  For  example,  figures  from  the 
Europa website show that the Women TechEU programme has already supported 184 deep-
tech  companies  founded  by  women  in  2021-2022926,  and  given  priority  to  women-led 
companies with regard to interviews for the EIC Accelerator scheme, enabling them to go 
from 8% in the pilot EIC in 2019, to 29% in June 2021927. 

Additional analysis carried out by the study team928 on how various projects contribute to the 
achievement  of  the  SDGs  shows  that,  as  expected,  projects  funded  under  WIDERA 
contribute the most to SDG 5 – Gender Equality (26.7%, or 87 out of 326 projects assessed), 
followed  by  projects  funded  under  Pillar  1,  with  11.7%  (659  out  of  5,569  projects)  –  in 
particular, projects funded under the MSCA, with 18.2% (8 out of 79 projects).  

922 Accessed 10 September 2023. 
923 https://erc.europa.eu/projects-statistics/advanced-analytics-tool-eris. 
924 https://eismea.ec.europa.eu/programmes/european-innovation-ecosystems/women-
techeu_en. 
925 https://eic.ec.europa.eu/eic-funding-opportunities/business-acceleration-services/eic-women-
leadership-programme_en. 
926 Ibid. 
927 https://eic.ec.europa.eu/news/statement-gender-diversity-eic-2021-06-03_en. 
928 For more details, see Annex 6.  

547 

 
 
Overall, Horizon Europe appears to be a frontrunner among the other Multiannual financial 
framework  (MFF)  programmes,  thanks  to  its  heightened  gender  equality  provisions.  Data 
extracted from the Europa website provides preliminary insights into the contribution made 
by  Horizon  Europe  to  promoting  gender equality,  based on  a  scoring system  (Table  6)929. 
The figures show that Horizon Europe contributed a total of EUR 96.6 million over 2021 and 
2022 to interventions for which the improvement of gender equality is a principal objective 
(score 2). Examples of such contributions include the Women Leadership Programme (EIC), 
the EU Prize for Women Innovators (EIC), and the Supernovas (EIT). Under Score 1 (i.e. 
interventions that have gender equality as an important and deliberate objective but not as 
the main reason for the intervention), Horizon Europe has contributed approximately EUR 
1,684 million. These include projects such as those funded under the EU’s Cancer Mission.  

Gender score 

2021 

Score 2 

Score 1 

37.7 

160.0 

2022 

58.9 

Total 

96.6 

1,524.3 

1,684.3 

Score 0* 

11,195.9 

10,390.9 

21,586.8 

Score 0 

0.0 

265.0 

265.0 

Total 

11,393.6 

12,239.1 

23,632.7 

Table 6. Contribution to gender equality (million EUR). Source: Europa website, available at: 
https://commission.europa.eu/strategy-and-policy/eu-budget/performance-and-reporting/programme-performance-
statements/horizon-europe-performance_en#contribution-to-horizontal-priorities (accessed September 2023). 

Note:  
- 
- 

- 

- 

Score 2: interventions for which the principal objective is to improve gender equality; 
Score  1: interventions that  have  gender  equality as an important and deliberate objective, but not  as the main 
reason for the intervention; 
Score  0:  non-targeted  interventions  (interventions  that  are  expected  to  have  no  significant  bearing  on  gender 
equality); 
Score 0*: score to be assigned to interventions with a likely but not yet clear positive impact on gender equality. 

1.3.3.  EU added value 

The evidence collected for the elaboration of this case study indicates that the main areas of 
EU  added  value  of  the  actions  undertaken  under  the  Framework  Programmes,  and 
particularly during the first stages of Horizon Europe, are as follows:  

•  Promoting  the  creation  of  knowledge  on  and  the  full  integration  of  the  gender 
dimension  in  R&I.  An  important  element  of  EU  added  value  with  respect  to  gender 
equality is the creation of a knowledge base through EU-funded research, at a scale that 
is  not  possible  at  country  level.  For  example,  EU  support  allows  longer  and  bigger 
projects  with  multiple  partners  from  different  countries,  which  are  important  factors  in 
conducting field work and pooling together knowledge. This aspect can be seen across 
several programme parts, such as the ERC, MSCA and Pillar 2 (Clusters 2 and 4). At 
the same, the integration of the gender dimension in R&I as a default in RI, RIAs and 

929 This tracking is part of an EC commitment deriving from a 2020 Inter-institutional agreement, 
available at: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32020Q1222(01). 

548 

 
 
Cofund  actions  helps  to  raise  awareness,  exchange  practices  and  promote  gender 
equality in R&I across Europe.  

•  Supporting the development of Gender Equality Plans. Although having in place a 
Gender  Equality  Plan  or  similar  measure  was  already  mandatory  by  law  in  several 
European  countries930,  this  situation  varied  greatly  between  countries.  In  addition,  in 
many cases, this situation was heightened by a lack of resources, including insufficient 
financial resources and expertise at national level to help organisations develop a GEP. 
The role of the Framework Programme in funding projects to fill this gap by providing the 
necessary expertise and financial resources to help institutions develop their GEPs, was 
considered very positive. This assessment relates mostly to the SwafS projects under 
Horizon  2020931,  which  targeted  the  development  of  GEPs  in  many  European 
organisations.  However,  new  WIDERA  projects  continue  to  provide  support,  but  now 
focus on developing more inclusive GEPs. Introducing the requirement to have a GEP 
in place in order to obtain Horizon Europe funding has also produced important results 
and  contributed  to  institutional  changes.  This  is  viewed  very  positively  by  both 
interviewees and in the position papers submitted to the OPC932. Some doubts remain 
as  to  whether  this  support  is  sufficient  to  foster  the  development  of  effective  and 
sustainable GEPs in a large number of organisations across Europe. The introduction of 
GEPs as an eligibility criterion is in general welcomed by organisations, although there 
are still concerns regarding the effectiveness and implementation of the GEPs that have 
been developed for the purposes of participating in Horizon Europe. 

•  Pooling  together  experience,  expanding  the  knowledge  base  and  providing 
support in relation to the GEPs. European projects funded under both Horizon 2020 
and Horizon Europe aim to create a solid knowledge base on Gender Equality Plans, 
which  can  be  exploited  by  organisations  across  Europe.  For  example,  following  the 
development and publication of the Gender Equality Plans by organisations, the main 
challenges are how to implement these plans, how to improve specific capacities and 
capabilities  within  the  institutions,  and  how  to  generate  mutual  learning  among 
institutions. In addition, interviews indicate that institutions often face a lack of support 
and of contacts with actors who possess longstanding experience in this particular field. 
Interviewees indicated that one of the main benefits of participating in Horizon Europe 
projects is to be able to connect with more experienced partners and mentors in other 
countries  to  build  and  define  a  GEP  that  is  both  effective  and  sustainable.  This  is 
particularly important for institutions with less experience in gender equality matters in 
R&I,  especially  in  STEM  scientific  fields,  in  which  the  need  for  progress  on  gender 
equality is more acute, such as in agriculture and some fields of life sciences. Another 
relevant  point  in  relation  to access  to  knowledge  and  experience  is  the importance of 
networking  and  the  exchange  of  best  practices.  Interviews  carried  out  during  the 
Evaluation study  on  Excellent  Science  under  Horizon  2020  indicated  that  the  Horizon 

930 For example, in Spain, the Organic Law for Effective Equality between Women and Men 
(3/2007) introduced gender equality as a basic principle for public action, applicable to 
universities and research centres. This law made it compulsory for institutions and companies 
with more than 50 employees to adopt and implement gender equality plans. Having a GEP has 
been an ERA priority and recommended tool since the 2012 ERA Communication, and several 
EU MS had already committed to this through dedicated Council Conclusions and in their ERA 
NAPs. 
931 Such as the SPEAR project, CALIPER project, TARGET project and PLOTINA project. 
932 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 

549 

 
 
2020  action  ‘sister  projects’933  and  the  new  Horizon  Europe  GENDERACTIONplus 
project  are  particularly  useful  in  this  regard,  as  they  have  fostered  the  exchange  of 
information and awareness between projects in the area of gender equality. Similarly, 
interviewees  indicate  that  fostering  the  opportunity  for  more  joint  activities  and 
exchanges of best practice, such as more training activities and events, would help to 
strengthen  the  community  and  its  capacities  for  knowledge  exchange  and  the 
identification  of  best  practices,  and  ultimately  aid  the  implementation  and  updating  of 
GEPs. Lastly, the European Commission has taken several actions providing extensive 
information  to  support  the  implementation  of  the  GEPs  (see  the  section  on 
Effectiveness).  

•  Promoting gender equality values and knowledge, in and beyond the EU. European 
projects funded under Horizon Europe aim to share values and knowledge about gender 
equality, and to exchange best practices with third  countries. Funding available under 
‘Reforming  and  enhancing  the  European  R&I  System’  is  designed  to  go  beyond  the 
requirement to have a GEP as an eligibility criterion; in projects such as the AGRIGEP934 
project, in line with the call topic, the team aims to learn from more advanced partners 
and gain support on how to integrate gender equality issues into teaching and training 
materials for agricultural and life sciences. Part of the objective of this is to share gender 
equality insights with foreign students who come from third countries, with the intention 
that these students will take this knowledge back to their home countries. The project 
BUDGET  IT935,  which  began  in  2023,  aims  to  use  gender+  budgeting936  to  transform 
institutions in order to advance inclusive gender+ equality and enhance the reputation, 
inclusiveness and research excellence of the widening countries of Bosnia, Serbia and 
Turkey, assisted by leading university counterparts in Italy and Spain.  

•  Dissemination of and access to knowledge on gender equality. In spite of the efforts 
made under previous Framework Programmes and the establishment of specific tools 
such  as  GEAR,  interviews  and  a  number  of  position  papers  submitted  the  OPC937 
indicate  that  it  is  often  still  difficult  for  institutions  to  gain  access  to  the  information, 
knowledge,  resources  and  expertise  necessary  for  the  development  of  effective  and 
sustainable GEPs. Under Horizon Europe, the INSPIRE project is expected to set up a 
European Centre of Excellence on inclusive gender equality in R&I to address this gap. 
In  addition,  the  EU  Award  for  Gender  Equality  Champions,  which  recognises  and 
celebrates outstanding results achieved in the implementation of GEPs by academic and 
research organisations, can be used as a promotional tool to showcase success stories 
and promote role models. Other actions include funding to promote gender equality, for 
example through the establishment of an EU Gender Equality Competence Facility, and 

933 https://h2020.genderaction.eu/resources/sister-projects/. 
934 Assessment and implementation of Agriculture and Life Science Universities’ first Gender 
Equality Plans in widening countries. AGRIGEP Project, available at: 
https://cordis.europa.eu/project/id/101094158. 
935 BUDGET IT project; https://budget-it.eu/. 
936 Gender budgeting is a strategy to achieve equality between women and men by focusing on 
how public resources are collected and spent. For more information, see: 
https://eige.europa.eu/gender-mainstreaming/tools-methods/gender-budgeting. 
937 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 

550 

 
 
the continued monitoring of the state of gender equality in R&I through the publication of 
‘She Figures’ reports938. 

1.4.  Success factors and barriers 

The following main factors fostering the progress and impact of the actions to promote gender 
equality have been identified:  

•  The importance of building consortia involving diverse types of partners. One of 
the main factors for success identified during interviews with projects addressing different 
aspects  of  gender  (e.g.  gender  studies  or  GEPs)  concerns  the  types  of  consortia 
involved.  Interviews  carried  out  for  this  report  and  during  the  Evaluation  study  on 
Excellent  Science  under  Horizon  2020  indicate  that  the  inclusion  of  various  types  of 
institutions,  from  different  countries,  with  diverse  degrees  of  experience  on  gender 
equality, allows the better exchange of knowledge and best practices. In addition, many 
of the more experienced partners have already participated in previous projects relating 
to gender equality, which is seen as positive for knowledge diffusion. The inclusion of 
third-country partners is also seen as a success factor, as it allows the exchange and 
spillover of knowledge beyond Europe.  

in  particular  sister  projects.  Some 

•  Cooperation  and  exchange  between  projects.  Another  success  factor  concerns 
interviewees 
cooperation  between  projects, 
highlighted the importance of combining efforts, arguing that a stronger impact can be 
achieved by working together. Therefore, cooperation across funded projects should be 
continued  and  further  promoted.  In  addition,  the  sharing  of  and  access  to  knowledge 
from  previous  successful  projects,  as  well  as  access  to  tools  such  as  GEAR,  were 
mentioned in interviews as a factor contributing to the success of actions. This knowledge 
is seen as critical to laying the foundations for new projects, in order to avoid duplications 
and ensure progress, and to build GEPs.  

•  Communication.  According  to  interviewees,  convincing  people,  and  in  particular 
researchers,  of  the  importance  of  gender  equality,  especially  the  contribution  gender 
equality can make to excellent science, is not always straightforward. The importance of 
highlighting available evidence and positive examples, such as how the integration of the 
gender  dimension  is  applied  in  specific  fields  of  life  sciences  and  engineering,  can 
contribute  to  achieving  a  higher  impact,  as  this  can  trigger  action  by  individuals  and 
organisations.  Such  measures  include  actions  already  taken  by  the  European 
Commission,  such  as  the  Gendered  Innovations  2  report939  published  in  2020,  or  the 
videos produced as an output of the Horizon 2020 ACT project940. The creation under 
Horizon Europe of the EU Award for Gender Equality Champions can also serve as an 
excellent communication tool to showcase success stories and best practices.  

938 Extracted from the REA website, available at: https://rea.ec.europa.eu/news/tackling-gender-
equality-research-and-innovation-2022-03-07_en. 
939 https://reserch-and-innovation.ec.europa.eu/knowledge-publications-tools-and-
data/publications/all-publications/gendered-innovation-2-how-inclusive-analysis-contributes-
research-and-innovation_en. 
940 https://cordis.europa.eu/project/id/788204. 

551 

 
 
The  main  barriers  and  challenges  to  progress  on  gender  equality  in  R&I  that  have  been 
identified, include: 

•  Limited  resources.  One  important  barrier  identified  in  interviews  is  limited  financial 
resources, time and access to qualified experts. These factors are perceived as hindering 
the implementation and dissemination of project activities (e.g. the promotion of project 
results through more effective communication activities). In addition, some interviewees 
thought that larger project budgets would also allow actions to have a greater impact on 
the  integration  of  the  gender  dimension  into  research  content  and  into  teaching  and 
training materials. Interviewees also mentioned that more resources and longer projects 
would allow more opportunities for networking and engagement with policymakers and 
other stakeholders. Fewer financial resources create fewer incentives for institutions to 
participate.  Second,  with  regard  to  the  implementation  of  GEPs,  lack  of  resources 
hinders those effective institutional changes that go beyond the creation of a document 
and  can  entail  real  changes  at  institutional  level.  Interviewees  argue  that  this  type  of 
institutional  change  needs  to  be  backed  by  resources,  know-how  and  an  effective 
monitoring mechanism (which also requires specific budget). 

• 

Insufficient  monitoring  of  Gender  Equality  Plans.  Despite  the  fact  that  European 
associations,  universities,  beneficiaries  and  other  stakeholders  generally  view  the 
requirement  from  2022  of  having  a  GEP  as  an  eligibility  criterion  to  apply  for Horizon 
Europe as a positive step towards addressing gender equality in R&I, several concerns 
were  raised  in  relation  to  the  implementation  of  GEPs  and  their  impact  on  long-term 
institutional  changes.  Interviewees  and  position  papers  submitted  to  the  public 
consultation  point  to  the  absence  of  a  monitoring  system.  Nevertheless,  annual 
compliance  checks  are  planned,  with  pilot  compliance  checks  currently  being 
implemented  on  a  limited  number  of  beneficiaries  of  the  2022  calls.  In  addition,  an 
upcoming study from the European Commission will provide first insights on the impact 
of the requiring GEPs as an eligibility criterion. 

•  Avoiding strict quotas. As mentioned previously, one of the European Commission’s 
objectives  is  to  improve  gender  balance  throughout  the  Framework  Programme  and 
promote gender balance among research teams. Despite it not being a target of Horizon 
Europe,  five  position  papers  submitted  to  the  OPC941  raised  concerns  about  the 
effectiveness of the hypothetical possibility of implementing a strict 50:50 gender balance 
among the researchers participating in a consortium. For example, among the concerns 
expressed, one referred to the risk of creating partly ‘artificial’ consortia in which some 
members are selected mainly to fulfil the 50:50 (sub)criterion rather than because of the 
need  for/complementarity  of  their  specific  expertise  within  the  consortium.  Another 
referred  to  how  the  ranking  criterion  would  be  used,  as  it  might  discourage  the 
participation of women when a strict 50:50 ratio is seen as optimal, leading to the risk of 
women  researchers  being  removed  from  the  consortia  in  disciplines  in  which  there is 
usually a higher representation of women (such as in SSH). 

•  The potential for the integration of the gender dimension into R&I has not yet been 
fully  realised.  The  information  provided  and  actions  taken  by  DG  RTD  have  helped 
applicants  to  comply  with  the  integration  of  the  gender  dimension  in  R&I  content. 
Nevertheless, a few of those interviewed for this case study believed there is a lack of 
clarity among beneficiaries on how to integrate the gender dimension in research and 

941 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 

552 

 
 
innovation,  particularly  in  those  scientific  fields  that  are  perceived  as  being  less 
connected  to  the  topic  of  gender,  such  as  natural  sciences  and  engineering  and 
technology. In some cases, such integration is confused with having a gender-balanced 
consortium  or  team.  Several  position  papers  submitted  to  the  OPC942,  as  well  as 
respondents (especially among the unsuccessful applicants) to the survey on Horizon 
Europe conducted May-July 2023, also expressed concerns about how this integration 
is assessed during the evaluation process. In particular, they highlighted a lack of clarity 
regarding the weight given to the gender dimension and the experience of evaluators on 
the topic.  

1.5.  Conclusions  

Based  on  desk  research,  the  analysis  of  monitoring  data,  representative  interviews  and 
survey data, this case study provides initial insights into the relevance, effectiveness and EU 
added  value  of  gender  equality  actions,  in  particular  under  Horizon  Europe,  following  the 
assessment of gender equality as a cross-cutting issue under Horizon 2020 carried out during 
the Evaluation study on Excellent Science under Horizon 2020.  

Gender balance under Horizon Europe has improved in comparison to Horizon 2020. 
The  inclusion  of  new  variables  in  the  data  collection  (such  as  data  on  researchers)  has 
improved  the  monitoring  of  female  participation  in  projects  and  helps  to  give  a  better 
indication of gender balance across research teams in the various programme parts. This 
opens up the possibility of targeted actions in programmes parts in which female participation 
is very low (without compromising research excellence). In addition, the self-declaration of 
gender  identity  by  researchers  under  three  categories:  woman,  man  and  non-binary  –  a 
novelty  under  Horizon  Europe  –  is  good  step  forward  towards  greater  inclusivity  in  the 
Framework Programme.  

Under Horizon Europe, the development and implementation of Gender Equality Plans 
has further advanced and shifted towards a focus on inclusive and sustainable GEPs. 
In  general,  the  requirement  to  have  a  GEP  is  perceived  positively  by  organisations  and 
beneficiaries participating in the FP (including Horizon Europe) and wider stakeholder groups 
(in the context of the OPC), inspiring discussions and actions at institutional level. However, 
there is still room for significant progress to be made in addressing the remaining disparities 
in the application of gender equality principles across countries and fields of science, and 
concerns  are  raised  regarding  the  implementation  and  effectiveness  of  GEPs  in  the  long 
term. Efforts already being developed to monitor the impact of GEPs at institutional level may 
provide  valuable  insights  regarding  their  effectiveness,  and  hence  the  relevance  and 
usefulness of this eligibility criterion. In addition, communication, exchange and cooperation, 
as well as sufficient resources, are mentioned as the most important factors for the successful 
implementation and impact of projects. The same factors were also identified in relation to 
the successful implementation of GEPs. 

Despite improvements and changes with regard to the integration of the gender dimension 
in  R&I  content  under  Horizon  Europe,  evidence  points  to  the  need  for  further 
improvements. Qualitative data from the interviews, as well as from the OPC and the survey 
of unsuccessful Horizon Europe applicants conducted in May-July 2023, indicate concerns 
about  the  effectiveness  of  the  integration  of  the  gender  dimension  in  R&I  content.  For 
example, there continues to be confusion among both applicants and beneficiaries between 

942 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 

553 

 
 
gender  balance  and  the  integration  of  the  gender  dimension  in  R&I.  Therefore,  more  and 
better  communication  –  including  clear  examples  of  best  practices  that  explain  the  link 
between excellent science and the integration of the gender dimension in R&I  – would not 
only help to increase awareness of its importance, but also substantially improve progress 
and  increase  potential  for  the  integration  of  the  gender  dimension  in  R&I.  In  addition  to 
general  communication  initiatives,  communication  and  exchange  on  the  topic  with  the 
National Contact Points (NCPs) could further help raise awareness, by supporting their direct 
communication towards applicants and beneficiaries. An effective monitoring system is also 
necessary to assess how integration is being carried out by the projects.  

In  general,  topics  that  go  beyond  gender  equality,  such  as  intersectionality  and 
diversity, are gaining importance, as demonstrated by references made in several position 
papers  submitted  to  the  OPC943,  as  well  as  during  interviews.  A  number  of  actions  have 
already been taken by the European Commission in this regard – for example, the possibility 
for  researchers  to  self-declare  their  gender  identity  under  three  categories  (woman,  man, 
non-binary), as well the inclusion of new topics under  WIDERA and Clusters 2 and 4 that 
reflect new avenues of research beyond gender equality – all of which confirm the importance 
attached to inclusive gender equality in the ERA Policy Agenda (which is already the focus 
of the WIDERA calls). However, there is still room to further reflect on how to better integrate 
these cross-cutting topics into the current or future Framework Programmes. 

Appendix 1. Literature sources 

•  Cordis webpages of selected projects. 

•  Council Decision (EU) 2021/764 of 10 May 2021 establishing the Specific Programme 

implementing Horizon Europe – the Framework Programme for Research and 
Innovation, and repealing Decision 2013/743/EU, available at: https://eur-
lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32021D0764 

•  European Commission, Communication from the Commission to the European 
Parliament, the Council, the European Economic and Social Committee and the 
Committee of the Regions A new ERA for Research and Innovation, 30 September 
2020, COM (2020) 628 final, available at: https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=COM%3A2020%3A628%3AFIN 

•  European Commission, Communication from the Commission to the European 
Parliament, the Council, the European Economic and Social Committee and the 
Committee of the Regions A New European Innovation Agenda, 5 July 2022, COM 
(2022) 332 final, available at: https://eur-lex.europa.eu/legal-
content/EN/TXT/PDF/?uri=CELEX:52022DC0332 

•  European Commission, Communication from the Commission to the European 
Parliament, the Council, the European Economic and Social Committee and the 
Committee of the Regions A Union of Equality: Gender Equality Strategy 2020-2025, 5 
March 2020, COM (2020) 152 final, available at: https://eur-lex.europa.eu/legal-
content/EN/ALL/?uri=COM:2020:0152:FIN  

943 Public consultation on the past, present and future of the EU’s Horizon research and 
innovation programmes 2014-2027. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-consultation-
future-eu-research-and-innovation-programmes-are-now-public-2023-04-19_en. 

554 

 
 
•  European Commission, Communication from the Commission to the European 
Parliament, the Council, the European Economic and Social Committee and the 
Committee of the Regions. A Reinforced European Research Area Partnership for 
Excellence and Growth. COM(2012) 392 final, available at: https://eur-
lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52012DC0392. 

•  European Commission, Directorate-General for Justice and Consumers, Strategic 

engagement for gender equality 2016-2019, Luxembourg: Publications Office of the 
European Union, 2016, available at: https://data.europa.eu/doi/10.2838/454429 

•  European Commission, Directorate-General for Research and Innovation, European 

Research Area policy agenda: overview of actions for the period 2022-2024, 
Luxembourg: Publications Office of the European Union, 2022, available 
at: https://data.europa.eu/doi/10.2777/52110 https://data.europa.eu/doi/10.2777/52110   

•  European Commission, Directorate-General for Research and Innovation, Horizon 
Europe guidance on gender equality plans, Luxembourg: Publications Office of the 
European Union, 2021, available at: https://data.europa.eu/doi/10.2777/876509 

•  European Commission, Directorate-General for Research and Innovation, Horizon 

Europe, gender equality – A strengthened commitment in Horizon Europe, 
Luxembourg: Publications Office of he European Union, 2021, available at: 
https://data.europa.eu/doi/10.2777/97891 

•  European Commission, Directorate-General for Justice and Consumers, Strategic 

engagement for gender equality 2016-2019, Luxembourg: Publications Office of the 
European Union, 2016, available at: https://data.europa.eu/doi/10.2838/454429 

•  European Commission, Directorate-General for Research and Innovation, Interim 

evaluation – Gender equality as a crosscutting issue in Horizon 2020, Luxembourg: 
Publications Office of the European Union, 2017, available at: 
https://data.europa.eu/doi/10.2777/054612 

•  European Commission, Directorate-General for Research and Innovation, Naujokaitytė, 
R., Mačiulienė, M., Stančiauskas, V.et al., Evaluation study on the implementation of 
cross-cutting issues in Horizon 2020 – Study in support of the ex-post evaluation of the 
European Framework Programme for Research and Innovation Horizon 2020, Dirham, 
S. (editor), Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/763665 

•  European Commission, Directorate-General for Research and Innovation, She figures 
2021 – Gender in research and innovation : statistics and indicators, Luxembourg: 
Publications Office of the European Union, 2021, available at: 
https://data.europa.eu/doi/10.2777/06090 

•  European Commission, Directorate-General for Research and Innovation, MORE4 – 
Support data collection and analysis concerning mobility patterns and career paths of 
researchers : survey on researchers outside Europe, Luxembourg: Publications Office 
of the European Union, 2021, available at: https://data.europa.eu/doi/10.2777/117072  

•  European Commission, Directorate-General for Research and Innovation, Synopsis 
report – Looking into the R&I future priorities 2025-2027, Luxembourg: Publications 

555 

 
Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/93927  

•  European Commission, Strategy for Equality between Women and Men 2010-2015, 

Luxembourg: Publications Office of the European Union, 2011, available at: 
https://data.europa.eu/doi/10.2767/71254. 

•  European Commission, Strategic engagement for gender equality 2016-2019, 
Luxembourg: Publications Office of the European Union, 2016, available at: 
https://data.europa.eu/doi/10.2838/454429 

•  European Council, Council Conclusions of 1 December 2015 on Advancing Gender 

Equality in the European Research Area, available at: 
https://data.consilium.europa.eu/doc/document/ST-14846-2015-INIT/en/pdf. 

•  European Council, Council Conclusions of 7 March 2011 on European Pact for Gender 

Equality (2011-2020), available at: https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=CELEX%3A52011XG0525%2801%29. 

•  European Council, Proposal for a Council Recommendation on a Pact for Research 

and Innovation in Europe. COM/2021/407 final, available at: https://eur-
lex.europa.eu/legal-content/EN/ALL/?uri=COM:2021:407:FIN. 

•  European Council, Council Conclusions of 26 November 2021 on Future governance of 

the European Research Area (ERA), available at: 
https://data.consilium.europa.eu/doc/document/ST-14308-2021-INIT/en/pdf 

•  European Parliament and the Council (2021), Regulation (EU) 2021/695 of the 

European Parliament and of the Council of 28 April 2021 establishing Horizon Europe – 
the Framework Programme for Research and Innovation, laying down its rules for 
participation and dissemination, and repealing Regulations (EU) No 1290/2013 and 
(EU) No 1291/2013, 28 April 2021, PE/12/2021/INIT, OJ L 170, 12.5.2021, pp. 1–68, 
available at: http://data.europa.eu/eli/reg/2021/695/oj    

•  European Union, Consolidated Version of the Treaty on the Functioning of the 

European Union, 26 October 2012, OJ L. 326/47-326/390; 26.10.2012, available at: 
http://data.europa.eu/eli/treaty/tfeu_2012/oj. 

•  Charter of Fundamental Rights of the European Union, available at: https://eur-

lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A12012P%2FTXT. 

• 

Ljubljana Declaration, available at: https://data.consilium.europa.eu/doc/document/ST-
12044-2021-INIT/en/pdf. 

•  UN Sustainable Development Goals, Goal 5, available at: 

https://sdgs.un.org/goals/goal5 

• 

• 

https://research-and-innovation.ec.europa.eu/funding/funding-opportunities/funding-
programmes-and-open-calls/horizon-europe_en. 

https://research-and-innovation.ec.europa.eu/strategy/strategy-2020-2024/democracy-
and-rights/gender-equality-research-and-innovation_en. 

556 

 
•  European Commission, Horizon 2020 administrative and monitoring data, 2022, 

Available at: 
https://www.europarl.europa.eu/RegData/docs_autres_institutions/commission_europe
enne/com/2022/0389/COM_COM(2022)0389_EN.pdf.  

•  Genderaction Project, information available at: https://genderaction.eu/eu-sister-

projects/ and https://h2020.genderaction.eu/resources/sister-projects/. 

•  BUDGET IT Project, Information available at: https://budget-it.eu/. 

•  Funding & tender opportunity - FAQ - Frequently Asked Questions. Available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/support/faq. 

•  European Research Council, Advanced analytics tool (ERIS). Available at: 
https://erc.europa.eu/projects-statistics/advanced-analytics-tool-eris. 

•  European Institute for Gender Equality, Gender budgeting. Available at: 

https://eige.europa.eu/gender-mainstreaming/tools-methods/gender-budgeting. 

•  Commission Staff Working Document Evaluation Accompanying the Document Report 

from the Commission to the European Parliament and the Council Ex-post evaluation 
of Horizon 2020, the EU Framework Programme for Research and Innovation. 
Available at: https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=SWD%3A2024%3A29%3AFIN&amp%3Bqid=1706528145182  

Appendix 2. Examples of gender-focused Horizon Europe projects signed 
under Clusters 2 and 4 

Projects 

Duration 
in 
months 

EU 
contribution  
(EUR) 

Number of 
participating 
organisations 

Cluster 2 – Topic ‘Feminisms for a new age of democracy’ 

CCINDLE 
Co-Creating Inclusive Intersectional 
Democratic Spaces Across Europe 

48 

2,746,211.25 

12 

FIERCE 
Feminist Movements Revitalizing Democracy 
in Europe 

36 

2,490,220 

PushBackLash 
Anti-Gender Backlash & Democratic Pushback 

36 

2.158,756 

11 

11 

RESIST 
Fostering Queer Feminist Intersectional 
Resistances against Transnational Anti-
Gender Politics 

48 

2,379,425.75 

10 

557 

 
 
Projects 

Duration 
in 
months 

EU 
contribution  
(EUR) 

Number of 
participating 
organisations 

UNTWIST 
UNTWIST: Policy recommendations to regain 
‘losers of feminism’ as mainstream voters 

36 

2,697,318.25 

9 

Cluster 2 – Topic ‘Gender and social, economic and cultural empowerment’ 

gEneSys 
Transforming Gendered Interrelations of 
Power and Inequalities for Just Energy 
Systems 

RE-WIRING 
Realising Girls’ and Women’s Inclusion, 
Representation and Empowerment 

36 

2,656,488.75 

36 

2,461,349.75 

8 

7 

ReIncluGen 
Rethinking Inclusion and Gender 
empowerment: a participatory action research 

36 

2,990,481.25 

12 

Cluster 4 – Topic ‘Tackling gender, race and other biases in AI (RIA)’ 

AEQUITAS 
Assessment And Engineering Of Equitable, 
Unbiased, Impartial And Trustworthy AI 
Systems 

BIAS 
Mitigating Diversity Biases of AI in the Labor 
Market 

FINDHR 
Fairness and Intersectional Non-Discrimination 
in Human Recommendation 

MAMMOth 
Multi-Attribute, Multimodal Bias Mitigation in AI 
Systems 

36 

3,493,992 

17 

48 

3,498,525.25 

9 

36 

3,341,637 

13 

36 

3,304,975 

13 

Table 7. Examples of gender-focused Horizon Europe projects signed under Clusters 2 and 4. Source: Horizon Dashboard 
and CORDIS data (accessed September 2023). 

558 

 
 
Appendix 3. Examples of gender-focused Horizon Europe projects signed 
under the MSCA and ERC 

Projects 

MSCA 

Duration 
in 
months 

EU contribution  
(EUR) 

UNGENDERED VALUES 
Disrupting gendered stereotypes and disclosing neglected 
value-laden motivations of academic entrepreneurship 

EuWoRD 
European Women and Religious Dissent: The Advent of 
Modernity and the Democratic Public Sphere 

24 

189,687.36 

24 

199,694.40 

CORESIST 
Resistance to anti-gender politics: The role of coalitions and 
affects 

24 

175,920.00 

EUTERPE 
European Literatures and Gender from a Transnational 
Perspective 

48 

2,316,333.60 

WomenACT-REPinHOST 
Latin American women activists’ practices of resistance to 
transnational repression in host societies 

24 

172,618.56 

ERC 

BILQIS 
Building Conceptual and Methodological Expertise for the Study 
of Gender, Agency and Authority in Islam 

60 

1,994,249.00 

F-WORD 
The world behind a word 

BeyondGenderBinary 
Challenging the gender binary: Empirically unravelling the 
limitation of the male-female categories 

54 

1,114,266.00 

66 

2,446,001.00 

TRACE 
Tracing Queer Citizenship over Time: Ageing, ageism and age-
related LGBTI+ politics in Europe 

60 

1,999,930.00 

INTERMAPS 
Mapping intersectionality: a conceptual and methodological 
model for the study of inequalities and discriminations 

60 

1,226,000.00 

Table 8. Examples of gender-focused Horizon Europe projects signed under the MSCA and ERC. Source: information 
provided by the European Commission and ERC ERIS tool, and CORDIS data (accessed September 2023). 

559 

 
Annex 2.13: International cooperation and association 

1.1. 

Introduction 

International  collaborative  research  and  researchers’  mobility  worldwide  are  fundamental 
principles  within  the  European  Union,  reinforcing  excellence  in  research  and  innovation 
(R&I),  strengthening  competitiveness,  addressing  global societal  challenges,  and  enabling 
the  construction  of  required  infrastructures.  The  objectives  in  the  EU  Framework 
Programmes (FPs) that pertain to international cooperation are implemented by opening the 
FPs  up  to  researchers  and  innovators  from  around  the  world,  and  by  supporting  targeted 
activities  based  on  priority  areas  for  cooperation  with  international  partner  countries  and 
regions.  The  Horizon  Europe  programme  is  the  most  open  and  by  far  the  largest  R&I 
programme  globally944.  The  present  case  study  has  been  implemented  through  extensive 
desk research, and the analysis of EC monitoring data, survey results and interviews with the 
key stakeholder that have allowed a more structured view of international collaboration and 
association under the Horizon Europe programme, with a particular focus on activities under 
Pillar 1, specifically the ERC, MSCA and INFRA, as well as activities in WIDERA and its 
two  programme  parts: 
‘Widening  Participation  and  Spreading  Excellence’  and 
‘Reforming and Enhancing the European R&I system’. The key added value of this case 
study  within  the  evaluation  overall  is  that  it  provides  a  concise  analysis  and  summary, 
including both qualitative and quantitative evidence, regarding the modalities and impacts of 
international  cooperation  under  the  Horizon  Europe  programme,  in  terms  of  spreading 
excellent science and enhancing the coherence of the European Research Area (ERA).  

1.2.  Background information 

1.2.1.  Policy context and the state of play 

The Horizon Europe FP pursues an ambitious general objective namely,  to strengthen the 
scientific and technological bases of the Union and to foster its competitiveness, including 
that of industry, to deliver on the EU’s strategic policy priorities, and to contribute to tackling 
global  challenges  including  the  UN  Sustainable  Development  Goals945.  Furthermore, 
Decision  (EU)  2021/764946  outlines  two  operational  objectives  for Horizon  Europe  that are 
tied  to  international  cooperation  and  association  (‘To  strengthen  international  cooperation’ 
and ‘Connecting to and developing research infrastructures across the European research 
area  (ERA)  and  providing  transnational  access’).  To  fulfil  these  general  and  operational 
objectives, the programme provides a number of international cooperation modalities. These 
range  from  higher-level  policy  tools  such  as  Association  agreements  and  other  bilateral 
scientific and technological (S&T) cooperation agreements, to more operational mechanisms 
such as outreach and support activities that focus on international programme participants, 
the flagging of topics and mentoring initiatives).  

944 European Commission, International Cooperation 2023.  
945 Regulation (EU) 2021/695. 
946 Council Decision (EU) 2021/764 of 10 May 2021 establishing the Specific Programme 
implementing Horizon Europe – the Framework Programme for Research and Innovation, and 
repealing Decision 2013/743/EU. 

560 

 
 
Throughout  this  case  study,  reference  is  made  to  the  following  key  groups  of  countries: 
Member  States  (MS),  associated  countries  (AC)  and  non-associated  third  countries  (TC). 
The  associated  countries  under  Horizon  Europe  comprise  Albania,  Armenia,  Bosnia  and 
Herzegovina, Faroe Islands, Georgia, Iceland, Israel, Kosovo, Moldova, Montenegro, New 
Zealand, North Macedonia, Norway, Serbia, Tunisia, Turkey, Ukraine, and the recently added 
United Kingdom947. The groupings of countries under the Horizon Europe programme have 
been  amended  significantly  in  comparison  to  its  predecessor,  Horizon  2020,  given  the 
changed status of major FP participants – most importantly, the United Kingdom (UK) and 
Switzerland (CH), two countries with a substantial history of participation in the Framework 
Programme). To simplify comparison between FPs, the tables and figures in this case study 
provide separate values for the UK and Switzerland in addition to the totals for MS, AC and 
TC. 

Table 1 below outlines the state of play with regard to international cooperation in key pillars 
of  the  Horizon  Europe  programme.  The  majority  of  projects  under  Horizon  Europe  are 
implemented by beneficiaries representing the Member States (84.5%). The overall share of 
participation by associated countries amounts to 5.3%, while third countries represent 3.4% 
of participations, Switzerland 2.3%, and the United Kingdom 4.4%. Compared with Horizon 
2020, the share of beneficiaries representing associated countries has increased and that of 
third  countries  has  decreased.  Significant  changes  can  be  seen  in  participations  from  the 
United  Kingdom,  with  its  share  having  decreased  significantly  between  Horizon  2020  and 
Horizon  Europe.  The  same  applies,  albeit  to  a  lesser  extent,  in  the  case  of  Switzerland. 
Nevertheless, it is important to note that these figures only represent participation patterns 
during 2021 and 2022. Given that the whole Framework programme runs until 2027, it is still 
too early to draw definitive conclusions.  

Total 
Horizon 
2020 

Total 
Horizon 
Europe 

Pillar 1 
Horizon 
2020 

Pillar 1 
Horizon 
Europe 

Pillar 2 
Horizon 
Europe 

Pillar 3 
Horizon 
Europe 

WIDER
A 
Horizon 
Europe 

191,568 

54,911 

56,868 

15,457 

34,904 

2,614 

1,936 

149,523 

46,408 

39,657 

12,599 

29,872 

2,334 

1,603 

78.1% 

84.5% 

69.7% 

81.5% 

85.6% 

89.3% 

82.8% 

9,207 

2,925 

2,539 

843 

1,751 

121 

210 

4.8% 

5.3% 

4.5% 

5.5% 

5.0% 

4.6% 

10.8% 

8,476 

1,891 

3,996 

979 

856 

18 

38 

Total 
participation 

Horizon 
Europe 
Member 
States 

Horizon 
Europe 
associated 
countries* 

947 Please note that, as of 1 January 2024, the UK is an associated country, given such a vry 
recent change in association status, this case study does not yet consider UK as an associated 
country. In addition, since late November 2023, the Commission is in exploratory talks with 
Switzerland on Switzerland's association to Horizon Europe and the Euratom R&T programme 
amongst other EU programmes. Association negotiations to follow.  

561 

 
 
 
Total 
Horizon 
2020 

Total 
Horizon 
Europe 

Pillar 1 
Horizon 
2020 

Pillar 1 
Horizon 
Europe 

Pillar 2 
Horizon 
Europe 

Pillar 3 
Horizon 
Europe 

WIDER
A 
Horizon 
Europe 

4.4% 

3.4% 

7.0% 

6.3% 

2.5% 

0.7% 

2.0% 

5,888 

1,250 

2,306 

329 

855 

55 

11 

3.1% 

2.3% 

4.1% 

2.1% 

2.4% 

2.1% 

0.6% 

18,474 

2,437 

8,370 

707 

1,570 

2,614 

74 

9.6% 

4.4% 

14.7% 

4.6% 

4.5% 

3.3% 

3.8% 

Third 
countries* 

Switzerland 

United 
Kingdom  

Table 1. Comparison of participation (shares) by different country groups under Horizon Europe Source: Horizon Dashboard, 
accessed 2023/10/22. *Numbers of associated countries for both periods refer to countries currently associated to the 
Horizon Europe programme. Third countries exclude Switzerland and the United Kingdom given their changed status (UK 
became an associated country in January 2024, Switzerland is expected soon to be one). Member States under Horizon 
2020 exclude the United Kingdom. 

In  terms  of  the  programme  parts  analysed  in  this  case  study,  Pillar  1  stands  out  for  its 
international dimension, with more than half (51.8%) of Pillar 1 project participants coming 
from non-associated third countries (excluding the UK and Switzerland). Compared with the 
Horizon 2020, similar trends can be observed across the whole FP, with significantly lower 
participation  rates  and  shares  of  participants  coming  from  the  United  Kingdom  and 
Switzerland.   Given that UK just recently changed its association status, as Switzerland is in 
the process of it, such decrease in participation is not surprising, participation numbers are 
expected to go up in the next years of HE. 

1.2.2.  Overall approach and scope of the case study 

This  case  study’s  analysis  of  international  cooperation  and  association  in  Pillar  1  and 
WIDERA actions under Horizon Europe relies on several interconnected methods: 

•  Desk research consisted of two stages: 1) a literature review mapping and synthesising 
evidence and findings from existing sources of information on international cooperation 
and Association in Framework Programmes, such as previous evaluations, studies, the 
results of public consultations, databases and similar material (listed in the footnotes and 
in Appendix 1); and 2) analysis of monitoring and administrative data on relevant Horizon 
Europe and Horizon 2020 projects, participants and applications (see Appendix 4).  

• 

Interviews. While desk research provides a good context and background information, 
the  study  team  implemented  13  interviews  with  stakeholders  who  possessed  a  good 
overview of cooperation between the various country groups. These interviews provided 
valuable information and evidence about the mainstreaming of international cooperation 
and association under the Framework Programme. More specifically it allowed the study 
team  to  extract  judgements  regarding  the  drivers  of  and  barriers  to  international 
cooperation and association, as well as its added value.  

562 

 
 
•  Surveys. Data from two surveys (the Survey of Horizon Europe beneficiaries, conducted 
May-July 2023, and the Survey of unsuccessful Horizon Europe applicants, conducted 
May-July  2023)  was  used  to  analyse  the  experience  of  international  programme 
participants  in  the  project  application  process  as  well  as  the  administrative  and 
management  processes  of  Horizon  Europe,  and  the  respective  added  value  of 
participation in the FP (see Appendixes 2 and 3).  

In summary, the present case study provides relevant information regarding the evaluation 
questions outlined in the table below.  

Dimension 

EQs 

Key contributions 

Relevance  

RV6 

Effectiveness  EFC5 
EFC6 
EFC14 

Efficiency 

EFF6 
EFF12 

This section of the report discusses the relevance of international 
cooperation activities in relation to delivering excellent science. More 
specifically, the case study analyses the transition of international 
cooperation from the Horizon 2020 programme to Horizon Europe; the 
implementation modalities used under the reference programmes and 
their relationship to the objectives of the FPs and to broader EU policy 
objectives including the reinforcement Europe’s relative position with 
regard to R&I.  

This section evaluates (to the extent that is currently possible) the 
activities, results and impacts of the international cooperation 
modalities, focusing on relevant actions under Pillar 1 and WIDERA. 
More specifically, the case study analyses how the FP reinforces the 
integration and coherence of the ERA and achieves the scientific 
objectives set for the Horizon Europe programme. Qualitative 
assessment through in-depth interviews also allows the assessment of 
internal and external factors that have contributed to or hindered 
progress. 

This section begins by focusing on the efficiency of outreach activities 
aimed at attracting international beneficiaries, explaining the benefits of 
international cooperation to potential programme participants and 
enabling the smooth participation of international partners in FP-funded 
projects. More specifically, the study team looked into the structure 
(strategies and tools) and implementation (the drivers and barriers, as 
perceived by the key targeted groups) of outreach and communication 
activities under Pillar 1 and WIDERA. In addition, this section provides 
an overview of the results of the surveys and public consultation in 
relation to the efficiency of the programme, as perceived by the 
international participants. 

EU added 
value 

EAV2 

The case study will determine the role of the Horizon Europe 
programme and, more specifically, of Pillar 1 and WIDERA, in 
strengthening international cooperation, including association.  

Table 2. Evaluation questions addressed by this case study. Source: defined by the study team, based on the Tender 
Specifications. 

563 

 
 
 
1.3.  Case study findings 

1.3.1.  Relevance  

Evaluation question 

RV6:  In  which  areas  is  the  participation  of  international  partners  and  associated  countries  most 
relevant  to  delivering  excellent  science?  How  does  this  participation  fit  into  the  objectives  of  the 
Framework Programme, including to reinforce Europe’s relative position in R&I? 

1.3.1.1. 

Relevance of international cooperation to the objectives of the EU’s 
external R&I policy and the Horizon Europe programme 

International research collaboration and the global mobility of researchers are foundational 
principles of the EU Framework Programmes. The previous FP, Horizon 2020, was led by 
a strategy for international cooperation in R&I set out by a Commission Communication948  in 
2012. This strategy guided the EU’s scientific and technological relations with third countries 
and  underpinned  the  international  reach  of  the  programme.  In  this  regard,  international 
cooperation was a cross-cutting priority of the Horizon 2020 programme. Its objectives were 
implemented by opening up the programme to researchers and innovators from around the 
world  and  supporting  targeted  activities  on  the  basis  of  priority  areas  for  cooperation  with 
international partner countries and regions. Horizon 2020’s political ambition was to maintain 
international cooperation activities at least at the level achieved under FP7. Despite an initial 
drop in participations in collaborative projects by  associated countries and non-associated 
third countries (due to discontinuation of dedicated international cooperation schemes that 
had existed under FP7, and changes to funding eligibility conditions for some countries), the 
programme caught up with the  objectives set for international cooperation  as a result of a 
number of corrective measures being taken (namely an increase in the numbers of topics 
that  were  specifically  relevant  for  international  cooperation,  an  improvement  in  framework 
conditions and a refinement of the communication strategy). Evidence and lessons learned 
from the Horizon 2020 Interim evaluation and the recommendations of the independent High-
Level Group on maximising the impact of EU R&I949 were used in building the Horizon Europe 
programme.  

The Global Approach to Research and Innovation950 adopted in May 2021 supersedes 
the previous strategy with the aim of better responding to a changing global context (including 
higher  spending  on  science  by  other  major  science  powers  compared  with  the  EU;  rising 
geopolitical  tensions  and  challenges  to  human  rights  and  fundamental  values  such  as 
academic freedom; the increasing willingness of certain countries to achieve technological 
leadership  through  discriminatory  measures;  and  the  instrumentalisation  of  R&I  for  global 

948 COM/2012/0497. Communication from the Commission to the European Parliament, the 
Council, the European Economic and Social Committee and the Committee of the Regions: 
Enhancing and focusing EU international cooperation in research and innovation: A strategic 
approach. 
949 COM/2018/02, Communication from the Commission to the European Parliament, the 
Council, the European Economic and Social Committee and the Committee of the Regions: 
Horizon 2020 interim evaluation: maximising the impact of EU research and innovation.  
950 COM/2021/252. Communication from the Commission to the European Parliament, the 
Council, the European Economic and Social Committee and the Committee of the Regions on 
the Global Approach to Research and Innovation Europe's strategy for international cooperation 
in a changing world. 

564 

 
 
influence  and  social  control).  The  new  approach  also  aims  to  align  the  EU’s  international 
cooperation with its current priorities. The new strategy focuses on the EU taking a leading 
role  in  supporting  international  R&I  partnerships,  and  on  delivering  innovative  solutions  to 
make  our  societies  green,  digital  and  healthy.  The  Global  Approach  sets  out  a  series  of 
concrete actions aimed at contributing to delivering these objectives. These are: 1) promoting 
common values and principles for international cooperation in R&I with international partners, 
in  areas  such  as  academic  freedom,  research  ethics  and  gender  equality;  2)  developing 
guidelines  for  dealing  with  foreign  interference  that  targets  European  universities  and 
research  organisations;  3) jointly  developing  targeted  bilateral  roadmaps  with  key  non-EU 
country partners with a strong research and innovation base, to achieve a level playing field 
and reciprocity in R&I cooperation; and 4) developing initiatives to support researchers and 
their organisations to accelerate sustainable and inclusive development in low and middle-
income countries951. The adoption of this approach reaffirms the importance of international 
cooperation in R&I. 

Horizon Europe is a key instrument for the implementation of this strategy. Reaffirming the 
EU's commitment to global openness, Horizon Europe, like its predecessor, remains open to 
the participation of researchers and innovators from all over the world. However, important 
changes  have  been  introduced  into  the  Horizon  Europe  programme,  in  the  field  of 
international  cooperation.  These  include  a  broadening  of  the  Association  policy,  with  the 
possibility of partial association and reciprocal access to programmes (discussed in Section 
1.3.2). Given the new openness of the Association policy, and due to the fact that the previous 
static  GDP-based  financial  contribution  model  led  to  many  countries  becoming  net 
contributors  to  the  programme  budget,  and  a  few  becoming  net-beneficiaries,  a  need  to 
adjust the financial contribution model was identified. This involves an automatic correction 
mechanism to avoid large imbalances between a given country’s financial contribution and 
the amount which entities from that country receive back from the programme. In addition, 
the  effects  of  the  COVID-19  pandemic,  coupled  with  geopolitical  tensions  and  complex 
societal  challenges,  have  necessitated  a  shift  in  the  EU’s  strategic  documents.  This  shift 
reinforces the concept of the strategic autonomy of the Union, whereby certain calls under 
the  FP  are  limited  to  or  targeted  towards  specific  countries  that  can  help  to  strengthen 
competitiveness. More specifically, Article 22(5) of the Horizon Europe Regulation explicitly 
allows  the  EU  to limit  the  access  of  third countries, including  associated  countries,  and  to 
potentially exclude them from specific call topics of the Horizon Europe programme in which 
the EU’s strategic or security interests are at stake. This means that the work programmes 
for certain calls under Pillar 2 can limit the access of legal entities from non-EU states. In 
addition,  Article  22(6)  of  the  Horizon  Europe  Regulation  has  been  applied  to  prevent  the 
participation of Chinese entities in innovation actions. It is important to note that in contrast 
to  Horizon  2020,  the  current  FP  does  not  have  a  target  set  for  the  level  of  international 
participation.  Looking  to  the  future  of  the  Framework  Programme,  the  ‘Horizon  Europe 
Strategic  Plan  2025–2027  Analysis’952  reports  that  R&I  is  at  the  forefront  of  the  EU’s 
preparedness, resilience, security and crisis response, and that R&I activities have a specific 
capacity 
the  societal,  economic  and 
environmental challenges facing Europe and the world. The results of the ‘Public consultation 
on  the  past,  present  and  future  of  the  European  R&I  FP  2014-2027’953  reveals  that  the 
majority of respondents (70%, or 1,367 respondents) highlight that improvements are needed 
with regard to international cooperation. A review of the position papers submitted, together 

innovative,  sustainable  solutions  to 

to  offer 

951 For example, the ‘Africa initiative' under Horizon Europe to strengthen cooperation with 
African countries. 
952 Horizon Europe Strategic plan 2025-2027 Analysis. 
953 Public consultation on the past, present and future of the European R&I FPs 2014-2027. 

565 

 
 
with responses to the consultation, is provided in Section 1.3.3.2. This confirms the continuing 
relevance of international cooperation activities to key stakeholders in the programmes.  

1.3.1.2. 

Fit between international participation and the objectives of the FP 

The previous section established the overall relevance of international cooperation and 
association under Horizon Europe. At the same time, it is necessary to assess in what 
ways  international  participation  is  most  relevant  to  delivering  excellent  science  and 
reinforcing  Europe’s  relative  position  in  R&I.  To  address  this,  the  next  subsection 
outlines  the  links  between  objectives  of  the  programme  parts  under  analysis,  the 
modalities  used  for  international  cooperation,  as  well  as  trends  in  international 
participation over time. 

The ERC and international cooperation 

The  fundamental  activity  of  the  European  Research  Council  (ERC)  is  to  provide 
attractive, long-term funding to support excellent investigators and their research teams 
to pursue ground-breaking, high-risk/high-gain research. There has been a high degree 
of continuity in the objectives of the ERC across Framework Programmes since 2007. 
The Scientific Council of the ERC has developed a Monitoring and Evaluation Strategy954 
that allows the ERC to continuously assess progress towards its four specific objectives, 
all of which have a strong international dimension.  

Based on the ERC’s Work Programme955, it can be expected that the majority of ERC-
funded research activities will be implemented within the territory of the Member States 
or associated countries. However, the ERC also provides several modalities that allow 
the  world’s  best  scientists  to  be  attracted  to  Europe.  Specifically,  researchers  of  all 
nationalities from non-associated third countries can apply for ERC funding, as long as 
they find a host in a Member State or associated country. While most ERC grants fund 
individual  researchers,  ERC  Synergy  Grants  bring  together  between  two  and  four 
principal investigators (PIs). The ERC Scientific Council decided to open this scheme up 
to researchers hosted in institutions in non-associated third countries956. Thanks to its 
production  of  excellent  science,  its  focus  on  frontier  research  and  its  broad  visibility 
across and beyond the EU, the ERC is highly attractive globally to both associated and 
third countries.  

Table 3 shows the breakdown of participation shares by country group. In line with the 
overall trend across Horizon Europe, most participations come from non-widening Member 
States.  The  vast  majority  (96.7%)  of  participations  in  the  ERC by  non-EU Member  States 
came from just eight countries: Israel (201 participations), Norway (70), the United Kingdom 
(34),  Turkey  (17), Switzerland  (13), the  United  States  (12), Iceland  (4) and  Canada (4). A 
detailed breakdown per country group and action type is provided in Table 38 of Appendix 4. 
The  top  fields  in  terms  of  participation  by  associated  as  well  as  non-associated  third 
countries are genomics, scientific computing and data processing, and systems biology. 

954 European Research Council, ERC Monitoring and Evaluation Strategy, 2018. 
955 ERC Work Programme 2022. 
956 Provided that they are not covered by Council sanctions, as set out in the ERC Work 
Programme. 

566 

 
 
Horizon 
Europe 
(all pillars) 

ERC 

StG 

CoG 

AdG 

SyG 

PoC 

Member States 

84.5% 

87.0% 

88.6% 

87.9% 

89.9% 

68.9% 

85.2% 

Associated 
countries 

5.3% 

10.5% 

10.5% 

6.2% 

8.4% 

9.6% 

13.9% 

Third countries 

3.4% 

0.8% 

0.5% 

1.6% 

0.2% 

8.9% 

0.2% 

Switzerland 

2.3% 

0.5% 

0.1% 

1.0% 

0.4% 

3.0% 

0.4% 

United Kingdom  

4.4% 

1.2% 

0.3% 

3.3% 

1.1% 

9.6% 

0.2% 

Participations 

54,911 

2,824 

991 

306 

526 

135 

460 

Table 3. Share of participations per action type and country group in ERC. Source: Horizon Dashboard, accessed 21 
October 2023. Note: Third countries excludes Switzerland and the United Kingdom given their status in Horizon Europe has 
recently changed, UK became an associated country on January 1, 2024 (this table was last updated in September 2023). 

As shown in the report specifically on the ERC carried out as part of this study957, the ERC 
contributes substantially to the production of scientific excellence in the EU. This is reflected 
in its publication track records, the scientific quality of the research produced, the many prizes 
and  awards  received  by  ERC  grantees,  and  the  ground-breaking  nature  of  ERC research 
projects. As a result, the ERC is able to attract excellent researchers from abroad to the EU. 
Specifically, under the previous FP, Horizon 2020, around 18% of ERC team members were 
non-ERA  nationals coming  from  almost  90 countries  worldwide958.  The  four most  frequent 
nationalities  among  non-ERA  ERC  team  members  under  Horizon  2020  were  researchers 
from India (18% of non-ERA nationalities), China (17%), the US (12%) and Russia (7%)959. 
The  ERC  Activity  Report  2020  also  indicates  that  the  largest  share  of  non-ERA  team 
members  was  found  in  the  category  of  postdoctoral  researchers.  Horizon  Europe 
administrative and monitoring data received by the study team shows that 66 nationalities 
are represented  among  the  total  of  2,364  PIs  and  corresponding  PIs  receiving  main  ERC 
grants. Of these, 27% are non-EU nationals, with the most common nationalities being those 
of the UK and Israel (each accounting for 24% of non-EU nationals) and the United States 
(12% of non-EU nationals). 

Another international aspect of the ERC that contributes to the excellence of science in the 
EU is the participation of international experts in the evaluation panels (17-18% of evaluators, 
according to the EC officials interviewed). The interviews conducted for this study suggest 
that  the  presence  of  international  know-how  and  perspectives  positively  influences  the 

957 Annex 1: Analysis and findings per programme part.  
958 European Research Council (2021). Annual report on the ERC activities and achievements in 
2020, available at: https://op.europa.eu/en/publication-detail/-/publication/09a3dcb6-9ccb-11eb-
b85c-01aa75ed71a1/language-en/format-PDF/source-283364380. 
959 It is important to note that these are participants’ nationalities, not their countries of residence 
or the countries of the organisations to which they are affiliated.  

567 

 
 
 
outcomes of evaluations and ensures that the proposals selected are relevant from a global 
perspective. 

Survey findings confirm the relevance of international cooperation to the ERC, as a large 
majority of the ERC beneficiaries believe that the ERC provides opportunities to increase 
the international visibility/outreach of research (90.8%, 218 out of 240) and international 
mobility  opportunities  (60.6%,  or  146  out  of  241  respondents).  However,  the  survey 
indicates that there is still  a need for more funding for collaboration with  international 
partners (non-EU countries): only 33% of ERC beneficiaries (80 out of 239) agreed to a 
‘large’ or ‘very large’ extent that there is sufficient funding for international collaboration 
(this  result  is  considerably  lower  than  that  seen  for  the  other  programme  parts  being 
analysed, as outlined in the sections that follow).  

MSCA and international cooperation 

The Marie Skłodowska-Curie Actions (MSCA) play a key role in terms of being open to the 
world, building international collaboration and attracting and retaining researchers in Europe 
through dedicated actions.  The programme is bottom-up and open to the entire world, i.e. 
there are no limitations on the researchers who can participate in terms of nationality960. The 
MSCA also feature a strong international dimension, as they: 1) allow institutions to set up 
strategic  collaborations  worldwide;  2)  attract  foreign  talents  to  Europe;  and  3)  provides 
European  researchers  with  access  to  unique  expertise,  facilities,  testing  environments  or 
data that are only available outside Europe961. 

In  this  regard,  all  MSCA  actions  support  international  cooperation  –  i.e.  MSCA  Doctoral 
Networks, MSCA Postdoctoral Fellowships (including Global Postdoctoral Fellowships, which 
fund  the  mobility  of  researchers  outside  Europe),  MSCA  Staff  Exchanges  and  MSCA 
COFUND962. As a result, the MSCA account for almost half of all non-associated third country 
participations in Horizon Europe (third countries including the UK and Switzerland, 48.7%; 
third countries excluding UK and Switzerland, 31%). Table 4 below provides an overview of 
international participation in the MSCA under Horizon Europe. Detailed participation rates per 
type of action and country group are provided in Table 47 of Appendix 4 of this case study.  

960 There are some limitations and specific requirements in relation to the participation of 
organisations/legal entities.  
961 MSCA Work Programme 2023-2024. 
962 European Commission Decision C(2021)4200 of 15 June 2021. 

568 

 
 
Horizon 
Europe 
(all pillars) 

MSCA 
(all 
actions) 

COFUND 

Post-
doctoral 
Fellow-
ships 

Doctoral 
Networks 

Staff 
Exchanges 

84.5% 

80.1% 

83.9% 

82.2% 

82.2% 

60.1% 

5.3% 

4.1% 

5.1% 

4.4% 

3.1% 

6.2% 

3.4% 

8.5% 

6.4% 

9.2% 

4.2% 

27.2% 

Member 
States 

Associated 
countries 

Third 
countries 

Switzerland 

2.3% 

2.4% 

1.5% 

2.0% 

3.1% 

1.4% 

United 
Kingdom 

Total 
participations 

4.4% 

5.0% 

3.1% 

2.3% 

7.4% 

5.1% 

54,911 

10,846 

955 

3,841 

4,914 

1,073 

Table 4. Share of participations in MSCA per action type and country group. Source: Horizon Dashboard, accessed 21 
October 2023. Note: Third countries excludes Switzerland and United Kingdom, given their changed status. 

As shown in Table 4 above, the participation of associated countries under different types of 
MSCA actions ranges between 3% and 6%, similar to the average for the whole of Horizon 
Europe of 5%. The area in which the MSCA stand out is in participation by non-associated 
third countries – especially with regard to Staff Exchanges, where total participation of non-
associated  third  countries  amounts  to  27.2%.  Analysis  of  participation  patterns  in  Staff 
Exchanges  (Table  51,  Appendix  4)  show  that  this  action  is  an  entry  point  for  many  non-
associated third countries and regions to participate in Horizon Europe (e.g. in the cases of 
Mexico and Argentina, more than half of participations in Horizon Europe programme stem 
from  Staff  Exchanges).  The  high  rates  of  international  cooperation  with  third  countries 
suggest  that,  given  their  collaborative  nature,  these  actions  allow  institutions  to  set  up 
strategic collaborations worldwide. 

In terms of broader EU policy objectives, the Study on international cooperation in the Marie 
Skłodowska-Curie actions963 concluded that the programme should remain bottom-up, but at 
the same time efforts should be made to better align it with external EU policy objectives, 
given its strong international dimension and the growing role of science diplomacy in foreign 
policies  and  strategies.  To  address  some  of  these  challenges,  the  MSCAdvocacy  project 
aims  to  map  and  provide  insights  (e.g.  identifying  key  challenges,  the  specificities  of  R&I 
ecosystems,  cultural  issues  relating  to  mobility)  on  the  cooperation  achieved  with  key 
international  partners  until  now  (i.e.  through  bilateral  agreements  and  existing  policy 
dialogues), according to the EC officials interviewed. Such a strategic outlook is expected to 
provide  a  smarter way  to  select  and  cooperate  with  partners  in  MSCA  projects,  based  on 
common  values  and  strategic  mutual  interests.  In  addition,  a  number  of  MSCA  support 
actions are designed to promote and support the implementation of the MSCA with regard to 
international  cooperation  (e.g.  facilitating  cooperation  between  MSCA  National  Contact 

963 Study on international cooperation in the Marie Skłodowska-Curie actions, 2019. 

569 

 
 
 
Points964; promoting the MSCA at an international level965; using ERA fellowships to foster 
the participation of Widening Countries, which also include several associated countries).  

The survey findings confirm the relevance of international cooperation to the MSCA: 60% of 
those  MSCA  beneficiaries  surveyed  who  represented  organisations  (272  out  of  456) 
considered collaboration with international (non-EU) partners to be important to a ‘large’ or 
‘very large’ extent to the uptake of their research findings/innovations. In addition, individual 
beneficiaries of the MSCA PF (40%, 258 out of 488 respondents), as well as organisations 
in  MSCA  collaborative  projects  (50.1%,  372  out  of  824)  agreed  to  a  ‘large’  or  ‘very  large’ 
extent that there is sufficient funding for international collaboration. 

Research infrastructures and international cooperation 

Research infrastructures (Ris) are facilities, resources and services that are used by research 
communities to conduct research and foster innovation in their fields966. The Member States 
play a pivotal role in both the development and financing of pertinent research infrastructures. 
Meanwhile, the EU contributes significantly to this ecosystem by promoting the establishment 
of novel facilities, facilitating widespread entry to national and European infrastructures, and 
ensuring coherence and efficacy across regional, national, European and global policies with 
regard to research infrastructures. Coordinated actions not only allow to the duplication of 
efforts to be avoided, but also enable necessary resources to be pooled so that the Union 
can  acquire  and  operate  RIs  at  world  level.  Under  the  previous  Framework  Programme, 
INFRA  was  successful  in  establishing  links  with  relevant  communities  worldwide.  These 
included global networks as well as connections with leading research communities in third 
countries.  According  to  the  findings  of  Phase  1  of  this  study,  the  INFRASUPP  calls  in 
particular  boosted  international  cooperation,  although  projects  such  as  OPEN  SESAME 
contributed to the optimal exploitation of SESAME in the Middle East, especially through the 
training of staff and building of human capacity.  

In relation to RI performance under the current Framework Programme, Table 5 provides an 
overview of participations in INFRA actions compared with the Horizon Europe programme 
as  a  whole.  Compared  with  Horizon  2020,  the  shares  of  participation  across  the  country 
groups  (when  adjusted  to  account  for  the  changed  status  of  Switzerland  and  the  United 
Kingdom) do not show any major fluctuations. Table 52 in Appendix 4 provides participation 
rates  per  associated  country  and  non-associated  third  country.  In  this  regard,  the  non-
associated third countries with the highest shares of participation include Switzerland (7.5%), 
the United States (3.2%) and South Africa (0.4%). In terms of associated countries, Norway 
(3.7%), Israel (0.4%) and Serbia (0.4%) lead the shares of participation. 

964 Call – HORIZON-MSCA-2021-NCP-01-01 – Trans-national cooperation among Marie 
Skłodowska-Curie National Contact Points (NCP). 
965 Call – HORIZON-MSCA-2021-INCO-01 – MSCA International Cooperation 2021. 
966 The 2015 Expert Advisory Group Implementation Report. 

570 

 
 
Research Infrastructures 

Horizon Europe (all pillars) 

Total 

1,787 

100% 

54,911 

100% 

Member States 

1,456 

81.5% 

46,408 

84.5% 

Associated countries 

105 

5.9% 

2,925 

5.3% 

Third countries 

Switzerland 

35 

57 

2.0% 

1,891 

3.4% 

3.2% 

1,250 

2.3% 

United Kingdom 

134 

7.5% 

2,437 

4.4% 

Table 5. Numbers and shares of participations per action type and country group. Source: Horizon Dashboard, accessed 21 
October 2023. Note: Third countries in this figure includes the UK and Switzerland. 

The  Horizon  Europe  INFRA  work  programmes  include  specific  calls  for  international 
collaboration  in  the  field  of  infrastructures,  most  notably  those  targeting  cooperation  on 
climate  change  in  Africa  (INFRA-2021-DEV-01-02)  or  bilateral  cooperation  with  Latin 
America (INFRA-2023-DEV-01-06). Interviews confirmed the relevance of this type of call, 
but questioned the long-term sustainability of such cooperation beyond the lifetimes of the 
projects  due  to  limited  availability  of  funds  in  the  partner  countries  to  continue  the 
cooperation.  Some  interviewees  indicated  that  a  longer-term  strategy  for  international 
cooperation  could  help  to  boost  its  effectiveness.  Furthermore,  one  key  policy  instrument 
boosting international cooperation is the ERIC Regulation. European Research Infrastructure 
Consortium (ERIC) has the potential to develop globally integrated RI activities by facilitating 
third country membership in Ris. However, this mechanism needs to be further strengthened, 
as indicated by the third report on the ERIC Regulation: ‘ERICs have the potential to become 
the legal instrument of choice for further developing globally integrated RI activities, provided 
that the participation of third countries in the ERICs is properly facilitated, to overcome the 
current legal and operational obstacles’967. A notable example of good practice is that of JIV-
ERIC968, which facilitates the observation of black holes by integrating together a set of radio 
astronomical observatories across Africa, Europe and Asia.  

Survey findings confirm the relevance of international cooperation to INFRA: 59.1% of the 
INFRA beneficiaries surveyed (88 out of 149) found that collaboration with international (non-
EU) partners is important to a ‘large’ or ‘very large’ extent for the uptake of their research 
findings/innovations. On the other hand, the survey also indicates that there is still a need for 
further funding for collaboration with international partners (non-EU countries): only 25.0% of 
INFRA  beneficiaries  (41  out  of  164)  agreed  to  a  ‘large’  or  ‘very  large’  extent  that  there  is 
sufficient funding for international collaboration.  

967 Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=COM:2023:488:FIN. 
968 JVI stands for Joint Institute for Very Long Baseline Interferometry. For more information, 
see: https://jive.eu/jiv-eric.  

571 

 
 
 
 
WIDERA and international cooperation 

According to the WIDERA Work Programme 2021-2022969, less advanced countries are eligible 
as hosts for the coordinator of Widening actions under this programme part. Such countries include 
the  Widening  Member  States,  but  also  all  associated  countries  that  share  equivalent 
characteristics  in  terms  of  R&I  performance,  i.e.  Albania,  Bosnia  and  Herzegovina,  Kosovo, 
Montenegro, North Macedonia, Serbia, Turkey, Armenia, Georgia, Moldova, Tunisia, Ukraine and 
the Faroe Islands. In exceptional cases, legal entities established in non-associated third countries 
may participate in Coordination and Support Actions970. The delivery of the programme can only 
be  optimised by  unlocking  the  potential  of  all partners. Most  notably, instruments  aimed at  the 
inclusion of Widening countries (including associated countries) encompass Teaming initiatives. 
These build capacity towards the European Excellence Initiative (through the aim of significantly 
strengthening cooperation between at least two organisations from the higher education sector in 
a  Widening  country  by  linking  them  to  at  least  two  internationally  leading  institutions).  and  the 
available instruments also include ERA fellowships (which enable excellent researchers who could 
not be  funded under  the MSCA  to undertake a  fellowship in  a  Widening  country).  International 
outreach  by  the  EURAXESS  initiative  links  the  EU  with  the  rest  of  the  world  and  promotes 
international collaboration, mobility and career development for researchers971.  

Table 6 below provides an overview of the state of play in WIDERA actions in terms of participation 
among  different  country  groups.  As  expected,  based  on  the  objectives  of  the  programme,  the 
participation of associated countries is higher than the Horizon Europe programme average. Table 
61 in Appendix 4 shows that the associated countries with the highest shares of participation are 
Serbia (2.5%), Turkey (2.3%) and Norway (1.7%). 

Horizon Europe 
(all pillars) 

WIDERA  
(all actions) 

Widening 
participation 
and spreading 
excellence 

Strengthening the 
ERA 

Total 

54,911 

100% 

1,936 

100% 

1,231 

100% 

705 

100% 

Member 
States 

Associated 
countries 

Third 
countries 

46,408 

84.5% 

1,603 

82.8% 

1,020 

82.9% 

583 

82.7% 

2,925 

5.3% 

210 

10.8% 

146 

11.9% 

64 

9.1% 

1,891 

3.4% 

38 

2.0% 

22 

1.8% 

16 

2.3% 

Switzerland 

 1,250 

2.3% 

11 

0.6% 

6 

0.5% 

5 

0.7% 

United 
Kingdom 

2,437 

4.4% 

74 

3.8% 

37 

3.0% 

37 

5.2% 

Table 6. Numbers and shares of participations in WIDERA per action type and country group. Source: Horizon Dashboard, 
accessed 21 October 2023. Note: Third countries in this Figure includes the UK and Switzerland. 

969 Horizon Europe Work Programme 2021-2022: Widening participation and strengthening the 
European Research Area. 
970 e.g. HORIZON-WIDERA-2021-ERA-01-09: Support for policy makers – Programme-level 
collaboration between national R&I programmes. 
971 WIDERA Work Programme 2023-2024. 

572 

 
 
 
Survey findings confirm the relevance of international cooperation to WIDERA: 56.6% of 
WIDERA  (Widening)  beneficiaries  (94  out  of  166)  and  46.9%  of  WIDERA  (ERA) 
beneficiaries (53 out of 96) who responded to the survey found that collaboration with 
international  (non-EU)  partners  is  important  to  a  ‘large’  or  ‘very  large’  extent  for  the 
uptake  of  their  research  findings/innovations.  In  addition,  the  survey  indicates  that 
WIDERA  beneficiaries  are  satisfied  with  the  funding  available  for  collaboration  with 
international partners, with 45.8% (80 out of 175) of WIDERA (Widening) beneficiaries 
and 46.1% of WIDERA (ERA) beneficiaries (52 out of 102) agreeing to a ‘large’ or ‘very 
large’ extent that there is sufficient funding for international collaboration.  

1.3.2.  Effectiveness 

Evaluation questions 

EFC5: How effective is the association of third countries to the Framework Programme in 
reinforcing the integration and coherence of the European R&I system with those of the 
associated countries, in terms of performance and of the convergence of values and principles? 

EFC6: To what extent has international cooperation and, more specifically, the association of third 
countries to the EU Framework Programme made a difference in achieving the scientific 
objectives of the Framework Programme? Has international cooperation, and specifically 
association, improved the EU’s scientific performance? 

EFC14:  What  is  the  impact  of  Framework  Programme  actions  supporting  Open  Science,  citizen 
science, gender equality, international cooperation, ethics and integrity, and scientific input into other 
EU policies? 

The present case study analyses the effectiveness of the various modalities for international 
cooperation offered by the FP (the most important among them being association), in terms 
of: 1) reinforcing the integration and coherence of the European R&I system with those of the 
associated countries, and 2) making a difference in achieving the scientific objectives of the 
programme. While most of the available international cooperation modalities (an example of 
which could be the Policy Support Facility) are relevant in increasing international cooperation 
in  all  Horizon  Europe  programme  parts,  this  case  study  focuses  on  those  provisions  and 
aspects that are specific to Pillar 1 and WIDERA (as determined by the outcomes of interview 
programme  with  EC  stakeholders).  Hence,  it  is  important  to  note  that  other  actions  are 
available that support international cooperation; however, these have a less direct influence 
on the programme parts under analysis. 

1.3.2.1. 

Effectiveness of international cooperation in reinforcing the integration and 
coherence of the European R&I system with the R&I systems of 
associated countries 

The role played by the association of third countries in the integration and coherence 
of the European R&I system with the R&I systems of associated countries 

To  evaluate  the  role  of  the  policy  of  association  in  relation  to  the  integration  and 
coherence of the European R&I system with the R&I systems of associated countries, 
we  first  need  to  understand  and  define  the  differences  between  groups  of  Horizon 
Europe associated countries. In general, when a third country becomes associated to 
the Framework Programme, legal entities from that associated country can participate 

573 

 
under conditions equivalent to those that apply to legal entities from EU Member States. 
This  form  of  cooperation  allows  non-EU  associated  countries  to  take  part  in  various 
Horizon Europe programme actions on similar terms to Member States972. In most cases, 
the participation of the country in the Framework Programme is based on an Association 
Agreement that defines the rules for participation (e.g. participation as observers in the 
governance  structures  of  the  programme,  financial  contributions).  Each  agreement 
establishes a Joint Research and Innovation Committee composed of members from the 
EU  and  from  the  non-EU  country  concerned.  In  the  cases  of  Norway  and  Iceland, 
association is provided through the European Economic Area (EEA) agreement and its 
protocol on  their participation in Union programmes.  Article 16 of the Horizon Europe 
Regulation defines four groups of associated countries (see Table 7). These groupings 
of associated countries are based on their institutional links to the EU, and the objectives 
of the Framework Programme, as highlighted in the sections that follow.  

Associated country group 

Horizon Europe associated countries 

Countries of the European Free Trade 
Association (EFTA) that are members of 
the European Economic Area (EEA) and 
share EU values and make important 
contributions to the European R&I 
landscape 

Norway, Iceland 

Acceding Countries, Candidate Countries and 
potential Candidate Countries 

EU Candidate countries: Albania, Bosnia and 
Herzegovina, Moldova, Montenegro, North 
Macedonia, Serbia, Turkey, Ukraine. Potential 
candidate countries: Georgia, Kosovo973. 

European Neighbourhood Policy countries 
(Eastern Partnership – Southern 
Neighbourhood) 

Eastern Partnership: Armenia, Georgia, the 
Republic of Moldova and Ukraine. 
Southern Neighbourhood: Israel, Tunisia. 

Other third countries and territories that fulfil a 
set of criteria related to their economic and 
political system and their R&I system974 

Faroe Islands 
New Zealand (partial association to Pillar 2) 
United Kingdom (as of 1 January 2024) 

Table 7. Key groups of associated countries, as defined in the Horizon Europe Regulation. Source: Horizon Europe 
Regulation, Article 16. 

Analysis of EC monitoring data, as well as an overview of previous evaluation studies 
and interviews conducted for the present case study show that the association of third 
countries to the Horizon Europe programme has bolstered the integration and coherence 
of the European R&I system, but the groups of associated countries have varying levels 

972 The only limitation to participation in the programme is linked to the protection of the EU’s 
strategic assets, interests, autonomy and security, as defined by Article 22(5) of the Regulation. 
973 This designation is without prejudice to positions regarding status, and is in line with UNSCR 
1244/1999 and the ICJ Opinion on the Kosovo declaration of independence. 
974 Most notably: having a strong science, technology and innovation capacity; a rules-based 
open market economy, including the protection of IPR; respect for human rights and democratic 
institutions; and policies for the economic and social well-being of citizens. 

574 

 
 
of integration with the European R&I ecosystem. The following key insights led to this 
conclusion.  

First,  associated  status  has  supported  the  development  of  R&I  capacities  in 
countries that have less advanced R&I ecosystems and has ‘exported’ the shared 
values and principles of the EU. In this regard, for Enlargement Countries (Acceding 
Countries,  Candidate  Countries  and  potential  Candidate  Countries)975,  association 
provides  an  important  opportunity  to  prepare  for  compliance  with  the  EU  acquis  on 
science, research and innovation, and to build R&I capacity at national level. This means 
the EU can ensure in advance that potential Member States are ready to be integrated 
into  the  European  Research  Area  (ERA).  In  addition,  the  ‘Evaluation  study  on  the 
external coherence and synergies of Horizon 2020 within the European research and 
innovation  support  system’976  concluded  that  the  experiences  of  Serbia  and  Moldova 
show  that  FP  instruments  made  a  difference,  with  direct  implications  for  capacity 
building,  researchers’  careers  and  to  some  extent,  the  countries’  R&I  regulatory 
frameworks, due to their governments’ desire to receive EU funding.  

Although almost all programme actions are open to the participation of associated countries 
that  are  fully  associated  to  the  programme977,  in  the  case  of  the  programme  parts  under 
analysis, there are a few instruments that are especially attractive to AC countries with 
low  R&I  capacities.  Most  notably,  these  includes  MSCA  Staff  Exchanges,  in  which  ACs 
account for 6.2% of participations (higher than the average for MSCA or Horizon Europe as 
a  whole).  More  specifically,  the  case  study  ‘Strengthening  human  capital  in  research  and 
innovation’978  suggests  that  a  large  majority  of  beneficiaries  surveyed  across  the  MSCA 
actions considered their participation in the MSCA to have contributed to their ability to attract 
funding  from  national,  regional  and  European  sources.  This  was  further  confirmed  in 
interviews  with  both  Horizon  Europe  beneficiaries  and  National  Contact  Points,  who 
considered  participation  in  the  MSCA  to  be  particularly  useful  in  making  participating 
organisations more competitive and able to attract funding from external sources, especially 
in the Widening Countries (which also include the associated countries). Other valuable tools 
include ‘Widening participation and spreading excellence’ (share of AC participation: 11.9%) 
and ‘Strengthening the ERA’ (share of AC participation: 9.1%) actions. The conclusions of 
Phase 1 of this study indicate that beneficiaries of the ‘Widening participation and spreading 
excellence’ programme under Horizon 2020 have an increased capacity to develop networks 
both locally and internationally as a result of their Widening projects. More specifically, being 
part of a consortium in an EU-funded project had a positive impact on the maturity and quality 
of the research group and on these institutions being considered reliable research partners. 
These conclusions are supported by interviews conducted during the preparation of this case 
study.   

During  the  interview  programme,  a  number  of  stories  were  identified  concerning  the 
successful  integration  of  associated  countries  into  the  European  R&I  system.  The 
recent  success  story  of  Tunisia  exemplifies  the  transformative  power  of  association  (and 
involves a number of participations surpassing initial estimates by the EC and Tunisia). With 
proactive  national  efforts  such  as  Information  Days,  outreach  activities  and  support  from 

975 At the time this case study was prepared, the EU Candidate countries comprised Albania, 
Bosnia and Herzegovina, Moldova, Montenegro, North Macedonia, Serbia, Turkey and Ukraine. 
Potential candidate countries were Georgia and Kosovo. 
976 Evaluation study on the external coherence and synergies of Horizon 2020 within the 
European research and innovation support system, 2023. 
977 Note: it is now possible for third countries to be partially associated, such as New Zealand. 
978 Annex 2: Case study 10 “Strengthening human capital in research and innovation”. 

575 

 
 
National Contact Points, the Tunisian research community quickly familiarised itself with the 
Horizon  Europe  programme  and  increased  its  participation  rates  compared  with  Horizon 
2020.  According  to  the  EC officials  interviewed,  the  successful  association  of  Tunisia  has 
triggered interest from other countries in the region who wish to follow the success of Tunisia 
and  its  increase  in  participation  rates  due  to  association.  Formal  negotiations  on  the 
association  of  Morocco  to  the  Horizon  Europe  programme  have  been  taking  place  since 
2021. At present, Morocco enjoys transitional measures and participates in the programme 
as a candidate associated country, but also as a low- or middle-income third country. Another 
notable  example  is  Ukraine.  Under  Horizon  2020,  Ukraine  became  one  of  the  top  seven 
associated countries in terms of two main indicators: the number of proposals supported and 
the  amount  of  funds  granted.  Despite  the  ongoing  Russian  war  of  aggression,  Ukrainian 
researchers  are  active  in  submitting  proposals  with  the  help  of  a  number  of  targeted 
measures aimed at supporting the Ukrainian research community979. In addition, Egypt has 
informally expressed an interest in association.  

The  above  examples  are  backed  up  by  the  increasing  shares  of  eligible  applications, 
eligible  proposals  and  retained  proposals  received  from  associated  countries  under 
successive EU Framework Programmes (Table 8). Across all three categories in the table, 
the  shares  show  that  the  impact  of  association  on  participation  is  most  notable  in  those 
countries  with  low  levels  of  R&I  performance.  The  EC  officials  interviewed  noted  that 
associated country status increases the interest of Member States in involving entities from 
associated  countries  in  consortia,  as  well  as  facilitating  enhanced  training  for  National 
Contact Points and offering improved access to pertinent information on participation in the 
Horizon Europe programme.  

Table 8. Shares of eligible applications, eligible proposals and retained proposals from associated countries under 
successive Framework Programmes. Source: developed on the basis of Horizon Dashboard data, accessed 24 October 
2023. 

As evidenced by the interview programme, long-standing traditions of collaboration with 
certain  associated  countries  have  led  to  the  close  integration  of  the  EU’s  R&I 
ecosystem  with  the  national  R&I  ecosystems  in  these  ACs.  Specifically,  the  research 
communities in Iceland and Norway view EU Framework Programmes as extensions of their 
national R&I landscape (i.e. fully integrated into their R&I ecosystem). In this regard, Norway 
and  Iceland's  deep  integration  with  EU  R&I  programmes  is  evident.  The  EEA  agreement 
provides  for  the  possibility  of  automatic  association  of  the  EEA  countries  to  the  Horizon 

979 Specific support for Ukrainian research and innovation. 

576 

 
 
 
Europe programme (through a technical amendment to Protocol 31 of the EEA Agreement). 
Both countries have a long history of association (Norway since 1987, Iceland since 1994). 
Their level of integration signifies the smooth operational relationships of both countries with 
the EU.  

Specific  international  cooperation  modalities  that  reinforce  the  integration  and 
coherence of the European R&I system with the R&I systems of associated countries 

Associated countries differ in terms of their performance within the Framework Programme. 
Interviews carried out for this case study revealed that the Eastern Partnership and Acceding 
Countries, Candidate Countries and potential Candidate Countries face numerous difficulties 
in attracting Horizon Europe funding. These difficulties include (but are not limited to) weak 
R&I systems (i.e. the level of public and private sector investment in research, development 
and  innovation  is  low);  finding  partners  for  collaborative  projects  and  host  institutions  for 
individual  projects  (due  to  limited  connections  with  partners  in  EU Member States);  and  a 
lack  of  high-quality  applications,  due  to  limited  experience  and  knowledge  about  key 
requirements  as  well  as  language  barriers.  The  European  Commission  has  designed  a 
number of specific measures to address these barriers and reinforce the integration of the 
R&I systems of associated countries with that of the EU:  

•  The Policy Support Facility (PSF) is a tool within the Framework Programmes that is 
geared towards improvements and synergies between national research systems. Since 
it  began  operating  in  2015,  the  Facility  has  provided  Member  States  and  associated 
countries with practical support for design, implementation and evaluation reforms that 
enhance  the  quality  of  their  R&I  investments,  policies  and  systems980.  Under  Horizon 
2020, all Member States participated in at least one exercise981. Among the associated 
countries,  Moldova,  Georgia,  Montenegro and  Ukraine  have  received specific support 
through the PSF. The Horizon Europe PSF was launched in February 2021. Since then, 
some  associated  countries  have  requested  specific  support for reforms  to  their  public 
R&D sectors982. Periodic feedback on the work of the PSF shows that the operational 
recommendations provided by leading experts and policy practitioners have catalysed 
changes  in  the  implementation  of  national  R&I  reforms983.  In  terms  of  participation, 
countries with both high and low R&D intensities have been active in PSF exercises. The 
EC officials interviewed for this study confirmed the positive effects of PSF support to 
associated  countries.  However, they  also  noted  that  the  successful implementation of 
PSF  recommendations  depends  on  a  pro-active  stance  being  taken  by  both  the 
governmental  institutions  in  the  countries  and  their  research  communities.  These 
findings are a continuation of the findings of the European Court of Auditors (2022)984, 
which confirmed that it was the responsibility of the participating country to respond to 
the recommendations that emerged from PSF activities. However, the Commission also 
uses the knowledge gained with regard to national R&I systems, mainly for its country 
assessments in  the context of  the  European  Semester, which  lead  to  country-specific 
recommendations. 

980 Statistics from the Policy Support Facility’s website, accessed at: 
https://ec.europa.eu/research-and-innovation/en/statistics/policy-support-facility  
981 Evaluation study on the implementation of cross-cutting issues in Horizon 2020 (2023). 
982 PSF, Support to Moldova on reforms in the public R&D sector, 2023.  
983 A New ERA for Research and Innovation: Staff Working Document, 2020. 
984 European Court of Auditors, Measures to widen participation in Horizon 2020 were well 
designed but sustainable change will mostly depend on efforts by national authorities, 2022. 

577 

 
 
through 

•  Contribution to the entry ticket to the Horizon Europe programme. In addition, the 
European Commission provides a contribution985 to the entry ticket to the Horizon Europe 
programme  for  some  Horizon  Europe  associated  countries  (specifically,  Enlargement 
countries,  all  Eastern  Partnership  (EaP)  Horizon  Europe  associated  countries  and 
Tunisia) 
the 
Neighbourhood,  Development  and  International  Cooperation  instrument  (NDICI). 
National  financial  contributions  to  the  FP  are  a  significant  burden  for  these  countries 
(influenced by their economic situation and low investment in R&D, but also by the effects 
of  COVID-19  and  the  Russian  war  on  Ukraine). Such  support is  important  not  only in 
economic terms, but also for their participation in the programme (e.g. Georgia uses the 
contribution to fund its national Horizon Europe office).  

for  Pre-accession  Assistance  (IPA)  and 

Instrument 

the 

•  Fostering bilateral cooperation between associated countries. The Horizon Europe 
associated  EaP  countries,  together  with  the  European  Commission,  invest  significant 
efforts  in  building  bilateral  cooperation  between  themselves  and  the  Member  States. 
More specifically, the Commission has launched regional meetings within EaP regional 
cooperation  in  the  field  of  R&I,  whereby  representatives  at  expert  level  from  all  five 
countries are invited to regular meetings986,987. Georgia signed a bilateral Memorandum 
of Understanding (MoU) with Armenia in October 2022, and with Moldova in May 2023. 
The outcomes of these MoUs are joint bilateral calls, which are expected to strengthen 
regional R&I cooperation. In terms of cooperation with the Member States, the German 
Ministry of Education and Science has provided very strong support for R&I cooperation 
with  Georgia,  while  France  provides  support  to  Armenia  (and  now  also  Ukraine). The 
bilateral  tools  are  expected  to  result  in  more  project  proposals  in  the  Horizon  Europe 
programme in the long-term perspective. 

•  Horizon Europe national offices. Georgia was the first country to establish a Horizon 
Europe national office in 2021, with the purpose of disseminating information about the 
programme to help beneficiaries prepare project applications and to guide them through 
implementation.  It  has  set  up  a  network  of  16  grant offices  at  16  different  universities 
over a period of two years. The national office network is a government-led and funded 
organisation. The initiative came from the previous Georgian Minister of Education and 
Science,  indicating  that  R&I  is  a  top  priority  for  the  country.  The  office  represents  a 
unique structure and has served as a role model for Moldova (which opened an office in 
May 2023) and Ukraine (whose office was opened  in October 2023 thanks to a financial 
contribution from the European Commission). According to the EC officials interviewed, 
the  Georgian  office  shows  signs  of  being  a  successful  intervention  and  may  have 
contributed  to  the  growing  number  of  applications  coming  from  Georgia  under  the 
successive FPs (71 eligible applications in 2018; 70 in 2019; 86 in 2020; 79 in 2021; and 
91  in  2022988).  Given  this  success,  such  a  structure  could  be  replicated  within  other 
associated countries and EU Member States, especially among the Widening group. 

In general, the Framework Programme actions supporting international cooperation that are 
outlined  in  this  section  are  important  in  bridging  gaps  and  fostering  international  scientific 

985 Under Horizon Europe, Ukraine and Moldova are given a full waiver; Armenia and Georgia 
are given a 50% waiver; and Tunisia is given a 65% waiver until the end of the FP (because the 
country is experiencing serious economic difficulties, reimbursement is used as a tool to 
guarantee stability across the whole Mediterranean region). 
986 Directorate-General for Research and Innovation, 5th Eastern Partnership Informal Working 
Group Meeting on Research and Innovation, 2023. 
987 Directorate-General for Research and Innovation, Fourth informal meeting within the Eastern 
Partnership Panel on research and innovation, 2022. 
988 Data retrieved from Horizon Dashboard on 2023/10/23. 

578 

 
 
collaborations.  However,  challenges  are  involved  in  assessing  the  effectiveness  of  these 
arrangements. For example, some instruments require a longer period to demonstrate their 
impact (e.g. despite the initial success of the Horizon Europe national office in Georgia, there 
is still a need to look into performance in the long term), while others (e.g. the Policy Support 
Facility)  require  separate  studies  and  evaluation  methods  to  better  grasp  their  complex 
influence  on  national  R&I  systems.  In  this  regard,  the  FP  would  benefit  from  a  more 
concentrated approach to governing those actions that support international cooperation.  

The role of national support and the contexts of the associated countries 

Lastly, it is important to note that, a country's participation rate and success in the 
FP  greatly  depends  on  its  individual  context  and  the  national  support  that  is 
available.  As  indicated  by  a  number  of  the  EC  officials  interviewed,  EU  support 
mechanisms cannot replace proactivity at national level. Those associated countries with 
lower  R&I  performance  inevitably  need  to  catch  up,  increase  their  R&D  funding,  and 
develop the capacities necessary to compete with countries that have longer experience 
of  participation.  This  is  especially  relevant  under  Pillar  1,  where  the  key  criterion  for 
participation  is  excellence.  In  this  regard,  Georgia  and  Tunisia  were  indicated  as 
examples of countries with a strong focus on building capacities and raising awareness 
of programme funding opportunities.  

In the case of the EFTA/EEA associated countries (Norway and Iceland), their strong 
performance  in  the  programme  parts  under  analysis  (a  detailed  overview  of  their 
participation  in  various  programme  parts  is  provided  in  Appendix  4)  and  across  the 
Framework Programme in general, is influenced by a number of factors at national level. 
More specifically: 

•  Norway has a strategic and efficient structure for encouraging participation in 
EU  Framework  Programmes.  Entities  such  as  the  Research  Council  of  Norway 
(which focuses on Pillars 1 and 2) and Innovation Norway (which focuses on Pillar 
3) make it easier to connect FP activities to national activities. Norway has a very 
active  and  well-organised  NCP  network.  In  addition,  Norwegian  R&D  spending  is 
high,  and  the  government  actively  promotes  participation  in  the  FP  via  financial 
allocations and subsidies such as Project Establishment Support (PES) and top-up 
funding  for  ERC  applicants.  Representatives  of  the  Norwegian  Research  Council 
who  were  interviewed  noted  a  culture  of  sharing  and  proactiveness  among 
institutional  actors  (including,  but  not  limited  to,  the  network  of  EU  advisors, 
reference groups, representatives of universities) as contributing to the shaping of 
the  EU  R&I  agenda  (e.g.,  through  participation  in  the  Programme  governance 
structures) and gaining information upfront (e.g. Norwegian universities have offices 
in Brussels, in addition to the office of the Research Council of Norway). These were 
identified  as  key  factors  contributing  to  Norway’s  success  in  the  Horizon  Europe 
programme.  
Iceland is a smaller country, and thus does not have as many resources to run similar 
structures  to  those  of  Norway.  At  the  same  time,  recent  administrative  decisions 
such  as  merging  research,  innovation  and  digital  affairs  under  a  single  ministry, 
signify Iceland's commitment to facilitating better information exchange and creating 
synergies.  
In  addition,  the  EFTA/EEA  associated  countries  actively  participate  in  the 
governance structures of the Framework Programmes. All associated countries 
have  the  right  to  participate  in  the  governance  structures  of  the  Horizon  Europe 

• 

• 

579 

 
programme (e.g. by joining the programme committee meetings as observers), but 
without voting rights. With regard to this, both the EC officials and representatives of 
the Norwegian Research Council interviewed underlined the need to be proactive in 
putting  forward  needs  and  ideas  for  improvement.  Since  Norway  and  Iceland  are 
associated via the EEA agreement, they also participate in the joint committee run 
by EU Central Services, which also covers topics beyond R&I. Both vehicles provide 
these countries with early access to relevant information (e.g. the topics of upcoming 
tools), which leads to better prepared ecosystems. These interviews also underlined 
the  proactive  stance  taken  by  Norway  in  building  bilateral  relations  with  similar 
countries including other Nordic countries, the Baltic states and major R&I players 
in the EU (e.g. Spain, Italy and Germany). 

The  EC  officials  who  were  interviewed  noted  that  due  to  the  successful  performance  of 
Norway  and  Iceland,  these  countries  are  encouraged  to  share  their  best  practices  and 
expertise (e.g. how to establish strong networks of NCPs) with Widening Countries and newly 
associated countries, as well as countries considering association to Pillar 2 (New Zealand, 
South Korea and Japan). In terms of both participation in research projects and spreading 
know-how,  the  EEA  associated  countries  provide  strong  input  for  a  stronger  European 
scientific base.  

1.3.2.2. 

Effectiveness of international cooperation in achieving the scientific 
objectives of the Framework Programme 

Among  the  specific  objectives  of  Horizon  Europe  developing,  promoting  and  advancing 
scientific excellence; supporting the creation and diffusion of high-quality new fundamental 
and applied knowledge, and of skills, technologies and solutions; supporting the training and 
mobility of researchers, and attracting talent at all levels. Set against this background, this 
section  of  the  case  study  aims  to  assess  the  effects  of  international  cooperation  and 
association  to  the  Framework  Programme  on  the  production  of  high-quality  research,  the 
contribution  to  prominent  topics,  and  how  international  cooperation  modalities  allow  the 
strengthening of human capital engaged in EU R&I activities, taking into account the limited 
data available at this stage.  

International co-publications funded by the Framework Programme 

One of the key ways to measure the scientific performance of the Framework Programmes 
is  through  the  share  of  international  scientific  co-publications  that  are  funded  by  the 
Framework  Programme989.  However,  it  is  too  early  yet  to  assess  data  on  publications 
produced under Horizon Europe. Hence, a bibliometric analysis of Horizon 2020 programme 
data990 is used to answer this evaluation question. During the period 2014-2022, a total of 
138,888  publications  were  linked  to  35,855  Horizon  2020  projects.  Table  9  provides  an 
overview of the contributions to this scientific output from non-EU Member States (i.e. Horizon 
2020  associated  countries  and  the  top  15  non-associated  third  countries).  Given  the  high 
share of publications in the sample that are related to the ERC (35.6%) and MSCA (22%), 
such an overview is of relevance to Pillar 1. 

989 Evidence Framework on the monitoring and evaluation of Horizon Europe. 
990 Conducted during the Phase 1 of this study.  

580 

 
 
H2020 programme associated countries 

Top 15 non-associated third countries 

Country 

Publications 

% of 
publications 

% of 
participations 
in Horizon 
2020 

Country 

Publications 

% of 
publications 

% of 
participations 
in Horizon 
2020 

Switzerland 

12,088 

8.7% 

3.1% 

US 

22,934 

16.5% 

1.1% 

Israel 

3,914 

2.8% 

1.1% 

China 

7,340 

5.3% 

0.3% 

Norway 

3,662 

2.6% 

1.8% 

Australia 

4,673 

3.4% 

0.2% 

Turkey 

947 

0.7% 

0.7% 

Canada 

4,161 

3.0% 

0.3% 

Ukraine 

869 

0.6% 

0.2% 

Japan 

4,088 

2.9% 

0.1% 

Serbia 

647 

0.5% 

0.3% 

Russia 

3,153 

2.3% 

0.1% 

Iceland 

382 

0.3% 

0.2% 

Brazil 

2,231 

1.6% 

0.1% 

Georgia 

135 

0.1% 

0.0% 

Chile 

1,532 

1.1% 

0.1% 

Armenia 

128 

0.1% 

0.3% 

South 
Korea 

1,532 

1.1% 

0.1% 

Tunisia 

115 

0.1% 

0.1% 

India 

1,446 

1.0% 

0.0% 

North 
Macedonia 

Moldova 

Bosnia and 
Herzegovina 

Albania 

89 

77 

57 

26 

0.1% 

0.1% 

South 
Africa 

1,070 

0.8% 

0.1% 

0.1% 

0.0% 

Singapore 

819 

0.6% 

0.0% 

0.04% 

0.1% 

Mexico 

786 

0.6% 

0.1% 

0.02% 

0.0% 

Argentina 

725 

0.5% 

0.1% 

Montenegro 

18 

0.01% 

0.0% 

New 
Zealand 

598 

0.4% 

0.0% 

Faroe 
Islands 

13 

0.01% 

0.0% 

Total 

41,428 

29.8% 

Total 

21,194 

15.26% 

Table 9. Contributions to Horizon 2020 scientific output from non-EU countries. Source: quantitative tracking results from 
Phase 1 of this study and the Horizon Dashboard, accessed on 25 October 2023. 

581 

 
 
 
 
 
 
 
The co-publication data show that researchers from associated countries contributed to more 
than 15% of publications under Horizon 2020. Among Horizon 2020  associated countries, 
researchers  from  Switzerland  led  the  contribution  count  and  were  involved  in  8.7%  of 
publications  produced  by  Horizon  2020  projects,  followed  by  Israel  (2.8%)  and  Norway 
(2.6%).  Among  the  non-associated  third  countries,  the  US  produced  the  largest  share  of 
publications,  with  16.5%  of  Horizon  2020-related  publications  involving  US  co-authorship, 
which is much higher than the participation ratio of any other non-EU country. Contributions 
by the UK (previously an EU Member State under Horizon 2020) accounted for 25.4% (35 
219) of publications.  

When examining participation shares under Horizon 2020, it becomes evident that there is a 
disparity  between  publication  outputs  and  participation  rates,  particularly  when  comparing 
associated  countries  with  non-associated  third  countries.  For  most  associated  countries, 
there is a close alignment between their percentage of publications and their Horizon 2020 
participation rate. In contrast, non-associated third countries such as the United States, China 
and  Australia  display  relatively  high  shares  of  publications,  at  16.5%,  5.3%  and  3.4% 
respectively, compared with their shares of participations, which are 1.1%, 0.3% and 0.2%, 
respectively.  This  divergence  suggests  that  these  non-associated  third  countries  have  an 
expansive influence on global research, even if their integration into Horizon 2020 projects is 
not as profound. 

This divergence might be better understood in the light of findings from Phase 1 of this study, 
which indicated a positive relationship between the mobility of researchers and the share of 
international scientific co-publications. When researchers from Europe relocate or collaborate 
with  institutions  abroad,  they  frequently  maintain  active  academic  ties  with  their  home 
countries. This enhances the collaboration dynamics between their home countries and their 
host nations. The mobility programmes under Pillar 1, especially the MSCA, facilitate this by 
enabling  researchers  to  form  connections  abroad.  Consequently,  while  the  formal 
participation in Horizon 2020 of some non-associated third countries might be limited, their 
scientific  influence  and  collaborative  ties,  fostered  through  mobility  and  co-publications, 
remain extensive and impactful. This highlights the pivotal role mobility programmes play in 
enriching the Union's science base and fostering global connections, even with countries that 
are not deeply integrated into the Framework Programmes. 

Contributions of international participants to different fields of science  

The study team also looked into the contributions made by international participants to 
different fields of science. To do so, a comparative data analysis was conducted using 
EuroSciVoc Level 1 classification991 across the programme parts under analysis during 
the two programming periods (i.e. Horizon 2020 and Horizon Europe). By segmenting 
data according to country groups (MS, AC, TC, CH and UK), the team aimed to discern 
the  key  strengths  in  the  scientific  contributions  of  international  partners.  Before 
discussing the results of this analysis, it is important to note that the label ‘field of science’ 
is  not  necessarily  attributed  consistently  to  each  signed  grant,  and  hence  these  data 
must be interpreted with caution. Also, the available data provide limited opportunities to 
discern which partner was responsible for a specific part of a collaborative project (which 
often represent interdisciplinary approaches).  

991 EuroSciVoc is a multilingual taxonomy that represents all of the main fields of science 
discovered from CORDIS content, organised through a semi-automatic process based on NLP 
techniques. For more, see: https://op.europa.eu/en/web/eu-vocabularies/euroscivoc. 

582 

 
 
Figure 1. Signed grants under the ERC, MSCA, Research Infrastructures and WIDERA, by Framework Programme, field of 
science and country group. Note: percentages represent the share of signed grants within the programme part that are 
allocated to a particular field of science. The sum of the percentages per programme part may exceed 100% because in 
some cases, more than one field of science is allocated to a signed grant. Source: Horizon Dashboard, accessed 25 
October 2023. 

The above analysis showed that the distribution of fields of science across the country groups 
is  similar  under  both  programming  periods.  In  this  regard,  'Natural  sciences'  consistently 
takes the top spot across all country groups in both programming periods and programme 
parts (with the exception of ‘Science with and for Society’ under Horizon 2020, where ‘Social 
sciences’  take  the  lead).  However,  the  data  also  indicate  that  international  partners 
supplement  areas  in  which  EU  Member  States  may  be  lacking.  For  instance,  projects 
involving  third  country  participants  in  Research  Infrastructures  contribute  more  to  the 
‘Engineering  and  technology’  field  under  Horizon  Europe  compared  with  the  thematic 
distribution of the Member States. 

Strengthening the EU’s human capital base through international cooperation 

The third dimension of analysis in relation to scientific excellence relates to strengthening of 
the EU’s human capital base through international cooperation. To examine this, the study 
team looked at the mobility patterns of MSCA Postdoctoral Fellowship fellows under Horizon 
Europe, to examine the mobility choices of fellows originating from associated countries and 
Third  countries.  Figure  2  illustrates  that  both  associated  country  fellows  and  third  country 
fellows show a pronounced flow towards EU non-Widening countries. Meanwhile, there is a 
discernible  but  relatively  limited  flow  from  associated  countries  and  third  countries  to  EU 
Widening countries. Such a distribution raises important questions as to how to ensure that 

583 

 
 
the entire EU region, especially Widening countries, can benefit from the global talent pool, 
while also offering appealing opportunities to researchers from outside the EU. 

Figure 2. Nationalities of MSCA PF fellows flowing to host institution, by country group. Source: EC monitoring data (cut-off 
date 26 July 2023). Note: Third countries in this Figure includes the UK and Switzerland. 

In terms of MSCA impacts in bringing the international excellence back to the EU through 
Horizon Europe programme, Figure 3 below shows that significant numbers of fellows 
of EU nationalities who use the MSCA Postdoctoral Fellowship scheme to return to their 
EU country of nationality. This suggests that MSCA continues to support fellows to return 
to the country of their nationality.  

Figure 3. MSCA PF fellows with the same nationality as the host country under Horizon Europe. Source: EC monitoring data 
(cut-off date 26 July 2023). Note: Third countries exclude Switzerland and the United Kingdom, given their changed status. 

Key changes affecting the scientific performance of the EU 

The Horizon Europe programme introduced a number of changes to the policy of association 
(as  discussed  in  the  section  on  relevance).  A  few  of  these  could  have  an  impact  on  the 
scientific performance of the Framework Programme.  

First, there have been changes in the country group status of key international partners, 
most notably the United Kingdom and Switzerland. Although Switzerland was associated 
to all parts of Horizon 2020 as of January 2017 (after a period of partial association), it is not 
currently  associated  to  Horizon  Europe992.  The  United  Kingdom,  formerly  an  EU  Member 
State under Horizon 2020, is associated to Horizon Europe as of January 2024 under a Trade 

992 https://research-and-innovation.ec.europa.eu/strategy/strategy-2020-2024/europe-
world/international-cooperation/bilateral-cooperation-science-and-technology-agreements-non-
eu-countries/switzerland_en. 

584 

 
 
 
 
and Cooperation Agreement. This was confirmed in a joint statement by the EC and the UK 
Government on 7 September 2023993.  Under Horizon 2020, both countries featured in the 
top 10 beneficiary countries in the programmes analysed: the UK ranked first, with 14.2% of 
participations and 16.8% of EU contribution received; Switzerland ranked seventh, with 3.9% 
of participations and 6.5% of EU contribution received. So far, the number of participations 
from the UK under Horizon Europe appears to be lower than under Horizon 2020, due to its 
participation in the first years of the Programme as a non-associated third country.  

Second, the Strategic Plan 2020-2024 of DG Research and Innovation994 outlines a specific 
objective  with  a  relevant  result  indicator  to  cover  the  broad  spectrum  of  R&I  activities 
contributing  to  a  ‘Stronger  Europe  in  the  world’,  namely  ‘Specific  Objective  4.1:  Regional 
research  and  innovation  strategies  and  broader association  policy  contribute  to  promoting 
common  European  R&I  values  and  creating  a  global  R&I  space’.  This  specific  objective 
continues  and  reinforces  the  EU’s  general  openness  to  international  participation,  with 
targeted actions involving specific key actors. More specifically, it opens up the possibility of 
including into the FP’s activities a wider range of countries that have a well-established R&I 
base,  through  the  option  of  partial  association  of  countries  beyond  the  vicinity  of  the 
EU995. Horizon Europe opens up the possibility for any country with a strong STI capacity to 
become  associated  to  the  programme,  as  long  as  it  fulfils  a  number  of  criteria  relating  to 
democratic institutions, human rights, market economy, protection of IPR, and so on.  New 
Zealand  has  taken  the  lead  and  became  associated  to  Pillar  2  of  Horizon  Europe  in  July 
2023.996 Canada, Japan, Singapore and South Korea are other countries interested in the 
possibility  of  joining  the  Framework  Programme997.  Association  negotiations  with  Canada 
were  concluded  in  November  2023,  and  Canada  is  expected  to  join  the  Framework 
Programme  in  2024.  Association  negotiations  with  the  Republic  of  Korea  are  ongoing. 
However, these countries must negotiate access and fulfil the general criteria set by the EU. 
This ensures that the EU gets a fair balance of benefits from  their association in terms of 
access to talents, institutions, facilities and resources, but also in teaming up with countries 
that  share  common  values  and  principles  with  the  EU  in  the  current  unstable  geopolitical 
context.  

The focus of these associations is on Pillar 2, which addresses global societal challenges on 
which  the  EC  sees  strong  mutual  interest  in  working  together  with  strong  international 
competitors to jointly address global challenges such as climate change, biodiversity, health 
and a secure energy supply. According to the EC officials interviewed for this study, Pillar 2 
is  where  the  mutual  benefits  of  association  are  expected  to  be  the  most  prominent. 
Association to Pillar 1, which emphasises the excellence of research in Europe, is not under 
active  consideration.  The  potential  ramifications  of  associating  third  countries  such  as 
Canada,  South  Korea and Japan  with  Pillar  1  warrant  careful  evaluation,  according to  the 
officials  interviewed.  Any  move  in  this  direction  –  particularly  with  regard  to  bottom-up 
programme parts such as the MSCA and ERC, could possibly lead to projects that are devoid 
of  a  European  dimension  and  instead  reinforce  the  dominant  position  of  some  important 
competitors  on  the  international  scene.  This  underlines  the  increased  emphasis  under 
Horizon Europe on the need to protect EU interests. 

993 https://ec.europa.eu/commission/presscorner/detail/en/statement_23_4375. 
994 Strategic Plan 2020-2024 of DG Research and Innovation. 
995 In Article 16 of the Horizon Europe Regulation, this country group is called 1D 'Other third 
countries or territories'. 
996 European Commission, New Zealand joins Horizon Europe Research and Innovation 
Programme, 2023. Accessed 1 August 2023. 
997 European Commission, New Zealand joins Horizon Europe Research and Innovation 
Programme, 2023. Accessed 1 August 2023. 

585 

 
 
Lastly, reciprocal access to programmes is a new requirement998 defined by Article 16.4 
of the Horizon Europe Regulation. This requirement entails that entities from the EU should 
be permitted access, to the extent possible, to participate in the equivalent programmes of 
associated  countries.  Here,  reciprocity  is  seen  as  more  than  just  a  mutual  exchange;  it 
represents  a  conscious  effort  to  intertwine  the  European  R&I  system  with  those  of 
international  partners.  It  is  important  to  note  that  the  reciprocity  clause  refers  only  to  the 
provision of access and not to dedicated funding. Each Association Agreement999 comes with 
an  Annex  2,  which  lists  equivalent  national  programmes,  calls  or  actions  available  for 
reciprocal  participation.  Annex  2  varies  from  country  to  country,  reflecting  the  specific 
programmes or initiatives each nation offers. Israel, for instance, has a comprehensive list of 
programmes  detailed  in  its  annex1000.  In  the  case  of  other  countries,  such  as  Armenia, 
programmes  are  currently  under  development  and  will  be  opened  up  once  launched. 
Nevertheless, the EU continually encourages these countries to establish R&I programmes 
that would be open to EU-based entities.  

Reciprocity is also a principle that the Horizon Europe programme also aims to establish with 
the  non-associated  third  countries.  More  specifically,  this  is  approached  through  the  S&T 
agreements  (e.g.  in  recognition  of  the  opening  up  of  US  National  Institutes  of  Health 
programmes to European researchers, any legal entity established in the United States is 
eligible to receive EU funding under Cluster 1 of Horizon Europe1001). Thus, under Horizon 
Europe there is a recognised necessity to establish a system to oversee this reciprocity.  

Monitoring  is  facilitated  through  Joint  R&I  Committees,  established  by  the  association 
agreements  with  associated  countries.  Each  committee  convenes  annually  to  review  the 
cooperation  and  participation  in  the  programme  of  the  relevant  associated  country,  and 
assesses the implementation of the agreement, including how the reciprocity clause is being 
implemented. One such example is Georgia, which has already shared data about the EU 
researchers making use of its national funding opportunities. 

Specific  international  cooperation  modalities  that  reinforce  the  FP’s  scientific 
excellence 

The  EU  facilitates  international  cooperation  to  increase  the  scientific  performance  of  the 
Framework Programme through a number of pillar-specific policy and support instruments. 
This section of the case study discusses the key instruments that are most pertinent to Pillar 
1 and WIDERA.  

Co-funding mechanisms. To foster the participation of legal entities that are not inherently 
eligible  for  funding  under  the  Horizon  Europe  programme,  several  third  countries  and 
territories have initiated co-funding arrangements. These arrangements vary in structure and 
purpose.  In  terms  of  Pillar 1,  co-funding  can  be  provided  by  third  country  participants  in 
Doctoral  Networks,  Staff  Exchanges  (SE)  actions  and  COFUND  under  the  Marie 
Skłodowska-Curie Actions, as well as for Research Infrastructure actions. Country-specific 
co-funding modalities are presented in Table 10 below.  

998 In the past, reciprocity clauses were defined only in the agreements with Switzerland and the 
Faroe Islands. 
999 It should be noted that the association of Norway and Iceland is conducted through EFTA and 
EEA agreements, which do not include an Annex on reciprocal access to programmes. 
1000 Annex 2 of the Agreement between the European Union and Israel on the participation of 
Israel in the EU’s Horizon Europe Framework Programme for Research and Innovation. 
1001 Horizon Europe Work Programme 2023-2024: Health. 

586 

 
 
Country 

Canada 

Brazil 

Japan 

Monaco 

Co-funding arrangements 

MSCA Staff Exchanges are facilitated by Mitacs Globalink Research Award or the 
Mitacs Accelerate International programme. These avenues offer funding for 
research projects in the EU, benefitting graduate students and postdoctoral fellows. 
The duration of research must fall between 12 and 24 weeks under Globalink and 
16-24 weeks under the Accelerate programme.  

To support the participation in Horizon Europe of entities established in Brazil, on 
19 November 2021 the Commission and three Brazilian funding agencies – the 
National Council for Scientific and Technological Development (CNPq), the 
Funding Agency for Studies and Projects (FINEP), and the Brazilian National 
Council of State Funding Agencies (CONFAP) – signed an Administrative 
Arrangement on the mechanisms to support the participation of entities established 
in Brazil in collaborative projects, Partnerships and Missions under Horizon Europe 
(2021-2027). 

The Japan Society for the Promotion of Science (JSPS) offers support for 
Japanese research entities involved in the MSCA Staff Exchange scheme. Support 
is detailed within the Japanese Core-to-Core Program and Bilateral Research 
Projects. 

The Principality of Monaco has instituted a co-funding mechanism for select 
Monegasque entities participating in collaborative projects under Horizon Europe. 
Funding aligns with Monaco's research and innovation priorities, and covers areas 
such as energy, bio-economics, digital technology, health, and Marie Skłodowska-
Curie actions. Eligible entities in Monaco can request up to 50% of eligible costs 
directly from the Prince’s Government, upon successful submission of a proposal. 

Republic of 
Korea 

Korean research entities that successfully join Horizon Europe programme projects 
open to international cooperation can seek matching funds from the Korean 
government, regardless of their cluster or topic. The Korean Ministry of Science 
and ICT provides co-funding to successful entities under both the Horizon Europe 
programme and the domestic competition organised by the National Research 
Foundation of Korea. 

Taiwan 

Taiwan’s National Science and Technology Council (NSTC) has announced 
through a formal letter addressed to the European Commission that it will continue 
to co-fund Taiwan’s participation under the Horizon Europe programme. 

Switzerland 

The Swiss government has put in place a number of schemes and procedures to 
ensure the funding of Swiss participations in Horizon Europe actions: a specific 
financial guarantee for Swiss participants in ERC Starting and Consolidator Grants 
2021; a financial guarantee for Swiss participants as associated partners in 
collaborative projects for previous calls under Horizon Europe in 2021 and 2022; a 
similar financial guarantee for Swiss participants as associated partners in 
collaborative projects under Horizon Europe programme 2023 calls; and direct 
funding mechanisms for mono-beneficiary projects (transfer grants and direct 
funding). 

Table 10. Countries with co-funding mechanisms for Horizon Europe programme Pillar 1. Source: Horizon Europe 
complementary funding mechanisms in third countries.1002 

1002 EU Grants: HE Complementary funding mechanisms in third countries. 

587 

 
 
the  Framework  Programme,  assessing 

Even  though  these  co-funding  arrangements  have  the  potential  to  enhance  the  scientific 
performance  of 
the  effectiveness  of  such 
arrangements  is  challenging,  as  highlighted  by  interviews  with  EC  officials.  The  European 
Commission  often  lacks  comprehensive  feedback  on  participation  due  to  its  limited 
involvement  in  funding  decisions  and  implementation  arrangements.  Consequently,  the 
Commission’s capacity to monitor and influence such arrangements remains limited. Notably, 
a significant portion of co-funding mechanisms primarily focus on Pillar 2 actions, and may 
not align well with bottom-up initiatives. In practice, funding for successful applications tends 
to  be  earmarked  for  calls  relating  to  collaborative  projects  that  resonate  with  a  country’s 
specific  interests  and  national  priorities.  This  misalignment  is  particularly  evident  in  the 
context  of  actions  such  as  the  MSCA  and  ERC.  It  is  also  important  to  note  that  it  occurs 
despite a significant number of MSCA and ERC opportunities being offered and consequently 
projects  being  funded  in  the  priority  areas  of  non-associated  third  countries.  An  important 
initiative in addressing this misalignment is the MSCAdvocacy project, which recognises the 
need to understand the national contexts of partner countries and to extend awareness about 
funding opportunities beyond traditional avenues. 

ERC Implementing Arrangements1003. Since 2012, a series of Implementing Arrangements 
have been negotiated by the ERC with funding organisations around the world. These provide 
opportunities  for  researchers/scientists  supported  by  non-European  funding  agencies  to 
temporarily  join  research  teams  run  by  ERC  grantees.  The  countries  involved  include 
Argentina,  Australia,  Brazil,  Canada,  India,  Japan,  South  Korea,  Mexico,  the  People’s 
Republic  of  China,  Singapore,  South  Africa,  the  United  States  and  Thailand  (agreement 
signed under the Horizon Europe programme). The Implementing Arrangements allow visits 
for non-associated  third  country  researchers  to  organisations  that  are implementing  ERC-
funded projects. The selection process is led by the third country (the European Commission 
is  not  involved),  and  is  based  on  the  criterion  of  excellence.  In  this  regard,  the  European 
Research  Council  allows  broad  flexibility  (i.e.  professors,  postdocs  and  even  doctoral 
students  can  apply  and  visit  the  organisations).  However,  interviews  with  EC  officials 
highlighted that participation is not uniform across countries due to differences in terms of the 
allocated  funding,  call  and  selection  processes.  The  number  of  visits  depends  on  the 
proactiveness of the third country. Brazil and South Korea were mentioned as examples of 
countries that had intensified efforts to send researchers to visit ERC-funded projects. 

ERC  Mentoring  Initiative.  Over  its  years  of  operation,  the  ERC  has  implemented  and 
supported  a  number  of  mechanisms  aimed  at  encouraging  quality  applications  from 
researchers  in  countries  with  modest  levels  of  participation  and  success  rates  in  the 
programme. The national agencies of EU Member States and associated countries also have 
support  mechanisms  for  PIs  initiating  such  proposals.  However,  such  programmes  often 
struggle to identify international experts that are able to help with their proposals. Hence, the 
ERC  Mentoring  Initiative1004,1005  aims  to  boost  existing  support  programmes  for  ERC 
applicants by helping to identify international experts who can provide coaching and advice 
via local offices1006 in the EU Member States and associated countries. Interviews suggest 

1003 ERC Implementing Arrangements. 
1004 ERC Mentoring Initiative. 
1005 ERC Guidelines for the Mentoring Initiative. 
1006 Currently, the following ERC Mentoring Initiatives are in place: Cyprus – the Research and 
Innovation Foundation (RIF); Czech Republic – the Czech National Expert Group (NEG) for the 
ERC; Estonia – the Estonian Research Council (ETAg); Hungary – the Hungarian Academy of 
Sciences (MTA); Italy – the National Research Council (CNR); Malta – the Malta Council for 
Science and Technology (MCST); Poland – the National Centre for Research and Development 
(NCBR), the National Science Centre (NCN) and the Foundation for Polish Science (FNP); 

588 

 
 
 
that  the  success  of  mentoring  initiatives  is  highly  dependent  on  the  engagement  and 
capacities of NCPs, which serve as links between the FP and local research communities. 
Currently, the efficiency of mentoring activities under the ERC is not evaluated in a structured 
manner, but interviewees confirmed that such an exercise is planned in the near future.  

1.3.3.  Efficiency 

Evaluation questions 

EFF6: To what extent has the Framework Programme (overall, and in the different areas 
supported) reached newcomers, non-traditional R&I actors such as citizens, different organisation 
types including civil society organisations, and beneficiaries from across Europe and 
internationally, and what lessons can be learned in terms of openness and engagement? 

EFF12:  What  could  be  done  in  the  future  to  improve  the  attractiveness  and  outreach  of  the 
Framework Programme to key targeted groups, given its objectives and core principles, in particular 
its focus on excellence? 

This section of the report on efficiency looks into how and to what extent Horizon Europe has 
reached  beneficiaries  internationally,  and  what  could  be  done  in  the  future  to  improve  the 
attractiveness and outreach of the Framework Programme.  

In general, the outcomes of the present evaluation suggest that despite the intention of the 
outreach mechanisms to reach international participants, evaluation of their ability to do so is 
limited by the lack of a unified monitoring and evaluation system. Although data on various 
aspects (e.g. numbers of job postings, numbers of NCPs, numbers of success stories) are 
available  online,  these  offer  limited  insights  into  the  effects  of  outreach  on  international 
participants. Furthermore, although the survey did not cover all of the outreach and support 
measures  identified  as  being  offered  by  the  FP,  a  lack  of  support  during  the  application 
process  was  indicated  across  all  groups  of  countries  and  signified  a  potential  area  for 
improvement.  In  this  regard,  the  interview  programme  revealed  that  organisations  and 
researchers face difficulties when attempting to navigate the extensive pool of information 
available.  Freeform  answers  received  from  survey  respondents  also  suggest  that 
beneficiaries face difficulties in interpreting changes related to the status of third countries in 
the programme and the rules governing their participation.  

1.3.3.1. 

Outreach of the Framework Programme internationally (overall and in the 
reference programmes) 

Desk research identified a number of outreach mechanisms employed under the Framework 
Programme  to  inform  international  applicants  and  beneficiaries  regarding  the  general 
openness of the programme, the R&I funding and collaboration opportunities available, and 
the application/evaluation process. Insights from the interviews built upon the results of the 
desk  research  (e.g.  helping  to  identify  those  most  relevant  to  the  programme  parts  under 
consideration),  and  also  to  obtain  stakeholder  perceptions  regarding  the  efficiency  of  the 
current arrangements in relation to the reference programme parts.  

Slovakia – the Horizon National Office, Slovak Centre of Scientific and Technical Information 
(CVTI SR); Turkey – the Scientific and Technological Research Council of Turkey (TÜBİTAK). 

589 

 
 
National Contact Points and their networks 

National Contact Points (NCPs) serve as an important means of disseminating information 
about  the  open  funding  calls  and  collaboration  opportunities  under  Horizon  Europe.  Two 
variants of NCPs for third countries are available: 1) NCPs in associated countries that are 
part of the NCP networks together with the NCPs of EU Member States; and (2) NCPs in 
non-associated  third  countries.  Analysis  of  those  NCPs  currently  established  in  various 
countries shows limited numbers in third countries – only 34 third countries have at least one 
NCP  established  (302  NCPs  in  total).  The  numbers  are  more  significant  in  associated 
countries (317 in total, average 16.3 per country). Insights derived from the interviews carried 
out for this case study suggest that the role and practices of NCPs are not consistent across 
the international partners. In particular, the efficiency of the work of NCPs is highly dependent 
on the motivation of the representatives and their institutional capacities (e.g. in terms of ERC 
networks, the South Korean NCP was mentioned as highly effective operation, and Georgian 
and Tunisian NCPs were noted as active participants in numerous NCP networks). Efficiency 
is also influenced by availability and turnover among NCP representatives (due to personal 
and  career  reasons).  In  addition,  the  NCP  Experience  Report1007  and  NCP  Best  Practice 
catalogue1008 point to the diversity and fragmentation of NCP networks and the limited flow 
of information between the European Commission and NCPs that form the basis for high-
quality advice. The networks of NCPs serve as mitigation mechanisms in addressing these 
problems. In relation to the specific programme parts under analysis, relevant activities are 
implemented by the following networks (Table 11). 

Programme 
part 

MSCA 

NCP network activities 

The MSCA-Net project facilitates transnational cooperation between NCPs for 
the MSCA to achieve a consistent and harmonised level of NCP support. In this 
regard, the project offers training and mentoring schemes to the community, with 
the goal of developing the capacities of MSCA NCPs. It also generates support 
materials such as guides1009 and handbooks in collaboration with the European 
Research Executive Agency. The project is continuation of work carried out 
under Horizon 2020; however, it currently gives an intensified focus to 
associated countries and non-associated third countries. The representatives of 
MSCA-Net interviewed noted that the community of the project is highly 
welcoming and cooperative in offering support and sharing know-how. 
Newcomers are integrated quickly through well-established processes that 
include EC officials. Meanwhile, the MSCAdvocacy project plays a crucial role in 
supporting non-associated third countries, as it aims to provide direct support in 
order to better align MSCA cooperation mechanisms in 20 key partner countries 
and six regions with the European Commission’s Global Approach to Research 
and Innovation. In this regard, the project is expected to provide analysis and 
recommendations on leveraging the international footprint of the MSCA at policy 
level and making MSCA cooperation broader, in line with shared EU values and 
principles. 

1007 NCP Experience Report 2019. 
1008 Best Practice catalogue, 2020. 
1009 Three policy briefs were developed on Open Science, Missions and gender aspects. These 
served a dual purpose: 1) the above elements are very important in successful proposals, so 
NCPs need to understand what lies behind these concepts in order to explain it well to potential 
beneficiaries; and 2) the Commission was very interested in the development of these guides 
because they enable links to be built between EU strategic priorities (e.g. the Green Deal) and 
MSCA actions that are bottom-up. 

590 

 
 
Programme 
part 

ERC 

NCP network activities 

Stakeholders who were interviewed noted that the ERC lacks similar support 
actions, and underlined this as a disadvantage that limits its community- and 
capacity building. The ERC addresses the issue of fragmentation (at least to 
some extent) through biannual meetings of its NCPs (after the COVID-19 
pandemic, face-to-face activities were resumed in 2023). However, the NCPs of 
non-associated third countries are not automatically invited to these meetings, 
which as a result limits their access to relevant information. 

Research 
Infrastructures 

RICH Europe gathers together experts from European and third countries to 
collect, organise and provide access to information on RI projects, transnational 
access opportunities, policy issues, stakeholders, national and regional 
initiatives on RIs, news, etc. 

WIDERA 

NCP_WIDERA.NET supports the capacity and effectiveness of WIDERA NCPs 
in low-performing Member States and associated countries, and fills knowledge 
gaps to improve access to funding opportunities under the Framework 
Programme. It hosts wideraexperts.eu, which is a free-of-charge portal that 
facilitates the matchmaking of experts with applicants from Widening countries 
who are preparing proposals within the Widening, ERA, Pillar 2 clusters and EIC 
Pathfinder schemes under Horizon Europe. In addition, the network provides 
opportunities for study visits to Spain, Germany, Italy and Finland. 

Table 11. Activities of NCP networks in the reference programmes. Source: prepared by the study team, based on 
interviews and information on the MSCA-Net, RICH Europe and NCP_WIDERA.NET websites 

EURAXESS and EURAXESS Worldwide 

EURAXESS provides a range of services to professional researchers, including access to 
thousands of job postings both in and outside of the EU. The programme parts under analysis 
employ  these  networks  to  varying  degrees.  In  the  case  of  MSCA,  beneficiaries  of  MSCA 
Doctoral  Networks  and  COFUND  have  an  obligation  to  post  vacancy  notices  on  the 
EURAXESS website, and host organisations willing to host MSCA Postdoctoral Fellowships 
researchers also use the portal to advertise their hosting offers. However, it mostly focuses 
on  promotion  towards  individual  researchers,  and  more  targeted  communication  towards 
organisations  is  lacking  due  to  resource  constraints  in  the  case  of  the  ERC,  Research 
Infrastructures and WIDERA. In relation to Research Infrastructures and WIDERA, there is 
no specific requirement to post vacancies to the platform. However, beneficiaries are free to 
use EURAXESS to raise awareness about their initiatives. In this regard, there is potential 
for  EURAXESS  to  increase  its  impact  by  enlarging  the  number  of  research  organisations 
publishing  vacancies  and,  in  particular,  by  increasing  the  number  of  non-academic 
institutions advertising their research vacancies on the EURAXESS platform. 

EURAXESS  Worldwide  is  an  all-in-one  networking  platform  and  information  tool  for 
talented  researchers  on  the  move  to  explore  and  develop  careers  in  the  EU.  The 
interview programme revealed the efforts of DG RTD to better coordinate the activities 
of  nine  worldwide  hubs  and  European  activities  (in  43  countries)  through  the 
establishment of EURAXESS Worldwide outreach liaison. This liaison is responsible for 
increasing  the  efficiency  of  communication  between  hubs,  following  up  on  commonly 
defined actions points, and enabling collaborating between major stakeholders and the 
European  Commission  under  various  programme  parts  and  through  the  international 
units  of  DG  RTD.  This  represents  a  shift  in  the  activities  of  EURAXESS,  given  its 

591 

 
previous focus on Pillar 1 activities. Current activities represent a broader mandate, with 
a  focus  on  spreading  information  about  collaborative  projects  under  different  Horizon 
Europe programme Missions too. A three-day EURAXESS Worldwide training event that 
took place in Brussels in March 2023, which involved with presentations by various EC 
services,  serves  as  an  example  of  the  emphasis  on  this  new  and  extended  role  of 
EURAXESS offices. 

Alumni associations 

Most notably, the Marie Curie Alumni Association (MCAA)1010 is an international non-profit 
organisation  established  and  supported  by  the  European  Commission,  but  entirely  run  by 
volunteer  members  and  with  a  bottom-up  approach  at  its  core.  The  MCAA  community 
connects  researchers  at  all  career  stages  from  all  scientific  disciplines,  encouraging 
networking and cooperation. According to interviewees, the association serves as a policy 
instrument  for science  diplomacy,  since its  representatives  (who  are located  in  regions  all 
around the world) participate in policy initiatives funded by DG EAC and DG RTD and other 
organisations. In this regard, the MCAA contributes to policy recommendations on topics that 
affect 
lives  of  researchers  (e.g.  Key 
Recommendations by the MCAA on the MSCA programme1011, published in 2023, discusses 
issues related to research assessment, the stability of early-career researchers and research 
management).  A  similar  entity  exists  in  the  context  of  the  ERC,  albeit  not  in  an  official 
capacity. 

the  MSCA  programme,  research,  and 

the 

Partner search tools and platforms 

The establishment of robust networks and relationships is essential to fostering productive 
international R&I collaborations. With regard to this need, the FP offers a range of partner 
search  tools  and  platforms  that  serve  as  repositories  in which  consortia  can  find  potential 
new partners. Although in the current digital landscape, there are many channels available 
via which suitable partners can be found to start research collaborations, the official support 
structures of the Horizon Europe programme continue to provide this type of support service, 
which  is  tailor-made  to  the  specific  needs  of  Horizon  Europe  programme  consortia.  More 
specifically, the Funding & tender opportunities portal offers the possibility to find partners 
and to post offers directly within the same portal used to access funding opportunities. It is 
important to note that the portal also serves a broader function as a key platform for finding 
information  regarding  the  funding  and  project  management  process.  In  addition,  the 
Enterprise Europe Network partner search tool manages an extensive database of business 
opportunities with an option to filter searches to find R&I partners. Most NCP networks also 
offer  partner  search  tools  to  their  communities.  In  relation  to  the  programme  parts  under 
analysis,  the  MSCA-NET  project  offers  a  dedicated  MSCA  Matchmaking  Platform.  The 
Research  Infrastructures  and  WIDERA  NCP  networks  currently  do  not  provide  such 
repositories,  and  the  ERC  does  not  have  an  NCP  network.  A  list  of  available  repositories 
supported by the Horizon Europe programme, together with a guide on how to best approach 
them, are outlined in the Horizon Europe Guide to an adequate partner search1012. 

1010 https://www.mariecuriealumni.eu/  
1011 Key Recommendations by the Marie Curie Alumni Association (MCAA) on the Marie 
Skłodowska-Curie Actions (MSCA) Programme, 2023. 
1012 Bridge2HE, Horizon Europe Guide to an adequate partner search, 2022. 

592 

 
 
Science Counsellors and EU delegations 

According  to  the  DG  RTD  Strategic  plan  2020-20241013,  the  DG  will  continue  to  carry  out 
targeted  outreach  activities  organised  by  Science  Counsellors  posted  in  EU  Delegations 
around  the  world  (currently  there  are  12).  Ruffini  (2023)1014  suggests  that  the  Science 
Counsellors  should  focus  on  two  key  priorities:  1)  promoting  the  EU’s  Framework 
Programmes,  and  2)  enhancing  coordination  with  Member  States’  science  attachés. 
According  to  the  EC officials  interviewed,  the  efficiency of  the  Science  Counsellors varies 
between  countries,  i.e.  according  to  the  willingness  of  the  country  to  participate  in  the 
Framework  Programme,  the  resources  dedicated  to  the  implementation  of  such  activities, 
and the portfolios of activities assigned to the counsellors.  

The role of Executive Agencies 

It is important to note that the entire portfolio of Excellent Science has been implemented by 
the  Executive  Agencies.  In  this  regard,  the  agencies  also  serve  as  information  sources 
regarding the application and evaluation process. More specifically, ERCEA communication 
and  outreach  activities  for  ERC  applicants  and  beneficiaries  are  based  upon  the  three 
objectives of the ERC’s external communication strategy: explaining why frontier research is 
vital  to  Europe’s  future,  attracting  the  best  ideas  and  the  brightest  minds,  and  sharing  a 
passion  for  frontier  science.  The  ERCEA  has  increasingly  diversified  its  communication 
activities and media tools and the channels it uses. These include the ERC’s website, social 
media, press releases and highlights, publications, the ERC’s newsletter Ideas, the European 
Commission’s intranet, and its various communication channels and networks1015. In the case 
of  REA,  the  agency’s  communication  and  outreach  activities  aimed  at  MSCA,  Research 
Infrastructures and WIDERA applicants and beneficiaries include: 1) the REA website; 2) a 
newsletter  for  REA  stakeholders  in  collaboration  with  the  operational  units;  3)  during 
missions, REA Project Officers promote calls whenever relevant; 4) Info Days – co-organised 
sessions on REA’s managed programme parts, together with the parent DGs’ relevant policy 
units (during the evaluation period, focus was given to the organisation of Horizon Europe 
Information Days); and 5) participation in the events of the NCPs1016. REA has also organised 
information,  training  and  networking  events  relating  to  various  programme  parts.  For 
example,  during  2021  and  2022,  the  Commission  organised  several  conferences  and 
workshops  dedicated  to  the  MSCA  and  contributed  to  leading  research  conferences.  The 
approaches applied by these agencies address the information needs of both EU-based and 
international applicants and beneficiaries – in other words, the information is not tailor-made 
for international participants. In addition, REA oversees the Research Enquiry Service, which 
serves as a contact point for future and current fellows, universities, research organisations, 
public institutions and businesses to answer all questions about participation in the European 
Union’s Research Framework Programmes. 

Dissemination activities 

Within the realm of international R&I collaborations, effective dissemination plays a pivotal 
role  not  only  in  conveying  essential  information,  but  also  in  showcasing  success  stories. 

1013 DG Research and Innovation Strategic Plan 2020-2024. 
1014 Ruffini, P.B., Science Counsellors of the European Union–a Case Study of Science 
Diplomacy, Diplomatica, 5(1), 95-120, 2023. 
1015 Study supporting the evaluation of CHAFEA, EACEA, EASME, ERCEA, INEA & REA 
(2017/2018-2021). 
1016 Ibid.  

593 

 
 
Examples  of  dissemination  tools  and  platforms  include  success  stories  on  international 
cooperation, the Horizon Dashboard, the Horizon Results platform and CORDIS. 

1.3.3.2. 

Attractiveness of the Framework Programme 

Application and subscription rates of different country groups in the Horizon Europe 
programme 

In assessing the attractiveness of the Framework Programme, we first explore its reach by 
analysing  application  rates.  Table  12  below  provides  an  overview  of  application  rates  for 
Member  States,  Horizon  Europe  associated  countries  and  non-associated  third  countries 
during  the  current  and  previous  Framework  Programmes.  Comparison  between  the  two 
programmes  shows  that  the  share  of  application  rates  across  key  Horizon  Europe 
programme country groups does not show any significant changes. However, Horizon 
Europe  attracted  fewer  applications  from  all  country  groups  during  its  first  two  years  of 
implementation,  which  may  be  explained  by  the  later-than-expected  launch  of  the 
programme.  

Horizon 2020 
(total) 

Horizon 2020 (first 
two years) 

Horizon Europe 
(total) 

Horizon Europe 
first two years 

Applica- 
tions 

Share 

Applica- 
tions 

Share 

Applica- 
tions 

Share 

Applica- 
tions 

Share 

Overall 

1,001,887  100%  282,041 

100%  289,821  100% 

237,795 

100% 

Member States 
(without UK) 

781,017  77.95%  218,211  77.37%  239,545  82.65%  195,251  82.11% 

United Kingdom 

66,455 

6.63% 

34,983 

12.40%  18,156  6.26% 

16,029 

6.74% 

Third countries 
(without UK and 
Switzerland) 

37,590 

3.75% 

9,346 

3.31% 

10,132  3.50% 

7,966 

3.35% 

Switzerland 

26,480 

2.64% 

5,782 

2.05% 

6,133 

2.12% 

4,940 

2.08% 

HE associated 
countries (without 
Switzerland) 

56,540 

5.64% 

13,681 

4.85% 

16,835  5.81% 

13,609 

5.72% 

Table 12. Application rates under Horizon 2020 and Horizon Europe and during the first two years of each programme –
Horizon 2020 (2014, 2015) and Horizon Europe (2021, 2022). Source: Horizon Dashboard, accessed 21 October 2023. 

After analysing application rates in the reference programmes in greater detail, however, 
some differences in application rates between groups can be identified. Based on the 
shares  of  eligible  applications  for  associated  countries,  the  most  attractive  part  was 
WIDERA (11%), followed by INFRA (5.5%); for non-associated third countries, the most 
attractive part was the MSCA (7%), followed by INFRA (3.5%).  

594 

 
 
 
The study team also compared application rates for the programme parts under analysis 
under Horizon 2020 and Horizon Europe programme (a detailed overview is provided in 
Appendix  5).  Overall,  while  application  rates  for  associated  countries  and  non-
associated third countries in most programmes either remained stable or witnessed a 
slight increase from Horizon 2020 to Horizon Europe, significant changes were observed 
in the application rates from the United Kingdom and Switzerland: 

•  For  the  MSCA,  the  application  rates  for  associated  countries  to  the  MSCA  remained 
stable (Horizon 2020, 4.2%; Horizon Europe, 4.3%). The same applies to Switzerland 
(Horizon  2020,  7.6%;  Horizon  Europe,  7.0%).  Application  rates  for  third  countries 
decreased  slightly  (Horizon  2020,  7.6%;  Horizon  Europe,  7.0%).  However,  the  most 
notable change can be seen in the case of the United Kingdom (Horizon 2020, 14.4%; 
Horizon Europe, 10.2%).  
In the context of the ERC, the share of applications increased from both Horizon Europe 
associated countries (Horizon 2020, 6.3%; Horizon Europe, 7.6%) and non-associated 
third countries (Horizon 2020, 1.2%; Horizon Europe, 1.6%). Again, for both the United 
Kingdom (Horizon 2020, 17.5%; Horizon Europe, 11.4%) and Switzerland (Horizon 2020, 
4%; Horizon Europe, 2%), application rates decreased significantly.  

• 

•  For  Research  Infrastructures,  the  application  rates  from  associated  countries,  non-
associated third countries and from Switzerland remained stable. However, application 
rates  from  the  United  Kingdom  dropped  significantly  (Horizon  2020,  9.4%;  Horizon 
Europe, 6.5%). 
Lastly, WIDERA saw a surge of applications from associated countries in comparison to 
those  seen  for  Science  with  and  for  Society  (SwafS)  and  the  Widening  participation 
programmes under Horizon 2020 (Horizon 2020, 7.8%; Horizon Europe, 11.0%). In terms 
of applications from non-associated third countries, Switzerland and the United Kingdom 
a decrease can be noted.  

• 

Table  13  shows  that  applicants  from  associated  countries  who  apply  for  Horizon  Europe 
programme funding are more successful than the programme average in the ERC (17.3%) 
and INFRA (59.8%). However, these applicants represent a fairly narrow group of countries 
with strong R&I performance (Israel and Norway). In the case of the MSCA, success rates 
for  third  countries  are  greater  than  the  programme  average  in  the  MSCA  (20.7%)  and 
WIDERA  (43.2%).  Under  Research  Infrastructures,  success  rates  are  distributed  evenly 
across country groups. In the case of WIDERA, success rates from associated countries are 
significantly  lower  (24.5%)  than  among  other  country  groups.  Given  the  large  number  of 
associated  countries  in  the  Widening  countries  group,  this  highlights  an  area  for 
improvement. 

Table 13. Success rates of different country groups under the reference Horizon Europe programme parts. Source: Horizon 
Dashboard, accessed 21 October 2023. Note: Third countries in this Figure include UK and Switzerland. 

595 

MSCAERCINFRAWIDERAHE totalOverall success rate16,6%14,6%55,7%32,0%16,9%Oversubscription rate74,0%75,0%34,0%53,0%71,0%Member States16,6%14,0%55,7%32,0%17,0%United Kingdom17,3%16,8%59,7%35,1%19,6%Third Countries20,7%12,0%52,6%43,2%20,7%Switzerland18,1%14,9%55,3%29,0%23,1%Associated Countries16,2%17,3%59,8%24,5%18,9% 
 
 
Lastly, it is important to note that heavy oversubscription is one of the biggest challenges in 
ensuring the efficient implementation of the Framework Programme despite the high rates of 
submissions by different country groups.  

Efficiency of the application and administrative processes under Horizon Europe 

The  evaluation  of  efficiency  also  explores  the  attractiveness  of  the  FP  as  perceived  by 
programme  participants.  In  this  regard,  the  two  surveys  of  Horizon  Europe  programme 
participants conducted during this study (covering all pillars of the programme) serve as the 
key  points  of  information.  Specifically,  the  survey  of  Horizon  Europe  beneficiaries  was 
conducted  in  May-July  2023,  and  received  responses  from  2,463  successful  applicants 
coming  from  EU  Member  States  (MS);  865  from  associated  countries;  and  78  from  non-
associated  third  countries,  representing  all  pillars  of  the  Framework  Programme1017.  The 
survey  of  unsuccessful  Horizon  Europe  applicants  was  conducted  in  May-July  2023,  and 
received responses from 6,055 respondents from EU Member States; 865 from associated 
countries; and 254 from non-associated third countries1018. To assess the attractiveness of 
the  FP,  the  study  team  looked  at  responses  to  questions  relating  to  the  experiences  of 
international programme participants in the project application process and in administrative 
and management processes: 

•  Successful applicants from associated and non-associated third countries demonstrate 
higher  levels  of  satisfaction  with  the  Horizon  Europe  programme  application 
process (see Table 19 in Appendix 2 for more details), especially when it comes to clarity 
of the aims and objectives of calls (68% of respondents from Member States said they 
were satisfied to a ‘large’ or ‘very large’ extent; the corresponding figures for ACs and 
TCs  were  71%  and  78%,  respectively).  Similar  findings  were  made  in  relation  to 
satisfaction with the design of the proposal template (MS, 60%; AC, 62%; TC, 72%), the 
transparency of evaluation process (MS, 53%; AC, 61%, TC, 71%) and the level of detail 
in the feedback provided (MS, 53%; AC, 59%; TC, 65%). The survey also evaluated the 
satisfaction of beneficiaries with the changes to the application process introduced under 
Horizon Europe (see Table 20 in Appendix 2). The majority of respondents in all three 
country groups were satisfied to a ‘large’ or ‘very large’ extent with the changes in relation 
to the clarity of the eligibility and funding rules for partners in associated countries and 
non-associated third countries (MS, 50%; AC, 51%; TC, 62%).  

• 

In  addition,  successful  applicants  from  non-associated  third  countries  and  associated 
countries perceived the efforts required to prepare the Horizon Europe programme 
proposal  more  favourably  than  those  representing  Member  States  (see  Table  22  in 
Appendix  2).  Among  those  beneficiaries  representing  non-associated  third  countries, 
67% agreed to a ‘large’ or ‘very large’ extent that the overall effort required to prepare 
the proposal were acceptable. The corresponding figure for respondents from associated 
countries was 58%, while from Member States it was just 48%. The same question was 
asked  in  the  survey  of  unsuccessful  applicants  (see  Table  32  in  Appendix  3).  In  this 
survey,  satisfaction  with  the  efforts  invested  was  much  lower  in  comparison  to  that 
expressed by successful applicants (30% of MS, 37% of AC and 48% of TC agreed to a 
‘large’ or ‘very large’ extent that the overall effort required to prepare a Horizon Europe 
programme proposal was acceptable). 

• 

In terms of the support received during the preparation of the project application, 
beneficiaries in all three country groups mostly indicated that they received no support 
(see  Table  24  in  Appendix  2).  Those  who  indicated  receiving  some  support  did  so 
through a dedicated department in their organisation (MS, 7%; AC, 7%;TC, 1%); from 

1017 A detailed overview of survey results per country group is provided in Appendix 3. 
1018 A detailed overview of survey results per country group is provided in Appendix 4. 

596 

 
 
National Contact Points (MS, 5%; AC, 6%; TC, 0%); from a consultancy firm/expert (MS, 
4%;  AC,  3%;  TC,  1%)  or  that  they  had  provided  such  support  to  the  consortium 
themselves (MS, 4%; AC, 3%; TC, 0%). The same question was asked in the survey of 
unsuccessful  applicants,  with  a  very  similar  level  of  negative  answers  in  terms  of  the 
support received (see Table 34 in Appendix 3).  

•  The  successful  applicants  surveyed  who  represented  associated  countries  and  non-
associated  third  countries  also  demonstrated  higher  levels  of  satisfaction  with  the 
Horizon  Europe  programme’s  administrative  and  management  processes  (see 
Table  21  in  Appendix  2).  More  specifically,  satisfaction  was  significantly  higher  with 
regard to project reporting requirements (MS, 49%; AC, 51%; TC, 67%); support in the 
event of technical issues (MS, 34%; AC, 39%; TC, 55%); the flexibility of the EC with 
regard to adapting the project’s objectives due to changed circumstances (MS, 47%; AC, 
50%;  TC,  60%);  and  the  quality  of  the  guidance  documents  relating  to  project 
implementation (MS, 51%; AC, 52%; TC, 63%).  

In  conclusion,  respondents  representing  associated  countries  and  non-associated  third 
countries were more satisfied with both the application process and the administrative and 
project  management  processes  compared  with  those  from  Member  States.  These 
perspectives  suggest  that  Horizon  Europe  programme  is  viewed  as  an  attractive  funding 
option,  characterised  by  streamlined  processes  and  potentially  reduced  administrative 
complexities from the standpoint of non-EU countries.  

Difficulties  in  including  partners  located  in  associated  countries  and/or  non-
associated third countries  

The survey also provided insights into the difficulties Horizon Europe beneficiaries faced in 
including  partners  located  in  associated  countries  or  non-associated  third  countries  in 
collaborative projects. Beneficiaries provided their responses in the form of freeform answers. 
Table 14 below provides an overview of the key issues highlighted in these answers. 

Key issues 

MSCA  
N=135 

INFRA 
N=45 

WIDERA 
N=47 

Description 

Status change of 
international 
partners 

39 

20 

8 

Difficulties in 
interpreting the 
Horizon Europe 
programme rules 

6 

7 

1 

The majority of the responses highlighted 
challenges and uncertainties posed by a 
change in the status of international 
partners. The examples provided mostly 
related to the inclusion of UK-based 
partners post-Brexit. More specifically, the 
respondents indicated a lack of guidelines 
as to how to navigate the uncertainty. 

Respondents identified a lack of knowledge 
on the international partner’s side with 
regard to funding and project rules and 
Horizon Europe programme reporting 
requirements. This created difficulties for 
consortium partners in the EU Member 
States, as they had to lead the non-EU 
beneficiaries through complex systems of 
reporting, online tools and other important 
administrative processes. 

597 

 
Key issues 

MSCA  
N=135 

INFRA 
N=45 

WIDERA 
N=47 

Description 

Difficulties in 
understanding 
the role of AC or 
TC partners in 
call texts 

4 

1 

0 

Other 

10 

3 

1 

Respondents highlighted a lack of clarity in 
Horizon Europe programme rules 
regarding how (and if it is possible) to 
include international partners in the 
proposal. Specifically, respondents 
indicated problems in understanding 
funding aspects (if and how much the 
international partners can be funded by the 
Commission), which calls are open to 
international participants, and how this 
needs to be justified in the proposal.  

Respondents highlighted other, more 
general issues such as miscommunication 
with international partners during the 
process of implementing the projects. 

No difficulties/ 
Not applicable 

75 

14 

38 

These respondents reported that they did 
not face any major difficulties. 

Table 14. Difficulties in including partners located in associated or third countries 

The freeform answers provided by respondents to the survey echo the key themes noted in 
the position papers submitted together with responses to the ‘Public consultation on the past, 
present  and  future  of  the  European  R&I  FP  2014-2027’1019.  Although  in  most  cases  these 
responses  highlighted  the  relevance  of  international  R&I  activities  to  the  programme  in 
general (e.g. international cooperation helps to address global challenges and needs to be 
further supported), a few more specific improvements were highlighted. These were: 1) the 
association of the United Kingdom and Switzerland; 2) addressing practical issues that arise 
as  a  result  of  the  variation  in  statuses  of  third  countries  (e.g.  countries  with  transitional 
arrangements,  countries 
in  negotiations),  such  as  budgetary  and  administrative 
consequences when the country’s status is changed; 3) complexities in building synergies 
with international initiatives (e.g. the Belmont Forum) under Horizon Europe compared with 
Horizon 2020; 4) the late communication of the work programmes as an obstacle to the better 
integration  of  non-EU  partners;  and  5)  the  alignment  of  EU  priorities  with  cross-border 
priorities such as the SDGs as a way to provide more opportunities under future Framework 
Programmes for researchers, innovators, societal actors and governments to collaborate. 

1.3.4.  EU added value 

Evaluation questions 

EAV2: What is the added value of the EU Framework Programme in addressing cross-cutting or 
transversal issues such as responsible research and innovation, co-design and co-creation, gender 
equality, social sciences and humanities, and international cooperation? 

1019 Public consultation on the past, present and future of the European R&I FPs 2014-2027. 

598 

 
 
To assess the added value of the Horizon Europe programme, the survey results were used 
as a key source of information. Successful applicants from the Member States, associated 
countries and non-associated third countries who responded to the survey agreed that the 
Horizon  Europe  programme  offers  multiple  benefits  in  comparison  to  the  national 
and/or  regional  level  (Table  15).  In  the  case  of  successful  applicants  from  EU  Member 
States, the most notable benefits include greater diversity in partners’ profiles (73%), greater 
international mobility opportunities for researchers (70%), and that the FP helps to better pool 
the skills and expertise necessary for research (58%), among other benefits. This suggests 
that  the  FP’s  openness  to  third  countries  allows  EU  participants  to  build  connections  and 
collaborate  with  the  best minds  in  the  world,  which is  a  necessary condition  for delivering 
excellent  science  research  in  complex  research  areas  (e.g.  artificial  intelligence,  quantum 
computing  and  clean  energy  technologies).  For  successful  applicant  from  associated 
countries,  the  most  notable  benefits  also  include  diversity  in  partner  profiles  (70%  of 
respondents  ‘agreed’  or  ‘strongly  agreed’),  international  mobility  opportunities  (67%),  and 
help in better pooling the skills and expertise necessary for research (57%). However, 59% 
of  respondents  agreed  that  the  level  of  competition  in  the  Horizon  Europe  programme  is 
higher than that at national and/or regional level.  

The top benefits identified by respondents from non-associated third countries were similar 
to those identified by beneficiaries from associated countries. However, the majority of non-
associated  third  country  respondents  also  stressed  Horizon  Europe’s  ability  to  provide 
funding  for  topics  not  covered  by  national/regional  programmes  (69%),  and  that  it  better 
addressed  grand  societal  challenges  (68%),  paid  attention  to  the  human  aspects  of 
innovation (65%) and provided access to research infrastructures (63%). As the majority of 
respondents  in  this  group  represent  low-  to  middle-income  third  countries,  these  findings 
suggest that Horizon Europe offers a valuable avenue for researchers from these countries 
to access programme funding, resources, knowledge and opportunities that might be limited 
or unavailable in their domestic settings. This underlines the importance of the FP in bridging 
the  research  and  innovation  gap,  promoting  inclusive  growth  and  fostering  collaborative 
efforts that benefit not just the European region but the global R&I landscape. 

Benefits offered by Horizon Europe 

Associated 
countries 
N=322 

Member 
States 
N=2463 

Third 
countries 
N=78 

Involves a higher level of competition for research 
funding 

59% 

67% 

73% 

Provides more diversity in the partners’ profiles 
(multi-sectoral, multi-disciplinary consortia, more 
geographical diversity of partners, etc.) 

70% 

73% 

79% 

Provides more international mobility opportunities for 
researchers  

67% 

70% 

74% 

Provides more opportunities for the 
commercialisation of research results, e.g. the 
applicability of results to new markets not present in 
the national environment  

37% 

41% 

47% 

Helps to better pool the skills and expertise 
necessary for research  

57% 

58% 

47% 

599 

 
Benefits offered by Horizon Europe 

Associated 
countries 
N=322 

Member 
States 
N=2463 

Third 
countries 
N=78 

Gives more attention to the human aspects of 
innovation (safety, ethics, user acceptance, etc.) 

40% 

41% 

65% 

Gives more attention for the environmental 
sustainability aspects of innovation 

38% 

41% 

54% 

Helps to achieve economies of scale in research, 
achieving the same result at lower costs 

37% 

35% 

49% 

Helps to better address grand societal challenges 
(e.g. climate change, demographics and ageing, 
etc.) 

52% 

52% 

68% 

Provides more access to research infrastructures 
(equipment and/or databases) 

52% 

51% 

63% 

Provides funding for research topic(s) or field(s) not 
covered by national or regional R&I funding 
programmes 

53% 

62% 

69% 

Gives more flexibility to respond to technological 
developments 

40% 

39% 

59% 

No additional benefits compared with national 
regional support 

12% 

10% 

36% 

Table 15. Benefits of Horizon Europe compared with programmes at national and/or regional level. Note: share of answers 
provided represent respondents who ‘Agreed’ and ‘Strongly agreed’ with the statements in the table above. A detailed 
overview of the data is provided in Appendix 2 of this case study. 

In  addition,  analysis  of  the  survey  of  Horizon  Europe  beneficiaries  aimed  to  understand 
whether  the  Horizon  Europe  programme  provides  sufficient  funding  opportunities  for 
research in collaboration with international partners (non-EU countries). As shown in the 
table  below,  beneficiaries  representing  both  Member  States  and  associated  countries 
expressed  a  more  reserved  view  than  participants  from  non-associated  third  countries. 
However, it is important to note that the beneficiaries from the non-associated third countries 
are  already  participating  in  a  Horizon  Europe-funded  project,  which  might  may  lead  to  a 
subjective  evaluation.  Notably,  the  relatively  reserved views  of  beneficiaries  from  Member 
States  might  potentially  signify  the  need  for  enhanced  awareness  of  the  rationale  and 
mechanisms  for  involving  international  partners  from  third  countries  into  Horizon  Europe 
project consortia. 

600 

 
Opportunities for 
collaboration with non-
EU countries 

To a very large extent 

To a large extent 

To a moderate extent 

To a small extent 

Not at all 

Do not know/Not 
Applicable 

Not answered 

Associated 
countries 

Member States 

Non-associated 
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

62 

76 

65 

38 

8 

38 

35 

19% 

205 

8% 

24% 

551 

22% 

20% 

629 

26% 

12% 

314 

13% 

2% 

64 

3% 

12% 

464 

19% 

11% 

236 

10% 

29 

24 

8 

3 

1 

7 

6 

37% 

31% 

10% 

4% 

1% 

9% 

8% 

Total 

322 

100% 

2,463 

100% 

78 

100% 

Table 16. Responses to the question ‘Extent to which the Horizon Europe programme provides sufficient funding 
opportunities in the research in collaboration with international partners’ by respondents from MS, TC and AC. 

1.4.  Success factors and perceived barriers 

This section of the case study focuses on the main factors and key determinants that drive 
the  international  dimension  of  the  Horizon  Europe  programme.  This  analysis  is  based  on 
evidence  gathered  via  most  of  the  analytical  tools  employed  in  this  report,  but  draws 
especially on interview data.  

1.4.1.  Success factors 

Association  Agreements.  According  to  the  EC  officials  interviewed  and  the  analysis  of 
participation trends across Framework Programmes, association automatically increases the 
participation  of  a  country  (e.g.  recent  association  of  Tunisia  resulted  in  increased 
participations  across  various  parts  of  the  programme,  while  the  non-association  of 
Switzerland to Horizon Europe has conversely decreased its number of participations). This 
relationship  is  influenced  by  several  factors.  First,  NCPs  in  all  associated  countries  are 
included in network meetings (e.g. WIDERA). This increases awareness among the NCPs 
as  to  how  to  participate  and  read  the  text  of  the  call  and  allows  them  to  disseminate  this 
information among the national research community. NCPs are also included in training as 
part  of  NCP  networks.  Second,  officials  from  associated  countries  can  also  attend 
programme committees as observers. This allows them to gain advance information about 
the work programme. In the case of Azerbaijan (which is not associated to Horizon Europe), 
a  decrease  in  participation  can  be  seen  compared  with  previous  programmes  and  in 
comparison, to smaller countries in the region (e.g. Georgia). Third, research entities from 
associated countries have much wider access to calls and actions under the FP, compared 

601 

 
with non-associated third countries (i.e. they also have access to mono-beneficiary actions 
under  Pillar  1  and  Pillar  3).  Lastly,  research  entities  from  associated  countries  are 
automatically  eligible  for  programme  funding,  which  is  not  the  case  for  entities  from  high-
income third countries.   

Creation  of  networking  opportunities  for  third  country  researchers.  A  number  of 
interviewees  (both  those  representing  the  position  of  the  EC  and  external  stakeholders) 
underlined the importance of building networks with research institutions from EU Member 
States as a way to foster international cooperation. As noted in the section on effectiveness, 
networking is especially relevant for countries with lower R&I performance and limited links 
with partners in EU Member States. In this regard, a number of modalities provide networking 
opportunities  for  researchers  at  different  stages  of  their  career  (e.g.  ERC  Implementing 
Arrangements,  COST  Actions),  as  well  as  young  people  from  third  countries  (e.g.  the 
Erasmus+ programme). 

Tools developed to address the fragmentation of international cooperation under Pillar 
1.  The  EC  has  initiated  a  number  of  new  mechanisms  (most  notably  the  MSCAdvocacy 
initiative) which aim to address fragmentation in cooperation with international partners. The 
effects  of  these  efforts  are  yet  to  be  seen;  however,  these  projects  will  provide  valuable 
insights into the national contexts of non-associated third countries and serve as an important 
stepping  stone  towards  improved  R&I  collaborations  (in  terms  of  both  the  extent  of 
participation and the potential to increase scientific performance). 

1.4.2.  Perceived barriers 

The  National  Contact  Points  are  designed  to  serve  as  important  links  between  EU 
Framework Programmes and local research communities. Even though their operations and 
effectiveness vary from country to country, NCPs increase awareness of the possibilities of 
the  Horizon  Europe  programme  and  provide  necessary  information  about  preparing 
proposals  and  finding  partners.  However,  the  surveys  of  unsuccessful  applicants  and 
beneficiaries indicate that respondents to a large extent did not take advantage of the support 
provided by NCPs. In this regard, the work of NCP networks and Communication and Support 
actions (such as MSCAdvocacy) play a crucial role in sharing best practices among the NCP 
community  and  tailoring  communication  strategies  to  the  varied  national  contexts  of  third 
countries. With regard to this, interviewees noted the lack of any coordinated NCP network 
for the ERC programme.  

Ambiguities  relating  to  eligibility  conditions/participation  rules.  Clear  eligibility 
conditions and participation rules for the Horizon Europe programme significantly influence 
the  participation  of  third  countries.  When  rules  are  transparent  and  unambiguous, 
international participants can easily understand the prerequisites and expectations, removing 
any  hesitancy  that  may  be  caused  by  uncertainty.  This  clarity  not  only  simplifies  the 
application  process  but  also  fosters  a  sense  of  fairness,  ensuring  that  all  participants  feel 
they  have  an  equal  opportunity.  Although  the  survey  results  demonstrate  high  levels  of 
satisfaction  with  the  Horizon  Europe  application  and  administrative  process  among 
applicants  representing  associated  countries  and  non-associated  third  countries,  the 
interviews,  freeform  answers,  and  the  results  of  the  public  consultation  point  to  difficulties 
being created by the changed status of large international partners (especially those from the 
United Kingdom). These findings are in line with the expert report on ‘How to make science 
diplomacy work’, which presented the views of international stakeholders on how EU should 
foster  international  cooperation  in  R&I.  These  experts  suggested  that  it  should  be  clearer 

602 

 
which  third  countries  have  co-funding  mechanisms  in  place,  and  how  these  work  in 
practice1020. 

international 
Limitations  of  Communication  and  Support  activities  aimed  at 
participants. Although programme participants from third countries perceive Horizon Europe 
favourably  in  terms  of  its  proposal  preparation  and  project  management  processes,  their 
awareness  of  the  funding  and  support  opportunities  provided  by  the  programme  remain 
limited.  Despite  numerous  tools  being  available  to  support  international  participation,  their 
effectiveness is hard to evaluate, given the complex range of efforts and the varying initiators 
of these tools (which extend beyond the vicinity of the European Commission). 

Lack of awareness among EU-based programme participants of the opportunities and 
benefits  of  third  country  participation  in  Horizon  Europe  projects.  According  to  the 
stakeholders interviewed, engagement with international partners is often perceived as being 
costlier  and  more  resource-intensive  (i.e.  involving  more  management).  However,  these 
perceptions  may  lead  to  the  exclusion  of  the  scientific  excellence  and  know-how  of 
international partners needed to solve complex societal challenges. In this regard, the survey 
of beneficiaries confirmed the limited awareness of the benefits of international cooperation 
as perceived by programme participants from EU Member States. Such perceptions indicate 
the  need  to  raise  awareness  of  the  rationale  and  mechanisms  for  involving  international 
partners from third countries within Horizon Europe project consortia. 

Impact is only realised in the longer term. Given the time it takes to realise the impact of 
international cooperation in projects and across the Framework Programme in general, it is 
difficult to demonstrate impact during the lifetime of the project, or even of the FP itself. At 
present, it is especially difficult with regard to the Horizon Europe programme, under which 
the current projects started only a short while ago. The identification of best practices and 
effective  approaches  would  be  facilitated  by  monitoring  approaches  that  can  gather 
information beyond the lifespan of projects in order to capture their long-term effects. 

1.5.  Conclusions  

Relevance. International cooperation remains a highly relevant issue in the context of the 
EU’s R&I ecosystem. In this regard, Horizon Europe continues to build upon the lessons 
learned under the Horizon 2020 programme. Important changes have been introduced into 
the Framework Programme as a result of the Global Approach to Research and Innovation, 
which  emphasises  the  need  for  the  EU  to  strengthen  multilateral  partnerships  in  order  to 
tackle global challenges. The Global Approach urges the EU to foster reciprocal cooperation 
with other nations based on mutual interest and principles of reciprocity, and highlights the 
importance  of  protecting  the  EU’s  strategic  assets.  It  has  brought  about  an  updated  and 
broadened policy for association to the Horizon Europe programme, which now allows the 
option of partial association for countries beyond the vicinity of the EU. It also provides for 
reciprocal access to programmes together with changes to the financial contribution model. 
The spectrum of activities related to international cooperation under Pillar 1 and WIDERA 
serve  the  needs  of  different  stakeholder  groups  and  ensure  that  international 
participants  at  different  stages  in  their  careers  can  participate  in  the  Framework 
Programme  and  contribute  to  delivering  excellent  science  and  reinforcing  Europe’s 
relative position. More specifically, the ERC, thanks to its production of excellent science, 
its  focus  on  frontier  research  and  its  broad  visibility  across  and  beyond  the  EU,  is  highly 
attractive to international participants from around the globe. The MSCA address the needs 
of researchers at different careers stages by providing training and research opportunities 

1020 https://sciencebusiness.net/sites/default/files/inline-files/ScienceBusiness%20 
%20International%20cooperation%20in%20Horizon%20Europe.pdf  

603 

 
 
that extend beyond national borders. In particular, MSCA Staff Exchanges actions serve as 
an  entry  point  for  many  associated  and  third  countries  to  participate  in  Horizon  Europe. 
Research  Infrastructures  actions,  meanwhile,  provide  a platform  for establishing  links  with 
relevant global communities and networks. WIDERA’s objectives and activities are the most 
relevant to building up the capacities of associated countries in the Widening group, which 
require additional support and capacity building to close the R&I performance gap.  Lastly, 
the results of the beneficiaries’ survey indicate that all of the programme parts analysed 
provide  significant  opportunities  to  increase  the  international  visibility/outreach  of 
research, further establishing its relevance. 

Effectiveness. The findings of the present case study suggest that the association of 
third countries to the Horizon Europe programme has strengthened the coherence 
and  integration  of  the  European  R&I  system.  Association  helps  to  advance  R&I 
capacities,  especially  in  associated  countries  with  less  developed  R&I  systems,  and 
prepares  them  for  alignment  with  the  European  Research  Area  (and,  in  the  case  of 
Enlargement  Countries,  their  alignment  with  the  EU  acquis  prior  to  accession). 
Significant  examples  include  Serbia  and  Moldova,  whose  experiences  underline  the 
transformative  impact  FP  instruments  can  have  on  capacity  building  and  research 
frameworks.  Countries  such  as  Ukraine  and  Tunisia  have  seen  heightened  shares  of 
applications  and  participations  in  the  Horizon  programme,  thereby  demonstrating  the 
beneficial effects of association. Moreover, this case study suggests that participation in 
the MSCA and WIDERA enhances the competitiveness of beneficiaries from Widening 
associated countries, as well as their capabilities for securing external funding. Because 
certain  associated  countries  still  face  challenges  in  obtaining  Framework  Programme 
funding  due  to  their  weak  R&I  systems,  difficulties  in  finding  partners  and  language 
barriers, the European Commission has introduced a set of support measures to close 
the outstanding gaps. These include the Policy Support Facility, as well as contributions 
to  the  entry  tickets  to  Horizon  Europe,  and  support  for  the  establishment  of  Horizon 
Europe national offices. While these initiatives promote international collaboration and 
help to build necessary capacities, robust data collection and evaluation approaches are 
needed in order to understand their effects. The present analysis also underlines that 
the  success  of  a  country’s  participation  in  the  programme  is  intrinsically  linked  to  its 
national support and context, and that support mechanisms cannot replace the need 
for  proactivity  and  adequate  R&D  funding  at  national  level.  Norway  and  Iceland 
exemplify  this  with  their  robust  performance,  driven  by  strategic  structures,  national 
commitments  and  active  participation  in  the  governance  structures  of  the  Framework 
Programmes. Their achievements not only set a benchmark but also position them as 
role  models  for  current  and  potential  associated  countries.  Lastly,  many  associated 
countries  perceive  the  Framework  Programmes  as  natural  extensions  of  their 
national  R&I  systems  –  a  testament  to  the  profound  influence  and  integration  of 
European research initiatives within their domestic landscapes. 

In  relation  to  Horizon  Europe's  overarching  objective  of  fostering  scientific  excellence, 
international  cooperation  and  association  has  brought  a  number  of  benefits.  First, 
analysis  of  international  co-publication  data  for  Horizon  2020  shows  that  authors 
representing associated countries and third countries both contributed significantly 
to the pool of FP-related publications (more than 15% and 29%, respectively). For most 
associated countries, there is a close alignment between their percentage of publications and 
their  participation  rate  in  Horizon  2020.  In  contrast,  non-associated  third  countries  display 
significant divergences between their shares of co-publications and their participation rates 
in  the  Framework  Programme.  These  variations  point  to  the  expansive  global  research 
influence of certain non-associated third countries, even if their integration into Horizon 2020 

604 

 
projects is not as profound. With regard to this, it is important to note that when researchers 
from Europe relocate or collaborate with institutions abroad, they frequently maintain active 
academic  ties  with  their  home  countries,  which  eventually  leads  to  more  co-publications. 
Furthermore, the analysis of contributions to different fields of science by different country 
groups shows that the distribution of fields of science across the country groups is similar for 
both programming periods. However, the data also indicate that the international partners 
supplement  scientific  areas  in  which  EU  Member  States  may  be  lacking  (e.g.  third 
country participants in both Research Infrastructure and ERC actions contribute more to the 
field of ‘Engineering and technology’ under Horizon Europe programme compared with the 
thematic distribution of the Member States). In terms of strengthening the EU’s human capital 
base, analysis of researchers’ mobility patterns in the MSCA PF shows  that these actions 
allow international excellence to be brought back to Europe.  

The present case study has also examined the major changes that have had an impact on 
the  excellence  of  Horizon  Europe.  While  they  require  further  investigation,  the  modalities 
provided by its new  association approach  (i.e. partial association, the reciprocity clause) 
provide novel ways to attract excellence to the EU. While association with countries beyond 
the vicinity of the EU is not considered in the present study, as New Zealand is the first such 
country and its association is only very recent, the present study allows some elaborate on 
the  potential impacts  and  drawbacks  of  such  an  association  to  Pillar 1.  In  this  regard,  the 
study  finds  that  the  potential  ramifications  of  associating  third  countries  with  high  R&I 
capacities warrant careful evaluation – especially with regard to bottom-up programme parts 
such as the MSCA and ERC – since their inclusion could possibly result in projects that lack 
a  European  dimension  and  which  reinforce  the  already  dominant  position  of  international 
scientific  competitors.  Lastly,  the  up  until  recently  non-associated  status  of  United 
Kingdom1021  and  still  non-associated  Switzerland  has  posed  challenges  for  the 
Framework  Programme  in  terms  of  excellence.  Compared  with  the  Horizon  2020 
programme, the participation of these countries in Horizon Europe is lagging behind, and has 
introduced levels of uncertainty into previously established international collaboration efforts. 

Efficiency. While the Horizon Europe programme remains an attractive funding avenue 
for  international  participants,  several  challenges  inhibit  its  full  potential.  Despite  well-
intentioned  outreach  mechanisms,  a  unified  monitoring  and  evaluation  system  for 
outreach is notably absent, thus hindering any comprehensive evaluation of the efficacy of 
outreach. Although valuable data are available, these often fail to provide in-depth insights 
into the impacts of international participant outreach. Beneficiaries expressed difficulties in 
navigating the expansive pool of information available, deciphering changes to programme 
rules  and  understanding  Third-Country  participant  roles.  This  is  corroborated  by  concerns 
raised during the ‘Public consultation on the past, present, and future of the European R&I 
FP 2014-2027’ with regard to persistent ambiguities relating to the participation conditions of 
countries whose status has changed under the Horizon Europe programme (i.e. the United 
Kingdom and Switzerland). Lastly, there is a pressing need to amplify communication efforts 
that  highlight  the  merits  of  international  collaboration  to  EU-based  researchers  and 
institutions. Addressing these challenges will not only enhance the programme’s efficacy, but 
will also bolster the standing of Horizon Europe as a premier funding option, characterised 
by streamlined processes and reduced administrative complexities. 

EU  added  value.  The  Horizon  Europe  programme,  as  underlined  by  the  result  of  the 
surveys carried out for this study, offers distinct advantages over national and regional 
initiatives,  particularly  in  fostering  international  collaboration,  pooling  research 
expertise,  and  addressing  global  societal  challenges.  Respondents  from  EU  Member 

1021 At the time this Case study was prepared, UK was not yet associated with Horizon Europe. 

605 

 
 
States,  associated  countries  and  non-associated  third  countries  consistently  highlighted 
benefits such as the enhanced diversity of partners and increased mobility opportunities. For 
non-associated  third  countries,  especially  those  in  low-  to  middle-income  bracket,  the 
programme  provides  essential  access  to  resources  and  collaborative  avenues  that  would 
otherwise  be  limited  in  their  local  contexts.  This  solidifies  Horizon  Europe’s  pivotal  role  in 
advancing global research and bridging disparities in innovation. 

Appendix 1. Literature sources 

•  Commission Communication on the Global Approach to Research and Innovation, 
‘Europe's strategy for international cooperation in a changing world’: https://eur-
lex.europa.eu/legal-content/EN/TXT/?uri=COM:2021:252:FIN 

•  Balland, P.A., & Ravet, J., Dynamic Network Analysis of the EU R&I Framework 

Programme. European Commission report, 2018.  

•  Craciun, D., & Orosz, K. Benefits and costs of transnational collaborative partnerships 
in higher education, EENEE Analytical Report No.36 prepared for the European 
Commission, 2018.  

•  Farrell M., Kalpazidou Schmidt, E., Mourzelas, M., Warrington, B., & Wood, J., Ex-post 

Evaluation of International Cooperation Activities of the Seventh Framework 
Programme’s Capacities Programme, Report for the European Commission, 2015.  

•  European Commission, Interim Evaluation of Horizon 2020. Staff Working Document. 

SWD(2017), 2017. 

•  European Commission, From Horizon 2020 to Horizon Europe – Monitoring Flash #2, 

2018. 

•  Project success stories: stories of especially successful EU-funded research projects. 

•  European Commission, Interim evaluation of Horizon 2020, 2017; Staff Working 

Document; and its Annex 1 and Annex 2; also other parts of Horizon 2020 interim 
evaluation. 

•  Monitoring flashes on international cooperation and association aspects of the 
Framework Programme, monitoring report ‘Keeping our eyes on the horizon’. 

•  European Commission (2018), Impact assessment of Horizon Europe, Staff Working 

Document. 

•  Horizon Europe Strategic Plan for 2021-2024. 

•  European Commission (2020), Science, Research and Innovation Performance (SRIP) 

of the EU 2020. 

•  European Commission websites on research and innovation and its monitoring and 

evaluation. 

606 

 
• 

International comparisons: OECD’s reports and Main Science and Technology 
Indicators; the NSF’s annual Science & Engineering Indicators; Clarivate’s Research 
Fronts. 

•  European Commission, Monitoring flash series: Horizon 2020 monitoring report, 2020: 

https://op.europa.eu/en/web/eu-law-and-publications/publication-detail/-
/publication/f164fa95-fed4-11ea-b44f-01aa75ed71a1 

•  Enhancing and Focusing EU International Cooperation in Research and Innovation: A 

Strategic Approach’ https://eur-lex.europa.eu/legal-
content/EN/TXT/?qid=1580995561233&uri=CELEX:52012SC0258 

•  Report on the Implementation of the Strategy for International Cooperation in Research 

and Innovation, 2018.  
https://ec.europa.eu/info/sites/default/files/research_and_innovation/strategy_on_resea
rch_and_innovation/documents/ec_rtd_inco-roadmap-2018-progress-report.pdf  

• 

‘Enhancing and focusing EU international cooperation in research and innovation: A 
strategic approach, (COM(2012) 497): https://eur-lex.europa.eu/legal-
content/en/ALL/?uri=CELEX%3A52012DC0497   

607 

 
 
 
Appendix 2. Survey results: successful applicants 

Type of organisation 

Associated 
countries 

Member States 

Non-associated 
third countries 

Total 

Count  Share  Count  Share  Count  Share  Count  Share 

Civil society/NGO 

19 

6% 

102 

4% 

External expert/consultancy 
firm 

Incubator 

0 

0 

0% 

16 

1% 

0% 

7 

0% 

Large enterprise 

21 

7% 

158 

6% 

Other 

17 

5% 

195 

8% 

Private research centre 

13 

4% 

136 

6% 

Public research centre 

27 

8% 

405 

16% 

Research funding 
organisation/agency 

12 

4% 

40 

2% 

SME (small or medium-sized 
enterprise) 

38 

12% 

356 

14% 

Spin-off 

1 

0% 

2 

0% 

8 

0 

0 

0 

8 

5 

8 

2 

1 

0 

10% 

129 

5% 

0% 

16 

1% 

0% 

7 

0% 

0% 

179 

6% 

10% 

220 

8% 

6% 

154 

5% 

10% 

440 

15% 

3% 

54 

2% 

1% 

395 

14% 

0% 

3 

0% 

Start-up (not older than 5 
years,  at the time of 
proposal submission) 

University/higher education 
institution 

2 

1% 

49 

2% 

1 

1% 

52 

2% 

172 

53% 

997 

40% 

45 

58% 

1214 

42% 

Total 

322 

100% 

2463 

100% 

78 

100% 

2865 

100% 

Table 17. Reponses to the question ‘Please describe the organisation that you are representing’ 

608 

 
 
 
 
Country/territory 

Total 

Country/territory 

Total 

Italy 

Spain 

Germany 

France 

Belgium 

Greece 

Netherlands 

United Kingdom 

Portugal 

Austria 

Sweden 

Finland 

Denmark 

Ireland 

Poland 

Norway 

Serbia 

Romania 

Slovenia 

Bulgaria 

Albania 

Argentina 

Canada 

China (People’s 
Republic of) 

Cuba 

Ecuador 

Ethiopia 

Kosovo (UN 
resolution) 

Pakistan 

South Africa 

Tunisia 

Uganda 

Afghanistan 

Anguilla 

Armenia 

Australia 

Bangladesh 

Belarus 

Bosnia and 
Herzegovina 

Botswana 

347 

335 

328 

214 

152 

146 

140 

124 

106 

102 

68 

64 

59 

59 

55 

48 

45 

42 

34 

32 

609 

3 

3 

3 

2 

2 

2 

2 

2 

2 

2 

2 

2 

1 

1 

1 

1 

1 

1 

1 

1 

 
Country/territory 

Total 

Country/territory 

Total 

Turkey 

Czechia 

Croatia 

Cyprus 

Estonia 

Lithuania 

Latvia 

Ukraine 

Israel 

Switzerland 

Hungary 

Luxembourg 

Iceland 

Malta 

Slovakia 

Moldova (Republic of) 

Georgia 

North Macedonia 

Thailand 

United States 

Chile 

Colombia 

Egypt 

Eswatini 

Faroe Islands 

Ghana 

Japan 

Kazakhstan 

Kenya 

Kyrgyzstan 

Korea (Republic of) 

Malawi 

Montenegro 

Palestine 

Philippines 

Réunion 

Rwanda 

Tajikistan 

Tanzania (United 
Republic of) 

Uruguay 

32 

31 

27 

22 

22 

20 

18 

15 

14 

14 

12 

10 

9 

9 

9 

8 

6 

6 

5 

5 

610 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

 
Country/territory 

Total 

Country/territory 

Total 

Malaysia 

Morocco 

Nigeria 

Total 

4 

4 

4 

Vietnam 

Other, unnamed 
country 

1 

1 

2,863 

Table 18. Responses to the question ‘In which Horizon Europe country is the organisation that you represent located?’ 

Element of project 
application process 

Ease of identifying relevant 
funding opportunities across 
the different Horizon Europe 
programme parts 

Ease of identifying relevant 
partners for the consortium  

Clarity of the aims and 
objective of the calls 

Feasibility of addressing the 
objectives set out in the 
calls  

Size of possible project 
funding as foreseen in the 
call, compared with the 
objectives set in the call  

Clarity of the general 
conditions and eligibility and 
funding rules provided on 
the funding tenders portal  

Design of the proposal 
template 

Transparency of the 
evaluation process 

Associated 
countries 

Third countries, 
non-associated 

Member States 

Count 

Share 

Count 

Share 

Count 

Share 

171 

53% 

44 

56% 

1,234 

50% 

171 

53% 

45 

58% 

1,219 

49% 

230 

71% 

61 

78% 

1,667 

68% 

230 

71% 

53 

68% 

1,486 

60% 

204 

63% 

46 

59% 

1,387 

56% 

218 

68% 

47 

60% 

1,644 

67% 

199 

62% 

56 

72% 

1,485 

60% 

195 

61% 

55 

71% 

1,333 

54% 

611 

 
 
 
 
Element of project 
application process 

Level of detail in the 
feedback provided in the 
evaluation report 

Timeliness of the funding 
decisions 

Associated 
countries 

Third countries, 
non-associated 

Member States 

Count 

Share 

Count 

Share 

Count 

Share 

190 

59% 

51 

65% 

1,302 

53% 

229 

71% 

54 

69% 

1,678 

68% 

Table 19. Summary of responses to the question ‘To what extent are you satisfied with the following aspects related to your 
Horizon Europe project application process?’ (count includes those who responded ‘To a large extent’ or ‘To a very large 
extent’) 

Change introduced 

Associated 
countries 

Non-associated 
third countries 

Member States 

Count 

Share 

Count 

Share 

Count 

Share 

Clarity of the eligibility and funding 
rules for partners in associated and 
third countries  

Clarity of instructions for the 
requested ethical review  

Clarity of instructions for the 
requested data management plan  

Clarity of instructions for the 
requested gender equality plan  

Clarity of instructions for the 
development of impact statements 
(i.e. project impact pathways) 

Clarity of instructions relating to the 
principle of ‘do no significant harm’ 

163 

51% 

48 

62% 

1,229 

50% 

176 

55% 

49 

63% 

1,210 

49% 

169 

52% 

50 

64% 

1,193 

48% 

193 

60% 

53 

68% 

1,340 

54% 

171 

53% 

47 

60% 

1,095 

44% 

168 

52% 

50 

64% 

1,142 

46% 

Table 20. Summary of responses to the question ‘To what extent are you satisfied with the following changes in the 
application process introduced under Horizon Europe?’ (count includes those who responded ‘To a large extent’ or ‘To a 
very large extent’) 

612 

 
Administrative and 
management processes 

The time taken for the 
process up to the signature 
of the grant agreement is 
adequate 

The burden of the 
administrative and legal 
requirements for granting 
procedures was 
proportionate  

The EC is sufficiently 
flexible in adapting the 
project objectives due 
changed circumstances 

The EC is sufficiently 
flexible with respect to 
changes in the project 
consortium 

The online reporting 
platform is user friendly 

Project reporting 
requirements require 
reasonable effort and costs  

Support in the event of 
technical issues with the 
online platforms is useful  

Guidance documents 
related to the project 
implementation phase in the 
annotated grant agreement 
are sufficient and clear 

Expenditure eligibility 
requirements include clear 
financial guidelines 

Rules for the calculation of 
personnel costs (esp. daily 
rates) are adequate and 
clear 

Associated 
countries 

Non-associated 
third countries 

Member States 

Count 

Share 

Count 

Share 

Count 

Share 

207 

64% 

62 

79% 

1,598 

65% 

155 

48% 

45 

58% 

1,237 

50% 

161 

50% 

47 

60% 

1,154 

47% 

165 

51% 

41 

53% 

1,184 

48% 

148 

46% 

39 

50% 

1,027 

42% 

164 

51% 

52 

67% 

1,215 

49% 

126 

39% 

43 

55% 

843 

34% 

171 

53% 

49 

63% 

1,255 

51% 

187 

58% 

54 

69% 

1,436 

58% 

173 

54% 

48 

62% 

1,291 

52% 

Table 21. Summary overview of responses to the question ‘Would you agree or disagree with the following statements about 
the administrative and management processes in your Horizon Europe Project?’ (count includes those who responded 
‘Agree’ or ‘Strongly agree’) 

613 

 
Statement relating to the effort 
required to prepare and submit 
the Horizon Europe project 

The application costs and the 
total time and resources needed 
are proportionate to the volume of 
funding requested in the proposal 

The efforts needed were 
proportionate to the complexity of 
the proposed project 

The efforts needed were 
proportionate to the number of 
consortium partners involved 

The efforts needed were in 
proportion with the chances of 
securing Horizon Europe funding 

The overall effort to prepare a 
Horizon Europe proposal was 
acceptable  

Associated 
countries 

Non-associated 
third countries 

Member States 

Count 

Share 

Count 

Share 

Count 

Share 

178 

55% 

51 

65% 

1,194 

48% 

199 

62% 

47 

60% 

1,423 

58% 

209 

65% 

48 

62% 

1,336 

54% 

162 

50% 

46 

59% 

1,011 

41% 

188 

58% 

52 

67% 

1,188 

48% 

Table 22. Summary of responses to the question ‘To what extent do you agree with the following statements about the effort 
needed to prepare and submit your Horizon Europe project’ (count includes those who responded ‘To a large extent’ or ‘To a 
very large extent’) 

Estimated efforts 

Less than 5 person-
days 

6 to 15 person-days 

16 to 25 person-days 

26 to 35 person-days 

36 to 45 person-days 

46 to 55 person-days 

56 to 65 person-days 

Associated 
countries 

Member States 

Non-associated 
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

20 

70 

67 

64 

37 

27 

19 

6% 

181 

7% 

22% 

531 

22% 

21% 

504 

20% 

20% 

454 

18% 

11% 

286 

12% 

8% 

6% 

171 

7% 

109 

4% 

614 

11 

21 

15 

8 

5 

6 

4 

14% 

27% 

19% 

10% 

6% 

8% 

5% 

 
Estimated efforts 

More than 65 person-
days 

Associated 
countries 

Member States 

Non-associated 
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

18 

6% 

215 

9% 

8 

10% 

Total 

322 

100% 

2,463 

100% 

78 

100% 

Table 23. Responses to the question ‘In your estimation, how many person-days did your organisation spend in preparing 
your Horizon Europe proposal?’ 

Support measures 

Associated 
countries 

Member States 

Non-associated 
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

Yes, we received help from a 
dedicated department within 
our organisation 

Yes, we received support from 
the National Contact Point 
(NCP) 

Yes, our organisation received 
support from a consultancy 
firm/expert within or outside the 
consortium  

My organisation provided this 
type of support to the 
consortium 

22 

7% 

164 

7% 

20 

6% 

123 

5% 

1 

0 

1% 

0% 

17 

5% 

106 

4% 

1 

1% 

9 

3% 

103 

4% 

0 

0% 

Table 24. Responses to the question ‘Did you receive any support for your Horizon Europe project application preparation?’ 

Previous experience in 
Framework Programmes 

Associated 
countries 

Member States 

Non-associated 
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

Do not know/Not Applicable 

9 

3% 

62 

3% 

Yes, as a coordinator and as a 
participant 

29 

9% 

311 

13% 

3 

1 

4% 

1% 

615 

 
Previous experience in 
Framework Programmes 

Associated 
countries 

Member States 

Non-associated 
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

Yes, as a coordinator of a 
consortium 

16 

5% 

162 

7% 

0 

0% 

Yes, as a participant 

119 

37% 

1,050 

43% 

No 

147 

46% 

867 

35% 

Missing answers 

2 

1% 

11 

0% 

27 

46 

1 

35% 

59% 

1% 

Total 

322 

100% 

2463 

100% 

78 

100% 

Table 25. Responses to the question ‘Before your current Horizon Europe project have you personally 
participated/coordinated previous framework programme?’ 

Associated 
countries 

Member States 

Non-associated 
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

44 

14% 

346 

14% 

8 

10% 

20 

6% 

- 

- 

2 

3% 

Additional funding source 

National/regional R&D public 
funding 
schemes/programmes  

Financial support specifically 
made available by my country 
for successful applicants in 
Horizon Europe projects in 
the form of a co-funding 
arrangement (only applies to 
third country participants) 

Table 26. Responses to the question ‘Have you applied for any additional funding for the research idea activities addressed 
in your Horizon Europe project?’ 

616 

 
 
 
Decision regarding additional 
funding 

Associated 
countries 

Member States 

Non-associated 
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

Do not know / cannot answer 

9 

3% 

61 

2% 

Yes, additional funding has been 
granted 

42 

13% 

173 

7% 

Yes, but additional funding was not 
granted 

4 

1% 

31 

1% 

2 

9 

0 

3% 

12% 

0% 

Yes, some funds were granted, but 
only from some of the additional 
funding sources that we have 
applied for 

No, the funding decision(s) is/are still 
pending 

7 

2% 

74 

3% 

5 

6% 

19 

6% 

144 

6% 

4 

5% 

Missing answers 

241 

75% 

1,980 

80% 

58 

74% 

Total 

322 

100% 

2,463 

100% 

78 

100% 

Table 27. Responses to the question ‘Has your organisation already received a decision regarding this additional funding?’ 

Opportunities for 
collaboration with non-
EU countries 

To a very large extent 

To a large extent 

To a moderate extent 

To a small extent 

Not at all 

Do not know/Not 
Applicable 

Not answered 

Associated 
countries 

Member States 

Non-associated 
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

62 

76 

65 

38 

8 

38 

35 

19% 

205 

8% 

24% 

551 

22% 

20% 

629 

26% 

12% 

314 

13% 

2% 

64 

3% 

12% 

464 

19% 

11% 

236 

10% 

617 

29 

24 

8 

3 

1 

7 

6 

37% 

31% 

10% 

4% 

1% 

9% 

8% 

 
Opportunities for 
collaboration with non-
EU countries 

Associated 
countries 

Member States 

Non-associated 
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

Total 

322 

100% 

2,463 

100% 

78 

100% 

Table 28. Responses to the question ‘In your opinion, to what extent does Horizon Europe provide sufficient funding 
opportunities for research in collaboration with international partners (non-EU countries)?’ 

Aspect offered by Horizon Europe 
compared with research funding 
available elsewhere 

Associated 
countries 

Member States 

Non-associated 
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

Involves a higher level of competition 
for research funding 

189 

59% 

1,646 

67% 

57 

73% 

Provides more diversity in partner 
profiles (multi-sectoral, multi-
disciplinary consortia, greater 
geographical diversity of partners, 
etc.) 

Provides more international mobility 
opportunities for researchers  

Provides more opportunities for the 
commercialisation of research results 
(e.g. the applicability of results to new 
markets not present in the national 
environment)  

Helps to better pool the skills and 
expertise necessary for research  

Gives more attention for the human 
aspects of innovation (safety, ethics, 
user acceptance, etc.) 

Gives more attention for the 
environmental sustainability aspects 
of innovation 

Helps to achieve economies of scale 
in research, achieving the same result 
at lower costs 

Helps to better address grand societal 
challenges (e.g. climate change, 
demographics and ageing, etc.) 

225 

70% 

1,808 

73% 

62 

79% 

216 

67% 

1,724 

70% 

58 

74% 

118 

37% 

999 

41% 

37 

47% 

185 

57% 

1,439 

58% 

37 

47% 

128 

40% 

1,013 

41% 

51 

65% 

122 

38% 

1,020 

41% 

42 

54% 

118 

37% 

856 

35% 

38 

49% 

166 

52% 

1,283 

52% 

53 

68% 

618 

 
Aspect offered by Horizon Europe 
compared with research funding 
available elsewhere 

Associated 
countries 

Member States 

Non-associated 
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

Provides more access to research 
infrastructures (equipment and/or 
databases) 

Provides funding for research topic(s) 
or field(s) not covered by national or 
regional R&I funding programmes 

Gives greater flexibility to respond to 
technological developments 

No additional benefits compared with 
national/regional support 

167 

52% 

1,264 

51% 

49 

63% 

172 

53% 

1,531 

62% 

54 

69% 

130 

40% 

956 

39% 

46 

59% 

40 

12% 

254 

10% 

28 

36% 

Table 29. Summary of responses to the question ‘Compared with the research funding available to you at national and or 
regional level Horizon Europe…’ (count includes those who responded ‘Agree’ or ‘Strongly agree’) 

619 

 
 
Appendix 3. Survey results: unsuccessful applicants 

Type of organisation 

Associated 
countries 

Member States 

Non-associated 
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

Civil society/NGO 

40 

5% 

255 

4% 

32 

13% 

External expert/consultancy 
firm 

Incubator 

Large enterprise 

Private research centre 

Public research centre 

Research funding 
organisation/agency 

7 

4 

37 

42 

84 

1% 

58 

1% 

0% 

10 

0% 

4% 

301 

5% 

5% 

309 

5% 

0 

0 

2 

7 

0% 

0% 

1% 

3% 

10% 

992 

16% 

33 

13% 

4 

0% 

32 

1% 

4 

2% 

SME (Small or medium-sized 
enterprise) 

103 

12% 

835 

14% 

15 

6% 

Spin-off 

1 

0% 

25 

0% 

13 

2% 

134 

2% 

0 

2 

0% 

1% 

Start-up (not older than 5 years 
at the time of proposal 
submission) 

University/higher education 
institution 

490 

57% 

2,704 

45% 

142 

56% 

Other 

Total 

40 

5% 

400 

7% 

17 

7% 

865 

100% 

6,055 

100% 

254 

100% 

Table 30. Responses to the question ‘Please describe the organisation that you are representing’ 

620 

 
 
 
Country 

Total 

Country 

Total 

Afghanistan 

Albania 

Algeria 

Angola 

Argentina 

Armenia 

Aruba 

Australia 

1 

17 

2 

1 

6 

2 

1 

6 

Kosovo (UN 
resolution) 

Latvia 

Lebanon 

Lesotho 

Liberia 

Lithuania 

Luxembourg 

Madagascar 

Austria 

222 

Malaysia 

Azerbaijan 

Bangladesh 

Belarus 

Belgium 

Bosnia and 
Herzegovina 

Botswana 

Brazil 

Brunei Darussalam 

Bulgaria 

Burkina Faso 

Cameroon 

2 

1 

1 

257 

17 

1 

2 

1 

65 

1 

4 

Malawi 

Mali 

Malta 

Mexico 

Moldova (Republic 
of) 

Montenegro 

Morocco 

Mozambique 

Namibia 

Netherlands 

New Zealand 

621 

8 

34 

1 

1 

1 

32 

29 

3 

4 

5 

1 

17 

2 

8 

8 

6 

4 

4 

317 

1 

 
Country 

Total 

Country 

Total 

Denmark 

127 

Romania 

Canada 

Chile 

China (People’s 
Republic of) 

Cyprus 

Colombia 

Congo (Democratic 
Republic of) 

Croatia 

Czechia 

3 

1 

4 

50 

9 

4 

91 

93 

Dominican Republic 

Ecuador 

Egypt 

Estonia 

Eswatini 

Ethiopia 

Faroe Islands 

Finland 

France 

Gabon 

Georgia 

1 

1 

3 

42 

1 

9 

2 

139 

483 

2 

8 

Niger 

Nigeria 

North Macedonia 

1 

15 

25 

Norway 

129 

Pakistan 

Peru 

Poland 

Portugal 

Rwanda 

São Tomé and 
Príncipe 

Seychelles 

Senegal 

1 

1 

159 

295 

131 

3 

2 

1 

2 

Serbia 

100 

48 

75 

1 

9 

909 

1 

Slovakia 

Slovenia 

Somalia 

South Africa 

Spain 

Sri Lanka 

622 

 
Country 

Total 

Country 

Total 

Germany 

Ghana 

Gibraltar 

Greece 

Grenada 

Guinea-Bissau 

Hungary 

Iceland 

India 

Indonesia 

Ireland 

Israel 

Italy 

Jamaica 

Japan 

Jordan 

Kazakhstan 

760 

10 

1 

329 

1 

1 

72 

5 

1 

1 

117 

57 

969 

1 

6 

2 

1 

Sudan 

Sweden 

Switzerland 

Taiwan 

Tanzania (United 
Republic of) 

Thailand 

Tunisia 

Turkey 

Uganda 

Ukraine 

United Kingdom 

United States 

Uruguay 

Vietnam 

Zambia 

Zimbabwe 

Other country 

2 

193 

45 

1 

6 

3 

16 

113 

10 

32 

289 

16 

1 

2 

5 

2 

5 

Kenya 

22 

Total 

7,174 

Table 31. Responses to the question ‘In which country is the organisation that you represent located?’ 

623 

 
 
 
Statement relating to the effort 
required to prepare and submit the 
Horizon Europe project 

The application costs and the total 
time and resources needed are 
proportionate to the volume of funding 
requested in the proposal 

The efforts needed were proportionate 
to the complexity of the proposed 
project 

The efforts needed were proportionate 
to the number of consortium partners 
involved  

The efforts needed were in proportion 
with the chances of securing Horizon 
Europe funding  

The overall effort to prepare a Horizon 
Europe proposal was acceptable 

Associated 
countries 

Member States 

Non-
associated 
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

418 

48% 

2,778 

46% 

147 

58% 

501 

58% 

3,370 

56% 

159 

63% 

483 

56% 

3,209 

53% 

154 

61% 

251 

29% 

1,441 

24% 

112 

44% 

316 

37% 

1,829 

30% 

121 

48% 

Table 32. Summary of responses to the question ‘To what extent do you agree with the following statements about the effort 
needed to prepare and submit your Horizon Europe proposal?’ (count includes those who responded ‘To a large extent’ or 
‘To a very large extent’) 

Estimated efforts 

Associated 
countries 

Member States 

Non-associated  
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

Less than 5 person-days 

70 

8% 

633 

10% 

6 to 15 person-days 

207 

24% 

1,520 

25% 

16 to 25 person-days 

175 

20% 

1,251 

21% 

26 to 35 person-days 

134 

15% 

904 

15% 

36 to 45 person-days 

111 

13% 

646 

11% 

46 to 55 person-days 

56 to 65 person-days 

48 

46 

6% 

343 

6% 

5% 

261 

4% 

36 

75 

45 

33 

22 

15 

14 

14% 

30% 

18% 

13% 

9% 

6% 

6% 

624 

 
Estimated efforts 

Associated 
countries 

Member States 

Non-associated  
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

NA 

Total 

More than 65 person-days 

74 

9% 

495 

8% 

0 

0% 

2 

0% 

14 

0 

6% 

0% 

865 

100% 

6,055 

100% 

254 

100% 

Table 33. Responses to the question ‘In your estimation, how many person-days did your organisation spend in preparing 
your Horizon Europe proposal?’ 

Support for proposal 
application 

Yes, we received help from a 
dedicated department in our 
organisation 

Yes, we received support from 
the National Contact Point 
(NCP) 

Yes, our organisation received 
support from a consultancy 
firm/expert (within or outside 
the consortium) 

My organisation provided this 
type of support to the 
consortium 

No, we did not receive any 
support 

Associated 
countries 

Member States 

Non-associated 
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

51 

6% 

410 

7% 

38 

4% 

259 

4% 

1 

1 

0% 

0% 

44 

5% 

243 

4% 

3 

1% 

26 

3% 

220 

4% 

81 

9% 

426 

7% 

1 

8 

0% 

3% 

Table 34. Responses to the question ‘Did you receive any support for your Horizon Europe proposal application 
preparation?’ 

625 

 
Funding area 

To conduct basic fundamental 
research in new and emerging 
research areas 

To conduct applied research setting 
the basis for the development of new 
products, services or processes (e.g. 
pilots) 

Associated 
countries 

Member States 

Non-
associated 
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

368 

43% 

2,067 

34% 

107 

42% 

420 

49% 

2,799 

46% 

115 

45% 

To conduct interdisciplinary research  

497 

57% 

3,049 

50% 

160 

63% 

To conduct academia-industry 
collaborative research  

For the demonstration and large-scale 
validation of new or improved 
products, processes or services, 
including with end-users  

Grants for the development of 
research infrastructures 

461 

53% 

3,077 

51% 

120 

47% 

344 

40% 

2,313 

38% 

102 

40% 

225 

26% 

1,333 

22% 

69 

27% 

Research contributing to policy-making 

288 

33% 

1,635 

27% 

121 

48% 

Pre-normative research allowing the 
development of international standards  

199 

23% 

1,079 

18% 

88 

35% 

Research in collaboration with 
international partners (non-EU 
countries) 

Research into technological 
applications in areas that are important 
for Europe’s industrial competitiveness 

To pursue your individual or 
institutional research agenda 

To strengthen the knowledge base and 
the R&D capacity of your organisation 

To enhance the R&D and/or innovation 
capabilities and skills of your 
organisation  

301 

35% 

1,636 

27% 

131 

52% 

429 

50% 

2,749 

45% 

125 

49% 

388 

45% 

2,971 

49% 

106 

42% 

438 

51% 

2,775 

46% 

119 

47% 

420 

49% 

2,839 

47% 

115 

45% 

To enhance your competitiveness 

412 

48% 

2,815 

46% 

128 

50% 

626 

 
Funding area 

To enhance the international visibility 
across the borders of your country 

To create or strengthen collaboration 
with leading research organisations 

To strengthen existing markets or 
create/receive access to new markets 
and global value chains  

To improve or develop new tools, 
methods or techniques  

To improve or develop new products, 
services or processes  

To reduce the environmental impacts 
of your products, processes or 
services 

To develop sustainable solutions that 
contribute to the green transition  

Associated 
countries 

Member States 

Non-
associated 
third countries 

Count 

Share 

Count 

Share 

Count 

Share 

538 

62% 

3,633 

60% 

147 

58% 

335 

39% 

2,270 

37% 

108 

43% 

234 

27% 

1,333 

22% 

85 

33% 

394 

46% 

2,912 

48% 

132 

52% 

341 

39% 

2,368 

39% 

111 

44% 

242 

28% 

1,427 

24% 

89 

35% 

320 

37% 

2,154 

36% 

104 

41% 

To promote institutional change in your 
organisation, region or country 

228 

26% 

1,274 

21% 

86 

34% 

To strengthen the application of 
responsible research and innovation 
principles 

To support policy-making or the 
development of standards-setting 
measures 

289 

33% 

1,660 

27% 

118 

46% 

228 

26% 

1,339 

22% 

95 

37% 

Table 35. Summary of responses to the question ‘In your opinion, to what extent does Horizon Europe provide sufficient 
funding opportunities in the following areas…’ (count includes those who responded ‘To a large extent’ or ‘To a very large 
extent’) 

627 

 
 
 
Associated 
countries 

Member States 

Non-associated 
third countries 

Count 

Share 

Count 

Count 

Share 

Count 

376 

43% 

2,451 

40% 

94 

37% 

380 

44% 

2,340 

39% 

86 

34% 

338 

39% 

1,948 

32% 

93 

37% 

451 

52% 

2,986 

49% 

117 

46% 

405 

47% 

2,717 

45% 

107 

42% 

359 

42% 

2,306 

38% 

79 

31% 

Our project addressed EU-level and 
or global challenges 

Necessary knowledge, expertise and 
skills could not be accessed without 
Horizon Europe  

Necessary research infrastructure 
databases and other tools could not 
be accessed without Horizon Europe 

Our project required the involvement 
of a variety of international partners 
and could not be implemented 
without them 

There was no alternative funding 
available for that research topic area  

The amount of funding available from 
other sources was insufficient 

Table 36. Summary of responses to the question ‘How important were the following reasons that prevented your 
organisation from implementing the project without Horizon Europe funding’ (count includes those who responded 
‘Important’ or ‘Very important) 

628 

 
 
 
 
Annex 4. Results of the analysis of monitoring data 

Horizon 
Europe 
(all 
pillars) 

ERC 

StG 

CoG 

AdG 

Syg 

PoC 

Total 

54,911 

2,824 

991 

306 

526 

135 

460 

Member States 

46,408 

2,457 

878 

269 

473 

93 

392 

Associated 
countries 

2,925 

296 

104 

19 

44 

Third countries 

1,891 

Switzerland 

1,250 

United Kingdom  

2,437 

24 

13 

34 

5 

1 

3 

5 

3 

10 

1 

2 

6 

13 

12 

4 

13 

64 

1 

2 

1 

Table 37. Participations in the ERC, per action type and country group (all actions) 

Horizon 
Europe 
(all 
pillars) 

ERC 

StG 

CoG 

AdG 

Syg 

PoC 

Member States 

84.5% 

87.0% 

88.6% 

87.9% 

89.9% 

68.9% 

85.2% 

Associated 
countries 

5.3% 

10.5% 

10.5% 

6.2% 

8.4% 

9.6% 

13.9% 

Third countries 

3.4% 

0.8% 

0.5% 

1.6% 

0.2% 

8.9% 

0.25 

Switzerland 

2.3% 

0.5% 

0.1% 

1.0% 

0.4% 

3.0% 

0.4% 

United Kingdom  

4.4% 

1.2% 

0.3% 

3.3% 

1.1% 

9.6% 

0.2% 

Table 38. Shares of participations, per action type and country group (all actions) 

Country 

Number of participations 

% of total 

Israel 

Norway 

7.1% 

2.5% 

201 

70 

629 

 
 
 
Country 

Number of participations 

% of total 

Turkey 

Iceland 

Serbia 

New Zealand 

Tunisia 

North Macedonia 

17 

4 

1 

1 

1 

1 

Israel 

201 

0.6% 

0.1% 

0.0% 

0.0% 

0.0% 

0.0% 

7.1% 

Table 39. Participations in the ERC from associated countries (all actions) 

Country 

Number of participations 

% of total 

Net EU 
contribution 
(EUR) 

United Kingdom 

Switzerland 

United States 

Canada 

Australia 

India 

Namibia 

Kenya 

Chile 

Ghana 

Mexico 

1.2% 

20,348,076.28 

0.5% 

15,645,535.25 

0.4% 

17,056,131.25 

0.1% 

2,286,622.50 

0.0% 

3,367,309.00 

0.0% 

697,500.00 

0.0% 

482,445.00 

0.0% 

380,995.00 

0.0% 

137,500.00 

0.0% 

81,725.00 

0.0% 

50,000.00 

34 

13 

12 

4 

1 

1 

1 

1 

1 

1 

1 

630 

 
Country 

Number of participations 

% of total 

Net EU 
contribution 
(EUR) 

Tanzania (United Republic of) 

United Kingdom 

Switzerland 

1 

34 

13 

0.0% 

26,000.00 

1.2% 

20,348,076.28 

0.5% 

15,645,535.25 

Table 40. Participations in the ERC from non-associated third countries (all actions) 

Country 

Country 
group 

Number of participations  % of total 

Net EU 
contribution 
(EUR) 

Germany 

France 

Netherlands 

Italy 

Israel 

Spain 

Belgium 

Austria 

Sweden 

Denmark 

Norway 

Member 
State (MS) 

Member 
State (MS) 

Member 
State (MS) 

Member 
State (MS) 

Associated 
country 

Member 
State (MS) 

Member 
State (MS) 

Member 
State (MS) 

Member 
State (MS) 

Member 
State (MS) 

Associated 
country 
(AC) 

21.3% 

296,595,607.36 

14.5% 

183,419,206.50 

11.8% 

178,361,528.00 

9.1% 

113,145,085.10 

7.0% 

110,581,842.50 

6.1% 

82,437,943.50 

5.4% 

77,462,842.75 

4.1% 

56,626,115.65 

3.5% 

50,937,126.50 

3.2% 

47,399,024.74 

2.3% 

34,034,685.00 

211 

144 

117 

90 

69 

60 

54 

41 

35 

32 

23 

631 

 
Country 

Country 
group 

Number of participations  % of total 

Net EU 
contribution 
(EUR) 

Finland 

Poland 

Ireland 

Portugal 

Turkey 

Czechia 

Greece 

United Kingdom 

United States 

Cyprus 

Slovenia 

Iceland 

Slovakia 

Serbia 

Luxembourg 

Member 
State (MS) 

Member 
State (MS) 

Member 
State (MS) 

Member 
State (MS) 

Associated 
country 
(AC) 

Member 
State (MS) 

Member 
State (MS) 

Third 
country (at 
a time of 
analysis) 

Third 
country 
(TC) 

Member 
State (MS) 

Member 
State (MS) 

Associated 
country 
(AC) 

Member 
State (MS) 

Associated 
country 
(AC) 

Member 
State (MS) 

2.1% 

28,738,090.50 

1.8% 

26,827,905.75 

1.8% 

25,589,227.50 

1.4% 

19,004,838.00 

0.8% 

13,992,337.50 

0.8% 

12,617,037.00 

0.5% 

7,541,105.00 

0.3% 

1,082,319.75 

0.3% 

381,475.00 

0.2% 

3,748,644.00 

0.2% 

3,702,404.75 

0.2% 

3,499,711.00 

0.2% 

1,776,265.00 

0.1% 

1,498,239.00 

0.1% 

1,496,821.00 

21 

18 

18 

14 

8 

8 

5 

3 

3 

2 

2 

2 

2 

1 

1 

632 

 
Country 

Hungary 

Lithuania 

Switzerland 

India 

Malta 

Ghana 

North Macedonia 

Country 
group 

Member 
State (MS) 

Member 
State (MS) 

Third 
country 
(TC) 

Third 
country 
(TC) 

Member 
State (MS) 

Third 
country 
(TC) 

Associated 
country 
(MS) 

Number of participations  % of total 

Net EU 
contribution 
(EUR) 

1 

1 

1 

1 

1 

1 

1 

0.1% 

1,175,215.00 

0.1% 

1,141,779.00 

0.1% 

780,750.00 

0.1% 

697,500.00 

0.1% 

190,789.24 

0.1% 

81,725.00 

0.1% 

71,500.00 

Country 

Germany 

France 

Spain 

Netherlands 

Italy 

Israel 

Sweden 

Table 41. Overview of participations in the ERC Starting Grants (StG) programme 

Country 
group 

Number of participations 

% of total 

Net EU 
contribution 

155% 

22.0% 

108% 

15.3% 

56% 

54% 

45% 

43% 

38% 

8.0% 

7.7% 

6.4% 

6.1% 

5.4% 

MS 

MS 

MS 

MS 

MS 

AC 

MS 

72 

41 

27 

19 

27 

7 

11 

633 

 
Country 
group 

Number of participations 

% of total 

Net EU 
contribution 

Country 

Belgium 

Austria 

Denmark 

Norway 

Finland 

Portugal 

Ireland 

MS 

MS 

MS 

AC 

MS 

MS 

MS 

United Kingdom 

TC 

Hungary 

Poland 

Czechia 

Cyprus 

Switzerland 

Greece 

Turkey 

United States 

Romania 

Lithuania 

Estonia 

Slovenia 

MS 

MS 

MS 

MS 

TC 

MS 

AC 

TC 

MS 

MS 

MS 

MS 

32% 

26% 

26% 

23% 

18% 

16% 

14% 

11% 

5% 

5% 

5% 

4% 

4% 

3% 

2% 

2% 

1% 

1% 

1% 

1% 

4.5% 

3.7% 

3.7% 

3.3% 

2.6% 

2.3% 

2.0% 

1.6% 

0.7% 

0.7% 

0.7% 

0.6% 

0.6% 

0.4% 

0.3% 

0.3% 

0.1% 

0.1% 

0.1% 

0.1% 

10 

9 

7 

8 

7 

15 

5 

10 

4 

3 

4 

1 

3 

3 

2 

2 

1 

1 

1 

1 

634 

 
Country 

Iceland 

Kenya 

New Zealand 

Chile 

Tanzania 

Country 
group 

Number of participations 

% of total 

Net EU 
contribution 

AC 

TC 

AC 

TC 

TC 

1 

1 

1 

1 

1 

1% 

1% 

1% 

1% 

1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

Table 42. Overview of participations in the ERC Consolidator grants (CoG) programme 

Country 

Country group 

% of total 

Net EU contribution 

Germany 

France 

Netherlands 

Italy 

Spain 

Israel 

Belgium 

Denmark 

Sweden 

Austria 

Ireland 

Norway 

Portugal 

MS 

MS 

MS 

MS 

MS 

AC 

MS 

MS 

MS 

MS 

MS 

AC 

MS 

58% 

41% 

18% 

28% 

28% 

6% 

7% 

5% 

6% 

8% 

4% 

5% 

9% 

635 

116% 

93% 

52% 

45% 

44% 

31% 

20% 

18% 

17% 

17% 

10% 

10% 

9% 

 
Country 

Country group 

% of total 

Net EU contribution 

Finland 

Czechia 

United Kingdom 

Greece 

Poland 

Slovenia 

Hungary 

Turkey 

Switzerland 

Estonia 

Luxembourg 

Romania 

Tunisia 

Canada 

MS 

MS 

TC 

MS 

MS 

MS 

MS 

AC 

TC 

MS 

MS 

MS 

AC 

TC 

3% 

6% 

5% 

4% 

4% 

2% 

3% 

1% 

2% 

1% 

1% 

1% 

1% 

1% 

7% 

6% 

6% 

5% 

4% 

4% 

3% 

2% 

2% 

1% 

1% 

1% 

1% 

1% 

Table 43. Overview of participations in the ERC Advanced Grants (AdG) programme 

Country 

Germany 

France 

Country 
group 

MS 

MS 

United Kingdom 

TC 

Israel 

AC 

Number of participations 

% of total 

Net EU 
contribution 

22% 

21% 

13% 

12% 

16.3% 

15.6% 

9.6% 

8.9% 

18 

12 

12 

8 

636 

 
Country 

Country 
group 

Number of participations 

% of total 

Net EU 
contribution 

Spain 

Italy 

Netherlands 

Denmark 

United States 

Austria 

Switzerland 

Belgium 

Canada 

Czechia 

Norway 

Sweden 

Ireland 

Australia 

Portugal 

Croatia 

Slovenia 

Namibia 

Finland 

MS 

MS 

MS 

MS 

TC 

MS 

TC 

MS 

TC 

MS 

AC 

MS 

MS 

TC 

MS 

MS 

MS 

TC 

MS 

9 

9 

6 

4 

6 

4 

4 

3 

3 

2 

1 

1 

1 

1 

1 

1 

1 

1 

1 

11% 

10% 

7% 

7% 

7% 

4% 

4% 

3% 

3% 

2% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

8.1% 

7.4% 

5.2% 

5.2% 

5.2% 

3.0% 

3.0% 

2.2% 

2.2% 

1.5% 

0.7% 

0.7% 

0.7% 

0.7% 

0.7% 

0.7% 

0.7% 

0.7% 

0.7% 

Table 44. Overview of participations in the ERC Synergy Grants (Syg) programme 

637 

 
Country 

Country 
group 

Number of participation 

% of total 

Net EU 
contribution 

Italy 

Spain 

France 

Germany 

Netherlands 

Israel 

Sweden 

Belgium 

Austria 

Denmark 

Norway 

Ireland 

Portugal 

Finland 

Turkey 

Czechia 

Cyprus 

Poland 

Slovenia 

Luxembourg 

MS 

MS 

MS 

MS 

MS 

AC 

MS 

MS 

MS 

MS 

AC 

MS 

MS 

MS 

AC 

MS 

MS 

MS 

MS 

MS 

63% 

57% 

54% 

51% 

49% 

45% 

22% 

20% 

16% 

13% 

13% 

11% 

10% 

9% 

5% 

4% 

4% 

3% 

2% 

2% 

13.7% 

12.4% 

11.7% 

11.1% 

10.7% 

9.8% 

4.8% 

4.3% 

3.5% 

2.8% 

2.8% 

2.4% 

2.2% 

2.0% 

1.1% 

0.9% 

0.9% 

0.7% 

0.4% 

0.4% 

25 

34 

23 

36 

19 

8 

11 

8 

12 

4 

11 

4 

7 

4 

3 

4 

2 

3 

2 

1 

638 

 
Country 

Country 
group 

Number of participation 

% of total 

Net EU 
contribution 

Switzerland 

Iceland 

Greece 

Hungary 

United Kingdom 

Mexico 

TC 

AC 

MS 

MS 

TC 

TC 

2 

1 

1 

1 

1 

1 

2% 

1% 

1% 

1% 

1% 

1% 

0.4% 

0.2% 

0.2% 

0.2% 

0.2% 

0.2% 

Table 45. Overview of participations in the ERC Proof of Concept (PoC) grant programme 

Horizon 
Europe 
(all 
pillars) 

MSCA 
(all) 

COFUND 

PF 

DN 

SE 

Total 

54,911 

10,846 

955 

3,841 

4,914 

1,073 

Member States 

46,408 

8,686 

801 

3,156 

4,038 

645 

Associated countries 

2,925 

442 

Third countries 

1,891 

920 

Switzerland 

1,250 

259 

United Kingdom 

2,437 

539 

49 

61 

14 

30 

168 

154 

66 

354 

205 

292 

75 

88 

154 

363 

15 

55 

Table 46. Participations in the MSCA, per action type and country group 

Horizon 
Europe 
(all 
pillars) 

MSCA 
(all) 

COFUND 

CSA 

PF 

SE 

Member States 

84.5% 

80.1% 

83.9% 

82.2% 

82.2% 

60.1% 

Associated countries 

5.3% 

4.1% 

5.1% 

4.4% 

3.1% 

6.2% 

639 

 
 
 
Horizon 
Europe 
(all 
pillars) 

MSCA 
(all) 

COFUND 

CSA 

PF 

SE 

Third countries 

3.4% 

8.5% 

6.4% 

9.2% 

4.2% 

27.2% 

Switzerland 

2.3% 

2.4% 

1.5% 

2.0% 

3.1% 

1.4% 

United Kingdom 

4.4% 

5.0% 

3.1% 

2.3% 

7.4% 

5.1% 

Table 47. Shares of participations in the MSCA, per action type and country group 

Country 

Country 
group 

Number of participations  % of total 

Net EU 
contribution 

United Kingdom 

Turkey 

United States 

Australia 

Switzerland 

Serbia 

Norway 

TC 

AC 

TC 

TC 

TC 

AC 

AC 

Moldova (Republic of) 

AC 

Japan 

Singapore 

Montenegro 

Brazil 

Canada 

China 

TC 

TC 

AC 

TC 

TC 

TC 

30% 

3.1% 

28% 

2.9% 

25% 

2.6% 

16% 

1.7% 

14% 

1.5% 

6% 

5% 

4% 

4% 

3% 

2% 

2% 

2% 

2% 

0.6% 

0.5% 

0.4% 

0.4% 

0.3% 

0.2% 

0.2% 

0.2% 

0.2% 

27 

26 

24 

16 

11 

6 

3 

4 

4 

2 

2 

2 

2 

2 

640 

 
 
Country 

Country 
group 

Number of participations  % of total 

Net EU 
contribution 

Korea (Republic of) 

Iceland 

Ukraine 

Israel 

New Zealand 

India 

Mexico 

South Africa 

Taiwan 

United Arab Emirates 

TC 

AC 

AC 

AC 

AC 

TC 

TC 

TC 

TC 

TC 

2 

1 

1 

1 

1 

1 

1 

1 

1 

1 

2% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

0.2% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

Table 48. Overview of participations in MSCA COFUND (associated countries and third countries) 

Country 

Country 
group 

Number of participation 

% of 
total 

Net EU 
contribution 

United States 

Norway 

United Kingdom 

Switzerland 

Canada 

Australia 

Israel 

Turkey 

TC 

AC 

TC 

TC 

TC 

TC 

AC 

AC 

102 

237% 

6.2% 

130% 

3.4% 

88% 

2.3% 

75% 

2.0% 

31% 

0.8% 

22% 

0.6% 

16% 

0.4% 

13% 

0.3% 

32 

48 

22 

18 

12 

9 

8 

641 

 
Country 

Japan 

Mexico 

Chile 

New Zealand 

Brazil 

South Africa 

China 

India 

Colombia 

Serbia 

Iceland 

Palestine 

Tanzania 

Argentina 

Ecuador 

Hong Kong 

Lebanon 

Peru 

Bosnia and 
Herzegovina 

Cabo Verde 

Country 
group 

Number of participation 

% of 
total 

Net EU 
contribution 

TC 

TC 

TC 

AC 

TC 

TC 

TC 

TC 

TC 

AC 

AC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

AC 

AC 

6 

6 

4 

2 

4 

4 

3 

3 

2 

2 

2 

2 

2 

2 

2 

2 

1 

1 

1 

1 

642 

7% 

6% 

5% 

4% 

4% 

4% 

4% 

3% 

3% 

2% 

2% 

2% 

2% 

2% 

2% 

2% 

2% 

2% 

1% 

1% 

0.2% 

0.2% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.0% 

0.0% 

 
Country 

Country 
group 

Number of participation 

% of 
total 

Net EU 
contribution 

Costa Rica 

Ethiopia 

Fiji 

Guam 

Holy See 

Iran (Islamic 
Republic of) 

Kenya 

Korea (Republic of) 

Monaco 

Pakistan 

São Tomé and 
Príncipe 

Singapore 

Thailand 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

Table 49. Overview of participations in MSCA PF (associated countries and third countries) 

Country 

Country 
group 

Unique 
participants 

Number of 
participations 

United Kingdom 

Switzerland 

Norway 

United States 

TC 

TC 

AC 

TC 

363 

154 

93 

87 

221 

82 

48 

76 

643 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

% of 
total 

7.4% 

3.1% 

1.9% 

1.8% 

 
Country 

Israel 

Canada 

Australia 

Japan 

South Africa 

China (People's 
Republic of) 

Serbia 

Turkey 

Iceland 

Georgia 

Brazil 

Botswana 

Argentina 

India 

Kenya 

Thailand 

Montenegro 

Ghana 

Ukraine 

Taiwan 

Country 
group 

Unique 
participants 

Number of 
participations 

AC 

TC 

TC 

TC 

TC 

TC 

AC 

AC 

AC 

AC 

TC 

TC 

TC 

TC 

TC 

TC 

AC 

TC 

AC 

TC 

14 

15 

12 

12 

8 

7 

8 

6 

4 

4 

4 

3 

2 

3 

3 

3 

2 

2 

2 

2 

644 

30 

15 

13 

13 

10 

9 

8 

6 

5 

4 

4 

3 

3 

3 

3 

3 

2 

2 

2 

2 

% of 
total 

0.6% 

0.3% 

0.3% 

0.3% 

0.2% 

0.2% 

0.2% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.0% 

0.0% 

0.0% 

0.0% 

 
Country 
group 

Unique 
participants 

Number of 
participations 

Country 

Chile 

Colombia 

Ecuador 

Indonesia 

Korea (Republic of) 

Malaysia 

Monaco 

Vietnam 

Costa Rica 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

Moldova (Republic of) 

AC 

Albania 

New Zealand 

Tunisia 

Singapore 

Bolivia 

Egypt 

Ethiopia 

Hong Kong 

Jordan 

Kyrgyzstan 

AC 

AC 

AC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

% of 
total 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

2 

2 

2 

2 

2 

2 

2 

2 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

2 

2 

2 

2 

2 

2 

2 

2 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

645 

 
Country 

Country 
group 

Unique 
participants 

Number of 
participations 

Liechtenstein 

Mauritius 

Mexico 

Morocco 

Namibia 

Papua New Guinea 

Philippines 

Saudi Arabia 

Senegal 

United Arab Emirates 

Zambia 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

% of 
total 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

Table 50. Overview of participations in Doctoral Networks (DN) 

Country 

Country 
group 

Number of participations 

% of total 

Net EU 
contribution 

United States 

United Kingdom 

China 

Turkey 

Argentina 

Canada 

Japan 

TC 

TC 

TC 

AC 

TC 

TC 

TC 

70% 

55% 

27% 

22% 

19% 

17% 

17% 

6.5% 

5.1% 

2.5% 

2.1% 

1.8% 

1.6% 

1.6% 

56 

41 

24 

17 

12 

13 

13 

646 

 
Country 
group 

Number of participations 

% of total 

Net EU 
contribution 

Country 

Ukraine 

Brazil 

Australia 

Switzerland 

Serbia 

Chile 

Mexico 

Israel 

South Africa 

Azerbaijan 

Egypt 

Malaysia 

Thailand 

Uzbekistan 

Cuba 

Morocco 

Colombia 

AC 

TC 

TC 

TC 

AC 

TC 

TC 

AC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

Korea (Republic of) 

TC 

Vietnam 

TC 

Moldova (Republic of) 

AC 

16% 

16% 

16% 

15% 

10% 

9% 

9% 

8% 

7% 

6% 

5% 

5% 

5% 

5% 

5% 

5% 

4% 

4% 

4% 

3% 

1.5% 

1.5% 

1.5% 

1.4% 

0.9% 

0.8% 

0.8% 

0.7% 

0.7% 

0.6% 

0.5% 

0.5% 

0.5% 

0.5% 

0.5% 

0.5% 

0.4% 

0.4% 

0.4% 

0.3% 

13 

13 

9 

10 

9 

8 

7 

6 

4 

5 

5 

5 

5 

5 

4 

4 

4 

4 

4 

3 

647 

 
Country 

Country 
group 

Number of participations 

% of total 

Net EU 
contribution 

Kazakhstan 

Kyrgyzstan 

Singapore 

Montenegro 

New Zealand 

Indonesia 

Lebanon 

Nigeria 

Rwanda 

Tajikistan 

Tunisia 

Georgia 

Norway 

French Polynesia 

Bolivia 

Burkina Faso 

Costa Rica 

Ecuador 

Hong Kong 

India 

TC 

TC 

TC 

AC 

AC 

TC 

TC 

TC 

TC 

TC 

AC 

AC 

AC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

3% 

3% 

3% 

2% 

2% 

2% 

2% 

2% 

2% 

2% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

0.3% 

0.3% 

0.3% 

0.2% 

0.2% 

0.2% 

0.2% 

0.2% 

0.2% 

0.2% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

3 

3 

2 

2 

2 

2 

2 

2 

2 

2 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

648 

 
Country 

Country 
group 

Number of participations 

% of total 

Net EU 
contribution 

Iran 

Kenya 

Kuwait 

Mauritius 

Mongolia 

Mozambique 

Panama 

Peru 

Saudi Arabia 

Tanzania 

Turkmenistan 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

TC 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

Table 51. Overview of participations in Staff Exchanges (SE) 

649 

 
 
Participations 
in INFRA 

Share of 
participations 
in INFRA 

Participations 
in Horizon 
Europe 
(all pillars) 

Share of all 
Horizon 
Europe 
participations 

Total 

1,787 

100% 

54,911 

100% 

Member States 

1,456 

81.5% 

46,408 

84.5% 

Associated countries 

105 

Third countries 

Switzerland 

35 

57 

United Kingdom 

134 

5.9% 

2.0% 

3.2% 

7.5% 

2,925 

1,891 

1,250 

2,437 

5.3% 

3.4% 

2.3% 

4.4% 

Table 52. Table 5.16. Participations in Research Infrastructures, per action type and country group 

Country 

Norway 

Israel 

Serbia 

Turkey 

North Macedonia 

Iceland 

Georgia 

Albania 

Armenia 

Ukraine 

Moldova (Republic of) 

Number of  
participations 

% of total 

3.7% 

0.4% 

0.4% 

0.3% 

0.2% 

0.2% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

67 

8 

7 

6 

3 

3 

2 

2 

2 

2 

1 

650 

 
 
Country 

Number of  
participations 

Bosnia and Herzegovina 

Montenegro 

1 

1 

% of total 

0.1% 

0.1% 

Table 53. Participations in Research Infrastructures from associated countries 

Country 

Number of  
participations 

% of total 

Switzerland 

United States 

South Africa 

Canada 

Japan 

Senegal 

Australia 

Cote d'Ivoire 

Kenya 

Bangladesh 

Azerbaijan 

New Caledonia 

Burkina Faso 

Mali 

China (People's Republic of) 

7.5% 

3.2% 

0.4% 

0.3% 

0.2% 

0.2% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

134 

57 

8 

5 

3 

3 

2 

2 

1 

1 

1 

1 

1 

1 

1 

651 

Net EU 
contribution 
(EUR) 

7,428,453.29 

5,899,371.25 

44,062.50 

359,625.00 

1,144,600.00 

0.00 

37,250.50 

6,840.00 

98,250.00 

95,625.00 

52,903.75 

11,750.00 

2,518.00 

988.00 

988.00 

 
Country 

Number of  
participations 

% of total 

Korea (Republic of) 

Peru 

Taiwan 

Uruguay 

1 

1 

1 

1 

0.1% 

0.1% 

0.1% 

0.1% 

Net EU 
contribution 
(EUR) 

0.00 

0.00 

0.00 

0.00 

Table 54. Participations in Research Infrastructures from non-associated third countries 

Horizon Europe 
(all pillars) 

WIDERA  
(all actions) 

Widening 
participation 
and spreading 
excellence 

Strengthening 
the ERA 

Total 

54,911 

100% 

1,936 

100% 

1,231 

100% 

705 

100% 

Member States 

46,408 

84.5% 

1,603 

82.8% 

1,020 

82.9% 

583 

82.7% 

Associated 
countries 

2,925 

5.3% 

210 

10.8% 

146 

11.9% 

64 

9.1% 

Third countries 

1,891 

3.4% 

38 

2.0% 

22 

1.8% 

16 

2.3% 

Switzerland 

1,250 

2.3% 

11 

0.6% 

6 

0.5% 

5 

0.7% 

United Kingdom 

2,437 

4.4% 

74 

3.8% 

37 

3.0% 

37 

5.2% 

Table 55. Participations in WIDERA, per action type and country group 

Country 

Number of participations 

% of total 

Serbia 

Turkey 

Norway 

North Macedonia 

Albania 

2.5% 

2.3% 

1.7% 

0.5% 

0.5% 

48 

44 

32 

10 

10 

652 

 
 
Country 

Number of participations 

% of total 

Bosnia and Herzegovina 

Montenegro 

Israel 

Tunisia 

Ukraine 

Armenia 

Moldova (Republic of) 

Georgia 

Iceland 

Faroe Islands 

Kosovo * UN resolution 

New Zealand 

10 

10 

9 

8 

7 

5 

5 

4 

4 

2 

1 

1 

0.5% 

0.5% 

0.5% 

0.4% 

0.4% 

0.3% 

0.3% 

0.2% 

0.2% 

0.1% 

0.1% 

0.1% 

Table 56. Participations in WIDERA from associated countries 

Country 

Number of participations 

% of total 

Net EU 
contribution 
(EUR) 

United Kingdom 

Switzerland 

United States 

Angola 

Kenya 

Australia 

3.8% 

623,581.24 

0.6% 

345,635.00 

0.3% 

291,275.00 

0.2% 

0.00 

0.2% 

498,601.25 

0.2% 

241,500.00 

74 

11 

5 

4 

3 

3 

653 

 
Country 

Number of participations 

% of total 

Net EU 
contribution 
(EUR) 

Brazil 

China (People's Republic of) 

Korea (Republic of) 

South Africa 

India 

Costa Rica 

Algeria 

Bhutan 

Canada 

Central African Republic 

Chile 

Colombia 

Egypt 

Equatorial Guinea 

Mozambique 

Nigeria 

Paraguay 

São Tomé and Príncipe 

Sri Lanka 

3 

2 

2 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

0.2% 

33,750.00 

0.1% 

223,087.50 

0.1% 

137,152.50 

0.1% 

348,690.00 

0.1% 

160,560.00 

0.1% 

120,750.00 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.1% 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

Table 57. Participations in WIDERA from non-associated third countries  

654 

 
 
Annex 5. Application and success rates 

Horizon 2020 
(total) 

Horizon 2020  
(first two years) 

Horizon Europe 
(total) 

Horizon Europe  
(first two years) 

Applica- 
tions 

Share 

Applica- 
tions 

Share 

Applica- 
tions 

Share 

Applica- 
tions 

Share 

Overall 

249,996 

100% 

58,963 

100% 

55,140 

100% 

51,669 

100% 

Member 
States 
(without the 
UK) 

United 
Kingdom 

Third 
countries 
(without the 
UK and 
Switzerland) 

176,292 

70.5% 

40,006 

67.8% 

42,116 

76.4% 

39,824 

77.1% 

35,955 

14.4% 

10,250 

17.3% 

5,629 

10.2% 

5,401 

10.5% 

18,936 

7.6% 

4,667 

7.9% 

3,845 

7.0% 

3,173 

6.1% 

Switzerland 

8,432 

3.4% 

1,678 

2.8% 

1,203 

2.2% 

1,167 

2.3% 

Horizon 
Europe 
associated 
countries 
(without 
Switzerland) 

10,381 

4.2% 

2,362 

4.0% 

2,347 

4.3% 

2,104 

4.1% 

Table 58. Application and success rates in the MSCA. Source: Horizon Dashboard, accessed 21 October 2023. 

Horizon 2020 
(total) 

Horizon 2020  
(first two years) 

Horizon Europe 
(total) 

Horizon Europe  
(first two years) 

Applica- 
tions 

Share 

Applica- 
tions 

Share 

Applica- 
tions 

Share 

Applica- 
tions 

Share 

Overall 

61,762 

100% 

20,224 

100% 

22,152 

100% 

20,944 

100% 

Member 
States 
(without the 
UK) 

United 
Kingdom 

43,839 

71.0% 

14,560 

72.0% 

17,151 

77.4% 

16,128 

77.0% 

10,824 

17.5% 

3,710 

18.3% 

2,517 

11.4% 

2,471 

11.8% 

655 

 
 
 
Horizon 2020 
(total) 

Horizon 2020  
(first two years) 

Horizon Europe 
(total) 

Horizon Europe  
(first two years) 

Applica- 
tions 

Share 

Applica- 
tions 

Share 

Applica- 
tions 

Share 

Applica- 
tions 

Share 

759 

1.2% 

211 

1.0% 

346 

1.6% 

313 

1.5% 

Third 
countries 
(without the 
UK and 
Switzerland) 

Switzerland 

2,452 

4.0% 

603 

3.0% 

446 

2.0% 

419 

2.0% 

Horizon 
Europe 
associated 
countries 
(without 
Switzerland) 

3,888 

6.3% 

1,140 

5.6% 

1,692 

7.6% 

1,613 

7.7% 

Table 59. Application and success rates in the ERC. Source: Horizon Dashboard, accessed 21 October 2023. 

Horizon 2020 
(total) 

Horizon 2020  
(first two years) 

Horizon Europe 
(total) 

Horizon Europe  
(first two years) 

Applica 
tions 

Share 

Applica 
tions 

Share 

Applica 
tions 

Share 

Applica 
tions 

Share 

13,141 

100% 

4,546 

100% 

4,663 

100% 

2,838 

100% 

10,350 

78.8% 

3,497 

76.9% 

3,809 

81.7% 

2,298 

81.0% 

1,237 

9.4% 

509 

11.2% 

305 

6.5% 

212 

7.5% 

486 

3.7% 

185 

4.1% 

164 

3.5% 

89 

3.1% 

Overall 

Member 
States 
(without the 
UK) 

United 
Kingdom 

Third 
countries 
(without the 
UK and 
Switzerland) 

656 

 
 
 
Horizon 2020 
(total) 

Horizon 2020  
(first two years) 

Horizon Europe 
(total) 

Horizon Europe  
(first two years) 

Applica 
tions 

Share 

Applica 
tions 

Share 

Applica 
tions 

Share 

Applica 
tions 

Share 

403 

3.1% 

140 

3.1% 

129 

2.8% 

93 

3.3% 

665 

5.1% 

215 

4.7% 

256 

5.5% 

146 

5.1% 

Switzerland 

Horizon 
Europe 
associated 
countries 
(without 
Switzerland) 

Table 60. Application and success rates in Research Infrastructures. Source: Horizon Dashboard, accessed 21 October 
2023. 

Horizon 2020 
(total) 

Horizon 2020  
(first two years) 

Horizon Europe 
(total) 

Horizon Europe  
(first two years) 

Applica 
tions 

Share 

Applica 
tions 

Share 

Applica 
tions 

Share 

Applica 
tions 

Share 

Overall 

26,401 

100% 

8,724 

100% 

7,338 

100% 

6,216 

100% 

Member 
States 
(without the 
UK) 

United 
Kingdom 

Third 
countries 
(without UK 
and 
Switzerland) 

21,541 

81.6% 

7,027 

80.5% 

6,170 

84.1% 

5,262 

84.7% 

1,750 

6.6% 

792 

9.1% 

225 

3.1% 

205 

3.3% 

627 

2.4% 

188 

2.2% 

98 

1.3% 

64 

1.0% 

Switzerland 

413 

1.6% 

118 

1.4% 

35 

0.5% 

23 

0.4% 

HE 
associated 
countries 
(without 
Switzerland) 

2,070 

7.8% 

599 

6.9% 

810 

11.0% 

662 

10.6% 

Table 61. Application and success rates in WIDERA. Source: Horizon Dashboard, accessed 2023/10/21 

657 

 
 
 
Annex 2.14: Case study on the impact of the Framework 
Programmes on fostering research integrity and ethics 

1.1. 

Introduction 

As stated in the Strategic Plan for Horizon Europe 2021-2024, ‘research ethics and integrity 
are a prerequisite for research excellence and a critical factor in achieving socially relevant 
impact’1022. 

Given  the  importance  of  ethics  and  integrity  in  EU-funded  research,  the  Framework 
Programmes (FPs) have stimulated research ethics and integrity. Among the means used 
are the following: 

1.  Supporting  projects  that  address  the  topics  of  ethics  and  research  integrity,  such  as 
under the SwafS (Horizon 2020) and WIDERA (Horizon Europe) programme parts; and 

2.  Setting  standards  and  policies  for  all  activities  funded  by  the  FP  (e.g.  ethics  review 

processes).  

Under Horizon Europe and previous Framework Programmes, all projects funded have been 
required to respect the European Code of Conduct for Research Integrity1023 and to adhere 
to the relevant ethical principles and legislation, including when activities are conducted in a 
non-EU country, in order to prevent so-called ‘ethics dumping’. The process used to assess 
and address the ethical dimensions of activities funded under Horizon Europe is the Ethics 
Appraisal Scheme. This procedure applies to all activities funded under Horizon Europe, and 
comprises several elements: when preparing a proposal, applicants are required to conduct 
an ethics self-assessment starting with the completion of an ethics issues table; prior to the 
start of the project, an ethics review procedure is conducted; and during the project, ethics 
checks, reviews and audits may be conducted. 

The  present  case  study  examines  whether  the  actions  of  the  Framework  Programmes  in 
relation to research integrity and ethics in EU R&I systems have generated positive effects 
in light of the criteria listed below. Key to this an assessment of the impact the FP has had 
on  the  convergence  of  national,  regional  or  international  ethics  and  integrity  procedures 
towards  EC  practices  is.  This  analysis  includes  drivers  and  motivations,  as  well  as  the 
barriers to implementing ethics and integrity procedures at those levels. In this way, gaps can 
be identified and ways forward proposed. At the same time, the case study evaluates how 
standards  such  as  ethics  appraisal  procedures  are  being  implemented  under  Horizon 
Europe. 

The  case  study  zooms  in  on  Pillar  1  and  WIDERA,  and  although  its  primary  focus  is  on 
Horizon  Europe,  where  relevant,  it  considers  the  outcomes  of  Horizon  2020  projects  and 
identifies long-term results from projects funded by previous Framework Programme. 

The  case  study  is  structured  as  follows.  Section  1.2  provides  background  information, 
including an overview of the evaluation criteria and evaluation questions relevant to this study 

1022 European Commission, Directorate-General for Research and Innovation, Horizon Europe 
strategic plan 2021-2024, Publications Office of the European Union, 2021, 
https://data.europa.eu/doi/10.2777/083753. 
1023 The 2023 edition of the Code of Conduct can be consulted online: Allea, The European 
Code of Conduct for Research Integrity, available at:https://allea.org/wp-
content/uploads/2023/06/European-Code-of-Conduct-Revised-Edition-2023.pdf. 

658 

 
 
and  of  the  policy  context  regarding  the  increasing  importance  of  ethics  and  integrity  in 
achieving excellence in research. This section also provides further information pertaining to 
the  integration  of  ethics  and  integrity  into  the  Framework  Programmes,  and  introduces 
projects  that  have  been  established  which  focus  on  research  ethics  and  integrity  under 
Horizon  2020  and  Horizon  Europe.  Lastly,  this  section  presents  the  methodological  tools 
employed in the present case study and the research questions upon which it aims to shed 
light. 

Section  1.3  presents  the  findings  of  the  case  study.  It  begins  by  providing  an  overview  of 
actions under Horizon Europe and Horizon 2020 that foster research ethics and integrity1024, 
as  well  as  presenting  some  participation  data.  This  is  followed  by  an  analysis  of  project-
specific  results  and  impacts,  and  the  identification  of  more  general  impacts,  as  well  as 
success factors and barriers. Information is then provided on the added value of projects at 
EU level, together with insights regarding the integration of ethics and integrity into research 
within  the  Framework  Programme.  Section  1.4  presents  the  conclusions  and  lessons 
learned. 

1.2.  Background information 

1.2.1.  Policy context 

Ethical research conduct entails the application of fundamental ethical principles to scientific 
research.  The  European  Code  of  Conduct  for  Research  Integrity,  revised  in  20231025, 
stipulates  that  research  integrity  ‘encompasses  the  basic  responsibility  of  the  research 
community to formulate the principles of research, to define the criteria for proper research 
behaviour, to maximise the quality, reliability, and robustness of research and its results, and 
to respond adequately to threats to, or violations of, good research practices.’ 

In  November  2021,  the  European  Commission  approved  the  Pact  for  Research  and 
Innovation  in  Europe,  which  includes  a  future  governance  structure  for  the  European 
Research  Area  (ERA)1026.  The  Pact  sets  out  the  priorities  of  the  ERA,  streamlines  its 
governance and applies a common set of values and principles for research and innovation 
across all EU Member States, including on issues relating to ethics and integrity.  

1.2.2. 

Integration of ethics and integrity into the Framework Programmes: the Ethics 
Appraisal Scheme 

The process used to assess and address the ethical dimension of activities funded under the 
Framework Programme is the Ethics Appraisal Scheme. The purpose of this procedure is to 
ensure  full  compliance  with  ethical  rules,  so  that  all  applicants  submit  an  ‘ethics-ready’ 
proposal. The Ethics Appraisal Scheme applies to all activities funded under Horizon 2020 
and Horizon Europe, and includes a self-assessment during proposal preparation, an ethics 
review  procedure  conducted  prior  the  start  of  the  project,  and  ethics  checks,  reviews  and 

1024 The term 'ethics and integrity in research' is now preferred as it encompasses a broader scope 
than 'research ethics and integrity,' which is often associated primarily with the ethics review of 
research. However, due to its traditional usage, this study will use 'research ethics and integrity' 
to refer to the broader activities beyond just ethics review. 
1025  Allea, The European Code of Conduct for Research Integrity, available at: 
https://allea.org/wp-content/uploads/2023/06/European-Code-of-Conduct-Revised-Edition-
2023.pdf. 
1026 European Commission, The Pact for Research and Innovation in Europe, available at: 
https://research-and-innovation.ec.europa.eu/news/all-research-and-innovation-
news/commission-adopts-proposal-pact-research-and-innovation-europe-2021-07-16_en. 

659 

 
 
audits  that  may  be  performed  during  project’s  implementation  (see  Appendix  2  for  more 
information  on  the  application  of  the  Ethics  Appraisal  Scheme).  Key  points  for  attention 
include  the  principle  of  proportionality,  the  right  to  privacy,  the  right  to  the  protection  of 
personal  data,  the  right  to  the  physical  and  mental  integrity  of  a  person,  the  right  to  non-
discrimination,  and  the  need  to  ensure  protection  of  the  environment  and  high  levels  of 
protection for human health.  

Under  Horizon  Europe,  the  Ethical  Appraisal  Scheme  has  been  finetuned  compared  to 
Horizon 2020. Specifically, there was a shift towards a risk-based ethics appraisal process 
in  which  proposals  that  raise  serious  or  complex  ethics  issues  are  identified,  which  has 
significantly  simplified the  process  (process  fully  outlined  in  Appendix  2).  Furthermore, 
there are novelties under Horizon Europe, such as the declarations applicants need to fill out. 
Key improvements of Horizon Europe with regard to ethical dimensions are:  

•  Research integrity has been fully incorporated into guidance documents with reference 
to the European Code of Conduct for Research Integrity, as well as in the Model Grant 
Agreement1027. The explicit declaration of conformity with the European Code of Conduct 
represents a novelty under Horizon Europe. This means that applicants must formally 
confirm  that  their  activities  are  compliant  with  the  European  Code  of  Conduct  for 
Research Integrity published by All European Academies. In addition, they must declare 
that  appropriate  procedures,  policies  and  structures  are  in  place  to  foster responsible 
research practices, to prevent questionable research practices and research misconduct, 
and  to  handle  allegations  of  breaches  of  the  principles  and  standards  in  the  Code  of 
Conduct1028. 

•  The  Guide  for  proposal  submission  and  evaluation  provides  guidance  for  (potential) 
applicants  and  includes  minimum  scientific  quality  requirements  (e.g.  research 
publishing  protocols).  Compared  with  the  revised  2015  version  of  the  Guide1029,  the 
information on ethics and integrity in the 2023 version is more extensive, and includes, 
among other things, a reference document on the consistent use of the GDPR and the 
processing of personal data for research purposes1030. 

•  The ethics self-assessment is now included in part A of the proposal, with web-based 
forms  being  generated  by  the  submissions  IT  system.  In  this  way,  applicants  receive 
guidance from the IT system on how to submit the required information in a structured 
manner. Supporting documents can be provided in part B of the proposal, but this is only 

1027 Article 14 of the Model Grant Agreement (MGA) stipulates that ‘the action must be carried 
out in line with the highest ethical standards and the applicable EU, international and national 
law on ethical principles. The beneficiaries must commit to and ensure the respect of basic EU 
values (such as the respect for human dignity, freedom, democracy, equality, the rule of law and 
human rights, including the rights of minorities).’ See: European Commission, Programme Guide 
Horizon Europe, available at: https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/guidance/programme-guide_horizon_en.pdf. 
1028 Horizon Europe Programme. Standard Application Form (HE RIA, IA). Version 8.0. 
Consulted on May 28, 2024 through https://ec.europa.eu/info/funding-
tenders/opportunities/docs/2021-2027/horizon/temp-form/af/af_he-ria-ia_en.pdf 
1029 European Commission, Grants Manual – action on: Proposal submission and evaluation, 
available at: 
https://ec.europa.eu/research/participants/data/ref/h2020/grants_manual/pse/h2020-guide-
pse_en.pdf. 
1030 European Commission, Horizon Europe (HORIZON), Programme Guide, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/guidance/programme-guide_horizon_en.pdf. 

660 

 
 
expected  for  applications  involving  considerable  issues  or  risks,  unlike  Horizon  2020, 
where it was requested by default.  

•  A  more  pronounced  distinction  is  made  between  projects  in  which  serious  and/or 
complex  issues  are  present,  and  those  without  such  issues  (as  identified  during  the 
screening process). While the bulk of the signed grants receive full ethics clearance after 
the ethics screening, meaning that no specific conditions or reporting duties are required, 
this novel process substantially limits the administrative burden and complexity. 

•  A specific section on the use of artificial intelligence (AI) has been added to the appraisal 
procedure under Horizon Europe. This represents an increased focus on the topic, and 
a  growing  recognition  of  the  ethical  issues  it  raises.  The  European  Commission  has 
published a guiding document entitled ‘Ethics guidelines for trustworthy AI’, developed 
by the High-Level Expert Group on Artificial Intelligence. This document is often referred 
to during ethics evaluations1031. Other EC guidance frameworks on several aspects of AI 
ethics  have  been  developed  since,  such  as  the  ‘Ethics  By  Design  and  Ethics  of  Use 
Approaches for Artificial Intelligence’ guidance note1032.  

• 

In the application form, applicants need to confirm that the research has an exclusive 
focus on civil applications (as opposed to military applications) or, in the case of dual-
use  research,  that  the  research  is  in  compliance  with  relevant  regulations.  This  is  no 
longer subject to an assessment by ethics reviewers. A guidance note for applicants on 
this was developed in 20211033. Concerns relating to dual-use and security-related issues 
are currently part of a dedicated security review1034. 

•  Research that could be misused from a security perspective is no longer covered under 
the Ethics Appraisal Scheme (but is also dealt with under the dedicated security review). 
In  addition,  the  scheme  no  longer  contains  a  specific  section  on  misuse  from  a  non-
security perspective.  Potential issues are covered in other sections, and now fall under 
‘cross-cutting issues’). A guidance note has been developed on this topic1035. 

1.2.3.  Research ethics and research integrity projects as part of Horizon Europe and 

Horizon 2020 

In line with the integration of research ethics and integrity as a fundamental principle under 
Horizon  2020  and  Horizon  Europe,  a  portfolio  of  research  ethics  and  research  integrity 
projects was also developed as part of (and funded by) Horizon 2020 Science with and for 
Society  (SwafS),  and  is  under  development  under  Horizon  Europe  WIDERA  (Widening 

1031 https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai. 
1032 European Commission, Ethics By Design and Ethics of Use Approaches for Artificial 
Intelligence (2021), available at: https://ec.europa.eu/info/funding-
tenders/opportunities/docs/2021-2027/horizon/guidance/ethics-by-design-and-ethics-of-use-
approaches-for-artificial-intelligence_he_en.pdf. 
1033 European Commission, Guidance note - Research with an exclusive focus on civil 
applications, 2021), available at: https://ec.europa.eu/info/funding-
tenders/opportunities/docs/2021-2027/horizon/guidance/guidance-note-research-focusing-
exclusively-on-civil-applications_he_en.pdf.  
1034 https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/common/guidance/how-to-handle-security-sensitive-projects_en.pdf. 
1035 European Commission, Guidance note – Potential misuse of research, 2021, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/guidance/guidance-note-potential-misuse-of-research-results_he_en.pdf. 

661 

 
 
participation  and  strengthening  the  European  Research  Area).  For  further  discussion  on 
these projects, refer to Section 1.3.1. 

1.2.4.  Overall approach and scope of the case study 

To make an informed assessment as to whether the efforts of the Framework Programmes 
in relation to research integrity and ethics in the EU R&I systems have generated positive 
effects  with  respect  to  the  evaluation  criteria  of  effectiveness  and  EU  added  value,  the 
present case study relies on the following methods:   

•  Extensive desk research was carried out on documents, policy reports and other sources 
relating to research integrity and ethics and its main elements; in particular, with regard 
to  the  changes  implemented  under  Horizon  Europe  and  the  long-term  impact  of  the 
SwafS projects funded under Horizon 2020. A list of desk research sources is provided 
in Appendix 1. 

•  Analysis of the administrative, monitoring and reporting data of projects funded by the 

Framework Programmes on this area.  

•  Analysis  of  the  survey  of  Horizon  Europe  beneficiaries,  conducted  in  May-July  2023, 
provided information about the beneficiaries’ perceptions regarding ethics and integrity 
topics under the Framework Programmes. 

• 

Lastly,  interviews  with  beneficiaries  and  stakeholders  constituted  key  sources  of 
information for this case study. Interviews were held with project coordinators from SwafS 
projects  funded  under  Horizon  2020  (5)  and  from  WIDERA  projects  funded  under 
Horizon Europe (2), as well as with stakeholders at European level (4).  

1.3.  Case study findings 

This section begins by providing descriptive statistics relating to the Ethics Appraisal Scheme 
for Horizon Europe projects. It then goes on to present an overview of projects under Horizon 
Europe and Horizon 2020 that have had a clear focus on ethics and/or integrity in research, 
with  an  emphasis  on  the  results  and  impacts  of  relevant  Horizon  2020  projects.  Lastly, 
general findings are outlined with regard to impacts, success factors and barriers, and EU 
added value with respect to ethics and integrity in research is analysed. 

1.3.1.  Descriptive statistics on the Ethics Appraisal Scheme for Horizon Europe 

projects 

Monitoring data1036 on the ethics screening of proposals under Horizon Europe allows further 
insights to be shed on the implementation of this process.  

The step of ethics screening aims to identify proposals involving serious or complex ethics 
issues. If any such serious and/or complex issues are identified, the proposal is subject to a 
full ethics assessment, as a result of which ethics requirements are likely to be defined. Other 
proposals, for which the ethics screening step does not identify serious or complex issues, 
are cleared without such ethics requirements. Nevertheless, ethics experts may still require 
the  appointment  of  an  ethics  advisor  or  ethics  advisory  board  and/or  advise  subsequent 
ethics checks or reviews to be carried out on cleared proposals. In all cases, cleared projects 
need  to  comply  with  the  obligatory  self-declarations  submitted  in  the  proposal  as  well  as 

1036 Received from the European Commission 22 August 2023.  

662 

 
 
general ethical standards and rules. These obligations are included in the Ethics Summary 
Report  and  may  be  monitored  through  ethics  checks  or  reviews  during  the  project’s 
implementation. All relevant documents need to be kept on file and relevant documents must 
be submitted upon request. 

Analysis of the monitoring data shows that out of the 7,671 Horizon Europe projects signed 
by  the  time  the  data  were  extracted,  6,882  were  cleared  and  782  projects  were 
conditionally cleared1037. These data do not allow the identification of how many proposals 
underwent  the  full  ethics  assessment  process;  only  the  outcomes  at  the  time  the  grant  is 
signed are recorded.  

In the case of both conditionally cleared and cleared proposals, it may be recommended that 
they  are  subject  to  ethics  checks  and  reviews  during  their  implementation  when  deemed 
necessary  to  reassess  the  global  situation  or specific  ethics  issues,  or when  the  proposal 
does not contain all necessary information. Among the 6,882 cleared proposals, an ethics 
check was recommended for 290 projects and an ethics review for nine projects. Among the 
782 conditionally cleared projects, an ethics check was recommended for 221 projects and 
an ethics review for nine projects. 

Based  on  this  process,  all  782  conditionally  cleared  projects  are subject  to  specific ethics 
requirements with which they need to comply. In addition, 317 cleared projects have specific 
ethics requirements defined for the project as a result of the ethics checks or reviews, yet the 
ethics requirements for the cleared projects are on average lower in number per project than 
for the conditionally cleared projects. 

This means that a total of 1,099 projects under Horizon Europe (or 14.3% of the in total 
of 7,671 projects at the time the data were extracted)  have given rise to  one or several 
specific  ethics  requirements.  The  inclusion  of  ethics  requirements  often  involves  the 
appointment  of  an  ethics  advisor  or  ethics  advisory  board  from  an  early  stage  on  in  the 
project.  Additional  provisions  about  this  process  and  a  description  of  the  role  of  this 
advisor/board are provided. Commonly, an added requirement is the production of an ethics 
report  after  certain  stages  of  the  project  by  the  advisor/board.  Another  relatively  common 
requirement  is  the  provision  of  more  information,  for  example  to  explain  the  process  of 
obtaining informed consent, to justify compliance with ‘3Rs’ principles for animal research, or 
to explain how sensitive data will be treated in a GDPR-compliant manner. 

Figure  1  provides  an  overview  of  the  proportion  of  projects  with  ethics  requirements,  per 
programme  part so  far  under  Horizon  Europe.  Some  variation  exists  between  programme 
parts,  with  Cluster  5  (Climate,  Energy  &  Mobility),  European  Innovation  Ecosystems 
(EIE), MSCA and ERC having the lowest shares of projects with ethics requirements. 
All  four  of  these  programme  parts  thus  have  shares  below  the  average  of  14.3%  for  all 
Horizon Europe grants (yet this average is strongly driven by the higher numbers of ERC and 
MSCA  grants  compared  with  the  number  of  grants  in  other  programme  parts).  Other 
Clusters have some of the highest shares, with 55.6% for Cluster 2 (Culture, Creativity & 
Inclusive  Society)  56.6%  of  Cluster  3  (Civil  Security  for  Society)  and  44.1%  of  Cluster  1 
(Health) projects being assigned ethics requirements.  

1037 For 7 projects, no information is available in the dataset for this variable (among which one 
Framework Partnership Agreement and one Identified Beneficiary Agreement). 

663 

 
 
ERC (N=2063)

MSCA (N=2533)

INFRA (N=79)

Cluster 1 (N=229)

Cluster 2 (N=142)

Cluster 3 (N=53)

Cluster 4 (N=500)

Cluster 5 (N=413)

Cluster 6 (N=311)

EIC (N=620)

EIE (N=166)

WIDERA (N=353)

EU Missions…

Average = 14.3%

11,6%

6,4%

29,1%

44,1%

55,6%

56,6%

17,0%

2,7%

35,4%

25,2%

4,2%

15,3%

21,8%

0%

20%

40%

60%

80%

100%

Figure 1. Proportion of Horizon Europe projects for which one or more ethical requirements were requested, as a share of all 
projects in each programme part. Note: projects relating to the European Union Agency for the Space Programme (EUSPA, 
n=13), Joint Undertakings (JU, n=53) and one project classified as horizontal IBA, are not shown in the graph. Source: data 
received from the European Commission 22 August 2023. 

1.3.2.  Overview of projects with a clear focus on ethics and integrity in research 

under Horizon Europe and Horizon 2020  

Table 1 in Appendix 2 provides an overview of the relevant projects under Horizon 2020 that 
had a clear focus on ethics and/or integrity in research1038. These projects add up to a total 
budget  of  approximately  EUR 50  million,  and  involved  211  partner  organisations,  with  the 
following distribution: around half of these organisations being higher or secondary education 
services  (62;  47%)  and  the  remaining  half  roughly  equally  divided  between  research 
organisations (19; 15%), private for-profit entities (25; 19%) and others (25; 19%). 

Figure 2 below shows participations in the aforementioned projects, by EU-28 Member States 
(including the UK). For comparison, these data are presented alongside data for all Horizon 
2020  projects.  This  reveals  that  some  EU  countries  are  far  more  likely  to  participate  in 
projects  related  to  ethics  and  integrity  in  research,  presumably  because  the  topic  is  well 
developed and there is a relative abundance of organisations with relevant expertise. This is 
the  case  in  countries  such  as  Austria,  Belgium,  Ireland  and  the  Netherlands.  Conversely, 
France, Italy and Spain participated less often in projects with a focus on ethics and integrity 
in research under Horizon 2020, compared with their participation in Horizon 2020 projects 
more generally. 

1038 The list does not include projects with a focus on Responsible Research and Innovation. 
These projects were identified in European Commission, Directorate-General for Research and 
Innovation, Monachello, R., Delaney, N., Tornasi, Z., Research ethics and research integrity : 
achievements in Horizon 2020 and recommendations on the way forward, Delaney, N. (editor), 
Tornasi, Z. (editor), Luxembourg: Publications Office of the European Union, 2020, available 
at: https://data.europa.eu/doi/10.2777/63976. 

664 

 
 
 
16%
14%
12%
10%
8%
6%
4%
2%
0%

i

n
a
p
S

y
n
a
m
r
e
G

y
l
a
t
I

e
c
n
a
r
F

s
d
n
a
l
r
e
h
t
e
N

i

m
o
d
g
n
K
d
e
t
i
n
U

i

m
u
g
e
B

l

n
e
d
e
w
S

e
c
e
e
r
G

a
i
r
t
s
u
A

k
r
a
m
n
e
D

l

a
g
u
t
r
o
P

d
n
a
n
i
F

l

d
n
a
o
P

l

d
n
a
e
r
I

l

i

a
h
c
e
z
C

i

a
n
a
m
o
R

y
r
a
g
n
u
H

i

a
n
e
v
o
l
S

s
u
r
p
y
C

a
i
r
a
g
u
B

l

i

a
n
o
t
s
E

a
i
t
a
o
r
C

a
i
k
a
v
o
l
S

i

a
n
a
u
h
t
i
L

a
i
v
t
a
L

a
t
l
a
M

g
r
u
o
b
m
e
x
u
L

Horizon 2020

Projects related to Ethics and Integrity in Science

Figure 2. Participation in ethics and integrity in research projects by country as a percentage of all such projects: comparison 
with national shares of all Horizon 2020 projects. Source: Horizon Dashboard, accessed 7-8 July 2023. 

Projects in the area of ethics and integrity in research continue to be supported under Horizon 
Europe. Under the thematic priority ‘Reforming and enhancing the European R&I System’, 
and  within  the  WIDERA  programme  part,  three  relevant  projects  (with  project  acronyms 
PREPARED,  iRECS  and  BEYOND)  were  signed  in  July  2023,  with  a  combined  EU  net 
contribution of more than EUR 10 million and close to 50 participating organisations. 

1.3.3.  Results and impacts of Horizon 2020 projects on ethics and integrity in 

research 

This  section  focuses  on  the  outcomes  of  the  Horizon  2020  projects  listed  in  Table  1  in 
Appendix 2, about which more information on (longer-term) results is available than for the 
recent  and  ongoing  Horizon  Europe  projects.  Analysis  of  these  Horizon  2020  projects  is 
mostly  based  on  project  reporting  documents,  complemented  by  insights  from  interviews. 
The  aims  of  these  projects  can  be  broadly  categorised  into  the  following  themes:  1) 
increasing  the  knowledge  base  and  raising  awareness  and  understanding  of  the  need  to 
conform  to  high  standards  of  research  ethics  and  integrity;  2)  developing  tools  to  support 
compliance  with  norms  and  standards;  and  3)  setting  up  a  network  of  experts  and 
disseminating its insights to a broad group of stakeholders. In the paragraphs that follow, we 
further elaborate on the results and impacts of these projects in relation to these three main 
types of objectives. 

Increasing the knowledge base, raising awareness and understanding of the need to 
conform to high standards of research ethics and integrity 

PRINTEGER1039 is a good example of a project pursuing the above goal. While integrity in 
science  is  sometimes  perceived  as  an  external  control  system  with  items  that  need  to  be 
ticked off a list, the central premise of PRINTEGER is that excellent science requires a culture 
in  which integrity  takes  centre  stage.  Other projects  among  those  presented  in  Table  1  in 
Appendix  2  have  a  similar  ambition,  sometimes  with  a  narrower  focus  –  for  example, 

1039 Promoting Integrity as an Integral Dimension of Excellence in Research, https://printeger.eu/. 

665 

 
 
 
 
HYBRIDA1040, which addressed specific ethics and integrity issues in relation to organoids1041, 
or  ROSiE1042  which  focused  on  open  science  and  citizen  science  from  a  multidisciplinary 
perspective. Box 1 below provides further examples.  

Box 1. Results of the EnTIRE, SIENNA and SHERPA projects, in relation to increasing 
the knowledge base and raising awareness 

These three projects expanded the knowledge base in relation to research integrity and 
ethics by mapping the diversity of relevant practices in general, and in relation to emerging 
technological  developments.  It  is  important  to  note  that  all  of  these  projects  strove  to 
operationalise  their  findings,  and  make  these  insights  as  accessible  as  possible  to 
stakeholders. 

EnTIRE developed an online open-source, wiki-based platform, designed by and for the 
research integrity and research ethics community, called the ‘Embassy of Good Science’. 
This  is  a  platform  providing  access  to  normative  frameworks,  fostering  the  active 
participation of the scientific community, and supporting the quality of science by making 
knowledge and experience more accessible. The stakeholder consultation mapped the 
local standards, rules and practices prevailing across the EU, and made these sometimes 
implicit  norms  more  explicit.  This  information  was  processed  in  a  systematic  and 
structured  manner  to  allow  interested  parties  to  understand  and  compare  the  varied 
landscape. All of this information is made available in an interactive website. Six peer-
reviewed  articles  were  published  as  part  of  the  project,  with  20  additional  publications 
listed. 

SIENNA looked into ethical, legal and human rights issues related to developments in AI 
and robotics, human genomics and human enhancement, in particular to raise awareness 
in a timely fashion. For each of these technologies, an ethical framework was developed. 
These frameworks were, in turn, operationalised in the form of the following tools: 

• 

• 

• 

• 

operational guidelines for research ethics committees for these technologies; 

codes  of  responsible  conduct  and  other  ethics  tools  for  researchers  who  develop 
these technologies; 

proposals for revisions to existing ethical frameworks; and 

proposals for revisions to existing legal frameworks. 

Lastly, a report was developed, containing insights on how to repeat this process for other 
emerging technologies that potentially raise consequential ethical questions. The results 
of the project were communicated in various ways, with consortium members reported to 
have reached around five million stakeholders over the course of the entire process – for 
instance,  by  means  of  the  project  website,  a  newsletter,  policy  briefs,  social  media, 
academic publications and talks, public talks, webinars, workshops and conferences. 

SHERPA placed a spotlight on smart information systems (SIS), the combination of AI 
and big data analytics. It developed 10 case studies and other materials on the ethical 
and security aspects of SIS, and brought together a community of around 30 experts who 
shared their thoughts on an ongoing basis, and in turn reached out to a much broader 
stakeholder list (a total of around 1,000 people). The project generated 18 peer-reviewed 
articles,  undertook  a  variety  of  dissemination  and  communication  activities,  hosted  21 

1040 Embedding a comprehensive ethical dimension to organoid-based research and resulting 
technologies, https://hybrida-project.eu/. 
1041 Miniaturised and simplified versions of an organ produced in vitro. 
1042 Fostering responsible Open Science in Europe, https://rosie-project.eu/. 

666 

 
 
advocacy activities/events, and reached out to 22 selected policy-makers across the EU 
who have a say in shaping the policy framework for SIS. Guidelines on ethics by design 
were also developed, which were turned into a Horizon Europe guidance document.  

Developing  a  broad  range  of  tools  that  can  be  applied  by  various  stakeholders  to 
support compliance with norms and standards 

Based on desk research of relevant Horizon 2020 projects, it appears that tools developed 
in  these  projects  typically  endeavour  to  be  hands-on  and  easy  to  apply  by  organisations 
interested in the topic that would otherwise struggle to set appropriate standards for research 
ethics in a coherent and structured manner. By adopting these tools, stakeholders such as 
ethics committees and integrity offices in research organisations and research funding bodies 
can strengthen their capacity. As an illustration of the issue faced, an international survey 
among researchers across Europe and the United States, revealed that many researchers 
are confident in their personal abilities to ensure integrity in research, but lack appropriate 
training and support from their organisation1043. A recurrent feature of these projects is that 
they provide sufficient flexibility, given that a ‘one-size-fits-all’ policy would not be applicable 
across  the  EU  and  beyond  due  to  the  diversity  of  systems  and  regulations.  Box  2  below 
provides more information about two projects in this area. 

Box 2. Results of the SOPs4RI and VIRT2UE projects, in relation to developing various 
tools on research integrity and ethics 

The SOPs4RI project website states that “‘SOPs4RI aims to stimulate transformational 
processes across European Research Performing Organisations and Research Funding 
Organisations (RPOs and RFOs)’. Under this project, 121 tools spanning nine different 
topics were developed for research performing organisations, and 29 tools across 6 topics 
were  developed  for  research  funding  organisations.  The  full  toolbox,  including  all  the 
above modules, was finalised in December 2022, in line with expectations and following 
a  four-year  process.  The  tools  developed  were  underpinned  by  a  literature  review, 
interviews with experts and a survey among practitioners, as well as extensive piloting. 
As  a  measure of  its  success,  the  SOPs4RI  Project  guidelines  are also  featured  in  the 
Horizon Europe Programme Guide. Another indication of the project’s added value to the 
research  community is  the  fact  that  nine  peer-reviewed  articles  were  published  on  the 
topic using the insights from the toolbox and discussing how it was developed. More than 
80 events took place during the implementation of this project in which members from the 
Consortium  participated  actively.  While  many  of  these  were  not  organised  by  the 
SOPs4RI team, they presented opportunities to raise awareness. 

VIRT2UE developed a toolbox of educational resources, online and offline courses and 
group  sessions  using  participatory  exercises  to  train  individuals,  who  can  then  share 
insights relating to research ethics and integrity within their community across Europe (i.e. 
the project is a ‘train-the-trainers’ endeavour). VIRT2UE's learning materials are guided 
by the principles listed in the European Code of Conduct for Research Integrity (ECoC) 
and  are  modular  in  nature,  offering  some  flexibility  to  adapt  to  specific  audiences  and 
circumstances. Trainers can pick and choose their own didactic material on the basis of 

1043 Allum, N., Reid, A., Bidoglia, M., Gaskell, G., Aubert-Bonn, N., Buljan, I., Fuglsang, S., 
Horbach, S., Kavouras, P., Marušić, A., Mejlgaard, N., Pizzolato, D., Roje, R., Tijdink, J., & Veltri, 
G. Researchers on research integrity: a survey of European and American researchers. 
F1000Res. 2023 Feb 16;12:187. doi: 10.12688/f1000research.128733.1. 

667 

 
 
length, purpose of training, target audience and so on. Four peer-reviewed articles were 
published in the context of this project.  

Two out of the Horizon 2020 projects - Path2Integrity1044 and INTEGRITY1045 - focus on the 
student population, developing educational tools to instil the importance of ethics and integrity 
among potential future researchers and explaining how to approach scientific outputs with a 
critical mind (see Box 3).  

Box  3.  Results  of  the  Path2Integrity  and  INTEGRITY  projects  in  relation  to  learning 
materials for students 

Path2Integrity developed role models and rotating role-playing exercises for adolescents 
(in secondary school and university) as part of the Learning Card Programme for teaching 
and  learning  research  integrity.  It  also  launched  an  awareness-raising  campaign  for 
educational  organisations  on  the  topic.  This  knowledge  was  then  transferred  via  a 
network of 150 disseminators, and a campaign was carried out in schools and universities 
in  15  EU  countries.  Three  publications  were  published  in  peer-reviewed  journals,  and 
there were 44 other publications – mainly case studies on trust in science among students 
in  various  countries  and  institutional  settings.  The  project  also  set  up  a  permanent 
network of trainers and community of research integrity educators as a founding member 
of the Network of Education and Research Quality (NERQ), which remains active after 
the ending of the EU funding. Project materials are available online at no cost. The project 
also  carried  out  a  mapping  exercise  of  secondary  school  and  university  curricula  that 
include research integrity, research ethics or associated fields. 

INTEGRITY  was  another  project  involving  student  training  in  research  integrity,  with 
empowerment and an evidence-based approach at its core. It has designed a variety of 
educational tools for upper-level high school students, undergraduate students and PhD 
students  using  online  courses,  web-based  narratives  and  similar  tools.  In  total,  10 
educational tools for high school students, a simulation game for undergraduate students, 
three thematic online courses for PhD students, a massive open online course (MOOC) 
for  PhD  students,  and  an  online  module  for  supervisors,  were  developed  and 
implemented in a handful of institutions. The project’s website lists other organisations 
that  expressed  interest  in,  for  example,  the  MOOC  developed  by  the  project,  which  is 
among  the  most  popular  provided  by  COURSERA,  a  well-established  provider  of 
MOOCs. 

Setting  up  a  network  of  experts  and  disseminating  its  insights  to  a  broad  group  of 
stakeholders 

Several projects have aimed to build a community of research ethics and integrity experts. 
ENERI1046  is  notable  in  this  respect,  developing  governance  for  the  advancement  of 
responsible research and innovation by all stakeholders that is sensitive to society’s needs 

1044 Rotatory role-playing and role-models to enhance the research integrity culture, 
https://cordis.europa.eu/project/id/824488. 
1045 INTEGRITY: empowering students through evidence-based, scaffolded learning of 
Responsible Conduct in Research (RCR), https://cordis.europa.eu/project/id/824586. 
1046 European network of research ethics and research integrity, 
https://cordis.europa.eu/project/id/710184. 

668 

 
 
and demands, as well as promoting an ethics framework for research and innovation. Other 
projects,  such  as  Path2Integrity,  INTEGRITY,  VIRT2UE  and  ENTIRE,  have  also  aimed  to 
foster  community-building.  Indeed,  as  the  boxes  in  this  section  illustrate,  many  (if  not  all) 
projects  contain  a  community-building  and  outreach  component,  and  typically  list  their 
outreach efforts as part of their deliverables. 

Combining objectives 

Individual  projects  have  typically  combined  the  various  aspects  mentioned  in  the  boxes 
above.  Another  example,  DEFORM1047,  placed  a  strong  emphasis  on  expanding  the 
knowledge base and raising awareness by making the (potential) financial consequences of 
research  misconduct  more  explicit.  A  related  objective  is  to  “propose  a  methodology  and 
guidelines  for  anticipating  and  prevent  the  appearance  of  this  type  of  practices”,  which  is 
closer to the second type of activities. Lastly, the project also had a networking component, 
bringing together experts on the topic and sharing their insights to a broader audience. Box 
4 below provides another example of a project with multiple aims. 

Box 4. Results of the TRUST project, in relation to various objectives 

The TRUST project had multiple ambitions. It aimed to improve adherence to high ethical 
standards  in  research  around  the  globe  via  a  collaborative  platform  that  included 
organisations  from  middle-  and  low-income  countries.  By  raising  standards  worldwide, 
this project aimed to reduce the risk of ‘ethics dumping’, whereby unethical practices are 
‘outsourced’ outside the EU-27. In particular, it set out to develop: 1) a Global Code of 
Conduct  for  North‐South  Collaboration  (GCC);  2)  a  Fair  Research  Contracts  Webtool; 
and 3) a Compliance and Ethics Follow‐up Tool. 

The  impact  of  the  project’s  work  has  been  manifold  and  goes  beyond  its  stated  aims, 
according  to  the  project’s  reporting  on  the  CORDIS  page.  The  GCC  has  indeed  been 
developed and disseminated as a reference tool for forming equitable partnerships and 
fostering high standards, as has an online tool for stakeholders who would not otherwise 
have access to lawyers or relevant information, and a follow-up tool for conditions of high 
vulnerability.  In  addition  to  achieving  these  aims,  the  project  has  made  a  significant 
contribution  in  terms  of  informing  how  to  foster  integrity  and  ethics  for  research  with 
implications for indigenous communities through the San Code of Research Ethics. The 
project developed the Global Code of Conduct for Research in Resource-Poor Settings, 
which  has  become  a  mandatory  reference  document  for  EU Framework  Programmes, 
the  EDCTP  and  other  organisations.  The  name  of  the  code  was  later  changed  to  the 
TRUST  Code:  A  Global  Code  of  Conduct  for  Equitable  Research  Partnerships.  The 
project  has  also  resulted  in  five  peer-reviewed  articles.  Lastly,  the  team  has  been 
successful  in  attracting  high-profile  media  attention,  which  in  turn  fosters  broader 
dissemination.  An  important  achievement  of  the  project  was  identifying  and  selecting 
trainees  for  the  ‘train-the-trainer’  programme,  and  creating  a  database  of  trainers 
accessible via a common European access point. 

While a fully fledged impact analysis of projects on research ethics and integrity developed 
with support from Horizon 2020 is beyond the scope of this case study (and would require 
substantial data on the take-up of tools, the impact on beneficiaries, and so on), the desk 
research  and  interviews  carried  out  for  the  present  study  suggest  that  these  have  been 
largely  successful.  Generally  speaking,  the  stated  objectives  (for example, in  terms  of  the 

1047 Define the global and financial impact of research misconduct, 
https://cordis.europa.eu/project/id/710246. 

669 

 
 
tools  developed  or  network  events  organised)  were  met;  the  requested  deliverables  were 
completed,  and  milestones  were  met  in  line  with  the  agreed  timing.  Furthermore,  these 
projects have made a contribution to the formation of platforms for experts in research ethics 
and research integrity across the EU-27. In addition, they have led to contributions in peer-
reviewed journals and have expanded the knowledge base. Lastly, the various tools that have 
been  developed  have  seen  some  adoption  by  national  stakeholders,  universities  and 
research organisations. More information about the success factors – in particular, in terms 
of how the projects have had a lasting impact on research and integrity practices within the 
EU – as well as areas for improvement and potential barriers, are highlighted below. 

1.3.4.  Findings on impacts, success factors and barriers 

Horizon 2020 has been instrumental in establishing a network of experts spanning the 
EU-27 and beyond  

Interviews carried out for this case study indicate that in some countries and regions within 
the EU, it is sometimes difficult to identify and collaborate with experts on ethical research 
and integrity, especially outside of Northern Europe. In this sense, it should thus be noted 
that the building and maintenance by these projects of a network of such experts from various 
Member States with sufficient expertise and critical mass is an important achievement. The 
European Network for Research Ethics and Integrity (ENERI) is a prime example. It positions 
itself  as  a  network  of  networks  across  the  EU  bringing  together  key  stakeholders  and 
researchers. ENERI began as a three-year Horizon 2020 project in 2016, but has remained 
active, and its e-community has become a permanent platform. This platform could be a step 
towards an open database of experts on research ethics and research integrity, which could 
potentially address the difficulties faced by researchers and other stakeholders in identifying 
whom to contact with questions on these topics. This is a good example of an institutional 
change which, among other things, demonstrates a meaningful impact that would not have 
been achieved in the absence of the project, and whose impact has lasted beyond the lifetime 
of the project funding. Interviewees pointed out that the  relevant EU-funded projects were 
indeed key to kickstarting and develop the community. 

With  respect  to  other  projects,  it  has  reportedly  been  hard  to  maintain  the  momentum  of 
experts and other stakeholders once the projects were complete. SIENNA, for instance, has 
a  LinkedIn  group  of  stakeholders  that  remains  active  to  some  extent,  but  interactions  are 
limited. 

Horizon  2020  brought  together  the  community  on  research ethics  and  integrity,  but 
attention should also be paid to opening the community up to new organisations 

Interviewees highlighted that under Horizon 2020, the same experts were often involved in 
several projects, and in turn in projects from other programming periods as well. This has 
both positive and negative implications. On the positive side, collaborating with people who 
have been involved for a sufficiently long period of time allows coherence between projects 
and leads to some degree of continuity between activities over time. From a more negative 
viewpoint, interviewees point out that working with ‘the usual suspects’ in projects on integrity 
and responsible innovation may lead to an ‘echo chamber’ effect. Interviewees reported that, 
once you have worked in a project as an expert, there is a tendency to contribute to follow-
up projects, as this significantly increases the likelihood of a successful application. A careful 
balance  needs  to  be  struck  between  ensuring  continuity  and  fostering  the  participation  of 
newcomers  who  could  bring  fresh  perspectives  and  may  be  more  critical  towards  past 

670 

 
achievements, as well as further expanding the community of experts. Interviewees pointed 
out  that  the  balance  may  currently  be  skewed  too  much  towards  continuity  and  the 
involvement of the same partners, especially at organisation level. 

The  projects  have  been  successful  in  developing  learning  tools  related  to  research 
ethics and integrity and in expanding the knowledge base, yet the Embassy of Good 
Science could focus on becoming a broader repository of information 

As  the  preceding  section  illustrates,  the  projects  funded  under  Horizon  2020  have  been 
successful  in  developing  tools  and  guidance  material  for  policy-makers  in  the  EU-27  and 
beyond. The peer-reviewed articles that have resulted from these work streams indicate the 
generally high quality of the outputs produced, as well as the relative novelty of the insights 
gathered.  Interviewees  also  pointed  out  that  many  projects  generated  novel  insights  and 
contributed to the understanding of research ethics and integrity. As another positive point, 
there was general agreement among interviewees that, within the Framework Programme, 
there  has  been  an  increasing  tendency  for  projects  to  build  upon  previous  projects,  in  an 
effort to avoid ‘reinventing the wheel’. 

The  Embassy  of  Good  Science1048  (which  was  the  initiative  of  the  VIR2UE  and  EnTIRE 
projects)  is  an  important  achievement  in  this  respect.  This  is  an  online  wiki-platform  that 
serves as a hub for those interested in good scientific practices in relation to research ethics 
and integrity. As a minimum, it is a repository for the materials so far developed, allowing the 
research  community  to  build  upon  existing  work  and  providing  a  ‘one-stop  shop’  for other 
stakeholders to access information. It also allows users –who are logged in with their ORCID 
researcher  ID  to  add  new  information,  and  to  update  and  refine  existing  information.  The 
Embassy of Good Science is thus another example of a tangible output that still exerts an 
impact  even  after  funding  from  Horizon  has  ceased  (with  the  explicit  aim  of  letting  the 
research community to take over, with 93 unique contributors so far. 

However, based on an examination of its website, it appears that the majority of content on 
the Embassy of Good Science platform has been produced in the context of the VIR2UE and 
EnTIRE projects, with material from other relevant projects often not visible or even absent. 
Interviewees  who  were  not  involved  in  these  two  projects  were  not  always  aware  of  the 
website’s potential. Those who were involved in the above projects sometimes highlighted 
that there are technical issues with regard to adding content such as interactive tools in a 
user-friendly manner, as its wiki-based structure is not well suited to this purpose. 

Aside  from  adding  existing  material  in  a  consistent  manner,  it  also  remains  to  be  seen 
whether the community will actually make meaningful contributions in the absence of funding. 
This could be monitored, in addition to statistics on visits to the website and downloads of the 
material, so as to gain a better understanding of the continued impact. Interviewees pointed 
out that continued support for the updating of the Embassy’s website would be a welcome 
(and  cost-efficient)  way  to  cement  its  position  as  a  one-stop  repository  of  information.  It 
should  be  noted  that  foreseen  projects  on  research  ethics  funded  by  the  Framework 
Programme specify structured cooperation with the e-platform Embassy of Good Science, 
with the aim of placing outputs on the repository1049. 

1048 The Embassy of Science website, https://www.embassy.science. 
1049 https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2023-2024/wp-11-widening-participation-and-strengthening-the-european-research-
area_horizon-2023-2024_en.pdf. 

671 

 
 
Interviewees expressed the need to strengthen the dissemination of project results 

A recurring observation made by interviewees is that, thanks to EU funding, a great deal of 
material (such as guides, tutorials, good practices, ethical codes, toolboxes and so on) has 
been  developed  on  the  topic  of  research  integrity  and  ethics.  This  has  improved  the 
understanding of the importance of these topics and what needs to be done in order to foster 
high  ethical  and  integrity  standards.  All  of  the  projects  under  scrutiny  have  conducted 
considerable  outreach  efforts,  participating  in  many  events  and  reaching  out  to  many 
stakeholders.  SOPs4RI,  for  instance,  has  a  strategy  for  dissemination  that  encompasses 
selected social media tools, taking part in academic conferences, contributing to academic 
journals and so on for the duration of the project, with 82 events listed on its website at which 
the toolbox was discussed or at least brought to the attention of stakeholders. These types 
of actions are deemed necessary to generate a sizeable impact on the ground, with many 
projects  aiming  to  foster  behavioural  change  at  organisational  (universities,  research 
organisations, funding bodies, etc.) and even individual level (researchers at various levels 
of seniority, and students in secondary or higher education).  

At the same time, there was a perception among interviewees that many of these outputs are 
still not shared widely among the university and research community, and are therefore not 
sufficiently put into practice. The courses that were developed by INTEGRITY, for instance, 
have been made part of the curriculum in a handful of educational organisations (identified 
during desk research). This implies that there is vast untapped potential for introducing these 
courses  into  other  institutions,  which  may  necessitate  further  (if  limited)  support.  Some 
interviewees  indicated  that  a  funding  scheme  focusing  on  dissemination  and  follow-up  on 
previous projects would add value from this perspective. 

A related concern expressed by interviewees is that some of the exercises carried out on this 
topic have been rather technocratic and/or bureaucratic in nature. Interviewees expressed a 
need for practical tools that are accessible with few hurdles (including for non-experts in the 
matter), rather than more technical notes. In other words, there is an emerging sentiment that 
the  knowledge  base  is  sufficiently  well  developed,  and  that  the  most  value  is  now  in 
disseminating the key insights widely and in an easily digestible manner.  

In a similar spirit, further efforts could be made to collaborate with stakeholders to put 
outputs into practice 

A  related  opportunity  raised  by  interviewees  is  to  focus  more  on  implementation  in  future 
projects  relating  to  integrity  and  ethics  in  science.  Tools  such  as  ethical  codes  could  be 
shared  more  widely  to  inspire  national  frameworks,  university  curriculums,  the  approval 
procedures of research funding organisations and so on, so as to increase their impact. This 
focus on the implementation or adoption of research ethics and research integrity tools would 
require close collaboration with interested stakeholders from the early stages of the project 
cycle, to ensure these tools and insights are adapted to the specific requirements of projects 
and that there is sufficient buy-in and a sense of ownership. According to interviewees, this 
is  relatively  uncommon  at  the  moment,  especially  with  regard  to  actors  operating  at  the 
national  level  of  government  (while  it  is  often  reported  to  be  easier  to  engage  with  pan-
European organisations). 

672 

 
Relevant calls are sometimes deemed overly prescriptive 

Some interviewees indicated that calls on research ethics and integrity under Horizon 2020 
and Horizon Europe calls have been rather specific, with a clear agenda for what they want 
to  achieve  and  what  proposals  should  look  like.  These  interviewees  argued  that  there  is 
limited  room  for  creative  or  outside-the-box  ideas,  and  suggest  these  calls  could  be 
complemented  by  calls  whose  terms  of  reference  ask  for  a  particular  problem  to  be 
addressed, but which are more open-ended as to exactly how to tackle the issues at hand. 

Fostering ethics and integrity in research requires the strengthening of  the skills of 
researchers and research staff in relation to these topics 

Interviews and desk research indicate that researchers at all levels of seniority require skills 
in how to properly apply research ethics and research integrity, which could represent another 
area  of  focus  for  future  projects.  As  an  illustration,  the  recently  published  European 
Competence  Framework  for  Researchers,  ‘ResearchComp’,  stresses  the  importance  to 
researchers in the EU of developing transferable skills – of which applying research ethics 
and integrity principles is an integral component.  

Ethics and integrity in research would surely be more widely applied if (aspiring) researchers 
could  access  more  formal  training  more  easily.  Two  examples  of  institutions  providing 
comprehensive  training  programmes  on  research  ethics  and  research  integrity  are  the 
Science  Foundation  Ireland  (SFI)  in  Ireland  and  the  Austrian  Agency  for  Research 
Integrity1050. 

The  rising  focus  on  ethics  and  integrity  is  perceived  positively,  and  the  changes 
introduced  under  Horizon  Europe  appear  to  address  concerns  regarding  the 
associated administrative burden for researchers and applicants 

The  OPC  on  the  past,  present  and  future  of  the  European  Research  and  Innovation 
Framework Programmes 2014-20271051 confirms that many stakeholders regard ethics and 
integrity in science as crucial, and consider it appropriate to consider this in the application 
and implementation phases of projects. This is often seen as a factor that can enable a higher 
quality of research. As a case in point; slightly more than half of all respondents (1,037 out of 
1,933)  thought  it  was  ‘essential’  or  of  ‘high  priority’  to  further  elaborate  this  theme  in  the 
Strategic Plan 2025-2027, according to the synopsis report on the consultation1052. 

At the same time, there is widespread concern that the inclusion of ethics and integrity makes 
the application process more cumbersome, and that not all organisations have the capacity 
to comply with the necessary requirements. The consultation revealed that similar positions 

1050 This organisation participated in the Horizon 2020 VIR2TUE project (Virtue based ethics and 
Integrity of Research: Train-the-Trainer program for Upholding the principles and practices of the 
European Code of Conduct for Research Integrity). 
1051 The results of this consultation were published in April 2023. The process lasted 12 weeks. 
For more information, see: https://research-and-.innovation.ec.europa.eu/news/all-research-and-
innovation-news/results-public-consultation-future-eu-research-and-innovation-programmes-are-
now-public-2023-04-19_en. 
1052 Synopsis report of the Public consultation on the past, present and future of the European 
research and innovation Framework Programmes 2014-2027, 
https://op.europa.eu/en/publication-detail/-/publication/e8f722ec-dff7-11ed-a05c-
01aa75ed71a1/language-en/format-PDF/source-284554549.  

673 

 
 
were  expressed  with  regard  to  other  transversal  requirements,  such  as  those  related  to 
Gender Equality Plans or Open Science. While these horizontal requirements are seen to 
have their merits, they may lead to additional costs and overheads, as well as the need to 
involve experts on cross-cutting issues. Some participants in the consultation indicated that 
the  increasing  trend  to  ‘tick  more  boxes’  may  lead  to  barriers  for  stakeholders  with  less 
experience  and  administrative  capacity,  especially  SMEs  and  organisations  based  in 
widening countries. 

A  survey  conducted  among  research  funding  organisations  across  the  EU  as  part  of  the 
SOPs4RI  project  is  revealing  in  this  respect.  Among  the  respondents,  38% (33  out  of  86) 
were  not  aware  of  the  research  integrity  funding  requirements  for  Horizon  2020  projects. 
However,  out  of  those  respondents  who  were  aware,  85%  (45  out  of  53  respondents) 
approved of the formal inclusion of ethical issues in the application procedure.  

Specifically  with  regard  to  the  ethics  review  procedure,  it  should  be  noted  that  National 
Research  Ethics  Committees  (and  likewise,  committees  at  institutional  level)  have  not  yet 
been established in every EU country. Where such committees are in place, the resources 
and  capabilities  at  their  disposal  appear  to  vary,  and  training  and  support  facilities  more 
generally are not always available. Even in the EU countries that are considered frontrunners, 
many  ‘low-hanging  fruits’  such  as  setting  up  an  ethics  committee  or  establishing 
whistleblower procedures have not yet been addressed1053. 

This  absence  of  such  committees  could  inhibit  organisations  from  such  countries  from 
applying  for  funding  from  EU  Framework  Programmes,  especially  for  projects  that  raise 
ethical  questions,  as  these  committees  can  offer  substantial  support  in  the  application 
process. In addition, the OPC on the past, present and future of the European Research and 
Innovation Framework Programmes 2014-2017 revealed that the new rules are not always 
clearly communicated, and are sometimes deemed too complex. This finding is in line with a 
2021  study,  which  showed  a  discrepancy  between  the  understanding  of  ethics  issues 
displayed  by  the  applicants  of  Horizon  2020  projects,  and  the  understanding  of  expert 
reviewers.  As  outlined  in  Section  1.2.2,  more  information  and  guidance  (documents)  on 
complying with ethics and integrity requirements have been provided under Horizon Europe, 
and its risk-based approach has simplified the procedure for most applications (i.e. those that 
do  not  pose  a  considerable  ethical  risk).  The  survey  of  Horizon  Europe  beneficiaries, 
conducted  May-July  2023,  asked  about  beneficiaries’  experiences  with  the  ethics  self-
assessment  procedure.  Its  findings  revealed  that  overall,  their  experiences  with  this 
procedure were positive a large or moderate extent, with only a small minority stating that 
they had ‘negative’ or ‘rather negative’ experiences (see Figure 3). 

In the same survey, respondents indicated a favourable opinion of the support provided to 
carry out self-assessments, which was most commonly classified as good, with only a small 
minority of respondents deeming the support to be poor (see Figure 4). Even though some 
concerns  were  expressed  regarding 
imposed  by  horizontal 
requirements  during  the  application  stage  (see  above),  the  survey  results  thus  appear  to 
indicate  that  the  ethics  self-assessment  process  in  itself  is  clear  and  sufficiently  well 
supported. 

the  additional  burden 

1053 Gans Combe, C., Faucheux, S., Kuszla, C., Petousi, V., & Garani, T. DEFORM Main 
Findings – A Policy Report (January 09, 2019), available at SSRN: 
https://ssrn.com/abstract=3389414 or http://dx.doi.org/10.2139/ssrn.3389414. 

674 

 
 
What is your experience with the ethics self-assessment in your 
project? 

60%

50%

40%

30%

20%

10%

0%

48,5%

48,3%

56,4%

21,1%

21,0%

16,2%

43,1%

48,5%

26,8%

25,7%

38,4%

30,3%

5,9%

3,8%

2,8%

0,2%

1,6%

0,1%

0,7%

0,0%

3,5%

0,0%

2,0%

0,0%

ERC PIs (237
responses)

MSCA Orgs
(472
responses)

MSCA PFs
(815
responses)

INFRA (153
responses)

WIDERA (171
responses)

WIDERA (ERA)
(99 responses)

Positive to a large extent (no issues faced)

Positive to a moderate extent (clear, minor challenges were encountered
but resolved)
Rather negative (faced some issues)

Figure 3. Assessment of beneficiaries’ experiences with the ethics self-assessment. Source: survey of Horizon Europe 
beneficiaries, conducted May-July 2023. 

How would you rate the support (e.g. information, guidance) available 
to you in carrying out the ethics self-assessment of your project?

50,6%

48,6%

44,4%

56,1%

41,4%

60%

50%

40%

36,9%

30%

24,6%

20%

10%

0%

19,8%

10,2%

8,6%

15,3%

10,3%

2,4%

12,4%

7,0%

2,3%

18,1%

17,1%

19,7%

2,9%

4,5%

1,5%

17,1%

8,6%

5,7%

ERC PIs (178
responses)

MSCA Orgs
(340
responses)

MSCA PFs
(599
responses)

INFRA (105
responses)

WIDERA (132
responses)

WIDERA (ERA)
(70 responses)

Excellent

Good

Fair

Poor

Do not know/cannot answer

Figure 4. Beneficiaries’ assessments of support (from the European Commission) in carrying out the ethics self-assessment. 
Source: survey of Horizon Europe beneficiaries, conducted May-July 2023. 

Some  interviewees  also  indicated  that  the  current  procedure  is  better  at  differentiating 
between the majority of projects in which no thorny issues are present, and those that do 
present such issues, and that this lowers the overall burden on applicants in the majority of 
projects (for instance, those in which data protection issues exist, but these are not overly 
complex or sensitive). 

675 

 
 
 
Other interviewees voiced concerns that the appointment of an ethics advisor or board does 
not in itself guarantee compliance with ethical standards and legal obligations – especially if 
their  recommendations  are  not  taken  to  heart.  These  interviewees  called  for  a  more 
prescriptive approach, with more detail about the steps necessary to ensure compliance. 

In addition to these results, the survey asked applicants – for ERC and MSCA programme 
parts  only  –  about  the  impact  that  participation  in  Horizon  2020  projects  had  on  the 
development of their skills. One such question concerned their skills in relation to research 
ethics  and  research  integrity.  As  Figure  5  shows,  participation  in  the  ERC, and  especially 
MCSA, supports the development of such skills – often to a large or very large extent (with 
233  and  814  respondents,  respectively).  In  response  to  a  different  survey  question  (not 
depicted), one in four applicants (25.3%, or 210 out of 814 respondents) reported that they 
received  training  or  planned  to  receive  training  in  research  ethics  within  the  framework  of 
their MSCA project. 

To what extent, if at all, will participation in your project have an impact 
on the development of your skills in research ethics and integrity?

30,3%

30,7%

21,9%

20,6%

24,9%

22,9%

35%

30%

25%

20%

15%

10%

5%

0%

15,0%

5,4%

10,3%

8,6%

7,3%

2,1%

To a very large
extent

To a large
extent

To a moderate
extent

To a small
extent

Not at all

Do not know/not
applicable

MCSA PFs

ERC PIs

Figure 5. Assessment of the impact of participation in MSCA or ERC projects on the development of research ethics and 
integrity skills. Source: Survey of Horizon Europe beneficiaries, conducted May-July 2023. 

1.3.5.  EU added value 

Efforts to promote ethics and integrity at EU level are welcome, but ‘one-size-fits-all’ 
frameworks should be avoided 

A first rationale for developing ‘a common language’ for research integrity and ethics is that 
most  scientific  research  involves  partners  from  various  EU  countries  and  beyond,  posing 
potential  risks  of ‘integrity  dumping’  in countries  with  weak  standards  and  policies.  This  is 
especially  relevant  in  the  case  of  projects  that  benefit  from  support  at  EU  level,  as  the 
European Commission applies standards for research applications for all applications across 
the  Union,  indicating  the  need  for  a  uniform  and  harmonised  framework  (i.e.  the  Ethics 
Appraisal Procedure for Horizon Europe-funded projects) that can also inspire projects that 
do not benefit from EU funding.  

Many countries in the EU and beyond (including middle- and low-income countries) grapple 
with similar issues in relation to ethics and integrity in research. This means that there are 

676 

 
 
probably economies of scale that can be developed, as well as good practices that can be 
shared  with  regard  to  ethical  codes  and  the  conducting  of  communication  and  outreach 
activities. In addition, several interviewees highlighted that it would be difficult to adequately 
fund projects in this area in the absence of support at European level. National governments 
would reportedly find it difficult to allocate funding for projects with the level of ambition and 
scale  possible  under  the  Framework  Programmes.  Lastly,  EU  projects  are  able  to  bring 
together expertise from across the EU-27, which is hard to replicate at national level. While 
some initiatives do take place at national level, these reportedly lack the scale and ambition 
of  the  Horizon  projects.  Interviewees  also  pointed  out  that,  thanks  to  the  efforts  of  the 
European  Commission,  the  EU  could  become  a  global  frontrunner  in  setting  and 
implementing ethics and integrity standards in research.  

Furthermore, the ethical codes that are developed at national level are sometimes inspired 
by publications by international organisations such as the European Science Foundation, the 
OECD or the European Commission. For instance, the ‘Codes of ethics for scientific research 
in Belgium’ explicitly refer to these international organisations as being instrumental in their 
development. While no reference is made within the documents themselves, the Estonian 
code of conduct and its Greek counterpart are both reportedly influenced by efforts made as 
part of Horizon 2020, according to European Commission sources. In these three cases, the 
adoption of a code of conduct inspired (at least in part) by work developed under Horizon 
2020, enables them to be classified as an institutional change. In other cases, it is unclear 
whether the efforts of the European Commission have played a role, or how important these 
were. Another example relates to awareness of ethics dumping as an issue, especially with 
regard  to  projects  that  involve  partners  from  emerging  economies.  The  Trust  Code  has 
developed guidelines in this area that have so far been adopted by around 20 organisations. 
For this reason, some interviewees indicated that it makes sense to develop ethical codes or 
standards and similar learning materials at EU level, rather than at the level of Member States 
or  individual  research  organisations.  One  major  achievement  in  the  area  of  such  EU 
leadership is the revision of the European Code of Conduct for Research Integrity (ECoC), 
which was led by the All European Academies (Allea) in collaboration with various experts 
and  stakeholders,  but  in  which  the  European  Commission  played  an  important  catalysing 
role. The European Code of Conduct for Research Integrity was also included in the 2021 
Council Conclusions on a ‘Global approach to Research and Innovation – Europe’s strategy 
for international cooperation in a changing world’. 

At the same time, recognition should be given to efforts conducted at the national (or even 
sub-national)  level  of  government,  as  well  as  by  individual  universities  and  research 
organisations. According to some interviewees, a common framework on research ethics and 
integrity at EU level may not be desirable, as this might not reflect the diversity of practices 
across  all  Member  States  (one  example  being  their  differing  approaches  to  dealing  with 
plagiarism).  However,  many  EU  countries 
lack  national  codes,  guidelines  or 
recommendations in relation to research ethics and integrity. Table 2 in Appendix 2 provides 
an overview of national guidelines or codes on ethics and integrity among the EU Member 
States. 

1.4.  Conclusions  

The projects funded on research ethics and research integrity under Horizon 2020 have made 
a meaningful contribution to this field of research, in terms of raising awareness of the main 
issues  and  generating  novel  insights.  There  was  a  perception  among  interviewees  that 
funding and support from the Commission had enabled projects to be more ambitious than 
might otherwise have been the case. 

677 

 
One important result of EU funding is the development of networks of experts on the topic of 
research ethics and integrity. More specifically, it is encouraging that some of these networks 
have remained active even after the funding ended. This points to a long-term impact. At the 
same time, the geographical coverage of these networks remains uneven and some concern 
is expressed about an over-reliance on a relatively small pool of experts who will participate 
in future projects. 

The  relevant  work  conducted  under  Horizon  2020  can  and  does  serve  as  guidance  for 
organisations  at  international,  national  and  sub-national  levels,  thanks  to  the  manifold 
learning materials, guidance documents and other tools that have been developed. There is 
some evidence that the framework developed has inspired the development of ethical codes 
and similar documents in the Member States, and some of the training modules have been 
applied in EU countries. 

However, the evidence base for the impact of the various tools developed remains weak, with 
relatively  few  cases  of  institutional  change  being  documented  as  resulting  from  the  work 
developed under Horizon 2020 projects in this area. Systematically collecting the institutional 
changes that have resulted from Framework Programme projects on research integrity and 
ethics would enable the identification of good practices and areas for future work, and could 
guide further work. This would also facilitate the assessment of the relevant Horizon 2020 
projects.  As  one  case  in  point,  it  is  hard  to  assess  the  extent  to  which  codes  of  conduct 
developed  at  the  national  level  have  been  inspired  by  the  work  developed  by  the 
Commission. 

In  this  respect,  the  sub-optimal  dissemination  of  insights  from  the  various  projects  on 
research ethics and integrity across universities, research organisations and funding bodies 
could be considered a weakness or a potential area of focus for future endeavours. Various 
interviewees  expressed  the  sentiment  that  more  could  be  done  to  reach  out  to  potential 
stakeholders  and  set  up  collaborations  in  order  to  realise  change  on  the  ground,  even  in 
countries  with  a  relatively  well  established  research  culture.  In  other  words,  the  potential 
impact  achieved  by making broader and  more  frequent  use  of  the  tools  that  have  already 
been developed could be greater than that of developing additional materials (especially if 
such materials are technocratic in nature and therefore hard to apply by policy-makers – a 
related concern among some interviewees). One concrete example would be to further enrich 
the Embassy of Good Science website with additional content, cementing its position as the 
central  repository  and  strengthening  its  brand  among  the  research  community  and  other 
relevant actors.  

In a similar spirit, projects with a strong focus on how to implement certain tools are perceived 
to be welcome. Researchers often appear to lack the necessary skills in relation to ethics and 
integrity in science, in part due to a dearth of formal training courses. More attention being 
given to these issues in formal courses would pay dividends in the long run.  

Lastly, the procedure used to submit proposals under the Framework Programme  “ethics-
ready’ has been the subject of changes under Horizon Europe, among other things to make 
it more streamlined for the majority of proposals in which there are no obvious ethical issues 
at play, and to provide more guidance to applicants. While some concerns remain with regard 
to the added burden of this procedure overall, the support for the self-assessment is viewed 
favourably by successful applicants. 

678 

 
 
Appendix 1. Literature sources 

•  Allea, The European Code of Conduct for Research Integrity, 2023, available at: 

https://allea.org/wp-content/uploads/2023/06/European-Code-of-Conduct-Revised-
Edition-2023.pdf. 

•  Allum, N., Reid, A., Bidoglia, M., Gaskell, G., Aubert-Bonn, N., Buljan, I., Fuglsang, 
S., Horbach, S., Kavouras, P., Marušić, A., Mejlgaard, N., Pizzolato, D., Roje, R., 
Tijdink, J., & Veltri, G., Researchers on research integrity: a survey of European 
and American researchers. F1000Res. 2023 Feb 16;12:187, doi: 
10.12688/f1000research.128733.1. 

•  Austrian Agency for Research Integrity website, https://oeawi.at/en/training-

overview. 

•  Codes of ethics for scientific research in Belgium, 

https://research.kuleuven.be/en/integrity-ethics/integrity/practices/belspo-code. 

•  Cordis EU Research Results webpage, Creating and enhancing TRUSTworthy, 

responsible and equitable partnerships in international research, 
https://cordis.europa.eu/project/id/664771. 

•  Cordis EU Research Results webpage, Define the global and financial impact of 

research misconduct, https://cordis.europa.eu/project/id/710246. 

•  Cordis EU Research Results webpage, European Network of Research Ethics and 

Research Integrity, https://cordis.europa.eu/project/id/710184.  

•  Cordis EU Research Results webpage, INTEGRITY: empowering students through 

evidence-based, scaffolded learning of Responsible Conduct in Research (RCR), 
https://cordis.europa.eu/project/id/824586. 

•  Cordis EU Research Results webpage, Mapping Normative Frameworks for Ethics 

and Integrity of Research, https://cordis.europa.eu/project/id/741782. 

•  Cordis EU Research Results webpage, Shaping the ethical dimensions of smart 

information systems (SIS) – a European perspective, 
https://cordis.europa.eu/project/id/786641. 

•  Cordis EU Research Results webpage, Stakeholder-informed ethics for new 

technologies with high socio-economic and human rights impact, 
https://cordis.europa.eu/project/id/741716. 

•  Cordis EU Research Results webpage, Virtue based ethics and Integrity of 

Research: Train-the-Trainer program for Upholding the principles and practices of 
the European Code of Conduct for Research Integrity, 
https://cordis.europa.eu/project/id/787580. 

•  Council Conclusions on a Global approach to Research and Innovation – Europe's 

strategy for international cooperation in a changing world, adopted on 28 

679 

 
September 2023, available at: https://data.consilium.europa.eu/doc/document/ST-
12301-2021-INIT/en/pdf.  

•  De Waele, I., Wizel, D., Puljak, L., & Koporc, Z. (2021), Ethics appraisal procedure 
in 79,670 Marie Skłodowska-Curie proposals from the entire European HORIZON 
2020 research and innovation program (2014–2020): A retrospective analysis, 
PLoS ONE 16(11): e0259582, available at: 
https://doi.org/10.1371/journal.pone.0259582. 

•  Embassy of Science Website, https://www.embassy.science.  

•  EUR-Lex, Regulation (EU) 2021/695 of the European Parliament and of the 

Council of 28 April 2021 establishing Horizon Europe – the Framework Programme 
for Research and Innovation, laying down its rules for participation and 
dissemination, and repealing Regulations (EU) No 1290/2013 and (EU) No 
1291/2013 (Text with EEA relevance), available at: https://eur-
lex.europa.eu/eli/reg/2021/695/oj.  

•  European Commission, Ethics Guidelines for Trustworthy AI, 2019, available at: 

https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai. 

•  European Commission, Communication from the Commission to the European 
Parliament, the Council, the European Economic and Social Committee and the 
Committee of the Regions A new ERA for Research and Innovation, 30 September 
2020, COM (2020) 628 final, available at: EUR-Lex - 52020DC0628 - EN - EUR-
Lex (europa.eu). 

•  European Commission, Directorate-General for Research and Innovation, 

Monachello, R., Delaney, N., & Tornasi, Z., Research ethics and research integrity: 
achievements in Horizon 2020 and recommendations on the way forward, 
Delaney, N. (editor), Tornasi, Z. (editor), Luxembourg: Publications Office of the 
European Union, 2020, available at: https://data.europa.eu/doi/10.2777/63976. 

•  European Commission, Directorate-General for Research and Innovation, Ethics 
for Researchers: Facilitating Research Excellence in FP7, available at: ethics-for-
researchers_en.pdf (europa.eu). 

•  European Commission, Directorate-General for Research and 

Innovation, European Research Area policy agenda: overview of actions for the 
period 2022-2024, Luxembourg: Publications Office of the European Union, 2022, 
available at: https://data.europa.eu/doi/10.2777/52110 
https://data.europa.eu/doi/10.2777/52110. 

•  European Commission, Directorate-General for Research and Innovation, 

Almerud, M., Ricksten, M., O’Neill, G., et al., Knowledge ecosystems in the new 
ERA : using a competence-based approach for career development in academia 
and beyond, Núñez, L., De Coen, A. (eds.), Luxembourg: Publications Office of the 
European Union, 2022, available at: https://data.europa.eu/doi/10.2777/150763. 

680 

 
•  European Commission, Directorate-General for Research and Innovation, A new 
horizon for Europe: impact assessment of the 9th EU framework programme for 
research and innovation, Luxembourg: Publications Office of the European Union, 
2018, available at: https://data.europa.eu/doi/10.2777/194210. 

•  European Commission, Directorate-General for Research and Innovation, Interim 
evaluation of Horizon 2020: Commission staff working document, Luxembourg: 
Publications Office of the European Union, 2017, available 
at:  https://data.europa.eu/doi/10.2777/220768. 

•  European Commission, Directorate-General for Research and Innovation, Kadunc, 
M., & Bruno, N., Keeping our eyes on the Horizon : monitoring flash series : a 
Horizon 2020 monitoring report, Kadunc, M., Bruno, N. (eds.), Luxembourg: 
Publications Office of the European Union, 2020, available 
at:  https://data.europa.eu/doi/10.2777/303024. 

•  European Commission, Directorate-General for Research and Innovation, Horizon 

Europe: Strategic Plan 2021-2024, Luxembourg: Publications Office of the 
European Union, 2021, available at: https://data.europa.eu/doi/10.2777/083753. 

•  European Commission, Directorate-General for Research and Innovation, 

Monachello, R., Delaney, N., & Tornasi, Z., Research ethics and research integrity 
: achievements in Horizon 2020 and recommendations on the way forward, 
Delaney, N., Tornasi, Z. (eds.), Luxembourg: Publications Office of the European 
Union, 2020, available at: https://data.europa.eu/doi/10.2777/63976. 

•  European Commission, Directorate-General for Research and Innovation, 

Delaney, N., Iagher, R., & Tornasi, Z., Institutional changes towards responsible 
research and innovation : achievements in Horizon 2020 and recommendations on 
the way forward, Luxembourg: Publications Office of the European Union, 2020, 
available at: https://data.europa.eu/doi/10.2777/682661. 

•  European Commission, Directorate-General for Research and Innovation, Iagher, 
R., Monachello, R., Warin, C., et al., Science with and for society in Horizon 2020: 
achievements and recommendations for Horizon Europe, Delaney, N., Tornasi, Z. 
(eds.), Luxembourg: Publications Office of the European Union, 2020, available 
at: https://data.europa.eu/doi/10.2777/32018. 

•  European Commission, EU Grants: How to complete your ethics self-assessment, 
available at: https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/common/guidance/how-to-complete-your-ethics-self-assessment_en.pdf. 

•  European Commission, Grants Manual - Section on: Proposal submission and 

evaluation, available at: 
https://ec.europa.eu/research/participants/data/ref/h2020/grants_manual/pse/h202
0-guide-pse_en.pdf. 

•  European Commission, Horizon 2020 Online Manual, available at: 

https://ec.europa.eu/research/participants/docs/h2020-funding-guide/cross-cutting-
issues/ethics_en.htm. 

681 

 
•  European Commission, Horizon Europe (HORIZON), Programme Guide, available 

at: https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/guidance/programme-guide_horizon_en.pdf. 

•  European Commission, Horizon Europe Strategic Plan 2025-2027 analysis, 

available at: https://research-and-innovation.ec.europa.eu/funding/funding-
opportunities/funding-programmes-and-open-calls/horizon-europe/strategic-
plan_en. 

•  European Commission, The Pact for Research and Innovation in Europe, available 
at: https://research-and-innovation.ec.europa.eu/news/all-research-and-innovation-
news/commission-adopts-proposal-pact-research-and-innovation-europe-2021-07-
16_en. 

•  European Commission; Horizon Europe's first Strategic Plan 2021-2024, available 

at: https://ec.europa.eu/commission/presscorner/detail/en/IP_21_1122. 

•  European Commission, Guidelines on Serious and Complex Cases, Horizon 2021-

2027 Online Manual, available at: https://ec.europa.eu/info/funding-
tenders/opportunities/docs/2021-2027/horizon/guidance/guidelines-on-serious-
and-complex-cases_he_en.pdf. 

•  European Commission, Guidance on How to Handle Security-Sensitive Projects, 
Horizon 2021-2027 Online Manual, available at: https://ec.europa.eu/info/funding-
tenders/opportunities/docs/2021-2027/common/guidance/how-to-handle-security-
sensitive-projects_en.pdf. 

•  European Commission, Ethical Guidance for Research with a Potential for Human 

Enhancement, Horizon 2021-2027 Online Manual, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/guidance/ethical-guidance-for-research-with-a-potential-for-human-
enhancement-sienna_he_en.pdf. 

•  European Commission, 2021, Ethics by Design and Ethics of Use Approaches for 

Artificial Intelligence, Horizon 2021-2027 Online Manual, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/guidance/ethics-by-design-and-ethics-of-use-approaches-for-
artificial-intelligence_he_en.pdf. 

•  European Commission, Guidance note – Research with an exclusive focus on civil 

applications (2021), available at: https://ec.europa.eu/info/funding-
tenders/opportunities/docs/2021-2027/horizon/guidance/guidance-note-research-
focusing-exclusively-on-civil-applications_he_en.pdf. 

•  European Commission, Guidance note – Potential misuse of research (2021), 
available at: https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/guidance/guidance-note-potential-misuse-of-research-
results_he_en.pdf.   

682 

 
•  European Commission, Widening Participation and Spreading Excellence, 

available at: https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/wp-call/2023-2024/wp-11-widening-participation-and-strengthening-
the-european-research-area_horizon-2023-2024_en.pdf. 

•  European Parliament and the Council (2021), Regulation (EU) 2021/695 of the 
European Parliament and of the Council of 28 April 2021 establishing Horizon 
Europe – the Framework Programme for Research and Innovation, laying down its 
rules for participation and dissemination, and repealing Regulations (EU) No 
1290/2013 and (EU) No 1291/2013, 28 April 2021, PE/12/2021/INIT, OJ L 170, 
12.5.2021, pp. 1–68, available at: http://data.europa.eu/eli/reg/2021/695/oj. 

•  European Union, Charter of Fundamental Rights of the European Union, 26 

October 2012, 2012/C 326/02, OJ C 326, 26.10.2012, pp. 391–407, available at: 
http://data.europa.eu/eli/treaty/char_2012/oj  

•  Fair Research Contracting Website, http://frcweb.cohred.org. 

•  Gans Combe, C., Faucheux, S., Kuszla, C., Petousi, V., & Garani, T., DEFORM Main 

Findings – A Policy Report (January 09, 2019), available at SSRN: 
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3389414. 

•  Hybrida, Embedding a comprehensive ethical dimension to organoid-based 

research and resulting technologies, https://hybrida-project.eu/ 

•  Mejlgaard N., Bouter, L.M., Gaskell, G., Kavouras, P., Allum, N., Bendtsen, A.K., 

Charitidis, C.A., Claesen, N., Dierickx, K., Domaradzka, A., Reyes Elizondo, A., 
Foeger, N., Hiney, M., Kaltenbrunner, W., Labib, K., Marušić, A., Sørensen, M.P., 
Ravn, T., Ščepanović, R., Tijdink, J.K., Veltri, G.A.. Research integrity: nine ways 
to move from talk to walk, Nature 2020 Oct;586(7829):358-360, doi: 
10.1038/d41586-020-02847-8. PMID: 33041342. 

•  Nature, Nature addresses helicopter research and ethics dumping, Nature, 2022 

Jun;606(7912):7. doi: 10.1038/d41586-022-01423-6. PMID: 35637322. 

•  NPCFlanders, Report Science with and for society in Horizon 2020 - achievements 

and recommendations for Horizon Europe, available at: 
https://ncpflanders.be/documents/report-science-with-and-for-society-in-horizon-
2020-achievements-and-recommendations-for-horizon-europe. 

•  Prepared Project, The TRUST Code – A Global Code of Conduct for Equitable 
Research Partnerships, 2018, available at: https://prepared-project.eu/the-trust-
code/. 

•  Primanova.com, Artificial Intelligence and Ethics in EU-funded Projects, available at: 
https://www.privanova.com/resources/artificial-intelligence-and-ethics-in-eu-
funded-projects.  

•  Printeger – Promoting Integrity as an Integral Dimension of Excellence in 

Research, https://printeger.eu/. 

683 

 
•  ROSIE – Fostering responsible Open Science in Europe, https://rosie-project.eu/. 

•  SOPs4RI – Standard Operating Procedures for Research Integrity, 

https://sops4ri.eu/. 

•  Synopsis report of the public consultation on the past, present and future of the 

European research and innovation Framework Programmes 2014-2027, available at: 
https://op.europa.eu/en/publication-detail/-/publication/e8f722ec-dff7-11ed-a05c-
01aa75ed71a1/language-en/format-PDF/source-284554549. 

•  The TRUST Code: A Global Code of Conduct for Equitable Research 

Partnerships, http://www.globalcodeofconduct.org/. 

684 

 
 
 
Appendix 2. Additional information 

Application of the Ethics Appraisal Scheme 

During the application phase, applicants must fill in the ethics issues table – a checklist of 
questions  with  yes/no  answers.  If  the  answer  to  one  of  the  questions  is  ’yes’,  a  self-
assessment  is  mandatory.  In  this  self-assessment,  applicants  must  describe  the  ethical 
dimensions  of  the  project  and  its  compliance  with  ethical  standards  (by  filling  in  an  IT-
generated document).  

During  the  evaluation  phase,  a  pre-screening  may  take  place  (which  is  optional,  and 
relevant in particular to calls that generally pose no considerable issues according to the self-
assessments  submitted).  During  this  phase,  at  least  two  evaluators  (qualified  staff  and/or 
independent ethics experts) evaluate the proposed activities. If one or both of the evaluators 
finds that there are ethics issues, the proposal moves to the screening phase. If no issues 
are identified, the application receives ethics clearance. 

The ethics screening is carried out on those applications on which no pre-screening took 
place, and those for which, during the pre-screening, ethical issues were identified by at least 
two ethics evaluators. The screening assesses the ethical aspects of the project’s objectives, 
methodology and potential impacts so as to identify proposals with serious or complex issues. 
For  proposals  that  raise  serious  and/or  complex  ethics  issues  (e.g.  in  case  of  severe 
intervention on humans, the lack of an appropriate ethics framework in the country in which 
the  research  will  be  performed,  etc.)  and/or  when  human  embryonic  stem  cells  or  human 
embryos are involved, the ethics screening is followed by an ethics assessment prior to the 
signature of the grant agreement. This assessment is a more in-depth analysis by experts of 
the ethical issues at hand. It focuses on compliance with ethical rules and standards, relevant 
European legislation, international conventions and declarations, national authorisations and 
ethics approvals, the proportionality of the research methods proposed and the applicants’ 
awareness  of  the  ethical  aspects  and  social  impact  of  their  planned  research1054.  The 
European Commission provides dedicated guidance for such applications1055.  

In  cases  where  no  ethical  issues  are  identified  throughout  the  aforementioned  process, 
clearance  is  awarded  and  no  further  action  is  required.  Proposals  that  give  rise  to  ethical 
questions but which address them appropriately in the proposal also receive clearance with 
no further action required. Sometimes the clearance is conditional. In such cases, the ethics 
summary  report  will  list  one  or  more  ethics  requirements,  which  become  contractual 
obligations. The latter is only the case for serious and/or complex ethics issues or when an 
external,  independent  ethics  advisor  or  board  is  requested.  Pre-grant  ethics  requirements 
must  be  fulfilled  prior  to  the  signing  of  the  grant  agreement.  Post-grant  requirements  are 
included in the form of deliverables that must be submitted. Such post-grant requirements 
may  include  the  appointment  of  an  ethical  expert  or  board  that  regularly  reports  to  the 
Commission or Executive Agencies. Lastly, projects are rejected for funding in cases where 
no ethics clearance is awarded. 

1054 European Commission, Horizon 2020 Online Manual, available at: 
https://ec.europa.eu/research/participants/docs/h2020-funding-guide/cross-cutting-
issues/ethics_en.htm. 
1055 https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/guidance/guidelines-on-serious-and-complex-cases_he_en.pdf. 

685 

 
 
 
 
Horizon 2020 projects related to ethics and integrity in research 

Call ID 

Start/ 
end 
year 

Project name 

Project 
acronym 

Net EU 
contribution 
(EUR) 

Number of 
participants 

H2020-GARRI-
1-2014 

2015-
2018 

H2020-GARRI-
2-2014 

2015-
2018 

H2020-GARRI-
1-2015 

2016-
2018 

H2020-GARRI-
1-2015 

2016-
2019 

H2020-SwafS-
16-2016 

2017-
2021 

H2020-SwafS-
17-2016 

2017-
2021 

H2020-SwafS-
18-2016 

2017-
2021 

Promoting Integrity 
as an Integral 
Dimension of 
Excellence in 
Research 

Creating and 
enhancing 
TRUSTworthy, 
responsible and 
equitable 
partnerships in 
international 
research 

Define the global 
and financial 
impact of research 
misconduct 

European Network 
of Research Ethics 
and Research 
Integrity 

Mapping 
Normative 
Frameworks for 
Ethics and 
Integrity of 
Research 

Improving the 
guidelines for 
Informed Consent, 
including 
vulnerable 
populations, under 
a gender 
perspective 

Stakeholder-
informed ethics for 
new technologies 
with high socio-
economic and 
human rights 
impact 

PRINTEGER 

1,978,780 

8 

TRUST 

2,141,173 

13 

DEFORM 

999,713 

2 

ENERI 

1,499,000 

12 

EnTIRE 

3,770,000 

12 

I-CONSENT 

3,077,398 

11 

SIENNA 

3,996,778 

14 

686 

 
 
Call ID 

Start/ 
end 
year 

Project name 

Project 
acronym 

Net EU 
contribution 
(EUR) 

Number of 
participants 

H2020-SwafS-
21-2017 

2018-
2021 

H2020-SwafS-
22-2017 

2018-
2021 

H2020-SwafS-
22-2017 

2018-
2021 

H2020-SwafS-
27-2017 

2018-
2021 

H2020-SwafS-
02-2018 

2019-
2021 

H2020-SwafS-
02-2018 

2019-
2021 

H2020-SwafS-
03-2018 

2019-
2022 

PROmoting 
integrity in the use 
of RESearch 
results 

Shaping the 
ethical dimensions 
of smart 
information 
systems (SIS)  – a 
European 
perspective 

Participatory 
Approaches to a 
New Ethical and 
Legal Framework 
for ICT 

Virtue based 
ethics and Integrity 
of Research: 
Train-the-Trainer 
program for 
Upholding the 
principles and 
practices of the 
European Code of 
Conduct for 
Research Integrity 

Rotatory role-
playing and role-
models to 
enhance the 
research integrity 
culture 

INTEGRITY: 
empowering 
students through 
evidence-based, 
scaffolded learning 
of Responsible 
Conduct in 
Research (RCR) 

Standard 
Operating 
Procedures for 
Research Integrity 

PRO-RES 

2,800,000 

14 

SHERPA 

2,865,948 

12 

PANELFIT 

2,789,031 

15 

VIR2TUE 

2,800,000 

14 

Path2Integrity 

2,239,549 

8 

INTEGRITY 

2,497,036 

11 

SOPs4RI 

3,999,860 

13 

687 

 
Call ID 

Start/ 
end 
year 

Project name 

Project 
acronym 

Net EU 
contribution 
(EUR) 

Number of 
participants 

Develop the 
governance for the 
advancement of 
responsible 
research and 
innovation by all 
stakeholders, 
which is sensitive 
to society needs 
and demands and 
promote an ethics 
framework for 
research and 
innovation 

Embedding a 
comprehensive 
ethical dimension 
to organoid-based 
research and 
resulting 
technologies 

Ethics for 
Technologies with 
High Socio-
Economic Impact 

H2020-SwafS-
16-2019 

2020-
2023 

H2020-SwafS-
28-2020 

2021-
2024 

H2020-SwafS-
29-2020 

2021-
2023 

PRO-ETHICS 

2,987,934 

15 

HYBRIDA 

2,995,785 

8 

TechEthos 

3,994,593 

17 

H2020-SwafS-
30-2020 

2021-
2024 

Responsible Open 
Science in Europe 

ROSiE 

2,227,961 

12 

Table 1. Overview of Horizon 2020 projects related to ethics and integrity in research. Source: Horizon Dashboard, 
accessed 7-8 July 2023. 

Overview of national guidelines or codes on ethics and integrity 

EU country 

Document 

Link 

Austria 

Austrian Agency for Research 
Integrity 
Guidelines for Good Scientific 
Practice 

OeAWI_Guidelines-for-Good-Scientific-
Practise.pdf (eneri.eu) 

Belgium 

Codes of ethics for scientific 
research in Belgium 

belspo-code (kuleuven.be) 

Denmark 

The Danish Code of Conduct for 
Research Integrity 

Guidelines for Good Scientific 
Practice 

https://eneri.eu/wp-
content/uploads/2018/10/The-danish-
code-of-conduct-for-research-integrity-
pub.pdf 

688 

 
 
EU country 

Document 

Link 

Estonia 

Code of Ethics for Estonian 
Scientists 

Estonian Code of Conduct for 
Research Integrity 

UVVU GVP ENG 15052009.doc 
(ufm.dk) 

https://www.akadeemia.ee/_repository/fil
e/ALUSDOKUD/Code_ethics2002.pdf 
https://www.eetika.ee/en/ethics-
estonia/estonian-code-conduct-
research-integrity 

Finland 

Responsible conduct of research 
and procedures for handling 
allegations of misconduct in Finland 

HTK_ohje_2012.pdf (tenk.fi) 

France 

Integrity and responsibility in 
research practices 

http://www.cnrs.fr/comets/IMG/pdf/guide
2017-en.pdf 

Germany 

Recommendations on Academic 
Integrity 

https://www.wissenschaftsrat.de/downlo
ad/archiv/4609-15_engl.pdf 

Greece 

Code of ethics and conduct of 
scientific research foundation for 
research and technology – Hellas 

https://www.forth.gr/files/Code_of_Cond
uct_for_Research_FORTH_ENG.pdf 

Ireland 

National policy statement on 
ensuring research integrity in Ireland 

Netherlands 

Code of conduct for research 
integrity 

https://hea.ie/assets/uploads/2017/04/Na
tional-Policy-Statement-on-Ensuring-
Research-Integrity-in-Ireland-2014.pdf 

Netherlands2BCode2Bof2BConduct2Bfo
r2BResearch2BIntegrity_2018_UK.pdf 
(eneri.eu) 

Poland 

Good Manners in Science. A Set of 
Principles and Guidelines 

https://ken.pan.pl/images/stories/pliki/go
odmanners.pdf 

Good scientific research practice  
Recommendations 

http://lms.lt/archyvas/files/active/0/polish
_reserach_practice.pdf 

Spain 

Code of Good Scientific Practices of 
CSIC 

Sweden 

Good Research Practice 

http://enacti2ng-
itn.cbm.uam.es/images/Documentation/
CBP_CSIC_en.pdf 

https://www.vr.se/download/18.5639980
c162791bbfe697882/1555334908942/G
ood-Research-Practice_VR_2017.pdf 

Table 2. Overview of national guidelines or codes on ethics and integrity in the EU Member States. Source: ENERI.eu and 
desk research. 

689 

 
 
 
 
 
Annex 2.15: Co-design and co-creation of R&I through citizen 
engagement 

1.1. 

Introduction 

The engagement of citizens and other non-traditional research actors is crucial in tackling 
societal challenges, as well as to create value for all parts of society. Innovative solutions, 
based  on  Europe’s  base  of  excellent  science  and  research,  can  contribute  to  improving 
people’s  lives,  from  everyday  problems  to  the  green  and  digital  transitions  and  economic 
development.  Hence,  people  are  at  the  heart  of  the  EU’s  research  and  innovation  (R&I) 
agenda.  Research  conducted  in  Phase  1  of  the  study  has  revealed  that  Horizon  2020  (in 
particular the ‘Science with and for Society’ programme, also referred to as SwafS) provided 
EU added value by enhancing various aspects of interactions between science and society, 
including science education, science communication, citizen science and citizen engagement 
in  science.  However,  the  available  evidence  also  indicates  a  need  to  further  develop 
methodologies for measuring the impact of such actions.  

The  ambition  of  this  case  study  is  to  analyse  the  modalities  for  implementing  citizens’ 
engagement  under  Horizon  Europe  and,  more  specifically,  to  gain  insights  into  how  the 
programme encourages non-traditional R&I actors (most notably citizens) to work together 
with the research community throughout the R&I process. Within this scope, the case study 
will  examine  the  innovative  mechanisms  through  which  such  approaches  have  been 
implemented, as well as any effects that are visible so far, the (expected) impact of these 
actions in the short to medium term, and the drivers and barriers that could further affect this 
impact. The key added value of the present case study within the evaluation as a whole is 
that  it  will  provide  a  concise  analysis  and  a  summary  –  together  with  qualitative  and 
quantitative evidence – regarding the modalities of the Horizon Europe programme in terms 
of spreading excellent science, tackling global challenges and promoting innovation through 
citizen engagement mechanisms. The case study has been implemented through extensive 
desk research and analysis of EC monitoring and survey data. In addition, interviews have 
been carried out with key stakeholders to gain a more structured view of citizen engagement 
activities under the Horizon Europe Framework Programme (FP).  

1.2.  Background information 

1.2.1.  Policy context 

The  origins  of  citizen  engagement  are  documented  in  a  2001  EC  White  Paper  on 
governance1056  that  signalled  an  ambition  to  better  connect  democratic  institutions  with 
European citizens in response to a crisis of legitimacy with regard to institutions and scientific 
expertise. In this context, the Commission called for more socially robust science and greater 
involvement of citizens in policy-making and implementation. Consequently, EU institutions 
have  fostered  a  variety  of  participatory  and  deliberative  practices  to  enhance  citizens’ 
engagement in EU decision-making and policy-making1057. One practical response to this call 
was a proposal to increase the frequency of online and offline interactions between citizens 
and EU institutions. The most recent and ambitious practice has been the Conference on the 
Future  of  Europe  (CoFE1058).  During  this  event,  participants  discussed  the  necessity  for 
escalating citizen participation to inform policy-making. The recommendations that emerged 
emphasised  further  resolute  action  to  boost  citizen  and  youth  involvement  at  EU  level;  to 

1056 European Governance: A White Paper COM(2001) 428. 
1057 Alemanno, A. . Study requested by the European Parliament's Committee on Constitutional 
Affairs, 2022. 
1058 CoFE: Putting Vision into Concrete Action (COM/2022/404). 

690 

 
 
 
develop a 'full civic experience' for Europeans and to ensure that their voices are heard not 
just during elections, but on an ongoing basis. 

As  part  of  its  commitment  to  engaging  broader  stakeholder  groups  in  R&I,  the  European 
Commission  has  backed  extensive  research into  deliberative  and  participatory  democracy 
under its FPs. This has led to the establishment of Science and Society Programme under 
FP6,  which  was  later transformed  into  the  Science  in  Society  Programme  under  FP7  and 
SwafS (‘Science with and for Society’) under Horizon 2020. The principles of the co-creation 
and co-design of R&I through the engagement of citizens and civil society have now been 
transitioned into an operational objective of Horizon Europe1059 and marked as a dedicated 
impact pathway1060. Horizon Europe places citizens at its core. Under such a framework, all 
societal actors (researchers, citizens, policy-makers, businesses, civil society organisations, 
etc.)  work  together  throughout  the  R&I  process  to  better  align  both  the  process  and  its 
outcomes  with  the  values,  needs  and  expectations  of  European  society.  The  EU's 
commitment to citizen engagement is also evident through its policy initiatives. For instance, 
the  European  Green  Deal1061  underscores  the  EU’s  commitment  to  fostering  innovative 
solutions by mobilising local communities and citizens. The Commission Communication ‘A 
new ERA for Research and Innovation’1062 further identifies value creation through knowledge 
sharing and knowledge creation as a pivotal objective, and promotes the improved translation 
of research results to benefit the economy and society. It embeds the opening of the research 
system towards society (and vice versa) as a policy priority, to achieve greater societal impact 
and  increased  trust  in  science.  In  addition,  it  continues  to  deepen  and  strengthen  Open 
Science approaches to R&I. The political basis for action on citizen and societal engagement 
has been further strengthened by the Council Recommendations on the ‘Pact for R&I’1063. 
Indeed, under this Pact, social responsibility is laid down as a principle for working together, 
and  ‘active  citizen  and  societal  engagement  in  R&I’  is  presented  as  an  action  to  increase 
society’s participation in the European Research Area (ERA). 

While it is important to understand the policy context, the present case study also needs to 
acknowledge  societal  perceptions  of  science,  technology  and  innovation.  According  to 
Special  Eurobarometer  ‘European  citizens’  knowledge  and  attitudes  towards  science  and 
technology’1064,  conducted  in  2021,  it  is  evident  that  most  respondents  felt  that  decisions 
about science and technology should be made by professionals (52%), with the public always 
kept informed. Meanwhile, 32% thought that the public should be consulted and that public 
opinion  should  be  taken seriously into  consideration.  Furthermore,  72% thought  that  such 
decisions should mainly rely on expert advice. However, a significant share of respondents 
(61%) felt that involving non-experts in research and technological development ensures that 
societal needs and expectations are met. Interestingly, most respondents reported that they 
engaged with science and technology by watching documentaries, discussing related issues, 
visiting museums, or studying in their free time – despite citing a lack of time, knowledge, or 

1059 Council Decision (EU) 2021/764 of 10 May 2021 establishing the Specific Programme 
implementing Horizon Europe – the Framework Programme for Research and Innovation, and 
repealing Decision 2013/743/EU. 
1060 Key Impact Pathway 6: ‘Strengthening the uptake of research and innovation in society’. See 
Annex V of Regulation (EU) 2021/695 establishing Horizon Europe and SWD(2023) 132 final on 
the Evidence Framework on Monitoring and Evaluation of Horizon Europe. 
1061 COM(2019) 640 final: Communication from the Commission to the European Parliament, the 
European Council, the Council, the European Economic and Social Committee and the 
Committee of the Regions: The European Green Deal. 
1062 COM(2020) 628 final: Commission Staff Working Document accompanying the document 
"Communication from the Commission to the European Parliament, the Council, the European 
Economic and Social Committee and the Committee of the Regions: A new ERA for Research 
and Innovation". 
1063 https://data.consilium.europa.eu/doc/document/ST-14136-2021-INIT/en/pdf 
1064 Directorate-General for Communication, Special Eurobarometer #516: ‘European citizens’ 
knowledge and attitudes towards science and technology’, 2021. 

691 

 
 
interest  as  barriers  to  more  active  engagement.  With  the  Eurobarometer  providing  robust 
data concerning public opinion across the European Union, it is evident that there is potential 
for greater citizen and end-user engagement in both research and innovation.  

1.2.2.  Key definitions 

Table 1 below summarises the key terms and concepts associated with citizen engagement, 
as outlined in the strategic documents of Horizon Europe.  

Term 

Citizens 

Definition 

Individuals acting on their own initiative and not on behalf of their employer or 
sectoral interests. This definition does not refer to any legal citizenship(s) that 
people may or may not hold. 

Civil society 

Ensemble of citizens and civil society organisations that are active in the public 
sphere but distinct from government and business. 

Civil society 
organisations 
(CSOs) 

All non-state, not-for-profit structures, such as citizens’ associations, patient 
groups, professional societies or groups, consumer groups, humanitarian 
organisations, non-governmental organisations (NGOs), foundations and 
charities. 

End-users 

Public, private or civil (i.e. civil society, see above) organisations that constitute 
potential users of the R&I outputs. 

Engagement 

The involvement of citizens and civil society in co-designing R&I agendas, in co-
creating R&I contents, and/or in co-assessing R&I outcomes. 

Social 
innovation 

Social innovation concerns the development of new products, methods and 
services for and with society, involving citizens, public authorities, business and 
industry, and academia in their design, development and implementation.  

Co-design 
activities 

Activities including workshops, focus groups or other means to develop R&I 
agendas, roadmaps or policies.  

Co-creation 
activities 

Activities such as citizen science or user-led innovation, which involve citizens or 
end-users directly in the development of new knowledge or innovations, through a 
range of different levels of participation. 

Co-
assessment 
activities 

Activities such as assisting in the monitoring and evaluation of the progress of a 
project, portfolio of projects, policies or programmes, to help ensure an iterative or 
even continuous process of interaction with citizens, civil society and end-users 
throughout the project cycle with regard to the quality, utilisation and (potential) 
impact of project outputs. 

Table 1. Key definitions of terms used in the case study. Source: Horizon Europe Programme Guide (2023, v3.0) 

1.2.3.  Overall approach and scope of the case study 

This case study’s analysis of citizen and societal engagement activities under the Horizon 
Europe programme relies on several interconnected methods: 

Desk  research  consisted  of  two  stages:  (1)  A  literature  review,  which  mapped  and 
synthesised evidence and findings from existing information sources on citizen and societal 

692 

 
engagement in Framework Programmes, such as previous evaluations, studies, databases 
and  similar  material;  and  (2)  Analysis  of  monitoring  and  administrative  data  on  relevant 
Horizon Europe topics.  

Interviews. While desk research provides a good context and preliminary information, the 
study  team  has  implemented  12  interviews  for  this  case  study.  These  interviews  have 
provided  valuable  information  and  evidence  on  the  promotion  of  citizen  and  societal 
engagement  under  the  Framework  Programme.  More  specifically,  they  have  allowed 
judgements to be extracted regarding the drivers, barriers, benefits and added value of citizen 
and societal engagement. Interviews carried out for the present case study include two main 
groups of interviewees: (1) Stakeholders who have a good overview of citizen and societal 
engagement activities within the Framework Programme, including but not limited to relevant 
Commission  officials, 
funding  organisations, 
representatives of citizen science associations, Science Europe, and other networks related 
to the topic under analysis; and (2) Participants in Horizon Europe projects focusing on citizen 
and societal engagement.  

representatives  of  national 

research 

A survey  was  used  to  collect  and  analyse  data  regarding  the  extent  to  which  citizen  and 
societal  engagement  methods  have  been  applied  during  the  implementation  of  Horizon 
Europe projects.  

In  summary,  the  case  study  will  provide  information  on  the  evaluation  questions  (EQs) 
outlined in Table 2 below.  

Dimension 

Key contributions 

Relevance  
RV1 

Coherence 
CH1 

Efficiency 
EFF6 
EFF12 
EFF17 

Effectiveness 
EFC14 

•  Relevance of citizen and societal engagement activities in delivering 
excellent science, given the needs of non-traditional research actors 
such as citizens. 

•  Citizen and societal engagement activities across different parts of 

the FP, and their coherence. 

•  External coherence with other EU programmes serving similar 

objectives.  

•  Modalities of implementation of citizen and societal engagement 

across the project cycle at FP level. Such an inventory is needed, 
since citizen engagement is relevant at all stages of R&I – from the 
identification and conceptualisation of R&I priorities to the testing and 
piloting, implementation, utilisation, valorisation and impact 
assessment of R&I results. 
Incentives and barriers to developing citizen and societal engagement 
activities. 

• 

•  The extent to which the citizen and societal engagement modalities 
are encouraged in the topic call texts across the diverse programme 
parts (content analysis of the texts of work programmes). 

•  The extent to which citizen and societal engagement modalities are 
implemented in the diverse programme parts (using data from the 
survey). 

EU added value 
EAV2 

•  The role of Horizon Europe in strengthening citizen and societal 

engagement in R&I, in comparison to what is available at 
national/regional level. 

Table 2. Evaluation questions addressed by this case study. Source: defined by the study team, based on Tender 
Specifications. 

693 

 
1.3.  Case study findings 

1.3.1.  Relevance  

As noted in Section 1.2.1, over time, the European Commission has stepped up its efforts to 
promote  the  societal  uptake  of  science  and  innovative  solutions.  Under  Horizon  2020, 
responsible  research  and  innovation  (RRI)  was  enumerated  as  a  priority  across  all  R&I 
activities,  I  order  to  deepen  the  relationships  between  science  and  society  and  to  be 
responsive  to  societal  challenges.  This  cross-cutting  issue  was  embodied  by  –  but  not 
exclusive to – the Science with and for Society programme (SwafS). Projects aimed to push 
the  boundaries  of  citizen  engagement  in  R&I,  building  evidence  of  positive  impacts  and 
pooling experience and knowledge between countries. The study on ‘Cross-Cutting Issues 
in Horizon 2020’1065 concluded that while the implementation of RRI had indisputably made 
strides towards strengthening the bond between science and society, a lack of systematic 
monitoring of its outcomes (e.g. ambiguities in flagging practices and the undefined scope of 
citizen and societal engagement) cast doubt on its depth and efficacy. Nevertheless, based 
on insights from Phase 1 of the present study, a number of Member States have picked up 
on the importance of various dimensions of RRI, and certain countries have launched national 
funding mechanisms to provide funding for RRI (e.g. through specific calls for citizen science 
activities1066 and the adoption of Open Science guidelines1067).  

Despite  the  active  efforts  made  during  the  implementation  of  Horizon  2020,  the  Interim 
evaluation1068 of the FP called for greater outreach to civil society in order to better explain 
results  and  impacts  as  well  as  the  contribution  that  R&I  can  make  to  tackling  societal 
challenges, and to better involve civil society actors in the co-design and implementation of 
programmes.  As  a  result,  the  current  Framework  Programme,  Horizon  Europe,  has 
strong  ambitions  for  Open  Science,  as  well  as  citizen  and  societal  engagement  in 
research, to become common practice across the European Research Area. In addition, 
citizen and societal engagement is part of the Open Science policy of the EU, embedded in 
the  new  European  Research  Area  (ERA)  policy  agenda1069  to  achieve  greater  impact  and 
increased trust in science. In this regard, the focus has shifted from RRI to a more overarching 
approach through the following Horizon Europe features:  

•  The engagement of citizens and civil society organisations is mainstreamed throughout 
the programme. This does not mean that the citizen and societal engagement activities 
need to appear within every funded project, but it is expected that such activities will have 
a place across the programme in those projects in which they will increase the chances 
of projects delivering on their objectives. This orientation towards openness is apparent 
in  the  Horizon  Europe  regulation,1070  both  as  a  programme  principle  (‘the  programme 

1065 Evaluation study on the implementation of cross-cutting issues in Horizon 2020 (2023). 
1066 For example, the Flemish government in Belgium released two citizen science calls in 2017 
and 2019. For more information, please refer here.  
1067 For example, the Latvian Open Science Strategy (2021-2027) was launched in 2021 and 
contains references to European Framework Programmes and their requirements (e.g. 
researchers can expect conditions similar to international research programmes. Similar 
requirements have been introduced into the Horizon Europe Framework Programme and other 
international research programmes in which Latvian researchers have participated, as well as in 
the Dutch National Open Science Plan and Programme, Open Science Policy Austria, and the 
Romanian National Action Plan on Open Science.  
1068 Interim evaluation of Horizon 2020 (2017).  
1069 European Commission. (2021). ERA Policy Agenda. Retrieved from 
https://commission.europa.eu/system/files/2021-11/ec_rtd_era-policy-agenda-2021.pdf 
1070 Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 2021 
establishing Horizon Europe – the Framework Programme for Research and Innovation, laying 
down its rules for participation and dissemination, and repealing Regulations (EU) No 1290/2013 
and (EU) No 1291/2013. 

694 

 
 
shall  promote  co-creation  and  co-design  through  engagement  of  citizens  and  civil 
society’)  and  as  an  operational  objective  (‘improving  the  relationship  and  interaction 
between  science  and  society,  including  the  visibility  of science in  society  and  science 
communication, and promoting the involvement of citizens and end-users in co-design 
and co-creation processes’). 

•  Work programmes may provide additional incentives or lay down obligations to adhere 
to Open Science practices, including citizen and societal engagement. In this regard, the 
citizen and societal engagement is also operationalised through topic texts.  

•  Applicants are encouraged to consider Open Science practices and to include citizens 
and societal engagement in their methodology, as this aspect is part of the evaluation 
process  (the  excellence  criterion).  Specific  guidance  regarding  citizen  and  societal 
engagement can be found in the Horizon Europe Programme Guide1071.  

•  Periodic  and  final  project  reporting  also  includes  a  section  on  citizen  and  end-user 
engagement  in  project  activities.  These  data  feed  into  one  of  the  nine  Key  Impact 
Pathways1072 (specifically, KIP6), which focuses on citizens and end-users co-creating 
knowledge  and innovations, with  the  goal  of  developing solutions  and knowledge  that 
are taken up by society. 

The  modalities  outlined  above  signal  the  ongoing  importance  and  continuing  relevance  of 
citizen and societal engagement in Horizon Europe. However, a few of the interviews with 
stakeholders  revealed  some  reservations  concerning  the  incorporation  and  transition  of 
citizen  and  societal  engagement  principles  from  Horizon  2020  to  Horizon  Europe.  These 
interviewees  highlighted  a  perceived  disjunction:  while  Horizon  2020  had  established  a 
dedicated  avenue  for  discourse  on  public  engagement  through  its  SwafS  programme, 
Horizon  Europe  appears  to  have  sidestepped  the  experience  accumulated  under  the 
previous  programme.  This  notion  has  also  been  expressed  in  a  number  of  research 
articles1073,1074  as  well  as  in  a  petition1075  by  the  research  community.  This  perceived 
fragmentation  is  seen  as  a  missed  opportunity  to capitalise  on  the  profound insights  into 
citizen  and  societal  engagement  amassed  during  the  implementation  of  Horizon  2020.  In 
navigating  the  new  structures  of  Horizon  Europe,  the  researchers  find  themselves  in 
somewhat uncharted territory, due to an absence of clear guidance on effectively integrating 
citizen  and  societal  engagement.  This  ambiguity  is  exacerbated  by  the  dispersal  of  the 
knowledge  acquired  under  Horizon  2020,  which  is  now  spread  thinly  across  various 
initiatives.  While 
ideal  of 
mainstreaming, they contended that the inclusion of more dedicated calls – specifically, calls 
underlining  citizen  and  societal  engagement  –  would  provide  a  well-rounded  approach  to 
fostering growth in this field. 

interviewed  generally  agreed  on 

the  stakeholders 

the 

On the flip side, Commission officials who were interviewed offered a contrasting narrative. 
They do not view the above changes as a discontinuation, but as an evolution. In this regard, 
they argue, the transition symbolises the maturity of the community involved in citizen and 
societal  engagement,  which  is  now  ripe  for  more  seamless  integration  into  mainstream 

1071 Horizon Europe Programme Guide (2023, v3.0). 
1072 Evidence Framework on monitoring and evaluation of Horizon Europe, 2023. 
1073 Meier, A., & Byland, R., A New Horizon for Society? Analysing the Integration of Responsible 
Research and Innovation in Horizon Europe, Brussels: SwissCore, 2020. 
1074 Griessler, E., Braun, R., Wicher, M., & Yorulmaz, M., The drama of responsible research and 
innovation: The ups and downs of a policy concept. In: Putting Responsible Research and 
Innovation into Practice: A Multi-Stakeholder Approach, pp. 11-34, Cham: Springer International 
Publishing, 2022. 
1075 Sis.Net, Call for Action: Horizon Europe Needs a Specific Programme for Funding Science, 
Society and Citizens’ Initiatives, 2018. 

695 

 
 
research across all parts of  Horizon Europe. The interviewed EC officials emphasised the 
potential benefits for communities of concentrating on citizen science, institutional academic 
shifts and science communication. Integrating these groups, they argue, not only enhances 
the  quality  and  societal  impact  of  research  but  also  presents  more  expansive  financial 
prospects.  It  was  highlighted  that  the  restructured  approach  under  Horizon  Europe  is  not 
about sidelining the foundation laid by SwafS under Horizon Europe, but rather building upon 
it. 

1.3.2.  Coherence 

Citizen  and  societal  engagement  is  woven  throughout  the  funding  instruments  of  Horizon 
Europe,  including  the  European  clusters  and  missions.  Citizen  and  societal  engagement 
approaches are also being embedded into other European funding programmes, as well as 
by  international  organisations  and  EU  Member  States  in  their  national  science  policies  to 
involve stakeholders across the quadruple helix in R&I. This section of the report provides an 
overview  of  the  modalities  of  implementation  and  assesses  the  internal  and  external 
coherence of such activities. 

1.3.2.1. 

Coherence of implementation across pillars 

Horizon Europe hosts a myriad of practices for citizen and societal engagement. These differ 
in  terms  of  their  setting  (local  or  international  cooperation),  the  organising  entity  (the 
Commission, project beneficiaries), the project/policy cycle, the target groups, the purposes 
(awareness raising, education, the co-production of knowledge) and time span (e.g. single 
event vs. long-term initiatives), as described in the sub-sections below. These approaches to 
public engagement overlap and diverge in various ways. While some can be used in multiple 
settings, others have been designed for more specific contexts and serve as inspiration for 
other  organisations.  Coherence  between  activities  is  ensured  through  the  operational 
objective  of  involving  citizens  and  end-users  in  the  co-design  and  co-creation  processes 
across pillars. However, stakeholders who were interviewed noted the absence of a central 
logic organising Horizon Europe’s citizen and societal engagement mechanisms – i.e. while 
individual instruments have their specific purposes and strengths, their collective potential is 
not fully harnessed due to overlaps and missing connections.  

Pillar 1: Excellent Science. Given the bottom-up nature of most projects under the Excellent 
Science pillar, the principles of co-creation, co-design and citizen and societal engagement 
are  not  explicitly  required  in  its  work  programmes  and  call  texts.  However,  a  number  of 
modalities and incentivisation mechanisms exist that support such endeavours: 

to  encourage  outstanding  science  communicators  among 

•  The  European  Research  Council  (ERC)  has  launched  the  Public  Engagement  with 
Research  Awards  and  the  Science  Journalism  Initiative,  encouraging  researchers  to 
engage  with  the  public.  More  specifically,  the  Public  Engagement  with  Research 
Awards1076  define  public  engagement  as  the  involvement  of  the  public  in  the  design, 
conduct or dissemination of activities funded by the ERC. Through this prize, the ERC 
wishes 
its  principal 
investigators.  The  prizes  are  distributed  across  three  categories:  Involve  (citizen 
science), Inspire (public outreach) and Influence (media and policy). The winners of the 
2020 prizes showcased innovative approaches to citizen and societal engagement (e.g. 
with the project RecognizeYourself the winner of the 2020 Involve prize, Mariska Kret 
from Leiden University, brings science to the zoo by involving the public in the study of 
great  apes’  emotions).  The  Science  Journalism  Initiative1077  supports  organisations  in 
hosting journalists at research organisations, in order to provide opportunities to media 

1076 ERC Public Engagement with Research Awards, 2023. 
1077 ERC, Science Journalism Initiative, 2023: https://erc.europa.eu/apply-grant/science-
journalism-initiative 

696 

 
 
representatives  to  learn  and  work  on  in-depth  reporting  projects  by  immersing 
themselves in the research environment and interactions with researchers. The ERC has 
also taken steps to promote innovative citizen and societal engagement methodologies 
through specific reports and factsheets (e.g. Citizen Science in ERC projects1078) aimed 
at  informing  the  research  community.  Lastly,  as  underlined  in  interviews  with 
Commission officials, the ERC’s Proof of Concept grants create complementary funding 
opportunities for ERC grantees willing to explore not only the commercial but also the 
social innovation potential of their work.  

•  The  work  programmes  of  the  MSCA  (Marie  Skłodowska-Curie  Actions)1079  under 
Horizon Europe encourage applicants to engage with society at large in order to make 
the results of their research visible and accessible to citizens – and, where relevant, to 
involve citizens, civil society and end-users in the co-creation of research content. Both 
Doctoral Networks and Postdoctoral Fellowships underline the importance of preparing 
young researchers (through training activities) for the increased research collaboration 
and  information-sharing  made  possible  by  new  (digital)  technologies1080.  In  addition, 
MSCA COFUND activities place a strong emphasis on Open Science approaches in their 
training activities1081. Most importantly, the objective of the MSCA and Citizens action is 
to bring research and researchers closer to the public at large through the organisation 
of European Researchers’ Night1082 and Researchers at Schools activities1083. European 
Researchers’ Night is Europe’s largest event for science communication and citizen and 
societal  engagement,  attracting  over  1.5  million  visitors  every  year  in  over  400  cities 
around  Europe  to  showcase  the  diversity  of  European  R&I  and  its impact  on  citizens’ 
daily  lives.  Researchers  at  Schools  activities  are  organised  throughout  the  year  by 
European Researchers’ Night projects and reflect local academic cultures. The initiative 
aims to foster direct interaction between researchers, primary and secondary school 
teachers and pupils and includes presentations in schools, hands-on experiments, visits 
to labs, games, discussions and role-playing. The initiative is expected to reach out to 
more than 2,400 schools and involve more than 224,000 pupils by the end of 2023. In 
addition,  a  number  of  supporting  actions  focus  on  aspects  of  citizen  and  societal 
engagement.  For  example,  ‘MSCA  Feedback  to  Policy  2023’1084  calls  for  studies, 
analyses and reports into trends and developments in the MSCA that are relevant to the 
EU Missions, including the citizen and societal engagement activities. Lastly, MSCA-NET 
prepared a policy brief on Open Science1085 in 2023. In addition to other key pillars of 
Open Science, this discusses the citizen and societal engagement practices. In relation 
to this, the policy brief provides guidelines for the MSCA National Contact Points, which 
spread information to applicants in the European Research Area and beyond. In addition, 
Science is Wonderful! is a major science fair funded under the MSCA which celebrates 
the value and impact of EU-funded research, putting MSCA fellows in the spotlight. The 
event takes place every year or every two years and gives primary and secondary school 
students the chance to interact with leading researchers and innovators, to learn more 
about their work in engaging formats and ask questions about scientific careers. 

1078 European Research Council, Citizen Science in ERC Projects: Mapping ERC Frontier 
Research, 2022. 
1079 MSCA Work Programmes 2021-2022 and 2023-2024. 
1080 e.g. collaborative tools, opening up access to publications and to other research outputs 
including data, FAIR data management, societal engagement and citizen science, etc. 
1081 MSCA Work Programmes 2021-2022 and 2023-2024. 
1082 European Researchers’ Night is a research communication and promotion event that takes 
place across EU Member States and Horizon Europe associated countries.  
1083 Researchers at Schools activities bring top researchers to schools and allow teachers and 
pupils to engage with them on important topics faced by European societies, such as climate 
change, sustainable development, health and nutrition. 
1084 HORIZON-MSCA-2023-FTP-01-01: MSCA Feedback To Policy 2023. 
1085 MSCA-NET Policy brief: Open Science, 2023. 

697 

 
 
•  Research  Infrastructures  actions  aim  to  endow  Europe  with  world-class  sustainable 
research  infrastructures  that  are  open  and  accessible  to  the  best  researchers  from 
Europe and beyond. In terms of citizen and societal engagement, the European Open 
Science  Cloud  (EOSC)  constitutes  an  operational,  open  and  FAIR  Open  Science 
ecosystem.  Where  relevant,  projects 
funded  under  EOSC  actions  encourage 
researchers to pay special attention to the involvement of external users, including the 
citizen scientists1086. 

Pillar 2: Global Challenges and European Industrial Competitiveness comprises clusters 
of research and innovation activities aimed at maximising integration and synergies across 
its  respective  thematic  areas,  while securing  high  and  sustainable levels  of  impact  for the 
Union in relation to the resources expended. Approximately half of Horizon Europe’s budget 
is allocated to the implementation of Pillar 21087. This pillar supports the creation and better 
diﬀusion of high-quality new knowledge, technologies and sustainable solutions; reinforces 
European industrial competitiveness; strengthens the impact of R&I in developing, supporting 
and implementing Union policies; and supports the uptake of innovative solutions in industry 
(in particular, in SMEs and start-ups) and society to address global challenges. This reflects 
the recognition that the success of research in Europe is dependent on extensive and high-
quality citizen and societal engagement, and on the optimal leveraging of societal capabilities 
and intelligence. 

The EC officials interviewed for this study noted that the Pillar 2 provides opportunities for 
mainstreaming citizen and societal engagement approaches such as citizen science, and for 
applying  such  approaches  within  thematic  research  communities.  Notably,  citizen  and 
societal  engagement  modalities  vary  between  clusters.  According  to  the  EC  officials 
interviewed, the approach used depends on the topic of the cluster – for example, in the case 
of  highly  specialised  and  industrial  topics,  citizen  participation  is  less  expected.  However, 
even  in  such  clusters,  aspects  of  citizen  and  societal  engagement  are  encouraged.  For 
example, the Cluster 2 2023-2024 work programme ‘Civil Security for Society’1088 indicates 
that  citizens  and  communities  should  be  engaged  in  the  field  of  security  research  (e.g.  in 
designing  the  requirements  for  security  research  and  assessing  the  societal  impact  of 
security  technologies).  Another  example  ca  be  seen  in  the  Cluster  4  2023-2024  work 
programme  ‘Digital,  Industry  and  Space’1089,  in  the  form  of a  public  procurement  call  for  a 
Citizens’  Hackathon  European  Championship  2023  which  aimed  to  deliver  a  number  of 
innovative solutions co-created with citizens across Europe, evaluated in local hackathons 
and  before  progressing  to  a  final  at  the  European  level  at  which  the  best  ideas  would 
compete. Similar examples can also be found in other clusters (e.g. 1, 5 or 6).  

Pillar 3: Innovative Europe emphasises the role of open innovation and aims to reinforce 
close  cross-border  collaboration  between  multiple  actors,  including  academia,  the  public 
sector, industry and individual entrepreneurs, while supporting the development of disruptive 
and market-creating innovations and enhancing European innovation ecosystems. A number 
of instruments for citizen and societal engagement exist through three of its programmes: 

•  The  European  Innovation  Council  (EIC)  aims  to  identify  and  support  breakthrough 
technologies  and  game-changing  innovations  to  create  new  markets  and  scale  up 

1086 HORIZON-INFRA-2023-EOSC-01-01 (WP 2023-2024). 
1087 Total Horizon Europe Budget: EUR 95,517 million. Global Challenges and European 
Industrial Competitiveness budget: EUR 53,516 million. As indicated in European Commission, 
Directorate-General for Research and Innovation, Horizon Europe, budget – Horizon Europe – 
the most ambitious EU research & innovation programme ever, Luxembourg: Publications Office 
of the European Union, 2021.  
1088 Horizon Europe Work Programme 2023-2024 'Civil Security for Society'. 
1089 Horizon Europe Work Programme 2023-2024, ‘Digital, Industry and Space’. 

698 

 
 
internationally1090.  Although  its  focus  is  on  industrial  players,  the  EIC  emphasises  the 
links  between  all  innovation stakeholders  (including  the citizens)  through  a  number  of 
competitions and awards. More specifically, the European Social Innovation Competition 
acts as a beacon for social innovators across Europe, incentivising and rewarding early-
stage ideas that shape society for the better. This prize acknowledges social innovation 
as  a  potent  approach  for  responding  to  societal  challenges,  and  encourages 
engagement between universities, engineering schools and civil society organisations to 
create  new  connections,  new  sources  of  sustainable  growth  and  meaningful  job 
opportunities. In addition, the European Capital of Innovation Award1091 acknowledges 
that  traditional  city  innovation  ecosystems  are  opening  up,  and  rewards  cities  that 
implement new and innovative activities that engage citizens1092.  

•  European  Innovation  Ecosystems  (EIE)  aim  to  foster  open  innovation  ecosystems 
among national, regional and local innovation. In particular, the activities of its CONNECT 
programme focus on creating links between innovators (the private sector, SMEs, start-
ups)  and  civil  society  organisations,  citizens  and  individual  inventors  among  other 
quadruple helix stakeholders. The inclusion of broader stakeholder groups in this way 
ensures that the innovations produced match existing needs, values and expectations in 
society,  thereby  accelerating  their  deployment  and  uptake  in  order  to  tackle  societal 
challenges1093. 

•  The  European 

Institute  of 

Innovation  and  Technology  (EIT)  supports 

the 
development  of  long-term  pan-European  partnerships  among  organisations  from  the 
business,  education  and  research  sectors.  Activities  relating  to  citizen  and  societal 
engagement centre around nine EIT Knowledge and Innovation Communities1094 (KICs). 
Each  KIC  has  a  distinct  societal  focus,  reflecting  priority  thematic  areas  at  national, 
European and international levels. The EIT’s KICs build their activities independently in 
a bottom-up manner by being directly linked to the actors in the knowledge triangle. More 
importantly, the EIT ensures the development of specific societal indicators in the areas 
of activity of each KIC, and carries out regular monitoring in line with the Horizon Europe 
framework  for  societal  impact.  For  example,  the  Climate-KIC  provides  a  number  of 
opportunities  for  citizen  and  societal  engagement  through  the  organisation  of 
‘climathons’1095 
in  climate  action)  and  a 
ClimateLaunchPad1096 competition. Other KICs have also targeted citizen and societal 
engagement activities, notably EIT Health1097, EIT Food1098 and EIT Urban Mobility1099.  

(engaging  cities  and  citizens 

The underlying premise of the ‘Widening participation and strengthening the European 
Research  Area  (WIDERA)’  programme  part  is  that  R&I  policy  can  only  achieve  greater 
excellence if everyone within the EU progresses. Despite numerous developments relating 
to  the  ERA,  large  disparities  remain  between  those  countries  that  lead  in  R&I  and  those 
countries that lag behind. More specifically, Widening actions strengthen collaborative links 
across  Europe  and  open  up  European  R&I  networks,  as  well  as  contributing  to  improving 

1090 European Innovation Council Work Programme 2023. 
1091 EIC, European Capital of Innovation Award, 2023. 
1092 Past winners include Barcelona (2014), Amsterdam (2016), Paris (2017), Athens (2018), 
Nantes (2019), Leuven (2020), Dortmund (2021), Vantaa (2021), Aix-Marseille Provence 
Metropole (2022) and Haarlem (2022).  
1093 EIC Work Programme 2023-2024. 
1094 EIT Climate-KIC, EIT InnoEnergy, EIT Digital, EIT Health, EIT Raw Materials, EIT Food, EIT 
Manufacturing, EIT Urban Mobility, EIT Culture & Creativity 
1095 For more information, see: https://climathon.climate-kic.org/  
1096 For more information, see: https://climatelaunchpad.org/  
1097 EIT Health, Value-based healthcare, 2023. 
1098 EIT Food, 2023Public engagement, 2023. 
1099 In 2021, EIT Urban Mobility launched a call for proposals entitled ‘Citizen Engagement’. 

699 

 
 
research management capacities in widening countries, supporting national policy reforms, 
and exploiting the potential of the Union's talent pool through targeted actions.  

The implementation of the ERA part of the programme is related to the European Research 
Area  Policy  Agenda  (2022-2024).  In  the  context  of  this  agenda,  the  2020  ERA 
Communication1100  calls  for  a  valorisation  of  research  results  that  places  citizens  at  the 
forefront; people expect science to be a driving force that will support the transition towards 
a greener and fairer society and economy. Hence, the inclusion of citizens and civil society 
is  evident  through  various  parts  of  the  WIDERA  work  programmes1101.  For example,  ERA 
fellowship activities include training in key transferable skills including communication, public 
engagement  and  citizen science1102.  Meanwhile,  ERA  Chairholders  can  be  citizens  of any 
country  in  the  world.  One  of  four  key  strands  of  the  ‘Reforming  and  enhancing  the  EU 
research and innovation system’ destination addresses the challenge of deepening the ERA, 
and  includes  Open  Science,  higher  education  and  researchers,  citizen  science,  science 
education, and gender and ethics. It aims underpin a new ERA that benefits from the creation, 
circulation and use of knowledge. A number of other initiatives are also being implemented 
to  bring  science  closer to  citizens,  such  as  a  Eurobarometer survey  of  European  citizens’ 
knowledge  and  attitudes  towards  science  and  technology  in  2024;  the  European  City  of 
Science; the European Union Contest for Young Scientists (EUCYS); and the EuroScience 
Open  Forum  (ESOF).  Most  importantly,  a  number  of  calls  for  Coordination  and  Support 
Actions  (CSA)  focus  on  citizen  and  societal  engagement  elements  such  as  citizen 
science1103, 
in  science1104,  science  education1105  and 
communication1106.  However,  stakeholders  who  were  interviewed  underlined  that  the 
mechanisms  offered  by  activities  under  the  Horizon  Europe  WIDERA  programme  could 
benefit from greater funding being dedicated to citizen and societal engagement activities, 
referencing how the SwafS programme was implemented under Horizon 2020 (as discussed 
in the Section 3.1. of this case study).  Nevertheless, dedicated calls remain, ensuring that 
topics  relating  to  citizen  science  and  societal  engagement  are  not  overlooked.  Notably, 
projects such as IMPETUS1107 (which offers cascade funding opportunities for citizen science 
initiatives) and European Citizen Science1108 stand out in this regard. 

increasing  societal 

trust 

1100 Communication from the Commission to the European Parliament, the Council, the 
European Economic and Social Committee and the Committee of the Regions: a new ERA for 
Research and Innovation. COM(2020) 628 final. 
1101 WIDERA Work Programmes 2021-2022 and 2023-2024. 
1102 HORIZON-WIDERA-2023-TALENTS-02-01: ERA Fellowships (WIDERA Work Programme, 
2023-2024). 
1103 HORIZON-WIDERA-2021-ERA-01-61: Supporting and giving recognition to citizen science 
in the European Research Area; HORIZON-WIDERA-2021-ERA-01-60: A capacity-building and 
brokering network to make citizen science an integral part of the European Research Area. 
1104 HORIZON-WIDERA-2021-ERA-01-44: Societal trust in science, research and innovation. 
1105 HORIZON-WIDERA-2022-ERA-01-70: Open schooling for science education and a learning 
continuum for all. 
1106 HORIZON-WIDERA-2022-ERA-01-60: A European competence centre for science 
communication. 
1107 Impetus (Grant agreement ID: 101058677). 
1108 European Citizen Science (Grant Agreement ID: 101058509). 

700 

 
 
The EU R&I Missions. Based on insights from the Lamy Report1109 and Horizon 2020 Interim 
evaluation1110,  Horizon  Europe  adopted  a  mission-oriented,  impact-focused  approach  to 
addressing global challenges. These Missions represent a narrowly defined set of activities 
aimed at delivering verifiable results according to a planned timescale that can be used to 
measure  progress  in  overcoming  particular  societal  challenges.  They  serve  as  important 
measures connecting different parts of the programme, as well as connecting stakeholders 
who would otherwise not have common ground on which to collaborate. At the core  of the 
five  EU  R&I  Missions  is  the  inclusion  of  bottom-up  and  citizen-engaged  public-private 
partnerships.  As  outlined  in  the  Horizon  Europe  Work  Programme  2023-20241111,  the 
Commission  invites  researchers  and  innovators,  as  well  as  citizens  and  all  interested 
stakeholders, to take part in the implementation and monitoring of these EU R&I Missions.  

Interviews  carried  out  for  this  study  have  revealed  that  the  Missions  play  a  crucial  role  in 
citizen and societal engagement efforts across the Framework Programme. This view was 
shared  by  EC  officials,  beneficiaries  and  representatives  of  networks  related  to  civic 
engagement. However, the interviews also underlined the need to understand the specific 
implementation context of each Mission (e.g. the characteristics of the innovation ecosystem 
concerned,  the  technological  nature  of  solutions)  in  order  to  successfully  match  relevant 
citizen and societal engagement activities. The logic of Missions predefines the inclusion of 
citizens; however, according to the officials interviewed, some Missions are missing out on 
the potential for social innovations and co-creation. In relation to this, a recently published 
Staff  Working  document,  ‘EU  Missions  two  years  on:  assessment  of  progress  and  way 
forward’1112,  sheds  light  on  citizen  and  stakeholder  engagement  in  the  preparation  and 
implementation of the Missions:   

•  All Missions went through a preparatory phase during which detailed implementation 
plans were co-developed, followed by Horizon Europe work programmes, in cooperation 
with policy actors, citizens and other stakeholders. For example, citizen and stakeholder 
engagement  is  a  major  element  of  the  ‘Soil’  Mission,  as  it  is  key  to  achieving  the 
Mission’s  goal  of  anchoring  care  for  the  soil  into  society.  In  the  initial  phase  of  the 
Mission, a survey of citizens’ perceptions concerning soil-related challenges received 
more than 2,500 replies, with 80% of respondents indicating that the health of the soil 
matters  to  their  quality  of  life.  However, despite  active  efforts  being  made  to  engage 
stakeholders, in some Missions citizen and societal engagement activities have not had 
a visible effect on the Mission formulation1113.  

•  Citizen  and  stakeholder  engagement  in  the  implementation  of  Missions  varies 
depending on their context1114. In case of the ‘Climate adaptation’ Mission, there is a 
strong  focus  on  communication  and  awareness-raising  activities  (e.g.  through  the 
community of practice and the Mission forum). However, the Staff Working document 
concludes that the further engagement of citizens at a strategic level does not appear 
justified, and that further opportunities will emerge as the project achieves its first results. 
It is important to note, however, that some Mission projects (e.g. CLIMAS) specifically 

1109 In 2020, as part of the Starfish 2030 Mission, Pascal Lamy, the President of the ‘Ocean’ 
Mission Board, wanted to consult European citizens to gather their opinions on five areas of 
work proposed by the Mission's Committee of Experts, which brought together scientists, 
representatives of civil society and economic sectors. An online questionnaire gathered 6,600 
responses in 20 days. Analysis of these responses revealed a strong desire for commitment and 
a strong need for knowledge. 
1110 The Horizon 2020 Interim evaluation emphasised that the next Framework Programme 
should make it easier for citizens to understand the value of investments in R&I and should 
maximise the impact of investments by setting clearer targets and expected impacts when 
addressing global challenges. 
1111 Horizon Europe Work Programme 2023-2024: General Introduction.  
1112 Commission Staff Working document: COM(2023) 457 final. 
1113 Ibid.  
1114 Ibid.  

701 

 
 
focus on citizen and societal engagement in climate adaptation processes. Also, citizen 
and stakeholder engagement remains important to the implementation of the ‘Cancer’ 
Mission,  as  evidenced  by  the  results  of  an  online  survey  carried  out  by  an  external 
study1115. Meanwhile, the ‘Ocean’ Mission’s activities appear to place a strong emphasis 
on  promoting  citizen  and  societal  engagement,  encouraging  ocean  literacy  and 
education  with  a  special  emphasis  on  citizen  science  approaches  (e.g.  the  Plastic 
Pirates campaign, the EU4Ocean Coalition, student and school initiatives through the 
Blue School Network). In the case of the ‘100 Climate-neutral and Smart cities by 2030’ 
Mission, the Staff Working document suggests that evidence regarding how citizens are 
being involved in the Mission will only become clear once the first wave of the Climate 
City Contract has been assessed. Given the behavioural changes expected from local 
stakeholders  and  private  households,  awareness  of  the  Mission  needs  to  increase 
further  among  European  citizens  –  including  how  they  can  become  more  involved. 
Lastly, the ‘Soil Deal for Europe’ Mission places a strong emphasis on co-producing and 
disseminating knowledge through living labs and lighthouses. This mission goes beyond 
research, incorporating both soil literacy and citizen science, as well as involving citizens 
in  activities  such  as  designing  experiments  and  collecting  data.  Progressing  its 
implementation  involves  national  info  days  to  engage  stakeholders  and  gather 
feedback.  The  recently  launched  Mission  Soil  Manifesto  reaches  out  effectively  to 
mobilise support for the mission's goals. Soil literacy and education efforts can be seen 
through the strengthening of soil advisors' skills, the promotion of citizen science, and 
the integrating of soil education into schools and universities, thereby contributing to the 
Education for Climate Coalition. 

1.3.2.2. 

Synergies with other EU programmes and relevant national, regional and 
international initiatives 

Horizon  Europe  manifests  a  clear  shift  towards  robust  citizen  and  societal  engagement, 
setting a precedent for a new paradigm in R&I processes. This shift is not merely internal; it 
reflects and complements broader European and international strategies, representing the 
EU’s commitment to creating a participatory, inclusive and democratic research environment. 
By  embedding  citizen  and  societal  engagement  within  its  Open  Science  policy,  the  EU 
demonstrates its dedication to making the scientific process transparent and democratic. The 
ERA Policy Agenda (2022-2024) further accentuates the significance of value creation, with 
the aim of derive societal and economic benefits from the outcomes of research. This citizen 
and societal engagement narrative is also evident in other European funding programmes. 
For example, Erasmus+ plays a pivotal role in enhancing democratic participation, focusing 
on the strengthening of citizens’ understanding of the European Union, fostering social and 
intercultural competences and promoting active citizenship1116. In the case of EU Cohesion 
policy,  the  recent  collaboration  between  the  European  Commission’s  DG  REGIO  and  the 
OECD emphasises the centrality of citizens in shaping investment decisions, underlining the 
role  of  effective  partnerships  between  governments,  stakeholders  and  citizens1117.  In  a 
broader international context, Horizon Europe’s alignment with co-creation and with citizen 
and societal engagement resonates profoundly with global strategies, including the United 
Nations’  Sustainable  Development  Goals1118.  The  EU’s  advocacy  for  Open  Science, 
harmonising  with  objectives  such  as  the  democratisation  of  knowledge,  further  echoes 
international frameworks such as the UNESCO Recommendations on Open Science1119. This 

1115 Most respondents (64%) in the external study’s online survey agreed that the Mission 
encourages broad engagement and the active participation of stakeholders and citizens. 
1116 Erasmus+, Priorities of the Erasmus+ Programme, 2023. 
1117 OECD Working Papers on Public Governance,2022 Engaging citizens in cohesion policy DG 
REGIO and OECD pilot project final report, 2022. 
1118 United Nations, Open Science to Achieve United Nations 2030 Agenda, 2023. 
1119 UNESCO,2022 An introduction to the UNESCO Recommendation on Open Science, 2022. 

702 

 
 
alignment not only positions the EU as a frontrunner in championing democratic, inclusive 
research, but also underlines its dedication to bridging knowledge divides on a global scale. 

1.3.3.  Efficiency 

The  results  of  the  survey  and  desk  research  have  enabled  the  study  team  to  create  an 
inventory of mechanisms for mainstreaming citizen and societal engagement throughout the 
policy/project cycle across Horizon Europe. In this regard, the interviews proved  especially 
relevant  to  evaluating  the  efficiency  and  key  implementation  challenges  of  the  various 
modalities.  

1.3.3.1. 

Multi-stakeholder approach towards policy design and setting the research 
agenda  

In the context of R&I policy design, active citizen and societal engagement plays a pivotal 
role in ensuring that the wider societal implications of scientific developments are thoroughly 
considered  through  a  well-structured  citizen  and  societal  engagement  process.  Such 
processes  contribute  to  the  creation  of  concrete  policy  options  that  align  with  the  values, 
expectations and concerns of the public. Moreover, increasing demands by citizens  – both 
within  and  beyond  the  EU’s  borders  –  to  actively  engage  with  scientific  and  technological 
matters,  underlines  the  importance  of  inclusive  policy  design.  For  the  EU,  capitalising  on 
citizen and societal engagement offers an opportunity to bolster the productivity of EU-funded 
R&I endeavours and to strengthen the profile of Union institutions in the eyes of the public. 
Against this backdrop, efforts have been made to develop means for more constructive and 
effective communication and dialogue between R&I actors and citizens, in order to enable 
society  to  exert  an  influence  on  R&I  governance.  Most  notably,  public  consultations  have 
been  employed  to  enrich  policy  design  processes  from  a  societal  perspective.  During  the 
public  consultation  process,  citizens,  stakeholders  and  experts  collaborate  in  concerted 
processes  to  identify  visions,  needs  and  concerns,  and  to  transform  these  into  research 
topics of high value to society.  

While  public consultation  remains  a  standard practice  for participative  policymaking,  there 
are other methodologies that can be used complement it. Citizen assemblies, focus group 
discussions  and  stakeholder  workshops  are  some  alternatives  that  can  provide  in-depth 
insights,  especially  on  complex  issues.  Nevertheless,  the  core  tenet  remains  unchanged: 
whether through a broad-based public consultation or a focused group discussion, the aim is 
to ensure policies are not designed in isolation, but are reflective of the collective aspirations 
and concerns of the community they intend to serve. 

The  European  Commission’s  commitment  to  fostering  an  inclusive  and  participatory 
approach to shaping its research agendas is evident from its endeavours through the Horizon 
Europe programme. However, certain areas still exhibit a divergence between this intention 
and the actual influence of public input. For instance, while the consultation on the Horizon 
Europe Strategic Plan saw an impressive number of responses, deeper examination reveals 
the under-representation of the voices of citizens and non-governmental organisations (for 
example, while they expressed their views in the survey, the opinions regarding their precise 
needs  provided  in  the  position  papers  were  limited).  Moreover,  in  the  definition  and 
implementation  of  Missions such  as  the  Cancer Mission,  the  pre-defined  areas  of  interest 
and heavy reliance on expert judgments indicate that participatory processes may, in some 
instances, be more symbolic than genuinely influential1120. This highlights the importance of 
not  simply initiating  public  consultations,  but  ensuring  they  are genuinely  open-ended  and 
can meaningfully impact the final outcomes. Preliminary patterns suggest that a genuinely 
participatory  and  co-designed  process  might  have  yielded  different,  perhaps  more  widely 
encompassing, Mission formulations. As the Horizon Europe programme moves ahead, it is 

1120 Commission Staff Working document: COM(2023) 457 final. 

703 

 
 
vital to not only conduct public consultations, but to ensure they have the breadth, depth and 
receptiveness  necessary  to  transform  citizens’  insights  into  actionable  research  agendas. 
Only  then  can  the  potential  of  public  consultations  be  truly  realised,  thus  necessitating 
broader  outreach  and  a  commitment  to  integrating  varied  perspectives  into  the  EU's  R&I 
ecosystem. 

1.3.3.2. 

Steering of R&I initiatives towards broader citizen and societal 
engagement 

At programme level, understanding the benefits of citizen and societal engagement is crucial. 
Since the bulk of Horizon Europe projects are prepared and orchestrated by researchers, it 
is  imperative  that  they  can  discern  the  advantages  and  incentives  lined  up  for  them.  The 
interviews and desk research carried out for this study allowed the evaluation team to identify 
a  number  of  mechanisms  defined  by  the  European  Commission  that  steer  the  Horizon 
Europe research community towards considering aspects of citizen and societal engagement 
in their proposals. These mechanisms are as follows: 

•  The wording of work programmes and topics. In addition to the overarching presence 
of citizen and societal engagement under Horizon Europe, particular work programmes 
may offer further incentives or stipulations that emphasise its significance. Applicants to 
R&I  programmes  pay  careful  attention  to  the  wording  and  objectives  outlined  in  the 
descriptions  of  call  topics,  and  align  their  submissions  accordingly.  By  placing  an 
emphasis  on  citizen  and  societal  engagement  within  these  textual  guidelines, 
programmes can catalyse a discernible shift in the orientations of projects. In this regard, 
the handbook for the drafters of work programmes offers guidance on how and when to 
incorporate citizen and societal engagement, as well as providing standardised wording 
and illustrative examples from previous work programmes. In addition, the EC officials 
interviewed noted that early drafts of work programmes are shared with them and timely 
feedback is provided in order to optimise citizen and societal engagement. However, the 
drafting of topics is not without its challenges. The interviews conducted for this study 
revealed  complexities  regarding  the  use  of  technology  readiness  levels  (TRLs)  in  call 
texts, as it can often inadvertently exclude social sciences and humanities – sectors that 
ought to be seamlessly integrated into research initiatives for broader citizen and societal 
engagement. In addition, there is the challenge of a project’s depth versus its breadth. 
EC officials noted that smaller-scale projects tend to engage the public more efficiently 
(because they tend to be closer to local communities and know them better). Despite 
attempts to introduce smaller projects into work programmes, resistance has primarily 
emerged  from  executive  agencies,  which  favour  larger  projects  due  to  their  limited 
staffing capacities for programme management. 

•  The Horizon Europe Programme Guide1121 serves as another important instrument for 
incorporating citizen and societal engagement. This dedicates an entire section to the 
nuances of citizen and civil society engagement within research paradigms, and includes 
key  definitions,  examples  of  previous  projects,  useful  resources  and 
training 
opportunities.  Serving  as  a  foundational  document,  it  informs  potential  beneficiaries 
about the imperatives of citizen and societal engagement.  

•  Operationalisation of the criterion of excellence. One of the significant evolutions in 
the operation of the Framework Programmes concerns the application of the criterion of 
excellence.  Under  Horizon  Europe,  this  has  entailed  the  recognition  of  citizen  and 
societal  engagement  as  an  integral  element  denoting  the  quality  and  excellence  of 
research. The essence of this criterion of excellence mandates that every proposal under 
Horizon  Europe  should  clearly  elucidate  the  modalities  used  to  engage  citizens,  civil 
society  and  end-users,  wherever  pertinent.  However,  the  external  stakeholders 

1121 Horizon Europe Programme Guide (2023, Version 3.0). 

704 

 
 
interviewed  raised  concerns  regarding  its  ambiguity.  While  citizen  and  societal 
engagement finds its place in the criteria used to evaluate excellence, it is prefaced by 
the words ‘if relevant’. The lack of any precise guidelines regarding its incorporation may 
therefore result in such engagement being overlooked or poorly integrated. Furthermore, 
the stakeholders interviewed expressed a pronounced need for clearer guidance to be 
given to the experts who evaluate proposals. Evaluator briefings, while structured, often 
lack  a  comprehensive  overview  of  elements  such  as  co-creation,  citizen  and  societal 
engagement  and  Open  Science,  leading  to  inconsistencies  in  proposal  assessments. 
For example, some interviewees noted that many proposals reveal gaps in knowledge 
that stem from a limited understanding of Open Science – the understanding of which is 
often limited to merely open access. This situation shows the need for targeted training, 
especially at national level. Although Europe has made significant strides in promoting 
citizen  and  societal  engagement  and  Open  Science,  the  national  frameworks  within 
Member States often fall short, thus requiring a more aligned and informed approach to 
proposal evaluations. 

•  Dedicated  calls.  Despite  a  shift  in  its  structural  approach,  under  Horizon  Europe  the 
European Commission has continued its commitment to fostering projects centring on 
citizen and societal engagement. Dedicated calls continue, ensuring that topics related 
to  citizen  science  and  societal  engagement  are  not  overlooked.  In  particular,  projects 
such  as  IMPETUS1122  (which  offers  cascade  funding  opportunities  for  citizen  science 
initiatives) and European Citizen Science1123 stand out in this regard. However, as noted 
in  the  section  of  this  report  on  relevance,  the  community  of  citizen  and  societal 
engagement  practitioners  and  researchers  raised  some  concerns  regarding  the 
significantly lower number of dedicated calls.  

• 

Information days on different programme parts play an important role in accentuating 
the citizen and societal engagement narrative and directing the attention of the research 
community  towards  topics  that  require  broader  citizen  and  societal  engagement, 
according to the EC officials interviewed. According to these interviewees, information 
days provide an opportunity to explain the need for citizen and societal engagement. 

•  Highlighting successful examples. Over the last decade, numerous organisations in 
Europe have garnered significant experience in citizen and societal engagement (most 
importantly  as  a  result  of  the  Horizon  2020  SwafS  programme).  The  EC  makes  a 
concerted  effort  to  provide  practical  examples  of  citizen  engagement  to  the  research 
community  through  the  tools  discussed  above  (e.g.  in  the  texts  of  work  programmes, 
during information days and in the programme guide). Efforts have also been made to 
publish catalogues of success stories in various programme parts, e.g. the ‘Results pack 
on  Citizen  Science’1124  showcases  12  EU-funded  projects  that  are  developing  good 
practices, while the ‘Communication and citizen engagement initiatives in line with the 
Horizon  Europe  Mission  A  Soil  Deal  for  Europe’1125  presents  examples  of  soil-related 
activities and initiatives launched by, co-developed with or targeted at European citizens. 
Horizon  Europe  work  programmes  (especially  those  under  WIDERA)  refer  to  projects 
under  the  Horizon  2020  SwafS  programme,  and  encourage  close  cooperation  to  be 
developed with its project beneficiaries1126. According to interviews with EC officials and 
external stakeholders, the sharing of best practices and know-how is especially valuable, 

1122 Impetus (Grant agreement ID: 101058677). 
1123 European Citizen Science (Grant Agreement ID: 101058509). 
1124 European Commission,2022 Results pack on Citizen Science, 2022. 
1125 European Commission, Communication and citizen engagement initiatives in line with the 
Horizon Europe Mission A Soil Deal for Europe, 2022.  
1126 HORIZON-WIDERA-2021-ERA-01-91: Ensuring reliability and trust in quality of Research 
Ethics expertise in the context of new/emerging technologies call text states: ‘This work package 
should also pay particular attention to cooperation with the projects resulting from call H2020-
SwafS-2020-1 TechEthos, HYBRIDA and ROSiE.’ 

705 

 
 
because individuals preparing proposals in various thematic areas may not be fully aware 
of the resources and expertise required to craft effective citizen and societal engagement 
initiatives. More specifically, the interviews noted difficulties in assigning an appropriate 
budget  for  societal  engagement  activities;  due  to  the  absence  of  specific  guidelines 
regarding  citizen  and  societal  engagement,  research  institutions  often  approach 
engagement organisations with minimal budgets, sometimes treating them akin to PR 
agencies. However, genuine and impactful citizen and societal engagement requires a 
dedicated budget. Another misconception relates to the necessary expertise: according 
to  the  stakeholders  interviewed,  although  some  researchers  may  feel  confident  about 
undertaking  citizen  and  societal  engagement  merely  because  it's  a  call  requirement, 
genuine  engagement  requires  a  depth  of  knowledge.  In  this  regard,  the  best  way  for 
those new to the field to find the most efficient approach is through collaboration with 
experienced entities. On the European stage, resources and support can be accessed 
via  the  European  Citizen  Science  Association  (ECSA)  and  EU-Citizen.Science.  While 
there is a benefit to these platforms, those stakeholders interviewed who had experience 
in citizen and societal engagement noted that they had mostly discovered them through 
previous project participations and basic internet searches.  

1.3.3.3. 

Societal engagement in specific R&I activities  

Horizon  Europe’s  R&I  ecosystem  is  marked  by  an  evolution  in  the  integration  of  citizen 
engagement,  emphasising  a  shift  from  passive  observation  to  active  participation.  In  this 
intricate  network,  citizens  and  stakeholders  are  engaged  not  as  mere  spectators;  but  are 
ingrained  within  processes  as  users,  co-designers,  data  providers,  thus  underpinning  the 
depth and relevance of research outcomes. This multiplicity of roles suggests an adaptable 
approach. For instance, while some projects might orient their entire focus on the basis of 
citizen  inputs,  others  embed  these  perspectives  within  specific  work  packages  (WPs). 
Moreover,  feedback-driven  WPs  are  emerging  as  pivotal  touchpoints,  offering  continuous 
evaluation  and  redirection  throughout  a  project’s  lifecycle.  While  citizen  engagement  is 
manifested in various forms including deliberative platforms, open innovation  programmes 
and  living  labs,  the  present  analysis  focuses  predominantly  on  three  key  areas:  financial 
support to third parties, citizen science, and hackathons. Chosen for their recurrence during 
interviews,  these  domains  provide  a  detailed  understanding  of  both  the  challenges  and 
opportunities  in  enhancing  citizen-centric  methodologies  within  Horizon  Europe's  R&I 
framework. 

•  Financial  Support  for  Third  Parties  (FSTP),  commonly  referred  to  as  ‘cascade 
funding’,  is  a  distinct  financial  mechanism  devised  by  the  European  Commission.  Its 
design  aims  to  stimulate  (social)  innovation  by  redirecting  public  funds  towards  a 
particular set of beneficiaries. Unlike the traditional approach, the beneficiaries under 
this  umbrella  are  not  direct  members  of  the  initial  eligible  consortium,  but  rather  civil 
society organisations, SMEs and other (often smaller and less structured) entities. FSTP 
works through a competitive open call mechanism. The central theme of such open calls 
is intricately woven into the objectives of the parent project, with the topic’s specificity 
determining the breadth of its audience reach. The goals of open calls are diverse, and 
can  focus  on  selecting  initiatives  for  acceleration  or  incubation  purposes  and/or 
integrating more participants into a project to extend its scope or address specific tasks. 
According to the stakeholders interviewed, this mechanism holds the potential to unlock 
the  doors  of  European  funding  for  entities  that  would  conventionally  remain  on  the 
sidelines.  For smaller entities,  navigating  the intricate  web  of  EC Regulations can  be 
overwhelming. Within this scenario, cascade funding emerges as a beacon, simplifying 
the complex requirements. Notably, Horizon Europe has already embraced a number of 
cascading grants calls. However, the analysis of topic texts in the work programmes for 
2021-2022 and 2023-2024 shows that FSTP is rarely used in relation to broader citizen 
and societal engagement efforts. For example, in the 2021-2022 work programme for 
clusters  under  Pillar  2,  only  11  topics  encouraged  researchers  to  consider  the 

706 

 
engagement of citizens and provided opportunities for FSTP (not necessarily aimed at 
the general public of civic society). However, some notable examples of projects that 
aim  to  encourage  societal  engagement  in  R&I  were  identified  under  the  WIDERA 
programme1127. 

The  operational  intricacies  of  cascade  funding,  as  highlighted  in  interviews,  present 
several challenges. Foremost among these is the issue of record-keeping. Beneficiaries 
with  experience  of  FSTP  have  pointed  out  the  complexity  of  the  existing  reporting 
system,  emphasising  the  need  to  develop  a  more  balanced  approach  that  melds 
responsibility with ease of reporting. While the present reporting structure does account 
for  final  beneficiaries,  extracting  this  information  necessitates  intense  scrutiny,  as 
indicated  by  the  EC  officials  interviewed.  Another  challenge  lies  in  the  actual 
implementation of the FSTP, which demands hands-on experience. The distribution of 
these grants poses obstacles for a range of stakeholders, from project coordinators to 
the  institutional staff  responsible  for disbursing  funds.  Larger  institutions  grapple  with 
intricate bureaucratic processes that are not always conducive to third-party payments. 
Meanwhile,  smaller  entities  wrestle  with  cashflow  issues,  often  lacking  immediate 
access to the required funds. 

• 

Citizen science, which entails the public’s active engagement in scientific endeavours, 
has  carved  out  a  significant niche  within  European  R&I  initiatives.  As  pivotal  facet  of 
Open Science, citizen science is intertwined with outreach activities, scientific education 
and diverse modalities for public engagement, collectively serving as pillars for RRI. In 
recent years, several European bodies have acknowledged the paramount importance 
of citizen science. In 2016, the Council recognised it as an imperative for Open Science, 
and by April 2018, the Open Science Policy Platform1128 (OSPP) had integrated citizen 
science into its octet of Open Science objectives. As a further testament to its growing 
prominence,  following  the  Horizon  2020  interim  assessment,  the  SwafS  work 
programme  for  2018-2020  strategically  pivoted  towards  ‘exploring  and  supporting 
citizen science’, fostering a portfolio-centric strategy to realise this orientation. Citizen 
science  offers  benefits  to  researchers,  citizens,  policymakers  and  society,  making 
science more relevant, speeding up the discovery of new knowledge, assisting in policy 
monitoring,  raising  public  awareness  about  science,  and  promoting  evidence-based 
policy  decisions.  The  European  Commission  has  thus  taken  steps  to  promote  these 
benefits,  showcasing  excellent  examples  of  citizen  science  and  exploring  citizen 
engagement  methodologies  in  various  R&I  projects  (e.g.  ERC  projects1129,  the 
‘Exploration  of  citizen  engagement  methodologies  in  European  R&I  projects’  by  the 
Working  Group  on  Consumer  and  Citizen  Engagement1130,  ‘Experience  with  Citizen 
Science in Horizon 2020’1131, and the CORDIS results pack on citizen science1132). In 
calls under Work Programmes 2021-2022 and 2023-2024, citizen science is mentioned 
to the greatest extent in WIDERA and EU Missions, with a number of calls specifically 

1127 For example, HORIZON-WIDERA-2023-ERA-01-08 in relation to citizen science. 
1128 https://openscience.eu/ 
1129 European Research Council, Citizen Science in ERC projects: Mapping ERC Frontier 
Research, 2022.  
1130 European Commission,, Exploration of citizen engagement methodologies in European R&I 
projects by the Working Group on Consumer and Citizen Engagement, 2023. 
1131 European Commission, 2020Citizen science and citizen engagement: Achievements in 
Horizon 2020 and recommendations on the way forward, 2020. 
1132 An example can be found here: https://op.europa.eu/en/publication-detail/-
/publication/26c8a71d-bb98-11ec-b6f4-01aa75ed71a1 

707 

 
 
focusing on increasing awareness about citizen science and increasing the capacities 
of both researchers and citizens1133.  

•  Hackathons – namely, intense collaborative events in which individuals from diverse 
backgrounds  work  collectively  to  address  predefined  challenges  –  have  surged  in 
popularity and applicability within the European Union. This underlines a shift whereby 
the  concerns  of  citizens  have  become  the  focal  point  for  ideation  and  the  crafting  of 
solutions. For instance, multifaceted challenges such as climate change, air pollution 
and participatory democracy are dissected through a citizen-centric lens, delving into 
what  citizens  can  achieve  in  collaboration  with  researchers,  to  tangibly  improve  their 
lives and communities. During interviews, several references were made to initiatives 
by  the  EIT  Climate  KIC,  including  ‘climathons’1134  (city-based  hackathons  serving  as 
idea incubators) and Climate Launchpad1135 (a contest to generate transformative ideas 
and  envisage  potential  change).  Both  of  these  initiatives  gained  international 
recognition, with various funders supporting iterations beyond Europe. To ensure the 
smooth organisation of hackathons within R&I activities, the European Commission has 
developed  a  set  of  guidelines1136.  However,  while  hackathons  present  numerous 
advantages,  the  stakeholders  interviewed  pointed  out  certain  drawbacks.  Their  most 
prominent concerns related to inadequate follow-up with citizens after such events, with 
some feeling that ideas were collected merely for the sake of doing so. In addition, there 
were challenges in attracting a diverse range of participants (e.g. due to relevance of 
the topics, and the reimbursement of researchers). 

Based on an analysis of these forms of concrete citizen engagement in R&I activities, it can 
be  concluded  that  while  the  landscape  of  citizen  and  societal  engagement  offers 
opportunities, it also presents challenges (common to all approaches). A significant challenge 
revolves around the incentives for citizens to participate. It is true that incentivising citizens 
to  contribute  actively  and  meaningfully  poses  a  considerable  obstacle,  especially  in  the 
absence of monetary benefits. In this context, it is of paramount importance to note that while 
the  professionals  involved  in  these  projects  receive  compensation  for  their  contributions, 
citizens  (typically  retirees  and  students)  often  do  not.  This  discrepancy  can  skew  the 
outcomes, given the potential biases of this narrowed participant demographic. Overarching 
guidelines  at  EU  level,  which  often  discourage  or  outright  prohibit  financial  rewards  or 
reimbursements for citizen participants, only exacerbate this issue. Participation requires not 
just  a  person’s  presence,  but  also  their  time,  effort,  and  sometimes  unique  insights. 
Restricting financial incentives could dampen enthusiasm and limit participation only to those 
unburdened  by  financial  constraints.  Stakeholders  pinpointed  potential  solutions  to  this 
challenge. For instance, while it might not be feasible to provide fees, reimbursing travel and 
related  expenses  could  incentivise  participation.  In  addition,  recent  changes  to  financial 
regulations now permit certain entities to claim costs  – a model that could be extended to 
individual citizens, especially in projects that necessitate broad public involvement. Part of 
the  challenge  undoubtedly  springs  from  misunderstandings  or  a  lack  of  awareness  about 
existing  provisions.  Enlightening  both  institutions  and  citizens  about  these  opportunities  is 
vital. Lastly, fostering a culture of genuine citizen and societal engagement within both civil 
society and research organisations is crucial. Such a shift requires capacity-building, training, 
and an emphatic focus on the essence of citizen involvement. 

In  conclusion,  the  success  of  citizen  and  societal  engagement  in  R&I  hinges  on 
acknowledging the invaluable contributions of citizens, as well as dismantling barriers and 

1133 e.g. HORIZON-MISS-2021-OCEAN-05-03: Piloting citizen science in marine and freshwater 
domains; HORIZON-WIDERA-2021-ERA-01-60: A capacity-building and brokering network to 
make citizen science an integral part of the European Research Area. 
1134 Climate KIC, Climathon, 2023.  
1135 Climate Launchpad, 2023. 
1136 European Commission, Valorising research through citizens’ engagement: how to run 
hackathons with citizens, 2022. 

708 

 
 
harnessing the resources available for enhanced collaboration. Greater awareness is needed 
regarding the resources and funding necessary for effective citizen and societal engagement. 
Through the use of established, dedicated methodologies, reaching out and connecting with 
citizens can become a more streamlined and impactful process. 

1.3.3.4. 

Co-assessment with citizens and other stakeholders 

As  defined  in  the  Horizon  Europe  Programme  Guide1137,  co-assessment  tasks  involve 
activities such as citizens assisting with the consistent monitoring, evaluation and provision 
of governance feedback for projects, policies or programmes, potentially on a repeated or 
continuing  basis.  However,  the  research  team  found  no  active  co-assessment  activities 
taking  place  within  the  implementation  of  Horizon  Europe.  The  EC  officials  interviewed 
confirmed  that  co-assessment  activities  would  benefit  the  programme,  and  would  allow 
confirmation  as  to  whether  the  input  of  citizens  into  policy  design  and  research  agenda-
setting had been successfully integrated during the programme’s implementation (e.g. if the 
visions, needs and concerns identified during public consultations had been integrated into 
the work programmes and implemented during R&I activities). The EC officials also confirmed 
the current absence of such initiatives. 

1.3.4.  Effectiveness 

Given the current early stage of Horizon Europe, evaluating its effectiveness is not an easy 
task, and is complicated yet further by limited scope of project reporting data (which, among 
other key indicators and dimensions, outlines the citizen and societal engagement activities 
of the projects funded). Despite these challenges, the desk research and interviews carried 
out for this cases study delved into the monitoring system that relates to the societal uptake 
of R&I, and its limitations. In addition, by analysing Horizon Europe’s work programmes and 
topics, the study team identified those areas of the programme in which societal involvement 
was most anticipated. Furthermore, results from the beneficiary survey offered preliminary 
insights into the volumes of relevant activity under each Horizon Europe pillar. 

1.3.4.1. 

The monitoring system for KIP6 and its limitations 

Overall,  Horizon  Europe  aims  to  be  more  impact-driven.  Key  to  driving,  tracking  and 
ultimately assessing the impact of the programme is the introduction of Key Impact Pathways 
(KIPs). These impact pathways describe the logic of the intervention and the routes via which 
the programme is expected to eventually lead to effects over time. This approach enables 
policy-makers and the wider public to gain regular insights into the effects and benefits of the 
programme.  Although  historically,  R&I  evaluations  have  primarily  focused  on  measuring 
scientific outputs and impacts, there is increasingly widespread interest and evident political 
necessity in assessing the broader societal and economic returns of R&I spending. Hence, 
the pathways are grouped into scientific  impact (KIPs 1-3), societal impact (KIPs 4-6) and 
economic impact (KIPs 7-9). Horizon Europe is expected to achieve societal impact in terms 
of  addressing  the  EU’s  policy  priorities  (as  well  as  global  challenges  including  the  UN 
Sustainable Development Goals) by pursuing the principles of Agenda 2030 and the goals of 
the Paris Agreement through R&I; by delivering benefits and impact through R&I missions 
and  European  Partnerships;  and  by  strengthening  the  uptake  of  innovations  in  society, 
ultimately contributing to people’s well-being. In this regard, R&I activities are expected to 
contribute  to  objectives  that  have  been  democratically  defined  by  society.  This  approach 
constitutes a major advance for science and for society, putting scientific knowledge at the 
service of society.  

This focus on societal value is an important novelty of Horizon Europe. None of the 55 KPIs of 
the  previous  Framework  Programme,  Horizon  2020,  focused  on  societal  impacts.  In  terms  of 

1137 Horizon Europe Programme Guide (2023, v3.0). 

709 

 
 
strengthening  the  uptake  of  R&I  in  society,  Horizon  2020  included  efforts  to  mainstream  and 
monitor  citizen  and  societal  engagement  through  the  cross-cutting  issue  of  Science  and 
Society/responsible  research  and  innovation  (RRI).  Horizon  Europe’s  impact  monitoring 
framework  provides  an  intervention  logic  for  tracking  progress  towards  scientific,  societal  and 
economic impacts across the whole programme. As its title indicates, this framework focuses on 
providing a practical way to monitor (and regularly report on) impacts as they emerge along the 
impact pathways. To make the framework feasible (both in terms of implementation and its ability 
to regularly generate relevant, reportable data), as well as being uniformly applicable across the 
entire  FP,  the  framework  relies  extensively  on  proxy  indicators,  and  is  limited  by  the  extent  of 
existing  data.  Societal  impact  is  tracked  through  three  impact  pathways,  each  with  their  own 
background,  data  sources  and  indicators  as  defined  in  the  ‘Indicator  and  methodology 
handbook’1138. In the context of the present case study, KIP6 ‘Strengthening the uptake of R&I in 
society’ is the most relevant (see Table 3 below). 

Indicators 

Short-term indicator: 
Co-creation 

Medium-term 
indicator: 
Engagement 

Long-term: Societal 
R&I uptake 

Assessment 
indicators 

Methods of 
assessment 

FP projects in which 
EU citizens and end-
users contribute to the 
co-creation of R&I 
content. 

Number and share of 
FP beneficiaries with 
citizen and end-user 
engagement 
mechanisms after the 
FP project. 

FP co-created 
scientific results and 
innovative solutions. 

This indicator tracks 
the number and share 
of Horizon Europe 
projects in which EU 
citizens, workers and 
other end-users 
contribute to the co-
creation of R&I 
content. In addition, 
the indicator estimates 
the role of such 
contributions. This is 
calculated on the basis 
of a simple count of all 
ongoing and 
completed Horizon 
Europe projects in 
which EU citizens, 
workers and other end-
users contribute to the 
co-creation of R&I 
content. 

This indicator 
measures the effects 
of the heightened 
attention to and 
demand for co-creation 
in Horizon Europe 
through the 
development of citizen 
engagement 
mechanisms in 
beneficiary entities 
(such as social 
dialogue and social 
partner projects, 
citizen forums, 
participatory research, 
co-creation facilities, 
experimental sites and 
similar). 

This long-term 
indicator focuses on 
the uptake and 
outreach of 
Framework 
Programme co-
created scientific 
results and 
innovative solutions. 
Its aim is to assess 
the extent of the 
uptake and outreach 
of the scientific 
results (e.g. in the 
fields of science, 
innovation, the 
economy and 
society) and 
innovative solutions 
originating from the 
programme. 

Data sources 

EC administrative and 
monitoring data 
submitted by HE 
participants. 

EC administrative and 
monitoring data 
submitted by HE 
participants. 

Study will be 
launched for 
assessment. 

Table 3. KIP6 – ‘Strengthening the uptake of R&I in society’. Source: SWD(2023) 132 final: Evidence Framework on 
monitoring and evaluation of Horizon Europe. 

1138 Indicator and methodology handbook, 2022: https://op.europa.eu/en/publication-detail/-
/publication/2236c81c-c9bd-11ec-b6f4-01aa75ed71a1/language-en/format-PDF/source-
256388146 

710 

 
 
Insights  gathered  during  the  desk  research  and  interview  programme  carried  out  for  the 
present  study  show  that  establishing  causal  relationships  between  R&I  projects  (and  their 
portfolios)  and  observed  changes  in  society  poses  a  theoretical  challenge,  as  well  as 
empirical  and  methodological  problems.  More  specifically,  the  following  limitations  in 
monitoring the progress along KIP6 were noted: 

•  Delay  in  observing  impacts:  it  is  challenging  to  demonstrate  short-term  effects  of 
programmes that are inherently designed to produce long-term outcomes and impacts. 
Ripple effects may only be noticed years after a project’s conclusion, making monitoring 
and measurement difficult. By their nature, R&I activities generate impacts that manifest 
themselves  far  in  the  future,  with  a  considerable  time  gap.  This  extended  timeframe 
often makes it impossible to trace and attribute the exact source of the impact.  

•  Lack of assessment frameworks and impact grids to estimate the aggregated impact 
of EU-funded projects. The data collected either do not point precisely to the results and 
impacts achieved, or they do not capture qualitative aspects of the implementation (in 
other  words,  it  may  not  be  clear  what  the  data  collected  means,  and  how  to  assess 
whether the expected progress has been made). Lastly, an evaluation goes beyond an 
assessment of what has happened; it also considers why it has happened. Hence, the 
impact scheme needs to consider the diversity of situations/factors that could lead to a 
particular societal impact (e.g. the nature of the research project, the strategies used for 
citizen and societal engagement, the urgency of solutions). 

•  Perceptions of societal impact vary between pillars. According to the Staff Working 
Document (SWD) on the Evidence Framework on monitoring and evaluation of Horizon 
Europe1139, the data monitored under KIPs 4-6 are particularly relevant to Pillar 2 and the 
implementation  of  the  Missions.  Nevertheless,  these  KIP  indicators stretch  across  the 
whole  FP,  so  monitoring  data  will  be collected  from  any project  that  produces  eligible 
results.  

•  According to the stakeholders interviewed, the limited approaches to the evaluation of 
societal impact also result from the inability of the citizen engagement researcher and 
practitioner  community  to  capture  and  evaluate  tangible  societal  impact.  Despite 
numerous  efforts,  a  concrete  methodology  to  measure  and  highlight  added  value, 
especially  in  quantitative  terms,  has  not  yet  been  established.  According  to  the 
stakeholders interviewed, many attempts – even by renowned institutions across Europe 
that  claim  to  specialise  in  this  domain  –  have  produced  vague  and  predominantly 
qualitative  results.  The  framework  of  KIPs  provides  a  solution  for  overcoming  these 
problems,  as  it  has  the  potential  to  unify  the  criteria  for  impact  evaluation  across  the 
Member States and institutions.  

•  Post-project accountability. The interviews also highlighted the issue of monitoring the 
impacts of projects beyond their lifetime, as well as the absence of a control mechanism 
to validate promises of societal impact beyond a project’s lifecycle. Notably, this does 
not  imply  a  lack  of  commitment  on  the  part  of  the  project  beneficiaries,  but  external 
factors often play a role. Beneficiaries may change their career paths, shift their project 
focus, face new priorities, or encounter logistical challenges, making sustained tracking 
and follow-up not only challenging but at times, unfeasible. 

Navigating the challenges of evaluating societal impact is a shared endeavour, with research 
organisations around the world confronting similar issues to those encountered in Horizon 
Europe. As governments globally pivot towards emphasising societal impact in their research 
the  UK's  Research  Excellence 
funding  criteria  and  evaluation  mechanisms  (e.g. 

1139 SWD on the evidence framework on monitoring and evaluation of Horizon Europe, 
SWD(2023) 132 final. 

711 

 
 
Framework1140; the Engagement and Impact Assessment by Australian Research Council1141; 
the Performance-Based Research Fund in New Zealand1142), the quest for reliable evaluation 
tools  remains  paramount.  Both  national  and  regional  research  funding  programmes  are 
actively  exploring  methods  to  integrate  societal  dimensions  into  their  assessments, 
underlining  the  universal  need  for more  refined  instruments  to  gauge  the  broader societal 
ramifications of scientific endeavours. 

1.3.4.2. 

Indications of progress based on thematic analysis and survey results 

The effectiveness of mainstreaming citizen and societal engagement within Horizon Europe 
is a focal point of inquiry. An in-depth examination of the FP’s work programmes and topics 
has  identified  domains  in  which  expectations  of  societal  involvement  are  heightened. 
Furthermore, data from the beneficiary survey provide preliminary insights into the volumes 
of such activity under each of Horizon Europe’s pillars. Prior to the presentation of results, 
however, it is important to draw attention to certain methodological notes: 

1.  Since  the  reporting  of  the  first  project  results  funded  under  Horizon  Europe  is  still 
underway, data from the beneficiary survey provide preliminary insights into the societal 
engagement  activity  volumes  across  the  programme.  The  survey  of  Horizon  Europe 
beneficiaries  was  conducted  between  May  and  July  2023,  and  received  complete 
responses  from  5,414  beneficiaries.  While  this  survey  provides  valuable  insights  into 
citizen  and  societal  engagement  under  Horizon  Europe,  several  limitations  must  be 
considered.  Despite  the  best  efforts  of  the  study  team,  the  sample  may  not 
comprehensively represent all project activities. In addition, those who chose to respond 
to  the  survey  may  inherently  be  more  engaged  or  hold  stronger  opinions  concerning 
Horizon Europe than those who did not participate. This could skew the results towards 
a particular sentiment or viewpoint.  

2.  To  understand  the  extent  of  societal  involvement,  the  study  team  undertook  an 
exhaustive  analysis  of  the  work  programmes  spanning  two  periods  –  2021-2022  and 
2023-2024 – across key programme components. One primary objective of this was to 
identify  and  annotate  topic  calls  that  explicitly  encouraged  or  hinted  at  the  active1143 
engagement  of  citizens  and  end-users  (e.g.  ‘Proposals  should  address  societal 
engagement  to  foster  acceptance  of  the  solutions  in  the  local  and  regional 
communities’).  Several  challenges  emerged  during  this  analysis.  First,  variations  in 
terminology  –  work  programmes  employed  a  range  of  phrases  specific  to  their 
respective sectors, which made automated content analysis problematic. For instance, 
while general discussions might refer to ‘citizens’, the Health cluster specifically targets 
‘patients’.  Such  variation  necessitated  a  more  nuanced,  manual  approach  to  ensure 
accurate  interpretation.  Second,  there  were  concerns  over  subjectivity:  owing  to  the 
diverse phrasing and the subtleties in certain topic calls, the analysis had to occasionally 
rely on the judgement of the study team. As a result, despite the best efforts being made 
to remain objective, there may be instances in which the interpretation is influenced by 
the team's understanding and perspectives on citizen and societal engagement. 

The  sections  that  follow  present  a  compilation  of  the  observations  gathered  during  the 
analysis  of  topics  and  survey  data.  These  observations  map  out  indications  of  progress 
towards citizen and societal engagement across the Framework Programme. 

1140 UK REF (2021). 
1141 Australian Research Council: Engagement and Impact Assessment. 
1142 Tertiary Education Commission: Performance-Based Research Fund. 
1143 Topics that referred only to raising the awareness of citizens (without an active engagement 
component) were excluded from the count.  

712 

 
 
FP-wide distribution 

First,  the  study  team  looked  into  the  distribution  of  the  survey  results  across  the  whole 
Framework  Programme  (FP  and  individual  pillars,  Figure  1).  Overall,  48% of  beneficiaries 
who  responded  to  the  question  indicated  that  they  planned  to  engage  citizens,  citizen 
representatives  and/or end-users  in  the co-creation  of R&I  content.  The  highest shares  of 
positive responses can be seen for Pillar 2 (54%) and WIDERA (58%).  

Yes

No

Do not know/Not Applicable

%
8
4

%
9
2

%
4
2

%
6
3

%
4
3

%
0
3

%
4
5

%
6
2

%
0
2

%
7
3

%
1
3

%
2
3

%
8
5

%
3
2

%
9
1

F P   ( N = 5 3 2 0 )

P I L L A R   1  
( N = 1 6 6 0 )

P I L L A R   2  
( N = 3 0 5 0 )

P I L L A R   3  
( N = 3 4 0 )

W I D E R A   ( N = 2 7 0 )

Figure 1. Responses by FP beneficiaries to the question ‘Do you plan to engage citizens/citizen representatives/end-users in 
the co-creation of R&I content of your project?’ Source: prepared by the study team, based on the results of beneficiaries’ 
survey. 

When asked how they planned to engage citizens/end-users in their project activities (Figure 
2), most FP beneficiaries indicated co-design approaches (i.e. activities such as workshops, 
focus groups or other means to develop R&I agendas, roadmaps and policies, including deep 
discussions  on  the  implications,  ethics,  benefits  and  challenges  related  to  R&I  courses  of 
action or technology development), with the highest shares occurring under Pillar 1 (45%) 
and Pillar 3 (46%). 

Co-design

Co-creation

%
1
4

%
2
3

%
7
2

%
5
4

%
3
3

%
2
2

%
9
3

%
2
3

%
9
2

Co-assesment
%
6
4

%
8
2

%
6
2

%
4
4

%
2
3

%
4
2

F P   ( N = 4 0 2 8 )

P I L L A R   1  
( N = 7 3 6 )

P I L L A R   2  
( N = 2 8 6 6 )

P I L L A R   3  
( N = 1 8 0 )

W I D E R A   ( N = 2 4 6 )

Figure 2. Responses by FP beneficiaries to the question ‘Please indicate what are these activities to engage citizens/end-
users in your project’. Source: prepared by the study team, based on the results of beneficiaries’ survey. 

Note: only those who answered ‘Yes’ to the question ‘Do you plan to engage citizens/citizen representatives/end-users in 
the co-creation of R&I content in your project?’ were asked about specific engagement activities. 

The next subsections detail the survey results per pillar, together with the topic analysis.  

713 

 
 
 
 
 
Pillar 1 ‘Excellent Science’ 

Given Pillar 1’s focus on excellence (and, to a large extent, its bottom-up nature), the topic 
analysis yielded limited quantitative results. Although some mentions of citizen and societal 
engagement could be discerned, these mostly related to:  

•  Specific prizes and initiatives (e.g. ERC-2022-PERA, ERC-2024-PERA);  

•  The  need  to  train  researchers  on  topics  relating  to  citizen  and  societal  engagement. 
HORIZON-MSCA-2022-CITIZENS-01, for example, refers to ‘improved communication 
skills  and  competences  to  interact  with  a  non-research  audience’.  Meanwhile, 
HORIZON-MSCA-2021-DN-01-01:  MSCA  Doctoral  Networks  indicates  that  ‘Doctoral 
Networks  should  adequately  prepare  doctoral  candidates  for  increased  research 
collaboration and information-sharing made possible by new (digital) technologies (e.g. 
collaborative  tools,  opening  access  to  publications  and  to  research  data,  FAIR  data 
management, public engagement and citizen science, etc.)’; or  

•  Specific  infrastructures  that  should  accommodate  citizen  scientists  (e.g.,  HORIZON-
INFRA-2023-EOSC-01 indicated that ‘special attention should be put on involving user 
group(s) also from outside the H2020 INFRAEOSC community including – when relevant 
– citizen scientists and “the long tail” of science.’ 

Figure 3 shows that according to the results of the survey, ERC beneficiaries are the least 
keen of those under Pilar 1 to engage citizens, citizen representatives and end-users in the 
co-creation  of  R&I  content  in  their  projects  (18%  of  respondents).  Beneficiaries  of  other 
programme parts under Pillar 1 display a more positive outlook, with approximately 38% of 
researchers in the MSCA (both individual and collaborative projects) and INFRA planning to 
engage citizens. When asked how they planned to engage citizens/end-users in their project 
activities, most respondents under pillar 1 (Figure 4) indicated that they planned to use co-
design approaches (i.e. activities such as workshops, focus groups or other means to develop 
R&I agendas, roadmaps and policies, including deep discussions on the implications, ethics, 
benefits and challenges relating to R&I courses of action or technology development).  

%
0
1

,

8
4

%
0
6

,

2
3

%
0
3

,

9
1

Yes

No

Do not know/Not Applicable

%
0
7

,

8
3

%
0
3

,

6
3

%
0
0

,

5
2

%
0
7

,

7
3

%
0
8
9
2

,

%
0
5

,

2
3

%
0
8

,

8
3

%
0
2

,

4
3

%
0
0

,

7
2

E R C   ( N = 2 3 3 )

M S C A - P F   ( N = 8 0 8 ) M S C A - O R G   ( N = 4 6 7 )

I N F R A   ( N = 1 5 2 )

Figure 3. Responses by Pillar 1 beneficiaries to the question ‘Do you plan to engage citizens/citizen representatives/end-
users in the co-creation of R&I content in your project?’ Source: prepared by the study team, based on the results of 
beneficiaries’ survey. 

714 

 
 
Co-design

Co-creation

Co-assesment

%
0
1

,

1
5

%
0
3

,

3
5

%
0
4

,

4
2

%
0
8
7
1

,

%
0
0

,

5
5

%
0
6

,

3
4

%
0
1

,

3
2

%
0
0

,

5
1

%
0
4

,

5
5

Do not know/Not Applicable

%
0
4

,

2
7

%
0
9

,

8
3

%
0
0
2
3

,

%
0
0

,

2
1

%
0
5

,

4
3

%
0
2

,

6
3

%
0
9
6

,

E R C

M S C A - P F  

M S C A - O R G  

I N F R A

Figure 4. Responses by Pillar 1 beneficiaries to the question ‘Please indicate what are these activities to engage 
citizens/end-users in your project?’ Source: prepared by the study team, based on the results of beneficiaries’ survey.  

Note: only those who answered ‘Yes’ to the question ‘Do you plan to engage citizens/citizen representatives/end-users in 
the co-creation of R&I content in your project?’ were asked about specific engagement activities. 

Pillar 2 ‘Global Challenges and European Industrial Competitiveness’  

The  analysis  of  topics  under  Pillar  2  ‘Global  Challenges  and  European  Industrial 
Competitiveness’  provided  a  much  richer  set  of  quantitative  data  (given  both  its  focus  on 
societal  challenges  and  the  number  of  calls  per  cluster).  Figures  5  and  6  reveal  that  the 
highest share of topics encouraging citizen engagement appear in Cluster 1 (Health), Cluster 
2 (Culture) and Cluster 6 (Food). However, a noticeable decrease in such topics can be seen 
between  WP  2021-2022  and  WP  2023-2024.  In  terms  of  mentioning  citizen  science  as  a 
potential methodological approach, Cluster 6 (Food) leads the count, followed by Cluster 1 
(Health), Cluster 5 (Climate) and Cluster 4 (Climate).  

WP 2021-2022 

WP 2023-2024 

Cluster 6: Food

12%

41%

Cluster 6: Food

7%

27%

Cluster 5: Climate

2%

Cluster 4: Digital

Cluster 3: Civil security

Cluster 2: Culture

0%

0%

0%

18%

10%

Cluster 5: Climate

Cluster 4: Digital

23%

Cluster 3: Civil security

Cluster 2: Culture

14%

1%

2%

0%

0%

8%

8%

Cluster 1: Health

2%

35%

Cluster 1: Health

2%

27%

30%

0% 10% 20% 30% 40% 50%

0%

10% 20% 30% 40%

Citizen science

Societal engagement

Citizen science

Societal engagement

Figures 5 & 6. Share of topics encouraging societal engagement and citizen science in EU Missions. Source: analysis of 
Pillar 2 Work Programmes 2021-2022 and 2023-2024 performed by the study team.  

Note: the total number of topics analysed under WP 2021-2022 is 707 (Cluster 1: Health, 48; Cluster 2: Culture, 50; Cluster 
3: Civil security, 64; Cluster 4: Digital, 175; Cluster 5: Climate, 189; Cluster 6: Food, 181). The total number of topics 
analysed under WP 2023-2024 is 647 (Cluster 1: Health, 44; Cluster 2: Culture, 56; Cluster 3: Civil security, 48; Cluster 4: 
Digital, 132; Cluster 5: Climate, 191; Cluster 6: Food, 176) 

715 

 
 
 
 
 
The  analysis  of  the  results  of  the  beneficiaries’  survey  (Figure  7  and  Figure  8),  when 
compared against the analysis of topics under WP 2021-2022 (as most of the beneficiaries 
who responded to the survey represent projects under this programming period), reveals the 
extent to which citizen engagement correlates with beneficiaries’ plans to engage citizens, 
citizen representatives and end-users in their R&I activities. Specifically, the top three clusters 
in the topic analysis are the same as those with the largest shares of positive responses in 
the survey. As mentioned in previous sections, the aspect of societal and citizen engagement 
differs depending on the specificities of the topics concerned (i.e. more technical topics might 
require  less  engagement).  However,  these  results  also  show  that  the  way  in  which  the 
Commission  defines  topics  can  play  an  important  role  in  mainstreaming  the  citizen  and 
societal engagement narrative within these projects.  

%
0
8

,

0
7

%
0
3

,

2
6

%
0
5

,

5
2

%
0
2

,

2
1

%
0
8

,

7
1

%
0
4

,

1
1

Yes

No

Do not know/Not Applicable

%
0
0

,

2
5

%
0
7

,

6
2

%
0
3

,

1
2

%
0
2

,

3
3

%
0
6

,

3
3

%
0
2

,

3
3

%
0
9

,

5
4

%
0
5

,

1
3

%
0
6

,

2
2

%
0
4

,

9
6

%
0
7

,

8
1

%
0
9

,

1
1

C L U S T E R   1  
( N = 3 7 7 )

C L U S T E R   2  
( N = 3 3 2 )

C L U S T E R   3  
( N = 1 5 0 )

C L U S T E R   4  
( N = 6 3 9 )

C L U S T E R   5  
( N = 7 4 5 )

C L U S T E R   6  
( N = 8 0 7 )

Figure 7. Responses by Pillar 2 beneficiaries to the question ‘Do you plan to engage citizens/citizen representatives/end-
users in the co-creation of R&I content in your project?’ Source: prepared by the study team, based on the results of 
beneficiaries’ survey. 

%
0
8

,

9
6

%
0
3

,

4
6

%
0
2

,

3
5

Co-design
%
0
7

%
0
5
4
6

,

,

6
6

Co-creation

Co-assesment

%
0
8
2
6

,

%
0
5

,

1
6

%
0
2

,

6
4

%
0
6

,

7
4

%
0
9

,

1
6

%
0
7

,

6
4

%
0
9

,

1
3

Do not know/Not Applicable

%
0
2

,

0
7

%
0
0

,

2
5

%
0
2
3
5

,

%
0
2

,

9
6

%
0
6
9
5

,

%
0
6

,

8
4

%
0
3
4

,

%
0
6
2

,

%
0
6
2

,

%
0
0

,

0
1

%
0
7
4

,

%
0
5
4

,

C L U S T E R   1  

C L U S T E R   2

C L U S T E R   3

C L U T E R   4

C L U S T E R   5

C L U S T E R   6

Figure 8. Responses by Pillar 2 beneficiaries to the question ‘Please indicate what are these activities to engage 
citizens/end-users in your project’. Source: prepared by the study team, based on the results of beneficiaries’ survey. 

Note: only those who answered ‘Yes’ to the question ‘Do you plan to engage citizens/citizen representatives/end-users in 
the co-creation of R&I content in your project?’ were asked about specific engagement activities. 

Pillar 3 ‘Innovative Europe’  

The analysis of topics under Pillar 3 provided fewer opportunities for quantitative analysis. 
Given  the  industry-focused  nature  of  these  activities,  such  results  were  to  be  expected. 

716 

 
 
 
However, calls  for certain  actions  (e.g.  the  EIE  CONNECT  programme,  which  focuses  on 
creating links between innovators and civil society) and with regard to competitions (e.g. the 
EIC European Social Innovation Competition and the European Capital Innovation Award) 
underline the need to engage with civil society. However, the result of the survey (Figure 9 
and Figure 10) show that despite the lack of encouragements provided in the programme, 
beneficiaries in the Pathfinder, Transition, EIE and Accelerator programme parts do plan to 
engage citizens in their R&I activities (most notably in the EIE, with 53% of responses being 
positive).  Co-design  is  the  most  widely  preferred  method  of  collaboration  across  all 
programme parts analysed.  

%
0
0

,

8
3

%
0
0

,

1
3

%
0
0
1
3

,

Yes

No

%
0
0

,

4
4

%
0
0

,

8
2

%
0
0

,

8
2

Do not know/Not Applicable
%
0
5

,

2
5

%
0
9

,

3
3

%
0
6

,

3
1

%
0
8

,

1
4

%
0
7
4
3

,

%
0
5

,

3
2

P A T H F I N D E R   ( N = 1 5 8 )

T R A N S I T I O N   ( N = 2 5 )

E I E   ( N = 5 9 )

A C C E L E R A T O R   ( N = 9 8 )

Figure 9. Responses by Pillar 3 beneficiaries to the question ‘Do you plan to engage citizens/citizen representatives/end-
users in the co-creation of R&I content in your project?’ Source: prepared by the study team based on the results of 
beneficiaries’ survey. 

%
0
4

,

9
6

%
0
6

,

0
3

%
0
6

,

8
2

Co-design

Co-creation

%
0
6

,

3
6

%
0
5

,

4
5

%
0
3

,

7
2

Co-assesment

%
0
0

,

1
7

%
0
8

,

4
5

%
0
2

,

5
4

%
0
6
3
6

,

%
0
5
5
4

,

%
0
4

,

6
3

P A T H F I N D E R

T R A N S I T I O N

E I E

A C C E L E R A T O R

Figure 10. Responses by Pillar 3 beneficiaries to the question ‘Please indicate what are these activities to engage 
citizens/end-users in your project’. Source: prepared by the study team, based on the results of beneficiaries’ survey. 

Note: only those who answered ‘Yes’ to the question ‘Do you plan to engage citizens/citizen representatives/end-users in 
the co-creation of R&I content in your project?’ were asked about specific engagement activities. 

WIDERA 

WIDERA work programmes provided a potent case for analysis (see Figure 11), with 41% of 
calls under the 2021-2022 work programme highlighting the need for proposals to address 
citizen and societal engagement, compared with 33% under the 2023-2024 work programme. 
Compared  with  other  programme  parts,  citizen  science  was  mentioned  more  often  as  a 
potential participatory approach (32% of topics in 2021-2022; 27% in 2023-2024).  

717 

 
 
 
 
Societal engagement

Citizen science

%
1
4

%
3
3

%
2
3

%
7
2

W P   2 0 2 1 - 2 0 2 2

W P   2 0 2 3 - 2 0 2 4

Figure 11. Share of topics encouraging societal engagement and citizen science in WIDERA. Source: analysis of WIDERA 
Work Programmes 2021-2022 and 2023-2024 performed by the study team. 

Note: the total number of topics analysed under WP 2021-2022 is 44. The total number of topics analysed under WP 2023-
2024 is 34. 

The results of the survey confirm the broad application of citizen and societal engagement 
methods  in  both  parts  of  WIDERA,  with  a  positive  response  from  53%  of  beneficiaries 
representing Widening projects and an impressive 68% in the ERA programme parts. Such 
results can, however, be expected given the nature of the programme. 

Plans to engage with citizens 

Types of citizen engagement activities 

Do not know/NA

22,40%

14,00%

No

25,30%

18,00%

Yes

52,40%

68,00%

WIDERA-WIDENING (N=170)

WIDERA-ERA (N=100)

WIDERA-ERA

WIDERA-WIDENING

Do not know/NA

4,40%
4,70%

Co-assesment

Co-creation

Co-design

36,80%
39,50%

58,80%

44,20%

70,60%
70,90%

Figures 12 &13. Responses by WIDERA beneficiaries to the questions ‘Do you plan to engage citizens/citizen 
representatives/end-users in the co-creation of R&I content in your project?’ and ‘Please indicate what are these activities to 
engage citizens/end-users in your project?’ Source: prepared by the study team, based on the results of beneficiaries’ 
survey. 

Note: only those who answered ‘Yes’ to the question ‘Do you plan to engage citizens/citizen representatives/end-users in 
the co-creation of R&I content in your project?’ were asked about specific engagement activities. 

EU Missions 

Analysis of the work programmes of the EU Missions (see Figures 14 and Figure 15) reveals 
a strong focus on citizen and societal engagement across all Missions, especially under WP 
2021-2024, with the ‘Ocean’ (83%), ‘Climate’ (67%) and ‘Cancer’ (56%) Missions leading the 
count.  Under  WP  2023-2024,  however,  the  shares  of  topics  including  citizen  and  societal 
engagement decreased. Due to the fact that the beneficiaries’ survey did not include projects 
funded under the EU Missions work programmes, it is not possible to evaluate indications 
towards progress.  

718 

 
 
 
 
 
 
WP 2021-2022 

WP 2023-2024 

Societal engagement

Citizen science

Societal engagement

Citizen science

83%

80%

67%

56%

50% 47%

17%

17%

11%

16% 20%

33%

100%

64% 67%
45%

50%

44%

33%

0%

0%

0%

0%

0%

Figures 14 & 15. Shares of topics in EU Missions that encourage societal engagement and citizen science. Source: analysis 
of EU Missions Work Programmes 2021-2022 and 2023-2024 performed by the study team. 

Note: the total number of topics analysed under WP 2021-2022 is 76 (Climate, 12; Cancer, 9; Ocean, 23; Cities, 8; Soil, 19; 
Cross-mission, 5. The total number of topics analysed under WP 2023-2024 is 33 (Climate, 3; Cancer, 4; Ocean, 11; Cities, 
3; Soil, 9; Cross-mission, 3). 

1.3.5.  EU added value 

The European Union has played a pivotal role in advancing citizen and societal engagement 
across  its  Member  States.  Impetus  and  investments  provided  by  the  EU  have  catalysed 
broader and deeper engagement, ensuring that societal integration and citizen participation 
are  prioritised.  Without  the  EU’s  proactive  approach,  the  evolution  of  such  engagement 
initiatives  at  national  level  would  probably  have  been  much  slower.  The  European 
Commission  has  laid  significant  groundwork  for  the  fostering  of  citizen  and  societal 
engagement.  Its  financial  and  ideological  support  has  enabled  numerous  projects  to 
effectively  implement  their  citizen  and  societal  engagement  strategies,  demonstrating 
profound impact. Such initiatives would arguably have faced significant challenges without 
the backing of the Commission.  

The  majority  of  participants  in  the  interview  programme  agreed  that  in  the  absence of  the 
Framework Programmes facilitated by the EU, national-level progress on citizen and societal 
engagement  would  either  have  stagnated  or  progressed  at  a  much  slower  pace.  Some 
nations  may  have  been  quicker  in  their  response  to  the  changing  landscape,  but  it  is 
international projects, underpinned by EU support, that  have truly ignited swift advance at 
both  national  and  regional  levels.  The  funding  mechanisms  of  the  EU,  even  if  they  are 
occasionally  criticised,  have  undeniably  spurred  action.  The  conditions  embedded  within 
funding calls compel researchers and organisers to consider citizen and societal engagement 
more  deeply.  As  the  EU  has  consistently  amplified  its  emphasis  on  citizen  and  societal 
engagement  in  its  work  programmes,  it  has  prompted  corresponding  shifts  in  national 
strategies and priorities. The most recent examples of such shifts include those in Latvia and 
Romania.  More  specifically,  the  Latvian  Open  Science  Strategy1144,  launched  in  2021, 
references  the  EU  Framework  Programmes  and  their  requirements  (i.e.  researchers  can 
expect  conditions  similar  to  those  under  international  research  programmes).  Meanwhile, 
Romania’s National Strategic Framework for Open Science1145, adopted in 2022, details the 

1144 Latvian Open Science Strategy 2021-2027.  
1145 White Paper on the Transition to Open Science 2023-2030. 

719 

 
 
 
 
 
 
 
implementation of Open Science principles, challenges and actions. It also refers to the EU 
Framework Programmes, and builds upon the Council Conclusions on Research assessment 
and the implementation of Open Science1146 as well as the Recommendation of the European 
Commission on access to and preservation of scientific information1147 among other key EU 
strategic documents.  

1.4.  Success factors and perceived barriers 

Citizen  and  societal  engagement  are burgeoning  areas  within  Horizon  Europe  that  aim  to 
integrate the collective intelligence and expertise of diverse stakeholders to foster richer and 
more inclusive research and innovation processes. Based on the findings of this case study, 
several  drivers  and  barriers  play  a  pivotal  role  in  influencing  the  effectiveness  of  these 
approaches. 

1.4.1.  Barriers 

•  Vague definitions: the terms ‘co-creation’, ‘co-design’ and ‘citizen engagement’ remain 
ambiguous, and carry different meanings for various stakeholders. In addition, the way 
in which societal engagement is defined in the work programmes analysed in this case 
study also differs (with some instances of very detailed explanations as to why and how 
citizens  should  be  engaged,  and  others  that  simply  mention  them  alongside  other 
stakeholder groups). This lack of clarity may inadvertently reduce engagement efforts to 
mere box-ticking exercises. 

•  Lack of visibility and understanding: a number of modalities and incentives for citizen 
and societal engagement exist across the pillars of Horizon Europe. However, there is a 
noticeable gap in the flow of information from EU institutions to researchers, leading to 
the  inadequate  integration  of  co-creation,  co-design  and  citizen  engagement  into  R&I 
practices. As a result, many stakeholders in the field of R&I find it challenging to relate 
citizen and societal engagement to their activities (e.g. why citizens need to be included; 
how resource-intensive the engagement process is; what the most relevant and effective 
strategies  are in  different  contexts).  This  gap  in  understanding  is  also  an  issue  in  the 
context of proposal evaluation (e.g. limited guidance for evaluators on how to approach 
the citizen and societal engagement aspects in evaluations).  

•  Limited  incentives  for  citizens:  collaborating  with  non-academic  partners  can  be 
intricate. A skills gap exists in relation to working with public actors or citizens unfamiliar 
with  scientific  routines,  which  needs  to  be  addressed  in  order  to  ensure  effective  co-
creation. In addition, numerous stakeholders who were interviewed expressed concerns 
regarding the lack of payment for citizens’ contributions. This challenge raises questions 
about how citizens' contributions can be recognised and rewarded under Horizon Europe 
and future Framework Programmes. 

•  Practical  implementation  challenges:  the  inclusion  of  participatory  elements  in  R&I 
strategies  does  not  in  itself  guarantee  inclusive  and  effective  stakeholder  and  public 
engagement.  According  to  various  stakeholder  groups,  how  balanced  and  credible 
citizen  and  societal  engagement  activities  are  depends  on  how  these  elements  are 
implemented  in  practice.  The  research  community  often  faces  hurdles in  the  planning 
and implementation of public engagement practices (e.g. a lack incentives; a frequent 
tendency to view citizen and societal engagement as an administrative burden; and a 
lack of training on how to effectively engage with the public).  

1146 Council Conclusions on Research assessment and implementation of Open Science, 2022. 
1147 EC Recommendation (EU) 2018/790 of 25 April 2018 on access to and preservation of 
scientific information. 

720 

 
 
1.4.2.  Drivers 

Despite  various  shortcomings,  existing  EU  participation  instruments  provide  a  solid 
foundation through the following set of drivers: 

•  Previous EU Initiatives: past projects conducted under Horizon 2020 serve as valuable 
repositories of lessons learned, best practices and methodologies to drive forward co-
creation  efforts.  In  this  regard,  the  Commission  puts  significant  efforts  into  informing 
potential  beneficiaries  with  regard  to  the  insights  gained  (especially  under  SwafS 
programmes) by publishing reports and guidance, and adding references into call texts.  

•  The criterion of excellence and the Horizon Europe participant guide serve as key 
guiding  elements  in  encouraging  the  participation  of  citizens  and  other  non-traditional 
research  actors.  Both  of  these,  underline  the  importance  of  citizen  and  societal 
engagement, and provide guidance as to how to apply it when preparing Horizon Europe 
project  proposals.  Even  so,  most  interviewees  felt  that  more  could  be  done  to 
mainstream citizen and societal engagement across different channels. 

•  Based on an analysis of topic call texts, a large number of topics highlight the need 
to include non-traditional research actors. In addition, each programme pillar offers 
diverse possibilities for the inclusion of citizens or rewarding those who implement such 
activities during the project’s implementation. This again signifies the active efforts made 
by the Commission to mainstream citizen and societal engagement narratives.  

•  The  evaluation  and  monitoring  system,  while  still  in  development,  provides  an 
incentive for project beneficiaries to consider citizen and societal engagement activities 
during the implementation of their projects. 

In addition to programme-level drivers, the national context also serves as a driver for the 
broader application of public engagement in R&I: 

•  Well-articulated  national  strategies:  countries  such  as  Germany  and  Spain,  which 
have in place robust citizen science strategies and vibrant citizen science activities, have 
demonstrated increased citizen and societal engagement efforts in R&I. 

In summary, while the push towards citizen and societal engagement under Horizon Europe 
offers  transformative  potential  for  research  and  innovation,  it  is  essential  to  navigate  and 
address the relevant drivers and barriers effectively. Only through such a nuanced approach 
can these methodologies realise their full promise, fostering research that is both holistic and 
reflective of diverse perspectives. 

1.5.  Conclusions  

This  section  summarises  the  preceding  analysis,  and  provide  the  main  conclusions  and 
lessons learned.  

Relevance 

Citizen and societal engagement is an integral part of the EU’s Open Science policy and 
an agreed priority for the European Research Area. This is reflected in Horizon Europe, 
which emphasises citizen and societal engagement across the Framework Programme. 
The intentional mainstreaming of citizen and societal engagement throughout Horizon 
Europe is far more than a symbolic gesture – it is a pragmatic strategy to harness the 
collective  intelligence,  insights  and  creativity  of  Europe’s  diverse  populace.  This  is 
particularly  critical  as  Europe  grapples  with  multifaceted  and  interrelated  challenges, 

721 

 
ranging  from  climate  change  to  socioeconomic  disparities,  for  which  singular  expert-
driven solutions may no longer suffice. 

By embedding co-creation and co-design principles into the very fabric of the Framework 
Programme,  the  EU  affirms  its  commitment  to  an  Open  Science  ethos  that  places 
citizens  and  civil  society  at  the  heart  of  research  endeavours.  This  holistic  approach, 
while  challenging  to  implement  due  to  the  vast  diversity  of  stakeholder  perspectives, 
paves  the  way  for  more  grounded,  relevant  and  actionable  research  outcomes.  It  is 
evident  that  for  such  citizen  and  societal  engagement  efforts  to  achieve  their  full 
potential,  they  must  be  underpinned  by  clear  guidance,  continuous  dialogue,  and  a 
shared understanding among all stakeholders while building on the efforts undertaken 
by Horizon 2020.  

Coherence 

Horizon Europe boasts a diverse spectrum of citizen and societal engagement practices. 
These  vary  in  multiple  dimensions:  their  setting,  the  entities  that  organise  them,  their 
relevance to different phases of the project or policy cycle, and the fact that they cater 
to various target groups and differ in their duration. Despite these variations, a thread of 
coherence is woven through the operational objective of involving citizens and end-users 
in co-design and co-creation processes across all pillars. This common objective binds 
the  diverse  practices  together,  ensuring  their  alignment  with  the  overarching  goals  of 
Horizon Europe. 

It  is,  however,  essential  to  acknowledge  that  while  citizen  and  societal  engagement 
stands as a foundational objective, its interpretation and application can differ between 
individual  parts  of  the  programme.  Some  programme  parts  might  stress  more  direct 
public participation in R&I (e.g. in the Clusters and Missions); others might lean towards 
a more prize-based approach that rewards those who have excelled in their citizen and 
societal engagement efforts (e.g. the ERC prizes), or capacity-building approach aimed 
at  bolstering  the  relevant  skills  in  Open  Science  (e.g.  in  the  MSCA).  This  plurality  of 
understanding and implementation underscores the flexibility and adaptability of Horizon 
Europe, but also highlights the need for continuous dialogue and clarity in defining and 
realising citizen and societal engagement across its various facets. 

Efficiency 

Using comprehensive desk research and insights from interviews, the study team has 
successfully  catalogued  the  various  mechanisms  used  for  citizen  and  societal 
engagement throughout the Horizon Europe policy and project cycle, namely: 

•  The European Commission's efforts to foster an inclusive approach in R&I policy 
design  through  Horizon  Europe,  especially  through  public  consultations,  are 
commendable. The inclusion of diverse stakeholders and citizens promises richer, 
more nuanced policy outcomes that align with broader societal values and concerns. 
Initiatives such as the public consultation on the Green Deal call and the Horizon 
Europe  foresight  are  exemplary  of  the  Commission’s  commitment  to  such 
engagement.  However,  certain  challenges  remain.  The  divergence  between  the 
intent behind these consultations and their actual influence, as seen in the Horizon 
Europe  Strategic  Plan  and  certain  Missions  such  as  the  Cancer  Mission,  raises 
questions  about  the  depth  of  citizen  and  stakeholder  impact.  While  the  numbers 
might indicate successful outreach, the substance and the changes that result from 
these  consultations  must  be  critically  assessed.  Going  forward,  Horizon  Europe 

722 

 
must  ensure  that  its  public  engagement  mechanisms  are  not  just  participatory  in 
name, but are robust platforms where diverse voices have a genuine influence on 
policy outcomes. 

•  Steering  R&I  initiatives  towards  societal  engagement.  Horizon  Europe’s 
intention  to  place  citizen  and  societal  engagement  at  the  forefront  of  its  R&I 
framework  is  evident.  Various  mechanisms,  such  as  the  wording  of  work 
programmes, the Horizon Europe Programme Guide, and the inclusion of dedicated 
calls,  serve  to  emphasise  this  commitment.  This  attention  to  detail  provides  a 
roadmap for researchers to integrate societal considerations effectively. However, 
challenges exist, particularly with regard to the clarity of guidelines, discrepancies in 
evaluations, and budgetary constraints that can inhibit genuine citizen and societal 
engagement.  As  observed,  while  there  is  a  rich  repository  of  best  practices  and 
success stories, their reach and applicability depend on effective dissemination and 
awareness.  The  present  situation  suggests  a  dichotomy  whereby  the  intent 
regarding citizen and societal engagement is clear, but its actual operationalisation 
requires further refinement. For Horizon Europe to truly capitalise on the benefits of 
citizen  and  societal  engagement,  there  needs  to  be  synergy  between  the 
mechanisms employed, the resources allocated, and the guidance provided.  
•  Societal engagement in concrete R&I activities. Horizon Europe's R&I ecosystem 
emphasises  holistic  citizen  and  societal  engagement,  transitioning  from  passive 
observation to active involvement in diverse roles, such as users, co-designers and 
data  providers.  While  various  engagement  platforms  exist,  the  present  analysis 
accentuates three areas: Financial  Support for Third  Parties, citizen science,  and 
hackathons. Each of these presents unique opportunities, from redirecting funds to 
smaller  entities  through  FSTP,  to  integrating  public  insights  into  scientific 
endeavours.  However,  challenges  persist,  especially  in  selecting  the  right 
engagement  method  and  ensuring  genuine  and  incentivised  citizen  participation. 
Addressing  these  challenges,  through  avenues  such  as  monetary  incentives  and 
leveraging  past  experiences,  is  crucial  to  ensuring  robust  citizen-centric  R&I 
outcomes under Horizon Europe. 

•  Project  reporting.  Citizen  and  societal  engagement  is  a  cornerstone  of  Horizon 
Europe, echoed in every facet of the programme. As emphasised by EC officials in 
interviews, project submissions are evaluated not only for their scientific merit but 
also  for  their  alignment  with  key  indicators  of  citizen  and  societal  engagement, 
notably  Key  Impact  Pathway  6.  This  ensures  a  twofold  approach:  projects  must 
demonstrate ongoing citizen and end-user participation throughout their lifecycle, as 
well as sketching out sustainable citizen and societal engagement strategies post-
completion.  

•  Co-assessment  tasks,  which  encompass  citizen-led  monitoring,  evaluation  and 
feedback  on  various  initiatives,  remain  largely  untapped  under  Horizon  Europe. 
Although these activities represent a logical extension of the EC's existing citizen 
and societal engagement through public input during policy formulation, the research 
team and corroborating interviews with EC officials reveal a conspicuous absence 
of any active co-assessment ventures. 

Effectiveness 

Horizon  Europe  signifies  a  pivotal  evolution  in  the  evaluation  of  R&I,  marking  a 
pronounced  shift  towards  gauging  impact,  particularly  through  the  introduction  of  Key 
Impact Pathways (KIPs). These pathways outline the logic and mechanisms by which 
the FP  expects to  manifest its  effects over time,  emphasising societal, economic and 
scientific  impacts.  In  contrast  to  its  predecessor,  Horizon  2020,  Horizon  Europe 
prominently  underlines  societal  value,  marking  a  remarkable  advance  of  the  former's 

723 

 
lack of KPIs targeting societal impacts. However, challenges have been identified in the 
monitoring system for KIP6, which focuses on strengthening the uptake of R&I in society. 
The  delays  inherent  in  observing  long-term  impacts,  the  absence  of  comprehensive 
assessment frameworks, discrepancies regarding perceptions of societal impact across 
different  programme  pillars,  and  inadequacies  in  the  guidelines  for  reporting  societal 
impacts, all represent significant hurdles. In particular, the lack of a well-defined method 
for assessing tangible societal impact, despite multiple attempts, underlines the complex 
nature of this challenge. Furthermore, post-project impact monitoring, coupled with the 
changing dynamics and priorities of project beneficiaries, magnifies this complexity. As 
the emphasis on societal impact grows globally, the challenges faced by Horizon Europe 
resonate universally, emphasising the need for robust, innovative and reliable evaluation 
tools  that  can  capture  the  broad  spectrum  of  societal  implications  tied  to  scientific 
pursuits. 

The results of the analysis of work programmes reveal the highest shares of topics that 
focus on citizen and societal engagement occur under Pillar 2, WIDERA and Missions. 
This  results  from  the  logic  behind  the  funding  mechanism,  which  focuses  on  solving 
societal problems and not on scientific excellence or practical application in the market. 
However, an alarming trend towards a smaller share of topics encouraging researchers 
to consider citizen and societal engagement under WP 2023-2024 can be seen in Pillar 
2,  WIDERA  and  Missions.  With  regard  to  actual  citizen  and  societal  engagement 
activities planned (as indicated in the beneficiaries’ survey), WIDERA, Cluster 2, Cluster 
6  and  Cluster  1  lead  with  highest  shares  of  beneficiaries  confirming  that  they  plan  to 
conduct citizen and societal engagement activities. It is important to note that other parts 
of the FP (in which societal engagement is not widely present in the WPs) also display 
encouraging  signs  with  regard  to  the  willingness  of  the  Horizon  Europe  research 
community to include citizens as possible partners in the co-creation, co-design and co-
assessment  of  their  R&I  activities.  However,  the  results  of  the  survey  should  be 
approached with the caution, as beneficiaries were only asked about their plans and not 
about actual activities carried out.  

EU added value 

The European Union, through its strategic vision and financial mechanisms, has been a 
linchpin  in  promoting  citizen  and  societal  engagement  across  its  Member  States.  By 
creating a robust framework and championing the ethos of citizen involvement, the EU 
has undeniably accelerated the pace of citizen and societal engagement initiatives that 
might otherwise have languished at national level. While individual Member States might 
have  ventured  into  these  territories,  it  was  the  collective  influence  of  EU-backed 
international projects that has galvanised rapid advances both regionally and nationally. 
The  EU’s  funding  mechanisms,  although  subject  to  occasional  criticism,  have  been 
pivotal  to  this  journey,  shaping  research  agendas  and  compelling  the  deeper 
contemplation of citizen and societal engagement. Yet, with these advancements come 
inherent challenges: the tangible and often substantial costs associated with establishing 
meaningful  public  engagement.  Such  endeavours  require  the  creation  of  conducive 
environments, investments in time, and the evolution of structures to truly harness public 
participation. 

Literature sources 

•  Alemanno, A (2022) Towards a permanent citizens' participatory mechanism in the 
EU. Study requested by the European Parliament's Committee on Constitutional 

724 

 
Affairs. Available here: 
https://www.europarl.europa.eu/thinktank/en/document/IPOL_STU(2022)735927 

•  Australian Research Council (2018): Engagement and Impact Assessment. 

Available here: https://dataportal.arc.gov.au/EI/NationalReport/2018/ 

•  COM(2019) 640 final: Communication from the Commission to the European 

Parliament, the European Council, the Council, the European Economic and Social 
Committee and the Committee of the Regions: The European Green Deal. 
•  COM(2020) 628 final: Commission Staff Working Document accompanying the 

document "Communication from the Commission to the European Parliament, the 
Council, the European Economic and Social Committee and the Committee of the 
Regions: A new ERA for Research and Innovation". 

•  COM(2023) 457 final: Commission Staff Working document "Communication from 
the Commission to the European Parliament, the Council, the European Economic 
and Social Committee and the Committee of the regions EU Missions two years 
on: assessment of progress and way forward" 

•  Council Conclusions on Research assessment and implementation of Open 

Science, 2022. Available here: 
https://www.consilium.europa.eu/media/56958/st10126-en22.pdf 

•  Council Decision (EU) 2021/764 of 10 May 2021 establishing the Specific 

Programme implementing Horizon Europe – the Framework Programme for 
Research and Innovation, and repealing Decision 2013/743/EU, available at: 
https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32021D0764 
•  Council Recommendation on a Pact for Research and Innovation in Europe 

(2021):  Available here: https://data.consilium.europa.eu/doc/document/ST-14136-
2021-INIT/en/pdf 

•  Special Eurobarometer #516: ‘European citizens’ knowledge and attitudes towards 

science and technology’, 2021. Survey conducted by Kantar at the request of the 
European Commission, Directorate-General for Research and Innovation. Survey 
co-ordinated by the European Commission, Directorate-General for 
Communication (DG COMM “Media monitoring and Eurobarometer Unit”). 
Available here: https://lbg.ac.at/wp-
content/uploads/2021/11/ebs_516_science_and_technology_report_EN.pdf 
•  Commission Recommendation (EU) 2018/790 of 25 April 2018 on access to and 

preservation of scientific information. Available here: https://eur-
lex.europa.eu/eli/reco/2018/790/oj 

•  Erasmus+, Priorities of the Erasmus+ Programme, 2023. Available here: 

https://erasmus-plus.ec.europa.eu/projects/priorities-2019-2024 

•  European Commission (2022), Valorising research through citizens’ engagement: 

how to run hackathons with citizens. Available here: 
https://op.europa.eu/en/publication-detail/-/publication/1585e969-ec57-11ec-a534-
01aa75ed71a1/language-en/format-PDF/source-283906489 

•  European Commission, Directorate-General for Research and Innovation, Warin, 

C., Delaney, N., Tornasi, Z., Citizen science and citizen engagement – 
Achievements in Horizon 2020 and recommendations on the way forward, 
Delaney, N.(editor), Tornasi, Z.(editor), Publications Office, 2020, 
https://data.europa.eu/doi/10.2777/05286 

•  European Commission, Directorate-General for Research and Innovation, 

Communication and citizen engagement initiatives in line with the Horizon Europe 
Mission A Soil Deal for Europe – Report on dissemination and exploitation 
practices in Member States and associated countries, Publications Office of the 
European Union, 2022, https://data.europa.eu/doi/10.2777/704413 

•  European Commission, Directorate-General for Energy, Brenner-Fließer, M., 

Rikos, E., Tuiskula, H. et al., Exploration of citizen engagement methodologies in 

725 

 
European R&I projects 3.0 – Consumer, Citizen & Engagement Working Group – 
June 2023, Brenner-Fließer, M.(editor), Rikos, E.(editor), Tuiskula, H.(editor), 
Höffken, J.(editor), Birch Riley, L.(editor), Kuivalainen, M.(editor), Ktenidis, 
P.(editor), Publications Office of the European Union, 2023, 
https://data.europa.eu/doi/10.2833/78191 

•  European Commission, Directorate-General for Research and Innovation, 

European Research Executive Agency, Publications Office of the European Union, 
CORDIS results pack on citizen science, Publications Office of the European 
Union, 2022, https://data.europa.eu/doi/10.2830/376577 

•  European Commission. (2021). ERA Policy Agenda. Retrieved from 

https://commission.europa.eu/system/files/2021-11/ec_rtd_era-policy-agenda-
2021.pdf 

•  European Governance: A White Paper COM(2001) 428: 

https://ec.europa.eu/commission/presscorner/detail/en/DOC_01_10. 

•  European Parliament and the Council (2021), Regulation (EU) 2021/695 of the 
European Parliament and of the Council of 28 April 2021 establishing Horizon 
Europe – the Framework Programme for Research and Innovation, laying down its 
rules for participation and dissemination, and repealing Regulations (EU) No 
1290/2013 and (EU) No 1291/2013, 28 April 2021, PE/12/2021/INIT, OJ L 170, 
12.5.2021, pp. 1–68, available at: http://data.europa.eu/eli/reg/2021/695/oj    
•  European Research Council, Citizen Science in ERC projects: Mapping ERC 
Frontier Research, 2022. Available here: https://erc.europa.eu/projects-
statistics/citizen-science-and-frontier-research 

•  European Commission, Directorate-General for Research and Innovation, 

Naujokaitytė, R., Mačiulienė, M., Stančiauskas, V. et al., Evaluation study on the 
implementation of cross-cutting issues in Horizon 2020 – Study in support of the 
ex-post evaluation of the European Framework Programme for Research and 
Innovation Horizon 2020, Dirham, S.(editor), Publications Office of the European 
Union, 2023, https://data.europa.eu/doi/10.2777/763665 

•  SWD (2023)132 final. (2023). Evidence Framework on monitoring and evaluation 

of Horizon Europe. Brussels, 27.4.2023. 

•  Griessler, E., Braun, R., Wicher, M., & Yorulmaz, M., The drama of responsible 
research and innovation: The ups and downs of a policy concept. In: Putting 
Responsible Research and Innovation into Practice: A Multi-Stakeholder 
Approach, pp. 11-34, Cham: Springer International Publishing, 2022. 

•  Horizon Europe (HORIZON). Programme Guide. Version 3.0. 

https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/guidance/programme-guide_horizon_en.pdf. 

•  European Commission, Directorate-General for Research and Innovation, 

Stančiauskas, V., Brozaitis, H., Notten, A. et al., Study to support the monitoring 
and evaluation of the framework programme for research and innovation along key 
impact pathways – Indicator methodology and metadata handbook, Stančiauskas, 
V.(editor), Brozaitis, H.(editor), Notten, A.(editor), Hollanders, H.(editor), 
Papageorgiou, H.(editor), Manola, N.(editor), Zagame, P.(editor), Le-Mouel, 
P.(editor), Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2777/44653 

•  European Commission, Directorate-General for Research and Innovation, Interim 
evaluation of Horizon 2020 – Commission staff working document, Publications 
Office, 2017, https://data.europa.eu/doi/10.2777/220768 

•  Meier, A., & Byland, R., A New Horizon for Society? Analysing the Integration of 
Responsible Research and Innovation in Horizon Europe, Brussels: SwissCore, 
2020. Available here: https://www.swisscore.org/wp-
content/uploads/RRI_in_Horizon_Europe.1.0.pdf 

726 

 
•  MSCA-NET Policy brief: Open Science, 2023. Available here: 

https://horizoneuropencpportal.eu/sites/default/files/2023-04/task-3.6-
open_science_brief.pdf 

•  OECD (2022), "Engaging citizens in cohesion policy: DG REGIO and OECD pilot 

project final report", OECD Working Papers on Public Governance, No. 50, OECD 
Publishing, Paris, https://doi.org/10.1787/486e5a88-en. 

•  Tertiary Education Commission, New Zealand: Performance-Based Research 

Fund. Available here: https://www.tec.govt.nz/funding/funding-and-
performance/funding/fund-finder/pbrf/ 

•  An introduction to the UNESCO Recommendation on Open Science. (2022). 

https://doi.org/10.54677/xoir1696 

•  Work Programme Cluster 3: Civil Security for Society (2023-2024) 
•  Work Programme Cluster 4: Digital, Industry and Space (2023-2024) 
•  Work Programme EIC (2023-2024) 
•  Work Programmes MSCA (2021-2022; 2023-2024) 
•  Work Programmes WIDERA (2021-2022; 2023-2024). 

727 

 
 
 
Annex 3: Benchmarks  

Annex 3.1: Benchmark on the Swiss National Science Foundation 

1.1. 

Introduction 

This benchmark study focuses on the Swiss National Science Foundation (SNSF), which 
supports fundamental scientific research across all academic disciplines. Given the SNSF’s 
focus  on  supporting  excellent  researchers  through  bottom-up  and  top-down  funding 
mechanisms  to  advance  research  in  Switzerland1148,  the  lessons  learned  from  this 
programme  hold  particular  value  for  the  Horizon  Europe  Framework  Programme.  This 
analysis does not aim to provide a comprehensive overview of the portfolio and the activities 
carried out by this funding agency; instead, its purpose is to highlight pertinent insights from 
the SNSF that can contribute to and inspire discussions on the relevance, efficiency and 
effectiveness of the EU´s Framework Programme.  

More specifically, this report sheds light on the balance between bottom-up and top-down 
approaches to research funding, and provides examples of successful initiatives supporting 
gender equality, as well as instruments promoting research in line with political or societal 
priorities.  The  context  of  Switzerland  and  its  discontinuous  and  partial  association  to  the 
Framework  Programme  allows  the  present  study  to  examine  the  consequences  of  non-
association  on  research  output  and  international  collaboration.  The  objectives  and 
implementation of the SNSF programme and its initiatives is expected to bring lessons and 
inspiration for the EU Framework Programme.  

This  benchmark  relies  on  extensive  desk  research  of  available  information,  data  and 
publications relating to the SNSF, as well as documentation and data linked to Horizon 2020 
and Horizon Europe. In addition, it draws upon qualitative insights gained through interviews 
with representatives of the SNSF and other stakeholders (e.g. the Swiss State Secretariat for 
Education, Research and Innovation (SERI)).  

The structure of this report is as follows: Section 1.2 introduces the methodological approach 
and evaluation questions addressed in this report. Section  1.3 briefly outlines the relevant 
Swiss policy context and Switzerland's history of association to the Framework Programme. 
It  also  introduces  the  principle  features  of  the  SNSF  and  its  different  funding  schemes. 
Section 1.4 presents noteworthy statistics regarding participation patterns under the SNSF. 
This  is  mainly  intended  to  inform  the  evaluation  questions  regarding  efficiency  and 
effectiveness,  and  provides  inspiration  on  further  engaging  women  in  research  and 
supporting the research careers of female researchers. Section 1.5 provides insights into the 
balance  within  the  programme  between  different  approaches  to  funding  research,  and 
specifically focuses on the perceived optimal balance in (collaborative) bottom-up versus top-
down funding approaches in the benchmark initiative. Section 1.6 focuses on Switzerland’s 
association to the Framework Programme, and provides insights into the implications of non-
association for Switzerland and for research in Europe.  Section  1.7 identifies key success 
factors and barriers and how these can be considered from the perspective of the Framework 
Programme. The conclusions of this analysis are presented in Section 1.8.  

1148 Throughout this benchmark report, the following conversion rate is used: 1 Swiss Franc 
(CHF) = EUR 1.03. 

728 

 
 
1.2.  Methodology and evaluation questions addressed 

This benchmark study relies extensively on multiple methods of research.  Desk research 
was carried out on the SNSF website1149, as well as documentation and reports. These were 
examined carefully, and have provided important information on the objectives, functioning, 
resources  and  results  of  SNSF  programmes.  Meanwhile,  data  provided  by  the  SNSF 
dashboards have provided rich insights into programme budgets, as well participation in and 
the outputs of the programmes. In addition, relevant information and news articles about the 
status and impacts of the  non-association of Switzerland to the Framework Programme 
have been reviewed. Lastly, documents, work programmes, reports, and monitoring data on 
Horizon 2020 and Horizon Europe have been reviewed to investigate the transferability of 
the results to the various part of the EU Framework Programme. 

The above desk research was complemented by interviews with representatives of SNSF 
and other stakeholders, with the objective of gathering more in-depth information concerning 
the  operations  of  the  SNSF,  the  expected  and  perceived  impacts  of  the  programmes,  the 
strengths and weaknesses of the different funding schemes, as well as the consequences of 
non-association to Horizon Europe. In total, eight interviewees were heard in the context of 
this benchmark. Five of the interviewees were from the SNSF and three were national-level 
stakeholders.  

The main findings of this benchmark study are arranged according to the evaluation criteria 
of relevance, efficiency, effectiveness, and EU added value. 

1.3.  Context and objectives of the benchmark initiative 

1.3.1.  The Swiss R&I policy context 

Switzerland  is  known  for  its  high  spending  on  R&D  relative  to  its  gross  domestic  product 
(GDP). Private sector funding for R&D is comparatively high – accounting for more than two-
thirds of Swiss R&D expenditure (over 3% of GDP1150) – but the public sector in Switzerland 
at  various  levels  also  invests  in  research  and  innovation.  In  Switzerland,  basic  research 
mainly  takes  place  at  universities  and  federal institutes of  technology.  Applied  research  is 
primarily the domain of the private sector and universities of applied sciences.  

In Switzerland, Art. 64 of the Swiss Federal Constitution obliges the Confederation to promote 
scientific research and innovation1151. Specifically, the Research and Innovation Promotion 
Act  (RIPA)1152  provides  that  the  Confederation  is  responsible  for  supporting  research  and 
innovation through the SNSF and Innosuisse, the Swiss Agency for Innovation Promotion. 
The  RIPA  Act  also  lays  the  basis  for  the  funding  of  the  Swiss  Academies  of  Arts  and 
Sciences1153 as well as support for around 30 research institutions of national importance.  

1149 https://www.snf.ch/en.  
1150 https://www.sbfi.admin.ch/sbfi/en/home/research-and-innovation/research-and-innovation-in-
switzerland.html#tab__content_sbfi_en_home_themen_forschung-und-innovation-in-der-
schweiz_jcr_content_par_tabs, accessed 3 April 2023. 
1151 https://www.fedlex.admin.ch/eli/cc/1999/404/en, accessed 3 April 2023. 
1152 https://www.fedlex.admin.ch/eli/cc/2013/786/en, accessed 3 April 2023. 
1153 https://akademien-schweiz.ch/en/uber-uns/, accessed 3 April 2023. 

729 

 
 
 
Innosuisse, the Swiss Innovation Agency, aims to promote science-based innovation in the 
interest of the economy and society in Switzerland.1154 The SNSF and Innosuisse coordinate 
their strategies and working together1155 to strengthen the interaction between science and 
innovation, with the aim of ensuring the best possible implementation of research results.1156 
Furthermore,  in  Switzerland,  the  cantons  are committed to  promoting  research  as  funding 
bodies of the universities.1157 

1.3.2.  Association of Switzerland to the Framework Programmes 

Researchers  and  organisations  from  Switzerland  have  participated  in  the  EU  Framework 
Programmes for Research and Innovation since 1987, under the first Framework Programme 
(FP1,  1984–1988)  but  the  extent  to  which  they  have  been  able  to  participate  has  evolved 
over time:1158 

•  Until  the  end  of  2003  (FP1–FP6),  researchers  based  in  Switzerland  only  had  a  few 
opportunities to take part in projects, as Switzerland was considered a  third country. 
While participation in such opportunities was initially funded by the project participants 
themselves, from 1992 onwards, funding was provided by the Swiss Confederation. 

•  From 2004 to 2013 (FP6 and FP7), Switzerland had the status of an associated country 
in the programme. This meant that participants from Switzerland had similar rights and 
obligations  to  participants  from  EU  Member  States;  thus,  Swiss  participants  received 
direct EU funding and had the opportunity to coordinate projects. 

•  Due  to  a  political  event  in  Switzerland  (the  ‘mass  immigration  initiative’),  in  2014, 
Switzerland’s association to Horizon 2020 was suspended. Later in 2014, agreements 
were  made  and  Switzerland  became  partially  associated  to  the  Framework 
Programme. This meant that Switzerland was treated as an associated country for some 
parts of the Framework Programme, and as a third country for others. This status lasted 
until January 2017, when Switzerland became fully associated to all parts of Horizon 
2020.  

Since  Horizon  Europe  began,  Switzerland  has  once  more  been  considered  a  non-
associated third country for the submission of project proposals to Horizon Europe. This 
means that researchers and innovators in Switzerland can participate only to a limited extent. 
It also means that Swiss participants are funded directly by the Swiss Confederation. More 
specifically1159: 

1154 https://www.innosuisse.ch/inno/en/home/about-us/mission.html, accessed 3 April -2023. 
1155 e.g. through joint programmes such as BRIDGE, 
https://www.innosuisse.ch/inno/en/home/promotion-of-national-projects/bridge.html, accessed 3 
April 2023) and NRP 78, https://www.snf.ch/en/ROoig00P3wqY8Sws/page/nrp/nrp78?, 
accessed 3 April 2023). 
1156 https://www.innosuisse.ch/inno/en/home/about-us/partnership-snsf.html, accessed 3 April 
2023. 
1157 https://www.sbfi.admin.ch/sbfi/en/home/research-and-innovation/research-and-innovation-in-
switzerland.html#tab__content_sbfi_en_home_themen_forschung-und-innovation-in-der-
schweiz_jcr_content_par_tabs, accessed 3 April 2023. 
1158 SERI, The Impact of Switzerland’s Participation in the EU Framework Programmes, 2019. 
1159 In addition to this, and beyond the programme, non-association to Horizon Europe meant 
that Swiss participation in the European Strategy Forum on Research Infrastructures (ESFRI) 
was discontinued in September 2022. 

730 

 
 
•  Swiss  organisations  can  participate  in  the  majority  of  collaborative  Horizon 
Europe projects. Funding to support such participation is provided by the Swiss State 
Secretariat  for  Education,  Research  and  Innovation  (SERI)  as  part  of  the  transitional 
measures (more details on this are provided in the pages that follow).1160  

•  However, Swiss researchers cannot participate in mono-beneficiary projects, which 
include  grants  from  the  European  Research  Council  (ERC),  Postdoctoral  Fellowships 
from  the  Marie  Skłodowska-Curie  Actions  (MSCA),  and  Accelerator  grants  from  the 
European Innovation Council (EIC). Furthermore, participation is very limited in strategic 
areas such as quantum and space research. Lastly, no access is granted to Swiss actors 
under the current status of the Digital Europe Programme. 

Figure  1  shows  the  eligibility  for  participation  and  funding  of  Swiss  researchers  and 
innovators  under  Horizon  Europe,  based  on  the  Horizon  budget  for  the  2023  work 
programmes. To compensate for the lack of funding available for participation in collaborative 
grants, and the impossibility of participation in the mono-beneficiary programme parts, the 
Swiss  State  Secretariat  for  Education,  Research  and  Innovation  (SERI)  has  introduced  a 
series of transitional measures (see Figure 2).1161,1162 These measures include funding for 
i)  participation  in  collaborative  projects,  and  ii)  support  via  funding  programmes  that  are 
aimed  to  mirror  the  ERC,  MSCA  and  EIC  calls  and  will  be  continued  until  an  association 
agreement is reached. The Swiss government has so far decided on transitional measures 
amounting to over EUR 1.9 billion covering the years 2021, 2022 and 2023.  

1160 https://www.sbfi.admin.ch/sbfi/en/home/research-and-innovation/international-cooperation-r-
and-i/eu-framework-programmes-for-research/horizon-europe/transitional-measures.html, 
accessed 3 April 2023. 
1161 https://www.sbfi.admin.ch/dam/sbfi/en/dokumente/2019/02/horizon-
europe.pdf.download.pdf/Fact-sheet_Horizon_en.pdf, accessed 3 April 2023. 
1162 https://www.sbfi.admin.ch/sbfi/en/home/research-and-innovation/international-cooperation-r-
and-i/eu-framework-programmes-for-research/horizon-europe/transitional-measures.html, 
accessed 3 April 2023. 

731 

 
 
 
Figure 1. Eligibility and funding of Swiss participants under Horizon Europe (2023 work programmes). Source: Swiss 
Confederation. Federal Department of Economic Affairs, Education and Research (EAER); State Secretariat for Education, 
Research and Innovation (SERI).1163 

Box 1. Transitional measures 

The  Swiss  government  has  taken  transitional  measures  to  ensure  researchers  and 
innovators  in  Switzerland  are  funded  when  they  participate  in  Horizon  Europe  projects, 
and that they have access to similar programmes at national level to the EU programme 
in which they are unable to participate (see Figure 2). 

Specifically, the Swiss government has assigned funds to SNSF to introduce national-level 
Starting, Advanced and Consolidator grants, mimicking the ERC grants. Since the SNSF 
already had similar grants in its portfolio (e.g. the SNSF Eccellenza1164), which targeting 
the same groups of researchers (e.g. mirroring the ERC Starting Grant), the SNSF board 
decided not to retain the two overlapping instruments but to integrate them, adopting the 
same names used by the ERC (e.g. the SNSF Starting Grant). In the second round of this 
Starting Grant, the SNSF adopted all of the evaluation criteria used by the ERC.  

According  to  interviewees,  the  integration  of  these  instruments  was  challenging, 
particularly in terms of communication channels to inform both Swiss and EU researchers 
on how the transitional system worked and how Swiss organisations could still be part of 
consortia. Nevertheless, the interviewees considered these transitional measures to have 
been successful. In 2022, the SNSF approved EUR 178.2 million for projects under the 
transitional measures. 

While the SNSF is the body mandated by the Swiss government to carry out parts of the 
transitional measures (e.g. SNSF Starting Grants, etc.), Figure 2 highlights that not all of 
the transitional measures are carried out by the SNSF, but also by Innosuisse among 
others. However, these measures are beyond the scope of this benchmark report.  

Figure 2. Switzerland’s transitional measures. Note: *Accordingly, there are no replacement measures. Source: Swiss 
transitional measures for the Horizon package (2021-2027), State Secretariat for Education, Research and Innovation 
(SERI).1165 

1163 https://www.sbfi.admin.ch/sbfi/en/home/research-and-innovation/international-cooperation-r-
and-i/eu-framework-programmes-for-research/horizon-europe.html/, accessed 3 April 2023. 
1164 https://www.snf.ch/en/DGCP4SuM4eU2O1bz/funding/careers/eccellenza. 
1165 https://www.sbfi.admin.ch/sbfi/en/home/research-and-innovation/international-cooperation-r-
and-i/eu-framework-programmes-for-research/horizon-europe/transitional-measures.html. 

732 

 
 
 
Despite  these  transitional  measures,  the  Federal  Council’s  objective  is  to  ensure 
Switzerland's full association to Horizon Europe as soon as possible.1166 

1.3.3.  The Swiss National Science Foundation 

The  Swiss  National  Science  Foundation  (SNSF)  was  founded  in  1952  as  a  private 
foundation. From that time onwards, its main task has been to provide financial support to 
research  projects  in  all  academic  disciplines,  and  to  promote  young  scientists  at  different 
stages in their careers.1167  

The  Research  and  Innovation  Promotion  Act  (RIPA)1168  outlines  the  tasks  the  SNSF  must 
carry out using the contributions granted by the Confederation (Art. 101169). These are: 

•  Research funding within the framework of the research instruments set up by the SNSF; 

•  Funding  programmes  and  networked  research  projects  at  national  and  international 

level; 

• 

Implementing national funding programmes that have been approved and commissioned 
by the Federal Council, particularly the National Research Programmes1170 and National 
Centres of Competence in Research1171; 

•  Supporting  Switzerland’s  participation  in  international  programmes  that  have  been 

approved and commissioned by the Federal Council; 

1166 https://www.sbfi.admin.ch/sbfi/en/home/research-and-innovation/international-cooperation-r-
and-i/eu-framework-programmes-for-research/horizon-europe.html/, accessed 3 April 2023. 
1167 https://www.snf.ch/en/SmhGfltIQkzzCxJc/page/aboutus/profile/history, accessed 7 April 
2023. 
1168 https://www.fedlex.admin.ch/eli/cc/2013/786/en, accessed 7 April 2023. 
1169 https://www.fedlex.admin.ch/eli/cc/2013/786/en, accessed 7 April 2023. 
1170 The SNSF participates in the procedures that precede decisions on the National Research 
Programmes, the National Centres of Competence in Research and further funding programmes 
that are assigned to it. The National Research Programmes (NRPs) are the federal 
government’s research funding tools to support research projects that develop knowledge useful 
in guiding decisions and practices – and thereby help to solve current social and economic 
challenges At the request of the Federal Department of Economic Affairs, Education and 
Research (EAER), the Federal Council periodically decides on the thematic focus of and budget 
allocations to new NRPs. Once the Federal Council has made its decision, the State Secretariat 
for Education, Research and Innovation (SERI) will instruct the SNSF to implement the approved 
NRPs. Currently, there are NRPs on COVID-19, antimicrobial resistance and digital 
transformation, among other topics. https://www.sbfi.admin.ch/sbfi/en/home/research-and-
innovation/research-and-innovation-in-switzerland/promotion-instruments/national-research-
programmes-nrp.html, accessed 7 April 2023.  
1171 The National Centres of Competence in Research (NCCRs) are an established federal 
funding instrument for top-level research in Switzerland. NCCRs are long-term research projects 
on themes of strategic importance to Switzerland as a location for research and innovation. The 
process of launching a new series of National Centres of Competence in Research begins with a 
public call for proposals by the Swiss National Science Foundation, SNSF. 
https://www.sbfi.admin.ch/sbfi/en/home/research-and-innovation/research-and-innovation-in-
switzerland/promotion-instruments/the-national-centres-of-competence-in-research-nccr.html, 
accessed 7 April 2023. 

733 

 
 
•  Supporting  measures  for  the  evaluation  and  exploitation  of  the  results  of  funded 

research. 

In this context, the SNSF aims to foster research in Switzerland as well as the international 
competitiveness and integration of Swiss research, and to support young researchers and 
contribute  to  addressing  ‘important  current  problems  through  research  programmes  and 
centres of competence’.1172 

The  SNSF  offers  a  range  of  research funding  schemes,  each  with  its  own  budget.  The 
SNSF is active in various research domains and disciplines: Human and Social Sciences, 
Mathematics, Natural and Engineering Sciences, and Biology and Medicine.1173 The SNSF 
distinguishes between five categories of funding: 

•  Project funding: under this umbrella, the SNSF awards grants to qualified researchers 
who are already paid by their home institutions, but need additional support to carry out 
a research project. 

•  Funding in the context of programmes: the SNSF awards grants for research projects 
with  pre-defined  thematic  or  conceptual/organisational  parameters  suggested  by 
researchers,  research  institutions,  or  political  actors.  Such  programmes  comprise,  in 
particular: 

–  The SNSF’s own programmes; 

–  The programmes and integrated research projects in which the SNSF decides 

to participate at national and international level; 

–  The national programmes opted for and commissioned by the Federal Council, 
in  particular  the  National  Research  Programmes  and  the  National  Centres  of 
Competence in Research; and 

–  The international programmes in which Switzerland participates, as chosen and 

commissioned by the Federal Council. 

•  Career funding. This funding targets researchers looking for a fellowship or salary to 
gather additional funding for their project in order to promote their research career. 

• 

Infrastructure  funding.  The  SNSF  awards  infrastructure  grants  as  initial  financing  to 
directly  support  specific  research  goals.  It  does  not  award  grants  for  infrastructures 
aimed at integrating different research areas. 

•  Funding  of  Science  Communication.  The  SNSF  awards  grants  to  promote  science 
communication. The aim of this is to foster communication between researchers, as well 
as communication between science and society. 

1172 SNSF Mission Statement, available at: 
https://www.snf.ch/media/en/MRzx9NGujOzMbXSv/snf_leitbild_e.pdf, accessed 7 April 2023. 
1173 SNSF research domains and disciplines, available at: 
https://www.snf.ch/media/en/Ii1WiTqVDFTDp5ts/allg_disziplinenliste.pdf, accessed 7 April 2023. 

734 

 
 
In line with the principle of fair competition, applications are evaluated using a competitive 
procedure, through comparisons with similar applications. 

The SNSF represents an interesting benchmark for the Framework Programme, as it allows 
insights to be gained into the balance between the ‘top-down’ and ‘bottom-up’ programmes 
implemented, how this has evolved over time, and the extent to which the current approach 
enables the evolving needs of society to be met. In addition, it provides useful insights into 
which thematic areas are prioritised, how these are defined, and why. Moreover, an analysis 
of Switzerland’s non-association to the Framework Programme – and the consequences this 
might  engender  for  R&I  in  both  Switzerland  and  in  the  EU  –not  only  informs  criterion  of 
effectiveness in this report (e.g. in terms of the implications of non-association on scientific 
excellence  and  collaboration  in  Europe),  but  provides  insights  into  the  EU  added  value 
(compared with the use of national-level funding instruments).  

1.4.  Efficiency and effectiveness 

In 2022, the SNSF approved EUR 937 million for new projects and fellowships abroad under 
its regular funding schemes. In addition, to support the implementation of Horizon Europe 
transitional  measures,  the  Swiss  government  assigned  a  further  EUR  178  million  to  the 
SNSF to support researchers in the shift towards the association of Switzerland to Horizon 
Europe.1174 

In  2022,  the  SNSF  evaluated  a  total  of  7,542  applications  and  approved  2,733  of  them, 
amounting to a success rate of 36%1175. Almost half of the SNSF budget was used to finance 
approximately  1,500  research  projects  in  Switzerland.  In  addition,  405  fellowships  were 
awarded  to  enable  early-career  researchers  to  conduct  a  research  stay  abroad,  and 
approximately  1,500  open  access  journal  articles  were  financed  under  the  Science 
Communication  funding  scheme.  Several  key  figures  relating  to  participation  patterns  are 
discussed below1176: 

•  Disciplines1177:  in  2022,  44%  of  the  grants  awarded  went  to  Humanities  and  Social 
Sciences;  28%  went  to  Mathematics,  Natural  Sciences  and  Engineering,  and  29%  to 
Biology and Medicine. However, looking more closely at the distribution of funding, grants 
to  Humanities  and  Social  Sciences  tend  to  be  smaller  than  those  for  other  fields  –  in 
other words, more grants were made to Humanities and Social Sciences than in other 
fields, but these grants were smaller.  

•  Age:1178  the  median  age  of  SNSF  applicants  and  project  participants  in  2022  was  48 

years. 

•  Gender:1179 as Appendix 2 illustrates, the SNSF actively targets women as beneficiaries, 
achieving  an  increase  in  women  beneficiaries  over  recent  years.  The  proportion  of 
female researchers funded has increased from 10% in 1991 to more than one-third in 
2021. Calculations by the SNSF show that the odds of receiving funding are about equal 

1174 These and subsequent figures and budget include the ‘regular’ SNSF funding schemes 
and transitional measures, but exclude the National Centres of Competence in Research 
(NCCRs), as on the figures are derived from the SNSF’s data portal and annual report. 
1175 https://www.snf.ch/en/g8XOw4OxWxBfnLiQ/page/annual-report. 
1176 These figures are based on the SNSF data portal, available at: www.data.snf.ch. 
1177 https://data.snf.ch/key-figures/disciplines.  
1178 https://data.snf.ch/key-figures/demographics. 
1179 https://data.snf.ch/stories/the-impact-of-confounding-factors-en.html. 

735 

 
 
for female and male applicants. The SNSF’s approach to supporting and fostering the 
career progression of women in research can serve as an inspiration for the Framework 
Programme  on  how  to  stimulate  gender  equality  and  improve  career  prospects  for 
women in particular.  

• 

Internationalisation1180:  

–  Between  2018  and  2022,  2,709  mobility  fellowships  were  awarded.  These 
were  awarded  to  young  researchers,  who  mainly  visited  Europe  (1,351, 
especially to the UK), closely followed by North America (1,218). 

–  Between  2018  and  2022,  there  were  a  total  of  24,910  project  collaborations. 
More than 15,500 collaborations involved European collaborators, while almost 
6,000 involved North American researchers, and around 1,660 involved Asian 
partners.  

1.5.  Relevance: balance between the approaches used to fund research  

An  important  point  of  discussion  relates  to  the  balance  between  top-down  and  bottom-up 
funding  approaches.  While  in  top-down  approaches,  research  projects  are  bound  to  pre-
defined  thematic  or  conceptual  parameters,  in  bottom-up  approaches,  project  topics  and 
objectives are open to all fields.  

Under the SNSF, a distinction is made between top-down funding schemes, which are mainly 
supported  under  the  ‘Programmes’  funding  scheme,  and  bottom-up  funding,  which  is 
mainly  provided  by  the  ‘Projects’  funding  scheme.1181  In  its  2021-2024  action  plan,  the 
SNSF  announced  that  it  would  continue  to  invest  the  vast  majority  of  its  budget  (i.e. 
approximately 80%) in bottom-up funding formats, thus enabling researchers to pursue 
and develop their own ideas and priorities.1182 

Interviewees  corroborated  these  figures,  indicating  that,  in  Switzerland,  there  is  more 
bottom-up than top-down funding for research.  The most widespread bottom-up funding 
opportunities under the SNSF focus mainly on collaborative projects, and are implemented 
under  the  largest  bottom-up  funding  scheme,  ‘Projects’.  In  terms  of  topics  funded, 
projects in Biology and Medicine received the highest amount of funding in 2022 (EUR 215 
million). Since 2017, applicants to ‘Projects’ have been allowed to include partners from 
national or foreign institutions in their applications. The contributions of such project 
partners are allowed to total up to 20% of the budget granted.1183 Since then, 37% of 
approved ‘Projects’ funding grants reported at least one collaboration with a researcher 
from a Swiss institution; 22% included at least one partner from a research institution in 
Europe (excluding Switzerland); and 14% partnered with at least one researcher from 
an institution outside Europe. On average, the projects that include project partners have 
an average of 3.4 partners.  

1180 https://data.snf.ch/key-figures/internationality. 
1181 While it was traditionally the case that all ‘Programmes’ funding schemes were top-down and 
‘Projects’ were bottom-up, this has changed slightly over time. For instance, the National 
Centres of Competence in Research (NCCRs) are positioned under ‘Programmes’, but this has 
turned into a bottom-up programme. 
1182 https://www.snf.ch/en/CBGkfq5CP6BAkNu3/page/theSNSF/profile/strategy/action-plan. 
1183 https://data.snf.ch/stories/project-partners-and-networked-research-en.html. 

736 

 
 
•  Projects  in  Social  Sciences  and  Humanities  (SSH)  are  more  often  collaborative  than 
projects  in  Mathematics,  Natural  and  Engineering  Sciences  (MINT),  and  slightly  more 
than in Life Sciences (LS). In other words, the percentage of grants funded with project 
partners in SSH is slightly higher than in MINT and LS; 

•  Among  collaborative  projects  (i.e.  those  with  at  least  one  partner),  projects  in  Life 

Sciences on average have the highest number of partners. 

As indicated in Appendix 2, the National Centres of Competence in Research (NCCRs) 
programme  also  represents  a  bottom-up  collaborative  funding  initiative  that  connects 
scientific communities in Switzerland and beyond. The research groups that collaborate in 
an NCCR can either profit directly, or can participate without receiving direct funding.  

The  SNSF  also  funds  research  on  pre-defined  themes  aimed  at  generating  scientific 
knowledge that can help to solve contemporary problems of national importance, as well as 
strengthening Swiss research in areas of strategic importance to the future of Swiss science, 
business  and  society.  One  of  the  key  types  of  funding  instrument  under  the  SNSF 
‘Programmes’ funding is the National Research Programmes (NRPs). These are run in close 
cooperation with the Swiss government, and have had an impact on Swiss institutions and 
the national regulatory framework (see Appendix 2).  

1.6.  EU  added  value:  the  non-association  of  Switzerland  to  Horizon 

Europe  

As outlined in Section 1.3.2, as of 1 January 2021, Switzerland is no longer associated to the 
EU Framework Programme for R&I, and is classified as a third country. This means that while 
Swiss  researchers  and  innovators  can  participate  in  some  programme  parts  of  Horizon 
Europe (see Figure 1), they are only funded directly by the Swiss Confederation.  

The EU considers Switzerland’s association to EU programmes in the context of its broader 
bilateral  relationship.  To  stabilise  and  further  develop  bilateral  relations,  the  EU  and 
Switzerland have been conducting exploratory talks on a broad negotiation package since 
February  2022.  Among  other  things,  this  includes  institutional  issues  and  Switzerland’s 
participation in EU programmes.1184 After several rounds of talks at various levels (political, 
diplomatic  and  technical)  between  the  Swiss  and  EU  delegations  on  the  approach  of  the 
package,  in  June  2023  the  Federal  Council  approved  the  parameters  for  a  negotiating 
mandate with the EU. 1185 Discussions continue, with a view to resolving outstanding issues 
and  defining  common  ‘landing  zones’.  If  the  exploratory  talks  continue  to  progress 
satisfactorily, the EU and Switzerland will prepare for the adoption of a negotiating mandate. 
Switzerland expects this step to have a positive impact on Switzerland's swift association to 
Horizon Europe and Erasmus+.” 

The  previous  association  to  Horizon  2020  was  extremely  beneficial  to  Switzerland, 
according  to  the  SERI  impact  report  carried  out  in  2019.1186  This  report  indicated  that  the 
Framework  Programme  was  an  integral  part  of  the  Swiss  R&I  landscape,  and  that 

1184 https://www.admin.ch/gov/en/start/documentation/media-releases.msg-id-87349.html, 
accessed 30 August 2023. 
1185 https://www.admin.ch/gov/en/start/documentation/media-releases.msg-id-95910.html, 
accessed 30 August 2023. 
1186 SERI, The Impact of Switzerland’s Participation in the EU Framework Programmes, 2019, 
available at: https://www.sbfi.admin.ch/sbfi/en/home/services/publications/data-base-
publications/impact-2019.html. 

737 

 
 
Switzerland’s participation in the EU Research and Innovation programmes provided a big 
boost to the country’s economic competitiveness, fostered knowledge generation, stimulated 
exchange,  and  positively  impacted  the  whole  of  Swiss  society.  Under  the  previous 
Framework Programme (Horizon 2020), Switzerland received 3.6% of the EU net contribution 
to the whole programme (namely, EUR 2.4 billion out of the EUR 68.3 billion total).1187 Only 
seven countries received a higher share. Excellent Science was by far the most important 
pillar for Switzerland under Horizon 2020, with the country receiving an EU net contribution 
of  almost  EUR  1.6  billion.  Out  of  this,  EUR  266.91  million  was  granted  to  Swiss 
organisations under the MSCA, and EUR 983.4 million was granted under ERC funding. 
This contrasts with the situation that as a third country under Horizon Europe, researchers in 
Switzerland can no longer participate in MSCA fellowships nor obtain grants from the ERC. 
Together, under Horizon 2020 these actions represented more than half of the total European 
funding granted to Switzerland. One interviewee indicated that during a previous period of 
non-association  by  Switzerland  to  the  Framework  Programme,  exclusion  from  the  mono-
beneficiary  grants  (ERC,  EIC  and  MSCA)  in  particular  was  the  biggest  problem  faced  by 
Swiss researchers. 

The desk research and interviews carried out for this benchmark study point to some negative 
consequences stemming from Switzerland’s present non-association that are noticeable right 
now,  or are  expected  in  the  (near)  future.  It  is  important  to  take  these  consequences  into 
account when examining the added value of the Framework Programme: 

Fewer  international  collaborations,  but  not  a  steep  decline:  it  has  been  argued  that 
Switzerland’s  exclusion  from  Horizon  Europe  is  affecting  the  country’s  international 
collaborations.  The  Horizon  Dashboard  does  indeed  confirm  that  Swiss  organisations  are 
now less involved in EU projects, but the decline does not appear as highly pronounced as 
might be expected. Data from the Horizon Dashboard1188 indicate that under Horizon 2020, 
Switzerland  contributed  5,882  or  3.1%  of  all  participations.  By  comparison,  Swiss 
participations represent 2.26% (999) of all Horizon Europe participations so far. Eliminating 
Pillar 1 (Excellent Science), this disparity  in shares diminishes (1.9% under Horizon 2020, 
and only slightly lower at 1.7% under Horizon Europe so far).  

Several factors identified during the desk research and interviews can help to explain why 
international collaborations by Swiss organisations under the Framework Programme have 
so  far  not  dropped  significantly.  First,  Swiss  organisations  often  have  a  strong  record  of 
participation under the programme, possess relevant equipment and expertise, and are often 
seen as reliable partners in the context of actions funded by the programme. Second, Swiss 
researchers contribute to international consortia using their own budgets (i.e. supported by 
Switzerland), which means that consortia that include Swiss partners may ultimately benefit 
from  higher  levels  of  funding.  The  fact  that  Swiss  parties  can  no  longer  coordinate 
collaborative Horizon projects is, however, regarded as an important disadvantage. Not being 
able  to  coordinate  projects  hinders  an  organisation’s  capacity  to  explore  new  research 
pathways and agendas and to strengthen its reputation.  

Less research output: non-association to Horizon Europe may also have had an impact on 
the  Switzerland's  science  output.  A  recent  study  indicated  that  in  certain  fields,  such  as 
STEM, more than 15% of all Swiss publications in a given year were funded by Horizon 2020. 
Even  though  domestic  funding  alternatives  have  been  set  up,  non-association  to  Horizon 

1187 Source: Horizon Dashboard, accessed 24 May 2023, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/horizon-
dashboard. 
1188 Horizon Dashboard, available at: https://ec.europa.eu/info/funding-
tenders/opportunities/portal/screen/opportunities/horizon-dashboard, accessed 24 May 2023. 

738 

 
 
Europe may pose challenges in these fields.1189 In addition, one interviewee indicated that 
because projects with Swiss coordinators enjoyed fairly high success rates under Horizon 
2020, they augmented the excellence of and impacted their project partners and, as such, 
contributed to the excellence of the European R&I as a whole.  

Less  prestige:  although  Switzerland  has  taken  a  number  of  transitional  measures  (see 
Box 1)  and  provides  funding  opportunities  that  mirror  Horizon  Europe’s  ERC  grants  and 
MSCA  fellowships,  the  interviewees  fear  that  these  may  be  regarded  by  the  research 
community  as  less  prestigious  and  less  competitive  (although  there  is  actually  stiff 
competition for these domestic instruments). This could lead to Swiss researchers being less 
valued than peers from elsewhere in Europe who are funded by the Framework Programme, 
thus potentially hampering the career prospects of Swiss researchers.  

More difficulties in attract and retaining top international scientists: interviewees and 
other stakeholders1190 indicated that, without association to Horizon Europe, Switzerland may 
be regarded as a less attractive destination for top international researchers. In other words, 
researchers  may  not  consider  coming  to  Switzerland,  and  those  who  are  already  in 
Switzerland may even leave the country. This is because access to European programmes 
has a positive impact on attracting the best talent (e.g. many top scientists are interested in 
obtaining ERC grants), and Switzerland was particularly interesting to them, given its strong 
national  and  European  funding.1191  In  addition,  non-association  may  hinder  the  futures  of 
early-stage researchers, since it restricts their access to the European research network and 
its related instruments (e.g. MSCA, Erasmus+). In addition, one interviewee noted that non-
association might entail negative consequences for the R&I landscape across Europe as a 
whole. This is because under Horizon 2020, Swiss institutions were highly involved in the 
Excellent Science pillar, and hence contributed significantly to the European talent pool by 
attracting excellent scientists and innovators from around the world.  

1.7.  Critical  success  factors  and  barriers,  and  the  transferability  of 

findings to the EU Framework Programme 

This section of the report presents the most important success factors and barriers identified 
in  its  analysis.  Also  presented  are  lessons  that  can  be  of  interest  to  the  EU’s  Framework 
Programme, and which relate to the evaluation questions on efficiency and effectiveness (for 
example  with  regard  to  stimulating  gender equality  and  improving  the  career  prospects  of 
women),  as  well  as  the  relevance  of  topics,  and  insights  into  EU  added  value  given  the 
present  context  of  uncertainty  surrounding  Switzerland’s  association  to  the  Framework 
Programme. 

With  regard  to  the  balance  between  bottom-up  and  top-down  funding  approaches, 
interviewees expressed the view that the existing balance between these approaches in the 
SNSF  portfolio  receives  positive  feedback  from  researchers.  They  emphasised  that 
researchers  enjoy  ample  options  under  Swiss  funding  schemes,  and  generally  value  the 
allocation  of  a  major  portion  of  funding  to  bottom-up  collaborative  calls.  Interviewees 

1189 ‘Switzerland’s non-association to Horizon Europe could hit STEM subjects hard – with 
consequences for innovation’, available at: https://sciencebusiness.net/news-byte/switzerlands-
non-association-horizon-europe-could-hit-stem-subjects-hard-consequences.  
1190 https://www.swissinfo.ch/eng/sci-tech/why-is-horizon-europe-important-for-switzerland-and-
the-european-union-/46792306; https://www.srf.ch/news/schweiz/blockade-im-eu-dossier-uni-
rektoren-alarmiert-die-situation-ist-untragbar?ns_source=mobile&srg_sm_medium=tw. 
1191 https://www.swissinfo.ch/eng/sci-tech/why-is-horizon-europe-important-for-switzerland-and-
the-european-union-/46792306. 

739 

 
 
underline  that  this  focus  on  bottom-up  collaborative  grants  contributes  to  both  the  Swiss 
economy and society, even though topics are not pre-determined beforehand.  

Among the top-down approaches to research funding, the NRP programme stands out thanks 
to  its  distinctive  approach  to  selecting  the  topics.  While  the  Swiss  government  ultimately 
determines  the  topics  of  the  calls,  these  themes  are  suggested  by,  among  others,  Swiss 
researchers, universities, and other contributors, with the SNSF subsequently conducting an 
extensive feasibility study to assess the merits and rationale behind the topics proposed. 

Under the European Union´s Framework Programmes, the balance of funding approaches 
has slightly evolved over time, with Horizon 2020 placing a greater emphasis on top-down 
approaches  compared  with  previous  framework  programmes.  Phase  1  of  the  present 
evaluation  support  study  focused  on  the  ex-post  evaluation  of  Horizon  20201192  and 
highlighted that, although the Framework Programme included both bottom-up and top-down 
approaches (e.g. focusing on specific societal challenges), there were concerns regarding 
the availability of sufficient funds for collaborative, basic bottom-up research. Interviews 
indicated that, even though collaborative, basic bottom-up research was being carried out by 
some partnerships and some programme parts, there was room to increase support for this 
type of research (and its visibility within the Framework Programme).  

The area of gender equality give rise to some findings of particular interest: the SNSF has 
implemented  effective  measures  to  promote  women’s  involvement  in  academia  and  their 
progression to professorships. Since the 1990s, the SNSF has been developing and refining 
instruments  aimed  at  providing  funding  for  research  projects  led  by  excellent  female 
researchers  (e.g.  PRIMA).  This  includes,  in  recent  years,  specific  budgetary  lines  to  fund 
high-quality  proposals  from  female  scientists  that  exceeded  the  threshold  but  could  not 
otherwise be funded due to budget constraints (under SNSF Starting Grants).  

The  study  team  cannot  identify  analogous  instruments  under  the  EU  Framework 
Programmes that encourage female participation in individual grants. There are, however, 
alternative  measures,  such  as  ERC  Scientific  Council's  Gender  Equality  Plan  for  2021-
20271193  which  includes  extensions  to  eligibility  for  women  with  children,  modified  CV 
templates for applicants, and the introduction of anti-bias briefings for members of evaluation 
panels, as well as amplified communication efforts to encourage female applicants. 

Switzerland’s  National  Centres  of  Competence  in  Research  (NCCRs)  constitute  an 
interesting  example  in  terms  of  instruments  fostering  the  development  of  research 
agendas in line with political priorities. Their large size in terms of the number of research 
groups involved, their flexible and autonomous organisation and budgeting, and the fact that 
they run for very long periods (up to 12 years) are some of the factors that have led to the 
success  of  NCCRs.  This  contrasts  with  the  shorter  and  usually  more  rigid  organisation  of 
similar projects under the EU’s Framework Programmes. 

With regard to the consequences of non-association, even though the transitional measures 
introduced by Switzerland to mimic the FP funding instruments are found to be successful, 

1192 European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European framework programmes 
for research and innovation for excellent science: Horizon 2020: phase 1 final study report, 
Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/967813. 
1193 
https://erc.europa.eu/sites/default/files/document/file/ERC_ScC_Gender_Equality_Plan_2021-
2027.pdf. Prior to this plan, there were plans covering the periods 2007-2013 and 2014-2020. 

740 

 
 
some interviewees noted that Switzerland may not be able to maintain the same intensity 
and quality of knowledge production, international collaborations, and contribution to shaping 
the European research agenda. This is considered by interviewees to be a negative situation 
for both Swiss and European research. 

1.8.  Conclusions 

This  benchmark  report  presents  various  aspects  of  the  SNSF  portfolio,  viewed  from  the 
perspective  of  the  transferability  of  its  results  to  the  Framework  Programme  in  relation  to 
efficiency,  effectiveness,  relevance  and  EU  added  value.  The  present  analysis  does  not 
provide a comprehensive picture of the portfolio and the activities carried out by this funding 
agency.  It  does,  however,  shed  light  on  a  number  of  interesting  insights  and  approaches 
linked to several aspects that are relevant to the design of the EU´s Framework Programme 
These  include  the  balance  in  research  funding  between  ‘bottom-up’  and  ‘top-down’ 
approaches; examples of successful Swiss initiatives to support gender equality; instruments 
used to promote research that aligns with political or societal priorities; and the consequences 
of Switzerland’s non-association to the Framework Programme. 

Appendix 1. Literature sources 

•  European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European framework 
programmes for research and innovation for excellent science: Horizon 2020: phase 1 
final study report, Luxembourg: Publications Office of the European Union, 2023, 
available at: https://data.europa.eu/doi/10.2777/967813. 

•  European Commission, She Figures 2021, in: https://www.rathenau.nl/en/science-
figures/personnel/women-science/share-female-professors-netherlands-and-eu-
countries, accessed 13 April 2023. 

•  State Secretariat for Education, Research and Innovation, , Status Update: Swiss 

participation in Horizon Europe and related programmes and initiatives, 2023, available 
at: https://www.sbfi.admin.ch/dam/sbfi/en/dokumente/2019/02/horizon-
europe.pdf.download.pdf/Fact-sheet_Horizon_en.pdf, accessed 3 April 2023. 

•  Federal Council of Switzerland, Federal Constitution of the Swiss Confederation, 

available at: https://www.fedlex.admin.ch/eli/cc/1999/404/en 

•  Federal Council of Switzerland, Talks conducted by the President of the Swiss 

Confederation in Davos focused on security situation in Europe and the work of the UN 
Security Council, 2023, available at: 
https://www.admin.ch/gov/en/start/documentation/media-releases.msg-id-92581.html. 

•  Federal Council of Switzerland, Federal Act of 14 December 2012 on the Promotion of 

Research and Innovation (RIPA), available at: 
https://www.fedlex.admin.ch/eli/cc/2013/786/en  

•  Federal Council of Switzerland, Talks conducted by the President of the Swiss 

Confederation in Davos focused on security situation in Europe and the work of the UN 
Security Council, available at: https://www.admin.ch/gov/en/start/documentation/media-
releases.msg-id-92581.html 

741 

 
•  Feresin, E., ‘What’s on the horizon for Swiss research in Europe?’, Swissinfo.ch, 2022, 
available at: https://www.swissinfo.ch/eng/sci-tech/what-s-on-the-horizon-for-swiss-
research-in-europe-/47490296 

•  Flücker, Y. ‘Why is Horizon Europe important for Switzerland and the European 

Union?’, Swissinfo.ch, 2021, available at: https://www.swissinfo.ch/eng/sci-tech/why-is-
horizon-europe-important-for-switzerland-and-the-european-union-/46792306 

•  Rutzer, C., ‘Swiss STEM research at risk due to lack of Horizon Europe funding?’, 
LinkedIn post, 2023, available at: https://lk.linkedin.com/posts/christian-rutzer-
38978392_horizoneurope-innoscape-switzerland-activity-7038462335228768256-
bD9D 

•  Matthews, D., EU and Switzerland talk about Horizon Europe at Davos, Science 

Business, 2023, available at: https://sciencebusiness.net/news/Horizon-Europe/eu-and-
switzerland-talk-about-horizon-europe-
davos#:~:text=Because%20Switzerland%20is%20not%20associated,they%20want%2
0to%20join%20consortia. 

•  State Secretariat for Education, Research and Innovation, Facts and Figures on the 
Swiss Participation in the EU-Framework Programmes for Research and Innovation, 
available at: https://www.sbfi.admin.ch/sbfi/en/home/research-and-
innovation/international-cooperation-r-and-i/eu-framework-programmes-for-research/f-f-
swiss-participation.html 

•  Science Business, Switzerland’s non-association to Horizon Europe could hit STEM 
subjects hard - with consequences for innovation, Science Business, 2023, available 
at: https://sciencebusiness.net/news-byte/switzerlands-non-association-horizon-
europe-could-hit-stem-subjects-hard-consequences 

•  SNSF, Funding Regulations, available at: 

https://www.snf.ch/en/97JmAecSwqB12X8m/page/funding/documents-
downloads/regulations-funding-regulations  

•  SNSF, General implementation regulations for the Funding Regulations, available at: 

https://www.snf.ch/en/Zi65otScMDqzyALN/page/funding/documents-
downloads/regulations-general-implementation-regulations  

•  Federal Department of Economic Affairs, Education and Research, Swiss Science 
Council, Evaluation of the Swiss National Science Foundation, 2022, available at: 
https://wissenschaftsrat.ch/images/stories/pdf/en/SWR_2022_SNF_evaluation_annexe
_I-IV.pdf; 
https://wissenschaftsrat.ch/images/stories/pdf/en/SWR_2022_SNF_evaluation_annexe
_IX-XIII.pdf  

•  State Secretariat for Education, Research and Innovation, The Impact of Switzerland’s 

Participation in the EU Framework Programmes, 2019, available at: 
https://www.sbfi.admin.ch/dam/sbfi/en/dokumente/webshop/2019/impact-studie-
2019.pdf.download.pdf/Impact_Studie_en.pdf  

•  Washington, O. Uni-Rektoren alarmiert: «Die Situation ist untragbar», Schweizer Radio 
und Fernsehen,, 2023, available at: https://www.srf.ch/news/schweiz/blockade-im-eu-

742 

 
dossier-uni-rektoren-alarmiert-die-situation-ist-
untragbar?ns_source=mobile&srg_sm_medium=tw  

•  SNSF website, available at: www.snsf.ch/en  

•  Swiss government, SERI and Innosuisse websites and relevant pages.  

Appendix 2. SNSF support programmes 

SNSF support for women in science 

Over recent decades, the SNSF has devoted attention to engaging women in research 
and supporting  their careers in research. In the 1990s, a programme was set up to 
support women who encountered a delay in their careers during their PhD or postdoctoral 
research (e.g. due to taking maternity leave). This programme ran until 2016, after which 
the  SNSF  decided  to  support,  through  its  PRIMA  grants1194,  excellent  women 
researchers who display high potential for obtaining a professorship. This support can be 
seen as important, given that women account for 23% of professors in Switzerland,1195 
compared with 26.2% across the EU-28.1196 

PRIMA  grants  are  awarded  to  outstanding  female  researchers  in  all  disciplines  at 
postdoctoral  stage  or  with  equivalent  research  experience,  and  who  are  pursuing  an 
academic career with a view to a professorship in Switzerland. The grants include project 
funds  and  the  beneficiary’s  salary,  and  run  for  a  maximum  of  five  years.  If  a  PRIMA 
grantee  obtains  a  professorship  at  a  higher  education  institution  (HEI)  in  Switzerland 
before the grant expires, she can still use the remaining PRIMA funds. This enhances the 
appeal of PRIMA grantees to HEIs. 

In 2023, PRIMA funding was  integrated into the SNSF Starting Grants.  The PRIMA 
budget  continues  to  be  reserved  exclusively  for women,  but  is  administered under  the 
SNSF Starting Grants. This means that the highest-scoring applications will be funded, 
regardless of gender, but that an additional amount of projects by female applicants 
will also receive funding through the PRIMA budget. This has several advantages: 1) 
the budget per project is larger than under the original separate PRIMA grants; and 2) 
women  do  not  have  to  choose  which  scheme  they  wish  to  apply  for.  In  addition,  the 
eligibility criteria have been relaxed over time. For instance, the obligation for applicants 
to have international mobility experience has been removed, as such travel was perceived 
as being more difficult for women with a family.  

Interviews carried out for this benchmark study indicate that this ‘positive discrimination’ 
is well received by researchers and organisations, and that PRIMA has always been 
a highly appreciated instrument that has never encountered strong opposition.  

Source: compiled by the authors, based on the SNSF website and interviews.1197 

1194 https://www.snf.ch/media/en/TMyhiwVNNo08vdv3/snf_prima_reglement_e.pdf. 
1195 https://www.snf.ch/en/U5Idhmrcn15Xuqty/news/prima-and-eccellenza-become-snsf-
professorial-fellowships. 
1196 Source: European Commission, She Figures 2021, in: https://www.rathenau.nl/en/science-
figures/personnel/women-science/share-female-professors-netherlands-and-eu-countries, 
accessed 13 April 2023. 
1197 https://www.snf.ch/en/soMWfibVDpNMVR36/funding/careers/prima. 

743 

 
 
National Centres of Competence in Research (NCCRs)  

Established  in  1999,  the  NCCR  programme1198  aims  to  strengthen  Swiss  research  in 
strategically important fields and seeks to promote innovative, top-tier research initiatives 
with  a  long-term  perspective.  NCCRs  are research  networks  or ‘centres  of  excellence’ 
that  may  run  for  up  to  12  years,  divided  into  four-year-long  phases,  and  bringing 
together  between  20  and  40  research  groups.  NCCRs  mainly  focus  on  conducting 
fundamental research, but also address the transfer of knowledge and technologies, as 
well as issues such as equal opportunities, education and training, and outreach. 

The budget for each series of NCCRs is determined by parliament. In addition to federal 
funds, NCCRs receive funding from HEIs and third parties, and have a high degree of 
financial and organisational autonomy. 

So far, five calls1199 have been launched for NCCRs, and 42 NCCRs have been funded. 
Among those currently underway are NCCR AntiResist; NCCR Automation, and NCCR 
Bio-Inspired Materials.1200 A sixth call was scheduled towards the end of 2023.  

Interviewees indicated that the NCCRs are regarded as extremely successful, and have 
changed  the  research  landscape  in  Switzerland.  An  earlier  interim  assessment  of  the 
SNSF showed that between 2001 and 2014, NCCRs produced approximately 30,000 
scientific  publications.  The  same  report  indicates  that  NCCRs  have  created  138 
assistant  professorships  and  that  5,200  young  researchers  have  completed  their 
doctorates in NCCR doctoral programmes.1201 A more recent report confirms that, since 
the  launch  of  the  initiative,  each  of  the  centres  supported  has  produced  hundreds  of 
scientific  publications,  engaged  in  hundreds  of  transfer  activities,  and  have  fostered 
collaboration with dozens of (international) research institutions and parties in the public 
or  private  sector.1202  The  long-term  impact  of  the  NCCRs  is  ensured  through  the 
commitment  of  the  HEIs  with  which  they  are  associated,  and  which  also  contribute 
financially to setting up and maintaining the NCCRs. 

The example of the NCCR SwissMap – The Mathematics of Physics illustrates the 
success of the initiative. Started in 2014, the home institutions of this NCCR are the 
University of Geneva and ETH Zurich. By 2022, the NCCR employed 223 people. In terms 
of outputs, it has produced 1,067 scientific publications, organised 2,055 academic events 
(e.g.  individual  talks,  conference  talks  or invited  keynote  lectures),  and  engaged  in  43 
public communications. 

1198 The applicants (established researchers) present a proposal, accompanied by a letter of 
support from the home institution, indicating that this initiative is of strategic importance to the 
institution, and outlining how the sustainability of the centre will be ensured after the project 
ends. A peer review process follows, after which the shortlisted proposals are evaluated by 
SERI, whose recommendations are passed to the Federal Department of Economic Affairs, 
Education and Research (EAER), which makes the final selection. This approach is different has 
changed since the beginning of the programme, when programmatic content was more 
predefined in certain disciplines. 
1199 The years of the calls were: 1999, 2003, 2008, 2011 and 2017.  
1200 Other currently running NCCR are: NCCR Catalysis; NCCR Digital Fabrication; NCCR 
Evolving Language; NCCR MARVEL – Materials Revolution: Computational Design and 
Discovery of Novel Materials; NCCR Microbiomes; NCCR Molecular Systems Engineering; 
NCCR On the Move – The Migration-Mobility Nexus; NCCR PlanetS – the Origin, Evolution and 
Characterisation of Planets; NCCR RNA & Disease; NCCR SPIN – Spin Qubits in Silicon; and 
NCCR SwissMAP  – The Mathematics of Physics. 
1201 https://www.snf.ch/SiteCollectionDocuments/nccr_brochure_e.pdf. 
1202 https://www.snf.ch/media/en/dM2ruuUsDX7esqDv/nccr-guide_2022.pdf. 

744 

 
 
National Research Programmes (NRPs) and their pre-defined topics 

The National Research Programmes (NRPs) in Switzerland are the federal government’s 
research funding tools for supporting research projects that develop knowledge that can 
be  useful  in  guiding  political  decisions  and  practice  –thereby  helping  to  solve  current 
social  and  economic  challenges.  The  selection  of  the  topics  addressed  in  the  NRPs 
follows a specific procedure: 

-  Every two to three years, federal offices, research institutes, research groups or 
individuals can propose topics and potential priorities for an NRP to the State 
Secretariat  for  Education,  Research  and  Innovation  (SERI).  SERI’s  website 
provides information on when and how NRP topics can be submitted. SERI drafts 
programme proposals on the basis of these submissions, and commissions the 
SNSF to conduct feasibility studies. Then, for each of the proposed programmes 
that receives a positive assessment in the feasibility study, the SNSF prepares 
a call for proposals.  

-  At  the  request  of  the  Federal  Department  of  Economic  Affairs,  Education  and 
Research  (EAER),  the  Federal  Council  judges  the  merits  of  these  calls  for 
proposals,  defines  budgets,  and  commissions  the  SNSF  to  implement  the 
resulting NRPs. There have been rare exceptions to this approach, such as when 
the Federal Council unilaterally decided to launch an NRP (NRP 78) on COVID-
19 following the outbreak of the pandemic. 

Since  the  1970s,  84  NRPs  have  been  launched  on  a  variety  of  topics.  The  first  NRP 
focused on cardiovascular diseases. The most recent ones have focused on Baukultur 
(the production and culture of the built environment), biodiversity, gender medicine and 
plant breeding. 

According  to  interviewees  contacted  for  this  study,  the  NRPs  not  only  have  a  positive 
impact on knowledge production, but also contribute to fostering knowledge transfer 
towards  industry  and  professionals  in  the  field.  This  is  reflected,  for example,  by  the 
involvement  of  professional  associations  in  the  research  and  publications  produced  in 
specialised practical journals.  

Moreover, the NRPs also contribute to political decision-making: specifically, there are 
close  and  frequent  interactions,  from  the  early  stages  of  the  process,  between 
researchers  and  practitioners  (who  are  also  involved  in  the  sounding  boards  for  the 
projects, for example) and political committees. This was mentioned in interviews as one 
of the most important drivers of the initiative's success: that the NRPs motivate the people 
involved to leave their ‘silos’ and engage with practitioners and policymakers. A number 
of challenges to this programme were also highlighted during the interviews, among which 
was the absence of a systemic monitoring framework to assess the impacts of this funding 
instrument, as well as the length of the submission and selection processes. 

745 

 
 
 
Annex 3.2: Benchmark on the United Kingdom Research & 
Innovation (UKRI)  

1.1. 

Introduction 

This report presents an analysis of United Kingdom Research & Innovation (UKRI). UKRI 
is a non-departmental public body sponsored by the Department for Science, Innovation and 
Technology (DSIT) in the United Kingdom (UK). UKRI brings together the seven disciplinary 
research  councils,  as  well  as  Research  England  (responsible  for supporting  research  and 
knowledge exchange at higher education institutions in England), and Innovate UK, the UK’s 
innovation agency. This combination enables UKRI to work across the whole research and 
innovation system. 

This  exercise  aims  to  examine  several  aspects  of  particular relevance  to  Horizon  Europe, 
and to Excellent Science in particular. The present benchmark study looks more closely into 
the organisational model of UKRI, and especially the extent to which it encourages bottom-
up initiatives as opposed to a top-down approach, individual and collaborative projects, and 
projects that span different fields. The benchmark study also includes an in-depth look into 
the  strategic  plan  for  UKRI,  and  how  overarching  policy  priorities  are  designed  and 
implemented  in  order  to  encourage  projects  to  be  more  ‘directional’  towards  a  broader 
societal ambition. The analysis in this benchmark report focuses specifically on the Strategic 
Priorities  Fund  (SPF)  and  Strategic  Themes  Fund  (STF),  as  these  are  designed  to  foster 
interdisciplinary  research  and  offer  considerable  flexibility  with  regard  to  topics  –  and  are 
therefore  especially  insightful  in  relation  to  the  Excellent  Science  programme  parts  of  the 
European Framework Programmes for Research and Innovation. The objectives of the SPF 
and STF programmes and their implementation are therefore expected to hold lessons for 
EU Framework Programmes (FPs).  

In  addition,  the  benchmark  study  provides  evidence  on  the  importance  of  the  UK’s 
association  to  Horizon  Europe,  as  well  as  more  information  about  the  contingency  plan 
developed  by  the  UK  in  the  event  that  association  had  not  taken  place1203.  At  the  time  of 
writing  and  when  interviews were  carried  out,  the  United  Kingdom  was considered  a  third 
country  and  the  subject  of  considerable  uncertainty  regarding  association.  This  situation 
sheds light on the added value of the Framework Programme and the impacts of participation 
and support. 

This report is structured as follows: Section 1.2 presents the methodological approach used 
and the evaluation criteria addressed in this report. Section 1.3 briefly outlines the relevant 
UK policy context and introduces the main features of UKRI and the SPF and STF. Section 
1.4 presents the results and findings of the analysis in relation to the participation patterns of 
the organisation and its key results and impacts, including the ‘association’ history of UK to 
the Framework Programme. Section 1.5 identifies the key factors that have influenced the 
progress  of  the  programme,  including  the  barriers  and,  where  relevant,  the  solutions  that 

1203 On 7 September 2023, a joint Statement by the European Commission and the UK 
Government on the UK's association to Horizon Europe (and Copernicus) was made public. This 
stated that an agreement in principle had been reached under the Trade and Cooperation 
Agreement. This notes that ‘UK researchers will be able to fully participate in the Horizon Europe 
programme on the same terms as researchers from other associated countries, including leading 
consortia, from the 2024 Work Programmes and onwards – including any 2024 calls opening this 
year.’ Transitional arrangements for 2023 calls will remain in place. For more information, see: 
https://ec.europa.eu/commission/presscorner/detail/en/statement_23_4375. 

746 

 
 
 
have  been  implemented  to  address  those  barriers.  The  results  of  the  analysis,  including 
critical  success  factors  and  barriers,  are  subsequently  used  to  formulate  lessons  for  the 
Framework Programme in Section 1.7. Section 1.8 presents the conclusions of the report. 

1.2.  Methodology and evaluation questions addressed 

The main findings of this benchmark study are arranged according to the evaluation criteria 
of relevance, efficiency, effectiveness, and EU added value. 

It relies on a variety of methods to shed light with respect to the above criteria: 

•  Desk  research  of  various  sources  of  information  has  been  harnessed  throughout  the 
process.  These  sources  include  the  websites,  evaluation  reports,  strategy  and 
implementation reports of UKRI and it constituent parts, as well as relevant information 
on Horizon 2020 and Horizon Europe.  

•  Desk research was complemented by interviews with representatives of UKRI  (5) and 

other stakeholders such as beneficiaries from the Strategic Priorities Fund (2). 

1.3.  Context 

1.3.1.  The R&I policy context in the United Kingdom 

The Innovation Scoreboard ranks the United Kingdom as a strong innovator with an overall 
performance  of  117.8%.  This  number  represents  a  weighted  average  of  performance 
indicators, with 100% representing the average across the EU. In other words, the  United 
Kingdom performs relatively well in comparison to the EU average.  

According to OECD data, the UK’s total gross domestic spending on R&D in 2020 was close 
to 3% of GDP – well above the 2.19% average for the EU-27. In addition, public spending on 
R&D has increased substantially in recent years, in particular between 2021 and 20221204. As 
another illustration of its leadership on innovation, the United Kingdom ranked fourth out of 
132 countries in an innovation ranking by the World Intellectual Property Organization (WIPO) 
in 2022.  

The  UK  Innovation  and  Development  Road  Map,  published  in  July  2020,  charts  the  way 
forward  for  policy-making  in  the  area  of  research  and  innovation.  This  document  also 
provides an overview of the R&I funding landscape1205. 

1.3.2.  History, organisation and governance of UKRI 

UKRI is a central actor in the innovation ecosystem because the organisation comprises of 
seven different research councils, structured by scientific fields: 

•  Arts and Humanities Research Council (AHRC); 
•  Biotechnology and Biological Sciences Research Council (BBCRC); 
•  Economic and Social Research Council (ESRC); 

1204 Department for Business, Energy & Industrial Strategy, BEIS research and development 
(R&D) budget allocations 2021 to 2022; https://www.gov.uk/government/publications/beis-
research-and-development-rd-budget-allocations-2021-to-2022. 
1205 See https://www.gov.uk/government/publications/uk-research-and-development-
roadmap/uk-research-and-development-roadmap. 

747 

 
 
•  Engineering and Physical Sciences Research Council (EPSRC); 
•  Medical Research Council (MRC); 
•  Natural Environment Research Council (NERC); 
•  Science and Technology Facilities Council (STFC). 

Innovate  UK,  the  national  innovation  agency,  which  focuses  on  business-led  innovation 
regardless of sector and technology, and Research England, an organisation that engages 
with higher education providers, are also part of UKRI’s architecture. 

These nine organisations predate the establishment of UKRI by several decades, and have 
developed a reputation for encouraging, supporting and funding research excellence. UKRI 
was  established  on  1  April  2018  by  the  Higher  Education  and  Research  Act,  at  the 
suggestion of an influential report published in 2016 that recommended the creation of a new 
public body alongside the aforementioned organisations1206.  

UKRI provides the majority of public funding for R&I in the UK, with a diverse portfolio 
of investments spanning all disciplines and economic activities through its councils. It also 
has a statutory advisory role towards the UK Government regarding R&I policy. UKRI’s 
budget  for  the  three  financial  years  2022-23  to  2024-25  adds  up  to  a  total  of  GBP  25.1 
billion1207. UKRI is fully funded by the Department for Science, Innovation and Technology 
(DSIT) through a process called the Spending Reviews and Budgets. 

Of particular interest to this benchmark is gaining an insight into how strategic priorities are 
defined  at  both  overall  and  at  council-specific  level  to  optimise  the  relevance  of  the 
programme.  The  organisation's  strategic  objectives  are  people  &  careers,  places,  ideas, 
innovation and impacts1208.  

The main governance bodies of UKRI are the UKRI Board and the Executive Committee. 
The latter provides strategic advice to the Board and is the day-to-day coordinating body for 
the activities of the organisation. While UKRI’s overall strategy is set at organisational level, 
its implementation is conducted at council level, with considerable autonomy being given to 
each council to design its own programmes and to contribute to the overall strategy. Every 
council  has  its  own  governance  structures  and  bodies  in  place  to  keep  abreast  of 
developments within its sphere of interest, and to set its own policies. 

1.3.3.  The Strategic Priorities Fund and Strategic Themes Fund 

Within the broad portfolio of UKRI, the Strategic Priorities Fund and Strategic Themes Fund 
are particularly insightful to focus on in this benchmark on Excellent Science, mainly due to 
their  multidisciplinary  and  directional  nature,  which  correspond  to  the  relevance  and 
effectiveness perspectives of this benchmark, and to the criterion of efficiency with regard to 
engagement  with  the  scientific  community  and  other  relevant  stakeholders.  The  analysis 
therefore focuses on exploring these funds in greater depth. 

1206 Department for Business, Innovation and skills, Department for Energy Security and Net 
Zero, Case for the creation of UK Research and Innovation, available at: 
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/fil
e/527803/bis-16-291-ukri-case-for-creation.pdf.  
1207 UKRI-090822-BudgetAllocationExplainer-2022To2023-2023To2024-2024To2025.pdf. 
1208 Strategic objectives of UKRI, according to its five-year Strategic Plan. Source: UKRI, UKRI 
International Strategic Framework, 2022, available at: https://www.ukri.org/wp-
content/uploads/2022/10/UKRI-300922-InternationalStrategicFramework.pdf. 

748 

 
 
The  Strategic  Priorities  Fund  (SPF)  is  an  investment  scheme  worth  GBP  830  million, 
funded through the National Productivity Investment Fund (NPIF)1209. It invests in 34 themes, 
typically cutting across different scientific and research areas managed by UKRI. Two waves 
of funding have taken place, one in 2018 and one in 2019. While these calls are over, the 
successful projects will continue to receive funding until 2023-24. The aims of the SPF are 
threefold: 

•  To increase high-quality multi- and interdisciplinary research and innovation; 
•  To  ensure  UKRI  investment  corresponds  effectively  with  government  research  and 

innovation priorities; 

•  To respond to strategic priorities and opportunities. 

The SPF was announced in the Industrial Strategy White Paper in December 20171210. An 
interim evaluation, as well as insights from interviewees, reveals that the scheme has been 
relatively successful in achieving these aims1211. Its projects were multi-disciplinary by nature, 
involving at least two research councils. There was strong governmental involvement in the 
preparatory  phases  of  the  almost  all  programme  bids,  often  involving  several  government 
agencies  and  departments. Overarching  government  priorities  in  research  and innovation, 
such as research on productivity or on modern slavery, are well reflected in the SPF, and the 
fund  made  a  sizeable  difference  to  the  levels  of  investment  in  these  key  priority  areas. 
Interviewees also note that calls under the SPF were not very prescriptive, allowing enough 
leeway to include novel ideas and experiment. This flexibility and room for manoeuvre was 
deemed  important  in  tackling  societal  problems  in  an  interdisciplinary  or  multidisciplinary 
manner.  These  findings  are  relevant  in  the  context  of  ongoing  discussions  regarding  the 
balance  between  top-down  and  bottom-up  research,  and  the  need  to  encourage 
multidisciplinary  research  in  order  to  address  societal  challenges  under  the  Framework 
Programmes.  

At the same time, the evaluation identified room for improvement in some areas of efficiency 
and in effectiveness in sustainability of research outcomes. One area for improvement is that 
the Fund was set up over a brief time span, and as a consequence, awareness of possible 
applications could have been improved. The guidance and information provided concerning 
what is expected could also have been improved, as could the transparency of the outcome 
(which is in turn related to the need for a clear scoring process). The Fund’s approval process 
was  also  found  to  be  relatively  slow.  Lastly,  interviewees  expressed  concerns  that  some 
projects  would  benefit  from  follow-up  funding  that  cannot  be  guaranteed  by  the  SPF. 
Infrastructures or networks that were created by the SPF (by providing financial resources 
for a fixed duration) risk being dismantled after the projects end due to a lack of funding, even 
if they have proved successful and could deliver value for money over a longer period. 

1209 The National Productivity Investment Fund (NPIF) was established in 2016 by the UK 
Treasury to provide additional capital investment in areas critical to productivity such as housing, 
transport, digital infrastructure and R&D. The NPIF was extended to run until 2023-24, and had 
an overall budget of GBP 37 billion. For more information, see: 
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/fil
e/752202/Budget_2018_red_web.pdf. 
1210 Department for Business, Energy & Industrial Strategy, 
https://www.gov.uk/government/publications/industrial-strategy-building-a-britain-fit-for-the-
future. 
1211 Technopolis Group, 2022, Strategic Priorities Fund (SPF) Baseline and Interim Process 
Evaluation – Main Report (v3), available at: https://www.ukri.org/publications/strategic-priorities-
fund-baseline-and-interim-process-evaluation/. 

749 

 
 
The UK Strategic Themes Fund (STF) can be considered the successor of the SPF, as it 
shares close similarities in its ambitions, which are: 

•  To  take  a  coordinated  and  collaborative  approach  across  the  UKRI  and  its  partner 

organisations to tackle societal challenges; 

•  To  develop  funding  opportunities  for  multi-disciplinary  work  in  relation  to  the  strategic 

themes and to unlock additional funding from other sources; 

•  To build and strengthen international partnerships. 

In addition, the STF seeks to address some of the shortcomings that were identified in the 
SPF, such as its time-consuming process, as well as reducing competition for funding across 
different councils, and allowing more bottom-up initiatives from the research community to 
tackle societal issues (see Section 1.6). These are relevant lessons to consider with regard 
to  ensuring  a  balance  of  top-down  and  bottom-up  approaches  and  the  efficiency  of 
procedures. 

The  STF  aims  to  leverage  the  potential  of  the  R&I  landscape  to  address  the  key  societal 
challenges set out in the five-year UKRI strategy ’Transforming Tomorrow Together 2022 to 
2027; (which are, in turn, heavily influenced by the priorities of the UK Government). The five 
themes are: 

•  Building a green future; 
•  Building a secure and resilient world; 
•  Creating opportunities, improving outcomes; 
•  Securing better health, aging and well-being; 
•  Tackling infections. 

GBP 75 million is allocated to each of these five themes, with the overall budget totalling GBP 
375 million. This investment spans the six years from 2023 to 2029, with an initial commitment 
of up to GBP 185 million from 2023 to 2025. 

For each of the five themes, an action plan has been developed. Each includes key partner 
organisations (from within and beyond UKRI), the exact scope and sub-themes that are part 
of the remit, as well as initial funding opportunities. As the first calls were opened in 2023, it 
is too early to report any official data concerning patterns of participation or success stories 
(aside from anecdotal information from interviewees that the first impressions are favourable). 

The effectiveness of these programmes in terms of results will be discussed further in the 
section that follows. 

1.4.  Effectiveness: results 

1.4.1.  Activities and budget of UKRI 

Although this benchmark report mainly examines the project-based research and innovation 
funding provided by UKRI, it is important to note that its investment portfolio is much broader, 
ranging  from  infrastructure  investments  to  programmes  with  higher  education  providers. 
Information  on  the  distribution  of  the  UKRI  budget  by  activity,  by  organisation  and  project 
partners is presented in Figure 1, Figure 2 and Figure 3 in Appendix 2. Funding for higher 
education providers and research grants make up the largest shares of the budget.  

Table  1  below  presents  a  breakdown  of  the  UKRI  budget  by  council  and  by  ringfenced 
spending category for 2021-2022, the most recent year for which this type of information is 
available.  As  the  name  implies,  these  ringfenced  spending  categories  are  laid  down  in 

750 

 
advance,  and  funds  cannot  (easily)  be  moved  from  one  spending  category  to  another. 
Strategic Programmes is one of these ringfenced spending categories. These are a set of 
programmes that are deemed of strategic importance to addressing the transversal issues 
highlighted  in  the  UKRI  Corporate  Plan,  and  includes  the  SPF  and  STF.  The  Strategic 
Programmes make up 16% of the UKRI budget. 

C
R
H
A

C
R
S
B
B

C
R
S
P
E

C
R
S
E

K
U

I

C
R
M

C
R
N

E
R

C
F
T
S

I

R
K
U

l

a
t
o
T

Research and  
Innovation 
Budgets 

Science 
infrastructure 
capital 

Official 
Development 
Assistance 

Corporate 
funding 

Strategic 
Programmes 

110 

364 

946 

183 

667 

709 

352 

1,172 

554 

37 

5,693 

65 

145 

36 

82 

60 

282 

213 

51 

934 

139 

139 

2 

7 

5 

1 

4 

7 

4 

5 

111 

5 

151 

26 

31 

129 

33 

496 

116 

24 

363 

61 

33 

1,355 

Total budget 

138 

467 

1,225 

253 

1,166 

914 

440 

1,822 

833 

371 

8,272 

Table 1. Budgets by council and ringfenced spending category, 2021-22 in GBP million. Source: UKRI 2021/22 budget 
allocations for UK Research and Innovation and author’s calculations, available at: https://www.ukri.org/wp-
content/uploads/2022/06/UKRI-270521-UKRI-Allocation-Explainer-2021-22-FINAL-PDF.pdf.  

1.4.2.  Projects and participation in the Strategic Priorities Fund1212 

Within the Strategic Programmes, there are multiple programmes that are quite distinct, with 
two  programme  parts  being  funded  entirely  by  Research  England.  As  mentioned,  the 
Strategic Priorities Fund (SPF) is part of the Strategic Programmes of the UKRI. The SPF 
was allocated a total amount of GBP 192 million in the budgetary year 2021-2022, the most 
recent year for which this type of data are available, with all organisations being represented 
except Research England (which was not the lead organisation for any SPF projects in that 
year) (see Table 2). 

1212 The STF was introduced in 2023, and hence no further data are currently available. 

751 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
C
R
H
A

C
R
S
B
B

C
R
S
P
E

C
R
S
E

K
U

I

C
R
M

C
R
N

E
R

C
F
T
S

I

R
K
U

l

a
t
o
T

Industrial Strategy Challenge Fund 

16 

44 

468 

15 

543 

Future Leaders Fellowships 

89 

Strength in Places Fund 

69 

89 

69 

Fund for International Collaboration 

4 

4 

4 

13 

8 

9 

5 

1 

48 

Strategic Priorities Fund 

9 

27 

52 

9 

10 

15 

22 

42 

8 

192 

Skills, Talent and Others 

1 

4 

33 

24 

17 

3 

2 

5 

4 

25 

116 

Dual Support 

289 

289 

Table 2. Budgets by council and programme part within the Strategic Programmes, 2021-22 in GBP million. Source: UKRI 
2021/22 budget allocations for UK Research and Innovation and author’s calculations, available at: https://www.ukri.org/wp-
content/uploads/2022/06/UKRI-270521-UKRI-Allocation-Explainer-2021-22-FINAL-PDF.pdf. 

In total, 34 projects were funded by the SPF. These are grouped into different thematic areas 
(see Table 3). The coverage of these different thematic areas by several projects illustrates 
the  effectiveness  of  the  programme  in  funding  research  in  the  main  priority  areas.  It  is 
furthermore worth noting that each of the seven councils leads at least one of these projects, 
and partners with at least one other UKRI organisation, with considerable variation between 
one project and another. The ‘Quantum technologies for fundamental physics’ programme, 
for instance, is led by the STFC with the EPSRC as the only partner. At the other end of the 
spectrum,  the  National  Interdisciplinary  Circular  Economy  Research  (NICER)  programme, 
involves six UKRI organisations (AHRC, BBSRC, ESRC, EPSRC, NERC and Innovate UK). 

Number of projects 

Combined budget (in GBP 
million) 

Environment 

Health, well-being and human 
rights 

Biology and bio-medicine 

Productivity and technical 

200.9 

104.5 

89.2 

237.4 

9 

6 

4 

9 

752 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Number of projects 

Combined budget (in GBP 
million) 

AI & Digital 

Infrastructure 

5 

1 

115.9 

71.2 

Table 3. Thematic areas of the projects funded by the SPF. Source: UKRI website, https://www.ukri.org/what-we-do/our-
main-funds-and-areas-of-support/strategic-priorities-fund/. 

In  terms  of  collaborations  and  efficiency  in  engaging  stakeholders,  many  of  the  above 
projects  involve  partners  from  various  stakeholder  groups  outside  UKRI  (e.g.  private 
companies,  governmental  bodies 
research 
from  different 
organisations,  and  so  on).  As  an  illustration,  the  project  on  AI  and  Data  Science  for 
Engineering,  Health  and  Government  (ASG)  involves  a  total  of  32  partners,  ranging  from 
Rolls-Royce (an engineering company) to the City of Bradford Metropolitan District Council. 
As another example, the Advanced Pain Discovery Platform (APDP) is a partnership between 
the MRC, ESRC BBSRC, as well as Versus Arthritis (a charity), Eli Lilly (a pharmaceutical 
company) and the Medical Research Foundation. 

levels  of  government, 

1.5.  EU  added  value:  association  of  the  United  Kingdom  to  Horizon 

Europe 

The United Kingdom formally left the Framework Programme (Horizon 2020) when it left the 
EU’s legal framework at the end of 2020. In January 2021, the UK Government announced 
that the country would associate to Horizon Europe by means of a Protocol to the Trade and 
Cooperation  Agreement.  Association  means  that  researchers  and  innovators  can  apply  to 
most Horizon programmes and applications are treated equally with those from EU Member 
States. At the time of writing and when interviews were carried out for the present report, the 
United Kingdom was considered a third country, with considerable uncertainty regarding the 
prospect  of  association.  This  situation  sheds  light  on  the  added  value  of  the  Framework 
Programme and the impacts of participation and support. 

While  association  was  still  to  be  formalised  at  that  time,  applicants  based  in  the  United 
Kingdom  could  still  apply  to  Horizon  Europe  calls.  More  formally,  ‘The  General  Annexes 
attached to the main Horizon Europe work programmes (2021-2022 and 2023-2024) ensure 
that UK applicants are treated as if the UK is an associated country throughout the process, 
from admissibility and eligibility to evaluation, up until the preparation of grant agreements. 
However, grant agreements can only be signed if the association has come into force. The 
same treatment is also granted to any applicants from other associated countries engaged 
with the European Commission in an active process of association.’1213 

In addition, the UK government made a commitment that it would fund successful Horizon 
Europe applicants from the United Kingdom in the event that the association agreement ran 
into  further  delays  or  did  not  take  place.  The  so-called  ‘Horizon  Europe  guarantee’ 
programme came into force in November 2021, and was intended to ensure that successful 
applicants would be able to access funds, either from the EU or from the UK government (in 
the event of non-association).  

1213 https://spice-spotlight.scot/2023/07/13/eu-uk-relations-developments-on-the-horizon/. 

753 

 
 
 
The net EU contribution to the UK under Horizon 2020 constituted EUR 7.48 billion, or 11.5% 
of the total, with 18,456 participations and 3,590 unique participants (9.64% and 7.70% of all 
countries respectively). In other words, the UK was an important actor in Horizon 2020, with 
Excellent  Science  representing  a  major proportion  of  all participations.  Breakdowns of  UK 
participations  by  pillar  and  by  type  of  action  are  presented  in  Figure  4  and  Figure  5  in 
Appendix 2. 

The number of participations from the UK under Horizon Europe, at 2,205 in August 2023 or 
4.43% of the total, compares unfavourably with the previous FP (in which participants from 
the  UK  made  up  9.64%  of  all  participations)1214.  This  partly  reflects  the  fact  that  UK 
organisations who win ERC grants (for which they are still eligible) are technically not funded 
through EU funds, but rather through a replica scheme from UKRI. This technicality means 
that the drop in participation rates is not as large as the above numbers suggest. There is 
strong  evidence,  however,  that  even  when  taking  this  arrangement  into  account,  fewer 
researchers  and  organisations  in  the  UK  are  participating  in  Pillar  2  projects  under  the 
Horizon Europe framework in comparison to the recent past1215. Uncertainty related to the 
Brexit and the possibility of non-association is the most likely reason for this decline, probably 
leading to hesitation regarding the inclusion of UK organisations into international research 
consortia, especially in a leading role1216. 

In  terms  of  the  effectiveness  of  international  collaboration,  the  UK  set  up  around  31,000 
collaborative  links  with  other  countries  through  Horizon  2020.  This  allowed  participants  to 
operate  globally,  reaping  economies  of  scale  and  working  with  the  most  advanced 
organisations globally1217. Maintaining these links was the most important reason for the UK 
to  push  for  association,  according  to  most  interviewees  from  UKRI  (which  is  the  body 
representing UK interests with regard to the association). While the UK government could 
potentially  have  replaced  EU funding  streams  with  domestic  ones,  the  network  and  reach 
that the EU Framework Programme provides is not so easily replaced. Indeed, interviewees 
highlighted that this would be almost impossible to do so on a bilateral basis1218. Such benefits 
thus  constitute  a  highly  important  element  of  the  EU  added  value  of  the  Framework 
Programme. 

In addition, as Figure 4 in the Appendix 2 illustrates, the UK was successful in attracting Marie 
Skłodowska-Curie Actions (MSCA) under Horizon 2020, allowing organisations active in UK 
to  attract  and  retain  more  talent,  and  providing  training  opportunities  for  their  staff.  Non-
association would therefore also have potentially negatively affected the pool of talent in the 
UK’s R&I system. Interviewees also point out that the EU Framework Programme, with its 
seven-year life span, offers stability and predictability for its participants – which in turn leads 
them to focus on long-term achievements. For these reasons, an independent and influential 
review of the Research, Development and Innovation (RDI) Organisational Landscape of the 
United Kingdom, published in March 2023, called for association to be quickly achieved, and 

1214 Data from Horizon Dashboard, accessed 5 August 2023. 
1215 Sciencebusiness, Here’s what the first two years of Horizon Europe look like in numbers, 
available at: https://sciencebusiness.net/news/Horizon-Europe/heres-what-first-two-years-
horizon-europe-look-numbers 
1216 https://theconversation.com/horizon-europe-how-the-uks-delay-in-rejoining-eu-funding-
scheme-is-damaging-scientific-research-
209279#:~:text=Until%202020%2C%20the%20UK%20was,Brexit%20changed%20that. 
1217 Russell Group, Horizon Europe: Maintaining UK leadership in global R&D collaboration, 
2022, available at: https://russellgroup.ac.uk/media/6054/benefits-of-horizon-europe.pdf 
1218 One important caveat is that interviewees highlighted that UKRI (and the United Kingdom 
more generally) collaborates extensively with European and other research partners outside of 
the Horizon Europe framework, for instance through large-scale research infrastructures. 

754 

 
 
indicated that failure to do so would be highly damaging to the UK’s RDI endeavours1219. Next 
to international collaboration, attracting a broader pool of talent to the UK R&I system is thus 
considered an important aspect of the added value of the Framework Programme. 

The UK outlined a contingency plan to transition to a new R&D programme in the event that 
association did not materialise. This plan reflects both of the above elements – international 
collaboration  and  talent.  To  outline  the  plan  in  brief:  the  UK  funds  earmarked  for  Horizon 
Europe would flow instead to the transition measures. Aside from the guarantee programme 
(mentioned  above)  already  in  place,  the  plan  envisaged:  1)  funding  for  successful 
applications; 2) increased funding for existing programmes for innovation and talent support; 
3) the creation of the Talent and Research Stabilisation Fund to support those UK entities 
most affected by non-association; and 4) a funding scheme for those projects in which UK 
applicants apply as third-country applicants in a consortium with applicants from the EU or 
associated  countries.  The  three  measures  listed  were  designed  to  provide  stability  for  a 
relatively  short  period,  and  thus  ensure  that  the  immediate  fall-out  resulting  from  non-
association would not be too damaging. In addition, the plan foresaw longer-term measures 
to secure the UK’s place in collaborative research and R&D. These included, among others, 
a  new  flagship  talent  offer,  with  high-value,  long-term  fellowships;  a  strong  international 
mobility  offer;  additional  funds  for  bottom-up  collaborations  with  researchers  in  partner 
countries  around  the  globe;  a  strong  increase  in  core  Innovate  UK  funding;  and  higher 
spending on (research and technology) infrastructures1220. 

1219 Department for Business, UK’s Research, Development and Innovation Organisational 
Landscape: Final Report and Recommendations, available at: 
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/fil
e/1141484/rdi-landscape-review.pdf.  
1220 Department for Science, Innovation and Technology and Department for Business, Energy & 
Industrial Strategy, Supporting UK R&D and collaborative research beyond European 
programmes, available at: https://www.gov.uk/government/publications/supporting-uk-rd-and-
collaborative-research-beyond-european-programmes. 

755 

 
 
1.6.  Relevance and efficiency: critical success factors and barriers 

A  common  feature  of  all  of  the  projects  funded  by  UKRI  is  that  they  follow  the  Haldane 
principle  that  ‘decisions  about  what  to  spend  research  funds  on  should  be  made  by 
researchers  rather  than  politicians’.  This  principle  is  enshrined  in  the  UKRI  Framework 
Document1221. Interviewees commonly referred to it as a lodestar that guides their funding 
decisions and policy approach. UKRI puts this into practice by making funding decisions 
through peer review. While this process varies somewhat from one council to another and 
between  programmes,  the  general  principle  is  well  established.  The  Arts  and  Humanities 
Council, for example, has a network of around 1,000 experts who review applications and 
ultimately decide which proposals merit funding, with ad hoc expertise for specialist projects. 
Interviewees consider this an important element in ensuring that only research of high quality 
gets funded. Detailed guidance is provided to peer reviewers. The Medical Research Council, 
as one example, has developed a 36-page guidance document1222. Among other things, this 
minimises  the  risk  of  conflicts  of  interest  and  subjective  factors  in  the  decision-making 
process. 

Another manifestation of the Haldane principle is that, even when the broad research topics 
to  be  funded  are  set  in  a  top-down  manner,  reflecting  government  priorities,  they  leave 
sufficient  degrees  of  freedom  for  the  research  community  to  put  these  into  practice1223. 
This  is  evident in  the  SPF  and  STF,  for instance.  According  to  interviewees,  priorities  are 
sufficiently well targeted that they catalyse the research community and government actors 
to  work  together  on  a  joint  vision,  while  also  being  sufficiently  broad  to  allow  grass-roots 
initiatives to flourish.  In other words, even where funds can be considered to follow a top-
down approach, with the SPF and STF  being the most notable examples, the pre-defined 
topics are sufficiently broad to incorporate bottom-up elements. Indeed, a mid-term review of 
the  SPF  included  the  openness  and  flexibility  to  propose  different  ideas  as  a  major 
strength1224. 

Several interviewees pointed out that there has been something of a shift towards bottom-
up funding schemes. While it is hard to gain a complete picture of the overall situation at 
UKRI level, due to the considerable autonomy of its constituent bodies, bottom-up funding 
schemes appear more prevalent than under the EU Framework Programme, again according 
to  interviewees.  The  AHRC,  for  instance,  provides  two-thirds  of  its  budget  to  bottom-up 
schemes  (research  grants  and  PhD  funding),  with  the  remaining  third  going  to  top-down 
scheme (but, as previously noted, with considerable freedom for applicants to formulate novel 

1221 The Framework Document sets out the broad parameters within which UKRI operates, and 
how its relationship between the DSIT (previously the Department for Business, Energy & 
Industrial Strategy – BEIS) is defined: Department for Business, Energy and Industrial Strategy, 
UKRI Framework Document, available at: https://www.ukri.org/wp-
content/uploads/2020/10/UKRI-111020-UKRIFrameworkDocument.pdf. 
1222 UKRI Medical Research Council, Guidance for Peer Reviewers, available at: 
https://beta.ukri.org/wp-content/uploads/2023/06/MRC-280623-GuidanceforPeerReviewers-
EmbeddingDiversityUpdates-June-2023.pdf. 
1223 UKRI, Principles of assessment and decision making, available at: https://www.ukri.org/wp-
content/uploads/2021/03/UKRI-310321-Principles-of-Assessment-and-Decision-Making-V2.pdf. 
1224 Technopolis Group, Strategic Priorities Fund (SPF) Baseline and Interim Process Evaluation 
– Main Report (v3), 2022, available at: https://www.ukri.org/publications/strategic-priorities-fund-
baseline-and-interim-process-evaluation/. 

756 

 
 
ideas)1225. This is a conscious decision, reflecting the idea that the research community is 
well placed to identify the most promising areas for funded research. 

As previously described, the UKRI’s five-year plan sets it broad direction and general topics 
of importance. These are made more concrete, or ‘unpacked’ in the Strategic Delivery Plans 
of the nine constituent councils. In short, these delivery plans detail the activities the councils 
will undertake with respect to UKRI’s broad strategic objectives, over the three-year spending 
review period. Interviewees highlighted that this model works well, and ensures a measure 
of ‘directionality’ with regard to the various projects funded, as well as encouraging 
multi-disciplinary and collaborative research. 

Interviewees also highlighted that the development of an overall strategy at UKRI level is a 
collaborative  process,  involving  all  of  the  different  councils.  Its  joint  development  and 
influence over the (funding) activities fosters an overall culture of collaboration. The SPF and 
STF  perpetuate  this  culture of  collaboration,  with  interviewees  pointing  out  that  this  had  a 
lasting impact in encouraging cross-disciplinary research. According to interviewees, the SPF 
was  particularly  successful  in  this  respect.  The  significant  size  of  this  fund,  specifically 
intended to set up joint projects involving different research councils, different government 
departments and other stakeholders, provided powerful incentives to collaborate. Its lasting 
impact  reflects  both  the  creation  of  platforms  and  structures  spanning  various  (research) 
communities  and  a  more  general  shift  in  mindset  towards  collaboration,  according  to 
interviewees. 

Another novel example of the efforts to ‘break the silos’ and bring research communities 
and  other  stakeholders  together  is  the  ‘Sector  Champions’  initiative,  which  is  currently 
under development. This initiative aims to identify a handful of economic sectors that are both 
important for the future of the UK economy and which span the boundaries of different sectors 
(for instance, the creative industries). After these are identified, the initiative will formulate 
suggestions to improve support for research and innovation activities through the different 
councils that are relevant to the economic sectors concerned. While this strategy is still under 
development and relatively little information is available, this overall approach of setting up 
collaborative platforms to cater to the needs of sectors that do not fall within the boundaries 
of a single council is illustrative of a drive to push for further research cooperation. 

One possible shortcoming of the SPF, according to interviewees, is that the process often 
felt rushed. This was also confirmed by an independent evaluation, which states that there 
was insufficient time to get the Fund’s cross-departmental priorities right, to develop ideas, 
and to fine-tune the bidding and selection processes1226. Another potential area for attention 
is  that  the  approval  processes  and  general  administrative  procedures  (such  as 
reporting requirements) were quite  (resource-)intensive and  time-consuming, relying 
in part on outside funding. This meant that approval from the Treasury and other ministries 
was  often  needed  for  investments  through  the  SPF.  In  one  case,  a  large investment  was 
delayed by three to four months while it awaited approval from the Treasury, which in turn 
complicated the meeting of targets set for its first year of operation. 

Another  issue  with  the  SPF  was  that  the  allocation  process  was  (too)  competitive  in 
nature, which was deemed not conducive to a spirit of mutual trust and collaboration and led 
to negotiations between councils over budget that were not productive to advancing research 

1225 UKRI, Arts and Humanities Research Council, Strategic Delivery Plan, 2022-25, 
https://www.ukri.org/wp-content/uploads/2022/09/AHRC-010922-StrategicDeliveryPlan2022.pdf. 
1226 Technopolis Group, Strategic Priorities Fund (SPF) Baseline and Interim Process Evaluation 
– Main Report (v3), 2022, available at: https://www.ukri.org/publications/strategic-priorities-fund-
baseline-and-interim-process-evaluation/. 

757 

 
 
and addressing societal problems. Although it is too early for its successor, the STF, to be 
subject to a formal evaluation, there are indications that it has learned from these lessons. 
For one, it allocates a pre-set, equal amount of budget (i.e. GBP 75 million) to each of the 
five objectives (to be revised, if necessary, after the next Spending Review). It is also more 
‘open’ or ‘bottom-up’ in nature, with fewer content requirements. The funding also comes 
from  internal  sources  rather  than  from  the  NPF,  leading  to  faster  processes,  with  the 
additional bonus that the funding is more stable, with less risk that the projects will need to 
be discontinued after the initial funding ends. This aspect is further enhanced by the fact that 
the individual projects of the strategic themes are often smaller in size, which also contributes 
to leaner and faster procedures with lower overheads.  

1.7. 

Transferability of findings to the EU Framework Programme 

This section presents some potential lessons that can be drawn from the case of UKRI, which 
may be of interest to the EU´s Framework Programme, and which relate to the evaluation 
criteria of efficiency and effectiveness of processes (peer review approach and processes); 
the  relevance  of  topics;  directionality  and  engagement  of  the  scientific  community  and 
stakeholders;  and  insights  into  EU  added  value  resulting  from  the  context  of  uncertainty 
regarding the country’s association to the Framework Programme. 

First,  the  peer-review  approach  to  evaluation  was  valued  by  interviewees  as  a  way  to 
involve the research community in decisions about which research to fund. The European 
Research Council (ERC) Frontier Research Grants use a peer review process that is similar 
to the one used by the UKRI councils, as illustrated by the guidance document for the Frontier 
Research  Grants1227.  Other  programme  parts  of  Excellent  Science  also  employ  a  panel 
review process, whereby experts are selected from a database. The experience of the United 
Kingdom confirms the importance such a panel review process, in large part due to the broad 
agreement  among  interviewed  stakeholders  that  the  current  system  within  UKRI  is  fit  for 
purpose (albeit with some variations across the councils). 

In a similar spirit, the flexibility and openness towards the incorporation of ideas seen in top-
down  calls  in  which  only  a  broad  theme  is  pre-defined  could  merit  closer  scrutiny.  In 
particular, this may be a suitable model to closer incorporate EU Missions into the Framework 
Programme  for  Research  and  Innovation,  given  the  similarities  between  the  Strategic 
Themes and the EU Missions approach1228. This flexibility could encourage the development 
of  groundbreaking  ideas  that  may  not  have  been  ‘on  the  radar’  in  advance.  The  UKRI 
approach could serve as an inspiration, given that several position papers regarding the past, 
present and future of the European Research & Innovation Framework Programmes have 
recommended  broader,  less  prescriptive  topics,  especially  for  lower  TRL  (Technology 
Readiness Level) calls1229. One of the objectives of the FP is to foster scientific collaboration 
across disciplines in order to advance novel ideas that can tackle broad societal problems. 
According  to  interviewees,  the  councils  in  UKRI  have  made  considerable  progress  in  this 

1227 https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/experts/guide-for-
peer-reviewers_he-erc-syg_en.pdf. 
1228 https://research-and-innovation.ec.europa.eu/funding/funding-opportunities/funding-
programmes-and-open-calls/horizon-europe/eu-missions-horizon-europe_en.  
1229 This has been suggested by the League of European Research Universities (LERU), LERU 
key messages for the current and future European R&I Framework Programme, available at: 
https://www.leru.org/files/LERU_statement_EU-FP_consultation_2023-02.pdf; and by the Guild 
of European Research-Intensive Universities, Lessons learned and way forward: the Guild’s 
recommendations for Horizon Europe, available at: https://www.the-
guild.eu/publications/position-papers/lessons-learned-and-way-forward-the-guild-s-
recommendations-for-horizon-europe.pdf. 

758 

 
 
area  by  developing  a  solid  strategy  for  engagement  with  the  research  community  in 
several fields. This enables the research community to increasingly exchange information, 
work  together  and  propose  joint  bids.  It  is  important  to  note  that,  even  though  UKRI  was 
established  in  part  to  support  cross-cutting  research,  it  took  dedicated  efforts  to  instil  a 
mindset whereby this is seen as a natural thing to do. A 2021 position paper from the Initiative 
for Science in Europe (ISE) called for additional actions to mainstream the multidisciplinary 
perspective within Horizon Europe. Examples included creating more calls across different 
clusters, incentivising interdisciplinary research, and combining various disciplines within a 
single project1230. The experience of UKRI could offer further inspiration on how to set up calls 
in  this  manner, which  are interdisciplinary  in  nature  and  encourage  research  communities 
from different fields to work together. 

The  EC  public  consultation  on  the  past,  present  and  future  of  the  European  Research  & 
Innovation  Framework  Programmes  2014-2027  revealed  that  there may  be some  room  to 
further streamline the application and approval process for Horizon Europe projects. In this 
respect, lessons could be drawn from the STF which (in comparison to its predecessor, the 
SPF) has succeeded in cutting back the bureaucratic procedure without losing sight of its 
mandate to foster interdisciplinary research through a broadly mission-based approach. 

Interviews carried out with representatives of UKRI also revealed that access to networks 
and  collaborative  platforms  across  the  EU-27  and  beyond  is  perceived  as  the  most 
important benefit that association brings. Indeed, given that, compared with its predecessor, 
Horizon  Europe  is  open  a  wider  range  of  countries,  such  as  New  Zealand,  the  case  for 
association  is  even  more  compelling.  This  could  be  regarded  as  an  endorsement  of  the 
European Commission's new approach to strengthening the ties with trusted partners around 
the globe with a solid scientific base and a robust research track record1231. 

Lastly, some lessons can also be drawn from the strategic thinking that took place in the UK 
to prepare for possible non-association. In particular, the flagship talent offer is based on the 
success  of  the  ERC  and  MSCA,  but  aims  to  introduce  some  targeted  innovations  with 
regard to training and mobility that may of in interest. One case in point is the possibility 
to  offer  fellowships  of  longer  duration  to  foreign  senior-level  researchers.  These 
innovations could be followed up on and considered in relation to possible improvements for 
future programming periods. 

1.8.  Conclusions 

This  benchmark  report  zooms  in  on  United  Kingdom  Research  &  Innovation  (UKRI)  and 
identifies certain elements that may be transferable to the European Framework Programme, 
and which relate to elements of its efficiency, effectiveness, relevance and EU added value. 
Among other aspects, the report examines aspects relating to the association of the United 
Kingdom, the balance between top-down and bottom-up funding approaches, and avenues 
for fostering interdisciplinary research and innovation.  

This benchmark report shows that the United Kingdom has a well performing R&I system, 
with UKRI at its centre. The very creation of UKRI is indicative of the UK government’s drive 
to unite multidisciplinary teams behind a common mission, thereby addressing national (and 
international) policy priorities by pursuing research excellence. The organisational structure 
of  UKRI  is  a  strong  point  in  the  UK’s  R&I  landscape,  and  ensures  the  relevance  and 

1230 Initiative for Science in Europe, Executive Summary: Position on Horizon Europe, available 
at: https://initiative-se.eu/wp-content/uploads/2021/07/ISE-position-Horizon-Europe.pdf. 
1231 https://ec.europa.eu/commission/presscorner/detail/en/ip_23_3728.  

759 

 
 
directionality of its actions. UKRI’s very broad strategy is defined at organisational level, with 
considerable input from its constituent councils. This strategy is then operationalised by its 
constituent  councils.  This  arrangement  delivers  a  good  balance  between  tackling  societal 
challenges and pursuing government priorities, while not being overly prescriptive as to how 
to achieve these goals. In addition, recent initiatives by UKRI, such as the SPF, the STF, and 
the Sector Champions initiative, have given impetus to the organisation's overall mission to 
bring  together  research  communities  from  different  scientific  fields  in  order  to  work  on  a 
common goal, and to provide (pecuniary) incentives to do so. UKRI's approach to fostering 
interdisciplinary research and innovation is of particular relevance in this respect. The above 
elements,  together  with  the  use  –  widely  appreciated  by  interviewees  –  of  a  peer  review 
process in which researchers are involved in evaluating the merits of funding proposals, can 
be considered the main success factors of UKRI. 

The context at the time of writing of uncertainty regarding the UK’s association to Horizon 
Europe  provides  further  insights  regarding  the  EU  added  value  of  the  Framework 
Programme. Association of the UK to Horizon Europe is perceived as important by all of the 
stakeholders interviewed, in large part because it provides access to a large network of talent, 
infrastructure, data and knowledge. While the funding from Horizon Europe can be replaced 
by domestic sources, replicating this network outside the EU Framework Programme would 
be a far more challenging and time-consuming endeavour. The data suggest that continuing 
insecurity  at  that  time  with  regard  to  whether  association  would  take  place,  was  already 
negatively  impacting  the  participation  of  United  Kingdom  researchers  in  Horizon  Europe 
projects, in spite of efforts from both sides to the contrary. Meanwhile, the UK Government 
had designed a contingency plan to minimise the impact on UKRI and the broader research 
community in the event that association negotiations were not successful. 

Appendix 1. Literature sources 

•  Department for Business, UK’s Research, Development and Innovation Organisational 

Landscape: Final Report and Recommendations, 2023, available at: 
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachme
nt_data/file/1141484/rdi-landscape-review.pdf. 

•  Department for Business, Energy and Industrial Strategy, Industrial Strategy: Building a 

Britain fit for the future, 2018, available at: 
https://www.gov.uk/government/publications/industrial-strategy-building-a-britain-fit-for-
the-future. 

•  Department for Business, Energy and Industrial Strategy, UKRI Framework Document, 
2018, available at: https://www.ukri.org/wp-content/uploads/2020/10/UKRI-111020-
UKRIFrameworkDocument.pdf. 

•  Department for Business, Energy and Industrial Strategy, BEIS research and 
development (R&D) budget allocations 2021 to 2022, 2021, available at: 
https://www.gov.uk/government/publications/beis-research-and-development-rd-
budget-allocations-2021-to-2022/beis-research-and-development-rd-budget-
allocations-2021-to-2022. 

•  Department for Business, Innovation and skills, Department for Energy Security & Net 

Zero,  Case for the creation of UK Research and Innovation, 2016, available at: 
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachme
nt_data/file/527803/bis-16-291-ukri-case-for-creation.pdf. 

760 

 
•  Department for Science, Innovation and Technology & Department for Business, 

Energy and Industrial Strategy, Supporting UK R&D and collaborative research beyond 
European programmes, 2022, available at: 
https://www.gov.uk/government/publications/supporting-uk-rd-and-collaborative-
research-beyond-european-programmes. 

•  Economic and Economic Research Council, Strategic Delivery Plan 2022-25, available 

at: https://www.ukri.org/wp-content/uploads/2022/09/ESRC-010922-
StrategicDeliveryPlan2022.pdf. 

•  European Commission, 2023, European Innovation Scoreboard 2023 – Country profile 
United Kingdom, available at: https://ec.europa.eu/assets/rtd/eis/2023/ec_rtd_eis-
country-profile-uk.pdf. 

•  European Commission, EU Missions in Horizon Europe, available at: https://research-
and-innovation.ec.europa.eu/funding/funding-opportunities/funding-programmes-and-
open-calls/horizon-europe/eu-missions-horizon-europe_en. 

•  European Commission, ‘New Zealand joins Horizon Europe Research and Innovation 

Programme’, available at: 
https://ec.europa.eu/commission/presscorner/detail/en/ip_23_3728. 

•  European Commission, Joint Statement by the European Commission and the UK 

Government on the UK's association to Horizon Europe and Copernicus, available at: 
https://ec.europa.eu/commission/presscorner/detail/en/statement_23_4375. 

•  Fiona Lettice, Horizon Europe: how the UK’s delay in rejoining EU funding scheme is 
damaging scientific research, The Conversation, https://theconversation.com/horizon-
europe-how-the-uks-delay-in-rejoining-eu-funding-scheme-is-damaging-scientific-
research-209279. 

•  GOV.UK, UK Research and Development Roadmap, 2021, available at: 

https://www.gov.uk/government/publications/uk-research-and-development-
roadmap/uk-research-and-development-roadmap. 

•  Guide for Peer Reviewers to the Synergy Grant call, available at: 

https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/experts/guide-
for-peer-reviewers_he-erc-syg_en.pdf. 

•  Guild of European Research-Intensive Universities, Lessons learned and way forward: 

The Guild’s recommendations for Horizon Europe, available at: https://www.the-
guild.eu/publications/position-papers/lessons-learned-and-way-forward-the-guild-s-
recommendations-for-horizon-europe.pdf. 

•  HM Treasury, Budget 2018, available at: 

https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachme
nt_data/file/752202/Budget_2018_red_web.pdf. 

• 

Initiative for Science in Europe, Executive Summary: Position on Horizon Europe, 
available at: https://initiative-se.eu/wp-content/uploads/2021/07/ISE-position-Horizon-
Europe.pdf. 

761 

 
• 

League of European Research Universities (LERU), LERU key messages for the 
current and future European R&I Framework Programme, available at: 
https://www.leru.org/files/LERU_statement_EU-FP_consultation_2023-02.pdf.  

•  Russell Group, Horizon Europe: Maintaining UK leadership in global R&D collaboration, 

2022, available at: https://russellgroup.ac.uk/media/6054/benefits-of-horizon-
europe.pdf. 

•  Sciencebusiness.net, ‘Here’s what the first two years of Horizon Europe look like in 

numbers’, 2023, available at: https://sciencebusiness.net/news/Horizon-Europe/heres-
what-first-two-years-horizon-europe-look-numbers. 

•  Scottish Parliament, Spice Spotlight, EU-UK relations – Developments on the Horizon, 

available at: https://spice-spotlight.scot/2023/07/13/eu-uk-relations-developments-on-
the-horizon/. 

•  Technopolis Group, Strategic Priorities Fund (SPF) Baseline and Interim Process 

Evaluation – Main Report (v3), 2022, available at: 
https://www.ukri.org/publications/strategic-priorities-fund-baseline-and-interim-process-
evaluation/. 

•  UKRI Investment and Outputs Data 2022-23, available at: https://www.ukri.org/wp-
content/uploads/2023/08/UKRI-070823-InvestmentsOutputsData2022To2023.pdf. 

•  UKRI Medical Research Council, Guidance for Peer Reviewers, 2023, available at: 

https://beta.ukri.org/wp-content/uploads/2023/06/MRC-280623-
GuidanceforPeerReviewers-EmbeddingDiversityUpdates-June-2023.pdf.  

•  UKRI, Principles of assessment and decision making, 2021, available at: 

https://www.ukri.org/wp-content/uploads/2021/03/UKRI-310321-Principles-of-
Assessment-and-Decision-Making-V2.pdf. 

•  UKRI, 2021/22 budget allocations for UK Research and Innovation, available at: 
https://www.ukri.org/wp-content/uploads/2022/06/UKRI-270521-UKRI-Allocation-
Explainer-2021-22-FINAL-PDF.pdf.  

•  UKRI, UKRI International Strategic Framework, 2022, available at: 
https://www.ukri.org/wp-content/uploads/2022/10/UKRI-300922-
InternationalStrategicFramework.pdf. 

•  UKRI, UKRI strategy 2022 to 2027: transforming tomorrow together, 2022, available at: 

https://www.ukri.org/publications/ukri-strategy-2022-to-2027/. 

•  UKRI, 2022-23 – 2024-25 budget allocations for UK Research and Innovation, 

available at: UKRI-090822-BudgetAllocationExplainer-2022To2023-2023To2024-
2024To2025.pdf. 

•  UKRI, Arts and Humanities Research Council, Strategic Delivery Plan, 2022-25, 
available at: https://www.ukri.org/wp-content/uploads/2022/09/AHRC-010922-
StrategicDeliveryPlan2022.pdf. 

762 

 
•  UKRI, How we are governed, available at: https://www.ukri.org/about-us/nerc/how-we-

are-governed/. 

•  UKRI, How we’re funded, available at: https://www.ukri.org/wp-

content/uploads/2022/08/UKRI-050822-FundingExplainer-HowWereFunded.pdf. 

•  UKRI, Our Budget, available at: https://www.ukri.org/about-us/who-we-are/our-budget/. 

•  UKRI, The Strategic Priorities Fund, available at: https://www.ukri.org/what-we-do/our-

main-funds-and-areas-of-support/strategic-priorities-fund/. 

•  World Intellectual Property Organisation, Global Innovation Index 2022: What is the 

future of innovation-driven growth?, 2022, available at: 
https://www.wipo.int/global_innovation_index/en/2022/. 

763 

 
  
 
 
 
Appendix 2. Additional information 

Activities and budget of UKRI 

5% 1%

7%

19%

Research grants

Challenge-led funding

Funding to higher education providers

15%

11%

11%

Skills and training

8%

Infrastructure

Institutes, centres, facilities and
catapults
Responsive innovation

24%

International partnerships

Public engagement

Figure 1. Budget of UKRI by activity, 2022-23, as a percentage of the total budget. Source: UKRI website, 
https://www.ukri.org/publications/explainer-ukris-investment-portfolio/explainer-how-we-manage-investment-in-research-and-
innovation. 

Centrally managed 
funding; 330

Core AHRC; 71

Core BBSRC; 300

Core IUK; 669

Core EPSRC; 621

Core ESRC; 121

Core MRC; 548

Core RE; 1730

Core NERC; 288

Core STFC; 531

Figure 2. Budget of UKRI by organisation, 2022-2023, in GBP million. Source: UKRI website, https://www.ukri.org/about-
us/who-we-are/our-budget/. 

764 

 
 
 
 
4% 6%

25%

4%

61%

Academic/University/Learned Society
Private
Other (including unknown)

Charity/Non-profit
Public bodies

Figure 3. Distribution of project partners of competitively awarded applications from the 2022-23 UKRI budget as a 
percentage of the total number of partner organisations. Source: UKRI Investment and Outputs Data 2022-23, available at: 
https://www.ukri.org/wp-content/uploads/2023/08/UKRI-070823-InvestmentsOutputsData2022To2023.pdf. 

765 

 
 
 
 
 Association of the United Kingdom to Horizon Europe 

Figure 4. Number of participations from the United Kingdom under Horizon 2020 by pillar/theme. Source: Horizon 
Dashboard, consulted 3 August 2023. 

766 

 
 
Figure 5. Number of participations from the United Kingdom under Horizon 2020, by type of action. Source: Horizon 
Dashboard, consulted 3 August 2023. 

767 

 
 
 
 
Annex 3.3: Benchmark on the Austrian Science Fund 

1.1. 

Introduction 

The Austrian Science Fund (FWF)1232 is the national funding organisation for basic research 
in Austria. It aims to support Austrian science and basic research at an international level. 
The  FWF  aims  to  contribute  towards  cultural  development,  the  advancement  of  Austria’s 
knowledge-based society, and to creating value and wealth in Austria. Specifically, through 
the funding of top-quality projects and by enhancing the competitiveness of research facilities, 
the  FWF  aims  to  strengthen  Austria's  international  performance  in  science  and  research. 
Based  on  the  principle  of  research-driven  education,  the  FWF  seeks  to  develop  Austria's 
human resources for science and research. In addition to this, it aims to stimulate interactions 
between science and research and the economy and society. In 2021, the Austrian federal 
government provided EUR 150 million to the FWF for a new programme, excellent=austria, 
which aims to help universities and other research institution to further develop their research 
strengths.  

The  excellent=austria  programme  is  a  suitable  benchmark  initiative  for  multiple  reasons. 
Given the novelty of the programme, its rationale, intentions and design and their links with 
technological, scientific, socio-economic developments and emerging needs is scrutinised in 
the  present  report to  inform  evaluation  questions  in  relation  to  relevance.  Meanwhile,  the 
coherence (potential complementarities, overlaps, and synergies) between this programme 
and the FP will be assessed. The FWF’s ‘Clusters of Excellence’ action enables groups of 
researchers,  led  by  a  Director  of  Research,  to  achieve  outstanding  cooperative  research 
results (over a period of 60 months). Thus, the design and implementation of this programme 
can provide some first lessons in terms of efficiency.  

The lessons drawn in this benchmark report are applicable for the assessment of Excellent 
Science-related aspects of the Framework Programme, particularly with regard to the ERC 
and WIDERA programme parts.  

1.2.  Methodology and evaluation questions addressed 

The  present  benchmark  study  relies  on  extensive  desk  research.  The  FWF  Austria  and 
excellent=austria websites, documents, evaluation reports and annual reports, programme 
guides and calls are assessed, along with the FWF dashboard providing information on the 
objectives, functioning, participation and results of the research funding. As with the other 
benchmarks, documents from the Framework Programme are also examined to assess the 
transferability of findings. 

The study aimed to complement the above desk research using qualitative evidence gained 
through interviews with FWF Austria stakeholders. Indeed, interviews with representatives of 
the  excellent=austria  programme  were  an  important  source  of  information  regarding  the 
rationale and first lessons learned from these new programmes. However, the availability of 
interviewees is an important limitation of this case study. Given that the programme is in an 
early phase of its implementation (one pillar just recently confirmed its first round of funded 
projects, the second is the in the process of evaluation, and the third is yet to be launched), 
it  was  only  possible  to  interview  FWF  officials  and  not  beneficiaries  of  the  programme. 

1232 Fonds zur Förderung der wissenschaftlichen Forschung, FWF. 

768 

 
 
Nevertheless,  interviews  with  the  top-level  executives  allowed  the  study  team  to  extract 
important information regarding the rationale and expected impacts of the programme.  

Table 1 below presents a list of the evaluation questions addressed in this benchmark report 
and the key criteria compiled for each of these questions. 

Evaluation 
questions 

Judgement criteria 

RV1, RV2, RV4, 
RV7 

•  Rationale for and balance of the excellent=austria portfolio. 
•  Extent to which excellent=austria can meet the needs of target groups 

in terms of activities and thematic areas. 

CH1, CH2 

•  Coherence with 1) other programme parts of FWF Austria; 2) other 

national, regional and international initiatives; and 3) EU programmes 
(including Horizon Europe). 

•  Positioning of FWF Austria and excellent=austria within the European 

and international R&I context. 

EFF1, EFF2, EFF3, 
EFF7, EFF11, 
EFF12 

•  Rationale for and design of the implementation of the funding 

(administration and management, project application and selection 
processes, funding allocation). 

•  Strengths and weaknesses of the funding allocation and 

implementation. 

•  Perceptions and satisfaction regarding the application and selection 

processes and implementation. 

•  Key factors that influence the progress (enablers, drivers) or lack of 
progress (barriers) of the programme towards its objectives and 
impact. 

1.3.  Context and objectives of the benchmark initiative 

Table 1. Evaluation criteria 

1.3.1.  Austrian R&I policy context 

The  Austrian  innovation  system  relies  heavily  on  its  robust  research  funding,  which  is 
allocated on the basis of current key indicators and strategic development plans at federal 
level. Austrian  R&I  policy is guided  by  the  Austrian  Research,  Technology  and  Innovation 
(RTI) Strategy, which provides direction for the federal government  – particularly for those 
ministries whose areas of responsibility include research, technology and innovation1233. The 
first  Austrian  RTI  strategy  ran  from  2011  to  2020.  A  second  RTI  strategy  was  adopted  in 
2021,  with  three  overarching  objectives  (further  discussed  in  Section  1.4.1).  The 
implementation of the RTI strategy is led by a widespread and firmly established culture of 
evaluation1234. The Austrian research landscape is characterised by 22 public universities, 21 
universities  of  applied  sciences,  five  central  non-university  research  institutions,  3,489 
research companies and a total of 5,084 research performing units, including many leading 

1233 RTI Strategy 2030 Strategy for Research, Technology and Innovation of the Austrian 
Federal Government,2020. https://era.gv.at/public/documents/4489/RTI_Strategy_2030-1-1.pdf  
1234 Most evaluation reports are available to the public through the online repository of the 
Austrian Platform for Research and Technology Policy Evaluation (fteval).  

769 

 
 
international  companies1235.  Funding  is  primarily  managed  by  three  major  agencies:  the 
Austrian  Science  Fund  (FWF),  the  Austrian  Research  Promotion  Agency  (FFG),  and  the 
Austria  Wirtschaftsservice  (AWS).  The  FWF  (discussed  in  further  sections)  funds  basic 
research; the FFG focuses on funding applied research associated with industry, and AWS 
is the federal development bank in Austria for business-related economic development. 

Based on the 2023 Statistics Austria estimate12361237, EUR 15.5 billion will be spent in Austria 
in 2023 on research and development (R&D). This means that the country’ research intensity, 
i.e. R&D expenditure as a share of nominal gross domestic product (GDP) will be 3.22%. s 
an RTI location, Austria has shown dynamic development over recent decades. For example, 
Austria’s R&D intensity in 1993 was 1.45%; by 2003 it had risen to 2.17%1238. With its current 
ratio of R&D expenditure to GDP, Austria exceeds the 3% target set by the European Union, 
and has the third-highest R&D intensity in the EU, surpassed only by Sweden (3.35%) and 
Belgium  (3.22%)1239.  In  terms  of  global  comparisons,  according  to  the  Austrian 
Research and Technology Report (2022), Austria is also ahead of highly innovative 
countries such as the USA and China.1240 The same report indicates that Austria has 
made  a  significant  progress  in  terms  of  core  scientific  indicators  (e.g.  citation  rate  and 
international patent applications). However, it is important to note that most funding in Austria 
goes towards industrial research and not basic research.  

1.3.2.  Austrian participation in the European Framework Programmes 

through 

the  EU Framework  Programmes 

Along  with  this  national  research  funding,  there  is  also  international  research  aid,  most 
notably 
for  Research  and  Technological 
Development  (i.e.  Horizon  2020,  and  now  Horizon  Europe).  Austrian  applicants  to  and 
participants  in  the  EU  Framework  Programmes  for  Research  enjoy  support  from  1)  the 
Austrian  Research  Promotion  Agency,  FFG  (which  informs  and  supports  business 
enterprises, universities, independent research institutes and other organisations interested 
in  international  collaborations);  2)  the  regional  agencies,  which  provide  on-site  advice  in 
collaboration  and  consultation  FFG;  and  3)  advisory  units  within  the  institutions  (e.g. 
excellence-related  measures  at  Austrian  universities  were  promoted  through  performance 
agreements, the establishment of Christian Doppler laboratories as incentive systems, and 
newly developed third-party funding strategies).  

Under  Horizon  2020,  Austrian  RTI  organisations  reached  a  total  of  more  than  EUR 1.95 
billion in net funding acquired from the European Commission, and signed 3,426 grants  – 
39.6% with higher education institutions (HEIs), 30% with Austrian companies, and 24.8% 
with non-university research institutions1241. The success rate of Austrian applicants under 
Horizon 2020 was among the highest in the EU, amounting to 17.31% (compared with the 
Horizon  2020  average  of  15.30%).  Austria’s  performance  in  the  activities  of  European 

1235 RTI Strategy 2030 Strategy for Research, Technology and Innovation of the Austrian 
Federal Government,2020. https://era.gv.at/public/documents/4489/RTI_Strategy_2030-1-1.pdf  
1236 In accordance with the R&D Statistics Ordinance, Federal Law Gazette II/396/2003 as 
amended, Statistics Austria conducts a mandatory full survey every two years concerning the 
funding and conduct of research and experimental development (R&D) in all economic sectors in 
Austria. 
1237 Statistics Austria, Press release 13 050-078/23, 2023, R&D expenditure rise to around €15.5 
billion. https://www.statistik.at/fileadmin/announcement/2023/04/20230421FuE2023EN.pdf  
1238 Ibid. 
1239 Based on data the latest data currently available, covering the year 2021. 
1240 Austrian Research and Technology Report, 2022. 
https://www.bmbwf.gv.at/en/Topics/Research/Research-in-Austria/Services/FTB.html  
1241 Horizon Dashboard: R&I Country Profile Austria. Data extracted 18 July 2023. 

770 

 
 
Research  Council  (ERC)  was  particularly  successful,  with  4.5  grants  per  1  million 
inhabitants1242. In relation to Horizon Europe, the Austrian RTI Strategy 2030 stipulates the 
aim  of  increasing  participation  in  European  partnerships  by  following  a  more  strategic 
approach  at  national  level  and  by  supporting  participation  with  an  efficiently  coordinated 
portfolio  of  funding.  Even  though  it  is  too  early  to  draw  conclusions  as  to  the  success  of 
Austrian participation in Horizon Europe, Austrian participants have already been awarded 
EUR  694.6  million  in  net  EU  contributions  and  signed  931  grants,  with  a  success  rate  of 
21.77%1243. This reflects not only the high level of Austrian research, but also the efficiency 
of its advisory, support and supervision structures.  

1.3.3.  The Austrian Science Fund (FWF) 

Established in 1968, the Austrian Science Fund is the central funding organisation for basic 
research as well as artistic-scientific research in Austria. The Austrian Science Fund (FWF) 
is  managed  by  the  Federal  Ministry  of  Education,  Science  and  Research  via  a  multi-year 
programme and through the approval of the annual work programmes. The purpose of the 
FWF is ‘to support the ongoing development of Austrian science and basic research at a high 
international level’1244. The fund commits itself to the principles of transparency and fairness, 
gender  equality  mainstreaming  and  equal  opportunities  –  in  addition  to  the  principles  of 
excellence  and  competition,  independence  and  internationality.  The  above  purposes  are 
achieved through three key interrelated objectives: 

1.  To strengthen Austria’s international performance and capabilities in science 
and  research,  as  well  as  the  country’s  attractiveness  as  a  location  for  high-level 
scientific activities, primarily by funding top-quality research projects for individuals 
and teams and by enhancing the competitiveness of Austria's innovation system and 
its research facilities; 

2.  To  develop  Austria’s  human  resources  for  science  and  research  in  both 
qualitative  and  quantitative  terms,  based  on  the  principle  of  research-driven 
education; 

3.  To emphasise and enhance the interactive effects of science and research with 
all other areas of culture, the economy and society, and in particular to increase 
the acceptance of science and research through concerted public relations activities. 

FWF Austria allocates funding through a variety of funding schemes (see  Table 2 below), 
majority of which are ‘bottom-up’ (i.e. not programme-specific, and driven by the curiosity of 
the  research  community).  Through  these  measures,  the  FWF  supports  researchers  and 
ideas that are pioneering by virtue of their scientific quality, and which lay a broad foundation 
for  improving  the  ability  to  tackle  the  future  challenges  facing  society1245.  Projects  are 
selected  through  an  international  peer  review  process.  To  arrive  at  a  budget  across  all 
disciplines,  the  number  of  applications  and  number  of  proposals  granted  in  the  past  is 
used1246. 

1242 Factsheet: Austrian Research and Technology Report, 2022. 
https://www.bmbwf.gv.at/en/Topics/Research/Research-in-Austria/Services/FTB.html  
1243 Horizon Dashboard: R&I Country Profile Austria. Data extracted 18 July 2023. 
1244 Corporate website of the FWF Austrian Science Fund, About the FWF, 2023, accessed on 
17 July 2023. https://www.fwf.ac.at/ueber-uns/leitbild 
1245 Austrian Research and Technology Report, 2022. 
https://www.bmbwf.gv.at/en/Topics/Research/Research-in-Austria/Services/FTB.html 
1246 Janger, J., Schmidt, N., & Strauss, A. International differences in basic research grant 
funding – a systematic comparison,. Austrian Institute of Economic Research, 2019. 

771 

 
 
Programmes 

Short description / objectives 

excellent=austria 

Clusters of Excellence 

Emerging fields 

FWF Distinguished Professor 

Established to strengthen Austria’s position as a 
top location for research within the international 
scientific community (discussed in detail in the 
forthcoming sections). 

Projects 

Principal investigator projects 

Principal investigator projects (international) 

Project funding provides researchers with the 
freedom and flexibility to carry out pioneering 
projects in all areas of basic research. In addition 
to national projects, tailored funding is available for 
bi- or trilateral projects.   

Careers 

Career funding opportunities support researchers 
at different career stages, from PhD to postdoc 
and beyond. These give researchers the chance to 
gain a foothold in cutting-edge research, both in 
Austria and abroad.  

Clinical research 

Clinical research (international) 

Arts-based research 

FWF Wittgenstein Award 

ESPRIT 

Erwin Schrödinger 

Elise Richter 

Elise-Richter-PEEK 

START 

doc.funds 

doc.funds.connect 

772 

 
Programmes 

Short description / objectives 

Collaborations 

Special research areas 

Research groups 

International (multilateral activities) 

International (joint seminars) 

Top Citizen Science 

Science communication 

Book publications 

Digital publications 

Academic journals 

Peer-reviewed publications 

These funding programmes are designed to 
expand and strengthen cooperative research 
within teams of all sizes, across institutional and 
regional borders.  

Communications 

Disseminating research findings and entering into 
a discourse with the public – funding is available 
for citizen science projects, science 
communication, and the publication of research 
results. 

Table 2. FWF portfolio of funding programmes. Source: FWF Annual Report, 2022. 

In  addition  to  its  thematically  open  programmes,  the  FWF  offers  targeted  funding 
opportunities  such  as  the  Quantum  Austria  Funding  Initiative,  AI  Mission  Austria,  ASMET 
Research  Award,  Belmont  Forum,  EJP  Rare  Diseases,  ERA-NET  CHIST-ERA,  ERA-NET 
ERA  PerMed,  ERA-NET  HERA,  ERA-NET  NORFACE,  ERA-NET  QuantERA,  ERA-NET 
TRANSCAN, Alternative Methods to Animal Testing, the European Biodiversity Partnership 
(Biodiversa+), the European ERA4Health Partnership, the European Water4All Partnership, 
the  Gottfried  and  Vera  Weiss  Award,  netidee  SCIENCE,  Herzfelder-Stiftung  Projects, 
Quantum  Austria,  the  Rückenwind  funding  bonus,  and  Sustainable  Food  Systems. 
Researchers  can  also  apply  for  funding  for  international  research  projects.  Thanks  to 
numerous  agreements  with  partner  organisations  across  Europe  and  beyond  (including 
Belgium/Flanders, the Czech Republic, France, Germany, Hungary, Italy/South Tyrol, Japan, 
Luxembourg,  Poland,  Slovenia,  Switzerland  and  Taiwan),  researchers  can  choose  from  a 
wide range of cooperating countries. 

1.3.4.  Participation patterns in FWF Austria 

Even  though  the  main  focus  of  this  case  study  is  on  the  excellent=austria  initiative,  this 
section  looks  at  participation  patterns  across  the  whole  of  FWF  Austria.  This  is  primarily 
influenced by a lack of data regarding participation patterns in excellent=austria (i.e. at the 

773 

 
time of writing, only the Clusters of Excellence pillar has been fully launched, with the FWF 
Distinguished Professor pillar set to be launched in early 2024 and the funding decisions for 
Emerging  fields  pillar  to  be  published  in  spring  2024).  However,  providing  an  overview  of 
participation patterns across the whole programme allows an understanding of the research 
community served by both FWF Austria and excellent=austria. The information presented in 
this  section  is  based  on  FWF  Austria  Annual  Reports  (2018-20221247)  and  the  FWF-
Dashboard1248.  

Table 3 and Figure 1 provide an overview of the various programme indicators, including the 
number  of  applications,  projects  funded,  amounts  of  funding  requested  vs  granted,  and 
approval rates. 

The data in Table 3 and Figure 1 show increases in the amounts of funding requested and 
the numbers of applications submitted over the years. However, according to the FWF’s 2022 
Annual  Report,  in  spite  of  excellent  reviews,  too  many  researchers  were  prevented  from 
implementing their projects at Austrian research institutions due to a limited funding budget 
in addition to the effects of high inflation. This need for funding will remain a constant in the 
future as the country’s standing as a location for research continues to grow. 

Applications 

Approvals 

Drafts and 
proposals 

Approved 
projects 

Approval 
rate 

2018 

2,501 

2019 

2,489 

2020 

2,980 

2021 

3,316 

2022 

2,783 

684 

707 

708 

732 

743 

2,468 

2,483 

2,953 

3,339 

2,751 

658 

666 

680 

732 

689 

26.7% 

26.8% 

23.0% 

21.9% 

25.0% 

Table 3. Data on FWF Austria applications and approvals, 2018-2022. Source: FWF-Dashboard data retrieved 18 July 2023. 

1247 Accessed via the Publications section of the FWF Austria corporate website on 18 July 2023. 
1248 FWF-Dashboard data retrieved 18 July 2023. https://dashboard.fwf.ac.at/   

774 

 
 
 
Figure 1. Amounts requested vs amounts granted under FWS. Note: blue line = total amounts requested; green line = total 
amounts granted. Source: FWF-Dashboard data retrieved 18 July 2023. 

Viewed through the lens of research disciplines (see Table 4), the portfolio of FWF Austria 
for the period 2018-2022 is led by Natural Sciences and Engineering (41.5%), followed by 
Biology  and  Medical  Sciences  (34.8%),  with  the  lowest  share  of  projects  approved  being 
attributed to Social Sciences and Humanities (23.4%). 

According to the 2022 Annual Report, the FWF approved funding for 463 applications in the 
Projects category, 147 applications in Careers, 125 applications in Collaborations, and eight 
applications in Communications. Approximately 4,800 researchers were funded by the FWF 
in 2020. Out of these participants, 52.2% were male, 47.7% were female, and 0.1% were 
non-binary. Another positive trend is the high share of funding given to junior researchers (35 
years  or  younger)  in  recent  years,  reaching  a  share  of  70%  in  2022.  These  statistics 
emphasise  the  important  role  the  FWF  plays  in  the  development  of  professional  research 
staff in Austria. 

Research discipline 

Applications 

Approvals 

Approval 
rate 

Number 

Share 

Number 

Share 

Biology and Medical Sciences 

4,886 

35% 

1,256 

34.8% 

23.9% 

Natural Sciences and 
Engineering 

5,610 

40% 

1,497 

41.5% 

25.7% 

Social Science and Humanities 

3,572 

25% 

852 

23.4% 

23.4% 

Table 4. Application data averaged across research disciplines, 2018-2022. Source: FWF-Dashboard data retrieved on 18 
July 2023. 

775 

 
 
1.4.  Results 

The main focus of this case study is the excellent=austria programme (‘the FWF’s excellence 
initiative),  which  is  tasked  with  reinforcing  areas  of  strength  and  developing  new  and 
innovative fields of research. The sections that follow focus on relevance of the three pillars 
that  make  up  this  programme  in  the  context  of  Austrian  R&I,  their  coherence  with  other 
national and international instruments (including Horizon Europe), and the efficiency of the 
programme (to the extent that is possible, given the implementation status of the pillars).  

1.4.1.  Relevance of the excellent=austria programme 

The  relevance  of  the  excellent=austria  programme  is  established  under  the  Austrian  RTI 
Strategy  20301249. This  was  adopted  on  23  December  2020,  and  created  a  new  strategic 
framework for Austrian research policy. The objectives of this strategy were defined following 
an  analysis  of  Austria’s  strengths  and  weaknesses  in international  rankings.  This  analysis 
placed  the  country  in  a  good  upper-mid  position  globally  –  but  not  on  a  par  with  the  top 
performers.  The  objectives  of  the  strategy  were  also  informed  by  the  expertise  of  a  wide 
range  of  stakeholders,  OECD  reviews,  the  European  Commission’s  concept  of  Smart 
Specialisation,  and  other  cross-cutting  issues  such  as  the  UN  Sustainable  Development 
Goals  (SDGs).  The  RTI  Strategy  2030  defined  strategic  fields  of  activity  that  are  geared 
towards  the  following  overarching  goals,  which  focus  on  outputs  as  well  as  impacts, 
excellence  and  openness:  1)  Become  an  international  innovation  leader  and  strengthen 
Austria  as  an  RTI  location;  2)  Focus  on  effectiveness  and  excellence;  and  3)  Focus  on 
knowledge, talent and skills. 

The objective of establishing Austria as an innovation leader has also given rise to a wide 
range  of  sub-strategies  with  relevance  to  RTI,  including  the  Excellence  Initiative1250.  The 
OECD Review of Austrian Innovation Policy 20181251 provided the key input for formulating 
the necessary adjustments to the R&I policy context in terms of excellence. More specifically, 
the  OECD  recommended  introducing  an  excellence  initiative  and  strengthening  the 
competitive  component  of  basic  research  funding  through  an  increase  in  FWF  budget  for 
existing formats, and innovations in the programme portfolio. In addition, the OECD review 
recommended  taking  action  with  regard  to  established  researchers,  as  well  as  career 
development opportunities for young researchers.   

Following  these  developments,  in  2021  the  FWF  –  in  partnership  with  Austria’s  Federal 
Ministry of Education, Science and Research (BMBWF) – launched an excellence initiative 
entitled  ‘excellent=austria’.  Under  the  excellent=austria  programme,  the  Austrian  federal 
government provided the Austrian Science Fund (FWF) with EUR 150 million in funding over 
the  next  three  years  to  support  universities  and  other  research  institutions  in  further 
developing their research strengths. The goals of the excellent=austria programme are: 1) to 
support  outstanding  basic  research  on  open  topics,  in  line  with  the  highest  international 
standards, and with plenty of freedom for unconventional approaches; 2) to promote gender 
equality  and  diversity  to  a  greater  extent,  and  to  create  attractive  career  opportunities  for 
long-term  collaborations  (national  and 
excellent 
international)  to  leverage  synergies;  4)  to  strengthen  Austrian  universities,  universities  of 

junior  researchers;  3) 

to  expand 

1249 RTI Strategy 2030 Strategy for Research, Technology and Innovation of the Austrian 
Federal Government, 2020. https://era.gv.at/public/documents/4489/RTI_Strategy_2030-1-1.pdf 
1250 Other sub-strategies include Location Strategy 2040, Digital Action Plan, AI Strategy 2030, 
Austrian Space Strategy 2030+, R&I Mobility Agenda 2026, FIT4UrbanMission, and Circular 
Economy. 
1251 OECD Reviews of Innovation Policy, Austria 2018 https://www.oecd-ilibrary.org/science-and-
technology/oecd-reviews-of-innovation-policy-austria-2018_9789264309470-en 

776 

 
 
applied  sciences,  and  non-university  research  institutions  in  terms  of  their  international 
competitiveness; 5) to enhance the international reputation of Austrian research institutions; 
and 6) to boost the transfer of research results into business and society. 

The  programme  promotes  bottom-up,  cutting-edge  research  in  line  with  the  highest 
international standards, while leaving scope for unconventional approaches. The programme 
is distributed across three pillars, to be launched over different timeframes (see Table 5).  

Pillar 

Clusters of Excellence 

Emerging Fields 

Objectives 

Strengthening 
outstanding research 
areas by means of 
collaborations across 
institutions, disciplines 
and national borders. 

Identifying and 
supporting high-
potential, forward-
looking research 
topics. 

FWF Distinguished 
Professor 

Recruiting and 
appointing renowned 
researchers at Austria’s 
universities. 

Implementation 
status 

Launched, and funding 
decision published.  

Call has been 
launched. Results will 
be published in early 
2024. 

Scheduled to be 
launched in early 2024. 

Table 5. Application data averaged across research disciplines, 2018-2022. Source: FWF corporate website 
excellent=austria, 2023. Accessed 19 July 2022. 

Based on this structure, the excellence initiative funds large research networks (Clusters of 
Excellence),  smaller  inter-institutional  teams  (Emerging  Fields)  and  the  recruitment  to 
Austrian universities of highly qualified professors (FWF Distinguished Professor).  

1.4.1.1. 

Clusters of Excellence 

The Clusters of Excellence (CoE) form the first pillar of the excellent=austria funding initiative, 
aiming  at  strengthening  Austria’s  position  as  a  leading  research  destination  globally.  By 
transcending the limitations of the existing programme portfolio of the Austrian Science Fund 
(FWF),  these  clusters  facilitate  extensive  collaborative  research  to  achieve  exceptional 
outcomes,  and  establish  Austria  as  a  prominent  global  research  hub.  The  CoE  follow  a 
bottom-up approach, allowing researchers from Austrian institutions to achieve outstanding 
cooperative  results  within  a  specific  field  or  through  interdisciplinary  efforts.  By  firmly 
establishing these research areas, the CoE play a pivotal role in shaping and strengthening 
core research domains at participating institutions, while also providing exceptional research 
training, career development and education for young researchers. 

The initial call1252 for the CoE offered up to EUR 70 million in funding over 10 years for large-
scale basic research projects. The CoE annual programme budget ranges between EUR 2 
and EUR 7 million, with 60% funded by the FWF and the remaining 40% provided by the lead 
and collaborating institutions. The clusters will operate for five years initially, with the potential 
for a five-year extension based on positive evaluations. Structurally, the CoE are composed 
of  a  lead  research  institution  and  a  minimum  of  two  collaborating  institutions  (up  to  a 
maximum of seven). The Director of Research – an outstanding researcher from the leading 
institution – serves as the coordinator, and manages the research programme. Coordination 

1252 Application Guidelines Letter of Intent Clusters of Excellence, 2021. 

777 

 
 
is supported by a Board of Directors (BoD) representing collaborating institutions, and may 
include individuals with non-research specific expertise. Key researchers conduct research 
under the guidance of the Director of Research and the BoD. All CoE encompass four distinct 
units: 1) the research unit, led by the Director of Research; 2) the training unit, which focuses 
on  advancing  the  careers  of  young  researchers;  3)  the  communication  and  transfer  unit, 
which promotes knowledge exchange with stakeholders; and 4) the management unit, which 
ensures optimal organisational and financial management of the CoE. 

The final funding decision for the initial call was made by the FWF Board at the end of March 
2023, on the basis of hearings before an international jury. A total of 56 letters of intent were 
received;  35  consortia  submitted  proposals;  11  teams  were  shortlisted;  and  five  clusters 
ultimately received funding. The FWF and the participating research institutions will provide 
a total investment of EUR 135 million for the first five years1253. The five clusters chosen will 
advance basic research in the areas of energy storage, quantum technologies, global health, 
the future of knowledge and Eurasian cultural heritage (see Table 6 below for details). The 
activities of the projects were scheduled to launch in October 2023.  

Title 

Board of Directors and 
institutions 

Description 

Knowledge in 
Crisis 

Tim Crane (Director of Research, 
Central European University) 
Marian David (Universität Graz) 
Katalin Farkas (Central European 
University) 
Max Kölbel (Universität Wien) 
Hans Bernhard Schmid 
(Universität Wien) 
Paulina Sliwa (Universität Wien) 
Charlotte Werndl (Universität 
Salzburg) 

Quantum 
Science Austria 

Gregor Weihs (Director of 
Research, Universität Innsbruck) 
Markus Aspelmeyer (Universität 
Wien) 
Francesca Ferlaino 
(Österreichische Akademie der 
Wissenschaften) 
Johannes Fink (Institute of 
Science and Technology Austria) 
Armando Rastelli (Universität 
Linz) 
Oriol Romero-Isart (Universität 
Innsbruck) 
Hannes-Jörg Schmiedmayer 
(Technische Universität Wien) 

The overwhelming influx of information 
on the internet blurs the ability to 
discern truth from falsehood, while the 
rejection of scientific standards and 
expertise further compounds the issue. 
This crisis raises profound philosophical 
enquiries into knowledge, truth, 
science, ethics, politics and our 
fundamental relationship with reality. 
This Cluster of Excellence undertakes a 
novel and interdisciplinary exploration 
of these questions, aiming to 
comprehend the multifaceted 
knowledge crisis and to develop 
strategies to address and redefine our 
approach to knowledge. Total funding 
allocation: EUR 8.9 million 

Pioneering contributions from Austria 
have been instrumental in the 
development of the Second Quantum 
Revolution, leading to the advancement 
of modern quantum science and the 
emergence of quantum technologies 
that surpass classical counterparts. The 
Cluster of Excellence ‘Quantum 
Science Austria’ focuses on 
fundamental research to expand the 
knowledge and drive future innovations. 
Total funding allocation: EUR 21 
million. 

1253 FWF, Clusters of Excellence, 2023 (in German), accessed on 19 July 2023. 

778 

 
 
Title 

Board of Directors and 
institutions 

Description 

Materials in 
Energy for 
Conversion and 
Storage  

Günther Rupprechter (Director of 
Research, Technische Universität 
Wien) 
Ulrike Diebold (Technische 
Universität Wien) 
Stefan Freunberger (Institute of 
Science and Technology Austria) 
Leticia González (Universität 
Wien) 
Julia Kunze-Liebhäuser 
(Universität Innsbruck) 

Microbiomes 
Drive Planetary 
Health 

EurAsian 
Transformations 

Michael Wagner (Director of 
Research, Universität Wien) 
Andreas Bergthaler 
(Österreichische Akademie der 
Wissenschaften, CeMM) 
Christina Kaiser (Universität 
Wien) 
Bernhard Lendl (Technische 
Universität Wien) 
Christine Moissl-Eichinger 
(Medizinische Universität Graz) 
Alexander Moschen (Universität 
Linz) 
Leonid Sazanov (Institute of 
Science and Technology Austria) 
Angela Sessitsch (Austrian 
Institute of Technology) 

Claudia Rapp (Director of 
Research, Österreichische 
Akademie der Wissenschaften) 
Birgit Kellner (Österreichische 
Akademie der Wissenschaften) 
Tijana Krstić (Central European 
University) 
Melanie Malzahn (Universität 
Wien) 
Walter Pohl (Österreichische 
Akademie der Wissenschaften) 
Robert Rollinger (Universität 
Innsbruck) 
Jens Oliver Schmitt (Universität 
Wien) 

Central to this Cluster of Excellence is 
the development of nanocatalysts that 
facilitate swift and reliable chemical 
reactions. This requires the 
consolidation of expertise from diverse 
fields such as surface chemistry, 
surface physics, materials science, and 
computer simulation. Through this 
collaborative effort, the Cluster of 
Excellence is taking a significant stride 
towards achieving climate neutrality, 
offering a promising solution in the 
global fight against climate change. 
Total funding allocation: EUR 20.6 
million. 

Microbiomes play a vital role in the well-
being of our planet and its inhabitants. 
Within this Cluster of Excellence, a 
diverse team of 30 scientists across 
multiple disciplines and seven research 
sites collaboratively strive to unravel the 
fundamental principles governing 
environmental microbiomes and human 
microbiomes. By comprehending the 
intricate workings of microbiomes, the 
ultimate objective is to decipher how 
they influence the overall health of our 
planet. Total funding allocation: EUR 21 
million. 

The ‘Eurasian Miracle’, spanning over 
three millennia from Central Europe to 
Asia, encompasses historical 
transformations with lasting impacts on 
the modern world. Preserved in diverse 
languages and writings, historical 
sources from this period provide a rich 
cultural heritage that awaits thorough 
analysis. In this endeavour, 31 Austrian 
scholars aim to foster the development 
of a new generation of researchers 
equipped with comprehensive 
competencies in research-oriented 
teaching. Total funding allocation: 
EUR 9.2 million. 

Table 6. Clusters of Excellence so far launched. Source: FWF, Clusters of Excellence, 2023 (in German), accessed 19 July 
2023. 

779 

 
1.4.1.2. 

Emerging Fields 

The second pillar of the excellent=austria programme is the establishment of the ‘Emerging 
Fields’  programme,  with  a  mission  to  ramp  up  basic  research  in  fields  with  particularly 
significant  potential  for  the  future.  The  objectives  of  this  programme  are:  1)  to  give 
researchers the opportunity to pursue particularly innovative, original or high-risk ideas that 
are often not sufficiently supported through traditional funding instruments; 2) to challenge 
fundamental  elements  of  traditional  models  or  approaches  in  individual  disciplines;  3)  to 
combine theories and methods from different disciplines; 4) to harness new technologies in 
order to identify new problems or look at old ones from a new perspective; and 5) to fund 
research that has the potential to transform a field of research and/or create a paradigm shift 
within or between disciplines. The implementation of the programme is expected to enable 
Austrian research institutions to serve as international pioneers, and to strengthen the setting 
of priorities in the Austrian research area in the long term.  

The call for proposals was launched in 2022, and decisions on funding will be made in early 
2024. The first call1254 invites researchers from all areas of basic research to form teams 
of 3-7 outstanding researchers, led by a coordinator. Projects must consist of staff at one 
or more Austrian research institutions and, if applicable, national or international cooperation 
partners. In particular, the programme is open to interdisciplinary teams and to researchers 
undertaking  arts-based  research  who  apply  aesthetic  and  artistic  methods,  as  well  as  to 
transdisciplinary  approaches  that  allow  researchers  to  involve  non-academic  participants 
from  outside  the  scientific  community.  An  interim  evaluation  will  be  conducted  after  three 
years to decide on the continuation of a project, and the research work can then, if necessary, 
be adapted to any changes in the research environment. The total amount of funds available 
for the first call is EUR 24 million. For individual teams, up to EUR 6 million will be available 
(100% funding by FWF, no co-funding by institutions). The FWF plans to issue a new call for 
Emerging Fields proposals every two years.  

1.4.1.3. 

FWF Distinguished Professor 

The third pillar of the programme, the FWF Distinguished Professor, is yet to be launched, 
but  plans  to  offer  universities  and  other  research  facilities  more  opportunities  to  support 
excellent researchers in all scientific disciplines, to retain them in Austria and to secure their 
services for the Austrian research system (with a focus on individuals, not research teams). 
A bottom-up approach will lead the selection process, with no requirement for co-financing 
by institutions. During the interviews with FWF officials conducted for this study, it was noted 
that  only  formal  government-recognised  universities  will  be  eligible  for  funding,  with  non-
university  research  institutions  being  excluded.  The  funding  tool  will  allow  institutions  to 
receive reimbursements for attracting high-class researchers (including recruitment fees) in 
addition to the budget allocated for actual research efforts.  

1.4.2.  Coherence of the excellent=austria programme 

Alignment with other Austrian R&I programmes (broader ecosystem). Implementation 
of  the  Austrian  RTI  Strategy  is  directly  linked  to  the  country’s  Research  Financing  Act 
(FoFinaG1255),  which  requires  the  federal  government  to  agree  on  a  Pact  on  Research, 
Technology and Innovation every three years. These Pacts provide financing and planning 
security  to  facilitate  growth,  and  contain  measures  explicitly  designed  to  achieve  the 

1254 Emerging fields, Application guidelines, 2022. 
1255 Forschungsfinanzierungsnovelle 2020 NR: GP XXVII RV 239 AB 308 S. 43. BR: AB 10406 
S. 911 (In German). 
https://www.ris.bka.gv.at/Dokumente/BgblAuth/BGBLA_2020_I_75/BGBLA_2020_I_75.html  

780 

 
 
objectives of the RTI Strategy and to pursue the fields of activity specified within it. In the 
case of the FWF, the three-year financing agreement provided the Fund with several years 
of planning security for the first time.1256,1257 The priorities defined in the RTI Pacts are derived 
from  regular  evaluation  of  the  measures  implemented  and  progress  towards  the  three 
objectives. The close alignment between the Austrian Science Fund (FWF) and the RTI Pacts 
demonstrates  the  FWF’s  commitment  to  coherence  with  national  research  and  innovation 
policies. By directly linking the implementation of the Fund’s strategy to the FoFinaG and the 
RTI  Pacts,  the  FWF  ensures  that  its  efforts  are  in  sync  with  broader  national  goals  and 
priorities.  This  alignment  enhances  coordination  and  collaboration  between  the  FWF  and 
other  stakeholders  in  the  research  and  innovation  ecosystem,  fostering  a  coherent  and 
integrated approach to advancing scientific progress and societal impact in Austria. 

Synergies  with  other  FWF  projects.  The  FWF  recognises  the  importance  of  facilitating 
researchers with ongoing cooperative projects to participate in the preparation of Clusters of 
Excellence  applications.  In  this  regard,  researchers  engaged  in  ongoing  projects  such  as 
special  research  programmes  (SFB),  research  groups  (FG),  doctoral  programmes  (DK), 
doc.funds, doc.funds.connect, research units (ZK), and #ConnectingMinds are encouraged 
to apply. However, provisions are in place to ensure that there is no duplication of funding or 
simultaneous submission of identical proposals. While a consortium is permitted to submit a 
proposal even if there is some overlap in content with an ongoing research group project, if 
the  proposal  is  approved,  the  research  team  must  be  terminated  prematurely  and  any 
remaining funds must be returned to the FWF before the CoE can commence. By allowing 
researchers involved in ongoing projects to transition into a CoE, the FWF promotes synergy 
and collaboration between different initiatives. This approach ensures the efficient allocation 
of  resources  and  fosters  the  integration  of  existing  research  efforts  into  a  larger,  more 
comprehensive CoE framework. Nevertheless, interviews with FWF officials reveal that the 
research  community  has  not  yet  taken  advantage  of  such  opportunities  in  the  application 
process.  

Synergies  with  Horizon  Europe  and  future  Framework  Programmes.  No  formal  links 
exist between the excellent=austria initiative and the European Framework Programmes (i.e. 
no  direct  measures  or  promotional  aspects).  However,  according  to  the  FWF  officials 
interviewed, links do arise as a natural consequence of researchers looking for other funding 
opportunities that focus on excellence. According to the Austrian Research and Technology 
Report  20191258,  the  country’s  success  in  the  Framework  Programmes  should  also  be 
attributed to strong national research funding in competitive fields (e.g. 80% of researchers 
with an ERC Advanced Grant under Horizon 2020 had previously received funding from the 
Austrian Science Fund). It is important to note that other programmes in the FWF portfolio 
have  more  explicit  links.  For  example,  when  applying  for  funding  under  the  START 
programmes,  applicants  are  also  required  to  submit  their  application  for  an  ERC  Starting 
Grant.  

1256 Research and technology report, 2022. 
https://www.bmbwf.gv.at/en/Topics/Research/Research-in-Austria/Services/FTB.html  
1257 A total of EUR 806 million for pioneering basic research represents a 27% increase in the 
funding budget compared with the previous three years. 
1258 Austrian Research and Technology Report 2019. 
https://www.bmbwf.gv.at/en/Topics/Research/Research-in-Austria/Services/FTB.html  

781 

 
 
1.4.3.  Efficiency of excellent=austria programme  

1.4.3.1. 

Communication and stakeholder engagement 

The  launch  of  the  excellent=austria  programme  attracted  significant  attention  within  the 
research  community,  primarily  due  to  the  unprecedented  amount  of  funding  offered.  To 
effectively  communicate  the  initiative  and  the  funding  opportunities  available,  the  FWF 
employed a range of conventional communication tools, using its website as a central hub 
for the dissemination of information. The website served as a comprehensive and accessible 
platform  at  which  researchers  and  institutions  could  find  detailed  information  about  the 
excellent=austria programme. Key elements such as the programme’s objectives, application 
guidelines, evaluation criteria and submission deadlines were clearly outlined. This ensured 
that  potential  applicants  were  well  informed  with  regard  to  the  funding  framework  and  the 
process  of  submitting  proposals.  Another  important  communication  channel  was  the 
Assembly  of  Delegates  of  the  FWF.  Consisting  of  university  representatives  and  non-
university research organisations closely linked to the academic landscape, the Assembly of 
Delegates  provides  a  direct  line  of  communication  between  the  FWF  and  the  research 
community.  

In  terms  of  community  engagement,  the  FWF  plays  a  very  active  role  in  the  proposal 
submission process. After the submission of letters of intent under the Clusters of Excellence 
programme,  the  FWF  invested  significant  resources  in  ensuring  that  organisations 
collaborated  with  each  other  –  in  other  words,  instead  of  a  few  mediocre  proposals being 
received, applicants were encouraged to find partners working in similar fields and to submit 
the  proposals  together.  As  a  result,  strong  proposals  were  prepared  in  collaboration  with 
partners with whom teams did not have strong links beforehand. According to the interview 
programme,  researchers  acknowledged  the  value  of  the  FWF’s  involvement  in  bringing 
together diverse expertise and encouraging the formation of partnerships. The collaborative 
nature of such proposals allowed the inclusion of complementary expertise and perspectives, 
ultimately  leading  to  more  impactful  and  innovative  research  initiatives.  This  proactive 
approach to promoting collaboration and partnership formation highlights the efficiency of the 
FWF’s efforts in maximising the quality and potential impact of research proposals within the 
Clusters of Excellence programme.  

1.4.3.2. 

Evaluation and monitoring 

The evaluation process is fundamental to the work of the FWF, serving as the cornerstone of 
quality assurance in all its funding programmes. The key document in this regard is the FWF 
Rules  on  Quality  and  Transparency  for  Evaluations1259.  At  all  stages  of  the  evaluation 
process, the FWF relies on a finely tuned system to avoid conflicts of interest and to ensure 
that all operations are checked. Since the establishment of the FWF, international peer review 
has  been  the  chosen  instrument  for  conducting  evaluations.  Esteemed  and  qualified 
scientists, considered the peers of applicants, are engaged to provide their expert opinions 
on  various  stages  of  a  project’s lifecycle,  including:  1) the  pre-funding  decision  evaluation 
(ex-ante); 2) project process and interim evaluation; and 3) the post-project evaluation (ex-
post).  The  sections  that  follow  will  discuss  the  specificities  of  each  step  in  the  evaluation 
process of the excellent=austria initiative.  

Pre-funding  decision  evaluation.  The  first  stage  of  evaluation  occurs  before  a  funding 
decision is made. When researchers submit applications for project support, these proposals 
undergo a rigorous evaluation by a panel of international experts. This peer review process 

1259 FWF Rules on Quality and Transparency Evaluations, Studies and Research Policy-related 
Services, accessed 25 July 2023. 

782 

 
 
ensures that the proposed projects meet high standards of scientific quality, innovation and 
relevance. Experts with relevant experience and qualifications closely related to the research 
field concerned assess the applications thoroughly, considering the feasibility, methodology 
and potential impact of the proposed research. 

•  The  first  selection  of  projects  under  the  Cluster  of  Excellence  programme  followed  a 
complex multi-stage process. First, consortia submitted letters of intent. Those consortia 
whose  letters  of  intent  met  the  pre-defined  criteria  were  invited  to  submit  CoE  pre-
proposals, subject to an international review according to quality criteria. Selected pre-
proposals were then invited to draft full proposals. The final funding decision was made 
by the FWF Board on the basis of hearing before an international jury. According to the 
interview  programme,  the  second  selection  of  the  projects  was  scheduled  for  late  in 
2023,  with  some  adjustments  based  on  a  forthcoming  evaluation  study  by  external 
consultants.  

•  The evaluation of the Emerging Fields programme is based on a multi-phase decision-
making  process  consisting  of  a  pre-selection  by  a  jury  of  experts,  written  reviews  by 
international reviewers, and a jury hearing. Each submission must include a three-page 
synopsis  and  a  full  proposal.  The  first  review  stage  focuses  solely  on  the  synopsis  to 
identify proposals with unique and/or high-risk perspectives. If a proposal is not selected, 
it may be submitted to other programmes within the FWF portfolio. During the second 
stage,  the  selected  proposals  are  evaluated  by  three  international  reviewers.  On  the 
basis  of  this  evaluation,  the  FWF  board  selects  a  shortlist  of  applicants  to  invite  to  a 
hearing (stage 3). Based on the jury’s suggestions, made after considering the results of 
these hearings, the FWF Board will decide on the approval of Emerging Fields proposals 
in early 2024. 

It  is  important  to  note  that  the  composition  of  juries is  established  with  respect  to  specific 
programmes  (the  same  applies  for  the  Clusters  of  Excellence  and  Emerging  Fields).  The 
juries consist of high-ranking international experts, based on their expertise in the relevant 
subject areas. The FWF’s peer-review process is highly international in nature: over the past 
decade and more, all referees’ reports have come from outside Austria, with one-third of them 
coming from non-German speaking countries in Europe, and a further one-third from further 
afield.  Furthermore,  the  FWF  attempts  to  ensure  that  at  least  25%  of  reports  come  from 
female referees1260.  

Project  progress  evaluation.  The  progress  of  the  excellent=austria  projects  is  monitored 
through  a  standard  FWF  procedure  involving  annual  reports  on  scientific  progress.  In 
addition, interim evaluations will be conducted to ensure ongoing quality assurance. During 
these  interim  assessments  (in  the  CoE  programme,  envisaged  after  five  years  of 
implementation), the progress and achievements of the projects will be carefully reviewed by 
an international expert jury. On the conclusion of successful interim evaluations, projects are 
expected to be extended by another five years. In this regard, the feedback received from 
expert evaluators provides valuable insights to consortia on how to fine-tune their work and 
address challenges that may arise during the project’s implementation. 

Post-Project Evaluation (ex-post). After the conclusion of a funded project, the final stage of 
evaluation  takes  place  in  all  FWF-funded  projects.  International  experts  examine  the 
outcomes and results of the research, assessing the scientific contributions, societal impact 
and overall success of the project. This evaluation provides essential feedback for continuous 

1260 FWF, Evaluation in the FWF, 2023,. accessed 14 July 2023. 

783 

 
 
improvement and informs future funding decisions. At the point, no excellent=austria projects 
have been concluded, and thus none of them have yet received such an evaluation.  

1.4.3.3. 

International participation 

Participation by international partners (even using their own funding) is not envisaged under 
this  programme.  The  excellent=austria  programme  focuses  primarily  on  increasing  the 
capacities of Austrian research institutions and Austrian-based scientists.  

1.4.3.4. 

Equal opportunities and fair treatment 

The  Austrian  Science  Fund  (FWF)  actively  supports  equal  opportunities  in  all  of  its 
programmes.  In  this  regard,  the  review  of  a  proposal  must  not  put  researchers  at  a 
disadvantage for non-research related reasons such as age, gender, career breaks, etc. The 
FWF Equality Plan (2023-20241261) defines relevant future actions whose aim, among others, 
is to significantly increase the number of applications from female researchers in accordance 
with their potential. 

In terms of Clusters of Excellence, a good gender balance (or at least one that reflects the 
distribution within a particular discipline) is required when selecting the key researchers and 
members  of  the  Board  of  Directors.  The  Emerging  Fields  pillar  limits  the  number  of  male 
principal investigators to roughly two-thirds of the total number of the team1262. In addition, 
the FWF revised its application guidelines for all projects in 2019 by adding a requirement for 
applications to discuss potential sex- and gender-relevant aspects of the project, and how 
these will be implemented. 

1.5.  Critical success factors and barriers 

This section presents the critical success factors for the FWF excellent=austria programme, 
as well as the most significant barriers, as identified through desk research and interviews. It 
is important to note that, given the present early phase of implementation, the successes and 
barriers may not yet be as evident as in other benchmarking case studies conducted under 
this study.  

1.5.1.  Success factors  

Synergies  between  Austrian  R&I  institutions:  one  of  the  key  success  factors  of  the 
programme  lies  in  its  focus  on  creating  networks  and  fostering  collaborations  between 
research and innovation institutions that have not previously worked together. This emphasis 
on  partnership  building  is  evident  not  only  in  the  calls  for  proposals,  but  also  during  the 
evaluation  process,  during which  the  FWF  actively  promotes  and  encourages  cooperative 
efforts. In addition, limiting the number of Clusters of Excellence applications per organisation 
encourages  universities  and  researchers  to  engage  in  collaborative  ventures  in  order  to 
maximise the benefits of the funding, leading to stronger inter-institutional ties and knowledge 
exchange. 

Curiosity-driven  approach:  the  programme’s  commitment  to  providing  freedom  and 
flexibility in the development of proposals and the implementation of the activities funded is 
instrumental  in  its  success.  The  consortia  participating  in  the  Clusters  of  Excellence  and 

1261 FWF Equality Plan 2023-2024. 
1262 These limits are set as follows: a maximum of five men out of seven team members, or four 
out of six, three out of five, two out of four, and two out of three. 

784 

 
 
Emerging  Fields,  as  well  as  individual  applicants  to  the  FWF  Distinguished  Professor 
category, have the liberty to choose the thematic focus of their proposals according to their 
own  priorities  and  strengths.  The  Emerging  Fields  initiative,  in  particular, 
fosters 
interdisciplinary connections, paving the way for innovative ideas and breakthroughs that can 
address complex challenges from fresh perspectives. 

Long-term funding perspective: the excellent=austria programme’s funding model, which 
offers  support  for a  duration  of  five  years  (with  the  potential  for a  further  extension of  five 
years in the case of the Clusters of Excellence), greatly enhances its appeal to researchers 
and  institutions.  This  long-term  perspective  provides  stability  and  continuity,  allowing 
research  projects  to  unfold  their  full  potential  and  make  substantial  progress  over  an 
extended period. This longer duration of funding also encourages researchers to embark on 
ambitious and high-impact projects that require time to yield significant results. 

Institutional support: in the case of the Clusters of Excellence, the programme’s success 
is bolstered by the robust support received from the respective institutions involved. Beyond 
financial  contributions,  these  institutions  actively  engage  in  supporting  the  projects  by 
involving  their  senior  personnel  in  Boards  of  Directors  responsible  for  offering  scientific, 
strategic and management guidance. This institutional commitment enhances the research 
teams’ capabilities, and ensures a strong ecosystem that can sustain and amplify the impacts 
of the initiatives funded. 

Development of opportunities for younger researchers: the programme demonstrates a 
proactive approach to nurturing young talent, particularly within the context of the Clusters of 
Excellence. By dedicating a key element of activities to a training unit, the programme places 
a  significant  focus  on  the  inclusion  of  young  researchers  and  the  consideration  of  post-
doctoral career perspectives. This approach not only promotes the professional development 
of early-career researchers but also contributes to the sustainability of research excellence 
in Austria by fostering a new generation of skilled and motivated scientists. 

1.5.2. 

 Barriers   

Given the current early stage of programme implementation, only a limited number of barriers 
could be identified. However, the interview programme highlighted one key challenge during 
the application process, namely the co-funding requirement (in the Clusters of Excellence 
programme).  Requiring  each  consortium  to  secure  40%  of  its  funding  from  other  sources 
presented  a  significant  hurdle,  as  it  necessitated  that  institutions  carefully  assess  their 
strategic  areas  of interest  and  make  a  long-term  commitment  to  the  project,  ensuring  that 
only  the  most  promising  and  well-prepared  proposals  could  participate  in  the  competitive 
process. In addition, the co-funding requirement underlined the need for effective financial 
planning and resource allocation, demanding a rigorous evaluation of the proposed initiatives’ 
feasibility and potential for success. 

1.6. 

Transferability of findings to the EU Framework Programmes 

The excellent=austria programme yields interesting avenue from which to reflect on the future 
of certain actions under the Framework Programmes.  

Collaborative  bottom-up  curiosity-driven  research  and  long-term  perspectives.  The 
FWF’s  Clusters  of  Excellence  programme  provides  an  exemplary  model  for  collaborative, 
bottom-up, curiosity-driven research with a notable long-term perspective. A key highlight of 
this programme is the extended duration of support, which enables the  training of multiple 
cohorts of PhD students and post-docs, leveraging accumulated knowledge and exploring 

785 

 
 
 
diverse  avenues  of  research  and  application  based  on  the  findings.  Moreover,  the 
programme’s  success  is  bolstered  by  its  substantial  budget,  which  ranges  from  EUR 2  to 
EUR 7  million  annually  for  the  whole  programme,  which  is  concentrated  in  up  to  seven 
institutions. Its primary focus on generating breakthroughs and further pushing the frontiers 
of  disciplines  enhances  the programme’s  potential  to  make  significant contributions to  the 
scientific landscape. A noteworthy parallel can be drawn between the Clusters of Excellence 
and the ERC grants within the Framework Programme, both of which embrace a bottom-up 
approach.  However,  the  Clusters  of  Excellence  distinguish  themselves  by  incorporating  a 
collaborative dimension across multiple Austrian institutions. This inclusion of collaborative 
efforts  enriches  the  programme  by  expanding  opportunities  for  innovative  research  ideas, 
facilitating access to new collaborative networks, providing essential training for researchers, 
and  granting  access  to  vital  research  infrastructures  and  equipment.  This  comprehensive 
approach  positions  the  Clusters  of  Excellence  programme  as  a  driver  of  groundbreaking 
discoveries and a catalyst for advancing knowledge across diverse fields of research. 

1.7.  Conclusions 

in  various  domains  of  knowledge.  Furthermore, 

This benchmark report presents the main characteristics of three initiatives organised under 
the excellent=austria programme of FWF Austria. In doing so, it shows that the programme 
represents a new dimension in research funding in Austria, and at least partially addresses 
the long-term problem of the underfunding of excellent basic research. A key aspect of the 
programme is its strict adherence to the ‘bottom-up’ principle, fostering an environment of 
freedom for unconventional approaches that adhere to the highest international standards. 
This principle aligns the programme’s core objectives with those of the prestigious European 
Research  Council  (ERC).  By  endorsing  this  approach,  the  programme  empowers 
researchers  to  pursue  innovative  ideas  and  breakthroughs,  thereby  potentially  yielding 
the 
transformative  advances 
excellent=austria programme fosters collaborative endeavours by offering an opportunity for 
diverse research teams from different institutions across Austria to form partnerships. Such 
collaborations  aim  to  leverage  synergies  and  facilitate  the  collaboration  of  previously 
unacquainted  partners.  In  many  ways,  this  mirrors  the  overarching  goals  of  the  WIDERA 
programme part, providing a platform to encourage interdisciplinary exchange and the cross-
pollination of ideas, which could lead to a proliferation of novel insights and discoveries. Even 
though  the  success  of  the  programme  is  yet  to  be  seen,  the  positive  effects  of  new 
collaborations  were  already  noted  during 
the 
excellent=austria  programme  represents  the  first  time  an  FWF-funded  programme  has 
focused  not  only  on  basic  research,  but  also  encouraged  participating  consortia  to  define 
ways in which the transfer of research results to business and society will be boosted. In the 
long  term,  the  programme  is  expected  to  enhance  the international  reputation  of  Austria’s 
research and innovation ecosystem.  

interview  programme.  Lastly, 

the 

Appendix 1. Literature sources 

•  Corporate website of FWF Austria. https://www.fwf.ac.at/en/ 

•  Annual FWF Austria reports, 2018-2022. 

•  Austrian Research and Technology reports, 2018-2022. 

https://www.bmbwf.gv.at/en/Topics/Research/Research-in-Austria/Services/FTB.html  

786 

 
 
•  RTI Strategy 2030 Strategy for Research, Technology and Innovation of the Austrian 

Federal Government, 2020. 
https://era.gv.at/public/documents/4489/RTI_Strategy_2030-1-1.pdf  

•  OECD Reviews of Innovation Policy: Austria 2018. https://www.oecd-

ilibrary.org/science-and-technology/oecd-reviews-of-innovation-policy-austria-
2018_9789264309470-en 

•  FWF Dashboard. https://dashboard.fwf.ac.at/ 

• 

Janger, J., Schmidt, N., & Strauss, A., International differences in basic research grant 
funding – as systematic comparison. Austrian Institute of Economic Research, 2019 

787 

 
 
Annex 3.4: Benchmark on the Australian Research Council  

1.1. 

Introduction 

The  present  report  focuses  on  the  Australian  Research  Council  (ARC).  Given  the  ARC’s 
focus  on  fostering  the  uptake  of  innovative  solutions  in  industry  and  society  in  order  to 
address  global  challenges,  lessons  learned  from  this  programme  will  be  valuable  to  the 
Horizon  Europe  Framework  Programme  –  in  particular,  its  programme  parts  relating  to 
excellent science.   

This  benchmark  report  aims  to  derive  lessons  and  inspiration  from  the  ARC  that  can 
contribute to the relevance, effectiveness, efficiency and added value of the ERC, MSCA and 
WIDERA  in  Europe.  This  benchmark  relies  on  extensive  desk  research  of  available ARC-
related information, data and publications, as well as documentation concerning the ARC and 
its  work  programmes.  Furthermore,  this  benchmark  draws  on  qualitative  insights  gained 
through interviews with ARC stakeholders and beneficiaries.  

The report is structured as follows. Section 1.2 presents the methodological approach and 
evaluation questions addressed by this study. Section 1.3 discusses the relevant Australian 
policy context, as well as the intervention logic and features of the ARC. Section 1.4 presents 
the results and the findings of the benchmark study in relation to patterns of participation in 
ARC programmes, as well as its key results and impacts. Section 1.5 identifies key drivers 
or  enabling  factors  that  have  influenced  the  ARC,  as  well  as  barriers  to  progress  and,  if 
applicable, the solutions to those barriers implemented in the ARC open call. These results, 
and  the  critical  success  factors  and  barriers  identified,  are  subsequently  translated  into 
lessons for the Framework Programme in Section 1.6. 

1.2.  Methodology and evaluation questions addressed 

This benchmark study relies on desk research on the ARC. Especially important in this regard 
are the ARC Strategy 2022-2025 report, ARC selection reports, and the Australian Research 
Council Annual Report 2021-2022. These represent rich sources of information with regard 
to its functioning and resources, as well as participants in the programme, but also into recent 
changes that have been made to the programme. In addition, documents and reports relating 
to  Horizon  Europe  and  its  work  programmes  have  also  been  examined  to  investigate  the 
transferability of the results of this study to the various parts of the Framework Programme.  

Furthermore,  the  above  desk  research  is  complemented  by  qualitative  evidence  from 
interviews  carried  out  with  ARC  representatives  and  ARC  beneficiaries.  These  interviews 
provide  more  in-depth  insights  into  patterns  of  participation,  the  perceived  impacts  of  the 
ARC, as well as its strengths and weaknesses. Interviews were carried out with a total of six 
individuals.  

Given the uniqueness of the ARC, the benchmark analysis in this report focuses in particular 
on the organisation’s Linkage Program and its funding schemes. The Linkage Programme 
aims to encourage and extend cooperative approaches to research, and to improve the use 
of research outcomes by strengthening links within the innovation system in Australia and 
internationally.  The  Linkage  Program  promotes  national  and  international  research 
partnerships  between  researchers  and  industry,  community  organisations  and  publicly 
funded  research  agencies.  In  this  context,  the  benchmark  provides  lessons  that  are  of 
particular relevance to for the ERC, MSCA and WIDERA programmes under Horizon Europe.  

788 

 
Table 1 below presents a list of evaluation criteria that will be addressed in this benchmark 
report. 

Evaluation criteria 

Judgement criteria 

Efficiency 

Effectiveness 

Relevance  

Added value 

• 

• 
• 

• 

• 

• 

• 

• 

• 

Key factors that influence the application, participation and 
cooperation of target groups. 
Actions taken to encourage the participation of target groups. 
Perceptions regarding the sustainability of international, 
intersectoral collaborations; measures taken to improve it.  
Patterns of participation in the programme. 

Identification of the outcomes and impacts of the actions, and of 
success factors and difficulties in supporting industry-academia 
and cross-sectoral cooperation. 
Exploitation of the research outcomes by the broader innovation 
system in Australia and internationally. 

The appropriateness of the ARC’s activities and themes in relation 
to emerging needs and policy priorities, as well as scientific, 
technological, and socio-economic developments. 

The value generated by the Australian Research Council in terms 
of international and intersectoral collaborations aimed at 
promoting scientific excellence and capacity building, as well as 
collaborations with quadruple helix partners; 
Expected results and effects at national level, compared with 
European-level instruments. 

Table 1. Evaluation criteria addressed in this benchmark report 

1.3.  Context and objectives of the benchmark initiative 

1.3.1.  Australian policy context 

The Australian Research Council (ARC) was established as an independent body in 2001, 
under  the  Australian  Research  Council  Act  20011263.  The  ARC  superseded  the  Australian 
Research Grants Committee (ARGC), which had made recommendations on the allocation 
of  funding  support  to  research  proposals  submitted  by  university  researchers  in  Australia 
since 1965.  

The ARC was formed in response to the Australian federal government’s white paper, ‘Higher 
Education: a policy statement’, which outlined the government’s strategy for the long-term 
development of Australia’s higher education system. From playing a role in its early days of 
providing  both  research  funding  and  research  policy  advice,  the  ARC  now  acts  as  an 
independent  body  with  a  broad  range  of  advisory  functions  and  full  administrative 
responsibility for the assessment of grant applications.  

The  ARC  allocates  funding  to  individual  schemes,  most  of  which  fall  under  the  National 
Competitive Grants Program (NCGP). It also carries out Excellence in Research for Australia 

1263 Parliament of Australia, Australian Research Council Act 2001, Act No. 8 of 20 April 2022, 
retrieved 4 May 2023. https://www.legislation.gov.au/Series/C2004A00773 

789 

 
 
(ERA)1264 exercises, which assess research quality in Australia’s higher education institutions 
(HEIs)  and  evaluate  the  country’s  research  capacity  in  comparison  to  international 
benchmarks.  

Through  its  periodic  Engagement  and  Impact  Assessment  (EI)1265,  the  ARC  also  aims  to 
assess  how  well  research  outputs  are  translated  into  impacts  beyond  academia.  The  EI 
facilitates  strategic  planning  to  further  strengthen  research  engagement  and  impact,  and 
helps to identify and promote the contributions that made by universities to the economic, 
social, environmental and cultural domains of the country and beyond. 

1.3.2.  Australian Research Council’s National Competitive Grants Program (NCGP) 

The Australian Research Council administers the National Competitive Grants Programme 
(NCGP),  which  supports  high-quality  fundamental  and  applied  research  through  two  main 
funding  programmes,  Discovery  and  Linkage.  The  Discovery  Program  focuses  on  the 
fundamental ‘blue-sky’ research, and supports individuals and small teams. Five schemes 
are available under the Discovery Program1266:  

•  Australian Laureate Fellowships (FL) 
•  Discovery Early Career Researcher Award (DECRA) 
•  Discovery Indigenous (IN) 
•  Discovery Projects (DP) 
•  Future Fellowships (FT). 

The  Linkage  Program  focuses  on  encouraging  and  extending  cooperative  approaches  to 
research  by  promoting  national,  international  and  intersectoral  research  partnerships. 
Funding is available under the following schemes1267: 

•  ARC Centres of Excellence (CoE); 
• 

(ITRP), 

including 

Industrial 
Industrial  Transformation  Research  Program 
Transformation  Research  Hubs  (ITRH)  and  Industrial  Transformation  Training 
Centres (ITTC); 
Linkage Infrastructures, Equipment and Facilities (LIEF); 
Linkage Projects (LP); 

• 
• 
•  Special Research Initiatives (SRI); 
• 
•  Supporting Responses to Commonwealth Science Council Priorities (CS); and 
• 

Learned Academies Special Projects (LASP); 

Industry Fellowships (IF).  

1264 Australian Research Council, Excellence in Research for Australia,, n.d., available at: 
https://www.arc.gov.au/evaluating-research/excellence-research-australia 
1265 Australian Research Council, Engagement and Impact Assessment, n.d., available at: 
https://www.arc.gov.au/evaluating-research/ei-assessment 
1266 Australian Research Council (2023b, February 10). Australian Research Council Annual 
Report 2021-22, Transparency Portal, available at: https://www.transparency.gov.au/annual-
reports/australian-research-council/reporting-year/2021-22 
1267 Australian Research Council (2023b, February 10). Australian Research Council Annual 
Report 2021-22, Transparency Portal, available at: https://www.transparency.gov.au/annual-
reports/australian-research-council/reporting-year/2021-22 

790 

 
 
Two structures provide support to the administration of the NCGP. The Selection Advisory 
Board  provides  recommendations  on  which  applications  should  receive  funding,  with  a 
projected budget for each1268. The National Competitive Grants Program Appeals Committee 
deals with appeals submitted to the ARC, and makes recommendations as to whether such 
appeals should be considered or dismissed1269.  

One of the annual targets set by the ARC is to have 100% of applications receive a minimum 
of two detailed assessments. As its 2021-2022 Annual Report indicates, the ARC relies on 
external parties to provide these assessments. Approximately 7,000 assessors support the 
ARC peer review process; however, uncertainty created by the COVID-19 pandemic resulted 
in  no  guarantees  being  given  that  applications  will  always  receive  a  minimum  of  two 
assessments1270.  

All  current  and  future  funding  opportunities  available  under  the  NCGP  are  published  and 
handled by GrantConnect1271 – Australia’s whole-of-government grants information system. 
Meanwhile, the ARC’s Research Management System (RMS)1272 serves as the focal point 
for  the  preparation  and  submission  of  research  proposals,  award  announcements  and 
resubmissions.  

Through  the  various  activities  supported,  NCGP  grants  should  lead  to  expected  outputs. 
These include the advancement of career opportunities through diverse pathways, as well as 
the increased visibility of high-quality research. In addition, the programme aims to foster the 
growth of research networks and strategic collaborations, both nationally and internationally. 
It also seeks to enhance the attractiveness of research careers and to facilitate better access 
to  research  facilities  on  a  global  scale.  Furthermore,  the  initiative  strives  to  improve  the 
employability of researchers and to bolster the exchange of knowledge between academia 
and  business.  Ultimately,  this  will  lead  to  the  refinement  of  both  the  scientific  and  the 
transferable  skills  of  researchers,  contributing  to  a  more  dynamic  and  impactful  research 
landscape. 

The  anticipated  impacts  of  the  NCGP  encompass  economic,  commercial,  environmental, 
social  and  cultural  benefits  for  Australia,  as  well  as  an  expanded  knowledge  base  and 
strengthened  research  ecosystem.  Figure  1  outlines  the  intervention  logic  of  the  ARC’s 
National Competitive Grants Programme (NCGP). 

1268 Australian Research Council, Selection Advisory Committees, n.d., available at: 
https://www.arc.gov.au/about/our-organisation/committees/selection-advisory-committees 
1269 Australian Research Council, Appeals against NCGP administrative processes, n.d., 
available at: https://www.arc.gov.au/about-arc/our-organisation/committees/appeals-against-
ncgp-administrative-processes 
1270 Australian Research Council (2023b, February 10). Australian Research Council Annual 
Report 2021-22, Transparency Portal, available at: https://www.transparency.gov.au/annual-
reports/australian-research-council/reporting-year/2021-22 
1271 Australian Research Council, GrantConnect – the Australian Government’s grant information 
system, n.d., available at: https://www.arc.gov.au/funding-research/apply-funding/grantconnect  
1272 Australian Research Council, Research Management System (RMS) Information, n.d., 
Available at: https://www.arc.gov.au/manage-your-grant/research-management-system-rms-
information 

791 

 
 
Figure 1. ARC National Competitive Grants Programme intervention logic. Source: compiled by the authors. 

1.4.  Results 

1.4.1.  Relevance  

Interviews with stakeholders carried out for the present study provide examples  of how the 
ARC’s  various  funding  schemes  are  tailored/relevant  to  different  and  distinct  research 
objectives and target groups. Each scheme – from the Industrial Transformation Research 
Program (ITRP), with its emphasis on collaborative industry projects, to the ARC Centres of 
Excellence scheme, which focuses on fostering extensive partnerships  – serves a specific 
yet complementary role within the broader framework. The ITRP’s Industrial Transformation 
Research  Hubs  (ITRH)  and  Industrial  Transformation  Training  Centres  (ITTC)  address 
industrial challenges while also investing in skills development. The Linkage Infrastructure, 
Equipment  and  Facilities  (LIEF)  scheme  ensures  the  shared utilisation  of  critical  research 
infrastructures.  In  addition,  the  Linkage  Projects  (LP)  scheme  facilitates  internationally 
competitive  research  endeavours,  with  a  focus  on  end-user  relevance  and  commercial 
application.  The  Special  Research  Initiative  (SRI)  scheme,  administered  by  the  NCGP, 
enhances the coherence of the programme by strategically addressing key priority areas and 
emerging fields vital to Australia’s global competitiveness. While alternative regional funding 
schemes  are  available  in  Australia,  the  Linkage  Program  is  renowned  for  its  competitive 
nature, placing a strong emphasis on its unique standing and prestige. 

Interviews  carried  out  with  Australian  Research  Council  stakeholders  reveal  that  grant 
funding  is  significantly  influenced  by  the  National  Interest  Test  (NIT).  All  applications  for 
funding under the National Competitive Grants Program (NCGP) are required to include an 
NIT  statement.  The  purpose  of  this  is  to  demonstrate  the  societal  benefits  (economic, 
commercial,  environmental,  social,  or  cultural)  of  the  proposed  research  beyond  the 
academic community. The NIT statement is a simple, cohesive statement demonstrating to 
an  average  member  of  the  general  public  how  the  proposed  research  can  be  of  value  to 
Australia.  One  of  the  stakeholders  interviewed  supported  the  idea  of  justifying  research 
funding  to  taxpayers  and  considered  the  NIT  to  be  a  reasonable  criterion.  However, 
interviewee acknowledged the growing challenge of intense competition for grants, making it 

792 

 
 
more challenging to obtain funding. A similar justification is also required when applying to 
Horizon  Europe  funding,  including  in  Pillar  1.  Organisations  and  individuals  applying  to 
Horizon Europe need to explain the impact their research will have for society. 

Applications  for  funding  must  also  take  into  account  the  Industrial  Transformation  Priority 
areas  set  by  the  Australian  government.  In  interviews,  stakeholders  emphasised  that  it  is 
important for project proposals to show how they align with national research priority areas 
in order to secure funding, even if the underlying research idea is strong. 

This  alignment  with  government  priorities,  particularly  in  areas  such  as  sustainable 
infrastructure and sustainability, is seen as vital, and reflects the government’s emphasis on 
green energy and reducing environmental impact. Nonetheless, while alignment with such 
priorities  is  preferred,  it  was  noted  that  it  is  not  an  absolute  requirement  for  funding. 
Interviewees  highlighted  that  compelling  projects  led  by  excellent  investigators  can  still 
receive funding even if they do not align perfectly with the priority areas. It was also noted 
that  these  priorities  may  shift  with  a  change  in  government,  and  can  sometimes  be  quite 
narrowly  focused.  Overall,  however,  it  was  evident  that  government  priorities  play  a 
significant role in shaping research funding across the various grant schemes. Furthermore, 
the  stakeholders  agreed  that  the  priority  areas  set  by  the  government  rightly  reflect  the 
emerging needs, as well as scientific, technological and socio-economic developments. 

1.4.2.  Efficiency and added value: the contributions of partner organisations 

One  of  the  distinctive  features  of  grant  schemes  under  the  Linkage  Program  is  the 
requirement for partner organisations to contribute cash and/or in-kind support to show their 
commitment to the funded projects. This requirement has been in place for approximately 15 
years. The stakeholders emphasised the underlying reasons for this approach to funding: 

•  Demonstration  of  commitment:  cash  contributions  demonstrate  a  commitment  from 
the participating organisations. This shows that the project is taken seriously, and has 
tangible support. 

•  Leveraging  resources:  cash  contributions  can  amplify  the  impact  of  the  funding 
provided by the government. They allow a larger overall budget, which can lead to more 
extensive research, better equipment and broader outreach efforts. 

•  Ensuring sustainability: when organisations provide cash contributions, they have a 
vested interest in the success of the project. This can lead to a greater commitment to 
sustaining the project’s outcomes beyond the duration of the funding period. 

• 

In-kind  contributions:  in  addition  to  cash,  organisations  can  provide  in-kind 
contributions. These can consist of resources, equipment or services that are relevant to 
the project, and which can complement and enhance the project's outcomes. 

•  Facilitating innovation and risk-taking: with additional financial support, projects may 
enjoy more leeway to take calculated risks and explore innovative approaches. This can 
lead to breakthroughs and advances in research and development. 

•  Enhancing  credibility  and  competitiveness:  projects  including  cash  contributions 
often appear more credible  and competitive in grant applications. It signals to funding 
agencies that there is strong support and belief in the project's potential success. 

793 

 
Evaluation of the ARC Linkage Program for 2013-2018 offers some evidence regarding the 
sizes and types of partner organisations’ contributions financial contributions (see Figures 2 
and 3 below). In general, partner organisations tend to contribute more in-kind than in cash. 
A  tendency  can  also  be  seen  whereby  projects  that  offered  higher  in-kind  or  cash 
contributions were more likely to be selected. 

Figure 2. Total partner organisation contributions (cash and in-kind) in funded projects. Source: ARC. Note: data for 2018 C 
were not available at the time this report was prepared. 

Figure 3. Average partner organisation cash and in-kind contributions per application and funded project. Source: ARC. 
Note: complete data for 2018 C were not available at the time this report was prepared. 

The  present  analysis  has  also  found  that  there  is  some  flexibility  when  it  comes  to  slight 
overlaps  between  the  scope  of  research  under  different  funds,  which  may  be  deemed 
permissible. In terms of supplementing funding through external sources, while established 
processes  exist  that  are  overseen  by  the  ARC,  it  was  noted  that  in  some  cases,  informal 
arrangements  can  allow  such  a  combination  of  funds.  Although  a  clear  ethical  boundary 
exists  against  ‘double-dipping’  for  the  same  project,  there  are  avenues  for  strategic 

794 

 
 
 
collaboration  and  the  supplementing  of  funds  from  various  sources  under  specific 
circumstances and with appropriate oversight. 

Furthermore,  in  a  similar  manner  to  the  ‘hop-on  facility’  under  Horizon  Europe,  the  ARC 
allows new partners to join already-ongoing projects in situations where the existing partner 
organisations within the consortium agree. Allowing new partners to join ongoing projects is 
considered beneficial, and can lead to a more dynamic, inclusive and ultimately successful 
research endeavour. This possibility enables consortia to leverage additional expertise and 
resources, promotes diversity, and enables the project to adapt to evolving challenges and 
opportunities. 

1.4.3.  Efficiency: application and evaluation process  

The findings from the interviews carried out for this study point to the challenging landscape 
of grant applications in Australia, particularly for projects under the Linkage Program, where 
success rates hover below 20%. These low success rates underline the competitive nature 
of schemes under the Linkage Program. In terms of the specific evaluation criteria applied, 
the  strength  of  the  team,  the  project’s  innovation  potential,  its  benefits  to  society  and  its 
feasibility were emphasised as pivotal factors contributing to a successful proposal.  

However, stakeholders who were interviewed acknowledged that the submission process for 
Linkage  Program  grants  is  detailed  and  lengthy,  prompting  the  suggestion  of  a  more 
streamlined  approach,  especially  for  industry  partners.  According  to  the  interviewees, 
Application is demanding, often stretching to more than 100 pages. Each chief investigator 
is required to compile a comprehensive list of their publications, adhering to a specific format. 
For seasoned researchers with a decade or more of contributions,  this can be an onerous 
task, resulting in lists in excess 50 entries per investigator. Multiplied across a team, this can 
result in the challenge of documenting 200-300 articles. 

Furthermore, the requirement to address the requirement for ROPE (Performance Relative 
to  Opportunity),  which  entails  a  researcher  articulating  their  achievements  in  the  light  of 
personal  life  challenges  and  other  pertinent  factors,  introduces  an  additional  layer  of 
complexity. These complexities highlight an inherent mismatch between the academic rigour 
expected and the practical expectations of industry partners, who prioritise a straightforward, 
concise  approach  over  an  exhaustive  account  of  extensive  publication  records  in  high-
ranking journals. The need for a more streamlined and direct application process is evident. 

The evaluation process also operates according to a single-blind format. Under this setup, 
whereas the assessors are privy to the identity of the applicants, applicants remain unaware 
of their assessors’ identities. This process received criticism from stakeholders, who called 
greater transparency. 

1.4.4.  Efficiency: grant funding allocation 

The  interviews  conducted  with  stakeholders  from  the  Australian  Research  Council  (ARC) 
provided valuable insights into the utilisation and potential limitations of ARC funding. One 
noteworthy  observation  was  that  once  researchers  have  secured  ARC  funding,  there  is  a 
perceived lack of stringent oversight or monitoring mechanisms. This raised concerns among 
some stakeholders regarding the productivity and outcomes of funded projects. 

The allocated funding can cover a range of items such as personnel costs, equipment and 
supplies,  travel  and  subsistence,  networking  and  dissemination  activities,  while  major 
equipment  needs  are  addressed  by  the  Linkage  Infrastructure,  Equipment  and  Facilities 

795 

 
(LIEF) scheme. The Linkage Program seeks specific project outcomes, and while flexibility 
is  offered  within  the  budgets  allocated,  significant  deviations  in  certain  budget  categories 
require 
to  accommodate  changing 
circumstances,  but  annual  and  financial  reports  must  be  submitted.  The  administering 
organisations  –  typically  universities  –  oversee  the  use  of  funds,  and  changes  within 
expenditure categories during the course of the project need to be reported.  

justification.  Budget  adjustments  are  permitted 

1.4.5.  Effectiveness and efficiency: industry-academia collaborations 

The Linkage Program places a strong emphasis on creating and sustaining collaborations 
between  industry  and  academia.  The  interviews  with  ARC  stakeholders  illuminated  the 
intricacies  of  industry-academia  collaborations  and  cross-sectoral  cooperation.  One 
significant  challenge  lies  in  aligning  the  differing  priorities  and  long-term  objectives  of 
academic institutions and industrial stakeholders. Academia often emphasises fundamental 
research  and  long-term  scientific  goals,  while  industry  tends  to  have  more  immediate, 
commercially  driven  imperatives.  This  divergence  can  create  tension  in  setting  mutually 
agreeable research agendas. 

Intellectual property (IP) issues further compound the collaborative landscape. Determining 
the  ownership  of  and  rights to  innovations  and  discoveries  generated  within  a  partnership 
can be a delicate process. Balancing the need for open academic discourse with industry’s 
interest  in  protecting  proprietary  knowledge  can  be  a  formidable  task,  requiring  careful 
negotiation and legal frameworks. 

Moreover, the motivations that drive industry partners to participate in these collaborations 
can  vary  widely.  While  researchers  may  hold  the  ARC  in  high  esteem  for  its  prestigious 
funding  opportunities,  industry  partners  are  often  more  inclined  to  engage  if  they  foresee 
direct  and  practical  benefits.  Some  industry  partners  may  enter  into  collaborations  out  of 
social  responsibility  or  contractual  obligations,  but  tangible,  industry-specific  advantages 
remain a primary motivator. 

From the perspective of the involvement of industry partners, there usually a lengthy delay 
between  initiating  negotiations  concerning  potential  involvement  in  government-funded 
research projects, and the actual start of the projects if they are deemed successful. In other 
words,  businesses  find  it  hard  to  plan  in  advance  their  potential  involvement  in  a  project, 
taking into account a company’s reliance on specific business cycles. 

Participation in such partnerships extends across the business spectrum, encompassing both 
larger multinational corporations and smaller enterprises. Notably, larger companies tend to 
contribute more substantial resources to these collaborations, reflecting their greater capacity 
for investment and a recognition of the long-term benefits derived from such engagements. 

1.4.6.  Effectiveness: results and participation patterns 

This section of the report provides an overview of six funding schemes available under 
the Linkage Program. It analyses in detail the patterns in participation, financial resources, 
distribution  of  funding  by  participating  organisations,  discipline  panels  and  strategic 
research priorities. The metrics provided below are drawn from publicly available data, 
and in more detail the latest rounds of the funding scheme; however, where information 
was available, they also shed light on previous rounds of funding.  

796 

 
The  Industrial  Transformation  Research  Program  (ITRP)1273  aims  to  support 
collaborative  projects  between  university  researchers  and  industry  in  response  to  the 
challenges faced by industry through the development of new products, processes and 
services. The programme consists of two schemes –Industrial Transformation Research 
Hubs  (ITRH)1274,  and  Industrial  Transformation  Training  Centres  (ITTC)1275.  ITRH  are 
designed  to  provide  funding  for  collaborative  research  that  benefits  industry  partners, 
while  ITTC  place  an  emphasis  on  the  advancement  of  skills  by  providing  innovative 
higher-degree and postdoctoral training for research focusing on end-users in industries 
important to Australia’s future.  

Comparing funding by ITRH scheme round over recent years, the highest success rate 
(50%) and the largest amount of funding (AUD 36,418,096) are seen in 2021. Conversely, 
the lowest amount of funding was allocated in 2022, namely AUD 17,768,172. 

Industrial Transformation 
Research Hubs (ITRH) 

ITRH 2019 
(2018-
2019) 

ITRH 2020 
(2019-
2020) 

ITRH 2021 
(2020-
2021) 

ITRH 2022 
(2021-
2022) 

Number of applications  

Number of awards 

13 

4 

16 

5 

16 

8 

12 

4 

Success rate 

30.8% 

31.3% 

50% 

33.3% 

Total funding (AUD) 

18,104,299 

24,950,000 

36,418,096 

17,768,172 

Table 2. Industrial Transformation Research Hubs (ITRH) funding, by round. Source: ARC selection outcome reports. 

The table below lists all of the Industrial Transformation Research Hubs that received funding 
from the ARC in 2022. Interestingly, as shown in the table, the contributions from participating 
organisations are three times higher (76% of the total) than the funding received from ARC 
(24%). Broken down by type of participating organisation, the highest contributions are made 
by partner organisations – accounting for AUD 31,444,458 – more than half (55.8%) of the 
total contributions made by organisations to successful project applications.  

An  almost  identical  distribution  of  funding  can  be  observed  for  the  years  2021,  2020  and 
2019, with less than on-third of funding for successful applicants being provided by the ARC, 
and more than two-thirds coming from participating organisations, with partner organisations 
contributing the highest share of financial contributions per type of participating organisation. 

1273 Australian Research Council, Industrial Transformation Research Program, n.d., available at: 
https://www.arc.gov.au/funding-research/funding-schemes/linkage-program/industrial-
transformation-research-program 
1274 Australian Research Council, Industrial Transformation Research Hubs available at: 
https://www.arc.gov.au/funding-research/funding-schemes/linkage-program/industrial-
transformation-research-program/industrial-transformation-research-hubs 
1275 Australian Research Council, Industrial Transformation Training Centres, n.d., available at: 
https://www.arc.gov.au/funding-research/funding-schemes/linkage-program/industrial-
transformation-research-program/industrial-transformation-training-centres 

797 

 
 
Research Hub title 

Administering 
organisation 

ARC funds 
approved over the 
project lifetime 
(AUD) 

Contributions by 
participating 
organisations (AUD) 

ARC Research Hub for 
Advanced Manufacture 
of Targeted 
Radiopharmaceuticals 

ARC Research Hub to 
Advance Timber for 
Australia’s Future Built 
Environment 

ARC Research Hub for 
Fire Resilience 
Infrastructure Assets, 
and Safety 
Advancements 
(FRIASA) in Urban, 
Resources, Energy and 
Renewables Sectors 

ARC Research Hub for 
Carbon Utilisation and 
Recycling 

University of 
Queensland 

4,808,669 

University of 
Queensland 

2,959,803 

22,239,418 

University of New 
South Wales 

4,999,700 

16,798,035 

Monash University 

5,000,000 

17,320,895 

Total funding (AUD) 

17,768,172 

56,358,348 

Table 3. Industrial Transformation Research Hubs (ITRH) funded in 2022. Source: ARC selection outcome reports. 

With regard to Industrial Transformation Training Centres (ITTC), between 2019 and 2022, 
the largest amount of funding allocated was in 2021, to eight successful applications, and 
totalled  AUD 37,770,864.  The  success  rate  among  the  applications  submitted  was  also 
highest in 2021. Conversely, the 2020 scheme round had the lowest success rate (16.1%); 
however, it  also shows  the highest  number of  applications  compared  with  the  other years 
analysed. 

Following a similar pattern to funding under the ITHR, the funding provided by the ARC to 
Industrial Transformation Training Centres (ITTC) in 2022 accounted for slightly more than 
one-third  (34%)  of  total  contributions  for the  successful applications,  with  66% of  the  total 
funding coming from the participating organisations. Looking at previous funding rounds, the 
shares of ARC financial contributions to successful applicants were at most 35% of the total 
contributions made (29% in 2019, 33% in 2020, and 28% in 2021). 

798 

 
 
 
 
Industrial 
Transformation 
Training Centres 
(ITTC) 

Number of 
applications  

Number of awards 

ITTC 2019 
(2018-2019) 

ITTC 2020 
(2019-2020) 

ITTC 2021 
(2020-2021) 

ITTC 2022 
(2021-2022) 

25 

6 

31 

5 

25 

8 

21 

6 

Success rate 

24% 

16.1% 

32% 

28.6% 

Total funding 
(AUD) 

24,072,165 

24,334,385 

37,770,864 

29,285,604 

Table 4. Industrial Transformation Training Centres (ITTC) funding, by round. Source: ARC selection outcome reports. 

799 

 
Industrial 
Transformation 
Training Centre title 

Administering 
organisation 

ARC funds 
approved over the 
project lifetime 
(AUD) 

Contributions by 
participating 
organisations (AUD) 

ARC Training Centre 
for Advanced Building 
Systems Against 
Airborne Infection 
Transmission 

ARC Training Centre 
for Biofilm Research 
and Innovation 

ARC Training Centre 
for Environmental and 
Agricultural Solutions 
to Antimicrobial 
Resistance 

ARC Training Centre 
for Hyphenated 
Analytical Separation 
Technologies 

ARC Training Centre 
for Innovative 
Composites for the 
Future of Sustainable 
Mining Equipment 

ARC Training Centre 
for Next-Gen 
Architectural 
Manufacturing 

Queensland 
University of 
Technology 

4,939,486 

7,060,038 

Flinders University 

4,930,205 

7,648,913 

University of 
Queensland 

University of 
Tasmania 

University of 
Wollongong 

4,508,426 

8,389,379 

4,958,927 

11,423,983 

4,969,602 

11,625,228 

The University of New 
South Wales 

4,978,958 

11,021,833 

Total funding (AUS) 

29,285,604 

57,169,374 

Table 5. Industrial Transformation Training Centres (ITTC) funded in 2022. Source: ARC selection outcome reports. 

The  graphs  below  provide  insights  into  the  shares  of  financial  contributions  made  to  the 
budget  of  successful  applications,  by  type  of  participating  organisation.  For  the  ITRH  and 
ITTC  schemes,  the  most  significant  contributors  were  organisations  representing  industry 
(83%  and  86%,  respectively).  The  same  trend  can  be  observed  in  previous  years,  with 
industry  being  the  largest  financial  contributor  among  all  the  participating  organisations  in 
both the ITHR and the ITTC schemes. 

800 

 
 
ITRH
3% 3% 3%

7%

83%

Government

Industry Body

Non-profit

Education

Other

ITTC

0%

1%

5%

7%

86%

Government

Industry Body

Non-profit

Education

Other

Figure 4. ITPR Partner organisation contributions, by type of participating organisation in 2022. Source: ARC selection 
outcome reports. 

The  Industrial  Transformation  Research  Program  (ITRP)  also  has  a  set  of  Industrial 
Transformation Priorities identified by the ARC. The selected applications must show a strong 
research  focus  towards  one  or  more  priority  areas,  with  a  clear  objective  to  support  the 
development  of  research  outcomes  that  benefit  industry  partners.  As  shown  in  the  graph 
below,  the  priority  of  Advanced  Manufacturing  received  the  highest  focus  among  all  the 
priority areas in the ITRP schemes in 2022. Projects in the ITTC scheme also focused heavily 
on priorities such as Medical Technologies (20%) and Pharmaceuticals as well as the Food, 
Beverage  and  Agribusiness  (18%)  priority  area.  With  regard  to  the  ITRH,  cutting-edge 
research was also largely directed towards achieving advances in priority areas such as Oil, 
Gas and Energy resources (16%), as well as Food, Beverage and Agribusiness (14%). 

ITTC

28%

8%3%

18%

20%

10%

13%

3%

ITRH

35%

11%

14%

11%

11%

16%

3%

Advanced Manufacturing

Cyber Security

Defence

Food, Beverage and
Agribusiness

0%

20%

40%

60%

80%

100%

Figure 5. ITPR distribution of funding, by priority area in 2022. Source: ARC selection outcome reports. 

The ARC Centres of Excellence scheme1276 runs periodically and aims to provide financial 
support to facilitate significant collaboration between universities, publicly funded research 
organisations,  other  research  bodies,  governments  and  businesses  to  undertake  highly 
innovative  and  potentially  transformational  research  that  would  consequently  lead  to 
achieving international standing and significant advancement of capabilities and knowledge. 
The focal point of the scheme is to build critical mass with a new capacity for interdisciplinary 
and cross-sectoral research. 

When  looking  at  the  funding  allocated  to  successful  applicants  in  the  ARC  Centres  of 
Excellence  scheme,  the  table  below  shows  that,  overall,  the  amount  of  funding  increased 

1276 Australian Research Council, ARC Centres of Excellence, n.d., available at: 
https://www.arc.gov.au/funding-research/funding-schemes/linkage-program/arc-centres-
excellence 

801 

 
 
 
 
substantially from the first scheme round in 2011 to the most recent one in 2023. The success 
rates of the scheme rounds varied between no less than 9 % to 12 %.  The number of eligible 
applications to receive funding was also largely similar throughout the years. 

ARC 
Centres of 
Excellence 

CE 2011 
(2010-
2011) 

CE 2014 
(2013-2014) 

CE 2017 
(2016-2017) 

CE 2020 
(2019-2020) 

CE 2023 
(2022-2023) 

Number of 
applications 

Number of 
applications 
funded 

111 

103 

13 

12 

97 

9 

93 

9 

101 

11 

Success rate 

12% 

12% 

9% 

10% 

11% 

Total 
funding 
(AUD) 

255,900,00 

284,999,996 

283,500,000 

308,790,287 

384,905,274 

Table 6. ARC Centres of Excellence (CE) funding, by round. Source: ARC selection outcome reports. 

The  table  below  lists  all  of  the  applications  approved  for  ARC  Centres  of  Excellence 
commencing  in  2023.  Overall,  ARC’s  financial  contribution  to  successful  applications 
accounts  for  just  over  one-third  (36%)  of  the  total  contributions  made;  however,  all  of  the 
successful applications individually received the maximum amount of funding that ARC can 
offer (approximately 5,000,000 per year). In previous funding rounds, the total amounts of 
funding approved by the ARC for successful applications were slightly lower. 

Research Hub title 

Administering 
organisation 

ARC funds 
approved over the 
project lifetime 
(AUD) 

Contributions by 
participating 
organisations (AUD) 

ARC Centre of 
Excellence for 
Indigenous and 
Environmental 
Histories and Futures 

ARC Centre of 
Excellence for the 
Elimination of 
Violence Against 
Women 

ARC Centre of 
Excellence for the 
Weather of the 21st 
Century 

James Cook 
University 

35,000,000 

54,070,197 

Monash University 

69,999,990 

78,693,634 

802 

 
Research Hub title 

Administering 
organisation 

ARC funds 
approved over the 
project lifetime 
(AUD) 

Contributions by 
participating 
organisations (AUD) 

RMIT University 

34,948,820 

37,278,565 

Swinburne University 

35,000,000 

236,174,241 

University of Adelaide 

35,000,000 

53,450,715 

University of 
Melbourne 

35,000,000 

46,423,476 

University of New 
South Wales 

35,000,000 

42,575,848 

University of 
Queensland 

104,956,464 

131,119,372 

ARC Centre of 
Excellence in Optical 
Microcombs for 
Breakthrough Science 

ARC Centre of 
Excellence for 
Gravitational Wave 
Discovery of 
Technology 

ARC Centre of 
Excellence in Plants 
for Space 

ARC Centre of 
Excellence for the 
Mathematical 
Analysis of Cellular 
Systems 

ARC Centre of 
Excellence for Carbon 
Science and 
Innovation 

ARC Centre of 
Excellence for Green 
Electrochemical 
Transformation of 
Carbon Dioxide 

ARC Centre of 
Excellence for 
Indigenous Futures 

ARC Centre of 
Excellence in 
Quantum 
Biotechnology 

Total funding (AUD) 

384,905,274 

679,786,048 

Table 7. ARC Centres of Excellence (CE) funded in 2023. Source: ARC selection outcome reports. 

The graph below shows the distribution of funding by discipline panel each year. It can be 
noted  that  funding  for  the  MPCE  panel  only  commenced  in  2020,  with  no  funding  for  this 
discipline  being  allocated  in  the  previous  rounds  of  the  scheme.  Similarly,  a  substantial 
portion of funding to the PCE panel is observed for the years 2011, 2014 and 2017 with no 
funding  being  allocated  in  subsequent  years.  Overall,  as  shown  in  the  figure,  there  are 

803 

 
 
considerable variations in the distribution of funding across the different discipline panels over 
time. 

100%

80%

60%

40%

20%

0%

15%

54%

8%
8%
15%

2011

8%

25%

8%

25%

33%

2014

11%

78%

11%

2017

22%

22%

22%

11%

22%

2020

55%

9%

18%

9%
9%

2023

BSB

EMI

HCA

PCE

SBE

EIC MPCE

Figure 6. ARC Centres of Excellence funding distribution, by discipline panel. Source: ARC selection outcome reports. 

Note: BSB, Biological Sciences and Biotechnology; EIC, Engineering, Information and Computing Sciences; HCA, 
Humanities and Creative Arts; MPCE, Mathematics, Physics, Chemistry and Earth Sciences; SBE, Social, Behavioural and 
Economics Sciences; EMI, Engineering, Mathematics and Informatics; PCE, Physics, Chemistry and Earth Sciences. 

The Linkage Infrastructure, Equipment and Facilities (LIEF) scheme1277 is designed to 
provide funding to eligible organisations for research infrastructures, equipment and facilities. 
By  participating  in  collaborative  initiatives,  expensive  national  or  international  research 
infrastructures,  equipment  and  facilities  can  be  shared  between  industry  and  higher 
education organisations. As part of the scheme, ARC has also created the LIEF Register, 
which provides an overview of the infrastructures, equipment and facilities that have been 
funded so far under the LIEF scheme. The Register aims to prevent duplicative applications.  

The table below provides an overview of the LIEF scheme funding, by funding round. As can 
be seen, the numbers of applications and numbers of awards have decreased slightly over 
the years. Meanwhile, the total amount of funding allocated has increased by more than 22% 
from 2020 to 2023. 

Linkage 
infrastructures, 
equipment and 
facilities (LIEF) 

LIEF 2020 
(2019-2020) 

LIEF 2021 
(2020-2021) 

LIEF 2022 
(2021-2022) 

LIEF 2023 
(2022-2023) 

Number of 
applications 

164 

Number of awards 

47 

138 

47 

138 

46 

133 

45 

1277 Australian Research Council, Linkage Infrastructure, Equipment and Facilities (LIEF), n.d., 
available at: https://www.arc.gov.au/funding-research/funding-schemes/linkage-program/linkage-
infrastructure-equipment-and-facilities 

804 

 
 
 
Linkage 
infrastructures, 
equipment and 
facilities (LIEF) 

LIEF 2020 
(2019-2020) 

LIEF 2021 
(2020-2021) 

LIEF 2022 
(2021-2022) 

LIEF 2023 
(2022-2023) 

Success rate 

28.7% 

34.1% 

33.3% 

33.8% 

Total funding 
(AUD) 

30,672,752 

38,827,812 

32,611,722 

39,515,942 

Table 8. Linkage Infrastructures, Equipment and Facilities (LIEF) funding, by scheme round. Source: ARC selection 
outcome reports. 

The LIEF scheme has funded successful applications under five discipline panels (see figure 
below).  Throughout  the  years,  the  lowest  amounts  of  funding  have  been  allocated  to  the 
Social,  Behavioural  and  Economics  Sciences  (SBE)  panel,  while  the  highest  shares  of 
funding  have  gone  to  applications  under  the  Mathematics,  Physics,  Chemistry  and  Earth 
Sciences (MPCE) panel. 

100%

80%

60%

40%

20%

0%

2%

49%

2%

30%

17%

2020

2%

45%

11%

28%

15%

2021

4%

39%

4%

41%

11%

2022

2%

31%

7%

40%

20%

2023

BSB

EIC

HCA MPCE

SBE

Figure 7. Linkage Infrastructures, Equipment and Facilities (LIEF) funding by a discipline panel. Source: ARC selection 
outcome reports. 

Note: BSB, Biological Sciences and Biotechnology; EIC, Engineering, Information and Computing Sciences; HCA, 
Humanities and Creative Arts; MPCE, Mathematics, Physics, Chemistry and Earth Sciences; SBE, Social, Behavioural and 
Economics Sciences. 

The  Linkage  Projects  (LP)  scheme1278  focuses  on  supporting  internationally  competitive 
research projects and teams to address challenges or opportunities of relevance to research 
end-users.  The  scheme  fosters  the  establishment  and strengthening  of  research  alliances 
between  higher  education  organisations  and  research  end-users  to  apply  advanced 
knowledge and ideas as a basis for securing commercial and other benefits of research. 

For the  Linkage  Project scheme,  the  number of  approved  applications  has  grown steadily 
from 175 in 2019 to 181 in 2021. The highest success rate – almost 41% – can be observed 
in 2019. A total of AUD 78,187,269 were allocated in that year. 

1278 Australian Research Council, Linkage Projects, n.d., available at: 
https://www.arc.gov.au/funding-research/funding-schemes/linkage-program/linkage-projects 

805 

 
 
 
Linkage projects (LP) 

LP 2019 
(2019-2020) 

LP 2020 
(2020-2021) 

LP 2021 
(2021-2022) 

Number of applications 

Number of awards 

432 

175 

715 

195 

Success rate 

40.5% 

27.4% 

569 

181 

32% 

Total funding (AUD) 

78,187,269 

90,170,327 

85,906,568 

Table 9. Linkage Projects (LP) funding by scheme round. Source: ARC selection outcome reports. 

The  graph  below  shows  the  shares  of  successful  Linkage  Projects  by  discipline  panel. 
Interestingly, over the years, only four disciplines have been represented: 1) BEM, Biological 
Sciences, Biotechnology, Environmental, Medical and Health Sciences; 2) ET, Engineering 
and Technology; 3) HSE, Humanities and Creative Arts, Social, Behavioural and Economic 
Sciences; and (4) PMI, Physical, Mathematical and Information Sciences. The distribution of 
funding by discipline panel is fairly equal, with slightly greater focus being given to the HSE 
and BEM panels. 

100%

80%

60%

40%

20%

0%

16%

39%

28%

17%

2019

15%

33%

33%

18%

2020

22%

25%

31%

22%

2021

BEM ET

HSE

PMI

Figure 8.Linkage Projects (LP) distribution of funding, by discipline panel. Source: ARC selection outcome reports. 

The Special Research Initiative (SRI) scheme1279, administered by the NCGP, is designed 
to  provide  funding  to  support  new  fields  of  research  that  can  improve  Australia’s  global 
competitiveness and address the government’s key priority areas. For instance, the scheme 
that ran in 2018 focused on proposals for projects exploring the development of innovative 
technologies,  as  well  as  projects  to  investigate  and  remediate  per-  and  poly-fluoroalkyl 
substance (PFAS)-contaminated areas, including soil, groundwater, waterways and marine 
systems.  In  2020,  round  1  of  the  scheme  was  directed  towards  projects  that  focused  on 
developing  Australia’s  research  excellence  in  Antarctic  Science  and  supported  excellent 
research in one or more of the Australian government’s Antarctic Science priority research 

1279 Australian Research Council, Special Research Initiatives n.d., available at: 
https://www.arc.gov.au/funding-research/funding-schemes/linkage-program/special-research-
initiatives 

806 

 
 
 
themes.  Another  strand,  round  2,  was  targeted  towards  advancing  understanding  of 
Australian and Aboriginal and Torres Strait Islander society, history and culture. 

Special Research Initiatives 
(SRI) 

SRI 2020 round 1 
(2019-2020) 

SRI 2021 round 2 
(2020-2021) 

Number of applications 

Number of awards 

Success rate 

20 

2 

10% 

692 

49 

7.1% 

Total funding (AUD) 

56,000,000 

11,759,398 

Table 10. Special Research Initiatives (SRI) funding, by round. Source: ARC selection outcome reports. 

1.4.7.  Effectiveness and Efficiency: monitoring impact 

The  ARC  implements  the  Engagement  and  Impact  Assessment  process,  which  evaluates 
how well researchers engage with the end-users of research. This process aims to provide 
an overall assessment of research carried out in Australia as a whole, Including but not limited 
to research carried out under funding grants from ARC programmes. It also assesses how 
well  universities  translate  research  into  impacts  beyond  academia,  including  economic, 
social,  environmental  and  cultural  impacts.  The  inaugural  Engagement  and  Impact 
Assessment  took  place  in  2018.  The  following  deliverables  are  produced  as  part  of  the 
process: 

•  A national assessment report; 

•  Engagement narratives that received a ‘high’ rating by the experts; 

• 

Impact studies that received a ‘high’ rating by the experts; and 

•  Engagement and impact outcomes: ratings in terms of engagement and impact (low, 

moderate, high) for Fields of Research at each institution.  

Interviews with ARC stakeholders revealed that currently, there is no direct measurement at 
agency  level  of  the  impacts  on  research  or  society  of  funding  from  Linkage  Program. 
However, while long-term assessment of research impacts may not be systematically tracked 
by the funding agency, universities have their own assessment processes in place Hence, 
although universities may produce their own reports regarding the impact of funded research, 
there is no centralised impact assessment mechanism with regard to the Linkage Program. 
Past attempts to assess engagement and impact have been made, but were discontinued. 
These  findings  highlight  the  need  for  a  more  systematic  and  comprehensive  approach  to 
assessing  the  broader  impacts  of  ARC-funded  research  on  society  and  various  sectors. 
Establishing  a  structured  evaluation  framework  could  provide  valuable  insights  into  the 
effectiveness and contributions of ARC-supported projects. 

807 

 
1.5.  Critical success factors and barriers 

The critical success factors identified with regard to the Linkage Program are as follows: 

•  Sufficient focus on researchers’ professional development and transferable skills: 
participation in the programme leads to heightened visibility, improved leadership skills 
and  increased  academic  contributions,  underlining  its  positive  impact  on  researchers’ 
professional development through collaborations with industry partners.  

•  Merit-based funding decisions: funding decisions are primarily based on the merits of 
individual researchers and projects, rather than the overall performance of their affiliated 
institutions, thereby ensuring a level playing field. Stakeholders emphasised that Linkage 
Program  funding  schemes  do  not  differentiate  on  the  basis  of  the  institution,  thus 
promoting fairness and equal opportunity. 

•  Prestigious  reputation:  the  Linkage  Program  is  highly  regarded  within  the  academic 
and research community. From the standpoint of excellence, the programme is known 
for funding and supporting high-quality, impactful research projects. Researchers who 
secure  funding  through  the  Linkage  Program  are  often  seen  as  achieving  a  notable 
milestone in their academic careers, as it represents an endorsement of the excellence 
and significance of their work. 

•  Restriction  on  the  inclusion  of  overseas  researchers:  Linkage  Program  funding  is 
exclusively available to researchers based in Australia. While international researchers 
are  eligible  to  apply,  they  must  hold  resident  status  in  the  country  in  order  to  be 
considered. 

•  Clear  focus  on  industry-relevant  issues:  the  effectiveness  of  the  Linkage  Program 
stems from its sharp focus on addressing the real-world challenges faced by industry. 
This targeted approach ensures that research efforts are aligned with practical needs, 
increasing the likelihood of successful outcomes. 

• 

for  commitment: 

Incentives 
from  participating 
organisations  enables  them  to  demonstrate  a  genuine  commitment  to  the  project’s 
success.  This  cost-sharing  mechanism  helps  to  distribute  financial  responsibility,  and 
ensures that all stakeholders are invested in the project’s outcomes. 

requiring  cash  contributions 

The following critical barriers to the operations of the Linkage Program were identified: 

•  Complex application process: the application process for Linkage Program grants is 
exceptionally detailed and extensive, posing a significant challenge for both researchers 
as well as industry partners, who may seek a more streamlined approach. 

•  Lack  of  mechanism  for  impact  assessment:  the  lack  of  a  centralised  impact 
assessment  mechanism  makes  it  difficult  to  evaluate  the  broader  societal  impacts  of 
Linkage Programme-funded research projects. 

•  Lack of oversight of project implementation by the managing authority: a lack of 
oversight with regard to project implementation may hinder the successful execution of 
projects,  potentially  leading to  inefficiencies,  financial  risks  and  a  diminished  ability  to 
demonstrate the positive impacts of the initiatives. 

808 

 
1.6. 

Transferability of findings to the EU Framework Programme 

The  Linkage  Program's  approach  of  allowing  additional  members  to  join  ongoing  projects 
serves as a valuable example of good practice that could be further integrated into the EU 
Framework Programme. Similar approaches are found under Horizon Europe – specifically 
in the context of the ‘hop-on’ facility. This allows legal entities from countries that are low-
performing in terms of R&I to join collaborative R&I actions that have already been selected, 
subject to agreement by the respective consortium and provided that legal entities from such 
countries are not already participating in the action. However, this approach could be further 
explored and transferred to other funding programmes in order to enhance their inclusiveness 
and increase their added value.  

The Linkage Program’s requirement for partner organisations to contribute cash and/or in-
kind support to funded projects presents a commendable model that could be successfully 
integrated  into  the  EU  Framework  Programme.  This  approach  has  demonstrated  its 
effectiveness  over  a  track  record  of  approximately  15  years.  By  mandating  such 
contributions,  the  EU  Framework  Programme  could  achieve  several  important  objectives. 
First, such a requirement ensures a clear demonstration of commitment from participating 
organisations, underlining that they take participation in the project seriously and that it enjoys 
tangible support. In addition, leveraging resources through cash contributions amplifies the 
impact  of  government  funding,  allowing  a  larger  overall  budget  that  can  result  in  more 
extensive  research,  enhanced  equipment,  and  broader  outreach  efforts.  Moreover,  this 
approach fosters sustainability, as organisations with a financial stake in the project are more 
likely  to  remain  dedicated  to  sustaining  the  project  outcomes  beyond  the  funding  period. 
Alongside cash, in-kind contributions add further value by providing resources, equipment or 
services pertinent to the project, complementing and enhancing its overall impact. 

1.7.  Conclusions 

This benchmark report indicates that the Australian Research Council’s Linkage Program is 
successful in fostering productive collaborations between academia and industry, effectively 
tackling the real-world challenges faced by the industrial sector. While  acknowledging that 
access to certain critical documents has been limited, it is important to note that this report 
provides  a  nuanced  (albeit  not  comprehensive)  overview  of  the  programme’s  portfolio. 
Notably,  the  absence  of  systematic  monitoring  or  assessment  of  grant  impacts  by  the 
Australian Research Council limits the scope for conclusive insights into project outcomes. 
Nevertheless,  it is  evident  that  the  Linkage  Program  embodies  a  well-balanced  approach, 
catering  to  a  wide  spectrum  of  research  needs  and  target  groups.  Significantly,  the 
requirement for partner organisations to contribute tangible support to the funded projects, 
alongside the flexibility for additional partners to join ongoing projects, emerge as standout 
practices. These approaches are worthy of further exploration, with a view to their potential 
integration into the EU Framework Programme in order to bring further benefits in terms of 
enhanced inclusivity and additional added value. 

Appendix 1. Literature sources 
•  Australian Research Council, Appeals against NCGP administrative processes, n.d., 

available at: https://www.arc.gov.au/about-arc/our-
organisation/committees/appeals-against-ncgp-administrative-processes  

•  Australian Research Council, ARC Centres of Excellence, n.d., available at: 

https://www.arc.gov.au/funding-research/funding-schemes/linkage-program/arc-
centres-excellence  

809 

 
•  Australian Research Council, Australian Research Council Annual Report 2021-22, 

Transparency Portal, 2023b (February 10), available at: 
https://www.transparency.gov.au/annual-reports/australian-research-
council/reporting-year/2021-22 

•  Australian Research Council, GrantConnect – the Australian Government’s grant 

information system, n.d., available at: https://www.arc.gov.au/funding-
research/apply-funding/grantconnect 

•  Australian Research Council, Industrial Transformation Research Hubs, n.d.-b, 

available at: https://www.arc.gov.au/funding-research/funding-schemes/linkage-
program/industrial-transformation-research-program/industrial-transformation-
research-hubs  

•  Australian Research Council, Industrial Transformation Research Program, n.d., 

available at: https://www.arc.gov.au/funding-research/funding-schemes/linkage-
program/industrial-transformation-research-program  

•  Australian Research Council, Industrial Transformation Training Centres, n.d., available 

at: https://www.arc.gov.au/funding-research/funding-schemes/linkage-
program/industrial-transformation-research-program/industrial-transformation-
training-centres  

•  Australian Research Council, Linkage Infrastructure, Equipment and Facilities (LIEF), 

n.d., available at: https://www.arc.gov.au/funding-research/funding-
schemes/linkage-program/linkage-infrastructure-equipment-and-facilities  

•  Australian Research Council, Linkage Projects, n.d., available at: 

https://www.arc.gov.au/funding-research/funding-schemes/linkage-
program/linkage-projects   

•  Australian Research Council, Research Management System (RMS) Information, n.d., 
available at: https://www.arc.gov.au/manage-your-grant/research-management-
system-rms-information  

•  Australian Research Council, Selection Advisory Committees, n.d., available at: 

https://www.arc.gov.au/about/our-organisation/committees/selection-advisory-
committees  

•  Australian Research Council, Selection Outcome Reports, n.d., available at: 

https://www.arc.gov.au/funding-research/funding-outcome/selection-outcome-
reports  

•  Australian Research Council, Special Research Initiatives, n.d., available at: 
https://www.arc.gov.au/funding-research/funding-schemes/linkage-
program/special-research-initiatives  

810 

 
Annex 4: Evidence table by programme part and evaluation criteria 

This  document  presents  the  key  findings  emerging  from  the  analysis  and  data  collection 
carried  out  for  the  present  study.  These  findings  are  presented  as  direct  answers  to  the 
specific Evaluation Questions for this study. 

811 

 
 
Methods 
and 
analyses 

Literature 
review 

Monitoring 
data 

Interviews 

1.1.  Relevance 

Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

RV1* 

How relevant have the 
Framework Programme 
activities, including 
partnerships, been to 
deliver excellent 
science given the 
stakeholders’ needs 
and considering the 
scientific, technological 
and/or socio-economic 
problems and issues 
identified at the time of 
its design and over 
time? 

The strength of the logical 
links between the 
objectives of the 
programmes analysed, 
and the policy and socio-
economic needs/problems 
identified in strategic policy 
documents. 

Analysis of the programme 
design (‘bottom-up’ versus 
‘top-down’ programmes 
and calls), and how well 
they fit with stakeholders’ 
needs. 

The extent to which policy 
feedback has been 
integrated into programme 
design/annual work 
programmes. 

A breakdown of resources, 
participation and spending 
in comparison to the 
priorities set. 

Key findings 

Evidence 

ERC: Under Horizon Europe, the ERC continues to operate on a 'bottom-up' 
basis without predetermined priorities, with the objective of providing attractive 
long-term funding to support excellent investigators and their research teams 
to pursue ground-breaking, high-risk/high-gain research. Evidence compiled 
from interviews with ERC beneficiaries and from the position papers submitted 
to the open public consultation shows that this approach is highly appreciated 
by beneficiaries. For example, several position papers emphasise the 
importance of researchers being able to formulate their own paths of 
investigation and the ‘blue-sky’ nature of ERC calls, which gives more scope 
to push the boundaries of research disciplines – a prelude to delivering 
excellent science and frontier research. This approach is also very attractive to 
top researchers, and hence contributes to achieving the objectives of the ERC. 
In addition, 96% (231 out of 241 respondents) of the successful ERC 
applicants who responded to the survey confirmed that their ERC grant, to a 
‘very large’ or ‘large’ extent, gives them the freedom to pursue their specific 
research agenda, while 95% (229 out of 241) indicated that it gives them 
opportunities to conduct basic/fundamental research (ERC-BEN-025). 

The ERC Scientific Council has made changes to the ERC panel structure in 
order to ensure balance between panels in terms of the number of proposals 
assessed in each call, the disciplinary coherence within and across panels, 
and the place of interdisciplinary research within the panel structure.  

MSCA: The MSCA are highly relevant in nurturing excellence through the 
mobility of researchers across borders, sectors and disciplines. The 
programme's setup and guiding principles actively support international, 
intersectoral and interdisciplinary mobility through initiatives such as 
Postdoctoral Fellowships, Doctoral Networks, Staff Exchanges and COFUND. 

812 

 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

The MSCA encourage researchers from any nationality, within or outside 
Europe, to pursue outstanding research and to advance their skills and 
careers. Under Horizon Europe, the MSCA account for the highest share of 
third country participations across programme parts in Pillar 1 and WIDERA. 
They also attract a substantial number of newcomers (14%), with the majority 
originating from private for-profit organisations. 

Survey findings confirm the relevance of the MSCA in fostering the cross-
border, intersectoral and interdisciplinary mobility of researchers. The majority 
of successful MSCA applicant organisations and successful PF applicants 
surveyed agreed to a ‘very large’ or ‘large’ extent that the MSCA provide 
sufficient funding opportunities to conduct interdisciplinary research (68% or 
338 out of 493, for beneficiary organisations; 77% or 628 out of 820 for PF 
fellows) and academia-industry collaborative research (70% or 334 out of 491 
for beneficiary organisations; 52% or 430 out of 821 for PF fellows). In parallel, 
the majority of successful MSCA PF applicants find that the international 
mobility opportunities (87%, or 719 out of 825) and interdisciplinary 
cooperation opportunities (77% or 628 out of 824) provided by the MSCA 
project respond to their needs to a ‘very large’ or ‘large’ extent. 

The MSCA programme has also considered feedback from various 
stakeholders, resulting in a few changes under Horizon Europe compared to 
the previous Framework Programme. Postdoctoral Fellowships have 
expanded their provisions to include an optional placement of up to six months 
in a non-academic organisation located in an EU Member State or Horizon 
Europe associated country, with the placement taking place at the end of the 
fellowship. Under Horizon Europe, the Staff Exchanges action enables 
interdisciplinary secondments within Europe, resulting in a reduction in person-
months to a maximum of 360 to enhance project manageability. Within the 
COFUND action, the MSCA now cover minimum remuneration, which can be 
used more flexibly. In addition, COFUND applications that score at least 85%, 

813 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

but do not secure funding due to budget constraints are now eligible for the 
prestigious Seal of Excellence quality label. 

Under Horizon Europe, all MSCA fellows now benefit from a newly introduced 
long-term leave allowance, specific needs allowances for individuals with 
disabilities, and accessible family allowances for all recipients. Furthermore, 
the MSCA place a heightened emphasis on effective supervision, adequate 
mentoring and career guidance, as underlined by the introduction of the MSCA 
Guidelines on Supervision. Complementing the objectives of the European 
Green Deal and maintaining the MSCA's organic approach, the European 
Commission has introduced the MSCA Green Charter, which promotes the 
sustainable implementation of research activities. Under Horizon Europe, the 
MSCA continue to operate on a bottom-up basis, providing full flexibility for 
researchers to come up with their own research topics. This approach is highly 
appreciated by the beneficiaries. The MSCA are open to all domains of 
research and innovation, chosen freely by the applicants in a fully bottom-up 
manner, addressed under the Treaty on the Functioning of the European 
Union. The MSCA complement top-down collaborative research activities, 
notably contributing to the EU Missions.  

INFRA: it is stressed in the new ERA that research infrastructures (Ris) are 
critical for a well-functioning research and innovation system in Europe. This is 
also apparent in the concrete actions of the ERA, notably the action 
‘strengthening RIs’1280. It was also confirmed in interviews that developing 
world-class pan-EU RIs is important for Europe's competitiveness and its 
scientific excellence, as well as its ability to address societal challenges.  

1280 Available at: https://research-and-innovation.ec.europa.eu/strategy/strategy-2020-2024/our-digital-future/european-research-area_en. 

814 

 
 
 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Key findings 

Evidence 

Methods 
and 
analyses 

WIDERA: 

Widening: The Widening actions mainly have a bottom-up approach, with 
researchers having flexibility over the topics to be researched with the support 
of the programme. The stakeholders and beneficiaries consulted, from both 
Horizon 2020 and Horizon Europe, valued the bottom-up collaborative 
opportunities provided by the programme part. Nevertheless, under Horizon 
Europe, in addition to their “classic” bottom-up approach, some actions also 
include calls related to more dedicated themes, for instance the Twinning 
Green Deal call. In line with EU priorities, this will spur projects in the following 
topics: climate research, green technologies, renewable energy, sustainable 
mobility, biodiversity research and the sustainable use of natural resources 
(land, water, air).  

ERA: The ‘ERA’ destination of the WIDERA programme is highly relevant and 
corresponds closely to the key European strategic priorities relating to 
enhancing the European R&I system, including supporting investments and 
reforms in research and innovation, improving access to excellence, 
translating R&I into the economy, and deepening the integration and 
coordination between national R&I systems. 

EOSC Partnership: According to interviewees, the three objectives of the 
EOSC Partnership, as described in 2021 in the SRIA and the Memorandum of 
Understanding (MoU), remain relevant. However, some interviewees called for 
more concrete objectives. The benefits of working together, combining 
research results and avoiding duplication also remain important, and are even 
increasing in importance. The three objectives of the EOSC Partnership also 
refer to the various answers to the question ‘What is the EOSC?’, and many 
aspects have not yet been implemented. The partnership itself is also still 
considered relevant. As one interviewee put it: ‘The partnership is a pre-
requisite if we want to achieve the objectives of EOSC.’ 

815 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

RV2* 

To what extent have 
the supported activities, 
including partnerships, 
taken into account the 
latest technological, 
scientific and/or socio-
economic 
developments at the 
national, European and 
international level? 
What are the emerging 
needs that the 
Framework Programme 
has not covered? 

The extent to which the 
design of the FP actions 
allows adaptability to 
developments or emerging 
needs. 

Systematic mapping of the 
latest technological, 
scientific and/or socio-
economic developments at 
national/European and 
international levels. 

The extent to which the FP 
projects/programmes 
analysed refer to these 
developments; the 
contribution of the FP to 
addressing these 
developments; an 
overview of the scientific 
fields covered and of 
overlaps with emerging 
(versus stable or declining) 
fields and needs. 

Identification of gaps and 
emerging needs that are 
not covered. 

Key findings 

Evidence 

ERC: According to the Horizon Dashboard (accessed 6 September 2023), 
under Horizon Europe, the field of science with the greatest number of ERC 
signed grants is natural sciences, followed by medical and health sciences, 
and engineering and technology. In closer detail, the fields of proteins, 
oncology and climate changes are those with the most signed grants. In terms 
of participation, using a keyword methodology, genomics, physical chemistry 
and quantum optics and quantum information were identified as the fields with 
the most participation.  

Through its ‘bottom-up’ approach, the ERC was able to fund research 
essential to tackling emerging needs. For example, at the height of the 
COVID-19 pandemic, over 180 ongoing or completed ERC projects generated 
knowledge and technology that contributed to the response to this pandemic, 
including diagnostics and treatments, medical devices or immunity, infection 
and pathology. In addition, the ERC has contributed to the EU priorities 
‘Europe fit for the digital age’ and the European Green Deal (EGD).  

53% of the ERC successful applicants (123 out of 230) who replied to the 
survey (ERC-BEN-044) ‘strongly agreed’ or ‘rather agreed’ that, compared 
with the research funding available at national and/or regional level, Horizon 
Europe helps to better address grand societal challenges (e.g. climate change, 
demographics and ageing, etc.). 

MSCA: The MSCA also showed a high degree of responsiveness and 
flexibility in addressing the newly emerging needs resulting from challenges 
such as the Russian invasion of Ukraine and the COVID-19 crisis. In response 
to the Russian invasion of Ukraine, the MSCA has taken proactive steps to 
support affected doctoral candidates and postdoctoral researchers from 
Ukraine. Through the dedicated MSCA4Ukraine scheme launched in 2022, a 
funding allocation of EUR 25 million has been designated to aid displaced 
researchers, allowing them to continue their scientific pursuits in Europe and 

Methods 
and 
analyses 

Literature 
review 

Monitoring 
data 

Analysis of 
research 
topics 

Interviews 

Survey 

Case 
studies 

Benchmarks 

816 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

to contribute to the rebuilding of Ukraine's R&I capacity when conditions 
permit. The scheme received over 600 applications from 28 countries, and 111 
post-doctoral and 13 doctoral fellowships were awarded to researchers from 
Ukraine. 

In early 2021, REA conducted a survey to gather information about the 
challenges faced by participating organisations managing MSCA projects 
during the pandemic. The survey aimed to assess the effectiveness of 
measures taken by the European Commission and REA to provide support 
during similar future situations. The findings suggest that the proposed 
flexibility measures were generally seen as appropriate and effective, allowing 
organisations to adapt to restrictions and constraints. Most respondents to the 
survey viewed positively the ability to fully pay MSCA researchers, even if they 
could not work at the host organisation’s premises. Respondents also 
considered the extended flexibility regarding project deliverables and 
milestones as appropriate. It was also found that the communication relating to 
these flexibility measures was generally timely and was supported by REA 
project officers.  

As an entirely ‘bottom-up’ and ideas-driven programme, the MSCA provide 
great flexibility for research creativity in all scientific fields, leading to 
significant and innovative contributions to major societal challenges and 
Commission priorities, including emerging ones such as the EU’s response to 
COVID-19. As discussed in DG EAC’s strategic plan for 2020-2024, there are 
already more than 100 MSCA projects directly related to COVID-19 research 
issues alone, and the MSCA can support many more projects in the years to 
come. The current COVID-19 crisis dramatically highlights the critical need to 
provide researchers with the right skills, including the necessary resilience 
skills, and to facilitate their outreach and networking, not only to address 
immediate health challenges, but also to anticipate the broader societal, 

817 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

ecological and economic implications of the outbreak and to enhance Europe’s 
preparedness in the future. 

Based on data from the Horizon Europe Dashboard, the most-researched 
topics under all MSCA signed grants relate to AI (artificial intelligence), 
including machine learning, statistical data processing and applications that 
use signal processing (e.g. speech, image, video), numerical analysis, 
simulation optimisation, and modelling tools. In terms of research fields, the 
largest share of MSCA projects fall within the field of life sciences (23%), 
followed by engineering (22%) and social sciences (19%).The results so far 
indicate that the projects signed are well in line with the European 
Commission’s priority areas and main orientations.  

INFRA: Based on data from the Horizon Europe Dashboard (accessed August 
2023), the most-covered fields of science under INFRA are natural sciences 
(65% of signed grants) followed by social sciences (31%). Fields of science 
that are less well covered include engineering and technology (18%), medical 
and health sciences (16%), humanities (6%) and agricultural sciences (6%).  

An important document in ensuring the relevance of INFRA in terms of 
addressing scientific and socio-economic developments is the Landscape 
Analysis performed by ESFRI. Until now, this document was part of the ESFRI 
Roadmap, and identified gaps and needs in the RI landscape. These 
emerging needs were then covered immediately in the ESFRI projects 
identified in the roadmap, which are funded through INFRA. Interviews with 
stakeholders confirm that other aspects included in the previous Landscape 
Analysis were covered by INFRA, such as the need for interdisciplinary 
collaboration between RIs to address societal challenges. In 2024, the most 
comprehensive Landscape Analysis to date is expected. This will be based on 
very broad stakeholder consultations, and will be key to ensuring the 
relevance of future INFRA work programmes.  

818 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

There are no indications of any gaps or emerging needs that have not been 
covered by INFRA. Again, the next Landscape Analysis from ESFRI will be 
instrumental to ensuring INFRA responds to newly identified gaps and needs. 

WIDERA: 

Widening: According to data from the Horizon Dashboard (accessed June 
2023), the topics most widely dealt with in WIDERA (Widening) signed grants 
are ecosystems, climate change, educational sciences, innovation 
management oncology, pollution, and the sustainable economy. The field in 
which by far the most projects are active is natural sciences. These findings 
indicate that Widening projects are largely in line with the latest technological, 
scientific and/or socio-economic developments, and address EU Missions 
such as beating cancer and tackling climate change. In addition, the analysis 
of research trends based on the publications generated by SEWP under 
Horizon 2020 indicates that SEWP has generated publications on prominent 
topics.  

Almost 70% of WIDERA (Widening) respondents to the beneficiary survey (82 
out of 169 respondents) indicated that, compared with the research funding 
available to participants at national and/or regional level, Horizon Europe 
better helps to address grand societal challenges (e.g. climate change, 
demographics and ageing, etc.). Only 4% (7 out of 169) ‘rather disagreed’ or 
‘strongly disagreed’ (Survey of Horizon Europe beneficiaries, conducted May-
July 2023).  

ERA: The analysis of WIDERA work programmes and data from the Horizon 
Dashboard reveal that the following topics are most widely covered under 
WIDERA calls (ERA destination): responsible research and innovation, 
ecosystems, pandemics, gender equality, civil society, employment, open 
access, climate change, research ethics and integrity. The field in which the 
majority of projects are carried out is social sciences. These preliminary 

819 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

findings indicate that the ERA destination of the WIDERA programme is 
closely aligned with recent social, economic and technological developments. 

WIDERA projects under the ‘Strengthening the ERA’ thematic priority were 
highly relevant to the needs of institutions to develop their capacities and the 
skills of their staff, in order to better respond to current socioeconomic needs, 
and to increase competitiveness and the overall knowledge base. Case study 
findings suggest that by, for example, helping universities to build the 
competencies needed by researchers and by facilitating the commercialisation 
of research findings, acceleration services make European higher education 
institutions (HEIs) better aligned with the needs of the market and more 
capable of forging alliances with the private sector. Likewise, 65% (66 out of 
102) of WIDERA (ERA) beneficiaries surveyed agreed to a ‘very large’ or 
‘large’ extent that their project responds to their institution’s need to enhance 
its R&D and/or innovation capabilities and skills, followed by the need to 
strengthen the knowledge base and R&D capacity in their organisation (63%, 
or 65 out of 102 respondents) and to enhance their competitiveness (61%, or 
62 out of 101 respondents). 

EOSC Partnership: Achievements have been made, but the three objectives 
have not yet been fully met. For example, as mentioned by one interviewee: 
‘Only about 2% of research data is findable on the internet’. Meanwhile, the 
EOSC minimal viable infrastructure has still to be constructed. 

RV3* 

Has the Framework 
Programme tackled the 
right issues given the 
scientific positioning of 
the European Union 

Analysis of important 
scientific, technological 
and socioeconomic 
problems and issues and 
how they have changed 
over time between 2014 

Literature 
review 

Monitoring 
data 

ERC: Despite its sole focus on excellent science and its ‘bottom-up’ approach, 
the ERC has contributed to the EU priorities of ‘Europe fit for the digital age’ 
and the European Green Deal (EGD). In particular, under Horizon 2020, ERC 
funded over 700 projects, worth more than EUR 1 billion, in the fields of 
artificial intelligence, quantum technologies or the digital age. Quantum 
technologies continued to be the top funded field under Horizon Europe. In the 

820 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

since the programme 
started and over time? 

and 2020 and beyond (EU 
strategic and policy 
documents, statistical 
data, equivalent 
documents from 
international organisations 
such as the OECD, UNDP, 
US institutions) – 
concordance between EU 
FP objectives and global 
trends. 

Timeliness of research: 
estimated share of 
analysed FP 
projects/programmes that 
have addressed scientific 
topics while these were at 
a nascent stage. 

Key findings 

Evidence 

case of the EGD, Horizon 2020 funded over 950 projects, worth more than 
EUR 2 billion, in the fields of clean, affordable and secure energy, farm-to-fork 
and biodiversity. Compared with Horizon 2020, genomics, climate change and 
machine learning appear more prominent under Horizon Europe, which shows 
the increasing momentum of AI and climate change in the research landscape. 

MSCA: As mentioned above, based on data from the Horizon Europe 
Dashboard, the most widely researched topics under all MSCA signed grants 
relate to AI (artificial intelligence) such as machine learning, statistical data 
processing and applications that use signal processing (e.g. speech, image, 
video), numerical analysis, simulation optimisation and modelling tools. In 
terms of research fields, the biggest share of MSCA projects fall under the life 
sciences field (23%), followed by engineering (22%) and social sciences 
(19%). The results so far indicate that the projects signed are well in line with 
the European Commission’s priority areas and main orientations. 

INFRA: Whereas the approach of INFRA under Horizon 2020was generally 
bottom-up, the approach under Horizon Europe is more challenge-oriented. 
This means that some topics focus on addressing specific societal challenges 
– especially those relating to the digital and green transitions, but also to 
cancer, for example. A complete overview of challenge-oriented topics is 
provided in Case Study 6. These challenges are closely related to EU priorities 
and also the Horizon Missions, and are thus highly relevant.  

WIDERA: 

Widening: As mentioned above, Widening projects are largely in line with the 
latest technological, scientific and/or socio-economic developments, and 
address the EU Missions such as beating cancer and tackling climate change. 
Machine learning and artificial intelligence (AI) are also common. ‘Oncology’ 
was also one of the most frequent topics under Horizon 2020 and ‘climatic 
change’ was the 10th most frequent topic, while machine learning and AI were 

Methods 
and 
analyses 

Interviews 

Case 
studies 

Bibliometric 
analysis  
(Horizon 
2020) 

821 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

studied less often (all data from Horizon Dashboard, accessed June 2023). 
This suggests that climate change, machine learning and AI are gaining more 
interest (at least at this stage of WIDERA).  

ERA: As already indicated, ERA projects addressed a number of topics that 
are highly relevant to tackling the most relevant societal issues of recent 
years, e.g. climate change, gender equality, democracy, pandemics, Open 
Science, research ethics, etc. The WIDERA Work Programme 2023-24 
included a specific call under the ERA destination to address the topic of AI 
(‘Next generation AI and Human Behaviour: promoting an ethical approach’). 

RV4.1* 

To what extent have 
the Framework 
Programme activities to 
deliver excellent 
science addressed the 
needs of groups 
targeted for 
application/participation 
in terms of tools and 
thematic areas 
covered?  

Literature 
review 

Monitoring 
data 

Interviews 

Case 
studies 

The extent to which the 
needs of the target groups 
are addressed in terms of 
available tools and of 
thematic areas covered 
(and adaptability to 
changing needs); 
needs/gap analysis; 
appropriateness of 
activities’ design and 
implementation modes. 

Knowledge and 
awareness of supporting 
tools and thematic areas 
among the targeted 
groups. 

ERC: Under Horizon Europe, the ERC has maintained the same approach as 
under previous FPs of supporting excellent investigators and their research 
teams to pursue ground-breaking, high-risk/high-gain research, using a 
‘bottom-up’ approach. As mentioned above, evidence from interviews with 
ERC beneficiaries and position papers from the open consultation suggest that 
its approach is highly appreciated by beneficiaries.  

MSCA: The MSCA programme part has considered and aligned its focus to 
better reflect the European Commission’s key political priorities (2019-2024). 
With regard to this, the MSCA issued its ‘Guidelines for the Inclusion of 
Researchers at Risk’ in 2021. These Guidelines address the need to support 
researchers who face precarious situations or are at risk for political, social, or 
human rights-related reasons in their home countries. The Guidelines aim to 
provide a safe and supportive environment for these researchers, allowing 
them to continue their scientific work and to develop their careers without 
threat or constraint. The document contains useful background on researchers 
at risk, along with recommendations on improving their recruitment practices.  

822 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

In line with the goals of the European Green Deal and the Sustainable 
Development Goals (SDGs), the MSCA have also issued the MSCA Green 
Charter, a code of good practices for individuals and institutions receiving 
MSCA funding. The MSCA Green Charter is not binding; however, all MSCA 
projects at the final reporting stage will be asked to report on the ways in 
which they have sought to minimise the environmental impact of their research 
activities, and how they applied the principles of the Green Charter.  

Preliminary findings from interviews indicate that the MSCA achieves the 
desired structuring impacts in the overall delivery of Excellent Science in the 
EU. These impacts are most relevant in the areas of international cooperation, 
cooperation with the private sector, and research excellence. For widening 
countries, the MSCA proved to be relevant in narrowing the gap between the 
research infrastructures of widening and non-widening countries by providing 
a leveraged field for all EU researchers, and standardised sets of rules.  

The study finds that the research and innovation community are highly aware 
of the supporting tools available, and the thematic areas covered by the MSCA 
programme. In this regard, the MSCA Network of National Contact Points 
(MSCA-Net) plays a crucial role in assisting researchers and institutions in 
their respective countries with information, guidance and support relating to 
participation in the MSCA programme. The MSCA NCP Network consists of 
experts located in each EU Member State and in associated countries. These 
experts serve as central points of contact and provide a range of services, 
including information and guidance about different funding opportunities; 
proposal preparation and feedback; networking and partner search; as well as 
organising information events and workshops. 

According to interviewees, the support provided by the European Commission 
is highly appreciated, as it enables participating organisations and researchers 
to access various information channels concerning eligibility criteria, proposal 
preparation process, and available training courses. Interviewees also 

823 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

welcomed the responsiveness and support received from the national contact 
points (NCPs). However, it was observed that this support varies depending 
on the extent of an NCP’s experience.  

Nevertheless, organisations, researchers and NCPs face difficulties when 
attempting to navigate the extensive pool of information available, which 
interviewees say is widely scattered. Greater centralisation would be 
welcomed, especially given that numerous changes have been made to the 
MSCA programme over recent years, making it difficult to keep track of the 
evolving rules and requirements. 

INFRA: As indicated under the previous question, the Horizon Europe work 
programmes have introduced important changes to the support for research 
communities. While under Horizon 2020, the term ‘user community’ referred to 
scientists interested in the use of a certain piece of infrastructure, equipment 
or instrument, under Horizon Europe the term refers to scientists carrying out 
research in a certain challenge domain (e.g. cancer). This is related to the 
discontinuation of the support for advanced and starting communities 
(INFRAIA) under Horizon 2020, which has been replaced by INFRASERV 
calls. This change has brought important changes to how consortia are built 
and their composition. Interviewees were generally positive towards the 
challenge-oriented approach, as it increases the relevance of the projects. 
However, this change has brought a heavier administrative and management 
burden for applicants and beneficiaries, given that consortia are much larger 
(with some projects and proposals having more than 150 participants). 
Moreover, funding is now more widely dispersed across many partners, which 
–together with the higher administrative burden – reduces the interest of some 
partners in participating in such projects. Case Study 6 provides a full 
overview of the effects of the challenge-oriented approach. Furthermore, in 
Case Study 5, alternative models to fund transnational access and virtual 

824 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

access (TNA/VA) are discussed and explored, including potential ways to 
reduce administrative burden (e.g. cascade funding). 

WIDERA:  

Widening: Stakeholders and beneficiaries highly value the bottom-up 
collaborative opportunities provided by this programme part, which 
indicates that a wide range of thematic areas can be covered. In addition, 
following the new ERA objectives and action plan and the New European 
Innovation Agenda, the Widening work programmes under Horizon Europe 
address gaps present under Horizon 2020, namely the possibility to reinforce 
the links between science and business, and the opportunity to foster 
excellence in universities and placed-based innovation ecosystems (e.g. 
through Excellence Hubs, the European Excellence Initiative (EEI), and ERA 
Talents), the opportunity to foster balanced brain circulation across the ERA 
(e.g. through the ERA Fellowships) and to connect institutions to networks of 
excellence (e.g. through Hop-On Facilities). Nevertheless, several 
stakeholders indicated that they are not yet fully aware of the new actions 
and that some of them seem somewhat complex at first glance (e.g. 
Pathways to Synergies).1281  

Furthermore, some interviewees and stakeholders (for instance, through the 
position papers submitted to the open public consultation (OPC)) have 
expressed concerns about the design and implementation of the 
Widening actions. For example, some interviewees and stakeholders 
stressed that the WIDERA work programmes are somewhat confusing, since 
the Widening component and the ‘Strengthening the ERA’ component are 

1281 e.g. Initiative for Science in Europe (ISE). (2023). ISE Response to the European Commission Public Consultation on the past, present and future 
of the European Research & Innovation Framework programmes 2014-2027. 

825 

 
 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

quite different, yet are addressed together. Second, some stakeholders1282 and 
interviewees have indicated that the Widening actions could benefit from 
actions supporting R&I activities (e.g. more Research and Innovation Actions 
(RIA) or Innovation Actions (IA) to complement the typical Coordination and 
Support Actions (CSA)).  

ERA: Evidence from desk research and interviews shows that ERA part of the 
programme is relevant to contemporary scientific needs and issues: for 
example, the ERA destination puts significant emphasis on supporting the 
development of competences among research administrators and managers. 
In particular, the competences and skills of research managers in dealing with 
research ethics relating to new and emerging fields in which they have little or 
no experience – e.g. research ethics in the application of and research into 
artificial intelligence.  

Study evidence also shows that Horizon Europe WIDERA (ERA) actions were 
mostly relevant in addressing the needs of beneficiary institutions to increase 
their international visibility and strengthen their collaborative ties with other 
organisations. According to evidence from the beneficiary survey, 80% of 
WIDERA (ERA) beneficiary respondents (82 out of 102) agreed that their 
project responded to their organisation’s need to enhance its international 
visibility (across the borders of your country) to a ‘very large’ or ‘large’ extent. 
This was followed by their need to create or strengthen collaboration with 
leading research organisations (77%, or 79 out of 103 respondents) (see 
Figure 23). A number of beneficiaries interviewed likewise confirmed that their 
project was highly relevant in the context of their institution’s need to build 
connections with peers in other countries, to build networks of expertise, and 

1282 e.g. European Regions Research and Innovation Network (ERRIN) (2023) Errin Input To The Public Consultation On The Past, Present And 
Future Of The European Research & Innovation Framework Programmes 2014-2027. 

826 

 
 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

RV4.2* 

Are the activities as 
they exist today 
appropriate to address 
the scientific needs? 
What is missing? 

The extent to which the 
needs of the target groups 
are addressed in terms of 
available tools and 
thematic areas covered 
(and adaptability to 
changing needs); 
needs/gap analysis; 
appropriateness of 
activities’ design and 
implementation mode. 

Knowledge and 
awareness of supporting 
tools and thematic areas 
among targeted groups. 

Methods 
and 
analyses 

Literature 
review 

Monitoring 
data 

Interviews 

Case 
studies 

Key findings 

Evidence 

to be able to learn from examples of good practice from institutions in other 
countries 

ERC: see RV4.1 

INFRA: Interviewees are very positive about the grants for the development of 
RIs. Individual support to the ESFRI and other world-class RIs throughout their 
entire lifecycles is deemed very important in enabling them to remain at the 
state-of-the-art. The survey of successful applicants to INFRA, however, 
indicates that funding for the development of RIs may be insufficient: only 49% 
of INFRA beneficiaries (80 out of 165) and 26% of unsuccessful applicants to 
INFRA (24 out of 92) indicated that there are (to a ‘large’ or ‘very large’ extent) 
sufficient funding opportunities for the development of research infrastructures 
(see INFRA-BEN-024 in the survey of successful applicants and INFRA-
APPL-014 in the survey of unsuccessful applicants).  

WIDERA: 

Widening: Since many of the Widening actions are bottom-up, a wide range 
of scientific fields can be covered. Some stakeholders indicate that the 
Widening actions could support more Research and Innovation Actions (RIA) 
or Innovation Actions (IA) to complement the Coordination and Support 
Actions (CSA), in order to pursue concrete R&I activities and stimulate 
scientific excellence. In addition, it has been argued that researchers from 
widening countries need greater capacity to lead R&I projects (i.e. not only 
coordinating, but also leading and driving the R&I project).1283 

1283 e.g. European Regions Research and Innovation Network (ERRIN) (2023). Errin Input To The Public Consultation On The Past, Present And 
Future Of The European Research & Innovation Framework Programmes 2014-2027. 

827 

 
 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

ERA: Projects under the ‘Strengthening the ERA’ thematic priority were also 
highly relevant to institutions’ needs to develop their capacities and the skills of 
their staff, in order to better respondent to current socioeconomic needs, to 
increase competitiveness and the overall knowledge base. The case study 
findings suggest that, for example, by helping universities to build the 
competencies needed for researchers and facilitating the commercialisation of 
research findings, acceleration services make European HEIs better aligned 
with market needs and more capable of forging alliances with the private 
sector. Likewise, 65% (66 out of 102) of surveyed WIDERA (ERA) beneficiary 
respondents agreed to a ‘large’ or ‘very large’ extent that their project 
responds to their institution’s need to enhance its R&D and/or innovation 
capabilities and skills, followed by the need to strengthen the knowledge base 
and R&D capacity of their organisation (63%, or 65 out of 102 respondents) 
and to enhance their competitiveness (61%, or 62 out of 101 respondents). 

To what extent have 
the Framework 
Programme activities to 
deliver excellent 
science demonstrated 
to be flexible to cope 
with changing 
circumstances in 
Europe and in the 
world? 

RV5* 

Related to RV2. 

Evidence of activities in 
the analysed programmes 
that address newly 
emerging needs (e.g. 
migration, participatory 
democracy, COVID-19, 
vaccines, the green and 
digital transitions, etc.) in 
terms of content or 
implementation modalities. 

The extent to which the 
design of the FP actions 

Literature 
review 

Monitoring 
data 

Interviews 

Survey 

Case 
studies 

Benchmarks 

ERC: As mentioned previously, the ERC takes a ‘bottom-up’ approach, with a 
focus on excellence. However, the programme has been able to fund research 
used to tackle emerging needs. For example, at the height of the pandemic, 
over 180 ongoing or completed ERC projects generated knowledge and 
technology that contributed to the response to the COVID-19 pandemic, 
including diagnostics and treatments, medical devices or immunity, infection 
and pathology. In another example, the ERC started an initiative, following the 
ERC Scientific Council’s statement supporting Ukraine and its research 
community. With ‘ERC for Ukraine’, the ERC appealed to its grantees to 
provide temporary employment to researchers and support staff from Ukraine 
and seeking refuge in the EU, under the initiative ‘ERC for Ukraine’. 

48% of successful ERC applicants (109 out of 228) who replied to the survey 
(ERC-BEN-044) ‘strongly agreed’ or ‘rather agreed’ that, compared with the 

828 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

allows adaptability to 
developments or emerging 
needs; evidence of quick 
adaptation to changing 
circumstances. 

The extent of 
gaps/recently emerged 
aspects of scientific 
developments not covered 
by Framework Programme 
resources. 

Share of Framework 
Programme resources (in 
terms of number of 
projects, participants and 
funding) allocated to the 
most pressing needs. 

Gaps and differences 
compared with other 
similar funders analysed 
under the benchmarking 
analysis. 

Key findings 

Evidence 

research funding available at national and/or regional level, Horizon Europe 
provides greater flexibility to respond to pressing socio-economic needs. 
(Survey of Horizon Europe ERC beneficiaries, conducted May-July 2023) 

MSCA: In response to the Russian invasion of Ukraine, the MSCA has taken 
proactive steps to support affected doctoral candidates and postdoctoral 
researchers from Ukraine. Through the dedicated MSCA4Ukraine scheme 
launched in 2022, a funding allocation of EUR 25 million has been designated 
to assist displaced researchers, allowing them to continue their scientific 
pursuits in Europe and to contribute to the rebuilding of Ukraine's R&I capacity 
when conditions permit. The scheme received over 600 applications from 28 
countries, and 111 post-doctoral and 13 doctoral fellowships were awarded to 
researchers from Ukraine. 

In early 2021, REA conducted a survey to gather information about the 
challenges faced by participating organisations managing MSCA projects 
during the pandemic. The survey aimed to assess the effectiveness of the 
measures taken by the European Commission and REA to provide support 
during similar future situations. The findings suggest that the proposed 
flexibility measures were generally seen as appropriate and effective, allowing 
organisations to adapt to restrictions and constraints. Most respondents to the 
survey positively viewed the ability to fully pay MSCA researchers, even if they 
could not work in the host organisation premises. Respondents also 
considered the extended flexibility with regard to project deliverables and 
milestones as appropriate. It was also found that the communication regarding 
flexibility measures was generally timely, and supported by REA project 
officers.  

As an entirely ‘bottom-up’ and ideas-driven programme, the MSCA provide 
great flexibility for research creativity in all scientific fields, leading to 
significant and innovative contributions to major societal challenges and 
Commission priorities, including emerging ones such as the EU response to 

Methods 
and 
analyses 

Bibliometric 
analysis  
(Horizon 
2020) 

829 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

COVID-19. As discussed in the DG EAC’s strategic plan for 2020-2024, there 
are already more than 100 MSCA projects directly related to COVID-19 
research issues alone, and the MSCA can support many more projects in the 
years to come. The current COVID-19 crisis dramatically highlights the critical 
need to provide researchers with the right skills, including the necessary 
resilience skills, and to facilitate their outreach and networking, not only to 
address immediate health challenges, but also to anticipate the broader 
societal, ecological and economic implications of the outbreak, and to enhance 
Europe’s preparedness in the future. 

INFRA: The INFRA Work Programme 2021-2022 offered grants without a call 
for proposals, in accordance with Article 195 (b) of the Financial Regulation 
2018/1046 to address the COVID-19 exceptional emergency. This allowed the 
setup of projects such as: 1) the ISIDORe1284 consortium, in the form of a 
Research and Innovation Action, which brings together the capacities of 
European ESFRI infrastructures and coordinated networks to assemble the 
largest and most diverse research and service-providing instrument in Europe 
for studying infectious diseases, from structural biology to clinical trials; and 2) 
Beyond-COVID (BY-COVID), which aims to provide comprehensive open data 
on SARS-CoV-2 and other infectious diseases across scientific, medical, 
public health and policy domains. Coordination and synergies between both 
are planned, but it is still too early to assess the impact of these actions.  

WIDERA: 

Widening: The Widening WIDERA actions are addressing newly emerging 
needs, both through calls that explicitly mention contributing to these needs as 
their objectives (e.g. Support for R&I policy-making in the Western Balkans 
and Twinning Western Balkans Special, Excellence Hubs), as well as by 
supporting projects that address these topics. For instance, BRAINN 

1284 https://cordis.europa.eu/project/id/101046133  

830 

 
 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

(Twinning) will develop an innovative applied neuroscience approach to 
characterise the effects of COVID-19 on brain health; TWINN2SET (Twinning) 
will build a network to foster a sustainable energy transition in Greece; and 
ACRONYM (Twinning) aims to advance expertise on migration and asylum 
governance in Europe.  

In addition, more than half of respondents to the survey of WIDERA 
(Widening) beneficiaries (specifically, 89 out of 168) indicated that, compared 
with the research funding available to participants at national and/or regional 
level, Horizon Europe offers more flexibility to respond to changing socio-
economic needs. Only 6% of respondents (10 out of 168) ‘rather’ or ‘strongly’ 
disagreed with this statement (Survey of Horizon Europe beneficiaries, 
conducted May-July 2023; WIDERA-WIDE-BEN-045) 

ERA: As previously mentioned, preliminary desk research and interview 
evidence indicate that the ERA part of the WIDERA programme is highly 
relevant to addressing newly emerging challenges/ changing circumstances in 
Europe and the world. Calls under the ERA part of the programme include 
support for developing skills and competences in research ethics and integrity 
(especially in newly emerging fields such as AI), democratic societies, green 
transitions/ecosystems and other emerging societal and scientific issues. 

According to the case study findings, FP actions under WIDERA (ERA 
thematic priority) in the area of promoting citizen science/science education 
also align closely with the objective of bringing science closer to society 
(objective 14 of the European Research Area Policy Agenda), including by 
promoting stronger public understanding, connection to and engagement with 
science, which also brings science closer to societal challenges. A number of 
interviewees indicated that EU projects such as the European City of Science 
were of particular relevance in the context of the COVID-19 pandemic, when 
society’s trust in science and scientists was of paramount importance. 
Activities that brought science closer to the general public were particularly 

831 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

relevant in this context, and helped to nurture and maintain the public’s trust in 
science 

EOSC Partnership: In interviews, it was stated that the relevance of the 
EOSC has been demonstrated through the possibility for researchers working 
on the same topic to use each other’s data. In the case of COVID-19, all RNA 
research data was combined into a big data space. As a result of this 
interoperability, work on a vaccine progressed much faster than anyone had 
expected. 

The EOSC Partnership has also demonstrated flexibility, as taskforces of the 
EOSC Association have managed to provide input into a new draft SRIA. In 
interviews, it was questioned why the work in these taskforces was still 
voluntary. 

In which areas is the 
participation of 
international partners 
and associated 
countries the most 
relevant to deliver 
excellent science?  

RV6.1*  EU performance in terms 
of cooperation with third 
country participants and 
associated countries in 
Framework Programme 
activities to deliver 
excellent science; EU 
financial contribution 
allocated to third country 
participants and 
associated countries. 

Literature 
review 

Monitoring 
data 

Interviews 

Case 
studies 

Benchmarks 

Analysis of the areas of 
international cooperation, 
and assessment of 

ERC: According to the Horizon Dashboard (accessed 6 September 2023), 
participations from associated countries and non-associated third countries 
remained relatively stable under Horizon 2020 (the period considered being 
2015-2021), ranging from 116 signed grants in 2021 to 225 signed grants in 
2016. Under Horizon Europe, performance has not changed significantly, with 
183 grants signed in 2022 and 125 grants signed so far in 2023. However, it is 
important to consider two elements: the non-association of Switzerland to 
Horizon Europe, which was the second most active partner under Horizon 
2020, and the inclusion of the UK under the group of non-EU countries. So far, 
the net EU contribution to associated countries plus non-associated third 
countries corresponds to 12% (EUR 471.6 million) of the ERC's total net EU 
contribution. This share is slightly lower than the share observed during 
Horizon 2020 (15%). The top fields in terms of participation of associated 

832 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

contribution to EU FP 
objectives. 

countries plus non-associated third countries are genomics, scientific 
computing and data processing, and systems biology.  

MSCA: The MSCA programme is dominated by participations from EU 
Member States (77%), while third countries accounted for 20% of 
participations. Participations from associated countries account for just 4%. 
When broken down by action type, Staff Exchanges (SE) managed to attract 
the highest share of participations from third countries across all action types 
under the MSCA programme (35%). So far, the net EU contribution to 
associated countries and third countries in the MSCA account for 6.1% (EUR 
88.9 million) of the total MSCA net EU contribution. However, the survey 
indicates that more funding is needed to enhance collaborations with 
international partners (non-EU countries): only 40.0% (195 out of 488) of 
MSCA beneficiary organisations agrees to a ‘large’ or ‘very large’ extent that 
there is sufficient funding for international collaboration. 

In terms of external policy objectives, the Study on international cooperation in 
the Marie Skłodowska-Curie actions concluded that the programme will remain 
bottom-up, but at the same time efforts should be made to better align it with 
the EU external policy objectives, given its strong international dimension and 
the growing role of science diplomacy in foreign policies and strategies. To 
address some of these challenges, the MSCAdvocacy project was launched in 
2022. This aims to map and provide insights (e.g., key challenges, specificities 
of R&I ecosystems, cultural issues related to mobility) into the cooperation 
achieved until now with key international partners (i.e. through bilateral 
agreements and existing policy dialogues), according to the EC officials 
interviewed. Such a strategic outlook is expected to yield smarter ways of 
cooperating and selecting partners for MSCA projects, based on common 
values and strategic interests. 

INFRA: According to the Horizon Dashboard (accessed August 2023), 
associated countries represent 6% of participations in INFRA so far under 

833 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

Horizon Europe, compared with 9% under Horizon 2020. Moreover, the 
participation of third countries in INFRA accounts for 13% of participations, 
compared with only 3% under Horizon 2020. However, it is important to note 
that 85% of all third country participations are by the UK and Switzerland, 
which were respectively a Member State and an associated country under 
Horizon 2020.  

Interviewees and the survey indicate that international cooperation is 
important: 59.1% of the INFRA beneficiaries surveyed (88 out of 149) found 
that collaboration with international (non-EU) partners is important (to a ‘large’ 
or ‘very large’ extent) for the uptake of their research findings/innovation 
(INFRA-BEN-041). However, the survey indicates that more funding is needed 
for collaboration with international partners (non-EU countries): only 25.0% of 
INFRA beneficiaries (41 out of 164) agreed to a ‘large’ or ‘very large’ extent 
that there is sufficient funding for international collaboration (INFRA-BEN-024). 
In addition, only 26.2% agreed to a ‘large’ or ‘very large’ extent that there is 
sufficient funding for pre-normative research, allowing the development of 
international standards (INFRA-BEN-024). 

Horizon Europe INFRA work programmes include specific calls for 
international collaboration in the field of infrastructures, most notably those 
targeting cooperation on climate change in Africa (INFRA-2021-DEV-01-02) or 
bilateral cooperation with Latin America (INFRA-2023-DEV-01-06). Interviews 
confirm the relevance of this type of call, but also indicate that there are 
concerns regarding the sustainability of such cooperation after the end of 
these projects. Some interviewees indicated the need for a long-term strategy 
for this type of collaboration, with a focus on specific geographical and topical 
areas. 

WIDERA:  

834 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

Widening: According to the Horizon Dashboard (accessed June 2023), so far 
under Horizon Europe, Member States have received 93% of the EU net 
contribution (EUR 590 million), while non-Member States have received 7% 
(EUR 41 million). Serbia is the associated Widening Country receiving the 
highest EU net contribution (EUR 14 million), followed by Turkey (EUR 9 
million). The novel Widening actions aim to spur both place-based innovation 
systems, but also to foster international cooperation (e.g. through the Hop-On 
Facilities and European Excellence Initiative (EEI)). International collaboration 
opportunities are highly appreciated by Widening organisations, given the 
capacity such opportunities enable them to build, as well as the visibility they 
gain.  

ERA: According to the Horizon Dashboard, Member States have so far 
received over around 93% (over EUR 108 million) of the total EU net 
contribution of EUR 116 million. Meanwhile, non-Member States have 
received around 7% (roughly EUR 8 million). The highest amount of funding 
received by non-Member States was by Norway (over EUR 3 million), Serbia 
(EUR 0.8 million) and Turkey (EUR 0.7 million). Actions supporting 
programme-level collaboration between national R&I policy-makers provide 
support to both associated and non-associated third countries. 

How does this 
participation fit into the 
objectives of the 
Framework 
Programme, including 
to reinforce Europe’s 
relative positioning? 

RV6.2*  EU performance in terms 

of cooperation with 
participants from third 
countries and associated 
countries in Framework 
Programme activities to 
deliver excellent science; 
EU financial contribution 

Literature 
review 

Monitoring 
data 

Interviews 

ERC: See RV6.1. 

INFRA: International collaboration between RIs is considered very important 
to fostering excellent science. An example of good practice is JIV-ERIC, which 
enables the observation of black holes by integrating together astronomical 
observatories in Europe, Asia and Africa; however, other examples exist. 

835 

 
 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

allocated to participants 
from third country and 
associated countries. 

Analysis of the areas of 
international cooperation, 
and assessment of 
contribution to EU FP 
objectives. 

Methods 
and 
analyses 

Case 
studies 

Benchmarks 

Key findings 

Evidence 

WIDERA: 

Widening: One of the premises of the Widening actions is that R&I policy can 
strive towards excellence if everyone progresses. The Widening actions 
stimulate associated Widening Countries (Albania, Bosnia & Herzegovina, 
Kosovo, Montenegro, North Macedonia, Serbia, Turkey, Armenia, Georgia, 
Moldova, Morocco, Tunisia, Ukraine, and the Faroe Islands) to engage in the 
Framework Programme.  

The network analysis conducted in the context of this study1285 provides more 
information about collaborations within WIDERA. Collaborations between 
widening and non-widening Member States account for 31.6% of the 
collaborations in Horizon Europe excluding WIDERA. Within WIDERA, the 
share of cross-collaborations between widening and non-widening Member 
States is higher, at 40.0%. Moreover, when examining the new connections 
within WIDERA’s Widening calls, 4,404 collaborations were found within 
Widening, of which 3,631 were new, meaning that the two entities had not 
previously collaborated anywhere else under Horizon Europe. The largest 
share of new collaborations were formed between Greek organisations, 
followed by Croatian organisations and Greek-Bulgarian collaborations. 

RV7 

To what extent does 
the Framework 
Programme provide the 
right balance between 
bottom-up and top-
down approaches to 

Identification of bottom-up 
and top-down approaches; 
assessment of appropriate 
distribution in the given 
context. 

Literature 
review 

Interviews 

Case 
studies 

ERC: As mentioned previously, under Horizon Europe, the ERC has retained 
the same ‘bottom-up’ approach. This approach has proved relevant to meeting 
the ERC’s objectives of encouraging the research of the highest quality in 
Europe through competitive funding, as well as supporting investigator-driven 
frontier research across all fields, based on scientific excellence. Beneficiaries 
and other stakeholders also consider this approach to be appropriate.  

1285 For more information and details, see Annex 6 ‘Quantitative research track findings’, Entities network analysis. 

836 

 
 
 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

funding and between 
support for basic and 
applied research given 
its place in the 
European research and 
innovation landscape? 

Benchmarks  MSCA: Under Horizon Europe, the MSCA continue to operate in a fully 
bottom-up manner. The MSCA are expected to complement top-down 
collaborative research activities, contributing notably to the EU Missions. In 
position papers submitted to the open public consultation on the past, present 
and the future of research and innovation framework programmes, 
stakeholders recommended an increase in bottom-up approaches to funding.. 
For example, Masaryk University saw the MSCA as fundamentally based on 
bottom-up principles, and called upon Horizon Europe to do more to promote 
broader and more inclusive calls from widening countries (WCs). 

Interview findings highlight this bottom-up approach to funding as being 
essential for the MSCA programme’s success in fostering excellent science. 
The MSCA is highly appreciated by researchers as it allows them to propose 
research topics and ideas based on their own interests and without a 
predefined thematic focus. The survey results reflect these findings, with an 
overwhelming majority of successful MSCA PF applicants (91%, or 755 out of 
828) indicating that the freedom to pursue their specific research agenda 
responded to their needs to a ‘very large’ or ‘large’ extent. Exceptions exist, 
however, for programmes proposed under COFUND. The new or existing 
doctoral or postdoctoral programmes in COFUND can cover any research 
discipline (bottom-up approach), but can also focus on specific disciplines, 
notably when they are based on national or regional Research and Innovation 
Strategies for Smart Specialisation (RIS3 strategies). 

INFRA: As indicated previously, INFRA has shifted to a more challenge-
oriented (top-down) approach under Horizon Europe, compared with its 
bottom-up approach under Horizon 2020. While interviewees are generally 
positive, they also indicate that ‘blue-sky’ research important for addressing 
future challenges, and thus a balance should be maintained. Only 39.4% of 
INFRA beneficiaries surveyed (65 out of 165) agreed to a ‘large’ or ‘very large’ 
extent that there are sufficient funding opportunities for conducting 

837 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

basic/fundamental research in new and emerging research areas (INFRA-
BEN-024). A slightly higher share of INFRA beneficiaries (48.8%, or 80 out of 
164) agreed to a ‘large’ or ‘very large’ extent that there are sufficient funding 
opportunities for research into technological applications in areas that are 
important for addressing global challenges (INFRA-BEN-024). Moreover, the 
majority of INFRA beneficiaries (58.2%, or 96 out of 165) found that there are 
(to a ‘large’ or ‘very large’ extent) enough funding opportunities for 
interdisciplinary research (INFRA-BEN-024).  

Some interviewees also expressed concerns about the increased dependence 
of funding on (top-down) political priorities and the instability that can result 
from this. The time horizon for politics is usually shorter than that required for 
scientific endeavours. As new priorities are set, the continuity of projects may 
be compromised, reducing the long-term impact of projects and of the 
programme. Some interviewees therefore point to the need for longer-term 
roadmaps (at least beyond typical project lifetimes of 3-4 years) that could 
facilitate longer-term planning and stability. 

WIDERA: 

Widening: While almost all of the WIDERA Widening calls adopt a bottom-up 
approach, in which researchers can define their research topics, Horizon 
Europe has introduced the novelty of a Twinning call dedicated to more 
specific themes, namely the Twinning Green Deal call (WIDERA-2023-
ACCESS-02-02). Projects funded under this call will need to address climate 
research, green technologies, renewable energy, sustainable mobility, 
biodiversity research or the sustainable use of natural resources (land, water, 
air). In general, the bottom-up nature of the Widening actions is highly 
appreciated by beneficiaries and stakeholders. It allows research groups in 
widening countries to strengthen their critical mass and spur excellence in any 
specific field that is relevant to them.  

838 

 
 
Evaluation question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods 
and 
analyses 

Key findings 

Evidence 

ERA: ERA takes a top-down approach, with each call seeking a unique activity 
for each topic the call aims to strengthen. These include Prioritizing 
Investment and Reform; Translating R&I Results Into the Economy; Open 
Science; Higher Education Institutions and Researchers; Science 
Communication Science Education; and Gender Equality, Ethics and Integrity 

EOSC Partnership: Some interviewees appreciated the bottom-up 
contributions of research projects under Horizon 2020, but the more recent 
Coordination and Support Actions (e.g. with EOSC FOCUS) are now 
considered especially relevant for aligning projects and ensuring the take-up of 
results and improving the sustainability of results. For the development of a 
federated European infrastructure, research projects have limitations; 
application and implementation have now become much more important. 

1.2.  Coherence 

Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

How coherent has 
the Framework 
Programme been to 

CH1.1* 

Internal coherence 

Overlaps, gaps, synergies and 
complementarities (in terms of 

Literature 
review 

ERC:  ERC grants are evaluated and selected on the basis of the 
sole criterion of scientific excellence. Having a solely bottom-up and 
excellence-driven approach makes the ERC a distinctive programme 

839 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

deliver excellent 
science, in 
particular: (i) 
between 
Framework 
Programme parts 
covered by this 
study; (ii) with other 
parts of the 
Framework 
Programme not 
covered by this 
study; 

objectives, thematic areas, value 
chains, technological readiness levels 
covered, types of beneficiaries and 
activities supported): 

• 

• 

Between different specific 
programmes under the 
Excellent Science pillar (e.g. 
the MSCA, ERC, research 
Infrastructures); 

Between the Excellent 
Science pillar and other 
pillars/specific programmes 
of Horizon 2020/Horizon 
Europe; 

Perceptions of stakeholders regarding 
internal consistency between the 
objectives and the actions funded 
under the Excellent Science pillar of 
Horizon 2020/Horizon Europe. 

Level of specificity of the synergy 
objectives (in different Horizon 2020a 
and Horizon Europe work 
programmes. 

Key findings 

Evidence 

part within the Framework Programme. However, due to these 
distinctive features, the ERC is less likely to display strong synergies 
with other programme parts. Position papers submitted to the open 
public consultation, as well as survey respondents both confirm this 
aspect: 

•  Most respondents to the public consultation (56%, or 500 

out of 895) did not know whether they see any unexploited 
potential for complementarities between the six clusters 
and the ERC. However, a smaller percentage (24%; 215 
out of 895) saw unexploited potential for 
complementarities with Cluster 1 (Health).  

•  Most respondents (71.5%, or 178 out of 250) to the survey 

(ERC-BEN-022) reported having no plans for joint 
activities with other projects under Horizon Europe. 
Meanwhile, 10% (25 out of 250) said they planned to hold 
joint activities with another ERC project, and 12.4% (31 
out of 250) either did not know or indicated that the 
question was not applicable to their situation. 

Nevertheless, ERC Proof of Concept proposals that do not receive 
funding are now automatically awarded  the Seal of Excellence, and 
are thus eligible to receive funding from the Structural Funds. In 
addition, over recent years, the ERC has deepened its collaboration 
with the EIC, with the objective of exploiting potential synergies and 
complementarities between the two organisations. Examples of such 
collaboration include a joint workshop on energy storage organised 
in November 2022, and a joint workshop on cell and gene therapy in 
June 2021. The potential for further collaboration is very large, as 
demonstrated by the fact that 25 out of 42 recipients of the 
European Innovation Council’s new ‘Transition’ funding originated 
from research funded by the ERC. Another initiative includes the 

Methods and 
analyses 

Interviews 

Case studies 

Network 
analysis 

Surveys 

840 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

Next Generation Innovation Talent Schemes1286, open to PhD 
candidates and/or postdoctoral researchers participating in projects 
funded by the ERC. 

In addition, according to the network analysis carried out under this 
study (see Annex 6), and the degree to which researchers 
participate in different programmes (the connectedness of 
programme parts), the ERC is one of programme parts with the 
fewest number of connections. A specific analysis of the trajectories 
of Horizon 2020 ERC researchers under Horizon Europe shows that 
around 8% of PIs from ERC projects in Horizon 2020 participated in 
a Horizon Europe project. Most frequently, they participate in Pillar 2 
of Horizon Europe (52% of participations in Horizon Europe by PIs 
that participated in the ERC under Horizon 2020), followed by Pillar 
1 (24%), Pillar 3 (18%) and WIDERA (6%). In Pillar 1, the most 
frequent destination programme part is MSCA (15% of all 
participations by Horizon 2020 ERC PIs under Horizon Europe), 
followed by INFRA (5%). In Pillar 2, the most frequent destination 
programme part is Cluster 4 ‘Digital, Industry and space’ (20% of all 
participations by Horizon 2020 ERC PIs under Horizon Europe), 
followed by Cluster 1 ‘Health’ (10%). In Pillar 3, almost all PIs went 
to the EIC, accounting for 18% of all participations by Horizon 2020 
ERC PIs under Horizon Europe. A more detailed analysis of the 
trajectories of ERC PIs is available in Case Study 9 on Knowledge 
Creation; here, a distinction is also made by the type of ERC grant. 

MSCA: The present study indicates that the MSCA are coherent 
with other Horizon Europe programme parts. Based on the findings 
of the survey of Horizon Europe beneficiaries, 16% of successful 
MSCA applicant organisations (81 out of 500) reported that their 

1286 https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/topic-details/horizon-eic-2023-talents-01-01 

841 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

MSCA project is a continuation of research activities carried out 
under the previous Framework Programme, Horizon 2020 (i.e. it is 
based on work carried out in the previous research project). The 
figure for successful MSCA PF applicants stands at 5% (39 out of 
867). 

Findings from the interview programme highlight the internal 
coherence between different programme parts of Pillar 1 under 
Horizon Europe. The MSCA were found to be especially coherent 
with the ERC. Both programmes are bottom-up or investigator-
driven, and share a similar emphasis on the stimulation of 
excellence and the production of high-quality research. The ERC 
and MSCA are also seen as the main instruments for the 
development and attraction of talent to the European Union. Despite 
these similarities, the interviewees see complementarities rather 
than overlaps between these two programme parts. To illustrate this, 
the MSCA provide funding only for fellows recruited by the 
beneficiary, whereas ERC funding supports not only principal 
investigators (PIs), but also the researchers sub-contracted to the 
project, helping PIs build their own research teams. While both 
programme parts offer funding opportunities for researchers at 
various career stages, the ERC focuses on supporting researchers 
with a track record of significant research achievements. It aims to 
support pioneering and high-risk/high-reward research projects 
conducted by independent investigators. Against this backdrop, the 
interviewees also see the ERC as complementary to the MSCA, as 
researchers consider ERC funding opportunities to be a natural next 
step in their careers after participating in the MSCA. 

Notable parallels exist between MSCA Staff Exchanges and ERA 
Talents. Both schemes promote innovative collaborations in R&I 
through the exchange of staff and the sharing of knowledge at all 
stages in the innovation chain. However, MSCA Staff Exchanges 

842 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

encompass collaborations on a global scale, spanning sectors and 
disciplines, while ERA Talents primarily seeks to foster inter-sectoral 
collaboration, with a distinct emphasis on widening countries. In 
contrast to MSCA Staff Exchanges, the ERA Talents scheme also 
permits intra-country exchanges under specific circumstances, 
provided they contribute to the preservation or enhancement of 
regional innovation ecosystems. 

Evidence from the interviews regarding the similarities that exist 
between the ERA Fellowship scheme and the MSCA Postdoctoral 
Fellowship (PF) scheme. However, for the ERA Fellowship scheme, 
a dedicated budget specifically covers postdoctoral fellowships in 
host organisations located in countries with low R&I performance. 
The objective of ERA Fellowships is to incentivise brain circulation 
towards widening countries. Researchers that are considering 
applying to the ERA Fellowships may do so by applying directly to 
the ERA Fellowship scheme or through the MSCA Postdoctoral 
Fellowships, which offer the option of automatically resubmitting an 
application to the ERA Fellowships if it does not pass the MSCA 
threshold. 

A previous evaluation of Horizon 2020 considered the MSCA 
coherent with the European Institute of Innovation and Technology 
(EIT). Horizon Europe continues to encourage synergies between 
these two initiatives, especially given that financial rules notably 
prohibit the same activity from being funded by two different EU 
programmes. Organisations can participate in different MSCA 
projects and in EIT KICs’ activities by setting up complementary 
exchange projects under MSCA Staff Exchanges, by sending 
recruited researchers under MSCA PF, DN and COFUND actions on 
short-term secondments to organisations involved in the EIT KICs’ 
networks, or by providing non-academic placements for MSCA PF 
fellows in EIT KICs partner organisations. The findings of the 

843 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

previous evaluation suggest that most synergies between the MSCA 
and the EIT have been developed through MSCA Doctoral Networks 
(previously known as Innovative Training Networks) and MSCA 
COFUND.  

The present study finds that instances of synergies between these 
two programmes also exist under Horizon Europe. For example, the 
project SmartOptoelectronics (MSCA-PF-2021) aims to develop 
machine learning (ML) methods to establish trends between the 
structural properties and electrochemical performance of perovskite 
lanthanum-based oxides, based on high-throughput experimental 
data. The postdoctoral researcher’s host organisation, the Institute 
for Energy Research in Catalonia, is a partner of EIT InnoEnergy. 
The host institute offers unique opportunities to reinforce the 
technology transfer competencies of the postdoctoral researcher by 
carrying out an industrial secondment and through involvement in 
the KIC Innoenergy community. 

INFRA: Horizon Europe work programmes include efforts to foster 
the collaboration of research infrastructures with other actors along 
the innovation value chain, including technology 
infrastructures/centres and industry (see, for example, the call 
INFRA-2022-DEV-01-02 on Cooperation, synergies and networking 
between research infrastructures and technology infrastructures). 
Interviewees indicate that there is room to better integrate research 
infrastructures into other programme parts by specifically including 
references to them in call texts. Survey respondents who 
participated in INFRA indicated that they collaborate most with other 
projects from INFRA (31.40%, or 55 out of 175 indicate that this is 
one of the three most important programmes for collaboration), 
followed by Cluster 6 ‘Food, Bioeconomy, Natural Resources, 
Agriculture & Environment’ (9.10%, or 16 out of 175), and Cluster 1 
‘Health’ (4.60%, or 8 out of 175) (INFRA-BEN-022). According to the 

844 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

network analysis carried out under this study (see Annex 6), INFRA 
is most connected to Cluster 4 ‘Digital, Industry and Space’, Cluster 
6 ‘Food, Bioeconomy, Natural Resources, Agriculture & 
Environment’, Cluster 1 ‘Health’, as well as with WIDERA. Analysis 
of ERIC participation across Horizon Europe also indicates strong 
links with Cluster 1 and Cluster 6 (with 11 and 13 ERIC 
participations  in these programme parts, respectively).  

According to an internal mapping of synergies conducted by DG 
RTD in Spring 2023, there are also synergies between the MSCA 
programme part and INFRA: the MSCA allow research 
infrastructures to meet the training needs of their researchers and 
other staff, also contributing to their career development. Moreover, 
the same mapping identified synergies with the Next Generation 
Innovation Talents Scheme, which offers internships to (among 
others) PhD candidates from the Horizon Europe INFRA programme 
part. 

With regard to coherence between INFRA under the successive 
Framework Programmes, the survey indicates a high level of 
continuity: 42.4% of INFRA beneficiaries (75 out of 177) indicated 
that their project is a continuation of a research project under 
Horizon 2020, while 17.5% (31 out of 177) indicated that their project 
is a continuation of a project under FP7 (INFRA-BEN-023).  

WIDERA: 

Widening: Interviewees and stakeholders commented on the design 
and implementation of the WIDERA programme part. Some 
indicated that the Widening component and the ‘Strengthening the 
ERA’ component are quite different, yet were addressed together in 
a single Work Programme. In addition, several stakeholders 
wondered whether some actions that are categorised under 

845 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

Widening (e.g. Excellence Hubs, the European Excellence Initiative, 
COST), would not be better positioned under the ‘Strengthening the 
ERA’ component of the work programme, or perhaps another 
programme part. In general, the Widening actions are seen as a 
distinct set of actions aimed at supporting widening institutions and 
countries in improving their R&I systems.  

To complement the Horizon 2020 (Widening) actions1287, several 
new actions have been introduced into Horizon Europe. In the 
WIDERA work programmes, the ERA Talents scheme has been 
positioned as being complementary to ERA Chairs, Excellence 
Hubs, ERA Fellowships and MSCA Staff Exchanges. ERA 
Fellowships are, of course, very similar to MSCA Fellowships; 
meanwhile, the Excellence Hubs are positioned as complementary 
but different from initiatives such as Digital hubs or the EIT Regional 
Innovation Scheme (RIS) due to their strategic orientation, broader 
scope and alignment with widening eligibility criteria. In addition, 
since this action aims to foster a real place-based innovation culture 
in Widening countries, it encourages synergies to be sought with 
European Innovation Ecosystems (EIE), EIC Pathfinder and the 
European Institute of Innovation & Technology (EIT), as well as with 
the initiative ‘Partnerships for Regional Innovation’ run by the JRC. 
However, it is somewhat early to evaluate whether this indeed is the 
case. Another new instrument, ‘Pathways to Synergies’, has been 
set up to stimulate synergies between Horizon Europe and Cohesion 
Policy programmes, mainly the European Fund for Regional 
Development (ERDF), INTERREG and the Resilience and Recovery 
Fund (RRF). These projects are expected to start in June 2024.1288  

1287 Horizon Europe Work Programmes WIDERA 
1288 https://rea.ec.europa.eu/funding-and-grants/horizon-europe-widening-participation-and-spreading-excellence/pathways-synergies_en 

846 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

The Targeted Stakeholder Consultation on the European Research 
& Innovation framework revealed that a large majority of 
respondents (67%, or 595 out of 889) did not know whether they see 
any unexploited potential for complementarities between the six 
clusters and Widening and ERA, followed by 16% (141 out of 889) 
who see unexploited potential for complementarities with Cluster 1 – 
Health.  

The survey results also examine whether WIDERA Widening 
projects plan to implement activities in collaboration with projects 
funded under other Horizon Europe programme parts (WIDERA-
WIDE-BEN-022). Most respondents indicates that they either did not 
have joint activities planned with projects under Horizon Europe 
(38.7%, or 74 out of 191) or did not know (20.9%, or 40 out of 191). 
One-fifth of respondents (41 out of 191) reported having joint 
activities with another Widening project. (Survey of Horizon Europe 
beneficiaries, conducted May-July 2023) 

To maximise its impact on R&I, support should ideally be coherent 
and should work on synergies. Taken as a whole, the Widening 
actions mainly focus on collaborative approaches and Coordination 
and Support Actions (CSA). This differs substantially from Pillar 1 
actions, which mainly focus on individual actions and only focus to a 
limited extent on CSA. ERA: it is expected that the actions funded 
under ‘Reforming and enhancing the EU research and innovation 
system’ destination will be coherent and will produce synergies with 
other Horizon Europe actions. The actions focusing on the topic of 
open access are expected to produce synergies with Research 
Infrastructure projects under the EOSC, whereas WIDERA actions 
focusing on citizen science will produce synergies with Horizon 
Europe EU Missions. 

847 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

WIDERA (ERA) actions were especially complementary to other 
Horizon Europe programme parts that contribute to the development 
of human research capacities, as well as the development of 
researchers’ careers and the organisational capacities of institutions. 
In particular, support for university transformation/acceleration 
services under WIDERA is complementary to the MSCA, by 
contributing to the same overarching objective of strengthening 
Europe's human capital base and improving the working conditions 
of researchers and the attractiveness of research careers. Whereas 
the MSCA contributes to this overarching objective primarily though 
supporting dedicated actions on the mobility and training of 
researchers, WIDERA (ERA) support focuses on fostering structural 
changes in institutions, thereby contributing to the improvement of 
working conditions and the career development of researchers. 
Case study evidence provides examples of how WIDERA projects in 
the field of acceleration services for universities aim to boost 
university transformation in areas such as the development of 
research careers and talent support (including qualifications and 
research careers, research assessment, digitalisation, talent 
circulation and mobility, life-long learning, etc.). Similarly, the 
WIDERA project ‘Creating Framework Conditions for Research 
Management to Strengthen the European Research Area’ aims to 
create a roadmap for the future of research management (RM) in 
Europe, thereby contributing to the overall development of Europe’s 
human research capacities by developing European-wide standards 
and roadmaps. 

Citizen science-oriented projects under the WIDERA programme are 
also complementary to the MSCA Researchers’ Night initiative. 
Together (although through different means) these address the 
overarching goal of bringing science closer to the public and making 
research careers more visible and attractive, especially for young 

848 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

people. In some cases, activities in the area of citizen science that 
are funded under WIDERA have resulted in synergies and spill-over 
effects with other parts of Horizon 2020/Horizon Europe. For 
example, the MSCA Alumni Association was involved in the 
European City of Science and EuroScience Open Forum (ESOF 
20221289) in Leiden. At the ESOF 2022 Marie Skłodowska-Curie 
Actions Satellite Event in Leiden, more than 120 MSCA fellows, 
supervisors, coordinators, alumni, and policy-makers gathered to 
network, debate and contribute to priority topics for the research 
ecosystem1290. 

Contributing to gender equality in research is also a significant area 
of complementarity between WIDERA (ERA) and other parts of 
Horizon Europe, particularly the MSCA. Whereas the MSCA include 
measures to address gender imbalances in research careers (mainly 
though support to mobility and training of individuals), WIDERA 
(ERA) supports expert policy and knowledge support, as well as 
institutional transformation in the field of gender equality (e.g. 
supporting the Centre of excellence on inclusive gender equality in 
Research & Innovation, the Living Lab for gender-responsive 
innovation, and the implementation of inclusive gender equality 
plans). 

Similarly, in some cases, WIDERA projects in the area of university 
transformation have complemented the Research Infrastructures 
programme. Based on case study evidence, some WIDERA (ERA) 
projects focusing on the professionalisation of research 
management complemented similar projects supported under the RI, 

1289 Frigge, W., Evaluation Report EuroScience Open Forum (ESOF2022), available at: https://leiden2022.nl/sites/default/files/2023-
06/D3.2%20ESOF2022%20Programme%20and%20evaluation%20report.pdf 
1290 https://www.esof.eu/marie-sklodowska-curie-actions-msca/ 

849 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

e.g. the RI Train1291 and RI Trains Plus1292 projects, which aim to 
improve and professionalise the training of managerial and 
leadership staff in research infrastructures. Both programmes 
provide targeted support for the professional development of 
research managers, while using different support measures. 
Whereas the WIDERA project aimed to foster longer-term structural 
changes by developing a roadmap for the future of research 
management in Europe and a community to support its delivery. 
Meanwhile, support under the RI programmes include training and 
competency requirements for the current and future managers of 
European research infrastructures. 

Significant complementarities exist between WIDERA (ERA) support 
and the RI programme in the field of support for Open Science and 
the reusability/exploitation of research results. Whereas the RI 
programme supports the development of European infrastructure 
that enables Open Science and the sharing of research results (e.g. 
through support for initiatives such as the European Open Science 
Cloud), WIDERA (ERA) supports the systematic gathering of 
evidence on the impacts of Open Science practices (e.g. ‘Modelling 
and quantifying the impacts of open science practice’) and by 
supporting the capacity building of institutions/institutional 
transformation in the field of Open Science (e.g. actions to support 
capacity building for institutional open access publishing across 
Europe). 

EOSC Partnership: Attention has mainly been directed towards the 
internal coherence of the European partnership, and less towards 
collaboration with and learning between different partnerships. 

1291 http://ritrain.eu/ 
1292 https://ritrainplus.eu 

850 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

Literature 
review 

Interviews 

Case studies 

Network 
analysis 

Surveys 

How coherent has 
the Framework 
Programme been to 
deliver excellent 
science, in 
particular: iii) with 
other EU 
programmes 
serving similar 
objectives (e.g. 
Structural Funds, 
European Fund for 
strategic 
investments (smart 
specialisation 
strategies), 
Erasmus+ (notably 
European 
Universities, (iv) 
with relevant 
national, regional or 
international 
initiatives 

CH1.2.* 

External coherence 

Overlaps, gaps, synergies and 
complementarities (in terms of 
objectives, thematic areas, value 
chains, technological readiness levels 
covered, types of beneficiaries and 
activities supported): 

• 

• 

Between the Excellent 
Science pillar and other EU 
programmes serving similar 
objectives – ESIF (smart 
specialisation strategies), 
Erasmus+ (strategic 
partnerships, European 
universities); 

Between the Excellent 
Science pillar and other 
similar national/regional and 
international R&I funding 

Interviewees also report that not enough time had yet been available 
to collaborate with initiatives on Open Science in third countries.  

ERC: Proposals under the ERC Proof of Concept that have 
achieved a pass mark in all three evaluation steps but cannot be 
funded due to a lack of available budget for the call are automatically 
awarded the Seal of Excellence1293. In addition, survey results (ERC-
BEN-021) indicate that most respondents (74.1%, or 186 out of 250) 
did not apply for any additional funding for the research 
idea/activities addressed in their ERC project proposal, while almost 
one-fifth of respondents (47 out of 250) applied for national/regional 
R&D public funding schemes programmes. The interviews with PIs 
confirm that national/regional funding is in some cases applied for in 
order to complement the research activities under the ERC grant – 
for instance, to explore a new research path that became apparent 
during the research but did not fit into the scope of the ERC project.  

One additional aspect of external coherence concerns the strength 
of links between the ERC and key EU strategic priorities in terms of 
research excellence. In this regard, the ERC has taken steps by, for 
example, in September 2022, taking part in the annual European 
Research and Innovation Days1294, which bring together policy-
makers, researchers, entrepreneurs and citizens to debate and 
shape the future of R&I. The panel discussed the importance of 
science communication and public engagement, particularly among 
young citizens. The ERC also provides additional funding for ERC 
grantees within the scope of the Public Engagement with Research 

1293 https://research-and-innovation.ec.europa.eu/funding/funding-opportunities/seal-excellence_en.  
1294 European Research Council (2022). Annual Report on the ERC activities and achievements in 2022. Available at: https://op.europa.eu/s/yqiQ. 

851 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

programmes (e.g. DFG in 
Germany, ANR in France). 

Award1295, with the objective of recognising and rewarding 
outstanding science communicators among those PIs who 
successfully engage audiences outside their domain with ERC-
funded research. Lastly, the ERC has been able to contribute to the 
green and digital transitions, one of the priority areas for the ERA. 
Such contributions include, for example, projects in the fields of 
energy and mobility1296.  

MSCA: The new Euratom Programme 2021-2025 has a single set of 
objectives for both direct and indirect actions, seeking to enhance 
synergies with Horizon Europe, in particular in the medical 
applications of radiation. This reinforces its education and training 
actions in the nuclear field and opens the MSCA up to nuclear 
researchers. In this regard, researchers in the fields covered by the 
Euratom programme will be eligible to apply for MSCA Postdoctoral 
Fellowships, given the reference to possible synergies between 
Horizon Europe and the Euratom programme. By aligning its 
objectives and actions with the MSCA, the Euratom Programme 
seeks to maximise collaboration, knowledge sharing and innovation, 
ultimately fostering advances and discoveries regarding the medical 
applications of radiation within the nuclear research community. 

Both the MSCA and Erasmus+ are EU initiatives aimed at 
supporting the mobility, training and career development of 
individuals, including researchers. The legal basis of these 
programmes encourages synergies, provided the same activities 
cannot be funded twice. In this regard, various avenues for 

1295 ERC Work Programme 2024. Available at: https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2023/wp_horizon-erc-2023_en.pdf  
1296 European Research Council (2022). ERC Frontier Research contribution to the European Green deal. Available at: 
https://erc.europa.eu/sites/default/files/2022-08/H2020_factsheet_EuropeanGreenDeal.pdf 

852 

 
 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

synergies are open – for example, participants in Erasmus+ Mobility 
Projects can integrate into MSCA research teams through 
traineeships, short-term physical mobility and staff exchanges. 
MSCA researchers can join Erasmus+ intensive programmes to 
share their expertise. Organisations, in particular those involved in 
mobility projects with third countries, can use their networks and 
existing partnerships to set up collaborative research and innovation 
exchange projects under MSCA Staff Exchanges.  

The connections and expertise cultivated in Erasmus Mundus 
programmes can serve as a foundation for establishing doctoral 
initiatives, including those offering joint or multiple degrees, 
especially within the MSCA Doctoral Networks and COFUND. This 
collaboration can leverage the educational networks of Erasmus 
Mundus to facilitate the development of advanced research-oriented 
doctoral programmes under the MSCA, fostering a seamless 
progression from Master’s-level research to doctoral studies. Such 
examples already exist in the MSCA under Horizon Europe. For 
example, the MSCA Doctoral Network programme EXBRINER is 
implemented through an intersectoral and multidisciplinary network 
composed of HEIs, research centres and industrial partners. 
Drawing from over a decade of collaborative involvement in the 
Erasmus Mundus Joint Doctorate/Master’s in Membrane 
Engineering, the Consortium shares a common, globally acclaimed 
infrastructure for implementing highly competitive and jointly 
supervised training and research activities.  

Likewise, European Universities established as part of the 
Erasmus+ European Universities Initiative, are encouraged to exploit 
the links created in order to develop doctoral and postdoctoral 
programmes, including joint programmes, under the MSCA Doctoral 
Networks and COFUND, as well as collaborative projects under 
MSCA Staff Exchanges. European University Alliances can also 

853 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

exploit their partner networks by participating in outreach activities 
under the MSCA and Citizens action. Such synergies are already 
present in Horizon Europe. To illustrate this, UNITA, one of the 
European University Alliances, is implementing an MSCA project 
supported by the MSCA and Citizens action that aims to expand the 
outreach of the European Researchers’ Night and increase the 
public recognition of R&D activities in rural and mountain areas. 

INFRA: With regard to INFRA’s coherence with other similar 
programmes at European and national/regional level, it appears that 
INFRA projects under Horizon Europe are not typically continuations 
of projects funded by other programmes. Only 6.2% of INFRA 
beneficiaries (11 out of 177) in the survey indicated that their 
research project is a continuation of a project funded under 
national/regional R&D public funding programmes, and only 2.3% (4 
out of 177) indicated it was a continuation of another European 
research project (e.g. funded under the ERDF, ESF, etc.) (INFRA-
BEN-023). 

Similar results are found when analysing administrative data using a 
semantic textual similarity method. This shows that 10 out of 81 
INFRA projects under Horizon Europe (12%) received 
complementary funding, adding up to a total of EUR 44.3 million. 

WIDERA: 

Widening: Under WIDERA, synergies between funding sources are 
envisaged in the Teaming and Excellence Hubs in order to ensure 
the sustainability of their results (work programmes). A general 
overview of where Horizon Europe and other EU funding sources 
complement each other reveals 15 cases in which WIDERA and 
other EU funding schemes have supported thematically related 

854 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

projects. The amount of EU funds received as complementary 
funding amounts to EUR 20.1 million1297. 

In Teaming under Horizon Europe, complementary funding from 
national, regional, European or private sources is required to support 
the new or upgraded Centres of Excellence. While under Horizon 
2020, issues were sometimes experienced by some projects in 
relation to securing complementary funding (as well as with its timely 
arrival)1298, efforts have been made to mitigate these. For instance, 
under Horizon Europe, second-stage proposals should include a 
binding commitment in relation to the necessary complementary 
funding. Some interviewees (e.g. national ministries and 
beneficiaries) indicated this synergy is critically important to creating 
an impact and ensuring that the country concerned is sufficiently 
engaged and stands behind the Teaming project. Nevertheless, 
stakeholders still refer to this complementary funding as potentially 
complicated1299. Some interviewees point out that there is still a 
mismatch between Structural Funds and FP funding, in the sense 
that the Structural Funds have to be allocated early on (often before 
particular Horizon Europe calls come out), and that it is difficult to 
then renegotiate the allocations. One positioning paper indicates that 
under Horizon 2020, the failure to demonstrate access to 
complementary funding was one of the main shortcomings found 

1297 Based on Corda data. For more information, see the Main Report, Table ‘Overview of complementary funding (in EUR million)’. 
1298 European Commission, Directorate-General for Research and Innovation, Budraitis, M., Pranckevičius, P., Dėlkutė, R. et al., Evaluation study of 
the European framework programmes for research and innovation for excellent science – Horizon 2020 – Annexes, Luxembourg: Publications Office 
of the European Union, 2023, available at: https://data.europa.eu/doi/10.2777/353383 
1299 e.g. the Initiative for Science in Europe (ISE) (2023), ISE Response to the European Commission Public Consultation on the past, present and 
future of the European Research & Innovation Framework programmes 2014-2027. 

855 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

during the Stage 1 evaluation process1300. In addition, several 
operational programmes that could serve as a source for 
complementary funding, provide support for infrastructure and 
equipment only when there is sufficient demand from the private 
sector, which is not always the case for research activities at lower 
TRLs.  

With regard to Excellence Hubs, the relevant Frequently Asked 
Questions (FAQs) elucidate that a commitment from regional 
authorities in terms of participation in the proposal is sufficient, and 
that financial commitments may be included in the investment plans, 
to be elaborated during the lifetime of the project.1301 

The continuity analysis1302 provides information on the continuation 
of research projects. Within Horizon Europe WIDERA, we see the 
largest knowledge inflow from Horizon 2020’s Pillar 1 into Horizon 
WIDERA (512 researchers). Meanwhile, 494 researchers flowed 
from FP7 Cooperation into WIDERA. 

ERA: WIDERA support for programme-level collaboration between 
national R&I policy-makers is expected to result in the identification 
of common R&I priorities being agreed among the participating 
national and regional R&I programmes, as well as the 
implementation of multiannual joint calls and other joint R&D 
activities between participant Member States. In particular, this 
group of actions under the ERA part of Horizon Europe 
complements the European Partnerships identified in the Strategic 
Plan for Horizon Europe by providing the possibility for Member 

1300 Guild of European Research-Intensive Universities (2023), Lessons learned and way forward: the Guild’s recommendations for Horizon Europe. 
1301 https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/support/faq/30740, accessed 29 August 2023. 
1302 More information can be found in the network analysis in Annex 6 ‘Quantitative research track findings’.  

856 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

States, associated countries and civil society organisations to 
maintain existing collaborations and establish new ones on priorities 
of their choice. However, as already mentioned in the overview of 
project implementation (see Section 5.2.2.), the progress of 
WIDERA in this field has so far been limited, with only three grants 
for programme-level collaboration between national R&I policy-
makers signed since 2021. 

The available evidence also indicates that there are significant 
potential synergies between Horizon Europe ERA projects and 
Erasmus+; in particular, the European University Alliances initiative. 
A case study found that a project consortium implementing activities 
in the field of the professionalisation of research management (RM) 
and aiming to develop a roadmap for the RM profession involved 
one of the European Alliances as an associated partner. This 
collaboration is expected to result in mutual benefits and synergies: 
the results of the ERA project are highly relevant to developing the 
skills and competences of research managers leading European 
Alliances, while the involvement of an Alliance in the WIDERA 
project provided the project consortium with the necessary 
knowledge in the field of research infrastructure management. 
Moreover, WIDERA support in the field of fostering the 
professionalisation of RM is also expected to contribute to European 
University Alliances in terms of identifying ways and measures to 
make RM positions compatible across different countries – a 
challenge faced by multiple Alliances. Synergies between WIDERA 
(ERA) projects focusing on the professionalisation and development 
of RM and European University Alliances is further confirmed by the 
recent progress report on the University Alliance Projects supported 
under Horizon 2020. This report emphasised the crucial role played 
by professional research managers in the success of Alliances: 
professionalisation would therefore ensure that there are leaders 

857 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

who have a full overview of an institution's research capacities and 
who have the authority to make resource allocation decisions. 

In some cases, ERA projects have also been complementary to 
equivalent national-level initiatives in Member States: e.g. the EU 
Contest for Young Scientists builds upon national-level events, as 
participants in the EU-level contest are projects nominated by the 
national organiser in each participating country (typically, the 
winners of contests organised at national level). 

Coherence between Horizon Europe ERA projects and other, non-
FP funding schemes is also evidenced by complementarities in 
funding: according to the survey evidence, around 19% of 
beneficiaries (21 out of 108) applied for additional funding for the 
research idea/activities addressed in their Horizon Europe project 
(including applications for national/regional R&D public funding 
schemes, other European funding schemes, international research 
funding schemes, private R&D funding or other sources). Moreover, 
the majority of these organisations (57%, or 12 out of 21) were 
granted all or at least part of the additional funding they applied for. 

CH2* 

What is the 
positioning of the 
Framework 
Programme in 
supporting 
excellent science 
within the overall 
European research 
and innovation 
landscape (incl. 

The strength of links/correspondence 
between the objectives of the 
Excellent Science pillar and the key 
EU strategic priorities in terms of 
research excellence (e.g. the ERA 
priorities for 2025); 

The existence of mechanisms for 
achieving synergies between the 
activities supported under Excellent 

Literature 
review 

Interviews 

Case studies 

Surveys 

ERC: The ERC has been able to contribute to the green and digital 
transitions, which is one of the ERA priority areas. This contribution 
includes, for example, projects in the fields of energy and mobility 
(see the report ‘ERC frontier research contribution to the European 
Green Deal’). 

On 29 September 2022, the ERC took part in the annual European 
Research and Innovation Days, which bring together policy-makers, 
researchers, entrepreneurs and citizens to debate and shape the 
future of R&I. The panel discussed the importance of science 

858 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

R&I funds at 
national, regional 
and European 
level) and beyond 
(at international 
level)? 

Science and other 
national/regional/EU/international 
programmes, e.g.: 

• 

Financial 
complementarity/synergy 
mechanisms (pooling a 
critical mass of resources); 

•  Mechanisms for the 

dissemination of good 
practices and policy 
learning. 

Other complementarity/synergy 
mechanisms – e.g. awareness raising 
and communication activities. 

communication and public engagement, particularly among young 
citizens (see the Annual Report on the ERC activities and 
achievements in 2022).  

The ERC also provides additional funding for ERC grantees within 
the scope of the Public Engagement with Research Award, with the 
objective of recognising and rewarding outstanding science 
communicators among those PIs who successfully engage 
audiences outside their domain with ERC-funded research. 

In addition, the survey results (ERC-BEN-021) indicate that most 
respondents (74.1%, or 186 out of 250) did not apply for any 
additional funding for the research idea/activities addressed in the 
ERC project, while one-fifth (47 out of 250) applied for 
national/regional R&D public funding schemes programmes.  

MSCA: By fostering international competition among scientists and 
institutions in both academic and non-academic sectors, the MSCA 
contribute in particular to the goals of the ‘Jobs, growth and 
investment’ agenda, the EU Global Strategy, and the SDGs. This is 
achieved through the creation and sharing of high-quality knowledge 
across countries, sectors and disciplines. 

The structure of the MSCA programme are especially well aligned 
with the following Horizon Europe specific objective: ‘to develop, 
promote and advance scientific excellence, to support the creation 
and diffusion of high-quality new fundamental and applied 
knowledge, of skills, technologies and solutions, to support training 
and mobility of researchers, to attract talent at all levels and 
contribute to the full engagement of the Union's talent pool in actions 
supported under the Programme’. The MSCA address this specific 
objective by equipping researchers with new knowledge and skills 
through mobility and exposure across borders, sectors and 

859 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

disciplines, enhancing training and career development systems 
while also refining institutional and national recruitment processes, 
all while upholding the principles of the European Charter for 
Researchers and the Code of Conduct for the recruitment of 
researchers. 

The MSCA play a significant role in contributing to (and are in line 
with the priorities of) the European Research Area (ERA), a unified 
research and innovation space in Europe that aims to enhance the 
coordination and collaboration of research activities across EU 
Member States and associated countries. The MSCA programme is 
especially well aligned with the ERA priority area, thereby deepening 
a well-functioning internal market for knowledge. In this regard, the 
MSCA are coherent with the ERA’s objective of fostering 
international collaboration. The MSCA contribute to this by 
supporting researchers from around the world to work in Europe, 
and enabling European researchers to collaborate with international 
partners. ERA also promotes gender equality and diversity in R&I. 
The MSCA include measures to address gender imbalances in 
research careers and to encourage the participation of 
underrepresented groups, thus aligning with the ERA’s commitment 
to inclusivity. Another key component of ERA is the development of 
research careers. The MSCA provide researchers with opportunities 
to develop their skills, gain interdisciplinary experience, and 
transition between different career stages. This aligns with the 
ERA’s emphasis on supporting researchers at all stages of their 
careers and promoting sustainable research careers. Also, both the 
ERA and MSCA emphasise the importance of Open Science and 
open access to research results. Researchers funded by the MSCA 
are encouraged to share their findings openly, contributing to the 
dissemination of knowledge and fostering collaboration across 
Europe and beyond. Lastly, another key objective of the ERA is to 

860 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

promote researchers' mobility and cross-border collaboration. This 
mobility element is central to the MSCA, as it directly supports 
researchers to move between countries, institutions and sectors. 

INFRA: INFRA has provided significant support over time to the 
structuring of the landscape of pan-EU research infrastructures, 
particularly through its support for infrastructures included in the 
ESFRI Roadmap. The EU added value of providing support to 
connect infrastructures and associated services at EU level has 
been shown over time, yet there are concerns as to the long-term 
sustainability of these efforts. These concerns not only relate to the 
increasing number of pan-EU infrastructures, but also to the 
commitment of Member States to financially sustain their national 
laboratories and equipment and to contribute to non-national 
laboratories and equipment. This is especially visible in the case of 
ERICs: ERICs are, in principle, funded through contributions from 
their member countries; however, this contribution is typically not 
sufficient to cover the intended joint activities such as the 
coordination of R&I agendas. Nor does it cover the investments 
needed to keep ERICs at the state-of-the-art. The INFRA 
programme part thus remains a very important source of funding for 
the financing of ERICs, whereas the aim was to decrease the 
dependence of pan-EU RIs on European funding from the 
Framework Programme.  

WIDERA: 

Widening: Under WIDERA, synergies between funding sources are 
envisaged in the Teaming and Excellence Hubs in order to ensure 
the sustainability of their results (work programmes). A general 
overview of where Horizon Europe and other EU funding sources 
have complemented each other reveals 15 cases in which WIDERA 
and other EU funding schemes have supported thematically related 

861 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

projects. The EU funds received as complementary funding amounts 
to EUR 20.1 million. 

In Teaming under Horizon Europe, complementary funding (from 
national, regional, European or private sources) is required to 
support the new or upgraded Centres of Excellence. While under 
Horizon 2020, some projects experienced issues1303, efforts have 
been made to mitigate these as described above. Nevertheless, 
stakeholders still refer to this as potentially complicated1304, and 
some interviewees still pointed out that, in their opinion, the 
Structural Funds are somewhat inflexible in the sense that they must 
be allocated early on, often prior to the particular Horizon Europe 
calls coming out, and that it is difficult to renegotiate these 
allocations later. In addition, several operational programmes that 
could serve as sources for complementary funding, provide support 
for infrastructure and equipment only when there is sufficient 
demand from the private sector, which is not always the case for 
research activities at lower TRLs. With regard to Excellence Hubs, 
the FAQs elucidate that a commitment from regional authorities in 
terms of participation in the proposal is sufficient, and that financial 
commitments may be included in the investment plans, to be 
elaborated during the lifetime of the project.1305 

The survey indicates that 63.6% of successful WIDERA Widening 
respondents (119 out of 189) did not apply for any additional funding 

1303 European Commission, Directorate-General for Research and Innovation, Budraitis, M., Pranckevičius, P., Dėlkutė, R. et al., Evaluation study of 
the European framework programmes for research and innovation for excellent science – Horizon 2020 – Annexes, Luxembourg: Publications Office 
of the European Union, 2023, available at: https://data.europa.eu/doi/10.2777/353383 
1304 e.g. Initiative for Science in Europe (ISE) (2023). ISE Response to the European Commission Public Consultation on the past, present and future 
of the European Research & Innovation Framework programmes 2014-2027. 
1305 https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/support/faq/30740, accessed 29 August 2023.  

862 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

for the research idea and activities addressed in their WIDERA 
project. Meanwhile, 17.6% (33 out of 189) applied for national or 
regional R&D public funding, and 6.4% (12 out of 189) applied for 
other European funding schemes (e.g. the ERDF). Another 12.2% 
(23 out of 189) indicated that they did not know (WIDERA-WIDE-
BEN-020). (Survey of Horizon Europe beneficiaries, conducted May-
July 2023) 

ERA: In general, it is expected that actions funded under the ‘ERA’ 
destination of the programme will be highly complementary to the 
EU policy priorities in R&I, as well as with other national-level R&I 
programmes. As previously mentioned, the entire ‘Reforming and 
enhancing the EU research and innovation system’ destination is 
built around the key ERA strategic policy priorities. One of the key 
areas of intervention is programme-level collaboration between 
national R&I policy-makers, which should enhance synergies and 
complementarities between national programmes, e.g. by supporting 
the implementation of multiannual joint calls, resulting in the funding 
of transnational collaborative R&I projects. 

More specifically, actions aimed at supporting projects in the area of 
citizen science will complement other EU and national programmes 
by identifying the most promising citizen science initiatives for 
transnational upscaling, and proposing how to unlock the necessary 
funding commitments required (e.g. from EU and national 
programmes and funders, as well as philanthropic and/or 
commercial sources).  

EOSC Partnership: In the tripartite governance structure, Member 
States are also involved in the EOSC Partnership, but some 
interviewees raised concerns about the number of national Open 

863 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

CH3* 

What could be 
done to improve the 
coherence of the 
Framework 
Programme 
interventions in this 
area with other 
initiatives to better 
deliver on the 
European Union 
policy objectives? 

Evidence on potentially fruitful 
mechanisms for achieving 
synergies/complementarities between 
the Excellent Science pillar and other 
EU/national/ regional/international 
initiatives; 

Evidence on other measures to 
improve the 
complementarities/synergies between 
the Excellent Science pillar and other 
EU/national/regional/ international 
initiatives, in terms of achieving 
common objectives. 

Science initiatives popping-up. European coordination and alignment 
remain needed. 

Literature 
review 

Interviews 

Case studies 

ERC: As mentioned previously, high-quality ERC Proof of Concept 
proposals that cannot be funded are automatically awarded the Seal 
of Excellence. Even without receiving an ERC Grant, there are 
opportunities for career development through public funding linked to 
ERC projects, such as jobs in ERC teams, Visiting Research 
Fellowships, International Arrangement funding and mentoring. 

MSCA: To better guide Member States as to the opportunities 
available in terms of establishing a national- or regional-level 
scheme to fund Seal of Excellence recipients, the European 
Commission issued a Commission Notice on Synergies between 
Horizon Europe and ERDF programmes. The document outlines the 
new opportunities available to the managing authorities of Cohesion 
Policy programmes, Horizon Europe NCPs and Horizon Europe 
project promoters/proposers. This document is intended to make it 
easier to understand how schemes can be established and 
operationalised, taking stock of national or regional programmes 
supported by the ERDF. 

However, evidence from the position papers on the past, current and 
future of research and innovation programmes (specifically, the 
Coimbra Group) suggests that a recurrent re-allocation of the 
existing Horizon Europe budget to new European Commission-led 
initiatives outside the FP, such as under the European Chips Act, 
the Health and Emergency Response Agency or the Hydrogen 
Valley, creates uncertainty for researchers and innovators, and 

864 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

severely hinders programmes such as the MSCA which (based on 
success rates) appears underfunded. 

INFRA: ERICs are seen as a ‘highly successful mechanism to 
initiate and integrate many RIs at European level, aligning national 
investments and research priorities, pooling resources and 
expertise’1306. An ERIC is a specific legal form that enables Member 
States to cooperate in operating and financing world-class research 
infrastructures. The creation of ERICs is expected to lead to less 
duplication in the RI landscape. By August 2023, 26 ERICs had 
been created and at least another seven ERICs were in the pipeline. 
Each ERIC has an average of 14 member countries, and some span 
almost the entire EU. Moreover, every Member State is part of at 
least one ERIC.  

Another, similar instrument is now being fostered by DG CNNECT – 
the European Digital Infrastructure Consortium, created to help 
interested Member States speed up and simplify the implementation 
of multi-country projects in the digital area. There are indications that 
consortia formed as an ERIC could also implement the types of 
activities expected from EDICs – thus, further alignment between 
both instruments would be necessary to avoid the duplication of 
legal instruments and to ensure coherence. 

WIDERA: 

Widening: In the case of Teaming actions, fruitful synergies are 
produced in the setting up and modernisation of Centres of 
Excellence, as this requires commitments of complementary funding 

1306 Report from the Commission to the Council and the European Parliament, Third report on the application of Council Regulation (EC) No 723/2009 
of 25 June 2009 on the Community legal framework for a European Research Infrastructure Consortium (ERIC), 2023, available at: https://eur-
lex.europa.eu/legal-content/EN/TXT/?uri=COM:2023:488:FIN. 

865 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

by national or regional authorities or private sources. As outlined in 
the questions above, although steps have been taken to better align 
the various funding sources, according to some interviewees there is 
still room for improvement to maximise the potential synergies 
between different types of funding. The Guild positioning paper in 
the context of the OPC1307 recommends to the EC that in situations 
where Teaming requires cofunding for infrastructure and activities at 
low TRLs, these costs should be eligible from the ERDF budget to 
spur synergies. It further recommends that this should be clearly 
communicated to the national authorities responsible. In addition, an 
entirely new instrument, Pathways to Synergies, has been set up to 
stimulate synergies between Horizon Europe and Cohesion Policy 
programmes, mainly the ERDF, INTERREG and the Resilience and 
Recovery Fund (RRF). This pilot aims to incentivise the concrete 
realisation of promising sequential synergies via two different 
pathways – upstream, focusing on human resource development 
and internationalisation; or downstream, focusing on the valorisation 
and upscaling of research results towards marketable solutions. In 
March 2023, a call for proposals was launched. 

Furthermore, in Horizon Europe, a Seal of Excellence is awarded to 
Teaming applicants that exceed the evaluation thresholds set out in 
the work programme, but cannot be funded due to a lack of available 
budget for the call. This Seal of Excellence offers a unique 
opportunity for funding bodies other than the European Commission 
to benefit from the high-quality Horizon Europe evaluation process. 
Regional or national funders can decide to set up a funding scheme 
to support Seal of Excellence recipients using institutional, regional, 

1307 The Guild of European Research-Intensive Universities (2023). Lessons learned and way forward: The Guild’s recommendations for Horizon 
Europe. 

866 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

national or EU funding such as the European Structural and 
Investment Funds.  

As suggested in the evaluation study on Excellent Science under 
Horizon 2020 and underlined in several public consultation papers in 
the context of the OPC1308, it may be worth considering the 
integration of Widening elements across the Framework Programme 
to further enhance synergies. For instance, funding reserve-listed 
proposals submitted by researchers and organisations from 
widening countries (which have thus achieved the threshold) using 
(part of) the Widening budget, could stimulate synergies among 
Widening and other Framework Programme parts.  

ERA: Preliminary evidence indicates that progress has been limited 
in terms of implementing ERA actions in the area of programme-
level collaboration between national R&I policy-makers. Only three 
grants have so far been supported under the topic of ‘Support for 
policy makers – Programme level collaboration between national 
R&I programmes’. Interview evidence also confirms that whereas 
WIDERA (ERA) has succeeded in supporting a wide range of 
projects in the field of citizen science/science and society, as well as 
projects to support university transformation across various fields 
(such as gender equality, entrepreneurship, open access policies 
and practices, RRIs/research ethics etc.), progress with regard to 
support for programme-/policy-level collaboration between policy-
makers has been more limited. This suggests that greater efforts 
should be dedicated to raising the awareness of national policy-
makers about the opportunities provided by Horizon Europe for 

1308 e.g., ISE; European Regions Research and Innovation Network (ERRIN) (2023), Errin Input To The Public Consultation On The Past, Present 
And Future Of The European Research & Innovation Framework Programmes 2014-2027. 

867 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

CH4 

How is the level of 
coherence among 
partnerships, and 
between 
partnerships and 
the Framework 
Programme 
activities in this 
area? Are 
partnerships more 
effective in 
achieving synergies 
for scientific impact, 
compared to other 
modalities of the 
programme? 

Overlaps, gaps, synergies and 
complementarities in terms of 
objectives, activities supported and 
target groups involved in different 
types of partnerships under Horizon 
2020/Horizon Europe; 

Overlaps, gaps, synergies and 
complementarities between different 
types of partnerships under Horizon 
2020/ Horizon Europe and the 
specific programmes under the 
Excellent Science pillar. 

programme-level collaboration, as well as other types of support for 
policy-makers 

EOSC Partnership: Discussions are taking place regarding the 
future, and on new forms for  the partnership for the next FP. Parts 
of the EOSC are now under public procurement by DG CNCT. 
Stakeholders in the EOSC are curious as to how this will affect the 
partnership and, for example, whether core components of the 
EOSC currently developed will be taken up. 

Literature 
review 

Interviews 

Case studies 

EOSC Partnership: Attention has mainly been directed towards the 
internal coherence of the European partnership, and less towards 
collaboration with and learning between different partnerships. 
Interviews indicate that further synergies could be developed 
between partnerships.  

868 

 
 
 
1.3.  Efficiency 

Evaluation 
question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

EFF1 

EFF3 

How efficient have 
the implementation 
processes of (i) 
administration and 
management, (ii) 
project application 
and selection 
processes, (iii) 
funding allocation, 
(iv) forms of 
implementation 
(e.g. partnerships, 
collaborative 
research, blending; 
bottom-up/top-down 
actions)? 

What can be 
learned in terms of 
implementation 
processes from the 
experience of 
applicants and 
participants?  

Cost-effectiveness of 
programmes’ administrative 
arrangements (ratio between 
administrative and 
operational budgets) 

Level of satisfaction among 
applicants, beneficiaries and 
other stakeholders with the 
administrative set-up and the 
administration and 
management processes of 
the respective programme 
part 

Level of ineligible proposals 
suggesting the complexity of 
the application process 

Level of competition at 
selection stage and relevant 
success rates (all/high-
quality proposals)  

Indicators related to the 
timeliness of project 
evaluation, selection and 
grant conclusion procedures:  
(time-to-inform (TTI), time- 

Key findings 

Evidence 

Horizon Europe: The average TTG for Horizon Europe was 275.5 days, 
which exceeds the 8 months (244 days) target by 32 days. This may be 
explained by the European Research Council’s (ERC) long timeframe for TTG 
(maximum of 395 days), as only the Pillar 1 application and selection process 
did not fit within the target (292.3 days). All Pillars were fairly close to the 
target date. Overall TTS for Horizon Europe is 106.6 days, which also 
exceeds the maximum limit of 91 days. The total average TTI is 156.6 days, 
which is just above the maximum estimate of 152 days. The average share of 
ineligible proposals under Horizon Europe is 4.0%, with Pillar 2 having the 
highest at 11.2%. The overall success rate among all eligible proposals is 
16.2%, and 27.7% among only high-quality proposals. When looking at the 
distribution of funding, Pillar 2 has the highest average project size (EUR 6 
million) followed by Widera (EUR 2.7 million) and Pillar 3 (EUR 2.7 million). 

ERC: The average TTG for the ERC was 356.7 days, which is within the 
maximum benchmark of 395 days. Similarly, TTI was 193.7 days and TTS 
107.2 days, both falling within the maximum allocated timeframes. The 
average success rate was 14.3%, and the average success rate among high-
quality proposals was 57%, indicating a relatively high rate of competitiveness 
overall. The average share of ineligible proposals for the ERC was 1.2%. 
Almost all payments were pre-financing, with an average total TTP of 11.5 
days, with 91.9% being made within the target. The ERC has so far funded 1 
792 projects via Action Grants, and 355 projects via Lump Sums – the highest 
Lump Sum usage in Horizon Europe.   

MSCA: The average TTG for MSCA was 244.4 days, which is just over the 
maximum limit of 243 days (8 months). TTS is almost within the target, at 91.2 
days. TTI is 152.5 days, which falls just over the maximum number of days 
(152). The average success rate for the MSCA is 16.1% among eligible 

Methods and 
analyses 

Literature 
review 

Monitoring 
data 

Interviews 

Surveys 

869 

 
 
 
 
 
 
 
Evaluation 
question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods and 
analyses 

Key findings 

Evidence 

to-sign (TTS), time-to-grant 
(TTG) 

The extent to which 
applicants, beneficiaries, 
parent DGs and other 
stakeholders are satisfied 
with the quality of the 
evaluation and selection 
process 

Distribution of funding 
allocation by different parts of 
the programme 

proposals, and 21.4% among high-quality proposals. The share of ineligible 
proposals is 1.3%, which is the same as the Pillar 1 average. The average 
project size for the MSCA is the smallest under this pillar, averaging EUR 0.4 
million. The MSCA are mostly funded via Unit Grants (2 786), but have also 
received 57 Lump Sums and seven Action Grants. 

The position paper submitted to the open public consultation on the past, 
current and future of research and innovation programmes by the Coimbra 
Group highlights two issues in relation to the implementation of the MSCA – 
on the one hand, its continuing low success rates; on the other, the 
participation limits placed on institutions without substantial discretionary 
research funding available to subsidise participation.  

Furthermore, the paper submitted by Masaryk University suggests there is a 
lack of clarity in the rules governing the MSCA Staff Exchanges action that 
effectively hinders many research teams and individuals from further 
participation in the scheme. MSCA Staff Exchanges promote collaborations 
with other funding mechanisms such as Erasmus+ or the European Structural 
and Investment Funds, as long as these funding sources do not duplicate 
each other. At the same time, participants in SE projects are required to work 
on the projects full-time. To participate in MSCA SE projects and receive 
payment from sources other than MSCA grants, the only choices are to work 
more than a full-time workload, or rely on non-research-related resources. 
However, this is rarely feasible, especially for institutions in widening countries 
(WC).  

In addition, feedback from stakeholders indicates that it could be beneficial to 
revise the MSCA allowances and correction factors in order to account for 
heterogeneity between countries, in order to maintain the attractiveness of the 
MSCA programme and for it to continue to act as a reference for PhD 
candidates and early-stage researchers. Adjusting the country correction 
coefficients (CCCs) for living allowances would be particularly relevant to the 
Doctoral Networks and Postdoctoral Fellowships schemes, as the current 

870 

 
 
Evaluation 
question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods and 
analyses 

Key findings 

Evidence 

level of the CCCs might potentially explain the low number of early-stage 
researchers choosing certain countries over others as research destinations.   

INFRA: The average TTG for INFRA was 226.3 days, which falls within the 
benchmark limit of 243 days. Average TTS was 130.3 days, which is well over 
the benchmark of 91 days, and average TTI was 96.8 days which was fairly 
efficient. Overall, INFRA had the highest average success rate, both among 
all applications (53.4%), and among high quality application (65.8%). It also 
had the highest share of ineligible proposals (11.3%). This may be explained 
by the nature of the programme and the fact that it received only 167 
proposals. Projects had an average EC contribution of EUR 6.5 million. 
Evidence from the survey suggests that INFRA applicants may have had a 
more difficult time finding relevant funding opportunities and greater difficulties 
with the clarity of the instructions for the development of impact statements, 
which may have contributed to the relatively high rates of ineligibility. 

WIDERA: WIDERA was the most efficient programme part, with an average 
TTG of 220.7 days, a TTS of 94.1 days, and a TTI of 117.8 days. The share of 
ineligible proposals was 2.8%, and the average success rate was 27.8% 
among all proposals and 48.2% among high-quality proposals. TPP was 35.8 
days, falling within the maximum target of 30 days, with 72% of payments 
being made within the target.  

Widening: 

With new actions introduced, some unexpected hiccups have occurred in the 
application and implementation of these projects. Many of these have already 
been identified and are being addressed by REA and DG RTD. Various 
stakeholders (e.g. NCPs) have emphasised that the widening landscape is 
now rather large and contains many different topics, and that potential 
applicants still need to familiarise themselves with the new actions. The 
survey partly corroborates this view, with half of successful applicants (97 out 
of 196 respondents) indicating that they were unsatisfied, or only satisfied to a 

871 

 
 
Evaluation 
question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods and 
analyses 

Key findings 

Evidence 

moderate or small extent, with the ‘ease of identifying relevant funding 
opportunities across different Horizon Europe programme parts’ (Survey of 
Horizon Europe beneficiaries, conducted in May-July, 2023).  

 Other concerns raised by stakeholders during interviews or in the positioning 
papers of stakeholders in the context of the OPC are as follows:  

With regard to the Hop On Facility, some applicants indicated that it is 
burdensome to search for the contact information of eligible project 
coordinators and to go through the long application and approval processes 
and arrangements.1309 At the same time, many potential applicants in non-
widening countries are unaware of this funding scheme.  

ERA: Administrative burden and long processes of proposal evaluation (which 
often took around half a year) were in some cases identified as a source of 
difficulties, particularly for project coordinators. Some organisations also 
indicated that they experienced difficulties in relation to the new Certificate of 
Financial Statement (CFS). In some cases, this new form wasn't recognised 
by Accountancy Europe and the Dutch Association of Accountants, which 
presented significant challenges for beneficiary organisations, as they were 
unable to upload final reports without the CFS. 

EOSC Partnership: Many interviewees stressed that EOSC projects are not 
the most cost-effective way to contribute to the infrastructure objective of the 
Partnership. Building consortia and proposals of projects that will come to an 
end and which are independent of the EOSC Association or other EOSC 
projects is not a cost-effective way to implement the more operational and 
tangible infrastructure tasks that lie ahead. Especially in the wake of Horizon 
2020 projects, a need was felt to improve interactions between the EOSC 
projects in order to reduce fragmentation and enhance the take-up of project 
results by the EOSC community – and, for example, enhance the EOSC 

1309 https://errin.eu/news/errin-input-horizon-2014-2027-consultation 

872 

 
 
 
Evaluation 
question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

How did these 
processes cater for 
flexibility needs in 
implementation? 
What have been 
the barriers or 
drivers? How could 
they be improved or 
what else could be 
done to maximise 
the benefits of the 
Framework 
Programme 
implementation to 
deliver excellent 
science? To what 
extent have the 
programme 
implementation 
processes in this 
area influenced the 
types of projects 
selected? 

EFF2 

Evidence that 
implementation processes 
and arrangements were 
adapted according to the 
changing environment. 

Level of amendments 
requested and approved. 

Differences between 
characteristics of all 
proposals and retained 
proposals in terms of EU 
contributions, project length 
and numbers of participants.  

Differences between the 
shares of applicants and 
beneficiaries by country and 
organisation type.  

Methods and 
analyses 

Literature 
review 

Monitoring 
data 

Interviews 

Case studies 

Key findings 

Evidence 

branding. One interviewee stated that the introduction of the Partnership was 
cost-effective compared with the prior situation: without co-programming.  

Horizon Europe: According to the REA, HADEA and ERCEA Annual Activity 
Reports, COVID-19 continues to impact implementation and smooth 
transition. Beneficiaries requested a large number of project extensions or 
suspensions, which led to many amendments and the rescheduling of project 
reviews and payments. This has led to projects remaining in the portfolio for a 
longer time, and longer times to amend (TTA).  

In addition, the large number of grants transferred to the funding agencies, as 
well as the impact of Brexit and COVID-19, required high levels of staff 
flexibility and resilience. This was due to the resulting increases in staff 
administrative hurdles – in particular with regard to budget implementation 
and the workload created by overlapping timelines and additional 
requirements (such as modified security screening) for the various phases of 
the call and grant management processes. 

Based on the administrative data received by the study team, in 2021, 2022 
and the beginning of 20231310, the ERC, with an EU contribution of around 
EUR 3.6 billion to 2,147 projects, was the most prominent programme part in 
Pillar 1. It was followed by MSCA (approximately EUR 1.2 billion to 2 850 
projects) and INFRA (approximately EUR 523.5 million to 81 projects). In 
WIDERA, ‘Widening participation and spreading excellence’ contributed a 
total of EUR 701.6 million to 254 projects, while ‘Reforming and enhancing the 
European R&I system’ made a total EC contribution of EUR 128.1 million to 
56 projects.  

1310 Data for 2023 cover the period up to the date when the data was received by the study team, and do not reflect the total figures for 2023. 

873 

 
 
 
 
 
 
Evaluation 
question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods and 
analyses 

Key findings 

Evidence 

Stark differences in proposal 
comparison could suggest a 
preference for certain project 
characteristics, whereas 
differences in the share of 
applicants vs beneficiaries by 
country/organisation type 
could suggest one of three 
things, 1) excellent (or poor) 
proposals from applicants 
from a given 
country/organisation; 2) the 
influence of the programme’s 
implementation on project 
selection; or 3) differences 
simply due to the random 
variations. 

ERC: In the first year of Horizon Europe (as stated in the 2021 work 
programme), the ERC Scientific Council reviewed the ERC panel structure to 
redefine the profiles of panels, enrich the descriptors and consider previous 
application numbers. This also resulted in the addition of two new panels: SH7 
– Human Mobility Environment and Space, and PE11 – Materials 
Engineering. Similarly, under WP 2024, in the domain of Life Sciences, the 
Scientific Council aims to make changes to the LS3 panel title and the LS5 
subtitle and descriptors to make disciplinary coverage more explicit and to 
encourage applicants. In the domain of Social Sciences and Humanities, a 
new panel (SH8) is set to be added to restore the balance in coverage 
between social sciences and humanities, and to limit the number of proposals 
submitted to the SH3, SH4, SH5 panels.  

As stated in WP 2023, for ERC Starting and Consolidator Grants from 2023 
onwards, the reference date for calculating the eligibility period is the certified 
date of the successful defence (and not the award) of the applicant’s PhD 
degree. 

MSCA: In 20221, REA adapted the application and evaluation guidelines, as 
well as training materials, to the requirements and features of Horizon Europe. 
This included the manual for evaluators, the e-learning tool for experts and 
vice-chairs, the training and manual for project officers (PO), the guide for 
applicants, and the reworked Part B template prepared in close cooperation 
with the DGs EAC and RTD.  

The application process has been streamlined under Horizon Europe, and a 
number of changes have been introduced in specific actions. Postdoctoral 
Fellowships now allow for an additional placement of up to six months in a 
non-academic European organisation based in an EU Member State or 
Horizon Europe associated country, provided the placement takes place at the 
end of the fellowship. Under the Doctoral Networks, the number of eligible 
fellow-months in projects was reduced to 360 (or up to 540 in the case of 
industrial or joint doctorates). Staff Exchanges has made interdisciplinary 

874 

 
 
Evaluation 
question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods and 
analyses 

Key findings 

Evidence 

secondments within Europe possible and reduced the number of person-
months to 360, to make projects more manageable. Under the COFUND 
action, the EU contribution now covers minimum remuneration and can be 
used more flexibly.  

INFRA: Interviewees indicated that the introduction of more agile and flexible 
implementation modes for certain types of calls – most notably those involving 
access to transnational users – would be appreciated. An example of an 
alternative implementation mode mentioned in the interviews would be 
cascade funding. There are, however, some doubts regarding the introduction 
of this type of system: 1) participants are used to applying the existing mode 
based on the definition and reporting of transnational access costs; 2) they 
usually do not have experience in managing cascade funding systems; 3) 
there would be transition costs related to the change of system, to be born 
both by both EC and applicants. 

WIDERA:  

Widening: (also see question EFF1.) As discussed above, the Teaming 
application procedure and implementation has changed significantly under 
Horizon Europe. There are no longer two phases, but one (i.e. no separate 
phases for business plans and execution). In addition, the application 
procedure now consists of two stages instead of one (the first requiring a 
shorter version of the proposal to be submitted, to spare the time of 
applicants).  

ERA: Coordination actions could be further enhanced by founding a yearly 
European policy forum for regional and local R&I ecosystems – a meeting 
place for in-depth dialogue on R&I practices and challenges across European 
regions, which could be developed on the Week of Innovative Regions in 
Europe (WIRE). This would allow policy-makers, public authorities, academia 
and businesses to meet and exchange knowledge, as well as further 

875 

 
 
Evaluation 
question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods and 
analyses 

Key findings 

Evidence 

developing and strengthening regional innovation ecosystems and creating 
connections between them1311. 

EOSC Partnership:  The process of updating the SRIA starts within the 
thematic taskforces of the EOSC Association, where the community comes 
together to specify in which direction development should be taken. This then 
continues its lifecycle as a working document of the Commission and, after 
the new SRIA is accepted, forms the basis for new Horizon Europe calls for 
projects. Since EOSC-A is open to all, and the taskforces are open and 
transparent, it means all relevant actors can have a seat at the table.  

EFF4.1 

To what extent are 
project application, 
management and 
reporting being 
performed by 
organisations other 
than those 
performing the 
research and 
innovation 
activities? 

The extent to which external 
organisations (e.g. 
specialised consultancy 
companies) are being 
involved in application, 
management and reporting 
processes. This was 
approached in two ways. 
First, a survey asked 
respondents if they had used 
external help at various 
stages of project cycle. 
Second, projects were 
identified in which at least 
one member of the 
consortium was a project 
management organisation. 

Literature 
review 

Monitoring 
data  

Surveys 

Horizon Europe: Across Horizon Europe, the majority of survey respondents 
reported receiving some form of help to prepare their application. Around half 
(51%) reported receiving support from a dedicated department in their 
organisation; one-fifth (19%) from the NCP; and 17% from an external 
consultant. A minority of respondents (6%) reported providing this type of help 
to their consortia (i.e. they are consultancies themselves). Meanwhile, 29% 
reported receiving no help at all. 

ERC: in the survey of successful ERC applicants, 25.3% of respondents (64 
out of 253) reported receiving support from a consultancy firm or expert (ERC-
BEN-016). Among unsuccessful ERC applicants, almost 15.2% drew on 
support from a consultancy firm or expert (223 out of 1467; ERC-APPL-016). 

MSCA (excluding PF): The following shares of survey respondents indicated 
they had used outside resources: 46% had received help from a dedicated 
department within their organisation; 25% received help from an NCP; 23% 

1311 https://errin.eu/news/errin-input-horizon-2014-2027-consultation 

876 

 
 
 
Evaluation 
question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods and 
analyses 

Key findings 

Evidence 

Manually checks were 
carried out to ensure that this 
organisation’s work portfolio 
only included project 
management and 
consultation, and no 
research activities. In this 
way, we can be sure that the 
projects identified had used 
project management 
services.  

The scope and types of 
services contracted to 
external organisations. 

Identification of the 
underlying reasons for 
employing external 
consultants. 

Estimated costs of services 
related to the application and 
management process. 

from a consultancy; 18% said their organisation provided this type of help to 
the consortium; and 26% reported receiving no help. 

MSCA PF: The following shares of survey respondents indicated they had 
used outside resources: 64% had received help from a dedicated department 
within their organisation; 13% received help from an NCP; 5% from a 
consultancy; and 29% reported receiving no help. 

INFRA: The following shares of survey respondents indicated they had used 
outside resources: 21% received help from a dedicated department within 
their organisation; 18% received help from an NCP; 15% from a consultancy; 
42% said their organisation provided this type of help to the consortium; and 
33% reported receiving no help. 

WIDERA: The following shares of survey respondents indicated they had 
used outside resources: 26% received help from a dedicated department 
within their organisation; 29% received help from an NCP; 19% from a 
consultancy; 10% said their organisation provided this type of help to the 
consortium; and 42% reported receiving no help. 

Widening: In the survey of successful WIDERA Widening applicants, 22.6% 
of respondents (14 out of 62) reported receiving support from a consultancy 
firm or expert within or outside the consortium (WIDERA-WIDE-BEN-013). Of 
these applicants, 64.3% (9 out of 14) indicated they had paid the consultancy 
firm for services in preparing the application. The minimum amount paid was 
EUR 1,000-2,000, while the maximum was EUR 10,000. Among unsuccessful 
WIDERA Widening applicants, almost 20% sought support from a consultancy 
firm or expert within our outside their consortium (26 out of 131; WIDERA-
WIDE-APPL-008).  

ERA: According to the results of the beneficiary survey, only one coordinating 
institution confirmed that it had received support from a consultancy 

877 

 
 
 
Evaluation 
question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods and 
analyses 

Key findings 

Evidence 

EFF4.2 

What are the 
underlying reasons 
and implications 
(e.g. in terms of 
costs, quality of 
applications, R&I 
activities) for the 
beneficiaries and 
for the 
Commission? 

The extent to which external 
organisations (e.g. 
specialised consultancy 
companies) are being 
involved in application, 
management and reporting 
processes. This was 
approached in two ways. 
First, a survey was run and 
respondents were asked if 
they have used external help 
at various stages of the 
project cycle. Second, 
projects were identified in 
which at least one member of 
the consortium is a project 
management organisation. 
These were manually 
checked to ensure that the 
work portfolio of such 
organisations only includes 
project management and 
consultation; no research 
activities. In his way, we can 
be sure that the projects 

firm/expert (within or outside the consortium) when preparing its project 
proposal, at a total cost of EUR 15 000. 

Horizon Europe: Based on 658 valid responses to the question, ‘What was 
the total amount that your organisation paid to the consultancy firm/expert for 
the above services in project application/proposal writing?’, the estimated 
median fees paid are EUR 7,500 for consortia, and EUR 2 000 for single 
beneficiaries. We estimate that applicants have spent between EUR 39 million 
and EUR 55 million on consultancy fees so far across Horizon Europe, 
representing 0.2% of the total Horizon Europe budget. 

ERC: As a single-beneficiary programme, the median consultancy fee 
reported by survey respondents was EUR 2,000. We estimate that applicants 
have spent a total of between EUR 4.99 million and EUR 5.99 million on 
consultancy fees to date, or 0.1% of the programme part’s total EU 
contribution. 

MSCA (excluding PF): As a consortium-based programme part, the median 
consultancy fee reported by survey respondents was EUR 7,500. We estimate 
that applicants have spent a total of between EUR 3.33 million and EUR 4.73 
million on consultancy fees to date, or between 0.4 and 0.5% of the 
programme part’s total EU contribution. 

MSCA PF: As a single-beneficiary programme part, the median consultancy 
fee reported by survey respondents was EUR 2,000. We estimate that 
applicants have spent a total of between EUR 1.22 and EUR 1.83 million on 
consultancy fees to date, or between 0.2% and 0.3% of the programme part’s 
total EU contribution. 

INFRA: As a consortium-based programme part, the median consultancy fee 
reported by survey respondents was EUR 7,500. We estimate that applicants 
have spent a total of between EUR 0.21 million and EUR 0.29 million on 

878 

 
 
 
Evaluation 
question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods and 
analyses 

Key findings 

Evidence 

identified had used project 
management services.  

Scope and types of services 
contracted to external 
organisations. 

Identification of the 
underlying reasons for 
employing external 
consultants. 

Estimated costs of services 
related to the application and 
management process. 

Estimated costs of proposal 
preparation; variation in costs 
across different parts of the 
programme. 

Types of measures to lower 
costs of application and to 
address oversubscription 
issues. 

Types of simplification 
measures introduced into the 
implementation of specific 
programme parts and 
actions; impacts of 
simplification measures on 

EFF8.1 

How proportionate 
were the costs of 
application and 
participation borne 
by different 
stakeholders 
groups, taking into 
account the 
associated 
benefits? Are the 
administrative costs 
borne by applicants 
and participants 
lower, higher or 
constant if 

consultancy fees to date, or between 0.04% and 0.05% of the programme 
part’s total EU contribution. 

WIDERA: As a consortium-based programme part, the median consultancy 
fee reported by survey respondents was EUR 7,500. We estimate that 
applicants have spent a total of between EUR 1.29 million and EUR 1.90 
million on consultancy fees to date, or around 0.2% of the programme part’s 
total EU contribution. 

Literature 
review 

Monitoring 
data 

Interviews 

Case studies 

Horizon Europe: Survey data indicates that the median value for the 
coordinators was 36-45 days, suggesting that, on average, consortium 
coordinator spends between 36 and 45 person-days preparing a Horizon 
Europe proposal. For partner organisations, this effort is smaller. The median 
partner spends between 16-25 person-days to prepare a Horizon Europe 
proposal. 

Following the observations made by the European Court of Auditors in its 
annual report, the funding agencies report having planned and implemented 
further simplifications. These include the increased use of simplified forms of 
funding (including Lump Sums and unit costs), focused communication 
campaigns targeting more ‘error-prone’ types of beneficiaries with higher-
than-average error rates (such as SMEs and newcomers), and enhanced 
training for external audit firms performing audits on behalf of the 
Commission. 

879 

 
 
 
 
 
 
 
 
 
 
Evaluation 
question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods and 
analyses 

Key findings 

Evidence 

compared with the 
previous 
Framework 
Programme? 
Please quantify 
them to the extent 
possible. How do 
costs and benefits 
of applying to the 
Framework 
Programme 
balance out on the 
side of applicants? 

the administrative burden on 
applicants and beneficiaries 
and the workload of the 
Commission services; further 
scope for simplification; 
evidence of effective take-up 
of new simplification 
measures.  

Evidence of streamlining and 
harmonisation of funding 
rules and procedures across 
different programme parts 
and actions, systems for 
electronic data management 
and electronic data exchange 
between the administration 
and beneficiaries. 

Impact of the simplification 
measures on the 
administrative costs of the 
Commission services and the 
timeliness of programme 
management functions. 

According to submissions to the open public consultation, an increase in 
application requirements (Gender Equality Plan (GEP); Data Management 
Plan (DMP); ethics considerations, etc.) necessitates the spending of 
additional time, expertise and financial resources on applications, escalating 
the administrative costs borne by the applicants. In addition, some also cited 
the need to seek experts in order to complete the additional sections of the 
application template. Nonetheless, most stakeholders did not have an opinion 
(55%, or 868 out of 1,578), or thought that the effort needed to participate in 
Horizon Europe was similar to that for Horizon 2020 is similar (36% or 568 out 
of 1 578). 

Lump Sums were introduced by the EC as means of simplification. The 
majority of these (413) have been used in Pillar 1, followed by Pillar 3 with 97. 
However, according to submissions to the open public consultation, the roll-
out of the Lump Sum model has added another layer of complexity and 
additional workload to the pre-award proposal submission process, shifting 
the administrative burden from the EC to participants. This appears to have 
had the opposite impact to simplification. The greater uncertainty, as well as 
the lack of sufficient understanding and data regarding the merits and defects 
of Lump Sums, need to be further assessed. 

Under Horizon Europe, the process for grant applications has been 
streamlined, creating a single-entry point for grant applications called the 
Funding and Tenders Portal. Most submissions to the open public 
consultation indicate that while centralising information and proposal 
submission tools in the Funding and Tenders Portal is a progressive move, its 
complexity and error-proneness demand simplification to ensure enhanced 
user-friendliness. Technical malfunctions are still widespread and need to be 
addressed. 

ERC: The effort involved in preparing proposals for mono-beneficiary 
applications (such as the ERC and MSCA PF) is comparable to that seen in 
the consortium coordinators’ sample. The median ERC Principal Investigator 

880 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation 
question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods and 
analyses 

Key findings 

Evidence 

or MSCA Postdoctoral Fellow spends between 36 and 45 days preparing a 
Horizon Europe application. 

As stated in WP 2023, from 2021 onwards, it was no longer possible for 
applicants to opt out of the submission of Research Data Management plans. 
In addition, the evaluation elements of the Proof-of-Concept Grant were 
modified slightly. As stated on the ERC’s website, the Scientific Council 
agreed at the plenary meeting in February 2023 to the use of Lump Sum 
funding for Advanced Grants awarded following the 2024 call. 

The comments noted above in submissions to the open public consultation 
also apply to the ERC.  

MSCA: The effort involved in preparing proposals for mono-beneficiary 
applications such as the ERC and MSCA PF is comparable to that seen in the 
consortium coordinators’ sample. The median ERC Principal Investigator or 
MSCA Postdoctoral Fellow spends between 36 and 45 days preparing a 
Horizon Europe application.  

Widening: As discussed above under Teaming, the application procedure 
now consists of two stages instead of one (the first requiring a shorter version 
of the proposal to be submitted, to spare the time of applicants), in order to 
mitigate the costs of proposal preparation. In addition, the issues experienced 
under Horizon 2020 in relation to complementary funding have been mitigated 
under Horizon Europe.  

Are the 
administrative costs 
borne by applicants 
and participants 
lower, higher or 
constant if 

EFF8.2  Estimated costs of proposal 

preparation, variation in costs 
between different parts of the 
programme. 

Literature 
review 

Monitoring 
data 

ERC: According to the survey, the median ERC participant spends between 
6% and 10% of the project budget on administrative tasks. 62% of ERC 
respondents to the survey ‘agreed’ or ‘strongly agreed’ with the statement, 
‘The burden of the administrative and legal requirements for granting 
procedures was proportionate.’ 

881 

 
 
Methods and 
analyses 

Interviews 

Case studies 

Key findings 

Evidence 

MSCA: According to the survey, the median MSCA PF participant spends 
between 6% and 10% of the project budget on administrative tasks. 46% of 
MSCA PF respondents to the survey ‘agreed’ or ‘strongly agreed’ with the 
statement, ‘The burden of the administrative and legal requirements for 
granting procedures was proportionate.’ 

Out of MSCA participants (excluding PF) who responded to the survey, 68% 
‘agreed’ or strongly agreed with the statement, ‘The burden of the 
administrative and legal requirements for granting procedures was 
proportionate. 

INFRA: 47% of INFRA respondents to the survey ‘agreed’ or ‘strongly agreed’ 
with the statement, ‘The burden of the administrative and legal requirements 
for granting procedures was proportionate.’ 

WIDERA: 61% of WIDERA respondents to the survey ‘agreed’ or ‘strongly 
agreed’ with the statement, ‘The burden of the administrative and legal 
requirements for granting procedures was proportionate.’ 

Evaluation 
question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

compared with the 
previous 
Framework 
Programme? 
Please quantify 
them to the extent 
possible. How do 
costs and benefits 
of applying to the 
Framework 
Programme 
balance out on the 
side of applicants? 

Types of measures to reduce 
the cost of application and 
address oversubscription 
issues. 

Types of simplification 
measures introduced in the 
implementation of specific 
programme parts and 
actions; impact of 
simplification measures on 
the administrative burden of 
applicants and beneficiaries 
and the workload of the 
Commission services; further 
scope for simplification; 
evidence of effective take-up 
of the new simplification 
measures.  

Evidence of the streamlining 
and harmonisation of funding 
rules and procedures across 
different programme parts 
and actions; systems for 
electronic data management 
and electronic data exchange 
between the administration 
and beneficiaries. 

Impact of the simplification 
measures on the 
administrative costs of the 

882 

 
 
Evaluation 
question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods and 
analyses 

Key findings 

Evidence 

Commission services and the 
timeliness of programme 
management functions. 

EFF9 

How to lower costs 
of applications and 
increase benefits 
from participation 
for the applicants 
(i.e. cost of writing 
proposals) and 
Commission 
services (i.e. cost of 
administrating and 
running the 
programme)? 

Literature 
review 

Monitoring 
data 

Surveys 

Horizon Europe: The beneficiaries survey shows that a median consortium-
run project allocates between 6% and 10% of its project budget to 
implementing administrative tasks such as project reporting, project financial 
management and similar. The reported share differs between coordinators 
and partners. The median value for coordinators falls between 11% and 15%, 
while the mode (most frequently chosen) value is between 6% and 10%, 
which provides grounds to assume 6-10% for the total population of projects. 

Submissions to the open public consultation cite several ways in which 
administrative burden could be reduced. These include;  

• 

• 

• 
• 
• 

• 
• 

A faster process to avoid mobilising staff between the proposal and 
contract stages. 
AI support systems to assist with the completion of some 
documents (such as reporting requirements, cost calculations etc.). 
Streamlined and simpler templates.  
Improved guidance and information. 
Increasing the number of two-stage calls, with the initial step being 
shorter and less resource-intensive. 
Clearer communication to avoid miscommunication and confusion.  
Standardisation and harmonisation of practices across different 
aspects.  
Reduction in the frequency and volume of reporting. 

• 
•  More use of Lump Sums. 

883 

 
 
 
Evaluation 
question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods and 
analyses 

Key findings 

Evidence 

EFF12 

Evidence drawn from survey 
and interview responses 
regarding motivations to 
participate and obstacles. 

Interviews 

Case studies 

Surveys 

What could be done 
in the future to 
improve the 
attractiveness and 
outreach of the 
Framework 
Programme to key 
targeted groups 
given its objectives 
and core principles, 
in particular its 
focus on 
excellence? 

ERC: As mentioned previously, the ERC’s sole mission is ‘to encourage the 
highest quality research in Europe through competitive funding and to support 
investigator-driven frontier research across all fields, based on scientific 
excellence.’ As a result, ERC actions do not target specific groups. This 
approach is highly appreciated by beneficiaries, as demonstrated in the 
survey results and interviews. Among others, motivations for applying include 
the positive impact on the researcher’s career due to the prestige of receiving 
an ERC grant, and the quantity of funding and duration of the project (both of 
are greater than for national funding). The freedom to pursue ground-breaking 
research is also one of the main reasons to apply. However, a few position 
papers and replies to the survey by unsuccessful ERC applicants refer to the 
very low success rate, which discourages participation and results in many 
other very competent researchers not being funded. Furthermore, 
participation by private for-profit organisations is very low. 

MSCA: Interviews findings suggest a clear need to revise the unit 
contributions for the MSCA, since the variation in funding coefficients between 
countries increases complexity. According to interviewees, in some cases, the 
coefficients may not perfectly reflect day-to-day realities or even regional 
differences within countries. Currently, the MSCA is undergoing a review of its 
Lump Sum and unit contributions, and should therefore address the concerns 
expressed. 

From the perspective of successful applicant organisations participating in the 
Doctoral Networks action, the interview findings point to the need to better 
align the MSCA programme with national doctoral programmes in many EU 
Member States. Under the current situation, participating organisations are 
compelled to seek additional funding to cover the fourth year of a PhD, as 
opposed to national/regional funding programmes in which funding is usually 
provided for a duration of 48 months. Stakeholders also called for further 

884 

 
 
Evaluation 
question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods and 
analyses 

Key findings 

Evidence 

clarification of the rules guiding the Joint Doctorates (DN-JD), due to the vast 
array of national practices in doctoral education across Europe and beyond. 

INFRA: Evidence shows that private for-profit entities represent only 9% of 
participations in INFRA. The connection between RIs and the private 
sector/industry could be further fostered. One potential mechanism to foster 
RI-industry collaboration could be to strengthen the participation of RIs in 
other parts of the Framework Programme beyond INFRA in which industry is 
more active. A better connection between research and technology 
infrastructures could increase the impact of RIs and boost industrial 
collaboration. 

WIDERA: 

Widening: The networks of NCPs in the widening countries appears to be 
important in increasing awareness and guiding widening institutions in 
applying to WIDERA and Horizon Europe as a whole. It is important to 
continue training and strengthening these.  

With regard to improving the attractiveness of specific actions, one of the 
positioning papers submitted to the open public consultation on the past, 
present and future of the FP suggests that the budget for Twinning projects 
should be increased, and that a separate budget should be allocated for top-
class partners to provide incentives for institutions in non-widening countries 
to participate.1312 For the Hop-On Facility, the number of widening partners per 
project could be increased to spur participation. It has been underlined that in 
general, non-widening countries continuously need to be made aware and 
incentivised to participate in the Widening actions.  

1312 European Regions Research and Innovation Network (ERRIN) (2023). ERRIN Input To The Public Consultation On The Past, Present And 
Future Of The European Research & Innovation Framework Programmes 2014-2027. 

885 

 
 
 
Evaluation 
question 

ID 

Judgement 
criteria/indicators 
identified ex ante 

Methods and 
analyses 

Key findings 

Evidence 

In the future, the FP could integrate Widening elements across the Horizon 
programme to widen participation in all pillars.  

ERA: Interview evidence confirms that whereas WIDERA (ERA) has 
succeeded in supporting a wide range of projects in the field of citizen 
science/science and society, as well as projects to support university 
transformation in various fields (such as gender equality, entrepreneurship, 
open access policies and practices, RRIs/research ethics etc.), progress with 
regard to support for programme/policy-level collaboration between policy-
makers has been more limited. This suggests that greater efforts should be 
dedicated to raising awareness among national policy-makers about the 
cooperation opportunities provided by Horizon Europe for programme-level 
collaboration, as well as other types of support for policy-makers. 

1.4.  Effectiveness 

Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

What are the main 
results and 
(expected and 
unexpected) 

EFC1.1 

Horizon Europe performance targets: 
the extent to which programme parts 
have met their objectives regarding the 
Key Impact Pathways indicators. 

Literature 
review 

Horizon Europe: The European Commission has reported on the 
short-term KIP1 indicator, i.e. the number of peer-reviewed 
scientific publications (although still under development), based on 

886 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

outcomes and 
impacts from the 
projects having the 
delivery of 
excellent science 
as their core 
focus?  

Examples of outstanding results and 
impacts already achieved. 

The evaluation of the outputs and 
results of Horizon Europe projects in 
terms of several indicators, such as: 

Number of FP peer-reviewed scientific 
publications. 

Share of publications from funded 
projects in top journals. 

Number of researchers that have 
benefitted through the FP. 

Evaluation of scientific impact: 

Number and share of peer-reviewed 
publications from FP projects that are 
core contributions to their scientific 
fields. 

Number and share of upskilled FP 
researchers with improved working 
conditions. 

Share of FP beneficiaries that have 
developed new transdisciplinary/trans-
sectoral collaborations thanks to the 
FP. 

Methods and 
analyses 

Monitoring 
data 

Interviews 

Case studies 

Benchmarks 

Surveys 

Bibliometric 
analysis 
(H2020) 

Key findings 

Evidence 

periodic reporting1313. Specifically, 323 projects have produced a 
first-period report (i.e. 3.5% of all projects), out of which 49 (0.5% 
of all projects) contain publication data. Importantly, 119 
publications were flagged as peer-reviewed and had a valid DOI 
(out of a total of 241 publications). From these publications, 42 
were matched and were identified as foreground patent (i.e. 
published after the start of the project). 

ERC: Under Horizon Europe, the ERC’s mission continues to be 
to encourage the highest-quality research in Europe through 
competitive funding, and to support investigator-driven frontier 
research across all fields, based on scientific excellence. The 
bibliometric analysis carried out in the evaluation study on 
Excellent Science under Horizon 2020 shows that, for the period 
2014-2020, ERC-funded research contributed to 49,496 
publications (35.6% of the total number of Horizon 2020 
publications). In addition, the bibliometric analysis shows that each 
ERC-related publication has an average of 24.4 citations (the third 
highest number across all Horizon 2020 programme parts after 
FET and Societal Challenge 1 (SC1) ‘Health, demographic change 
& wellbeing’), while 5.6% of ERC-related publications are among 
the top 1% most-cited publications (also the third-highest 
percentage under Horizon 2020, after SC1 ‘Health, demographic 
change & wellbeing’ (7.4%) and SC5 ‘Climate action, environment, 
resource efficiency & raw materials’ (5.9%). Also, after SC1 and 
SC5, the ERC has the third-highest field-normalised citation score 
(2.32), meaning that an ERC publication is cited on average 2.32 
times more often than the average for the field. The bibliometric 
analysis also shows that at 90.8%, the share of publications 

1313 On 18 October 2023. 

887 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

produced by the ERC that are in open access is the highest 
among all Horizon 2020 programme parts.  

Since 2015, the ERCEA has been conducting a qualitative 
evaluation of completed ERC projects. The latest edition, 
published in the Annual Report on the ERC activities and 
achievements in 2022, uses a random sample of 450 projects and 
concludes that 21% of projects led to a scientific breakthrough and 
59% led to a major scientific advance. Only one-fifth of them were 
found to produce incremental or no appreciable scientific advance.  

Data from the Survey of Horizon Europe ERC beneficiaries shows 
that almost all respondents (236 out of 238) stated that their ERC 
project has produced or is likely to produce research publications 
(in the form of academic articles, monographs, books, etc.). Most 
respondents also expect new/improved research methodologies, 
new large-scale datasets and new or improved tools, methods or 
techniques. Conversely, only 1% (2 out of 238 respondents) 
expected their ERC project to produce large-scale product 
validation and market replication. These types of expected 
outcomes are in line with the types of projects funded by the ERC, 
which focuses on bottom-up fundamental research. 

MSCA: As part of this evaluation, the study team analysed the 
impact of MSCA funding on both novelty and disruptiveness. This 
analysis revealed that there are minimal differences between 
funded and non-funded MSCA applicants before (as is desirable) 
as well as after funding. Exactly 1% of documents by funded 
applicants are novel at the time of application. Non-funded 
applicants are very close, with 0.9% novel documents. At two and 
at five years after application, 5% of documents published by 

888 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

funded applicants are novel. For non-funded applicants, the 
shares are very similar, respectively, 5% and 4.5%.  

Similarly, 0.8% of documents by funded applicants are disruptive 
publications at the time of application. Non-funded applicants are 
very close, with 0.7% disruptive documents. At two and at five 
years after application, 3% of documents published by funded 
applicants are disruptive. For non-funded applicants, the shares 
are very similar, respectively, at 3% and 5%. This is unsurprising, 
given the programme’s competitiveness and relatively low success 
rates – there is a wide pool of excellent researchers who, due to 
budget limitations, are not able to participate in the MSCA 
programme.  

While the MSCA do not increase the capacity of funded applicants 
to produce more novel and/or disruptive research, they do provide 
a critical support system for researchers to continue to do so. By 
providing financial support, resources and opportunities for 
professional development, the MSCA help researchers to maintain 
their research activities and potentially build on existing work. This 
can be particularly important in ensuring the continuity of projects 
with the potential for significant impact. 

INFRA: INFRA is an important enabler of excellent science, given 
that RIs provide the necessary equipment, tools and services for 
scientific research and innovation. The main outputs from INFRA 
projects, as indicated by successful applicants to INFRA in the 
survey, are research publications (83.5%, or 137 out of 164 
respondents indicated their project has produced, or is likely to 
produce these); new or improved tools, methods, or techniques 
(63.4%, or 104 out of 164); new large-scale datasets/data 
produced (57.9%, or 95 out of 164); new/ improved software, 
protocols/guidelines for conducting research (56.7%, or 93 out of 

889 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

164); training programmes (52.4%, or 86 out of 164); and 
recommendations for policy-makers/inputs into national or EU 
policies (50.6%, or 83 out of 164) (see INFRA-BEN-027). 

The main results indicated in the survey by successful applicants 
to INFRA were strengthening relationships with leading partners in 
Europe (81.6%, or 129 out of 158 indicate they will achieve this to 
a ‘large’ or ‘very large’ extent); increasing international visibility 
through collaboration with leading global partners (75.9%, or 120 
out of 148); improving access to physical and knowledge 
infrastructures, facilities and data resources (74.7%, or 118 out of 
158); improving the skills, knowledge and competences of 
researchers (74.2%, or 118 out of 159); and contributing to 
emerging areas of science and technology (63.5%, or 99 out of 
156) (see INFRA-BEN-031). 

WIDERA: 

Widening: While there is only limited bibliometric information so 
far on Horizon Europe, the evaluation study on Excellent Science 
under Horizon 2020 underlined the importance of the Widening 
programme part in Horizon 2020 for widening countries in terms of 
the production of science. In addition, the Widening actions also 
appeared to be key to the production of excellent science in 
widening countries, with 28% of the highly cited Horizon 2020 
publications by widening countries being produced under the 
Widening actions. Moreover, the share of highly cited publications 
in Horizon 2020 Widening actions increased from 6% in 2014 to 
12% in 2020. 

In the survey of successful WIDERA Widening participants, 89.3% 
of respondents (158 out of 177) indicated that the project has or 
would lead to research publications; 71.2% to new training 

890 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

programmes (126 out of 177); and 65.5% (116 out of 177) to 
education material (WIDERA-WIDE-BEN-027). Meanwhile, 37.3% 
of respondents (66 out of 177) indicated that the project would 
(probably) lead to testing, demonstrating and piloting; 25.4% of 
respondents (45 out of 177) that it would lead to the development 
of new or improved products, services or processes; and 12.4% 
(22 out of 177) that it would lead to patents. In addition, 60.6% of 
respondents agreed to a ‘large’ or ‘very large’ extent with the 
statement that their project is pushing or will push the frontiers of 
knowledge (103 out of 170) (WIDERA-WIDE-BEN-031).  

ERA: The development of collaborative ties and networks of 
expertise across borders was also identified as one of the key 
outcomes of WIDERA projects under the thematic priority 
‘Strengthening the ERA’. Survey results indicate that networking 
and the development of cross-border ties between institutions was 
among the most frequently reported effects of this group of 
projects. According to the results of the Horizon Europe 
beneficiary survey, 78% of WIDERA (ERA) beneficiaries (78 out of 
99 respondents) agreed to a ‘large’ or ‘very large’ extent that their 
project has resulted/is likely to result in strengthening relationships 
with leading partners in Europe. This was followed by ‘Increasing 
international visibility through collaboration with leading global 
partners’ (67%; 67 out of 99 of WIDERA (ERA) beneficiary 
respondents). In-depth case study and interview evidence 
confirms that nearly all beneficiary stakeholders confirmed that 
network building is one of the main outcomes of Horizon Europe 
support. The development of clusters and groups of experts 
discussing possibilities and pathways for transformational change 
in different thematic areas (e.g. entrepreneurship, gender equality 
or the professionalisation of research managers in universities) 
has already occurred during the project inception phase. 

891 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

EFC1.2 

Horizon Europe performance targets: 
the extent to which the programme 
parts have met their objectives. 

Examples of outstanding results and 
impacts already achieved. 

Perceptions of beneficiaries regarding 
success factors and barriers to 
achieving these objectives.  

Is the delivery of 
the projects’ results 
all together leading 
to the achievement 
of the programme’s 
scientific 
objective(s)? What 
more is needed to 
be able to reach 
the excellent 
science objectives 
and by which 
timeframe? 

Methods and 
analyses 

Interviews 

Case studies 

Benchmarks 

Surveys 

Bibliometric 
analysis 
(Horizon 
2020) 

Key findings 

Evidence 

ERC: As mentioned previously, the nature of the ERC is to 
contribute to delivering on Horizon Europe’s scientific objectives. 
Interviews and the position papers from the open public 
consultation emphasise the importance of having a substantial 
budget to fund bottom-up basic research and to retain top 
researchers in Europe.  

INFRA: According to interviews and previous evaluations of the 
FP, INFRA has made a substantial contribution to the structuring 
and development of pan-EU research infrastructures, allowing 
them to remain at the vanguard of technological and scientific 
progress, and offering researchers support to access these 
infrastructures. Support from INFRA via its various instruments, as 
well as the increase in infrastructures included in the ESFRI 
Roadmap, has resulted in an increase in the number of pan-EU 
infrastructures. Interviews indicate, however, that there is a risk of 
excessive fragmentation with regard to the user communities, 
which could lead to difficulties regarding the sustainability of 
support.  

WIDERA: 

Widening: Some of the core scientific objectives of the Widening 
programme part are to increase research excellence, to improve 
research management and administrative capacities at institutions, 
to attract top researchers to widening countries, and to strengthen 
scientific capacities in widening organisations. While it still too 
early to assess whether the Horizon Europe Widening actions 
have been successful in these objectives, preliminary evidence 
points to positive results.  

Under Horizon Europe, the survey indicates that many 
researchers expect (high-quality) research outputs. In addition, the 

892 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

focus on capacity building and strengthening the research 
management and administration skills of the coordinating 
institutions is often highlighted as critical in the WIDERA 
(Widening) work programmes. In the survey of successful Horizon 
Europe Widening applicants, slightly over three-quarters of 
respondents (138 out of 180) agreed that their project strengthens 
the knowledge base and R&D capacity in the organisation to a 
‘large’ or ‘very large’ extent, and enhances the R&D and/or 
innovation capabilities and skills in the organisation (136 out of 
180) (WIDERA-WIDE-BEN-025). 

Recommendations by interviewees and stakeholders to further 
foster the programme part’s scientific objectives  include more 
Research and Innovation Actions (RIAs) in the Widening actions; 
integrating (part of) the Widening budget into other programme 
parts (e.g. by funding proposals submitted by researchers from 
widening countries who are on the reserve list)1314; and providing 
schemes that do not require non-widening countries to participate 
(i.e. widening-only schemes), so that widening institutions can truly 
drive collaborative projects and improve their leadership 
capabilities1315. 

ERA: From a longer-term perspective, the outcomes of WIDERA 
(ERA) are expected to contribute to structuring effects at 
institutional level and across the ERA, which are ultimately likely to 
increase the overall excellence and productiveness of institutions 
in European research systems. For example, according to the 

1314 e.g. Initiative for Science in Europe (ISE) (2023). ISE Response to the European Commission Public Consultation on the past, present and future 
of the European Research & Innovation Framework programmes 2014-2027. 
1315 e.g. European Regions Research and Innovation Network (ERRIN) (2023). Errin Input To The Public Consultation On The Past, Present And 
Future Of The European Research & Innovation Framework Programmes 2014-2027. 

893 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

case study findings, projects focusing on the professionalisation of 
research management (RM) are expected to enhance the 
efficiency of national research and innovation systems, primarily 
by freeing up valuable research time (i.e. allowing researchers to 
focus first and foremost on research tasks) and creating a more 
supportive environment for researchers. The expected impact on 
the excellence of research systems resulting from WIDERA (ERA) 
projects is also evidenced by the survey results: 45% of survey 
respondents (45 out of 99) agreed to a ‘large’ or ‘very large’ extent 
that their project had resulted in /was likely to result in pushing the 
frontiers of knowledge. Meanwhile, 40% of respondents (39 out of 
97) agreed to a ‘large’ or ‘very large’ extent that their project had 
resulted in/was likely to result in an increase in research 
productivity (e.g. the number of peer-reviewed publications) 

ERC: Evidence from the two ERC-related case studies, based 
mostly on interviews with ERC beneficiaries and desk research, 
points to several success factors that explain the contribution the 
ERC makes to support new fields of research. These are: 

• 

• 

• 

The curiosity-driven design of ERC grants, which enables 
changes to be accommodated in a rapid manner during the 
course of projects, bringing opportunities for new research in 
rapidly evolving fields. 

Composition of the team and the length of the project, as the 
ERC allows larger and more diverse teams and longer 
projects. 

The evaluation process and governance, such as the 
application of scientific excellence as the sole selection 

EFC2.1 

What internal or 
external factors 
have influenced 
progress or lack of 
progress of the 
Framework 
Programme 
interventions in this 
area towards their 
impact?  

Analysis of factors that have 
influenced the progress (enablers, 
drivers) or lack of progress (barriers) of 
the FP interventions towards their 
impact?  

Identification of solutions to barriers or 
further support for drivers. 

Prioritisation of solutions to barriers or 
further support for drivers. 

Literature 
review 

Interviews 

Case studies 

Benchmarks 

Surveys 

894 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

criterion, and the implementation of standing evaluation 
panels of peers, as well as the independence of the ERC’s 
Scientific Council as the main governing body of the ERC. 

In terms of barriers encountered, the relatively low success rates 
for ERC grants imply that an important proportion of excellent 
research ideas and researchers cannot be funded, and the 
architecture around the evaluation panels might be a factor 
hindering the realisation of the full potential of highly novel and 
emerging research, since there is not always a natural fit with one 
of the evaluation panels – especially in the case of 
multidisciplinary or highly novel research ideas.  

The survey of successful ERC applicants under Horizon Europe 
indicates that the most relevant obstacles to carrying out project 
are (ERC-BEN-036) are the insufficient information and assistance 
provided by the host institution (23.7% of respondents. or 55 out of 
232) and obstacles related to the COVID-19 pandemic (18.3% of 
respondents, or 42 out of 230). 

MSCA: Based on interview findings, this study finds the following 
barriers that might potentially influence the lack of progress of the 
MSCA intervention towards its anticipated impacts:  

• 

Country coefficients: attracting top talent constitutes a 
hurdle, especially for widening countries, which compete 
against non-widening countries. According to interviewees, 
the variations in funding coefficients between countries 
increases this complexity – in some cases, the coefficients 
may not perfectly reflect day-to-day realities or even regional 
differences within countries.   

895 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

• 

• 

• 

Transferable skills: interview findings suggest that 
institutions grapple with enhancing the quality of training for 
researchers, especially on transferable skills such as 
intellectual property rights, project management and 
entrepreneurship.  
Human resources and working conditions: while the 
MSCA promotes mobility, interviewees noted that third 
country researchers often face challenges in visa processes, 
cultural adaptation and balancing personal commitments. 
This mobility is juxtaposed against the career uncertainties of 
temporary positions and limited advancement opportunities, 
especially in certain regions. Another systemic challenge 
relates to the fact that in certain countries such as Ireland, 
PhD students cannot be legally granted full-time employment 
contracts, leading to various complications that can only be 
dealt with by enacting changes to national labour law. 
Building sustainable collaborations and knowledge 
transfer: collaborations between academic and non-
academic sectors present additional challenges. One of the 
primary concerns expressed by most interviewees is the 
alignment of academic research goals with the often short-
term and profit-driven objectives of the private sector. 
Intellectual property rights (IPR) are a significant point of 
contention, with both parties needing clarity with regard to 
ownership, knowledge sharing and commercialisation rights. 
Small and medium-sized enterprises (SMEs) might find the 
complexity of the MSCA scheme daunting, deterring them 

896 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

from participating due to a lack of familiarity with such 
academic-industry collaborations. Knowledge security issues, 
especially when dealing with sensitive technologies, require 
strict protocols and agreements, which can sometimes hinder 
collaboration and the free flow of information.  

•  Open access: open access (OA) publishing, while heralded 
as a move towards democratising scientific knowledge, 
presents several challenges, as expressed by the 
interviewees. One of their foremost concerns is the high cost 
associated with OA publishing rates, which some believe is 
counterintuitive to the very ethos of accessibility that OA aims 
to achieve. In addition, many OA journals suffer from a low 
Impact Factor (IF), a metric often used to gauge the 
significance of a publication, potentially undermining the 
perceived value of a researcher's work. Some criticism was 
also expressed towards OA repositories, which often lack the 
peer-review process altogether, raising concerns about the 
accuracy and validity of data. 

Based on the interview findings, the study finds the following 
drivers potentially influence the progress of the MSCA intervention 
towards its anticipated impacts: 

• 

Clear guidelines and support materials: evidence 
stemming from the interviews suggests that the MSCA 
provide clear guidelines, work programmes and an 
applicant’s guide tailored to each action, ensuring 

897 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

• 

• 

researchers and institutions understand the programme’s 
expectations and adhere to its requirements. 
Strong focus on ensuring adequate supervision or 
mentoring: the MSCA emphasis on the quality of supervision 
is considered another success factor in raising research 
excellence in beneficiary organisations. The MSCA thus 
enable a shift in recognising that good supervision is 
instrumental to ensuring that early-stage and experienced 
researchers receive the guidance, support and mentorship 
they need to excel in their projects and in further 
employment. 
Critical administrative support for the participating 
organisations: interviewees pointed out that the main drivers 
for achieving structuring impacts through the MSCA 
programme are the administrative offices of the participating 
organisations. Researchers participating in MSCA actions 
might come and go, or move between projects and 
institutions; however, the administrative offices typically offer 
continuity, thus enabling and facilitating the transfer of good 
practices that go beyond participation in MSCA. 

Overall, ‘Case Study 4. ‘How the MSCA contribute to more 
sustainable and greener research’ reveals that the extent to which 
sustainable research practices can be implemented varies greatly 
depending on the scope of the research project and the field of 
research. For example, projects in the social sciences 
demonstrate minimal sustainability concerns due to their inherently 
low-impact nature. In contrast, resource-intensive projects in other 
fields present a higher risk of environmental or social harm. 

898 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

Therefore, interviewees were particularly concerned that 
researchers may inadvertently be encouraged to exaggerate the 
sustainability practices envisaged and the potential environmental 
impact of their projects. The case study also reveals other 
prominent barriers:   

• 

• 

• 

Inherent variations between MSCA actions: interviewees 
noted that the impact of specific sustainable practices is 
heavily influenced by the MSCA action concerned. For 
instance, while MSCA Postdoctoral Fellowships, which 
typically last three years, focus on post-fellowship outcomes, 
MSCA COFUND and Doctoral Networks, due to their broader 
scope and longer-term commitment, tend to create more 
enduring changes within institutions. While these factors 
touch on more than just the influence of the MSCA Green 
Charter, interviewees specifically highlighted their relevance 
to the effectiveness of the Charter.  
Sustainable travel: a prevalent challenge faced by many 
participating organisations and researchers is managing 
sustainable travel. While the MSCA encourage collaboration 
with international partners, the programme does not provide 
sufficient funding for sustainable international travel. A 
significant challenge is the notable difference in cost between 
eco-friendly forms of transport such as trains and more 
polluting forms such as air travel. This cost difference often 
makes it hard for researchers to choose the greener travel 
options. 
Difficulties with carbon offsetting: options for carbon 
offsetting are not always available locally. Depending on 
location and availability, researchers may struggle to 
effectively offset their carbon footprint. 

899 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

•  Materials and protocols issues: the nature of certain 
scientific research sometimes require materials and 
procedures that do not align with sustainability. Researchers 
and organisations find it difficult to look for green alternatives 
for essential but toxic reagents. Many research fields, such 
as molecular biology, rely heavily on single-use plastics, with 
finding contamination-resistant alternatives being a 
challenging task. 
Administrative constraints: policies on purchasing goods 
and resources largely depend on an institution’s governing 
bodies; therefore, financial and logistical limitations usually 
constrain researchers from opting for more eco-friendly 
options, thus limiting sustainable choices for researchers 
even if they are willing. 

• 

The case study also reveals the following notable drivers:  

• 

• 

Training: training courses and their associated materials are 
useful to institutions, consortia and researchers who are 
already heading in a sustainable direction (e.g. training is 
useful to help operationalise sustainable research practices 
when researchers have already displayed an interest in doing 
so).  
Existing suggestions: the detailed discipline-specific 
measures that are currently recommended in the Green 
Charter are comprehensive, as indicated in the interview 
programme. For example, the Guidance Materials suggest 
that organisations which require practical tools to manage 
environmental responsibilities should obtain the International 
Organization for Standardization (ISO) 14000 family 
Environmental Management, an internationally recognised 
benchmark. 

900 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

• 

Balance between the structure of the Green Charter and 
external position: the synthesis between the current 
structure in the Green Charter, and its external position vis-à-
vis the MSCA’s structure (as a non-binding code of good 
practice balancing specific practices and general themes) 

INFRA: The main barriers indicated in the survey by INFRA 
beneficiaries are as follows: the COVID-19 pandemic (22.8% of 
respondents, or 36 out of 158, indicated that this barrier presented 
challenges to a ‘large’ or ‘very large’ extent); the insufficient 
amount of funding received (17.2%, or 27 out of 157); the 
excessive research, teaching, managerial or business workloads 
of researchers from the participating organisations (17.2%, or 27 
out of 157); lack of administrative support within the respondent’s 
organisation (11.4%, or 18 out of 158); an the regulations and 
rules laid down by the Horizon Europe grant agreement (10.1%, or 
16 out of 157) (INFRA-BEN-034).  

Moreover, in interviews, beneficiaries indicated that at least some 
of the actions under the INFRA work programme would be more 
efficient and effective if the call texts allowed the participation of 
smaller consortia. The large size of consortia for some grants 
leads to complex management and high coordination costs in 
addition to diluting the funding between a greater number of 
partners. This negatively impacts the effectiveness of such grants. 
Moreover, interviewees fear that the sustainability of the services 
and tools created by projects (especially challenge-oriented 
projects) is not guaranteed beyond the lifetime of the project.  

WIDERA: 

Widening: Several factors were identified as being influential to 
the progress of the Widening programme part towards its impact: 

901 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

•  Good project management and support within widening 

institutions is considered important in the case Widening 
projects that are perceived as complex – for instance, 
because of the need to secure complementary funding, or 
due to the large and diverse consortia required.  
The interest, support and involvement of national 
governments in Widening projects, as well as that of 
industry and SMEs, also came up as an important factor, 
especially in ensuring the impact and sustainability of 
projects, but also in identifying consortium partners so that 
applications can be submitted.  
Similarly, the interest of non-widening countries in 
participating is critical.  
The amount of research costs eligible in the project is 
important in creating scientific impact. 
For Teaming, the fact that the period of funding has been 
extended under Horizon Europe compared with Horizon 
2020 is appreciated, and is said to contribute to more and 
better outcomes.  
International collaboration and networking with advanced 
international organisations are highlighted as being highly 
important and beneficial.  

• 

• 

• 

• 

• 

In the survey of successful applicants to Horizon Europe WIDERA 
(Widening), respondents stressed that the most important barriers 
to carrying out the project are as follows (WIDERA-WIDE-BEN-
034):  

1)  The excessive research, teaching, managerial or 

business workloads of researchers from the participating 
organisations; 

2)  The COVID-19 pandemic; 

902 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

3)  Lack of administrative support within the organisation. 

ERA: According to evidence from the case study and survey, the 
successful achievement of project outputs and results (and their 
structural impacts on ERA, from a longer-term perspective) will 
depend on a complex set of success factors and the ability to 
overcome barriers: 

• 

• 

The active dissemination of information and awareness 
among the local research community and the wider public 
regarding the project’s activities is one of the key success 
factors, and is particularly important in the case of ERA 
project activities that are directly linked to activities at 
national/local level – such as in the case of the EUCYS (EU 
Contest for Young Scientists), in which the EU-level contest 
gathers together the winners of national-level young scientist 
contests. Awareness among young people of these contests 
at national level varies significantly from country to country; 
whereas authorities in some countries put great efforts into 
raising awareness of these events, and attract a lot of interest 
from young people, in other countries these events are not 
well known among the youth.  
The success of projects depends on ensuring effective 
communication and coordination between different project 
partners and their varying needs, which was identified by 
multiple stakeholders as one of the key potential challenges – 
particularly in the case of projects that focus on university 
transformation. This critical success factor is also confirmed 
by evidence from the survey, in which the excessive 
research, teaching, managerial or business workloads of 
researchers from the participating organisations was 
identified as a key challenge/potential barrier (39%, or 39 out 

903 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

• 

• 

• 

100 beneficiary respondents agreed to a ‘very large’, ‘large’ 
or ‘moderate’ extent that this constitutes a challenge when 
carrying out their WIDERA project). 
According to a number of stakeholders interviewed, the 
success of a project largely depends on engaging policy 
stakeholders and top-level management and ensuring their 
commitment to institutional transformation.  
In the case of projects in the field of university 
transformation/acceleration services, one of the key success 
factors identified is the ability of project coordinators and 
participants to ensure the active involvement of wider groups 
of stakeholders in the project’s activities (e.g. data gathering, 
stakeholder consultation process). According to the case 
study evidence, the successful development of a roadmap for 
the future of research management (RM) in Europe will 
largely depend on the effective mobilisation and involvement 
of the 120 national ambassadors, to ensure their active 
participation in co-creation and consultation activities, 
communication with local research communities, and data 
gathering efforts. 
The prior experience of project coordinators and their ability 
to draw on previous good practice examples was also found 
to be a key success factor. For example, in the case of the 
2022 European Capital of Science, one of the project’s key 
success factors was the prior experience of coordinators with 
the European Capitals of Culture initiative and their ability to 
use and amplify this in the field of citizen science. More 
specifically, this experience contributed to project 
coordinators applying a new and innovative approach to 
citizen science: instead of a top-down ‘science 
communication’ approach, the organisers applied grassroots, 

904 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

EFC2.2 

Analysis of factors that have 
influenced the progress (enablers, 
drivers) or lack of progress (barriers) of 
FP interventions towards their impact? 
Identification of solutions to barriers, or 
further support for drivers. 

Prioritisation of solutions to barriers or 
further support for drivers. 

Literature 
review 

Interviews 

Case studies 

Benchmarks 

Surveys 

What could be 
done to address 
these in the short 
and longer term? 
Are there any 
factors that are 
more or less 
effective than 
others, and, if so 
what lessons can 
be drawn from 
this? 

905 

bottom-up approach in which individuals from local 
communities took the initiative to promote science. 

ERC: The main recommendation made by interviewees and 
stakeholders is to increase the ERC’s funding capacity in order to 
fund more projects and researchers. 

MSCA: From the perspective of successful applicant organisations 
participating in the Doctoral Networks action, the interview findings 
point to the need to better align the MSCA programme with 
national doctoral programmes in many EU Member States. Under 
the current situation, participating organisations are compelled to 
seek additional funding to cover the fourth year of their PhD, in 
contrast to national/regional funding programmes in which the 
funding is usually provided for the duration of 48 months. 
Stakeholders also called for better clarification of the rules guiding 
the Joint Doctorates (DN-JD), due to the vast array of national 
practices in doctoral education across Europe and beyond. 

The interview findings also suggest a clear need for a revision of 
the unit contributions of the MSCA, since the variations in funding 
coefficients between countries increases complexity. According to 
interviewees, in some cases, these coefficients may not perfectly 
reflect the day-to-day realities or even regional differences within 
countries. Currently, the MSCA is undergoing a review of its Lump 
Sum and unit contributions, and should therefore address the 
concerns expressed.  

INFRA: Interviewees pointed to the need for continuity beyond 
individual projects and also the need for longer-term roadmaps. 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

Moreover, smaller consortia would foster the efficiency and 
effectiveness of projects.  

WIDERA: 

Widening: Interviewees and stakeholders provided several 
recommendations, namely: including more Research and 
Innovation Actions (RIAs) in the Widening actions; integrating (part 
of) the Widening budget into other programme parts (e.g. by 
funding proposals submitted by researchers from widening 
countries who are placed on the reserve list); if necessary, 
encouraging participation in Widening actions by industry and 
governments – particularly those from non-widening countries;  

ERA: As evidenced by the case study and survey data on success 
factors in ERA projects, the key recommendation for the ERA 
Widening part is to increase the awareness of the initiative on the 
part of potential ERA project beneficiaries. The target groups for 
these kinds of awareness-raising initiatives could include both 
policy-makers and wider society/young people who are potential 
beneficiaries of ERA projects in the field of citizen science. 

EFC3 

Number of dissemination and outreach 
activities, such as conferences, 
workshops, press releases, 
publications, flyers, exhibitions, 
trainings, social media, websites, 
communication campaigns, etc. 

To what extent 
dissemination, 
exploitation and 
communication 
measures have 
enabled to reach 
the targeted 
outcomes and 
impacts? What 

Literature 
review 

Monitoring 
data 

Interviews 

Case studies 

Horizon Europe: Several tools have been introduced under 
Horizon Europe to improve the communication, dissemination and 
exploitation of results stemming from the research and activities 
funded by the programme: Horizon Results Booster, Horizon 
Results Platform and Horizon Results TV. 

Horizon Results Booster (HRB) is an initiative of the European 
Commission that aims to strengthen the stream of innovations to 
market, and to maximise the impact of EC-funded research. HRB 
offers free consultancy services to ongoing or completed research 

906 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

further actions are 
needed to 
maximise the 
impact of the 
Framework 
Programme 
interventions in this 
area? 

Amount of funding dedicated to 
dissemination and communication 
activities. 

Identification of needs to further 
maximise the impact of dissemination 
and valorisation. 

Methods and 
analyses 

Surveys 

Key findings 

Evidence 

projects funded under FP7, Horizon 2020 or Horizon Europe that 
wish to go beyond their Dissemination and Exploitation (D&E) 
obligations. The timeframe for the implementation of HRB is from 
July 2020 to June 20241316.  

The HRB includes three main avenues of support: 

1)  The Portfolio Dissemination & Exploitation Strategy 
package includes three services: modules A and B 
focus on strengthening beneficiaries’ capacity to 
disseminate research results, while Module C aims to 
foster a project’s exploitation potential.  

2)  Business Plan Development helps beneficiaries to 

develop a business strategy.  

3)  Go To Market helps beneficiaries to prepare their 

product for commercialisation1317. 

By March 2023, the HRB had provided 600 services, involving 
more than 2,789 beneficiaries, 209 experts and 7,363 
organisations: 57% of these services belonged to the first category 
(the Portfolio Dissemination and Exploitation Strategy service); 
12% related to Business Plan Development; and 10% involved Go 
To Market services. In addition, the HRB helped to develop nine 
EU Dissemination and Exploitation reports providing analyses of 
the D&I clusters. More than half of entities benefitting from HRB 
services of are private organisations, showing that there is a high 
degree of interest from the private sector. The other half of 

1316 https://www.horizonresultsbooster.eu/about 
1317 https://www.horizonresultsbooster.eu/about 

907 

 
 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

beneficiaries comprise a mix of public institutions, universities and 
non-profit organisations.1318.  

The Horizon Results Platform1319 is an EC initiative that aims to 
make information about EU-funded research results and their 
creators more widely accessible. It categorises results according 
to the EU Missions they contribute to: 

- 
- 
- 
- 
- 

Adaptation to climate change (128 results) 
Cancer (27 results) 
Climate-neutral and smart cities (139 results) 
Oceans, seas and waters (94 results) 
Soil health and food (41 results) 

In addition, users of the platform can also search for results 
relating to policy (470 results), and to contribution to specific UN 
SDGs (2,262 results). They can also find information on 
beneficiaries looking for funding, loans or investment (772 results), 
or for help on their way to market (734). It is also possible to find 
organisations looking for technical or infrastructure help or 
fellowship (542 results), or for collaboration (994). It should be 
noted that the platform currently only provides results for one 
Horizon Europe project; all other results come from Horizon 2020 
projects.  

1318 https://www.meta-group.com/news/Pages/Horizon-Results-Booster-hits-400-completed-services-milestone.aspx 
1319 https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/horizon-results-platform. Figures in these section correspond to 
those published in the Horizon Results Platform in June 2023. 

908 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

The Horizon Results Platform TV1320 (HRP TV) is an EC online 
learning tool that aims to support and inspire FP beneficiaries in 
their efforts to bring their solutions to market. 

The Horizon Impact Awards was an EC initiative launched in 
2019 that aims to show the wider socio-economic benefits of EU 
investments in R&I that have been carried out in projects from FP7 
or Horizon 2020 that have already closed. Applications were 
assessed by a jury on the basis of the following cumulative criteria: 
the impact and societal benefits achieved, and the pathways that 
lead to societal benefits (i.e. describing in detail the scale and 
sustainability of their actions, including those that emerge after the 
lifetime of the project). A total of six prizes of EUR 25,000 each 
were envisaged. Two rounds of the awards were carried out (in 
20191321 and 20201322), but the Commission decided to cancel the 
2022 contest for the Horizon Impact Awards1323. 

ERC: the ERC has a mandate to communicate to the scientific 
community, key stakeholders, the media and the public at large. 
According to the External Communication Strategy 2023, the 
ERC’s communication tools and channels are its website, 
multimedia (video productions, infographics and podcasts), social 
media, its online magazine, email alerts and publications (with a 
focus on reader-friendly online formats). In addition, the ERC is 
frequently mentioned in the news, and plans to explore 

1320 https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/horizon-results-platform/hrptv 
1321 More information about the winners of the 2019 award is available here: https://research-and-innovation.ec.europa.eu/news/all-research-and-
innovation-news/commission-announces-winners-horizon-impact-award-2019-2019-09-26_en  
1322 More information about the winners of the 2020 award is available here: https://research-and-innovation.ec.europa.eu/news/all-research-and-
innovation-news/commission-announces-winners-horizon-impact-award-2020-2020-09-23_en  
1323 https://research-and-innovation.ec.europa.eu/funding/funding-opportunities/prizes/horizon-impact-award_en  

909 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

collaborations with, for example, science television shows. Lastly, 
the NCPs, host institutions, representations and the European 
Parliament are also channels of communication. The ERC’s 
annual communication and dissemination indicators and activities 
are presented in its Annual Activity Reports. Figures show that the 
ERC’s media mentions have more than doubled compared since 
2014, and the number of subscribers to the ERC newsletter and 
news updates is 2.5 times higher than at the beginning of Horizon 
2020, while unique visitors to its website have increased from 
500,000 in 2014 to around a million in 2022. Social media 
followers have also increased significantly, especially users of 
Twitter (now known as X) and LinkedIn.  

Under its Work Programme 2023, the ERC allocated EUR 1.5 
million to the ERC Science Journalism Initiative, which aims to 
support an organisation or a consortium of organisations to set up 
a funding scheme that would facilitate stays at research 
institutions of between three and five months for journalists in 
order to give them access (for example, via ERC grantees) to a 
better understanding of frontier research, while respecting 
journalistic independence.  

MSCA: The MSCA and Citizens action aims to bring research and 
researchers closer to the public at large, to increase awareness of 
R&I activities, and to boost public perceptions and recognition of 
science and research education. These objectives are pursued 
mainly through the organisation of the European Researchers’ 
Night, the largest communication and promotion event taking 
place across EU Member States and Horizon Europe associated 
countries. The indicative budget for the 2021 MSCA and Citizens 
call was EUR 15.42 million, with EUR 15,537,841 already 
disbursed.  

910 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

INFRA: The current dissemination, exploitation and 
communication efforts of INFRA appear to be only of limited 
importance to INFRA beneficiaries. Out of the INFRA beneficiaries 
that responded to the survey, only 28.7% (or 42 out of 146) found 
that EC platforms and measures aimed at facilitating exploitation, 
such as the Horizon Results Booster, Horizon Results Platform 
and IPR Helpdesk were important for the uptake of their research 
findings and/or innovation to a ‘large’ or ‘very large’ extent (INFRA-
BEN-041). Meanwhile, 36.90% (55 out of 149) found the EC’s 
dissemination activities and platforms, such as the Horizon 
Dashboard and CORDIS were important to a ‘large’ or ‘very large’ 
extent (INFRA-BEN-041), and 30.40% of INFRA beneficiaries (45 
out of 146) found that EC communication activities to stakeholder 
groups were important to a ‘large’ or ‘very large’ extent (INFRA-
BEN-041). Lastly, 29.7% of INFRA beneficiaries (44 out of 148) 
found EC activities aimed at communicating research to citizens 
was important to a ‘large’ or ‘very large’ extent (INFRA-BEN-041). 

Some interviewees pointed to the need of more communication 
and synchronisation between different INFRA projects, especially 
in the context of the EOSC. It is difficult for project coordinators to 
keep track of current and past activities and results, leading to a 
duplication of efforts and unexploited synergies. Projects such as 
the ERIC Forum1324, which brings together all ERICs to discuss 
common challenges and solutions, are found to be very beneficial. 

WIDERA:  

Widening: 44.6% of Widening beneficiary respondents to the 
survey of indicated that they had been producing or would produce 
recommendations for policy-makers and provide inputs into 

1324 https://cordis.europa.eu/project/id/823798 

911 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

national or EU policies (WIDERA-WIDE-BEN-027, Survey of 
Horizon Europe beneficiaries, conducted May-July 2023). 
Furthermore, as mentioned in EFF.1.1, 89.3% of these 
respondents (158 out of 177) indicated their project had led or 
would lead to research publications; 71.2% (126 out of 177) to 
new training programmes, and 65.5% (116 out of 177) to 
educational materials (WIDERA-WIDE-BEN-027). 

More than half of the survey respondents foresaw that their project 
results would be used for academic purposes (58.3%, or 98 out of 
168); around a quarter (45 out of 168) foresaw the creation and 
provision of a service, and another quarter (40 out of 168) foresaw 
the development, manufacturing or marketing of a product or 
process as an exploitation outcome. In all, 16.7% (28 out of 168) 
indicated that they did not foresee any exploitation activities as 
part of their WIDERA Widening project (WIDERA-WIDE-BEN-
039).  

Fewer than half of respondents to the survey indicated that EC 
platforms and communication activities are important or very 
important in facilitating the uptake of research findings and 
innovations. In particular, the EC’s activities communicating about 
research to stakeholder groups were found to be important or very 
important by the smallest share of respondents (37.7%, or 61 out 
of 162 respondents), followed by EC platforms aimed at facilitating 
exploitation (e.g. Horizon Results Booster, Horizon Results 
Platform, IPR Helpdesk), as indicated by 38.0% of respondents 
(62 out of 163) (WIDERA-WIDE-BEN-041).  

In addition, the call Dissemination and Exploitation Support Facility 
has been developed. This call is open to Widening participants 
from Horizon 2020 and Horizon Europe, and is aimed at scaling up 
research results and improving the sustainability of actions (e.g. 

912 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

translating innovations into new products, the feeding of results 
into policy-making). 

ERA: A significant share of respondents indicated that their project 
had resulted in (or was likely to result in) research roadmaps 
(39%, or 41 out of 105); social innovations such tools, guides and 
strategies (35%, or 37 out of 105); new or improved tools, 
methods or techniques (31%, or 33 out of 105); and codes of 
conduct (17 out of 105). Moreover, 61% of WIDERA (ERA) 
beneficiaries surveyed (61 out of 99) agreed to a ‘very large’ or 
‘large’ extent that their projects had achieved/were likely to 
achieve the development of policy-making and standards-setting 
measures. 

EFC4 

To what extent has 
the Framework 
Programme in this 
area contributed to 
achieving the 
European Union 
policy priorities and 
the Sustainable 
Development 
Goals (SDGs)? 

The extent to which the projects 
analysed contribute to the 'Open 
Innovation, Open Science, Open to the 
World' policy agenda, ERA priorities, 
and/or the twin green and digital 
transitions. 

The extent to which FP funding 
contributed to the Sustainable 
Development Goals: 

Number/share of projects addressing 
different SDGs, by programme 
part/year/etc. 

Number/share of research outputs 
addressing SDGs. 

Literature 
review 

Monitoring 
data 

Interviews 

Case studies 

Surveys 

Bibliometric 
analysis 
(Horizon 
2020) 

ERC: According to the Horizon Dashboard (accessed 13 June 
2023), topics dealt with most often by ERC signed grants are 
proteins, oncology, climate change, history, machine learning, 
DNA, RNA, neurobiology and ecosystems. In addition, an analysis 
carried out on ERC projects under Horizon 2020 shows that 
significant contributions were made to the European Green Deal 
and Europe fit for the digital age in the fields of energy and climate 
action, and quantum technologies and ICT.  

According to the SDG analysis performed for this study, the SDGs 
to which ERC projects contribute the most under Horizon Europe 
(in terms of EU contribution) are SDG 3 (Good health and well-
being, 41.2% or EUR 1,692.9 million); SDG 9 (Industry and 
innovation, 12.6% or EUR 516.7 million); and SDG 7 (Affordable 
and clean energy, 10.6% or EUR 437.4 million). These SDGs 
reflect the growing importance of certain challenges in recent 
years with respect to health (the COVID-19 pandemic, but also the 

913 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Methods and 
analyses 

Analysis of 
SDG data 

Key findings 

Evidence 

importance of, for instance, cancer research) and clean energy 
(related to climate change and the energy crisis). Compared with 
Horizon 2020 (in terms of total number of projects), SDG 7 
appears to be more widely covered in the ERC under Horizon 
Europe. However, the documents o which the SDG analysis is 
based differ between Horizon 2020 and Horizon Europe: the 
Horizon Europe analysis was carried out using the proposal texts, 
while under Horizon 2020 the analysis was based on publications. 
This makes it difficult to compare the contributions made to the 
SDGs by each of the two Framework Programmes. For a more in-
depth analysis, please refer to Annex 6. 

MSCA: In line with the European Green Deal and the provisions 
guiding the 9th Framework Programme for Research and 
Innovation, the Marie Skłodowska-Curie Actions (MSCA) 
introduced a bespoke Green Charter that lays down a set of 
general principles and objectives promoting the sustainable 
implementation of research activities.  

The Charter applies to all MSCA-funded projects and participants, 
who are expected to commit to its principles on a ‘best effort’ 
basis; therefore, the Charter does not impact the bottom-up nature 
or the scientific excellence of the MSCA, but rather encourages 
sustainable thinking in research management. Some of the 
measures suggested by the Charter include reducing waste, 
promoting green purchasing, ensuring events are sustainable, 
using low-emission transport (and teleconferencing when 
possible), using renewable energy, raising awareness of 
sustainability, and sharing best practices. 

The case study on how the MSCA contribute to more sustainable 
and greener research found a dichotomy in awareness among 
MSCA respondents: while the European Green Deal is widely 

914 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

recognised, the MSCA community was found to be less aware of 
the MSCA Green Charter and its guidelines. Notably, despite this 
gap, many institutions involved in MSCA actions have proactively 
embraced strategies to foster sustainable research activities. 
Furthermore, a significant majority (72%) of respondents 
expressed their willingness to align with the environmental 
considerations outlined in the Green Charter across all action 
types, indicating a strong inclination toward sustainable practices 
within the MSCA research community (For more information on 
the conditions, best practices and barriers that contribute to the 
implementation of green/sustainable research activities, please 
refer to the Case Study 4 on How MSCA contribute to more 
sustainable and greener research). 

Furthermore, the case study assessed the MSCA’s contribution to 
the United Nations’ Sustainable Development Goals (SDGs). The 
SDGs most addressed by MSCA-funded projects are SDG 3 – 
‘Good health and well-being (48%), SDG 4 – ‘Quality education’ 
(30%), and SDG 16 – ‘Peace, justice and strong institutions’ 
(24%). The SDGs covered least are SDG 1 – ‘Affordable and 
clean energy’ (3%) and SDG 2 – ‘Decent work and economic 
growth’ (3%). The SDGs covered the most also received the 
highest shares of EU contribution. 

INFRA: One of the actions included in the ERA Policy Agenda that 
was approved in 20221325 was directed towards Ris, namely: 
‘Strengthen sustainability, accessibility and resilience of research 
infrastructures in the ERA’. INFRA thus plays an important role 

1325 European Commission, Directorate-General for Research and Innovation, European Research Area policy agenda: overview of actions for the 
period 2022-2024, Luxembourg: Publications Office of the European Union, 2022, available at: https://data.europa.eu/doi/10.2777/52110 

915 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

with regard to the ERA, as it provides funding for the development 
of RIs and the accessibility of RIs to researchers across Europe.  

According to the SDG analysis performed for this study (see 
Annex 6), the SDGs to which INFRA contributes most (in terms of 
EU contribution) are: SDG 3 – ’Good health and well-being’ 
(17.30%); SDG 16 – ‘Peace, justice and strong institutions’ 
(13.6%); and SDG 9 – ‘Industry, innovation and infrastructure’ 
(11.80%). Compared with Horizon 2020, it appears that a higher 
share of projects contributes to SDG 16, while a lower share of 
projects contribute to SDG 9, as well as SDG 4 – ‘Quality 
education’, SDG 11 – ‘Sustainable cities and communities’. 
However, the documents on which the SDG analysis is based 
differ between Horizon 2020 and Horizon Europe: the Horizon 
Europe analysis was carried out using the proposal texts, while in 
Horizon 2020 the analysis was based on publications. This makes 
it difficult to compare the contribution made to SDGs by each of 
the two Framework Programmes.  

WIDERA: The SDG analysis shows that under WIDERA, the 
SDGs most frequently covered by Horizon Europe projects so far 
are: SDG 16 – ‘Peace, justice and strong institutions’ (42.3%, 138 
out of 326 projects); SDG 4 – ‘Quality education’ (39.9%, 130 out 
of 326 projects); and SDG 3 – ‘Good health and well-being’ 
(38.3%, 125 out of 326 projects). 

Widening: According to the Horizon Dashboard (accessed in June 
2023), the topics most widely dealt with by WIDERA (Widening) 
signed grants are ecosystems, climate change, educational 
sciences, innovation management, oncology, pollution, and the 
sustainable economy. Just over half of respondents to the survey 
(52.8%, or 95 out of 180) stressed that their WIDERA Widening 
project developed sustainable solutions that contribute to the 

916 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

green transition (WIDERA-WIDE-BEN-025). In addition, only 8.7% 
(15 out of 172) of respondents indicated that no green policy 
measures (such as prevention to minimize the production of 
waste, harmful substances; were being applied or would be 
applied in the implementation of their WIDERA Widening project 
(WIDERA-WIDE-BEN-028).  

ERA: According to the Horizon Dashboard (assessed September 
2023), the topics most widely dealt with in WIDERA (ERA) signed 
grants include EuroSciVoc concept-related keywords including 
ecosystems, gender inequality, pandemics, agricultural sciences, 
big data, business and management, business models, computer 
and information sciences, coronaviruses, crisis management, and 
democracy. 

ERC: As discussed above in the section on relevance, the ERC is 
attractive to researchers from both associated countries and non-
associated third countries. According to the Horizon Dashboard 
(accessed 31 August 2023), non-Member States have so far 
accounted for 13% of all ERC participations under Horizon Europe 
(347 out of 2,727) and received 12% (EUR 471.5 million) out of a 
total EU net contribution of EUR 3.9 billion. These figures are 1 
percentage point lower than those observed under Horizon 2020, 
which may be explained by the difficulties connected with the 
association of Switzerland to Horizon Europe, which was the 
second largest participant under Horizon 2020, as well as with the 
fact that until very recently, the UK was not yet an associated 
country. In total, 44 EU and non-EU countries have been hosting 
ERC projects in Horizon Europe, which compares with a total of 65 
countries under Horizon 2020. Up to the time of writing, the non-

EFC5* 

Percentage of third countries in the 
projects analysed (by programme, 
country, etc.) 

Impact on coherence with AC systems 
in terms of performance (output 
indicators) and convergence of values 
and principles (e.g. gender, modern 
HR practices, etc.); examples and 
narrative. 

Literature 
review 

Monitoring 
data 

Interviews 

Case studies 

How effective is the 
association of third 
countries to the 
Framework 
Programme in 
reinforcing the 
integration and 
coherence of the 
European R&I 
system with those 
of the associated 
countries, in terms 
of performance and 
of convergence of 

917 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

values and 
principles? 

Member States with the highest number of participations in the 
ERC programme part have been Israel (187 participations), 
Norway (68 participations) and the United Kingdom (31 
participations).  

MSCA: As discussed previously, the MSCA programme largely 
consists of participations from EU Member States (77%), while 
third country participations account for 20%. associated countries 
account for just 4% of participations. When broken down by action 
type, Staff Exchanges (SE) managed to attract the highest share 
of participations from third countries across all action types in the 
MSCA programme (35%). So far, the net EU contribution to 
associated countries (EUR 71.2 million) accounts for 4.9% of the 
total net EU contribution to the MSCA, while the net EU 
contribution to third countries under the MSCA (EUR 17.7 million) 
accounts for 1.2%. However, the survey indicates that more 
funding is needed to enhance collaborations with international 
partners (non-EU countries): only 40.0% of MSCA beneficiary 
organisations (195 out of 488) agreed to a ‘large’ or ‘very large’ 
extent that there is sufficient funding for international collaboration. 

INFRA: As discussed previously in the section on relevance, 
associated countries represent 6% of participations in INFRA so 
far under Horizon Europe, compared with 9% under Horizon 2020. 
Meanwhile, third countries account for 13% of participations in 
INFRA under Horizon Europe, compared with only 3% under 
Horizon 2020 (Horizon Dashboard, accessed in August 2023). 
However, it is important to note that 85% of all third country 
participations are by the UK and Switzerland, which were a 
Member State and an associated country, respectively, under 
Horizon 2020. Other third countries with a notable number of 
participations are the United States (eight, or 0.4% of all Horizon 
Europe INFRA participations) and South Africa (five, or 0.3%). 

918 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

Among the associated countries, those with the highest numbers 
of participations are Norway (67, or 3.8%), Israel (eight, or 0.7%), 
Serbia (seven, or 0.4%) and Turkey (six, or 0.4%). The biggest 
change compared with Horizon 2020 is that the Russian 
Federation no longer participates in the Framework Programme; 
previously, it had been the top participating third country under 
Horizon 2020 (45, or 0.6% of all Horizon 2020 INFRA 
participations).  

WIDERA: 

Widening: According to the Horizon Dashboard (accessed in June 
2023), the Member States have so far already received 93% (EUR 
590 million) of the net EU contribution, while non-Member States 
have received 7% (EUR 41 million). The non-associated third 
countries with the greatest numbers of participations in the 
Widening actions of WIDERA so far are the UK (33 participations), 
Switzerland (four participations), Angola and Brazil (two 
participations each). The grants signed have mainly been in the 
field of natural sciences.  

See also RV6.1 and RV6.2. 

ERA: According to the Horizon Dashboard, the Member States 
have so far already received over around 93% (over EUR 108 
million) out of the total EU net contribution of EUR 116 million, 
while non-Member States have received around 7% (roughly EUR 
8 million). The non-associated third countries with most 
participations were the UK (34 participations), Switzerland (five 
participations), Kenya (three participations) and the US (three 
participations). 

919 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

To what extent has 
international 
cooperation and, 
more specifically, 
association of third 
countries to the EU 
Framework 
Programme made 
a difference in 
achieving the 
scientific objectives 
of the Framework 
Programme?  

EFC6.1* 

Number/percentage/budget share of 
topics mentioning international 
cooperation in WPs. 

Percentage of third country and 
associated country participants in 
Horizon 2020/Horizon Europe projects 
analysed. 

Percentage of EU financial 
contribution allocated to third 
country/associated country 
participants.  

Output indicators of scientific 
performance based on international 
cooperation; qualitative assessment 
and examples of positive impact on EU 
scientific performance. 

Methods and 
analyses 

Literature 
review 

Monitoring 
data 

Interviews 

Case studies 

Key findings 

Evidence 

ERC: See the answers to EFC5 and RV6.  

MSCA: See the answer to EFC5. 

INFRA: (see also the answer to question RV6) Interviewees and 
the majority of the INFRA beneficiaries surveyed indicated that 
international cooperation is important to very important. Moreover, 
only a quarter of INFRA beneficiaries agreed to a ‘large’ or ‘very 
large’ extent that there is enough funding for international 
cooperation, while a similar share agreed to a ‘large’ or ‘very large’ 
extent that there is enough funding for the development of 
international standards. Certain topics in INFRA specifically target 
international cooperation, e.g. on climate change in Africa (INFRA-
2021-DEV-01-02) or with Latin America (INFRA-2023-DEV-01-06). 
Interviews confirm the relevance of this type of call, but also 
indicate that there are concerns about the sustainability of such 
cooperation after the end of projects. Some interviewees indicated 
the need for a long-term strategy for this type of collaboration, with 
a focus on specific geographical and topical areas. Moreover, the 
European Commission sees ERICs as the instrument of choice for 
international collaboration through third country membership, and 
indicates that this needs to be further facilitated1326.  

WIDERA: 

Widening: (see answers to questions RV6, EFC5 and EFC.6.2). 
According to the Horizon Dashboard (accessed in June 2023), the 
Member States have so far already received 93% (EUR 590 

1326 Report from the Commission to the Council and the European Parliament, Third report on the application of Council Regulation (EC) No 723/2009 
of 25 June 2009 on the Community legal framework for a European Research Infrastructure Consortium (ERIC), 2023, available: https://eur-
lex.europa.eu/legal-content/EN/TXT/?uri=COM:2023:488:FIN. 

920 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

million) of the total EU net contribution, while the non-Member 
States have received 7% (EUR 41 million). In terms of 
participations, the Member States account for 82% of all 
participations, while non-Member States made up 18%.  

ERA: According to the Horizon Dashboard, the Member States 
have so far already received around 93% (over EUR 108 million) 
out of the total net EU contribution of EUR 116 million, while the 
non-Member States have received around 7% (around EUR 8 
million). In terms of participations, Member States have 83% of the 
total participation, while non-Member States have 17%. 

Has international 
cooperation, and 
specifically 
association, 
increased the EU 
scientific 
performance? 

EFC6.2* 

Number/percentage/budget share of 
topics mentioning international 
cooperation. 

Percentage of third country and 
associated Participants in the Horizon 
2020/Horizon Europe projects 
analysed. 

Percentage of EU financial 
contribution allocated to third 
country/associated country 
participants.  

Output indicators of scientific 
performance based on international 
cooperation; qualitative assessment 
and examples of positive impact on EU 
scientific performance. 

Literature 
review 

Monitoring 
data 

Survey  

Interviews 

ERC: (see answers to EFC5 and 6.1, and RV6). In addition to the 
information already added in the previous evaluation questions, 
the survey shows that 33.5% (80 out of 239) of successful ERC 
applicants agreed to a ‘large’ or ‘very large’ extent that Horizon 
Europe provides sufficient funding opportunities for research in 
collaboration with international partners (ERC-BEN-024). 

MSCA: See the answer to EFC5. 

Case studies 

INFRA: See the answers to EFC6.1, EFC5 and RV6.  

WIDERA: 

Widening: (see answers to questions RV6, EFC5 and EFC.6.1.) 
The Evaluation study on Excellent Science under Horizon 2020 
points to an increased capacity to develop networks both locally 
and internationally as a result of Widening projects. More 
specifically, being part of an international consortium had a 

921 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

positive impact on the maturity and quality of research groups and 
of these institutions being considered reliable research partners.  

Moreover, more than half of survey respondents with a WIDERA 
Widening project (56.6%, or 94 out of 166 respondents) indicated 
that collaboration with international (non-EU) partners is important 
to a ‘large’ or ‘very’ large extent in facilitating the uptake of 
research findings and innovations (WIDERA-WIDE-BEN-041).  

ERA: Case study findings indicate that taking part in international 
consortia that include partners from different countries is one of 
the key sources of knowledge expected to result in policy 
standards-setting measures (e.g. recommendations, methods, 
tools, roadmaps, educational materials, training programmes, etc.) 
that will empower beneficiary institutions (as well as institutions 
outside the projects) to undertake transformation. According to the 
results of the Horizon Europe beneficiaries’ survey, a large 
majority of WIDERA (ERA) beneficiaries (82%, or 86 out of 105) 
indicated that their project had resulted in/is likely to result in 
recommendations for policy-makers/inputs to national or EU 
policies, educational materials (63%, or 66 out of 105 
respondents), or training programmes (62%, or 65 out of 105 
respondents. 

EOSC Partnership: The EOSC Partnership has contributed to a 
large extent to Open Science. Indicator data for this will be further 
explored. 

To what extent 
have the 
partnerships 
funded under the 
Framework 
Programme 
contributed to 

EFC7 

Number of projects; the extent to 
which projects in the partnerships 
analysed have achieved the objectives 
of excellent science; examples of 
outstanding results/impacts already 
achieved. 

Literature 
review 

Interviews 

Case studies 

922 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

deliver on excellent 
science objectives? 

What has been the 
contribution of the 
Framework 
Programme as a 
whole, and of the 
core programme 
parts covered in 
this procurement, 
in particular to: 
• reinforcing and 
extending the 
excellence of the 
Union's science 
base? 

EFC8.1 

Detailed analysis of the various 
aspects, including quantitative 
assessment based on monitoring data 
and parallel studies (e.g. MSCA talent 
circulation, knowledge ecosystems in 
the new ERA study, etc.) and 
qualitative assessment of contribution 
and attribution; examples and 
narrative. 

Examples: 

• 

Scientific output indicators, 
numbers of researchers linked to 
FP participation, etc. 

Literature 
review 

Interviews 

Case studies 

Benchmarks 

Surveys 

Bibliometric 
analysis  
(Horizon 
2020) 

ERC: The ERC contributes substantially to the EU’s science base. 
This is reflected in its track record of publications, the scientific 
quality of the research produced, the many prizes and awards 
given to ERC grantees, and the ground-breaking nature of ERC 
research projects. For example, as stated in Annex 1, 14 ERC 
grantees have so far been awarded Nobel Prizes, seven of whom 
have received them in the last three years; 11 ERC grantees have 
been awarded with the Wolf prize, six of them in the last three 
years; and six ERC grantees have won a Fields Medal. Even in 
the field of SSH, two ERC grantees have won the Gold Medal of 
the Swedish Society for Anthropology and Geography; two ERC 
grantees have won the Holberg Prize; and one ERC grantee has 
won the Stein Rokkan Prize. In addition, 44.9% (106 out of 236) of 
ERC principal investigators (PIs) who responded to the survey 
indicated that they expect to have their work recognised through 
prizes or awards as a result of participating in the ERC (ERC-
BEN-032), a higher value than the 34.1% of such respondents in 
the survey of Horizon 2020 ERC beneficiaries conducted in 2022.  

In the survey of successful ERC participants, 99.2% of 
respondents (236 out of 238) indicated that their project has led or 
will lead to research publications (ERC-BEN-027). Furthermore, 
42.7% of ERC beneficiaries (103 out of 241) indicated that their 
project strengthens their opportunities to access the latest 
scientific and other knowledge and databases, and 55.8% (134 out 
of 240) said it enabled them to access good research facilities, to 
a ‘large’ or ‘very large’ extent (ERC-BEN-025). 

923 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

Two case studies have been developed by the study team to 
assess the extent to which the ERC contributes to trending 
research pathways in the fields of life sciences, physical sciences 
and engineering on the one hand, and social sciences and 
humanities (SSH) on the other. With respect to life sciences, 
physical sciences and engineering, ERC-funded research has 
contributed the most in terms of quantity of outputs in the fields of 
physics, astronomy and engineering, while in terms of impact, it 
has contributed the most in the fields of biology and clinical 
medicine. In the fields of social sciences and humanities, ERC-
funded research has contributed the most in terms of quantity of 
outputs in the fields of social sciences and historical studies, while 
in terms of impact, it has contributed the most in the fields of 
economics and business.  

MSCA: This study finds that MSCA beneficiary organisations 
agree to a ‘large’ or ‘very large’ extent that their MSCA project 
responds to their needs to strengthen the knowledge base and 
R&D capacity (76%), and to enhance R&D and/or innovation 
capabilities and skills (76%).  

The findings of the survey of successful Horizon Europe applicants 
also highlight the extent to which the expected outputs have 
been/are likely to be produced by successful MSCA PF applicants 
and successful MSCA applicant organisations through the Horizon 
Europe-funded projects. Successful applicants report a strong 
emphasis on research-related outputs, with 98% of MSCA PF 
fellows indicating research publications as a result. Looking at the 
outputs produced (or likely to be produced) by beneficiary 
organisations participating in the MSCA, they also tend to prioritise 
research publications (94%),  

924 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

Successful MSCA applicant organisations and successful MSCA 
PF applicants were further asked whether any exploitation 
activities were foreseen as a part of their project. Around half of 
respondents in each group (51% and 48%, respectively) indicated 
that they anticipated the project results being used for academic 
purposes. One-third of MSCA beneficiary organisations also 
indicated that the project results will be exploited to develop, 
create, manufacture or market a product or process.  

As part of the evaluation, the study team analysed the impact of 
MSCA funding on both novelty and disruptiveness. This analysis 
revealed only minimal differences between MSCA applicants who 
received funding and those who did not, both before (as is 
desirable) as well as after receiving the funding. Exactly 1% of the 
documents of funded applicants were novel at the time of their 
application. Non-funded applicants are very close, with 0.9% 
documents being novel. Two and five years after application, 5% 
of documents published by funded applicants were novel. For non-
funded applicants, the shares are very similar, at 5% and 4.5% 
respectively. Similarly, 08% of the documents of funded applicants 
were disruptive publications at time of application. Non-funded 
applicants are very close, with 0.7% documents being disruptive. 
Two and five years after their application, 3% of documents 
published by funded applicants were disruptive. For non-funded 
applicants, the shares are very similar, at3% and 5% respectively. 
This is unsurprising, given the programme’s competitiveness and 
its relatively low success rates – there is a wide pool of excellent 
researchers who, due to budget limitations, are not able to 
participate in the MSCA programme.  

While the MSCA do not increase the capacity of funded applicants 
to produce more novel and/or disruptive research, they do provide 
a critical support system for researchers to continue to do so. By 

925 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

providing financial support, resources and opportunities for 
professional development, the MSCA help researchers to maintain 
their research activities and potentially build on existing work. This 
can be particularly important in ensuring the continuity of projects 
that have the potential for significant impact. 

Evidence from the position papers submitted to the past, current 
and future of research and innovation programmes suggests that 
there is a gap between the R&I activities of doctoral fellows, as 
well as between academic and non-academic competences. While 
significant progress has been made in terms of multi-disciplinarity, 
for example, improvements in the area of inter- and trans-
disciplinarity would be welcomed.  

The MSCA Green Charter has contributed significantly to 
increasing awareness of sustainable research practices across 
Europe. While some participants expressed a desire for further the 
integration of cross- and interdisciplinary sustainable practices, the 
existing Green Charter was effective in standardising several 
sustainable practices. The Green Charter was also effective at 
raising awareness of sustainability issues as well as promoting 
awareness, for example, of the high carbon emissions related to 
regular intercontinental flights. 

INFRA: INFRA plays an important role in reinforcing and 
extending the excellence of the Union’s science base. It increases 
the accessibility of world-class RIs and facilities, which provide 
researchers with the necessary tools, equipment and training to 
perform their research. In addition, INFRA fosters the development 
of world-class pan-EU RIs, which increases the EU’s capacity to 
perform excellent science and enhances EU competitiveness.  

926 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

In the survey of successful INFRA participants, 83.5% of 
respondents (137 out of 164) indicated that the project has led or 
would lead to research publications (INFRA-BEN-027). 
Furthermore, 59.8% (98 out of 164) of INFRA beneficiaries 
indicated that their project strengthens the knowledge base and 
R&D capacity in their organisation to a ‘large’ or ‘very large’ extent 
(INFRA-BEN-025). Moreover, 52.9% (or 82 out of 155) agreed that 
the project would probably increase their research productivity to a 
‘large’ or ‘very large’ extent (INFRA-BEN-031).  

WIDERA: 

Widening: Historically, Widening actions have contributed to 
increasing the research excellence of the Union (e.g. under 
Horizon 2020). Widening actions have produced large numbers of 
highly cited publications in widening countries, with 28% of the 
highly cited Horizon 2020 publications from widening Member 
States being produced as a result of Widening actions.1327 In 
addition, Widening actions also contributed to the international 
positioning and reputation of researchers and research groups 
from widening countries, and thanks to the Widening actions, 
networks have been strengthened. Interviews conducted with 
Horizon 2020 Widening beneficiaries in the context of this study 
still confirm the above findings. 

Several other novelties have been introduced under Horizon 
Europe that also aim to enhance scientific capacity and research 
excellence. To give an example, through the Hop On Facilities, 

1327 European Commission, Directorate-General for Research and Innovation, Budraitis, M., Pranckevičius, P., Dėlkutė, R. et al., Evaluation study of 
the European framework programmes for research and innovation for excellent science: Horizon 2020: Annexes, Luxembourg: Publications Office of 
the European Union, 2023, available at: https://data.europa.eu/doi/10.2777/353383 

927 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

there is now the possibility for widening institutions to join 
collaborative R&I actions with the aim, among others, of improving 
research excellence in widening institutions. Also, Excellence 
Hubs should improve access to excellence for R&I actors in 
widening countries (according to the WIDERA Work Programmes 
21-22 and 23-24).  

In the survey of successful WIDERA Widening participants, 89.3% 
of respondents (158 out of 177) pointed out that the project has 
led or would lead to research publications (WIDERA-WIDE-BEN-
027). More than three-quarters (76.7%, 138 out of 180) of 
WIDERA Widening participants who responded to the survey 
agreed that their project strengthened the knowledge base and 
R&D capacity within the organisation, and more than 90% (162 out 
of 178) indicated that the project enhanced the organisation’s 
international visibility to a ‘large’ or even a ‘very large’ extent 
(WIDERA-WIDE-BEN-025). In addition, 70.8% agreed that the 
project would probably increase their research productivity 
(WIDERA-WIDE-BEN-031). A majority of unsuccessful WIDERA 
Widening applicants (62.5%, or 25 out of 40) reported that they 
expected to produce fewer research outputs and innovations 
compared with those thy had anticipated under the Horizon 
Europe project (WIDERA-WIDE-APPL-025).  

Furthermore, almost 90% (158 out of 180) of WIDERA Widening 
beneficiaries who responded to the survey said that the project 
creates or strengthens collaboration with leading research 
organisations to a ‘large’ or ‘very large’ extent (WIDERA-WIDE-
BEN-025).  

ERA: WIDERA (ERA) is expected to produce outcomes 
reinforcing and extending the excellence of the Union's science 
base, mostly by strengthening networks, enhancing cooperation 

928 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

between policy-makers, increasing exchanges of good practice 
and creating a common framework and standards for R&I in 
Europe. As previously mentioned, from a longer-term perspective 
the above outcomes of WIDERA (ERA) support should contribute 
to structuring effects at institutional level and across the ERA, 
which are ultimately likely to increase the overall excellence and 
productiveness of institutions within European research systems. 
For example, according to the case study findings, projects 
focusing on the professionalisation of research management are 
expected to enhance the efficiency of national R&I systems, 
primarily by freeing up valuable research time (by allowing 
researchers to focus first and foremost on research tasks) and 
creating a more supportive environment for researchers. The 
expected impacts on the excellence of research systems as a 
result of WIDERA (ERA) projects is also evidenced by the survey 
results: 45% of survey respondents (45 out of 99) agreed to a 
‘large’ or ‘very large’ extent that their project has resulted in/is 
likely to result in pushing the frontiers of knowledge. Meanwhile, 
40% of respondents (39 out of 97) agreed to a ‘large’ or ‘very 
large’ extent that their project has resulted in/is likely to result in an 
increase in research productivity (e.g. the number of peer-
reviewed publications produced). 

EFC8.2 

What has been the 
contribution of the 
Framework 
Programme as a 
whole, and of the 
core programme 
parts covered in 

Detailed analysis of the various 
aspects, including quantitative 
assessment based on monitoring data 
and parallel studies (e.g. MSCA talent 
circulation, knowledge ecosystems in 
the new ERA study, etc.) and 
qualitative assessment of contribution 

Literature 
review 

Interviews 

Case studies 

Benchmarks 

ERC: The survey of Horizon Europe ERC beneficiaries indicates 
that for 83% of ERC respondents (198 out of 233), opportunities to 
improve skills and competences was, to a ‘large’ or ‘very large’ 
extent, a motivating factor in applying for ERC funding (ERC-BEN-
025). However, among unsuccessful applicants, this percentage 
was only 47% (682 out of 1,367 respondents) (ERC-APPL-023). 
According to the survey of ERC beneficiaries, the skills most 

929 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

this procurement in 
particular to: 
• improving the 
skills of Europe’s 
researchers and 
facilitating the 
emergence of new 
talented 
researchers and 
cross-border and 
cross-sector 
mobility? 

and attribution; examples and 
narrative. 

Examples: 

• 

• 

Training in the context of FP 
participation; skills developed and 
strengthened.  

International and inter-sectoral 
mobility in the context of FP 
participation and appreciation of 
the attractiveness of the ERA for 
researchers from abroad, etc. 

Methods and 
analyses 

Surveys 

Key findings 

Evidence 

developed through the ERC projects are ‘project and people 
management’ and ‘scientific methods and/or techniques’, both with 
83% of respondent (192 out of 233) stating to a ‘large’ or ‘very 
large’ extent. ‘Thinking’ was indicated to a ‘large’ or ‘very large’ 
extent by 78% of respondents (196 out of 237) (see Figure 5). 
‘The sustainable implementation of research activities’ and 
‘commercialisation of research outputs’ were the skills developed 
the least, according to respondents (each with a share of 18%; 41 
out of 230, and 43 out of 234 respondents, respectively). 

The ERC is presently supporting more than 600 researchers who 
are involved in ongoing projects. Notably, 468 of these 
researchers are involved in upskilling activities (training, 
mentoring/coaching, mobility, and access to R&I infrastructures) in 
projects funded by the Framework Programme (KIP2)1328. The 
upskilling of researchers mostly concerns researchers who are in 
the earlier stages of their research careers (i.e. first-stage 
researchers and recognised researchers). Conversely, the ERC so 
far is supporting 417 established researchers (i.e. senior 
researchers and top-grade researchers)1329. 

In the survey of Horizon Europe ERC beneficiaries, 61% of 
respondents (146 out of 241) stated that their ERC project 
responds to a ‘large’ or ‘very large’ extent to their needs for 
international mobility opportunities (ERC-BEN-025). Moreover, 
84% (196 out of 233) strongly agreed or rather agreed that, 
compared with the research funding available at national and/or 
regional level, Horizon Europe provides more international mobility 
opportunities for researchers ((ERC-BEN-044). In one interview, 

1328 This reflects the short-term KIP2 indicator, and is provided by the EC. 
1329 Categorisation as defined in the Frascati 2015 Manual, and identified as a short-term indicator for KIP2. 

930 

 
 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

the PI indicated that being awarded an ERC grant had enabled 
them to choose among various host institutions across Europe; 
they confirmed that they had moved from one EU country to 
another because of the ERC grant. In interviews, several 
examples were given of mobility opportunities for team members 
or the recruitment of international team members, which were also 
said to have been enabled by the bottom-up character and size of 
grants, which provide a greater degree of freedom to define a 
relevant research approach including international elements. A 
similar share is seen among unsuccessful applicants, with 53% of 
respondents (698 out of 1,366) stating that the Horizon Europe 
responds to a ‘large’ or ‘very large’ extent to their needs for 
international mobility opportunities (ERC-APPL-023).  

Overall, intersectoral (e.g. industry-academia) mobility remains 
very limited under ERC grants, which is to be expected given the 
design and objectives of these grants. The survey (ERC-BEN-028) 
shows that only one out of 237 ERC PIs indicated that their ERC 
project had contributed to them moving from industry to academia 
at the start of the project. Nevertheless, 20% considered that the 
ERC project responds to a ‘large’ or ‘very large’ extent to their 
needs for intersectoral (especially industry-academia) mobility 
opportunities (ERC-BEN-025).  

MSCA: This study finds that the MSCA programme is highly 
effective in providing international mobility opportunities. This is 
reflected in the findings of the survey, in which an overwhelming 
majority of successful MSCA PF applicants (87%) agreed that the 
international mobility opportunities provided by their MSCA project 
respond to a ‘large’ or ‘very large’ extent to their needs. In 
addition, an overwhelming majority of MSCA PF fellows (95%) 
agreed that the opportunities provided by their MSCA project 
respond to their need to improve their skills and competences. 

931 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

Among successful MSCA PF applicants, 74% agreed to a ‘large’ 
or ‘very large’ extent that the host organisation provides adequate 
access to research training and continuous development. 
Similarly, the findings of the Survey of Horizon Europe 
beneficiaries also suggest that successful MSCA applicant 
organisations share a generally optimistic view regarding the 
potential for organisational-level improvements in terms of training 
and opportunities for the continuous professional development for 
researchers. Successful MSCA applicant organisations largely 
anticipate a positive impact, with the majority of respondents 
(74%) expecting improvements to a ‘large’ or ‘very large’ extent. 

To assess the potential impact of the MSCA on the acquisition of 
skills, the survey of successful Horizon Europe applicants asked 
MSCA PF fellows to what extent (if at all) would participation in 
their Horizon Europe projects have an impact on the development 
of their skills in various areas. Among MSCA Postdoctoral Fellows, 
there is a a strong belief that participation in Horizon Europe 
projects will have a substantial impact on the development of core 
research skills, particularly in scientific methods and/or techniques 
(93%), critical and abstract thinking (89%), and the communication 
of research results (87%). However, while most areas received a 
positive response, in categories such as the commercialisation of 
research outputs, entrepreneurship and digital technologies, fewer 
than 40% of respondents expected a significant impact. 

INFRA: INFRA contributes to the training of researchers, 
especially with regard to the use of world-class equipment and 
tools offered by RIs. The survey results indicate that around half of 
projects produce training programmes, and a majority of projects 
improve the skills of researchers as well as their careers. The key 
survey results are summarised below: 

932 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

•  One of the top five outputs indicated by INFRA beneficiaries 

• 

• 

• 

surveyed is training programmes: 52.40% of survey 
respondents (86 out of 164) indicated they provided training 
programmes (INFRA-BEN-033). 
42.60% of beneficiaries (67 out of 157) indicated that their 
project is, to a large or very large extent expected to lead 
improvements with regard to the quality of training and 
opportunities for continuous professional development 
(INFRA-BEN-033).  
74.20% (118 out of 159) of INFRA beneficiaries surveyed 
said their project improves (or is likely to improve) the skills, 
knowledge and competences of researchers to a ‘large’ or 
‘very large’ extent (INFRA-BEN-031). 
52.90% of INFRA beneficiaries surveyed (83 out of 159) said 
their project improves (or is likely to improve) the career 
prospects of researchers to a ‘large’ or ‘very large’ extent 
(INFRA-BEN-031).  

INFRA also contributes significantly to the cross-border mobility of 
researchers by offering TNA/VA to RIs. A large majority of INFRA 
beneficiaries (78.9%, or 119 out of 151) reported that INFRA 
provides more international mobility opportunities for researchers 
than national and/or regional funding (INFRA-BEN-045). Figures 
on TNA/VA under Horizon Europe are not yet available, but under 
Horizon 2020 a total of 16,165 unique researchers received 
access to RIs through INFRA1330.  

1330 European Commission, Directorate-General for Research and Innovation, Budraitis, M., Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of 
the European framework programmes for research and innovation for excellent science: Horizon 2020: annexes, Luxembourg: Publications Office of 
the European Union, pp. 848-858, 2023, available at: https://data.europa.eu/doi/10.2777/353383. 

933 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

In addition, preliminary evidence for KIP2 ‘Strengthening human 
capital in R&I’ indicates that Horizon Europe is so far supporting 
more than 5,000 researchers via INFRA. Of these researchers, 
30%% are involved in upskilling activities (training, 
mentoring/coaching, mobility and access to R&I infrastructures) in 
projects funded by the programme (KIP2). Those researchers 
involved in upskilling activities are the ones that are in the earlier 
stages of their research careers (i.e. first-stage researchers and 
recognised researchers). 

WIDERA: 

Widening: The Evaluation study on Excellent Science under 
Horizon 2020, underlined that SEWP beneficiaries agree that 
Widening actions have contributed significantly to improving the 
skills of researchers in specific areas. This was again confirmed in 
new interviews carried out with Horizon 2020 beneficiaries. In 
particular, Twinning has been stressed by many interviewees and 
stakeholders as a proven concept for capacity building and 
improving skills, under both Horizon 2020 and Horizon Europe. 
Twinning actions truly allow the exchange of knowledge and skills, 
and are also seen as an instrument enabling engagement in 
deeper relationships with leading European institutions. Therefore, 
many interviewees and stakeholders (e.g. in the positioning 
papers submitted to the public consultation on the past, present 
and future of the FP) call for an increased budget for Twinning. 
Some voice support for an increased budget per project, as well 
as the introduction of a separate budget for advanced partners, to 
incentivise participation. Several Twinning projects have been 
affected by COVID-19, in the sense that physical exchanges, 
mobilities, conferences, training and so on were impossible. 
Nevertheless, all interviewees spoken to in the context of this 
study emphasised the added value provided by this action in terms 

934 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

of skills development and networking. Almost 90% of successful 
WIDERA Widening beneficiaries (152 out of 172 respondents) in 
the survey indicated that the improvement of the skills, knowledge 
and competences of researchers is to a ‘large’ or ‘very large’ 
extent likely to be achieved (WIDERA-WIDE-BEN-031).  

Several actions under Horizon Europe are devoted specifically to 
fostering mobility. The ERA Fellowships are assigned for 
excellent researchers undertaking cross-border mobility to a 
widening country. During the first call under Horizon Europe 
(HORIZON-WIDERA-2022-TALENTS-02), 52 grants were signed 
according to the Horizon Dashboard (accessed on 31 August 
2023). The top participating countries were Czechia, Portugal and 
Greece. While this instrument is widely appreciated, one 
interviewee indicated that since funding is not granted under the 
well-known MSCA initiative, the prestige associated with this grant 
may be lower. In addition, ERA Talents aims to support the 
training and mobility of researchers, innovators and other R&I 
talents across sectors, with a particular focus on widening 
countries. It encourages inter-sectoral collaboration in R&I through 
cross-sectoral exchange of staff. The Horizon Dashboard indicates 
that under the first call (HORIZON-WIDERA-2022-TALENTS-03), 
nine grants were signed, including 86 unique participants. The 
widening countries with greatest numbers of participants are 
Greece and Slovenia.  

Almost 90% of successful WIDERA Widening beneficiaries (152 
out of 170) indicated in the survey that WIDERA funding provides 
more opportunities for international mobility than national or 
regional funding (Survey of Horizon Europe beneficiaries, 
conducted May-July 2023).  

935 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

ERA: Study evidence also indicates that WIDERA (ERA) projects 
have contributed or are likely to contribute to the development of 
the skills and competences of administrative, managerial and 
academic staff at universities. According to the survey findings, 
77% of WIDERA (ERA) beneficiaries (77 out of 99 respondents) 
agreed to a ‘large’ or ‘very large’ extent that their project had 
achieved or is likely to achieve an improvement in the skills, 
knowledge and competences of researchers, with another 63% 
(63 out of 99) agreeing to a ‘large’ or ‘very large’ extent that their 
project will contribute to improving the career prospects of 
researchers. Likewise, some projects in the field of citizen 
science/science and society helped to equip young researchers 
with entrepreneurial skills. Case study evidence shows that in 
some cases, citizen science-oriented activities supported under 
WIDERA (e.g. brainstorming sessions during the TalentOn 
competition event for young researchers) resulted in spin-off 
initiatives/start-up companies initiated by young researcher 
beneficiaries. Moreover, case study evidence confirms that citizen 
science-oriented projects – such as, for example, the EU Contest 
for Young Scientists (EUCYS) – are particularly effective in 
increasing the visibility and attractiveness of research careers and 
motivating young talented people to choose a research career 
path. 

EFC8.3 

What has been the 
contribution of the 
Framework 
Programme as a 
whole, and of the 
core programme 

Detailed analysis of the various 
aspects, including quantitative 
assessment based on monitoring data 
and parallel studies (e.g. MSCA talent 
circulation, knowledge ecosystems in 
the new ERA study, etc.) and 

Literature 
review 

Interviews 

Case studies 

INFRA: INFRA funds the development of RIs, particularly those 
identified in the ESFRI roadmap, which outlines the essential RIs 
needed across Europe for the next 10-20 years. In this way, 
INFRA has contributed to the 41 ESFRI landmarks that exist 
today. Under Horizon Europe, a total of eight design studies have 
been funded, with a net EU contribution of EUR 22 million (as of 

936 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

parts covered in 
this procurement in 
particular to: 
• supporting world 
class physical and 
knowledge 
infrastructures and 
facilities? 

qualitative assessment of contribution 
and attribution; examples and 
narrative. 

Benchmarks 

Surveys 

Example: 

• 

New research infrastructures, and 
so on 

August 2023). These design studies are aimed at discovering new 
ESFRI projects. In addition, 10 preparatory phase projects have 
been granted, with a net EU contribution of EUR 30 million. These 
projects are aimed at bringing ESFRI projects towards legal, 
financial and technical maturity. Lastly, four implementation 
projects were granted, with a net EU contribution of EUR 5 million. 
Beyond INFRADEV, INFRATECH also plays an important role in 
the development of Ris, as it funds joint research on the next 
generation of tools, equipment and methods. A total of EUR 161 
million has been granted in this destination so far (as of August 
2023). 

MSCA: Survey findings reveal that 79% of successful MSCA PF 
applicants agree to a ‘large’ or ‘very large’ extent that their MSCA 
project responds to their needs for access to good research 
facilities/equipment/infrastructures. Among successful MSCA 
applicant organisations, 61% reported that they expect it to lead, 
to a ‘large’ or ‘very large’ extent, to improved access to physical 
and knowledge infrastructures, facilities and data resources.  

WIDERA: 

Widening: The Evaluation study on Excellent Science under 
Horizon 2020 found that the Widening actions often support the 
development and better use of physical and knowledge 
infrastructure and facilities. For instance, several Twinning actions 
addressed capacity building and learning about specific 
infrastructures or research facilities that were present in partner 
institutions. Teaming actions aim to set up or modernise Centres 
of Excellence, although infrastructure costs in these cases need to 
be covered by complementary funding. Newly introduced 
Widening actions also aim to support world-class infrastructures 
and facilities. For instance, one of the expected outcomes of the 

937 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

Excellence Hubs is to contribute to common investment plans for 
R&I, including infrastructures that leverage national, regional and 
European funds as well as private capital in a synergetic manner 
(WIDERA work programmes). Another of the expected impacts of 
‘Destination 2: attracting and mobilising the best talents’ (which 
includes ERA Chairs, ERA Talents and ERA Fellowships) is the 
better use of research infrastructures.  

ERA: As previously mentioned, ERA projects are likely to result in 
‘soft’ infrastructures paving the way for structural impacts across 
the ERA, including policy standards-setting measures (e.g. 
recommendations, methods, tools, roadmaps, educational 
materials, training programmes etc.) that will empower beneficiary 
institutions and institutions outside the projects to undertake 
transformation. 

EFC8.4 

What has been the 
contribution of the 
Framework 
Programme as a 
whole, and of the 
core programme 
parts covered in 
this procurement, 
in particular to: 
• supporting 
scientific 
breakthroughs, 
higher risk 
research and 

Detailed analysis of the various 
aspects, including quantitative 
assessment based on monitoring data 
and parallel studies (e.g. MSCA talent 
circulation, knowledge ecosystems in 
the new ERA study, etc.) and 
qualitative assessment of contribution 
and attribution; examples and 
narrative. 

Example: 

• 

Research in future and emerging 
areas, etc. 

Literature 
review 

Interviews 

Case studies 

Analysis of 
research 
topics 

Benchmarks 

Surveys 

ERC: As mentioned under question EFC8.1, two case studies 
have been developed by the study team to assess the extent to 
which the ERC contributes to trending research pathways in the 
fields of life sciences, physical sciences and engineering on the 
one hand, and social sciences and humanities (SSH) on the other.  

•  With respect to life sciences, physical sciences and 

engineering, ERC-funded research has contributed most in 
terms of quantity of outputs in the fields of physics, 
astronomy and engineering, covering topics such as 
‘Perovskite Solar Cells; Solar Cell; Formamidine,’ with a total 
of 418 scientific publications under Horizon 2020, and 
‘Exoplanets; Kepler; Atmosphere’ (401 scientific 
publications). In terms of impact, ERC-funded research has 
contributed most to the fields of biology and clinical medicine, 

938 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

research in 
emerging areas of 
science and 
technology? 

covering topics such as ‘Single-Cell Analysis; Small 
Cytoplasmic RNA; Single Cell RNA Seq’ (28.2% of the ERC-
sponsored research outputs on this topic feature among the 
top 1% most-cited publications) or ‘Nasopharyngeal Swabs; 
Serologic Tests; COVID-19' (72%% of ERC-sponsored 
research outputs on this topic feature in the top 10% most-
cited.) 

• 

In the field of social sciences and humanities, ERC-funded 
research has contributed most in terms of quantity of outputs 
in the fields of social sciences and historical studies, covering 
topics such as ‘Genome; Neanderthals; Anatomically Modern 
Humans’ (70 scientific publications). In terms of impact, it has 
contributed most in the fields of economics and business, 
covering topics such as ‘Platforms; Collaborative 
Consumption; Peer to Peer’ (26.3% of ERC-sponsored 
research output on this topic features in the top 1% most-
cited publications). 

79% (191 out of 241) of successful ERC applicants who replied to 
the survey agreed that Horizon Europe to a ‘large’ or ‘very large’ 
extent provides sufficient funding opportunities to conduct 
basic/fundamental research in new and emerging research areas.  

MSCA: 79.3% of successful MSCA applicants who replied to the 
survey agreed that Horizon Europe to a ‘large’ or ‘very large’ 
extent provides sufficient funding opportunities to conduct 
basic/fundamental research in new and emerging research areas. 

INFRA: INFRA contributes to scientific breakthroughs in an 
indirect way, by funding the development of and access to world-
class RIs that researchers can use to perform high-risk research. 
In the survey of INFRA beneficiaries, the majority (63.5%, or 99 

939 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

out of 155) indicated that their project contributes to emerging 
areas of science and technology, while 62.8% (98 out of 156) 
indicated that their project pushes/will push the frontiers of 
knowledge to a ‘large’ or ‘very large’ extent.  

WIDERA: 

Widening: In the survey of successful WIDERA Widening 
participants, 60.6% of respondents (103 out of 170) agreed to a 
large or very large extent  with the statement that their project is 
pushing or will push the frontiers of knowledge, and almost 65% 
agreed to a large or very large extent that their project contributes 
to emerging areas of science and technology (WIDERA-WIDE-
BEN-031, Survey of Horizon Europe beneficiaries, conducted 
May-July 2023). Furthermore, as indicated in RV2, the analysis of 
research trends based on publications generated in SEWP under 
Horizon 2020 indicates that SEWP has generated publications on 
prominent topics (see Annex 6). ERA: in the survey of WIDERA 
(ERA) beneficiaries, 45% of respondents (45 out of 99) agreed 
that their project results have, to a ‘large’ or ‘very large’ extent, 
resulted in/are likely to result in pushing the frontiers of knowledge. 
Meanwhile, 40% of respondents (39 out of 97) agreed that their 
project has, to a ‘large’ or ‘very large’ extent, resulted in/is likely to 
result in increased research productivity (e.g. the number of peer-
reviewed publications produced). 

EFC8.5 

What has been the 
contribution of the 
Framework 
Programme as a 
whole, and of the 

Detailed analysis of the various 
aspects, including quantitative 
assessment based on monitoring data 
and parallel studies (e.g. MSCA talent 
circulation, knowledge ecosystems in 

Literature 
review 

Interviews 

ERC: Under Horizon 2020, around 18% of ERC team members 
were non-ERA nationals coming from almost 90 countries 
worldwide. The four most frequent nationalities among non-ERA 
ERC team members under Horizon 2020 were researchers from 
India (18% of non-ERA nationals), China (17%), the US (12%) and 

940 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

core programme 
parts covered in 
this procurement, 
in particular to: 
• making Europe 
more attractive for 
world class 
researchers from 
abroad? 

the new ERA study, etc.) and 
qualitative assessment of contribution 
and attribution; examples and 
narrative. 

Examples: 

International and inter-sectoral mobility 
in the context of FP participation and 
appreciation of the attractiveness of 
the ERA for researchers from abroad, 
etc. 

Methods and 
analyses 

Case studies 

Benchmarks 

Surveys 

Key findings 

Evidence 

Russia (7%). The ERC Activity Report of 2020 also stated that the 
largest share of non-ERA team members was found in the 
category of postdoctoral researchers. Preliminary administrative 
and monitoring data on Horizon Europe received by the study 
team in May 2023 shows that 66 nationalities could be found 
among the total of 2,484 PIs and corresponding PIs of the main 
ERC grants. Of these, 10% have a non-ERA nationality, with the 
most common nationalities being those of the United States (33% 
of non-ERA nationals), China (15%) and India and Canada (both 
8%).  

MSCA: Under Horizon Europe, the MSCA are particularly 
successful in attracting world-class researchers from abroad. 
Based on an analysis of the mobility patterns of MSCA PF fellows 
by nationality, 38% of fellows represent third countries, while 
researchers from associated countries account for 3% of all 
researchers. The analysis also shows that the programme 
attracted researchers from 61 non-EU Member States, with the 
highest shares of MSCA fellows coming from India (19%), Canada 
(14%), United Kingdom (7%) and Iran (6.5%). 

INFRA: By investing in world-class RIs, Europe becomes more 
attractive to researchers from abroad. Figures are not yet available 
for TNA/VA under Horizon Europe; however, TNA/VA is also 
available (to a limited extent) to researchers from outside Europe. 
This stimulates international collaboration and excellent science. 
Under Horizon 2020, 19% of TNA/VA users came from associated 
countries and 9% came from third countries.  

WIDERA: 

Widening: The Evaluation study on Excellent Science under 
Horizon 2020 highlighted that widening countries sometimes 

941 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

experience difficulties in attracting international talent. This 
challenge was particularly notable in the ERA Chairs action – a 
fact that was also underlined in the Court of Auditors’ report1331. As 
a consequence, some novelties have been introduced into the 
ERA Chairs actions under Horizon Europe. Specifically, the ERA 
chairholder now has to be identified in the proposal, and there is 
greater flexibility over the appointment of the ERA chairholder. For 
instance, the chairholder can be employed part-time and can 
choose to carry out multiple stays at the coordinating institution, 
rather than a single, continuous stay. While these changes are 
generally appreciated, one interviewee questioned whether the 
commitment of the ERA Chairholder and the results achieved 
would be the same. In addition, several stakeholders still indicate 
that attracting excellent researchers is a major challenge, 
especially if the host institution is located in a rural areas or has 
generally less attractive salary conditions. The positioning paper 
by the Guild of European Research-Intensive Universities in the 
context of the OPC1332 raises the possibility of hiring an employee 
of the host institution, if the recruitment of external experts 
continues to be unsuccessful after several open calls. The Guild 
also recommends using the MSCA calculation method to 
determine salary levels for ERA chairholders, rather than following 
national guidelines. Under Horizon Europe’s WIDERA destination 

1331 European Court of Auditors, Measures to widen participation in Horizon 2020 were well designed but sustainable change will mostly depend on 
efforts by national authorities. Special report No 15, 2022, Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2865/359822.  
1332 e.g. The Guild of European Research-Intensive Universities, Lessons learned and way forward: The Guild’s recommendations for Horizon 
Europe, 2023. 

942 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

‘attracting and mobilising the best talents’, ERA Talents and ERA 
Fellowships are also included. 

ERA: ERA projects (particularly those focusing on acceleration 
services/university transformation) are also likely to result in 
institutional-level changes/improvements in standards and 
practices, thereby contributing to better working conditions, and 
the career and professional development of staff. When asked if 
they expect participation in the Horizon Europe project to lead to 
improvements at organisational level, around 58% of WIDERA 
(ERA) beneficiary respondents (57 out of 99) indicated they expect 
improvements in the quality of training and opportunities for 
continuous professional development to a ‘large’ or ‘very large’ 
extent. This was followed by 41% of respondents (40 out of 97) 
who expected improvements in the provision of career advice for 
researchers 

ERC: The Evaluation study on Excellent Science under Horizon 
2020 found that the ERC has a structuring effect by increasing the 
visibility and reputation of host institutions: interviewees 
considered that the prestige associated with ERC grants increased 
the attractiveness and credibility of host institutions. Both for 
grantees and host institutions, an ERC grant is considered a 
standardised label of excellence by the research community. This 
effect is even more significant for less well-known universities and 
institutions. It also contributes to the observation by interviewees 
that host institutions seek to attract ERC grantees to their 
institution.  

In general, non-academic sectors play only a limited role in the 
ERC, and synergies between academia and industry are limited. 

EFC9.1 

Detailed analysis of the various 
aspects, including indicators in existing 
studies/scoreboards and information 
collected through parallel studies (e.g. 
MSCA talent circulation, knowledge 
ecosystems in the new ERA study, 
etc.),  as well as qualitative 
assessment of contribution and 
attribution; examples and narrative. 

Literature 
review 

Interviews 

Case studies 

Benchmarks 

Surveys 

What has been the 
contribution of the 
core programme 
parts covered in 
this procurement in 
particular to: 
• consolidating the 
European 
Research Area. 
Does it have a 
structuring effect 
on the national 
research systems 

943 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

in terms of 
changes to national 
and institutional 
policies and 
practices? 

For instance, data from the Horizon Dashboard show that under 
both Horizon 2020 and Horizon Europe, private for-profit entities 
have represented only 2% of total participations. Although there 
are exceptions, for many projects that offer potential for 
commercialisation, connecting with industry still remains a difficult 
task.  

MSCA: Overall, participation in the MSCA is considered to 
contribute to participating organisations’ ability to attract funding 
from national and regional sources. A large majority of 
beneficiaries surveyed across different actions regarded their 
participation in the MSCA as having contributed to a ‘large’ or ‘very 
large’ extent to their ability to attract funding from national, regional 
and European sources. This was further confirmed by interviews 
with both Horizon Europe beneficiaries and NCPs, who regarded 
participation in the MSCA as particularly useful in making 
participating organisations more competitive and able to attract 
funding from external sources. This structuring effect was 
particularly important for small and recently created organisations 
wishing to increase their collaborations in Europe, as well as for 
organisations from widening countries trying to make their 
institutions as competitive as their counterparts in non-widening 
ones. 

Similarly, participation in the MSCA appears relevant in enhancing 
the international visibility of organisations. Above 70% of both 
beneficiaries and unsuccessful applicants across actions regarded 
the MSCA as responding to their organisation’s need to increase 
its visibility – which, according to most interviewees, represents an 
important factor in strengthening their research capacities. 

Most survey respondents considered that participation in the 
MSCA responded, to a ‘moderate’, ‘large’ or ‘very large’ extent, to 

944 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

their organisation’s need to promote institutional change at 
organisational, regional or national level. Interview evidence 
shows a similar trend, although with important nuances. In terms 
of administrative or organisational practices, participation in the 
MSCA only rendered significant effects where administrative 
procedures were not already influenced by national legislation, or 
by the institutional environment. Therefore, these effects were 
easily identifiable in widening countries or institutions with more 
recent participation in their country’s academic environment. 

The MSCA were found to have a broader structuring impact, 
influencing both beneficiary organisations and wider 
national/regional frameworks. A prime example is the 
institutionalisation of gender equality policies due to MSCA 
involvement, as mentioned by several interviewees. Furthermore, 
the rigour of the MSCA evaluation process has been adopted by 
certain national programmes, thereby setting high-quality 
benchmarks for application assessments. Guidelines and 
principles stemming from the MSCA, such as the need for a 
management plan or the adoption of Open Science practices, are 
also applied as requirements for national funding programmes. 
(For more information on this, see Case Study 3. The structuring 
impacts of the MSCA.)  

Preliminary findings from interviews have shown that the 
structuring impacts of the MSCA go beyond participating 
organisations, and are implemented within national research 
systems, particularly in widening countries. Guidelines and 
principles stemming from the MSCA, such as the need for a 
management plan or the adoption of Open Science practices, may 
also be applied as requirements for national funding programmes, 
while others, such as gender equality measures, are envisioned to 
be applied. Similarly, some countries have created funding 

945 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

programmes directed towards recipients of the MSCA Seal of 
Excellence. 

INFRA: According to interviews and desk research, INFRA 
contributes significantly to consolidating the ERA, especially by 
reducing fragmentation and duplication in the RI landscape. It 
does this both by developing ESFRI RIs, and also by fostering 
collaboration between national RIs and facilities. The ERIC 
Regulation has also been a very successful mechanism in 
fostering collaboration between countries and reducing 
fragmentation and duplication.  

Moreover, INFRA projects also aim to contribute to more effective 
policy-making. Around half of INFRA beneficiaries surveyed 
(50.6% or 83 out of 164) say their project is likely to lead to 
recommendations for policy-makers or inputs into national or EU 
policies (INFRA-BEN-027). However, only a minority of 
respondents (32.90%, or 54 out of 164) indicate that there is 
enough research under INFRA that contributes to policy-making 
(INFRA-BEN-024). Furthermore, only 29.8% of INFRA 
beneficiaries (48 out of 161) say their project supports policy-
making or the development of standards-setting measures 
(INFRA-BEN-026). 

WIDERA:  

Widening: The Widening actions are intended to induce structural 
changes in regional or national R&I landscapes or the institutions 
involved, with the aim of fostering R&I performance. Since 
structural changes may take some time to emerge, the present 
evaluation support study focuses in depth only on those structural 
changes introduced as a result of Horizon 2020 SEWP actions.  

946 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

The Widening actions have proven effective in inducing diverse 
types of structural changes, and as both project beneficiaries and 
national level-stakeholders interviewed perceived them as having 
a positive effect on R&I performance. Case Study 7 (see Annex 2) 
shows that structural changes at institutional level appear to be 
widespread and common across the SEWP Teaming, Twinning 
and ERA Chairs projects analysed, and show some similarities 
with the institutional changes observed under MSCA actions. 
Evidence also indicates that changes occur more commonly at 
institutional level than at regional or national levels. However, this 
may partly be attributed to the fact that institutional changes may 
be more narrowly defined and potentially applicable to all projects, 
and are hence easier to identify than those that occur at national 
or regional levels.  

Among the national or regional structural changes most frequently 
identified are contributions to the development of regional or 
national R&I strategies, as well as the implementation of actions 
leading to increased knowledge and expertise in the building, use 
and operation of new infrastructures, facilities and laboratories. 
The mechanisms through which these changes occur usually 
relate to the design and objectives of the project, and are also 
associated with the degree of political involvement as well as 
alignment with national or regional policies and priorities. Case 
Study 7 further emphasises that a more consolidated and robust 
monitoring approach (e.g. including a definition and specific 
categories of impact) would better facilitate the monitoring of the 
progress made by the actions in this regard.  

In the survey of Horizon Europe beneficiaries, 44.6% of successful 
WIDERA widening applicants (79 out of 177) indicated they intend 
to produce recommendations for policy-makers and inputs into 

947 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

national or EU policies (WIDERA-WIDE-BEN-027, conducted 
May-July 2023).  

ERA: In interviews carried out for the case studies as part of this 
study, project stakeholders confirm that their ongoing projects are 
likely to result in policy-setting standards, guidelines and tools that 
will enhance university transformation and produce structuring 
effects across the ERA. For example, one of the final deliverables 
envisaged in one WIDERA (ERA) project focusing on acceleration 
services includes policy recommendations to support the 
acceleration of entrepreneurial and innovative HEIs across 
Europe, which can be freely used by institutions and policy-makers 
across Europe. Similarly, some of the key deliverables planned in 
a project focusing on the professionalisation of research 
management (RM) include standardisation measures for the RM 
profession, together with related terminology and ultimately a 
roadmap for improving the RM system in Europe. These 
deliverables will be open for use by institutions across Europe. 

In general, most ERA projects are still during their inception 
phase, and it is too early to estimate their precise results; 
however, stakeholders expect that networking, the sharing of good 
practice experiences, as well as roadmaps, knowledge creation 
and policy standards, will have structuring effects across the ERA 
in the long term. However, the successful implementation of 
projects and the realisation of the above outcomes and impacts at 
institutional level and across the ERA will depend on a number of 
factors, such as the ability of project partners to ensure the active 
dissemination of information and the awareness of project 
activities among the local research community/wider public, as 
well as effective communication and coordination between project 
partners with regard to their varying needs, and the commitment of 
policy stakeholders and top-level management to undertake 

948 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

transformational changes. In particular, the success of ERA 
projects focusing on university transformation depends on the 
ability of projects to mobilise wider groups of stakeholders whose 
active engagement is crucial for data gathering and stakeholder 
consultation processes that will feed into development of policy 
standards, recommendations and roadmaps 

ERC: It is still too early to assess the resulting IPR and 
commercialisation of results arising from Horizon Europe ERC 
grants; however, for Horizon 2020, data from the Horizon 
Dashboard (accessed on 1 September 2023) show a total of 759 
IPR applications from 442 ERC projects, of which 87% are 
patents. This figure corresponds to 16% of all IPR applications 
reported under Horizon 2020 so far. This share compares with 
38% of all peer-reviewed publications, if one considers the 
contribution made by the ERC to total scientific production under 
Horizon 2020. These figures are consistent with the 
commercialisation attitudes, experiences and behaviour reported 
by ERC grantees: the survey indicates that only 29% of ERC PI 
respondents (59 out of 241) regarded their ERC project responds 
to a ‘large’ or ‘very large’ extent to their needs with respect to the 
commercialisation of research results. In terms of skills gained, as 
previously reported, only 18% (43 out of 234) of respondents 
indicated they had (to a ‘large’ or ‘very large’ extent) gained skills 
related to commercialising research results as a result of 
participating in the ERC project, and 33% (78 out of 233) had 
gained skills in relation to IPR management.  

Nonetheless, a significant number of beneficiaries in the survey 
(22%, or 53 out of 238) expected the ERC project to result in/ be 

EFC9.2 

Detailed analysis of the various 
aspects, including indicators in existing 
studies/scoreboards and information 
collected through parallel studies (e.g. 
MSCA talent circulation, knowledge 
ecosystems in the new ERA study, 
etc.) and qualitative assessment of 
contribution and attribution; examples 
and narrative. 

Literature 
review 

Interviews 

Case studies 

Benchmarks 

Surveys 

Examples 

Uptake of practices, standards and 
approaches in regions, institutions, 
etc. 

Intellectual property developed, the 
commercialisation of research results. 

What has been the 
contribution of the 
core programme 
parts covered in 
this procurement, 
in particular to: 
• making the 
Union's research 
and innovation 
system more 
performing and 
competitive on a 
global scale (incl. 
spin-offs/start-ups 
creation, job 
maintained and 
created, Intellectual 
Property activity 
deriving from 
scientific results, 
knowledge 

949 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

valorisation and 
market uptake)? 

likely to result in a patent, and 22% (51 out of 231) foresaw patent 
activities as exploitation activities of the ERC project. Meanwhile, 
15% (34 out of 231) foresaw the establishment of spin-off or start-
up companies. The ERC website indicates that 400 start-ups have 
been identified as being founded or co-founded by ERC PIs. 
These figures thus illustrate the potential of bottom-up 
fundamental research supported by the ERC to lead to IPR and 
commercial solutions.  

MSCA: The MSCA actions have a fairly limited effect on IPR 
activity and on the creation of spin-offs and start-ups. According to 
the survey, only 31% of beneficiary organisations (151 out of 500) 
and 18% of MSCA PF fellows (152 out of 830) believe their project 
is likely to produce patents. Only one in five beneficiary 
organisation (122 out of 500) and one in 10 MSCA PF fellows (83 
out of 830) believe their projects will result in new or improved 
products, services or processes. 

INFRA: INFRA has an indirect impact on Europe’s 
competitiveness. By developing world-class RIs that are 
accessible to researchers and industry, excellent research is 
facilitated, as well as innovation and scientific breakthroughs. Due 
to this indirect impact, only a minority of INFRA projects lead to 
patents (7.3%, or 12 out of 164 INFRA beneficiaries indicate this is 
a likely result of their project – INFRA-BEN-027). Similarly, only a 
small minority of projects lead to the establishment of spin-off or 
start-up companies (only 3.60%, or 7 out of 152 INFRA 
beneficiaries indicate this is likely to be a result – INFRA-BEN-
039).  

WIDERA: 

950 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

Widening: As the bulk of Widening actions represent Coordination 
and Support Actions (CSAs), the Widening actions have a fairly 
limited effect on IPR activity and on the creation of spin-offs and 
start-ups. This was also underlined in the Evaluation study on 
Excellent Science under Horizon 2020. In the survey of successful 
WIDERA Widening respondents, 12.4% of respondents (22 out of 
177) reported that their project was likely to produce patents, and 
25.4% (45 out of 177) that it was likely to produce new or 
improved products, services or processes (WIDERA-WIDE-BEN-
027, Survey of Horizon Europe beneficiaries, conducted May-July 
2023). In addition, 28.8% of respondents (49 out of 170) reported 
that their project is (to a ‘large’ or ‘very large’ extent) likely to 
achieve entry to new markets or (global) value chains (WIDERA-
WIDE-BEN-031). More than half of successful WIDERA widening 
applicants who responded (56.8%, or 96 out of 169) strongly 
agreed or rather agreed that, compared with national or regional 
funding, Horizon Europe provides more opportunities for the 
commercialisation of research results (e.g. the applicability of 
results to new markets not present in the national environment) 
(WIDERA-WIDE-BEN-045). 

That said, efforts have been made under Horizon Europe to better 
include the private sector in projects. The Horizon Dashboard 
shows that under Horizon Europe, private for-profit entities 
constitute 12.7% of participations in Widening actions, compared 
with just4.7% under Horizon 2020 (Horizon Dashboard accessed 1 
September, 2023). For example, the action Excellence Hubs, 
takes a quadruple helix approach and aims, among other things, 
to strengthen linkages between science and business, to foster the 
uptake of innovative technologies, and to develop new business 
opportunities (as set out in the WIDERA work programmes). It 
remains to be assessed whether those efforts positively impact the 

951 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

creation of spin-offs and start-ups, IPR activity, knowledge 
valorisation and market uptake when the Widening projects are 
further advanced.  

ERA: Support under the ERA thematic priority is expected to have 
indirect impacts on the innovativeness and competitiveness of EU 
research systems, in particular by fostering structural changes 
across the ERA in the fields of entrepreneurship and 
innovativeness. An example of such a project outcome is provided 
by one of the projects covered in Case Study 8: WIDERA Actions 
on enhancing the European RI system, in which each partner has 
developed a strategic vision statement towards becoming an 
entrepreneurial and innovative university. In these statements, the 
institutions narrow down and identify key areas of institutional 
transformation in the areas of entrepreneurship and 
innovativeness that they aim to undertake though the 
implementation of the project. 

What has been the 
contribution of the 
core programme 
parts covered in 
this procurement in 
particular to: 
• ensuring 
successful 
implementation 
and results of EU 
missions (applies 

EFC9.3 

Detailed analysis of the various 
aspects, including indicators in existing 
studies/scoreboards and information 
collected through parallel studies (e.g. 
MSCA talent circulation, knowledge 
ecosystems in the new ERA study, 
etc.) and qualitative assessment of 
contribution and attribution; examples 
and narrative. 

Literature 
review 

Interviews 

Case studies 

Benchmarks 

Surveys 

ERC: As mentioned previously, the ERC works according to a 
bottom-up approach. Nevertheless, research funded by the ERC 
contributes to delivering on the EU Missions. For example, 
Horizon Dashboard data (accessed on 6 September 2023) shows 
that the second most popular field of science in terms of ERC 
signed grants under Horizon Europe is oncology, which 
contributes to the EU’s Mission on Cancer; the third most popular 
is climate change, which contributes to the EU Missions on climate 
change. While oncology was already the second most popular 
field under Horizon 2020, climate change has gained significant 
momentum.  

952 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

only to Horizon 
Europe)? 

MSCA: The Feedback to Policy Coordination and Support Action, 
running in parallel with the MSCA Work Programme 2023-2024, 
seeks to facilitate stronger connections between MSCA projects 
and key EU policy priorities, thereby raising the visibility of their 
contributions. The incorporation of secondments, research visits 
and non-academic placements within MSCA projects is 
encouraged in order to further enable knowledge transfer and 
exchange, particularly in relation to the five EU Mission goals. To 
further enhance synergies and provide coordinated input into the 
relevant EU policy-making services, four MSCA Cluster events 
have already taken place under Horizon Europe, with the aim of 
showcasing the outcomes and results of MSCA projects relating to 
the thematic areas of the EU Missions. 

INFRA: By shifting to a challenge-oriented approach (with topics 
on cancer, climate change, etc.), the aim for INFRA under Horizon 
Europe is to increase its alignment with and contribution to the EU 
Missions. However, interviewees indicated that such alignment 
could be further increased. An example of good practice to follow 
would be the collaboration between several projects and initiatives 
working on combatting cancer, including projects from INFRA and 
the Health Cluster of Pillar II. However, some interviewees 
expressed the opinion that the EU Missions are mainly a tool for 
communication and engagement towards citizens.  

WIDERA: 

Widening: As indicated in RV2.1, the topics most widely dealt with 
in WIDERA (Widening) signed grants are ecosystems, climate 
change, educational sciences, innovation management, oncology, 
pollution and the sustainable economy (according to the Horizon 
Dashboard, accessed in June 2023). These findings indicate that 

953 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

many Widening projects contribute to EU Missions such as 
beating cancer and climate change. 

In ERA Talents, participants are encouraged to improve training 
and lifelong learning for researchers, innovators and other R&I 
talents. The WIDERA work programmes (21-22 and 23-24) 
recommend training in specific skills that are in demand in industry 
or non-academic sectors in thematic areas such as those of the 
EU Missions.  

ERA: Some of the activities supported under the ERA thematic 
priority indirectly contribute to the EU Missions, e.g. as part of 
Leiden 2022 European City of Science (a citizen science-oriented 
project funded under WIDERA) bright young minds aged between 
21 and 35 years were put to the test in the EU TalentOn contest, 
Researchers were placed into teams and challenged to think 
about five EU Missions and the best means to address their goals. 

EFC10 

Long-term impact 
on researchers' 
careers; 
institutional and 
systemic impact on 
doctoral and 
postdoctoral 
training 
programmes, and 
working / 
employment 
conditions 

Detailed analysis of the various 
aspects, including survey information 
and information from parallel studies 
(e.g. MSCA talent circulation, 
knowledge ecosystems in the new 
ERA study, etc.) and qualitative 
assessment of contribution and 
attribution; examples and narrative. 

Training in the context of FP 
participation. 

Literature 
review 

Interviews 

Case studies 

Surveys 

ERC: 71% (171 out of 241) of successful ERC applicants (ERC-
BEN-025) stated that the ERC project gives them, to a ‘large’ or 
‘very large’ extent, the opportunity to gain a research position with 
better career prospects. The percentage is even higher (91%, or 
218 out of 240 respondents) in relation to increasing the 
international visibility and outreach of their research. However, 
only 51% (124 respondents) stated that, to a ‘large’ or ‘very large’ 
extent, participating in the ERC gives them more attractive 
contractual conditions or a better salary. With regard to aspects 
relating to contractual or salary conditions, it should be noted that 
these factors are also affected by national or institutional 
employment conditions and standards.  

954 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

Perceptions of beneficiaries with 
regard to career prospects. 

Implementation of inspirational 
frameworks (C&C, research careers 
framework, HRS4R, etc.) 

Most respondents to the survey of Horizon Europe successful 
applicants (ERC-BEN-032) expected that their participation in 
ERC will impact the following aspects of their career development:  

• 

• 

• 

Strengthening their capacity to conduct frontier research 
(93%, or 219 out of 236 respondents); 
Strengthening their reputation as a researcher (91%, or 214 
respondents); 
Establishing and/or consolidating their own research group 
(81%, or 191 respondents); 

•  Obtaining a new fellowship/research grant (76%, or 180 

• 

• 

respondents); 
Becoming an established researcher (70%, or 165 
respondents); 
Progressing towards a higher level of responsibility/moving 
into a more senior position (e.g. full professorship) (50%, or 
118 respondents). 

The above aspects were confirmed during the illustrative 
interviews carried out with ERC PIs. Some ERC PIs mentioned, 
for instance, that their ERC grant enabled them to focus solely on 
research, and to establish their position as a top scholar.  

In addition, 84% (196 out of 233 ERC successful applicants) 
‘strongly agreed’ or ‘rather agreed’ that Horizon Europe provides 
more international mobility opportunities for researchers compared 
with the research funding available to them at national and/or 
regional level. In addition, 72% (168 respondents) and 75% (171 
respondents), respectively, strongly agreed or rather agreed that 
receiving funding is more likely to make them pursue their career 
within Europe (as opposed to seeking opportunities outside 

955 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

Europe), and that it helps to better pool the skills and expertise 
necessary for research (ERC-BEN-044).  

MSCA: Among the programme parts in Pillar 1 and WIDERA 
(Widening participation), the MSCA place a profound emphasis on 
the quality of supervision and mentorship provided to researchers 
who are funded under its various schemes. The MSCA Guidelines 
on Supervision aim to ensure that researchers receive effective 
support and mentorship. Effective supervision is a critical 
component of the MSCA programme's commitment to nurturing 
the next generation of research talent. 

According to the survey findings, only slightly more than half of 
successful MSCA PF applicants agreed to a ‘large’ or ‘very large’ 
extent that the host organisation has a career development 
strategy for researchers (55%). However, slightly more MSCA PF 
applicants agreed that the host organisation provides access to 
career advice (59%). Meanwhile, satisfaction with the supervision 
provided is more pronounced among successful applicants, with 
75% of applicants agreeing to a ‘large’ or ‘very large’ extent that 
the host organisation provides adequate supervision, and that 
supervision arrangements are clearly defined. 

The present study also finds that participation in the MSCA 
programme is anticipated to lead to better career opportunities and 
prospects for individual researchers. This is reflected in the 
findings of the survey, in which successful MSCA PF fellows 
agreed that the opportunities to gain a research position with 
better career prospects provided by the MSCA project respond to 
their needs to a ‘large’ or ‘very large’ extent (87%). In parallel, an 
absolute majority of successful MSCA applicant organisations 
(90%) believe to a ‘large’ or ‘very large’ extent that their MSCA 

956 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

project has improved or is likely to improve the career prospects of 
researchers.  

The MSCA also play a significant role in promoting better 
recruitment practices and working conditions for researchers 
across Europe and beyond. They aim to create an attractive and 
supportive research environment by encouraging participating 
organisations to adopt the principles outlined in the European 
Charter for Researchers and the Code of Conduct for the 
Recruitment of Researchers. The MSCA encourage host 
organisations to widely advertise open positions to be filled by 
fellows, provide clear eligibility criteria, and ensure that the 
selection process is fair and unbiased. In addition, while MSCA-
funded projects are often time-bound, host organisations are 
encouraged to provide researchers with clear and fair employment 
conditions, including fixed-term contracts that align with the 
duration of the research project. Researchers should have access 
to social security benefits and other employment rights in the host 
country. 

Successful MSCA applicant organisations also expressed strong 
optimism regarding improvements in recruitment practices (51%) 
and the working conditions of researchers (49%). However, 
expectations are more modest concerning the working conditions 
of other employees, with a more even distribution across different 
levels of anticipated improvement. From the perspective of 
successful MSCA PF applicants, more than half of fellows (60%) 
agreed to a ‘large’ or ‘very large’ extent that their host organisation 
is committed to improving the stability of researchers’ working 
conditions. MSCA PF fellows also agreed ‘large’ or ‘very large’ 
extent that their host organisation protects the intellectual property 
rights of researchers. However, less than half of MSCA PF fellows 

957 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

believe that appropriate complaints/appeals procedures are in 
place within their host organisations (47%). 

Numerous factors relating to recruitment practices and working 
conditions can significantly impact researchers' fellowships during 
their mobility period. Among the challenges faced by successful 
MSCA PF applicants, the most prominent include taxation and 
salary disparities between different countries, which affected 30% 
of respondents. In addition, a notable concern is the lack of 
information regarding details of the fellowship/grant such as 
salary, duration and other arrangements prior the project's 
commencement. Only 11% of fellows reported facing difficulties in 
obtaining visas or work/residence permits, indicating that this is a 
relatively manageable hurdle. Furthermore, less than 10% 
encountered obstacles in securing an employment contract with 
full social security coverage in the host institution. Lack of 
transparency during the recruitment process at the host 
organisation emerged as the issue of least concern to participating 
researchers, with only 7% of respondents expressing concerns. 

Findings from the interviews show that the MSCA contribute to an 
overall improvement in human resource practices and working 
conditions for researchers. However, external barriers hinder 
researchers’ career progression (e.g. lack of funding for 
permanent positions, excessive bureaucracy, and the need for 
collective agreements). The survey of Horizon Europe 
beneficiaries provides more insight into the challenges faced by 
MSCA PF fellows in terms of the quality and transparency of 
recruitment practices and working conditions. Responses to the 
survey indicate that several researchers grappled with uncertainty 
regarding their final salary due to the host institution's inability to 
calculate it prior to the contract being finalised. Others 
encountered substantial deductions from living, family, and 

958 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

mobility allowances, impacting their financial stability. Other issues 
arose for working parents – particularly mothers with children – 
with regard to adapting work conditions to accommodate their 
family responsibilities. Furthermore, a significant concern emerged 
from contracts that prohibited simultaneous employment with other 
institutions, a crucial detail that was not disclosed during the 
application phase or after the grant agreement was signed. These 
challenges collectively highlight the need for clearer 
communication and support structures within the framework of the 
MSCA. 

INFRA: As already indicated earlier under EFC 8.2, INFRA 
contributes to the training of researchers, especially with regard to 
the use of the world-class equipment and tools offered by RIs; 
around half of beneficiaries surveyed indicated that their project 
produces training programmes. Similarly, around half of INFRA 
beneficiaries surveyed (52.9%, or 82 out of 155) indicated that 
their project improves the career prospects of researchers 
(INFRA-BEN-031). At the organisational level, however, only a 
minority of INFRA beneficiaries in the survey (25.5%, or 40 out of 
157) indicated that the working conditions of researchers are 
expected to improve to a ‘large’ or ‘very large’ extent (INFRA-BEN-
033).  

WIDERA:  

Widening: The evidence collected in this study points to some 
success stories in which entirely new research streams and 
educational programmes were built by smartly combining 
Widening grants (often with other grants as well). These include 
'Cetocoen Excellence', a Teaming project in Czechia.  

959 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

Under Horizon Europe, 80.0% of successful WIDERA Widening 
beneficiaries who responded to the survey indicated that their 
project is likely to improve the career prospects of the researchers 
(i.e. 136 out of 170) (WIDERA-WIDE-BEN-031, Survey of Horizon 
Europe beneficiaries, conducted in May-July 2023). At 
organisational level, three-quarters of Widening respondents (128 
out of 171) indicated that their project is expected their project to 
improve the quality of training and opportunities for continuous 
professional development to a ‘large’ or ‘very large’ extent, and 
65.8% (112 out of 170) indicated it would improve the quality of 
scientific supervision (WIDERA-WIDE-BEN-033). 

In terms of working and employment conditions, almost half of 
survey respondents (82 out of 170) believed to a ‘large’ or ‘very 
large’ extent that the project would improve the working conditions 
of researchers at institutional level, and over 40% (71 out of 170) 
believes it would also improve the working conditions of other 
employees within the institution (WIDERA-WIDE-BEN-033).  

ERA: As mentioned previously, ERA projects are likely to produce 
impacts on researchers’ careers and training opportunities, mainly 
though institutional capacity building and acceleration services. 
ERA projects (particularly those focusing on acceleration 
services/university transformation) are also likely to result in 
institutional-level changes/improvements to standards and 
practices, thereby contributing to enhanced working conditions, 
and the career and professional development of staff. When asked 
if they expect participation in the Horizon Europe project will lead 
to improvements at organisational level, around 58% of WIDERA 
(ERA) beneficiary respondents (57 out of 99) indicated that they 
expect improvements in the quality of training and opportunities for 
continuous professional development to a ‘large’ or ‘very large’ 
extent. This was followed by 41% (40 out of 97) of respondents 

960 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

Take-up and 
impact of the Seal 
of Excellence in the 
Member States 
and associated 
countries and their 
link with the 
Structural and 
Investment Funds 
(ESIFs) 

EFC11* 

Number of MS applying the Seal of 
Excellence principle. 

Number of researchers/projects 
funded by national authorities through 
the application of the Seal of 
Excellence principle. 

Qualitative information about the 
impact of the Seal of Excellence 
principle, in terms of inspiring national 
evaluation systems/criteria for funding. 

The extent to which ESIFs have been 
used to fund Seal of Excellence 
recipients. 

Literature 
review 

Monitoring 
data 

Interviews 

961 

who expected improvements to the provision of careers advice for 
researchers 

MSCA: Under Horizon Europe, the MSCA continue to award Seal 
of Excellence certificates to project proposals submitted that have 
been ranked above a predefined quality threshold, but were not 
funded due to budgetary constraints. The Seal of Excellence 
scheme also facilitates the building of synergies between funding 
programmes at EU and national levels, including the potential to 
seek alternative funding sources. In this regard, funding bodies 
can also take advantage of the well-established and high-quality 
Horizon Europe evaluation process to promote territorial 
development.  

So far, the European Commission has awarded MSCA Seal of 
Excellence certificates to a significant number of applicants for 
MSCA Postdoctoral Fellowships (1,956 in 2021 and 1,607 in 
2022). In addition, 21 Seals of Excellence were also awarded to 
applicants who presented outstanding proposals but fell short of 
receiving funding from the COFUND action in 2021.  

The study findings indicate that between Horizon 2020 and 
Horizon Europe, the number of Seal of Excellence schemes at 
institutional, regional and national level increased steadily. 
Currently, 22 funding schemes are in place across 14 EU Member 
States for MSCA Seal of Excellence holders who applied for 
Postdoctoral Fellowships and COFUND. National, regional or 
institutional funding schemes are available in Austria, Belgium 
(Flanders), Bulgaria, Czechia, Cyprus, France, Italy, Lithuania, 
Poland, Romania, Slovakia, Slovenia, Spain and Sweden. Out of 
the 22 existing support schemes, half have been established on a 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

national scale, with four operating at a regional level, and the 
remaining seven being funded institutionally. 

Four schemes currently make use of Cohesion Policy funds under 
the European Regional Development Fund (ERDF) and the 
European Social Fund Plus (ESF+). For example, at a regional 
level, France's Hauts-de-France region has been allocated EUR 
9.7 million from the ERDF for excellent research and innovation 
projects, including those awarded the MSCA Seal of Excellence. 
Eligible beneficiaries include HEIs, research organisations and 
technical centres as well as businesses.  

More recently, new schemes have been launched and funded 
under National Recovery and Resilience Plans (NRRP), 
demonstrating successful synergies between Horizon Europe and 
the Recovery and Resilience Facility (RRF). For example, in Italy, 
the Ministry of Research launched an initiative to support 
approximately 400 researchers under the National RRP. In 
Slovenia, a new initiative funded by the RRF provides financial 
support to successful Slovenian researchers who have been 
awarded the MSCA Seal of Excellence when applying with a host 
organisation abroad under the Individual or Postdoctoral 
Fellowships calls since 2019  

Furthermore, numerous HEIs and research organisations across 
Europe offer their own initiatives to support Seal of Excellence 
recipients. These initiatives can be found in Austria, Italy and 
Spain. Recipients of the COFUND Seal of Excellence from the 
2021-2027 national programming period are already being 
supported by Czechia under a national call launched in 2022, with 
additional calls foreseen for consecutive years.  

962 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

To better guide Member States with regard to the available 
opportunities to establish a national or regional level scheme to 
fund the Seal of Excellence recipients, the European Commission 
has also issued a Commission Notice on Synergies between 
Horizon Europe and ERDF programmes. This document outlines 
the new opportunities available to the managing authorities of 
Cohesion Policy programmes, Horizon Europe NCPs and Horizon 
Europe project promoters/proposers. The document is intended to 
make it easier to understand how such a scheme can be 
established and operationalised, taking stock of national or 
regional programmes supported by the ERDF. 

WIDERA 

Widening: Under Horizon Europe, Seals of Excellence are 
awarded to Teaming applications that exceed all evaluation 
thresholds but cannot be funded due to a lack of budget. At the 
end of 2022, the European Commission awarded 11 Seal of 
Excellence certificates to proposals submitted under the Horizon 
Europe Widening call ‘Teaming for Excellence’.1333 Interviews with 
national-level stakeholders from widening countries reveal that 
some of them fund (to the extent possible) projects that have 
received a Seal of Excellence. Two unsuccessful WIDERA 
Widening applicants who responded to the survey indicated that 
there should be clearer communication regarding the Seal of 
Excellence, and that national stakeholders should be further 
encouraged to fund such projects (WIDERA-WIDE-APPL-030, 
Survey of Horizon Europe unsuccessful applicants, conducted 
May-July 2023).  

1333 https://research-and-innovation.ec.europa.eu/news/all-research-and-innovation-news/commission-awards-seal-excellence-outstanding-teaming-
excellence-proposals-2022-12-19_en 

963 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

EFC12 

Number/extent of results shared.  

Channels through which results are 
shared. 

Enablers/drivers versus barriers, and 
solutions or further support to 
encourage these. 

Literature 
review 

Monitoring 
data 

Interviews 

Case studies 

Surveys 

How have different 
types of results 
(e.g. software, 
knowledge, data) 
been made 
available for the 
research 
community, 
innovation and 
policy actors, and 
general public? 
How can this be 
improved? 

964 

ERA: Analysis of administrative data shows that none of the 
proposals under WIDERA (ERA destination) received the Seal of 
Excellence. 

Horizon Europe: See the answer to question EFC3 above. 

ERC: According to the External Communication Strategy 2023, 
the ERC’s communication tools and channels are its website 
(which attracted around one million visits in 2022); multimedia 
(video productions, infographics and podcasts); social media 
(including Twitter and LinkedIn); the ERC online magazine; email 
alerts; and publications (with a focus on reader-friendly online 
formats). In addition, the ERC is frequently mentioned in the news, 
and plans to explore collaborations with science TV shows, for 
example. Lastly, NCPs, host institutions, representations and the 
European Parliament are also channels of communication. For 
more information, see the answer to question EFC3. 

MSCA: The MSCA is relevant in its aim to bridge the gap between 
research and the general public, increasing awareness of R&I 
activities and increasing the interest of young people in research 
and scientific careers. The MSCA and Citizens action plays an 
important role in this regard. MSCA and Citizens targets a broad 
audience, irrespective of their scientific background, with a 
particular emphasis on families, students and those who may face 
barriers to engaging with STEAM fields (science, technology, 
engineering, arts, and mathematics) or research endeavours. 
Inclusivity is a central principle, as MSCA and Citizens aims to 
democratise access to science and research for everyone. MSCA 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

and Citizens also promotes gender balance and inclusiveness in 
Open Science and responsible research and innovation.  

The MSCA achieve these goals by organising Researchers’ Night, 
the largest research communication and promotion event across 
EU Member States and Horizon Europe associated countries, and 
the implementation of the Researchers at School initiative. 

INFRA: As yet there is no evidence available regarding the 
dissemination of the results of Horizon Europe projects. The 
evaluation study on Excellent Science under Horizon 2020 found 
that the median INFRA project allocates 1.2% of its budget to 
dissemination and communication – double the share across 
Horizon 2020 as a whole, indicating how important communication 
and dissemination is to INFRA. The main audience for these 
activities (as found in the evaluation study on Excellent Science 
under Horizon 2020) is the science community, followed by 
industry, policy-makers and the general public. Some of the most 
common platforms for communication and dissemination identified 
in the above evaluation study are the website, scientific 
publications, open access repositories, social media, workshops 
and conferences.  

WIDERA:  

Widening: As mentioned above in the Widening actions, efforts 
have been undertaken in the form of new actions to further include 
these quadruple helix actors. The survey shows that 60.7% of 
Widening beneficiary respondents (102 out of 168) indicated that 
civil society representatives and/or governmental institutions play 
a role as stakeholders in the execution of their project (Survey of 
Horizon Europe beneficiaries, conducted May-July 2023). This is 

965 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

most often as an informant (45.8%; 77 out of 168), followed by 
project partner (21.4%; 36 out of 168) (WIDERA-WIDE-BEN-040).  

More information on outreach and communication is provided in 
the answer to question EFC3.1.  

In addition, a call on Dissemination and Exploitation Support 
activities has been developed under Widening. This call is open to 
Widening participants from Horizon 2020 and Horizon Europe, and 
aims to scale up research results and improve the sustainability of 
actions (e.g. translating innovations into new products, feeding 
results into policy-making). Projects under the first call (HORIZON-
WIDERA-2023-ACCESS-05) began in June 2023.  

ERA: The available evidence indicates that dissemination and 
communication activities have been a high priority for the projects 
supported under the ERA priority, particularly those projects in the 
field of acceleration services/university transformation, where 
wider structural impacts on the ERA are highly dependent on 
effective dissemination and communication strategies in order to 
reach the target groups outside the project consortium. In addition 
to the traditional dissemination measures and instruments that 
projects have used/intend to use (websites, conferences, public 
speeches, academic publications, newsletters, press releases), 
some projects have applied innovative tools to disseminate project 
outputs and results to wider publics. For example, in the case of 
the RM Roadmap project, the project consortium involved a 
partner – Crowdhelix Open Innovation platform – to facilitate and 
boost its dissemination strategy. Crowdhelix uses specialised 
software known as an ‘intelligent recommender system’, which 
allows the project consortium to match specific project outcomes 
and dissemination items with specific target groups by using 
natural language processing and machine learning. Using this 

966 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

software makes it easier for stakeholders to access the 
information that is the most relevant to them, and streamlines the 
dissemination process for the project. 

Some projects have aimed to amplify and increase the impact of 
their results by organising a dedicated dissemination event. For 
example, in the case of the European City of Science (Leiden 
2022), a legacy session was organised after the project ended, 
during which the project stakeholders discussed all of the 
significant events and achievements of the project. 

To what extent 
have the projects 
selected 
contributed to: 
• the improvement 
of the ‘ethics by 
design approach’ in 
research ethics 
and integrity ethics 
appraisal process 
and guidance 
(including at 
national, regional 
and institutional 
level)? 

EFC13.1* 

The extent of implementation of 
inspirational frameworks (C&C, 
research careers framework, HRS4R, 
etc.) 

Qualitative information about impact in 
terms of inspiring national evaluation 
systems/criteria for funding, among 
them gender equality plans, 
intersectoral mobility frameworks and 
research integrity frameworks. 

Enablers/drivers versus barriers; and 
solutions or further support to 
encourage these. 

Literature 
review 

Monitoring 
data 

Interviews 

Case studies 

Surveys 

ERC: Given its nature, the ERC contributes to cross-cutting issues 
such as ethics and research integrity through, on the one hand, 
funding bottom-up research on these topics, and on the other 
hand, through the development of individual researchers’ skills in 
these areas. To illustrate the former, data from the Horizon 
Dashboard (accessed on 10 September 2023) shows five signed 
ERC grants (for the projects MORE, NOTCOM, REASONF1RST, 
Abortion Figurations and PatentsInHumans) on the topic of ethics 
principles (compared with seven signed grants under Horizon 
2020).  

With respect to skills development, 43% of survey respondents (99 
out of 233) stated that the ERC had contributed to a ‘large’ or ‘very 
large’ extent to developing their skills in research ethics and 
integrity. Around half (115 out 237) of ERC beneficiaries (ERC-
BEN-042) reported no issues and a positive experience with the 
ethics self-assessment of their ERC project, while 21% (50 out of 
237) reported only minor issues. 62% of ERC beneficiaries (115 
out of 187; ERC-BEN-043) considered the support available to 

967 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

carry out the ethics self-assessment of the ERC project (e.g. 
information, guidance) to be excellent or good.  

MSCA: Survey findings indicate the experiences of successful 
MSCA applicant organisations and MSCA PF applicants’ with 
ethics self-assessment in their projects. Almost half of successful 
MSCA applicant organisations (48%) and more than half of 
successful MSCA PF applicants (56%) reported that their 
experiences were, ‘to a large extent’, positive, indicating that they 
faced no issues in the ethics self-assessment. This high 
percentage of positive feedback suggests that the EU's process 
for ethics self-assessment is generally clear, accessible and 
manageable for most applicants, enabling the successful 
promotion of high standards of ethical conduct in research and 
innovation. In addition, the majority of successful MSCA applicant 
organisations surveyed indicated that the support available in 
carrying out the ethics self-assessment was excellent or good 
(73%, or 245 out of 340). Similar findings can be seen for 
successful MSCA PF applicants (78%, or 469 out of 599).  

Furthermore, with respect to skills development, 67% of survey 
respondents (497 out of 744) stated that the MSCA contributed to 
a ‘large’ or ‘very large’ extent to developing their skills in research 
ethics and integrity. 

INFRA: Under the INFRA work programmes, limited mention is 
made of ethics (only nine mentions in in WP 21-22, and six 
mentions in WP 23-24). When ethics is mentioned, this is mostly in 
relation to data management. Of the INFRA beneficiaries 
surveyed, only one out of 153 respondents (0.7%) expressed 
negative opinions about the ethics self-assessment in their project; 
43.1% (66 out of 153) were positive to a ‘large’ extent, and 26.8% 
(41 out of 153) were positive to a ‘moderate’ extent; the remaining 

968 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

29.4% (45 out of 153) indicated that they did not know or that the 
question was not applicable (INFRA-BEN-043). In addition, the 
majority of INFRA beneficiaries surveyed indicated that the 
support available when carrying out the ethics self-assessment 
was excellent or good (61.9%, or 65 out of 105). Only a very small 
minority indicated that it was poor (2.9%, or 3 out of 105) (INFRA-
BEN-044). Lastly, around half of INFRA beneficiaries surveyed 
found that the instructions for the requested ethical review were 
clear to a ‘large’ or ‘very large’ extent (51.9%, or 94 out of 181) 
(INFRA-BEN-006).  

WIDERA: 

Widening: While the activities undertaken under Widening should 
respect ethical principles, there is no concrete mention of 
objectives relating to adopting an ‘ethics by design’ approach, or of 
‘research integrity’. In addition, there was no specific indicator of 
the impact of Widening on these topics. Nevertheless, Case Study 
7 (Annex 2) outlines that the creation or revision of guidelines and 
procedures related to ethics at institutional or regional/national 
level is one of the structural changes that is sometimes triggered 
by Widening projects aimed at fostering research excellence and 
collaboration. Specifically, based on the results of the survey of 
successful SEWP beneficiary organisations carried out for the 
Evaluation study on Excellent Science under Horizon 2020, the 
case study outlines that slightly fewer than half of the survey 
participants 36 out of 81) acknowledged that their Widening 
project had influenced (or would influence) the development or 
improvement of standards or appraisal processes in relation to 
research ethics and integrity in the host institution. 

ERA: RRI and research ethics are among the most widely covered 
topics in ERA projects. Based on the Horizon Dashboard and an 

969 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

To what extent 
have the projects 
selected 
contributed to: 
• the development 
of inclusive gender 
equality policies at 
national level, and 
opening to 
intersectionality, in 
line with the new 
ERA 
Communication? 
How could 
effectiveness be 
increased in this 
regard? 

EFC13.2* 

The extent to which inspirational 
frameworks (C&C, research careers 
framework, HRS4R, etc.) have been 
implemented. 

Qualitative information about impact in 
terms of inspiring national evaluation 
systems/criteria for funding, among 
which are gender equality plans, 
intersectoral mobility frameworks, 
research integrity frameworks. 

Enablers/drivers versus barriers; and 
solutions or further support to 
encourage these. 

Literature 
review 

Monitoring 
data 

Interviews 

Case studies 

Surveys 

970 

analysis of thematic coverage/keywords, 9% of participations in 
ERA projects ((153 participations) reported the keywords 
‘Responsible Research and Innovation (RRI)’, while another 71 
participations mentioned ‘Ethics in research and innovation’.  

ERC: Similar to the question before, data from the Horizon 
Dashboard (accessed on 10 September 2023) shows two signed 
ERC grants (the projects BILQIS and F-WORD) on the topic of 
‘gender studies’ (compared with 22 signed grants under Horizon 
2020). However, data from the ERC ERIS tool show a longer list of 
projects addressing ‘gender studies’. 

In addition, only 39.8% of successful ERC applicants who 
responded to the survey (100 out of 251) were satisfied to a ‘large’ 
or ‘very large’ extent with the clarity of the instructions relating to 
the requested gender equality plans (ERC-BEN-010). 

INFRA: According to Case Study 12 on gender equality, INFRA 
has a slightly lower share of female researchers compared with 
the Horizon Europe average (25.8% compared with 31.5% 
overall). However, Case Study 12 also shows that INFRA has a 
relatively high share of projects flagged as covering gender-related 
topics (52% of projects, or 42 out of 81, compared with only 5% 
across Horizon Europe as a whole). Furthermore, the majority of 
INFRA beneficiaries surveyed said that the guidelines for the 
requested gender equality plan were clear (59.6%, or 109 out of 
183) (INFRA-BEN-006).  

WIDERA: 

Widening: Case Study 7 (see Annex 2) outlines that endorsing 
and applying the EU Charter for Researchers and the Code of 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

Conduct for the Recruitment of Researchers, or principles of these 
such as gender balance are some of the structural changes that 
are sometimes triggered by Widening projects to improve working 
conditions. The case study outlines (based on the results of a 
survey of successful SEWP beneficiary organisations carried out 
for the Evaluation study on Excellent Science under Horizon 2020 
outlines that around half of respondents (39 out of 81) indicated 
that their Horizon 2020 Widening project had resulted (or will 
result) in advances towards the achievement of gender equal 
opportunities through gender mainstreaming or gender equality 
policies within their institutions. 

In addition, 63.3% (122 out of 193) of successful WIDERA 
Widening applicants who responded to the relevant survey for the 
present evaluation are to a ‘large’ or ‘very large’ extent satisfied 
with the clarity of the instructions relating to the requested gender 
equality plans (WIDERA-WIDE-BEN-006, Survey of Horizon 
Europe beneficiaries, conducted May-July 2023).  

ERA: Gender was one of the most widely covered topic areas in 
ERA projects. According to the analysis of topics based on 
Horizon Dashboard data (September 2023), there have been 107 
participations (6% of all participations) with the keyword ‘gender 
equality’; 47 participations with the keywords ‘gender research’; 41 
participations with the keywords ‘sexual orientation’; and 17 
participations with the phrase ‘gender in innovation development’. 

To what extent 
have the projects 
selected 
contributed to: 

EFC13.3* 

The extent to which inspirational 
frameworks (C&C, research careers 

Literature 
review 

ERC: See answer to question EFC 13.1. 

971 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

• the promotion of 
research integrity 
(procedures, 
training, education) 
and catalysing 
relevant 
frameworks and 
approaches at 
national, regional 
and institutional 
level? 

framework, HRS4R, etc.) have been 
implemented. 

Qualitative information on impact in 
terms of inspiring national evaluation 
systems/criteria for funding, among 
which gender equality plans, 
intersectoral mobility frameworks, 
research integrity frameworks. 

Enablers/drivers versus barriers; and 
solutions or further support to 
encourage these. 

Methods and 
analyses 

Monitoring 
data 

Interviews 

Case studies 

Surveys 

Key findings 

Evidence 

INFRA: In the INFRA work programmes, there is no mention of 
‘research integrity’. There is no evidence that INFRA contributes to 
the promotion of research integrity.  

WIDERA: 

Widening: This response is also mentioned under EFC13.1. While 
the Widening activities undertaken should respect ethical 
principles, there is no concrete mention of objectives relating to 
the adoption of an ‘ethics by design’ approach, or of ‘research 
integrity’. In addition, there is no specific indicator for the impact of 
Widening actions on these topics. Nevertheless, Case Study 7 
outlines that the creation or revision of guidelines and procedures 
relating to ethics at institutional or regional/national level is one of 
the structural changes that is sometimes triggered by Widening 
projects aimed at fostering research excellence and collaboration. 
Specifically, the case study outlines (on the basis of the results of 
the survey of successful SEWP beneficiary organisations carried 
out for the Evaluation study on Excellent Science under Horizon 
2020 that slightly fewer than half of survey participants (36 out of 
81) acknowledged that their Horizon 2020 Widening project had 
influenced (or would influence) the development or improvement 
of standards or appraisal processes in research ethics and 
integrity in the host institution. 

ERA: As mentioned above, RRIs and research ethics are among 
the topics most widely covered in ERA projects. Based on the 
Horizon Dashboard and an analysis of thematic 
coverage/keywords, 9% of participations in ERA projects (153 
participations) reported the keywords ‘Responsible Research and 
Innovation (RRI)’, while another 71 participations mentioned 
’Ethics in research and innovation’. Case study evidence shows 
that structural impacts in ethic/integrity are expected to be 

972 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

EFC14* 

What is the impact 
of Framework 
Programme actions 
supporting Open 
Science, citizen 
science, gender 
equality, 
international 
cooperation, ethics 
and integrity, and 
scientific input into 
other EU policies? 

Related to the evaluation questions 
EFC9, EFC13 above, e.g. with regard 
to their inspiring role, and the 
implementation of plans/guidelines at 
national, regional or institutional levels. 

Evidence of increases in Open 
Science, citizen science, gender 
equality, international cooperation, the 
implementation of ethics and integrity 
plans, etc.  

Literature 
review  

Interviews 

Case studies 

Surveys 

973 

achieved through the institutional transformation and acceleration 
services that are supported under the programme. 

ERC: See answer to questions EFC 13.1 and 13.2 and the case 
studies on gender equality, ethics and research integrity. 

MSCA: The MSCA work programmes under Horizon Europe 
encourage applicants to engage with society at large in order to 
make the results of their research visible to citizens, and to involve 
citizens, civil society and end-users in the co-creation of research 
content where relevant. Both the Doctoral Networks and 
Postdoctoral Fellowships underline the importance of preparing 
the young researchers, through training activities, for the 
increased research collaboration and information-sharing made 
possible by new (digital) technologies. Most importantly, the 
objective of the MSCA and Citizens action is to bring research and 
researchers closer to the public at large through the organisation 
of European Researchers’ Night and Researchers at Schools 
activities. In addition, several supporting actions focus on aspects 
of societal engagement. For example, ‘MSCA Feedback to Policy 
2023’ calls for analyses and reports to look into trends and 
developments in the MSCA that are relevant to the EU Missions, 
including citizen engagement activities. Lastly, MSCA-NET 
prepared a policy brief on Open Science in 2023, which discusses 
societal engagement practices in addition to other key pillars of 
Open Science. The related policy brief provides guidelines for the 
network of MSCA National Contact Points, who spread the 
information for applicants across the European Research Area 
and beyond. 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

The study found that slightly more than one-third of successful 
MSCA applicant organisations (38%) and successful MSCA PF 
applicants (39%) plan to engage citizens’ representatives/end-
users in the co-creation of R&I content in their MSCA projects. Out 
of those who plan to do so, a significant share of successful 
applicant organisations and applicants (55%, respectively) plan to 
actively involve citizens or end-users in co-design activities. These 
typically involve organising workshops, focus groups, or similar 
approaches to collaboratively develop R&I agendas, roadmaps 
and policies. 

In terms of the impact of the MSCA on EU policies, the survey 
results indicate that this impact could be further strengthened. 
Less than half of the MSCA beneficiary organisations surveyed 
(26%, or 130 out of 500) and MSCA PF fellows (24%, or 200 out 
of 830) say their project is likely to lead to recommendations for 
policy-makers or inputs into national or EU policies. MSCA 
recipients also believe that the MSCA do not provide sufficient 
funding opportunities for research that contributes to policy-making 
(36%, or 294 out of 818 for MSCA PF fellows; 32% or 157 out of 
490 beneficiary organisations). 

INFRA: INFRA does not have a specific objective to support 
citizen science, gender equality or ethics and integrity (see also 
EFC13.1, EFC13.2 and EFC13.3). Only a minority of INFRA 
beneficiaries surveyed (38.80%, or 59 out of 152) indicated that 
they plan to engage citizens, citizens’ representatives or end-users 
in the co-creation of R&I content in their project (INFRA-BEN-037); 
mostly, they indicate the use of co-design activities (such as 
workshops or focus groups), followed by co-assessment and co-
creation (INFRA-BEN-038).   

974 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

However, INFRA does contribute significantly to the objective of 
Open Science, mainly through its support for the EOSC and by 
providing access to RIs. Interviewees mentioned that INFRA 
support for the EOSC could be more streamlined and 
synchronised. Given the large number of projects running in 
parallel with limited coordination between projects, interviewees 
indicated that this is likely to lead to a duplication of efforts and 
unexploited synergies.  

Regarding the impact of INFRA on international cooperation, 
please refer to questions RV6 and EFC6.1.  

The survey results indicate that the impact of INFRA on EU 
policies could be further strengthened. Around half of INFRA 
beneficiaries surveyed (50.6%, or 83 out of 164) say their project 
is likely to lead to recommendations for policy-makers or inputs 
into national or EU policies (INFRA-BEN-027). However, only a 
minority of respondents (32.90%, or 54 out of 164) indicated that 
there is enough research under INFRA that contributes to policy-
making (INFRA-BEN-024). Furthermore, only 29.8% of 
beneficiaries (48 out of 161) say their project supports policy-
making or the development of standards-setting measures 
(INFRA-BEN-026). 

WIDERA: 

Widening: As mentioned under EFC13.1, EFC13.2, EFC13.3 and 
EAV2, the Widening actions generally support the development of 
gender equality principles, and the promotion of research integrity 
in the sense that many Widening countries and institutions have 
aligned their policies with European guidelines. As mentioned 
previously, the Widening actions also stimulate international 
cooperation in even more ways than under Horizon 2020, with 

975 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

new actions under Horizon Europe (e.g. the Excellence Hubs and 
European Excellence Initiative (EEI), which strengthen capacity for 
excellence in HEIs and their surrounding ecosystems). With 
regard to Open Science, Case Study 7 (based on results of the 
survey successful SEWP beneficiary organisations carried out 
under the Evaluation study on Excellent Science under Horizon 
2020) shows that a majority of approximately three-quarters of 
respondents (55 out of 75), strongly agree or rather agreed that 
the Widening projects had effectively enhanced their institution's 
competence in implementing Open Science practices.  

ERA: Study evidence shows that ERA projects, most of which 
focus on or cover at least some CCIs (gender, Open Science, 
RRIs, etc.) will have the impact on EU policies mainly through the 
dissemination of knowledge and new standards in the form of 
roadmaps, education materials and policy recommendations. 
According to the results of the Horizon Europe beneficiaries’ 
survey, a large majority of WIDERA (ERA) beneficiaries (82%, or 
86 out of 105) indicated that their project had resulted in/is likely to 
result in recommendations for policy-makers/inputs into national or 
EU policies, followed by educational materials (63%, or 66 out of 
105 respondents), and training programmes (62%, or 65 out of 
105 respondents). Similarly, significant shares of respondents 
indicated that their project had resulted in or is likely to result in 
research roadmaps (39% or 41 out of 105), social innovations 
such as tools, guides, strategies (35%, or 37 out of 105), new or 
improved tools, methods or techniques (31%, or 33 out of 105), 
and codes of conduct (16%, or 17 out of 105). 

976 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Level of 
participation of the 
ERIC legal entities 
(European 
Research 
Infrastructure 
Consortium) and 
efficiency of the 
process to 
participate. 

EFC15 

Number of ERIC legal entities.  

Participation of ERIC legal entities in 
Horizon Europe. 

The extent to which applicants agree 
that the requirements and process for 
participation are clear, reasonable, 
proportionate and timely. 

Methods and 
analyses 

Literature 
review 

Monitoring 
data 

Interviews 

Case studies 

Key findings 

Evidence 

INFRA: INFRA plays an important role in the development and 
operation of ERICs. The ERIC Regulation facilitates new 
European RIs with members from different countries, and is 
regarded as a ‘highly successful mechanism to initiate and 
integrate many RIs at European level, aligning national 
investments and research priorities, pooling resources and 
expertise’1334. A significant portion of the ESFRI landmarks (60%) 
are set up as ERICs. Since its creation in 2009, a total of 26 
ERICs have been established1335, and at least another seven 
ERIC applications are in the pipeline. On average, each ERIC has 
14 members, with some ERICs spanning almost the entire EU. 
Furthermore, every Member State is a member of at least one 
ERIC, and nine associated countries are also members of an 
ERIC. 

Under Horizon Europe, there have been 113 participations by 
ERICs, of which 10 have been in the role of coordinator (Horizon 
Dashboard, accessed in July 2023). Under Horizon 2020, there 
were 279 participations by ERICs, of which 32 were as a 
coordinator (see the evaluation study on Excellent Science under 
Horizon 2020). Under Horizon Europe, the most popular 
programme part for ERIC participations was INFRA (82 
participations), followed by Cluster 6 ‘Food, Bioeconomy Natural 
Resources, Agriculture and Environment’ (13 participations), and 
Cluster 1 ‘Health’ (11 participations). Within INFRA, ERICs mostly 
participate in INFRASERV (26% of all participations in INFRA by 

1334 Report from the Commission to the Council and the European Parliament, Third report on the application of Council Regulation (EC) No 723/2009 
of 25 June 2009 on the Community legal framework for a European Research Infrastructure Consortium (ERIC), 2023, available at: https://eur-
lex.europa.eu/legal-content/EN/TXT/?uri=COM:2023:488:FIN. 
1335 Ibid.  

977 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

EFC16.1 

The extent to which the Member 
States and associated countries have 
implemented the ESFRI Roadmap.  

Literature 
review 

Interviews 

Case studies 

To what extent has 
the Framework 
Programme 
contributed to: 
• facilitate the work 
of ESFRI 
(European Strategy 
Forum on 
Research 
Infrastructures) and 
foster the 
implementation by 
Member States 
and associated 
countries of the 
ESFRI Roadmaps? 

ERICs) and INFRATECH (24% of all participations by ERICs in 
INFRA).  

INFRA: INFRA Work Programmes under Horizon Europe 
contribute significantly to the development of projects and 
landmarks included in the ESFRI Roadmap, notably through the 
INFRADEV destination, which funds design studies, preparatory 
phase projects and implementation projects; each related to 
another lifecycle phase of ESFR RIs. In addition, the INFRATECH 
destination is important to ensuring ESFRI RIs remain at the state-
of-the-art. INFRAEOSC also directs funding for ESFRIs, notably 
for the creation of FAIR data and its integration into the EOSC. In 
particular, the Cluster Projects under Horizon 2020, which brought 
together ESFRI RIs and other world-class RIs within the same 
discipline, have been considered especially effective (see the 
evaluation study on Excellent Science under Horizon 2020 and 
Case Study 6 of the present study). Under Horizon Europe, the 
Science Clusters will come together in a ‘cluster-of-clusters’1336 
project that aims to foster integration between the clusters. Some 
interviewees indicate, however, that further dedicated funding for 
each Cluster would be beneficial. Moreover, this programme part 
is aligned with the needs reflected in the ESFRI Landscape 
Analysis. Its limited budget implies a limited selection of topics (i.e. 
not all topics identified as needs can be supported through EC 
funding). 

1336 HORIZON-INFRA-2023-EOSC-01-01 - Build on the science cluster approach to ensure the uptake of EOSC by research infrastructures and 
research communities. Funded under: https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/topic-details/horizon-infra-
2023-eosc-01-01;callCode=null;freeTextSearchKeyword=;matchWholeText=true;typeCodes=1 

978 

 
 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

To what extent has 
the Framework 
Programme 
contributed to: 
• Increase access 
to research 
infrastructures? 

To what extent has 
the Framework 
Programme 
contributed to: 
• Career evolution 
of researchers 
thanks to access to 
research 
infrastructures 
('TNA generation' 
)? 

To what extent has 
the general bottom-
up approach for 
access provision to 

EFC16.2 

Analysis of current access to research 
infrastructures. 

EFC16.3 

Qualitative information about impact 
on the career evolution of researchers. 

INFRA: With regard to access to infrastructures, during Horizon 
2020, INFRA actions significantly increased transnational and 
virtual access (TNA/VA) to and between RIs. Data received from 
the European Commission in October 2022 regarding the number 
of instances of transnational or virtual access shows that 16,712 
unique researchers had received access to RIs through the INFRA 
programme so far (see the evaluation study on Excellent Science 
under Horizon 2020). Projects under Horizon Europe have not yet 
reported figures on TNA/VA in their interim reports. 

INFRA: Interviews within the scope of both the evaluation study on 
Excellent Science under Horizon 2020 and the present study 
indicate that TNA/VA significantly advances the career prospects 
of the researchers involved. TNA/VA enables the exchange of 
knowledge between researchers; and allows them to apply and 
test their methods in a different environment; it allows them to 
learn new scientific methods and receive training on how to use 
the RI and the related services they access. It also improves their 
chances of publishing in peer-reviewed journals. Its positive effect 
on career development is not directly measurable, however, as the 
current reporting information does not allow the systematic 
analysis of the impact of TNA/VA on the career development of 
the researchers involved. 

EFC17.1 

The extent to which the approach to 
access provision has influenced 

Literature 
review 

INFRA: Case Study 6 of the present study provides an in-depth 
analysis of the impact of the new challenge-oriented approach to 
the provision of access under Horizon Europe (compared with the 
bottom-up approach under Horizon 2020). Its main conclusion is 

979 

 
 
 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Research 
Infrastructures 
influenced 
research in 
addressing global 
challenges and EU 
policy priorities? 
How effective is the 
new challenge-
driven approach in 
that regard? 

What has been the 
impact on 
effectiveness of 
R&I processes and 
outcomes of the 
discontinuation of a 
dedicated science-
society programme 
part (e.g. FP6 
Science and 
Society, FP7 
Science in Society, 

Methods and 
analyses 

Interviews 

Case studies 

research addressing global challenges 
and EU policy priorities. 

Examples of the influence of the 
challenge-driven approach on this 
bottom-up approach. 

EFC17.2 

Qualitative assessment of impact and 
lessons learned; gap analysis. 

Literature 
review 

Interviews 

980 

Key findings 

Evidence 

that there is a need for a clear long-term vision on the provision of 
access to RIs. Under Horizon 2020, the general bottom-up 
approach enabled a level of continuity for user communities, which 
were formed around similar scientific disciplines and RI 
instruments. In Horizon Europe, RIs have been asked to form 
larger, interdisciplinary consortia that come together around 
societal challenges. While interviewees are generally positive 
about the increased contribution RIs make to societal challenges, 
significant transition costs have been incurred by RIs in switching 
to this new approach, and there are concerns about the 
sustainability of the services developed beyond the project 
lifetime. Interviewees therefore ask for a strategic, roadmap-based 
approach to provide a stable framework for the development of 
RIs and the provision of access. Furthermore, the increase in the 
sizes of consortia has resulted in greater complexity and the 
dilution of funding, potentially reducing the effectiveness of these 
projects.  

ERA: Qualitative assessment shows that there is high level of 
continuity between the activities funded under Horizon 2020 
SwafS and Horizon Europe ERA, both in terms of thematic 
coverage and the types of activities supported, e.g. bringing 
science closer to society, support for responsible research and 
innovation (RRI), ethics and integrity, gender equality in research 
etc, as well as some of the projects funded under SwafS (e.g. 
EUCYS, the EU Contest for Young Scientists). 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

Horizon 2020 
Science with and 
for Society)? What 
are the lessons to 
be drawn from 
this? 

To what extent 
have actions 
implemented under 
the Framework 
Programme to 
strengthen the 
European 
Research Area at 
national and 
institutional level 
(in open science, 
Responsible 
Research and 
Innovation, citizen 
science, gender 
equality, 
international 
cooperation, ethics 
and integrity) 
contributed to the 
attainment of key 
EU values and 
principles (as laid 

EFC18 

The extent to which inspirational 
frameworks and, more specifically, key 
EU values and principles, have been 
implemented. 

Literature 
review 

Interviews 

Case studies 

Qualitative information regarding 
impact in terms of inspiring national 
and institutional-level measures, e.g. 
with respect to the implementation of 
gender equality plans, research 
integrity frameworks, international and 
intersectoral cooperation, Open 
Science, etc. 

ERC: See answers to questions EFC 13.1 and 13.2 and the case 
studies on gender equality, ethics and research integrity. 

One beneficiary mentioned that they had hired an expert on ethics 
and research integrity to advise their project on those topics. In 
terms of the GEP requirement, this is a positive step, but lacks an 
efficient implementation and monitoring system. The gender 
dimension should be integrated into research, but more 
information should be made available about this.  

MSCA: Interview findings indicate that the MSCA contribute 
strongly to Open Science among the participating institutions. 
Participation in the MSCA contributes towards normalising Open 
Science practices at institutional level, and strengthens national 
Open Science strategies in those countries where such strategies 
already exist. Interviewees view Open Science as a leading 
example of a European value that is implemented successfully in 
and through the MSCA. This process is considered an example of 
how other values (e.g. sustainability) could be implemented as 
part of the MSCA application process.  

The MSCA have a pronounced structuring impact on participating 
organisations, and bring added value by promoting gender 
equality. In addition, the MSCA have structuring impacts that go 

981 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

down i.e. by the EU 
Charter of 
Fundamental 
Rights and EU 
treaty)? 

beyond MSCA projects and are spread /institutionalised within 
beneficiary organisations and national/regional systems. This is 
particularly evident in relation to gender equality policies, which 
are becoming institutionalised due to participation in MSCA 
actions. 

The MSCA also strongly promote international cooperation, as 
well as cross-sectoral and cross-border mobility. These impacts 
can involve specific forms of cooperation. For example, the MSCA 
Green Charter has encouraged the creation of a pan-European 
green laboratory, contributing to the development of an 
interdisciplinary network of researchers, all of whom are interested 
in improving their sustainable research practices. 

INFRA: As already discussed in the answers to EFC14 and 
EFC13, INFRA does not have a specific objective to support 
citizen science, gender equality or RRI. Engagement with 
citizens/citizen representatives/end-users is limited to a minority of 
INFRA projects (INFRA-BEN-037). However, INFRA does 
contribute significantly to the objective of Open Science, mainly 
through its support for the EOSC and by providing access to RIs. 
For the impact of INFRA on international cooperation, please refer 
to questions RV6 and EFC6.1.  

WIDERA: 

Widening: By triggering structural changes at institutional and 
regional/national levels as a result of Widening projects, several 
widening countries and institutions have aligned their policies with 
European guidelines (although, of course, other EU projects and 
practices also trigger such changes). Case Study 7 (see Annex 2) 
outlines that such structural changes can, among other things, be 
aimed at improving working conditions, career progression and 

982 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Methods and 
analyses 

Evidence 

skills (such as endorsing and applying the EU Charter for 
Researchers and the Code of Conduct for the Recruitment of 
Researchers or one of its principles, such as gender balance) or 
by fostering research excellence and collaboration (such as 
developing or revising guidelines on Open Science practices or 
research ethics). The above case study finds that the Widening 
actions often trigger structural changes at institutional level, and 
the evidence points to a greater number of changes occurring at 
institutional level than at regional or national levels. The 
mechanisms through which such changes occur at regional or 
national level usually relate to a project’s design and objectives, 
and are also associated with the degree of political involvement in 
the project. The case study also recommends that a more 
consolidated and robust monitoring approach would facilitate the 
monitoring of the progress made by the actions. 

ERA: As already mentioned, WIDERA actions (ERA) will 
contribute directly to the key strategic priorities of strengthening 
the European Research Area at national and institutional level. 
Preliminary desk research shows that the topics covered under 
WIDERA (ERA) actions generally included responsible research 
and innovation (RRI), EU research policy, gender equality, 
inclusion, STEM, ethics and responsible research. 

Horizon Europe: There have been no ERA-NETs under Horizon 
Europe1337. 

To what extent 
have ERA-Nets 
contributed to 
strengthening the 

EFC19 

The extent to which ERA-Nets have 
facilitated particular partnerships. 

1337 Source: https://www.era-learn.eu/  

Literature 
review 

Interviews 

983 

 
 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

European 
Research Area? 

Qualitative information on the 
contribution ERA-Nets have made to 
transnational cooperation and 
competition (e.g. effective national 
research systems, researcher mobility, 
gender equality, open access and 
international cooperation). 

1.5.  EU added value 

Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

What is the EU 
added value of the 
Framework 
Programme 
interventions, 
including of the 
different 
instruments 
mobilised (such as 

EAV1.1 

Types of value generated by projects 
analysed, as opposed to at national 
level (e.g. access to skills, improving 
research capacities, leverage of private 
and public investments, better access to 
infrastructures, pooling of research, 
reducing commercial and/or research 
risks, faster implementation, 
interdisciplinarity, improved 

Key findings 

Evidence 

Methods and 
analyses 

Case studies 

Key findings 

Evidence 

ERC: Investing in discovery-driven research at EU level is 
considered essential to positioning Europe at the forefront of 
excellent science. The analysis and examples in Case Studies 1 and 
2, for instance, demonstrate the importance of supporting prominent 
fields at an early stage in their development, not only to be at the 
forefront of scientific research in the world, but also to create 
relevant and high-quality knowledge and insights by the time the 
topic gains societal interest. Interviewees also stress the importance 
of the sole selection criterion of excellence for truly bottom-up 

Methods 
and 
analyses 

Literature 
review 

Interviews 

Case 
studies 

Surveys 

984 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Evidence 

Methods 
and 
analyses 

partnerships), to 
deliver excellent 
science? 

mobility/international career prospects 
of researchers, etc.). 

Key reasons why projects could not be 
implemented at national level (e.g. not 
being able to address pan-European 
issues, not finding alternative funding, 
insufficient scale/scope of funding 
instruments at national level). 

Examples of exceptional EAV of 
projects funded under the Excellent 
Science pillar. 

research: high-risk/high-gain research needs to allow for unexpected 
outcomes or for failure. The ERC is considered the main funding 
instrument in Europe for funding bottom-up fundamental research, 
and its grants are considered both more substantial and more 
prestigious by the research community than national funding 
opportunities. ERC grants offer researchers the possibility to gain 
funding for high-risk/high-gain research projects of sufficient scale to 
be relevant, to combine complementary research steps or expertise 
into a single project, and to generate impact in research and beyond 
without the prerequisite to define these impacts ex-ante. As an 
alternative, national funding opportunities do not always offer this 
same degree of excellence-driven research funding, nor the same 
scale of funding within a single grant. Interviewees point to these 
elements as important factors in the added value of the EU-level 
approach. 

The survey of successful ERC beneficiaries provides further insights 
into the differences between Horizon research funding and national 
or regional funding (ERC-BEN-044). For example, 93% of 
respondents (215 out of 232) strongly agreed or rather agreed that 
the Horizon research funding provides a higher amount of funding, 
while 89% (207 out of 233) said that it involves higher level of 
competition for research funding, and 84% (207 out of 233) strongly 
agreed or rather agreed that it provides more international mobility 
opportunities for researchers. 

MSCA: The findings of this study indicate that the majority of 
unsuccessful MSCA applicant organisations (78%) and unsuccessful 
MSCA PF applicants (60%) did not implement their projects in the 
absence of MSCA support. Of those who applied for alternative 
sources of funding to implement the activities proposed in their 
MSCA application, 82% of unsuccessful MSCA applicant 

985 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Evidence 

Methods 
and 
analyses 

organisations and 62% unsuccessful MSCA PF applicants 
implemented their planned projects with some or significant 
adaptations.  

For the overwhelming majority of unsuccessful PF applicants, these 
adapted projects were usually smaller in scope (63%), less complex 
in terms of research methods (66%), smaller in terms of the number 
of research outputs and innovations to be produced (61%), and less 
favourable in terms of research conditions (63%). However, most 
unsuccessful PF applicants indicated that the duration of their 
adapted project was longer than the duration of the planned MSCA 
project (52%). 

For unsuccessful MSCA applicant organisations, the adapted 
projects were smaller in terms of the number of research (sub)areas 
covered (in 85% of cases), as well as in research outputs and 
innovations to be produced (in 81%), and were complex/diverse in 
terms of research methods (in 84% of cases). Furthermore, the 
adapted projects were shorter in duration (74%) as well as less 
diverse in terms of the project partners involved (82%). The 
overwhelming majority of unsuccessful MSCA applicants also 
indicated that the adapted projects constituted a smaller number of 
consortium partners involved (89%) compared with the planned 
MSCA project. 

This study further indicates a strong level of agreement among 
successful MSCA applicant organisations with regard to various 
aspects of Horizon Europe funding, compared with the available 
national and/or regional research funding. A significant 93% of 
respondents strongly agreed or rather agreed that Horizon Europe 
provides more international mobility opportunities for researchers, 
highlighting its role in promoting global collaboration and knowledge 

986 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Evidence 

Methods 
and 
analyses 

exchange. Similarly, 89% of respondents expressed agreement that 
Horizon Europe offers greater diversity in partner profiles, including 
multi-sectoral and multi-disciplinary consortia, as well as a broader 
geographical distribution of partners. Furthermore, 80% of 
respondents perceived there to be a higher level of competition for 
research funding within Horizon Europe, indicating MSCA’s 
reputation as a competitive and prestigious programme. In addition, 
70% of respondents acknowledged that Horizon Europe provides a 
higher amount of funding in comparison to national or regional 
schemes, reflecting its pivotal role in financially supporting research 
endeavours. 

INFRA: Interviewees indicated that INFRA provides clear added 
value by enabling the development of world-class pan-EU Ris; 
promoting collaboration among existing RIs in Europe and beyond; 
reducing fragmentation and duplication in the RI landscape; and 
providing access to RIs across Europe and beyond. In the survey of 
INFRA beneficiaries, 82.2% of respondents (or 125 out of 152) 
indicated that INFRA provides more access to RIs, equipment 
and/or databases compared with national and/or regional funding). 
They also indicated that it brings together a more diverse set of 
partners (80.9%, or 123 out of 152 respondents); and provides more 
international mobility opportunities for researchers (78.9%, or 119 
out of 151) (INFRA-BEN-045). Only 7.8% of INFRA beneficiaries (11 
out of 141) indicated that Horizon Europe provides no additional 
benefits compared with national/regional support (INFRA-BEN-045). 

WIDERA:  

Widening: Evidence points to the importance of the Widening 
actions in supporting countries less advanced in R&I to increase 
their participation in transnational networks, to strengthen their 

987 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Evidence 

Methods 
and 
analyses 

reputations, set up research agendas, projects and teams, and to 
increase their scientific, innovation and management capabilities. 
The Widening beneficiaries interviewed largely agreed that there are 
no comparable grants in their regions or countries, and that their 
projects would be smaller in size, scope and internationality if they 
had been funded at national or regional level. The survey of 
successful WIDERA Widening beneficiaries provides insights into 
the differences between Horizon research funding and national or 
regional funding (WIDERA-WIDE-BEN-045). For example, 89% of 
respondents (152 out of 170) strongly agreed or rather agreed that 
Horizon research funding provides more international mobility 
opportunities for researchers; 88% (150 out of 170) strongly agreed 
or rather agreed that Horizon research funding provides greater 
diversity in partners’ profiles; and 84% (142 out of 170) strongly 
agreed or rather agreed that it provides higher amounts of funding. 
One successful example is the Czech Horizon 2020 Teaming 
project, Cetocoen Excellence1338. The coordinator of this project 
indicated that it would not have obtained the same results without 
the Teaming project, and that it would not have had the same 
budget, resources, outcomes and respect. 

ERA: One the most significant areas of added value provided by 
Horizon Europe support for ERA actions is the opportunity provided 
to beneficiary organisations to form diverse, cross-border networks 
of expertise that are crucial to achieving the project results, which 
could not be achieved without such EU support. EU-level support 
enabled the involvement of a wide range of stakeholders from 
different Member States, which is a key source of knowledge in 
building tools and roadmaps that respond to relevant needs and can 

1338 https://cordis.europa.eu/project/id/857560  

988 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Evidence 

Methods 
and 
analyses 

be adapted by institutions across the EU. According to the results of 
the Horizon Europe beneficiaries survey, 89% of ERA beneficiaries 
(84 out of 95) agreed Horizon Europe provides greater diversity in 
partners’ profiles (e.g. multi-sectoral, multi-disciplinary consortia; 
more geographical diversity of partners etc.) compared with what 
could have been achieved using research funding available to them 
at national and/or regional level. The overwhelming majority of 
respondents (82%, or 76 out of 93) also agreed that the programme 
provides more international mobility opportunities for researchers.  

Other key sources of EU added value for beneficiaries and the EU 
as a whole that stem from Horizon Europe support for ERA actions 
(compared with the funding that would have been available at 
national level) include their contribution to excellence, which stems 
from the higher level of competition at EU level (compared with 
regional or national level); providing beneficiary organisations with 
funding for research topics or fields not covered by national or 
regional R&I funding programmes; pooling skills and expertise 
necessary for research; addressing grand societal challenges; and 
providing a higher amount of funding compared with what is 
available at national/regional level. 

EOSC Partnership: 

As the president of the EOSC Association Karel Luyben explains: 
‘perhaps the best way of seeing the effectiveness of the Partnership 
is that now with the Partnership we start to converge, rather than 
diverge. During Horizon 2020 it was diverging: going in all directions. 
Now there is alignment, and we work in consultation.’. Others have 
emphasised that there is still a lot of EU value yet to be added, since 
there are still very few concrete services on offer to researchers as a 
result of the EOSC, and many new national initiatives emerge that 

989 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

What would have 
happened if the 
Framework 
Programme had not 
existed? Could the 
stakeholders have 
implemented their 
activities in another 
way, including 
through other 
national or regional 
support? 

EAV1.2 

The extent to which beneficiaries could 
have implemented their projects at 
national/regional level; 

The extent to which these national 
projects would have had the same: 

- Scope; 

- Scale; 

- Timeframe; and 

- Results (in terms of excellence). 

Key findings 

Evidence 

are not necessarily avoiding duplications and may not us EOSC 
results. 

ERC: Results from the survey (ERC-APPL-026) point to a small 
majority of unsuccessful ERC applicants (53.7%, or 676 out of 1 258 
respondents) applying for alternative sources of funding (e.g. 
national/regional funds, grants from international organisations, etc.) 
to implement the activities proposed in their rejected ERC 
application. Of those who applied for alternative funding (ERC-
APPL-027), around half (49%, or 332 out of 676) were successful, 
with a large majority of these (89%, or 294 out of 332) receiving their 
funding from national or regional R&D public funding schemes 
(ERC-APPL-029).  

However, when applying to alternatives sources of funding, most 
respondents (84%, or 575 out of 675) indicated that they had to 
implement some kind of adaptation of the project (in some cases 
significant), for example in terms of scope, research conditions or 
project duration (ERC-APPL-030). Interviewees considered this to 
as having significant negative impacts on the outcomes of the 
project, especially if the adaptations entailed a smaller scope or 
smaller teams. Indeed, one of the key success factors identified in 
Case Studies 1 and 2 (mainly based on qualitative information 
extracted from the interviewees and position papers submitted to the 
open public consultation) refers to the opportunity offered by ERC 
funding, compared with other sources of funding, to build larger, 
more diverse and more multidisciplinary teams, and to have a longer 
project duration, which is helpful for developing more (and more 
novel) research outcomes.  

Methods 
and 
analyses 

Literature 
review 

Interviews 

Case 
studies 

Surveys 

990 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Evidence 

Methods 
and 
analyses 

Among those who made adaptations to the applications:  

• 

• 

• 

• 

• 

91.8% (501 out of 546) applied with a project that was 
smaller in scope, in terms of the number of research 
(sub)areas covered (ERC-APPL-031).  
86.8% (447 out of 515) applied with a less complex and 
diverse set of research methods (ERC-APPL-032).  
88.6% (465 out of 525) expected to produce a smaller 
number of research outputs and innovations (ERC-APPL-
033).  
91.3% (464 out of 508) applied with a shorter project 
duration (ERC-APPL-034).  
84.6% (423 out of 500) received less favourable research 
conditions, such as in relation to salary (ERC-APPL-035).  

Among those who did not apply for alternative sources of funding 
and were unable to carry out their project in absence of Horizon 
Europe funding (ERC-APPL-0316), 81.4% (447 out of 549) 
considered the amount of funding available from other sources to be 
insufficient (a ‘very important’ or ‘important’ factor), and 78.8% (438 
out of 556) felt that there was no alternative funding available for that 
research topic/area. 

On top of this, among those applicants who were successful (ERC-
BEN-044), 93% (215 out of 232) strongly agreed or rather agreed 
that Horizon Europe, provides a higher amount of funding compared 
with the research funding available to them at national and/or 
regional level,, and 69% (158 out of 230) strongly agreed or rather 
agreed that it provides funding for research topic(s) or field(s) not 
covered by national or regional R&I funding programmes. These 
figures indicate the broad coverage of topics by the ERC, thanks to 
the bottom-up character of the programme part. One ERC 

991 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Evidence 

Methods 
and 
analyses 

beneficiary interviewed, for example, stressed that without ERC 
funding, it would not have been possible to pursue the same type of 
research, or a type of research capable of competing internationally.  

MSCA: The survey provided insights into the perceptions of 
unsuccessful MSCA applicant organisations and unsuccessful 
MSCA PF applicants regarding the importance of the various 
reasons that prevented them from implementing projects without 
Horizon Europe funding. Among unsuccessful MSCA applicant 
organisations, the most prevalent obstacle was the lack of 
alternative funding available for the type of research they wished to 
pursue, with 72% of respondents considering this a ‘very important’ 
or ‘important’ reason. In addition, 63% of organisations indicated that 
the amount of funding available from other sources was insufficient. 
Concerns about salaries and working conditions also emerged as 
another significant factor, with 56% of organisations indicating that 
salary and working conditions would have been low or 
unsatisfactory. For unsuccessful MSCA PF applicants, the need for 
diverse international partners was identified as the most critical 
factor, with an overwhelming 89% of respondents emphasising its 
importance. Second, the scarcity of alternative funding for their 
specific research domain proved to be a substantial barrier, as 
highlighted by 79% of respondents. Lastly, 68% of respondents 
stressed the inaccessibility of the knowledge, expertise and skills 
essential to the project without Horizon Europe funding. 

INFRA: A large majority of unsuccessful INFRA applicants surveyed 
(78.9%, or 60 out of 76) stated that they did not apply for alternative 
sources of funding to implement the activities proposed in their 
INFRA application (INFRA-APPL-018), indicating that stakeholders 
in general would not be able to implement their activities in another 
way if the Framework Programme did not exist. Of those 

992 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Evidence 

Methods 
and 
analyses 

unsuccessful applicants that did apply for alternative funding, 73.3% 
(11 out of 15) indicated that this involved significant adaptations 
compared with their Horizon Europe application (INFRA-APPL-022). 
The main reasons for not applying for alternative funding are the 
need for a variety of international partners (91.6%, or 55 out of 60 
reported this reason as ‘important’ or ‘very’ important); the lack of 
alternative funding for their research topic/area (86.7%, or 52 out of 
60); and the fact that their project addressed EU-level (and/or) global 
challenges (80.0%, or 48 out of 60) (INFRA-APPL-029).  

WIDERA: 

Widening: Of those unsuccessful WIDERA widening applicants that 
responded to the survey (survey of unsuccessful Horizon Europe 
applicants, conducted May-July 2023), almost 80% (257 out of 323) 
did not apply for alternative funding such as national, regional or 
international funding to implement the activities proposed in the 
WIDERA project, while 14.6% (47 out of 323) did apply for 
alternative funding (WIDERA-WIDE-APPL-018). Of those that did 
apply, 36.3% (17 out of 47) were successful, 10.6% (5 out of 47) 
were not, and almost 50% (23 out of 47) were still awaiting a 
decision.  Of those who received alternative funding, 58.8% (10 out 
of 17) received it from a national or regional source, 23.5% (3 out of 
17) received international funding such as from the ERDF, ESF or 
CF. In addition, of those that did apply to another funding source, 
almost half (23 out of 47) made significant adaptations to their 
proposal, 36.2% (17 out of 47) made some adaptations, and 8.5% (4 
out of 47) made minimal-to-no adaptations. Those who made 
adaptations, did so with respect to: 

• 

the project’s scope, with 75% (30 out of 40) reducing its 
scope in terms of the number of research areas covered;  

993 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Evidence 

Methods 
and 
analyses 

• 

• 

• 

• 

• 

research methods, with 82.5% (33 out of 40) covering a 
less complex and diverse set of research methods;  
research outputs, with 62.5% (25 out of 40) expecting to 
produce a smaller number of research outputs and 
innovations; 
duration, with 70% (28 out of 40) planning a project of 
shorter duration; 
the number of partners, with 77.5% (31 out of 40) 
involving a smaller number of consortium partners; 
partner profiles, with 66.7% (26 out of 39) including less 
diverse partners (in terms of internationality, sectors etc.).  

Among those unsuccessful applicants who decided not to apply for 
alternative funding, the most common reasons for doing so were that 
the project required the involvement of a variety of international 
partners and that there was simply no alternative funding available 
for that research topic (WIDERA-WIDE-APPL-029).  

ERA: The significant additionality brought by EU support is also 
confirmed by the assessment of unsuccessful applicants under the 
ERA thematic priority. A large majority of unsuccessful applicants 
(81%, 67 out of the 83 unsuccessful respondents) indicated that they 
did not implement their project without Horizon Europe support (and 
either did not apply for additional funding, or applied for alternative 
funding but were unsuccessful)). Fewer than 16% (13 out of 83) of 
the unsuccessful applicants indicated they had applied for 
alternative funding, with slightly more than half of them (8 out of 13) 
reporting that their application for alternative funding (from 
national/regional or other alternative funding sources) had been 
successful, and they had implemented the project or the activities 
proposed in their Horizon Europe application. This shows that at the 

994 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

EAV2 

The extent to which projects under the 
Excellent Science pillar addressed 
cross-cutting or transversal issues. 

Examples of exceptional EAV in terms 
of addressing cross-cutting or 
transversal issues in the projects funded 
under the Excellent Science pillar. 

What is the added 
value of the EU 
Framework 
Programme in 
addressing cross-
cutting or 
transversal issues 
such as 
Responsible 
Research and 
Innovation, co-
design and co-
creation, gender 
equality, Social 
Sciences and 
Humanities, 
international 
cooperation? 

Key findings 

Evidence 

time of evaluation, fewer than 10% of unsuccessful applicants (8 out 
of 83) received alternative funding for the activities planned in their 
WIDERA project. 

ERC: As mentioned earlier under questions EFC 13.1, 13.2, 13.3 
and 14, given its nature, the ERC contributes to cross-cutting issues 
through funding bottom-up research on the one hand, and through 
the development of individual researchers’ skills on the other. To 
illustrate the former, data from the Horizon Dashboard (accessed on 
10 September 2023) shows two signed ERC grants on the topic of 
gender studies (compared with 22 grants signed under Horizon 
2020) and five signed ERC grants on the topic of ethics principles 
(compared with seven grants signed under Horizon 2020).  

With respect to skills development, as shown above, 43% of survey 
respondents (99 out of 233) stated that the ERC had contributed to a 
‘very large’ or ‘large’ extent to developing their skills in research 
ethics and integrity. Around half of ERC beneficiaries (115 out 237) 
reported no issues and a positive experience with the ethics self-
assessment of their ERC project (ERC-BEN-042), and 21% (50 out 
of 237) reported only minor issues. Meanwhile, 62% of ERC 
beneficiaries (115 out of 187) considered the support (e.g. 
information, guidance) available for carrying out the ethics self-
assessment of their ERC project to be excellent or good (ERC-BEN-
043).  

With respect to participation by gender, the share of women 
applicants to the ERC has improved slightly over time. Data on ERC 
calls between 2007-2021 show a share of 28% of female ERC 
applicants, while preliminary EC administrative and monitoring data 

Methods 
and 
analyses 

Monitoring 
data 

Interviews 

Case 
studies 

995 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Evidence 

Methods 
and 
analyses 

for Horizon Europe provided to the study team in May 2023 show a 
share of 33% ERC applicants being women. To incentivise 
applications from women researchers, in addition to Horizon Europe-
wide actions, the ERC has been carrying out changes to its 
application rules, for example with respect to maternity leave 
conditions. 

With respect to co-design and co-creation, only 19.3% of the ERC 
beneficiaries that responded to the survey (45 out of 233) indicated 
that they plan to engage citizens and citizen representatives or end 
users in the co-creation of R&I content in their project (ERC-BEN-
038). 

Lastly, the ERC contributes significantly to social sciences and 
humanities. Horizon Dashboard data (accessed on 11 September 
2023) shows that 19% of ERC signed grants so far are in the fields 
of SSH. This corresponds to EUR 873.5 million of net EU 
contribution. For more information, see Case Study 2.  

MSCA: The survey findings indicate the experiences of successful 
MSCA applicant organisations and MSCA PF applicants with the 
ethics self-assessment in their projects. A large majority of 
successful MSCA applicant organisations (67%) and successful 
MSCA PF applicants (76%) reported positive experiences to a ‘large’ 
extent, indicating that they faced no issues with the ethics self-
assessment. This high percentage of positive feedback suggests 
that the EU's process for ethics self-assessment is generally clear, 
accessible and manageable for most applicants, enabling the 
successful promotion of high standards of ethical conduct in R&I. 
Furthermore, the majority of successful MSCA applicant 
organisations believed that their participation in MSCA had, to a 

996 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Evidence 

Methods 
and 
analyses 

‘large’ or ‘very large’ extent (52%) aided their efforts to reinforce RRI 
principles within their institutions.  

The MSCA actions fully support and encourage participating 
organisations and researchers to follow the principles of Open 
Science. Within this context, the study found that slightly more than 
one-third of successful MSCA applicant organisations (38%) and 
successful MSCA PF applicants (39%) plan to engage 
citizens/citizen representatives/end-users in the co-creation of R&I 
content in their MSCA projects. Out of those who plan to do so, a 
significant share of successful applicant organisations and 
applicants (55%) plan to actively involve citizens or end-users in co-
design activities, which typically involve organising workshops, focus 
groups or similar approaches to collaboratively developing research 
and innovation agendas, roadmaps and policies. 

The MSCA have a pronounced structuring impact on participating 
organisations, and bring added value by promoting gender equality. 
The structuring impacts of the MSCA extend beyond MSCA projects, 
and are spread/institutionalised within beneficiary organisations and 
national/regional systems. This is particularly evident in relation to 
the gender equality policies that are being institutionalised as a 
result of participation in MSCA actions. However, the study finds that 
there is variation in gender balance across the different types of 
actions. For instance, the MSCA Doctoral Networks have a higher 
percentage of men researchers (67%) compared with women 
researchers (33%), indicating an imbalance skewed towards men in 
this particular action. Conversely, MSCA and Citizens, COFUND 
and Support Activities have a more balanced gender distribution. 
These findings can be considered indicative, as many projects are 
still in an early stage of implementation, and the accuracy of the 

997 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Evidence 

Methods 
and 
analyses 

available information is therefore limited due to a lack of data on the 
researchers recruited to DN and COFUND projects. 

INFRA: As mentioned earlier, INFRA does not contribute 
significantly to transversal issues such as gender equality, RRI, and 
co-design and co-creation. Only a minority of beneficiaries (38.80%, 
or 59 out of 152) indicate that they plan to engage citizens/citizen 
representatives/end-users in the co-creation of R&I content in their 
project (INFRA-BEN-037). Among those beneficiaries who do, 
72.4% (42 out of 58) plan to use co-design activities the most (such 
as workshops or focus groups). These are followed by co-
assessment (36.2%, or 21 out of 28) and co-creation (34.5%, or 20 
out of 28) (INFRA-BEN-038). Meanwhile, only 31% (28 out of 91) of 
unsuccessful applications to INFRA indicate that INFRA responds 
(to a ‘large’ or ‘very large’ extent) to their organisation’s need to 
strengthen the application of RRI principles (INFRA-APPL-015), and 
even fewer INFRA beneficiaries (25.3%, or 41 out of 162) indicate 
that this programme part responds to this need (INFRA-BEN-025). 
In terms of social science and humanities, some interviewees 
indicate that not enough funding is directed towards this discipline, 
which is important in understanding societal change and could 
contribute significantly to the SDGs.  

In terms of international cooperation, interviewees and survey 
responses indicate that INFRA plays an important role in fostering 
international cooperation between RIs. Of the INFRA beneficiaries 
surveyed, 59.1% (88 out of 149) indicated that collaboration with 
international (non-EU) partners is important to a ‘large’ or ‘very large’ 
extent for the uptake of their research findings/innovation (INFRA-
BEN-041). However, as already discussed in questions RV6 and 
EFC6.1, international cooperation with RIs outside Europe could be 

998 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Evidence 

Methods 
and 
analyses 

further fostered. The participation of associated countries and third 
countries remains limited (6% and 13%, respectively, but comprising 
significant shares by the UK and Switzerland, which are third 
countries under Horizon Europe) (in the meantime, the UK is an 
Association Country to Horizon Europe). Other third countries with a 
high number of participations are the United States (eight, or 0.4% of 
all Horizon Europe INFRA participations) and South Africa (five, or 
0.3%). Among the associated countries, those with the highest 
numbers of participations are Norway (67, or 3.8%) Israel (eight, or 
0.7%), Serbia (seven, or 0.4%) and Turkey (six, or 0.4%). Moreover, 
the survey indicates that more funding is needed for collaboration 
with international partners (non-EU countries): only 25.0% (or 41 out 
of 164) of INFRA beneficiaries agreed to a ‘large’ or ‘very large’ 
extent that there is sufficient funding for international collaboration 
(INFRA-BEN-024). In addition, only 26.2% agreed (to a ‘large’ or 
‘very large’ extent) that there is sufficient funding for pre-normative 
research allowing for the development of international standards 
(INFRA-BEN-024). Furthermore, a more strategic and long-term 
approach to international cooperation would be welcomed by RIs to 
ensure sustainability beyond project lifetimes. 

WIDERA:  

Widening:  

Responsible Research and Innovation. Interviews with Horizon 
2020 Widening beneficiaries conducted during the two phases of 
this study point to the influence of the Widening actions and the 
Framework Programme in general on responsible research and 
innovation (RRI) in their institution, and sometimes even in their 
region or country. Specifically, the creation or revision of ethics 
guidelines in organisations or countries is regarded as a structural 

999 

 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Evidence 

Methods 
and 
analyses 

change sometimes triggered by Widening actions that aim to foster 
research excellence and collaborations1339. Case Study 7 (Annex 2) 
indicates that slightly under half of the SEWP survey respondents in 
the Evaluation study on Excellent Science under Horizon 2020 
acknowledged that their Widening project had influenced the 
development or improvement of standards or appraisal processes 
with regard to research ethics and integrity in the host institution. 
Furthermore, the survey indicates that almost half of successful 
WIDERA widening applicants (48.5%, or 83 out of 171) had a 
positive experience with ethics self-assessment (WIDERA-WIDE-
BEN-043).  

Co-design and co-creation. Around half of WIDERA Widening 
beneficiaries who responded to the survey (52.4%, or 89 out of 170) 
indicated that they plan to engage citizens and citizen 
representatives or end users in the co-creation of R&I content in 
their project. A quarter of respondents (25.3%, or 43 out of 170) did 
not plan to do so, and another 22.4% (38 out of 170) did not know 
(WIDERA-WIDE-BEN-037). In addition, novel actions introduced 
under Horizon Europe – in particular, Excellence Hubs – engage 
quadruple helix partners, and hence society. Such large-scale 
projects often cannot be funded at national level, thus the EU 
provides particular added value here. Nevertheless, it remains to be 
examined what the concrete outcomes and impacts are of such 
Excellence Hubs projects are.  

Gender Equality. The Horizon Dashboard (accessed 3 September 
2023) indicates that among the Horizon 2020 SEWP participants, 
46.9% were women – slightly lower than the share for the Horizon 

1339 See Case Study 7, Annex 2. 

1000 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Evidence 

Methods 
and 
analyses 

2020 Excellent Science pillar as a whole (47.9%) However, women 
account for 33.9% of coordinators in SEWP, which is higher than the 
share of female coordinators for the Excellent Science pillar as a 
whole (23.1%). Also, both shares are higher in SEWP than across 
Horizon 2020 as a whole (the respective Horizon 2020 averages 
being 42.1% and 23.0%). In addition, the endorsement and 
application of EU Charters and Codes of Conduct and principles 
relating to gender balance are often identified as structural changes 
that are triggered by Widening actions, affecting either the institution, 
or in some cases the region or country. In particular, Case Study 7 
shows that half of respondents to the survey of SEWP beneficiaries 
carried out in the Evaluation study on Excellent Science under 
Horizon 2020 indicated that their Widening project had resulted in 
advances towards achieving gender equal opportunities within their 
institutions. 

Social Sciences and Humanities. According to Horizon 2020 EC 
monitoring and reporting data, 18% of SEWP projects were tagged 
with SSH. This share is the same is under the Excellent Science 
pillar (18%), somewhat higher than the Societal Challenges pillar 
(16%), and lower than the SwafS pillar (59%). While it is unclear 
what the particular added value of Widening actions is in terms of 
SSH, there are good examples of WIDERA Widening projects that 
aim to raise awareness of the importance of research being socially 
engaged. One example is the BETTER Life1340 project, the idea of 
which was to explore ‘how to make life science researchers (with the 
focus on early career researchers) more aware that their research 
can be socially engaged, that it can have impact on society, and to 

1340 BETTER Life: Bringing Excellence to Transformative Socially Engaged Research in Life Sciences through Integrated Digital Centers, available at: 
https://cordis.europa.eu/project/id/101071314 

1001 

 
 
 
Evaluation 
question 

ID 

Judgement criteria/indicators 
identified ex ante 

Key findings 

Evidence 

Methods 
and 
analyses 

support them in such endeavours’. The project also triggered 
interest in the city of Prague and at a wider regional level. 

International cooperation. As previously mentioned, the Widening 
actions truly spur international cooperation, and this is extremely 
valued by widening institutions. This type of international research 
often cannot be funded via national funding sources, and the 
Widening actions are therefore indispensable to widening countries. 
Not only do such cooperations allow engagement (to a limited 
extent) with excellent R&I projects, but they provide opportunities to 
significantly strengthen skills, mobility, capacities, reputation, 
visibility, and so on.  

ERA: As previously mentioned in the overview of project 
implementation under the WIDERA ERA thematic priority, the 
majority of ERA projects focus on cross-cutting and transversal 
issues – in particular, Open Science and the reusability of research 
results, RRI, research ethics, gender equality, inclusion, citizen 
engagement in science, and related topics. Bearing in mind the 
above figures on the additionality of Horizon Europe support and the 
fact that the majority of projects could not be implemented at 
national/regional level without EU support, the available evidence 
confirms that WIDERA (ERA) projects provide very significant EU 
added value in terms of addressing cross-cutting and transversal 
issues. 

1002 

 
 
Annex 5: Literature and information sources 

In this annex we provide a continuously updated compilation of literature and information sources 
consulted while implementing the study. 

Main report and answers to the evaluation questions (Annex 1) 

European Research Council under H2020 and Horizon Europe 

•  Assessment of the Union Added Value and the economic impact of the EU Framework 

Programmes (FP7, Horizon 2020), https://op.europa.eu/en/publication-detail/-
/publication/af103c38-250d-11e9-8d04-01aa75ed71a1/language-en 

•  Bonaccorsi, A., (2015), Ex-post Evaluation of the Seventh Framework Programme, 

Support paper to the High Level Expert Group, IDEAS Specific Programme Analytical 
Evaluation, retrieved through: https://www.kowi.de/Portaldata/2/Resources/fp7/fp7-
ideas-report.pdf  

•  ERC at a glance: https://erc.europa.eu/about-erc/mission 

•  ERC facts and figures: https://erc.europa.eu/about-erc/facts-and-figures  

•  ERC frontier research contribution to a Europe fit for the digital age: 

https://erc.europa.eu/projects-statistics/frontier-research-empowering-digital-europe 

•  ERC frontier research contribution to the European Green Deal: 

https://erc.europa.eu/projects-statistics/frontier-research-european-green-deal 

•  ERC news: https://erc.europa.eu/news-events/news  

•  ERC Research Information System. https://erc.europa.eu/projects-statistics/advanced-

analytics-tool-eris 

•  ERC website 

•  European Commission (2022). Performance of European Partnerships: Biennial 

Monitoring Report 2022 on partnerships in Horizon Europe. 

•  European Commission C(2013)8632 of 10 December 2013, ERC work programme 

2014, 
https://erc.europa.eu/sites/default/files/document/file/ERC_Work_Programme_2014.pdf 

•  European Commission C(2014)5008 of 22 July 2014, ERC work programme 2015, 

https://erc.europa.eu/sites/default/files/document/file/ERC_Work_Programme_2015.pdf  

•  European Commission C(2015) 5086 of 28 July 2015, ERC work programme 2016, 

https://erc.europa.eu/sites/default/files/document/file/ERC_Work_Programme_2016.pdf  

1003 

 
 
 
•  European Commission C(2016) 4616 of 25 July 2016, ERC work programme 2017, 
https://ec.europa.eu/research/participants/data/ref/h2020/wp/2016_2017/erc/h2020-
wp17-erc_en.pdf  

•  European Commission Decision C(2021) 4860, ERC Work Programme 2022, 

https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2022/wp_horizon-erc-2022_en.pdf 

•  European Commission Decision C(2021) 930, ERC Work Programme 2021, 

https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2021/wp_horizon-erc-2021_en.pdf 

•  European Commission Decision C(2022) 4861, ERC Work Programme 2023, 

https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2023/wp_horizon-erc-2023_en.pdf  

•  European Commission, “Public consultation on the past, present and future of the 
European Research & Innovation Framework Programmes 2014-2027”. Available: 
https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-Horizon-
Europe-interim-evaluation/public-consultation_en.  

•  European Commission, Directorate-General for Research and Innovation, European 

Research Council Executive Agency, Annual report on the ERC activities and 
achievements in 2019, Publications Office, 2020, 
https://data.europa.eu/doi/10.2828/020505 

•  European Commission, Directorate-General for Research and Innovation, European 

Research Council Executive Agency, Annual report on the ERC activities and 
achievements in 2018, Publications Office, 2019, 
https://data.europa.eu/doi/10.2828/36521  

•  European Commission, Directorate-General for Research and Innovation, European 

Research Council Executive Agency, Annual report on the ERC activities and 
achievements in 2017, Publications Office, 2018, 
https://data.europa.eu/doi/10.2828/397027  

•  European Commission, Directorate-General for Research and Innovation, European 

Research Council Executive Agency, Annual report on the ERC activities and 
achievements in 2016, Publications Office, 2017, 
https://data.europa.eu/doi/10.2828/990429 

•  European Commission, Directorate-General for Research and Innovation, European 

Research Council Executive Agency, Annual report on the ERC activities and 
achievements in 2015, Publications Office, 2016, 
https://data.europa.eu/doi/10.2828/37878 

•  European Commission, Directorate-General for Research and Innovation, European 

Research Council Executive Agency, Annual report on the ERC activities and 
achievements in 2014, Publications Office, 2015, 
https://data.europa.eu/doi/10.2828/3513    

1004 

 
 
•  European Commission, Directorate-General for Research and Innovation, In-depth 

Interim evaluation of H2020: Commission staff working document, Publications Office, 
2017. 

•  European Commission, European Research Council Executive Agency, Annual report 
on the ERC activities and achievements in 2021, Publications Office of the European 
Union, 2022, https://data.europa.eu/doi/10.2828/064534 

•  European Commission, European Research Council Executive Agency, Annual report 
on the ERC activities and achievements in 2022 –, Publications Office of the European 
Union, 2023, https://data.europa.eu/doi/10.2828/45086 

•  European Commission, European Research Council Executive Agency, Annual report 
on the ERC activities and achievements in 2021 –, Publications Office of the European 
Union, 2022, https://data.europa.eu/doi/10.2828/064534 

•  European Commission, European Research Council Executive Agency, Annual report 

on the ERC activities and achievements in 2020, Publications Office, 2021, 
https://data.europa.eu/doi/10.2828/789744  

•  European Commission, European Research Council Executive Agency, Mapping ERC 
frontier research Horizon 2020 Research and Innovation Framework Programme 
(2014-2020) – , Publications Office of the European Union, 2023, 
https://data.europa.eu/doi/10.2828/636104 

•  European Commission. Spotlight reports on diverse fields of science (2011-2019) 

https://op.europa.eu/ 

•  European Research Council (2018), ERC Monitoring and Evaluation Strategy 2018, 
retrieved through: https://erc.europa.eu/sites/default/files/content/pages/pdf/erc-
monitoring-and-evaluation-2018_brochure.pdf  

•  European Research Council (2021), Qualitative Evaluation of completed Projects 

funded by the European Research Council 2020, retrieved through:  
https://erc.europa.eu/sites/default/files/document/file/2021-qualitative-evaluation-
projects.pdf 

•  European Research Council (2022). COVID-19 Frontier research in the spotlight. 

https://erc.europa.eu/sites/default/files/2022-08/COVID19-Frontier_research_in_the-
spotlight.pdf. 

•  European Research Council (2022). EU4Health. Available at: 

https://erc.europa.eu/sites/default/files/2022-08/H2020_factsheet_EU4Health.pdf. 

•  European Research Council (2022). European Green Deal. Available at: 

https://erc.europa.eu/sites/default/files/2022-
08/H2020_factsheet_EuropeanGreenDeal.pdf. 

•  European Research Council (2022). Gender statistics. Available at: 

https://erc.europa.eu/sites/default/files/2022-11/Gender_statistics_June_2022.pdf. 

1005 

 
 
•  F. Meyer-Krahmer (2017). Meta-Evaluation of Article 185 Initiatives, Report of the 

Expert Group. 

•  Other websites: 

–  https://ec.europa.eu/info/funding-

tenders/opportunities/portal/screen/opportunities/horizon-dashboard 

–  https://en.wikipedia.org/wiki/Holberg_Prize. 

–  https://en.wikipedia.org/wiki/Stein_Rokkan_Prize_for_Comparative_Social_Sci

ence_Research. 

–  https://en.wikipedia.org/wiki/Swedish_Society_for_Anthropology_and_Geogra

phy#Retzius_Medal. 

–  https://research-and-innovation.ec.europa.eu/funding/funding-

opportunities/seal-excellence_en.  

•  Mapping ERC frontier research: https://erc.europa.eu/mapping-science-frontiers 

•  Monti et al. (2017) “Future Funding of the EU: Final report and recommendations of the 

high level group on own resources”, December 2016. 

•  N. Gøtke et al. (2016). Analysis of ERA-NET Cofund actions under Horizon 2020: Final 

report of the expert group. https://op.europa.eu/en/publication-detail/-
/publication/74c34f43-b147-11e6-871e-01aa75ed71a1. 

•  Nobel Prizes Reports. Available at: 

https://www.nsf.gov/news/special_reports/nobelprizes/. 

•  Performance of European Partnerships. Biennial Monitoring Report 2022 on 

Partnerships in Horizon Europe. 2. Contribution of European Partnerships. 
https://ec.europa.eu/research-and-innovation/sites/default/files/bmr-2022/ec_rtd_bmr-
2022-chapter-2.pdf 

•  SBFI. EU Framework Programmes for Research and Innovation. 

https://www.sbfi.admin.ch/sbfi/en/home/research-and-innovation/international-
cooperation-r-and-i/eu-framework-programmes-for-research.html.  

Marie Skłodowska-Curie Actions under H2020 and Horizon Europe 

•  ARIS - Načrt za okrevanje in odpornost - Javni poziv za (so)financiranje projektov in 

programov za krepitev mednarodne mobilnosti slovenskih raziskovalcev in 
raziskovalnih organizacij ter za spodbujanje mednarodne vpetosti slovenskih 
prijaviteljev. (2023). http://www.arrs.si/sl/NOO/razpisi/23/poziv-mobilnost-23.asp  

•  Combra Group. Position Paper on the past, present and future of the European 
Research & Innovation Framework programmes 2014-2027. February 2023 

1006 

 
 
•  Commission Notice Synergies between Horizon Europe and ERDF programmes 

2022/C 421/03 (OJ C, C/421, 04.11.2022, p. 7, CELEX: https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=CELEX:52022XC1104(02)   

•  Contribution of Masaryk University to Public Consultation on the past, present and 

future of the European Research & Innovation Framework programmes 2014-2027. 
February 2023 

•  Council Decision (EU) 2021/764 of 10 May 2021 establishing the Specific Programme 

implementing Horizon Europe – the Framework Programme for Research and 
Innovation, and repealing Decision 2013/743/EU (Text with EEA relevance) 

•  Council Decision (EU) 2021/764 of 10 May 2021 establishing the Specific Programme 

implementing Horizon Europe – the Framework Programme for Research and 
Innovation, and repealing Decision 2013/743/EU (Text with EEA relevance) (OJ L 167 
12.05.2021, p. 1, ELI: http://data.europa.eu/eli/dec/2021/764/oj)  

•  Ejermo, O., Fassio, C., & Källström, J. (2019). Does Mobility across Universities Raise 
Scientific Productivity? Oxford Bulletin of Economics and Statistics, 82(3), 603–624. 
https://doi.org/10.1111/obes.12346   

•  European Commission (2021). Marie Skłodowska-Curie Actions 2021-2027. 

Developing talents, advancing research.  

•  European Commission, DG EAC (2021). Marie Skłodowska-Curie Actions guidelines 

on supervision. https://data.europa.eu/doi/10.2766/508311.  

•  European Commission, DG EAC (2021). Marie Skłodowska-Curie Actions Green 

Charter. https://data.europa.eu/doi/10.2766/720690.  

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 
Marie Skłodowska-Curie actions guidelines on supervision, Publications Office, 2021, 
https://data.europa.eu/doi/10.2766/508311   

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 
Synergies between the Marie Skłodowska-Curie actions and the European Institute of 
Innovation and Technology, Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2766/42217   

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 
Dėlkutė, R., Nikinmaa, J., Pupinis, M. et al., Study on mobility flows of researchers in 
the context of the Marie Sklodowska-Curie Actions – Analysis and recommendations 
towards a more balanced brain circulation across the European Research Area – Final 
report, Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2766/401134   

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 
Marie Skłodowska-Curie actions guidelines on supervision, Publications Office, 2021, 
https://data.europa.eu/doi/10.2766/508311   

1007 

 
 
•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 

Study on international cooperation in the Marie Skłodowska-Curie actions: final report, 
Publications Office, 2019, https://data.europa.eu/doi/10.2766/621682 

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 
Souto-Otero, M., Humburg, M., Franke, J., FP7 ex post and H2020 interim evaluation 
of Marie Skłodowska-Curie actions (MSCA): final report, Publications Office, 2017, 
https://data.europa.eu/doi/10.2766/103174 

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 
Dėlkutė, R., Nikinmaa, J., Pupinis, M., et al., Study on mobility flows of researchers in 
the context of the Marie Sklodowska-Curie Actions: analysis and recommendations 
towards a more balanced brain circulation across the European Research Area: final 
report, Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2766/401134 

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 
Study on business participation and entrepreneurship in Marie Skłodowska-Curie 
actions (FP7 and Horizon 2020): final report, Publications Office of the European 
Union, 2017, https://op.europa.eu/en/publication-detail/-/publication/decfab92-5ae2-
11e7-954d-01aa75ed71a1/language-en/format-PDF/source-273489664  

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 
Pupinis, M., Brožaitis, H., Navikas, V., et al., Review of Marie Skłodowska-Curie 
actions unit costs in preparation for Horizon Europe: final report, Publications Office, 
2020, https://data.europa.eu/doi/10.2766/36137 

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 
Synergies between the Marie Skłodowska-Curie Actions and Erasmus+ in the area of 
higher education, Publications Office of the European Union, 2022, 
https://op.europa.eu/en/publication-detail/-/publication/1f4e362a-824b-11ec-8c40-
01aa75ed71a1  

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 
Study of business participation and entrepreneurship in Marie Skłodowska-Curie 
actions (FP7 and Horizon 2020): final report, Publications Office, 2017, 
https://data.europa.eu/doi/10.2766/121877 

•  European Commission, Directorate-General for Research and Innovation, Science, 

research and innovation performance of the EU 2022 – Building a sustainable future in 
uncertain times – Executive summary, Publications Office of the European Union, 
2023, https://data.europa.eu/doi/10.2777/497405   

•  European Commission, Directorate-General for Research and Innovation, Almerud, M., 
Ricksten, M., O’Neill, G. et al., Knowledge ecosystems in the new ERA – Using a 
competence-based approach for career development in academia and beyond, Núñez, 
L.(editor), De Coen, A.(editor), Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2777/150763   

•  European Commission, Directorate-General for Research and Innovation, The 
European Charter for Researchers – The code of conduct for the recruitment of 
researchers, Publications Office, 2006. 

1008 

 
 
•  European Commission, Directorate-General for Research and Innovation, Interim 
Evaluation of Horizon 2020: commission staff working document, Annex 2, 2018, 
https://op.europa.eu/en/publication-detail/-/publication/4cedc9a0-d8cc-11e8-afb3-
01aa75ed71a1/language-en# 

•  European Commission, Directorate-General for Research and Innovation, Ince, E., 

Janger, J., Knowledge ecosystems in the new ERA : talent circulation and intersectoral 
mobility : analytical report with a mapping of talent mobility and causes of brain drain, 
Reda Nausedaite, R.(editor), Fraioli, M.(editor), Moreno, T.(editor), Colassin, C.(editor), 
Costas Comesaña, R.(editor), Núñez, L.(editor), Cincera, M.(editor), Ricksten, 
M.(editor), Anna Strauss-Kollin, A.(editor), Daniel Holmberg, D.(editor), Dudek, 
J.(editor), Hoed, M.(editor), Hofmann, K.(editor), Janger, J.(editor), Publications Office 
of the European Union, 2022, https://data.europa.eu/doi/10.2777/701074 

•  European Commission, Directorate-General for Research and Innovation, Evaluation 
study on the external coherence and synergies of Horizon 2020 within the European 
research and innovation support system, Draft Final Report, 2022. 

•  European Commission, Directorate-General for Research and Innovation, Evaluation 

study on the implementation of cross-cutting issues in Horizon 2020, Draft Final Report, 
2022. 

•  European Commission, Directorate-General for Research and Innovation, Daimers, S., 
Seus, S., Nof Afghani, A., Kroll, H. Howoldt, D., Evaluation study on the relevance and 
internal coherence of Horizon 2020 and its policy mix, Interim Report 2, Version 1, 
2022.  

•  European Commission, European Research Executive Agency, Marie Skłodowska-

Curie actions under the Seventh Framework Programme for Research and 
Technological Development: a taste of success, Publications Office, 2020, 
https://data.europa.eu/doi/10.2848/49626 

•  European Commission. (2011). Principles for Innovative Doctoral Training. 

•  European Commission. (2020). COMMUNICATION FROM THE COMMISSION TO 

THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND 
SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS A new ERA for 
Research and Innovation (COM/2020/628 final). European Commission. https://eur-
lex.europa.eu/legal-content/EN/TXT/?uri=COM%3A2020%3A628%3AFIN   

•  European Commission. (2021). Work Programme of the Euratom Research and 

Training Programme 2021-2022. Retrieved from https://ec.europa.eu/info/funding-
tenders/opportunities/docs/2021-2027/euratom/wp-call/2021-2022/wp_euratom-2021-
2022_en.pdf   

•  European Commission. (2022). Commission Notice Synergies between Horizon 
Europe and ERDF programmes (2022/C 421/03) https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=CELEX%3A52022XC1104%2802%29   

•  FP7 ex-post evaluation PEOPLE Specific Programme (2007-2013): Rationale, 

implementation and achievements by Dr Dragana Avramov- (2015). 
https://www.kowi.de/Portaldata/2/Resources/fp7/fp7-people-report.pdf  

1009 

 
 
•  Funding opportunities under MSCA. (2023). Research and Innovation. https://research-
and-innovation.ec.europa.eu/funding/funding-opportunities/seal-excellence/funding-
opportunities-under-msca_en   

• 

Inspireurope+ Submission to Public Consultation on the European Research and 
Innovation Framework Programmes 2014-2027 

•  MSCA Seal of Excellence awarded to high-quality Postdoctoral Fellowships proposals. 

(2023). Marie Skłodowska-Curie Actions. https://marie-sklodowska-curie-
actions.ec.europa.eu/news/award-of-msca-seal-of-excellence-certificates-postdoctoral-
fellowships-2022   

•  MSCA Seal of Excellence awarded to outstanding COFUND proposals. (2023). Marie 

Skłodowska-Curie Actions. https://marie-sklodowska-curie-
actions.ec.europa.eu/news/msca-seal-of-excellence-awarded-to-outstanding-cofund-
proposals   

•  MSCA Seal of Excellence awarded to outstanding researchers. (2022). Marie 

Skłodowska-Curie Actions. https://marie-sklodowska-curie-
actions.ec.europa.eu/news/msca-seal-of-excellence-awarded-to-outstanding-
researchers   

•  Netz, N., Hampel, S., & Aman, V. (2020). What effects does international mobility have 
on scientists’ careers? A systematic review. Research Evaluation, 29(3), 327–351. 
https://doi.org/10.1093/reseval/rvaa007   

•  OECD (2021b), "Reducing the precarity of academic research careers", OECD 

science, technology and industry policy papers, No. 113, OECD Publishing, Paris. 
https://doi.org/10.1787/0f8bd468-en   

•  OECD. (2019b). “What are the characteristics and outcomes of doctoral graduates?”, in 

Education at a glance 2019: OECD indicators. OECD Publishing, Paris. 
https://doi.org/10.1787/8389c70e-en   

•  OP JAK a MSCA. (2022). ČZU. https://www.czu.cz/cs/r-7584-projekty-a-spoluprace-s-

praxi/r-11162-visual-projekty/op-jak-a-msca.html   

•  Powell, S., & Green, H. (Eds) (2007). The doctorate worldwide. McGraw-Hill Education: 

Maidenhead. 

•  Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 
2021 establishing Horizon Europe – the Framework Programme for Research and 
Innovation, laying down its rules for participation and dissemination, and repealing 
Regulations (EU) No 1290/2013 and (EU) No 1291/2013 (Text with EEA relevance) 

•  Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 
2021 establishing Horizon Europe – the Framework Programme for Research and 
Innovation, laying down its rules for participation and dissemination, and repealing 
Regulations (EU) No 1290/2013 and (EU) No 1291/2013 (Text with EEA relevance) 
(OJ L 170 12.05.2021, p. 1, ELI: http://data.europa.eu/eli/reg/2021/695/oj)     

1010 

 
 
•  Sarrico, C. S. (2022). The expansion of doctoral education and the changing nature 

and purpose of the doctorate. Higher Education, 84(6), 1299–1315. 
https://doi.org/10.1007/s10734-022-00946-1   

•  Sarrico, C. S. (2022). The expansion of doctoral education and the changing nature 

and purpose of the doctorate. Higher Education, 84(6), 1299–1315. 
https://doi.org/10.1007/s10734-022-00946-1   

•  Science Foundation, Boman, J., Beeson, H., Sanchex, M. Barrioluengo, M., What 

comes after a PhD? Findings from the DocEnhance survey of doctorate holders on 
their employment situation, skills match, and the value of the doctorate, DocEnhance 
Deliverable 1.2, 2021, https://docenhance.eu/wordpress/wp-
content/uploads/2021/12/DocEnhance-D1.2_Report-on-career-tracking-of-PhD-
graduates.pdf  

•  The University of Cambridge Response to the Public Consultation on the EU R&I 

Framework Programmes. February 2023 

•  Universities of the Netherlands. Position Paper Horizon Europe and Beyond. February 

2023. 

Future and emerging technologies under H2020 

•  Amunts, K., Ebell, C., Muller, J., Telefont, M., Knoll, A., & Lippert, T. (2016). The 

human brain project: creating a European research infrastructure to decode the human 
brain. Neuron, 92(3), 574-581 

•  Calogero, Maria Oddo, Experiences from the FET Flagships Interim Evaluation: 

https://www.era-learn.eu/documents/flagships_cmo_20181108_v2.pdf 

•  Carrozza MC. et al. (2017) FET FLAGSHIPS Interim Evaluation. Final report 

•  Commission Staff Working Document, FET Flagships: A novel partnering approach to 

address grand scientific challenges and to boost innovation in Europe  

•  European Commission (2022). Horizon 2020 Work Programme 2018-2020 2. Future 

and Emerging 
Technologies;https://ec.europa.eu/research/participants/data/ref/h2020/wp/2018-
2020/main/h2020-wp1820-fet_en.pdf 

•  European Commission. (2015). Horizon 2020 indicators: Assessing the results and 

impact of Horizon 2020. 
https://ec.europa.eu/programmes/horizon2020/en/news/horizon-2020-indicators-
assessing-results-and-impact-horizon. 

•  European Commission. (2020) Interim evaluation of Horizon 2020 (key documents), 

available at https://ec.europa.eu/info/publications/interim-evaluation-horizon-2020_en 

•  European Commission. Study on assessing the contribution of the Framework 

Programmes to the development of human research capacity: 
http://ec.europa.eu/research/evaluations/pdf/archive/other_reports_studies_and_docu
ments/fp_hrc_study_final_report.pdf 

1011 

 
 
•  FLAG ERA (2021) D6.5 Landscape analysis in the domain of the Graphene Flagship  

•  FLAG-ERA II (2021) Deliverable D6.6 International collaboration funding opportunities 

for the Graphene Flagship  

•  Frégnac, Yves. (2021). How blue is the sky? Eneuro, 8(2). 

•  Graphene Flagship Annual Report: 2020 and 2021 https://graphene-

flagship.eu/research/annual-report/ 

•  Graphene Info (2022) https://www.graphene-info.com/ 

• 

• 

http://ec.europa.eu/information_society/newsroom/cf/dae/document.cfm?doc_id=6812  

https://ec.europa.eu/newsroom/document.cfm?doc_id=42760 

•  Markram, H. (2012). The human brain project. Scientific American, 306(6), 50-55 

•  The FET Flagship model: A brief overview and FAQs on Partnering Projects 

http://ec.europa.eu/information_society/newsroom/cf/dae/document.cfm?doc_id=8204  

Research Infrastructures under H2020 and Horizon Europe 

•  Commission implementing decision of 15.12.201 amending Implementing Decision 
C(2021)1940 final on the adoption of the work programme for 2021-2022 within the 
framework of the Specific Programme implementing Horizon Europe – the Framework 
Programme for Research and Innovation and on its financing, as regards Missions, 
available at https://eur-lex.europa.eu/legal-
content/EN/TXT/HTML/?uri=PI_COM:C(2021)9128&from=EN  

•  ENRIITC D3.2 Strategy for innovation and industry-RI cooperation, available at 

https://ec.europa.eu/research/participants/documents/downloadPublic?documentIds=0
80166e5e2ae7e05&appId=PPGMS  

•  ERIC Forum, Funding models for access to ERIC multinational / transnational services: 
Key recommendations https://www.eric-forum.eu/wp-content/uploads/2020/09/One-
pager_ERIC-Forum-policy-brief.pdf  

•  ESFRI cluster projects (2019), Position papers on expectations and planned 

contributions to the EOSC, accessible via 
https://zenodo.org/record/3675081#.YvJ29nZByUk  

•  ESFRI Roadmap 2016, available at https://www.esfri.eu/esfri_roadmap2016/ 

•  ESFRI White paper, 2020, available at https://www.esfri.eu/esfri-white-paper/24-

mobility-and-training-researchers-and-ri-staff  

•  ESFRI. Roadmap 2021: Projects and Landmarks. Available at: 

https://roadmap2021.esfri.eu/projects-and-landmarks/view-the-table/. 

1012 

 
 
•  European Commission (2022). Directorate-General for Research and Innovation, 

European Research Area policy agenda: overview of actions for the period 2022-2024, 
Publications Office of the European Union. Available at: 
https://data.europa.eu/doi/10.2777/52110. 

•  European Commission (2023). Directorate-General for Research and Innovation, 

Budraitis, M., Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European 
framework programmes for research and innovation for excellent science: Horizon 
2020: annexes, Publications Office of the European Union. Available at: 
https://data.europa.eu/doi/10.2777/353383. 

•  European Commission Decision C(2022)2975 of 10 May 2022: Horizon Europe Work 

Programme 2021-2022, Research Infrastructures. Available: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2021-2022/wp-3-research-infrastructures_horizon-2021-2022_en.pdf. 

•  European Commission Decision C(2023)2178 of 31 March 2023: Horizon Europe Work 

Programme 2023-2024, Research Infrastructures. Available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2023-2024/wp-3-research-infrastructures_horizon-2023-2024_en.pdf. 

•  European Commission, “Public consultation on the past, present and future of the 
European Research & Innovation Framework Programmes 2014-2027”. Available: 
https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-Horizon-
Europe-interim-evaluation/public-consultation_en.  

•  European Commission, Directorate-General for Research and Innovation, Sustainable 

European research infrastructures: A call for action: Commission staff working 
document: long-term sustainability of research infrastructures, Publications Office, 
2017, https://data.europa.eu/doi/10.2777/76269  

•  European Commission, Directorate-General for Research and Innovation, Europe 2020 

flagship initiative Innovation Union: SEC(2010) 1161, communication from the 
Commission to the European Parliament, the Council, the European Economic and 
Social Committee and the Committee of the Regions, Publications Office, 2011, 
https://data.europa.eu/doi/10.2777/47750  

•  European Commission, Directorate-General for Research and Innovation, Supporting 
the transformative impact of research infrastructures on European research: report of 
the High-Level Expert Group to assess the progress of ESFRI and other world class 
research infrastructures towards implementation and long-term sustainability, 
Publications Office, 2020, https://data.europa.eu/doi/10.2777/3423  

•  European Commission, Directorate-General for the Information Society and Media, 

Arnold, E., Hammerschmidt, R., Mahieu, B., et al., Evaluation of pertinence and impact 
of research infrastructure activity in FP7 - EPIRIA: final report, Publications Office, 
2014, https://data.europa.eu/doi/10.2759/62665  

•  Horizon Dashboard (accessed in August 2023). 

1013 

 
 
•  Kimmo Halme, 4FRONT Oy, 2015, Summary Report of the Capacities Specific 

Programme of FP7 for the High Level Expert Group, available at 
https://www.kowi.de/Portaldata/2/Resources/fp7/fp7-capacities-report.pdf  

•  Other websites: 

–  https://www.synchrotron-soleil.fr/en/ 

–  https://www.bbmri-eric.eu/news-events/apply-now-mba-research-

infrastructure-management/ 

–  https://training.prace-ri.eu/  

–  https://www.esfri.eu/national-roadmaps  

–  European Commission, European Research Infrastructure Consortium (ERIC), 

available at: https://research-and-innovation.ec.europa.eu/strategy/strategy-2020-
2024/our-digital-future/european-research-infrastructures/eric_en. 

–  https://ec.europa.eu/info/funding-

tenders/opportunities/portal/screen/opportunities/topic-details/infraeosc-04-2018.  

–  https://jive.eu/jiv-eric 
–  https://ec.europa.eu/info/funding-

tenders/opportunities/portal/screen/opportunities/topic-details/horizon-infra-2023-
eosc-01-
01;callCode=null;freeTextSearchKeyword=;matchWholeText=true;typeCodes=1 

•  Project websites: 

–  https://www.openaire.eu/   

–  https://prace-ri.eu/  

–  https://network.geant.org/ 

–  https://attract-eu.com/ 

–  https://cordis.europa.eu/project/id/101004761   

–  https://cordis.europa.eu/project/id/871112  

–  https://cordis.europa.eu/project/id/730957 

–  https://cordis.europa.eu/project/id/823798   

–  https://cordis.europa.eu/project/id/823711   

–  https://cordis.europa.eu/project/id/646713   

–  https://cordis.europa.eu/project/id/871072   

–  https://www.cremlinplus.eu/   

1014 

 
 
–  https://cordis.europa.eu/project/id/730974  

–  https://cordis.europa.eu/project/id/730879 

–  https://cordis.europa.eu/project/id/730957   

–  https://cordis.europa.eu/project/id/101008004 

–  https://cordis.europa.eu/project/id/730897   

–  https://cordis.europa.eu/project/id/871124  

–  https://cordis.europa.eu/project/id/654156   

–  https://cordis.europa.eu/project/id/101008503  

–  https://training.prace-ri.eu/  

–  https://cordis.europa.eu/project/id/823798   

–  https://cordis.europa.eu/project/id/676536 µ 

–  https://cordis.europa.eu/project/id/730890  

–  https://cordis.europa.eu/project/id/646713  

–  http://www.rich2020.eu/tas_calls 

–  https://cordis.europa.eu/project/id/730943  

•  Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 
2021 establishing Horizon Europe – the Framework Programme for Research and 
Innovation, laying down its rules for participation and dissemination, and repealing 
Regulations (EU) No 1290/2013 and (EU) No 1291/2013. 

•  Report from the Commission to the Council and the European Parliament, 2023, Third 
report on the application of Council Regulation (EC) No 723/2009 of 25 June 2009 on 
the Community legal framework for a European Research Infrastructure Consortium 
(ERIC). Available: https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=COM:2023:488:FIN. 

•  Research Infrastructures Work Programmes (2014-2015, 2016-2017, 2018-2020, 

2021-2022, 2023-2024). 

•  Rizzuto, C. et al. (2021). D3.2 Procurement rules, VAT exemptions practises and 

economic activities (ERIC Forum project). 

1015 

 
 
Spreading  Excellence  and  Widening  Participation  sources  and  WIDERA  under 
H2020 and Horizon Europe 

•  COM(2022)16 Final. Communication from the Commission to the European 

Parliament, the Council, the European Economic and Social Committee and the 
Committee of the Regions on a European Strategy for Universities.  

•  COM/2022/332 Final. Communication from the Commission to the European 

Parliament, the Council, the European Economic and Social Committee and the 
Committee of the Regions: A New European Innovation Agenda.  

•  Cordis webpages of selected projects. Available at: https://cordis.europa.eu/. 

•  EUR-Lex. Outermost Regions. Available at : https://eur-lex.europa.eu/EN/legal-

content/glossary/outermost-regions.html 

•  European Association of Research and Technology Organisations (EARTO). (2023). 

EARTO Answer to the EC Consultation on the past, present and future of the European 
Research & Innovation Framework Programmes 2014-2027. 

•  European Commission (2021). Directorate-General for Research and Innovation, The 

new European Research Area, Publications Office of the European Union. Available at: 
https://data.europa.eu/doi/10.2777/2736 

•  European Commission (2022). Commission Awards Seal of excellence for Outstanding 

Teaming-for-Excellence Proposals. Available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/commission-awards-
seal-excellence-outstanding-teaming-excellence-proposals-2022-12-19_en. 

•  European Commission (2022). Directorate-General for Education, Youth, Sport and 
Culture, Dėlkutė, R., Nikinmaa, J., Pupinis, M. et al., Study on mobility flows of 
researchers in the context of the Marie Sklodowska-Curie Actions – Analysis and 
recommendations towards a more balanced brain circulation across the European 
Research Area – Final report, Publications Office of the European Union. Available at: 
https://data.europa.eu/doi/10.2766/401134. 

•  European Commission (2023). Directorate-General for Research and Innovation, 

Budraitis, M., Pranckevičius, P., Dėlkutė, R. et al., Evaluation study of the European 
framework programmes for research and innovation for excellent science – Horizon 
2020 – Annexes, Publications Office of the European Union. Available at: 
https://data.europa.eu/doi/10.2777/353383. 

•  European Commission Decision C (2015)5668 of 14 August 2015, Horizon 2020 Work 

Programme 2014- 2015, 15. Spreading Excellence and Widening Participation 
(revised) 

•  European Commission Decision C(2017)2468 of 24 April 2017, Horizon 2020 Work 
Programme 2016- 2017, 15. Spreading Excellence and Widening Participation 
(revised) 

1016 

 
 
•  European Commission Decision C(2019)4575 of 2 July 2019, Horizon 2020 Work 
Programme 2018- 2020, 15. Spreading Excellence and Widening Participation 
(revised) 

•  European Commission Decision C(2022)2975 of 10 May 2022, Horizon Europe, Work 
Programme 2021-2022, 11. Widening participation and strengthening the European 
Research Area. 

•  European Commission Decision C(2022)7550 of 6 December 2022, Horizon Europe, 
Work Programme 2023-2024, 11. Widening participation and strengthening the 
European Research Area. 

•  European Commission, “Public consultation on the past, present and future of the 
European Research & Innovation Framework Programmes 2014-2027”. Available: 
https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-Horizon-
Europe-interim-evaluation/public-consultation_en.  

•  European Commission, Directorate-General for Research and Innovation, A new ERA 

for research and innovation: staff working document, Publications Office, 2020, 
https://data.europa.eu/doi/10.2777/605834    

•  European Commission, Directorate-General for Research and Innovation, Interim 
evaluation of Twinning and ERA chairs in Horizon 2020, Publications Office, 2018, 
https://data.europa.eu/doi/10.2777/998775 

•  European Commission, Directorate-General for Research and Innovation, Peck, F., 
Review of issues related to methods, criteria and indicators for widening actions, 
Publications Office, 2018, https://data.europa.eu/doi/10.2777/636491  

•  European Commission, Directorate-General for Research and Innovation, Puukka, J., 
Spreading excellence and widening participation in Horizon 2020: analysis of FP 
participation patterns and research and innovation performance of eligible countries, 
Publications Office, 2018, https://data.europa.eu/doi/10.2777/42382 

•  European Commission, Directorate-General for Research and Innovation, Meyer-

Krahmer, F., Grech, J., Sánchez, B., et al., Horizon 2020 Commission Expert Group for 
the evaluation of the Horizon 2020 Policy Support Facility: final report: (report-only 
version without annexes), Publications Office, 2020, 
https://data.europa.eu/doi/10.2777/433930 

•  European Commission, Directorate-General for Research and Innovation, Knowledge 
ecosystems in the new ERA : talent circulation and intersectoral mobility: pathways for 
a balanced talent circulation in EU, Núñez, L.(editor), Fraioli, M.(editor), Hoed, 
M.(editor), Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2777/83842 European Commission Decision 
C(2015)5668 of 14 August 2015, Horizon 2020 Work Programme 2014-2015, 15. 
Spreading Excellence and Widening Participation (revised)  

•  European Commission, Directorate-General for Research and Innovation (2023). 

Budraitis, M., Pranckevičius, P., Dėlkutė, R. et al., Evaluation study of the European 
framework programmes for research and innovation for excellent science – Horizon 

1017 

 
 
2020 – Phase 1 final study report, Publications Office of the European Union. Available 
at: https://data.europa.eu/doi/10.2777/967813. 

•  European Commission, European Research Executive Agency, Dobranskyte-Niskota, 
A., Fernandez-Wyttenbach, A., Salas, ERA chair dynamics in the widening programme 
– Report on geographical aspects (2014-2022), Publications Office of the European 
Union, 2023, https://data.europa.eu/doi/10.2848/719715 

•  European Commission, European Research Executive Agency, Spreading excellence 

and widening participation impact report: H2020 results and outlook to Horizon Europe, 
2022, https://data.europa.eu/doi/10.2848/30035; 

•  European Commission. Shaping EU Research and Innovation Policy – New European 

Innovation Agenda. Available at: https://research-and-
innovation.ec.europa.eu/strategy/support-policy-making/shaping-eu-research-and-
innovation-policy/new-european-innovation-agenda_en. 

•  European Court of Auditors, Measures to widen participation in Horizon 2020 were well 

designed but sustainable change will mostly depend on efforts by national authorities. 
Special report No 15, 2022, Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2865/359822 

•  European Regions Research and Innovation Network (ERRIN). (2023). Input To The 
Public Consultation On The Past, Present And Future Of The European Research & 
Innovation Framework Programmes 2014-2027.  

•  Foundation for Science and Technology Portugal (FCT), (2023). Position paper on the 
ex-post evaluation of Horizon 2020, and on the mid-term evaluation of Horizon Europe 
and its future. 

•  Horizon Europe Programme. Information document (2022). Testing of the ERA Hub 

concept – pilot phase. Accessed on October 20, 2023 through: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2022/information_horizon-widera-2022-era-01-30_en.pdf 

• 

• 

Initiative for Science in Europe (ISE). (2023). ISE Response to the European 
Commission Public Consultation on the past, present and future of the European 
Research & Innovation Framework programmes 2014-2027. 

JPI Urban Europe: https://jpi-urbaneurope.eu/news/expand-ii-helped-advance-urban-
living-labs/  

•  Masaryk University (MUNI). (2023). Contribution of Masaryk University to Public 

consultation on the past, present and future of the European Research & Innovation 
Framework programmes 2014-2027. 

•  Mutual Learning – PSF: https://ec.europa.eu/research-and-

innovation/en/statistics/policy-support-facility/mutual-learning  

•  Polish Chamber of Commerce for High Tech Technology (IZTECH) Working Group on 
EU Framework Programme Horizon Europe. (2023). Position Paper on the past, 

1018 

 
 
present and future of the European Research & Innovation Framework programmes 
2014-2027. 

•  Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 
2021 establishing Horizon Europe – the Framework Programme for Research and 
Innovation, laying down its rules for participation and dissemination, and repealing 
Regulations (EU) No 1290/2013 and (EU) No 1291/2013. 

•  Replies Of The European Commission To The European Court Of Auditors’ Special 
Report. Measures to widen participation in Horizon 2020 were well designed but 
sustainable change will mostly depend on efforts by National Authorities.  

•  Specific Support – PSF: https://ec.europa.eu/research-and-
innovation/en/statistics/policy-support-facility/specific-
support?field_rio_geo_coverage_target_id=All&page=1  

•  The Guild of European Research-Intensive Universities (2023). Lessons learned and 

way forward: The Guild’s recommendations for Horizon Europe. 

•  WIDERA (widening) webpages. 

–  European Commission. Excellence Hubs. Available at: 

https://rea.ec.europa.eu/funding-and-grants/horizon-europe-widening-
participation-and-spreading-excellence/excellence-hubs_en.  

–  European Commission. Pathways to Synergies. Available at: 

https://rea.ec.europa.eu/funding-and-grants/horizon-europe-widening-
participation-and-spreading-excellence/pathways-synergies_en. 

–  European Union. Funding & Tenders Portal. Available at: 

https://ec.europa.eu/info/funding-
tenders/opportunities/portal/screen/support/faq/30740, accessed 29/08/2023. 

Science with and for Society under H2020 

•  European Commission, Directorate-General for Research and Innovation, Stroyan, J., 
Franke, J., Wilkinson, C., Ex-post evaluation of science in society in FP7: final report, 
Publications Office, 2016, https://data.europa.eu/doi/10.2777/544495  

•  Regulation (EU) No 1291/2013 of the European Parliament and of the Council 

establishing Horizon 2020 (annex I), consulted through: 
https://ec.europa.eu/info/research-and-innovation/funding/funding-
opportunities/funding-programmes-and-open-calls/horizon-2020_en  

•  European Commission, Directorate-General for Research and Innovation, Delaney, N., 

Iagher, R., Tornasi, Z., Institutional changes towards responsible research and 
innovation: achievements in Horizon 2020 and recommendations on the way forward, 
Publications Office, 2020, https://data.europa.eu/doi/10.2777/682661  

•  European Commission, Directorate-General for Research and Innovation, Iagher, R., 

Monachello, R., Warin, C., Science with and for society in Horizon 2020: achievements 

1019 

 
 
and recommendations for Horizon Europe, Delaney, N.(editor), Tornasi, Z.(editor), 
Publications Office, 2020, https://data.europa.eu/doi/10.2777/32018  

•  RRI Tools, A practical guide to RRI, key lessons from RRI tools, https://rri-

tools.eu/documents/10184/16301/RRI+Tools.+A+practical+guide+to+Responsible+Res
earch+and+Innovation.+Key+Lessons+from+RRI+Tools  

•  Erwan Dagorne, Niklas Gudowky, CIMULACT Deliverable 5.1. Inspiration Catalogue 

for consulting different groups, available at http://www.cimulact.eu/wp-
content/uploads/2018/04/D5.1_Inspiration-Catalogue-for-consulting-different-groups-
compressed.pdf  

•  Mathias Wullum Nielsen, Niels Mejlgaard, Emil Alnor, Eric Griessler, Ingeborg 
Meijer,NEW HORRIZON, ensuring societal readiness, a thinking tool, 2018, 
https://newhorrizon.eu/wp-
content/uploads/2019/10/Deliverable_6.1_THINKING_TOOL.pdf  

•  FutureTDM, Deliverable D5.3, FutureTDM Practitioner Guidelines, 

https://project.futuretdm.eu/wp-content/uploads/2017/07/FutureTDM_D5.3-FutureTDM-
practitioner-guidelines.pdf  

•  ERAC Secretariat, ERAC Opinion on the European Research Area Roadmap 2015-

2020, 2015,  https://era.gv.at/era/era-2000-2021/era-roadmap/european-era-roadmap-
2015-2020/  

•  European Commission, Directorate-General for Research and Innovation, European 

Research Area policy agenda: overview of actions for the period 2022-2024, 
Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2777/52110  

•  Evaluation Study on the Implementation of the Cross-cutting Issues in Horizon 2020, 
Annex 8: Fostering the functioning and achievement of European Research Area, 
version of June 2022. 

•  SwafS Work Programmes (2014-2015, 2016-2017, 2018-2020) 

•  European Commission, Directorate-General for Research and Innovation, MORE4 : 

Support data collection and analysis concerning mobility patterns and career paths of 
researchers : survey on researchers in European higher education institutions, 
Publications Office, 2021, https://data.europa.eu/doi/10.2777/132356 

• 

Institute of Sociology Czech Academy of Sciences, GENDERACTION, Final 
Dissemination and Communication Report, 2021, https://genderaction.eu/wp-
content/uploads/2022/03/741466_D7_2_Communication-and-Dissemination-
Report_web.pdf 

•  Charter of Fundamental Rights of the European Union, 2012, https://eur-

lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:12012P/TXT  

•  Handbook for Organisations Aimed at Strengthening Responsible Research and 

Innovation (2019), available at: https://www.rri-practice.eu/wp-
content/uploads/2019/06/RRI-Practice-Handbook-for-Organisations.pdf  

1020 

 
 
• 

Joint declaration on mainstreaming RRI across Horizon Europe, 2020, available at 
https://www.tandfonline.com/doi/full/10.1080/23299460.2020.1764837  

•  Communication from the Commission to the European Parliament, the Council, the 
European Economic and Social Committee and the Committee of the regions - A 
Reinforced European Research Area Partnership for Excellence and Growth /* 
COM/2012/0392 final */, available at: https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=celex%3A52012DC0392  

•  Koerber, A. (2021). Is it Fake News or Is it Open Science? Science Communication in 

the COVID-19 Pandemic. J. Business Tech. Commun. 35 (1), 22–27. 
doi:10.1177/1050651920958506  

Efficiency of all analysed programme parts 

•  European Commission, Directorate-General for Research and Innovation, Interim 
Evaluation of Horizon 2020: commission staff working document, Annex 2, 2018, 
https://op.europa.eu/en/publication-detail/-/publication/4cedc9a0-d8cc-11e8-afb3-
01aa75ed71a1/language-en# 

•  European Commission, Research Executive Agency, Annual Activity Report 2022. 
https://commission.europa.eu/system/files/2023-06/REA_AAR_2022_final_en.pdf 

•  European Commission, Research Executive Agency, Annual Activity Report 2020, 
Annexes, https://ec.europa.eu/info/sites/default/files/annual-activity-report-2020-
research-executive-agency-annexes_en.pdf 

•  European Commission, Research Executive Agency, Annual Activity Report 2019, 

Annexes, https://ec.europa.eu/info/sites/default/files/rea_aar_2019_annexes_en.pdf 

•  European Commission, Research Executive Agency, Annual Activity Report 2018, 

Annexes, https://ec.europa.eu/info/sites/default/files/rea_aar_2018_annexes_final.pdf 

•  European Commission, Research Executive Agency, Annual Activity Report 2017, 

Annexes, 
https://ec.europa.eu/info/sites/default/files/file_import/rea_aar_2017_annexes_final.pdf 

•  European Commission, Research Executive Agency, Annual Activity Report 2016, 
Annexes, https://ec.europa.eu/info/sites/default/files/rea_aar_2016_annexes_0.pdf 

•  European Commission, Research Executive Agency, Annual Activity Report 2015, 
Annexes, https://ec.europa.eu/info/sites/default/files/activity-report-2015-rea-
annex_march2016_en.pdf 

•  European Commission, Research Executive Agency, Annual Activity Report 2015, 
Annexes, https://ec.europa.eu/info/sites/default/files/research_executive_agency_-
_activity_report_2014_annex.pdf 

•  European Commission, Study Supporting the Evaluation of the Research Executive 

Agency (2015-2018), Final Report, 2019, doi: 10.2777/299516 

1021 

 
 
•  European Commission, European Research Council Executive Agency, Annual Activity 
Report 2020, Annexes, https://ec.europa.eu/info/system/files/annual-activity-report-
2020-ercea-annexes_en.pdf 

•  European Commission, European Research Council Executive Agency, Annual Activity 

Report 2019, Annexes, 
https://ec.europa.eu/info/sites/default/files/ercea_aar_2019_annexes_en.pdf 

•  European Commission, European Research Council Executive Agency, Annual Activity 

Report 2018, Annexes, 
https://ec.europa.eu/info/sites/default/files/ercea_aar_2018_annexes_final.pdf.pdf  

•  European Commission, European Research Council Executive Agency, Annual Activity 

Report 2017, Annexes, 
https://ec.europa.eu/info/sites/default/files/file_import/rea_aar_2017_annexes_final.pdf  

•  European Commission, European Research Council Executive Agency, Annual Activity 
Report 2016, Annexes, https://ec.europa.eu/info/sites/default/files/file_import/aar-ercea-
2016_annex_en_0.pdf  

•  European Commission, European Research Council Executive Agency, Annual Activity 
Report 2015, Annexes,  https://ec.europa.eu/info/sites/default/files/activity-report-2015-
ercea-annex_march2016_en.pdf  

•  European Commission, European Research Council Executive Agency, Annual Activity 

Report 2014, Annexes, 
https://ec.europa.eu/info/sites/default/files/research_executive_agency_-
_activity_report_2014_annex.pdf  

•  European Commission, Evaluation of the European Research Council Executive 

Agency (ERCEA): commission staff working document, 2020 

•  European Commission, Evaluation of the Operation of ERCEA (2012-2015), Final 

Report, 2016, doi:10.2777/75945 

•  European Commission, Study Supporting the Evaluation of the European Research 
Council Executive Agency (2015-2018), Final Report, 2019, doi: 10.2777/487469 

•  European Commission, Study on the Proposal Evaluation System for the EU R&I 

framework programme, Annexes, 2020, doi: 10.2777/246820 

•  European Court of Auditors, The majority of simplification measure brought into 

Horizon 2020 have made life easier for beneficiaries, but opportunities to improve still 
exits, Special Report, 2018 

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 
Dėlkutė, R., Nikinmaa, J., Pupinis, M., et al., Study on mobility flows of researchers in 
the context of the Marie Sklodowska-Curie Actions: analysis and recommendations 
towards a more balanced brain circulation across the European Research Area: final 
report, 2022, https://data.europa.eu/doi/10.2766/401134 

1022 

 
 
•  European University Association, Member Consultation: A Contribution to the Horizon 

2020 Mid-Term Review, 2016 

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 
Study on international cooperation in the Marie Skłodowska-Curie actions: executive 
summary, Publications Office, 2019, https://data.europa.eu/doi/10.2766/932249 

•  European Commission, FP7 ex post and H2020 interim evaluation of Marie 

Sklodowska-Curie Actions (MSCA), Final Report, 2017, 
https://op.europa.eu/en/publication-detail/-/publication/98885a02-c849-11e7-9b01-
01aa75ed71a1/language-en/format-PDF/source-273411412 

•  European Commission, Report on the H2020 Simplification Survey, 24.09.2015 – 

23.10.2015, 
https://ec.europa.eu/research/participants/data/ref/h2020/other/events/survey/h2020_si
mplification-survey_final-report_en.pdf 

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 
Study of business participation and entrepreneurship in Marie Skłodowska-Curie 
actions (FP7 and Horizon 2020): final report, 2017, 
https://data.europa.eu/doi/10.2766/121877 

•  Veugelers, R., 2017. Supporting the next generation of biomedical researchers in 

Europe: The impact from ERC funding on career development for young biomedical 
researchers 

•  European Commission, Directorate-General for Research and Innovation, Ince, E., 

Janger, J., Knowledge ecosystems in the new ERA: talent circulation and intersectoral 
mobility. Analytical report with a mapping of talent mobility and causes of brain drain, 
Reda Nausedaite, R.(editor), Fraioli, M.(editor), Moreno, T.(editor), Colassin, C.(editor), 
Costas Comesaña, R.(editor), Núñez, L.(editor), Cincera, M.(editor), Ricksten, 
M.(editor), Anna Strauss-Kollin, A.(editor), Daniel Holmberg, D.(editor), Dudek, 
J.(editor), Hoed, M.(editor), Hofmann, K.(editor), Janger, J.(editor), Publications Office 
of the European Union, 2022, https://data.europa.eu/doi/10.2777/701074 

•  European Commission, Directorate-General for Research and Innovation, Iagher, R., 

Monachello, R., Warin, C., Science with and for society in Horizon 2020: achievements 
and recommendations for Horizon Europe, Delaney, N.(editor), Tornasi, Z.(editor), 
Publications Office, 2020, https://data.europa.eu/doi/10.2777/32018 

Phase 1 Quantitative research track findings 

•  Machlup, F. (1962). The production and distribution of knowledge in the United States 

(Vol. 278). Princeton university press 

•  Bloom, N., Jones, C. I., van Reenen, J., & Webb, M. (2020). Are ideas getting harder to 

find? American Economic Review, 110(4), 1104-1144. 
https://doi.org/10.1257/aer.20180338 

•  Mokyr, Joel. 2005. “Useful Knowledge as an Evolving System: The View from 

Economic History.” In The Economy As an Evolving Complex System III, edited by 

1023 

 
 
Lawrence E Blume and Steven N Durlauf. Oxford University Press. 
http://ideas.repec.org/p/esi/evopap/2004-23.html. 

•  Chu, Johan S. G., and James A. Evans. 2021. “Slowed Canonical Progress in Large 
Fields of Science.” Proceedings of the National Academy of Sciences 118 (41): 
2021636118. https://doi.org/10.1073/PNAS.2021636118 

•  Abramo, G., & D’Angelo, C. A. (2014). How do you define and measure research 

productivity? Scientometrics, 101(2), 1129-1144 

•  Reed, M. S., Ferré, M., Martin-Ortega, J., Blanche, R., Lawford-Rolfe, R., Dallimer, M., 
& Holden, J. (2021). Evaluating impact from research: a methodological framework. 
Research Policy, 50(4), 104147 

•  Bornmann, L., L. Leydesdorff and R. Mutz (2013). The use of percentiles and percentile 
rank classes in the analysis of bibliometric data: opportunities and limits. Journal of 
Informetrics, 7(1), 158–165 

•  European Commission (2022) Monitoring the open access policy of Horizon 2020 - 

Final report. https://doi.org/10.2777/268348. Available at:  
https://op.europa.eu/en/publication-detail/-/publication/56cc104f-0ebb-11ec-b771-
01aa75ed71a1 

•  European Commission (2022) Directorate-General for Research & Innovation - 

Associated Countries. Available at: 
https://ec.europa.eu/research/participants/data/ref/h2020/grants_manual/hi/3cpart/h202
0-hi-list-ac_en.pdf 

•  Skwarek-Maruszewska, A., Costa Dantas Faria, J. and Neicu, D., Bibliometric analysis 
of JRC’s research performance using Scopus/Scival tools (2014-2018), Internal Report, 
Joint Research Centre, Brussels, 2019. JRC119686.  

•  Costa Dantas Faria, J., Bibliometric analysis of JRC’s research performance using 
Scopus/Scival tools (2016-2020), Internal Report, Joint Research Centre, Brussels, 
2021. JRC127047.  

• 

Lehto, S. and Totaro, S., JRC contribution to the ex post evaluation of Horizon 2020 
and Euratom research and training programme 2014-2020: Analysis of peer-reviewed 
publications 2014-2020 and bibliometric indicators, Internal Report, Joint Research 
Centre, Brussels, 2022. 

•  European Commission (2022) Euratom Research and Training – Performance. 
Available at: https://ec.europa.eu/info/strategy/eu-budget/performance-and-
reporting/programme-performance-overview/euratom-research-and-training-
performance_en 

• 

• 

• 

https://en.wikipedia.org/wiki/Open_access 

https://www.scopus.com 

https://www.scival.com 

1024 

 
 
• 

https://osdg.ai/  

•  CORDIS data: https://data.europa.eu/data/datasets/cordish2020projects?locale=en 

•  PATSTAT data: https://www.epo.org/searching-for-patents/business/patstat.html  

Phase 2 Quantitative research track findings 

•  Aghion, P., Dewatripont, M., Stein, J.C., 2008. Academic freedom, private-sector focus, 

and the process of innovation. RAND J. Econ. 39, 617–635. doi:10.1111/J.1756-
2171.2008.00031.X 

•  Azoulay, P., Graff Zivin, J.S., Manso, G., 2011. Incentives and creativity: evidence from 

the academic life sciences. RAND J. Econ. 42, 527–554. doi:10.1111/J.1756-
2171.2011.00140.X 

•  Becher, T. & Trowler, P. (2001). Academic Tribes and Territories: Intellectual Enquiry 

and the Culture of Disciplines. 2nd ed. London: McGraw-Hill Education (UK). 

•  Blondel, V.D., Guillaume, J.-L., Lambiotte, R., Lefebvre, E., (2008). Fast unfolding of 

communities in large networks. J. Stat. Mech. Theory Exp. 2008, P10008. 

•  Bornmann, L. (2014) How are excellent (highly cited) papers defined in bibliometrics? A 

quantitative analysis of the literature, Research Evaluation, Vol. 23(2). 
https://doi.org/10.1093/reseval/rvu002 

•  Carayol, N., Agenor, L., Oscar, L., 2019. The Right Job and the Job Right: Novelty, 

Impact and Journal Stratification in Science. SSRN Electron. J. 
doi:10.2139/SSRN.3347326 

•  Chaturvedi, S., Prytkova, E., Ciarli, T., Nomaler, Ö., 2023. What is the Future of 

Automation? Using Semantic Analysis to Identify Emerging Technologies, PLLARS 
deliverable. PILLARS. 

•  Ciarli, T., Chavarro, D., Arza, V., Confraria, H., 2022. Study on factors impeding the 
productivity of research and the prospects for open science policies to improve the 
ability of the research and innovation. Brussels. 

•  Elsevier (2023) The ScienceDirect topic classification. Available at: 

https://www.sciencedirect.com/topics/ 

•  Elsevier (2023) Serial titles classified using the ASJC (All Science Journal 

Classification) scheme. Available at: 
https://service.elsevier.com/app/answers/detail/a_id/12007/supporthub/scopus/ 

•  European Commission, Directorate-General for Research and Innovation, A new 
horizon for Europe – Impact assessment of the 9th EU framework programme for 
research and innovation, Publications Office, 2018, 
https://data.europa.eu/doi/10.2777/194210. 

1025 

 
 
•  Foster, J.G., Shi, F. &Evans, J. (2021) “Surprise! Measuring Novelty as Expectation 

Violation.” SocArXiv. https://doi.org/10.31235/osf.io/2t46f 

•  Franzoni, C., Stephan, P., Veugelers, R., 2022. Funding Risky Research. Entrep. 

Innov. Policy Econ. 1, 103–133. doi:10.1086/719252 

•  Ghirelli, C., Havari, E., Meroni, E.C., Verzillo, S., 2023. The Long-Term Causal Effects 
of Winning an ERC Grant, IZA Discussion Papers. Institute of Labor Economics (IZA). 

•  Girvan, M. and Newman, M.E.J.(2002) Community structure in social and biological 

networks, PNAS, Vol. 99(12). https://doi.org/10.1073/pnas.122653799 

•  Grassano, N., Rotolo, D., Hutton, J., Lang, F., Hopkins, M.M., 2017. Funding Data from 
Publication Acknowledgments: Coverage, Uses, and Limitations. J. Assoc. Inf. Sci. 
Technol. 68, 999–1017. doi:10.1002/ASI.23737 

•  Guo, J., Pan, J., Guo, J., Gu, F. & Kuusisto, J. (2019) Measurement framework for 

assessing disruptive innovations, Technological Forecasting and Social Change, Vol. 
139. https://doi.org/10.1016/j.techfore.2018.10.015   

• 

• 

• 

Ioannidis, J.P.A., 2011. Fund people not projects. Nat. 2011 4777366 477, 529–531. 
doi:10.1038/477529a 

Jones, B., 2021. Science and Innovation: The Under-Fueled Engine of Prosperity, in: 
Kearney, M.S., Ganz, A. (Eds.), Rebuilding the Post-Pandemic Economy. Aspen 
Institute Press, Washington D.C., pp. 271–310. 

Jones, B.F., Summers, L.H., Azoulay, P., Bechtold, S., Harhoff, D., Jones, C., 
Schnitzer, M., Stern, S., Trajtenberg, M., Watzinger, M., Kennedy, H., 2020. A 
Calculation of the Social Returns to Innovation. Cambridge, MA. doi:10.3386/W27863 

•  Klavans, R. & Boyack, K.W. (2017) Research portfolio analysis and topic prominence, 

Journal of Informetrics, Vol. 11(4). https://doi.org/10.1016/j.joi.2017.10.002 

• 

Lin, Y., Evans, J.A., Wu, L., 2022. New directions in science emerge from 
disconnection and discord. J. Informetr. 16, 101234. doi:10.1016/J.JOI.2021.101234 

•  Office of the Director of National Intelligence (2023) ForeST: Forecasting Science & 

Technology. Available at: https://www.iarpa.gov/research-programs/forest 

•  Perianes-Rodriguez, A., Waltman, L. & Van Eck, N-J. (2016) Constructing bibliometric 

networks: A comparison between full and fractional counting, Journal of Informetrics, 
Vol. 10(4). https://doi.org/10.1016/j.joi.2016.10.006 

•  Pezzoni, M., Veugelers, R., Visentin, F., 2022. How fast is this novel technology going 
to be a hit? Antecedents predicting follow-on inventions. Res. Policy 51, 104454. 
doi:10.1016/j.respol.2021.104454 

•  Porter, A.L., Garner, J., Carley, S.F. & Newman, N.C. (2019) Emergence scoring to 

identify frontier R&D topics and key players, Technological Forecasting and Social 
Change, Vol. 146. https://doi.org/10.1016/j.techfore.2018.04.016 

1026 

 
 
•  Shibayama, S., Yin, D. & Matsumoto, K. (2021) Measuring novelty in science with word 

embedding, PLoS One, Vol. 16(7). https://doi.org/10.1371/journal.pone.0254034 

•  Small, H., Boyack, K.W. & Klavans, R. (2014) Identifying emerging topics in science 

and technology, Research Policy, Vol. 43(8). 
https://doi.org/10.1016/j.respol.2014.02.005. 

•  Waltman, L. & Schreiber, M. (2013) On the calculation of percentile-based bibliometric 
indicators, Journal of the American Society for Information Science and Technology, 
Vol. 64(2).  https://doi.org/10.1002/asi.22775 & 
https://arxiv.org/ftp/arxiv/papers/1205/1205.0646.pdf 

•  Wang, J., Lee, Y.N., Walsh, J.P., 2018. Funding model and creativity in science: 

Competitive versus block funding and status contingency effects. Res. Policy 47, 1070–
1083. doi:10.1016/J.RESPOL.2018.03.014 

•  Wang, J., Veugelers, R., Stephan, P., 2017. Bias against novelty in science: A 
cautionary tale for users of bibliometric indicators. Res. Policy 46, 1416–1436. 
doi:10.1016/j.respol.2017.06.006 

•  Wasserman, S. & Faust, K. (2012) Social Network Analysis: Methods and Applications. 
Chapter 14. Triads. Available at: https://www.cambridge.org/core/books/abs/social-
network-analysis/triads/2E61DF0F9CF8EB680BBD8A3B76F7E26C 

•  Wintjes, R. and Hollanders, H. (2011) Innovation pathways and policy challenges at the 
regional level: smart specialisation, MERIT Working Papers, No. 2011-027. Available 
at: https://econpapers.repec.org/paper/unmunumer/2011027.htm 

Phase 1 case studies 

Case study 1: ERC impact on creating new or pushing existing frontiers of science 

•  European Commission. (2017). Interim evaluation of Horizon 2020. Commission staff 

working document: annex 2. https://op.europa.eu/en/publication-detail/-
/publication/4cedc9a0-d8cc-11e8-afb3-01aa75ed71a1/language-en.   

•  European Commission. (2005). The European Challenge. 

http://erc.europa.eu/publication/frontier-research-european-challenge-high-level-expert-
group-report-0.   

•  Science Business. (2016). How the European Research Council came to be. (2016). 
Science Business. https://sciencebusiness.net/news/80035/How-the-European-
Research-Council-came-to-be. 

•  Chronology of Basic Research Policy in the European Research Area. 
https://ec.europa.eu/research/future/basic.research/brperaen.html 

•  Technology and report 2021. Catching technological waves: Innovation with equity. 

https://unctad.org/system/files/official-document/tir2020_en.pdf. 

•  European Commission. (2009). Green Paper on innovation. Document drawn up on the 
basis of COM(95) 688 final, Bulletin of the European Union Supplement 5/95, 2009. 

1027 

 
 
•  Commitment and coherence: Ex‐Post‐Evaluation of the 7th EU Framework Programme 

(2007‐2013). 
https://www.ffg.at/sites/default/files/downloads/page/fp7_final_evaluation_expert_group
_report.pdf.  

•  Barcelona European Council, 15 and 16 March 2002. Available at 

https://ec.europa.eu/invest-in-
research/pdf/download_en/barcelona_european_council.pdf.  

•  Council of the European Union. (2017). In-depth interim evaluation of Horizon 2020. 

https://www.eumonitor.nl/9353000/1/j4nvgs5kjg27kof_j9vvik7m1c3gyxp/vkeo7y9icmzx/f
=/9791_17_add_3.pdf.   

•  European Research Council. (2020). Qualitative Evaluation of completed Projects 

funded by the European Research Council. 
https://erc.europa.eu/sites/default/files/document/file/2021-qualitative-evaluation-
projects.pdf.  

• 

2021 Research Fronts. Institutes of Science and Development, Chinese Academy of 
Sciences. National Science Library. Clarivate. 
https://english.cas.cn/newsroom/news/202112/t20211208_293637.shtml.  

•  China Academy of Sciences. https://english.cas.cn/.  

•  US National Science Foundation. https://www.nsf.gov/  

•  Research Australia, https://researchaustralia.org/.  

Case  study  2:  Achievement  of  commercial  and/or  social  innovation  potential  of 
ERC projects that received ERC Proof of Concept funding 

•  European Commission (2017), Interim evaluation of Horizon 2020, Staff Working 

Document, and its Annex 1 and Annex 2, https://op.europa.eu/en/publication-detail/-
/publication/fad8c173-7e42-11e7-b5c6-01aa75ed71a1/language-en/format-
PDF/source-140646035.  

•  High-Level Expert Group Report (2015), Frontier Research: The European Challenge, 
https://erc.europa.eu/sites/default/files/document/file/high_level_expert_group_report_f
ull_report.pdf.  

•  European Research Council (2011), ERC Work Programme, 

https://erc.europa.eu/sites/default/files/document/file/erc_2011_work%20programme.pd
f.  

•  European Research Council (2021), Annual report on the ERC activities and 

achievements in 2020, https://erc.europa.eu/content/annual-report-erc-activities-and-
achievements-2020.  

Case study 5: Structuring impact of MSCA ITN on doctoral programmes 

•  Biswas, R., Schiller, A., Casolani, C., Daoud, E., Dode, A., Genitsaridi, E., Jacquemin, 

L., Liyanage, N., Lourenco, M., Makani, P., Parameshwarappa, V., Riha, C., l 

1028 

 
 
Santacruz, J., Shabbir, M., Simoes, J., Trpchevska, N., & Schoisswohl, S. (2021). 
Doctoral Studies as part of an Innovative Training Network (ITN): Early Stage 
Researcher (ESR) experiences. Open Research Europe, 1, 34. 
https://doi.org/10.12688/openreseurope.13094.2  

•  Doonan, F., Taylor, L., Branduardi, P., & Morrissey, J. P. (2018). Innovative training 
networks: overview of the Marie Skłodowska-Curie PhD training model. FEMS 
Microbiology Letters, 365(19). https://doi.org/10.1093/femsle/fny207   

•  EURAXESS LAC. (2019c). EURAXESS Brazil & Latin America and the Caribbean 

(LAC) - Quarterly Newsletter (No. 1). 
https://euraxess.ec.europa.eu/sites/default/files/qnl1_2019.pdf   

•  European Commission. (2011). Principles for Innovative Doctoral Training. Directorate-

General for Research and Innovation (RTD). 
https://euraxess.ec.europa.eu/sites/default/files/policy_library/principles_for_innovative
_doctoral_training.pdf   

•  European Commission. (2017a). FP7 ex post and H2020 interim evaluation of Marie 
Skłodowska-Curie actions (MSCA) (NC-02-17-778-EN-N). Luxembourg: Publications 
Office of the European Union. https://doi.org/10.2766/103174   

•  European Commission. (2017b). Interim Evaluation of Horizon 2020: Commission staff 
working document (KI-01-17-599-EN-C). Luxembourg: Publications Office of the 
European Union. https://doi.org/10.2777/78199  

•  European Commission. (2017c). Horizon 2020 Work Programme 2018–2020 - 3. Marie 

Skłodowska-Curie actions (C(2020)6320). 
https://ec.europa.eu/research/participants/data/ref/h2020/wp/2018-2020/main/h2020-
wp1820-msca_en.pdf   

•  European Commission. (2017d). Study of business participation and entrepreneurship 
in Marie Skłodowska-Curie actions (FP7 and Horizon 2020) (NC-02-17-274-EN-N). 
Luxembourg: Publications Office of the European Union. 
https://doi.org/10.2766/121877   

•  European Commission. (2019a). Study on International Cooperation in the Marie 

Skłodowska-Curie Actions (NC-03-19-797-EN-N). Luxembourg: Publications Office of 
the European Union. https://doi.org/10.2766/621682  

•  European Commission. (2020). Review of Marie Skłodowska- Curie actions unit costs 
in preparation for Horizon Europe (NC-04-20-171-EN-N). Luxembourg: Publications 
Office of the European Union. https://doi.org/10.2766/36137   

•  European Commission. (2021). Monitoring the open access policy of Horizon 2020 (KI-

05-21-227-EN-N). Luxembourg: Publications Office of the European Union. 
https://doi.org/10.2777/268348   

•  European Commission. (2021-2022). Study on Mobility Flows of Researchers in the 

Context of the MSCA. [2022].  

1029 

 
 
•  European Parliament & Council of the European Union. (2013). Regulation of the 

European Parliament and of the Council of 11 December 2013 establishing Horizon 
2020 - the Framework Programme for Research and Innovation (2014–2020) (No. 
1291/2013). Luxembourg: Publications Office of the European Union. 
http://data.europa.eu/eli/reg/2013/1291/oj  

•  Gilbert, R. (2009). The Doctorate as Curriculum: A Perspective on Goals and 

Outcomes of Doctoral Education. In Changing Practices of Doctoral Education, Ed. 
David Boud & Alison Lee, 54-68. New York: Routledge  

•  Hasgall, A., Saenen, B., Borrell-Damian, L., Deynze, F., Seeber, M., & Huisman, J. 
(2019b). Survey - Doctoral education in Europe today: approaches and institutional 
structures. European University Association. 
https://eua.eu/downloads/publications/online%20eua%20cde%20survey%2016.01.201
9.pdf  

• 

Idea Consult & CHEPS. (2013). Exploration of the implementation of the Principles for 
Innovative Doctoral Training in Europe: Final Report (ARES (2011) 932978). 
Luxembourg: Publications Office of the European Union. 
https://ris.utwente.nl/ws/portalfiles/portal/5141476/IDT+Final+Report+FINAL.pdf   

•  Kehm, B. M. (2020). Reforms of Doctoral Education in Europe and Diversification of 

Types. Structural and Institutional Transformations in Doctoral Education, 85–104. 
https://doi.org/10.1007/978-3-030-38046-5_4   

•  Meuskens, I., Higson, S., & Linke, D. (2019). Innovative training networks: a new way 
of collaboration-propped PhD training. Medical Microbiology and Immunology, 209(3), 
215–216. https://doi.org/10.1007/s00430-019-00647-0  

•  Puukka, J. (2018). Widening Participation in Horizon 2020 report: Analysis of FP 

participation patterns and R&I performance of eligible countries. (KI-03-18-051-EN-N). 
Luxembourg: Publications Office of the European Union. https://doi.org/10.2777/42382  

•  Sabic, N. (2014). Comparative Analysis of Doctoral Education in Europe. Annals of the 

University of Bucharest / Political science series, 16(1), 129-147. https://nbn-
resolving.org/urn:nbn:de:0168-ssoar-397492  

•  The ERA Steering Group for Human Resources and Mobility. (2014). Using the 

Principles for Innovative Doctoral Training as a Tool for Guiding Reforms of Doctoral 
Education in Europe: Report of the ERA Steering Group Human Resources and 
Mobility. ERA SGHRM. 
https://cdn5.euraxess.org/sites/default/files/principles_for_innovative_doctoral_training.
pdf 

Case study 6: FET Graphene Flagship 

•  Carrozza MC. et al. (2017) FET FLAGSHIPS Interim Evaluation. Final report 

https://ec.europa.eu/newsroom/document.cfm?doc_id=42760 

•  FET Flagships https://ec.europa.eu/digital-single-market/en/fet-flagships  

1030 

 
 
•  The FET Flagship model: A brief overview and FAQs on Partnering Projects 

http://ec.europa.eu/information_society/newsroom/cf/dae/document.cfm?doc_id=8204  

•  Commission Staff Working Document, FET Flagships: A novel partnering approach to 

address grand scientific challenges and to boost innovation in Europe 
http://ec.europa.eu/information_society/newsroom/cf/dae/document.cfm?doc_id=6812  

•  Graphene Flagship Annual Report: 2020 and 2021 https://graphene-

flagship.eu/research/annual-report/ 

•  Graphene Flagship project deliverables: FP7 D14.5 International collaboration report; 
FP7 D14.7 International collaboration report update; FP7 D15.1 TIR update; C1 D16.6 
Evaluation and recommendation report; C1 D19.4 Updated S&T Roadmap: Updated 
and upgraded roadmap document; C2 D18.1 PPs and AM Integration; C2 D18.2 
International collaborations; C2 D18.6 Analysis of flagship output (Core1); C2 D19.2 
Updated TIR; C2 D19.6 Overview of industry needs where GRM may provide solutions; 
C3 D16.1 Best Practices and IP Tools; C3 D16.7 Patent/Application output of the 
Graphene Flagship, FP7 through Core 2; C3 D18.3 Analysis of Flagship output; C3 
D18.8 PPs and AMs integration; C3 D18.9 “European funding landscape on Graphene 
and 2D materials” 

•  FLAG ERA (2021) D6.5 Landscape analysis in the domain of the Graphene Flagship  

•  FLAG‐ERA (2021) D6.4. Confidential deliverable 

•  FLAG ERA (2021) Deliverable D6.6 International collaboration funding opportunities for 

the Graphene Flagship 

Case study 7: FET Human Brain Flagship 

•  Amunts, K., Ebell, C., Muller, J., Telefont, M., Knoll, A., & Lippert, T. (2016). The 

human brain project: creating a European research infrastructure to decode the human 
brain. Neuron, 92(3), 574-581 

•  Carrozza MC. et al. (2017) FET FLAGSHIPS Interim Evaluation. Final report 

https://ec.europa.eu/newsroom/document.cfm?doc_id=42760 

•  FET Flagships https://ec.europa.eu/digital-single-market/en/fet-flagships 

•  Frégnac, Yves. (2021). How blue is the sky? Eneuro, 8(2) 

https://doi.org/10.1523/ENEURO.0130-21.2021 Available at: 
https://www.eneuro.org/content/8/2/eneuro.0130-21.2021 

•  The FET Flagship model: A brief overview and FAQs on Partnering Projects 

http://ec.europa.eu/information_society/newsroom/cf/dae/document.cfm?doc_id=8204  

•  European Commission staff working document FET Flagships: A novel partnering 
approach to address grand scientific challenges and to boost innovation in Europe 
http://ec.europa.eu/information_society/newsroom/cf/dae/document.cfm?doc_id=6812  

•  Human brain project https://www.humanbrainproject.eu/en/about/overview/ 

1031 

 
 
•  Horizon 2020 Work Programme 2018-2020 2. Future and Emerging Technologies 

https://ec.europa.eu/research/participants/data/ref/h2020/wp/2018-2020/main/h2020-
wp1820-fet_en.pdf 

•  HBP, 2020. SP11 Management and Coordination – Public Key Results for SGA2 Year 

2 

•  Markram, H. (2012). The human brain project. Scientific American, 306(6), 50-55 

•  Frégnac, Y., & Laurent, G. (2014). Neuroscience: Where is the brain in the Human 

Brain Project? Nature, 513(7516), 27-29. 

Case  study  8:  Impact  of  the  framework  programme  on  the  creation  of  new 
excellent services 

•  European Commission, Directorate-General for Research and Innovation, Solutions for 
a sustainable EOSC: a FAIR Lady (olim Iron Lady) report from the EOSC Sustainability 
Working Group, Publications Office, 2020, https://data.europa.eu/doi/10.2777/870770  

•  ESFRI (2008) European roadmap for Research Infrastructures, Roadmap 2008, 

available at https://www.esfri.eu/sites/default/files/esfri_roadmap_update_2008.pdf  

•  COM(2016) 178 final, European Cloud Initiative – Building a competitive data and 
knowledge economy in Europe, available at: https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=CELEX%3A52016DC0178  

•  SWD(2018) 83 final, Commission Staff Working Document, Implementation Roadmap 
for the European Open Science Cloud, https://ec.europa.eu/transparency/documents-
register/detail?ref=SWD(2018)83&lang=en  

•  ESFRI cluster projects (2019), Position papers on expectations and planned 

contributions to the EOSC, p.3, accessible via: 
https://zenodo.org/record/3675081#.YvJ29nZByUk  

•  Project websites:  

–  https://www.eric-forum.eu/  

–  https://cordis.europa.eu/project/id/824087 

–  https://cordis.europa.eu/project/id/824068  

–  https://cordis.europa.eu/project/id/823852  

–  https://cordis.europa.eu/project/id/824064  

–  https://cordis.europa.eu/project/id/823782  

–  https://cordis.europa.eu/project/id/863463  

–  https://cordis.europa.eu/project/id/863353 

1032 

 
 
–  https://cordis.europa.eu/project/id/863410  

–  https://cordis.europa.eu/project/id/863448  

–  https://cordis.europa.eu/project/id/863420 

–  https://projectescape.eu/services 

•  Other websites:  

–  https://research-and-innovation.ec.europa.eu/strategy/strategy-2020-2024/our-

digital-future/open-science/european-open-science-cloud-eosc_en  

–  https://www.eoscsecretariat.eu/eosc-governance/eosc-executive-board  

–  https://ec.europa.eu/info/funding-

tenders/opportunities/portal/screen/opportunities/topic-details/infraeosc-04-
2018 

–  https://ec.europa.eu/info/funding-

tenders/opportunities/portal/screen/opportunities/topic-details/infraeosc-02-
2019  

–  https://marketplace.eosc-portal.eu/   

–  https://human-organ-atlas.esrf.eu/ 

–  https://identify.plantnet.org/nl 

–  https://zenodo.org/record/6552377#.Y2jD6HbMKUl  

Case study 9: Fostering knowledge creation through transnational access 

•  Regulation (EU) No 1291/2013 of the European Parliament and of the Council 

establishing Horizon 2020 (annex I), consulted through: https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=celex%3A32013R1291 

•  European Commission, Directorate-General for Research and Innovation, Sustainable 

European research infrastructures: a call for action: Commission staff working 
document: long-term sustainability of research infrastructures, Publications Office, 
2017, https://data.europa.eu/doi/10.2777/264074  

•  European Commission, Directorate-General for Research and Innovation, European 

charter of access for research infrastructures: principles and guidelines for access and 
related services, Publications Office, 2016, https://data.europa.eu/doi/10.2777/524573  

•  European Commission (2011), Reporting transnational access and service activity 

costs. Available at https://ec.europa.eu/research/participants/data/ref/fp7/89703/fp7-
reporting-ta-service-activity-costs_en.pdf  

1033 

 
 
•  Commission Decision C(2013) 8199 of 10 December 2013 authorising the use of 

reimbursement on the basis of unit costs for actions involving trans-national access 
under the Research Infrastructures Part of the Horizon 2020 Framework Programme. 
Available at: 
https://ec.europa.eu/research/participants/data/ref/h2020/other/legal/unit_costs/unit-
costs_tna-infra_en.pdf   

•  European Commission, Directorate-General for Research and Innovation, Sustainable 

European research infrastructures: a call for action: Commission staff working 
document: long-term sustainability of research infrastructures, Publications Office, 
2017, https://data.europa.eu/doi/10.2777/76269 

•  Project websites:  

–  https://cordis.europa.eu/project/id/730897  

–  https://cordis.europa.eu/project/id/871124  

–  http://www.calipsoplus.eu/    

–  https://www.ensar2.eu/ 

–  https://cordis.europa.eu/project/id/653706   

–  https://www.laserlab-europe.eu/   

–  https://cordis.europa.eu/project/id/654168 

–  https://cordis.europa.eu/project/id/101007417 

–  https://cordis.europa.eu/project/id/730976  

–  https://cordis.europa.eu/project/id/730957  

–  https://cordis.europa.eu/project/id/101008004  

•  Other websites:  

–  https://www.esfri.eu/esfri-white-paper/23-access-ris-including-trans-national-

access 

–  https://ec.europa.eu/info/funding-

tenders/opportunities/portal/screen/opportunities/topic-details/infraia-03-2020   

–  https://www.synchrotron-soleil.fr/en/  

Case  study  11:  Contribution  of  framework  programme  in  integrating  research 
groups from widening countries 

•  References above  

1034 

 
 
 
•  COM(2020) 628 final. A new ERA for Research and Innovation {SWD(2020) 214 final}. 

Brussels, 30.9.2020. 

•  Horizon 2020: https://ec.europa.eu/programmes/horizon2020/en/h2020-

section/spreading-excellence-and-widening-participation 

•  Horizon Dashboard: https://ec.europa.eu/info/funding-

tenders/opportunities/portal/screen/opportunities/horizon-dashboard 

•  Evaluation study on the External coherence and synergies of Horizon 2020 within the 

European R&I system 

•  Evaluation study on the Relevance and internal coherence of Horizon 2020, and its 

policy mix. 

•  Evaluation study on the implementation of the cross-cutting issues in Horizon 2020 (in 
particular ‘Fostering the functioning and achievement of the European Research Area’, 
‘Widening participation across the Union in research and innovation and helping to 
close the research and innovation divide in Europe’, ‘Science and Society: Responsible 
Research and Innovation’, ‘Gender equality’). 

•  Meyer-Krahmer, F., Grech, J., Sánchez, B., Wilson, D. and Zagar, P. (2019): Final 

Report of the Horizon 2020 Commission Expert Group for the evaluation of the Horizon 
2020 Policy Support Facility. European Union. DOI: 10.2777/433930  https://eur-
lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52020DC0628&rid1  

•  European Research Area and Innovation Committee (2017), ERAC Opinion on the 
Interim Evaluation of Horizon 2020 and preparations for the next Framework 
Programme. 

•  European Commission (2020), Science, Research and Innovation Performance (SRIP) 

of the EU 2020. 

•  Research and Innovation Observatory – Horizon 2020 Policy Support Facility. 

Case  study  12:  Impact  of  framework  programme  in  improving  quality  (and 
coverage) of research in widening countries 

•  References above 

• 

Leiden Ranking: https://www.leidenranking.com/  

•  Horizon 2020: https://ec.europa.eu/programmes/horizon2020/en/h2020-

section/spreading-excellence-and-widening-participation 

•  Horizon Dashboard: https://ec.europa.eu/info/funding-

tenders/opportunities/portal/screen/opportunities/horizon-dashboard 

•  European Commission Decision C (2022)2975 of 10 May 2022, Horizon Europe. Work 
Programme 2021-2022. 11. Widening participation and strengthening the European 
Research Area. https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-

1035 

 
 
2027/horizon/wp-call/2021-2022/wp-11-widening-participation-and-strengthening-the-
european-research-area_horizon-2021-2022_en.pdf 

•  European Commission DG RTD – Unit G1, Kroll, A., Spreading Excellence and 

Widening Participation under Horizon 2020 and Horizon Europe. 
https://bestprac.eu/fileadmin/mediapool-bestprac/documents/WS-
Ghent/BESTPRAC_Spreading_Excellence_and_Widening_Participation_Adeline_Kroll
_EC.pdf 

•  Evaluation study on the External coherence and synergies of Horizon 2020 within the 

European R&I system 

•  Evaluation study on the Relevance and internal coherence of Horizon 2020, and its 

policy mix. 

•  Evaluation study on the implementation of the cross-cutting issues in Horizon 2020 (in 
particular ‘Fostering the functioning and achievement of the European Research Area’, 
‘Widening participation across the Union in research and innovation and helping to 
close the research and innovation divide in Europe’, ‘Science and Society: Responsible 
Research and Innovation’, ‘Gender equality’). 

•  Meyer-Krahmer, F., Grech, J., Sánchez, B., Wilson, D. and Zagar, P. (2019): Final 

Report of the Horizon 2020 Commission Expert Group for the evaluation of the Horizon 
2020 Policy Support Facility. European Union. DOI: 10.2777/433930  https://eur-
lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52020DC0628&rid1  

•  European Research Area and Innovation Committee (2017), ERAC Opinion on the 
Interim Evaluation of Horizon 2020 and preparations for the next Framework 
Programme. 

•  European Commission (2020), Science, Research and Innovation Performance (SRIP) 

of the EU 2020. 

•  Research and Innovation Observatory – Horizon 2020 Policy Support Facility. 

Case  study  13:  Contribution  of  the  Framework  Programme  to  some  emerging 
areas  of  science  and  technology  such  as  artificial  intelligence,  quantum 
computing, clean energy technologies 

•  A new approach to measuring government investment in AI-related R&D - OECD.AI. 

(2021). OECD.AI Policy Observatory. https://oecd.ai/en/wonk/government-investment-
ai-related-r-and-d    

•  Canada lays the groundwork to become a powerhouse in quantum technology. (2022). 
Science|Business. https://sciencebusiness.net/news/canada-lays-groundwork-become-
powerhouse-quantum-technology    

•  Communication on 'Accelerating Clean Energy Innovation' (ACEI) adopted in 2016. 

COM (2016) 763.  

1036 

 
 
•  Correia, A., & Reyes, I. (2020). AI research and innovation: Europe paving its own way, 
R&I paper series, working paper 2020/15. Publications Office of the European Union. 
https://doi.org/10.2777/264689   

•  European Commission. (2018) Interim evaluation of Horizon 2020, Staff Working 

Document.   

•  European Research Council. (2021). Mapping ERC Frontier Research.   

•  Giles, M. (2022, June 9). President Trump has signed a $1.2 billion law to boost US 

quantum tech. MIT Technology Review. 
https://www.technologyreview.com/2018/12/22/138149/president-trump-has-signed-a-
12-billon-law-to-boost-us-quantum-tech/    

• 

Johansson, M.P., Krishnasamy E., Meyer, N., Piechurski, C. (2021). Quantum 
Computing – A European Perspective. Partnership for Advanced Computing in Europe.   

•  Kurek, M. (2020). Quantum Technologies Patents, Publications & Investments.  

•  Midterm Report of the Quantum Technologies Flagship, September 2020.  

•  Nurton, J. (2020). Patenting Trends in Renewable Energy. World Intellectual Property 
Organisation. https://www.wipo.int/wipo_magazine/en/2020/01/article_0008.html   

•  OECD. (2019). Measuring the Digital Transformation: A Roadmap for the Future  

•  The Foundation for Law and International Affairs. (2017). Notice of the State Council 

Issuing the New Generation of Artificial Intelligence Development Plan. 
https://www.unodc.org/ji/en/resdb/data/chn/2017/new_generation_of_artificial_intelligen
ce_development_plan.html   

Case  study  14:  Impact  of  the  Framework  Programme  on  fostering  diffusion  of 
knowledge and open science 

•  EOSC Executive Board reports: https://www.eoscsecretariat.eu/eosc-governance/eosc-

executive-board-outputs 

•  European Commission, Horizon 2020 Online Manual: Data Management. 

https://ec.europa.eu/research/participants/docs/h2020-funding-guide/index_en.htm  

•  European Commission, 2017, Guidelines to the Rules on Open Access to Scientific 

Publications and Open Access to Research Data in Horizon 2020. 
https://ec.europa.eu/research/participants/data/ref/h2020/grants_manual/hi/oa_pilot/h2
020-hi-oa-pilot-guide_en.pdf  

•  European Commission, 2019, Open Science. 

https://ec.europa.eu/info/sites/default/files/research_and_innovation/knowledge_public
ations_tools_and_data/documents/ec_rtd_factsheet-open-science_2019.pdf  

•  European Commission, Directorate-General for Research and Innovation, 2016, 

European charter of access for research infrastructures: principles and guidelines for 

1037 

 
 
access and related services, Publications Office, 
https://data.europa.eu/doi/10.2777/524573  

•  European Commission, Directorate-General for Research and Innovation, 2021, 

Monitoring the Open Access Policy of Horizon 2020: final report, Publications Office, 
https://data.europa.eu/doi/10.2777/268348  

•  European Commission, Directorate-General for Research and Innovation, 2020, 

Iagher, R., Open access to publications and data: achievements in Horizon 2020 and 
recommendations on the way forward, Tornasi, Z.(editor), Delaney, N.(editor), 
Publications Office, https://data.europa.eu/doi/10.2777/28803  

•  European Commission, 2020, A European strategy for data. COM(2020) 66 final. 

2020a.   

•  European Commission, Directorate-General for Research and Innovation, 2017, 

O'Carroll, C., Hyllseth, B., Berg, R., et al., Providing researchers with the skills and 
competencies they need to practise Open Science, Publications Office, 2017, 
https://data.europa.eu/doi/10.2777/121253  

•  European Commission, Directorate-General for Research and Innovation, 2020, Baker, 
L., Cristea, I., Errington, T., et al., Reproducibility of scientific results in the EU: scoping 
report, Lusoli, W.(editor), Publications Office, 2020, 
https://data.europa.eu/doi/10.2777/341654  

•  European Commission, 2020, Proposal for a Regulation of the European Parliament 

and of the Council on European data governance (Data Governance Act). 
COM/2020/767 final. 2020b. European Commission, Directorate-General for Research 
and Innovation, 2017, Evaluation of research careers fully acknowledging Open 
Science practices : rewards, incentives and/or recognition for researchers practicing 
Open Science, Cabello Valdes, C.(editor), Rentier, B.(editor), Kaunismaa, E.(editor), 
Metcalfe, J.(editor), Esposito, F.(editor), McAllister, D.(editor), Maas, K.(editor), 
Vandevelde, K.(editor), O'Carroll, C.(editor), Publications Office, 2017, 
https://data.europa.eu/doi/10.2777/75255  

•  Van Hesteren, D. et al, 2021, European Data Portal: Open Data Maturity in Europe. 

Report 2021. European Data Portal. 
https://euagenda.eu/upload/publications/landscaping-insight-report-n7-2021.pdf  

• 

Leonelli, S., 2013, Why the Current Insistence on Open Access to Scientific Data? Big 
Data, Knowledge Production, and the Political Economy of Contemporary Biology. Bull 
Sci Technol Soc. 2013; 33(1–2): 6–11.   

•  Neylon, C., 2017, Compliance Culture or Culture Change? The role of funders in 

improving data management and sharing practice amongst researchers. Res Ideas 
Outcomes. 3: e14673.  

•  Science Europe, 2019, Practical Guide To the International Alignment of Research 
Data Management. https://www.scienceeurope.org/our-resources/practical-guide-to-
the-international-alignment-of-research-data-management/  

1038 

 
 
•  Science Europe, 2021, Practical Guide to Sustainable Research Data 

https://scienceeurope.org/our-resources/practical-guide-to-sustainable-research-data/  

•  Spichtinger D, Siren J: The Development of Research Data Management Policies in 

Horizon 2020. In Research Data Management - A European Perspective. 
Berlin/Boston: De Gruyter SAUR, 2018; 11–23.  

•  Spichtinger D. Data Management Plans in Horizon 2020: what beneficiaries think and 

what we can learn from their experience [version 2; peer review: 2 approved, 1 
approved with reservations]. Open Res Europe 2022, 1:42. 
https://doi.org/10.12688/openreseurope.13342.2  

•  Case study 15: Impact of the Framework Programme in spreading excellence across 

the Union 

•  Horizon 2020. Work Programme 2018-2020. 15. Spreading Excellence and Widening 

Participation. European Commission Decision C(2019)4575 of 2 July 2019. 

•  European Commission, European Research Executive Agency, Spreading excellence 

and widening participation impact report: H2020 results and outlook to Horizon Europe, 
2022, https://data.europa.eu/doi/10.2848/30035 

• 

https://research-and-innovation.ec.europa.eu/funding/funding-opportunities/funding-
programmes-and-open-calls/horizon-2020_en 

• 

https://ec.europa.eu/programmes/horizon2020/en/h2020-section/excellent-science 

•  Horizon Dashboard, https://ec.europa.eu/info/funding-

tenders/opportunities/portal/screen/opportunities/horizon-dashboard 

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 
Dėlkutė, R., Nikinmaa, J., Pupinis, M., et al., Study on mobility flows of researchers in 
the context of the Marie Skłodowska-Curie Actions: analysis and recommendations 
towards a more balanced brain circulation across the European Research Area: final 
report, Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2766/401134    

• 

• 

• 

• 

• 

https://marketplace.eosc-portal.eu/services/inode-open-data-dialog 

https://cordis.europa.eu/project/id/863410 

https://marketplace.eosc-portal.eu/services/cos4bio 

https://cordis.europa.eu/project/id/863463 

Interim Evaluation of Horizon 2020, 2017. https://research-and-
innovation.ec.europa.eu/knowledge-publications-tools-and-data/publications/all-
publications/interim-evaluation-horizon-2020-book-version_en 

•  Assessment of the Union Added Value and the economic impact of the EU Framework 

Programmes (FP7, Horizon 2020). https://op.europa.eu/en/publication-detail/-
/publication/af103c38-250d-11e9-8d04-01aa75ed71a1/language-en 

1039 

 
 
•  Case study on the impact of SEWP in improving quality (and coverage) of research in 

widening countries 

•  Case study on achievement of commercial and/or social innovation potential of ERC 

projects that received ERC Proof of Concept funding 

•  Case study on structuring impact of MSCA ITN on doctoral programmes 

•  Case study on the contribution of the Framework Programme to some emerging areas 
of science and technology such as artificial intelligence, quantum computing, clean 
energy technologies 

•  Case study on the ERC impact on creating new or pushing existing frontiers of science 

•  Case study on the impact of MSCA IF on strengthening human capital in research and 

innovation 

•  Case study on the impact of the Framework Programme on the creation of new 

excellent services 

•  Case study on fostering knowledge creation through transnational or virtual access 

(TNA/VA) 

•  Case study on the contribution of SEWP to integrating research groups from widening 

countries  

Phase 2 case studies 

The list of literature and information sources consulted while preparing Phase 2 case studies will 
be compiled and provided together with the Second Interim Report. 

Case  Study  1:  Contribution  of  the  ERC  to  emerging  areas  in  the  Life  Sciences, 
Physical Sciences and Engineering 

•  Commission Implementing Decision adopting the 2023 Work Programme, in the 

framework of the Specific Programme Implementing Horizon Europe – The Framework 
Programme for Research and Innovation (2021-2027) in relation to the component 
‘European Research Council' under Pillar I ‘Excellent Science’ – C/2022/4861 final. 
https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=PI_COM:C(2022)4861. 

•  Cordis webpages of selected projects. 

•  DATAVERSE, Replication Data for the Comparison of Nationally Funded Outputs – 

EU27 and selected third countries, https://doi.org/10.34894/DNWIXV. 

•  ERC (2022). ERC Grantees Awarded Nobel Prizes. Available at: 

https://erc.europa.eu/news-events/events/follow-nobel-week-3-erc-grantees-awarded-
2022-nobel-prizes.  

•  ERC (2022). Press Release. Available at: https://erc.europa.eu/news-

events/news/2022-Nobel-physics-prize. 

1040 

 
 
•  ERC at a glance. Available at: https://erc.europa.eu/about-erc/erc-glance. 

•  ERC (2022). Assessing the Influence of ERC-funded Research on Patented Inventions, 

https://erc.europa.eu/sites/default/files/2023-01/Assessing_the_Influence_ERC-
funded_Research_Patented_Inventions.pdf. 

•  ERC (2023). How Frontier Research Spurs Patented Inventions. Available at: 

https://erc.europa.eu/news-events/news/new-study-reveals-how-frontier-research-
spurs-patented-inventions. 

•  ERC (2022). ERC Funded Research Wins New EU Innovation Grants. Available at: 

https://erc.europa.eu/news/erc-funded-research-wins-most-new-eu-innovation-grants. 

•  European Commission, Directorate-General for Research and Innovation, Science, 

research and innovation performance of the EU 2022 – Building a sustainable future in 
uncertain times, Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/78826. 

•  European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European framework 
programmes for research and innovation for excellent science: Horizon 2020: phase 1 
final study report, Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/967813. 

•  European Commission, European Research Council Executive Agency, Mapping ERC 
frontier research Horizon 2020 Research and Innovation Framework Programme 
(2014-2020), Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2828/636104. 

•  European Parliament and Council (2021). Regulation (EU) 2021/695 of the European 
Parliament and of the Council of 28 April 2021 establishing Horizon Europe – the 
Framework Programme for Research and Innovation, laying down its rules for 
participation and dissemination, and repealing Regulations (EU) No 1290/2013 and 
(EU) No 1291/2013, 28 April 2021, PE/12/2021/INIT, OJ L 170, 12.5.2021, pp. 1-68. 
Available at: http://data.europa.eu/eli/reg/2021/695/oj.   

•  European Research Council (2020). COVID-19 Frontier research in the spotlight. 

Available at: https://erc.europa.eu/sites/default/files/2022-08/COVID19-
Frontier_research_in_the-spotlight.pdf.  

•  European Science Vocabulary. Available at: https://op.europa.eu/en/web/eu-

vocabularies/dataset/-
/resource?uri=http://publications.europa.eu/resource/dataset/euroscivoc. 

•  Horizon Dashboard (accessed 6 July 2023). 

•  Nobel prize winners who have received NSF funding. 

https://www.nsf.gov/news/special_reports/nobelprizes/. 

1041 

 
 
Case Study 2: Contribution of the ERC to emerging areas in the Social Sciences 
and Humanities  

•  Commission Implementing Decision adopting the 2023 Work Programme, in the 

framework of the Specific Programme Implementing Horizon Europe – The Framework 
Programme for Research and Innovation (2021-2027) in relation to the component 
‘European Research Council' under Pillar I ‘Excellent Science’ - C/2022/4861 final, 
available at: https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=PI_COM:C(2022)4861. 

•  Cordis webpages of selected projects. 

•  ERC at a glance. Available at: https://erc.europa.eu/about-erc/erc-glance. 

•  European Commission, Directorate-General for Research and Innovation, Science, 

research and innovation performance of the EU 2022 – Building a sustainable future in 
uncertain times, Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/78826  

•  European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European framework 
programmes for research and innovation for excellent science: Horizon 2020: phase 1 
final study report, Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/967813.  

•  European Commission, European Research Council Executive Agency, Mapping ERC 
frontier research Horizon 2020 Research and Innovation Framework Programme 
(2014-2020), Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2828/636104.    

•  European Parliament and Council (2021). Regulation (EU) 2021/695 of the European 
Parliament and of the Council of 28 April 2021 establishing Horizon Europe – the 
Framework Programme for Research and Innovation, laying down its rules for 
participation and dissemination, and repealing Regulations (EU) No 1290/2013 and 
(EU) No 1291/2013, 28 April 2021, PE/12/2021/INIT, OJ L 170, 12.5.2021, pp. 1-68. 
Available at: http://data.europa.eu/eli/reg/2021/695/oj    

•  European Science Vocabulary. Available at: https://op.europa.eu/en/web/eu-

vocabularies/dataset/-
/resource?uri=http://publications.europa.eu/resource/dataset/euroscivoc. 

•  Field-Weighted Citation Impact in SciVal. Available at: 

https://www.elsevier.com/__data/assets/pdf_file/0020/53327/ELSV-13013-Elsevier-
Research-Metrics-Book-r12-WEB.pdf. 

•  Horizon Dashboard (accessed 6 July 2023). 

• 

• 

https://en.wikipedia.org/wiki/Holberg_Prize. 

https://en.wikipedia.org/wiki/Stein_Rokkan_Prize_for_Comparative_Social_Science_Re
search. 

1042 

 
 
• 

https://en.wikipedia.org/wiki/Swedish_Society_for_Anthropology_and_Geography#Retz
ius_Medal. 

Case Study 3: The structuring impacts of the MSCA 

•  Case Study on „Structuring Impact of MSCA ITN on Doctoral Programmes“  

•  ERA Steering Group for Human Resources and Mobility. “Using the Principles for 

Innovative Doctoral Training as a Tool for Guiding Reforms of Doctoral Education in 
Europe: Report of the ERA Steering Group Human Resources and Mobility.,” 2014. 
https://euraxess.ec.europa.eu/sites/default/files/policy_library/sghrm_idtp_report_final.p
df   

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 
Marie Skłodowska-Curie actions guidelines on supervision, Publications Office, 2021. 
https://data.europa.eu/doi/10.2766/508311  

•  European Commission, Directorate-General for Research and Innovation, The 
European Charter for Researchers – The code of conduct for the recruitment of 
researchers, Publications Office, 2006. 

•  European Commission. (2023-forthcoming). Evaluation study on Excellent Science in 
the European Framework Programmes for Research and Innovation, Annexes. (Case 
Study on „structuring impact of MSCA ITN on doctoral programmes).   

•  European Commission. “Commission Communication on a European Strategy for 

Universities,” 2022.  

•  European Commission. Directorate General for Education, Youth, Sport and Culture. 
FP7 Ex Post and H2020 Interim Evaluation of Marie Skłodowska-Curie Actions 
(MSCA): Final Report. LU: Publications Office, 2017. 
https://data.europa.eu/doi/10.2766/103174  

•  European Commission. Directorate General for Education, Youth, Sport and Culture. 
Interim Evaluation of Horizon 2020: Commission Staff Working Document: Annex 2, 
2017. https://op.europa.eu/en/publication-detail/-/publication/4cedc9a0-d8cc-11e8-
afb3-01aa75ed71a1/language-en.  

•  European Commission. Directorate General for Research and Innovation. and Centre 
for Strategy & Evaluation Services LLP (CSES). Towards a 2030 Vision on the Future 
of Universities in Europe. LU: Publications Office, 2020. 
https://data.europa.eu/doi/10.2777/510530.  

•  Hasgall, Alexander, and Ana Maria Peneoasu. “Doctoral Education in Europe: Current 

Developments and Trends,” 2022.  

• 

Lambert, W. Marcus, Nanda Nana, Suwaiba Afonja, Ahsan Saeed, Avelino C. Amado, 
and Linnie M. Golightly. “Addressing Structural Mentoring Barriers in Postdoctoral 
Training: A Qualitative Study.” Preprint. Scientific Communication and Education, 
August 23, 2022. https://doi.org/10.1101/2022.08.20.504665.  

1043 

 
 
•  Sabic, Norbert. “Comparative Analysis of Doctoral Education in Europe.” Analele 

Universităţii Din Bucureşti. Seria Ştiinţe Politice 16, no. 1 (2014): 129–47.  

•  Steen, Kaylee, Jay Vornhagen, Zara Y. Weinberg, Julie Boulanger-Bertolus, Arvind 
Rao, Margery Evans Gardner, and Shoba Subramanian. “A Structured Professional 
Development Curriculum for Postdoctoral Fellows Leads to Recognized Knowledge 
Growth.” Edited by Sina Safayi. PLOS ONE 16, no. 11 (November 22, 2021): 
e0260212. https://doi.org/10.1371/journal.pone.0260212. 

Case Study 4: How the MSCA contribute to more sustainable and greener research  

•  Council Recommendation of 16 June 2022 on learning for the green transition and 

sustainable development 2022/C 243/01   

•  Eco-Management and Audit Scheme (EMAS). (n.d.). Green Business. https://green-

business.ec.europa.eu/eco-management-and-audit-scheme-emas_en     

•  European Commission Decision C (2021)4200 of 15 June 2021, Horizon Europe Work 

Programme 2021- 2022, 2. Marie Skłodowska-Curie Actions.  

•  European Commission Decision C (2022)7550 of 6 December 2022, Horizon Europe 

Work Programme 2023- 2024, 2. Marie Skłodowska-Curie Actions.  

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 

Learning for the green transition and sustainable development: staff working document 
accompanying the proposal for a Council recommendation on learning for 
environmental sustainability, Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2766/02392   

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 

Tasiopoulou, E., Billon, N., Finlayson, A., et al., Education for environmental 
sustainability : policies and approaches in European Union Member States : final 
report, Tasiopoulou, E. (editor), Billon, N. (editor), Finlayson, A. (editor), Siarova, H. 
(editor), Pribuišis, K. (editor), Gras-Velazquez, A. (editor), Mulvik, I. (editor), Bajorinaitė, 
M. (editor), Sabaliauskas, E. (editor), Fronza, V. (editor), Vežikauskaitė, J. (editor), 
Disterheft, A. (editor), Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2766/391    

•  European Commission, Directorate-General for Research and Innovation, Kadunc, M., 
Bruno, N., Keeping our eyes on the Horizon: monitoring flash series: a Horizon 2020 
monitoring report, Kadunc, M. (editor), Bruno, N. (editor), Publications Office, 2020, 
https://data.europa.eu/doi/10.2777/303024    

•  European Commission, European Research Executive Agency, MSCA cluster event on 

the European Green Deal: meeting report and key messages for policy consideration, 
Publications Office of the European Union (2022).   

•  European Commission, Joint Research Centre, GreenComp, the European 

sustainability competence framework, Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2760/13286   

1044 

 
 
•  European Parliament, Directorate-General for Internal Policies of the Union, Schreiber, 
H., Tisch, A., Jones, M., et al., Green public procurement and the EU action plan for the 
circular economy, European Parliament, 2017, https://data.europa.eu/doi/10.2861/3120     

•  Evaluation study on Excellent Science in the EU framework programmes for Research 

and Innovation. Phase 1 Final Report and its annexes.   

•  Marie Skłodowska-Curie Actions Green Charter – Guidance Materials (2021).  

•  Marie Skłodowska-Curie Actions Green Charter (2021).  

•  Science Europe. (n.d.). Greening Research Organisations: Reducing our Carbon 
Footprint. https://www.scienceeurope.org/news/greening-research-organisations-
reducing-our-carbon-footprint/. 

Case  Study  5:  Analysis  of  INFRA’s  impact  on  transnational  and  virtual  access 
support 

•  CEA (2023). Position Paper regarding the ‘Public consultation on the past, present and 
future of the European Research & Innovation Framework programmes 2014-2027’, 
available at: https ://ec.europa.eu/info/law/better-regulation/have-your-
say/initiatives/13460-Horizon-Europe-interim-evaluation/public-consultation_en. 

•  Cordis webpages of selected projects. 

•  EARTO (2023). Position Paper regarding the ‘Public consultation on the past, present 
and future of the European Research & Innovation Framework programmes 2014-
2027’, available at: https://ec.europa.eu/info/law/better-regulation/have-your-
say/initiatives/13460-Horizon-Europe-interim-evaluation/public-consultation_en. 

•  ERIC Forum (2020). Policy Brief, Funding models for Access to ERIC 

Multinational/Transnational Services, p. 25, Available at: https://www.eric-forum.eu/wp-
content/uploads/2020/09/ERIC-Forum_Policy-Brief.pdf 

•  ERIC Forum (2023). Position Paper regarding the ‘Public consultation on the past, 

present and future of the European Research & Innovation Framework programmes 
2014-2027’, available at: https://ec.europa.eu/info/law/better-regulation/have-your-
say/initiatives/13460-Horizon-Europe-interim-evaluation/public-consultation_en. 

•  European Commission Decision C(2022)2975 of 10 May 2022: Horizon Europe Work 

Programme 2021-2022, Research Infrastructures, p. 61, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2021-2022/wp-3-research-infrastructures_horizon-2021-2022_en.pdf. 

•  European Commission, Directorate-General for Research and Innovation, European 

Research Area policy agenda: overview of actions for the period 2022-2024, 
Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/52110. 

•  European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European framework 
programmes for research and innovation for excellent science: Horizon 2020: annexes, 

1045 

 
 
Publications Office of the European Union, pp. 848-858, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383. 

•  Helmholtz Association (2023). Position Paper regarding the ‘Public consultation on the 

past, present and future of the European Research & Innovation Framework 
programmes 2014-2027’, available at: https://ec.europa.eu/info/law/better-
regulation/have-your-say/initiatives/13460-Horizon-Europe-interim-evaluation/public-
consultation_en. 

•  Horizon 2020 Call ‘Implementing the European Open Science Cloud (H2020-
INFRAEOSC-2018-2020)’, available at: https://ec.europa.eu/info/funding-
tenders/opportunities/portal/screen/opportunities/topic-details/infraeosc-04-2018.' 

•  Horizon Dashboard (accessed July 2023). 

•  European Commission, European Research Infrastructure Consortium (ERIC), 

available at: https://research-and-innovation.ec.europa.eu/strategy/strategy-2020-
2024/our-digital-future/european-research-infrastructures/eric_en. 

• 

canSERV, Current Calls - All services provided by canSERV. canSERV - Cutting Edge 
Cancer Research Services Across Europe, available at: https://www.canserv.eu/calls/ 

•  Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 
2021 establishing Horizon Europe – the Framework Programme for Research and 
Innovation, laying down its rules for participation and dissemination, and repealing 
Regulations (EU) No 1290/2013 and (EU) No 1291/2013. 

Case Study 6: Analysis of the use of the challenge-oriented approach in the INFRA 
Work Programmes 

•  CNR (2023). Position Paper regarding the ‘Public consultation on the past, present and 
future of the European Research & Innovation Framework Programmes 2014-2027’, 
available at: https://ec.europa.eu/info/law/better-regulation/have-your-
say/initiatives/13460-Horizon-Europe-interim-evaluation/public-consultation_en. 

•  Cordis webpages of selected projects. 

•  DESY (2023). Position Paper regarding the ‘Public consultation on the past, present 
and future of the European Research & Innovation Framework Programmes 2014-
2027’, available at: https://ec.europa.eu/info/law/better-regulation/have-your-
say/initiatives/13460-Horizon-Europe-interim-evaluation/public-consultation_en. 

•  EARTO (2023) and CEA (2023) Position Paper regarding the ‘Public consultation on 
the past, present and future of the European Research & Innovation Framework 
Programmes 2014-2027’, available at: https://ec.europa.eu/info/law/better-
regulation/have-your-say/initiatives/13460-Horizon-Europe-interim-evaluation/public-
consultation_en. 

•  EMBL (2023). Position Paper regarding the ‘Public consultation on the past, present 
and future of the European Research & Innovation Framework Programmes 2014-
2027’, available at: https://ec.europa.eu/info/law/better-regulation/have-your-
say/initiatives/13460-Horizon-Europe-interim-evaluation/public-consultation_en. 

1046 

 
 
•  ESFRI (2021). ESFRI Roadmap Part 2 – Landscape Analysis, available at: 

https://roadmap2021.esfri.eu/landscape-analysis/. 

•  European Commission Directorate-General for Research and Innovation, European 

Research Area policy agenda: overview of actions for the period 2022-2024, 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/52110. 

•  European Commission Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European framework 
programmes for research and innovation for excellent science: Horizon 2020: annexes, 
Luxembourg: Publications Office of the European Union, pp. 848-858, 2023, available 
at: https://data.europa.eu/doi/10.2777/353383. 

•  European Commission, EU missions in Horizon Europe, 2023, available at: 

https://research-and-innovation.ec.europa.eu/funding/funding-opportunities/funding-
programmes-and-open-calls/horizon-europe/eu-missions-horizon-europe_en. 

•  European Commission Decision C(2022)2975 of 10 May 2022: Horizon Europe Work 

Programme 2021-2022, Research Infrastructures, p. 7, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2021-2022/wp-3-research-infrastructures_horizon-2021-2022_en.pdf. 

•  European Commission, ‘Public consultation on the past, present and future of the 

European Research & Innovation Framework Programmes 2014-2027’, available at: 
https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-Horizon-
Europe-interim-evaluation/public-consultation_en. 

•  FCT, ‘Public consultation on the past, present and future of the European Research & 

Innovation Framework Programmes 2014-2027’, 2023, available at: 
https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13460-Horizon-
Europe-interim-evaluation/public-consultation_en. 

•  Helmholtz (2023). Position Paper regarding the ‘Public consultation on the past, 

present and future of the European Research & Innovation Framework Programmes 
2014-2027’, available at: https://ec.europa.eu/info/law/better-regulation/have-your-
say/initiatives/13460-Horizon-Europe-interim-evaluation/public-consultation_en. 

•  Horizon 2020 call ‘Implementing the European Open Science Cloud (H2020-
INFRAEOSC-2018-2020)’, available at: https://ec.europa.eu/info/funding-
tenders/opportunities/portal/screen/opportunities/topic-details/infraeosc-04-2018. 

•  Uncan.eu, project website, available at: https://uncan.eu/. 

•  Masaryk University (2023). Position Paper regarding the ‘Public consultation on the 

past, present and future of the European Research & Innovation Framework 
Programmes 2014-2027’, available at: https://ec.europa.eu/info/law/better-
regulation/have-your-say/initiatives/13460-Horizon-Europe-interim-evaluation/public-
consultation_en. 

•  Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 
2021 establishing Horizon Europe – the Framework Programme for Research and 

1047 

 
 
Innovation, laying down its rules for participation and dissemination, and repealing 
Regulations (EU) No 1290/2013 and (EU) No 1291/2013, available at: https://eur-
lex.europa.eu/eli/reg/2021/695/oj.  

•  Science Business (2023). Position Paper regarding the ‘Public consultation on the past, 
present and future of the European Research & Innovation Framework Programmes 
2014-2027’, available at: https://ec.europa.eu/info/law/better-regulation/have-your-
say/initiatives/13460-Horizon-Europe-interim-evaluation/public-consultation_en. 

Case Study 7: Impact of the widening actions on inducing structural change  

•  Cordis webpages of selected projects. 

•  Commission Staff Working Document 221 final (2017). Interim evaluation of Horizon 
2020 Annex II. https://op.europa.eu/en/publication-detail/-/publication/4cedc9a0-d8cc-
11e8-afb3-01aa75ed71a1/language-en/format-PDF/source-288823388.  

•  European Commission Decision C (2015)5668 of 14 August 2015, Horizon 2020 Work 

Programme 2014- 2015, 15. Spreading Excellence and Widening Participation 
(revised). 

•  European Commission Decision C(2017)2468 of 24 April 2017, Horizon 2020 Work 
Programme 2016- 2017, 15. Spreading Excellence and Widening Participation 
(revised). 

•  European Commission Decision C(2019)4575 of 2 July 2019, Horizon 2020 Work 
Programme 2018- 2020, 15. Spreading Excellence and Widening Participation 
(revised). 

•  European Commission Decision C(2022)2975 of 10 May 2022, Horizon Europe, Work 
Programme 2021-2022, 11.Widening participation and strengthening the European 
Research Area.  

•  European Commission Decision C(2022)7550 of 6 December 2022, Horizon Europe, 
Work Programme 2023-2024, 11. Widening participation and strengthening the 
European Research Area. 

•  European Commission, Directorate-General for Research and Innovation, Horizon 
2020 indicators – Assessing the results and impact of Horizon, Publications Office, 
2015, https://data.europa.eu/doi/10.2777/71098.  

•  European Commission, Directorate-General for Research and Innovation, Interim 
evaluation of Twinning and ERA chairs in Horizon 2020, Publications Office, 2018, 
https://data.europa.eu/doi/10.2777/998775. 

•  European Commission, Directorate-General for Research and Innovation, Puukka, J., 
Spreading excellence and widening participation in Horizon 2020: analysis of FP 
participation patterns and research and innovation performance of eligible countries, 
Publications Office, 2018, https://data.europa.eu/doi/10.2777/42382. 

•  European Commission, Directorate-General for Research and Innovation, A New 

Horizon for Europe – Impact Assessment of the 9th EU Framework Programme for 

1048 

 
 
Research and Innovation, Publications Office, 2018, 
https://data.europa.eu/doi/10.2777/194210.  

•  European Commission, European Research Executive Agency, Spreading excellence 

and widening participation impact report: H2020 results and outlook to Horizon Europe, 
2022, https://data.europa.eu/doi/10.2848/30035. 

•  European Commission, European Research Executive Agency, Dobranskyte-Niskota, 

A., Fernandez-Wyttenbach, A., Salas, , ERA chair dynamics in the widening 
programme – Report on geographical aspects (2014-2022), Publications Office of the 
European Union, 2023, https://data.europa.eu/doi/10.2848/719715. 

•  European Commission, Directorate-General for Research and Innovation, Budraitis, M., 

Pranckevičius, P., Dėlkutė, R.et al., Evaluation study of the European framework 
programmes for research and innovation for excellent science – Horizon 2020: 
annexes, Publications Office of the European Union, 2023, 
https://data.europa.eu/doi/10.2777/353383.  

•  European Commission, Horizon 2020 Online Manual, 

https://ec.europa.eu/research/participants/docs/h2020-funding-guide/grants/grant-
management/reports/continuous-report_en.htm.   

•  European Court of Auditors, Measures to widen participation in Horizon 2020 were well 

designed but sustainable change will mostly depend on efforts by national authorities. 
Special report No 15, 2022, Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2865/359822. 

•  European Regions Research and Innovation Network (ERRIN) (2023) ERRIN Input To 
The Public Consultation On The Past, Present And Future Of The European Research 
& Innovation Framework Programmes 2014-2027. 

•  Replies Of The European Commission To The European Court Of Auditors’ Special 
Report. Measures to widen participation in Horizon 2020 were well designed but 
sustainable change will mostly depend on efforts by National Authorities.  

•  Research Executive Agency, Feedback-to-Policy Survey, 2022, 

https://rea.ec.europa.eu/funding-and-grants/horizon-europe-widening-participation-and-
spreading-excellence/project-results-and-feedback-policy_en.  

•  SWD(2017) 221 final (2017). Interim evaluation of Horizon 2020 Annex II. 

https://op.europa.eu/en/publication-detail/-/publication/4cedc9a0-d8cc-11e8-afb3-
01aa75ed71a1/language-en/format-PDF/source-288823388. 

•  Universities of the Netherlands (UNL), Netherlands Federation of University Medical 
Centres (NFU) & The Royal Netherlands Academy of Arts and Sciences (KNAW) 
(2023) Position Paper Horizon Europe and Beyond. 

Case Study 8: WIDERA Actions on enhancing the European RI system 

•  Communication from the Commission to the European Parliament, the Council, the 

European Economic and Social Committee and the Committee of the Regions on 
achieving the European Education Area by 2025. 

1049 

 
 
•  EUCYS 2021 Salamanca: the future of science, https://eucys2021.usal.es/book/ 

•  European Research Area Policy Agenda - Research and Innovation Overview of 

actions for the period 2022-2024. European Commission, 2021. 

•  Flecha, R., Puigvert, L., and Racionero-Plaza, S. (2023). ‘Achieving student well-being 
for all: Educational contexts free of violence’, NESET report, Luxembourg: Publications 
Office of the European Union. doi: 10.2766/463854 

•  Gareth O’Neill and Helena Acheson, Progress of University Alliance Projects: Projects 
funded under Horizon 2020, Luxembourg: Publications Office of the European Union, 
2023. https://op.europa.eu/en/publication-detail/-/publication/ef36cf58-e57e-11ed-a05c-
01aa75ed71a1/language-en/format-PDF/source-285015820 

•  Horizon Europe Work Programme 2021-2022: 11. Widening participation and 
strengthening the European Research Area. European Commission Decision 
C(2022)2975 of 10 May 2022. 

•  Horizon Europe Work Programme 2023-2024: 11. Widening participation and 

strengthening the European Research Area. European Commission Decision C(2023) 
2178 of 31 March 2023 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

http://ritrain.eu/ 

https://acceleratefuturehei.eu/ 

https://allinteract.eu/impact/ 

https://catalisi.eu/ 

https://crowdhelix.com/ 

https://eucys2021.usal.es/ 

https://leiden2022.nl 

https://ritrainplus.eu 

https://www.esof.eu/marie-sklodowska-curie-actions-msca/ 

https://www.rmroadmap.eu/ 

Ljubljana Declaration: Gender Equality in Research and Innovation. 
https://www.gov.si/assets/ministrstva/MIZS/Dokumenti/PSEU/Ljubljana-Declaration-on-
Gender-Equality-in-Research-and-Innovation-_endorsed_final.pdf 

•  RM Roadmap Ambassadors Network, 

https://static1.squarespace.com/static/633ae0b47dc8ac471e5978a9/t/6429e8d3302b4
96a51e6f569/1680468185491/RM+Roadmap+Ambassadors+Network+V1.pdf 

•  Survey of Horizon Europe beneficiaries, conducted in May-July, 2023. 

1050 

 
 
•  Towards Inclusive Science Communication: Reflections And Successful Actions, 
Fundación Española para la Ciencia y la Tecnología, F. S. P. (FECYT), 2021. 
https://www.fecyt.es/es/system/files/publications/attachments/2022/03/comunicacion_in
clusiva_ciencia_accesible.pdf 

•  Wendy Frigge, Evaluation Report EuroScience Open Forum (ESOF2022), 

https://leiden2022.nl/sites/default/files/2023-
06/D3.2%20ESOF2022%20Programme%20and%20evaluation%20report.pdf 

Case Study 9: Impact of the Framework Programmes on creating high-quality new 
knowledge  

•  Coimbra Group. (2023). Position Paper: Past, present, and future of the European 

Research & Innovation Framework Programme 2014-2027. 

•  Commission presents its evaluation of the 7th Framework Programme for Research 
(2016). https://ec.europa.eu/commission/presscorner/detail/de/MEMO_16_146.  

•  Cordis websites of selected projects. 

•  Duerr, F., & Herkommer, A., (2019). Why does interdisciplinary research matter? 
Advanced Optical Technologies, 8(2). https://doi.org/10.1515/aot-2019-0026. 

•  ERC. (2022). ERC reveals mapping of its funded research. https://erc.europa.eu/news-
events/news/erc-reveals-mapping-its-funded-research, accessed on August 30, 2023.  

•  European Association of Research and Technology Organisations (EARTO). (2023). 

EARTO Answer to the EC Consultation on the past, present and future of the European 
Research & Innovation Framework Programmes 2014-2027. 

•  European Commission Decision C(2022)2975. Horizon Europe. Work Programme 

2021-2022. 12. Missions. https://ec.europa.eu/info/funding-
tenders/opportunities/docs/2021-2027/horizon/wp-call/2021-2022/wp-12-
missions_horizon-2021-2022_en.pdf.  

•  European Commission, Directorate-General for Research and Innovation, A new 
horizon for Europe: impact assessment of the 9th EU Framework Programme for 
Research and Innovation, Publications Office, 2018, 
https://data.europa.eu/doi/10.2777/194210.  

•  European Commission, Directorate-General for Research and Innovation, Nixon, J., 
Study to support the monitoring and evaluation of the Framework Programme for 
Research and Innovation along key impact pathways – Indicator methodology and 
metadata handbook, Nixon, J.(editor), Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2777/44653.  

•  European Commission, Directorate-General for Research and Innovation, Horizon 

Europe – Strategic plan 2021-2024, Publications Office of the European Union, 2021, 
https://data.europa.eu/doi/10.2777/083753. 

•  European Commission, Directorate-General for Research and Innovation, Science, 

research and innovation performance of the EU 2022: building a sustainable future in 

1051 

 
 
uncertain times, Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2777/78826. 

•  European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European Framework 
Programmes for Research and Innovation for excellent science: Horizon 2020: 
annexes, Publications Office of the European Union, 2023, 
https://data.europa.eu/doi/10.2777/353383. 

•  European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R. et al., Evaluation study of the European Framework 
Programmes for Research and Innovation for excellent science: Horizon 2020: phase 1 
final study report, Publications Office of the European Union, 2023, 
https://data.europa.eu/doi/10.2777/967813.  

•  European Commission, Directorate-General for Research and Innovation, Naujokaitytė, 
R., Stančiauskas, V., Cakić, M. et al., Evaluation study of the European framework 
programmes for research and innovation for an innovative Europe – Final report – 
Phase 1, Denham, S.(editor), Publications Office of the European Union, 2023, 
https://data.europa.eu/doi/10.2777/144504. 

•  European Commission, Directorate-General for Research and Innovation, Naujokaitytė, 
R., Mačiulienė, M., Stančiauskas, V. et al., Evaluation study on the implementation of 
cross-cutting issues in Horizon 2020: study in support of the ex-post evaluation of the 
European Framework Programme for Research and Innovation Horizon 2020. Annex 3, 
Case study report, Dirham, S.(editor), Publications Office of the European Union, 2023, 
https://data.europa.eu/doi/10.2777/331671. 

•  European University Association (EUA). (2023). EUA input to the public consultation on 

the past, present and future of the European Research & Innovation Framework 
Programmes 2014-2027. 

• 

• 

Initiative for Science in Europe (ISE). (2023). ISE response to the European 
Commission public consultation on the past, present and future of the European 
Research & Innovation Framework Programmes 2014-2027. 

League of European Research Universities (LERU). (2023). LERU key messages for 
the current and future European R&I Framework Programme. 

•  Hristova, M. and Lehto, S., JRC contribution to the interim evaluation of Horizon 

Europe and Euratom research and training programme. Analysis of peer-reviewed 
publications 2021-2022 and bibliometric indicators, Internal Report, Joint Research 
Centre, Brussels, 2023. 

•  Olsen, R., (2020). Europe needs blue sky research more than ever. Consulted through 
https://www.coimbra-group.eu/europe-needs-blue-sky-research-more-than-ever, 
accessed on August 30, 2023. 

•  Open Public Consultation on the Past, Present and Future of The European Research 

and Innovation Programmes (2014-2027). https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-

1052 

 
 
consultation-future-eu-research-and-innovation-programmes-are-now-public-2023-04-
19_en. 

•  Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 

2021. Establishing Horizon Europe – the Framework Programme for Research and 
Innovation, laying down its rules for participation and dissemination, and repealing 
Regulations (EU) No 1290/2013 and (EU) No 1291/2013. 

•  Regulation (EU) No 1291/2013 of the European Parliament and of the Council of 11 

December 2013 establishing Horizon 2020 - the Framework Programme for Research 
and Innovation (2014-2020) and repealing Decision No 1982/2006/EC. 

•  Science Europe (2023). Science Europe Response to the Public Consultation on 

Horizon 2020 and Horizon Europe. DOI: 10.5281/zenodo.7664436.  

•  SWD (2023)132 final. (2023). Evidence Framework on monitoring and evaluation of 

Horizon Europe. Brussels, 27.4.2023. 

•  Technical Research Centre of Finland (VTT). (2023). VTT Technical Research Centre 
of Finland’s input to the European Commission consultation on the future of the EU 
Research and Innovation Framework Programmes.  

•  The European Science Vocabulary (EuroSciVoc). https://op.europa.eu/en/web/eu-

vocabularies/euroscivoc, accessed on August 30, 2023. 

•  The Guild of European Research-Intensive Universities (2023). Lessons learned and 
way forward: The Guild’s recommendations for Horizon Europe. https://www.the-
guild.eu/publications/position-papers/lessons-learned-and-way-forward-the-guild-s-
recommendations-for-horizon-europe.pdf. 

Case Study 10: Strengthening human capital in research and innovation  

•  European Commission, Directorate-General for Education, Youth, Sport and Culture, 
Marie Skłodowska-Curie actions guidelines on supervision, Publications Office, 2021, 
https://data.europa.eu/doi/10.2766/508311    

•  European Commission, Directorate-General for Research and Innovation, Nixon, J., 
Study to support the monitoring and evaluation of the framework programme for 
research and innovation along key impact pathways: baseline and benchmark report, 
Nixon, J. (editor), Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2777/98156.   

•  European Commission, Directorate-General for Research and Innovation, Nixon, J., 
Study to support the monitoring and evaluation of the framework programme for 
research and innovation along key impact pathways: indicator methodology and 
metadata handbook, Nixon, J. (editor), Publications Office of the European Union, 
2022, https://data.europa.eu/doi/10.2777/44653    

•  European Commission, Directorate-General for Research and Innovation, Evaluation 
study on the external coherence and synergies of Horizon 2020 within the European 
research and innovation support system: case study report, Publications Office of the 
European Union, 2023, https://data.europa.eu/doi/10.2777/054469    

1053 

 
 
•  European Commission, Directorate-General for Research and Innovation, Naujokaitytė, 
R., Mačiulienė, M., Papageorgiou, H., et al., Evaluation study on the implementation of 
cross-cutting issues in Horizon 2020: study in support of the ex-post evaluation of the 
European Framework Programme for Research and Innovation Horizon 2020, Dirham, 
S. (editor), Publications Office of the European Union, 2023, 
https://data.europa.eu/doi/10.2777/763665  

•  European Commission, Directorate-General for Research and Innovation, Newcomers 
in EU R&I programmes: main trends in Horizon 2020, first evidence from Horizon 
Europe, Publications Office of the European Union, 2023, 
https://data.europa.eu/doi/10.2777/911220    

•  European Commission, Directorate-General for Research and Innovation, Country 
participation: from Horizon 2020 to Horizon Europe, Publications Office of the 
European Union, 2022, https://data.europa.eu/doi/10.2777/210660    

•  European Commission, Directorate-General for Research and Innovation, Naujokaitytė, 
R., Mačiulienė, M., Papageorgiou, H., et al., Evaluation study on the implementation of 
cross-cutting issues in Horizon 2020: study in support of the ex-post evaluation of the 
European Framework Programme for Research and Innovation Horizon 2020, Dirham, 
S. (editor), Publications Office of the European Union, 2023, 
https://data.europa.eu/doi/10.2777/763665    

•  European Commission. (2017). Interim Evaluation of Horizon 2020. Staff Working 

Document. SWD(2017).  

•  European Commission. (2021-2022). Work Programme 2021-2022: Widening 

participation and strengthening the European Research Area. 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2021-2022/wp-11-widening-participation-and-strengthening-the-european-
research-area_horizon-2021-2022_en.pdf     

•  European Commission. (2023). European Commission announces support for 124 

researchers who fled the war through #MSCA4Ukraine. Marie Skłodowska-Curie 
Actions. https://marie-sklodowska-curie-actions.ec.europa.eu/news/european-
commission-announces-support-for-124-researchers-who-fled-the-war-through-
msca4ukraine     

•  European Commission. (2023). Evaluation study on Excellent Science in the European 
Framework Programmes for Research and Innovation. Phase 1 Final Study Report and 
its annexes.  

•  European Commission. (2023). Work Programme 2023: European Research Council 

https://erc.europa.eu/sites/default/files/document/file/ERC_WorkProgramme_2023.pdf     

•  European Commission. (2023). Work Programme 2023-2024. Research Infrastructures 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2023-2024/wp-3-research-infrastructures_horizon-2023-2024_en.pdf     

•  European Commission. (2023). Work Programme 2023-2024: Marie Skłodowska-Curie 

Actions. https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/wp-call/2023-2024/wp-2-msca-actions_horizon-2023-2024_en.pdf      

1054 

 
 
•  European Commission. (2021). Work Programme 2021-2022: Marie Skłodowska-Curie 

Actions. https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/wp-call/2021-2022/wp-2-msca-actions_horizon-2021-2022_en.pdf    

•  European Commission. (2023-2024). Work Programme 2023-2024: Widening 

participation and strengthening the European Research 
Area.https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/wp-call/2023-2024/wp-11-widening-participation-and-strengthening-the-
european-research-area_horizon-2023-2024_en.pdf      

•  European Research Council (ERC). (2022). Science Journalism Initiative. 

https://erc.europa.eu/apply-grant/science-journalism-initiative    

•  Gender equality in research and innovation. (2023, May 5). Research and Innovation. 

Retrieved from https://research-and-innovation.ec.europa.eu/strategy/strategy-2020-
2024/democracy-and-rights/gender-equality-research-and-innovation_en     

•  Horizon Dashboard data, accessed on 21 September 2022. Data provided by the 

European Commission indicate that 16,712 distinct researchers have so far received 
access to RIs through INFRA - under FP7, this figure reached 21,060.  

•  Horizon Europe Dashboard (Projects & Results).   

•  MSCA NET. (2023, May 2). MSCA-NET Project Aims at Improving, Professionalising 
and Harmonising Services of the Marie Skłodowska-Curie Actions (MSCA) NCPs. 
https://msca-net.eu/    

•  Regulation (EU) 2021/695 of the European Parliament and of the Council of 28 April 
2021 establishing Horizon Europe – the Framework Programme for Research and 
Innovation, laying down its rules for participation and dissemination, and repealing 
Regulations (EU) No 1290/2013 and (EU) No 1291/2013 (Text with EEA relevance).  

•  Research project database (CORDIS).  

•  Survey of Horizon 2020 beneficiary organisations, conducted on August-September 

(2022).  

•  Survey of Horizon 2020 ERC beneficiaries, conducted on August-September (2022).  

•  Survey of MSCA fellows (IF, ITN, COFUND, RISE), conducted for the Study on mobility 

flows in the context of the MSCA (2022). 

Case Study 11: Impact of the Framework Programmes on fostering the diffusion 
of knowledge and open access 

•  Euraxess, Proposal for a Council Recommendation on a European Framework for 

Research Careers, 2023, available at: 
https://euraxess.ec.europa.eu/euraxess/news/proposal-council-recommendation-
european-framework-research-careers, accessed 18 October 2023. 

•  European Commission, Directorate-General for Research and Innovation, Specific 

support to Malta – ‘Open Access – An Opportunity for Malta’ – Recommendations for 

1055 

 
 
the Development of a National Policy for Open Access to Publications, Research Data 
and related Issues, Luxembourg: Publications Office of the European Union, 2020, 
available at: https://data.europa.eu/doi/10.2777/835. 

•  European Commission, Directorate-General for Research and Innovation, Horizon 

Europe, open science – Early knowledge and data sharing, and open collaboration, 
Luxembourg: Publications Office of the European Union, 2021, available at: 
https://data.europa.eu/doi/10.2777/18252. 

•  European Commission, 2012/417/EU: Commission Recommendation of 17 July 2012 

on access to and preservation of scientific information. 

•  European Commission, Directorate-General for Research and Innovation, European 
Research Area policy agenda – Overview of actions for the period 2022-2024, 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/52110 

•  European Commission, Directorate-General for Research and Innovation, Johnson, R., 

Operationalising Open Research Europe as a collective publishing enterprise, 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/061886. 

•  European Commission, Directorate-General for Research and Innovation, Nixon, J., 
Study to support the monitoring and evaluation of the Framework Programme for 
Research and Innovation along key impact pathways – Indicator methodology and 
metadata handbook, Nixon, J.(editor), Luxembourg: Publications Office of the 
European Union, 2022, available at: https://data.europa.eu/doi/10.2777/44653. 

•  European Commission, Directorate-General for Research and Innovation, Science, 

research and innovation performance of the EU 2022: building a sustainable future in 
uncertain times, Luxembourg: Publications Office of the European Union, 2022, 
available at: https://data.europa.eu/doi/10.2777/78826. 

•  European Commission, Rates of Open Access Scientific Publications, 2022, available 

at: https://ec.europa.eu/assets/rtd/srip/2022/figure-6-1-6.xlsx. 

•  European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European Framework 
Programmes for Research and Innovation for excellent science: Horizon 2020: 
annexes, Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383. 

•  European Commission, Research and Innovation Funding Opportunities, 

https://ec.europa.eu/research/participants/data/ref/h2020/grants_manual/amga/h2020-
amga_en.pdf. 

•  European Council, Improving Conditions for Research Careers in Europe: Council 

Adopts Conclusions, 2021, available at: 
https://www.consilium.europa.eu/en/press/press-releases/2021/05/28/improving-
conditions-for-research-careers-in-europe-council-adopts-conclusions/. 

1056 

 
 
•  European Research Area Policy Agenda, Overview of Actions for the Period 2022-
2024, available at: https://research-and-innovation.ec.europa.eu/system/files/2021-
11/ec_rtd_era-policy-agenda-2021.pdf 

•  Horizon Europe (HORIZON). Programme Guide. Version 3.0. 

https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/guidance/programme-guide_horizon_en.pdf. 

• 

• 

• 

https://ncpflanders.be/infosheets/open-science-under-horizon-europe, accessed on 
14/07/2023.  

Initiative for Science in Europe (ISE) (2023). ISE Response to the European 
Commission Public Consultation on the past, present and future of the European 
Research & Innovation Framework programmes 2014-2027. 

Ludwig Boltzmann Gesellschaft (2023). Ludwig Boltzmann Gesellschaft position paper 
public consultation on the past, present and future of the European Research & 
Innovation Framework Programmes 2014-2027 

•  Nelson Memo, Ensuring Free, Immediate, and Equitable Access to Federally Funded 

Research, Executive Office of the President – Office of Science and Technology Policy, 
2022, available at: https://www.whitehouse.gov/wp-content/uploads/2022/08/08-2022-
OSTP-Public-Access-Memo.pdf, accessed on 17 October 2023. 

•  Owens, B., Catching up on Open Access, 2023, available at: 

https://www.universityaffairs.ca/features/feature-article/catching-up-on-open-access/. 

•  Public consultation on the past, present and future of the EU’s Horizon research and 

innovation programmes 2014-2027, available at: https://research-and-
innovation.ec.europa.eu/news/all-research-and-innovation-news/results-public-
consultation-future-eu-research-and-innovation-programmes-are-now-public-2023-04-
19_en  

•  RDA Bulgarian Node, National Plan for the Development of Open Science Initiatives in 

Bulgaria. https://rda.bg/en/node/47, accessed on 18 October 2023. 

•  ResearchComp: The European Competence Framework for Researchers, available at: 

https://research-and-innovation.ec.europa.eu/jobs-research/researchcomp-european-
competence-framework-researchers_en 

•  Science Europe, Science Europe Response to the Public Consultation on Horizon 
2020 and Horizon Europe, 2022, available at: https://www.scienceeurope.org/our-
resources/science-europe-response-to-the-public-consultation-on-horizon-2020-and-
horizon-europe/. 

•  Science Europe (2023). Science Europa response to the public consultation on Horizon 

2020 and Horizon Europe 2023. 

•  Select Committee on Science and Technology Tenth Report, 2004, 

www.publications.parliament.uk/pa/cm200304/cmselect/cmsctech/399/39903.htm, 
accessed on 18 October 2023. 

1057 

 
 
•  Slovakia, National Strategy for Open Science 2021-2028, available at: 

https://www.openaire.eu/blogs/slovak-national-strategy-for-open-science-2021-2028. 

•  SPARC Europe, An Analysis of Open Science Policies in Europe, v7, 2021, available 

at: https://sparceurope.org/what-we-do/open-data/sparc-europe-open-data-resources/, 
accessed on 18 October 2023. 

•  SWD(2023)132 final. (2023). Evidence Framework on monitoring and evaluation of 

Horizon Europe. Brussels, 27.4.2023. 

•  UNESCO Recommendation on Open Science. Draft text, available at: 

https://unesdoc.unesco.org/ark:/48223/pf0000378841. 

•  Websites on Open Science environments and platforms (Zenodo, OpenAire, Open 

Research Europe, EOSC) 

•  Work Programmes INFRA (2021-2022; 2023-2024). 

•  Work Programmes SwafS (2014-2015; 2016-2017; 2018-2020). 

•  Work Programmes WIDERA (2021-2022; 2023-2024). 

Case  Study  12:  Gender  equality  in  the  European  Framework  Programmes  for 
Research and Innovation 

•  Cordis webpages of selected projects. 

•  Council Decision (EU) 2021/764 of 10 May 2021 establishing the Specific Programme 

implementing Horizon Europe – the Framework Programme for Research and 
Innovation, and repealing Decision 2013/743/EU, available at: https://eur-
lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32021D0764 

•  European Commission, Communication from the Commission to the European 
Parliament, the Council, the European Economic and Social Committee and the 
Committee of the Regions A new ERA for Research and Innovation, 30 September 
2020, COM (2020) 628 final, available at: https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=COM%3A2020%3A628%3AFIN 

•  European Commission, Communication from the Commission to the European 
Parliament, the Council, the European Economic and Social Committee and the 
Committee of the Regions A New European Innovation Agenda, 5 July 2022, COM 
(2022) 332 final, available at: https://eur-lex.europa.eu/legal-
content/EN/TXT/PDF/?uri=CELEX:52022DC0332 

•  European Commission, Communication from the Commission to the European 
Parliament, the Council, the European Economic and Social Committee and the 
Committee of the Regions A Union of Equality: Gender Equality Strategy 2020-2025, 5 
March 2020, COM (2020) 152 final, available at: https://eur-lex.europa.eu/legal-
content/EN/ALL/?uri=COM:2020:0152:FIN  

•  European Commission, Communication from the Commission to the European 
Parliament, the Council, the European Economic and Social Committee and the 

1058 

 
 
Committee of the Regions. A Reinforced European Research Area Partnership for 
Excellence and Growth. COM(2012) 392 final, available at: https://eur-
lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52012DC0392. 

•  European Commission, Directorate-General for Justice and Consumers, Strategic 

engagement for gender equality 2016-2019, Luxembourg: Publications Office of the 
European Union, 2016, available at: https://data.europa.eu/doi/10.2838/454429 

•  European Commission, Directorate-General for Research and Innovation, European 

Research Area policy agenda: overview of actions for the period 2022-2024, 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/52110 https://data.europa.eu/doi/10.2777/52110   

•  European Commission, Directorate-General for Research and Innovation, Horizon 
Europe guidance on gender equality plans, Luxembourg: Publications Office of the 
European Union, 2021, available at: https://data.europa.eu/doi/10.2777/876509 

•  European Commission, Directorate-General for Research and Innovation, Horizon 

Europe, gender equality – A strengthened commitment in Horizon Europe, 
Luxembourg: Publications Office of the European Union, 2021, available at: 
https://data.europa.eu/doi/10.2777/97891 

•  European Commission, Directorate-General for Justice and Consumers, Strategic 

engagement for gender equality 2016-2019, Luxembourg: Publications Office of the 
European Union, 2016, available at: https://data.europa.eu/doi/10.2838/454429 

•  European Commission, Directorate-General for Research and Innovation, Interim 

evaluation – Gender equality as a crosscutting issue in Horizon 2020, Luxembourg: 
Publications Office of the European Union, 2017, available at: 
https://data.europa.eu/doi/10.2777/054612 

•  European Commission, Directorate-General for Research and Innovation, Naujokaitytė, 
R., Mačiulienė, M., Stančiauskas, V.et al., Evaluation study on the implementation of 
cross-cutting issues in Horizon 2020 – Study in support of the ex-post evaluation of the 
European Framework Programme for Research and Innovation Horizon 2020, Dirham, 
S. (editor), Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/763665 

•  European Commission, Directorate-General for Research and Innovation, She figures 
2021 – Gender in research and innovation: statistics and indicators, Luxembourg: 
Publications Office of the European Union, 2021, available at: 
https://data.europa.eu/doi/10.2777/06090 

•  European Commission, Directorate-General for Research and Innovation, MORE4 – 
Support data collection and analysis concerning mobility patterns and career paths of 
researchers: survey on researchers outside Europe, Luxembourg: Publications Office 
of the European Union, 2021, available at: https://data.europa.eu/doi/10.2777/117072  

•  European Commission, Directorate-General for Research and Innovation, Synopsis 
report – Looking into the R&I future priorities 2025-2027, Luxembourg: Publications 
Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/93927  

1059 

 
 
•  European Commission, Strategy for Equality between Women and Men 2010-2015, 

Luxembourg: Publications Office of the European Union, 2011, available at: 
https://data.europa.eu/doi/10.2767/71254. 

•  European Commission, Strategic engagement for gender equality 2016-2019, 
Luxembourg: Publications Office of the European Union, 2016, available at: 
https://data.europa.eu/doi/10.2838/454429 

•  European Council, Council Conclusions of 1 December 2015 on Advancing Gender 

Equality in the European Research Area, available at: 
https://data.consilium.europa.eu/doc/document/ST-14846-2015-INIT/en/pdf. 

•  European Council, Council Conclusions of 7 March 2011 on European Pact for Gender 

Equality (2011-2020), available at: https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=CELEX%3A52011XG0525%2801%29. 

•  European Council, Proposal for a Council Recommendation on a Pact for Research 

and Innovation in Europe. COM/2021/407 final, available at: https://eur-
lex.europa.eu/legal-content/EN/ALL/?uri=COM:2021:407:FIN. 

•  European Council, Council Conclusions of 26 November 2021 on Future governance of 

the European Research Area (ERA), available at: 
https://data.consilium.europa.eu/doc/document/ST-14308-2021-INIT/en/pdf 

•  European Parliament and the Council (2021), Regulation (EU) 2021/695 of the 

European Parliament and of the Council of 28 April 2021 establishing Horizon Europe – 
the Framework Programme for Research and Innovation, laying down its rules for 
participation and dissemination, and repealing Regulations (EU) No 1290/2013 and 
(EU) No 1291/2013, 28 April 2021, PE/12/2021/INIT, OJ L 170, 12.5.2021, pp. 1–68, 
available at: http://data.europa.eu/eli/reg/2021/695/oj    

•  European Union, Consolidated Version of the Treaty on the Functioning of the 

European Union, 26 October 2012, OJ L. 326/47-326/390; 26.10.2012, available at: 
http://data.europa.eu/eli/treaty/tfeu_2012/oj. 

•  Charter of Fundamental Rights of the European Union, available at: https://eur-

lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A12012P%2FTXT. 

• 

Ljubljana Declaration, available at: https://data.consilium.europa.eu/doc/document/ST-
12044-2021-INIT/en/pdf. 

•  UN Sustainable Development Goals, Goal 5, available at: 

https://sdgs.un.org/goals/goal5 

• 

• 

https://research-and-innovation.ec.europa.eu/funding/funding-opportunities/funding-
programmes-and-open-calls/horizon-europe_en. 

https://research-and-innovation.ec.europa.eu/strategy/strategy-2020-2024/democracy-
and-rights/gender-equality-research-and-innovation_en. 

•  European Commission, Horizon 2020 administrative and monitoring data, 2022, 

Available at: 

1060 

 
 
https://www.europarl.europa.eu/RegData/docs_autres_institutions/commission_europe
enne/com/2022/0389/COM_COM(2022)0389_EN.pdf.  

•  Genderaction Project, information available at: https://genderaction.eu/eu-sister-

projects/ and https://h2020.genderaction.eu/resources/sister-projects/. 

•  BUDGET IT Project, Information available at: https://budget-it.eu/. 

•  Funding & tender opportunity - FAQ - Frequently Asked Questions. Available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/support/faq. 

•  European Research Council, Advanced analytics tool (ERIS). Available at: 
https://erc.europa.eu/projects-statistics/advanced-analytics-tool-eris. 

•  European Institute for Gender Equality, Gender budgeting. Available at: 

https://eige.europa.eu/gender-mainstreaming/tools-methods/gender-budgeting. 

•  Commission Staff Working Document Evaluation Accompanying the Document Report 

from the Commission to the European Parliament and the Council Ex-post evaluation 
of Horizon 2020, the EU Framework Programme for Research and Innovation. 
Available at: https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=SWD%3A2024%3A29%3AFIN&amp%3Bqid=1706528145182 

Case Study 13: International cooperation and association 

•  Work Programmes WIDERA (2021-2022; 2023-2024). 

•  Commission Communication on the Global Approach to Research and Innovation 

“Europe's strategy for international cooperation in a changing world”. 

•  Balland, P.A., and Ravet, J. (2018). Dynamic Network Analysis of the EU R&I 

Framework Programme. European Commission report.  

•  Craciun, D. and Orosz, K. (2018). Benefits and costs of transnational collaborative 

partnerships in higher education. EENE Analytical Report No.36 prepared for the 
European Commission.  

•  Farrell M., Kalpazidou Schmidt E., Mourzelas M., Warrington B., Wood J. (2015), Ex-
post Evaluation of International Cooperation Activities of the Seventh Framework 
Programme’s Capacities Programme, Report for the European Commission. 

•  European Commission (2017). Interim Evaluation of Horizon 2020. Staff Working 

Document. SWD(2017). 

•  From Horizon 2020 to Horizon Europe - Monitoring Flash #2.  European Commission 

(2018).  

•  Project success stories: stories of particularly successful EU-funded research projects. 

•  European Commission (2017), Interim evaluation of Horizon 2020, Staff Working 
Document, and its Annex 1 and Annex 2, also other parts of Horizon 2020 interim 
evaluation. 

1061 

 
 
•  Monitoring flashes on international cooperation and association aspects of the 
Framework Programme, monitoring report “Keeping our eyes on the Horizon”. 

•  European Commission (2018), Impact assessment of Horizon Europe, Staff Working 

Document. 

•  Horizon Europe Strategic Plan for 2021-2024. 

•  European Commission (2020), Science, Research and Innovation Performance (SRIP) 

of the EU 2020. 

•  European Commission websites on research and innovation and its monitoring and 

evaluation. 

• 

International comparisons: OECD’s reports and Main Science and Technology 
Indicators, the NSF’s annual Science & Engineering Indicators, Clarivate’s Research 
Fronts. 

•  European Commission (2020) Monitoring flash series: Horizon 2020 monitoring report 

• 

• 

‘Enhancing and Focusing EU International Cooperation in Research and Innovation: A 
Strategic Approach’ 

2018 Report on the Implementation of the Strategy for International Cooperation in 
Research and Innovation 

•  Enhancing and focusing EU international cooperation in research and innovation: A 

strategic approach" (COM(2012) 497) 

•  European Commission (2023) New Zealand joins Horizon Europe Research and 

Innovation Programme. Accessed 2023/08/01/. 

•  Horizon Europe Strategic plan 2025-2027 Analysis 

•  Public consultation on the past, present and future of the European R&I FPs 2014-

2027 

•  Work Programmes ERC (2021; 2022; 2023; 2024).  

•  Work Programmes MSCA (2021-2022; 2023-2024).  

•  Work Programmes INFRA (2021-2022; 2023-2024). 

•  Work Programmes SwafS (2014-2015; 2016-2017; 2018-2020). 

•  Work Programmes WIDERA (2021-2022; 2023-2024). 

1062 

 
 
Case  Study  14:  Impact  of  the  Framework  Programmes  on  fostering  research 
integrity and ethics  

•  Allea, The European Code of Conduct for Research Integrity, 2023, available at: 

https://allea.org/wp-content/uploads/2023/06/European-Code-of-Conduct-Revised-
Edition-2023.pdf. 

•  Allum, N., Reid, A., Bidoglia, M., Gaskell, G., Aubert-Bonn, N., Buljan, I., Fuglsang, S., 
Horbach, S., Kavouras, P., Marušić, A., Mejlgaard, N., Pizzolato, D., Roje, R., Tijdink, 
J., & Veltri, G., Researchers on research integrity: a survey of European and American 
researchers. F1000Res. 2023 Feb 16;12:187, doi: 10.12688/f1000research.128733.1. 

•  Austrian Agency for Research Integrity website, https://oeawi.at/en/training-overview. 

•  Codes of ethics for scientific research in Belgium, 

https://research.kuleuven.be/en/integrity-ethics/integrity/practices/belspo-code. 

•  Cordis EU Research Results webpage, Creating and enhancing TRUSTworthy, 

responsible and equitable partnerships in international research, 
https://cordis.europa.eu/project/id/664771. 

•  Cordis EU Research Results webpage, Define the global and financial impact of 

research misconduct, https://cordis.europa.eu/project/id/710246. 

•  Cordis EU Research Results webpage, European Network of Research Ethics and 

Research Integrity, https://cordis.europa.eu/project/id/710184.  

•  Cordis EU Research Results webpage, INTEGRITY: empowering students through 

evidence-based, scaffolded learning of Responsible Conduct in Research (RCR), 
https://cordis.europa.eu/project/id/824586. 

•  Cordis EU Research Results webpage, Mapping Normative Frameworks for Ethics and 

Integrity of Research, https://cordis.europa.eu/project/id/741782. 

•  Cordis EU Research Results webpage, Shaping the ethical dimensions of smart 

information systems (SIS) – a European perspective, 
https://cordis.europa.eu/project/id/786641. 

•  Cordis EU Research Results webpage, Stakeholder-informed ethics for new 

technologies with high socio-economic and human rights impact, 
https://cordis.europa.eu/project/id/741716. 

•  Cordis EU Research Results webpage, Virtue based ethics and Integrity of Research: 
Train-the-Trainer program for Upholding the principles and practices of the European 
Code of Conduct for Research Integrity, https://cordis.europa.eu/project/id/787580. 

•  Council Conclusions on a Global approach to Research and Innovation – Europe's 

strategy for international cooperation in a changing world, adopted on 28 September 
2023, available at: https://data.consilium.europa.eu/doc/document/ST-12301-2021-
INIT/en/pdf.  

1063 

 
 
•  De Waele, I., Wizel, D., Puljak, L., & Koporc, Z. (2021), Ethics appraisal procedure in 
79,670 Marie Skłodowska-Curie proposals from the entire European HORIZON 2020 
research and innovation program (2014–2020): A retrospective analysis, PLoS ONE 
16(11): e0259582, available at: https://doi.org/10.1371/journal.pone.0259582. 

•  Embassy of Science Website, https://www.embassy.science.  

•  EUR-Lex, Regulation (EU) 2021/695 of the European Parliament and of the Council of 
28 April 2021 establishing Horizon Europe – the Framework Programme for Research 
and Innovation, laying down its rules for participation and dissemination, and repealing 
Regulations (EU) No 1290/2013 and (EU) No 1291/2013 (Text with EEA relevance), 
available at: https://eur-lex.europa.eu/eli/reg/2021/695/oj.  

•  European Commission, Ethics Guidelines for Trustworthy AI, 2019, available at: 

https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai. 

•  European Commission, Communication from the Commission to the European 
Parliament, the Council, the European Economic and Social Committee and the 
Committee of the Regions A new ERA for Research and Innovation, 30 September 
2020, COM (2020) 628 final, available at: EUR-Lex - 52020DC0628 - EN - EUR-Lex 
(europa.eu). 

•  European Commission, Directorate-General for Research and Innovation, Monachello, 
R., Delaney, N., & Tornasi, Z., Research ethics and research integrity: achievements in 
Horizon 2020 and recommendations on the way forward, Delaney, N. (editor), Tornasi, 
Z. (editor), Luxembourg: Publications Office of the European Union, 2020, available at: 
https://data.europa.eu/doi/10.2777/63976. 

•  European Commission, Directorate-General for Research and Innovation, Ethics for 
Researchers: Facilitating Research Excellence in FP7, available at: ethics-for-
researchers_en.pdf (europa.eu). 

•  European Commission, Directorate-General for Research and Innovation, European 

Research Area policy agenda: overview of actions for the period 2022-2024, 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/52110 https://data.europa.eu/doi/10.2777/52110. 

•  European Commission, Directorate-General for Research and Innovation, Almerud, M., 

Ricksten, M., O’Neill, G., et al., Knowledge ecosystems in the new ERA: using a 
competence-based approach for career development in academia and beyond, Núñez, 
L., De Coen, A. (eds.), Luxembourg: Publications Office of the European Union, 2022, 
available at: https://data.europa.eu/doi/10.2777/150763. 

•  European Commission, Directorate-General for Research and Innovation, A new 
horizon for Europe: impact assessment of the 9th EU framework programme for 
research and innovation, Luxembourg: Publications Office of the European Union, 
2018, available at: https://data.europa.eu/doi/10.2777/194210. 

•  European Commission, Directorate-General for Research and Innovation, Interim 
evaluation of Horizon 2020: Commission staff working document, Luxembourg: 
Publications Office of the European Union, 2017, available at:  
https://data.europa.eu/doi/10.2777/220768. 

1064 

 
 
•  European Commission, Directorate-General for Research and Innovation, Kadunc, M., 
& Bruno, N., Keeping our eyes on the Horizon: monitoring flash series: a Horizon 2020 
monitoring report, Kadunc, M., Bruno, N. (eds.), Luxembourg: Publications Office of the 
European Union, 2020, available at:  https://data.europa.eu/doi/10.2777/303024. 

•  European Commission, Directorate-General for Research and Innovation, Horizon 

Europe: Strategic Plan 2021-2024, Luxembourg: Publications Office of the European 
Union, 2021, available at: https://data.europa.eu/doi/10.2777/083753. 

•  European Commission, Directorate-General for Research and Innovation, Monachello, 
R., Delaney, N., & Tornasi, Z., Research ethics and research integrity: achievements in 
Horizon 2020 and recommendations on the way forward, Delaney, N., Tornasi, Z. 
(eds.), Luxembourg: Publications Office of the European Union, 2020, available at: 
https://data.europa.eu/doi/10.2777/63976. 

•  European Commission, Directorate-General for Research and Innovation, Delaney, N., 
Iagher, R., & Tornasi, Z., Institutional changes towards responsible research and 
innovation: achievements in Horizon 2020 and recommendations on the way forward, 
Luxembourg: Publications Office of the European Union, 2020, available at: 
https://data.europa.eu/doi/10.2777/682661. 

•  European Commission, Directorate-General for Research and Innovation, Iagher, R., 
Monachello, R., Warin, C., et al., Science with and for society in Horizon 2020: 
achievements and recommendations for Horizon Europe, Delaney, N., Tornasi, Z. 
(eds.), Luxembourg: Publications Office of the European Union, 2020, available at: 
https://data.europa.eu/doi/10.2777/32018. 

•  European Commission, EU Grants: How to complete your ethics self-assessment, 
available at: https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/common/guidance/how-to-complete-your-ethics-self-assessment_en.pdf. 

•  European Commission, Grants Manual - Section on: Proposal submission and 

evaluation, available at: 
https://ec.europa.eu/research/participants/data/ref/h2020/grants_manual/pse/h2020-
guide-pse_en.pdf. 

•  European Commission, Horizon 2020 Online Manual, available at: 

https://ec.europa.eu/research/participants/docs/h2020-funding-guide/cross-cutting-
issues/ethics_en.htm. 

•  European Commission, Horizon Europe (HORIZON), Programme Guide, available at: 

https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/guidance/programme-guide_horizon_en.pdf. 

•  European Commission, Horizon Europe Strategic Plan 2025-2027 analysis, available 
at: https://research-and-innovation.ec.europa.eu/funding/funding-opportunities/funding-
programmes-and-open-calls/horizon-europe/strategic-plan_en. 

•  European Commission, The Pact for Research and Innovation in Europe, available at: 

https://research-and-innovation.ec.europa.eu/news/all-research-and-innovation-
news/commission-adopts-proposal-pact-research-and-innovation-europe-2021-07-
16_en. 

1065 

 
 
•  European Commission; Horizon Europe's first Strategic Plan 2021-2024, available at: 

https://ec.europa.eu/commission/presscorner/detail/en/IP_21_1122. 

•  European Commission, Guidelines on Serious and Complex Cases, Horizon 2021-

2027 Online Manual, available at: https://ec.europa.eu/info/funding-
tenders/opportunities/docs/2021-2027/horizon/guidance/guidelines-on-serious-and-
complex-cases_he_en.pdf. 

•  European Commission, Guidance on How to Handle Security-Sensitive Projects, 
Horizon 2021-2027 Online Manual, available at: https://ec.europa.eu/info/funding-
tenders/opportunities/docs/2021-2027/common/guidance/how-to-handle-security-
sensitive-projects_en.pdf. 

•  European Commission, Ethical Guidance for Research with a Potential for Human 

Enhancement, Horizon 2021-2027 Online Manual, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/guidance/ethical-guidance-for-research-with-a-potential-for-human-
enhancement-sienna_he_en.pdf. 

•  European Commission, 2021, Ethics by Design and Ethics of Use Approaches for 

Artificial Intelligence, Horizon 2021-2027 Online Manual, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/guidance/ethics-by-design-and-ethics-of-use-approaches-for-artificial-
intelligence_he_en.pdf. 

•  European Commission, Guidance note – Research with an exclusive focus on civil 

applications (2021), available at: https://ec.europa.eu/info/funding-
tenders/opportunities/docs/2021-2027/horizon/guidance/guidance-note-research-
focusing-exclusively-on-civil-applications_he_en.pdf. 

•  European Commission, Guidance note – Potential misuse of research (2021), available 

at: https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/guidance/guidance-note-potential-misuse-of-research-results_he_en.pdf.   

•  European Commission, Widening Participation and Spreading Excellence, available at: 
https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/wp-
call/2023-2024/wp-11-widening-participation-and-strengthening-the-european-
research-area_horizon-2023-2024_en.pdf. 

•  European Parliament and the Council (2021), Regulation (EU) 2021/695 of the 

European Parliament and of the Council of 28 April 2021 establishing Horizon Europe – 
the Framework Programme for Research and Innovation, laying down its rules for 
participation and dissemination, and repealing Regulations (EU) No 1290/2013 and 
(EU) No 1291/2013, 28 April 2021, PE/12/2021/INIT, OJ L 170, 12.5.2021, pp. 1–68, 
available at: http://data.europa.eu/eli/reg/2021/695/oj. 

•  European Union, Charter of Fundamental Rights of the European Union, 26 October 

2012, 2012/C 326/02, OJ C 326, 26.10.2012, pp. 391–407, available at: 
http://data.europa.eu/eli/treaty/char_2012/oj  

•  Fair Research Contracting Website, http://frcweb.cohred.org. 

1066 

 
 
•  Gans Combe, C., Faucheux, S., Kuszla, C., Petousi, V., & Garani, T., DEFORM Main 

Findings – A Policy Report (January 09, 2019), available at SSRN: 
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3389414. 

•  Hybrida, Embedding a comprehensive ethical dimension to organoid-based research 

and resulting technologies, https://hybrida-project.eu/ 

•  Mejlgaard N., Bouter, L.M., Gaskell, G., Kavouras, P., Allum, N., Bendtsen, A.K., 
Charitidis, C.A., Claesen, N., Dierickx, K., Domaradzka, A., Reyes Elizondo, A., 
Foeger, N., Hiney, M., Kaltenbrunner, W., Labib, K., Marušić, A., Sørensen, M.P., 
Ravn, T., Ščepanović, R., Tijdink, J.K., Veltri, G.A., Research integrity: nine ways to 
move from talk to walk, Nature 2020 Oct;586(7829):358-360, doi: 10.1038/d41586-020-
02847-8. PMID: 33041342. 

•  Nature, Nature addresses helicopter research and ethics dumping, Nature, 2022 

Jun;606(7912):7. doi: 10.1038/d41586-022-01423-6. PMID: 35637322. 

•  NPCFlanders, Report Science with and for society in Horizon 2020 - achievements and 

recommendations for Horizon Europe, available at: 
https://ncpflanders.be/documents/report-science-with-and-for-society-in-horizon-2020-
achievements-and-recommendations-for-horizon-europe. 

•  Prepared Project, The TRUST Code – A Global Code of Conduct for Equitable 

Research Partnerships, 2018, available at: https://prepared-project.eu/the-trust-code/. 

•  Primanova.com, Artificial Intelligence and Ethics in EU-funded Projects, available at: 

https://www.privanova.com/resources/artificial-intelligence-and-ethics-in-eu-funded-
projects.  

•  Printeger – Promoting Integrity as an Integral Dimension of Excellence in Research, 

https://printeger.eu/. 

•  ROSIE – Fostering responsible Open Science in Europe, https://rosie-project.eu/. 

•  SOPs4RI – Standard Operating Procedures for Research Integrity, https://sops4ri.eu/. 

•  Synopsis report of the public consultation on the past, present and future of the 

European research and innovation Framework Programmes 2014-2027, available at: 
https://op.europa.eu/en/publication-detail/-/publication/e8f722ec-dff7-11ed-a05c-
01aa75ed71a1/language-en/format-PDF/source-284554549. 

•  The TRUST Code: A Global Code of Conduct for Equitable Research Partnerships, 

http://www.globalcodeofconduct.org/. 

Case Study 15: Co-design and co-creation of R&I through citizen engagement 

•  Alemanno, A (2022) Towards a permanent citizens' participatory mechanism in the EU. 
Study requested by the European Parliament's Committee on Constitutional Affairs. 
Available here: 
https://www.europarl.europa.eu/thinktank/en/document/IPOL_STU(2022)735927 

1067 

 
 
•  Australian Research Council (2018): Engagement and Impact Assessment. Available 

here: https://dataportal.arc.gov.au/EI/NationalReport/2018/ 

•  COM(2019) 640 final: Communication from the Commission to the European 

Parliament, the European Council, the Council, the European Economic and Social 
Committee and the Committee of the Regions: The European Green Deal. 

•  COM(2020) 628 final: Commission Staff Working Document accompanying the 

document "Communication from the Commission to the European Parliament, the 
Council, the European Economic and Social Committee and the Committee of the 
Regions: A new ERA for Research and Innovation". 

•  COM(2023) 457 final: Commission Staff Working document "Communication from the 
Commission to the European Parliament, the Council, the European Economic and 
Social Committee and the Committee of the regions EU Missions two years on: 
assessment of progress and way forward" 

•  Council Conclusions on Research assessment and implementation of Open Science, 

2022. Available here: https://www.consilium.europa.eu/media/56958/st10126-en22.pdf 

•  Council Decision (EU) 2021/764 of 10 May 2021 establishing the Specific Programme 

implementing Horizon Europe – the Framework Programme for Research and 
Innovation, and repealing Decision 2013/743/EU, available at: https://eur-
lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32021D0764 

•  Council Recommendation on a Pact for Research and Innovation in Europe (2021):  
Available here: https://data.consilium.europa.eu/doc/document/ST-14136-2021-
INIT/en/pdf 

•  Special Eurobarometer #516: ‘European citizens’ knowledge and attitudes towards 

science and technology’, 2021. Survey conducted by Kantar at the request of the 
European Commission, Directorate-General for Research and Innovation. Survey co-
ordinated by the European Commission, Directorate-General for Communication (DG 
COMM “Media monitoring and Eurobarometer Unit”). Available here: 
https://lbg.ac.at/wp-
content/uploads/2021/11/ebs_516_science_and_technology_report_EN.pdf 

•  Commission Recommendation (EU) 2018/790 of 25 April 2018 on access to and 

preservation of scientific information. Available here: https://eur-
lex.europa.eu/eli/reco/2018/790/oj 

•  Erasmus+, Priorities of the Erasmus+ Programme, 2023. Available here: 

https://erasmus-plus.ec.europa.eu/projects/priorities-2019-2024 

•  European Commission (2022), Valorising research through citizens’ engagement: how 
to run hackathons with citizens. Available here: https://op.europa.eu/en/publication-
detail/-/publication/1585e969-ec57-11ec-a534-01aa75ed71a1/language-en/format-
PDF/source-283906489 

•  European Commission, Directorate-General for Research and Innovation, Warin, C., 
Delaney, N., Tornasi, Z., Citizen science and citizen engagement – Achievements in 

1068 

 
 
Horizon 2020 and recommendations on the way forward, Delaney, N.(editor), Tornasi, 
Z.(editor), Publications Office, 2020, https://data.europa.eu/doi/10.2777/05286 

•  European Commission, Directorate-General for Research and Innovation, 

Communication and citizen engagement initiatives in line with the Horizon Europe 
Mission A Soil Deal for Europe – Report on dissemination and exploitation practices in 
Member States and associated countries, Publications Office of the European Union, 
2022, https://data.europa.eu/doi/10.2777/704413 

•  European Commission, Directorate-General for Energy, Brenner-Fließer, M., Rikos, E., 

Tuiskula, H. et al., Exploration of citizen engagement methodologies in European R&I 
projects 3.0 – Consumer, Citizen & Engagement Working Group – June 2023, Brenner-
Fließer, M.(editor), Rikos, E.(editor), Tuiskula, H.(editor), Höffken, J.(editor), Birch 
Riley, L.(editor), Kuivalainen, M.(editor), Ktenidis, P.(editor), Publications Office of the 
European Union, 2023, https://data.europa.eu/doi/10.2833/78191 

•  European Commission, Directorate-General for Research and Innovation, European 
Research Executive Agency, Publications Office of the European Union, CORDIS 
results pack on citizen science, Publications Office of the European Union, 2022, 
https://data.europa.eu/doi/10.2830/376577 

•  European Commission. (2021). ERA Policy Agenda. Retrieved from 

https://commission.europa.eu/system/files/2021-11/ec_rtd_era-policy-agenda-2021.pdf 

•  European Governance: A White Paper COM(2001) 428: 

https://ec.europa.eu/commission/presscorner/detail/en/DOC_01_10. 

•  European Parliament and the Council (2021), Regulation (EU) 2021/695 of the 

European Parliament and of the Council of 28 April 2021 establishing Horizon Europe – 
the Framework Programme for Research and Innovation, laying down its rules for 
participation and dissemination, and repealing Regulations (EU) No 1290/2013 and 
(EU) No 1291/2013, 28 April 2021, PE/12/2021/INIT, OJ L 170, 12.5.2021, pp. 1–68, 
available at: http://data.europa.eu/eli/reg/2021/695/oj    

•  European Research Council, Citizen Science in ERC projects: Mapping ERC Frontier 
Research, 2022. Available here: https://erc.europa.eu/projects-statistics/citizen-
science-and-frontier-research 

•  European Commission, Directorate-General for Research and Innovation, Naujokaitytė, 
R., Mačiulienė, M., Stančiauskas, V. et al., Evaluation study on the implementation of 
cross-cutting issues in Horizon 2020 – Study in support of the ex-post evaluation of the 
European Framework Programme for Research and Innovation Horizon 2020, Dirham, 
S.(editor), Publications Office of the European Union, 
2023, https://data.europa.eu/doi/10.2777/763665 

•  SWD (2023)132 final. (2023). Evidence Framework on monitoring and evaluation of 

Horizon Europe. Brussels, 27.4.2023. 

•  Griessler, E., Braun, R., Wicher, M., & Yorulmaz, M., The drama of responsible 
research and innovation: The ups and downs of a policy concept. In: Putting 
Responsible Research and Innovation into Practice: A Multi-Stakeholder Approach, pp. 
11-34, Cham: Springer International Publishing, 2022. 

1069 

 
 
•  Horizon Europe (HORIZON). Programme Guide. Version 3.0. 

https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-
2027/horizon/guidance/programme-guide_horizon_en.pdf. 

•  European Commission, Directorate-General for Research and Innovation, 

Stančiauskas, V., Brozaitis, H., Notten, A. et al., Study to support the monitoring and 
evaluation of the framework programme for research and innovation along key impact 
pathways – Indicator methodology and metadata handbook, Stančiauskas, V.(editor), 
Brozaitis, H.(editor), Notten, A.(editor), Hollanders, H.(editor), Papageorgiou, H.(editor), 
Manola, N.(editor), Zagame, P.(editor), Le-Mouel, P.(editor), Publications Office of the 
European Union, 2022, https://data.europa.eu/doi/10.2777/44653 

•  European Commission, Directorate-General for Research and Innovation, Interim 

evaluation of Horizon 2020 – Commission staff working document, Publications Office, 
2017, https://data.europa.eu/doi/10.2777/220768 

•  Meier, A., & Byland, R., A New Horizon for Society? Analysing the Integration of 

Responsible Research and Innovation in Horizon Europe, Brussels: SwissCore, 2020. 
Available here: https://www.swisscore.org/wp-
content/uploads/RRI_in_Horizon_Europe.1.0.pdf 

•  MSCA-NET Policy brief: Open Science, 2023. Available here: 

https://horizoneuropencpportal.eu/sites/default/files/2023-04/task-3.6-
open_science_brief.pdf 

•  OECD (2022), "Engaging citizens in cohesion policy: DG REGIO and OECD pilot 
project final report", OECD Working Papers on Public Governance, No. 50, OECD 
Publishing, Paris, https://doi.org/10.1787/486e5a88-en. 

•  Tertiary Education Commission, New Zealand: Performance-Based Research Fund. 

Available here: https://www.tec.govt.nz/funding/funding-and-performance/funding/fund-
finder/pbrf/ 

•  An introduction to the UNESCO Recommendation on Open Science. (2022). 

https://doi.org/10.54677/xoir1696 

•  Work Programme Cluster 3: Civil Security for Society (2023-2024) 

•  Work Programme Cluster 4: Digital, Industry and Space (2023-2024) 

•  Work Programme EIC (2023-2024) 

•  Work Programmes MSCA (2021-2022; 2023-2024) 

•  Work Programmes WIDERA (2021-2022; 2023-2024). 

Phase 1 benchmarks 

Benchmark  1:  National  Science  Foundation  (NSF):  Established  Program  to 
Stimulate  Competitive  Research  (EPSCoR)  and  the  Broadening  Participation 
Portfolio (US) 

1070 

 
 
•  Evaluation of the National Science Foundation’s Experimental Program to Stimulate 

Competitive Research (EPSCoR): Final Report (2014). 
https://www.ida.org/~/media/Corporate/Files/Publications/STPIPubs/2015/P-5221.ashx 

• 

L. Harris, (2017). Established Program to Stimulate Competitive Research (EPSCoR): 
Background and Selected Issues. 
https://crsreports.congress.gov/product/pdf/R/R44689/5 

•  EPSCoR Factsheet: https://nsf.gov/news/factsheets/EPSCoR_Factsheet_final.pdf 

•  NSF and EPSCoR website 

• 

• 

https://www.nsf.gov/ 

https://beta.nsf.gov/funding/initiatives/epscor  

•  NSF 21-088; Dear Colleague Letter: Envisioning the Future of NSF EPSCoR. 

https://www.nsf.gov/pubs/2021/nsf21088/nsf21088.jsp  

•  Established Program to Stimulate Competitive Research (EPSCoR): Background and 

Selected Issues (2017), Congressional Research Service. R44689. 
https://crsreports.congress.gov  

•  Wu, Y. (2010). Tackling undue concentration of federal research funding: An empirical 
assessment on NSF's Experimental Program to Stimulate Competitive Research 
(EPSCoR). Research Policy, 39(6), 835-841. https://doi.org/10.1007/978-3-030-18704-
0_7 

•  Wu, Y. (2020). The Funding of Academic Research in the U.S. In: America's Leaning 

Ivory Tower. SpringerBriefs in Political Science. Springer, Cham. 
https://doi.org/10.1007/978-3-030-18704-0_1    

•  Melkers, J., & Wu, Y. (2009). Evaluating the improved research capacity of EPSCoR 

states: R&D funding and collaborative networks in the NSF EPSCoR program. Review 
of Policy Research, 26(6), 761-782. 

Benchmark 2: Canada Excellence Research Chairs Programme (CERC) 

•  Canada's Science, Technology and Innovation Strategy. 

https://www.canada.ca/en/news/archive/2014/12/canada-science-technology-
innovation-strategy.html 

•  Website of the CERC programme: https://www.cerc.gc.ca 

•  SSHRC/NSERC Evaluation Division (2020). Evaluation of the Canada Excellence 

Research Chairs Program and the Canada 150 Research Chairs Program. Final report. 
https://www.cerc.gc.ca/about-au_sujet/publications/2020/CERC-C150-Evaluation-
Report-En.pdf  

•  Evaluation of the Canada Excellence Research Chairs Program and the Canada 150 

Research Chairs Program: Management Response and Action Plan. 

1071 

 
 
https://www.cerc.gc.ca/about-au_sujet/publications/evaluation_response-
evaluation_reponse_2020-eng.aspx 

•  SSHRC/NSERC Evaluation Division (2014). Evaluation of the Canada Excellence 

Research Chairs (CERC) Program. Final Evaluation Report. 
https://www.cerc.gc.ca/about-
au_sujet/publications/CERC_Evaluation_Report_FinalE.pdf 

•  Management Response to the Evaluation of the Canada Excellence Research Chairs 

(CERC) Program Final Evaluation Report. https://www.cerc.gc.ca/about-
au_sujet/publications/evaluation_response_2014-eng.aspx 

•  New Frontiers in Research Fund. https://www.sshrc-crsh.gc.ca/funding-

financement/nfrf-fnfr/index-eng.aspx 

•  Canada First Research Excellence Fund. https://www.cfref-apogee.gc.ca/home-

accueil-eng.aspx  

•  Canada’s Science, Technology and Innovation strategy, 

https://www.canada.ca/en/news/archive/2014/12/canada-science-technology-
innovation-strategy.html  

Benchmark 3: Max Planck Centres and Dioscuri Centres of Scientific Excellence  

•  Governance: Mission, Managing Risks and Compliance. p.34. Consulted through: 

https://www.mpg.de/18150641/governance-mission-managing-risks-and-
compliance.pdf 

•  Max Planck Schools – Cutting-Edge Research with International Appeal. 
https://www.bmbf.de/bmbf/en/academia/max-planck-schools/max-planck-
schools_node.html  

•  Max Planck Website: https://www.mpg.de/ 

•  Website of the Max Planck Centres and Dioscuri Centres 

•  Horizon Dashboard: https://ec.europa.eu/info/funding-

tenders/opportunities/portal/screen/opportunities/horizon-dashboard 

Benchmark 4: National Natural Science Foundation of China (NSFC) 

• 

• 

gov.cn / Government of the PRC (2012): ”Opinions on Deepening the Reform of the 
Science and and Technology System and Accelerating the Construction of a National 
Innovation System" (关于深化科技体制改革加快国家创新体系建设的意见), 
http://www.gov.cn/jrzg/2012-09/23/content_2231413.htm 

Liu Yin (2022-06-06):】85% of NSF projects have been categorized （分类评审方面）
and evaluated Date 2022-06-06, in: Science and Technology Daily, 
https://www.nsfc.gov.cn/publish/portal0/tab440/info85855.htm 

1072 

 
 
•  NCSTE 2022 / National Center for Science and Technology Evaluation (2022): NSF 

Performance Evaluation Report 2021 (国家自然科学基金 2021 年度 绩效评价报告), p. 
72 ff. https://www.nsfc.gov.cn/Portals/0/fj/fj20220413_01.pdf  

•  NSFC 2022: Application requirements, 申请规定, 

https://www.nsfc.gov.cn/publish/portal0/tab1100/ (Accessed Sept. 27, 2022) 

•  NSFC: “Reform Initiatives for 2022” (Chinese: 2022年度国家自然科学基金改革举措), 

https://www.nsfc.gov.cn/publish/portal0/tab1099/ (Accessed Sept. 17, 2022) 

•  NSFC Yearbook 2012. https://www.nsfc.gov.cn/nsfc/cen/xmtj/pdf/2012_table.pdf 

•  NSFC Yearbook 2021. https://www.nsfc.gov.cn/publish/portal0/tab1374/ (Accessed 

Sept. 5, 2022) 

•  OECD Main Science and Technology Indicators. 

https://stats.oecd.org/viewhtml.aspx?datasetcode=MSTI_PUB&lang=en# (Accessed 
Sept. 5, 2022) 

•  Poo, M. (2021): Innovation and reform: China’s 14th Five-Year Plan unfolds. National 

Science Review 8, doi: 10.1093/nsr/nwaa294 

•  Poo, M. (2022): New focus on basic research in China’s advancement in science and 
technology. National Science Review – Editorial, https://doi.org/10.1093/nsr/nwac014  

•  RMRB 2011 / People’s Daily (人民日报)Where are the trillions of research funding 

going? (万亿科研经费到底去哪了), 
http://www.ipe.cas.cn/dzqk/2016/01/mtsm/201607/t20160704_4634932.html 

Phase 2 benchmarks 

The list of literature and information sources consulted while preparing Phase 2 case studies 
will be compiled and provided together with the Second Interim Report. 

Benchmark 1: The Swiss National Foundation (SNSF) 

•  European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R., et al., Evaluation study of the European framework 
programmes for research and innovation for excellent science: Horizon 2020: phase 1 
final study report, Publications Office of the European Union, 2023, 
https://data.europa.eu/doi/10.2777/967813. 

•  European Commission, She figures 2021, in https://www.rathenau.nl/en/science-
figures/personnel/women-science/share-female-professors-netherlands-and-eu-
countries, accessed on 13-07-2023. 

•  Federal Confederation Switzerland. (2023). Status Update: Swiss participation in 

Horizon Europe and related programmes and initiatives. 

1073 

 
 
https://www.sbfi.admin.ch/dam/sbfi/en/dokumente/2019/02/horizon-
europe.pdf.download.pdf/Fact-sheet_Horizon_en.pdf.  accessed on 3-04-2023. 

•  Federal Council Switzerland. Federal Constitution of the Swiss Confederation. 

https://www.fedlex.admin.ch/eli/cc/1999/404/en. 

•  Federal Council Switzerland. (2023). Talks conducted by the President of the Swiss 

Confederation in Davos focused on security situation in Europe and the work of the UN 
Security Council. https://www.admin.ch/gov/en/start/documentation/media-
releases.msg-id-92581.html.   

•  Federal Council Switzerland. Federal Act of 14 December 2012 on the Promotion of 
Research and Innovation (RIPA). https://www.fedlex.admin.ch/eli/cc/2013/786/en. 

•  Federal Council Switzerland. Talks conducted by the President of the Swiss 

Confederation in Davos focused on security situation in Europe and the work of the UN 
Security Council. https://www.admin.ch/gov/en/start/documentation/media-
releases.msg-id-92581.html. 

•  Feresin, E. (2022). What’s on the horizon for Swiss research in Europe? Newsarticle: 
https://www.swissinfo.ch/eng/sci-tech/what-s-on-the-horizon-for-swiss-research-in-
europe-/47490296. 

•  Flücker, Y. (2021). Why is Horizon Europe important for Switzerland and the European 
Union? Newsarticle: https://www.swissinfo.ch/eng/sci-tech/why-is-horizon-europe-
important-for-switzerland-and-the-european-union-/46792306. 

• 

LinkedIn Post by Rutzer C., Swiss STEM research at risk due to lack of Horizon Europe 
funding?  https://lk.linkedin.com/posts/christian-rutzer-38978392_horizoneurope-
innoscape-switzerland-activity-7038462335228768256-bD9D. 

•  Matthews, D., (2023). EU and Switzerland talk about Horizon Europe at Davos. 

Newsarticle: https://sciencebusiness.net/news/Horizon-Europe/eu-and-switzerland-talk-
about-horizon-europe-
davos#:~:text=Because%20Switzerland%20is%20not%20associated,they%20want%2
0to%20join%20consortia. 

•  SBFI Admin. Facts and Figures on the Swiss Participation in the EU-Framework 

Programmes for Research and Innovation. 
https://www.sbfi.admin.ch/sbfi/en/home/research-and-innovation/international-
cooperation-r-and-i/eu-framework-programmes-for-research/f-f-swiss-participation.html. 

•  Science Business. (2023). Switzerland’s non-association to Horizon Europe could hit 

STEM subjects hard - with consequences for innovation. Newsarticle: 
https://sciencebusiness.net/news-byte/switzerlands-non-association-horizon-europe-
could-hit-stem-subjects-hard-consequences. 

•  SNSF. Funding Regulations. Regulations of the Swiss National Science Foundation on 

research grants. 
https://www.snf.ch/en/97JmAecSwqB12X8m/page/funding/documents-
downloads/regulations-funding-regulations. 

1074 

 
 
•  SNSF. General implementation regulations for the Funding Regulations. 
https://www.snf.ch/en/Zi65otScMDqzyALN/page/funding/documents-
downloads/regulations-general-implementation-regulations. 

•  Swiss Confederation. Federal Department of Economic Affairs, Education and 

Research EAER. Swiss Science Council SSC. (2022). Evaluation of the Swiss National 
Science Foundation. 
https://wissenschaftsrat.ch/images/stories/pdf/en/SWR_2022_SNF_evaluation_annexe
_I-IV.pdf; 
https://wissenschaftsrat.ch/images/stories/pdf/en/SWR_2022_SNF_evaluation_annexe
_IX-XIII.pdf. 

•  Swiss Confederation. SERI (2019). The Impact of Switzerland’s Participation in the EU 

Framework Programmes. 
https://www.sbfi.admin.ch/dam/sbfi/en/dokumente/webshop/2019/impact-studie-
2019.pdf.download.pdf/Impact_Studie_en.pdf. 

•  Washington O. (2023). Uni-Rektoren alarmiert: «Die Situation ist untragbar». 

Newsarticle: https://www.srf.ch/news/schweiz/blockade-im-eu-dossier-uni-rektoren-
alarmiert-die-situation-ist-untragbar?ns_source=mobile&srg_sm_medium=tw  

•  Website SNSF and all its pages. www.snsf.ch/en.  

•  Websites and relevant pages from the Swiss government, SERI, and Innosuisse. 

Benchmark 2: Austrian Science Fund (FWF Austria) 

•  Corporate website of FWF Austria; 

•  Annual FWF Austria reports 2018-2022; 

•  Austrian Research and Technology reports (2018-2022); 

•  RTI Strategy 2030 Strategy for Research, Technology and Innovation of the Austrian 

Federal Government (2020); 

•  OECD Reviews of Innovation Policy: Austria 2018 

•  FWF-Dashboard 

• 

Janger, Schmidt & Strauss (2019) International differences in basic research grant 
funding – as systematic comparison. Austrian Institute of Economic Research. 

Benchmark 3: Australian Research Council (ARC) 

•  Appeals against NCGP administrative processes | Australian Research Council. (n.d.). 

https://www.arc.gov.au/about-arc/our-organisation/committees/appeals-against-ncgp-
administrative-processes  

1075 

 
 
•  ARC Centres of Excellence | Australian Research Council. (n.d.). 

https://www.arc.gov.au/funding-research/funding-schemes/linkage-program/arc-
centres-excellence  

•  Australian Research Council. (2023b, February 10). Australian Research Council 

Annual Report 2021-22Transparency Portal. https://www.transparency.gov.au/annual-
reports/australian-research-council/reporting-year/2021-22 

•  GrantConnect—the Australian Government’s grant information system | Australian 

Research Council. (n.d.). https://www.arc.gov.au/funding-research/apply-
funding/grantconnect   

• 

• 

• 

• 

• 

Industrial Transformation Research Hubs | Australian Research Council. (n.d.-b). 
https://www.arc.gov.au/funding-research/funding-schemes/linkage-program/industrial-
transformation-research-program/industrial-transformation-research-hubs  

Industrial Transformation Research Program | Australian Research Council. (n.d.). 
https://www.arc.gov.au/funding-research/funding-schemes/linkage-program/industrial-
transformation-research-program  

Industrial Transformation Training Centres | Australian Research Council. (n.d.). 
https://www.arc.gov.au/funding-research/funding-schemes/linkage-program/industrial-
transformation-research-program/industrial-transformation-training-centres  

Linkage Infrastructure, Equipment and Facilities (LIEF) | Australian Research Council. 
(n.d.). https://www.arc.gov.au/funding-research/funding-schemes/linkage-
program/linkage-infrastructure-equipment-and-facilities  

Linkage Projects | Australian Research Council. (n.d.). https://www.arc.gov.au/funding-
research/funding-schemes/linkage-program/linkage-projects   

•  Research Management System (RMS) Information | Australian Research Council. 

(n.d.). https://www.arc.gov.au/manage-your-grant/research-management-system-rms-
information  

•  Selection Advisory Committees | Australian Research Council. (n.d.). 

https://www.arc.gov.au/about/our-organisation/committees/selection-advisory-
committees  

•  Selection Outcome Reports | Australian Research Council. (n.d.). 

https://www.arc.gov.au/funding-research/funding-outcome/selection-outcome-reports  

•  Special Research Initiatives | Australian Research Council. (n.d.). 

https://www.arc.gov.au/funding-research/funding-schemes/linkage-program/special-
research-initiatives 

Benchmark 4: United Kingdom Research & Innovation (UKRI) 

•  Department for Business, UK’s Research, Development and Innovation Organisational 

Landscape: Final Report and Recommendations, 2023, available at: 
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachme
nt_data/file/1141484/rdi-landscape-review.pdf. 

1076 

 
 
•  Department for Business, Energy and Industrial Strategy, Industrial Strategy: Building a 

Britain fit for the future, 2018, available at: 
https://www.gov.uk/government/publications/industrial-strategy-building-a-britain-fit-for-
the-future. 

•  Department for Business, Energy and Industrial Strategy, UKRI Framework Document, 
2018, available at: https://www.ukri.org/wp-content/uploads/2020/10/UKRI-111020-
UKRIFrameworkDocument.pdf. 

•  Department for Business, Energy and Industrial Strategy, BEIS research and 
development (R&D) budget allocations 2021 to 2022, 2021, available at: 
https://www.gov.uk/government/publications/beis-research-and-development-rd-
budget-allocations-2021-to-2022/beis-research-and-development-rd-budget-
allocations-2021-to-2022. 

•  Department for Business, Innovation and skills, Department for Energy Security & Net 

Zero, Case for the creation of UK Research and Innovation, 2016, available at: 
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachme
nt_data/file/527803/bis-16-291-ukri-case-for-creation.pdf. 

•  Department for Science, Innovation & Technology and Department for Business, 

Energy & Industrial Strategy, Supporting UK R&D and collaborative research beyond 
European programmes, 2022, available at: 
https://www.gov.uk/government/publications/supporting-uk-rd-and-collaborative-
research-beyond-european-programmes. 

•  Economic and Economic Research Council, Strategic Delivery Plan, 2022-25, available 

at: https://www.ukri.org/wp-content/uploads/2022/09/ESRC-010922-
StrategicDeliveryPlan2022.pdf. 

•  European Commission, 2023, European Innovation Scoreboard 2023 – Country profile 
United Kingdom, available at: https://ec.europa.eu/assets/rtd/eis/2023/ec_rtd_eis-
country-profile-uk.pdf. 

•  European Commission, EU Missions in Horizon Europe, available at: https://research-
and-innovation.ec.europa.eu/funding/funding-opportunities/funding-programmes-and-
open-calls/horizon-europe/eu-missions-horizon-europe_en. 

•  European Commission, New Zealand joins Horizon Europe Research and Innovation 

Programme, available at: 
https://ec.europa.eu/commission/presscorner/detail/en/ip_23_3728. 

•  European Commission; Joint Statement by the European Commission and the UK 

Government on the UK's association to Horizon Europe and Copernicus, available at: 
https://ec.europa.eu/commission/presscorner/detail/en/statement_23_4375. 

•  Fiona Lettice, Horizon Europe: how the UK’s delay in rejoining EU funding scheme is 
damaging scientific research, The Conversation, https://theconversation.com/horizon-
europe-how-the-uks-delay-in-rejoining-eu-funding-scheme-is-damaging-scientific-
research-209279. 

1077 

 
 
•  GOV.UK, UK Research and Development Roadmap, 2021, available at: 

https://www.gov.uk/government/publications/uk-research-and-development-
roadmap/uk-research-and-development-roadmap. 

•  Guide for Peer Reviewers to the Synergy Grant call, available at: 

https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/experts/guide-
for-peer-reviewers_he-erc-syg_en.pdf. 

•  Guild of European Research-Intensive Universities, Lessons learned and way forward: 

The Guild’s recommendations for Horizon Europe, available at: https://www.the-
guild.eu/publications/position-papers/lessons-learned-and-way-forward-the-guild-s-
recommendations-for-horizon-europe.pdf. 

•  HM Treasury, Budget 2018, available at: 

https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachme
nt_data/file/752202/Budget_2018_red_web.pdf. 

• 

• 

Initiative for Science in Europe, Executive Summary: Position on Horizon Europe, 
available at: https://initiative-se.eu/wp-content/uploads/2021/07/ISE-position-Horizon-
Europe.pdf. 

League of European Research Universities (LERU), LERU key messages for the 
current and future European R&I Framework Programme, available at: 
https://www.leru.org/files/LERU_statement_EU-FP_consultation_2023-02.pdf.  

•  Russell Group, Horizon Europe: Maintaining UK leadership in global R&D collaboration, 

2022, available at: https://russellgroup.ac.uk/media/6054/benefits-of-horizon-
europe.pdf. 

•  Sciencebusiness.net, Here’s what the first two years of Horizon Europe look like in 

numbers, 2023, available at: https://sciencebusiness.net/news/Horizon-Europe/heres-
what-first-two-years-horizon-europe-look-numbers. 

•  Scottish Parliament, Spice Spotlight, EU-UK relations – Developments on the Horizon, 

available at: https://spice-spotlight.scot/2023/07/13/eu-uk-relations-developments-on-
the-horizon/. 

•  Technopolis Group, Strategic Priorities Fund (SPF) Baseline and Interim Process 

Evaluation – Main Report (v3), 2022, available at: 
https://www.ukri.org/publications/strategic-priorities-fund-baseline-and-interim-process-
evaluation/. 

•  UKRI Investment and Outputs Data 2022-23, available at: https://www.ukri.org/wp-
content/uploads/2023/08/UKRI-070823-InvestmentsOutputsData2022To2023.pdf. 

•  UKRI Medical Research Council, Guidance for Peer Reviewers, 2023, available at: 

https://beta.ukri.org/wp-content/uploads/2023/06/MRC-280623-
GuidanceforPeerReviewers-EmbeddingDiversityUpdates-June-2023.pdf.  

•  UKRI, Principles of assessment and decision making, 2021, available at: 

https://www.ukri.org/wp-content/uploads/2021/03/UKRI-310321-Principles-of-
Assessment-and-Decision-Making-V2.pdf. 

1078 

 
 
•  UKRI, 2021/22 budget allocations for UK Research and Innovation, available 

at:https://www.ukri.org/wp-content/uploads/2022/06/UKRI-270521-UKRI-Allocation-
Explainer-2021-22-FINAL-PDF.pdf.  

•  UKRI, UKRI International Strategic Framework, 2022, available 

at:https://www.ukri.org/wp-content/uploads/2022/10/UKRI-300922-
InternationalStrategicFramework.pdf. 

•  UKRI, UKRI strategy 2022 to 2027: transforming tomorrow together, 2022, available at: 

https://www.ukri.org/publications/ukri-strategy-2022-to-2027/. 

•  UKRI, 2022-23 – 2024-25 budget allocations for UK Research and Innovation, 

available at: UKRI-090822-BudgetAllocationExplainer-2022To2023-2023To2024-
2024To2025.pdf. 

•  UKRI, Arts and Humanities Research Council, Strategic Delivery Plan, 2022-25, 
available at: https://www.ukri.org/wp-content/uploads/2022/09/AHRC-010922-
StrategicDeliveryPlan2022.pdf. 

•  UKRI, How we are governed, available at: https://www.ukri.org/about-us/nerc/how-we-

are-governed/. 

•  UKRI, How we’re funded, available at: https://www.ukri.org/wp-

content/uploads/2022/08/UKRI-050822-FundingExplainer-HowWereFunded.pdf. 

•  UKRI, Our Budget, available at: https://www.ukri.org/about-us/who-we-are/our-budget/. 

•  UKRI, The Strategic Priorities Fund, available at: https://www.ukri.org/what-we-do/our-

main-funds-and-areas-of-support/strategic-priorities-fund/. 

•  World Intellectual Property Organisation, Global Innovation Index 2022 What is the 

future of innovation-driven growth?, 2022, available 
at:https://www.wipo.int/global_innovation_index/en/2022/. 

1079 

 
 
 
 
Annex 6: Quantitative research track findings 

1.1.  Efficiency indicators 

This annex presents preliminary data and findings from the analysis of selected efficiency-
related indicators. In line with its proposed methodology, the study team calculated data for 
the following key metrics: 

•  The efficiency of proposal evaluation processes in the programmes analysed: time-to-
inform (TTI), time-to-sign (TTS), time-to-grant (TTG), the share of eligible/ineligible 
proposals, and the success rate for all applications, as well as for high-quality 
applications. 

•  The efficiency of grant management processes in the programmes analysed: average 

grant size, time-to-pay, and the number of projects funded via different types of funding 
schemes. 

1.1.1.  Results at the level of Horizon Europe 

The data and corresponding indicators in the tables below present overall efficiency figures 
for the Horizon Europe programme. These figures are presented by thematic priority/pillar 
and year.  

The figure below presents the indicators used to assess the efficiency of the application and 
selection process. According to official Horizon Europe targets, time-to-inform (TTI) should 
be up to five months after the call for proposals is closed. Time-to-sign (TTS) should be no 
longer  than  three  months  after  the  notification  on  the  evaluation  outcome  reaches  the 
applicants. Time-to-sign is only calculated for successful proposals (i.e. awarded projects). 
This process should not exceed eight months; the summation of TTI and TTS is called time-
to-grant (TTG).  

Horizon 
Europe 
thematic 
priority 

Total number of 
applications 
analysed1341 

Average TTI 
(days) 

Average TTS 
(days) 

Average TTG 
(days) 

Pillar 1 

34 336 (68.1%) 

171.8 

98.8 

292.3 

Pillar 2 

11 219 (22.3%) 

111.6 

128.5 

237.5 

Pillar 31342 

3 770 (7.5%) 

140.7 

110.2 

240.0 

1341 The total number of applications (n) is based on the data availability for TTI, i.e. if there are 
no data available for the date of the proposal information letter then the proposal is not included 
in this table. 
1342 Data excludes EIT and EIC Accelerator. 

1080 

 
 
 
Horizon 
Europe 
thematic 
priority 

Total number of 
applications 
analysed1341 

Average TTI 
(days) 

Average TTS 
(days) 

Average TTG 
(days) 

WIDERA 

1 107 (2.2%) 

117.9 

99.1 

223.8 

Total 

50 432 (100%) 

154.9 

106.9 

273.0 

Table 1. Overview of project application and selection process indicators, by thematic priority. Source: compiled by the study 
team using eCORDA data. 

The application and selection process for 2021 exceeded the target time for TTG by 51 days 
(295.3). In 2022 the TTG improved to 246.8 days. Similarly, TTI for 2021 (158.9) exceeded 
the maximum target of 152 days by approximately 7 days, whereas TTI for 2022 was within 
the target. It is unsurprising that the initial years were less efficient, given that quite a few 
changes were introduced into Horizon Europe, requiring administrative shifts. Data for 2023 
are quite limited so far, and do not yet constitute the entire year, prohibiting the drawing of 
meaningful conclusions. 

Call closure 
year 

Total number of 
applications 
analysed 

Average TTI 
(days) 

Average TTS 
(days) 

Average TTG 
(days) 

2021 

2022 

24 975 (51.3%) 

158.9 

118.9 

25 451 (48.7%) 

151.1 

20231343 

6 (0.01%) 

46.5 

Total1344 

50 432 (100%) 

154.9 

106.9 

92.7 

62.3 

295.3 

246.8 

109.3 

273.0 

Table 2. Overview of project application and selection process indicators, by call closure year. Source:compiled by the study 
team using eCORDA data, calculations are presented per master call year. 

The  average  share  of  successful  proposals  out  of  all  eligible  proposals  is16%,  indicating 
relatively  high  competition. The  average  success  rate  for high-quality  proposals  is  27.7%. 
Out of the four thematic priorities, WIDERA has the highest average success rates, at 27.8% 
for all proposals and 40.4% for high-quality proposals. When looking at the ineligibility rates, 
4% of proposals received under Horizon Europe were deemed ineligible, a higher rate than 
the Horizon 2020 average of 1.1%. 

1343 Data for 2023 cover the period up to the date when data were received by the study team, 
and do not reflect the total figures for 2023. 
1344 Data exclude EIT and EIC Accelerator. 

1081 

 
 
 
Horizon 
Europe 
thematic 
priority 

Total number of 
applications 
analysed1345 

Share of 
ineligible 
proposals1346 

Average success 
rate (all 
proposals)1347 

Average 
success rate 
(only high-
quality 
proposals) 

Pillar 1 

34 547 (67.1%) 

1.3% 

15.3% 

29.9% 

Pillar 2 

12 044 (23.4%) 

11.3% 

19.0% 

31.3% 

Pillar 31348 

3 802 (7.9%) 

6.2% 

10.2% 

19.2% 

WIDERA 

1 110 (2.2%) 

2.8% 

27.8% 

40.4% 

Total 

51 503 (100%) 

4.0% 

16.0% 

29.8% 

Table 3. Overview of the level of competition, by thematic priority. Source: compiled by the study team using eCORDA data. 

The table below presents the success rates and the shares of ineligible proposals in 2021, 
2022 and the initial months of 2023. There were almost twice as many ineligible proposals in 
2022 (5.7%) than in 2021 (2.2%); however, the success rate was also higher in 2022 (17.0%). 
Since only six applications were analysed in the dataset for 2023, the success rate is 100%, 
but this cannot be taken as representative for the entire year. 

Call 
closure 
year 

Total number of 
applications 
analysed 

Share of 
ineligible 
proposals 

Average 
success rate (all 
proposals) 

Average success 
rate (only high-
quality proposals) 

2021 

25 023 (48.6%) 

2.2% 

14.9% 

2022 

26 474 (51.4%) 

5.7% 

17.0% 

20231349 

6 (0.01%) 

- 

100% 

Total1350 

51 503 (100%) 

4.0% 

16.0% 

28.4% 

31.0% 

100% 

29.8% 

Table 4. Overview of the level of competition, by call closure year. Source: compiled by the study team using eCORDA data, 
calculations are presented per master call year. 

1345 Total number of applications includes all applications submitted for evaluation. Also applies 
to Table 4. 
1346 Eligible proposals are those with the status Main, No Money, Rejected, Reserve and 
Withdrawn; Ineligible proposals are those with the status Inadmissible or Ineligible. 
1347 Share of successful proposals out all eligible applications (also applies to Table 4). 
1348 Data exclude EIT and EIC Accelerator. 
1349 The data for 2023 cover the period up to the date when data were received by the study 
team, and do not reflect the total figures for 2023. 
1350 Data exclude EIT and EIC Accelerator. 

1082 

 
 
 
The time between when payment was requested and when it was made (time-to-pay, TTP) 
serves  as  an  indicator  for  evaluating  the  efficiency  of  the  project  implementation  and 
management process. Average TTP under Horizon Europe was 17.1 days. There are two 
main types of payments, which also have different targets – pre-financing, which has a target 
of  30  days;  and  normal  payments,  with  a  target  of  90  days.  Over 99.2% of  the  payments 
analysed  in  this  study  were  pre-financed,  with  one  project  sometimes  receiving  multiple 
payments. Overall, 82.1% of payments were made on time, which is slightly lower than the 
total share under Horizon 2020 (91.5%). 

Horizon 
Europe 
thematic 
priority 

Total 
number 
of 
projects 
analysed 

Total 
number 
of 
payments 
made1351 

Average 
TTP: 
pre-
financing 

Share of 
pre-
financing 

Average 
TTP: 
normal 
payments 

Share of 
normal 
payments 

Average 
TTP: 
total 

Share of 
payments 
within the 
target1352 

Pillar 1 

Pillar 2 

Pillar 
31353 

WIDERA 

Total 

2 389 
(52.3%) 

1 652 
(36.2%) 

269 
(5.9%) 

254 
(5.6%) 

4 564 
(100%) 

2 469 

13.5 

99.6% 

66.3 

0.2% 

13.6 

89.1% 

2 533 

19.3 

99.5% 

80.8 

0.2% 

19.4 

76.8% 

337 

18.8 

94.5% 

50.1 

3.8% 

19.9 

79.5% 

271 

25.4 

99.3% 

74.5 

0.7% 

25.8 

72.0% 

5 610 

17.0 

99.2% 

62.5 

0.5% 

17.1 

82.1% 

Table 5. Overview of payments data and TTP, by thematic priority1354. Source: compiled by the study team using eCORDA 
data. 

To  offer  a  point  of  reference  and  a  broad  perspective,  the  study  team  computed  average 
project size and quantity of EC contribution by year, thematic priority and type of funding. So 
far, the total budget committed is EUR 21 961.9 million, with Pillar 2 accounting for over half 
of this total.  

Horizon 
Europe 
thematic 
priority 

Pillar 1 

Pillar 2 

Pillar 31355 

2021 

2022 

2023 

Total (average) 

1.2 

5.9 

2.1 

1.2 

6.6 

2.7 

0.9 

- 

- 

1.2 

6.3 

2.5 

Total budget 
committed (until 
20 June 2023) 

6 101.7 

14 031.4 

985.0 

1351 Negative payments and regularisation are excluded.  
1352 The target for pre-financing is 30 days; for other payments the target is 90 days.  
1353 Data exclude EIT and EIC Accelerator. 
1354 Refers only to projects with the status ‘closed’ or ‘signed’. 
1355 Data exclude EIT and EIC Accelerator. 

1083 

 
 
 
Horizon 
Europe 
thematic 
priority 

WIDERA 

Total 

2021 

2022 

2023 

Total (average) 

Total budget 
committed (until 
20 June 2023) 

2.6 

2.3 

2.8 

3.0 

1.7 

1.2 

2.7 

2.7 

843.7 

21 961.9 

Table 6. Average project size, calculated based on EU contribution by thematic priority, expressed in million EUR1356. 
Source: compiled by the study team using eCORDA data. 

Type of action 

Pillar 1 

Pillar 2 

Pillar 3 

WIDERA 

RIA (Research and 

Innovation Actions 

CSA (Coordination and 

Support Actions) 

EIC grants 

49 

97 

- 

ERC grants1357 

2 373 

IA (Innovation Actions) 

JU CSA (Joint 

Undertakings) 

JU IA 

JU RIA 

Pre-commercial 

Procurement 

Programme Cofund 

- 

- 

- 

- 

- 

- 

1 225 

1 

9 

223 

- 

- 

479 

19 

79 

166 

2 

9 

108 

276 

- 

- 

- 

- 

- 

- 

2 

252 

- 

- 

- 

- 

- 

- 

- 

- 

1356 Calculations exclude projects with the status ‘rejected’, ‘suspended’ or ‘terminated’. The 
same applies to Table 7. 
1357 Includes ERC Proof of Concept and Synergy Grants. 

1084 

 
 
 
Type of action 

Pillar 1 

Pillar 2 

Pillar 3 

WIDERA 

TMA (Transnational 

Mobility Activities) 
MSCA1358 

2 570 

- 

- 

49 

Total 

5 089 

2 211 

387 

311 

Table 7. Number of projects, by thematic priority and type of action 

To  alleviate  the  administrative  and  financial  burden,  Horizon  Europe  implemented  Lump 
Sums,  which  are  predetermined,  fixed  amounts  specified  in  the  grant  agreement.  This 
measure  is  aimed  at  facilitating  easier  access  to  the  programme,  particularly  for  small 
organisations and newcomers. During the funding period analysed, a total of 1 565 proposals 
applied  Lump  Sum  funding.  Furthermore,  most  proposals  were  funded  through  Action 
Grants, amounting to 30 401 proposals. Unit grants constituted the second-largest funding 
category, supporting 17 606 proposals. These figures are reflected in the table below. 

Horizon 
Europe 
thematic 
priority 

Action 
Grants 

Framework 
Partnership 

Lump 
Sum 

Specific 
Grant 

Horizon 
Prizes 

Unit 
Grant 

Pillar 1 

15 591 

Pillar 2 

10 805 

Pillar 31359 

3 003 

WIDERA 

1 002 

1 

10 

- 

1 

919 

82 

563 

1 

2 

8 

- 

1 

66 

17 532 

- 

- 

- 

- 

- 

74 

Total 

30 401 

12 

1 565 

11 

66 

17 606 

Table 8. Number of proposals by type of funding1360. Source: compiled by the study team using eCORDA data. 

1.1.2.  Detailed analysis: Pillar 1 programmes 

In  line  with  the  analysis  for  Horizon  Europe  overall,  we  provide  the  corresponding  figures 
relating only to the Programme parts in Pillar 1. 

The  European  Research  Council  (ERC)  has  a  higher  TTG  than  the  rest  of  the  FP  –  a 
maximum of 395 days, as defined by Horizon Europe framework. The TTG for the for ERC 

1358 Includes MSCA Postdoctoral Fellowships; Postdoctoral Fellowships – European 
Fellowships/Global Fellowships, Cofund Doctoral/Postdoctoral programmes, Doctoral Networks; 
Staff Exchanges. 
1359 Data excludes EIT and EIC Accelerator. 
1360 Excludes proposals with the status ‘inadmissible’ or ‘ineligible’. 

1085 

 
 
 
in the period analysed by the study team is 356.7, and thus falls within this limit. European 
Research  Infrastructures  (INFRA)  was  the  most  efficient  programme  part,  with  a  TTG  of 
226.3 days and a TTI of 96.8 days. This may be partially explained by the relatively small 
number of applications and the thematic targets of this programme part. 

Horizon 
Europe 
thematic 
priority 

Total number of 
applications 
analysed1361 

Average TTI 
(days) 

Average TTS 
(days) 

Average TTG 
(days) 

ERC 

16 336 (47.6%) 

193.7 

107.2 

356.7 

MSCA 

17 835 (51.9%) 

152.5 

91.2 

244.4 

INFRA 

165 (0.5%) 

96.8 

130.3 

226.3 

Total for Pillar 1 

34 336 (100%) 

171.8 

98.8 

292.3 

Table 9. Overview of project application and selection process indicators, by programme. Source: compiled by the study 
team using eCORDA data. 

Among  those  years  assessed,  2021  emerged  as  least  efficient  in  terms  of  TTG,  with  an 
average of 314.4 days. In 2021, Pillar 1 also slightly exceeded the maximum TTS of 91 days 
by 8 days, with a total 98.8 days. As yet, the data for 2023 are insufficient to draw meaningful 
conclusions. 

Call closure 
year 

Total number of 
applications 
analysed 

Average TTI 

Average TTS 

Average TTG 

2021 

2022 

18 749 (57.3%) 

169.0 

116.7 

314.4 

15 583 (42.7%) 

175.3 

20231362 

4 (0.01%) 

45.3 

Total for Pillar 1 

34 336 (100%) 

171.8 

73.3 

59 

98.8 

261.2 

105.3 

292.3 

Table 10. Overview of project application and selection process indicators, by year. Source: compiled by the study team 
using eCORDA data. 

The overall success for Pillar 1 was 15.3%, and the success rate of high-quality proposals 
was 29.9%. INFRA was the most successful programme part, with an average success rate 
of  53.4%.  It  was  also  the  programme  part  with  the  highest  share  of  ineligible  proposals, 
11.4%. These numbers are probably linked to the nature of the programme part, as it aims 
to  support  the  development  and  operation  of  large-scale  research  infrastructures  across 

1361 The total number of applications (n) is based on the availability of data for TTI. 
1362 The data for 2023 cover the period up to the date when data were received by the study 
team, and do not reflect the total figures for 2023. 

1086 

 
 
 
various  scientific  domains.  The  ERC  had  the  lowest  share  of  ineligible  proposals  and  the 
lowest success rate. 

Horizon 
Europe 
thematic 
priority 

Total number of 
applications 
analysed 

Share of 
ineligible 
proposals1363 

Average 
success rate 
(all proposals) 

Average success 
rate (only high-
quality 
proposals)1364 

ERC 

16 500 (47.8%) 

1.2% 

14.0% 

57.4% 

MSCA 

17 880 (51.8%) 

1.3% 

16.1% 

INFRA 

167 (0.5%) 

11.4% 

53.4% 

21.4% 

65.8% 

Total for 
Pillar 1 

34 547 (100%) 

1.3% 

15. 3% 

29.9% 

Table 11. Overview of the level of competition, by programme. Source: compiled by the study team using eCORDA data. 

Call closure 
year 

Total number of 
applications 
analysed 

Share of 
ineligible 
proposals 

Average 
success rate (all 
proposals) 

Average 
success rate 
(only high-
quality 
proposals) 

2021 

2022 

20231365 

Total for 
Pillar 1 

18 789 (54.4%) 

1.2% 

13.8% 

27.8% 

15 754 (45.6%) 

1.3% 

17.0% 

32.1% 

4 (0.01%) 

- 

100% 

100% 

34 547 (100%) 

1.3% 

15.3% 

29.9% 

Table 12. Overview of the level of competition, by call closure year. Source: compiled by the study team using eCORDA 
data. 

On  average,  89.1%  of  the  payments  were  made  within  the  target.  The  most  efficient 
programme part was the ERC, with an average TTP of 11.5 days and 91.9% of payments 
made on time. Over 95% of Pillar 1 is financed via pre-financing. There were only four projects 
in the Marie Skłodowska-Curie Actions (MSCA) with Normal payments, and one in INFRA. 

1363 Eligible proposals are those with the status ‘Main’, ‘No Money’, ‘Rejected’, ‘Reserve’ and 
‘Withdrawn’; ineligible proposals are those with the status ‘Inadmissible’ or ‘Ineligible’. 
1364 High-quality proposals are defined as proposals evaluated as ’above threshold’ in the final 
stage of evaluation. 
1365 The data for 2023 cover the period up to the date when data were received by the study 
team, and do not reflect the total figures for 2023. 

1087 

 
 
 
ERC 

MSCA 

INFRA 

Total for 
Pillar 1 

1 128 (47.2%) 

1 187 (49.7%) 

74 (3.1%) 

2 389 (100%) 

1 185 (48.0%) 

1 201 (48.6%) 

83 (3.4%) 

2 469 (100%) 

11.5 

15.1 

19.1 

13.5 

99.9% 

99.6% 

96.5% 

99.6% 

- 

- 

55 

100 

66.3 

0.3% 

1.2% 

0.2% 

11.5 

15.2 

20.1 

13.6 

91.9% 

87.6% 

69.9% 

89.1% 

Total number of 
projects 
analysed 

Total number of 
payments 
made1366 

Average TTP: 
pre-financing 

Share of pre-
financing 

Average TTP: 
normal 
payments1367 

Share of normal 
payments 

Average TTP: 
total 

Share of 
payments within 
the target1368 

Table 13. Overview of the payments data and TTP, by programme1369. Source: compiled by the study team using eCORDA 
data.As expected, INFRA has the largest average project size, followed by the ERC and MSCA. 

Horizon Europe 
thematic priority 

2021 

2022 

2023 

Total for Pillar 1 

ERC 

MSCA 

1.9 

0.5 

1.6 

0.6 

- 

- 

1.8 

0.5 

1366 Negative payments and regularisation are excluded.  
1367 As of the data cut-off point in May 2023, only four normal payments had been made in 
MSCA, and one in INFRA. 
1368 The target for pre-financing target is 30 days, for other payments 90 days.  
1369 Refers to only projects with the status ‘Closed’ or ‘Signed’. 

1088 

 
 
 
 
Horizon Europe 
thematic priority 

2021 

2022 

2023 

Total for Pillar 1 

INFRA 

Total per year 

7.2 

1.2 

6.4 

1.2 

0.9 

0.9 

6.5 

1.2 

Table 14. Average project size, calculated on the basis of EU contribution, expressed in million EUR1370. Source: compiled 
by the study team using eCORDA data. 

Horizon 
Europe 
type of 
action 

RIA 
(Research 
and 
Innovation 
Actions) 

CSA 
(Coordination 
and Support 
Actions) 

ERC 
(European 
Research 
Council) 
Grants1371 

TMA 
(Transnational 
Mobility 
Activities) 
MSCA1372 

Total 

ERC 

MSCA 

INFRA 

Total for 

Pillar 1 

- 

- 

49 

49 

2 

64 

31 

97 

2 372 

- 

2 375 

- 

- 

2 569 

2 633 

- 

81 

2 375 

2 569 

5 089 

Table 15. Number of projects, by programme and type of action1373. Source: compiled by the study team using eCORDA 
data. 

The largest share of ERC proposals was funded via Action Grants (15 439), followed by Lump 
Sums  (801).  Most  MSCA  proposals  were  funded  via  Unit  Grants  (17  532),  although  117 
proposals had Lump Sum funding, demonstrating the growing use of this novel funding type.  

Horizon Europe 
type of funding 

Action 
Grants 

Horizon 
Prizes 

Lump Sums 

Specific 
Grant 

Unit Grant 

ERC 

MSCA 

15 439 

8 

66 

- 

801 

117 

- 

- 

- 

17 532 

1370 Calculations exclude projects with the status ‘Rejected’, ‘Suspended’ or ‘Terminated’. Also 
applies to Table 15. 
1371 Includes ERC Grants, Proof of Concept Grants and Synergy Grants. 
1372 Includes MSCA Postdoctoral Fellowships; Postdoctoral Fellowships – European Fellowships/ 
Global Fellowships, Cofund Doctoral/Postdoctoral programmes, Doctoral Networks; Staff 
Exchanges.  
1373 One Framework Partnership Agreement (FPA) was excluded from the analysis as it did not 
receive any EU contribution. 

1089 

 
 
 
Horizon Europe 
type of funding 

Action 
Grants 

Horizon 
Prizes 

Lump Sums 

Specific 
Grant 

Unit Grant 

INFRA 

144 

Total for Pillar 1 

15 591 

- 

66 

1 

919 

2 

2 

- 

17 532 

Table 16. Number of proposals, by programme and type of funding1374 

Source: compiled by the study team using eCORDA data. 

1.1.3.  Detailed analysis: WIDERA 

The  section  that  follows  takes  a  closer  look  at  the  efficiency  indicators  for  WIDERA, 
specifically in  the  programme  parts  ‘Widening  participation  and  spreading  excellence’  and 
‘Reforming and enhancing the European R&I system’. 

The average TTG across both parts was 223.8 days, which is within the limit of target eight 
months (244 days). It is also significantly lower than the TTG for Horizon Europe as a whole 
(273.0 days). For the most part, TTS is also not far from the target of three months, averaging 
99.1 days. Looking at TTI, WIDERA is relatively efficient, averaging 117.9 days – well below 
the maximum of 152 days.  

Horizon Europe 
thematic priority 

Total number of 
applications 
analysed1375 

Average TTI 

Average TTS 

Average TTG 

Widening 

participation and 

spreading 

excellence 

Reforming and 

enhancing the 

European R&I 

system 

933 (84.3%) 

120.2 

97.0 

226.3 

174 (15.7%) 

105.5 

108.9 

212.3 

Total for WIDERA 

1 107 (100%) 

117.9 

99.1 

223.8 

Table 17. Overview of project application and selection process indicators, by programme. Source: compiled by the study 
team using eCORDA data. 

1374 Calculations exclude proposals with the status ‘Inadmissible’ or ‘Ineligible’. 
1375 The total number of applications (n) is based on the availability of data for TTI, i.e. if no data 
were available for the date of the proposal information letter, the proposal is not included in this 
table. 

1090 

 
 
 
In  2021  TTI,  TTS  and  TTG  were  slightly  longer  than  the  maximum  targets,  which  may  be 
explained by bureaucratic delays due to the changes made to the Framework Programme. 
This situation improved in 2022. 

Call closure 
year 

Total number of 
applications 
analysed 

Average TTI 

Average TTS 

Average TTG 

2021 

2022 

351 (31.7%) 

130.4 

97.6 

754 (68.1%) 

112.3 

100.1 

20231376 

2 (0.2%) 

49 

Total for Pillar 1 

1 107 (100%) 

117.9 

- 

99.1 

245.9 

212.2 

- 

223.8 

Table 18. Overview of project application and selection process indicators, by year. Source: compiled by the study team 
using eCORDA data. 

WIDERA has the highest success rate of the key thematic priorities analysed in the Excellent 
Science pillar, averaging 27.8% over the initial years of Horizon Europe. Similarly, it has a 
relatively high success rate of high-quality proposals averaging 40.4%. In terms of ineligible 
proposals, ‘Reforming and enhancing the European R&I system’ received the highest share, 
totalling 12.6%, most of which were submitted in 2022. 

Horizon Europe 
thematic priority 

Total number of 
applications 
analysed 

Share of 
ineligible 
proposals1377 

Average 
success rate 
(all proposals) 

Average 
success rate 
(only high-
quality 
proposals) 

Widening 
participation and 
spreading 
excellence 

Reforming and 
Enhancing the 
European R&I 
system 

Total for 
WIDERA 

936 (84.3%) 

1% 

26.5% 

39.1% 

174 (15.7%) 

12.6% 

35.5% 

48.2% 

1 110 (100%) 

2.8% 

27.8% 

40.4% 

Table 19. Overview of the level of competition, by programme. Source: compiled by the study team using eCORDA data. 

1376 The data for 2023 cover the period up to the date when data were received by the study 
team, and do not reflect the total figures for 2023. 
1377 Eligible proposals are those with the status ‘Main’, ‘No Money’, ‘Rejected’, ‘Reserve’ or 
‘Withdrawn’; ineligible proposals are those with the status ‘Inadmissible’ or ‘Ineligible’. 

1091 

 
 
 
Call closure 
year 

Total number of 
applications 
analysed 

Share of 
ineligible 
proposals 

Average 
success rate 
(all proposals) 

Average 
success rate 
(only high-
quality 
proposals) 

2021 

2022 

2023 

351 (31.6%) 

0.9% 

29.6% 

43.5% 

757 (68.2%) 

2 (0.2%) 

3.7 

- 

26.8% 

38.3% 

100% 

100% 

Total for Pillar 1 

1 110 (100%) 

2.8% 

27.8% 

40.4% 

Table 20. Overview of the level of competition, by call closure year. Source: compiled by the study team using eCORDA 
data. 

The average TTP was 25.8 days, which falls within the maximum targets of 30 days for pre-
financing and 90 days for normal payments. Overall, only 72.0% of payments were made on 
time.  The  majority  of  payments  were  pre-financing,  with  no  normal  payments  being  made 
under ‘Widening participation and spreading excellence’. 

Widening participation 
and spreading 
excellence 

Reforming and 
enhancing the 
European R&I 
system 

Total for WIDERA 

Total number of 
projects analysed1378 

204 (80.3%) 

50 

(19.7%) 

254 

(100%) 

Total number of 
payments made1379 

Average TTP: pre-

financing 

213 (78.6%) 

58 (21.4%) 

271 (100%) 

25.2 

26 

25.3 

Share of pre-financing 

100% 

96.6% 

99.3% 

Average TTP: normal 

payments 

- 

74.5 

74.5 

1378 The total number of projects (n) is based on the availability of data for TTP, i.e. if there are 
no data available for the date of proposal information letter, the proposal is not included in the 
table. 
1379 Negative payments and regularisation are excluded. 

1092 

 
 
 
 
Widening participation 
and spreading 
excellence 

Reforming and 
enhancing the 
European R&I 
system 

Total for WIDERA 

Share of normal 

payments 

Average TTP: total 

Share of payments 
within the target1380 

0% 

25.3 

3.5% 

27.7 

0.7% 

25.8 

75.6% 

58.6% 

72.0% 

Table 21. Overview of the payments data and TTP, by programme. Source: compiled by the study team using eCORDA 
data. 

To provide a reference point and a general overview, the study team calculated the average 
project size per year and per programme part. On average, a project in WIDERA cost EUR 
2.7 million. 

Horizon Europe 
thematic priority 

Widening 

2021 

2022 

2023 

Total WIDERA 

participation and 

2.7 

2.8 

- 

2.8 

spreading excellence 

Reforming and 

enhancing the 

European R&I 

system 

2.3 

2.3 

1.7 

2.3 

Total per year 

2.6 

2.8 

1.7 

2.7 

Table 22. Average project size, calculated on the basis of EU contribution, expressed in million EUR1381. Source: compiled 
by the study team using eCORDA data. 

Most projects used Coordination and Support Actions (CSAs) to fund 262 projects in total. 
Following this, 49 projects were funded via Transnational Mobility Activities (TMA), and nine 
via Research and Innovation Actions. 

1380 The target for pre-financing target is 30 days; for other payments, the target is 90 days. 
1381 Calculations exclude projects with the status ‘Rejected’, ‘Suspended’ or ‘Terminated’.  

1093 

 
 
 
 
Horizon Europe 
type of action 

RIA (Research 
and Innovation 
Actions) 

CSA 
(Coordination and 
Support Actions) 

TMA 
(Transnational 
Mobility Activities) 

Total 

Widening 

participation and 

spreading 

excellence 

Reforming and 

enhancing the 

European R&I 

system 

Total WIDERA 

- 

9 

9 

205 

49 

255 

47 

- 

56 

262 

49 

3111382 

Table 23. Number of projects, by programme and type of action. Source: compiled by the study team using eCORDA data. 

Most proposals in WIDERA were funded via Action Grants (1 002), followed by Unit Grants 
(74). Only one proposal was funded via Lump Sum funding, one via Framework Partnership, 
and one via Specific Grant. 

Horizon Europe 
type of funding 

Action 
Grants 

Framework 
Partnership 

Lump 
Sum 

Specific 
Grant 

Unit Grant 

Widening 

participation and 

spreading 

excellence 

Reforming and 

enhancing the 

European R&I 

system 

851 

151 

Total for 

WIDERA 

1 002 

1 

- 

1 

- 

1 

1 

1 

- 

1 

74 

- 

74 

Table 24. Number of proposals by a programme and type of funding1383. Source: compiled by the study team using eCORDA 
data. 

1382 One Framework Partnership Agreement (FPA) was excluded from the analysis, as it did not 
receive any EU contribution. 
1383 Calculations exclude proposals with the status ‘Inadmissible’ or ‘Ineligible’. 

1094 

 
 
 
 
1.2.  Network analysis 

1.2.1.  Methodology 

In this report, the study team conducts a network analysis of participating organisations and 
individuals.  The  basis  for  this  analysis  is  administrative  data  on  Horizon  Europe,  with  the 
analysis of individual researchers being carried out using data on participating researchers’ 
projects1384 and analysis of entities being carried out using data on participating entities1385. 
In addition, Tracking of Research Results (TRR) data for FP7 and Horizon 2020 were used 
to trace involvement over time. 

The analysis encompasses four main sections:  

1. 

2. 

3. 

4. 

Analysis across regions, which compares regional networks based on individual 
researchers’ country of origin, the locations of participating organisations, and 
regional  concentrations  of  the  most-networked  organisations.  This  section 
produced the top 1% most-networked (for a detailed definition of this, see the 
relevant section), which was used in the efficiency analysis under questions EFF 
5 and 6, as well as EFC 5. 

Unique collaborations within WIDERA. The results fed into evaluation question 
EFF5. 

Connectedness of Horizon Europe programme parts. The results of this analysis 
fed into CH1, as well as EQ 38 and 39, and CS9. 

Continuity of researchers across the FP, which includes analyses at the levels 
of pillar and programme part, as well as one analysis at call level (for the ERC). 
These results fed into CS9. 

Different  identifying  factors  are  used  for  each  level  of  analysis.  For  the  organisation-level 
analyses,  the  organisation’s  PIC  was  used  to  identify  organisations,  and  official  Horizon 
Europe project numbers were used to denote projects. Thus, collaboration can be defined as 
two organisations which worked on the same project. 

For analyses at the individual researcher level, researchers’ full names were used to identify 
them, and project numbers were used to identify projects. It was noted in the inception report 
that ORCID numbers would be used to identify and connect researchers; however, ORCID 
numbers were not available in the data for 6 808 individuals (11.2%). Therefore, full names 
were used instead. Full names do present their own complications, as there is no perfect way 
to ensure that two individuals with identical full names will not be treated as a single individual. 
It  may  be  tempting  to  tie  individuals  to  research  institutions  in  order  to  guarantee  their 
uniqueness; however, this risks treating the same individual as multiple people in the case of 
researchers who have changed research institutions. To mitigate the risk of overestimating 
the  number  of  connections  due  to  identical  full  names,  the  research  team  repeated  the 
analysis  using  a  subset  of  the  data  for which  ORCID numbers  were  available.  This  found 
very similar results, which speaks to the robustness of the method. Because the full name 
data are more complete, these results are what is presented in this report. 

1384 Last updated 14 April 2023. 
1385 Last updated 20 June 2023. 

1095 

 
 
 
In the continuity analysis, two different datasets featuring individual researchers were used: 
publication  data  for  Horizon  2020,  and  participation  data  for  Horizon  Europe.  The 
methodology and limitations of this method is further laid out in the relevant section.  

1.2.2.  Concepts related to the network analysis 

To understand the concept of connectedness in our data,  three aspects are considered: 
‘degree’, ‘closeness’ and ‘betweenness centrality’. Centrality measures the level of the 
‘central-ness’ of each node, which in the present case are countries or programme parts. The 
meaning of ‘central’ varies by the type of centrality measurement used, and a node that is 
highly central by one measurement may be low according to another. It is therefore important 
to consider all three measurements. 

•  Degree centrality is a simple measure of how many connections or ‘edges’ each 
node has. For example, in the geographic analysis, the degree of each country 
indicates how many different countries the node was connected to through Horizon 
Europe research projects. 

•  Closeness centrality, as the name implies, looks at how close each node is to every 
other node in the network. It is the average shortest path of the node to every other 
node in the network, normalised so that the score is between 0 and 1. A score closer 
to 1 indicates that the paths from that node to every other node are, on average, 
short. A number close to 0 indicates longer average paths.  

•  Betweenness centrality calculates the ratio of the shortest paths that go through the 
node,  divided  by  the  total  shortest  paths  between  two  nodes.  In  simpler  terms, 
betweenness centrality measures how important a node is to the information flow of 
the network. The number reported is an absolute, unweighted value, and will increase 
in  size  as  the  size  of  the  network  increases.  It  is  therefore  not  possible  to  compare 
betweenness scores between different network analyses; however, the rankings can 
be compared. 

In addition, the density of each graph is given. Density is represented by a number between 
0 and 1, which indicates the connectedness of the network as a whole. For example, if every 
country is connected to every other country through at least one project, the density would 
be  1.  If  no  country  were  connected  to  any  other  (for  example,  if  researchers  worked 
exclusively with researchers of their own nationality), the density would be 0. 

1.2.3.  Analysis across regions 

Regional disparities in R&I were noted as a challenge in the Horizon Europe Strategic Plan 
2025-2027 analysis. The divide between Member States is sharp, and has been increasing 
in recent years, most notably after the pandemic. Regional disparities can lead not only to 
economic disparities, as each euro invested through EU funding is predicted to generate up 
to EUR 11 in GDP over the next 25 years, but they also lead to consequences regarding the 
diffusion of breakthrough technologies, which is currently sub-optimal due to under-utilisation 
of interregional linkages and regional innovation ecosystems1386. 

To analyse the extent to which regional disparities exist under the Horizon Europe FP, three 
network  analyses  were  conducted  to  examine  different  facets  of  the  question.  The  first 
analysis  uses  individual  researchers’  country  of  origin  to  find  out  which  nationalities 
collaborate most under Horizon Europe. Then, as researchers are known to be highly mobile, 

1386 European Commission, Directorate-General for Research and Innovation, Horizon Europe 
strategic plan 2025-2027 analysis, Luxembourg: Publications Office of the European Union, 
2023, available at: https://data.europa.eu/doi/10.2777/637816. 

1096 

 
 
 
and country of residence was not available in the data, participating organisations were used 
as a proxy for researchers’ country of residence, and a second analysis was performed using 
that data. Lastly, the centrality statistics provided by the second analysis were used to create 
a subset of the top 1% most-networked organisations under Horizon Europe. The regional 
distribution of those highly-networked organisations was then examined. 

The results of this analysis have fed into the main report under EFF 6 and 7 as well as EFC5 
with regard to the integration of third countries and associated countries. 

1.2.3.1. 

Analysis using individual researchers’ country of origin 

This analysis uses Horizon Europe administrative data (from the Corda database), May 2023 
release.  The  full  dataset  comprises  63  083  rows,  with  each  row  representing  a  unique 
researcher/project pair. Each project (represented by a nine-digit number) may have multiple 
researchers associated with it, and each researcher may be involved in multiple projects. The 
nationality data provided for each researcher was used to connect countries through projects, 
thus revealing the extent to which researchers of certain nationalities are more connected 
than researchers of other nationalities. 

To prepare the data, 2 653 rows with missing nationality information were removed. Of those, 
2 580 came from the MSCA programme. In total, there were 7 040 MSCA project/researcher 
pairs, so the lack of nationality information led to a 37% reduction in the available MSCA data 
for the geographic part of the analysis. 

The density of this graph is 0.368. A high density indicates a high level of interconnectedness, 
and we see that globally, over one-third of nationalities are connected to one another through 
researchers participating in Horizon Europe-funded projects. As this is the first year in which 
this particular analysis has been carried out, as yet there is no baseline for comparison. In 
subsequent years, it may be possible to compare how this density changes over time. 

Figure  1  presents  a  visualisation  of  the  top  collaborations  between  Member  States  and 
associated  countries  under  Horizon  Europe.  A  complete  visualisation  of  all  countries  and 
edges is extremely dense and difficult to interpret. Therefore, a sparser image showing only 
the top collaborations is presented1387. The size of the label corresponds to the country’s 
betweenness centrality, and the thickness of the lines corresponds to the number of times 
that researchers from the two connected countries have participated in a research project 
together. All edges visualised represent at least 10 938 collaborations. Countries are colour-
coded into non-widening (blue), widening (green), and associated (purple). 

Immediately, it can be seen that researchers of Italian, Spanish, German, French and Greek 
origin have thus far collaborated the most under Horizon Europe, with the highest number of 
collaborations between nationalities occurring between Spanish and Italian researchers. 

1387 The top collaborations for this visual are the edge weights that lie two log standard 
deviations above the log mean edge weight for the dataset. 

1097 

 
 
 
 
Figure 1. Visual representation of top collaborations between nationalities through research projects. Source: compiled by 
the study team using eCORDA data. 

The  table  below  shows  the  centrality  measurements  for  researchers  across  all  EU-27 
countries. The results of this analysis echo the results of the co-publication network analysis 
carried out in the ‘Evaluation study on Excellent Science under Horizon 2020’, which found 
the top 5 highest-ranked countries by closeness centrality in 2021 to be Germany, Belgium, 
Italy, Spain and Greece. 

The present analysis, which has been carried out using a different dataset and is therefore 
not directly comparable, nevertheless  supports the above results. It finds that researchers 
from Germany, France, Italy and Spain continue to be highly central to the European research 
ecosystem under the new FP by degree, closeness and betweenness centrality. Overall, non-
widening  countries  scored  the  highest,  probably  due  to  their  well-developed  research 
infrastructure.  Among  the  widening  countries,  researchers  from  Greece,  Portugal,  Poland 
and Czechia were found to be more central. Researchers from Luxembourg had the lowest 
centrality rankings. 

1098 

 
 
 
 
In  contrast  to  the  results  of  the  organisation-based  analysis  in  the  next  section,  individual 
Romanian  researchers  are  not  so  poorly  connected  and  collaborate  more  often  than 
Romanian  organisations.  This  is  supported  by  a  finding  from  a  co-publication  network 
analysis carried out under the ‘Evaluation study on Excellent Science under Horizon 2020’: 
between 2014 and 2021, there was a large increase in the closeness centrality of Romanian 
researchers.  In  the  present  study,  this  is  interpreted  to  mean  that  individual  Romanian 
researchers  are better  connected  than  Romanian  research  organisations,  probably  due  to 
researcher mobility. 

Country 

Region 

Degree 

Closeness centrality 

Betweenness centrality 

Germany 

Non-widening 

142 

Italy 

Non-widening 

142 

Spain 

Non-widening 

139 

France 

Non-widening 

137 

Netherlands 

Non-widening 

135 

Greece 

Widening 

130 

Belgium 

Non-widening 

129 

Portugal 

Widening 

119 

Austria 

Non-widening 

115 

Finland 

Non-widening 

108 

Poland 

Widening 

108 

Czechia 

Widening 

107 

Denmark 

Non-widening 

106 

Sweden 

Non-widening 

106 

Ireland 

Non-widening 

104 

Romania 

Widening 

Slovenia 

Widening 

99 

96 

0.938 

0.938 

0.921 

0.910 

0.899 

0.874 

0.869 

0.822 

0.804 

0.776 

0.776 

0.772 

0.768 

0.768 

0.760 

0.741 

0.731 

1099 

519.1 

571.9 

542.2 

446.9 

361.1 

363.6 

285.9 

222.9 

171.1 

123.9 

106.4 

135.1 

126.7 

126.3 

94.5 

80.7 

49.6 

 
 
Country 

Region 

Degree 

Closeness centrality 

Betweenness centrality 

Bulgaria 

Widening 

Hungary 

Widening 

Cyprus 

Widening 

Croatia 

Widening 

Lithuania 

Widening 

Slovakia 

Widening 

Estonia 

Widening 

Latvia 

Widening 

Malta 

Widening 

95 

95 

88 

88 

86 

81 

78 

72 

68 

Luxembourg 

Non-widening 

60 

0.727 

0.727 

0.704 

0.704 

0.697 

0.682 

0.673 

0.655 

0.644 

0.623 

76.9 

76.6 

48.2 

37.5 

48.3 

37.2 

24.6 

13.9 

15.6 

11.5 

Table 25. Centrality measurements for the EU-27, all Horizon Europe. Source: compiled by the study team using eCORDA 
data. 

1.2.3.2. 

Pillar 1 and WIDERA geographic network analysis 

The  general  methodology  used  for the  geographic  analysis  of Pillar 1  is  the same as  that 
used for the full FP. The Pillar 1 dataset comprises 14 135 rows, 2 584 of which were removed 
due to lack of nationality data. Of these, 2 580 came from the MSCA programme, out of which 
2 525 were from the HORIZON-MSCA-2021-DN-01 call. 

Figure 2 visualises a subset of the data, following the same logic as the full FP geographic 
visual.  In  order  to  present  a  more  readable  image,  only edge  weights  of  two  log standard 
deviations  above  the  mean  are  depicted.  In  this  case,  only  edge  weights  above  344  are 
shown (7% of all edges). It can be seen that – as with the full FP – Germany, Italy, Spain, 
Greece  and  France  were  very  central.  The  widespread  collaboration  pairings  between 
countries happened between Spain and Italy (11 960 projects had collaborators from both 
countries), followed by Italy and Germany (8 624), and Italy and France (7 770). The density 
of the full network is 0.373, slightly higher than the full FP. The densities of Pillar 2 and Pillar 
3 (analyses not presented here) are 0.369 and 0.263, respectively, making Pillar 1 the most 
globally connected Pillar in Horizon Europe. 

1100 

 
 
Figure 2. Visual representation of countries connected via Pillar 1 and WIDERA research projects. Source: compiled by the 
study team using eCORDA data. 

The table below presents the centrality measurements for all EU-27 countries. The results 
are comparable to those of the full FP, with countries in almost the same order of centrality. 
Once more, widening countries are less well connected than non-widening countries, with 
the exception of Luxembourg. In this subset, we find that Germany and not Italy is the most 
central country, by all three measurements. 

Country 

Region 

Degree 

Closeness centrality 

Betweenness Centrality 

Germany 

Non-widening 

93 

France 

Non-widening 

90 

Italy 

Non-widening 

90 

Spain 

Non-widening 

90 

0.927 

0.902 

0.902 

0.902 

252.0 

211.0 

198.3 

278.8 

1101 

 
 
 
Country 

Region 

Degree 

Closeness centrality 

Betweenness Centrality 

Greece 

Widening 

Portugal 

Widening 

83 

83 

Belgium 

Non-widening 

77 

Netherlands  Non-widening 

77 

Poland 

Widening 

73 

Austria 

Non-widening 

70 

Finland 

Non-widening 

70 

Czechia 

Widening 

69 

Sweden 

Non-widening 

69 

Hungary 

Widening 

62 

Denmark 

Non-widening 

61 

Ireland 

Non-widening 

58 

Croatia 

Widening 

Romania 

Widening 

Slovenia 

Widening 

Slovakia 

Widening 

Bulgaria 

Widening 

Malta 

Widening 

Cyprus 

Widening 

Lithuania 

Widening 

55 

55 

53 

50 

49 

46 

45 

43 

0.849 

0.849 

0.808 

0.808 

0.783 

0.765 

0.765 

0.759 

0.759 

0.721 

0.716 

0.701 

0.687 

0.687 

0.678 

0.664 

0.660 

0.647 

0.643 

0.635 

1102 

149.1 

181.9 

118.0 

109.3 

87.0 

61.4 

83.3 

134.2 

77.6 

58.8 

49.8 

38.0 

25.7 

26.9 

38.7 

18.6 

17.2 

14.5 

16.1 

17.2 

 
 
Country 

Region 

Degree 

Closeness centrality 

Betweenness Centrality 

Estonia 

Widening 

Latvia 

Widening 

39 

38 

Luxembourg  Non-widening 

22 

0.620 

0.616 

0.558 

10.8 

8.1 

0.8 

Table 26. Centrality measurements for the EU-27 under Pillar 1 and WIDERA. Source: compiled by the study team using 
eCORDA data. 

1.2.3.3. 

Analysis using organisations as a proxy for country of residence  

This section of the annex examines collaborations between different organisations, such as 
higher  and  secondary  education  institutions,  private  for-profit  institutes  and  others,  which 
collaborated  with  one  another  through  Horizon  Europe  projects.  Because  individual 
researchers are highly mobile, while organisations are fixed in place, this analysis illuminates 
another side of the regional trends within the ERA. 

Each instance of collaboration between two organisations is counted as a single ‘degree’ for 
each of those organisations, as described by degree centrality in the Methodology section 
above. The more times that an organisation collaborates, the higher its degree will be. To 
summarise regional disparities within the ERA, all of the degrees of all organisations within 
each Member State and visualised this in Figure 3. The map presents an aggregation of all 
instances of collaboration by all organisations within each Member State  – for example, if 
University W in Germany collaborated once with Institute X in Germany, and College Y in 
Germany collaborated twice with University Z in the Netherlands, than Germany would have 
a total degree of 4 and the Netherlands would have a total degree of 2. The map indicates 
that regions with higher aggregate degrees are host to organisations that collaborate more 
(in  absolute  numbers).  Spain  shows  the  highest  number  of  collaborations,  followed  by 
Germany,  Italy  and  France,  meaning  that  organisations  within  those  countries  are 
collaborating the most. 

To add nuance to the above picture, Table 27 presents the density of degree weighted by 
each  country’s  population.  When  population  density  is  taken  into  account,  Cyprus, 
Luxembourg,  Slovenia  and  Estonia  emerge  at  the  top  of  Member  States  and  associated 
countries, meaning that their organisations collaborated more with other institutions relative 
to the size of the countries’ populations. Cyprus and Estonia saw the greatest increases in 
innovation  index  score  between  2016  and  2023  according  to  the  European  Innovation 
Scoreboard (EIS), so it is unsurprising to see them close to the top of the list. In contrast, 
some  regions  perform  poorly  by  both  measures.  Romanian  and  Hungarian  organisations 
collaborate less  than  others in  absolute  as  well  as  relative  terms.  Romania  was  given  the 
lowest score of all EU Member States by the EIS. 

1103 

 
 
Figure 3. Aggregate instances of collaboration (degree), by Member State. Source: compiled by the research team using 
eCORDA data. 

Name 

Total degree  Weighted degree1388 

Cyprus 

7 459 

Luxembourg 

4 839 

Slovenia 

12 291 

Estonia 

Iceland 

6 372 

1 583 

Finland 

22 620 

0.82 

0.74 

0.58 

0.47 

0.41 

0.41 

1388 Retrieved from Eurostat (DEMO_GIND), data as of 1 January 2022. 

1104 

 
 
 
 
Name 

Total degree  Weighted degree1388 

0.39 

0.37 

0.36 

0.35 

0.32 

0.31 

0.29 

0.27 

0.25 

0.25 

0.24 

0.23 

0.19 

0.17 

0.16 

0.14 

0.14 

0.11 

0.11 

0.11 

Belgium 

46 076 

Greece 

38 089 

Norway 

19 436 

Malta 

1 881 

Austria 

28 874 

Denmark 

18 065 

Netherlands 

51 702 

Ireland 

13 939 

Switzerland 

22 094 

Portugal 

25 923 

Latvia 

4 604 

Sweden 

23 778 

Spain 

92 156 

Lithuania 

4 764 

Croatia 

6 256 

Czechia 

15 072 

Italy 

79 999 

Bulgaria 

7 244 

France 

74 114 

Germany 

90 371 

1105 

 
 
Name 

Total degree  Weighted degree1388 

Slovakia 

5 412 

Hungary 

8 223 

Montenegro 

413 

Serbia 

4 188 

Liechtenstein 

24 

Romania 

10 951 

Poland 

16 285 

North Macedonia 

626 

Moldova (Republic of) 

801 

0.10 

0.09 

0.07 

0.06 

0.06 

0.06 

0.04 

0.03 

0.03 

Table 27. Total and weighted degree, EU Member States and associated countries. Source: compiled by the research team 
using eCORDA and Eurostat data. 

1.2.3.4. 

Analysis using the top 1% most-networked organisations 

The  top  1%  subset  was  created  by  identifying  those  participating  organisations  with  the 
highest (top 1%) closeness centrality. In network analysis, if a node has a high closeness 
centrality, this indicates not only that it has many direct connections to other nodes in the 
network, but also that it has short paths to other nodes with which it lacks direct connections. 
Nodes with high closeness centrality are seen as spreading information more quickly through 
the  network.  The  nodes  in  this  analysis  are  organisations,  and  the  diagram  can  be 
interpretated  as  showing 
Individual 
organisations  were  identified  by  their  PIC,  and  collaboration  was  defined  as  two  or  more 
organisations being listed under the same project number. The sample consists of 18 017 
total unique organisations; therefore, the top 1% was a set of 180 organisations. 

they  are  well-established  and 

influential. 

that 

So  far,  34.5%  of  Horizon  Europe  funding  has  gone  to  just  1%  of  participating 
organisations. This implies that the remaining 65.5% of all funding was distributed among 
17  837  organisations.  The  top  1%  most-networked  organisations  are  primarily  located  in 
Western  Europe,  with  Italy,  Germany,  the  Netherlands,  Spain,  the  United  Kingdom  and 
France  hosting  half  of  them.  In  addition,  nine  Member  States  (Bulgaria,  Croatia,  Cyprus, 
Latvia,  Lithuania,  Hungary,  Malta,  Romania  and  Slovakia)  do  not  host  any  of  the  top  1% 
most-networked  institutions.  Highly  influential  and  well-established  organisations  are 
concentrated in Western Europe, and receive disproportionately more funding. As time goes 
on, this may lead to deepening disparities and the slower diffusion of research results. 

1106 

 
 
Pillar 

Total 
participations 
by the top 1% 

Top 1% as a 
share of all 
participations 

Total funding 
for top 1% 
(million EUR) 

Share of 
funding  
given to top 1% 

Horizon Europe 
overall 

14 591 

Pillar 1 

Pillar 2 

Pillar 3 

WIDERA 

6 122 

7 369 

652 

448 

26.3 

38.0 

20.8 

31.6 

23.8 

7 886.29 

3 572.42 

3 718.83 

357.73 

237.31 

34.5 

51.0 

26.5 

36.1 

28.0 

Table 28. Participations and funding to top 1% most-networked organisations. Source: compiled by the study team using 
eCORDA data. 

Participations  by  the  180  most-networked  organisations  are  presented  in  the  three  tables 
below  according  to  programme  part,  region  and  Member  State.  In  each  table,  total 
participations  by  the  top  1%  most  networked  organisations  is  compared  with  total 
participations in that programme part, region or country. A higher share of participations by 
the  top  1%  indicates  that  Horizon  Europe  resources  may  be  disproportionately  funnelled 
towards a small number of well-established institutions. 

Part name1389 

Total participations 
by the top 1% 

Total participations in 
Horizon Europe 
overall 

Top 1% as a share 
of all participations 
(%) 

ERC 

INFRA 

MSCA 

WIDERA Widening 

WIDERA ERA 

Cluster 1 

Cluster 2 

Cluster 3 

1 686 

624 

3 812 

319 

129 

1 063 

412 

272 

3 277 

1 832 

10 986 

1 175 

705 

4 758 

1 760 

1 606 

1389 EIT and EIC Accelerator were excluded from the data. 

1107 

51.4 

34.1 

34.7 

27.1 

18.3 

22.3 

23.4 

16.9 

 
 
 
Part name1389 

Total participations 
by the top 1% 

Total participations in 
Horizon Europe 
overall 

Top 1% as a share 
of all participations 
(%) 

Cluster 4 

Cluster 5 

Cluster 6 

EIC 

EIE 

2 174 

2 018 

1 430 

622 

30 

9 312 

10 295 

7 656 

1 646 

420 

Horizon Europe overall 

14 591 

55 428 

23.3 

19.6 

18.7 

37.8 

7.1 

26.3 

Table 29. Participations of top 1%, by programme part. Source: compiled by the study team using eCORDA data. 

The  table  below  provides  an  overview  of  the  share  of  participations  by  the  top  1%  most 
networked  organisations  by  region  (Widening,  Non-widening,  Associated,  Third).  Non-
widening  countries  experienced  the  highest  share,  due  to  their  stronger  research 
infrastructure. 

Region 

Total participations 
by top 1% 

Total participations in 
Horizon Europe overall 

Top 1% as a share of 
all participations (%) 

Widening 

1 487 

Non-widening 

11 083 

Associated 

1 567 

Third 

454 

9 303 

36 706 

6 155 

3 264 

16.0 

30.2 

25.5 

13.9 

Table 30. Participations of top 1%, by region. Source: compiled by the study team using eCORDA data. 

Over  half  of  participations  from  Denmark  were  by  organisations  in  the  top  1%  of  most-
networked organisations. Sweden and Netherlands also have high percentages (45.2% and 
40.9%, respectively). This is in line with the findings of the European Innovation Scoreboard, 
which ranked Denmark highest out of all EU Member States in 2023 in the overall innovation 
index, with Sweden and the Netherlands being next-highest1390.  

1390 European Commission, Directorate-General for Research and Innovation, Hollanders, H., 
European Innovation Scoreboard 2023, Luxembourg: Publications Office of the European Union, 
2023, available at: https://data.europa.eu/doi/10.2777/119961 

1108 

 
 
 
Member State 

Total participations 
by the top 1% 

Total participations in 
Horizon Europe overall 

Top 1% as a share of 
all participations (%) 

Denmark 

Luxembourg 

Sweden 

756 

119 

680 

Netherlands 

1 493 

Finland 

Ireland 

Greece 

Slovenia 

512 

374 

773 

196 

Italy 

1 658 

Belgium 

928 

France 

1 367 

Austria 

434 

Germany 

1 560 

Spain 

1 202 

Estonia 

Portugal 

Czechia 

Poland 

66 

257 

126 

69 

1 371 

256 

1 506 

3 654 

1 288 

1 056 

2 369 

611 

5 367 

3 011 

5 141 

1 634 

6 270 

6 152 

369 

1 490 

737 

876 

55.1 

46.5 

45.2 

40.9 

39.8 

35.4 

32.6 

32.1 

30.9 

30.8 

26.6 

26.6 

24.9 

19.5 

17.9 

17.2 

17.1 

7.9 

Table 31. Participations by the top 1% most-networked, by Member State. Source: compiled by the study team using 
eCORDA data. 

A further subset of the top 0.1% of organisations ranked by the highest degree is presented 
in Table 32. Again, the numerical value for degree represents the number of shared projects 
under Horizon Europe, given by unique identifying project number. 

1109 

 
 
None of these organisations were newcomers to EU funding programmes. The Fraunhofer 
Society tops the list, with 3 354 shared Horizon Europe projects. This is unsurprising, given 
that it is the largest organisation for applied research in Europe.  

Country 

Participant 

Entity 
type 

Degree 

Germany 

Fraunhofer Gesellschaft zur Forderung der Angewandten 
Forschung EV 

REC 

3 354 

Spain 

Agencia Estatal Consejo Superior de Investigaciones 
Cientificas 

REC 

2 554 

France 

Centre National de la Recherche Scientifique CNRS 

REC 

2 552 

Italy 

Consiglio Nazionale delle Ricerche 

REC 

2 344 

Belgium 

Katholieke Universiteit Leuven 

HES 

2 076 

France 

Commissariat à l’Energie Atomique et aux Energies 
Alternatives 

REC 

2 040 

Netherlands  Technische Universiteit Delft 

HES 

1 921 

Greece 

Ethniko Kentro Erevnas kai Technologikis Anaptyxis 

REC 

1 917 

Finland 

Teknologian Tutkimuskeskus VTT Oy 

REC 

1 774 

Italy 

Alma Mater Studiorum – Universita di Bologna 

HES 

1 690 

Denmark 

Danmarks Tekniske Universitet 

HES 

1 686 

Germany 

Deutsches Zentrum für Luft – und Raumfahrt EV 

REC 

1 560 

Italy 

Politecnico di Milano 

HES 

1 524 

Switzerland 

Eidgenoessische Technische Hochschule Zuerich 

HES 

1 521 

Netherlands 

Nederlandse Organisatie voor Toegepast 
Natuurwetenschappelijk Onderzoek TNO 

REC 

1 519 

Netherlands  Wageningen University 

HES 

1 463 

1110 

 
 
 
Country 

Participant 

Norway 

Sintef AS 

Denmark 

Aarhus Universitet 

Entity 
type 

Degree 

REC 

1 452 

HES 

1 451 

Table 32. Top 0.1% entities with the highest degree. Source: compiled by the study team using eCORDA data. 

There  were  no  newcomers  to  the  most  central  organisations  within  either  the  ERC  or  the 
MSCA. Tables listing their most-networked organisations are presented below, and analysed 
in  further  detail  in  section  7  of  this  annex.  Most  notably,  the  Spanish  National  Research 
Council (CSIC) and not Fraunhofer, is the most central organisation within the ERC. 

Country 
name 

Participant legal name 

Entity 
type 

Degree 

Spain 

Agencia Estatal Consejo Superior de Investigaciones 
Cientificas 

REC 

2 554 

France 

Centre National de la Recherche Scientifique CNRS 

REC 

2 552 

Italy 

Consiglio Nazionale delle Ricerche 

REC 

2 344 

Belgium 

Katholieke Universiteit Leuven 

HES 

2 076 

France 

Commissariat à l’Energie Atomique et aux Energies 
Alternatives 

REC 

2 040 

Netherlands 

Technische Universiteit Delft 

HES 

1 921 

Italy 

Alma Mater Studiorum – Università di Bologna 

HES 

1 690 

Denmark 

Danmarks Tekniske Universitet 

HES 

1 686 

Germany 

Deutsches Zentrum für Luft – und Raumfahrt EV 

REC 

1 560 

Italy 

Politecnico di Milano 

HES 

1 524 

Switzerland 

Eidgenoessische Technische Hochschule Zuerich 

HES 

1 521 

Denmark 

Aarhus Universitet 

HES 

1 451 

Table 33. Most central organisations within the ERC. Source: compiled by the study team using eCORDA data. 

1111 

 
 
Country 
name 

Participant legal name 

Entity 
type 

Degree 

Germany 

Fraunhofer Gesellschaft zur Forderung der Angewandten 
Forschung EV 

REC 

3 354 

Spain 

Agencia Estatal Consejo Superior de Investigaciones 
Cientificas 

REC 

2 554 

France 

Centre National de la Recherche Scientifique CNRS 

REC 

2 552 

Italy 

Consiglio Nazionale delle Ricerche 

REC 

2 344 

Belgium 

Katholieke Universiteit Leuven 

HES 

2 076 

France 

Commissariat à l’Energie Atomique et aux Energies 
Alternatives 

REC 

2 040 

Netherlands 

Technische Universiteit Delft 

HES 

1 921 

Greece 

Ethniko Kentro Erevnas kai Technologikis Anaptyxis 

REC 

1 917 

Finland 

Teknologian Tutkimuskeskus VTT Oy 

REC 

1 774 

Italy 

Alma Mater Studiorum - Università di Bologna 

HES 

1 690 

Denmark 

Danmarks Tekniske Universitet 

HES 

1 686 

Germany 

Deutsches Zentrum für Luft – und Raumfahrt EV 

REC 

1 560 

Table 34. Most central organisations in MSCA. Source: compiled by the study team using eCORDA data. 

1.2.3.5. 

Conclusions from the analysis across regions 

Regional disparities exist within the ERA, whether in terms of the nationalities of  individual 
researchers, the locations of research organisations, or the density of the most-networked 
organisations.  German,  Spanish,  Italian,  French  and  Greek  researchers  and  research 
organisations  collaborate  with  others  the  most.  High  concentrations  of  top  1%  most-
networked  organisations  exist  in  Germany,  Italy,  the  Netherlands,  France  and  Spain.  In 
contrast, researchers from the nine Member States that are not host to any of the top 1% 
most-networked  institutions  (Bulgaria,  Croatia,  Cyprus,  Latvia,  Lithuania,  Hungary,  Malta, 
Romania and Slovakia) show lower levels of collaboration than researchers from countries 
with a high concentration of top 1% most-networked institutions. 

1112 

 
 
Collaboration  between  researchers  across  regions  hastens  the  flow  of  knowledge  and 
improves the quality of scientific research1391. It can therefore be expected that regions with 
a  higher  degree  of  collaboration  also  produce  high-quality  scientific  output.  One  way  to 
measure the quality of scientific output is through highly cited publications. Figure 4 shows a 
map  of  highly  cited  publications  created  for  the  report  ‘Science,  Research  and  Innovation 
Performance  of  the  EU  2022’,  in  which  a  clear  concentration  of  highly  cited  publications 
across Western Europe can be seen. High concentrations of highly cited publications also 
come  from  Cyprus,  Estonia  and  Slovenia,  whose  institutions  collaborated  more  in 
comparison to their national population sizes than any other Member States. 

In conclusion, our findings support the assertion by made in ‘The Horizon Europe Strategic 
Plan 2025-2027 analysis’ that regional disparities exist within the ERA ecosystem. 

Figure 4. Percentage of highly cited publications (top 10%) in 2018, per NUTS 2 level. Source: European Commission, 
Directorate-General for Research and Innovation (2022), Science, research and innovation performance of the EU 2022 – 
Building a sustainable future in uncertain times1392. 

1.2.4.  WIDERA collaborations 

Below, Figure 5 shows a visual depiction of the 1 001 organisations based in EU Member 
States  that  have  participated  in  Horizon  Europe  WIDERA  thus  far.  Organisations  from 
widening  Member  States  tend  to  be  clustered  together  in  groups,  forming  ties  between 
themselves  within  the  WIDERA  programme.  These  are  represented  by  the  purple  dots 

1391 European Commission, Directorate-General for Research and Innovation, Horizon Europe 
strategic plan 2025-2027 analysis, Luxembourg: Publications Office of the European Union, 
2023, available at: https://data.europa.eu/doi/10.2777/637816. 
1392 Luxembourg: Publications Office of the European Union, available at: 
https://data.europa.eu/doi/10.2777/78826. 

1113 

 
 
 
 
forming  clusters  on  the  left  and  bottom  of  the  picture.  Organisations  from  non-widening 
Member States are more central within the programme overall. These are depicted by the 
light blue dots that are more concentrated in the centre of the visual. This indicates that even 
within  the  WIDERA  programme,  organisations  from  non-widening  countries  remain  highly 
connected via shared projects. 

Figure 5. Graphic representation of WIDERA collaborations. Source: compiled by the study team using eCORDA data. 

Table  35  provides  a  numerical  breakdown  of  cross-collaborations  between  organisations 
from  widening  and  non-widening  countries.  Figures  for  Horizon  Europe  overall,  excluding 
WIDERA, are presented as a baseline for comparison. These results are analysed in further 
detail under EFF5.  

Total connections 

Widening/non-widening 
collaboration 

WIDERA 

Horizon Europe (excluding 
WIDERA) 

614 878 

194 498 

13 478 

5 384 

1114 

 
 
 
 
WIDERA 

Horizon Europe (excluding 
WIDERA) 

Within widening collaborations 

Within non-widening collaborations 

Cross-collaborations as a share of 
total (%) 

4 082 

4 012 

39.94% 

49 440 

370 940 

31.63% 

Table 35. Cross-region collaborations within WIDERA. Source: compiled by the study team using eCORDA data. 

Following  the  methodology  outlined  in  the  inception  report,  the  present  study  further 
examined new connections within WIDERA’s Widening Participation calls. A total of 4 404 
collaborations were found within Widening, 3 631 of which were new, meaning the two 
organisations had not previously collaborated anywhere else under Horizon Europe. 
Although the share of new collaborations may at first seem high, similarly high shares were 
also found in Pillar 2’s Mission Cancer and Cybersecurity calls1393. Based on the patterns of 
collaboration in previous framework programmes (FP6, FP7 and Horizon 2020), a majority of 
new collaborations seems to be the rule rather than the exception1394.  

Out of these collaborations, 2 553 were between two organisations based in Member States 
(as opposed to associated or third countries). Figure 6 provides a breakdown of these 2 553 
new collaborations. Organisations from Greece took part in the highest number, with 367, 
followed by organisations from Bulgaria, with 228. 

Table 36 shows clusters of countries in which organisations have formed new collaborations 
during  Widening.  Only  those  pairs  with  more  than  20  new  collaborations  are  shown.  The 
highest number of new collaborations were formed between organisations within the same 
country: 127 new collaborations were formed between organisations from Greece, and 72 
new collaborations were formed between organisations from Croatia. 67 new collaborations 
involved organisations in Greece and Bulgaria. Of the new collaborations within Widening, 
61% had at least one widening country and 24% were between widening countries. 

1393 Details can be found in the Annex of the Evaluation support study on Horizon Europe’s 
contribution to a Resilient Europe. 
1394 European Commission, Directorate-General for Research and Innovation, Ravet, J., Balland, 
P., Dynamic network analysis of the EU R&I Framework Programme, Publications Office, 2018, 
https://data.europa.eu/doi/10.2777/651302. 

1115 

 
 
 
 
Figure 6. Histogram of new collaborations in Widening Participation calls. Source: compiled by the study team using 
eCORDA data. 

Member State pair 

Number of new collaborations 

EL / EL 

HR / HR 

EL / BG 

CY / CY 

CY / EL 

CZ / SK 

BG / BG 

BG / EL 

EL / CY 

127 

72 

67 

61 

54 

54 

52 

51 

50 

1116 

 
 
 
Member State pair 

Number of new collaborations 

SK / SK 

EL / MT 

EE / EE 

BG / RO 

CZ / CZ 

PT / PT 

BE / HR 

ES / ES 

HR / CY 

MT / MT 

CY / BG 

HR / SI 

SI / HR 

BG / CY 

CY / HR 

SK / CZ 

EL / RO 

EE / CZ 

MT / EL 

44 

43 

40 

39 

39 

32 

30 

28 

26 

25 

24 

24 

24 

23 

23 

23 

22 

20 

20 

Table 36. New collaborations, by Member State pair. Source: compiled by the study team using eCORDA data. Note: BE, 
Belgium; BG, Bulgaria; CY, Cyprus; CZ, Czechia; EE, Estonia; ES, Spain; EL, Greece; HR, Croatia; MT, Montenegro; PT, 
Portugal; RO, Romania; SI, Slovenia; SK, Slovakia. 

1117 

 
 
1.2.5.  Connectedness of Horizon Europe programme part 

The  connectedness  of  programme  parts  examines  the  degree  to  which  researchers 
participate in different programme parts under the FP. The results of this analysis are used 
in CH1, as well as in EQ 38 and 39 and CS9. 

The connectedness of programme parts was analysed by looking at individual researchers. 
Each row in the CORDA database represents a unique researcher/project pair and includes 
the name of the programme part under which that project falls. Using this methodology, it 
was possible to trace individual researchers across programme parts. 

The  full  FP  dataset  contained  62,859  rows.  The  overall  density  was  0.955,  meaning  that 
almost every part was connected to every other part through at least one researcher. Out of 
the 12 programme parts, all but three achieved the maximum degree of 11. The three parts 
that did not were Cluster 3, the EIE, and the ERC – all of which had degrees of 9, meaning 
that  each  of  these  programme  parts  was connected  by  shared  researchers  to  9  of the  11 
other programme parts. 

Figure 7 presents a visualisation of the connectedness of the programme parts via individual 
researchers.  The  names  of  the  programme  parts  are  colour-coded  by  pillar:  Pillar  1  and 
WIDERA in blue, Pillar 2 in green and Pillar 3 in purple. The thickness of the edges indicates 
the number of researchers who had a project in each of the two connected nodes. 

Figure 7. Connectedness of programme parts. Source: compiled by the study team using eCORDA data. 

The table below presents the top 20 thickest edges from the above graph, together with their 
number  of  connections  associated  with  each  of  them.  Out  of  all  the  Horizon  Europe 
programme  parts,  Cluster  5  and  Cluster  4  share  the  greatest  numbers  of  researchers  in 
common. Clusters 4, 5 and 6 have seen a very large number of participations so far, with 
over 27 000 between the three programme parts – so it is to be expected that these three 
parts will have a high level of overlap. When it comes to thematic similarity, Cluster 5 (Climate, 
Energy and Mobility), and Cluster 4 (Digital, Industry and Space) may be more closely related 

1118 

 
 
 
than  they  first  appear,  as  many  of  Cluster  4’s  calls  addressing  the  twin  green  and  digital 
transitions target circularity and energy efficiency. 

In addition, Pillar 1 researchers participated more often in projects in other pillars than they 
did in an additional project within Pillar 1.  

Pair 

Number of connections 

Cluster 5 / Cluster 4 

1 717 

727 

668 

472 

444 

302 

271 

261 

221 

209 

207 

193 

191 

185 

153 

145 

138 

Cluster 6 / Cluster 4 

Cluster 6 / Cluster 5 

Cluster 4 / Cluster 3 

Cluster 4 / Cluster 1 

MSCA / Cluster 4 

INFRA / Cluster 4 

INFRA / Cluster 6 

Cluster 4 / Cluster 2 

EIC / Cluster 4 

WIDERA / Cluster 4 

WIDERA / Cluster 6 

Cluster 5 / Cluster 1 

WIDERA / Cluster 5 

MSCA / Cluster 5 

Cluster 6 / Cluster 1 

INFRA / Cluster 1 

1119 

 
 
Pair 

Number of connections 

Cluster 3 / Cluster 1 

WIDERA / INFRA 

135 

135 

Table 37.  Researcher connections (top 20 pairs). Source: compiled by the study team using eCORDA data. 

1.2.6.  Continuity of programmes between successive FPs 

This section looks at trends in the way individual researchers from previous FPs participate 
in Horizon Europe. 

The  present  evaluation  of  Horizon  Europe  is  taking  place  early  in  the  FP’s  lifecycle.  As  a 
result, as yet there is an insufficient amount of data available on co-publications1395; however, 
good-quality  data  on  participation  at  an  individual  level  are  available.  Conversely,  while 
abundant publication  data are  available  for  FP7  and  Horizon  2020, individual  participation 
data were not released. Working on the assumption that project  collaborators also publish 
together, we use both types of data to examine the flow of researchers from the previous two 
Framework Programmes into Horizon Europe. The two analyses below show the manner in 
which researchers who published under FP7 and Horizon 2020 projects went on to participate 
in Horizon Europe.  

The main limitation of this methodology is that those who publish under a project funded by 
the EU are not necessarily participants in the project. This becomes obvious when looking at 
the ERC under Horizon 2020, a mono-beneficiary programme with a single leading principal 
investigator (PI) per project, but sometimes with more than 200 researchers publishing under 
the same project. The strength of this method is that it makes it possible to capture some 
aspects of the knowledge flow, as researchers who publish under a project can be assumed 
to  bring  their  knowledge  of  that  topic  to  the  next  project  in  which  they  participate  –  a 
knowledge flow that could not be captured by using data based purely on participations. 

As  mentioned  above  in  the  section  on  methodology,  researchers  were  identified  in  both 
datasets  using  their  full  names,  as  Orcid  IDs  were  only  available  for  around  11%  of 
individuals.  Both  datasets  were  encoded  with  UTF-8  to  ensure  compatibility  between  the 
names. Duplications were removed on the basis of the level of analysis: for analysis at pillar 
level, a researcher was counted once if they published under or participated in a given pillar. 
For  analysis  by  programme  part,  likewise  each  researcher  was  counted  once  for  their 
publication under or participation in a given programme.  

Pillar level – full Framework Programme  

In  the  text  that  follows,  we  examine  the  ways  in  which  researchers  who  published  under 
previous FP pillars went on to participate under Horizon Europe. Duplications based on pillar 
were removed for this table, meaning that the resulting subset included only one instance of 
a  researcher  publishing  under  or  participating  in  any  given  pillar,  leading  to  a  clearer 
interpretation of the results. After removing all duplicates, the merged dataset comprised 1 
843 244 rows. FP7-GA was removed prior to the analysis because it did not fit well into any 
of the main pillars; however, this represented only a small fraction of the data (<1%) and will 

1395 Only around 800 publications have been recorded as of October 2023. 

1120 

 
 
 
not affect the conclusions. The results of this analysis feed into the effectiveness analysis in 
the main report and in CS9. 

Overall,  continuity  –  as  measured  by  the  number  of  researchers  who  published  under  a 
previous FP and went on to participate in Horizon Europe – is low. The largest flow, in terms 
of absolute numbers, occurred between FP7 Cooperation and Horizon Europe Pillar 2. Over 
7 500 researchers who published under FP7 Cooperation went on to participate in Horizon 
Europe Pillar 2. The two pillars are closely related thematically, with strong overlap between 
topics in Health and Security. However, these represent only a small fraction – 7 502 out of 
386 553, or around 2% of the total number of researchers who published under Cooperation. 

Within  Horizon  Europe  Pillar  1  and  WIDERA,  we  see  the  largest  knowledge  inflow  from 
Horizon  2020  Pillar  1  into  Horizon  Europe  Pillar  1,  which  is  not  surprising,  as  it  was  the 
thematic predecessor. This is reflected with further granularity in the analysis of specific Pillar 
1 programme parts, in which a large number of researchers who published under Horizon 
2020 MSCA can be seen to go on to participate in Horizon Europe MSCA. 

In relation to inflows from FP7, it is surprising to see that Horizon Europe Pillar 1 saw more 
inflow  from  Cooperation  than  Ideas,  as  Ideas  was  the  thematic  predecessor.  However, 
Cooperation was the largest pillar of FP7, with two-thirds of the overall budget and 386 553 
individual researchers publishing under the programme part, as opposed to 329 958 under 
Ideas. That difference represents 17% more researchers, while we also see that Pillar 1 saw 
18.7% more inflow from Cooperation than Ideas – a comparable difference, suggesting that 
the size of the programme was the main factor. 

Previous FP pillar under which they published 

Horizon Europe pillar in which 
they went on to participate 

7 502 

5 821 

5 812 

3 567 

3 473 

3 472 

2 568 

2 091 

1 764 

1 485 

FP7 Cooperation 

Horizon 2020 Pillar 3 

Horizon 2020 Pillar 1 

Horizon 2020 Pillar 2 

FP7 People 

FP7 Ideas 

Horizon 2020 Pillar 1 

FP7 Capacities 

FP7 Cooperation 

FP7 Ideas 

1121 

Pillar 2 

Pillar 2 

Pillar 2 

Pillar 2 

Pillar 2 

Pillar 2 

Pillar 1 

Pillar 2 

Pillar 1 

Pillar 1 

 
 
Previous FP pillar under which they published 

Horizon Europe pillar in which 
they went on to participate 

1 294 

FP7 People 

942 

933 

850 

808 

672 

595 

568 

512 

499 

494 

451 

346 

299 

288 

265 

264 

251 

226 

217 

Horizon 2020 Pillar 3 

FP7 Capacities 

Horizon Europe Pillar 1 

Horizon 2020 Pillar 1 

Horizon 2020 Pillar 2 

FP7 Cooperation 

FP7 Ideas 

Horizon 2020 Pillar 1 

FP7 People 

FP7 Cooperation 

Horizon Europe Pillar 2 

FP7 People 

Horizon 2020 Pillar 3 

FP7 Ideas 

Horizon 2020 Pillar 2 

FP7 Capacities 

Horizon Europe Pillar 2 

Horizon 2020 Pillar 3 

FP7 Capacities 

1122 

Pillar 1 

Pillar 1 

Pillar 1 

Pillar 2 

Pillar 3 

Pillar 1 

Pillar 3 

Pillar 3 

WIDERA 

Pillar 3 

WIDERA 

WIDERA 

WIDERA 

WIDERA 

WIDERA 

Pillar 3 

WIDERA 

Pillar 3 

Pillar 3 

Pillar 3 

 
 
Previous FP pillar under which they published 

Horizon Europe pillar in which 
they went on to participate 

213 

161 

94 

76 

28 

25 

9 

3 

Horizon 2020 Pillar 2 

Horizon Europe Pillar 1 

Horizon Europe Pillar 1 

FP7 Nuclear 

WIDERA 

WIDERA 

Pillar 3 

Pillar 2 

Horizon Europe Pillar 3 

WIDERA 

FP7 Nuclear 

FP7 Nuclear 

FP7 Nuclear 

Pillar 1 

Pillar 3 

WIDERA 

Table 38. Researchers who went on to participate in Horizon Europe, by pillar. Source: compiled by the study team using 
eCORDA data. 

The research team then followed the paths of entities that were newcomers to Horizon 2020, 
to find out how they went on to participate in Horizon Europe. Because newcomers were not 
flagged  at  the  individual  level,  entities  data  were  used  for  this  analysis.  Newcomers  are 
flagged  by  the  first  Framework  Programme  in  which  they  participated,  rather  than  by  the 
programme part within that framework – hence, an entity that first came into Horizon 2020 
under SC5 and went on to participate in SC1 will be counted as a newcomer in both cases. 
There were 29 318 unique newcomer organisations under Horizon 2020, 4 667 of which have 
so far participated in Horizon Europe. 

Certain  trends  become  immediately  clear:  entities  that  were  newcomers  to  Horizon  2020 
largely went on to participate in Pillar 2, which is to be expected as it is the largest pillar under 
Horizon  Europe.  The  biggest  overlap  being  between  SC  3  and  Cluster  5  is  also  to  be 
expected, as both programmes focus on clean energy. Those who went on to participate in 
Pillar 1 did so largely under the MSCA. 

Horizon 2020 programme 

Horizon Europe 
programme 

Number of 
connections 

H2020-SC3 

H2020-LEITs-ICT 

H2020-SC4 

556 

507 

503 

Cluster 5 

Cluster 4 

Cluster 5 

1123 

 
 
Horizon 2020 programme 

Horizon Europe 
programme 

Number of 
connections 

H2020-SC2 

H2020-MSCA 

H2020-SC1 

H2020-SC5 

H2020-SC5 

H2020-SC5 

H2020-SC3 

Cluster 6 

MSCA 

Cluster 1 

Cluster 5 

Cluster 4 

Cluster 6 

Cluster 4 

H2020-LEITs-Manufacturing 

Cluster 4 

H2020-LEITs-ICT 

H2020-SC4 

H2020-SC7 

H2020-MSCA 

H2020-SC3 

H2020-SC2 

H2020-LEITs-ICT 

H2020-SC1 

Cluster 5 

Cluster 4 

Cluster 3 

Cluster 4 

Cluster 6 

Cluster 5 

MSCA 

MSCA 

H2020-LEITs-Manufacturing 

Cluster 5 

H2020-LEITs-ICT 

H2020-SC2 

Cluster 6 

Cluster 4 

H2020-Innvoation in SMEs 

Cluster 4 

1124 

501 

431 

333 

295 

273 

259 

236 

229 

223 

185 

180 

177 

155 

144 

140 

140 

139 

132 

132 

131 

 
 
Horizon 2020 programme 

Horizon Europe 
programme 

Number of 
connections 

H2020-MSCA 

Cluster 6 

H2020-LEITs-Materials 

Cluster 4 

HORIZON.2.4 

H2020-SC7 

H2020-LEITs-ICT 

H2020-LEITs-ICT 

H2020-FET 

H2020-MSCA 

H2020-SC3 

H2020-SC6 

Cluster 6 

Cluster 4 

Cluster 1 

Cluster 3 

Cluster 4 

Cluster 1 

MSCA 

Cluster 2 

H2020-LEITs-Materials 

Cluster 5 

130 

128 

124 

121 

118 

114 

111 

111 

106 

106 

103 

Table 39. Participation of Horizon 2020 newcomers in Horizon Europe (only connections greater than 100 reported). Source: 
compiled by the study team using eCORDA data. 

1.2.6.1. 

Continuity of Pillar 1 

This section follows the methodology used in the previous section, mapping researchers who 
published  under  previous  Framework  Programmes  to  their  participation  under  Horizon 
Europe.  For  this  analysis,  a  subset  consisting  of  only  those  who  participated  in  Horizon 
Europe Pillar 1 was used. Duplicates were removed at programme part level; therefore, each 
figure  in  Table  40  represents  researchers  who  published  in  the  programme  part  listed  in 
column 2 and participated in the programme part listed in column 3. The subset comprises 
27 879 rows of unique researcher/programme part pairs. 

Figure  8  shows  the  aggregated  sum  of  connections  by  those  who  published  under 
FP7/Horizon  2020  and  went  on  to  participate  in  Horizon  Europe,  by  programme  part1396. 
MSCA has seen the biggest inflow of such researchers, followed by INFRA. Meanwhile, ERC 
and  WIDERA  –  Strengthening  the  ERA  saw  the  fewest.  Table  40  below  provides  a  more 
detailed look at the programmes from which those researchers came. 

1396 Note: because of the way in which the figures were calculated, this aggregation cannot be 
interpreted as distinct researchers, but as the number of instances that researchers who 
published under one programme also participated in another. 

1125 

 
 
 
Figure 8. Participation in Horizon Europe Pillar 1 of researchers who published under FP7/Horizon 2020 projects. Source: 
compiled by the study team using eCORDA data. 

The table below presents programme part pairs with greater than 100 connections. At the top 
of the list that the largest group comprises 1 043 researchers who published under Horizon 
2020 MSCA and went on to participate in Horizon Europe MSCA. Next come 837 researchers 
who published under FP7 Ideas ERC and went on to participate in Horizon Europe’s MSCA. 
The capacity in which those 837 researchers participated in the MSCA is further explored in 
Table 41. 

Number of 
researchers 

Previous programme part 
under which they published 

Horizon Europe programme 
part in which they went on to 
participate 

1 043 

H2020-MSCA 

837 

803 

700 

644 

529 

FP7-IDEAS-ERC 

H2020-ERC 

FP7-PEOPLE 

H2020-INFRA 

FP7-IDEAS-ERC 

518 

FP7-INFRASTRUCTURES 

433 

H2020-MSCA 

1126 

MSCA 

MSCA 

MSCA 

MSCA 

INFRA 

INFRA 

INFRA 

INFRA 

 
 
 
Number of 
researchers 

Previous programme part 
under which they published 

Horizon Europe programme 
part in which they went on to 
participate 

FP7-PEOPLE 

H2020-ERC 

FP7-HEALTH 

FP7-ICT 

INFRA 

INFRA 

MSCA 

MSCA 

FP7-PEOPLE 

WIDERA - Widening 

H2020-LEITs-ICT 

MSCA 

H2020-MSCA 

WIDERA - Widening 

419 

404 

382 

355 

265 

261 

260 

252 

FP7-ICT 

252 

FP7-INFRASTRUCTURES 

INFRA 

MSCA 

249 

247 

224 

219 

211 

205 

197 

181 

179 

178 

171 

FP7-IDEAS-ERC 

WIDERA - Widening 

H2020-INFRA 

H2020-FET 

MSCA 

MSCA 

H2020-ERC 

WIDERA - Widening 

FP7-HEALTH 

FP7-ENVIRONMENT 

H2020-SC1 

INFRA 

INFRA 

MSCA 

H2020-SEWP 

WIDERA - Widening 

MSCA 

INFRA 

MSCA 

H2020-SEWP 

FP7-KBBE 

FP7-NMP 

1127 

 
 
Number of 
researchers 

Previous programme part 
under which they published 

Horizon Europe programme 
part in which they went on to 
participate 

160 

155 

146 

142 

139 

138 

137 

132 

128 

125 

118 

H2020-LEITs-ICT 

FP7-KBBE 

H2020-FET 

H2020-SC2 

FP7-IDEAS-ERC 

H2020-ERC 

H2020-SEWP 

H2020-SC1 

INFRA 

MSCA 

INFRA 

INFRA 

ERC 

ERC 

INFRA 

INFRA 

FP7-ICT 

WIDERA - Widening 

H2020-SC5 

INFRA 

FP7-HEALTH 

WIDERA - Widening 

118 

FP7-INFRASTRUCTURES 

WIDERA - Widening 

112 

110 

108 

H2020-INFRA 

WIDERA - Widening 

FP7-JTI 

FP7-ENVIRONMENT 

MSCA 

MSCA 

Table 40. Horizon 2020 Pillar 1 researchers who went on to participate in Horizon Europe, and the programme parts in 
which they participated (only pairs with over 100 connections reported. Source: compiled by the study team using eCORDA 
data. 

The study team then went on to examine more closely those 837 researchers who published 
under FP7-IDEAS-ERC and went on to participate in Horizon Europe MSCA, to determine in 
what capacity they are participating. Table 41 breaks down these participations by action. It 
can be seen that the largest share (66%) are now participating in Doctoral Networks. 

1128 

 
 
Action 

Share of total (n=837) 

Doctoral Networks 

Staff Engagement 

Postdoctoral Fellowships 

Citizens 

Researchers at Risk 

66% 

18% 

15% 

1% 

<1% 

Table 41. Breakdown of MSCA participation among those who published under FP 7-IDEAS-ERC. Source: compiled by the 
study team using eCORDA data. 

1.2.6.2. 

Journeys of Horizon 2020 ERC researchers 

To  support  Case  Study  9,  Creating  high-quality  new  knowledge,  PIs  from  the  ERC  under 
Horizon 2020 were followed into Horizon Europe to discover the programme parts in which 
they were most likely to participate. This analysis, unlike the continuity analysis above, uses 
participation  data  from  both  Horizon  2020  and  Horizon  Europe.  The  full  analysis  of  this 
dataset can be found in CS9. 

1.3.  Research  Trend  Analysis  -  Results  and  Interpretation  of  Funding 

and Topical Overlap 

1.3.1. 

Introduction 

This report aims to provide readers with information about the topical focus of the Horizon 
2020 programme parts and the Social Challenges, as defined by the Framework Programme. 
Next  to  this,  the  analysis  looks  to  discover  topical  overlaps  between  Horizon  2020 
Programmes  and  Social  Challenges,  as  well  as  topical  overlaps  at  national  and  supra-
national  level.  However,  this  analysis  can  also,  in  part,  be  characterised  as  investigating 
overlaps  in  funding,  with  its  focus  being  on  those  topics  that  have  received  overlapping 
funding from different programme parts, funders and other (national) funding sources1397.  

The idea of analysing the disciplinary and topical focus of funders and nations has a long 
history in bibliometrics, as well as in the fields of the economics of innovation and the science 
of science, whereby the idea of smart specialisation (in industry) has a partial overlap with 
science, technology and innovation (STI) studies looking at basic science as an enabler of 
industrial specialisation.  

In  addition  to  this,  there is  a  literature  on  –  and  funding  programmes1398  in  –  science  and 
technology (S&T) forecasting. Thus, one of the main goals of this study is the identification 

1397 These sources can be specific funding programmes, but could also be an aggregate of 
national funding.  
1398 https://www.iarpa.gov/research-programs/forest 

1129 

 
 
 
and assessment of emerging S&T capabilities, as well as an analysis of funder priorities and 
other foresight indicators. 

This report combines the two, using comparable definitions of topic, as well as metrics that 
measure both  topic  prominence  (‘Hot  topics’)  and  topic emergence  (‘New  topics’).  As  it  is 
notoriously  difficult  to  predict  the  future,  the  results  of  the  latter  metric  are  open  to 
interpretation. 

1.3.2.  Topics 

In science, a topic represents a body of literature that is defined by its focus on a specific 
problem or issue, a common methodological focus or a specific scientific viewpoint. This body 
of  literature  is  also  defined  by  its  own  language;  using  keywords,  key-phrases,  and  a 
combination of these that are peculiar to the scientific topic in question1399. At a higher level, 
a collection of topics might represent a subject area, which in turn, as a collection of subject 
areas, might represent a research domain or scientific discipline. The top level of this tree is 
represented by the distinction between Alpha and Beta1400; STEM1401 and SSH1402. 

The  classification  of  science  depends  on  the  vocabularies  and  classifications  that  are 
employed to distinguish these topics, subject areas, etc., from one another. Although at the 
micro level, uncontrolled keywords and key-phrases are the norm, once established as part 
of  a  literature,  these  are  added  to  controlled  vocabularies  and  classifications  as  being 
representative of a certain topic, subject area, and so on.  

As  the  present  study  uses  the  Scopus  citation  index  and  the  databases  connected  to  it  – 
SciVal  and  ScienceDirect  –  as  its  data  source,  the  collection  of  vocabularies  and 
classifications used can be described as follows: 

Vocabulary 

Control 

Source 

Free text (keywords mined 
from title, abstract, full text) 

Uncontrolled 

ScienceDirect, Scopus 

Author Keywords 

Semi-controlled 

Scopus 

Index Keywords 

Controlled 

Scopus 

TopicName 

Controlled 

ScienceDirect1403, 
SciVal 

1399 Becher, T., & Trowler, P., Academic Tribes and territories: intellectual enquiry and the 
cultures of disciplines (2nd edition), Buckingham: Open University Press/SRHE, 2001. 
1400 The top-level classification used in this analysis assigns research outputs to five classes: 
Health Sciences, Life Sciences, Physical Sciences, Social Sciences and Humanities, and 
Multidisciplinary Research. 
1401 STEM, Science, technology, engineering, and mathematics. 
1402 SSH, Social sciences and humanities. 
1403 https://www.sciencedirect.com/topics 

1130 

 
 
 
Vocabulary 

Control 

Source 

TopicCluster 

Controlled 

SciVal 

Subject Area 

Controlled 

Scopus1404, SciVal 

Table 42. A short overview of vocabularies and sources. Source: authors’ own interpretation. 

to 

(i.e. 

issues 

(among  others)  semantic 

Through  extensive  and  iterative  testing  and  analysis,  the  study  team  decided  to  use  the 
TopicName controlled vocabulary as its main source of data on topics. The reasoning behind 
this  is  that  to  contextualise  separate  keywords  is  rather  difficult  without  in-depth  domain 
knowledge  due 
the  problems  of 
homonyms/synonyms1405). The study team understands that new topics emerge from natural 
language,  often  as  instances  of  new  combinations  and  interpretations  of  pre-existing 
keywords; however, for the present purposes, the use of controlled vocabularies makes more 
sense in terms of interpretability. The method used here is similar to the theoretical network 
concept of triadic closure1406, whereby a combination of three keywords in close proximity is 
used to define a topic1407. This further enhances the interpretability of the results. A downside 
of  using  a  controlled  vocabulary  to  calculate  certain  metrics  is  that  a  sort  of  aggregation 
problem appears to exist. The main compounders of this effect are an insufficient number of 
publications  (and  as  such,  a  lack  of  topical  keywords),  or  an  insufficient  number  of  years 
being  present  in  the  (sub-)dataset  to  enable  the  emergence  algorithm  to  run  (even with  a 
dummy). This means that for some programme parts, it has not been possible to calculate 
an emergence score.  

Using  a  refinement  of  the  topical  analysis  of  the  Horizon  2020  Programmes  and  Social 
Challenges (SCs), topical overlap – in terms of the number of overlapping programme parts 
and SCs – has been calculated for each of the top 200 topics1408. In doing so, the study team 
has  tried  to  pinpoint  those  topics  for  which  EC-sponsored  outputs  have  been  distributed 
across  several  programme  parts  and  SCs.  As  an  additional  measure, 
the 
ProminencePercentile per topical triad has been used to approximate the current importance 
of the topics. A coloured scale is used to highlight the amount of EC-sponsored outputs per 
programme part, SC and topic. 

In addition, additional data on topical productivity per country has been collected for the EU-
27 countries, a select number of G7 and BRICS countries (China, Japan, South Korea, the 
United Kingdom and the United States), as well as for the world as a whole1409. The analysis 

1404 https://service.elsevier.com/app/answers/detail/a_id/12007/supporthub/scopus/ 
1405 https://en.wikipedia.org/wiki/Homonym 
1406 Wasserman, S., & Faust, K., 2012, available at: 
https://www.cambridge.org/core/books/abs/social-network-
analysis/triads/2E61DF0F9CF8EB680BBD8A3B76F7E26C 
1407 Here, the study team diverges from the method used to calculate the FET Score (in the 
‘Evaluation study on Excellent Science under Horizon 2020’, Phase 1 of the Excellent Science 
Study). The FET Score method used free text keywords denoting technologies, while the aim 
here is to define topics using a taxonomy and keyword triads. 
1408 The top 200 ranking is achieved by looking at research productivity per topic for the whole of 
Horizon 2020, i.e. publication totals per topic. 
1409 SciVal uses a full count method for distributing country affiliations across TopicNames as 
opposed to a fractional count method (Perianes-Rodriguez, A., Waltman, L., & Van Eck, N.J., 

1131 

 
 
 
 
also includes a calculation of the total for the EU-27 and EU-28, as well as adding data which 
represents EC-sponsored outputs1410 in the Top 200 topics, as calculated for the EU-27 and 
selected third countries. As an additional measure, the ProminencePercentile per topical triad 
is used to approximate the ‘momentum’ of the topics1411.  

1.3.3.  Data 

The present analysis relies on three datasets: EC-reported publications data, Scopus/SciVal-
generated data, and EC administrative data. The EC-reported data have been validated using 
Scopus, and are used for the internal comparison, while the Scopus/SciVal-generated data 
is used for the external comparison. Both datasets are comprehensive and of high quality; 
however,  there  is  an  important  difference,  in  that  the  EC-reported  data  has  a  funding 
connection by virtue of being reported, while the Scopus/SciVal-generated publications can 
only be attributed to a funder by virtue of having that information in the acknowledgement. 
This  will  undoubtedly  result  in  discrepancies  between  the  data  reported  and  the  data 
acknowledged.  That  said,  the  present  study  separates  the  comparison  of  those  data  that 
have the same source (i.e. which are generated from affiliation data), and those that stem 
from the reported outputs. Because EC-sponsored data was used in the external comparison, 
as a total, this is highlighted by separating this data column1412 from the others in the analysis. 
The same applies to the total EU-27 and EU28 outputs, which, as a construct, can be found 
in separate columns1413. 

With  regard  to  reported  outputs,  this  study  uses  a  significantly  larger  publication  set, 
comprising 196,712 unique DOIs versus the 138,888 DOIs that used in the ‘Evaluation study 
on Excellent Science under  Horizon 2020’  – an increase of approximately 40%.  This is of 
significant benefit because, especially for NLP1414-based work, the more data points that are 
available, the more robust the analysis and results.  

In terms of data collected on topics, data on 23,793 unique topical triads were gathered, of 
which  23,765  were  matched  to  EC-funded  outputs.  In  addition,  data  were  collected  from 
Elsevier’s SciVal module1415 for each of the countries and institutes/funders analysed. As the 
full set resulting from the above is a vast matrix, the analysis presented here is confined to 
the  top  200  topics,  ranked  on  the  basis  of  the  total  number of  outputs  published  for  each 
topic, and related to the focal variables of the analysis (be they programme parts and Social 
Challenges, or countries and institutions or funders). 

The process of assessing, cleaning, matching and validating was the same as that used in 
the ‘Evaluation study on Excellent Science under Horizon 2020’, thus ensuring a similar, high-
quality dataset that has all the attributes necessary to perform a proper analysis. The dataset 

2016). To overcome the issue of ‘over-count’, affiliation counts of all countries were aggregated. 
Thus, the TopicName count reached is higher than in reality, but the percentage count is closer 
to reality. 
1410 Please note that the output total for the EU-27, EU-28 and EC columns (columns N,L&J) 
have not been added to the ‘Publications per Topic’ column (column B), as these number 
represent a product of the outputs already counted per EU-27/EU-28 country as well as outputs 
sponsored by the EC and also counted as a national output. To signify this difference we have 
‘corralled’ these two columns using bars as well as highlighting them in bold. 
1411 A coloured scale is used to highlight the amount of sponsored outputs per country, country 
group and topic. 
1412 Column J, https://doi.org/10.34894/DNWIXV. 
1413 Column L&N, https://doi.org/10.34894/DNWIXV. 
1414 NLP: natural language processing. 
1415 https://www.SciVal.com/landing. 

1132 

 
 
 
has been enriched with specific data and metrics, as was the case in the Evaluation study on 
Excellent  Science  under  Horizon  2020.  In  addition,  project-level  administrative  data  from 
Horizon 2020 were used in order to collect the funding-related variables needed to calculate 
the  funding  amounts  per  topic.  Other  information,  such  as  specific  Subject  Area  and 
Disciplinary classifications1416, were used to create the appropriate concordances.  

1.3.4.  Metrics 

As the above text notes, two relatively new metrics or indicators were tested that have been 
specifically  devised  for  measuring  the  visibility  and  momentum,  as  well  as  the  novelty  or 
emergence  of  research  topics.  These  are  the  ‘Prominence  Percentile’  for  measuring 
momentum, and the ‘Emergence Score’ to assess the novelty of a given topic. In addition, 
Top1%  and  Top10%  percentile  indicators  (PPTop1417)  are  employed,  as  used  in  the 
bibliometric analysis in the Evaluation study on Excellent Science under Horizon 2020.  

1.3.4.1. 

Indicators 

1.  The  Prominence  Percentile1418  is  an  indicator  of  the  momentum  or  visibility  of  a 
particular  topic.  Prominence  does  not  signify  ‘importance’.  Calculating  a  topic’s 
prominence combines three metrics to indicate the momentum of the topic, namely:  

•  Citation count in year n for papers published in n and n-1; 

•  Scopus view count in year n for papers published in n and n-1; and 

•  Average CiteScore1419 for year n 

Due  to  the  nature  of  certain  research  fields,  some  topics  will  never become  ‘prominent’. 
This does not mean the topic is not important; however, there is a correlation between the 
prominence  (momentum)  of  a  particular topic  and  the  amount  of  funding  per  publication 
within that topic. On average, the higher the momentum, the more money per publication 
is available for research into that topic. 

To arrive at a single and stable percentile score, the yearly figures from 2016 to 2021 were 
averaged  to  smooth  them.  For  the  purposes  of  reference,  please  note  that  the  overall 
average Prominence Percentile of all Topics is 78. 

2.  The  Emergence  Score1420  looks  at  four  S&T  attributes  –  novelty,  persistence, 
community and growth – and seeks to identify emergent terms, i.e. topical content that 
evidences these four attributes. To do so, topical content is extracted to discern terms 
or  phrases  that  show  high  growth,  along  with  evidence  of  novelty,  persistence  and 
community: 

•  Novelty – newness; this can pertain to technologies, technical sub-systems, 

functions, and/or uses; 

1416 https://service.elsevier.com/app/answers/detail/a_id/15181/supporthub/scopus/, and 
https://service.elsevier.com/app/answers/detail/a_id/12007/supporthub/scopus/. 
1417 Bornmann, L., 2014, available at: https://doi.org/10.1093/reseval/rvu002. 
1418 Klavans, R., & Boyack, K.W., 2017, available at: https://doi.org/10.1016/j.joi.2017.10.002. 
1419 https://en.wikipedia.org/wiki/CiteScore. 
1420 Porter, A.L., Garner, J., Carley, S.F., & Newman, N.C., 2019, available at: 
https://doi.org/10.1016/j.techfore.2018.04.016. 

1133 

 
 
 
•  Persistence  –  indicating  some  identity  and  momentum,  e.g.  shared  use  of 

acronyms, ongoing community interest; 

•  Community – as in ‘community of practice’, implying multiple players, not all 
within a single unit, and connected in some manner (e.g. citation connections 
in R&D literature or patent analyses); 

•  Growth – pertaining to increasing R&D outputs and/or to gains in other aspects 

(e.g. funding, number of players involved). 

3.  The Top Percentiles indicator looks at the impact of science, and measures world-class 
science  by  using  the  citations  accrued  to  calculate  the  number  and  share  of  peer-
reviewed  publications  that  have  resulted  from  projects  financed  by  EC-funded 
programmes and which have made core contributions to scientific fields, calculated as 
a percentage of the top most-cited publications (PPTop1421): 

•  Top1% – publications that are in the top 1% most highly cited publications in 

their field; and 

•  Top10% – publications that are in the top 10% most highly cited publications 

in their field. 

The Top Percentiles (PPTop) indicators are also part of the KIPs (Key Impact Pathways) 
framework: under KIP1: Creating high-quality new knowledge1422,1423. 

The above indicators are used to show how many EC-sponsored publications are leading 
in their field. This is done by calculating the percentage of EC-sponsored publications that 
are in the top percentiles as a proportion of the total EC-sponsored publications for each 
topic. As such, the share of EC-sponsored output that is published in a certain topic can be 
seen, along with the topic’s momentum, denoted using the prominence percentile. 

4.  To signify overlap, the number of overlapping Horizon 2020 Programmes and Social 
Challenges are shown, out of a total of 22. Coloured scales are used to enhance the 
interpretation  of  the  results  presented.  The  coloured  scales  denote  the  number  of 
outputs per programme, social challenge, country, funder and country bloc per topic 
covered.  

1.3.5.  Practicalities 

Similar  to  the  bibliometric  report  from  the  Evaluation  study  on  Excellent  Science  under 
Horizon 2020, the data used are of such a scale that only the main results can be represented 

1421 Waltman, L., & Schreiber, M., 2013, available at: https://doi.org/10.1002/asi.22775. 
1422 See: https://research-and-innovation.ec.europa.eu/strategy/support-policy-making/shaping-
eu-research-and-innovation-policy/evaluation-impact-assessment-and-monitoring/horizon-
europe-programme-analysis_en. 
1423 European Commission, Directorate-General for Research and Innovation, Nixon, J., Study to 
support the monitoring and evaluation of the framework programme for research and innovation 
along Key Impact Pathways – Indicator methodology and metadata handbook, Nixon, J.(editor), 
Luxembourg: Publications Office of the European Union, 2022, available at: 
https://data.europa.eu/doi/10.2777/44653. 

1134 

 
 
 
within  the  confines  of  this  report.  The  results  are  therefore  shared  in  full,  in  spreadsheet 
format, using our Dataverse1424. The datasets containing the full results can be found at: 

•  First Interim Report Results: 

https://dataverse.nl/dataverse/research_trend_analysis_H2020 

•  Second Interim Report Results: 

https://dataverse.nl/dataverse/Research_Trend_Analysis-
Funding_and_Topical_Overlap 

•  Final Report Results: 

https://dataverse.nl/dataverse/Excellent_Science_Phase_2_Final_Report 

1.3.6.  Results 

1.3.6.1. 

Internal analysis of Horizon 2020 topics 

Figure 9 presents the top 35 topics in relation to the programme parts and the number of EC-
sponsored outputs1425 reported for each of these topics.  

The first thing to notice is that the assignment of a topic to a single programme part or Social 
Challenge is the exception in this listing. If we look at the number of overlapping topics (in 
column  E),  we  find  that  on  average  there  is  an  overlap  of  6.3  programme  part/Social 
Challenge per topic (for the Top 200), with a 21 programme/challenge overlap as a maximum, 
and one programme part/Social Challenge per topic as the (obvious) minimum. 

A maximum of 95% overlap (21 programme parts/Social Challenges) is achieved for the topic 
‘Object Detection; Deep Learning; IOU (Intersection Over Union)’ which can be interpreted 
as  a  general-purpose  technology  (GPT),  and  as  such  may  find  application  across  a  large 
number of funding areas. Combining this information with the high ProminencePercentile of 
99.98  for  this  topic  might  explain  the  interest  of  a  diverse  set  of  areas  and  funders  in  a 
technology  with  high  momentum/visibility.  That  said,  one  could  wonder  whether  the 
fragmentation of funding for a single high-prominence GPT is in the interest of the funder or 
of research in the area as a whole. The Flagship projects (as analysed in the Evaluation study 
on Excellent Science under Horizon 2020) have shown that concentrating funding works well, 
and one might wonder if this also applies to the above. 

Also visible is the fact that instances of high overlap (40% overlap or more, i.e. 10 or more 
programme parts/Social Challenges overlapping) occur almost exclusively within very high 
prominence topics. This underlines an earlier finding that there is a correlation between the 
prominence (momentum) of  a particular topic and the amount of funding available for that 
topic. On average, the higher the prominence or momentum, the more funding is available 
for research on that topic. In our sample, the notable exception is the topic ‘Nasopharyngeal 
Swabs;  Serologic  Tests;  COVID-19’,  which  obviously  has  a  different  kind  of  urgency  and, 
while of lower prominence, has received funding from a diverse set of programme parts and 
Social Challenges. 

1424 https://dataverse.nl/dataverse/UNU-MERIT. 
1425 Please note that the number of unique publications (in column B) is lower than the number of 
combined publications for all programmes and challenges (the sum of each row). This is due to 
double-reporting of same outputs by multiple projects and across multiple programmes and 
challenges. 

1135 

 
 
 
The  above  conclusion  is  further  exemplified  by  Euratom,  which,  while  producing  high 
numbers  of  outputs  on  a  diverse  range  of  topics,  covers  a  relatively  low  number  of  high-
prominence topics. However, it is also usually the sole funder of these topics. 

Figure 9. Top 35 Topics x Horizon 2020 programme parts and Social Challenges. Note: the set of top 200 topics can be 
retrieved at https://doi.org/10.34894/QIJZK8. Source: authors’ own calculations 

The present report finds that there is a high incidence of overlap in funding between the ERC 
and MSCA. SEWP and INFRA also share a large number of topics with the aforementioned 
programme parts, although the number of outputs differs significantly. LEITs-ICT, FET and 
SC1 also share topics with the MSCA and ERC, and while the incidence of overlap appears 
to be lower, the number of publications in shared topics is larger than those for SEWP and 
INFRA.  

Although such overlaps might seemingly lack efficiency, the reason between overlaps with 
the ERC and MSCA can be explained by the breadth of research covered by both programme 
parts.  In  addition,  instances  of  MSCA  fellows  working  on  ERC-funded  projects  could 
contribute to this higher incidence of overlap. The lower levels of overlap in other programme 
parts  can  also  be  explained  by  looking  at  their  more  singular  focus.  LEITs  ICT  obviously 
researches  ICT-related  topics  also  visible  in  Figure  10,  and  such  a  focus  is  applicable  to 
many  other  programme  parts,  with  Euratom  again  being  the  prime  example.  The  above 
reasoning is confirmed by the more macro-level analysis of subject areas and disciplinary 
divisions presented below. 

1.3.6.2. 

Internal analysis of Horizon 2020 Subject Areas and Disciplines 

Following on from the micro-level analysis presented above, to the study also looks at higher-
level classifications of research funded through  Horizon 2020 and the meso-level ‘Subject 
Areas’ and macro-level ‘Disciplines’ covered by this research. For this, the ASJC (All Science 

1136 

Top 200 TopicNames12345678910111213Publications per Programme/Social Challenge713204269016197116321124295898174667151764600440934861724No.Publications per TopicTopic names Av. ProminencePercentile (2016-2021)# Overlapping programmes (22)ERCMSCALEITs-ICTFETSC1SEWPINFRAEURATOMSC2SC5SC3SC4SC71969Perovskite Solar Cells; Solar Cell; Formamidine100.001141931312123104911172895Molybdenum Disulfide; Monolayer; Van Der Waals99.9993491791147075813814Object Detection; Deep Learning; IOU99.9821206129261404019174106327304489Exoplanets; Kepler; Atmosphere99.725401921145377Nasopharyngeal Swabs; Serologic Tests; COVID-1996.0913574217202117666375Mobile Communication Systems; Slicing; Software Defined Networking98.898423336513147374Cavity; Mechanical Oscillators; Resonators99.54518570217898360Quantum Optics; Thermalization (Energy Absorption); Eigenvalues and Eigenfunctions99.474252587519345Network Function Virtualization; Transfer Function; Network Services99.286917312121010329Planet; T Tauri Star; Pre-main Sequence Stars98.994238129611318Intestine Flora; Ruminococcaceae; Microorganisms99.991016574391523612318Topological Insulators; Topology; Quantum Hall Effect99.7651976693213311Integrated Assessment Model; Carbon; Global Temperature Increase99.851141521624816214303Human-Robot Interaction; Humanoid Robot; Man-Machine Systems99.411066105648633115302Edge Computing; Internet Of Things; Location Awareness99.7811773202622415416295Divertors (Fusion Reactors); Beryllium; Deuterium92.3732229417294Boson; Partons; Higgs Bosons99.8441851333218294Ion Temperature; Turbulence; Stellarators93.4931729119292Bitcoin; Ethereum; Internet Of Things99.5814333112411591421674620288Precoding; Millimeter Waves; Antenna99.745458814911021287Exosomes; Extracellular Vesicles; Microrna99.98969983834466222284Neutron Stars; Mergers; Gravitational Waves99.74519977171923276Demand Response; Demand Side Management; Energy Conservation99.93914422416176124275Divertors (Fusion Reactors); Poloidal Flux; Plasma Density92.89127525275Graphite; Terahertz; Photodetectors99.84810540517012126268Internet Of Things; Traffic Engineering; Denial-Of-Service Attack99.821045141871218141227266Weyl; Metalloids; Paul Adrien Maurice Dirac99.887204491323528261Star Formation; Galaxies; Molecular Gases98.844207651029257Connectome; Functional Magnetic Resonance Imaging; Functional Connectivity99.9369748379401030256Origanum; Superconductivity; Topology99.37520531401431256Plastics; Marine Debris; Litter99.971678541115486322132255Fiberoptic Cable; Radio over Fiber; Passive Optical Networks98.683442420433248Homomorphic Encryption; Computer Security; Lattice Ideal97.27681789338734243Color Centers; Nitrogen; Nanodiamonds99.63711037124132735240Genome; CRISPR Associated Endonuclease Cas9; Gene Editing99.98111284552410451 
 
 
Journal Classification) scheme1426 and coding1427 is employed, as used by the Scopus citation 
index. Again, the reported DOIs are used, and these are linked to subject area codes, and to 
those disciplines to which these subject areas adhere. As a paper can be linked to multiple 
projects (and, as such, programme parts), there is some double-counting in this dataset1428. 
By contemplating the fact that a single piece of knowledge can be the result of a number of 
linked  experiments,  e.g.  projects,  one can  imagine  the  occurrence  of  such  double counts. 
Especially since the subject under analysis is knowledge constructs, i.e. topics or concepts, 
rather than productivity or impact metrics, we believe – as in the Evaluation study on Excellent 
Science  under  Horizon  2020  –  that  this  approach  is  justified.  However, as  one  paper  can 
cover multiple subject areas, a fractional count has been employed for these. This again is 
justified by the fact that one topic or concept can intersect with a number of subject areas. 

In  the  study  team’s  Dataverse,  available  at  https://doi.org/10.34894/1PUX77,  one  can  see 
the subdivision of Subject Areas and Disciplines across each of the 22 programme parts and 
Social  Challenges.  The  percentage  of  research  is  visualised  for  each  Subject  Area  and 
Discipline, and subsequently for each of the programme parts and Social Challenges. Figure 
10, below, shows this disciplinary division of the Horizon 2020 programme parts. 

Figure 10. Division of Horizon 2020 research across the main Disciplines. Source: authors’ own calculations. 

Figure 10 also allows some interesting insights to be deduced. These point to an emphasis 
among  Horizon  2020  programmes  on  the  Physical  Sciences  and  the  Life  Sciences,  with 
Health  and  SSH seemingly less  well  covered. However, looking  at  the  meso-level  data  in 
Figure  11,  the  picture  is  somewhat  more  nuanced.  For  instance,  for  Health  and  Medical 
Sciences,  lesser  attention  is  indeed  paid  by  most  programmes,  but  some  do  focus  on 
Medicine as a subject area (Innovation in SMEs), or on Immunology and Microbiology (LEITs 
Nanotech). That said, this does corroborate the findings of the external funder comparison, 
presented in Figure 13, which show that the EC programmes have little overlap with the topics 
covered  by  the  research  funded  by  more  Health  Sciences-focused  funders  such  as  the 
National Institutes of Health (NIH) in the US.  

1426 See: https://service.elsevier.com/app/answers/detail/a_id/12007/supporthub/scopus/. 
1427 See: https://service.elsevier.com/app/answers/detail/a_id/15181/supporthub/scopus/. 
1428 Approximately 4.5%. 

1137 

ERCFETFTIINFRAInnoSMEsLEITsBioLEITsICTLEITsManLEITsMatLEITsNanLEITsSpaMSCASEWPSWAFSEURATOMDisciplines% of Total% of Total% of Total% of Total% of Total% of Total% of Total% of Total% of Total% of Total% of Total% of Total% of Total% of Total% of TotalHealth Sciences6.02.710.15.730.55.31.50.42.37.60.36.710.28.30.7Life Sciences17.711.816.714.223.949.62.20.96.123.53.316.126.04.60.4Multidisciplinary5.35.80.64.95.22.60.90.42.63.22.63.63.64.20.6Physical Sciences62.678.070.472.339.641.491.392.987.564.792.667.255.038.098.0Social Sciences & Humanities8.41.82.12.90.81.04.15.31.51.01.26.45.244.90.2 
 
 
 
Figure 11. Division of Horizon 2020 research across the ASJC Subject Areas. Source: authors’ own calculations. 

1.3.6.3. 

Comparison of internal funding  

Below, a selection of topics are highlighted that perform well in terms of impact, as well as 
interesting ‘hot’ and emerging topics, for a number of the funding programme parts and Social 
Challenges. 
More can be found in our Dataverse, available at https://doi.org/10.34894/Q1NT0D. 

A comparison of topic clusters (Topic_TOP200) reveals that topics are largely concentrated 
in the ‘appropriate’ programmes, with the most obvious examples being Euratom and SwafS. 
A similar correspondence can be found in the focus on materials science and engineering in 
the LEITs, FET and INFRA, as well as in topics matching each of the Social Challenges. The 
ERC,  MSCA  and  SEWP  are  broadest  in  terms  of  the  range  of  topics  covered,  but  as 
explained earlier, this is to be expected. 

From this internal comparison of funding programme parts, it is interesting to note that there 
is a surprising overlap of topics across programme parts and Social Challenges. In itself, this 
is  not  an  issue  –  especially  for  those  topics  that  require  a  degree  of  funding  that  one 
programme alone cannot cover1429 (without resorting to focusing on an almost-singular topic), 
or for topics that can be researched from different angles1430. However, this situation does 
require higher-level coordination to ensure the efficiency of efforts, which could be a potential 
policy issue. 

For instance, work on sustainable energy, drug development and therapies, and genetics-
related  work  appear to  span  multiple  programme  parts.  Obviously,  COVID-19  had  a  large 
impact on the work carried out during the final year of Horizon 2020. It is interesting to note 
that  the  EC was  able  not  only  to  mobilise  the  necessary  resources,  but  also  to  effect  this 
sudden change in direction in response to the pandemic. 

1429 This might include topics in which research involves large or costly infrastructures, such as 
special laboratory set-ups or computational facilities with unique specifications. 
1430 Multidisciplinary, mixed methodologies and suchlike. 

1138 

ERCFETFTIINFRAInnoSMEsLEITsBioLEITsICTLEITsManLEITsMatLEITsNanLEITsSpaMSCASEWPSWAFSEURATOMSubject Area% of Total% of Total% of Total% of Total% of Total% of Total% of Total% of Total% of Total% of Total% of Total% of Total% of Total% of Total% of TotalAgricultural and Biological Sciences2.60.64.82.62.27.70.30.30.31.12.33.26.51.10.0Arts and Humanities2.00.20.00.40.00.40.30.10.50.10.01.30.43.00.1Biochemistry, Genetics and Molecular Biology10.05.67.08.913.826.91.00.43.510.00.88.012.42.30.3Business, Management and Accounting0.30.10.40.10.00.00.51.40.20.00.10.50.67.40.0Chemical Engineering2.72.92.72.31.89.60.66.98.47.10.23.12.70.30.4Chemistry6.88.04.07.04.39.90.84.214.012.50.97.06.20.41.2Computer Science7.89.87.48.35.61.946.120.73.91.58.49.18.314.71.3Decision Sciences0.30.20.30.40.20.01.62.40.10.00.30.40.41.90.0Dentistry0.00.00.00.00.40.00.00.00.10.00.00.00.00.00.0Earth and Planetary Sciences7.00.61.011.41.10.20.41.10.10.140.54.52.40.90.6Economics, Econometrics and Finance0.80.10.00.30.00.20.10.30.10.00.00.40.80.80.0Energy0.81.011.51.31.01.40.76.84.61.00.72.02.43.414.3Engineering5.69.219.64.79.05.522.834.413.98.59.810.010.84.914.0Environmental Science1.90.51.93.11.27.30.33.63.38.95.23.54.57.30.9Health Professions0.10.10.40.10.20.10.10.00.20.10.20.20.30.1Immunology and Microbiology1.80.51.41.72.213.50.10.00.21.60.11.42.30.30.0Material Science6.013.77.86.65.92.95.25.925.718.12.67.26.71.513.0Mathematics5.83.23.93.41.01.57.84.00.80.72.74.72.62.33.3Medicine5.82.69.75.229.65.11.40.42.17.20.36.29.66.50.6Multidisciplinary5.35.80.64.95.22.60.90.42.63.22.63.63.64.20.6Neuroscience2.74.82.60.32.70.30.80.20.10.30.01.92.00.80.0Nursing0.10.00.00.20.30.10.00.00.00.10.00.20.21.60.0Pharmacology, Toxicology and Pharmaceutics0.60.31.00.73.01.10.10.02.010.60.01.62.70.10.0Physics and Astronomy18.128.910.624.18.71.26.65.312.76.421.716.08.42.249.0Psychology1.10.70.30.20.30.00.20.10.00.00.00.70.31.80.0Social Sciences3.80.41.11.60.40.21.41.10.50.80.73.02.629.90.1Veterinary0.00.00.00.20.00.00.00.00.00.10.00.10.20.00.0 
 
 
 
The above conclusions are amplified when the analysis is extended by looking at the number 
of publications per topic in relation to the overall EC funding these topics receive1431. Using 
these data, it becomes clear that the ERC has been most efficient programme part in terms 
of  allocating  funding  across  a  wide  spectrum  of  topics,  while  also  safeguarding  sufficient 
funding for those topics that are not as prominent or productive as some of the leading ones 
(leading in terms of total productivity). This is less evident for most of the other programme 
parts, with the exception of the MSCA. Again, this will be an effect of the concentration of 
topics and funding into the ‘appropriate programme parts, but it could also be influenced by 
those topics that require significant research infrastructure, and which one can assume will 
also be concentrated in more topically focused programme parts. 

Below,  a selection  of  findings  are presented  for a  number of  programme  parts  and Social 
Challenges. This section looks in particular at productivity in terms of publications, and impact 
in terms of the Top 1% and 10% cited publications. In addition, it examines the momentum 
and visibility of a few of the topics funded, using the prominence and emergence scoring from 
the calculations carried out. 

ERC – as a programme that funds a broad array of research, an equally broad number of 
topics receive input from the ERC. Quite a number of these are leading topics in terms of 
impactful, highly cited research that accounts for a large share of funded research on these 
topics that features in the top 1 % and top 10 % of the world’s most highly cited publications 
in their field. From the top 200, we select: 

In terms of topical impact: 

•  Work  on  solar energy  production  and  solar cells  (Perovskite  Solar Cells;  Solar Cell; 
Formamidine), of which 63% of the EC-sponsored research on this topic features in the 
Top 10%. 

•  Work  in  the  area  of  astronomy  (Neutron  Stars;  Mergers;  Gravitational  Waves)  has 
included  EC-sponsored  work,  of  which  71%  of  the  research  funded  by  the  ERC 
features in the top 10%, and 20% of the output funded features in top 1%.  

•  Work in the area of photocatalysis (Alkene; Cross-coupling Reactions; Photocatalysis), 
in  which  a  staggering  84%  of  ERC-funded  research  on  the  topic  features  in  the  top 
10%. 

•  Work  on  immunotherapy  and  drug  development  (Nivolumab;  Pembrolizumab; 
Immunotherapy) features 34% of related ERC-sponsored research in the top 1% of the 
world’s most highly cited publications on that topic. 

In terms of momentum and novelty: 

•  Work on groundwater and water storage (GRACE; Groundwater; Water Storage) has 
both a high emergence score, as well as a high average prominence percentile. 

•  Work on thermoacoustics has a similarly a good emergence score, and a high average 

prominence percentile. 

• 

In  terms  of  the  number  of  publications  funded,  high  emergence  scores  and  high 
average  prominence  percentiles, ERC funded  research  on  Beyond  5G  (Beyond  5G; 

1431 Data and analysis are available at: https://doi.org/10.34894/M9JGU2. 

1139 

 
 
 
Massive  MIMO;  Intelligent  Reflecting  Surface)  and  on  Migration  (Transnationalism; 
Human Migration; Dual Citizenship) scores highly. 

FET – this programme specifically funds frontier research. Although basic science is funded, 
there is a certain focus on applied science. Here, research is funded on similar topics to those 
of  the  ERC,  but  also  materials  science  and  engineering-focused  topics  of  a  more  applied 
nature. 

In terms of topical impact: 

•  Work  on  solar energy  production  and  solar cells  (Perovskite  Solar Cells;  Solar Cell; 
Formamidine), a topic also sponsored heavily by the ERC. Of FET-sponsored research 
on this topic, 69% features in the top 10%, and 29% features in the top 1%. 

•  Work  on  organic  polymers  (Organic  Polymers;  Triazines;  Porous),  of  which  68%  of 
FET-sponsored research on this topic features in the top 10%, and 29% features in the 
top 1%. 

•  Work on electrochemical capacitors (Electrochemical Capacitors; Silicon Nanowires; 
Graphite), of which 87% of FET-sponsored research on this topic features in the top 
10% in this field, and a staggering 57% features in the top 1%. 

•  Work  in  the  field  of  mechanotransduction1432  and  the  topic  (Mechanotransduction; 
Focal  Adhesions;  Extracellular  Matrix).  Even  though  the  number  of  FET-funded 
publications is not large, 83% of FET-sponsored research on this topic has managed 
to feature in the top 10% in this field, and 26% features in the top 1%. 

In terms of momentum and novelty: 

•  Work on adsorption properties (Effective Properties; Random Sequential Adsorption; 

Fiber) 

•  Work on elastomers (Elastomers; Actuator; Nematic) 

•  Work on photonics (Filamentation; Terahertz; Supercontinuum Generation) 

INFRA – this programme appears to be most successful in sponsoring materials science-
related research. 

In terms of topical impact: 

•  Work  in  the  area of  materials  science  (Neural  Networks;  Potential  Energy Surfaces; 
Materials Science), with a relatively high number of publications sponsored, of which 
64% of INFRA-sponsored research on this topic features in the top 10% in this field, 
and 24% features in the top 1%. 

•  Work  in  the  area  of  materials  science  and  high-entropy  ceramics1433  (Thermal 
Conductivity;  Solid  Solution;  High-Entropy  Ceramics),  which  86%  of  the  INFRA-
sponsored research on the topic features in the top 10% in this field, and a staggering 
43% features in the top 1%. 

1432 https://www.sciencedirect.com/topics/medicine-and-dentistry/mechanotransduction. 
1433 https://www.sciencedirect.com/topics/materials-science/high-entropy-ceramics. 

1140 

 
 
 
•  Work  on  COVID-19  (Pharmacotherapy;  Prolongation;  COVID-19),  in  which  77%  of 
INFRA-sponsored  research  on  the  topic  features  in  the  top  10%  in  this  field,  and  a 
staggering 31% features in the top 1%. 

In terms of momentum and novelty: 

•  Work in the field of astronomy (Asymptotic Giant Branch Stars; Iron; Dwarf Galaxies) 
appears to attract particular attention, with an exceptionally high emergence score and 
a high prominence percentile as well. 

SEWP  –  the  Spreading  Excellence  and  Widening  Participation  programme  has  seen 
successful collaborations on a number of topics. 

In terms of topical impact: 

•  Work in the field of health and medicine (Multifactorial Inheritance; Summary Statistic; 
Single  Nucleotide  Polymorphism),  in  which  a  staggering  88%  of  SEWP-sponsored 
research on the topic features in the top 10% in this field, and 32% features in the top 
1%. 

•  Other  work  in  the  same  field  (Anthocyanins;  Gene;  Dihydroflavanol  4-Reductase) 
shows similar impact, with 73% of SEWP-sponsored research on this topic featuring in 
the top 10% in this field, and 18% featuring in the top 1%. 

In terms of momentum and novelty: 

•  Again, work in the field of medicine on MicroRNA (Exosomes; Extracellular Vesicles; 
Microrna)  has  a  prominence  percentile  of  almost  100%,  denoting  a  very  ‘hot’  topic, 
which also has a very high emergence score. From the interpretation of the indicators, 
this would appear to be a highly topic, attracting considerable funding. 

SC1 – this Social Challenge, which focused on ‘Health, demographic change and well-being’, 
has funded a large body of health-related research. 

In terms of topical impact: 

•  Work on COVID-19 (ARIMA; Mathematical Modeling; COVID-19) has generated SC1-
funded research, 74% of which features in the top 10% in its field, and 29% features in 
the top 1%. 

•  Other  work  on  COVID-19  (Interleukin  6;  Rheumatology;  COVID-19)  has  generated 
SC1-funded  research,  73%  of  which  features  in  the  top  10%  in  its  field,  and  a 
staggering 54% features in the top 1%. 

In terms of momentum and novelty: 

•  Work 

in 

the  area  of  chemical  engineering  (Fluorocarbon;  Fluorochemicals; 
Perfluorooctane)  has  a  prominence  percentile  of  nearly  100%,  denoting  a  very  ‘hot’ 
topic,  as  well  as  a  very  high  emergence  score.  As  with  the  SEWP-funded  example 
above, attracting considerable funding. 

1141 

 
 
SC5  –  this  Social  Challenge,  which  focused  on  ‘Climate  action,  environment,  resource 
efficiency  and  raw  materials’  has  funded  a  sizeable  body  of  research  related  to  topics 
focusing on climate change, the environment and (natural) resource usage.  

In terms of topical impact: 

•  Work  on  the  maritime  environment  (Oceans  And  Seas;  Carbonate  System;  Ocean 
Acidification) has generated  SC1-funded research, 60% of which features in the top 
10% in its field, and 15% features in the top 1%. 

In terms of momentum and novelty: 

•  Work  in  the  area  of  the  circular  economy  (Industrial  Symbiosis;  Sustainable 
Development;  Circular  Economy)  has  both  a  high  prominence  percentile  of  nearly 
99.5%, denoting that this as a ‘hot’ topic, as well as a high emergence score. Looking 
at these indicators, attracting considerable funding. 

Note on the prominence and emergence analysis: in the spreadsheet that can be retrieved 
from  https://doi.org/10.34894/Q1NT0D,  under  the  tab  ‘Topic_TOP200’,  the  Top  200  main 
topics published can be found, arranged by number of publications per Horizon 2020 sub-
programme and Social Challenge. These topics consist of three closely connected keywords, 
as explained earlier. Furthermore, the number of highly cited publications in the top 1% and 
top 10% can be found (in red, in the columns denoted by a 1), as a share of the total number 
of EC-sponsored publications on each of the given topics, signalling the impact achieved by 
EC-sponsored research in these topics. Under the tab ‘TopicKeywords_TOP200’, there are 
the single keywords that make up the topics, with the same calculations and metrics as for 
‘Topic_TOP200’.  

Under  the  tab  ‘TopicEmergence’  one  can  find  the  keyword  constructs  that  define  a  topic. 
These  topics  are  made  up  of  three  keywords  that  are  connected  through  similarity  and 
instances of appearance in same publications. These topics, as defined, also add some initial 
context  to  the  keywords  by  virtue  of  ‘connectedness’,  and  can  then  also  be  seen  as  a 
rudimentary  S&T  concept.  Added  to  this  are  both  ‘emergence  scores’  as  well  as  average 
‘prominence percentiles’, denoting the novelty and momentum of a given topic, respectively. 
Under  the  tab  ‘TopicKeywordEmergence’,  one  can  find  the  various  ‘emerging’  topic 
keywords, as distributed across each Horizon 2020 sub-programme and Social Challenge, 
and the publications in which they appear. Added to this are the ‘emergence scores’, which 
denote the novelty value of each topic keyword. Interestingly, one can see that for these ‘new’ 
topics, there is little difference between the publication count for the keyword constructs and 
for the individual topic keywords. 

1.3.6.4. 

External analysis of Horizon 2020 topics 

Figure 12 shows the top 50 topics in the world and at a regional level, for both the EU-27 and 
the EU-28, and for the number of EC-funded outputs reported for each of these topics.  

1142 

 
 
Figure 12. Top 50 topics in terms of productivity worldwide, in the EU-27, EU-28 and EC-funded. Note: the data presented 
can be found at: https://doi.org/10.34894/6CO0VO. The full set of 23,755 topics for all 196 countries and territories can be 
retrieved from: https://doi.org/10.34894/KQRWEN. Source: authors’ own calculations. 

The  above  analysis  presents  interesting  information1434,  some  of  which  might  require 
additional insights in order to fully understand the implications of these findings. Out of the 
10 topics with the highest outputs among all Horizon 2020 programmes, only five appear in 
the Top 50 topics researched and funded1435 by the EU-27 and EU-28 and by selected third 
countries.  These  are  ‘Perovskite  Solar  Cells;  Solar  Cell;  Formamidine’,  ‘Molybdenum 
Disulfide; Monolayer; Van Der Waals’, ‘Object Detection; Deep Learning; IOU’, ‘Exoplanets; 
Kepler;  Atmosphere’  and  ‘Nasopharyngeal  Swabs;  Serologic  Tests;  COVID-19’.  The  fifth 
topic  is  on  COVID-19-related  testing.  The  Nano  science  and  technology-related  topic 
Molybdenum Disulfide (MoS2) appears in the top 10, while the topic relating to Exoplanets 
appears in the top 30 for the EU-27 and selected third countries, and in the top 40 worldwide. 
The other topics featured in the top 10 of EC-funded research are not present at all in this 
top 50. This points to a partial “disconnect” between topics prioritised in the main knowledge 
economies, such as China and the USA, and those prioritized by the EC. Whether this is a 
result from national Smart Specialization1436 strategies or other causes we cannot ascertain, 
and this would require additional research. 

COVID-19 related research features somewhat more prominently in the comparator set than 
that it does in the Horizon 2020 programmes and SCs set. On a whole Health, Medicine and 
Life Sciences related research features less (high) on the EC sponsored topic list than it does 
elsewhere. We find in this Top 50 listing, that 12 Topics World-wide, and 15 Topics for the 
EU-27  and  5  selected  knowledge  economies  dataset  contain  Health,  Medicine  and  Life 
Sciences  related  topics.  The  EC  has  funded,  under  Horizon  2020,  only  5  Topics  in  these 
disciplinary categories in the Top 50 most-researched topics. One might question whether 
the EC needs to sponsor research that is already well covered in the region, but likewise, the 

1434 Please see: https://dataverse.nl/dataverse/Excellent_Science_Phase_2_Final_Report. 
1435 By local, national, or other funding. 
1436 See Wintjes, R., & Hollanders, H., 2011, available at: 
https://ideas.repec.org/p/unm/unumer/2011027.html. 

1143 

Topic NamesAv. ProminencePercentile (2016-2021)WorldECEC % of worldEU-27EU-28Topic NamesAv. ProminencePercentile (2016-2021)WorldECEC % of worldEU-27EU-28Topic NamesAv. ProminencePercentile (2016-2021)32 Country totalECEC% of 32EU-27EU-28Topic NamesAv. ProminencePercentile (2016-2021)32 Country totalECEC% of 32EU-27EU-28Object Detection; Deep Learning; IOU99.976511019508140.802019324451Perovskite Solar Cells; Solar Cell; Formamidine99.99913496879691.951085512831Object Detection; Deep Learning; IOU99.97651710938141.142019324451Perovskite Solar Cells; Solar Cell; Formamidine99.99913354549692.731085512831Boson; Partons; Higgs Bosons99.84075520112940.572124223092Molybdenum Disulfide; Monolayer; Van Der Waals99.99322348258952.5781079118Perovskite Solar Cells; Solar Cell; Formamidine99.99913354549692.731085512831Molybdenum Disulfide; Monolayer; Van Der Waals99.99322254738953.5181079118Perovskite Solar Cells; Solar Cell; Formamidine99.99913496879691.951085512831Object Detection; Deep Learning; IOU99.976511019508140.802019324451Nivolumab; Pembrolizumab; Immunotherapy99.99409340581400.411306214772Object Detection; Deep Learning; IOU99.97651710938141.142019324451Radiological Findings; Clinical Features; COVID-1993.3389644410770.17903011100Exoplanets; Kepler; Atmosphere99.72145166764892.9366428366Boson; Partons; Higgs Bosons99.84075279832941.052124223092Exoplanets; Kepler; Atmosphere99.72145127394893.8466428366Nivolumab; Pembrolizumab; Immunotherapy99.99409409041400.341306214772Nasopharyngeal Swabs; Serologic Tests; COVID-1996.08741370893771.02965012164Microrna; Growth Arrest Specific Transcript 5; Small Nucleolar RNA99.9869627251900.3328063227Nasopharyngeal Swabs; Serologic Tests; COVID-1996.08741230603771.63965012164Nasopharyngeal Swabs; Serologic Tests; COVID-1996.08741370893771.02965012164Mobile Communication Systems; Slicing; Software Defined Networking98.8884555683756.7324272753Intestine Flora; Ruminococcaceae; Microorganisms99.9887271473181.171093012307Mobile Communication Systems; Slicing; Software Defined Networking98.8884541553759.0324272753Intestine Flora; Ruminococcaceae; Microorganisms99.9887363733180.871093012307Cavity; Mechanical Oscillators; Resonators99.5432274743745.0020612447Molybdenum Disulfide; Monolayer; Van Der Waals99.99322254738953.5181079118Cavity; Mechanical Oscillators; Resonators99.5432256883746.5820612447Molybdenum Disulfide; Monolayer; Van Der Waals99.99322348258952.5781079118Quantum Optics; Thermalization (Energy Absorption); Eigenvalues and Eigenfunctions99.4746658153606.1924172928Nasopharyngeal Swabs; Serologic Tests; COVID-1996.08741230603771.63965012164Quantum Optics; Thermalization (Energy Absorption); Eigenvalues and Eigenfunctions99.4746647583607.5724172928Psychological Support; Mindfulness; COVID-1995.8156632682760.23818110435Network Function Virtualization; Transfer Function; Network Services99.2791759053455.8424012691Radiological Findings; Clinical Features; COVID-1993.3389622569770.34903011100Network Function Virtualization; Transfer Function; Network Services99.2791745773457.5424012691Microrna; Growth Arrest Specific Transcript 5; Small Nucleolar RNA99.9869631312900.2928063227Planet; T Tauri Star; Pre-main Sequence Stars98.9872183933293.9237024466Embedding; Named Entity Recognition; Entailment99.61639219461580.7265808118Planet; T Tauri Star; Pre-main Sequence Stars98.9872165733295.0137024466Bitcoin; Ethereum; Internet Of Things99.57769294932920.9967538252Intestine Flora; Ruminococcaceae; Microorganisms99.9887363733180.871093012307Exosomes; Extracellular Vesicles; Microrna99.98226181442871.5865307361Intestine Flora; Ruminococcaceae; Microorganisms99.9887271473181.171093012307Embedding; Named Entity Recognition; Entailment99.61639291541580.5465808118Topological Insulators; Topology; Quantum Hall Effect99.7634884743183.7524432836Psychological Support; Mindfulness; COVID-1995.8156617121760.44818110435Topological Insulators; Topology; Quantum Hall Effect99.7634865423184.8624432836Silver Nanoparticles; Green Synthesis; Biofabrication99.938624655220.0928793113Integrated Assessment Model; Carbon; Global Temperature Increase99.84679104483112.9837775070Microstructure; Titanium Alloy (TiAl6V4); Inconel (Trademark)99.97513168901630.9772928606Integrated Assessment Model; Carbon; Global Temperature Increase99.8467977963113.9937775070Electrode; Cobaltous Sulfide; Electrode Materials99.9890523669250.1130763415Human-Robot Interaction; Humanoid Robot; Man-Machine Systems99.41343123883032.4549995987Connectome; Functional Magnetic Resonance Imaging; Functional Connectivity99.9301162782571.5851866706Human-Robot Interaction; Humanoid Robot; Man-Machine Systems99.4134394663033.2049995987Nanofluid; Stagnation Point Flow; MHD Flow99.898262363810.0012641606Edge Computing; Internet Of Things; Location Awareness99.78148199773021.5141794987Bitcoin; Ethereum; Internet Of Things99.57769158342921.8467538252Edge Computing; Internet Of Things; Location Awareness99.78148122643022.4641794987Microstructure; Titanium Alloy (TiAl6V4); Inconel (Trademark)99.97513232281630.7072928606Divertors (Fusion Reactors); Beryllium; Deuterium92.37084247629511.9115421765Polymer Solar Cells; Polymers; Organic Photovoltaics99.98453152361030.6852145955Divertors (Fusion Reactors); Beryllium; Deuterium92.37084224529513.1415421765Exosomes; Extracellular Vesicles; Microrna99.98226230842871.2465307361Boson; Partons; Higgs Bosons99.84075520112940.572124223092Electrode; Cobaltous Sulfide; Electrode Materials99.9890514892250.1730763415Boson; Partons; Higgs Bosons99.84075279832941.052124223092Biochar; Soil; Black Carbon99.9688721120420.2043124925Ion Temperature; Turbulence; Stellarators93.4869438002947.7417202041Genome; CRISPR Associated Endonuclease Cas9; Gene Editing99.98123143012401.6838684713Ion Temperature; Turbulence; Stellarators93.4869434382948.5517202041Technology Acceptance Model; Mobile Payment; E-Learning99.7581821041160.0837934560Bitcoin; Ethereum; Internet Of Things99.57769294932920.9967538252Collaborative Filtering; Recommender Systems; Factorization99.8349313747670.4937654268Bitcoin; Ethereum; Internet Of Things99.57769158342921.8467538252Collaborative Filtering; Recommender Systems; Factorization99.8349321015670.3237654268Precoding; Millimeter Waves; Antenna99.73862138682882.0841214910Transcatheter Aortic Valve Implantation; Heart Valve Prosthesis; Aortic Stenosis99.814661318380.0669677953Precoding; Millimeter Waves; Antenna99.73862100182882.8741214910Connectome; Functional Magnetic Resonance Imaging; Functional Connectivity99.9301202332571.2751866706Exosomes; Extracellular Vesicles; Microrna99.98226230842871.2465307361Biochar; Soil; Black Carbon99.9688712985420.3243124925Exosomes; Extracellular Vesicles; Microrna99.98226181442871.5865307361Polymer Solar Cells; Polymers; Organic Photovoltaics99.98453201431030.5152145955Neutron Stars; Mergers; Gravitational Waves99.7412781702843.4832613804Oxygen Production; Electrocatalysts; Catalyst99.99305128671641.2734113740Neutron Stars; Mergers; Gravitational Waves99.7412763572844.4732613804Edge Computing; Internet Of Things; Location Awareness99.78148199773021.5141794987Demand Response; Demand Side Management; Energy Conservation99.93287183962761.5050525875Cyanogen; Graphitic Carbon Nitride; Photocatalysts99.9913112807520.4120322352Demand Response; Demand Side Management; Energy Conservation99.93287116352762.3750525875Plastics; Marine Debris; Litter99.97391197132561.3069238139Graphite; Terahertz; Photodetectors99.83762113142752.4323262813Exoplanets; Kepler; Atmosphere99.72145127394893.8466428366Graphite; Terahertz; Photodetectors99.8376280692753.4123262813Biosorbents; Congo Red; Adsorption99.913381908650.0319402095Divertors (Fusion Reactors); Poloidal Flux; Plasma Density92.8942527772759.9012781536Plastics; Marine Debris; Litter99.97391123952562.0769238139Divertors (Fusion Reactors); Poloidal Flux; Plasma Density92.89425240027511.4612781536Cyanogen; Graphitic Carbon Nitride; Photocatalysts99.9913118960520.2720322352Internet Of Things; Traffic Engineering; Denial-Of-Service Attack99.81568156792681.7137124278Zinc Air Batteries; Electrocatalysts; Catalyst99.9869612279740.6024972809Internet Of Things; Traffic Engineering; Denial-Of-Service Attack99.8156896562682.7837124278Sentiment Classification; Data Mining; Product Review99.8031118713470.2530683632Weyl; Metalloids; Paul Adrien Maurice Dirac99.8779689102662.9919262175Strain Sensor; Flexible Electronics; Sensor99.98122122781461.1935334091Weyl; Metalloids; Paul Adrien Maurice Dirac99.8779667092663.9619262175Demand Response; Demand Side Management; Energy Conservation99.93287183962761.5050525875Star Formation; Galaxies; Molecular Gases98.8354109372612.3948166041Edge Computing; Internet Of Things; Location Awareness99.78148122643022.4641794987Star Formation; Galaxies; Molecular Gases98.835484532613.0948166041Mechanical Properties; Sisal; Coir99.8260718244490.2729733395Connectome; Functional Magnetic Resonance Imaging; Functional Connectivity99.9301202332571.2751866706Thrombus Aspiration; Brain Ischemia; Solitaire99.8217711766450.3855596173Connectome; Functional Magnetic Resonance Imaging; Functional Connectivity99.9301162782571.5851866706Genome; CRISPR Associated Endonuclease Cas9; Gene Editing99.98123180412401.3338684713Plastics; Marine Debris; Litter99.97391197132561.3069238139Fatty Liver; Proton Density (Concentration); Keratin-1899.87913117151050.9050305974Plastics; Marine Debris; Litter99.97391123952562.0769238139Fog Computing; Network Protocol; Internet of Things99.75941179372101.1742564917Origanum; Superconductivity; Topology99.3702652342564.8918081934Cultural Ecosystem Services; China; Natural Capital99.91966116652201.8963007472Origanum; Superconductivity; Topology99.3702640132566.3818081934Cause-Related Marketing; Corporate Social Responsibility; Corporate Philanthropy99.7907217657190.1155406746Fiberoptic Cable; Radio over Fiber; Passive Optical Networks98.6841546092555.5316901967Consensus Problem; Formation Control; Topology99.8351311649330.2816882003Fiberoptic Cable; Radio over Fiber; Passive Optical Networks98.6841536372557.0116901967Oxygen Production; Electrocatalysts; Catalyst99.99305176101640.9334113740Homomorphic Encryption; Computer Security; Lattice Ideal97.2666251502484.8213311561Demand Response; Demand Side Management; Energy Conservation99.93287116352762.3750525875Homomorphic Encryption; Computer Security; Lattice Ideal97.2666236102486.8713311561ARIMA; Mathematical Modeling; COVID-1996.49677171781610.9438475093Color Centers; Nitrogen; Nanodiamonds99.6343270392433.4521942516Alliance Portfolios; Firm; Open Innovation99.798211289320.2861277379Color Centers; Nitrogen; Nanodiamonds99.6343251792434.6921942516Alliance Portfolios; Firm; Open Innovation99.798216990320.1961277379Genome; CRISPR Associated Endonuclease Cas9; Gene Editing99.98123180412401.3338684713Sodium-ion Batteries; Electrode; Ion Storage99.9878411229590.5325052845Genome; CRISPR Associated Endonuclease Cas9; Gene Editing99.98123143012401.6838684713Transcatheter Aortic Valve Implantation; Heart Valve Prosthesis; Aortic Stenosis99.814661692480.0569677953Star Formation; Galaxies; Galactic Evolution99.44382121092401.9846256035Cause-Related Marketing; Corporate Social Responsibility; Corporate Philanthropy99.7907210665190.1855406746Star Formation; Galaxies; Galactic Evolution99.4438288812402.7046256035Maximum Entropy; Ecosystem; Environmental Space99.82452168391190.7150776070Magnetic Bubbles; Magnetism; Domain Walls99.6930961672403.8920692371Maximum Entropy; Ecosystem; Environmental Space99.82452102331191.1650776070Magnetic Bubbles; Magnetism; Domain Walls99.6930947332405.0720692371Exoplanets; Kepler; Atmosphere99.72145166764892.9366428366Single-Cell Analysis; Small Cytoplasmic RNA; Single Cell RNA Seq99.9212281842382.9119472595Prescription Drug Monitoring Programs; Narcotic Analgesic Agent; Morphinomimetic Agent99.841411018270.0711311573Single-Cell Analysis; Small Cytoplasmic RNA; Single Cell RNA Seq99.9212268992383.4519472595Nanocellulose; Nanocrystal; Nanowhiskers99.96453166631230.7442764587Optical Networks; Traffic Grooming; Fiberoptic Cable98.3096362982383.7826732959Rolling Bearing; Rotating Machinery; Failure Analysis99.8270910152270.2719332371Optical Networks; Traffic Grooming; Fiberoptic Cable98.3096352802384.5126732959Climate Change Adaptation; Urban Climate; Resilience99.7792816650910.5541345647Divertors (Fusion Reactors); Pedestal; Magnetism92.8677630522367.7312831624Social Media; Online Reviews; Brand Community99.8912810072120.1240455004Divertors (Fusion Reactors); Pedestal; Magnetism92.8677628452368.3012831624Social Media; Online Reviews; Brand Community99.8912816407120.0740455004Gravitational Waves; LIGO (Observatory); Compact98.9657351002224.3521322625Precoding; Millimeter Waves; Antenna99.73862100182882.8741214910Gravitational Waves; LIGO (Observatory); Compact98.9657339712225.5921322625Fatty Liver; Proton Density (Concentration); Keratin-1899.87913163721050.6450305974Cultural Ecosystem Services; China; Natural Capital99.91966163642201.3463007472Cancer; Photoacoustics; Theranostic Nanomedicine99.976710002640.6415781768Cultural Ecosystem Services; China; Natural Capital99.91966116652201.8963007472Cultural Ecosystem Services; China; Natural Capital99.91966163642201.3463007472AERONET; Aerosol; Aerosol Property99.2371171632203.0726342953Electric Vehicle; Vehicle-To-Grid; Charging99.887139766760.7836254275AERONET; Aerosol; Aerosol Property99.2371153252204.1326342953Zinc Air Batteries; Electrocatalysts; Catalyst99.9869615894740.4724972809Bis(1,3,5-Benzenetricarboxylate)Tricopper(II); MIL-101; Adsorption99.96888124742171.7428433443Ecosystem; Plant Communities; Ecosystem Stability99.8370697461621.6653236110Bis(1,3,5-Benzenetricarboxylate)Tricopper(II); MIL-101; Adsorption99.9688886892172.5028433443Heat Transfer; Heat Transfer Enhancement; Automobile Radiators99.9387915767160.1017182061Communication Systems; Optical Wireless; Orthogonal Frequency Division Multiplexing (OFDM)99.6277114752171.8925713668ARIMA; Mathematical Modeling; COVID-1996.4967796981611.6638475093Communication Systems; Optical Wireless; Orthogonal Frequency Division Multiplexing (OFDM)99.627776422172.8425713668Internet Of Things; Traffic Engineering; Denial-Of-Service Attack99.81568156792681.7137124278Deuterium Plasma; Tungsten; Helium Ions97.231427182177.98826886Internet Of Things; Traffic Engineering; Denial-Of-Service Attack99.8156896562682.7837124278Deuterium Plasma; Tungsten; Helium Ions97.231423202179.35826886Geopolymers; Coal Ash; Alkali99.9130415342990.6534043941Mechanical Properties; Divertors (Fusion Reactors); Spark Plasma Sintering97.1231122342179.71861947Nanocellulose; Nanocrystal; Nanowhiskers99.9645395981231.2842764587Mechanical Properties; Divertors (Fusion Reactors); Spark Plasma Sintering97.12311197621710.98861947Electric Vehicle; Vehicle-To-Grid; Charging99.8871315313760.5036254275Fog Computing; Network Protocol; Internet of Things99.75941179372101.1742564917Human-Robot Interaction; Humanoid Robot; Man-Machine Systems99.4134394663033.2049995987Fog Computing; Network Protocol; Internet of Things99.7594177832102.7042564917Strain Sensor; Flexible Electronics; Sensor99.98122152791460.9635334091Localization; Stimulated Emission; Point Spread Functions99.6362663772073.2521592655Osimertinib; Mutation; ErbB-1 Genes99.784069434110.1231883578Localization; Stimulated Emission; Point Spread Functions99.6362653312073.8821592655Thrombus Aspiration; Brain Ischemia; Solitaire99.8217715269450.2955596173Separatrix; Plasma Turbulence; Simulation85.32846138020614.93672776Catalyst; Cobalt Phosphide; Water Splitting99.983669428910.9716941951Separatrix; Plasma Turbulence; Simulation85.32846111520618.48672776 
 
 
 
question can be raised as to whether it would be desirable for the EC to have topical gaps in 
its funding portfolio.  

A further observation is that the UK clearly played a substantial role in strengthening the core 
EU research topics during Horizon 2020, when EU-27 versus EU-28 topics and publication 
numbers per topic are compared. If, following Brexit, these topics have remained core topics, 
the departure of the UK will have had a noticeable impact on EU publication numbers. 

1.3.6.5. 

Comparison with external funders  

For this external comparison against the performance of the European Commission, we have 
chosen a select number of large third-country funders from China, Japan, the United Kingdom 
and the United States, namely: 

•  National Natural Science Foundation of China (NNSFC, China)  

• 

Japan Science and Technology Agency (JSTA, Japan) 

•  Engineering and Physical Sciences Research Council (EPSRC, UK) 

•  National Institutes of Health (NIH, USA) 

•  National Science Foundation (NSF, USA) 

Using SciVal data, an overview of the topics funded by each of the above funders has been 
generated, along with prominence percentiles, and the scholarly output generated. By virtue 
of their calculation using SciVal data, the final two metrics can differ slightly, but the overall 
comparison  remains  valid1437.  To  further  enable  comparison,  funded  publications  as  a 
percentage of total output on each topic has been added.  

Funder 

Total 
number 
of 
topics 

Number of 
topics that 
overlap with 
EC 

Number of 
different 
topics 

Number of funded topics 
with more than a 1% 
share of worldwide 
output 

EC 

984 

NNSFC 

198 

JSTA 

6,275 

0 

198 

229 

0 

0 

6,046 

EPSRC 

197 

8 

189 

68 

7 

1,062 

14 

Number of 
topics with 
more than 1% 
share not in 
common with 
EC 

0 

0 

1,044 

14 

1437 As mentioned previously, there is a difference in data gathered via the EC beneficiary survey 
and through the text-mining of acknowledgements. Here, the research team has attempted to 
separate the two, with the external comparison relying completely on Scopus/SciVal data. These 
data have been retrieved using an API. Due to time lag, these figures may differ slightly; 
however, this should not affect the overall robustness of the results and analysis. 

1144 

 
 
 
Funder 

Total 
number 
of 
topics 

Number of 
topics that 
overlap with 
EC 

Number of 
different 
topics 

Number of funded topics 
with more than a 1% 
share of worldwide 
output 

NIH 

7,164 

NSF 

1,483 

296 

173 

6,868 

1,310 

2,847 

193 

Number of 
topics with 
more than 1% 
share not in 
common with 
EC 

2,753 

172 

Figure 13. External funder comparison – topics. Source: authors’ own calculations. Note: the dataset can be found at: 
https://doi.org/10.34894/SKYJTH. 

Looking first at productivity, in terms of those topics with a worldwide share of 1% or more of 
funded publications on a specific topic, we find that for 7% of all research topics funded by 
the EC, 1% of the worldwide publications are funded by the EC programmes. Comparing this 
with the NSF, we find that that the NSF funds 13% vs the EC’s 7%, while for the EPSRC this 
share is also 7%. Meanwhile, almost 40% of all topics funded by the NIH fall within this 1%-
or-more category. The NNSFC seems to have rather low share at 3.5%. 

Of specific interest is the similarity between the EC and NNSFC-funded research; all (100%) 
of topics funded by the NNSFC are matched by the EC. Also interesting is the dissimilarity 
between the EC and the EPSRC and JSTA, with which only approximately 4% of research 
topics match. For the NIH and NSF, the corresponding figures are 4% and 11%, respectively. 

1.3.7.  Conclusions 

•  From  the  internal  analysis  of  topical  overlap,  the  present  analysis  finds  that  overlap 
does indeed exist between Horizon 2020’s programme parts and Social Challenges. 
These overlaps differ per topic – which is important to note, as topics that represent a 
more broadly applicable technology (a ‘general-purpose technology’ or GPT) or topics 
that have a certain urgency or can be contributed to by different programme parts and 
Social  Challenges  (such  as  COVID-19)  are  found  to  exhibit  higher  levels  of  overlap 
than topics with a much narrower focus. 

•  The internal analysis also shows that programme parts with a broad focus, such as the 
ERC,  MSCA  and  SEWP,  support  research  on  a  large  number  of  prominent  topics. 
Other  programme  parts  (such  as  Euratom  or  the  LEITs)  and  Social  Challenges  are 
more  specialised  in  their  focus  and  policy  goals.  This  is  mirrored  by  the  topics  they 
fund,  and  their  overlap  with  other  programme  parts  and  Social  Challenges  is  only 
minimal – especially when looking at the publication numbers outside of the funding 
programme part or SC. One can note that there usually is a clear ‘home’ for specific 
topics, with only a small number of publications appearing in other clusters.  

•  The  external  analysis  shows  that  EC-funded  research  is  in  line  with  those  topics 
researched nationally in the broader EU. EC-funded research also shows a high degree 
of overlap with Chinese research – both nationally, and with that funded by the National 
Natural Science Foundation of China (NNSFC) – as well as with other third countries, 
such as the United States and Japan. A certain differentiation effect can be noted at 
national  level,  but  this  is  to  be  expected,  and  is  in  line  with  the  smart  specialisation 
strategies employed at national level. 

1145 

 
 
•  The external analysis clearly shows the effect of Brexit on the topics funded. The UK 
has  been  an integral  part of  the  European  Research  Area  and  environment,  both  in 
terms  of  strengthening  topics  that  are  of  importance  to  the  EU-27  and  the  EC,  and 
relying on EU research to fill its domestic gaps.  

•  The external analysis also indicates that with respect to research funded in topics in 
the field of Health Science, the EC appears to be lagging in terms of topics covered, 
as well as in the productivity of those topics funded. 

1.3.8.  Lessons learned 

•  The ERC and MSCA produce the bulk of Horizon 2020’s new knowledge on a large 
number of topics, while other programme parts and SCs aim to boost competitiveness 
or to solve specific societal challenges. Overall, the focus and funding of the different 
programme parts and Social Challenges are proportionate (in terms of the distribution 
of funding between topics),  and cover programme-specific topics. The exceptions to 
this are Health Sciences and, to some extent, also SSH-related topics. 

•  The  above  gap  in  the  EC’s  funding  portfolio  in  terms  of  Health  Sciences-related 
research  may  require  attention,  especially  as  this  field  is  the  core  of  the  first  Social 
Challenge  (3.1).  Compared  with  the  US  and  China,  the  EC  lags  behind  in  terms  of 
topics  covered  and  the  funding  given  to  those  topics  that  are  covered.  If  the  EC  is 
interested  in  competing  in  this  area,  additional  funding  for  research  in  these  topics 
would be advisable. Of course, this gap can also be attributed to smart specialisation 
strategies at national and supra-national (regional) level, whereby countries or country 
groups select a number of specific/priority areas for investments in basic and applied 
knowledge, while keeping this funding focused on national strengths (for instance, in 
terms of resources) as well as comparative advantages in terms of access to skilled 
manpower and knowledge capital. 

• 

Looking at excellent science and the data from the present analysis, closer cooperation 
with the UK in terms of research activities is also desirable. The re-admission of the UK 
to Horizon Europe and the ERA as an Associate Member has thus been a logical step 
(for both sides). A further strengthening of this research cooperation would call for an 
extension of this association beyond 2027. 

1.4.  Counterfactual  analysis/econometric  modelling  –  novelty  and 

disruptiveness (ERC and MSCA-IF) 

The Evaluation study on Excellent Science under Horizon 2020 analysed the role of ERC 
and MSCA-IF grants/fellowships under Horizon 2020 in promoting the productivity, visibility 
and careers of scientists who applied for funding. Among the MSCA’s funding programmes, 
both the Evaluation study on Excellent Science under Horizon 2020 and the present report 
focus  on  MSCA  Individual  Fellowship  (MSCA-IF)  under  Horizon  2020.  For  simplicity, 
throughout  the  rest  of  this  annex,  these  fellowships  are  referred  to  simply  as  ‘MSCA’. 
Standard bibliometric indicators were used to assess productivity and visibility: namely, the 
number of publications and citations. The study team also investigated three aspects of a 
scientist’s career, i.e. whether MSCA or ERC funding allowed funded scientists to 1) improve 
their ability to raise further competitive funds; 2) expand their research portfolio by publishing 
on topics that differ from their past topics; or 3) move to more highly regarded organisations, 
compared  with  other  similar,  non-funded  applicants.  The  research  team  did  not  find  any 
significant  effects  of  ERC  and  MSCA  funding  on  the  productivity  or  visibility  of  funded 

1146 

 
 
applicants in comparison to similar, non-funded peers, five years after funding. Some impacts 
were found on MSCA applicants’ careers: recipients of an MSCA individual fellowship are 
more  likely  to  access  further  EC  funding  than  similar  applicants  who  did  not  receive  the 
fellowship, three to five years after their successful MSCA application. No significant effects 
of  ERC/MSCA  funding  were  seen  on  other  aspects  of  the  funded  applicants’  careers 
compared  with  similar  applicants  who  did  not  receive  ERC/MSCA  funding,  including  with 
respect to the probability that funded scientists will move on to research topics on which they 
had not conducted research during their pre-application career.  

But  what  about  the  novelty  of  the  published  research  with  respect  to  existing  global 
knowledge? Novel and/or disruptive research generates findings that are useful in advancing 
the frontiers of knowledge and are, therefore, a component of scientific excellence (Ioannidis, 
2011; Jones, 2021; Jones et al., 2020) – even if novel and disruptive research is usually less 
successful in the short term, as measured by the number of citations it receives. For instance, 
while Carayol et al. (2019) find that papers which combine new terms tend to be more often 
cited, Wang et al. (2017) show that novel research has a higher variability in its citations, is 
published in lower-impact factor journals, and only begins to receive more citations several 
years after publication. It is therefore important to understand whether, despite not producing 
more citations, researchers funded by the ERC/MSCA produce outputs that are more novel 
in comparison to similarly excellent scientists that did not receive funding. 

One of the benefits of ERC/MSCA grants is to allow the scientists funded to focus on research 
for a number of years, by providing support for human and physical capital resources. These 
are exceptional  conditions  in  academia,  and  may  allow the  scientists  awarded  to  work  on 
more experimental or disruptive research that they would not/could not carry out without such 
funding (Aghion et al., 2008; Azoulay et al., 2011; Wang et al., 2018). Moreover, ERC/MSCA 
grant  schemes  appear  promising  in  contributing  to  novel  science  because  they  target 
individuals or small teams, who are in a better position  to produce breakthrough  research 
than larger project consortia, as recent evidence suggests (Lin et al., 2022).  

This  report therefore  studies  the  effect  of  ERC/MSCA  funding  under  Horizon  2020  on  the 
publication  of  novel  or  disruptive  research1438,1439.  Does  access  to  ERC  or  MSCA  funding 
enable researchers to engage in more experimental or disruptive research than they would 
in  the  absence  of  such  funding?  Do  to  the  fact  that  it  is  impossible  to  know  what  a  given 
scientist would have done in the absence of ERC/MSCA funding, the study team follows the 
literature in comparing funded ERC/MSCA scholars with researchers who have applied and 
who have a similar likelihood of being funded (within the given discipline) (Ghirelli et al., 2023; 
Jacob and Lafgren, 2011; Ayoubi et al., 2019; Jaffe, 2002). In other words, they are compared 
with  researchers  who  possess  similar  potential  to  produce  novel  and  disruptive  scientific 
output, but who have to do so using a different source of funding, if at all. Using a sample of 
scientists  is  used  that  are  similar  to  the  awarded  as  a  control  has  the  advantage  that 
considering  individuals  with  the  same  propensity  to  apply  for  the  schemes,  and  the  same 
characteristics with regard to attracting funding. Some of these are awarded (treated); others 
are not (non-treated). It is possible that those applicants who are not awarded ERC/MSCA 
funding  will  find  other  sources  of  funding.  However,  only  a  few  European  countries  offer 
funding  schemes  for  individual  researchers  that  are  similar  to  the  ERC/MSCA  in  terms  of 
design  and  amount  of  funding  (others  will  need  to  apply  to  schemes  that  fund  projects 

1438 For the ERC, the present study focuses on Advanced (ERC-ADG), Consolidator (ERC-
COG), and Starting (ERC-STG) grants, and excludes applicants who applied to the following 
programmes: Synergy grants (ERC-SyG), Proof of concept (ERC-PoC and ERC-PoC-LS), and 
Low value grant (ERC-LVG). 
1439 For the MSCA, the present study focuses on both European (MSCA-IF-EF) and Global 
(MSCA-IF-GF) fellowships under the MSCA Individual Fellowship programme. 

1147 

 
 
 
targeting research consortia). In addition, results from the survey carried out in the Evaluation 
study on Excellent Science under Horizon 20201440 suggest that 61% of ERC applicants who 
did not receive funding could not implement the activities proposed in their project (30%), or 
had to change them substantially (31%). Only 13% of non-funded ERC applicants managed 
to carry out their proposed project with no or minimal changes. For the MSCA, 50% of non-
funded  applicants  could  not  carry  out  their  proposed  research;  15%  had  to  adapt  their 
proposals significantly, and only 19% managed to carry out the proposed project with no or 
minimal changes. In summary, the design used in the present study allows us to assess the 
additional value of ERC/MSCA funding in producing more novel and disruptive research vis-
à-vis researchers who are similarly talented and who may access other sources of funding. 
Thus, the findings tell us something about the additional effect ERC/MSCA funding has with 
respect to alternative sources of funding for similar applicants. 

To  measure  novelty  and  disruptiveness,  the  study  team  follows  the  literature  that  uses 
citation-based metrics.1441 The advantage of this is that these are relatively simple measures 
to interpret. The disadvantage is that they rely on citation behaviour1442, and that novelty may 
be correlated with multidisciplinarity. Future work may dig deeper into measures of novelty 
and disruptiveness that capture alternative, more specific aspects in relation to expanding 
the frontiers of scientific. Several variations of the disruptiveness measures are computed, to 
account for sensitivity in their construction. These measures are discussed at greater length 
in Section 4.2, following the presentation of the data. 

1.4.1.  Data 

The present study uses CORDA data to retrieve information about applicants, the evaluation 
of  their  projects,  and  funding.  To  retrieve  information  about  applicants’  publications, 
OpenAlex data were used1443. 

To identify publications generated by applicants, the study team developed a procedure to 
identify  high-confidence  matches  between  the  applicants  listed  in  CORDA  and  authors  in 
OpenAlex, based on the precision of their author identifiers: unique identifiers such as ORCID 
were  assigned  higher  confidence  than  more  ambiguous  identifiers,  such  as  two-word 
common names. Records with a combination of matches (e.g. ORCID and Scopus ID and 
Name) were also assigned higher confidence. Lastly, manual checks were performed on a 

1440 See: European Commission, Directorate-General for Research and Innovation, Budraitis, M., 
Pranckevičius, P., Dėlkutė, R. et al., Evaluation study of the European framework programmes 
for research and innovation for excellent science – Horizon 2020 – Evaluation study on Excellent 
Science under Horizon 2020 final study report, Luxembourg: Publications Office of the European 
Union, 2023, available at: https://data.europa.eu/doi/10.2777/967813. 
1441 For example, Uzzi, B., Mukherjee, S., Stringer, S., & Jones, B., Atypical combinations and 
scientific impact. Science, 342 (6157):468–472, 2013; Wang, J., Veugelers, R., & Stephan, P., 
Bias against novelty in science: A cautionary tale for users of bibliometric indicators. Research 
Policy, 46(8):1416–1436, 2017; Funk, R.J., & Owen-Smith, J., A dynamic network measure of 
technological change. Management Science, 63(3):791–817, 2017; Wu, L., Wang, D., & Evans, 
J.A., Large teams develop and small teams disrupt science and technology. Nature, 
566(7744):378–382, 2019; Bornmann, L., Devarakonda, S., Tekles, A., & Chacko, G., Disruptive 
papers published in scientometrics: meaningful results by using an improved variant of the 
disruption index originally proposed by Wu, Wang, and Evans (2019). Scientometrics, pp. 1–7, 
2020. 
1442 The present analysis measures these by discipline, to control for different practices across 
disciplines. 
1443 https://openalex.org. The study team chose to use OpenAlex, because the data are open, 
and because it offers a broader coverage than proprietary databases. Information from 
OpenAlex was complemented with information from Scopus for some variables. 

1148 

 
 
 
sample of matches to ensure their consistency. Of the 43,644 ERC applicants, a large sample 
of 31,299 (72%) were identified with high confidence in OpenAlex. Of the 48,822 applicants 
to MSCA, a large sample of 35,223 (72%) were identified with high confidence in OpenAlex.  

Following the literature on impact evaluations of research funding, to identify the causal effect 
of each funding programme, the present analysis restricts its sample to ‘similar’ applicants in 
order  to  control  for  non-observable  features  of  the  applicants  that  could  influence  their 
outcomes in terms of novelty and disruptiveness (Ghirelli et al., 2023; Jaffe, 2002). ‘Similar’ 
is  defined  in  slightly  different  ways  for  the  MSCA  and  ERC,  due  to  differences  in  the 
underlying  data  available  in  CORDA.  For  the  MSCA,  we  have  information  about  each 
applicant's score and can, therefore, focus on applicants that are just above and below the 
funding  threshold  (by  panel  and  year).  The  funding  threshold  is  given  by  the  score  and 
position of the last funded applicant after ranking them by score. That is, MSCA applicants 
that  have  been  assessed  as  very  similar by  their  panel in  terms  of  overall  quality  (and  all 
considered fundable). For the ERC, only information about the score category was available. 
The study team included applicants who were funded, as well as those who were considered 
fundable in the second step of the evaluation but were either included in the reserve list or 
not funded due to lack of funds (scored A). All applicants who were rejected at either step 
one (C) or two (B) were excluded. Because information on individual scores was not available 
for the for ERC, and the study team were therefore unable to select those just below or above 
the funding threshold, wider variation can be seen in the characteristics of the applicants (as 
assessed  by  the  panel),  which  may  influence  the  output  variable.  Details  about  how  the 
present  analysis  defines  the  sample  of  ‘similar’  applicants  and  controls  for  other 
characteristics that might still influence the outcomes after funding, are provided in the two 
sections below on the ERC and MSCA. 

Out of the population of 9,472 ‘similar’ ERC applicants in category A (funded or on the reserve 
list),  a  large  sample  of  8,328  (88%)  were  identified  with  high  confidence  in  OpenAlex  as 
authors of publications. Of the population of 5,370 ‘similar’ MSCA applicants (funded and not 
funded), a large sample of 4,272 (80%) were identified with high confidence in OpenAlex as 
authors  of  publications.  The  large  matched  samples  for  the  ERC  and  MSCA  are 
representative of the population by disciplinary panel, gender and country. For the MSCA, 
81% of those funded were matched with high confidence, and 78% of non-funded applicants 
were matched with high confidence, suggesting that the matching introduced no bias. For the 
ERC, the  difference  is  slightly  larger:  92% of  those  funded  were  matched,  compared  with 
81%  of  non-funded  applicants,  suggesting  that  non-funded  applicants  are  slightly  under-
represented. Because the matched sample is representative of the population by disciplinary 
panel, gender, country and programme (Starting, Consolidating and Advanced), and because 
all applicants ranked A must show a competitive level of publication and citation records, we 
see no remaining reason to believe that the 10 percentage point difference in the matching 
is due to some systematic bias, for instance in the reporting the ORCID IDs or in publishing. 
The  difference  is  likely  to  be  driven  by  the  availability  in  the  CORDA  data  of  publication 
records  only  for  funded  applicants.  To  improve  the  matching,  publications  were  used,  so 
funded applicants have a higher probability of being matched with authors in OpenAlex due 
to  the  construction  of  the  procedure  used.  However,  this  is  unrelated  to  the  publication 
records of the applicants1444. 

1444 For more details on the patterns of non-matched applicants and the resulting sample, please 
see Appendix A (Data Appendix) of Annex 8 prepared for the Evaluation study on Excellent 
Science under Horizon 2020 (pp. 1073-1084) in: European Commission, Directorate-General for 
Research and Innovation, Budraitis, M., Pranckevičius, P., Dėlkutė, R., Evaluation study of the 
European framework programmes for research and innovation for excellent science: Horizon 

1149 

 
 
 
 
1.4.2.  Novelty and disruptiveness 

The present study seeks to analyse the impact of funding on both novelty and disruptiveness, 
using indicators widely used in the literature. To identify novelty, the indicator proposed by 
Foster et al. (2015)1445 was used; for disruptiveness, various indicators proposed by Bornman 
et al. (2019)1446, as well as Wu and Yan (2019)1447, were used. It is important to stress that 
these  indicators  measure  specific  aspects  of  novelty  and  disruptiveness  that  rely  on  the 
behaviour of scientific communities in citing and combining existing knowledge, as discussed 
below.  

1.4.2.1. 

Novelty indicator 

The  novelty  indicator  identifies  publications  that  cite  documents  that  arise  from  different 
scientific communities, i.e. which combine different topics in novel ways (that have not been 
combined  before).  The  novelty  measure  was  calculated  on  the  basis  of  combinations  of 
journals obtained from the reference lists of the papers in focus. First, the algorithm produces 
a co-occurrence matrix of papers and journals by year. Then, it applies the Louvain network 
community algorithm to identify clusters of journals by year: we can refer to these as scientific 
communities; i.e. groups of scientists that work on similar topics and cite each other (Blondel 
et  al.,  2008).  It  should  be  noted  that  these  communities  are  not  equivalent  to  disciplines. 
Some  of  these  communities  may  overlap  in  terms  of  subdisciplines,  while  others  may  be 
interdisciplinary.  The  communities  identified  depend  on  how  researchers  have  cited  each 
other  between  1998  and  2021.  Therefore,  while  the  two  are  related,1448  the  measure  of 
novelty  that  we  use  is  not  equivalent  to  interdisciplinarity.  It  is  equivalent  to  combining 
scientific knowledge in different ways that may yield novel directions for scientific knowledge 
according to the literature (e.g. Strumsky and Lobo, 2015; Lin et al, 2022).1449 One possible 
concern with this combinatorial view of novelty is that excludes novelty resulting from single 
discoveries. However, Strumsky and Lobo (2015) suggest that the latter type of novelty are 
limited in number, and have decreased over time. It should be acknowledged, however, that 
there are multiple ways of measuring the novelty of research outputs.  

Foster’s  novelty  indicator  is  computed  as  the  number  of  novel  inter-community  journal 
combinations as a proportion of the total number of journal combinations, as follows: 

𝐹𝑁 =

𝑁𝑜𝑣𝐶𝑜𝑚𝑏
𝐶𝑜𝑚𝑏

2020: annexes, Luxembourg: Publications Office of the European Union, 2023, available at: 
https://data.europa.eu/doi/10.2777/353383. 
1445 Foster, J.G., Rzhetsky, A., & Evans, J.A., Tradition and innovation in scientists’ research 
strategies. American Sociological Review, 80(5):875-908, 2015. 
1446 Bornmann, L., Devarakonda, S., Tekles, A., & Chacko, G., Do disruption index indicators 
measure what they propose to measure? the comparison of several indicator variants with 
assessments by peers. 
1447 Wu, Q. & Yan, Z., Solo citations, duet citations, and prelude citations: New measures of the 
disruption of academic papers. arXiv preprint, arXiv:1905.03461, 2019. 
1448 Fontana, M., Iori, M., Montobbio, F., & Sinatra, R., New and atypical combinations: An 
assessment of novelty and interdisciplinarity. Res. Policy, 49, 104063, 2020, 
doi:10.1016/J.RESPOL.2020.104063. 
1449 Strumsky, D., & Lobo, J., Identifying the sources of technological novelty in the process of 
invention. Res. Policy, 44, 1445-1461, 2015, doi:10.1016/J.RESPOL.2015.05.008; Lin, Y., 
Evans, J.A., & Wu, L., New directions in science emerge from disconnection and discord. J. 
Informetr. 16, 101234, 2022, doi:10.1016/J.JOI.2021.101234. 

1150 

 
 
 
 
where FN is our measure of Foster novelty; NovComb is the number of novel combinations 
of journals; and Comb is all combinations of journals. The indicator ranges between 0 and 
1, with 1 being the highest level of novelty. 

1.4.2.2. 

Disruptiveness indicator 

The study team have computed several correlated indicators of disruptiveness. All of these 
are based on the same rationale: a publication is disruptive if a large number of subsequent 
publications that cite it, but do not cite its references. In other words, scientists who cite a 
given paper that is disruptive do not feel the need to cite its references because the disruptive 
work has opened up a new scientific trajectory, which differs from the trajectory from which it 
has  developed.  In  other  words,  a  disruptive  paper  acts  a  knowledge  gateway  between 
existing  and  new  knowledge.  Differently  from  the  measure  of  novelty,  the  measure  of 
disruptiveness does not rely on communities. Similarly to the novelty measure, it does rely 
on  citation  practices,  which  may  differ  by  discipline.  To  address  this,  for  each  paper,  we 
compute the novelty and disruptiveness of a publication within its main discipline.  

Figure 14 below illustrates different measures of disruptiveness based on the same principle. 
The variables used in Figure 14 are defined as follows:  

• 
• 

• 

I: number of papers that cite only the paper in focus, and none of its references; 
J: number of papers that cite the paper in focus, and at least one of its 
references; 
J5: number of papers that cite the paper in focus, and at least five of its 
references; 

•  K: number of papers that cite at least one of the references of the paper in focus, 

• 

but not the paper itself; 
JxC: number of common references between the citing papers and the paper in 
focus. 

Figure 14. Related measures of disruptiveness 

1151 

 
 
 
 
Each of the above, related measures are explained below: 

DI1: this indicator was proposed by Wu et al. (2019) and measures the extent to which a 
paper is focus is more recognised in the literature than its references (i.e. cited in isolation, 
rather than together with its references). According to Bornmann et al. (2019), ‘High positive 
values of this indicator should be able to point to disruptive research; high negative values 
should reflect developmental research’ (p. 7). 

DI1NoK: because the variable K in the calculation of DI1 is often very large, leading to very 
small numbers in the indicator (affecting its ability to discriminate between disruptive and 
non-disruptive research), Wu & Yan (2019) proposed removing K from the equation. High 
positive values in this indicator should reflect disruptive research. 

DI5: this is a version by Bornman et al. (2019) of the DI1 indicator, which requires a minimum 
number  of  five  references  shared  between  the  paper  in  focus  and  the  citing  paper  to 
calculate J. High positive values should reflect disruptive research. 

DI5NoK:  This  is  a  version  by  Bornman  et  al.  (2019)  of  the  DI1NoK  indicator,  requiring  a 
minimum number of five references shared between the paper in focus and the citing paper 
to calculate J, and removing K. High positive values should reflect disruptive research. 

DeIn: this indicator can be defined as ‘the average number of bibliographic coupling links 
per cited reference of the focal paper’s citing papers’ (Bornmann et al., 2019). In this case, 
the value of DeIn is expected to decrease as the level of disruptiveness of a paper increases. 
DeIn measures how much a paper depends on earlier work, as opposed to being disruptive. 

For the present analysis, the novelty and disruptiveness indicators above were calculated 
using all publications in the OpenAlex database up to 20211450. Following this calculation, 
we identified the range of values for the top quartile and the top decile for each discipline, 
year and indicator. The number of papers in the top quartile and decile for ERC and MSCA 
applicants were calculated, normalised by discipline and year. The share of papers in the 
top  quartile  and  decile  of  the  novelty  and  disruptiveness  distribution  for  each  ERC  and 
MSCA applicant were also calculated.  

1.4.3.  ERC and MSCA applicants compared with European and USA researchers 

Figure  15  and  Figure  16  compare  the  novelty  of  the  publications  published  by  ERC  and 
MSCA applicants, respectively, with publications published by all researchers globally (who 
have  at  least  one  publication  in  OpenAlex),  based  in  the  USA  and in  Europe  (EU-27  plus 
UK). First, we computed the Foster novelty score for each publication in OpenAlex. Next, we 
computed in which percentile of the Foster novelty score each publication falls, by year and 
field of study (FOS level 0 in OpenAlex). Publications in the first percentile constitute the 1% 
publications in the whole of OpenAlex with the lowest Foster novelty score; publications in 
the 100th percentile constitute the 1% publications in the whole of OpenAlex with the highest 
Foster novelty score. Lastly, we allocated each publication to USA or European researchers, 
and  to  researchers  who  have  applied  to  the  ERC  and  MSCA.  For  the  ERC,  we  further 
distinguish between ERC_A: publications by applicants with the evaluation outcomes ‘Main’ 
(funded), ‘Reserve’ in Step 2, and ‘No Money’ in Step 2; ERC_B: publications by applicants 
with  the  evaluation  outcomes  ‘No  Money’  in  Step  1  or  ‘No_Money’  in_Step_2;  ERC_C: 
publications by applicants with the outcome ‘Rejected’ in Step 1. For the MSCA, we further 
distinguished between MSCA_A1: publications by funded applicants who obtained a score 

1450 For disruptiveness, the study team used only publications with at least 10 references and 10 
citations. 

1152 

 
 
 
above  the  ‘highest  score’  of  our  restricted  sample  of  similar  applicants  (see  definition  in 
Section  4.4);  MSCA_A2:  publications  of  the  sample  of  similar  funded  applicants  in  our 
matched sample, who obtained a score around the threshold (see definition in Section 4.4); 
MSCA_B1:  publications  by  non-funded  applicants  in  our  matched  sample,  who  obtained 
scores around the funding threshold (see definition in Section 4.); MSCA_B2: publications by 
applicants who obtained a score above 70, but were not funded; and MSCA_C: publications 
by applicants who obtained a score below 71 and were not funded. Lastly, for each of the 
above categories of scientists, we computed the share of their publications that fall into each 
of  the  10  deciles  of  the  Foster  novelty  score  distribution.  The  first  decile  comprises  those 
publications  with  the  lowest  Foster  novelty  score;  the  10th  decile  comprises  those 
publications with the highest Foster novelty score. 

By design, the share of scientists that fall into each decile is 10%, and constitutes the baseline 
for reference. Overall, we find that, averaged across all disciplines, scientists in the USA and 
Europe (inclusive of the UK) have published a higher share of novel publications than the 
rest of the world. Almost no difference is observed between the USA and Europe (including 
the UK). Among these groups, publications by all applicants to the ERC (funded and non-
funded) are on average more novel than the average for the USA and for Europe (including 
the UK). ERC applicants have a lower share of publications in the first three deciles compared 
with the averages for researchers in US and Europe (including the UK), and a higher share 
in all higher deciles. Among ERC applicants, we find little difference between those who were 
funded (ERC_A) and those who were dropped either in the first evaluation phase (ERC_C) 
or  second  evaluation  phase  (ERC_B).  If  anything,  ERC  applicants  who  did  not  make  it 
through the first phase actually show publications in the top deciles of the novelty distribution. 

The results for the MSCA are very similar: applicants had a lower share of publications with 
lower  novelty  scores,  and  more  publications  with  high  novelty  scores  compared  with  the 
average scientist in the US and Europe (including the UK). 

Results for both the ERC and MSCA differ when split according to discipline, but all convey 
the  same  broad  message:  ERC  and  MSCA  applicants  produce,  on  average,  more  novel 
research than the average researcher in the USA or Europe (including the UK). 

Of course, these results cannot be taken as an indicator of the impact of ERC and MSCA 
funding, due to self-selection and endogeneity. Thus, we next use these indicators to analyse 
if  MSCA  and  ERC  scientists  who  were  funded  under  Horizon  2020  published  even  more 
novel and disruptive documents following their funding, than similar non-funded colleagues 
who were close to the funding threshold (and who also published more novel research than 
the European average (including the UK). 

1153 

 
 
Figure 15. Percentage of papers, ordered by Foster’s novelty decile, produced by ERC applicants, compared with global, USA, and European researchers. 

Note: the figure reports the distribution of publications by Foster’s novelty decile from all scientists located anywhere in the world (Global), the US (USA), Europe (EU), and from scientists that 
have applied for ERC funding (ERC). We first compute the Foster novelty score for each publication in OpenAlex. Next, we computed in which percentile of the Foster novelty score of each 
publication falls by year and field of study (FOS level 0 in OpenAlex). Publications in the first percentile constitute the 1% publications in the whole of OpenAlex with the lowest Foster novelty 
score; publications in the 100th percentile constitute the 1% publications in the whole of OpenAlex with the highest Foster novelty score. Lastly, we allocated each publication to one of the 
following categories, and computed for each category the share of publications that fall into each of the 10 deciles of the Foster novelty score distribution. Global publications: all publications in 
OpenAlex – distributed equally across deciles by construction (10% in each decile), and shown as a point of reference; EU: all publications by authors affiliated to one of the 27 countries of the 
European Union or the UK; US: all publications by authors affiliated to the USA; ERC: total number of publications by ERC applicants; ERC_A: publications by applicants with evaluation the 
outcomes ‘Main’ (funded), ‘Reserve’ in Step 2, and ‘No Money’ in Step 2; ERC_B: publications by applicants with the evaluation outcome ‘No Money’ in Step 1; ERC_C: publications of 
applicants with the outcome ‘Rejected’ in Step 1. The first decile comprises those publications with the lowest Foster novelty scores; the 10th decile comprises those publications with the 
highest Foster novelty score. 

1154 

 
 
 
Figure 16. Percentage of papers, ordered by Foster’s novelty decile, produced by MSCA applicants, compared with global, USA and European researchers 

Note: the figure reports the distribution of publications by Foster’s novelty decile from all scientists located anywhere in the world (Global), the US (USA), Europe (EU), and from scientists that 
have applied for MSCA funding (MSCA). We first compute the Foster novelty score for each publication in OpenAlex. Next, we compute in which percentile of the Foster novelty score each 
publication falls by year and field of study (FOS level 0 in OpenAlex). Publications in the first percentile constitute the 1% of publications in the whole of OpenAlex with the lowest Foster novelty 
scores; publications in the 100th percentile constitute the 1% of publications in the whole of OpenAlex with the highest Foster novelty scores. Lastly, we allocate each publication to one of the 
following categories, and compute for each category the share of publications that fall into each of the 10 deciles of the Foster novelty score distribution. Global publications: all publications in 
OpenAlex – distributed equally across deciles by design (10% in each decile), and shown as a point of reference; EU: all publications by authors affiliated to one of the 27 countries of the 
European Union or the UK; US: all publications by authors affiliated to the USA; MSCA total number of publications by MSCA applicants; MSCA_A1: publications by funded applicants who 
obtained a score above the ‘highest score’ of our restricted sample of similar applicants (see definition in Section 4.4); MSCA_A2: publications by the sample of ‘similar’ funded applicants in our 
matched sample, who obtained a score around the threshold (see definition in Section 4.4); MSCA_B1: publications by non-funded applicants in our matched sample, who obtained scores 
around the funding threshold (see definition in Section 4.); MSCA_B2: publications by applicants who obtained a score above 70, but were not funded; MSCA_C: publications by applicants who 
obtained a score below 71 and were not funded. The first decile comprises those publications with the lowest Foster novelty scores; the 10th decile comprises those publications with the 
highest Foster novelty scores.

1155 

 
 
 
1.4.4. 

Impact of MSCA funding on novelty and disruptiveness 

1.4.4.1. 

Data and empirical strategy 

To evaluate the impact of an MSCA-Individual Fellowship under Horizon 2020, we apply a 
regression  discontinuity  design  (RDD)  to  those  applicants  falling  into  two  key  categories: 
funded  and  reserve.  This  approach  enables  us  to  compare applicants  who  were  awarded 
with applicants who could have been funded. However, our data shows a lack of a clear-cut 
discontinuity,  meaning  that  we  observe  different  funding  outcomes  for  applicants  with  the 
same evaluation scores. To address this issue, in line with previous studies (Howell, 2017; 
Zhao and Ziedonis, 2012; Baruffaldi et al, 2020), we employed a fuzzy RDD. 

To ensure we compared ‘similar’ applicants (with a similar probability of being funded, and a 
similar  scientific  trajectory)  within  the  same  cohort1451,  we  initially  sorted  all  applicants  in 
ascending order based on their scores. We then determined the ‘cut-off score’ as the score 
of the non-funded applicant with reserve status who achieved the highest score. The ‘lowest 
score’ was the score of the non-funded applicant (in the same year and panel, i.e. cohort) 
with reserve status who achieved the lowest score. The ‘highest score’ was the sum of the 
cut-off score and the difference between the cut-off and the lowest score (in the same year 
and panel, i.e. cohort). Therefore, the distance between the cut-off and the lowest and highest 
scores are the same. Next, we selected all non-funded applicants who fall within the range 
between  the  lowest  score  and  the  cut-off  score  and  who  were  fundable  (reserve  or  no-
money), as well as all funded applicants who fall within the range between the cut-off score 
and the highest score. In other words, we encompass all applicants falling within the same 
range, both below and above the cut-off score, with the range being defined by the cut-off 
score  and  the  lowest  score  that  was  considered  fundable.  From  this  restricted  sample  of 
‘similar’  applicants,  we  remove  the  outliers:  all  applicants  whose  number  of  novel  and 
disruptive publications at the time of application is below the first quartile minus 1.5 times the 
interquartile range (Q1 - 1.5*IQR) and above the third quartile plus 1.5 times the interquartile 
range  (Q3  +  1.5*IQR)1452  following  the  funding.  We  estimated  the  impact  on  each  period 
separately  to  assess  any  difference  as  the  applicants  progressed  in  their  careers.  We 
removed the first year after the application to exclude publications already in the pipeline prior 
to the application, and thus not resulting from the funding. 

Our sample of ‘similar’ applicants with observed novelty outputs between two and five years 
after funding comprised 3,688 applicants (1,828 of whom were funded, and 1,860 of whom 
were not funded). For disruptiveness outputs between two and five years after funding there 
were 3,172 applicants (1,559 of them funded and 1,613 of them not funded)1453. 

The table below describes the variables used in the estimations, while Table 44 and Table 
45  provide  descriptive  statistics  for  those  applicants  falling  into  our  restricted  sample  of 
‘similar’ applicants for the novelty and disruptiveness measures, respectively. Applicants who 

1451 i.e. those who applied to the same programme and year and were allocated to the same 
panel. 
1452 For outlier cleaning, we focused only on those applicants with at least one novel (or 
disruptive) publication, as the sample is dominated by applicants with zero novel (or disruptive) 
publications. So, focusing on those applicants with at least one novel/disruptive publication at the 
time of application, we removed all applicants below the Q1 - 1.5*IQR and above the Q3 + 
1.5*IQR. 
1453 For novelty and disruptiveness measures, there were different numbers of observations due 
to the publication data available for each measure. 

1156 

 
 
 
were not funded correspond to those applicants who were just below the cohort-specific cut-
off points, while funded applicants are those who were just above the cohort-specific cut-offs. 

Variable 

Description 

Funding status  Equal to 1 if the applicant has obtained a grant 

Score 

The score that the applicant received, on a scale from 85.4 to 96.4 

Cut-off 

Cohort-specific (i.e. panel- and year-specific) cut-off points, which are the 
score of the last non-funded person within each cohort 

Distance 

The distance of the applicant’s score from the cohort-specific cut-off point 

Foster’s 
novelty 2 
years (3, 4, 
and 5 years) 

Share of publications in the top 10% of all publications in OpenAlex (by 
discipline and year) by the Foster’s Novelty score from the first year to the 
second (third, fourth, and fifth) year after the application  
(FosterNovelty_2y = number of publications in the top 10% by Foster’s 
Novelty two years after the application / the total number of publications two 
years after the application) (log-transformed) 

Disruptiveness 
DI1 2 years (3, 
4, and 5 
years)  

Share of publications in the top 10% out of all publications in OpenAlex (by 
discipline and year) using the disruptiveness measure DI1 from the first year 
to the second (third, fourth, and fifth) year after the application  
(DI1_2y = number of publications in the top 10% by DI1 two years after the 
application / the total number of publications with at least 10 references and 
citations two years after the application) (log-transformed) 

Gender 

Equal to 1 if the applicant is male 

Academic 
seniority 

Logarithm of the researcher’s total number of productive years at the time of 
application  
(Academic seniority = year of the call – year of the first publication + 1) 

Publication 
history 

Logarithm of the number of average yearly publications at the time of 
application 

Citation 
history 

Logarithm of the number of average yearly citations at the time of 
application 

Novelty history   Logarithm of the share of average yearly publications in the top 10% by 

Foster’s Novelty at the time of application 
(Novelty History = (number of publications in the top 10% by Foster’s 
Novelty at the time of application / the total number of publications at the 
time of application) / (Academic Seniority)) 

Disruptiveness 
history 

Logarithm of the share of average yearly publications in the top 10% by 
disruptiveness measure DI1 at the time of application 
(Disruptiveness History = (number of publications in the top 10% by DI1 at 
the time of application / the total number of publications with at least 10 
references and citations at the time of application) / (Academic Seniority)) 

1157 

 
 
Variable 

Description 

Country 

Nationality of the applicant at the time of application 

Year 

Year of the call 

Panel 

Thematic category under which the applicant applied 

Table 43. Variables used in models on MSCA and variable descriptions 

The table below reports the average, median, standard deviation, minimum and maximum 
number and  share of  novel documents  (those  in  the  top  10% novelty  score)  published  by 
scientists who were funded or not funded by the MSCA, between two and five years after 
application. The number of novel publications is the number of publications that are among 
the top 10% most novel documents in the whole of the OpenAlex repository in a given year 
and discipline. The share is that number divided by all documents published by the scientist 
and available in OpenAlex. 

Funded 

Obs. 

Mean 

Median 

SD 

Min 

Max 

Academic 
seniority 

Publication 
history 

1,828 

7.379 

7 

4.465 

1,828 

2.701 

2.167 

2.673 

Citation history 

1,828 

32.882 

15.268 

88.624 

Foster novelty 
history 

Foster novelty 
2y_count 

Foster novelty 
2y_share 

Foster novelty 
3y_count 

Foster novelty 
3y_share 

Foster novelty 
4y_count 

Foster novelty 
4y_share 

1,828 

0.010 

1,828 

0.336 

1,828 

0.053 

1,388 

0.492 

1,388 

0.052 

895 

0.648 

895 

0.054 

0.037 

0.817 

0.142 

1.061 

0.133 

1.229 

0.126 

0 

0 

0 

0 

0 

0 

0 

1158 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

25 

64.167 

2,596.600 

1.000 

5 

1 

6 

1 

6 

1 

 
 
 
Foster novelty 
5y_count 

Foster novelty 
5y_share 

659 

0.739 

659 

0.053 

0 

0 

1.361 

0.124 

0 

0 

6 

1 

Not funded 

Obs. 

Mean 

Median 

SD 

Min 

Max 

Academic 
seniority 

Publication 
history 

1,860 

7.540 

7 

4.443 

1,860 

2.879 

2.200 

3.128 

Citation history 

1,860 

34.919 

15.817 

87.448 

Foster novelty 
history 

Foster novelty 
2y_count 

Foster novelty 
2y_share 

Foster novelty 
3y_count 

Foster novelty 
3y_share 

Foster novelty 
4y_count 

Foster novelty 
4y_share 

Foster novelty 
5y_count 

Foster novelty 
5y_share 

1,860 

0.009 

1,860 

0.369 

1,860 

0.051 

1,439 

0.544 

1,439 

0.051 

934 

0.653 

934 

0.047 

704 

0.781 

704 

0.044 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.027 

0.845 

0.134 

1.098 

0.124 

1.179 

0.109 

1.334 

0.102 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

25 

55 

1,722.583 

0.500 

5 

1 

6 

1 

6 

1 

6 

1 

Table 44. Novel publications: MSCA descriptives. Notes: Academic seniority: years since first publication; Publication 
history: number of average yearly publications at the time of application; Citation history: number of average yearly citations 
at the time of application; Foster novelty history: share of novel publications published prior to the year of application; Foster 
novelty #y_count: number of novel documents published # years after the funding application; Foster novelty #y_share: 
share of novel documents published # years after funding application. 

The  descriptive  statistics  already  show  that  there  were  minimal  differences  between  the 
funded  and  non-funded  MSCA  applicants  both  before  (as  is  desirable)  as  well  as  after 
funding. At the time of application, exactly 1% of all documents that had been produced by 
the funded applicants were novel. Non-funded applicants are very close to this, with 0.9% of 
documents being novel. Two and five years after application, 5% of documents published by 

1159 

 
 
 
the funded applicants were novel. For non-funded applicants, the shares are very similar – 
5% and 4.5%, respectively.  

The table below reports the average, median, standard deviation, minimum and maximum 
number and share of disruptive documents published by the scientists funded and not funded 
by MSCA, between two and five years after application. The number of disruptive publications 
is the number of publications that are among the top 10% most disruptive documents in the 
whole of the OpenAlex repository in a given year and discipline1454. The share is that number 
divided by all documents published by the scientist and available in OpenAlex. 

Funded 

Obs. 

Mean 

Median 

SD 

Min 

Max 

Academic 
seniority 

1,559 

7.679 

7 

4.289 

Publication history 

1,559 

2.742 

2.286 

2.085 

Citation history 

1,559 

30.888 

16.400 

56.012 

DI1 history 

1,559 

0.0084 

num_di1_2y 

1,559 

0.104 

share_di1_2y 

1,559 

0.032 

num_di1_3y 

1,183 

0.140 

share_di1_3y 

1,183 

0.034 

num_di1_4y 

748 

0.179 

share_di1_4y 

748 

0.035 

num_di1_5y 

540 

0.207 

share_di1_5y 

540 

0.033 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.0312 

0.384 

0.132 

0.449 

0.130 

0.512 

0.124 

0.573 

0.106 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

25 

27.320 

1,468.167 

0.500 

3 

1 

4 

1 

4 

1 

5 

1 

1454 Using the basic Wu et al. (2019) definition: DI1. 

1160 

 
 
 
 
Not funded 

Obs. 

Mean 

Median 

SD 

Min 

Max 

Academic seniority 

1,613 

7.842 

7 

4.268 

Publication history 

1,613 

2.809 

2.286 

2.113 

Citation history 

1,613 

30.015 

16.938 

48.738 

DI1 history 

1,613 

0.0077 

num_di1_2y 

1,613 

0.109 

share_di1_2y 

1,613 

0.032 

num_di1_3y 

1,242 

0.159 

share_di1_3y 

1,242 

0.039 

num_di1_4y 

798 

0.199 

share_di1_4y 

798 

0.041 

num_di1_5y 

586 

0.253 

share_di1_5y 

586 

0.045 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.0259 

0.414 

0.133 

0.489 

0.138 

0.515 

0.124 

0.614 

0.124 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

25 

20.333 

1,097.16 

0.500 

4 

1 

5 

1 

3 

1 

4 

1 

Table 45. Disruptive publications: MSCA descriptives. Notes: Academic seniority: years since first publication; Publication 
history: number of average yearly publications at the time of application; Citation history: number of average yearly citations 
at the time of application; DI1 history: share of disruptive publications published prior to the year of application; num_di1_#y: 
number of novel documents published # years after funding application; share_di1_#y: share of novel documents published 
# years after funding application. 

The descriptive statistics above already show minimal differences between funded and non-
funded  MSCA  applicants,  both  before  (as  is  desirable)  as  well  as  after  funding.  Of  the 
documents by funded applicants, 0.8% were disruptive publications at the time of application. 
Non-funded applicants are very close, with 0.7% disruptive documents. Two and five years 
after application,  3% of  the  documents  published  by  funded  applicants  are disruptive.  For 
non-funded applicants, the shares are very similar, at 3% and 5%, respectively. 

After the exploratory analysis, an econometrics analysis was implemented that allowed the 
study team to control for the applicants’ characteristics and to identify the causal impact of 

1161 

 
 
 
MSCA  funding  using  a  fuzzy  regression  discontinuity  design  (RDD).  To  arrive  at  an 
estimation, the following two-stage equation model was used:  

Where, in the first-stage, the outcome variable Fundedit is a binary variable that takes the 
value of 1 when the applicant was ‘Funded’ and 0 when they were ‘Not Funded’ in year t; 
I(Scoreit > Cut-offit) takes the value of 1 when the score of the applicant i is above the cohort-
specific cut-off and functions as an instrument for being funded; Distanceit is the distance of 
the applicant i’s score from the cohort-specific cut-off point in year t; Genderi is the gender of 
the  applicant  i  (1  is  male,  0  is  female);  AcademicSeniorityit   is  the  researcher’s  academic 
seniority at the time of evaluation (logarithm of the difference between the year of application 
and  the  first  year  of  publication  +  1);  the  log-transformed  number  of  average  yearly 
publications (log(PublicationHistory)it) and citations (log(CitationHistory)it) before application; 
and the log-transformed share of average yearly novel publications before application or the 
log-transformed share of average yearly disruptive publications before application (log(Y)it), 
depending on the outcome variable that is used in the second stage. The study team also 
controlled for unobserved trends, including country (countryi), year (yeart), panel (paneli), and 
cohort (cohortit as yearXpanel) fixed effects. 

In the second-stage, Yit is the log-transformed version of the share of publications in the top 
10% by Foster’s Novelty and the share of publications in the top 10% by the disruptiveness 
measure  DI1  from  the  first year to  the  second,  third,  fourth  and  fifth year after application; 
Fundedit is the main independent variable as our predicted funding variable in the first stage 
and  the  coefficient  of  interest  β  represents  the  effect  of  being  funded;  Distanceit  is  the 
distance of the applicant i’s score from the cohort-specific cut-off point in year t; I(Scoreit > 
Cut-offit) takes the value of 1 when the score of the applicant i is above the cohort-specific 
cut-off  at  time  t;  the  study  team  controlled  for  gender  (1  is  male,  0  is  female)  (Genderi); 
academic seniority at the time of evaluation (logarithm of the difference between the year of 
application  and  the  first  year  of  publication  +  1)  (log(AcademicSeniority)it);  the  log-
transformed  number  of  average  yearly  publications  (log(PublicationHistory)it)  and  citations 
(log(CitationHistory)it) before application; and log-transformed share of average yearly novel 
publications  before  application  when  Yit  is  Foster’s  Novelty  or  log-transformed  share  of 
average  yearly  disruptive  publications  before  application  when  Yit  is  the  disruptiveness 
measure  DI1;  to  control  also  for  unobserved  trends,  the  study  team  included  country 
(countryi), year (yeart), panel (paneli), and cohort (cohortit as yearXpanel) fixed effects. 

1162 

 
 
  
1.4.4.2. 

Results 

Table 46 reports the results for the novelty measure, and Table 47 for the disruptive measure. 

log(FosterNovelty_2y) 

log(FosterNovelty_3y) 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

0.006 

0.007 

0.006 

0.004 

-0.008 

-0.007 

-0.007 

-0.008 

(0.007) 

(0.007) 

(0.007) 

(0.007) 

(0.007) 

(0.007) 

(0.007) 

(0.007) 

-0.003 

-0.002 

-0.002 

0.001 

0.004 

0.004 

0.004 

0.008* 

(0.003) 

(0.004) 

(0.003) 

(0.004) 

(0.004) 

(0.004) 

(0.004) 

(0.004) 

-0.002 

-0.002 

-0.003 

-0.003 

-
0.0004 

-0.001 

-0.002 

-0.003 

(0.003) 

(0.004) 

(0.004) 

(0.004) 

(0.004) 

(0.004) 

(0.004) 

(0.004) 

0.016***  0.016***  0.015***  0.015***  0.015***  0.015***  0.014***  0.015*** 

(0.003) 

(0.003) 

(0.003) 

(0.003) 

(0.003) 

(0.004) 

(0.004) 

(0.004) 

0.003 

0.003 

0.004 

0.003 

-0.003 

-0.002 

-0.002 

-0.004 

(0.004) 

(0.005) 

(0.005) 

(0.005) 

(0.005) 

(0.005) 

(0.005) 

(0.005) 

-0.003* 

-0.003 

-0.004 

-0.003 

-0.002 

-0.002 

-0.004 

-0.003 

(0.002) 

(0.002) 

(0.002) 

(0.002) 

(0.002) 

(0.002) 

(0.002) 

(0.003) 

0.997***  1.002***  0.972***  0.978***  0.902***  0.900***  0.867***  0.873*** 

(0.061) 

(0.062) 

(0.062) 

(0.063) 

(0.066) 

(0.067) 

(0.067) 

(0.068) 

0.002 

-0.002 

-0.001 

-0.008 

0.003 

0.002 

0.002 

-0.007 

(0.005) 

(0.005) 

(0.005) 

(0.006) 

(0.005) 

(0.006) 

(0.006) 

(0.007) 

no 

no 

no 

yes 

yes 

yes 

yes 

no 

no 

yes 

no 

no 

no 

no 

no 

yes 

yes 

yes 

yes 

no 

no 

yes 

no 

no 

Funded 

Distance 

Gender: Male 

log(Academic 
seniority) 

log(Publication 
history) 

log(Citation 
history) 

log(Novelty 
history) 

Distance*I(Score 
> cut-off) 

Country FE 

Years FE 

Panel FE 

1163 

 
 
 
log(FosterNovelty_2y) 

log(FosterNovelty_3y) 

(1) 

no 

(2) 

no 

(3) 

no 

(4) 

yes 

(5) 

no 

(6) 

no 

(7) 

no 

(8) 

yes 

3,688 

3,688 

3,688 

3,688 

2,827 

2,827 

2,827 

2,827 

0.072 

0.072 

0.068 

0.069 

0.067 

0.067 

0.062 

0.064 

35.470 

34.731 

32.470 

32.509 

25.261 

24.361 

22.638 

22.511 

Years X Panel 
FE 

Observations 

R2 

F Statistic 

Robust Standard Errors in Parentheses | *** p-value < 0.001, ** p-value < 0.01, * p-value < 0.05 

Note: the table reports RDD estimates for the share of novel publications two, three, four and five years after application. 
For each outcome variable (i.e. Foster Novelty 2/3/4/5y), we report estimates for a different model in each column (models 
1, 5, 9 and 13: no fixed effects – OLS (ordinary least squares); models 2, 6, 10, and 14: country and years fixed effects; 
models 3, 7, 11, and 15: country and panel fixed effects; models 4, 8, 12, and 16: country and cohort (panelXyear) fixed 
effects). The difference between funded and non-funded applicants was estimated using Eq 1 and Eq 2. The reason behind 
the decrease in the number of observations as we move from publications two years to five years after application is mainly 
because we can observe more applicants for the two-year variable than for three, four or five years, due to the application 
year (i.e. two years consists of 2014, 2015, 2016, 2017, 2018 and 2019 applicants, while five years consists of 2014, 2015 
and 2016 applicants). 

Table 46. Novel publications: impact of MSCA funding 

log(FosterNovelty_4y) 

log(FosterNovelty_5y) 

(9) 

(10) 

(11) 

(12) 

(13) 

(14) 

(15) 

(16) 

-0.011 

-0.010 

-0.009 

-0.007 

-0.021* 

-0.015 

-0.012 

-0.010 

(0.011) 

(0.011) 

(0.010) 

(0.010) 

(0.012) 

(0.012) 

(0.012) 

(0.011) 

0.003 

0.002 

0.00004 

0.003 

0.007 

0.005 

0.004 

0.006 

(0.007) 

(0.007) 

(0.007) 

(0.008) 

(0.008) 

(0.008) 

(0.008) 

(0.008) 

-0.007* 

-0.007 

-0.009* 

-
0.009** 

-
0.012** 

-0.012** 

-0.015*** 

-0.016*** 

(0.004) 

(0.005) 

(0.005) 

(0.005) 

(0.005) 

(0.005) 

(0.005) 

(0.005) 

0.017***  0.017*** 

0.017*** 

0.017*** 

0.010** 

0.011** 

0.010** 

0.010** 

(0.004) 

(0.004) 

(0.004) 

(0.004) 

(0.004) 

(0.005) 

(0.005) 

(0.005) 

Funded 

Distance 

Gender: Male 

log(Academic 
seniority) 

log(Publication 
history) 

-
0.012** 

-0.010* 

-0.011* 

-0.012* 

-
0.014** 

-0.013** 

-0.017** 

-0.018** 

1164 

 
 
 
 
 
log(FosterNovelty_4y) 

log(FosterNovelty_5y) 

(9) 

(10) 

(11) 

(12) 

(13) 

(14) 

(15) 

(16) 

(0.006) 

(0.006) 

(0.006) 

(0.006) 

(0.006) 

(0.007) 

(0.007) 

(0.007) 

-0.001 

-0.002 

-0.003 

-0.002 

-
0.0003 

-0.001 

-0.001 

-0.001 

(0.002) 

(0.002) 

(0.003) 

(0.003) 

(0.003) 

(0.003) 

(0.003) 

(0.003) 

0.709***  0.722*** 

0.695*** 

0.704*** 

0.530*** 

0.547*** 

0.527*** 

0.553*** 

(0.068) 

(0.069) 

(0.069) 

(0.070) 

(0.075) 

(0.077) 

(0.077) 

(0.078) 

0.011 

0.011 

0.015 

0.004 

0.013 

0.014 

0.013 

0.006 

(0.010) 

(0.011) 

(0.011) 

(0.012) 

(0.011) 

(0.011) 

(0.012) 

(0.013) 

no 

no 

no 

yes 

yes 

yes 

yes 

no 

no 

yes 

no 

no 

no 

no 

no 

no 

no 

no 

yes 

no 

yes 

yes 

yes 

yes 

no 

yes 

no 

no 

no 

yes 

no 

no 

log(Citation 
history) 

log(Novelty 
history) 

Distance*I(Score 
> cut-off) 

Country FE 

Years FE 

Panel FE 

Years X Panel 
FE 

Observations 

1,829 

1,829 

1,829 

1,829 

1,363 

1,363 

1,363 

1,363 

R2 

0.067 

0.068 

0.066 

0.067 

0.052 

0.053 

0.054 

0.057 

F Statistic 

16.339 

15.904 

15.209 

15.037 

9.217 

9.018 

9.085 

9.415 

Robust Standard Errors in Parentheses | *** p-value < 0.001, ** p-value < 0.01, * p-value < 0.05 

Note: the table reports RDD estimates for the share of novel publications two, three, four and five years after application. 
For each outcome variable (i.e. Foster Novelty 2/3/4/5y), we report estimates for a different model in each column (models 
1, 5, 9 and 13: no fixed effects – OLS (ordinary least squares) models 2, 6, 10 and 14: country and years fixed effects; 
models 3, 7, 11 and 15: country and panel fixed effects; models 4, 8, 12 and 16: country and cohort (panelXyear) fixed 
effects).  The  difference  between  funded  and  non-funded  applicants  was  estimated  using  Eq  1  and  Eq  2.  The  reason 
behind the decrease in the number of observations as we move from publications two years to five years after application 
is mainly because we can observe more applicants for the two-year variable than for three, four or five years, due to the 
application year (i.e. two years consists of 2014, 2015, 2016, 2017, 2018 and 2019 applicants, while five years consists 
of 2014, 2015 and 2016 applicants). 

Table 46 (cont). Novel publications: impact of MSCA funding  

1165 

 
 
 
 
 
log(DI1_2y) 

log(DI1_3y) 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

0.008 

0.007 

0.007 

0.005 

0.002 

0.001 

0.0001 

-0.003 

(0.007) 

(0.007) 

(0.007) 

(0.007) 

(0.009) 

(0.009) 

(0.009) 

(0.009) 

-0.006* 

-0.008** 

-0.006 

-0.006 

-0.009** 

-0.009* 

-0.006 

-0.004 

(0.004) 

(0.004) 

(0.004) 

(0.004) 

(0.004) 

(0.005) 

(0.005) 

(0.005) 

0.001 

-0.001 

-0.003 

-0.004 

0.002 

0.0004 

-0.002 

-0.003 

(0.004) 

(0.004) 

(0.004) 

(0.004) 

(0.004) 

(0.005) 

(0.005) 

(0.005) 

0.003 

0.004 

0.005 

0.004 

0.003 

0.004 

0.005 

0.005 

(0.004) 

(0.004) 

(0.004) 

(0.004) 

(0.004) 

(0.004) 

(0.005) 

(0.005) 

0.025*** 

0.026*** 

0.025*** 

0.024*** 

0.026*** 

0.025*** 

0.024*** 

0.023*** 

(0.005) 

(0.005) 

(0.005) 

(0.005) 

(0.006) 

(0.006) 

(0.006) 

(0.007) 

-0.003 

-0.004* 

-0.005** 

-0.004* 

-0.005** 

-0.006** 

-0.007** 

-0.007** 

(0.002) 

(0.002) 

(0.002) 

(0.003) 

(0.002) 

(0.003) 

(0.003) 

(0.003) 

0.140** 

0.115 

0.119 

0.110 

0.128 

0.099 

0.080 

0.081 

(0.071) 

(0.073) 

(0.073) 

(0.074) 

(0.079) 

(0.081) 

(0.081) 

(0.082) 

0.006 

0.009 

0.005 

0.007 

0.010 

0.010 

0.005 

0.003 

(0.005) 

(0.006) 

(0.005) 

(0.006) 

(0.006) 

(0.007) 

(0.007) 

(0.008) 

Funded 

Distance 

Gender: Male 

log(Academic 
seniority) 

log(Publication 
history) 

log(Citation 
history) 

log(Disruptiveness 
history) 

Distance*I(Score > 
cut-off) 

Country FE 

Years FE 

Panel FE 

no 

no 

no 

Years X Panel FE 

no 

no 

no 

yes 

no 

yes 

no 

yes 

yes 

yes 

yes 

yes 

yes 

no 

no 

no 

no 

no 

no 

no 

no 

yes 

yes 

no 

yes 

no 

yes 

no 

no 

Observations 

3,172 

3,172 

3,172 

3,172 

2,425 

2,425 

2,425 

2,425 

1166 

 
 
 
log(DI1_2y) 

log(DI1_3y) 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

R2 

0.013 

0.012 

0.010 

0.009 

0.013 

0.011 

0.008 

0.008 

F Statistic 

5.362 

4.826 

3.714 

3.379 

4.057 

3.343 

2.441 

2.233 

Robust Standard Errors in Parentheses | *** p-value < 0.001, ** p-value < 0.01, * p-value < 0.05 

Note:  the  table  reports  RDD  estimates  for  the  share  of  disruptive  publications  two,  three,  four  and  five  years  after 
application. For each outcome variable (i.e. DI1 2/3/4/5y), we report estimates for a different model in each column (models 
1, 5, 9 and 13: no fixed effects – OLS; models 2, 6, 10 and 14: country and years fixed effects; models 3, 7, 11 and 15: 
country and panel fixed  effects; models 4, 8, 12 and 16: country and cohort (panelXyear) fixed effects). The difference 
between funded and non-funded applicants was estimated using Eq 1 and Eq 2. The reason behind the decrease in the 
number of observations as we move from publications two years to five years after application is mainly because we can 
observe more applicants for the two-year variable than for three, four and five years, due to the application year (i.e. two 
years consists of 2014, 2015, 2016, 2017, 2018 and 2019 applicants, while five years consists of 2014, 2015 and 2016 
applicants). 

Table 47. Disruptive publications: impact of MSCA funding 

log(DI1_4y) 

log(DI1_5y) 

(9) 

(10) 

(11) 

(12) 

(13) 

(14) 

(15) 

(16) 

0.001 

0.002 

-0.002 

-0.003 

0.00004 

0.003 

-0.003 

-
0.0002 

(0.012) 

(0.013) 

(0.012) 

(0.012) 

(0.014) 

(0.014) 

(0.014) 

(0.014) 

-0.008 

-0.006 

-0.002 

0.001 

-0.006 

-0.005 

-
0.0001 

0.0002 

(0.009) 

(0.009) 

(0.009) 

(0.009) 

(0.009) 

(0.009) 

(0.009) 

(0.010) 

0.003 

0.003 

0.0004 

-0.001 

0.004 

0.003 

-0.002 

-0.002 

(0.005) 

(0.005) 

(0.005) 

(0.006) 

(0.006) 

(0.006) 

(0.006) 

(0.006) 

0.002 

0.003 

0.003 

0.002 

0.009 

0.011* 

0.011* 

0.010 

(0.005) 

(0.005) 

(0.005) 

(0.006) 

(0.006) 

(0.006) 

(0.006) 

(0.006) 

0.016** 

0.017** 

0.015** 

0.015* 

0.013 

0.016* 

0.012 

0.012 

(0.007) 

(0.007) 

(0.008) 

(0.008) 

(0.008) 

(0.008) 

(0.009) 

(0.009) 

Funded 

Distance 

Gender: Male 

log(Academic 
seniority) 

log(Publication 
history) 

log(Citation history) 

-0.005 

-0.006* 

-0.005 

-0.005 

-0.005 

-0.009** 

-0.008* 

-0.007* 

1167 

 
 
 
 
 
log(DI1_4y) 

log(DI1_5y) 

(9) 

(10) 

(11) 

(12) 

(13) 

(14) 

(15) 

(16) 

(0.003) 

(0.003) 

(0.003) 

(0.004) 

(0.003) 

(0.003) 

(0.004) 

(0.004) 

log(Disruptiveness 
history) 

Distance*I(Score > 
cut-off) 

0.122 

0.092 

0.063 

0.047 

0.069 

0.057 

0.049 

0.030 

(0.095) 

(0.097) 

(0.097) 

(0.098) 

(0.103) 

(0.104) 

(0.104) 

(0.106) 

0.001 

-0.002 

-0.006 

-0.012 

-0.003 

-0.008 

-0.012 

-0.016 

(0.012) 

(0.013) 

(0.013) 

(0.014) 

(0.012) 

(0.013) 

(0.014) 

(0.015) 

Country FE 

Years FE 

Panel FE 

no 

no 

no 

Years X Panel FE 

no 

no 

no 

yes 

yes 

yes 

yes 

yes 

yes 

yes 

no 

yes 

no 

yes 

yes 

no 

no 

no 

no 

no 

no 

no 

no 

no 

no 

no 

yes 

no 

yes 

Observations 

1,546 

1,546 

1,546 

1,546 

1,126 

1,126 

1,126 

1,126 

R2 

0.008 

0.008 

0.006 

0.006 

0.010 

0.013 

0.010 

0.009 

F Statistic 

1.602 

1.468 

1.027 

0.976 

1.341 

1.661 

1.295 

1.198 

Robust Standard Errors in Parentheses | *** p-value < 0.001, ** p-value < 0.01, * p-value < 0.05 

Note:  the  table  reports  RDD  estimates  for  the  share  of  disruptive  publications  two,  three,  four  and  five  years  after 
application. For each outcome variable (i.e., DI1 2/3/4/5y), we report estimates for a different model in each column (models 
1, 5, 9 and 13: no fixed effects – OLS; models 2, 6, 10 and 14: country and years fixed effects; models 3, 7, 11 and 15: 
country and panel fixed  effects; models 4, 8, 12 and 16: country and cohort (panelXyear) fixed effects). The difference 
between funded and non-funded applicants was estimated using Eq 1 and Eq 2. The reason behind the decrease in the 
number of observations as we move from publications two years to five years after application is mainly because we can 
observe more applicants for the two-year variable than for three, four and five years, due to the application year (i.e. two 
years consists of 2014, 2015, 2016, 2017, 2018 and 2019 applicants, while five years consists of 2014, 2015 and 2016 
applicants). 

Table 47 (cont). Disruptive publications: impact of MSCA funding 

The results can be summarised as follows: 

•  MSCA  applicants,  on  average,  publish  more  novel  research  than  the  average  for 
scientists from Europe (including the UK) or the USA. This result holds across most 
fields of study, although they differ in terms of magnitude. This indicates that scientists 
who apply for MSCA are also more likely to produce more novel research. 

•  An MSCA Individual Fellowship does not lead funded applicants to publish even more 
novel  or  disruptive  science  than  their  peers  who  applied  to  the  same  panel-year, 
received a similar evaluation, but did not receive funding. Both funded and non-funded 

1168 

 
 
 
 
MSCA  applicants  publish  around  5%  of  novel  documents  and  3%  of  disruptive 
documents, on average. 

•  The  likelihood  that  a  scientist  will  publish  at  least  one  novel  or  disruptive  document 
among those scientists who received MSCA funding is no higher than that for scientists 
who applied to the same panel-year and received a similar evaluation, but no MSCA 
funding. 

•  What appears to explain the likelihood of publishing novel or disruptive science after 
an  application  for  funding,  is  having  published  more  novel  outputs  before  the 
application. That is, applicants who tend to write publications that are more novel and/or 
disruptive than the average MSCA applicant in the past, also continue to do so after 
receiving MSCA funding, and more so than those who published less novel/disruptive 
science in the past.  
Interestingly,  writing  many  papers  negatively  influences  novelty,  but  positively 
influences disruptiveness. In other word, selecting more productive applicants based 
on  the  number  of  papers  published  may  lead  to  less  novelty  but  more  disruptive 
research after funding. 

• 

•  Also  interesting  is  the  finding  that  receiving  many  citations  negatively  influences 
disruptiveness.  That  is,  selecting  applicants  who  are  more  visible/successful,  based 
only  on  their  number  of  citations,  may  lead  to  less  disruptive  research  after  MSCA 
funding.  Taken  together  with  the  third  point,  this  result  is  consistent  with  studies 
showing that novel research and inventions take time to be cited or reused (Pezzoni et 
al., 2022; Wang et al., 2017). Usually, novelty is associated with high rewards but also 
with  high  risk,  because  it  needs  time  to  be  understood  and  find  legitimisation.  For 
instance, Wang et al. (2017) find that novel papers have less probability of being among 
the most-cited in the short term. Similarly, looking at technological trajectories, Pezzoni 
et al. (2022) find that novel technologies require a longer time to take off. 

1.4.5. 

Impact of ERC funding on novelty and disruptiveness 

1.4.5.1. 

Data and empirical strategy 

To investigate whether differences in novelty and disruptiveness are due to the funding, we 
narrow down the population of funded and non-funded applicants, in order to better assess 
the  impact  among  comparable  applicants  with  similar  past  levels  of  novelty  and 
disruptiveness. Initially, we eliminated all applicants who did not meet the funding threshold, 
as identified in the CORDA data with a dummy variable1455. Subsequently, we only focused 
on  those  applicants  who  received  a  ‘score  A.  These  two  steps  effectively  filtered  out  a 
substantial  portion  of  unsuccessful  applicants,  enabling  us  to  concentrate  on  non-funded 
applicants  of  the  highest  quality,  who  closely  resemble  the  funded  applicants1456. 
Consequently, the restricted sample consists of top non-funded applicants who were eligible 
for funding but did not receive it, as well as funded applicants. Within this restricted sample, 
we further refined our analysis by excluding outliers, which entailed removing all applicants 
whose average number of novel and disruptive publications at the time of application is below 

1455 In CORDA, the threshold is a variable that takes the value ‘yes’ if the applicant is above the 
threshold identified by the evaluators, and ‘no’ otherwise. 
1456 We exclude ERC-PoC, ERC-PoC-LS and ERC-LvG because the scoring system used is 
different from the rest of the schemes, and therefore not comparable. We also had to exclude 
ERC-SyG because panel information (which is a fundamental parameter in our analysis) was 
missing and also because these are not individual grants. 

1169 

 
 
 
the 1st quartile minus 1.5 times the interquartile range and above the 3rd quartile plus 1.5 
times the interquartile range1457. 

Table 48 below describes the variables used in the estimations, while Table 49 and Table 50 
provide  descriptive  statistics  of  the  outcome  variables  pre-  and  post-application  year.  We 
distinguish between two groups: the funded applicants (i.e. Score A and funded) and non-
funded applicants (i.e. Score A & top non-funded). 

Variable 

Description 

Funding status 

Equal to 1 if the applicant has obtained a grant 

Funding year 

Year of the funding 
(equal to 0 if the applicant was not funded and equal year of the funding, if 
funded) 

Year 

The time variable (when we observe our outcome variables) 

Foster’s Novelty 

Share of publications in the top 10% of all publications in OpenAlex (by 
discipline and year), by Foster’s Novelty at time t  
(FosterNovelty = number of publications in the top 10% by Foster’s 
Novelty at time t / the total number of publications at time t) (log-
transformed) 

Disruptiveness 
DI1  

Share of publications in the top 10% of all publications in OpenAlex (by 
discipline and year), by disruptiveness measure DI1 at time t  
(DI1 = number of publications in the top 10% by DI1 at time t / the total 
number of publications with at least 10 references and citations at time t) 
(log-transformed) 

Publication 
history 

Logarithm of the number of average yearly publications at time t-1 

Citation history 

Logarithm of the number of average yearly citations at time t-1 

Academic 
seniority 

Logarithm of the total number of productive years at time t  
(year t - year of the first publication + 1) 

1457 For outlier cleaning, we only focus on those applicants with at least one novel (or disruptive) 
publication, as the sample is dominated by applicants zero novel (or disruptive) publications. 
Therefore, by focusing on applicants with at least one novel/disruptive publication at the time of 
application, we remove all applicants below the Q1 - 1.5*IQR and above the Q3 + 1.5*IQR. 

1170 

 
 
 
Variable 

Description 

Gender 

Equal to 1 if the applicant is Male 

Panel 

The thematic category to which the applicant applies  

Table 48. Variables used in models on the ERC, and variable descriptions 

The shares of publications in the top 10% by Foster’s novelty and disruptiveness measure 
DI1 are measures of the novelty and disruptiveness of the applicants’ publications at time t 
(i.e. the extent to which they contribute to the most novel and disruptive scientific outputs in 
a given year-discipline, among the observable publications of scientists). The publication and 
citation history variables indicate researchers’ productivity and visibility history (i.e. at time t-
1)1458. Lastly, academic seniority represents the applicants' total number of academic years 
at year t, measured as the difference between year t and the year of the first publication for 
which we have records.  

The table below reports the average, median, standard deviation, minimum and maximum 
number and share of novel documents published by scientists funded and not funded by the 
ERC  (any  grant  type),  at  the  time  of  application  and  five  years  after  the  application.  The 
number of novel publications is the count of publications that are among the top 10% most 
novel documents in the whole of the OpenAlex repository in a given year and discipline. The 
share  is  that  number  divided  by  all  documents  published  by  the  scientist  and  available  in 
OpenAlex. 

Funded 

Obs. 

Mean 

Median 

SD 

Min 

Max 

Academic seniority 

4,593 

15.269 

14 

6.417 

3 

31 

Publication history 

4,593 

4.316 

3.222 

3.828 

0.3 

34.391 

Citation history 

4,593 

157.387 

83.333 

210.144 

Foster10_count (t=0) 

4,593 

0.314 

Foster10_share (t=0) 

4,593 

0.050 

Foster10_count (t=2) 

4,258 

0.315 

0 

0 

0 

0.677 

0.134 

0.666 

0 

0 

0 

0 

1,618.778 

3 

1 

3 

1458 Publication and citation history measures are not used in the models, but they are included 
in the descriptive statistics to provide more insights about the sample. All variables are 
normalised by the number of years (average number of publications/citations over a researcher’s 
career). 

1171 

 
 
 
 
Foster10_share (t=2) 

4,258 

0.051 

Foster10_count (t=5) 

2,353 

0.358 

Foster10_share (t=5) 

2,353 

0.047 

0 

0 

0 

0.136 

0.736 

0.123 

0 

0 

0 

1 

3 

1 

Not funded 

Obs. 

Mean 

Median 

SD 

Min 

Max 

Academic seniority 

2,417 

15.569 

14 

6.557 

3 

31 

Publication history 

2,417 

4.571 

3.385 

3.974 

0.3 

35.5 

Citation history 

2,417 

162.763 

88.231 

211.623 

0.167 

1,635.357 

Foster10_count (t=0) 

2,417 

0.274 

Foster10_share (t=0) 

2,417 

0.047 

Foster10_count (t=2) 

2,219 

0.297 

Foster10_share (t=2) 

2,219 

0.047 

Foster10_count (t=5) 

1,144 

0.362 

Foster10_share (t=5) 

1,144 

0.054 

0 

0 

0 

0 

0 

0 

0.624 

0.131 

0.664 

0.135 

0.740 

0.141 

0 

0 

0 

0 

0 

0 

3 

1 

3 

1 

3 

1 

Table 49. Novel publications: ERC descriptives. Notes. Academic seniority: years since first publication; Publication history: 
number of publications at t-1; Citation history: Number of citations at t-1; Foster10_count: number of novel documents 
published; Foster10_share: share of novel documents published; t=#: number of years since application. 

The descriptive statistics already show minimal differences between funded and non-funded 
ERC applicants at the time of application and five years later. Exactly 5% of documents by 
the funded applicants were novel at the time of application. Non-funded applicants are very 
close, at just below 5%. Five years after application, these shares remain the same: 5% of 
documents published by both funded and non-funded applicants are novel. 

The table below reports the average, median, standard deviation, minimum and maximum 
number and share of disruptive documents published by scientists funded and not funded by 
ERC (any grant type), at time of application and up to five years after application. The number 
of  disruptive  publications  is  the  count  of  publications  that  are  among  the  top  10%  most 
disruptive  documents  in  the  whole  of  the  OpenAlex  repository  in  a  given  year  and 
discipline1459. The share is that number divided by all documents published by the scientist 
and available in OpenAlex.  

1459 Using the basic Wu et al. (2019) definition: DI1.   

1172 

 
 
 
 
Funded 

Obs. 

Mean 

Median 

SD 

Min 

Max 

4,701 

15.287 

14 

6.435 

3 

31 

4,701 

4.271 

3.267 

3.544 

0.3 

33.286 

Academic 
seniority 

Publication 
history 

Citation history 

4701 

152.350 

83.667 

199.573 

DI1_count (t=0) 

4,701 

0.148 

DI1_share (t=0) 

4,701 

0.031 

DI1_count (t=2) 

4,372 

0.130 

DI1_share (t=2) 

4,372 

0.027 

DI1_count (t=5) 

2,432 

0.094 

DI1_share (t=5) 

2,432 

0.025 

DI5_count (t=5) 

2,432 

0.133 

DI1_share (t=5) 

2,432 

0.025 

0 

0 

0 

0 

0 

0 

0 

0 

0.529 

0.118 

0.579 

0.112 

0.352 

0.113 

0.406 

0.113 

Not funded 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1,557.933 

17 

1 

16 

1 

6 

1 

3 

1 

Obs. 

Mean 

Median 

SD 

Min 

Max 

2,506 

15.658 

14 

6.606 

3 

31 

2,506 

4.578 

3.429 

3.816 

0.3 

32.852 

Academic 
seniority 

Publication 
history 

Citation history 

2,506 

161.038 

89.436 

203.434 

0.167 

1,635.36 

DI1_count (t=0) 

2,506 

0.146 

DI1_share (t=0) 

2,506 

0.031 

0 

0 

0.456 

0.118 

0 

0 

8 

1 

1173 

 
 
 
 
DI1_count (t=2) 

2,277 

0.126 

DI1_share (t=2) 

2,277 

0.031 

DI1_count (t=5) 

1,174 

0.110 

DI1_share (t=5) 

1,174 

0.031 

DI5_count (t=5) 

1,174 

0.136 

DI1_share (t=5) 

1,174 

0.031 

0 

0 

0 

0 

0 

0 

0.429 

0.118 

0.405 

0.134 

0.419 

0.134 

0 

0 

0 

0 

0 

0 

10 

1 

4 

1 

3 

1 

Table 50. Disruptive publications: ERC descriptives. Notes. Academic seniority: years since first publication; Publication 
history: number of publications at t-1; Citation history: Number of citations at t-1; DI1_count: number of disruptive documents 
published; DI1_share: share of disruptive documents published; t=#: numbers of year since application. 

The descriptive statistics already show minimal differences between funded and non-funded 
ERC applicants at the time of application and five years after. Of the documents by funded 
applicants, 3% were disruptive at the time of application. Non-funded applicants show exactly 
the  same  share.  Five  years  after  application,  this  share  has  fallen  slightly  for  funded 
applicants (2.5%), but not for non-funded applicants (3%). 

Econometric  testing  was  implemented  to  control  for  the  applicant  characteristics  and  to 
identify the causal impact of ERC funding using a difference-in-differences (DiD) event study. 
We compare the outcomes in the pre-application and post-application periods for funded and 
not  funded  applicants,  where  funding  is  the  ‘treatment’.  As  the  data  allow  us  to  observe 
publication outcomes over the years, there is variation in the treatment timing in ERC funding 
data  for  the  periods  2014-2020,  and  since  the  parallel  trends  assumption  potentially  only 
holds after conditioning on observed covariates, we follow the event study setting (DiD with 
multiple time periods) proposed by Callaway and Sant’Anna (2021)1460, 1461. Accordingly, we 
specify the following equation for our DiD estimation: 

Y𝑖, 𝑡 = α1

g,t + α2

g,t Gg + α3

g,t 1{T = t} + 𝛽 (Gg  1{T=t}) + Xi'θ + 𝜖it   

(3) 

Where Y𝑖,𝑡 is one of our output variables (share of novel or disruptive publications, all log-
g,t Gg is a binary variable that equals 0 if the 
transformed) for applicant i in year t; α1
applicant is not funded, or equals 1 if the applicant is funded in year g (i.e. the funding year); 
g,t 1{T = t}  is the coefficient for the group dummy; 𝛽 is the average treatment effect on the 
α3
treated (ATT); Gg  1{T=t is the coefficient for the post-treatment dummy; and Xi'θ + 𝜖it  is a 
vector of control variables that includes gender and academic seniority. 

g,t + α2

To  calculate  the  average  treatment  effect  in  DiD  with  a  multiple  time  period  setting,  we 
followed  the  guidelines  suggested  by  Callaway  and  Sant’Anna  (2021).  Accordingly,  the 
group-time  ATTs  are  estimated  using  the  doubly  robust  approach;  standard  errors  are 

1460 Callaway, B., & Sant’Anna, P.H.C., Difference-in-Differences with multiple time periods. 
Journal of Econometrics, 225(2), 200–230, 2021, available at: 
https://doi.org/10.1016/j.jeconom.2020.12.001. 
1461 For the parallel trends assumption, we rely on a pre-test of parallel trends assumption 
parameter that is provided by the output of the software created by the authors (namely, att_gt). 
In all cases, the parallel trends assumption holds (conditioned on observed covariates). 

1174 

 
 
 
clustered  at  the  individual  level  using  the  multiplier  bootstrap  with  1,000  iterations.  In  our 
analysis,  we  estimate  our group-time  ATTs  by  considering  five  years  before  and  after the 
application. 

1.4.5.2. 

Results 

Figure  17  reports  the  results  for  our  novelty  measure,  and  Figure  18  for  our  disruptive 
measure.  

The results can be summarised as follows: 

•  On  average,  ERC  applicants  publish  more  novel  research  than  average  scientists  in 
Europe  (including  the  UK)  or  the  USA.  This  result  holds  across  most  fields  of  study, 
although they differ in terms of magnitude. It indicates that scientists who apply for ERC 
are also more likely to produce more novel research. 

• 

•  Across  funding  schemes  and  disciplinary  panels,  ERC  funding  does  not  lead  funded 
applicants to publish even more novel or disruptive academic documents than their peers 
who applied to the same panel-year, received a similar evaluation, but did not receive 
funding. Of the documents published by both funded and non-funded ERC applicants, 
on average around 5% are novel, and 3% are disruptive, on average. 
In the case of Starting Grants in the field of Physics, the ERC applicants who received 
funding  consistently  published  even  more  novel  research  than  non-funded  applicants, 
but the difference year by year is not statistically significant. 
In the case of Starting Grants in the field of Social Sciences, we find the opposite result: 
funded applicants consistently published less novel and disruptive research than non-
funded  applicants.  However,  again  in  this  case,  the  difference  year  by  year  is  not 
statistically significant. 

• 

1175 

 
 
1176 

 
 
 
 
Figure 17. Novel publications: impact of ERC funding 

Note: the figure reports DiD estimates for the share of novel publications published since the beginning of an applicant’s 
career, divided by the years since first publication. In each row, we report estimates for a different panel (top: Physics (PE); 
middle: Life Sciences (LS); bottom: Social Sciences and Humanities (SH)). In each column, we report estimates for different 
ERC programmes (left: Advanced (ADG); middle: Consolidator (COG); right: Starting (STG)). The difference between 
funded and non-funded applicants was estimated using Eq 3. We report point estimates and confidence intervals for the 
year of application (0), five years before, and five years after. The size of the confidence intervals increases with time, as the 
sample decreases. 

1177 

 
 
 
 
 
1178 

 
 
 
1179 

 
 
 
Figure 18. Disruptive publications: impact of ERC funding 

Note: the figure reports DiD estimates for the share of disruptive publications published since the beginning of an applicant’s 
career, divided by the years since first publication. In each row, we report estimates for a different panel (top: Physics (PE); 
middle: Life Sciences (LS); bottom: Social Sciences and Humanities (SH)). In each column we report estimates for different 
ERC programmes (left: Advanced (ADG); middle: Consolidator (COG); right: Starting (STG)). The difference between 
funded and non-funded applicants was estimated using Eq 3. We report point estimates and confidence intervals for the 
year of application (0), five years before, and five years after. The size of the confidence intervals increases with time, as the 
sample decreases. 

1.5.  SDG analysis 

1.5.1.  SDG analysis: results at the level of Horizon Europe pillars 

This  is  a  methodological  annex  dedicated  to  the  analysis  of  SDG  data.  Its  main  aim  is  to 
present data and insights regarding how different parts of Horizon Europe have contributed 
to the Sustainable Development Goals (SDGs). The initial sample was a full list of Horizon 
Europe projects, but this was narrowed down to observations for which data relating to SDGs 
was available. As the Horizon Europe programme is still ongoing, the share of projects with 
the status ‘closed’ is as yet insufficient to conduct an analysis. The sample therefore did not 
consider the status of the project.  

In the breakdowns, we present two values: absolute and relative. The absolute value refers 
to the number of projects/publications/million EUR from a certain group (pillar or programme 
part)  that  contributed  to  a  given  SDG.  The  relative  value  refers  to  the  relative  share  of 
contribution made to a particular SDG, as a proportion of the overall contribution made to all 
SDGs by that group (pillar or programme part). 

1180 

 
 
 
The  SDG  labels  were  assigned  to  the  projects  using  the  OSDG  tool1462.  In  following  this 
approach,  we  first  assembled  a  text  corpus  for each  proposal  text. SDG  labels  were  then 
assigned to each project, using the entire project text corpus. Assigning SDG labels was a 
three-stage process: in the first stage, AI and machine-learning models were used to assign 
preliminary SDG labels. Expert-curated ontology was then used to double-check and verify 
the  initial  labelling.  In  the  final  step,  the  results  were  aggregated  from  the  text  segment 
(paragraph)  level  to  project  level,  taking  into  account  the  size  of  each  project  corpus,  the 
amount of SDG-related content in the corpus, and the relative distribution of different SDGs. 
To receive an SDG label, at least 15% of text segments in the corpus have to be SDG-related, 
and each SDG included in the final label must account for at least 10% of all the SDG-relevant 
content in the project corpus. The OSDG tool supports  multi-label classification, i.e. it can 
assign more than one SDG label to relevant projects. 

Table 51 presents an overview of the distribution of projects among the pillars of the Horizon 
Europe Framework Programme. At the time of analysis, there were almost no closed projects 
(0.6%).  Consequently, the analysis encompassed all projects, regardless of status. Out of 
9,459  projects,  8,964  contained  data  relating  to  SDGs,  and  almost  63%  of  these  projects 
were pursued under Pillar 1. 

Horizon 
Europe 
thematic 
priority 

Total number of 
projects 

Number of 
projects with 
status ’Closed’ 

Total number of 
projects analysed 

Total number of 
projects analysed 
in projects with 
status ‘Closed’ 

Number 

Share 

Number 

Share 

Number 

Share 

Number 

Share 

Pillar 1 

6,088 

64.4% 

27 

48.2% 

5,636 

62.9% 

27 

48.2% 

Pillar 2 

2,217 

23.4% 

Pillar 3 

818 

8.6% 

3 

9 

5.4% 

2,195 

24.5% 

16.1% 

807 

9.0% 

3 

9 

5.4% 

16.1% 

WIDERA 

336 

3.6% 

17 

30.4% 

326 

3.6% 

17 

30.4% 

Total 

9,459 

100% 

56 

100% 

8,964 

100% 

56 

100% 

Table 51. Overview of the project status under Horizon Europe. Source: data from Corda. 

The composition of Horizon Europe projects in terms of the pillars under which they fall and 
the SDGs they relate to is shown in Table 52. This reveals that overall, projects focused 
most intensively on: SDG3  – Good Health and  Well-being  (44%); SDG7 – Affordable 
and Clean Energy (24.2%); SDG9 – Industry, Innovation and Infrastructure (22.9%); and 
SDG16 – Peace, Justice and Strong Institutions (21%). The shares of projects relevant to 
particular SDGs tended to be similar across pillars, with the exception of SDGs 4, 5 and 16 
(Quality Education; Gender Equality and Peace, Justice and Strong Institutions), which were 
included significantly more widely in WIDERA than under the other pillars. The least attention 
was devoted to the SDG10  – Reduced Inequalities (1.4%), while SDG1 – No poverty was 
given similarly little coverage under Pillar 3. 

1462 For a detailed explanation of this tool, see: 
https://arxiv.org/ftp/arxiv/papers/2211/2211.11252.pdf. 

1181 

 
 
 
Pillar 1 

Pillar 2 

Pillar 3 

WIDERA 

Total 

Thematic 
Priority 

Number  Share  Number  Share  Number  Share  Number  Share  Number  Share 

SDG1 

235 

4.2% 

59 

2.7% 

2 

0.2% 

5 

1.5% 

301 

3.4% 

SDG2 

129 

2.3% 

125 

5.7% 

37 

4.6% 

11 

3.4% 

302 

3.4% 

SDG3 

2,867 

50.9% 

538 

24.5% 

418 

51.8% 

125 

38.3% 

3,948 

44.0% 

SDG4 

1,118 

19.8% 

106 

4.8% 

44 

5.5% 

130 

39.9% 

1,398 

15.6% 

SDG5 

659 

11.7% 

179 

8.2% 

63 

7.8% 

87 

26.7% 

988 

11.0% 

SDG6 

214 

3.8% 

87 

4.0% 

24 

3.0% 

17 

5.2% 

342 

3.8% 

SDG7 

1,136 

20.2% 

768 

35.0% 

226 

28.0% 

42 

12.9% 

2,172 

24.2% 

SDG8 

193 

3.4% 

138 

6.3% 

98 

12.1% 

32 

9.8% 

461 

5.1% 

SDG9 

1,198 

21.3% 

584 

26.6% 

221 

27.4% 

54 

16.6% 

2,057 

22.9% 

SDG10 

87 

1.5% 

33 

1.5% 

1 

0.1% 

6 

1.8% 

127 

1.4% 

SDG11 

676 

12.0% 

379 

17.3% 

26 

3.2% 

26 

8.0% 

1,107 

12.3% 

SDG12 

205 

3.6% 

372 

16.9% 

110 

13.6% 

17 

5.2% 

704 

7.9% 

SDG13 

743 

13.2% 

602 

27.4% 

215 

26.6% 

42 

12.9% 

1,602 

17.9% 

SDG14 

325 

5.8% 

124 

5.6% 

17 

2.1% 

19 

5.8% 

485 

5.4% 

SDG15 

224 

4.0% 

154 

7.0% 

14 

1.7% 

14 

4.3% 

406 

4.5% 

SDG16 

1,148 

20.4% 

515 

23.5% 

81 

10.0% 

138 

42.3% 

1,882 

21.0% 

Total 

5,569 

98.8% 

2,194 

100% 

806 

99.9% 

326 

100% 

8,895 

99.2% 

Table 52. Number of Horizon Europe projects by SDG. Source: data from Corda and projects’ descriptions of action (DoA). 

Comparison of SDG coverage under the Horizon Europe and its predecessor, Horizon 
2020, reveals a significant increase in the importance of the SDGs. Overall, 99.2% of 
projects  analysed  in  Horizon  Europe  covered  at  least  one  SDG  –  an  increase  of  27 
percentage points compared with Horizon 2020. All SDGs experienced improvements except 
for SDG6 – Clean Water and Sanitation, SDG8 – Decent Work and Economic Growth, and 
SDG9 – Industry, Innovation and Infrastructure.  

1182 

 
 
Thematic priority 

Total, Horizon Europe 

Total, Horizon 2020 

SDG1 

SDG2 

SDG3 

SDG4 

SDG5 

SDG6 

SDG7 

SDG8 

SDG9 

SDG10 

SDG11 

SDG12 

SDG13 

SDG14 

SDG15 

SDG16 

Total 

3.4% 

3.4% 

44.0% 

15.6% 

11.0% 

3.8% 

24.2% 

5.1% 

22.9% 

1.4% 

12.3% 

7.9% 

17.9% 

5.4% 

4.5% 

21.0% 

99.2% 

2.0% 

0.3% 

26.6% 

8.0% 

1.0% 

3.9% 

12.1% 

7.4% 

24.5% 

0.7% 

10.6% 

6.7% 

17.3% 

3.2% 

3.5% 

6.6% 

72.2% 

Table 53. Comparison of the shares of Horizon Europe and Horizon 2020 projects by SDG. Source: data from Corda and 
projects’ descriptions of action (DoA). 

Table  54  presents  the  total  EU  allocated  funding  for  each  SDG  by  pillar,  with  amounts 
assigned according to the coverage of a particular SDG by a project. The largest amount 
associated  with  achieving  a  particular  SDG  under  Horizon  Europe,  EUR 5,753.8 
million, is devoted to SDG3  – Good  Health and Well-being, and constitutes nearly a 
quarter  of  all  funds  contributed  by  the  EU.  SDG  3  was  consistently  the  most  heavily 
funded SDG across all pillars except Pillar 2, under which the largest share of EU funding 
was associated with SG7 – Affordable and Clean Energy. 

1183 

 
 
Pillar 1 

Pillar 2 

Pillar 3 

WIDERA 

Total 

Thematic 
priority 

Number 

Share 

Number 

Share  Number 

Share  Number 

Share 

Number 

Share 

SDG1 

57.3 

0.9% 

41.7 

0.3% 

0.1 

0.0% 

2.0 

0.2% 

101.1 

0.4% 

SDG2 

28.2 

0.5% 

213.7 

1.5% 

50.9 

1.8% 

4.5 

0.5% 

297.3 

1.3% 

SDG3 

2,224.3 

36.3% 

2,474.2 

17.8% 

887.1 

31.5% 

168.3 

19.9% 

5,753.8 

24.3% 

SDG4 

270.5 

4.4% 

132.4 

1.0% 

224.4 

8.0% 

84.8 

10.0% 

712.2 

3.0% 

SDG5 

134.9 

2.2% 

144.1 

1.0% 

72.7 

2.6% 

78.2 

9.2% 

429.9 

1.8% 

SDG6 

71.2 

1.2% 

190.2 

1.4% 

22.0 

0.8% 

5.2 

0.6% 

288.7 

1.2% 

SDG7 

579.6 

9.5% 

2,912.9 

20.9% 

299.9 

10.6% 

43.0 

5.1% 

3,835.4 

16.2% 

SDG8 

75.7 

1.2% 

155.5 

1.1% 

250.7 

8.9% 

28.5 

3.4% 

510.4 

2.2% 

SDG9 

667.6 

10.9% 

1,204.2 

8.7% 

236.6 

8.4% 

35.2 

4.2% 

2,143.6 

9.0% 

SDG10 

41.8 

0.7% 

27.2 

0.2% 

0.0 

0.0% 

3.7 

0.4% 

72.7 

0.3% 

SDG11 

233.0 

3.8% 

821.0 

5.9% 

97.6 

3.5% 

16.9 

2.0% 

1,168.5 

4.9% 

SDG12 

59.2 

1.0% 

938.2 

6.7% 

107.6 

3.8% 

6.6 

0.8% 

1,111.6 

4.7% 

SDG13 

313.4 

5.1% 

1,240.7 

8.9% 

120.4 

4.3% 

20.4 

2.4% 

1,694.9 

7.1% 

SDG14 

175.3 

2.9% 

393.2 

2.8% 

14.7 

0.5% 

9.3 

1.1% 

592.5 

2.5% 

SDG15 

89.8 

1.5% 

367.4 

2.6% 

8.4 

0.3% 

11.7 

1.4% 

477.3 

2.0% 

SDG16 

497.1 

8.1% 

829.8 

6.0% 

92.8 

3.3% 

165.8 

19.6% 

1,585.5 

6.7% 

Total 

6,044.2 

98.6% 

13 888.8 

99.8% 

2,817.0 

99.9% 

846.0 

100% 

23,596.0 

99.5% 

Table 54. EU contribution by SDG (in million EUR). Source: data from Corda and projects’ descriptions of action (DoA). 

Direct comparisons between EU contributions under Horizon Europe and Horizon 2020 are 
not possible, for the following reasons: 1) the Horizon 2020 analysis was based on closed 
projects;  2)  the  present  early  stage  of  the  Horizon  Europe  FP;  3)  the  analysis  of  EU 
contribution  to  the  SDGs  under  Horizon  2020  was  based  on  publications  data,  while  the 
corresponding  analysis  of  Horizon  Europe  is  based  on  projects’  proposal  texts  (DoA). 
However,  looking  at  the  overall  trend,  we  can  see  that  the  focus  on  sustainable 

1184 

 
 
development  has  not  changed  significantly,  with  SDG9  –  Industry,  Innovation  and 
Infrastructure and SDG3 – Good Health and Well-being remaining the most intensively 
funded SDGs.  

1.5.2.  SDG analysis: results at the level of Pillar 1 and WIDERA 

Table 55 provides a closer look at the distribution of projects under Pillar 1 and WIDERA. 
The total number of projects analysed (and used for further analysis) excludes projects with 
missing information about SDG performance.  

Total 
number of 
projects 

Number of 
projects with 
status ‘Closed’ 

Total number 
of projects 
analysed 

Total number of 
projects analysed in 
projects with status 
‘Closed’ 

Horizon 
Europe 
thematic 
priority 

ERC 

MSCA 

INFRA 

2,774 

3,233 

81 

Total Pillar 1 

6,088 

WIDERA 

336 

1 

24 

2 

27 

17 

2,366 

3,189 

81 

5,636 

326 

1 

24 

2 

27 

17 

Table 55. Overview of the project status. Source: data from Corda. 

The breakdown of programme parts by SDG presented in Table 56 below reveals that SDG3 
– Good Health and Well-being corresponded to around a half of projects under Pillar 1. SDG3 
was  also  the  SDG  most  widely  addressed  under  the  ERC  and  MSCA  programme  parts. 
SDG16 – Peace, Justice and Strong Institutions was the most widely addressed by projects 
under both Research Infrastructures (INFRA) and WIDERA. Because Pillar 1 (especially the 
ERC and MSCA) accounts for the projects, its focus on SDGs repeated the trend observed 
for the total Horizon Europe programmes.  

ERC 

MSCA 

INFRA 

Total Pillar 1 

WIDERA 

Total,  
Horizon Europe 

Thematic 
priority 

Number  Share  Number  Share  Number  Share  Number  Share  Number  Share  Number  Share 

SDG1 

133 

5.6% 

101 

3.2% 

SDG2 

38 

1.6% 

86 

2.7% 

1 

5 

1.2% 

235 

4.2% 

5 

1.5% 

301 

3.4% 

6.2% 

129 

2.3% 

11 

3.4% 

302 

3.4% 

SDG3 

1,305 

55.2% 

1,542 

48.4% 

20 

24.7% 

2,867 

50.9% 

125 

38.3% 

3,948 

44.0% 

SDG4 

155 

6.6% 

955 

29.9% 

8 

9.9% 

1,118 

19.8% 

130 

39.9% 

1,398 

15.6% 

1185 

 
 
ERC 

MSCA 

INFRA 

Total Pillar 1 

WIDERA 

Total,  
Horizon Europe 

Thematic 
priority 

Number  Share  Number  Share  Number  Share  Number  Share  Number  Share  Number  Share 

SDG5 

72 

3.0% 

579 

18.2% 

SDG6 

63 

2.7% 

149 

4.7% 

8 

2 

9.9% 

659 

11.7% 

87 

26.7% 

988 

11.0% 

2.5% 

214 

3.8% 

17 

5.2% 

342 

3.8% 

SDG7 

500 

21.1% 

625 

19.6% 

11 

13.6% 

1,136 

20.2% 

42 

12.9% 

2,172 

24.2% 

SDG8 

92 

3.9% 

98 

3.1% 

3 

3.7% 

193 

3.4% 

32 

9.8% 

461 

5.1% 

SDG9 

630 

26.6% 

545 

17.1% 

23 

28.4% 

1,198 

21.3% 

54 

16.6% 

2,057 

22.9% 

SDG10 

67 

2.8% 

20 

0.6% 

SDG11 

245 

10.4% 

426 

13.4% 

SDG12 

54 

2.3% 

150 

4.7% 

0 

5 

1 

0.0% 

87 

1.5% 

6 

1.8% 

127 

1.4% 

6.2% 

676 

12.0% 

26 

8.0% 

1,107 

12.3% 

1.2% 

205 

3.6% 

17 

5.2% 

704 

7.9% 

SDG13 

291 

12.3% 

426 

13.4% 

26 

32.1% 

743 

13.2% 

42 

12.9% 

1,602 

17.9% 

SDG14 

101 

4.3% 

211 

6.6% 

13 

16.0% 

325 

5.8% 

19 

5.8% 

485 

5.4% 

SDG15 

85 

3.6% 

134 

4.2% 

5 

6.2% 

224 

4.0% 

14 

4.3% 

406 

4.5% 

SDG16 

336 

14.2% 

772 

24.2% 

40 

49.4% 

1,148 

20.4% 

138 

42.3% 

1,882 

21.0% 

Total 

2,320 

98.1% 

3,170 

99.4% 

79 

97.5% 

5,569 

98.8% 

326 

100.0% 

8,895 

99.2% 

Table 56. Number of Horizon Europe projects, by SDG. Source: data from Corda and projects’ descriptions of action (DoA). 

Table 57 below reflects an overall increase in the coverage of SDGs under Pillar 1 and 
WIDERA,  compared with  equivalent  programmes under  Horizon  2020.  In  addition,  all 
programme parts have seen some shift in the coverage of the SDGs. For example, the ERC 
under Horizon Europe has seen an increase in the importance of SG7, which was not covered 
as extensively under Horizon 2020. Meanwhile, the MSCA under Horizon Europe has seen 
an increase in the coverage of SDG4 and SDG16 (whereas the most important SDGs under 
Horizon 2020 were SDG9 and SDG 13). In INFRA, it appears that attention has shifted from 
SDG4  to  SDG  16  under  Horizon  Europe.  Lastly,  WIDERA  has  seen  a  shift  in  focus  from 
SDG9 to SDG 16. 

1186 

 
 
Thematic 
priority 

ERC 
(Horizon 
Europe) 

ERC 
(Horizon 
2020) 

MSCA 
(Horizon 
Europe) 

MSCA 
(Horizon 
2020) 

INFRA 
(Horizon 
Europe) 

INFRA 
(Horizon 
2020) 

WIDERA 
(Horizon 
Europe) 

SEWP + 
SwafS 
(Horizon 
2020) 

SDG1 

5.6% 

3.4% 

3.2% 

3% 

1.2% 

6% 

1.5% 

1.6% 

SDG2 

1.6% 

0% 

2.7% 

0.5% 

6.2% 

0% 

3.4% 

0.8% 

SDG3 

55.2% 

44.93% 

48.4% 

38.4% 

24.7% 

29.3% 

38.3% 

28.7% 

SDG4 

6.6% 

1.92% 

29.9% 

7.4% 

9.9% 

45.3% 

39.9% 

47.8% 

SDG5 

3.0% 

0.77% 

18.2% 

1.3% 

9.9% 

2% 

26.7% 

7.6% 

SDG6 

2.7% 

1.83% 

4.7% 

3.3% 

2.5% 

6.7% 

5.2% 

3.2% 

SDG7 

21.1% 

5.88% 

19.6% 

6.5% 

13.6% 

14% 

12.9% 

6.8% 

SDG8 

3.9% 

2.5% 

3.1% 

3.6% 

3.7% 

14.7% 

9.8% 

17.9% 

SDG9 

26.6% 

22.2% 

17.1% 

14.7% 

28.4% 

62% 

16.6% 

22.7% 

SDG10 

2.8% 

1.25% 

0.6% 

0.8% 

0.0% 

1.3% 

1.8% 

0.4% 

SDG11 

10.4% 

5.4% 

13.4% 

9.4% 

6.2% 

21.3% 

8.0% 

14.3% 

SDG12 

2.3% 

1.92% 

4.7% 

2.8% 

1.2% 

3.3% 

5.2% 

3.6% 

SDG13 

12.3% 

12% 

13.4% 

14.2% 

32.1% 

33.3% 

12.9% 

14.3% 

SDG14 

4.3% 

1.83% 

6.6% 

4.3% 

16.0% 

11.3% 

5.8% 

4.4% 

SDG15 

3.6% 

2.31% 

4.2% 

4.3% 

6.2% 

6% 

4.3% 

4.4% 

SDG16 

14.2% 

5% 

24.2% 

7.2% 

49.4% 

24% 

42.3% 

16.7% 

Total 

98.1% 

77.3% 

99.4% 

85.87% 

97.5% 

99.3% 

100.0% 

84.5% 

Table 57. Comparison of the shares of Horizon Europe and Horizon 2020 projects by SDG. Source: data from Corda and 
projects’ descriptions of action (DoA). 

Overall, the trends are similar to total EU contributions under Horizon Europe. As mentioned 
previously, in Pillar 1 and WIDERA, the highest EU contributions related to SDG3, SDG7, 
SDG9 and SDG16, as well as SDG 14 – Climate Action, which was slightly lower than the 
total, and was mainly targeted by projects under INFRA. 

1187 

 
 
Thematic 
Priority 

ERC 

MSCA 

INFRA 

Number 

Share 

Number 

Share 

Number 

Share 

SDG1 

52.0 

1.3% 

SDG2 

11.5 

0.3% 

5.0 

8.3 

0.3% 

0.6% 

0.4 

8.3 

0.1% 

1.6% 

SDG3 

1,692.9 

41.2% 

440.9 

29.4% 

90.5 

17.3% 

SDG4 

96.8 

2.4% 

142.1 

9.5% 

31.7 

6.1% 

SDG5 

43.1 

1.0% 

85.9 

5.7% 

SDG6 

44.8 

1.1% 

24.1 

1.6% 

5.9 

2.3 

1.1% 

0.4% 

SDG7 

437.4 

10.6% 

116.7 

7.8% 

25.4 

4.9% 

SDG8 

54.9 

1.3% 

15.7 

1.0% 

5.1 

1.0% 

SDG9 

516.7 

12.6% 

88.8 

5.9% 

62.0 

11.8% 

SDG10 

40.4 

1.0% 

1.4 

0.1% 

0.0 

0.0% 

SDG11 

171.2 

4.2% 

51.5 

3.4% 

10.3 

2.0% 

SDG12 

22.5 

0.5% 

26.8 

1.8% 

9.8 

1.9% 

SDG13 

212.3 

5.2% 

50.7 

3.4% 

50.4 

9.6% 

SDG14 

89.0 

2.2% 

32.1 

2.1% 

54.2 

10.4% 

SDG15 

65.3 

1.6% 

16.5 

1.1% 

8.0 

1.5% 

SDG16 

282.8 

6.9% 

142.9 

9.5% 

71.3 

13.6% 

Total 

4,041.5 

98.3% 

1 494.9 

99.7% 

507.8 

97.0% 

Thematic 
Priority 

Total, Pillar 1 

WIDERA 

Total,  
Horizon Europe 

Number 

Share 

Number 

Share 

Number 

Share 

SDG1 

57.3 

0.9% 

2.0 

0.2% 

101.1 

0.4% 

1188 

 
 
SDG2 

28.2 

0.5% 

4.5 

0.5% 

297.3 

1.3% 

SDG3 

2,224.3 

36.3% 

168.3 

19.9% 

5,753.8 

24.3% 

SDG4 

270.5 

4.4% 

84.8 

10.0% 

712.2 

3.0% 

SDG5 

134.9 

2.2% 

78.2 

9.2% 

429.9 

1.8% 

SDG6 

71.2 

1.2% 

5.2 

0.6% 

288.7 

1.2% 

SDG7 

579.6 

9.5% 

43.0 

5.1% 

3,835.4 

16.2% 

SDG8 

75.7 

1.2% 

28.5 

3.4% 

510.4 

2.2% 

SDG9 

667.6 

10.9% 

35.2 

4.2% 

2,143.6 

9.0% 

SDG10 

41.8 

0.7% 

3.7 

0.4% 

72.7 

0.3% 

SDG11 

233.0 

3.8% 

16.9 

2.0% 

1,168.5 

4.9% 

SDG12 

59.2 

1.0% 

6.6 

0.8% 

1,111.6 

4.7% 

SDG13 

313.4 

5.1% 

20.4 

2.4% 

1,694.9 

7.1% 

SDG14 

175.3 

2.9% 

9.3 

1.1% 

592.5 

2.5% 

SDG15 

89.8 

1.5% 

11.7 

1.4% 

477.3 

2.0% 

SDG16 

497.1 

8.1% 

165.8 

19.6% 

1,585.5 

6.7% 

Total 

6,044.2 

98.6% 

846.0 

100.0% 

23 596.0 

99.5% 

Table 58. EU contributions under Horizon Europe by SDG (in million EUR). Source: data from Corda and projects’ 
descriptions of action (DoA). 

Shares  of  EU  contributions  under  Pillar  1  of  Horizon  Europe  were  similar  to  those  of  its 
predecessor in the Horizon 2020 programme, with funding mainly focusing on SDG3 – Good 
Health  and  Well-being,  and  SDG  9  –  Industry,  Innovation  and  Infrastructure.  Despite  the 
present early stage of the programme, SDG7 – Affordable and Clean Energy, and SDG16 – 
Peace, Justice and Strong Institutions, appear more targeted than under Horizon 2020. While 
the same is true for WIDERA with respect to SDG16 – Peace, Justice and Strong Institutions, 
the increased intensity of EU contributions is most noticeable with respect to SDG5 – Gender 
Equality relative to the equivalent programme parts under Horizon 2020. 

1189 

 
 
 
 
1.6.  Application and participation patterns 

Table 59 presents a summary of Horizon Europe funding up to this point1463. The three pillars 
and  WIDERA  combined  have  together  provided  a  total  of  EUR  24.7  billion  to  56,028 
participations. Of this, almost EUR 1.6 billion has been provided to newcomers, defined as 
organisations that did not participate in Horizon 2020. Table 60 presents breakdowns of the 
individual programme parts within Pillar 1, each of which will be further analysed in its own 
section. 

Number of 
participations 

Number of 
unique 
newcomers 

Amount 
of funding 
(million 
EUR) 

Amount of 
funding to 
newcomers 
(million EUR) 

Application 
success rate 
(%) 

Horizon 
Europe 
overall 

56 028 

3 961 

24 657.08 

1 592.70 

15.89 

Pillar 1 

16 095 

554 

6 913.40 

31.22 

15.25 

Pillar 2 

35 387 

2 492 

14 051.78 

1 294.51 

18.99 

Pillar 3 

2 666 

191 

2 844.18 

247.85 

9.41 

WIDERA 

1 880 

97 

847.72 

19.12 

27.80 

Table 59. Overview of participation under Horizon Europe. Source: compiled by the evaluation team using CORDA data, 
June 2023 release. 

Programme 
part 

Number of 
participations 

Number of 
unique 
newcomers 

Amount of 
funding 
(million 
EUR) 

Amount of 
funding to 
newcomers 
(million 
EUR) 

Application 
success 
rate (%) 

ERC 

3 277 

11 

4 879.87 

3.05 

14.00 

MSCA 

10 986 

515 

1 510.04 

16.39 

16.09 

INFRA 

1 832 

28 

523.49 

11.78 

53.38 

1463 Latest update: 26 June 2023. 

1190 

 
 
 
 
Programme 
part 

Number of 
participations 

Number of 
unique 
newcomers 

Amount of 
funding 
(million 
EUR) 

Amount of 
funding to 
newcomers 
(million 
EUR) 

Application 
success 
rate (%) 

WIDERA 
Widening 

WIDERA 
ERA 

1 175 

705 

58 

39 

719.64 

14.81 

26.54 

128.08 

4.31 

35.53 

Table 60. Participation patterns, by programme part. Source: compiled by the evaluation team using CORDA data, June 
2023 release. 

So far under Horizon Europe, the majority of participating entities (10 902) are located 
in non-widening countries. Of all funding, 76.3% has gone to non-widening countries. In 
Pillars 1, 2 and 3, the majority of entities come from, and a higher share of funding goes to, 
non-widening  countries.  In  contrast,  due  to  its  specific  objective,  WIDERA  has  almost  1.7 
times more participating entities from widening countries than from non-widening. However, 
despite this, WIDERA’s funding for widening participants still totalled slightly less than that 
given to participant from non-widening countries. 

Pillar 

Region 

Number 
of 
entities 

Number of 
unique 
newcomers 

Share of 
funding 
(% of 
pillar 
total) 

Share of 
funding to 
newcomers 
(% of 
region 
total) 

Application 
success 
rate (%) 

Associated 

1 181 

304 

10.14 

4.86 

20.63 

Non-
widening 

HE 
overall 

10 902 

2 281 

76.29 

5.20 

22.46 

Third 

3 101 

783 

0.95 

14.48 

21.34 

Widening 

2 855 

593 

12.62 

6.06 

19.56 

Associated 

73 

18 

20.17 

0.28 

19.56 

Non-
widening 

Pillar 1 

929 

321 

73.09 

0.67 

20.19 

Third 

474 

154 

0.50 

4.14 

23.96 

Widening 

165 

61 

6.24 

1.36 

18.18 

1191 

 
 
Pillar 

Region 

Number 
of 
entities 

Number of 
unique 
newcomers 

Share of 
funding 
(% of 
pillar 
total) 

Share of 
funding to 
newcomers 
(% of 
region 
total) 

Application 
success 
rate (%) 

Associated 

478 

201 

6.04 

8.64 

22.36 

Non-
widening 

Pillar 2 

4 257 

1 440 

77.63 

5.22 

24.88 

Third 

1 219 

505 

1.39 

15.84 

20.72 

Widening 

1 036 

346 

14.94 

6.83 

20.54 

Associated 

51 

17 

6.92 

20.44 

9.64 

Pillar 31464 

WIDERA 

Non-
widening 

Third 

Widening 

Associated 

Non-
widening 

Third 

Widening 

377 

132 

86.04 

16.72 

11.46 

43 

71 

37 

55 

35 

91 

20 

22 

20 

16 

17 

44 

0.04 

85.99 

3.36 

7.01 

13.23 

8.95 

6.31 

4.27 

30.00 

47.71 

0.67 

31.35 

0.31 

20.60 

41.56 

45.67 

3.12 

24.29 

Table 61. Overview of participation patterns regional overview. Source: compiled by the evaluation team using CORDA data, 
June 2023 release. 

Higher  or  secondary  educational  establishments  (HES)  made  up  the  majority  of 
participants  under  Horizon  Europe,  followed  by  private,  for-profit  institutes  (PRC)  and 
research organisations (REC). Pillar 1 and WIDERA followed this overall trend, while Pillars 
2 and 3 saw a higher share of PRCs than HES. In all pillars, public bodies (PUB) and NGOs, 
as well as citizen and societal engagement projects (OTH) constituted the lowest share of 
participations and funding. Non-traditional actors will be further explored in subsection 1.6.5. 

1464 Excludes EIT. 

1192 

 
 
 
Pillar 

Legal 
entity 
type 

Number of 
participations 

Share of funding 
(% of pillar total) 

Application 
success rate (%) 

HES 

19 484 

36.07 

OTH 

4 187 

6.05 

Horizon Europe overall 

PRC 

16 816 

29.09 

PUB 

3 031 

3.50 

REC 

12 508 

25.30 

HES 

9 236 

69.19 

OTH 

500 

Pillar 1 

PRC 

2 464 

PUB 

410 

0.92 

3.14 

0.72 

REC 

3 485 

26.03 

HES 

8 649 

23.64 

OTH 

3 242 

6.01 

Pillar 2 

PRC 

12 996 

36.72 

PUB 

2 351 

5.19 

REC 

8 149 

28.44 

HES 

760 

OTH 

235 

Pillar 3 

PRC 

1 118 

PUB 

99 

REC 

452 

1193 

13.32 

13.89 

61.90 

2.24 

8.65 

19.11 

23.74 

21.87 

26.09 

24.46 

18.67 

24.22 

20.24 

22.32 

23.52 

20.63 

23.95 

23.38 

26.35 

26.49 

9.82 

17.50 

10.16 

24.06 

10.53 

 
 
Pillar 

Legal 
entity 
type 

Number of 
participations 

Share of funding 
(% of pillar total) 

Application 
success rate (%) 

HES 

839 

OTH 

210 

WIDERA 

PRC 

238 

PUB 

171 

44.66 

22.81 

7.10 

2.50 

REC 

422 

22.93 

27.97 

26.37 

24.50 

36.25 

28.69 

Table 62. Participation patterns, by entity type. Source: compiled by the evaluation team using CORDA data, June 2023 
release. 

Among EU Member States, the highest share of funding (15.9%) went to Germany, followed 
by 11.1% to France. The shares of funding generally follow the numbers of participants; however, 
some  small  discrepancies  can  be  seen.  Italy  saw  the  second-highest  number  of  participants 
(5,389) but only the fifth-highest share of funding (8.5%). The share of Italy’s funding that went to 
newcomers was also comparatively low, at 3.7%. France saw a comparatively high share of its 
funding go to newcomers, at almost 10%. 

Member 
State 

Number of 
participants 

Number of 
unique 
newcomers 

DE 

FR 

ES 

NL 

IT 

BE 

EL 

SE 

AT 

6 369 

5 219 

6 207 

3 726 

5 389 

3 050 

2 372 

1 536 

1 650 

423 

338 

322 

268 

287 

161 

132 

106 

110 

Share of 
funding  
(% of 
Horizon 
Europe 
overall) 

Share of 
funding to 
newcomers 
(% of country 
total) 

Application 
success rate 
(%) 

15.89 

3.98 

23.61 

11.10 

9.96 

24.36 

10.47 

3.58 

20.88 

5.96 

3.69 

2.72 

4.67 

5.72 

5.19 

25.17 

19.40 

25.36 

19.14 

21.57 

22.36 

8.96 

8.53 

6.99 

3.63 

3.17 

3.15 

1194 

 
 
Member 
State 

Number of 
participants 

Number of 
unique 
newcomers 

Share of 
funding  
(% of 
Horizon 
Europe 
overall) 

Share of 
funding to 
newcomers 
(% of country 
total) 

Application 
success rate 
(%) 

DK 

FI 

PT 

IE 

PL 

CZ 

SI 

CY 

RO 

EE 

LU 

HU 

BG 

LT 

HR 

SK 

LV 

MT 

1 389 

1 316 

1 498 

1 076 

889 

742 

612 

451 

560 

374 

259 

422 

350 

259 

278 

235 

194 

121 

93 

83 

93 

71 

72 

76 

34 

25 

65 

32 

19 

35 

48 

31 

41 

20 

10 

11 

2.79 

2.64 

2.22 

2.13 

1.25 

1.12 

0.84 

0.62 

5.59 

4.70 

5.06 

7.41 

4.46 

4.48 

4.70 

5.29 

23.40 

22.94 

18.06 

20.57 

19.17 

22.08 

23.01 

17.60 

0.61 

11.14 

17.64 

0.52 

12.03 

21.41 

0.46 

0.40 

7.12 

4.20 

19.51 

20.11 

0.35 

14.94 

21.20 

0.32 

7.52 

20.50 

0.23 

13.84 

18.68 

0.20 

10.13 

22.06 

0.18 

10.41 

21.58 

0.13 

8.19 

21.87 

Table 63. Participation patterns by Member State, ranked by share of funding. Source: compiled by the evaluation team 
using CORDA data, June 2023 release. 

1195 

 
 
The evaluation team conducted a network analysis to find the top 1% most-networked entities 
within Horizon Europe. The  result is a list of the 180 participating entities with the highest 
closeness centrality, meaning that these entities are the most well-connected within Horizon 
Europe so far (details can be found in Annex 6. Quantitative track). This is presented together 
with participation by the Leiden top 50 European1465 entities, due to the overlap that exists 
between the two lists. 

Pillar 1 saw much higher rates of participation by highly connected entities than the 
average across Horizon Europe. In fact, the share of participations by Leiden top 50 entities 
was nearly double the Horizon Europe average (16.7% vs 8.9%), and the share of the top 
1%  of  most-networked  entities  was  10  percentage  points  higher  in  Pillar  1  than  across 
Horizon Europe overall (32.4% vs 22.1%). 

Highly  connected  entities  received  a  larger  share of  funding  in  Pillar 1  compared  with  the 
average  across  Horizon  Europe.  The  180  most-networked  entities  received  51%  of  all 
Pillar 1 funding. Leiden top 50 entities received 22.3% of all Pillar 1 funding, compared with 
just 8.3% across Horizon Europe overall. 

Pillar 

Total Leiden 
top 50 
participations 

Share of 
Leiden 
top 50 
(%) 

Share of 
funding to 
Leiden top 
50 (%) 

Total top 1% 
participations 

Share 
of top 
1% (%) 

Share of 
funding 
to top 1 
(%) 

Horizon 
Europe 
overall 

4 958 

8.85 

8.26 

14 591 

26.3 

34.5 

Pillar 1 

2 713 

16.86 

22.32 

6 122 

38.0 

51.0 

Pillar 2 

1 893 

5.35 

5.91 

7 369 

20.8 

26.5 

Pillar 3 

WIDERA 

215 

137 

8.06 

3.12 

7.29 

1.72 

652 

448 

31.6 

36.1 

23.8 

28.0 

Table 64. Participation patterns by the Leiden top 50, and by top 1% most-networked entities. Source: compiled by the 
evaluation team using CORDA data, June 2023 release. 

1.6.1.  ERC 

So  far,  the  ERC  has  seen  3  277  participations  and  allocated  nearly  EUR  5  billion  to 
researchers,  making  it  the  largest  Pillar  1  programme  part  by  budget.  The  specific 
objectives of the ERC are to attract excellent researchers from all over the world, meaning it 
is a highly competitive programme. This is reflected in the relatively low success rate of 16.4% 
for  applications  –  almost  5  percentage  points  lower  than  that  for  Horizon  Europe  overall. 
Participation by newcomers is therefore also low, with only 11 unique new participants to the 
programme. 

1465 Organisations that were included among the top 50 universities in Europe, according to the 
CWTS Leiden ranking. 

1196 

 
 
 
Table  65  shows  the  breakdown  of  funding  by  region.  Following  the  overall  trend  under 
Horizon Europe, most participations came from non-widening countries. 

Programme 
part name 

Region 

Total 
participations 

Number of 
unique 
newcomers 

ERC 

Associated 

695 

Non-
widening 

Third 

Widening 

2 303 

116 

163 

0 

5 

6 

0 

Share of 
funding  
to newcomers 
(compared 
with total for 
country type) 

Application 
success 
rate (%) 

0.00 

19.31 

0.27 

17.06 

3.09 

0.06 

11.82 

8.44 

Table 65. ERC participation patterns, by region. Source: compiled by the evaluation team using CORDA data, June 2023 
release. 

The vast majority of non-Member State participations in ERC (96.7%) came from just 
five  countries:  the  UK,  Israel,  Switzerland,  Norway  and  Turkey.  Out  of  3  277  ERC 
participations, 933 were based in non-Member States. Non-Member States with more than 
five participations are listed below, together with their numbers of participations. 

Country 

Participations 

United Kingdom 

Israel 

Switzerland 

Norway 

Turkey 

443 

190 

78 

61 

18 

Table 66. Non-EU Member States with more than five participations. Source: compiled by the evaluation team using 
CORDA data, June 2023 release. 

The largest share of funding under the ERC went to higher education institutions. Research 
organisations saw the next largest share, at 24.6%. Public, private and other research entities 
saw very few participations and little funding from the ERC.  

1197 

 
 
Programme 
part name 

Legal 
entity 
type 

Total 
participations 

Share of funding 
(compared with part 
total) 

Application 
success rate (%) 

ERC 

HES 

OTH 

PRC 

PUB 

REC 

2 375 

74.62 

13 

56 

17 

0.11 

0.52 

0.21 

816 

24.55 

16.25 

15.12 

16.88 

10.00 

18.28 

Table 67. Patterns of participation in the ERC, by type of entity. Source: compiled by the evaluation team using CORDA 
data, June 2023 release. 

The  most-networked  entities  made  up  51%  of  all  ERC  participations,  which  nearly 
twice  the  rate  of  Horizon  Europe  overall.  This  reflects  the  ERC’s  specific  objective  of 
funding excellent research, and coincides with the high share of participations by HES, as 
shown in the table above. Table 68 shows the number and share of ERC participations by 
entities that were in the top 1% most-networked entities, as ranked by closeness centrality 
(see section 1.2 of this Annex for further details).  

Total Leiden 
top 50 
participations 

Share of 
Leiden 
top 50 
(%) 

Share of 
funding to 
Leiden 
(%) 

Total top 1% 
participations 

Share 
of top 
1 (%) 

Share of 
funding to 
top 1 (%) 

ERC 

834 

25.45 

19.29 

1 686 

51.4 

53.7 

Pillar 1 

2 713 

16.86 

22.32 

6 122 

38.0 

51.0 

Table 68. Patterns of participation in the ERC by Leiden top 50, and by top 1% most-networked entities. Source: compiled 
by the evaluation team using CORDA data, June 2023 release. 

The table below presents the top 12 most-networked organisations and their participations in 
the  ERC.  Of  all  the  highly  networked  entities,  the  French  National  Centre  for  Scientific 
Research (CNRS) has the greatest number of participations in ERC, with 158. A list of 
the top 12 most-networked organisations under Horizon Europe is presented side-by-side for 
comparison.  Notably,  the  Fraunhofer  Society,  which  is  the  largest  applied  research 
organisation  in  Europe,  did  not  have  any  participations  within  the  ERC  as  of  June  2023. 
Organisations that appear in one of the two top 12 lists, but not in the other, are highlighted 
in blue. 

1198 

 
 
 
ERC 

Horizon Europe overall 

Country 

Participant 
legal name 

Number of 
participations 
in ERC 

Country 

Participant legal name 

France 

Spain 

French 
National 
Centre for 
Scientific 
Research 
(CNRS)  

Spanish 
National 
Research 
Council 
(CSIC) 

158 

Germany 

Fraunhofer Society 

34 

Spain 

Spanish National Research Council 
(CSIC) 

Belgium 

KU Leuven 

31 

France 

French National Centre for 
Scientific Research (CNRS) 

Netherlands 

Denmark 

Italy 

Delft 
University of 
Technology 

Technical 
University of 
Denmark 
(DTU) 

Polytechnic 
University of 
Milan 

23 

Italy 

National Research Council of Italy 
(CNR) 

23 

Belgium 

KU Leuven 

23 

France 

French Alternative Energies and 
Atomic Energy Commission (CEA) 

Switzerland 

ETH Zurich 

19 

Netherlands 

Delft University of Technology 

Denmark 

Aarhus 
University 

Italy 

Italy 

France 

University of 
Bologna 

National 
Research 
Council of 
Italy (CNR) 

French 
Alternative 
Energies 
and Atomic 
Energy 

17 

14 

Greece 

Centre for Research and 
Technology (CERTH) 

Finland 

VTT Technical Research Centre of 
Finland Ltd 

11 

Italy 

University of Bologna 

10 

Denmark 

Technical University of Denmark 
(DTU) 

1199 

 
 
ERC 

Horizon Europe overall 

Country 

Participant 
legal name 

Number of 
participations 
in ERC 

Country 

Participant legal name 

Commission 
(CEA) 

German 
Aerospace 
Center 

Germany 

2 

Germany 

German Aerospace Centre 

Table 69. Most central organisations within the ERC, compared with Horizon Europe overall. Source: compiled by the 
evaluation team using CORDA data, June 2023 release. 

The study team also analysed the participation of individual ERC beneficiaries by gender. So 
far, there have been 380 distinct ERC projects. Almost twice as many men as women have 
participated, and almost twice as much funding has gone to men as to women.  The 
survey  of  ERC PI  beneficiaries  and  unsuccessful  applicants  conducted  by  the study team 
revealed  no  visible  difference  between  men  and  women  in  their  perceptions  of  the 
programme. 

Gender 

Total participants  Amount of funding (million EUR) 

Male 

Female 

246 

132 

Non-binary 

2 

415.87 

220.66 

1.65 

Table 70. ERC participants and funding by gender. Source: compiled by the evaluation team using CORDA data, June 2023 
release. 

1.6.2.  MSCA 

The MSCA constitute the largest Pillar 1 programme by number of participations, with 10,986 
as of June 2023. Altogether, these participants have received over EUR 1.5 billion in funding. 
Like the ERC, the MSCA focus on excellence; however, a higher share of funding has gone 
to newcomers to the MSCA, compared with the ERC (1% compared with 0.6%). 

Because the specific objectives of MSCA actions are very different from one another, Table 
71 further breaks down participation by the main actions. Doctoral Networks saw the largest 
number of participations (4,911), as well as the highest amount of funding (EUR 728.8 million) 
within  MSCA.  Notably,  although  Cofund  saw  the  highest  share  of  participations  by 
newcomers  (12.9%),  it  also  saw  the  lowest  share  of  funding  to  newcomers  of  any  single 
programme part.  

1200 

 
 
Action 

Number of 
participations 

Amount of 
funding 
(million 
EUR) 

Number of 
newcomers1466 

Share of 
funding to 
newcomers (%) 

Doctoral Networks 

4 911 

728.83 

405 

Postdoctoral 
Fellowships 

Staff Exchanges 

Cofund 

MSCA & Citizens 

Other (actions with 
30 or fewer 
participations each)  

4 149 

568.36 

889 

591 

384 

73.55 

89.46 

16.75 

62 

33.08 

73 

70 

76 

28 

3 

2.31 

0.41 

5.35 

<0.00 

3.03 

<0.00 

Table 71. MSCA participations, by action. Source: compiled by the evaluation team using CORDA data, June 2023 release. 

The regional participation patterns over different MSCA actions largely mirrored the 
Horizon  Europe  overall  regional  participation  patterns:  non-widening  countries  saw 
the greatest share of participants and the greatest share of newcomers. The share of 
funding to newcomers was exceptionally high within third-country Staff Exchanges, with 67% 
of SE third-country funding going to newcomers.  

Action 

Region 

Number of 
participations 

Number of 
newcomers 

Share of funding to 
newcomers (% of 
regional/action total) 

Associated 

511 

Doctoral 
Networks 

Postdoctoral 
Fellowships 

Non-
widening 

Third 

Widening 

Associated 

Non-
widening 

3 612 

354 

434 

691 

2 762 

54 

270 

44 

37 

7 

49 

4.90 

1.82 

7.56 

5.30 

0.00 

0.52 

1466 Newcomer calculations by call may differ slightly than calculations by pillar. 

1201 

 
 
 
Action 

Region 

Number of 
participations 

Number of 
newcomers 

Share of funding to 
newcomers (% of 
regional/action total) 

Staff 
Exchanges 

Cofund 

Third 

Widening 

Associated 

Non-
widening 

Third 

Widening 

463 

233 

111 

355 

251 

172 

Associated 

56 

Non-
widening 

Third 

Widening 

365 

76 

94 

Associated 

1 413 

11 

6 

7 

22 

26 

15 

1 

38 

9 

28 

70 

Non-
widening 

7 327 

401 

MSCA overall 

Third 

1 157 

Widening 

1 089 

92 

92 

NA1467 

0.55 

3.15 

5.16 

66.99 

5.26 

0.00 

0.00 

NA 

0.00 

1.34 

1.37 

14.39 

3.70 

Table 72. MSCA regional breakdown by action. Source: compiled by the evaluation team using CORDA data, June 2023 
release. 

The  majority  of  MSCA  participations  came  from  HES.  MSCA  also  saw  a  large  number  of 
participations by PRCs and RECs. Only a small number of participations came from OTH. 

1467 No EU funding was recorded as going to PF or Cofund third countries. 

1202 

 
 
 
Programme 
part name 

Legal 
entity 
type 

Number of 
participations 

Share of funding 
(compared with part 
total) 

Application 
success rate (%) 

MSCA 

HES 

OTH 

PRC 

PUB 

REC 

6 208 

385 

2 246 

311 

1 836 

68.62 

1.47 

9.84 

1.89 

18.18 

18.31 

20.88 

19.52 

19.49 

19.59 

Table 73. MSCA participations, by entity type. Source: compiled by the evaluation team using CORDA data, June 2023 
release. 

Similar to the ERC, the focus of the MSCA on attracting excellence resulted in a high share 
of participations by (and funding to) top organisations. The MSCA provided almost half of 
its total funding to the 180 entities that appear in the top 1% most-networked.  

Total Leiden 
participations 

Share of 
Leiden 
top 50 
(%) 

Share of 
funding to 
Leiden top 
50(%) 

Total top 1 
participations 

Share 
of top 1 
(%) 

Share of 
funding to 
top 1 (%) 

MSCA 

1 700 

15.47 

26.07 

3 812 

34.7 

49.3 

Pillar 1 

2 713 

16.86 

22.32 

6 122 

38.0 

51.0 

Table 74. MSCA participation patterns by Leiden top 50, and by top 1% networked. Source: compiled by the evaluation team 
using CORDA data, June 2023 release. 

The table below presents the top 12 most-networked entities participating in the MSCA. The 
entities  are the same  as  the  top  12  most-networked  organisations  across  Horizon  Europe 
overall. Of all highly networked entities, the French National Centre for Scientific Research 
has participated in MSCA the most, with 189 participations so far. 

Country 

Participant legal 
name 

Entity type 

Number of 
participations 

France 

Spain 

French National Centre 
for Scientific Research 
(CNRS) 

Spanish National 
Research Council 
(CSIC) 

REC 

REC 

189 

121 

1203 

 
 
 
Country 

Participant legal 
name 

Entity type 

Number of 
participations 

Belgium 

KU Leuven 

Netherlands 

Delft University of 
Technology 

Denmark 

Technical University of 
Denmark (DTU) 

HES 

HES 

HES 

Italy 

Italy 

University of Bologna 

HES 

National Research 
Council of Italy (CNR) 

REC 

Germany 

Fraunhofer Society 

REC 

France 

French Alternative 
Energies and Atomic 
Energy Commission 
(CEA) 

Germany 

German Aerospace 
Centre 

Greece 

Finland 

Centre for Research 
and Technology 
(CERTH) 

VTT Technical 
Research Centre of 
Finland Ltd 

REC 

REC 

REC 

REC 

98 

53 

53 

52 

44 

18 

17 

8 

5 

4 

Table 75. Top 12 most central organisations within MSCA, compared with Horizon Europe overall. Source: compiled by the 
evaluation team using CORDA data, June 2023 release. 

1.6.3. 

INFRA 

The INFRA programme, which is smaller than the ERC and MSCA, has so far seen 1 832 
participations  and  given  over  EUR  500  million  in  funding.  Almost  EUR  12  million  (roughly 
2.3%  of  the  total)  went  to  INFRA  newcomers,  which  is  the  largest  share  of  funding  to 
newcomers under Pillar 1.  

1204 

 
 
Programme 
part name 

Region 

Number of 
participations 

Number of 
newcomers 

Share of 
funding to 
newcomers 
(compared 
with regional 
total) 

Application 
success 
rate (%) 

INFRA 

Associated 

252 

Non-
widening 

1 138 

4 

16 

Third 

96 

61468 

Widening 

346 

2 

3.39 

65.95 

1.90 

62.11 

0.00 

0.92 

44.09 

62.85 

Table 76. INFRA participations by region. Source: compiled by the evaluation team using CORDA data, June 2023 release. 

The largest share of funding (64%) within INFRA went to REC entities, which is more than 
double the Pillar 1 average of 26%. 

Part 
name 

Legal 
entity 
type 

Number of 
participations 

Share of funding (compared 
with total for programme 
part) 

Application 
success rate (%) 

HES 

OTH 

INFRA 

PRC 

PUB 

REC 

653 

102 

162 

82 

833 

20.36 

6.77 

7.07 

1.98 

63.82 

60.49 

66.45 

47.92 

76.11 

65.89 

Table 77. INFRA distribution by entity type. Source: compiled by the evaluation team using CORDA data, June 2023 release. 

As INFRA does not specifically target excellence in academic research, participation by the 
top  Leiden  entities  makes  up  only  9.8%  of  all  participations  in  INFRA,  which  is  7.1 
percentage  points  lower  than  their  Pillar  1  average.  Participation  by  the  top  1%  most-
networked entities is higher, at 34.1%, but still below their average representation under Pillar 
1 of 38%. 

1468 These six participants were all associated partners that did not request funds. 

1205 

 
 
 
Total Leiden 
participations 

Share of 
Leiden 
top 50 
(%) 

Share of 
funding to 
Leiden top 
50 (%) 

Participations 
by top 1% 

Share 
of top 
1% (%) 

Share of 
funding to 
top 1% 
(%) 

INFRA 

179 

9.77 

4.69 

624 

34.1 

31.7 

Pillar 1 

2 713 

16.86 

22.32 

6 122 

38.0 

51.0 

Table 78. INFRA participation patterns by Leiden top 50, and by top 1% most-networked. Source: compiled by the 
evaluation team using CORDA data, June 2023 release. 

1.6.4.  WIDERA 

WIDERA has seen 1 880 participations so far, and 97 unique newcomers. Nearly EUR 20 
million of the WIDERA budget has gone to newcomers – the highest amount of any of the 
excellent science programmes.  

In line with its specific objectives as a programme, over one-third (726 out of 1 880) of 
WIDERA’s  participations  have  come  from  widening  countries.  Although  45.7%  of 
WIDERA’s funding went to widening countries, an even higher share, 47.7%, went to non-
widening countries, which saw only 185 participations. Non-widening countries thus appear 
better-positioned to secure a greater share of funding than widening countries, even within 
WIDERA. 

Pillar 

Region 

Number of 
participations 

Number of 
unique 
newcomers 

Associated 

272 

WIDERA 

Non-
widening 

Third 

830 

52 

Widening 

726 

20 

16 

17 

44 

Share of 
funding 
(compared 
with pillar 
total) 

Share of 
funding to 
newcomers 
(compared 
with 
regional 
total) 

Application 
success 
rate (%) 

6.31 

4.27 

30.00 

47.71 

0.67 

31.35 

0.31 

20.60 

41.56 

45.67 

3.12 

24.29 

Table 79. WIDERA participation, by region. Source: compiled by the evaluation team using CORDA data, June 2023 
release. 

Most WIDERA participations came from HES, and 44.7% of WIDERA’s funding so far has 
gone to HES. OTH entities received 22.8% of funding – a much larger share of funding 
than the Horizon Europe average of 6.1% (see Table 80). OTH participants in WIDERA are 
further examined in section 1.6.5. 

1206 

 
 
 
Pillar 

Legal 
entity 
type 

Number of 
participations 

Share of funding (compared 
with total for programme 
part) 

Application 
success rate (%) 

HES 

OTH 

WIDERA 

PRC 

PUB 

REC 

839 

210 

238 

171 

422 

44.66 

22.81 

7.10 

2.50 

22.93 

27.97 

26.37 

24.50 

36.25 

28.69 

Table 80. WIDERA participation by legal entity. Source: compiled by the evaluation team using CORDA data, June 2023 
release. 

Participation and funding to highly-networked entities was lower in WIDERA than in 
Pillar 1. This is to be expected, as WIDERA focuses on increasing participation in widening 
countries, which have fewer highly-networked entities (see Annex 6 for a breakdown of the 
most-networked entities by Member State).  

Programme 
part name 

Total Leiden 
top 50 
participations 

Share 
of 
Leiden 
top 50 

Share of 
funding 
to 
Leiden 

Total top 1% 
participations 

Share 
of top 
1% 

Share of 
funding 
to top 
1% 

Widening 
participation 

Reforming 
and 
enhancing 

81 

56 

6.89 

1.21 

319 

27.1 

29.4 

7.94 

7.85 

129 

18.3 

20.1 

Pillar 1 

2 713 

16.86 

22.32 

6 122 

38.0 

51.0 

Table 81. WIDERA participation patterns by Leiden top 50, and by top 1% most-networked. Source: compiled by the 
evaluation team using CORDA data, June 2023 release. 

The specific objective of the WIDERA programme is to increase participation among widening 
countries. Participation can be bolstered by creating links between entities in non-widening 
countries, which tend to have more resources, and entities in widening countries. Network 
analysis  was  used  to  examine  the  types  of  collaborations  that  occurred  within  WIDERA, 
compared  with  Horizon  Europe  overall.  This  analysis  found  that  the  share  of 
widening/non-widening cross-collaborations accounted for 39.9% of all collaborations 
in WIDERA: 8 percentage points higher than the Horizon Europe average. 

1207 

 
 
Total 
connections 

Widening/ 
non-
widening 
collaboration 

Within 
widening 
collaborations 

Within non-
widening 
collaborations 

Share of 
cross-
collaborations 

WIDERA 

13 478 

5 384 

4 082 

4 012 

39.94% 

Horizon 
Europe 
(excluding 
WIDERA) 

614 878 

194 498 

49 440 

370 940 

31.63% 

Table 82. Connections between entities in widening and non-widening countries. Source: compiled by the evaluation team 
using CORDA data, June 2023 release. 

1.6.5.  Focus on non-traditional actors 

This  section  analyses  the  participation  patterns  of  non-traditional  actors,  which  are  those 
entities that are not part of a traditional research organisation such as a university, private 
for-profit  research  organisation,  or  publicly  funded  research  organisation.  This  includes 
categories  such  as  small  and  medium-sized  enterprises  (SMEs),  civil  society,  and  non-
governmental organisations (NGOs). 

Participation by non-traditional actors was analysed using Corda data, survey data, and case 
studies. Financial Support for Third Parties (FSTP), also known as Cascade Funding, is a 
mechanism  that  gives  some  specific  calls  the  ability  to  distribute  small  grants  (between 
EUR 10 000 and EUR 250 000) to third parties. As cascade funding recipients are not listed 
in CORDA, it was not possible to directly measure the quantitative efficiency of this funding 
tool. 

1.6.5.1. 

Citizen engagement 

The focus of the Framework Programmes on citizen engagement began under FP6 with the 
Science  and  Society  programme,  and  currently  constitutes  a  Key  Impact  Pathway  (KIP6) 
under Horizon Europe. As Case Study 15: Co-Creation, co-design and citizen engagement 
explores  the  topic  of  citizen  science  in  great  detail,  only  a  summary  of  the  findings  is 
presented here. 

The primary key challenges discovered by the case study were: 

• 

Implementation of the FSTP budget. Money designated as cascade funding is 
considered part of the project budget. This creates a strain on the Horizon Europe 
participating organisations, as pre-payments run out and interim payments are 
used entirely to fund the third parties. It was suggested by one interviewee that 
cascade funding should be treated separately from the project budget. 

•  The complexity of managing the FSTP. The administrative learning curve for this 

type of support is very steep, which represents a barrier to newcomers and smaller 
organisations who may want to use FSTP for citizen engagement.  

1208 

 
 
 
The key drivers discovered were: 

•  FSTP is an effective mechanism for citizen engagement, according to 

interviewees. Large-scale FSTP was looked upon especially positively. 

With  regard  to  citizen  engagement  more  generally,  the  key  challenges  discovered were  a 
lack of clear definitions, a lack of visibility and understanding on the part of research teams, 
limited resources, limited incentives for citizens, and practical implementation challenges. 

The key drivers of success for FSTP include the emphasis placed on citizen engagement in 
the  Horizon  Europe  strategic  plan,  lessons  learned  from  previous  initiatives,  capacity 
building, and the evaluation and monitoring system that is currently in development.  

1.6.5.2. 

Participation patterns among non-traditional actors 

The  legal  entity  category  Other  (OTH)  includes  NGOs,  SMEs,  start-ups,  incubators, 
museums and other non-traditional R&I actors. This section examines participation patterns 
among this subgroup. 

Table  83  shows  the  distribution  of  OTH  participations.  The  WIDERA  programme 
‘Reforming and enhancing the European R&I system’ saw the largest share, at 18.3% 
–  more  than  twice  the  Horizon  Europe  average.  Two  calls  in  particular  –  ERA-01-61, 
‘Supporting and giving recognition to citizen science in the European Research Area’, and 
the  European  Union  Contest  for  Young  Scientists  (EUCYS)  –  are  specifically  aimed  at 
broadening  engagement  in  citizen  science.  Under  all  Pillar  1  programme  parts,  the 
participation of OTH-type entities is lower than the Horizon Europe average. 

Programme part 

Total OTH 
participations 

Total 
participations 

Share 
(%) 

Horizon Europe overall 

4 121 

55 4281469 

7.43 

ERC 

MSCA 

INFRA 

WIDERA (Widening participation) 

WIDERA (Reforming and 
enhancing) 

13 

385 

102 

81 

129 

3 277 

10 986 

1 832 

1 175 

0.40 

3.50 

5.57 

6.89 

705 

18.30 

Table 83. OTH participations, by programme part. Source: compiled by the evaluation team using CORDA data, June 2023 
release. 

The majority of OTH participations came from Belgium, followed by Spain, Italy and France. 
Belgium’s OTH participants also received a far larger share of funding than OTH entities from 
any other country. Out of this funding, 25% went to the Cost Association, the Belgian entity 

1469 Excludes EIT and EIC Accelerator. 

1209 

 
 
 
tasked with implementing the WIDERA Cost SGA. Cost Association is obliged to distribute at 
least half of its budget to researchers from widening countries. This means that at least EUR 
76.5 million of Belgium’s OTH entity funding went towards research in widening countries. 

Country 

Total participations  Total funding (million EUR) 

Belgium 

Spain 

Italy 

France 

Germany 

Netherlands 

Greece 

Portugal 

Austria 

Denmark 

Romania 

Finland 

Ireland 

Bulgaria 

Estonia 

Poland 

Sweden 

Hungary 

Czechia 

845 

378 

371 

307 

291 

273 

142 

100 

94 

82 

78 

76 

65 

63 

61 

57 

56 

55 

46 

1210 

607.69 

72.67 

74.70 

182.75 

93.40 

94.12 

39.92 

18.95 

38.91 

22.29 

11.73 

21.38 

40.31 

6.48 

10.81 

32.31 

12.12 

7.67 

9.91 

 
 
Country 

Total participations  Total funding (million EUR) 

Cyprus 

Slovakia 

Slovenia 

Lithuania 

Croatia 

Latvia 

Luxembourg 

Malta 

44 

37 

36 

32 

29 

19 

16 

7 

8.99 

6.43 

4.21 

7.78 

3.57 

3.15 

2.70 

0.52 

Table 84. Distribution of OTH participations, by Member State. Source: compiled by the evaluation team using CORDA data, 
June 2023 release. 

The  present  study  finds  that  very  few  OTH-type  entities  are  participating  as 
coordinators  in  Horizon  Europe  projects  (4%  of  the  total  OTH  entities  which  have 
participated in Horizon Europe so far). For comparison, this is 13 percentage points below 
the rate at which non-OTH type entities participate as coordinators. Some of the few OTH 
coordinators,  such  as  the  Cost  Association  and  the  Marie  Curie  Alumni  Association,  have 
direct ties to EU funding infrastructure. 

Figure 19. Proportions of coordinators and participants, by entity. Source: compiled by the evaluation team using CORDA 
data, June 2023 release. 

1211 

 
 
 
The  survey  of  successful  applicants  was  used  to  better  understand  the  composition  of 
participating OTH type entities. In response to the question, ‘Please describe the organisation 
that you are representing in the project’, almost half (48%) of respondents representing 
OTH entities reported that they worked for civil society organisations or NGOs. At a 
confidence  interval  of  95%,  we  estimate  that  the  true  proportion  of  NGOs  in  the  overall 
Horizon Europe population is between 4.9 and 6.0%.  

Organisation type 

Total  As a share of OTH (%) 

Civil society/NGO 

Other (free response) 

SME (Small or medium-sized enterprise) 

Research funding organisation/agency 

University/higher education institution 

Private research centre 

Public research centre 

Start-up (not older than five years at the time of the proposal 
submission date) 

External expert/consultancy firm 

Incubator 

Large enterprise 

203 

117 

34 

21 

11 

9 

9 

6 

5 

4 

4 

47.99 

27.66 

8.04 

4.96 

2.60 

2.13 

2.13 

1.42 

1.18 

0.95 

0.95 

Table 85. Types of OTH entities among respondents to a survey of successful applicants to Horizon Europe. Source: survey 
of Horizon Europe beneficiaries, conducted May-July 2023. Question: ‘Please describe the organisation that you are 
representing in the project’. Combined number of responses: 423. 

The second-largest share of OTH participants was for SMEs. According to Corda data, there 
have been 10,266 participations by SMEs in Horizon Europe so far. Pillar 2 saw the largest 
number of participations, with 7 717, and the highest share of funding, at  10.4% (shown in 
Table 86). 

1212 

 
 
 
 
Pillar 

Share of funding (compared with pillar 
total) 

Number of 
participations 

Horizon Europe 
overall 

Pillar 1 

Pillar 2 

Pillar 3 

WIDERA 

16.93 

0.86 

10.35 

5.50 

0.22 

10 266 

1 408 

7 717 

945 

196 

Table 86. Participations by SMEs. Source: compiled by the evaluation team using CORDA data, June 2023 release. 

1213 

 
 
 
 
Annex 7: Synopsis of the Survey programme 

The survey programme consisted of ten online questionnaires and covered all three pillars of 
Horizon Europe, and its results provided evidence for the present study and for other parallel 
evaluation studies of the Horizon Europe programme. A total of 112 395 organisations and 
individuals  were  contacted,  and  18  342  valid  responses  (completed  and  partial)  were 
obtained. A response rate of 16.3% was achieved. 

1.1. 

Survey scope 

Survey programme consisted of ten online questionnaires and served two main purposes. 
Firstly, it provided evidence to answer the evaluation questions; in particular, the survey data 
provided evidence on needs and motives for participating in Horizon Europe, perceptions of 
expected project outcomes and impacts, and obstacles encountered during the application 
and implementation phases. Secondly, the survey data fed into other methods, particularly 
the counterfactual impact analysis. For this purpose, the survey targeted both Horizon Europe 
beneficiaries and unsuccessful applicants: 

Survey of Horizon Europe beneficiaries, conducted in May-November 2023, consisted of 
7 different survey questionnaires tailored for different programme parts of Horizon Europe: 

•  Survey  of 

the  Horizon  Europe  MSCA  Postdoctoral  Fellowships  beneficiary 

researchers; 

•  Survey  of  the  Horizon  Europe  MSCA  Doctoral  Networks,  Staff  Exchanges  and 

COFUND researchers; 

•  Survey of Horizon Europe ERC beneficiary Principal Investigators (PIs); 

•  Survey of beneficiary organisations under the Horizon Europe collaborative actions, 

including: 

Pillar  I:  MSCA  (Doctoral  Networks;  Staff  exchanges;  and  COFUND),  Research 

Infrastructures. 

Pillar II: Global Challenges & European Industrial Competitiveness (Clusters 1-4). 

Pillar III: beneficiaries of European Innovation ecosystems; 

Horizontal actions: WIDERA. 

•  Survey  of  beneficiary  organisations  under  the  Horizon  Europe  Pillar  II:  Cluster  5  and 

Cluster 6 programme parts; 

•  Survey  of  beneficiary  organisations  under  the  Horizon  Europe  Pillar  III:  European 

Innovation Council (Pathfinder and Transition grants); 

•  A  separate  questionnaire  is  developed  for  beneficiary  organisations  under  Horizon 

Europe Pillar III: European Innovation Council (EIC) Accelerator grants.  

1214 

 
 
Survey  of  Horizon  Europe  unsuccessful  applicants,  conducted  in  May-July,  2023 
consisted of 3 survey questionnaires: 

•  Survey  of  unsuccessful  applicants  to  individual  researcher-driven  Horizon  Europe 
actions  (survey  of  unsuccessful  applicants  to  Horizon  Europe  MSCA  Postdoctoral 
fellowships and ERC); 

•  Survey  of  unsuccessful  applicants  to  the  Horizon  Europe  collaborative  actions, 

including: 

Pillar  I:  unsuccessful  applicants  to  MSCA  (Doctoral  Networks;  Staff  exchanges;  and 

COFUND), Research Infrastructures. 

Pillar 

II:  unsuccessful  applicants 
Competitiveness (Clusters 1-6) grants. 

to  Global  Challenges  &  European 

Industrial 

Pillar  III:  unsuccessful  applicants  to  European  Innovation  ecosystems  grants  and 

European Innovation Council (Pathfinder and Transition grants). 

Horizontal actions: WIDERA. 

•  Finally,  the  survey  questionnaire  addressed  to  unsuccessful  applicants  to  Horizon 

Europe Pillar III: European Innovation Council (EIC) Accelerator grants. 

The survey programme covered all three pillars of Horizon Europe and its results provided 
the evidence for the present study and for other parallel evaluation studies of Horizon Europe 
programme. The present study mostly refers to the survey results covering Horizon Europe 
Pillar I and WIDERA programme parts, including the survey of beneficiaries and unsuccessful 
applicants  under  ERC,  MSCA  Postdoctoral  Fellowships  (PFs),  Doctoral  Networks,  Staff 
Exchanges,  COFUND,  beneficiary  organisations  and  unsuccessful  applicants  under  the 
MSCA  collaborative  actions  (DNs,  SEs  and  COFUND),  Research  Infrastructures  and 
WIDERA.  

1.2. 

Survey sampling and preparation of survey contact data 

Based on the survey strategy presented in the Inception report, the survey invitations were 
sent to all the Horizon Europe beneficiaries and unsuccessful applicants under the relevant 
programme parts. Bearing in mind that the first Horizon Europe funding calls were announced 
in 2021 and the population of the survey target group is limited, this strategy allows collecting 
survey answers from the maximum number of respondents, which will increase the reliability 
of survey results. 

Before sending the survey invitations, the following steps were taken to clean and prepare 
the CORDIS contact data received from the European Commission: 

•  First, the contact list of unsuccessful applicants was cleaned by removing from this list 
the proposals that did not meet the eligibility requirements for Horizon Europe funding, as 
well  as  other  types  of  irrelevant  applications  (ineligible,  withdrawn,  inadmissible, 
proposals on reserve list). 

•  Second  the  contact  dataset  was  cleaned  by  removing  the  irrelevant  calls,  programme 
parts  and  types  of  contacts  (e.g.  Horizon  –  EIT  applicants;  IBA  actions  (Actions  by 

1215 

 
 
invitation  (named  Beneficiaries  and  SGAs);  MSCA  PFs  and  ERC  host  organization 
contacts; partnerships, etc.) 

•  Third, the contact data was cleaned of duplicate contacts to ensure that each individual 
included in the final list of survey contacts would receive only one survey invite regarding 
one  project.  In  cleaning  duplicates,  we  employed  the  logic  that  accounted  for  the 
differences  in  population  sizes  among  different  groups  of  survey  respondents.  For 
example,  since  populations  for  unsuccessful  applicants  are  larger,  if  the  same  contact 
person  was  identified  both  under  successful  and  unsuccessful  applicants,  the  contact 
person was be removed from the latter dataset. Similarly, if the same contact appeared 
under two different Horizon Europe programme parts, the contact under the programme 
part with the larger number of applicants (population) was removed. 

•  Fourth, the contact data was cleaned by removing from the list the contact persons that 
explicitly  asked  not  to  be  contacted  in  the  similar  exercises  in  the  future  during  the 
execution of the survey programme during the Phase 1. 

•  Finally, the contact data was cleaned of invalid e-mail addresses. 

After taking these steps the final batches of survey contact data were produced amounting 
to  112  395  unique  contacts  (30 143  successful  applicants  and  82  252  unsuccessful 
applicants). 

1.3. 

Survey progress and response rates 

The survey was launched on May 19, 2023 using the Alchemer1470 tool. On June 1, survey 
reminders  were  sent  to  all  the  beneficiaries  covered  in  the  survey  (both  individuals  and 
organisations). On June 5, survey reminders were sent to the unsuccessful applicants under 
the programme parts with the lowest number of survey responses (Research Infrastructures, 
Cluster 3, EIC-EIE, EIC-Accelerator, and WIDERA). The detailed survey schedule with the 
final response rate (as of July 5, 2023) per programme part is presented in the Table 1. 

1470 https://www.alchemer.com/  

1216 

 
 
 
SUCCESSFUL APPLICANTS 

Organisations 

Programme part 

Complete 
Responses 

Partial 
responses 

Total responses 
(complete + 
partial) 

Population/sent 
(valid) 

Response 
rate 
(completes) 

Response 
rate (total) 

Launch 
date 

Reminder 
round 1 

Reminder 
round 2 

MSCA (COFUND, SE, DN) 

INFRA 

HORIZON.2.1 

HORIZON.2.2 

HORIZON.2.3 

HORIZON.2.4 

472 

152 

385 

336 

150 

653 

67 

35 

65 

51 

25 

88 

HORIZON.2.5 

772 

191 

539 

187 

450 

387 

175 

741 

963 

2726 

17,31% 

19,77% 

May 22 

June 1 

June 19-21 

1203 

12,64% 

15,54% 

May 22 

June 1 

June 19-21 

2864 

13,44% 

15,71% 

May 22 

June 1 

June 19-21 

1413 

23,78% 

27,39% 

May 22 

June 1 

June 19-21 

796 

18,84% 

21,98% 

May 22 

June 1 

June 19-21 

3842 

17,00% 

19,29% 

May 22 

June 1 

June 19-21 

5230 

14,76% 

18,41% 

May 22 

June 1 

June 19-21 

HORIZON.2.6 

826 

184 

1010 

4799 

17,21% 

21,05% 

May 22 

June 1 

June 19-21 

WIDERA 

272 

Horizon-EIC (Pathways and Transition) 

187 

41 

23 

313 

210 

1157 

23,51% 

27,05% 

May 22 

June 1 

June 19-21 

968 

19,32% 

21,69% 

May 22 

June 1 

June 19-21 

1217 

 
 
HORIZON-EIC- EIE 

HORIZON-EIC Accelerator 

HE ERC PIs 

60 

98 

237 

HE MSCA Postdoctoral Fellowships 

814 

11 

4 

17 

65 

71 

102 

308 

354 

Individuals 

19,48% 

23,05% 

May 22 

June 1 

June 19-21 

27,68% 

28,81% 

May 22 

June 1 

June 19-21 

254 

879 

907 

26,13% 

28,00% 

May 22 

June 1 

June 19-21 

2060 

39,51% 

42,67% 

May 22 

June 1 

- 

Total (successful applicants) 

5414 

867 

6281 

28627 

18,91% 

21,94% 

UNSUCCESSFUL APPLICANTS 

Organisations 

Programme part 

Complete 
Responses 

Partial 
responses 

Total responses 
(complete + 
partial) 

Population/sent 
(valid) 

Response 
rate 
(completes) 

Response 
rate (total) 

Launch 
date 

Reminder 
round 1 

Reminder 
round 2 

MSCA (COFUND, SE, DN) 

INFRA 

HORIZON.2.1 

HORIZON.2.2 

814 

93 

847 

739 

62 

10 

64 

66 

876 

103 

911 

805 

7077 

11,50% 

12,38% 

May 19 

- 

June 19-21 

656 

14,18% 

15,70% 

May 19 

June 5 

June 19-21 

10537 

8,04% 

8,65% 

May 19 

6335 

11,67% 

12,71% 

May 19 

- 

- 

June 19-21 

June 19-21 

1218 

 
 
 
 
 
HORIZON.2.3 

HORIZON.2.4 

HORIZON.2.5 

HORIZON.2.6 

WIDERA 

286 

786 

828 

848 

492 

Horizon-EIC (Pathways and Transition) 

755 

HORIZON-EIC-EIE 

HORIZON-EIC Accelerator 

133 

351 

20 

59 

71 

92 

28 

69 

12 

11 

306 

845 

899 

940 

520 

824 

145 

362 

2251 

12,71% 

13,59% 

May 19 

June 5 

June 19-21 

8557 

9,19% 

9,87% 

May 19 

8533 

9,70% 

10,54% 

May 19 

8687 

9,76% 

10,82% 

May 19 

- 

- 

- 

June 19-21 

June 19-21 

June 19-21 

3021 

16,29% 

17,21% 

May 19 

June 5 

June 19-21 

6730 

11,22% 

12,24% 

May 19 

- 

June 19-21 

962 

13,83% 

15,07% 

May 19 

June 5 

June 19-21 

2180 

16,10% 

16,61% 

May 19 

June 5 

June 19-21 

Individuals 

HE ERC PIs 

1363 

110 

1473 

6778 

20,11% 

21,73% 

May 22 

HE MSCA Postdoctoral Fellowships 

1955 

160 

2115 

9948 

19,65% 

21,26% 

May 22 

- 

- 

June 19-21 

June 19-21 

Total (unsuccessful applicants) 

10290 

834 

11124 

82252 

12,51% 

13,52% 

Table 1. Phase 2 survey launch schedule and response rate (as of July 5, 2023; final). Source: prepared by the study team. 

1219 

 
 
 
 
 
To  ensure  that  there  are  no  underrepresented  groups  among  different  programme  parts 
covered in the survey, the study team has also assessed the distribution of survey responses 
by gender (for MSCA PFs and ERC applicants) and type of organisation (for applicants under 
collaborative actions). The analysis showed that there were no underrepresented groups in 
the  survey  responses  and  the  response  pool  reflected  the  structure  of  Horizon  Europe 
applicant  population  (e.g.  within  the  survey  responses  among  successful  ERC  PIs  group 
around 37,9% were females and 58,9% were males – this distribution by gender among the 
survey respondents closely reflects the overall population of successful Horizon Europe ERC 
applicants  –  according  to  administrative  data,  around  34.5%  of  all  the  successful  ERC 
applicants  were  females,  around  95% were  males  of  ERC PIs  successful  applicants).  For 
more details on distribution of survey responses by gender and organisation type please see 
Table 3 and Table 4. 

The survey was closed on July 5. After closing the survey, the survey data was cleaned and 
prepared  for  the  analysis.  Most  importantly,  partial  responses  to  survey  were  assessed, 
cleaned and added to the final survey dataset. The study team applied the following criteria 
for cleaning out all the partial responses: 

•  Partial responses that only have answers to background questions were removed. This 
eliminated  responses  that  did  not  provide  relevant  information  to  answer  evaluation 
questions (i.e. in addition to background information).  

•  Partial  responses  with  suspicious  answer  patterns  were  excluded  (e.g.  a  person  has 
always picked the first answer option in all the multiple choice answers, or always selected 
the same response option in matrices (e.g. always either "very important" or "important"). 

•  Finally, respondents who spent less than 2 minutes to complete the entire survey were 
also  be  excluded.  The  average  time  to  complete  survey  (for  partial  responses)  varied 
significantly depending on the questionnaire, ranging from 9.5 to 33 minutes1471.  

Table 2 below provides the number of partial responses removed after each cleaning step. 

After survey  data cleaning  process  was  completed,  the data  was  transferred  to  frequency 
tables (by different programme part) in MS Excel format and distributed to all the study teams. 

1471 Please note that the average time spent to complete the survey was calculated by assessing 
the survey start date/time against the date/time the survey questionnaire was last edited by a 
respondent. In some cases, respondents returned to the survey several hours or the next day 
after starting the survey, which significantly increases the average time spent on the survey per 
respondent. This means that time spent on survey questionnaire varied from 5 seconds (in 
cases when a respondent closed the questionnaire shortly after opening it) to over 11 hours (in 
case when a respondent returned to the survey the next day after starting the survey). 

1220 

 
 
 
Questionnaire 

Survey of beneficiary 
organisations under Horizon 
Europe Pillar III: European 
Innovation Council (EIC) 
Accelerator grants 

Survey of unsuccessful 
applicants to the Horizon 
Europe collaborative actions 

Survey of the Horizon Europe 
MSCA Postdoctoral 
Fellowships beneficiary 
researchers 

Survey of beneficiary 
organisations under the 
Horizon Europe Pillar III: 
European Innovation Council 
(Pathfinder and Transition 
grants) 

Survey of beneficiary 
organisations under the 
Horizon Europe collaborative 
actions 

Survey of Horizon Europe 
ERC beneficiary Principal 
Investigators (PIs) 

Total number 
of partial 
responses 
(before 
cleaning) 

Partial responses 
removed after Step 
1 (removing 
responses with 
background 
information only) 

Partial 
responses 
removed after 
Step 2 
(responses 
with suspicious 
answer pattern) 

Partial responses 
removed after Step 
3 (responses that 
took less than 2 
minutes to complete 
the entire 
questionnaire) 

Total number 
of partial 
responses 
(after cleaning) 

Average time 
spent to 
answer 
questionnaire 
(in minutes) 

26 

21 

1625 

835 

138 

72 

49 

26 

1079 

680 

34 

17 

1 

89 

1 

0 

15 

0 

1221 

0 

4 

25.5 min 

148 

553 

16 min 

0 

0 

1 

0 

65 

33 min 

23 

9.5 min 

383 

30.5 min 

17 

17 min 

 
 
Total number 
of partial 
responses 
(before 
cleaning) 

Partial responses 
removed after Step 
1 (removing 
responses with 
background 
information only) 

Partial 
responses 
removed after 
Step 2 
(responses 
with suspicious 
answer pattern) 

Partial responses 
removed after Step 
3 (responses that 
took less than 2 
minutes to complete 
the entire 
questionnaire) 

Total number 
of partial 
responses 
(after cleaning) 

Average time 
spent to 
answer 
questionnaire 
(in minutes) 

509 

182 

43 

14 

270 

17 min 

806 

421 

45 

29 

7 

3 

3 

2 

375 

30.5 min 

11 

21 min 

Questionnaire 

Survey of unsuccessful 
applicants to individual 
researcher-driven Horizon 
Europe actions (survey of 
unsuccessful applicants to 
Horizon Europe MSCA 
Postdoctoral fellowships and 
ERC) 

Survey of beneficiary 
organisations under the 
Horizon Europe Pillar II: 
Cluster 5 and Cluster 6 
programme parts 

Survey of unsuccessful 
applicants to Horizon Europe 
Pillar III: European Innovation 
Council (EIC) Accelerator 
grants 

Total 

4311 

2283 

159 

168 

1701 

Table 2. Number of partial responses removed during the survey data cleaning process and average time spent to answer questionnaire (partial responses, by questionnaire). Source: prepared 
by the study team. 

1222 

 
 
 
 
 
SUCCESSFUL APPLICANTS 

Females 

Females (%) 

Males 

Males (%) 

Non-binary 

Non-binary 
(%) 

Prefer not to 
answer/ 
Blanks/ 
Prefer to 
self-describe 

Prefer not to 
answer/ 
Blanks/ 
Prefer to 
self-describe 
(%) 

Total 
responses 

HE ERC PIs 

96 

37,9% 

149 

58,9% 

HE MSCA Postdoctoral Fellowships 

391 

44,6% 

468 

53,4% 

Total (successful applicants) 

487 

43,1% 

617 

54,7% 

1 

1 

2 

0,4% 

0,1% 

0,2% 

7 

16 

23 

2,8% 

1,8% 

253 

876 

2,0% 

1129 

UNSUCCESSFUL APPLICANTS 

Females 

Females (%) 

Males 

Males (%) 

Non-binary 

Non-binary (%) 

Prefer not 
to answer/ 
Blanks/ 
Prefer to 
self-
describe 

Prefer not to 
answer/ 
Blanks/ 
Prefer to 
self-describe 
(%) 

Total 
responses 

HE ERC PIs 

553 

36.3% 

889 

60,60% 

HE MSCA Postdoctoral Fellowships 

889 

42,1% 

1155 

54.7% 

Total (unsuccessful applicants) 

1442 

40,1% 

2044 

56,8% 

6 

13 

19 

0,60% 

0,6% 

0,5% 

40 

54 

94 

2,70% 

1488 

2.5% 

2111 

2,6% 

3599 

Table 3. Distribution of survey responses by gender (as of June 19, 2023; final). Source: prepared by the study team. 

1223 

 
 
 
 
Programme part 

University/ 
HEI 

University/ 
HEI (%) 

Public 
research 
centre 

Public 
research 
centre (%) 

Private 
research 
centre 

Private 
research 
centre (%) 

SME 

SME 
(%) 

Large 
enterprise 

Large 
enterprise 
(%) 

SUCCESSFUL APPLICANTS 

MSCA (COFUND, SE, DN) 

356 

66,0% 

INFRA 

73 

39,0% 

HORIZON.2.1 

188 

41,8% 

HORIZON.2.2 

217 

56,1% 

HORIZON.2.3 

37 

21,1% 

HORIZON.2.4 

180 

24,3% 

HORIZON.2.5 

226 

23,5% 

HORIZON.2.6 

321 

31,8% 

WIDERA (Widening) 

110 

55,8% 

WIDERA (ERA) 

Horizon-EIC (Pathfinder) 

Horizon-EIC (Transition) 

45 

84 

5 

38,8% 

45,9% 

18,5% 

74 

64 

64 

29 

29 

126 

132 

176 

37 

16 

43 

2 

3,5% 

43 

8,0% 

25 

4,6% 

4,3% 

12 

6,4% 

1 

0,5% 

5,8% 

56 

12,4% 

13 

2,9% 

4,1% 

34 

8,8% 

5 

1,3% 

3,4% 

38 

21,7% 

24 

13,7% 

9,1% 

175 

23,6% 

109 

14,7% 

6,9% 

203 

21,1% 

134 

13,9% 

6,7% 

162 

16,0% 

38 

3,8% 

4,1% 

10 

5,1% 

3,4% 

9 

7,8% 

4,9% 

32 

17,5% 

3,7% 

11 

40,7% 

1 

1 

4 

2 

0,5% 

0,9% 

2,2% 

7,4% 

19 

8 

26 

16 

6 

67 

66 

68 

8 

4 

9 

1 

13,7% 

34,2% 

14,2% 

7,5% 

16,6% 

17,0% 

13,7% 

17,4% 

18,8% 

13,8% 

23,5% 

7,4% 

1224 

 
 
SUCCESSFUL APPLICANTS 

Programme part 

University/ 
HEI 

University/ 
HEI (%) 

Public 
research 
centre 

Public 
research 
centre (%) 

Private 
research 
centre 

Private 
research 
centre (%) 

SME 

SME 
(%) 

Large 
enterprise 

HORIZON-EIC-EIE 

HORIZON-EIC Accelerator 

7 

0 

10,0% 

0,0% 

1 

0 

1,4% 

0,0% 

0 

0 

0,0% 

18 

25,7% 

0,0% 

72 

70,6% 

0 

0 

Large 
enterprise 
(%) 

0,0% 

0,0% 

Total (successful applicants) 

1849 

35,9% 

793 

15,4% 

298 

5,8% 

875 

17,0% 

357 

6,9% 

SUCCESSFUL APPLICANTS 

Programme part 

Start-up / 
spin-off/ 
incubator 

Start-up / 
spin-off / 
incubator 
(%) 

Research 
funding 
organisation/ 
agency 

Research 
funding 
organisation / 
agency (%) 

Civil 
society 
/ NGO 

Civil 
society 
/ NGO 
(%) 

External 
expert / 
consultancy 
firm 

External 
expert / 
consultancy 
firm (%) 

Other 

Other 
(%) 

Total 
Responses 

MSCA (COFUND, SE, DN) 

INFRA 

HORIZON.2.1 

HORIZON.2.2 

HORIZON.2.3 

6 

4 

8 

4 

3 

1,1% 

2,1% 

1,8% 

1,0% 

1,7% 

0 

3 

16 

10 

5 

0,0% 

1,6% 

7 

1 

1,3% 

0,5% 

3,6% 

34 

7,6% 

2,6% 

38 

9,8% 

2,9% 

4 

2,3% 

0 

1 

3 

0 

0 

0,0% 

9 

1,7% 

539 

0,5% 

20 

10,7% 

187 

0,7% 

42 

9,3% 

450 

0,0% 

34 

8,8% 

387 

0,0% 

29 

16,6% 

175 

1225 

 
 
 
SUCCESSFUL APPLICANTS 

Programme part 

Start-up / 
spin-off/ 
incubator 

Start-up / 
spin-off / 
incubator 
(%) 

Research 
funding 
organisation/ 
agency 

Research 
funding 
organisation / 
agency (%) 

Civil 
society 
/ NGO 

Civil 
society 
/ NGO 
(%) 

External 
expert / 
consultancy 
firm 

External 
expert / 
consultancy 
firm (%) 

Other 

Other 
(%) 

Total 
Responses 

HORIZON.2.4 

HORIZON.2.5 

HORIZON.2.6 

WIDERA (Widening) 

WIDERA (ERA) 

Horizon-EIC (Pathfinder) 

Horizon-EIC (Transition) 

HORIZON-EIC-EIE 

21 

19 

10 

1 

1 

5 

5 

7 

2,8% 

2,0% 

1,0% 

0,5% 

0,9% 

2,7% 

18,5% 

10,0% 

HORIZON-EIC Accelerator 

28 

27,5% 

5 

22 

20 

4 

4 

0 

0 

7 

0 

0,7% 

21 

2,8% 

2,3% 

42 

4,4% 

2,0% 

103 

10,2% 

2,0% 

5 

2,5% 

3,4% 

12 

10,3% 

0,0% 

0,0% 

10,0% 

0,0% 

1 

0 

6 

0 

0,5% 

0,0% 

8,6% 

0,0% 

6 

14 

16 

3 

1 

1 

0 

2 

0 

0,8% 

30 

4,1% 

740 

1,5% 

105 

10,9% 

963 

1,6% 

96 

9,3% 

1010 

1,5% 

18 

9,1% 

197 

0,9% 

23 

19,8% 

116 

0,5% 

0,0% 

4 

1 

2,2% 

183 

0,0% 

27 

2,9% 

22 

24,3% 

70 

0,0% 

2 

2,0% 

102 

Total (successful applicants) 

122 

2,4% 

96 

1,9% 

274 

5,3% 

47 

0,9% 

435 

8,5% 

5146 

1226 

 
 
 
UNSUCCESSFUL APPLICANTS 

Programme part 

University / 
HEI 

University / 
HEI (%) 

Public 
research 
centre 

Public 
research 
centre (%) 

Private 
research 
centre 

Private 
research 
centre (%) 

SME 

SME 
(%) 

Large 
enterprise 

Large 
enterprise 
(%) 

MSCA (COFUND, SE, DN) 

637 

72,7% 

134 

15,3% 

INFRA 

60 

58,3% 

24 

23,3% 

HORIZON.2.1 

467 

51,3% 

140 

15,4% 

HORIZON.2.2 

475 

59,0% 

HORIZON.2.3 

90 

29,4% 

61 

29 

7,6% 

9,5% 

HORIZON.2.4 

297 

35,1% 

121 

14,3% 

HORIZON.2.5 

290 

32,3% 

127 

14,1% 

HORIZON.2.6 

342 

36,4% 

196 

20,9% 

WIDERA (Widening) 

215 

51,4% 

WIDERA (ERA) 

59 

57,8% 

93 

10 

22,2% 

9,8% 

Horizon-EIC (Pathfinder) 

361 

46,5% 

162 

20,9% 

Horizon-EIC (Transition) 

20 

41,7% 

7 

14,6% 

21 

1 

43 

25 

17 

62 

56 

58 

21 

3 

43 

3 

2,4% 

39 

4,5% 

27 

3,1% 

1,0% 

5 

4,9% 

0 

0,0% 

4,7% 

97 

10,6% 

22 

2,4% 

3,1% 

60 

7,5% 

4 

0,5% 

5,6% 

65 

21,2% 

37 

12,1% 

7,3% 

176 

20,8% 

102 

12,1% 

6,2% 

172 

19,1% 

88 

9,8% 

6,2% 

139 

14,8% 

28 

3,0% 

5,0% 

25 

6,0% 

2,9% 

4 

3,9% 

7 

0 

1,7% 

0,0% 

5,5% 

127 

16,4% 

22 

2,8% 

6,3% 

11 

22,9% 

2 

4,2% 

1227 

 
 
UNSUCCESSFUL APPLICANTS 

Programme part 

University / 
HEI 

University / 
HEI (%) 

Public 
research 
centre 

Public 
research 
centre (%) 

Private 
research 
centre 

Private 
research 
centre (%) 

SME 

SME 
(%) 

Large 
enterprise 

HORIZON-EIC-EIE 

HORIZON-EIC Accelerator 

23 

0 

15,9% 

0,0% 

5 

0 

3,4% 

0,0% 

5 

0 

3,4% 

33 

22,8% 

0,0% 

229 

62,2% 

1 

0 

Large 
enterprise 
(%) 

0,7% 

0,0% 

Total (unsuccessful applicants) 

3336 

44,3% 

1109 

14,7% 

358 

4,8% 

1182 

15,7% 

340 

4,5% 

UNSUCCESSFUL APPLICANTS 

Programme part 

Start-up / 
spin-off / 
incubator 

Start-up / 
spin-off / 
incubator 
(%) 

Research 
funding 
organisation / 
agency 

Research 
funding 
organisation / 
agency (%) 

Civil 
society 
/ NGO 

Civil 
society / 
NGO 
(%) 

External 
expert / 
consultancy 
firm 

External 
expert / 
consultancy 
firm (%) 

Other 

Other 
(%) 

Total 
Responses 

MSCA (COFUND, SE, DN) 

INFRA 

HORIZON.2.1 

HORIZON.2.2 

HORIZON.2.3 

3 

2 

21 

11 

3 

0,3% 

1,9% 

2,3% 

1,3% 

1,0% 

1 

1 

7 

1 

4 

0,1% 

1,0% 

3 

2 

0,3% 

1,9% 

0 

0 

0,0% 

11 

1,3% 

876 

0,0% 

8 

7,8% 

103 

0,8% 

42 

4,6% 

11 

1,2% 

61 

6,6% 

911 

0,1% 

90 

11,2% 

1,3% 

17 

5,6% 

7 

3 

0,9% 

71 

8,8% 

805 

1,0% 

41 

13,4% 

306 

1228 

 
 
 
 
UNSUCCESSFUL APPLICANTS 

Programme part 

Start-up / 
spin-off / 
incubator 

Start-up / 
spin-off / 
incubator 
(%) 

Research 
funding 
organisation / 
agency 

Research 
funding 
organisation / 
agency (%) 

Civil 
society 
/ NGO 

Civil 
society / 
NGO 
(%) 

External 
expert / 
consultancy 
firm 

External 
expert / 
consultancy 
firm (%) 

Other 

Other 
(%) 

Total 
Responses 

HORIZON.2.4 

HORIZON.2.5 

HORIZON.2.6 

WIDERA (Widening) 

WIDERA (ERA) 

Horizon-EIC (Pathfinder) 

Horizon-EIC (Transition) 

23 

22 

18 

4 

0 

39 

2 

2,7% 

2,5% 

1,9% 

1,0% 

0,0% 

5,0% 

4,2% 

HORIZON-EIC- EIE 

27 

18,6% 

HORIZON-EIC Accelerator 

131 

36,2% 

6 

6 

4 

4 

0 

3 

1 

2 

0 

0,7% 

18 

2,1% 

0,7% 

35 

3,9% 

0,4% 

65 

6,9% 

1,0% 

17 

4,1% 

0,0% 

11 

10,8% 

0,4% 

2,1% 

6 

1 

0,8% 

2,1% 

1,4% 

20 

13,8% 

0,0% 

0 

0,0% 

8 

12 

7 

3 

4 

6 

0 

4 

0 

0,9% 

32 

3,7% 

845 

1,3% 

91 

10,0% 

899 

0,7% 

83 

8,7% 

940 

0,7% 

29 

7,0% 

418 

3,9% 

11 

10,8% 

102 

0,8% 

0,0% 

7 

1 

0,9% 

776 

2,1% 

48 

2,8% 

25 

13,1% 

145 

0,0% 

2 

0,6% 

362 

Total (successful applicants) 

306 

4,1% 

40 

0,5% 

327 

4,3% 

65 

0,9% 

473 

6,3% 

7536 

Table 4. Distribution of survey responses by organisation type (as of June 19; final). Source: prepared by the study team 

1229 

 
 
After  the  contact  details  of  the  researchers  participating  in  the  MSCA  Doctoral  Networks, 
Staff  Exchanges  and  COFUND  action  became  available,  additional  survey  of  the  MSCA 
researchers participating in DN, COFUND and SE actions was launched on 14 November 
2023. Its planned duration was 2 weeks. The survey was closed on 28 November 2023 at 
23:59  CET.  In  addition  to  the  original  invitation  to  participate  in  the  survey,  reminders  to 
participate were sent on 20 November and 27 November 2023. This survey was sent to 1 
516  individuals  (803  doctoral  candidates,  674  researchers  on  staff  exchanges  and  39 
researchers participating in COFUND). Table 5 shows the survey statistics as they are after 
closing the survey on 28 November. 

Type of 
action 

Number of 
invitations 
sent to 
participate 
(a) 

Bounced and 
unsubscribed 
emails (b) 

Completed 
replies (c) 

Partial 
replies 
(d) 

Total 
completed 
and partial 
replies (e) 

Response 
rate 
considering 
completed 
and partial 
replies (f) 

Doctoral 
Networks 

Staff 
Exchanges 

803 

674 

COFUND 

39 

Total 

1516 

25 

30 

2 

57 

454 

106 

560 

71.98% 

279 

26 

67 

5 

346 

53.73% 

31 

83.78% 

759 

178 

937 

61,8% 

Table 5. Response rate in the survey of the MSCA researchers’ (as of November 28, 2023; final).  Source: prepared by the 
study team 

1230 

 
 
GETTING IN TOUCH WITH THE EU 

In person 
All over the European Union there are hundreds of Europe Direct centres. You can find the 
address of the centre nearest you online (european-union.europa.eu/contact-eu/meet-us_en). 

On the phone or in writing 
Europe Direct is a service that answers your questions about the European Union. 
You can contact this service: 
  by freephone: 00 800 6 7 8 9 10 11 (certain operators may charge for these calls), 
  at the following standard number: +32 22999696,  
  via the following form: european-union.europa.eu/contact-eu/write-us_en. 

FINDING INFORMATION ABOUT THE EU 

Online 
Information about the European Union in all the official languages of the EU is available on 
the Europa website (european-union.europa.eu). 

EU publications 
You can view or order EU publications at op.europa.eu/en/publications. Multiple copies of free 
publications can be obtained by contacting Europe Direct or your local documentation centre 
(european-union.europa.eu/contact-eu/meet-us_en). 

EU law and related documents 
For access to legal information from the EU, including all EU law since 1951 in all the official 
language versions, go to EUR-Lex (eur-lex.europa.eu). 

EU open data 
The portal data.europa.eu provides access to open datasets from the EU institutions, bodies 
and agencies. These can be downloaded and reused for free, for both commercial and non-
commercial purposes. The portal also provides access to a wealth of datasets from European 
countries. 

 
 
 
 
 
 
 
 
The study presents lessons learned and recommendations 
for policy on excellent science under Horizon Europe.  
This is one of several support studies feeding into the 
European Commission's interim evaluation of the European 
framework programme for Research and Innovation Horizon 
Europe. 

Studies and reports