Assessment of the Horizon 2020 support to the European University Initiative : analytical report prepared within the study "Towards a European Excellence Initiative".
Date: 2023-11-16
Institutions: Directorate-General for Research and Innovation
Subjects: EU research policy, Framework Programme for Research and Development, call for proposals, higher education, honour, innovation, report, research expenditure (EU), research infrastructure, university

Assessment of the Horizon 2020 
support to the European 
Universities Initiative  

Analytical Report prepared within the study "Towards 
a European Excellence Initiative" 

Tetra Tech and the Centre for Strategy & Evaluation Services (CSES) 

Independent 
Expert 
Report 

 
 
 
 
 
 
Assessment of the Horizon 2020 support to the European Universities Initiative  

European Commission 
Directorate-General for Research and Innovation 
Directorate A - ERA & Innovation 
Unit A.3 R&I Actors and Research Careers 
Contact  Stijn Delauré 
Email  

RTD-A2-UNIT-SUPPORT@ec.europa.eu 
Stijn.DELAURE@ec.europa.eu 
RTD-PUBLICATIONS@ec.europa.eu 

European Commission 
B-1049 Brussels 

Manuscript completed in September 2023 

This document has been prepared for the European Commission, however it reflects the views only of the authors, and the 
European Commission shall not be liable for any consequence stemming from the reuse. 

PDF  

ISBN 978-92-68-07405-3 

doi:10.2777/74484 

KI-04-23-832-EN-N 

Luxembourg: Publications Office of the European Union, 2023 

© European Union, 2023 

The reuse policy of European Commission documents is implemented by Commission Decision 2011/833/EU of 12 December 
2011  on  the  reuse  of  Commission  documents  (OJ  L  330,  14.12.2011,  p.  39).  Unless  otherwise  noted,  the  reuse  of  this 
document 
licence 
(https://creativecommons.org/licenses/by/4.0/). This means that reuse is allowed provided appropriate credit is given and any 
changes are indicated. 

is  authorised  under  a  Creative  Commons  Attribution  4.0 

(CC  BY  4.0) 

International 

Image credits for cover page and throughout: © skypicsstudio # 286372753, © MicroOne # 288703015,  
© creativeteam # 323412491, © Viktoriia # 345410470, © Yurii # 372950117, 2022. Source: Stock.Adobe.com. 

 
 
 
 
 
 
 
 
 
 
EUROPEAN COMMISSION 

Assessment of the Horizon 
2020 support to the European 
Universities Initiative  

Analytical Report prepared within the study 
"Towards a European Excellence Initiative" 

Tetra Tech and the Centre for Strategy & Evaluation Services (CSES) 

Authors: Fionn O’Sullivan, Irina Jefferies, Ninon Gautier, Fahad Anwad 

Senior Study Experts: Dr Jennifer Brennan, Dr Conor O’Carroll 

2023                              Directorate-General for Research and Innovation 

EN 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Table of contents 

Executive summary 

1. 

Introduction 

8 

14 

1.1  Overview ........................................................................................... 14 
1.2  Context .............................................................................................. 14 

1.3  How  Horizon  funding  is  expected  to  help  achieve  ultimate 
objectives .......................................................................................... 16 
1.4  Evaluation questions ....................................................................... 17 

1.5  Sources of information used for the evaluation ........................... 18 
1.5.1  Secondary sources ............................................................................ 18 

1.5.2  Primary sources - online survey ......................................................... 18 
1.5.3  Primary sources - interviews .............................................................. 19 
1.6  Assessing the strength of the evidence and addressing potential 
bias .................................................................................................... 19 

1.7  Note on names of alliances and alliance projects ........................ 20 

2. 

The Transformational potential of Horizon funding 

21 

2.1  Views  of  Member  State  representatives  on  transformational 
potential ............................................................................................ 21 
2.2  Views of key stakeholders on transformational potential ........... 21 
2.3  Results from the survey of alliance members on transformational 
potential ............................................................................................ 23 

2.4  EQ2: How far do participating universities show transformational 
progress on research and innovation issues? ............................. 24 
2.4.1  TM1: Developing shared R&I strategies ............................................ 24 

2.4.2  TM2: Sharing capacity, infrastructure, and resources ....................... 30 
2.4.3  TM3: Strengthening human capital (inc. gender equality and research 
assessment reform) ........................................................................... 34 
2.4.4  TM4:  Reinforcing  cooperation  in  R&I  with  other  sectors,  incl. 
knowledge valorisation ....................................................................... 38 

2.4.5  TM5: Transitioning to knowledge- and digitally driven HEIs, conducting 
open science ...................................................................................... 42 
2.4.6  TM6: Embedding citizens and society ............................................... 46 

2.4.7  Overall answer to EQ2 ....................................................................... 50 
2.5  EQ1: What is the transformational potential of the alliances? .... 51 

2.5.1  What  was  the  distinct  contribution  of  the  Horizon  2020  funding  to 
developing R&I capabilities? .............................................................. 52 

2 

 
2.6  EQ5:  What  are  the  strengths,  weaknesses,  opportunities  and 
threats for the establishment and consolidation of the alliances?
 ........................................................................................................... 60 
2.7  Strengths ........................................................................................... 60 

2.8  Weaknesses ...................................................................................... 61 

2.9  Opportunities .................................................................................... 62 
2.9.1  Main reasons to continue supporting alliances through Horizon Europe 
or its legacy programme ..................................................................... 62 

2.10  Threats .............................................................................................. 62 

3. 

Impacts on HEI participation in innovation systems 

64 

3.1  EQ3: What is the EUI’s impact on cooperation between the HEIs 
and wider innovation ecosystems? ............................................... 64 
3.2  Overall assessment ......................................................................... 65 
3.3  EQ  4:  To  what  extent  have  the  European  Universities  engaged 
internationally, beyond the EU? ..................................................... 66 

3.4  EQ6: Has the EUI affected university and university policies?... 71 
3.4.1  Impacts on universities ....................................................................... 71 

3.4.2  Impacts on national R&I policy ........................................................... 72 

4. 

Demonstration effects on HEIs not part of alliances 

74 

4.1  EQ 7: To what extent can European universities inspire other HEIs 
that  are  not  part  of  the  alliances  and  induce  change  in  the 
university sector with their good practices? ................................ 74 

4.2  Expansion of alliance memberships .............................................. 74 
Increases in associate membership .............................................. 75 
4.3 

4.4  Good practices ................................................................................. 75 
4.5  EQ8: To what extent have the European Universities inspired the 
broader university sector in the way they engage with their local 
knowledge ecosystems? ................................................................. 77 
4.6  EQ 9: To what extent have the European Universities inspired the 
broader  university  sector  to  engage  and 
induce  change 
internationally?................................................................................. 77 

4.7  EQ10:  what  are  the  strengths,  weaknesses,  opportunities,  and 
threats  of  the  Horizon  2020  support  mechanism  for  European 
Universities in terms of impact on other HEIs in Europe, and on 
international cooperation beyond Europe by the alliances? ...... 77 

5. 

Conclusions 

80 

5.1  Overview ........................................................................................... 80 

5.2  Overall assessment ......................................................................... 80 
5.3  How  far  has  the  funding  supported  the  transformation  of  R&I 
capabilities?...................................................................................... 80 

3 

 
5.4  Wider impacts of the Horizon 2020 funding .................................. 82 
5.5  Challenges  in  supporting  the  deepening  and  widening  of  the 
alliances’ R&I collaborations .......................................................... 83 

Annex 1: List of interviewees 

Annex 2: Documents reviewed 

84 

86 

Annex 3: Intervention logic for the Horizon funding for Alliances R&I  89 

Annex  4:  Correspondence  between  EUI  Transformational  Modules  and 
90 

European Research Area priorities 

List of Tables 

Table 1 Thematic focuses for alliances that chose this approach ....................................... 16 

Table 2 Progress against key activity indicators for each TM .............................................. 51 

List of Figures  

Figure 1 Has your university developed the following: ................................................................ 24 

Figure  2  Were  non-university  stakeholders  from  a)  business  b)  local/regional  government 
c) research d) civil society involved at all stages in creating the R&I strategy, including 
defining goals? ........................................................................................................... 25 

Figure  3  Is  there  an  action  plan  translating  the  strategy  into  activities,  with  responsibilities  for 
different stakeholders? ............................................................................................... 26 

Figure 4 Have any new partnerships been initiated because of the R&I strategy? ..................... 26 

Figure 5 Have any R&I activities been initiated because of the R&I strategy (within the university, 
or shared with other partners)? .................................................................................. 27 

Figure 6 Have there been any applications for new resources or funding at the European, national 
or local level as a result of the R&I strategy? ............................................................. 27 

Figure 7 Did your participation in the alliance influence the development or updating of: ........... 28 

Figure 8 To what extent has collaborating with other universities within  TM1 Developing shared 
R&I strategies and roadmaps, helped your university advance? ................................ 30 

Figure 9 Has your university developed plans for sharing research infrastructure with other alliance 
members? .................................................................................................................. 31 

Figure 10 Does your university have an up-to-date inventory of research infrastructure which can 
be shared with other universities? .............................................................................. 31 

Figure 11 Does your university have policies and/or guidelines defining how other universities can 
access its research infrastructure? ............................................................................. 31 

Figure 12 Did your participation in the alliance influence the development or updating of: ......... 33 

Figure 13 To what extent has collaborating with other universities within TM2  Sharing capacity, 
infrastructure, and resources, helped your university advance?................................. 33 

4 

 
 
 
Figure 14 Does your university have an explicit HR strategy or action plan for researchers (e.g. 
Human Resources Strategy for Researchers HRS4R)? ............................................. 35 

Figure 15 Does your university have an equality, diversity, and inclusion  strategy / action plan?
 ................................................................................................................................... 35 

Figure 16 Does your university have any of these support structures for female researchers? .. 36 

Figure 17 Does your alliance have a formal strategy for reforming research assessment? ........ 36 

Figure  18  To  what  extent  has  collaborating  with  other  universities  within  TM3  Strengthening 
human capital, helped your university advance? ........................................................ 37 

Figure  19  TM3:  Did  your  university’s  participation  in  the  alliance  have  any  influence  on  the 
development or updating of the following: .................................................................. 38 

Figure 20 Does your university have R&I collaboration spaces or other initiatives to encourage 
engagement with external actors?.............................................................................. 39 

Figure 21 Does your university offer training for doctoral students and/or research staff to develop 
entrepreneurial skills? ................................................................................................ 39 

Figure 22 Does your university have an inventory of existing R&I collaboration partners from other 
sectors? ...................................................................................................................... 40 

Figure  23  To  what  extent  has  collaborating  with  other  universities  within  TM4  Reinforcing 
cooperation in R&I with other sectors, helped your university advance? .................... 40 

Figure  24  TM4:  Did  your  university’s  participation  in  the  alliance  have  any  influence  on  the 
development or updating of the following: .................................................................. 41 

Figure 25 Has your university developed an Open Science policy for digital research outputs? . 42 

Figure 26 Does your university have a designated person, unit or structure for data governance 
and data management? ............................................................................................. 43 

Figure 27 Does your university have measures to support researchers in publishing open access?
 ................................................................................................................................... 44 

Figure 28 Does your university provide training for researchers on open science skills? ........... 44 

Figure 29 To what extent has collaborating with other HEIs within TM5 Transitioning to knowledge- 
and digitally driven HEIs, conducting Open Science, helped your university advance?
 ................................................................................................................................... 45 

Figure  30  TM5:  Did  your  university’s  participation  in  the  alliance  have  any  influence  on  the 
development or updating of the following: .................................................................. 46 

Figure 31 Does your university have a department, centre, lab, network, testbeds, or other initiative 
to support citizen/end-user engagement? .................................................................. 47 

Figure  32  Does  your  university  hold  activities  to  promote  citizen/end-user  engagement  in  co-
design, co-creation, and co-evaluation of science? .................................................... 47 

Figure 33 Has your university developed or implemented activities to make it more environmentally 
sustainable? ............................................................................................................... 48 

Figure 34 Does your university participate in activities to transfer academic knowledge into policy?
 ................................................................................................................................... 48 

Figure 35 To what extent has collaborating with other universities within TM6 Embedding citizens 
and society helped your university progress? ............................................................ 49 

Figure  36  TM6:  Did  your  university’s  participation  in  the  alliance  have  any  influence  on  the 
development or updating of the following ................................................................... 50 

Figure 37 To what extent has collaborating with other universities helped your university advance?
 ................................................................................................................................... 53 

5 

 
Figure  38  In  which  areas  within  individual  TMs  did  participating  in  an  alliance  have  the  most 
significant influence on R&I capabilities? ................................................................... 54 

Figure 39 Breakdown by widening and non-widening countries - percentage of respondents saying 
collaboration helped their HEI advance 'very' or 'quite significantly' ........................... 55 

Figure  40  Breakdown  by  geographic  region  (north/west  and  south/central/east  Europe)  - 
percentage of respondents saying collaboration helped their HEI advance 'very' or 'quite 
significantly' ................................................................................................................ 58 

Figure  41  Breakdown by  alliances  with  /  without  a  thematic  focus  - percentage of  respondents 
saying collaboration helped their HEI advance 'very' or 'quite significantly' ................ 59 

Figure 42 Breakdown by small and large HEIs - percentage of respondents saying collaboration 
helped their HEI advance 'very' or 'quite significantly' – ............................................. 60 

Figure  43:  Have  any  HEIs  outside  your  alliance  expressed  interest  in  any  of  your  alliance's 
activities or practices (number of responses)? ........................................................... 66 

Figure 44 Has the work of the alliance had any impact on the broader universities sector in your 
country? ..................................................................................................................... 71 

6 

 
 
 
 
 
List of Acronyms 

Acronym / 
Abbreviation 

COARA 

DG RTD  

EEA 

EEI 

EHEA 

EIT 

ERA 

ERC 

EUA 

EUI 

FAIR 

HE 

HEI 

IP 

LERU 

MSCA 

R&I 

SwafS 

SWOT 

TM  

Meaning 

Coalition for Advancing Research Assessment 

Directorate-General for Research and Innovation  

European Economic Area 

European Excellence Initiative 

European Higher Education Area 

European Institute of Innovation and Technology 

European Research Area 

European Research Council 

European University Association 

European Universities Initiative 

Findable, accessible, interoperable, reusable 

Higher Education  

Higher Education Institution 

Intellectual property  

League of European Research Universities 

Marie Skłodowska-Curie Actions 

Research & Innovation 

Science with and for Society 

Strengths, Weaknesses, Opportunities, Threats 

Transformational Module 

7 

 
Executive summary 

Following calls for proposals under Horizon 2020 ‘Science with and for Society’ (SwafS) in 
2019 and 2020, EU grants of up to EUR 2 million were offered to 39 European Universities 
alliances to implement institutional changes that would develop their research and innovation 
(R&I)  capabilities,  both  as  individual  HEIs  and  collectively  as  alliances.  The  funding  was 
intended to support the development and initial implementation of joint strategies and action 
plans,  where  possible  in  connection  with  other  EU  or  national  funding  sources.  The 
capabilities targeted by the funding related to six different ‘transformational modules’ (TMs). 

This analytical report presents an assessment of the effectiveness and impact of the funding 
on institutional changes within the alliances in the period 2019-2022. The assessment was 
undertaken  as  part  of  the  study  entitled  “Towards  a  European  Excellence  Initiative".  This 
report  examines  the  effectiveness  and  impact  of  the  grants  in  the  period  2019-2022, 
addressing twelve evaluation questions (EQs) which cover issues including what has been 
achieved so far, whether the funding has the potential to transform the R&I performance of 
the  alliances  and  their  members,  and  whether  the  projects  have  had  any  impacts  on  the 
broader Higher Education (HE) sector in Europe or beyond. The first ten of these questions 
are answered in this report, while EQs 11 and 12 (covering learning and recommendations 
on future financing) are addressed in the accompanying policy report (the second output for 
this study). 

The evaluation evidence comprises:  

•  A  review  of  programme  data  and  of  reports  and  policy  briefs  from  the  participating 

alliances  

•  An online survey with responses from 117 representatives of HEIs that participate in an 

alliance. 

• 

• 

In-depth interviews with 35 representatives of HEIs that belong to 27 different alliances 

Interviews with 7 HE stakeholder organisations and 14 organisations from 12 Member 
States  

•  Data and policy reports produced by other HE stakeholder organisations 

Overall, the evaluation provides good evidence that the Horizon 2020 funding has contributed 
positively to the development of R&I capacities among most of the participating HEIs. While 
the funding has not in itself delivered a transformation in capacities, there are positive signs 
that it has helped to put many alliances on a pathway which could achieve this in due course, 
with an important caveat that this is dependent on continuing support. 

This  progress  needs  to  be  considered  against  the  background  of  ongoing  challenges  in 
transnational  collaboration  between  HEIs,  whether  in  the  education  or  R&I  dimensions. 
These include different systems, legal and regulatory contexts, national policy objectives and 
language barriers. While the Horizon-funded R&I projects have helped the participating HEIs 
advance, the combination of these challenges can impede the speed of collaboration or lead 
to ambitions being scaled back. 

The potential for R&I transformation is also weakened by the siloed nature of the two main 
funding streams (Erasmus+ and Horizon) for the alliances, and the different administrative 
rules and timescales associated with each scheme. 

8 

 
There is almost universal consensus that moving from the development of R&I capabilities to 
delivery of R&I outcomes will require sustained and more predictable funding over several 
years. However, while longer-term funding is a necessary condition for transformation, it is 
not sufficient, and buy-in from academics at different levels within HEIs and a commitment to 
changing institutional cultures are also essential ingredients. 

There is also some anecdotal evidence of the alliances having a wider impact, such as an 
HE  stakeholder  organisations  noting  that  conversations  with  university  vice  rectors  for 
research frequently turn to the subject of the alliances, or that applications for Erasmus grants 
are also now starting to highlight synergies with R&I work. 

In terms of impacts on Member States, some governments have provided some additional 
funding to their own universities to participate in alliances (often for the education and R&I 
elements together). However, apart from this funding, there are no significant signs that the 
Horizon funding has yet had any impact on R&I policy more widely at Member State level.  

What have the R&I projects delivered? 

Results in delivering the transformational modules are promising and indicate good but early 
progress in the transformation process, taking into account that this process will necessarily 
be a long-term endeavour requiring ongoing support.  

TM1: Developing shared R&I strategies and roadmaps 

This TM (in many ways providing groundwork for the other five TMs) involved HEIs working 
to  develop  a  common  research  and  innovation  agenda,  identifying  specific  topics  for 
collaboration along with accompanying action plans to put these strategies into practice. The 
evaluation found that a substantial majority of HEIs have made progress on both an overall 
R&I strategy and in subsequent and related R&I activities as a result of their participation in 
an alliance.  

Around two thirds of HEIs responding to the online survey have developed an R&I strategy 
(67%), with nearly all of these (65%) also having developed this with other alliance members, 
often involving external stakeholders though with different degrees of intensity. These R&I 
strategies  have  allowed  alliance  members  to  make  further  significant  advances  including 
developing new partnerships, initiating R&I activities and making new applications for funding 
or resources. 

Engaging researchers in capacity-building projects has often been a challenge. However, the 
evaluation has identified various examples of good practice. A common theme in all these 
examples is linking the development of joint research strategies with funding opportunities 
for researchers, or at least the future prospect of such opportunities.  

TM2: Sharing capacity, infrastructure and resources  

Sharing infrastructure and resources within alliances covers both large-scale infrastructure 
such  as  instruments  or  laboratories,  research  services  and  human  resources  including 
research support staff. Survey results on TM2 indicate good progress in this area stemming 
from the HEIs participation in an alliance, especially regarding the development of plans with 
other universities on sharing research infrastructure.  

However, the evidence suggests this progress reflects an initial approach to collaboration up 
to this point (e.g. granting access to HEIs respective research infrastructure) rather than a 
more  ambitious  approach  entailing  a  strategic  assessment  of  requirements  across  the 
alliance. The latter could involve, for instance, some members specialising only in particular 
assets  and  relying  on  other  members  HEIs  to  provide  other  infrastructure  (rather  than 
investing in this themselves) – but this has not yet materialised in any alliance. 

9 

 
TM3: Strengthening researchers’ careers, including through gender equality plans 

TM3 focuses on the aim of nurturing and rewarding research talent and giving researchers 
the  opportunity  to  pursue  career  paths  across  their  alliance.  The  survey  results  indicate 
collaboration  has  had  a  significant  positive  influence  in  developing  an  HR  strategy, 
developing equality, diversity and inclusion plans and helping revise frameworks for research 
assessment  for  around  a  third  of  all  participating  HEIs,  and  has  provided  lesser  (but  still 
valuable) support to around another half. The results also suggest participation in an alliance 
has at times added value in helping develop support structures for female researchers – but 
only for a small number of participating HEIs.  

TM4: Reinforcing cooperation in R&I with other sectors 

TM4  aims  to  help  alliances  reinforce  cooperation  with  non-academic  sectors,  especially 
academia-business collaboration, and to promote the use of academic outputs for the benefit 
of society. 

There is good evidence that the R&I grants have helped to enhance cooperation between 
universities and wider innovation ecosystems, though the scale of impact from this is not easy 
to judge. Examples include initiating or enhancing cooperation with national, regional, and 
local  authorities,  extending  cooperation  to  R&I  in  the  context  of  existing  city  twinning 
arrangements, and strengthening relations with businesses and other stakeholders through 
new advisory boards with business expertise. Several alliances have also created new tools 
to support such cooperation, including databases of external partners and researchers and 
other IT applications that promote interaction among innovation system stakeholders. 

TM5: Transitioning to knowledge- and digitally-driven HEIs, conducting open science  

TM5  aims  to  support  alliances  in  moving  towards  a  more  open  and  collaborative  way  of 
undertaking  research  and  sharing  findings,  including  by  developing  and  implementing 
strategies  for  open  access  and  research  data  management.  The  evidence  indicates  the 
funding  has  supported  substantial  progress  across  different  capabilities  relating  to  open 
science and digital transformation. For instance, nearly all alliance members have already 
developed or are developing an open science policy for research outputs. Again, the benefits 
have  been  most  significant  for  around  a  third  of  alliance  members,  with  a  further  half 
indicating collaboration has been valuable although not transformative. 

TM6: Embedding citizens and society in R&I 

TM6 aims to ensure that academic work is publicly engaged, and that research is socially 
accountable  and  acceptable,  increasing  its  potential  to  address  societal  needs.  Survey 
results indicate that the HEIs responding to the survey have made significant progress in a 
range of areas relevant to TM6, with many having either created an institutional arrangement 
to support engagement with citizens or end-users or started to develop one. 

What is the transformational potential of the Horizon-funded R&I projects? 

There  is  a  common  view  across  stakeholders  that  the  European  Universities  alliances 
represent a significant development of the HE sector in Europe (across all their dimensions 
– not just R&I). However, it was widely noted that the three years since the start of the earliest 
R&I  projects  is  too  short  a  timescale  to  see  transformative  impacts  in  the  R&I  dimension, 
particularly given the impacts of the pandemic in slowing the pace of work. The responses 
from the survey of HEIs highlights a nuanced picture of the potential for future transformation: 

• 

around one third of the 39 alliances said that the Horizon-funded R&I project has helped 
their institution advance ‘very significantly’ or ‘quite significantly’. The evaluators interpret 
this as confirming that the project does indeed have the potential to promote significant 
transformation – albeit this will take more time.  

10 

 
•  For just  over half  the  participating  HEIs,  the  projects  have  helped  ‘somewhat’.  This is 
interpreted  as  indicating  that  the  support  has  been  valuable  and  helped  with  the 
member’s  own  efforts  (often  aligned  with  national  policies)  –  but  is  not  likely  to  be 
transformational in its own right.  

•  For a small minority – around a tenth of HEIs – participation in an alliance seems to have 
made no difference to their efforts to build R&I capabilities. (It is important to bear in mind 
that these are aggregate results – and that no HEIs said it received no benefit from any 
TM). 

The fact that a third of respondents think the projects will significantly support their institution’s 
efforts to develop its R&I capabilities is a key result and suggests a reasonably high degree 
of success for the intervention, particularly given the relatively modest level of funding.  

It is notable however that interviewees from higher education stakeholder organisations and 
Member States organisations were generally more sceptical of the transformative potential 
of the funding, in particular because of the relatively small value of grants. At the very least, 
this  scepticism  highlights  the  need  to  continue  and  extend  the  scope  of  funding  for  R&I 
transformation, but in the view of the evaluators does not undermine the generally positive 
picture highlighted by the evaluation. 

The  evaluation  also  highlighted  significant  differences  in  the  distribution  of  benefits,  with 
some  HEIs  gaining  more  than  others.  There  are  much  stronger  benefits  for  HEIs  in 
widening countries compared to those in non-widening ones, providing evidence for the 
hypothesis that institutional changes in the transformational modules will tend to accelerate 
access to excellence in less R&I intensive institutions/regions.  

However,  there  are  exceptions  to  this  overall  picture.  For  instance  across  all  countries, 
smaller HEIs (in terms of student numbers) have generally benefitted more than larger 
ones. It is also the case that the funding was welcomed by all types of HEIs (even if some 
benefitted more than others). 

What  are  the  strengths  and  weaknesses  of  the  funding  approach,  and  what  are  the 
opportunities and threats for consolidating alliances’ R&I dimension? 

The table below summarises the strengths and weaknesses of the initiative and as well as 
opportunities  and  threats  that  highlight  how  achievements  so  far can  be consolidated  and 
threats to future progress addressed.  

SWOT analysis for the establishment and consolidation of the alliances R&I dimension 

Opportunities 

•  Continue to build on early successes  
•  Capitalise  in  growing  interest  among 

non-alliance HEIs 

•  Create 

a 

stable, 
programmatic  approach  to  supporting 
R&I capabilities 

long-term, 

Strengths 
•  Supports  a  broad  EU-wide  R&I 

agenda for HEIs 

•  Helps implement the ERA agenda 
•  Promotes 

HE 

sector 

internationalisation  

•  Focus  on  building  R&I capacity  adds 

value  

•  Particularly valuable for 
smaller HEIs 
those  with  less  developed  R&I 
capabilities  

- 
- 

-  HEIs with less internationalisation 

experience  

11 

 
 
- 

those in widening countries 

Weaknesses 

•  Small  grants  compared 

to  high 

expectations 

•  Administrative  burden 

(especially 
different Horizon and Erasmus rules / 
funding periods) 

•  Uncertain funding future 
• 

Limitations  on  use  of  the  grants  for 
actual  research  (as  opposed  to  R&I 
capacity-building) 

Threats 
•  Premature  removal  of  support  would 
undermine  what  has  been  achieved  so 
far 

•  Continuing  with  a  piecemeal,  under-

funded project-based approach 

•  Forcing too strict an alignment between 
alliances’ education and R&I dimensions 
•  Policy-makers and funding focusing too 
much  on  the  alliances,  rather  than 
thinking about the overall HEI sector  
•  Spreading alliance efforts too thinly with 

•  Difficulties 

numerous policy objectives 
in 

aligning  MS 

and 
Framework Programme funding in ways 
that  promote  balanced  R&I  capability 
development  

How far have the European Universities engaged beyond the EU on R&I issues? 

The alliances show a moderate level of engagement with other universities beyond the EU. 
It is notable that neither Erasmus nor the Horizon 2020 support focused on this issue, and 
proposals incorporating measures for this was an element of only a few alliance strategies. 
There are cases where this is happening as part of the core work of the alliances (especially 
with universities in Switzerland, Ukraine, and the United Kingdom) but generally on the basis 
of strong pre-existing links between individual alliance members rather than as a result of the 
Horizon-funded R&I projects.  

Impacts on HEI’s own policies and the R&I policies of national governments for 
the HE sector 

The  addition  of  an  R&I  dimension  to  the  European  Universities  has  helped  to  stimulate 
growing interest from HEIs not currently participating in one. This is motivated in some cases 
by interest in their work in helping HEIs achieve their own institutional missions, but also from 
a  “fear  of  missing  out”.  There  have  been  instances  where  non-member  HEIs  have  either 
joined  an  alliance  as  full  or  associate  members  (or  are  planning  to  do  so).  It  is  hard  to 
distinguish  the  relative  importance  of  the  education  and  R&I  dimensions  in  this  trend, 
however.  

There is also some evidence of other HEIs learning from the experience of alliance partners 
through formal and informal means, although it is hard to determine how significant this is 
yet. For instance, there is no evidence to suggest the alliances’ R&I work is yet influencing 
the way the wider HEIs cooperate with wider innovation ecosystems – indeed this would be 
surprising  given  the  relatively  short  implementation  period  of  the  R&I  projects  to  date, 
uncertainties  about  future  funding  and  the  fact  that  significant  changes  in  innovation 
ecosystems would only be expected over the mid-long term. The same is also true of any 
influence of the alliances on the ways that wider HEIs engage internationally. 

Some  Member  States  have  provided  some  additional  funding  to  their  own  universities  to 
participate in alliances (covering both education and R&I dimensions). However, apart from 
this funding, there are no significant signs that the Horizon funding has yet had any impact 
on R&I policy more widely. This contrasts with the situation regarding educational policy at 
national level where more significant changes are evident. This may also reflect the fact that 

12 

 
 
the educational dimension of the alliances has received greater funding, and also the fact 
that advances in this dimension may require greater legislative or regulatory change than for 
R&I (and have therefore stimulated a more active response from policy-makers). 

13 

 
1. 

Introduction  

1.1  Overview 

This report constitutes one of the deliverables within the study entitled: “Towards a European 
Excellence  Initiative”,  undertaken  for  the  European  Commission  by  a  consortium  led  by 
CSES  and  involving  Tetra  Tech,  SPI,  Panteia  and  a  number  of  independent  researchers. 
This study aims to support DG RTD in the implementation of the ERA Policy Agenda Action 
13, namely to “raise excellence in science and value creation in Europe’s university sector 
and increase its global competitiveness, including through a European Excellence Initiative 
and the consolidation of the European Universities Initiative”. 

In this context, this Analytical Report presents an evaluation of the Horizon 2020 support 
to the European University Initiative’s (EUI) research and innovation (R&I) dimension, 
provided  through  ‘Science  with  and  for  Society’  (SwafS)  grants.1  It  assesses  the 
effectiveness and impact of the funding on institutional changes of the European Universities 
alliances  in  the  period  of  2019  –  2022  and  the  progress  made  by  the  higher  education 
institutions, answering twelve evaluation questions (see below). 

The Analytical Report is complemented by a Policy Report presenting recommendations to 
the Commission in relation to a possible future European Excellence Initiative. The overall 
study also builds on other work to assess the R&I dimension of the alliances, in particular the 
monitoring methodology and indicators to measure institutional change, as developed by the 
study ‘Knowledge ecosystems for the new ERA’.2 

1.2  Context 

The EUI supports the emergence of European Universities consisting of alliances of different 
universities and higher education institutions (HEIs) which on purpose have been drawn from 
different  geographical  areas  of  the  European  Union,  sometimes  also  including  associate 
partners from beyond its borders.3 

Calls  for  proposals  under  Erasmus+  were  launched  in  2019  and  2020  with  the  aim  of 
developing  systemic  and  sustainable  cooperation  between  alliance  members,  focused  on 
education objectives such as joint degrees and enabling mobility within alliances for both staff 
and students. In addition, additional funding from Horizon 2020 was provided to 39 out of a 
total of 41 European Universities alliances in two calls also launched in 2019 and 2020. These 
received grants of up to EUR 2 million to implement institutional changes that would develop 
their research and innovation (R&I) capabilities, both as individual HEIs and collectively as 
alliances. 

The  funding  was  intended  to  support  the  development  and  initial  implementation  of  joint 
strategies and action plans, where possible in connection with other EU or national funding 
sources. The funding is therefore distinctive in focusing on  developing capacity – in 
an environment where (even at national level) funding for research is often focused on 

1 https://research-and-innovation.ec.europa.eu/news/all-research-and-innovation-news/european-
commissions-supports-erasmus-european-universities-pilot-additional-34-million-horizon-2020-2020-07-
24_en#:~:text=Today%2C%20the%20European%20Commission%20announced%20that%20the%20first,an
d%20transformation%20in%20their%20research%20and%20innovation%20missions.  
2 Bergman, F. et al (2022) ‘Knowledge ecosystems in the new ERA: a comprehensive analysis of the state of 
play, the design of monitoring mechanisms, and creation of a toolbox of support measures’. 
3 The term ‘alliance’ is used throughout this report as a shorthand denomination for ‘European University’. 

14 

 
 
carrying  out  research  (e.g.  by  providing  funds  for  salaries,  equipment  and 
consumables).  

The  Horizon  SwafS  funding  called  for  work  to  be  undertaken  within  the  context  of  six 
transformational modules (TMs). Individual alliances could develop work plans covering all 
or  some  of  these,  and  could  also  focus  on  other  related  themes  of  their  own  choosing  if 
relevant to R&I. 

•  TM1: Developing shared R&I strategies and roadmaps 

•  TM2: Sharing capacity, infrastructure and resources  

•  TM3: Strengthening researchers’ careers, including through gender equality plans, 

•  TM4: Reinforcing cooperation in R&I with other sectors  

•  TM5: Transitioning to knowledge- and digitally-driven HEIs, conducting open science  

•  TM6: Embedding citizens and society in R&I. 

It  is  notable  that  there  is  a  strong  correspondence  between the  six  TMs  and  the  20 
actions of the European Research Area (ERA) policy agenda (see Annex 4), and also 
with  the  nine  broad  priority  areas  constituting  a  vision  for  the  higher  education 
institution of the future, as defined in a separate 2021 study commissioned by DG RTD.4  

What does transformational change mean? 

Taking into account the close alignment between the six TMs and actions of the ERA policy 
agenda,  ‘transformational  progress’  –  a  key  element  of  the  evaluation  questions  –  is 
understood  in  this  report  as  progress  in  developing  and  implementing  joint  actions 
across the alliances within the framework of the TMs. 

The R&I projects developed and implemented by the alliances have varied in their focus, with 
around  two  thirds  (27  out  of  39)  choosing  a  broad  trans-disciplinary  approach,  and  the 
remaining third focusing on particular themes, as highlighted in Table 1. 

4 Javorka,  Z  and  Allinson,  R  (2021)  Knowledge  ecosystems  in  the new  era  –  policy  brief  for  the European 
Strategy for Universities 

15 

 
 
 
 
 
Alliance R&I project 

CIVICA RESEARCH 

EU-CONEXUS-
RESEARCH FOR 
SOCIETY 
EU4ART Differences 
EUGLOHRIA 
SMART-ER 

reSEArch-EU 
AURORA RI 

Table 1 Thematic focuses for alliances that chose this approach 

Thematic focus 

Social sciences – around the following four themes: 
Democracy in the 21st Century; Societies in transition and 
crises of the Earth; Europe revisited; data driven 
technologies for social sciences 
Smart urban coastal sustainability 

Artistic processes and crafts 
Global health and wellbeing 
Complex societal challenges under the UN SDG11 (Make 
cities and human settlements inclusive, safe, resilient and 
sustainable) 
Research and innovation relevant to coastal areas 
i) Sustainability and Climate Change, ii) Digital Society and 
Global Citizenship, iii) Health and Wellbeing, and iv) Culture, 
Diversity 
Climate, democracy and global health 
Film and Media Arts 

CIRCLE U. ERIA 
FILMEU_RIT 
INVEST4EXCELLENCE  Sustainable development 
RE-EURECA-PRO 
Re-UNITA 

Responsible consumption and production 
Cultural heritage, renewable energies and circular economy 

1.3  How Horizon funding is expected to help achieve ultimate objectives 

Resulting from the Horizon support, the European Commission expects to see a variety of 
successful  models  for  transformation  in  R&I  capabilities,  in  synergy  with  the  education 
dimension of the alliances. The logic for this process is set out in the form of a diagram in 
Annex 3 and can be summarised as follows: 

•  Outputs from work plans developed and delivered by the alliance. These reflect activities 

and are closely linked to the six TMs. 

•  Outcomes, which are expected to be achieved over time as a result of and activities. 

These include:  

-  Long-term  cooperation  between  the  alliance  members  and  other  R&I  ecosystem 

actors  

-  Progress towards institutional transformation/modernisation of HEIs 

-  Strengthened  cooperation  towards  raising  excellence,  global  competitiveness  and 

general attractiveness for international talent and investment 

-  A portfolio of successful cooperation models for modernisation and transformation in 

R&I capabilities  

-  Synergies between the EEA and the ERA in relation to the European Universities 

alliances. 

•  The ultimate impacts of the Horizon 2020 support are expected to include: 

-  Enhanced scientific and technological capacity of the alliance members 

-  A higher excellence profile for the participating HEIs and their staff 

16 

 
 
-  Strengthened networks for ongoing R&I cooperation.  

A key point is to highlight that systemic and sustainable collaboration is only likely to emerge 
through an ongoing transformational process lasting several years. As the 17 earliest 
R&I projects from the first call in 2019 only began in earnest in 2020 (and those for the 2020 
call  at  the  start  of  2021),  there  has  only  been  three  years  of  collaboration  to  date  for  the 
longest established projects. Their work has also coincided with the covid pandemic whose 
effects  was  a  significant  complicating  factor.  As  a  result,  the  evaluation  questions  can 
mainly be answered in relation to the potential future impact of the funding – i.e. at the 
stage of outcomes rather than impacts. 

1.4  Evaluation questions  

The overall study answers the following evaluation questions (EQs). Following the terms 
of reference for  this  study, this analytical report addresses EQs 1  - 10, while  the 
accompanying  policy  report  focuses  on  EQs  11  and  12  which  set  out 
recommendations for the European Commission. 

Impact within the alliance 

•  EQ1: To what extent do the participating universities show transformational progress on 

the R&I dimension because of the Horizon 2020 support to the EUI? 

•  EQ2:  What  is  the  impact  and  transformational  potential  of  the  EUI  on  the  level  of 

cooperation with the wider innovation eco-systems surrounding the universities? 

•  EQ3: To what extent have the European Universities engaged internationally (beyond 

the EU)? 

•  EQ4: In general, what are the strengths, weaknesses, opportunities, and threats of the 
Horizon 2020 support mechanism for European Universities on the establishment and 
future consolidation of the alliances? 

•  EQ5:  What  is  the  transformational  potential  of  the  transnational  and  integrated 
cooperation  among  HEIs  in  European  Universities  alliances  as  supported  by  Horizon 
2020, in synergy with Erasmus+? 

Impact outside the alliance 

•  EQ6:  To  what  extent  has  the  EUI,  as  supported  by  Horizon  2020  in  synergy  with 

Erasmus+, affected universities and university policies in EU Member States? 

•  EQ7: To what extent can European universities inspire other HEIs that are not part of the 

alliances and induce change in the university sector with their good practices? 

•  EQ8:  To  what  extent  have  the  European  Universities  inspired  the  broader  university 

sector in the way they engage with their local knowledge ecosystems? 

•  EQ9:  To  what  extent  have  the  European  Universities  inspired  the  broader  university 

sector to engage and induce change internationally? 

17 

 
 
•  EQ10: In general, what are the strengths, weaknesses, opportunities, and threats of the 
Horizon 2020 support mechanism for European Universities in terms of impact on other 
HEIs in Europe, and on international cooperation beyond Europe by the alliances? 

Recommendations for the future 

•  EQ11: What are the recommendations for improvements and lessons to learn as regards 
future support to European  Universities from Horizon Europe (or following Framework 
programmes)? 

•  EQ12: What are the recommendations towards more sustainable funding for (a selection 
of)  the  alliances,  including  how  to  create  synergies  between  different  EU  funding 
programmes  and  with  national  funding,  to  reinforce  the  impact  of  future  support 
mechanisms? 

1.5  Sources of information used for the evaluation 

The evaluation draws on evidence from a range of primary and secondary sources, including 
the following. 

1.5.1  Secondary sources 

•  Data from previous surveys providing performance indicators for the universities involved 

in the EUI initiative  

•  Policy  briefs  and  monitoring  reports  (periodic  technical  reports)  for  the  SwafS  grants, 

submitted by the alliances  

•  Policy  documents  and  reviews  provided  by  higher  education  (HE)  stakeholder  and 

alliance umbrella organisations  

•  Data  on  grant  applications  from  the  Horizon  Dashboard  –  the  repository  of  data  on 

Horizon grants for the period 2014-2020. 

•  The findings of a separate progress report on the alliances’ R&I work commissioned from 
independent  consultants  by  DG  RTD.5  This  used  many  of  the  same  sources  as  this 
evaluation (i.e. policy briefs and other monitoring reports) and key points from its analysis 
are compared with the findings from the evaluation primary research. 

1.5.2  Primary sources - online survey 

An online survey of the alliances and their individual member universities was implemented 
during  October  –  December  2022.  This  made  use  of  the  monitoring  methodology  and 
indicators developed to assess progress on R&I issues by the study ‘Knowledge ecosystems 
for the new ERA’.6 

There were 117 responses from different HEIs to the survey, including participation by 29 
alliances, out of a total of 39 that received Horizon grants (74%).7 Of the 117 responses, 41 

5 Report on Progress of University Alliances to Realise Transformation Modules (Draft 2) 
6 Study on ‘Knowledge ecosystems in the new ERA: a comprehensive analysis of the state of play, the design 
of monitoring mechanisms, and creation of a toolbox of support measures. 
7 i.e. in some cases several different HEIs from the same alliance responded to the survey. 

18 

 
 
(35%) were from institutions that received funding from the Horizon call in 2019 and 76 from 
the call in 2020 (65%).  

All charts and tables in the rest of this report are based on the data from this survey, except 
where otherwise indicated. 

1.5.3  Primary sources - interviews 

Interviews were carried out with a total of 65 different individuals from the following groups 
(see Annex 1 for further details):  

•  Coordinators or other representatives from HEIs in 27 of the 39 European Universities 

alliances that received funding from the 2019 and 2020 calls for proposals. 

•  A  range  of  stakeholder  organisations  working  in  the  HE  sector  at  European  level, 

including those representing groups of alliances or collections of universities. 

•  Representatives of 14 national Member States organisations including both government 
and  HE  sector  officials.  Countries  that  responded  to  requests  for  interview  included 
Austria,  Belgium,  Bulgaria,  Denmark,  Finland,  France,  Germany,  Hungary,  Italy, 
Lithuania, Poland, Slovenia, Spain and Sweden.8 

These interviews were carried out on the basis of not attributing comments to any individual 
or organisation (except with the explicit agreement of the interviewee). 

1.6  Assessing the strength of the evidence and addressing potential bias 

The variety of data for this evaluation have been purposely selected so that conclusions can 
be drawn by comparing different primary and secondary sources i.e. triangulation. This helps 
to ensure that potential bias in research findings is reduced or avoided. Such bias can arise 
in a number of ways, for instance by the evaluators placing undue emphasis on a single data 
source  or  where  respondents  do  not  answer  truthfully  if  they  fear  this  might  prejudice 
relationships  or  future  funding.  Alternatively,  respondents  (e.g.  alliance  coordinators 
responding to the online survey) may not have a full understanding of what is happening ‘on 
the ground’. However, we believe we have effectively mitigated these potential  biases (by 
using  a  range  of  sources  for  the  evaluation,  by  making  survey  and  interview  responses 
confidential, and drawing on the collective views of all 117 respondents to the survey) and 
other ones, and therefore that overall the evaluation draws reasonable conclusions based on 
the evidence collected.  

However, there is one key issue that arose in the analysis process for the evaluation which 
requires further discussion. This stems from the distinct viewpoints of several HE stakeholder 
interviews  and  of  the  consultants  who  produced  the  separate  progress  report  mentioned 
above – both of which contrasted to some degree with the overall picture from responses to 
the  online  survey  of  HEI  representatives.  It  should  be  pointed  out  that  these  differences 
represent different nuances regarding the results of the Horizon funding, rather than outright 
contradiction. 

In particular there is a greater degree of scepticism about the results of the Horizon 2020 
funding  among  some  stakeholders,  and  a  more  nuanced  assessment  of  progress  by  the 

8 Details of representatives for consultation in 19 Member States were provided by DG RTD for this study. For 
five  Member  States  (Netherlands,  Estonia,  Slovakia,  Czech  Republic  and  Portugal)  it  was  not  possible  to 
obtain  an  interview  during  the  evaluation  timescale.  The  evaluators  assess  however  that  there  is  a  good 
evidence base for the conclusions drawn. 

19 

 
 
separate consultant study compared to the picture suggested by the combined views of the 
117 respondents to the survey.  

While it is important to highlight this issue and bear it in mind when reviewing the conclusions, 
we believe that the different viewpoints in responses can be in fact be largely reconciled. In 
particular,  a  significant  area  of  difference  lies  in  the  distinction  between  what  has  been 
achieved through the implementation of the SwafS projects, and what  may be achieved in 
future, given the groundwork that the projects have now undertaken.  

This  issue  is  addressed  explicitly  below  in  the  answer  to  evaluation  question  1  on  the 
transformational potential of the Horizon-funded R&I projects.  

In discussing the results of the online survey we have also included actual examples from 
the work of the alliances identified from alliance project documents. These are set out in 
boxes in the discussion of each TM  – in order to illustrate what results from the survey 
mean in practice in specific cases. The fact that there are compelling real-world examples 
to exemplify results for particular advances  also helps to validate  the survey findings in 
each case. 

1.7  Note on names of alliances and alliance projects 

The rest of this document refers to both alliances, and the R&I projects delivered by each 
alliance  using  the  Horizon  funding.  Typically  the  names  are  so  similar  that  the 
correspondence  is  obvious  (e.g.  EUGLOH  alliance,  EUGLOHRIA  project;  UNA  Europa 
alliance, UNA-RESIN project etc). Where this is not the case, the link between an alliances 
and its R&I project is made clear in the text or a footnote). 

20 

 
2. 

The Transformational potential of Horizon funding 

This  section  addresses  EQ1  (“What  is  the  transformational  potential  of  the  Horizon  2020 
funding) and EQ2 (“How far do participating universities show transformational progress on 
R&I issues”). It summarises the evidence from different perspectives on the transformational 
potential from the alliances’ collaboration on R&I issues.  

This includes the views of Member State representatives and HE stakeholder organisations 
as  discussed  in  interviews,  as  well  as  the  collective  views  of  alliance  member  HEIs  as 
revealed through both the online survey and interviews. In general terms the latter are more 
strongly  positive  than  the  former  -  suggestions  on  how  different  perceptions  can  be 
interpreted is discussed below. 

2.1  Views of Member State representatives on transformational potential 

Interviews  were  carried  out  with  representatives  of  different  government  and  R&I 
bodies in 14 Member States. These highlighted strong overall support for the Horizon 
initiative,  a  perception  that  the  projects  implemented  so  far  have  been  value  for  the 
participating institutions, and agreement that  – in theory at least – these projects have the 
potential to help transform the R&I capabilities of participating institutions.  

However,  there  was  considerable  nuance  within  this  overall  view  about  what  the  Horizon 
2020 funding will achieve. A key point highlighted by Member States’ representatives (as well 
as by HE stakeholder organisations and members of alliance HEIs) is that relatively small 
grants  (which  are  low  compared  to  other  R&I  funding  in  general)  are  unlikely  to  have  a 
strategic impact on their own – achieving this will require sustained and more predictable 
funding over several years beyond the timescale of the Horizon funding.  

There is a common view that the education and mobility elements of the alliances have been 
more transformative than the R&I dimension, reflecting more time and resources, as well as 
the fact that the alliances were designed with education goals in mind with R&I ‘bolted on’ at 
a later date. Some countries noted that their own national excellence initiatives were more 
influential in promoting change in HEIs, even though the international element of the alliances 
is valued. It is also widely considered too early to see significant changes. As one interviewee 
summarised  it,  the  Horizon  2020  funding  and  timescales  “are  not  yet  enough  for 
transformation”. 

It is notable that the views of Member States’ representatives are generally more sceptical of 
the  transformative  potential  of  the  Horizon  funding  than  the  picture  reviewed  through  the 
collective views of the 117 representatives from alliance members who completed the survey. 
Views from the survey (as discussed in detail below), present a more positive picture of the 
overall  results,  with  around  a  third  of  respondents  indicating  collaboration  within  their 
alliances is making a significant contribution in developing their R&I capabilities, and about 
half saying it has been useful (albeit to a lesser extent). 

2.2  Views of key stakeholders on transformational potential 

There is a common view across stakeholders interviewed for this evaluation ((see Annex 1 
for details) that across all the dimensions of the alliance (including educational aspects) the 
alliances  represent  a  significant  transformation  of  the  HE  sector  in  Europe.  For  instance, 
universities  have 
in-depth  bilateral  relationships  with  each  other. 
Subsequently  the  formation  of  networks  of  universities  (e.g.  The  Guild,  CESAER  Group, 
LERU etc) has helped to represent the collective voice of institutions to policy makers but has 
not  involved  substantial  collaboration  between  members.  In  general  terms  the  alliances 

traditionally  had 

21 

 
represent  a  significant  change,  providing  a  mechanism  with  great  diversity  and  therefore 
strength.  

However,  in  terms  of  the  specific  impact  of  the  Horizon  2020  funding,  the  interviews 
highlighted more nuanced opinions regarding its transformational potential. These range from 
quite negative (“the funding provided will not lead to transformative change”) to more positive 
views. As with Member State representatives, a point  made repeatedly is that the level of 
funding is low, in particular compared to the transformational ambitions of the initiative. It was 
also repeatedly mentioned that the absence of certainty about future funding is a significant 
limiting factor. It was also widely noted that three years that have passed since the start of 
the  earliest  R&I  projects  (i.e.  those  from  the  2019  call)  is  also  too  short  to  see  such 
transformative impacts, particularly given the impact of the pandemic. Other critical points on 
transformational potential included: 

•  The potential for R&I transformation is weakened by the siloed nature of the two main 
funding streams (Erasmus+ and Horizon). For the more critical stakeholders, this reflects 
the lack of a truly transformational vision for the European Universities, which would need 
to be based on a programmatic funding approach that connects funding from regional, 
national and EU levels. 

•  Perceptions  that  the  funding  has  been  too  prescriptive  and  has  not  provided  enough 
flexibility for alliances to decide themselves how to transform their R&I performance.9 

• 

In  some  cases  work  on  joint  research  is  a  continuation  of  what  has  been  previously 
undertaken e.g. universities which already have an interdisciplinary research hub have 
used the funding to expand this – but this falls short of a genuine common R&I strategy. 

•  Some also mentioned that most of the Horizon 2020 funding has been used to develop 
structures for collaboration (rather than for undertaking research) – although others cited 
this as a positive feature (i.e. this is a necessary step in creating capabilities and therefore 
a solid foundation for sustained ongoing collaboration). 

•  A  concern  was  also  raised  that  the  initiative  (and  forthcoming  European  Excellence 
Initiative)  would  focus  on  strengthening  the  capabilities of  widening  countries  –  at  the 
expense of excellence (while also accepting that focusing on widening is important and 
valuable). This view is summed up in the comment “the question is what is the initiative 
for - if we want to make sure there are players in Europe that are the best in the world, 
we  need  a  different  type  of  thinking”.  This  concern  is  of  particular  interest  as  field 
research for this evaluation indicate that the results of the Horizon funding are indeed 
significantly stronger in widening than non-widening countries, as discussed below. 

However,  more  positive  points  were  also  made  by  stakeholder  organisations  even  while 
accepting the limitations of what relatively low levels of funding could achieve. It was noted 
for instance that the alliances have led some Member States to provide co-funding for the 
alliances  (including  their  R&I  dimension  e.g.  Ireland  and  Austria  have  done  this  recently). 
Another interviewee mentioned that conversations with university vice rectors for research 
frequently turn to the subject of the alliances, and that change is also visible in particular in 
terms of applications for Erasmus grants, which are now starting to highlight synergies with 
R&I elements. 

9 However, there is an argument that this is more a matter of perception than fact, as the Calls for Proposals 
highlighted  the  fact  that  alliances  did  not  need  to  submit  proposals  for  work  under  every  transformational 
module, and also that they could propose other themes if they wished. 

22 

 
 
It  is  also  widely  recognised  that  collaborative  transformation  of  R&I  capabilities  will 
necessarily be a long-term process given that universities are typically bottom-up institutions 
and unlike, for instance, private companies in the speed at which they can change.  

It  was  also  suggested  by  some  stakeholders  that  the  size  of  the  participating  universities 
could be an important issue in the relative pace of change. For instance, larger institutions 
are more likely than smaller ones  to already be embedded in a rich ecosystem, with well-
developed partnerships with industry and other players. An important related point raised 
by some HEI representatives is that smaller universities often prioritise the alliances 
to a greater extent than larger ones who may have other means for R&I cooperation 
(with  collaboration  in an  alliance  being  just one  of  these).  Notably,  the  survey  results 
confirm smaller HEIs tend to benefit more than larger ones – see Figure 42 below. 

Results may also be more apparent in new areas of work compared to those where existing 
ways  of  working  create  inertia.  Examples  include  sharing  research  infrastructure,  where 
some stakeholder organisations have noticed significant positive change. A similar point was 
made by an HEI interviewee who noted that while change in the education field is slow due 
to differing laws and regulations, the way that institutions engage with their R&I ecosystems 
or  conduct  citizen  science  is  not  regulated,  providing  the  potential  for  faster  and  more 
significant advances.  

Finally,  other  interviewees  highlighted  the  fact  that  while  the  longer-term  funding  is  a 
necessary condition for transformation, it is not sufficient, and that buy-in from academics at 
different levels within HEI and broader culture change are also essential ingredients. 

2.3  Results  from  the  survey  of  alliance  members  on  transformational 

potential 

The rest of this section discusses the results of the online survey, interviews and reviews of 
programme  documents  for  participating  HEIs.  This  provided  a  greater  amount  of  both 
quantitative  and  qualitative  data  than  interviews  with  stakeholders  and  Member  State 
representatives. The evidence is therefore presented in the following manner: 

•  Firstly, progress so far against each of the six TMs is presented to answer EQ2 (‘How 
far do participating universities show transformational progress on research & innovation 
issues?’).  For  each  TM,  the  extent  of  progress  on  various  R&I  issues  is  described, 
followed  by  an  assessment  of  a  key  issue  –  how  far  is  the  Horizon  2020  funding 
responsible for this progress (compared with other factors)? 

•  Evidence  is  then  considered  across  all  the  TMs  to  assess  the  overall  picture  (EQ1)  – 
what  do  the  results  indicate  about  the  potential  of  European  Universities  alliances  to 
transform the R&I performance of HEIs across Europe? 

•  The assessment then moves on to consider strengths and weaknesses of measures to 
promote  the  alliances’  R&I  capabilities,  as  well  as  opportunities  and  threats  for  their 
future establishment and consolidation (EQ5). 

•  A  final  assessment  of  EQs  1,  2  and  5  is  then  made,  triangulating  between  different 
sources to assess how the same finding is backed up by different sources (or whether 
the evidence is more nuanced or contradictory). 

23 

 
2.4  EQ2:  How  far  do  participating  universities  show  transformational 

progress on research and innovation issues? 

This section reviews results for each TM before providing an overall answer to the evaluation 
question. 

2.4.1  TM1: Developing shared R&I strategies 

This TM (in many ways providing groundwork for the other five TMs) involved HEIs working 
to  develop  a  common  research  and  innovation  agenda,  identifying  specific  topics  for 
collaboration along with accompanying action plans to put these strategies into practice.  

The  separate  study  commissioned  by  DG  RTD  identified  a  number  of  challenges  that 
alliances had encountered in developing shared R&I strategies.10 These included differences 
in  legal  and  financial  frameworks,  differences  in  research  culture  and  organisation  across 
institutions,  difficulties  in  aligning  concepts  and  priorities,  and  a  lack  of  involvement  of 
researchers in alliance collaboration. Although identified as challenges in this separate study, 
the fieldwork for this evaluation has also identified substantial progress within this TM, as set 
out in the rest of this sub-section. 

Figure 1 below illustrates that around two thirds of HEIs responding to the online survey have 
developed an R&I strategy (67%), with nearly all of these (65%) also having developed this 
with other alliance members.  

Figure 1 Has your university developed the following: 

80%

70%

60%

50%

40%

30%

20%

10%

0%

67%

65%

N=74 

25%

7%

25%

10%

Yes

No

Don’t know

An individual R&I strategy

A joint R&I strategy (with other HEIs)?

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Institutional  innovations  and  activities  that  underpin  and  advance  common 
research strategies  

A  key  advance  for  the  CIVICA  alliance  is  the  creation  of  a  governance  structure  to 
oversee the implementation of the R&I project funded through Horizon 2020. This includes 
a Permanent Design Team with vice presidents for research and senior faculty members 
from member HEIs which meets regularly to discuss project implementation, incorporate 
best practice from different members and to provide an overarching strategic body. The 

10 Report on Progress of University Alliances to Realise Transformation Modules (2023 – unpublished) 

24 

 
  
 
alliance identifies this body as contributing significantly to collaboration required to build 
a joint research agenda.11 

The  4EU+  alliance  cites  the  creation  of  a  legal  entity  (4eu+  European  University 
Alliance e.V.), the consolidation of its general secretariat and the establishment of a 
Virtual Development Office to pursue joint research collaboration (among other things) 
as significant steps enhancing the alliance’s ability to work towards a common research 
agenda.12 

Other alliances have developed their research agenda through other measures. Key steps 
in helping the Arqus alliance develop its Joint Research Action Plan (now being rolled 
out) included four large researcher networking workshops on the two focus areas of 
the project (artificial intelligence and climate change), leading to promising ideas on joint 
research projects involving Arqus researchers.13 

Figure 2 indicates that external stakeholders were nearly always involved (the first four bars 
sum  to  81%),  though  with  different  degrees  of  intensity i.e.,  a  range  of  stakeholders were 
usually involved, but not at all stages of defining the R&I strategy. The goal embodied in the 
quadruple helix model (involvement of stakeholders from business, government, academia, 
and societal actors) was in fact only fully realised for 21% of respondents (sum of the first 
two bars). 

Figure  2  Were  non-university  stakeholders  from  a)  business  b)  local/regional 
government  c) research  d)  civil  society  involved  at  all  stages  in  creating  the  R&I 
strategy, including defining goals? 

45%

N=74 

16%

15%

14%

5%

5%

50%

45%

40%

35%

30%

25%

20%

15%

10%

5%

0%

Stakeholders
from all these
groups involved,
at all stages

Stakeholders
from all these
groups involved,
but not at all
stages

Stakeholders
from some of
these groups
involved, at all
stages

Stakeholders
from some of
these groups
involved, but not
at all stages

None of these
groups were
involved

Don’t know

Source: survey of HEIs participating in alliances supported by Horizon 2020 

11 CIVICA Research Midterm Policy Brief 
12 TRAIN4EU-ERA Policy Brief 
13 Arqus R&I-ERA Policy Brief 

25 

 
 
 
 
 
 
Figure 3 highlights that 75% of respondent HEIs either already have an action plan to ensure 
R&I strategies have practical results or are developing one.  

Figure  3  Is  there  an  action  plan  translating  the  strategy  into  activities,  with 
responsibilities for different stakeholders? 

N=74 

35%

42%

50%

40%

30%

20%

10%

0%

14%

9%

Yes

In development

No

Don’t know

Source: survey of HEIs participating in alliances supported by Horizon 2020 

The  following  three charts  also  indicate significant  advances  based  on  having  an  R&I 
strategy. Figure 4 shows that 80% of respondents have developed or are developing new 
partnerships  as  a  result  of  the  strategy,  Figure  5  indicates  that  79%  have  initiated  R&I 
activities  as  a  result  of  the  strategy,  and  Source:  survey  of  HEIs  participating  in  alliances 
supported by Horizon 2020 

Figure 6 shows that 75% of respondents have already made new applications for funding or 
resources or are in the process of doing so.14  

Figure 4 Have any new partnerships been initiated because of the R&I strategy? 

50%

40%

30%

20%

10%

0%

47%

32%

N=74 

Yes

In development

9%

No

11%

Don’t know

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Development of a cross-alliance R&I partnering service 

The EUTOPIA alliance has established a Grants, Legal and Innovation Office (GLENN), 
with functions determined from a baseline study of all alliance members and staffed by 
R&I  experts  from  different  members.  The  office  has  developed  a  GLENN  response 
protocol  –  a  collective  procedure  to  be  activated  when  EUTOPIA  universities  need  to 
coordinate quickly in preparing joint applications at short notice. In parallel, the alliance 

14 NB the combined figures for ‘yes’ and ‘in development’ in Figures 3, 4 and 5 sometimes exceed the 67% of 
respondents  who  answered  ‘yes’  on  whether  their  HEI  had  developed  an  R&I  strategy  i.e.  Figure  1.  The 
explanation for this is that some respondents who answered ‘in development’ for Figures 3, 4 and 5 said that 
their HEI already had an R&I strategy, while others said it did not (with the implication being that the strategy 
is in the process of developed). 

26 

 
 
 
 
developed  an  MSCA  partnering  tool,  with  400  profiles  of  senior  researchers  and  their 
research  priorities  accessible  from  a  dedicated  website.  This  allows  postdoctoral 
candidates to identify supervisors and was used for the MSCA-PF call 2021.15 

Figure 5 Have any R&I activities been initiated because of the R&I strategy (within the 
university, or shared with other partners)? 

50%

40%

30%

20%

10%

0%

41%

38%

Yes

In development

9%

No

N=74 

12%

Don’t know

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure  6  Have  there  been  any  applications  for  new  resources  or  funding  at  the 
European, national or local level as a result of the R&I strategy? 

60%

40%

20%

0%

49%

26%

Yes

In development

19%

No

N=74 

7%

Don’t know

Source: survey of HEIs participating in alliances supported by Horizon 20201617 

Responses for this question were compared with data on successful applications for Horizon 
grant funds from the Horizon Dashboard (database on grants) to see if relevant trends could 
be identified (e.g. an increase in grant awards after the start of a Horizon 2020 project).18 Five 
alliances  were  randomly  selected  for  this  exercise,  and  the  performance  of  their  31 
constituent  HEI  members  assessed.  The  results  (not  shown  here)  in  fact  reveal  a  small 
annual increase in the number of grants won by these HEIs for 2018 through to 2020, followed 
by 41% fall from 2020 to 2021.  

However,  this  very  significant  fall  is  judged  to  be  more  likely  to  reflect  the  hiatus  as  the 
Horizon 2020 programme came to an end rather than due to any effect of the alliances. While 
there are examples of alliance members increasing their funding as a result of collaboration, 
given the timescales, it is unlikely that this would yet be significant across all alliances. 

The charts above indicate that a substantial majority of HEIs responding to the survey 
have made progress on both an overall R&I strategy and in subsequent and related 

15 EUTOPIA-TRAIN 1st Periodic Technical Report 
16 For ease of comprehension, results have been rounded up in this and all of the following charts from the 
online survey. As a result, in some charts the totals sum to 99% or 101%. 
17 e.g. Marie S. Curie, INFRA, clusters, Invest EU or other. 
18 https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/horizon-dashboard  

27 

 
 
 
 
R&I activities, including development of partnerships and grant applications. An important 
qualifier  to  this  (and  for  similar  results  for  other  TMs  as  described  below)  is  that  in  a 
substantial  proportion of  cases this work is ‘in development’  – indicating important 
progress,  but  also  indicating  that  alliances  are  still  in  the  early  stages  of  the 
transformative process. 

FILMEU – successful funding bids resulting from the Horizon R&I funding  

The Horizon 2020 Funding has made a significant contribution in helping members of the 
FILMEU alliance secure funding for joint research interests. While the different member 
institutions had previously worked together, the funding enabled them to develop a joint 
research agenda and to define research clusters (which have researcher from at least 3 
institutions) on specific themes. Results so far include three successful bids for Horizon 
funds and one for a European Institute of Innovation & Technology grant.19 

A  key  question  that  follows  is  the  extent  to  which  the  Horizon  2020  funding  for  alliances 
contributed to these changes. 

Figure  7  illustrates  that  participating  in  an  alliance  was  indeed  a  significant  factor  in 
prompting  these  outcomes:  51%  (i.e.,  28%  +23%)  said  participation  was  either  very  or 
quite significant in developing an R&I strategy, 66% said the same about the formation of 
new R&I partnerships and 62% said this about new funding applications. 

It is notable that the corresponding figures for those saying that participation in an alliance 
had no influence or very minor influence on the R&I results was 36% for an R&I strategy (5% 
+ 31%), 23% for new partnerships and 23% for new funding applications. This illustrates 
that around a quarter to a third of HEIs would probably have made advances in these 
areas in any case without the Horizon 2020 support (i.e., deadweight). Responses to 
the stakeholder consultation suggest that these are more likely to be larger and better-
resourced universities. 

Figure 7 Did your participation in the alliance influence the development or updating 
of: 

60%

50%

40%

30%

20%

10%

0%

50%

N=74 

31%

28%

23%

19%

16%

8%

5%

7%

5%

4%

38%

24%

19%

No influence at all

Very minor influence

Quite significant influence

Very significant influence

12%

Don’t know

3%

4%

3%

Not applicable

An R&I strategy

New partnerships for R&I

Applications for new
funding or resources

19 Interview with FILMEU alliance HEI 

28 

 
 
 
 
Source: survey of HEIs participating in alliances supported by Horizon 2020 

Successful mechanisms to engage researchers across alliance institutions 

Engaging with researchers in delivering R&I projects has been a challenge in some cases, 
in  part  because  institutional  capability  building  exercises  are  often  removed  from  the 
process  of  producing  research  outputs  which  are  a  key  criterion  for  assessing  career 
success.  However,  the  monitoring  reports  and  policy  briefs  provided  by  the  alliances 
outline a number of different strategies which have helped to promote this engagement. 
A common theme for all these examples is linking the development of joint research 
strategies with funding opportunities for researchers, even if indirectly. 

EUTOPIA-TRAIN has undertaken a number of activities of direct interest to researchers. 
In addition to the GLENN Response Protocol (see above), the alliance has also mapped 
funding  opportunities  to create an online  atlas displaying the main research and 
mobility opportunities from each EUTOPIA university.20 

CIVICA-RESEARCH launched an alliance-wide call for research proposals in spring 
2021,  receiving  27  applications  from  cross-institutional  research  teams,  and 
crowding in a total of EUR 400 000 in additional funding from alliance members. This 
was  regarded  as  a  strong  success:  “this  high  number  of  applications…  [represents]  a 
bottom-up  dynamic  that  is  crucial  for  the  alliance  in  order  to  feed  reflection…  on  the 
scientific policy of CIVICA for the coming years… a total of 11 projects were selected… 
within these projects the CIVICA scientific teams are exploring future collaborations that 
one day will lead to applications to bigger grants at the national or European level”.21 

SMART-ER: “A total of 66 events were organised by the ECIU SMART-ER partners, 
with an audience of over 1 340 researchers. The internal engagement events proved 
to be an efficient way of building the motivation of researchers, resulting in their active 
participation  in  the  calls  issued  by  the  SMART-ER  Seed  Programme”.  This  had  total 
funding of EUR 380 000 and received 50 applications for two calls, with 435 researchers 
participating. Efforts were deliberately made to ensure that the application process was 
streamlined, to reduce the bureaucracy for applicants.22  

Figure 8 presents responses to a further question on the specific contribution of the Horizon 
funding. This shows 29% saying that the Horizon funding has been very or quite significant 
in helping develop an R&I strategy, and 65% saying this it has been somewhat helpful. 

20 EUTOPIA-TRAIN 1st Periodic Technical Report 
21 CIVICA Research Periodic Technical Report 
22 SMART-ER Mid-Term Report 

29 

 
 
 
Figure  8  To  what  extent  has  collaborating  with  other  universities  within  TM1 
Developing shared R&I strategies and roadmaps, helped your university advance? 

70%

60%

50%

40%

30%

20%

10%

0%

65%

N=74 

11%

18%

7%

Very significantly

Quite significantly

Somewhat

Not at all

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure 7 and Figure 8 highlight a consistent trend repeated in roughly similar proportions for 
different TMs throughout the online survey results: 

•  A substantial minority of HEIs (typically between a quarter and a third, depending on the 
issue)  indicate  that  participation  in  an  alliance  has  provided  substantial  benefits  in 
developing R&I capabilities. 

•  A substantial majority (typically between half and two thirds) indicate that the initiative 

has been less transformational but nonetheless of value. 

•  A relatively small proportion – around 10% for all TMs (except for TM4 where it is 19%) 

– say participation has not helped them advance at all. 

The  issue  of  which  kinds  of  HEIs  are  most  likely  to  say  participation  was  very  or  quite 
significant,  somewhat  helpful  or  of  no  benefit  (e.g.  those  in  specific  geographic  areas)  is 
discussed further below. 

2.4.2  TM2: Sharing capacity, infrastructure, and resources 

Sharing infrastructure and resources within alliances covers both large-scale infrastructure 
such  as  instruments  or  laboratories,  research  services  and  human  resources  including 
research  support  staff.  The  separate  DG  RTD  progress  report  on  this  issue  noted  that 
although some alliances have reported clear progress in this area, many have not yet done 
so. This contrasts with the picture provided by the online survey results – reconciliation of 
these contrasting viewpoints is discussed at the end of this subsection. 

Survey results on issues falling under TM2 indicate substantial progress in this area. 
Figure 9 below shows that 92% of respondents have either developed or are in the process 
of developing plans to share research infrastructure with other members of their alliance.  

30 

 
 
Figure 9 Has your university developed plans for sharing research infrastructure with 
other alliance members? 

N=65 

60%

50%

40%

30%

20%

10%

0%

55%

37%

Yes

In development

2%

No

6%

Don’t know

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure 10 confirms that 58% of HEIs have developed an inventory of research infrastructure 
(e.g. scientific equipment, archives, computing systems etc.), and Figure 11 shows that 76% 
have developed or are developing policies to support sharing. 

Figure 10 Does your university have an up-to-date inventory of research infrastructure 
which can be shared with other universities? 

N=65 

70%

60%

50%

40%

30%

20%

10%

0%

58%

34%

Yes

No

Don't know

8%

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure  11  Does  your  university  have  policies  and/or  guidelines  defining  how  other 
universities can access its research infrastructure? 

N=65 

60%

50%

40%

30%

20%

10%

0%

48%

28%

15%

9%

Yes

In development

No

Don’t know

31 

 
 
 
 
 
 
Source: survey of HEIs participating in alliances supported by Horizon 2020 

EUGLOH:  measures  to  start  sharing  research  infrastructure  among  alliance 
members 

The European University Alliance for Global Health has worked to promote the sharing of 
infrastructure  through  a  work  programme  of  different  elements.  These  include  the 
development  of  a  Memorandum  of  Understanding  covering  the  basic  financial  and 
legal aspects of common use of core facilities, a handbook of operational regulations 
for  the  facilities,  and  a  study  tour  and  workshops  to  share  good  practice  among 
academics and research infrastructure managers, including cost models, IP issues and 
data  management  provisions.  A  web  interface  is  now  being  created  to  provide 
researchers from alliance members with access to information on the various facilities.23 
While these measures provide a good basis for sharing research infrastructure, it is too 
soon yet to determine what the results have been in opening up the alliance’s facilities to 
all of its members. 

Figure 12 and Figure 13 then consider the key issue of how far respondents attributed these 
developments to their participation in an alliance. Figure 12 addresses the question of which 
areas alliance participation were most useful: 

•  The most significant of these relates to the development of plans with other universities 
on  sharing  research  infrastructure  –  with  69% saying  that  the  alliances  had  a  very  or 
quite significant influence in supporting this. (22% said the alliance had no or very minor 
influence). 

•  The equivalent figure for development of inventories is lower (56%), but higher than the 

proportion saying the alliance had no or only very minor influence (32%).  

• 

In terms of developing a policy on sharing infrastructure, 41% say the influence alliance 
was  very  or  quite  significant,  matching  the  proportion  saying  participation  had  no 
influence or only a very minor influence (also 41%). 

23 EUGLOHRIA 1st Periodic Technical Report 

32 

 
 
 
 
Figure 12 Did your participation in the alliance influence the development or updating 
of: 

N=65 

50%

45%

40%

35%

30%

25%

20%

15%

10%

5%

0%

38%

23%

18%

32%

29%

46%

23%

17%

9%

9%

9%

12%

12%

2%

5%

5%

5%

5%

An inventory of research
infrastructure

A policy on sharing access
to research infrastructure

Plans with other
universities on sharing
research infrastructure

No influence at all

Very minor influence

Quite significant influence

Very significant influence

Don’t know

Not applicable

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure  13  illustrates that  36% said this  participation  was  very  or  quite  significant  in 
helping  them advance  in sharing  infrastructure,  capacity, and  resources  across  the 
alliance. 52% said it helped somewhat (compared with just 12% who indicated that factors 
not related to participation in the alliance were what made the difference). 

Figure 13 To what extent has collaborating with other universities within TM2 Sharing 
capacity, infrastructure, and resources, helped your university advance? 

52%

N=65 

28%

8%

12%

60%

50%

40%

30%

20%

10%

0%

Very significantly

Quite significantly

Somewhat

Not at all

Source: survey of HEIs participating in alliances supported by Horizon 2020 

33 

 
 
 
 
 
Arqus alliance: how collaboration is strengthening research support 

Universities  with  strong  research  offices  tend  to  be  more  successful  in  research  grant 
applications. These offices provide administrative and technical support, for instance for 
grant  applications,  partner  searches  and  research  management.  Arqus  has  set  up  a 
network  of  these  offices  so  that  alliance  members  can  learn from  each  other,  allowing 
large universities with well-established research offices to support those members with 
smaller ones. 

Although  responses  from  HEI  alliance  members  indicate  progress  towards  realising  the 
objectives of TM2, i.e. transformational progress, a key caveat needs to be made – which 
helps  to  align  this  finding  with  the  conclusion  of  relatively  limited  progress  in  the  separate 
report commissioned by DG RTD. 

Transformational progress on TM2 could mean more basic or more ambitious approaches: 

•  Different alliance members have developed effective approaches to start providing each 
other with access to their own libraries, laboratories or other infrastructure (as illustrated 
by the examples for EUGLOH and Arqus in this sub-section). 

•  Alternatively, a much more ambitious concept of transformation might involve creating a 
common research infrastructure, taking a strategic view across the alliance on who would 
provide what, with some members then specialising only in particular assets and relying 
on  other  member  HEIs  for  other  infrastructure  (rather  than  investing  in  these 
themselves). 

So  far,  the  evidence  suggests  that  progress  reflects  the  more  basic  rather  than  the  more 
ambitious approach to collaboration. It is worth noting in this context that  different national 
regulations limiting the autonomy of universities are a key barrier to cooperation in this and 
other  areas  (in  Sweden,  for  example,  specific  parliamentary  approval  is  required  before 
universities sign contracts with foreign entities).24 

Varying levels of ambition in sharing research infrastructure 

Within the EDUC alliance some universities have well-developed research infrastructure 
strategies,  with  central  coordinator  positions  within  the  rectorate  and  involvement  in 
research infrastructures at national and European levels. In contrast, others are only at 
the start of the process of understanding the concept of research infrastructures.25 

2.4.3  TM3: Strengthening human capital (inc. gender equality and research assessment 

reform) 

TM3 focuses on the aim of nurturing and rewarding research talent and giving researchers 
the  opportunity  to  pursue  career  paths  across  their  alliance.  The  separate  progress  study 
commissioned by DG RTD noted the most common and significant barrier to this mobility is 

24  EUA  (2022)  The  European  Universities  Initiative  and  system  level  reforms  –  current  challenges  and 
considerations for the future. 
25 EDUC-SHARE Project – Periodic Technical Report 

34 

 
 
 
 
different frameworks for human resources (underpinned e.g. by different labour or pension 
laws in different Member States).  

The  same  study  notes  that  while  the  comprehensive  mapping  and  analysis  exercises 
undertaken in the last 18 months will provide a strong practical basis for mobility initiatives, 
in  some  cases  the  underlying  issues  have  been  addressed  by  reducing  the  level  of 
ambition (e.g. reducing the number and duration of cross-alliance postdoctoral contracts, as 
was the case with the CIVICA alliance). These caveats should be taken into account when 
assessing the extent of progress in this TM described in the rest of this sub-section. 

Figure  14  indicates  that  73%  of  respondents  have  an  HR  strategy  or  action  plan  for 
researchers or are developing one. 

Figure  14  Does  your  university  have  an  explicit  HR  strategy  or  action  plan  for 
researchers (e.g. Human Resources Strategy for Researchers HRS4R)? 

57%

70%

60%

50%

40%

30%

20%

10%

0%

26%

15%

N=61 

2%

Yes

In development

No

Don't know

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure  15  shows  that  a  very  large majority  (89%)  of  universities  have  developed  equality, 
diversity and inclusion strategies (e.g. covering gender, ethnicity, sexual orientation, health, 
religion, disability etc.), with nearly all of the remainder currently doing so. 

Figure  15  Does  your  university  have  an  equality,  diversity,  and  inclusion  strategy  / 
action plan? 

100%

89%

80%

60%

40%

20%

0%

N=61 

10%

Yes

In development

2%

No

0%

Don't know

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure  16  highlights  significant  differences  in  the  extent  of  measures  to  promote  gender 
equality for female researchers. While 80% have a gender equality unit, and 54% a gender 
equality network, there are significantly lower levels of other measures such as programmes 
for mentoring (38%), training (33%) or scholarships (30%). This indicates areas where future 
funding support might more actively encourage these kinds of schemes.  

35 

 
 
Figure  16  Does  your  university  have  any  of  these  support  structures  for  female 
researchers? 

80%

N=61 

54%

38%

39%

43%

33%

25%

21%

23%

25%

49%

30%

Yes

No

21%

Not sure

12%

8%

90%
80%
70%
60%
50%
40%
30%
20%
10%
0%

Gender Equality
Office/Unit

Gender Equality
Network

Mentoring
Programme

Training
Programme

Scholarships

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure 17 highlights that while relatively few HEIs (16%) have yet adopted a formal strategy 
to  reform  research  assessment,  a  much  larger  proportion  (49%)  are  in  the  process  of 
developing one. 

Figure  17  Does  your  alliance  have  a  formal  strategy  for  reforming  research 
assessment? 

N=61 

60%

50%

40%

30%

20%

10%

0%

49%

16%

23%

11%

Yes

In development

No

Don’t know

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure 18 and Figure 19 identify the extent to which respondents credit participating in an 
alliance with helping their institution make progress in strengthening human capital for R&I. 
Figure 18 shows that 30% report that collaborating with other universities has made a very 
significant or quite a significant difference, compared to 59% who say that it has made some 
difference and 11% who report no advance. It is notable that these proportions are very 
similar to those reported above for TMs 1 and 2 using the same question. 

36 

 
 
 
Figure  18  To  what  extent  has  collaborating  with  other  universities  within  TM3 
Strengthening human capital, helped your university advance? 

59%

N=61 

20%

10%

11%

70%

60%

50%

40%

30%

20%

10%

0%

Very significantly

Quite significantly

Somewhat

Not at all

Source: survey of HEIs participating in alliances supported by Horizon 2020 

In terms of in which areas the alliance made most difference, Figure 19 shows that: 

•  Half  of  respondents  said  collaboration  had  had  a  very  or  quite  significant  influence  in 
developing an HR strategy, with the other half saying it had no or very minor influence. 

• 

44% said the alliance was very or quite significant in developing an equality, diversity 
and inclusion plan, although a larger number (50%) said it had no or minor influence). 

•  Roughly  the  same  proportion  said  the  alliance  was  very  or quite  significant in  helping 
revise frameworks for research assessment (43%) compared to 44% who said it made 
no difference or little difference). 

• 

64% said it had no impact or little impact in helping develop support structures for female 
researchers, with only 23% saying collaboration here had been quite or very significant.  

The benefits of developing a joint strategy on talent development 

These  benefits  have  “become  quite  concrete,  setting  up  a  joint  academy  with  shared 
training modules within the consortium… it’s wonderful to open up training courses for our 
own staff and to have participants from other universities in them. It improves the quality of 
staff training courses as well as the number of courses.”26 

26 Interview with alliance representative, HEI in the Netherlands 

37 

 
 
 
Figure 19 TM3: Did your university’s participation in the alliance have any influence 
on the development or updating of the following: 

31%

25%

25%

18%

34%

31%

38%

26%

16%

13%

15%

8%

7%

7%

2%

0%

3%

2%

40%

35%

30%

25%

20%

15%

10%

5%

0%

28%

25%

18%

16%

8%

5%

No influence at all

Very minor influence

Quite significant influence

Very significant influence

Don’t know

Not applicable

A human resource
strategy or action
plan for research
staff

An equality,
diversity or
inclusion strategy
or action plan

Support structures
for female
researchers

Its framework for
research
assessment

N=61 

Source: survey of HEIs participating in alliances supported by Horizon 2020 

2.4.4  TM4:  Reinforcing  cooperation  in  R&I  with  other  sectors,  incl.  knowledge 

valorisation 

This  TM  aims  to  support  alliances  to  reinforce  cooperation  with  non-academic  sectors, 
especially academia-business collaboration, and to promote the use of academic outputs for 
the benefit of society. The separate DG RTD progress report notes that initiatives to reinforce 
cooperation with other sectors vary from cautious to highly ambitious, reflecting the realities 
of different institutional cultures and development pathways for this issue, and the fact that 
some  areas  have  a  relatively  unsophisticated  private  sector,  limiting  cooperation  with 
universities. The results on progress for this TM should therefore be interpreted in light of this 
variety i.e. some universities will have made substantial progress, but from a low base, while 
others with stronger initial capabilities may also have advanced but less far.  

Different capabilities for engaging in R&I collaboration with other sectors 

Among  the  members  of  the  YUFE  alliance,  the  average  age  of  technology/knowledge 
transfer offices is 16 years. One member established its office 36 years ago while another 
does not yet have any such office. The scale of resourcing also varies very significantly, 
with office staff sizes varying from 1.5 to 20 Full Time Equivalent employees.27 

27 YUFERING Project – 1st Periodic Technical Report 

38 

 
 
 
 
 
 
 
The three figures below illustrate existing high capacity to engage with other R&I ecosystem 
actors.  For  instance,  Figure  20  shows  that  93%  of  respondents  have  created  spaces  or 
initiatives for collaboration (e.g. innovation labs, hubs, science parks or other initiatives such 
as networking events) or are in the process of doing so. These findings should be interpreted 
in light of results which suggest that collaboration in an alliance was a significant factor for 
around a third of respondents (see below). 

Figure 20 Does your university have R&I collaboration spaces or other initiatives to 
encourage engagement with external actors? 

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%

84%

N=57 

9%

Yes

In development

2%

No

5%

Don’t know

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure 21 illustrates that 90% offer entrepreneurship training for researchers or are planning 
to do so. 

Figure  21  Does  your  university  offer  training  for  doctoral  students  and/or  research 
staff to develop entrepreneurial skills? 

N=57 

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%

86%

4%

Yes

In development

5%

No

5%

Don’t know

Source: survey of HEIs participating in alliances supported by Horizon 2020 

39 

 
 
 
 
 
Figure  22  highlights  that  around  three  quarters  of  HEI  respondents  have  inventories  of 
existing collaboration partners in other research or education organisations, around two thirds 
have  this  for  private  and  public  sector  partners,  and  around  a  half  do  so  for  civil  society 
organisations. 

Figure  22  Does  your  university  have  an  inventory  of  existing  R&I  collaboration 
partners from other sectors? 

74%

67%

65%

49%

80%

70%

60%

50%

40%

30%

20%

10%

0%

14%

11%

Source: survey of HEIs participating in alliances supported by Horizon 2020 

The  issue  of  how  far  the  Horizon  funding  has  helped  the  respondents  develop  their  R&I 
ecosystem capabilities is highlighted below. Around a third of respondents (32%) said the 
alliance was either very or quite significant in helping them advance, and around half 
(49%)  said  it  had  helped  somewhat.  These  proportions  are  notably  similar  to  the  same 
answers for the other TMs reviewed above. 

Figure  23  To  what  extent  has  collaborating  with  other  universities  within  TM4 
Reinforcing cooperation in R&I with other sectors, helped your university advance? 

60%

50%

40%

30%

20%

10%

0%

49%

N=57 

23%

19%

9%

Very significantly

Quite significantly

Somewhat

Not at all

Source: survey of HEIs participating in alliances supported by Horizon 2020 

40 

 
 
How  far  does  sharing  good  practices  on  engaging  with  innovation  ecosystems  help 
alliance members? 

The EUGLOHRIA project28 undertook work to help members learn from each other, including 
mapping  each  partner’s  academia-business-society  innovation  value  chains  (e.g.  faculties, 
technology transfer organisations, incubators, policy institutions) and identifying best practices. 
Of  four  out  of  five  alliance  members  answering  the  question,  three  said  collaboration  on  R&I 
ecosystems  had  helped  ‘somewhat’,  and  one  said  it  had  helped  ‘quite  significantly’.  This 
suggests  R&I  dimension  of  the  alliances  has  value  in  strengthening  those  with  weaker 
capabilities, but also that benefits are often less substantial for those with significant experience 
and capacity.  

The relative benefit of participating in an alliance is illustrated for different elements of TM4 
in Figure 24 below. This shows: 

•  Very substantial benefits in terms of developing entrepreneurial training (with 60% saying 
participation was either very or quite significant, against 35% who said it had very minor 
or no influence). 

•  Substantial benefits in developing collaboration spaces or initiatives – with 46% saying 
the alliance was very or quite significant (albeit with 49% saying it had no effect or only 
a minor effect). 

•  Some benefits in terms of developing inventories of R&I collaboration partners (with 36% 
saying this was very or quite influential, but with 55% saying the alliance had no influence 
or only minor influence). 

Figure 24 TM4: Did your university’s participation in the alliance have any influence 
on the development or updating of the following: 

45%
40%
35%
30%
25%
20%
15%
10%
5%
0%

39%

25%

35%

32%

16%

14%

14%

42%

28%

18%

11%

7%

4%

7%

4%

2%

4%

2%

An inventory of
existing R&I
collaboration partners

R&I collaboration
spaces or initiatives

Entrepreneurial skills
training for doctoral
students or research
staff

N=57 

No influence at all

Very minor influence

Quite significant influence

Very significant influence

Don’t know

Not applicable

Source: survey of HEIs participating in alliances supported by Horizon 2020 

The  evidence  for  TM4  therefore  suggests  that  the  SwafS  funding  has  helped  a 
substantial minority of alliances to make significant progress (and a majority to make 
useful if less substantial advances) in this area. A caveat to this assessment was however 

28 i.e. the EUGLOH alliance 

41 

 
 
 
 
highlighted  in  a  stakeholder  interview,  which  noted  that  previous  Horizon  2020  funding 
opportunities  had  already  requested  collaborations  between  academia  and  other  sectors, 
prior to the SwafS calls, and in this sense created some impetus which the R&I funding 
for alliances has helped to sustain, rather than to initiate in its own right. 

2.4.5  TM5:  Transitioning  to  knowledge-  and  digitally  driven  HEIs,  conducting  open 

science 

TM5  aims  to  support  alliances  in  moving  towards  a  more  open  and  collaborative  way  of 
undertaking  research  and  sharing  findings,  including  by  developing  and  implementing 
strategies for open access and research data management. DG RTD’s separate study on the 
progress of the alliances highlighted that they have already made substantial progress in the 
implementation of open science, an assessment which aligns with the results of the online 
survey for this evaluation. 

Survey results for TM5 indicate a high degree of progress across different capabilities relating 
to open science and digital transformation. Figure 25 shows that nearly all alliance members 
responding to the survey have developed, are in the process or are planning to develop an 
open  science  policy  for  research  outputs  (e.g.  a  policy  for  producing  FAIR  (findable, 
accessible, interoperable, reusable) results). 

Figure 25 Has your university developed an Open Science policy for digital research 
outputs? 

39%

45%
40%
35%
30%
25%
20%
15%
10%
5%
0%

26%

20%

11%

Yes, developed
and implemented

Yes, developed
but not
implemented

Development in
progress

Not yet
developed but
planned

N=54 

0%

4%

Not planned

Don’t know

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure  26  shows  that  86%  of  respondents  have  put  arrangements  in  place  for  data 
governance and management or are currently developing these. 

42 

 
 
Figure 26  Does your university have a  designated person, unit or  structure for  data 
governance and data management? 

80%

70%

60%

50%

40%

30%

20%

10%

0%

69%

N=54 

17%

Yes

Under development

7%

No

7%

Don't know

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure  27  below  illustrates  that  nearly  all  HEIs  have  put  in  place  measures  to  help 
researchers  to  publish  open  access  outputs,  with  61%  providing  financial  support,  85% 
providing non-financial support, and 94% in total providing some combination of the two. 

EUTOPIA  TRAIN  –  challenges  and  solutions  in  implementing  on  open  science 
practices 

The  EUTOPIA  TRAIN  project  identified  specific  open  science  challenges  from  the 
differences in members’ recognition and reward systems, all operating in particular local 
contexts. It has addressed this by working to align these systems across a set of open 
science-related  dimensions  and  principles  through  a  Framework  Policy  for 
Assessment. It has also implemented a range of complementary activities to upskill staff 
and  promote  a  common  approach  on  open  science  issues  across  the  alliance.  These 
include  developing  an  Open  Science  and  Research  Data  Management  course  for 
researchers  and  data  stewards,  creating  guidance  documents,  webinars,  video 
recordings and learning materials on the EUTOPIA website to allow partner universities 
to embed these in their own internal systems.29 

29 EUTOPIA-TRAIN 1st Periodic Technical Report. 

43 

 
 
 
Figure 27 Does your university have measures to support researchers in publishing 
open access? 

60%

50%

40%

30%

20%

10%

0%

54%

N=54 

31%

7%

Yes, both financial
and non-financial
support

Yes, financial support Yes, non-financial

support

2%

No

6%

Don’t know

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure  28  also  highlights  that  the  very  large  majority  of  HEIs  provide  open  science  skills 
training  for  researchers  (e.g.  data  management,  communication,  public  engagement, 
participatory research methodologies). 

Figure  28  Does  your  university  provide  training  for  researchers  on  open  science 
skills?  

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%

83%

Yes

N=54 

9%

No

7%

Don’t know

Source: survey of HEIs participating in alliances supported by Horizon 2020 

The  following  two  charts  highlight  the  contribution  of  participating  in  an  alliance  to  the 
outcomes. Figure 29 illustrates that 37% of respondents felt that participation has made 
a very or quite significant contribution, compared to 54% who noted that it has helped 
them ‘somewhat’, and 9% who said it has made no difference. 

44 

 
 
 
 
 
Figure 29 To what extent has collaborating with other HEIs within TM5 Transitioning 
to  knowledge-  and  digitally  driven  HEIs,  conducting  Open  Science,  helped  your 
university advance? 

54%

N=54 

26%

11%

9%

60%

50%

40%

30%

20%

10%

0%

Very significantly

Quite significantly

Somewhat

Not at all

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure 30 indicates that the benefits of participation in an alliance were: 

•  Quite  valuable  in  helping  HEIs  develop  Open  Science  training  for  researchers  (50% 
saying participation was very or quite influential, compared to 41% saying it had no or 
little influence). 

•  Also  reasonably  strong  in  supporting  development  of  an  Open  Science  policy  (44% 
saying very significant or quite significant, compared to 52% citing low or no influence). 

•  Weaker (though still significant for some) in terms of helping HEIs support researchers 
in  open  access  publishing  (very  or  quite  significant  =  35%,  not  significant  =  57%), 
although these results are not particularly surprising given that open access policies and 
mandates by funders have been in place for a number of years (i.e. the entire Horizon 
2020 period). The influence of alliance collaboration is also weaker in terms of developing 
measures  for data  governance  and  management  (very  or  quite  significant  =  28%, not 
significant = 57%). 

45 

 
 
Figure 30 TM5: Did your university’s participation in the alliance have any influence 
on the development or updating of the following: 

35%

30%

25%

20%

15%

10%

5%

0%

31%

30%

22%

33%

33%

31%

28%

28%

26%

N=54 

24%

19%

No influence at all

17%

Very minor influence

13%

13%

9%

9%

6%

7%

6%

2%

6%

4%

4%

0%

An open science
policy

A designated
person, unit or
structure for data
governance

Measures to
support researchers
publishing in open
access

Training for
researchers in open
science skills

Quite significant influence

Very significant influence

Don’t know

Not applicable

Source: survey of HEIs participating in alliances supported by Horizon 2020 

2.4.6  TM6: Embedding citizens and society 

TM 6 aims to ensure that academic work is publicly engaged and that research is socially 
accountable  and  acceptable  and  increasing  its  potential  to  address  societal  needs.  The 
separate  progress  study  noted  that  alliance  HEIs  are  at  very  different  levels  in  terms  of 
readiness for public engagement here, and that there are regulatory, cultural and language 
barriers to progression. It also highlighted difficulties in sustaining engagement given impacts 
are likely to take considerable time. 

Survey results also indicate that the HEIs responding to the survey have also made significant 
progress in a range of areas relevant to TM6. Figure 31 shows that 88% have either created 
or are developing an institutional arrangement to support engagement with citizens or end-
users (e.g. science shops, maker spaces, a fablab, living labs). 

46 

 
 
 
Figure 31 Does your university have a department, centre, lab, network, testbeds, or 
other initiative to support citizen/end-user engagement? 

70%

60%

50%

40%

30%

20%

10%

0%

65%

N=52 

23%

10%

2%

Yes

Under development

No

Don't know

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure  32  similarly  illustrates  that  the  large  majority  (79%)  of  HEIs  already  have  or  are 
developing  activities  to  promote  citizen  involvement  in  scientific  research  (e.g.  regular 
dialogues, meetings, workshops, or other events, excluding one-off events). 

Figure 32 Does your university hold activities to promote citizen/end-user engagement 
in co-design, co-creation, and co-evaluation of science?  

60%

70%

60%

50%

40%

30%

20%

10%

0%

N=52 

6%

19%

15%

Yes

Under development

No

Don't know

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure 33 indicates that 89% of HEIs have developed or are developing activities to become 
more environmentally sustainable (e.g. a zero-pollution strategy, sustainable food strategy 
for catering, sustainable mobility plan etc.). 

47 

 
 
 
 
Figure  33  Has  your  university  developed  or  implemented  activities  to  make  it  more 
environmentally sustainable? 

70%

60%

50%

40%

30%

20%

10%

0%

60%

N=52 

15%

8%

8%

4%

6%

Yes, developed
but not
implemented

Yes, developed
and implemented

Development in
progress

No but planned No, not planned

Don’t know

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure 34 highlights that at least 80% of HEIs also have or are developing activities to support 
the  transfer  of  academic  knowledge  into  policy  (e.g.  regional  strategic  boards  or  other 
strategy development processes). 

Figure 34 Does your university participate in activities to transfer academic knowledge 
into policy? 

N=52 

80%

70%

60%

50%

40%

30%

20%

10%

0%

67%

13%

Yes

Under development

4%

No

15%

Don't know

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure  35  shows  that  around  a  third  of  responding  HEIs  say  that  participation  in  an 
alliance has helped them advance very or quite significantly in relation to TM6, with 
56% saying it has helped them somewhat and 10% saying it did not help them at all. 
Once again, these figures are similar to the same results for other TMs. 

48 

 
 
 
 
Figure  35  To  what  extent  has  collaborating  with  other  universities  within  TM6 
Embedding citizens and society helped your university progress? 

56%

N=52 

29%

6%

10%

60%

50%

40%

30%

20%

10%

0%

Very significantly

Quite significantly

Somewhat

Not at all

Source: survey of HEIs participating in alliances supported by Horizon 2020 

One HEI described how it had been able to learn from one of its alliance partners in Italy 
which has a very strong tradition of engaging with civic communities, and then to use this 
experience to refine its existing approach to citizen science.30 

Figure 36 illustrates that participation in an alliance has been: 

•  Of significant benefit in helping alliance members develop activities to transfer academic 
knowledge into policymaking (with 50% saying the influence was very or quite significant, 
compared to 42% saying the effect was minor or negligible). 

•  Also, of significant benefit in helping members develop initiatives for citizen engagement, 

(very or quite significant = 55%; minor or no influence = 38%). 

• 

Less  significant  in  terms  of  promoting  environmental  sustainability,  with  63%  saying 
alliance membership had no influence on them compared to 29% saying the effect was 
very or quite significant. 

30 Interview with alliance coordinator 

49 

 
 
 
 
 
Figure 36 TM6: Did your university’s participation in the alliance have any influence 
on the development or updating of the following 

42%

40%

N=52 

25%

25%

23%

13%

13%

6%

35%

27%

15%

15%

4%

4%

4%

No influence at all

Very minor influence

Quite significant influence

Very significant influence

Don’t know

Not applicable

45%

40%

35%

30%

25%

20%

15%

10%

5%

0%

Initiatives to support
citizen/end-user
engagement on a regular
basis

Activities to make it more
environmentally
sustainable

Activities to transfer
academic knowledge into
policy making

Source: survey of HEIs participating in alliances supported by Horizon 2020 

2.4.7  Overall answer to EQ2 

The  preceding  discussion  highlights  the  fact  that  the  alliances  have  made  good 
progress in all of the TMs. A key finding of the separate progress assessment of projects 
from the 2019 call commissioned by DG RTD also notes that progress with respect to grant 
agreement commitments has been satisfactory, and that the vast bulk of the activities 
planned  for  the  first  reporting  period  have  been  completed.31  This  progress  is 
particularly notable given the difficult circumstances created by the pandemic.  

Table 2 summarises this progress, using the EUI R&I monitoring indicators selected for this 
evaluation. This illustrates that alliances have delivered between 75% and 87% of the main 
deliverables for each TM (NB a score of less than 100% does not indicate a deficiency, as 
alliances undertook different activities depending on their specific needs and work contexts. 

31 Report on Progress of University Alliances to Realise Transformation Modules 

50 

 
 
 
 
Table 2 Progress against key activity indicators for each TM 

Transformational module 

Maximum 
possible 
score 

Average 
score 

Sharing 

capacity, 

TM1:  Developing  shared  R&I 
strategies and roadmaps32 
TM2: 
infrastructure and resources 
TM3: Strengthening human capital 
TM4:  Reinforcing  cooperation  in 
R&I with other sectors 
TM5:  Transitioning  to  knowledge- 
and 
HEIs, 
digitally-driven 
conducting open science 
TM6:  Embedding  citizens  and 
society 

6 

4 

4 
3 

4 

4 

Average 
score as 
% of 
maximum 
76% 

4.55 

3.05 

76% 

3.06 
2.57 

76% 
86% 

3.46 

87% 

3.35 

84% 

Number of 
respondents 

74 

65 

61 
57 

54 

52 

As noted in the separate progress study, many of the activities in the first reporting period 
were  primarily  concerned  with  data  collection,  mapping  exercises,  benchmarking  and  the 
analysis  of  results  of  this  work.  This  groundwork  was  essential  to  develop  a  mutual 
understanding of different members’ capabilities and needs, and to provide a key basis for 
future progress. However, many outputs will not necessarily lead directly to impacts.  

The  view  of  this  evaluation  is  that  the  results  in  terms  of  delivering  the  transformational 
modules  are highly  promising,  and  indicate  good,  but  early  progress  in  the  transformation 
process,  albeit  this  process  will  necessarily  be  a  long-term  endeavour,  requiring 
ongoing support. This leads to the broader question of the transformational potential of the 
alliances. 

2.5  EQ1: What is the transformational potential of the alliances? 

The evidence suggests a nuanced picture in terms of the potential for the Horizon 2020 
funding to transform the R&I capabilities of the alliances and their members: 

•  For a significant minority of alliances – around one third of the 39 alliances that received 
support  –  the  funding  does  indeed  have  the  potential  to  promote  significant 
transformation.  

•  For  around  half  the  alliances,  the  funding  received  is  supportive  of  the  member 
institutions  own  efforts  (often  supported  by  national  priorities),  but  not  likely  to  be 
transformational.  

32 For example, for TM1, the six points that could be scored include a positive response to the following 1) Has 
your alliance developed an R&I strategy? 2) Were non-university stakeholders from all four groups involved? 
3) Is there an associated action plan? 4) Have new partnerships been initiated because of the R&I strategy? 
Have any R&I activities been initiated because of the strategy? Have there been any new funding applications 
because of the R&I strategy?  

51 

 
 
 
•  For a small minority – perhaps a tenth of HEIs – participation in an alliance seems to be 
making little difference to efforts to build and refine R&I capabilities. NB these results in 
the  aggregate  obscure the  fact  that  among  this  sub-group  there are  some  benefits  to 
participating – but that these may be concentrated across a small number of TMs rather 
than across a wide range of areas. 

The evidence also suggests that while HEIs in all EU Member States have benefitted 
from  the  initiative,  the  gains  are  significantly  greater  for  those  in  the  widening 
countries.  

Reconciling the views of different groups consulted for this evaluation  

Some HE stakeholder organisations questioned the transformational potential of the SwafS 
funding,  based  on  interactions  with  their  own  members  as  well  as  the  relatively  small 
amount of funding (up to EUR 2 million SwafS grants per alliance) and relatively short period 
(just under 3 years, for the longest running R&I projects) for implementation.  

These results align with the approximately half of HEI respondents to the online survey who 
identified that collaboration within different TMs had helped them ‘somewhat’ (and with the 
roughly 10% of respondents who said it made no difference). However, they do not reflect 
the approximate third of respondents who said that collaboration has been quite significant 
or very significant in helping them advance.  

The most obvious reconciliation in differences is that for those respondents who indicate 
that  collaboration  has  been  a  significant  help,  this  relates  to  creating  the  potential  for 
improved R&I impacts which will predominantly be delivered over much longer timescales. 
However, full realisation of this potential will depend on whether the alliances find funding 
to continue their work and also on their ability to attract funding for actual research (rather 
than the development of joint strategies and structures etc which has been the focus so 
far).  

This interpretation confirms the value and effectiveness of the Horizon 2020 grants – i.e. 
the  initiative  has  had  considerable  success.  However,  it  also  highlights  the  need  for 
continued  support  at  EU  level,  and  also  that  there  are  other  risks  to  this  potential,  for 
instance,  posed  by  the  varying  levels  of  complementary  national  or  regional  funding 
available to different members of the same alliance. 

These  points  can  be  substantiated  further  with  the  following  evidence,  some  of  which 
summarises  for  the  initiative  as  a  whole  data  that  has  already  been  presented  above  for 
individual TMs.  

2.5.1  What was the distinct contribution of the Horizon 2020 funding to developing R&I 

capabilities? 

Figure 37 provides a comparative assessment of responses for all TMs together (reprising 
the results presented above for each TM individually).  

52 

 
 
Figure  37  To  what  extent  has  collaborating  with  other  universities  helped  your 
university advance? 

65%

28%

7%

80%

60%

40%

20%

0%

52%

35%

59%

30%

12%

11%

49%

32%

19%

54%

37%

56%

35%

9%

10%

TM1

TM2

TM3

TM4

TM5

TM6

Very or quite significantly

Somewhat

Not at all

Source: survey of HEIs participating in alliances supported by Horizon 202033 

The figure shows that: 

• 

the  proportion  of  HEIs  saying  that  collaborating  in  an  alliance  has  helped  them 
advance ‘very significantly’ or ‘quite significantly’ ranges from 28% to 37%, with 
an average across all TMs of 33%. 

•  On average, across all TMs, 56% said collaboration helped them ‘somewhat’ and 11% 

said it did not help them at all.  

These average figures for different activities undertaken within a TM, and across all six TMs 
hide  significant  variation  within  these  totals,  and  the  fact  that  participating  in  an 
alliance has been particularly valuable on specific subjects (and less so on others). This 
provides  support  for  the  conclusion  that  where  respondents  said  participation  under  a 
particular TM did not help at all, this does not mean that they received no benefit from all 
TMs. 

Areas where collaboration through the alliance has delivered particular value (i.e. => 50% of 
respondents said collaboration was very or quite significant) include: inventories and plans 
for sharing infrastructure, new partnerships for R&I, new funding applications, entrepreneurial 
skills training, initiatives to support engagement with citizens, an over-arching R&I strategy, 
activities to transfer academic knowledge into policy making, and open science skills training. 
Highlighting these areas should not diminish the appreciation of benefits in other areas, even 
if less prevalent. 

33 This chart presents results from six separate survey questions, so the numbers of responses vary for each 
TM (the numbers were TM1: 74, TM2: 65, TM3: 61, TM4: 57, TM5: 54, TM6: 52) 

53 

 
 
 
 
Figure 38 In which areas within individual TMs did participating in an alliance have the 
most significant influence on R&I capabilities? 

TM2: Plans with other universities on sharing research
infrastructure

TM1: New partnerships for R&I

TM1: Applications for new funding or resources

TM4: Entrepreneurial skills training for doctoral students
or research staff

TM2: An inventory of research infrastructure

TM6: Initiatives to support citizen/end-user engagement
on a regular basis

TM1: An R&I strategy

TM6: Activities to transfer academic knowledge into
policy making

TM5: Training for researchers in open science skills

TM3: A human resource strategy or action plan for
research staff

TM4: R&I collaboration spaces or initiatives

TM5: An open science policy

TM3: An equality, diversity or inclusion strategy or
action plan

TM3: Its framework for research assessment

TM2: A policy on sharing access to research
infrastructure

TM5: Measures to support researchers publishing in
open access

TM4: An inventory of existing R&I collaboration partners

TM6: Activities to make it more environmentally
sustainable

TM5: A designated person, unit or structure for data
governance

69%

66%

62%

60%

57%

56%

51%

50%

50%

49%

46%

44%

44%

43%

42%

35%

35%

29%

28%

TM3: Support structures for female researchers

23%

22%

9%

23%

11%

23%

15%

35%

5%

32%

11%

38%

6%

36%

12%

42%

41%

8%

9%

49%

2%

49%

52%

51%

44%

42%

57%

54%

63%

57%

64%

5%

4%

5%

13%

17%

7%

11%

8%

15%

13%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Very significant or quite significant influence Minor or no influence

Don't know/not applicable

Source: survey of HEIs participating in alliances supported by Horizon 202034 

34 This chart presents results from six separate survey questions, so the numbers of responses vary for each 
TM (the numbers were TM1: 74, TM2: 65, TM3: 61, TM4: 57, TM5: 54, TM6: 52) 

54 

 
 
Figure 39 below compares responses by HEIs saying that collaboration helped them advance 
‘quite  significantly’  or  ‘very  significantly’  for  HEIs  based  in  widening  countries  with  those 
based in non-widening countries.35 This illustrates the following: 

•  HEIs in widening countries benefitted significantly more compared to non-widening ones 
–  with  this  being  the  case  for  five  out  of  six  TMs.  This  provides  evidence  for  the 
hypothesis that institutional changes in the transformational modules will tend to 
accelerate access to excellence in less R&I intensive institutions/regions. 

•  The  exception  is  TM2  where  respondents  from  HEIs  in  non-widening  countries 
said  that  cooperation  was  more  likely  to  help  them  than  those  from  widening 
countries. It is not entirely clear why this is the  case, but it could be that HEIs in non-
widening countries have greater resources or infrastructure to share, making cooperation 
under  this  TM  more  valuable  for  them.  (For  instance,  participating  in  the  EUI  may 
encourage HEIs with large amounts of infrastructure to start to create an inventory of it 
and create rules about access and sharing, as they now have to find a way to share it 
with their alliance partners). 

•  Although  HEIs  based  in  non-widening  countries  benefit  relatively  less,  the  proportion 
saying  that  collaboration  helped  them  quite  significantly  or  very  significantly  is  still 
substantial, averaging 30% of respondents from these countries. 

Figure  39  Breakdown  by  widening  and  non-widening  countries  -  percentage  of 
'quite 
respondents  saying  collaboration  helped  their  HEI  advance 
significantly' 

'very'  or 

TM1 - Shared R&I strategies

TM2 - Sharing capacity, infrastructure, resources

TM3 - Strengthening human capital

TM4 - Cooperation with other sectors

TM5 - Knowledge- and digitally-driven HEIs /
open science

26%

21%

24%

26%

50%

39%

58%

45%

36%

44%

TM6 - Embedding citizens and society

28%

50%

0%

10% 20% 30% 40% 50% 60% 70%

HEIs from non-Widening countries

HEIs from Widening countries

Source: survey of HEIs participating in alliances supported by Horizon 202036 

35  Bulgaria,  Croatia,  Cyprus,  Czech  republic,  Estonia,  Greece,  Hungary,  Latvia,  Lithuania,  Malta,  Poland, 
Portugal, Romania, Slovakia, Slovenia 
36 This chart presents results from six separate survey questions, so the numbers of responses vary for each 
TM (the numbers were TM1: 74, TM2: 65, TM3: 61, TM4: 57, TM5: 54, TM6: 52) 

55 

 
 
 
56 

 
 
 
 
A key implication of this analysis is that while the Horizon funding for R&I has significantly 
promoted widening objectives, this is not its only benefit. While the identification of HEIs 
in non-widening countries with ‘excellence’ is too simplistic, the data does suggest that the 
Horizon funding has to some extent served the objective of promoting excellence, but 
to a lesser extent than supporting widening goals. 

However, it is still the case that universities from both groups of countries can make 
beneficial use of the funds, albeit with a different focus to support capacity building. 
For example, in widening countries universities often have very poor resources to support 
academics/researchers  submitting  proposals  to  the  Framework  Programmes.  Hiring 
dedicated  staff  to  provide  support  across  all  areas  would  therefore  be  beneficial.  In  non-
widening countries, universities may have better research support office but may still want to 
focus on a specific pillar, and therefore e.g. hire a specialist dedicated to ERC support only.  

Figure 40 below presents the same analysis, dividing responses into those from countries in 
north  and  western  Europe  and  those  in  south,  central  and  eastern  Europe.37  Figure  40 
provides a very similar picture to Figure 41 (which is to be expected as the only difference 
between  the  south  central  and  eastern group  and  the  widening  countries  is  that  the latter 
does not include Spain and Italy). It indicates that: 

•  Respondents  from  HEIs  in  south,  central  and  eastern  European  countries  were 
significantly more likely to say that collaboration had helped their HEI advance (this was 
the  case  for all  TMs  except TM5,  where respondents  from  HEIs in  north and  western 
Europe were slightly more likely to say this). 

• 

In relation to TM2, while Figure 41 shows greater benefit here for non-widening countries, 
Figure 40 indicates that countries in south, central and eastern Europe benefit more. This 
is explained by HEI respondents from Spain and Italy being relatively more likely to say 
they benefitted from this TM. 

37 North and western Europe: Austria, Belgium, Denmark, Finland, France, Germany, Ireland, Luxembourg, 
Netherlands,  Norway,  Sweden,  UK.  South,  central  and  eastern  Europe:  Bulgaria,  Croatia,  Cyprus,  Czech 
Republic,  Estonia,  Greece,  Hungary,  Italy,  Latvia,  Lithuania,  Malta,  Poland,  Portugal,  Romania,  Slovakia, 
Slovenia, Spain. 

57 

 
 
Figure 40 Breakdown by geographic region (north/west and south/central/east Europe) 
- percentage of respondents saying collaboration helped their HEI advance 'very' or 
'quite significantly'  

TM1:Shared R&I strategies

TM2:Sharing capacity, infrastructure, resources

43%

43%

57%

57%

TM3:Strengthening human capital

33%

67%

TM4:Cooperation with other sectors

TM5:Knowledge-and digitally driven HEIs/open
science

TM6:Embedding citizens and society

43%

57%

52%

47%

44%

56%

Northern and western Europe

Southern, Central and Eastern Europe

0% 10% 20% 30% 40% 50% 60% 70% 80%

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Figure 41 assesses the same data, breaking this down for those alliances which have a 
thematic focus compared to those which do not (see  

58 

 
 
 
Table  1  for  a  list  of  these  alliances).  The  chart  suggests  there  are  significant  differences 
between thematic and non-thematic alliances in the following areas: 

•  TM2  –  Sharing  capacity,  infrastructure  and  resources  –  where  46%  of  alliances 
with  a  thematic  focus  said  that collaboration  had  helped them advance  quite  or 
very significantly, compared to 27% among non-thematic alliances. This is a very high 
proportion  compared  to  other  results in  this survey.  It is not  very clear why  this is  the 
case, but one explanation could be that working within a few thematic areas increases 
the relevance of other alliance members’ infrastructure and resources. 

• 

In contrast, for TM4 – cooperation with other sectors, and TM6 – embedding citizens and 
society, the data suggest that non-thematic alliances found collaboration more valuable 
than thematic ones. The reasons for this are not readily apparent, however. 

Figure  41  Breakdown  by  alliances  with  /  without  a  thematic  focus  -  percentage  of 
respondents  saying  collaboration  helped  their  HEI  advance 
'quite 
significantly'  

'very'  or 

TM1: Shared R&I strategies

TM2: Sharing capacity, infrastructure, resources

TM3: Strengthening human capital

TM4: Cooperation with other sectors

TM5: Knowledge-and digitally driven HEIs/open
science

TM6: Embedding citizens and society

30%

26%

27%

46%

32%

29%

36%

28%

38%

40%

42%

30%

HEIs in alliances with no thematic focus

HEIs in alliances with a thematic focus

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Finally, Figure 42 compares the same results for ‘small’ (i.e. with 15 000 students or fewer 
according to the ETER database38) with those for ‘large’ universities (those with more than 
15,000). This highlights the fact that smaller HEIs are substantially more likely to say that 
collaboration has helped them than larger ones. Specifically 

38 https://www.eter-project.com/  

59 

 
 
 
 
•  For four out of the six TMs (TMs 1, 3, 4 and 6) smaller HEIs were around twice as likely 
to say collaboration had helped them advance ‘quite’ or ‘very’ significantly.  

•  For TM2 (sharing resources) and TM5 (knowledge and digitally-driven HEIs) the results 

for smaller and larger universities were more or less the same.  

Figure  42  Breakdown  by  small  and  large  HEIs  -  percentage  of  respondents  saying 
collaboration helped their HEI advance 'very' or 'quite significantly' –  

TM1: Shared R&I strategies

24%

TM2: Sharing capacity, infrastructure, resources

TM3: Strengthening human capital

TM4: Co-operation with other sectors

20%

24%

37%

35%

45%

44%

TM5: Knowledge- and digitally-driven HEIs/open
science

TM6: Embedding citizens and society

33%

36%

26%

50%

56%

Large universities (>15 000 students)

Small universities (<15 000 students)

0%

10%

20%

30%

40%

50%

60%

Source: survey of HEIs participating in alliances supported by Horizon 2020 

2.6  EQ5: What are the strengths, weaknesses, opportunities and threats 

for the establishment and consolidation of the alliances? 

The following analysis assesses the internal strengths and weaknesses of the initiative and 
as well as opportunities and threats relating to the broader external environment and the way 
that the initiative’s strengths can be built on and threats to objectives addressed.  

2.7  Strengths 

There is strong agreement across all consultees that the Horizon 2020 funding has supported 
a broad EU-wide R&I agenda for HEIs and helped to integrate the education and research 
dimensions of the alliances. Respondents welcome and strongly support the Horizon 2020 
top  up  funding  for  the  alliances’  R&I  dimension  as  adding  value  to  the  higher  education 
elements previously addressed through Erasmus+. A common theme of comments is that 
the  combination  of  academic/education  domain  and  research/innovation  focused  projects 
delivers value in its own right and also creates synergies between the two policy areas.  

The funding for the R&I dimension is considered a valuable impetus for a transformational 
process  which  builds  on  and  further  extends  previous  internationalisation  in  the  higher 
60 

 
 
 
 
education  sector  and  helps  HEIs  to  develop  a  strategic  R&I  approach.  The  support  is 
particularly valuable  for smaller HEIs,  those  with  less  developed  R&I capabilities  and  less 
experience in internationalisation and in particular, for those in widening countries. However, 
there are also benefits for HEIs that do not meet these criteria, even for those with strong 
existing  R&I  capabilities.  This  highlights  the  value  of  learning  from  each  other  and  from 
developing common research agendas. 

The Horizon 2020 funding for alliances adds value by supporting a broad agenda which aligns 
closely with other major policy initiatives e.g. the European Research Area agenda. It is also 
distinctive  in  focusing  on  institutional  capacity  building  and  development,  which  can  be 
challenging to secure funding for. For instance, the SwafS grants was the only element within 
Horizon Europe to focus on building R&I capacity, with the bulk of the budget dedicated to 
direct  funding  of  research  (ERC,  MSCA,  the  various  pillars),  paying  for  salaries, 
consumables, equipment etc. directly related to research activity. While there are some other 
funding  sources  for  capacity-building  (e.g.  the  EEA  &  Norway  Grants’  bilateral  research 
programmes  or  the  Structural  Funds)39  the  values  of  this  are  substantially  lower  than  for 
research funding.40 

One Member State also highlighted that selection of partners under the current mechanism 
is strict and the calls include many conditions, which encourages participation by HEIs that 
have the potential to deliver their workplans successfully.  

2.8  Weaknesses 

Despite strong appreciation of the potential for R&I support (through the Horizon 2020 grants 
and future initiatives), a number of weaknesses can also be identified.  

Two key points were raised repeatedly across all stakeholders consulted for the survey. The 
first is that the grants were relatively small, and therefore unlikely to promote transformational 
in their own right. This led some of those with more sceptical views to judge that advances in 
capabilities reported by alliances would most likely occur where HEIs had already decided to 
prioritise these, rather than because of the Horizon funding. 

A second, overwhelming concern raised by very many consultees  relates to the continuity 
and sustainability of funding for R&I work. There are common concerns that the Horizon 2020 
funding  will  be  seen  as  a  “one-off”,  providing  limited  project-based  funding  for  long-term, 
transformational  goals.  A  related  point  is  that  if  alliances  are  unsure  about  the  stability  of 
funding  over  the  long-term  they  are  very  likely  to  limit  the  ambition  of  their  collaborative 
projects to ensure that they are able to deliver the things they commit to. 

Among other things, uncertainty and relatively short funding timescales are cited as hindering 
the potential for other social partners to engage (in particular industry, which requires some 
certainty  to  commit  in  this  way).  In  the  absence  of  clear  indication  on  future  funding,  the 
continuation  of  established  R&I  projects  is  considered  in  jeopardy.  Member  State 
representatives  and  alliance  HEIs  alike  highlighted  the  limitations  of  other  co-funding 
(national or other state funds) to achieve this. A related point raised by some consultees in 
non-widening countries is that future initiatives would overly focus on widening countries. 

Stakeholders also pointed to a lack of flexibility (e.g. to depart from work plans submitted an 
application  stage),  the  heavy  bureaucracy  involved  in  responding  to  funding  calls  and 

39 For instance, see pp. 8-9 of Science Europe’s position statement – Towards Strengthened R&I Systems 
Across Europe.  
40 While the Structural Funds are widely used for this purpose the value of funding is not easy to determine 
even with the data search facility (https://cohesiondata.ec.europa.eu/themes/1/14-20). Building a full picture of 
capacity building funding would therefore require substantial further research. 

61 

 
 
burdensome reporting processes. A very common issue raised is that funding the alliances 
from  two  separate  instruments  (Erasmus+  and  Horizon  2020)  with  different  requirements 
heightens the administrative burden at  both the selection and implementation stages, with 
HEI representatives highlighting this as challenging and often confusing.  

Some  consultees  mentioned  that  the  focus  of  the  Horizon  funding  on  ‘overheads’  i.e. 
development  of joint  agendas  and  plans little  left  to support actual  R&I  activities,  and  that 
there  is  a  strong  need  for  further  funding  for  actual  research.  Others  noted  however  the 
advantage of having resources for capacity-building exercises, noting these can be harder to 
find funding for.  

2.9  Opportunities 

Stakeholders firmly believe in the potential and opportunities for the establishment and future 
consolidation of the alliances offered by the Horizon 2020 top-up funding for R&I activities. 
Its continuation is thought indispensable to support the exploration of the R&I dimension of 
alliances, as it creates conditions for HEIs to work together across borders, providing a solid 
foundation for excellent research. Member State representatives believed that realising the 
R&I  potential  of  alliances  would  contribute  to  strengthening  national  competitiveness  and 
raise the quality of R&I by elevating the best universities to an even higher level. This could 
also alleviate the loss of top universities in the EU caused by the UK’s departure from the 
Union. 

Should  the  Horizon  funding  for  the  R&I  dimension  of  alliances  be  renewed,  its  focus  on 
capacity  building  presents  an  opportunity  to  redress  the  existing  R&I  capacity  imbalance 
between  widening  and  non-widening  countries.  HEIs  in  both  widening  and  non-widening 
countries can benefit significantly from the funding in this view, whether they require cross-
cutting support to enhance their competitiveness to attract funding for research (a situation 
more  common  in  widening  countries)  or  for  fine-tuning  a  specific  element  of  their  R&I 
capabilities such as research support.  

2.9.1  Main  reasons  to  continue  supporting  alliances  through  Horizon  Europe  or  its 

legacy programme 

Stakeholder  groups  unanimously  agreed  that  the  main  reason  to  continue  supporting  the 
R&I dimension of alliances within the Horizon Europe or its legacy programme is to ensure 
the future of the successful pilot activities that have already taken place with the Horizon 2020 
funding. 

Stakeholders believed that it is crucial to ensure that the R&I dimension remains part of the 
alliances’  work.  However,  they  shared  concerns  whether  this  would  be  possible  in  the 
absence of funding from Horizon. In particular, while some Member State governments will 
support  the  alliances  from  their  own  resources,  not  all  will  do  this,  given  other  spending 
priorities. This has significant potential to unbalance the work of the alliances – i.e. where the 
opportunities for those member HEIs that do not receive support from their governments both 
to  benefit  themselves  and  contribute  to  the  benefits  of  other  members  is  significantly 
circumscribed. 

2.10  Threats 

There is broad consensus among stakeholders that the main threat to the establishment and 
future  consolidation  of  the  alliances  comes  from  an  uncertain  funding  environment,  with  a 
corresponding need to ensure that funding continues into the medium or long term. A point 
made repeatedly is that the R&I funding needs a programmatic, rather than project-based 
approach.  

62 

 
Although  it  is  conceded  by  some  stakeholders  that  at  some  point  universities  themselves 
should take on responsibilities for collaboration (including finding funding to support this), for 
the medium term at least (i.e. several more years) there is also a risk of prematurely pulling 
support before collaborative arrangements have had time to mature and before real benefits 
in  terms  of  improved  R&I  performance  materialises.  A  possible  result  could  be  that  many 
alliances become purely education-focused. In turn, this would undermine support for wider 
objectives e.g. ERA policy goals. 

Other threats relating to the scoping of future funding were also identified by consultees. For 
instance,  one  HE  stakeholder  organisation  highlighted  the  risks  in  forcing  to  strict  an 
alignment  between  the  education  and  R&I  dimensions  of  the  alliances.  Specifically,  many 
alliance  HEIs  may  want  to  have  separate  partners  for  each  dimension,  rather  than  to  be 
compelled to work with the same ones.  

A  concern  was  also  raised  that  the  Commission  should  avoid  focusing  too  much  on  the 
alliances  as  an  instrument  for  achieving  R&I  policy  objectives,  given  that  the  alliances 
represent only a relatively small proportion of HEIs (i.e. there are approximately 600 HEIs in 
alliances  compared  to  more  than  5  000  HEIs  in  total  in  Europe).  It  was  highlighted  that  it 
would  be  mistaken  to  overly-focus  other  instruments  such  as  Structural  Funding  on  the 
alliances. A related point was made in relation to the contention that the alliances provide a 
key  locus  for  research  which  is  then  ‘transferred  outwards’,  spurring  innovation.  But 
innovation  doesn’t  necessarily  work  like  this,  and  bilateral  collaboration  might  be  equally 
successful.  

Finally,  caution  was  also  mentioned  in  relation  to  the  number  of  policy  objectives  that 
alliances  are  tasked  with  achieving.  As  one  stakeholder  noted  “the  mismatch  between 
ambitions and means puts an enormous strain on the alliances, the institutions involved, and 
the people working to make them a success”.  

63 

 
 
3. 

Impacts on HEI participation in innovation systems 

3.1  EQ3: What is the EUI’s impact on cooperation between the HEIs and 

wider innovation ecosystems? 

The discussion of progress under TM4 above (see pages 38 to 42 above) indicates that the 
Horizon funding has made a significant difference to around a third of participating HEIs and 
provided some help to around a further half. As noted in the same section, the survey results 
identified  particular  benefits  from  collaboration  in  terms  of  developing  entrepreneurship 
training courses, collaboration spaces and inventories of R&I collaboration partners, but there 
are  further  examples,  as  illustrated  from  the  midterm  reports  and  feedback  from  the 
interviewed alliances.41 These include: 

•  Facilitation of cooperation with national, regional, and local authorities:  

-  Local partners (including municipalities) participating in an advisory board of around 

25 partners, from each member university  

-  A  university  professor  being  appointed  as  an  adviser  to  the  city  government  of 
Tilburg (with nearly a quarter of a million inhabitants) on labour market and welfare 
issues 

-  An extension of existing cooperation on city twinning between Brno and Rennes to 

include R&I collaboration 

-  Enhancement  of  links  with  regional  and  national  government  in  Spain,  and 

introductions for them to new potential new eco-system partners.  

•  Creation of research infrastructure that could attract organisations external to the 

alliance such as other research institutes, industry partners and NGOs:  

-  Research infrastructure which will also benefit by the research organisations which 

are not part of the alliance 

-  Measures  to  create  an  academic  environment  in  which  cross-boundary  research 
and  collaborative  education  is  facilitated,  including  cooperation  with  public  and 
private actors from across society 

-  Enhanced  outreach 

to  external  university  stakeholders 

through  shared 

infrastructure 

•  Development and strengthening of relations with businesses and other potential 

stakeholders, notably through:  

-  The creation of advisory board including business expertise,  

-  A database of external partners and researchers  

-  Creation  of  an  IT  application  enabling  interaction  among  the  stakeholders  of  the 

ecosystems of the member institutions  

The varied examples identified from interviews and project documents highlight substantial 
work  in  this  key  area across  different  alliances.  Perceptions  that  good  progress  has  been 
made  are  also  confirmed  by  Member  State  representatives  who  generally  had  a  positive 
outlook  on  the  impact  of  alliances  in  this  field.  While  they  could  not  necessarily  provide 
concrete  examples,  these  consultees  benefit  from  an  overview  of  the  bigger  picture  at 

41  Some  of  these  are  also  highlighted  in  the  Report  on  Progress  of  University  Alliances  to  Realise 
Transformation Modules (REA, 2022).  

64 

 
 
national level, and their observations generally validated the transformational potential of the 
Horizon funding here. 

Interviews with Member State representatives from widening countries  also noted that the 
EUI has strong potential to broaden their universities’ networks and enhance linkages with 
industry,  public  authorities  and  NGOs  depending  on  the  focus  of  the  alliance  (innovation, 
citizen  engagement).  In  two  instances,  Member  State  representatives  highlighted  visible 
improvements,  such  as  a  growing  collaboration  with  the  outer  ecosystem,  engaging  and 
involving  the  associated  partners  such  as  industry  players  and  the  creation  of  a  new 
institutional culture based on the alliances’ work in the country.  

A  further  key  point  made  by  consultees  is  that  benefits  will  often  be  greater  for  smaller 
universities with more modest relationships with existing innovation ecosystems. The visibility 
generated by the participation in a European alliance was also singled out by others as crucial 
for potential collaborations, especially for smaller universities and smaller Member States. 

3.2  Overall assessment 

While  much  of  the  evaluation  evidence  indicates  good  progress  in  developing  HEI 
relationships  with  their  broader innovation  ecosystems  there are also  some  caveats  to  be 
made. 

The assessment of HE stakeholder organisations was often more cautious than the account 
given so far, emphasising that advances are likely to be tentative and will need continuing 
support and development before genuine transformation of cooperation with wider innovation 
ecosystems occurs.  

For instance, one interviewee expressed scepticism about the role of alliances in this field, 
contrasting their transnational nature with the fact that the regional level is key in many eco-
systems (however, see the box below on FILMEU for an alternative view). This tallies with 
an  observation  in  the  separate  progress  assessment  commissioned  by  DG  RTD  which 
highlighted the fact that the authors could find no reference to Research and Innovation Smart 
Specialisation  Strategies  (RIS3)  what  now  exist  in  most  European  regions  with  close 
involvement with various actors including universities. 

As  another  stakeholder  pointed  out,  development  of  eco-systems  depends  on  long-term 
relationships, and trust, and that short-term projects don’t work for this purpose. For instance, 
to engage citizens, businesses or other stakeholders HEIs need to be there after the project 
ends. While an overall assessment of progress within TM4 does therefore present important 
indications of progress, it highlights again the need for a programmatic approach to support 
for alliances R&I dimension and a greater degree of stability and predictability in support. 

FilmEU – addressing fragmentation in European film and media innovation systems 

FILMEU  stands  out  through  the  potential  it  has  identified  for  the  alliance  to  address 
significant innovation ecosystem barriers. While the alliance’s HEIs already have a track 
record  of  collaboration  with  industry  partners,  the  R&I  funding  has  helped  them  develop 
plans to strengthen these links. The film and media production sector is huge across Europe 
but highly fragmented, with many companies being very fragile. This creates great diversity, 
which is a strength but also a weakness (e.g. it makes cooperation harder). The alliance 
hopes that setting up the European University will help to solidify sectoral cooperation. For 
instance, the alliance is experimenting with programmes to allow masters level students to 
produce film scripts in an incubator, and then to be supported in taking these to market in 
discussions with producers and distributors. This will help to reduce the risk of innovation 
and compensate for sectoral fragmentation and a high proportion of often fragile SMEs. 

65 

 
3.3  EQ 4: To what extent have the European Universities engaged 

internationally, beyond the EU? 

The alliances show a moderate level of engagement with other universities beyond the 
EU. It is notable that neither Erasmus nor the Horizon 2020 support focused on engagement 
beyond the EU, and proposals incorporating measures for this was an element of only a few 
alliance strategies. There are nonetheless cases where this is happening as part of the core 
work  of  the  alliances,  especially  with  universities  in  Switzerland,  Ukraine,  and  the  United 
Kingdom, and in several other instances as described below. 

Responses  to  the  survey  of  the  alliances,  presented  in  Figure  43,  suggest  that  there  has 
been interest from external HEIs from within, but also from beyond the EU. Most significantly, 
24%  (18)  of  the  74  respondents  that  answered  this  question  for  TM1  indicated  that  HEIs 
outside their alliance, from within and/or from beyond the EU, have expressed interest in their 
activities or practices.  

Figure  43:  Have  any  HEIs  outside  your  alliance  expressed  interest  in  any  of  your 
alliance's activities or practices (number of responses)? 

TM1

TM2

TM3

TM4

TM5

TM6

4

4

2

2

0

16

2

8

3

2

2

7

3

1

4

6

8

10

12

14

16

18

20

Yes, from within the EU and also from beyond the EU

Yes, from beyond the EU (only)

N = 74 (TM1), 65 (TM2), 61 (TM3), 57 (TM4), 54 (TM5), 52 (TM6) 

Source: survey of HEIs participating in alliances supported by Horizon 2020 

The desk research and interviews carried out for this evaluation confirmed this general picture 
on  engagement  beyond  the  EU.  First,  there  is  engagement  with  Bologna  process 
countries.42  The  countries  that  stakeholders  most  often  referred  to  were  Switzerland 
and Ukraine (nine and eight times respectively), with this engagement taking several forms. 

42  A  novelty  under  the  2022  Erasmus+  call  is  that  the  alliances  can  now  accept  associated  partners  from 
Bologna Process countries / the European Higher Education Area (EHEA), namely 49 European countries, 
including for example Ukraine, the United Kingdom and Switzerland. 

66 

 
 
 
•  Switzerland: The 4EU+ European University Alliance includes the University of Geneva 
as  a  member43.  Another  seven  HEI  interviewees  referred  to  engagement  with 
Switzerland as part of their alliance’s work, with three saying that Swiss universities could 
become part of the alliance. One specified that Swiss HEIs are interested in developing 
connections with European universities and can pay for activities out of their own pocket. 
Switzerland is becoming less attractive to researchers since its exclusion from Horizon 
Europe, the world's largest research funding programme – a development noted more 
widely44.  It  was  suggested  that  Swiss  universities  might  be  even  more  interested  in 
joining European university alliances in future to help compensate for this. 

•  Ukraine:  Taras  Shevchenko  National  University  of  Kyiv  is  part  of  EUniWell  –  the 
European University for Well-Being.45 The Arqus European University Alliance has the 
National University of Kyiv-Mohyla Academy46 and the Aurora Alliance has V.N. Karazin 
Kharkiv  National  University  as  associate  partners.47  The  Aurora  representative 
interviewed highlighted that this is their most important international cooperation, trying 
to  help  this  Ukrainian  university  keep  going.  Mariupol  State  University  in  Ukraine  is 
welcomed as an associate institution to Transform4Europe48. Ukrainian universities are 
also in the process of becoming associate partners of ENHANCE and of UNIVERSEH. 
Una Europa is also collaborating with Ukrainian universities, and EPICUR has identified 
two  universities  to  work  with.  The  EPICUR  Inter-University  Campus  will  also  help 
Ukrainian students. 

Five interviewees also mentioned engagement with the UK: 

•  The  London  School  of  Economics  and  Political  Science  (LSE)  is  a  full  member  of 
CIVICA,49  EUniWell  includes  the  University  of  Birmingham,50  Aurora  includes  the 
University  of  East  Anglia  as  a  member,51  and  UNA  Europa  (UNI-Resin)  includes  the 
University of Edinburgh. 

•  One HEI interviewee indicated its alliance is investigating the UK model of Centres for 
Doctoral  Training52  with  London  and  Cambridge  Universities,  looking  to  implement  it 
among alliance members if they can find the funding. They indicated also having close 
links  with  the  Russell  Group,  which  represents  24  leading  UK  universities.  More 
generally, they are trying to increase cooperation with UK institutions. 

43 4EU+ Alliance, https://4euplus.eu/4EU-9.html. 
44  Courrier  expat,  Alerte.  La Suisse  devient  moins  attractive  pour  les  chercheurs,  03.01.2023,  available  at: 
https://www.courrierinternational.com/article/alerte-la-suisse-devient-moins-attractive-pour-les-chercheurs. 
45 European University for Well-Being, https://www.euniwell.eu/about/universities. 
46  Arqus, Arqus kicks  off  the  process  of  welcoming  the  National  University  of  Kyiv-Mohyla  Academy as  an 
associate  partner  of  the  Alliance,  21.12.2022,  available  at:  https://arqus-alliance.eu/news/arqus-kicks-off-
naukma/. 
47 Aurora European Universities, https://aurora-universities.eu/. 
48 Aef info, L’université de Saint-Étienne rejoint l’alliance européenne Transform4Europe, Dépêche n° 682679, 
21.11.2022, available at: https://www.aefinfo.fr/depeche/682679. 
49 CIVICA, https://www.civica.eu/. 
50 European University for Well-Being, https://www.euniwell.eu/about/universities. 
51 Aurora European Universities, https://aurora-universities.eu/. 
52 A Centre for Doctoral Training (CDT) is one of the approaches that some UK Research Councils use to fund 
and train doctoral students in England, Scotland, Wales, and Northern Ireland. 

67 

 
 
 
 
In  addition,  interviewees  mentioned  the  participation  of  Icelandic,  North  Macedonian  and 
Turkish HEIs in the alliances: 

•  Aurora has the University of Iceland as a member and the University of Tetova (North 

Macedonia) as an Associate Partner.53 

•  The UNIgreen Alliance includes the Agricultural University of Iceland as a partner.54 

•  The FORTHEM Alliance has the University of Agder (Norway) as a member.55 

•  The European Engineering Learning Innovation and Science Alliance (EELISA) includes 

Istanbul Technical University (Turkey).56 

•  The University of Montenegro should join the Ulysseus alliance on 16 December 2022.57 

Two  interviewees  mentioned  cooperation  with  Norway,  one  mentioned  cooperation  with 
Serbia and another with a research lab in Israel.58 

Beyond EHEA countries, the European universities’ engagement has remained limited, 
but anecdotal evidence provides promising examples. 

Certain alliances have developed international partnerships: 

•  EUTOPIA  already  has  four  global  partners  from  three  continents:  Monash  University 
(Australia), Université Internationale de Rabat (Morocco), Stellenbosch University (South 
Africa), and Kyungpook National University (South Korea). 

•  The  Tecnológico  de  Monterrey  (Mexico)  is  an  associate  member  of  the  European 

Consortium of Innovative Universities (ECIU). 

•  Developing  partnerships  with  universities  in  Africa  and  the  Mediterranean  region  is  a 
distinctive and central aim of CIVIS. The alliance seeks to address joint interests with 
these partners and find joint solutions to common problems together, which is why they 
have  a  CIVIS  sub-project  "African  and  Mediterranean  partnerships"  led  by  the  Aix-
Marseille  Université.  One  Member  State  representative  mentioned  this  international 
positioning objective as an example of the EUI boosting internationalisation. 

53 Aurora European Universities, https://aurora-universities.eu/. 
54 UNIgreen, https://unigreen-alliance.eu/index.html. 
55 FORTHEM European University Alliance, https://www.forthem-alliance.eu/. 
56 EELISA, https://eelisa.eu/partners/. 
57 Aef info, Universités européennes : Ulysseus (université Côte d'Azur) va accueillir deux nouveaux membres, 
Aef Dépêche n° 682983, 25.11.2022, available at: https://www.aefinfo.fr/depeche/682983. 
58  While  Israel  is  not  a  part  of  the  EHEA,  it  is  an  associate  country  under  Article  7  of  the  Horizon 2020 
Regulation.  Legal  entities  from  Associated  Countries  can  participate  under  the  same  conditions  as  legal 
entities from the Member States. The following countries were associated with Horizon 2020: Iceland, Norway, 
Albania,  Bosnia  and  Herzegovina,  North  Macedonia,  Montenegro,  Serbia,  Turkey,  Israel,  Moldova, 
Switzerland, Faroe Islands, Ukraine, Tunisia, Georgia, Armenia. 

68 

 
 
 
 
Other stakeholders consulted mentioned lighter forms of engagement: 

•  CIVICA’s Scientific Advisory Board includes two members who work outside of Europe 
for example, which can help offer an external perspective. The Scientific Advisory Board 
will notably evaluate the common R&I strategy. 

•  Una Europa has the President at the African Academy of Sciences as an external advisor 

and makes a strategic priority of engaging with universities in Africa.59 

•  FilmEU  has  engaged  with  a  university  in  Australia  and  another  in  the  USA,  including 
through training of trainers and cooperation on innovation in film and extended reality. 

•  The FORTHEM Alliance coordinator indicated having close links with the U15 Group of 

Canadian Research Universities. 

Una Europa’s active approach to working with African HEIs 

The  Una Europa 2030 strategy  focuses  on  pushing  for  internationalisation  through 
collaboration with Africa, by acquiring European as well as external funding to support this.60 
The  alliance  held  a  high-level  meeting  with  six  African  vice  chancellors  in  June  2022, 
focusing on identifying fitting research funding and the required support services. They aim 
at  creating  structures  to  bring  funding  and  experts  together  and  share  best  practices. 
Another alliance also said it is working on a joint roadmap with African universities for early 
career  researchers.  They  also  invited  six  African  universities  to  their  webinars  on  open 
science. 

A few alliance representatives indicated that they were expecting engagement beyond 
the  EU  to  come,  including  by  working  with  other  alliances  to  identify  opportunities 
after the pilot phase. These alliances have understood that the interest is not to work 
only  between  themselves.  They  are  aware  of  the  need  to  go  beyond  the  alliance,  to 
generate links between alliances, and beyond. 

•  For CHARM EU, this is part of their next stage. They want to bring HEIs in Africa into the 
specification  of  challenges  and  share  best  practices  on  R&I  with  HEIs  in  Mexico  for 
instance. 

•  UNIVERSEH are thinking of integrating Indian universities as associated partners of the 

alliance. 

•  EUniWell are looking at universities in South Africa. 

•  The Unite! universities committed to providing as many high-quality mobility opportunities 
to their students and staff as possible,  which requires engaging with HEIs beyond the 
EU. 

Most  alliances  therefore  seem  to  be  working  towards  further 
engagement. 

international 

59  Una  Europa,  Una  Europa  Talk:  Europe  and  Africa,  Shaping  our  Future  Together,  https://www.una-
europa.eu/calendar/una-europa-talk-europe-and-africa. 
60 Una Europa, Una Europa 2030, available at: https://www.una-europa.eu/opportunities/2030. 

69 

 
 
 
In  general,  the  degree  of  interaction  beyond  the  EU  depends  on  the  pre-existing 
relations  of  the  alliances’  members  at  the  individual  level.  The  European  universities 
have  had  opportunities  to  engage  beyond  the  EU  mainly  based  on  existing  international 
relations between HEIs or regions they belong to, which may have extended to the alliance 
at  some  point.  Una  Europa  includes  research  intensive  HEIs  with  pre-existing  in-depth 
international  linkages,  especially  regarding  work  on  global  challenges  for  example.  The 
University of Minnesota (USA) wants to become an associate member of Aurora, based on 
the  long-standing  cooperation  with  the  University  of  Iceland,  in  relation  to  both  R&I  and 
education, and because they have shared interest on sustainability and climate change.  

Another  stakeholder  confirmed  that  some  universities  work  strongly  with  their  Swiss 
colleagues, based on strong pre-existing relations, especially in the applied sciences.  The 
University of Granada was already internationalised with Latin America and North Africa, for 
historical, cultural, linguistic reasons, which has helped with developing Arqus’ international 
links. 

In a few cases, the alliance has further strengthened existing links: 

•  One EU-CONEXUS member had previous bilateral collaboration with Deakin University 

in Australia, which became more interested when they heard of the alliance.  

•  The Technological University of the Shannon works with universities in South Africa and 
Brazil, a research cooperation that the RUN-EU alliance has indirectly increased. 

Based  on  this,  there  is  scope  for  European  universities  to  expand  their  international 
engagement.  For  example,  Scuola  Sant’Anna,  an  Italian  research  university  member  of 
ELIISA, already has relations with international HEIs, which the alliance could use. While the 
EU remains a primary target, Scuola Sant’Anna has expanded its portfolio of activities in Asia 
(especially in China), Russia, the Middle East, Africa, and South America. These links could 
be the basis for the alliance’s engagement. 

While there is evidence of engagement beyond the EU, in general, participating HEIs have 
level  and  the  alliances  themselves.  As  mentioned, 
prioritised  the  European 
international engagement generally originates from the HEIs rather than the alliances 
because the latter focus on delivering their workplan. While alliances have often tried to 
engage beyond the EU, it has been difficult to find the resources to integrate this into their 
work. Even if there is a lot of interest in cooperation with China and the USA for instance, the 
framework set-up has not helped this. One interviewee was therefore not sure that the EUI 
was  necessarily  the  place  where  HEIs  engage  internationally  beyond  the  EU.  For  large 
French  universities  for  example,  engagement  beyond  the  EU  is  stronger  under  other 
initiatives like the I-SITE.61 Some participating French HEIs have tried to connect the I-SITE 
and the European university, but this engagement beyond the EU is not the priority of the 
alliance, which focuses on having an impact at the European level. 

Nevertheless,  one  Member  State  representative  indicated  that  to  make  world-class 
universities, they need to be visible in an increasingly global ecosystem.  In addition, there 
are no HEIs in Europe with a million students to work with large universities in Africa or Latin 
America, where global university hubs have very large numbers of students. Critical mass is 
important for such international engagement, so European Universities could be a solution. 
Another Member State representative confirmed that in terms of global reach, HEIs in third 

61 Initiatives Science, Innovation, Territoires, Economie (Science, Innovation, Territories, Economy Initiatives). 
The I-SITEs, which is part of the Programme d'Investissements d'Avenir, are universities that develop more 
concentrated,  distinctive,  and  internationally  recognised  thematic  scientific  assets,  and  that  use  them  as  a 
driving force and a support point for their development strategy and partnership with the economic world. 

70 

 
 
countries  are interested in  large European  alliances,  which  have  a  significant  potential  for 
partnership. 

There  could  therefore  be  a  greater  focus  beyond  the  EU  in  terms  of  enhancement 
funding. Yet, to be able to do this, the European universities need to reach a certain 
level  of  development,  but  they  have  different  baselines  due  to  the  pre-existing 
interactions of their members. 

3.4  EQ6: Has the EUI affected university and university policies? 

The  evaluation  assessed  two  interconnected  aspects  of  this  question:  (i)  impacts  on 
universities and their policies and (ii) impacts on government policy on research & innovation 
issues. 

3.4.1  Impacts on universities 

The online survey of HEIs asked respondents if they thought the alliances had impacted at 
all on the broader universities sector in their country, as indicated in Figure 44 below, which 
shows that more than half thought this was the case. However, this overall finding requires 
further explanation and nuance. 

Figure  44  Has  the  work  of  the  alliance  had  any  impact  on  the  broader  universities 
sector in your country? 

60%

50%

40%

30%

20%

10%

0%

56%

Yes

37%

Don’t know

7%

No

Source: survey of HEIs participating in alliances supported by Horizon 2020 

Owing to the close relationship between the Erasmus+ and Horizon 2020 funding streams 
on the work of the alliances, stakeholders often consider the overall impact without making 
firm  distinctions  between  the  education  and  R&I  dimensions.  Based  on  the  feedback 
collected,  it  is  undeniable  that  the  establishment  of  alliances  influences  the  educational 
dimension of universities. It is not surprising that the EUI’s impact has been more visible in 
the  education  dimension,  which  is  more  substantially  subsidised  through  Erasmus+.  This 
impact is for instance reflected in the efforts of creation or accreditation of joint degrees which 
have been facilitated by the work of the alliances and the cooperation dynamics established 
within them. Regarding the R&I dimension, the feedback collected appears more nuanced 
on the EUI’s impact on universities and their strategic policies.  

Overall, however, participation of national HEIs in alliances appears to have influenced 
their approach to international relations and cooperation and could contribute to the 
improvement  of  the  universities’  internationalisation  strategies  or  development  of  a 
global vision. In addition, HEIs participating in the alliances have increased their focus on 
internationalisation  and  mobility  resulting  in  increased  cooperation  within  the  university 

71 

 
 
sector.  Alliances  are  also  considered  important  for  increased  participation  in  networks  at 
national, EU and international level.  

Across the board, stakeholders highlight the interest of the university sector in the work of 
alliances and the wish to join in for those which are not members. This was noted in several 
interviews, reflecting a “fear of missing out”.  

As discussed in the answer to EQ3, the creation of a new institutional culture as a result of 
the  alliances’  work  in  the  country  is  an  important  development  which  could  enhance  the 
positioning of HEIs in the research sector on both national and EU level. Highlighted by both 
the Member States and HEI representatives the potential of the EUI in this area is evident.  

Furthermore, the alliances are considered as suitable testbeds to pilot novel approaches in 
the university sector, specifically in the context of ERA, which if successful, can be scaled up 
to the wider university sector. Notably the interactions of one of the alliances with a Member 
State Ministry of Education contributed to clarifying the government’s position on point 13 of 
the ERA policy agenda (cooperation on R&I), resulting in the allocation of additional funding 
to proposals supporting inter-university cooperation. The alliances as a pilot action could also 
give  researchers  interesting new  pathways  for development  in  an  increasingly  barrier-free 
European academic environment.  

In the context of R&I, stakeholders also note the following potential contributions of the EUI 
to the university sector overall: 

•  Generating  research  infrastructure  used  by  other  universities  and  best  practices  in 

research management which can be scaled up; and 

•  Attracting new funding, notably European Institute of Innovation & Technology (EIT) kick 

funding, Horizon Europe, Marie Curie. 

The  work  of  alliances  has  piqued  the  interest  of  authorities  in  several  Member  States,  as 
relayed  by  the  consulted  stakeholders.  In  the  context  of  the  UNIVERSEH  alliance,  the 
University  of  Toulouse  works  regularly  with  the  regional  authority.  It  is  regularly  consulted 
and participates in the development and implementation of regional plans for research and 
innovation,  such  as  the  development  of  regional  key  challenges  focused  on  the  driving 
sectors  of  the  regional  economy.  In  one  Member  State,  the  regional  government  closely 
follows the latest developments and activities of HEIs participating in alliances while another 
noted that the alliances have influenced changes on student status at regional level. Finally, 
another Member State government expects that the alliances will contribute to sharpening its 
focus on excellence in research.  

3.4.2  Impacts on national R&I policy  

Some  Member  States  have  provided  some  additional  funding  to  their  own  universities  to 
participate in alliances62. However, apart from this funding, there are no significant signs 
that  the  Horizon  funding  has  yet  had  any  impact  on  R&I  policy  more  widely.  This 
contrasts  with  the  situation  regarding  educational  policy  at  national  level  where  some 
changes are evident.  

Regarding the development of the educational and mobility aspects of the alliances, recent 
research by the EUA identified ten Member States where the European Universities Initiative 

62  Member  states  that  have  supported  their  alliances  in  addition  to  Horizon  funding  include,  for  example, 
France, Germany, the Netherlands, Portugal and Ireland. In Portugal, the national government has provided 
additional funding for the alliances that is solely related to R&I issues (the only member state to do this). 

72 

 
 
has prompted policy discussions at national level (e.g. in relation to academic autonomy on 
issues such as accreditation, language of instruction and admission of students).63 These are 
significant developments in their own right, given the barriers to transnational collaboration 
within the European Universities Initiative created by legal, regulatory or cultural differences 
between Member States.  

However, the same is not yet true of national policies on R&I. While there are some early 
indications  of  future  change,  only  a  few  examples  were  identified  by  this  evaluation  – 
and these appear to be relatively minor in scope rather than significant policy changes. 
These examples include: 

•  The HE sector in one Member State has participated significantly in the alliances, with 
universities represented in 14 of the 39 alliances that received Horizon funding. This level 
of participation is reflected in national government interest in the EUI, and participation 
in  an  alliance  (including  R&I  elements)  is  now  embedded  in  many  performance 
agreements between universities and the government (e.g. on the digital strategy and 
universities development plan). 

•  One of the alliances helped to change a Member State’s government’s policy regarding 
point 13 of the ERA policy agenda (cooperation on R&I). This intervention clarified the 
government’s policy position and allowed more funding to be used by HEIs for projects 
supporting inter-university cooperation.  

• 

In  the  same  Member  State,  the  work  of  the  alliances  has  helped  to  promote  wider 
knowledge  and  understanding  of  European  policy  developments  on  research 
assessment reform. According to one interviewee awareness of this initiative and related 
developments (e.g. COARA) would have been much lower without the Horizon-funded 
R&I projects. 

•  The  Ministry  of  Education  and  Science  in  a  widening  Member State  has said  that  the 

government’s next strategy on R&I will include references to the EUI. 

• 

In another Member State, the government anticipates that the experience of the alliances 
will contribute to discussions on excellence initiatives at national level. 

•  One  of  the  EU4ART  members,  the  Academy  of  Fine  Arts  of  Rome,  fosters  regular 
discussion with the national regulatory body for quality management, the Italian National 
Agency for the Evaluation of Universities and research institutes (Agenzia Nazionale di 
Valutazione del Sistema Universitario e della Ricerca – ANVUR). 

The  EUA  notes  that  collaboration  on  the  education  side  of  the  alliances  started 
significantly earlier than that for R&I, indicating that policy changes for the latter might 
emerge in due course. It is worth pointing out also that the R&I dimension of the European 
Universities Initiative has been closely reflected in R&I policy at EU level, as witnessed for 
instance in the high degree of correspondence between the TMs and the ERA policy agenda 
(see  Annex  4).  Some  interviewees  also  mentioned  that  that  ERA  policy  is  more  likely  to 
influence national policy than the EUI (notwithstanding the fact that the EUI TMs for R&I relate 
very closely to the ERA policy agenda – see Annex 4). 

63  EUA  (2022)  The  European  Universities  Initiative  and  system  level  reforms  –  current  challenges  and 
considerations for the future. The members states identified include Greece, Belgium, Hungary, Italy, Romania, 
Slovenia, Denmark, Spain and the Netherlands. 

73 

 
 
4.  Demonstration effects on HEIs not part of alliances 

4.1  EQ  7: To  what  extent can European universities inspire  other  HEIs 
that are not part of the alliances and induce change in the university 
sector with their good practices? 

One  of  the  aspirations  for  the  alliances  is  that  inclusive  models  of  cooperation  and  the 
exchange  of  practice  have  positive  demonstration  effects  in  Member  States.  Most 
stakeholders  consulted  indicated  that  some  HEIs  that  are  not  part  of  the  alliances  have 
indeed expressed interest in the European universities (see also answer to EQ 6 above).  

The  best  evidence  of  this  interest  is  the  expansion  of  the  existing  alliances  (i.e.  new 
members)  and  the  growing  number  of  applications  and  alliances.  According  to  several 
Member State  representatives,  HEIs’  interest  in  the  EUI  has  grown  over  the  years.  More 
universities are interested after seeing the success of others and want to become part of an 
existing alliance or are looking for partners to create a new one. While this is not necessarily 
linked to the R&I dimension specifically, this shows the potential for European universities to 
inspire other HEIs and to induce change in the university sector, including on R&I issues. 

4.2  Expansion of alliance memberships 

There is a political incentive from the European Council to expand the European Universities, 
and some HEIs have joined alliances since the initial calls for proposals. Given the integration 
of the Horizon 2020 projects into the alliances, it is difficult to distinguish the relative influence 
of the different components of the EUI. Nevertheless, according to some stakeholders, the 
R&I funding has increased the attractiveness of the alliances. Examples of the expansion of 
the memberships of existing alliances include the following: 

•  EPICUR started with four member HEIs, each of which brought at least one extra partner 

onboard. Similarly, over time CIVICA has grown from eight to ten members. 

•  A number of new members have recently joined alliances or are about to do so, including  

-  At the end of 2022, the New Sorbonne University joined YUFE – Young universities 
for the future of Europe,64 and the University of Münster joined the Ulysseus alliance. 

-  At 

time, 

the  same 

the  Jean-Monnet  University  of  Saint-Etienne 

joined 
Transform4Europe, "opening the way to sustainable cooperation in education and 
research".  Two  additional  members  –  the  Catholic  University  of  Portugal  and  the 
University of Primorska in Slovenia will also join the alliance soon, bringing the total 
number of members to ten.65 

-  The Bauhaus-Universität Weimar will join the EU4ART alliance, and several other 

HEIs have expressed their interest in doing the same. 

-  The  EDUC  alliance  reported  that  new  members  will  join  soon,  funding  their 

participation from their own resources. 

64  AEF  info,  Universités  européennes  :  Paris-III  rejoint  l’alliance  Yufe,  qui  associe  neuf  autres  membres, 
Dépêche n° 684793, 23.12.2022, available at: https://www.aefinfo.fr/depeche/684793.  
65 Aef info, L’université de Saint-Étienne rejoint l’alliance européenne Transform4Europe, Dépêche n° 682679, 
21.11.2022, available at: https://www.aefinfo.fr/depeche/682679. 

74 

 
 
 
 
In addition, there are also examples of the alliances attracting other HEIs that did not initially 
join one: 

•  After the Technological University of the Shannon joined the RUN-EU alliance, two Irish 

HEIs applied for the second call and were successful. 

• 

• 

In Italy, the interest has grown, and many universities reportedly want to be part of the 
alliances.  

In Germany, the first wave of alliances also inspired other large HEIs to join the second 
call. The interviewee who mentioned this said the EUI became more attractive with the 
addition of the research and innovation dimension. 

•  Similarly, in Iceland, two new HEIs joined alliances after seeing the University of Iceland 
involved in one. The interviewee who mentioned this emphasised that they wanted to 
participate to address both education and research objectives, also highlighting the value 
of the R&I dimension. 

4.3 

Increases in associate membership 

In addition, there have been instances where HEIs that are not part of an alliance expressed 
interest  in  becoming  associate  partners,  because  they  see  the  alliances  as  politically 
influential. Examples include: 

•  The Unite! Alliance added two associate partners from Austria and Poland. 

•  Transform4Europe will have three new associate members  

•  HEIs  outside  the  alliance  have  approached  member  universities  of  EC2U.  In  another 
case, an HEI in Toulouse has indicated its interest in joining UNIVERSEH as a third party 
to the member University of Toulouse.  

While  there  is  no  information  on  the  link  between  this  phenomenon  and  the  alliances’ 
research and innovation dimension, these developments will further reinforce the network of 
partnerships  within  the  alliance  and  between  the  alliance  members  and  non-alliance 
universities, increasing the potential for learning and sharing of good practices.  

Lastly, there are some notable examples of interest in associate membership from beyond 
the EU: 

•  Ulysseus has received a series of enquires from third countries (in Africa, China, India, 
and the USA) asking for more information, according to a Member State representative. 

•  A Canadian university wants to work on research exchange with EUniWell. 

4.4  Good practices 

Beyond the issues of expanding memberships, there is some evidence of other HEIs learning 
from the experience of alliance partners, although it is hard to determine how significant this 
is. It is also important to note that the SwafS grants are not the only element promoting such 
developments.  For  instance,  the  Horizon  Coordination  and  Support  Actions  also  fund 
“acceleration  services  in  support  of  the  institutional  transformation  of  Higher  Education 

75 

 
Institutions”  to  promote  the  uptake  of  good  practices  to  HEIs  that  are  not  part  of  the 
alliances.66 Examples of learning between alliance and non-alliance HEIs include: 

•  One alliance coordinator highlighted close collaboration between HEIs (including those 
not  participating  in  alliances)  on  PhDs,  workshops,  and  seminars.  A  Member  State 
representative similarly said that the HEIs that do not participate in alliances still follow 
their work and are influenced by this.  

•  Another Member State representative also indicated that there is already some uptake 

of good practices by other HEIs in countries in the Western Balkans.  

•  Another stakeholder said that influence on other HEIs is more prevalent where alliances 
have a specific thematic focus. They mentioned for instance that other HEIs in Portugal 
are  benchmarking  some  of  their  work  against  activities  undertaken  by  the  FILMEU 
alliance, as are other HEIs in film/media/arts-related fields that are not part of the alliance, 
including Aalto University in Finland and Lund University in Sweden. Following FILMEU 
practice, these universities have now started to use a joint challenge-based approach for 
cross-curriculum projects based on a common issue, for instance.  

•  An interviewee noted that universities in Rennes in France that were not selected for the 
French  excellence  initiative  for  governance  reasons  are  creating  a  new  experimental 
institution to work together, informed by the experience of the alliances. In particular, this 
group will learn from the EDUC alliance on the "HR Excellence in Research" label. 

•  The  Hungarian  universities  participating  in  the  alliances  have  made  a  significant  step 
forward in transnational cooperation, increasing research cooperation as well as mobility, 
curriculum  development  and  joint  training  initiatives.  This  has  been  discussed  at 
conferences and meetings with other HEIs and is having some impact, with an increasing 
number of HEIs in Hungary starting to enhance their international cooperation efforts.  

•  The  4EU+  alliance  already  ensures  a  wide  uptake  and  exploitation  of  the  project 
outcomes, through targeted dissemination and communication to various HEIs outside 
the alliance. 4EU+ strives to disseminate project outcomes to a wide audience through 
participation in European and international networks initiatives such as the EUA, LERU, 
CE7, UNICA, Coimbra Group, and IARU. 

•  One  of  the  objectives  of  RIS4CIVIS  is  to  share  their  experience,  lessons  learnt  and 
results, especially the Institutional Transformation model, with other HEIs, to ensure that 
they  can  benefit  from  this  model,  gain  inspiration  from  it,  and  apply  it  in  their  own 
institutions,  thereby  building  their  capacities  both  in  terms  of  R&I  and  in  terms  of 
European cooperation. 

The examples above highlight the fact that there are many good examples of networking and 
sharing of good practice between alliance members and other HEIs, both through formal and 
informal means.  

It is hard to assess the overall impact of such developments at this stage. One HE stakeholder 
organisation made an important caveat which should be borne in mind on this issue: “at this 
stage the alliances are still focused on experimenting and learning. They have organised a 
lot on this [e.g. events to share best practice etc]. But it’s one thing to attend a conference an 
hear about changes – the alliances are prolific – but another to start the process within your 
own  institutions. At  the  moment  it’s  more  about  dissemination  than  transmission”.  Despite 
this caveat it is clear that the alliances are becoming a fixed feature in the European higher 

66 https://cordis.europa.eu/programme/id/HORIZON_HORIZON-WIDERA-2022-ERA-01-51 

76 

 
 
education landscape, with research and innovation elements playing an increasing part of 
their work. This creates the potential for further influence on the wider sector in future. 

4.5  EQ8:  To  what  extent  have  the  European  Universities  inspired  the 
broader  university  sector  in  the  way  they  engage  with  their  local 
knowledge ecosystems? 

Since  it  is  premature  to  observe  the  EUI’s  impact  on  cooperation  between  with  the  wider 
innovation  ecosystems  surrounding  participating  universities  (see  above3.1),  the  same 
applies  to  their  influence  on  the  broader  university  sector.  Nevertheless,  the  evidence 
supporting its transformational potential in this area and in inspiring other HEIs that are not 
part  of  the  alliances  suggests  that  the  European  Universities  can  inspire  the  broader 
university sector in the way they engage with their local knowledge ecosystems on the long 
term.  However,  this  will  probably  depend  on  the  sustainability  of  the  funding  of  the  R&I 
dimension  of  the  alliances  as  it  needs  time  for  the  entire  local  knowledge  ecosystems  to 
change the way they cooperate, especially where it is not common practice yet.  

For now, despite the use of different data collection methods – desk research, online survey 
of alliances, and interviews – there is no evidence suggesting that the European Universities 
have  yet  inspired  the  broader  university  sector  in  the  way  they  engage  with  their  local 
knowledge ecosystems. There is only one concrete example of a coordinator indicating that 
the  broader  university  sector  now  engages  more  with  their  local  ecosystems.  Through 
EPICUR,  Karlsruhe  Institute  of  Technology  started  internships  for  students  from  other 
universities in their start-ups. In addition, they are discussing at the regional level to inspire 
the broader university sector in the way they engage with their local knowledge ecosystems. 

This may change in future. For instance, in Poland, interviewees suggested it is likely that 
the European universities will inspire Polish HEIs that are not part of the alliances when it 
comes  to  cooperation  with  other  sectors,  such  as  business,  state  institutions  or  non-
governmental organisations, in various forms of applied or socially engaged studies.  

4.6  EQ  9:  To  what  extent  have  the  European  Universities  inspired  the 
induce  change 

to  engage  and 

broader  university  sector 
internationally? 

At the time of this assessment, there is no evidence from the various sources of information 
reviewed to indicate that the R&I dimension of the alliances has inspired change in the way 
the  wider  university  sector  engages  internationally.  When  this  issue  was  discussed  in 
interviews with representatives of alliance HEI members and other stakeholders they could 
only point to the potential of the European universities to do this, and no significant examples 
were identified. This situation might change with the further consolidation and development 
of the alliances’ R&I work. One Member State representative highlighted for instance that the 
EUI in general puts pressure on national HEIs to further internationalise and in doing so helps 
the  country’s  science  base  to  become  more  international  and  more  fluent  in  cross-border 
cooperation projects. 

4.7  EQ10:  what  are  the  strengths,  weaknesses,  opportunities,  and 
threats  of  the  Horizon  2020  support  mechanism  for  European 
Universities  in  terms  of  impact  on  other  HEIs  in  Europe,  and  on 
international cooperation beyond Europe by the alliances?  

The SWOT analysis below explores the Horizon 2020 support mechanism to the European 
universities in terms of impact on other HEIs in Europe, and on the alliances’ international 
cooperation beyond Europe. The desk research, stakeholder feedback, and answers to the 
previous evaluation questions informed this analysis.  

77 

 
Strengths 

As  mentioned  above,  the  main  strength  of  the  Horizon  2020  funding  mechanism  is  its 
existence, as it supports the R&I dimension of the European universities, which can impact 
on other HEIs in Europe and on their international cooperation beyond Europe. The scheme 
leverages the existing structure of the European universities to enable them to build capacity 
in R&I in a complementary manner to  the Education dimension. This R&I dimension adds 
value to the European universities and strengthens their potential on the international scene 
and their global competitiveness. If the Horizon 2020 support mechanism demonstrates real 
impact on R&I, this capacity building element can inspire others. 

Weaknesses 

While the potential of the Horizon 2020 support mechanism to European Universities in terms 
of  impact  on  other  HEIs  in  Europe  and  international  cooperation  beyond  Europe  is  not 
disputed, it has several shortcomings. 

Most of the shortcomings mentioned above also apply to the external impact of the European 
universities, but there are also some that are specific to it. First, the limited budget of the 
Horizon 2020 support does not facilitate impacts on external HEIs – the potential for this 
to happen is more likely where alliances demonstrate real impact in raising research quality 
and quantity. This itself is only likely to emerge with more time – so up to this point in time 
the potential for influence is not strong.  

Second, the Horizon 2020 support focuses on the alliances and their members, which 
naturally limits the scope for cooperating with other HEIs within and beyond Europe. 
In addition, this support does not facilitate cooperation with partners outside Europe, which 
is therefore more complicated. Even if evidence is anecdotal, alliances have often tried to 
find partners beyond the EU. Nevertheless, it is difficult to find the resources to integrate them 
into their work. There is some interest in cooperation with the USA and China for example, 
but the framework set-up has not helped with this since the evaluation criteria for the funding 
calls did not cover the extra-EU dimension for instance and the mechanism does not directly 
support this. A clear ambition to promote and strengthen international cooperation with 
HEIs beyond the EU is therefore lacking. 

Thirdly, it is usually the case that other HEIs outside alliances already have at least some 
significant international links, albeit predominantly in terms of networks between researchers 
and academic staff. 

Opportunities 

Nevertheless, stakeholders believed in the potential of the Horizon 2020 support mechanism 
for European Universities to impact on other HEIs in Europe, and on international cooperation 
beyond Europe in the future. The Horizon 2020 support mechanism is innovative and could 
potentially have wider leverage. It offers funding for HEIs to work on R&I in a structural way 
that can potentially transform international cooperation if other HEIs replicate the model they 
use  for  instance.  It  also  encourages  the  diffusion  of  good  practices  internationally  and 
indirectly  supports  discussions  on  joint  research  degrees  or  projects,  which  are  useful  for 
promoting international cooperation beyond the European universities. It allows for synergies 
between exchange programmes, which can go beyond the alliances, and research specific 
projects. By supporting the R&I dimension of the alliances and international cooperation, it 
may help to bring together high-class international researchers to solve global challenges. 

In  practice,  it  is  left  to  the  alliances  to  take  the  lead  on  international  cooperation 
beyond  Europe.  This  means  they  can  use  their  members’  existing  links  with  other  HEIs 
within and beyond Europe. The stocktaking exercises that participating HEIs carry out as part 

78 

 
of  the  SwafS  projects  can  also  lead  to  the  identification  of  common  frameworks  to  start 
collaboration  with  international  partners.  The  Horizon  2020  support  can  also  support  the 
definition of common research themes capitalising on the participating HEIs’ experience in 
the  field,  which can  then  create  interest  among  external HEIs.  It  might  in  time  lead  to  the 
development of European centres of excellence able to cooperate internationally.  

For instance, the ENGAGE.EU will create a University think tank tackling the business and 
economic dimensions of the SDGs and involving Communities for Societal Change (CSC) 
for example.67 This could create success stories to inspire other HEIs to cooperate, and to 
promote  European  research  internationally,  therefore  reinforcing  the  attractiveness  of 
European universities. 

More  generally,  the  R&I  dimension  of  the  European  universities  simply  creates  a  lot  of 
interest.  Countries  like  China  have  expressed  interest  in  the  alliances  and  their  R&I 
dimension. The Horizon 2020 support mechanism is relevant to African countries that need 
more exchange to help them build their research capacity. 

Threats 

Stakeholders  referred  to  a  few  threats  to  the  potential  for  the  Horizon  2020  support 
mechanism  to  impact  on  other  HEIs  in  Europe,  and  on  international  cooperation  beyond 
Europe. 

First,  while  leaving  the  lead  on  international  cooperation  to  the  participating  HEIs  is  an 
opportunity, it is also a threat as it depends on countries and HEIs and their pre-existing links 
and priorities. Some HEIs may have limited links and others may prioritise cooperation with 
the  alliance  members  for  which  they  receive  funding  for  instance.  In  addition,  there  is  no 
homogeneity at country level regarding research priorities, so HEIs and alliances may have 
too broad a spectrum of topics to choose from and focus on. This lack of specialisation could 
limit their visibility. 

67  ENGAGE.EU,  ENGAGE.EU:  The  European  University  for  Business  Knowledge,  Economic  Capability-
Building and Societal Development, 21.10.2021, available at: https://www.engageuniversity.eu/mission/. 

79 

 
 
 
5.  Conclusions  

5.1  Overview 

This section summarises key conclusions for EQs 1 – 10 from the rest of this report. Following 
the  specification  of  outputs  in  the  terms  of  reference,  full  answers  to  the  remaining  two 
evaluation questions, i.e. EQ 11 (what lessons can be learned / recommendations made for 
the future) and EQ 12 (how can funding for alliances be more sustainable, including synergies 
with other EU / national funding) are set out in the accompanying Policy Report. 

The  evaluation  provides  good  evidence  that  the  Horizon  2020  funding  has  contributed 
positively  to  the  development  of  R&I  capacities  among  most  of  the  participating  HEIs. 
While  the  funding  has  not  in  itself  delivered  a  transformation  in  capacities,  there  are 
positive signs that it has helped to put many alliances on a pathway which could achieve 
this in due course, with an important caveat that this is dependent on continuing support. 

5.2  Overall assessment 

This Analytical Report presents key findings from an assessment of the Horizon 2020 SwafS 
grants for R&I projects implemented by 39 of the European Universities alliances from 2019 
- 2022. It is based on a review of programme and policy documents, interviews with 7 higher 
education  stakeholder  organisations  and  with  14  organisations  from  12  Member  States, 
interviews with 35 representatives of HEIs that belong to 27 different alliances, and an online 
survey responded to by 117 representatives of HEI alliance members. 

The R&I projects were distinctive in focusing on developing alliances’ collective capabilities 
for  collaboration  (e.g.  baseline  assessments,  strategies  and  structures)  within  the  six 
transformational modules, rather than ‘downstream’ R&I activities (such as research). There 
is good evidence that the R&I projects are successfully delivering the workplans set out in 
their  grant  agreements,  with  the  large  bulk  of  the  activities  planned  for  the  first  reporting 
period completed. This is particularly notable given the difficult circumstances created by the 
pandemic. 

It is important to note this evaluation has been carried out at a relatively early moment in what 
is widely expected to be a long-term process of institutional and cultural change. As the 17 
projects  from  the  first  2019  call  began  in  earnest  in  2020,  even  the  longest  established 
projects  have  only  had  three  years  to  collaborate  on  R&I  issues  (with  less  time  for  the 
remaining 22 alliances). Furthermore, the level of Horizon funding was relatively modest  – 
with grants of up to EUR 2 million for each alliance. 

5.3  How  far  has  the  funding  supported  the  transformation  of  R&I 

capabilities? 

Given limited timescales and budgets, a key question is what the Horizon funding has made 
so  far.  While  there  is  strong  support  for  the  value  of  the  alliance’s  R&I  work  among  all 
consultees for this evaluation, there are varying viewpoints on how transformative the funding 
has been up till now. 

Consultees from higher education stakeholder organisations (and to a lesser extent Member 
State  representatives)  tend  to  be  more  sceptical  of  the  transformative  potential  of  the 
projects. A point made repeatedly is that the level of funding is low compared to the ambitions 
of  the  initiative,  and  that  the  absence  of  certainty  about  future  funding  is  a  significant 
limitation. 

80 

 
In contrast, evidence from consultations and the survey with HEIs suggests a more positive 
picture, albeit with the same concerns about the need for a stable and predictable funding 
environment going forward.  

•  For a significant minority of alliances – around one third of the 39 alliances receiving 
support – responses from HEIs suggest the funding does indeed have the potential to 
promote significant transformation, if activities are continued.  

▪  On average, 33% of the 117 survey respondents said R&I collaboration within their 
alliance has helped their HEI advance ‘very significantly’ or ‘quite significantly’ (the 
figures range from 28% to 37% for different transformational modules).  

▪  For  those  saying  that  collaboration  in  the  Horizon  projects  helped  them,  areas  in 
which particular value was added (i.e. => 50% of respondents said collaboration was 
very or quite significant) include: inventories and plans for sharing infrastructure, new 
partnerships  for  R&I,  new  funding  applications,  entrepreneurial  skills  training, 
initiatives  to  support  engagement  with  citizens,  an  over-arching  R&I  strategy, 
activities to transfer academic knowledge into policy making, and open science skills 
training.  Highlighting  these  areas  should  also  not  diminish  the  appreciation  of 
benefits in other areas even if less prevalent.  

▪  An  important  caveat  is  that  for  a  substantial  proportion  of  cases  outputs  from 
collaborative work are still in development rather than complete. This underlines that 
important progress has been made, but also that alliances are still in the early stages 
of  the  transformative  process.  Where advances  have  been  made,  these  can  best 
therefore be understood as providing good foundations for future transformation. For 
instance,  while  there  are  examples  of  alliance  members  obtaining  new  research 
funding as a result of collaboration, this is not yet widespread across all alliances. 

•  For around half of the alliances, the responses received suggest that the funding has 

helped the alliance member own efforts but is not likely to be transformational.  

•  For a small minority – perhaps a tenth of HEIs – participation in an alliance seems to 
be  making  little  difference  to  efforts  to  build  and  refine  R&I  capabilities.  For  this  sub-
group it is however important to note that there are still benefits to participating – but that 
these may be concentrated across a small number of TMs, rather than across the wide 
range of areas for transformation. In other words, where respondents said participation 
under a particular TM did not help at all, this does not mean that they received no benefit 
from all TMs. 

There  are  indications  that  HEIs  in  the  latter  two  groups  (where  collaboration  helped 
somewhat,  or  not  significantly),  tend  to  be  larger,  to  already  have  well-developed  R&I 
capacities,  and  to  be  institutions  whose  own  strategies  and  national  policy  contexts  have 
already created a significant impetus towards developing R&I capabilities. 

While HEIs in all EU Member States have benefitted from the initiative, there is also 
evidence  that  the  gains  are  significantly  greater  for  universities  in  the  widening 
countries compared to those non-widening ones – with this being the case for five out of six 
transformational modules. This provides evidence for the hypothesis that institutional 
changes in the transformational modules will tend to accelerate access to excellence 
in less R&I intensive institutions/regions. 

•  Although  HEIs  based  in  non-widening  countries  benefit  relatively  less,  the  proportion 
saying  that  collaboration  helped  them  quite  significantly  or  very  significantly  is  still 
substantial, averaging 30% of respondents from these countries.  

81 

 
•  A  key  implication  of  this  analysis  is  that  while  the  Horizon  funding  for  R&I  has 
significantly promoted widening objectives, this is not its only benefit. While the 
identification  of  HEIs  in  non-widening  countries  with  ‘excellence’  is  too  simplistic,  the 
data does suggest that the Horizon funding has served the objective of promoting 
excellence, but to a lesser extent than supporting widening goals. 

•  However,  it  is  still  the  case  that  universities  from  both  groups  of  countries  can  make 
beneficial use of the funds, albeit with a different focus to support capacity building. 

With one exception, the evidence does not generally indicate that alliances that focus on a 
particular theme advanced further in developing their R&I capabilities. The one exception to 
this finding relates to TM2 – Sharing capacity, infrastructure and resources – where 46% of 
alliances with a thematic focus said that collaboration had helped them advance quite or very 
significantly, compared to 27% among non-thematic alliances. The reasons for this are not 
very clear, but it may be that focusing on a few thematic areas increases the relevance of 
other alliance members’ infrastructure and resources. 

Finally,  it  is  also  clear  that  smaller  universities  benefit  substantially  more  from 
collaboration than larger ones. This was the case for TM1: Shared R&I strategies, TM3: 
Strengthening human capital, TM4: Co-operation with other sectors and TM5: Knowledge- 
and  digitally-driven  HEIs/open  science  (but  not  for  TM2:  Sharing  capacity,  infrastructure, 
resources, or for TM6: Embedding citizens and society. 

5.4  Wider impacts of the Horizon 2020 funding 

There is extensive evidence that the R&I grants have helped to enhance cooperation 
between  universities  and  wider  innovation  ecosystems,  though  the  scale  of  impact 
from this is not easy to judge. Examples include initiating or enhancing cooperation with 
national,  regional,  and  local  authorities,  extending  cooperation  to  R&I  in  the  context  of 
existing city twinning arrangements, and strengthening relations with businesses and other 
stakeholders through new advisory boards with business expertise. Several alliances have 
also created new tools to support such cooperation, including databases of external partners 
and researchers and other IT applications that promote interaction among innovation system 
stakeholders. 

Most consultees indicated that some HEIs that are not part of the alliances have expressed 
interest  in  the  European  universities,  though  it  is  not  always  easy  to  distinguish  between 
interest  in  the  longer-standing  education  work  compared  to  R&I  elements.  The  clearest 
evidence of this is the expansion of the existing alliances (i.e. new members) and the growing 
number  of  applications  and  alliances.  In  some  Member  States  there  has  been  greater 
participation in the second 2020 call for proposals, reflecting this interest and also what some 
interviewees  highlighted  as  a  “fear  of  missing  out”  given  that  the  alliances  are  now 
becoming a fixed feature of the European HE landscape.  

There is also some anecdotal evidence of the alliances having a wider impact, such as an 
HE  stakeholder  organisation  noting  that  conversations  with  university  vice  rectors  for 
research  frequently  turn  to  the  subject  of  the  alliances,  or  another  consultee  saying  that 
applications for Erasmus grants are also now starting to highlight synergies with R&I work. 

In terms of impacts on Member States, some governments have provided some additional 
funding to their own universities to participate in alliances (often for the education and R&I 
elements together). However, apart from this funding, there are no significant signs that 
the Horizon funding has yet had any impact on R&I policy more widely. This contrasts 
with  the  situation  regarding  educational  policy  at  national  level  where  some  significant 
changes are beginning to be made to support the European University alliances.  

82 

 
The Horizon funding has prompted a moderate level of engagement on R&I issues with 
other universities beyond the EU, though a key factor here is the degree of pre-existing 
interaction between individual alliance members. It is notable that neither Erasmus nor 
Horizon 2020 support focused on engagement beyond the EU, and few of the R&I projects 
included measures for this. There are nonetheless cases where this is happening as part of 
the core work of the alliances, especially with universities in Switzerland, Ukraine, and the 
United Kingdom, and in several other instances. A few alliance representatives indicated that 
they were expecting further engagement beyond the EU to come, including by working with 
other alliances to identify opportunities after the pilot phase.  

5.5  Challenges  in  supporting  the  deepening  and  widening  of  the 

alliances’ R&I collaborations 

A number of learning points and challenges were identified in the evaluation process and are 
discussed in detail in the accompanying Policy Report. Some key points can be summarised 
here: 

•  There are continuing challenges in transnational collaboration between HEIs, whether in 
the education or R&I dimensions. These include different systems, legal and regulatory 
contexts,  national  policy  objectives  and  language  barriers.  The  combination  of  these 
challenges can impede the speed of collaboration or lead to ambitions being scaled back.  

•  There  is  almost  universal  consensus  that  moving  from  the  development  of  R&I 
capabilities  to  delivery  of  R&I  outcomes  will  require  sustained  and  more  predictable 
funding over several years. One challenge to this is the fact that while some Member 
States provide additional funding for HEIs that are alliance members, many do not. 

•  The potential for R&I transformation is weakened by the siloed nature of the two main 
funding  streams  (Erasmus+  and  Horizon),  and  the  different  administrative  rules  and 
timescales associated with each scheme. 

•  Engaging  researchers  in  capacity-building  projects  has  often  been  a  challenge. 
However, the evaluation has identified various examples of good practice. A common 
theme in all these examples is linking the development of joint research strategies with 
funding  opportunities  for  researchers,  or  at  least  the  future  prospect  of  such 
opportunities.  

•  Several interviewees highlighted the fact that while longer-term funding is a necessary 
condition for transformation, it is not sufficient, and that buy-in from academics at different 
levels within HEIs and a commitment to changing institutional cultures are also essential 
ingredients. 

83 

 
 
 
ANNEX 1: LIST OF INTERVIEWEES 

A: Coordinators and other representatives from alliance HEIs 

NAME 

ORGANISATION 

ALLIANCE 

University of Iceland 

University of Granada 

Dorothy Kelly 
Fernando Galan 
Ulfar Gislason  
Florence Voitier-Sienzonit  University of Toulouse 
Lisa Hasenbein  
Emilie Domanico 
Elsa Bedos 
Aurelien Krejbich 
Sarah Thompson  
Sébastien Le Picard 

CIVICA 

Technical University of Munich 
Universite de Montpellier 

Arqus R.I 

Aurora R.I 
Beyond UNIVERSEH 
BoostEuroTeQ 
CHARM-EU 

CIVICA Research 

Isabel Mercedes 

Guido van Leerzem 
Boudewijn Haverkort  

Patrick Reurink 

Université de Rennes 1 
Universidad 
Madrid  

Politecnica 

EDUC-SHARE 

de 

EELISA innoCORE 

University of Tilburg 

ENGAGE EU R-I 

Norwegian University of Science 
and Technology 
Karlsruhe Institute of Technology  EPICUR-Research 

ENHANCERIA 

TU Dresden 

Michael Zacherele  
Claudia Reichert 
Till Ansgar Baumhauer 
Jean-Marc Ogier  
Sandrine Lacombe 
Beatrix Busse  
Dalila Kurtalija Sabanic  
Manuel José Damasio  
Prof. Stefan Müller-Stach   Mainz University 
Cadiz University 
Laura Martin  
Leoben University 
Aga Kosciuszko  
University of Poitiers 
Ludovic Thilly  
Technological  University  of  the 
Shannon 

University of La Rochelle 
Université Paris - Saclay 
University of Cologne 
University of Gothenburg 
University of Lusofona 

Patrick Murray  

EU4ART_differences 

EU-CONEXUS-RFS 
EUGLOHRIA 
EUniWell Research 
EUTOPIA-TRAIN 
FILMEU_RIT 
FIT FORTHEM 
reSEArch-EU 
RE-EURECA-PRO 
RI4C2 

RUN-EU PLUS 

Kokvedgaard 

Miriam 
Zeiten  
Elena del Giorgio  
Isabelle Kratz  
Teresa Amido  
Sath Amanfo 
Emily Palmer  
Roberto Zanino  
Marios Demetriades  

University of Copenhagen  

Universita degli Studi Di Milano  
Sorbonne University  
Saarland University 

TRAIN4EU+ 

T4ERI 

University of Edinburgh 

Una.Resin 

Politecnico di Torino 
University of Cyprus 

UNITE.H2020 
YUFERING 

84 

 
 
 
 
B: Representatives of Member State organisations 

NAME 

Gunhild Kiesenhofer-
Widhalm  

Didier Flagothier 

Milena Damyanova, Yanita 
Zherkova, Zornitsa 
Georgieva, Ivana 
Radonova 
Jonas Hussum 
Johannesen  
Saara Vihko 
Siegfried Martin-Diaz 
Ella Bouquet 

Anne-Sophie Gallou 

Claudia Hefftler 

Nils Woerner 

Agota David  

Francesca Spigarelli  
Fulvio Esposito  
Simona Rataj 

Ewa Kociańska Lange 

Cecilia Cabello Valdes  

ORGANISATION 

COUNTRY 

Federal Ministry of Education, 
Science and Research  
Département de la Recherche et du 
Développement Technologique 

Austria 

Belgium 

Ministry of Education and Science 

Bulgaria 

Ministry of Higher Education 

Ministry of Education and Culture 
Département des Stratégies de 
l’Espace Européen de 
l’Enseignement Supérieur et de la 
Recherche  
Federal Ministry of Education & 
Research 
German Rectors’ Conference 
National Research, Development & 
Innovation Office 
Università degli Studi di Macerata 
Università di Camerino 
University of Ljubljana 
National Centre for Research and 
Development 
Fundación Española para la Ciencia 
y la Tecnología  

Denmark 

Finland 

France 

Germany 

Hungary 

Italy 

Slovenia 

Poland 

Spain 

C HE stakeholder organisations 

NAME 

Doris Alexander  
Jorg Langwaldt 
Matthias Bjornmalm  
Prof Marjolijn Brussard 
Toni Leander 
Stephane Bergmans  
Anna-Lena Claeys-Kulik 
Emmanuelle Gardan 
Benjamin Martínez Sanchis  

Silvia Gomez Recio 

Laura Keustermans 

ORGANISATION 
FOREU1 – grouping of universities from the 2019 
Horizon call 
CESAER Group  

UAS4EUROPE 

European Universities Association (EUA) 

Coimbra Group  

Young European Research Universities Network 
(YERUN) 
League of European Research Universities (LERU) 

85 

 
 
 
 
ANNEX 2: DOCUMENTS REVIEWED 

A: Mid-term reports 

•  PERIODIC TECHNICAL REPORT AT MIDTERM ARQUS RESEARCH & INNOVATION 

•  CIVICA Research Periodic Technical Report 

•  EDUC-SHARE Periodic Technical Report 

•  EPICUR Activities for Citizens Dialogue 

•  EPICUR project Studies on Trust in Science in different (European) Countries 

•  EPICUR project Periodic Technical Report 

•  EU4ART_differences Periodic Technical Report Part B 

•  EU-CONEXUS RESEARCH FOR SOCIETY Periodic Technical Report Part B 

•  THE EUROPEAN ALLIANCE FOR GLOBAL HEALTH (EUGLOHRIA) Periodic Technical 

Report Part B 

•  EUTOPIA Periodic Technical Report Part B 

•  FIT FORTHEM Periodic Technical Report Part B 

• 

reSEArch-EU Periodic Technical Report Part B 

•  RIS4CIVIS Periodic Technical Report Part B 

•  ECIU University Research Institute for Smart European Regions Mid-term report Part B 

•  TORCH Periodic Technical Report Part B 

•  TRAIN4EU Periodic Technical Report Part B 

•  Una.Resin Periodic Technical Report Part B 

•  UNITE.H2020 Periodic Technical Report Part B 

•  YUFERING 1st Periodic Technical Report 

B: Policy briefs 

•  ARQUS RESEARCH & INNOVATION European Research Area (ERA) Policy Brief 

•  CIVICA Research Mid-term Policy Brief 

86 

 
 
•  EDUC-SHARE European Research Area (ERA) Policy Brief 

•  EPICUR Research European Research Area (ERA) Policy Brief 

•  EU4ART_differences European Research Area (ERA) Policy Brief 

•  EU-CONEXUS  RESEARCH  FOR  SOCIETY  European  Research  Area  (ERA)  Policy 

Brief 

•  THE  EUROPEAN  ALLIANCE  FOR  GLOBAL  HEALTH  (EUGLOHRIA)  European 

Research Area (ERA) Policy Brief 

•  EUTOPIA European Research Area (ERA) Interim policy report 

•  FIT FORTHEM European Research Area (ERA) Policy Brief 

•  FIT FORTHEM Alliance Universities Selected Good Practices in R&I 

• 

reSEArch-EU First Policy Brief 

•  RIS4CIVIS European Research Area (ERA) Policy Brief 

•  ECIU  University  Research  Institute  for  Smart  European  Regions  European  Research 

Area (ERA) Policy Brief 

•  TORCH European Research Area (ERA) Policy Brief 

•  TRAIN4EU European Research Area (ERA) First Policy Brief 

•  Una.Resin Publication of a short policy brief on interim findings 

•  UNITE.H2020 European Research Area (ERA) Policy Brief 

•  YUFERING European Research Area (ERA) Policy Brief 

C: Other documents 

•  Berger,  F.  et  al  (2021)  Knowledge  ecosystems  in  the  new  ERA  –  Work  Package  5  - 
Developing  a  monitoring  methodology  and  dashboard  concept  on  (institutional) 
transformation in the area of R&I at universities in Europe 

•  Claeys-Kulik  A.,  Pruvot  E.,  Estermann  T.  and  Jørgensen  T.  (2022)  Briefing  on  The 
European  Universities  Initiative  and  system  level  reforms  -  current  challenges  and 
considerations for the future 

•  European  Research  Area  (2022)  ERA  POLICY  AGENDA  –  ACTION  13  (Action  13: 
HIGHER EDUCATION INSTITUTIONS  – Empowering higher education institutions to develop in 
line with the ERA, and in synergy with the European Education Area). 

87 

 
 
•  Gunn, A (2021) The European Universities Initiative: A Study of Alliance Formation in 
Higher Education. Article in Curaj, A. et al European Higher Education Area: Challenges 
for a New Decade 

•  Horizon  Programme  (2020)  H2020  Programme  Guidance  document  “Support  for  the 

Research and Innovation Dimension of European Universities” 

•  Horizon Programme (2020) Work Programme 2018-2020 “Science with and for Society” 

• 

Jávorka Z., Allinson R (2021) Knowledge ecosystems in the new ERA: Policy Brief for 
the European Strategy for Universities (ES4U)  

•  Kalodimou V “From Horizon 2020 to Horizon Europe: Changes in Widening countries” 

•  PPMI  (2020)  Study  on  the  impact  and  transformational  potential  of  the  European 

University Initiative 

•  PPMI (2022) Final interim report “Study on the Outcomes and Transformational Potential 

of the European Universities Initiative” 

•  Unpublished  study  commissioned  by  DG  Research  &  Innovation  (2022)  Report  on 

Progress of University Alliances to Realise Transformation Modules, REA 

88 

 
 
ANNEX 3: INTERVENTION LOGIC FOR THE HORIZON FUNDING FOR ALLIANCES R&I  

89 

 
 
5.  Promote gender equality and foster 

TM3: Strengthening human capital 

4: 

ANNEX 
EUI 
TRANSFORMATIONAL  MODULES  AND  EUROPEAN  RESEARCH 
AREA PRIORITIES 

CORRESPONDENCE 

BETWEEN 

ERA Priority Area 

EUI transformational module for R&I 
ERA Priority Area: deepening a truly functioning internal market for knowledge 
1.  Enable Open Science, including 
through the European Open 
Science Cloud (EOSC) 

TM5:  Mainstreaming  of  open  science  and 
reforming  research  and  academic  career 
assessment systems 
TM5:  Mainstreaming  of  open  science  and 
reforming  research  and  academic  career 
assessment systems 
TM5:  Mainstreaming  of  open  science  and 
reforming  research  and  academic  career 
assessment  systems  +  TM3  careers  +  TM6 
citizens + TM4 cooperation with other sectors 
TM3: Strengthening human capital 

2.  Propose an EU copyright and data 
legislative framework for research 

3.  Reform the Assessment System for 

research, researchers and 
institutions 

4.  Promote attractive research 

careers, talent circulation and 
mobility 

inclusiveness 

6.  Protect academic freedom in 

Europe 

7.  Upgrade EU guidance for a better 

knowledge valorisation 

8.  Strengthen research infrastructures 

TM4:  Reinforcing  cooperation  in  R&I  with 
other sectors 
TM2:  Sharing  capacity,  infrastructure  and 
resources 
All modules 

9.  Promote international cooperation 
ERA Priority Area: Taking up together the challenges posed by the twin green and 
digital transition, and increasing society’s participation in the ERA 
TM1: common R&I agenda  
10.  Make EU research and innovation 
missions and partnerships key 
contributors to the ERA 

11.  An ERA for green transformation 
12.  Accelerate the green/digital 

transition of Europe’s key industrial 
ecosystems 

TM1: common R&I agenda 
TM4:  Reinforcing  cooperation  in  R&I  with 
other sectors 

13.  Empower Higher Education 

All TMs 

Institutions 

14.  Bring Science closer to citizens 

TM6:  Involving  and  engaging  citizens,  civil 
society and public authorities in R&I 

ERA Priority Area: Amplifying access to research and innovation excellence across 
the union  
15.  Build-up research and innovation 

TM4:  Reinforcing  cooperation  in  R&I  with 
other sectors 

ecosystems to improve excellence 
and competitiveness 

16.  Improve EU-wide access to 

all TMs 

excellence 

17.  Enhance public research 

TM3: human capital 

institutions’ strategic capacity 

ERA Priority Area: Advancing concerted research and innovation investments and 
reforms 

90 

 
 
18.  Support the development of EU 

All TMs 

countries’ national processes for the 
ERA implementation 

19.  Establish an ERA monitoring 

system 

20.  Support research and innovation 

investments and reforms 

Source  (ERA  Priority  Areas):  https://research-and-innovation.ec.europa.eu/system/files/2021-
11/ec_rtd_era-policy-agenda-2021.pdf 

91 

 
 
 
GETTING IN TOUCH WITH THE EU 

In person 
All over the European Union there are hundreds of Europe Direct centres. You can find the address of 
the centre nearest you online (european-union.europa.eu/contact-eu/meet-us_en). 

On the phone or in writing 
Europe Direct is a service that answers your questions about the European Union. 
You can contact this service: 
-  by freephone: 00 800 6 7 8 9 10 11 (certain operators may charge for these calls), 
-  at the following standard number: +32 22999696,  
-  via the following form: european-union.europa.eu/contact-eu/write-us_en. 

FINDING INFORMATION ABOUT THE EU 

Online 
Information about the European Union in all the official languages of the EU is available on the Europa 
website (european-union.europa.eu). 

EU publications 
You can view or order EU publications at op.europa.eu/en/publications. Multiple copies of free 
publications can be obtained by contacting Europe Direct or your local documentation centre 
(european-union.europa.eu/contact-eu/meet-us_en). 

EU law and related documents 
For access to legal information from the EU, including all EU law since 1951 in all the official language 
versions, go to EUR-Lex (eur-lex.europa.eu). 

EU open data 
The portal data.europa.eu provides access to open datasets from the EU institutions, bodies and 
agencies. These can be downloaded and reused for free, for both commercial and non-commercial 
purposes. The portal also provides access to a wealth of datasets from European countries. 

 
 
 
 
 
 
 
 
 
Following calls for proposals under Horizon 2020 in 2019 and 
2020, EU grants of up to EUR 2 million were offered to 39 
European University alliances to implement institutional 
changes that would develop their research and innovation (R&I) 
capabilities, both as individual HEIs and collectively as 
alliances. The funding was intended to support the development 
and initial implementation of joint strategies and action plans, 
where possible in connection with other EU or national funding 
sources. This analytical report presents an assessment of the 
effectiveness and impact of the funding on institutional changes 
within the alliances in the period 2019-2022. The assessment 
was undertaken as part of the study entitled Towards a 
European Excellence Initiative". 

Studies and reports