COMMISSION STAFF WORKING DOCUMENT Mid-term evaluation of the Erasmus+ programme (2014-2020) Accompanying the document REPORT FROM THE COMMISION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF REGIONS Mid-term evaluation of the Erasmus+ programme (2014-2020)
Date: 2018-01-31
Institutions: Directorate-General for Education, Youth, Sport and Culture, European Commission
Subjects: EU programme, cooperation in the field of education, education, education policy, sport, vocational training, youth policy

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 31.1.2018 SWD(2018) 40 final  COMMISSION STAFF WORKING DOCUMENT Mid-term evaluation of the Erasmus+ programme (2014-2020) Accompanying the document REPORT FROM THE COMMISION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF REGIONS  Mid-term evaluation of the Erasmus+ programme (2014-2020) {COM(2018) 50 final}  
 
 
Table of Contents 

1. 

2. 

3. 

4. 

5. 

INTRODUCTION ............................................................................................................................ 5 

BACKGROUND TO ERASMUS+ AND ITS PREDECESSOR PROGRAMMES ........................ 7 

IMPLEMENTATION OF ERASMUS+ UNTIL 2016 .................................................................. 12 

METHOD OF THE EVALUATION ............................................................................................. 14 

FINDINGS ..................................................................................................................................... 20 

5.1.  Effectiveness ............................................................................................... 20 

5.1.1.Outcomes for learners and practinioners………………………………........20 

5.1.2.Outcomes for organisations and systems……………………………….......31 

5.1.3. Transversal questions about effectiveness………………………………….40 

5.2.  Relevance .................................................................................................... 44 

5.3.  Coherence ................................................................................................... 54 

5.4.  Efficiency and simplification ..................................................................... 63 

5.5.  European added value ............................................................................... 72 

6. 

CONCLUSIONS ........................................................................................................................... 79 

ANNEX 1: PROCEDURAL INFORMATION  ............................................................... 78 

ANNEX 2: STAKEHOLDER CONSULTATION  .......................................................... 82 

ANNEX 3: METHODS AND ANALYTICAL MODELS ............................................... 91 

ANNEX 4: EVALUATION: CRITERIA AND QUESTIONS ...................................... 103 

ANNEX 5A: INTERVENTION LOGIC OF ERASMUS+ AND ITS PREDECESSOR 

PROGRAMMES .......................................................................................... 107 

ANNEX 5B: FACTSHEET ON ERASMUS+................................................................ 113 

ANNEX 5C: SPECIFIC OBJECTIVES OF ERASMUS+ ............................................. 115 

ANNEX 5D: FACTSHEETS ON PREDECESSOR PROGRAMMES ......................... 117 

ANNEX 5E: LEGAL BASIS INDICATORS ................................................................. 127 

ANNEX 6: GLOSSARY  ................................................................................................ 135 

1 

 
 
 
 
 
 
 
 
Executive Summary 

The 2014-2020 Erasmus+ programme and its predecessor programmes (2007-2013) 
have  provided  millions  of  people  in  Europe  and  beyond  with  opportunities  to  learn, 
volunteer  or  teach  abroad.  It  is  one  of  the  EU's  best-known  successes.  Its  impact  goes 
beyond  individuals,  as  it  also  has  a  positive  impact  on  education,  training,  youth  and 
sport  organisations  or  related  systems  and  policies  in  developing  cross-country 
cooperation. 

This  mid-term evaluation of Erasmus+  looks at the current programme until the end of 
2016.  It  also  includes  an  evaluation  of  the  long-term  effects  of  the  predecessor 
programmes.  It  is  based  on  a  very  reliable  methodology  (see  5.3  for  its  strengths  and 
limitations)  and  over  a  million  responses  from  interested  parties.  It  shows  that  at  mid-
term, the programme is on track to achieve or exceed the vast majority of the targets 
set in the Erasmus+ Regulation (EU) No 1288/2013. 

Effectiveness and European added value 

1) The evaluation finds that the Erasmus+ programme is highly valued by the general 
public as well as by its stakeholders. This finding is linked to the available evidence that 
the programmes under evaluation – both under their intra-EU and external dimensions - 
achieve  a  broad  range  of  concrete and  positive impacts  on  their  beneficiaries: learners 
(students, apprentices, volunteers, young people, etc.);  practitioners (teachers, trainers, 
youth workers, staff, etc.); participating organisations (schools, universities, youth and 
sport  organisations,  providers  of  vocational  education  and  training  and  of  adult 
education, etc.).  

2)  Though  less  visible,  the  evaluation  confirms  the  systemic  effect  of  the  evaluated 
programmes  on  education,  training,  youth  and  sport  policies  and  systems,  directly 
through the critical mass reached at least in the higher education sector  or indirectly in 
funding  policy  cooperation  (Open  Method  of  Coordination).  This  systemic  effect  goes 
together  with  partial  progress  made  in  the  area  of  dissemination  of  results  of  the 
programme.  However,  the  evidence  of  the  exploitation  of  project  results  by  policy 
makers  and  the  effective  engagement  of  the  latter  when  they  are  not  included  in  the 
project itself is not always clear. In this sense, the evaluation found that the dissemination 
of results is one of the aspects of Erasmus+ where there is room for further improvement. 
The evaluation also noted that the impact of funded projects on national systems could be 
more systematic if there were more cooperation projects fit for mainstreaming, focussed 
on  fewer  priorities  at  EU  level  and  further  efforts  made  for  mainstreaming  these  at 
national level. The systemic impact of the actions of a new kind introduced only in 2014 
(KA3, alliances, etc) is meant to be evaluated at final stage, after 2020. 

3)  The  evaluation  considers  there  is  potential  for  better  definition  of  actions  to 
maximise the programme's impact in Adult Education, sport, Jean Monnet activities and 
the  Student  Loan  Guarantee  Facility.  Considering  the  funding  available  at  EU  level, 
evidence shows that: the  contribution in  the  adult  learning sector is  diluted due to the 
wide size of the target population and the fragmented and diverse nature of the sector; in 
the  field  of  sport,  resources  should not  be spread  too  thinly  to  have  meaningful  result; 
the Student Loan Guarantee Facility has not yet lived up to volume expectations partly 
due  to  delays  in  its  launch.  Regarding  Jean  Monnet  activities,  there  is  a  need  to 
(notably  school  pupils')  awareness  and 
strengthen 
understanding of European integration. 

the  youngest  generation's 

4) In light of the impacts achieved in all other areas, the evaluation highlights the strong 
European  added  value  of  Erasmus+  and  its  predecessor programmes,  compared  to 

2 

 
 
what  could  have  been  achieved  with  similar  programmes  focused  on  separate 
geographical areas. This added value is the result of greater impact due to a much higher 
volume  and  wider  scope  of  funded  activities,  fairer  access  to  learning  mobility,  deeper 
EU  integration,  a  clearer  international  dimension  and  mainstreamed  best  practices. 
However,  the  programme's  potential  for  stimulating  learning  innovation  seems  to  be 
lagging behind and could be further exploited in the future. 

5)  The  evaluation  concludes  that,  in  the  absence  of  Erasmus+  and  its  predecessors, 
there  would  be  clear  negative  effects  on  learning  mobility  abroad,  transnational 
cooperation  among  organisations,  including  with  partner  countries,  integration  between 
European countries as well as the attitude of participants towards the EU. 

Erasmus+  is  more  coherent,  more  relevant  and  only  partly  more  efficient  than  its 
predecessors. 

1)  The  main  structural  change  of  the  Erasmus+  programme,  compared  to  its 
predecessors, is its integrated nature which has contributed to enhance the programme's 
internal  coherence.  Erasmus+  covers  learning  in  all  its  contexts  –  whether  formal  or 
non-formal, including youth work and sport – and at all levels of the lifelong learning 
continuum:  from  early  childhood  education  and  schools,  vocational  education  and 
training (VET), adult learning to higher education, including its international dimension. 
The  evaluation  highlighted  the  positive  effects  of  this  integrated  approach  underpinned 
by the lifelong learning logic. 

The evaluation highlights the following  positive consequences  of the Erasmus+ design, 
which has resulted in: 

  a  sharp  increase  of  cross-sectoral  cooperation  between  education  and  training 

sectors, youth and sport; 

  an  improved  geographical  balance  with  small  countries  and  countries  from 

Central and Eastern Europe being better integrated; 

  a simplified architecture in three key actions; 
  a  single  brand  name  which  has  contributed  to  the  programme's  increased 

visibility and a progressively strong adherence by the sectors covered. 

2) In terms of policy relevance, the evaluation shows that stakeholders see Erasmus+ as 
being  more  clearly  aligned  with  EU  policies  and  priorities  than  predecessor 
programmes.  However,  a  majority  of  programme  countries  call  for  more  flexibility  at 
national  level  and  the  evaluation  has  shown  that  to  maximize  the  impact  of  the 
programme priorities could be reduced and better focused. 

3)  The  evaluation  found  a  high  complementary  between  Erasmus+  and  other  EU 
policies and programmes relevant to education, training, youth and sport (e.g. Europeran 
Social Fund, Horizon 2020). Although the level of synergies differs, it is notable that the 
evaluation detected very few overlaps. 

4)  When  it  comes  to  budget,  the  evaluation  concludes  that  more  is  needed  for  the 
programme to reach a critical mass in sectors other than higher education. The demand 
largely  exceeds  the  funding  available  including  in  higher  education.  The  evaluation, 
including  the  public  consultation,  suggests  that  a  further  reconsideration  of  the 
programme's financial envelope is needed. Without prejudice to negotiations on the next 
Multi  Financial  Framework,  the  evaluation  shows  that  the  budget  could  be  differently 
shared between the programme sectors, in particular at the advantage of sectors showing 
the  highest  performance,  but  which  have  received  relatively  less  funding  up  until  now, 
such as school education and vocational education and training. 

3 

 
 
 
5)  The  evaluation  shows  that  Erasmus+  mobility  actions  are  clearly  cost-effective, 
especially  learners'  mobility  (with  a  cost  for  the  EU  of  15€  per  day/learner).  Cost-
effectiveness of other actions remains harder to quantify. The management costs (6% of 
Erasmus+  administrative  and  operational  budget)  are  deemed  reasonable,  especially 
when  compared  to  similar  national  schemes  (14%  in  average).  The  overall  efficiency 
stemming from the merge of 7 predecessor programmes is not yet clear. More efficiency 
gains are expected to materialise during the growing phase foreseen by the budget profile 
of Erasmus+ until 2020. This will have to be evaluated at final stage. 

6)  In  terms  of  programme  management,  the  division  of  responsibilities,  as  inherited 
from predecessor programmes, between the Commission, National Authorities, National 
Agencies  and  EACEA,  is  overall  clear  and  fit  for  purpose.  A  majority  of  programme 
countries  wish  more  flexibility in  implementing  the  budget.  A  truly  performance-based 
approach  has  been  adopted,  though  some  indicators  need  to  be  fine-tuned  and  less 
information collected while being better exploited. 

7)  However,  there  is  clearly  a  repeated  call  for  further  simplification.  Following  a 
difficult transitional period/learning process, there is broad agreement that Erasmus+ has 
brought  major  improvements  (e.g.  simplified  grants,  digitalisation,  VET  Charter, 
linguistic support, etc.) but that procedures of application and reporting could be further 
simplified  to  reduce  the  administrative  burden  on  beneficiaries.  Applicants  for  small 
projects  are  too  often  expected  to  meet  the  same  requirements  as  applicants  for  large 
ones.  IT  tools  are  not  inter-operable  and  enough  user-friendly.  The  application  process 
could more clearly focus on those criteria that matter most for effectiveness. A specific 
challenge  is  to  improve  the  efficiency  of  the  decentralised  international  credit  mobility 
action which includes more than 12 different budget envelopes for partner countries. 

Overall,  the  evaluation  found  that  all  the  evaluated  predecessor  programmes  were/are 
highly  effective,  whereas  Erasmus+  is  more coherent,  more  relevant  and  only  partly 
more efficient than its predecessors. 

4 

 
 
 
 
 
 
 
 
 
1. 

INTRODUCTION 

The  Union  Programme  for  Education,  Training,  Youth  and  Sport  for  the  period  2014-
2020 'Erasmus+', and its predecessor programmes for the years  2007-2013, support learning 
mobility  of  individuals  worldwide,  transnational  cooperation  between  organisations  and 
promote Member States' reforms in the education, training, youth and sport fields. This is how 
the EU invests in people with a view to unlocking individuals' potential regardless of age or 
background, in support of Member States' efforts to develop human resources in Europe and 
beyond. 

Erasmus+ contributes to the objectives of the Europe 2020 Strategy1 and more specifically to 
the Strategic Framework for European Cooperation in Education and Training (ET 2020)2, the 
European Youth Strategy3 and the EU policy in the field of sport4. Erasmus+ also contributes 
to  the  EU's  more  recent  overall  political  objectives,  such  as  the  European  Pillar  of  Social 
Rights and the EU Global Strategy on Foreign and Security Policy.  

The  Erasmus+  programme  (2014-2020)5  has  integrated  all  previously  existing  EU 
programmes  in  the  domains  of  Education,  Training,  Youth  and  Sport,  and  includes  an 
international dimension that is funded by different external action instruments. In the fields of 
Education, Training and Youth, the programme pursues its objectives through three types of 
actions:  learning  mobility  abroad,  transnational  cooperation  projects  and  policy  support. 
Separately,  the  Jean  Monnet  activities  promote  teaching  and  research  on  the  European 
integration6,  while  the  Erasmus+  supports  transnational  cooperation  activities  in  the  field  of 
sport, focusing in particular on grassroots level sport. 

The  Regulation  establishing  the  Erasmus+  Programme  stipulates  that  a  mid-term  evaluation 
report accompanied, if appropriate, by a legislative proposal to amend the Regulation, shall be 
submitted  by  the  Commission  to  the  European  Parliament,  the  Council,  the  European 
Economic  and Social  Committee  and the Committee of the Regions  by 31 December 2017. 
Furthermore,  the  mid-term  evaluation  shall  include  the  ex-post  evaluation  of  the  Erasmus+ 
predecessor programmes over the period 2007-2013 i.e. Lifelong Learning, Youth in Action, 
Erasmus Mundus, ALFA, Tempus, Edulink and sport preparatory actions, taking into account 
their long-term results and impact. Therefore, this report covers actions for the period 2007-
2016  in  all  programme  countries  (Member  States  of  the  European  Union,  the  Former 
Yugoslav  Republic  of  Macedonia,  Iceland,  Liechtenstein,  Norway,  Turkey)  and  partner 
countries (neighbouring the European Union and other partner countries). The  baseline for 
this  evaluation  is  the  period  2007-2013  unless  otherwise  specified.  In  total,  the  period 
under evaluation corresponds to a total budget of over EUR 15 billion. 

The  objectives  of  the  mid-term  evaluation  are  to  assess  five  evaluation  criteria  a)  the 
effectiveness  of  the  measures  taken  to  achieve  the  Erasmus+  programme's  objectives, 
including  the  contribution  made  to  the  realisation  of  the  Europe  2020  strategy;  b)  the 
continued  relevance  of  all  of  its  objectives;  c)  the  Programme's  internal  and  external 

1 http://ec.europa.eu/europe2020/index_en.htm 
2 http://ec.europa.eu/education/policy/strategic-framework/index_en.htm 
3 http://ec.europa.eu/youth/policy/youth_strategy/index_en.htm 
4 http://ec.europa.eu/sport/policy/index_en.htm 
5  Regulation  (EU)  No  1288/2013  of  the  European  Parliament  and  of  the  Council  of  11  December  2013  establishing 
'Erasmus+':  the  Union  programme  for  education,  training,  youth  and  sport  and  repealing  Decisions  No  1719/2006/EC,  No 
1720/2006/EC and No 1298/2008/EC http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32013R1288  
6  Article  10  of  the  Regulation  (EU)  No  1288/2013;  see  ICF,  stand-alone  report  on  Jean  Monnet  actions  ("ICF/JMO" 
hereinafter), ICF, vol. 2. The relevance of the institutions designated in Article 10(c) of Regulation (EU) No 1288/2013 and 
pursuing  an  aim  of  European  interest  and  funded  under  Jean  Monnet  activities  (about  €29  million  in  2016)  has  not  been 
evaluated due to its discretionary nature. 

5 

 
 
                                                            
coherence; d) the efficiency of the Programme and the scope for simplification; and lastly e) 
its  European  added  value.  The  results  of  the  evaluation  will  be  used  to  improve  the 
implementation of the current programme until 2020 where possible. They will also be used 
to feed into the impact assessment for its successor programme.  
The mid-term evaluation assesses the performance of the different (Key) actions7 included in 
Erasmus+ and evaluates to which extent findings differ across fields and sectors8 included in 
the programme and across the different target levels (individuals, organisations and systems). 
It also assesses the extent to which the programmes have contributed to policy development 
and implementation in the participating countries. The evaluation does not cover actions that 
were discontinued during the previous programming period (2007-2013)9 as these were not to 
be  evaluated,  under  the  Terms  of  Reference.  However,  feedback  from  stakeholders  on  the 
interruption of actions discontinued with Erasmus+ was collected and analysed. 

This  Staff  Working  Document  on  the  mid-term  evaluation  of  Erasmus+  and  predecessor 
programmes  has  been  carried  out  according  to  a  Roadmap  published  in  January  201610.  It 
draws mainly, among other sources11, on the National Reports submitted by the programme 
countries  in  accordance  with  Article  21(4)  of  Regulation  (EU)  No  1288/2013,  previous 
evaluations  and  other  studies12  and  the  final  deliverables  prepared  by  ICF  Consulting 
Services  Ltd,  (hereinafter  ICF),  under  contract  to  European  Commission  (DG Education, 
Youth,  Sport  and  Culture)13.  The  final  report  delivered  by  ICF  provides  answers  to  all 
evaluation questions14 defined in the Terms of Reference and related to the aforementioned 
five  evaluation  criteria.  The  final  report  of  the  contractor  contains  also  recommendations 
addressed to the Commission.  

7  The  actions  of  the  programme  are  defined  in  Articles  6-10  (education  and  training),  12-15  (youth)  and  17  (sport)  of 
Regulation (EU) No 1288/2013. They include Key Actions 1 (mobility for learners and practitioners), 2 (cooperation among 
organisations) and 3 (policy support), Jean Monnet activities, sport activities, and their respective specific actions. 
8 The fields of the programme are defined in Article 1.3 of the legal base. They consist of education and training (including 
five sectors: school education, vocational education and training, higher education (HE), international higher education and 
adult learning), youth, and sport. 
9  The  external  cooperation  agreements  in  HE,  training  and  youth  with  the  USA  and  Canada  were  discontinued  in  2011 
following unilateral decisions by these two countries to interrupt co-funding in the aftermath of the economic crisis. 
10 http://ec.europa.eu/smart-regulation/roadmaps/docs/2015_eac_014_evaluation_erasmus_en.pdf 
11 See annex 3 for sources, data collection and method of analysis 
12  The  Evaluation  of  the  EU  Development  Co-operation  Support  to  Higher  Education  in  Partner  Countries  (2007-2014), 
August  2017,  Particip,  LeA,  ECDPM,  Ecorys  for  the  Directorate-General  for  International  Cooperation  and  Development 
(European  Commission)  includes  the  ex  post  evaluation  of  the  four  predecessor  programmes  in  the  field  of  international 
higher education. 
13 See annex 1 for evaluation process 
14 See annex 4 for evaluation questions 

6 

 
 
 
 
                                                            
2.  BACKGROUND TO ERASMUS+ AND ITS PREDECESSOR PROGRAMMES 

Erasmus+ is the EU Programme in the field of education, training, youth and sport, with 
a  budget  of  EUR  16.45  billion  for  the  period  2014-202015.  It  provides  opportunities  for 
people  of  all  ages  (university  students  but  also  school  pupils,  trainees,  apprentices,  etc.)  to 
study,  be  trained,  volunteer  and  learn  in  other  countries.  It  also  fosters  the  professional 
development  of  practitioners  and  supports  cooperation  on  tangible  results,  networking  and 
share  of  knowledge  among  organisation  and  institutions  in  the  fields  covered  by  the 
programme.  

Through  cooperation  in  formal,  informal  and  non-formal  learning,  the  Programme  aims  to 
address  the  following  challenges:  economic  recovery  and  high  youth  unemployment;  skills' 
mismatches,  low  employability  and  education  poverty;  global  competition  for  talents; 
Information  and  Communication  Technology  potential  and  digital  divide;  social  exclusion 
and intolerance; lack of trust in the EU and low participation in democratic life; threats to the 
integrity of sport and, more generally, to common European values. 

The  general  objectives16  of  Erasmus+  are  to  contribute  to  the  Europe  2020  strategy  for 
growth  and  jobs17,  including  the  headline  education  targets18,  as  well  as  the  strategic 
framework for European cooperation in education and training ('ET 2020'), including related 
benchmarks. Erasmus+ also aims to contribute to achieving the objectives of the  EU Youth 
Strategy  and  the  EU  Work  Plan  for  Sport,  to  promote  the  sustainable  development  of 
partner  countries  in  the  field  of  higher  education  and  youth,  as  well  as  to  foster  European 
values19. 

Specific objectives20 tackled by the programme include the improvement of the level of key 
competences  and  skills,  with  particular  regard  to  their  relevance  for  the  labour  market  and 
their  contribution  to  a  cohesive  society;  the  promotion  of  solidarity  and  participation  in 
democratic  life  in  Europe  and  the  labour  market;  the  improvement  of  quality,  innovation, 
excellence  (including  in  European  studies)  and  internationalisation  at  the  level  of 
organisations  and  practitioners;  support  to  the  modernisation  of  education  and  training 
systems,  in  particular  through  evidence-based  policy  cooperation;  the  enhancement  of  the 
European/international  dimension  of  its  sectors,  including  with  partner  countries  in 
complementarity  with  the  Union's  external  action;  the  promotion  of  the  Union's  linguistic 
diversity and intercultural awareness; cross-border threats to the integrity of sport; support to 
good  governance  in  sport  and  dual  careers  of  athletes;  and  the  promotion  of  voluntary 
activities in sport. 

following  predecessor  European 
Erasmus+ 
interventions21 implemented during the period 2007-2013: Lifelong Learning (LLP)22, Youth 

integration  of 

results 

from 

the 

the 

as 

the 

from 

EUR 

as  well 

85  million 

appropriations, 

European  Development 

15 The Programme has an overall indicative financial envelope of 14.774 billion EUR under Heading 1 "Smart and Inclusive 
Growth" and of 1.680 billion EUR under Heading 4 "Global Europe" of the EU Budget for the seven years (2014-2020), EU-
Fund 
28 
https://ec.europa.eu/programmes/erasmus-plus/programme-guide/part-a/what-is-the-budget_en  
16 Article 4 of the Regulation (EU) No 1288/2013 
17 https://ec.europa.eu/info/strategy/european-semester/framework/europe-2020-strategy_en  
18 In EU average: rate of early school leavers below 10%; at least 40% of people aged 30–34 having completed HE. 
19 In accordance with Article 2 of the Treaty on European Union 
20 Articles 5, 11 and 16 of the Regulation (EU) No 1288/2013 
21  See  Annex  5b  and  5d  for  more  detailed  factsheets  on  each  programme;  see  also  annex  1  of  ICF's  final  report  for  a 
comparative tables of Erasmus+ and predecessor programmes 
22  LLP  accounting  for  84%  of  the  budget  for  education  and  training  2007-2013  comprised  4  sector-based  programmes: 
Comenius  (SE);  Leonardo  da  Vinci  (VET),  Erasmus  (HE),  Grundtvig  (AE),  transversal  programme  (policy  cooperation; 
language learning; ICT-based innovation, dissemination of results) and the Jean Monnet programme. 

7 

 
 
                                                            
in  Action  (YiA),  Preparatory  Actions  in  sport  (as  of  2009),  as  well  as  in  the  sector  of 
international  higher  education:  Erasmus  Mundus II  (as  of  2009,  with  the  rest  of  the  world), 
Edulink (as of 2008, with African, Caribbean and Pacific Group of States), Tempus IV (with 
neighbouring  countries),  Alfa III  (with  Latin  America) and  programmes  of  cooperation  with 
certain industrialised countries. 

The current Erasmus+ programme has the following architecture: 

Key  Action  1:  Learning  mobility  of  individuals  (opportunities  for  students,  trainees, 
apprentices, young people and volunteers, as well as for professors, teachers, trainers, youth 
workers,  staff  of  educational  institutions  and  civil  society  organisations  to  undertake  a 
learning and/or professional experience in another country) 

Key  Action  2:  Cooperation  for  innovation  and  the  exchange  of  good  practices 
(transnational  or  international  projects  promoting  cooperation,  innovation,  exchange  of 
experience and know-how between different types of organisations and institutions involved 
in education, training and youth or in other relevant fields) 

Key  Action  3:  Support  for  policy  reform  (actions  supporting  national  authorities  and 
stakeholders in defining and implementing new and better coordinated policies in the field of 
education, training and youth). 

Jean  Monnet  activities  (actions  aimed  at  improving  the  quality  of  teaching  on  European 
integration  studies,  as  well  as  projects  and  operating  grants  aimed  at  promoting  discussion, 
reflection on EU issues and enhancing knowledge about the EU and its functioning) 

Sport (cooperation projects, events, studies  and other  initiatives  aimed at implementing EU 
strategies and priorities in the field of sport) 

Erasmus+ relies on management modes inherited from predecessor programmes. 

The  European  Commission  bears  the  overall  responsibility  for  the  supervision  and 
coordination of the agencies in charge of implementing the Programme at national level. The 
European  Commission  manages  the  budget  and  sets  priorities,  targets  and  criteria  for  the 
Programme.  Furthermore,  it  guides  and  monitors  the  overall  implementation,  and  evaluates 
the  Programme  at  European  level  after  having  received  the  National  Reports  from 
participating countries, as described under "method". It also manages directly few actions of 
the programme23.  

At  European  level,  the  European  Commission's  Education,  Audiovisual  and  Culture 
Executive Agency (EACEA)24 is responsible for the implementation of centralised actions of 
the programmes25 (direct management) which account for a small share of the total budget. 

The largest share of the budget is implemented through indirect management. The European 
Commission  entrusts  implementation  and  promotion  tasks  to  National  Agencies  (NAs)26 
established  in  each  of  the  33  countries  participating  to  the  Erasmus+  programme  (named 
hereafter "programme countries"27) which implement those actions of the programme with the 

23  These  actions  consist  mainly  of  administrative  expenditure  (studies,  external  communication  and  dissemination,  IT 
systems, etc.), policy coordination and support actions, politically sensitive and new actions, pilot projects and preparatory 
actions. 

24 https://eacea.ec.europa.eu/homepage_en  
25  Concerning  Erasmus+  these  actions  are:  Joint  Master  Degrees,  large-scale  European  Voluntary  Service  under  KA1, 
Knowledge  Alliances  and  Sector  Skills  Alliances  under  KA2,  most  of  the  KA3  actions,  Jean  Monnet  activities  and  sport 
actions. 
26 Currently, 57 NAs appointed and supervised by the National Authorities (NAUs). Since 2014, their performance is also 
controlled by Independent Audit Bodies identified in each country. 
27 EU Member States, EFTA/EEA countries, the Former Yugoslav Republic of Macedonia and Turkey. 

8 

 
 
                                                            
highest volume28 so as to bring the programme as close as possible to its beneficiaries and to 
adapt to the diversity of national education, training and youth systems. 

No  direct  support  is  given  to  individual  beneficiaries29;  all  support  is  channelled  through 
participating organisations, which distribute it to individual learners or practitioners. 

2.1.  Main changes in the programmes over the period under evaluation 

In  line  with  the  subsidiarity  principle  and  Articles  165  and  166  of  the  Treaty  on  the 
Functioning  of  the  European  Union,  the  EU  intervention  aims  at  supporting  and 
supplementing the actions of Member States, while fully respecting their responsibility for the 
content  of  education,  training,  youth  and  sport  policies.  Meanwhile,  more  than  its 
predecessors,  Erasmus+  has  been  designed  with  a  view  to  better  aligning  spending  actions 
with the priorities set at EU level in each policy area. 

The integrated structure of the programme is the most noticeable change in scope brought by 
Erasmus+. Other important developments between the two programming periods have taken 
the form of an increased budget allocation (+40%); a simplified structure in 3 Key Actions; 
streamlined  ways  of  implementation  and  systematic  monitoring,  as  evaluated  under 
"effectiveness", "coherence" and "efficiency". 

a)  Erasmus+  does  not  only  draw  on  the  legacy  of  its  predecessors  but  also  brings  them 
altogether into a single integrated programme, including its international dimension funded 
by  the  EU  external  action  instruments  (2%  of  all  Erasmus+  student  mobility  activities  have 
involved  partner  countries)30.  The  suppression  of  (sub-)  programmes  per  field  (sector) 
contributes to the "effectiveness", "efficiency" and "internal coherence" of the programme, as 
evaluated below. 

b)  Erasmus+  benefits  from  increased  resources  with  over  EUR  16.4  billion  spread  over 
seven  years  as  opposed  to  a  total  budget  of  slightly  more  than  a  total  of  EUR  9  billion 
allocated to its predecessors over 2007-2013. 

Examples of changes with Erasmus+31 

• Opportunities for over 4 million people to study, train, volunteer or gain professional experience 
abroad (1.8 million learners and practitioners over 2014-2016) 

•  A  stronger  focus  on  improving  young  people’s  job  prospects  to  tackle  youth  unemployment 
(more and shorter traineeships since 2014) 

•  A  more  inclusive  programme  supporting  people  with  fewer  opportunities  (11.5%  of  learner 
participants) 

•  Opportunities  for  participants  to  study  worldwide  (12%  of  all  higher  education  participating 
organisations) 

• Language learning support for mobility participants (14% of learners in long term mobility)  

• New funding for actions in the field of sport (1,8 % of the budget – heading 1) 

28  NAs  manage  KA1  mobility  (except  Erasmus  Mundus  Joint  Master  Degrees),  KA2  strategic  partnerships,  structured 
dialogue between young people and decision-makers under KA3 
29 Students, trainees, apprentices, pupils, adult learners, young people, volunteers, professors, teachers, trainers, youth 
workers, professionals of organisations active in the fields of education, training and youth 
30  Development  Cooperation  Instrument  (DCI),  European  Neighbourhood  Instrument  (ENI),  Partnership  Instrument  for 
cooperation with 3rd countries (PI), Instrument for Pre-accession Assistance (IPA), European Development Fund (EDF). The 
Youth  in  Action  programme  had  already  integrated  its  international  component  with  the  European  Voluntary  Service  and 
Youth in the World opened to partner countries, with an emphasis on neighbouring countries. 
31 See Annex 5b for more detailed factsheet 

9 

 
 
 
                                                            
c)  For  simplification  purposes,  following  the  conclusions  of  the  Impact  Assessment  for 
Erasmus+32,  the  current  programme  is  based  on  three cross-cutting  Key  Actions33  which 
apply  to  all  education  and  training  sectors  and  to  the  youth  field:  Key  Action  1  (KA1) 
supports  learning  mobility  of  individuals;  Key  Action  2  (KA2)  fosters  cooperation  for 
innovation  and  the  exchange  of  good  practices,  including  collaborative  platforms,  and  Key 
Action 3 (KA3) supports policy reforms. This integrated approach does not apply to the two 
stand-alone strands for sport and for European integration studies ("Jean Monnet"). The fact 
that  the  nature  of  the  funded  activities  has  not  fundamentally  changed  with  Erasmus+,  as 
reflected below, makes it possible, however, to draw conclusions about the likely impact of 
the  current  programme  based  on  what  can  be  observed  ex  post  from  the  impact  of  its 
predecessors. 

Main type of actions per Erasmus+ Key Actions 

Learning mobility of 
individuals (KA 1)  

innovation  and 
Cooperation  for 
exchange of good practices (KA2)  

Support for policy reform 
(KA3) 

-  Mobility of learners 
and practitioners 
-  Erasmus Mundus 

Joint Master Degrees 

- 

Student Loan 
Guarantee Facility  

Strategic Partnerships 
IT support platforms 

- 
- 
-  Knowledge Alliances (HE) 
- 
-  Capacity Building (HE and 

Sector Skills Alliances (VET) 

Youth) 

-  Open method of Coordination 

(OMC) 

-  Prospective initiatives 
-  EU transparency and 
recognition tools 

-  Dissemination & exploitation  
- 

Policy dialogue with 
stakeholders, third countries 
and international organisations 

In bold: most significant novelties of Erasmus+ 

d) The integrated nature of the programme means that cross-field approaches or tools can be 
used for the purpose of grant application, monitoring, audit or dissemination.  

This  has  materialised  through  on-line  application,  single  audit  system,  generalisation  of 
simplified  grants  (lump  sums,  unit  costs),  performance-based  monitoring  system,  a  single 
Project results platform, etc. In designing Erasmus+, specific attention has been paid to make 
it  more  result-oriented  than  its  predecessors  and  more  agile  to  adapt  to  policy  evolutions 
notably in the light of the socio-economic challenges mentioned above. 

2.2. 

Intervention logic 

The  EU  mandate  in  the  education,  training,  youth  and  sport  areas  sets  the  scope  for  the 
intervention  logic34.  An  overarching  intervention  logic  was  designed  to  cater  for  all  the 
programmes  in  scope  of  the  evaluation,  irrespective  of  the  programming  period.  This 
approach  was  followed  in  order  to  assess  and  compare  more  swiftly  results  across 
programmes  in  order  to  make  a  more  comprehensive  assessment  of  the  impacts  achieved 
during  the  baseline  period.  As  a  matter  of  fact,  although  each  evaluated  programme  had 
different architecture, most of the supported actions (mobility, cooperation and policy reform 
support) were transversally implemented – although with a different level of intensity - under 

32 Erasmus+ Impact Assessment, Commission Staff Working Paper, SEC(2011) 1402 of 23.11.2011 
33 For instance, the number of activities has been reduced from 75 to 11 in the sole case of LLP. 
34 See Annex 5a for a more detailed visualisation of the intervention logic 

10 

 
 
 
 
 
                                                            
all programmes. For that purpose, the programme's specific objectives and actions have been 
grouped according to the level at which the results are expected35. 

Furthermore,  considering  the  overwhelming  dominance  of  the  external  factors  of  Member 
States' policy making and spending in these areas, it is not easy to attribute and quantify the 
specific effects of the EU intervention.  

Spill-over  between  intervention  levels  is  expected,  as  shown  in  the  graphic  below, 
explaining why most of the specific objectives are expected to deliver results at more than one 
level.  For  instance,  the  mobility  of  learners  and  practitioners  (KA1)  can  –  in  addition  to 
individual-level results – improve the performance of the organisations and have an impact on 
systems,  especially  in  terms  of  outcome  recognition.  Also,  the  performance  of  individual 
organisations  (KA2)  is  expected  to  benefit  from  European  cooperation  in  the  fields  of  the 
programme,  including  through  its  modernising  effects  on  national  systems  and  reforms 
prompted by the open method of coordination and KA3. 

  Systems, to improve policies 

in programme and partner countries; 

  Organisations, to promote cooperation 
and positive changes in work methods; 

  Individuals, to increase 

participants’ competences 
as well as change their attitudes, 
practices and perceptions; 

Source: Roadmap for the evaluation, January 2016 

35 See Annexes 5a and 5c showing respectively at which level actions and objectives aim to produce results 

11 

 
 
 
 
 
 
                                                            
3. 

IMPLEMENTATION OF ERASMUS+ AND PREDECESSOR PROGRAMMES UNTIL 2016 

In  the  period  2007–2016,  the  programmes  have  funded  mobility  for  more  than  4.3 million 
learners and more than 880,000 practitioners. Many more have benefited from short-term 
blended  mobility  and/or  other  forms  of  transnational  exchanges  as  part  of  cooperation 
projects. In the current programming period (2014–2016) alone, at the time of the mid-term 
evaluation  (i.e.  without  taking  into  account  fully  2016  data),  the  programme  had  already 
benefited over 1.4 million learners and 400,000 practitioners. Approximately 67 000 projects 
have  been  contracted  in  the  same  period,  which  represent  a  reduction  in  yearly  number  by 
around 50% compared to 2007-2013. Erasmus+ has indeed been designed  to fund fewer, but 
more multifaceted projects with larger average numbers of participants. Comprehensive 
information is available from the Erasmus+ programme – Annual Report (taking into account 
fully 2016 data) 36. 

The  Erasmus+  programme  has  been  fully  implemented  as  from  2015  without  major 
difficulties37.  The  international  higher  education  strand  and  the  Student  Loan  Guarantee 
Facility38 were launched only in the second year of the programme, when arrangements were 
ultimately  made  respectively  with  the  External  Relations  instruments  and  the  European 
Investment  Fund.  The  implications  of  the  newly  created  European  Solidarity  Corps  on 
Erasmus+, more specifically on the implementation of the European Voluntary Service under 
the  Youth  Chapter,  only  materialised  in  2017;  it  is,  for  that  reason,  out  of  the  scope  of  this 
evaluation. 

Erasmus+  was  allocated  EUR  16.4  billion  for  the  2014-2020  period39.  Despite  the  overall 
significant budget increase compared to the previous programming period (+40%), Erasmus+ 
has  only  experienced  a  limited  budget  increase  in  2014-2016  according  to  the  programme's 
budgetary profile which concentrates the funding increase over 2017-202040, as shown below. 
It is therefore too early at mid-term to evaluate the full impact of the additional resources 
allocated to the current programme. 

36 http://ec.europa.eu/programmes/erasmus-plus/about/statistics_en  
37 The projects managed by the Greek National Agency for youth (0.22% of the Erasmus+ budget in 2015) were suspended 
from December 2014 to April 2016 due to an insufficient management assurance. Certain OECD-related activities could 
partially not be funded (€ 1 million) until the general framework for the cooperation between the Commission and the OECD 
was finalised on time. See DG EAC's AAR 2015. 
38 The Student or Master Loan Guarantee Facility (SLGF) is a new scheme under Erasmus+ (max. EUR 520 million 
budgeted), aimed at fostering HE degree-mobility by easing access to student loans, irrespective of socio-economic 
background. The guarantee managed by the European Investment Fund (EIF) enables students enrolled in a full Master’s 
programme abroad to apply for a comparatively favourable Erasmus+ backed loan provided by participating financial 
intermediaries. ICF, stand-alone report on the Evaluation of the SLGF ("ICF/SLGF" hereinafter), ICF, vol. 3 
39 The Programme has an overall indicative financial envelope of 14.774 billion EUR under Heading 1 of the EU Budget and 
of 1.680 billion EUR under Heading 4 and the EDF for the seven years (2014-2020), EU-28 appropriations. 
40 Consequently, the final budget available to cover Erasmus+ actions in 2015 - EUR 2.115 billion – represents only a slight 
increase compared to 2014 (+2.1%). 

12 

 
 
 
                                                            
Source: Erasmus+ activity report 2015 

Distribution of Erasmus+ budget across sectors (2014-2016) 

Source: ICF calculations based on EplusLink 

The most substantial part of the Erasmus+ budget is allocated to education and training and, 
within  this  field,  to  higher  education,  as  shown  in  the  graph  above41.  In  terms  of  actions, 
mobility has the largest share of the budget, as displayed below. 

Key data, 2014 - 2016 

Projects received 

Contracted projects 

Amounts  allocated  (in 
million EUR) * 
Participants 

KA1 

108,904 

49,073 

3,798.9 

KA2 

34,298 

6,936 

1,469.4 

KA3 

4,456 

2,127 

205.9 

Jean Monnet 

2,425 

762 

126.2 

Sport 

1,325 

238 

67.3 

2,049,140 

1,231,267 

184,403 

944,245 

- 

Source: Erasmus+ Programme Annual Reports 2014-2016 (taking into account fully 2016 data) 
*Amounts not only for project grants but including also operating grants 

41 "Other" is compound of operating grants to national agencies and administrative expenditure 

13 

 
 
 
 
 
 
 
 
                                                            
4.  METHOD OF THE EVALUATION 

Considering  the  high  quality  of  data  collected  through  several  complementary  techniques 
from  various  sources,  the  sophisticated  analysis  and  triangulation  of  evidence,  the  positive 
opinion of the Regulatory Scrutiny Board and the improvement stemming from its comments, 
as well as the significant feedback received (with more than a million responses processed in 
total), this evaluation can be considered very reliable and valid. 

4.1.  Evaluation framework 

To undertake the combined mid-term evaluation of Erasmus+ and the final evaluation of its 
predecessor programmes, as outlined above in section 1, the Commission has drawn upon a 
set  of  evaluation  questions  relating  to  five  main  criteria:  effectiveness,  relevance, 
coherence, efficiency and EU added value, as per the evaluation Roadmap42. 

To ensure comparable approaches, the evaluation framework developed by the external 
evaluator43  classifies  actions  across  the  three  levels  of  intervention  of  the  programmes  and 
uses  that  as  the  basis  of  comparison  over  10  years,  across  all  sectors  and  various  target 
groups, namely: 

  at System level 
  at Organisation level 
  at  Individual  level,  divided  into  actions  for  learners  and  practitioners  across  all 
different  areas (education and training,  youth  and sport) and  – within education and 
training – different sectors (school education (SE), vocational education and training 
(VET),  higher  education  (HE)  and  adult  education  (AE)).  For  the  purpose  of  the 
evaluation, the overarching terms of Learners and Practitioners are used44. 

In addition, the analysis of effectiveness has been based on the intervention logic represented 
graphically in annex 5a accompanied by a detailed elaboration of expected outcomes (results 
directly  for  individual  beneficiaries;  impact  indirectly  for  a  wider  population,  participating 
organisations or education, training, youth and sport systems)45. All specific objectives of the 
programme (annex 5c) are reflected in the outcomes or impacts of the intervention logic. 

The purpose of the evaluation is to evaluate the outcome of the programme on target groups, 
not to audit each type of activities. Therefore this intervention logic is a powerful simplified 
model  to  develop  questionnaires  based  on  the  most  relevant  levels  of  outcomes.  It  is  worth 
noting that these three levels of intervention are not equivalent to the three Key Actions, as 
explained above46. A given Key Action can indeed have an effect on more than one level of 
intervention (see 2.2). Key Actions have been set within Erasmus+ but these three levels of 
interventions apply to both programming periods47. 

42 http://ec.europa.eu/smart-regulation/roadmaps/docs/2015_eac_014_evaluation_erasmus_en.pdf 
43 ICF, Annex 14 (not published), where a matrix maps and compares the types of actions in the predecessor programmes and 
the corresponding actions in the Erasmus+ programme 
44 Learners refer to all individuals involved in formal, non-formal and informal education as pupils, students, apprentices, 
volunteers, young  people, etc.  Practitioners  refer  to  those involved  in  the  same  respect as  teachers (including prospective 
teachers), trainers, youth workers, educators, coaches, organisation leaders or staff, etc. 
45  See  annex  5a.  This  framework  of  outcomes  has  resulted  from  a  combination  of  review  of  programme  documentation, 
literature on results and impacts of similar types of interventions and scoping interviews with policy officers in charge of the 
programme. ICF, 3.3 
46 For instance, individuals are also targeted within KA2 activities in the case of school pupil or adult mobility. 
47  The  predecessor  programmes  in  international  higher  education  had  a  variety  of  intervention  logics  of  which  only  some 
elements  are  reflected  in  the  evaluation  intervention  logic.  Doing  an  evaluation  programme  by  programme  would  have 
fragmented the data collection and would not have allowed to use some of the more advanced analytical techniques. 

14 

 
 
                                                            
This  unprecedented  combination  of  the  ex  post  evaluation  of  predecessor  programmes 
and the mid-term evaluation of the current programme is a legal requirement. It implies to 
evaluate differently the two periods, as laid down in article 21(2) of the Erasmus+ Regulation: 
[The mid-term evaluation report] "shall also take into account the results of an evaluation of 
the  long-term  impact  of  the  predecessor  programmes  (Lifelong  Learning,  Youth  in  Action, 
Erasmus  Mundus  and  other  international  higher  education  programmes)."  Whereas 
Erasmus+  is  evaluated  against  all  aforementioned  criteria,  its  predecessors  are  mainly 
evaluated in relation to effectiveness and relevance48. However, since  the predecessors are 
systematically  taken  as  a  baseline  for  comparison  with  Erasmus+,  they  have  been 
evaluated also for comparison purposes in terms of coherence, efficiency and European added 
value. 

Systemic  impact  as  well  as  long  term  effects  of  projects  should  mainly  be  attributed  to 
previous programmes, whilst short-term effects on individuals have rather been evaluated in 
the context of the current programme. The results at the level of individuals are unlikely to be 
different  between  the  two  programming  periods  as  the  types  of  actions  are  highly  similar. 
What has mainly changed in that respect is the scale and reach of the programme as of 2014. 
The  programme  is  designed  so  that  the  systemic  impact  of  projects  (Key  Actions  2  and  3) 
should  only  be  evaluated  "by  the  end  of  the  programme"  as  per  article  21(1)b  of  the  legal 
basis. At mid-term, systemic and long term effects can also be attributed by extrapolation 
to Erasmus+ due to the continued nature of basic activities (mobility; cooperation). 

4.2.  Data collection and analysis 

Evaluation  findings  have  been  based  on  a  wide  range  of  data  sources  systematically 
triangulated  i.e.  cross-checked  against  each  other:  programme  documentation,  programme 
data  and  views  of  the  managing  bodies,  views  of  direct  beneficiaries  as  well  as  non-
beneficiaries,  EU  and  national  stakeholders,  policy  makers,  programme  countries,  bodies  in 
charge of implementation of other comparable programmes and the general public49. 

The  evaluation  combined  a  number  of  techniques  for  data  collection  and  analysis.  Most 
results are reached by a mix of evidence combining a quasi-counterfactual assessment based 
on beneficiaries’ surveys  (i.e. comparisons between  beneficiaries and a control group),  self-
reported feedback of beneficiaries (i.e. monitoring surveys), as well as qualitative interviews 
and case studies. While monitoring surveys could only give findings as from 2014, evaluation 
surveys  covered  2007-2016  and  enabled  the  evaluators  to  compare  the  two  programme 
generations. Monitoring data have been fully used and supplemented with surveys carried out 
by the external evaluator, providing more objective findings (with questions not based on self-
perception),  more  comprehensive  data  (covering  as  well  the  2007-2013  period  and  KA2 
participants)  and  more  comparable  responses  (as  questions  were  harmonised  across  sectors 
and periods for comparison purpose). 

Collection tools presented below covered all programme countries, sectors and main types of 
actions  (except  where  mentioned  otherwise).  The  quality  of  the  one  million  responses 
processed  was  in  most  cases  exceptionally  good.  All  quantitative  data  was  statistically 

48 The Terms of reference of this evaluation specify that "The questions are in many cases phrased in a general way, however 
the intention is that they cover the entire Erasmus+ programme (where applicable also the predecessor programme)." For 
instance:  "What  are  the  long-term  impacts  of  the  predecessor  programmes?  We  are  interested  in  the  impact  of  all 
actions/elements of the predecessor's programmes, in particular those actions/elements that are continued in new Erasmus+ 
programme. We are also interested in impact of actions/elements that have discontinued  to the extent that it might help to 
design the future programme."or "What conclusions can be drawn on the likely impact of Erasmus+ programme given the 
fact that significant parts of their actions are continuation of predecessor's programmes?" 
49 Primary data were mainly collected from November 2016 to May 2017. 

15 

 
 
                                                            
processed and presented using visualisation tools, while all qualitative data was systematically 
coded. 

As  required  by  the  Erasmus+  Regulation,  programme  countries  have  submitted  their  own 
evaluation reports on the implementation and impact of decentralised actions of Erasmus+ in 
their respective territories. Their findings are summarised in the  National reports synthesis 
(NRS) annexed to the external evaluator's report50. 

In  line  with  Better  Regulation  rules,  an  Open  Public  Consultation  (OPC)  gathered  the 
opinions of the general public and all interested groups on all evaluation criteria.51 Different 
questions  were  asked  depending  on  the  level  and  scope  of  knowledge  of  the  programmes 
declared  by  each  respondent.  Consultation  findings  are  presented  separately  in  the  synopsis 
drawn by the external evaluator52. More widely, responses from all sources are presented in 
Annex 2. 

Box  1  provides  an  overview  of  the  data  sources  from  which  the  evidence  was  drawn.  A 
detailed description of the individual methods is provided in Annex 3.  

  Literature review (131 sources about 2007-2016) 
  Key informant interviews (190 interviews about 2007-2016) 
  National reports (34 countries mainly about 2014-2016) 
  Open Public Consultation (covering 2007-2016 with 4,786 responses of which 1,800 responses 

were fully exploitable; 24 position papers submitted) 
  Social Media analysis (725,678 posts over 12 months) 
  Benchmarking (18 comparable national/transnational programmes) 
  Programme database analysis (all beneficiaries 2007-2016) including network analysis 
  Monitoring surveys (over 955,000 respondents - KA1 since 2014) 
  Online Linguistic Support (523,238 participants since 2014) 
  Beneficiary surveys and control groups (over 47,000 respondents over 2007-2016) 
  Survey of socio-economic organisations (947 responses over 2007-2016) 
  Agencies survey (130 responses from National Agencies or the Executive Agency EACEA) 
  Experts survey (1,122 responses from project assessors over 2007-2016) 
  Case studies (38 organisations or policy cases; 233 respondents over 2007-2016) 
  Review of selected projects’ outputs (386 project reports over 2007-2016) 
  Expert panel assessment of projects’ outputs (100 projects) 
 
  Student Loan Guarantee Facility (SLGF) (219 survey respondents; 33 interviews) 

Jean Monnet activities (2,350 survey respondents; 5 interviews) 

50 National reports synthesis (NRS hereinafter), ICF, vol. 6 
51 https://ec.europa.eu/programmes/erasmus-plus/resources/consultations/erasmus-plus-mid-term-evaluation-2017_en 
52 Synopsis of the open Public Consultation (OPC hereinafter), ICF, vol. 5 

16 

 
 
                                                            
4.3.  Method strengths and limitations 

Overall,  the  quality  of  available  data  for  the  predecessor  programmes  was  good,  while 
for  Erasmus+  it  was  very  good.  This  reflects  improvements  made  in  terms  of  monitoring 
since  2014.  The  evaluation  used  the  wealth  of  the  programme  data  available  from  the 
Erasmus+ management or monitoring tools, including some data on results (e.g. satisfaction-
rate)  since  2014.  Although  the  period  within  the  scope  of  the  evaluation  was  2007-2016, 
programme data was not fully available for 200753.  

The main issues that affected the analysis of participation patterns include the following: 

i) the absence of a harmonised definition, across sectors, of people with fewer opportunities 
and  from  disadvantaged  backgrounds  (programme  data  were  completed  with  other 
findings)54;  

ii) the  lack  of  a  complete  picture  on  the  type  of  participating  organisations  (some  40%  of 
organisations  were  classified  as  "other"  in  the  programme  dataset  limiting  the  analysis  of 
participation per type of organisation).  

It  should  also  be  noted  that  the  analysis  over  time  of  networking  among  participating 
organisations  or  repeated  participation  of  individuals  in  the  programme  could  not  be  fully 
pursued,  as  prior  to  2014  the  same  organisations  could  be  recorded  under  multiple  names, 
which made such an analysis difficult to execute. These limitations have however remained 
marginal or have been overcome through the use of other sources. 

The monitoring surveys of beneficiaries carried out by the Commission concern all learners 
or practitioners taking part in mobility under KA1 (since 2014 only)55. All are requested to fill 
in an online survey on completion of their mobility. Therefore the  reliability of the data is 
strong. Given the sample sizes (first two years of the programme) the data is considered to be 
strongly reliable even though it was too early to include 2016 data. This being said the link 
between  the  monitoring  questions  and  the  indicators  they  are  supposed  to  measure  can  be 
weak  in  some  cases.  Above  all,  most  questions  are  based  on  the  self-perception  of 
beneficiaries in contrast to more objective measurements such as the assessment of language 
proficiency  after  mobility  (OLS56).  This  is  why  monitoring  data  were  supplemented  with 
primary data collection, such as surveys carried out by the external evaluator, providing more 
objective and comparable responses. 

Series  of  surveys  of  beneficiary  learners  and  staff  and  related  control  groups  covered  all 
programme target groups and predecessor programmes as well (2008-2016 in total), both for 
mobility  and  cooperation,  over  2007-2016.  Overall,  findings  could  be  generalised  to  the 
whole programme with a sufficient degree of confidence because of the sample sizes of all 
surveys  and  the  distribution  of  various  background  variables  within  the  survey  samples. 
Given  the  limited  data available  on the  background  of  beneficiaries, it nevertheless remains 
impossible  to  quantify  the  degree  of  comparability  of  the  samples  to  the  programme 
population57. 

53 Monitoring data for 2016, although not yet confirmed, were also analysed for outputs and participation patterns. 
54 See sections on effectiveness and relevance below for encouraging results in terms of social inclusiveness 
55 See annex 5e reporting on all output and result indicators from the legal basis against targets set in DG EAC's Strategic 
Plan. As these legal basis indicators are based primarily on direct beneficiaries’ perceptions of the programme contribution, 
they are only meaningful when asked shortly after the mobility experience. It would not have been possible to reconstruct 
them even if the evaluation surveys had asked comparable questions to the beneficiaries of 2007-2013. 
56 The Online Linguistic Support (OLS) was introduced with Erasmus+. Before and at the end of their mobility, participants 
(HE, VET or EVS) take the language assessment to measure their level and progress in the language. Language courses are 
also provided to them on-line. 
57 There is no database that would provide data on the overall target population. Moreover for several surveys (pupils, young 
people,  sport  staff)  or  certain  actions  (KA2,  predecessor  programmes)  there  is  often  no  contact  database  of  direct 

17 

 
 
                                                            
 
A  partially  counterfactual  approach  ("quasi-experimental")  was  also  used  to  quantify  the 
contribution  of  the  programme  to  the  results  measured.  This  was  done  by  assessing  the 
difference in results between Erasmus+ beneficiaries and their "peers" who did not take part. 
The  challenge  is  to  attribute  the  difference  in  outcomes  between  both  groups  to  the 
programme  isolating  any  other  underlying  factor  (attribution).  Control  groups  of  non-
beneficiaries  were  therefore  set  up.  In  an  ideal  world,  these  should  be  selected  on  random 
basis.  However,  a  random  allocation  is  neither  feasible  in  practical  terms  nor  desirable  for 
ethical reasons. By default, a ‘matched sample’ was sought where individuals are similar in 
some  background  characteristics  such  as  gender,  age,  etc.  There  were  no  significant 
differences between the control group and the treatment group for most variables, and where 
there  were,  they  concerned  variables  that  were  not  likely  to  strongly  influence  the  findings. 
Most  control  groups  reached  a  significant  size  allowing  comparison  with  beneficiaries, 
except in sport and Adult Education sectors where the control groups were too small. As a 
consequence,  the  control  group  of  VET  staff  was  also  used  for  Adult  Education  and  the 
control  group  of  Youth  staff  was  also  used  for sport,  as  these  were  found  to  be  sufficiently 
comparable  against  a  range  of  background  characteristics.  However  this  counterfactual 
approach  cannot  exclude  the  possibility  that  the  difference  in  a  given  variable  between 
beneficiaries and non-beneficiaries could be partially due to the selection into the programme 
rather than the effect of the programme. 

In addition to the above ‘post’ surveys (i.e. beneficiaries were surveyed after they took part in 
the  programme),  the  external  evaluator  carried  out  pre-surveys  in  two  sectors  where  short 
term mobility exchanges apply (i.e. school pupils and Youth exchanges). This was done as far 
as  possible  within  the  time  frame  of  the  evaluation  assignment  for  the  purpose  of  pre/post 
comparison. In  spite of repeated recruitment efforts, the sample sizes suffered from  a high 
attrition from the pre to the post survey, especially for the Youth sector where comparison 
of  profiles  prior  to  Erasmus+  mobility  could  not  be  done.  Pre-post  results  for  the  Youth 
sector  have  therefore  not  been  interpreted  considering  the  high  likelihood  of  sample  bias. 
This  allowed  nevertheless  for  the  school  sector  to  draw  conclusions  on  selection  into  the 
programme and to observe changes during the time of the mobility experience. 

To  get  a  better  understanding  of  their  quality  and  dissemination  potential,  a  review  of 
selected  project  outputs,  mainly  collected  from  National Agencies,  was  carried  out.  Given 
that  only  relatively  few  Erasmus+  funded  projects  were  completed  at  the  time  of  this 
evaluation,  the  sample  remained  inevitably  somewhat  biased  towards  predecessor 
programmes. 

As  for  the  Open  Public  Consultation  (OPC),  a  large  number  of  partial  responses  only 
contained very basic background information on respondents and had therefore to be marked 
as unsuitable for analysis. In any case, in contrast to surveys, public consultation findings can 
never be representative due to the selection bias inherent to any open recruitment. 

Lastly, some programme countries have  delivered their  national reports with  a four month 
delay. This, in turn, has delayed the adoption of the Commission report to take into account 
all programme countries' views. The period of time between the deadlines set by the legislator 
respectively for the delivery of the national reports and the Commission report was reduced 
from 9 months (LLP) to 6 months (Erasmus+), which has not allowed the external evaluator 
to make a full synthesis of all findings58. 

beneficiaries, or not standardised ones, hence there is no background data on the total population of beneficiaries to compare 
with respondents' profile. This is why respondents have been recruited through organisations. 
58 ICF has delivered separate reports, respectively volumes 1 (main report) and 6 (separate synthesis of national reports). 

18 

 
 
 
                                                                                                                                                                                          
4.4. 

Judgement on the validity and reliability of the findings 

The  external  evaluator  (ICF)  contracted  for  this  assignment59  has  carried  out  since  May 
2016  all  tasks  as  required  under  the  scrutiny  of  an  inter-service  group  (ISG)  60  and  the 
daily  steer of DG  Education,  Youth, Sport and Culture (EAC).  The only  significant change 
compared to the initial work plan was a two-month delay of the (sub)contractor in launching 
the beneficiary surveys61. The ISG was consulted at each stage of the evaluation process and 
reviewed  each  deliverable  produced  by  the  contractor  as  well  as  this  Staff  Working 
Document62. 

The  Regulatory  Scrutiny  Board,  responsible  for  the  independent  quality  control  of  this 
evaluation,  acknowledged  in  its  positive  opinion  with  comments  the  significant  efforts  of 
data and evidence collection and noted the good methodology used63. This document has been 
improved following its comments. 

Based on the above elements of the evaluation method and as further described in Annex 3, 
the reliability and validity of the evaluation can be regarded as strong64. 

Elements of the method 

Validity of overall judgements 

Definition of results and impacts 

Assessment of reliability and 
validity 

Strong 

Strong 

Validity of measurement tools to collect data on results and impacts   Strong 

Reliability of overall evaluation design 

Collection of qualitative data 

Interpretation of qualitative data 

Strong 

Strong 

Strong  

Generalisation of findings to the whole programme 

Medium to strong 

Counterfactual assessment 

Surveys  

Programme data 

Medium 

Medium 

Medium  

Source: ICF, 3.5; see section above for assessment grounds 

59 ICF Consulting Services Ltd under specific contract – EAC-2016-0219 implementing Framework contract EAC/22/2013 
60 Commission services (BUDG, DEVCO, EAC, EACEA, EEAS, EMPL, FPI, HOME, JRC, NEAR, SG), and EESC 
61 DG EAC wrote to ICF to mitigate that issue in March 2017. The delay had no consequence on the final report. 
62 ISG meetings on 9 September 2015 (Kick-off), 18 July 2016 (Inception report); 21 November 2016 (Interim report 1); 24 
March 2017 (Interim report 2); 10 July 2017 (Draft final report); 11 September 2017 (Final report). The ISG was thereafter 
consulted  on  the  drafting  of  this  Staff  Working  Document  (29  September  2017)  and  informed  of  the  Regulatory  Scrutiny 
Board's positive opinion (27 November 2017). 
63 Ares(2017)5629740 - 17/11/2017 
64 ICF, 3.7 

19 

 
 
 
 
                                                            
5.  FINDINGS 

5.1.  Effectiveness 

The assessment of the effectiveness65 of Erasmus+ is overall very positive, especially when 
compared  to  the  situation  before  the  integration  of  predecessor  programmes66.  It  has  been 
informed by 15 evaluation questions that looked into the extent to which: 

  outcomes (i.e. outputs, results and impacts) were delivered with particular attention to 

disadvantaged groups 

  spill-over, sustainable or unintended effects took place 
  certain  fields  or  actions  were  more  effective  than  others  (including  discontinued 

actions) 
results were disseminated and exploited. 

 

The  programme  reaches  or  exceeds  the  vast  majority  of  its  indicator  targets  set  in  the 
legal  basis67,  except  in  the  case  of  the  Student  Loan  Guarantee  Facility  (SLGF)  which  is 
missing the target by a high margin, as detailed below68. However, some output targets might 
have been set at a low level initially (sometimes below predecessor programme levels). The 
quality of the data on participation of disadvantaged groups although on target is unclear due 
to various definitions and practices (see below). Lastly, for Jean Monnet activities there is no 
exact  data  on  numbers  of  students  reached  and  this  indicator  is  likely  to  be  very 
overestimated.  These  limitations  concern  only  a  few  indicators  and  the  vast  majority  are 
without doubt on target.  

Incoming  mobility  of  students  from  higher  education  international  strand  is  also  below 
target  but  the  action  started  in  2015  only  and  the  overall  trend  over  ten  years  is  positive. 
Evaluated  programmes  have  been  important  for  the  EU's  global  outreach,  as  elaborated 
hereafter69. 

The Open Public Consultation confirms that the programme is achieving its objectives to 
a  (very)  large  extent.  OPC  respondents  highlighted  two  elements  as  particularly  helpful  in 
terms  of  effectiveness:  the  new  integrated  structure  of  the  programme70  and  the  overall 
increase  in  funding71.  One  sustainable  result  of  the  programme  and  its  predecessors  is  the 
cultural  shift  in  the  perception  of  mobility  and  its  positive  image,  at  least  in  the  higher 
education  sector  and  progressively  in  VET.  These  long-term  changes  are  yet  to  fully 
materialise in the other sectors. 

5.1.1.  Outcomes for learners and practitioners  

Compared to the last three years of predecessor programmes, Erasmus+ has supported during 
the first two years72 a much higher volume of mobility within and outside the EU in higher 
education  (the  sector  that  has  most  beneficiaries  overall),  but  also  a  higher  number  of 
practitioners  in  VET  and  schools73.  The  repeated  participation  of  individuals  is  kept 

65  Effectiveness  analysis  considers  to  which  extent  a  programme  has  achieved  its  objectives,  using  appropriate  points  of 
comparison and elaborating an opinion on the role of this programme in delivering the observed changes. 
66 ICF, 6; NRS 5; OPC 3.3 and 5.1.2 
67 Erasmus+ annual reports: http://ec.europa.eu/programmes/erasmus-plus/about/statistics_en  
68 ICF, 6.2.1; tab. 6.3 and annex 3 JMO, 1.2.2 
69 Evaluation of the EU Development Co-operation Support to Higher Education in Partner Countries (2007-2014), (2017), 
Particip,  LeA,  ECDPM,  Ecorys  for  the  Directorate-General  for  International  Cooperation  and  Development  (European 
Commission)  
70  One  EU  level  organisation  representing  over  30  HEIs;  one  sub-national  organisation  representing  a  region;  and  one 
national agency for HE cooperation.  
71 One EU-level organisation representing over 30 HEIs. 
72 See section 3 for increasing budget profile as of 2017 
73 OPC: 84% agree that Erasmus+ provides more opportunities for mobility than predecessor programmes (n = 552). 

20 

 
 
                                                            
reasonably low, indicating a continuous impact on renewed cohorts.  In contrast, as intended 
with the design of Erasmus+74 there has been a significant decrease in mobility for pupils - 
due to the fact that individual mobility of pupils was not embedded in the programme as such 
any more - and a slight decrease for VET learners75. 

5.1.1.1. Outcomes for learners 

Beyond contrasted evolutions in outputs across sectors, drawing on a variety of sources76, the 
evaluation found positive and sustainable results for all individual learners taking part in 
mobility77. Firstly, monitoring surveys outline that most learners themselves are satisfied with 
their  mobility  experience78.  The  rate  of  formal  recognition  of  participation  in  Erasmus+  is 
high but, more importantly for those concerned; the rate of recognition of learning outcomes 
is also rather high79. The vast majority agree that the programme helped them improve more 
than 6 competences out of a list of 9 (see fig.)80. To be noted, higher education students are 
comparatively least positive in their self-assessment81.  

Share of learners who (strongly) agree that they developed transversal competence 

74  Erasmus+  no  longer  provides  for  (KA1)  individual  pupil  mobility,  which  is  now  only  limited  to  mobility  activities 
embedded in (KA2) cooperation projects. 
75 However as mobility in VET is more dominated by traineeships in companies, it is likely that the programme now reaches 
more typical VET learners. The number of adults reached through mobility remained besides stable. 
76 Sources: monitoring surveys, evaluation surveys including control groups, case studies, literature review. For instance, the 
Erasmus  impact  study  (2014)  outlines  how  the  programme  has  improved  prospects  for  a  successful  career  by  giving  HE 
students skills that are sought after by employers: 
http://ec.europa.eu/dgs/education_culture/repository/education/library/study/2014/erasmus-impact_en.pdf 
77 For mobile learners but also for those taking part in KA2 blended learning mobility and its equivalent in the past 
78 93% of learners were very satisfied or rather satisfied with their mobility experience; ICF, 6.4.3 per sector 
79 ICF, tab. 5.11: Over 2014-2016, formal recognition of participation in Erasmus + ranges from 80% (VET with Europass) 
to 87% EVS and 88% youth exchanges (Youthpass). Academic recognition is at 83% for HE. 
80 More than 80% learners declare having improved six or  more competences from the proposed  set of nine competences, 
except in HE (71%); ICF, fig. 6.4 
81 This could be explained because their competences are already at higher level when entering the programme, if it was not 
for  the  fact  that  those  HE  students  in  traineeships  show  a  different  pattern.  ICF,  fig.  6.3  and  6.4).  Moreover,  HE  learners 
within Europe are less satisfied than others; e.g. with mentoring (68% (rather) satisfied against 81%). IFC, fig.6.7 to 6.9 

21 

 
 
 
                                                            
Source: ICF calculations based on Erasmus+ participant monitoring surveys. Multiple answer question. N = 730,254 

In line with the intervention logic82, all programmes under review are expected to contribute 
to changes in learners’ skills and competences as well as attitudes. In the longer term, they are 
expected to result in better completion rates and more positive employment outcomes. Both 
have  materialised  as  detailed  below.  As  far  as  the  general  indicators  of  Erasmus+  are 
concerned, trends are also positive and the Education and Training Monitor 2017 highlights a 
further  decrease  in  the  rate  of  early  school  leavers,  and  a  tertiary  attainment  target  that  is 
within reach83. 

Most  result  indicators  set  in  the  legal  basis  of  Erasmus+  are  on  or  above  target84.  The 
evaluation found, using a variety of sources, clear evidence of results for mobile learners in: 
foreign  language  skills  development,  especially  for  those  with  low  entry  levels;  feeling  of 
belonging  to  the  EU;  willingness  to  be  mobile  and  to  work  abroad;  shorter  transition  from 
education to work85; positive perception of the value of education and positive feeling towards 
the sending organisation which are key precursors for retention and attainment in  education 
and  training.  As  evidenced  for  those  benefiting  from  KA1  mobility,  the  programme  is 
associated with stronger rate of completion of studies86. 

However the result indicators set in the programme legal basis and related targets set in DG 
EAC's strategic plan have some limitations in that they are mostly based on the self-perceived 
contribution of KA1 reported by beneficiaries. Some of the targets were exceeded and others 
not reached (e.g. participants declaring that they have increased their language skills; young 
participants  declaring  being  better  prepared  to  participate  in  social  and  political  life). 
However, there are concerns about the plausibility of the target values in both cases87. 

The evaluation does not permit isolating how many of these results are the direct effect of the 
selection of the most motivated into the programme. For instance, Jean Monnet students do 
show better results than other higher education students when it comes to knowledge of the 
EU. This however can be expected given that the programme reaches mainly those who have 
shown an interest in the EU topics88. 

The most successful objectives of the Programme, which are also the most relevant, according 
to OPC respondents, have been a) developing the skills and competences of learners; and 
b)  promoting  the  European  dimension  of  education,  training  and  youth  activities89.  Other 
studies also confirm this in the relevant sectors90. Higher education who have participated in 
the  programme  are  not  only  more  likely  to  be  employed,  but  also  more  likely  to  secure 
management positions. On average, 64% of Erasmus students, compared to 55% of their non-
mobile  peers  hold  such  positions  within  5-10  years  from  graduation.  This  holds  even  more 
true for Erasmus students from Central and Eastern Europe, where around 70% of them end 
up  in  managerial  jobs91.  The  same  can  be  said  for  the  international  dimension  of  higher 

82 Section 2.2 above and Annex 4a (intervention logic) 
83 http://ec.europa.eu/education/policy/strategic-framework/et-monitor_en  
84 See annex 5e reporting on all output and result indicators from the legal basis. 
85 For mobile learners in KA1 only; this has not been demonstrated for other types of actions 
86 Measured as a proxy by the absence of repetition of a class within the typical duration of mobility 
87 ICF, 6.4.2 and tab. 6.10. As there was no comparable data from the predecessor programmes (no baseline data) for many 
indicators, “it seems that the targets were defined somewhat arbitrarily”. 

89  Respectively  86%  and  80%  of  OPC  respondents  (n  >  1400)  thought  that  these  two  objectives  were  achieved  to  a  ‘very 
large’ or ‘large’ extent. 
90 ICF, tab 6.13 and 6.14 for analysis per sector; there is less evidence of positive impact of mobility in the Adult Learning 
sector, but it could be simply not researched. 
91  Follow-up  to  the  2014  Erasmus  Impact  Study  focusing  on  regional  analysis  of  the  benefits  of  the  Erasmus  programme, 
2016 

22 

 
 
                                                            
 
education,  with  clear  evidence  of  strengthened  capacities  and  employability  of  individual 
students and scholars who have participated in mobility92. 

Albeit  mostly  based  on  the  self-reported  perceptions  of  participants,  most  National  Reports 
refer also to improvement of the level of key competences and skills of learners with a high 
level of similarity with the ones listed above. On top of technical/professional knowledge and 
skills93, the impact of Erasmus+ on employability is underlined by several countries mostly in 
the  case  of  HE  and  VET94,  transversal  skills95  or  soft  skills  are  also  frequently  identified. 
These  include  communication  skills96,  social  and  civic  skills97,  and  more  widely  personal 
development98.  

The case studies confirmed the strongly positive influence the programme plays on personal 
development  and  maturity  of  young  people.  Both  sources  frequently  outline  that  learners 
gain  in  confidence,  independence,  ability  to  cope  with  new  environments  and  open-
mindedness through their mobility. Participation in the programme also leads to development 
of learners’ social capital99. In contrast to very positive effects of all forms of mobility, there 
is  less  clear  evidence  of  impact  of  transnational  partnerships  on  learners  (excluding  their 
mobility component) except in the school sector100. For example, a survey conducted as part 
of  the  eTwinning101  evaluation,  indicated  that  55%  of  teachers  believe  that  the  platform  is 
helpful in motivating their pupils102. 

At  programme  level,  the  areas  where  learner  surveys  have  identified  strongest  positive 
difference between the results of participants and non-participants103 include: willingness 
to  move  abroad  permanently  (+31%  likeliness  compared  to  non-participants);  stronger 
feeling of being an EU citizen (+19%); shorter transition to employment (+13%)104; value 
attached to education (+8%) and also openness to  immigration  and minorities (+2.6%). The 
figure  below  gives  an  overview  of  all  areas  evaluated  according  to  the  degree  of  difference 
found105. 

92 Evaluation of the EU Development Co-operation Support to Higher Education in Partner Countries (2007-2014), (2017), 
Particip, LeA, ECDPM, Ecorys, already mentioned 
93 BEde, DK, LU, HU, MT, NL, AT, PT, NO 
94 BEnl, FR, MT, NL, FI, SE, IS, NO 
95 CY, HU, NL, NO 
96 LT, PL, UK, IS 
97 DE, LT, LU, MT, NL, SK, UK 
98 BEnl, DK, DE, IE, EL, ES, CY, LT, LU, MT, NL, PL, PT, FI, SE, UK, IS, TR, (e.g. for learners and youth:, self-esteem, 
self-awareness) 
99  55.7%  of  beneficiaries  were  in  frequent  contact  with  persons  they  met  during  participation  in  the  programme  (at  least 
monthly  basis)  and  another  30.8%  were  occasionally  in  contact  (at  least  once  a  year).  VET  beneficiaries  have  more 
occasional or rare contact with people they met through the programme. 
100  For  instance,  pupils  who  benefitted  from  partnerships  have  higher  degree  of  independence,  motivation  and  self-
confidence, better communication skills, increased enthusiasm for learning European languages. 
101 eTwinning offers to the school practitioner community a platform to develop and share on-line their transnational projects 
(involving more than 183,000 schools in September 2017): https://www.etwinning.net/en/pub/index.htm  
102  Education  for  Change  (2013)  Study  of  the  impact  of  e-Twinning  on  participating  pupils,  teachers  and  schools 
http://bookshop.europa.eu/en/study-of-the-impact-of-etwinning-on-participating-pupils-teachers-and-schools-pbNC3112371/ 
103 ICF, tab. 6.12 for estimations of treatment effect and statistical significance 
104 Measured for HE and VET beneficiaries only; not measured for school pupils as not yet relevant 
105 ICF, 6.4.4.4 - 7 and Annex 3 for detailed analysis per sector 

23 

 
 
 
 
                                                            
Comparison of results for learner beneficiaries and control group 

Source: ICF Beneficiary surveys: only areas where the difference is statistically significant i.e. unlikely due to a sampling interference 

The  share  of  beneficiaries  who  took  less  than  three  months  to  find  a  job  stood  at  68.5%, 
while the percentage of the overall control group was at 59.2%. This trend mirrors the positive 
expectation  of  most  beneficiaries  towards  Erasmus+106.  This  is  particularly  strong  for  the 
higher education107 and VET beneficiaries, in contrast with the youth sector108. Erasmus+ also 
leaves a positive effect on entrepreneurship109. Overall, this confirms the findings of several 
previous studies about the effects of learners’ mobility on employability110. 

The  pre/post  analysis  for  pupils  confirms  the  positive  contribution  of  Erasmus+  in  five 
similar  result  areas,  the  order  of  which  appears  nevertheless  specific  to  the  school  sector. 
There are statistically significant differences in results across pre- and post-survey of school 
pupils  in  EU  citizenship  (+7  score  points),  digital  competence  (+4),  racism  and  xenophobia 
(+3),  self-confidence  in  education  (+4),  civic  participation  and  volunteering  (negative 
difference). 

However, there is likely selection of already more performant pupils into the programme 
at the level of schools, since the pre-post survey also shows already a significant difference 
between participants and the control group at entry into the programme. Pupils participating 
in  short-term  mobility  speak  a  foreign  language  more  often  than  the  control  group 
(+13 percentage  points  or  p.p.)  and  feel  more  often  as  EU  citizens  already  before  going  on 
mobility  (+9  p.p.).  Cultural  awareness  (+7  points  in  index  score  or  p.i.s.),  positive  attitudes 
towards immigration and minorities (+6 p.i.s.), digital competence (+5 p.i.s.) and perceptions 

106 They are 92% in general to believe so according to the beneficiary survey. 
107 70% of Erasmus Mundus alumni found their first job within three months after graduation according to the tracer study in 
Evaluation  of  the  EU  Development  Co-operation  Support  to  Higher  Education  in  Partner  Countries  (2007-2014),  (2017), 
Particip, LeA, ECDPM, Ecorys, already mentioned 
108 Beneficiaries in the youth sector experience longer transition than the control group, which could be due to the specific 
profile of this target group. 
109 The interim evaluation of the EIT indicates a self-reported rate of adoption of entrepreneurial skills of 69% for Erasmus 
mobile students compared to 83% for EIT-KICs. The foundation rate for new start-ups is 7% with Erasmus+ (6% with EIT-
KICs). 
110 ICF, 6.4.5 for findings of the literature review; ICF, annex 11 (literature reviewed) 

24 

Large positive differenceMedium positive differenceStatistically significant yet small positive differenceMedium negative differenceShort transition to employment (0-3 months)Willingness to work abroad Openness to other cultures and minoritiesActive civic engagement in an organisation (in the past)Feeling of belonging to the EUPositive perception of the value of educationProblem solving behavioursCivic competence - perceived importance of community engagement activitiesCompletion of studies (in a typical duration)Sense of initiativePositive feelings towards their school/ education org. Civic competence - perceived importance of civic behaviours and activitiesDigital competence - use of various online resourcesLearning to learn Positive feelings towards the EU 
 
 
 
                                                            
of  the  value  of  their  education  (+4  p.i.s.)  were  also  higher.  This  is  confirmed  by  the  case 
studies where some respondents claim that participation is seen as a certain form of reward. 

Foreign language skills development is more important in those sectors where learners tend to 
have lower entry levels (i.e. in VET and youth more than in higher education). The analysis of 
OLS foreign language tests (pre/post mobility) proved that the weaker the proficiency entry 
level, the stronger the linguistic improvement due to mobility111. 

Comparison of levels of proficiency before and after mobility, by sector 

Legend: share of learners in each sector at a given level of proficiency; blue: before mobility (first language assessment), red: at the end of 
the mobility (final language assessment) 
Source: ICF calculations based on OLS data as of 31/08/2016. Higher education: n= 500,410; VET: n=19,441; EVS: n=3,387 

Beneficiaries  were  asked  in  case  studies  to  describe  the  most  important  contribution  of  the 
programme. The word-cloud below visualises the most cited testimonies. 

What do you see as the most important contribution of the programme for yourself?  

Source: ICF case studies –interviews with learners: 200 mentions or expressions were collected; the size of the font captures the frequency 
with which a given result was mentioned. 

111 ICF, fig. 6.5 

25 

 
 
 
 
 
 
                                                            
the  achievement  of  programme  objectives112, 
When  asked  about  reasons  hindering 
respondents  identified  the  following  barriers  during  the  public  consultation,  in  descending 
order  of  frequency:  linguistic  barriers;  financial  difficulties  and  complex  application 
process113. The factors that enable positive results at the level of individual learners are related 
to their motivation to participate as well as the quality of the learning experience including 
foreign  language  learning  support,  the  support  received  in  the  host  organisation  and  their 
integration  among  other  students  and  local  communities114.  Lastly,  levels  of  recognition  of 
learning  outcomes  vary  according  to  types  of  actions115,  but  signing  a  learning  agreement 
remains a strong predictive indicator for later recognition. Only a small share of learners does 
not sign one116. 

Although financial barriers are a major obstacle to mobility117, the Student Loan Guarantee 
Facility,  which  was  an  innovation  in  Erasmus+,  has  so  far  failed  to  attract  financial 
intermediaries in  sufficient numbers118– especially for the incoming  student segment119. First 
beneficiaries  are,  however,  satisfied.  The  Student  Loan  Guarantee  Facility  already  shows 
signs  of  social  fairness120,  but  its  visibility  at  this  early  stage  is  far  from  being  sufficient 
throughout the supply chain121. 

5.1.1.2. Outcomes for disadvantaged learners  

The current programme is rightly perceived as paying more attention to the participation of 
disadvantaged people122. To respond to the criticisms regarding the predecessor programmes 
– notably in the field of higher education – of being elite programmes, Erasmus+ has put in 
place actions to strengthen the participation of disadvantaged groups compared to predecessor 
programmes123.  In  the  period  under  review,  11.5%  of  participants  under  KA1  had  special 
needs or came from a disadvantaged background. This share rose to 30% for learners in the 
Youth field between 2014 and 2016124. The number of participants with special needs or fewer 
opportunities  has  more  than  doubled  since  the  predecessor  programmes,  i.e.  more  than 
proportionally compared to the overall budget increase. The indicator targets have even been 

112 This question was only addressed to OPC respondents who indicated that they have detailed or some knowledge of the 
Erasmus+ objectives and actions’. 
113 Other barriers mentioned in the OPC: lack of cooperation between actors involved at their level; differences in educational 
systems hindering their cooperation; lack of information about the Programme and poor quality of courses. 
114 Monitoring surveys show that the integration of learners in local communities is more difficult than in the host institution. 
HE mobile learners (within programme countries) feel least well integrated to both; ICF, fig. 6.9 
115  Recognition  of  learning  outcomes  is  not  compulsory.  Europass  is  a  tool  helping  to  document  and  to  make  learning 
outcomes  visible.  In  VET,  where  the  most  common  form  of  documentation  is  Europass,  recognition  using  ECVET  credit 
remain  small  by  nature  (credits  points  are  not  commonly  used  in  VET  systems)  but  is  progressing  (3  percentage  points 
between  2014  and  2015).  The  degree  of  recognition  is  highest  in  HE  thanks  to  ECTS,  particularly  within  programme 
countries and among students in traineeships. The degree of recognition within the youth sector is higher for those in youth 
exchanges, albeit of shorter duration, than for those in EVS. 
116 In VET, 12% of learners did not sign a learning agreement prior to departure on mobility. 
117 E.g. according to Portugal’s recent estimations the contribution of families to HE mobility can reach 50%; NRS, 3.4 
118 After two years, 6 intermediaries in 5 countries and 162 students supported in 2016, well below initial expectations 
119 ICF, stand-alone report on the SLGF, 4.4 for reasons and flaws in the design of the loan facility 
120 71% of the first loan beneficiaries (n=65) said in 2016 they would not have been able to study abroad without the loan. 
43% of SLGF beneficiaries (mostly from Spain) in 2015 or 2016 had parents not holding a university degree; ICF 
standalone report, 4.4.3 and 4.4.4 respectively 
121 93% of respondents met at student fairs (80% for the exhibitors) had no prior knowledge of the loan facility. 
122 Several case study interviewees; NRS, 3.4 for national examples of promoting the participation of vulnerable groups 
123 Top up grants  and grants to fund accompanying persons are now available for those who may need such assistance. A 
youth  inclusion  strategy  entails  financial  incentives  and  specific  project  format  to  foster  the  participation  of  young  people 
with fewer opportunities and from disadvantaged backgrounds. 
124  ICF,  annex  2,  section  2.2.3.  However  there  are  concerns  about  the  reliability  of  data  given  the  collection  method  and 
differences  in  the  breadth  of  definitions  used  across  sectors.  Survey  data  suggests  that  the  differences  between  sectors  are 
much smaller. ICF, 6.3 

26 

 
 
                                                            
exceeded125,  though  data  reliability  is  weak126.  This  is  why  some  countries  call  for  more 
consistent definitions and rules for disadvantaged participants127. 

According  to  case  studies,  the  programme  reaches  out  to  those  who  are  easier  to  reach 
among the disadvantaged, and not to those who are disengaged or at risk of marginalisation. 
This  is  not  specific  to  Erasmus+  as  other  benchmarked  programmes  struggle  with  the  same 
challenge.  This  is  partly  due  to  the  fact  that  the  participating  organisations,  even  under  an 
action  as  inclusive  as  the  European  Voluntary  Service  (EVS)128,  tend  to  select  the  most 
motivated and performing  individuals. In  spite of the specific measures mentioned above to 
foster  social  inclusiveness,  the  evaluation  found  that  more  specific  support  and  adequate 
funding  would  be  needed129.  The  lack  of  spending  flexibility  limits  the  capacity  to  face 
unforeseen situations inherent to vulnerable groups (see Efficiency)130 72% of respondents to 
the  consultation  indicated  that  there  has  not  been  sufficient  progress  in  this  area  under  the 
current  Programme  and  that  this  aspect  deserves  more  attention  and  funding131.  This  was 
also flagged by the majority of countries in their National Reports132. 

To mitigate the limitation of the related programme indicator mentioned above, beneficiaries 
were personally asked about their social background133. Evaluation survey responses indicate 
that  the  self-reported  participation  of  people  from  a  minority  background  or  with  learning 
difficulties is rather small. Yet, responses highlight that disadvantaged learners show more 
positive results than the others, in particular in completing formal learning and in boosting 
their  self-confidence  in  their  education  capacities.  However,  no  significant  difference  was 
noted for them in terms of benefiting from shorter transition periods to employment following 
their mobility experience.  

These  survey  results  are  statistically  significant134,  but  the  pre/post  surveys  of  school  pupils 
and the case studies indicate that they are likely due to selection into the programme, at least 
for a part135 and cannot be confirmed with strong evidence from data collected or the literature 
reviewed136. 

125 The numbers of participants with special needs or fewer opportunities supported were for education and training 8,000 in 
2013 (LLP) and 15,000 in 2014 (Erasmus+) with a target of 40,000 in 2020 ; for Youth 18,700 in 2013 (YiA) and 43,000 in 
2016 (Erasmus+) with a target of 37,000 in 2020 (Annual Activity Report 2016 of DG EAC). 
126  When  compared  with  other  sources  of  evidence  (surveys,  participation  of  disadvantaged  schools)  it  seems  that  the 
programme  data  may  have  overestimated  the  participation  of  disadvantaged.  Moreover  the  data  from  predecessor 
programmes is incomplete, hindering comparison over a longer period of time. See ICF, 6.2.2 about data limitations and 6.3 
on other measurements ; ICF, annex 2 for details 
127 FR, IT, NL 
128 The persons reached through youth exchanges might be more varied but there is less data on their profile. NRS, 3.4 
129  OPC:  According  to  the  position  papers  issued  by  two  national  level  organisations  and  one  EU  level  organisation 
(representing 50 regional and local stakeholders in the field of education and training), the lack of funding has a particularly 
negative  impact  on  beneficiaries  from  disadvantaged  backgrounds.  Disadvantaged  people  are  less  likely  to  participate  on 
their  own  initiative  in  activities  on  offer  without  receiving  pro-active  guidance.  Current  traveling  grants  hinder  the 
participation  of  certain  targets  groups  (e.g.  youth  coming  from  rural  areas)  according  to:  one  national  level  organisation 
representing 8 members in the youth and social work sector; one organisation representing 34 NGOs active in the youth and 
social sector; and one national organisation representing 22 religious organisations. NRS, 3.4 
130 OPC: positions of a non-EU ministry and 34 NGOs from 18 Member States active in the youth /social sector 
131 OPC, fig 6.2: A few responses suggested to improve the communication towards people from disadvantaged backgrounds, 
especially in rural areas. However, around one-third of the respondents stated that Erasmus+ already does enough in this area. 
132 NRS, 3.4: BEde, BEnl, BG, CZ, DE, EE, IE, EL, HR, IT, CY, LV, LT, LU, HU, MT, NL, AT, PT, RO, SI, SK, SE, UK, 
IS, FY, TR 
133 Learners were asked questions about possible disadvantage such as having a disability, the extent to which their parents 
were unemployed or families received social support or whether they repeated a class in the past. 
134 ICF, 6.8.2 for detailed survey findings 
135 If the grant does not enable the full group of learners to join, it is unlikely that those at the margin would participate. 
136  Most  of  evidence  from  these  two sources is  about  participation  rather than  results of  disadvantaged  groups.  In the  rare 
cases where their results were compared with other beneficiaries the results were positive. A particularly under-researched 
area is the multiplier effect from staff to disadvantaged groups. 

27 

 
 
                                                            
Most  interviewees  in  case  studies  believe  in  the  potential  of  the  programme  to  provide 
powerful  results  for  disadvantaged  groups.  Several  of  them  were  clear  that  more  than  other 
groups, disadvantaged learners would not have gone abroad to study had it not been for 
the  programme.  Meanwhile,  the  participation  of  organisations  with  high  shares  of 
disadvantaged learners remains  low. The case studies suggest that specific top-up points for 
organisations with high share of disadvantaged people should not have been discontinued in 
the  current  programme  and  that  accompanying  disadvantaged  learners  (e.g.  in  European 
Voluntary Service) requires a particular care and specific measures. 

On the external dimension, the picture is somewhat similar. All the predecessor programmes 
promoted inclusiveness in higher education from a gender perspective. Some promoted other 
under-represented,  disadvantaged  groups.  However  their  specific  evaluation  noted  that  the 
mobility programmes only partly succeeded in achieving equitable participation137. 

Beyond  mobility,  the  other  types  of  action  have  also  inspired  new  ways  of  working  with 
people  from  a  disadvantaged  background,  especially  in  sectors  other  than  higher 
education138.  This  was  reported  by  44%  of  practitioner  respondents139,  although  only  19% 
stated  that  the  activity  they  took  part  focused  on  integration  of  disadvantaged  groups.  This 
implies that there are significant spill over effects even where it is not the main purpose to 
target these groups (i.e. impacts beyond results). 

Lastly, certain actions perform better in terms of social inclusiveness. Studies in the Youth 
sector  found  that  international  youth  projects  or  mobility140  have  had  significantly  more 
impact on young people with fewer opportunities, especially in learning to learn, developing 
cultural  awareness  and  building  self-confidence  (+3.6%),  likely  due  to  the  lower  starting 
points141.  The  collaborative  platform  eTwinning  seem  slightly  more  able  to  reach 
disadvantaged groups than the rest of the programme142. Some of the adult education actions 
are  succeeding  in  engaging  with  hard  to  reach  groups,  but  given  the  low  budget  share  the 
scale of these actions have only been small.  

5.1.1.3. Outcomes for practitioners 

Within  Erasmus+,  practitioner  mobility has overall seen a significant increase compared 
to the past143. It remains more common in higher education than in all other education and 
training sectors put together, although higher education practitioners have comparatively more 
opportunities  abroad  through  other  schemes.  It  should  be  noted  that  an  important  share  of 
practitioners  in  the  adult  education  sector  teach  primarily  young  people  (aged  below  24)144. 
This  could  also  be  related  to  the  limited  effectiveness  of  the  programme  in  reaching  out  to 
practitioners who are not employed in large organisations. 

137 Evaluation of the EU Development Co-operation Support to Higher Education in Partner Countries (2007-2014), (2017), 
Particip, LeA, ECDPM, Ecorys, already mentioned 
138  Practitioner  beneficiaries  show  greater  use  of  strategies  to  enhance  education  attainment  of  students  than  their  control 
group.  Staff  from  organisations  with  high  share  of  learners  with  disadvantaged  background  more  frequently  took  part  in 
activities  focusing  on  disadvantaged  learners.  A  majority  of  sport  projects  focus  on  social  inclusion  and  physical 
enhancement activities. 
139 Evaluation survey of practitioners: even more true for sport (68%) and youth organisations (71%), but also for schools 
(57%) and AE (52%); less for HE (32%); ICF, 6.5.2.2 and fig. 6.32 to 6.34 for breakdown by sector 
140 RAY, 2015; Schroer, 2003; Sherraden et al., 2008 
141 RAY, 2015a 
142 ICF, 6.4.8.2 
143 Increase in all sectors where reliable data is available except the sector of adult learning where no additional funding was 
available in 2014. There is no reliable data on practitioner mobility for predecessor programmes in the sector of youth and 
higher education. 
144 ICF, 6.3 for highlights about the profile of practitioners: The teaching staff taking part in VET and HE sectors represents a 
more balanced sample of subjects than school staff, where nearly half are foreign language teachers. 

28 

 
 
                                                            
In addition, collaborative platforms expand participation in the programme (eTwinning, 
School  Education  Gateway145,  EPALE146)  and  are  associated  with  positive  results  for 
beneficiary practitioners147. In particular eTwinning, the platform for schools, attracts a very 
large number of teachers and pupils, in the EU and beyond, as mentioned previously. In 11 
countries, eTwinning seems to be reaching to more than 10% of all school teachers148. As a 
stepping stone into the programme, as noted by some interviewees, it does reach to a broader 
audience than other types  of action149. After  stagnation in  2011-2013, its audience has been 
sharply increasing under Erasmus+, pointing at a strong potential of this kind of intervention 
for the future. 

Practitioners are even more satisfied with their mobility experience (98% compared with 
93% for learners) and very positive about the contribution to their competence development 
according  to  the  monitoring  surveys.  For  most  of  the  competences  the  differences  across 
sector are minor150. 

Practitioners who (strongly) agree that they developed a given key competence or skill 

Source: ICF calculations based on Erasmus+ participant monitoring surveys. This is a multiple answer question therefore the data does not 
add to 100%. N = 227,319 

Most National Reports refer to results for practitioners with a high level of similarity with the 
benefits  listed  above,  in  particular  for  education  and  training  staff:  language  skills151,  new 

145 The School Education Gateway offers collaborative space and on-line resources to all professionals in the field of school 
education: https://www.schooleducationgateway.eu/en/pub/index.htm 
146 Electronic Platform for Adult Learning in Europe launched under the LLP and animated by National Support Services 
for stakeholders to exchange and develop their professional practice: https://ec.europa.eu/epale/ 
147  Based  on  extrapolation  from  eTwinning  findings  since  primary  data  was  not  collected  specifically  about  the  School 
Education Gateway and EPALE platforms for this programme-level evaluation and even though it is difficult to differentiate 
their contribution from other actions of the programme 
148 BG, EE, EL, HR, CY, LV, LT, MT, RO, SI, SK, by increasing order; from 2,2% in Germany to more than 35% in Malta; 
ICF, fig. 6.24: CSS data on registered teachers per country and Eurostat data on the overall population 
149  More  than  183,000  schools  registered  by  September  2017  based  on  CSS  Monitoring  data.  It  does  not  mean  that  the 
registered users actually take part in a project every year. Only in 2016, 29,752 teachers took part in at least one project. ICF, 
6.4.9 for profile of users compared to other participants 
150 ICF, 6.4.7.5 and fig. 6.17 for a breakdown per sector 
151 DE, LV, FI, UK, IS, TR 

29 

 
 
 
                                                            
teaching  methods152  and  job  satisfaction153;  as  well  as  for  youth  organisations’  staff: 
professional development154 and new professional methods155. 

Evaluation surveys found in addition, when comparing with a control group, that participation 
in  the  programme  is  associated  with  wider  networking  and  cooperation,  stronger 
attachment to Europe, and greater use of digital resources, as shown below156: 

Main counterfactual results for practitioners 

Source: ICF beneficiary survey for practitioners 

From  virtual  collaboration on Erasmus+  funded platforms, beneficiary practitioners  also 
observe positive results, though admittedly with smaller intensity than in the case of mobility 
or  partnerships.  Even  though  it  is  difficult  to  differentiate  its  contribution  from  other 
actions157,  in  certain  areas  practitioners  associate  eTwinning  with  more  positive  results  than 
the  control  group,  such  as  international  networking,  but  also  digital  skills  (the  collaborative 
platform  might  attract  practitioners  that  are  already  digitally  skilled,  but  can  also  enhance 
such skills). Findings of other sources support the positive results from ICF’s general surveys: 
another  distinct  evaluation  of  eTwinning  revealed  that  a  vast  majority  of  teachers  find  the 
platform  helpful  to  network  across  Europe  (64%)  or  to  improve  relationships  between 
teachers and learners (62%)158. 

More generally, all practitioners interviewed cited positive results for themselves as well as 
their learners and organisations159. They found their Erasmus+ experience as an opportunity 
to go ‘outside of the ordinary’ which can be important for retention in the teaching profession, 
a  crucial  issue  in  many  countries160.  As  practitioners  appear  mostly  motivated  by  individual 
improvement,  it  is  not  surprising  to  find  a  clearer  contribution  to  development  of  staff  than 

152 BEnl, FR, CY, LV, LT, MT, FI, UK 
153 BEnl, FI, UK, IS 
154 BEnl, EL, CY, LT, PT 
155 BEnl, CY, LT 
156 Involvement in volunteering is also higher but can result from the programme selecting the most engaged staff. 
157 ICF’s beneficiary survey covered also eTwinning participants. However most of respondents who took part in eTwinning 
also  took  part  in  other  activities  funded  by  Erasmus+.  It  is  therefore  not  possible  to  judge  whether  the  difference  is  due 
specifically to eTwinning. ICF, fig 6.26 
158 Education for Change (2013), mentioned above 
159  Professional  development,  motivation,  open-mindedness  were  commonly  cited  in  the  context  of  case  studies  (most 
significant change approach) next to a range of skills. The surveys also found an association (though a weaker one) with job 
satisfaction and, in the school sector, wider use of strategies to improve pupil attainment. 
160 This is particularly true in schools and VET; less in the adult education, youth and sport sectors where "out of curriculum" 
freedom is more easily available by other means. 

30 

Large positive differenceMedium positive differenceStatistically significant yet small positive differenceMedium negative differenceNetworking - frequent contact with foreign counterpartiesFeeling EuropeanJob satisfactionCooperation with international partners - exchange on topics and methods of professional interestInvolvement in volunteering or community activitiesPerception of the importance of political and civic participationUse of digital resources and media 
 
 
 
                                                            
hard  evidence  of  evolution  of  pedagogical  or  institutional  practices.  Other  studies  point  to 
positive  impact  on  teaching  practices,  social  competences  and  motivation  for  professional 
cooperation161. 

Recognition  is  meanwhile  less  formalised  for  practitioners  than  for  learners  by  nature. 
Only around half of the practitioner survey respondents recorded having received some form 
of  recognition162.  While  a  majority  of  practitioners  noted  informal  recognition  by  peers, 
effective  recognition  by  the  hierarchy  is  less  common.  At  the  same  time,  nearly  half  of 
respondents  state  that  participation  in  the  programme  helped  them  achieve  new  roles  or 
positions, which is very positive. 

It should also be noted that several of the results discussed in this section also have a spill-
over effect from individual level to organisational level, as elaborated below. 

5.1.2.  Outcomes for organisations and systems 

As for mobility (KA1), the evaluated programmes reached or exceeded the vast majority of 
the  targets  when  it  comes  to  the  number  of  projects  funded  (KA2  and  KA3).  Compared  to 
predecessor  programmes,  this  represents  a  strong  increase  in  the  number  of  higher 
education projects163, but also a strong decrease in all other sectors. This is due to the lower 
budget  available  at  the  start  of  the  programme  but  also  to  the  fact  that  Erasmus+  aims  to 
focus  on  funding  fewer  large-scale  projects164.  Erasmus+  is  nonetheless  perceived  as 
providing more opportunities for cooperation than it was previously possible165. 

However,  the  fact  that  Erasmus+  and  its  predecessor  programmes  have  produced  a  high 
volume  of  intellectual  outputs  is  not  necessarily  an  indicator  of  project  quality166.  The 
panel  experts  have  pointed  out  several  issues  with  the  selection  process. They  consider  that 
the quality of applications selected is in general quite average. According to monitoring data, 
applications  are  frequently  scored  higher  than  the  final  outputs.  Many  projects  (other  than 
KA2  Alliances)  lack  a  continuous  quality  review,  external  support,  or  solid  needs  analysis. 
National Reports refer to some enhancing-effectiveness factors (e.g. experienced staff on the 
applicants'  side  -  a  condition  more  often  met  by  large  organisations  and  therefore  in  higher 
education, and good cooperation between partners); they also outline challenges which hinder 
effectiveness (e.g. high-quality projects rejected due to lack of funding). 

On  the  contrary,  the  evaluators  find  that  fewer  outputs  with  clearer  added  value  might 
possibly  meet  more  clearly  the  innovation  objective  of  the  programme.  According  to  the 
qualitative  assessment  of  outputs  carried  out  by  the  expert  panel,  an  area  for  further 
improvement  of  Erasmus+  is  the  quality  of  KA2  (and  to  a  more  limited  extent  KA3  which 
cannot  be  fully  evaluated  at  mid-term)  outputs  and  especially  their  effective  potential  for 
mainstreaming beyond participating organisations. Although the programme often produces 
something new for participating organisations167, this innovation does not enough follow the 
state  of  the  art  developments  in  a  given  sector.  While  the  programme  does  fund  good  and 

161 In the literature evidence of impact on adult education is somewhat scarce and absent for sport and Jean Monnet; ICF, 
6.4.7. 
162  85%  of  practitioners  got  international  experience  accepted  as  training;  58%  state  it  was  acknowledge  by  hierarchy  or 
peers; 45% state that it helped them attain a new function/ level of seniority; 26% received a financial reward. 
163 Prior to 2014, the higher education sector was rather under-represented in the volume of cooperation projects compared to 
its share of mobility funding. 
164 See above section 3 
165 OPC: 80% (n = 558) strongly agreed or agreed 
166 Sources to assess project quality: monitoring surveys, case studies, expert panel assessment of KA2/KA3 outputs 
167  European  cooperation  in  education  and  training:  added  value  and  impact,  L’Observatoire  Erasmus+,  Pluricité  and 
Synoptic.Pro, note 6, November 2017 

31 

 
 
                                                            
innovative  activities168,  these  are  relatively  hidden  among  many  standard  projects.  The  vast 
majority  of  case  studies  identified  some  form  of  innovation  or  change  in  the  organisations 
reviewed  (e.g.  use  of  materials  or  methods  shared  during  the  projects)169.  Yet  often  these 
innovations remain small scale or light touch and limited to a specific part of the organisation. 
For  many  funded  projects,  it  appears  that  the  main  benefits  arise  from  the  participation 
process  rather  than  the  output  itself.  However,  where  there  are  instances  of  high  quality 
outputs, these could be better supported upstream and thereafter mainstreamed building upon 
the appreciated precedent of the 2016 specific call in the area of social inclusion170. 

5.1.2.1. Outcomes for organisations171 

Over  2007-2016,  more  than  940,000  organisations  were  contracted,  at  an  average  of 
approximately  115,000  organisations  per  year  for  the  predecessor  programmes  but  80,000 
under  Erasmus+172.  During  the  public  consultation,  74%  of  respondents  thought  that  the 
Programme  improves  the  quality,  innovation  and  internationalisation  in  education,  training 
and youth organisations173. When asked in the monitoring survey about expected changes to 
their  institutions,  more  than  55%  (up  to  85%)  of  practitioner  respondents  stated  that  most 
such changes (10 out of 11) had indeed taken or were taking place174. Later, when asked in the 
evaluation survey about changes that had happened after their participation175, a vast majority 
of practitioners once again agreed with a high number of types of changes listed, such as the 
introduction of new teaching materials176 or new assessment methods177. In the youth sector, 
75% of practitioners agreed with most statements about organisational results. Lastly, most of 
other  socio-economic  actors  (companies,  public  authorities,  civil  society  bodies)  stated  that 
their objectives were (fully) met178. For them, the most commonly cited barriers concern the 
complexity  of  the  application  procedure  and  the  low  success  rates  in  applications,  which  is 
due to budget availabilities179. 

Furthermore, the level of cross-sectoral cooperation increased under the current programme 
by 23 percentage points180. Paradoxically, interviewees are often sceptical about the reality of 
this  key  evolution.  This  contrasts  with  OPC  or  agency  respondents,  whom,  to  a  very  high 

168 74% of higher education practitioners in the higher education sector reported the creation of new research projects and 
37% the creation of new spin offs. 
169 Case studies were selected at random using a rather broad set of criteria; ICF, 6.5.2.4 and Annex 9 
170 An envelope of EUR 13 million was set aside to support the upscaling of good practices from grass-roots level in the area 
of social inclusion; Official Journal C 99/05 of 15/03/2016. 
171 Results at organisation level were assessed mainly through monitoring survey, practitioner survey (staff were asked not 
only about their own development but also about changes in their organisation), survey of socio-economic actors and case 
studies, supplemented with more transversal sources (OPC, NRS, literature, etc)  
172 ICF, Annex 2; multiple participation of organisations remains unclear, as detailed below in section 4.3 and annex 3 
173 n = 1333. Of significance for the Jean Monnet strand, 70% of respondents thought that promoting excellence in teaching 
and research in European integration activities had been effective to a (very) large extent (n = 888). 
174 The indicator of DG EAC's Strategic Plan is “% of organisations that have developed/adopted innovative methods and/or 
materials,  improved  capacity;  outreach  methodologies,  etc.”  However,  all  findings  are  based  on  surveys  of  staff,  not 
organisations. Hence there is only a partial match between the official indicator and available data.  
175 In contrast to evaluation surveys, monitoring survey ask shortly after participation about what mobile staff intends to do 
on return to their organisations. The findings are very positive but they can capture the intention rather than the actual effect. 
176 81% SE, 72% VET, 72% HE, 77% AE 
177 68% SE, 63% VET, 59% HE, 68% AE 
178 ICF, 6.5.2.3 and fig. 6.40: survey of participant organisations other than education, training, youth and sport ones 
179  Response  patterns  differ  according  to  the  profile  of  respondent.  Companies  cite  as  most  important  barrier  the  lack  of 
knowledge about the programme. Public authorities first cite lack of staff available to take part in the programme. 
180 ICF, 6.10: with 47% of projects in the predecessor period involving multiple sectors/fields, compared to 70% of projects 
so far under Erasmus+. A similar finding holds for pairs of organisation linked in a same partnership, with one third of pairs 
under the predecessor programme being cross-sectoral partnerships, whereas half of pairs under Erasmus+ are cross-sectoral. 

32 

 
 
                                                            
extent,  recognise  the  effectiveness  of  cross-sector  cooperation181.  In  the  same  vein,  the 
network analysis shows promising results in the form of an increase in the number of sectors 
participating  per  project.  There  are  notably  many  more  higher  education  institutions  taking 
part in other strands of the programme than in the past. Cross-sector participation seems more 
challenging in other education and training sectors where organisations are smaller in average. 
Reciprocally,  youth  organisations  strongly  contribute  to  the  international  dimension  of  the 
programme  and  sport  organisations  take  part  in  other  sectors  of  the  programme  even  more 
than in the their own strand of Erasmus+. This contrast between perception and observation 
suggests that there is a need for better communication to potential applicants about cross-
sectoral opportunities. 

In  terms  of  improved  geographical  balance,  the  centrality  of  countries  in  the  programme 
network is changing. Small countries and countries from central and eastern Europe are better 
integrated182.  The  main  exception  remains  the  country  coverage  of  Jean  Monnet  activities, 
with about 25% of its grants going to two countries only183. 

The surveys also  demonstrate a  clear internationalisation of  organisations, in  and outside 
the EU184. In the monitoring survey, practitioners in general185 most strongly believe that their 
mobility  will  lead  (85%)  or  has  already  led  (74%)  to  internationalisation  of  their  sending 
institution,  as  shown  in  the  figures  below.  For  instance,  59%  of  concerned  respondents 
believe that the Jean Monnet grant helped them to allocate additional funding for teaching or 
research  about  the  EU.  The  fact  that  practitioners  have  developed  strong  networks,  as 
demonstrated  above,  corroborates  other  findings  on  the  internationalisation  of  their 
organisations. Participant organisations are significantly more likely than non-participants to 
be engaged in  transnational cooperation, except in the sport sector186. This holds true to the 
largest extent for higher education institutions187 and to the least for schools188, as confirmed 
in case studies. Specific findings for the international dimension of the programme underline 
the  strong  contribution  of  the  predecessor  programmes  to  the  internationalisation  of  higher 
education  institutions  in  partner  countries  through  curriculum  development  and  the 
improvement  of  management  practices  of  universities189.  But  even  in  the  case  of  school 
education,  a  vast  majority  of  respondents  believe,  for  example,  that  Comenius  transnational 
partnerships strengthened the European dimension of schools190 and the survey conducted as 
part  of 
the  eTwinning  platform  evaluation  has  observed,  as  a  result,  a  certain 
internationalisation of schools. It is particularly important for organisations in the youth and 

181 80% (n = 471) of OPC respondents (strongly) agreed that the integrated nature of Erasmus+ has strengthened cooperation 
across sectors. 90% of agency respondents strongly or rather agree with this opinion. 
182 ICF, 6.3 for participation patterns and fig. 6.3 as for the evolution of the centrality of the top 20 countries 
183 ICF/JMO, 1.2.3 and fig. 1.5: Italy (16% of grants, considering funding from Heading 1 only) and Spain (9%). Countries in 
Northern and South-Eastern Europe are not very often among the JM beneficiaries when compared to Western or Southern 
Europe. Meanwhile Jean Monnet covered 82 different countries since 2007 (with Heading 4). 
184 Internationalisation is a broad concept that has a variety of layers. It can mean simply the fact of having contacts with an 
organisation  from  another  country  but  also  integration  of  an  international  dimension  into  regular  activities  (for  example 
through eTwinning), teaching in a foreign language, hosting staff/ learners from other countries, etc 
185 Higher education staff is more optimistic about the internationalisation of their institution, higher education international 
staff about spin-off effects, while school education staff believes more strongly in the use of new teaching methods and 76% 
of youth staff believe in the effect on the quality of the project they develop in their organisations. ICF, fig. 6.18 and 6.20 
186 In contrast to other sectors, there was no significant difference in the likelihood of engaging in transnational partnerships 
between Erasmus+ sport participants and non-participants but this needs to be taken cautiously as the control group for sport 
was reconstructed in combining the control group for VET and the too few non-participants who responded for sport. See 
Annex 3. 
187 In the HE sector, internationalisation is more advanced than in other sectors (ICF, Annex 11 on literature). 
188 ICF, tab.6.33 for share of organisation with an international strategy (79% v. 68% in control group) 
189 Evaluation of the EU Development Co-operation Support to Higher Education in Partner Countries (2007-2014), (2017), 
Particip, LeA, ECDPM, Ecorys, already mentioned 
190 ICF, Annex 11 (GES and ZSB, 2010) 

33 

 
 
 
                                                            
sport  sectors  which  work  on  specific  issues  for  which  they  have  few  counterparts  in  their 
countries191. Given their  scale and the number of organisations reached, it can  be  concluded 
that the programmes strongly contribute to strengthening the internationalisation of education, 
training, and youth sectors. 

Whether mobility has led or will lead to changes in the sending institution 

Source: ICF calculations based on Erasmus+ participant monitoring surveys. 

In contrast, the internationalisation effect is less clear-cut on sport organisations. Competitive 
sport  is  already  much  internationalised  independently  from  any  EU  intervention,  whereas 
grassroots  sport  organisations  have  fewer  international  opportunities.  In  the  latter  case 
Erasmus+ has not reached the required critical mass. Yet, sport organisations' knowledge and 
usage  of  the  sport  guidelines  promoted  by  the  EU  (on  dual  careers192  or  health  enhancing 
physical  activity193)  suggest  some  alignment  of  sport  actions  with  EU  policies  (see  fig. 
below)194.  A  total  of  45%  of  organisations  strongly  comply195  with  the  good  governance  in 
sport principles196. 

191 ICF, 6.5.2.4 and Annex 9 (case studies) 
192 http://ec.europa.eu/assets/eac/sport/library/documents/dual-career-guidelines-final_en.pdf 
193 http://ec.europa.eu/assets/eac/sport/library/policy_documents/eu-physical-activity-guidelines-2008_en.pdf 
194 ICF, fig 6.38 and 6.39 
195 ICF, tab. 6.23 and 6.24; organisations are the least compliant with the requirement of an appeal procedure 
196 http://ec.europa.eu/assets/eac/sport/library/policy_documents/xg-gg-201307-dlvrbl2-sept2013.pdf 

34 

 
 
 
                                                            
Usage of the EU guidelines in the field of sport 

Source: ICF beneficiary surveys (among practitioner respondents aware of these guidelines)) 

Only  4  out  of  10  participant  organisations  have  strong  measures  in  place  to  combat  doping 
and violence, while almost half have measures to combat discrimination197. 

Share of respondents who comply with the anti-doping, violence and 
discrimination in sport principles (% of organisations) 

Principle 

Strong measures 

Medium measures 

Low measures 

Anti-doping 

Violence 

Discrimination 

40% 

41% 

48% 

27% 

38% 

33% 

33% 

21% 

19% 

Note: Index (1 (strongly agree)-4 (strongly disagree), 1-1.5=strong measures, 1.51-2=medium measures, less is low measures 

More generally, another very positive monitoring finding is that overall 91% of practitioners 
state that they have shared their own knowledge with students and/or other persons after their 
participation  in  Erasmus+198.  The  programme  is  particularly  seen  as  transformational  for 
youth  organisations,  where  members  of  staff  are  significantly  more  optimistic  about  the 
contribution  of  the  programme  to  their  organisations  than  in  other  sectors199.  Interestingly, 
71%  of  practitioners  in  the  sport  sector  state  that  the  programme  contributed  to  new 
cooperation  with  civil  society200.  The  literature  review  points  as  well  to  strong  positive 
impact, particularly on: internationalisation (all sectors), teaching capacity (school education, 
VET), and staff development (VET, youth)201, although evidence of benefits for organisations 
on sport, adult education and Jean Monnet activities is somewhat scarce. 

Other  results  are  softer  and  there  is  less  evidence  of  deep  changes  of  institutional  or 
pedagogical practices202. The qualitative findings, however, suggest that a key contribution 
that  was  not  measured  quantitatively  is  the  self-reflection  and  soft-benchmarking  that 
happens  through  the  opportunities  for  practitioners  to  see  how  other  organisations  tackle  a 
given issue. In all case studies, concrete changes at organisational level were indeed cited203. 
In many cases these are light adjustments rather than major structural evolutions; however an 
accumulation  of  light  changes  over  time  can  result  in  a  step  change.  Frequently  noted 
mentions  include  not  only  internationalisation  (most  commonly  cited)204,  but  also  soft 
benchmarking, implementation of new tools, materials, increased positive image and with that 

197 ICF, tab. 6.25 to 6.28 
198 ICF, Fig. 6.29; from 97% of HE International staff to 86% of practitioners of Adult Education. 
199 ICF, fig. 6.36 
200 ICF, fig. 6.37 
201 ICF, tab. 6.30 and 6.31 and Annex 11 
202 Findings are less robust, in the absence of indicators of concrete change, such as improvement in student outcomes. 
203 ICF, 6.5.2.4 and Annex 9 for a summary of case studies and effects identified at organisation level 
204 ICF, tab. 6.29 

35 

 
 
 
                                                            
associated empowerment of practitioners and learners. Other effects for organisations include 
the  development  of  national  partnerships,  cooperation  with  companies205  or  quality 
frameworks  specifically  for  youth  work.  On  average,  58%  of  practitioners  reported  as  an 
outcome the creation of quality assurance approaches206. 

The  National  Reports  confirm  that  Erasmus+  has  three  main  impacts  on  participating 
organisations, namely their internationalisation207, the quality of education and training they 
provide208 and the professionalisation of youth work209. Both mobility actions and Strategic 
Partnerships  allow  for  the  development  of  new  methods,  often  innovative  at  least  for 
participating  organisations,  which  can  have  a  positive  impact  on  quality.  However,  the 
evidence gathered at EU level in that respect is not sufficiently strong to assess whether the 
programme  is  transforming  the  practices  of  practitioners  and  organisations.  Though 
practitioners are positive about the fact that they implement the lessons learnt and results from 
cooperation  projects  and  mobility  exchanges,  there  is  only  anecdotal  evidence  that  the 
programme is changing institutions’ approaches to pedagogy and their target groups. Deeper 
qualitative  work  of  a  larger  scale  would  be  needed  to  see  these  types  of  results  at  final 
evaluation stage. 

5.1.2.2. Outcomes for systems and policies210 

The  general  literature  shows  little  evidence  of  system  level  impacts  of  preceding 
interventions,  in  contrast  to  the  abundant  findings  about  individual  impacts.  Where  such 
evidence  exists,  it  is  positive.  But  it  is  very  fragmented  in  all  sectors,  although  stronger  in 
higher education, VET and youth. 

This  is  why,  since  2014,  Erasmus+  focuses  through  KA3  on  funding  actions  that  are 
specifically designed to have system level effects211. Comparable actions existed under LLP 
and Youth in Action, but some new features were introduced under the current programme212, 
enabling the Commission to address more rapidly emerging needs (e.g. the Paris Declaration, 
the New Skills Agenda for Europe). The systemic impact of Erasmus+ is only expected "by 
the  end  of  the  Programme"213  and  therefore  can  only  be  evaluated  at  final  stage214.  In 
particular,  most  KA3  projects  were  not  finalised  at  mid-term.  Nonetheless  some  National 
Reports outline KA3's policy effects although limited by the relatively small funding allocated 
to KA3215. Not all KA3 case studies could identify system level effects. Although KA3 has 

205 62% of VET staff report influence on cooperation with companies 
206 Truer for sport (67%) and above all youth organisations (76%) than for education and training ones; ICF, 6.5.2.2 and fig. 
6.35 to 6.37 for breakdown by sector 
207 BEde, BEnl, CZ, DK, EE, IE, EL, ES, FR, HR, CY, LT, HU, MT, NL, PL, RO, SI, FI, UK, IS, FY, LI, NO, TR 
208 BEnl, DE, EE, IE, EL, ES, FR, HR, LT, HU, MT, PL, RO, SK, FI, SE, UK, FY, NO, TR;  
209 BEnl, IE, EL, ES, IT, LT, NL, PL, SI, SK, FI, SE, UK, NO 
210 Results at organisation level were assessed mainly through monitoring survey, practitioner survey (staff were asked not 
only about their own development but also about changes in their organisation), survey of socio-economic actors and case 
studies, supplemented with more transversal sources (OPC, NRS, literature, etc)  
211 Including support to European networks and tools fostering transparency and recognition of skills and qualifications 
212 Under Key Action 3 (KA3) the programme funds Structured dialogue between young people and policy makers; national 
contact points in relation to EU tools (EQF, EQAVET, ECVET, adult learning, etc.) and policies (Eurodesk, ENIC); support 
to  EU  level  civil  society  bodies;  actions  aimed  to  strengthen  evidence-based  policy  making  (studies,  cooperation  with  the 
OECD);  activities  related  to  EU-level  OMCs;  forward  looking  projects  and  projects  on  specific  EU-level  priorities 
(apprenticeships, social inclusion, etc.). 
213 Article 21 (1) (b) of Regulation 1288/2013 
214 The vast majority of interviewees, who were provided with a list of KA3 projects, were not aware of them and many were 
unclear about the difference between KA2 and KA3. 
215 e.g. in the youth field: BEnl, IE, FR, LV, NL, PL, PT, NO, TR. However, a too low number of KA3 projects result in 
limited policy-effects (e.g. HR, HU, PT); NRS 5.1.6, 5.2 and 5.5 

36 

 
 
 
                                                            
the highest potential to produce systemic impact216, further involvement of decision-makers is 
required.  It  seems  therefore  too  early  to  conclude  on  the  additional  systemic  impact 
brought specifically by Erasmus+. 

However, even if effectiveness varies across instruments and countries (some use them much 
more strategically than others217), some policy networks financed through Erasmus+ have 
proven  influential  since  2007:  the  national  contact  points  for  EQF218  or  EQAVET219,  the 
Bologna  experts220  or  the  partnership  between  the  Commission  and  the  European  Youth 
Forum221. Studies and cooperation with OECD222 funded by Erasmus+ and LLP have provided 
Member States with stronger evidence, according to the evaluation of ET2020223. 

On  the  international  dimension,  the  general  finding  is  that  the  projects  funded  by  previous 
programmes  have  contributed  to  policy  and  institutional  reforms  in  partner  countries, 
through improved quality assurance, standardisation of higher education and a rapid extension 
of EU-supported Bologna principles. This has had an evidenced impact on the harmonisation 
of programmes and curricula in partner countries as well as on mobility and the recognition of 
studies and qualifications within and between non-European regions and Europe224. 

Furthermore, the effectiveness of the programmes evaluated should also be assessed through 
their  contribution  to  policies  via  the  Open  Method  of  Cooperation  (OMC),  which  they 
have  substantially  supported  over  the  period  2007-2016225.  The  evaluation  of  the  OMC  in 
education and training (ET2020) noted that according to national stakeholders the outputs of 
OMC  activities  have  been  used  to  influence  national  policies  and  practices.  Therefore  the 
programme  plays  a  major  role  in  supporting  financially  the  OMC  process,  which  in  turn 
influences  national  education  and  training  policies,  including  those  priorities  of  ET2020 
which  are  not  easy  to  tackle  through  mobility  and  cooperation  actions  (e.g.  policies  on 
qualifications  design  or  early  school  leaving  policies).  The  programmes  had  also  a  certain 
systemic  effect  in  the  field  of  youth.  The  most  recent  evaluation  of  the  European  Youth 
Strategy226  concluded  that  the  OMC  was  rather  positive  about  the  influence  on  national 
policies and structures in the youth field, while the results in the youth field might not always 
be measurable in a way that is easily comparable with the area of education and training. At 
the  same  time,  the  activities  funded  by  Youth  in  Action  and  Erasmus+  were  identified  as 
strong  pillars  of  the  OMC,  regarding  the  structural  dialogue  with  young  people  and  the 
increasing  use  of  the  Youthpass  to  promote  learning  outcomes  of  volunteers.  Sport  is  too 
recent a programme sector to be evaluated at interim stage in relation to systemic impact227. 

216 e.g. LT, NO, TR 
217 The system level impact of policy instruments, such as the EQF or ECVET, is to a large extent determined by national 
policy factors and priorities. The fact that Ersamus+ finances the related networks makes for an indirect causality link. 
218 ICF (2012) Evaluation of the Implementation of the European Qualifications Framework Recommendation  
219  ICF  (2013)  Evaluation  of  implementation  of  the  European  Quality  Assurance  Reference  Framework  for  Vocational 
Education and Training (EQAVET); EQAVET national points are only funded through the programme since 2016. 
220 ICF (2013) Study to examine the impact of the national teams of Bologna experts on the implementation of the Bologna 
process; National Reports mention also the support to the Bologna and Copenhagen processes (BEnl, IT, RO, FI, NO, TR) 
221 Ecorys (2013) Evaluation of the European Commission-European Youth Forum Operating Grant Agreements 
222 E.g. PISA and PIAAC surveys and their subsequent analysis, both highly influential sources of evidence 
223 Ecorys (2014) Interim Evaluation of the strategic framework for European cooperation in education and training 
224 Evaluation of the EU Development Co-operation Support to Higher Education in Partner Countries (2007-2014), (2017), 
Particip, LeA, ECDPM, Ecorys, already mentioned 
225 The programmes fund at EU level working groups and research and co-fund at national level the implementation of EU 
tools (e.g. National Contact Points for the European Qualifications Framework); ICF, 6.5.3 
226 ICF (2016) Evaluation of the EU Youth Strategy and the Council Recommendation on the mobility of young volunteers 
227  There  is  no  formal  OMC  in  the  field  of  sport  but  similar  activities  take  place  under  the  EU  Work  Plan  for  sport.  The 
Commission adopted a report on its implementation in January 2017. The responses of 105 policymakers and stakeholders 
suggest that the Work Plan is perceived as influential by the respondents, although no concrete examples of effects on sport 
organisations or sport policies were given. 

37 

 
 
                                                            
Overall,  88%  of  respondents  to  the  consultation  agreed  that  Erasmus+  has  increased 
systemic  impact  compared  to  the  predecessor  programmes228,  whilst  46%  estimated  that 
the programmes had been effective in modernising education, training and youth systems229. 
As  shown  in  the  figure  below,  around  80%  of  the  key  informant  interviewees  saw  some 
positive  influence  of  the  programme  on  policy,  of  which  half  were  able  to  cite  concrete 
examples.  This  can  be  regarded  as  a  good  performance230.  Interviewees  tend  to  be  more 
positive  about  the  systemic  influence  of  the  programme  in  smaller  and  less  wealthy 
countries.  Generally,  most  frequent  examples  cited  dealt  with  EU  tools  related  to 
qualifications.  It  seems  that  interviewees  no  longer  think  of  systemic  changes  as  resulting 
from the scale of funded mobility. Lastly, the examples remained particularly rare in the sport 
sector231. 

Share of key informants interviewed who were able to cite concrete policy/ system changes 
linked to Erasmus+ or its predecessor programmes 

Legend: As positive were coded responses which named a concrete and influential project or a clear area of influence supported via one of 
the programmes. As neutral were coded interviews that thought there was some influence but did not give a concrete example. As  negative 
were coded interviews which said they are not aware of any such influence. 

The interviewees mainly cited four types of changes at system level232. Occasionally, even a 
single  funded  project  can  be  used  as  a  strategic  input  at  national  policy  level  (e.g.  no 
alternative  resource  to  design  work-based  learning  legislation  would  have  been  available  in 
Latvia).  More  frequently,  examples  were  found  where  an  effort  has  been  made  to  pool 
together the findings of several projects to accompany a  systemic development inspired by 
European priorities (e.g. the Polish reform based on ECVET). Thirdly, the scale of mobility 
actions can trigger a change in national policy to remove barriers (e.g. Erasmus+ beneficiaries 
are no longer charged a visa fee when going to Turkey). Lastly, programme funded activities 
can raise awareness of an issue at policy level (e.g. early development of Lifelong Learning 
Platforms at regional level in Spain). 

System  level  influence  via  individual  cooperation  projects  remains  an  exception.  This 
does not so much depend on the type of action but rather on the own merits of a given project. 
Effects  at  system  level  are  more  likely  to  happen  if  the  project  is  aligned  with  country 
priorities233.  Even  when  small  scale  projects  prove  to  be  innovative,  unless  they  had  a  high 
level influential stakeholder involved from the beginning234 they typically would not have had 

228 OPC: n = 424 
229 OPC: n = 1,206; it must be noted nevertheless that 262 respondents rather selected “no opinion” and that, as seen under 
5.1, no more than 44% regarded this objective as extremely relevant. 
230 The interviews were carried out mainly with key policy-makers who generally were not directly involved in projects. 
231  Relatively  few  projects  focus  on  systemic  issues  stated  in  the  programme  objectives  (combating  threats  to  sport, 
governance; dual careers of athletes). An high share of sport organisations taking part are besides small local bodies. 
232 ICF, 6.5.3.1 for more examples; ICF, 6.6 for effectiveness per sector and type of action 
233 National Reports of FI, NO 
234 Outlined in several National Reports, be it at local (LT, NL, NO, TR) or national (CY, IS) level 

38 

 
 
 
                                                            
the  capacity  to  reach  out  effectively  to  decision  makers.  It should  nonetheless  be  noted that 
Knowledge  Alliances  and  Sector  Skills  Alliances,  two  actions  of  a  new  kind  and  more 
ambitious within KA2, could not be evaluated at mid-term but might have a certain impact on 
systems at the time of the final evaluation of Erasmus+. 

The evaluation shows that a better way for the programmes to achieve system level impact is 
by  reaching  out  to  a  critical  mass  of  practitioners  and  organisations.  This  has  been  the 
case in higher education where mobility actions, by their volume, led to system level changes 
e.g.  implementation  at  national  level  of  European  credit  systems  (ECTS)  and  joint  degrees 
(Erasmus Mundus). Erasmus increased the employability advantage for Erasmus alumni over 
their  non-mobile  peers  by  45%  and  half  of  all  European  graduates  who  studied  or  trained 
abroad  benefited  from  it  according  to  ex-post  impact  research235.  In  several  of  the  other 
sectors, the programme is also close to reaching a critical mass. KA1 alone supported in 2014 
1.6% of higher education students, close to 1% of VET students and 0.36% of school staff236. 
However the impact on VET and adult education systems is more difficult to detect because 
of the scale and the heterogeneous nature of these sectors, which would require more targeted 
actions considering available funding. In KA2, eTwinning is also instrumental in reaching out 
to  a  critical  mass  of  teachers,  as  described  above.  Other  types  of  activity  do  not  reach  the 
same volume, but can be influential in areas where system level changes are expected237. For 
the  youth  sector,  although  an  impact  on  systems  is  observed  through  other  channels,  it  is 
difficult  to  evaluate  to  what  extent  the  programme  succeeds  in  reaching  a  critical  mass238. 
Other  sectors  such  as  adult  education  or  sport  remain  too  small  to  see  system  level  results 
through this channel. 

Last  but  not  least,  Erasmus+  is  expected,  in  line  with  its  legal  basis,  to  contribute  to  the 
Europe  2020  headline  target  in  education239,  namely  (a)  higher  education  attainment  and 
(b) tackling early school leaving. Though the EU countries seem well on target to deliver on 
these  two  dimensions,  the  contribution  of  the  programme  and  its  predecessor  to  their 
evolution  is  mostly indirect. The choice of these overarching indicators for the programme 
can  indeed  be  questioned  in  terms  of  testability  and  plausibility.  Though  the  programme 
recitals cover both targets, this is done together with many other priorities. It is true that, as 
seen  above,  some  countries  did  make  policy  changes  linked  to  these  two  benchmarks.  The 
programme  also  reaches  a  high  number  of  learners  via  KA1  and  KA2,  whose  attitude 
towards  education/training  is  more  positive  when  compared  to  control  groups.  Such  a 
positive attitude can be analysed as a precursor of education retention and as reducing the risk 
of  drop-outs.  However  early  school  leavers  or  people  not  attracted  by  higher  education  are 
more  likely  to  be  found  among  disadvantaged  people.  As  seen  above,  both  the  share  of 
organisations targeting hard-to-reach groups and the share of individuals particularly at risk of 
exclusion  are  low  within  the  programme.  The  programme  does  improve  attractiveness  of 
participating higher education institutions but it cannot be deduced that it attracts people who 
would  not  have  otherwise  studied  at  higher  education  level.  Lastly,  the  review  of  selected 
projects  reveals  a  weak  alignment  between  the  transnational  partnerships  funded  and  these 
two key priorities. Though some potential does exist, the causality link is only indirect. 

As regards other ET2020 European benchmarks mentioned in the general objectives of the 
programme,  employability  of  young  people  has  also  improved  in  recent  past  (77%  are 

235 DG EAC's Annual Activity Report 2016; ex post Erasmus Impact Study (2014) 
236 ICF, tab. 6.33 
237  In  the  school  practitioner  survey,  61%  stated  that  they  improved  governance  and  quality  assurance  approaches.  In  the 
evaluation sample, 7% o HE staff who is in departments that teach about the EU has applied for Jean Monnet grant. 
238 The overall population of youth staff/organisations concerned is not determined (no comparator). 
239 The Europe 2020 strategy aims to reduce early school-leaving rates to a level below 10% (10.7% in 2016) and to enable at 
least 40% of 30-34 year-olds to have completed tertiary or equivalent education (39.1% in 2016). 

39 

 
 
                                                            
employed  i.e.  +1  percentage  point  from  2014  to  2015).  In  contrast,  other  impact  indicators 
more  related  to  social  inclusiveness  have  registered  deterioration  over  the  last  years,  with 
worse  levels  of  basic  skills  and  with  lower  rates  of  participation  in  youth  out-of-school  or 
physical activities240, which could justify a strengthened focus on social inclusion. 

5.1.3.  Transversal questions about effectiveness 

When considering all objectives together, 71% of respondents to the consultation thought 
that  the  programme  is  achieving  its  objectives  to  a  (very)  large  extent.  Education  and 
training  providers  who  responded  to  the  consultation  claimed  that  lack  of  funding  and 
insufficient  language  skills  are  among  the  barriers  hindering  the  achievement  of  Erasmus+ 
objectives. 

5.1.3.1. Most effective actions and fields  

The majority of respondents and countries241 believe that mobility is essential in ensuring the 
effectiveness  of  Erasmus+242.  Actions  cited  as  most  effective  by  agency  respondents  are 
mobility  ones243  and  strategic  partnerships,  with  the  latter  mentioned  with  more  mixed 
views244.  Moreover,  longer  term  mobility  actions  are  associated  with  stronger  results  for 
learners in beneficiary surveys and by a few countries245. For practitioners, what matters most 
seems to be participation itself, irrespective of the type of action246. Lastly, KA2 Alliances are 
perceived by agencies as being less effective for organisations247. It can meanwhile be noted 
that respondents tend to consider the actions closest to their interest as more effective. 

Comparing  the  relative effectiveness  of  Erasmus+  fields  cannot  be  done  without  keeping in 
mind  the  diversity  of  needs  they  address  and  funding  available  at  EU  level.  Nevertheless, 
30 years  of  Erasmus  with  a  larger  and  targeted  budget  allocation  makes  of  higher 
education248 the strongest field in effectiveness. In contrast, less funded or more recent fields, 
such as adult education or sport, not only miss the critical mass but also a similar strong focus 
to guarantee an equivalent level of effectiveness. School education and VET fields appear in 
an  intermediary  position,  indicating  at  this  stage  the  highest  potential  for  improving  the 
overall effectiveness of an EU programme in the future249. 

More  generally,  the  respondents  to  the  consultation  have  considered  factors  which  could 
maximize the effectiveness of the programme, as follows: 

240 See DG EAC's Annual Activity reports: https://ec.europa.eu/info/publications/annual-activity-reports-2015_en  
241 NRS, 5.2 and OPC, 3.3 for nuances per objective: from developing skills (86%) or promoting the European dimension 
(80%) to modernising systems (46%). ‘No opinion’ responses were excluded. Slightly more respondents who had ‘detailed 
knowledge’ of the programme thought that objectives were being achieved than those who had ‘some knowledge’.  
242 BEnl, IT, CY, HU, MT, SI, FI, UK, TR whereas a few see KA2 as more effective due to long-lasting effects (RO, FI). 
Lastly, DK, EE and NO do not found KA1 and KA2 different in effectiveness. 
243 Covering KA1 and KA2 mobility activities with the exception of the Student Loan Guarantee Facility 
244 ICF, tab. 6.41 to 6.44 for details and breakdown per sector; the key reason why KA2 strategic partnerships are appreciated 
in the school and adult learning sector is that they offer mobility options. When asked about system level effects, only ¼ of 
respondents cited actions under KA3. The majority cited KA2 types of actions. 
245 ICF, tab. 6.40; National reports of CY, NL, FI  
246 However two issues of method affect the breakdowns of preference per type of action for practitioners. Staff frequently 
participates in multiple types of actions and the sample sizes are rather small for some types of actions. 
247 Less clear when it comes to reporting about least effective actions, as few respondents opted to name some. 
248 Without including Jean Monnet activities and the Student Loan Guarantee Facility in this judgement 
249 Confirmed by several National Reports (BE, CZ, LT, PL, RO, FI, NO, CH) 

40 

 
 
 
 
 
 
                                                            
To what extent do you think the following topics need to be addressed to maximize the impact 
of any successor to the Erasmus+ Programme? 

Source: Erasmus+ OPC survey, 28 February – 31 May 2017, ICF analysis. Base indicated next to each topic 

According to the public consulted, the programme should be better funded, more inclusive 
and  user-friendly  while  its  opportunities  should  be  better  known  and  its  outcomes  more 
widely exploited and publicised. 

5.1.3.2. Visibility and dissemination of results 

Erasmus+ is very well-known across Europe and visible beyond. A majority of citizens have 
heard  about  Erasmus+  and  student  exchange  programmes  come  third  in  the  list  of  the  EU's 
positive results250. 

The  dissemination  and  exploitation  of  project  results  are  much  more  effective  and 
systematic with Erasmus+ compared to the predecessor programmes, albeit with nuances 
according to the geographical level considered251. The monitoring was much less performant 
in  2007-2013,  when  it  was  scattered  over  different  tools  covering  only  parts  of  previous 
programmes.  Good  practice  examples  are  used  to  raise  awareness  of  the  general  public;  to 
guide  potential  applicants;  to  inform  the  work  of  practitioners,  researchers,  social  partners, 
organisations  and  policy  makers.  This  new  approach  has  been  recognised  in  the  public 
consultation252  and  by 
transparency, 
accountability and visibility254, drawing on the lessons learnt from the interim evaluations of 
predecessor programmes. 

the  experts253.  It  addresses 

the  objectives  of 

250 Respectively 53% and 25% of respondents to the standard Eurobarometer. Visibility is evidenced on social media. 
251 Sources: Desk research, KI interviews, case studies, expert panel assessment and agency survey; ICF, 6.9 
252  Number  of  OPC  respondents  mentioning  the  empowerment  of  policy-makers  through  dissemination  when  asked  to 
indicate what changes happened for systems in each sector: SE (31); VET(41); AE (34); ICF, Tab. 5.1 
253 A conclusion from the expert panel assessment workshop is that the branding enhances the attractiveness of outputs. 
254  77%  of  agency  respondents  strongly  or  rather  agree  that  the  strategy  has  contributed  to  better  communication  of  the 
projects results and 58% that it would have met its objective of transparency and accountability. 

41 

 
 
 
                                                            
At  EU  level,  the  establishment  for  the  first  time  of  an  overarching  dissemination  and 
exploitation  strategy255  with  regularly  reviewed  indicators  and  a  single  project  results 
platform256 represent clear improvements in comparison to the past257. Specific steps have been 
taken  both  by  the  Commission  and  the  Executive  Agency  to  ensure  that  the  exploitation  of 
projects’ results effectively serves to inform policy making258. 

The project results platform offers a common and accessible space for disseminating most 
project results whilst also highlighting good practices and success stories (including from LLP 
and  Youth  in  Action).  Its  value  in  comparison  to  the  past  lies  in  structured  and  selected 
material.  Insights  gained  from  EU  interviews  suggest  that  the  platform  is  actually  used  to 
inform the work of policy makers at EU level. Content-wise, the main limitation is the lack of 
complementary  external  reference  materials  that  could  inspire  users  and  a  lack  of  success 
stories in certain sectors (e.g. adult learning) due to the insufficient quality of reports. 

At national level, the assessment is also positive about the strategy, setting clear rules for the 
selection  of  good  practice  examples.  However,  the  evidence  of  the  exploitation  of  project 
results  by  policy  makers259  and  the  effective  engagement  of  the  latter  when  they  are  not 
included  in  the  project  itself  is  not  clear.  The  vast  majority  of  National  Agencies  mainly 
disseminate  examples  of  good  practices  to  potential  applicants  or  beneficiaries,  and  only 
rarely to inform policy making bodies. Moreover, though valued by over half of respondents 
as shown in the graph below, 37% of agency respondents do not agree that the dissemination 
platform is user-friendly. This sentiment, more mixed than at EU level, concurs with insights 
gained from national interviews and reports260. 

Source: ICF experts’ survey 

255 DG Education, Youth, Sport and Culture, March 2015; ICF, 6.9, table for a description per level of intervention 
256 Initially named VALOR following the LLP interim evaluation and building on predecessors (ADAM, EVE), the Project 
Results Dissemination Platform acts as a repository for all funded projects under Erasmus+ and some of those funded under 
its predecessors, highlighting best practices and success stories in terms of policy relevance, communication potential, impact 
or design: http://ec.europa.eu/programmes/erasmus-plus/projects  
257 Under Erasmus+, dissemination channels notably include in addition to the project results platform, DG EAC or EACEA's 
websites,  social  media,  collaborative  platforms  (eTwinning,  School  Education  Gateway,  EPALE,  Youth  Portal),  videos, 
newsletters, electronic and paper-based publications, etc. A distinct evaluation of this strategy was on-going in October 2017 
and preliminary evidence seems to confirm these findings. 
258  E.g.  the  project  results  platform  enabled  in  2015  to  retrieve  160  projects  that  supported  integration  of  asylum  seekers. 
These were used for policy communication. 
259  Only  6%  of  agency  respondents  actually  strongly  agree  with  the  statement  that  the  programme  results  are  adequately 
exploited for policy purposes. 
260 BEnl, IE, MT, FI. The search function of the platform, for example, was described as difficult to use. 

42 

 
 
 
                                                            
Furthermore National Reports refer not only to the dissemination platform, but also to other 
dissemination  channels  (e.g.  events,  websites,  social  media).  Despite  all 
tangible 
dissemination  efforts,  some  observe  that  the  results  of  projects  are  seldom  further 
exploited261.  While  this  can  be  due  to  factors  that  are  external  to  the  programme  itself  (e.g. 
professionals’  lack  of  time)  some  regret  a  lack  of  concentration  of  resources  on  the  most 
interesting  projects262  and  an  insufficient  involvement  of  national  authorities  and/or 
municipalities in their dissemination263. 

At  project  level,  the  expert  panel  confirmed  that  dissemination  is  effective  in  90%  of  the 
cases  they  assessed264.  The  platform  is  also  of  use  for  people  looking  for  partners. 
Interviewees  and  participants  were  aware  of  the  dissemination  and  exploitation  strategy 
requirements, but often perceived these as burdensome. Despite the standardised processes in 
place (except in the case of Jean Monnet activities265), project leaders often do not have the 
capacity to effectively exploit results. This might explain why, according to OPC respondents, 
the dissemination of results is one of the aspects of Erasmus+ that should be improved in the 
future. 

261 BEnl, DE, IE, HU, MT, NL, PL, PT, RO, SI, SK, FI, FY, NO, TR 
262 BG, CY, HU, FI, SE, FY, TR 
263 SI, IS 
264 The vast majority of projects reviewed: had all or some projects’ products disseminated online (90 out of 100) and none of 
the  projects  without  online  outputs  were  funded  under  the  current  programme;  93  had  several  (55)  or  at  least  a  few 
dissemination activities (38). Outputs were made available fully (57) or partially to the target group(s) and the wider public in 
most cases. 85 outputs were assessed as of high to good quality (41 and 39 respectively) and generally considered relevant 
and coherent, but nevertheless bringing rather limited innovation. 
265  Even  though  there  is  a  requirement  of  disseminating  the  JM  results  to  audiences  beyond  the  stakeholders  directly 
involved, no specific targets are defined.; ICF/JMO 2.1 

43 

 
 
 
 
 
                                                            
5.2.  Relevance 

The assessment of the relevance266 of Erasmus+ is generally positive267 in regard to both its 
alignment with policy priorities and its adaptability to socio-economic needs. The relevance 
of  the  programme,  compared  where  appropriate  with  its  predecessors,  has  been  assessed 
against 8 evaluation questions that overall looked into the extent to which Erasmus+ is: 

responsive to the needs of stakeholders, sectors and society at large 

  aligned with EU and national policy priorities 
 
  visible 
  suitable for attracting different target audiences, including hard-to-reach groups 

The  vast  majority  of  interviewees  and  most  of  OPC  respondents268  consider  that  the 
programme shows a high degree of alignment with key EU priorities namely as set in the 
Europe  2020  Strategy,  ET2020  Joint  Report,  the  Youth  Strategy,  and to  some  lesser  extent, 
the Union's external action, including its development objectives, and the EU Work Plan for 
sport269. 

Erasmus+ is strongly relevant to the rationale of the European project: it fosters the intra-
European  integration  through  learning.  For  instance,  Erasmus+  and  its  predecessors  have 
played  an  important  role  in  supporting  the  Member  States  and  beyond  in  implementing  the 
Bologna  process  with  the  introduction  of  the  ECTS.  Had  it  not  been  for  Erasmus+,  much 
fewer students would have benefited from learning mobility or would have been much more 
oriented  towards  non-EU  English-speaking  countries,  according  to  interviews270.  The 
consequences  of  a  scenario  where  Erasmus+  would  not  exist  are  elaborated  further  in  the 
section on its European added value below. 

The international higher education dimension of the evaluated programmes has contributed to 
achieving  socio-economic  development  objectives.  The  EU  programmes  for  international 
cooperation  in  higher  education  have  contributed  to  fostering  regional  integration  as  a  key 
objective in the EU’s relations with partner regions and supporting EU policies in the field of 
legal migration and mobility271. 

266 Relevance analysis looks at the relationship between the needs of participants and problems in society and the objectives 
of a programme and hence touches on aspects of design. 
267 Consensus among key informants, at EU and national level, concurring with findings of the agency and experts surveys, 
the OPC and the synthesis of the NAUs reports; ICF 4, NRS 3, OPC 3.1 and 5.1.3 
For  instance,  agencies’  respondents  perceive  a  clear  alignment  between  the  programme  and  the  EU  priorities,  e.g.  in 
improving  the  quality  of  education  and  training  (74%  of  respondents  perceiving  this  as  strong  alignment),  increasing  the 
number  of  those  who  benefit  from  mobility  (70%),  opening  up  education  and  training  to  the  wider  world  (68%).  Similar 
insights are also available for the youth field: support to volunteering and youth organisations (83% of respondents rated it as 
‘strong’),  engaging  participation  of  young  people  in  policy  making  (55%),  non-discrimination  and  access  to  social  rights 
(52%). Conversely, least alignment is found in the following areas: increasing cognitive skills in reading, math and science 
(28%  whilst  24%  claimed  they  ‘don’t  know’),  increasing  higher  education  attainment  (44%  whilst  33%  ‘don’t  know’), 
capacity building in higher education in partner countries (39%, whilst 39% ‘don’t know’), access to quality employment – 
youth strand (28% whilst 14% ‘don’t know’), capacity building of youth sector in Partner countries (34% whilst 14% ‘don’t 
know’). ICF, Fig. 4.1 and 4.2 
268 96% (n = 606) of OPC respondents agreed that Erasmus+ is well aligned with EU policies and priorities. 
269 Sport pilot projects launched in 2016-2017 (dealing with health-enhancing activities or refugees) might help mitigate this 
lower perception of relevance. 
270  ICF,  tab.4.6  -  Illustrative  examples  of  perceived  (mis-)alignment  of  Erasmus+  with  national  policies  (KIIs  and  case 
studies) 
271  Many  partner  countries  which  are  not  members  of  the  EHEA  (European  Higher  Education  Area)  are  influenced  by 
Bologna principles through Erasmus+. See, Evaluation of the EU Development Cooperation Support to Higher Education in 
Partner Countries 2007–2014, Particip, LeA, ECDPM, Ecorys (2017), already mentioned. 

44 

 
 
                                                            
Other  evidence272  and  the  consultation273  reveal  in  addition  that  the  alignment  with 
European  policy  priorities  is  stronger  with  Erasmus+  than  with  its  predecessors,  as 
shown below. 

Share of projects that focus on issues that are high on the European policy agenda 

40%

30%

20%

10%

0%

f
o
e
g
a
t
n
e
c
r
e
P

s
t
n
e
d
n
o
p
s
e
r

32% 

36% 

5% 

10% 

2% 

Very high

High

Average

Low

Very low

38% 

14% 

I don't
know

Erasmus+ (n = 977)

Predecessor programmes, 2007 - 2013 (n = 902)

Source: ICF, experts’ survey 

On  the  one  hand,  the  review  of  topic  coverage  of  funded  projects  undertaken  for  this 
evaluation  and  the  expert  survey  confirm  that  these  align  to  a  high  extent  with  EU-level 
general priorities, such as promoting lifelong learning, equity or social cohesion. For instance, 
73%  of  expert  respondents  as  opposed  to  52%  in  the  case  of  predecessor  programmes, 
estimate  that  a  high  share  of  the  funded  projects  which  they  assessed  align  with  EU-level 
priorities.  This  is  also  consistent  with  the  data  analysis  of  cooperation  projects,  albeit  with 
variations in topic prioritised before and after 2014, as shown in the table below: 

First 10 topics of cooperation projects under the Erasmus+ and LLP 

LLP  

Erasmus+ 

1 

2 

3 

4 

5 

6 

7 

8 

EU citizenship 

Teaching and learning of foreign 
languages 

Creativity and culture 

Environment / sustainable 
development 

ICT - new technologies - digital 
competences 

Health and wellbeing 

Reinforcing links between education 
and working life 

18% 

17% 

7% 

7% 

Comparing educational systems 

5% 

22% 

ICT - new technologies - digital 
competences 

18% 

Curricula/teaching methods 

Creativity and culture 

Teaching and learning of foreign languages  11% 

15% 

Skills and labour market issues 

10% 

20% 

18% 

11% 

EU Citizenship 

International cooperation, international 
relations, development cooperation 

Entrepreneurial learning - entrepreneurship 
education 

9% 

9% 

9% 

272 Stronger alignment than its predecessors according to agencies' respondents e.g. in opening up education and training to 
the  wider  world  (64%  of  programme  agencies’  respondents  perceive  stronger  alignment),  enhancing  creativity  and 
innovation,  including  entrepreneurship  at  all  levels  of  education  (61%),  increasing  the  number  of  those  who  benefit  from 
mobility (56%), or in the youth field strengthening cooperation with policy makers (58%). Results are more balanced e.g. in 
improving  the  quality  of  education  and  training  (52%  as  opposed  to  44%  who  feel  that  alignment  is  the  same),  capacity 
building  of  higher  education  in  partner  countries  (51%  as  opposed  to  43%),  increasing  employability  of  young  graduates 
(51% as opposed to 40%) or in increasing the transparency of qualifications and skills (48% as opposed to 48%). Conversely, 
areas  where  alignment  is  felt  to  be  the  same  than  in  the  past  by  most  respondents  include  increasing  cognitive  skills  in 
reading, mathematics and science (60% versus 34%), facilitating access to education and training for all (57% versus 32%), 
increasing  higher  education  attainment  (58%  versus  37%),  reducing  early  school  leaving  (53%  versus  43%),  increasing 
participation  in  lifelong  learning  (22%  versus  55%),  and  in  the  youth  field:  non-discrimination  and  access  to  social  rights 
(12% as opposed to 81%), access to quality employment (24% as opposed to 68%), access to quality education for all young 
people (27% as opposed to 68%).; ICF, 4.2.1 and ICF, annex 4  
273  83%  (n  =  468)  of  OPC  respondents  (strongly)  agreed  that  Erasmus+  has  stronger  emphasis  on  promoting  youth 
employment compared to predecessor programmes. 

45 

 
 
 
 
                                                            
 
 
 
9 

Natural Sciences 

4% 

Intercultural/intergenerational education 
and (lifelong)learning 

7% 

10 

Sport 

Source: 
In red: significant decrease under Erasmus+. NB: most projects of the sport strand are centralised, so do not appear here. 

calculations 

based 

total 

ICF 

the 

on 

of 

4% 
Epluslink 

Health and wellbeing 
LLPLink274 
(% 
and 

5% 
decentralised 

projects) 

On the other hand - when analysing projects more in-depth, beyond statistical data - some key 
challenges  are  only  addressed  marginally  in  practice.  The  extent  to  which  the  projects 
reviewed  align  with  more  specific  priorities  (e.g.  early  school  leaving,  higher  education 
attainment,  adults’  participation  in  adult  learning,  etc.)  is  not  clear  enough.  For  instance, 
across  the  sampled  projects  in  higher  education,  none  focused  on  the  attainment  headline 
target275 and in school education, only a very small number dealt with early school leaving276. 
There is still room for improvement in the number of activities in topics on key competences 
and basic skills, even though these have been a priority in many calls for project applications. 
Lastly,  in  the  sport  field,  respondents  at  national  level  often  felt  a  gap  between  broad  EU 
priorities and the need for more customised ones277. 

Most interviewees also consider that Erasmus+ has proven to be flexible in adjusting to new 
emerging  EU-level  challenges278  (e.g.  refugee  crisis,  intolerance  or  violent  radicalisation 
were  often  mentioned),  particularly  through  its  annual  work  programming279.  Several 
National  Reports  value  as  well  the  flexibility  of  Erasmus+  objectives  to  address  emerging 
challenges280. 

Examples of flexibility and relevance to the EU policy agenda 

Following  the  Paris  Declaration  on  promoting  citizenship  and  the  common  values  of  freedom, 
tolerance  and  non-discrimination  through  education281,  the  prevention  of  radicalisation  through 
education, youth outreach and sport activities was identified as a priority within the updated EU Agenda 
on Security. More than EUR 200 million was allocated to support Erasmus+ projects fostering inclusion 
and promoting fundamental values, on top of a specific EUR 13 million call launched in March 2016 to 
identify and spread best practices. 

In  the  context  of  the  refugee  crisis  of  2015,  Erasmus+  started  as  of  2016  to  provide  online  language 
assessment and courses for newly arrived third country nationals, through the online linguistic support 
(OLS) in dedicating 100.000 additional licences for a period of three years. 

274  Data  included  for  all  actions  for  which  ‘topic’  information  was  available,  including  LLP  (COM06,  COM07,  COM13, 
GRU06, LEO04) and Erasmus+ (KA200, KA201, KA202, KA203, KA204, KA205, KA219) mobility actions 
275 Europe 2020 targets of "reducing the share of early leavers of education and training to less than 10%" and "increasing the 
share of the population aged 30 to 34 having completed tertiary or equivalent education to at least 40%" by 2020 
276 Nonetheless it is the third most popular topic in the school field in the period 2014-2016: 553 projects were funded in total 
under the topic "Early school learning/combating failure in education" for EUR 84 million. 
277 ICF, tab. 4.6, quotations from KIIs: "Erasmus+ seems to focus more on scientific issues or general issues in sports"; "At 
the grassroots level, there are continuing issues around retaining volunteers to support local clubs and local organisations (not 
reflected in Erasmus+ priorities)" 
278  Particularly  positive  among  EU  level  stakeholders  in  school  and  higher  education  sectors,  whilst  perceptions  are  more 
mixed among EU-level stakeholders in the sport sector and among national stakeholders. For instance, the good alignment of 
the  programme  with  the  Modernisation  of  Higher  Education  communication,  the  New  Skills  Agenda  or  the  increased 
attention paid to work-based learning (ErasmusPro) were often valued. 
279  Although  challenges  are  not  always  explicitly  stated;  for  instance,  the  2017  Programme  guide  briefly  presents  the 
European Solidarity Corps (part describing EVS) as a means to help "resolve challenging situations" without specifying the 
latter. 
280 EE, IE, NL, SI, NO, etc 
281 http://ec.europa.eu/education/news/20160316-paris-declaration-education_en 

46 

 
 
 
 
                                                            
Meanwhile, 79% of OPC respondents indicated that in their view there are no other priorities 
or actions that Erasmus+ should address. For the remaining share of respondents, the main 
suggestions282 included in descending order  of frequency: i) promoting European citizenship 
and  civic  education;  ii) fostering  inclusion  and  diversity,  especially  the  integration  of 
migrants;  iii) linking  education  more  strongly  with  enterprise  and  promoting  international 
internships. 

In contrast, the context has changed radically since the Jean Monnet programme was founded. 
There  is  now  substantial  volume  of  teaching  and  research  about  the  EU,  particularly  in  EU 
countries, irrespective of Erasmus+ support. Because of their focus on higher education, the 
evaluation  shows  that  the  relevance  of  Jean  Monnet  activities  could  be  enhanced  by 
focussing more on other segments of the learning continuum such as schools and VET. Given 
the  low  turnout  in  the  European  Parliament  elections283  and  the  lack  of  trust  in  the  EU 
institutions284, there is a need to strengthen Europeans’ understanding of the EU. This is the 
underpinning  challenge  that  Jean  Monnet  aims  to  address.  Overall,  only  56%  of  Europeans 
state that they understand how the EU works - a number that has nevertheless improved by 9 
percentage  points  since  2004.  However,  Jean  Monnet  activities  target  a  population  that  has 
been tested as already more knowledgeable about the EU than non-beneficiaries. One third of 
staff  respondents  dedicate  substantial  share  of  their  teaching  to  students  outside  EU-related 
fields of study. However, most of those reached remain students who would still have studied 
European integration285. 

The extent to which all the ambitious objectives of the programme are achievable over time is 
also sometimes questioned. Some stakeholders call for greater emphasis on the promotion 
of  social  inclusion.  In  the  aftermath  of  the  2008-2009  economic  crisis,  Erasmus+  was 
primarily focused on economy-related priorities. Since 2016, a shift towards social inclusion 
is  observed  in  its  implementation.  Responses  seem  to  indicate  that  the  social  inclusion 
objective could be explicitly defined and go together with a higher and more visible budget to 
achieve  a  more  tangible  impact286.  However,  views  remain  mixed  on  this  issue.  Many 
stakeholders, not only from the youth sector but also from education and training, believe that 
too much emphasis is put on employability to the detriment of the cultural value of education 
or  the  youth  sector;  whereas  some  other  stakeholders  insist  on  the  continued  relevance  of 
employability.  Some  National  Reports  refer  to  needs  that  could  be  further  addressed, 
including  i) the  objectives  connected  to  the  Paris  Declaration  and  prevention  of  violent 
radicalisation287; ii) traineeships to enhance labour-market inclusion288; iii) transversal, social 
and  personal  skills  –  hence  aligning  well  with  data  collated  by  the  external  evaluation  at 
national level. 

282 For more suggestions see OPC 3.1 and NRS 3.1 
283 http://www.europarl.europa.eu/elections2014-results/en/turnout.html  
284 Standard Eurobarometer 86 – Wave EB86.2 
285 ICF/JMO, 2.1 and fig.2.1 and 2.3 
286 Several interviewees welcome the KA3 call on social inclusion (13 million EUR in 2016) but a quarter of them see it as 
too  modest  to  effectively  achieve  tangible  impact.  Other  social  inclusion-related  measures  (e.g.  social  inclusion  emphasis 
recently  put  on  Erasmus+  KA2  transnational  partnerships)  were  very  scarcely  mentioned.  ICF,  4.6  about  the  efforts  to 
strengthen the social inclusion dimension of Erasmus+ since 2016 
287 FR, LU, SK 
288 BEnl, FI, IT, TR. In particular, VET-higher education permeability was felt insufficiently addressed. There is however no 
consensus  on  labour-market  inclusion.  CZ,  EE,  ES,  HR,  IT,  LT,  PL  call  for  tighter  link  with  companies.  A  number  of 
interviewees expressed the view that Erasmus+ is probably not the best instrument for targeting the labour market.  

47 

 
 
 
 
                                                            
The alignment of Erasmus+ objectives with national policies is almost equally recognised, 
by  most  of  respondents289. Overall,  a vast majority of experts  surveyed  estimate that a high 
share of the projects funded align very highly to fairly with national level priorities290. Experts 
felt that there has been an improvement on this matter between Erasmus+ and its predecessor 
programmes, as illustrated below. 

Share of projects that focus on issues that are high on the policy agenda of my country 

50%
40%
30%
20%
10%
0%

f
o
e
g
a
t
n
e
c
r
e
P

s
t
n
e
d
n
o
p
s
e
r

40% 

25% 

3% 

13% 

4% 

Very high

High

Average

Low

Very low

39% 

16% 

I don't
know

Erasmus+ (n = 983)

Predecessor programmes, 2007 - 2013 (n = 906)

Source: ICF experts’ survey 

Nevertheless  views  from  national  interviewees  are  more  mixed291,  with  organisations  being 
more complimentary than policy makers292 or agencies’ respondents293. The most noticeable 
difference  is  found  in  the  sport  field  where  no  stakeholders  and  only  one out  of  4  policy 
makers see Erasmus+ as strongly relevant to solving national challenges. 
Most of the national authorities294 consider as well that Erasmus+ decentralised actions295 are 
relevant to  country  needs, although fewer countries refer to national priorities  in the adult 
education, sport and Jean Monnet sectors. Furthermore some countries ask for more flexibility 
to allow the programme to adapt better to local needs296. 

Examples of relevance to national needs or need for more flexibility 

Relevance to national needs 

Need for more flexibility 

- 

- 

In  Denmark,  the  internationalisation  of  education  is  a 
national  priority.  The  Danish  higher  education 
institutions  receive  an  “internationalisation  rate”  for 
every  student  they  receive  or  send,  including  under 
Erasmus+. 

Ireland  explains  that  Erasmus+  growing  emphasis  on 
youth employment has aligned the programme with the 

-  Hungary  and  Lithuania  require  more 
flexibility  to  local/national  needs  in  the 
youth sector. 

-  Germany  considers  that  call  priorities 
should  be  regularly  aligned  with  societal 
realities  in  a  more  targeted  manner.  It 
requests  the  possibility  to  set  national 

289 86% (n = 527) of OPC respondents (strongly) agreed that Erasmus+ is well aligned with national policies and priorities. 
Interviews, agencies and experts’ surveys; for instance most of experts consider that the projects funded generally align well 
(from high to average alignment) with both EU and national policy priorities. 
290 68% vs. 46% under the predecessor programmes (p.m. Erasmus+ aligned with EU priorities: 73%); ICF, fig.4.11. 
291 Examples of alignment between national priorities and the programme were reported by at least half of interviewees and 
by at least one type of interviewee per case study. ICF, tab. 4.5 shows that Erasmus+ objectives or actions align well with the 
vast majority of policy priorities/needs commonly reported per sectors by KIIs. 
292 Except in the school education sector, where the share of policy makers with a strong positively feeling of the alignment 
was conversely slightly higher. ICF/Technopolis, fig. 4.5 to 4.10 for breakdown per sector. 
293 60% agencies’ respondents disagree or strongly disagree with the statement that Erasmus+ is better aligned with national 
priorities than its predecessor programmes. 
294 BEde, BEfr, BEnl, BG, CZ, DE, EE, IE, EL, ES, FR, HR, IT, LT, LU, HU, MT, NL, AT, PL, PT, RO, SI, SK, SE, UK, 
FY, LI, NO, TR; NRS, tab 3.1 for details per sector 
295 National Reports were not required to cover centralised actions (e.g. sport field, Jean Monnet, etc). 
296 DK, DE, IE, LV, LT, HU, SI, SK, etc 

48 

 
 
 
                                                            
 
 
 
Irish national agenda allowing important synergies. 

priorities to respond better to the needs of 
national target groups. 

Source: ICF 4.1, NRS 3.1 

Most position papers collected during the consultation highlight that Erasmus+ is one of the 
EU’s most successful programmes and that it remains highly relevant. The majority of the 
consultation respondents (59%) thought that the current Erasmus+ objectives are extremely 
relevant  to  the  current  challenges  and  needs,  especially  as  regards  the  development  of 
skills and competences, the European dimension of education and youth activities, languages 
and intercultural awareness. In contrast, fewer respondents saw the relevance of the objectives 
regarding the international strand, Jean Monnet, policy support and sports297. 

Relevance  of  the  programme  can  also  be  assessed  in  relation  to  its  capacity  to  fund 
innovative projects, considering the needs for innovation in a context of global competition 
between economies and particularly between education systems. While evidence shows that a 
wide range of applicants are well targeted, the innovative capacities are rather limited298. The 
experts’  survey  indicates  that  the  projects  are  most  frequently  (42%)  moderately  innovative 
with a relatively small share that was considered as highly or very highly innovative (15% and 
2%,  respectively).  At the same time, similarly to the increased  quality of applications299, an 
improvement  can  be  seen  in  terms  of  innovation  when  comparing  Erasmus+  with  its 
predecessor programmes. This could further improve at the final evaluation stage when KA2 
Alliance-type projects are sufficiently advanced or have been finalised to be evaluated. 

More generally, key evidence suggests that the programme is  well suited to attract a wide 
range of target audiences across the fields it covers. It is particularly suitable for reaching 
out  to  learners  with  different  profiles300,  with  one  noticeable  exception:  disadvantaged 
groups, as elaborated below301. 

A  deeper  analysis  shows  that  the  needs  of  different  stakeholders  are  generally  well 
addressed,  albeit  with  variations  according  to  sector,  organisation  size  and  level  of 
intervention.  The  majority  of  the  interviewees  estimate  that  the  programme  strongly 
contributes  to  addressing  the  socio-economic  needs  of  the  learners302.  Monitoring  survey 
highlights that the motivations of learners are fairly well aligned with programme objectives 
and  related  to  key  competences.  Foreign  language  skills  is  the  most  mentioned  need  in  all 
sectors.  In  VET,  learners’  motivations  for  participation  are  particularly  well  aligned  with 
programme objectives on key competences, including technical skills (80%). At least 50% of 
agencies’  respondents  believe  that  Erasmus+  meets  the  needs  of  target  groups  including 
practitioners303 better than its predecessor programmes. They all agree that learners’ needs in 
higher education and VET are met by both KA1 and KA2, but the needs of school pupils are 
partially  or  weakly  met  by  KA1,  despite  the  fact  that  teachers  take  part  in  the  action.  On  a 

297 OPC 3.1; ‘extremely relevant’ objectives for the Programme include: skills and competences of individual learners (73%); 
the  European  dimension  of  education  and  youth  activities  (70%);  the  quality,  innovation  and  internationalisation  in 
education, training and youth organisations (68%); the teaching and learning of languages and intercultural awareness (68%) 
the skills and competences of practitioners (62%); the Union's external action (54%); excellence in teaching and research in 
European integration (48%); policy reforms at national level (44%); sports objectives of Erasmus+ (33%) 
298 ICF, 6.13 and 6.18. The share of innovative methods considered low or very low was 24% and 5% respectively 
299  ICF,  fig.4.16  :  according  to  the  expert  survey,  improvement  in  application  quality  is  found  between  Erasmus+  and  its 
predecessor programmes; currently, almost one third of the applications is considered as of high quality by experts (18% for 
predecessor  programmes),  whereas  most  frequently  (42%)  of  average  quality  (37%  for  predecessors).  Caution:  37%  of 
experts have no opinion as for predecessor programmes. 
300 Beneficiary survey, ICF 4.4 and 6.3 
301 Drawing on the definition set out in Erasmus+ Programme guide (p.9) 
302 ICF, 4.2 and 4.3 
303 ICF, 4.3.3 about practitioners' needs 

49 

 
 
                                                            
qualitative note, many examples in interviews suggest strong relevance of the programme to 
learners’  needs304.  Lastly,  as  detailed  above  in  the  section  on  effectiveness,  participants’ 
satisfaction  is  very  high  (81%  to  100%  claiming  being  very  or  rather  satisfied  with  their 
mobility  experience). In contrast,  interviewees have  more mixed views on the programme’s 
ability to meet the needs of organisations. The needs of small organisations especially are not 
met sufficiently305. Lastly, agencies’ staff consider that the needs of schools, VET and youth 
systems are addressed by KA3 (highly relevant), while the relevance for higher education and 
adult learning systems would be more moderate. 

The discontinuation of a small number of (Lifelong Learning Programme and Youth in 
Action)  actions  had  only  a  limited  impact  on  the  extent to  which  stakeholders’  needs  were 
addressed, except in two areas:306 

  many  interviewees  claimed  that  pupil  mobility  (and  to  a  smaller  extent  adult  learner 
mobility) should return to a KA1-type of funding, instead of under the more complex 
coverage of KA2307.  

  discontinued  local  youth  initiatives  previously  funded  by  Youth  in  Action  were 
considered  more  accessible  than  current  Erasmus+  transnational  youth  initiatives308. 
The former enabled young people to initiate their own projects, whereas the latter are 
seen as too complex for informal groups of young people, especially  NEETs (young 
people not in Education, Employment, or Training), to apply309. 

Knowledge about Erasmus(+) 2009 vs 2016 

Source: Standard Eurobarometer 86 and Special Eurobarometer survey 316 

All  findings  reveal  that  the  programme  is  perceived  as  increasingly  visible  and 
attractive310.  Erasmus+  is  well  known  to  the  audience  in  general  (see  fig.  above)311  and 
especially to the young generations. Its large visibility also, including outside the EU, stands 
out  when  compared  with  that  of  other  EU  programmes  such  as  Horizon2020,  Europe  for 
Citizens or EU Aid Volunteers312. It can also be positively noted that the degree of repeated 
participation appears to be low for learners (less than 10%), whilst it is expected for the more 

304 ICF, 4.3.2 and 4.3.3, text boxes with examples of responsiveness to needs reported by interviewees (case studies) 
305 ICF, 4.3.4 about organisations' needs 
306 Interviews at national level and National Reports 
307 National reports of DK, FR, IT, CY, FI, CY,UK, etc; ICF, 4.3, including illustrative quotations from KIIs  
308 BEnl, HU, MT, CH 
309 ICF, 4.3, including illustrative quotations from KIIs  
310 Social media analysis, Agencies survey (76%), National reports, Eurobarometer survey and outputs to Erasmus+ 30 years 
anniversary campaign. See also section on coherence below. 
311 Less visible in the sport community (e.g. HU) or in reaching dual VET (DE, AT) 
312 ICF 4.5, 6.9 and annex 19 (not published). See Annex 3 about social analysis performed 

50 

 
 
 
 
 
                                                            
permanent population of staff313. It is nevertheless found that the extent to which Erasmus+ 
reaches  out  to  potential  organisations  varies  across  the  different  fields.  The  visibility  of  the 
Jean Monnet activities could, in particular, be stronger314. A few National Reports315 link the 
lower participation of primary education and early childhood education and care in Erasmus+ 
to  a  lower  awareness  of  the  programme  in  these  sectors.  Some  National  Reports  make 
suggestions on how to increase visibility316. 

Furthermore,  a  number  of  barriers  to  participation  are  identified  limiting  the  extent  to 
which the programme addresses the needs of stakeholders. At the individual level, financial 
barriers317  and  administrative  burdens318  are  more  often  reported  than  are  geographical 
disparities319.  At  the  organisation  level,  the  programme  offers  less  room  for  small  scale 
projects than in the past, reducing the extent to which it reaches grassroots level organisations 
in all their diversity320. Evidence suggests that smaller sized organisations often have neither 
the  experience,  nor  the  resource  capacity  that  a  successful  application  requires321.  Many 
stakeholders  believe  that  funding  has  become  too  highly  competitive,  making  it  only 
accessible  to  the  most  experienced/largest  organisations,  preventing  access  to  potential 
newcomers,  hence  reducing  the  number  of  those  who  may  benefit  from  a  European 
experience322. 
Despite the specific  attention  paid  to  social  inclusion323  with  varied results  across  sectors324 
and  to  widening  the  participation  of  disadvantaged  target  groups325,  recognised  by  a 
majority  of  interviewees326  and  30%  of  agencies,  evidence  suggests  that  there  is  room  for 
improvement327.  The  share  of  disadvantaged  groups  represents  almost  10%  of  KA1 
participants (almost 25% in the youth field as well), as detailed in the section on effectiveness 
above,  due  to  the  programme’s  specific  efforts  in  the  area  (see  fig  below).  Erasmus+  has 
offered new experiences abroad to many people who would have otherwise not been able to 

313  Based  on  the  monitoring  data,  the  vast  majority  of  VET  learners  (95%)  and  students  in  international  higher  education 
mobility (91%) are first time participants in EU funded mobility actions (Erasmus+ or LLP). This is a positive finding as the 
benefits  of  mobility  on  attitudes  are  likely  to  diminish  with  repeated  participation.  Concerning  staff  mobility,  while  the 
majority of staff in schools, VET, adult learning and international higher education are first time participants, most staff in 
youth and higher education are repeated participants. 
314  Even  among  higher  education  practitioners  who  teach  about  the  EU,  only  55%  have  heard  about  the  Jean  Monnet 
activities; ICF/JMO, 2.5 
315 DE, RO, FI, etc 
316 NRS 3.3: SI, IS 
317 HU, PT  
318 Participants expressed concerns about complex administrative procedures related to application and reporting: BEnl, BG, 
CZ, DK, DE, EE, EL, ES, FR, HR, IT, CY, LV, LT, LU, HU, MT, NL, AT, RO, SK, FI, SE, UK, IS, FY, LI, NO, PL, PT, 
CH, TR; see section on efficiency below. 
319 Few countries report issues of unequal participation by regions. FR observes a reduction in the participation of overseas 
territories. BG and ES mention the specific needs of small towns and rural areas. 
320 EE, IE as regards vulnerable groups; ES, EL, PL, FY concerning newcomers 
321 Large number of national interviews; also denoted across the National Reports BE, BG, CZ, DE, FR, HR, CY, LV, LT, 
RO, SK, FI, NO, etc 
322  Both  data  programme  analysis  and  the  expert  survey  identifies  major  unbalances  in  competition  for  grants  among  the 
different sectors; ICF, 4.4 and tab. 4.10 for success rates for Erasmus+ per sector 
323 Interviewees most often referred to the call on policy experimentation (KA3) on social inclusion across sectors. 
324 In the field of adult education, only 15% of agency respondents (lowest share) strongly agree that the programme offers 
more opportunities for disadvantaged target groups than in the past. In contrast, the highest share is found in youth (44%). 
325 ICF, 4.6: 57% of agency respondents perceive the actual participation of disadvantaged groups in Erasmus+ as higher or 
the same compared to other national or international programmes. It should be mentioned, however, that for 36% of agency 
respondents, the absence of relevant statistical data was an obstacle to provide any estimates on this issue. IFC, Annex 10 for 
the most frequently targeted disadvantaged groups as per the expert panel assessment of Erasmus+ project outputs. 
326  KIIs  (national  level)  assessed  the  relevance  of  Erasmus+  to  hard-to-reach  or  specific  disadvantaged  groups  positively 
almost in all sectors 
327 ICF 4.3 and 4.6, similar findings from NRS 3.4 and OPC 3.1 

51 

 
 
                                                            
afford it. In countries where there is no mobility programme targeting disadvantaged group, 
Erasmus+ is particularly appreciated. Where programmes supporting mobility do exist, they 
do  not  have  such  a  priority  or  –  if  they  do  –  they  do  not  promote  it  to  such  an  extent  as 
Erasmus+ does328. 

Perceived level of participation of disadvantaged groups (Erasmus+ compared to other 
comparable national/international programmes/initiatives 

30% 

27% 

36% 

7% 

40%

30%

20%

10%

0%

Higher level 

Equivalent 

Lower level 

No opinion/data/n.a. 

Source: ICF survey with National Agencies (n = 117) 

However, according to a vast majority of National Reports329 and a shared perception among 
all  groups  of  respondents,  young  people  from  disadvantaged  backgrounds  and  with 
special  needs  are  not  sufficiently  addressed.  The  view  prevails  that  Erasmus+  has  even 
limited  potential  to  reach  those  organisations  that  address  specifically  this  group  of  people. 
Even  the  European  Volunteer  Service  (EVS)  remains  strongly  competitive.  Most  National 
Reports and findings  from the public  consultation welcome the additional funding available 
for  disadvantaged  participants330  but  do  not  consider  it  sufficient  on  its  own.  Some 
countries331  call  for  clearer  definitions  and  streamlined  rules  applying  to  disadvantaged 
participants  in  Erasmus+.  According  to  the  expert  panel,  while  a  good  share  of  projects 
viewed  considered  disadvantaged  groups,  this  was  often  not  substantiated  with  outputs 
suitable to them. Moreover, the Student Loan Guarantee Facility (SLGF) is commonly found 
to be insufficiently tailored to address the needs of the disadvantaged who are risk averse to 
go abroad for a full Master programme or to take up loans even if repayments are not income-
contingent332,  although  the  first  evidence  from  the  beneficiaries  surveys  rather  shows  many 
beneficiaries  are  first-generation  higher  education  students  or  coming  from  families  with 
financial  difficulties333.  Despite  some  progress  made  in  the  programme  design,  it  can  be 
concluded  that  there  is a  certain  dissonance  between  its  ambition  and actual  project  outputs 
specifically addressing the disadvantaged target population. The views whether more can be 
done at European level to address these challenges were nevertheless mixed among national 
policy-makers. 

328  The  inclusion  of  disadvantaged  groups  is  not  specifically  encouraged  by  the  Baltic-American  Freedom  Foundation, 
Denmark-USA programme, the Fulbright programme, German-Turkish Youth Bridge and Prämienprogramm 
329 BEde, BEnl, BG, CZ, DE, EE, IE, CY, EL, ES, FR, HR, IT, LV, LT, LU, HU, MT, NL, AT, PL, PT, RO, SI, SK, FI, SE, 
IS, FY, TR 
330 In addition to the social inclusion dimension in mobility (e.g. top-ups), in 2016, more than €200 million were prioritised 
under KA2 as well as €13 million for dissemination and €14 million for policy experimentation under KA 3. 
331 FR, IT, NL 
332 According to literature reviewed, interviewed students and 79% of agency respondents 
333 ICF, stand-alone report on SLGF and annex 8 on SLGF survey results 

52 

 
 
 
 
 
 
 
                                                            
Examples of support to disadvantaged individuals or hard-to-reach group 

In Poland, the NA implements the ‘Foreign mobility of disabled and disadvantaged students’ project co-
financed  under  Erasmus+  and  the  ESF.  This  project  funds  scholarships for  students  with  disabilities  and 
provides additional funds for disadvantaged students receiving grants from their home universities. 

In Turkey, as a result of a KA2 project implemented by the Sincan Public Training Centre, adult training is 
delivered to inmates at the Prison using a tailored curriculum and method. 

Source: NRS, 3.4 (PL, TR) 

53 

 
 
 
 
5.3.  Coherence 

The  assessment  of  the  coherence  of  Erasmus+  is  overall  very  positive,  both  internally 
(compared  to  its  predecessors)  and  externally  (compared  to  other  existing  interventions).  It 
has been informed by 10 evaluation questions that looked into the extent to which: 

 
the programme’s internal coherence improved compared to its predecessors 
 
the Erasmus+ objectives were consistent, mutually supportive across fields 
  synergies or duplications between fields and actions existed and were dealt with 
 
the centralised and decentralised actions complemented each other 
  Erasmus+ design provided appropriate support to the EU policy agenda 
  Erasmus+ was coherent with relevant policies and programmes and complementary to 
other  interventions  in  the  same  fields  at  EU  or  national/international  levels 
respectively. 

5.3.1.  Internal coherence334 

The merger of seven predecessor programmes into Erasmus+ has improved the coherence 
of  the  EU  intervention  according  to  the  vast  majority  of  respondents335.  As  mentioned 
before, Erasmus+ now covers learning in all its contexts – whether formal or non-formal, 
including  youth  work–  and  at  all  levels  of  the  lifelong  learning  continuum:  from  early 
childhood education and schools, vocational education and training (VET) through to higher 
education and adult learning. The evaluation highlights the positive effects of this integrated 
approach  underpinned  by  the  lifelong  learning  logic.  It  underlines  the  necessity  of  the 
integrated approach in today's education and training environment, where the borders between 
the traditional education and training sectors have become less clear-cut336. 

Different  sources  agreed  on  the  added  value  of  the  integration  of  several  programmes 
within  a  single  structure.  The  objectives  are  seen  as  mutually  supportive  across 
education/training,  youth  and  sports,  as  well  as  across  actions337.  These  fields  are  seen  as 
naturally  fitting  in  a  programme  allowing  cross-sector  fertilisation,  which  -  according  to 
90%  agency  respondents  -  was  not  always  possible  in  the  past.  Around  3/4  of  these 
respondents  feel  that  the  integrated  programme  enables  increased  synergies,  while  2/3 
estimate it avoids internal overlaps. 

All these improvements are confirmed by desk research, national reports and a vast portion of 
the  interviewees  who  appreciate  that  the  programme  has  been  designed  with  better  logical 
linkages  between  objectives  and  actions338.  It  is  very  often  valued  for  its  clearer  and 
simplified architecture with only three Key Actions (KAs)339, cross-cutting priorities (e.g. 
social inclusion, prevention of violent radicalisation, etc.) and a much lower fragmentation of 
actions than denoted in the past (e.g. Grundtvig, international cooperation340), with only a few 

334  Source  of  this  section  (where  not  mentioned  otherwise):  ICF,  5.2;  NRS  4.1;  OPC  3.2  and  5.1.3.  The  evaluation  of  the 
coherence in relation to Jean Monnet actions was evaluated based on desk research. 
335 Interviews, agency survey; ICF, 5.2.4: no contradiction in objectives noted between JMO and the rest of Erasmus+ 
336 ICF, Conclusion D: Many VET providers are falling under the definition of 'schools' (in school-based VET) but they can 
also be providers of adult education. Similarly, higher education organisations are also providers of adult education in many 
cases.  The  cooperation  between education  and  training  (in  particular  for  schools  but  also beyond)  and  civil  society  (youth 
sector) is a common reality on the ground. 
337 ICF, tab. 5.3 
338 ICF, 5.2.1 
339 See above section on background. 89% (n = 619) of OPC respondents (strongly) agreed that the structure into the three 
Key Actions is working well. 65% of agency respondents (strongly/rather) agree that it increases EU added value. 
340 The merger of Tempus, Alfa III and Edulink into the Erasmus+ Capacity Building action allows for a strengthening of co-
operation between different regions, however the focus on intra-regional co-operation for partner countries remains limited, 
according to the Evaluation of the EU Development Co-operation Support to Higher Education in Partner Countries (2007-
2014), (2017), Particip, LeA, ECDPM, Ecorys, already mentioned. 

54 

 
 
                                                            
exceptions341.  Almost  all  country reports  call  for  stability  of  the  structure  and  scope of  the 
programme  in  the  future.  The  stronger  attention  given  in  the  design  and  implementation  of 
Erasmus+  to  budget  based  on  performance342,  monitoring  of  individual  outcomes343  and 
dissemination  of  project  results344  is  also  valued.  As  detailed  under  "effectiveness"  above, 
actions  are  delivering  at  more  than  one  level,  with  spill-over  effects  (e.g.  individual  staff 
mobility having impact on sending organisations) and common support tools (e.g. widening 
scope of transparency and recognition tools). 

The  advantages  of  Erasmus+  programme  listed  by  the  national  respondents  include  its 
strong  branding  and  its  greater  visibility345,  especially  among  policy-makers.  This  is 
particularly  true  for  VET,  adult  education  and  even  for  the  youth  sector  in  contrast  to 
stakeholders'  initial  fears.  European  challenges  (such  as  the  integration  of  refugees  or  the 
attention to disadvantaged groups in higher or adult education) are now addressed at least in a 
more  flexible  way346.  Moreover,  lifelong  learning  is  approached  more  as  a  lifecycle 
(including non-formal and informal learning) rather than  in a segmented  way. Even  if there 
remains  residual  concern347  that  the  brand  “Erasmus”  is  traditionally  linked  to  higher 
education, the fact that the Erasmus+ provides one place to look for funding opportunities 
in the related sectors with a standardised and more transparent administration is appreciated. 

Meanwhile,  some  concerns  have  been  reported  at  national  level.  In  two-thirds  of  the 
countries,  different  NAs  continue  to  deal  with  distinct  sectors,  replicating  the  way  they 
functioned under the previous programme generation348. Some National Reports confirm that 
there is no systematic exploitation of potential synergies between different types of actions349. 
The  disadvantages  inherent  to  any  one-size-fits-all  approach  are  naturally  pointed  out350. 
Respondents from various organisations and sectors noted that the adult education strand has 
shrunk in size and lost its specificity. The youth chapter would be less integrated, less visible 
for  instance  within  KA2  (or  would  now  link  to  more  abstract  EU  policy  targets  than  in  the 
past). Whilst the objectives of the sport chapter align well with EU priorities, some potential 
applicants do not always see how to make them fit in their local perspective. More generally, 
an integrated programme is likely to offer less autonomy to sectors and less room for country-
specific  priorities  compared  to  its  predecessors.  There  is  for  instance  the  sentiment  among 
national authorities that the integration of the programme committees has led to discussions 
becoming more abstract for each policy area integrated in Erasmus+. 

looking  at  specific  examples, 

When 
increased 
decentralisation was positively viewed by the vast majority of respondents because it brings 
the  programme  closer  to  its  beneficiaries  and  helps  engage  local  partners,  while  centralised 
actions  (e.g.  in  KA3)  complement  well  other  actions  of  the  programme.  Meanwhile,  policy 

further  nuances  emerge.  The 

341  NRS:  mobility  in  the  field  of  school  education  and  VET  is  not  included  under  KA1  but  can  be  covered  under  KA2, 
creating some confusion among schools (e.g. DK, FR, IT, CY, LV). 
342 Article 18(7) and (8) of Regulation (EU) No 1288/2013 about performance based budget: in practice, around 25% of the 
funds allocated to National Agencies for KA1 are distributed on the basis of the level of implementation achieved, whilst the 
remaining  budget  is  allocated  essentially  on  the  basis  of  the  size  of  the  population.  For  KA2  and  3,  qualitative  criteria  to 
increase EU added value apply (e.g. link with OMC; resource commitments of project partners). 
343 Article 21 (idem) on monitoring of performance and results 
344 Article 22 (idem) on the dissemination of results 
345 76% of agency respondents strongly or rather agree that the integrated programme is more visible /better branded than the 
predecessor programmes; confirmed by social media analysis 
346 ICF, tab. 5.9 
347 CY, EE, HU, SI, etc. Besides, for a small portion (estimated less than 10%) of respondents Erasmus+ still remains viewed 
as associated with the mobility experience of HE students. 
348 Nevertheless certain National Agencies have merged since 2014. Their number has been reduced from 63 in 2013 (with 
12 NAs common to education and youth) to 57 (16 common NAs) in June 2017. 
349 BEnl, EE, HU, SI, SK, UK, NO 
350 ICF, tab. 5.9 

55 

 
 
                                                            
makers  and  NAs  were  clearly  more  positive  than  EU  level  organisations  which  look  for  a 
more direct access to funding. The main critical perceptions relate to KA2 suggesting that 
too broad scope of the action and the decentralisation of strategic partnerships may result in 
differences in implementation or a prevalence of nationally-driven agenda351. Some argue that 
this  could  even  push  partnerships  or  cross-country  organisations  to  apply  rather  in  larger 
countries.  Besides  to  attract  talent  to  Europe,  some  suggest  it  could  make  more  sense  to 
centralise international credit mobility. However, providing concrete examples of synergies or 
overlaps has proven more difficult than anticipated for most interviewees at national level or 
agency  respondents.  The  only  area  where  some  overlaps  have  been  noticed  concerns  those 
projects  in  the  sport  sector  which  focus  on  social  inclusion  and  could  be  youth  projects  as 
well. However, overlaps might be more often due to misunderstanding at application stage 
than to the design of the programme as such352. 

Examples of internal synergies or overlaps 

Synergies 

Overlaps  

"Yet, a significant difficulty has been experienced 
with the adult education and VET actions […]. 
Coverage and scope of these fields mostly overlap 
and cannot easily be differentiated by the 
beneficiaries." (TR) 

-  Between fields: "It is more likely that a 

- 

- 

- 

young person who has done an Erasmus will 
also go on youth exchanges or EVS 
projects." 

"An integrated programme stimulates cross-
sectoral collaboration in a valuable way. 
Regional development projects are easier 
[…] to communicate and are also 
administratively simpler when all the target 
groups are part of the same programme." 
(SE) 

"Our organisation takes part in an activity 
under KA 3 in the youth chapter and uses the 
outputs for advocacy in the education field." 
(Structured dialogue with young people) 

-  There is evidence of education institutions 

successfully applying to implement sport 
projects that are also tied to the education 
and youth field. 

-  The last example on the right column has a 
counterpart, as meanwhile it reduces the 
participation of purely sport-focused organisations. 

-  Between KAs: Synergies are reported at 

- 

- 

- 

organisation level between e-Twinning and 
KA1 and KA2 in the school sector. 

"The way in which KA1 and KA2 
complement each other provides 
opportunities for cooperation between long-
term network partners." (FYROM) 

"Within KA2, there is a close cross-sectorial 
cooperation, which benefits all sectors. […] 
There are strategic partnership projects that 
have led to development of tools and 
databases to heighten the quality assurance 

"Student mobility within the field of school 
education falls within KA2. This has led to 
confusion among the schools given that student 
mobility for all other sectors falls under KA1. It is 
recommended that student mobility for pupils in the 
field of school education be moved to KA1." (DK) 

-  Mobility opportunities under KA1 and KA2 "for 
Erasmus+ Youth in Action the logic is not kept. 
[…]. KA1 and KA2 in Erasmus+ seem to be partly 
overlapping in practice, […]. This leads to the point 
that applicants are partly going for their mobility 
projects to KA2 because of the higher funding" 
(Youth NAs' input) "KA1 could benefit from 

351 Agency survey, OPC position papers from 3 EU-level organisations representing together some 70 organisations 
352 Common input for the mid-term evaluation 2017 from the NAs Erasmus+ Youth in Action, August 2016 

56 

 
 
 
 
 
 
 
 
 
                                                            
of VET students’ mobility projects, which 
have been implemented by the vocational 
colleges working within KA1" (DK) 

focusing on training and capacity building tackling 
learners directly and KA2 more on intellectual 
outputs that are tangible enough and with long term 
impact." (Youth NAs' input) "There is a certain 
overlap between youth worker mobility and 
strategic partnerships for good practices" (BE-nl) 

-  KA2 (Strategic partnerships) and KA3 (Forward 

looking cooperation initiatives) could overlap (ICF). 
"KA2 is often used for policy reform too at national 
level" (Youth NAs' input). 

-  Between actions: KA3 Policy 

- 

experimentations and Forward looking 
initiatives: (one is top-down, the other is 
bottom-up) are meant in their design to be 
the two sides of the same coin. (ICF) 

"The inconsistencies arise from differences in 
grants given per participant for training activities 
versus youth exchanges [two sub-actions in KA1]. 
The grants for training activities are higher, which 
increases the incentive to apply for funding for 
training activities." (DK) 

Source (where not mentioned otherwise): ICF, 5.2.2 

Another area where findings are more contrasted concerns cross-sector opportunities. Whilst 
a  significant  number  of  respondents353  see  this  as  a  clear  improvement  under  the  new 
programme (e.g. for youth), many others354 remain more uncertain (e.g. unclear remit of adult 
education355) or have concerns about its actual materialisation (e.g. for sports356). A majority 
of respondents asked are either not interested or not convinced to take part in cross-sectoral 
projects  (e.g.  finding  different  partners  require  more  efforts).  However,  as  detailed  under 
5.1.2, the level of cross-sectoral cooperation increased under the current programme by 23 
percentage points 

5.3.2.  External coherence357 

At programme level, complementarities are mainly found with the European Social Fund358 
(ESF)  and  the  Research  framework  programme  Horizon  2020,  and  to  a  lesser  extent  with 
INTERREG,  Erasmus  for  Young  Entrepreneurs  programme  (EYE)  and  in  the  field  of  sport 
with  the  Third  Health  programme.  For  instance,  in  Greece,  the  participation  of  pupils  in 
eTwinning  was  boosted  nation-wide  by  the  combined  support  of  European  Investment  and 
Structural Funds to teachers' training (ICT skills) and classroom equipment (digital tablets)359. 
Potential  synergies  that  were  noted  by  interviewees  include:  Erasmus+  and  the  European 
Training  Foundation  (ETF);  Europe  for  Citizens  and  Culture/Creative  Europe  programmes; 
the EU Programme for Employment and Social Innovation (EaSI) which has a component of 

353 High share of agencies’ respondents; DK, EE, NL, PL, PT, SI, NO, etc 
354 Over half of interviews (at national level particularly), several OPC position papers, DE, EE, MT, NL, SI, SK, FY, TR… 
355  Overlaps  are  observed  between  VET  and  adult  education  (CY,  PT,  TR);  LLP  interim  evaluation  COM(2011)  413, 
section 5 
356 The borders between sport and other strands are rather blurred. There are more sport organisations participating in other 
strands than in the sport one. Reciprocally, more than 5% of participants in the sport strand are higher education institutions. 
Many grassroots sport projects could be funded under the youth strand. 
357 Source of this section (where not mentioned otherwise): ICF, 5.3; OPC 3.2; NRS 4.2 
358 The ESF legal basis recommends taking up good practices from Erasmus+ to further develop them under ESF. Erasmus+ 
mobility  grants  can  be  topped-up  by  ESF  or  followed  by  placement  services  financed  by  ESF.  Under  the  ESF  investment 
priority  “Improving  the  quality  of  tertiary  education”,  for  instance,  the  ESF  may  support  activities  to  encourage  the 
development of Erasmus activities in a university. 
359 Workshop DG REGIO/KPMG, 18 November 2016, organised in the context of the study mentioned below 

57 

 
 
                                                            
entrepreneurship  education,  etc.  Few  potential  overlaps  were  alleged  by  some  interviewees 
(e.g. in the case of strategic partnerships) but without any concrete examples360. 

The  integration  of  the  programme  has  also  enhanced  the  external  coherence  in  removing 
certain  overlaps361.  Insights  on  the  coherence  of  the  programme  against  other  EU  level 
programmes were gained from both a majority of EU level interviews and the vast majority of 
agency  and  OPC362  respondents.  Those  provided  examples  of  (potential)  synergies  or 
overlaps, confirming desk research and benchmarking. 

Examples of external complementarities, synergies or overlaps 

Complementarities 

Synergies 

Overlaps  

With the European Social Fund (ESF), Youth Employment initiative 

"For example, why does ESF 
funds projects to establish 
placement/internship structures if 
Erasmus+ has done so for the 
past 10 years (and beyond)?" 

NB: This might not necessarily 
be an overlap, as Erasmus+ 
supports transnational activities, 
whereas the ESF focuses on 
activities implemented in a 
national context. 

- 

- 

International visits, transfer 
of good practice from 
abroad, the training of 
teachers, etc are supported 
by the ESF. Topics which 
are dealt with include digital 
education, language 
training, and recognition of 
results from formal and 
informal learning, 
cooperation between 
schools and employers, etc 
(KPMG363) 
Former LLP ‘People on the 
labour market’ initiative is 
now run through ESF, 
providing mobility 
opportunities for people 
after studies. 

-  A number of projects under 
the operational programme 
‘Employment, Human 
Resources and Social 
Cohesion’ are co-funded by 
the ESF, the Youth 
Employment Initiative and 
Erasmus+ (CY). 

- 

- 

In Latvia, Erasmus+ and the 
ESIF in the field of 
education and science are 
managed by the same 
Ministry, helping to increase 
synergies. (KPMG) 

In Sweden, the ESF is used 
to fund measures in 
preparation and as a follow-
up on the Erasmus+ 
mobility. Although 
Erasmus+ has a smaller 
budget, by combining the 
two funds, both objectives 
of labour mobility and 
labour market activation are 
better achieved. (KPMG) 

-  Discussion between 

stakeholders of ESIF and 
Erasmus+ has increased in 
the Czech Republic since 

360 Existing or potential synergies were most often reported than overlaps across the nine programmes discussed. In the OPC, 
only  one  EU  level  organisation  identified  overlaps  between  Erasmus+  and  national  funds  in  education  and  training,  in 
particular  in  the  case  of  Germany  (DAAD  funds  for  joint  degree  programmes)  and  Norway  (national  travel  support  for 
outgoing mobility). One organisation was concerned about potential overlaps between Erasmus+ and the European Solidarity 
Corps. ICF, NRS 4.2 
361 Mobility of doctoral researchers to/from partner countries, which overlapped with Marie Skłodowska-Curie actions 
362 80% of respondents (n = 1485) believed that Erasmus+ does not overlap with other funding opportunities, compared to 
3% of respondents who believed it ‘fully overlaps’. 
363  Study  on  the  co-ordination  and  harmonisation  of  ESI  Funds  and  other  EU  policies,  DG  REGIO/KPMG/Prognos,  to  be 
published in 2018 

58 

 
 
 
                                                            
2014. The Czech Ministry 
of Education has set up a 
centre (DZS) which co-
ordinates EISF, Erasmus+, 
EQF, EQAVET or 
Europass. Beneficiaries find 
it easier to apply for 
Erasmus+; ESIF is only 
used to provide additional 
resources and authorities are 
trying to avoid financing the 
same beneficiaries twice. 
(KPMG) 

With the European Regional Development Fund (ERDF), INTERREG 

ERDF is moving towards smart 
specialisation strategies for 
regions, which is related with 
training (ESF, Erasmus+). 

-  Some INTERREG 
projects prepare framework 
for mobility activities under 
Erasmus+ (e.g. Danish-
German projects) 

-  For instance, Erasmus+ 
supports mobility, whereas 
the ERDF support 
investments in 
infrastructure, such as the 
renovation of buildings. Due 
to these investments, Latvia 
became more attractive to 
students and academics 
from other Member States. 
(KPMG) 

With the Framewok Programmes for Research and Innovation 

On the international 
dimension of the previous 
programmes364, "the EU’s 
approach to supporting 
higher education on the one 
hand and research and 
innovation on the other was 
complementary… 

…and has created some 
synergies, for example: 

- Links between Erasmus 
Mundus and Tempus IV with 
the Seventh Framework 
Programme (FP 7), the 
EU's main instrument for 
funding research; 

- Edulink’s connection with 
the ACP Science & 
Technology Programme 
(ACP S&T)… 

…However, formal and 
institutionalised attempts to 
connect the major programmes 
targeted at universities were very 
limited (they existed only in the 
case of ACP HE Institutions), and 
there is room (and demand) for 
creating more synergies." 

"Erasmus+ complements 
Horizon 2020 very well 
[since the focus of each 
programme is rather 

The share of EU students in 
in the EIT Label-
programmes has slightly 
declined in recent years, 

Horizon 2020 (e.g. priority 6 on 
societal challenges) tackles same 
issues as Erasmus+, on larger 
scale but without synergy. For 

364 Evaluation of the EU Development Co-operation Support to Higher Education in Partner Countries (2007-2014), (2017), 
Particip, LeA, ECDPM, Ecorys, already mentioned 

59 

 
 
 
                                                            
complementary and partly 
intends to fulfil the same 
goals] e.g. Joint Master 
Degrees (Erasmus Mundus) 
in Erasmus+ and Joint 
Doctorates in Marie Curie 
Actions" 

which could be due to the 
alignment of the EIT 
programmes with Erasmus 
Mundus actions. 

(Interim evaluation of EIT) 

example, early school leaving 
was addressed in 20-30 projects 
under Horizon 2020 and with 
over 100 projects under 
Erasmus+. 

With Young Entrepreneurs programme (EYE) 

Many EYE participants 
have gone through Erasmus 
mobility while they were 
students. Individuals are 
later more open to 
international mobility. 

Source (where not mentioned otherwise): ICF, 5.3  

Despite  this  promising  picture,  most  of  the  examples  supplied  remained  rather  general.  In 
particular,  few  examples  were  provided  where  ESF  would  have  funded  follow-up  of 
Erasmus+  projects  (KA2).  More  synergies  should  be  encouraged  between  ESF  and  policy 
actions (KA3) for designated bodies. Considering the call for action set in the Erasmus+ legal 
basis365,  some  respondents  suggest  that  communication  between  services in  charge  of  given 
programmes could be strengthened at EU level. In particular, no clear example of dedicated 
coordination tools between EC services was reported. 

At  policy  level,  the  topical  coverage  of  the  selected  projects  reviewed  reveals  a  clear  and 
satisfactory  alignment  with  different  EU-level  policies366.  Most  interviewees  (particularly 
at  EU  level)  perceive  Erasmus+  as  supporting  key  EU  policy  agendas367,  especially  in 
education  and  training,  similarly  to  its  predecessors  but  in  a  more  streamlined  manner. 
Most  of  EU  interviewees  could  confirm  it  with  examples  of  complementarities.  Out  of  the 
80%  of  reviewed  projects368  aimed  at  contributing  to  the  main  EU  policies/strategic 
documents analysed as part of desk research369, two-thirds focused on ET2020 topics, mainly 
social inclusion370. However, across the sampled projects, only a very small number focused 
on  the  Europe  2020  headline  targets,  despite  these  being  the  two  first  key  indicators  of 
Erasmus+371.  Lastly,  the  Jean  Monnet  projects  reviewed  appeared  to  be  the  least  aligned, 
being perceived as too focussed on experts' needs372. 

365 Articles 3.2 (b) and 25 of Regulation (EU) No 1288/2013 define complementarity 
366 Confirmed across the sources mentioned under this section by a high share of respondents or projects reviewed 
367 Europe 2020* (for most interviewees), ET2020*(to a high extent), European Youth Strategy* (although perceived as a bit 
less visible), Copenhagen process and tools (often mentioned for VET), Modernisation of higher education agenda (strong 
consistency), EC Communications on ‘New Skills Agenda for Europe’* or 'European higher education in the world'* (albeit 
both mentioned to a lesser extent), EU Work Plan for sport* (probably consistent although not as clear at national level as it 
is at EU level), etc. (*): analysed through Desk Research 
368 ICF, Annex 10 (published) and 17 (not published) 
369  Marked  with  an  asterisk  (*)  in  the  footnote  above.  For  instance,  over  half  of  the  sport  projects  reviewed  aligned  with 
priority objectives listed in the EU Work Plan or sport. ICF, tab. 5.13 
370 Only three of the 18 comparator programmes reviewed pay special attention to the participation of disadvantaged groups 
e.g.  the  Causeway  programme,  the  Nordplus  Higher  Education  and  Adult  programme  programmes  and  the  UK  Sport's 
IDEALS programme in sport. Attention paid under Erasmus+ to widening participation was seen as complementary in most 
other cases. 
371 Early school leaving and Higher education attainment as per Annex 1 of Regulation (EU) No 1288/2013 
372 Project review, JMO beneficiaries and non-beneficiaries survey, desk review 

60 

 
 
 
 
 
                                                            
Examples of support to EU policy agenda 

-  ET2020 working groups (WG): as per the reviewed ET2020 priorities for 2016-2018, the WGs 
have  now  a  recurrent  mandate  to  support  and  to  follow-up  the  Erasmus+  KA3  policy 
experimentation.  For  instance,  in  2016,  synergies  were  found  for  the  following  topics: 
‘Strengthening  teacher  training  and  education by  using  the  opportunities  of  new  technologies"  ; 
‘VET  teachers  and  trainers  in  work-based  learning/apprenticeship  (VET)’  ;  ‘Employment  and 
Skills: validation of informal and non-formal learning in Education and Training’, etc.  

-  Building on the legacy of LLP, Erasmus+ support the review and implementation of several EU 
transparency  and  recognition  tools,  by  (co-)funding  the  Secretariats  and  national  bodies  for 
ECVET  and  EQAVET,  the  Euroguidance  network,  the  national  Europass  centres  and  EQF 
Coordination Points. 

Source: ICF, 5.3  

Erasmus+  programme  is  also  coherent,  to  a  high  extent,  with  interventions  pursued  at 
national  and  international  level  which  have  similar  objectives.  A  vast  majority  of 
interviewees  consider  it  to  be  overall  well  aligned  with  national  priorities  in  all  sectors. 
Benchmarking of comparable programmes373 but also National Reports374, interviews, agency 
survey  and  desk  research  did  not  detect  any  notable  overlaps375  and  provided  interesting 
examples  of  complementarities  at  both  levels.  The  area  most  often  mentioned  by  VET 
interviewees  in  that  respect  is  school  and  work-based  learning.  The  higher  education 
interviewees  offered  several  examples  of  national-level  programmes  that  show  significant 
coherence with Erasmus+376. 

The  benchmarking  confirmed  a  fair  alignment  between  Erasmus+  and  national 
comparable programmes in terms of objectives, target groups377 and duration of actions378. 
Although few overlaps were reported379 and efforts have already been made to clear them380, 
the existence of parallel funding opportunities381 could likely be better communicated to the 
public at national level, considering the aforementioned call for action set in the legal basis. 
Indeed,  despite  the  fact  that  several  NAs  run  some  schemes  in  parallel  to  Erasmus+,  many 
beneficiary organisations and NA respondents382 claimed they were not aware of any similar 
national support. 

Countries themselves did not identify any specific overlaps between Erasmus+ and other 
national  and  international  programmes,  but  rather  more  synergies.  Erasmus+  is  different 

373  Out  of  58  national  or  international  schemes  shortlisted  as  supporting  actions  similar  to  those  of  Erasmus+,  18 were 
benchmarked against the programme. Action-wise, the vast majority support mobility. Several NAs run such programmes in 
parallel to Erasmus+ (ICF, Annexe 12). 
374 NRS, 4.2 
375 Few National reports (e.g. FR) mention cases where synergies could be improved. For instance, to avoid the duplication of 
similar tools at local level, the Europass could be made more flexible with a second part for information customised to local 
requirements. 
376 Nordplus Higher Education (Nordic countries); CEEPUS (various countries); National Science Centre programmes (PL); 
Norway/EEA funds instrument (various countries), etc. 
377 High alignment in higher education, VET and youth ; lower in school and adult education and in sport 
378 ICF, tab. 5.15. For example, some comparator  schemes, especially at the school level and in adult learning, offer much 
longer average mobility stays for the practitioners. 
379  20%  of  OPC  respondents  considered  that  Erasmus+  is  ‘fully’  or  ‘partially’  overlapping  with  national  funding 
opportunities (for student mobility or volunteering activities). However, most of them specified that even if there is a partial 
overlap, Erasmus+ is still very much required either due to: a lack of funding even with the overlap, different objectives of 
the funding even when targeted at the same beneficiaries or differences in geographical areas covered. 
380 54% of agency respondents declared their agency had taken specific actions to ensure complementarity between Erasmus+ 
and other programmes (e.g. joint events, information sessions) 
381 For mobility in education and training (DK, EE, CY, HU, FI, etc) or youth (EE) 
382 Case studies for the former; ICF, tab.5.16 for the latter 

61 

 
 
                                                            
from  (inter)national  initiatives  in  many  respects383,  e.g.  wider  number  of  beneficiaries  and 
projects; wider geographical scope; different target groups or sectors than national initiatives; 
mobility for longer periods; no other initiatives in some smaller countries to support certain 
sectors  (e.g.  youth)  or  policy  reform  similar  to  Key  Action  3.  A  few  national  reports  even 
mention cases of synergies384. 
Although aiming to achieve comparable goals in most cases385, none of the national schemes 
reviewed can match the pan-European scope of Erasmus+, not to mention its worldwide 
dimension,  making  it  the  best-placed  programme  to  develop  internationalisation.  In  most 
cases  it  is  also  more  generous  than  other  similar  schemes.  Furthermore  Erasmus+  offers 
some unique actions that none of its comparators does (or, at least, not to the same extent), 
such  as  electronic  support  platforms  (eTwinning,  EPALE  etc.),  stakeholder  dialogue  or 
knowledge  building  for  evidence-based  policymaking.  Over  half  of  agency  respondents 
strongly  or  rather  agree  that  Erasmus+  is  doing  better  than  the  latter  in  all  five  areas 
considered386. This can be regarded as a real added value of Erasmus+, as developed under the 
related section below. 

383 CY, EE, MT, NL, SI 
384  FI,  SE,  NO.  In  Norway  a  number  of  VET  schools,  use  Erasmus+  mobility  strategically  to  offer  students  specialised 
training in a sub field that is not available at their own school or even in the country. 
385 ICF, tab. 5.15. The level of alignment of goals is particularly high in formal education and relatively lower in the sport 
field. 
386  Better  alignment  with  EU  policy  priorities  (especially  in  higher  education  and  youth);  broader  geographical  coverage; 
more  opportunities  for  disadvantaged  target  groups  (except  in  adult  education);  broader  topic  coverage;  more  funding  for 
comparable activities. 

62 

 
 
 
 
                                                            
5.4.  Efficiency and simplification 

When it comes to assessing the efficiency of the programme387, including in comparison to its 
predecessors, the overall picture is positive388, stemming from the analysis of 24 evaluation 
questions that cover the following areas: 

 

 

 

 

the cost-effectiveness of the main types of actions 

the efficiency of implementation and management modes 

the efficiency gains through changes in the integrated programme 

the efficiency of monitoring arrangements and measures to identify and prevent fraud 
and irregularities. 

Overall  the  costs  of  management  for  the  EU  of  the  entire  programme  are  reasonable 
(6% of the Erasmus+ administrative and operational budget)389. This is particularly clear 
when  compared  to  other,  much  smaller,  comparable  national  actions,  which  appear  more 
costly (on average, 14% of their respective budget)390. 

Most of respondents to the consultation believe that the user-friendliness of the programme 
has improved over time391. However, work on simplification (e.g. online application forms) 
has already started in 2017. The level of dissatisfaction has decreased as of 2016 after a steep 
learning curve. 
Nonetheless,  there  is  still room  for  improvement  in  several  features  of  the  programme392, 
especially  concerning  further  simplification  of  procedures  and  tools  (e.g.  on-going 
development  of  e-forms),  more  flexibility  on  budget  allocations,  adjustment  of  the  level  of 
simplified  grants  as  already  proposed  in  2017,  as  well  as  the  reversal  of  the  low  success 
rates393 in some particular fields and sectors. 

5.4.1.  Cost-effectiveness 

With regard to learning mobility of individuals under KA1, the relationship between the costs 
(inputs) and effects (results and impacts), indicates a positive cost-effectiveness394 especially 
for learners but also for staff395. Considering the positive outcomes identified above under 
"effectiveness",  the  costs  per  individual  appear  clearly  reasonable.  This  is  further  supported 
below by the evidence of European added value of these types of actions, also compared to 
similar  programmes  reviewed  at  national  level.  The  average  cost  of  a  mobility  activity  is 
1,500€  per  learner  (15€  per  day/learner)396  and  between  700€  and  900€  per  staff  member 

387  Efficiency  considers  the  relationship  between  the  resources  used  and  the  results  of  the  programme  with  a  view  to 
maximise yield for given resource or minimise costs for a given output. 
388 ICF, 7; NRS, 6; OPC, 3.4 
389 This includes the operating grant for National Agencies as well as the administrative expenditure of the Commission and 
EACEA.  In  2015,  these  management  costs  represented  19%  of  the  budget  managed  directly  by  the  Commission  (mainly 
covering costs for the development of IT tools for NA as programme stakeholders), 10% for EACEA and 5% of the budget 
managed  indirectly  by  NAs,  which  appears  well  in  line  with  the  different  nature,  complexity  and  volume  of  activities 
managed respectively at each level; ICF, 7.4, tab. 7.18 
390 Share of administrative costs in comparator programmes, from 10% (EEA) to 20% (Causeway Ireland); ICF, fig. 7.15 
391 OPC, 3.4 
392 ICF, 7.1 and 7.2; NRS, 6.1; OPC, 3.4 and 5.1.4 
393 Funded projects out of the total number of applications 
394 Cost-effectiveness analysis judges costs against the benefits achieved. 
395 Although generally positive, more positive for learners than for staff: ICF, 7.3.4 and 7.3.5 
396  Although  the  format  of  the  mobility  activities  is  quite  standardised,  the  differences  in  average  grant  support  are 
determined by several factors, such as the average duration of a typical activity, the contribution to higher subsistence costs 
in  the  case  of  staff  mobility  as  well  as  the  contribution  to  travel  costs,  which  are  normally  higher  in  the  case  of  mobility 
activities from or to Partner Countries. The cost-effectiveness of learner mobility is clearly shown when comparing the low 
costs to the results identified (see section on effectiveness). The costs per learner vary between roughly 900€ in the youth 

63 

 
 
                                                            
 
(200€ per day/staff)397. An increase in costs per learner has been measured compared to 
the  predecessor  programme  as  regards  higher  education  (+9%)  and  VET  (+1%)398.  An 
increase  in  costs  per  staff  member  is  similarly  observed  in  higher  education  (+22%),  adult 
education  (+19%)  and  VET  (+2%),  while  a  decrease  is  noted  in  the  case  of  school  staff  (-
6%)399. Although the impact of a mobility activity is more significant by nature for learners 
than  for  staff,  the  effects  on  staff  also  create  positive  spill-over  effects  at  the  level  of  the 
organisations/institutions  as  well  as  on  the  learners  who  are  not  necessarily  taking  part  in 
mobility  activities.  Hence,  the  overall  ratio  of  costs  versus  effects  can  be  considered 
positive  in  all  cases.  There  is  also  an  European  added  value  of  the  programme  actions  for 
staff  as  for  learners,  since  the  comparison  with  national  programmes  is  favourable  for 
Erasmus+, as demonstrated below under "added value". 

As  the  grant  amounts  mentioned  above  may  appear  low,  the  degree  to  which  these  grants 
cover  expenses  incurred  has  been  examined.  On  average  for  42%  of  KA1  learners  the 
Erasmus+ grant covered most of expenses (76-100%), but as the graph shows below, wide 
differences  are  made  between  learner  categories400.  Erasmus+  is  successful  in  leveraging 
complementary funding from national or regional budgets401. 

Share of expenses covered by the grant – by type of learner 

Source: ICF calculations based on Erasmus+ participant monitoring surveys402. 

On  the  other  hand,  the  conclusion  is  more  nuanced  as  regards  the  cost-effectiveness  of 
cooperation  projects.  This  is  because  there  are  a  variety  of  multilayer  effects  having  a 

sector and 4700€ in the higher education international sector (average cost: 1,500€ per learner). The costs per day of mobility 
vary between 10€ in HE sector (programme countries) and 60€ in VET (15€ on average per mobile learner per day). 
397  The  cost  effectiveness  of  staff  mobility  is  also  positive  though  the  judgement  is  more  nuanced.  The  average  costs  per 
mobile member of staff is between 700€ and 900€. The costs are lowest in HE (programme countries) and highest in higher 
education international. The costs per day of mobile member of staff are around 200€ (ranging between 100€ in youth and 
close to or more than 300€ in adult education, schools and higher education international).  
398 IFC, tab. 7.7 
399 IFC, tab. 7.10 
400 The Erasmus+ grant covered the majority of expenses for HE international students (e.g. Erasmus Mundus scholarship is 
intended to cover all costs: travel, master participation, subsistence) and VET students (72% and 68% respectively), whereas 
this was the case for only 19% of HE trainees (whom are paid/ compensated by their employers) and 8% HE students (as the 
grant is not expected to cover all costs of living, but only the additional costs of mobility abroad). 
401  9 to 15%  of  participating  learners  do  not  receive  a  grant  from  the  programme  (i.e.  so  called  zero  grant  beneficiaries), 
which suggest an additional added value of the programme and a spill-over effect; ICF, 7.3.6 
402 The participants who did not receive grant (about 8,5%) are also included into these results. 

64 

0%10%20%30%40%50%60%70%80%HE traineeshipsHE internationalHE studentsVET learnersAverages0-25%26-50%51-75%76-100% 
 
 
 
 
                                                                                                                                                                                          
impact  on 

the  solid  assessment  of  cost-effectiveness403.  These 

significant 
include 
differentiated  budget  items,  as  well  as  variations  between  their  types  and  sizes  across  and 
within sectors (around  180,000€ per project on average)404. In  spite  of efforts made  for this 
evaluation,  the  conclusion  on  cost-effectiveness  for  cooperation  projects  cannot  be 
generalised as for mobility activities. Although quantitatively, the cost-effectiveness ratio can 
seem  positive  at  first  sight  (widened  participation,  multiple  layers  of  effects),  on  a  more 
qualitative  note,  there  are  elements  that  can  be  improved  in  terms  of  efficiency  of 
implementation  (application  process)  and  effectiveness  (quality  of  project  outputs)405.  As  a 
general  trend,  cooperation  projects  seem  to  be  very  well  designed  and  conceived  at 
application  stage  although  they  are  not  able  to  keep  the  same  level  of  excellence  at  the 
implementation  stage.  The  majority  of  projects  receive  indeed  lower  quality  scores  on 
completion  than  at  application  stage.  In  contrast,  given  the  relatively  low  costs  of 
collaborative  platforms  for  the  EU,  these  types  of  actions  appear  as  particularly  cost-
effective. 

5.4.2.  Size of budget 

Budget  envelopes  for  most  of  the  sectors  were  regarded  as  insufficient406  and  strongly 
correlated  to  low  success  rates407.  In  some  actions  or  countries,  this  had  led  to  a 
discouragement  of  applicants  over  the  first  years  of  implementation  of  Erasmus+.  The 
potential of the programme for broader organisational  and system level effects, as discussed 
under "effectiveness", could be enhanced by reaching a critical mass across all the fields of 
education, training and youth. 

The demand largely exceeds the funding available. In KA2 the competition and the scores 
for successful projects are high408, which makes it more difficult for organisations with no or 
little  experience  to  access  the  programme.  Given  the  increased  demand  for  the  cooperation 
projects,  the  budget  allocated  to  KA2  and  KA3  did  not  appear  to  be  sufficient  yet  in 
absolute terms. Even in KA1, success rates can be low depending on countries and actions, 
meaning  that  more  learners  could  benefit  from  the  programme  if  there  were  more  funds 
available409.  Without  prejudice  to  negotiations  on  the  next  Multi  Financial  Framework,  the 
evaluation, including the public consultation, points to the need for reconsidering the budget 
envelope and the need for reviewing the budgetary distribution between programme sectors to 
have  a  better  alignment  to  their  relative  effectiveness.  However,  there  is  no  evidence  that 
seriously  questions  the  current  overall  balance  between  Key  Actions.  The  vast  majority  of 

403 ICF, 7.3.5 and 7.3.6 
404 Depending on the year chosen the average project size varies from 160,000€ to 200,000€. The smallest projects (around 
55,000€) are in the youth sector while the largest ones in higher education and VET (250,000€ to 270,000€). 
405  These aspects  refer  mostly  to the  following:  a)  "one  size  fits all"  regarding  application  forms (KA2  mainly) that  is  not 
proportionate,  meaning  that  small  projects  have  to  supply  the  same  information  as  much  larger  ones;  b)  the  ratio  between 
complexity of forms compared to the type of results expected that is not balanced (in KA2 mainly); and c) low success rates 
vs complicated application forms. ICF, 7.1 and 7.4; NRS, 6.1.2 and OPC, 3.4 
406  The  European  Parliament  (CULT  committee)  study,  2016,  Erasmus+:  decentralised  implementation  -  first  experience 
found that the views of the national agencies on the adequacy of programme funding to be almost equally split. The concern 
of insufficient budget size is also confirmed by the review of National Agencies’ reports. The insufficiency of budget was 
mentioned  in  67  of  the  2007-2013  NA  reports,  the  most  problematic  being  Grundtvig.  Over  2014-2015,  the  second  most 
frequent  problem  reported  by  the  national  level  related  to  the  insufficiency  of  budget,  especially  for  KA2.  Two  NAs 
suggested  a  greater  flexibility  in  budget  transfers.  Only  51%  of  OPC  respondents  (strongly)  agreed  that  the  budget  of  the 
Erasmus+ Programme is sufficient to achieve the objectives set for the Programme (n = 584). ICF, 7.2.3 
407 For instance, from 3% (Not-for-profit European sport Events), 4% (Knowledge Alliances) or 15% (Strategic Partnerships 
for Youth) to 43% (Jean Monnet strand) in 2014. However, the 2016 selection resulted in improved success rates; e.g. 31% 
for Strategic Partnerships for Schools; ICF, tab. 4.10 for full data. 
408 No KA2 selection was above 25% rate over 2014-15. 
409 ICF, 7.1; NRS, 6.1.1 and OPC, 3.4 

65 

 
 
                                                            
respondents to the consultation agreed with the current budget distribution between the three 
Key Actions set in the Erasmus+ Regulation410. 

5.4.3.  Implementation modes and user-friendliness 

The  division  of  responsibilities,  as  inherited  from  predecessor  programmes,  between  the 
Commission,  National  Authorities,  National  Agencies  and  EACEA,  is  overall  clear  and  fit 
for purpose411. It has not significantly changed over the two programming periods, if it was 
not for the decentralisation of the management of the higher education international (non-EU) 
programmes  and  some  changes  in  the  level  of  management  of  certain  types  of  cooperation 
projects. The only outstanding challenge stems from the rigidity inherent to the addition of the 
development-related requirements of the EU external cooperation funds which are also used 
to  promote  the  international  dimension  of  higher  education412.  This  requires  a  complicated 
management of multiple small budget envelopes with different rules at decentralised level and 
a disproportionate effort compared to the numbers of beneficiaries413. 

The  guiding  role  of  the  Commission  with  regard  to  the  programme  management  has  been 
acknowledged  by  63%  of  agencies,  due  to  efforts  made  in  terms  of  coordination  and 
communication. However, many National Agencies expressed a need for EACEA to enhance 
its cooperation with the national level.  

Above  all,  there  is  a  general  call  for  further  simplification  of  the  management  IT  tools. 
Lowering the burden of the application procedure through redefining the type as well as the 
volume of information required depending on the actions of the programme and making better 
use  of  the  reporting  system  (see  below)  would  contribute  to  continued  improvement  of  the 
programme's implementation414. 

5.4.4.  Efficiency gains and simplification through changes in the integrated 

programme 

Overall, efficiency gains were acknowledged by a majority of countries415, sectors416 and 
most  of  National  Agencies417.  In  general,  countries  recognised  a  certain  simplification  of 
administrative  procedures,  while  participants  expressed  concerns  about  application  and 
reporting. 

Due to the lack of time for adjustment between the adoption and the entry into force of 
the legal basis (i.e. less than 3 months) and for business continuity reason (e.g. the constraints 
of the academic year), all players (the European Commission, the implementing bodies of the 
programme  and  a  large  spectrum  of  programme  stakeholders)  had  to  go  through  a 
challenging inception phase to adapt to the many novelties of Erasmus+ over the first two 
years  of  its  implementation.  This  was  mainly  due  to  the  late  adoption  of  the  Erasmus+ 
Regulation (end of 2013), coupled with the necessity to implement the majority of the actions 

410 71% of OPC respondents (strongly) agreed that this distribution is appropriate (n = 480). 
411 80% of OPC respondents (n = 497) believe that the distribution of Erasmus+ Programme activities between centralised 
(through EACEA) and decentralised management (through National Agencies) is effective. 
412 See above background section about the different external instruments 
413 ICF, 7.4.4 and 7.5 
414 ICF, 7.1 and 7.4 
415 In 19 countries a positive opinion seems to prevail (BEnl, BG, EE, IE, EL, ES, FR, IT, CY, HU, PL, PT, SI, SE, UK, IS, 
FY, CH,TR). In 6 other countries, stakeholders see insufficient efficiency gains or no gains overall (CZ, LT, MT, NL, RO, 
SK, FI,). In a few countries, national authorities report different opinions depending on the NA, sector or key action; ICF, 7.5 
and NRS, 6.3 
416  It  is  evidenced  through  a  number  of  National  Reports  that  the  youth  sector  received  least  well  the  changes  in  the 
programme, claiming the new youth chapter to be less efficient than the Youth in Action Programme. NRS, 6.3 
417 79% of agencies states that they have seen efficiency gains compared to predecessor programmes. Nearly two in five say 
they have seen large efficiency gains; ICF, fig. 7.19 

66 

 
 
                                                            
already  by  the  first  trimester  of  2014418,  which  significantly  reduced  the  time  for  the 
Commission,  the National agencies and the  Executive Agency to  put  in place  the necessary 
normative framework and technical infrastructure to implement the programme smoothly. The 
new architecture of the programme, although it is now positively valued, led to an overhaul of 
previous  administrative  rules,  definition  of  new  criteria  and  new  ways  of  operating.  A  new 
generation  of  IT  tools  has  replaced  the  previous  one  and  new  automated  support  has  been 
introduced for a number of management tasks. 

National agencies and authorities recognised that initial challenges were gradually overcome 
and  that  there  has  been  a  steep  learning  curve  with  substantial  efficiency  improvement  and 
simplification  over  the  period  2014-2016.  The  challenge  of  digitalisation  that  was  put  in 
practice through the online management of applications and reports is a case in point, and so 
is  the  generalisation  of  simplified  grants419.  The  programme  management  has  now  reached 
cruising speed. 

On the positive side, different stakeholders and countries have acknowledged the difference 
made  with  the  introduction  of:  a)  simplified  forms  of  grants,  mainly  lump  sums  and  unit 
costs420, b) uniform application forms, c) digitalised application procedure421, d) possibility to 
apply as an institution instead of as a participant, and e) a single website to get easier access to 
all programme information 

On  the  negative  side  some  other  stakeholders  and  countries  did  not  consider  that  the 
integrated  programme  makes  it  easier  for  potential  applicants  to  understand  the  funding 
opportunities and even claimed that the standardised approaches (e.g. applications, reporting) 
made  work  more  time  consuming  than  in  the  past422.  Although  acknowledging  some 
administrative  simplification  introduced  with  Erasmus+,  according  to  National  Reports423, 
participants  express  concerns  about  too  complex  application  and  reporting  procedures.  For 
instance, beneficiaries responded that forms tend to be repetitive and some questions are quite 
long and difficult to interpret. 

Overall there are a number of areas that need further improvement and simplification, such 
as:  a)  better  field-customisation  of  the  unified  procedures,  b)  further  simplification  of  the 
application and reporting procedures, c) making the programme more accessible to smaller 

418  The  first  deadline  for  submitting  grant  proposals  for  mobility  and  cooperation  actions  under  Erasmus+  was  1 February 
2014  
419 Study Mid-term review of simplified grants used in the Erasmus+ programme, DG EAC/PPMI, July 2017 
420 Simplified grants gave overall satisfaction to both National Agencies and projects’ coordinators. They have reduced their 
workload,  simplified  budget  planning,  as  well  as  project  management  and  reporting.  They  give  more  flexibility  to 
participants,  who  can  manage  what  they  receive  according  to  their  needs.  Most  respondents  advocated  against  any  major 
changes  in  the  funding  rules,  due  to  efforts  that  were  required  to  adjust  to  the  current  system.  It  has  nevertheless  been 
suggested adjusting the level rates in some cases, especially for the longest distance bands, as well as introducing a number of 
fine-tuning changes. See for specific evaluation findings: Study Mid-term review of simplified grants used in the Erasmus+ 
programme, DG EAC/PPMI, July 2017 
421 92% of OPC respondents (strongly) agreed that the digitalisation of Erasmus+ is a progress (n = 338), and 89% that the 
user-friendliness of IT tools in the Erasmus+ Programme has improved over time (n = 597), in particular as from 2016, when 
IT tools were substantially streamlined. 
422  31%  of  agency  respondents  (rather/strongly)  disagree  that  the  integrated  programme  makes  it  easier  for  potential 
applicants  to  understand  the  funding  opportunities.  This  negative  sentiment  is  corroborated  by  several  key  interviews.  A 
majority of interviewees (national) claimed that the standardised approaches (e.g. applications, reporting) make work much 
time consuming than in the past. Benefits were however pointed by several interviewees. 
423 BEnl, BG, CZ, DK, DE, EE, IE, EL, ES, FR, HR, IT, CY, LU, HU, MT, LT, LV, NL, AT, PT, RO, SK, FI, SE, UK, IS, 
FY, LI, NO, CH, TR.  

67 

 
 
                                                            
organisations424,  d)  decreasing  the  time  required  for  project  management,  especially  in 
KA2425, and e) making the terminology used more user-friendly426. 

In  comparison  with  the  previous  programmes,  the  main  changes  which  agencies  associate 
with efficiency gains are online applications and reporting as well as the fact that there is a 
smaller diversity of types of actions to manage. However, the agencies are not very positive 
about  the  evolution  of  their  own  administrative  workload.  There  is  also  room  for 
improvement  concerning  the  lowering  of  the  administrative  burden,  as  shown  in  the  graph 
below427. 

To what extent do you agree with the following statements about efficiency gains 
for your agency under Erasmus + compared to predecessor programmes? 

Source: Survey of programme agencies  

The  rationale  of  the  economies  of  scale  expected  from  the  consolidation  of  several 
programmes into one was that an integrated programme with fewer but bigger actions/projects 
would  be  simpler  to  administer,  while  the  use  of  unit  costs  would  further  drive  the 
management costs down. If the model proposed in 2011 by the Commission to have a single 
National  Agency  per  programme  country  had  been  adopted  in  2013,  it  would  have  allowed 
according to estimates "to reach a cumulative effect of these simplifications to a productivity 

424 Regarding the performance of success rates, country reports (BEnl, CZ, DE, EE, EL, ES, FR, HR, IT, CY, LT, HU, MT, 
NL,  AT,  PL,  PT,  RO,  SK,  SE,  UK,  FY,  CH,  TR)  alert  that  success  rates  are  very  low  in  some  of  the  actions  and  fields. 
Success rates appear to be particularly low in KA2, the youth field and adult education sectors. 
425 Administrative procedures in KA2 are perceived particularly time-consuming (DK, LT, LV, HU, MT, NL, AT, LI, CH).  
426 For instance, in the context of school education, references to "innovation", "intellectual outputs" and "multiplier events" 
cannot be expected to be well understood by stakeholders 
427 ICF, 7.5 

68 

 
 
 
 
                                                            
increase  of  around  40%"428.  This  objective  however  could  not  be  a  target  for  Erasmus+,  as 
most countries continued to manage the programme through several National Agencies. 

Management fees for NAs – LLP, Youth in Action and Erasmus+ (% of overall funding) 

Source: ICF calculations based on a review of annual work programmes (% of overall funding) 

The management costs of National Agencies compared to the programme value are smaller 
than  under  predecessor  programmes,  when  looking  at  LLP  and  Youth  in  Action.  The 
comparison  looks  inevitably  less  positive  when  taking  into  account  the  overall  budgets  of 
predecessor international higher education programmes (Erasmus Mundus, Tempus, etc.), as 
their  complexity  reduces  the  average  efficiency  of  Erasmus+.  This  is  because  the 
decentralised  actions  with  partner  countries  have  specific  criteria  and  12  different  budget 
envelopes within the EU external relations funds. In any case, the comparison is not a simple 
one. There are important limitations in assessing the change of management fees: a) the base 
is not comparable between 2007-2013 and 2014-2016, as some actions that used to be grants 
managed  by  National  Agencies  have  been  moved  to  agencies'  core  budget  and  b)  the 
comparison with management costs of the Executive Agency is not available since EACEA 
used  to  manage  all  international  higher  education  mobility  actions,  of  which  an  important 
share has been decentralised to National Agencies. 

Nonetheless more efficiency gains are still expected to materialise, as the Erasmus+ budget 
will  increase  in  the  second  half  of  the  programming  period  whereas  the  management  costs 
will proportionally decrease. It is therefore too early to conclude at mid-term and the actual 
cumulative effect on efficiency will have to be measured after 2020. 

5.4.5.  Monitoring mechanisms 

The  programme  monitoring  has  seen  major  improvements  with  Erasmus+  compared  to 
the predecessor programmes. Much more qualitative (e.g. beneficiary surveys showing results 
beyond  outputs)  and  comprehensive  data  is  now  available  (e.g.  international  HE  mobility 
being  now  covered  as  well)429.  Data  is  also  more  systematically  used  and  disseminated 
through  new  tools  dedicated  to  Erasmus+,  namely  an  on-line  Dashboard  for  internal 
management purpose and an Annual Report for public accountability purpose430. 

Erasmus+  was  expected  to  have  a  much  clearer  performance-based  management  than 
predecessor  programmes  and  indeed  arrangements  have  been  put  in  place  to  allow  regular 
monitoring:  key performance indicators have been  defined  in  the legal basis as  well as  in 

428 Erasmus+ Impact Assessment, Commission Staff Working Paper, SEC(2011) 1402 of 23.11.2011 
429 See respective sections on method and effectiveness, as well as annex 3 
430 http://ec.europa.eu/programmes/erasmus-plus/about/statistics_en  

69 

 
 
 
 
 
                                                            
DG  EAC's  Strategic  Plan431,  data  is  collected  on  ongoing  basis  in  relation  to  most  of  these 
indicators and targets have been set432. 

However, as partly discussed in the section on effectiveness, to lower the reporting burden 
on beneficiaries433, there is room for improvement when it comes to: a) clarity and relevance 
of  some  output  indicators,  as  well  as  of  the  quality  of  the  data434,  b)  robustness  of  the  self-
reported results indicators, c) proportionality between the related burden on beneficiaries and 
the  actual  use  of  the  data,  d) balance  of  monitoring  efforts  according  to  types  of  action,  e) 
user-friendliness  and  further  inter-operability  of  IT tools435,  f)  and  promotion  of  truly 
performance-based management at the level of programme agencies. At Commission's level, 
the  reporting  is  used  primarily  for  accountability;  greater  use  of  the  information  should  be 
made for programme monitoring and continuous improvement purposes436. 

When looking at comparable national programmes, the monitoring arrangements of Erasmus+ 
are  nevertheless  in  line  with  existing  practices  and  perform  well,  given  the  size  of  the 
programme437.  Availability  of  objective  data  for  results  (e.g.  based  on  actual  pre-post 
language proficiency assessment through the OLS), as well as availability of large volume of 
self-reported  feedback  collected  through  systematic  beneficiary  surveys  are  two  areas  in 
which Erasmus+ monitoring practice can even be considered better than that of other national 
comparable programmes438. 

Overall, the monitoring mechanisms are effective when it comes to providing an up-to-date 
view of programme implementation to the Commission, the agencies and national authorities 
but  also  to  the  general  public,  in  particular  via  the  Erasmus+  Project  results  platform439,  as 
discussed under "dissemination". However, there is less evidence of the use of the data for 
management  and  planning  purposes,  which  may  be  due  to  the  fact  that  the  monitoring 
arrangements have only been enhanced since a couple of years440.  

5.4.6.  Anti-fraud measures 

Overall  there  is  a  limited  scope  for  fraud  in  the  types  of  actions  carried  out  under  the 
programme. The main concerns are project coordinators failing to honour obligations to other 
partners, and possible multiple submissions of project applications. 

According  to  the  programme  legal  basis,  a  system  of  controls  has  been  in  place  to  prevent 
fraud and irregularities at the EU and national levels441. The number of fraud cases reported is 
very low compared to the scale of the programme, with agencies proceeding to recover the 

431 https://ec.europa.eu/info/publications/strategic-plan-2016-2020-education-and-culture_en  
432 See annex 5e reporting on all output and result indicators from the legal basis 
433 Disproportionately more information is collected for KA1 compared to other Key Actions; ICF, 7.6.1 
434 E.g. various definitions of disadvantaged learners or why is country coverage is relevant indicator for Jean Monnet 
435 As a legacy of the past, several IT tools co-exist supporting various monitoring functions, such as control and detection of 
irregularities, accountability and transparency, as well as programme management. See annex 3 for description of tools 
436 Only anecdotic examples have been found of agencies that take ownership of the data available and use it for their own 
purposes of management and evaluation; ICF, 7.6 
437  Looking  at  the  comparator  schemes,  the  level  of  monitoring  is  highly  variable.  For  instance,  four  programmes  collect 
feedback  from  the  participants  (CEEPUS,  CTEP,  Denmark-USA  programme,  German-French  elementary  school  teacher 
exchange);  others  gather  regular  annual  monitoring  data  (AKTION).  Some  perform  regular  evaluations,  for  example,  the 
IACOBUS Programme started in 2014 and its activities in 2014-2015 have already been evaluated. The final evaluation of 
the  International  Inspiration Programme  (2007-2014)  was  conducted  by  an  independent  evaluator.  Prämienprogramm  has 
annual reports for 2013 and 2014 and was audited by the German Federal Court of Auditors. 
438 ICF, 7.6.5 
439 http://ec.europa.eu/programmes/erasmus-plus/projects  
440 ICF, 7.6.1 
441 Guide for National Agencies: Implementing the Erasmus + programme 

70 

 
 
                                                            
necessary  amounts442.  The  vast  majority  of  National  Agencies  (71%)  consider  that  the 
measures  are  fully  sufficient443.  Therefore  the  anti-fraud  strategy  in  vigour  is  considered 
appropriate. 
A few agency respondents  made  suggestions, such as  using IT-tools  to verify plagiarism444. 
As  the  programme  moves  to  its  later  phases,  more  checks  could  be  performed  on  the 
accumulated population of grants. In any case, the ratio of administrative burden compared to 
the low number of cases reported and the money recovered is to be carefully considered. 

442 No more than one or two cases per country in average and an overall low error rate indicated through primary checks of 
less than 1%, in line with the predecessor programmes (DG EAC's Annual Activity Report 2016). 
443 ICF, fig. 7.23 
444 ICF, 7.7 

71 

 
 
 
 
                                                            
5.5.  European added value 

The  evaluation  highlights  the  strong  European  added  value445  of  EU  programmes  in  the 
fields of education, training and youth446.  

The  additional  benefit  of  Erasmus+  was  compared  to  what  could  be  achieved  at  regional, 
national  or  other  international  level  (see  5.5.1),  to  what  was  achieved  by  EU  predecessor 
programmes  (see  5.5.2)  as  well  as  to  what  would  be  the  most  likely  consequences  of 
discontinuing the programme (see 5.5.3).  
The  European  added  value  of  Erasmus+  has  been  informed  by  6  evaluation  questions  and 
assessed against the following criteria that examined to what extent Erasmus+ supports: 

  a scale of actions between countries which would not otherwise be achieved (volume 

effects); 
target groups or sectors that would not be otherwise covered (scope effects); 

 
  processes  which  participating  countries  translate  further  into  their  own  practice 

(process effects) 

  cooperation between programme countries (integration effects) 
 

innovation  which  would  not  be  otherwise  mainstreamed  so  widely  (innovation 
effects); 

5.5.1.  European  added  value  as  compared  to  what  could  be  achieved  at 

regional, national or other international level 

Firstly, two evidence sources show clear added value of the programme due to its strong 
volume effects447, except in the cases of the Student Loan Guarantee Facility448, sport449 and 
Jean  Monnet  activities450.  The  programme  manages  to  reach  out  to  substantially  more 
learners, staff and organisations than all comparable programmes reviewed taken together, as 
well  as  groups  that  are  much  more  rarely  covered  otherwise.  The  benchmarking  analysis 
shows in particular that the scale of Erasmus+ is much bigger than that of other comparable 
schemes  reviewed  in  the  fields  of  education,  training,  youth  and  sport.  There  are  only  few 
other actions that support credit mobility and they are of a much smaller scale451. In terms of 
"market share", it is likely that Erasmus+ represents about half of credit mobility in EU higher 
education452.  It  is  also  likely  that  the  share  of  Erasmus+  in  VET  mobility  is  significant, 

445 European added value analysis looks for changes due to the EU programme over and above what could reasonably have 
been expected from actions by other players let alone or from no action at all. It draws on the findings of the other evaluation 
criteria, to conclude whether the programme is still justified. 
446 ICF, 8; NRS 7; OPC 3.5 
447 The assessment of volume effects is mostly based on the benchmarking of Erasmus+ against comparator programmes and 
the agency survey. 
448 For the Student Loan Guarantee Facility (SLGF), the European added value has not yet reached its full potential given the 
very limited scale of the action at the time of this evaluation. 
449  In  the  sport  sector,  the  assessment  of  European  added  value  is  made  difficult  by  the  fact  that  the  strand  is  too  recent. 
Another challenge is the fact that limited funding appears spread over too many different issues. 
450  The  European  added  value  of  Jean  Monnet  activities  has  declined  over  the  years  since  the  programme  was  initially 
launched, as the volume of research and teaching about the EU grew and has become a common practice in higher education 
within the EU. This growth cannot be related to Jean Monnet funding as this represents only a small share of research and 
teaching activities. There is however potential for greater European added value targeting other audiences which have fewer 
opportunities to learn about the EU. 
451 ICF, fig. 8.3: The comparable programmes comprise from 10 to 3,522 participants each. 
452  In  2015  the  total  number  of  EU  higher  education  student  who  benefitted  from  degree  mobility  had  reached  331,078 
(Source: Eurostat (2017), Degree mobile graduates from abroad, figure lacks input for Spain, Greece and Poland. Available 
at:  http://ec.europa.eu/eurostat/web/products-datasets/-/educ_uoe_mobg01).  Higher  education 
learner  credit  mobility 
supported by Erasmus+ that year had reached nearly the same number (301,267 individual students). As these two modes are 
mutually exclusive it can be assumed that there were around 650,000 mobile students in total. However this calculation is 
only  rough  as  the  Eurostat  data  covers  full  degree  mobility  and  not  credit  mobility.  There  is  no  comparable  data  about 

72 

 
 
 
 
                                                            
 
although  available  data  is  not  of  sufficient  quality  to  calculate  it453.  Most  respondents  from 
National Agencies consider that Erasmus+ funds more than 75% of actions of a given type, as 
illustrated below454. 

Perceived market share of Erasmus+ (number of responses) 

Source: Survey of programme agencies 

Secondly,  three  evidence  sources  point  to  the  conclusion  that  Erasmus+  scope  effects  are 
significant455.  Comparable  national  and  international  actions  do  exist  (e.g.  bilateral 
programmes,  philanthropic  actions,  etc.)  but  these  are  more  focused  on  mobility  than  on 
cooperation  actions456.  The  Erasmus+  programme  covers  fields  rarely  covered  by  other 
international interventions except in higher education. Only in the latter sector was it possible 
for  two-thirds  of  National  Agencies  to  identify  a  number  of  comparable  although  much 
smaller  programmes457.  For  all  other fields  or  types  of  actions  less  than half  of  respondents 
could identify comparators458. 

Erasmus+ has its own and unique boosting effect on mobility and cooperation in every 
field it targets. The findings from programme benchmarking459 and agencies' survey (see fig. 
below)  reinforce  each  other  to  provide  a  solid  case  in  that  respect.  None  of  the  comparable 
schemes  combine  as  much  as  Erasmus+,  in  terms  of  country,  sector  as  well  as  action 
coverage.  Moreover  with  a  dedicated  strand  (KA3)  since  2014,  the  EU  programme 

mobility for staff nor for students in other sectors. "Identifying mobile students, as well as their types of learning mobility, is 
a  key  challenge  for  developing  international  education  statistics  since  current  international  and  national  statistical  systems 
only report domestic educational activities undertaken within national boundaries" (OECD, Education at a Glance 2017). 
453 A pilot study on IVET and general youth learning mobility (Eurostat, 2015) estimated that roughly 3% of VET learners 
from 16 countries (estimated as about 320,000 learners by ICF) benefit from mobility actions, but the study acknowledges 
that the data is not of sufficiently good quality to be extrapolated. ICF, 8.2.2.2: Erasmus+ supported around 150,000 VET 
learners in these 16 countries over the first three years (common duration of VET studies). 
454 Only when looking at system level actions a relatively high number of agency respondents thought the share of Erasmus+ 
was below 50% 
455 Based on the agency survey, data about student mobility in general as well as data on comparable programmes 
456  Out  of  the  58  comparable  programmes  shortlisted  for  benchmarking  purpose,  2/3  supported  mobility,  while  only  1/3 
supported cooperation actions. 
457 ICF, fig. 8.1 
458 The agencies were asked about their awareness of other comparable interventions. The results  as regards sectors shows 
that around two thirds of respondents were aware of comparable actions in HE, slightly less than half in SE or in the Youth 
sector, less than one third in AE, fewer in VET. As for types of action, 43% of respondents from National Agencies knew of 
similar programmes that covered mobility of learners; this share was only of 42% and 38% for mobility of practitioners and 
cooperation among organisations respectively. 
459 Out of the total 58 benchmark programmes, around half cover SE, over 35% HE, below 20% VET or the Youth sector, 
10% AE. Their target groups are students in nearly 80% of the 58 schemes, teachers at around 22%, youth at around 17% and 
adults only around 10%. 

73 

 
 
 
 
 
                                                                                                                                                                                          
encompasses  system  level  cooperation  actions  that  are  very  rarely  supported  by  other 
programmes. 

In  addition,  the  evaluated  EU  programmes  show  specific  added  value  to  the  benefit  of 
disadvantaged  groups  whose  opportunities  for  mobility  largely  result  from  the  EU 
intervention,  more  than  from  national  initiatives.  This  is  all  the  more  true  in  the  youth  and 
sport fields where there are fewer comparable programmes460. Over two thirds of the agency 
respondents  agreed  with  the  notion  that  Erasmus+  offered  more  opportunities  for 
disadvantaged  groups  in  comparison  to  other  similar  schemes.  In  all  comparable  fields, 
Erasmus+ stands out as having the largest and most inclusive outreach461. 

Comparison between Erasmus+ and other comparable initiatives 

Source: Survey of agencies 

Both volume and scope effects magnify the impacts of the programme as detailed above. 

Thirdly, there are clear examples that the evaluated EU programmes have influenced other 
comparable programmes both in terms of process or types of actions supported. 

At  national  level,  Erasmus+  and  its  predecessors  have  contributed  to  shaping  the 
framework  for  student  and  staff  mobility  or  youth  exchanges  across  Europe462.  For 
instance,  quality  frameworks  -  in  particular  the  concept  of  a  European  Quality  Charter  for 
Mobility  introduced  first  for  higher education  and  extended  to  VET  due  to  its  success  - are 
also  used  for  national  schemes463.  Best  practices  of  EU  programme  management  are  often 
reported  to  be  transferred  to  national  programmes464.  Other  comparable  actions  sometimes 
follow the same rules465 as Erasmus+. Generally speaking, among National Agencies which 
manage  programmes  other  than  Erasmus+,  71%  of  respondents  acknowledge  a  spill  over 
effect  from  the  EU  programmes  under  review  to  their  own  interventions466.  These  are good 
examples  of  unintended positive effects of the evaluated programmes.  Furthermore, 91%  of 
respondents to the public consultation strongly agreed or agreed that lessons learnt from the 

460 Only 1 out of 6 benchmarking programmes had specific measures addressing the needs of disadvantaged groups. Only 3% 
were directed at the sport field. 
461 See sections on effectiveness and relevance about the share of participants with a disadvantaged background 
462 E.g. At EU level, the "Bologna process" for HE or the "Copenhagen process" for VET. At national level, the European 
concept of youth exchanges has inspired for instance the national programme "Youth meetings". 
463  E.g.  The  quality  assurance  and  administration  of  Nordplus  mobilities  (among  DK,  EE,  LV,  LT,  FI,  SE,  IS,  NO)  are 
inspired by Erasmus+. 
464 E.g. accessibility grants for participants with special needs (NordPlus Higher Education) 
465 E.g. Erasmus Belgica (among BE-de, BE-fr, BE-nl) follows the same rules as Erasmus+. 
466 ICF, fig. 8.5. N.B. the number of agency responses (n=60) is particularly limited on this point. 

74 

 
 
 
 
                                                            
Erasmus+ actions (which they were most aware of) are being applied elsewhere. In particular, 
processes  established  to  manage  mobility  have  been  mainstreamed  into  other  national  and 
European interventions. 

Several  spill-over  effects  on  process  have  also  been  identified  at  EU  level.  Erasmus  for 
young  entrepreneurs  shares  comparable  approaches  with  Erasmus+  mobility.  Schemes 
launched  recently,  such  as  the  European  Solidarity  Corps467  or  the  EU  AID  scheme  for 
volunteers  in  humanitarian  aid468,  follow  similar  selection  principles  as  the  European 
Voluntary Service (EVS). 

Fourthly, as discussed in the effectiveness section, the network analysis showed that there are 
overall  increasingly  good  levels  of  integration  and  cooperation  between  programme 
countries  with  some  that  were  initially  more  peripheral  moving  to  more  central  positions. 
Trends  in  interconnections  show  that  the  programme  is  not  necessarily  dominated  by  large 
countries. From a systemic perspective, the greatest added value comes from cooperation at 
the level of staff and organisations and that of policy makers and stakeholders. As regards the 
latter, many Open Method of Coordination activities organised in the context of ET2020, the 
EU  Youth  strategy  and the  EU  Work  Plan  for  sport  are  funded  through  Erasmus+.  The  EU 
programmes  reviewed  definitely  enable  cross-country  cooperation  and  integration  at  a  scale 
that is incomparable to other actions. 

While  its  volume,  scope,  process  and  integration  effects  are  substantial,  the  innovation 
effects  of  Erasmus+  appear  to  be  more  modest.  This  is  the  only  type  of  added  value 
analysed  where  the  results  are  less  positive.  The  evaluation  of  effectiveness  showed  above 
that while there are some examples of innovations that emanate from funded projects, these 
are rather ad-hoc, soft and of modest scale compared to the volume of projects funded rather 
than  significant  and  mainstreamed.  Although  Erasmus+  has  potential  to  enhance  innovation 
(collaborative approaches, specific KA3, brand attractiveness), its added value cannot be said 
as emanating significantly so far from a role model in that respect. 

5.5.2.  European  added  value  of  Erasmus+  as  compared  to  what  was 

achieved by its predecessor programmes 

Compared  to  its  predecessors,  Erasmus+  manages  to  reach  out  to  more  learners, 
practitioners and organisations. This scale potential results from the integration of several 
programmes into one and from a 40% increase in budget in comparison to 2007-2013. This 
increase has not yet fully materialised as most of it will be observed as of 2017. Meanwhile 
the integration has generated efficiency gains469. 

Through its well-known branding, the current programme benefits from a better visibility in 
media/social  media470,  but  also  in  policy  making  accompanied  with  a  strengthened 
alignment with policy priorities471. The fragmented nature of its predecessors did not give 
such  a  visibility.  Increased  attention  has  been  continuously  paid  to  synergies  between  the 
programme  and  policy  priorities  in  education,  training,  youth  and  sport.  In  particular,  the 
introduction of a Key Action devoted to system level support (KA3)  is a novelty valued by 

467 https://europa.eu/youth/solidarity_en 
468 http://ec.europa.eu/echo/what/humanitarian-aid/eu-aid-volunteers_en 
469  In  particular  thanks  to  the  decentralised  management  mode  through  agencies  but  also  increasingly  on  the  side  of 
beneficiaries; besides certain processes (e.g. to manage mobility) have been streamlined from one sector to the others. See 
section on efficiency 
470 See section on relevance 
471 See section on external coherence 

75 

 
 
 
                                                            
most respondents concerned. It is nevertheless too early to evaluate whether this programme 
will have achieved stronger system level changes. 

Thanks  to  the  integration  of  several  programmes,  the  degree  of  cross-sectoral  cooperation 
has  increased  compared  to  the  period  2007-2013.  This  is  mainly  due  to  the  rising 
participation of higher education organisations in other sectors and to a lesser extent to some 
increase in participation of companies and public authorities. 

The Erasmus+ programme has also been designed to pay more attention than previously to 
information about performance, which helps to allocate the budget more appropriately. The 
monitoring  of  Erasmus+  is  much  more  developed  than  in  the  past.  The  dissemination  and 
exploitation  of  project  results  is  underpinned  by  a  clearer  strategy  and  more  systematic 
publication on line. This represents actual progress, even though in both cases newly set-up 
tools  are  not  all  as  user-friendly  as  expected  and  though  data  remains  insufficiently 
analysed472. 

The international dimension, where it exists (i.e. in higher education and youth), has been 
made  clearer  with  Erasmus+.  The  integrated  programme  has  addressed  the  fragmentation 
issues raised in the interim evaluations of its predecessors (Erasmus Mundus, Tempus, Alfa). 
There  remains  room  for 
the  management,  visibility  and 
communication with regard to international actions. 

improvement  notably 

in 

From  one  programme  generation  to  the  next,  the  question  of  repetitive  actions  has  to  be 
assessed473. Here  again added value  is  reflected in the  differentiated level  of  repetitions. 
Each  year,  EU  programmes  reach  out  to  a  new  cohort  of  learners  (i.e.  though  mobility  is 
repetitive  it  addresses  different  audiences  each  year),  whereas  the  repeated  participation  of 
practitioners  and  organisations  in  the  funded  activities  over  time  enables  the  respective 
actions to achieve deeper effect. 

Overall,  most  of  respondents  to  the  Open  Public  Consultation  value  Erasmus+  for  its 
undisputed  benefits474.  More  than  95%  of  respondents  agreed  or  strongly  agreed  that  it  is 
funding  activities  which  would  not  have  been  made  possible  otherwise,  and  that  it 
contributes  to  improving  what  is  already  available  at  other  geographical  levels  for  the 
education, training, youth and sports sectors475. 

Most of the National Authority reports confirm the above findings by stating that Erasmus+ 
and its predecessor programmes produce effects that could not have been attained through 
actions  at  national  level  alone.  Country  reports  confirmed  by  the  public  consultation 
responses  highlight  the  impacts  of  Erasmus+  on  mobility  within  the  EU  and  beyond, 
cooperation  opportunities  and  internationalisation  of  organisations,  quality  improvements 
based  on  learning  from  others,  as  well  as  the  promotion  of  common  European  values  and 
intercultural awareness. Erasmus+  added  value is more strongly  praised in  the youth sector, 
while in the sports field, the European added value is rather acknowledged where activities go 
beyond  professional  sport,  such  as  cross-border  cooperation  between  grassroots-level  actors 
476. 

472 See section on efficiency 
473 See section on effectiveness 
474  OPC,  3.5  (n=1264  to  1044).  Opinions  among  individuals  replying  in  their  private  capacity  and  representatives  of  an 
organisation are consistent. 
475  Meanwhile  86%  (n=1111)  agreed  or  strongly  agreed  that  Erasmus+  is  providing  additional  support  to  already  existing 
activities at the national, European or international level. 
476 See for instance National Report of FI 

76 

 
 
                                                            
5.5.2  European  added  value  as compared  to  what  would  be  the  most  likely 
consequences of discontinuing the programme 

The absence of Erasmus+ would result in a steep decrease in the volume, but also in the 
quality,  equity  and  efficiency  of  both  mobility  and  cooperation  among  the  programme 
countries  and  worldwide477.  This  would  entail  in  turn  a  negative  impact  on  education, 
training, youth and sport sectors, as detailed under effectiveness above. 

As  shown  in  the  efficiency  section,  Erasmus+  is  managed  particularly  efficiently  due  to 
economies of scale. A decline in efficiency would result from the inevitable downscaling 
of coordinating structures. According to National Reports, collaboration between institutions 
and organisations at European level would be more complex to realise without Erasmus+. If 
National  Agencies478,  European  networks  with  central  support  function  and  European 
collaborative platforms would not exist, the efficiency of comparable schemes, not to say their 
mere existence, would probably be put at risk in many countries. 

Both for mobility and cooperation actions, the absence of Erasmus+ might be compensated in 
certain countries and sectors (mainly  higher education)  only in  scaling up other  comparable 
schemes. However, when asked in a Delphi survey about plausible alternatives to Erasmus+, 
the  experts  consider  that  these  partial  and  uncoordinated  compensatory  initiatives  would 
be highly unlikely to reach the scale of Erasmus+479. To achieve the full range of its results, 
a  greater  variety  of  actions  than  the  three  main  types  of  actions  under  Erasmus+  would  be 
needed. No single alternative would result in comparable results. 

Unequal access to mobility and cooperation abroad would once again become a reality, 
as  was  the  case  when  these  EU  programmes  did  not  exist.  National  Reports  note  that 
participation in alternative activities would probably be linked more directly to organisational 
or local resources. Therefore, differences in international opportunities between countries as 
well  as  between  sectors  would  increase  as  some  would  not  be  able  to  afford  funding 
international activities from national funds. As a consequence of the above, differences would 
also increase between wealthy and less wealthy organisations or individuals, as confirmed by 
the  monitoring  survey  among  higher  education  learners480  and  the  Delphi  survey  among 
experts481. 

This  in  turn  would  lead  to  a  decline  in  positive  adherence  to  the  European  project, 
especially  among  young  people,  and  a  much  weaker  international  outlook  of  staff.  It  would 
also  marginally  negatively  affect  their  transversal  skills,  especially  foreign  language 
acquisition,  though  other  channels  to  develop  these  skills  would  continue  being  used.  Over 
time the connections between organisations and staff would fade out, which would eventually 
have  a  knock-on  effect  on  learners  and  outcome  recognition.  Together  this  would  lead  to 
much smaller international outlook of education, training, youth and sport sectors, including 
beyond  the  EU,  which  would  generate  negative  externalities  given  the  globalised  nature  of 

477 Source: Delphi survey and ICF, 8.4. extrapolating from evidence presented in part 5 of this document 
478 NAs would certainly not survive a cut of EU programmes, even though they often manage other schemes since these are 
of a much smaller scale, as shown above. 
479 Confirmed for the international dimension of the programmes in Evaluation of the EU Development Co-operation Support 
to  Higher  Education  in  Partner  Countries  (2007-2014),  (2017),  Particip,  LeA,  ECDPM,  Ecorys,  already  mentioned: 
"Probably no single EU MS or even group of MS on their own would be well placed position to take the lead in organising 
and managing a highly complex programme in support of global HE." 
480 Two in five higher education learners would have not gone abroad without receiving the Erasmus+ grant (ICF, tab. 6.12). 
However, one in five would have probably gone abroad also without receiving it with the remaining one third being unclear. 
This suggests that there is some level of deadweight in the higher education mobility supported but this remains a minority of 
beneficiaries. Comparable data is not available for VET in Erasmus+ monitoring survey. 
481  According  to  the  Delphi  expert  survey,  given  the  budgetary  constraints,  alternative  funding  sources,  including  from 
organisations and beneficiaries themselves, would not compensate in the same scale. This would in turn deepen the financial 
barriers to access. Higher education would most likely see some compensation but inequality between sectors would increase. 

77 

 
 
                                                            
modern  economies.  In  other  words,  bringing  Erasmus+  to  an  end  would  leave  negative 
effects way beyond its direct beneficiaries. 

In  a  nutshell,  it  might  be  worth  quoting  the  High  Level  Group  on  Education  and  Training 
concluding under the Estonian Presidency of the EU Council that "The Erasmus+ programme 
and  its  preceding  programmes  have  'created  the  first  generation  of  young  Europeans  and 
therefore  have  the  highest  possible  added  value  for  Europe  as  well  as  the  potential  to 
promote common European values, empower young people, tackle socio-economic challenges 
and foster competitiveness." 482 

482 Meeting of the High Level Group on Education and Training, 14-16 June 2017, Tallinn 

78 

 
 
 
 
                                                            
6.  CONCLUSIONS 

Overall,  the  programmes  evaluated  have  proved  to  be  highly  effective.  Erasmus+  appears 
more coherent, relevant and only partly more efficient than its predecessors.  

The  external  evaluator  made  11  recommendations483  to  the  Commission  to  address 
weaknesses of the current Erasmus+  programme. These  recommendations  will be followed-
up  as  stated  in  the  Commission  report  on  the  mid-term  evaluation  of  the  Erasmus+ 
programme (2014-2020). 

  Five  recommendations  are  cross-cutting  (with  a  view  to  boosting  respectively: 
inclusiveness for more vulnerable groups; strategic investment in the sectors with highest 
potential  performance  (schools,  VET,  youth);  policy  prioritisation;  relevance  of 
innovation  potential  at  sector  level;  systemic  impact  of  projects  through  further 
involvement of policy makers); 

  Four  recommendations  concern  specific  areas  (aiming  at  addressing  the  relative 
weaknesses  of  the  Student  Loan  Guarantee  Facility;  Jean  Monnet  activities;  adult 
education; sport); 

  Two  recommendations  are  more  process-oriented  (aiming  at  simplification  and  better 

monitoring). 

Effectiveness 

The  current  programme  and  its  predecessors  deliver  a  unique  package  of  outcomes  in  the 
field  of  education,  training,  youth  and  sport.  When  concerns  are  expressed,  these  relate  to 
specific parts of the programme or to certain modalities of disbursement. 

At  mid-term,  Erasmus+  is  on  track  to  achieve  or  to  exceed  most  of  its  performance 
indicators,  as  set  in  the  legal  basis.  Beneficiaries  report  above  90%  satisfaction  rates  for 
learners  and  even  higher  for  staff.  In  the  period  2007–2016,  all  programmes  reviewed  have 
supported  together  more  than  4.3 million  learners  and  880,000  practitioners  as  well  as 
940,000  organisations.  In  terms  of  volume  of  outputs,  the  main  exception  concerns  the 
Student Loan Guarantee Facility. Even if the feedback received from its first users has been 
positive, it has not yet lived up to volume expectations due to delays of its launch, low take-up 
among financial institutions and lack of awareness among students. 

The  evaluation  went  beyond  these  indicators,  which  are  not  all  equally  relevant.  A 
counterfactual  approach  (quantifying  the  contribution  of  the  programme  in  measuring  the 
difference in results between beneficiaries and non-beneficiaries) was used for the first time 
across  the  programme,  making  this  evaluation  very  reliable  and  more  valid  than  previous 
ones.  However  such  an  approach  cannot  exclude  the  possibility  that  the  difference  in 
outcomes between beneficiaries and non-beneficiaries could be partially due to the selection 
into the programme (in attracting the most motived individuals for instance) rather than purely 
the effect of the programme. Based on this very sound methodology, the evaluation identified 
a  broad  range  of  results  and  impacts  at  the  level  of  learners  (with  particularly  outstanding 
effects  of  mobility),  practitioners,  organisations  (in  particular  their  internationalisation, 
including  beyond  the  EU)  and  also,  to  a  certain  extent,  systems  and  policies  (fostering  the 
recognition of learning outcomes across borders). 

At individual level 

483 ICF, 9 

79 

 
 
                                                            
For most individuals, including when compared with the situation of non-beneficiaries, their 
experience  abroad  has  contributed  not  only  to  their  skills'  development  and  future  career 
prospects  but  also  to  deep  changes  to  their  personal  development.  The  programmes 
evaluated  have  stimulated  networking  among  both  learners  and  staff.  Finally,  they  have 
contributed to the development of openness to other cultures and positive attitude towards 
the European project. Yet the evaluation points to the need to do more to reach out to the 
more vulnerable in society and to facilitate the participation of smaller-size organisations. 

At organisational level 

The  programmes  reviewed  have  demonstrated  a  range  of  soft  effects  on  organisations, 
particularly a clear internationalisation of organisations, in and outside the EU. The changes 
at  organisational  level  are  however  progressive  and  small  scale  and  continued 
participation is needed for deeper transformations. 

At system level 

In the long run, the programmes have led to a cultural shift in Europe in the way learning 
mobility  abroad  is  perceived  and  its  learning  outcomes  are  validated  and  recognised. 
The  programmes  evaluated  have  also  been  important  for  increasing  the  EU's  global 
outreach.  

Though less visible, the evaluation also confirmed that the programmes reviewed have had 
a direct impact on education, training, youth and sport policies where the related budget 
was large enough or had an indirect effect by funding policy cooperation between authorities. 
The  system  level  changes  through  critical  mass  are  much  clearer  in  the  higher  education 
sector than in other sectors which receive comparatively much less funding. 

Systemic  effectiveness  is  less  in  evidence  for  Jean  Monnet,  adult  learning  and  sport, 
where  funded  EU  actions  lack  the  critical  mass  and/or  a  sufficiently  relevant  focus.  The 
evaluation underlines in particular that the impact in the adult learning sector is diluted due to 
the fragmented and diverse nature of the sector. 

More  generally,  the  resources  for  mainstreaming  project  results  are  spread  across  too  many 
topics at EU level and are insufficient at national level. Although local innovation (producing 
something new for participating organisations) is achieved by the programme, this innovation 
does  not  enough  follow  the  state  of  the  art  developments  in  a  given  sector.  Lastly,  the 
evidence  of  the  exploitation  of  project  results  by  policy  makers  is  not  always  clear, 
especially when the latter are not engaged in the project from the beginning. 

Coherence 

The  evaluation  found  a  high  external  coherence  between  Erasmus+  and  other  relevant  EU 
policies and programmes (e.g. European  Social Fund, Horizon 2020).  Although  the level of 
synergies differs, it is notable that the evaluation detected very few overlaps. 

The internal coherence of the programme resulting from the lifelong learning coverage 
is strong as well. The further integration of fields often kept separate at domestic level into a 
single  EU  programme  is  increasingly  perceived  as  facilitating  complementarities  and 
international outlook. This consistent scope sends the message that the learning opportunities 
offered  are  equally  important  for  the  development  of  people,  be  it  through  formal  or  non-
formal and informal learning. 

There  is  a  strong  degree  of  cross-sectoral  cooperation  within  Erasmus+,  which  has 
increased sharply compared to predecessor programmes. The majority of cooperation projects 
in  all  sectors  include  at  least  one  organisation  which  can  be  considered  as  coming  from 
80 

 
 
another  sector.  There  is  an  overall  consensus  that  an  integrated  programme  boosted  the 
promotion of the actions and the visibility of the different sectors. 

Coherence can still be improved in relation to sport in reducing overlaps with non formal 
education activities. 

Relevance 

The evaluation showed that Erasmus+ is better aligned with EU policies than its predecessor 
programmes and entails sufficient flexibility to adapt to EU-level emerging needs. However, 
to maximise the programme's impact, the evaluation recommends that priorities at selection 
stage be reduced and better focused. 

The  contribution  of  Erasmus+  to  a  more  cohesive  Union  is  also  clearly  evidenced.  The 
programme  reaches  out  to  disadvantaged  people  more  than  previous  EU  programmes  or 
comparable national schemes, although more still needs to be done to reach out to the more 
vulnerable  in  society  as  outlined  above.  The  geographical  balance  has  also  improved  with 
small countries and countries from Central and Eastern Europe being better integrated than in 
the predecessor programmes. 

The  evaluation  also  noted  that  there  is  potential  for  introducing  better  targeted  actions  to 
maximise the relevance of Jean Monnet activities. Teaching and research on EU matters is 
now relatively wide-spread in higher education. In contrast, there is a need to strengthen the 
understanding  of  European  integration  and  sense  of  belonging  to  Europe  amongst  the 
youngest generations. 

Efficiency 

The  economies  of  scale  linked  to  the  integration  of  several  programmes  did  not  deliver  all 
efficiency  gains  that  had  initially  been  anticipated  in  2011  based  on  a  "one  agency-one 
country" model which  was not  adopted by the legislator in  2013. Nevertheless, the  costs of 
programme  management  appear  reasonable  (6%  of  Erasmus+  administrative  and 
operational  budget)  and,  in  any  case,  lower  than  for  similar  programmes  at  national  level 
(14% in average). More efficiency gains are expected to materialise during the growing phase 
foreseen by the budget profile of Erasmus+ until 2020. 

The hybrid combination of different  programme management modes and bodies is fit for 
purpose  with  a  good  overall  coordination  and  no  major  inefficiencies  identified.  Through 
decentralised  actions,  the  programmes  get  close  to  their  target  audience  and  offer  the 
possibility  to  align  with  national  priorities.  Very  positive  feedback  was  received  on  the 
support provided by and to National Agencies. Centralised actions align more with EU level 
priorities and help to respond to urgent political priorities. Some National Agencies report a 
need  for  more  communication  with  the  Executive  Agency  (EACEA)  to  maximise  synergies 
between centralised and decentralised actions. 

The  first  two  years  of  Erasmus+'  implementation  were,  however,  very  challenging  for  all 
players.  This  was  mainly  due  to  the  late  adoption  of  the  Erasmus+  Regulation  (end  of 
2013),  coupled  with  the  necessity  to  implement  the  majority  of  the  actions  under  the  new 
architecture and with a new generation of IT tools already by the first trimester of 2014, 
which  significantly  reduced  the  time  for  the  Commission,  the  National  agencies  and  the 
Executive  Agency  to  put  in  place  the  necessary  normative  framework  and  technical 
infrastructure to implement the programme smoothly. National agencies and authorities have 
recognised that initial challenges have been gradually overcome and that there has been a 
steep  learning  curve  with  substantial  efficiency  improvement  over  the  period  2014-2016. 
Despite this difficult start, the programme has now reached its cruising speed. 

81 

 
 
Simplification 

The integration has led to greater simplicity of programme architecture, notwithstanding 
the separate chapters for Jean Monnet and sport, which is beneficial for both beneficiaries and 
those  in  charge  of  management.  It  also  makes  programme  monitoring  more  comprehensive 
and  clearer,  although  there  is  room  for  a  smarter  collection  and  a  better  exploitation  of 
data,  enhancing  transparency  and  accountability.  The  counterfactual  approach  (results 
compared between beneficiaries and non-beneficiaries) should be pursued and developed for 
evaluating the next programme. 

The procedural burden on beneficiaries (application, reporting, etc.) is the main recurrent 
issue.  Many  agree  that  Erasmus+  has  considerably  simplified  a  number  of  processes 
(simplified grants, digitalisation, VET Charter, linguistic support, etc.). However, procedures 
and IT management tools impose a burden on beneficiaries which is not always proportionate 
to the grant they receive.  

By  contrast,  the  evaluation  notes  that  the  simplification  process  may  have  gone  too  far  as 
regards  cooperation  projects.  Applicants  for  small  strategic  partnerships  are  too  often 
expected to meet the same requirements as applicants for large ones. Furthermore, it is hard to 
clearly assess the cost-effectiveness of these actions considering their relatively average level 
of  innovation.  There  is  not  enough  differentiation  made  in  the  way  of  handling  smaller 
(mainly  collaborative)  and  larger  (innovative)  projects.  The  evaluation  recommends 
simplifying  the  application  forms,  reviewing  the  award  criteria  to  better  reflect  key  success 
factors  for  effectiveness  and  strengthening  the  review  at  mid-term  in  particular  for  bigger 
projects. 

Another challenge hampering efficiency is linked to the funding for international higher 
education, which remains managed separately for each global region in a too rigid manner. 

European added value 

30  years  after  its  beginnings  in  the  field  of  higher  education,  Erasmus+  has  become  a 
flagship  programme  of  the  EU.  The  programme  is  highly  valued  by  the  general  public  as 
well as by all stakeholders. It is consistently identified by citizens as  one of the three most 
positive results of European integration. 

Erasmus+ remains unique. Whereas there are other schemes funding comparable actions at 
national level, these are much smaller both in volume and scope. It is unlikely that alternative 
measures  would  be  able 
its 
undisputable European added value. 

to  compensate  for  Erasmus+  funding,  demonstrating 

For the future, without prejudice to negotiations on the next Multi Financial Framework, the 
evaluation  concludes  that  it  would  be  relevant  to  reconsider  the  overall  budget.  It  also 
suggests  modifying  the  share-out  between  the  programme  sectors,  whereby  potential 
increases could be directed to sectors showing the highest performance, but which have, until 
now,  received  relatively  less  funding.  School  education  and  vocational  education  and 
training  (VET),  where  the  impact  is  not  yet  as  widespread  as  in  higher  education,  were 
identified  as  having  the  most  promising  potential  for  an  expanded  participation  in 
Erasmus+ activities in the coming years. The unmet demand and the potential for maximising 
effectiveness  call  for  a  stronger  investment  for  the  future  development  of  the  programme.

82 

 
 
Annex 1: Procedural information 

1.  LEAD DG, DeCIDE PLANNING/CWP REFERENCES 

  DG Education, Youth, Sport and Culture (DG EAC) 
  EAC/2015/014 

2.  ORGANISATION AND TIMING 

This  evaluation  has  been  steered  by  DG  Education,  Youth,  Sport  and  Culture  since  July 
2017484 under the scrutiny of an inter-service group (ISG) comprising of representatives of 
DG BUDG, DEVCO, EAC, EEAS, EMPL, FPI, HOME, JRC, NEAR, SG, as well as EESC. 

The ISG was consulted at each stage of the evaluation process and reviewed each deliverable 
produced by the contractor as well as this Staff Working Document. 

ISG meetings took place on: 

  9 September 2015 (Kick-off), 

  18 July 2016 (Inception report); 

  21 November 2016 (Interim report 1); 

  24 March 2017 (Interim report 2); 

  10 July 2017 (Draft final report); 

  11 September 2017 (Final report) 

The ISG was consulted on the draft Staff Working Document on 29 September and informed 
about the Regulatory Scrutiny Board's opinion on 27 November. 

3.  EXCEPTIONS TO THE BETTER REGULATION GUIDELINES 

None 

4.  CONSULTATION OF THE RSB (IF APPLICABLE) 

The Regulatory Scrutiny Board was consulted on 16 October and met on 15 November 
2017. 

The Board acknowledged in its opinion with comments  485 the significant efforts of data and 
evidence collection. It also noted the good methodology. The Board gave a positive opinion, 
with comments to improve the document. All have been addressed. 

The following changes have been made to this Staff Working Document, in response to the 
Board’s main considerations: 

484 Ares(2015)3111961 - 24/07/2015 - Invitation to participate in an inter-service steering group for the 
evaluation of Erasmus+ programme 
485 Ares(2017)5629740 - 17/11/2017 

83 

 
 
 
                                                            
Regulatory 
Scrutiny Board's 
recommendation 

(1) The 
conclusions are not 
sufficiently 
balanced and 
precise. They do 
not accurately 
reflect some of the 
programme 
strengths and 
weaknesses that the 
evaluation and its 
supporting study 
identified. 

They also do not 
provide enough 
guidance for future 
decisions about the 
programme. 

Changes made to the Staff Working Document 

Both the Executive summary and the conclusions have been revised to outline 
more clearly the weaknesses pointed out by the external evaluator in its 
conclusions and across the Staff Working Document. 

Any evaluation Staff Working Document has to remain backward-looking 
according to Better Regulation guidelines. For that reason, the 
recommendations are addressed in the Commission report to Council and 
Parliament, without anticipating the conclusions of a future Impact assessment. 

The Commission can accept all 11 recommendations received from the external 
evaluator (section 9 of ICF's final report) to a smaller or larger extent. 

- 5 recommendations are cross-cutting (innovation; inclusiveness; strategic 
investment in the best place sectors; systemic impact of projects; policy 
prioritisation at selection stage) 

- 4 recommendations concern a specific sector (SLGF; Jean Monnet actions; 
Adult education; Sport) 

- 2 recommendations deal with process (simplification; monitoring) 

(2) The report does 
not clearly define 
the scope of the 
evaluation and the 
baseline/benchmar
ks against which it 
assesses the 
programme. 

The baseline for this evaluation is the period 2007-2013 unless otherwise 
specified. This is now stated explicitly in the introduction and in the section 
about the evaluation method (4.1 and 4.2). In addition, it has been specified 
more systematically, especially across the section on effectiveness (5.1), where 
the scope of a given finding concern all evaluated programmes (e.g. findings 
from evaluation surveys of beneficiaries cover 10 years) or Erasmus+ only (e.g. 
findings from monitoring surveys concern by definition only the current 
programme). 

The choice of an intervention logic (2.2) covering the two programming periods 
has been further justified (4.1) 

To better support the claim that Erasmus+ has achieved or exceeded most of its 
targets, a new annex (5e) has been produced reporting on all indicators from the 
legal basis against targets set in DG EAC's Strategic Plan 

The Staff Working Document (5.1, 5.4.1, 5.4.4, 5.4.5) has been completed with 
more information about measures taken since 2014 to improve monitoring, 
efficiency and to simplify the programme implementation. It is also explained 
now why the Impact assessment regarding efficiency gains cannot be regarded 
as a reference (5.4.4). 

The Commission has not waited for the mid-term evaluation to start 
working on simplification (e.g. application forms). The level of complaints 
about the administrative burden has decreased as of 2016 after a steep learning 
curve on the new programme. 

(3) The report does 
not adequately 
address 
simplification and 
efficiency 
improvement. 

84 

 
 
5.  EVIDENCE, SOURCES AND QUALITY 

Sources of evidence 

The  following  box  provides  an  overview  of  the  data  sources  from  which  the  evidence  was 
drawn. A detailed description of the individual methods is provided in Annex 3. 

  Literature review (150 sources) 

  Key informant interviews (190 interviews) 

  National reports (34 countries) 

  Open  Public  Consultation  (4,786  responses  of  which  1,800  responses  were  fully 

exploitable; 24 position papers submitted) 

  Social media analysis (725,678 posts) 

  Benchmarking (18 comparable national/transnational programmes) 

  Programme database analysis (all beneficiaries) including network analysis 

  Monitoring surveys (950,000 respondents - KA1) 

  Online Linguistic Support (523,238 participants) 

  Beneficiary surveys and control groups (over 47,000 respondents) 

  Survey of socio-economic organisations (947 responses) 

  Agencies  survey  (130  responses  from  National  Agencies  or  the  Executive  Agency 

EACEA) 

  Experts survey (1,122 responses from project assessors) 

  Case studies (38 organisations or policy cases; 233 respondents) 

  Review of selected projects’ outputs (386 outputs) 

  Expert panel assessment of projects’ outputs (100 projects) 

  Jean Monnet actions (13,183 survey respondents; 5 interviews) 

  Student Loan Guarantee Facility (SLGF) (219 survey respondents; 33 interviews) 

  eTwinning Plus pilot action (2,967 respondents; 31 interviews) 

85 

 
 
 
 
 
Expert advice 

National Reports 

Programme countries have submitted their own evaluation reports on the implementation and 
impact  of  decentralised  actions  of  Erasmus+  in  their  respective  territories,  as  required  by 
Erasmus+  legal  basis.  Their  findings  are  summarised  in  the  National  reports  synthesis 
(NRS) drawn by the external evaluator486. 

Source 

Scope 

Volume 

National Reports 
(ICF, stand lone 
synthesis – "NRS") 

Programme country reports on 
the impact of decentralised 
actions submitted by June 2017 
as per Article 21(4) of 
Regulation (EU) 1288/2013 
and DG EAC’s guidelines487 

34 National reports 
(MT and UK transmitted 
their report to the 
Commission with a four 
month delay) 

Expert Panel 

Erasmus+  and  predecessor  programmes  fund  a  high  number  of  cooperation  projects  which 
result  in  outputs  (e.g.  handbook,  toolkits,  methodologies,  etc.)  that  are  aimed  to  be  used 
within  the  organisations  that  took  part  in  the  project  but  also  beyond.  To  get  a  better 
understanding  of  their  quality  and  dissemination  potential,  an  expert  panel  reviewed 
collectively the actual outputs of a selection of 100 cooperation projects. 

Source 

Scope 

Volume 

Expert panel 
assessment of projects’ 
outputs 
(ICF, Annexe 18) 

Assess projects’ outputs 
against a set of pre-
determined criteria  

100 projects 
(144 outputs) 

External consultant 

The external evaluator (ICF) contracted for this assignment488 has carried out since May 2016 
all tasks as required under the scrutiny of an inter-service group (ISG) and the daily steer of 
DG Education, Youth, Sport and Culture. 

Primary data were mainly collected from November 2016 to May 2017. 

The  only  significant  change  compared  to  the  initial  work  plan  was  limited  to  a  two-month 
delay of the (sub) contractor in launching the beneficiary surveys489. 

486 National reports synthesis (NRS) annexed to ICF final report 
487 Ares(2016)576506 of 2 February 2016 
488 ICF Consulting Services Ltd under specific contract – EAC-2016-0219 implementing Framework contract EAC/22/2013 
489  DG  EAC  wrote  to  ICF  to  mitigate  that  issue  in  March  2017.  The  delay  had  no  consequence  on  the  final 

report. 

86 

 
 
 
 
 
                                                            
Annex 2: Stakeholder consultation synopsis report  

This evaluation has drawn on data from direct consultations with various stakeholder groups 
involved in the implementation of the Erasmus+ and predecessor programmes.  

A4.1 Overview of main stakeholders consulted 

The table below provides an overview of the stakeholders consulted over the lifetime of the 
evaluation  through:  semi-structured  interviews  (for  the  purpose  of  the  key  informant 
interviews (KIIs) and the case studies), online surveys and an open public consultation. Other 
consultation activities are described at the end of the section.  

Table 1. 

Overview of stakeholder consultation  

Type of stakeholders consulted  Method  

Nbr of 
respondents/cases/records 

Beneficiary and control group 
surveys  

Learners: 24,037   beneficiaries 
and 2,695 from control group  

Staff: 20,155 beneficiaries and 928 
from control group  

Socioeconomic actors survey 

947 valid responses  

Experts survey 

1,122 valid responses 

Programme agencies survey 

130 valid responses 

Key informant interviews 

59 at EU level  

Case studies 

131 in 15 countries  

233 respondents 

38 case studies 

Student Loan Guarantee Facility 
student fair survey 

119 students and 100 exhibitors  

Open Public Consultation (OPC) 

1,800 valid responses  

Student Loan Guarantee Facility 
interviews 

33 interviews 

Beneficiaries (learners and 
practitioners) of mobility actions 
and cooperation actions as well as 
control groups  

Organisations other than the 
primary target group, i.e. 
companies, public authorities, civil 
society (other than youth 
organisations) 

Assessors supporting project 
selection and those supporting 
evaluation of final reports  

Agencies in charge of programme 
implementation  

One respondent per sector  

Key stakeholders in education and 
training, youth and sport 
(EU/national level) 

Staff, learners, leadership, project 
leaders and other stakeholders if 
relevant  

Students participating in student 
fairs for future (mobile) master’s 
students and exhibitors at these 
fairs 

General public including key 
stakeholders active in education 
and training, youth and sport 

Other: focused consultation  

(Non)-participating financial 
intermediaries and their 
representatives; national student 
loan schemes; HEIs and their 
representatives; National 
Authorities and agencies; 
representatives of students and the 
youth 

87 

 
 
 
 
 
Students studying about the EU – 
beneficiary students and non-
beneficiary academic staff 

Students studying about the EU – 
beneficiary students and non-
beneficiary academic staff 

Jean Monnet students survey  

332 beneficiaries and 1,015 non-
beneficiaries  

Jean Monnet section in the 
beneficiary student survey 

120 beneficiaries and 5,822 non-
beneficiaries 

Staff teaching about the EU – 
beneficiaries and non-beneficiaries  

Jean Monnet staff survey 

560 beneficiaries and 443 non-
beneficiaries  

Staff teaching about the EU – 
beneficiaries and non-beneficiaries 

Jean Monnet section in the 
beneficiary staff survey 

210 beneficiaries and 4,681 non-
beneficiaries 

Interviews with professors who 
have never applied for Jean 
Monnet 

Staff taking part in the pilot 
project 

Teachers in programme and 
partner countries, EU-level actors 
and key stakeholders 

Source: ICF  

Jean Monnet interviews 

5 interviews  

eTwinning Plus pilot survey490 

405 responses to partner countries 
survey  

2,562 responses to programme 
countries survey 

eTwinning Plus interviews 

31 interviews 

Based on the above, 68,675 stakeholders’ views were collated and analysed among which: 

  66,383  through  the  seven  above-mentioned  parallel  online  surveys491  among  which 

47,815 through the sole beneficiary and control group surveys  

  492 through interviews 

  1,800 through the OPC 

As the table also reflects, the range of stakeholders consulted was broad, encompassing both 
direct  and  less  direct  beneficiaries  of  the  programmes  at  the  individual  (learners  and 
practitioners), organisation and system level comprising (not in specific order): 

  general  public  including  a  variety  of  key  stakeholders  active  in  education  and 
training,  youth  and  sport  with  different  levels  of  knowledge  and  experience  with 
Erasmus+ and/or predecessor programmes: addressed through the OPC492 

  all  current  programme 

target  groups,  both  for  KA1  (mobility)  and  KA2 
(organisational  cooperation):  addressed  though  the  surveys  of  beneficiary  learners 
and staff and related control groups  

490 The Staff Working Document does not address this part of the contractor's assignment, not initially set out in 

the Terms of reference, as eTwinningPlus pilot action is not funded by the Erasmus+ programme. 

491  The  Jean  Monnet  separate  student  and  staff  surveys  outlined  in  the  table  are  counted  here  as  one  survey 

package. 

492 Overall, those respondents who replied as ‘organisations’ were primarily active (in order) in the higher education, school 
education, vocational education and training; adult education and youth fields. The remainder included respondents from 
other sectors and lastly from the sport field. Those who replied as individuals were for the majority (59%) employed in 
education and training, youth and sports. The remainder comprised learners (29%) in one of the fields above and 12% 
reporting they had a different role in relation to these fields. For further details, see external evaluation final report volume 6.  

88 

 
 
 
 
                                                            
  socioeconomic actors (i.e. companies, public authorities, civic organisations, sectoral 

bodies, etc.): addressed through the socioeconomic actors survey 

  project  assessors  (i.e.  experts  contracted  by  the  EACEA  and/or  the  NAs  to  assess 

project applications and reports): addressed through the experts survey 

  NAs and EACEA staff members: addressed through the programme agencies survey 
  Selected  EU  and  national  level493  key  informants  involved  with  programme(s) 
implementation (EU level) or drawing on it and/or its predecessors: key stakeholders/ 
key  stakeholder  organisations  representatives,  EC  officials  (EC  and  agencies 
(EACEA  and  ETF),  ministries’  representatives  (policy-making)494.  Those  were 
addressed through the Key informant interviews (KIIs) 

  Selected national level495 informants benefitting from the programme(s) in the three 
fields  above  comprising:  beneficiary  organisations’  leadership,  practitioners  and 
learners, targeted funded project leaders, other key stakeholders (where applicable). 
Those were addressed through the case studies   

Next  to  the  consultation  methods  above  that  were  designed  and  implemented  to  collate 
stakeholders’ opinions on the current programme’s (and/or predecessors where appropriate) 
performance  against  all  or  several  of  the  main  evaluation  criteria,  the  evaluation  team  also 
undertook ad hoc stakeholders consultation. This served to inform the evaluation of specific 
part or activities of the programme namely: Jean Monnet programme496 and the Student Loan 
Guarantee  Facility  (SLGF)497.  In  each  case,  the  consultation  methods  comprised  tailored 
surveys and interviews. For an overview of the types of stakeholders reached in both cases, 
see table above.  

Besides,  to  gain  complementary  insights  and  to  ultimately  assess  the  difference  in  results 
between Erasmus+ beneficiaries and their counterparts who did not take part, control groups 
of non-beneficiaries (targeted at practitioners498 and learners499) were set-up.  

Overall,  the  mix  of  consultation  activities  described  above  enabled  the  evaluation  to 
effectively  address  a  relevant  breadth  of  stakeholders.  This  in  turn  enabled  the  evaluation 
team  to  gain  insights  from  a  range  of  relevant  key  players  on  different  and  meanwhile 

493  Focusing  on  15  Programme  countries  as  per  ICF  proposal  and  agreed  during  the  inception  phase  of  the 

evaluation study: BG, CY, CZ, DK, ES, FR, DE, HU, IE, IT, LV, PL, SE, TR, UK 

494 National Authorities in charge of the Erasmus+ programme were not consulted because they contributed to 

the evaluation directly through their national reports, as laid down by the programme legal basis. 

495  Focusing  on  15  Programme  countries  as  per  ICF  proposal  and  agreed  during  the  inception  phase  of  the 

evaluation study: BG, CY, CZ, DK, ES, FR, DE, HU, IE, IT, LV, PL, SE, TR, UK 

496  Considering  the  specific  features  of  the  Jean  Monnet  programme  as  one  of  the  two  standalone  Erasmus+ 
chapters (with Sport), the evaluation Steering Committee agreed  with ICF, during the inception phase, that 
the  latter  would  be  make  the  object  of  a  standalone  evaluation  (for  details,  see  external  evaluation  final 
report, volume 3) .     

497 As one the novelties introduced through Erasmus+, the SLGF made the object of a focused evaluation (for 

details, see external evaluation final report, volume 2).   

498 i.e. involved in all three fields and subfields (i.e. school education, VET, HE, adult education). 

499 i.e. enrolled in school education, VET, HE and Erasmus+ Youth exchanges and EVS.  

89 

 
 
 
                                                            
complementary  dimensions  of  the  programme(s)’s  performance  and/or  on  suggestions 
towards the future programming period. 

With regard to the representativeness of the data, sample sizes (more particularly applicable 
to the main surveys) and targets had been agreed upon by the ISG during the inception stage 
of  the  evaluation  study.  In  practice,  the  number  of  responses  received  and  processed  have 
been  on  or  above  targets  in  most  cases.  In  the  case  of  the  beneficiary  and  control  group 
surveys the satisfactory response rate offered to the evaluation a solid and reliable basis for 
data comparison amongst beneficiaries and sectors. A noticeable exception though was in the 
sport and adult education fields where the control groups were too small. For details on the 
actual  response  rates  against  minimum  sample  sizes  defined  for  the  above-mentioned  main 
surveys’ target and control groups see evaluation final report, volume 1 (section 3).  
Concerning the other surveys, the minimum sample sizes500 were attained. In the case of the 
programme  agencies,  no  ad  hoc  target  had  been  fixed,  but  all  programme  countries  were 
covered. 

In the case of the OPC, the number of total responses received and this of exploitable ones 
(i.e. complete responses) was considered statistically reliable. 

With regard to the key informant interviews (KIIs) undertaken to inform the main evaluation 
report as well as accompanying standalone reports, the number of interviews foreseen at the 
start  of  the  evaluation  process  was  attained  in  all  cases.  The  main  types  of  pre-identified 
stakeholders were also consulted.  

The same applied to the case studies (i.e. at organisational level (spread across all education 
and training, youth and sport fields), at system level and focusing on Jean Monnet). With the 
exception of one case study initially foreseen that could be not carried out, all the others were 
effectively  conducted  either  on-site  (organisation  level  case  studies)  or  over  the  phone 
(system level and Jean Monnet ones).  

In  addition  to  the  above,  other  consultation  related  activities  are  worth  being  mentioned. 
These have not been listed in table 1: 

  Expert  panel  assessment  of  project  outputs:  experts  in  education  and  training, 
youth  and  sport  fields  were  sub-contracted  by  ICF  to  assess  a  number  of  selected 
project outputs and take part in a virtual expert panel assessment 

  Delphi survey of experts: experts in the fields above were sub-contracted by ICF to 
take part in the survey that whose main purpose was to inform the analysis of the EU 
added value and cost-effectiveness of the current programme 

  Informal workshop organised by two EU-level organisations (EUCIS-LLLP and the 
European Youth Forum -EYF) in May 2017, prior to the closure of the OPC. It was 
agreed  with  DG  EAC  that  the  event  could  be  an  opportunity  for  ICF  to  grasp 
additional insights from key stakeholders ideally before OPC closure. 

A4.2 Consistency of results across consultation activities  

The  table  below  presents  key  results  per  consultation  activity,  organised  by  evaluation 
criteria, as well as the level of (1) consistency of results across consultation activities and (2) 
complementarity of results across consultation activities.  

500 i.e. at least 200 respondents to the socio-economic actors and to the experts surveys. At least 200 students 

enrolled in JM courses or modules; at least 50 practitioners from Jean Monnet actions  

90 

 
 
 
                                                            
Overall,  there  was  a  large  degree  of  convergence  in  results  from  the  different  consultation 
activities.  

91 

 
 
 
 
f
o
y
c
n
e
t
s
i
s
n
o
C

s
s
o
r
c
a
s
t
l
u
s
e
r

n
o
i
t
a
t
l
u
s
n
o
c

s
e
i
t
i
v
i
t
c
a

y
t
i
r
a
-
t
n
e
m
e
l
p
m
o
C

s
s
o
r
c
a
s
t
l
u
s
e
r

f
o

n
o
i
t
a
t
l
u
s
n
o
c

s
e
i
t
i
v
i
t
c
a

High 

High 

Mid-term evaluation of Erasmus+ and ex-post evaluation of predecessor programmes  

Table 2. 

Main results of the consultation processes and level of their consistency and complementarity 

Headline results 

n
o
i
t
a
u
l
a
v
E

n
o
i
r
e
t
i
r
c

Surveys 

KIIs 

Case studies 

OPC 

Concrete examples of 
alignment of 
Erasmus+ with key 
national policies were 
provided though not 
by all respondents 

Several similar 
findings as those 
obtained through KIIs 

The relevance of the E+ 
is positively rated by 
most respondents.  

Some programmes’ 
objectives are perceived 
to be notably more 
relevant than others in 
addressing the current 
challenges and needs 
within education, 
training, youth and sport 

Relevance 

Beneficiaries surveys: 

Learners with different 
backgrounds can benefit from 
the programme.  

The attractiveness of 
disadvantaged groups is an area 
for improvement though 

Experts survey: 

Most respondents estimate that a 
high share of projects funded 
align to high or average extent 
with EU-and national level 
priorities   

Improvement in quality of 
applications and project reports 
is found between E+ and 
predecessor programmes 

Programme agencies survey: 

Over half of disagree that E+ is 
better aligned with national 
priorities than predecessor 
programmes 

Mixed reports on the 
programme’s ability to meet the 

Most interviewees see 
strong alignment with key 
EU priorities and  high to 
medium level of alignment 
of Erasmus+ with key 
national policies depending 
on sectors  

A vast majority of 
interviewed estimate that 
E+ best addresses learners’ 
needs. Those at national 
level confirm that E+ 
actions are relevant in 
terms of their positive 
effects at practitioner level 
too.  

Learners with different 
backgrounds can benefit 
from the programme. The 
attractiveness of 
disadvantaged groups is an 
area for improvement 
though 

Mixed reports on the 
programme’s ability to 
meet the needs of 
education, training, youth 

October,  2017 

92 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
f
o
y
c
n
e
t
s
i
s
n
o
C

s
s
o
r
c
a
s
t
l
u
s
e
r

n
o
i
t
a
t
l
u
s
n
o
c

s
e
i
t
i
v
i
t
c
a

y
t
i
r
a
-
t
n
e
m
e
l
p
m
o
C

s
s
o
r
c
a
s
t
l
u
s
e
r

f
o

n
o
i
t
a
t
l
u
s
n
o
c

s
e
i
t
i
v
i
t
c
a

High 

High 

Mid-term evaluation of Erasmus+ and ex-post evaluation of predecessor programmes  

Headline results 

n
o
i
t
a
u
l
a
v
E

n
o
i
r
e
t
i
r
c

Surveys 

KIIs 

Case studies 

OPC 

needs of education, training, 
youth and sport at organisation 
and system level. 

Most respondents believe that 
the single programme is more 
visible 

A number of obstacles for 
disadvantaged groups to 
participate in E+ are reported. 

and sport at organisation 
and system level.  

A number of obstacles for 
disadvantaged groups to 
participate in E+ are 
reported. 

Coherence  

Programme agencies survey: 

Most agencies perceive 
synergies and lack of overlaps 
across programme actions as 
well as externally (other EU or 
national programmes) 

The coherence of the programme 
between centralised and 
decentralised actions is valued 
by most respondents 

Respondents’ views on the 
benefits of the integrated 
programme are positive  

Not all respondents 
were able to give 
examples of 
synergies 
overlaps/duplications 
between different 
types of actions and 
in different sectors 

Most respondents  
believe that E+ does not 
overlap with other 
funding opportunities 

Need to ensure synergies 
with EU level 
programmes is pointed 
by some respondents. 

Examples of 
complementary national 
schemes to E+ are 
supplied in some OPC 
position papers 

Most interviewees 
(EU/national) consider that 
Erasmus+ objectives and 
actions are mutually 
supportive in the different 
fields.  

Not all respondents were 
able to give examples of 
synergies 
overlaps/duplications  

No clear consensus on 
increased E+ 
decentralisation: not 
positively perceived by all 

The integrated programme 
is valued  by most 
interviewees 

October,  2017 

93 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
f
o
y
c
n
e
t
s
i
s
n
o
C

s
s
o
r
c
a
s
t
l
u
s
e
r

n
o
i
t
a
t
l
u
s
n
o
c

s
e
i
t
i
v
i
t
c
a

y
t
i
r
a
-
t
n
e
m
e
l
p
m
o
C

s
s
o
r
c
a
s
t
l
u
s
e
r

f
o

n
o
i
t
a
t
l
u
s
n
o
c

s
e
i
t
i
v
i
t
c
a

High 

High 

Mid-term evaluation of Erasmus+ and ex-post evaluation of predecessor programmes  

Headline results 

n
o
i
t
a
u
l
a
v
E

n
o
i
r
e
t
i
r
c

Surveys 

KIIs 

Case studies 

OPC 

Most 
interviewees  have 
the 
positive  views  on 
effects  of  E+  in  particular 
at individual and to varying 
extent 
organisation 
levels. 

at 

More  mixed  views  on 
examples  of  system-level 
projects 
related 
and 
influence  

71% of respondents 
thought that as a whole, 
the programme is 
achieving its objectives to 
a ‘very large’ or ‘large’ 
extent. There were some 
notable differences 
between programme 
objectives, levels of 
intervention and sectors 
though 

Learners  expressed  a 
of 
broad 
positive  changes  they 
associate with E+.  

range 

practitioners 
All 
cited 
interviewed 
positive 
results  on 
themselves as well as 
their 
and 
learners 
organisations 

In  all  case  studies 
concrete  changes  at 
organisational 
level 
were cited 

A  lot  of  the  evidence 
is  about  participation 
disadvantaged 
of 
groups 
than 
results for them.  

rather 

Effectiveness 

Beneficiaries surveys 

Participation  in  E+  is  associated 
with  stronger  attachment  to  the 
EU, networking, and greater use 
of digital resources 

The recognition of practitioners’ 
achievement  during  mobility  is 
uneven  

Disadvantaged 
positive 
indicators 

learners 

results  on 

a 

show 
few 

Socio economic actors survey 

respondents  agree 

Most 
their objectives have been met 

that 

Only  a  small  number  said  their 
projects led to implementation of 
approaches 
tackle 
to 
disadvantaged group 

Programme agencies survey 

The  quality  scores  at  project 
robust 
selection  are  not  a 
predictor 
at 
completion 

quality 

of 

Most 
D&E 

respondents  value 
the 
strategy.  More  mixed 

October,  2017 

94 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
f
o
y
c
n
e
t
s
i
s
n
o
C

s
s
o
r
c
a
s
t
l
u
s
e
r

n
o
i
t
a
t
l
u
s
n
o
c

s
e
i
t
i
v
i
t
c
a

y
t
i
r
a
-
t
n
e
m
e
l
p
m
o
C

s
s
o
r
c
a
s
t
l
u
s
e
r

f
o

n
o
i
t
a
t
l
u
s
n
o
c

s
e
i
t
i
v
i
t
c
a

High 

High 

Mid-term evaluation of Erasmus+ and ex-post evaluation of predecessor programmes  

Headline results 

n
o
i
t
a
u
l
a
v
E

n
o
i
r
e
t
i
r
c

Surveys 

KIIs 

Case studies 

OPC 

The majority of 
respondents welcome 
the use of unit costs 

Several findings 
concurrent with those 
collated under KIIs. 

Several key informants 
commented spontaneously 
on insufficient levels of 
funding for certain types of 
actions or sectors.  

Mixed views on efficiency 
gains are observed by the 
programme beneficiaries 

The set of monitoring tools 
is commonly viewed as 
rather complex 

Most respondents see 
improvements in the 
user-friendliness of E+ IT 
tools and reporting 
procedures  

A slightly smaller 
majority of respondents 
agree that the 
management of the E+ 
has been simplified for 
them 

Mixed views on the 
budget distribution 
between the three Key 
Actions. In particular 
KA2 is considered as 
underfunded 

views  on 
platform 

the  project  results 

Efficiency 

Beneficiaries surveys 

The size of budgets is considered 
too small, in particular in the 
case of projects applying for 
KA2 

Programme agencies survey 

Most respondents value  the 
balance between centralised and 
decentralised actions  

Mixed views on E+ management 
tools  

Most respondents see efficiency 
gains compared to predecessor 
programmes.  

Mixed views on efficiency gains 
are observed by the programme 
beneficiaries 

The set of monitoring tools is 
considered rather complex 

Socio economic actors survey 

Vast majority of respondents are 
very positive about the support 

October,  2017 

95 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Mid-term evaluation of Erasmus+ and ex-post evaluation of predecessor programmes  

Headline results 

n
o
i
t
a
u
l
a
v
E

n
o
i
r
e
t
i
r
c

Surveys 

KIIs 

Case studies 

OPC 

received by EACEA or NAs 

EU added value  Programme agencies survey 

Though less specifically, most 
interviewees see a clear 
European added value in 
E+ 

Most respondents claimed 
that their organisation 
could not have 
developed similar 
projects/actions 
without Erasmus+ 
funding  

The programme funds more 
mobility and cooperation actions 
than comparator programmes.  

It covers system-level 
cooperation actions that are very 
rarely covered by other 
programmes 

Most of the respondents 
believe that E+ brings 
certain benefits to the 
actions implemented.  

Respondents also 
identified other areas 
where they believed E+ 
has added value to the 
actions implemented on 
the EU, national and  
international level  

f
o
y
c
n
e
t
s
i
s
n
o
C

s
s
o
r
c
a
s
t
l
u
s
e
r

n
o
i
t
a
t
l
u
s
n
o
c

s
e
i
t
i
v
i
t
c
a

y
t
i
r
a
-
t
n
e
m
e
l
p
m
o
C

s
s
o
r
c
a
s
t
l
u
s
e
r

f
o

n
o
i
t
a
t
l
u
s
n
o
c

s
e
i
t
i
v
i
t
c
a

High 

High 

October,  2017 

96 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Annex 3: Methods and analytical models 

As the differences in the architecture, objectives and activities of the current and predecessor 
programmes501  made  it  difficult  to  make  a  straightforward  comparison  between  their 
respective actions, the current evaluation did not look at programmes along their own types of 
actions  but  through  a  customised  matrix.  To  ensure  comparable  approaches,  the  evaluation 
framework developed by the external evaluator502 classifies actions across the three levels of 
intervention  of  the  current  programme  (see  fig.  below)  and  uses  that  as  the  basis  of 
comparison, namely: 

■  System level 

■  Organisation level 

■  Individual  level  divided  into  actions  for  learners  and  practitioners  across  all 
different areas (education and training, youth and sport) and sectors (school (SE), 
vocational (VET), higher (HE) and adult education (AE)). For the purpose of the 
evaluation, the overarching terms of Learners and Practitioners are used503. 

501 ICF, 3 and Annexe 1; see section 2 above for background on Erasmus+; under the Erasmus+ the activities 
were  simplified,  (as  a  simple  example  among  many  others,  ‘ad  personam’  Jean  Monnet  chair  is  now 
integrated in the Jean Monnet Chair). 

502 ICF, Annexe 14 (not published), where a matrix maps and compares the types of actions in the predecessor 

programmes and the corresponding actions in the Erasmus+ programme 

503 Learners refer to all individuals involved in formal, non-formal and informal education as pupils, students, 
apprentices,  volunteers,  young  people,  etc.  Practitioners  refer  to  those  involved  in  the  same  respect  as 
teachers (including prospective teachers), trainers, youth workers, educators, coaches, etc. 

97 

 
 
 
                                                            
Summary of analytical techniques used 

Desk research and interviews 
An  extensive  literature  review  was  carried  out  across  varied  sources.  In  addition  to  the 
contextualisation  of  other  data,  the  main  purpose  was  to  review  research  notably  on 
programme  outcomes,  including  success  factors  and  obstacles  to  learning  mobility  and 
transnational  cooperation  in  the  fields  of  education,  training,  youth  and  sport,  mainly 
encountered through predecessor programmes. This helped to inform whether and where the 
programmes  have  had  impacts  at  the  individual,  organisational  and  system  levels  and  was 
central  to  the  analytical  framework  underpinning  this  evaluation.  A  full  account  of  the 
literature reviewed is annexed to the contractor's report. External statistics were also used to 
put the programme in perspective504. 

Key informant interviews (KIIs) have been conducted at both national and European level 
targeting  EC/agencies  officials,  EU  key  stakeholder  organisation  representatives,  national 
policy  makers  and  key  stakeholders  in  15  selected  countries,  as  well  as  managers  of 
comparable  programmes for benchmarking  purpose.  All were  selected using  category based 
purposeful sampling 505. 

Source 

Scope 

Volume 

Literature review 
(ICF, Annexe 11 and 15 - not 

Desk research: review of other 
evaluations, NA yearly reports (since 

131 sources reviewed 
More than 400 yearly reports 

504  For  instance,  although  the  market  share  of  Erasmus+  was  assessed  in  the  HE  field  where  more  data  is 
available,  this  calculation  was  rough  as  Eurostat  deals  with  full  degree  mobility  and  not  credit  mobility 
mainly supported by the EU. 

505 This means that interviewees were spread across each (sub-)field of the programme, at both EU and national 
level, including policymakers and key stakeholders. The country selection (BG, CY, CZ, DE, DK, ES, FR, 
HU,  IE,  IT,  LV,  PL,  SE,  TR  and  UK)  took  into  account  a  variety  of  volume  of  participation  in  the 
programme;  the  country  size  (covering  more  than  50%  of  the  population);  geographical  distribution 
(including  one  non-EU  country)  and  the  existence  of  national  programmes  for  benchmarking  in  different 
sectors. 

98 

 
 
 
                                                            
Source 

Scope 

Volume 

published) 

Interviews of Key informants 
(ICF, Annexe 16 - not 
published) 

2008), studies, academic papers at 
EU, national, international, etc506 
Semi-guided interviews of selected 
stakeholders in all sectors, at EU 
level and in 15 countries 

from National Agencies 

190 interviews: 
59 at EU level  
131 at national level 

National reports 

Programme countries have submitted their own evaluation reports on the implementation and 
impact  of  decentralised  actions  of  Erasmus+  in  their  respective  territories,  as  required  by 
Erasmus+  legal  basis.  Their  findings  are  summarised  in  the  National  reports  synthesis 
(NRS)  drawn  by  the  external  evaluator507.  These  reports  involved  the  review  of 
documentation  and  statistics  concerning  the  programme  and  related  policies  and  ad  hoc 
consultations  with  relevant  stakeholders.  The  most  often  used  data  collection  methods  were 
interviews and surveys (both used by 25 out of 34 countries).  In addition 11 countries  used 
focus groups. 

Source 

Scope 

Volume 

National Reports 
(ICF, stand lone synthesis – 
"NRS") 

Programme country reports on the 
impact of decentralised actions 
submitted by June 2017 as per Article 
21(4) of Regulation (EU) 1288/2013 
and DG EAC’s guidelines508 

34 National reports 
(MT and UK transmitted 
their report to the 
Commission with a four 
month delay) 

Open Public Consultation 

The objective of the Open Public Consultation (OPC) was to gather the opinions, including 
forward-looking perspectives, of the general public and various stakeholders, on all evaluation 
criteria509.  Different  (closed  and  open-ended)  questions  were  asked  depending  on  the  level 
and scope of knowledge of the programmes declared by each respondent. Most respondents 
were  either  organisations  or  practitioners.  In    any  case,  consultation  findings  can  never  be 
representative  due  to  the  selection  bias  inherent  to  any  open  recruitment,  but  they  can  be 
triangulated with other sources to inform the evaluation. Alongside their responses, the OPC 
respondents submitted 24 position papers, which are included as well in the synopsis drawn 
by the external evaluator510. 

506 The Terms of reference set for this evaluation contain a list of literature references, which have been enriched 

thereafter. 

507 National reports synthesis (NRS) annexed to ICF final report 

508 Ares(2016)576506 of 2 February 2016 

509 

https://ec.europa.eu/programmes/erasmus-plus/resources/consultations/erasmus-plus-mid-term-evaluation-

2017_en 

510 Synopsis of the open Public Consultation (OPC) annexed to ICF final report 

99 

 
 
 
                                                            
Source 

Scope 

Volume 

Open Public Consultation 
(ICF, stand lone "OPC" 
synopsis) 

To gather the opinions and 
perspectives of the general public 
and other stakeholders, the OPC 
was available in all official EU 
languages on the dedicated 
European Commission website 
from 28 February to 31 May 2017 

4,786 complete (n = 
1,219) and partial 
responses of which 1,800 
responses were fully 
exploitable (53% 
answered in their personal 
capacities; 47% on behalf 
of an organisation) 511 
24 position papers 
submitted 

Social media analysis 

Social  media  analysis  was  used  to  collect  evidence  about  the  awareness  on  the  programme 
and its public image, globally and by sector. All relevant posts identified512 were analysed for 
trends  in  the  volume  of  conversation  about  Erasmus+,  audience  profile,  and  content  of 
discussions  both  in  terms  of  topic  and  sentiment.  The  volume  of  information  shared  was 
compared  with  the  volume  for  other  EU  programmes  (e.g.  Marie  Curie,  MEDIA,  Citizens 
programme). 

Source 

Scope 

Volume 

Social media analysis 
(ICF, Annex 19 - not 
published) 

Twitter, Facebook and Instagram 
were analysed between 5 
November 2015 and 4 November 
2016. Limited for qualitative 
analysis to posts in English, French, 
Spanish and German. 

725,678 posts 
on the topic of Erasmus+ 
(almost 2,000 posts every 
day)  

Benchmarking 

Erasmus+  was  benchmarked  across  the  different  evaluation  criteria  against  18  selected 
comparator  national  and  transnational  mobility  and  cooperation  programmes513.  Three 
different  types  of  benchmarking  were  undertaken:  Strategic  benchmarking  helped  to 
compare 
the  programmes.  Process 
benchmarking  compared  efficiency  and  effectiveness  of  their  management.  Financial 
benchmarking  assessed  the  respective  financial  efficiency  and  effectiveness  of  the 

the  relevance,  coherence  and  added  value  of 

511 A large number of partial responses only contained very basic background information about respondents and 

were therefore removed from the analysis as unsuitable for analysis. 

512 The Crimson Hexagon platform was used to identify the topics of conversations on social media according to 
the frequency of use in posts of certain selected keywords (“Erasmus+”, “ Erasmus+”, “Erasmus plus” and 
“Erasmusplus”). Results for Erasmus+ were fine-tuned using secondary keywords, such as youth, school or 
schools,  higher  education  or  higher  ed,  VET  or  vocational,  adult  learning,  sport,  etc.  Source: 
https://www.crimsonhexagon.com/ 

513 18 national or international schemes were selected out of 58 for detailed benchmarking purpose and spread as 
follows:  HE  (7),  VET  (5),  SE  (4),  youth  (2),  sport  (2),  AE  (1).  Action-wise,  the  vast  majority  support 
mobility. ICF, fig 5.5 about geographical balance 

100 

 
 
                                                            
programmes  compared.  The  mapping  was  not  expected  to  be  comprehensive  but  to  inform 
about the scale and profiles of comparable initiatives514. The large difference in scope and size 
made  comparison  between  programmes  fairly  complex.  The  contractor  only  had  access  to 
information  available  in  the  public  domain,  which  means  that  in  a  few  cases,  it  was  not 
possible to collect data for some indicators. 

Source 

Scope 

Benchmarking with 
comparator programmes 
(ICF, Annex 12) 

Strategic/ process/ financial 
benchmarking against national or 
international schemes sharing some 
similarities with Erasmus+ 

Volume 

18 comparable 
programmes 
out of 58 shortlisted 
schemes 
15 interviews 

Programme data  

The evaluation used the wealth of the programme data available as regards  inputs (funding) 
and  outputs  (numbers  of  projects,  numbers  of  beneficiaries,  etc.).  Moreover  data  on  certain 
results  (e.g. satisfaction-rate)  is  available  since  2014  directly  from  the  management  or 
monitoring tools set up for Erasmus+. 

Overall  the  quality  of  data  was  good  for  the  predecessor  programmes  and  even  very 
good for Erasmus+. This shows improvement made in terms of monitoring since 2014 (see 
table of monitoring tools below)515, albeit with certain limits516. The quality and completeness 
of  the  data517  was  best  for  the  Erasmus+  programme,  followed  by  the  LLP  and  Youth  in 
Action518. Fewer data was available for Erasmus Mundus and Tempus and even less for the 
remaining higher education international programmes519 and sport pilot actions520. The main 

514 The initial mapping identified across Europe 58 schemes which were comparable to Erasmus+. Around half 
covered SE, over 35% HE, below 20% VET or the Youth sector, 10% AE. Their target groups were students 
in nearly 80% of the 58 schemes, teachers at around 22%, youth at around 17% and adults only around 10%. 

515 Erasmus+ Dashboard, annual reports, Business Objects (BO) reports, Epluslink and Mobility Tool databases 

516 Data is not always available for predecessor programmes. The data on Jean Monnet participants is in general 
not very reliable. It reflects the intention as it is based on applications rather than the actual output, contains 
double counting where the same persons take part in more than one activity and captures indistinctly those 
involved in dissemination activities only. 

517 ICF, Tab. 3.3 and 6.2 for a summary of the datasets coverage 

518 Although data on YiA participants is only available for the European Volunteer Service, but not for Youth 

exchanges. 

519  For  the  remaining  higher  education  international  programmes,  Alfa  and  Edulink  in  particular,  only 
consolidated findings were available from their distinct evaluation: European Commission, Evaluation of the 
EU  Development  Co-operation  Support  to  Higher  Education  in  Partner  Countries  (2007-2014),  Revised 
draft final report (main report), March 2017. 

520 For sport pilot actions only the funding and number of selected projects were available. 

101 

 
 
                                                            
issues affected the analysis of participation patterns due to an incomplete coverage of data on 
i) organisations521; ii) participants with few opportunities and disadvantaged backgrounds522. 

Overview of Monitoring arrangements 

Level and purpose  IT systems and tools 

Project 
management 

level  – 

E+Link –main system for management of decentralised grants (NAs) 
Pegasus –main system for managing centralised grants (EACEA) 
Mobility Tool – system for managing individual mobility actions (NAs) 

Project level – 
grant evaluation  

OEET – online expert evaluation tool  
For project assessors 

Programme level 
– reporting and 
management 

Business Object reporting  ‘live’ Dashboard visualising key performance 
indicators according to criteria defined by each user 
For Commission services and agencies 

Programme level 
– dissemination 

Erasmus+ Project results platform – summary overview of all projects (KA2 
and KA3) with success stories flagged  - For all audiences 

Other tools 

EU Survey – online survey administered for all KA1 beneficiaries through the 
mobility tool  
OLS – pre-post testing on language competence  

PDM/URF – DGIT system for creation of unique organisational ID for all 
beneficiaries of EU programmes 
HERMES – DGIT system for management of application forms 

The  monitoring  surveys  of  beneficiaries  carried  out  by  DG  EAC  concern  all  learners  or 
practioners  who  take  part  in  mobility  under  KA1  over  2014-2016523.  The  reliability  of  the 
data  is  strong.  Given  the  sample  sizes  (first  two  years  of  the  programme)  the  data  is 
considered  to  be  strongly  reliable  even  though  it  was  too  early  to  include  2016  data.  This 
being said most questions are based on the self-perception of beneficiaries524 in contrast to the 
more objective OLS525 assessment of progress in language proficiency for instance; even for 

521 A unique ID for each participating organisation has been introduced only since 2014. Its absence beforehand 
has  affected  the  comparison  with  predecessor  programmes  and  limited  the  network  analysis  to  Erasmus+ 
(without KA1). Furthermore, although the number of types of organisations used was very large, some 40% 
of organisations were classified as "other" in the programme dataset affecting the analysis of participation 
per type of organisation. 

522 See sections on effectiveness and relevance below 

523 All KA1 individual beneficiaries are requested to fulfil an online survey on completion of their mobility. See 

ICF, fig. 3.2 for breakdown by sector. 

524  The  most  factual question (programme  result  indicator)  is  the rate of  formal  recognition of participation  in 

Erasmus+. 

525  The  Online  Linguistic  Support  (OLS)  was  introduced  with  the  Erasmus+  programme  to  help  mobility 
participants improve their knowledge of the relevant foreign language, so that they can make the most out of 
their  experience  abroad.  Before  and  at  the  end  of  the  Erasmus+  mobility,  Erasmus+  participants  take  the 
language  assessment  to  measure  their  level  and  progress  in  the  language.  OLS  provides  as  well  online 
language courses. 

102 

 
 
 
                                                            
very  similar  questions  the  responses  offered  vary  across  sectors  affecting  the  comparability 
across target groups. 

In  addition  to  descriptive  and  multivariate  statistical  analysis  of  programme  data,  a  social 
network  analysis  showed  more  specifically  how  organisations  cooperate  and  participate  in 
Erasmus+  and  assessed  in  particular  cross-sector  and  trans-national  cooperation.  It  is  to  be 
noted that the analysis of repeated participation in the programme could not be pursued, as it 
would have required substantial data cleaning across previous programmes' data sets (due to 
multiple names used for the same organisations prior to 2014). 

The analysis of programme data has been  enriched with other secondary data: contextual 
indicators  from  the  Education  and  training  monitor,  the  Youth  indicators,  relevant 
Eurobarometer  studies  or  the  Sport  indicators  available  on  Eurostat.  Furthermore,  external 
data sources on education and training institutions with high share of disadvantaged students 
have  been  compared  across  a  sample  of  countries  with  the  programme  data  to  identify  the 
schools that have (or have not) engaged with Erasmus+. 

Most result indicators are based on self-perception and the link between the survey questions 
and  the  indicator  measured  can  sometimes  be  weaker.  That  is  why  monitoring  data  were 
complemented through primary data collection such as surveys. 

Source 

Scope 

Volume 

Programme databases 
(ICF, Annexe 2) 

Monitoring surveys of 
Erasmus+ beneficiaries 
(Commission data) 
(ICF, Annexe 2) 

Extracts from Commission IT 
systems for programme 
management over 2008-2016526 
All Erasmus+ beneficiaries of KA1 
have been surveyed on completion 
since 2014 

Records concern all 
beneficiaries and in some 
case also applicants  

Over 955,000 
respondents 
of which: 
730,254 learners 
227, 319 staff 

Online Linguistic Support 
assessment (ICF, 3.4.2) 

Learners in KA1 (mostly higher 
education students) since 2014 

total sample size: 523,238 
participants (95% in HE) 

Network analysis 
(ICF, Annex 18 – not 
published) 

Based on programme data for KA2 
and KA3 (or equivalent actions) 
from Erasmus+, LLP, Youth in 
Action, Erasmus Mundus and 
Tempus527 

Based on programme data 
from Erasmus + (KA2 and 
KA3), LLP, Youth in 
Action, Erasmus Mundus 
and Tempus 

526 Some indicators were unavailable for 2007. 2016 data for centralised actions were not yet fully available at 
the time of the data analysis. The sample for 2016 therefore represents a robust indication but not the final 
state of the respective indicators. 

527  For  International  Higher  Education,  data  was  available  only  for  centralised  projects.  For  Tempus  the  data 
analysed  only  covered  leading  organisations  as  no  data  was  available  about  partner  organisations.  Other 
Higher Education international programmes were not covered. 

103 

 
 
 
 
 
                                                            
Surveys of beneficiaries and control groups 

through  factual  questions  whether 

To  collect  more  comprehensive,  objective  and  comparable  quantitative  data  on  programme 
results, the external evaluator has carried out ten specific surveys of beneficiary learners and 
staff  and  related  control  groups  covering  all  programme  target  groups,  both  for  KA1 
(mobility) and KA2 (organisational cooperation)528. The Erasmus+ monitoring survey shows 
a  subjective  valuation  by  beneficiaries  themselves  of  their  experience,  whereas  the  external 
evaluation  surveys  show  more  objectively, 
the 
beneficiaries demonstrate different attitudes, competences or beliefs than a control group that 
was asked the same questions. As often as possible the questions used in the survey had been 
previously  tested  in  large  scale  surveys  such  as  PISA,  TALIS  or  Eurobarometer  and  in  any 
case were as often as possible factual, so as to enable a comparison with the control group of 
non-beneficiaries.  This  means  that  to  strength  the  validity  of  surveys  statements  about  self-
perceived contribution of the programme were as often as possible avoided. The latter surveys 
were  disseminated  to  all  contactable  beneficiaries  of  Erasmus+  as  well  as  predecessor 
programmes529. Overall findings could be generalised to the whole programme with sufficient 
confidence because the sample sizes for all surveys were large enough in all cases to make 
judgements  and  the  distribution  of  various  background  variables  within  the  survey  samples 
was assessed as acceptable530. 

In  addition  to  the  standard  tests  of  significance  and  regressions,  a  quasi-experimental 
approach was used to assess the contribution of the programme to the results measured531. As 
a proxy of counterfactual assessment532, control groups of non-beneficiaries were set up533 

528 Results at the level of organisations were measured through responses of staff 

529 ICF/GfK, 3.3.3 on survey dissemination and recruitment methods as well as sample sizes (tab. 3.6) 

530  However  given  the  limited  background  variables  about  the  beneficiaries  it  is  not  possible  to  assess  how 
comparable  are  the  samples  compared  to  the  programme  population.  For  several  surveys  (pupils,  young 
people, sport staff) there is no contact database of direct beneficiaries hence there is no background data on 
the  population  to  compare  with.  Respondents  have  been  recruited  through  organisations.  For  all  the  staff 
surveys though there is database for KA1 beneficiaries (Erasmus+ only) KA2 beneficiaries, for whom there 
is  no  database  that  would  provide  data  on  the  overall  population,  were  also  surveyed.  Respondents  have 
been recruited among beneficiaries who took part not only in the current programme for which population 
data is available for KA1 but also predecessor programmes. Except learners in higher education and VET 
the information about background characteristics of the overall population for the other groups is weak or 
inexistent.  Where  available  the  background  information  is  besides  not  standardised  across  datasets.  For 
example,  the  database  of  HEI  students  who  took  part  in  Erasmus  Mundus  identifies  their  country  of 
residence, whereas the HEI student database of Erasmus students identifies their nationality or the country 
they are being sent from. For all databases there are gaps in terms of the completeness of the information. 
Even  where  population  variables  of  interest  are  available,  there  are  missing  cases,  meaning  that  it  is  not 
possible to say with confidence what the profile of the population is. 

531 The analysis includes tests of statistical significance, regressions, pre-post measurement and propensity score 
matching; ICF, Annexe 3 for detailed statistical analysis of survey results; ICF/GfK, 3.4 concerning method 
of statistical analysis and counterfactual estimation applied for the post surveys and the pre-post surveys 

532 Counterfactual attempts to measure what would have been the outcome in the absence of the programme. 

533  In  addition  to  open  recruitment  through  social  media,  control  groups  were  recruited  in  asking  either 
participating  organisations  or  unsuccessful  applicants  to  disseminate  the  survey  respectively  to  non-
participants or to their staff. 

104 

 
 
 
                                                            
with  similar  profiles  to  compare  their  responses  with  those  made  by  respondents  who 
participated in the Erasmus+ programme. This has been achieved in matching respondents in 
the  treatment  group  (i.e.  Erasmus+  beneficiaries)  with  ‘similar’  individuals  in  the  control 
group,  to  come  up  with  a  ‘matched  sample’  where  subjects  are  alike  in  some  background 
characteristics,  called covariates,  such as  gender, age,  etc.534. Overall  the comparability was 
considered  satisfactory  even  if  there  are  some  minor  though  statistically  significant 
differences  on  certain  surveys  and  certain  variables.  For  most  variables  there  were  no 
significant  differences  between  the  control  group  and  the  treatment  group  and  where  they 
were these concerned variables that were not likely to strongly influence the findings. In other 
words,  this  might  not  be  considered  as  genuine  counterfactual  assessment,  as  the  design 
cannot exclude that the differences in result variables measured between the beneficiaries and 
the  control  group  would  be  related  to  the  selection  into  the  programme  rather  than  the 
programme. Nevertheless it does control for various characteristics of both beneficiaries and 
non-beneficiaries  when  judging  the  differences  in  results.  Most  control  groups  reached  a 
significant  size  allowing  comparison  with  beneficiaries,  except  in  sport  and  Adult 
education sectors where the control groups were too small535. 

In addition to the above ‘post’ surveys (i.e. beneficiaries were surveyed after they took part in 
the  programme),  the  external  evaluator  carried  out  pre-surveys  in  two  sectors  where  short 
term  mobility  exchanges  apply  (i.e.  school  pupils  and  Youth  exchanges)  for  the  purpose  of 
pre/post comparison. This allowed - for the school sector only536 - to draw conclusions on 
selection  into  the  programme  and  to  observe  changes  during  the  time  of  the  mobility 
experience. The sample  sizes  suffered from a high  attrition from the pre to  the post  survey, 
especially for the youth sector where comparison of profiles prior to Erasmus+ mobility could 
not  been  done.  Pre-post  results  for  the  youth  sector  have  therefore  not  been  interpreted 
considering the high likelihood of sample bias. 

The  surveys  of  learners  and  staff  were  complemented  with  a  survey  of  socio-economic 
organisations 
the  programme:  companies,  public  authorities,  civic 
organisations, sectoral bodies, etc. 

taking  part 

in 

Other surveys and interviews targeting beneficiaries of specific activities were carried out in 
the case of Jean Monnet actions and the Student Loan Guarantee Facility (SLGF)537. 

534 The match between co-variates was sought on: Age; Gender; Country; Academic level (i.e. whether they had 
repeated a grade); Socio-economic background; Rural/urban area; Objective migrant status (i.e. at least one 
parent born in a different country); Disadvantaged status (those who said ‘yes, and I feel disadvantaged by 
this’  to  any  of  the  statements  in  the  question  about  disabilities,  problems  and  obstacles);  Highest  parents’ 
education level (i.e. by either the mother or father); Volunteering experience (Youth survey only). 

535 As a consequence, the control group of VET staff was also used for adult education and the control group of 
youth  staff  was  also  used  for  sport,  as  these  were  found  to  be  sufficiently  comparable  against  a  range  of 
background characteristics. 

536 ICF/GfK, tab. 3.7 

537 ICF, 3.3.4 and 3.3.5 

105 

 
 
 
 
                                                            
Source 

Scope 

Volume 

Beneficiary surveys and 
control groups 
(ICF, Annexe 3 and 13 – 
not published) 

Beneficiaries of mobility actions 
(learners and practitioners) and 
cooperation actions (practitioners) 
as well as control groups (non-
beneficiaries) 

Socio-economic actors 
survey 
(ICF, Annexe 6) 

Organisations other than the 
primary target group, i.e. 
companies, public authorities, civil 
society (other than youth 
organisations) 

Topic-specific surveys 

Over 47,000 
respondents538 
Learners: 24,037 
beneficiaries and 2,695 
from control group  
Staff: 20,155 beneficiaries 
and 928 from control 
group  

947 valid responses  

Student Loan Guarantee 
Facility student fair survey 
(ICF, Annexe 8 and 
standalone SLGF section) 

Students participating in student 
fairs for future (mobile) Master 
students and exhibitors of these 
fairs 

219 respondents 
119 students and 100 
exhibitors  

Student Loan Guarantee 
Facility interviews 
(ICF, 3.4.5) 

Jean Monnet surveys 
(ICF, Annexe 7 and 
standalone Jean Monnet 
report) 

(Non)- Participating Financial 
Intermediaries and their 
representatives; National student 
loan schemes; HEIs and their 
representatives; National 
authorities and Agencies; 
Representatives of students and the 
youth 

Students studying and Staff 
teaching about the EU – 
beneficiaries and non-beneficiaries 

33 interviews 

2,350 respondents 
Supplemented with 330 
beneficiaries and 10,503 
non-beneficiaries 
from the general 
beneficiary survey 

Jean Monnet interviews and 
case studies 
(ICF, 3.4.4) 

Interviews with professors who 
have never applied for Jean 
Monnet (AU, CZ, UK, EL, US) 

5 interviews 
4 case studies 

Surveys of staff involved in the programme implementation 

538 Response rates are not available for these surveys, due to the fact that a large proportion of the responses were 
collected  through  institutions  or  by  a  snowball  method.  If  there  is  no  way  of  knowing  how  many  people 
were  actually  contacted,  ICF's  technical  report  (ICF,  annex  3a)  maps  how  many  invitations  were  sent  to 
which  contact  list.  For  instance,  for  the  sport  sector  197  out  of  a  total  of  329  participating  organisations 
replied to the survey. 

106 

 
 
 
 
 
 
 
 
                                                            
National  agencies  and  EACEA,  as  key  managing  bodies  in  the  implementation  of  the 
programme,  were  subject  to  a  dedicated  programme  agency  survey,  especially  useful  to 
inform efficiency of the programme management. 

To  complement  data  on  projects,  in  particular  about  the  quality  of  applications  and  final 
reports but also the efficiency of the selection process, a survey of project assessors was also 
carried out. They are experts assessing project applications and reports. 

Lastly,  a  so-called  Delphi  survey  was  disseminated  among  these  experts  to  collect  project 
assessors views about the added value of the programme and thus to provide inputs into the 
analysis of cost effectiveness. They were asked for each sector to elaborate about hypothetical 
impact of dismissing Erasmus+ and about alternatives539. 

Source 

Agencies survey 
(ICF, Annexe 4) 

Experts (project assessors) 
survey and Delphi survey 
(ICF, Annexe 5) 

Scope 

Agency staff in charge of 
programme implementation (57 
National Agencies and Executive 
Agency EACEA) 
One respondent per sector 
designated by each agency  

Assessors supporting project 
selection and those supporting 
evaluation of final reports 
(centralised and decentralised 
projects) 

Volume 

130 valid responses 

1122 valid responses 

213 responses (Delphi 
survey) 540 

539 What would have happened in absence of Erasmus +? What are the most likely reactions when it comes to 
alternative/  compensatory  measures?  What  would  have  been  the  most  likely  effect  of  the  absence  of 
Erasmus +? What alternative actions could lead to comparable results as Erasmus +? 

540 That is to say between 25 and 40 expert responses per sector. 

107 

 
 
 
 
 
 
 
 
                                                            
Case based and qualitative data collection 

On  a  qualitative  note,  the  case  studies  served  to  complement  insights  gained  from  key 
informant  interviews  and  surveys,  in  particular  about  how  the  programme  generates 
organisational and system level changes including spill over effects from individuals who are 
direct beneficiaries to other individuals and to organisations. Case studies were selected using 
a combination of criterion-based selection and random selection. 

The most significant change approach was applied to case study interviews. It is based on 
open  ended  questions  which  ask  beneficiaries  to  give  account  in  their  own  words  of  the 
difference  the  programme  makes  to  them.  Indeed,  the  surveys  measure  always  predefined 
results  from  the  perspective  of  the  programme  design.  However  the  participants  themselves 
may judge how the programme has benefited to them differently.  

Erasmus+  and  predecessor  programmes  fund  a  high  number  of  cooperation  projects  which 
result  in  outputs  (e.g.  handbook,  toolkits,  methodologies,  etc.)  that  are  aimed  to  be  used 
within  the  organisations  that  took  part  in  the  project  but  also  beyond.  To  get  a  better 
understanding  of  their  quality  and  dissemination  potential,  a  review  of  selected  project 
outputs, mainly collected from national agencies, was carried out. Given that only relatively 
few  Erasmus+  funded  projects  were  completed  at  the  time  of  this  evaluation,  the  sample  is 
somewhat biased towards predecessor programmes. All qualitative fieldwork was based on a 
standard  protocol:  the  outputs  were  captured  in  standardised  grids,  reviewed  and  where 
appropriate enriched  with  feedback  asked  by  interviewers.  Lastly,  an  expert  panel  reviewed 
collectively the actual outputs of cooperation projects. 

Source 

Case studies 
(ICF, Annexe 9) 

Scope 

Volume 

Cases mainly at organisation level 
to assess spill over effects and 
changes at organisation and system 
levels 
Staff, learners, leadership and other 
stakeholders if relevant  

38 cases 
of which 
32 at organisation level, 
4 at policy level, 
4 for Jean Monnet actions 

Review of selected 
projects’ outputs 
(ICF, Annexe 17 – not 
published) 

Extract information from 
cooperation projects and system 
level project actions, mainly from 
predecessor programmes 

233 respondents 
386 project reports541 

Expert panel assessment of 
projects’ outputs 
(ICF, Annexe 10) 

Assess projects’ outputs against a 
set of pre-determined criteria  

100 projects 
(144 outputs) 

541 The sample of 386 projects selected for review has the following characteristics: 292 projects are funded by 
LLP  (of  which  100  are  transversal  actions)  and  70  by  Erasmus+;  250  projects  are  centralised  and  136 
decentralised.  The  projects  were  spread  across  the  sectors/strands  as  follows:  school  (65),  VET  (49),  HE 
(25), adult learning (33), Jean Monnet (50), Youth (42) and sport (22). 

108 

 
 
 
 
                                                            
Annex 4:  Evaluation criteria and evaluation questions 

Source: Terms of Reference for Mid-term evaluation of Erasmus+ (January 2016) 

Effectiveness 

  To  what  extent  have  the  various  programme  fields  (both  within  Erasmus+  and  its 
predecessor  programmes)  delivered  the  expected  outputs,  results  and  impacts?  What 
negative and positive factors seem to be influencing outputs, results and impacts? 

  What are the long-term impacts of the predecessor programmes? We are interested in 
the impact of all actions/elements of the predecessor's programmes, in particular those 
actions/elements  that  are  continued  in  new  Erasmus+  programme.  We  are  also 
interested  in  impact  of  actions/elements  that  have  discontinued  to  the  extent  that  it 
might help to design the future programme. 

  Please  identify,  describe  and  quantify  (if  possible)  the  spill  over  effects  between 

various actions (clusters of actions) of the Programme. 

  What  are  the  differences  in  impact  of  programme  actions  on  specific  disadvantaged 
groups of the population who traditionally do not engage in transnational activities as 
compared to other groups that benefit from the programme? 

  Are there positive /negative effects that existed in the previous programmes, but that 

no longer exist with the new programme? 

  What conclusions can be  drawn on the likely impact of Erasmus+  programme given 
the  fact  that  significant  parts  of  their  actions  are  continuation  of  predecessor's 
programmes? 

109 

 
 
 
 
 
  What have been the unintended effects and their magnitude of the Erasmus+ and the 

predecessor programmes if any? 

  To what extent are the effects likely to last after the intervention ends? 

  How  effective  are  the  forms  of  cooperation  and  the  types  of  actions  under  the 
Programme  for  the  purpose  of  supporting  the  political  priorities?  Which  fields  and 
actions of the Programme are the most effective considering the needs? 

  How  and  to  what  extent  is  the  existence  of  one  integrated  Programme  a  help  or  a 
hindrance in promoting greater effectiveness and visibility for the EU's activities in the 
field  of  education  and  training,  youth  and  sport,  compared  with  a  sector-specific  or 
target  group-specific  approach  to  Programme  design,  using  sector-specific  action 
names (e.g. Comenius, Leonardo, Grundtvig)? 

  To what extent are the Programme results adequately disseminated and exploited? 

Relevance 

  To  what  extent  are  current  and  emerging  key  socio-economic  needs  and  challenges 
that Europe is facing reflected in the policy priorities, objectives and actions/activities 
of the Programme? 

  To  what  extent  are  needs  of  different  stakeholders  and  sectors  addressed  by 
Programme  objectives?  How  successful  is  the  Programme  in  attracting  and  reaching 
target  audiences  and  groups  within  different  fields  of  the  programme's  scope?  How 
well is the Erasmus+ programme known to the education and training, youth and sport 
community? In case some target groups are not sufficiently reached, what factors are 
limiting their access and what actions could be taken to remedy this? 

  To what extent is the design of the programme oriented and focused towards the hard-
to-reach  groups or specific disadvantaged groups of the population who traditionally 
do  not  engage  in  transnational  activities?  In  case  some  target  groups  are  not 
sufficiently reached, what factors are limiting their access and what actions could be 
taken to remedy this? 

  Based  on  the  analysis  of  the  impact  of  predecessor  programmes,  are  there  any 
elements that have been discontinued and could have a possible value added in future 
generation of programmes? 

Coherence 

  To  what  extent  are  the  objectives  of  different  programme  fields  within  Erasmus+ 
programme  consistent  and  mutually  supportive?  What  evidence  exists  of  synergies 
between  the  different  programme  fields  and  actions?  How  well  do  different  actions 
work  together?  To  what  extent  there  exist  duplications,  overlaps,  or  other 
disadvantageous issues between the programme fields and how are they dealt with? 

  To  what  extent  are  the  centralised  and  decentralised  actions  coherent?  How  do  they 

interact / complement each other? 

110 

 
 
 
 
 
 
 
 
 
  To  what  extent  is  the  Erasmus+  programme  coherent  with  relevant  EU  policies  and 
programmes  with  similar  objectives  such  as  Creative  Europe  Programme,  European 
Structural  and  Investment  Funds  (ESIF)  or  other  EU's  funding  programmes  such  as 
Horizon 2020, financial instruments relating to justice and citizenship, health, external 
cooperation  or  pre-accession  assistance  programmes?  To  what  extent  has  the 
Erasmus+  programme  proved complementary to other  EU interventions/initiatives in 
the field of education and training, youth and sport? 

  To  what  extent  does  the  Erasmus+  programme  design  (including  needs,  objectives, 
activities, outputs and effects) provide appropriate links and support to the EU policy 
agendas,  in  particular  policy  initiatives  and  political  priorities  within  the  EU  2020 
Strategy,  the  Education  and  Training  Strategic  Framework  ET  2020,  the  European 
youth strategy or the EU policy priorities in the field of sport? 

  To  what  extent  is  the  Erasmus+  programme  coherent  with  various  interventions 
pursued  at  national  and  international  level  which  have  similar  objectives?  To  what 
extent has the Erasmus+ programme proved complementary to other Member States' 
interventions/initiatives in the field of education, training, youth and sport? 

Efficiency 

  What  is  the  cost-effectiveness  of  various  actions  (clusters  of  actions)  of  Erasmus+ 
programme  and  its  predecessor  programmes?  How  the  relative  costs  and  outcomes 
(effects)  of  various  programme  actions  compare  within  and  across  the  programme 
fields? Provide quantifications. 

  To  what  extent  is/was  the  size  of  budget  appropriate  and  proportionate  to  what  the 
Programme  and  its  predecessors  set  out  to  achieve?  To  what  extent  is/was  the 
distribution  of  funds  across  the  programme  fields  and  Key  Actions  appropriate  in 
relation to their level of effectiveness and utility? 

  What  is  the  prospect  of  other  policy  instruments  or  mechanisms  in  providing  better 

cost-effectiveness ratio? 

  To  what  extent  is  the  implementation  and  management  structure  of  centralised  and 
decentralised actions appropriate, efficient, and well-functioning? How efficient is the 
cooperation  between  the  different  management  bodies  (Commission  –Executive 
Agency  –National  Agencies  –European  Investment  Fund  –  National  Authorities  – 
Independent  Audit  Bodies  –  Erasmus+  Committee),  and  to  what  extent  does  the 
Commission fulfil its guiding role in the process? How has this changed between the 
two programming period? What are the areas for improvements? 

  Are  there  differences  in  efficiency  of  Programme  management  and  implementation 
between  National  Agencies,  the  Commission  Executive  Agency,  the  European 
Investment  Fund  or  between  different  programming  periods?  If  so,  what  are  the 
differences  and  what  are  the  underlying  reasons  for  them?  Compare  the  strategies, 
approaches and outcomes of the different National Agencies. 

  To  what  extent  are  the  monitoring  mechanisms  applied  by  the  Commission,  the 
Executive  Agency,  the  National  Agencies  and  the  European  Investment  Fund 

111 

 
 
 
 
 
 
 
efficient/cost  effective?  To  what  extent  are  the  monitoring  mechanisms  of  the 
beneficiaries  and  participants  by  National  Agencies  and  the  Executive  Agency 
effective  and  proportionate?  To  what  extent  are  internal  monitoring  mechanisms  of 
activities  of  the  National  Agencies  and  the  implementation  of  the  Programme  at 
national  level  effective  and  proportionate?  What  are  the  areas  for  improvement, 
considering the need for a smooth and effective implementation of the Programme? 

  To  what  extent  do  the  indicators  identified  for  the  Programme  in  the  legal  base 
correspond  to  the  monitoring  purposes?  How  could  the  overall  management  and 
monitoring system be improved? 

  To  what  extent  are  the  management  support  tools  (e.g.  Epluslink,  Mobility  tool) 

adequate and sufficient to support a sound management of the Programme? 

  What  are  the  efficiency  effects  of  the  integration  of  previous  programmes  and 
the  advantages  and 
actions/activities 
into  Erasmus+  programme?  What  are 
disadvantages  of  the  integration  of  previous  programmes  and  actions  into  Erasmus+ 
programme? 

  To  what  extent  has  the  system  of  simplified  grants  resulted  in  a  reduction  of  the 
administrative  burden  for  National  Agencies,  Executive  Agency  and  programme 
beneficiaries  and  participants?  Are  there  differences  across  actions  or  fields?  What 
elements  of  the  programme  could  be  changed  to  further  reduce  the  administrative 
burden and simplify the programme's implementation, without unduly compromising 
its results and impact? 

  To  what  extent  have  the  anti-fraud  measures  allowed  for  the  prevention  and  timely 

detection of fraud? 

European added value 

  What  is  the  additional  value  and  benefit  resulting  from  EU  activities,  compared  to 
what  could  be  achieved  by  MS  at  national  and/or  regional  levels?  What  does  the 
Erasmus+ programme offer in addition to other education & training support schemes 
available at both international and national levels? 

  What  is  the  benefit  and  added  value  of  the  Erasmus+  programme  compared  to  the 
benefit  of  the  predecessor  programmes?  What  is  the  added  value  of  repetitive 
actions/activities of the Erasmus+ programme and the predecessor programmes? 

  What would be the most likely consequences of stopping the Erasmus+ programme? 

  To  what  extent  does  the  Erasmus+  programme  promote  cooperation  between 

participating countries? 

112 

 
 
 
 
 
 
  
 
 
Annex 5a: The intervention logic of Erasmus+ and its predecessor programmes 

113 

 
 
 
Programmes’ aims 

Drawing  on  the  legacy  of  its  predecessors  (Lifelong  Learning,  Youth  in  Action,  Erasmus  Mundus,  Tempus,  Alfa,  Edulink  and  the 
Preparatory Actions in Sport), Erasmus+ aims to contribute to similar general objectives in the fields of education and training, youth 
and sport. As for its predecessors, these objectives have been defined based on EU level priorities set out in key policy documents (e.g. 
Europe 2020, ET 2020, Youth Strategy, EU Sport Action Plan and ‘Increasing the impact of EU development policy: an Agenda for 
Change’542  and  since  then  confirmed  through  various  European  Commission  Communications  such  as:  ‘New  Skills  Agenda  for 
Europe’543 ; ‘Renewed EU agenda for higher education’544, etc.). These objectives jointly with the programme’s rationale are specified 
in the legal basis545 and the programme guide546.  

Next to these overarching objectives, Erasmus+ is aimed to help achieve a number of specific objectives applying to the respective fields 
it covers. These notably relate to: 

■  improving the level of key competences; 
■  improving quality, innovation, excellence and internationalisation of education, training and youth organisations; 
■  promoting the emergence of European lifelong learning area; 
■  enhancing the international dimension of education, training and youth; 
■  promoting teaching and research about the EU; 
■  promoting good governance in sport, volunteering in sport and tackle threats to integrity in sport.  

542 October 2011: https://eeas.europa.eu/sites/eeas/files/com3a20113a06373afin3aen3atxt.pdf  

543 June 2016: https://ec.europa.eu/education/news/20160610-education-skills-factsheet_en  

544 May 2017: https://ec.europa.eu/education/sites/education/files/he-com-2017-247_en.pdf  

545 Available at : http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32013R1288&from=EN   

546 Erasmus+ Programme Guide, Part A: http://ec.europa.eu/programmes/erasmus-plus/sites/erasmusplus/files/files/resources/erasmus-plus-programme-guide_en.pdf  

114 

 
 
 
 
                                                            
The predecessor programmes and Erasmus+ share a number of specific objectives that remained common to both periods. Over both 
programming periods specific emphasis has been put for instance on: 

■  Competence development of participating learners; 
■  Professional development of staff; and 
■  Foreign language learning;  

However,  the  predecessor  programmes  and  the  current  programme  were  designed  in  quite  different  contexts.  Hence,  a  number  of 
differences in objectives can be noted: 

■  The Erasmus+ programme has stronger emphasis on high level policy objectives (and result-oriented approaches); 
■  The emphasis on employability is also clearer in the current programme which was designed in a period when young people were 

facing high unemployment, unlike the period of design of the predecessor programmes; 

■  The  predecessor  programmes  on  the  other  hand  had  specific  objectives  about  the  quality  and  volume  or  mobility  exchanges  and 
quality  and  volume  of  organisational  cooperation.  This  ‘Europeanisation’  was  perceived  as  an  objective  in  its  own  right  in  the 
predecessor programmes while in the current programme it seems to become a means to achieve other ends rather than a goal on its 
own; 

■  The predecessor programmes also emphasised quite strongly the use of ICT in education and training and the introduction of ICT-

based pedagogies was an objective of the programme while it does not figure in the Erasmus+ legal basis; 

■  The youth in action programme put much more emphasis on the objectives of youth participation and the citizenship dimension of 

the youth programme than the current programme. 

There were also some issues that were topical at a given point in time for which specific actions were implemented temporarily. This is 
for instance the case of the situation of Roma population across the EU which was rather high on the policy agenda in the period 2009-
2010. Regarding Erasmus+, the issue of social inclusion has become for instance prominent during the refugee crisis since 2015 and the 
issue of violent radicalisation became urgency after the terrorist attacks that started in 2015. 

Overall,  despite  progressive  adjustments  as  above,  the  aims  of  Erasmus+  have  not  radically  changed  in  comparison  to  those  of  its 
predecessors.  A  noticeable  difference  between  the  current  and  the  previous  programmes  is  the  integrated  architecture  of  Erasmus+. 
Bringing  together  the  education  and  training  (including  Jean  Monnet  programme),  youth  and  sport  fields  into  a  single  integrated 
programme  is  in  particular  expected  to  foster  synergies,  cross-fertilisation  and  to  stimulate  new  forms  of  cooperation  that  did  not  or 
failed to materialise in the past. 

115 

 
 
 
 
 
Inputs  

To  operate  the  main  types  of  actions  and  achieve  the  expected  outcomes  and  long-last  impacts  discussed  below,  three  main  types  of 
inputs underpin the Erasmus+ programme:  

■  funding range of actions; 
■  system and management structures, and; 
■  support measures for dissemination and knowledge management. 

Whilst these do not differ much in theory from those offered under the predecessor programmes, Erasmus+ inputs are in practice rather 
different. A number of novelties have been indeed brought into Erasmus+. Besides the integrated structure mentioned above, the most 
noticeable  changes  have  taken  the  form  of:  an  increased  budget  allocation;  a  renewed  internal  structure  of  the  programme;  the 
introduction of new implementation and monitoring approaches and support measures for dissemination and knowledge management. 

Types of actions 

A major evolution compared to the predecessor programmes has been the change in programme structure. Rather than being structured 
by  sectors  with  each  sector  having  embedded  a  variety  of  types  of  actions  each  specific  to  a  given  sector,  the  programme  was 
restructured according to main categories of types of actions (Key Actions - KA) which are common to the education and training and 
youth  fields.  The  sport  field  and  the  Jean  Monnet  programme  have  made  the  object  of  separate  chapters.  While  there  are  still  some 
specificities in the fields and subfields, the main types of actions are shared. These are: 

■  Mobility of individuals: through KA1 (in education and training and youth);  
■  Cooperation  partnerships:  through  KA2  (in  education  and  training  and  youth)  and  other  cooperation  actions  in  sport  and  Jean 

Monnet) ; and 

■  System level projects: through KA3 and ad hoc actions in sport and Jean Monnet 

This logic of these three main types of actions corresponds to those levels at which the programme aims to trigger change: individual, 
organisation and system. This is an improvement in the logic of the programme compared to the myriad of actions with different names 
under the predecessor programmes.   

These  broad  categories  of  actions  are  further  subdivided  into  a  small  number  of  types  of  actions  which  as  often  as  possible  share  a 
common name if they are common to several sectors. This enables to cater for a variety of needs within a broad category  of activities 
funded.  

116 

 
 
 
Expected outcomes and long-lasting impact 

Erasmus+ aims to deliver outcomes and long-lasting impact at its three levels of intervention as the figure above illustrates. This can be 
further summarised as follows: 

■  At individual level: the programme is aimed to bring positive changes at both learners (students, trainees, apprentices, young people 
and  volunteers)  and  practitioners  (teachers,  trainers,  youth  workers)  in  the  form  of  (not  exhaustive):  improved  skills  and 
competences  (including  soft  skills),  self-empowerment  and  self-esteem,  better  awareness  of  the  EU  values,  etc.  For  practitioners, 
additional outcomes are expected such as: enhanced motivation, opportunities to test and implement new practices, ability to address 
the  needs  of  the  disadvantaged,  etc.  The  achievement  of  these  outcomes  is  in  turn  expected  to  generate  long-lasting  impact  at 
individual  level  (e.g.  enhanced  employability,  entrepreneurship,  active  participation  in  society,  participation  in  formal/non-formal 
education  or  training,  etc.)  but  also  at  organisation  and  system  levels  (e.g.  improved  education  attainment  and  completion  rates,  
employability, transition to further levels of education, solidarity and career progression of staff).  

■  At  organisation  level:  the  transnational  cooperation  project  opportunities  offered  by  the  programme  are  expected  to  generate  the 
following types of outcomes (not exhaustive): development and/or implementation of new pedagogies or curricula, implementation 
of  new  organisational  practices,  enhanced  networking  with  foreign  partners  (including  outside  Europe  and  from  other  fields), 
improving the dialogue between the academic research arena and policy makers. etc. The achievement of these outcomes is in turn 
expected  to  generate  long-lasting  impact  at  system  level  notably  in  the  form  of  better  quality  of  teaching,  youth  work  and  sport 
activities, sustainable partnerships, increased levels of participation in sport, physical activity and voluntary activity, etc.  

■  At system level: much greater systemic impact than in the past (e.g. KA3 clearly sets a framework for system level-oriented actions) 
is expected overall. Anticipated outcomes at both EU and national levels relate to achieving: stronger awareness about key  policy 
challenges in education and training, youth and sport; enhanced mutual learning and good practice exchanges among policy makers 
and key stakeholders; better understanding of key EU tools and policies; supporting research and training about the EU, etc. This is 
in turn aimed to help achieve long-lasting impact in the form of:  
–  Enhanced quality, efficiency and equity of education and training systems and youth policies through the OMC;  
–  Effective implementation of reforms converging with the OMC at national level  
–  Effective implementation of EU tools for assessment, transparency and recognition of skills and qualifications acquired through 

formal, non-formal and informal learning at national level; 

–  Increased  visibility  of  the  external  dimension  of  the  programme  (both  within  and  outside  Europe)  and  credibility  to  support 

structural reforms in partner countries, etc. 

117 

 
 
 
 
 
The different levels of intervention and related types of actions are not to be seen in isolation but on the contrary as aiming to contribute 
to commonly shared objectives and to generate mutually reinforcing outcomes and impacts. More than in the past, spill-over effects are 
expected to materialise across Erasmus+. Overall, the logic of the programme is that the simplification it offers should help reach greater 
and  long-lasting  impact  at  the  individual,  organisation  and  system  levels  and  contribute  to  the  achievement  of  the  key  EU  strategic 
documents mentioned above. 

118 

 
 
Annex 5b: Erasmus+ Factsheet 

119 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
120 

 
 
 
 
 
 
 
 
Annex 5c: Specific objectives of Erasmus+ by level of intervention 

Specific objective 

Education and training 

a  To improve the level of key competences and 
skills, with particular regard to their relevance 
for the labour market and their contribution to a 
cohesive society, in particular through 
increased opportunities for learning mobility 
and through strengthened cooperation 
between the world of education and training 
and the world of work 

b  To foster quality improvements, innovation 

excellence and internationalisation at the level 
of education and training institutions, in 
particular through enhanced transnational 
cooperation between education and training 
providers and other stakeholders 

c  To promote the emergence and raise 

awareness of a European lifelong learning area 
designed to complement policy reforms at 
national level and to support the modernisation 
of education and training systems, in particular 
through enhanced policy cooperation, better 
use of Union transparency and recognition 
tools and the dissemination of good practices 

d  To enhance the international dimension of 

education and training, in particular through 
cooperation between Union and partner-
country institutions in the field of VET and in 
higher education, by increasing the 
attractiveness of European higher education 
institutions and supporting the Union's external 
action, including its development objectives, 
through the promotion of mobility and 
cooperation between the Union and partner-
country higher education institutions and 
targeted capacity-building in partner countries 

e  To improve the teaching and learning of 

languages and to promote the Union's broad 
linguistic diversity and intercultural awareness 

Envisaged effect at: 

Individual 
level 

Institutional 
level 

Systemic level 

EU 

National 

**** 

*** 

** 

* 

*** 

**** 

** 

** 

** 

** 

**** 

*** 

**** 

**** 

* 

* 

Envisaged effect in partner countries 

**** 

**** 

**** 

**** 

** 

* 

* 

* 

* 

f  To promote excellence in teaching and 

research activities in European integration 
through the Jean Monnet activities worldwide 

*** 

**** 

121 

 
 
 
 
 
 
 
 
 
Specific objective 

Youth 

a  To improve the level of key competences and 
skills of young people, including those with 
fewer opportunities, as well as to promote 
participation in democratic life in Europe and 
the labour market, active citizenship, 
intercultural dialogue, social inclusion and 
solidarity, in particular through increased 
learning mobility opportunities for young 
people, those active in youth work or youth 
organisations and youth leaders, and through 
strengthened links between the youth field and 
the labour market 

b  To foster quality improvements in youth work, 
in particular through enhanced cooperation 
between organisations in the youth field and/or 
other stakeholders 

c  To complement policy reforms at local, 

regional and national level and to support the 
development of knowledge and evidence-
based youth policy as well as the recognition 
of non-formal and informal learning, in 
particular through enhanced policy 
cooperation, better use of Union transparency 
and recognition tools and the dissemination of 
good practices 

d  To enhance the international dimension of 

youth activities and the role of youth workers 
and organisations as support structures for 
young people in complementarity with the 
Union's external action, in particular through 
the promotion of mobility and cooperation 
between the Union and partner-country 
stakeholders and international organisations 
and through targeted capacity-building in 
partner countries 

Sport 

a  To tackle cross-border threats to the integrity 
of sport, such as doping, match-fixing and 
violence, as well as all kinds of intolerance and 
discrimination 

b  To promote and support good governance in 

sport and dual careers of athletes 

c  To promote voluntary activities in sport, 
together with social inclusion, equal 
opportunities and awareness of the importance 
of health-enhancing physical activity through 
increased participation in, and equal access to, 
sport for all 

Envisaged effect at: 

Individual 
level 

Institutional 
level 

Systemic level 

EU 

National 

**** 

*** 

** 

* 

*** 

**** 

** 

** 

** 

** 

**** 

*** 

**** 

**** 

* 

* 

Envisaged effect in partner countries 

**** 

**** 

* 

** 

** 

*** 

** 

**** 

*** 

*** 

**** 

**** 

**** 

**** 

* 

** 

Source: Terms of reference of the mid-term evaluation

122 

 
 
 
 
 
 
Annex 5d: Predecessor programmes factsheet 

ERASMUS+ predecessor programmes 

1. The Lifelong Learning Programme 
The Lifelong Learning Programme (LLP)547 was designed to enable people, at 
any  stage  of  their  life,  to  take  part  in  stimulating  learning  experiences,  as 
well  as  developing  education  and  training  across  Europe.  Nearly  €7  billion, 
which ran from 2007-2013, funded a range of transnational learning mobility 
exchanges, study visits and networking activities. Most of the activities of the 
LLP continue under the new Erasmus+ programme 2014-2020. 

Over  the  course  of  its  lifespan,  the  LLP  provided  support  to  school  pupils,  university 
students,  adult  learners,  and  a  variety  of  projects  under  the  following  main  sub-
programmes: 

  Comenius for schools 
  Leonardo da Vinci for vocational education and training 
  Erasmus for higher education 
  Grundtvig for adult education 
 

Jean  Monnet  actions,  designed  to  stimulate  teaching,  reflection,  and  debate  on 
European integration 

Comenius 

The  Comenius  sub-programme  focused  on  all  levels  of  school  education,  as well  as  the 
individuals  involved,  including  pupils,  teachers,  local  authorities,  and  education 
institutions, among others. It aimed to: 

Improve and increase the mobility of pupils and staff across the EU 

 
  Enhance and increase school partnerships across the EU 
  Encourage language learning, ICT for education, and better teaching techniques 

547 http://ec.europa.eu/education/lifelong-learning-programme_en  

123 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                                            
 
  Enhance the quality and European dimension of teacher training 
 

Improve approaches to teaching and school management. 

The  total  budget  for  Comenius  in  the  2007-2013  period  was  EUR  1,190  million  and 
benefited to nearly 980,000 participants. 

The sub-programme funded the following main actions: 

•  Mobility of pupils and school staff  
•  Bilateral and multilateral partnerships between various schools in the EU 
•  Multilateral  projects  and  networks  relating  to  improving  language  learning,  ICT 

for education, and better teaching techniques. 

Leonardo da Vinci 

The Leonardo da Vinci sub-programme funded practical projects in the field of vocational 
education and training. The total budget for Leonardo da Vinci in the 2007-2013 period 
was EUR 1,820 million and benefited to around 590,000 participants. 

The sub-programme was aiming to: 

  Enhance the competitiveness of the European labour market by helping European 
citizens  to  acquire  new  skills,  knowledge  and  qualifications  and  have  them 
recognised across borders 

  Support  innovations  and  improvements  in  vocational  education  and  training 

systems and practices. 

Leonardo da Vinci funded the following main actions: 

•  Mobility of vocational education and training students and staff   
•  Bilateral  and  multilateral  partnerships  between  various  vocational  education  and 
training  providers  in  the  EU  for  transfer  of  innovation,  experience  or  good 
practices. 

•  Multilateral  projects  and  networks  relating  to  improving  the  quality  of  training 
systems  through  the  development  of  innovative  contents,  methods  and 
procedures for vocational education and training.  

124 

 
 
125 

 
 
 
 
Erasmus 

The  total  budget  for  Erasmus  programme  in  the 
2007-2013 period was EUR 3.1 million. In addition 
to the 230,000 students supported each year (1.6 
million  over  2007-2013),  Erasmus  also  provided 
opportunities  for  over  300,000  academic  and 
administrative staff in higher education, with 4,000 
institutions and 33 countries participating. 

The sub-programme was aimed to: 

The Erasmus sub-programme 
supported student and staff 
exchanges and transnational 
cooperation in the field of higher 
education. 

• 

Improve  and  increase  the  mobility  of  students  and  staff  in  higher  education  to 
study, teach and train across the EU 

•  Enhance  and  increase  higher  education  institutions  partnerships  and  networks, 
thus  promoting  innovation,  quality  and  relevance  of  higher  education  across  the 
EU.  

The Erasmus programme supported the mobility through grants and provided co-funding 
to transnational cooperation projects and networks. 

Grundtvig 

The  Grundtvig  sub-programme  focused  on  the  teaching  and  study  needs  of  adult 
learners, as well as developing the adult learning sector in general. The total budget for 
Grundtvig  programme  in  the  2007-2013  period  was  EUR  420  million  and  benefited  to 
nearly 170,000 participants. 

Covering teachers, trainers, staff, and adult learners, among others, the sub-programme 
aimed to: 

Increase the number of people in adult education 
Improve mobility conditions in adult learning 
Improve the quality and cooperation between adult education organisations 

 
 
 
  Develop innovative educational and management practices 

126 

 
 
 
  Ensure social inclusion through adult education 
  Support innovative ICT-based educational content, services, and practices.  

Grundtvig supported:  

•  Mobility of adult learners and adult learning staff    
•  Bilateral and multilateral partnerships between various adult learning providers in 

the EU 

•  Multilateral  projects  and  networks  relating  to  improving  the  quality  of  adult 
learning  through  the  development  of  innovative  contents,  methods  and 
procedures  for  adult  learning  and  making  adult  learning  more  accessible  to  the 
potential users.  

Jean Monnet 

The  Jean  Monnet  sub-programme  was  a  component  of  LLP  focusing  on  promoting 
teaching  and  research  on  European  integration  matters.  The  total  budget  for  the  Jean 
Monnet programme in the 2007-2013 period was EUR 140 million and benefited to over 
1,100 practitioners. 

It consisted of three key activities: 

  The Jean Monnet Action, designed to stimulate teaching, research, and reflection 
on European integration, consisting of Jean Monnet Chairs, Centres of Excellence, 
and Modules, among others; 

  Support for six specific academic institutions; 
  Support  for  Europe-wide  associations  active  in  the  area  of  European  integration 

research. 

These  were  complemented  by  conferences,  thematic  groups,  and  policy  support  within 
the European Commission. 

127 

 
 
 
 
7. Youth in Action 
Youth  in  Action548  was  the  Programme  of  the  European  Union  for  young 
people  from  2007  to  2013.  It  also  aimed  to  promote  out  of  school  mobility 
within  and  beyond  the  EU’s  borders,  non–formal  learning  and  intercultural 
dialogue, and encouraged the inclusion of all young people regardless of their 
educational, social and cultural background. 

YiA aimed to inspire a sense of active 
European citizenship, solidarity and 
tolerance among young Europeans and to 
involve them in shaping the Union's 
future.  

The  total  budget  for  the  programme  in  the 
2007-2013 period was EUR 885 million. 
Benefiting  to  close  to  1  million  participants, 
young people and youth workers, it had also 
contributed  to  the  effective  recognition  of 
non-formal  learning  with  265,000  Youthpass 
-  the  Youth  in  Action  learning  opportunities 
certificate - delivered since 2007. 

In  order  to  achieve  its  objectives,  the  Youth  in  Action  Programme  implemented  the 
following actions: 

  Action 1 - Youth for Europe: groups of young people from different countries plan 
together their Youth Exchange to learn about each other’s cultures; networking of 
similar  projects  in  order  to  strengthen  their  European  aspect;  support  to  young 
people’s participation in the democratic life at all levels. 

  Action 2 - European Voluntary Service:  young people take part individually or in 
groups in non-profit, unpaid activities, within and outside the European Union 
  Action  3  -  Youth  in  the  World:  cooperation  with  Partner  Countries  from  other 

parts of the world (exchange of good practice, etc) 

  Action  4  -  Youth  Support  Systems:  support for  organisations  and  youth  workers 

(training, networking, partnerships, etc) 

548 http://eacea.ec.europa.eu/youth/programme/about_youth_en.php  

128 

 
 
 
 
                                                            
 
  Action  5  -  Support  for  European  cooperation  in  the  youth  field:  between  those 
responsible  for  youth  policy,  those  active  in  youth  work  and  young  people, 
(seminars, Structured Dialogue, etc) 

8. Erasmus Mundus 2009-2013 
Erasmus Mundus II549 was a cooperation and mobility programme in the field 
of  higher  education  over  2009-2013.  The  total  budget  for  the  programme  in 
the  2007-2013  period  was  EUR  950  million  and  benefited  to  around  13,000 
participants in the 2007-2013 period. 

The Erasmus Mundus programme provided support to: 

 

  Higher  education  institutions  that  wished  to  implement  joint  programmes  at 
postgraduate  level  (Action  1)  or  to  set-up  inter-institutional  cooperation 
partnerships  between  universities  from  Europe  and  targeted  Third-Countries 
(Action 2); 
Individual students, researchers and university staff who wish to spend a study / 
research  /  teaching  period  in  the  context  of  one  of  the  above  mentioned  joint 
programmes 
cooperation 
or 
partnerships (Action 1 and Action 2); 
  Any organisation active in the 
field of higher education that wishes 
to  develop  projects  aimed  at 
enhancing  the  attractiveness  and 
visibility 
higher 
education worldwide (Action 3). 

Erasmus Mundus aimed to enhance the quality of 
European higher education and to promote 
dialogue and understanding between people and 
cultures through cooperation with the third 
countries 

European 

of 

549 http://eacea.ec.europa.eu/erasmus_mundus/index_en.php  

129 

 
 
 
 
                                                            
 
9. Tempus 

programme550 

Tempus 
the 
modernisation  of  higher  education  in  the  EU's 
surrounding  area.  The  total  budget  for  Tempus  IV 
was  EUR  258.7  million,  with  around  3,000 
participants in the 2007-2013 period. 

supported 

Tempus promoted a people-to-
people approach. 

The  programme  aimed  to  promote  voluntary  convergence  of  the  higher  education 
systems in the Partner Countries with EU developments in the field of higher education. 
With  regards  to  the  Western  Balkans,  Tempus  contributed  to  preparing  the  candidate 
and potential candidate countries for a participation in the integrated Life Long Learning 
Programme. 

Tempus  promoted  through  institutional  cooperation  the  modernisation  of  higher 
education systems in the Partner Countries of Eastern Europe, Central Asia, the Western 
Balkans and the Mediterranean region.  

In  addition,  Tempus  provided  support  to  consortia  of  institutions  composed  mainly  of 
universities  or  university  associations.  Non-academic  partners  could  also  be  part  of  a 
consortium.  

The  Tempus  programme  was  implemented  in  close  coordination  with  the  Erasmus 
Mundus programme which provided scholarships also to third country students allowing 
them  to  participate  in  top-level  Master  courses  and  Doctorate  programmes  outside  the 
EU. 

550 http://eacea.ec.europa.eu/tempus/programme/about_tempus_en.php  

130 

 
 
 
 
                                                            
 
10.  Alfa 
The ALFA III Programme551 aimed at the modernisation of Higher Education 
in  Latin  America  with  a  view  to  promoting  sustainable  and  equitable 
development in the region. The total budget for the programme in the 2007-
2013  period  was  EUR  950  million  (EU  contribution  of  EUR  75  million)  and 
benefited  to  153  participating  institutions  from  the  EU  and  341  from  Latin 

America. 

This programme aimed to ensure a process 
of ownership by the Latin American 
countries through the creation of networks 
and synergies between universities in Latin 
America and Europe.   

this 

regard,  ALFA 

In 
III  aimed  at 
strengthening  bilateral  and  multilateral 
relations  between  the  two  regions,  where 
higher education institutions play a leading 
role in the process of improving the quality 
of national education systems which in turn 
enables the socio-economic development. 

The  third  phase  of  the  programme  –  ALFA  III  2007-2013  –  financed  a  diversity  of 
projects  to  improve  the  quality,  relevance  and  accessibility  of  higher  education  in  Latin 
America and further regional integration through the creation of a higher education area. 
The  participating  countries  were  the  28  Member  States  of  the  European  Union  and  18 
countries of Latin America: Argentina, Bolivia, Brazil, Chile, Colombia, Costa Rica, Cuba, 
Ecuador,  El  Salvador,  Guatemala,  Honduras,  Mexico,  Nicaragua,  Panama,  Paraguay, 
Peru,  Uruguay  and  Venezuela.  Non-government  organisations,  chambers  of  commerce, 
professional associations, private companies had an associate role. 

551 http://ec.europa.eu/europeaid/where/latin-america/regional-cooperation/alfa/index_en.htm_en  

131 

 
 
 
 
                                                            
 
11.  Edulink 
The  total  budget  for  Edulink552  in  the  2006-2013 
period was EUR 58.3 million.  

that  will  enable  ACP  students 

The programme aimed to increase access to quality 
education 
to 
undertake  postgraduate  studies,  and  to  promote 
student  retention  in  the  region,  while  increasing 
the competitiveness of the institutions themselves, 
through 
regional  and  multilateral  networking 
between  higher  education  institutions,  capacity 
building  and 
intra-ACP  academic  mobility  of 
students and staff. 

The scheme provided support to: 

The programme was designed to 
foster co-operation in higher 
education between the countries of 
the African, Caribbean and Pacific 
Group of States (ACP States) and 
the EU 

  Higher education institutions to set up inter-institutional cooperation partnerships 

 

between universities from different countries within the ACP regions;  
Individual students, researchers and university staff to spend a study / research / 
teaching period in the context of one of the cooperation partnerships.  

552 http://www.acp-hestr.eu/  

132 

 
 
 
 
                                                            
 
12.  Preparatory actions in sport 2009-2013 

The main objective of Preparatory Actions553 was to prepare future EU actions 
in the field of sport. The funding in the 2007-2013 period for the preparatory 
actions was EUR 14.5 million. 

The  actions  funded  transnational  projects 
put  forward  by  public  bodies  or  civil 
society  organisations  in  order  to  test 
suitable  networks  and  good  practices  in 
the field of  sport. The preparatory actions 
also  served  to  support  knowledge-base  in 
the  sport  area  through  studies,  surveys 
and conferences. 

Those were mostly aimed to support 
cooperation among sport organisations, 
through small scale partnerships and the 
organisation of raising awareness events. 

For instance, the areas covered in the preparatory actions in 2012-2013 included:  

  Strengthening of good governance and dual careers in sport through support for 
the  mobility  of  volunteers,  coaches,  managers  and  staff  of  non-profit  sport 
organisations, 

  Protecting  athletes,  especially  the  youngest,  from  health  and  safety  hazards  by 

improving training and competition conditions, 
  Promoting traditional European sports and games, 
  Supporting the 'fight against match-fixing', 
  Promoting physical activity supporting active ageing, 
  Awareness-raising about effective ways of promoting sport at municipal level, 
  Trans-frontier  joint  grassroots  sport  competitions  in  neighbouring  regions  and 

Member States. 

553 http://ec.europa.eu/sport/policy/preparatory-actions/preparatory-actions_en.htm  

133 

 
 
 
 
                                                            
 
Annex 5e: Indicators of the Erasmus+ programme legal basis 

Overview of outputs (legal basis and strategic plan indicators) 

Legend: D = DG EAC dashboard; AR = Annual report; ICF survey = ICF survey of beneficiaries; O = Other 

Commission data (annual reports) 

Target 

State of play 

Legal 
basis 

2014 

2015 

2016 

Total (if 
applicable) 

2014–2016 

Judgeme
nt 

Comment 

Source 

Definition 

Learning mobility opportunities through Erasmus+ 

 - Higher education 
(without international) 

236,0
00 

239,0
00 

248,0
00 

723,000 

869,615 

 - VET 

Student Loan 
Guarantee 

78,00
0 

78,00
0 

80,00
0 

236,000 

279,982 

200,000 
(full period)  247 

Practitioners supported by the programme (Erasmus+, education and training) 

 - HE practitioners 
(excluding 
international) 

 - VET practitioners 

 -Schools practitioners 

 - Adult learning  

Assuming 
even 
distribution 
across 
years: 
150,000 

50,00
0 

159,177 

11,00
0 

As  above: 
33,000 

50,556 

15,00
0 

As  above: 
45,000 

54,341 

As  above: 
9,000 

12,971 

3,000 

Substantia
lly above 
target 

Substantia
lly above 
target 

Substantia
lly below 
target 

Above 
target 

Substantia
lly above 
target 

Substantia
lly above 
target 

Substantia
lly above 
target 

Number of 
students and 
trainees 
participating in 
the programme, 
by country, 
sector, action 
and gender 

Number of 
students who 
received the 
guarantee  

Number of 
practitioners 
supported by the 
programme, by 
country and in 
the sector of 
education and 
training 

D 

D 

O 

D 

D 

D 

D 

No target was set 
for years 2014 and 
2015 so it is 
assumed that it 
was the same as in 
2016 

Idem 

Idem 

Idem 

134 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Participants with special needs or fewer opportunities 

Education and training 

15,00
0 

Special needs 
(4,034) 

Fewer 
opportunities 
(9,580)  

Disadvantaged 
background 
(51,852) 

Outmost 
regions 
(9,148) - used 
in HE 

The categories are not 
mutually exclusive. 
Summing up the 
different categories 
would lead to double 
counting. However, 
indicator of 
disadvantaged young 
people in higher 
education alone is 
substantially above 
the target 

Substantia
lly above 
target 
according 
to 
programm
e data but 
issues of 
data 
quality 

D 

Assuming 
even 
distribution 
across 
years: 
45,000 

Participants with 
special needs or fewer 
opportunities (youth) 

International higher education 

Assuming 
even 
distribution 
- 64,800 

21,60
0 
(2017) 

Special needs 
– 6,551 

Fewer 
opportunities - 
76,737 

Substantia
lly above 
target 

Data concerns 
numbers of 
organisations in 
general including EU 
ones (capacity 
building and 
international student 
and practitioner 
mobility). 
Considering that in 
2016 there were more 
grants awarded than 
in 2015 the number is 
likely to be 
significantly over 
target even given that 
not all the 
organisations 
captured in this 
indication are from 
partner countries 
(some are from 
programme countries)  

A
R 

Involvement of non-EU 
higher 
education 
institutions from partner 
countries 

n/a 

1,000 

1,100 

2,200 

2,523 in 2015 
alone 

Substantial
ly above 
target 

students 

and 
EU 
practitioners  going 
to 
partner  countries  and 
vice versa 

 - Outgoing total 

 - Incoming total 

38,800 

39,448 

On target 

D 

n/a 

3,800 

4,000 

7,800 

11,016 (2,670 
learners) 

Substantial
ly above 
target 

The cap on number of 
outgoing students 
towards neighbouring 
countries was lifted in 
2016 

n/a 

15,000  16,000 

31,000 

28,432 

Below 

Where the numbers 

135 

Number of 
participants with 
special needs or 
fewer 
opportunities 
supported by the 
programme 

Number of 
participants with 
special needs or 
fewer 
opportunities 
supported by the 
programme 
(Erasmus+, 
youth) 

The number of 
partner country 
higher education 
institutions 
(HEIs) involved 
in credit and 
degree mobility 
and cooperation 
actions, i.e. 
capacity-building 
projects under 
the Erasmus+ 
programme and 
having signed an 
institutional 
agreement with 
an EU HEI 

Number of 
higher education 
students and 
practitioners 
(134) receiving 
support (a 
scholarship) to 
study in a partner 
country, as well 
as the number of 
students and 
practitioners 
coming from a 
partner country 
to study in a 
programme 
country 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Number of young 
people engaged 
in mobility 
actions supported 
by the 
programme, by 
country, sector, 
action and gender 

Number of 
practitioners 
supported by the 
programme, by 
country and for 
the youth sector  

Number of youth 
organisations 
from both 
programme 
countries and 
partner countries 
involved in 
international 
mobility and 
cooperation 
actions under the 
Erasmus+ 
programme 

D 

D 

A
R 

Youth mobility 

(16,413 
learners) 

target 

for incoming mobility 
are not used the 
budgets are used to 
fund outgoing 
mobility  

Learning 
opportunities 
Erasmus+ (youth) 

mobility 
through 

69,00
0 

70,000 

70,00
0 

209,000 

277,913 

Practitioners  supported 
by 
programme 
(Erasmus+, youth) 

the 

21,00
0 

21,000 

22,00
0 

64,000 

101,437 

Substantia
lly above 
target 

Substantia
lly above 
target 

Assuming 
2015 is the 
same as 
2014: 
16,600 

87,773 
(mobility) and 
3150 
(Strategic 
Partnerships) 

Substantia
lly above 
target 

5,500 

5,600 

Involvement  of  EU  and 
non-EU 
youth 
organisations 
(Erasmus+) 

Services/information networks 

indicator is 
not 
included in 
the 
strategic 
plan) 

Number  of  users  of 
Euroguidance  

n/a 

n/a 

n/a 

n/a 

Approx. 
62,000  (2014–
2015) 

The exact 
indicators 
reported differ 
according to 
reports. This 
data reflects 
participants in 
activities  

Number of users of 
Euroguidance  

O 

Users 
(enquiries answered) 

of  Eurodesk 

140,0
00 

140,00
0 

140,0
00 

420,000 

524,566 
(2014–2015  - 
n/a for 2016) 

Substantia
lly  above 
target 

The average 
per year is 
around 
250,000 rather 
than 140,000 

O 

The number of users 
of Eurodesk (as a 
support organisation 
to Erasmus+, 
Eurodesk, a network 
of 1,200 youth 
professionals, worked 
with Eurodesk in 34 
countries in 2016, 
making information 
on learning mobility 
accessible to young 
people and youth 
professionals) 

136 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Jean Monnet 

Students trained through 
Jean Monnet activities 

215,00
0 

235,00
0 

260,00
0 

710,239 

667,199 

The indicator 
on number of 
participants 
is not reliable 
(double 
counting) 
and 
inconsistency 
with other 
sources of 
data on 
participants 
reached 

Below target 
according to 
programme 
data and 
issues of data 
quality 

Worldwide scope of 
Jean Monnet activities 

Sport 

78 

80 

81 

81 

81 

On target 

Number of students 
receiving training 
through Jean Monnet 
activities 

Number of countries 
where Jean Monnet 
activities have been 
performed 
successfully, 
increasing knowledge 
in partner countries 

D 

D 

Estimates 
based on 
survey: 

- 75 % of 
respondents 
have 100 or 
fewer 
members 
(including no 
members) 

- 58 % have 
10 or fewer 
employed 
practitioners 

This data is 
not captured 
in Erasmus+ 
monitoring 
systems 

ICF 
survey 
(practi
tioners
) 

Substantially 
above target 

Size of membership 
of sport organisations 
(% of small grassroots 
organisations with 
fewer than 1000 
members) applying 
for, and taking part in, 
the programme, by 
country 

Size of membership of 
sport organisations 

30 % 
(2017) 

137 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Overview of results (legal basis and strategic plan indicators) 

Result indicator 

Legal 
basis  Target  

Definition 
strategic plan 

in 

2014–2016 performance 

Comment 

Judgeme
nt 

Source of 
data 

Education and training  

83 % 

Better skills for 
participants (Erasmus+, 
education and training) 

% of Erasmus+ 
participants 
declaring that 
they have 
improved their 
key competences 
and/or their skills 
relevant for 
employability 

87 % of beneficiaries state that they 
have improved six or more competences 
from the proposed list  

More than 80 % of learners agreed that 
they improved the following 
competences: cooperation in teams, 
planning and organisation tasks, 
planning and organising learning 
activities, problem solving skills, seeing 
the value of different cultures  

83 % of learners state that they believe 
they improved their professional skills  

44 % 

% of Erasmus+ 
participants 
indicating that 
participation in 
the programme 
contributed to 
finding a job 

88 % believe that their chances to find a 
job improved 

92 % of respondents agree that their 
opportunities for finding a job improved 

Beneficiaries from VET and HE 
experience shorter transition to 
employment than control group 

Employability of 
participants (Erasmus+) 

55 % 

Innovation from 
participating 
organisations 
(Erasmus+) 

Formal recognition of 
participation (Erasmus+; 
education and training) 

HE – 
100 % 

% of 
organisations that 
have 
developed/adopte
d innovative 
methods and/or 
materials, 
improved 
capacity; 
outreach 
methodologies, 
etc 

% of Erasmus+ 
participants who 
have received a 
certificate, 
diploma or other 
kind of formal 
recognition of 
their participation 
in the programme 

When asked about expected changes to 
their institutions, for 10 (out of 11) 
changes more than 55 % (up to 85 %) of 
practitioner respondents state that they 
agree these changes have or will happen  

When asked about changes that have 
happened after the mobility of project, 
more than 55 % of respondents agree 
with a high number of types of changes 
asked about 

Introduction of new teaching materials 
(81 % schools, 72 % VET, 72 % HE, 
77 % adult education) 

New assessment or evaluation methods 
(68 % schools, 63 % VET, 59 % HE, 
68 % adult education) 

100 % (estimate) 

The monitoring surveys of beneficiaries 
do not ask the question about whether 
learners received a certificate, etc. 
However, considering that the issuing 
of a transcript to mobile learners is one 
of the requirements of the HE 
University Charter which is a 
precondition to receive Erasmus+ 
funding, it can be assumed the rate of 
recognition of participation rate is 
100 % 

138 

On target 

Responses range from 
71 % for HE studies 
(long-term mobility) 
to 88 % long-term 
mobility in VET 

Erasmus+ 
monitoring 
survey of 
beneficiari
es 
(learners)  

The responses range 
from 72 % for HE 
studies to 89 % for 
VET 

From 82 % for HE 
studies to 90 % for 
international HE  

Ranging from 90 % 
for pupils (blended 
mobility) to 94 % for 
higher education 

Taking part in an 
Erasmus+ 
intervention appears 
to improve the 
transition from 
education to 
employment with a 
13 % increase in the 
indicator 

The Erasmus+ 
monitoring survey as 
well as ICF survey is 
based on a survey of 
practitioners taking 
part in the programme 
not organisations. 
Hence there is only a 
partial match between 
the definition and this 
data 

Recognition of 
learning outcomes 
(academic 
recognition) is at 
83 %. It ranges from 
75 % for international 
students to 89 % for 
traineeships  

Above 
target  

Idem  

ICF survey 
of 
beneficiary 
learners 

On target 
(possibly 
above) 

There is 
not a 
perfect 
match with 
the 
definition 
but the 
data 
collected 
from 
mobile 
practitione
rs gives 
very 
positive 
feedback 

On target 

Erasmus+ 
monitoring 
survey of 
practitione
r 
beneficiari
es  

ICF survey 
of 
beneficiary 
practitione
rs 

Erasmus+ 
monitoring 
survey of 
beneficiari
es(learners
)  

 
 
 
 
 
 
Result indicator 

Legal 
basis  Target  

Definition 
strategic plan 

in 

2014–2016 performance 

Comment 

Judgeme
nt 

Source of 
data 

88 % (2015)  

Above 
target 

The most common 
tool for recognition is 
Europass which is 
received by 80 % of 
participants  

VET – 
68 % 

Practitio
ners 

(not part 
of the 
targets) 

HE – 
95 % 

% of Erasmus+ 
participants in 
long-term 
mobility 
declaring that 
they have 
increased their 
language skills 

85 % of practitioners got international 
experience accepted as training (from 
83 % in HE to 89 % in schools) 

58 % state it was acknowledged by 
hierarchy or peers (53 % in HE to 65 % 
in AE) 

45 % state that it helped them attain a 
new function/ level of seniority (36 % 
in schools to 48 % in adult education) 

26 % received a financial reward (21 % 
in schools – 28 % in HE) 

84 % HE studies 

77 % international HE 

76 % HE traineeships 

Note: 14 % of students are fluent in the 
language of the mobility before 
departing on mobility 

Erasmus+ 
monitoring 
survey of 
beneficiari
es(learners
)  

ICF survey 
of 
practitione
rs 

Erasmus+ 
monitoring 
survey of 
beneficiary 
learners 

OLS data 
on pre-post 
tests 

Given that HE 
students start with a 
higher level of foreign 
language skills it 
would be expected 
that the improvement 
is more modest than 
in VET where the 
entry level is lower. 
The indicator has 
been set assuming 
that the duration of 
mobility makes the 
biggest difference but 
the entry level makes 
a very important 
difference  

Below 
target if 
taking as 
share of all 
students/ 
above 
target if 
excluding 
those who 
were 
already 
fluent 
before 
starting 
mobility 

On target 

VET – 
87 % 

86 % VET learners 

Note the data for VET does not cover 
only long-term mobility as suggested 
by the indicator 

According to OLS data:  

HE – 34 % large improvement, 14 % 
small improvement, 34 % no change 
and 19 % regressed score 

VET – very similar trends - 34 % large 
improvement, 13 % small 
improvement, 35 % no change, 18 % 
regressed score 

This data measures 
actual change in a 
score on a before and 
after language test  

It shows that there is a 
big difference in self-
perception values and 
actual competence 
development 

87 %  of  young  people 
in  youth 
exchanges state having improved six or 
more competences  

More  than  77 %  of  young  people  in 
youth exchanges state having improved 
problem solving (78 %), practical skills 
(planning etc. – 78 %), learning to learn 
(82 %), 
social 
competence  (92 %),  cultural  awareness 
and expression (93 %) and intercultural 
competences (84 %) 

interpersonal 

and 

Language skills of 
participants 

Youth  

skills 

Better 
for 
participants  (Erasmus+, 
youth) 

77 % 

%  of  Erasmus+ 
participants 
that 
declaring 
have 
they 
improved 
their 
key  competences 
and/or  their  skills 
relevant 
for 
employability 

Above 
target 

Erasmus+ 
monitoring 
survey  of 
beneficiari
es(learners
) 

Social 
political 
and 
participation  of  young 

80 % 

%  of  Erasmus+ 
young 

74 % of EVS participants state that they 
intend  to  participate  more  actively  in 

The question asked in 
Erasmus+ 
the 

Below 
target 

Erasmus+ 
monitoring 

139 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
people (Erasmus+) 

participants 
declaring 
being 
better prepared  to 
in 
participate 
social 
and 
political life 

social  and  political 
community  

life  of 

their 

73 % state that they are more interested 
in  knowing  what  happens  in  the  word 
daily 

71 %  are  more  aware  of  social  and 
political concepts 

monitoring 
does 
not 
correspond 
definition  

survey 
exactly 
the 
to 

It is not clear why the 
target was set so high 

survey  of 
beneficiari
es(learners
) 

ICF survey 
of learners 

Erasmus+ 
monitoring 
survey  of 
beneficiary 
learners 

OLS  pre-
post 
language 
tests 

Erasmus+ 
monitoring 
survey  of 
beneficiary 
learners 

Below 
target 
according 
to 
Erasmus+ 
monitoring 
data 
but 
strong 
improveme
nt based on 
OLS 

Substantial
ly 
above 
target  

(but  target 
was 
set 
very low) 

Above 
target 

ICF survey 
of sport 
practitione
rs 

There 
is  a  statistically  significant 
difference  between  young  people  who 
took  part  in  youth  exchanges  or  EVS 
when  it  comes  to  active  citizenship 
in 
(voting, 
community  activities  and  attitude  to 
protest strategies.  

participation 

etc), 

to  participate 

According  to  the  ICF  survey:  55 %  of 
youth beneficiaries think it is important 
to engage in political discussions (50 % 
it 
control  group),  89 %  believe 
is 
important 
local 
in 
(69 %  control 
community  activity 
active 
group), 
membership 
is 
in  control  group), 
important  (62 % 
80 %  believe  that  organising  a  local-
level  community  activity  is  important 
(69 % control group) 

believe 
in  an  association 

77 % 

81 %  of  participants  in  EVS  state  they 
improved their foreign language skills 

According  to  OLS  data  79 %  of  young 
people  taking  part  in  youth  actions 
improved  their  foreign  language  skills 
(69 %  large  improvement  and  10 % 
small improvement) 

The  data  used  for  the 
analysis  in  this  report 
index  which 
is  an 
responses 
combines 
perceived 
the 
on 
importance of a range 
of  civic  and  political 
activities 

When  looking  at  the 
detailed  responses  the 
differences  with  the 
control 
are 
large  in  some  cases 
(see left cell) 

group 

the 

self-

Though 
perceived 
improvement is below 
the 
the 
target 
improvement 
based 
on  OLS  is  large  and 
substantially  greater 
than for other groups 

77 %  of  EVS  volunteers  and  88 %  of 
young  people  in  youth  exchanges  state 
having  received  some  form  of  formal 
recognition  

The  vast  majority 
receive 
Youthpass 
(87 %  EVS  and  88 % 
youth exchanges) 

67 % of respondents state that they used 
the results to improve governance or 
manage quality  

27 % of respondents state that their 
project focused on the topic of good 
governance  

31 % state that their project focused on 
dual careers 

(multiple choice question) 

Data is not available 
through Erasmus+ 
monitoring surveys 
which are not carried 
out among sport 
beneficiaries 

The indicators are 
developed based on 
survey of sport 
organisations 
practitioners  

62 % of respondents state they used the 
results to combat threats to sport  

Idem 

Above 
target 

ICF survey 
of sport 
practitione
rs 

10 % state that their project focused on 

140 

skills 

Language 
of 
participants  (Erasmus+, 
youth) 

90 % 

35 % 

Formal  recognition  of 
participation (Erasmus+, 
youth) 

Sport 

Results increasingly 
used to improve good 
governance in sport and 
dual careers 

Results increasingly 
used to fight against 
threats to sport 

50 % 
(2017) 

50 % 
(2017) 

%  of  Erasmus+ 
participants 
in 
voluntary  actions 
that 
declaring 
have 
they 
their 
increased 
language skills 

%  of  Erasmus+ 
participants  who 
have  received  a 
(for 
certificate 
a 
example 
Youthpass), 
diploma  or  other 
formal 
kind  of 
recognition 
of 
their  participation 
in the programme 

Percentage of 
participants 
(expressed as % 
of Erasmus+ 
sport 
organisations) 
who have used 
the results of 
cross-border 
projects to 
improve good 
governance and 
dual careers 

Percentage of 
participants 
(expressed as 
Erasmus+ sport 
organisations) 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
that use the 
results of cross-
border projects to 
combat threats to 
sport 

combating match fixing  

12 % state the project focused on 
combating doping 

32 % state the project focused on 
combating racism and violence  

(multiple choice question) 

Percentage of 
participants 
(expressed as % 
of Erasmus+ 
sport 
organisations) 
who have used 
the results of 
cross-border 
projects to 
enhance social 
inclusion, equal 
opportunities and 
participation rates 

68 % of respondents state that their 
organisation introduced new ways of 
working with disadvantaged young 
people 

Idem 

73 % report that they implemented new 
ways to reach out to people 

53 % state the project focused on social 
inclusion and equal opportunities  

67 % state the project focused on 
encouraging participation in physical 
activities  

(multiple choice question) 

Results increasingly 
used to enhance social 
inclusion, equal 
opportunities and sport 
participation rates 

50 % 
(2017) 

Source: ICF, tab. 6.3 and 6.13 

Above 
target 

ICF survey 
of sport 
practitione
rs 

141 

 
 
 
 
Annex 6: Glossary 

Term or acronym 

Meaning or definition 

Action 

Activity 

A  strand  or  measure  of  the  Erasmus+  Programme.  Examples  of 
Actions are: Strategic Partnerships, Erasmus Mundus Joint Master 
Degrees, Sectors Skills Alliances, etc. 

A set of tasks carried out as part of a funded project. An activity 
can  be  of  different  types  (mobility  activities,  cooperation 
activities, etc.). In the framework of Jean Monnet, an Activity  is 
equivalent to an Action (see definition above). 

Adult education 

All forms of non-vocational adult education, whether of a formal, 
non-formal or informal nature. 

Adult learner 

Applicant 

Beneficiary 

Europass 

Any  person  who,  having  completed  or  is  no  longer  involved  in 
initial education or training, returns to some forms of continuing 
learning  (formal,  non-formal  or  informal),  with  the  exception  of 
school and VET teachers/trainers. 

participating  organisation  or  informal  group  that  submits  grant 
application. Applicants may apply either individually or on behalf 
of a other organisations involved in the project. In the latter case, 
the applicant is also defined as coordinator. 

If the project is selected, the applicant becomes beneficiary of an 
Erasmus+  grant.  The  beneficiary  sings  a  grant  agreement  with  – 
or  is  notified  of  a  grant  decision  by  –  the  National  or  Executive 
Agency that has selected the project. If the application was made 
on  behalf  of  other  participating  organisations,  the  partners  may 
become co-beneficiaries of the grant. 

Europass  is  a  portfolio  of  five  different  documents  and  an 
electronic  folder  aiming  to  contain  descriptions  of  the  entire 
holder's  learning  achievements,  official  qualifications,  work 
experience, skills and competences, acquired over time. Europass 
also  includes  the  European  Skills  Passport,  a  user-friendly 
electronic  folder  that  helps  the  holder  to  build  up  a  personal, 
modular inventory of his/her skills and qualifications. The aim of 
Europass  is  to  facilitate  mobility  and  improve  job  and  lifelong 
learning prospects in Europe. 

Fields 

Education and training, youth, and sport. 

Key Action (KA) 

The actions of Erasmus+ for education, training and youth include 
Key  Actions  1  (mobility  for  learners  and  practitioners),  KA2 
(cooperation among organisations) and KA3 (policy support). 

Informal learning 

Learning resulting from daily activities related to work, family or 
leisure which is not organised or structured in terms of objectives, 
time  or  learning  support;  it  may  be  unintentional  from  the 

142 

 
 
learner's perspective. 

Learners 

Learners  refer  to  all  individuals  involved  in  formal,  non-formal 
and 
informal  education  as  pupils,  students,  apprentices, 
volunteers, young people, etc. 

Level 
intervention 

of 

Three  levels  are  distinguished  to  evaluate  the  effects  of  the 
programmes:  individual  level,  divided  into  actions  for  learners 
and practitioners, organisation level and system level. 

Lifelong learning 

All  general  education,  vocational  education  and  training,  non-
formal learning and informal learning undertaken throughout life, 
in  knowledge,  skills  and 
resulting 
competences  or  participation  in  society  within  a  personal,  civic, 
cultural,  social  and/or employment-related  perspective,  including 
the provision of counselling and guidance services. 

improvement 

in  an 

Majority 
majority / most of 

/vast 

"Majority of …" is used for [50 to 60%[ of the respondents 
"Vast majority" for [56 to 75%[ ; "Most of…" for 75% and more 

Non-formal learning  Learning which takes place through planned activities (in terms of 
learning  objectives  and  learning  time)  where  some  form  of 
learning  support  is  present,  but  which  is  not  part  of  the  formal 
education and training system. 

Participants 

Participating 
organisation 

In  the  context  of  Erasmus+  participants  are  considered  those 
individuals  fully  involved  in  a  project  and,  in  some  cases, 
receiving part of the European Union grant intended to cover their 
costs  of  participation  (notably  travel  and  subsistence).  Under 
certain  Actions  of  the  Programme  (i.e.  Strategic  Partnerships)  a 
distinction  is  hence  to  be  made  between  this  category  of 
participants  (direct  participants)  and  other  individuals  indirectly 
involved in the project (e.g. target groups). 

Any organisation or informal group of young people involved in 
implementation  of  an  Erasmus+  project.  Participating 
the 
organisations  can  be  applicants  or  partners  (also  defined  as  co-
applicants, if they are identified at time of submission of the grant 
is  granted,  applicants  become 
application).  If 
beneficiaries  and  partners  may  become  co-beneficiaries  if  the 
project is financed through a multi-beneficiary grant. 

the  project 

Partner 
(organisation) 

Participating  organisation  involved  in  the  project  but  not  taking 
the role of applicant. 

Partner Countries 

countries  which  do  not  participate  fully  in  the  Erasmus+ 
Programme, but which may take part (as partners or applicants) in 
certain Actions of the Programme.  

Practitioners 

Practitioners  refer  to  those  involved  in  formal  and  non-formal 
education  as  teachers  (including  prospective  teachers),  trainers, 
youth workers, educators, coaches, organisation leaders or staff. 

143 

 
 
Programme 
Countries 

Sectors 

Vocational 
education and 
training (VET) 

Youthpass 

EU  and  non  EU  countries  (FY,  IS,  LI,  NO,  TR)  that  have 
established  a  National  Agency  which  participate  fully  in  the 
Erasmus+ Programme.  

The  education  and  training  field  is  compounded  of  five  sectors: 
School  education,  vocational  education  and  training,  higher 
education, international higher education and adult learning 

Education  and  training  which  aims  to  equip  people  with 
knowledge,  know-how,  skills  and/or  competences  required  in 
particular occupations or more broadly on the labour market. For 
the purpose of Erasmus+ projects focusing on initial or continuing 
vocational education and training are eligible under VET actions. 

The  European  tool  to  improve  the  recognition  of  the  learning 
outcomes  of  young  people  and  youth  workers  from  their 
participation in projects supported by the Erasmus+ Programme.  

144 

 
 
 
 
 
Abbreviations 

  AE: Adult Education 

  DG EAC: Directorate General for Education and Culture 

  EACEA: Educational, Audiovisual & Culture Executive Agency 

  EaSI: Employment and Social Innovation 

  EC: European Commission 

  ECAS: European Commission Authentication System 

  ECHE: Erasmus Charter for Higher Education 

  ECTS: European Credit Transfer and Accumulation System 

  ECVET: European Credit System for Vocational Education and Training 

  EECS: European Economic and Social Committee 

  EIB: European Investment Bank 

  EIT: European Institute of Innovation and Technology 

  EMJMD: Erasmus Mundus Joint Master Degree 

  EP: European Parliament 

  EPALE: Electronic Platform for Adult Learning in Europe 

  EPRP: Erasmus+ Project Results Platform 

  EQAVET: European Quality Assurance in Vocational Education and Training 

  EQF: European Qualifications Framework 

  ERDF: European Regional Development Fund 

  ESF: European Social Fund 

  ESIF: European Structural & Investment Funds 

  E&T: Education and Training 

  ET 2020: Education and Training 2020 strategic framework for European cooperation 

in education and training 

  ETF: European Training Foundation 

  EU: European Union 

  EUR: Euro 

  EVS: European Voluntary Service  

  EYE: Erasmus for Young Entrepreneurs 

  HE(I): Higher education (institution) 

  HERE: Higher Education Reform Experts 

  Horizon 2020: The EU Framework Programme for Research and Innovation 

  ICF: Consulting Service Ltd 

145 

 
 
 

 

 

ICT: Information and Communication Technology 

INTERREG: European Territorial Cooperation (ETC), also known as Interreg 

ISG: Interservice Group 

  JMO: Jean Monnet Operation (subject of a stand-alone evaluation report) 

  KA1: Erasmus+ Key Action 1 

  KA2: Erasmus+ Key Action 2 

  KA3: Erasmus+ Key Action 3 

  KII: Key Informant Interview ("interview" in the text) 

  LLP: Lifelong Learning Programme 

  NA: National Agency 

  NGO: Non-governmental organisation 

  NRS: National Report Synthesis (subject of a stand-alone evaluation report° 

  OECD: Organisation of Economic Cooperation and Development 

  OER: Open Educational Resources 

  OLS: Online Linguistic Support 

  OMC: Open Method of Coordination 

  OPC: Open Public Consultation (subject of a stand-alone evaluation report) 

  SE: School Education 

  SLGF: Student [/Master] Loan Guarantee Facility 

  VET: Vocational Education and Training 

  WG: Working Group 

  YiA: Youth in Action 

Add document title 1 

Add title 2 

146