Promoting the relevance of higher education : main report.
Date: 2018-07-10
Institutions: Directorate-General for Education, Youth, Sport and Culture
Subjects: EU Member State, EU employment policy, European citizenship, comparative education, database, education policy, educational system, higher education, knowledge economy, personal development, teaching quality

Promoting the Relevance of Higher 
Education 

Main Report 

 
 
 
EUROPEAN COMMISSION 

Directorate-General for Education, Youth, Sport and Culture 
Directorate B — Youth, Education and Erasmus+ 
Unit B.1 — Higher Education 

Contact: Nadia Manzoni 

E-mail: nadia.manzoni@ec.europa.eu  

European Commission 
B-1049 Brussels 

 
 
 
 
 
 
EUROPEAN COMMISSION 

Promoting the Relevance of 
Higher Education  

Main Report 

prepared by: 

Hans Vossensteyn 
Renze Kolster  
Frans Kaiser    
Jon File 

    Jeroen Huisman 
    Marco Seeber 
    Martina Vukasovic 

      Kai Muehleck 
      Christoph Gwosc 

2018 

Directorate-General for Education, Youth, Sport and Culture 
EAC-2015-0446 

EN 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
     
 
 
 
 
 
 
Disclaimer 
Neither  the  European  Commission  nor  any  person  acting  on  behalf  of  the 
Commission  is  responsible  for  the  use  that  might  be  made  of  the  following 
information. 

Europe Direct is a service to help you find answers  
to your questions about the European Union. 

Freephone number (*): 
00 800 6 7 8 9 10 11 

(*) The information given is free, as are most calls (though some operators, phone boxes 

or hotels may charge you). 

More information on the European Union is available on the Internet (http://europa.eu). 

Luxembourg: Publications Office of the European Union, 2018 

© European Union, 2018  
Reuse is authorised provided the source is acknowledged.  
The reuse policy of European Commission documents is regulated by Decision 2011/833/EU (OJ L 
330, 14.12.2011, p. 39). 
For any use or reproduction of photos or other material that is not under the EU copyright, permission 
must be sought directly from the copyright holders 

Catalogue number: NC-04-18-216-EN-N 
ISBN: 978-92-79-80311-6 
doi: 10.2766/048735 

Image © Source: istockphoto.com | Rawpixel 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Acknowledgements 

The project team would like to thank the following for their important 
contributions to this study 

Our senior advisors 

Professor Kate Purcell 
Warwick Institute for Employment Research, University of Warwick, UK  

Professor Ivan Svetlik 
Rector of the University of Ljubljana, Slovenia 
Former President of the Higher Education Council 

Professor Frans van Vught 
Honorary Professor, CHEPS 
Former Rector and President of the University of Twente 
Chairman of the Dutch Committee for the Review of Higher Education and Research  

Our case study researchers 

Prof. em. Paul Axelrod, York University, Canada (Ontario) 

Dr. Aleš Vlk, Tertiary Education and Research Institute (Czech Republic) 

Dr Martina Vukasovic and prof. Jeroen Huisman, CHEGG (Denmark) 

Emmanuel Boudard, La Rochelle Consult (France) 

Dr Kai Muehleck and Christoph Gwosć, DZHW (Germany) 

Leon Cremonini MA, CHEPS (Ireland) 

Renze Kolster M.Phil. and prof. Hans Vossensteyn, CHEPS (the Netherlands) 

Dr Marco Seeber, CHEGG (Spain) 

Our respondents 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Table of Contents 

2.1 
2.2 
2.3 
2.4 
2.5 
2.6 
2.7 

Abbreviations ........................................................................................................ 6 
Executive Summary ............................................................................................... 7 
The objective and structure of the study ................................................................ 7 
Key findings ....................................................................................................... 7 
Recommendations ................................................................................................ 10 
Introduction ................................................................................................... 12 
1 
1.1  Objective and structure of this study ..........................................................12 
Analytical Framework ................................................................................12 
1.2 
Research design .......................................................................................16 
1.3 
2  Policy levers for higher education relevance ....................................................... 20 
Introduction ............................................................................................20 
Importance of the relevance of higher education ..........................................20 
Policy levers for personal development ........................................................23 
Policy levers for sustainable employment ....................................................25 
Policy levers for active citizenship ...............................................................30 
Policy effectiveness ..................................................................................32 
Reflections and conclusions .......................................................................33 
Indicators for higher education relevance ........................................................... 35 
Introduction ............................................................................................35 
Indicators and monitoring of relevance in case study countries ......................35 
Review of indicators measuring the relevance of higher education ..................36 
Reflections and conclusions .......................................................................50 
4  Analytic and diagnostic tool .............................................................................. 53 
Introduction to the analytic and diagnostic tool ............................................53 
Country score cards ..................................................................................57 
Conclusions based on the analytic and diagnostic tool ...................................68 
5  Conclusions and recommendations .................................................................... 71 
Conclusions .............................................................................................71 
Recommendations ....................................................................................75 
Bibliography ........................................................................................................ 78 

3.1 
3.2 
3.3 
3.4 

4.1 
4.2 
4.3 

5.1 
5.2 

3 

5 

 
 
 
 
 
 
 
 
 
 
 
 
Abbreviations 

ABP 

AD 

Admissions Post Bac 

Associate Degree 

CALOHEE 

Comparing Achievements in Learning Outcomes in Higher Education in Europe 

CHEGG 

Centre for Higher Education Governance Ghent 

CHEPS  

Center for Higher Education Policy Studies 

COMUE 

Communautés inter-académique d'universités et d'établissements 

DG-EAC 

Directorate General Education and Culture 

DZHW  

Deutsches Zentrum für Hochschul- und Wissenschaftsforschung 

ESS 

EU  

European Social Survey 

European Union 

EU-LFS 

European Union Labour Force Survey 

EU-SILC 

European Union Statistics on Income and Living Conditions 

ICT 

Information and Communication Technologies 

ISCED  

ISCO   

HE 

HEA 

HEI 

International Standard Classification of Education 

International Standard Classification of Occupations 

Higher Education 

Higher Education Authority 

Higher Education Institution 

KLIM 

Key Indicators of the Labour Market 

KOAB   

Kooperationsprojekt Absolventenstudien 

KPI 

Key Performance Indicator 

MESR   

Ministère de lʼEnseignement Supérieur, de la Recherche et de lʼInnovation 

MinOCW 

Ministerie van Onderwijs, Cultuur en Wetenschappen 

MOOC   

OECD   

PETITE  

PIAAC   

Massive Open Online Course 

Organisation for Economic Cooperation and Development 

Pôles Étudiants Pour l'innovation, le Transfert et l'Entrepreneuriat 

Programme for the International Assessment of Adult Competencies 

STEM   

Science, Technology, Engineering and Medicine 

StraNES 

Stratégie Nationale d’Enseignement Supérieur 

UAS 

WVS 

Universities of Applied Sciences 

World Values Survey 

6 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Executive Summary 

The objective and structure of the study 
Europe’s  knowledge  economies  need  high-level  skills,  the  capacity  to  innovate  and  to 
support  democratic  societies.  In  this  perspective,  the  main  objective  of  this  study  is  to 
provide  a  comprehensive  analysis  of  the  relevance  of  higher  education  and  how  this  is 
promoted  in  various  countries  in  Europe  and  beyond.  The  evidence,  conclusions  and 
recommendations are intended to support EU Member States in developing and improving 
policies that promote the relevance of higher education: for students; for graduates; for 
employers; and for society. On the basis of a review of relevant literature, policy documents 
and databases, consultation with national experts and eight in-depth country case studies 
(Canada  [Ontario],  the  Czech  Republic,  Denmark,  France,  Germany,  Ireland,  the 
Netherlands  and  Spain)  that  included  interviews  with  many  national  stakeholders,  this 
study seeks to answer the following questions:  
  How is the relevance of higher education defined in different member states? 
  Which policy levers do member states use to promote higher education relevance? 
  Which  national  policies  concerning  higher  education  relevance  appear  to  be  effective 

and good practice examples? 

  Which  indicators  are  informative  in  assessing  higher  education  relevance  at  system 

level? 

  How can an analytic and diagnostic tool be designed and developed that can provide a 

systematic assessment of the relevance of higher education systems? 

Key findings 

Higher  education  is  relevant  when  it  contributes  to  personal  development, 
sustainable employment and active citizenship 
Relevance is understood differently by different stakeholders in higher education. These 
understandings  can  refer  to  the  competencies  of  individual  students  as  well  as  to  their 
collective outcomes for society. To analyse national policies concerning higher education 
relevance, and indicators to measure this relevance, we adopted the three main objectives 
of higher education formulated by the Council of Europe (2007): personal development, 
sustainable employment and active citizenship.  

Personal development relates to individual growth at the psychological, cognitive, social 
and  moral  levels.  In  terms  of  sustainable  employment,  higher  education  provides 
students  with  the  skills  to  secure  and  sustain  suitable  employment.  Active  citizenship 
encompasses the development of (inter)cultural skills, a sense of citizenship, and political 
literacy  and  participation.  In  addition,  higher  education  relevance  relates  to  different 
“users” of higher education: students, graduates, employers and society. 

Countries  differ  in  the  priorities  they  give  to  the  three  higher  education  relevance 
dimensions  and  the  different  higher  education  user  groups.  Sustainable  employment 
receives the most explicit attention, primarily as a result of the increasing emphasis on the 
contribution  of  higher  education  to  the  knowledge  economy  and  because  it  is  arguably 
easier  to  measure  than  personal  development  and  active  citizenship.  The  latter  two 
dimensions  are  addressed  more  implicitly  in  most  countries  as  they  are  assumed  to  be 
embedded  in  higher  education  practice.  Compared  to  most  other  studies,  this  report 
broadens the concept of relevance beyond employability. 

Countries utilise a variety of policies to address higher education relevance 
The eight case study countries promote higher education relevance using a wealth of policy 
levers,  categorised  for  the  purposes  of  this  study  into  one  of  the  following  four  groups: 
regulation, funding, organisation and information policies. 

7 

 
 
Personal development 

Personal  development  is  more  often  addressed  implicitly  than  explicitly.  Explicit  levers 
include  the  integration  of  personal  development  in  the  learning  outcomes  of  degree 
programmes  (regulation),  financial  support  for  under-represented  student  groups  to 
secure equal access (funding), supporting students’ mental health (organisation), and 
through student satisfaction and engagement surveys (information). 

Sustainable employment 

Sustainable  employment  receives  strong  policy  attention  in  most  countries,  particularly 
through  the  “hard”  policy  levers  of  regulation  and  funding.  Laws  and  regulations  in 
several  countries  include  a  link  between  the  number  of  study  places  and  labour  market 
needs, or define explicit labour market functions and learning outcomes for specific higher 
education sectors. Quality assurance and accreditation regulations often explicitly refer to 
sustainable  employment  criteria.  Typical  funding  policies  to  promote  sustainable 
employment include: 
  Performance funding (rewarding graduation and employment outcomes) 
  Scholarships and loans to stimulate graduation and particular professional fields 
  Strategic investment programmes guided by quality and employability criteria. 

Sustainable  employment  is  often  promoted  by  organisation  policies  such  as  the 
establishment  of  career  guidance  centres;  integrating  employers’  representatives  in 
programme advisory bodies and accreditation processes; offering new types of degrees or 
programmes; and regulated access for specific target groups. Finally, information policies 
address employability by means of student, graduate and employer surveys. In addition, 
platforms to inform students’ study choices increasingly include employment information. 

Active citizenship 

Overall, the active citizenship dimension of higher education is primarily supported through 
regulation and funding. Regulations for active citizenship often include:  
  The obligation to educate students for active citizenship 
  The facilitation of student participation in higher education governance 
  The stimulation of flexible curricula that enable students to engage in civic activity. 

Active citizenship is promoted by funding policies that: 
  Aim  to  expand  access  by  providing  financial  support  to  students  from  poor  socio-

economic backgrounds 

  Funding, recognising and awarding credits for involvement in student organisations 

The only organisational policy used to promote active citizenship is the Irish “Campus 
Engage” initiative, which stimulates students to volunteer with local organisations as an 
extra-curricular  activity.  Finally,  information  policies  include  student  surveys  that 
measure how students are integrated into civic activities and how this affects their social 
values. 

Insight into the impact and effectiveness of HE relevance policies is limited 
Overall, there is limited knowledge about the effectiveness of policies used to promote the 
relevance  of  higher  education  in  the  eight  case  study  countries.  In  most  countries,  the 
efficacy of policy levers is neither systematically evaluated nor monitored. If evaluations 
take  place,  they  often  focus  on  implementation  rather  than  on  outcomes.  However,  the 
limited evidence available (endorsed in stakeholder interviews) demonstrates that: 
  Labour market information allows students to make better educational choices (Spain) 
  Extra funding can increase the attractiveness  of STEM disciplines for  female students 

(Germany) 

  Organising part-time studies for the unemployed increases their employability (Ireland) 
  The introduction of associate degree programmes and excellence education tracks show 

positive effects for all three relevance dimensions (Netherlands) 

8 

 
 
 
  A  comprehensive  policy  approach  to  promote  employability  in  various  ways  creates 

strong awareness and relevant activities (France) 

More evaluations are needed to obtain enhanced insights into the effectiveness of policy 
levers and the reasons behind their success or failure. Based on the evidence available we 
can conclude that:  
  Policy instruments need to be designed for their specific national contexts 
  The  level  at  which  policies  need  to  be  developed  (national  or  institutional)  has  to 

carefully considered 

  Policy effectiveness improves when relevant stakeholders are involved in policy design 

and implementation processes 

Higher education relevance is assessed by many different indicators across Europe, 
however,  the  information  gathered  is  scattered  and  its  value  not  realised  by  most 
stakeholders 
There  are  many  readily  available  indicators  that  offer  insights  into  the  relevance  of 
higher education. Our main conclusions on the available data are: 
  Although  various  indicators  exist  for  each  of  the  dimensions  of  relevance,  most 
indicators suffer from a lack of periodicity and/or limited geographical coverage which 
prevents longitudinal and international analysis 

  Despite a variety of indicators that address sustainable employment, in most countries 
these  do  not  cover  key  aspects  such  as  labour  market  transitions,  graduate  careers, 
skills mismatches or graduate and employer views 

  While  various  aspects  of  personal  development  are  measured  in  several  countries  – 
including happiness, perceived health, trust in others and motivation - there is limited 
awareness among stakeholders of such indicators 

  Indicators  in  general  provide  an  impression  of  the  overall  performance  of  a  system 
rather than evidence of causal relationships between higher education and its societal 
outcomes and impact. 

An  analytic  and  diagnostic  tool  can  be  used  to  monitor  national  performance  in 
higher education relevance and to stimulate transparency and policy debate 
We have developed an analytic and diagnostic tool in the form of ‘country score cards’ that 
provide  per  country  a  1-page 
overview  of  the  state  of  the  art 
regarding  the  relevance  of  higher 
education. The ‘country score cards’ 
present: 
  The main system characteristics 

(first time entry, attainment, and 
educational expenditure) 

  The main national policy levers 

(regulations, funding, 
organisation and information) 
that are utilised 

  A selection of valid and 

comparable indicators per 
relevance dimension 

Regardless of the limitations of using 
monitoring  indicators  –  such  as 
limited  scope  and  availability  of 
robust, reliable time series data, and 
difficulties of causality – the insights 
gained  from  the  selected  indicators 

9 

 
 
imply positive effects of higher education in many countries. Skills levels and private and 
social returns are positively related to higher education and seem to be good measures of 
relevance. 

The indicators  of the ‘country  score cards’ are presented in a ‘wheel chart’.  These can 
guide  the  actions  of  higher  education  decision-makers.  Indicators  for  personal 
development include the level of trust in other people, happiness and perceptions of health. 
The indicators for sustainable employment are unemployment, private and public returns, 
relative  earnings,  vertical  mismatch,  and  the  distribution  of  ICT  skills.  The  indicators 
selected for active citizenship are efficacy in political participation and social background. 

Recommendations 
Based on the outcomes of this study we present a number of recommendations for national 
and EU policy-makers to promote the relevance of higher education. 

Governments should develop more explicit policy designs to boost relevance 
Though most policy makers and stakeholders across the eight case study countries agree 
that  the  relevance  of  higher  education  is  related  to  the  dimensions  of  personal 
development, sustainable employment and active citizenship, most countries give explicit 
policy  attention  to  sustainable  employment,  while  aspects  of  personal  development  and 
active  citizenship  are  often  addressed  implicitly.  Relevance  policies  can  be  made  more 
explicit in the following ways: 

  National policy mixes should target a proper balance between the three dimensions of 

higher education relevance; 

  Governments  should  be  clear  and  explicit  in  defining  and  communicating  the  specific 
relevance  dimensions,  aspects,  goals  and  targets  that  they  regard  as  important, 
including the reasons for these priorities;  

  National  policy  levers  need  to  be  explicit  about  the  expected  roles  of  different 

stakeholders. 

The  eight case studies  point at several examples of instruments that  “work”. These  can 
guide national governments in developing their own policy instruments: 

Sustainable employment 

  Provide extra funding for study programmes that address labour market shortages; 
  Undertake graduate and employer surveys to monitor the graduate labour market; 
  Involve labour market representatives in advisory committees and the quality assurance 

of education programmes; 

  Organise/improve career orientation and guidance. 

Personal development 

  Provide targeted funding for under-represented groups; 
  Integrate personal development explicitly in programme learning outcomes; 
  Measure levels of personal development in student (evaluation) surveys.  

Active citizenship 

  Allow credit to be awarded for extra-curricular activities and prior learning (non-formal 

learning); 

  Measure levels of active citizenship in student (evaluation) surveys. 

10 

 
 
 
 
 
 
 
Stimulate the collection of evidence on the effectiveness of higher education relevance 
policies, and monitor, share and adopt successful policy practices 
To  improve  the  limited  attention  for  policy  evaluation  and  monitoring,  the  European 
Commission and national governments should take the following actions: 

  To initiate more systematic national and international comparative empirical research 

on the impact and effectiveness of higher education relevance policies; 

  To link the higher education relevance agenda to other higher education policy areas, 

for example, modernisation, quality assurance and internationalisation; 

  Use good practice examples to inspire national practices, e.g.: 

o  Mandatory evaluations of national policies (Denmark and the Netherlands); 
o  The use of indicators addressing all three relevance dimensions (Germany: adequate 

employment of graduates, satisfaction of graduates and employers); 

o  Share  objective  and  experience-based  information  about  study  programmes  at  a 

central study portal (Studiekeuze 123 in the Netherlands). 

Governments as well as the European Commission should stimulate the collection of 
more robust data on the relevance of higher education 
It is desirable to organise coordinated action across national borders to build up a more 
solid knowledge base derived from commonly defined relevance indicators. 

  National governments should systematically collect information on the indicators of HE 

relevance using internationally shared definitions and should monitor outcomes; 

  Interaction between decision makers, practitioners and data providers at European and 

national levels could improve the quality and awareness of available data; 

  The  European  Commission  and  national  governments  should  invest  more  effort  in 
international studies that enhance the international knowledge base on the dimensions 
of HE relevance. Initiatives such as the recent recommendation of the European Council 
on tracking graduates, including the European pilot graduate survey (EUROGRADUATE), 
are promising steps in this direction; 

  The analytic and diagnostic tool designed and developed within this project is a useful 
starting point for the systematic monitoring of relevance indicators and can serve as an 
input for a qualitative policy debate on higher education relevance; 

  Linked to the analytic and diagnostic tool, national governments should be encouraged 
to  utilize  existing  indicators  on  the  personal  development  and  active  citizenship 
dimensions, such the level of trust in other people, level of happiness, self-confidence 
for political participation, and levels of social representation; 

  The European Commission and member states should supplement statistical indicators 
with in-depth analyses to improve the understanding of the relationship between higher 
education and its outcomes. 

A  feasible  approach  to  develop  an  explicit  policy  approach  to  all  three  dimensions  of 
relevance  is  for  national  governments  to  gradually  adopt  various  elements  of  these 
recommendations, while carefully integrating them into their specific contexts. 

11 

 
 
 
 
1  Introduction 

1.1  Objective and structure of this study 

The main objective of this study is to provide a comprehensive analysis of the relevance 
of higher education and how this is promoted in various countries in Europe and beyond. 
The  study  provides  evidence,  conclusions  and  recommendations  to  support  EU  Member 
States in developing and improving policies that promote the relevance of higher education 
for students/graduates, employers and society at large. 

In  order  to  meet  this  objective,  we  first  developed  an  analytical  framework  that 
distinguishes three dimensions of relevance and three important (groups of) stakeholders. 
This  analytical  framework  is  explained  in  Section  1.2.  In  Section  1.3,  we  explain  the 
research methodology of the project. The framework and methods are subsequently used 
to address the following questions: 

  Which  policy  levers  are  available  and  used  by  public  authorities  to  support  the 

relevance of higher education for societal needs? (Chapter 2) 

  What  conclusions  and  recommendations  can  be  drawn  about  the  advantages, 
disadvantages, applicability and limitations of these different policy levers? (Chapter 
2) 

  Which indicators can be found in the literature and in practice that can be used to 
assess the relevance of higher education teaching and learning activities for societal 
needs? (Chapter 3) 

  How can the relevance of higher education systems be captured in an analytic and 

diagnostic tool that provides a snapshot performance analysis? (Chapter 4) 

  What  overall  conclusions  and  recommendations  can  be  drawn  from  the  project? 

(Chapter 5) 

1.2  Analytical Framework 

1.2.1  Conceptualising relevance of higher education 
Based on an extensive literature review that confirmed our expectation that relevance is a 
multidimensional concept, we decided to conceptualise the relevance of higher education 
along  two  dimensions:  (1)  for  what  is  higher  education  relevant  (objectives  of  higher 
education), and (2) to whom higher education is relevant (“users”  of  higher education). 
Using work undertaken by the Council of Europe (2007), the analytical framework of this 
study  distinguishes  three  dimensions  of  the  teaching  and  learning  function  of  higher 
education: 

  Personal development 
  Sustainable employment  
  Active citizenship 

These three dimensions are the result of a wide consultation by the Council with a variety 
of stakeholders and reflect the purposes of higher education that can be found in academic 
literature (Castells, 2001; Clark, 1983; Marginson, 2011; Trow, 1975). At the same time 
they reflect the contemporary policy discourses on relevance. 

Personal  development  relates  to  a  process  of  change  and  transformation  at  a 
psychological, social, cognitive and moral level. Accordingly, higher education contributes 
to the development of personal development skills that are primarily useful for personal 
growth. More specifically, personal growth relates to attitudes towards oneself, perception 
of oneself, trust in others, values and well-being.  

12 

 
 
 
Sustainable employment can be regarded as an outcome of higher education as higher 
education is generally understood to provide students with the appropriate skills with which 
they can find and retain employment. Key terms associated with this task are, inter alia, 
basic skills, foundation skills, transferable skills, technical and vocational skills, 21st century 
skills, and lifelong learning. The latter is particularly important for employees to anticipate 
changing skills demands in domestic and international labour markets. 

Active citizenship comprises a number of skills, attitudes and values that help individuals 
to contribute to societal development. Firstly, this includes the development of intercultural 
skills, cultural literacy and instilling the values of tolerance, multiculturalism and diversity 
as  well  as  a  sense  of  global  citizenship.  Secondly,  preparation  for  active  citizenship  in 
democratic  societies  also  includes  the  development  of  political  literacy,  increasing  the 
interest and motivation for political participation (voting, involvement in political parties or 
activism in non-governmental organisations) and providing opportunities for volunteering 
and service learning.  

The  distinction  between  personal  development,  sustainable  employment  and  active 
citizenship serves to organise the analysis, including explorations of the linkages between 
the different dimensions (e.g. promoting personal development may also positively affect 
the employability and active citizenship of graduates). Moreover, a number of aspects that 
are rather prominent in policy discourse concerning higher education – knowledge society, 
regional development, innovation or entrepreneurship – cut across these three dimensions. 
For example, a knowledge society implies a knowledge economy which is closely related 
to  sustainable  employment.  But  knowledge  and  education  also  contribute  to  personal 
development  and  to  inclusive  and  stable  democratic  societies  (i.e.  active  citizenship).  It 
should  be  stressed  that  in  this  report,  a  higher  education  system  is  considered 
relevant if it is organised in such a way that it contributes to all three dimensions 
of relevance. 

The three dimensions can be linked to three higher education stakeholder groups. First, 
the direct users of higher education: students and graduates. Second, by using their skills 
and knowledge, graduates make higher education relevant to employers of highly educated 
labour.  Third,  graduates  also  contribute  to  society  at  large  which  benefits  from  highly 
educated citizens in a variety of ways. 

Thus,  the  concept  of  relevance  comprises  (1)  three  distinct  dimensions  –  personal 
development,  sustainable  employment  and  active  citizenship  and  (2)  three  distinct 
stakeholder  groups  identified  for  the  purpose  of  this  study  –  students/graduates, 
employers and society. As such, the concept of relevance is related to the concept of quality 
of higher education, but for the purposes of this study it is considered as distinct. While 
relevance concerns the identification and focus on the purposes of higher education, quality 
concerns  the  extent  to  which  a  higher  education  system,  institution  or  programme 
addresses such purposes. 

Through the literature review (see Annex 2) we identified aspects connected to the three 
dimensions of the teaching and learning function of higher education and in relation to the 
three groups of stakeholders. 

The  outputs/outcomes  of  higher  education  provide  insight  into  the  performance  and 
relevance  of  the  higher  education  system  for  those  involved  (including  policy  makers). 
Table  1.1  shows  the  aspects  of  these  outcomes  and  outputs  that  are  related  to  the 
relevance of higher education. 

13 

 
 
 
 
 
Table 1.1: The grid with aspects of outcomes and outputs of higher education relevance 

Stakeholders 

Students/Graduates 

Employers 

Society 

  Attitudes towards self and perception of self 
(sense of identity, self-respect, self-worth, 
self-efficacy)  

  Trust in others, cooperative spirit 
  Values (ethical standards) 
  Motivation 
  Health/well-being 
  Happiness 
  Student/graduate satisfaction with personal 

development & related HE aspects 

  Qualifications 
  Knowledge  
  Basic skills 
  Transferable skills (communication skills, 

creativity, critical thinking etc.) 
  Technical & professional skills 
  Competencies 
  Cultural capital (cultural literacy, aesthetic 

appreciation, etc.) 

  Social capital 
  Social prestige (associated with degree, 

knowledge, etc.) 

  Successful transition to labour market  
  Adequacy of employment 
  Earnings  
  Career opportunities 
  Long-term job security 
  Job satisfaction  
  Social prestige (associated with job) 
  Student/graduate satisfaction with 

sustainable employment & related HE 
aspects 

  Democratic values 
  Tolerance, intercultural skills & values 
  Political literacy, civic skills and (sense of) 

ability to have influence 

  Social participation & inclusion (e.g. 

participation in associations, unions, or 
social & community project volunteering) 

  Political participation (e.g. campaigning, 
protesting, voting, running for political 
functions) 

  Lower risk of becoming a criminal and/or 

becoming a victim of crime,  

  Student/graduate satisfaction with active 

citizenship & related HE aspects 

  Social dimension of HE 

  Responsible 
employees 

  Cooperative spirit 
in company, good 
teamwork 
  Motivated 
employees 

  Healthy employees 

  Employees able to 
flexibly adapt to 
changing tasks and 
challenges 

  Creative employees 
  Low staff turnover 
  Sufficient supply of 
highly educated 
employees 

  Diversity and good 

mix of 
qualifications, skills 
and competencies  

  Employers’ 

satisfaction with 
skills & 
competences 

  Employees’ 
productivity  

  Employee 

participation 
  Corporate Social 
Responsibility 

  Level of trust in society 
  Level of happiness 
  Level of health 

  Economic productivity  
  Extent of employment/ 

unemployment 

  Impact of HE on public 

revenues 

  Impact of HE on social 

security system 

  Impact of sustainable 

employment on families, 
friends, associations, 
neighbourhoods 
  Highly educated 

workforce 

  Innovation capacity of 

economy 

  Knowledge-based 

economy 

  Impact of HE on growth 
  External effects of skills 
and competencies on 
families, friends, 
associations, 
neighbourhoods 

  Strong civil societies, 
lively communities 

  Strong & lively 
democracies 

  External effects of 

democratic values, civic 
skills etc. on families, 
friends, associations, 
neighbourhoods 

  Less crime 
  Social dimension of HE 

Dimensions 
of HE 

Personal 
development 

Sustainable 
employment 

Active 
citizenship 

14 

 
 
 
1.2.2  Categorising policy levers 
To organise and structure the analysis of the variety of policy levers impacting on the three 
dimensions  of  higher  education  relevance  –  personal  development,  sustainable 
employment,  and  active  citizenship  –  in  relation  to  the  three  stakeholder  groups  – 
students/graduates,  employers  and  society,  we  distinguish  between  different  types  of 
policy instruments. Based on the typologies of Hood and Margetts (2007) and Van Vught 
and De Boer (2015), four categories of instruments are distinguished: regulation, funding, 
organisation and information. These instruments differ in their capacity to affect behaviour. 
Regulation  and  funding  are  generally  understood  as  “hard”  or  “strong”  tools  and 
organisation and information as “weak(er)” tools. 

  Regulation: The first category of policy instruments is regulations. These are intended 
to command and to forbid, to commend and to permit. Regulations vary by the degree 
of  restriction  they  seek  to  place  on  the  behaviour  of  higher  education  institutions  or 
students.  For  example,  authorities  regulate  admission  to  higher  education,  set  entry 
requirements  and  allow  and  forbid  certain  higher  education  institutions  to  offer 
particular types of programmes. Regulations may also impact on the procedures or the 
substance regarding the contents of curricula in higher education (see Berdahl [1983], 
for  a  distinction  between  substantive  and  procedural  autonomy).  In  unitary  states, 
regulations are developed at the national level, while in federal states they may be at 
either federal or state level. 

  Funding: The second category of policy instruments concerns funding. Funding enables 
governments to use financial sticks and carrots to influence behaviour. Authorities may, 
for instance, provide a bonus for every student higher education institutions are able 
to attract in a field with severe labour market shortages or to exempt students in these 
fields  from  paying  fees  or  to  offer  students  a  grant.  Also,  governments  may  charge 
higher tuition fees to students with high employability prospects and lower tuition fees 
to students in ‘national priority’ fields such as nursing and teacher training. Other uses 
of  funding  include  additional  budgets  for  innovative  teaching  programmes  or  for 
excellence  trajectories  and  performance-based  funding  linked  to  employability 
indicators. 

  Organisation: The third category is organisation. Within this category fall all kinds of 
operational  activities  directly  influencing  higher  education  structures.  An  example 
would be  setting up an agency to improve the match between student qualifications 
and the demands of the labour market (at the national level) or the establishment of 
student career centres in universities (organisational level). Also national or local public 
agencies that guide pupils in making a choice for a particular study area would fit here. 
Organisational aspects also have to do with the structures and procedures regarding 
teaching and learning. One can think of pathways to and within higher education (e.g. 
rules 
institutions  and  programmes),  and  structural 
characteristics  such  as  the  types  of  (sub-)degrees  being  offered,  the  duration  of 
studies,  the  opportunities  for  part-time  provision,  and  the  integration  of  online 
education provision (e.g. MOOCs). 

transition  between 

for 

  Information:  The  fourth  category  of  instruments  concerns  the  provision  of 
information.  From  its  specific  position  in  society,  government  often  is  ‘a  store  of 
information'. Compared with other institutions, governmental agencies are often better 
positioned to collect data and to develop rather broad, panoramic overviews of societal 
conditions.  Examples  would  be  to  make  employability  data  broadly  available,  or  to 
publish skills forecasts and information on the supply and quality of education. Through 
information and marketing campaigns, government may attract students to enrol in a 
discipline  or  field  that  is  characterised  by  a  significant  shortage  of  graduates  on  the 
labour market. 

15 

 
 
1.2.3  Overall analytical framework 
Using  the  conceptual  distinction  between  three  dimensions  of  relevance  and  the  three 
(most important) stakeholder groups as well as the policy lever typology developed above, 
Figure  1.1  presents  the  basic  model  that  guides  the  analysis:  policy  levers  are  used  by 
governments – in their specific national contexts – to reach (a combination of) dimensions 
of relevance that address (a combination of) stakeholder groups. 

Figure 1.1: Relationships between policy levers, dimensions and relevance for stakeholder groups 

of higher education 

1.3  Research design 

To answer the  research questions of this project, different research methods have been 
used. These are described below. 

1.3.1  Literature and indicator review 
An extensive review of the higher education literature was carried out to better understand 
and explore the notion of the relevance of higher education. First of all, the literature review 
resulted  in  the  conceptualisation  and  operationalisation  of  the  “relevance”  of  higher 
education. Based on this, the analytical framework for this study was designed (see Section 
1.1). Second, a review of the literature was carried out to explore the policy levers that 
are  applied  to  promote  the  relevance  of  higher  education  in  various  higher  education 
systems. Finally, a review of several, mostly international, databases was undertaken to 
critically assess the availability of indicators on the relevance of higher education and to 
select  indicators  that  can  be  used  to  measure  the  three  relevance  dimensions  of  higher 
education (see Chapter 3). The indicators that are readily available (and sufficiently robust) 
were used to develop an analytic and diagnostic tool for policy makers, researchers and 
practitioners. 

1.3.2  Selection of case studies  
To  study  in-depth  how  policy  levers  are  used  and  to  explore  their  potential  impact,  we 
carried out eight country case studies in Canada (Ontario), the Czech Republic, Denmark, 
France,  Germany,  Ireland,  the  Netherlands  and  Spain.  To  select  the  eight  case  study 

16 

 
 
 
 
 
countries  envisaged  for  this  study,  a  two-stage  approach  was  taken.  In  a  first  step,  17 
potential case study countries were selected that appeared to be interesting on the basis 
of the following criteria: 

  Countries  with  a  well-documented  record  of  past  and  present  higher  education 

policies; 

  Countries  with  a  relatively  high  number  of  country-specific  policies  related  to  the 
relevance of higher education based on our international policy literature review; 
  A  balance  in  the  size  and  regional  distribution  of the  European  countries  selected, 
thus representing different higher  education traditions and a variety  of geographic 
locations; 

  Two non-European countries that are relatively active regarding issues relating to the 
relevance of higher education as demonstrated in the international policy literature. 

Using these criteria, the following countries were selected: Australia, Belgium (Flanders), 
Canada  (Ontario),  the  Czech  Republic,  Croatia,  Denmark,  England,  Finland,  France, 
Germany, Ireland, Italy, Latvia, the Netherlands, Norway, Slovenia and Spain. 

For these 17 countries, a more systematic analysis of the respective national regulations, 
strategies, policies and monitoring instruments was made to provide a more targeted and 
transparent  overview  of  the  countries’  approaches  towards  the  relevance  of  higher 
education (see short country fiches in Annex 3). To put these analyses into context, the 
country  fiches  also  provide  a  brief  introduction  to  the  main  characteristics  of  the  higher 
education systems. These country fiches were developed in close consultation with national 
higher education experts and researchers.  

In  a  second  step,  based  on  the  analysis  of  the  country  fiches  for  the  17  pre-selected 
countries, we developed a more specific list of criteria to select eight case study countries 
for a more in-depth analysis of their national policy approaches towards the relevance of 
higher education and their effectiveness: 

  To  include  countries  that  were  mentioned  several  times  in  the  international  policy 
literature  review  as  having  a  relatively  high  number  of  country-specific  policies 
connected to the relevance of higher education; 

  To  select  countries  which  have  already  applied  relevance  policies  for  a  number  of 

years. This enables monitoring the impact and effectiveness of such policies; 

  To select countries that have a well-documented record of past and present higher 
education policies. This allows a more thorough analysis and comprehensive insight 
into country-specific policies, contexts and the potential relationships between policy 
levers and relevance outcomes; 

  To  select  a  mix  of  countries  in  terms  of  their  addressing  all  three  dimensions  of 
relevance  (personal  development,  sustainable  employment  and  active  citizenship), 
but also countries that strongly address one, or two of the three dimensions. 

  To seek a balance in the size and regional distribution of European countries, thus 
including  countries  with  different  higher  education  traditions  and  representing  a 
variety of geographic locations. 

  To include one non-European country that is active and – at first sight – successful 
in promoting the relevance of higher education (this was a requirement in the Tender 
Specifications). 

Applying  these  criteria  to  the  countries  mentioned  previously  and  after  consulting 
representatives from DG-EAC, the following eight countries were selected as case studies: 
Canada  (Ontario),  the  Czech  Republic,  Denmark,  France,  Germany,  Ireland,  the 
Netherlands and Spain (see Annex 1). The conclusions and recommendations presented in 
this report are primarily based on these eight case studies and comparisons across them.  

17 

 
 
 
 
1.3.3  Case study protocol 
A case study protocol was developed to guide the case study researchers. Most cases were 
conducted by core-research team members. The DZHW team conducted the German case 
study, the CHEGG team conducted the Danish and Spanish case studies, and the CHEPS 
team  carried  out  the  studies  on  the  Netherlands  and  Ireland.  Three  case  studies  were 
undertaken by external national experts: Canada (Ontario), the Czech Republic and France. 
The  national  experts  drafting  these  case  studies  worked  under  the  supervision  of  core-
research team members. 

The most important part of the case study protocol were the guidelines for conducting the 
case studies. The case studies were based on the following principles: 

1.  Secondary  analysis  of  national  policy  documents  (white  papers,  evaluation  reports, 

strategic documents, HEI documents, research articles, etc.); 

2.  At least ten expert interviews; 
3.  Analysis of the (national) statistical indicators that are regarded as most important for 

the relevance of higher education. 

1.3.4  Interview protocol 
The  method  of  interviews  with  experts  is  often  termed  ‘elite  interviewing’  and  is  used 
especially to corroborate other (written or interviewed) sources, establish the viewpoints 
and opinions of stakeholders involved in a process, and to reconstruct an event or process. 
Such interviews are normally semi-structured, with a predetermined set of basic questions 
formulated in a protocol. Interview questions focused on the following major issues: 

  Perceptions of what the relevance of higher education is (dimensions) and on how 

important it is in the system. 

  Which  policies  explicitly  addressing  higher  education  relevance  have  been 
implemented in the areas of regulation, funding, organisation and information? Are 
these well designed and do they address the appropriate target audience? 

  Which data and information is available and used to assess the relevance of higher 

education? Which data is lacking or is not of sufficient quality?  

  What seem to be the effects of these policies? What policies appear to work and which 

ones not? 

  Various  stakeholders  were  requested  to  reflect  on  national  policy  levers, 
measurements and potential effectiveness from their organisational perspective. The 
main  stakeholders  included  representatives  from  ministries,  higher  education 
institutions,  rectors’  conferences,  employer  organisations,  student  organisations, 
accreditation agencies and educational advisory bodies. These stakeholders were also 
asked  to  reflect  on  how  national  policies  relate  to  their  interests  and  whether  and 
how national policies are translated into their own policy instruments. 

1.3.5  Analytic and diagnostic tool 
The  diagnostic  tool  aims  at  giving  a  snapshot  impression  of  a  country’s  profile  and 
performance regarding the main dimensions and indicators of higher education relevance, 
as well as an overview of a country’s policies used to promote higher education relevance. 
For  relevance  policies,  the  diagnostic  tool  offers  information  about  the  scope,  focus  and 
instruments of the relevance policies applied in a country. The indicator profiles provide an 
impression  of  the  country’s  performance  with  regard  to  the  three  higher  education 
relevance  dimensions:  (1)  personal  development,  (2)  sustainable  employment,  and  (3) 
active citizenship. Indicators have been selected that are crucial to these three dimensions 
and  that  provide  comparable  data  across  the  case  study  countries.  To  select  such 
indicators, the long list of indicators found in the literature (see Chapter 3 and Annex 2) 
was used as the point of departure. To develop an impression of the “contribution” of higher 

18 

 
 
 
education to achieving relevance in the three dimensions, the scores on the indicators for 
persons  with  higher  education  attainment  are  compared  to  persons  with  medium 
education.  This  approach  offers  insights  into  the  “value  added”  of  higher  education  for 
relevance.  We  acknowledge  that  factors  other  than  education  levels  affect  personal 
development,  active  citizenship,  and  sustainable  employment.  These  include  social 
background, primary and secondary education, or context conditions such as the impact 
of the European economic crisis on graduate employment. In this sense, (relative) scores 
cannot be fully attributed to higher education experience. 

1.3.6  Generating conclusions and recommendations 
The conclusions and recommendations of this study were derived in an iterative process 
from the data and information collected in this study. Within the project meetings of the 
core research team the main findings from the different parts of the study were discussed, 
such as the literature review, the database review, the consultation of national experts and 
the  case  studies.  In  these  meetings  the  information  was  structured  along  the  three 
relevance dimensions, the four types of policies as well as by country. The information was 
summarised  in  tables  comparing  countries,  relevance  dimensions  and  types  of  policy 
levers. This allowed the team to recognise patterns and striking examples and to identify 
issues  to  explore  further.  The  eight  in-depth  case  studies  also  explicitly  explored 
stakeholders’ perceptions of relevant policies, evidence on their effectiveness and impact 
as well as stakeholder reflections on “what works”. By reading each other’s reports, team 
members were better able to understand the wealth of information and to select the most 
prominent findings and lessons from this study. Two interim reports were used to gather 
feedback  from  DG-EAC  and  the  Senior  Advisors  of  this  study  on  the  quality  and  main 
outcomes of the study, on how to present these and how to translate the outcomes into 
conclusions and policy-relevant recommendations. 

19 

 
 
 
 
2  Policy levers for higher education relevance 

2.1  Introduction 

In this chapter we analyse and compare the information from the eight case studies in a 
search  for  common  patterns,  good  practice  examples  and  overarching  conclusions.  The 
main issues and questions addressed are: 

1.  What importance is given to the different relevance dimensions of higher education in 

the eight case study countries? (Section 2.2) 

2.  Which  policy  levers  are  used  to  stimulate  the  three  relevance  dimensions  of  higher 
education  in  the  case  study  countries  and  what  types  of  policy  levers  –  regulation, 
funding,  organisation  or  information  –  are  most  frequently  applied?  (Sections  2.3  - 
personal development, 2.4 - sustainable employment and 2.5 - active citizenship) 

3.  Are  the  relevance  policies  effective  based  on  evidence,  evaluations  or  stakeholder 

perspectives? (Section 2.6) 

4.  Can  we  discern  any  good  practice  examples  in  terms  of  types  of  policy  levers  used, 
successful  policy  implementation  and  monitoring  practices?  What  overall  conclusions 
can be drawn in terms  of how national practices relate to the  conceptual framework 
used for this study and to specific national contexts? 

2.2  Importance of the relevance of higher education 

Following  the  conceptual  framework  of  the  project,  the  relevance  of  higher  education  is 
described  along  two  lines:  (1)  which  dimensions  of  higher  education  relevance  are 
addressed, and (2) who is being addressed (or benefits)? 

Higher  education  policies  in  the  eight  higher  education  systems  tend  to  cover  all  three 
relevance  dimensions  –  although  the  term  “relevance”  is  not  always  used  within  the 
national context (e.g. the term “relevance” does not always have a national equivalent). 

Table 2.1 presents the dimensions of relevance and the beneficiaries that are particularly 
significant  in  the  eight  systems  considered.  Sustainable  employment  is  important  in  all 
eight cases. Personal development and active citizenship are less frequently indicated as a 
high  priority,  and  in  some  cases  they  are  framed  by  interviewees  and  policy  papers  as 
ultimately  enhancing  the  chances  of  employability.  With  regard  to  beneficiaries, 
students/graduates and society at large are more frequently addressed than employers. 
Not  surprisingly,  employers  are  regarded  as  a  target  group  only  on  issues  regarding 
sustainable employment. 

Table 2.1: The priorities given by countries to relevance dimensions and beneficiaries 

Stakeholders  

Students/Graduates 

Employers 

Society 

Dimensions of HE 

Personal development 

Sustainable employment 

Germany, Ireland, 
Netherlands 

Czech Republic, Denmark, 
France, Germany, Ireland, 
Netherlands, Spain 

Czech Republic, Germany, 
Ireland, Netherlands, 
Ontario, Spain 

Active citizenship 

France, Germany, 
Netherlands 

Czech Republic, Ireland, 
Netherlands 

Czech Republic, Denmark, 
Germany 

Czech Republic, France, 
Germany, Spain 

Employability-related aspects of higher education clearly dominate policy documents and 
monitoring  systems  in  the  Czech  Republic,  Denmark,  France  and  Ontario.  Yet  the 
understanding  of  employability-related  aspects  is  not  fully  aligned  with  the  concept  of 
“sustainable employment”. In these countries there is a discussion about whether the focus 

20 

 
 
 
 
 
 
 
should  be  on  general  transferable  competences  and  soft  skills  or  on  directly  applicable 
professional  skills  and  field-specific  knowledge.  Danish  policies  focus  on  avoiding  high 
unemployment as this is regarded as a major societal risk. 

In  most  countries,  the  dimension  of  personal  development  appears  to  be  important  for 
most stakeholders. Nevertheless, it is seldom regarded as an explicit policy objective but 
rather a key characteristic that is implicitly addressed in higher education. In France, this 
implicit treatment of personal development was stressed. In the Netherlands the impact of 
higher education on economic development is a priority, but stakeholders also stress the 
benefits  of  personal  development  for  sustainable  employment.  In  Germany,  federal  and 
state  level  policy  makers  regard  personal  development  as  a  crucial  task  of  higher 
education. Some policy levers are directed at personal development (see below). Moreover, 
the role of higher education in promoting active citizenship as well as its long-term benefits 
for economic and regional development is emphasised by German policy makers at both 
the federal and state levels. 

Expansion of 
student 
participation in 
Germany 

In  the  German  state  of  Lower  Saxony,  the  government  has  strengthened  the 
participatory  culture  within  higher  education  institutions  in  the  new  Higher 
Education Act issued in 2015. The rights of students and student organisations 
to  participate  in  the  governance  of  higher  education  institutions  have  been 
extended in various ways: students now can submit topics to be discussed in the 
governing  bodies  of  higher  education  institutions;  they  have  to  agree  to  the 
appointment  of  the  Vice-President  of  the  Department  for  Study,  Teaching  and 
Student Affairs; students can also be appointed as avocational Vice-Presidents, 
and  student  representatives  can  attend  and  vote  (advisory)  in  meetings  of 
institutional  advisory  bodies  such  as  the  Hochschulrat  and  Stiftungsrat.  These 
changes also encourage the social commitment of students and institutions. 

Compared  to  other  countries,  Ireland  has  rather  detailed  regulations  concerning  the 
functions  of  the  different  types  of  higher  education  institutions.  Particularly  noteworthy 
with respect to relevance is the required vocational/professional orientation of Institutes 
of Technology, though their role may change substantially with their intended “upgrading” 
to  Universities  of  Technology.  The  focus  on  sustainable  employment  appears  to  have  a 
rather short-term focus, thus risking losing sight of broader societal impact and perhaps 
also personal development and active citizenship. 

Professional 
orientation of IoT 
in Ireland 

In  Ireland,  the  Institutes  of  Technology  offer  higher  education  and  training 
programmes with a vocational orientation. The institutes have a high degree of 
autonomy in deciding how they link study programmes to the professional field. 
Yet, all institutes have developed links with employer bodies in their regions and 
offer career guidance to their students (Kolster & Westerheijden, 2014). 

In Spain, employability is pursued particularly through the accreditation procedures and in 
some regions also by providing career information to prospective students. 

In  countries with binary higher  education systems, such as Denmark, Germany and the 
Netherlands,  the  Universities  of  Applied  Sciences  have  a  stronger  focus  towards 
professional employability than the universities have.  In France this is also  very  explicit 
with  the  Grandes  Écoles  and  the  two-year  professional  diploma  institutions  offering 
curricula for well-defined jobs while universities have a broader labour market preparation 
approach that revolves around generic transversal skills.  

Many  governments  have  issued  strategic  documents  setting  the  framework  for  policy 
instruments. Table 2.2 shows in which countries national higher education policy strategies 
give attention to particular dimensions of relevance. 

21 

 
 
 
 
 
 
Table 2.2: Strategies related to higher education relevance dimensions 

Dimensions of HE  

Personal development 

Sustainable employment 

Active citizenship 

Countries 

Canada (Ontario) 

Czech Republic 

Denmark 

France 

Germany 

Ireland 

Netherlands 

Spain 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

From Table 2.2 it can be seen that – except for Ontario – all case study countries have 
higher education strategies that address the relevance of higher education. As indicated 
before, most priority is given to the sustainable development dimension. Only in Spain and 
the Netherlands are the personal development and active citizenship dimensions explicitly 
targeted in their national higher education strategies. 

In France, the main aim of the national higher education strategy (StraNES) is that 50% 
of the relevant age group obtains a higher education qualification. Professional insertion 
(e.g. internships) is the second priority. This ambition builds on the 2007 “Plan Campus” 
and  “Future  Investment  Plan”  to  modernise  universities  and  their  infrastructures  and  to 
boost  the  quality  of  teaching  and  research.  Bachelor  graduates  were  to  become  more 
professionally oriented to enhance employability. 

The  2011  Dutch  strategy  for  higher  education  highlighted  the  sustainable  employment 
dimension.  The  subsequent  2015  Dutch  strategy  for  higher  education  stresses  the 
importance  of  quality  in  higher  education,  and  -  although  the  term  ‘relevance’  is  not 
explicitly used - the strategy is strongly linked to enhancing all three relevance dimensions, 
with particular attention given to personal development (Bildung). 

In Germany there is no overarching national strategy for higher education. Arguably, this 
is due to the division of competences between the federal level and the level of the states, 
the latter being primarily responsible for (higher) education. Some of the federal strategies 
pay attention to sustainable employment. 

Over  the  past  decade,  the  Irish  government  has  promoted  the  sustainable  employment 
dimension  of  the  relevance  of  higher  education,  both  through  strategies  and  connected 
policies. The economic downturn after 2007 has been mentioned as a key event triggering 
the focus on sustainable employment.  

In  Spain,  the  national  strategic  document  for  higher  education  (Estrategia  Universidad 
2015) stresses the concept of higher education as a public service. As such, the Spanish 
strategy  appears  to  integrate  the  personal  development,  sustainable  employment  and 
active citizenship dimensions in the development of society. 

An integrated 
strategic 
conception of the 
relevance of higher 
education in Spain 

In  Spain,  the  Estrategia  Universidad  emphasises  the  social  dimension  of  the 
universities, which is expected to contribute to the modernisation of the society 
by  introducing  new  ideas,  fostering  a  knowledge-based  critical  spirit,  to 
contribute to social cohesion, cultural and civic values, and to being an agent for 
economic and technological development. 

22 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2.3  Policy levers for personal development 

Higher  education  has  the  potential  to  strongly  contribute  to  personal  development, 
particularly  as  the  majority  of  students  attend  higher  education  at  an  age  when  they 
develop into independent adults and responsible actors in society. This dimension of higher 
education relevance can, inter alia, be connected to attitudes towards self and perception 
of self, cooperative attitudes, trust in others, motivation, happiness and health. Table 2.3 
illustrates the policy levers employed to promote this dimension of relevance in the eight 
higher education systems. 

Table 2.3: Personal development policy levers by type of policy instrument 

Regulation 

Funding 

Organisation 

Information 

Policy 
instrument 
type 

Countries 

Canada (Ontario)  Obligatory degree level 

expectations and 
learning outcomes in 
terms of personal 
development 

Special funding for 
(underrepresented) 
target populations (since 
2014) to “create a space 
to learn for every eligible 
student” and “realising 
academic potential” 

Centre for Innovation 
on Campus Mental 
Health established to 
share best practices 
for combatting mental 
health problems of 
students 

HEIs participate in 
National Survey of 
Student Engagement 
(with some measures 
of students’ 
developmental 
trajectories) 

Czech Republic 

Denmark 

France 

Germany 

Ireland 

Netherlands 

Law dictated functions 
of higher education 
institutions 

Law dictated functions 
of higher education 
institutions 

Spain 

Specific student loans for 
vulnerable groups (at 
the same time student 
support to be reformed: 
decrease grants, 
increase loans) 

Quality Pact for Teaching 
in Higher Education 

Support of student 
mobility e.g. by the 
German Academic 
Exchange Service 

Student Support 

Performance agreements 

Funds available for 
quality initiatives 
through adjustments in 
study finance system 

Demand-based funding 

Student Survey of AG 
Hochschulforschung 
(federal law) 

Organisation of 
excellence education 

Associate degree 

Study choice 123 

Table 2.3 shows that the Czech Republic, France and Spain do not have explicit policies 
that address the personal development dimension. However, this absence does not mean 
that the personal development dimension is not given any consideration. In fact, in France 
– as stated in Section 2.2 – personal development is a function inherently expected from 
higher education. Likewise, in Spain (and in Ireland), the conception that higher education 
is very important for personal development is strong among students and their families. 
This  seems  to  be  at  odds  with  the  official  policy  conception  that  higher  education  has 
foremost to provide economic returns. 

23 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Higher education systems that do explicitly address the personal development dimension 
of relevance do so using a variety of policy instruments. Three systems use regulations 
to connect higher education to personal development outcomes (Ontario, Ireland and the 
Netherlands). In Ontario regulations are the strongest and require universities to prescribe 
“degree level expectations” and “learning outcomes” for all undergraduate and graduate 
programmes on  criteria related to aspects  of personal development (e.g. autonomy and 
professional capacity). 

Funding policies are most often used to address aspects of personal development. These 
include  student  financial  support  schemes,  special  funds  for  specific  target  groups  and 
funds  for  specific  initiatives.  Student  financial  support  is  connected  to  personal 
development  in  Denmark,  Germany  and  Ireland  because  it  enables  students  to  attain 
higher education to increase their personal development.  It  should be  noted that this is 
not  the  only  reason.  In  Denmark  student  financial  support  is  also  connected  to  active 
citizenship and in Ireland also to sustainable employment. 

Ontario,  Denmark,  and  Germany  provide  special  funds  for  specific  target  groups.  These 
funds are particularly aimed at enhancing access to higher education for underrepresented 
(Ontario) or vulnerable (Denmark) groups.  

Both  Germany  and  the  Netherlands  have  made  funds  available  for  initiatives  related  to 
personal development. Germany established a large strategic fund to enhance institutional 
advice  and  service  capacities,  study  conditions,  the  quality  of  teaching  and  retention 
outcomes. In the Netherlands a short/medium term fund – available through adjustments 
in the study finance system – is to be, inter alia, used to appoint 4,000 extra teachers to 
promote more personal and innovative education. This fund is also related to the higher 
education strategy debate in the Netherlands around the concept of “Bildung” (see example 
below). 

Personal 
development in 
terms of “Bildung” 
in the Netherlands 

In the Netherlands, a renewed emphasis on personal development has emerged 
in the policy discourse. Various stakeholders as well as the national strategic plan 
for  higher  education  (MinOCW,  2015)  indicate  that  the  strong  focus  on 
employability and skills for the future labour market – 21st century skills – should 
be  balanced  with  attention  for  academic  development  and the  critical  thinking 
skills  of  students.  As  such  the  term  “Bildung”  has  been  reintroduced  into  the 
policy debate. Various stakeholders are trying to move public policy away from 
neoliberal thinking in terms of efficiency, outputs and performance and revert to 
traditional values of educating critical, analytical and self-conscious individuals 
who can act as responsible citizens in a sustainable society. 

Organisational  policy  levers  used  to  promote  personal  development  are  observed  in 
Ontario and in the Netherlands. In Ontario, personal development is connected to students’ 
mental  health.  More  specifically,  the  Ministry-funded  Centre  for  Innovation  on  Campus 
Mental  Health  is  expected  to  promote  and  share  best  practices  in  this  field  among 
stakeholders.  In the Netherlands two recently introduced new types of higher education 
contribute to personal development. Excellence education is to challenge talented students 
beyond the standard curriculum. Short and professional Associate Degrees widen access 
to  higher  education  and  provide  personal  development  opportunities  for  students  that 
might not otherwise have attended higher education. 

Personal development is stimulated by information provision in Ontario, the Netherlands 
and Germany. In Ontario, the engagement survey shows how students are integrated into 
their study programmes, while the Dutch National Student Survey as well as the German 
Student  Survey  of  AG  Hochschulforschung  explore  how  satisfied  students  are  about 
personal development opportunities.  

24 

 
 
 
 
 
 
This discussion on policy levers that promote personal development illustrates that there 
are relatively few policy levers explicitly used towards this end. However, as observed by 
interviewed  stakeholders,  the  personal  development  dimension  is  often  addressed 
implicitly, e.g. in France and Spain. An interpretation could be that governments leave the 
promotion of personal development for a large part to the higher education institutions and 
students  themselves.  However,  when  policies  to  address  personal  development  are 
developed,  they  tend  to  be  connected  to  a  broad  range  of  issues,  such  as  widening 
participation, financial support to underrepresented student groups, (mental) health, and 
the types of education programmes available. 

2.4  Policy levers for sustainable employment 

Higher education has an important role in providing students with the appropriate skills to 
find  and  remain  in  employment.  Sustainable  employment  in  the  literature  is  related  to 
basic skills, foundation skills, transferable skills, technical and vocational skills, 21st century 
skills  and  lifelong  learning.  Table  2.4  shows  the  different  policy  levers  the  reviewed 
countries use to support sustainable employment. 

Table 2.4: Sustainable employment policy levers by type of policy instrument 
Funding 

Organisation 

Regulation 

Policy 
instrument 
type 

KPIs (e.g. graduation 
rates, employer 
satisfaction) determine 
a very small percentage 
of institutional budgets 

Ontario Higher Educ. 
Quality Council: “light 
touch” monitoring 
performances and 
outcomes 

Countries 

Canada 
(Ontario) 

Czech 
Republic 

Denmark 

Strategic mandates to 
address “jobs, economic 
development and 
innovation”. 

Control on enrolments 
for licensed professions 

Obligatory degree level 
expectations and 
learning outcomes in 
terms of employment 

Strict regulation of 
apprenticeship training 
at colleges 

Quality assurance 
overall, and inclusion of 
external stakeholders 

Accreditation standards 
/ accreditation profile 

Curriculum content for 
licenced professions 

Curricula to develop in 
close interaction with 
the labour market. 
Employability an 
important criterion for 
new programs 

Institutional 
accreditation (since 
2013) includes attention 
to (quality &) relevance 
for the labour market 

Cap on enrolment based 
on labour market 
statistics since 2014 

Development contracts 
(can) include labour 
market indicators. 

Performance-based 
funding formula 

Funding to promote 
specific fields of study 

Operational programme 
OP VK (EU Social Fund) 

Funding connected to 
institutional plans 

Taximeter has for a few 
years included a  
completion bonus, 
stimulating HEIs to 
deliver their graduates 
“on time” to the labour 
market 

Information 

Umbrella organisations 
of colleges and 
universities publish 
annual employment 
surveys of graduates 

HEQCO national study 
of performance 
regarding access, value 
to students and value to 
society 

Promotion of particular 
fields of study  

New university founded 
in 2003: “employment 
sensitive” University of 
the Ontario Institute of 
Technology 

Advisory bodies at 
curriculum level (with 
members from world of 
work) in college sector 

Introduction of two 
institutional profiles 
(academic and 
professional) 

Regional guidance 
centres: info on labour 
market situation 

Student choice portal: 
employment rates, 
earnings, graduate 
satisfaction 

25 

 
 
 
 
Policy 
instrument 
type 

Countries 

France 

Germany 

Ireland 

Regulation 

Funding 

Organisation 

Information 

Reduction in number of 
ba/ma titles 

Student entrepreneurs 
supported by Centres 
for Innovation, Transfer 
and Entrepreneurship 
(PEPITE) 

Universities to have 
career orientation office 
(Plan for Success) 

Recognition of other 
skills to stimulate 
flexibility and study 
success 

Information system 
with APB (admission 
post bac) 

Plan for Success: career 
orientation info for 
(prospective) students 
& graduates  

Universities to present 
indicator data on study 
success and 
employment 

National Pact for 
Women in STEM 
professions 

National Pact for 
Women in STEM 
professions 

Eased access for 
persons with vocational 
HE qualifications 

Career centres 

Dual study programmes 
(Berlin) 

Graduate Panel Study of 
DZHW 

KOAB Graduate Study 

Consultancy and 
information for holders 
of higher education 
entrance qualifications 
(Berlin) 

Plan campus: 
excellence also w.r.t. 
employability 

PIA funds: develop 
skills 

Funding formula 
includes employment 
rate 

Five-years contracts 
(COMUE) aimed to 
include employability 

Higher Education Pact 
2020 

Student support system 
BAföG 

Programme 
“Advancement by 
education: Open 
University” 

National Pact for 
Women in STEM 
professions  

Programme for the 
development of 
universities of applied 
sciences (Lower 
Saxony) 

Student Support 

Springboard initiative 

Compact Framework 

Graduate destination 
survey (first 
destination) 

Employers’  insight 
survey 

The HE laws (2007 & 
2013) regulate: 
•  universities to have 

clear missions 
•  reduce number of 
bachelor & master 
titles/programmes 
•  to monitor students & 

graduate careers 

•  quotas professional & 
technical students 
•  focus on internships 
•  accreditation to 

include employability 

Preparing students for 
prof. careers/self-
employment (German 
Framework Act for HE, 
HE Acts of Berlin and 
Lower Saxony) 

Open access for persons 
with vocational HE 
qualifications  

Eased access to HE for 
refugees 

Law dictates functions 
of higher education 
institutions 

Quality assurance: 
institutions to seek 
regular feedback from 
employers, graduates, 
labour market 
representatives  

Course redundancy 
regulations (macro 
efficiency) 

Netherlands 

Law dictates functions 
of higher education 
institutions 

Performance funding 
and performance 
agreements 

Top sector policy 

Study choice 123 

Organisation of 
excellence education  

National student 
satisfaction survey 

Quality assurance 
overall 

Macro-efficiency check 

Recognition exceptional 
achievement 

Quality Assurance 
procedure foresees that 
degree programmes 
add value in terms of 
professional outcomes 
and skills acquired. 

Acredita offers an 
accreditation label for 
good/exceptional 
performance (e.g. for 
employability). 

Funds available through 
adjustments in study 
finance system 

Associate degree 

Centres of Expertise / 
Entrepreneurship 

Employability indicators 
have been used only in 
Catalonia. 

Some regional 
authorities try to 
coordinate the supply of 
programmes (partly as 
an efficiency measure: 
reduce overlap, small 
programmes, etc.; 
partly in light of labour 
market relevance). 

HBO and University 
Monitor 

National campaigns for 
attractiveness study 
fields 

In some regions (e.g. 
Catalonia) surveys 
among graduates are 
used to inform policy 
decision on education 
provision and students’ 
choice. 

Spain 

26 

 
 
 
 
 
 
 
 
Table  2.4  clearly  indicates  that  sustainable  employment  has  a  high  priority  in  the  eight 
case study countries. All countries promote sustainable employment by means of various 
policy levers in the domains of regulation, funding, organisation and information policies. 

The regulations regarding sustainable employment are specifically related to: 

  Enrolment to study places and quotas (Canada, Czech Republic, Denmark, France, 

Germany) 

  Functions of sectors or higher education institutions; learning outcomes relating to 

sustainable employment (Canada, France, Germany, Ireland, Netherlands) 

  Quality assurance and accreditation (Czech Republic, Denmark, France, Ireland, 

Netherlands, Spain). 

Ontario  aims  to  influence  sustainable  employment  by  a  rather  strict  regulation  of 
apprenticeship  training  at  colleges.  All  higher  education  institutions  are  required  to 
formulate  degree  level  expectations  and  learning  outcomes  related  to  personal 
development  and  employability.  Their  strategic  mandates  have  to  address  criteria, 
including jobs, innovation and economic development. There is some control on enrolments 
for specific programmes, which is partially related to labour market needs. 

Cap on enrolment 
(dimensionering) 
in Denmark 

In  2014  Denmark  introduced  a  cap  on  enrolment  (dimensionering)  for  some 
study programmes. The enrolment quotas in general are defined in accreditation 
decisions  for  each  study  programme,  but  based  on  graduate  unemployment 
statistics, the ministry can decide to reduce the intake over three years in specific 
programmes which are characterized by higher than average unemployment in 
the past (seven years out of ten). 

In binary higher education systems the UAS sector generally provides professional training 
while research universities offer general academic programmes. In addition, institutions in 
France, Ireland and the Netherlands are expected to develop more explicit strategic profiles 
targeting specific niches, competences or educational approaches. 

Quality assurance is another type of regulation to influence sustainable employment. Often 
quality evaluations require study programmes or institutions to prove or check the labour 
market  relevance  of  their  education.  In  Spain  and  the  Netherlands,  the  national 
accreditation  agencies  award  a  special  accreditation 
for  good/exceptional 
performance in specific areas, in Spain amongst others for employability. 

label 

Accreditation label 
for exceptional 
performance in 
employability in 
Spain 

Recently, upon the request of the Catalan government, the regional accreditation 
agency has launched a new Acredita procedure assessing programmes in terms 
of  their  efforts  and  results  regarding  employability  and  internationalisation.  If 
both efforts (“enablers”) and results are outstanding, the programmes receives 
a special label (around 10% of the programmes). 

Funding policies are used in all case study countries to promote sustainable employment. 
Common are funding arrangements that address: 

  Employment outcomes (Canada, Czech Republic, Denmark, France, Ireland, 

Netherlands, Spain) 

  Student financial support (Germany, Ireland) 
  Specific initiatives (France, Netherlands) 

Graduation rates in various countries constitute part of regular performance-based funding 
formula,  such  as  in  Ontario,  Denmark,  France  and  the  Netherlands.  The  French  regular 
funding formula for universities includes indicators on retention and graduates to enhance 
study  success  and  future  employment.  The  Danish  Taximeter  funding  system  has  since 

27 

 
 
 
 
 
 
2009 included a completion bonus to stimulate higher education institutions to deliver their 
graduates “on time” to the labour market. 

More  specific  indicators  on  employment  outcomes  are  also  featured  in  some  countries, 
such as the Czech Republic, France and Ontario. The Czech funding formula includes the 
number of employed graduates. However, because the unemployment rate of graduates is 
relatively low and is rather similar across universities and disciplines in the Czech Republic, 
this  policy  lever  does  not  generate  a  strong  incentive.  In  France,  the  large  investment 
schemes  not  only  aim  to  increase  the  quality  of  teaching  but  also  the  employability  of 
graduates.  Currently,  the  inclusion  of  employability  statistics  in  the  five-year  contracts 
between the ministry and individual universities is being discussed. Likewise, the German 
Higher  Education  Pact  2020  –  the  largest  joint  funding  programme  of  the  federal 
government and states – primarily aims to secure a sufficient supply of highly educated 
personnel. At the same time, it supports open access to higher education, which serves 
the social dimension. 

Large scale 
programmes 
(Pacts) to boost 
quality and 
employability in 
Germany 

In Germany, the  Higher Education Pact 2020 is a joint initiative of the federal 
government  and  the  Länder  governments.  The  programme  aims  to  create  1.5 
million additional study places in the period 2007-2023. Jointly the authorities 
provide 38.5 billion Euros (about 52% comes from the federal budget). The first 
stage  of  the  Higher  Education  Pact  2020  focused  on  opening  and  expanding 
higher  education  to  cater  for  the  demand  for  highly  skilled  personnel  and  to 
stimulate the  innovation  capacity  of  Germany  (BMBF,  2007).  It  thus  is  mainly 
directed  towards  sustainable  employment.  The  third  stage  of  the  Higher 
Education Pact focuses on access for persons with a vocational qualification as 
well as to invest 10% of the budget to reduce drop-outs (BMBF, 2014). 

In 2008, the federal government issued a memorandum on the National Pact for 
Women  in  STEM  professions  with  the  aim  to  reduce  the  shortage  of  skilled 
workers  in  Germany  in  the  fields  of  science,  technology,  engineering  and 
mathematics by better utilising the potential of women. A concrete objective is 
to increase the proportion of female first-year students in science and technical 
subjects by 5 percent. Furthermore, the proportion of women among new hires 
in  STEM  professions  has  to  increase  to  a  level  at  least  as  high  as  among  the 
graduates  in  the  respective  subjects.  The  Pact  provided  information,  offered 
internships, and organised “technical boot camps” for women. This initiative can 
be clearly associated with sustainable employment. 

Student support systems are usually connected to the social dimension of higher education 
but may also be connected to sustainable employment. In Germany the student support 
system, inter alia, explicitly aims to assure a highly skilled workforce. Likewise, although 
less explicit, stakeholders in Ireland also argue that the student support system may be 
connected to sustainable employment. 

Organisational  policy  levers  show  quite  a  broad  spectrum  of  initiatives  to  promote 
sustainable employment. We see the organisation instrument reflected in: 

  Establishment of institutions and agencies. For example, the quality assurance 

organisation in Canada and the centres of expertise in the Netherlands. 

  Advisory bodies that connect employers and higher education institutions (Canada 

and the Netherlands) 

  Career guidance centres / advice for students (Denmark, Germany, France) 
  Orientation distinctions between higher education institutions (Czech Republic) 
  Additional or new types of degrees or programmes (Czech Republic, France, 

Germany, Netherlands) 

28 

 
 
 
 
 
  Access to higher education / fields of study / additional programmes for specific 

target groups (Germany, Ireland, Netherlands) 

As  the  overview  shows,  several  organisational  policy  levers  can  be  highlighted.  In 
Germany, higher education was opened for persons with vocational qualifications in 2009. 
A  somewhat  comparable  policy  lever  is  the  introduction  of  the  Associate  Degree  in  the 
Netherlands and the professional bachelor in France. The Dutch associate degree (AD) aims 
to  support  persons  with  a  vocational  secondary  education  qualification  to  enter  higher 
education after some years of professional work (lifelong learning) or who are uncertain 
about whether they want to take a full 4-year professional bachelor degree. In this respect 
the  AD  qualification  is  related  to  students’  personal  development  and  to  sustainable 
employment.  

Renovating 
degrees and titles 
in France 

In  France,  the  2013  law  decreased  the  number  of  titles  for  both  bachelor  and 
master  degrees  to  facilitate  student  choice  and  recognition  by  employers.  The 
previous  number  of  1400  Bachelors,  2200  professional  Bachelors  and  1800 
Masters  with  5900  specialisations  have  been  reduced  to  only  a  few  discipline-
related  degrees  (arts,  humanities,  languages,  management,  economics,  law, 
humanities  and  social  sciences,  science,  technology  and  health)  with  45 
specialisations (previously 322), 173 professional Bachelors and 245 Masters. 

Since  2013,  professional  bachelors  are  stronger  focused  on  transversal  skills 
such  as  ICT,  oral  and  writing  proficiency,  language  proficiency,  teamwork  and 
working methods to promote sustainable employment. 

In  parallel,  the  government  has  introduced  a  new  student  status  for 
entrepreneurs  who  create  start-up  companies.  They  are  called  student 
entrepreneurs  (“étudiant  entrepreneur”)  and  are  allowed  to  develop  an 
entrepreneurial  project  and  are  awarded  the  "student-entrepreneur"  degree 
(D2E).  

Denmark,  France  and  Germany  have  established  career  guidance  centres  to  support 
students in their education choices and labour market orientation. In Denmark, such career 
guidance centres are particularly organised at regional level. This organisational instrument 
is also used for specific target groups, such as women in STEM programmes in Germany 
and  talented  students  in  the  Dutch  excellence  programmes  (honours  programmes  and 
University  Colleges).  In  France,  student  entrepreneurs  can  benefit  from  special  facilities 
established  by  universities,  called  Student  Centres  for  Innovation,  Transfer  and 
Entrepreneurship (PEPITE) of which 29 were established by the government in 2016. 

The Springboard 
initiative in Ireland 

As  part  of  the  Government’s  Jobs  Initiative  and  part  of  the  Action  Plan  for 
Education  2016-2019,  the  Springboard  initiative  was  launched  in  2011.  The 
Springboard  initiative  provides  part-time  places  in  higher  education  to 
unemployed persons who would benefit from re-skilling or up-skilling in areas of 
identified  skills  need.  The  initiative  provides  access  to  higher  education  for 
specific target groups with a focus on sustainable employment. 

Policies addressing information provision to prospective students are also widely used. In 
particular, the connection between funding and employment outcomes makes it necessary 
to collect information that can also serve other purposes. We observe convergence in the 
policy levers connected to information, such as: 

  Data collected through graduate and employer surveys (Canada, Denmark, 

Germany, Ireland, Netherlands, and Spain) 

  Data collected on the study performance of students (Canada) 
  Information to promote certain disciplines (Czech Rep., Germany, Netherlands) 

29 

 
 
 
 
 
 
  Information feeding into student choice information (Denmark, France, Germany, 

Netherlands, Spain).  

Data  on  employment  outcomes  is  not  only  collected  by  government.  The  umbrella 
organisations of colleges and universities in Canada publishes annual employment surveys 
of graduates demonstrating how graduates perform on the labour market. Similar surveys 
are organised in the Czech Republic, Demark, Germany, Ireland, the Netherlands and some 
regions in Spain (e.g. Catalonia). Each country has its own approach to graduate surveys. 
The information collected in most cases is for purposes other than funding. For example, 
in Spain where employment outcomes (e.g. whether graduates found a job appropriate to 
the programme) are used in the evaluation of newly established study programmes. In the 
Czech Republic the government actively promotes particular fields of study in case of labour 
market  shortages.  The  Netherlands  has  initiated  several  information  campaigns  to 
stimulate student participation in disciplines with a strong labour market demand, such as 
teacher  training  and  science  and  engineering.  In  Germany,  as  well  as  several  surveys 
among students and graduates, the National Pact for Women in STEM professions provides 
information for female students on their future career opportunities. 

Employment data may also be used as student choice information. In Denmark, an online 
student choice portal provides information on graduate employment rates, earnings and 
graduate satisfaction. Similar information is presented on the Dutch Student Choice 123 
website. The French government has set up a special information system “Admissions Post 
Bac”  (ABP)  to  inform  prospective  students  about  their  study  choices,  graduation  rates, 
employment  rates  and  career  opportunities.  The  National  Plan  for  Study  Success  urges 
universities  to  establish  career  offices  while  the  government  has  also  established  29 
Student  Centres  for  Innovation,  Transfer  and  Entrepreneurship  (PEPITE)  to  support 
entrepreneurial students. 

In sum, sustainable employment is not only promoted by various policy levers in all case 
study countries, but there also appears to be a variety of levers amongst the four policy 
types. An example of a country utilising a fairly broad range of policy levers for a single 
dimension  is  Denmark.  In  Denmark,  study  programmes  need  to  be  developed  in  close 
interaction  with  the  labour  market,  and  labour  market  relevance  is  an  explicit  and 
important  criterion  for  establishing  new  programmes.  The  institutional  accreditation 
processes  (since  2013)  focus  on  quality  and  relevance  of  programmes  for  the  labour 
market.  In  addition,  a  cap  on  enrolments  that  is  based  on  labour  market  statistics  was 
introduced  in  2014.  Furthermore,  the  development  contracts  with  individual  higher 
education  institutions  may  include  labour  market  indicators.  Finally,  information  and 
indicators help students with study and career choices. 

2.5  Policy levers for active citizenship 

Active citizenship comprises several aspects – widening access for disadvantaged groups, 
promoting and stimulating participation of students in the governance of higher education 
and/or in voluntary work and civic society organisations. Table 2.5 provides an overview 
of the policy levers developed to promote this dimension. 

The Czech Republic and Ontario seem not to have policy levers specifically addressing the 
active citizenship dimension of relevance. The Czech Republic has however a long tradition 
of  student  participation  in  higher  education  governance  and  the  most  recent  higher 
education strategy highlights that one of the missions of higher education institutions is to 
“help […] build a democratic, open, tolerant and cohesive education and cultural society” 
(see Czech country report, Annex 1). Ontario allocates specific funding for students from 
under-represented populations but explicitly links this to their personal development and 
not to their role as active citizens. This obviously does not imply that higher education in 
the  two  systems  discussed  does  not  prepare  students  for  an  active  role  in  democratic 

30 

 
 
 
societies, but rather that in these two systems no policy lever has been developed with an 
explicit aim to promote active citizenship. 

Table 2.5: Active citizenship policy levers by type of policy instrument 

Policy lever 
type 

Countries 

Canada (Ontario) 

Czech Republic 

Denmark 

France 

Germany 

Ireland 

Netherlands 

Spain 

Regulation 

Funding 

Organisation 

Information 

Student loans to 
vulnerable groups to 
increase access, hence 
link to active citizenship 

Flexibility in curricula 
and recognition of civic 
service activities 

Stimulation funds for 
student involvement in 
associative life  

Recognition of student 
experience in student 
associations by 
awarding credits for it 

Student Survey of AG 
Hochschulforschung 

Campus Engage 
initiative 

Irish student 
engagement survey 

Study choice 123 

Student Monitor 

Preparing students for 
responsible behaviour 
in free, democratic 
state (German 
Framework Act for 
Higher Education, 
Higher Education Acts 
of Berlin and Lower 
Saxony) 

Strengthening 
participation of 
students in academic 
self-government 
(Amendment of Higher 
Education Act of Lower 
Saxony, 2015) 

Law dictates functions 
of higher education 
institutions 

Law dictated functions 
of higher education 
institutions 

Students are formally 
involved in the 
university, faculty and 
departmental level 
decision-making, in 
the design and 
accreditation of study 
programmes 

Five countries have specific regulations concerning active citizenship: 

  higher education legislation specifically referring to the role of higher education in 

educating students for active citizenship in Ireland and the Netherlands; 

  strengthening participation of students in higher education governance: Germany 

and Spain; 

  allowing flexibility of curricula so that students can also engage in civic service 

during their studies: France. 

31 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Curriculum 
changes 
supporting active 
citizenship in 
France 

In 2015, a national plan for improving student life on campuses was presented 
(MESR,  2015).  Several  actions  focus  specifically  on  the  active  citizenship 
dimension  of  relevance,  including  (1)  recognition  of  skills  acquired  through 
student  associations  or  elected  student  governance  structures,  (2)  flexibility 
when it comes to workload and examination modalities or (3) allowing for a study 
break for up to one year for personal or professional experiences outside of the 
university environment. 

Funding policy levers used to promote active citizenship include: 

  expanding access to higher education to students of poorer socio-economic 

backgrounds through student loans in Denmark and grants and loans in Germany; 

  allocating funding to (all) universities to stimulate student engagement and to 

recognise student engagement by awarding study credits for it: France. 

The  only  country  that  has  explicit  organisational  policy  levers  concerning  active 
citizenship is Ireland which, through its “Campus Engage” initiative, promotes engagement 
of students not only in regionally relevant economic activities but also in societal activities. 

Campus Engage 
initiative in Ireland 

The ‘Campus Engage’ initiative engages students in civic society activities. This 
project is run by the Irish University Association (IUA) and the Dublin Institute 
of Technology. Through a website students can volunteer for local organisations 
on a pro bono and non-curricular basis. https://www.studentvolunteer.ie/  

With  regards  to  information  policy  levers,  in  Germany  and  the  Netherlands  various 
student surveys focus on how students are integrated in student life and engagement and 
in what ways this benefits them regarding social values. 

Overall, the active citizenship dimension of higher education is primarily supported through 
regulation and funding, while organisation and information policies are rare. 

2.6  Policy effectiveness 

Although  the  relevance  of  higher  education  appears  to  be  an  important  topic  for  policy 
interventions in all eight case study countries and all countries have developed interesting 
policy  initiatives  to  address  particular  relevance  dimensions  or  aspects,  there  is  little 
evidence  on  the  effectiveness  of  the  policy  levers  used.  First,  many  policies  have  been 
initiated only recently and therefore have not been evaluated (yet). Second, policies are 
rarely  closely  monitored  or  evaluated,  with  the  exceptions  of  in  Germany  and  the 
Netherlands. Third, when policies are monitored or  evaluated, the  relevance dimensions 
as defined in this study are not the priority. Fourth, monitoring in most countries is mostly 
focused on sustainable employment indicators rather than indicators connected to personal 
development  and  active  citizenship.  Fifth,  overall  outcome  indicators  (e.g.  employment 
five years after graduation) do not necessarily say something about the effectiveness of 
policy levers connected to sustainable employment, but may be the result of exogenous 
factors. Lastly, data retrieved through indicators may suffer from time lags (i.e. showing 
the “performance” of a higher education system of a decade ago) or may not yet show the 
full (long term) impact of a policy lever (e.g. active citizenship monitored five years after 
graduation, while it may take ten years to see the full impact).  

The  limitations  described  above  can  be  connected  to  the  limited  insights  into  the 
effectiveness  of  policy  levers  in  the  eight  case  study  countries.  However,  there  are 
exceptions to this general observation, illustrated by the following examples.  

32 

 
 
 
 
 
 
 
 
 
 
Good practices of studying policy effectiveness 

Robust monitoring 
of employability 
(Catalonia, Spain) 

Catalonia has a robust monitoring system covering the employability of students 
and  providing  information  to  families  to  influence  their  educational  choices. 
Officials say that this instrument has been very effective.  

Evaluation of the 
Springboard 
Initiative (Ireland) 

In Ireland, a 2015 evaluation of the Springboard Initiative, which provides part-
time places in higher education to unemployed people, reported that a majority 
of Springboard graduates do indeed find employment or become self-employed 
within two years of completing their course (HEA, 2015). 

Most policies are 
evaluated 
(Germany) 

Most policies are 
evaluated (the 
Netherlands) 

In  Germany,  the  majority  of  policies  are  evaluated.  Often  the  evaluations  and 
reports are connected to the relevance of higher education because they often 
assess effects on aspects of personal development or sustainable employment. 
A  good  example  is  the  “National  Pact  for  Women  in  STEM  Professions”.  Its 
evaluation shows that the programme has motivated women to start studying 
and working in a STEM field. 

In the Netherlands it is a legal obligation to evaluate policies within five years. 
As  an  illustration,  two  recent  policy  evaluations  demonstrated  direct effects  in 
relation  to  relevance.  The  evaluation  of  excellence  education  has  shown  that 
students  experience  a  positive  influence  on  personal  development,  active 
citizenship and employability. Associate degrees particularly appear to have an 
impact on graduate employment. Other policy levers have unintended relevance 
effects.  For  example,  performance-based  funding  triggered  a  debate  to 
reintroduce more attention for personal development (Bildung). 

These  examples  illustrate  that  different  types  of  policies  (can)  have  visible  impact. 
However,  international  practice  also  shows  that  many  policies  are  not  evaluated  or 
monitored so their effects are unknown and policy instruments are not able to be optimised. 
Moreover, if evaluations are carried out, then relevance dimensions are not always taken 
into  account  and  often  such  studies  do  not  clearly  show  the  conditions  for  success  or 
reasons for failure. However, based on the stakeholder interviews we gained some generic 
insights on how policy effectiveness studies could be better supported: 

  governments need to carefully consider the appropriateness of policy instruments in 

their specific settings and also to reflect on which instruments fit best with the 
challenges in their systems; 

  governments should consider which instruments would be best organised at the 

system level and which instruments could be best left to higher education 
institutions; 

  policy effectiveness may be improved by the inclusion of all relevant stakeholders in 

the design and implementation processes. 

2.7  Reflections and conclusions 

The eight country case studies on the relevance of higher education have been analysed 
and compared in this chapter. The full case study reports can be found in Annex 1. The 
comparative analysis of the eight case studies exemplifies that countries use a variety of 
policy  levers  to  maintain  and  improve  higher  education  relevance.  This  variety  confirms 
that  relevance  of  higher  education  is  a  multidimensional  concept:  it  can  be  stimulated 
through a large variety of policies.  

As expected on the basis of the initial literature review, the analysis of policies in the eight 
case study countries, enriched by the multiple stakeholder interviews, clearly demonstrate 

33 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
that countries put the strongest emphasis on sustainable employment. This stems firstly 
from  the  sheer  number  of  policy  levers  explicitly  addressing  this  relevance  dimension. 
Secondly, policy documents and policy-makers as well as other stakeholders often indicate 
that  personal  development  and  active  citizenship  are  somehow  implicitly  expected  from 
higher  education.  As  such,  personal  development  and  active  citizenship  are  less 
prominently translated into policy levers. 

The  focus  on  a  particular  dimension  is  however  not  cast  in  stone.  The  priority  given  to 
personal  development,  sustainable  employment  or  active  citizenship,  may  change  over 
time. A country’s  changing economic and social situation may partly explain the priority 
given to the relevance dimensions and the policy levers used at a moment in time. As the 
Irish, Danish and Dutch case studies  show, economic challenges can trigger the priority 
given to sustainable employment, while in more affluent times personal development and 
active citizenship may receive more attention. The attention may also be dynamic in other 
ways. Some stakeholders indicated that – next to the three relevance dimensions defined 
in  this  study  –  also  other  dimensions  can  be  important  such  as  knowledge  creation, 
autonomy, economic and socio-cultural development. 

Among the policy levers applied, regulations and funding instruments – the “hard” policy 
levers  –  appear  to  dominate  policy  initiatives.  In  most  countries,  funding  arrangements 
take into account student and/or graduate numbers, basic funds for teaching programmes, 
and funding of larger strategic plans for the modernisation and quality improvement of the 
higher education system. If an explicit relationship is made between funding policies and 
the  relevance  of  higher  education,  then  in  most  countries  the  strongest  focus  is  on 
sustainable employment. Stakeholders identify funding instruments often as most suitable 
to  pursue  aspects  of  relevance  dimensions  –  they  support  new  initiatives  –  and  regard 
them  as  more  acceptable  than  regulations  which  often  forbid  certain  actions.  However, 
funding instruments can be of a more temporary nature. When funding stops, it is often 
difficult to maintain structures and staff. Policy levers in the domains of organisation and 
information  provision  –  often  called  the  “soft”  policy  levers  –  are  used  less  frequently. 
Nevertheless, information policy levers are used in all case study countries, again mainly 
for the sustainable employment dimension. 

The classification of levers into relevance dimensions or types of instruments is not always 
clear  cut.  For  example,  policy  levers  connected  to  regulations  may  also  include  funding 
elements. The Danish ‘development contracts’ exemplify this. Moreover, quality assurance 
systems  are  often  rather  specific  with  respect  to  requirements  related  to  sustainable 
employment.  Yet,  the  accreditation  may  also  address  personal  development  and  active 
citizenship  through  learning  outcomes  requirements,  particularly  if  these  are  explicit 
objectives of the study programmes or institutions. Student financial support can also be 
mentioned here: in Denmark and Germany this is connect to personal development, while 
in Ireland it is also connected to sustainable employment. 

Though all countries studied apply a large number of policy levers to promote the relevance 
of  higher  education,  structural  monitoring  and  evaluation  of  implementation  and 
effectiveness appears to be scarce. Only limited studies and databases are available on the 
impact on relevance policies and the interviews with various stakeholders in the case study 
countries demonstrated that the knowledge on policy effectiveness in many cases is rather 
generic and not founded in solid analysis. This leaves ample room for governmental action 
to  better  analyse  the  implementation  and  effectiveness  of  recent  and  current  policy 
initiatives. 

34 

 
 
 
 
3  Indicators for higher education relevance 

3.1  Introduction 

This chapter presents the project’s findings with respect to used and usable indicators to 
measure  higher  education  relevance.  The  insights  are  based  on  an  in-depth  review  of 
existing  international  indicators  and  the  country  case  studies.  The  main  issues  and 
questions addressed are: 

1.  How is the relevance of higher education monitored, and what indicators are used 

by policy actors to this end? These questions are discussed in Section 3.2. 

2.  Which  indicators  can  be  found  in  the  international  literature  that  can  be  used  to 
assess  the  relevance  of  the  teaching  and  learning  activities  of  higher  education 
systems to societal needs? This question is discussed in Section 3.3. 

3.  What overall conclusions can be drawn from the used and usable indicators? This 

question is addressed in Section 3.3. 

3.2  Indicators and monitoring of relevance in case study countries 

As part of the country case studies we asked the interviewed policy actors to reflect on the 
used indicators for policy making, monitoring and public debates about higher education 
relevance. The overview is presented in Table 3.1. 

Throughout  the  different  case  studies,  policy  actors  stress  that  quantitative  data  should 
always  be  interpreted  carefully,  as  outcomes  very  much  depend  on  local  contexts  and 
conditions  (e.g.  regional  economic  development  and  the  number  of  employers  in  the 
region). In fact, policy actors in the Netherlands suggest that for a better interpretation of 
outcomes, quantitative data should be supplemented with qualitative insights. This is one 
of the lessons learned from the Dutch performance funding policy lever.  

Table  3.1  highlights  the  strong  focus  of  indicators  on  the  sustainable  employment 
dimension. Every case study country monitors graduates’ transition to the labour market. 
Yet, some countries do this more extensively than others. For example, Germany monitors 
the  transition,  the  adequacy  of  employment,  assessment  of  studies  regarding  labour 
market relevance and the professional development of graduates for up to ten years after 
graduation. In contrast, Ireland has a more narrow focus on the short-term transition of 
graduates to the labour market. 

The focus on sustainable employment indicators comes as no surprise. First, sustainable 
employment  indicators  are  often  associated  with  funding  schemes,  thus  requiring 
measurable indicators. Second, institutions may be hesitant to measure aspects related to 
personal development and active citizenship (or also more in depth aspects of sustainable 
employment) out of concern of misinterpretation and misuse, as outcomes may be used 
to hold institutions accountable. Third, sustainable employment aspects appear to be more 
easily  measured  than  aspects  related  to  personal  development  and  active  citizenship. 
Several policy actors mention in this respect that personal development over time is very 
visible in students, yet hard to measure quantitatively. According to the same policy actors, 
there could be proxy indicators that may say something about personal development and 
active  citizenship,  such  as  mobility  (incoming  and  outgoing  students)  as  mentioned  by 
German  and  Dutch  experts.  Active  citizenship  is  partly  measured  in  the  Student 
Engagement  Survey 
the  German  AG 
Hochschulforschung.  The  European  Social  Survey  or  the  European  Values  Survey  do 
provide a number of interesting indicators on personal development and active citizenship, 
such as social participation as mentioned in the Dutch case study. Apparently policy makers 

the  Student  Survey  of 

in  Ireland, 

in 

35 

 
 
 
are not always aware of existing indicators on both of these relevance dimensions, be it in 
national or international data sources. 

Table 3.1: Indicators used relating to higher education relevance dimensions 

Relevance 
Dimensions  

Countries 

Canada 
(Ontario) 

Czech Republic 

Denmark 

France 

Germany 

Ireland 

Netherlands 

Spain 

Personal development 

Sustainable employment 

Active citizenship 

Criteria for degree level 
expectations and learning 
outcomes 

Graduation rates and employer 
satisfaction (as KPIs) determine a very 
small percentage of institutional 
budgets 
Criteria for degree level expectations 
and learning outcomes 

Employment rates (6 months and two 
years after graduation) by field of 
study 

Unemployment shortly after 
graduation / tertiary educated 
population aged 25-29 (also 
considering the regional 
unemployment rate) 

Student careers and transitions to 
labour market 

Evaluate the cooperation with industry 
at their institution 

HE graduates working at ISCO 1-3 
positions 

Bachelor graduates with command of 
at least one foreign language 

Labour market indicators in: 
development contracts, (caps on) 
enrolment decisions, accreditation 
decisions – also for new programmes, 
and in educational choice information. 

Professional status, 18 months and 30 
months after graduation 

mobility (incoming and 
outgoing students) 

Transition of graduates to the labour 
market 

Adequacy of employment of HE 
graduates 

Professional development of 
academics 

Assessments of studies of HE 
graduates 

HE drop-out 

Graduate first destination (six to nine 
months after graduation) 

Student engagement on 
campus and active 
citizenship 

Suitable job in specific amount of time 
after graduation 

Social participation 

Labour market prospects 

Employer satisfaction 

Employability indicators, but only 
recently and not coherently or in a 
comprehensive way. 

Having identified which indicators are used – and in which areas indicators could be useful 
– the next section discusses which existing indicators could potentially be used to measure 
the  three  dimensions  of  higher  education  relevance  based  on  an  international  literature 
study. 

3.3  Review of indicators measuring the relevance of higher education 

This indicator  review  particularly  addresses  the  question  of  how  the  relevance  of  higher 
education  can  be  measured,  visualised,  and  monitored.  The  objective  is  to  provide  an 

36 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
overview of the state of the art regarding monitoring the relevance of higher education in 
Europe and some interesting benchmark countries. To this end, the chapter predominantly 
explores  the  availability  of  existing  international  comparative  indicators  that  grasp  the 
essence of the relevance of higher education in countries. Based on such an overview, one 
not only can identify aspects of  relevance that  are well-covered, but also aspects which 
cannot be measured satisfactorily with available statistics or that are, to the best of our 
knowledge, not addressed with statistical monitoring indicators so far. Such limitations can 
spur the debate on whether new indicators should and could be developed or existing ones 
improved. 

The review is based on extensive research on international data resources. International 
sources already offer a broad bandwidth of indicators and international data providers such 
as  the  OECD  and  Eurostat  make  considerable  efforts  to  collect  this  data  and  to  make 
comparable  data  collected  at  the  national  level.  National-level  data  may  well  offer 
additional indicators not covered by the present review. 

The  review  covers  a  considerable  number  of  data  sources  and  we  have  strived  towards 
retrieving indicators for all dimensions of higher education relevance set out earlier in this 
report.  The  review  identifies  aspects  for  which  no  or  only  few  indicators  were  detected. 
However, despite all efforts some indicators may have been overlooked.  

Tables 3.2 to 3.6 offer overviews of indicators by dimension. For easy access of information 
the  tables  in  this  chapter  do  not  show  all  indicators  that  have  been  identified.  A  few 
exemplary  indicators  are  presented  per  dimension.  Exemplary  indicators  were  chosen 
based on the criteria of (1) content validity, i.e. how well the indicator corresponds to the 
aspect,  (2)  up-to-date  and  regularly  refreshed  data  (periodical),  (3)  the  number  of 
stakeholder groups an indicator is relevant to, (4) the size of the stakeholder groups an 
indicator  is  relevant  to  (student/graduates  and  society  have  been  given  priority  over 
employers), and (5) the number of (European) countries covered. Aspects are shaded in 
green if an indicator is available that captures the content of the aspect well, is regularly 
updated, and covers the majority of EU member countries. If an indicator is available but 
any of the mentioned criteria is violated (low content validity, not regularly available, or 
covering less than half the EU  countries), the  aspect is shaded in orange.  If no suitable 
indicator is available, the aspect is shaded in red. 

3.3.1  Indicators on personal development 
Aspects  of  personal  development,  such  as  self-perception,  a  cooperative  spirit,  or  high 
ethical  standards  are  of  direct  use  to  students  and  graduates,  but  employers  and  the 
society are likely to benefit from personality characteristics of students and graduates as 
well. Table 3.2 shows results for all three stakeholder groups jointly as indicators for the 
stakeholders  overlapped  to  a  large  extent.  Crosses  in  the  table  mark  the  stakeholder 
group(s) an indicator is relevant for.  

For  all  aspects  associated  with  the  dimension  of  personal  development  at  least  one 
indicator  was  found.  Thus  the  overall  availability  of  indicators  seems  relatively  good. 
Quality and suitability of indicators varies between aspects and a number of caveats should 
be noted.  

  Attitudes towards self and perception of self: The indicator used as an example is part 
of the World Values Survey (WVS). It provides a set of self-image attributes, such as 
being reserved, trusting or doing a thorough job. Attributes are rated by respondents 
and jointly capture the aspect well. The WVS is repeated at intervals of four to six years 
and covers less than half of the EU countries which is the reason for categorizing it as 
not fully satisfactory.  

  Trust in others and cooperative spirit: The example indicator is provided by the  ESS 
and  uses  a  standard  instrument  for  measuring  trust  in  others.  The  ESS  offers 

37 

 
 
complementary indicators; jointly, these indicators give a good account of the aspect 
for students and graduates as well as the society.  

  Values  (ethical  standards),  responsible  employees:  This  WVS  indicator  measures  to 
what  extent  respondents  assess  actions  as  justifiable  that  are  unlawful  or  challenge 
social  norms  (e.g.  bribery,  free-riding  on  public  transport).  Content  validity  seems 
satisfactory, but the number of countries is not. 

  Motivation, motivated employees: The ESS provides an indicator on the overall intrinsic 
motivation for their job of economically active persons with further items on the job-
related  motivation  being  available.  The  indicators  jointly  capture  the  aspect  of 
motivation  of  students,  graduates,  and  employers.  However,  the  indicator  is  not 
updated regularly.  

  Health, healthy employees: The indicator is a self-assessment on the general level of 
health of the respondent on a 5-point scale. This is a relatively gross measure that may 
be prone to measurement error but it is very clearly related to the aspect of health and 
thus  seems  satisfactory.  It  could  be  complemented  by  additional  indicators  of  the 
survey programme of the EU-SILC. 

  Happiness:  The  ESS  offers  an  indicator  measuring  the  level  of  happiness  on  an  11-
point  scale  covering  the  aspect  well.  Some  complementary  indicators  would  be 
available in the ESS. 

  Students’  and  graduates’  satisfaction  with  personal  development:  EUROSTUDENT  IV 
asked  students  whether  they  assess  their  study  programme  as  a  good  basis  for 
personal development.  This indicator covers the aspect well but was not repeated in 
subsequent rounds of EUROSTUDENT. 

All  in  all,  interesting  indicators  were  found  for  all  considered  aspects  of  personal 
development. However, only for three aspects were indicators identified that fulfilled the 
criteria of good content validity, regular updates and a good coverage of EU countries. 

38 

 
 
 
 
 
 
Table 3.2: Examples of indicators on personal development 

Aspect 

Example of indicator 

Source 

Attitudes towards self and 
perception of self (sense 
of identity, self-respect, 
self-worth, self-efficacy) 

Self-perception by highest 
educational attainment level  

World Values Survey 
2016, http://www.worldvaluessurv
ey.org/WVSOnline.jsp 

Trust in others, 
cooperative spirit, 
cooperative spirit in 
company, good 
teamwork, & level of trust 
in society 
Values (ethical 
standards), responsible 
employees 

Extent of trust in other people 
by highest level of education 

European Social 
Survey http://www.europeansocial
survey.org/data/  

Justifiability of different 
actions by highest educational 
attainment level  

World Values Survey 
2016, http://www.worldvaluessurv
ey.org/WVSOnline.jsp  

Motivation, motivated 
employees  

Health, well-being, 
healthy employees, level 
of health 

“I would enjoy working in my 
current job even if I did not 
need the money” by highest 
level of education. 
Self-perceived health by sex, 
age and educational 
attainment (%) 

Happiness, level of 
happiness 

Level of happiness by highest 
level of education 

Students’/graduates’ 
satisfaction with personal 
development & related 
aspects in HE  

Students’ assessment of study 
programme as good basis for 
personal development 

European Social 
Survey http://www.europeansocial
survey.org/data/  

European Commission, Eurostat 
2016, EU-
SILC, http://ec.europa.eu/eurostat
/data/database 
European Social 
Survey http://www.europeansocial
survey.org/data/  
Eurostudent 
(2011), https://eurostudent.his.de
/eiv/report/data_overview.jsp?ssid
=D2E755EAAD04E01F512CDFD7A
65D4030&sel_lang=&cnt_oid=1  

Students/ 
graduates 
X 

Stakeholders 
Employers 

Society 

Periodical 

Yes 

Quality criteria 
Comparable 

Yes, across 
countries 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Yes 

Yes, across 
countries 

Yes 

Yes, across 
countries 

No (only data for 
5th wave, 2010) 

Yes, across 
countries 

X 

X 

Yes 

Yes 

No (only for 
period 2008-
2011 available) 

Yes, across 
countries 

Yes, across 
countries 

Yes, across 
countries  

Geographical 
coverage 
Generally up to 
61 countries 
worldwide for 
the wave 2010-
2014 (including 
11 European 
countries)  
21 mostly 
European 
countries 

Generally up to 
61 countries 
worldwide for 
the wave 2010-
2014 (including 
11 European 
countries)  
27 mostly 
European 
countries 

31 European 
countries and 
two EU 
aggregates 
21 mostly 
European 
countries 
20 EHEA 
countries 

39 

 
 
 
 
 
 
 
 
 
 
 
 
 
3.3.2  Indicators on sustainable employment 
Indicators were more group-specific for this dimension and are thus presented separately 
for each group. 

Table 3.3 shows the indicators for the stakeholder group students and graduates.  

  Qualifications: A large variety of indicators on qualifications of graduates are available 
in the Eurostat database. The indicator in table 3.3 shows the distribution of graduates 
of a chosen level of education across fields of study. Content validity seems satisfactory, 
indicators are updated regularly, and cover >30 OECD countries, mostly including all 
EU28 countries. 

  Basic  skills:  OECD’s  survey  of  adult  skills  provides  an  indicator  on  literacy  by 
educational attainment. Complementary information is available on basic skills such as 
numeracy or use of ICT by educational attainment. Jointly these indicators give a good 
account  of  the  basic  skills  and  related  differences  in  educational  groups;  however 
indicators will presumably not be updated more frequently than every ten years. 

  Transferable skills: The proficiency in problem solving in technology-rich environments 
is defined as “using digital technology, communication tools and networks to acquire 
and  evaluate  information,  communicate  with  others  and  perform  practical  tasks” 
(OECD, 2013). It pictures a transferable skill needed for a large variety of occupations, 
however for other transferable skills, e.g. creativity or critical thinking, no indicator has 
been found in the international data sources considered.  

  Knowledge,  technical  &  professional  skills,  competencies:  No  indicators  on  these 
aspects could be identified in the international data sources considered, even though 
the  borders  between  basic  &  transferable  skills  and  knowledge,  technical  and 
professional  skills,  and  competencies  may  not  always  be  distinct.  Projects,  such  as 
CALOHEE, strive for progress in measuring skills. 

  Cultural capital: Eurostat offers an indicator on the cultural activities of persons (e.g. 
visiting cinema, theatre, concerts, cultural sites etc.). The indicator captures the aspect 
quite well, even though it does not measure cultural capital directly, but via activities 
that build on or may lead to cultural capital. 

  Social  capital:  The  indicator  shown  in  Table  3.3  measures  the  frequency  of  socially 
meeting with friends, relatives, or colleagues. International data sources offer several 
indicators  related  to  social  capital  but  none  of  these  indicators  provides  a 
comprehensive  picture  of  the  size  and  quality  of  the  social  network  of  students  and 
graduates.  

  Social  prestige  associated  with  the  education  or  the  occupation  of  graduates: 
Sociological research has developed a number of scales measuring the social prestige 
of occupations (e.g. Ganzeboom et al., 1992), but none of them are available in the 
international data sources considered. 

  Successful transition to the labour market: The Bologna Process Implementation Report 
of  2015  offers  an  indicator  on  involuntary  unemployment  of  tertiary  education 
graduates by the number of years since graduation. While this certainly provides an 
interesting  piece  of  information  it  does  not  give  a  very  fine-grained  picture  of  the 
transition process and misses the counter-part of unemployment, i.e. employment. The 
EU-LFS ad-hoc module of 2009 provides several complementary indicators; however, 
these indicators are not available regularly. 

  Adequacy of employment: Available indicators on adequacy of employment cover the 
aspect  of  over-qualification.  This  measure  captures  vertical  mismatch  well,  though 
somewhat roughly. Measures of horizontal mismatch are, to the best of our knowledge, 

40 

 
 
 
not regularly available in international data sources. Thus an important dimension of 
this aspect is not covered with current data. 

  Earnings:  Education  at  a  glance  gives  a  good  account  of  the  relative  earnings  of 
graduates of different educational levels. The indicator is updated, but not regularly. 
The  OECD  regularly  provides  a  measurement  of  the  private  net  financial  returns  of 
higher education as compared to other levels of education. 

  Career  opportunities:  The  ESS  provides  a  subjective  assessment  of  the  career 
opportunities  offered  by  the  current  job  using  a  5-point  scale.  While  the  indicator  is 
clearly  related  to  the  aspect  in  question  it  seems  questionable  whether  a  subjective 
assessment alone is a very reliable indicator. Preferably, career opportunities could be 
measured directly by the employment history of graduates.  

  Long-term job security: The indicator is provided by the ESS and measures job security 
by the status of the employment contract. A permanent contract is obviously closely 
related to job security. For capturing long-term job security it is limited, however, e.g. 
it  pictures  the  prospects  to  remain  in  employment  after  (voluntary)  job  changes  or 
whether  persons  without  a  permanent  contract  attain  a  permanent  contract  later  in 
their career 

 

Job satisfaction: The ESS offers one indicator measuring job satisfaction on an 11-point 
scale. This is an overall measure capturing the aspect well, with some complementary 
indicators on areas of satisfaction or dissatisfaction being available  

  Student/graduate satisfaction with sustainable employment, & related aspects in HE: 
To the best of our knowledge, no indicator on this aspect is available in international 
data sources. 

All  in  all,  only  for  few  aspects  were  indicators  found  fulfilling  all  three  criteria  of  good 
content validity, regular updates, and good coverage of EU countries. Lacks of information 
are specifically visible for the realm of skills and competencies. For labour market-related 
aspects short-comings are mostly related to content validity, illustrating the limitations of 
the – necessarily – concise scope of the EU-LFS. 

Table 3.4 shows the indicators for the stakeholder group employers. 

  Employees  able  to  flexibly  adapt  to  changing  tasks:  Table  3.4  shows  an  indicator 
measuring  the  proficiency  in  problem  solving  in  technology-rich  environments.  This 
indicator  has  already  been  listed  above  as  a  measurement  for  transversal  skills. 
Problem solving in technology-rich environments is clearly related to being flexible with 
regard to different tasks. At the same time it can hardly be seen as a comprehensive 
measure of this aspect and therefore has no satisfactory content validity.  

  Creative  employees  and  low  turnover  in  staff:  For  these  aspects  no  good  indicators 

were found in the international data sources considered. 

  Diversity of and good mix of qualifications, skills, and competencies: The distribution 
of  graduates  across  education  levels,  programme  orientation,  and  field  of  education 
informs on the provision of skilled labour and has already been used as an example for 
the  aspect  of  “qualifications”.  The  indicator  does  not  however  give  information  on 
whether these qualifications are “good” in the sense that they meet the needs of the 
employers. 

  Sufficient supply of highly educated employees: The indicator provides information on 
the labour force, differentiating by sex, age, and education. It gives an overall picture 
of  available  formal  qualifications  in  an  economy  but  is  not  suited  to  assess  the 
sufficiency of supply 

  Employers’ satisfaction with skills & competencies: The European Commission’s Flash 
Eurobarometer  provided  insights  into  the  employers’  satisfaction  with  HE-graduate 

41 

 
 
skills in 2010. The indicator seems to measure the aspect quite well but the survey is 
not repeated regularly. 

  Employees’ productivity: The International Labour Organisation publishes regularly Key 
Indicators  of  the  Labour  Market  (KILM).  It  is  not  known  whether  the  KILM  report 
provides this analysis regularly. 

The choice of indicators on sustainable employment specifically relevant for employers is 
not satisfactory. Surveys among employers are rarely done. Information on the suitability 
of graduates’ qualifications is therefore typically derived from data on the labour market 
success. Subjective assessments from the point of view of employers would be a valuable 
complementary source giving a more direct account. 

Table 3.5 shows the indicators for the stakeholder group society. 

  Economic  productivity:  The  indicator  for  the  aspect  of  economic  productivity  has 
already  been  listed  for  employees’  productivity  (Table  3.4).  It  is  not  known  whether 
the KILM report provides this analysis regularly. 

  Extent  of  employment  and  unemployment:  The  indicator  chosen  measures  the 
unemployment  rate,  which  is  an  important  standard  indicator  for  assessing  the 
condition  of  a  labour  market  and  the  economic  performance  of  an  economy.  The 
Eurostat indicator allows differentiating by age groups, sex, and four different groups 
of educational attainment levels (based on ISCED 2011). Eurostat offers a wide range 
of further labour market indicators.  

 

 

Impact of higher education on public revenues and social security systems: The public 
provision of HE-teaching induces costs for the state, but may also contribute to public 
revenues  (e.g.  taxes).  Income,  employment,  and  unemployment  also  impact  on  the 
revenues and expenses of social security systems. The indicators provided by the OECD 
estimate the state’s net financial returns of a man/woman attaining tertiary education 
in comparison with a man/woman attaining upper secondary or post-secondary non-
tertiary education.  

Impact of sustainable employment on families, friends, associations, neighbourhoods: 
No suitable indicator has been found in international data sources. 

  Highly educated workforce: The database of the World Bank provides an indicator on 
the quantity of the highly educated workforce. The indicator measures the percentage 
of  persons  with  tertiary  education  as  part  of  the  total  labour  force.  The  indicator  is 
closely  related  to  the  aspect,  but  the  measurement  is  quite  rough.  The  statistics 
available do not differentiate levels of education within the group of tertiary education 
or different kinds of degrees.  

 

Innovation capacity of economy: Eurostat offers a number of statistics related to the 
innovation  capacities  of  economies.  The  indicator  in  Table  3.5  pictures  the  share  of 
research and development personnel as percentage of the total labour force or of all 
employed  persons.  Complementary  indicators  are  e.g.  the  number  of  patent 
applications or patents granted or innovation activities and expenditures in enterprises. 
Indicators  on  the  innovation  capacities  seem  to  be  available  in  good  quality  and 
quantity but it should be noted that current indicators do not directly offer insights into 
the relevance of higher education for innovation capacities. 

  Knowledge-based  economy:  The  indicator  shows  the  share  of  knowledge-intense 
industries and services as proportion of the total creation of value. The indicator gives 
a  direct  measurement  of  the  degree  to  which  the  economy  is  knowledge-based  but 
covers only a small number of countries. It is not known if similar statistics could easily 
be made available for a larger set of countries. 

42 

 
 
 
 

Impact  of  higher  education  on  growth:  Despite  there  being  a  vast  literature  on  the 
correlation of higher education and economic growth, international databanks do not 
provide  standard  indicators  on  this  aspect.  However,  this  is  not  surprising  as 
quantifying the impact of changes in a highly educated workforce on economic output 
requires  rather complex economic modelling. The indicator presented in Table 3.5 is 
insightful  and  seems  to  measure  the  aspect  quite  well.  It is  not  known  whether  this 
study is repeated periodically; less than half of the EU28 countries are covered.  

  External  effects  of  skills  and  competencies  on  families,  friends,  associations, 
neighbourhoods:  It  is  difficult  to  determine  the  types,  causes,  and  scales  of  such 
effects. International statistics do not supply any indicators on this aspect. The indicator 
in Table 3.5 describes positive effects that higher education graduates from developing 
countries have on their home communities after their return from participating in higher 
education abroad and can be viewed as a tentative approach to capture external effects, 
but is most likely not suited for industrial countries. 

Compared to the stakeholder group of employers, the availability of suitable indictors on 
the societal level is considerably better but still not satisfactory for most aspects. Available 
indicators for a highly educated workforce seem to be easily improved for a better content 
validity. For the aspect  of a knowledge-based society a suitable indicator is at hand but 
only for eight countries. It seems likely that equivalent information is available for further 
countries.  

43 

 
 
 
 
Table 3.3: Indicators on sustainable employment (user group: students and graduates) 

Example of indicator 

Source 

Distribution of graduates at education level and 
programme orientation by sex and field of 
education 

Mean literacy proficiency by level of educational 
attainment 

Proficiency in problem solving in technology-rich 
environments, by educational attainment 

European Commission, Eurostat 
2016, http://ec.europa.eu/eurostat/data/database  

Periodical 
Yes 

OECD 2013a, Table A3.9 (L), p. 285.  

presumably 

OECD 2013a, Table A3.10 (P), p. 287. 

presumably 

Persons participating in cultural activities in the last 
12 months by sex and educational attainment level 
Frequency of socially meeting with friends, relatives 
or colleagues by highest level of education  

European Commission, Eurostat 
2016, http://ec.europa.eu/eurostat/data/database 
European Social 
Survey http://www.europeansocialsurvey.org/data/  

Unemployment ratio of tertiary education 
graduates aged 20-34, by the numbers of years 
since graduation 
Percentage of people aged 25-34 with tertiary 
education (ISCED 5-6) who are vertically 
mismatched (ISCO 4-9) 
Private costs & benefits of men & women attaining 
tertiary education 
Relative earnings of workers, by educational 
attainment, age group, and gender 
“Current job offers good opportunities for 
advancement” by highest level of education 

Employment contract with limited or unlimited 
duration by highest level of education. 
Satisfaction with main job by highest level of 
education  
[see social prestige associated with degree etc.] 

EC/EACEA/Eurydice 2015, p. 188. 

EC/EACEA/Eurydice 2015, p. 193. 

OECD 2015a, Table A7.3a, p. 147, Table A7.3b, p. 
148.  
OECD 2015a, Table 6.1a, p. 125. 

European Social 
Survey http://www.europeansocialsurvey.org/data/  

European Social 
Survey http://www.europeansocialsurvey.org/data/  
European Social 
Survey http://www.europeansocialsurvey.org/data/  

Yes 

Yes 

Yes 

Yes 

Yes 

Quality criteria 

Comparable 
Yes, across 
countries  

Geographical coverage 
34 OECD countries, including 32 
European countries and 26 
countries of EU28 

Yes, across 
countries  

Yes, across 
countries  

Yes, across 
countries  
Yes, across 
countries 

20 OECD countries, 4 OECD sub-
national entities, 1 partner 
country 
17 OECD countries and 4 sub-
national entities 

22 European countries  

21 mostly European countries 

Yes 

34 EHEA countries 

Yes, across 
countries 

37 EHEA countries 

Yes 

Both tables: 25 OECD countries  

Basically yes, but not 
annually 
No (only data for 2nd 
and 5th wave, 2004 + 
2010) 
Yes 

No (only data for 5th 
wave, 2010) 

Yes, across 
countries 
Yes, across 
countries 

Yes, across 
countries 
Yes, across 
countries 

32 OECD countries, 2 partner 
countries 
25 European countries (2nd 
wave), 27 mostly European 
countries (5th wave) 
21 mostly European countries 
(7th round) 
27 mostly European countries 

Aspect 

Qualifications 

Knowledge  
Basic skills 

Transferable skills (communication 
skills, creativity, critical thinking etc.) 
Technical & professional skills 
Competencies 
Cultural capital (cultural literacy, 
aesthetic appreciation, etc.) 
Social capital 

Social prestige (associated with 
degree, knowledge, etc.) 
Successful transition to labour 
market  

Adequacy of employment 

Earnings 

Earnings  

Career opportunities 

Long-term job security 

Job satisfaction  

Social prestige (associated with job) 
Student/graduate satisfaction with 
sustainable employment, & related 
aspects in HE 

44 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Table 3.4: Indicators on sustainable employment (user group: employers) 

Aspect 

Example of indicator 

Source 

Employees able to flexibly adapt to 
changing tasks and challenges 

Distribution of skills and readiness to use 
information and communication technologies for 
problem solving, by educational attainment 

Creative employees 
Low turnover in staff  
Sufficient supply of highly educated 
employees 

Labour force by sex, age and education (in 1,000) 

Diversity of and good mix of 
qualifications, skills, competencies 
needed 
Employers’ satisfaction with skills & 
competences 

Employees’ productivity  

Distribution of graduates at education level and 
programme orientation by sex and field of 
education 
Employers’ satisfaction with graduate skills (“HE-
graduates recruited in the last 3-5 years have the 
skills required to work in respondents’ companies”)  
Tertiary level of educational attainment and labour 
productivity 

Education at a Glance 2016: OECD 
Indicators, Table A1.6 (P) (Web 
only), http://dx.doi.org/10.1787/8889333
96517  

Periodical 
Presumably only at 
long intervals of ~10 
years 

Quality criteria 

Comparable 
Yes, across 
countries  

Geographical coverage 
23 OECD countries and 3 sub-
national entities 

International Labour Organization ILOSTAT 
Database 
(2016) http://www.ilo.org/ilostat/faces/h
elp_home/data_by_subject?_adf.ctrl-
state=ujhckfms7_1204&_afrLoop=140158
83796839  
European Commission, Eurostat 
2016, http://ec.europa.eu/eurostat/data/
database  
European Commission 2010, p. 7.  

Yes 

Yes 

No 

ILO Key indicators of Labour Market, 9th 
edition, Figure 3.3, p. 14. 

Unknown  

Yes, across 
countries 

Data for 187 Member states 
worldwide, additional data for sub-
national entities  

Yes, across 
countries  

Yes, across 
countries  

Yes, across 
countries  

34 OECD countries, including 32 
European countries and 26 
countries of EU28 
31 European countries 

74 countries 

45 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Table 3.5: Indicators on sustainable employment; user group: society 

Example of indicator 

Source 

Quality criteria 

Stakeholders: Society 
Aspects 

Economic productivity 

Extent of employment/ unemployment 

Impact of HE on public revenues 

Impact of HE on social security system 

Impact of sustainable employment on 
families, friends, associations, 
neighbourhoods 
Highly educated workforce 

Innovation capacity of economy 

Knowledge-based economy 

Impact of HE on growth 

Tertiary level of educational attainment and labour 
productivity.  
Employment rates by sex, age and educational 
attainment level (% of respective educational level) 

Public costs & benefits of men & women attaining 
tertiary education 
Public costs & benefits of men & women attaining 
tertiary education 

ILO Key indicators of Labour Market, 9th 
edition, Figure 3.3, p. 14. 
European Commission, Eurostat 
2016, http://ec.europa.eu/eurostat/data/d
atabase  
OECD 2015a, Table A7.4a, p. 149, Table 
A7.4b, p. 150.  
OECD 2015a, Table A7.4a, p. 149, Table 
A7.4b, p. 150.  

Labour force with tertiary education (% of total) 

Total R&D personnel and researchers by sectors of 
performance, as % of total labour force and total 
employment, and by sex 
Proportion of R&D-intensive industries and 
knowledge-intensive services in total creation of 
value 
Contributions of higher, medium and low skills 
growth to output growth, in % (1982-2005)   

World Bank 2016, World Development 
Indicators, http://databank.worldbank.org/
data/reports.aspx?source=2&Topic=4   
European Commission, Eurostat 
2016, http://ec.europa.eu/eurostat/data/d
atabase 
Expertenkommission Forschung und 
Innovation 2016, p. 127 

Holland et al., 2013, p. 46 f. 

Unknown (probably 
not) 

Periodical 
Unknown  

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Geographical coverage 
37 countries 

33 European countries and various 
Euro area and European Union 
aggregates   
Both tables: 25 OECD countries  

Both tables: 25 OECD countries  

39 countries worldwide 

35 European countries, EU28, 
EU15, and Euro area aggregates 

8 OECD countries shown in 
publication; more countries should 
be available 
15 countries (mostly European) 

22 countries covering Africa, 
Middle East, Asia, Russia, Latin 
America 

Comparable 
Yes, across 
countries  
Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Limited (Skill 
levels applied by 
EU KLEMS are not 
based on an 
international 
classification. 
Comparability 
across countries 
cannot be 
assured.) 
Difficult to judge 
(not much 
information on 
study 
methodology 
available) 

External effects of skills and 
competencies on families, friends, 
associations, neighbourhoods 

Provision of community benefits through HE-
graduates 

Martel & Bhandari, 2016, p. 
16. http://www.iie.org/Who-We-
Are/News-and-Events/Press-Center/Press-
Releases/2016/2016-04-26-IFP-Tracking-
Study-1 

Not yet (first impact 
study of an 
International 
Fellowships Program) 

46 

 
 
 
 
 
 
 
 
 
 
 
3.3.3  Indicators on active citizenship 
Higher education institutions are expected to prepare students for a life as active citizens 
in democratic societies. This could result in an understanding of how democracies work, 
support of democratic  values, tolerance, and intercultural skills. The democratic ideal of 
equitable  participation  is  also  reflected  in  the  social  dimension  of  higher  education. 
Indicators are presented in Table 3.6. 

  Democratic values: The WVS offers a number of items measuring the overall support 
for democracy and democratic institutions as well as knowledge about key elements of 
democracy.  The  WVS  covers  a  large  number  of  countries,  but  only  11  European 
countries.  

  Tolerance,  intercultural  skills  and  values:  The  ESS  asks  respondents  for  the 
consequences of immigration e.g. regarding the economy or the cultural life. Further 
indicators measure the attitudes of respondents towards the immigration of different 
social groups. These indicators are clearly related to the aspect of tolerance with regard 
to immigrants, but can hardly be seen as an indicator of intercultural skills.  

  Political literacy, civic skills, and (sense of) ability to take influence: Several questions 
related  self-efficacy  in  politics  are  part  of  the  core  questionnaire  of  the  ESS  and 
measure, e.g. the belief to have a say in politics, whether politicians care about what 
people  think,  or  whether  the  respondent  –  him  or  herself  -  finds  it  easy  to  become 
politically  active  or  participate  in  political  groups.  These  items  cover  the  aspect 
satisfactorily.  

  Social  participation,  strong  civil  societies,  and  lively  communities:  The  ESS  regularly 
offers four indicators measuring whether respondents participated in different kinds of 
social  associations  in  the  last  12  months  (e.g.  environment,  peace,  animal 
organisations), capturing the aspect well.  

  Political participation, strong and lively democracies: The ESS regularly offers a set of 
indicators measuring whether respondents were politically active in the last 12 months 
in a variety of ways (e.g. contacted politicians, worked in political action group, etc.), 
giving a comprehensive account of the aspect of political participation.  

  Lower risk of becoming criminal or becoming a victim of crime, less crime: The WVS 
provides several indicators related to this aspect, measuring whether respondents or 
their  immediate  family  had  become  victims  of  crime,  the  perceived  safety  of  the 
neighbourhood or the attitudes towards criminal acts. Jointly these indicators capture 
the aspect well but only a minority of EU28 countries is covered.  

  Employee  participation:  Membership  in  trade  unions  is  related  to  employee 

participation, but does not give a comprehensive account of the aspect. 

  Corporate  social  responsibility,  external  effects,  student/graduate  satisfaction  with 
active citizenship: To the best of our knowledge no valid indicators are available in the 
international data sources considered. 

  Social dimension of higher education: EUROSTUDENT regularly provides an indicator of 
the  social  dimension  of  higher  education,  measuring  to  what  extent  the  student 
population in European countries reflects the socioeconomic composition of the overall 
population. EUROSTUDENT regularly covers gender as another dimension as well. To 
date, there is no source regularly providing comparable information for a larger number 
of countries for equal participation of persons with an immigrant background. 

Looking at the availability of indicators on active citizenship we find a mixed picture. For 
political literacy and social as well as political participation suitable indicators are regularly 
provided by the ESS. The WVS is a richer source of data in this regard and offers indicators 
on two further aspects  but only for a smaller number  of European countries. Monitoring 

47 

 
 
information  on  the  social  dimension  of  higher  education  is  regularly  provided  by 
EUROSTUDENT.  Complementary  indicators  on  graduates  can  to  some  extent  be  drawn 
from ad-hoc modules of EU-LFS and EU-SILC. It should, however, be noted that such ad-
hoc modules are not regularly available. No regular, comparable information is available 
on  the  social  dimension  with  respect  to  an  immigration  background.  The  aspects  of 
employee participation and corporate social responsibility are specifically relevant to the 
employers as a stakeholder group. Again a lack of suitable data needs to be acknowledged 
for  this  group.  Last  but  not  least,  regular,  comparable  indicators  on  students’  and 
graduates’ satisfaction with being prepared for active citizenship in higher education are 
not available to our knowledge. This reflects the fact that the vast majority of indicators 
are  provided  by  general  or  working  age  population  surveys  and  that  there  is  a  lack  of 
surveys focussing on the realm of higher education. 

48 

 
 
 
 
 
Table 3.6: Indicators on active citizenship 

Aspects 

Example of indicator 

Source 

Democratic values 

Appreciation of democracy by highest 
educational attainment level  

World Values Survey 
2016, http://www.worldvaluessurvey.org/
WVSOnline.jsp  

Students/ 
graduates 
X 

Tolerance, intercultural skills & 
values 

Political literacy, civic skills and 
(sense of) ability to take 
influence 
Social participation & inclusion 
(e.g. participation in 
associations, unions, or social & 
community projects, 
volunteering), strong civil 
societies, lively communities 
Political participation  (e.g. 
campaigning, protesting, voting, 
running for political functions), 
strong & lively democracies 
Lower risk of becoming criminal 
and/or becoming a victim of 
crime, less crime 

Employee participation 

Corporate Social Responsibility 
External effects of democratic 
values, civic skills etc. on 
families, friends, associations, 
neighbourhoods 
Social dimension of HE: 
socioeconomic background 

Social dimension of HE: 
immigrant background 

Student/graduate satisfaction 
with active citizenship & related 
aspects in HE   

Attitudes towards the consequences 
of immigration for respondent’s 
country 
Confidence in own ability to 
participate in politics by highest level 
of education 
Having worked voluntarily for 
environment/peace/animal 
organisations during the last 12 
months by highest level of education 

European Social 
Survey http://www.europeansocialsurvey.
org/data/  
European Social 
Survey http://www.europeansocialsurvey.
org/data/ 
European Social 
Survey http://www.europeansocialsurvey.
org/data/  

Participation in various political 
actions by highest level of education   

European Social 
Survey http://www.europeansocialsurvey.
org/data/  

Justifiability of criminal/illegal 
activities by highest educational 
attainment level  

World Values Survey 
2016, http://www.worldvaluessurvey.org/
WVSOnline.jsp  

Membership in trade union or similar 
organisation by highest educational 
attainment level 

European Social 
Survey http://www.europeansocialsurvey.
org/data/  

Representation of students from high, 
medium and low educational 
backgrounds 
Relative chances to obtain higher 
education for second-generation 
immigrants 

EUROSTUDENT 2015, Figure 3.3, p. 54 

Muehleck 2013, p. 30 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Stakeholders 
Employers 

Society 

Periodical 

Quality criteria 
Comparable 

Geographical coverage 

X 

Yes 

X 

X 

X 

Yes 

Yes 

Yes 

X 

Yes 

X 

Yes 

Yes 

X 

X 

Yes, across 
countries 

Yes, across 
countries 

Yes, across 
countries 

Yes, across 
countries 

Generally up to 61 countries 
worldwide for the wave 2010-
2014 (including 11 European 
countries)  
21 mostly European countries 
(7th round) 

See above 

See above 

Yes, across 
countries 

See above 

Yes, across 
countries 

Yes, across 
countries 

Generally up to 61 countries 
worldwide for the wave 2010-
2014 (including 11 European 
countries) 
See above 

X 

X 

Yes (operationalisation of 
indicator changes to some 
extent between repetitions) 
No 

Yes, across 
countries  

Yes, across 
countries  

23 EHEA countries 

12 EU countries 

49 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3.4  Reflections and conclusions 

A  first  conclusion  is  that  suitable  indicators  were  identified  for  all  dimensions  of  higher 
education relevance and all three stakeholder  groups. Similarly, for most aspects of the 
grid  of  the  relevance  of  higher  education  indicators  were  found  that  are  related  to  the 
indicator.  However,  only  for  14  out  of  53  considered  aspects,  are  indicators  readily 
available  in  international  data  sources  that  match  all  applied  criteria  of  good  content 
validity, regular updates and coverage of the majority of EU28 countries. 

The  availability  of  suitable  indicators  is  slightly  better  for  the  dimension  of  personal 
development than for sustainable employment and active citizenship. However, important 
aspects of the first dimension are not captured satisfactorily. 

3.4.1  Personal development: indicators, limitations and improvements 
For the personal development dimension, the supply of data is fairly good compared to the 
situation  for  the  other  two  dimensions.  Valid  indicators  are  regularly  available  for  the 
majority of EU28 countries on three aspects: trust in others, health and overall happiness. 
A  comparison  of  groups  with  different  levels  of  educational  attainment  can  indicate  the 
relevance  of  higher  education  for  these  aspects,  though  group  differences  cannot  be 
interpreted  as  causal  effects  of  higher  education.  Data  on  four  aspects  of  personal 
development was less satisfactory: 1) attitude towards self, 2) values, 3) motivation and 
4) student/graduate satisfaction with personal development and related aspects in higher 
education. Indicators are available, but due to lack of periodicity or limited geographical 
coverage, the current supply of data is not good. 

We  would  like  to  highlight  the  limited  data  on  two  aspects:  attitudes  towards  self  and 
values.  A  mature  personality,  being  self-reflective  and  developing  a  moral  compass  are 
seen  as  important  goals  of  higher  education  in  several  European  countries.  The  World 
Values Survey offers indicators for a smaller number of European countries; the European 
Values  Survey  covers  more  countries  but  is  only  reported  at  intervals  of  8-10  years. 
Regular  comparative  data  allowing  monitoring  the  majority  of  EU28  countries  on  these 
aspects is not available. Indicators of self-perception and ethical standards of graduates 
would shed more light on these personal development aspects. 

3.4.2  Sustainable employment: indicators, limitations and improvements 
For  the  sustainable  employment  dimension,  by  far  the  largest  number  of  aspects  and 
indicators has been collected. For the user group of students and graduates three aspects 
are  covered  by  indicators  matching  all  three  quality  criteria:  qualifications  of  higher 
education graduates, cultural capital and earnings. 

A lack of information is specifically visible for skills and competencies. The OECD survey of 
adult  skills  (PIAAC)  provides  important  data  of  high  quality  for  about  half  of  all  EU28 
countries.  The  reason  for  not  categorising  this  data  as  “satisfactory”  is  that  it  will 
presumably only be updated at long intervals of about ten years thus limiting its capacity 
for  regular  monitoring.  Moreover,  PIAAC  focusses  on  basic  and  a  few  transversal  skills. 
Thus  it  does  not  give  a  comprehensive  picture  on  skills  relevant  for  higher  education 
graduates.  Besides  more  transversal  and  generic  skills,  professional  skills  would  be 
interesting.  However,  measuring  subject-specific  skills  in  the  field  of  higher  education 
comparatively poses methodological challenges not yet fully solved. Moreover, several case 
study countries did not participate in the last round of PIAAC.  

Availability of data on most labour market aspects has been categorised as insufficient as 
well, often due to limited content validity. The EU-LFS regularly offers highest quality data 
on a large number of European countries. But its scope of variables is quite limited, because 
each additional question involves considerable costs as the number of respondents is very 

50 

 
 
 
high. This limited scope often prevents giving a detailed account of relevance aspects. EU-
LFS-ad-hoc modules or alternative data sources, such as the European Social Survey, can 
complement this information but do (often) not provide indicators regularly. Only rough 
measures  are  available  in  the  regular  EU-LFS  for  the  successful  transition  to  the  labour 
market, long-term prospects of graduates, or for adequacy of employment (e.g. horizontal 
mismatch is not covered).  

Regular  data  on  the  satisfaction  of  students  or  graduates  with  how  higher  education 
prepares for sustainable employment or supports personal development is not available. 
Only EUROSTUDENT asks students on their preparedness for the national and international 
labour  market,  however,  important  aspects  of  sustainable  employment  such  as  having 
acquired different kinds of skills are not covered. 

The relevance of higher education for employers is clearly less well covered by indicators. 
Availability of indicators on sustainable employment for this group has not been found to 
be satisfactory for any aspect. This hints at a specific lack of data. While information on 
students  and  graduates  or  the  society  at  large  can  often  be  extracted  from  general 
population  surveys  or  register  data  this  is  not  the  case  for  employers  or  companies. 
Surveys among employers and companies are organised rarely.  

The aspects referring to the society as user group are covered relatively well with suitable 
indicators on four out of 10 aspects: (1) (un)employment rates, (2) the impact of higher 
education  on  public  revenues,  (3)  impact  on  the  social  security  system,  and  (4)  the 
innovation capacity of the economy. Indicators on economic productivity, the knowledge-
based  economy  or  the  impact  on  growth  fall  short  of  being  satisfactory  mostly  because 
they do not fully grasp complex phenomena. Another drawback is that the effect of higher 
education on a certain outcome is even less clear at societal level than at the individual 
level as there is no equivalent to comparing groups with different levels of education. Still, 
valid estimates of the impact of higher education on public revenues and the social security 
system are regularly available. Such estimates are not available for the impact of higher 
education  on  the  innovation  capacity  of  the  economy,  its  productivity,  or  growth  which 
often presupposes complex statistical models instead of aggregate level indicators. 

Capacities to monitor the relevance of higher education for sustainable employment could 
be enhanced in a variety of ways: for the user group of students and graduates, indicators 
would  be  specifically  useful  if  they  covered:  earnings  to  subjects,  the  time  between 
graduation  and  first  employment,  a  match  between  subjects  and  occupation  (horizontal 
mismatch),  or  the  competences  of  graduates.  What  is  more,  subjective  assessments  of 
graduates on the usefulness of their studies for the labour market would add an interesting 
piece of information. The OECD provides an interesting indicator on the private returns to 
higher education but only for a few countries. 

Employer surveys could provide important complementary information on higher education 
relevance by directly measuring the view of employers of graduates. 

Information on the economic relevance of higher education for societies can be improved 
potentially.  Though  the  OECD  provides  a  good  indicator  on  the  public  returns  to  higher 
education, this information is not available for several countries. Indicators on economic 
productivity, the knowledge-based economy or the impact of higher education on growth 
are not satisfactory. Available indicators mostly cannot grasp such complex phenomena or 
relationships as they presuppose complex statistical modelling. Nevertheless, the effect of 
higher  education  on  economic  growth  might  be  a  specifically  helpful  indicator.  It 
summarizes a variety of effects of higher education in one indicator and is thus very telling.  

3.4.3  Active citizenship: indicators, limitations and improvements 
The  data  situation  for  the  active  citizenship  dimension  is  also  rather  mixed.  The  ESS 
provides good data on three out of eleven aspects: (1) political literacy, civic skills, and 
sense of ability to influence politics, (2) social participation and inclusion, and (3) political 

51 

 
 
participation.  But  indicators  on  democratic  values  as  well  as  tolerance  and  intercultural 
skills and values are only regularly available in international data sources for a minority of 
the EU28-countries (provided by the World Values Survey). Additionally, no regular data 
is  available  for  measuring  students’  or  graduates’  satisfaction  with  higher  education 
preparation for active citizenship. 

Indicators on equitable participation in higher education have also been categorised as not 
fully  satisfactory.  We  would  like  to  hint  at  three  shortcomings:  (1)  there  is  no  source 
regularly  providing  indicators  on  the  participation  of  students  with  an  immigration 
background as one important group for the social dimension (only since the fifth round of 
EUROSTUDENT are such data are collected). (2) In international sources, there are to the 
best of our knowledge hardly any regular indicators shedding light on the social dimension 
and  qualitative  differences  within  higher  education  for  example  types  of  institutions 
(private vs. public, professional higher education vs. research universities, or Ivy League 
institutions  vs.  other  institutions).  (3)  EUROSTUDENT  focusses  on  the  current  student 
population, but results may be different for graduates. General or working-age population 
surveys  could  provide  such  information  but  either  do  not  regularly  cover  the  variables 
needed (e.g. the EU-LFS) or hold too few cases for reliable regular monitoring of the social 
dimension (e.g. the European Social Survey).  

Democratic values and tolerance are  often mentioned as goals of the agendas of higher 
education  policies  of  European  countries  at  national  level  as  well  as  at  European  level 
(Yerevan  Communiqué  of  the  Bologna  ministers).  Regular  comparable  indicators  on 
whether higher education graduates hold democratic values and intercultural skills could 
support  such  political  goals.  Relevance  policies  could  be  supported  by  students’  and 
graduates’ assessments of whether their studies fostered active citizenship and in which 
way.  

Generally,  there  is  a  specific  lack  of  (comparative)  data  on  subjective  assessments  of 
higher  education  by  students  and  graduates.  Such  assessments  could  be  specifically 
informative for improving higher education as they capture relevance in the view of those 
having  received  higher  education.  Similarly,  there  is  hardly  any  data  systematically 
covering  the  views  of  employers,  as  another  important  stakeholder  group,  on  higher 
education. 

It should be kept in mind that our assessment of the availability and suitability of indicators 
on the relevance of higher  education was nearly exclusively based on international data 
sources. Further indicators may well be available for individual countries. 

Based on the above collection of indicators, we have chosen ten indicators for a diagnostic 
tool (see Chapter 4). These indicators are deemed to be useful for policy-makers to quickly 
assess some important aspects reflecting the relevance of higher education. Most of the 
indicators  capture  the  content  of  the  aspect  well,  are  regularly  updated,  and  cover  the 
majority of EU member countries (i.e. they are shaded in green in the tables above). Some 
indicators  –  those  on  relative  earnings,  vertical  mismatch,  the  distribution  of  skills  and 
social  representation  –  violate  at  least  one  of  the  criteria  mentioned  earlier  (they  are 
shaded  orange  in  the  tables  above).  They  have  been  chosen  due  to  lack  of  better 
alternatives. 

52 

 
 
 
 
 
 
 
 
 
 
4  Analytic and diagnostic tool 

4.1  Introduction to the analytic and diagnostic tool 

The analytic and diagnostic tool aims to give a brief overview of countries’ policies on the 
relevance of higher education. Moreover, it shows countries’ profiles based on indicators 
linked to the relevance of higher education. For both, relevance policies and profiles, the 
tools can only give a rough outline and are by no means comprehensive. For policies, the 
overview gives an idea of the breadth, focus, and instruments of the relevance policies of 
the countries and thus analyses what countries do to improve on the relevance of higher 
education.  The  key  indicator  profiles  give  a  diagnosis  of  how  countries  score  on  certain 
relevance indicators. They offer an impression of the profile of higher education graduates 
with regard to the three dimensions of the relevance of higher education used in this study: 
(1) personal development, (2) sustainable employment, and (3) active citizenship. Aspects 
of all three dimensions are covered by suitable indicators. However, for some indicators 
there is room for improvement and different or complementary indicators could have been 
chosen.  

The  indicators  draw  on  a  comparison  of  results  for  persons  with  higher  educational 
attainment  and  those  with  medium-level  education.  They  are  meant  to  describe  and 
quantify – at least to some extent – the relevance of higher education. Differences between 
higher  education  graduates  and  persons  without  degrees  can  be  caused  by  a  variety  of 
factors, in large part external to higher education, e.g. the labour market structure, the 
welfare state system, or other branches of education. Thus the indicators cannot provide 
an assessment of the higher education system as such. However, they can reveal where 
room for improvement exists and where the outcomes of higher education policies seem 
promising. In this sense they can guide the actions of higher education decision makers. 

4.1.1  Limitations of the analytic and diagnostic tool 
An important caveat pertains to relationships and causality. The term “relevance of higher 
education”  suggests  that  higher  education,  at  least  to  some  extent,  causes  a  certain 
desired outcome, be it at an individual or the societal level. Similarly, the aggregate level 
indicators  presented  suggest  a  relationship  between  higher  education  and  a  certain 
outcome,  e.g.  attitudes,  skills,  unemployment,  income  etc.  Over  the  last  five  decades, 
theoretical and empirical research has indeed identified various interrelations between the 
level of education acquired and outcomes such as earnings, health, political participation, 
happiness,  or  economic  growth  (e.g.  Bassanini  and  Scarpetta,  2001;  Carstensen  and 
Jungbauer-Gans, 2016; Enders and Bornmann, 2001; Field, 2009; Harmon et al., 2003; 
Holland  et  al.,  2013;  Hout,  2012;  Krueger  and  Lindahl,  2001;  Recotillet,  2007; 
Spangenberg et al., 2012). This evidence supports the assumption of a positive impact of 
higher  education  on  several  of  the  key  indicators  presented  below.  It  must  be  stated, 
however, that aggregate level indicators are rough measures. Although they are useful for 
showing developments, country-differences, and group-differences, they are neither suited 
for proving causal relationships nor do they tell us about the size of the influence higher 
education may have on a certain outcome. The relationship between higher education and 
its potential outcomes is much more  complex than aggregate level indicators can show. 
Therefore, it would not be advisable to deduce policies based only on the results displayed 
by the following indicators.  Instead, context data on a national level and more in-depth 
analyses  based  on  micro-level  data  would  be  necessary  to  get  a  more  comprehensive 
picture and to arrive at potentially adequate policies. 

Such context and micro-level data should include detailed information on higher education, 
such as educational trajectories, styles of teaching and learning, international mobility and 
institutional characteristics, as well as the mid- and long-term development of graduates. 

53 

 
 
 
To date, such information is missing in many European countries as their graduate surveys 
rarely track graduates for more than just the first 12-18 months after graduation (Mühleck 
et al., 2016). Analyses across countries are particularly promising, as they can give insights 
into the influence of national context  conditions (e.g.  structures  of the higher  education 
system and the labour market) and impact of national level policies. To date, such analyses 
are  hardly  possible  due  to  the  lack  of  comparative  and  comparable  data.  Drawbacks  of 
existing comparative data and the incompatibility of national graduate studies support the 
call  for  initiating  a  new  comparative  study  focussing  on  higher  education  graduates 
(Mühleck et al., 2016). 

4.1.2  Choice and design of key indicators 
Ten key indicators have been selected to give a concise overview of the relevance of higher 
education. For most indicators we, firstly, display either a mean value or a percentage for 
persons with higher education (e.g. the mean value for the level of trust in other people 
or the share of unemployment). Secondly, depicted on the second horizontal axis, a ratio 
is  shown  to  compare  persons  with  higher  education  against  persons  with  medium-level 
education. This ratio is generated by dividing the value for both groups. A ratio larger than 
one  indicates  that  the  value  for  persons  with  higher  education  is  larger  than  for  their 
counterparts  with  medium-level  education.  If  the  ratio  is  less  than  one,  the  opposite  is 
true.  A  ratio  of  exactly  one  means  there  is  no  difference  in  the  characteristic  attribute 
between the two groups. For most of the key indicators a European or an OECD average 
is offered to compare. 

Personal development 

Trust  in  others,  overall  happiness,  and  health  are  non-monetary  returns  to  higher 
education.  They  are  either  indirectly  linked  to  personal  development,  like  health  and 
happiness, or are seen as direct expressions of a mature personality, like cooperativeness 
and trust. It would have been desirable to further consider indicators on aspects such as 
attitudes towards self, perception of self, or ethical standards. Such indicators are available 
in the World Values Survey (WVS), but only for a limited number of European countries. 

a)  Level  of  trust  in  other  people:  Trust  in  other  people  is  seen  as  one  indicator  of  a 
reflective and developed personality. Higher levels of mutual trust are seen as beneficial 
for social cohesion and the society as a whole. Higher levels of education are assumed 
to  go  together  with  higher  levels  of  interpersonal  trust.  The  European  Social  Survey 
(ESS) offers complementary indicators covering the perceived level of cooperative spirit 
in society or the value of being helpful. 

•  Question: “Using this card, generally speaking, would you say that most people can be trusted, or that 

you can’t be too careful in dealing with people?”  

•  Measurement: Mean value on 11-point scale, 0 = ”you can’t be too careful” to 10 = ”most people can 

be trusted”. 

•  Levels of education: higher education: ISCED1997 5A-6; medium-level education: ISCED1997 3-4. 
• 
•  Source: Own calculations of ESS, 7th wave (2014). 

International aggregate: 18 EU countries. 

b)  Happiness: The pursuit of happiness can be regarded as an overarching goal of human 
behaviour and education may serve as a helpful tool. Education may increase directly 
one’s benefits and the economic returns of education may enable one to consume other 
desired  goods.  Complementary  indicators  on  satisfaction  with  various  aspects  of  life 
are available in the ESS. 

•  Question: “Taking all things together, how happy would you say you are?” 
•  Measurement: Share of respondents for the three highest categories of 11-point scale, 0 = ”extremely 

unhappy” to 10 = “extremely happy” 

•  Levels of education: higher education ISCED1997 5A-6; medium-level education: ISCED1997 3-4.  
• 
•  Source: Own calculations of ESS, 7th wave (2014). 

International aggregate: 18 EU countries. 

54 

 
 
 
c)  Health: Education may affect one’s own health condition, by e.g. the awareness of the 
importance  of  a  healthy  lifestyle,  ability  to  purchase  better  quality  of  food,  or 
occupations  with  working  conditions  less  detrimental  to  health.  At  the  same  time 
prestigious  occupations  may  involve  long  working  hours  and  time  pressure.  The 
indicator  could  be  complemented  by  additional  indicators  of  EU-SILC,  e.g.  on  long-
standing illness or health-problems. 

•  Question: Question on how a person perceives his/her health in general.  
•  Measurement: Share of respondents for the two highest categories of 5-point scale, 1 = ”very bad” to 5 

= “very good” 

•  Levels of education: higher education: ISCED2011 5-8, medium-level education: ISCED2011 3-4.  
• 
•  Source: Own calculations of European Commission, Eurostat Database (2016). 

International aggregate: 28 EU countries. 

Sustainable employment 

A variety of indicators are available related to sustainable employment. Still, availability of 
information is far from satisfactory in several respects (see chapter 3).  

d)  Unemployment: The indicator chosen measures the unemployment rate, which is an 
important  standard  indicator  for  assessing  the  condition  of  a  labour  market  and 
economic performance.  

• 
• 
• 
• 

Measurement: Share of unemployed in % for persons aged 25-54.  
Levels of education: higher education: ISCED2011 5-8, medium-level education: ISCED2011 3-4. 
International aggregate: 28 EU countries. 
Source: European Commission, Eurostat Database (2016). 

e)  Relative earnings: Earnings are an important economic return to higher education. 

•  Measurement:  Relative  earnings  of  full-time  full-year  employees  aged  25-64  years.    Income  of 
employees with upper secondary education = 100. An index value of 1.9, for instance, means that the 
wage  level  of  employees  with  tertiary  education  is  90  %  higher  than  that  of  their  peers  with  upper 
secondary education. 

•  Levels of education: tertiary education, upper secondary education.  
• 
•  Source: OECD, Education at a Glance 2016, OECD indicators. 

International aggregate: 22 EU countries.  

f)  Vertical mismatch: This indicator looks at over-qualification rates of higher education 
graduates and may provide insights into current inefficiencies in the coordination of the 
higher education system and the labour market.   

•  Measurement:  Share  of  people  aged  25-34  with  tertiary  education  with  occupations  not  traditionally 
regarded as requiring a tertiary qualification (occupations with International Standard Classification of 
Occupations (ISCO) levels 4-9).  

•  Levels of education: higher education: ISCED1997 5-6.  
• 
•  Source: Source: European Commission/EACEA/Eurydice, 2015, The European Higher Education Area in 

International aggregate: EHEA.  

2015: Bologna Process Implementation Report, Luxembourg, p. 193. 

g)  Returns: private costs and benefits of higher education: A ‘simple’ but comprehensive 
way of looking at the monetary advantage  of tertiary education is by calculating net 
financial  returns  across  the  graduate’s  work  life.  Private  net  returns  can  be  used  by 
potential students as a criterion for the decision whether to study in higher education. 
The indicator is available for both sexes; women were chosen for this report as they 
represent the majority of higher education graduates. 

•  Measurement: Net present value for the difference of private costs and benefits for a woman attaining 
tertiary  education  as compared with  a  woman  attaining  upper secondary/post-secondary  non-tertiary 
education in US-$ (PPP) over the entire life span.  

•  Levels of education: tertiary education, upper secondary/post-secondary non-tertiary education.  
• 
•  Source: Source: OECD, Education at a Glance 2016, OECD indicators. 

International aggregate: 22 EU countries.  

Returns:  public  costs  and  benefits  of  higher  education:  Public  net  returns  to  higher 
education  can  be  used  by  the  state  as  decision  support  when  reflecting  upon  public 

55 

 
 
investment  in  human  capital.  The  indicator  is  available  for  both  sexes;  women  were 
chosen for this report as they represent the majority of higher education graduates. 

•  Measurement: Net value public costs and benefits for a woman attaining tertiary education as compared 
with a woman attaining upper secondary/post-secondary non-tertiary education in US-$ (PPP) over the 
entire life span.  

•  Levels of education: tertiary education, upper secondary/post-secondary non-tertiary education.  
• 
•  Source: OECD, Education at a Glance 2016, OECD indicators. 

International aggregate: 22 EU countries.  

h)  Distribution of skills to use ICT: The indicator measures a transferable skill needed for 

a large variety of occupations.  

•  Measurement: Proficiency in problem solving and the use of information and communication technologies 
(ICT).  Share  of  persons  classified  as  having  “good  ICT  and  problem-solving  skills”  (highest  of  5 
categories) based on skills tests.  

•  Levels of education: higher education: ISCED2011 5-8, medium-level education: ISCED2011 3-4.  
• 
•  Source: OECD, Education at a Glance 2016, OECD Indicators. 

International aggregate: 17 EU countries and 3 EU subnational entities.  

Active citizenship 

The  ESS  regularly  provides  a  number  of  indicators  measuring  political  participation  or 
indicators  closely  related  to  political  participation  which  can  be  compared  across  groups 
with  different  levels  of  educational  attainment.  For  the  diagnostic  tool,  we  have  chosen 
self-confidence  for  political  participation,  which  is  a  well-established  instrument  for 
measuring  political  self-efficacy.  Other  subjective  aspects  of  active  citizenship,  such  as 
democratic values or intercultural tolerance are only available in certain waves of the ESS, 
but  not  regularly.  The  World  Values  Survey  (WVS)  holds  further  indicators,  but  only  for 
less than half of the EU countries.  

i)  Self-confidence  for  political  participation:  Higher  education  is  assumed  to  capacitate 
graduates with skills that are beneficial for an active participation in democracies.  

•  Question: “And using this card, how confident are you in your own ability to participate in politics?” 
•  Measurement: Mean value on 11-point scale, 0 = ” not at all confident” to 10 = ”completely confident” 
•  Levels of education: higher education: ISCED1997 5A-6, medium-level education: ISCED1997 3-4.  
• 
•  Source: Own calculations of ESS, 7th wave (2014). 

International aggregate: 18 EU countries. 

j)  Social  representation:  Equal  participation  of  social  groups  means  that  the  social 
composition of the student body reflects the social composition of the overall populace. 
We have placed this aspect within the dimension of active citizenship; however, it has 
the capacity to boost the relevance of higher education in all three dimensions, as a 
broader share of the population benefits from higher education. 

•  Measurement: Index value relating the share of students whose fathers have attained higher education 
to the share of males aged 40-59 years with higher education in the total population. An index value of 
1 indicates that there are exactly as many students from a certain educational background as expected 
based  on  the  distribution  of  educational  attainment  in  the  male  population.  Values  over  1  indicate 
overrepresentation of the group, values below 1 underrepresentation.  

•  Levels of education: higher education. 
• 
•  Source: EUROSTUDENT V 2012-2015 Synopsis of indicators, 2015, Social and Economic Conditions of 

International aggregate: 24 European countries. 

Student Life in Europe, Bielefeld, p. 54. 

Most of the indicators of the three dimensions stem from internationally renowned sources 
such as the OECD, Eurostat or the ESS. Due to the various measures of quality assurance 
(e.g. with respect to sampling methods and the number of respondents), the quality of the 
data  is  generally  quite  good.  The  data  are  internationally  comparable  and  have  a  good 
geographical  coverage  with  up  to  28  EU  countries.  Most  of  the  indicators  are  regularly 
available, i.e. they are being updated either at two-year intervals (e.g. ESS) or some of 
them  even  on  an  annual  basis.  However,  there  are  also  indicators,  e.g.  the  skills 
measurements, which are supposed to be updated only in long intervals. For other aspects, 
such as the impact of sustainable employment on the creativity of employees, low turnover 

56 

 
 
 
in  staff,  specific  effect  of  higher  education  on  economic  growth  or  innovation  suitable 
indicators  are  either  not  available  at  all  or  not  on  a  regular  basis.  These  are  severe 
limitations for monitoring capacities and it is recommended to take actions for improving 
the regular supply of data (see conclusions and recommendations). 

4.2  Country score cards 

The  country  score  cards  (i.e.  the  analytic  and  diagnostic  tool)  for  the  eight  case  study 
countries Canada (Ontario), the Czech Republic, France, Denmark, Germany, Ireland, the 
Netherlands  and  Spain  are  presented  below.  Conclusions  from  the  diagnostic  tool  are 
presented at the end of this chapter. 

57 

 
 
 
 
Canada-Ontario
Canada-Ontario

Main system characteristics

0%

20%

40%

60%

80%

100%

first time entry

attainment

0,6

0%

20%

40%

60%

80%

100%

source: OECD (2016) Education at a Glance

exp HE (% GDP)

exp ed (% GDP)

0

0

0,5

2,5

Providers: structural diversity
   ○ Unitary system
   ○ Binary system
   ○ Multi-sector system

1

5

1,5

2

7,5

10

Regulated learning outcomes
Quality assurance & accreditation
Regulated study places and quotas
Functions of sectors of HEIs
Stimulated student engagement
Relevance indicators in funding formula
Large strategic budgets
Short/medium term funds for special initiatives
Student financial support incentives
Special funds for specific target groups

Main policy levers

PD SE AC
R
R

R/O

F

Development / Performance agreements 
Student counselling & career guidance (centres)
Additional/new types of degrees / programmes
Centres of Expertise or Innovation (regional)
Access for specific target groups
Student satisfaction / engagement surveys / monitors
Graduate / employer surveys
Promotion of particular disciplines/programmes
Student choice portal (central website)

PD SE AC
F

O

I

I

PD = Personal development, SE = Sustainable employment, AC = Active citizenship

R = Regulation, F = Funding, O = Organisation, I = Information

Personal development

Key indicators

Level of trust in other people
0

1

2

3

4

5

6

7

8

9

10

Health

higher 
educated

0

20

40

60

80

100

0,4

0,8

1,2

1,6

2

0

0,4

0,8

1,2

1,6

2

higher/medium

10

20

30

40

50

60

70

80

90

100

higher educated

higher/medium

Happiness

0

0

higher educated

higher/medium

0

0,4

0,8

1,2

1,6

Sustainable employment

Unemployment
0

4

8

12

16

2

20

Returns: costs and benefits for a woman attaining HE 
120000
40000
(US$, PPP)

80000

0

160000

200000

higher educated

higher/medium

private c/b

public c/b

0

0,4

0,8

1,2

1,6

2

0

40000

80000

120000

160000

200000

Relative earnings

higher/medium

0,0

0,4

0,8

1,2

1,6

2,0

Vertical mismatch

higher educated

0

10

20

30

40

50

Active citizenship

Distribution of skills to use ICT

20

0

40

60

80

higher 
educated

higher/medium

0

1

2

3

4

Self-confidence for political participation
1

0

2

3

4

5

6

7

8

9

10

Social representation (>1=overrepresented)

higher educated

58 

higher/medium

0,0

0,4

0,8

1,2

1,6

2,0

higher 
educated

0,0

0,5

1,0

1,5

2,0

2,5

Data for Canada-Ontario
EU/OECD average

o Data for other participating countries

 
 
 
 
Czech Republic
Czech Republic

Main system characteristics

0%

20%

40%

60%

80%

100%

first time entry

0,7

attainment

0,2

0%

20%

40%

60%

80%

100%

source: OECD (2016) Education at a Glance

exp HE (% GDP)

exp ed (% GDP)

0

0

0,5

2,5

1

5

4,0

1,5

2

1,3

7,5

10

Providers: structural diversity
   ○ Unitary system
   ○ Binary system
   ○ Multi-sector system

Regulated learning outcomes
Quality assurance & accreditation
Regulated study places and quotas
Functions of sectors of HEIs
Stimulated student engagement
Relevance indicators in funding formula
Large strategic budgets
Short/medium term funds for special initiatives
Student financial support incentives
Special funds for specific target groups

Main policy levers

PD SE AC
R
R
R

F

F

Development / Performance agreements 
Student counselling & career guidance (centres)
Additional/new types of degrees / programmes
Centres of Expertise or Innovation (regional)
Access for specific target groups
Student satisfaction / engagement surveys / monitors
Graduate / employer surveys
Promotion of particular disciplines/programmes
Student choice portal (central website)

PD SE AC

O

I
I

PD = Personal development, SE = Sustainable employment, AC = Active citizenship

R = Regulation, F = Funding, O = Organisation, I = Information

Key indicators

7

8

9

10

Health

higher 
educated

higher/medium

3

4

5

6

5,9

1,1

0

20

40

60

80

100

76,7

1,3

0,4

0,8

1,2

1,6

2

0

0,4

0,8

1,2

1,6

2

10

20

30

40

50

60

70

80

90

100

Personal development

Level of trust in other people
0

1

2

higher educated

higher/medium

Happiness

0

0

higher educated

higher/medium

54,3

1,2

1,2

1,6

0

0,4

0,8

Sustainable employment

Unemployment
0

4

8

12

16

higher educated

2,2

higher/medium

0,5

2

20

Returns: costs and benefits for a woman attaining HE 
120000
40000
(US$, PPP)

80000

0

private c/b

public c/b

82.500

160000

200000

162.800

0

0,4

0,8

1,2

1,6

2

0

40000

80000

120000

160000

200000

Relative earnings

higher/medium

1,9

0,0

0,4

0,8

1,2

1,6

2,0

Vertical mismatch

higher educated

17,0

0

10

20

30

40

50

Active citizenship

Distribution of skills to use ICT

20

0

higher 
educated

higher/medium

40

60

60

80

2,5

0

1

2

3

4

Self-confidence for political participation
1

0

2

3

4

higher educated

higher/medium

5

6

7

8

9

10

Social representation (>1=overrepresented)

4,8

higher 
educated

2,0

1,3

0,0

0,5

1,0

1,5

2,0

2,5

59 

0,0

0,4

0,8

1,2

1,6

2,0

Data for Czech Republic
EU/OECD average

o Data for other participating countries

 
 
 
Denmark
Denmark

Main system characteristics

0%

20%

40%

60%

80%

100%

first time entry

0,9

attainment

0,4

0%

20%

40%

60%

80%

100%

source: OECD (2016) Education at a Glance

exp HE (% GDP)

exp ed (% GDP)

0

0

0,5

2,5

Providers: structural diversity
   ○ Unitary system
   ○ Binary system
   ○ Multi-sector system

1

5

6,4

1,5

2

1,7

7,5

10

Regulated learning outcomes
Quality assurance & accreditation
Regulated study places and quotas
Functions of sectors of HEIs
Stimulated student engagement
Relevance indicators in funding formula
Large strategic budgets
Short/medium term funds for special initiatives
Student financial support incentives
Special funds for specific target groups

Main policy levers

PD SE AC
R
R
R

F

F
F

F
F

Development / Performance agreements 
Student counselling & career guidance (centres)
Additional/new types of degrees / programmes
Centres of Expertise or Innovation (regional)
Access for specific target groups
Student satisfaction / engagement surveys / monitors
Graduate / employer surveys
Promotion of particular disciplines/programmes
Student choice portal (central website)

PD SE AC
R/F
O

I

PD = Personal development, SE = Sustainable employment, AC = Active citizenship

R = Regulation, F = Funding, O = Organisation, I = Information

Personal development

Key indicators

Level of trust in other people
0

1

2

3

4

5

6

7

8

9

10

8,5

1,1

Health

0

20

40

60

higher 
educated

higher/medium

1,1

100

80

80,5

0,4

0,8

1,2

1,6

2

0

0,4

0,8

1,2

1,6

2

10

20

30

40

50

60

70

80

90

100

80,3

1,1

higher educated

higher/medium

Happiness

0

0

higher educated

higher/medium

0

0,4

0,8

1,2

1,6

Sustainable employment

Unemployment
0

4

8

12

16

2

20

higher educated

higher/medium

5,0

1,0

Returns: costs and benefits for a woman attaining HE 
120000
40000
(US$, PPP)

80000

0

160000

200000

private c/b

public c/b

25.600

74.300

0

0,4

0,8

1,2

1,6

2

0

40000

80000

120000

160000

200000

Relative earnings

higher/medium

1,3

0,0

0,4

0,8

1,2

1,6

2,0

Vertical mismatch

higher educated

15,1

0

10

20

30

40

50

Active citizenship

Distribution of skills to use ICT

20

0

higher 
educated

higher/medium

40

60

80

55

1,8

0

1

2

3

4

Self-confidence for political participation
1

0

2

3

4

higher educated

60 

higher/medium

5

6

7

8

9

10

Social representation (>1=overrepresented)

7,4

1,3

higher 
educated

2,0

0,0

0,5

1,0

1,5

2,0

2,5

0,0

0,4

0,8

1,2

1,6

2,0

Data for Denmark
EU/OECD average

o Data for other participating countries

 
 
 
 
France
France

Main system characteristics

0%

20%

40%

60%

80%

100%

first time entry

attainment

0,3

exp HE (% GDP)

exp ed (% GDP)

0

0

0,5

1

1,5

1,5

2

5,3

5

2,5

7,5

10

0%

20%

40%

60%

80%

100%

source: OECD (2016) Education at a Glance

Providers: structural diversity
   ○ Unitary system
   ○ Binary system
   ○ Multi-sector system

Regulated learning outcomes
Quality assurance & accreditation
Regulated study places and quotas
Functions of sectors of HEIs
Stimulated student engagement
Relevance indicators in funding formula
Large strategic budgets
Short/medium term funds for special initiatives
Student financial support incentives
Special funds for specific target groups

Main policy levers

PD SE AC
R
R
R
R

R/F/O

F
F

Development / Performance agreements 
Student counselling & career guidance (centres)
Additional/new types of degrees / programmes
Centres of Expertise or Innovation (regional)
Access for specific target groups
Student satisfaction / engagement surveys / monitors
Graduate / employer surveys
Promotion of particular disciplines/programmes
Student choice portal (central website)

PD SE AC
F
O
O
O

I

I

PD = Personal development, SE = Sustainable employment, AC = Active citizenship

R = Regulation, F = Funding, O = Organisation, I = Information

Personal development

Key indicators

Level of trust in other people
0

1

2

3

4

5

6

7

8

9

10

Health

0

20

40

60

6,7

1,2

higher 
educated

higher/medium

1,1

100

80

80,3

0,4

0,8

1,2

1,6

2

0

0,4

0,8

1,2

1,6

2

10

20

30

40

50

60

70

80

90

100

65,2

1,3

higher educated

higher/medium

Happiness

0

0

higher educated

higher/medium

0

0,4

0,8

1,2

1,6

Sustainable employment

Unemployment
0

4

8

12

16

2

20

higher educated

higher/medium

5,8

0,6

Returns: costs and benefits for a woman attaining HE 
120000
40000
(US$, PPP)

80000

0

160000

200000

private c/b

public c/b

0

0,4

0,8

1,2

1,6

2

0

40000

80000

120000

160000

200000

Relative earnings

higher/medium

1,4

0,0

0,4

0,8

1,2

1,6

2,0

Vertical mismatch

higher educated

25,6

0

10

20

30

40

50

Active citizenship

Distribution of skills to use ICT

20

0

40

60

80

higher 
educated

higher/medium

0

1

2

3

4

Self-confidence for political participation
1

4

3

0

2

higher educated

higher/medium

5

6

7

8

9

10

Social representation (>1=overrepresented)

6,3

1,3

higher 
educated

1,7

0,0

0,5

1,0

1,5

2,0

2,5

61 

0,0

0,4

0,8

1,2

1,6

2,0

Data for France
EU/OECD average

o Data for other participating countries

 
 
 
Germany
Germany

Main system characteristics

0%

20%

40%

60%

80%

100%

first time entry

0,6

attainment

0,3

0%

20%

40%

60%

80%

100%

source: OECD (2016) Education at a Glance

exp HE (% GDP)

exp ed (% GDP)

0

0

0,5

2,5

1

5

4,3

1,5

2

1,2

7,5

10

Providers: structural diversity
   ○ Unitary system
   ○ Binary system
   ○ Multi-sector system

Regulated learning outcomes
Quality assurance & accreditation
Regulated study places and quotas
Functions of sectors of HEIs
Stimulated student engagement
Relevance indicators in funding formula
Large strategic budgets
Short/medium term funds for special initiatives
Student financial support incentives
Special funds for specific target groups

Main policy levers

PD SE AC
R
R

R/O

R

F

F

F
F

F

Development / Performance agreements 
Student counselling & career guidance (centres)
Additional/new types of degrees / programmes
Centres of Expertise or Innovation (regional)
Access for specific target groups
Student satisfaction / engagement surveys / monitors
Graduate / employer surveys
Promotion of particular disciplines/programmes
Student choice portal (central website)

PD SE AC
F
O

I

I

O

I
O/I

PD = Personal development, SE = Sustainable employment, AC = Active citizenship

R = Regulation, F = Funding, O = Organisation, I = Information

Personal development

Level of trust in other people
0

1

2

Key indicators

3

4

5

6

8

9

10

7

7,0

Health

higher 
educated

0,4

0,8

1,2

1,2

higher/medium

1,6

2

0

0,4

0,8

1,2

1,6

2

10

20

30

40

50

60

70

80

90

100

0

20

40

60

80

100

73,5

1,1

higher educated

higher/medium

Happiness

0

0

higher educated

higher/medium

72,6

1,2

1,2

1,6

0

0,4

0,8

Sustainable employment

Unemployment
0

4

8

12

16

higher educated

2,3

higher/medium

0,6

2

20

Returns: costs and benefits for a woman attaining HE 
40000
120000
(US$, PPP)

80000

0

160000

200000

private c/b

public c/b

0

0,4

0,8

1,2

1,6

2

0

40000

80000

120000

160000

200000

Relative earnings

higher/medium

1,6

0,0

0,4

0,8

1,2

1,6

2,0

Vertical mismatch

higher educated

16,7

0

10

20

30

40

50

Active citizenship

Distribution of skills to use ICT

20

0

higher 
educated

higher/medium

0

1

40

60

80

53

2,0

2

3

4

Self-confidence for political participation
1

0

4

2

3

higher educated

62 

higher/medium

5

6

7

8

9

10

Social representation (>1=overrepresented)

7,3

1,3

higher 
educated

2,0

0,0

0,5

1,0

1,5

2,0

2,5

0,0

0,4

0,8

1,2

1,6

2,0

Data for Germany
EU/OECD average

o Data for other participating countries

 
 
 
 
Ireland
Ireland

Main system characteristics

0%

20%

40%

60%

80%

100%

first time entry

attainment

0,4

exp HE (% GDP)

exp ed (% GDP)

0

0

0,5

1

1,5

2

1,2

5,2

5

2,5

7,5

10

0%

20%

40%

60%

80%

100%

source: OECD (2016) Education at a Glance

Providers: structural diversity
   ○ Unitary system
   ○ Binary system
   ○ Multi-sector system

Regulated learning outcomes
Quality assurance & accreditation
Regulated study places and quotas
Functions of sectors of HEIs
Stimulated student engagement
Relevance indicators in funding formula
Large strategic budgets
Short/medium term funds for special initiatives
Student financial support incentives
Special funds for specific target groups

Main policy levers

PD SE AC
R
R

R
O

R

F

R

F

O
F
F

Development / Performance agreements 
Student counselling & career guidance (centres)
Additional/new types of degrees / programmes
Centres of Expertise or Innovation (regional)
Access for specific target groups
Student satisfaction / engagement surveys / monitors
Graduate / employer surveys
Promotion of particular disciplines/programmes
Student choice portal (central website)

PD SE AC
F

O
I/R
I

I/R

PD = Personal development, SE = Sustainable employment, AC = Active citizenship

R = Regulation, F = Funding, O = Organisation, I = Information

Personal development

Key indicators

Level of trust in other people
0

1

2

3

4

5

6

8

9

10

7

7,1

Health

higher 
educated

0

20

40

60

80

100

91,3

1,1

higher/medium

1,0

0,4

0,8

1,2

1,6

2

0

0,4

0,8

1,2

1,6

2

10

20

30

40

50

60

70

80

90

100

58,5

1,1

higher educated

higher/medium

Happiness

0

0

higher educated

higher/medium

0

0,4

0,8

1,2

1,6

Sustainable employment

Unemployment
0

4

8

12

16

2

20

higher educated

higher/medium

5,2

0,5

Returns: costs and benefits for a woman attaining HE 
120000
40000
(US$, PPP)

80000

0

160000

200000

private c/b

public c/b

0

0,4

0,8

1,2

1,6

2

0

40000

80000

120000

160000

200000

Relative earnings

higher/medium

0,0

0,4

0,8

1,2

Vertical mismatch

higher educated

1,6

1,6

36,9

Distribution of skills to use ICT

20

0

higher 
educated

2,0

higher/medium

40

60

80

45

2,5

0

1

2

3

4

0

10

20

30

40

50

Active citizenship

Self-confidence for political participation
1

3

2

0

4

5

6

7

8

9

10

Social representation (>1=overrepresented)

higher educated

higher/medium

0,0

0,4

0,8

5,7

1,2

1,2

higher 
educated

1,1

0,0

0,5

1,0

1,5

2,0

2,5

1,6

2,0

Data for Ireland
EU/OECD average

o Data for other participating countries

63 

 
 
 
Netherlands
Netherlands

Main system characteristics

0%

20%

40%

60%

80%

100%

first time entry

0,7

attainment

0,4

0%

20%

40%

60%

80%

100%

source: OECD (2016) Education at a Glance

exp HE (% GDP)

exp ed (% GDP)

0

0

0,5

2,5

Providers: structural diversity
   ○ Unitary system
   ○ Binary system
   ○ Multi-sector system

1

5

5,5

1,5

2

1,7

7,5

10

Regulated learning outcomes
Quality assurance & accreditation
Regulated study places and quotas
Functions of sectors of HEIs
Stimulated student engagement
Relevance indicators in funding formula
Large strategic budgets
Short/medium term funds for special initiatives
Student financial support incentives
Special funds for specific target groups

Main policy levers

PD SE AC
R
R
R
R
R
R

F

F

F

Development / Performance agreements 
Student counselling & career guidance (centres)
Additional/new types of degrees / programmes
Centres of Expertise or Innovation (regional)
Access for specific target groups
Student satisfaction / engagement surveys / monitors
Graduate / employer surveys
Promotion of particular disciplines/programmes
Student choice portal (central website)

PD SE AC
F
F

O

I

O
O

I
I
I
I

I

I

PD = Personal development, SE = Sustainable employment, AC = Active citizenship

R = Regulation, F = Funding, O = Organisation, I = Information

Personal development

Key indicators

Level of trust in other people
0

2

1

3

4

5

6

7

8

9

10

7,7

1,1

Health

higher 
educated

0

20

40

60

80

100

86,0

higher/medium

1,1

0,4

0,8

1,2

1,6

2

0

0,4

0,8

1,2

1,6

2

10

20

30

40

50

60

70

80

90

100

75,2

1,1

higher educated

higher/medium

Happiness

0

0

higher educated

higher/medium

0

0,4

0,8

1,2

1,6

Sustainable employment

Unemployment
0

4

8

12

16

2

20

higher educated

3,3

higher/medium

0,5

Returns: costs and benefits for a woman attaining HE 
120000
40000
(US$, PPP)

80000

0

private c/b

public c/b

114.100

160000

200000

179.300

0

0,4

0,8

1,2

1,6

2

0

40000

80000

120000

160000

200000

Relative earnings

higher/medium

1,5

0,0

0,4

0,8

1,2

1,6

2,0

Vertical mismatch

higher educated

17,8

0

10

20

30

40

50

Active citizenship

Distribution of skills to use ICT

20

0

higher 
educated

higher/medium

40

1,7

60

64

80

0

1

2

3

4

Self-confidence for political participation
1

3

4

0

2

5

6

7

8

9

10

Social representation (>1=overrepresented)

higher educated

64 

higher/medium

0,0

0,4

0,8

5,9

1,2

1,2

higher 
educated

1,3

0,0

0,5

1,0

1,5

2,0

2,5

1,6

2,0

Data for Netherlands
EU/OECD average

o Data for other participating countries

 
 
 
 
Spain
Spain

Main system characteristics

0%

20%

40%

60%

80%

100%

first time entry

0,7

attainment

0,4

0%

20%

40%

60%

80%

100%

source: OECD (2016) Education at a Glance

exp HE (% GDP)

exp ed (% GDP)

0

0

0,5

2,5

1

5

4,3

1,5

2

1,3

7,5

10

Providers: structural diversity
   ○ Unitary system
   ○ Binary system
   ○ Multi-sector system

Regulated learning outcomes
Quality assurance & accreditation
Regulated study places and quotas
Functions of sectors of HEIs
Stimulated student engagement
Relevance indicators in funding formula
Large strategic budgets
Short/medium term funds for special initiatives
Student financial support incentives
Special funds for specific target groups

Main policy levers

PD SE AC
R
R

R

Development / Performance agreements 
Student counselling & career guidance (centres)
Additional/new types of degrees / programmes
Centres of Expertise or Innovation (regional)
Access for specific target groups
Student satisfaction / engagement surveys / monitors
Graduate / employer surveys
Promotion of particular disciplines/programmes
Student choice portal (central website)

PD SE AC

I

PD = Personal development, SE = Sustainable employment, AC = Active citizenship

R = Regulation, F = Funding, O = Organisation, I = Information

Personal development

Key indicators

Level of trust in other people
0

2

1

3

4

5

6

7

8

9

10

6,4

1,1

Health

higher 
educated

0

20

40

60

80

100

87,2

higher/medium

1,0

0,4

0,8

1,2

1,6

2

0

0,4

0,8

1,2

1,6

2

10

20

30

40

50

60

70

80

90

100

59,9

1,0

higher educated

higher/medium

Happiness

0

0

higher educated

higher/medium

0

0,4

0,8

1,2

1,6

Sustainable employment

Unemployment
0

4

8

12

16

2

20

higher educated

higher/medium

0,6

12,8

Returns: costs and benefits for a woman attaining HE 
120000
40000
(US$, PPP)

80000

0

private c/b

public c/b

59.500

160000

200000

176.200

0

0,4

0,8

1,2

1,6

2

0

40000

80000

120000

160000

200000

Relative earnings

higher/medium

1,4

0,0

0,4

0,8

1,2

1,6

2,0

Distribution of skills to use ICT

20

0

40

60

80

higher 
educated

higher/medium

Vertical mismatch

higher educated

38,8

0

1

2

3

4

0

10

20

30

40

50

Active citizenship

Self-confidence for political participation
1

3

4

2

0

higher educated

higher/medium

5

6

7

8

9

10

Social representation (>1=overrepresented)

6,5

1,2

higher 
educated

0,0

0,5

1,0

1,5

2,0

2,5

65 

0,0

0,4

0,8

1,2

1,6

2,0

Data for Spain
EU/OECD average

o Data for other participating countries

 
 
 
4.2.1  Introduction to the wheel charts 
The wheel charts display the relative scores on the relevance indicators of the case study 
countries at a glance (excluding Canada – Ontario). As such the charts convey the national 
‘relevance  profiles’.  They  show  on  which  indicators  the  country  scores  well  or  less  well 
compared to the other case study countries. The charts can be used to compare countries 
with each other. Using the example of Denmark, a guideline for interpretation is provided 
below. All wheel charts are presented on the next page. 

Figure 4.1: Wheel chart of Denmark’s scores on relevance indicators as compared to the other case study countries 

The  length  of  the  rays  represent  the  standardised  scores  on  the  indicators.  A  long  ray 
means that the country scores higher than average. A short ray means that the score is 
below the average. Looking by way of example at the trust indicator (the long blue ray), 
we can see that the level of trust in other people among Danish highly educated is – as 
compared to the other six case study countries – relatively high. However, looking at the 
difference  in  trust  between  the  highly  and  medium  educated  populations  (the  second, 
short,  ray),  Denmark  scores  relatively  low.  This  means  that  the  difference  between  the 
highly and medium educated populations in terms of trust is higher in other countries. This 
could be interpreted as higher education having a comparatively weak effect as compared 
to other countries. 

66 

 
 
 
 
 
 
 
 
 
67 

 
 
 
 
 
 
 
 
4.3  Conclusions based on the analytic and diagnostic tool 

In  summary,  the  insights  gained  by  applying  the  selected  indicators  show  the  positive 
effects  of  higher  education  in  many  areas  and  countries.  For  all  three  dimensions  the 
results underscored the relevance of higher education by suggesting relationships between 
tertiary  educational  attainment  and  a  wide  range  of  positive  outcomes,  such  as  health, 
skills,  earnings,  or  political  participation.  However,  the  use  of  such  indicators  has  its 
caveats.  Most  rely  on  bivariate  relationships  between  two  variables  (often:  tertiary 
educational  attainment  and  a  proposed  outcome).  While  the  results  often  support  the 
expected correlation between the two, the differences between educational groups are not 
tested for statistical significance. Moreover, it must be kept in mind that the exact size of 
the effect of higher education cannot be identified, let alone that causal relationships can 
be proven:  

  The  actual  relationship  could  have  the  reverse  direction  (e.g.  higher  happiness 

levels encouraging a person to pursue more education, or facilitating this); 

  The pattern could be the result of a more indirect causal chain (e.g. higher education 
leading to higher income leading to better healthcare provision, rather than higher 
education leading directly to better health); 

  The outcome of interest can be (additionally/instead) influenced by other individual 
or system-level variables, which are not taken into account (e.g. economic strength 
of a country). 

Additionally,  the  indicators  themselves  may  be  influenced  by  restrictions  with  regard  to 
their validity and reliability. Bearing these qualifications in mind, the diagnostic tool shows 
that persons with higher education score considerably better on all indicators than persons 
with medium-level education. 

Regarding personal development, persons with tertiary education report higher levels of 
trust in other people in all countries. In France and Germany, the mean value of trust in 
other people among higher educated exceeds the level of trust of persons with secondary 
education by more than 20%. In all other countries the difference is about 10%. Compared 
to  their  peers  with  upper  secondary  education,  tertiary-educated  persons  also  indicate 
better levels of health in all countries, with the largest difference in the Czech Republic. 
Additionally, self-reported happiness levels tend to be higher among persons with tertiary 
degrees for all countries except Spain. Again, differences are relatively large in the Czech 
Republic, France and Germany. 

With regard to sustainable employment, five indicators are considered. Overall they show 
positive  relationships  between  tertiary  education,  individual  skills,  and  labour-market 
related  outcomes.  With  the  exception  of  Denmark,  the  risk  of  being  unemployed  is 
considerably lower for persons with higher education in all countries. Denmark’s focus on 
sustainable  employment  of  graduates  may  be  a  political  reaction  to  this.  For  three 
countries,  the  unemployment  rate  of  tertiary  educated  is  about  half  of  the  size  of  the 
unemployment rate of persons with medium education (Czech Republic, Ireland, and the 
Netherlands).  In  all  countries,  higher  educated  persons  earn  more  than  persons  with 
medium-level education. The difference is the smallest in Denmark (earnings about 30% 
higher) and the largest for the Czech Republic (earnings nearly twice as high). 

Private  and  public  net  benefits  of  higher  education  as  compared  to  upper  secondary 
education are positive in all countries. Returns vary considerably across countries. Private 
returns  are  (in  descending  order)  especially  high  for:  the  Netherlands,  Spain,  and  the 
Czech  Republic.  In  Denmark,  private  returns  are  relatively  low.  Public  returns  are  the 
highest  in  the  Netherlands  and  the  Czech  Republic  and  somewhat  lower  in  Spain  and 
Denmark.  No  data  was  available  for  Germany,  France,  and  Ireland.  The  indicators  on 
returns suggest that investments in higher education are expected to pay off financially in 

68 

 
 
 
 
all  countries  from  a  private  as  well  as  from  a  public  perspective.  Persons  with  higher 
education  are  more  likely  to  have  good  ICT  problem  solving  skills.  The  proportion  with 
good skills is about twice as high as among persons with upper secondary education in the 
countries that have participated in the first round of the OECD survey of adult skills. Ireland 
and  the  Czech  Republic  reported  relatively  large  differences  and  Germany  an  average 
difference. For the Netherlands, differences in skills levels are somewhat below average. 
No data was available for France and Spain.  

A certain proportion of  higher educated are vertically mismatched in all countries, there 
are, however, major differences. A group of four countries show a relatively small share of 
overqualified  persons  with  a  tertiary  degree,  ranging  between  15-18%:  Denmark, 
Germany,  the  Czech  Republic  and  the  Netherlands.  In  contrast  France  (26%),  Ireland 
(37%), and Spain (39%) report much higher mismatch levels.  

Regarding active citizenship, tertiary-educated persons are more confident regarding their 
own ability to exert an influence on their countries’ politics than persons with medium-level 
education.  Countries  do  not  differ  a  lot  in  the  size  of  this  difference.  In  contrast,  the 
representation of social groups within higher education varies across countries. The higher 
education systems of Germany, the Czech Republic, Denmark, and France are relatively 
selective, while the systems of the Netherlands and Ireland are more inclusive (no data on 
Spain). 

Though a comparative analysis of seven countries is limited, the indicators suggest some 
tentative patterns and relationships. Denmark is an example of a country with relatively 
little  advantages  for  higher  educated  persons  across  all  indicators.  Differences  between 
educational  groups  are  comparatively  small  on  indicators  on  personal  development  and 
even more so regarding indicators on sustainable employment. The earnings advantage of 
the higher educated is smallest across all countries and unemployment levels do not differ 
at all. Does this mean that higher education is of little relevance in Denmark? This question 
hints at the limitations of the analysis. Indicators on Denmark show high levels of social 
cohesion (highest overall levels of social trust, happiness, and self-confidence in political 
participation).  Private  (74,300€)  and  public  (25,600€)  returns  of  women  with  a  higher 
education degree are comparatively small but far from negligible. The proportion of higher 
educated  with  good  ICT  skills  is  nearly  twice  as  large  compared  to  persons  with  upper-
secondary education. Moreover, Denmark reported the smallest proportion of overqualified 
higher  educated  persons,  suggesting  a  good  match  with  the  labour  market,  which  is 
interesting given the strong political emphasis on employability in Denmark. Denmark has 
a relatively large proportion of persons with higher education and is known for its relatively 
strong  welfare  state.  Both  aspects  have  the  potential  to  mitigate  differences  between 
educational groups, which is typically seen as another desirable political goal. 

Spain reported small and near to negligible differences between educational groups in the 
indicators on personal development, while at the same time having relatively low overall 
levels of social trust and happiness. Economic advantages of higher educated are larger 
than  in  Denmark  but  still  comparatively  small.  Spain  has  the  largest  proportion  of 
overqualified persons with a higher education degree which may be one explanatory factor. 
Also the share of unemployed persons with a tertiary degree is largest among all countries, 
reflecting  overall  economic  difficulties.  At  the  same  time  and  somewhat  contradictorily, 
private net returns on higher education are comparatively high. Public returns are small 
compared  to  other  countries  but  still  considerable  (59,500€).  In  this  regard,  further 
investments in higher education could be expected to pay off. At the same time, levels of 
vertical mismatch and unemployment among higher education graduates are already quite 
high.  More  information  on  horizontal  mismatch  could  be  advantageous.  The  scope  and 
number  of  policy  levers  targeting  the  relevance  of  higher  education  is  relatively  small, 
suggesting room for more policy action. 

69 

 
 
 
 
 
 
 
 
 
 
The Netherlands and Ireland report relatively small differences in the aspects of personal 
development and active citizenship but medium to large differences in the indicators  on 
sustainable employment. Another commonality between both countries is the low level of 
social selectivity: persons with and without higher education do not differ a lot with regard 
to their social background. Family socialisation is very likely to impact on trust, happiness, 
health behaviour and political attitudes. Ireland is an example of an Anglo-Saxon welfare 
state  with  an  emphasis  on  equality  of  chances  (e.g.  by  access  to  education)  and  little 
redistribution. Thus differences in levels of education translate into economic differences, 
such  as  relative  earnings  or  levels  of  unemployment.  In  the  Netherlands,  economic 
differences between educational groups are medium to high as compared to the other case 
study  countries.  Public  and  private  returns  to  higher  education  are  the  highest  in  the 
Netherlands suggesting a strong economic relevance  of higher  education. This is also in 
line  with  the  relatively small level  of  vertical  mismatch  reported  for  the  Netherlands.  In 
contrast,  Ireland  shows  the  second-highest  proportion  of  overqualified  persons  with  a 
tertiary degree, reflecting economic difficulties and crowding-out effects at the expense of 
groups with lower levels of education. As a reaction, Ireland has put more policy focus on 
employability. Overall, both countries show a broad set of policy levers.  

Germany, France, and the Czech Republic are countries with medium to large differences 
between educational groups across all three relevance dimensions. All three countries have 
relatively  medium  to  small  proportions  of  persons  with  a  higher  education  degree  and 
higher  education  systems  that  are  comparatively  socially  selective.  Thus,  differences 
between educational groups may reflect both the relevance of higher education regarding 
beneficial  outcomes  as  well  as  social  selection  in  the  student  in-take  (i.e.  differences 
independent from higher education). Descriptive indicators do not allow for disentangling 
both factors. Besides these commonalities, the three countries differ in a variety of aspects. 
Economic advantages of the higher educated are only moderate in France. The potential 
benefits of higher education may well be impaired by the high level of over-qualification. 
In  the  Czech  Republic,  economic  advantages  are  more  pronounced  with  the  highest 
difference in earnings across all countries. Public and private returns on higher education 
are  comparatively high in this country as  well. This suggests a  relatively high degree  of 
economic relevance. For Germany, private economic benefits regarding employment and 
earnings are moderate as compared to other countries. Data on public and private returns 
was available neither for Germany nor for France.  

In summary, the insights gained by applying the selected indicators imply positive effects 
of  higher  education  in  many  areas  and  countries.  Skills  levels  and  returns  to  higher 
education  are  positively  related  to  higher  education  and  seem  to  be  good  measures  of 
relevance.  However,  the  scope  of  available  indicators  on  skills  of  higher  education 
graduates  is  still  very  limited.  Generally,  the  analysis  underscores  that  the  use  of 
monitoring  indicators  has  its  limitations.  Whether  higher  education  leads  to  positive 
outcomes as well as the degree of differences between educational groups depends on a 
large  variety  of  factors  and  context  conditions,  such  as  the  overall  size  of  the  higher 
education system, social selectivity in access to higher education, the labour market, the 
welfare  state,  or  the  overall  economic  situation.  Identifying  the  net  effect  of  higher 
education clearly asks for more complex analyses than are possible by using descriptive 
indicators. 

70 

 
 
 
 
 
 
5  Conclusions and recommendations 

Higher education provides valuable public and private benefits for various stakeholders in 
higher education and the wider society. As such, the relevance of higher education is an 
important  area  of  interest  for  policy  makers,  students,  graduates,  employers  and 
researchers. The main objective of the current study is to provide a comprehensive analysis 
of  the  relevance  of  higher  education  and  how  this  is  promoted  in  various  countries  in 
Europe and beyond. The study includes: 

  a thorough literature review to define and explore the concept of “the relevance of 

higher education”;  

  a mapping of national policies explicitly addressing higher education relevance; 
  eight  in-depth  country  case  studies  (Canada  [Ontario],  the  Czech  Republic, 
Denmark, France, Germany, Ireland, the Netherlands and Spain) exploring policy 
documents and stakeholder perspectives on higher education relevance; 

  an overview of indicators on the relevance of higher education in international data 

sources; and 

  an  analytic  and  diagnostic  tool  with  a  concise  set  of  indicators  that  provides  a 

snapshot critical performance analysis for policy makers. 

This  chapter  summarises  the  main  conclusions  of  this  study  (Section  5.1)  and  presents 
policy recommendations on promoting the relevance of higher education (Section 5.2). 

5.1  Conclusions 

5.1.1  The concept and importance of relevance 
A first conclusion is that the relevance of higher education is a broad and multi-dimensional 
concept. As such, the definitions of the relevance of higher education differ widely across 
countries  and  public  authorities  pay  different  levels  of  attention  to  relevance  and 
emphasise  different  aspects.  This  makes  international  comparisons  a  challenging 
endeavour, further complicated by differences between higher education systems in terms 
of national contexts, structures and traditions. 

Based on an elaborate literature review (see Annex 2 and 3), we defined the relevance of 
higher education in terms of its three main objectives as presented by the Council of Europe 
(2007): 1) personal development; 2) sustainable employment; and 3) active citizenship. 
Regarding  the  conceptualisation  and  importance  given  to  the  relevance  of  higher 
education, we arrive at the following key observations. 

1)  The literature review revealed that many aspects of higher education can be linked to 
“relevance”. The distinction between three dimensions – personal development, active 
citizenship  and  sustainable  employment  –  proved  to  be  helpful  in  uncovering  a  rich 
literature on these themes and a long list of factors that possibly affect the relevance 
of  higher  education.  Moreover,  in  the  literature,  policy  documents  and  country  case 
studies, other dimensions are seen as important to relevance, e.g. knowledge creation, 
personal  autonomy,  economic  development  and  socio-cultural  development.  The 
distinction  between  the  relevance  dimensions  may  not  always  be  clear-cut:  various 
aspects of dimensions may overlap or interrelate. 

2)  Sustainable employment is the dominant dimension of higher education relevance in a 
majority  of  European  countries.  The  following  factors  contribute  to  this.  Firstly, 
sustainable employment fits very well the contemporary narrative of the “usefulness” 
of  higher  education  for  the  knowledge  economy  and  the  instrumental  neo-liberal 
perspective  on  higher  education.  Secondly,  the  aftermath  of  the  economic  crisis 

71 

 
 
 
 
 
 
 
 
 
 
 
triggered  attention  for  (un)employment  levels,  mismatches  between  supply  and 
demand  and  youth  employment.  Finally,  from  a  pragmatic  perspective,  sustainable 
employment  is  considered  to  be  better  and  more  easily  measurable  than  personal 
development and active citizenship.  

3)  The  relative  importance  of  the  dimensions  of  relevance  appears  to  be  dynamic  over 
time,  with  some  aspects  of  relevance  gaining  importance  and  others  becoming  less 
prominent.  For  example  in  the  Czech  Republic  and  Ireland  sustainable  employment 
receives growing attention, while in the Netherlands personal development (Bildung) is 
gradually being put on an equal par with employability. 

5.1.2  What kind of policies do governments have in place? 
The review of policy literature and the eight in-depth case studies have identified a wealth 
of  policy  levers  explicitly  (or  implicitly)  designed  to  promote  the  relevance  of  higher 
education. These instruments have been categorised under the following types: regulation, 
funding, organisation and information. We analysed the policy instruments for each of the 
relevance  dimensions:  personal  development,  sustainable  employment  and  active 
citizenship.  The  analyses  focused  on  common  patterns  among  and  differences  between 
countries and on finding interesting or good practices. 

Personal development 

Personal  development  is  often  addressed  implicitly,  e.g.  in  France  and  Spain.  In  some 
countries  governments  have  a  specific  take  on  personal  development  as  they  stimulate 
personal development by integrating it in the learning outcomes of degree programmes. 
More  common  policies  relate  to  widening  participation  through  financial  support  for 
underrepresented students to secure equal access and personal development. Grants and 
scholarships in Germany and Ireland as well as loans in Denmark are envisaged to facilitate 
students  in  achieving  personal  development  goals.  Personal  development  can  also  be 
achieved  through  offering  support  for  students  with  mental  health  problems,  such  as  in 
Ontario (through the instrument of organisation). 

In  Ontario,  Germany,  and  the  Netherlands,  satisfaction  and  engagement  surveys  are  in 
place  to  monitor  how  students  experience  the  personal  development  function  of  higher 
education. In the Netherlands, the concept of “Bildung” is being reintegrated into the policy 
debate after a long dominance of a performance and employability orientation. 

Sustainable employment 

Sustainable  employment  appears  to  be  the  dominant  relevance  dimension  for  higher 
education in most countries studied. This is reflected in the intensity and wide spread of 
policy instruments. The “hard” policy levers of regulation and funding are dominant. 

Regarding regulations, typical sustainable employment policies include: 

  Regulating enrolments, including the determination of quotas (Canada, Czech 

Republic, Denmark, France, Germany) 

  Setting specific roles for higher education sectors, often in terms of learning 

outcomes (Canada, France, Germany, Ireland, Netherlands) 

  Making employability part of quality assurance and assessment arrangements, e.g. 

in accreditation procedures (Czech Republic, Denmark, France, Ireland, 
Netherlands, Spain). 

Funding policies to promote sustainable employment include: 

  Performance based-funding, including graduation/employment outcomes (Spain, 
Canada, Czech Republic, Denmark, France, Germany, Ireland, Netherlands) 

  Study finance for students (Germany, Ireland) 
  Specific funding initiatives (France, Germany, Netherlands), such as large 

investment programmes to improve higher education quality. 

72 

 
 
 
 
Sustainable employment is also promoted by various organisational policy levers: 

  Integrating employer representatives in advisory bodies, accreditation processes, 

and centres of expertise (Canada, Germany, Netherlands) 

  Setting up career guidance centres / organising advisory services for students 

(Denmark, Germany, France) 

  Additional or new types of degrees or programmes (Czech Republic, France, 

Germany, Netherlands) 

  Organising access to higher education / fields of study / additional programmes for 

specific target groups (Germany, Ireland, Netherlands). 

Policies addressing the information provision to prospective students include: 

  Data collected through student, graduate and employer surveys (Canada, Denmark, 

Germany, Ireland, Netherlands, and Spain) 

  Employment data feeding into student choice information or to promote certain 
disciplines (Czech Republic, Denmark, France, Germany, Netherlands, Spain). 

Active citizenship 

Not all countries have explicit policy levers addressing the active citizenship dimension of 
relevance (e.g. these are lacking in the Czech Republic and Ontario). Overall, the active 
citizenship  dimension  of  higher  education  is  primarily  supported  through  regulation  and 
funding, while organisation and information levers are rather rare. 

Regulations concerning active citizenship often include: 1) legislation that specifies that 
students  need  to  be  educated  for  active  citizenship  (Ireland,  Germany,  and  the 
Netherlands); 2) strengthening participation of students in higher education governance 
(Germany and Spain); 3) allowing flexible curricula to enable students to engage in civic 
service activities (France). 

Funding policy levers used to promote active citizenship include: 1) expanding access 
to higher education to students of poorer socio-economic backgrounds through the study 
finance system (Denmark and Germany); 2) funding, recognising and awarding credits for 
active involvement in student organisations (France). 

The only organisational policy lever used to promote active citizenship activities is the 
Irish  “Campus  Engage”  initiative,  which  invites  students  to  volunteer  with  local 
organisations on a pro bono basis for extra-curricular activities. 

Finally, information policies used concern student surveys that focus on how students 
are  integrated  in  student  life  and  engagement  and  how  that  benefits  their  social  values 
(Germany, Netherlands). 

5.1.3  Effectiveness of explicit relevance policies  
Overall, there is no comprehensive knowledge on the effectiveness of the policy levers in 
the eight case study countries. There are several explanations for this observation. First, 
many  policy  levers  have  been  implemented  recently.  Second,  policies  are  rarely  closely 
monitored  or  evaluated.  Third,  evaluations  do  not  necessarily  reflect  on  relevance 
dimensions. Fourth, monitoring on personal development and active citizenship is limited. 
Fifth,  the  overall  impact  of  policy  levers  may  be  affected  by  various  factors  outside  the 
scope of the policy lever. Lastly, evaluative data may have a time lag, thus not showing 
the result of the policy lever or its full (long term) impact. 

However, the available insights into effectiveness of policy levers illustrate that different 
policy  instruments  can  have  an  impact.  For  example,  evaluations  show  that:  (1)  labour 
market  information  allows  students  to  make  better  educational  choices  (Spain),  (2)  a 
funding  initiative  can  increase  the  attractiveness  of  the  STEM  disciplines  for  female 

73 

 
 
 
 
 
 
 
 
 
 
students  (Germany),  (3)  offering  part-time  places  in  higher  education  to  unemployed 
people  has  an  effect  on  their  sustainable  employment  (Ireland),  and  (4)  “excellence 
education” and the introduction of associate degrees show a positive impact on sustainable 
employment (Netherlands). 

The  above  mentioned  factors  indicate  that  more  evaluations  are  needed  to  gain  better 
insights  into  the  effectiveness  of  policy  levers,  as  well  as  into  the  conditions  for 
success/failure.  In  relation  to  this,  we  can  only  report  on  rather  generic insights  gained 
through the interviews with stakeholders: (1) governments need to carefully consider the 
appropriateness of policy instruments in their specific settings and also  reflect  on  which 
mix of instruments best fits with the pertinent challenges in their systems, (2) governments 
need  to  consider  which  instruments  would  be  best  organised  at  the  system  level  (e.g. 
through  regulation  and  funding)  and  which  instrument  should  better  be  left  to  higher 
education institutions themselves, and (3) effectiveness may be improved by the inclusion 
of all relevant stakeholders in the design and implementation processes.  

5.1.4  Data and indicators of relevance 
Based  on  the  review  of  several  international  data  sources,  an  inventory  was  made  of 
(potential) indicators of relevance. It shows that there are many available indicators and 
data  offering  insights  into  the  relevance  of  higher  education.  Chapter  3  and  Annex  2 
provide  substantial  lists  of  indicators  for  various  aspects  of  higher  education  relevance 
dimensions. These indicators have been assessed on their content validity (i.e. how well 
they measure an aspect of a higher education relevance dimension), refresh period (i.e. 
are they regularly updated), and country coverage (i.e. do they cover the majority of the 
EU28-countries). Only for a minority of relevance aspects an indicator matching all three 
criteria was found. In many cases indicators only provide rough measures, are available 
for only a few countries, or are not regularly updated.  

Our main conclusions on the supply of data on the relevance of higher education are: 

  For  the  personal  development  dimension,  the  supply  of  data  is  fairly  good  when 
compared to the other dimensions of relevance. Data on aspects such as 1) attitude 
towards self, 2) values, and 3) motivation are basically available, but due to lack of 
periodicity  or  limited  geographical  coverage,  the  current  supply  of  data  is  not 
satisfactory. 

  For  the  sustainable  employment  dimension,  plenty  of  indicators  are  available  but 
often content validity is not satisfactory. For the successful transition to the labour 
market, mid- and long-term careers  of graduates, or  for adequacy  of employment 
only rough measures are regularly available. 

  A lack of information was specifically visible for skills and competencies. 
  Data  on  aspects  such  as  political  literacy,  social  participation  and  political 
participation  are  regularly  available  in  international  data  sources.  Indicators  on 
democratic  values,  tolerance,  intercultural  skills  and  values  are  only  regularly 
available for a minority of the EU28-countries.  

  Many stakeholders have stated that personal development and active citizenship is 
very hard or impossible to measure. While data availability is far from satisfactory, it 
might be that stakeholders are not fully aware of the information already available in 
international data sources such as the ESS. 

  Data  on  the  subjective  views  of  stakeholder  groups  on  the  relevance  of  higher 
education  is  scarce.  No  regular  and  comparable  data  are  available  for  measuring 
graduates’ satisfaction with higher education regarding their personal development 
or for preparing them for the labour market or active citizenship.  

  Besides  some  examples  of  surveys  amongst  employers,  there  is  hardly  any  data 
collected  that  systematically  covers  the  views  of  employers  as  another  important 
stakeholder group on higher education. 

74 

 
 
 
 
This study focusses on aggregate level indicators, as this is a type of data prevalently used 
by decision makers and practitioners. It should be kept in mind that most indicators have 
limits on what they can tell about the relevance of higher education. They indicate a certain 
statistical  fact  but  are  typically  not  suitable  for  expressing  the  complex  relationship 
between higher education and a certain outcome. The indicators predominantly presented 
in  this  report  draw  on  a  comparison  of  results  for  persons  with  higher  educational 
attainment and those with medium-level education. Such indicators cannot prove a causal 
relationship or quantify the exact size of the influence higher education has on a certain 
outcome.  

Still, indicators can provide important information on the relevance  of higher education. 
This  study  has  provided  a  concise  set  of  indicators  that  can  be  used  for  measuring  the 
relevance of higher education in an international comparative way. These have been used 
to develop a diagnostic tool to monitor and give a snapshot of the performance of higher 
education  systems  in  terms  of  their  relevance  to  society  (see  Chapter  4).  Against  the 
backdrop of the above mentioned caveats, it is clear that such a monitoring tool does not 
give a comprehensive assessment of the higher education system as, inter alia, important 
context conditions cannot be systematically considered. 

Suggested indicators for personal development: 

  Level of trust in other people (ESS) 
  Happiness (ESS) 
  Health (EU-SILC) 

Suggested indicators for sustainable employment: 

  Unemployment (Eurostat) 
  Relative earnings (OECD, Education at a Glance) 
  Vertical mismatch (EC data)  
  Returns (OECD data) 
  Distribution of skills to use ICT (OECD data) 

Suggested indicators for active citizenship: 

  Self-confidence for political participation (ESS)  
  Social representation (EUROSTUDENT) 

In addition, this study identified a number of indicators that would be an interesting and 
valuable  asset  to  use  when  the  international  community  is  interested  in  promoting, 
monitoring and measuring the relevance of higher education (see Chapter 3). 

5.2  Recommendations 

Based on the outcomes of this study we present a number of recommendations for national 
and EU policy-makers to promote the relevance of higher education. 

Governments should develop more explicit policy designs to boost relevance 
Though most policy makers and stakeholders across the eight case study countries agree 
that  the  relevance  of  higher  education  is  related  to  the  dimensions  of  personal 
development, sustainable employment and active citizenship, most countries give explicit 
policy  attention  to  sustainable  employment,  while  aspects  of  personal  development  and 
active  citizenship  are  often  addressed  implicitly.  Relevance  policies  can  be  made  more 
explicit in the following ways: 

  National policy mixes should target a proper balance between the three dimensions 

of higher education relevance; 

75 

 
 
 
 
 
 
 
 
 
 
  Governments should be clear and explicit in defining and communicating the specific 
relevance  dimensions,  aspects,  goals  and  targets  that  they  regard  as  important, 
including the reasons for these priorities;  

  National  policy  levers  need  to  be  explicit  about  the  expected  roles  of  different 

stakeholders. 

The  eight case studies  point at several examples of instruments that  “work”. These  can 
guide national governments in developing their own policy instruments: 

Sustainable employment 

  Provide extra funding for study programmes that address labour market shortages; 
  Undertake graduate and employer surveys to monitor the graduate labour market; 
  Involve  labour  market  representatives  in  advisory  committees  and  the  quality 

assurance of education programmes; 

  Organise/improve career orientation and guidance. 

Personal development 

  Provide targeted funding for under-represented groups; 
  Integrate personal development explicitly in programme learning outcomes; 
  Measure levels of personal development in student (evaluation) surveys.  

Active citizenship 

  Allow  credit  to  be  awarded  for  extra-curricular  activities  and  prior  learning  (non-

formal learning); 

  Measure levels of active citizenship in student (evaluation) surveys. 

Stimulate the collection of evidence on the effectiveness of higher education relevance 
policies, and monitor, share and adopt successful policy practices 
To  improve  the  limited  attention  for  policy  evaluation  and  monitoring,  the  European 
Commission and national governments should take the following actions: 

  To initiate more systematic national and international comparative empirical research 

on the impact and effectiveness of higher education relevance policies; 

  To link the higher education relevance agenda to other higher education policy areas, 

for example, modernisation, quality assurance and internationalisation; 

  Use good practice examples to inspire national practices, e.g.: 

o  Mandatory evaluations of national policies (Denmark and the Netherlands); 
o 

The  use  of  indicators  addressing  all  three  relevance  dimensions  (Germany: 
adequate employment of graduates, satisfaction of graduates and employers); 
o  Share objective and experience-based information about study programmes at 

a central study portal (Studiekeuze 123 in the Netherlands). 

Governments as well as the European Commission should stimulate the collection of 
more robust data on the relevance of higher education 
It is desirable to organise coordinated action across national borders to build up a more 
solid knowledge base derived from commonly defined relevance indicators. 

  National governments should systematically collect information on the indicators of 
HE relevance using internationally shared definitions and should monitor outcomes; 
  Interaction between decision makers, practitioners and data providers at European 

and national levels could improve the quality and awareness of available data; 

  The  European  Commission  and  national  governments  should  invest  more  effort  in 
international  studies  that  enhance  the  international  knowledge  base  on  the 
dimensions  of  HE  relevance.  Initiatives  such  as  the  recent  recommendation  of  the 
European  Council  on  tracking  graduates,  including  the  European  pilot  graduate 
survey (EUROGRADUATE), are promising steps in this direction; 

76 

 
 
 
 
  The analytic and diagnostic tool designed and developed within this project is a useful 
starting point for the systematic monitoring of relevance indicators and can serve as 
an input for a qualitative policy debate on higher education relevance; 

  Linked  to  the  analytic  and  diagnostic  tool,  national  governments  should  be 
encouraged  to  utilize  existing  indicators  on  the  personal  development  and  active 
citizenship  dimensions,  such  the  level  of  trust  in  other  people,  level  of  happiness, 
self-confidence for political participation, and levels of social representation; 

  The  European  Commission  and  member  states  should  supplement  statistical 
indicators  with in-depth  analyses  to  improve  the  understanding  of  the  relationship 
between higher education and its outcomes. 

A  feasible  approach  to  develop  an  explicit  policy  approach  to  all  three  dimensions  of 
relevance  is  for  national  governments  to  gradually  adopt  various  elements  of  these 
recommendations, while carefully integrating them into their specific contexts. 

77 

 
 
 
 
 
 
 
 
 
 
 
Bibliography 

Bassanini, A. & Scarpetta, S. (2001): Does Human Capital Matter for Growth in OECD Countries? 

Evidence from Pooled Mean-Group Estimates, Economics Department Working Papers No. 282, 
OECD, Paris. 

Berdahl, R. (1983). Coordinating structures: the UGC and U.S. state coordinating agencies, in M. 
Shattock, (ed.), The structure and governance of higher education. Guildford: Society for 
Research into Higher Education. 

Bundesministerium für Bildung und Forschung (2007). Bekanntmachung der 

Verwaltungsvereinbarung zwischen Bund und Ländern über den Hochschulpakt 2020, 
Bundesanzeiger Nr. 171 of September 12th, 2007, Berlin.  

Bundesministerium für Bildung und Forschung (2014). Verwaltungsvereinbarung zwischen Bund 

und Ländern gemäß Artikel 91b Abs. 1 Nr. 2 des Grundgesetzes über den Hochschulpakt 2020.  

Carstensen, J., & Jungbauer-Gans, M. (2016). Health Returns to Education and Educational 

Systems. In Hadjar, A., & Gross, C. (Hrsg.). Educational Systems and Educational Inequalities. 
International Comparisons (S. 301-321). Bristol, UK: Policy Press 

Castells, M. (2001). Universities as dynamic systems of contradictory functions, in J. Muller, N. 
Cloete, and S. Badat, (eds.), Challenges of Globalisation. South African Debates with Manuel 
Castells. Cape Town: Maskew Miller Longman, pp. 206–223. 

Clark, B. R. (1983). The higher education system: academic organization in cross-national 

perspective, Berkeley, Calif.: University of California Press. 

Council of Europe / CoE (2007). Explanatory memorandum to recommendation rec (2007) 6 of the 
committee of ministers to member states on the public responsibility for higher education and 
research. 

Enders, J. & Bornmann, L. (2001). Karriere mit Doktortitel? Ausbildung, Berufsverlauf und 

Berufserfolg von Promovierten. Frankfurt a. M.: Campus. 

Field, J. (2009). Well-being and Happiness, IFLL Thematic Paper 4, Leicester: National Institute of 

Adult Continuing Education. 

Ganzeboom, H.B.G. De Graaf, P.M. & Treiman, D.J. (1992): A Standard International Socio-

Economic Index of Occupational Status. Social Science Research 21 (1), 1-56. 

Harmon, C., Oosterbeek, H., Walker, I. (2003): The Returns to Education: Microeconomics.  

Journal of Economic Surveys 17, 2, S. 115-155. 

Holland, D., Liadze, I., Rienzo, C., Wilkinson, D. (2013). The relationship between graduates and 

economic growth across countries. Department for Business Innovation & Skills, BIS Research 
Paper No. 110, London. 

Hood, C., and Margetts, H. Z. (2007). The tools of government in the digital age, Basingstoke: 

Palgrave Macmillan. 

Hout, M. (2012). Social and economic returns to college education in the United States. Annual 

Review of Sociology, 38, 379-400. 

Kolster, R., & Westerheijden, D. F. (2014). Employability of professional bachelors from an 

international perspective: final report. Enschede: CHEPS. 

Krueger, A.B. & Lindahl, M. (2001). Education for Growth: Why and For Whom? Journal of 

Economic Literature 39, 4, S. 1101-1136. 

Marginson, S. (2011). Higher Education and Public Good. Higher Education Quarterly, 65(4), 411-

433. 

Mühleck, K. (2013). Second-generation immigrants in Higher Education – sketching the European 

picture, in: A. Camilleri, and K. Mühleck, (eds.), Evolving diversity II. Participation of students 
with an immigrant background in European Higher Education. Brussels: Menon Network. 

Mühleck, K., Grabher, A., Hauschildt, K., Litofcenko, J., Mishra, S., Ryska, R., Unger, M., & 

Zelenka, M. (2016). Testing the Feasibility of European Graduate Study. Final Report of the 
EUROGRADUATE feasibility study, Hanover: German Centre for Higher Education Research and 
Science Studies. 

OECD (2013). Literacy, Numeracy and Problem Solving in Technology-Rich Environments. 

Framework for the OECD Survey of Adult Skills, Paris: OECD. 

Recotillet, I. (2007). PhD Graduates with Post

doctoral Qualification in the Private Sector: Does It 

Pay Off? Labour, 21(3), 473-502. 

Spangenberg, H., Mühleck, K. & Schramm, M. (2012). Erträge akademischer und nicht-

‐

akademischer Bildung. HIS: Forum Hochschule, 11/2012. Hannover: HIS. 

Trow, M. (1975). The Public and Private Lives of Higher Education. Daedalus, 104(1), 113-127. 

78 

 
 
 
 
 
Van Vught, F. A., and De Boer, H. (2015). Governance models and policy instruments, in J. 

Huisman, H. De Boer, D. D. Dill, and M. Souto-Otero, (eds.), Handbook of higher education 
policy and governance. London: Palgrave Macmillan. 

79 

 
 
 
 
 
 
 
 
 
 
 
 
HOW TO OBTAIN EU PUBLICATIONS 

Free publications: 

•  one copy: 

via EU Bookshop (http://bookshop.europa.eu); 

•  more than one copy or posters/maps: 

from the European Union’s representations (http://ec.europa.eu/represent_en.htm);  
from the delegations in non-EU countries (http://eeas.europa.eu/delegations/index_en.htm);  
by contacting the Europe Direct service (http://europa.eu/europedirect/index_en.htm) or 
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) (*). 

(*)  The information given is free, as are most calls (though some operators, phone boxes or hotels may 
charge you). 

Priced publications: 

•  via EU Bookshop (http://bookshop.europa.eu). 

 
 
 
 
l

C
a
t
a
o
g
u
e

n
u
m
b
e
r

N
C
-
0
4
-
1
8
-
2
1
6
-
E
N
-
N

doi: 10.2766/048735 

ISBN: 978-92-79-80311-6