EUROPEAN COMMISSION Brussels, 1.7.2020 SWD(2020) 121 final COMMISSION STAFF WORKING DOCUMENT



EVALUATION

of the Council Recommendation of 20 December 2012 on the validation of non-formal and informal learning







Accompanying the document COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS

European Skills Agenda for sustainable competitiveness, social fairness and resilience









{COM(2020) 274 final} - {SWD(2020) 122 final}



Table of contents 1.Introduction Purpose and scope 2.Background to the intervention Description of the intervention and its objectives 3.Implementation / State of Play 3.1 Baseline and points of comparison 3.2 Description of the current situation 4.Method Short description of methodology Limitations and robustness of findings 5.Analysis and answers to the evaluation questions 5.1 Effectiveness 5.2 Efficiency 5.3 Relevance 5.4 Coherence 5.5 EU added value 6.Conclusions and lessons learned 6.1 Conclusions

6.2 Lessons learned Annex 1: Procedural information Annex 2: Stakeholder consultation Annex 3: Methods and analytical models Glossary Term Meaning or definition Credit Confirmation that a part of a qualification, consisting of a coherent set of learning outcomes has been assessed and validated by a competent authority, according to an agreed standard. Credit is awarded by competent authorities when the individual has achieved the defined learning outcomes, evidenced by appropriate assessments and can be expressed in a quantitative value (e.g. credits or credit points) demonstrating the estimated workload an individual typically needs for achieving related learning outcomes. Guidance A continuous process that enables individuals to identify their capacities, skills and interests, through a range of individual and collective activities to make educational, training and occupational decisions and to manage their individual life paths in learning, work and other settings in which those capacities and skills are learned or used. Informal learning Learning resulting from daily activities related to work, family or leisure and is not organised or structured in terms of objectives, time or learning support. It may be unintentional from the learner's perspective. Examples of learning outcomes acquired through informal learning are skills acquired through life and work experiences, languages learned during a stay in another country, skills acquired through volunteering, youth work, sport or family life. Learning outcomes Statements regarding what a learner knows, understands and is able to do on completion of a learning process, which are defined in terms of knowledge, skills and responsibility and autonomy. National qualifications framework An instrument for the classification of qualifications according to a set of criteria for specified levels of learning achieved, which aims at integrating and coordinating national qualifications subsystems and improve the transparency, access, progression and quality of qualifications in relation to the labour market and civil society. Non-formal learning Learning which takes place through planned activities (in terms of learning objectives, learning time) where some form of learning support is present (e.g. student-teacher relationships). It may cover programmes to impart work skills, adult literacy and basic education for early school leavers; examples include in-company training, through which companies update and improve the skills of their workers such as ICT skills; structured on-line learning; courses organised by civil society organisations for their members, their target group or the public at large. Qualification A formal outcome of an assessment and validation process, which is obtained when a competent authority determines that an individual has achieved learning outcomes to given standards. Skills audit A process aimed at identifying and analysing the knowledge, skills and competences of an individual, including his or her aptitudes and motivations in order to define a career project and/or plan a professional reorientation or training project; the aim of a skills audit is to help the individual analyse his/her career background, to self-assess his/her position in the labour environment and to plan a career pathway, or in some cases to prepare for the validation of non-formal or informal learning outcomes. Third sector The sector of economy and society comprising non-governmental and non-profit organizations or associations, such as charities, voluntary and community groups, engaging in activities primarily serving a social or public purpose. Validation of non-formal and informal learning A process of confirmation by a competent authority that an individual has acquired learning outcomes acquired in non-formal and informal learning settings measured against a relevant standard and consists of the following four distinct phases: identification through dialogue of particular experiences of an individual, documentation to make visible the individual’s experiences, a formal assessment of those experiences and certification of the results of the assessment which may lead to a partial or full qualification. Acronym Meaning or definition ECTS European credit and transfer system ECVET European credit system for vocational education and training EQF European qualifications framework NGO Non-governmental organisation NQF National qualifications framework VET Vocational education and training 1.Introduction Purpose and scope This document evaluates the Council Recommendation of 20 December 2012 on the validation of non-formal and informal learning (hereafter “the Recommendation”). Evaluating the implementation and impact of EU legislation is a basic requirement of the Better Regulation framework. The Recommendation itself asks the Commission to report to the Council assessing and evaluating “the action taken in response to its provisions and on the experience gained and implications for the future”. The results of the evaluation will feed into the policy debate to put in practice the European Pillar of Social Rights, namely the first and fourth principles, with particular reference to the updated Skills Agenda (COM(2020) 274). In compliance with the Better Regulation Guidelines, the evaluation is based on the following criteria: –effectiveness: to what extent the objectives of the Recommendation have been achieved through action by Member States and the Commission; –efficiency: relationship between costs and benefits, relevant factors and proportionality of costs; –relevance: whether six years after the adoption of the objectives of the Recommendation, the measures it proposes to achieve them and the governance and support structures are still relevant; –coherence: whether the Recommendation remains coherent with other EU and national initiatives; –EU added value: whether the Recommendation has generated additional value compared to what action at national level alone would have produced.

The Recommendation asked Member States to take action no later than 2018, therefore the evaluation covers the period from the adoption of the Recommendation (end of 2012) to 2018 included. The geographic scope includes all EU Member States. This requires an explanation: ·Croatia became a Member State on 1 July 2013. Therefore, information on the situation until 2012, date of the adoption of the Recommendation, does not cover Croatia. This is explicitly mentioned in figures. ·The United Kingdom left the European Union on 31 January 2020. As this evaluation covers the period 2012-2018, in this Staff Working Document Member States always include the United Kingdom. 2.Background to the intervention Description of the intervention and its objectives The Recommendation addresses an important element of lifelong learning. Its objective is to support all people to develop their personal and professional skills and get them recognised, so that they can play an active role in society and the labour market. The lifelong learning approach focuses on the knowledge and skills that people develop throughout their life in any learning setting, either formal (e.g. institutional education and training programmes), non-formal (e.g. in-company training) or informal (e.g. work and life experience). The Recommendation should therefore be seen in the European policy context defined by initiatives such as the European area of lifelong learning, the strategic framework for European cooperation in education and training (Education and Training 2020) and more recently the 2016 Skills Agenda, with particular reference to the Upskilling Pathways initiative, and the European Education Area. Its implementation is related with the use of other tools developed within this policy process, such as Europass, the European Qualifications Framework (EQF), the European Credit System for Vocational Education Training (ECVET) and contributes to putting in practice the first and fourth principles of the European Pillar of Social Rights. Impact assessment. In this context, a Council Recommendation on validation of non-formal and informal learning was considered appropriate to address the following problems identified through the impact assessment: while there was widespread consensus on the benefits of skills validation for individuals, the economy, and society, in most Member States people had limited opportunities to validate their skills and where such opportunities were available, their use was limited. Besides, people who had their skills validated in one country might not be able to use the results in another Member State, as the differences in approach, scope and conceptual framework made it difficult to compare validation results across countries. Addressing these problems meant pursuing the general objectives of providing individuals in all Member States with opportunities to validate their skills acquired outside formal education and training systems and to use the results of validation to study or work anywhere in Europe. Figure 2.1. The two general objectives Two specific objectives were defined. First, establishing of national arrangements for the validation of non formal and informal learning, linked to the national qualifications framework. Second, making sure that such national arrangements had an appropriate level of quality and coherence, so that their results could be more easily understood and compared across countries, thanks in particular to the link to the EQF.



Figure 2.2. The two specific objectives Five operational objectives indicated the action to take to pursue the specific and general objectives. They focused on national qualifications frameworks, validation mechanisms, the value of skills identification and documentation, the need for cooperation, and awareness raising. Figure 2.3. The five operational objectives Recommendation. As adopted by the Council, the Recommendation does not explicitly mentions the objectives of the intervention logic, but its provisions relate to them. In its first invitation the Recommendation does invite Member States to allow individuals who developed skills through non-formal and informal learning to have them validated (cf. first general objective) and use them for their careers and further learning (cf. second general objective). For this purpose, Member States should put in place, by 2018, arrangements for the validation of non-formal and informal learning, which may give priority to certain areas or sectors, so that people may eventually obtain through validation a formal qualification or credit towards it (Invitation 1.1, referring to specific objective 1). People should be able to take advantage of each of the four phases of validation – identification, documentation, assessment and certification – separately or in a coordinated process, so that individual validation pathways are flexible and adapted to specific individual needs (Invitation 1.2 to Member States, related to operational objectives 2 and 3).

The Recommendation also provides a number of principles to be applied in setting up validation arrangements (Invitation 1.3 to Member States). They concern the link to national qualifications frameworks and the EQF (specific objective 1), the role of information and guidance (operational objective 5), qualification standards and use of learning outcomes (operational objective 1), the use of European transparency documents (specific objective 2), as well as specific target groups, quality assurance, skills audits, credit systems and validation staff – all complementary elements to pursue the various objectives. Invitation 1.4 to Member States focuses on cooperation among stakeholders (operational objective 4). The Recommendation asks the Commission to organise peer learning activities (operational objective 4) and support the national efforts by updating the European guidelines for validation of non-formal and informal learning and the European Inventory of validation of non-formal and informal learning (hereafter “Inventory”). As concerns governance, the Recommendation indicates the EQF advisory group, set up following the 2008 EQF Recommendation and confirmed by the 2017 EQF Recommendation, as the policy body most appropriate for its follow up. The Recommendation emphasises that validation promotes mobility (recitals 1 and 2), and Invitation 1 to Member States refers to the opportunity for people to use their validated skills for their careers and further learning (referring to general objective 1). Supporting people to use the results of validation across countries (second general objective) is pursued through European transparency tools: if validation arrangements are properly linked to national qualifications frameworks aligned to the EQF (Invitation 1.3.a, pursuing second specific objective and the first operational objective), then validation leads to qualifications, or parts of them, which are easier to understand and compare across Europe. Skills that are documented, but not assessed and certified in the form of qualifications (cf. operational objective 3 and Invitation 1.2), can still be made transparent in all Member States through the use of such instruments as the Youthpass or the Europass Mobility (cf. Invitation 1.3.i). In its recitals and definitions, the Recommendation stresses that validation of non-formal and informal learning is a transversal policy area, which contributes to implementing policies related to education and training, youth, employment and social inclusion. Validation is part and parcel of the lifelong learning approach and the shift to learning outcomes – knowledge, skills and competences acquired throughout life in a variety of settings, from school to work experience to volunteering and open learning resources. Given its cross-sector nature, implementing validation requires coordinated action by many institutional, social and economic actors. As a direct result, validation is found embedded in much of the EU policy on vocational education and training, higher education, adult learning (cf. Upskilling Pathways), school education, employment policy and youth policy. The figure below summarises the intervention logic. Figure 2.4. Intervention logic NQFs: National qualifications frameworks EQF: European qualifications framework



3.Implementation / State of Play 3.1 Baseline and points of comparison In 2012, some validation arrangements were in place in 20 Member States, with huge differences in scope and stage of development. Four Member States had well-established validation arrangements leading to qualifications (Finland, France, the Netherlands and Portugal). France was the only country where all qualifications in the national qualifications directory (with the exception of a few regulated professions) could also be obtained on the basis of validation of non-formal and informal learning. Seven further countries (Denmark, Germany, Luxembourg, Romania, Spain, Sweden and the United Kingdom) had either a national system in its initial phase or a well-established, but partial, system of validation in one or more areas. Limited arrangements were also available in Austria, Belgium, the Czech Republic, Estonia, Italy, Ireland, Lithuania, Slovakia and Slovenia, e.g. in Belgium (Flanders) and Slovenia validation could support people obtaining vocational qualifications. In other Member States some opportunities for validation were also available, but mostly in the form of single initatives without any systematic character. Validation was part of the policy debate in all Member States, but only 13 Member States declared to have some form of validation strategy in place or in development. Data on the take-up of validation opportunities were very scarce in 2012. The figure below summarises the situation in Member States before the adoption of the Recommendation, as concerns the availability of validation arrangements and related features described below.



Figure 3.1–1. Validation in Member States in 2012 Source: Inventory 2010 In 2010 validation could lead to partial or full qualifications in 24 Member States, while in six countries validation could grant access to formal education programmes. The link to national qualifications frameworks (NQFs) was uncertain, as at the time NQFs were being developed in all Member States, while they were fully operational only in four Member States and were approaching operation in a further nine countries. At the time 16 Member States had referenced national qualification levels to the EQF. In 2010, the quality of validation was assured through a specific framework in five countries, while 11 others applied wider quality assurance frameworks. While 24 Member States reported equivalence of standard between validation and formal education and training, that was rather a declaration of principle, as it also included countries that did not have a validation system in place. Often validation only concerns a limited set of skills, which would not be enough for the award of a full qualification. Validation results could however be certified in formal credit that may by further cumulated towards a full qualification. In 2010 this was possible in 10 Member States. Data from the 2010 Inventory shows that 17 Member States used the four stages of validation (identification, documentation, assessment and certification of skills) as a reference. While the Recommendation had not yet been proposed, the four stages had been detailed in the European guidelines for validating non-formal and informal learning, first released in 2009. There was little information about how guidance services supported people who might benefit from validation – e.g. conducting skill reviews, providing information on validation opportunities, and advising and assisting individuals throughout validation processes.

In 2010, skills audits to facilitate validation or res-skilling of disadvantaged groups were offered in ten Member States, while five countries organised specific targeted initiatives for migrants or refugees. Mandatory professional requirements (in terms of training, experience, qualification) for validation staff were reported in 17 Member States in 2010. Country reports in the 2010 Inventory show that in 14 Member States there was a good level of involvement of most relevant stakeholders in validation activities. 3.2 Description of the current situation 3.2.1 By the end of 2018, it appears that all Member States have taken some action towards the objectives of the Recommendation, though not enough to make validation opportunities available to all. Some degree of validation arrangements are in place in all Member States, almost all of which have or are developing some form of validation strategy. Validation is still primarily connected to education and training institutions, but validation opportunities have also increased in the labour market and the third sector – more recent growth has especially concerned the labour market. In 17 Member States there are arrangements for skills validation provided by labour market actors, often promoted by employers or associations of employers, sometimes in collaboration with public sector institutions. In 20 Member States there are validation arrangements made available in the third sector. This includes initiatives associated with youth work or volunteering, as well as validation opportunities developed by charities or other non-governmental organisations (NGOs) to support specific target groups such as third country nationals or people with a disability. Labour market actors such as Chambers of Commerce may be able to validate and certify skills, while in the third sector the focus is more on the documentation of skills. While information on such validation opportunities was not available in 2012, the large share of countries concerned suggests a significant increase in opportunities in Member States, as also confirmed by a huge majority (186 out of 261) of respondents to the public consultation. While some developments took place in all Member States, in a few countries (Croatia, Hungary, Slovakia) arrangements remain quite limited and in most, they are not comprehensive. Three systems (in Belgium-Flanders, Italy and Portugal) have been developed as broad frameworks covering all areas, adding them to the wide national arrangements already in place in 2012 (in Denmark, Finland, France, Luxembourg and Sweden). All other Member States show a variety of less comprehensive approaches. Sometimes good projects are not considered up-scalable, such as the German ValiKom project. Wide systems may be under development, but still focus on vocational education and training, as in Poland. Different frameworks often coexist, e.g. addressing the ‘labour market route’ and the ‘education route’ as in the Netherlands, or the subsectors of education and training, as is the case in Spain or Lithuania. All arrangements have some specific limitations. For instance, validation may only be available to workers with work experience, as is the case in the otherwise comprehensive systems in France and Luxembourg. The figure in the next page summarises the situation of validation in Member States before the adoption of the Recommendation (cf. Figure 3.1–1) and in 2018, as concerns the availability of validation arrangements and related features.



Figure 3.2–1. Validation in Member States 2012 – 2018 Source: Inventory 2010 and 2018. 3.2.2. Some data on participation are now available from most Member States, but they are often not comprehensive, e.g. referring only to a specific sub-sector of education and training, and fragmentary. However, the external study concludes, based on the 2018 Inventory triangulated with other sources, that in 14 Member States (BE, BG, CY, EL, ES, FI, IT, LU, LV, MT, PL, PT, SE, SI, SK) there has been an upward trend in take-up of validation opportunities after 2012. Use remained stable in three countries, while it slightly decreased in countries (Denmark, France and the Netherlands) with a solid validation system, as well as in Romania. The figure below summarises the trend in the use of validation comparing data from 2016 and 2018, also showing the lack of data for several countries. Figure 3.2-2. Trend in the use of validation (2018 compared to 2016) Source: Inventory 2016 and 2018. Cf. External study, Table 6. In 22 Member States validation has become more accessible, that is people find it easier to apply. This may have been achieved through embedment in labour policies (Italy), adult education policy (Estonia), equality of standards (as in Lithuania) or specific measures, such as the German ValiKom initiative. Compared to the policy attention to validation and the growth of opportunities, the increase in take-up by people may appear limited. Besides, the external study found evidence of some ‘evaporation effect’: people who start a validation procedure and do not complete it. In fact, engaging in validation processes requires a serious commitment by individuals, and forms of active support to individuals, such as paid time made available by the employer or financial aid, are not common. Barely more than one in ten validation beneficiaries responding to the public consultation (8 out of 62, or 11%) had received an incentive or support to participate in validation. Figure 3.2-3. Response to public consultation question 17 (asked to individuals).

Have you received financial support? (External study, Figure A.3.19.) 3.2.3. The increase in validation arrangements is correlated to the sustained development of national qualifications frameworks (NQFs) based on learning outcomes and their connection to validation arrangements. Both developments are related to the shift towards learning outcomes in defining and describing qualifications standards, which occurred in all parts of education and training. This approach promotes the value and facilitates the assessment of skills developed outside formal education and training. By the end of 2018, almost all Member States had an NQF referenced to the EQF. In 24 Member States, validation can lead to partial or full qualifications included in the NQF and in 17 it enables people to access formal education programmes leading to qualifications included in the NQF. In 13 countries, any qualification included on the NQF can be acquired through validation. 3.2.4. In most Member States, validation can lead to qualifications applying the same standards used in formal programmes, although in a few countries equivalence of standards is still being developed. In many countries, there is a mixed approach, so that validation can also lead to qualifications that are not the same as those delivered through formal education and training. This equivalence – still in development in a few countries – has been strongly promoted by the gradual shift away from input based standards, restricting time, learning content and context, towards standards 7based on learning outcomes. Still, the external study points out that in some countries (BG, ES, SI) results of formal education retain a higher status and in others (EL, MT) the opportunity to obtain tertiary qualifications through validation is largely theoretical. Credibility of validation results is also pursued through dedicated quality assurance frameworks, as done in 13 Member States. Five of them also use wider quality frameworks, which is the approach of other ten countries. A majority of organisations responding to the public consultation (111 out of 160) agreed that validation arrangements in their country met clearly established quality standards and slightly fewer (100 out of 157) acknowledged that this led to reliable and credible results.

Figure 3.2–4. Quality assurance of validation in 2010 and 2018

Source: Inventory 2010 and 2018. External study, Table 4.10. 3.2.5. In 2018, formal credit towards a qualification could be awarded through validation in 22 Member States, and in about as many countries people could be granted exemption from formal programmes, more frequently in higher education. However, Member States have not given priority to strong synergies between credit systems and validation. The external study has found little information on whether credit obtained through validation is converted in the European Credit Transfer System (ECTS), which might support its portability to higher education programmes in other countries. Figure 3.2–5. Validation leading to formal credit – 2010 and 2018 Source: Inventory 2010 and 2018. External study, Figure 13. 3.2.6. While “all four stages of validation are in place in the vast majority of Member States”, comparison with the baseline is difficult for several reasons. First, because the information collected in the 2018 Inventory refers to the use in three areas (education and training, labour market and third sector) not applied in 2010. Second, because at national level different terminology may be used, sometimes to reflect specific practices – e.g. the phases of identification and documentation may be carried out together (Luxembourg) and be both covered by a phase called ‘counselling’ (Spain). Finally, the practical use of the stages depends on the purpose and context of each individual validation pathway. For instance, exemption from parts of a formal programme following validation may be considered or not as a stage of certification (into formal credits) based on the organisation of that specific programme and the system of which it is part. Therefore, while 16 Member States explicitly refer to the four stages as presented in the Recommendation, in others all stages are covered in validation processes, but called otherwise. The real change compared to 2010 is that validation is now commonly referred to as a process including all stages from identification to certification. 3.2.7. In 26 Member States people can take advantage of information and guidance related to validation and in 15 countries guidance is available to support people throughout all phases of a validation process. In the majority of Member States people are offered information and guidance on the outcomes and benefits of validation and on assessment – with particular reference to continuing vocational training and adult learning. Responses to the public consultation confirmed the perception of a certain improvement in information and guidance since 2012 (89 of 150 responding organisations, about 60%) as well as of guidance during the validation process (93 out of 160 responding organisations). On the other hand, one of the reasons why take-up was smaller than expected may indeed be that relevant guidance provision, while increased, is not very effective in reaching out to potential beneficiaries. Only 4 of the 64 beneficiaries responding to the public consultation found their validation opportunity thanks to a guidance centre. The coordination of validation initiatives with existing guidance arrangements is not always clear. 3.2.8. Provision directed to specific target groups has also increased. In 2018, 24 Member States systematically made skills audits available – an explicit invitation of the Recommendation. Figure 3.2–5. Skills audits made available in 2010 and 2018 Source: Inventory 2010 and 2018. External study, Table 12. A good number of countries carry out validation initiatives addressing specific target groups such as low-skilled adults or long term-unemployed. Targeted validation initiatives for migrants and refugees – sometimes as specific projects rather than systematic provision – were available in 23 countries. A few arrangements are open to all migrants and refugees (FI, DE, NL, SE and with some conditions AT and DK), while in eleven Member States, such measures are project-based. and as they address specific categories their visibility is relatively limited: about half organisations responding to the public consultation (81 out of 160) considered that validation arrangements in their country targeted disadvantaged groups (long-term unemployed, migrants, disabled persons) either to a high (18) or to some extent (63). Besides, the cost, complexity and length of validation procedures, along with fragmented provision and uncertain value, remain major barriers for individuals from disadvantaged groups. Figure 3.2–6 Targeted initiatives for migrants and refugees Source: Inventory 2010 and 2018. External study, Table 12. Several Member States are implementing the Recommendation on Upskilling Pathways in close coordination with the development of arrangements for validation. as a way to provide low skilled adults with credit for the skills they have, as a possible bridge to further learning opportunities. This is often driven by the need to upskill workers who have no vocational qualifications, or low skilled unemployed people who public employment services are seeking to return to the labour market. 3.2.9. In 2018, mandatory professional requirements for validation staff – counsellors, practitioners and assessors involved in validation – were reported in 23 Member States. In many countries competence development opportunities are not specifically targeting validation staff. Dedicated training for validation staff is available in EL, NL, PT and is planned in DK and IT. 3.2.10. In 2018, 18 Member States explicitly report that validation arrangements are developed and implemented through multi-stakeholder cooperation. However, analysis of country reports shows a much more frequent participation of a variety of stakeholders in one or the other phase of development and operation of validation arrangements or in activities supporting validation. For instance, labour market actors such as chambers of commerce, employer organisations and trade unions often contribute to standard definition and skills assessment. Public employment services and youth organisations are frequently involved as providers of relevant information and guidance. Interviews to national experts suggest that cooperation of a variety of stakeholders also appears to work as a major factor influencing the efficient of validation provision, facilitating a balanced distribution of costs and burdens. 3.2.11. As asked by the Recommendation, the EQF advisory group has taken over the role of policy group following the implementation. Validation issues are discussed in meetings of the EQF advisory group, including in particular the update of the European guidelines for validating non-formal and informal learning. Within the EQF advisory group, ten Member States volunteered to present one-off reports on validation, based on an agreed structure. Following the adoption of the Recommendation, the Commission has asked Member States to designate each one member and an alternate to this purpose (or assign this task to the member and alternate already following the EQF Recommendation) and has invited new stakeholders’ group to join the group, namely the European Youth Forum and the European Volunteering Centre. The EQF advisory group started meeting in this configuration in its 21st meeting (26-27 September 2013) and validation has been on the agenda in all 30 meetings since then (until November 2019). An extraordinary meeting dedicated to validation took place on 5 May 2019 in Berlin, adjoining the third Validation Biennale, a relevant event organised by third parties. Within the EQF advisory group, ten Member States volunteered to present one-off reports on validation, based on an agreed structure.

3.2.12. In the context of the EQF advisory group, ten peer learning activities specifically addressing validation issues have been organised: on non-formal qualifications (2013, SE), peer learning activities on learning outcomes and validation (2014, IE), on validation between individual pathways and collective strategies (2016, FR), on validation and refugees (2016, NL), on non-governmental stakeholders and the Recommendation (2017, PT), on funding validation (2017, BE), on non-formal qualifications in NQFs (2018, AT), on skills assessment and validation (2018, DE), on validation and volunteering (2019, BE), on peer review and validation (2019, LT). 3.2.13. In 2015 the Commission and Cedefop, following intensive consultation with national authorities and stakeholders, released a revised version of the European guidelines for validation of non-formal and informal learning, fully aligned with the Recommendation. The European guidelines have proven a major instrument to support national validation developments, confirmed by national experts interviewed and suggested by the impressive number of downloads from Cedefop’s website. Three updates of the European Inventory of validation non-formal and informal learning were produced, in 2014, 2016 and 2018. 3.2.14. The 2015 Joint Report by the Council and the Commission on the ‘Education and Training 2020’ strategic framework mentioned the need to reinforce work on validation, as well as on transparency and comparability of qualifications. It specifically made the link to incoming migration and to the innovative learning methods provided by digital technologies. It didn’t however provide any information on the implementation of the Recommendation, as Member States had been asked to take action by 2018. Reporting has rather been provided to authorities and stakeholders through the Inventory (updates of 2014, 2016, 2018) and other documents, such as the regular updates on developments of national qualifications frameworks published by Cedefop. 3.2.15. The Europass framework, for which the Recommendation suggested to explore further developments, has been radically revised in the framework of the Skills Agenda. A 2016 Commission proposal became in 2018 a Decision of the European Parliament and the Council, establishing a common framework for the provision of better services for skills and qualifications (Europass). The decision states in its Article 3 that the Europass online platform shall provide available information or links to available information on seven topics, one of which is “opportunities for validation of non-formal and informal learning” and that it will make available web-based tools for documenting and assessing skills. 3.2.16. The European Social Fund (ESF) has been a major source of funding for national validation activities, proving particularly effective in supporting wider developments rather than smaller projects. In particular, ten Member States with validation arrangements at an early stage have used ESF resources. Three of them (CY, EL, PL) explicitly relate these ESF-funded activities to the Recommendation. ESF support was however also used to improve validation systems already established, for instance in PT and BE (French community). In most Member States there has been some validation work supported through the ESF, either in projects explicitly focusing on validation or in wider skills development activities. The Erasmus+ programme (2014–2020) has supported EU level activities (Key Action 3, Support for policy reform), such as the peer learning activities on validation issues mentioned above, and a number of strategic partnerships (Key Action 2, Cooperation for Innovation and Exchange of Good Practices). While validation is a complementary issue in many partnerships with another focus throughout all sub-sectors of education and training, it is the core theme in at least 50 projects. Specific objectives of these partnerships include the recognition of skills acquired through volunteering, the validation of skills of specific groups (migrants, prisoners, Roma), and the validation of skills in specific economic sectors such as tourism, construction, wellness, public administration, and maritime. Since 2016, EU resources also became available through the Structural Reform Support Programme (SRSP). The SRSP has supported the implementation of a national qualifications framework (BG) and is currently supporting three Member States (BE, ES, NL) to improve their skills validation systems. Other relevant projects include raising the quality of adult education systems (HU, PT) and fostering excellence and labour market relevance in vocational education and training (ES, EL, FR, LU). 3.2.17. Only a few Member States (EL, ES, IT, SE) are able to provide information on whether validation beneficiaries find it easier to enter and move within the labour market. This very limited picture is positive. For instance, almost two thirds of respondents to an Italian survey on beneficiaries reported labour market progress following validation. Validation leading to specific professional accreditation (e.g. in security in EL and personal care in ES) was also reported to result in jobs. 3.2.18. There is little factual information on how validation beneficiaries have engaged in further learning. Sometimes validation processes are indeed embedded in upskilling initiatives run by public instances (e.g. in BE, IT and PT) or social partners (as in DK, FI, IE, SE). While in a majority of countries validation is a way that people may use specifically to access further learning (cf. 3.2.5), there is almost no data on whether a positive validation experience makes people more motivated to go further on their learning pathway. 3.2.19. In many Member States the costs of validation are not identified, as funding its public provision is part of funding the education and training system or employment policy measures. This is true for countries with well-established systems such as Denmark, Finland and Sweden, and for many countries where provision and related costs are more decentralised. A few countries (BE, ES, FR, IT, LU, NL, RO) do have specific budget items for validation. Two of them (Belgium and Italy) report developments inspired by the Recommendation. A proper cost-benefit study was carried out in Sweden, finding evidence of long-term benefits. As mentioned above, the ESF has provided the resources for many validation activities, especially the development of initial arrangements. In several countries, use of ESF resources was related to developments in line with the Recommendation, or directly inspired by it (Cyprus, Greece and Poland). Several representatives of national authorities have reported that validation benefits exceed its costs, sometimes referring to specific analyses. In some countries, distribution of costs over the actors seems unbalanced. The main factors influencing the efficiency of validation have been identified in multi-stakeholder cooperation and effective targeting. A partnership of many actors allows a more balanced distribution of costs and a fairer allocation of burdens. The improved dialogue triggered by the Recommendation has for instance helped mainstreaming resources in Finland and debate sustainability in Sweden. Inappropriate targeting of measures addressing specific groups has resulted in fragmented management and short-term planning, with limited results. 4.Method Short description of methodology This evaluation was organised following the Better Regulation Guidelines, addressing the five criteria of effectiveness, efficiency, coherence, relevance and EU added value. An external study supporting the evaluation was carried out. The study included desk research, field research, including a public consultation, and the development of conclusions and lessons learnt. The methodology is described in Annex 3. Limitations and robustness of findings The European Inventory on validation and its updates are the main source of information on the situation of validation in Member States. Country reports are prepared by external consultants and proofread by representatives of relevant national authorities (since the 2014 update this is done by members of the EQF advisory group). While those consultants retain their responsibility for the deliverables sent to the Commission, namely for their analytical dimension, national representatives check the accuracy and may clarify misunderstandings, e.g. in relation to the legal framework for validation, ensuring a good quality of the information on national validation activities. From one update to the next, information collection has become more structured and has gone into more detail, also with reference to the Recommendation. While this may require some adjustments, it remains to a large extent possible to compare over the years information on the main categories, such as the availability of validation arrangements, the provision of guidance, the link to qualifications frameworks and to institutional education and training. The Inventory update of 2010 and to some extent the update of 2014 do represent a solid baseline for the evaluation of a Recommendation adopted in 2012. The 2018 update was conceived with the explicit purpose of gathering a robust knowledge base supporting the evaluation of the Recommendation. Further reliable information sources are also available. The one-off reports that some Member States have presented to the EQF advisory group are accurate and up to date. The study on skills audits published early in 2019 provides a detailed picture of the situation in all Member States in 2018. Other useful literature is also available on relevant EU initiatives and national developments, such us the regular reports on national qualifications framework prepared by Cedefop. Input from interviews, expert meetings and the public consultation have allowed to complete and clarify the information available in the Inventory and other literature, supporting a cross-reference analysis. Input has sometimes complemented limited information, e.g. representatives of authorities in several countries have declared that in their country the benefits of validation exceed its costs – an issue on which secondary data are scarce. In other cases, input has corroborated findings from desk research; for instance, results from the public consultation show that some validation opportunities are available in most Member States, but their provision is often limited and typically not available to everybody, thus confirming the picture provided by the 2018 update of the Inventory. While the overview of validation in Member States provided by the European Inventory is a good information source on most relevant issues, there are however some objective limitations as concerns the availability and comparability of some categories of information. In spite of progress, data on the take-up of validation opportunities and the costs of validation remain limited. This makes it difficult to fully assess the effectiveness of the Recommendation, in terms of increased number of people benefitting from validation opportunities, and its efficiency, in terms of proportionality of costs compared to the benefits brought to individuals, organisations and society. While in the 2018 update of the Inventory most Member States reported some information on the take-up of validation arrangements, data are not comprehensive. They are typically collected on some but not all subsectors in which validation arrangements are in place. For instance, they are collected for candidates to lower level vocational training qualifications, but not for people seeking exemption from parts of programmes in higher education or higher vocational training. Besides, as data were mostly not available before the adoption of the Recommendation, comparison is not possible or not reliable. Information about increase or decrease in take-up by potential beneficiaries is contained in the country reports of the 2018 update of the Inventory, but it either refers to comparison 2016 or to an assessment based on partial and local data. However, available data and triangulation with interviews and the public consultation make it possible to formulate well-grounded assessments. Clear information on the costs of validation is limited as such costs are mostly part of wider cost items related to education or employment policy. While public funding is the most common resource for validation, it often is associated with other sources, from individuals that pay a fee to companies and private organisations to specific projects. Measuring the cost of validation is objectively difficult as it is a process where several actors may be involved, such as dedicated validation services, information and guidance centres, education and training institutions, Chambers of commerce and non-governmental organisations.

These limitations contribute to make it difficult to assess the direct impact of the Recommendation on the overall positive evolution. Relevant national documents rarely refer to the Recommendation. In many cases national efforts that implement the provision of the Recommendation may also be seen as the continuation of own national strategies and initiatives. However, interviews with representatives of national authorities did result in some explicit statements on the impact of the Recommendation. The public consultation has also contributed to gather views on the role of the Recommendation in increasing validation opportunities. A reasonably valid assessment of the impact of the Recommendation is therefore possible. 5.Analysis and answers to the evaluation questions 5.1 Effectiveness Evaluation question 1 on achieving the objectives of the Recommendation (Effectiveness) Question 1.1 – Progress. To what extent has there been progress towards the state of affairs described by the two statements below? 1.1.a Individuals have easy access to opportunities to have their skills validated; 1.1.b Individuals can use the results of validation to learn or work in Europe. Question 1.2 – Contribution. To what extent has the adoption of the Recommendation contributed to any such progress? The information reported in section 3 shows that there has been some progress in making it easier for people to access validation opportunities (evaluation question 1.1.a) and in making it possible to use validation results to learn and study across Europe (evaluation question 1.1.b). It also shows that there still is ample room for further progress, namely in making validation opportunities available to any individual at any stage of life, along with appropriate support. Taking into account the non-binding nature of the Recommendation, the action taken by Member States shows a good level of effectiveness, though more and better action is needed to achieve the objectives of the Recommendation. 1.1.a – People have more validation opportunities than in 2012 because Member States have responded to the first invitation of the Recommendation and set up or improved validation arrangements [3.2.1], taking advantage of the principles suggested in invitation 1.3 [3.2.3 to 3.2.9] and widening stakeholders’ cooperation [3.2.10], as called for by invitations 1.4 and 1.5. However, the relative fragmentation of provision, restricted access, and lack of support still prevent many people in taking advantage of validation opportunities [3.2.2]. Figure 5.1.1 Validation arrangements in place in 2010 and 2018 Source: Inventory 2010 and 2018. External study, Figure 3. While 132 of 160 organisations responding to the public consultation (83%) considered that validation opportunities were available in their country, only 54 of 65 EU individual respondents shared this view and 30 individuals believed they were not available. This suggests that fragmented provision of opportunities makes them less visible to individuals and confirms that information and guidance do not sufficiently reach out to potential users [cf. 3.2.7]. Besides, many respondents added a free text comment underlining the limitations of validation arrangements, with access to validation being limited to certain categories or made difficult by cumbersome procedures. The external study found that most validation arrangements are not comprehensive, all envisage some limitation to access and some are still quite limited in their scope [3.2.1]. On the one hand, this is in line with the pragmatic approach of the Recommendation, which in its invitation 1.1 said that Member States could “prioritise certain areas and/or sectors”. On the other hand, it should be clear that action by 2018 was a first step towards validation available to everybody, which remains the objective and is far from being achieved. Some interviews and discussions in the expert group showed awareness that the Recommendation was “only the beginning of a longer-term approach to validation”. Most respondents to the public consultation (198 out of 257) agreed that people should be able to validate their skills in all cases, while 55 thought this should happen in particular cases. Progress has been real, but limited, also with reference to disadvantaged groups. In spite of an increase in specific opportunities targeting them, in practice for many people from such groups access remains difficult [3.2.8]. As reported in Section 3.2.2, more people have used a validation service, though not as many as could be expected, and in a few countries, use of validation has actually slightly decreased. Individuals who might benefit from validation may not use it because they lack the necessary support. This is confirmed by responses to the public consultation: 23 of 62 beneficiaries of validation were satisfied with the guidance during the validation process, 17 were not satisfied and 15 had not had any, while financial support had only been available to seven of 62 responding beneficiaries [3.2.2, Figure 3.2-3]. Figure 5.1-2. Responses to public consultation question 15 (individuals). Were you guided and supported during the validation process?

External study, Figure A.3.17. The increase in opportunities and, however limited, in their use has also taken advantage of the now widely shared understanding that validation is a process including different stages [3.2.6], each of which has its own value and should be separately available to people (invitation 1.2). This seems to be a frequent case, as for instance validation initiatives in the third sector [3.2.1], skills audits for the unemployed, and initiatives targeting specific groups [3.2.8] do not go beyond identification and documentation. The public consultation confirms this: on the one hand most responding organisations were well aware of the four stages, with more than 100 out of 161 considering each phase was reasonably accessible in their country; on the other hand only 14 of 64 responding beneficiaries had gone through the whole process and received a qualification, while others didn’t go beyond identification (15), documentation (7) or some form of assessment (11). Figure 5.1-3. Responses to public consultation question 4 (organisations). To what extent can people in your country find skills identification (I), documentation (D), assessment (A) and certification(C) ? External study, Figure A.3.6. Figure 5.1-4. Responses to public consultation question 16 (individuals). What steps did your validation experience include? External study, Figure A.3.20. I = Identification of skills

D = Documentation of skills

A = Assessment

C = certification More relevant actors are playing a role in validation [3.2.10], and this may have contributed to make it easier for some people to access validation. Out of 160 organisations responding to the public consultation, 114 considered that all relevant parties were involved in national validation developments to a high (51) or some extent (63). On the other hand, the fact that most validation arrangements are not comprehensive and many people do not have access to validation or do not take advantage of it [3.2.2], suggests that more strategic coordination of actors, each of which may already be playing an active role in validation, would more effectively pursue the objectives of the Recommendation. 1.1.b – More people find it easier now than in 2012 to use their validation results across countries, namely when they obtain a full qualification through validation. In fact validation can now lead to qualifications included in the qualifications framework (invitation 1.1.b) in most Member States – though in only about half Member States validation can lead to any qualification in the framework [3.2.3]. This is facilitated by widespread equivalence of standards between validation and formal education and training, called for by invitation 1.3.h [3.2.4]. The figure below shows that, thanks to the close link between validation arrangements and national qualifications frameworks referenced to the EQF, implementing invitation 1.3.a, in 2018 in most countries people who obtain a qualification through validation will be able to use it to learn or work across Europe.

Figure 5.1–5 Validation and national qualifications frameworks in 2018 Source: External study, Figure 10. Inventory 2018. Synthesis Report, p.27. National qualifications frameworks are increasingly including qualifications awarded outside institutional education and training, sometimes called “non-formal” or “non-traditional” qualifications, such as the more and more common micro-credentials. Recent research has highlighted the need for formal education and training, credit systems and qualifications frameworks to take into account micro-credentials and related developments, which could have a disruptive impact, but could also “bring together the world of education and training and the world of work” and boost the flexibility of learning pathways. A more coordinated development and implementation of validation and qualifications frameworks might generalise validation as a pathway to all qualifications. 1.2 – The contribution of the Recommendation to the (partial) progress at national level seen above took several forms, as learned from interviews with national experts. In systems at an earlier stage of development, the Recommendation has helped shaping them (IT, MT). It has also supported existing systems by adding weight to national action (BE, ES), making the approach more comprehensive (SI), giving a new strategic direction (FI, SE) or making validation more visible at national level (CZ). It also influenced initiatives targeting specific groups (BE, IT, PT, SI). Evaluation question 2 on follow-up and support activities (Effectiveness) Question 2.1 – Follow-up. To what extent have the support actions (invitation 2) been taken by the Commission and Member States? Question 2.2 – Support. To what extent has the Commission implemented the actions called for in invitation 3? The information reported in section 3 [3.2.1 to 3.16] shows that the follow-up and support actions called for in invitations 2 and 3 have been implemented. The only exception concerns invitation 2.b on using the Education and Training 2020 Joint Report, cf. next paragraph. 2.1 – The follow-up activities envisaged by the second invitation of the Recommendation have been mostly implemented. The EQF advisory group had become the validation policy group [3.2.11] and Cedefop has provided a major contribution to the follow-up, including through its reporting activity. The 2015 Joint Report by the Council and the Commission on the ‘Education and Training 2020’ strategic framework, prepared less than three years after the adoption of the Recommendation, has not proven to be an effective reporting tool for validation[3.2.14]. 2.2 – Some contribution to the above mentioned progress may have come from the support activities envisaged in invitation 3, which have all been implemented as seen in sections [3.2.12, 13, 15, 16], mostly covered by the Erasmus+ budget. National efforts to widen validation opportunities have taken advantage in particular of the European guidelines [3.2.13]. Some suggestions for improvements were collected. For instance, peer learning activities could become more effective through closer coordination with similar activities related to relevant initiatives and with a higher profile. Evaluation question 3 on the impact of the Recommendation (Effectiveness) Question 3 – Impact. To what extent, where progress has been identified, is it possible to state that individuals have benefitted from validation, and in particular, 3.1 Find it easier to enter and move within the labour market; 3.2Engage in learning opportunities throughout their career. At this stage, the limited information available does not allow to draw any well-grounded conclusion on the impact of validation on its beneficiaries: whether validation helps them finding a job or a better job, and whether they engage in further learning. Only a couple of countries carried out targeted surveys on validation beneficiaries, which suggested a positive impact [3.2.17 and 3.2.18], but they cannot support an answer to this question at European level. As data on uptake in general remains rather limited, a systematic follow up of validation beneficiaries is at this stage not possible. 5.2 Efficiency Evaluation question 4 on efficiency Question 4.1 – Costs and benefits. What are the costs (such as expenditure and administrative burden) and the benefits associated with the implementation of the Recommendation for the different stakeholders, at local, national and EU level? Question 4.2 – Factors. What factors influenced the efficiency with which the results identified were achieved? What factors may have caused cost/benefit differences between Member States? What good or bad practices can be identified? Question 4.3 – Proportionality of costs. To what extent are the costs of the actions suggested by Recommendation proportionate to the benefits brought to individuals, economy and society? There is not enough information on costs and benefits directly related to the Recommendation (or indeed to validation activities themselves) to be able to compare them or to assess the proportionality of costs. Interviews and the public consultation express positive views, but don’t represent solid evidence. Stakeholder cooperation is recognised as an efficiency factor. Considerations of the cost-effectiveness of the Recommendation depend on a very thin information basis (cf. section 4.2). It is very difficult to identify the cost of validation as separate from costs of wider education or labour action, and detecting costs determined by the Recommendation, and therefore reviewing their proportionality, is not feasible. As there is a certain consensus that some national action might not have been taken without the Recommendation, this suggests that costs may have been higher. However, any such increase in cost cannot be identified, as the stimulus to action due to the Recommendation typically had an impact on existing systems and ongoing initiatives. Besides, only a few countries have dedicated funding frameworks for validation, which is often covered through wider budgetary items such as education or employment. In several countries early-stage arrangements have developed into better established systems, often supported by ESF, which in the long run should result in better cost-effectiveness. Information on benefits is also very limited, as seen above [3.2.17 and 3.2.18]. The qualitative judgement of national representatives, as gathered in the external study through interviews, is that validation benefits exceed its costs [3.2.19], although this refers to validation rather than to the Recommendation. A positive view on proportionality of cost, with reference to the Recommendation, was expressed by more than half organisations responding to the public consultation (86 out of 160); however most others (57, more than one third) did not feel able to reply, confirming the lack of solid information. When it comes to factors influencing efficiency, a positive result is that the Recommendation “has been reported to contribute to rationalising the use of financial and other resources in the development and provision of validation”, therefore increasing the efficiency of validation activities in Member States. To a certain extent, this happened because the Recommendation inspired comprehensive stakeholder partnerships, which appear to be a significant efficiency factor [3.2.10]. 5.3 Relevance Evaluation question 5 on relevance Question 5.1 – Relevance of objectives. To what extent are the objectives relevant in the current policy context? Do they address current needs? Question 5.2 – Relevance of measures. To what extent are these measures still relevant to achieve the objectives? Question 5.3 – Relevance of governance and support. To what extent are the provisions on governance and support still relevant to support the achievement of the objectives?

There is wide consensus among national authorities and stakeholders that the Recommendation remains relevant in today’s social and economic conditions. Specific measures may need adjustments, wider representation of stakeholders could make governance more relevant, and it would be helpful to include provisions for a solid evidence base. 5.1 – The objectives and principles of the Recommendation provide a useful contribution to addressing the rapidly evolving skills challenges. In several Member States the Recommendation has promoted national action and its potential has been highlighted with particular reference to the skills needs related to digitalisation and innovation of work practices. The Recommendation is seen as responding to the needs of the different stakeholders, although its high-level provisions may more easily support policy stakeholders than practitioners. Two thirds of the organisations responding to the public consultation (103 out of 156) agreed that it appropriately addressed their needs, though one in five (32) was not able to reply. Figure 5.3-1. Responses to public consultation question 10 (organisations). To what extent do you think the Council Recommendation appropriately addresses the needs of your organisation? External study, Figure A.3.12. The call for involvement of all relevant stakeholders was often highlighted and considered to deserve stronger emphasis at EU level. More guidance on how to put in place wide ranging stakeholders’ partnerships, building on the 2015 European guidelines for validating non-formal and informal learning, would be beneficial. 5.2 – Specific measures and principles are also mostly judged as still relevant. The flexible approach to the four stages of the validation process fits with the need for both national adaptations and individual pathways, although further guidelines might prove helpful. The link to national qualifications frameworks retains its importance, though developments could better take into account the increasingly rapid evolution of skills and occupational standards. Interviewees from several Member States have noted have noted the relevance of the Recommendation with reference to the inclusion of disadvantaged groups, including migrants. There was, however, some concern that so far disadvantaged groups had not been among the main beneficiaries of the Recommendation, probably because of inadequate outreach. It is possible to achieve higher relevance to the needs of disadvantaged groups through close coordination with interventions from other policy fields. While the four stages approach has been an effective factor promoting a shared understanding of validation throughout Europe [3.2.6], the meaning of the assessment and the certification phases might need some reflection to remain universally relevant in a changing context. Emerging practices in skills recognition, such as micro-credentials and other digital methods, may require a re-definition of the certification phase. The equivalence of standards, traditionally focused on formal education and training and their qualifications, may also need some reflection (cf. above, section 5.1). 5.3 – The relevance of follow-up and support measures, while acknowledged, could improve, as suggested by some interviewees and participants in the expert meetings, by involving a wider range of stakeholders (for instance through validation events) and ensuring a solid information basis that would convert the Inventory into an instrument for actual monitoring as is usually carried out in the framework of the open method of coordination. 5.4 Coherence Evaluation question 6 on coherence To what extent is the Recommendation coherent with other European policy initiatives and developments in related instruments? The Recommendation is coherent with European policies and tools in related fields. Some of its provisions, while conceptually coherent, may be less coherent with specific tools implemented nationally. The Recommendation is coherent with national validation policies, although some of its provisions are not fully implemented. Most national experts interviewed recognise the conceptual and thematic coherence of the Recommendation with other EU policy areas and instruments. The principle of learning outcomes is the red thread that ensures consistency across relevant EU policies and tools. While there is clear coherence with major tools such as the EQF and recent initiatives such as Upskilling Pathways, some national experts noted an insufficient practical and organisational coherence with specific policy tools. In particular, the synergies between validation and credit systems such as ECTS and ECVET, specifically called for in the Recommendation, do not seem to have been achieved beyond a few cases. While the 2018 Inventory reports the in most Member States validation can result in credit [3.2.5], the external study notes that there it is unclear how this relates to the use of ECTS and ECVET. Most Member States implement ECVET as an instrument to support flexible vocational pathways and rarely as a credit point system, which reduces the coherence of the Recommendation with this tool. On the other hand, use of ECTS is standard practice in higher education throughout Europe, but link to validation is equally not clear in many countries. The links between validation and recognition of small sets of skills is an issue that deserves further development, also in relation to innovative practices of skills recognition such as micro-credentials, which are often linked to short courses, but may also have an impact on validation activities. There is wide consensus among interviewees from Member States that the measures of the Recommendation are consistent both with each other and with national validation policies. This is particularly apparent in countries that adapted their relevant legal framework or strategies. General consistency with national policies coexists with different levels of practical implementation of the principles of the Recommendation. For instance, as seen above [3.2.1] access to validation, while not comprehensive in any country, is more or less wide depending on the scope of validation arrangements. Stakeholders who noted some lack of coherence referred indeed to the gap between the limited access made available by national action and the general access promoted by the Recommendation. 5.5 EU added value Evaluation question 7 on the EU added value Could the objectives of the Recommendation have been achieved sufficiently by the Member States acting alone? In particular, to what extent could the main findings (results/outputs) identified have been achieved without EU intervention? Were there benefits in replacing different national policy approaches with a more homogenous policy approach? To what extent are national validation arrangements converging? To what extent do the issues addressed by the Recommendation continue to require action at EU level? There is a certain consensus that in several countries the Recommendation has had a positive impact on national action towards its objectives. Appropriate public discourse would support the more common understanding of validation that is emerging throughout national policies and arrangements. Experts from about half Member States considered that further EU action in the field is necessary or appropriate, and could gain if it were more specific and closer to the operational level.

Interviews with national experts suggest that the Recommendation has had a certain positive impact on national action in a number of Member States, to different extents and in different forms. In countries where validation arrangements in 2012 were less developed, the Recommendation has been beneficial contributing to the policy debate (e.g. inter-ministerial discussions on validation in Slovakia) and to actual development (e.g. supporting the legislative process in Italy). Representatives from countries with better established arrangements also recognise that “the CR has given some impetus to already existing national validation strategies or actions”, or that the Recommendation has had a positive impact in shaping strategic choices. A huge majority of organisations responding to the public consultation (100 out of 160) agreed that the Recommendation had played a role in national action towards more and better validation opportunities and slightly less (91 out of 160) that it had enabled individuals to progress on their learning pathway or career. Free-text comments highlighted the role of the Recommendation in fostering the debate on validation and stimulating developments. Figure 5.5-1. Responses to public consultation question 12 (organisations). To what extent do you think the Council Recommendation has contributed to: 160 respondents External study, Figure A.3.14. A slightly smaller proportion of responding organisations (95 out of 160 or 60%) considered that the Recommendation had indeed contributed to making validation more available to people. National arrangements are largely conceived and practised “in accordance with national circumstances and specificities” (invitation 1 of the Recommendation), and the Recommendation does not aim at imposing one specific validation approach to the diverse national policies. However, a number of interviewed national officials and participants in expert meetings recognise that its principles have promoted a more common understanding of validation across Europe. The widespread view of validation as a four stage process mentioned above is a clear indication of this positive trend towards a shared understanding, which will facilitate people to use their validation results across countries. Documentary sources suggest however that in a few countries the concept of validation needs further clarification. Free-text comments of organisations responding to the public consultation note that more public discourse on validation would facilitate a common vision within and across countries. During interviews, 11 key informants also reported a certain impact on national policies in areas other than validation. Which specific areas varies between countries, with seven however agreeing on some impact on career guidance. Key informants from 11 countries, along with EU level key informants, considered continued EU action necessary to achieve the objectives agreed, while for five others EU action could play a role in promoting comprehensive national systems. Several experts interviewed noted that EU action could better support national action if provisions were less generic, for instance better defining target groups or more closely linking to the operational European guidelines. 6.Conclusions and lessons learned 6.1 Conclusions The ambitious objectives of the Recommendation have not been fully achieved, but there is evidence of significant – albeit fragmented – progress since 2012. All Member States have taken some relevant action by 2018, using to a large extent the principles suggested, although this has not yet resulted in systems open to everybody. The follow up and support activities carried out were appreciated by stakeholders, who nevertheless identified the need for further improvement and made suggestions for how to achieve this. The Recommendation called for stakeholders’ coordination and in cases where this has been put into practice, it has indeed proven to be a factor making national action more efficient. The objectives and measures of the Recommendation are considered still fully relevant in the current socio-economic situation. Their general coherence with the policy context is also acknowledged, although coordination with policy tools at operational level could improve. The EU added value is most visible in the higher policy profile and increased practice of validation observed to different degrees in all Member States, in some of which national action alone would not have reached the same extent. The evaluation has identified areas where further action is needed to achieve the goals of the Recommendation, to provide people with access to more and better validation opportunities, enabling them to access further learning and to put their skills to good use in European society and the labour market. Achievements Most countries’ arrangements are still far from being comprehensive, but there has been progress towards providing more validation opportunities. Some degree of validation arrangements have been set up or strengthened in almost all Member States, including by actors in the labour market and the third sector. To some extent, validation has become more accessible, and based on the limited available data, it appears that take-up by people has increased to some extent. The Recommendation is recognised to have contributed to this progress, by raising the profile of validation in the policy debate and in some countries stimulating action that might not have been taken otherwise. The four stages of validation are acknowledged to be a sound common reference framework, applied with terminological variants, and Member States have to a large extent adopted the principles suggested in the Recommendation when developing validation arrangements: there are clear links to national qualifications frameworks, now available across Europe, there is more information and guidance, many initiatives target disadvantaged groups, quality assurance has improved and equivalence of standards with formal education is common, transparency tools and formal credit systems are used to some extent in connection to validation. Challenges In spite of progress, most validation arrangements have limitations to access. In many countries, young people with no work experience cannot apply for validation that leads to full or partial qualifications. Validation can help them access formal education, but not the labour market. The Recommendation does not call for centralised, unified national validation systems, but does call for arrangements that “enable individuals” (with no qualifier) to have their skills validated. In action taken by 2018, Member States have prioritised certain sectors or areas. This was a realistic approach in 2012, as such pragmatically envisaged by the Recommendation, but it inevitably excludes some categories of potential users. While data remain limited, they suggest that fewer people than expected have engaged in validation, despite larger provision of opportunities and guidance. There may be several reasons. Of course, limiting access as mentioned above reduces the number of eligible individuals. Even for them, in practice access can be difficult because validation opportunities are provided by a variety of agencies with little coordination. Guidance, while available, may not have been effective in reaching out to potential beneficiaries. Finally, if there is no active support to individuals, such as paid leave or a financial contribution, they may not be able to engage in validation.

Reaching specific disadvantaged groups, such as long-term unemployed or migrants, remains a challenge, despite being the focus of an increasing number of initiatives. While data on their outcomes are limited, impact is often reduced by ineffective reach out, procedural complexity and uncoordinated provision on the one hand, and on the other hand, the high level of individual commitment required. This often results in inefficiencies, with potentially good initiatives not fully exploited. Scarcity of information is an issue through all aspects of validation provision. While there is good information on the structure of validation arrangements and the related policy and legal frameworks, there is little and fragmented information on their operation, costs, outcomes and impact. There are occasional surveys on beneficiaries and organisations involved, but no systematic tracking. The information available is not enough for any proper monitoring, and benchmarking could only be based on a few qualitative indicators with limited value, such as the existence or not of a specific legal framework. 6.2 Lessons learned Providing more validation opportunities is not enough. Providing support to individuals is necessary. A core principle of the Recommendation, and of European skills policy, is that everybody should have opportunities to validate their skills. In 2012 it was pragmatic to focus on providing “more opportunities”; in 2020 it is necessary to open up opportunities and support individuals to actually take advantage of them. People who are aware of the benefits of validation and of available opportunities may nevertheless wonder whether the benefits are worth the costs. They need to be motivated enough to invest time, energy and often money over a sustained period, all at the expense of work and family. To increase their motivation, investment should be shared with employers (e.g. paid leave, technical assistance), authorities (e.g. financial support, coaching, simplified procedures), and other actors such as unions, trade organisations, and education and training providers. Specific support should address the barriers limiting the access of individuals from disadvantaged groups. Stakeholders’ cooperation is not enough. There is a need for strategic coordination. While a variety of agencies providing validation can be a richness, responding to the needs of a diverse set of beneficiaries, a closer coordination of providers and cooperation with all stakeholders could improve both effectiveness – through better visibility, wider reach out, operational synergies – and efficiency, by distributing burdens, sharing facilities and peer learning. The Recommendation does stress stakeholders’ involvement and coordination (points 4 and 5). However, while many individual validation initiatives reach a good level of stakeholders’ cooperation, coordination is weaker between validation initiatives and between validation and other areas of skills policy. Providing more guidance is not enough. Closer cooperation and effective coordination between providers of guidance and validation is needed. While relevant guidance should be “made available to all individuals and organisations”, as the Recommendation says, in any time and place there is a need to tailor the media and the message to the audience: to reach out to them, to convey the right meaning. For instance, to illustrate to vulnerable workers the benefits of validation with a view to upskilling, and guiding them to the most suitable validation pathway. This would promote take-up in general, and in particular improve effective tailoring of validation initiatives addressing disadvantaged and vulnerable groups, making them more effective and efficient, in coordination with the implementation of the Recommendation on Upskilling Pathways. A formal link to national qualifications frameworks is not enough. Coordinated implementation of validation and national qualifications frameworks is needed. The emergence of qualifications frameworks and their alignment to the EQF, with their central concept of learning outcomes independent from learning settings and inputs, has been a major factor in promoting validation action in most Member States. While well-established in the Recommendation, and effectively pursued through the EQF advisory group, the link between validation and qualifications frameworks needs to develop in its practical implementation. A link to formal credit and qualifications is not enough. Validation should take advantage of innovative practices in skills recognition, such as micro-credentials. As micro-credentials recognise a small or very small set of skills, they can be a useful instrument to validate skills developed outside formal education and training. If they are developed and awarded respecting agreed standards of quality assurance and transparency, micro-credentials could make learning pathways more flexible. Appropriate developments, closely coordinated with qualifications frameworks, could build upon the experience gained through the implementation of ECVET and national experience with partial qualifications. The current level of information is not enough. Systematic information collection on validation is necessary. Validation has grown in all Member States in terms of policy significance and operational developments. Still, the poor and fragmented information available makes it difficult to meaningfully assess the relevance of policy measures, the effectiveness of their implementation, and the proportionality of investments. The Recommendation calls for updating the European Inventory of validation, which is recognised as a valuable tool, but does not provide for any systematic information collection at national level that could support the quality, accuracy and relevance of the Inventory. With a proper information basis, it would be possible to agree on indicators and monitor progress. Annex 1: Procedural information 1.Lead DG, Decide Planning/CWP references The evaluation of the Recommendation was led by the Directorate-General for Employment, Inclusion and Social Affairs. 2.Organisation and timing An evaluation roadmap published on 16 October 2018 explained the context, purpose and scope of the evaluation and informed stakeholders that an external evaluation study would be carried out, building upon the 2018 update of the European Inventory of validation and information provided by national authorities within the EQF advisory group, and that a public consultation would be launched. An Inter-service Steering Group (ISSG) was set up, coordinated by DG Directorate-General for Employment, Inclusion and Social Affairs (represented by staff from the lead policy unit, another policy unit and the evaluation unit) and including members from the Secretariat General, Education and Culture, the Joint Research Centre. The ISSG could take advantage of the technical assistance of expert staff from Cedefop and the European Training Foundation (ETF). The ISSG also acted as steering group for the external study supporting the evaluation, which was carried out by ICF Consulting Services. Contract VC/2019/120 was awarded after reopening of competition (tender VT/2018/035) within the multiple framework contract “Provision of services related to the implementation of Better Regulation Guidelines” (VT/2016/027). The contract VC/2019/120 started on 1st April 2019 and lasted eight months, until 31 December 2019. The draft final report was received on 4 December 2019, the inter-service steering group provide comments by 20 December 2019 and the contractor delivered the final report on 20 January 2020, along with the agreed annexes on the stakeholders’ consultation The external support study had been announced in the evaluation roadmap published on 16 October 2018. It included a public consultation, which remained open 13 weeks (7 August 2019 to 13 November 2019).

The preparation of the evaluation was discussed with the EQF advisory group in its meetings of 2nd October 2018, and interim results were presented in the meeting of 4-5 November 2019. 3.Exceptions to the better regulation guidelines All Better Regulation requirements were fulfilled. 4.Consultation of the RSB (if applicable) Not applicable. 5.Evidence, sources and quality A specific study supporting the evaluation of the Recommendation was carried out in 2019. The final report is available here . It included a public consultation (7 August to 13 November 2019), which resulted in 261 responses. The contractor also organised two expert workshops, focusing on validation in the labour market and on validation and qualifications frameworks. The main sources of information were: –the European inventory of validation non-formal and informal learning. The 2010 update, and to a lesser extent the 2014 update, provided the baseline information, while the 2018 update, finalised in summer 2019, provided information updated to the end of 2018; –interviews with 72 experts from national government bodies, qualifications authorities, learning providers, validation practitioners, and stakeholders’ associations; –literature from research and academia. Complementary information included a series of reports that national representatives of 6 countries (Germany, Latvia, Portugal, Luxembourg, Denmark, UK) have presented to the EQF Advisory Group over 2019 (one-off reports on validation). The present Staff Working Document relies upon the final report of the specific study as well as on direct consultation of the European inventory of validation of non-formal and informal learning, the national one-off reports on validation. Information acquired in peer learning activities of the EQF advisory group as well as through the ordinary coordination of the EQF advisory group. Available literature was equally consulted, in particular publications from the OECD. Annex 2: Stakeholder consultation Synopsis Report on stakeholder consultation activities In accordance with the Better Regulation Guidelines, this Annex provides a synopsis or summary of all the consultation activities conducted within the evaluation. The stakeholder consultation activities were carried out in the framework of the external study supporting the evaluation: ·Key Informant Interviews (KIIs) ·Expert group meetings ·Public consultation

1 - Key informant interviews (KIIs) In total, 72 KIIs were conducted. In some Member States, no KIIs could be conducted (DK, LV) while in several others fewer than two were completed (EE, LT, RO, UK). This was mostly due to lack of responsiveness or lack of detailed knowledge of the Recommendation among some of the targeted stakeholders. Table A2.1. Number of KIIs completed by Member State and at EU level Member State No. of KIIs completed Member State No. of KIIs completed Austria 2 Italy 4 Belgium 3 Latvia 0 Bulgaria 2 Lithuania 1 Croatia 3 Luxembourg 2 Cyprus 3 Malta 2 Czech Republic 3 Netherlands 4 Denmark 0 Poland 3 Estonia 1 Portugal 3 Finland 3 Romania 1 France 3 Slovakia 2 Germany 2 Slovenia 2 Greece 4 Spain 2 Hungary 2 Sweden 4 Ireland 5 United Kingdom 1 Number of EU-level KIIs completed 5 Total number of KIIs completed 72 The following table shows the type of stakeholders taking part in the KIIs for this evaluation study. Table A2.2 Overview of KIIs completed by stakeholder type Stakeholder type No. of KIIs completed Ministry of education representatives 21 No. of KIIs with EQF AG members 14 National VET agency representatives 12 Qualification authority representatives 11 validation organisation representatives 8 HEI and academia representatives 8 Chambers of commerce and crafts representatives 4 Labour market agency representatives 2 Ministry of labour representatives 1 EU umbrella organisation representatives 5 Total 72 The topic guide used to collect information from the KIIs included 25 questions on the five evaluation criteria, effectiveness (14), efficiency (3), relevance (3), coherence (2), EU added value (3). 2 - Expert group meetings Two Expert group meetings were held in Brussels to enable a reflection on the evaluation’s interim findings and to facilitate the exchange of experiences and observations among stakeholders the implementation of the CR (in different Member States and the EU) and on the topic of validation more generally. The two thematic meetings held were as follows: The role of employers and other labour market actors in validation arrangements Wednesday 13 November 2019 How validation arrangements relate to national qualifications frameworks (NQFs) and the shift to learning outcomes and flexible learning pathways Thursday 14 November 2019 A total of 15 participants included three from European umbrella organisations (the European Youth Forum and European Association for the Education of Adults) and 12 from ten countries (BE-Fr, CZ, ES, IE, IT, MT, PL, SE, SK and Norway), specifically from national ministries (2), national agencies for education and training (2), regional-level labour organisations (2), training and academic institutions (2), an organisation in charge of validation (3), and one from a business organisation. Below is a summary of the key messages from the Expert group meetings. The role of employers and other labour market actors in validation arrangements ·The CR is effective and relevant in helping Member States conceptualise validation and in creating momentum for multi-stakeholder collaboration involving employers and labour market actors. ·The Recommendation has generated considerable added value in those Member States where validation is still in its early stages of development. ·Limited evidence overall as to whether the Recommendation has enabled people to use their validation outcomes for entering the labour market and progressing within it. How validation arrangements relate to national qualifications frameworks (NQFs) and the shift to learning outcomes and flexible learning pathways ·The Recommendation has helped establish learning outcomes (knowledge/skills/competences) as a ‘currency’ for validation across the EU.

·The intended objectives of the Recommendation are restrained by the fragmentation of validation processes observed in many countries, compounded by the lack of a common vision among different stakeholder types in some cases. ·While the Recommendation has helped shape a more common understanding of validation across the EU, there is very limited evidence of validation outcomes being used for intra-EU mobility purposes. Overall conclusions ·The Recommendation is regarded as having continuously fed into discussions on validation in many Member States and has been used as for developing validation processes in those countries where they were mostly inexistent prior to 2012. ·Validation cannot operate separately and needs to be embedded in wider skills and lifelong learning strategies, which requires institutional change to facilitate multi-stakeholder collaboration. ·It may still be too early to assess the extent to which the Recommendation and other relevant EU-level instruments can be applied to improve links between validation and formal education systems, to develop flexible learning options as a gateway to validation and to facilitate EU mobility. ·Validation does have a bright future as it can be expected that skills will increasingly require updating to remain relevant to the rapid evolutions of the labour market. 3 - Public consultation The public consultation has served to gather views on the Council Recommendation from the wider community of experts and practitioners on validation on the one hand, and from people who have undergone a validation process. Certain questions were therefore only targeted at organisations acquainted with validation while some others were specifically targeted at individual end-users of validation. Following its translation into 22 other European languages, the public consultation was launched on 7 August 2019 and closed on 13 November 2019. It was disseminated to the relevant networks of DG EMPL with the targeted networks having been requested to disseminate the public consultation to their respective beneficiaries and partners. National ministry stakeholders taking part in the KIIs have also been asked to disseminate the public consultation to their relevant networks, while national-level validation and career guidance organisations have been encouraged to do likewise with their beneficiaries. The public consultation generated a total of 262 responses. In addition, ten organisations submitted a position paper together with their public consultation responses: seven at the national-level and three at the EU-level. Due to a few incomplete or ‘blank’ responses, sample sizes show small variation across the questions. To that end, sample sizes are provided below each chart for reference. Responses came from all Member States, plus Turkey (5), Albania, Armenia, Morocco North Macedonia, Norway, Pakistan and Switzerland (one each). Respondents from Italy were most numerous with 27 responses (10%), followed by 23 responses (9%) from Portugal and 18 (7%) from the United Kingdom. Cf. Figure A2/1 below. Figure A2.1 Overview of responses by country (External study, Figure A3.1) N=262, Source: public consultation results In terms of participant type, the 262 responses split unevenly between organisations (163 responses or 62%) and EU/non-EU citizens (99 responses or 38%). This sample size provides a good basis for analysis as both groups are sufficiently represented, whilst it also allows for further segmentation by type of organisation. When analysing the type of organisations in the sample, public authorities are most prevalent (44 responses), followed by NGOs (38 responses) and academic/research institutions (31 responses).

Table A2.3 Type of organisations in the sample (External study, Table A3.1) Type of organisation No. of responses Academic/research institution 31 Non-governmental organisation (NGO) 38 Public authority 44 Other 26 Company/business organisation 11 Business association 6 Trade union 7 Total 163 N=163, Source: public consultation results Questions for all respondents The first three public consultation questions were addressed to all respondents. Q1 - To your knowledge, are there possibilities for people in your country to validate the skills they have acquired outside school or university? Nearly three out of four respondents (72%) thought this was the case, 20% believed that such possibilities were not available, whilst 7% didn’t know. Rather unsurprisingly, a significantly larger share of representatives of organisations were aware of validation opportunities compared to citizens (28pp difference). Figure A2.2. Q1 - To your knowledge, are there possibilities for people in your country to validate the skills they have acquired outside school or university?

(External study, Figure A3.3) Source: public consultation results Well-established systems are well known: almost all respondents from France and Sweden believed that validation opportunities were available. A total of 88 open-text answers were received to further clarify and explain responses to Q1. Whilst the views are rather heterogeneous, most responses refer to the limited availability of validation arrangements. These are often provided through specific projects and/ or in relation to VET and hard skills. Put differently, well-developed and nationwide procedures are still seldom in place, thus further efforts are required according to the responses. Q2 - Do you think that people who acquired skills in the workplace or outside school should be able to have them validated? Nearly all respondents agreed, with 4 out of 5 selecting ‘Yes, in all cases’ and the others ‘Yes, but only in particular cases’. Two non-governmental organisations disagreed. Q3 - Have you personally participated in a programme to validate skills you acquired (through work, community groups, volunteering etc.) outside an education programme? One out of four respondents have participated in a programme to validate skills acquired outside an education programme as shown in the figure below. They were therefore invited to respond to questions . Questions for organisations Questions 4 to 13 were addressed to organisations. Q4 To what extent can people in your country obtain /identification/ documentation/ assessment/ certification/ of their skill? Some 60% of respondents believed in relation to all four stages that these can be obtained to a high extent or to some extent. A small minority (max 7%) believed there was no opportunity.

A total of 34 respondents provided further comments. Several comments describe hindering effects, including complex and lengthy validation processes, the limited possibilities for identification and documentation of skills, as well as limited awareness about the validation possibilities. As pointed out above, validation often covers only a set of professions and skills rather than being offered on a universal basis. Q5 Do you think that validation services in your country /consistently meet clear quality standards