Source: EURLEX
Language: en
Format: md

Table of contents

1.Introduction

Purpose and scope of the evaluation
   

2.What was the expected outcome of the quality framework of traineeships (qft)?

2.1 Description of the QFT and its objectives
   

2.2 Points of comparison
   

3.How has the situation evolved over the evaluation period?

4.Evaluation findings

4.1.To what extent was the QFT successful and why?

4.2.How did the QFT make a difference and to whom?

4.3.Is the QFT still relevant?

5.Conclusions and lessons learned

5.1 Conclusions

5.2 Lessons learned

Annex I: Procedural Information

Annex II. Methodology and Analytical models used

Annex III. Evaluation matrix

Annex IV. Overview of benefits and costs

Annex V. Stakeholders consultation - Synopsis report

Annex VI. Research questions of the evaluation study

Annex VII. The principles of the 2014 Quality Framework for Traineeships (QFT)

  

Table of figures

Figure 1 - QFT intervention logic
   

Figure 2 - EU-27 average unemployment, NEET and long-term unemployment rate (15-29 years old), 2008-2014
   

Figure 3 - EU27 average unemployment, NEET and youth long-term unemployment rate (15-29 years old), 2014 - 2021
   

Figure 4 - Share of young people (aged 20-29 years) estimated to potentially take up an OMT by EU Member State, 2014 and 2020, ordered by 2020 values from low to high
   

Figure 5 - Application of the QFT on the ground across the EU (country map)
   

Figure 6 - Trainee survey: job offers at the end of a traineeship, by sector
   

Table of tables

Table 1 - Regulatory approaches in 2014
   

Table 2 - Comparison between the 2013 Eurobarometer and the 2022 trainee survey
   

Table 3 - Regulatory approaches to traineeships in EU Member States
   

Table 4 - Conformity with QFT principles – OMTs

Table 5 - Conformity with QFT principles – ALMP Traineeships
   

Table 6 - Financial incentives in Member States
   

  

Glossary

|  |  |
| --- | --- |
| Term or acronym | Meaning or definition |
| ALMP traineeships | Traineeships as part of active labour market policies (ALMPs), organised by Public Employment Services (PES) in cooperation with employers, based on a tripartite agreement with the aim to help unemployed or inactive young people into employment. |
| Application on the ground | The degree to which regulation established is in fact abided by (i.e. practically applied) in traineeships taking place in the country. |
| EFQEA | Council Recommendation on a European Framework for Quality and Effective Apprenticeships |
| Conformity | The degree to which national legislation regulates traineeships as per the quality standards outlined in the QFT. |
| Cross-border (or transnational) traineeships | Traineeships that take place in an EU country other than the EU country where the trainee resides. |
| NEETs | Young people neither in employment nor in education and training |
| Open-market traineeships | Open-market traineeships (OMTs) are non-mandatory, bilateral, and private agreements agreed between a trainee and a traineeship provider (public/private/non-profit) without the involvement of a third party and without a formal connection to education or training. |
| PES | Public employment services |
| QFT | (Council Recommendation on a) Quality Framework for Traineeships |
| Traineeships | A limited period of work practice, whether paid or not, which includes a learning and training component, undertaken in order to gain practical and professional experience with a view to improving employability and facilitating transition to regular employment |
| YG / Youth Guarantee | (Council Recommendation on establishing the) Youth Guarantee |

1.
   Introduction

The Council Recommendation of 10 March 2014 on a Quality Framework for Traineeships (2014/C 88/01) (hereinafter QFT) aims at helping young people transition from education, unemployment or inactivity into the labour market through quality traineeships that enhance their skills and allow them to gain work experience. It sets out 21 quality principles
[1](#footnote1)
 for traineeships outside education curricula that can ensure high-quality learning content and adequate working conditions. The QFT is also an important reference point for determining what constitutes a good quality offer of traineeships under the reinforced Youth Guarantee
[2](#footnote2)
.

As emphasised in the European Pillar of Social Rights Action Plan, “[t]raineeships or internships allow to gain practical and first-hand experience. They are usefully facilitating young people’s access to the labour market. Yet, this aim can only be achieved if traineeships are of good quality and apply fair working conditions”. The Action Plan announced that the Commission will “[r]eview in 2022 the Council Recommendation on the Quality Framework for Traineeships, notably as regards working conditions”. The evaluation of the QFT also contributed to the 2022 European Year of Youth.

Purpose and scope of the evaluation

This report presents the findings of the Commission’s evaluation of the QFT. In line with the Better Regulation Guidelines, it assesses the extent to which the QFT is effective, efficient and coherent, provides EU added value and remains relevant to tackle present needs
[3](#footnote3)
.

Principle 20 of the QFT recommends that Member States “provide information to the Commission by the end of 2015 on the measures taken in accordance with this Recommendation”. Based on the information received, the 2016 Commission Staff Working Document (SWD) ‘Applying the Quality Framework for Traineeships’
[4](#footnote4)
 analysed the extent of Member States’ conformity with the quality principles of the QFT and pointed out remaining gaps. Its results were used for the present evaluation and are referred to below whenever relevant. The 2016 assessment did not, however, assess the extent to which the quality principles of the QFT were applied on the ground.

Moreover, despite the progress made since 2014, the youth unemployment rate remains at nearly double the unemployment rate of the overall labour force (15-64 years)
[5](#footnote5)
, indicating that young people continue to face structural challenges to access the labour market. Given the current geopolitical and macro-economic uncertainties and their impact on the labour market, the central objective of the QFT remains particularly pertinent: the improvement of the quality of traineeships, in particular as regards learning and training content and working conditions, with the aim of easing the transition from education, unemployment or inactivity to work. 

This evaluation report describes the current situation in the EU in terms of youth employment, the prevalence and quality of traineeships, the extent to which the QFT principles are adequately implemented
[6](#footnote6)
 across the 27 Member States and achieved the expected results. It reports on the main developments between the years 2014 and 2021. The evaluation report also assesses the extent to which the QFT principles are still fit for purpose in light of the current challenges. This includes a reflection on the relevance of quality aspects which are not covered by the QFT. Finally, the evaluation identifies whether traineeships and the implementation of the QFT may have had an impact on youth employment through quality traineeships (in particular learning content and working conditions), as well as on the prevalence of cross-border traineeships.

A key limitation of this evaluation report concerns the lack of systematic data collection on traineeships at EU level, in particular on open market traineeships (OMTs). In addition, information on traineeships across the EU is based on varying definitions, which poses problems of comparability for the analysis. Another challenge relates to isolating the impact of the QFT on youth employment from the impact of traineeships in general and of other youth employment measures. While the research conducted in support of this evaluation did not allow to establish a strong relation between the QFT (a non-binding instrument), the extent of its integration in the national legislation or frameworks and the quality of traineeships, some indications of the possible impacts of the QFT on the quality of traineeships and youth employment were found.

The limitations outlined above have consequences on e.g. the estimation of traineeship prevalence, the examination of the impact of the QFT on traineeships and the assessment of how well traineeships support the employability of young people. This evaluation is largely based on a dedicated study
[7](#footnote7)
 carried out in 2022 which is built on mapping activities, stakeholder interviews and surveys (both a targeted survey of former, current and potential future trainees, as well as a public consultation). In addition, research and studies on traineeships exist
[8](#footnote8)
, the results of which have been used in this evaluation.

2.
   What was the expected outcome of the quality framework of traineeships (qft)?

2.1 Description of the QFT and its objectives

Traineeships, also known as internships, are an important entry point into the labour market for young people. They complement traditional classroom instruction and allow obtaining important skills through practical experience.

At the time of adoption of the QFT in 2014, youth unemployment figures had reached all-time highs due to the impact of the global financial and economic crisis. Employment opportunities in general were scarce, and there were concerns that (low quality) traineeships would gain ground, including to substitute regular jobs. A 2013 Eurobarometer survey
[9](#footnote9)
 pointed out that 18% of traineeships did not provide a meaningful learning experience, 23% took place under substandard working conditions, and in effect did not constitute real traineeships but simply low-cost replacements for regular jobs.

At the same time, research increasingly confirmed that the value of traineeships in easing the transition to employment depended on their quality in terms of learning content and working conditions. For example, the Impact Assessment accompanying the Commission proposal for the QFT
[10](#footnote10)
 demonstrated a highly significant link between the quality of traineeships and the employment outcome. Quality traineeships bring direct productivity benefits, improve labour market matching and promote mobility, notably by decreasing search and matching costs both for enterprises and for trainees. 

The Impact Assessment also described the challenge caused by the fragmented, or even lacking, legislation applicable to traineeships in the EU. Furthermore, even where legislation existed, there were notable enforcement challenges.

Based on these findings, the Commission proposed the QFT to provide EU-wide quality standards for traineeships and to discourage traineeships with poor learning content or working conditions. The QFT complemented the objectives of the Youth Guarantee, which had been adopted one year earlier to address youth employment in a broader sense. 
[11](#footnote11)
 The QFT also contributes to the United Nations’ Sustainable Development Goal (SDG) 8, i.e. to “promote sustained, inclusive and sustainable economic growth, full and productive employment and decent work for all”, by supporting young people’s transition to the labour market whilst providing minimum quality standards.

For the purposes of the QFT, traineeships are understood as a limited period of work practice, whether paid or not, which includes a learning and training component, undertaken in order to gain practical and professional experience with a view to improving employability and facilitating transition to regular employment.

The QFT applies to Active Labour Market Policy (ALMP) traineeships as well as traineeships offered in the open market (OMTs). The QFT does however not cover work experience placements that are part of curricula of formal education or vocational education and training. Traineeships of which the content is regulated under national law and whose completion is a mandatory requirement to access a specific profession (medicine, architecture, etc.) are also not covered by the QFT.

The QFT has the following objectives:

„to facilitate a smooth transition from education to employment and to increase young people’s employability;

„to increase the quality and the transparency of the traineeship market ex ante, in order to make it easier for traineeship candidates to screen the offered quality before they make a commitment with a traineeship provider;

„to increase the awareness of rights and obligations among trainees and traineeships providers;

„to ensure more coherent regulatory approaches across Member States through common principles to be put in place in national legislation and/or national quality frameworks;

„to facilitate cross-border traineeships by addressing the need for transparent and clear information in view of the regulatory fragmentation across Member States.

The principles of the 2014 QFT

The 2014 Council Recommendation asks Member States to:

Conclusion of a written traineeship agreement

Learning and training objectives

Working conditions applicable to trainees

Rights and obligations

1.Improve the quality of traineeships, in particular as regards learning and training content and working conditions, with the aim of easing the transition from education, unemployment or inactivity to work by putting in practice the following principles for a Quality Framework for Traineeships:

2.Require that traineeships are based on a written agreement concluded at the beginning of the traineeship between the trainee and the traineeship provider;

3.Require that traineeship agreements indicate the educational objectives, the working conditions, whether an allowance or compensation is provided to the trainee by the traineeship provider, and the rights and obligations of the parties under applicable EU and national law, as well as the duration of the traineeship […];

4.Promote best practices as regards learning and training objectives in order to help trainees acquire practical experience and relevant skills; the tasks assigned to the trainee should enable these objectives to be attained;

5.Encourage traineeship providers to designate a supervisor for trainees guiding the trainee through the assigned tasks, monitoring and assessing his/her progress;

6.Ensure that the rights and working conditions of trainees under applicable EU and national law, including limits to maximum weekly working time, minimum daily and weekly rest periods and, where applicable, minimum holiday entitlements, are respected;

7.Encourage traineeship providers to clarify whether they provide coverage in terms of health and accident insurance as well as sick leave;

8.Require that the traineeship agreement clarifies whether an allowance or compensation is applicable, and if applicable, its amount;

9.Encourage the concerned parties to ensure that the traineeship agreement lays down the rights and obligations of the trainee and the traineeship provider, including, where relevant, the traineeship provider's policies on confidentiality and the ownership of intellectual property rights;

To reach these objectives, the QFT supports the improvement of working conditions and the learning content of traineeships through 21 quality principles
[12](#footnote12)
 which include provisions to enhance transparency of and information on traineeships (see box below). Of these, principles 19 and 20 are considered are considered action points for Member States rather than specific quality elements to be applied to traineeships. They are therefore not assessed as such in the evaluation. The QFT also outlines how the Commission intends to, inter alia, support Member States’ action through EU funds, the exchange of good practices, and monitoring. 

Reasonable duration

Proper recognition of traineeships

Transparency requirements

Cross-border traineeships

Use of European Structural and Investment Funds

Applying the Quality Framework for Traineeships

10.Ensure a reasonable duration of traineeships that, in principle, does not exceed six months, except in cases where a longer duration is justified, taking into account national practices;

11.Clarify the circumstances and conditions under which a traineeship may be extended or renewed after the initial traineeship agreement expired;

12.Encourage the practice of specifying in the traineeship agreement that either the trainee or the traineeship provider may terminate it by written communication, providing advance notice of an appropriate duration in view of the length of the traineeship and relevant national practice;

13.Promote the recognition and validation of the knowledge, skills and competences acquired during traineeships and encourage traineeship providers to attest them, on the basis of an assessment, through a certificate;

14.Encourage traineeship providers to include in their vacancy notices and advertisements information on the terms and conditions of the traineeship, in particular on whether an allowance and/or compensation and health and accident insurance are applicable; encourage traineeship providers to give information on recruitment policies, including the share of trainees recruited in recent years;

15.Encourage employment services and other providers of career guidance, if providing information on traineeships, to apply transparency requirements;

16.Facilitate the cross-border mobility of trainees in the European Union inter alia, by clarifying the national legal framework for traineeships and establishing clear rules on hosting trainees from, and the sending of trainees to, other Member States and by reducing administrative formalities;

17.Examine the possibility to make use of the extended EURES network and to exchange information on paid traineeships through the EURES portal;

18.Make use of the European Structural and Investment Funds, namely the European Social Fund and the European Regional Development Fund, in the programming period 2014-2020, and the Youth Employment Initiative, where applicable, for increasing the number and quality of traineeships, including through effective partnerships with all relevant stakeholders;

19.Take appropriate measures to apply the Quality Framework for Traineeships as soon as possible;

20.Provide information to the Commission by the end of 2015 on the measures taken in accordance with this Recommendation;

21.Promote the active involvement of social partners in applying the Quality Framework for Traineeships;

22.Promote the active involvement of employment services, educational institutions and training providers in applying the Quality Framework for Traineeships.

To understand how the QFT was expected to work, the intervention logic (
[Figure 1](#_Ref117846306)
) outlines the rationale of the intervention and the needs that triggered it. It also describes the objectives and expected outputs and the way how they were supposed to be achieved (actions).

Figure 1 - QFT intervention logic

2.2 Points of comparison

The points of comparison against which the intervention is assessed, concern young people’s situation on the labour market, the prevalence and quality of traineeships, as well as regulatory approaches at the time of adoption of the QFT in 2014. It is, in particular the parts on the prevalence and quality of traineeships, mainly based on the information provided in the 2013 Impact Assessment accompanying the proposal for a QFT
[13](#footnote13)
, which in turn largely relied on the 2013 Eurobarometer survey on traineeships
[14](#footnote14)
. Where possible, the 2013 Eurobarometer survey is directly referenced.

Young people’s labour market situation in 2014

In 2014, both the youth unemployment rate (15-29) and the share of young people who were neither in employment, education or training (NEETs) in the EU were high, standing at 18.7% and 15.7% respectively. Both rates had followed an upward trend from the onset of the global financial and economic crisis in 2008 (respectively 12.3% and 13.1%), reaching their peaks in 2013. Although variations between Member States were significant, these figures show the long-lasting impact the crisis had on young people.

Figure 2 - EU-27 average unemployment, NEET and long-term unemployment rate (15-29 years old), 2008-2014

![](./../../../resource.html?uri=IMMC:SWD%282023%299.ENG.xhtml.SWD_282023_299_ENG_xhtml_04002.jpg)
Source: Eurostat, youth unemployment rate by sex, age and country of birth
[15](#footnote15)
; young people neither in employment nor in education and training by sex, age and labour status; youth long-term unemployment rate (12 months or longer) by sex and age
[16](#footnote16)

Furthermore, the share of young people that were out of work for a prolonged period – 12 months or longer – stood at 7.5% in 2014. It had reached its peak in 2013 at 7.7%, with long-term consequences on young people’s skills and employability, future earnings and well-being.

Prevalence of traineeships in 2014

Although official statistics on traineeships do not exist, the 2013 Eurobarometer survey
[17](#footnote17)
 provided some indications on their prevalence. These findings should be interpreted with caution due to limitations on the representativeness of Eurobarometer surveys. The Eurobarometer results indicated that traineeships were widespread around the time of the adoption of the QFT, with 46% of respondents aged from 18 to 35 in the EU having undertaken at least one (and often more than one) traineeship. At the time, experts
[18](#footnote18)
 estimated that the number of traineeships was on the rise. One reason for this trend was that the majority of traineeships was unpaid or low-paid and represented a relatively affordable option for companies in times of crisis
[19](#footnote19)
. It might also have reflected the lack of better employment opportunities for young people in the context of high youth unemployment.

In terms of traineeship prevalence by sector, according to a 2012 Commission study
[20](#footnote20)
, OMTs were becoming widespread across all sectors. 9% of them had a cross-border nature.

The study also found that unpaid or low-paid traineeships were particularly present in the creative (e.g. culture, art, design, publishing) and media/journalism sectors. Furthermore, larger organisations were found more likely to offer traineeships compared to SMEs. However, the SME test which was carried out in 2012-2013
[21](#footnote21)
 showed that the majority of respondent SMEs (71.9%) did consider traineeships as a way to select and train future employees and highlighted the importance of traineeships for job creation.

Quality of traineeships in 2014

Before the adoption of the QFT, the quality of traineeships across the EU presented several areas of concern. The 2013 Eurobarometer results indicated that one in three traineeships was deemed unsatisfactory, either in terms of working conditions
[22](#footnote22)
 (25% of all traineeships), learning content (18%) or both. It also found that 32% of respondents received compensation, and only 41% of these indicated that this was sufficient to cover living costs. The widespread use of unpaid and/or low-paid traineeships entailed the risk of traineeships being abused to replace regular employment. It also raised concerns about equal opportunities, especially for young people from disadvantaged backgrounds.

Another common criticism concerned the (frequent) lack of social security and/or health/medical insurance coverage, entitlement to holidays, sick or holiday pay, and pension entitlements. Furthermore, the absence of adequate monitoring was also identified as an element that negatively affected traineeship quality.

The aforementioned 2012 Commission study moreover highlighted recurrent weaknesses in the learning content of traineeships. It found that in particular trainees in OMTs were frequently asked to perform tasks which were either irrelevant to their learning needs or perceived as too mundane.

The 2013 Eurobarometer highlighted that a lack of information was an important cause for the lower prevalence of cross-border traineeships, with 38% of those who would have been interested in doing a traineeship abroad citing insufficient information on traineeship regulations in other Member States as an obstacle.

Lastly, in terms of the extent to which traineeships contributed to fostering the labour market integration of young people, less than a third of respondents to the Eurobarometer (27%) had been offered a work contract after the end of their traineeship by the same employer who provided the traineeship.

Regulatory approaches in 2014

The regulatory approaches to traineeships varied widely across Member States and types of traineeships in 2014 (see 
[Table 1](#_Ref121907041)
). Nine Member States had a specific regulation on traineeships. ALMPs traineeships were more often regulated, as compared to OMTs
[23](#footnote23)
. While certain Member States (e.g. France) regulated all types of traineeships by law (i.a. forbidding OMTs to ensure that all traineeships are monitored by a third party institution) many other countries did not have in place any specific legislative framework for trainees. Less than half of the then 28 Member States had legislative provisions on traineeships’ duration, remuneration or social protection coverage. Additionally, in 11 Member States there were legislative and administrative barriers to transnational traineeships
[24](#footnote24)
.

Table 1 - Regulatory approaches in 2014

|  |  |
| --- | --- |
| Regulatory approach | Traineeships (no distinction made in type of traineeships) |
| Specific regulation on traineeships | EE, ES, FI, FR, HU, IT, PT, SI, SK (9) |

The widely diverse approach to regulation reflected the equally great variety of approaches to defining traineeships and/or trainees. As of 2012, most Member States had either a definition in their legislation or at least reached a common national understanding of the concept of a traineeship. However, this did not result in a homogenous or coherent definition of a traineeship across the EU. Among the common characteristics of traineeships that could be found in legislative frameworks across Member States prior to 2014 were: the general educational purpose; the practical element of learning; and the temporary nature of the traineeship. Some national regulations also addressed aspects related to information on working conditions and rights of the trainee during the traineeship
[25](#footnote25)
.

Even in countries where legislative and regulatory frameworks existed, the presence of these regulations did not guarantee that traineeships were of high quality, due to their inadequate enforcement, particularly in the open market
[26](#footnote26)
.

3.
   How has the situation evolved over the evaluation period?

Current state of play

Young people’s labour market situation

While high quality traineeships can help young people in their transition to employment, their situation on the labour market depends on many other factors, above all the economic situation. Before the COVID-19 pandemic began in 2020, the youth employment rate was steadily improving, though the crisis induced by the pandemic interrupted this trend. The information presented below aims to illustrate the context in which the QFT was implemented.

Figure 3 - EU27 average unemployment, NEET and youth long-term unemployment rate (15-29 years old), 2014 - 2021

Source: Eurostat (LFSA\_URGAN, TESEM150, YTH\_EMPL\_120)

Since the adoption of the QFT in 2014, youth unemployment in the EU followed a decreasing trend from 18.7% to 11.9% in 2019. In 2020, triggered by the Covid-19 pandemic, youth unemployment increased to 13.3%, while slightly decreasing again in the following year to 13%. The NEETs rate in the EU followed the same trend, falling from 15.7% in 2014 to 12.6% in 2019, with a spike in 2020 at 13.8% and decreasing again to 13.1% in the year after.

As for long-term youth unemployment, the effect of the Covid-19 pandemic manifested itself with a slight delay, as it followed a downward trend between 2014 and 2020 dropping from 7.5% to 3.2% but rose again to 3.5% in 2021.

Though compared to 2014, the labour market situation of young people has improved significantly, the uncertainty on the mid- and long-term impacts of the Covid-19 induced disruptions, the Russian invasion of Ukraine, the energy crisis and the rising costs of living call for caution. Furthermore, the youth unemployment rate remains at almost double the unemployment rate of the overall labour force (15 – 64 years)
[27](#footnote27)
, indicating the structural challenges that young people continue to face. Such challenges include the lack of professional experience compared to older workers, and if hired, being more prone to be employed under precarious conditions including temporary contracts.

Prevalence of traineeships

Robust and EU-wide comparable data on the prevalence of traineeships is lacking and therefore, estimations
[28](#footnote28)
 were made. The share of young people estimated to potentially take up a traineeship across the EU has increased from 16.6% in 2014 to 17.2% in 2020.
[29](#footnote29)
 Trainees with educational qualifications below tertiary education are not captured by this estimate. In the 2013 Eurobarometer survey on traineeships
[30](#footnote30)
, traineeships were found to be more prevalent among university graduates (61%), compared to persons with no tertiary qualifications (40%). The trainee survey (conducted as part of the 2023 study supporting this evaluation)
[31](#footnote31)
 also found that the majority of respondents (973 out of 1,836 (53%)) had tertiary educational attainment when undertaking their traineeship. As is the case with the 2013 Eurobarometer survey, the results of this trainee survey need to be taken with caution, in light of the limited representativeness of such a survey
[32](#footnote32)
.

Based on indications provided by the trainee survey
[33](#footnote33)
, the difference between women and men seems to have become more pronounced, with women increasingly more likely than men to undertake an OMT. 1,212 out of 1,836 (66%) of female respondents in the trainee survey and 606 out of 1,836 (33%) of male respondents had completed one traineeship after finishing their education. The 2013 Eurobarometer resulted in 49% of female respondents and 43% of male respondents with a traineeship experience
[34](#footnote34)
.

[35](#footnote35)
Figure 4 - Share of young people (aged 20-29 years) estimated to potentially take up an OMT by EU Member State, 2014 and 2020, ordered by 2020 values from low to high

Source: 2023 study supporting the evaluation. Note: the EU average is calculated using the data from the 20 Member States that have OMTs in their labour market (i.e. excluding Estonia, Finland, France, Italy, Malta, Sweden and Slovakia).

There are also indications that cross-border traineeships have become more common since 2014. The trainee survey
[36](#footnote36)
 shows that 363 out of 1,912 (19%) of respondents had undertaken a cross-border traineeship. This may suggest that the number of cross-border traineeships has increased since 2014
[37](#footnote37)
, as per the objective of the QFT.

The QFT refers to the use of the European Job Mobility Portal (EURES) to exchange information on paid traineeships. The 2016 EURES regulation
[38](#footnote38)
 introduced an obligation for Member States to share on the EURES portal information and guidance on (paid) traineeships at national level as from 2018. The trainee survey indicated that 239 out of 1,836 (13%) of respondents made use of the EURES portal to find a cross-border traineeship. 

Quality of traineeships

The main element of the QFT is the written traineeship agreement that indicates the educational objectives, adequate working conditions, rights and obligations, and a reasonable duration for traineeships (see recital 11 of the QFT
[39](#footnote39)
).

Assessing these quality elements, the 2023 study supporting this evaluation shows that there are indications that the quality of traineeships since 2014 has generally increased. This observation is based on the responses of national stakeholders to targeted consultations, the trainee survey, the vacancy analysis (all three conducted in the 2023 study supporting the evaluation
[40](#footnote40)
) as well as to the public consultation carried out as part of this evaluation.

The public consultation provided indications that overall the quality of traineeship offers has improved to a large or moderate extent (143 out of 239 respondents (60%)) and that there are more traineeships of good quality (120 of 169 (71%) of respondents in the public consultation).

When comparing the outcomes of the 2013 Eurobarometer and the 2022 trainee survey, a more mixed picture emerges:

[41](#footnote41)
Table 2 - Comparison between the 2013 Eurobarometer and the 2022 trainee survey (carried out under the 2023 study supporting the evaluation)

|  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | Eurobarometer, European Commission, 2013 | | Survey of trainees, 2022 (under the 2023 study supporting the evaluation) | | | Trend |
| Learning outcomes | 18% of respondents consider they have not learned anything professionally useful during their traineeship (N=3,464). | | 13% of respondents considered they have not learned anything professionally useful during their traineeship. (N=1,836) | | |  |
| Job outcomes | 27% of respondents who completed a traineeship were offered an employment contract at the end of their traineeship (N=5,945). | | 42% of respondents indicated that they were offered a job after their OMTs (N=1,836). | | |  |
| Written agreement | 35% trainees did not have a written traineeship agreement or contract with the host organisation or company (N=5,945). | | 17% did not sign such a written agreement. (N=1,836). | | |  |
| Transparency | 42% of respondents said that the advertisement made clear how much the traineeships was paid | | 21% of respondents said that the vacancy notice mentioned the amount of the allowance and/or compensation (N=1,836). | | |  |
| Certificate or reference letter | 64% of respondents stated that at the end of the traineeship the organisation or company gave a certificate or a letter of reference (N=5,945). | | 68% of respondents reported that they had received either a certificate (32%), a letter of reference (24%) or both (12%) (N=1,836). | | |  |
| Duration | | 15% of respondents stated that their traineeship lasted longer than 6 months (N=5,945). | | 22% of respondents stated that their traineeship lasted longer than 6 months (N=1,912). |  | |

As shown in the table above, compared to 2013, the share of trainee respondents who consider that their traineeship did not result in useful professional skills and work experience decreased to below 20% in 2022. Furthermore, while in 2013, 27% of respondents reported to have been offered a job after their traineeship, this increased to 42% (771 out of 1,836) in the 2022 trainee survey. On the written agreement principle, the share of trainees without such an agreement was much higher in 2013 (38%) than in 2022 (17%, or 312 out of 1,836 respondents). The percentage of respondents indicating to have received a certificate or reference letter also slightly increased. However, based on the two surveys, a negative trend can be observed for the following quality princples: transparancy of (information provided in the) vacancies (i.e. the amount of pay/allowance/compensation) and duration of traineeships
[42](#footnote42)
.

In addition, the 2023 study supporting this evaluation examined almost 2,000 vacancies in the 27 Member States on their alignment with the QFT principles on transparency (i.e. providing information in the vacancy on the terms and conditions of the traineeship). It found a higher alignment for ALMP traineeships compared to OMTs. Vacancies for OMTs in 15 Member States complied hardly or to a low extent with the quality principles on transparency, while this was the case in only six Member States for vacancies for ALMP traineeships.

Regulatory approaches

Similar to the situation in 2014, the 2023 study supporting this evaluation
[43](#footnote43)
 found that national legislation generally regulates ALMP traineeships to a high degree, but OMTs to a more modest degree. Since 2014, five Member States (BG, ES, LT, LU and RO) adopted legislation on OMTs. 12 Member States (BE, BG, DK, EE, EL, HR, IE, IT, LT, PT, RO, and SK) adopted legislation covering ALMP traineeships.

In addition, ALMP traineeships are more likely to be governed by specific laws on ALMPs, whereas OMTs are more likely to fall under general labour law or specific provisions on OMTs, if regulated at all. This is illustrated in the table below:

Table 3 - Regulatory approaches to traineeships in EU Member States

|  |  |  |  |
| --- | --- | --- | --- |
| Regulatory approach | 2014 [44](#footnote44) | 2021 | |
|  | Traineeships (no distinction made between types) | ALMP traineeships | OMTs |
| Specific measures in the labour code or in dedicated legislative instruments | EE, ES, FI, FR, HU, IT, PT, SI, and SK (9) | AT, BG, BE, DE, DK, EL, EE, ES, FI, FR, HR, IT, LT, LU, MT, PL, PT, RO, SK, SI, SE (21) | BE, BG, DE, ES, FR [45](#footnote45) , LU, LT, RO, PL, PT, SI (11) |
| General labour legislation | No information available | CZ, HU, LV, NL (4) | AT, CZ, DK, EL, HR, HU, IE, LV, NL, SE (10) |
| No regulation | No information available | CY, IE (2) | CY, EE, IT, FI, MT, SK (6) [46](#footnote46) |

Integration of QFT principles in national legislation or frameworks

Based on the 2016 Commission SWD
[47](#footnote47)
 and the 2023 study supporting this evaluation the degree of the reflection of the QFT principles in national legislation or frameworks (i.e. Member States’ ‘conformity’ with the QFT) can be assessed. However, and in particular given the non-binding nature of the QFT, it is not possible to establish a causal link between regulatory developments at national level and the adoption of the QFT.

A slight increase in terms of conformity can be observed for OMTs (four Member States fully/mostly aligned in 2016 versus seven Member States in 2021). A larger increase is observed for ALMP traineeships, with 18 Member States fully/mostly in conformity in 2021 versus 15 in 2016. Further analysis can be found in section 4.

Application of the QFT on the ground

The practical application of the QFT on the ground was assessed as part of the 2023 study supporting this evaluation through a vacancy analysis (comprising 1,972 vacancies on OMT as well as ALMP traineeships in all 27 Member States), desk research and interviews with stakeholders, and the trainee survey. The analysis found that the practical application of the QFT principles is lagging behind their legislative implementation in the majority of Member States, in particular for OMTs. Further analysis can be found in section 4.

  

4.
   Evaluation findings 

The analysis in this section is based on the evidence provided in sections 2 and 3 unless otherwise mentioned. The different research activities carried out in the 2023 study supporting this evaluation
[48](#footnote48)
 (i.e. targeted consultations, mapping and case studies) underpin the assessment of the five evaluation criteria (effectiveness, efficiency, coherence, added value and relevance). In addition, the public consultation conducted by the Commission supports the assessment of four out of five evaluation criteria, i.e. effectiveness, efficiency, added value and relevance.

For the following analysis, it is to be noted that the QFT is a non-binding instrument. It cannot be ascertained that changes observed since the adoption of the QFT in 2014 are a direct result of the QFT. Furthermore, the 21 principles
[49](#footnote49)
 of the QFT cannot all be considered “quality principles” as such. Principle 1
[50](#footnote50)
 is a general recommendation to Member States to improve the quality of traineeships by putting in practice the quality principles of the QFT and principles 19 and 20
[51](#footnote51)
 are considered action points for Member States rather than specific quality elements inherent to quality traineeships and are thus not assessed as such.

4.1.
   To what extent was the QFT successful and why?

Effectiveness

Integration of QFT quality principles in national legislation and frameworks

Since the adoption of the QFT there has been moderate progress in the take up of its quality principles in national legislation and frameworks, with notable differences across Member States and types of traineeships.

Since 2014, 14 Member States (BE, BG, DK, EL, EE, ES, HR, IE, IT, LT, LU, PT, RO, SK) have adapted or introduced legislation (either on OMTs or ALMP traineeships or both) integrating quality principles of the QFT in their legislative or policy frameworks. Such changes were more prevalent in national legislation/policy governing ALMP traineeships (identified in 12 Member States), than OMTs (identified in five Member States).

In the targeted consultations as part of the 2023 study supporting this evaluation
[52](#footnote52)
, some stakeholders argued that the legislation in certain Member States may not have been amended since 2014, because traineeship standards are adequately governed by existing legislation
[53](#footnote53)
 or because certain types of traineeships do not exist or are very rare
[54](#footnote54)
. For example, since the adoption of the QFT, Estonia, Finland, and Malta did not implement legislative changes on OMTs, which are rare in these countries
[55](#footnote55)
. This underlines that the extent to which the QFT principles have been integrated in national legislation depends on the overarching existing legislative framework for traineeships in the country as well as the (perceived) prevalence and uptake of specific types of traineeships in the labour market.

Examples of the integration of the QFT in national legislation

Luxembourg - A law specifically putting in place all principles of the QFT was introduced in 2020.

Finland - QFT principles are implemented in Finnish legislation on ALMP traineeships. For example, the Finnish work try-out programme describes in detail whether the trainee is entitled to an allowance or compensation, and the amount.

In some Member States (e.g. BG, ES, FI, IE)
[56](#footnote56)
, stakeholders (including youth representatives, PES and national policymakers) expressed the view
[57](#footnote57)
 that the introduction of new policies had led to an improved alignment with the QFT. For example, representatives from both the PES and a youth organisation underlined that the new labour market reform in Spain had increased conformity with the QFT. National policymakers in Bulgaria highlighted that the QFT drove legislative changes in relation to OMTs.

The mere fact that changes were made to national legislation or frameworks on traineeships does not mean that national legislation or frameworks have fully integrated all QFT quality principles. Therefore, the extent of the integration of the QFT quality principles has also been analysed, and is hereafter referred to as conformity (see 
[Table](#_Ref117760497)
4 on OMT and 
[Table](#_Ref117760504)
5 on ALMP traineeships)
[58](#footnote58)
. For the purpose of this evaluation, the conformity with the QFT principles under the dimensions ‘written agreement’, ‘learning and training objectives’, ‘working conditions’, ‘reasonable duration’, ‘proper recognition’, and ‘transparency requirements
[59](#footnote59)
 was assessed. This assessment builds on desk research and on targeted consultations using a scoring method explained in Annex II Methodology.

 

Table 4 - Conformity with QFT principles – OMTs

|  |  |  |
| --- | --- | --- |
| Degree of conformity | [60](#footnote60) 2016 | 2021 |
| Fully/mostly | BE, BG, LT, SI (4) | BE, BG, ES, LT, LU, RO, SI (7) |
| Partially | CZ, DE, ES, PL, PT, RO (6) | AT, CZ, DE, HU, NL, PL, PT (7) |
| Modestly | AT, DK, EL, HU, IE, LU, NL (7) | CY, EL, HR, IE, LV (5) |
| Not implemented |  | DK (1) |
| Not applicable | CY, EE, FI, MT, SE, SK (6) | [61](#footnote61) EE, FI, FR, IT, MT, SE, SK (7) |

Source: 2016 Commission SWD and 2023 study supporting this evaluation.

Table 5 - Conformity with QFT principles – ALMP Traineeships

|  |  |  |
| --- | --- | --- |
| Degree of conformity | 2016 | 2021 |
| Fully/mostly | AT, BE, BG, CY, DK, EL, FR, HU, LU, MT, PT, RO, SE, SI, SK (15) | AT, BE, BG, ES, FI, FR, HR, IE, IT, LT, LU, MT, PL, PT, RO, SE, SI, SK (18) |
| Partially | CZ, DE, EE, ES, FI, HR, IE, IT, LT, LV, NL, PL (12) | CY, CZ, DE, DK, EE, EL, HU, LV, NL (9) |
| Modestly | - | - |
| Not implemented | - | - |

Source: 2016 Commission SWD and 2023 study supporting this evaluation

A slight improvement in terms of conformity can be observed for OMTs with four Member States fully/mostly aligned in 2016 versus seven Member States in 2021. A larger improvement is observed for ALMP traineeships, with 18 Member States fully/mostly in conformity in 2021 versus 15 in 2016.

In 2021, the QFT principles that are most commonly
[62](#footnote62)
 implemented by legislation (either fully or partially) across the EU-27 are the same across OMTs and ALMP traineeships:

1.Trainees' rights and working conditions are respected including limits to maximum weekly working time, weekly rest periods, minimum holiday entitlements (covered in 18 Member States in legislation governing OMTs and in all 27 Member States for ALMP traineeships);

2.The traineeship is based on a written agreement (covered in 17 Member States in legislation governing OMTs and in 27 Member States for ALMP traineeships); 

3.The written agreement indicates key information including educational objectives, working conditions, if compensation is provided and how much, rights/obligations of all parties, and the duration (covered in 17 Member States in legislation governing OMTs and all 27 Member States in ALMP traineeships).

The QFT principles that are least
[63](#footnote63)
 implemented in legislative frameworks of Member States are also the same across OMTs and ALMP traineeships: 

1.Traineeship providers include in their vacancies information on the conditions of the traineeship (this is not implemented at all in six Member States for OMTs and in nine Member States for ALMP traineeships);

2.The duration of the traineeship does not exceed six months (in seven Member States for OMTs and in seven Member States for ALMP traineeships);

3.The knowledge, skills and competences acquired by the trainee are recognised by the traineeship provider through an assessment and a certificate (six Member States do not implement this at all for OMTs and seven Member States for ALMP traineeships).

Practical application on the ground

While some notable alignment of Member States’ traineeship standards to the QFT could be observed, the practical application of these regulatory standards on the ground is lagging behind in most Member States, notably for OMTs.

This assessment is based on the following research elements of the 2023 study supporting this evaluation
[64](#footnote64)
:

·The analysis of traineeship vacancies for both types (OMTs and ALMP traineeships) in each Member State with regard to their conformity with the transparency principles of the QFT;

·Desk research and interviews conducted with traineeship providers (employers) and other actors involved in the practical application of the QFT (e.g. national authorities);

·Research on the impact of the QFT on trainees and the society, through the trainee survey, desk research and interviews
[65](#footnote65)
.

[66](#footnote66)
[67](#footnote67)
[68](#footnote68)
[69](#footnote69)
[70](#footnote70)
[71](#footnote71)
The study observed that the quality parameters of ALMP traineeships are more closely aligned with Member States’ regulatory frameworks than in the case of OMTs. The practical application of the QFT principles for ALMP traineeships can be considered as low or very low in only two Member States, while all other Member States are assessed to moderately or to a high degree apply the QFT on the ground. For OMTs, the picture (see country map below) is less positive and more diverse with no Member States applying the QFT to a high degree on the ground, nine Member States to a moderate degree, eight Member States to a low extent, three Member States to a very low extent and for the rest (seven), OMTs were found not to be existing or very rare.

[72](#footnote72)
As regards the transparency principles of the QFT, a vacancy analysis as part of the 2023 study supporting this evaluation scanned through 1,972 vacancies in all 27 Member States (of which 1,272 were on OMTs and 700 concerned ALMP traineeships). 42% of the analysed OMT vacancies and 59% of the ALMP traineeship vacancies mentioned allowance or compensation. 21% of OMT vacancies and 44% of ALMP vacancies indicated the amount. These results support the view that the application of the QFT principles on the ground is higher for ALMP traineeships than for OMTs.

The trainee survey as part of the same study also suggests that the practical application of the QFT principles is higher in certain sectors than in others. For example, most respondents in the sectors ‘Financial and Insurance services’ and ‘Professional, scientific and technical activities’ stated that they had signed an agreement at the beginning of their traineeship (1,469 of 1,836 (80%) and 1,487 of 1,836 (81%) respectively). The QFT quality principles were found to be applied less in sectors such as arts, entertainment and recreation, health and social work, and education. Furthermore, sectors with a larger share of small enterprises were more likely not to apply the QFT principles due to the limited human and financial capacity to handle the administrative burden – or the perceived administrative burden – of ensuring quality traineeships.

More generally, a low level of awareness of the QFT principles and the potential benefits of quality traineeships amongst young people and employers alike hampered the QFT reaching its full potential in terms of effectiveness.

  

[73](#footnote73)
Figure 5 - Application of the QFT on the ground across the EU (country map)

![](./../../../resource.html?uri=IMMC:SWD%282023%299.ENG.xhtml.SWD_282023_299_ENG_xhtml_04003.jpg)

Monitoring and enforcement

Principle 19 of the QFT recommends that Member States take appropriate measures to apply the QFT as soon as possible. However, the monitoring and enforcement of the relevant national legislation governing traineeships is in many cases not fit for purpose. There are substantial variations between the existence and functioning of monitoring and enforcement mechanisms across Member States. Such mechanisms exist in all 27 Member States for ALMP traineeships, while for OMTs, they exist in 14 Member States
[74](#footnote74)
.

The case studies
[75](#footnote75)
 and interviews (as part of the targeted consultations)
[76](#footnote76)
 carried out for the 2023 study supporting this evaluation suggest that even where such mechanisms exist for OMTs, they have a limited impact on ensuring the practical application of regulations. This can be attributed to an insufficient capacity of labour inspectorates or a less systematic monitoring by labour inspectorates where traineeships are regulated as a specific employment relationship (instead of being covered by general labour law, which is more systematically monitored).

The decision of the Council of Europe’s Committee of Social Rights on the case brought by the European Youth Forum regarding the possibilities under Belgian law allowing for unpaid internships noted that the “labour inspectorate is not sufficiently effective in detecting and preventing “bogus internships”
[77](#footnote77)
. Moreover, point 163 of the decision explained that “[t]he Committee found that the inspection system which solely depends on individual complaints by interns, considering their disadvantaged situation, cannot be considered as sufficiently efficient in preventing misuses of unpaid internship contracts in violation of Article 4§1 of the Charter”
[78](#footnote78)
. 

A lack of clarity on who is responsible also weakens Member States’ monitoring and enforcement capacities. The 2023 study supporting this evaluation highlighted the cases of Belgium where there are ambiguities on which level of government is responsible and Italy, where a lack of cooperation between different national authorities makes it difficult to carry out controls effectively.
[79](#footnote79)
 

In the targeted consultations
[80](#footnote80)
 with national authorities and employers, some (EE and SK) stated that the application of the QFT on the ground is also impeded by a lack of awareness among both trainees and employers about the QFT quality principles (or relevant national legislation which they should observe). In addition, employers highlighted the following obstacles:

·The lack or the complexity of existing relevant legislative frameworks;

·Organisational changes due to the outbreak of the COVID-19 pandemic;

·Difficulties in cooperation between public employment services (PES) and employers, and insufficient PES capacity to properly match trainees with labour market needs.

More generally, consultations with EU and national stakeholders (in interviews as part of the targeted consultations) gave indications that the QFT may be lacking practical guidance for employers / traineeship providers on the overall implementation of the QFT.

Involvement of relevant stakeholders

The QFT (principles 21 and 22) recommends promoting the active involvement of key stakeholders, including social partners, public employment services, education institutions and training providers in applying the QFT.

The public consultation carried out in support of this evaluation gave indications that the active involvement of social partners and key stakeholders in implementing quality traineeships was considered very important or important by some stakeholders (i.e. rated as very important by all six trade union respondents (100%), two out of three business associations respondents (66%), important by 32 out of 85 public authorities (38%), 12 out of 34 former or current trainees (35%), and 23 out of 56 academic/research institutions (41%)). This aspect is deemed particularly relevant by social partners, with employer organisations and trade unions valuing stakeholders’ engagement the most (67% and 100% respectively).

The involvement of social partners through collective agreements (e.g. AT, FI and ES) tends to focus on working conditions. The case study on Austria
[81](#footnote81)
 observed that improvements for trainees mainly related to establishing a minimum pay in certain sectors (not a QFT principle), but that in the Information Technology sector the collective agreement also required a training plan to be defined (relevant to the QFT principle of defining learning and training objectives). In Finland, trade unions at sectoral level are involved in collective agreements, which also cover traineeships as regards pay, quality issues, rights and working conditions. The case study on Spain
[82](#footnote82)
 found that trade unions regularly engage in negotiations with employer organisations and/or the government in relation to traineeship relevant legislation, such as the recently adopted labour market reform (RDL 32/2021). This reform specifies the need for supervision and individual training plans.

Besides collective agreements, social partners are involved in discussions on regulation and/or policy-making with regards to traineeships (FI, HR, IT, LU, IE) and in monitoring the implementation of policies relevant to traineeships (FI and DK). It was not possible to deduce from the evidence gathered in the 2023 study supporting this evaluation whether such involvement has had an impact on the application of the QFT principles.

In all Member States, the case studies and interviews conducted in the targeted consultations
[83](#footnote83)
 found that PES are actively involved in the design, delivery and promotion of ALMP traineeship programmes, at times in cooperation with other national authorities and employers/employer organisations. PES also develop guidance and support for key actors (such as traineeship providers) on how to implement high quality traineeships.

Impact on easing labour market transitions

Quality traineeships can have a positive impact on young people through increased employability in terms of relevant skills, practical experience and access to professional networks and, as explained earlier, one of the key aims of the QFT is indeed to help young people transition from education, unemployment or inactivity into the labour market.

This was confirmed to be the case by the trainee survey
[84](#footnote84)
, carried out in 2022 as part of the 2023 study supporting this evaluation, where over 80% of respondents
[85](#footnote85)
 (strongly) agree that the tasks they did during their traineeship helped them gain practical experience and relevant skills as well as achieve their learning and training objectives
[86](#footnote86)
. 771 out of 1,836 (42%) of respondents were offered a job after their traineeship, and 717 (93%) of them reported that traineeship helped them at least to some extent to get a job offer. Even higher starting salaries have been identified in some Member States
[87](#footnote87)
.

The evidence gathered as part of the study does not allow to ascertain the impact of the implementation of the QFT itself on trainees, as it was not possible to determine to which extent the QFT has influenced the legislation. However, the study assessed which of the QFT principles
[88](#footnote88)
 were considered most useful for facilitating young people’s labour market integration based on the views gathered through the trainee survey and interviews conducted in the targeted consultations (see box below). On the QFT principles not mentioned in the box below, the views as to their impact on the labour market integration of young people were less pronounced.

QFT principles which are most likely to have a positive impact on labour market integration of young people:

Defining learning and educational objectives of the traineeship, including the appointment of a supervisor or a mentor. These elements were highlighted by employer organisations from Italy, Lithuania and Hungary as well as EU level employer organisations and trade union representatives from the Netherlands, Croatia, Bulgaria, France and Poland. In the trainee survey of the evaluation study, 89% of the respondents stated that ‘the possibility of trying out different tasks’ and 85% that ‘having access to training opportunities’ were important/essential for trainees in their traineeship.

A written traineeship agreement can be considered a particularly important element. Representatives from public employment services in particular highlighted that having a traineeship contract, with clearly defined educational objectives and standards, helps trainees understand what is expected from them, so that they can meet workplace requirements and increase their chances of obtaining employment following their traineeship. The QFT principle on a written traineeship agreement has, in the view of stakeholders from Estonia, Cyprus, Ireland, Luxembourg, Poland and Slovakia contributed to reducing the exploitation of trainees and to a greater awareness of their rights.

The trainee survey conducted as part of the evaluation study also highlighted sectoral differences, with respondents working in arts, entertainment and recreation and health and social work most likely to disagree that their traineeship made their transition from school to work easier (33% for both sectors (i.e. 38 out of 115 and 63 out of 191 respectively)). Trainee respondents from the Financial and Insurance and Information and Communication sectors were most likely to agree with this statement (96 out of 134 (72%) and 95 out of 134 (71%). These findings are mirrored when examining the actual outcomes of the traineeship following completion. Trainees in the Financial and Insurance sector were most likely to report that they had been offered a job after their traineeship (80 out of 134 respondents in the sector (60%)), followed by those in Construction (43 out of 76 (56%)). Trainees in education were most likely to report that they had not received an offer of a job after their traineeships (119 out of 172 (69%)), followed by those in health and social work (126 out of 191 (66%)) and arts and entertainment (74 out of 115 (64%)).

Figure 6 - Trainee survey: job offers at the end of a traineeship, by sector

Source: The 2022 trainee survey, 2023 study supporting this evaluation

Inclusiveness

Although inclusiveness is not a principle in the QFT, a large variety of stakeholders consulted in the 2023 study supporting this evaluation, including national authorities, the public employment services, youth organisations and the European Disability Forum, reported that vulnerable young people were less able to benefit from traineeships.

For example, young people from rural areas and in the outermost regions
[89](#footnote89)
, with disabilities, from a lower socio-economic background, with a migrant background, from the Roma community, or with lower educational attainment were identified as groups that may face obstacles in access, often linked to financial support. Young people from rural areas may often choose not to participate in traineeships in urban areas, as this would mean leaving their home regions without the guarantee of being hired afterwards. This suggests that additional support might be necessary for these groups. Companies were also reported to hire those trainees that are most skilled and/or educated, thus hampering access for young people with lower education level or skills. In addition, young people without networks (acquaintances, families, etc.) are likely to be less able to access traineeship opportunities, as traineeships are often found through such networks
[90](#footnote90)
.

Another barrier for vulnerable groups to access traineeships is the lack of traineeship programmes tailored to their needs, for example catering for specific training or upskilling needs or accessibility requirements in the case of disabilities.

To overcome such obstacles, some Member States have introduced measures to help vulnerable young people access traineeships. The following box presents some such examples.

Inclusive traineeships - Examples

Latvia: ALMP traineeship providers in Latvia must provide an employment contract in accordance with the Labour Law. For a trainee with disabilities, the State provides support for making adjustments to the workplace up to €1,000.

Ireland: A number of paid traineeships within the civil service are reserved for individuals from underrepresented groups, including those with a disability.

Italy: In Italy, the maximum duration of a traineeship is 12 months. However, this is extended to 24 months for people with a disability to take into account the additional time they may need to get used to the workplace.

Impacts on cross-border traineeships

The QFT (principle 16) recommends also that cross-border mobility of trainees in the European Union be facilitated inter alia by clarifying the national legal framework for traineeships and establishing clear rules on hosting trainees from, and the sending of trainees to, other Member States and by reducing administrative formalities. Principle 17 of the QFT recommends examining the possibility to make use of the extended EURES network (and to exchange information on paid traineeships through the EURES portal).

Though cross-border traineeships seem to have increased over the years (363 out of 1,912 (19%) of the 2022 trainee survey respondents had undertaken a cross-border traineeship), barriers remain. These barriers are linked to the lack of (financial) resources and information. In the trainee survey:

·478 out of 1,293 (37%) indicated that a lack of financial resources limits their possibilities to undertake a traineeship abroad;

·478 out of 1,293 (37%) also mentioned a lack of interest as their main reason for not looking for a cross-border traineeship, which might indicate a broader need to further disseminate information on traineeships and their positive effects on young people’s career pathways;

·349 out of 1,293 (27%) of trainees surveyed mentioned not being well-informed about cross-border traineeships.

The trainee survey indicated that 239 out of 1,836 (13%) of respondents made use of the EURES portal to find cross-border traineeship opportunities. The study supporting the Commission’s ex-post evaluation of EURES
[91](#footnote91)
 indeed highlighted difficulties in providing specific support services targeting youth regarding inter alia traineeships, mostly due to persisting differences in and a lack of harmonisation of national legislative frameworks for traineeships outside of education (e.g. legislative uncertainties regarding the definition of trainees).

Efficiency

It is difficult to provide quantitative estimations of the costs and benefits of the application of the QFT principles, even more so as it proved challenging to disentangle the benefits and costs related to providing traineeships in general from any additional or different benefits or costs related to the application of the QFT principles specifically. The analysis on costs and benefits is based on stakeholder consultations, the trainee survey and case studies that were carried out as part of the 2023 study supporting this evaluation
[92](#footnote92)
. While evidence on quantifiable benefits is challenging to obtain, the study provides qualitative evidence
[93](#footnote93)
 of a range of benefits of quality traineeships for young people, employers and society.

Benefits for young people

Benefits of the QFT for young people include: an increased number and increased quality of traineeships, reduced levels of exploitation of trainees including a better understanding of their rights, and an increased level of validation and certification of learning outcomes. National interviewees from Estonia, Cyprus, Ireland, Luxembourg, Poland and Slovakia noted that the QFT can be linked to reduced exploitation of trainees, by highlighting the importance of traineeships having a written contract/agreement. National consultees from Malta and Luxembourg stated that the obligation to have such contracts/agreements in place has helped trainees understand their rights. National stakeholders from Croatia noted that ALMP traineeships designed to meet the QFT have improved trainee rights, putting them on the same level as other employees. According to 1,338 out of 1,836 (62%) respondents to the trainee survey, traineeships also facilitate young people’s transition to the labour market. Cypriot national authorities noted that QFT implementation had a positive impact on the results of ALMP traineeships, with 77% of trainees in the personal care sector in 2017-2018 employed in that sector after their traineeships.

The trainee survey also highlighted key traineeship benefits such as trainees meeting their learning objectives and gaining skills. This is illustrated by 1,432 out of 1,836 (78%) respondents agreeing or strongly agreeing that their traineeship helped them achieve their learning and training objectives, and 1,506 out of 1,836 (82%) agreeing or strongly agreeing that they gained practical experience and relevant skills. Irish national stakeholders reported benefits for young people through improved skills, noting that the QFT helped highlight how their traineeship offer could be improved to align with best practice. A Career Traineeships pilot based on QFT principles was developed, and an evaluation
[94](#footnote94)
 of the pilot found that trainees enhanced their knowledge, skills, and proficiency in real work settings.

Benefits for employers

Based on the interviews carried out by the national experts with national stakeholders in the context of the targeted consultations
[95](#footnote95)
, benefits for employers include an increased understanding of the quality aspects of a traineeship and more transparency about traineeship requirements, including the rights and obligations of employers. These allow for improving the quality of traineeships which, in turn, can improve the employer’s reputation and capacity to attract better candidates. Quality traineeships also enable employers to benefit from financial incentives
[96](#footnote96)
 that are available, such as in the case of ALMP traineeships, to fill skills gaps in their companies by attracting new talent that they can train for their individual/specific future skills needs, and to improve productivity and reduce turnover by investing in these individuals’ professional development and career progression. Furthermore, quality traineeships allow employers to ‘try out’ workers. Consultees from Finland and Malta noted that the QFT benefits employers through making traineeship requirements transparent. In particular the requirement to have a contract in place benefits employers as it clarifies their rights and obligations.

Benefits for the society and the economy

The QFT has supported youth employment in the EU by providing a more stable EU-wide quality framework for easing the school-to-work transition via traineeships, by giving young people job-based skills and a chance to demonstrate their potential to employers. In the Irish case study it was reported that “[b]enefits for society include increased employability as a result of the production of relevant skills tailored to employment needs”
[97](#footnote97)
. However, it should also be noted that national authorities stressed the difficulty of assessing social impact, due to the problem of isolating the specific effect of the QFT on traineeships
[98](#footnote98)
.

Costs

The 2023 study supporting this evaluation has also examined the costs associated with the implementation of the QFT. Costs identified for employers include concluding agreements with public employment services or training centres, supervising the trainees, assessing and certifying the trainees’ skills, advertising the traineeship opportunities, and developing training plans. These costs are difficult to quantify and breaking them down to costs associated with offering traineeships in general versus costs associated with implementing the QFT as such proved to be impossible. Nevertheless, the following (qualitative) analysis describes administrative and adjustment costs linked to traineeships in general and provides indications, where possible, as to which costs may be attributed to the QFT.

Administrative costs for implementing OMTs and ALMP traineeships differ, with ALMP traineeships offering many more forms of financial support for employers, while entailing also administrative costs both for employers and for public authorities. Additional costs for public authorities include investing in public services and labour inspectorates to monitor compliance with national legislation. Applying for and managing subsidies is itself costly to employers, especially for small companies with limited capacity for these administrative tasks. In the Irish case study
[99](#footnote99)
, for example, stakeholders emphasised the significance of administrative costs for employers due to higher levels of quality assurance and compliance with contractual frameworks for ALMP traineeships. However, none of these costs can be directly attributed to the QFT.

Adjustment costs in adapting to the requirements of QFT principles were also noted by some of the consulted stakeholders, including: the costs associated with designing programmes, implementing new legislation, assessing trainees’ skills at the start of a traineeship, developing training plans to ensure learning objectives are met, supervising trainees, and certifying trainees’ skills at the end of a traineeship.

In general, costs were raised as a limiting factor for applying the QFT principles. National authorities and employers mentioned limited human and financial resources (mainly linked to the need to provide supervisors, specify learning objectives and cooperate with the public employment services in case of ALMP type of traineeships) as an obstacle. National authorities also referred to their limited resources for the promotion and communication about traineeships.

No evidence was found of significant enforcement costs associated with the implementation of the QFT and stakeholders did not identify such costs in the interviews or case studies. The key issue here is that enforcement of traineeship regulations would normally be undertaken by the national labour inspectorate or equivalent and any costs would be incurred by the relevant national authorities. Such inspections, however, normally focus on the enforcement of the relevant national legislation. Even where that legislation has been influenced to some extent by the QFT it is not possible to separately identify enforcement costs due to the QFT.

16 Member States use financial incentives to support quality improvements in traineeships. They are generally used in three main ways: to encourage or enable traineeships; to improve their quality; and/or to enable trainees to be kept on at the end of their traineeship. Financial incentives are mainly used in the case of ALMP traineeships.

Table 6 - Financial incentives in Member States

|  |  |
| --- | --- |
| Type of incentive | Member State |
| To encourage or enable traineeships | BG, CY, DE, EL, ES, HR, IT, LU, LV, MT, SK, SI, EL, PL, RO |
| To improve traineeship quality | BG |
| To enable trainees to be hired following their traineeship | BG, SK, IE, ES |

These types of incentives include public subsidies and/or social security contribution reductions for companies providing traineeships. Member States have also relied on financial support from European Structural and Investment Funds, such as the Youth Employment Initiative and the European Social Fund (ESF), as recommended by the QFT (principle 18). In targeted consultations
[100](#footnote100)
, public employment service representatives highlighted that employers can receive grants to support wages for mentors and supervisors. A national trade union noted that ESF funding is also used by employers, but requires a great deal of administrative commitment in terms of reporting.

Examples of financial incentives and use of European funds

Spain

The case study on Spain
[101](#footnote101)
 gives the example of subsidies to employers for ALMP traineeships funded through the Youth Guarantee in November 2020. The subsidies ranged from €694.82 to €810.62 per month per trainee, equal to 1.2 to 1.4 times the IPREM
[102](#footnote102)
 (with the higher rate applying to trainees with disabilities). Subsidised employers must pay a wage equal to 80% of the IPREM and social security contributions and must justify the training content of their traineeships. The implication is that the difference between the wage and the subsidy (i.e. 40-60% of the IPREM) covers the extra cost of providing the traineeship.

Bulgaria

The New Opportunity for Youth Employment initiative offers employers incentives to hire young people up to the age of 29, and subsidises the cost of a supervisor, transport costs and wages for trainees. This initiative also includes an exemption from a number of social security contributions if the employer subsequently signs a permanent contract with the trainee.

Germany

Employers who take on young people for in-company introductory training receive a subsidy that covers trainees’ pay and a lump sum for the trainees’ total social insurance contributions.

Based on the case studies and targeted consultations conducted in the 2023 study supporting this evaluation
[103](#footnote103)
, the most often held perception of stakeholders is that the total costs related to the implementation of the QFT are proportionate to the benefits. Throughout the research
[104](#footnote104)
, stakeholders highlighted benefits more often than costs, and while quantification was difficult, their inability to identify significant costs associated with the QFT in its current form suggests that such costs are unlikely to be large enough to outweigh the benefits that were more frequently identified.

Among the factors limiting the efficiency of the QFT are a lack of monitoring and enforcement and a lack of awareness. In particular, some employers do not seem to be aware of the potential benefits of offering a quality traineeship, nor of funding opportunities available to support employers offering such traineeships. In targeted consultations
[105](#footnote105)
, employer representatives in particular emphasised that the implementation of the QFT principles is also likely to be less efficient in small companies and micro-enterprises than in larger firms.

Coherence with other policies supporting quality jobs for young people

At EU level, the objectives, target groups and measures to implement the QFT display overall a good level of coherence with other relevant EU initiatives, policies as well as funds and programmes.

The QFT contributes to the European Pillar of Social Rights
[106](#footnote106)
, in particular principle 4 on ‘Active support to employment’ and the objectives of the 
[EU Youth Strategy](https://europa.eu/youth/strategy_en)
, in particular goal 7 ‘Quality Employment for All’, which, amongst others, calls for the recognition and validation of competencies acquired through internships, apprenticeships and other forms of work-based learning, volunteering and non-formal education as well as equal access to quality information and adequate support mechanisms to prepare young people for the changing labour market and future of work
[107](#footnote107)
. In addition, the QFT is coherent with the 
[Reinforced Youth Guarantee](https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32020H1104(01)&from=EN)

[108](#footnote108)
, which aims to ensure that all young people receive a good quality offer of employment, continued education, an apprenticeship or a traineeship within four months of becoming unemployed or leaving formal education and recommends to ensure that traineeship offers adhere to the minimum standards laid out in the QFT. Coherence with the 
[European Solidarity Corps](https://europa.eu/youth/solidarity_en)
 is found in the projects funded and supported under this initative, amongst which traineeships. Synergies also exist with other EU initiatives on traineeships and apprenticeships, such as the 
[Digital Opportunity Traineeships](https://erasmusintern.org/digital-opportunities)

[109](#footnote109)
 funded under Erasmus+, which aims to help companies fill vacancies with digitally competent candidates through traineeships. and the 
[European Framework for Quality and Effective Apprenticeships](https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1526484102559&uri=CELEX:32018H0502%2801%29)
 (EFQEA) that sets out similar quality principles for apprenticeships (more details below). It is also coherent with the goals of the main relevant EU funding mechanisms, including the Youth Employment Initiative (YEI)
[110](#footnote110)
, the ESF+
[111](#footnote111)
, NextGenerationEU
[112](#footnote112)
 and Erasmus+
[113](#footnote113)
.

As for coherence with relevant EU labour legislation, the Directive on adequate minimum wages in the European Union
[114](#footnote114)
 applies to (paid) trainees insofar as they fall under the definition of ‘worker’, as defined by national law with consideration to the case-law of the Court of Justice of the European Union. The Council Recommendation on Access to social protection for workers and the self-employed
[115](#footnote115)
 similarly only addresses the situation of trainees who are in an employment relationship.

Overall, there is also a fairly good level of coherence between the objectives, target groups and measures to implement the QFT, and the relevant employment, social, and education and training policies at national and regional level. These policies have the common goal to provide young people with relevant practical work experience and skills in a safe environment in which their rights are protected.

The way various national or regional measures are implemented might however lead to varying degrees of coherence across Member States. Overall, the majority of national and regional stakeholders consulted in the 2023 study supporting this evaluation
[116](#footnote116)
, including policymakers, social partners, youth organisations and chambers, were of the opinion that there was a better coherence of policies at EU level than at national level. These views often referred to the QFT not being fully integrated in national or regional policy, but only to a certain extent.

In particular, the degree of coherence with the QFT is generally higher for employment policies, as compared to policies in the fields of education, training or social inclusion. This reflects the fact that the QFT excludes traineeships which are a mandatory part of formal education or training programmes, and that the responsibilities for labour market and education policies are often dispersed between different parts or levels of government. The 2023 study supporting this evaluation pointed out that due to the complexity of labour law and its interaction with education and training systems, there remain some grey areas in which there is potential for duplication and incoherence. This notably concerns the interaction of policies supporting traineeships with those supporting apprenticeships.

In this regard, the European framework for quality and effective apprenticeships (EFQEA)
[117](#footnote117)
 of March 2018 sets out 14 criteria for quality and effective apprenticeships. There are many similarities with the QFT, for example addressing dimensions such as the written agreement, learning objectives or outcomess, transparency and working conditions, as well as EU-level monitoring and cooperation with other key stakeholders. Most stakeholders consulted under the 2023 study supporting this evaluation are of the opinion that the EFQEA has led to greater involvement of stakeholders, and has a clearer scope, a higher and more specific quality ambition in terms of more direct recommendations, a higher degree of regulation of apprenticeships, which are generally considered as employment contracts, and benefits from useful supporting measures under the European Alliance for Apprenticeships (EAfA) and the Apprenticeship Support Services. These perceptions are to be seen in light of fundamental differences between apprenticeships and traineeships, whereby apprenticeships are required to lead to a full qualification of professional or vocational education and training, while traineeships covered by the QFT are, at most, complimentary to education and training. In addition, apprenticeships are highly regulated, often on a tripartite basis, whereas traineeships (in particular the open market type) are either unregulated or partially regulated
[118](#footnote118)
.

There are also different degrees of coherence within the field of national employment policies. Policies on ALMP traineeships, which by nature are designed in the context of national employment integration measures, tend to be better aligned with the goals of the QFT. This is less so for policies regarding OMTs, due to the diversity of regulatory approaches to traineeships across the EU and the prominent role of employers in the determination of the conditions for these types of traineeships.

Some national and regional stakeholders (including policymakers, business representatives, education and training organisations and social partners) suggested (during interviews and expert meetings held in the targeted consultations
[119](#footnote119)
) a widening of the scope of the QFT to cover more types of traineeships, including traineeships which form a part of education and training programmes. However, at the validation workshop (September 2022) that took place in the context of the 2023 study supporting this evaluation
[120](#footnote120)
, some employer organisations and national authorities raised concerns that broadening the scope of the QFT could give rise to discrepancies with existing legislation on compulsory traineeships and/or traineeships that are part of education and might further complicate ongoing discussions on issues such as remuneration.

Finally, in some Member States (e.g. MT, RO, SK), national and EU funding has been a driver for promoting coherence with the QFT. National policymakers in Romania and Slovakia particularly underlined the role of the Youth Guarantee in ensuring the QFT principles were implemented. In Malta, youth organisations highlighted the role of national and EU funds (especially the ESF) in promoting synergies and coherence with the QFT, in particular by bringing national and regional policymakers from relevant fields (education and training, employment, social) together.

4.2.
   How did the QFT make a difference and to whom?

Despite the challenges in isolating the impact of the QFT from the impact of pre-existing national measures and of traineeships in general, the 2023 study supporting this evaluation suggests that the QFT has added value to the design and implementation of policies on both OMTs and ALMP traineeships in several Member States. The added value was found to have been greatest to national and regional level policy-makers.

In particular, the QFT has provided a common EU reference point for quality in traineeships, which has helped national authorities to align and improve national systems and legislation:

·In Bulgaria, interviewees stated that the QFT has specifically driven legislative changes regulating OMTs.

·In Spain, although legislation on traineeships was already in place, more recent legislative amendments are in line with the QFT, improving legislative and possibly also practical implementation.

·In Ireland too, it was reported that the QFT provided a framework for the development of the country’s own national policy on traineeships, and was of great benefit in terms of helping policymakers to do this quickly and effectively.

·In Greece, the QFT has been the impetus for a range of practical improvements to traineeships, such as putting into place a register of traineeship providers, a register of companies, and a code of ethics for traineeships.

The extent of the EU added value varies according to factors such as whether instruments and measures were already in place prior to the QFT. The EU added value appears to have been greatest in Member States where traineeship systems were less developed before the adoption of the QFT. This is supported by views gathered in the targeted consultations
[121](#footnote121)
 with representatives from the newer Member States. Stakeholders from Croatia
[122](#footnote122)
 for example highlighted that the QFT helped to lay the groundwork and set policy standards that had an extra weight in the national context. In Romania, recent legislation on traineeships was modelled on all the principles listed in the QFT. In Bulgaria, stakeholders emphasised that, since 2014, traineeships offered by large multinational companies have become more structured, including an individual traineeship plan and a mentor.

National authorities (such as ministries of labour and education and public employment services) all agree on the added value of the QFT in setting out a common EU framework as well as its impact on the ground. Youth, civil society and trade unions generally agree too, but they also consider that the EU added value is somewhat limited by the weaker application of the QFT on the ground.

As for employer organisations, the added value lies in an increased understanding of the quality elements of traineeships, and in case the QFT principles are applied, reputational benefits. Some employers are of the opinion that the QFT has limited added value because labour markets vary substantially across Member States, while others highlight that the QFT’s added value precisely lies in its flexible and non-binding nature.

4.3.
   Is the QFT still relevant?

The principles of the QFT remain relevant for improving the quality of traineeships and fostering the labour market integration of young people. Relying particularly on the public consultation survey, the targeted consultations (in particular the interviews by national experts with national stakeholders and the trainee survey), and the case studies carried out in the 2023 study supporting this evaluation
[123](#footnote123)
, stakeholders agree that while most QFT principles are pertinent, the following elements from the QFT tend to be particularly associated with better post-traineeship outcomes: learning objectives
[124](#footnote124)
, written agreement
[125](#footnote125)
 and the provision of a supervisor
[126](#footnote126)
. 

There were more divergent views on the relevance of limiting the duration of traineeships. On the one hand, a limited duration can prevent the replacement of regular jobs by traineeships
[127](#footnote127)
. On the other hand, sufficiently long traineeships can allow employers to see them as an investment in their future workforce and enable young people to acquire relevant competences
[128](#footnote128)
.

As for the relevance of the QFT itself, there are mixed views on whether the QFT should be revised, remain in place or even be discontinued. While a few stakeholders (from different stakeholder groups in BE, EL, FI, IE, IT, LT, LV, PT, RO) consider that the impact of a discontinuation would be limited given the level of integration of QFT principles in their national legislation
[129](#footnote129)
, other stakeholders (also from various stakeholder groups in AT, BG, DE, EL, IT, MT, PL) have underlined the importance of the existence of a common policy guidance at EU level which continues to give an impetus for the development of national policies on traineeships.

Employer organisations and national authorities generally called for retaining the QFT in its current form, while introducing flanking measures to support a better implementation on the ground, such as greater awareness raising and mutual learning, robust and comparable data on traineeships across the EU, more quantitative monitoring of implementation and a better coordination amongst different actors. The potential added value of such measures was also highlighted in the European Parliamentary Research Service (EPRS) study.
[130](#footnote130)
 

Trade unions and youth organisations, however, call for a more significant alteration of the nature and content of the QFT, notably by making its implementation binding, expanding its scope and adding additional quality elements
[131](#footnote131)
.

The 2023 study supporting this evaluation shows that the non-binding nature of the QFT has made it possible to take into account the diversity of national education and training and labour market environments and strike a balance between the need to ensure minimum standards and preserve a degree of flexibility – a view largely supported by national authorities and employer organisations
[132](#footnote132)
. At the same time, it has also allowed different regulatory approaches at the national level to continue, resulting e.g. in different levels of quality standards for traineeships, as highlighted by trade unions
[133](#footnote133)
.

National employer organisations highlight that having a set of binding principles that are applicable to all Member States would not be appropriate for fostering the stable labour market integration of young people because of the important diversity in national regulation on traineeships, in labour law and in the degree of social partner involvement in defining traineeship standards
[134](#footnote134)
.

Employer organisations and trade unions (AT, DE, DK, FI) point out that the relevance of the QFT also depends on the level of general social protection and labour standards applied in individual Member States. In Member States where these standards are generally high (such as AT, DE, DK, FI), the QFT is considered less relevant. Against this background, the flexibility and adaptability of the QFT in function of varying labour market needs were considered important.

In addition, national authorities, employer organisations, education and training organisations and trade unions considered
[135](#footnote135)
 that the scope of the QFT, limited to OMTs and ALMP traineeships, was a factor reducing its relevance for fostering stable labour market integration. Extending the scope to traineeships which are part of formal education or training programmes, could increase the relevance of the framework, as such traineeships also aim to increase young people’s employability.

The 2023 study supporting this evaluation
[136](#footnote136)
 also indicated quality principles which are considered to be missing from the QFT. Stakeholders from youth organisations as well as trade union representatives and some national authorities highlighted that the QFT currently does not sufficiently take into account that youth is not a homogeneous group. As a result, the QFT may thus be less relevant in addressing the specific needs of young people and to the broader Union aim of social cohesion and inclusion. Therefore, a possible additional quality element could concern ensuring a greater recognition of the diversity of situations of young people, in particular when it comes to the outreach to and accessibility of traineeship opportunities for vulnerable groups.

Remuneration has the potential to increase the quality of traineeships, which has been demonstrated also by previous research. For example, a study done by the International Labour Organization
[137](#footnote137)
 illustrates the tendency of remunerated traineeships to be more structured and formalised, thereby increasing the likelihood that young trainees gain relevant skills and competences. While trade unions, youth and civil society organisations as well as EU and international agencies consider remuneration an important aspect, not all employers consider it as an element increasing the quality of traineeships. Some employers argue that there are also negative consequences of remuneration, such as additional costs for the employer and a reduced numbers of traineeship offers, but also a lack of clarity on the legal status of trainees.

The current QFT only recommends transparency on whether an allowance or compensation is applicable, and if applicable, its amount. In the trainee survey, 220 out of 449
[138](#footnote138)
 (49%) of the trainee survey respondents would like to find information in the vacancy notice about entitlement to compensation or allowance and 193 of 449 (43%) information about its amount. As for being paid (not a QFT principle), 1,597 of 1,836 (87%) of trainees surveyed reported that being paid for their traineeship was essential or important for them. 1,542 of 1,836 (84%) of trainees surveyed reported that being paid at least the minimum wage was important or essential.

Remuneration is an incentive for trainees to remain in and complete a traineeship, and could contribute to guaranteeing equal access to traineeships and to the labour market. However, legislation on remuneration, if set at a low level, may prolong the labour market transition for young people and drive labour standards down. For example, in Croatia, a 2018 evaluation
[139](#footnote139)
 of a traineeship scheme found that the low wages of trainees in the scheme increased the probability of inactivity, and thus delayed their entry into the labour market. The reason for this has been linked to low wages of trainees (at the time between 29% and 43% of the average net wage) which disincentivised young people from taking part in the programme. The impact on wage standards was also noted, with stakeholders highlighting that the scheme lowered labour costs of young people and other workers. This potential adverse effect was also underlined in the case study on Italy
[140](#footnote140)
, in which interviewed stakeholders expressed concerns about the COVID-19 crisis resulting in a further increase in the number of traineeships (as opposed to entry level jobs). These stakeholders cautioned that an increased use of unpaid or low-paid traineeships might have a negative effect on entry-level wages for youth.

Young people’s access to social protection varies between Member States
[141](#footnote141)
, and the entitlements for trainees are not always clear. Research
[142](#footnote142)
 highlights that, currently, in the context of OMTs, employers do not have an obligation to contribute to social security for trainees in 10 out of 27 Member States
[143](#footnote143)
. For the remaining 17 Member States, social security coverage is either at the same level as any employee or only includes specific types of coverage.
[144](#footnote144)
 Against this background, many stakeholders, in particular youth and civil society organisations, consider that guaranteeing trainees’ access to social protection coverage would increase the relevance of the QFT. The trainee survey highlighted in particular the importance of the following aspects of social security:

­access to benefits related to accidents at work and occupational diseases (3,295 out of 3,787
[145](#footnote145)
 (87%) of trainee respondents considered this essential or important)

­being covered by health and sickness benefits (3,181 out of 3,787 (84%) of respondents); 

­having access to paid sick leave (2,991 out of 3,787 (79%) of respondents); 

­access to minimum income support ((2,954 out of 3,787 (78%) of respondents); 

­accumulating pension rights (2,613 out of 3,787 (69%) of respondents); 

­having access to unemployment benefits (2,535 out of 3,787 (67%) of respondents).

The public consultation carried out by the Commission confirmed these views.

Although the COVID-19 pandemic encouraged innovative approaches to traineeship supply and delivery, it has evidenced young people’s vulnerability to crises. It has moreover highlighted that the QFT might not sufficiently address changes in the labour market, including the increasing role of remote working and digital skills. Provisions on telework and online guidance are currently not included in the QFT and could increase its relevance.

Traineeships can be an opportunity for employers to address increasing skills mismatches due to megatrends, such as the digital transformation of the economy. Some employer organisations have suggested
[146](#footnote146)
 an increased focus on the learning component of traineeships. They notably argued that a mapping of traineeship providers’ skills needs and an assessment of trainees’ competences before and after their traineeships could strengthen the learning element in traineeships. As regards the acquisition of relevant skills, trainees themselves emphasised the importance of access to training activities during their traineeship.
[147](#footnote147)

Another important element is the provision of adequate mentorship, even more so in the context of digital traineeships
[148](#footnote148)
. However, some employer representatives and national authorities warn about additional costs, especially for SMEs
[149](#footnote149)
.

Consultations with EU and national stakeholders (in interviews as part of the targeted consultations)
[150](#footnote150)
 gave indications that the QFT may be lacking practical guidance for employers / traineeship providers on the overall implementation of the QFT. Conditions for employers to receive financial support for traineeships, for example by establishing trainee quotas, or by ensuring that the provision of EU and national funding and/or tax exemptions is subject to conformity with the QFT principles could, according to EU and national stakeholders consulted
[151](#footnote151)
, strengthen the relevance of the QFT. Finally, the QFT could also encourage companies to employ trainees after the traineeship, or help them find a job after the traineeship (i.e. post-placement support).

5.
   Conclusions and lessons learned

5.1 Conclusions

Traineeships continue to be an important pathway for young people to enter the labour market, whilst learning “on the job”. Quality traineeships, which reflect the principles of the QFT, contribute to increasing young people’s employability in terms of their skills and their access to professional networks, as highlighted in particular by the views of trainees through the trainee survey. Quality traineeships also bring distinct opportunities and benefits for employers to attract, train and retain young talent. This is particularly relevant in the context of the pronounced skills mismatches and skills shortages, which especially affect some sectors in the EU labour markets.

It needs to be reiterated that a key challenge in assessing the impact of traineeships and the QFT relates to the lack of systematic data collection at EU level and in Member States. Moreover, it is difficult to disentangle the impact the QFT may have had on the quality of traineeships and on youth employment from the impact of other factors (such as the economic situation, national contexts, existing national legislation and/or policies).

In terms of effectiveness, the provisions of the QFT that are perceived as most impactful on young people’s labour market integration seem to be those outlining the need to determine the learning and educational objectives of the traineeship and the written agreement.

The QFT provided added value as a reference point at EU level for Member States’ regulatory action on the quality of traineeships, in particular in Member States with less developed traineeship systems, where it helped to foster policy and legislative changes at national level. Since 2014, 14 Member States introduced legislative changes to cover traineeships
[152](#footnote152)
, which in some cases may be linked to the existence of the QFT though the analysis does not allow to establish a systematic causal link given that it is a non-binding instrument. The prevalence of specific legislative provisions as well as their conformity with the principles of the QFT were found to be higher for ALMP traineeships than for OMTs. Overall, a slight improvement in terms of conformity can be observed for OMTs with four Member States fully/mostly aligned in 2016 versus seven Member States in 2021. A larger improvement is observed for ALMP traineeships, with 18 Member States fully/mostly in conformity in 2021 versus 15 in 2016.

In addition, irrespective of the degree of implementation of the QFT principles in national legislation, there is a general room for improvement when it comes to the application of the QFT principles on the ground, as well as their monitoring and enforcement.

As for the QFT facilitating the cross-border mobility of trainees in the EU, there are indications that the number of transnational traineeships has increased, but young people still face difficulties due to a lack of financial means and the unavailability of relevant and sufficient information.

The evaluation lacked quantitative evidence on costs and benefits. Based on the views expressed by the consulted stakeholders, the costs appear to be low, while the benefits are considered relatively high. Overall, the cost for employers related to the implementation of the QFT is generally perceived as proportionate to the benefits, such as a better understanding of the determinants of traineeship quality, reputational advantages, an increased attractiveness to young talent, and a more affordable way to invest in potential future workers. However, the implementation of the QFT can be less cost effective for small and micro enterprises than for larger firms. Financial incentives, such as those supporting job offers after the completion of a traineeship, can help increase the benefits to traineeship providers compared to the costs involved. Benefits of the QFT for trainees include improved working conditions, as well as better training and learning content, which increase their chances of entering stable employment.

The QFT was found to be coherent with the objectives and actions of other EU level initiatives, strategies, programmes and funding instruments. As for coherence with national and regional policies, the evaluation found a mixed picture, with a higher level of coherence with measures on ALMP traineeships than with those on OMTs, where coherence was found to be more limited. Also, a higher degree of coherence was found with national and regional policies in the field of employment, compared to the policy fields of education, training and social policy.

Despite the many similarities in quality principles, the European framework for quality and effective apprenticeships (EFQEA)
[153](#footnote153)
 was perceived as leading to a greater involvement of stakeholders, and having a clearer scope, a higher and more specific quality ambition and benefits. These perceptions are to be seen in light of fundamental differences between apprenticeships and traineeships, where apprenticeships are required to attain full qualification of professional or vocational education and training, while traineeships covered by the QFT are, at most, complimentary to education and training. In addition, apprenticeships are highly regulated, often on a tripartite basis, whereas traineeships (in particular OMTs) are either unregulated or partially regulated.

Although the QFT is deemed still relevant and of added value, various stakeholder groups considered that it could be further strengthened, in particular when it comes to remuneration and social protection for trainees. However, not all employers consider remuneration an important aspect; they highlight also the negative consequences, such as additional costs for employers. Furthermore, outreach to as well as access for vulnerable groups was deemed important. Young people from rural areas, from a lower socio-economic and/or migrant background and with lower educational attainment were identified as groups that may face obstacles in accessing traineeship opportunities. In addition, there may be a need to update the QFT to the reality of today, notably when it comes to the increased practice of telework and the need to ensure guidance and mentorship in that context. Adding traineeships which are part of formal education and training is also a consideration to be made when it comes to increasing both the relevance and coherence of the QFT. However, not all stakeholders support this view. Some point to potential discrepancies with existing legislation on compulsory traineeships and mention that this could further complicate ongoing discussions on issues such as remuneration. Furthermore, to ease the transition to a stable job, the QFT could place more emphasis on post-placement support to be provided by traineeship providers.

As for the non-binding nature of the QFT, views diverge on whether this is appropriate for the purposes of the framework. While some (notably youth organisations and trade unions) consider the non-binding character as having led to a fragmented implementation of the QFT across the EU, others (notably employers and national authorities) argue that because of the diversity of national situations and limitations of the different instruments available at EU level, the QFT’s non-binding nature offers an adequate and flexible reference framework for national regulations.

5.2 Lessons learned

The collection of comparable data on the prevalence, quality and nature of traineeships across Member States, as well as their impact on youth employment, needs to be significantly improved. With such robust and comparable data, based on a common definition, the effectiveness of the QFT could be more accurately monitored at EU level. On the ground, the effectiveness of the QFT could be improved by national authorities through stronger monitoring and enforcement in conjunction with an increased awareness amongst various key stakeholders, in particular young people and employers. Drawing inspiration from the EFQEA, implementation of the QFT could benefit from a higher involvement of stakeholders (e.g. through dedicated networks) and by encouraging a higher degree of national legislation on OMTs (e.g. through stronger wording of the principles in the Recommendation). Employers could be better assisted in accessing financial support by providing practical guidance as well as by linking such financial incentives to the application of the QFT quality principles. Awareness-raising of the benefits of offering quality traineeships amongst employers would also help the application of the QFT. There is a need to provide more concrete and practical information to young people who may be interested in doing a cross-border traineeship. Raising awareness on EURES could be helpful.

The coherence of the QFT with national policies on employment, education and training and social policies could be further improved through a strengthened horizontal coordination across these policy areas. There is also a need to reflect on the scope of the QFT, in particular on extending it to traineeships which are part of curricula of formal education and training.

Regarding the relevance of the QFT, various stakeholders have recently argued in favour of quality standards that go beyond the QFT principles, condemning in particular the practice of unpaid traineeships and demanding social protection for trainees. For example, the European Parliament
[154](#footnote154)
, the European Economic and Social Committee
[155](#footnote155)
, the Committee of the Regions
[156](#footnote156)
, the ETUC
[157](#footnote157)
, the European Youth Forum
[158](#footnote158)
 and the Conference on the future of Europe have supported a ban on unpaid traineeships
[159](#footnote159)
. On the other hand, employers
[160](#footnote160)
 and national authorities
[161](#footnote161)
 have called for flexibility for Member States on these issues because of the important diversity in national regulation on traineeships, in labour law and in the degree of the involvement of social partners in defining traineeship standards. 

These views underline the findings of the evaluation as regards the question on how the QFT could be complemented by additional quality elements, including fair remuneration and access to social protection for trainees, but also the adaptation of the QFT to the needs of vulnerable groups, its update in light of recent developments such as telework and the increased demand for digital skills, conditions for employers to receive financial support for traineeships and a strengthened support to trainees during the traineeship (e.g. through mentorship) as well as after.

Any revision of the scope or content of the QFT will need to duly take into account the possibilities and limitations, including the principle of subsidiarity, of the different instruments available at EU level. In addition, changes would need to be carefully assessed in the light of potential additional costs, in particular for small and micro enterprises.

Annex I: Procedural Information

Organisation and timing

DG Employment, Social Affairs and Inclusion (DG EMPL) is the lead DG for the evaluation of the Council Recommendation on a Quality Framework for Traineeships of 10 March 2014 (2014/C 88/01).

The evaluation started with the publication of the Evaluation Roadmap
[162](#footnote162)
 on 27 July 2021 and has been carried out with the support of the Inter Service Group chaired by DG EMPL to which the following DGs were invited: CNECT, COMM, EAC, GROW, JRC, JUST, REGIO, SG, and SJ. The group met five times:

·06/09/2021 – roadmap, consultation strategy and technical specifications for the external study

·24/01/2022 - inception meeting for the external study and public consultation survey

·19/05/2022 - draft interim report of the external study

·18/10/2022 – draft final report of the external study

·21/10/2022 – draft evaluation report and way forward

Use of evidence

Both internal and external expertise was used to ensure good quality of the evaluation and related Staff Working Document. The main sources include:

·External study carried out by Ecorys consortium
[163](#footnote163)
. The study has been conducted in line with the Request for Services and the agreed inception. It includes all agreed components and is based on relevant qualitative and quantitative data although less quantitative information and evidence on efficiency was included than expected. The analysis and conclusions are sound while methodology and limitations are clearly outlined. The lessons learnt are relevant.

·The 2022 consultations carried out as part of the evaluation Public Consultation

·The 2021 study ‘Remuneration of Open-market traineeships in EU-27’ by the European Network of Public Employment Services, requested by the Commission

·The 2021 European Social Policy Network (ESPN) Thematic Report: Access to social protection for young people

·The EMCO multilateral surveillance biennial review on the Youth Guarantee in 2019, which had a dedicated part on the quality of traineeships

·The 2016 Commission Communication ‘The Youth Guarantee and Youth Employment Initiative – three years on’ and its the accompanying Staff Working Document 'Applying the Quality Framework for Traineeships'

·European Commission (2013), Impact Assessment accompanying the proposal for a Council Recommendation on a Quality Framework for traineeships (SWD(2013)495)

·European Commission, Flash Eurobarometer 378: the experience of traineeships in the EU, 2013

·European Commission (2012), Study on a comprehensive overview of traineeship arrangements in Member States.

·The 2012 Analytical document accompanying the Communication Towards a Quality Framework on Traineeships – Second-stage consultation of the social partners at European level under Article 154 TFEU (SWD(2012)407)

Annex II. Methodology and Analytical models used

1.Approach to the evaluation and analytical models

The evaluation was supported by an external study using mixed-method data collection approaches, combining qualitative and quantitative research methods within an overall analytical approach guided by a comprehensive evaluation framework. The objectives and purpose of the study, along with the key issues it focused on, indicated the necessity of an approach able to explore processes linked to the implementation of the Quality Framework for Traineeships, as well as their effects and outcomes.

(a)Intervention logic

During the inception phase, the intervention logic was developed by the contractor in cooperation with the Commission (DG EMPL and other services as part of the Interservice Group (ISG). Specifically, the intervention logic set out a high-level understanding of the rationale, inputs, activities, outputs, results and impacts of the QFT and provided a framework for a standardised approach to the evaluation whilst allowing for flexibility to explore the individual provisions called for in the QFT.  The intervention logic provides an articulation of activities, outputs, outcomes and impacts alongside paying greater attention to the causal links between them. This latter aspect is central to our analytical approach to the study particularly in terms of providing the basis to fully test the intervention logic and explore causality to determine the extent to which the QFT led to, and is responsible for, the outcomes and impacts anticipated. 

(b)Assessing process and outcomes

Using the intervention logics as an analytical basis for the evaluation, combined with the evaluation criteria specified in the tender specifications, provided an important underpinning to the required consideration of process-related themes as well as those more concerned with outcomes.

Exploration of processes linked to the earlier parts of the intervention logic, for example in exploring the degree to which inputs supported the implementation of principles of the QFT in national legislation. Assessment of outcomes and impacts focused on the later stages of the intervention logic, exploring how and the extent to which the actions undertaken through the implementation of the QFT led to the intended immediate outputs, longer term results, and broader impacts articulated in the model.

The different methodological approaches including secondary data analysis (Mapping), the consultation activities (Targeted consultations, Public consultation survey, Social Partners’ consultation) and the case studies provided a range of evidence that was triangulated to assess the degree to which process and outcome causal chains detailed in the intervention logic are supported.

However, as far as possible, this analytical approach needed to be complemented by an assessment of causality – i.e., the degree to which the activities implemented in response to the QFT have in reality led to the immediate and longer-term results they intend to generate. A range of other factors (external to the QFT) have influenced the results and impacts of the implementation of the QFT, including the economic context (e.g. levels of youth (un)employment, demand for skilled labour), and other factors, such as the Covid-19 pandemic.

The contractor thus also adopted a contribution analysis approach and operationalised the contribution analysis approach through collating and assessing, in a structured way, the range of evidence gathered from the range of methodologies deployed for the key evaluation tasks. Contribution analysis aims to build a credible ‘performance story’ , drawing upon the available sources of evidence to consider the extent to which the QFT, alongside other factors, contributed towards the observed outcomes. Situated within a wider theory-based evaluation approach, this is ideal for the evaluation, as it provides a way of explicitly defining and assessing the causal relationships and mechanisms within the intervention logic.

(c)Modelling and assessing costs and benefits

As reflected in the evaluation questions (see Annex III) relating to efficiency, it was essential that the study ensured the collection and analysis of available data on costs and benefits of the actions associated with the implementation of the QFT. This data was captured to the extent possible through the research tools developed for the different data collection tasks. The case studies provided the opportunity for more in-depth analysis of costs and benefits of the implementation of the Recommendation in selected Member States, using both qualitative and quantitative data where available.

Conceptually and practically, it is difficult to reliably assess the potential benefits and costs associated with QFT implementation in Member States, for several reasons:

·Benefits/costs that actors (employers, trainees, authorities) are typically aware of are those of introducing and implementing traineeships per se, rather than any additional or different benefits/costs due to adapting traineeships to QFT principles. Many stakeholders reported that costs and benefits associated with the QFT overlap with those of traineships in general and the QFT has not led to the emergence of new types of costs or benefits.

·Awareness of QFT among relevant actors is low. Thus even where benefits/costs can be identified, actors are unlikely to attribute them appropriately to the QFT. This is particularly true in countries, where the traineeship concept is long-established and no concrete changes have been associated with the QFT.

·Member States had traineeships in place prior to the QFT, sometimes closely resembling what is called for in the QFT. Even where specific traineeship developments are in line with the QFT Recommendation, most stakeholders noted that they have not monitored their effect. Hence it is not usually possible to identify which, if any, elements of these developments and their benefits/costs would occur anyway and which can be attributed to the QFT.

·Moreover, even where the QFT has impacted on the trainee landscape, and specific developments can be accurately attributed to the QFT, their often qualitative nature (e.g. improved clarity of contractual terms, educational objectives, rights and obligations) makes it very difficult for actors to quantify associated benefits/costs. The best that can be achieved in most cases is that they can name the benefits/costs and perhaps give some qualitative assessment of their importance.

A robust estimate of benefits/costs associated with QFT implementation requires a study with a different methodology than that used for this study. Specifically it would require (representative) quantitative data collected from the relevant actors, with a counterfactual aspect (e.g. a ‘before and after’ study of QFT implementation over time). The evidence from the stakeholder consultations, trainee survey and case-studies are largely qualitative in nature. Where stakeholders stated that costs or benefits arose directly due to the QFT, this is stated in the text.

(d)Evaluation framework

An overarching evaluation framework which guided the evaluation was developed (see Annex III).   

(e)Evaluation criteria

The evaluation assessed the implementation of the QFT against the five key evaluation criteria of relevance, effectiveness, efficiency, coherence and EU added value. An overview of these criteria in the context of this evaluation are set out in Table 1 below.

Table 1 Overview of evaluation criteria

|  |  |
| --- | --- |
| Evaluation criteria | Overview of key issues |
| Effectiveness | The assessment of effectiveness has been multi-faceted, covering the extent to which the principles of the QFT were effectively implemented (i.e. to meet objectives and expected results), but also how they were implemented and for which sub-groups and sectors they have been most/least beneficial. The analysis has examined the extent to which the QFT principles have been enshrined in and/or influenced national legislation/quality frameworks since 2014, the scope of such national provisions, as well as considering whether they already existed. In parallel, we have explored the existence and effectiveness of enforcement and/or monitoring mechanisms in ensuring adherence to the QFT/national frameworks, considering also any potential adverse effects. In addition, the evaluation explored the impact of QFT implementation on trainees, including their perceptions about the effectiveness of EU and/or national provisions to support quality traineeships in fostering sustainable labour market integration and tackling youth unemployment, and considering again any potential adverse effects. The effectiveness of the implementation of the QFT principles for different sub-groups and sectors within Member States was also assessed where data has allowed. The effectiveness and coherence (see below for the latter) of the implementation of the QFT with that of the EFQEA was compared, taking into account the differences in objectives and target groups. Evidence generated under all study tasks has contributed to the examination of the QFT’s effectiveness. |
| Efficiency | The examination of efficiency has explored the degree to which the objectives of the QFT have been achieved at optimal cost for different relevant stakeholders, and the factors which have contributed to this, including a consideration of the use of different sources of EU and national funding. Addressing the efficiency has involved limited quantitative data, but has been mainly supported by qualitative data from the case studies, existing (national/regional) evaluation reports and data generated from the study’s targeted and public consultations. The analysis includes an assessment of the administrative burden of the implementation and enforcement of the QFT for different stakeholders and at different levels. It has also examined the extent to which financial incentives have been used by Member States to increase the prevalence of quality traineeships, and whether any such incentives included contribution from relevant EU programmes. We have as far as possible based on data availability identified and carried out an estimation of the benefits – and potential benefits – for young people and explored the existence of any further benefits for wider society. The evaluation also explored the main obstacles preventing employers from offering traineeships, and the type of support which would be necessary to overcome them. Bringing together the results of the different analyses, we have provided an assessment of the degree to which the administrative costs of the implementation of the QFT are proportionate to the identified benefits, as well its overall cost-effectiveness, and an assessment of the factors which influenced efficiency. Finally, the study has explored whether the identified benefits could have been achieved at lower cost, by reducing administrative burden without compromising benefits. |
| Coherence | Examining coherence implies the need to assess linkages, synergies, complementarities and potential duplication with related EU, national and regional policies, instruments, initiatives, and recommendations (policy and programme coherence). In the context of this study, we have explored the coherence of the objectives, target groups and measures to implement the QFT within ALMP traineeships and OMTs with both (a) education, training, employment and social policies, at national and regional level and (b) relevant EU initiatives (including the EFQEA as mentioned under effectiveness. Evidence to address the coherence criterion has been drawn principally from desk-based analysis of relevant texts, but also from other sources, notably the detailed Member State level case studies, as appropriate. |
| EU added value | As specified in the Better Regulations Toolbox (tool #47), assessing EU added value involves looking for changes which can reasonably be argued are due to the EU intervention, over and above what could have been expected from national actions by the Member States. Building on the evidence gathered and analysis carried out to assess the other criteria, the study team has assessed the added value of the QFT compared to what could reasonably have been expected by Member States acting solely at national and/or regional levels. We have also examined the likely consequences of both (a) discontinuing the QFT at EU level and (b) continuing the QFT as it stands (no policy change scenario). Detailed evidence from the case studies have been key for assessing the added value at national level, alongside evidence available from results of the public and targeted consultations, as well as the mapping exercise. |
| Relevance | Assessment under this criterion has focused on examining the relevance of the QFT in relation to needs at several levels and from several perspectives. Firstly, we have assessed the degree to which the QFT principles are appropriate for fostering sustainable labour market integration, and which of the principles are most and least relevant in this regard. Secondly, the evaluation has explored the degree to which the QFT principles are still relevant to the main needs within the EU, taking into account a range of developments including the evolution of the concept of quality in traineeships over time, the impacts of the Covid-19 pandemic, the changing labour market and the current perspectives of key stakeholders. Thirdly, the assessment has considered whether any further dimensions were missing in the principles, and the likely impact of their inclusion on the quality of traineeships and their contribution to stable integration into the labour market. Finally, the study has examined the degree to which the status of the QFT as a non-binding Council Recommendation corresponded and continues to correspond to the needs and issues which it seeks to address. Evidence to assess relevance has been gathered through all study tasks, with a particular focus within the targeted consultation, mapping and case study tasks. |

2.Detailed methodology 

In this section, we set out in detail the methodological approach used in undertaking the study supporting the evaluation. We begin with an overview of the methodology, followed by a detailed presentation of the methods and tools implemented for each Task. The figure below provides an overview of the study methodology, including the main deliverables.  

Figure 1 Overview of the study methodology

![](./../../../resource.html?uri=IMMC:SWD%282023%299.ENG.xhtml.SWD_282023_299_ENG_xhtml_04004.jpg)

(a)Inception phase

(I)Preliminary literature review

During the inception phase, the contractor built on the review of relevant documentation carried out for the tender to undertake a wider preliminary literature review on the Quality Framework for Traineeships. This has laid the foundations for the subsequent Tasks of the study, with the following specific purposes:

·To gather further evidence to substantiate the evaluation questions and support in finetuning the evaluation framework;

·To become more familiar with data sources which will be used in the mapping of the state of play of implementation of the QFT against the point of comparison in 2014;

·To gather any further data from DG EMPL that is not in the public domain and that will aid the research, in particular the mapping task;

·To support the final selection of the case study countries;

·To fine-tune the research tools, including the interview topic guides (Task 1.1), the survey trainees (Task 1.2), and the open public consultation questionnaire (Task 4);

·To identify further potential consultees for the targeted consultations.

The contractor reviewed over 100 documents in this step of the inception phase. An overview of relevant resources served as a living document throughout the study and was shared with national experts as a starting point for their research. ‘High’ relevance sources were used as a key input into the mapping task.

(II)Scoping interviews

The contractor conducted two scoping interviews in the inception phase as well as two consultation meetings with the High-level Advisors.

(III)Estimation of prevalence of traineeships

Comparable EU-wide statistical data on the prevalence of OMTs is lacking, which could be linked to the fact that regulatory approaches to traineeships in Member States vary, entailing different categorisations, rights and obligations. As part of the estimation of the prevalence of traineeships in the EU27 for the study, a scoping activity was undertaken to assess the advantages and disadvantages of different ways of estimating traineeship prevalence. The scoping activity focused on Eurostat databases due to their geographical completeness in the area of employment, unemployment, and education. The scoping resulted in the following proposed proxy.

To provide an estimate of the number of traineeships, we used the number of students enrolled in tertiary education
[164](#footnote164)
 for multiplied by the activity rate
[165](#footnote165)
 to estimate the number of young people that may have completed a traineeship. This calculation was completed for the years 2014-2020; for clarity of presentation and given the period of time which the evaluation covers, we chose to display the timepoints in 2014 and 2020, and the evolution over that period. The number of young people estimated to be undertaking a traineeship was then calculated as a share of the total youth population
[166](#footnote166)
.

·Rationale: Based on the study findings, the majority of trainees either are in the process of attaining or have attained tertiary education qualifications. We can then estimate the number of students that may do a traineeship by using the number of students in tertiary education multiplied by the activity rate for that age group (i.e. If there are 100,000 enrolled students in tertiary education and activity rate is 70% we can assume that 70,000 of them would go on to do a traineeship). Using data on the number of people aged 20-34 in Member State, the number of estimated number of traineeships can be expressed as a percentage share of the number of 20–34 year-olds to allow for comparison between Member States
[167](#footnote167)
.

·Advantages: Data is largely available for EU27 and for the evaluation period. This variable corresponds well with the study preliminary findings (graduates from 20 to 29 years undertaking traineeships) and is straightforward to calculate.

·Caveats The main methodological limitation of this proxy is the fact that not all young people who undertake open market traineeships have a tertiary education meaning that the proxy does not capture traineeships of those with lower qualifications.

(b)Targeted consultations

The aim of the targeted consultations was to undertake a set of interlinked targeted consultation activities to help gather the views and opinions of a wide range of relevant stakeholders.

(I)Interview programme 

The interview programme provided one of the main sources of qualitative data for the study. The findings of the interviews were included in the triangulation of data to provide answers to all the evaluation questions relating to the effectiveness, efficiency, relevance, coherence and EU added value of the QFT. Specifically, the interviews helped to examine:

·National approaches to quality frameworks and legislation for traineeships, following the principles of the QFT;

·The impact of implementation of the QFT on the quality of traineeships, as well as on trainees, including any adverse effects, and on youth employment;

·The QFT’s relevance to socio-economic policy needs in the EU, and coherence with other EU and national/regional policies and initiatives on youth employment, education and training, and social policies;

·The costs and benefits associated with the implementation of the QFT for the main stakeholders involved

·The QFT’s EU added value, and any future needs for EU level support.

The interview programme also informed several other tasks of the study. It provided input to the mapping and the labour market review of traineeship offers (vacancy analysis). The interview programme also provided a source of evidence for the selection of and drafting of the case studies. The interview programme focused on two key groups of stakeholders:

·Main national stakeholders in charge of designing, implementing, monitoring, or ensuring compliance with legislative and/or quality frameworks in each Member State: e.g. (sub)-national authorities, public employment services, social partners, education and training providers. This also includes relevant stakeholders such as youth organisations and other organisations representing trainees.

·Relevant EU level stakeholders involved in youth employment policies and funding, with a focus on traineeships. This group included EU institutions, social partners, civil society organisations and other relevant organisations.

The contractor undertook a total of 124 interviews. Of these, 18 were conducted with EU-level stakeholders (including two scoping interviews) as outlined in the table below.

Table 2 Overview of EU level stakeholder interviews

|  |  |
| --- | --- |
| Organisation | Category |
| Eurochambers | EU level social partners |
| SPRINT project | EU-level organisations representing young trainees/young people/workers rights |
| Eurofound | Research and academia |
| Interns Go Pro | EU-level organisations representing young trainees/young people/workers rights |
| ETUC | EU level social partners |
| ILO | Research and academia |
| CESI Youth | EU level social partners |
| European Youth Forum | EU-level organisations representing young trainees/young people/workers rights |
| Eurofound | Research and academia |
| Fair Internship Initiative | EU-level organisations representing young trainees/young people/workers rights |
| BussinesEurope | EU level social partners |
| CEDEFOP | EU level institutions and policymakers |
| DG EAC | EU level institutions and policymakers |
| IndustrialAll European Trade Union | EU-level organisations representing young trainees/young people/workers rights |
| MEP Semedo's office | Europan Parliament MEP |
| DG EMPL, Directorate B | EU level institutions and policymakers |
| DG EMPL, Unit B1 | EU level institutions and policymakers |
| DG EMPL, Unit B3 | EU level institutions and policymakers |

Source: Ecorys 2022

107 national-level interviews were conducted. National experts were asked to conduct up to five interviews per Member States. The following interviews were undertaken:

Table 3 Interviews conducted on the national level

|  |  |  |
| --- | --- | --- |
| National level | | |
| Type of stakeholder | Countries covered | Number of interviews conducted |
| National and regional ministries and government bodies responsible for employment or education policies | AT, BE, BG, CY, CZ, DE, DK, EE, EL, FI, FR, HU, IE, IT, LT, LV, MT, PL, RO, SI, SK | 26 |
| Public Employment Services (PES) | BE, BG, EE, ES, HR, IT, LT, LU, LV, MT, RO, SE, SI, SK | 16 |
| Employer organisations | AT, BG, CY, CZ, DE, DK, EE, EL, ES, FI, HR, HU, IE, IT, LT, LU, MT, NL, PL | 22 |
| Trade unions | AT, BG, CY, CZ, DE, EL, ES, FI, FR, HR, LU, LV, NL, PL, SI | 16 |
| Civil society/youth organisations | AT, BE, CY, EL, ES, FI, HR, HU, IE, IT, LT, LU, LV, MT, NL, PL, RO, SI | 17 |
| Other stakeholders (e.g. research institutes; education and training organisations, etc.) | AT, BG, HU, IE, LV, NL, PL, PT, RO | 10 |
| Total of national level interviews | | 107 |

National experts conducted the interviews on the basis of an interview guide which included questions covering all evaluation criteria, including on quantifiable costs and benefits of implementation of the QFT for different stakeholders. A detailed and clear summary of the data collected during the interviews was gathered in a central interview grid to aid analysis. Overall, the national experts often reported difficulties in scheduling the interviews, particularly due to lack of knowledge of the QFT from national level stakeholders.

(II)Trainee survey 

To gather a comprehensive understanding of the prevalence and quality of traineeship experiences in each Member State, the contractor carried out an internet-based survey targeting the following types of respondents:

·Current trainees: trainees that are currently completing traineeships in one of the Member States.

·Former trainees: trainees that completed a traineeship in one of the Member States in 2014 or after.

·Potential future trainees: young people who have not done any traineeships yet.

The survey questionnaire was translated into all EU official languages. The survey ran from 1 March 2022 to 25 March 2022 and a total of 3,814 responses were collected, out of which 3,787 answers were from EU Member States. Out of these, 1,836 were from respondents belonging to the core target group (i.e. 18-30 year olds with traineeship experience in EU Member States from 2014 onwards). The analysis of survey results was carried out using both quantitative (for closed answers) and qualitative methods (for the open questions). The responses to the open-ended questions provided by the respondents in their native language were translated into English for better interpretation.

(III)Ex-ante online expert meeting

The ex-ante online expert meeting was held on 26 April 2022 and was attended by 36 participants from the national and EU level (see Table below) to discuss national approaches to the implementation of the QFT as well as possible next steps for the future. The outcomes of the ex-ante online expert meeting were triangulated with data and information gathered through the other research activities for this study to come to the final findings.

Table 4 Participants at the online expert meeting

|  |  |  |
| --- | --- | --- |
| Country | Organisation | Role/Job title |
| Austria | Arbeiterkammer Wien | Advisor for internships and political education |
| Croatia | Ministry of Labour, Pension System, Family and Social Policy | Senior Expert Advisor in Service for EU Policies and Implementation of LM related Projects |
| Estonia | Ministry of Social Affairs | Adviser at Work and Pension Policy Department |
| Finland | Confederation of Finnish Industries | Senior Advisor |
| Finland | The Central Organisation of Finnish Trade Unions SAK | Education and Labour Policy Specialist |
| France | UNML | Chargée de mission politiques publiques jeunesse |
| Germany | Federal Ministry of Labour and Social Affairs (Germany) | Ministerial Officer |
| Greece | Ministry Of Labour And Social Affairs | Head Of Vet Department |
| Hungary | Federation of the Chemical, Energy and General Workers’ Unions | Assistant |
| Hungary | Ministry for Innovation and Technology of Hungary | Planning officer |
| Ireland | ETBI | Work Based Learning Manager |
| Ireland | Ministry of Employment | N/A |
| Italy | ANPAL | Researcher |
| Italy | CGIL - Italian general confederation of labour | Head of the European affairs |
| Latvia | Employers' Confederation of Latvia | Sub-project manager |
| Lithuania | Lithuanian Public Employment Service | Head of Measures Implementation Organization Division |
| Lithuania | Ministry of Social Security and Labour of the Republic of Lithuania | Adviser at Labour market group unit |
| Lithuania | Ministry of Social Security and Labour ofthe Republic of Lithuania | Advisor of Labour Market Group |
| Malta | Public Employment Service | Head of Division |
| Malta | General Workers' Union | Secretary General |
| Malta | Ministry for Finance and Employment | Director Policy Development and Programme Implementation |
| Malta | Jobsplus | Department Manager Training Services |
| Malta | Jobsplus | Head of Division |
| Netherlands | SBB | Policy Advisor |
| Poland | Ministry of Family and Social Policy | Chief expert |
| Romania | National Agency for Employment | Inspector |
| Romania | CNSLR Fratia | Youth President |
| Romania | Ministry of Labour and Social Solidarity | Head of Employment Unit |
| Slovak Republic | Confederation of trade unions in Slovakia (KOZ SR) | International Secretary |
| Slovenia | Sindikat Mladi plus (trade union) | president |
| Spain | Spanish Confederation of Employers´Organizations | Senior Advisor |
| Spain | CCOO | International And Youth Secretariats On FSS-CCOO |
| Spain | SEPE - Employement Services Ministry Of Labour-Spain | Head of Unit at Employment Services |
| EU level | European Youth Forum | Policy and Advocacy Manager |
| EU level | ETUC - European Trade Union Confederation | Policy adviser |
| EU level | European Youth Forum | Policy Officer |
| EU level | Eurochambres | Senior Policy Advisor |

(IV)Online validation workshop

The online validation workshop was held on 20 September 2022 and attended by 36 experts, the Commission (DG EMPL) and Ecorys research team (see table below). Participants were sent an input paper in advance outlining the key findings and lessons learned of the study. During the workshop participants provided feedback on the findings which was used as evidence to finalise the study’s findings and the lessons learned. Following the workshop, written inputs were received from two participants, SME United and Business Europe.

Table 5 Participants at the validation workshop

|  |  |  |
| --- | --- | --- |
| Country | Organisation | Role/Job Title |
| Austria | Federal Chamber of Labor / Arbeiterkammer | Education Policy Advisor |
| Belgium | Ministry of Education, Netherlands | education attach'e |
| Belgium | VDAB | Expert work based learning |
| Denmark | Confederation of Danish Employers | Senior Advisor |
| Estonia | Estonian Unemployment Insurance Fund | Service Manager (Employer Services) |
| Estonia | Eesti Tööandjate Keskliit /Estonian Employers´Confederation | haridusnõunik/ education adviser |
| Estonia | Ministry of Social Affairs | Head of ALMPs |
| Greece | Ministry of Labour and Social affairs | Head of vet department |
| Hungary | Ministry for Technology and Industry | planning officer |
| Hungary | Ifjúsági Paktum Egyesület (Pact for Youth Association) | Project assistant |
| Hungary | Pact for Youth Association | president |
| Italy | Confindustria | Education Adviser |
| Italy | Confartigianato Imprese | Policy Advisor |
| Lithuania | The Ministry of Social Security and Labour of the Republic of Lithuania | Advisor ogf the Labour Market Group/National Youth Guarantee Coordinator |
| Lithuania | Ministry of Social Security and Labour | Adviser |
| Malta | Ministry for Finance and Employment | Director, Policy Development and Programme Implementation |
| Malta | Ministry for Finance and Employment | Manager I |
| Malta | Jobsplus | Head of Division |
| Malta | MTRM | CEO |
| Netherlands | UWV/NCO EURES | Business Adviser |
| Netherlands | SBB | Adviser |
| Portugal | Employment and Vocacional Training Institute (Public Employment Service) | Jurist |
| Romania | CNSLR Fratia | President Youth Commission |
| Romania | Ministry of Labour and Social Solidarity | Head of Employment Policies and Social Economy Unit, Employment Policies, Competences and Professional Mobility Directorate |
| Romania | Public Employment Service | Deputy Director |
| Spain | SPANISH CONFEDERATION OF EMPLOYERS ORGANIZATIONS | Senior Advisor |
| Spain | Ministry of Universities | Deputy Director General for Students' support and Institutional Relations |
| Spain | UGT Spain | Project Manager |
| Spain | Servicio Público de Empleo Estatal | Spanish Youth Guarantee Coordinator |
| Spain | Inspección de Trabajo y Seguridad Social (Labour and Social Security Inspectorate) | Inspector |
| Spain | ANECA - National Agency for Quality Assessment and Accreditation | N/A |
| Spain | ANECA - National Agency for Quality Assessment and Accreditation | Project officer |
| EU | SMEunited | Policy Adviser |
| EU | Eurofound | Research manager |
| EU | BusinessEurope | Senior Adviser |
| EU | ETUC, European Trade Union Confederation | Adviser |
| EU | European Youth Forum | Policy and Advocacy Manager | Youth Rights |
| EU | European Youth Forum | Policy Officer | Social and Economic Inclusion |
| EU | Eurochambres | Director |
| EU | Friedrich-Ebert-Stiftung | Policy Officer |

(c)Mapping 

The mapping aimed to:

·Present the situation in each of the 27 EU Member States since 2014 as regards traineeship quality and QFT implementation

·Present a clustering/grouping of Member States as regards traineeship quality and QFT implementation

In order to achieve this, the research team developed an analytical framework covering two aspects of QFT implementation: 1) implementation of specific principles of the QFT in national legislation/frameworks for traineeships; and 2) implementation of the QFT on the ground as detailed below.

1. Implementation of specific principles of the QFT in national legislation/frameworks for traineeships. To assess the degree to which the QFT has been implemented in national legislation and frameworks, the team compiled a standardised list of the QFT principles which can be directly implemented into national legislation/frameworks on traineeships in Member States. These are the principles that refer to standards and rights of trainees and/or the obligations of traineeship providers, that can be ‘transposed’ into legislative and regulatory frameworks on the national and regional level. The list consists of 12 principles as follows:

1.Traineeships are based on a written agreement
   

2.Written agreements indicate educational objectives, working conditions, whether an allowance or compensation is provided and how much, rights/obligations of all parties, duration

3.Tasks allow the trainee to work towards their learning and training objectives

4.Traineeship providers assign a supervisor for the trainee

5.Trainees' rights and working conditions under applicable law are respected including limits to max weekly working time, weekly rest periods, minimum holiday entitlements

6.Traineeship providers clarify if they provide trainees with health and accident insurance and sick leave

7.The written agreement clarifies if the trainee is entitled to an allowance or compensation, and the amount.

8.Traineeship providers include in their vacancies information on the conditions of the traineeship

9.The duration of the traineeship does not exceed six months, except when justified

10.The conditions for an extension or renewal of the traineeship are clarified

11.The written agreement includes information on how the trainee/ traineeship provider can terminate the traineeship
   

12.The knowledge, skills and competences acquired by the trainee are recognised by the traineeship provider through an assessment and a certificate

2. Implementation of the QFT on the ground: this refers to the degree to which the QFT principles – regardless of whether or not they are adequately implemented in legislation – are actually applied to traineeships taking place in practice.

On the basis of this analytical framework, the mapping was then conducted in four stages which are further detailed in the sections that follow.

(I)Desk research at the national level 

This subtask was designed to provide a foundation of evidence on both implementation of the QFT in national legislation and implementation of the QFT on the ground, which was then triangulated with findings from the consultation tasks outlined above.

National experts identified and analysed national sources, mainly relevant statistics and indicators; relevant legislation; national traineeship quality frameworks and monitoring mechanisms; evaluation reports, impact assessments, experts’ opinions, and other relevant academic and grey literature. The national experts gathered secondary data available at the national level, expanding on the evidence available at the EU level. Two briefing sessions were organised for the national experts in February 2021. During these sessions, the Ecorys team outlined the research aims and methodology, and presented the template to be used for national mapping, structured.

(II)Labour market review of traineeship vacancies

The main goal was to contribute to the evidence base for assessing the implementation of the QFT on the ground by checking the current quality of traineeship vacancies in each Member State through primary data collection.

The contractor analysed the alignment of current traineeship vacancies with the principles of the QFT which can be implemented through national legislation and/or quality frameworks governing traineeships (see Table 6 below). The analysis of traineeship vacancies thus focused on three main areas: the transparency of the vacancy, the duration of the traineeship and the working conditions set out in the vacancy.

Table 6.  The labour market review: overview of the QFT principles which can be implemented through national legislation and/or quality frameworks governing traineeships

|  |  |  |  |
| --- | --- | --- | --- |
| Areas of alignment with the QFT principles which can be implemented through national legislation and/or quality frameworks governing traineeships | TRANSPARENCY  of the traineeship vacancy | DURATION  of the traineeship | WORKING CONDITIONS  described in the traineeship vacancy |
| The principles of the QFT Recommendation which can be implemented through national legislation and/or quality frameworks governing traineeships | Written agreement  Insurance: health, accident and sick leave  Allowance or compensation for the work; and the amount of the allowance or compensation  Information about recruitment policies, including the share of the trainees in recent years | Duration of the traineeship  Conditions for an extension or renewal of the traineeship | Range of responsibilities of the trainee  Working time limits  Minimum holiday entitlements  Assigned supervisor for the trainee  Learning and training objectives of the trainee  Traineeship provider will provide a certificate upon completion of the traineeship |

Source: Ecorys, 2022

The initial aim was to review up to 2700 traineeship vacancies. However, a number of obstacles eventually made the achievement of this impossible. The biggest challenge has been the collection of ALMP traineeship offers. The research team faced problems with the availability of data, and data sources in Bulgaria, Croatia, Cyprus, the Czech Republic, Denmark, Estonia, Hungary, Latvia, Lithuania, Malta, Romania and Spain. In response, during the data collection phase, we implemented three additional steps to collect the missing data, following consultation and support from DG EMPL. The research showed that a number of countries (BG, HR, CY, CZ, DE, EE, HU, LV, LT, MT, RO, ES) either do not have a dedicated portal with ALMP traineeship offers or such vacancies are not publicly available because traineeships are internally managed by regional and national PES. These circumstances translated into an overall lower number of analysed vacancies in several countries.

Despite these challenges, we collected and analysed 1,972 traineeship offers in 27 EU Member States: 1,272 open market traineeship vacancies and 700 ALMP traineeship vacancies. In the final phase of this sub-task, after triangulation by the core research team and final review by the national experts, a total of 1,696 vacancies offers were included in the final analysis: 996 open market and 700 ALMP traineeships.

Each principle was assessed separately with a percentage scoring indicating the share of vacancies that addressed the principle. An average across all scores was then calculated to obtain an overall assessment of the degree of implementation of the QFT principles in traineeship vacancies. Based on this, each country was scored as follows:

·High: If a country obtains more than 88% average score in review criteria

·Moderate: If a country obtains 65 – 87% average score in review criteria

·Low: If a country obtains 31 – 64% average score in review criteria

·Very low: If a country obtains 0 – 30% average score in review criteria

The number of analysed vacancies were deemed sufficient to investigate to what extent vacancies are aligned with the QFT.

(III)Analysis of QFT implementation and traineeship quality at the national level

In order to bring together the findings of the research on degree of implementation of the QFT in national legislation and degree of implementation of the QFT on the ground, Ecorys prepared an assessment template consisting of three sections. The template was presented, discussed and approved by the High-Level Advisors during several consultations and completed by national experts throughout the research process from February to May 2022. The Ecorys team provided guidance and support in solving methodological and other research problems.

The assessment template consisted of the following sections, each of which contributed to the overall assessment of the degree of implementation of the QFT in each Member State.

Section A: Overview of the country situation in terms of traineeships, including information on the overall regulatory framework of both ALMP and open market traineeships in the country, an assessment of the evolution of the prevalence of traineeships in the country since 2014.

Section B: Implementation of the QFT in national legislation: this was structured by principle and covered each of the 12 principles of the QFT that can be implemented in national legislation. A separate asessment grid was included for ALMP and open market traineeships. National experts had to assess degree of implementation in national legislation of each principle and assign a score: Fully/mostly implemened; Partially implemented; Not implemented; Not applicable.

On the basis of the scores for each principle, national experts assigned an overall score on degree of implementation of the QFT in national legislation as follows:

·Fully/mostly implemented: 10 or more principles out of 12 implemented in national legislation

·Partially implemented: 6 to 9 principles out of 12 fully or partally implemented in national legislation

·Modestly implemented: 3 to 5 principles out of 12 fully or partally implemented in national legislation

·Not implemented: 2 or fewer principles out of 12 fully or partally mplemented in national legislation

·Not applicable: if none of the QFT's principles are implemented because this type of traineeship does not exist in the country.

Section C: Implementation of the QFT on the ground. The third section of the assessment framework included three assessment areas as follows:

1.Results of the labour market review of traineeship vacancies: National experts were asked to and validate the findings of the labour market. They were able to revise the scoring for their country based on evidence found during other research tasks (national interviews, desk research) but had to justify this revision with the relevant evidence.

2.Obstacles to implementation of the QFT principles in traineeships on the ground: National experts were asked to identify any obstacles to implementing quality traineeships on the ground faced particularly by traineeship providers.

3.Degree of impact of the the QFT on the ground: National experts were asked to assess the degree of impact of the QFT on trainees, based on the interview findings, desk research and any relevant results from the trainee survey. Specifically they were asked to assess:

oWhether there are sectors or subgroups for which the QFT is less effective

oWhether there is evidence of the QFT/quality traineeships increasing stable labour market integration since 2014

On the basis of these three assessment areas, national experts made an overall assessment of the degree of implementation of the QFT on the ground:

·High: There is a high quality of vacancies, lack of obstacles to QFT implementation on the ground, and evidence of positive QFT/traineeship impact on trainees and society

·Moderate: There is a moderate or low quality of vacancies, some obstacles to QFT implementation on the ground, but evidence of positive QFT/traineeship impact on trainees and society

·Low There is a moderate or low quality of vacancies, some obstacles to QFT implementation on the ground, and weak evidence of positive QFT/traineeship impact on trainees and society

·Very low: There is a weak quality of vacancies, major obstacles to QFT implementation on the ground, and evidence of low or negative QFT impact on trainees and society

The scores from sections B and C of the assessment template therefore present a final assessment of the implementation of the QFT which was used to establish how the situation on traineeships has evolved since 2014.

(IV)Comparative analysis and clustering of 27 EU Member States

The contractor conducted a comparative analysis of the implementation of the QFT in national legislation and the implementation of the QFT on the ground in EU Member States. The contractor then clustered EU countries according to this comparison into five distinct groups with similar trends and approaches to QFT implementation.

(d)Case studies

Seven case studies were undertaken, examining in-depth the implementation of the QFT in the following countries: Austria, Bulgaria, Greece, Ireland, Italy, Lithuania, Spain. The methodological approach for developing the seven case studies was based on the triangulation of evidence using:

·Available quantitative data (including proxies) at national/regional level on traineeship uptake, including country-specific contexts and detailed 2014-21 trends. The analysis of quantitative data also considered the information collected from the targeted consultations (Task 1) and the mapping (Task 2);

·A complete analysis of available qualitative data through desk research, providing insight into ongoing debates on the quality framework for traineeships within Member States and further contextual data and information, building on from the research conducted during the mapping (Task 2).

·Qualitative data based on four to five individual in-depth interviews with relevant stakeholders depending on the country context (e.g. implementing organisations and social partners, such as trade unions, employer organisations, PES, etc.);

·Focus groups to explore the experience of trainees participating in specific programmes. These were undertaken in Bulgaria, Ireland and Spain.

The case study template was drafted by the core research team at inception phase and revised following the interim report meeting. Revisions to the case study template were focused on gathering evidence on aspects of the study where data gaps remained, in particular:

·Quantitative evidence on the costs and benefits of implementing the QFT/quality traineeships. For the purposes of this, our high level economist was consulted and additional guidance was provided to the national experts on how to gather quantitative information on costs and benefits.

·Evidence on cross-border traineeships and their prevalence in the Member State

·Evidence on the extent to which traineeships are/are considered to replace regular entry-level jobs for young people

The final case study reports were drafted on the basis of all evidence gathered on implementation of the QFT in each country throughout the whole study. The case studies provided key evidence to underpin and validate the study findings.

(e)Public consultation

The European Commission published a public consultation on the Council Recommendation on a Quality Framework for Traineeships (QFT) on 23 March 2022 on its website Have your say. The consultation remained open until 13 June 2022. The consultation targeted all citizens, and in particular young people who have already done a traineeship or who are interested in doing one, and traineeship providers, as well as organisations representing young people, social partners, PES, public administration, civil society, businesses, academia, and researchers, along with organisations offering traineeships.

The survey included closed and open-ended questions that were filtered and routed, where necessary, to tailor the relevance of the survey to each stakeholder group. The survey was available in the 24 official languages of the EU.

(I)Methodology for analysing the results 

The research team analysed the results of the public consultation. Respondents’ views were examined through a combination of closed questions and open-ended questions. Closed questions provided respondents with either different pre-set categorical answers from which to choose (e.g. the extent to which the principles of the QFT have been implemented in their country or at the EU level, the contribution of the QFT to specific benefits at the national and EU level), or asked them to rate the effectiveness, efficiency, coherence and relevance of the QFT and/or actions implemented in response to it.

The analysis of results was carried out using both quantitative (to analyse the frequencies of the closed answers) and qualitative methods (for the open questions, in order to analyse complex concepts as well as to substantiate and interpret the quantitative data with relevant insights). In addition, the research team categorised the responses through a range of relevant typologies (type of respondent, types of organisations).

(II)Quantitative analysis (closed questions)

The quantitative data analysis included an analysis of frequency distribution for each of the variables related to the closed-ended questions. Depending on the particular variable, all of the data values were represented. Frequency distributions were also depicted as graphs (histograms) as applicable. Cross-tabulations between specific variables and characteristics of respondents (e.g. type of stakeholder categories, type of respondent) were also conducted, where possible, though the low number of responses has limited the possibilities for this. The responses to the closed questions of the questionnaire were analysed using Excel and R. The statistical significance of the differences observed could not be further tested due to the low number of responses received.

(III)Qualitative analysis (open questions)

As a first step, the answers related to open-ended questions were translated by the research team. With respect to qualitative data analysis, information was classified by related variable (number of question) and analysed to identify additional relevant information and trends. The information was used to enrich the analysis carried out on quantitative data. Such information is reflected in the report through the inclusion of examples which help to illustrate key issues or trends at EU and national level identified in the quantitative analysis.

(IV)Interpretation of results

The questionnaire was structured in a way that all the respondents had to fill in the descriptive section covering background information related to themselves, while the questions in the main sections differed according to the stakeholder category to which the respondent belongs. Each question was analysed separately in order to ensure a consistent analysis of the responses. The factual summary of the results was published on Have Your Say
[168](#footnote168)
.

(V)Respondents to the public consultation

The public consultation received 259 contributions, with the majority coming from Poland (37%) and Spain (20%). There is a big discrepancy in national representation, with only one response registered for at least a third of Member States which could decrease the quality of findings.

3.Summary of limitations and mitigation measures

There were a number of specific limitations associated with the scope and coverage of the research, the quality of available data and the methodology that was developed, given the constraints of the available resources for the study. These limitations were taken into account in the design and implementation of the study as outlined in the Table below.

Table 7 - Key limitations of the research

|  |  |  |
| --- | --- | --- |
| Limitation/challenge | Explanation | Mitigation measures taken |
| Lack of existing solid secondary evidence on traineeship prevalence, quality and impact on young people’s transition to the labour market | Due in part to the diversity of definitions of traineeships, as well as the range – and sometimes absence – of regulatory approaches to traineeships in Member States, there is a lack of solid evidence on traineeships in Europe. Firstly, there is no comparable EU-wide statistical data on the prevalence of open market traineeships. Data is not collected at the EU level on participation in open market traineeships, meaning that it is hard to reliably quantify the prevalence of traineeships in Member States, and understand which target groups are undertaking them. Several EU studies have used proxy indicators, but these do not provide a thorough scientific understanding. Data on ALMP traineeships is more readily available through ALMP participation data from Public Employment Services, yet this is not necessarily specific to traineeships, as the types of action defined do not include a separate ‘traineeship’ category, but rather a ‘training’ one. | This challenge was addressed in several ways. Firstly, the preliminary literature review during the inception phase allowed to identify relevant cross-country research on which further research efforts were built with relevant documentation on traineeships at the national level. However, given that this data is not available for each Member State and is not comparable across Member States, the contractor undertook a quantitative analysis using a proxy variable to provide an estimate of traineeship prevalence across EU Member States and how this has evolved since 2014. |
| Diversity in regulatory approaches to traineeships across Member States | Linked to the challenge on data availability is the fact that regulatory approaches to traineeships in Member States hugely vary. This makes it challenging to compare the degree to which regulation in response to the QFT Recommendation has been implemented, as well as to assess compliance and enforcement. | To address this, the mapping included a specifc legal review and analysis on the degree to which the QFT principles have been integrated into national legislation/frameworks and examine the enforcement/monitoring measures in place. This was undertaken by national experts and reviewed by legal experts in each Member State, and was triangulated with findings from the consultation tasks undertaken on the national level to ensure a precise and reliable understanding of the regulatory frameworks and degree of integration of the QFT in these frameworks for analysis of the effectiveness, relevance and efficiency of the QFT. |
| Diversity of stakeholders involved in implementing traineeships and the QFT | Responsibility for implementation of the QFT - in particular, its implementation into national law, enforcement and monitoring - can lie with a range of national authorities depending on the existing regulatory framework, and the structure and roles of the labour market institutions in different Member States. This means that it is not easy to identify the main interlocutor at the national level for the implementation of the Recommendation. | The contractor addressed this challenge through a extensive consultation programme in which the research team interviewed a diversity of actors, as relevant to the specific institutional and legal setup in that country. The contractor also utilised any EU-level entry points into identifying relevant national stakeholders, with the support of DG EMPL, in particular reaching out to the PES Network and the Youth Guarantee Coordinators in each Member State. |
| Gathering the views of traineeship providers | Implementation of the QFT also relies on traineeship providers themselves who are ultimately responsible for offering quality traineeships that abide by the principles of any QFT-related legislation in place in each Member State. Traineeships are provided by a wide range of employers and organisations– from public and private to third sector organisations. | The contractor engaged with the PES on ALMP traineeships during our consultation activities. |
| Lack of quantifiable evidence on costs and benefits of QFT implementation | Analysis of the efficiency of the QFT is hampered by a lack of quantifiable data on benefits and costs, and an absence of monitoring of the effects of its implementation. Conceptually and practically, it is very difficult to reliably assess the potential benefits and costs associated with QFT implementation in Member States, for several reasons:  ·Benefits/costs that actors (employers, trainees, authorities) are typically aware of are those of introducing and implementing traineeships per se, rather than any additional or different benefits/costs due to adapting traineeships to QFT principles. Many stakeholders reported that costs and benefits associated with the QFT overlap with those of traineships in general and the QFT has not led to the emergence of new types of costs or benefits.  ·Awareness of QFT among relevant actors is low. Thus even where benefits/costs can be identified, actors are unlikely to attribute them appropriately to the QFT. This is particularly true in countries, where the traineeship concept is long-established and no concrete changes have been associated with the QFT, as in the Austrian case-study, for example: “Given that the QFT has not led to any directly attributable implementation measures, no data on benefits or costs is available.”  ·Member States had traineeships in place prior to the QFT, often closely resembling what is called for in the QFT. Even where specific traineeship developments are in line with the QFT Recommendation, most stakeholders noted that they have not monitored their effect. Hence it is not usually possible to identify which, if any, elements of these developments and their benefits/costs would occur anyway and which can be attributed to the QFT.  ·Moreover, even where the QFT has impacted on the trainee landscape, and specific developments can be accurately attributed to the QFT, their often qualitative nature (e.g. improved clarity of contractual terms, educational objectives, rights and obligations) makes it very difficult for actors to quantify associated benefits/costs. The best that can be achieved in most cases is that they can name the benefits/costs and perhaps give some qualitative assessment of their importance. | The contractor addressed this challenge throughout the study research tasks through the following actions:  ·By adapting the case study templates and the case study interview guides to add additional questions on quantifying costs and benefits following the feedback received at interim phase.  ·The contractor drafted and provided additional guidance to the national experts conducting the interviews and the case study research on how to gather costs and benefits data, in consultation with our labour economist.  ·The contractor consulted regularly with our labour economist to explore all possible options for quantifying costs and benefits and ensure that the research tools were asking the right questions to gather this data.  ·Targeted questions on costs and benefits were asked at the expert meeting and at the validation workshop.  ·The contractor followed up with EU employer organisations encouraging them to submit written input to the evaluation study, which they did and were used in the analysis of costs and benefits to the extent possible and useful.  Despite the challenges, analysed the evidence from the stakeholder consultations, trainee survey and case-studies which, taken together, provide a picture of benefits and costs, though largely qualitative in nature. |
| Low level of awarenss of the QFT amongst stakeholders | Throughout the consultation tasks, it became clear that there is a low level of awareness of the QFT amongst different relevant stakeholders, including PES, representatives from Ministries of Labour/Education, employer representatives on the national level, trade union representatives on the national level and individual traineeship providers. | The contractor developed an information sheet on the QFT, explaining the objectives of the QFT, outlining the principles and the main actors involved in its implementation. This was shared with every stakeholder consulted on the EU and the national level throughout all consultation tasks. |

  

Annex III. Evaluation matrix

|  |  |  |  |
| --- | --- | --- | --- |
|  | | | |
| Evaluation criteria | Main research questions (as specified in the tender specifications) | Sub-questions | Potential judgement criteria / indicators |
| Current situation and developments 2014 - 2021 | 1.What are the most common characteristics of trainees across the 27 Member States (e.g. sex, age, education background, socioeconomic background, migrant background, ethnic minority status)? How have these developed since the baseline of 2014? | How have the characteristics of trainees evolved across the 27 Member States? since 2014? How can this be explained? | Numbers of trainees across all EU27 Member States:  a)disaggregated by sex, age, educational background, socioeconomic status, migrant background and ethnic minority status, where possible  b)broken down by year (2014-2021)  Comparison of numbers of trainees with the characteristics of the overall population of young people (e.g. proportions of male/female trainees or those with migrant background compared to those in the overall population)  Qualitative evidence (stakeholder consultations, desk research) on the evolution of the profile of trainees over time and the reasons for the evolution  Comparison of trainee characteristics across Member States |
|  | 2.What is the current prevalence of traineeships across the 27 Member States? In which sectors are they most prevalent and what are the typical working conditions (e.g. ALMP versus open market, duration, remuneration, social protection coverage)? How has all of this developed since the baseline of 2014? | What is the state of play of the  legislative implementation of the QFT Recommendation overall and, as far as possible, by MS and sector? How has it evolved since 2014?  What is the state of play of the practical implementation of the QFT Recommendation overall and, as far as possible, by MS and sector? How has it evolved since 2014?  What is the prevalence of traineeships across the 27 EU Member States and how has this evolved since 2014?? What proportions of young people take up this opportunity?  In which sectors/occupations/types of employers (e.g. SMEs vs. large companies) are traineeships most and least prevalent and how has this evolved since 2014? Why?  What are the typical working and learning conditions of traineeships? To what extent is remuneration offered? How have the conditions evolved since 2014? | Numbers of trainees across the EU27, currently and over time (since 2014)  Shares of young people taking up traineeship opportunities, currently and over time (since 2014)  Estimates on the prevalence of traineeships in different sectors, currently and over time (since 2014),  Evidence from the mapping and case studies on types of employers offering traineeships.  Prevalence of types of working conditions of traineeships, currently and over time (since 2014)  Qualitative evidence (stakeholder consultations, desk research) on the underlying reasons for the evolutions in the numbers and types of traineeships over time (since 2014)  Comparative analysis across Member States |
|  | 3.What is the public debate across the 27 Member States, if any, about the role of traineeships and the importance of their quality for young people (political, civil society, representations of young people)? How has it developed since the baseline of 2014? | To what degree is there public debate on the quality of traineeships and how has this evolved since 2014??  Which stakeholders are expressing opinions on the quality of traineeships (civil society, politicians, policymakers, representatives of young people, employers/business, training providers or their representatives, social partners)? To what degree do views differ according to different types of stakeholders? | Existence/non-existence of public debate on the role and quality of traineeships and nature of differences across Member States  Analysis of the sentiment expressed (positive/negative views) in the public debate on the role and quality of traineeships  Overview of the key stakeholders engaged in public debate (e.g. civil society, politicians, policymakers, representatives of young people, employers/business, training providers or their representatives, social partners) and any marked differences of opinion  Overview of the key topics, views and focus of public debate in the 27 Member States  Nature of the evolution in public debate over time (overall increase/decrease, increase in positive/negative views, etc.)  Qualitative evidence (stakeholder consultations, desk research) on the reasons for the evolution of over time |
| Relevance | 4.To what extent are the principles appropriate for fostering stable labour market integration? Which principles are likely to be the most and the least important? | To what degree are the QFT principles appropriate for fostering stable labour market integration? Which principles make the most/least important contribution to fostering sustainable labour market integration? Why? | Positive/negative views from stakeholders and literature on the appropriateness of the principles for fostering stable labour market integration  Positive/negative views from stakeholders and literature on the most/least important principles for fostering stable labour market integration  Analysis of concrete evidence and examples (where available) from the case studies (and/or existing evaluations) of the implementation of specific principles having a particularly important role in stable labour market integration of trainees  Evaluative judgement based on evidence collated on the alignment of the principles, and the most/least important principles, for fostering stable labour market integration |
|  | 5.Has the understanding of traineeship quality evolved over time? How well do the principles of the QFT still correspond to the needs within the EU? How well adapted is the QFT to a post-COVID-19 world, a changing labour market and the latest perspectives of stakeholders and citizens? | Has the understanding of the importance and nature of quality in traineeships evolved over time? If so, how and why? What differences exist between Member States or groups of Member States? Why?  To what degree do the QFT principles still correspond to needs within the EU? Why?  To what extent do the QFT principles respond to the concern that traineeships may be used to replace regular employment for young people?  To what extent and how is the QFT adapting to developments including the impact of the Covid-19 pandemic and the changing labour market?  To what degree is the QFT aligned with the current perspectives of stakeholders and individual citizens? | Existence/non-existence of changes in the understanding in traineeship quality over time  Existence/non-existence and types of differences between Member States  Qualitative evidence (from stakeholders and literature) on the degree to which the QFT principles correspond to existing EU needs and concerns of stakeholders, and are adapted to recent developments  Evaluative judgement based on evidence collated and expert opinion on the evolution of the understanding of traineeships, the degree to which the principles have corresponded to needs and continue to evolve to meet new needs in the light of key contextual changes (Covid-19, evolving labour market, new perspectives of stakeholders). |
|  | 6.Are there any additional dimensions that should have been added to the principles from the start or have in the interim proven to be lacking? How – and how likely – would such additions add to quality traineeships, and, in turn, their contribution to a stable labour market integration? | Are there any dimensions or elements that should have been added to the QFT principles from the start or have since been shown to be lacking? If so which and why?  To what extent would a principle related to remuneration of trainees improve the quality of traineeships? Would there be any adverse effects of such a principle?  What contributions would such new elements be likely to make to improving the quality of traineeships and stable labour market integration? | Qualitative evidence (from stakeholders and literature) on elements missing from the principles and views on potential new additions  Qualitative evidence (from stakeholders and literature) on contributions of new elements of the principles to improving the quality of traineeships and subsequent stable labour market integration  Evaluative judgement based on evidence collated and expert opinion on dimensions missing from the current principles, potential new additions, and the potential impact of any changes to the principles |
|  | 7.With due respect to the principle of subsidiarity, how well does the nature of the QFT as a (non-binding) Council Recommendation correspond to the needs and the socio-economic problems to be solved? Has this changed over time, and are their significant differences between Member States? | To what extent does the QFT’s status as a non-binding framework correspond to the needs and socio-economic problems to be tackled?  Has this changed over time and in light of any evolutions in needs or changes in the socio-economic context?  What differences exist between Member States and why? | Qualitative evidence (from stakeholders and literature) on the appropriateness of the non-binding status of the QFT for meeting needs and resolving problems  Analysis of the nature of the evolution of perspectives over time  Analytical overview of differences between Member States  Evaluative judgement based on evidence collated and expert opinion (including legal expertise) on the suitability of the non-binding nature of the QFT to meet needs |
| Effectiveness | 8.To what extent have the principles of the QFT been enshrined in national legislation and/or national quality frameworks since 2014? To what extent did they already exist? What is the scope of current national legislative frameworks and national quality frameworks? | To what extent have the principles of the QFT been enshrined in and/or influenced national legislation/quality frameworks since 2014? Which principles in particular and why?  What types of  legislative implementation tools/regulatory approaches have been used by Member States to enshrine the principles of the QFT?  To what extent and which principles were already enshrined in national legislation/frameworks before 2014?  To what extent are traineeships defined in current national legislation/frameworks?  What is the scope and content of current national legislative frameworks and national quality frameworks in relation to traineeships? To what degree to they cover all the principles? Which principles are most covered and which least? Why?  What have been the obstacles to the full transposition and implementation of the QFT in Member States? | Legal analysis of the transposition (or non-transposition) of the QFT principles into national legislation since 2014  Legal analysis of the transposition (or non-transposition) of the QFT principles into national quality frameworks since 2014  Evidence of the existence of quality frameworks corresponding to the QFT before 2014  Analytical overview of the scope of current national legislative frameworks and quality frameworks on both ALMP and open market traineeships  Evaluative judgement based on evidence collated and expert opinion (including legal expertise) on the degree to which the QFT has been enshrined in national legislation and frameworks  Evidence from interviews and meetings of the types of obstacles faced by Member States in transposing QFT principles. |
|  | 9.To what extent do enforcement and/or regular follow-up monitoring exist and to what extent do they confirm an adherence to national legislation and/or national quality frameworks and/or the overarching QFT? Is there evidence of adverse effects too? | Have enforcement and/or regular monitoring mechanisms been put in place to facilitate compliance with national legislation/quality frameworks and/or the QFT?  What types of mechanisms can be found, and how do they differ across the Member States?  Where they exist, how effective have they been in ensuring compliance? Which mechanisms have been most/least effective, and why?  Is there any evidence of any adverse effects of enforcement/monitoring? If so which and why? | Legal analysis of the existence/non-existence and nature of enforcement measures to ensure compliance to national legislation/quality frameworks and/or the overall QFT  Legal analysis of the existence/non-existence and nature of regular follow-up/monitoring mechanisms to ensure compliance to national legislation/quality frameworks and/or the overall QFT  Qualitative evidence (from stakeholders and desk research) on the effectiveness of such mechanisms in ensuring compliance  Qualitative evidence (from stakeholders and desk research) on any adverse effects  Evaluative judgement based on evidence collated and expert opinion (including legal expertise) on the effectiveness of enforcement, follow-up and monitoring arrangements |
|  | 10.What is the impact on trainees? How effective do (ex-)trainees perceive traineeships (and/or specific legislative frameworks and/or specific national quality frameworks) to be in terms of facilitating a stable labour market integration for young people, and contributing to youth employment? Is there evidence of adverse effects too? | What has been the impact of the implementation of the QFT on trainees?  Which specific elements of QFT implementation (specific principles, enforcement/follow-up measures, etc.) have had the most positive impacts on trainees and their stable labour market integration? Why?  To what degree do trainees (or ex-trainees) consider that the frameworks and principles governing traineeships (e.g. specific national legislation or quality frameworks and/or specific aspects of them) can be in facilitating stable labour market integration and fostering youth employment? Which elements/principles are of particular importance from their perspective? Why?  Are there additional principles that would enhance the quality of traineeships?  Has the way in which the QFT been implemented (e.g. specific principles or measures) had any adverse/negative effects on trainees? If so which and why? | Qualitative evidence (from stakeholders, case studies, surveys and literature) on the degree and nature of impacts (e.g. employability, skills development, development of transversal competences, development of social networks, etc.) of QFT implementation on trainees  Qualitative evidence (from stakeholders, case studies, surveys and literature) on the degree and nature of impacts on trainees of specific aspects of QFT implementation (e.g. specific principles, enforcement/follow-up measures, etc.)  Any quantitative evidence (e.g. from case studies) on the impacts of QFT implementation on young people e.g. access to stable employment, further education and training, etc.  Qualitative evidence (from stakeholders, surveys and literature) on any adverse effects of the implementation of the QFT principles  Views of trainees (and ex-trainees) (and/or their representatives) on the effectiveness of QFT implementation in facilitating stable labour market integration and fostering youth employment (particularly via the survey of trainees, but also the public consultation)  Views of trainees (and ex-trainees) on any adverse effects of QFT implementation (particularly via the survey of trainees, but also the public consultation)  Evaluative judgement based on evidence collated and expert opinion regarding the impact of the different aspects of QFT implementation (e.g. specific principles, enforcement/follow-up measures, etc.) on trainees |
|  | 11.How equally are the effects of the QFT, by way of national legislative frameworks and/or national quality frameworks, distributed within Member States? Are there sectors or (social) sub-groups for which the QFT has proven to be particularly successful or unsuccessful? | To what degree are the effects of the implementation of the QFT principles (by way of national legislative frameworks and/or national quality frameworks) distributed equally within Member States?  Is the QFT particularly successful or unsuccessful for specific economic sectors or target groups? If so, which and why?  What has been the impact of QFT implementation on traineeship providers/employers? Have specific elements of QFT implementation had particularly positive or negative impacts and why? | Qualitative evidence (from stakeholders and literature) on the equal distribution of effects of the QFT within Member States  Quantitative evidence (where available) on the existence of traineeships in different sectors and for different target groups  Qualitative evidence (from stakeholders, literature and surveys) on sectors and sub-groups in which the QFT has been particularly successful/unsuccessful  Views of employer representatives on the impact of QFT implementation on employers/traineeship providers (from interviews and case studies)  Evaluative judgement based on evidence collated and expert opinion on the impact and effectiveness of QFT implementation across sectors and sub-groups |
|  | 12.How do observed developments since the baseline of 2014 compare against the developments that were expected to be achieved when the QFT was developed/proposed? | To what degree do the developments since 2014 correspond to the expectations for the QFT when it was developed? How and in what ways?  Which expectations of the QFT have not been met and why? | Desk-based analysis on the degree to which the identified developments regarding the implementation and impacts of the QFT correspond to original expectations (based on key documents including the Council Recommendation and the underpinning impact assessment [169](#footnote169) )  Qualitative evidence (from stakeholders and literature) on the degree to which the identified developments correspond to original expectations, as well as the expectations that have not been met and why  Evaluative judgement based on evidence collated and expert opinion on the degree to which QFT implementation has met the original expectations |
|  | 13.While taking into account the differences in objectives and target groups, has one of the two quality frameworks, the QFT or the EFQEA, been more effective in improving the quality of traineeships and apprenticeships respectively, as well as the employability of trainees and apprentices, and why? | Accounting for the differences in target groups and objectives, has the QFT been more or less effective than the EFQEA in terms of improving the quality of traineeships/apprenticeships respectively? In what ways and why?  Has the QFT been more or less effective than the EFQEA in terms of improving the employability of trainees/apprentices respectively? In what ways and why? | Qualitative evidence (from stakeholders and literature) on the effectiveness of the QFT and the EFQEA in improving the quality of traineeships/apprenticeships (respectively)  Qualitative evidence (from stakeholders and literature) on the effectiveness of the QFT and the EFQEA in improving the employability of trainees/apprentices (respectively)  Review of any comparative quantitative evidence on the employability of trainees/apprentices (respectively)  Evaluative judgement based on the qualitative and quantitative evidence collated and expert opinion (including our experts working on the Apprenticeship Support Services), taking into account the differences in objectives and target groups, on the relative effectiveness of the QFT and EFQEA |
| Efficiency | 14.What was the quantifiable administrative burden associated with the adequate implementation of the QFT for the different stakeholders at various levels (national, regional, local), in terms of e.g. transposition of QFT principles, enforcement/follow-up monitoring and other compliance checks? | What was the nature of the administrative burden associated with the adequate implementation of the QFT (e.g. transposition of QFT principles, enforcement/follow-up monitoring and other compliance checks)?  What was the quantifiable administrative burden (i.e. administrative costs generated by the EU intervention, in addition to the already existing administrative costs) for different stakeholders at various levels (national, regional, local) in implementing the QFT?  What types of adjustment costs (e.g. staff costs for supporting trainees, setting learning objectives ,etc.) were there in implementing the QFT? Can any of these adjustment costs be quantified? Which stakeholders incurred these costs? | Quantitative (or quantifiable) evidence from stakeholders at national, regional and local stakeholders on the administrative burden associated with QFT implementation (e.g. additional staff time and training for implementation, enforcement and monitoring, etc.)  Administration expenditure data on ALMPs that offer traineeships within the scope of the QFT from public employment services, as available  Qualitative evidence from stakeholders at national, regional and local stakeholders on the nature of the administrative burden associated with QFT implementation  Evidence from stakeholders on the relative size and nature of the adjustment costs incurred to implement the QFT.  Economic analysis and evaluative judgement based on the data collected to ascertain the quantifiable administrative burden of the implementation and enforcement of the QFT |
|  | 15.To what extent have Member States encouraged or enabled traineeship prevalence and improved quality through financial incentives? Have any EU programmes (such as the YEI and ESF) contributed directly or indirectly to financial incentives? | What type of financial incentives have been put in place by Member States to increase the prevalence of traineeships and improve their quality?  To what extent are financial incentives in place across the EU27?  Have any EU programmes (such as the YEI or ESF) contributed to these financial incentives, either directly or indirectly? | Evidence (from the desk research/stakeholder consultations/case studies) on the existence, nature, and monetary value of financial incentives in different Member States (e.g for case study countries, the estimated total amount spent on financial incentives that aimed to increase prevalence or quality.)  Evidence (from the desk research/stakeholder consultations/case studies) on the contribution of EU programmes to financial incentives  Evaluative judgement based on the evidence collated and expert opinion on the existence and nature of financial incentives in different Member States, and the contributions of EU programmes to such incentives |
|  | 16.What can be approximated in terms of quantifiable benefits for young people? Are there other quantifiable benefits for society at large that could be linked back to the QFT? | What approximation can be made of the quantifiable benefits that can be linked to QFT implementation for young people?  What approximation can be made of the quantifiable benefits for employers?  Are there other quantifiable benefits which can be approximated linked to the QFT for wider society? If so, which? | Evidence (from the desk research/stakeholder consultations (in particular trainee survey/case studies) on the quantifiable benefits of QFT implementation for young people e.g. the increase in the proportion of traineeships/traineeship providers offering traineeships that give clear and improved educational objectives and working conditions, such as meeting minimum holiday entitlements and certification of skills  Evidence (from the desk research/stakeholder consultations/case studies) on the quantifiable benefits of QFT implementation for employers (e.g. by improving their reputation as quality traineeship providers/providing them with a more diverse, young workforce/resulting in tax exemptions/reductions and/or access to subsidies?  Evidence (from the desk research/stakeholder consultations/case studies) on the quantifiable benefits of QFT implementation for society at large e.g., a fall in youth unemployment/inactivity and associated reductions in welfare payments and increases in tax revenue).  Economic estimation (where possible) of the value of quantifiable benefits of QFT implementation |
|  | 17.What are the main obstacles preventing employers from offering traineeships? What kind of support would be necessary to overcome these obstacles? | What are the main obstacles preventing employers from offering quality traineeships?  What are the main reasons that employers offer quality traineeships?  To what degree are the obstacles to offering quality traineeships addressed by the QFT? Are there any additional principles that if added to the QFT would increase or reduce these obstacles?  What kind of further support would help overcome these obstacles? | Qualitative evidence (from desk research/stakeholder consultations/case studies) on the main obstacles for employers and the degree to which they are addressed by the QFT  Qualitative evidence (from desk research/stakeholder consultations/case studies) on the support needed to overcome the obstacles for employers  Evaluative judgement based on the evidence collected and expert opinion on the main obstacles stopping employers from offering traineeships, and the support need to overcome these obstacles |
|  | 18.To what extent are the administrative costs proportionate to the assessed benefits of QFT implementation? What has been the cost-effectiveness and how/why does it vary across the 27 Member States? What factors influenced the efficiency and how did they do so? | To what extent are the administrative costs proportionate to the benefits of QFT implementation?  What has been the cost-effectiveness of QFT implementation?  To what degree does cost-effectiveness vary across the 27 Member States? In what ways and why?  What factors influenced the efficiency of QFT implementation? In what ways and why? | Assessment of the proportionality of costs to benefits, based on the evidence collated on administrative burden (costs) and benefits  Assessment of the cost-effectiveness of the implementation of the QFT  Identification and analysis of any differences in cost-effectiveness across the 27 Member States  Qualitative evidence (from desk research/stakeholder consultations/case studies) on the factors which influenced the efficiency of QFT implementation |
|  | 19.Is there scope for reducing administrative burden without undermining the assessed benefits of QFT implementation? In other words, could benefits have been achieved at lower cost? | Could the observed benefits have been achieved at a lower cost? In what ways?  Is there scope to reduce the administrative burden without reducing the benefits of QFT implementation? | Qualitative evidence (from desk research/stakeholder consultations/case studies) on the scope to reduce administrative burden without diminishing the returns (benefits)  Evaluative and economic judgement based on the evidence collated and expert opinion on the scope for reducing costs while maintaining the benefits |
| Coherence | 20.To what extent have the objectives, target groups and measures to implement the QFT within the context of ALMPs been coherent with education and training, employment and social policies at national and regional level? How about open market traineeships? | To what degree has implementation of the QFT within ALMPs been coherent with national and regional education, training, employment and social policies in terms of (a) objectives, (b) target groups and (c) measures?  To what degree has implementation of the QFT within open market traineeships been coherent with national and regional education, training, employment and social policies in terms of (a) objectives, (b) target groups and (c) measures?  Are there differences across Member States? If so, why? | Text-based analysis on the extent of external coherence between QFT implementation (objectives, target groups, measures) and relevant national and regional education, training, employment and social policies, targeting young people (e.g. Youth Guarantee, activation measures for young people, etc.)  Qualitative evidence (from national/regional stakeholder consultations/case studies/public consultation) on the extent of external coherence between QFT implementation (objectives, target groups, measures) and relevant national and regional education, training, employment and social policies  Qualitative evidence (from desk-research/stakeholder consultations/case studies) on the extent of coherence with open market traineeships  Evaluative judgement on the basis of evidence collated, including assessment of (level of) complementarities, synergies, overlaps and contradictions |
|  | 21.Linked to the former, to what extent do the QFT support and usefully complement other policies (in particular those pursued at national level)? What is the level of complementarity or duplication? | To what degree does the QFT support and complement other policies, particularly at national level? In what ways? Are there any conflicts?  What is the level of complementarity or duplication?  Are there differences across Member States? If so, why? | Text-based analysis on the extent of external coherence between QFT implementation (objectives, target groups, measures) and relevant national and regional education, training, employment and social policies  Qualitative evidence (from national/regional stakeholder consultations/case studies) on the extent of external coherence between QFT implementation (objectives, target groups, measures) and relevant national and regional education, training, employment and social policies  Evaluative judgement on the basis of evidence collated, including assessment of (level of) complementarities, synergies, overlaps/duplication and contradictions |
|  | 22.To what extent have the objectives, target groups and measures to implement the QFT within the context of ALMPs been coherent with the relevant EU initiatives listed in section 2.3? How about open market traineeships? | To what degree has the implementation of the QFT (objectives, target groups and measures) been coherent with relevant EU initiatives including relevant Commission Communications (e.g. Youth Employment Support: a Bridge to Jobs for the Next Generation [170](#footnote170) ) or Recommendations (e.g. Effective Active Support to Employment (EASE) [171](#footnote171)  or strategies (e.g. European Youth Strategy [172](#footnote172) ), funding instruments (e.g. ESF, YEI, Erasmus+), other programmes/measures (e.g. EURES, European Solidarity Corps, the reinforced Youth Guarantee), etc.?  Are there differences across Member States in the degree of complementarity with EU initiatives? If so, why? | Text-based analysis (based on legal texts) on the extent of external coherence between QFT implementation (objectives, target groups, measures) and relevant EU initiatives  Qualitative evidence (from stakeholder and public consultations) on the extent of external coherence between QFT implementation (objectives, target groups, measures) and relevant EU initiatives  Evaluative judgement on the basis of evidence collated, including assessment of (level of) complementarities, synergies, overlaps/duplication and contradictions |
| EU added value | 23.What is the additional value resulting from the QFT compared to what could reasonably have been expected from Member States acting at national and/or regional levels? | What additional value has been generated from the QFT at the European level compared to what could reasonably be expected by Member States acting alone at national/regional levels?  To what degree is/was work already being done at national level to improve the quality and prevalence of traineeships (outside the scope/influence of the QFT)?  What type of added value has been generated? For example:  ·To what degree has the QFT led to more quality traineeships being implemented (volume effects)?  ·To what degree has the QFT broadened the focus of existing national/regional measures to include new areas or new target groups (scope effects)?  ·To what degree has the QFT led to structural changes in employment/education/training policy or frameworks at national level (role effects)?  ·Is there evidence of benefits to organisations implementing frameworks or delivering traineeships (training providers, employers, PES, monitoring bodies) from being involved in QFT implementation (process effects)?  To what degree has the Recommendation contributed to creating a ‘level playing field’ amongst Member States (upward convergence)? | Qualitative evidence (from desk research/stakeholder and public consultations/case studies) on added value  Quantitative evidence (where available) on added value  Evaluative judgement and expert opinion on the basis of evidence collated  Expert opinion based on evidence collected on degree of convergence of Member States’ regulatory approaches to traineeships since 2014. |
|  | 24.What would be the most likely consequences of discontinuing the QFT at EU level, and what would be the most likely prognosis for a no-policy-change scenario of continuing the QFT as it stands? | What would be the most likely consequences of:  (a) Discontinuing the QFT at EU level?  (b) Continuing QFT implementation as under current policy  Which stakeholders would be likely to be affected, in what ways and why?  What would be the probable impact on the quality and offer of traineeships? Why?  What would be the likely negative and positive effects? | Qualitative evidence (from desk research/stakeholder and public consultations/case studies) on the consequences of the two policy choices in terms of stakeholders affected and impact on traineeships and their quality  Evaluative judgement and expert opinion on the basis of evidence collated |

  
  

Annex IV. Overview of benefits and costs

|  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Table 1. Overview of costs and benefits identified in the evaluation [173](#footnote173) | | | | | | | |
|  | | Citizens/Consumers | | Businesses | | Administrations | |
|  | | Quantitative | Comment | Quantitative | Comment | Quantitative | Comment |
| Implementation costs associated with implementing new legislation in line with the QFT | | | | | | | |
| Costs:  Direct compliance costs     Adjustment costs | one-off | Not available | Not applicable to this stakeholder. | Not available | Implementation costs incurred by businesses likely to be low one-off costs of familiarisation with any new legislation in place. | Not available | For policy makers, these costs are estimated to be low (no monetary estimates provided) in part because some regulatory framework in line with the QFT was already in place. |
| Direct labour costs associated with designing quality traineeship schemes, drafting learning objectives, and supervising trainees | | | | | | | |
| Costs:  Direct compliance costs     Adjustment costs | one-off | Not available | Not applicable to this stakeholder. | Not available | These costs are generally higher for small and medium sized enterprises than for large companies. | Not available | Costs mainly associated to ALMP traineeships, such as designing the relevant schemes/programmes. |
| Costs of external services for advertising traineeship opportunities | | | | | | | |
| Costs:  Direct compliance costs     Adjustment costs | recurrent | Not available | Not applicable to this stakeholder. | Not available | Advertising/marketing costs to raise awareness of open market traineeships. No monetary estimates provided. | Not available | Advertising/marketing costs to raise awareness of ALMP traineeships. No monetary estimates provided. |
| Direct labour costs of investing in public services and labour inspectorates to monitor compliance. | | | | | | | |
| Costs:  Enforcement costs | recurrent | Not available | Not applicable to this stakeholder. | Not available | Not applicable to this stakeholder. | Not available | No monetary estimates provided. Enforcement of traineeship regulations is normally undertaken by the national labour inspectorate or equivalent, and any costs are incurred by the relevant national authorities. Such inspection, however, normally focuses on the enforcement of the relevant national legislation, and even where that legislation has been influenced to some extent by the QFT it is not possible to separately identify enforcement costs due to the QFT |
| Administrative costs the costs incurred for providing a written agreement (collecting the information that should be included in the written agreement and putting it down in the written agreement), costs of managing cooperation with PES, certifying trainees’ skills | | | | | | | |
| Costs:  Direct compliance costs     Administrative | recurrent | Not available | Not applicable to this stakeholder. | Not available | Evidence shows that these costs are higher for businessess offering ALMP traineeships than those offering open market traineeships in part due to the need to manage cooperation arrangements with PES which may include reporting obligations | Not available | Administrative costs for PES to offer ALMP traineeships in line with QFT are likely to be business as usual costs which are incurred for all ALMP offers. |
| Costs of financial incentives/subsidies to encourage uptake of traineeships aimed at both trainees and employers, including traineeship grants and reimbursing travel expenses for unemployed individuals | | | | | | | |
| Costs:  Direct compliance costs     Costs of financial incentives/subsidies | recurrent | Not available | Not applicable to this stakeholder. | Not available | Administrative costs of applying for and managing these subsidies were also reported by businesses receiving them. | Not available |  |
| Growth in number of quality traineeships | | | | | | | |
| Direct benefits    Improved market efficiency | recurrent | Not available | More opportunities for young people to undertake quality traineeships; No monetary estimates provided. | Not available | Improvement in skills and knowledge of employees and job applicants. No monetary estimates provided. | Not available | Increase in the share of the population with the skills needed for the labour market. |
| Traineeships more aligned with labour market needs | | | | | | | |
| Direct benefits    Improved market efficiency | recurrent | Not available | No monetary estimates provided. | Not available | No monetary estimates provided. | Not available | No monetary estimates provided. |
| Enhanced skills and certification of skills | | | | | | | |
| Direct benefits    Improved welfare | recurrent | Not available | More qualified future workforce. No monetary estimates provided. | Not available | More qualified future workforce. No monetary estimates provided. | Not available | Increase in the share of the population with the skills needed for the labour market. No monetary estimates provided. |
| Improved access and transition to labour market | | | | | | | |
| Direct benefits    Improved welfare | recurrent | Not available | "62% of respondents to the trainee survey agreed that traineeships made transitioning from school to work easier.  In Ireland, 98% of trainees moved into employment following the Career Traineeship programme of which 72% with employers that had originally hosted the trainees. No monetary estimates provided. " | Not available | More qualified future workforce, fewer skills mismatches. No monetary estimates provided. | Not available | Increase in the share of the population with the skills needed for the labour market. No monetary estimates provided. |
| Reduced exploitation and greater awareness of rights | | | | | | | |
| Direct benefits    Improved welfare | recurrent | Not available | Trainees in particular cited this as a benefit. No monetary estimates provided. | Not available | Increased in employee productivity. No monetary estimates provided. | Not available | No monetary estimates provided. |
| Reduced welfare payments and increased tax revenue | | | | | | | |
| Indirect benefits    Wider macroeconomic benefits | recurrent | Not available | Reduced welfare payments and increased tax revenue following on from supporting young people into employment. No monetary estimates provided. | Not available | Reduced welfare payments and increased tax revenue following on from supporting young people into employment. No monetary estimates provided. | Not available | Reduced welfare payments and increased tax revenue following on from supporting young people into employment. No monetary estimates provided. |
| Improved individuals’ participation in society | | | | | | | |
| Indirect benefits  Wider welfare benefits | recurrent | Not available | Individuals more integrated into society (through work). Increased social cohesion. No monetary estimates provided. | Not available | No monetary estimates provided. | Not available | Individuals more integrated into society (through work). Increased social cohesion. No monetary estimates provided. |

   

   

Annex V. Stakeholder consultations - Synopsis report

Introduction

This document provides a synopsis of the stakeholder consultations conducted for the evaluation of the Quality Framework for Traineeships (QFT). The consultations followed the consultation strategy and the Evaluation Roadmap
[174](#footnote174)
, published on the Commission’s Better Regulation website “Have Your Say”.

The aim of the consultation activities was to “gather information and views on the current state of play of both the quality of traineeships and the implementation of the QFT, as well as to assess whether the QFT is fit for purpose, in particular considering the evolving labour markets and stakeholders’ needs as well as the COVID-19 pandemic.” In addition, the consultation activities were “to contribute to a better understanding on whether traineeships and the QFT implementation have had an impact on youth employment, as well as on young people and the society at large”.

Overview of consultation activities

The Commission consulted a wide range of stakeholders, in order to ensure that the views of the relevant stakeholder groups are taken into account. The evaluation of the QFT included targeted consultations of national and regional authorities responsible for education, training and employment policies, social and economic partners, education and training providers, academic experts working on labour market issues, organisations representing young people, young (former, current and potential future) trainees, as well as other relevant stakeholders at European, national and regional level. A public consultation was open to all stakeholders as well as a wider public.

To adequately reach these stakeholders, different consultation activities and methods were used, such as interviews, targeted consultation meetings and case studies. A more detailed description of these activities can be found below in this chapter. Table 1 below presents the type of consultation tools used for each stakeholder category.

Table 1 Overview of stakeholders consulted through various consultation activities

|  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Public consultation | Evaluation roadmap | Targeted interviews | Trainee survey | Case studies | Expert meeting | Validation workshop | Consultation meeting |
| Ministries and institutions responsible for education and training policy | X | X | X |  | X | X | X |  |
| Ministries and institutions responsible for employment policies | X | X | X |  | X | X | X |  |
| Public Employment Services (PES) | X | X | X |  | X | X | X |  |
| Social and economic partners at national and EU level | X | X | X |  | X | X | X | X |
| Education and training providers | X | X | X |  | X |  |  |  |
| EU level institutions/ policymakers | X | X | X |  |  | X | X |  |
| Organisations representing young people and/or young trainees | X | X | X |  |  | X | X |  |
| Current, former, and potential trainees | X | X |  | X | X |  |  |  |
| Research /academia and other international organisations | X | X | X |  | X | X | X |  |
| EU citizen | X | X |  |  |  |  |  |  |

2.1 Evaluation Roadmap

The evaluation roadmap was open for consultation for four weeks between 28 July and 25 August 2021. It aimed at gathering the views of relevant stakeholders on the Commission's understanding of the problem and possible solutions and to share any relevant information that they may have on the implementation of the QFT and on traineeships.

Two contributions were received, representing academic and research institutions.
[175](#footnote175)
 In the course of the consultation activities, ad-hoc position papers were also received from one civil society organisation and one Member State.

2.2 Consultation of European social partner organisations

A dedicated consultation meeting was held with the European social partner organisations on 20 June 2022. Employers were represented by BusinessEurope, CEEMET and national employer representatives from HU, FR, and DK. Trade Unions were represented by ETUC and national trade union representatives from IT, LU, ES, SI, NO, TR, ES, LT, BE, HU and FI.

On the basis of a background document, the social partners presented their views on the current QFT principles as well as how the QFT could be improved in the future.

2.3 Public consultation

The aim of the public consultation
[176](#footnote176)
 was to provide a channel for all interested stakeholders to express their views on the QFT and traineeships in general. The consultation took the form of an e-survey (via the EUSurvey tool) with closed and open questions, addressing the key evaluation criteria (effectiveness, efficiency, relevance, coherence, and EU added value) and tailored to the different stakeholder groups. The response period was 12 weeks from 21 March to 13 June 2022.

The public consultation had 259 respondents. This included 169 organisations belonging to the categories indicated in the survey (65%), 70 individuals (27%), and 20 respondents representing other groups. Respondents were from 24 EU Member States and 4 non-EU countries. 89% (230 out of 259) of responses were received from seven of the 28 countries represented, showing a somewhat unbalanced geographic distribution of responses. Figure 1 shows the number of responses from each country represented in the public consultation.  

Figure 1 Breakdown of respondents by Member State

![](./../../../resource.html?uri=IMMC:SWD%282023%299.ENG.xhtml.SWD_282023_299_ENG_xhtml_04005.jpg)

 Source : QFT Public consultation 2022, N=259 

The largest group of respondents was public authorities (85 out of 259, 33%), with the majority of the authorities representing employment services (64 out of 85, 75%), and the remainder representing national, regional and local public authorities (12 out of 85, 14%), government bodies or ministries (7 out of 85, 8%), and training or education institutions (2 out of 85, 2%). The second largest group of respondents was EU citizens (69 out of 259, 27%), followed by academic/research institutions (56 out of 259, 22%) and those who indicated they belonged to a group not listed in the questionnaire (20 out of 259, 8%). The remaining respondents represented NGOs (11 out of 259, 4%), companies/business organisations (8 out of 259, 3%), trade unions (6 out of 259, 2%), business associations (3 out of 259, 1%) and a non-EU citizen (1 out of 259, 0.4%). Figure 2 provides a breakdown of respondents by main stakeholder groups.  

Figure 2 Breakdown of respondents by main stakeholder groups

![](./../../../resource.html?uri=IMMC:SWD%282023%299.ENG.xhtml.SWD_282023_299_ENG_xhtml_04006.jpg)

Source: QFT Public consultation 2022, N=259 

2.4 Targeted interviews with EU and national stakeholders

Targeted interviews with EU and national level stakeholders focused on two key stakeholder groups: national level stakeholders responsible for education and training, skills, and employment policies
[177](#footnote177)
 and EU level stakeholders directly and indirectly involved in education, training, and employment policies. The targeted semi-structured interviews were conducted from February to June 2022.

The targeted interviews included questions covering the evaluation criteria, as well as overarching questions to investigate the current prevalence of traineeships, main characteristics of trainees, the public debate around the topic, as well as any relevant change since the adoption of the QFT.

A total of 124 interviews of relevant stakeholders were undertaken as part of the evaluation. Of these, 17 were interviews with EU level stakeholders and 107 with national level stakeholders in all Member States.

Table 2 Targeted interviews completed

|  |  |  |
| --- | --- | --- |
| EU level | | |
| Stakeholder | Number of interviews conducted | |
| DG EMPL | 3 | |
| DG EAC | 1 | |
| Eurofound | 2 | |
| Cedefop | 1 | |
| Association of European Chambers and Industry (EUROCHAMBERS) | 1 | |
| Business Europe | 1 | |
| European Trade Union Confederation (ETUC) | 1 | |
| European Confederation of Independent Trade Unions (CESI) | 1 | |
| IndustrialAll Europe | 1 | |
| European Youth Forum | 1 | |
| Fair Internship Initiative | 1 | |
| Interns Go Pro | 1 | |
| SPRINT project | 1 | |
| International Labour Organization (ILO) | 1 | |
| Total EU level interviews | 17 | |
| National level | | |
| Type of stakeholder | Countries covered | Number of interviews conducted |
| National and regional ministries and government bodies responsible for employment or education policies | AT, BE, BG, CY, CZ, DE, DK, EE, EL, FI, FR, HU, IE, IT, LT, LV, MT, PL, RO, SI, SK | 26 |
| Public Employment Services (PES) | BE, BG, EE, ES, HR, IT, LT, LU, LV, MT, RO, SE, SI, SK | 16 |
| Employer organisations | AT, BG, CY, CZ, DE, DK, EE, EL, ES, FI, HR, HU, IE, IT, LT, LU, MT, NL, PL | 22 |
| Trade unions | AT, BG, CY, CZ, DE, EL, ES, FI, FR, HR, LU, LV, NL, PL, SI | 16 |
| Civil society/youth organisations | AT, BE, CY, EL, ES, FI, HR, HU, IE, IT, LT, LU, LV, MT, NL, PL, RO, SI | 17 |
| Other stakeholders (e.g., research institutes; education and training organisations, etc.) | AT, BG, HU, IE, LV, NL, PL, PT, RO | 10 |
| Total of national level interviews | | 107 |

Source: Ecorys, 2022

2.5 Case studies

Seven case studies were conducted as part of the consultations. As part of the case studies, additional in-depth interviews were conducted with representatives involved in the implementation and monitoring of the QFT, as well as three focus groups with young people/trainees. The case study interviews and focus groups were conducted from March to July 2022.

A total of 48 interviews and three focus groups (Bulgaria, Ireland, Spain) were conducted for the case studies, with stakeholders from the following categories:

·Ministries, governmental departments, and other public authorities responsible for employment and/or education and training policies

·Public Employment Services (hereafter ‘PES’);

·Social partners (trade unions, employer organisations, chambers of commerce, chambers of employees);

·Civil society organisations, including youth organisations and youth representatives;

·Research institutions

2.6 Internet-based survey of former, current and potential trainees

The trainee survey investigated the target groups’ experience and/or expectations with traineeships, their interest in and perceived challenges of undertaking traineeships, including cross border opportunities. For current and former trainees, the survey included questions on their background, their experience with finding a traineeship opportunity, as well as their working conditions, and the outcomes of their traineeship.

The trainee survey ran from 1 March to 25 March 2022 and a total of 3,814 responses were collected. Out of these, 1,836 (48%) were from respondents belonging to the core target group (i.e. 18-30 year olds with traineeship experience in EU Member States from 2014 onwards). The breakdown of all survey respondents is presented in table 3 below.

 Table 3 Breakdown of survey respondents

|  |  |  |
| --- | --- | --- |
| Category | Number of respondents | Share |
| Total respondents | 3,814 | 100% |
| Within the core target group | 1,836 | 48% |
| People with no traineeship experience | 702 | 18% |
| Respondents with only mandatory traineeship experience | 962 | 25% |
| People with traineeship experience before 2014 | 65 | 2% |
| Respondents with traineeship experience from non-EU countries | 75 | 2% |
| Respondents with traineeship experience in multiple EU countries | 84 | 2% |
| People with traineeship experience in the EU institutions | 90 | 2% |

Source: QFT online survey 2022

2.7 Expert meeting

An ex-ante expert meeting investigated key evaluation criteria, with a particular focus on national approaches to the implementation of the QFT. It was held online on 26 April 2022 to gather the views of selected consultees on preliminary findings on all evaluation criteria, identify examples of good and bad practices, and discuss ways forward to foster the implementation of the QFT, and quality traineeships more broadly, across the EU.

The expert meeting was attended by a total of 48 participants, 30 of whom were representatives of various stakeholders from 18 Member States (Austria, Croatia, Estonia, Finland, France, Germany, Greece, Hungary, Italy, Ireland, Latvia, Lithuania, Malta, the Netherlands, Poland, Romania, Slovenia and Spain). The remaining 18 were representatives from EU-level organisations (4), the European Commission (5) and the contractor research team (8), and one non-EU participant.

2.8 Validation workshop

A validation workshop was held on 20 September 2022 as a final consultation activity to validate the findings and for selected participants to share their views on lessons learned and next steps. Participants included representatives of national authorities in charge of traineeships (i.e., Ministries of employment, social affairs and/or education), PES, as well as representatives of social partners at EU and national level, and EU level civil society organisations.

The online validation workshop was attended by a total of 49 participants, 29 of whom were representatives of various stakeholders from 15 Member States (Austria, Belgium, Bulgaria, Denmark, Estonia, Greece, Hungary, Italy, Lithuania, Malta, the Netherlands, Portugal, Romania, Slovenia, Spain). The remaining 20 were representatives from EU-level organisations (7), the European Commission (5) and the contractor research team (8). As a follow up to the validation workshop, three additional written contributions were received from EU level trade union (1) and EU level employer organisations (2).

2.9 Challenges linked to the consultations

Implementation of the consultation strategy took place as planned. One of the main challenges was the low response rate for the pulic consultation. However, the extensive interview programme, and the dedicated survey targeting former, current, and potential future trainees allowed to collect the opinions of an important number of stakeholders, including young people.

Another challenge was the unresponsiveness of a number of national level stakeholders. Some of them felt that they did not have the relevant expertise to contribute to the evaluation in the context of the targeted interviews. However, the impact of this on the consultations was minimal, as the other targeted consultations conducted allowed to still gather in-depth views of a range of stakeholders across the EU27.

3 Methodology for data processing

The feedback on the evaluation roadmap went through a qualitative analysis to identify common trends and relevant insights.

The analysis of results of the public consultation was carried out using both quantitative and qualitative methods. The quantitative data analysis included analysis of frequency distribution for each of the variables related to the closed-ended questions, and cross-tabulations between specific variables and characteristics of respondents and between specific variables, though the low number of responses has limited the possibilities for this. For the qualitative data analysis, information was classified by related variable (number of question) and analysed to identify additional information and trends.

For the targeted interviews with EU and national level stakeholders, the write-ups from the interviews were exported into analytical grids, broken down by the different questions and by the respective evaluation criteria. Analytical grids were used to carry out an in-depth analysis of the data.

Information gathered from the case study interviews and focus groups was used in the analysis of each country case study report. The case study reports were used to inform the research analysis.

The trainee survey was analysed using both quantitative (to analyse the frequencies of the closed answers) and qualitative methods (for the open questions, to analyse complex concepts and substantiate and interpret the quantitative data with relevant insights), informing thus all parts of the research. The responses to the open-ended questions provided by the respondents in their native language were translated into English for better interpretation.

For the expert meeting, validation workshop and the consultation meeting of European social partner organisations, the outcomes of the discussions were collected in meeting reports, and they informed the whole analysis.

4 Overview of results of consultation activities

4.1 Effectiveness

|  |
| --- |
| National authorities responsible for education and training and employment policies, PES, social and economic partners at national and EU level (trade unions and employer organisations), EU and national level organisations representing young people and/or young trainees, and research /academia and other international organisations were all consulted on questions related to the effectiveness of the QFT, through the interviews, the public consultation, the case studies, the expert meeting and the validation workshop. While trainees were not directly asked about the effectiveness of the QFT, their feedback on their traineeship experience has been triangulated with the outcomes of the other consultation activities to assess the impact of the QFT. |

The consultations showed that the QFT principles have been implemented to varying degrees in national legislation since 2014. This was confirmed by interviews with national authorities, trade unions and civil society organisations in particular, and by the legal review undertaken in the framework of the evaluation. With regards to the implementation of the QFT on the ground, the majority (74%, 132 out of 178) of organisations responding to the public consultation indicated that the traineeships they offer comply with the QFT to either a large or moderate extent. Among these, 100% of respondents from public enterprises (5 out of 5) reported that their organisation complied to a large degree with QFT principles, followed by the majority of civil society representatives (67%, 4 out of 6), 65% (40 out of 62) of respondents from employment services, and 33% (3 out 9) of social partners. National authorities, youth organisations, and trade unions consulted during the interviews, the case studies, the expert meeting, and the validation workshop, however, highlighted several elements that affect the implementation of the QFT on the ground. These include lack of capacity of labour inspectorates or the PES to adequately monitor both open market and ALMP traineeships, lack of clarity with regards to the legislative framework on and/or definition of open market traineeships leading to unclear roles and responsibility in relation to monitoring, and different approaches to the involvement of trainees themselves in monitoring processes. On the other hand, interviewed employer representatives noted that excessive monitoring might discourage businesses from taking on trainees due to burdensome administrative and/or legal requirements or fear of sanctions (e.g. having to return subsidies received to offer traineeships).

The responses to the trainee survey show a positive impact of the QFT on trainees. 85% (1.561 out of 1.836) of trainees surveyed either strongly agreed or agreed that through their traineeship they learnt things that are useful professionally, whilst 83% (1.524 out of 1.836) noted that they acquired real-life work experience. In the public consultation, half of all respondents (50%, 154 out of 259) reported that the QFT helped young people move into stable employment either to a large or moderate extent, with public authorities standing out as the respondent group more likely to report that the QFT helped young people to secure employment to a large extent (31%, 26 respondents out of 85) and to a moderate extent (48%, 41). Furthermore, national authorities, PES, employer organisations, trade unions, youth organisations, and representatives from other international organisations participating in the interviews and the case studies reported that, besides job-specific competencies, traineeships allow for the development of transversal skills (e.g. time management, organisational skills, team work, communication skills, self-confidence) required at the labour market. However, despite seeing a positive impact on employability, these stakeholders overall agreed across all consultation activities that traineeships do not guarantee access to the labour market. This is further confirmed by the results of the trainee survey, with the majority (58%, 1.065 out of 1.836) of respondents reporting they did not receive a job offer after their traineeships.

Evidence from the consultations points to learning objectives and written agreements as the QFT principles that have a particularly positive impact on fostering young people’s stable labour market integration. This was in particular highlighted by the EU and national level employer organisations and trade unions participating in the interviews, expert group, and validation workshop. Interviewed youth organisations, trade unions, and national authorities recognised the key role played by the written agreement in both ensuring transparency on rights and obligations and setting out learning objectives. In the public consultation, concluding a written agreement at the beginning of the traineeship was deemed as very important by the vast majority of respondents (77%, 200 out of 259).

With regards to the impact of the QFT on sectors, consulted stakeholders across all categories had limited views and/or knowledge on sectoral differences in relation to quality traineeships. For example, in the public consultation, 39% of respondents (69 out of 178) representing organisations responded that they did not know if the QFT had a positive impact in their sector.

In terms of the impact of the QFT across social subgroups, national authorities, youth organisations, trade unions, employer organisations consulted during the interviews, case studies, and the expert group reported that the QFT principles are less effective for vulnerable groups as they do not directly address the obstacles they face in accessing traineeship opportunities and that marginalised youth are less able to benefit from traineeships due to financial barriers to accessing traineeships (i.e., lack of financial compensation), or address the challenges faced by specific groups (e.g. young people in rural areas, young people with disabilities, young people belonging to ethnic minorities). The trainee survey, moreover, revealed that trainees’ own networks often play a key role in accessing traineeship opportunities, with 33% (606 out of 1.836) of respondents indicating that they had found their traineeship through their friend circle, acquaintances, or families.

Lastly, regarding the extent to which the QFT has been effective in improving traineeship quality compared to the European Framework of Quality and Effective Apprenticeships (EFQEA) on apprenticeship opportunities, EU and national level trade unions, youth organisations and employer organisations consulted through the interviews and the expert group considered that the EFQEA has had greater impact on quality. This is due to several factors, including a more collaborative approach to apprenticeships, a higher degree of regulation in apprenticeships and the use of employment contracts. Lastly, the establishment of the European Alliance for Apprenticeships (EAfA) was identified by interviewed EU level employer organisations, trade unions, and youth organisations as a success factor in promoting quality apprenticeships.

4.2 Efficiency

|  |
| --- |
| National authorities responsible for education and training and employment policies, Public Employment Services, social and economic partners at national and EU level (trade unions and employer organisations) and research /academia and other international organisations were all consulted on questions related to the efficiency of the QFT, through the interviews, the public consultation, the expert meeting and the validation workshop. Trainees were not asked about the costs of implementation of the QFT in the survey of trainees, given the fact that they are unlikely to have such knowledge. However, trainees and potential trainees were consulted on the benefits of traineeships and of QFT implementation through the survey of trainees and the case study focus groups. |

  

Consultation activities undertaken for the evaluation provided limited evidence on the costs and benefits associated with the implementation of the QFT or quality traineeships. The following costs were identified:

·The majority of employer organisations and employers (7 out of 10, 70%) consulted in the public consultation agreed that there were administrative costs for employers involved in implementing the QFT. 48% of all organisations responding to the public consultation (89 out of 184) pointed to such costs (i.e. paperwork, submission of reports). EU and national level employer organisations consulted through the interviews, public consultation, case studies, expert meeting and validation workshop highlighted some specific costs, such as direct labour costs of identifying and training supervisors and certifying trainees’ skills, related to implementing the principles of the QFT. National authorities consulted during the interviews and case studies and employer organisations and employers consulted during the interviews, case studies, and validation workshop noted that, while these costs are often subsidised or reimbursed, applying for financial support is costly in itself, and may represent a burden.

·Adjustment costs for national authorities: Interviewees from national authorities mentioned costs incurred to adapt to the requirements of the QFT, including labour costs associated with designing and monitoring traineeship programmes, implementing new legislation in line with the QFT, investing in PES to monitor compliance, as well as costs associated with the provision of subsidies and grants.

·Furthermore, employer organisations and employers consulted during the interviews and case studies highlighted other costs associated with implementing traineeships in general, not specifically linked with the QFT. These include the costs of advertising traineeship opportunities, the costs of covering trainees with insurance against work accidents, and costs to cover a monthly allowance for trainees.

All types of stakeholders consulted across all consultation activities reported that measures introduced since the adoption of the QFT in 2014, and quality traineeships more generally, contribute to a range of benefits for young people, employers, and society. More than half of the organisations responding to the public consultation rated the benefits of implementing the QFT as quite high (44%, 77 out of 177) or very high (14%, 25 out of 177), with 47% of public authorities (37 out of 79) and 58% of academic/research institutions (32 out of 55) agreeing with this.

In terms of benefits for young people, during the interviews, and in responses to the trainee survey and to the public consultation, trainees, national authorities, and trade unions mentioned reduced exploitation of trainees and, in turn, an increase in the number of quality traineeships. Employer organisations and individual employers consulted during the interviews, the case studies, the validation workshop and the public consultation identified also improved skills levels and skills recognition as a benefit. Lastly, 62% of trainees surveyed through the trainee survey agreed that traineeships made their school-to-work transition easier. Similarly, EU level organisations interviewed and consulted during the expert meeting and validation workshop mentioned improved certification of skills as a benefit resulting in increased employability.

Turning to benefits for employers, national authorities consulted during the interviews argued that an improved understanding of the key features of quality traineeships is a direct benefit of the QFT as it allows businesses and the PES to better structure their traineeship programmes. Employer representatives interviewed highlighted the reputational benefits of offering quality traineeships, which can result in higher number of skilled young professionals applying to work for them. In addition, quality traineeships give employers the opportunity to assess the abilities of potential recruits without incurring in major financial risk. Views of employer organisations and employers responding to the public consultation were also relatively positive, with 6 out 10 (60%) reporting that the benefits of QFT were quite high and 2 out of 10 (20%) very high.

At societal level, all types of stakeholders consulted through all consultation activities, both at EU and national level, mentioned the potential of quality traineeships to have a positive impact in the medium and long term, by reducing youth unemployment and improving school-to-work transitions through fostering young people’s employability. However, national authorities interviewed stressed that the difficulties with isolating the specific effect of the QFT principles on traineeships make it harder to build a direct link between the QFT and broader societal impact.

The overall view of employer organisations and national authorities was that costs linked with the QFT and/or quality traineeships are generally low whereas benefits, especially those expected to occur in the future, are potentially large. However, national authorities in particular pointed out in the interviews that several factors can affect the cost-benefit ratio, including the fact that efficiency can only be achieved if the QFT results in quality traineeships, as low-quality opportunities would result in higher personal costs for trainees and lower benefit to employers.

4.3 Coherence

|  |
| --- |
| National authorities responsible for education and training and employment policies, Public Employment Services, and social and economic partners at national and EU level (trade unions and employer organisations) were consulted on questions related to the coherence of the QFT, through the interviews, the case studies, the expert meeting and the validation workshop. Given the specific national policy knowledge required to assess policy coherence of the QFT with national and EU policy, questions on coherence were not asked in the public consultation nor in the trainee survey. This was a methodological choice to ensure both surveys remained short and relevant to the stakeholder groups concerned, so as to increase response rates, and in view of the fact that other consultation activities were able to gather sufficient evidence on this evaluation criterion. |

  

National authorities, employer organisations, trade unions and youth organisations consulted through the interviews, the case studies, the public consultation, the expert group and the validation workshop overall agreed that there is a good level of coherence between the QFT and relevant policies and strategies at national and regional level. This overall coherence was considered to be largely due to a shared policy goal of providing young people with relevant, high-quality work experience and appropriate skills within a safe environment in which their rights are protected. However, stakeholders from national authorities highlighted in the interviews and case studies that there was generally a greater degree of coherence with national and regional policy for ALMP traineeships than for open market traineeships.

Moreover, factors contributing to good coherence include the following:

·Implementation of the QFT principles prior to the adoption of the QFT, as mentioned some national authorities consulted in the interviews and case studies.

·Introduction of new policies increase alignment, as pointed out by youth representatives and some national authorities interviewed and consulted in the case studies.

·National and EU funding for traineeships (e.g. European Social Fund; Youth Employment Initiative) has been a driver for promoting coherence with the QFT, by supporting the implementation of QFT principles and bringing national and regional actors together to develop education and training, or employment measures, as stressed by interviewed national authorities, PES and youth organisations.

Despite the overall positive feedback, a wide range of stakeholder groups consulted (national authorities, employer organisations, trade unions, youth organisations) also identified some factors hampering or limiting coherence with national and regional polices, including: lack of horizontal coordination mechanisms between policies and policy makers across relevant sectors (i.e., Ministries of education and employment), the relatively narrow scope of the QFT, which does not allow for cooperation between policymakers from different relevant fields, lack of harmonised vocational education and training systems which result in different quality criteria, monitoring procedures and approaches to work-based learning, as well as lack of ambition of the QFT compared to other EU level instruments (e.g. European Pillar of Social Rights) or national and regional policies, which limits coherence as national and regional measures may be more advanced.

Finally, national authorities, EU and national level social partners and youth organisations consulted in the interviews, case studies and expert meeting shared the view that the QFT has a good degree of coherence and complementarity with other related EU policies and funding mechanisms (e.g. Youth Guarantee, Youth Employment Initiative, European Social Fund) and that there is no duplication or overlap.

4.4 EU added value

|  |
| --- |
| National authorities responsible for education and training and employment policies, PES, social and economic partners at national and EU level (trade unions and employer organisations), EU and national level organisations representing young people and/or young trainees, research /academia and other international organisations were all consulted on questions related to the EU added value of the QFT, through the interviews, the public consultation, the case studies, the expert meeting and the validation workshop. Trainees were not asked about the EU added value of the QFT in the trainee survey given the fact that they are unlikely to have such knowledge and the priority given to simplifying the trainee survey in order to maximise response rates from this stakeholder group. |

The perception of EU added value resulting from the implementation of the QFT was overall positive, as evidenced in the consultations with national authorities, PES, youth organisations, trade unions, and employer organisations through the interviews, the public consultation, the case studies, the expert meeting and the validation workshop.

Interviewed national authorities, civil society, youth organisations, and trade unions recognised the value of the QFT in setting out common guidelines. National authorities from Member States with less developed systems stressed the added value brought by the QFT in fostering policy and legislative changes at national level. Employer organisations participating in the interviews and the validation workshop praised the flexibility of the QFT in allowing Member States to decide how and to what extent to implement its principles. The public consultation confirmed this positive view, with most respondents stating that the implementation of the QFT produced added value to a large extent or moderate extent by increasing the number of quality traineeships (71%, 119 out of 169), with public authorities standing out as the respondent group expressing the highest support for the QFT in this area, and encouraging young people to take up traineeships (71%, 119 out of 169), strongly supported by respondents from academic/research institutions.

However, despite recognising the important role of the QFT in providing overarching standards, a number of consultees questioned the extent to which the QFT had direct impact on legislation and on traineeships on the ground. These views were held by interviewed national authorities from Member States where legislation and frameworks on traineeships existed prior to 2014. According to them, the added value of the QFT lies mostly in fostering dialogue and increasing the debate around quality traineeships. Some employer organisations consulted in the interviews and the case studies also argued that existing differences between labour markets across the EU mean that overarching instruments such as the QFT bring little added value in practice. Views from respondents to the public consultation seem to align with these concerns as only 22% of respondents (32 out of 169) indicated that the QFT has prompted structural policy changes at the national level.

Lastly, stakeholder views at the EU level on discontinuing the QFT were mixed both across and within consulted groups. A number of social partners, national authorities, and youth organisations consulted through the interviews, case studies, and validation workshop commented that discontinuing the QFT or continuing as it stands would have no impact as its principles have been embedded into national legislation in most Member States. Other consultees from the same stakeholder categories and participating in the same consultation activities, however, thought that discontinuing the QFT would likely result in a loss of guidance and impetus for the further development of national frameworks, and called for the QFT to be strengthened. Some employer organisations and national authorities highlighted the need to invest in supporting actions (e.g. promoting awareness raising and mutual learning) and youth organisations and trade unions stressed the importance of adding new principles to the QFT and of reconsidering its non-binding nature.

4.5 Relevance

|  |
| --- |
| National authorities responsible for education and training and employment policies, PES, social and economic partners at national and EU level (trade unions and employer organisations) and research /academia and other international organisations were all consulted on questions related to the relevance of the QFT, through the interviews, the public consultation, the case studies, the expert meeting and the validation workshop. While trainees were not directly asked about the relevance of the QFT in the trainee survey, their feedback on their traineeship experience through the trainee survey has been triangulated with the outcomes of the other consultation activities to assess the relevance of the QFT. |

There is strong agreement across stakeholders consulted that the QFT continues to be highly appropriate to fostering the labour market integration of young people, with the learning content and the written agreements standing out as the QFT principles considered to be the most relevant in the context of a changing labour market. There is agreement across all stakeholder groups that establishing the learning objectives is key in increasing young peoples’ employability, with 98% (254 out of 259) of respondents to the public consultation identified them as either very important or important to ensure the quality of traineeships. Trade unions and employer organisations agree on the importance of the learning component, with 88% (7 out of 8) of companies/business organisations and 100% (6 out of 6) of trade union respondents to the public consultation identifying it as particularly important.

PES and national authorities interviewed in particular highlighted that the written agreement, reflecting the transparency on the rights and obligations, is a “bedrock” of the traineeship. The public consultation further supports this: concluding a written agreement was identified as a key element increasing traineeship quality by 77% (200 out of 259) of respondents, with 71% (24 out of 34) of former or current trainees identifying the existence of a written agreement as very important for quality traineeships.

During the interviews, a number of representatives from national authorities, PES, youth organisations, and trade unions highlighted that the QFT’s exclusive focus on open market and ALMP traineeships, and the exclusion of traineeships that are part of formal education or training programmes, might limit its relevance. Trade unions highlighted this also during the social partner consultation. However, both the employer organisations and the national authorities at the validation workshop stressed that broadening the scope of the QFT would result in clashes with existing legislation regulating compulsory traineeships and/or traineeships that are part of education.

All types of stakeholders agree that the QFT is still very relevant to respond to needs within the EU in terms of supporting young people to enter the labour market. However, adjustments to ensure that the QFT can respond to new challenges may be warranted. For example:

·Employer organisations at EU and national level participating in the interviews and validation workshop stressed that there is room to increase the relevance of the QFT in tackling skills mismatches through an increased focus on the learning component of the QFT. According to these stakeholders, this could entail increased efforts by traineeship providers to map their own skills needs to offer more targeted opportunities; greater focus on skills recognition; and greater support and guidance for employers to carry out skills assessments and provide adequate supervision. Employer organisations also mentioned the labour and skills shortages at the social partner consultation.

·Consulted stakeholders also noted that the push towards remote and hybrid work brought about by the pandemic may require adjustments for the QFT to remain relevant. EU level stakeholders and national authorities, trade unions, and PES taking part in the interviews, the case studies, and the expert group feared that remote/hybrid traineeships present risks (e.g. digital learning not suiting all individual learning styles, lack of socialisation negatively affecting motivation and engagement, and, in turn, trainees’ wellbeing, further marginalisation of disadvantaged groups lacking digital skills and/or access to adequate ICT equipment and increased costs for traineeship providers) which the QFT should address. However, EU level employer organisations participating in the validation workshop mentioned that the current lack of data on the prevalence remote traineeships across sectors means that no additional principles on remote working should be included in the QFT to increase its relevance.

·Employer organisations at the social partner consultation also highlighted the need for the QFT to focus more on digital and green transitions.

However, at the social partner consultation both employer representatives and trade unions regretted the lack of data on traineeships across the EU, making it difficult to understand the actual situation of traineeships. Trade unions also suggested that labour inspectorates are tasked to monitor traineeships, e.g. whether regular jobs are being replaced by traineeships.

Stakeholders’ views on the non-binding nature of the QFT as a Council Recommendation remain polarised. Youth organisations, trade union representatives, as well as a minority of national authorities consulted through the interviews, expert group, validation workshop and social partner consultation held the view that the non-binding nature of the QFT leaves excessive room for Member States to decide whether and how to implement the QFT, undermining its relevance and not leading to concrete legislative changes. EU and national level employer organisations as well as the majority of national authorities participating in the same consultation activities, on the other hand, stressed that a non-binding tool allows Member States to retain sufficient flexibility to take into account existing national industrial relations and education and training practices.

Lastly, the consultations showed differences in views on the need to include additional principles to increase the relevance of the QFT, with stakeholder groups clearly split around the issue of trainee remuneration and access to social protection as follows:

·Trade unions, civil society and youth organisations and EU and international agencies participating in the interviews, the case studies, the expert group, social partner consultation and the validation workshop agreed that a requirement to pay and/or reimburse trainees would increase the relevance of the QFT, as paid traineeships would result e.g. in higher productivity and better reputation of employers, and ensure increased accessibility for all groups of young people, including young people with disabilities. In the trainee survey, 87% (1.597 out of 1.836) of trainee respondents reported that being paid for their traineeship was essential or important for them. Trainees who received an allowance during their traineeships indicated that their compensation was either sufficient to a small extent (40%, 734 out of 1.836) or not sufficient at all (22%, 404 out of 1.836) to cover basic living costs. 76% (198 out of 259) of respondents to the public consultation stated that ensuring trainees are paid would increase the quality of traineeships to a large extent, including the majority of public authorities (91%, 77 out of 85), trade unions (100%, 6 out of 6), and current/former trainees (74%, 25 out of 34).

·Employer organisations at national and EU level consulted through the interview programme, the case studies, social partner consultation and the validation workshop, however, cautioned against the potentially negative consequences and/or the challenges of trainee pay. These include further blurring the distinction between trainees and workers, interfering with national labour law, as well as additional costs for traineeship providers (e.g. HR and administrative costs), potentially discouraging employers from offering traineeship opportunities at all. Costs could disproportionately impact SMEs, which tend to be the majority of traineeship providers; the trainee survey found that the vast majority of trainees undertook their traineeship in medium (50%, 478 out of 956) or small size companies (24%, 229 out of 956), as opposed to large employers (17%, 162 out of 956). However, a minority of employer organisations participating in the interviews recognised the benefits that paid traineeships can bring to traineeship providers, including avoiding reputational damage from offering unpaid opportunities and cultivating a greater sense of ownership from trainees, leading to higher productivity. Furthermore, the responses to the public consultation show that the majority of enterprises offering traineeships (60%, 3 out of 5) consider that ensuring trainees are paid would increase the quality of traineeships to a large extent.

Consultees from trade unions, youth and civil organisations at the national and EU level participating in the interviews, social partner consultation and the expert group highlighted that a new principle on access to social protection would increase the relevance of the QFT by setting trainees on a positive trajectory in the world of work. The trainee survey confirmed this, with 84% (3.181 out of 3.787) of respondents considering that being covered by health and sickness benefits as essential or important, and 78% (2.954 out of 3.787) of trainees surveyed indicating that having access to minimum income support is either essential or important. In the public consultation, ensuring access to a variety of additional benefits received broad support from respondents: health and sickness benefits (84%, 217 out of 259), minimum income (78%, 202 out of 250), and pension rights (76%, 197 out of 259). However, as with trainee pay, employer organisations, particularly at EU level, consulted through the interview programme and participating in the validation workshop and the social partner consultation, stressed that a principle on social protection would affect the flexibility that the QFT warrant, and result in increased costs and administrative burden, discouraging businesses from offering traineeships. When it comes to access to specific benefits, the public consultation showed that respondents from enterprises offering traineeships were the group most frequently held the view that access to unemployment benefits would increase the quality of traineeships only to a small extent (40%, 2 out of 5).

  

Annex VI. Research questions of the evaluation study

|  |  |
| --- | --- |
|  | Current situation and developments 2014 - 2021 |
| 1. | What are the most common characteristics of trainees across the 27 Member States (e.g. sex, age, education background, socioeconomic background, migrant background, ethnic minority status)? How have these developed since the baseline of 2014? |
| 2. | What is the current prevalence of traineeships across the 27 Member States? In which sectors are they most prevalent and what are the typical working conditions (e.g. ALMP versus open market, duration, remuneration, social protection coverage)? How has all of this developed since the baseline of 2014? |
| 3. | What is the public debate across the 27 Member States, if any, about the role of traineeships and the importance of their quality for young people (political, civil society, representations of young people)? How has it developed since the baseline of 2014? |
|  | Relevance |
| 4. | To what extent are the principles appropriate for fostering stable labour market integration? Which principles are likely to be the most and the least important? |
| 5. | Has the understanding of traineeship quality evolved over time? How well do the principles of the QFT still correspond to the needs within the EU? How well adapted is the QFT to a post-COVID-19 world, a changing labour market and the latest perspectives of stakeholders and citizens? |
| 6. | Are there any additional dimensions that should have been added to the principles from the start or have in the interim proven to be lacking? How – and how likely – would such additions add to quality traineeships, and, in turn, their contribution to a stable labour market integration? |
| 7. | With due respect to the principle of subsidiarity, how well does the nature of the QFT as a (non-binding) Council Recommendation correspond to the needs and the socio-economic problems to be solved? Has this changed over time, and are their significant differences between Member States? |
|  | Effectiveness |
| 8. | To what extent have the principles of the QFT been enshrined in national legislation and/or national quality frameworks since 2014? To what extent did they already exist? What is the scope of current national legislative frameworks and national quality frameworks? |
| 9. | To what extent do enforcement and/or regular follow-up monitoring exist and to what extent do they confirm an adherence to national legislation and/or national quality frameworks and/or the overarching QFT? Is there evidence of adverse effects too? |
| 10. | What is the impact on trainees? How effective do (ex-)trainees perceive traineeships (and/or specific legislative frameworks and/or specific national quality frameworks) to be in terms of facilitating a stable labour market integration for young people, and contributing to youth employment? Is there evidence of adverse effects too? |
| 11. | How equally are the effects of the QFT, by way of national legislative frameworks and/or national quality frameworks, distributed within Member States? Are there sectors or (social) sub-groups for which the QFT has proven to be particularly successful or unsuccessful? |
| 12. | How do observed developments since the baseline of 2014 compare against the developments that were expected to be achieved when the QFT was developed/proposed [178](#footnote178) ? |
| 13. | While taking into account the differences in objectives and target groups, has one of the two quality frameworks, the QFT or the EFQEA, been more effective in improving the quality of traineeships and apprenticeships respectively, as well as the employability of trainees and apprentices, and why? |
|  | Efficiency |
| 14. | What was the quantifiable administrative burden associated with the adequate implementation of the QFT for the different stakeholders at various levels (national, regional, local), in terms of e.g. transposition of QFT principles, enforcement/follow-up monitoring and other compliance checks? |
| 15. | To what extent have Member States encouraged or enabled traineeship prevalence and improved quality through financial incentives? Have any EU programmes (such as the YEI and ESF) contributed directly or indirectly to financial incentives? |
| 16. | What can be approximated in terms of quantifiable benefits for young people? Are there other quantifiable benefits for society at large that could be linked back to the QFT? |
| 17. | What are the main obstacles preventing employers from offering traineeships? What kind of support would be necessary to overcome these obstacles? |
| 18. | To what extent are the administrative costs proportionate to the assessed benefits of QFT implementation? What has been the cost-effectiveness and how/why does it vary across the 27 Member States? What factors influenced the efficiency and how did they do so? |
| 19. | Is there scope for reducing administrative burden without undermining the assessed benefits of QFT implementation? In other words, could benefits have been achieved at lower cost? |
|  | Coherence |
| 20. | To what extent have the objectives, target groups and measures to implement the QFT within the context of ALMPs been coherent with education and training, employment and social policies at national and regional level? How about open market traineeships? |
| 21. | Linked to the former, to what extent do the QFT support and usefully complement other policies (in particular those pursued at national level)? What is the level of complementarity or duplication? |
| 22. | To what extent have the objectives, target groups and measures to implement the QFT within the context of ALMPs been coherent with the relevant EU initiatives listed in section 2.3? How about open market traineeships? |
|  | EU added value |
| 23. | What is the additional value resulting from the QFT compared to what could reasonably have been expected from Member States acting at national and/or regional levels? |
| 24. | What would be the most likely consequences of discontinuing the QFT at EU level, and what would be the most likely prognosis for a no-policy-change scenario of continuing the QFT as it stands? |

  

Annex VII. The principles of the 2014 Quality Framework for Traineeships (QFT)

|  |  |
| --- | --- |
|  | The 2014 Council Recommendation asks Member States to: |
| 1. | Improve the quality of traineeships, in particular as regards learning and training content and working conditions, with the aim of easing the transition from education, unemployment or inactivity to work by putting in practice the following principles for a Quality Framework for Traineeships: |
|  | Conclusion of a written traineeship agreement |
| 2. | Require that traineeships are based on a written agreement concluded at the beginning of the traineeship between the trainee and the traineeship provider; |
| 3. | Require that traineeship agreements indicate the educational objectives, the working conditions, whether an allowance or compensation is provided to the trainee by the traineeship provider, and the rights and obligations of the parties under applicable EU and national law, as well as the duration of the traineeship […]; |
|  | Learning and training objectives |
| 4. | Promote best practices as regards learning and training objectives in order to help trainees acquire practical experience and relevant skills; the tasks assigned to the trainee should enable these objectives to be attained; |
| 5. | Encourage traineeship providers to designate a supervisor for trainees guiding the trainee through the assigned tasks, monitoring and assessing his/her progress; |
|  | Working conditions applicable to trainees |
| 6. | Ensure that the rights and working conditions of trainees under applicable EU and national law, including limits to maximum weekly working time, minimum daily and weekly rest periods and, where applicable, minimum holiday entitlements, are respected; |
| 7. | Encourage traineeship providers to clarify whether they provide coverage in terms of health and accident insurance as well as sick leave; |
| 8. | Require that the traineeship agreement clarifies whether an allowance or compensation is applicable, and if applicable, its amount; |
|  | Rights and obligations |
| 9. | Encourage the concerned parties to ensure that the traineeship agreement lays down the rights and obligations of the trainee and the traineeship provider, including, where relevant, the traineeship provider's policies on confidentiality and the ownership of intellectual property rights; |
|  | Reasonable duration |
| 10. | Ensure a reasonable duration of traineeships that, in principle, does not exceed six months, except in cases where a longer duration is justified, taking into account national practices; |
| 11. | Clarify the circumstances and conditions under which a traineeship may be extended or renewed after the initial traineeship agreement expired; |
| 12. | Encourage the practice of specifying in the traineeship agreement that either the trainee or the traineeship provider may terminate it by written communication, providing advance notice of an appropriate duration in view of the length of the traineeship and relevant national practice; |
|  | Proper recognition of traineeships |
| 13. | Promote the recognition and validation of the knowledge, skills and competences acquired during traineeships and encourage traineeship providers to attest them, on the basis of an assessment, through a certificate; |
|  | Transparency requirements |
| 14. | Encourage traineeship providers to include in their vacancy notices and advertisements information on the terms and conditions of the traineeship, in particular on whether an allowance and/or compensation and health and accident insurance are applicable; encourage traineeship providers to give information on recruitment policies, including the share of trainees recruited in recent years; |
| 15. | Encourage employment services and other providers of career guidance, if providing information on traineeships, to apply transparency requirements; |
|  | Cross-border traineeships |
| 16. | Facilitate the cross-border mobility of trainees in the European Union inter alia, by clarifying the national legal framework for traineeships and establishing clear rules on hosting trainees from, and the sending of trainees to, other Member States and by reducing administrative formalities; |
| 17. | Examine the possibility to make use of the extended EURES network and to exchange information on paid traineeships through the EURES portal; |
|  | Use of European Structural and Investment Funds |
| 18. | Make use of the European Structural and Investment Funds, namely the European Social Fund and the European Regional Development Fund, in the programming period 2014-2020, and the Youth Employment Initiative, where applicable, for increasing the number and quality of traineeships, including through effective partnerships with all relevant stakeholders; |
|  | Applying the Quality Framework for Traineeships |
| 19. | Take appropriate measures to apply the Quality Framework for Traineeships as soon as possible; |
| 20. | Provide information to the Commission by the end of 2015 on the measures taken in accordance with this Recommendation; |
| 21. | Promote the active involvement of social partners in applying the Quality Framework for Traineeships; |
| 22. | Promote the active involvement of employment services, educational institutions and training providers in applying the Quality Framework for Traineeships. |

:   [(1)](#footnoteref1)
     See Annex VII for the full list of the 21 quality principles. In addition to these, the QFT contains the (first point), which is not considered a quality principle on its own, but an overall recommendation to Member States to implement the (21) quality principles that follow thereafter.
:   [(2)](#footnoteref2)
     
    <https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32020H1104%2801%29>
:   [(3)](#footnoteref3)
     Please see Annex VI for the full list of research questions.
:   [(4)](#footnoteref4)
     Commission Staff Working Document ‘Applying the Quality Framework for Traineeships’{COM(2016) 646 final}, {SWD(2016) 323 final}
:   [(5)](#footnoteref5)
     In 2021, at the end of the evaluation period, youth unemployment rate in the EU was 13% whereas the overall unemployment rate was 7%.
:   [(6)](#footnoteref6)
     The adequate implementation includes (a) the reflection of QFT principles in national legislation and/or national quality frameworks; (b) the existence of enforcement and/or regular follow-up monitoring to verify compliance with national legislation and/or national quality frameworks; and (c) on-the-ground compliance with national legislation and/or national quality frameworks.
:   [(7)](#footnoteref7)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023).
:   [(8)](#footnoteref8)
     E.g. European Network of Public Employment Services (2021), Remuneration of Open-Market Traineeships in EU-27; European Commission (2018), Traineeships under the Youth Guarantee: Experience from the Ground, Publications Office of the European Union, Luxembourg; European Commission (2016), Applying the Quality Framework for Traineeships, SWD (2016) 324 Final; the Flash Eurobarometer 378 The experience of traineeships in the EU. November 2013; European Commission (2012), Study on a comprehensive overview of traineeship arrangements in Member States.
:   [(9)](#footnoteref9)
     Flash Eurobarometer 378. The experience of traineeships in the EU. November 2013.
:   [(10)](#footnoteref10)
     See e.g. the Impact Assessment accompanying the document Proposal for a Council Recommendation on a Quality Framework for Traineeships. SWD(2013)495 final. P. 15.
:   [(11)](#footnoteref11)
     The YG Recommendation aimed at ensuring that all young people up to the age of 25 years receive a good-quality offer of employment, continued education, an apprenticeship or a traineeship within four months of becoming unemployed or leaving formal education. In 2020, the YG was reinforced, and it was extended to all young people under the age of 30.
:   [(12)](#footnoteref12)
     The first point in the QFT is not considered a quality principle on its own, but an overall recommendation to Member States to implement the (21) quality principles that follow thereafter.
:   [(13)](#footnoteref13)
     European Commission (2013), Impact Assessment accompanying the proposal for a Council Recommendation on a Quality Framework for traineeships. SWD(2013)495.
:   [(14)](#footnoteref14)
      Flash Eurobarometer 378. The experience of traineeships in the EU. November 2013. The survey was carried out in the (then) 27 EU Member States and Croatia, among EU citizens aged 18-35. Some 12,921 EU respondents from different social and demographic groups were interviewed via telephone in theirmother tongue
:   [(15)](#footnoteref15)
     https://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=yth\_empl\_100&lang=en
:   [(16)](#footnoteref16)
     https://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=yth\_empl\_120&lang=en
:   [(17)](#footnoteref17)

     Flash Eurobarometer 378. The experience of traineeships in the EU. November 2013.
:   [(18)](#footnoteref18)
     European Commission (2012), Study on a comprehensive overview of traineeship arrangements in Member States.
:   [(19)](#footnoteref19)
     European Commission (2013), Impact Assessment accompanying the proposal for a Council Recommendation on a Quality Framework for traineeships. SWD(2013)495.
:   [(20)](#footnoteref20)

    Idem
:   [(21)](#footnoteref21)
     European Commission (2013), Impact Assessment accompanying the proposal for a Council Recommendation on a Quality Framework for traineeships. SWD(2013)495.
:   [(22)](#footnoteref22)
     Unsatisfactory working conditions refer to long working hours, lack of coverage for health and safety or occupational risks, lack of clarity of the applicable legal regimes, etc.
:   [(23)](#footnoteref23)
     European Commission (2012), Study on a comprehensive overview of traineeship arrangements in Member States.
:   [(24)](#footnoteref24)
     European Commission (2013), Impact Assessment accompanying the proposal for a Council Recommendation on a Quality Framework for traineeships. SWD(2013)495.
:   [(25)](#footnoteref25)
     European Commission (2012), Study on a comprehensive overview of traineeship arrangements in Member States.
:   [(26)](#footnoteref26)
     European Commission (2013), Impact Assessment accompanying the proposal for a Council Recommendation on a Quality Framework for traineeships. SWD(2013)495.
:   [(27)](#footnoteref27)
     7.1% in 2021 (Eurostat online code: LFSA\_URGAN)
:   [(28)](#footnoteref28)
     The traineeship prevalence proxy used is the number of students enrolled in tertiary education multiplied by the activity rate to estimate the potential number of young people that may have completed a traineeship (further details can be found in Annex II)
:   [(29)](#footnoteref29)
     See Annex II for the methodological explanation
:   [(30)](#footnoteref30)
     Flash Eurobarometer 378. The experience of traineeships in the EU. November 2013.
:   [(31)](#footnoteref31)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(32)](#footnoteref32)
     The trainee survey covered EU27 and targeted young people (aged 15-29) with traineeship experience or with interest in doing a traineeship. 3,787 responses were collected, of which 1,912 responses were in the main target group (> 18 years, with traineeship experience after 2014 or interested in doing a traineeship). Depending on the question, those respondents doing/having done a mandatory traineeship or an EU institution traineeship were eliminated, resulting in 1,836 responses.
:   [(33)](#footnoteref33)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(34)](#footnoteref34)
     European Commission, Flash Eurobarometer 378: the experience of traineeships in the EU, 2013. P. 10.
:   [(35)](#footnoteref35)
     Data on the number of students enrolled in tertiary education[educ\_uoe\_enrt01] was sourced from: 
    <https://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=educ_uoe_enrt01&lang=en>
     and data on the activity rate [tepsr\_wc160] was sourced from: 
    <https://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=tepsr_wc160&lang=en>
    Data on the number of students enrolled in tertiary education was not available for the Netherlands in 2014.
:   [(36)](#footnoteref36)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(37)](#footnoteref37)
     As 2014 data on cross-border traineeships is lacking, this is based on the European Commission Study on a comprehensive overview of traineeship arrangements in Member States (2012), which found that 9% of traineeships had a transnational nature at the time.
:   [(38)](#footnoteref38)
     
    <https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32016R0589&from=EN>
:   [(39)](#footnoteref39)
     https://eur-lex.europa.eu/legal-content/en/TXT/?uri=CELEX:32014H0327(01)
:   [(40)](#footnoteref40)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(41)](#footnoteref41)
     In addition to the aforementioned limitations regarding the representativeness of both surveys, the comparison between the two surveys needs to be taken with caution, as the surveys are not identical in e.g. questions asked, number and sample of respondents, and methodology applied. However, the 2013 Eurobarometer did serve as the basis for the 2022 trainee survey.
:   [(42)](#footnoteref42)
     The QFT recommends a reasonable duration of in principle 6 months.
:   [(43)](#footnoteref43)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(44)](#footnoteref44)
     Based on European Commission (2012), Study on a comprehensive overview of traineeship arrangements in Member States, which was the most relevant information available for 2014.
:   [(45)](#footnoteref45)
     In FR, open market traineeships are prohibited. Therefore, the specific legal measure should be seen in that light.
:   [(46)](#footnoteref46)
     To be noted that OMTs are rare or do not exist in EE, IT, FI, MT and SK.
:   [(47)](#footnoteref47)
     https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52016SC0324&from=HU
:   [(48)](#footnoteref48)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(49)](#footnoteref49)
     See Annex VII for a full list of the principles of the Quality Framework for Traineeships.
:   [(50)](#footnoteref50)
     1. Improve the quality of traineeships, in particular as regards learning and training content and working conditions, with the aim of easing the transition from education, unemployment or inactivity to work by putting in practice the following principles for a Quality Framework for Traineeships: (
    [QFT Recommendation](https://eur-lex.europa.eu/legal-content/en/TXT/?uri=CELEX:32014H0327(01))
    )
:   [(51)](#footnoteref51)
     19. Take appropriate measures to apply the Quality Framework for Traineeships as soon as possible; 20. Provide information to the Commission by the end of 2015 on the measures taken in accordance with this Recommendation; (
    [QFT Recommendation](https://eur-lex.europa.eu/legal-content/en/TXT/?uri=CELEX:32014H0327(01))
    )
:   [(52)](#footnoteref52)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(53)](#footnoteref53)
     E.g. stakeholders in AT, DE, NL and SI were of this opinion.
:   [(54)](#footnoteref54)
     Such as open market traineeships in EE, FI, FR, IT, MT, SE and SK.
:   [(55)](#footnoteref55)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(56)](#footnoteref56)
     Only on ALMP traineeships in the case of FI and IE.
:   [(57)](#footnoteref57)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(58)](#footnoteref58)
     Legend:Fully/mostly: 10 or more out of 12 principles are complied with. Partially: 6 to 9 out of 12 principles are complied with.Modestly: 3 to 5 out of 12 principles are complied with. Not implemented: 2 or fewer out of 12 principles are complied with. Not applicable. This type of traineeship is rare or does not exist.
:   [(59)](#footnoteref59)
     See recital 11 of the QFT Council Recommendation: ‘The main element of the Quality Framework for Traineeships is the written traineeship agreement that indicates the educational objectives, adequate working conditions, rights and obligations, and a reasonable duration for traineeships’ (https://eur-lex.europa.eu/legal-content/en/TXT/?uri=CELEX:32014H0327(01))
:   [(60)](#footnoteref60)
     In the 2016 SWD, four MS (FR, HR, IT, and LV) were categorised as “Not Applicable”, as the study concluded that in those countries open market traineeships were either illegal, very rare or nearly non-existent and there was no information available on conditions and thus on QFT compliance.
:   [(61)](#footnoteref61)
     Open market traineeships are prohibited in FR.
:   [(62)](#footnoteref62)
     The number of Member States having implemented fully or partially were counted to assess which principles have been implemented most commonly.
:   [(63)](#footnoteref63)
     The number of Member States not having implemented at all were counted to assess which principles have been implemented the least.
:   [(64)](#footnoteref64)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(65)](#footnoteref65)
     To be noted that it was not always possible to disentangle the impact on trainees of national legislation from the impact of practical implementation on the ground
:   [(66)](#footnoteref66)
     “Low” in CZ and EL
:   [(67)](#footnoteref67)
     However, no information could be gathered on AT.
:   [(68)](#footnoteref68)
     AT, BE, DE, ES, HU, LT, LU, NL, RO
:   [(69)](#footnoteref69)

     BG, CZ, HR, IE, LV, PL, SI, PT
:   [(70)](#footnoteref70)
     CY, DK, EL
:   [(71)](#footnoteref71)

     EE, FI, FR, IT, MT, SE, SK (To be noted that OMTs are prohibited in FR)
:   [(72)](#footnoteref72)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(73)](#footnoteref73)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(74)](#footnoteref74)
     AT, BE, BG, CY, EL, DE, DK, EL, ES, HU, IE, LT, LU, PT, RO, SI, SK.
:   [(75)](#footnoteref75)
     In particular those on IT and ES
:   [(76)](#footnoteref76)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(77)](#footnoteref77)
     The 
    [decision of the European Committee of Social Rights (ECSR) in European Youth Forum (YFJ) v. Belgium, Complaint No. 150/2017](https://hudoc.esc.coe.int/fre/?i=cc-150-2017-dmerits-en)
    , which became public on 16 February 2022.
:   [(78)](#footnoteref78)
     Idem. The Charter refers to the Revised European Social Charter. Article 4§1 of the Charter refers to the right of workers to “a remuneration such as will give them and their families a decent standard of living”.
:   [(79)](#footnoteref79)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(80)](#footnoteref80)
     Idem
:   [(81)](#footnoteref81)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(82)](#footnoteref82)
     Idem
:   [(83)](#footnoteref83)
     Idem
:   [(84)](#footnoteref84)
     Idem
:   [(85)](#footnoteref85)
     1,506 of 1,836 (82%) also indicated that the traineeship helped them gain real-life work experience and 1,524 of 1,836 (83%) reported that they acquired skills and competences specific to the sector they were working in
:   [(86)](#footnoteref86)
     Exact figures are: 496 of 1,836 (27%) strongly agree and 918 of 1,836 (50%) agree with the statement ”My tasks helped me achieve my learning and training objectives”.
:   [(87)](#footnoteref87)
     OECD. 
    [Evaluating Latvia’s Active Labour Market Policies](https://doi.org/10.1787/6037200a-en)
    . 2019.
:   [(88)](#footnoteref88)
     Those linked to: written agreement, learning and training objectives, working conditions, rights and obligations, reasonable duration, proper recognition, transparency, and cross-border mobility
:   [(89)](#footnoteref89)
     Guadeloupe, French Guiana, Martinique, Mayotte, La Réunion, Saint-Martin (FR), Azores and Madeira (PT), Canary Islands (ES).
:   [(90)](#footnoteref90)
     In the trainee survey carried out as part of the study supporting this evaluation, the most commonly reported way of finding a traineeship was via the trainees’ own networks, i.e. through friends, acquaintances, or families (33% of respondents).
:   [(91)](#footnoteref91)
     European Commission, Directorate-General for Employment, Social Affairs and Inclusion, Study supporting the ex-post EURES evaluation and the second biennial EURES report, Publications Office, 2021, https://data.europa.eu/doi/10.2767/98807
:   [(92)](#footnoteref92)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(93)](#footnoteref93)
     From the case studies and targeted consultations conducted under the 2023 Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(94)](#footnoteref94)

     ICF (2018) Developing Best Practice in Work-Based Learning - An Evaluation of the Career Traineeship Pilot Final Report 28 March 2018. Available at: 
    <https://www.solas.ie/f/70398/x/8948babc28/developing-best-practice-in-work-based-learning-an-evaluation-of-the-career-traineeship-pilot.pdf>
:   [(95)](#footnoteref95)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(96)](#footnoteref96)
     In addition to subventions at national level, some traineeships can be financially supported by the European Funds.
:   [(97)](#footnoteref97)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(98)](#footnoteref98)
     Idem
:   [(99)](#footnoteref99)
     Idem
:   [(100)](#footnoteref100)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(101)](#footnoteref101)
     Idem
:   [(102)](#footnoteref102)
     Indicador Público de Renta de Efectos Múltiples (IPREM): a reference standard for social benefits
:   [(103)](#footnoteref103)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(104)](#footnoteref104)
     Idem
:   [(105)](#footnoteref105)
     Idem
:   [(106)](#footnoteref106)
    <https://ec.europa.eu/info/strategy/priorities-2019-2024/economy-works-people/jobs-growth-and-investment/european-pillar-social-rights/european-pillar-social-rights-20-principles_en>
:   [(107)](#footnoteref107)
    [https://youth.europa.eu/strategy\_en#:~:text=The%20EU%20Youth%20Strategy%20focuses,European%20Youth%20Goals%20were%20developed](https://youth.europa.eu/strategy_en)
    .
:   [(108)](#footnoteref108)
     
    <https://ec.europa.eu/social/main.jsp?catId=1079&langId=en>
:   [(109)](#footnoteref109)
     
    <https://erasmusintern.org/digital-opportunities>
:   [(110)](#footnoteref110)
     
    <https://ec.europa.eu/social/main.jsp?catId=1176>
:   [(111)](#footnoteref111)

     
    <https://ec.europa.eu/esf/home.jsp>
:   [(112)](#footnoteref112)
     
    <https://ec.europa.eu/info/strategy/recovery-plan-europe_en>
:   [(113)](#footnoteref113)

     
    <https://erasmus-plus.ec.europa.eu/>
:   [(114)](#footnoteref114)
     
    <https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32022L2041&qid=1671007127164>
:   [(115)](#footnoteref115)
    <https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:OJ.C_.2019.387.01.0001.01.ENG&toc=OJ:C:2019:387:TOC>
:   [(116)](#footnoteref116)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(117)](#footnoteref117)

     
    <https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32018H0502%2801%29>
:   [(118)](#footnoteref118)

    European Commission (2012), Study on a comprehensive overview of traineeship arrangements in Member States.
:   [(119)](#footnoteref119)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(120)](#footnoteref120)
     Idem
:   [(121)](#footnoteref121)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(122)](#footnoteref122)
     HR became an EU Member State on 1 July 2013, while BG and RO joined the EU on 1 January 2007.
:   [(123)](#footnoteref123)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(124)](#footnoteref124)
     All interviewed stakeholder groups agreed on this in the targeted consultations. 254 out of 259 (98%,) of public consultation survey respondents considered learning and training objectives as either very important or important. Also in the public consultation survey, 7 out of 8 (88%) of companies/business organisations and 6 out of 6 (100%) of trade union respondents identified the learning component as particularly important.
:   [(125)](#footnoteref125)
     As expressed by PES and national authorities in targeted consultations. The public consultation further supports this: concluding a written agreement at the beginning of the traineeship was identified as a key element increasing traineeship quality by 200 out of 259 (77%) of respondents. 24 out of 34 (71%) of former or current trainees responding to the public consultation survey selected the existence of a written agreement as very important.
:   [(126)](#footnoteref126)
     Supervision is considered a relevant principle particularly by national authorities and PES (e.g. in IE, IT, MT, PL, RO). The majority of respondents to the trainee survey indicated that guidance and support from a supervisor would have been helpful in finding a job after the completion of their traineeships (76% or 1,395 out of 1,836).
:   [(127)](#footnoteref127)
     Employer organisations, national authorities, PES in BE, IE, LU, LT and IT.
:   [(128)](#footnoteref128)
     Youth organisations, trade unions, PES in BE, EL, IT and ES.
:   [(129)](#footnoteref129)
     It should however be noted that the analysis in section 4.1 gives a mixed picture for these countries in terms of conformity.
:   [(130)](#footnoteref130)
     European Parliamentary Research Service - 
    <https://www.europarl.europa.eu/thinktank/en/document/EPRS_STU(2022)699459>
:   [(131)](#footnoteref131)
     Expert meeting (April 2022), dedicated Social Partners’ consultation (June 2022), validation workshop (September 2022).
:   [(132)](#footnoteref132)
     Idem
:   [(133)](#footnoteref133)
     Idem
:   [(134)](#footnoteref134)
     Targeted consultations, validation workshop (September 2022) - Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(135)](#footnoteref135)
     Based on interviews and expert meetings held in the targeted consultations - Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(136)](#footnoteref136)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(137)](#footnoteref137)
     ILO. Interns and outcomes – Just how effective are internships. 2018. This publication stated that “paid internships produce – on average – better labour market outcomes than unpaid internships do”.
:   [(138)](#footnoteref138)
     Smaller number of respondents, as only some respondents were (depending on responses given to previous questions) routed to this question.
:   [(139)](#footnoteref139)
     Tomić, Iva & Zilic, Ivan. (2018). Working for 200 euro? The effects of traineeship reform on labor market outcomes in Croatia. EIZ-WP-1804.
:   [(140)](#footnoteref140)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(141)](#footnoteref141)
     See also the ESPN Thematic Report 
    [Access to Social Protection for Young People](https://ec.europa.eu/social/main.jsp?catId=738&langId=en&pubId=8454&furtherPubs=yes)
    . 2021.
:   [(142)](#footnoteref142)

     European Network of Public Employment Services (2021), Remuneration of Open-Market Traineeships in EU-27.
:   [(143)](#footnoteref143)
     AT, CY, DK, EE, FI, HR, LT, MT, PL, SE.
:   [(144)](#footnoteref144)

    Ibid.
:   [(145)](#footnoteref145)
     Larger number of respondents, as this question was not only routed to former, current, or potential trainees, but to all respondents.
:   [(146)](#footnoteref146)
     In targeted consultations and the validation workshop (September 2022) held as part of the Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(147)](#footnoteref147)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(148)](#footnoteref148)
     This point of view arose from the study consultations with EU level stakeholders and representatives from national authorities, trade unions, and PES from a number of countries (FI, IT, MT, SK, CY, PL).
:   [(149)](#footnoteref149)
     Validation workshop (September 2022) - Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(150)](#footnoteref150)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(151)](#footnoteref151)
     Idem
:   [(152)](#footnoteref152)
     Five Member States (BG, ES, LT, LU and RO) adopted legislation on OMTs, and 12 Member States (BE, BG, DK, EE, EL, HR, IE, IT, LT, PT, RO, and SK) adopted legislation covering ALMP traineeships.
:   [(153)](#footnoteref153)

     
    <https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32018H0502%2801%29>
:   [(154)](#footnoteref154)
     
    <https://www.europarl.europa.eu/doceo/document/TA-9-2022-0045_EN.html>
:   [(155)](#footnoteref155)
     
    <https://www.eesc.europa.eu/en/our-work/opinions-information-reports/opinions/how-guarantee-decent-work-young-people-and-ensure-inclusion-neets-through-proper-elaboration-national-recovery-plans-own>
:   [(156)](#footnoteref156)
     
    <https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52020IR3454&from=EN>
:   [(157)](#footnoteref157)
    <https://www.etuc.org/sites/default/files/press-release/file/2022-02/ETUC%20Open%20Letter_Ban%20Unpaid%20Internships.pdf>
:   [(158)](#footnoteref158)
     
    <https://www.youthforum.org/topics/no-more-unpaid-internships>
:   [(159)](#footnoteref159)
     
    [Report on the final outcome of the Conference on the future of Europe (May 2022)](https://prod-cofe-platform.s3.eu-central-1.amazonaws.com/2po250fn174z62m8g8c9ya9e62m7?response-content-disposition=inline%3B%20filename%3D%22Book_CoFE_Final_Report_EN_full.pdf%22%3B%20filename%2A%3DUTF-8%27%27Book_CoFE_Final_Report_EN_full.pdf&response-content-type=application%2Fpdf&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA3LJJXGZPDFYVOW5V%2F20220714%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Date=20220714T093221Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=b7fb57d806b79d800e16ffba37e5a3b3d24a73b22e445aa8be4507ffe678c566)
:   [(160)](#footnoteref160)
     Written contributions of BusinessEurope and SMEUnited to the consultation carried out as part of the evaluation of the Quality Framework for Traineeships. Not published.
      
    See also 
    <https://www.businesseurope.eu/publications/businesseurope-headlines-no-2022-37>
     as well as the Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(161)](#footnoteref161)
     Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(162)](#footnoteref162)
     https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13118-Quality-Framework-for-Traineeships-review-evaluation-\_en
:   [(163)](#footnoteref163)
    Study supporting the evaluation of the Quality Framework for Traineeships (2023)
:   [(164)](#footnoteref164)
     Data on the number of students enrolled in tertiary education[educ\_uoe\_enrt01] was sourced from: 
    <https://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=educ_uoe_enrt01&lang=en>
:   [(165)](#footnoteref165)
     Data on the activity rate [tepsr\_wc160] was sourced from: 
    <https://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=tepsr_wc160&lang=en>
:   [(166)](#footnoteref166)
     Data on the number of people aged 20-34 [DEMO\_R\_PJANGROUP] was sourced from: 
    <https://ec.europa.eu/eurostat/databrowser/view/DEMO_R_PJANGROUP__custom_2807666/default/table?lang=en>
:   [(167)](#footnoteref167)
     The choice to present the results as a percentage of the youth population is so that you can compare trends more easily between MS, as requested by the ISSG at the interim meeting. Comparing the raw numbers between countries is not very useful when they have such different population sizes e.g one MS might have a much higher number of traineeships compared to another MS, but this may be because they just have a higher population. Presenting the number as a share of the population allows you to make better comparisons between countries.
:   [(168)](#footnoteref168)
     https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13118-Quality-Framework-for-Traineeships-review-evaluation-/public-consultation\_en
:   [(169)](#footnoteref169)
     
    <https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52013SC0495&qid=1622466549066&from=EN>
:   [(170)](#footnoteref170)
     https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1594047420340&uri=CELEX%3A52020DC0276
:   [(171)](#footnoteref171)
     https://ec.europa.eu/social/main.jsp?langId=en&catId=89&furtherNews=yes&newsId=9939#navItem-3
:   [(172)](#footnoteref172)
     https://europa.eu/youth/strategy\_en
:   [(173)](#footnoteref173)
     Where there is a prior impact assessment, the table should contain as a minimum the costs/benefits identified in the IA with the information gathered on the actual cost/benefit. As available, the table should include the monetisation (€) of the costs/benefits based on any quantitative translation of the data (time taken, person days, number of records/equipment/staff etc. affected or involved represented in monetary value – see Standard cost model, for example). For all information presented, it should be included in the comments section whether it relates to all Member States or is drawn from a subset. An indication of the robustness of the data should be provided in Annex II on Methodology and analytical models used.
:   [(174)](#footnoteref174)
     European Commission, 
    <https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13118-Quality-Framework-for-Traineeships-review-evaluation-_en>
:   [(175)](#footnoteref175)
     European Commission, Have Your Say, Quality Framework for Traineeships review (evaluation): 
    <https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13118-Quality-Framework-for-Traineeships-review-evaluation-/feedback_en?p_id=26017674>
:   [(176)](#footnoteref176)
       The full report of the public consultation results can be found on the Commission’s Have Your Say portal: 
    <https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13118-Quality-Framework-for-Traineeships-review-evaluation-/public-consultation_en>
:   [(177)](#footnoteref177)
       These include primarily Ministries of Employment and/or Social Affairs and Ministries of Education, as well as other stakeholders such as national-level socio-economic partners and youth or civil society organisations representing young trainees.
:   [(178)](#footnoteref178)
     
       A useful source in this context may be the Impact Assessment that underpinned the Commission proposal upon which the Council Recommendation was based (https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52013SC0495&qid=1622466549066&from=EN).

[Top](#document1)