Table of contents 1.Introduction 2.Background description and programme objectives 2.1.Baseline and points of comparison 3.Implementation / state of play 3.1.Programming and budget 3.2.Type of operations funded 3.3.Financial and operational implementation 4.Method 4.1.Short description of the methodology 4.2.Limitations and robustness of findings 5.Analysis and answers to the evaluation questions 5.1.Effectiveness: How effective is the YEI, and other ESF-funded youth employment operations, in achieving their objectives? 5.2.Efficiency: How efficient has the YEI, and other youth employment operations funded by the ESF, been in the achievement of their objectives? 5.3.Relevance: How relevant is the YEI and the other ESF-funded youth employment operations? 5.4.Coherence: How coherent are YEI and the other ESF-funded youth employment operations among themselves, and with other actions in the same field? 5.5.EU added value: What is the EU added value of the YEI and other ESF-funded youth employment operations? 5.6.Sustainability: How sustainable are the YEI and the other ESF-funded youth employment operations? 6.Conclusions and lessons learned

6.1.OVERALL CONCLUSIONS 6.2.Lessons learned Annex 1: Procedural information Annex 2: Stakeholder consultation Annex 3: Methods and analytical models



Figures, Tables and Boxes Figure 1 Youth employment intervention logic Figure 2 Youth unemployment rate (15-24) (% of active population), 2014 Figure 3 The rate of young people (aged 15-24) not in employment, education or training, by labour market status (% of population), 2014 Figure 4 Young people in the EU28 (aged 18-24) having left education early, by country and labour status, in 2014 and 2018 Figure 5 Change in unemployment rates for young people aged 15-24 by level of education, 2014-2018 (in percentage points) Figure 6 Share of costs eligible for ESF funding by type of operation Figure 7 Implementation rates of YEI/ESF funding by Member State (expenditure declared to the managing authority vs funding allocated) Figure 8 Progress towards the milestones set for 2018 – YEI Figure 9 Share of NEETs (aged 15-24) receiving ESF/YEI funding as a proportion of the NEET population in 2014 by cluster of regions (%) Figure 10 How successful were the youth employment operations funded by the European Social Fund and the Youth Employment Initiative in providing support to the following target groups? Figure 11 Share of job offers with permanent contracts according to YEI evaluations Figure 12 To what extent do you agree/disagree that the following measures have been actually effective in helping young people enter quality and sustainable employment? (population: 399 organisations)

Figure 13 How would you qualify the following administrative arrangements for the implementation of youth employment operations under the ESF and YEI? Figure 14 Overview of budget allocation to clusters of regions Figure 15 To what extent are the YEI and ESF coherent with other youth and youth employment EU national/regional schemes? (n=399) Figure 16 In your opinion, how useful are the following actions in helping young people find a job, improve their professional skills or enter the labour market? (Group A1, A2 and B, n=976) Figure 17 The support you received helped you in… (Group A1 and A2, n=431, multiple answers allowed) Figure 18 To what extent do you agree or disagree that the following activities implemented under the Youth Employment Initiative were cost-effective? (Group C, n=206) Figure 19 To what extent are the YEI and ESF coherent with other youth and youth employment EU national/regional schemes? (Group C and D, n=399) Figure 20 Has the EU provided support that goes beyond what is addressed by national or regional programmes or support schemes aimed at youth employment? (Group C and D, n=282) Table 1 Funding for Youth Employment ESF + YEI, including amendments until 2018 Table 2 Comparative table of unit costs Table 3 Expenditure per entry: expenditure minimum-rate threshold of 75.1% - breakdown by region and type of intervention Table 4 Use of ESF and YEI funds in implementing national Youth Guarantee schemes, 2016-2019 Table 5 Employment rates by type of operation, upon completion and after 6 months Table 6 Labour market status of individuals who took part in ESF/YEI initiatives Table 7 Number of interviewees by Member State and type of organisation Box 1 Cooperation and partnerships in Germany and Portugal Box 2 Special support for hard-to-reach young people in Belgium Box 3 Common standards and tools developed in YEI programmes

Box 4 ‘From training to employment’ – the Voluntary Labour Corps project in Poland Box 5 Common standards and tools developed in the YEI programme Box 6 The impact of training on the employment rate in Italy Glossary Term or acronym Meaning or definition ALMP Active Labour Market Policies AIR Annual Implementation Report Beneficiary Beneficiary means a public or private body responsible for initiating or both initiating and implementing operations CIE Counterfactual impact evaluations CPR Common Provisions Regulation CSR Country-specific recommendations DG EMPL Directorate-General Employment, Social Affairs & Inclusion DG REGIO Directorate-General Regional and Urban Policy EAFRD European Agricultural Fund for Rural Development EMFF European Maritime and Fisheries Fund EQ Evaluation question ERDF European Regional Development Fund ESF European Social Fund FEAD Fund for European Aid to the Most Deprived GDP Gross domestic product ISCED International Standard Classification of Education JRC Joint Research Centre MS Member State IP Investment Priority NEET Person not in employment, education or training OP Operational Programme Operation Operation means a project, contract, action or group of projects selected by the managing authorities of the programmes concerned, or under their responsibility, that contributes to the objectives of a priority or priorities Participant A persons benefiting directly from an ESF intervention who can be identified and asked for their characteristics, and for whom specific expenditure is earmarked PES Public employment service SFC System for Fund Management in the European Union VET Vocational education and training YE Youth employment YEI Youth Employment Initiative YG Youth Guarantee 1.Introduction This evaluation takes stock of the activities and results achieved by providing EU support for youth employment under the Youth Employment Initiative and the European Social Fund. It builds on the evaluations carried out by the Member States of the Youth Employment Initiative, delivered by December 2018. It also assesses the extent to which the Youth Employment Initiative contributed to meeting the objectives of the Youth Guarantee. The evaluation assessed the effectiveness, efficiency, relevance, EU added value and the sustainability of youth employment activities funded by Youth Employment Initiative and other actions under the European Social Fund over the period 2014-2018. It covers all Member States, particularly those who funded operations under investment priority 8.ii of the European Social Fund and the dedicated investment priority for programming the Youth Employment Initiative resources. It also looked at whether the initiatives were complementary and coherent with other initiatives in this policy area over the same period (training, vocational education, lifelong learning, skills, mobility of young workers). The conclusions of the evaluation are relevant to the ongoing work in the final stages of the Youth Employment Initiative and the 2014-2020 European Social Fund investments in youth employment. The findings can also feed into the process to programme additional resources under the REACT-EU instrument for cohesion policy with the aim of crafting an effective response to the detrimental impacts of the COVID-19 crisis on employment and on society in general. In particular, support for youth employment and protecting vulnerable groups such as young people not in employment, education or training have been identified as some of the key areas for ESF support with additional resources allocated for 2020-2022. Member States will have flexibility in organising the recovery measures for young people, such as to provide incentives for job hiring. The results of the evaluation should feed into the next programming period, in the form of lessons learned (notably on cost effectiveness, outreach and main target groups). This will underpin the negotiation of operational programmes under the European Social Fund Plus and the design of operations by the Member States targeting youth employment. The findings can provide guidance for EU funding on how to further reform youth employment policies and vocational education and training and apprenticeships systems, co-funded by the European Social Fund+ and by the Resilience and Recovery Facility as part of the reinforced Youth Guarantee and the upcoming policy proposals on vocational education and training. The results will also feed into the Commission’s ex-post evaluation of the European Social Fund.



2.Background description and programme objectives Following the financial and economic crisis of the late 2000s, the employment conditions for young people in the EU deteriorated significantly. Following the Commission’s recommendations, the Council issued in April 2013 a Recommendation on establishing a Youth Guarantee. The Youth Guarantee is a political commitment made by the Member States to give every person under the age of 25 (or under 30, if the Member State so decides) a good-quality offer of employment, continued education, an apprenticeship or a traineeship within four months of becoming unemployed or leaving formal education. The Youth Employment Initiative is one of the main EU funding resources to support the Youth Guarantee schemes. However, the aim and scope of the programme are more focused than Youth Guarantee policy. The Initiative provides direct support exclusively to young people who are not in education, employment or training, including the long-term unemployed or those not registered as jobseekers living in regions where youth unemployment was over 25% in 2012. It thus aims to ensure that young people in parts of Europe with the most acute employment challenges can receive targeted support. The Youth Employment Initiative was integrated into the European Social Fund (chapter IV of Regulation (EU) No 1304/2013 of the European Parliament and of the Council on the European Social Fund and repealing Council Regulation (EC) No 1081/2006, the ‘ESF Regulation’), in order to provide support for action under the investment priority in Article 3(1)(a)(ii) of the ESF Regulation (the “IP8.ii”): ‘Sustainable integration into the labour market of young people, in particular those not in employment, education or training, including young people at risk of social exclusion and young people from marginalised communities, including through the implementation of the Youth Guarantee.’ Action under the Youth Employment Initiative often takes the form of pathways or packages of measures with the long-term objective of integrating young people into the active labour market. Under the Initiative, each individual participant receives an offer of employment and/or continued education and training, a traineeship or an apprenticeship, and therefore contributes to the implementation of national Youth Guarantee schemes. A single Youth Employment Initiative operation can cover both the offer itself and the range of support services (such as job search assistance, counselling, psychological support) leading to the job/education/training offer. However, the Initiative cannot directly fund reforms of systems and structures carried out under the Youth Guarantee. The European Social Fund provides additional assistance to youth employment policy, and has a broader scope than the YEI. It includes support for system-related activities and support for regions not eligible for Youth Employment Initiative funding.





2.1.Baseline and points of comparison The situation of youth employment at the start of the programming period for EU28 The economic and financial crisis of the late 2000s had severe effects on young people. Young people trying to make the transition from education or training into work faced a market with fewer jobs and unparalleled levels of competition, not only from their peers but also from adult workers with extensive work experience who had recently been laid off. As a result, they increasingly found themselves unable to get a job without prior experience and yet little chance of getting a job to gain that initial experience. At the beginning of the programming period, the unemployment rate of young people was about 30% higher than it was for older adults in 2014. In 2014, on average across the EU, more than one in five young people was unemployed, with wide variations between the Member States. Source: Eurostat, Labour Force Survey (lfsa_pganws), data extracted on 21 June 2018. The unemployment rate was particularly high for low-skilled young people, where one in three of the population did not have a job, twice the unemployment rate of high-skilled young people. In 2014, 12.5% of young people across the EU aged 15-24 were not in employment, education or training, with an equal share of inactive and unemployed, up from 10.9% in 2008. Again, there were huge variations between Member States (Figure 3). Source: Eurostat, Labour Force Survey (yth_empl_150), date of extraction 21 June 2019. In 2014, more than one in ten (11.2%) young people aged 18-24 left school early, meaning that they had completed at most lower secondary education and were not currently enrolled in further education or training (Figure 4). The rate of early leavers from education and training leaving varied widely between countries. At EU level, six out of ten (59.8%) early leavers from education and training were not employed.

Source: Eurostat, Labour Force Survey (edat_lfse_14), data extracted on 1 July 2019. At EU level, there was only a small gender gap in the rate of young people not in employment, education or training in 2014 (12.3% for men aged 15-24 v 12.7% for women), though some countries recorded more substantial differences. The gender gap is wider in the labour market status of these young people. At EU level, out of all young people not in employment, education or training, 60.2% of young men were unemployed and 39.8% were inactive, compared to 43.3% of young women unemployed and 56.7% inactive. In 2014, almost a third (31.4%) of young people aged 15-24 lived in households at risk of poverty or social exclusion. Poverty was therefore more prevalent amongst this age group than in the population as a whole (24.4%). In 2014, young people aged 15-24 were more likely (compared to the general population) to be at risk of poverty or social exclusion in all EU countries but Estonia. It was especially hard for young people with a migrant background (i.e. people born in a country other than the one in which they seek work) to find employment, compared to nationals. In 2014, the difference in unemployment rates at EU28 level between the two groups was 7.6 percentage points on average. Labour market trends for young people since 2014 Between 2014 and 2018, as the economic situation improved, youth unemployment rates fell in all Member States and by an average of 7 percentage points at EU28 level. However, not all Member States saw the same degree of improvement. There are also differences in the trend of unemployment rate based on education level. Although the overall improvement in the employment rate was higher for the low-skilled (primary/lower secondary education) than for medium (upper secondary education) - and highly (tertiary education) educated young people, the unemployment rate for low-skilled people (21.9%) remains double the rate of unemployed highly educated young people (11.0%).

Source: Eurostat, Labour Force Survey (yth_empl_100), data extracted on 21 June 2018. Over that period, the EU28 average rate of young people not in employment, education or training fell from 12.5% in 2014 to 10.5% in 2018. More importantly, it fell in all countries but Germany and Denmark, where it had only marginally increased. The improvement derives entirely from the fall in the number of unemployed young people (down 3.0 percentage points) as the number of inactive young people slightly increased (up 1.2 percentage points). The composition of the target population is thus shifting, with a higher share of inactive young people (58% in 2018). By 2018, the rate of early school leaving fell to 10.6% while the proportion of early school leavers that were not in employment fell from six in ten to five in ten (52.8%). By 2018, the rate of young people at risk of poverty or social exclusion had fallen from 31.5% in 2014 to 28.1%, though it remained higher than the rate for the overall population (21.8%). For young migrants (15-24 years), the unemployment rate fell from 29.2% in 2014 to 20.9% in 2018. Note that, given the inevitable time lags associated with the quantitative cross-EU data, all figures in this report predate the COVID-19 crisis.





3.Implementation / state of play 3.1.Programming and budget Originally, the resources for the Youth Employment Initiative consisted of: (1)a dedicated budget line (specific allocation) of €3.2 billion; (2)a matching contribution from the European Social Fund of €3.2 billion; (3)national co-financing for the European Social Fund. The specific allocation for the Youth Employment Initiative is not complemented with national co-financing. At the start of the programming period, the total EU budget allocated to the Initiative (under both the Youth Employment Initiative (YEI) and the European Social Fund (ESF)) was €6.4 billion (€7.67 billion including national co-financing of the European Social Fund). Given the persistently high level of youth unemployment, in June 2017, the European Parliament and the Council agreed to increase funding for the Initiative by another €1.2 billion, matched by an equivalent amount of ESF funding (€2.4 billion in total). The increase in ESF funding was also topped up by eligible funding from Member State own resources. All in all (combining EU funding on the dedicated YEI budget line, the matching funding under ESF and national co-financing), the total budget for YEI was €10.4 billion, as shown in Table 1 below. The total ESF budget allocated to Investment Priority 8.ii (non-YEI) is €8.2 billion, bringing the total investment in youth employment operations to around €18.6 billion. Member State ESF — IP 8ii YEI Total EU amount (in €000) Total (in €000) EU amount (in €000) Total (in €000) EU amount (in €000) Total (in €000) AT — — — — — — BE 61,539 133,209 125,788 188,682 187,327 321,891 BG 31,799 37,357 110,377 120,117 142,177 157,474 CY 6,798 7,998 36,274 39,474 43,072 47,472 CZ — — 27,200 29,600 27,200 29,600 DE 467,029 827,733 — — 467,029 827,733 DK — — — — — — EE — — — — — — ES 420,217 589,236 2,723,322 2,963,615 3,143,538 3,552,850 FI — — — — — — FR 195,976 249,915 944,660 1,117,509 1,140,635 1,367,424 GR — — 500,842 574,249 500,842 574,249 HR 35,540 41,812 202,590 220,466 28,130 262,277 HU 503,068 598,801 99,531 108,313 602,598 707,113 IE — — 136,291 204,436 136,291 204,436 IT 1,293,095 2,267,888 1,821,065 2,288,069 3,114,159 4,555,957 LT 17,453 20,533 63,565 69,174 81,018 89,707 LU 6,819 13,638 — — 6,819 13,638 LV — — 58,021 63,141 58,021 63,141 MT 4,800 6,000 — — 4,800 6,000 NL — — — — — — PL 1,256,028 1,488,182 537,635 585,074 1,793,663 2,073,256 PT — — 446,720 486,136 446,720 486,136 RO 421,124 496,769 302,237 328,905 723,361 825,674 SE 191,150 382,301 88,326 132,489 279,477 514,790 SI 73,000 91,250 18,423 20,726 91,423 111,976 SK — — 206,715 228,275 206,715 228,275 UK 549,841 984,149 397,265 578,361 947,105 1,562,510 EU 5,535,274 8,236,768 8,846,846 10,346,810 14,382,120 18,583,579 Source: SFC2014, based on data on the operational programmes reported by the Member States in AIR2018 (data extracted on 6 September 2019) Youth employment operations under both funds are programmed and implemented under the shared management system with the Member States. Member States identify relevant operations to support the objectives set in the operational programmes, in agreement with the European Commission. Therefore the range of measures supported by the European Social Fund and the Youth Employment Initiative varies significantly across Member States (also in view of the funding available to each Member State). There are 37 operational programmes running in 20 Member States including YEI (out of a total of 187 programmes under the European Social Fund). Only three Member States include YEI in more than one programme (Belgium and the UK with two each, and France with several regional programmes implementing YEI and a dedicated national operational programme for YEI). All Member States, including those not eligible for the YEI, can programme ESF resources to support youth employment operations. There are 58 operational programmes running in 20 Member States that include ESF investments under IP8.ii (access to youth employment) and that not covered by the YEI. Altogether only five countries (Austria, Denmark, Estonia, Finland and the Netherlands) did not programme ESF and YEI investment under the youth employment investment priority. This echoes the fact that these Member States did not receive a country-specific recommendation on youth employment under the European Semester process over the years, which means that these Member States did not present major policy challenges in this policy area. However, even though these Member States did not allocate specific budget to the youth employment investment priority, several operations are running in Austria, Denmark and Finland that are mainly focused on young people. During the negotiations of the 2014-2020 cycle of operational programmes, the Commission called for all action on youth employment to be programmed under the dedicated investment priority on youth employment (IP 8.ii), to ensure visibility and to concentrate action. Nevertheless, a map of all operations reported by the managing authorities under Thematic Objective 8 related to employment-related actions shows that ESF support for employment programmed under other investment priorities may also be accounted as youth employment. The costs of these operations are estimated at €2.1 billion, which can be extrapolated to an estimated €3.5 billion (assuming similar levels of implementation as the Youth Employment operations under IP8.ii: 66%). Based on these additional calculations, the total estimated funding for Youth Employment schemes under the European Social Fund and the Youth Employment Initiative is €22 billion, representing 18% of the total budget for the European Social Fund (including national co-financing). 3.2.Type of operations funded A total of 596 different ESF and YEI operations across all 28 Member States were identified under IP 8ii.

For ESF-funded operations, the budget was allocated as follows: ·24% (the largest share) to operations that support work-based learning, ·22% to guidance and support measures, ·13% to measures that support education and training, ·10% to financial incentives, ·13% to operations with multiple categories of operations that run under a single operation, with no dominant type. For example, a single operation could encompass a pathway of training and education combined with guidance and counselling for individuals. Given the specific scope and objectives of the Youth Employment Initiative, the share of different types of operations funded is different. This is confirmed by the high share of the budget (20%) allocated to multiple types of operations in a single approach. In particular, the higher share of financial incentives (26% compared to 10% for ESF-funded operations), such as wage subsidies, reflects the slightly different set-up of the Initiative. However, the provision of guidance and support accounts for a considerably smaller share in YEI operations (11% against 22% under the ESF). The share of funding allocated to work-based learning and education and training are relatively similar for both YEI and ESF-funded operations. Source: Mapping of operations eligible costs declared in AIR 2015 – 2018 3.3.Financial and operational implementation By the end of 2018, there had been about 3.8 million participations under ESF/YEI operations. Some 1.4 million participations operations resulted in employment immediately after participation, with further results including employment-focused education and training, and self-employment. To direct resources to youth employment measures at the start of the programming period, YEI investments were frontloaded in the first two years of the programming period, while ESF investments could be allocated over the full programming cycle. Some Member States planned to continue YEI schemes after 2018 from the start, with ESF funding. This initial frontloading of YEI funding explains the difference in implementation rates between YEI and ESF, with YEI financial execution overall more advanced in the Member States that receive YEI funding, in particular in Member States that receive a substantial share. The start of the actual implementation was delayed due to the time needed to set up control and monitoring systems that satisfy the Regulations and national legislation. The socio-economic context, absorption and administrative capacities of Managing Authorities and beneficiaries appear to be decisive factors for the progress in implementation which varies a lot between the different Member States.” Figure 7 Implementation rates of YEI/ESF funding by Member State (expenditure declared to the managing authority vs funding allocated) Source: SFC2014, based on AIR 2018, data extracted on 6 September 2019 MD=more developed region, LD=less developed region, TR=transition region. No regional data available for YEI funding.



4.Method 4.1.Short description of the methodology The evaluation questions, derived from the Better Regulation evaluation criteria, were broken down into a number of sub-questions.

The evaluation was based on three main sources of data: ·a study carried out by an external contractor; ·the twelve-week public consultation, which was carried out by and analysed with the assistance of the external contractor; ·an estimate of macroeconomic indirect effects of youth employment operations using the Dynamic Computable General Equilibrium model ‘Rhomolo’ developed by the European Commission’s Joint Research Centre. There were 1,376 responses to the public consultation. Over half (57.6%) were responses by citizens up to 33 years of age, almost a quarter of who had received funding under ESF or YEI operations. Some 21.8% of respondents were from organisations involved in managing and implementing of YEI and/or ESF-funded operations. Where data on the gender of respondents was provided, 58.7% of respondents were female. The study combined all information on the other two instruments in a single report, which provided the main materials and analysis for this evaluation. The study applied a mix of evaluation methods and tools, which are explained in further detail in Annex 3 to this Staff Working Document. In summary, the study involved: ·mapping and carrying out an in-depth analysis of youth-related thematic objectives, investment priorities, target populations and type of operations, based on the SFC2014 database, operational programmes, AIRs and related documentation, including the Commission’s country-specific recommendations. The study used data up to 31 December 2018, as reported by Member States by September 2019. For the mapping, the regions were categorised into four clusters in terms of their socioeconomic context, and used in the analysis and conclusions of this evaluation: oCluster A – regions with a strong initial socioeconomic context that made substantial progress over the period; oCluster B – regions with a strong initial socioeconomic context that made limited progress over the period; oCluster C – regions with a weak initial socioeconomic context that made visible progress; oCluster D – regions with a very weak initial socioeconomic context that made limited progress. ·reviewing available literature (policy documents, regulations, national and EU-wide evaluations, ESF websites and publications); ·a synthesis of the 2018 compulsory national YEI evaluation reports and screening of other ESF evaluations of youth employment programmes using the database of evaluations compiled by the Evaluation Helpdesk; ·the results of the public consultation; ·a comparative analysis of labour market trends and the role of EU-funded operations; ·a cost-effectiveness analysis; ·econometric analysis and provision of data to the Joint Research Centre (JRC) for the Rhomolo modelling simulations; ·case studies in 10 Member States, covering 20 operational programmes and a mix of YEI and ESF youth employment programmes; ·62 interviews with desk officers, managing authorities and other stakeholders, in particular for the case studies; ·five focus groups in the Member States and one at EU level;

·a set of fact sheets for all countries with YEI and/or ESF-funded youth employment programmes; ·a compilation of lessons learned and good practices. 4.2.Limitations and robustness of findings The findings and conclusions are reasonably robust, since they are based on a series of measurements from the multiple sources and tools used in the evaluation. However, the evaluation also noted several limitations. Data quality and timeliness. There is a significant time lag between reporting the outputs, results and financial implementation, as a result of work on data checking and reporting. The costs are declared only after checks by relevant authorities. Often, outputs are reported once operations are completed and declared, and the results reported later still. Lastly, since the data is based on participations in operations and not on individual participants, the calculation of certain ratios, such as success rate, is challenging. Detailed programme information. Detailed information on individual operations is usually not available, despite the obligation to record and store minimum data. Moreover, across the EU, the rules and standards for reporting operations are uneven. This means it is difficult to make a systematic comparison. To address this challenge, the evaluators focused on the cost of operations and made a number of assumptions such as estimated values by type of operation where no financial allocation was available, removing implausible values or the assumption that operations not covered by Article IP 8.ii reached similar implementation levels as those covered by Article IP 8.ii. Clearly defining youth employment operations. Regarding ESF support, a number of Member States decided to run youth employment schemes also across other investment priorities. As a result, the EU-level mapping exercise required clear definitions based on robust monitoring data, and estimates of the costs of operations not covered by the investment priority on youth employment (8.ii). The report makes it clear throughout what the underlying data is based on and where reliable comparisons can be drawn. For YEI support, the rules require that all YEI funding (including the matching ESF support) be programmed under IP 8.ii. Public consultation. The public consultation was in the form of an online survey run to gauge the views of a wide range of stakeholders, on a voluntary basis. This means that the information is not representative. Almost three quarters of the responses came from individuals and organisations in four Member States: Italy, Bulgaria, Spain and Slovakia, despite extensive efforts to publicise the consultation. This was highlighted in the report. Nonetheless, the public consultation provided a substantial amount of useful information and insights into the experiences and perceptions of youth employment operations. Comparative analysis of labour market trends. At the time of writing the report, data in the labour market policy database were only available up to 2017; excluding 2018. Regional clusters. YEI monitoring data are not broken down by type of region, so data on expenditure and participants may cover more than one type of region. Cost effectiveness. Due to issues of data timeliness and completeness, the lack of full information on the outputs and results made it difficult to draw overall conclusions on cost effectiveness. Thus the cost per result are not yet meaningful and the cost per participation (number of individual participants registered against the total eligible expenditure declared) are but a snapshot at the cut-off date of 31 December 2018. The approach taken was to combine partial findings from a number of sources (counterfactual impact evaluations available, micro-data from case studies in the Member States, econometric analyses, Rhomolo modelling and the public consultations) in order to draw valid, if only partial, conclusions. Econometric analysis. The explanatory power of the regressions – especially for unit costs – is low, due to the high degree of variability in the observations. This is the result of the complexity of youth employment schemes, mixed target groups and the fact that the data is still in the process of being consolidated (as mentioned above). Rhomolo analysis. General equilibrium models need to be based on a number of simplifying assumptions. Given the overall scale of EU youth employment operations in comparison with GDP figures and national programmes, the macro impacts identified by the model should be viewed with due perspective. Nonetheless, the work has been useful and informative. COVID-19 pandemic. Note that the field work for this staff working document was carried out before COVID-19 pandemic came to Europe. The Commission proposed the Recovery Plan for Europe on 27 May 2020, as part of the current and future multiannual financial frameworks. One of the aims of the Recovery Plan is to invest significant resources in measures to support youth employment and related reforms for the remainder of the current implementation period and over the next programming period, with a view to mitigating the consequences of this pandemic on young people and turning around the situation for youth employment for the better.



5.Analysis and answers to the evaluation questions 5.1.Effectiveness: How effective is the YEI, and other ESF-funded youth employment operations, in achieving their objectives? The section on effectiveness describes the extent to which YEI and ESF-funded operations have progressed towards the set objectives, broken down into operational (outputs), specific (results) and general (longer-term results/impacts) objectives. It covers both quantitative (success rates) and qualitative aspects (e.g. the quality of employment offers). This section also analyses the factors that influenced performance (socioeconomic context, implementation arrangements, type of action and outreach strategies).

Financial implementation and expected outputs are progressing, albeit slower than anticipated For the Youth Employment Initiative, progress on implementation has been below the targets set in the operational programmes, both in terms of financial implementation and number of participations. By the end of 2018, only France, Hungary, Lithuania, Latvia, Poland and Slovenia had met the 2018 milestone targets set at the start of the programming period. Despite action taken by the European Commission to facilitate the implementation of the Initiative in the early stages, Member States experienced delays in implementing the Initiative in 2014 and even in 2015, as they decided to first ensure compliance with ex ante conditionalities including set up of monitoring systems that complied with the Regulations and national legislation, before initiating the work. In addition, a common challenge across Youth Employment Initiative programmes was reaching young people not in employment, education or training. Since many of these young people were often not registered as unemployed, identifying the target group often required active canvassing and encouraging individuals to register. Member States and implementing organisations (beneficiaries of YEI support) often reported gathering the evidence on NEET status as possibly the most challenging legal requirement in relation to implementing the Initiative. Over time, most Member States implementing the Initiative broadened the eligibility of investments to include 25-29 year olds. This required refocusing the support to help young people overcome barriers to work and to fund outreach, rather than on developing their qualifications and skills. Faced with these considerable challenges to meet the initial targets for 2018, the 2017 revision of the multiannual financial framework provided resources for 2017-2020, which led to an extension of YEI funding until the end of the programming period. This extension was prompted by the need to ensure that operations already started and considered crucial by the Member States could continue. As a consequence, the operational programmes were amended and the implementation targets were revised. The EU average financial implementation rate reached 52% by the end of 2018. Source: SFC2014, based on AIR 2018, data extracted on 6 September 2019 (Cluster A – Strong start/substantial progress, Cluster B – Strong start/limited progress, Cluster C – Weak start/visible progress, Cluster D –Weak start/limited progress) According to the econometric analysis, the country specific context affects how the Initiative is implemented. In addition, the countries in most need of support often face the greatest implementation challenges. For example, countries with an unfavourable context at the start but where conditions are improving saw a positive correlation with progress in the project selection rate. By contrast, in general there was no correlation between the change in the youth unemployment rate and financial progress. The impact of socioeconomic factors on achieving the output targets, such as difficulty to reach target audience, lack of jobs or low education of participants, is more shaded. Other factors that hindered implementation were related to the administrative capacity of the managing authorities and the absorption capacity of beneficiaries. EU-funded youth employment operations have made a positive contribution to the integration of young people in the labour market Over 1.5 million completed participations in YEI were recorded in the monitoring system for unemployed young people under 25 years of age. This is slightly over half the total participations (2.7 million total participations). However, it is important to note that many participations continued after 2018 so they were not recorded in the study as completed. The 1.5 million represents on average 65% of the final target set for this population. Out of this total, 0.7 million unemployed people received an offer of training or employment immediately after completing the operation (with an average target achievement of 58%). Moreover, 0.9 million people unemployed at the start left the operation either in education or training, having gained a qualification or having found employment (60% of the target). This is encouraging progress, considering that YEI implementation is currently in its most active phase and will continue at least until the end of 2022, under cohesion policy rules. Therefore, it is likely that the overall YEI targets will be achieved. There were positive results for the long-term unemployed as result of participating in YEI:

The results for economically inactive participants have steadily improved, but the target for this group is relatively low (compared with implementation rates, for example, but also with the results for the long-term unemployed), as people are further from the labour market and less responsive to active labour market schemes than people who are unemployed. They are also often more difficult to reach out to and may require more comprehensive support, including motivation, counselling and social inclusion measures. Most participants who responded consider that their participation in EU youth employment operations has been beneficial The opinions expressed by participants during the public consultation were consistently positive regarding the effects of the EU support they received in helping them integrate into the labour market. The most frequent types of support the participants received were support to develop general skills (33.6% of respondents), help in looking for a job (33.4%), and gaining professional skills and qualifications (32.3%). Only 10.9% of respondents said that the support received did not help. The benefits of youth employment support go beyond access to employment The views expressed by the organisations responding to the public consultation echoed those expressed by individual participants. According to the organisations, the benefits of ESF/YEI funding are mostly linked to the development of skills and qualifications and to help young people find a job, including young people not in employment, education or training and other disadvantaged individuals.

Respondents from the organisations involved in running ESF/YEI also expressed broad agreement on the following positive results of the Initiative: ·promotion of new partnerships and exchanges; ·inclusion of marginalised people; ·development of soft skills and group work; ·youth empowerment; ·increased connection with labour market requirements; ·increased motivation and ·integration into society of young people not in employment, education or training. Overall, respondents from organisations are also more sceptical than individual respondents concerning the ESF/YEI’s effectiveness in helping young people improve their situation on the labour market in a sustainable manner. Identifying and reaching out to NEETs was often a challenge Despite the high levels of young people not in employment, education or training in a number of countries, actually identifying them to provide support proved difficult, in particular in Member States that had not yet set up policy channels and mechanisms to do so. Several Member States developed strategies to overcome this challenge. For example, they used social networks, media, newsletters or more innovative channels. In Bulgaria, concerts were organised and adverts were placed on fast food trays to reach young people. They also used street work to meet young people not in employment, education or training in public places where young people gather, e.g. in parks or shopping centres. Member States also invested in developing the experience and skills required to reach the ‘hardest to reach’ young people, working with professional youth workers who represent local organisations, promoting cooperation between the various bodies involved, especially ‘vertically’ (e.g. from local level to the public employment service), going beyond traditional outreach channels. One example is in Germany, where ESF funding successfully supported tailored activities (support chains and dedicated preventative systems of assisted transition from school to work through systematic multi-party cooperation). Resources were channelled to the population most in need The figures overleaf show that ESF/YEI funding reached a much higher proportion of young people not in employment, education or training in regions where the labour market situation of young people was worse at the start of the programming period than in regions with better initial favourable conditions, particularly when the situation continued to be unfavourable, e.g. when there has been limited progress. The higher coverage in regions where the labour market situation of young people was worse confirms that resources were channelled to the areas (and people) most in need. Source: SFC2014, based on AIR 2018, data extracted on 6 September 2019 and Eurostat, Labour Force Survey ( edat_lfse_22 ), data extracted on 21 June 2019. These findings concur with the Rhomolo modelling analysis, which found that EU youth employment funding went mostly to the regions most in need. The Rhomolo analysis also found that, in terms of indirect employment, the results are stronger for low-skilled participants, which was the main target group. At regional level too, in regions located in Southern European Member States where youth unemployment is high, EU funding has been the most beneficial for participants, considering also the high relative size of the support provided in those regions. Overall participation was well balanced from a gender perspective

Across the EU, participation is well balanced from a gender perspective, with a broadly equal share of men and women (51% against 49%) reached by YEI operations, although participation in ESF-funded operations is higher for men (54% against 46%). There are considerable differences among the Member States. YEI operations in Croatia and Greece reached more young women (66% for the YEI and 63% for youth employment operations under the ESF), while youth unemployment initiatives in Belgium reached more young men. The participation rate of young men in ESF-funded operations in Belgium, Germany, France and Sweden was over 60%. Cyprus, Croatia, Poland and Slovenia, by contrast, had above-average female participation rates. In some countries (Belgium and Spain), the national evaluations concluded that women benefited less from the operations under the ESF/YEI than men. Reaching out to specific groups of disadvantaged young people proved particularly challenging, requiring multiple methods According to the 2018 YEI evaluations, the most vulnerable young people are often under-represented, and potentially under-reported. This is in line with the replies submitted to the public consultation, which noted difficulties in reaching out to and supporting vulnerable young people in rural areas, those with disabilities, ex-offenders, those at risk of poverty and the homeless, and young people with multiple disadvantages. Source: Final Report, Public Consultation on the Evaluation of the EU Support to Youth Employment by the Youth Employment Initiative and the European Social Fund. Study supporting the 2019 evaluation of youth employment (VC/2018/0175) On the basis of the 2018 annual implementation reports, and bearing in mind that the categories below are not mutually exclusive, note that at least at EU(28) level: ·11.3% of YEI and 13.2% of the ESF participations reported were young migrants, participants with a foreign background or minorities (including marginalised communities such as Roma); ·4.5% of YEI and 5.2% of ESF participations were young people with disabilities; ·13.7% of YEI and 14.2% of ESF participations were people with other disadvantages; ·0.8% of YEI and 0.3% of ESF participations were homeless people or people affected by housing exclusion; ·19.3% of YEI and 18.7% of ESF participations were young people in rural areas. Under-reporting is quite likely for these indicators, since information on disadvantage is not always systematically collected by organisations or volunteered by participants. It is also important to note that not all EU countries have the same definition of the disadvantaged, or the same definition of the NEET category. In some regions particularly hit by the crisis, a substantial share of the young population could be considered disadvantaged at the beginning of the operation, while in other parts of Europe the definition covered only certain groups. Because of this, there are notable differences between the Member States. For example, Sweden recorded a significant share of migrants (38.7% of participations in YEI and 38.6% in ESF operations against an average of 11.3% of YEI and 13.2% of ESF) and a significant share of people with disabilities (69% of ESF compared to an average of 5.2%). The case studies illustrate the different methods used to reach out to disadvantaged groups. In Portugal, for example, the national operating programme did not define sub-groups of young people not in employment, education or training, but referred only to young unemployed and young inactive people. The indicators did not differentiate participants further. Nevertheless, some operations included an increase in the budget for vulnerable groups such as the long-term unemployed, ex-prisoners, people with disabilities, etc. The regional operating programme run in the Azores in Portugal reached all target groups of disadvantaged young people in all nine islands, except for young people with disabilities. Evidence from the case studies shows that some of the differences in target groups reached between the regions could be due to the type of organisations providing support rather than to the type of region. In Poland, for instance, there were differences in the targeting methods used by the local labour offices and the Voluntary Labour Corps. The local labour offices operated on a first come, first served basis with people registered with the public employment service. The Voluntary Labour Corps targeted NEETs including those not registered with the public employment service. Public employment services use additional eligibility criteria, including a lack of professional qualifications or qualifications not adapted to the needs of the labour market, disabilities, gender, particularly difficult social situations (e.g. people from jobless households, families using social assistance services, single parents and people from rural areas). Participants were mostly satisfied with the offers received but had contrasting views on the quality of the offer

One of the concerns of participants receiving YEI/ESF funding from the start was the quality and timeliness of the employment or training offers received. In practice, the criteria most used to assess the quality of job offers were whether the offer was for a permanent or full-time job, and the wage level. For most operational programmes for which data on the quality of job offers is available, over a third of the offers provided with EU support were for ‘permanent’ employment contracts (in the sense that they were for an indefinite period of time). Source: 2018 YEI Evaluation reports Data for FR relate to three national measures (‘Accompagnement APEC’, FR-GJ: ‘Garantie Jeunes’, FR-PA: ‘Parcours Autonomie’). Data for CY relate to ‘Acquisition of work experience for young unemployed graduates’. Data for ES refer to the situation 12 months after exit, and 6 months for UK-England. The results of the public consultation indicate that ESF/YEI participants who took part in the consultation and who started the operation unemployed were more likely to have transitioned to full-time employment (39.7%) than to part-time. However, among the long-term unemployed, only 19.5% found full-time employment after support, 7.3% found part-time work and half remained unemployed. When looking at responses from all types of participants, more respondents reported that they found a temporary job (20.9% of all ESF/YEI participants) than a permanent one (16.8%). Several YEI evaluation reports highlighted that for most participants, the wage level offered was an issue, as it was often at minimum wage level. This was a particular concern in (though not only in) Bulgaria, Croatia, Cyprus and Lithuania. In addition to the statistical data, the YEI evaluations carried out by the Member States revealed a high level of satisfaction with the support received by participants. In Croatia and Slovenia, for instance, the satisfaction rates are close to 70%, and in Italy, the participants surveyed are on average more satisfied with the support received through the YEI than participants in other employment support measures. There is, however, a general consensus among stakeholders in Italy that the quality of the offers should and could be further improved and that it should provide a form of certification of the skills acquired. The effectiveness of operations was influenced by the type of operations and, to a certain extent, socioeconomic factors and organisational arrangements The results of the public consultation indicate that paid apprenticeships are considered to be the most effective type of intervention by organisations involved in ESF/YEI, together with vocational education and training and basic skills training, which are considered useful for most target groups, including the most disadvantaged. This is generally supported by the evaluations carried out by Member States and the case studies providing specific examples. Community or voluntary work, as well as non-paid apprenticeships/traineeships, are considered the least effective types of operations in terms of access to employment, but not necessarily in terms of employability.

Source: Final Report, Public Consultation on the Evaluation of the EU Support to Youth Employment by the Youth Employment Initiative and the European Social Fund. Study supporting the 2019 evaluation of youth employment (VC/2018/0175) Socioeconomic and geographic factors clearly influenced the effectiveness of operations Although the econometric analysis indicated that the socioeconomic context does not seem to play a major role in achieving the targets for output indicators for the YEI, regions in less favourable economic situations recorded comparatively lower achievement rates for ESF operations. ESF operations in regions in a weak socioeconomic situation at the start and that made limited progress (Cluster D) had more difficulties in reaching the target number of individuals, whereas ESF investments in regions where the socioeconomic situation was improved made steadier progress towards their targets. The socioeconomic situation also had a greater bearing on the achievement of targets for result indicators than for outputs, independently of delayed reporting. A high share (47.4%) of organisations responding to the public consultation reported structural problems (such as the lack of jobs and the low education level of participants) as the factors hindering the achievement of ESF and YEI objectives, in addition to a lack of flexibility to adapt operations to changing socioeconomic contexts. This is understandable, given the difficult economic environment in which YEI and ESF youth employment operations were implemented in the aftermath of the late 2000s economic and financial crisis, in particular in southern Europe. The main geographical factor to emerge as having an influence on effectiveness of operations was the difference between urban and rural areas. This is based on information from the public consultation and the case studies, which provide some examples. For instance, in Spain actions taking place in large cities were considered more effective than those in rural areas, in particular as those run in rural areas typically tend to be more expensive due to the larger distances. They also suffer from having less infrastructure available and a smaller overall local job market. The administrative capacity, experience and organisation of managing authorities and beneficiaries also had an impact on effectiveness Structural challenges relate not only to the socioeconomic context, but also to the capacity of managing authorities and beneficiaries to deal with complex programming requirements, including eligibility criteria for young people not in employment, education or training under the YEI, and engaging with stakeholders. This last point was particularly relevant in regional programmes where the successful implementation of projects depends on cooperation between multiple governance levels and bodies. The 2018 national YEI evaluations revealed that prior experience and continuity are factors contributing to effective management. In the case studies, experience in managing similar measures was reported as helpful in implementing YEI. In Ireland, for instance, the long history of some operations (such as BTWEA, which has existed since 1993 or ‘Community Training Centres’) is considered to contribute to the success of YEI. Good governance was widely seen as essential to the success of the YEI evaluations. This is illustrated in Sweden, where the high quality implementation of the operations (the competence of staff, and low staff turnover) was regarded as a major success factor. Partnership and cooperation were also mentioned in several YEI evaluations and in the public consultation as crucial aspects to support implementation. For instance, in Brussels-Capital Region, cooperation between Actiris and other partners improved access to support for young people with a low level of qualifications or those living in jobless households. The managing authorities participating in the EU-level focus group also confirmed that cooperation among the several bodies involved, in particular among employers and employment services, is key to the efficient implementation of youth employment operations. This can only work well if the training offer is in line with companies’ need for skills or when there is a financial incentive for companies. The ability to reach the target population seems to have been the single biggest factor in effectiveness All in all, there seems to be a consensus among stakeholders that the factor with the single greatest impact on effectiveness was the ability to reach the target groups. Approximately six out of ten organisations responded to the public consultation that the main hindrance was the difficulty in reaching target groups, followed by structural problems such as the lack of jobs and the low level of education of participants. This was also confirmed in the EU-level focus group where the managing authorities agreed that reaching those furthest from the labour market was key to the success of the programmes. They identified three main essential aspects: local outreach work, local staff with the right qualifications and skills, and the use of innovative channels to reach young people. The composition of the target group also had an impact on the success rate of operations The composition of the target group also has an impact on effectiveness. The more ambitious the interventions were in terms of outreach to target groups requiring intensive and comprehensive support (for example people with disabilities or other disadvantaged people) the more difficult it was to achieve the financial targets and employment targets. This was also confirmed by the econometric analysis. For some disadvantaged participants, access to employment is not an immediate objective or even an option; instead the key outcomes were confidence building, gaining qualifications and some experience (not just employment).

Although not an explicit objective of YEI, youth employment operations led to structural changes in systems According to over half of the respondents from organisations (59%) to the public consultation, the employment operations run under the European Social Fund or the Youth Employment Initiative successfully contributed to the promotion of structural reforms in employment, education and training systems. A number of examples identified in the case studies confirmed this finding. The main impact of YEI and ESF funding for young people across operational programmes was in changes to the way public employment services and other bodies involved in employment policy work approach youth employment issues. Box 1 Cooperation and partnerships in Germany and Portugal In Saxony-Anhalt and North-Rhine-Westphalia, the two initiatives RÜMSA and KAoA, which bring together all bodies involved in youth employment in the regions, are bringing about a system change by coordinating support offered to young people. KAoA has the potential to spin-off into a broader municipal policy-making and coordination tool. In Portugal, implementation of the Youth Guarantee under the Youth Employment Initiative helped speed up and expand the public employment service’s work. The effect was to reach a more diverse target group of young people, extend geographic coverage and bring in new outreach channels (e.g. a specific position was created in job centres to assist companies and universities). The solid new network of partners working on the Youth Guarantee enabled the service to identify inactive young people (who were not yet registered at the public employment service). Macroeconomic effects are small, but positive, especially in the medium to long term The results from the modelling work carried out by the JRC’s Rhomolo analysis conclude that the ESF/YEI investments in human capital and education for young people produce macroeconomic returns on top of the direct positive results for participants, though they need some time to materialise. In the long term, youth employment operations are expected to provide a boost to overall EU GDP and employment levels, although at a modest rate. At EU level, the modelling found that, in addition to the jobs covered in direct relation to the funded operations, 11,000 jobs are expected to have been generated by 2023 thanks to the ESF and YEI investments in youth employment, with long-lasting effects generated by the structural policies and the change in productivity. The increase shows persistence over the long term (by 2030), indicating that GDP may be 0.06% higher than the baseline and over 35,000 jobs are expected to have been created. These positive but comparatively small effects should be read in the light of four key facts: ·the overall investment is modest compared to GDP (the whole Youth Employment package accounts for just 0.05% of the EU’s GDP) and uneven across regions; ·in addition to the overall macroeconomic effects, the operations provide direct benefits to those receiving support; ·these are just partial estimates given the programmes are still ongoing and reporting inevitably lags actual performance; ·EU cohesion policy supports investment in physical capital along with human capital, under the European Regional Development Fund. Such investments may have synergies with ESF/YEI investments in human capital. The key point here is that productivity-enhancing human capital investments ensure that jobs are created over the medium to long term. Although the effects may seem modest at EU level, they are greater in some regions, with a number of regions located in southern European states standing to reap most of the benefits, which can be quite substantial given the scale of EU support. 5.2.Efficiency: How efficient has the YEI, and other youth employment operations funded by the ESF, been in the achievement of their objectives? Three aspects of efficiency of EU youth employment operations are analysed below: ·First, the cost effectiveness of operations in delivering outputs and results, and the factors involved; ·Secondly, the impact of administrative arrangements on the effectiveness of support;

·Lastly, how visibility of the funding and the operations contributed to outreach and hence to successful delivery. Stakeholders viewed vocational education and training, as well as paid apprenticeships, traineeships and internships, as the most cost-effective type of operations The public consultation elicited broadly similar responses for the ESF and the YEI. Most respondents from organisations involved in implementing the ESF/YEI agreed on the cost effectiveness of vocational education and training activities (87.6% ESF, 83% YEI) followed by apprenticeships, traineeships and internships (83.3% ESF, 82.5% YEI), basic skills training (80.9% ESF, 79.1% YEI) and guidance and career support (80.4% ESF, 76.7%). In both cases, and in line with the public consultation responses on effectiveness, voluntary and community work were ranked the least cost-effective. Unit costs per output are comparable with the YEI and other ESF youth employment operations, with significant variations between Member States Mindful of the above limitations in terms of data availability, this evaluation resulted in an average cost per participation in ESF 8.ii (excluding YEI) investments of €1,854, which is higher than the average unit cost for Thematic Objective 8 (Promoting sustainable and quality employment and supporting local mobility) overall (at €1,390). These figures are also higher than the unit costs for youth operations given in the update of the ex-post evaluation of ESF 2007-2013 for Access to Employment (€1,215). Investments in YEI have similar average unit costs at €2,035 (EU average). The econometric analysis confirmed that there was no statistically significant difference between the unit costs for YEI and ESF youth employment operations outside YEI. A Commission study also concluded that ‘considering the rigorous data verification process that the data in the sample has undergone, […] unit costs for the unemployed established at EU level (Simplified Cost Options that Member States can use to reduce the administrative burden of providing evidence for eligibility of expenditure) reflect the specific training practices in those Member States’. The calculations of both costs per participation and simplified cost operations vary significantly between the Member States. There are some reasons for the substantially higher than average values for Germany, Ireland and Sweden. For Germany, the likely explanation is that typical operations in Germany last longer (12 months and over). In Ireland, operations target specific groups, such as the socially excluded, long-term unemployed, and youth, which increases the cost of the intervention. For Sweden, factors are both the type and the duration of operations. Note that the experience in running YEI and ESF youth employment support operations has become a solid basis for the Commission and Member States to work on developing simplified cost operations for the integration of young people not in employment or education. Developing EU-level simplified cost operations for this target group and such measures could be particularly relevant for transnational cooperation activities involving several Member States, as evidenced in the current period by the ESF Transnational cooperation network on youth mobility. Table 2 Comparative table of unit costs MS ESF monitoring data YEI monitoring data YEI evaluations 2018 LMP database 2017 LMP database 2017 SCO study based on LMP database SCO study based on extrapolated data from LMP database Overall unit cost 8.ii Overall unit costs TO8 Overall unit costs Overall unit costs Overall unit costs national Overall unit costs ESF co-funded Overall unit costs active population Overall unit costs active population (EUR) (EUR) (EUR) (EUR) (EUR) (EUR) (EUR) (EUR) BE 310 698 278 5,700 800 3,351 BG 17,711 16,035 1,648 1,278 2,500 2,500 596 CY 3,727 3,664 3,088 2,696 CZ - 2,831 5,084 700 4,800 521 DE 4,078 3,009 - 5,900 6,000 6,959 ES 805 626 1,951 2,772 FR 3,748 1,498 1,359 6,274 EL - 2,748 2,588 14,500 5,700 2,064 HR 4,925 4,136 4,754 4,300 5,400 4,299 689 HU 1,940 2,188 3,005 3,000 4,000 5,500 1,818 IE - 7,717 9,738 16,800 17,800 11,119 IT 1,017 691 2,578 3,900 3,676 LT - 1,678 1,023 3,600 5,800 1,359 LU 1,184 1,402 - 16,000 45,700 19,302 LV - 955 2,012 700 4,200 756 MT 680 1,551 - 8,700 5,600 2,256 PL 1,958 1,885 2,014 3,500 3,300 594 PT - 456 5,609 1602 2,500 4,800 994 RO 1,557 2,154 715 1,400 53 583 SE 5,564 6,759 3,524 15,100 7,303 SI 3,235 2,806 6,449 224 6,300 4,200 854 SK - 1,847 1,023 3,680 2,200 424 UK 2,265 1,706 2,257 5,863 EU 1,854 1,390 2,035 Sources: SFC2014, based on AIR 2018, data extracted on 6 September 2019. The figures for BG are subject to the same caveats as in the previous table, YEI 2018 evaluations. LMP database Data compiled by PPMI based on data extracted from the LMP database and provided by Member States for training of all unemployed The intensity of support is likely to be a major factor in determining the unit cost and the cost effectiveness of operations The focus on outreach to vulnerable and hard-to-reach groups means that effective operations require a significant investment in identifying, recruiting and supporting – often with a high level of intensity – those from vulnerable and hard-to-reach groups, which implies additional costs. This may further increase if inactive NEETs account for an increasing share of NEETs supported. This can also be related to the fact that training activities are also considered the most costly type of operations ( Table 3 ). In this context, it should be underscored that high costs are not in themselves indicators of low efficiency. Cost effectiveness is also positive when costly operations are linked to high effectiveness for specific target groups. Table 3 Expenditure per entry: expenditure minimum-rate threshold of 75.1% - breakdown by region and type of intervention Overall YEI more developed regions in transition regions less developed regions Work-based learning or first job experience 4,383 4,006 5,906 1,000 6,340 Education and training 7,869 1,027 12,295 Data gaps Data gaps Prevention strategies for early school leaving 964 964 Data gaps Data gaps Data gaps Guidance and support for individuals 469 457 904 537 Structural support for strengthening institutional capacity Data gaps Data gaps Data gaps Data gaps Data gaps Financial incentives to employers and unemployed 2,344 3,088 2,003 2,169 2,585 Source: Micro-data and own calculations, based on operations completed to 75% or more

The influence of the socioeconomic context on the cost per participant is not significant for outputs, but it does have some impact on results According to the econometric analysis carried out, the socioeconomic context tends does not make a statistically significant difference to the average cost per participant by country. Additional findings from the econometric analysis are that: ·costs tend to be higher in programmes run in the cluster of countries with a strong starting position and that made good progress (Luxembourg and Ireland, for example). Costs are comparatively lower in areas with greater need of support; ·changes in the unemployment rates are not correlated with cost per participant. The costs of measures are not directly related to the characteristics of the target population According to the econometric analysis, overall, the composition of the target population (different shares of inactive, low-skilled, disadvantaged participants, and shares of participants from minorities and with disabilities) is not strongly correlated with unit costs. The exception is for programmes that target a high share of inactive participants, which show statistically significantly lower unit costs. This finding would appear to contradict other sources that highlight more intense levels of provision for those young people from more disadvantaged backgrounds. For instance, most case studies highlight higher unit costs for hard-to-reach/disadvantaged groups (especially the long-term unemployed) for comparable forms of support. Administrative requirements have facilitated implementation, with some exceptions Overall, respondents had a positive perception of the administrative requirements and organisational arrangements to implement youth operations. Most respondents to the public consultation (300 responses from organisations) found the administrative arrangements for the implementation of projects to be appropriate (see Figure 13 ). The types of administrative requirements more often rated as appropriate are those related to project implementation (65.3%), project selection (64%), communication (62.7%), evaluation (61%), reporting and monitoring (55.7%), setting up standard cost options (51.3%) and audits (50.3%). A lower score was given to the arrangements for setting up the management and control systems, with 46.3% rating it as appropriate.

Figure 13 How would you qualify the following administrative arrangements for the implementation of youth employment operations under the ESF and YEI? Source: Draft Final Report, Public Consultation on the Evaluation of the EU Support to Youth Employment by the Youth Employment Initiative and the European Social Fund However, the case studies identified a number of areas for improvement, notably as regards the requirements to provide evidence to support the eligibility of costs regarding the support provided to young people not in employment, education or training and the problems related to the initial set-up and functioning of the IT systems for the operational programmes. This was found to be a general weakness in all cohesion policy programmes. Respondents to the public consultation were asked in an open question to provide any examples of gold plating or excessive administrative burden that they had experienced. However, the examples given do not seem to be deliberate on the part of public administrations or ESF/YEI operations, but more due to inefficiencies resulting from either work to implement EU regulations, or from incompatibility in national systems. Essentially, both the public consultation and the case studies noted improvements since the previous programming period and support for the EU’s simplification agenda, which will take time to deliver all of the anticipated benefits. Reporting and monitoring requirements are complied with, but delays are common The respondents had a generally positive perception on this aspect, after the initial transition period. Beneficiaries comply with the Commission’s reporting and monitoring requirements and systematically use the system for fund management for reporting purposes, in line with the guidance provided and the regulation. Compared to the previous period, the current ESF Regulation 2014-2020 sets out common output and result indicators (including those setting a common approach to monitoring/reporting on the sustainability of results through systematic measurement of longer-term results for participants) and the Commission has issued detailed definitions, guidance and support for setting up monitoring systems. Member States and the Commission use this framework to report progress, notably on the support provided to young and disadvantaged people.

The main issues are under-reporting, difficulties in measuring soft outcomes and in capturing the long-term benefits of YEI/ESF youth employment operations after the projects end. In addition, there is a significant time lag between reporting outputs, results and financial implementation, as a result of data checking and reporting requirements. Costs are declared only after the authorities carry out checks. Outputs are reported often only once operations are completed and declared, with results reported later still. The reporting of indicators focusing on employment and on the formal qualifications gained typically underestimates the total benefits to society resulting from active labour market operations, for example the benefits to health and levels of crime. The managing authorities recognise the value of soft outcomes, especially for disadvantaged groups with multiple problems and they are said to be highly valued by participants in the context of youth employment operations. The value is generally in the social skills built up (self-esteem/confidence, cultural and community interaction and interaction) and in the non-vocational skills sets that employers look for (including reliability, honesty/discipline, time keeping and personal organisation). However, Member States seldom monitor soft outcomes. This may be due to difficulties in devising a consistent measurement method and in verifying recorded data. Moreover, soft outcomes are specific to each operation, which makes them unsuited for tracking by common indicators. Managing authorities are aware of the value of capturing results after a period of time (six, 12 or more months after participation) and the value of tools such as ad hoc surveys, but practice varies. From an evaluation perspective, and for the purpose of making a pan-EU comparative analysis, managing authorities should be able to use national data sets for econometric analysis or to track individuals (e.g. through tax and national insurances numbers or surveys, especially for tracking results after a participant has left an operation). However, this raises data privacy issues, in particular since the entry into force of the GDPR. Visibility is an issue The visibility of EU funding is not only a legal requirement but also an important aspect of youth employment operations, since increasing awareness contributes to reaching the target population. Therefore, good visibility is key to boosting the efficiency of operations. Visibility covers both general awareness-raising and specific communication on activities. On both counts, there is room for improvement for the ESF and even more for the YEI. The public consultation, for instance, revealed that some respondents who had received EU funding for youth employment operations were not aware of the ESF or the YEI. Communication activities are essential in raising awareness. More work seems warranted to find effective channels to reach the target population. This appears clearly in the difference between the demand shown in responses to the public consultation for ‘new’ forms of communication and ‘old-style’ forms. Most respondents viewed social media campaigns to be the best communication channel (76.9%), followed at a distance by youth networks, clubs and online groups (42.7% overall) and word of mouth (36.5% overall). However, most participants had found out about the support measures they had received via employment services and employment info centres (52.9%), and much fewer by word of mouth, family or friends. Only 26.2% had found out about these measures via social media. 5.3.Relevance: How relevant is the YEI and the other ESF-funded youth employment operations? The following section refers to the extent to which the YEI and ESF-funded youth employment operations address the needs of young people, in particular young people not in employment, education or training. Given the changing socioeconomic context, especially the fall in the youth unemployment rate, this section also provides an analysis of how the implementation solutions have adapted to the changing needs. Youth employment operations funded by the ESF and the YEI addressed the most relevant needs and groups In line with the purposes of the Youth Guarantee, the YEI was created to provide support for young people not in employment, education and training to help them access the labour market. These needs were not only formulated in terms of providing jobs, but also in terms of developing their professional and personal skills to help young people make the transition to employment. The country-specific recommendations issued by the Commission to the Member States between 2014 and 2018, though not exhaustive, provide a good overview of the main needs at the beginning of the programming period: ·Improving basic training and skills; ·Reaching out to inactive young people and helping them find the way back to employment; ·Providing counselling and mentoring; ·Facilitating the transition from school to work; ·Offering quality traineeships and apprenticeships as a first step towards employment;

·Providing incentives to employers for hiring young people. The bulk of expenditure for ESF/YEI-funded operations is channelled to work-based learning, education and training in professional skills, guidance and support for individuals and combined operations as pathways to employment. For the YEI, a substantial share (26%) has been allocated to financial incentives to employers. The range of operations funded confirm that programmes were aligned with the identified needs of young people. Box 2 Special support for hard-to-reach young people in Belgium The Wallonia-Brussels AIR for 2018 in Belgium identified the difficulties experienced by young people not in employment, education or training and participating in YEI operations. They include their low skills and competence levels, lack of motivation, instability, problems linked to multiple disadvantages (e.g. access to housing, health, mental health problems and family problems). These issues require special support, such as individual and constant guidance, adaptation and close collaboration with specialist institutions (e.g. medical centres, mental health centres, diagnoses of learning difficulties, emergency accommodation) to prevent the young people from dropping out of operations. There is often a problem with attendance (absences and late arrivals). The Wallonian ‘Sac a dos’ project finances several activities carried out by the beneficiary. It includes individual guidance and counselling, workshops to help participants improve their self-confidence and self-esteem and developing personal skills (such as autonomy, living in a group, expressing oneself) and short projects (of about four days) where young people can give back to the community and discover or improve specific techniques in a range of sectors (such as in construction, or in the ‘green sector’). It also funds short training sessions (minimum two weeks) in several businesses (such as in a flower shop or a fitness centre) to give the young people work experience. In 2018, the project provided assistance to 50 young people, with the number rising each year (especially among homeless young people). There are no official statistics tracking the situation of young people once they have left the project, but it would appear that most go back to school or enrol in a vocational training course. Stakeholders and participants rated the youth employment operations as relevant and useful During the public consultation, participants reported that training to build general skills, support in overcoming barriers to work/training, and training to obtain qualifications were the most useful types of operations. Most organisations (over 90%) that responded to the public consultation consider the following actions as relevant or very relevant to help young people find quality and sustainable jobs: vocational-focused education and training, guidance and career support, paid apprenticeships/traineeships/internships and basic skills training. By contrast, they ranked support to set up a business (71%), find a job or opportunity abroad (70.3%) and support to get back to school (64.8%) as relatively less useful. The contributing organisations considered community or voluntary work and non-remunerated apprenticeships to be the least relevant (47.5%). Gender issues are taken into consideration initially, but there is a lack of active monitoring and targeting In general, programme documents contain detailed strategies on how attention to gender equality is mainstreamed. The monitoring systems meet the obligation to disaggregate data collected by gender but often did not define gender-specific indicators that extend beyond levels of participation and provide results from a gender perspective. The same applies to the targets. Although all Member States report outputs and results by gender, only 41% of the output indicators focusing on youth employment with targets are gender-specific targets and measure whether implementation meets with the programme’s ambitions for gender equality. In terms of results, this is slightly lower as only 28% of all result indicators are gender-specific targets, corresponding to fewer than half of the operating programmes. Only a small number of operating programmes (in Austria, Spain and Italy) included dedicated youth employment investments in gender equality (IP 8.iv), accounting for a total amount of €18 million. Other ESF/YEI operations (IP 8.ii) generally tend to focus on gender equality as a cross-cutting issue. The operations did not always reach the most vulnerable groups The strict eligibility criteria for young people not in employment, education or training mean that some vulnerable (young) people groups were not directly targeted by YEI operations, and sometimes not reached by ESF operations either. In focus group discussions, participants noted that YEI currently does not allow youth employment operations to focus on prevention work targeting groups at risk, for example potential early school leavers. Some countries were in favour of extending the definition of the target population (beyond those still at school) such as Italy where, due to severe unemployment in less developed regions, there is a need to extend the support to people up to the age of 34 years. In addition, some countries underlined the complexity of the NEET concept, as it requires verifying negative conditions. Programmes sought to address the most relevant and vulnerable groups by taking steps such as: ·defining specific output targets (e.g. Spain, Luxembourg, Romania); ·creating mechanisms to identify the most disadvantaged groups (e.g. Malta);

·putting a specific focus on disadvantaged young people (e.g. Germany, Spain and Greece). There are various ways in which operational programmes under the ESF address the most relevant target groups and tackle disadvantage. There are no strict eligibility requirements under the ESF for participation, so funding can be allocated to any such target groups, either under the dedicated investment priority for youth employment, or under broader types of objectives and investment priorities. Support for youth employment focused on regions most in need of support The budget allocation to different clusters of regions, as presented in Figure 14 overleaf, clearly shows how YEI focuses predominantly on regions where youth unemployment is most problematic (72% of the budget was allocated to regions in a very weak situation at the start of the process and that made limited economic progress). The Rhomolo modelling study showed that some regions (particularly in southern Europe) that depend greatly on youth employment reap much of the benefits of YEI. It also highlighted that the allocated YEI amounts are proportionately higher in regions that have much higher youth unemployment rates, NEET rates or social exclusion and at-risk-of-poverty rates. ESF funds are spread across different clusters of regions, but the fact that almost no youth employment operations are allocated to regions that had favourable youth employment figures in 2014 and that had improved further over recent years shows how ESF funding has been targeted to regions where youth employment is most problematic. When combining ESF and YEI funding, the regions with the highest youth unemployment figures also received over half of all the youth employment budget (56%). Regions that also had high youth unemployment at the start of the programming period but that improved over time were the second largest beneficiary of funding (28%). Figure 14 Overview of budget allocation to clusters of regions Source: SFC2014, based on AIR 2018, data extracted on 6 September 2019 (Cluster A – Strong start/substantial progress, Cluster B – Strong start/limited progress, Cluster C – Weak start/visible progress, Cluster D – Very weak start/limited progress). Operational programmes were flexible and able to adapt to changes Operational programmes were able to adapt to changes stemming from the business cycle such as the fall in youth unemployment and the rising share of inactive young people in the target group, but also to unexpected developments, such as the immigration crisis. Most Member States set specific objectives and target groups in a relatively broad way. As a result, minor annual adjustments to the programming did not have to go through formal amendment procedures. The additional budgets made available to YEI did, however, require formal amendments to the programmes. They are also evidence of how the procedures were flexible enough to provide for additional budget to be programmed where necessary and facilitated the adjustment of formal targets set in programmes. The budgetary changes made were mainly to the volume of operations supported, without leading to significant changes in the programme strategy. Most adaptations to the programme were to accentuate the focus of regional targeting, to broaden target groups and the type of actions. Only in a limited number of programmes was the underlying programme strategy revised (e.g. North Rhine-Westphalia, Saxony-Anhalt or the operational programme for knowledge education development to adapt to the changing socioeconomic context). Flexibility was thus a key factor in the adaptation of programmes and cooperation between programme stakeholders was an essential success factor in this regard. 5.4.Coherence: How coherent are YEI and the other ESF-funded youth employment operations among themselves, and with other actions in the same field?

The ESF Regulation (Article 4) requires that the strategy and actions set out in the operational programmes are consistent with, and respond to: ·the challenges identified in the national reform programmes, ·other national strategies (where relevant) that aim to combat unemployment, poverty and social exclusion, and ·the relevant Council recommendations, in order to contribute to achieving the headline targets of the Europe 2020 strategy on employment, education and poverty reduction. This section looks at coherence between YEI/ESF-funded youth employment operations among themselves and with the national policies. The design and programming of the ESF and the YEI contributed to their mutual complementarity On the one hand, YEI was designed to support similar operations to the ESF (such as employment and training, apprenticeship, hiring incentives, self-employment programmes), though it focuses exclusively on young people not in employment, education or training in eligible regions. YEI is therefore technically complementary to the ESF, available to the regions most affected by youth unemployment and providing support for particularly disadvantaged young people. On the other hand, ESF also offered system support or was often budgeted as a follow-up measure to operations run under the YEI. YEI and ESF youth employment operations are complementary and coherent with other EU schemes The issue of complementarity and coherence with other EU funds in the same areas was barely covered in the national YEI evaluations. The public consultation showed that most organisations involved in running youth employment operations considered that the YEI and ESF were coherent with each other and with other EU schemes in similar or related areas.

Source: Final Report, Public Consultation on the Evaluation of the EU Support to Youth Employment by the Youth Employment Initiative and the European Social Fund This point was confirmed by the case studies and the analysis of the policy framework for the ESF and other EU funding schemes. The YEI and youth employment measures under the ESF are specifically focused on access to the labour market for young people and helping them become active on the labour market. None of the other EU schemes replicate or unnecessarily duplicate YEI and youth employment measures under the ESF. Some funds (ERDF, EAFRD, EMFF) are geared towards creating sustainable jobs open to young people. The EAFRD and EMFF target young people by providing support for farming and fisheries ventures, including helping them to develop skills for these sectors. The EURES and Erasmus+ programmes have a strong complementarity with YEI and ESF youth employment measures. EURES facilitates information sharing on job opportunities while Erasmus+ facilitates mobility and learning for young people. Institutional coordination is essential element to ensure coherence Coordination is key to ensuring complementarity with other EU funds or programmes. Conversely, poor coordination can be an impediment to success. Although it did not find evidence of extensive overlaps, a study commissioned by DG REGIO found evidence of poor coordination only in a limited number of cases. There is overall complementarity between EU-funded youth employment operations and national schemes, but some cases of overlapping and replacement were reported

According to the results of the public consultation, most respondents (43%) considered that YEI/ESF youth employment programmes are complementary to national/regional programmes. This was confirmed by the case studies run in Italy, France and Germany. However, the analysis of YEI evaluations showed that in a few Member States, the YEI programmes sometimes replaced measures that used to be financed by the national budget, raising a question as to what extent EU resources have been used to complement national action and budgets. In general, there are clear lines of demarcation. In some countries and regions, this is facilitated by coordinating partnerships that take a holistic approach to youth employment operations across EU and national programmes. Several EU programmes have a focus on young people, but none have the clear employment focus or the breadth of the YEI or ESF youth employment operations. 5.5.EU added value: What is the EU added value of the YEI and other ESF-funded youth employment operations? To analyse EU added value, the evaluation used a wider framework extending beyond what could be achieved by Member States at national and/or regional levels. This framework helps assess the extent to which the YEI/ESF-funded youth employment operations was additional to national funding and programmes, and the ways in which the support helped reform and improve national systems. Without YEI and ESF funding, some operations would not have been implemented or would have been more limited At a time of economic crisis, EU youth employment operations enabled the running of operations that would have been difficult to achieve otherwise. According to the replies from the public consultation, the main effects of YEI were to strengthen approaches by providing extra funds, and by providing individualised support to participants. According to the information available to the Commission on national labour market policies, in several Member States, some of the labour market measures specifically designed to help disadvantaged young people (i.e. unemployed, employed but at risk and inactive young people) were fully funded by the ESF/YEI, i.e. there are no equivalent measures funded only through national/regional resources. This reflects the significant volume effect of YEI and ESF-funded operations. For instance, all apprenticeships were co-funded by ESF/YEI in Ireland, Spain, Portugal and Finland. This was also the case for traineeships in Bulgaria, Greece, Latvia, Malta, Portugal and Slovenia, for institutional training in Estonia, Ireland, Latvia, Malta, Portugal and Slovenia, and for employment incentives in Estonia, Cyprus, Lithuania, Spain and Slovakia. The case studies provided further examples where YEI and other ESF-funded operations produced effects at national and regional level that would not have been achieved without EU support. Box 3 Common standards and tools developed in YEI programmes A particularly striking example is the national YEI evaluation report for Bulgaria. If it were not for YEI funding, the level of youth unemployment and inactivity in the country would have been significantly higher: nearly 4,000 young people would be out of the labour market, approximately 9,000 people would have not received further vocational training, 19,000 would have not increased their qualification through internships, and some 26,500 would have been out of employment. YEI and ESF-funded operations covered target groups that were not covered by other national or regional interventions National evaluations, case studies and the public consultation provide evidence of the fact that the YEI enabled an expansion and refining of the scope of public employment operations. The YEI and the ESF brought to the forefront the needs of the target group (the NEETS in high youth unemployment regions) which would not have been a specific focus under other funding schemes. Box 4 ‘From training to employment’ – the Voluntary Labour Corps project in Poland The Voluntary Labour Corps runs daily activities for the benefit of young people. As part of the project, it was possible to test the entire support path from educational activities to employment, enriched with new activities to increase the effectiveness of support and ensure that the employment goal is met. Comprehensive support provided as part of the project (broader than the support provided in standard actions), improved the work with young people. The experience acquired fed into the activities carried out and, as far as possible, will lead to an expansion in the scope of standard forms of support. It was important to reach young people in the most difficult situations, who cannot access standard activities, and to work with them every day. To achieve this, each local coordinator looked after approximately 10 people, which required 24/7 engagement. They were in constant telephone contact with them, often taking them to classes, making sure that they did not give up the project. This is one of the most difficult aspects of this job, because it is associated with upbringing and teaching social behaviour, with the aim of helping young people to function independently. Equally important was to strengthen organisational structures at local and national level. The Voluntary Labour Corps has since gained recognition among young people as an institution providing information on the support available to them. The project led to the development of intensive, and sometimes tailor-made, approaches to target groups (vulnerable young people in different contexts), who might not otherwise have received such specific services. It also raised the level of ambition of support objectives. Though rather limited information is available on the possible effects of the operations, in some countries it is recognised that the YEI/ESF helped to raise awareness about young people not in employment, education or training to pay specific attention to helping this target group. The positive trends in NEET rates up until early 2020 are likely to be reversed as a result of the ongoing COVID-19 pandemic. Support to young people not in employment, education or training will continue to be at the core of the anti-crisis measures currently running in the Member States with funding from the current programmes, notably the ESF, as well as the future ESF+. As those instruments invest in both people and systems, they provide comprehensive support to respond to the challenges. Outreach to and specific measures for this target group have been identified as a key policy areas where Member States will step up their action. They will continue to learn from each other in the context of the Council Recommendation on a Bridge to Jobs – reinforcing the Youth Guarantee, which the Commission proposed on 1 July 2020.

Box 5 Common standards and tools developed in the YEI programme In Italy, the national YEI operational programme was highly innovative, in terms of the new method of collaboration brought in by the central and regional authorities, and the new common standards and tools that have now become a legacy of the programme. Most of the tools developed were mainstreamed in the 2015 reform of employment services and active labour market policies. The national YEI operational programme also spea