Glossary

Benchmarking Quantitative and qualitative comparison of the performance of an intervention, with that which is reputed to be the best in the same domain of intervention or in a related domain.

Effectiveness Measure of the relationship between the effects obtained and the initial objectives set.

European Social Fund (ESF) One of the Structural Funds which supports measures to prevent and combat unemployment and develop human resources and social integration into the labour market in order to promote a high level of employment, equality between men and women, sustainable development, and economic and social cohesion.

Evaluations Ex-ante evaluation A key element in the preparation of the programmes. It is the responsibility of authorities designated by the Member States. It consists mainly in an evaluation of the strengths, weaknesses and potential of the Member State, region or sector concerned and an assessment of the consistency of the strategy and targets selected with the specific features of the regions or areas concerned, including demographic trends, and the expected impact of the planned priorities for action.

Mid-term evaluation Evaluation carried out in the middle of the programme period examining the initial results of the assistance, their relevance and the extent to which the targets have been attained. It is carried out under the responsibility of the managing authority of the programme concerned, in cooperation with the Commission.

Ex-post evaluation Evaluation carried out by independent experts, not later than three years following the end of the programme period, under the responsibility of the Commission, in collaboration with the Member States and managing authorities. It shall cover utilisation of resources, the effectiveness and efficiency of the assistance and its impact. It shall draw conclusions regarding policy on economic and social cohesion, having covered the factors contributing to the success or failure of implementation and the achievement of results.

Gender equality The concept that all human beings are free to develop their personal ability and make choices without limitations set by strict gender roles; that the different needs of women and men are considered, valued and favoured equally.

Indicators Output indicator Tool for measuring physical outputs (for example: number of training participants) that demonstrate the progress made in implementing the actions financed.

Results’ indicator Tool for measuring the immediate effects on the direct beneficiaries of the actions financed (for example: number of participants who obtained the required qualifications).

Impact indicator Tool for measuring longer-term socioeconomic consequences of the actions financed going beyond the immediate effects on the direct beneficiaries (for example: number of people employed 12 months after the training).

Managing authority Authority designated by the Member State to manage an intervention (operational programme/single programme document).

Operational programme (OP) and Single Programming Document (SPD), called "programmes" in the report A Community support framework — which contains the strategy and priorities for action of the Structural Funds and the Member State, their specific objectives, the contribution of the funds and the other financial resources — is implemented through various operational programmes. An OP is a document prepared at central or regional level in a Member State and approved by the Commission, which takes the form of a coherent set of priorities comprising multiannual measures. The projects to be co-financed by the Structural Funds must fall within the scope of a measure. Under certain circumstances, a Community support framework and an operational programme can be merged to take the form of what is known as "a single programming document".

Programme period The multiannual framework within which Structural Funds’ expenditure is planned and implemented.

Project promoter An individual person or an organisation (e.g. local authorities, associations, enterprises, etc.) which can propose projects and apply to receive support from the Structural Funds.

Segregation (vertical and horizontal) in the labour market Differences between men’s and women’s participation in different sectors (horizontal) and in decision- making positions (vertical).

Structural Funds The Union’s principal policy instrument in support of the Treaty objectives of economic and social cohesion.

Vocational training Any form of education which prepares for a qualification for a particular profession, trade or employment or provides necessary skills for such a profession, trade or employment.

Executive summary

I. One of the objectives of the Structural Funds is to promote equal opportunities for men and women on the employment market. The main Community instrument for the promotion of such equal opportunity is the European Social Fund (ESF). This report deals with vocational training actions for women co-financed by the ESF during the 2000–06 programme period. An estimated 3 billion euro was earmarked for vocational training actions during this period.

II. The Court’s audit focused on the following two questions:

(a) were vocational training actions for women selected according to a clear analysis of labour market priorities?

(b) was monitoring information sufficient to evaluate whether such actions had achieved their objectives?

III. The audit was carried out at the Commission and in five Member States (Germany, Spain, France, Italy and the United Kingdom) in total accounting for 76 % of expenditure in this area. The audit conclusions presented in this report are based on an examination of the establishment of the programmes and of a sample of finalised projects.

IV. Overall, for the identification and selection of vocational training actions for women, it was found that the training actions contained within the programmes audited were not set up as a direct response to the analyses of labour market requirements and the measures established tended to leave a broad scope for actions to be implemented, failing to focus sufficiently on specific target groups.

V. Furthermore, project selection criteria in the Member States audited did not take sufficient account of these analyses of the labour market nor ensure that projects selected had objectives which were in line with overall programme objectives.

VI. The Court recommended that future operational programmes should be established in light of an analysis of the labour market and that the type of training and objectives decided upon should be a direct response to this analysis. Furthermore, the Court recommended that an effective project selection procedure be put in place to approve projects which respond best to the problems highlighted in the analysis.

VII. As regards the evaluation of the achievement of objectives, the Court found that monitoring information did not allow such an evaluation because the indicators established were too narrowly focused, targets were often not provided and the information itself was unreliable or incomplete. As a result the Court is of the opinion that neither the implementing authorities in the Member States nor the Commission were in a position to assess to what extent the objectives were attained.

VIII. The Court noted that there is a need for the Commission to continue to monitor the establishment by the Member States of appropriate and feasible indicators and that reliable data be collected, thereby allowing meaningful conclusions to be drawn on the efficiency and effectiveness of the co-financed actions.

Introduction

Background

1. Article 2 of the Treaty establishing the European Community lays down the promotion of equality between men and women as one of its fundamental principles. Council Regulation (EEC) No 2081/93 [1]declared the principle of equal opportunities for men and women on the employment market to be a Community goal to which structural measures should contribute. Equality between genders is the possibility for women and men alike to participate and be represented equally in the economy, in decision-making, and in social, cultural and civil life [2]. The Commission reports annually on the situation of equality between women and men, disclosing observed trends in areas where the main gender gaps exist, e.g. rates of employment and pay, sectoral and occupational segregation and the ratio of female representation in management [3]. Figure 1 illustrates the change over time of the gap between male and female employment rates. Whilst there is a gap in all Member States, it narrowed for the majority of the EU-27 countries between 2001 and 2007.

Figure 1

Absolute gender gap in employment rates (women and men aged 15–64) in EU Member States 2001 and 2007 (difference between male and female employment rates)

+++++ TIFF +++++

Source: Eurostat, EU Labour Force Survey (LFS), annual averages, EU-27.

NB: In all countries the employment rate for men was higher than the one for women.

2. This report deals with vocational training actions for women co-financed by the ESF during the 2000–06 programme period, which are the main specific ESF–funded means of promoting equal opportunities for men and women in the labour market [4]. In addition to co-financed measures, Member States may also implement national or regional programmes or actions without EU co-financing.

3. For the 2000–06 programme period, a dual approach to the use of ESF funds for the attainment of equality objectives was adopted. It combined specific actions with what is known as "gender mainstreaming". This is "the process of assessing the implications for women or men of any planned action, including legislation, policies or programmes, in all areas and at all levels [5]".

4. Specific actions to improve women’s access to and participation in the labour market include vocational training actions relating to women’s career development, access of women to new job opportunities, starting up businesses and the reduction of vertical and horizontal segregation on the basis of gender in the labour market. For the 2000–06 programme period, 4 billion euro was earmarked from the ESF for "positive labour market actions for women" [6]. The most significant of these specific actions related to vocational training, which accounted for an estimated 3 billion euro for the EU-25 [7]. For the 2007–13 programme period, the budget for specific actions has decreased significantly relative to the previous period as more emphasis has been placed upon mainstreaming.

5. In the context of positive labour market actions for women, choices have to be made about how best to improve gender equality in the labour market. On the one hand, for training to be considered successful it should normally lead to the realistic possibility of obtaining a job and as such the training people receive must be related to conditions existing in the labour market such as discernible skills shortages. On the other hand, there will also be some specific requirements of women as potential employees such as the existence of adequate childcare and flexible working arrangements. Such issues are important if positive actions are to achieve the objectives of improved and increased female participation in the workforce.

Responsibilities for the implementation of co-financed actions

6. Figure 2 sets out in simplified terms the programming and certain aspects of the implementation process of ESF-co-financed actions in the area of vocational training for women [8].

7. Projects co-financed by the ESF are implemented through multi-annual programmes (which can take the form of operational programmes (OPs) or single programming documents (SPDs)). These programmes contain established priorities, for each of which a number of measures are defined. Measures may include vocational training actions for women.

8. The programmes are drawn up by the Member States following a consultation process with the Commission (Employment DG) and regional, local and other competent public authorities, and the economic and social partners. The Commission appraises and approves the programmes.

9. A key element in this preparation, and in particular in the setting out of objectives and the allocation of financial resources to the priorities and measures, is an ex-ante evaluation of the strengths, weaknesses and potential of the Member State, region or sector concerned. This is the responsibility of authorities designated by the Member States and one of its aims is to asses the appropriateness of the strategy and targets selected given the specific situation of the regions or areas involved.

10. Member States are responsible for implementing the programmes and, in cooperation with the Commission, they must ensure that Community funds are used in keeping with the principles of sound financial management. The Commission retains overall responsibility for implementation of the budget.

Figure 2

Who does what?

+++++ TIFF +++++

11. Individual projects are proposed by project promoters — who are considered as being the beneficiaries in the context of receiving ESF support for organising training — and approved by designated national, regional or local authorities. The promoters are tasked with providing the competent managing authorities with monitoring information on project progress and the achievement of objectives. The Commission is provided with aggregated monitoring data on an annual basis.

12. For the programme period 2000–06, a mid-term evaluation of each programme had to be carried out by an independent assessor, under the responsibility of the managing authorities and in cooperation with the Commission. Some important items forming part of the evaluation were:

(a) an analysis of the continuing validity of the ex-ante evaluation’s analysis of strengths, weaknesses and potential;

(b) an assessment of the continuing relevance of the strategy;

(c) an assessment of the quantification of objectives;

(d) an evaluation of effectiveness, efficiency and likely impact; and

(e) an appraisal of the quality of implementation and monitoring arrangements.

13. The mid-term evaluation found that the analysis of efficiency, effectiveness and likely impact had been hampered by the late or slow start-up of many programmes, as few outputs or results were available by mid-2003. Problems with monitoring systems which failed to provide complete or reliable data were also highlighted [9].

14. In addition, Community structural assistance is subject to an ex-post evaluation of the utilisation of the resources and the effectiveness and efficiency of the assistance granted. This evaluation, to be carried out by independent experts, under the responsibility of the Commission, in collaboration with the Member States and management authorities, has amongst other things to consider the success or failure of implementation and the achievements and results.

Audit scope and approach

15. For the 2000–06 programme period, 75 % of "positive labour market actions for women" implemented were specific actions in the area of vocational training. The objective of the Court’s audit was to establish the extent to which ESF-co-financed vocational training actions for women were well targeted and adequately monitored. It addressed the following questions:

(a) were vocational training actions for women selected according to a clear analysis of labour market priorities?

(b) was monitoring information sufficient to allow an assessment of whether such actions had achieved their objectives?

16. The audit was carried out at the Commission and in five Member States (Germany, Spain, France, Italy and the United Kingdom) who were amongst the largest recipients of ESF money disbursed under the heading "positive labour market actions for women", in total accounting for 76 % of the expenditure of the EU-25. Within each Member State, one programme was selected according to the relative importance of vocational training activities in the measure, in four cases 75 % or greater. Five programmes were audited: Sachsen-Anhalt (Germany), Andalusia (Spain), Scotland (United Kingdom) and Lombardy (Italy), with the Ile-de-France region selected for France [10].

17. It is estimated that the five programmes contained 490 million euro of earmarked financing, that is 12 % of total spending on "positive labour market actions for women". Within this amount approximately 380 million euro was allocated to vocational training actions.

18. Eight projects per country were selected for on-the-spot audit, based upon a combination of financial significance and coverage of project types financed by the measure. The projects audited can be considered as being both typical of their programme and of other programmes falling under the heading of vocational training actions for women.

19. The review of the selection process and the analysis of monitoring information was carried out at the authorities responsible for implementing the programmes and by way of the on-the-spot audits of projects.

Observations

Were vocational training actions for women appropriately selected?

Background

20. In order to establish priorities for the 2000–06 programme period, the Member States had to carry out ex-ante evaluations, aimed at demonstrating the appropriateness of their selected strategy and justifying the proposed allocation of financial resources. This means that the identification of training as a priority and the objectives to be achieved through training should be based upon the skill needs and training and other demands emanating from the labour market.

21. Regulation (EC) No 1260/1999 established that the ex-ante evaluation of developed plans should include an appraisal of the situation in terms of equality between women and men, and of the implementation mechanisms and the expected impact of the strategy and assistance [11]. The eventual strategy chosen, and the resulting programme priorities were required to be coherent with the analysis forming part of the ex-ante evaluation. The Commission provided detailed guidance on the analysis of employment and labour market trends to be integrated in the ex-ante evaluation [12].

22. Appropriate project selection procedures needed to be put in place and applied consistently to ensure that the projects selected had training objectives which were consistent with overall programme objectives [13].

23. Considering the above, the Court examined whether:

(a) during the establishment of the programmes, an analysis of the various labour markets had been carried out by the regional or Member State authorities concerned and the objectives underlying the specific vocational training actions audited were established in light of such an appraisal; and

(b) the selection of both projects and participants were based upon criteria established in accordance with priorities identified in the relevant labour market and the programme objectives set.

Objectives not based upon the analyses

24. The analyses listed the main barriers for women entering the labour market and presented "gender gaps" in several important areas such as numbers working, rates of self-employment, occupancy of decision-making roles and labour market segregation.

25. The main weakness found during the audit, however, was that there was no clear link between the results of the analyses and the proposed training objectives. There was no clear rationale developed for the type of vocational training decided upon nor for the skills or sectors of activity targeted (see examples in textbox 1).

26. The measures developed following the analysis process mostly comprised broad actions with a general bias towards the participation of women rather than focused actions taking into account the way in which gender equality could best be advanced in the specific labour markets concerned. In three of the five programmes audited there was very little to differentiate between training actions intended exclusively for women and those open to both women and men. More resources in the form of training places were simply made available for women. In contrast, the situation illustrated in textbox 2 was an example of good targeting.

Textbox 1

In Lombardy, there was no specific analysis regarding areas of female underrepresentation despite one aim of the measure being to reduce horizontal and vertical segregation.

The French programme mentioned sectors providing employment opportunities, without analysing their potential for the creation of job opportunities for women. One aim of the measure was to give priority to training measures with potential for employment development but which had hitherto tended not to include women. However, there was no discussion of what type of measures were considered as falling into this category.

27. The weak association between the analysis of labour markets and subsequent programme objectives could have been partially compensated for at project level as project promoters were required to demonstrate how their individual projects took the characteristics of local labour markets into consideration. However, the extent to which this compensation could take place depended upon the level of information available to promoters (see textbox 3) and also on the quality of the project selection process (see paragraph 29).

28. Weaknesses of the type observed in the programmes audited, and described in paragraphs 25 and 26, were already well known. In a communication of the Commission from 2002 [14], it was stated, in the context of the strategy to achieve gender equality, that "in most of the programmes there is a significant gap between the analysis and programming strategy". It went on to say that any analysis of the gaps arising from the different socioeconomic circumstances of women and men was rare and, even when present, the strategy to overcome such gaps was not always consistent with the analysis. In spite of such awareness at an early stage in the programme period, the Commission did not take action on issues such as project selection, in an effort to remedy the known shortcomings.

Textbox 2

The Sachsen-Anhalt programme targeted specific groups of women for inclusion in its traineeship schemes: the long-term unemployed, women over 45 years of age or single mothers who had previously participated in training but were still unemployed 3 to 12 months following the training.

Textbox 3

In the case of the Lombardy programme, the project promoters had to demonstrate coherence between the qualifications targeted and inter alia the established priorities of the region. However, information on regional priorities was not included with either the programme or the call for applications.

Project selection did not take sufficient account of the analyses of labour markets nor of programme objectives

29. For three of the five programmes audited, the selection criteria did not ensure that the issues arising from the labour market analysis were sufficiently addressed nor provide a sound basis for the attainment of programme objectives (see textbox 4). In one case, calls for project proposals were not linked to a specific measure; instead projects were included as "positive labour actions for women" simply on the basis of the percentage of female trainees anticipated. In another case, whilst the selection criteria stemmed from the main objectives of the measure, the criterion evaluating a link with labour market priorities was not a determining factor when it came to the approval or rejection of projects.

Textbox 4

One of the objectives of the most important action implemented within the Andalusian programme "actions for vocational training for women" was that training courses should take into account local labour market conditions. Five priority selection criteria were established by "Ministerial Order" [15] and three of these criteria focused on local conditions. For two of these — "commitment to take on the person trained" and "inclusion of traineeship in companies" — most project applications did not include such commitments. Data on the third criteria — "employment rate regarding trainees of previous courses" — were also often not included.

The mid-term evaluation of the Lombardy programme stated that the project selection process did not satisfactorily take into account training actions needed in sectors where gender imbalances were most prevalent, and also that not enough attention had been paid to various forms of discrimination and inequalitiy in access to the labour market.

Although the audited measure of the Lombardy programme exclusively addressed women, the data show that 27 % of the training participants were male.

30. One of the main objectives of the measures was the promotion of women in sectors where there was a history of female under-representation. Despite this, for three of the programmes audited there was still an emphasis on areas of traditional female employment such as culture, health and cleaning.

31. The lack of specific target groups meant that the candidates selected were not always those most in need of support for entry or re-entry into the labour market, as the examples in textbox 5 show.

Textbox 5

The mid-term evaluation for the Lombardy programme found that training was mainly provided for people with an already "strong" position in the labour market (e.g. highly educated and/or young people who already had a good chance of finding a job without further training).

For the biggest action in the Andalusian programme, a mid-term evaluation, for which the results were available in 2005, concluded that the training measure was not suited to the needs of the women with the greatest difficulties as, on the one hand, the education level required excluded women who had only been to primary school and, on the other hand, the training hours were not suited to women who had to care for their families.

Did monitoring information make it possible to evaluate whether vocational training actions for women achieved their objectives?

Background

32. Measuring the outcomes of training actions is a difficult exercise because:

(a) a number of factors external to the training actions can play a major role with regard to the employability of individuals (e.g. evolution of labour supply and demand resulting from the overall economic situation);

(b) ESF-co-financed actions are only one part of a wider field of action implemented within Member States which have an influence on employment; and

(c) certain results are difficult to measure within the context of the attainment of programme objectives, but are of a beneficial nature in wider societal terms (e.g. re-learning how to adapt to routines including the respecting of time constraints, restoration of self-confidence, etc.).

33. In spite of such difficulties, some short- to medium-term achievements can still be measured. Indicators relevant to the objectives to be achieved could be established and corresponding targets set using benchmarks or other equivalent reference points regarding outputs, results and impact. This would constitute a basis for any subsequent monitoring and evaluation of measures and actions and hence for an appraisal of their effectiveness.

34. The following indicators provide a necessary minimum for evaluating vocational training actions [16]:

(a) output indicators: number of (female) participants, number of courses provided;

(b) results’ indicators: number of participants finishing courses, number of participants obtaining required qualifications; and

(c) impact indicators: number of trainees going on to further training, number of trainees finding a job and who were still employed a stipulated time period (e.g. 6 or 12 months) after the end of the course.

35. The Court examined whether:

(a) appropriate indicators had been identified and targets set for the measurement of output, results and impact set out in paragraph 34;

(b) the monitoring data for these indicators collected by the bodies implementing the actions were reliable; and

(c) in as far as could be ascertained from the available monitoring data, these targets had been attained.

Indicators too narrowly focused

36. Whilst output indicators had been established for all audited programmes, the situation regarding results’ and impact indicators varied. In one case both types of indicator existed, whereas in another, neither had been established. In the remaining three programmes, either results’ or impact indicators existed, but not both. The following aspects were noted.

(a) The employment situation from 6 to 12 months after the training was not used as an indicator at measure level for four programmes although, in some instances, the information was available at individual project level (see textbox 6). In addition, other issues such as job duration (i.e. short term or long term) and job level (in terms of quality and pay) were not considered.

(b) One of the programmes did not have a sufficient range of indicators to allow a worthwhile analysis to be made and in another case the indicators established were not of a type suitable for the measurement of the achievements of the training actions implemented (see textbox 7).

Textbox 6

In Lombardy, the authorities examined a sample of projects with regard to the employment status of trainees 12 months after the completion of the training actions and this was considered by the Court as being an example of "good practice".

In Andalusia, an indicator had been established concerning the success or otherwise of trainees in their subsequent search for employment. However, some weaknesses were observed with regard to data collection. For example, for one of the actions receiving 35 % of total expenditure for the measure, the relevant information was only available for 42 % of the training courses sampled.

Textbox 7

In Andalusia, the indicator "number of jobs obtained within six months after the actions" did not adequately address the specificities of the measures co-financed. As 40 % of the expenditure went on orientation and pre-training actions, more pertinent indicators might have dealt with numbers going on to further training or registering as job seekers. Furthermore, 20 % of the trainees were already in employment and therefore their responses skewed the apparent results of this indicator.

Some indicators for the Scottish programme were framed in too broad a fashion to be of much value for measuring the result of the training actions, e.g. "number of women/people gaining a part/full qualification". In terms of the indicator, the passing of one learning unit towards a full qualification was equivalent to obtaining the full qualification, even though the learning effect could not in any way be judged as comparable.

Targets not set for all measures

37. Targets to be achieved were not set for all measures audited [17]: in two cases there was no quantification at all and in another case, although a number of output and results’ indicators had been identified, the only quantified target was for "number of participants".

38. For the two measures where targets had been set, there was no explanation as to how they had been established and in one case the targets had to be revised significantly following the mid-term review (see textbox 8).

Monitoring data unreliable and sometimes incomplete

39. For two of the five programmes, output indicators concerning the number of participants were found to be unreliable. For four of the programmes, results’ indicators such as numbers completing training and gaining qualifications were also found to be unreliable.

Textbox 8

For the Scottish programme, quantified targets had been set up as per the table below. Despite enquiries by the Court, no information was provided as to how benchmarks had been established or initial and revised targets arrived at.

Indicator | Target (initial) | Target (revised) | Benchmarks |

% of women going into employment | 31 % | 5,51 % | 25-40 % |

% of women going on to further training | 42 % | 17,56 % | 20-30 % |

40. The following shortcomings were noted in respect of the underlying data:

(a) incompleteness, especially with regard to the question of what people who had been trained were doing 6 to 12 months after their training (see paragraph 36 also);

(b) problems relating to definitions or their interpretation; for example, in Scotland a declaration of intent to go on to further training was accepted as if the further training had taken place. Also in Scotland and in France, trainees who attended multiannual courses were counted several times, e.g. once for each year of attendance;

(c) discrepancies regarding the data provided; for example, in Scotland survey results on the occupation of people having completed training only corresponded to the reported results for half of the projects audited and in the annual implementation report for 2006, the number reported as having received guidance was 14371 whereas the total number of recorded participants was 11972.

41. If meaningful conclusions as to the achievements of the co-financed projects are to be drawn, it is essential that the monitoring data provided by the project promoters are reliable and complete. It was observed during the course of the audit that in most cases the managing authority and body delegated to carry out checks did not include any evaluation of the monitoring data provided by promoters.

42. For its part, the Commission, which receives the annual implementation reports, did not address the inconsistencies existing in these reports nor did it comment on the lack of impact indicators (such as "employment situation of trainees "x" months after the training"). Furthermore, it did not implement checks on the reliability of monitoring data when carrying out its on-the-spot audits [18], although the mid-term evaluations in respect of each programme had identified weaknesses in monitoring and reporting.

Assessment hampered due to lack of targets, benchmarks and reliable data on results and impact

43. Any assessment as to whether the co-financed training actions were effective was made difficult by insufficiencies with regard to indicators, targets (see paragraphs 36 to 38) and problems with the reliability of data (see paragraphs 39 and 40).

44. In respect of two of the programmes audited, the managing authorities endeavoured to evaluate whether the training had been correctly focused, by ascertaining whether jobs obtained following training bore a relationship to the type of training attended. Varied conclusions resulted from these analyses. In Lombardy, 77 % of participants finding work confirmed that the skills acquired were those required by their employer, whereas for one of the major actions in Andalusia, only 28 % of those obtaining work felt that their jobs were related to the courses attended.

Conclusions and recommendations

45. The audit on vocational training actions for women co-financed by the ESF for the 2000–06 period revealed a number of weaknesses in the establishment of the measures and the selection of the projects audited by the Court. There were also shortcomings with regard to the performance monitoring of the programmes.

Selection of vocational training actions

46. With regard to the identification and selection of vocational training actions for women, the Court concludes that, although the operational programmes listed the main barriers for women entering the labour market and presented gender gaps, training actions contained within the programmes audited were not set up as a direct response to an analysis of labour market requirements (see paragraphs 24 to 28).

47. The measures established by Member States and approved by the Commission tended to leave a broad scope within which actions could be implemented and failed to sufficiently focus training actions on specific target groups (see paragraphs 29 to 31).

48. Instead, in the programmes examined, it was left to project promoters to satisfactorily link the proposed training to real needs in the labour market. The relevant authorities did not, during the project selection process, verify adequately the validity of the justifications proposed by the project promoters (see paragraph 27).

49. Certain projects with the stated objective of promoting the employment of women in sectors where they were underrepresented still focused on training in traditional female occupations (see paragraph 30). For some projects audited, the candidates selected were not necessarily those most in need of support for entry to the labour market (see paragraph 31).

50. On the basis of the above, the Court recommends that:

(a) future operational programmes prepared in the Member States and approved by the Commission be established in light of an analysis of labour market conditions, both in terms of skills gaps to be addressed and specific actions needed to improve women’s access to and participation in the labour market; and that the type of training and objectives decided upon is a direct response to this analysis;

(b) a properly designed project selection procedure be put in place in the Member States and correctly implemented, approving those projects which respond best to solving the problems in the labour market identified by the above analysis and thereby optimising the added value of the measures.

Achievement of objectives

51. The monitoring information available for the programmes audited does not allow an evaluation of whether the vocational training actions for women achieved their stated objectives because:

(a) the indicators established were not sufficient in terms of range of attributes to be measured (see paragraph 36);

(b) targets were often not provided (see paragraph 37);

(c) monitoring information was unreliable or incomplete and therefore did not enable reliable overall conclusions to be drawn (see paragraphs 39 and 40).

52. As a result, neither the implementing authorities in the Member States nor the Commission are in a position to assess to what extent the objectives pursued by the specific measures were met.

53. On the basis of the above, the Court recommends that the Commission continues to monitor the establishment by the Member States of appropriate and feasible indicators and that reliable data be collected in the Member States, thereby allowing meaningful conclusions to be drawn on the efficiency and effectiveness of the co-financed actions.

This report was adopted by the Court of Auditors in Luxembourg at its meeting of 12 November 2009.

For the Court of Auditors

+++++ TIFF +++++

Vítor Manuel da Silva Caldeira

President

[1] Council Regulation (EEC) No 2081/93 amending Regulation (EEC) No 2052/88 (OJ L 193, 31.7.1993, p. 5).

[2] Communication from the Commission to the Council, the European Parliament, the European Economic and Social Committee and the Committee of the Regions "Towards a Community framework strategy on gender equality" (2001-05), COM(2000) 335 final.

[3] The latest report (COM(2008) 10 final) from the Commission to the Council, the European Parliament, the European Economic and Social Committee and the Committee of the Regions is dated 23.1.2008.

[4] Council Regulation (EEC) No 2084/93 amending Regulation (EEC) No 4255/88 (OJ L 193, 31.7.1993, p. 39) for the programme period 1994–99 and in particular Article 1(1)(d) thereof and Regulation (EC) No 1784/1999 of the European Parliament and of the Council (OJ L 213, 13.8.1999, p. 5) for the programme period 2000–06 and in particular Article 2(1)(b) and (e) thereof.

[5] Economic and Social Council (Ecosoc) agreed conclusions 1997/2 on mainstreaming the gender perspective into all policies and programmes in the United Nations system.

[6] Reported under code 25 of the classification of interventions laid down in Commission Regulation (EC) No 438/2001 regarding the management and control systems for assistance granted under the Structural Funds (OJ L 63, 3.3.2001, p. 21).

[7] There is no separate reporting code for expenditure on vocational training actions for women. Therefore the total amount dedicated can only be estimated on the basis of the information provided by the Member States audited.

[8] The basic process as described is applicable for any ESF-co-financed measure.

[9] For more information, see Special Report No 1/2007 concerning the implementation of the mid-term processes on the Structural Funds 2000–06 (OJ C 124, 5.6.2007).

[10] (1)Germany: OP Sachsen-Anhalt Objective 1 — 1999DE161PO003Selected priority 4, measure 4.51 ("specific measures for the qualification and integration of women");(2)Spain: OP Andalusia Objective 1 — 2000ES161PO003Selected priority 45, measures 45.16 ("improve the employability of women") and 45.17 ("promote the business activity of women");(3)France: SPD France Objective 3 — 1999FR053DO001Selected priority 5, measure 8/9 ("develop the access and participation of women in the labour market, favour the articulation of time use and professionalisation of actors");(4)United Kingdom: OP Scotland Objective 3 — 1999GB053PO002Selected priority 5, measure 5.1 ("addressing gender imbalance: promoting positive actions");(5)Italy: OP Lombardy Objective 3 — 1999IT053PO010Selected priority E, measure E1 ("promotion of the access and participation of women to the labour market").

[11] Article 41(2)(c) of Council Regulation (EC) No 1260/1999 of 21 June 1999 laying down general provisions on the Structural Funds (OJ L 161, 26.6.1999, p. 1), which specifically requires for an evaluation of the situation in terms of equality between men and women with regard to labour market opportunities and treatment at work, including the specific constraints on each group; an estimate of the expected impact of the strategy and assistance, particularly, on the integration of women and men into the labour market, on education and vocational training, on the establishment of women in business and on the reconciliation of family and working life.

[12] "The ex-ante evaluation of the Structural Funds interventions". Document aimed at national, regional or local authorities in charge of programming the 2000–06 Community structural interventions.

[13] Article 41(3) of Regulation (EC) No 1260/1999, which specifies that the ex-ante evaluation shall verify the relevance of the selection criteria once it has demonstrated consistency of the measures in the programme with the aims of the corresponding priorities.

[14] Communication from the Commission on the implementation of gender mainstreaming in the Structural Funds’ programming documents 2000–06, COM(2002) 748 final dated 20.12.2002.

[15] Ministerial Order (BOJA No 146, 19 December 2000, p. 18.647).

[16] The indicators were identified on the basis of a review by the Court of the programmes audited, of guidance issued by the European Commission in 1999 ("MEANS Collection" aimed at improving methods of evaluating structural policies) and of the specific 1999 Commission’s "Guidelines for systems of monitoring and evaluation of ESF assistance in the period 2000–06".

[17] A similar conclusion was already drawn in the Court’s Special Report No 7/2003 (OJ C 174, 23.7.2003) on the implementation of assistance programming for the period 2000 to 2006 within the framework of the Structural Funds. Paragraph 89 of this report states that "indicators were still not sufficiently quantified or relevant".

[18] The Court already addressed this issue, in particular in paragraph 120 of Special Report No 10/2006 on ex- post evaluations of Objective 1 and 3 programmes 1994 to 1999 (OJ C 302,12.12.2006): "To improve the evaluation process, better quality-control procedures need to be introduced and effectively applied by the Commission so that the problems encountered in the evaluations under review do not recur in future ex-post assessments. Such procedures should ensure (a) that relevant and reliable data is collected regularly and is available at each stage of the evaluation process (…)."

--------------------------------------------------