Special Report No 3/2002

concerning the Community Initiative "Employment-Integra" together with the Commission's replies

(pursuant to Article 248(4), second subparagraph, of the EC Treaty)

(2002/C 263/01)

CONTENTS

>TABLE>

SUMMARY

I. "Integra" was launched in 1996 as a new, fourth strand of the Community initiative (CI) "Employment and development of human resources"(1) for the programming period 1997 to 1999. From 1994 to 1997 Integra was an integral part of "Horizon", but the importance attached by the European Parliament and the Council to the subject of people excluded from the labour market resulted in recognising this as a separate issue. A Communication dated 10 July 1996 presented to the Member States by the Commission, stated Integra's objective as being "to promote measures to improve the access to the labour market and the employability of vulnerable groups who find themselves excluded from it, or at risk of being excluded from it."

II. Although target groups would continue to include those covered by Employment-Horizon(2), the Communication specified that particular emphasis should be given to actions that focus on the special needs of migrants, refugees and other similarly vulnerable groups likely to face greater discrimination in the labour market, due to the rise in racism and xenophobia in Europe(3).

III. With 418 million euro of appropriations made available the Member States initially selected approximately a thousand projects for implementation during the 1997 to 1999 period.

IV. The overall aim of the Court's audit was to ascertain whether:

- the selection procedures for choosing promoters and projects ensured that projects were innovative and targeted people most at risk of exclusion from the labour market,

- the experience acquired and lessons learnt were evaluated and followed up in a systematic, coherent and comparable way,

- monitoring and dissemination were performed effectively and used in creating new models for combating exclusion from the labour market.

V. The Commission did not explicitly provide definitions of the key words used in the guidelines nor of the population of each target group and did not establish a priorities/problems list. At neither Commission nor Member State level were any benchmarks or baseline criteria developed and it is therefore difficult to assess when and how projects were innovative and likely to give added value or to improve or change existing approaches.

VI. There is no evidence that the funds have been directed towards specific concentrations of the target groups, such as disadvantaged urban areas so that the distribution of Integra funds was the most efficient way of gathering knowledge about the target group and how to include them in the labour market.

VII. At the end of the programme the transnational aspect was seen as a positive and important factor by most promoters, but the audit revealed inconsistencies in the development phase, the selection phase and the funding policy. This had an adverse effect on the added value of the projects.

VIII. The experience acquired and lessons learnt were not evaluated and followed up in a systematic, coherent and comparable way, neither at Commission level nor at that of the Member States.

IX. The Commission and the Member States did not consider using indicators to describe the basic situations of the various target groups included in the initiative. Had they done so, they could then have formulated clear, unified and measurable criteria and created a baseline against which to evaluate the Integra initiative.

X. No evidence was found to prove that dissemination was performed systematically and effectively and used in creating new models for combating exclusion from the labour market for each target group.

XI. For none of the Member States visited was evidence found either at the national or transnational level of the organisation of systematic transfer of good practice. At promoter and local/regional level there was more evidence of such transfers. However, there was no systematic organisation of it, and promoter contact and networking with local authorities and other promoters appear to have been the most important factors.

XII. The Integra initiative has raised public awareness of excluded groups. Nevertheless, it is not possible to draw any firm conclusion on the extent to which the Integra initiative has resulted in new knowledge about how to promote measures to combat exclusion from the labour market in the different target groups and whether that knowledge has been effectively disseminated to political decision-makers.

XIII. The Court recommends:

- before launching innovative programmes such as Integra the Commission and Member States should identify the main areas (target groups and labour market segments) where innovative projects could contribute to the formulation of new policies,

- indicators should be developed at the beginning of such programmes to allow a continuous and systematic evaluation of the innovative projects so as to assess and compare the results and to make sure that the programmes will have an added value and are an improvement compared to other existing programmes,

- the Commission should develop a strategic plan for systematic transnational transfer of lessons learnt and good practice to social partners and policy-makers in order to make sure that the results are used in relevant policy implementation.

INTRODUCTION

Employment-Integra

1. The Commission launched the Community initiative "Employment and development of human resources" in 1994(4). The aim was to contribute to the development of human resources and improve labour market functioning, in order to enhance employment growth and promote both social solidarity in the European Union and equal opportunities for women in the labour market. From 1994 to 1997 Integra was an integral part of Employment-Horizon (see footnote 1 on page 2), but the importance attached by the European Parliament and the Council to the subject of excluded people resulted in recognising this as a separate issue.

2. In a Communication(5) to the Member States in 1996, the Commission added a new strand, Employment-Integra to the abovementioned Community initiative, for the programming period 1996 to 1999. The ultimate goal of this new strand was "to promote measures to improve the access to the labour market and the employability of vulnerable groups who find themselves excluded from it, or at risk of being excluded from it"(6). The target groups included the long-term unemployed, jobless single parents, the homeless, gypsies and travellers, prisoners and ex-prisoners, substance abusers, etc., already supported by the existing strand Employment-Horizon. However, Employment-Integra was to place the accent on the special needs of migrants, refugees and other similarly vulnerable groups who were more likely to be victims of discrimination in the labour market, because of such phenomena as racism and xenophobia.

3. The ESF Community Initiatives are the special financial instruments of structural policy which the Commission itself proposes to the Member States to support measures which will help solve problems having a particular impact at European level. These initiatives have the following key features which give them added value as compared with the other measures financed by the Social Fund:

(a) transnationality;

(b) innovation;

(c) bottom-up approach;

(d) dissemination;

(e) mainstreaming.

4. With 418 million euro of appropriations made available (Table 1) the Member States initially selected approximately a thousand projects for implementation during the 1997 to 1999 period.

Table 1

Budget implementation by Member State

>TABLE>

Source:

Commission of European Communities, December 2001.

5. Integra, as a strand of the Community initiative "Employment", is supposed to have a catalystic effect on innovation at Community level and at the same time encourage the organised transfer of experience and the diffusion of good practice among promoters and Member States. Its impact should therefore not be measured in terms of employment creation but in terms of new knowledge about innovative methods of improving the employability of the target groups.

Audit objective and approach

6. The overall aim of the audit was to investigate whether the Community initiative Employment-Integra provided the Commission and Member States with new knowledge and new methods to promote measures to combat exclusion from the labour market for the most vulnerable groups in society.

7. The main audit objectives were to ascertain:

(a) whether the selection procedures for choosing promoters(7) and projects ensured that projects were innovative and targeted people most at risk of exclusion from the labour market and that they presented a real transnational character;

(b) whether the experience acquired and the lessons learnt were evaluated and followed up in a systematic, coherent and comparable way;

(c) whether monitoring and dissemination were carried out effectively and used in creating new models for combating exclusion from the labour market.

8. The audit approach consisted of examination of the systems implemented on the basis of documentation review and interviews with key staff at Commission level and in four Member States (Denmark, Spain, France and the United Kingdom) where 16 projects were audited in the context of the three audit objectives mentioned above. The expenditure of these projects consists to a large extent of the salary cost of staff involved in the resocialising and training of the participants and of allowances paid to the latter.

BUDGET IMPLEMENTATION

9. Total commitments made available for the Integra initiative were 418 million euro. This was an increase of 8 % over the 385 million euro initially allocated due principally to the fact that Member States transferred funds from other CI Employment strands to Integra in order to satisfy requests for funding additional Integra projects.

10. By 31 December 2001 the Commission had paid 327 million euro to the Member States representing 78 % of the total commitment. The final closure of Integra is due to take place in 2003 together with the other strands of CI Employment after the Commission has received the required documents as foreseen by Regulation (EC) No 2064/97.

PRE-LAUNCH ACTIVITIES

General

11. Launching an innovative programme such as Integra requires shared understanding, between Commission, Member States and promoters of the theoretical concepts and intentions behind the programme, so that misunderstandings at the operational level are kept to a minimum. For successful implementation in the Member States it is essential for the pre-launch material to identify the key concepts and provide a clear definition of them, to establish clear objectives, clear criteria, and clear monitoring procedures and use statistical information or indicators about the target groups.

Commission level

12. The Commission's Communication to the Member States does not explicitly:

(a) offer any definition of when a project is "innovative" or identify what the innovation element in a project should consist of;

(b) define the population of each target group in Integra;

(c) identify clear objectives for each target group in Integra;

(d) provide indicators for monitoring the selection of Integra projects;

(e) use statistical material or indicators in the selection of target groups;

(f) establish benchmarks or baselines for the target groups or make use of previous experience of traditional ESF programmes to facilitate evaluation of projects.

13. The Commission Communication places particular emphasis on migrants, refugees and people in similar situations as a target group, but makes no distinction between the various categories within this very diverse group. As a result, essentially different categories, requiring totally different approaches, like refugees with no education at all and highly skilled refugees, are treated under the same heading, which makes evaluation extremely difficult.

Member State level

14. No list of problems or priorities was drawn up, for the potential target groups, prior to the distribution to potential promoters of information material on Integra. In this context little use was found to be made in the Member States visited of relevant statistical material or indicators beforehand in order to analyse the situation of each potential target group.

15. All the Member States visited had drawn up detailed documentation on Integra, which was sent to potential project tenderers or could be obtained on request. However, none of the Member States visited gave a clear definition of key words used in their pre-launch material. They did not develop benchmarks and baseline criteria for use by potential promoters as a means of distinguishing between "experimental" projects from the traditional, mainstream ones. Thus it was left to individual promoters to interpret and understand key concepts of the programme (see paragraph 3) and to assess for themselves "added value" and the "improvement of existing approaches".

SELECTION AND IMPLEMENTATION

General

16. At European level the Commission sets out the global framework within which the initiative has to be implemented at national level (see Table 2 - Selection of Projects). In line with the Communication to the Member States point 13, a technical assistance office (Europs) was established as an European support structure to assist the Commission. Europs established and maintained a Europe-wide database on projects and partnerships.

Table 2

Selection of projects

>TABLE>

Source:

Commission European Social Fund - Employment Integra - SR - February 1997.

17. At Member State level several national organisations are involved. The national monitoring committee, which is usually made up of representatives of a wide range of national authorities and also includes a representative from the Commission, monitors and evaluates the implementation of the initiative. national support structures (NSS) have been established to offer technical assistance at national level and to provide advice and support to individual projects, collecting and transmitting information.

18. In order to ensure that the objectives and priorities are achieved, active implementation should be monitored following a common methodology. This would facilitate monitoring and evaluation and putting the knowledge and experience acquired to effective use in the creation of new models for combating exclusion from the labour market. Especially where management of an initiative is decentralised, one of the prerequisites for its success is that the various entities responsible for implementing the programme follow clearly defined reporting guidelines.

Commission level

19. The lack of definition of each target group means that information from the Member States on various target groups cannot be used to monitor the effects of Integra on a Community scale. In the absence of guidelines for the drafting of the annual implementation reports by the Member States authorities and in the absence of a harmonised classification system, which caused considerable variations in the quality of information, it was practically impossible to have a systematic Community-wide overview of the implementation of Integra.

20. The contract between the provider of the technical assistance and the Commission ended in April 2000 and Europs had thereafter three months to close down. The database developed by Europs was handed over to the Commission. However, it was not maintained or updated by the Commission and because many of the projects had not been completed when Europs closed(8), the knowledge and experience acquired at project level could not be utilised.

21. In conclusion, the Commission did not develop a monitoring system to ensure that all necessary information was gathered and filed in a harmonised system and used effectively in creating new models for combating exclusion from the labour market.

Member State level

22. In line with the Commission Communication each Member State set up a single monitoring committee which was responsible for the Employment initiative as a whole and, therefore, also responsible for the final selection of the projects.

23. The Member States were aware of the risk that new promoters with potentially good innovative ideas, but little knowledge of or familiarity with the Structural Funds, could face difficulties in making the application correctly, and some counselling was given to project promoters in that respect.

24. In the interest of impartiality this pre-selection phase (administration, counselling and first screening of applications) was separate from the final selection of promoters. The Member States visited had different selection criteria and methods, but a careful examination of the applications took place, and even though the vast majority of the selected promoters were familiar with and had participated in other ESF programmes new promoters were also selected to participate in Integra. This is a positive aspect in the light of the fact that filling out the application form can be rather difficult and time consuming, especially for small and new promoters.

25. In all the Member States visited it proved very difficult to assess the innovative content of the selected projects, and in many cases it was difficult for the competent authorities to explain why a selected project had innovative aspects and was not just a different presentation of approaches that were already part of existing mainstream projects funded under the European Social Fund.

26. Due to the lack of clear priorities the most important indicator in the selection of project tenders and projects was a quality assessment of the project tenderer and of the application itself. It was found that the project's target group, the complementarity of projects or other indicators were not considered.

27. In none of the Member States visited did the selection procedures as implemented give particular emphasis to actions that focused on the special needs of migrants and refugees, as required in the Commission Communication.

EVALUATION

General

28. According to the Commission's communication, "During and at the end of the planning period the Commission shall evaluate, in partnership with the Member States, the results of the programmes submitted. In order to carry out this evaluation the Commission will use the objectives specified by Member States as the main benchmark against which to assess progress. As a function of the desired objectives and the implemented measures this evaluation will provide data related to the target groups including the final beneficiaries."

29. Neither the Commission nor the Member States used indicators to describe the basic situations of the various target groups included in the initiative. Had they done so, the indicators could have been used to formulate clear, unified and measurable criteria and to create a baseline against which to measure and evaluate the innovative aspects of the Integra initiative.

30. In experimental Initiatives like Employment-Integra the concepts of success and failure are equally important. Every project sets out to test a hypothesis, and "failure" in this regard is as valuable a lesson for the future formulation of labour market policy as "success". The Commission did not initiate a specific ex ante evaluation for the Integra initiative and its target groups and there was no indication by the Commission, nor by the Member States visited, of how to measure if and when a project or concept was successful or unsuccessful. As a consequence, lessons learnt and experience acquired at project level in different countries were not monitored in a coherent way and at Commission level it was not possible to evaluate whether concepts were good or bad.

31. None of the Member States visited, nor the Commission, carried out any economic analysis or cost benefit analysis of the economic efficiency of the different innovative projects. Value-for-money indicators of the successful inclusion in the labour market of one individual from a target group were not developed and an ex post evaluation was, therefore, not possible.

Commission level

32. An evaluation(9) of the whole Employment CI was carried out from August 1998 to January 2000, which means that the evaluation started after the launch of the programme and finished before the end of the programme. Because of the late involvement of evaluators and the late start in planning the evaluation process, the evaluators were limited to carrying out the evaluation on the basis of information available at the time or by means of case studies in selected Member States.

33. This evaluation found that on the basis of the available data on labour market outcomes, it is not possible to draw any conclusions for the initiative as a whole as regards target groups' access to job opportunities. It was further stated that the ultimate goal of the initiative is to improve the representation of target groups in mainstream programmes and their job placement in the labour market, but that without information on labour market outcomes it is difficult to assess the added value of the initiative.

34. A specific evaluation of Integra, in which the experience, knowledge and lessons learnt from innovative projects in all the Member States were evaluated and followed up in a systematic, coherent and comparable way, did not take place.

Member State level

35. In general the Member States visited had initiated several evaluation projects at different levels, such as national or international conferences and comprehensive reports on the Employment programme as a whole. However, similarly to the study initiated by the Commission, the Member States' evaluation reports mainly deal with the Employment initiative as one programme and not as several strands, each of which targets different vulnerable groups.

36. For lack of a basic strategy most of the evaluations took the traditional form of a general, quantitative description and did not take into account the fact that Integra was targeting vulnerable/disadvantaged groups with different needs. The reports did not evaluate the experience acquired and lessons learnt from all the different projects dealing with the same target group. None of the Member States visited had produced reports or guidelines with "best practice" for the inclusion into the labour market of the different target groups covered by the Integra programme.

37. In the Court's view, the evaluation of the Integra programme should result in drawing up a set of models showing how projects for the various target groups should be designed in order to offer the maximum chance that these "marginalised groups" would be successfully integrated into the labour market.

Project level

38. In March 1997, Europs, acting on behalf of the Commission, produced a "Guide to self-evaluation for Employment projects" which could be sent to projects by the NSS (see point 17) on the national monitoring committees' behalf. It was designed as an aid for promoters who were not used to making evaluations, rather than as a compulsory evaluation tool.

39. In all the Member States visited, the NSS had set up some evaluation tools to help the promoters in the evaluation of their projects. NSS paid personal visits to promoters and/or asked for periodic progress reports. It was observed that, in general, cooperation between promoters and NSS during the programme had been satisfactory and the promoters that were interviewed during the audit saw the evaluation as an important part of the project.

40. Apart from evaluating their own projects, the promoters involved in transnational networks had also participated in the evaluation of the network. As part of the latter evaluation, accounts of the experience acquired in the course of the transnational cooperation were collected and included in transnational evaluation reports.

41. Nevertheless, evaluations were still not synchronised and harmonised at project level, because the necessary monitoring instruments and measurable indicators were not available to promoters and the competent authorities did not develop an evaluation strategy before launching the programme.

42. The promoters visited were asked whether there had been any follow-up to their evaluation reports. None of them had received any feedback as regards the results of the innovative approach they had used in the projects, and none of them knew whether the project approach that had been developed was adopted (or rejected) by competent authorities or in other regions.

43. One consequence of an experimental programme like Integra is that there is no strategy to continue projects which, according to their evaluation reports, had demonstrably developed a successful approach. In some cases it had been possible for successful promoters to continue, because local authorities and/or labour market organisations were interested in the continuation of the project, for others it was not possible to continue with projects they deemed successful. In several cases promoters had been obliged to close down the activity and experienced staff had left for other work.

DISSEMINATION AND MAINSTREAMING

Dissemination

44. In fulfilling the objectives of Integra (see point 2), details of experiences, lessons learnt and best practice need to be disseminated from project, to local, to regional, to national and to Commission level, and from Commission level back to all Member States.

45. Many of the project promoters visited contacted local newspapers, labour market organisations and local authorities during and at the end of the project. But dissemination outside the local area was rare and often depended on the promoter's own network.

46. Judging by the example of the projects visited, there appeared to be a greater chance of information being disseminated successfully if the promoter had, at an early stage, such as the project-planning phase, set up a "project-steering group" consisting of representatives from the local municipality, labour market organisation or other interested parties.

47. These projects also appeared to be highly effective in terms of raising the profile of vulnerable groups in the labour market, possibly because the projects gave the labour market organisations insight into the problems and special needs of the target group, and at the same time made them aware of the potential of the "marginalised groups" as members of the workforce.

48. None of the Member States visited had, at the time of the visit, developed an easily accessible database where potentially interested promoters could find information on the experience acquired, lessons learnt, etc., on projects dealing with a specific target group.

49. At project level a variety of information material such as reports, leaflets, CD-ROM's, videos had been produced and local media, including radio and newspapers, had been used to disseminate project results. In the areas which had Integra projects, these appear to have raised public awareness of the target groups and contributed to their vocational integration.

50. Member States have produced material on the Integra programme, with information on some or all of the projects included in the programme, but none of the Member States visited had produced a code of best practice on how to integrate the different target groups into the labour market, despite most of the projects visited having been successful.

51. On the basis of the information available at Commission level and from the Member States visited, it is not possible to draw any firm conclusion on the extent to which Integra has resulted in new knowledge about combating exclusion from the labour market in the different target groups and whether that knowledge has been effectively disseminated to those responsible for political decisions.

Mainstreaming

52. Before mainstreaming is possible on a broader European level it is necessary to make an assessment of the innovative potential of project results for each target group, draw lessons and develop best practice, and make a selection of the most promising approaches for mainstreaming. Some of this was done by Thematic Focus Groups but no evidence was found to prove that transnational mainstreaming has taken place.

53. The establishment of transnational partnerships and networks might have had a transnational effect for projects in the partnership, but this was not treated in a systematic manner in the process of evaluation and mainstreaming.

54. No effective system of transfer of good practice was found to be working in any of the Member States visited. The evaluation procedure was not working in a way that made it possible to draw conclusions and select best practice for a target group and there was no structured system to link the Integra results for the different target groups to the national policy-making processes.

55. There was more evidence of such transfers at the promoter and local/regional levels, especially in the cases where the project-steering group consisted of actors from the municipality and labour market organisations. However, no systematic organisation of it took place, and promoter contacts and networking with local authorities and other promoters appear to be the most important factor.

OTHER ASPECTS

Unemployment rate

56. The projects visited provided some evidence that mainstreaming or continuation of projects and empowerment of the target groups were also linked to the unemployment rate in the geographical area. In some areas with low rates of unemployment there was a greater interest and involvement from labour market organisations and municipalities in looking at the target group as a potential workforce. In areas with high unemployment rates it appeared to be more difficult to accept the target group as a potential workforce. This factor makes comparative analysis of performance between areas with different unemployment rates very difficult.

Distribution of funds

57. The Communication states that the Employment initiative "....is applicable to the whole of the territory of the Union. However, particular emphasis will to be placed on the needs of the less-favoured regions. A number of measures will therefore apply only to priority regions (Objective 1, 2, 5b and 6)". On the other hand, for the measures under the Employment-Integra strand it is stated specifically that "greater emphasis should (...) be placed on actions targeted at disadvantaged urban areas".

58. There is no evidence that the funds were directed in any particular way to urban or other concentrations of the target groups, so that the actual distribution of Integra funds was the most efficient way of gathering knowledge about the target group and how to include it in the labour market.

Transnationality

59. The transnational partnership has been an in-built dimension of the Community initiatives since they were launched and in Integra, as well, projects are not funded unless brought together in transnational partnership with projects from other Member States. The objective is to transfer expertise, knowledge and experience and at the same time provide a tool for developing or testing innovative ideas which cannot be realised in a national context alone.

60. Through the databases established by Europs and the support structures in the Member States, it was possible for interested promoters to find and establish contact with potential transnational partners.

61. There was some inconsistency in the programme seeking to involve new promoters with experimental, innovative projects: but at the same time requiring them to have a transnational partner before an application could be approved. A "new promoter with experimental projects" might withhold his application because of this requirement.

62. Another inconsistency was that if a promoter had found a transnational partner and developed a common project with him before submitting the application, neither partner could be sure that his part of the project would be approved by his national committee. Even if approval was given, there was no guarantee that the transnational partners would receive their aid at the same time, so that the projects could be implemented in parallel. None of the Member States visited had addressed these problems, e.g. by establishing cooperation between the committees in the Member States as regards the selection procedure and the implementation of the programme.

63. Neither the Commission nor the Member States established guidelines on the amounts to be made available for transnational cooperation. This led to large differences in the amounts available for project promoters to implement the transnational partnership both within a Member State and between Member States. One consequence of this was that projects working together in a partnership did not all have the same amount of funds for use in the cooperation, and were thus not able to participate on equal terms(10).

64. This led to a situation whereby one Member State funded projects by means of a fixed amount per participant, regardless of the target groups' needs, and another made a quality assessment of the application, accepted that the transnational element was an important factor and financed visits to partners abroad for both trainers and participants. Thus, an inequality of treatment was created from the outset.

65. Some promoters visited had had no previous experience of transnational co-operation. Their initial expectations were very low: they saw the transnational element as time-consuming and an extra administrative burden, and only used the partnership for exchange of information and experiences. At the end of the project they still saw the transnational element as very time-consuming, but also viewed it as a positive and important factor for the project and one which they would like to develop in future projects. Amongst other things, receiving information about how other countries were dealing with problems had given them a greater awareness of alternative approaches to the target group.

66. Other promoters, often with experience from previous transnational partnerships, were involved at an early stage in networks comprising projects from several countries and had clear objectives for the partnership. Their involvement in and expectations of the transnational partnership started on a different level from the new promoters. But the absence of clear models, policies and procedures at central level led to critical comments on their part and point to a need for a more active role of and closer cooperation between Member States.

67. In general the transnational element was not a cornerstone of the approved projects, and was often mentioned as an additional activity, without a direct link to the primary purpose of the project.

CONCLUSIONS AND RECOMMENDATIONS

68. The Integra initiative, which was introduced in response to a rising awareness and concern about the access to the labour market and the employability of vulnerable groups, has generally been found overambitious, because it had too many objectives and was insufficiently focussed on the target groups due to a lack of overall guidance and coordination.

69. The pre-launch activities lacked the basic requirements for securing a unified understanding of the objectives and application of harmonised methodology among the Member States (see points 12 to 15).

70. Neither the Commission nor the Member States used indicators to describe the basic situations of the various target groups included in the initiative. Had they done so, the indicators could have been used to formulate clear, unified, identifiable and measurable criteria and to create a baseline against which to measure and evaluate the effect of the Integra initiative (see points 12 and 15).

71. In none of the Member States visited did the selection procedures as they were implemented, give particular emphasis to actions that focused on the special needs of migrants and refugees (see point 27).

72. The monitoring procedures set up by the Commission and in the Member States did not ensure that all necessary and adequate information was gathered and filed in a harmonised filing system which could subsequently be used for evaluation and for analysis (see points 19 to 21).

73. Had there been a specific evaluation of Integra, the experience, knowledge and know-how obtained from innovative projects in all the Member States could have been followed up in a systematic, coherent and comparable way, but the evaluation did not take place (see points 28 to 43).

74. Dissemination of good practice is likely to be influenced positively by setting up a local steering group at an early project stage (see point 46).

75. On the basis of the audit observations it is not possible to conclude that:

(a) the selection procedures applied for choosing promoters and projects ensured that projects were innovative and targeted people most at risk of exclusion from the labour market;

(b) the experience acquired and the lessons learnt were evaluated and followed up in a systematic, coherent and comparable way;

(c) monitoring and dissemination were carried out effectively and used in creating new models for combating exclusion from the labour market.

76. Furthermore, it cannot be concluded that the design of the Integra programme was the most cost effective and most efficient way of gaining experience in combating exclusion from the labour market and promoting inclusion in the labour market for the most vulnerable groups in society (see points 31and 56 to 58).

77. The Court recommends:

(a) before launching, in response to very generally formulated political incitations, innovative programmes such as Integra, the Commission and Member States should precisely identify the main areas (target groups and labour market segments) where innovative projects could contribute to the formulation of new policies;

(b) indicators should be developed at the beginning of such programmes to allow a continuous and systematic evaluation (ex ante, mid-term and expost) of the innovative projects so as to assess and compare the results and to make sure that the programmes have an added value and are an improvement compared to other existing programmes;

(c) the Commission should develop a strategic plan for systematic transnational transfer of lessons learnt and good practice to social partners and policy-makers in order to make sure that the results are used in policy implementation.

This report was adopted by the Court of Auditors in Luxembourg at its meeting of 17 and 18 July 2002.

For the Court of Auditors

Juan Manuel Fabra Vallés

President

(1) This CI was launched on 15 June 1994, in the context of Article 11 of Council Regulation (EEC) No 4253/88 as modified by Council Regulation (EEC) No 2082/93 (OJ L 193, 31.7.1993, p. 20). Originally it had three strands: Now, Youthstart and Horizon.

(2) These included "long-term unemployed, jobless single parents, the homeless, itinerants, gypsies and travellers, prisoners and ex-prisoners, substance abusers, etc." Communication to the Member States (OJ C 200, 10.7.1996, p.13).

(3) Communication to the Member States (OJ C 200, 10.7.1996, p. 19).

(4) Article 11 of Council Regulation (EEC) No 4253/88 as modified by Council Regulation (EEC) No 2082/93 (OJ L 193, 31.7.1993, p. 20).

(5) Communication to the Member States laying down amended guidelines for operational programmes or global grants which Member States are invited to propose within the framework of a Community initiative on employment and development of human resources aimed at promoting employment growth mainly through the development of human resources (OJ C 200, 10.7.1996, p.13).

(6) Paragraph 4(d) of the Commission Communication 96/C 200/06 (OJ C 200, 10.7.1996).

(7) Public or private person or organisation which requests and possibly obtains assistance in the framework of an intervention for a given project.

(8) See Special Report No 3/2000 on the European Social Fund and the European Agricultural Guidance and Guarantee Fund (Guidance section), Measures to assist the employment of young persons, points 59 and 60 (OJ C 100, 7.4.2000).

(9) On behalf of the Commission, Forschungsstelle für Berufsbildung, Arbeitsmarkt und Evaluation (FBAE) at FHVR Berlin and the Netherlands Economic Institute (NEI), Rotterdam developed "A methodology for the European evaluation of the Employment Initiative". This study was the basis for the subsequent evaluation of the Employment programme, including Integra, and on behalf of the Commission FBAE and NEI published a "European-wide Evaluation of the Community Initiative Employment" in March 2000.

(10) See Special Reports No 3/2000 on the European Social Fund and the European Agricultural Guidance and Guarantee Fund (Guidance section), Measures to assist the employment of young persons, point 51(c) (OJ C 100, 7.4.2000) and No 22/1998 concerning the management by the Commission of the implementation of measures to promote equal opportunities for women and men (OJ C 393, 16.12.1998).

The commission's replies

SUMMARY

II. In its Communication of 10 July 1996 on Integra,(1) the Commission also highlighted:

- the use of methods developed on the basis of practical experience in the field; and

- the application of integrated approaches in order to take account of the diversity of problems relating to exclusion from the labour market.

IV. The Commission believes that, given the objectives pursued by the Integra initiative, methodological questions inevitably arise regarding the tools for achieving those objectives. Exclusion is a complex phenomenon influenced by many factors. As many evaluations have shown, it is not easy to assess the effectiveness of operations, as the indicators often prove to be inadequate.

V. No precise and inflexible definition of target groups beyond those listed in the Communication to the Member States (see point II above) was attempted by the Commission in view of the often different circumstances prevailing in the different Member States. Concepts and definitions can differ from Member State to Member State, and a detailed "top-down" definition by the Commission risked limiting the scope of action of the Member States.

Moreover, selection of projects to participate in Integra was the prerogative of the selection committees established in each Member State by the Managing Authority for the initiative. The precise interpretation of innovation in each Member State was a responsibility of the selection committee.

For the successor initiative, "Equal"(2), the Commission provided a clear and comprehensive set of definitions, based on the typology which emerged from the evaluation of Employment. Nevertheless, despite this definition, appraisal of the innovative nature of a project remains the sole responsibility of the project selection committees, which base their judgment among other things on their knowledge of the situation on the ground and of local circumstances.

VI. The requirement for projects to be proposed by promoters working directly with the target groups led to an automatic concentration of effort on areas with a concentration of target groups. The specific problems of disadvantaged urban areas have also been specifically tackled by the Urban Community initiative programmes.

VII. Lessons were drawn from the implementation of the transnational dimension of the initiative and these lessons strongly influenced the design of the successor human resource Community initiative, Equal.

VIII. As regards evaluation and follow-up to lessons, an EU-wide mid-term evaluation was carried out from August 1998 to January 2000, and an ex post evaluation of the Employment initiative will be carried out in 2002 and 2003.

The Commission and the Member States have worked hard to draw lessons from the experience acquired during the initiative, in particular by:

- producing a series of publications,

- publishing hundreds of descriptions of good practice on the Web,

- holding regular meetings of working parties at European level,

- organising several conferences.

IX. The Commission would stress the importance of the work on indicators currently in progress as part of the "inclusion" process (following the Lisbon and Nice summits). This exercise highlights the difficulties in establishing harmonised indicators that can shed light on instances of exclusion.

X. The Commission produced guidance in 1997, entitled "Impact Guide for Employment Project Promoters". The Guide provides methods on how to plan a strategy to disseminate the successful outcomes of a project(3). This was followed in 1999 with a "Guide to Public Relations Activities for Project Promoters", to enable projects in the Community initiatives to realise their full potential through effective communication(4). As regards dissemination and mainstreaming applied to Integra, the new knowledge about combating exclusion from the labour market has been principally disseminated at EU, national and local level via publications and web sites.

XI. The Commission's web site at http://europa.eu.int/comm/equal contains the publications referred to in point X above, and many more giving examples of good practice, and is freely accessible by all labour market decision-makers in the EU. The majority of the publications produced by the Commission were printed and distributed widely throughout the EU.

Events organised at national and European level and the systematic incorporation of past experience in programmes that took over from Integra (in particular the Equal initiative) are indications of intense activity in the running of programmes. On the specific question of transnationality, a number of publications available on the Web describe what has been achieved through this form of cooperation.

XII. The final assessment of the effectiveness of Integra in achieving its objectives cannot be made in advance of the receipt and analysis by the Commission of the final national reports (due in June 2002) and of the publication of the ex post evaluation in 2003.

XIII. The Commission considers that, in the particular case of Integra, the main target groups of all strands of the Employment Initiative were well defined in the Communication to the Member States, but without the imposition of an overly 'top-down' definition that would have restricted the Member States' ability to fine-tune the focus of their Operation Programmes to the specific needs of their labour markets.

- Furthermore, the only valid indicators for a programme of innovative action such as Employment can be changes in labour market policies and practices that are deeply rooted and permanent. Experience of Adapt and Employment lead the Commission to the same conclusion as the Court in respect of indicators and programme follow-up, and this issue has been addressed in the design of the Equal initiative. However, it is obvious that most of these indicators will remain qualitative rather than quantitative in nature, thereby leaving broad scope for discretion.

- As regards the need to develop a strategic plan for systematic transfer of lessons and good practice, the Commission considers that the national action plans for employment, and the European employment strategy in general, form an important part of the strategy plan for the systematic improvement of human resources in the Member States, in which the human resource Community initiatives including Employment, play an important and active role.

PRE-LAUNCH ACTIVITIES

12. The Commission wishes to point out that:

(a) selection of projects to participate in Integra was the prerogative of the selection committees established in each Member State by the managing authority for the initiative. The precise interpretation of innovation in each Member State was a responsibility of the selection committee;

(b) as regards the definition of the population of the different target-groups in Integra, point 12 of the 1996 Communication to the Member States laying down amended guidelines for operational programmes in Employment(5) lists the target groups of the Integra strand. However these guidelines are broad and general, and the Member States were encouraged to interpret them in accordance with their own circumstances and priorities as set out in their Operational Programmes;

(c) concerning the identification of objectives, these were set out in general terms in the 1996 Guidelines (in section I 'Objectives') and in the Structural Fund regulations. Accordingly, the precise definition of the objectives for each of the target groups identified was to be set out in the Member States' Operational Programmes in the light of their national policies and priorities;

(d) and (e) the overall context for the selection of projects and target groups was provided in the Commission's White Paper on Growth, Competitiveness and Employment(6) and in Section VII of the Guidelines. It underlines, however, the difficulty in establishing indicators to measure exclusion, as shown by the exercise currently carried out in this domain, following the Lisbon and Nice summits;

(f) the establishment of benchmarks relating to identifiable target groups could give an inaccurate indication of the success of a programme like Integra where the target population is in a constant state of flux, and where qualitative effects on systems, and on factors impacting on labour market exclusion, are of greater relevance.

13. The Commission is aware of the difficulties involved in evaluating an initiative containing such a diversity of target groups and activities. Nonetheless it feels that the potential benefits from an approach that allows cross-fertilisation of activities and lessons outweigh the advantages of a comparative analysis of the projects.

14. The approach adopted was a holistic one, in which the problem of labour market exclusion in its broadest sense was tackled. Nonetheless, for example in the United Kingdom(7) the objectives of the initiative were based on an analysis of the British labour market, which identified the groups that were most disadvantaged. This analysis was published in September 1996 as a supplement to the OP(8) for Employment, in which the target groups for Integra were identified and the rationale for the proposed activities justified by statistical means. The United Kingdom 'Guidance for Applicants for the Employment nitiative 1997 to 1999 (Part 2)', annex 3 'beneficiary information', defines the proposed beneficiaries in some detail, while Annex 4 lists the 805 wards experiencing labour market disadvantage which were prioritised under Integra.

Similar information is available in the wider context of the French Operational Programme for Employment.

15. Clear definitions of key words were given in, for example, France, where the guide for applicants gave an explanation of key ideas including the assessment of the concept of innovation. Similarly, the United Kingdom Guidance for Applicants for the Employment initiative 1997 to 1999 gave definitions of the concepts of innovation, monitoring, multiplier effects, dissemination and mainstreaming as well as transnationality. This guidance was distributed to all applicants in advance of the submission of project proposals. In addition, in Annex 1 of the United Kingdom Guidance, applicants were provided with the precise selection criteria that were used to assess their understanding and application of these concepts.

SELECTION AND IMPLEMENTATION

18. The Commission had established general guidance for the monitoring and evaluation of ESF operations in 1995, and specific guidelines for the Employment initiative in 1999(9).

19. On the question of defining target groups, the Commission would point out that:

- the target groups of the Integra strand are defined in point 12 of the Communication to the Member States(10) laying down amended guidelines for operational programmes in Employment(11),

- it had a clear overview of the implementation of Integra throughout its period of application through its technical assistance office, Europs, which gathered information on the projects in cooperation with the national technical assistance offices for Integra,

- according to the Regulation, the Final Reports on the implementation of the Initiative are due to be sent to the Commission by 30 June 2002; analysis of these reports will provide the Commission with a detailed knowledge of the implementation of Integra in all Member States.

20. The aim of the Initiative was to improve the operation of the labour markets in the EU, and since the principal actors here are national, or even sub-national, it was to those levels that most mainstreaming attention was paid.

The bulk of dissemination concerning the projects was carried out via the national technical assistance offices, most of which continued in operation for a period following the closure of Europs.

At EU level, from 1997 onwards, dissemination was centred on the nine 'Thematic Focus Groups' (TFG) set up expressly for this purpose. These TFGs were led in all cases by Member States, and contained representatives from national technical assistance, national administrations and projects, and thus were not greatly affected by the close of Europs.

21. The initial lessons to be drawn from Integra, and the other strands of the Employment Community initiative, are described on the Commission's public internet site(12). These key lessons were classified according to the taxonomy of the European employment strategy to facilitate their use by Member States in the drawing up of their national action plans for employment.

In addition, in order to stimulate Member States to use the results of the Employment initiative, wherever there were any, the design of the current Equal initiative requires Member States to specify the lessons they learned from Adapt and Employment for the thematic priorities selected for Equal.

25. In Spain one of the main selection criteria for projects was "innovation" (as specified in the amended Operational Programme, Commission Decision C(97) 863).

In the United Kingdom the Employment OP specified additional selection criteria for Integra which asked applicants to identify how the project differed from existing provision, and did not duplicate other provisions.

26. As pointed out in paragraph II, the Commission Communication of 1996 set out clear objectives for Integra projects. In the United Kingdom at least, the selection criteria took into account the target group and complementarity with other similar programmes. The United Kingdom Guidance for Applicants for the Employment Initiative 1997 to 1999 (Part 2), Annex 1, shows clearly the selection criteria for Integra and the other strands of Employment, and includes the issues of targeting, labour market analysis, innovation, transnationality, outputs, monitoring and evaluation, dissemination, multiplier effect, capacity, budget, and complementarity.

27. Regarding the emphasis on migrants and refugees:

in Spain the call for projects mentioned that for Integra, priority would be given to the fight against racism and xenophobia, language training for immigrants, etc., despite the fact that the relative group size at that time was much lower in Spain than in other countries;

the United Kingdom OP specified that refugees and migrants were a target group. The selection criteria (contained, for instance in its Guidance for Applicants for the Employment Initiative 1997 to 1999 (Part 2)) gave specific weighting to activities in favour of refugees.

EVALUATION

29. Reliable indicators were not available during this period. The only relevant benchmark was one which compares instruments, techniques and methodologies developed through the Initiative with those in existence beforehand.

30. The inception of the Initiative can be linked directly to the Commission's White Paper on Growth, Competitiveness and Employment (in particular Chapter 8), which provided a wide-ranging background to the economic and social challenges facing the EU in 1994. The specific initiatives undertaken by the Commission in the period immediately following the White Paper should be seen in the light of the ambitious attempt by the Commission to effect broad changes on the economic and social fabric of the EU.

31. It will not be possible to assess the cost and possible benefits arising from the Initiative in each Member State and the extent of participation by members of the target groups until the Commission has received and analysed the final programme reports from the Member States. The due date for receipt of these reports is 30 June 2002.

32. Evaluation was carried out during the programme on the basis of general guidance for ESF operations established by the Commission(13). The EU-wide evaluation published in March 2000, which was classed as a 'mid-term' evaluation, relied both on national evaluation reports discussed in joint meetings and on specific investigations such as case studies. In addition, an ex post evaluation of the Employment Community initiative will be carried out during 2002 and 2003, for which information covering the entire initiative will be accessible.

33. Precise information on the outcomes of the initiative in each Member State will not be known until the Commission has received the final programme reports from the Member States. The due date for receipt of these reports is 30 June 2002.

34. Because Employment was a single Community initiative, no specific evaluation by strand was carried out. The EU-wide mid-term evaluation of the Employment initiative, which was carried out in cooperation with the evaluators in each Member State, contains considerable information on the implementation of Integra(14). Regarding the coherence of the evaluation approach followed for Employment, please see point 33.

36. The "Conclusions of the ESF mid-term evaluations" pointed out that the Community Initiative should not be evaluated in strictly quantitative terms. Evaluations should be focused on qualitative issues, such as processes designed to reach new populations affected by exclusion. A number of Member States have developed monitoring instruments to collect data about the implementation of the project in terms of content, financing and transnationality(15).

37. The design of Employment was premised on the existence of many different problems facing many different groups and subgroups, and the need for a large number of varied proposals to solve these problems. The ex post evaluation of the Employment initiative, due to be published in 2003, should draw the attention of policy-makers and programme-designers to different models of best practice, as well as providing an independent assessment of their potential.

41. Provision was made for evaluation during and at the end of the programme. The Commission has established general guidance for ESF operations(16). Evaluators working at EU level have been brought in to support and supplement national evaluators. The EU-wide evaluation establishes methodological guidelines to support national evaluators and set up a common framework on types of analyses to conduct and information to gather. The EU-wide evaluators can rely on both national evaluation reports discussed in joint meetings and specific investigations such as case studies.

42. The evaluation reports on individual projects formed part of the information and dissemination structures of the Member States, and through them, at EU level. In many cases the results of the project evaluations were incorporated into national reports and publications which were placed in the public domain.

43. According to the objective of the initiative, there is no provision for projects to continue indefinitely. If a project develops a successful approach, then that approach should be mainstreamed into 'normal' labour market policy and practice. Keeping it within the CIs would represent a failure to mainstream and hence run counter to a key principle of the initiative.

However, Member States had a strategy for dealing with the lessons of projects, through the process of mainstreaming.

DISSEMINATION AND MAINSTREAMING

44 and 45. As part of its dissemination activities, the Commission produced guidance in 1997, entitled 'Impact Guide for Employment Project Promoters'. The Guide provides suggestions on how to plan a strategy to disseminate the successful outcomes of a project. (CE-V/7-97-003-XX-C, 1997, p. 32). This was followed in 1999 with a 'Guide to Public Relations Activities for Project Promoters', to enable projects in the Community Initiatives to realise their full potential through effective communication (1999, p. 28).

In addition, the national Technical Assistance Offices (TAOs) provided guidance on their own projects.

For instance:

- in the United Kingdom the TAO produced, in July 1997, Guidance Note 7: Dissemination, which provided detailed guidance on projects, covering the objectives, means and techniques of dissemination,

- in Spain brochures were produced and three seminars were organised at national level in July 1998, November 1998 and June 1999. Many more seminars and information meetings were organised at regional and local level. Specific support was given to networks dealing with the target groups (Red Europea de Gitanos y Viajeros, Red Repris, Red Labora Vita).

46. The Commission agrees with this observation, and has already included the requirement to establish such wide steering groups in the design of the current Equal Community initiative.

48. The Commission, through its Technical Assistance Office for the Employment Initiative, Europs, set up an internet site that contained considerable information on experiences acquired and lessons learnt in Integra. When Europs ceased to operate in 2000 the contents of the web site were transferred to the Commission, and are currently freely available on the site: http://europa.eu.int/comm/equal.

Many Member States, including, for example, Spain set up databases containing basic information on projects.

50. The Member States were closely associated with the EU level work of the Thematic Focus Groups (TFGs), which produced a series of guidelines in the 'Innovations' series. Of specific relevance to Integra were the following publications:

- employment and solidarity(17),

- forging strong local partnerships(18),

- towards employment for all(19). Other publications in the series, while not directly stemming from the work of Integra projects, contain valuable lessons for the target groups of Integra (see Annex 1).

51. Clearer information about the extent to which Integra has had an impact on labour market exclusion will come from the ex post evaluation of the Employment Community initiative, to be published in 2003, and from the Final Reports on the implementation of the initiative at national level, which should be sent to the Commission by 30 June 2002.

53. The Commission, in co-operation with the Member States, published "Transnationality works! If you work at it!", a detailed publication in which transnationality was explored from three key perspectives. This publication was printed in the five largest official languages and widely distributed to key actors, decision makers, national authorities, and projects. It is available freely from the Commission's internet site at:http://europa.eu.int/comm/equal (Catalogue information: Innovations No 8, CE-19-98-487-XX-C, 1999, p. 60).

54. The Commission is not in position to confirm or contradict the Court's assertion that no strategic plan for systematic organisation of transfer of good practice existed in the Member States visited.

However, in the United Kingdom at least, there was a systematic mainstreaming plan, drawn up by the Managing Authority and agreed by the Monitoring Committee (see Employment MC meeting, 29 April 1997). This plan was implemented through the Technical Assistance Office, continuously updated and progress was reported back to the MC as a standing item in all subsequent meetings. A similar plan was implemented in France.

55. The beneficial effects of the involvement of such actors (local authorities, employers, social partners, etc.) in project steering groups were noted by the Commission during the period of implementation of the Employment initiative. For this reason the systematic inclusion of such actors in the steering groups of projects was built into the Equal Community initiative (a successor to Employment).

OTHER ASPECTS

58. As a result of the requirement to involve local actors in the planning, implementation and evaluation of the measures (point 6(d) of the Guidelines), there was a logical tendency to concentrate activities in those areas where such relevant local actors were in existence.

In Spain, for example, the selection criteria for Integra specifically mention priority for action in disadvantaged urban zones. In order to reinforce the complementarity of the activities of the different initiatives undertaken in such areas, Spain organised a joint Integra-Urban seminar in Córdoba.

60 to 63. and 65 to 67. Experience of Adapt and Employment led the Commission to the same conclusions as the Court, and this issue has been addressed in the design of the current Equal Initiative.

64. The interests of subsidiarity tended towards an automatic deference towards the preferences of Member States in this matter. The selection of projects and approval of their budgets was a national prerogative, which was based largely upon a detailed analysis of their proposed work programmes.

CONCLUSIONS AND RECOMMENDATIONS

68. The Commission considers that Integra, and the Employment Community initiative as a whole, was ambitious, as the problems it faced warranted, but not over-ambitious. The objectives of the initiative were clear and relatively simple, as stated in the 1996 Communication to the Member States laying down guidelines for the initiative, and have paved the way for success in most of the projects carried out on the ground.

69. As regards pre-launch activities, the objectives of the Initiative were set out both in the Guidelines (in section I 'Objectives') and in the Structural Fund regulations. The overall context was provided in the Commission's White Paper on Growth, Competitiveness and Employment(20), as indicated in point 3 of the guidelines.

70. The Guidelines laid down by the Commission in its Communication to the Member States(21) were broad and general, and the Member States were encouraged to interpret them in accordance with their own circumstances and priorities as set out in their Operational Programmes. The Commission considers that the establishment of benchmarks relating to identifiable target groups could give an inaccurate indication of the success of a programme such as Integra where the target population is in a constant state of flux, and where effects on systems, and on factors impacting on labour market exclusion, are of greater relevance.

71. Both in Spain and in the United Kingdom the Operational Programmes did specify that refugees and migrants were a target group. In the United Kingdom the selection criteria (contained, for instance in its Guidance for Applicants for the Employment Initiative 1997 to 1999 (Part 2)) gave specific weighting to activities in favour of refugees.

72. The initial results of Integra, and the other strands of the Employment Community initiative, are available on the Commission's public internet site. These key lessons were classified according to the taxonomy of the European employment strategy to facilitate their use by Member States in the drawing up of their national action plans for employment.

In addition, the ex post evaluation of Employment, and the final programme reports (due by 30 June 2002), will provide a concise summary of the key lessons at national and EU level and the results achieved.

73. Employment was a single Community initiative and no specific evaluation by strand was carried out. The EU-wide mid-term evaluation establishes methodological guidelines to support national evaluators and set up a common framework on types of analyses to conduct and information-gathering. In addition, the ex post evaluation of the Employment initiative will provide detailed information on the outcomes of the whole initiative, including Integra.

74. The Commission concurs with this observation concerning dissemination and has built the requirement into the current Equal Community initiative (a successor to Employment).

75. (a) Both in Spain and in the United Kingdom, the selection criteria covered the issue of innovation and targeting of assistance towards projects dealing with people at risk of exclusion from the labour market.

(b) A reliable conclusion concerning the experience acquired and the follow-up to the lessons learnt cannot be made in advance of the receipt and analysis by the Commission of the final reports on the initiative.

(c) Monitoring took place in each Member State via the formally constituted Monitoring Committee, of which the Commission was a member. Dissemination took place widely at project, local, national and EU-wide level. Neither were used to create new models for combating exclusion from the labour market, but both were important elements in the process of ensuring that the lessons learned by projects were passed to the widest possible audience of decision-makers.

76. A reliable conclusion concerning the value and effectiveness of Integra cannot be made in advance of the receipt and analysis by the Commission of the final reports on the Employment Initiative, which are due by 30 June 2002, and of the ex post evaluation, to be published in 2003.

77. As regards the Court's recommendations, the Commission would point out that:

(a) the experience of the first few years of the European Employment Strategy demonstrated the value of a more thematic approach to employment policy. Discrimination and inequality are rife and take different forms. If we want to open up the labour market to everyone, we must embark on a vast experiment. The Commission feels that it is mainly up to local bodies to ensure that the projects selected are innovative in nature.

(b) it shares the Court's concern that reliable and standardised indicators should be available as quickly as possible; this forms part of the joint work currently being undertaken by the Member States. Qualitative evaluation throughout the project is the best guarantee that such experimental programmes have added value.

(c) the experience gained in Adapt and Employment was the first stage in closer and more intense cooperation between the Commission and the Member States, in particular through the "thematic focus groups" set up at European level. These positive features were taken fully into account in the design of the Equal initiative.

Annex 1

A set of flagship publications, produced towards the end of the Initiative, includes the following booklets of direct relevance to Integra:

- Crossing the Job threshold: the output of the Thematic Focus Group "Crossing the Job Threshold". This publication points to the need to target resources on those individuals who are excluded, or at risk of exclusion, from the labour market. It is primarily of interest to those institutions and persons that can act as mediators between such disadvantaged groupings and employers, such as public employment services; private employment and placement services; vocational training providers; non-governmental organisations (NGOs) representing the interests of disadvantaged groups. Such institutions cannot easily change or develop their methods and approaches unless there is a supportive policy environment and so this publication is also addressed to the social partners and decision-makers concerned with employment policies at European, national, regional and local level (CE-V/1-00-016-XX-C, 2000, p. 28),

- Empowerment: output of the Thematic Focus Group on "Empowerment". This Group has looked into the ways in which Employment-Integra projects pursued the objective of 'empowering' excluded people and groups, as a strategy for their integration into employment. It identified projects most representative of this approach and worked at both national and European level to explore the nature and impact of empowerment strategies for employment and inclusion. (CE-V/1-00-013-XX-C, 2000, p. 30),

- Integra - Routes to Integration and employment for the socially excluded. Integra aims to promote measures to improve the access to the labour market and employability of those who find themselves excluded from it. (CE-97-96-475-XX-C, 1997, p. 20)

- Shaping active employment policies: Synthesis of Adapt & Employment strategic lessons: Adapt and Employment projects have explored a vast range of innovative models of good practice to improve and support active employment policies. These can be directly related to the four pillars of the European Employment Strategy (EES) and the Member States' national action plans for employment (NAPs): employability; entrepreneurship; adaptability; equal opportunities (CE-V/1-00-020-XX-C, 2000, p. 16).

- Territorial Approaches: the key principles: The term 'territorial approaches' refers to an emerging system of local governance. This system enables regions, and areas within regions, to respond flexibly and innovatively to the challenges of an increasingly open global economy, while simultaneously encouraging participation and stakeholding by citizens (CE-V/1-00-001-XX-C, 2000, p. 36).

(1) Communication to the Member States laying down amended guidelines for operational programmes or global grants which Member States are invited to propose within the framework of a Community initiative on employment and development of human resources aimed at promoting employment growth mainly through the development of human resources (96/C 200/06, OJ C200, 10.7.1996, p. 13).

(2) The Community initiative Equal concerning transnational cooperation to promote new means of combating all forms of discrimination and inequalities in connection with the labour market, for the programming period 2000 to 2006 (2000/C 127/02, OJ C127, 5.5.2000).

(3) CE-V/7-97-003-XX-C, 1997, p. 32.

(4) As above, 1999, p. 28.

(5) Communication to the Member States laying down amended guidelines for operational programmes or global grants which Member States are invited to propose within the framework of a Community initiative on employment and development of human resources aimed at promoting employment growth mainly through the development of human resources (96/C 200/06, OJ C 200, 10.7.1996, p. 13).

(6) Growth, Competitiveness, Employment - The Challenges and Ways Forward into the 21st Century - White Paper. ISBN 92-826-7423-1, 1994, Chapter 8(f).

(7) The United Kingdom presented two separate operational programmes (OPs) under the Employment Community initiative - one for Great Britain and one for Northern Ireland. The examples given in the Commission's reply refer to the OP for Great Britain.

(8) Operational programme. The form on which Member States' plans for action under the Community initiatives were presented to the Commission.

(9) Common guide for monitoring and evaluation, EC Structural Funds, European Commission, 1995 and, A methodology for the European evaluation of the Employment initiative, 1999.

(10) Op. cit. (see footnote 1 - document of 1996).

(11) "Target groups will continue to include those disadvantaged groups previously covered by Employment-Horizon, such as the long-term unemployed, jobless single parents, the homeless, itinerants, gypsies and travellers, prisoners and ex-prisoners, substance abusers, etc. However, a particular emphasis should be given to action that focuses on the special needs of migrants, refugees and other similarly vulnerable groups."

The definition of target groups under Employment-Horizon, referred to in the paragraph above, included "drug addicts, marginalised persons, immigrants, refugees, itinerants, isolated people, single-parent families, the homeless, prisoners and ex-prisoners." (OJ C 180, 1.7.1994, p.36).

(12) At http://europa.eu.int/comm/equal/ae.cfm Strategic Lessons.

(13) As previously cited, Common guide for monitoring and evaluation, EC Structural Funds, European Commission, 1995 and, A methodology for the European evaluation of the Employment initiative, 1999.

(14) Cited above.

(15) "Final EU-wide evaluation of Community Initiative Employment", March 2000, p. 107.

(16) Cited above, Common guide for monitoring and evaluation, EC Structural Funds, European Commission, 1995, and, A methodology for the European evaluation of the Employment initiative, 1999.

(17) Innovations No 7, CE-11-98-932-XX-C, 1998 p. 36.

(18) Innovations n°1, CE-11-98-924-XX-C, 1998 p. 28.

(19) Innovations No 9, CE-26-99-780-XX-C, 2000, p. 40.

(20) ISBN 92-826-7423-1.

(21) As previously cited, Communication to the Member States (OJ C 200, 10.7.1996, p. 19).