Special Report No 3/2002 concerning the Community Initiative "Employment-Integra" together with the Commission's replies
Date: 2002-07-18
Institutions: European Court of Auditors
Subjects: EU initiative, European Employment Strategy, aid to disadvantaged groups, employment policy, labour market, marginalisation

29.10.2002

EN

Official Journal of the European Communities

C 263/1

I

(Information)

COURT OF AUDITORS

SPECIAL REPORT No 3/2002

concerning the Community Initiative ‘Employment-Integra’ together with the Commission’s
replies

(pursuant to Article 248(4), second subparagraph, of the EC Treaty)

(2002/C 263/01)

CONTENTS

Paragraph

Page

SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

I-XIII

INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Employment-Integra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Audit objective and approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

BUDGET IMPLEMENTATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

PRE-LAUNCH ACTIVITIES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

General

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Commission level

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Member State level

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

SELECTION AND IMPLEMENTATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

General

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Commission level

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Member State level

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

EVALUATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

General

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Commission level

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Member State level

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Project level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

DISSEMINATION AND MAINSTREAMING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Dissemination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Mainstreaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

OTHER ASPECTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Unemployment rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Distribution of funds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Transnationality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

CONCLUSIONS AND RECOMMENDATIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1-8

1-5

6-8

9-10

11-15

11

12-13

14-15

16-27

16-18

19-21

22-27

28-43

28-31

32-34

35-37

38-43

44-55

44-51

52-55

56-67

56

57-58

59-67

68-77

The Commission’s replies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2

3

3

4

5

5

5

5

5

5

5

7

7

7

7

8

8

9

9

9

10

10

10

10

10

11

13

C 263/2

EN

Official Journal of the European Communities

29.10.2002

SUMMARY

I.
‘Integra’ was launched in 1996 as a new, fourth strand of the Community initiative (CI) ‘Employment
and development of human resources’ (1) for the programming period 1997 to 1999. From 1994 to 1997
Integra was an integral part of ‘Horizon’, but the importance attached by the European Parliament and the
Council to the subject of people excluded from the labour market resulted in recognising this as a separate
issue. A Communication dated 10 July 1996 presented to the Member States by the Commission, stated
Integra’s objective as being ‘to promote measures to improve the access to the labour market and the employ-
ability of vulnerable groups who find themselves excluded from it, or at risk of being excluded from it.’

II.
Although target groups would continue to include those covered by Employment–Horizon (2), the
Communication specified that particular emphasis should be given to actions that focus on the special needs
of migrants, refugees and other similarly vulnerable groups likely to face greater discrimination in the labour
market, due to the rise in racism and xenophobia in Europe (3).

With 418 million euro of appropriations made available the Member States initially selected approxi-

III.
mately a thousand projects for implementation during the 1997 to 1999 period.

IV.

The overall aim of the Court’s audit was to ascertain whether:

— the selection procedures for choosing promoters and projects ensured that projects were innovative and

targeted people most at risk of exclusion from the labour market,

— the experience acquired and lessons learnt were evaluated and followed up in a systematic, coherent and

comparable way,

— monitoring and dissemination were performed effectively and used in creating new models for combat-

ing exclusion from the labour market.

V.
The Commission did not explicitly provide definitions of the key words used in the guidelines nor of
the population of each target group and did not establish a priorities/problems list. At neither Commission
nor Member State level were any benchmarks or baseline criteria developed and it is therefore difficult to assess
when and how projects were innovative and likely to give added value or to improve or change existing
approaches.

VI.
There is no evidence that the funds have been directed towards specific concentrations of the target
groups, such as disadvantaged urban areas so that the distribution of Integra funds was the most efficient way
of gathering knowledge about the target group and how to include them in the labour market.

VII.
At the end of the programme the transnational aspect was seen as a positive and important factor by
most promoters, but the audit revealed inconsistencies in the development phase, the selection phase and the
funding policy. This had an adverse effect on the added value of the projects.

(1) This CI was launched on 15 June 1994, in the context of Article 11 of Council Regulation (EEC) No 4253/88 as modi-
fied by Council Regulation (EEC) No 2082/93 (OJ L 193, 31.7.1993, p. 20). Originally it had three strands: Now,
Youthstart and Horizon.

(2) These included ‘long-term unemployed, jobless single parents, the homeless, itinerants, gypsies and travellers, prisoners

and ex-prisoners, substance abusers, etc.’ Communication to the Member States (OJ C 200, 10.7.1996, p.13).

(3) Communication to the Member States (OJ C 200, 10.7.1996, p. 19).

29.10.2002

EN

Official Journal of the European Communities

C 263/3

VIII.
ent and comparable way, neither at Commission level nor at that of the Member States.

The experience acquired and lessons learnt were not evaluated and followed up in a systematic, coher-

IX.
The Commission and the Member States did not consider using indicators to describe the basic situ-
ations of the various target groups included in the initiative. Had they done so, they could then have formu-
lated clear, unified and measurable criteria and created a baseline against which to evaluate the Integra initia-
tive.

No evidence was found to prove that dissemination was performed systematically and effectively and

X.
used in creating new models for combating exclusion from the labour market for each target group.

For none of the Member States visited was evidence found either at the national or transnational level
XI.
of the organisation of systematic transfer of good practice. At promoter and local/regional level there was more
evidence of such transfers. However, there was no systematic organisation of it, and promoter contact and
networking with local authorities and other promoters appear to have been the most important factors.

XII.
The Integra initiative has raised public awareness of excluded groups. Nevertheless, it is not possible
to draw any firm conclusion on the extent to which the Integra initiative has resulted in new knowledge about
how to promote measures to combat exclusion from the labour market in the different target groups and
whether that knowledge has been effectively disseminated to political decision-makers.

XIII.

The Court recommends:

— before launching innovative programmes such as Integra the Commission and Member States should
identify the main areas (target groups and labour market segments) where innovative projects could con-
tribute to the formulation of new policies,

— indicators should be developed at the beginning of such programmes to allow a continuous and system-
atic evaluation of the innovative projects so as to assess and compare the results and to make sure that
the programmes will have an added value and are an improvement compared to other existing pro-
grammes,

— the Commission should develop a strategic plan for systematic transnational transfer of lessons learnt and
good practice to social partners and policy-makers in order to make sure that the results are used in
relevant policy implementation.

INTRODUCTION

Employment-Integra

market. From 1994 to 1997 Integra was an integral part of
Employment-Horizon (see footnote 1 on page 2), but the impor-
tance attached by the European Parliament and the Council to the
subject of excluded people resulted in recognising this as a sepa-
rate issue.

The Commission launched the Community initiative
1.
‘Employment and development of human resources’ in 1994 (1).
The aim was to contribute to the development of human resources
and improve labour market functioning, in order to enhance
employment growth and promote both social solidarity in the
European Union and equal opportunities for women in the labour

(1) Article 11 of Council Regulation (EEC) No 4253/88 as modified by
Council Regulation (EEC) No 2082/93 (OJ L 193, 31.7.1993, p. 20).

2.
In a Communication (2) to the Member States in 1996, the
Commission added a new strand, Employment-Integra to the

(2) Communication to the Member States laying down amended guide-
lines for operational programmes or global grants which Member
States are invited to propose within the framework of a Community
initiative on employment and development of human resources aimed
at promoting employment growth mainly through the development
of human resources (OJ C 200, 10.7.1996, p.13).

C 263/4

EN

Official Journal of the European Communities

29.10.2002

abovementioned Community initiative, for the programming
period 1996 to 1999. The ultimate goal of this new strand was
‘to promote measures to improve the access to the labour market
and the employability of vulnerable groups who find themselves
excluded from it, or at risk of being excluded from it’ (1). The tar-
get groups included the long-term unemployed, jobless single par-
ents,
the homeless, gypsies and travellers, prisoners and
ex-prisoners, substance abusers, etc., already supported by the
existing strand Employment-Horizon. However, Employment-
Integra was to place the accent on the special needs of migrants,
refugees and other similarly vulnerable groups who were more
likely to be victims of discrimination in the labour market, because
of such phenomena as racism and xenophobia.

3.
The ESF Community Initiatives are the special financial
instruments of structural policy which the Commission itself pro-
poses to the Member States to support measures which will help
solve problems having a particular impact at European level. These
initiatives have the following key features which give them added
value as compared with the other measures financed by the Social
Fund:

Table 1

Budget implementation by Member State

Member States

Commitments

Payments

Outstanding

(Mio EUR)

Spain
Italy
France
United Kingdom
Germany
Ireland
Greece
Belgium
Netherlands
Portugal
Finland
Austria
Sweden
Denmark
Luxembourg
Total

105,0
82,7
51,2
49,6
41,0
18,8
16,0
13,2
11,9
9,8
6,7
4,9
4,2
2,9
0,0
417,9

83,5
67,1
27,4
43,8
38,0
17,4
12,1
10,6
6,3
8,8
5,3
1,7
3,3
2,3
0,0
327,6

21,5
15,6
23,8
5,8
3,0
1,4
3,9
2,6
5,6
1,0
1,4
3,2
0,9
0,6
0,0
90,3

Source: Commission of European Communities, December 2001.

(a)

transnationality;

(b)

innovation;

(c) bottom-up approach;

(d) dissemination;

(e) mainstreaming.

4.
With 418 million euro of appropriations made available
(Table 1) the Member States initially selected approximately a
thousand projects for implementation during the 1997 to 1999
period.

5.
Integra, as a strand of the Community initiative ‘Employ-
ment’, is supposed to have a catalystic effect on innovation at
Community level and at the same time encourage the organised
transfer of experience and the diffusion of good practice among
promoters and Member States. Its impact should therefore not be
measured in terms of employment creation but in terms of new
knowledge about innovative methods of improving the employ-
ability of the target groups.

Audit objective and approach

6.
The overall aim of the audit was to investigate whether the
Community initiative Employment-Integra provided the Commis-
sion and Member States with new knowledge and new methods
to promote measures to combat exclusion from the labour mar-
ket for the most vulnerable groups in society.

7.

The main audit objectives were to ascertain:

(a) whether the selection procedures for choosing promoters (2)
and projects ensured that projects were innovative and tar-
geted people most at risk of exclusion from the labour mar-
ket and that they presented a real transnational character;

(b) whether the experience acquired and the lessons learnt were
evaluated and followed up in a systematic, coherent and com-
parable way;

(c) whether monitoring and dissemination were carried out effec-
tively and used in creating new models for combating exclu-
sion from the labour market.

(1) Paragraph 4(d) of the Commission Communication 96/C 200/06

(OJ C 200, 10.7.1996).

(2) Public or private person or organisation which requests and possibly
obtains assistance in the framework of an intervention for a given
project.

29.10.2002

EN

Official Journal of the European Communities

C 263/5

The audit approach consisted of examination of the sys-
8.
tems implemented on the basis of documentation review and
interviews with key staff at Commission level and in four Member
States (Denmark, Spain, France and the United Kingdom) where
16 projects were audited in the context of the three audit objec-
tives mentioned above. The expenditure of these projects consists
to a large extent of the salary cost of staff involved in the reso-
cialising and training of the participants and of allowances paid
to the latter.

BUDGET IMPLEMENTATION

9.
Total commitments made available for the Integra initia-
tive were 418 million euro. This was an increase of 8 % over the
385 million euro initially allocated due principally to the fact that
Member States transferred funds from other CI Employment
strands to Integra in order to satisfy requests for funding addi-
tional Integra projects.

10.
By 31 December 2001 the Commission had paid 327
million euro to the Member States representing 78 % of the total
commitment. The final closure of Integra is due to take place in
2003 together with the other strands of CI Employment after the
Commission has received the required documents as foreseen by
Regulation (EC) No 2064/97.

PRE-LAUNCH ACTIVITIES

General

11.
Launching an innovative programme such as Integra
requires shared understanding, between Commission, Member
States and promoters of the theoretical concepts and intentions
behind the programme, so that misunderstandings at the opera-
tional level are kept to a minimum. For successful implementa-
tion in the Member States it is essential for the pre-launch mate-
rial to identify the key concepts and provide a clear definition of
them, to establish clear objectives, clear criteria, and clear moni-
toring procedures and use statistical information or indicators
about the target groups.

Commission level

(c)

identify clear objectives for each target group in Integra;

(d) provide indicators for monitoring the selection of Integra

projects;

(e) use statistical material or indicators in the selection of target

groups;

(f)

establish benchmarks or baselines for the target groups or
make use of previous experience of traditional ESF pro-
grammes to facilitate evaluation of projects.

The Commission Communication places particular
13.
emphasis on migrants, refugees and people in similar situations
as a target group, but makes no distinction between the various
categories within this very diverse group. As a result, essentially
different categories, requiring totally different approaches, like
refugees with no education at all and highly skilled refugees, are
treated under the same heading, which makes evaluation extremely
difficult.

Member State level

14.
No list of problems or priorities was drawn up, for the
potential target groups, prior to the distribution to potential pro-
moters of information material on Integra. In this context little
use was found to be made in the Member States visited of relevant
statistical material or indicators beforehand in order to analyse
the situation of each potential target group.

15.
All the Member States visited had drawn up detailed
documentation on Integra, which was sent to potential project
tenderers or could be obtained on request. However, none of the
Member States visited gave a clear definition of key words used in
their pre-launch material. They did not develop benchmarks and
baseline criteria for use by potential promoters as a means of dis-
tinguishing between ‘experimental’ projects from the traditional,
mainstream ones. Thus it was left to individual promoters to
interpret and understand key concepts of the programme (see
paragraph 3) and to assess for themselves ‘added value’ and the
‘improvement of existing approaches’.

SELECTION AND IMPLEMENTATION

General

The Commission’s Communication to the Member States

12.
does not explicitly:

(a) offer any definition of when a project is ‘innovative’ or iden-
tify what the innovation element in a project should consist
of;

(b) define the population of each target group in Integra;

16.
At European level the Commission sets out the global
framework within which the initiative has to be implemented at
national level (see Table 2 – Selection of Projects). In line with the
Communication to the Member States point 13, a technical assis-
tance office (Europs) was established as an European support
structure to assist the Commission. Europs established and main-
tained a Europe-wide database on projects and partnerships.

C 263/6

EN

Official Journal of the European Communities

29.10.2002

Table 2

Selection of projects

Stages

Steps

Responsibility of

Planning

Pre-Launch Activities: Awareness
Raising and Information Giving

Member State & European
Commission

Call for proposals

January to 31 March ’97

Project Promoters develop and
submit their project ideas

Applicant and National Support
Structures (NSS)

Assessment

1 April to 31 July ’97

Check that proposal is eligible
for funding

Member State

.

.

Approval for selection of projects

Monitoring Committee & Member
State

.

Development

August ’97 onwards

Transnational partnerships are
formed and work programmes
agreed by partners

Applicant, NSS & EUROPS

.

Final Confirmation

31 December ’97 latest

Approval of Partnership and
start of activities

Member State

Source: Commission European Social Fund — Employment Integra — SR — February 1997.

29.10.2002

EN

Official Journal of the European Communities

C 263/7

At Member State level several national organisations are
17.
involved. The national monitoring committee, which is usually
made up of representatives of a wide range of national authori-
ties and also includes a representative from the Commission,
monitors and evaluates the implementation of the initiative.
national support structures (NSS) have been established to offer
technical assistance at national level and to provide advice and
support to individual projects, collecting and transmitting infor-
mation.

18.
In order to ensure that the objectives and priorities are
achieved, active implementation should be monitored following
a common methodology. This would facilitate monitoring and
evaluation and putting the knowledge and experience acquired to
effective use in the creation of new models for combating exclu-
sion from the labour market. Especially where management of an
initiative is decentralised, one of the prerequisites for its success
is that the various entities responsible for implementing the pro-
gramme follow clearly defined reporting guidelines.

Commission level

19.
The lack of definition of each target group means that
information from the Member States on various target groups
cannot be used to monitor the effects of Integra on a Community
scale. In the absence of guidelines for the drafting of the annual
implementation reports by the Member States authorities and in
the absence of a harmonised classification system, which caused
considerable variations in the quality of information, it was prac-
tically impossible to have a systematic Community-wide overview
of the implementation of Integra.

20.
The contract between the provider of the technical assis-
tance and the Commission ended in April 2000 and Europs had
thereafter three months to close down. The database developed
by Europs was handed over to the Commission. However, it was
not maintained or updated by the Commission and because many
of the projects had not been completed when Europs closed (1),
the knowledge and experience acquired at project level could not
be utilised.

Member State level

22.
In line with the Commission Communication each Mem-
ber State set up a single monitoring committee which was respon-
sible for the Employment initiative as a whole and, therefore, also
responsible for the final selection of the projects.

23.
The Member States were aware of the risk that new pro-
moters with potentially good innovative ideas, but little knowl-
edge of or familiarity with the Structural Funds, could face dif-
ficulties in making the application correctly, and some counselling
was given to project promoters in that respect.

24.
In the interest of impartiality this pre-selection phase
(administration, counselling and first screening of applications)
was separate from the final selection of promoters. The Member
States visited had different selection criteria and methods, but a
careful examination of the applications took place, and even
though the vast majority of the selected promoters were familiar
with and had participated in other ESF programmes new promot-
ers were also selected to participate in Integra. This is a positive
aspect in the light of the fact that filling out the application form
can be rather difficult and time consuming, especially for small
and new promoters.

25.
In all the Member States visited it proved very difficult to
assess the innovative content of the selected projects, and in many
cases it was difficult for the competent authorities to explain why
a selected project had innovative aspects and was not just a dif-
ferent presentation of approaches that were already part of exist-
ing mainstream projects funded under the European Social Fund.

26.
Due to the lack of clear priorities the most important
indicator in the selection of project tenders and projects was a
quality assessment of the project tenderer and of the application
itself. It was found that the project’s target group, the comple-
mentarity of projects or other indicators were not considered.

27.
In none of the Member States visited did the selection
procedures as implemented give particular emphasis to actions
that focused on the special needs of migrants and refugees, as
required in the Commission Communication.

In conclusion, the Commission did not develop a moni-
21.
toring system to ensure that all necessary information was gath-
ered and filed in a harmonised system and used effectively in cre-
ating new models for combating exclusion from the labour market.

EVALUATION

General

(1) See Special Report No 3/2000 on the European Social Fund and the
European Agricultural Guidance and Guarantee Fund (Guidance sec-
tion), Measures to assist the employment of young persons, points 59
and 60 (OJ C 100, 7.4.2000).

28.
According to the Commission’s communication, ‘During
and at the end of the planning period the Commission shall
evaluate, in partnership with the Member States, the results of the

C 263/8

EN

Official Journal of the European Communities

29.10.2002

programmes submitted. In order to carry out this evaluation the
Commission will use the objectives specified by Member States as
the main benchmark against which to assess progress. As a func-
tion of the desired objectives and the implemented measures this
evaluation will provide data related to the target groups includ-
ing the final beneficiaries.’

finished before the end of the programme. Because of the late
involvement of evaluators and the late start in planning the evalu-
ation process, the evaluators were limited to carrying out the
evaluation on the basis of information available at the time or by
means of case studies in selected Member States.

29.
Neither the Commission nor the Member States used
indicators to describe the basic situations of the various target
groups included in the initiative. Had they done so, the indicators
could have been used to formulate clear, unified and measurable
criteria and to create a baseline against which to measure and
evaluate the innovative aspects of the Integra initiative.

This evaluation found that on the basis of the available
33.
data on labour market outcomes, it is not possible to draw any
conclusions for the initiative as a whole as regards target groups’
access to job opportunities. It was further stated that the ultimate
goal of the initiative is to improve the representation of target
groups in mainstream programmes and their job placement in
the labour market, but that without information on labour mar-
ket outcomes it is difficult to assess the added value of the initia-
tive.

30.
In experimental Initiatives like Employment-Integra the
concepts of success and failure are equally important. Every project
sets out to test a hypothesis, and ‘failure’ in this regard is as valu-
able a lesson for the future formulation of labour market policy
as ‘success’. The Commission did not initiate a specific ex ante
evaluation for the Integra initiative and its target groups and there
was no indication by the Commission, nor by the Member States
visited, of how to measure if and when a project or concept was
successful or unsuccessful. As a consequence, lessons learnt and
experience acquired at project level in different countries were not
monitored in a coherent way and at Commission level it was not
possible to evaluate whether concepts were good or bad.

34.
A specific evaluation of Integra, in which the experience,
knowledge and lessons learnt from innovative projects in all the
Member States were evaluated and followed up in a systematic,
coherent and comparable way, did not take place.

Member State level

None of the Member States visited, nor the Commission,
31.
carried out any economic analysis or cost benefit analysis of the
economic efficiency of the different innovative projects. Value-for-
money indicators of the successful inclusion in the labour market
of one individual from a target group were not developed and an
ex post evaluation was, therefore, not possible.

35.
In general the Member States visited had initiated several
evaluation projects at different levels, such as national or interna-
tional conferences and comprehensive reports on the Employ-
ment programme as a whole. However, similarly to the study ini-
tiated by the Commission, the Member States’ evaluation reports
mainly deal with the Employment initiative as one programme
and not as several strands, each of which targets different vulner-
able groups.

Commission level

32.
An evaluation (1) of the whole Employment CI was car-
ried out from August 1998 to January 2000, which means that
the evaluation started after the launch of the programme and

(1) On behalf of the Commission, Forschungsstelle für Berufsbildung,
Arbeitsmarkt und Evaluation (FBAE) at FHVR Berlin and the Nether-
lands Economic Institute (NEI), Rotterdam developed ‘A methodol-
ogy for the European evaluation of the Employment Initiative’. This
study was the basis for the subsequent evaluation of the Employment
programme, including Integra, and on behalf of the Commission
FBAE and NEI published a ‘European-wide Evaluation of the Com-
munity Initiative Employment’ in March 2000.

36.
For lack of a basic strategy most of the evaluations took
the traditional form of a general, quantitative description and did
not take into account the fact that Integra was targeting vulnerable/
disadvantaged groups with different needs. The reports did not
evaluate the experience acquired and lessons learnt from all the
different projects dealing with the same target group. None of the
Member States visited had produced reports or guidelines with
‘best practice’ for the inclusion into the labour market of the dif-
ferent target groups covered by the Integra programme.

37.
In the Court’s view, the evaluation of the Integra pro-
gramme should result in drawing up a set of models showing
how projects for the various target groups should be designed in
order to offer the maximum chance that these ‘marginalised
groups’ would be successfully integrated into the labour market.

29.10.2002

EN

Official Journal of the European Communities

C 263/9

Project level

DISSEMINATION AND MAINSTREAMING

In March 1997, Europs, acting on behalf of the Commis-
38.
sion, produced a ‘Guide to self-evaluation for Employment projects’
which could be sent to projects by the NSS (see point 17) on the
national monitoring committees’ behalf. It was designed as an aid
for promoters who were not used to making evaluations, rather
than as a compulsory evaluation tool.

39.
In all the Member States visited, the NSS had set up some
evaluation tools to help the promoters in the evaluation of their
projects. NSS paid personal visits to promoters and/or asked for
periodic progress reports. It was observed that, in general, coop-
eration between promoters and NSS during the programme had
been satisfactory and the promoters that were interviewed dur-
ing the audit saw the evaluation as an important part of the
project.

40.
Apart from evaluating their own projects, the promoters
involved in transnational networks had also participated in the
evaluation of the network. As part of the latter evaluation, accounts
of the experience acquired in the course of the transnational
cooperation were collected and included in transnational evalua-
tion reports.

Nevertheless, evaluations were still not synchronised and
41.
harmonised at project level, because the necessary monitoring
instruments and measurable indicators were not available to pro-
moters and the competent authorities did not develop an evalu-
ation strategy before launching the programme.

The promoters visited were asked whether there had been
42.
any follow-up to their evaluation reports. None of them had
received any feedback as regards the results of the innovative
approach they had used in the projects, and none of them knew
whether the project approach that had been developed was
adopted (or rejected) by competent authorities or in other regions.

43.
One consequence of an experimental programme like
Integra is that there is no strategy to continue projects which,
according to their evaluation reports, had demonstrably devel-
oped a successful approach. In some cases it had been possible for
successful promoters to continue, because local authorities and/or
labour market organisations were interested in the continuation
of the project, for others it was not possible to continue with
projects they deemed successful. In several cases promoters had
been obliged to close down the activity and experienced staff had
left for other work.

Dissemination

44.
In fulfilling the objectives of Integra (see point 2), details
of experiences, lessons learnt and best practice need to be dis-
seminated from project, to local, to regional, to national and to
Commission level, and from Commission level back to all Mem-
ber States.

45.
Many of the project promoters visited contacted local
newspapers, labour market organisations and local authorities
during and at the end of the project. But dissemination outside
the local area was rare and often depended on the promoter’s
own network.

46.
Judging by the example of the projects visited, there
appeared to be a greater chance of information being dissemi-
nated successfully if the promoter had, at an early stage, such as
the project-planning phase, set up a ‘project-steering group’ con-
sisting of representatives from the local municipality, labour mar-
ket organisation or other interested parties.

47.
These projects also appeared to be highly effective in
terms of raising the profile of vulnerable groups in the labour
market, possibly because the projects gave the labour market
organisations insight into the problems and special needs of the
target group, and at the same time made them aware of the
potential of the ‘marginalised groups’ as members of the work-
force.

48.
None of the Member States visited had, at the time of the
visit, developed an easily accessible database where potentially
interested promoters could find information on the experience
acquired, lessons learnt, etc., on projects dealing with a specific
target group.

At project level a variety of information material such as
49.
reports, leaflets, CD-ROM’s, videos had been produced and local
media, including radio and newspapers, had been used to dissemi-
nate project results. In the areas which had Integra projects, these
appear to have raised public awareness of the target groups and
contributed to their vocational integration.

Member States have produced material on the Integra
50.
programme, with information on some or all of the projects
included in the programme, but none of the Member States vis-
ited had produced a code of best practice on how to integrate the
different target groups into the labour market, despite most of the
projects visited having been successful.

C 263/10

EN

Official Journal of the European Communities

29.10.2002

On the basis of the information available at Commission
51.
level and from the Member States visited, it is not possible to draw
any firm conclusion on the extent to which Integra has resulted
in new knowledge about combating exclusion from the labour
market in the different target groups and whether that knowledge
has been effectively disseminated to those responsible for politi-
cal decisions.

organisations and municipalities in looking at the target group as
a potential workforce. In areas with high unemployment rates it
appeared to be more difficult to accept the target group as a
potential workforce. This factor makes comparative analysis of
performance between areas with different unemployment rates
very difficult.

Mainstreaming

Distribution of funds

52.
Before mainstreaming is possible on a broader European
level it is necessary to make an assessment of the innovative
potential of project results for each target group, draw lessons and
develop best practice, and make a selection of the most promis-
ing approaches for mainstreaming. Some of this was done by
Thematic Focus Groups but no evidence was found to prove that
transnational mainstreaming has taken place.

The Communication states that the Employment initia-
57.
tive ‘.…is applicable to the whole of the territory of the Union.
However, particular emphasis will to be placed on the needs of
the less-favoured regions. A number of measures will therefore
apply only to priority regions (Objective 1, 2, 5b and 6)’. On the
other hand, for the measures under the Employment-Integra strand
it is stated specifically that ‘greater emphasis should (…) be placed
on actions targeted at disadvantaged urban areas’.

53.
The establishment of transnational partnerships and net-
works might have had a transnational effect for projects in the
partnership, but this was not treated in a systematic manner in
the process of evaluation and mainstreaming.

58.
There is no evidence that the funds were directed in any
particular way to urban or other concentrations of the target
groups, so that the actual distribution of Integra funds was the
most efficient way of gathering knowledge about the target group
and how to include it in the labour market.

No effective system of transfer of good practice was found
54.
to be working in any of the Member States visited. The evaluation
procedure was not working in a way that made it possible to draw
conclusions and select best practice for a target group and there
was no structured system to link the Integra results for the differ-
ent target groups to the national policy-making processes.

Transnationality

There was more evidence of such transfers at the pro-
55.
moter and local/regional levels, especially in the cases where the
project-steering group consisted of actors from the municipality
and labour market organisations. However, no systematic organi-
sation of it took place, and promoter contacts and networking
with local authorities and other promoters appear to be the most
important factor.

OTHER ASPECTS

Unemployment rate

56.
The projects visited provided some evidence that main-
streaming or continuation of projects and empowerment of the
target groups were also linked to the unemployment rate in the
geographical area. In some areas with low rates of unemployment
there was a greater interest and involvement from labour market

59.
The transnational partnership has been an in-built dimen-
sion of the Community initiatives since they were launched and
in Integra, as well, projects are not funded unless brought together
in transnational partnership with projects from other Member
States. The objective is to transfer expertise, knowledge and expe-
rience and at the same time provide a tool for developing or test-
ing innovative ideas which cannot be realised in a national con-
text alone.

60.
Through the databases established by Europs and the
support structures in the Member States, it was possible for inter-
ested promoters to find and establish contact with potential tran-
snational partners.

There was some inconsistency in the programme seeking
61.
to involve new promoters with experimental, innovative projects:
but at the same time requiring them to have a transnational part-
ner before an application could be approved. A ‘new promoter
with experimental projects’ might withhold his application because
of this requirement.

Another inconsistency was that if a promoter had found
62.
a transnational partner and developed a common project with

29.10.2002

EN

Official Journal of the European Communities

C 263/11

him before submitting the application, neither partner could be
sure that his part of the project would be approved by his national
committee. Even if approval was given, there was no guarantee
that the transnational partners would receive their aid at the same
time, so that the projects could be implemented in parallel. None
of the Member States visited had addressed these problems, e.g.
by establishing cooperation between the committees in the Mem-
ber States as regards the selection procedure and the implementa-
tion of the programme.

Neither the Commission nor the Member States estab-
63.
lished guidelines on the amounts to be made available for tran-
snational cooperation. This led to large differences in the amounts
available for project promoters to implement the transnational
partnership both within a Member State and between Member
States. One consequence of this was that projects working together
in a partnership did not all have the same amount of funds for use
in the cooperation, and were thus not able to participate on equal
terms (1).

This led to a situation whereby one Member State funded
64.
projects by means of a fixed amount per participant, regardless of
the target groups’ needs, and another made a quality assessment
of the application, accepted that the transnational element was an
important factor and financed visits to partners abroad for both
trainers and participants. Thus, an inequality of treatment was
created from the outset.

65.
Some promoters visited had had no previous experience
of transnational co-operation. Their initial expectations were very
low: they saw the transnational element as time-consuming and
an extra administrative burden, and only used the partnership for
exchange of information and experiences. At the end of the project
they still saw the transnational element as very time-consuming,
but also viewed it as a positive and important factor for the project
and one which they would like to develop in future projects.
Amongst other things, receiving information about how other
countries were dealing with problems had given them a greater
awareness of alternative approaches to the target group.

objectives for the partnership. Their involvement in and expecta-
tions of the transnational partnership started on a different level
from the new promoters. But the absence of clear models, poli-
cies and procedures at central level led to critical comments on
their part and point to a need for a more active role of and closer
cooperation between Member States.

In general the transnational element was not a corner-
67.
stone of the approved projects, and was often mentioned as an
additional activity, without a direct link to the primary purpose
of the project.

CONCLUSIONS AND RECOMMENDATIONS

68.
The Integra initiative, which was introduced in response
to a rising awareness and concern about the access to the labour
market and the employability of vulnerable groups, has generally
been found overambitious, because it had too many objectives
and was insufficiently focussed on the target groups due to a lack
of overall guidance and coordination.

69.
The pre-launch activities lacked the basic requirements
for securing a unified understanding of the objectives and applica-
tion of harmonised methodology among the Member States (see
points 12 to 15).

70.
Neither the Commission nor the Member States used
indicators to describe the basic situations of the various target
groups included in the initiative. Had they done so, the indicators
could have been used to formulate clear, unified, identifiable and
measurable criteria and to create a baseline against which to mea-
sure and evaluate the effect of the Integra initiative (see points 12
and 15).

71.
In none of the Member States visited did the selection
procedures as they were implemented, give particular emphasis to
actions that focused on the special needs of migrants and refugees
(see point 27).

66.
Other promoters, often with experience from previous
transnational partnerships, were involved at an early stage in net-
works comprising projects from several countries and had clear

72.
The monitoring procedures set up by the Commission
and in the Member States did not ensure that all necessary and
adequate information was gathered and filed in a harmonised fil-
ing system which could subsequently be used for evaluation and
for analysis (see points 19 to 21).

(1) See Special Reports No 3/2000 on the European Social Fund and the
European Agricultural Guidance and Guarantee Fund (Guidance sec-
tion), Measures to assist the employment of young persons, point
51(c) (OJ C 100, 7.4.2000) and No 22/1998 concerning the manage-
ment by the Commission of the implementation of measures to pro-
mote equal opportunities for women and men (OJ C 393, 16.12.1998).

73.
Had there been a specific evaluation of Integra, the expe-
rience, knowledge and know-how obtained from innovative
projects in all the Member States could have been followed up in
a systematic, coherent and comparable way, but the evaluation
did not take place (see points 28 to 43).

C 263/12

EN

Official Journal of the European Communities

29.10.2002

Dissemination of good practice is likely to be influenced
74.
positively by setting up a local steering group at an early project
stage (see point 46).

the most vulnerable groups in society (see points 31and 56 to
58).

On the basis of the audit observations it is not possible

75.
to conclude that:

77.

The Court recommends:

(a)

(b)

the selection procedures applied for choosing promoters and
projects ensured that projects were innovative and targeted
people most at risk of exclusion from the labour market;

the experience acquired and the lessons learnt were evaluated
and followed up in a systematic, coherent and comparable
way;

(c) monitoring and dissemination were carried out effectively
and used in creating new models for combating exclusion
from the labour market.

(a) before launching, in response to very generally formulated
political incitations, innovative programmes such as Integra,
the Commission and Member States should precisely identify
the main areas (target groups and labour market segments)
where innovative projects could contribute to the formula-
tion of new policies;

(b)

indicators should be developed at the beginning of such pro-
grammes to allow a continuous and systematic evaluation (ex
ante, mid-term and expost) of the innovative projects so as to
assess and compare the results and to make sure that the
programmes have an added value and are an improvement
compared to other existing programmes;

76.
Furthermore, it cannot be concluded that the design of
the Integra programme was the most cost effective and most effi-
cient way of gaining experience in combating exclusion from the
labour market and promoting inclusion in the labour market for

(c)

the Commission should develop a strategic plan for system-
atic transnational transfer of lessons learnt and good practice
to social partners and policy-makers in order to make sure
that the results are used in policy implementation.

This report was adopted by the Court of Auditors in Luxembourg at its meeting of 17 and 18 July 2002.

For the Court of Auditors

Juan Manuel FABRA VALLÉS

President

29.10.2002

EN

Official Journal of the European Communities

C 263/13

THE COMMISSION’S REPLIES

SUMMARY

II.

In its Communication of 10 July 1996 on Integra, (1) the Commission also highlighted:

— the use of methods developed on the basis of practical experience in the field; and

— the application of integrated approaches in order to take account of the diversity of problems relating to

exclusion from the labour market.

IV.
The Commission believes that, given the objectives pursued by the Integra initiative, methodological
questions inevitably arise regarding the tools for achieving those objectives. Exclusion is a complex phenom-
enon influenced by many factors. As many evaluations have shown, it is not easy to assess the effectiveness of
operations, as the indicators often prove to be inadequate.

No precise and inflexible definition of target groups beyond those listed in the Communication to the
V.
Member States (see point II above) was attempted by the Commission in view of the often different circum-
stances prevailing in the different Member States. Concepts and definitions can differ from Member State to
Member State, and a detailed ‘top-down’ definition by the Commission risked limiting the scope of action of
the Member States.

Moreover, selection of projects to participate in Integra was the prerogative of the selection committees estab-
lished in each Member State by the Managing Authority for the initiative. The precise interpretation of innova-
tion in each Member State was a responsibility of the selection committee.

For the successor initiative, ‘Equal’ (2), the Commission provided a clear and comprehensive set of definitions,
based on the typology which emerged from the evaluation of Employment. Nevertheless, despite this defini-
tion, appraisal of the innovative nature of a project remains the sole responsibility of the project selection com-
mittees, which base their judgment among other things on their knowledge of the situation on the ground
and of local circumstances.

The requirement for projects to be proposed by promoters working directly with the target groups led
VI.
to an automatic concentration of effort on areas with a concentration of target groups. The specific problems
of disadvantaged urban areas have also been specifically tackled by the Urban Community initiative pro-
grammes.

VII.
these lessons strongly influenced the design of the successor human resource Community initiative, Equal.

Lessons were drawn from the implementation of the transnational dimension of the initiative and

(1) Communication to the Member States laying down amended guidelines for operational programmes or global grants
which Member States are invited to propose within the framework of a Community initiative on employment and devel-
opment of human resources aimed at promoting employment growth mainly through the development of human
resources (96/C 200/06, OJ C200, 10.7.1996, p. 13).

(2) The Community initiative Equal concerning transnational cooperation to promote new means of combating all forms
of discrimination and inequalities in connection with the labour market, for the programming period 2000 to 2006
(2000/C 127/02, OJ C127, 5.5.2000).

C 263/14

EN

Official Journal of the European Communities

29.10.2002

As regards evaluation and follow-up to lessons, an EU-wide mid-term evaluation was carried out from
VIII.
August 1998 to January 2000, and an ex post evaluation of the Employment initiative will be carried out in
2002 and 2003.

The Commission and the Member States have worked hard to draw lessons from the experience acquired dur-
ing the initiative, in particular by:

— producing a series of publications,

— publishing hundreds of descriptions of good practice on the Web,

— holding regular meetings of working parties at European level,

— organising several conferences.

The Commission would stress the importance of the work on indicators currently in progress as part
IX.
of the ‘inclusion’ process (following the Lisbon and Nice summits). This exercise highlights the difficulties in
establishing harmonised indicators that can shed light on instances of exclusion.

The Commission produced guidance in 1997, entitled ‘Impact Guide for Employment Project Promot-
X.
ers’. The Guide provides methods on how to plan a strategy to disseminate the successful outcomes of a
project (1). This was followed in 1999 with a ‘Guide to Public Relations Activities for Project Promoters’, to
enable projects in the Community initiatives to realise their full potential through effective communication (2).
As regards dissemination and mainstreaming applied to Integra, the new knowledge about combating exclu-
sion from the labour market has been principally disseminated at EU, national and local level via publications
and web sites.

The Commission’s web site at http://europa.eu.int/comm/equal contains the publications referred to in
XI.
point X above, and many more giving examples of good practice, and is freely accessible by all labour market
decision-makers in the EU. The majority of the publications produced by the Commission were printed and
distributed widely throughout the EU.

Events organised at national and European level and the systematic incorporation of past experience in pro-
grammes that took over from Integra (in particular the Equal initiative) are indications of intense activity in
the running of programmes. On the specific question of transnationality, a number of publications available
on the Web describe what has been achieved through this form of cooperation.

XII.
The final assessment of the effectiveness of Integra in achieving its objectives cannot be made in
advance of the receipt and analysis by the Commission of the final national reports (due in June 2002) and of
the publication of the ex post evaluation in 2003.

XIII.
The Commission considers that, in the particular case of Integra, the main target groups of all strands
of the Employment Initiative were well defined in the Communication to the Member States, but without the
imposition of an overly ’top-down’ definition that would have restricted the Member States’ ability to fine-
tune the focus of their Operation Programmes to the specific needs of their labour markets.

(1) CE-V/7-97-003-XX-C, 1997, p. 32.
(2) As above, 1999, p. 28.

29.10.2002

EN

Official Journal of the European Communities

C 263/15

— Furthermore, the only valid indicators for a programme of innovative action such as Employment can be
changes in labour market policies and practices that are deeply rooted and permanent. Experience of
Adapt and Employment lead the Commission to the same conclusion as the Court in respect of indica-
tors and programme follow-up, and this issue has been addressed in the design of the Equal initiative.
However, it is obvious that most of these indicators will remain qualitative rather than quantitative in
nature, thereby leaving broad scope for discretion.

— As regards the need to develop a strategic plan for systematic transfer of lessons and good practice, the
Commission considers that the national action plans for employment, and the European employment
strategy in general, form an important part of the strategy plan for the systematic improvement of human
resources in the Member States, in which the human resource Community initiatives including Employ-
ment, play an important and active role.

PRE-LAUNCH ACTIVITIES

12.

The Commission wishes to point out that:

(a)

selection of projects to participate in Integra was the preroga-
tive of the selection committees established in each Member
State by the managing authority for the initiative. The precise
interpretation of innovation in each Member State was a
responsibility of the selection committee;

(b) as regards the definition of the population of the different
target-groups in Integra, point 12 of the 1996 Communica-
tion to the Member States laying down amended guidelines
for operational programmes in Employment (1) lists the tar-
get groups of the Integra strand. However these guidelines are
broad and general, and the Member States were encouraged
to interpret them in accordance with their own circumstances
and priorities as set out in their Operational Programmes;

(c) concerning the identification of objectives, these were set out
in general terms in the 1996 Guidelines (in section I ’Objec-
tives’) and in the Structural Fund regulations. Accordingly, the
precise definition of the objectives for each of the target
groups identified was to be set out in the Member States’
Operational Programmes in the light of their national poli-
cies and priorities;

(d) and (e)

the overall context for the selection of projects and
target groups was provided in the Commission’s White
Paper on Growth, Competitiveness and Employ-
ment (2) and in Section VII of the Guidelines. It under
lines, however, the difficulty in establishing indicators

to measure exclusion, as shown by the exercise cur-
rently carried out in this domain, following the Lis-
bon and Nice summits;

(f)

the establishment of benchmarks relating to identifiable tar-
get groups could give an inaccurate indication of the success
of a programme like Integra where the target population is in
a constant state of flux, and where qualitative effects on sys-
tems, and on factors impacting on labour market exclusion,
are of greater relevance.

13.
The Commission is aware of the difficulties involved in
evaluating an initiative containing such a diversity of target groups
and activities. Nonetheless it feels that the potential benefits from
an approach that allows cross-fertilisation of activities and lessons
outweigh the advantages of a comparative analysis of the projects.

14.
The approach adopted was a holistic one, in which the
problem of labour market exclusion in its broadest sense was
tackled. Nonetheless, for example in the United Kingdom (3) the
objectives of the initiative were based on an analysis of the Brit-
ish labour market, which identified the groups that were most dis-
advantaged. This analysis was published in September 1996 as a
supplement to the OP (4) for Employment, in which the target
groups for Integra were identified and the rationale for the pro-
posed activities justified by statistical means. The United Kingdom
’Guidance for Applicants for the Employment nitiative 1997 to
1999 (Part 2)’, annex 3 ’beneficiary information’, defines the pro-
posed beneficiaries in some detail, while Annex 4 lists the 805
wards experiencing labour market disadvantage which were pri-
oritised under Integra.

(1) Communication to the Member States laying down amended guide-
lines for operational programmes or global grants which Member
States are invited to propose within the framework of a Community
initiative on employment and development of human resources aimed
at promoting employment growth mainly through the development
of human resources (96/C 200/06, OJ C 200, 10.7.1996, p. 13).
(2) Growth, Competitiveness, Employment — The Challenges and Ways
Forward into the 21st Century — White Paper. ISBN 92-826-7423-1,
1994, Chapter 8(f).

(3) The United Kingdom presented two separate operational programmes
(OPs) under the Employment Community initiative — one for Great
Britain and one for Northern Ireland. The examples given in the Com-
mission’s reply refer to the OP for Great Britain.

(4) Operational programme. The form on which Member States’ plans
for action under the Community initiatives were presented to the
Commission.

C 263/16

EN

Official Journal of the European Communities

29.10.2002

Similar information is available in the wider context of the French
Operational Programme for Employment.

mission by 30 June 2002; analysis of these reports will pro-
vide the Commission with a detailed knowledge of the imple-
mentation of Integra in all Member States.

Clear definitions of key words were given in, for example,
15.
France, where the guide for applicants gave an explanation of key
ideas including the assessment of the concept of innovation. Simi-
larly, the United Kingdom Guidance for Applicants for the Employ-
ment initiative 1997 to 1999 gave definitions of the concepts of
innovation, monitoring, multiplier effects, dissemination and
mainstreaming as well as transnationality. This guidance was dis-
tributed to all applicants in advance of the submission of project
proposals. In addition, in Annex 1 of the United Kingdom Guid-
ance, applicants were provided with the precise selection criteria
that were used to assess their understanding and application of
these concepts.

SELECTION AND IMPLEMENTATION

18.
The Commission had established general guidance for the
monitoring and evaluation of ESF operations in 1995, and spe-
cific guidelines for the Employment initiative in 1999 (1).

On the question of defining target groups, the Commis-

19.
sion would point out that:

— the target groups of the Integra strand are defined in point
12 of the Communication to the Member States (2) laying
down amended guidelines for operational programmes in
Employment (3),

— it had a clear overview of the implementation of Integra
throughout its period of application through its technical
assistance office, Europs, which gathered information on the
projects in cooperation with the national technical assistance
offices for Integra,

20.
The aim of the Initiative was to improve the operation of
the labour markets in the EU, and since the principal actors here
are national, or even sub-national, it was to those levels that most
mainstreaming attention was paid.

The bulk of dissemination concerning the projects was carried
out via the national technical assistance offices, most of which
continued in operation for a period following the closure of
Europs.

At EU level, from 1997 onwards, dissemination was centred on
the nine ’Thematic Focus Groups’ (TFG) set up expressly for this
purpose. These TFGs were led in all cases by Member States, and
contained representatives from national technical assistance,
national administrations and projects, and thus were not greatly
affected by the close of Europs.

21.
The initial lessons to be drawn from Integra, and the other
strands of the Employment Community initiative, are described
on the Commission’s public internet site (4). These key lessons
were classified according to the taxonomy of the European employ-
ment strategy to facilitate their use by Member States in the draw-
ing up of their national action plans for employment.

In addition, in order to stimulate Member States to use the results
of the Employment initiative, wherever there were any, the design
of the current Equal initiative requires Member States to specify
the lessons they learned from Adapt and Employment for the the-
matic priorities selected for Equal.

— according to the Regulation, the Final Reports on the imple-
mentation of the Initiative are due to be sent to the Com-

In Spain one of the main selection criteria for projects
25.
was ‘innovation’ (as specified in the amended Operational Pro-
gramme, Commission Decision C(97) 863).

(1) Common guide for monitoring and evaluation, EC Structural Funds,
European Commission, 1995 and, A methodology for the European
evaluation of the Employment initiative, 1999.
(2) Op. cit. (see footnote 1 — document of 1996).
(3)

‘Target groups will continue to include those disadvantaged groups
previously covered by Employment-Horizon, such as the long-term
unemployed, jobless single parents, the homeless, itinerants, gypsies
and travellers, prisoners and ex-prisoners, substance abusers, etc.
However, a particular emphasis should be given to action that focuses
on the special needs of migrants, refugees and other similarly vulner-
able groups.’
The definition of target groups under Employment-Horizon, referred
to in the paragraph above, included ‘drug addicts, marginalised per-
sons, immigrants, refugees, itinerants, isolated people, single-parent
families, the homeless, prisoners and ex-prisoners.’ (OJ C 180, 1.7.1994,
p.36).

In the United Kingdom the Employment OP specified additional
selection criteria for Integra which asked applicants to identify
how the project differed from existing provision, and did not
duplicate other provisions.

As pointed out in paragraph II, the Commission Com-
26.
munication of 1996 set out clear objectives for Integra projects.
In the United Kingdom at least, the selection criteria took into

(4) At http://europa.eu.int/comm/equal/ae.cfm → Strategic Lessons.

29.10.2002

EN

Official Journal of the European Communities

C 263/17

account the target group and complementarity with other similar
programmes. The United Kingdom Guidance for Applicants for
the Employment Initiative 1997 to 1999 (Part 2), Annex 1, shows
clearly the selection criteria for Integra and the other strands of
Employment, and includes the issues of targeting, labour market
analysis, innovation, transnationality, outputs, monitoring and
evaluation, dissemination, multiplier effect, capacity, budget, and
complementarity.

27.

Regarding the emphasis on migrants and refugees:

in Spain the call for projects mentioned that for Integra, priority
would be given to the fight against racism and xenophobia, lan-
guage training for immigrants, etc., despite the fact that the rela-
tive group size at that time was much lower in Spain than in other
countries;

the United Kingdom OP specified that refugees and migrants were
a target group. The selection criteria (contained, for instance in its
Guidance for Applicants for the Employment Initiative 1997 to
1999 (Part 2)) gave specific weighting to activities in favour of
refugees.

EVALUATION

29.
Reliable indicators were not available during this period.
The only relevant benchmark was one which compares instru-
ments, techniques and methodologies developed through the Ini-
tiative with those in existence beforehand.

30.
The inception of the Initiative can be linked directly to
the Commission’s White Paper on Growth, Competitiveness and
Employment (in particular Chapter 8), which provided a wide-
ranging background to the economic and social challenges facing
the EU in 1994. The specific initiatives undertaken by the Com-
mission in the period immediately following the White Paper
should be seen in the light of the ambitious attempt by the Com-
mission to effect broad changes on the economic and social fabric
of the EU.

It will not be possible to assess the cost and possible ben-
31.
efits arising from the Initiative in each Member State and the
extent of participation by members of the target groups until the
Commission has received and analysed the final programme
reports from the Member States. The due date for receipt of these
reports is 30 June 2002.

32.
Evaluation was carried out during the programme on the
basis of general guidance for ESF operations established by the
Commission (1). The EU-wide evaluation published in March

(1) As previously cited, Common guide for monitoring and evaluation,
EC Structural Funds, European Commission, 1995 and, A methodol-
ogy for the European evaluation of the Employment initiative, 1999.

2000, which was classed as a ’mid-term’ evaluation, relied both
on national evaluation reports discussed in joint meetings and on
specific investigations such as case studies. In addition, an ex post
evaluation of the Employment Community initiative will be car-
ried out during 2002 and 2003, for which information covering
the entire initiative will be accessible.

Precise information on the outcomes of the initiative in
33.
each Member State will not be known until the Commission has
received the final programme reports from the Member States.
The due date for receipt of these reports is 30 June 2002.

34.
Because Employment was a single Community initiative,
no specific evaluation by strand was carried out. The EU-wide
mid-term evaluation of the Employment initiative, which was car-
ried out in cooperation with the evaluators in each Member State,
contains considerable information on the implementation of Inte-
gra (2). Regarding the coherence of the evaluation approach fol-
lowed for Employment, please see point 33.

36.
The ‘Conclusions of the ESF mid-term evaluations’ pointed
out that the Community Initiative should not be evaluated in
strictly quantitative terms. Evaluations should be focused on quali-
tative issues, such as processes designed to reach new populations
affected by exclusion. A number of Member States have devel-
oped monitoring instruments to collect data about the implemen-
tation of the project in terms of content, financing and transna-
tionality (3).

The design of Employment was premised on the existence
37.
of many different problems facing many different groups and sub-
groups, and the need for a large number of varied proposals to
solve these problems. The ex post evaluation of the Employment
initiative, due to be published in 2003, should draw the attention
of policy-makers and programme-designers to different models of
best practice, as well as providing an independent assessment of
their potential.

41.
Provision was made for evaluation during and at the end
of the programme. The Commission has established general guid-
ance for ESF operations (4). Evaluators working at EU level have
been brought in to support and supplement national evaluators.
The EU-wide evaluation establishes methodological guidelines to
support national evaluators and set up a common framework on
types of analyses to conduct and information to gather. The
EU-wide evaluators can rely on both national evaluation reports
discussed in joint meetings and specific investigations such as case
studies.

(2) Cited above.
(3)

‘Final EU-wide evaluation of Community Initiative Employment’,
March 2000, p. 107.

(4) Cited above, Common guide for monitoring and evaluation, EC Struc-
tural Funds, European Commission, 1995, and, A methodology for
the European evaluation of the Employment initiative, 1999.

C 263/18

EN

Official Journal of the European Communities

29.10.2002

The evaluation reports on individual projects formed part
42.
of the information and dissemination structures of the Member
States, and through them, at EU level. In many cases the results
of the project evaluations were incorporated into national reports
and publications which were placed in the public domain.

The Commission, through its Technical Assistance Office
48.
for the Employment Initiative, Europs, set up an internet site that
contained considerable information on experiences acquired and
lessons learnt in Integra. When Europs ceased to operate in 2000
the contents of the web site were transferred to the Commission,
and are currently freely available on the site: http://europa.eu.int/
comm/equal.

43.
According to the objective of the initiative, there is no
provision for projects to continue indefinitely. If a project devel-
ops a successful approach, then that approach should be main-
streamed into ’normal’ labour market policy and practice. Keep-
ing it within the CIs would represent a failure to mainstream and
hence run counter to a key principle of the initiative.

Many Member States, including, for example, Spain set up data-
bases containing basic information on projects.

However, Member States had a strategy for dealing with the les-
sons of projects, through the process of mainstreaming.

50.
The Member States were closely associated with the EU
level work of the Thematic Focus Groups (TFGs), which produced
a series of guidelines in the ’Innovations’ series. Of specific rel-
evance to Integra were the following publications:

DISSEMINATION AND MAINSTREAMING

As part of its dissemination activities, the Com-
44 and 45.
mission produced guidance in 1997, entitled ’Impact Guide for
Employment Project Promoters’. The Guide provides suggestions
on how to plan a strategy to disseminate the successful outcomes
of a project. (CE-V/7-97-003-XX-C, 1997, p. 32). This was fol-
lowed in 1999 with a ’Guide to Public Relations Activities for
Project Promoters’, to enable projects in the Community Initia-
tives to realise their full potential through effective communica-
tion (1999, p. 28).

In addition, the national Technical Assistance Offices (TAOs) pro-
vided guidance on their own projects.

For instance:

— in the United Kingdom the TAO produced, in July 1997,
Guidance Note 7: Dissemination, which provided detailed
guidance on projects, covering the objectives, means and
techniques of dissemination,

— in Spain brochures were produced and three seminars were
organised at national level in July 1998, November 1998 and
June 1999. Many more seminars and information meetings
were organised at regional and local level. Specific support
was given to networks dealing with the target groups (Red
Europea de Gitanos y Viajeros, Red Repris, Red Labora Vita).

— employment and solidarity (1),

— forging strong local partnerships (2),

— towards employment for all (3). Other publications in the
series, while not directly stemming from the work of Integra
projects, contain valuable lessons for the target groups of
Integra (see Annex 1).

51.
Clearer information about the extent to which Integra has
had an impact on labour market exclusion will come from the ex
post evaluation of the Employment Community initiative, to be
published in 2003, and from the Final Reports on the implemen-
tation of the initiative at national level, which should be sent to
the Commission by 30 June 2002.

53.
The Commission, in co-operation with the Member States,
published ‘Transnationality works! If you work at it!’, a detailed
publication in which transnationality was explored from three
key perspectives. This publication was printed in the five largest
official languages and widely distributed to key actors, decision
makers, national authorities, and projects. It is available freely
from the Commission’s internet site at:http://europa.eu.int/comm/
equal (Catalogue information: Innovations No 8, CE-19-98-487-
XX-C, 1999, p. 60).

54.
The Commission is not in position to confirm or contra-
dict the Court’s assertion that no strategic plan for systematic
organisation of transfer of good practice existed in the Member
States visited.

46.
The Commission agrees with this observation, and has
already included the requirement to establish such wide steering
groups in the design of the current Equal Community initiative.

(1)
(2)
(3)

Innovations No 7, CE-11-98-932-XX-C, 1998 p. 36.
Innovations no1, CE-11-98-924-XX-C, 1998 p. 28.
Innovations No 9, CE-26-99-780-XX-C, 2000, p. 40.

29.10.2002

EN

Official Journal of the European Communities

C 263/19

However, in the United Kingdom at least, there was a systematic
mainstreaming plan, drawn up by the Managing Authority and
agreed by the Monitoring Committee (see Employment MC meet-
ing, 29 April 1997). This plan was implemented through the
Technical Assistance Office, continuously updated and progress
was reported back to the MC as a standing item in all subsequent
meetings. A similar plan was implemented in France.

As regards pre-launch activities, the objectives of the Ini-
69.
tiative were set out both in the Guidelines (in section I ’Objec-
tives’) and in the Structural Fund regulations. The overall context
was provided in the Commission’s White Paper on Growth, Com-
petitiveness and Employment (1), as indicated in point 3 of the
guidelines.

The beneficial effects of the involvement of such actors
55.
(local authorities, employers, social partners, etc.) in project steer-
ing groups were noted by the Commission during the period of
implementation of the Employment initiative. For this reason the
systematic inclusion of such actors in the steering groups of
projects was built into the Equal Community initiative (a succes-
sor to Employment).

OTHER ASPECTS

70.
The Guidelines laid down by the Commission in its Com-
munication to the Member States (2) were broad and general, and
the Member States were encouraged to interpret them in accor-
dance with their own circumstances and priorities as set out in
their Operational Programmes. The Commission considers that
the establishment of benchmarks relating to identifiable target
groups could give an inaccurate indication of the success of a
programme such as Integra where the target population is in a
constant state of flux, and where effects on systems, and on fac-
tors impacting on labour market exclusion, are of greater rel-
evance.

58.
As a result of the requirement to involve local actors in
the planning, implementation and evaluation of the measures
(point 6(d) of the Guidelines), there was a logical tendency to con-
centrate activities in those areas where such relevant local actors
were in existence.

71.
Both in Spain and in the United Kingdom the Opera-
tional Programmes did specify that refugees and migrants were a
target group. In the United Kingdom the selection criteria (con-
tained, for instance in its Guidance for Applicants for the Employ-
ment Initiative 1997 to 1999 (Part 2)) gave specific weighting to
activities in favour of refugees.

In Spain, for example, the selection criteria for Integra specifically
mention priority for action in disadvantaged urban zones. In order
to reinforce the complementarity of the activities of the different
initiatives undertaken in such areas, Spain organised a joint Integra-
Urban seminar in Córdoba.

60 to 63. and 65 to 67.
Experience of Adapt and Employ-
ment led the Commission to the same conclusions as the Court,
and this issue has been addressed in the design of the current
Equal Initiative.

The interests of subsidiarity tended towards an automatic
64.
deference towards the preferences of Member States in this mat-
ter. The selection of projects and approval of their budgets was a
national prerogative, which was based largely upon a detailed
analysis of their proposed work programmes.

CONCLUSIONS AND RECOMMENDATIONS

68.
The Commission considers that Integra, and the Employ-
ment Community initiative as a whole, was ambitious, as the
problems it faced warranted, but not over-ambitious. The objec-
tives of the initiative were clear and relatively simple, as stated in
the 1996 Communication to the Member States laying down
guidelines for the initiative, and have paved the way for success
in most of the projects carried out on the ground.

72.
The initial results of Integra, and the other strands of the
Employment Community initiative, are available on the Commis-
sion’s public internet site. These key lessons were classified accord-
ing to the taxonomy of the European employment strategy to
facilitate their use by Member States in the drawing up of their
national action plans for employment.

In addition, the ex post evaluation of Employment, and the final
programme reports (due by 30 June 2002), will provide a concise
summary of the key lessons at national and EU level and the
results achieved.

73.
Employment was a single Community initiative and no
specific evaluation by strand was carried out. The EU-wide mid-
term evaluation establishes methodological guidelines to support
national evaluators and set up a common framework on types of
analyses to conduct and information-gathering. In addition, the ex
post evaluation of the Employment initiative will provide detailed
information on the outcomes of the whole initiative, including
Integra.

ISBN 92-826-7423-1.

(1)
(2) As previously cited, Communication to the Member States

(OJ C 200, 10.7.1996, p. 19).

C 263/20

EN

Official Journal of the European Communities

29.10.2002

The Commission concurs with this observation concern-
74.
ing dissemination and has built the requirement into the current
Equal Community initiative (a successor to Employment).

analysis by the Commission of the final reports on the Employ-
ment Initiative, which are due by 30 June 2002, and of the ex post
evaluation, to be published in 2003.

75.

(a) Both in Spain and in the United Kingdom, the selection cri-
teria covered the issue of innovation and targeting of assis-
tance towards projects dealing with people at risk of exclu-
sion from the labour market.

(b) A reliable conclusion concerning the experience acquired and
the follow-up to the lessons learnt cannot be made in advance
of the receipt and analysis by the Commission of the final
reports on the initiative.

(c) Monitoring took place in each Member State via the formally
constituted Monitoring Committee, of which the Commis-
sion was a member. Dissemination took place widely at
project, local, national and EU-wide level. Neither were used
to create new models for combating exclusion from the
labour market, but both were important elements in the pro-
cess of ensuring that the lessons learned by projects were
passed to the widest possible audience of decision-makers.

A reliable conclusion concerning the value and effective-
76.
ness of Integra cannot be made in advance of the receipt and

As regards the Court’s recommendations, the Commis-

77.
sion would point out that:

(a)

(b)

(c)

the experience of the first few years of the European Employ-
ment Strategy demonstrated the value of a more thematic
approach to employment policy. Discrimination and inequal-
ity are rife and take different forms. If we want to open up the
labour market to everyone, we must embark on a vast experi-
ment. The Commission feels that it is mainly up to local bod-
ies to ensure that the projects selected are innovative in nature.

it shares the Court’s concern that reliable and standardised
indicators should be available as quickly as possible; this
forms part of the joint work currently being undertaken by
the Member States. Qualitative evaluation throughout the
project is the best guarantee that such experimental pro-
grammes have added value.

the experience gained in Adapt and Employment was the first
stage in closer and more intense cooperation between the
Commission and the Member States, in particular through
the ‘thematic focus groups’ set up at European level. These
positive features were taken fully into account in the design
of the Equal initiative.

Annex 1

A set of flagship publications, produced towards the end of the Initiative, includes the following booklets of
direct relevance to Integra:

— Crossing the Job threshold: the output of the Thematic Focus Group ‘Crossing the Job Threshold’. This pub-
lication points to the need to target resources on those individuals who are excluded, or at risk of exclu-
sion, from the labour market. It is primarily of interest to those institutions and persons that can act as
mediators between such disadvantaged groupings and employers, such as public employment services;
private employment and placement services; vocational training providers; non-governmental organisa-
tions (NGOs) representing the interests of disadvantaged groups. Such institutions cannot easily change
or develop their methods and approaches unless there is a supportive policy environment and so this
publication is also addressed to the social partners and decision-makers concerned with employment poli-
cies at European, national, regional and local level (CE-V/1-00-016-XX-C, 2000, p. 28),

— Empowerment: output of the Thematic Focus Group on ‘Empowerment’. This Group has looked into the
ways in which Employment-Integra projects pursued the objective of ’empowering’ excluded people and
groups, as a strategy for their integration into employment. It identified projects most representative of
this approach and worked at both national and European level to explore the nature and impact of
empowerment strategies for employment and inclusion. (CE-V/1-00-013-XX-C, 2000, p. 30),

29.10.2002

EN

Official Journal of the European Communities

C 263/21

— Integra — Routes to Integration and employment for the socially excluded. Integra aims to promote measures to
improve the access to the labour market and employability of those who find themselves excluded from
it. (CE-97-96-475-XX-C, 1997, p. 20)

— Shaping active employment policies: Synthesis of Adapt & Employment strategic lessons: Adapt and Employ-
ment projects have explored a vast range of innovative models of good practice to improve and support
active employment policies. These can be directly related to the four pillars of the European Employment
Strategy (EES) and the Member States’ national action plans for employment (NAPs): employability; entre-
preneurship; adaptability; equal opportunities (CE-V/1-00-020-XX-C, 2000, p. 16).

— Territorial Approaches: the key principles: The term ’territorial approaches’ refers to an emerging system of
local governance. This system enables regions, and areas within regions, to respond flexibly and innova-
tively to the challenges of an increasingly open global economy, while simultaneously encouraging par-
ticipation and stakeholding by citizens (CE-V/1-00-001-XX-C, 2000, p. 36).