Opinion of the Committee of the Regions on the "Report from the Commission Mid-term review of structural interventions Objectives 1 and 6 (1994-1999). Developing a management culture through evaluations: towards best practice"
Date: 2000-04-13
Institutions: European Committee of the Regions
Subjects: EU programme, Structural Funds, management

C 226/64

EN

Official Journal of the European Communities

8.8.2000

Opinion of the Committee of the Regions on the ‘Report from the Commission “Mid-term
review of structural interventions Objectives 1 and 6 (1994-1999). Developing a management
culture through evaluations: towards best practice”’

(2000/C 226/17)

THE COMMITTEE OF THE REGIONS,

having regard to the report from the Commission on the ‘mid-term review of structural interventions
Objectives 1 and 6 (1994-1999). Developing a management culture through evaluations: towards best
practice’ (COM(1998) 782 final);

having regard to the decision taken by its Bureau on 2 June 1999 under Article 265, fifth paragraph, of
the Treaty establishing the European Community, to issue an opinion on this subject and to instruct
Commission 1 for Regional Policy, Structural Funds, Economic and Social Cohesion and Cross-border
and Inter-regional Cooperation to prepare it;

having regard to the draft opinion (CdR 389/99 rev. 2) adopted by Commission 1 on 2 February 2000
(rapporteur: Sir Ron Watson, Member of Sefton Metropolitan Borough Council Town Hall, UK/PPE),

adopted the following opinion at its 33rd plenary session on 12 and 13 April 2000 (meeting of 13 April).

1.

Introduction

2. The Mid Term Evaluation Process

1.1.
A continuous evaluation process for the Structural
Funds programmes was established during the 1998 reform
and includes ex-ante, mid term and ex-post evaluation of all
European Union co-financed programmes. The aim is to
ensure the effective disbursement of EU resources, in particular
with regard to reducing economic and social disparities in
objective 1 and 6 regions.

The key element of the Structural Funds is to reduce
2.1.
economic and social disparities within specific national stra-
tegies and priorities. As such the mid term evaluations provide
a tool for assessing what adjustments should be made to
programmes to reflect changing national economic circum-
stances.

1.2.
The Commission’s report includes an examination of
the overall process, the principal findings, the main outcomes
and adjustments made and the implications for future pro-
grammes.
It also makes some recommendations for the
future programming period. The COR wishes to add its own
recommendations.

This is particularly the case in the Cohesion Four
2.2.
countries, where the CSFs cover most,
if not the entire,
country, where the example of Ireland resulted in a re-direction
of resources to public infrastructure to sustain rapid economic
growth. Other exogenous factors such as the earthquake
emergency in Italy and the introduction of employment as
an over riding EU wide priority also led to shifts in the
programmes.

1.3.
The report proceeds to summarise the main elements
of more than a 100 mid term evaluations carried out in
relation to the 1994-1999 Objective 1 and 6 programmes.
These were mostly carried out between 1997 and 1998, thus
taking into account the Commission’s own guidelines, which
were published in May 1997.

2.3.
In May 1997, the Commission issued its guidelines for
priorities for interventions for the Structural Funds post the
mid term. These had been drawn up on the request of the
member states. They set out the changing EU wide policy
context and stipulated what elements should be covered in the
mid term evaluations.

1.4.
The mid term evaluation is a shared responsibility
between the member state, regions and the European Union
and on average 0,1 % of the total budgets have been utilised
for these evaluations.

2.4.
The mid term evaluations have tended to focus on
financial data supplied by the relevant monitoring systems. At
the stage of the mid term evaluation, little information about
the wider and longer-term impacts of the Structural funds will
not be available.

8.8.2000

EN

Official Journal of the European Communities

C 226/65

3. Lessons from the Mid Term Evaluations

3.1.
The extent of mid term evaluations (100+) bear witness
to the importance that both the Commission and the member
states attach to the evaluation process. A key point of the
mid term evaluations is to establish the extent to which a
programme lends itself to evaluation and to establish the
subsequent methodology. The main aims are to assess the
degree to which the programme has met its objectives at the
half way stage, determine the initial impact of the interventions,
and propose, where necessary, recommendations to improve
the management of the programme.

It is clearly essential that the evaluators are independent
3.2.
of both the managing authorities and the executive bodies.
The report does, however, give two examples of a highly
effective internal evaluations in Italy (the Evaluation Unit
of
the Budget Ministry in Italy, and Ireland). Whatever
methodologies are used, evaluations must be based on the
initially agreed criteria and objectives of the programmes to
ensure that progress (or lack of it) can be measured and
appropriate action taken.

3.3.
In some cases partnerships may also wish to engage
with suitably qualified research institutes, which may be
located within higher education institutions or the social
partners. In many cases, representatives of such bodies are
members of
the programme monitoring committees, yet
possess the technical knowledge and competence to undertake
evaluations and assessments. The COR believes it would be
wrong to rule this out, provided clear rules of engagement and
separation of duties exists.

In general, Programme Monitoring Committees have
3.4.
been keen to take on the responsibility for the implementation
of recommendations from the evaluations and appropriate sub
structures have been established to deal with the evaluations,
both in terms of managing the evaluations and taking forward
their recommendations.

3.5.
The involvement of regional authorities has varied,
depending on the political and institutional specificities of the
individual member states. There appears to have been more
direct involvement of regional authorities in case where the
Structural Funds have been delivered through SPDs rather than
CSFs, which by their nature tend to be national programmes.

The Commission’s report states that

3.7.
it has been
possible to assess some of the key macro-economic impacts of
the CSFs in the larger member states (the Cohesion Four, Italy
and Germany) at the mid term. There is a clear positive
impact on both economic growth and employment from the
Structural Funds interventions, although more emphasis needs
to be placed on the use of integrated models, which examines
the impact on both supply and demand.

Generally speaking, the impact on employment has
3.8.
been less than on economic growth in percentage terms, due
to an assumption of increased productivity where companies
become more efficient. There is clearly an inherent tension
between achieving economic growth and the creation of jobs
and while the Structural Funds have to have a proper
regard to the employment effect, economic efficiency of
the investments should be an overriding principle in the
assessments.

3.9.
For the smaller programmes (generally SPDs), results
from macro economic modelling are harder to achieve because
of the inter linkages with the wider economy. The report
quotes the Belgian experience as an example of best practice,
where the results showed a significant risk of a return to
slower growth rate after the completion of the SPD.

3.10.
The Commission’s report also includes a section on
the effectiveness of the interventions, including a review of
indicators used.
the evaluations led to a
substantial review of agreed physical and impact indicators.
Efficiency has not been covered in any great detail.

In many case,

3.11.
The Commission’s report accepts that it is difficult to
present an overview of the outputs and results from across the
programme because programmes are not comparable across
regions. Clearly this is to do partly with the lack of a common
core set of indicators agreed by the Commission and member
states prior to implementation. This aspect also requires
further analysis as the implication could be that indicators are
changed to suit
is clearly
acceptable to do so where economic and social circumstances
require this, this must not be a common practice.

local circumstances. While it

3.6.
In general, evaluation reports have been of a high
quality and standard, showing the progress, which is being
made, in this important field. This has been partly driven by
the MEANS Programme, which aims to facilitate evaluation
work across Structural Funds. The Commission’s report
expresses some concerns relating to the highly academic
nature, narrow scope (management rather than results and
impacts),
lack of independence and lack of support from
managing authorities of some of the evaluation reports.

3.12.
A key feature of the mid term evaluations were the
assessment of the monitoring systems and project selection
criteria. The current programmes contain a significant develop-
ments in the use of quantified indicators. However, a number
of programmes still show a weakness in information available
and, more generally, related to the quantification of indicators.
Clearly, many programmes do not fully utilise information
available as a proper management tool, combining financial
and physical outputs, results and impacts.

C 226/66

EN

Official Journal of the European Communities

8.8.2000

3.13.
In some cases, sophisticated project selection criteria
have been developed to promote transparency for the selection
process. Many mid term evaluations have questioned the
effectiveness of such systems as having transparent scoring
systems may not lead to the best results in terms of project
selection.

3.14. Most mid term evaluations have been a useful tool
and source of knowledge for decision-makers. In most cases,
programme managers have implemented recommendations of
the evaluations. An essential part of a successful evaluation has
been the substantial input from the partnerships themselves to
improve the delivery of the programmes.

to support a re-allocation of

The mid term evaluations have been used, in most
3.15.
case,
resource within the
CSFs/SPDs. In almost all case, the Commission report shows
that these financial re-allocations were made without affecting
the strategic priorities of the programmes. In some case, new
measures were introduced.

3.16. More generally, the Commission’s report concludes
that it has not always been easy to assess the extent to which
EU wide priorities have been addressed. Generally, more
emphasis has been given to employment, environment and
sustainable
information technology,
and
development,
reflecting shifts over time.

3.17.
In its conclusions, the Commission report stresses the
need for monitoring procedures to address financial as well as
programme results and impacts. This includes clear quantified
targets and indicators for monitoring and evaluation, more
appropriate project selection criteria, simplified management
procedures and synergies between the difference funds.

4. Future Challenges

4.1.
The Commission’s report emphasises the need to build
on the best practice which have been established by the
mid term evaluations. These include the soundness of the
evaluation, the involvement of the partnerships, the organis-
ation of the evaluations and the feedback role in supporting
programme decisions.

4.2.
Good evaluations are an important tool for good
management of the programme. This in turn depends on
adequate structures for managing evaluation activities within
the monitoring systems,
the partnership, development of
better integration between financial and physical indicators,
adequate quantification of baselines, progress in the area of
evaluation methodology through the MEANS programme, and
dissemination of best practice. The Commission is proposing
the publication of a guidance document on methodological
issues, including an indicative list of indicators.

5. Recommendations

5.1. While the Committee of the Regions (COR) acknowl-
edges the value of EU guidelines for the mid term evaluations,
it is concerned that these should be available to programme
managers early to inform the process more clearly.

5.2.
The next programming period sees the introduction of
a performance reserve and the COR wishes to stress the need
for early, clear guidance on a core set of EU wide indicators,
against which performance will be judged. Every effort should
be made to make these available prior to the start of the
programmes.

5.3.
The COR expresses its concern that more clarity is
needed in the design of evaluations to enable the production
of comparable reports across the European Union as to the
effectiveness of the Structural Funds.

5.4.
The COR would welcome a more pro-active approach
to the sharing of best practice among evaluation and pro-
gramme executives to ensure that programmes are designed
and delivered in such a manner as to facilitate evaluation to
establish best practice.

5.5.
The COR would also welcome the development of a
set of core common indicators which focus on results and
impacts rather than purely activities, commitment and spend
to facilitate a comparison of performance across the Structural
Funds. These indicators must cover not just GDP but also the
wider economic, social and environmental
impact of the
Structural Funds, particularly in relation to the new Objec-
tive 1.

5.6.
This would also enable a mid term evaluation at the
mid term rather than during the fourth year of implementation,
which has generally been the case during the current program-
ming period. The COR believes that the impact of the mid
term evaluations have been reduced because of the time lag
between evaluations and the implementation of their respective
recommendations.

5.7.
The COR welcomes the statement that the mid term
evaluation is a shared responsibility between the member
states and the Commission. In the light of the recently agreed
general regulations for the 2000-2006 programming period,
it believes that this principle should be extended to include
regional and local authorities.

5.8.
The COR would also welcome the opportunity to work
with the regional Policy Committee of the European Parliament
to promote a culture of rigorous evaluations and monitoring
by member states to assess the effectiveness and efficiency of
the Structural Funds interventions.

8.8.2000

EN

Official Journal of the European Communities

C 226/67

5.9. While COR recognises the need for independent
evaluators, the COR would be concerned if this became a
formal requirement as surely the aim is to improve programme
In cases, say, where
management and effective delivery.
technical knowledge and competence resides within higher
education institutions, technology institutes, etc., such bodies
should not be prohibited per se from carrying out evaluations
during the programme period. Clear rules should be established
so that selection of any evaluators is transparent and under-
stood within the partnership.

5.10.
The COR believes that better and more timely
dissemination of best practice is a must for the next program-

Brussels, 13 April 2000.

ming period. The Means Programme provide a vehicle for this
dissemination, including the development of tool kits and best
practice handbooks. It is essential that such activities are
addressed not
to evaluators but also the managing
authorities.

just

5.11. While the COR welcomes the use of macro-economic
modelling as an evaluation tool in the larger member states, it
is essential that models which integrate the supply and demand
side are used in the next generation of programmes.

The President

of the Committee of the Regions

Jos CHABERT