C 226/64 EN 8.8.2000 Official Journal of the European Communities
Opinion of the Committee of the Regions on the ‘Report from the Commission “Mid-term
review of structural interventions Objectives 1 and 6 (1994-1999). Developing a management
culture through evaluations: towards best practice”’
(2000/C 226/17)
THE COMMITTEE OF THE REGIONS,
having regard to the report from the Commission on the ‘mid-term review of structural interventions
Objectives 1 and 6 (1994-1999). Developing a management culture through evaluations: towards best
practice’ (COM(1998) 782 final);
having regard to the decision taken by its Bureau on 2 June 1999 under Article 265, fifth paragraph, of
the Treaty establishing the European Community, to issue an opinion on this subject and to instructCommission 1 for Regional Policy, Structural Funds, Economic and Social Cohesion and Cross-border
and Inter-regional Cooperation to prepare it;
having regard to the draft opinion (CdR 389/99 rev. 2) adopted by Commission 1 on 2 February 2000
(rapporteur: Sir Ron Watson, Member of Sefton Metropolitan Borough Council Town Hall, UK/PPE),
adopted the following opinion at its 33rd plenary session on 12 and 13 April 2000 (meeting of 13 April).
1.Introduction 2.The Mid Term Evaluation Process
2.1. The key element of the Structural Funds is to reduce1.1. A continuous evaluation process for the Structuraleconomic and social disparities within specific national stra-Funds programmes was established during the 1998 reformtegies and priorities. As such the mid term evaluations provideand includes ex-ante, mid term and ex-post evaluation of alla tool for assessing what adjustments should be made toEuropean Union co-financed programmes. The aim is toprogrammes to reflect changing national economic circum-ensure the effective disbursement of EU resources, in particularstances.with regard to reducing economic and social disparities in
objective 1 and 6 regions.
2.2. This is particularly the case in the Cohesion Four
countries, where the CSFs cover most, if not the entire,country, where the example of Ireland resulted in a re-direction 1.2. The Commission’s report includes an examination of
the overall process, the principal findings, the main outcomes of resources to public infrastructure to sustain rapid economic
growth. Other exogenous factors such as the earthquake and adjustments made and the implications for future pro-
grammes. It also makes some recommendations for the emergency in Italy and the introduction of employment as
an over riding EU wide priority also led to shifts in the future programming period. The COR wishes to add its own
recommendations. programmes.
2.3. In May 1997, the Commission issued its guidelines for1.3. The report proceeds to summarise the main elementspriorities for interventions for the Structural Funds post theof more than a 100 mid term evaluations carried out inmid term. These had been drawn up on the request of therelation to the 1994-1999 Objective 1 and 6 programmes.member states. They set out the changing EU wide policyThese were mostly carried out between 1997 and 1998, thuscontext and stipulated what elements should be covered in thetaking into account the Commission’s own guidelines, whichmid term evaluations.were published in May 1997.
2.4. The mid term evaluations have tended to focus on
financial data supplied by the relevant monitoring systems. At 1.4. The mid term evaluation is a shared responsibility
between the member state, regions and the European Union the stage of the mid term evaluation, little information about
the wider and longer-term impacts of the Structural funds will and on average 0,1 % of the total budgets have been utilised
for these evaluations. not be available.
8.8.2000 EN C 226/65 Official Journal of the European Communities
3.Lessons from the Mid Term Evaluations 3.7. The Commission’s report states that it has been
possible to assess some of the key macro-economic impacts ofthe CSFs in the larger member states (the Cohesion Four, Italy
and Germany) at the mid term. There is a clear positive3.1. The extent of mid term evaluations (100+) bear witnessimpact on both economic growth and employment from theto the importance that both the Commission and the memberStructural Funds interventions, although more emphasis needsstates attach to the evaluation process. A key point of theto be placed on the use of integrated models, which examinesmid term evaluations is to establish the extent to which athe impact on both supply and demand.programme lends itself to evaluation and to establish the
subsequent methodology. The main aims are to assess the
degree to which the programme has met its objectives at the
half way stage, determine the initial impact of the interventions,
and propose, where necessary, recommendations to improve 3.8. Generally speaking, the impact on employment has
the management of the programme. been less than on economic growth in percentage terms, due
to an assumption of increased productivity where companies
become more efficient. There is clearly an inherent tensionbetween achieving economic growth and the creation of jobs3.2. It is clearly essential that the evaluators are independentand while the Structural Funds have to have a properof both the managing authorities and the executive bodies.regard to the employment effect, economic efficiency ofThe report does, however, give two examples of a highlythe investments should be an overriding principle in theeffective internal evaluations in Italy (the Evaluation Unitassessments.of the Budget Ministry in Italy, and Ireland). Whatever
methodologies are used, evaluations must be based on the
initially agreed criteria and objectives of the programmes toensure that progress (or lack of it) can be measured and
appropriate action taken. 3.9. For the smaller programmes (generally SPDs), results
from macro economic modelling are harder to achieve because
of the inter linkages with the wider economy. The reportquotes the Belgian experience as an example of best practice, 3.3. In some cases partnerships may also wish to engage
where the results showed a significant risk of a return to with suitably qualified research institutes, which may be
slower growth rate after the completion of the SPD. located within higher education institutions or the social
partners. In many cases, representatives of such bodies are
members of the programme monitoring committees, yetpossess the technical knowledge and competence to undertake
evaluations and assessments. The COR believes it would be3.10. The Commission’s report also includes a section onwrong to rule this out, provided clear rules of engagement andthe effectiveness of the interventions, including a review of
separation of duties exists. indicators used. In many case, the evaluations led to a
substantial review of agreed physical and impact indicators.
Efficiency has not been covered in any great detail.
3.4. In general, Programme Monitoring Committees have
been keen to take on the responsibility for the implementation
of recommendations from the evaluations and appropriate substructures have been established to deal with the evaluations,3.11. The Commission’s report accepts that it is difficult toboth in terms of managing the evaluations and taking forwardpresent an overview of the outputs and results from across thetheir recommendations.programme because programmes are not comparable across
regions. Clearly this is to do partly with the lack of a commoncore set of indicators agreed by the Commission and member
states prior to implementation. This aspect also requires 3.5. The involvement of regional authorities has varied,
further analysis as the implication could be that indicators are depending on the political and institutional specificities of the
changed to suit local circumstances. While it is clearly individual member states. There appears to have been more
acceptable to do so where economic and social circumstances direct involvement of regional authorities in case where the
require this, this must not be a common practice. Structural Funds have been delivered through SPDs rather than
CSFs, which by their nature tend to be national programmes.
3.6. In general, evaluation reports have been of a high 3.12. A key feature of the mid term evaluations were the
assessment of the monitoring systems and project selection quality and standard, showing the progress, which is being
made, in this important field. This has been partly driven by criteria. The current programmes contain a significant develop-
ments in the use of quantified indicators. However, a number the MEANS Programme, which aims to facilitate evaluation
work across Structural Funds. The Commission’s report of programmes still show a weakness in information available
and, more generally, related to the quantification of indicators. expresses some concerns relating to the highly academic
nature, narrow scope (management rather than results and Clearly, many programmes do not fully utilise information
available as a proper management tool, combining financial impacts), lack of independence and lack of support from
managing authorities of some of the evaluation reports. and physical outputs, results and impacts.
C 226/66 EN 8.8.2000 Official Journal of the European Communities
3.13. In some cases, sophisticated project selection criteria 5. Recommendations
have been developed to promote transparency for the selectionprocess. Many mid term evaluations have questioned the
effectiveness of such systems as having transparent scoring5.1. While the Committee of the Regions (COR) acknowl-systems may not lead to the best results in terms of projectedges the value of EU guidelines for the mid term evaluations,selection.it is concerned that these should be available to programme
managers early to inform the process more clearly.
3.14. Most mid term evaluations have been a useful tool
and source of knowledge for decision-makers. In most cases,5.2. The next programming period sees the introduction ofprogramme managers have implemented recommendations ofa performance reserve and the COR wishes to stress the needthe evaluations. An essential part of a successful evaluation hasfor early, clear guidance on a core set of EU wide indicators,been the substantial input from the partnerships themselves toagainst which performance will be judged. Every effort shouldimprove the delivery of the programmes.be made to make these available prior to the start of the
programmes.
3.15. The mid term evaluations have been used, in most
case, to support a re-allocation of resource within the5.3. The COR expresses its concern that more clarity isCSFs/SPDs. In almost all case, the Commission report showsneeded in the design of evaluations to enable the productionthat these financial re-allocations were made without affectingof comparable reports across the European Union as to thethe strategic priorities of the programmes. In some case, neweffectiveness of the Structural Funds.measures were introduced.
5.4. The COR would welcome a more pro-active approach3.16. More generally, the Commission’s report concludesto the sharing of best practice among evaluation and pro-that it has not always been easy to assess the extent to whichgramme executives to ensure that programmes are designedEU wide priorities have been addressed. Generally, moreand delivered in such a manner as to facilitate evaluation toemphasis has been given to employment, environment andestablish best practice.sustainable development, and information technology,
reflecting shifts over time.
5.5. The COR would also welcome the development of a
set of core common indicators which focus on results and3.17. In its conclusions, the Commission report stresses theimpacts rather than purely activities, commitment and spendneed for monitoring procedures to address financial as well asto facilitate a comparison of performance across the Structuralprogramme results and impacts. This includes clear quantifiedFunds. These indicators must cover not just GDP but also thetargets and indicators for monitoring and evaluation, morewider economic, social and environmental impact of theappropriate project selection criteria, simplified managementStructural Funds, particularly in relation to the new Objec-procedures and synergies between the difference funds.tive 1.
5.6. This would also enable a mid term evaluation at the
mid term rather than during the fourth year of implementation, 4.Future Challenges
which has generally been the case during the current program-
ming period. The COR believes that the impact of the mid
term evaluations have been reduced because of the time lag4.1. The Commission’s report emphasises the need to buildbetween evaluations and the implementation of their respectiveon the best practice which have been established by therecommendations.mid term evaluations. These include the soundness of the
evaluation, the involvement of the partnerships, the organis-
ation of the evaluations and the feedback role in supporting
5.7. The COR welcomes the statement that the mid term programme decisions.
evaluation is a shared responsibility between the member
states and the Commission. In the light of the recently agreedgeneral regulations for the 2000-2006 programming period,4.2. Good evaluations are an important tool for goodit believes that this principle should be extended to includemanagement of the programme. This in turn depends onregional and local authorities.adequate structures for managing evaluation activities within
the partnership, development of the monitoring systems,
better integration between financial and physical indicators,
adequate quantification of baselines, progress in the area of 5.8. The COR would also welcome the opportunity to work
with the regional Policy Committee of the European Parliament evaluation methodology through the MEANS programme, and
dissemination of best practice. The Commission is proposing to promote a culture of rigorous evaluations and monitoring
by member states to assess the effectiveness and efficiency of the publication of a guidance document on methodological
issues, including an indicative list of indicators. the Structural Funds interventions.
8.8.2000 EN C 226/67 Official Journal of the European Communities
5.9. While COR recognises the need for independent ming period. The Means Programme provide a vehicle for this
evaluators, the COR would be concerned if this became a dissemination, including the development of tool kits and best
formal requirement as surely the aim is to improve programme practice handbooks. It is essential that such activities are
management and effective delivery. In cases, say, where addressed not just to evaluators but also the managing
technical knowledge and competence resides within higher authorities.
education institutions, technology institutes, etc., such bodies
should not be prohibited per se from carrying out evaluationsduring the programme period. Clear rules should be establishedso that selection of any evaluators is transparent and under-5.11. While the COR welcomes the use of macro-economicstood within the partnership.modelling as an evaluation tool in the larger member states, it
is essential that models which integrate the supply and demand 5.10. The COR believes that better and more timely
dissemination of best practice is a must for the next program- side are used in the next generation of programmes.
Brussels, 13 April 2000.
The President
of the Committee of the Regions
Jos CHABERT