# EUR-Lex - 52009SA0016 - EN

*|*

# 52009SA0016

**Special Report No 16/2009 - The European Commission’s management of pre-accession assistance to Turkey **

*

*

Abbreviations

AP : Accession Partnership

CFCU : Central Finance and Contracts Unit

DIS : Decentralised Implementation System

EADP : Eastern Anatolia Development Programme

EU : European Union

EUSG : European Union Secretariat-General

IPA : Instrument for Pre-Accession Assistance

JMC : Joint Monitoring Committee

MIPD : Multi-annual Indicative Planning Document

NP : National Programme

RACER : Relevant, Accepted, Credible, Easy and Robust

SMART : Specific, Measurable, Achievable, Relevant and Timely

TPA : Turkey Pre-Accession Assistance

Summary

I. Since 2002, pre-accession assistance to Turkey has funded the implementation of projects aimed at supporting Turkey in its efforts to meet the conditions required for accession to the European Union (see paragraphs 1 to 7).

II. The audit addressed the following three questions (see paragraph 8):

(a) Did the European Commission ensure that EU assistance was directed to the projects that add most value in achieving the EU’s Accession Partnership priorities?

(b) Did the European Commission ensure that the decentralised Implementation system resulted in timely and successful achievement of project outputs and objectives?

(c) Did the European Commission ensure that there was an effective system of performance monitoring and evaluation?

III. The Court’s audit examined a sample of completed projects, the system for programming new projects, and the functioning of the decentralised implementation system (see paragraphs 9 to 10).

IV. The first pre-accession assistance period (TPA 2002–06) suffered from many of the weaknesses common to previous pre-accession programmes: excessive delays, implementation problems, inadequate monitoring and evaluation (see paragraphs 36 to 42 and 53 to 61).

V. Nevertheless, the projects audited achieved their intended outputs and the Court assesses their results as likely to be sustained in the future (see paragraphs 48 to 52).

VI. The Commission has introduced measures aimed at addressing many of the weaknesses in the decentralised Implementation system, in particular since the introduction of the new instrument for pre-accession assistance (IPA 2007–13). Although this has already improved the selection process, the full impact of these changes can only be assessed as the IPA projects are implemented in the coming years (see paragraphs 19, 30 to 34 and 43 to 47).

VII. However, the Court identified that the Council and Commission strategy documents were insufficient in directing the EU assistance towards an achievable set of objectives within the pre-accession process. Moreover, the strategic and project objectives were not sufficiently specific to allow assessment of the project outcomes. The Commission did not have the information to demonstrate the effectiveness of the pre-accession assistance (see paragraphs 11 to 19 and 53 to 63).

VIII. On the basis of these observations, the Court recommends that the Commission address the remaining weaknesses in overall programming and performance management (see paragraphs 65 to 71).

Photo 1: One of the audited projects contributed to the overall objective of improving Turkish maritime safety and sea pollution prevention in line with EU requirements.

+++++ TIFF +++++

Source: European Commission.

Introduction

1. Turkey is currently one of three candidate countries for membership of the EU, along with Croatia and the former Yugoslav Republic of Macedonia. For each candidate country the EU has an Accession Partnership. The Accession Partnership for Turkey was adopted in 2001, and updated in 2003, 2006 and 2008. This is the Council decision that sets out the EU’s Accession Partnership priorities for Turkey to address in order to make progress in accession negotiations.

2. In order to help candidate countries face the challenges of European integration and implement the reforms needed to fulfil the criteria for EU membership, the EU provides financial support. Each year a national programme is drawn up which lists the projects to be implemented.

3. For Turkey, the 2002 to 2006 national programmes were funded under the Turkey Pre-Accession Assistance (TPA) scheme (see Table 1). TPA was replaced from 2007 by the Instrument for Pre-Accession Assistance (IPA [1]).

Table 1

EU-funded projects and annual budgets under TPA

National programme | Number of projects | Budget allocation (million euro) |

2002 | 18 | 126 |

2003 | 26 | 145 |

2004 | 38 | 237 |

2005 | 38 | 278 |

2006 | 44 | 463 |

4. The IPA for Turkey for the 2007–13 period was set at 4873 million euro. It consists of five components, the largest of which is Component I: Transition Assistance and Institution Building, with an allocation of 1665 million euro [2]. This is a key component of the EU’s assistance to meet the political and economic criteria (as defined at the Council meeting in Copenhagen in June 1993) [3] and implement the acquis [4], which are indispensable conditions for EU accession.

5. While the Commission retains overall responsibility for the management of the pre-accession assistance, the TPA and IPA are managed by Turkish authorities under a decentralised implementation system (DIS). The main beneficiary of each project (usually a ministry or other public body) is responsible for proposing project ideas, specifying the requirements and then for managing the project activities. New Turkish institutions were set up: the EU Secretariat-General (EUSG), responsible mainly for project identification and monitoring, and the Central Finance and Contracts Unit (CFCU), which is the contracting agent for all EU-funded projects.

6. The Commission was responsible for giving approval (accreditation) to the DIS institutions and the project management system before they could become operational. The Commission also approves the annual national programme (comprising the individual projects selected for EU funding), and the Commission’s staff in Ankara give prior approval for the tendering and contracting.

7. The Joint Monitoring Committee (JMC) for Turkey monitors the functioning of the DIS. It is chaired jointly by the Commission and the Turkish authorities and includes senior representatives of all the relevant bodies.

Audit scope and approach

8. The objective of the audit was to assess how well the Commission managed the pre-accession assistance to Turkey in terms of the following questions:

(a) Did the European Commission ensure that EU assistance was directed to the projects that add most value in achieving the EU’s Accession Partnership priorities?

(b) Did the European Commission ensure that the decentralised implementation system resulted in timely and successful achievement of project outputs and objectives?

(c) Did the European Commission ensure that there was an effective system of performance monitoring and evaluation?

9. The Court audited a sample of 11 of the 82 TPA projects from the 2002 to 2004 national programmes (NP), as projects from more recent years were not complete at the time. The projects were examined to assess the extent to which they had delivered their outputs and met their objectives. This examination also provided evidence of the types of weaknesses in the programming, implementation and monitoring systems. The sample covered a range of projects in terms of nature, size and geographical location. The simplest project in the sample consisted of a single contract, while the largest project had 312 grants. The value of the EU funding within the sampled projects ranged from 1,5 to 45 million euro (see Annex I).

10. The audit also examined the strategic planning and the procedures for selecting new projects for the first programming year (2007) of Component I of the IPA, until December 2008. This component was selected as it was the largest of the five components (52 % of the total 2007 budget for Turkey) and the most comparable with the TPA.

Map of Turkey — showing the on-the-spot audit sites

+++++ TIFF +++++

Source: European Court of Auditors.

Observations

Strategic planning

11. Accession Partnership priorities are not only to be addressed through EU assistance, but also from national resources and other donor assistance, and through national policy and legislation.

12. The Court examined the strategy documents and the programming system operating for the 2007 national programme to assess whether the Commission had a systematic approach to direct the EU financial assistance to where it could add most value in achieving the priorities.

Council and Commission strategy documents were insufficient in directing the EU assistance towards an achievable set of objectives

13. The Financial Regulation [5] requires SMART (specific, measurable, achievable, relevant and time-bound) objectives to be established for all policy measures covered by the EU budget. They should be set out in the annual activity statements [6] as part of the activity based budgeting and management processes. However, the Commission did not include such objectives in the activity statements for pre-accession expenditure in Turkey.

14. The 2006 Accession Partnership had 236 priorities for Turkey to meet the acquis requirements and address the Copenhagen criteria. There was no indication of the relative importance of the priorities listed; the only differentiation was in their timescale. The 2006 Accession Partnership indicated that short-term priorities should be achieved within one or two years and medium-term priorities within three to four years.

15. The Accession Partnership priorities were not consistently stated in specific, measurable terms. It proved to be unrealistic to achieve them all within the time periods specified in the Council decision.

16. In 2007, the Commission assessed the extent to which Turkey had met Accession Partnership priorities. This revealed that Turkey had already met some 60 priorities, 30 % of those due to be met according to the timeframes set in the Accession Partnership. The Court’s analysis confirmed this. Of the 2006 short-term priorities, 67 % were also found in the 2008 Accession Partnership indicating that they had not been completed within the planned timeframe (Table 2).

17. For example, the 2001 Accession Partnership set a priority for the "fight against organised crime, drugs, trafficking in persons, fraud, corruption and money-laundering" to be achieved within two years. This has been repeated in every subsequent Accession Partnership. The two year timescale was clearly not achievable for a priority of such magnitude.

18. In response to each Accession Partnership, Turkey is expected to develop a plan with the measures foreseen [7]. These plans do not specify the activities for which EU financial assistance would be required.

Table 2

Comparison of 2006 AP and 2008 AP short-term priorities

The 2006 AP listed 155 short-term priorities and the 2008 AP listed 139. |

Of the 2006 AP short-term priorities: 80 (52 %) were repeated with little or no change in the 2008 AP.24 (15 %) were repeated in the 2008 AP but with more precise or strengthened wording.51 (33 %) were not included in the 2008 AP. |

35 of the 2008 AP short-term priorities were new or were previously medium-term priorities. |

19. In 2007, the Commission introduced a Multi-annual Indicative Planning Document (MIPD). This is a broad strategy document intended to set the main priorities that should be addressed with the IPA funds over the following three years. The MIPD is an essential tool for this purpose and has potential to better direct EU funding. However, the level of aggregation limited how specific it could be. For some areas, the document did little more than restate the relevant Accession Partnership priorities, and in one case summarised seven priorities in a single point: "Strengthening of legal and judicial protection of religious freedoms". Consequently, the MIPD did not define the objectives for EU funding in a more specific, measurable or achievable manner than the Accession Partnership.

Project selection

20. Projects are considered for selection jointly by the Turkish authorities and the Commission, and included in a financing agreement signed by both parties after adoption of the corresponding national programme by the Commission.

21. The Court assessed whether the project selection process ensured that financed projects:

(a) directly addressed Accession Partnership priorities;

(b) had specific objectives and provided for performance monitoring;

(c) were well planned and feasible to implement.

Project selection under TPA was not rigorous enough

TPA project selection was not sufficiently guided by the Accession Partnership priorities

22. An effective process is necessary to ensure that the bottom-up project identification introduced with the DIS results in projects being developed and approved that are designed to achieve Accession Partnership priorities.

23. The Court’s audit of the projects from the 2002–04 national programmes found that there was no such process at that time, and for one of the 11 projects there was no corresponding Accession Partnership priority [8]. While this was clearly a worthwhile project (see paragraph 51 and Textbox 5), there was no mention of the need for it anywhere in the Accession Partnership and at the same time, there were over 100 other priorities for which no projects were selected. The objectives and scope of a further four of the audited projects [9], did not directly address Accession Partnership priorities.

TPA project objectives and indicators were insufficient for performance monitoring

24. For every project included in a national programme a full project description known as a project fiche is required. These should set out the project’s objectives, expected results and objectively verifiable indicators as well as implementation milestones.

25. The Court tested project fiches to check whether project objectives and expected results were SMART (Specific, Measurable, Achievable, Relevant, and Time-bound) and the indicators for monitoring the project’s performance were RACER (Relevant, Accepted, Credible, Easy, and Robust).

26. Although the project fiches provided the basis of a performance monitoring system, the Court found that only two of the 11 projects (Consumer Protection and Probation Service from the 2004 NP) had SMART objectives and RACER performance indicators. The probation service project fiche, for example, set seven indicators to measure the impact of probation as an alternative to prison sentencing.

27. The absence of baselines — the starting point against which to measure improvement — made any assessment of the extent to which projects achieved their objectives almost impossible. Textbox 1 shows that the Eastern Anatolia Development Programme’s component objectives could have been measurable. However, the project fiche gave neither indicators, nor baselines against which to demonstrate achievements, or set any targets.

Textbox 1

Eastern Anatolia Development Programme (EADP)

The 45 million euro EADP had four components whose objectives were:

Agriculture: Improve rural people and organisations’ agricultural capacities, income levels and quality of life in the programme area. (No indicators or baseline data on agricultural capacity, incomes, etc. were given.)

SME: Stimulate small and medium-sized enterprise activity by capacity building at both central and local level in the application of new methodologies to long-term innovative regional development. (No indicators or baseline data on SME activity were given.)

Tourism: Promote the region as a national and international tourist destination and help preserve the natural and historical sites. (No indicators or baseline data on national/international tourist numbers were given.)

Social Development: Redress the balance in favour of women’s access to literacy and income generating activities. (No indicators or baseline data on women’s participation were given.)

TPA project feasibility was not sufficiently considered

28. Once project objectives have been determined, the equipment, facilities, actions and activities to be undertaken within the project have to be specified. Of the 11 audited projects, only two fully established the specific project requirements: maritime safety from the 2002 national programme and tax administration from the 2004 national programme. Specifications for the other nine projects were inadequate to varying degrees.

29. For example, in the phytosanitary acquis [10] project, the facilities to be provided by the Turkish authorities had not been given due consideration. It was found only after the project had started that totally new buildings and significant changes in working practices would be required. This scale of work could not be undertaken within the budget and timescale established for the project. As a result, the project did not achieve the intended laboratory accreditation, though the EU-funded equipment and training were delivered as intended.

The project selection process improved with the introduction of the IPA

30. The Court’s audit of the project selection process for the 2007 national programme found that the Commission had introduced systematic and documented checks on each project proposal prior to it being considered for inclusion in the programme.

31. As an illustration, checks were made at an early stage that the project corresponded to a priority stated in the MIPD and the Accession Partnership. Project proposals where correspondence could not be established were rejected.

32. The Commission also recognised the need to improve indicators in order to better measure the achievement of project objectives and introduced checks on their quality. Guidance was given to the beneficiaries on how to define clear objectives and indicators to measure them.

33. In addition, the Commission obliged the Turkish authorities to prepare needs assessments or feasibility studies for each project, and introduced checks to ensure that this was done. The Commission recommended an economic assessment (usually in the form of a cost–benefit analysis) of investment projects. It also required market studies for the supply components of proposed projects to assess the real necessity and the appropriate budget for equipment. The Commission gave guidance to the Turkish authorities on how to prepare such studies.

34. The Court’s analysis showed that the new measures had been followed for the 2007 national programme, but there was still room for improvement in their application. Project proposals were not assessed for how effective and efficient they were likely to be in achieving a priority. Such an assessment would allow meaningful comparison between alternatives and the selection of projects likely to have the greatest impact.

35. The new measures should enable the Commission to select projects in a way that would reduce the problems noted in the audit of the 2002–04 national programme projects. However, the impact of the improvements can only be completely assessed when the 2007 national programme projects are fully implemented (2012).

Decentralised implementation system (DIS)

36. The DIS provides the legal and administrative framework for the Turkish authorities to manage EU-funded projects under Commission supervision. This requires appropriate and adequately resourced administrative bodies — i.e. the EUSG and CFCU (see paragraph 5) — and sufficient beneficiary capacity to implement projects.

37. The Court assessed whether the DIS successfully managed the projects through:

(a) timely implementation of the audited projects from the 2002–04 national programmes;

(b) achievement of the intended outputs, and potential for sustainable outcomes.

The DIS did not achieve timely implementation of the projects audited but remedial action has since been taken

38. In 2003, the Commission assessed the capacity of the DIS structure and staffing as satisfactory and gave approval for the DIS to implement the TPA on this basis. However, in practice the DIS had insufficient qualified and properly trained staff to process the volume of budget and projects adopted in the 2002–04 National Programmes, (see Textbox 2). Despite this, the Commission adopted national programmes with increased financial amounts and numbers of projects each year (see Table 1).

Textbox 2

Examples of delays caused by insufficient DIS staffing and experience

Road Transport project from the 2004 National Programme

The technical specifications were drafted at the end of 2004, but due to staff shortages it took the CFCU seven months to provide its comments. In total, finalising and approving the tender documents took more than a year and a half.

Phytosanitary acquis project from the 2002 National Programme

The contract preparation for the technical assistance for the project took a year in total, with 10 draft versions because of the beneficiary’s inexperience in drafting such contracts. This delayed the technical assistance by some 11 months.

39. In response to the backlog, the CFCU prioritised the tendering and contracting for the older projects so as not to miss the contracting deadlines and thereby lose the EU financial allocation. The result was that the current year projects did not progress as planned, leading to systematic delays [11]. As a consequence, the implementation period for every TPA national programme from 2002 to 2005 had to be extended by one year. Part of the 2006 NP also had to be extended.

40. The Court identified that the DIS did not achieve timely implementation for 7 of the 11 projects audited from the 2002–04 national programmes and did so only partially for the other four projects.

41. The first step in the implementation of a project is tendering, which can commence once the Commission has approved the national programme. Seven of the projects audited were not ready for tendering at that stage: the beneficiary ministry had not sufficiently prepared the project specification and other documentation to be able to publish the tender notices required to find suitable contractors. For example, there was a delay of 17 months from the approval of the project in the 2004 national programme to tendering for both the Customs Modernisation and the Cultural Rights projects.

Textbox 3

Customs Modernisation project from the 2004 National Programme

The 2004 NP was approved in January 2005. The procurement notice for vehicles and equipment (including x-ray scanners) was not published until 31 August 2006 and the contract was concluded on the last possible day, i.e. on 30 November 2006. Two of the procurement lots had to be cancelled because the offers had not met specifications. As there was no time to relaunch the tendering procedure, the delays resulted in the cancellation of some 3,7 million euro (21 % of total equipment budget of 18 million euro). The Commission had to grant a one-year extension of the implementation period (to 30 November 2008) to enable the remaining part of the project to be completed.

42. The contracting phase comprises the examination of the offers received, the selection of the contractor and the signature of the contract. This is mainly the CFCU’s responsibility. None of the supply contracts from the audited sample were signed according to schedule. Delays in tendering or contracting cause a corresponding reduction in the remaining time available to execute the project activities, which had significant consequences for some of the projects (see Textbox 3).

Photo 2: For the Customs Modernisation project, the EU-funded scanners at ports and border posts to examine vehicles for illegal trafficking of humans, drugs and guns.

+++++ TIFF +++++

Source: Turkish Customs Authority.

Photo 3: Suspicious contents detected by the EU-funded scanners.

+++++ TIFF +++++

Source: Turkish Customs Authority.

43. Following the problems that arose from low staffing levels at the CFCU, the Commission made approval of the 2005 and 2006 national programmes conditional on increased staff levels at the CFCU: to 80 for the 2005 national programme and 100 for the 2006 national programme.

44. In order to reduce the risks associated with last-minute contracting, the Commission introduced interim deadlines. This began with the 2005 national programme, but was more effective for the 2006 national programme, although this was in the context of a lower number of contracts to process (see Table 3).

45. New checks were also introduced into the project selection procedure for the 2007 national programme (see paragraph 30), which aimed to reduce the incidence of delays. An assessment of beneficiary absorption capacity was made and at the final stage of the project selection procedure "project maturity" was checked to ensure that the project was ready for immediate implementation.

Table 3

Contracting data

Programme | Contracting deadline | Total number of contracts | Last month contracting |

2002 NP | 30.11.2004 | 351 | 287 | 82 % |

2003 NP | 30.11.2005 | 501 | 454 | 91 % |

2004 NP | 30.11.2006 | 752 | 688 | 91 % |

2005 NP | 30.11.2007 | 604 | 470 | 78 % |

2006 NP | 30.11.2008 | 378 | 85 | 22 % |

46. The Commission also introduced new procedures to give the head of the CFCU responsibility for ensuring that beneficiary ministries have the necessary capacity, including staff, to implement their projects. With effect from the 2007 national programme, after self-assessments by the beneficiary ministries, the head of the CFCU undertakes "mini-accreditations".

47. As this process was still ongoing at the time of the audit, the impact of this measure cannot yet be assessed. However, the Commission has already found that the beneficiaries did not receive clear guidelines and that the CFCU had no satisfactory procedure to test the self-assessments.

The intended outputs of audited projects were eventually delivered and the results assessed as likely to be sustained

48. Projects use resources (inputs) to produce output and achieve objectives (intended results and outcomes). These terms are defined in Textbox 4.

Textbox 4

Definition of project outputs and objectives

Outputs —

That which is directly produced or accomplished with the resources allocated to a project.

Examples: provision of laboratory equipment (Phytosanitary Acquis); delivery of training courses (EADP).

Results —

Immediate changes that arise from the implementation of a project.

Examples: use of laboratory equipment to test plants for diseases; women who have learned a new skill through participation in a training course.

Impacts —

Longer-term consequences that can be observed after the completion of the project.

Examples: reduced occurrence of plant diseases; improved access for women to income-generating activities.

Outcomes — Results and impacts.

Objective — Statement of the outcomes intended to be achieved.

49. As described in paragraphs 24 to 27, there was an absence of SMART project objectives and indicators so the Commission was unable to demonstrate the extent of project management success in achieving objectives. The Court’s assessment was therefore limited to examining whether there was evidence to show that projects had achieved their intended outputs. The assessment was based on a review of the monitoring reports, interviews with all 11 main beneficiary ministries and 19 site visits by the Court auditors to obtain evidence of the outputs delivered.

50. The Court found no significant weaknesses in the delivery of project outputs: they were delivered fully in 3 [12] out of the 11 projects for the contracts audited and largely in the other eight (see Table 4). The few problems found resulted from weaknesses in the project design and selection phase (see paragraphs 20 to 29) and from the delays experienced during tendering and contracting.

Table 4

Outputs AND Sustainability

Criteria:

Y = entirely met; / = mostly met; N = not met.

| Projects |

Maritime safety | Phytosanitary acquis | Police forensic | Cultural rights | Consumer protection | Biocides and water | Customs modernisation | Tax administration | Road transport | Probation service | EADP |

Outputs delivered and contributed to objectives | Y | / | / | / | / | / | Y | / | / | Y | / |

Project likely to be sustainable | Y | N | Y | / | / | / | / | Y | / | Y | / |

51. A project is sustainable when it continues to deliver results and achieve impacts after the EU funding has stopped. Within the constraints of its work due to the maturity of the projects audited, the Court assessed that the results of 10 out of the 11 projects audited are likely to be sustained (see Table 4). The only exception was the phytosanitary acquis project from the 2002 national programme. Although it achieved most of the intended outputs, there were risks to its sustainability as it had not achieved accreditation (see paragraph 29), there were staff shortages at the laboratories and the supplier of the laboratory equipment was not providing the contracted maintenance.

52. Furthermore, a project is more likely to be sustainable when there is a high level of ownership on the part of the beneficiary as is illustrated in Textbox 5. A sense of ownership arises when the beneficiary feels actively involved in the initiative, for example when they have proposed it themselves, have provided input to the planning and design, have managed the implementation and believe that they benefit from its continuation. The likelihood that these conditions are met is favoured by the DIS system in Turkey with its bottom-up project identification and beneficiary management of project implementation. Although ownership is only one of several factors influencing sustainability, the Court found that suitable conditions for sustainability — project ownership coupled with the delivery of planned outputs — existed.

Textbox 5

Probation Service project from the 2004 National Programme

The technical assistance — by far the most important element of the project — was implemented efficiently and on time. However, there were delays in the supply of office equipment: the procurement notice was published over 17 months later than foreseen in the project fiche. The impact of the delay was minimised as the Ministry of Justice supplied alternative equipment from Turkish national funds so that the probation offices could become operational.

Performance monitoring and evaluation

53. Performance monitoring and evaluation is necessary to inform decision-making and demonstrate sound financial management. The Court examined whether the Commission had a sound basis on which to review the implementation of projects and assess the achievement of project objectives and the impact of EU-funded projects in achieving Accession Partnership priorities.

Implementation problems were not always reported

54. The DIS monitored whether projects were on track through regular meetings and monitoring reports.

55. Meetings were chaired by the beneficiary of each project to monitor progress and resolve problems faced during project implementation. The Commission, CFCU and EUSG were all required to attend the meetings to receive and provide up-to-date information. Minutes of meetings were not always prepared but the Court examined a small sample of those available for the audited projects. These confirmed that day-to-day implementation problems were identified and follow-up measures were decided, but some important issues were not documented. Although the EUSG is responsible for reviewing progress in implementation and, where necessary, recommending corrective measures to the JMC, the Court identified that the EUSG was not present at any of the meetings in the sample.

56. The DIS produced two types of project monitoring report. One was the CFCU’s monthly report on tendering, contracting and payment. These went to the EC Delegation for an operational check and to the Commission’s Finance Unit, for a financial check. The other was the beneficiary’s six-monthly report on the execution of project activities which went to the EUSG. However, the Court identified that a number of significant delays and problems in project implementation were not mentioned in these reports (see Table 5).

57. The reports provided an opportunity for monitoring progress against the milestones established in each project fiche for the different stages of project implementation. However, this was not done except for the project start and end dates: there was no reporting of the date when each interim milestone was achieved. Consequently, there was no complete record at the end of a project of the extent to which it was implemented according to plan.

Table 5

Weaknesses not mentioned in beneficiary's project monitoring reports and follow-up action taken

Biocidals and Water | Weaknesses relating to the laboratory facilities provided were not mentioned in the beneficiary’s monitoring report, but were detailed in the quarterly report by the EU consultant. The beneficiary took remedial action (financed by Turkish funds) and the project was successfully completed. |

Phytosanitary Acquis | Provision of inappropriate laboratory equipment and inadequate laboratory facilities was revealed by the auditors (see paragraphs 29 and 51, Textbox 2 and Table 4). The beneficiary took remedial action to refurbish the proposed laboratory facilities at one site after the Commission’s on-site visit. |

Customs Modernisation | Delays in the installation of the x-ray scanner were not mentioned in the beneficiary’s report, as the beneficiary had not planned the construction of an additional fitting for the scanner to be mounted, which it then did out of its own funds. |

58. "Interim evaluation reports" were prepared for the JMC each year by an external company contracted by the Commission. These summarised all projects that were active, i.e. in the process of being implemented at the cut-off date for the report. Their purpose was to review the implementation of each project in terms of relevance, efficiency and likely effectiveness, impact and sustainability. This was done primarily on the basis of the DIS monitoring report and interviews. An overall report was also prepared, which made recommendations for improvements to the functioning of the DIS systems and programming. These recommendations led to several JMC decisions.

59. As the interim evaluation reports were annual and did not give any assessment of the finished projects, the information was not timely or complete enough for performance management. Combined with the weaknesses identified in the DIS project monitoring reports and the lack of aggregation of the data that was available, there was no useful monitoring information on the stages at which delays occurred or by how much. Consequently there was a lack of complete, reliable and timely management information for scheduling the CFCU annual workload, to minimise bottlenecks and backlogs and to allow rapid intervention where milestones were not being met.

Overall data to assess project results was insufficient

60. In addition to the weaknesses in the formulation of project objectives and indicators identified in paragraphs 24 to 27, the performance indicators defined in the project fiches were not used in the monitoring reports for 10 of the 11 projects audited. This included the two projects that did have some well-developed indicators (see paragraph 26). Consequently, the Commission lacked information on how successful the projects were in achieving their objectives.

61. In many cases the results and impacts of projects will not be achieved until some time after the final payment for the project has been made, although this is key for assessing the performance of spending. As there was no final project report, the Commission did not have a system for collecting data for the indicators set out in the project fiche at project completion. There was no provision for monitoring and reporting the sustainability of the project and the achievement of project objectives after the final payment date.

62. In the six years of EU pre-accession assistance to Turkey, there has been no system of ex post evaluation of individual projects or of the effectiveness of the programme as a whole in terms of meeting the Accession Partnership priorities and of progressing Turkey towards EU accession.

63. Furthermore, the fact that most strategic objectives for the pre-accession financial assistance were not specific, measurable or achievable in the time set, means that there was insufficient basis for monitoring performance of the spending including value for money.

Conclusions and recommendations

64. The Court found weaknesses in the Commission’s management of pre-accession assistance to Turkey in the TPA period. Since then, the Commission has taken action to improve procedures for the IPA, although the effectiveness of the changes can only be assessed in the future. Improvements are still required in establishing priorities for and assessing the effectiveness of the funding.

Did the European Commission ensure that EU assistance was directed to the projects that add most value in achieving the EU’s accession partnership priorities?

65. There was no mechanism to ensure that the projects proposed and selected were those that represented the best use of EU financial resources in achieving the Accession Partnership priorities.

66. The Accession Partnership sets a multitude of priorities to be achieved to prepare Turkey for EU membership. EU assistance aims to complement Turkey’s own efforts to achieve these priorities. There was no clear hierarchy of objectives from the Accession Partnership down through the annual national programmes and approved projects to the individual project components and activities. There was a lack of specific criteria and a robust framework to determine the priorities to which the EU assistance should be directed. Specific, measurable and achievable objectives for that assistance were not set and timescales were not realistic.

67. Consequently, there was not a sound basis for monitoring performance.

68. Although project selection procedures were improved considerably with the IPA, proposals were still not assessed for how effective and efficient they were likely to be in achieving a strategic objective, thereby allowing meaningful comparison between alternatives or the selection of projects likely to have the greatest impact.

Recommendation on Strategic Objectives

The Commission should improve programming with a robust methodology to determine, the strategic objectives for which the EU financial assistance is most needed. The methodology should ensure that the logic for EU intervention in achieving each strategic objective is clearly demonstrated.

Recommendation on activity statements

The Commission should determine specific objectives for the pre-accession financial assistance programme and include these objectives in the activity statement that accompanies the preliminary draft budget.

Recommendation on project proposals

The Commission should encourage the Turkish authorities to develop project proposals such that the strategic objectives for EU funding can be achieved within realistic timescales.

Did the European Commission ensure that the decentralised implementation system resulted in timely and successful achievement of project outputs and objectives?

69. Despite having been approved by the Commission, the DIS institutions were understaffed for the 2002 to 2004 national programmes and did not achieve timely implementation of the projects audited or for the programmes as a whole. Nevertheless, although beset by implementation problems and delays, the DIS ensured that the audited projects mostly achieved their planned outputs and the results were likely to be sustained.

70. The Commission has introduced measures aimed at addressing many of the weaknesses in the DIS. The full impact of these improvements can only be assessed as the IPA projects are implemented in the coming years.

Recommendation on project Design

The Commission should continue with initiatives to improve project design and implementation by the DIS institutions. Measures such as compulsory needs assessments and better scheduling of contracting should be appropriately applied.

Did the European Commission ensure that there was an effective system of performance monitoring and evaluation?

71. The Commission did not have the information to demonstrate the effectiveness of the pre-accession assistance. The Court found that the project fiches provided the basis of a performance monitoring system by setting out project objectives and expected results with objectively verifiable indicators. However, the objectives set were often not specific, measurable, achievable, relevant and time-bound and the indicators were not sufficient to monitor the achievement of the objectives.

Recommendation on project Performance

The Commission should ensure that individual project proposals have specific, measurable, achievable, relevant and time-bound objectives in order that their contribution to achieving the strategic objectives can be demonstrated.

The Commission should improve the mechanism for reporting on the implementation of projects and the delivery of their activities and outputs, and should ensure that project performance is monitored using indicators set out in the project fiche to demonstrate the achievement of the project objectives.

The Commission should ensure that project outcomes (results and impacts) are reported at the end of each project and at appropriate intervals thereafter in order to provide performance information to inform future planning.

Recommendation on Programme Evaluation

The Commission should launch an evaluation of the entire programme of pre-accession assistance to Turkey.

This report was adopted by the Court of Auditors in Luxembourg at its meeting of 22 October 2009.

For the Court of Auditors

+++++ TIFF +++++

Vítor Manuel da Silva Caldeira

President

[1] IPA legal basis: Council Regulation (EC) No 1085/2006 of 17 July 2006 establishing an Instrument for Pre-Accession Assistance (IPA) (OJ L 210, 31.7.2006, p. 82).

[2] The other components are the cross-border Cooperation component, which applies to border regions between beneficiaries from candidate and potential candidate countries and between them and EU Member States (IPA-II); and the regional, human resources and rural development components (IPA III, IV and V).

[3] The Copenhagen political criteria require stability of institutions guaranteeing democracy, the rule of law, human rights and respect for and protection of minorities. The economic criteria require the existence of a functioning market economy and the capacity to cope with competitive pressure and market forces within the European Union. For more information on accession negotiation see: http://europa.eu/scadplus/glossary/accession_criteria_copenhague_en.htm

[4] The acquis communautaire — from hereon referred to as acquis — is the entirety of EU law and constitutes about 100000 pages of primary and secondary legislation. Turkey’s administrative and judicial authorities must be trained to implement and enforce the new EU legislation.

[5] Article 33 of the Financial Regulation (and Article 24 of the implementing rules) requires activity statements containing information on the achievement of all previously set specific, measurable, achievable, relevant and timed objectives.

[6] The most recent activity statement for the Enlargement DG is at page 667 of document referenced SEC(2008) 514: Preliminary Draft General Budget of the European Commission for the Financial Year 2009 — Working Document Part I.

[7] National Programme for the Adoption of the Acquis.

[8] The project involved creating a probation service as part of the Turkish prison reform to provide a range of services to the judicial authorities and to the prison service. In this new system, an alternative to prison was introduced giving the opportunity for offenders to serve their sentences in the community.

[9] Police Forensic, Cultural Rights, Road Transport, and the Eastern Anatolia Development Programme.

[10] The Phytosanitary acquis project aimed at alignment with EU plant health legislation and practices.

[11] Previous Court reports found similar problems in pre-accession assistance:

- Special Report No 12/2008 concerning the Instrument for Structural Policies for Pre-accession (ISPA), 2000–2006;

- Special Report No 4/2006 concerning Phare investment projects in Bulgaria and Romania;

- Special Report No 5/2004 concerning Phare support to prepare candidate countries for managing the Structural Funds;

- Special Report No 6/2003 concerning twinning as the main instrument to support institution-building in candidate countries.

[12] Maritime Safety (2002 NP), Customs Modernisation (2004 NP) and Probation Service (2004 NP).

--------------------------------------------------

ANNEX I

LIST OF PROJECTS AUDITED

Project name | Budgeted EU (million euro) | Contracted (million euro) | Paid EU (million euro) |

Maritime Safety | 2 | 2 | 2 |

Phytosanitary Acquis | 4 | 3 | 3 |

Police Forensic | 5 | 4 | 4 |

Cultural Rights | 3 | 2 | 1 |

Consumer Protection | 3 | 1 | 1 |

Biocides and Water | 5 | 4 | 4 |

Customs Modernisation | 17 | 12 | 11 |

Tax Administration | 5 | 5 | 5 |

Road Transport | 5 | 4 | 3 |

Probation Service | 2 | 1 | 1 |

Eastern Anatolian Development Programme (EADP) | 45 | 43 | 43 |

TOTAL cost to EU | 95 | 82 | 78 |

| | | |

TOTAL cost to Turkey | 11 | 8 | 7 |

EU + national contributions | 106 | 90 | 85 |

--------------------------------------------------