THE COURT’S OBSERVATIONS

INTRODUCTION

6.1. This chapter presents our findings for MFF sub-heading 1b ‘Economic, social and territorial cohesion’. In 2017, for the first time, payments subject to our audit relate to the closure of the 2007-2013 and the implementation of the 2014-2020 programming periods under the new control and assurance framework.

6.2. Box 6.1 gives an overview of the main activities and spending under this heading in 2017. For further explanation on the 2017 audit population see paragraphs 6.16 to 6.19.

Box 6.1 — MFF sub-heading 1b ‘Economic, social and territorial cohesion’ — 2017 breakdown Total payments for the year 35,7 - advances ( 68 ) -9,0 + clearings ( 68 ) ( 69 ) 0,5 + annual closure decisions for 2014-2020 pp ( 70 ) 2,5 - payments for the 2014-2020 MFF period -20,8 - interim payments for non-closed OPs for the 2007-2013 pp -0,9 Audited population, total (71) 8,0 Source: 2017 consolidated accounts of the European Union, Commission closure information and data from Commission systems.

THE COURT’S OBSERVATIONS

Brief description of ‘Economic, social and territorial cohesion’

Policy objectives and spending instruments

6.3. Spending under MFF sub-heading 1b ‘Economic, social and territorial cohesion’ focuses on reducing development disparities between the different Member States and regions of the EU and strengthening all regions’ competitiveness (1). These objectives are implemented through the following funds/instruments:

— the European Regional Development Fund (ERDF), which aims to redress the main regional imbalances through financial support for the creation of infrastructure and productive job-creating investment, mainly for businesses;

— the Cohesion Fund (CF), which, in the interests of promoting sustainable development, finances environment and transport projects in Member States with a per capita GNI of less than 90 % of the EU average (2);

— the European Social Fund (ESF), which aims to encourage a high level of employment and the creation of more and better jobs, including measures through the Youth Employment Initiative (YEI) (3) targeting regions with a high youth unemployment rate;

— other smaller instruments/funds, such as the European Neighbourhood Instrument (support for cross-border cooperation and political initiatives to strengthen ties between the EU and its neighbours) and the Fund for European Aid to the Most Deprived (FEAD — material assistance to help people out of poverty).

Implementation

6.4. Member States generally submit multiannual operational programmes (OPs) at the beginning of each programming period for the entire duration of an MFF (4). After the Commission has given approval, responsibility for implementing an OP is shared between the Commission (5) and the Member State. Beneficiaries receive reimbursement through Member States’ authorities, in line with the terms of OPs, with the EU budget co-financing the costs incurred and paid of eligible operations.

Control and assurance framework

2014-2020 programming period (annual acceptance of accounts)

6.5. An important element of the new control and assurance framework for the 2014-2020 programming period (6) is that, for each OP (or group of OPs), the Member State’s programme authorities (7) must submit an annual ‘assurance package’ (8) covering expenditure certified to the Commission in the accounting year (9). Using this package, the authorities confirm that the accounts are complete, accurate and true, as well as the effectiveness of their management systems and internal controls for the OP (or group of OPs) and the legality and regularity of the certified expenditure.

6.6. A key element of the assurance package is an annual control report prepared by the audit authority. This report discloses the residual error rate (10) for the OP (or group of OPs). The audit authority issues an audit opinion on the regularity of expenditure (11). This opinion can only be unqualified if the residual error rate remains below the 2 % materiality level set in the regulation (12). To determine this rate, the audit authority takes account of any irregularities it has identified by examining a representative sample of operations, which is obtained, as a general rule, by applying statistical sampling methods (13). The rate also reflects any financial corrections applied and registered in the accounts to compensate for those irregularities.

6.7. The Commission can accept accounts which are part of the assurance package and settle the outstanding EU contribution (14) only if it is able to conclude, taking into account the audit authority’s audit opinion, that the accounts are complete, accurate and true (15). Before accepting the accounts, the Commission performs mainly administrative checks of completeness and accuracy of the information included in the assurance packages received. This means that the Commission may accept the accounts even if the residual error rate is above the materiality threshold and therefore the audit authority’s opinion on regularity is qualified or adverse. However, in that situation the Commission may also interrupt payment of the balance (which includes the release of the 10 % retention (16)) and launch the necessary financial corrections.

THE COURT’S OBSERVATIONS THE COMMISSION’S REPLIES

6.8. After accepting the accounts, the Commission may also perform regularity audits to review an audit authority’s work. If these audits reveal any irregularities or serious deficiencies, the Commission may then impose further (net) financial corrections (17). The audit scope can also be expanded to expenditure included in previous accounts with regard to the risks identified, if necessary, within the regulatory limits of the period for keeping documents available (18). 6.8. Before accepting the accounts, the Commission performs checks of their completeness, accuracy and veracity based on the information included in the assurance packages. Subsequently, it also assesses the information including the audit opinions contained in the assurance packages in relation to legality and regularity and the functioning of systems, in view of the validation of the reported error rates and as a basis for its risk assessment for regularity audits.

6.9. Following its regularity audits and checks, the Commission (DG REGIO and DG EMPL) validates and concludes on the residual error rates and publishes the results in its annual activity reports (AARs) for the following year. In their 2016 annual activity reports, DGs REGIO and EMPL stated that they would finalise this process for the 2015/2016 accounts by the end of 2017 (19) and publish the results in the 2017 AARs. 6.9. DG REGIO and DG EMPL indeed intend to validate and conclude on the residual error rates and report on these in their next annual activity reports (AARs) as a general rule. However, this is not a legal obligation and, as mentioned in the Single audit strategy of the Directorates-General, regularity audits may still occur afterwards. Audits consequently may lead to further assessment of the residual error rates and additional financial corrections, where relevant, including after the year following reception of assurance packages.

2007-2013 programming period (closure of OPs)

6.10. The controls envisaged prior to the closure of OPs in the 2007-2013 programming period are comparable to those for acceptance of an annual assurance package in the 2014-2020 programming period, with the main difference being the timeframe. When expressing its opinion on the regularity of the expenditure declared for the entire programming period, the audit authority calculates a residual error rate. To do this, it takes account of its annual reported error rates and all financial corrections made during the programming period. This year we report for the first time in our annual report on the implementation of these measures (20).

Risks to regularity

6.11. Due to the changes in the control and assurance framework, we assess the risks to regularity at the level of each operation (see paragraph 6.12) and at the level of the systems designed to produce a reliable residual error rate (see paragraphs 6.13 to 6.15).

6.12. Expenditure under ‘Economic, social and territorial cohesion’ has a high level of inherent risk. In particular, beneficiaries may declare costs that are not eligible under the EU and national rules, or aid may be granted to beneficiaries or operations that do not meet the OP eligibility requirements.

6.13. The new control and assurance framework aims to ensure that the residual error rate for expenditure reimbursed from the EU budget remains below the materiality threshold (2 %). The related main risks concern both the audit authorities and the Commission. 6.13. The 2014-2020 control and assurance framework aims to ensure that the residual error rate for expenditure reimbursed from the EU budget remains below the materiality threshold (2 %) for each operational programme.

6.14. The audit authorities may underestimate the residual error rate for an OP (or group of OPs). This might occur because they base their audit work on an unrepresentative sample of operations, they do not adhere to international audit standards or they fail to detect errors. They may also incorrectly calculate the residual error rate or propose corrections that are insufficient to address the errors they have identified.

6.15. If the Commission did not detect one or more of these problems through its regularity audits and checks, it may have settled balances even though the actual residual error rates were above the 2 % materiality threshold and thus irregular. 6.15. As a result of regularity audits and checks, if the Commission concludes that the residual error rate, taking into account all recorded financial corrections, remains above the 2 % materiality level, it may launch the necessary financial correction procedure, including net financial corrections in the event of remaining serious deficiencies.

THE COURT’S OBSERVATIONS

Audit scope and approach

6.16. Taking account of the revised control and assurance framework (see paragraphs 6.5 to 6.9), this year we changed our audit approach for this spending area. Our objective was to contribute to the overall statement of assurance as described in Annex 1.1 and to assess the new control and assurance framework and the extent to which it can be relied upon. This was carried out with a view to possibly making increased use of it in the future and, in that context, identifying where further improvements are needed.

6.17. In 2017, therefore, we audited the certified expenditure included in the 2014-2020 assurance packages and, for 2007-2013, the expenditure underlying audit authorities’ final control reports. In the past, we have sampled from interim and final payments or clearings from the EU budget to OPs.

6.18. This was the first year Member State programme authorities submitted assurance packages with expenditure for the 2014-2020 programming period and the Commission applied its procedures for reviewing and accepting the accounts, and for concluding on reported error rates. By March 2017 the Commission had received 50 assurance packages covering accounts with expenditure for 71 out of 419 OPs. In May 2017 it accepted accounts with expenditure from 17 Member States (21), covering 70 OPs and expenditure of only 2,5 billion euros (22). This explains why we covered less expenditure than last year. Furthermore, around 48 % of this amount concerned operations co-financed by the ESF/YEI. This compares to an overall ESF/YEI allocation to OPs approved for 2014-2020 of 26 %.

6.19. This was also the first year the Commission settled the balance for OPs from the 2007-2013 programming period. In 2017, the Commission closed 141 (those it considered to be lower-risk) of the 440 OPs concerned, from 18 Member States (23).

6.20. For Part 1 of this chapter, which focuses on regularity, we examined:

(i) a sample of 217 transactions, designed to be statistically representative of the full range of spending under this MFF heading. We took the sample in two stages. First, we selected 22 assurance/closure packages (24) covering 51 of the 211 OPs for which there was a Commission settlement from the EU budget (25) in 2017. Within those OPs, we then selected 217 transactions for which the audit authorities had carried out audits of operations. This work contributed to the overall statement of assurance in line with Annex 1.1 ;

(ii) the work done by audit authorities to validate the information contained in the 22 assurance/closure packages, also using the sample of 217 transactions referred to above;

(iii) the Commission’s work when reviewing and validating the 2017 assurance/closure packages. We also looked at the relevant information in the AARs of DG REGIO and DG EMPL.

Box 6.2 contains a breakdown of the audited sample by Member State, while paragraphs 6.21 to 6.23 provide more detail on how we designed our approach.

Box 6.2 — ‘Economic, social and territorial cohesion’ — Breakdown of audited sample Source: European Court of Auditors.

THE COURT’S OBSERVATIONS

6.21. Our transaction testing consisted of a review for 217 transactions of the audit authorities’ checklists and the evidence they had to substantiate the results of their audit work and their audit opinions. Where necessary, we re-performed (26) the audit work (see paragraph 6.50).

6.22. As the audit authorities’ work is a critical part of the assurance and control framework, this year we placed greater emphasis on examining the key elements of their checks. We did this in order to obtain assurance regarding the effectiveness of controls. For each of the 22 assurance/closure packages in our sample, we examined the audit authority’s audit strategy, sampling approach, how it identified, quantified and extrapolated errors, and its work on the accuracy and reliability of financial corrections implemented at Member State level.

6.23. We also looked at the work done by the two responsible DGs to review and validate the annual accounts and closure decisions, and how they used the results of this work in their AARs. This helped us to evaluate the extent to which the Commission is able to rely on the regularity information produced by the Member States.

6.24. Part 2 of this chapter focuses on performance. We assessed the Member States’ systems for measuring the performance of the 113 physically completed projects for both programming periods included in our sample. We also assessed the extent to which those projects achieved their targets.

PART 1 — REGULARITY OF TRANSACTIONS

6.25. This part of the chapter consists of three sub-sections. The first concerns our testing of this year’s sample of 217 transactions with a view to obtaining insight on the main sources of errors and providing material for the overall statement of assurance. The second sub-section concerns our assessment of audit authorities’ work, and the third relates to the Commission’s work, including the AARs of the two responsible DGs. The results for all sub-sections provide a basis for us to conclude on the regularity information contained in the AARs (see also Box 6.3 ).

Box 6.3 — The building blocks of our approach Source: European Court of Auditors.

THE COURT’S OBSERVATIONS THE COMMISSION’S REPLIES

Results of our review of transactions and re-performance of audit work

6.26. Annex 6.1 provides an overview of the results of our transaction testing. We identified and quantified 36 errors which had not been detected by the audit authorities. Taking account of the 50 errors previously found by the audit authorities and corrections applied by programme authorities worth a total of 101 million euros covering both programming periods, we estimate the level of error to be 3,0 % (27). This applies to 2014-2020 payments and to closures from 2007-2013, and is lower than in previous years. 6.26. The Commission notes with satisfaction that the ECA’s estimate of the level of error this year for Cohesion policy represents a significant improvement compared to previous years. In particular, the Commission notes that the single SME Initiative programme present in the 2015-2016 accounts, with a specific regulatory framework compared to mainstream programmes due to its innovative nature, contributes 1 percentage point to this estimate. Based on all information it has obtained, the Commission concludes that, for the vast majority of programmes, there is no remaining material level of error in the 2015/2016 accounts. The Commission will follow up the individual additional errors identified by the ECA and will request additional financial corrections from the concerned programme authorities, where necessary. The Commission further notes that its estimated amount at risk at payment for the policy, as disclosed in the annual activity reports of DGs EMPL and REGIO, is within the range calculated by the ECA for the error rate (see footnote 27).

6.27. The number and impact of detected errors indicate persisting weaknesses with the regularity of the expenditure declared by managing authorities (28). 6.27. As mentioned in its reply to paragraph 6.19 of the 2016 annual report, the Commission addressed updated guidance to Member States for the 2014-2020 programming period which, combined with the required use of simplified costs options, should contribute to improving further the quality of management verifications in the future.

6.28. Audit authorities detected a number of these errors. They reported 50 quantifiable errors in the assurance/closure packages for the 217 transactions we sampled. These errors concerned ineligible costs (30), public procurement (12) and ineligible participants (eight). The Member State authorities applied corrections with a view to bringing the residual error rates below the materiality threshold of 2 %.

6.29. This was the first year which covered both 2014-2020 payments and closures relating to 2007-2013. The number and impact of the errors we found for transactions in the 2014-2020 period were higher than for transactions relating to closure in the 2007-2013 period. Financial instruments, followed by ineligible costs, contributed most to our estimated level of error. 6.29. The Commission underlines that the financial instrument (SME Initiative) with amounts certified in the 2015-2016 accounts constituted an important part of the 2014-20 expenditure declared in these accounts. This instrument, which contributed to one third of the estimated error rate, is governed by specific rules compared to other mainstream programmes under Regulation (EU) No 1303/2013, due to its innovative nature (see in particular paragraphs 6.37 and 6.47). Under Article 42 of Regulation (EU) No 1303/2013, the eligibility of such expenditure is to be finally assessed at closure of the concerned programme and by then the fund manager should correct irregular transactions and replace them with eligible ones.

6.30. Box 6.4 shows how the errors we found break down by category (before taking account of financial corrections), and paragraphs 6.31 to 6.43 provide more information on these errors.

Box 6.4 — ‘Economic, social and territorial cohesion’ — Breakdown of the errors we found Source: European Court of Auditors.

THE COURT’S OBSERVATIONS

Financial instruments contributed most to the estimated level of error for 2017

6.31. Financial instruments under shared management (FISMs) take the form of equity or quasi-equity investments, loans or guarantees. They are different in nature to traditional grants, which involve reimbursement of a beneficiary’s spending on eligible projects.

6.32. Member States can contribute to the set-up of financial instruments:

(a) under the responsibility of a managing authority at Member State level (in both MFF periods) (29);

(b) at EU level (2014-2020 period only). This category includes the SME Initiative (30), which is implemented as a separate, individual OP (31).

6.33. Where financial instruments are used for funding, only disbursements made to final recipients and fund managers’ costs and fees are eligible at programme closure, and unused contributions must be returned to the EU budget (32). The following paragraphs describe some of the difficulties caused by this approach.

THE COURT’S OBSERVATIONS THE COMMISSION’S REPLIES

Two of the four thematic audits we examined for 2007-2013 OPs did not detect significant errors at closure

6.34. The Commission only requires audit authorities to provide assurance at closure that the amounts paid to final recipients have been used as intended (33). To achieve this, it has recommended that audit authorities carry out ‘thematic’ audits (34) for financial instruments. 6.34 and 6.35. In 2011, the Commission coordinated with the audit authorities a methodology for auditing financial instruments under shared management for the programming period 2007-2013 (‘Common Audit Framework — Financial Engineering Instruments in the Context of Structural Funds’). Since then the Commission called for audits by audit authorities on the implementation of such instruments. The Commission also verified, when assessing closure declarations, that audit authorities had audited the implementation of financial instruments. Where this was not the case, the Commission interrupted the closure process and requested additional audit work. The Commission will follow up the two cases referred to by the ECA. For the case where the ECA estimates a material financial impact of the error on the residual error rate validated by the Commission, the Commission refers to its reply to box 6.5 below.

6.35. We found that not all audit authorities systematically performed these audits. Where they did, the work done was not always sufficient to identify material irregularities. We detected quantifiable errors in two of the four financial instruments in our sample of closures. In one of these two cases, the financial impact of the error materially affected the reported residual error rate validated by the Commission (i.e. the rate exceeded 2 %).

Box 6.5 — Example of ineligible loans approved for a FISM ERDF co-financing for loans must primarily (i.e. above 50 % of the total value of loans) be used to support SMEs (Regulation (EC) No 1083/2006). The financial instrument we audited in Spain had invested almost 80 % of the approved total value of loans in companies that were not SMEs. Neither the audit authority nor the Commission detected this breach of the basic eligibility requirement set by the legislator. Box 6.5 — Example of ineligible loans approved for a FISM Article 44 of Regulation (EC) No 1083/2006 does not preclude loans to be given also to large enterprises if all other conditions are fulfilled. Since no minimum level of funding to SMEs is mentioned in this provision, this article could also refer to the number of individual loans (and not necessarily to amounts).

Serious shortcomings in the implementation of one 2014-2020 OP under the SME Initiative

6.36. For our audit, we examined 30 transactions from the sole OP under the SME Initiative in the annual accounts submitted in 2017.

6.37. The EIF is the fund manager for the SME Initiative at EU level. Audit authorities are obliged to give an opinion on the legality and regularity of the expenditure included in the accounts of OPs under the SME Initiative. However, they have no legal right to carry out on-the-spot verifications to obtain assurance on regularity at financial intermediary level (35). To allow for an opinion on the accounts, the EIF signed an ‘agreed-upon procedure’ (36) contract with an external auditor. We found, however, that the contract for the OP we examined did not provide for any work at financial intermediary level, and the external auditor’s report included a disclaimer to the effect that it could not express any assurance. Despite this, the audit authority issued a positive opinion on the regularity of the accounts submitted to the Commission (37). 6.37. In an Agreed Upon Procedure, the external auditor does not issue an opinion but the audit findings and conclusions are to be used by the audit authority to issue its own audit opinion. For this purpose, the audit authority also carried out a system audit, at the level of both the managing and certifying authorities. The annual control report describes clearly how the audit authority reached its opinion. The Omnibus proposal, in its drafting resulting from the political agreement between the Council and Parliament, addresses this assurance gap.

6.38. We also detected deficiencies that seriously affected the eligibility of a number of investments made by financial intermediaries (38). Only companies meeting the EU definition of SME status (39) are eligible for the SME Initiative. However, financial intermediaries either did not always check this or did not check it correctly. In the 30 investments we examined, financial intermediaries had approved loans to five recipients without confirming their SME status; we therefore deem these investments to be ineligible. Another four loans were used, in part or in full, for ineligible activities (e.g. for refinancing other loans). These irregularities accounted for 1,0 percentage points of our estimated level of error for this chapter (40).

6.39. The situation was caused by procedural inadequacies at the financial intermediaries, which did not respect the legally binding eligibility criteria set out in the contracts signed with the EIF. This represents a serious regularity risk for other OPs approved under the SME Initiative and managed in similar circumstances. It also shows that the EIF’s ongoing monitoring requires further improvement. To that end, the EIF has undertaken to implement an action plan in order to raise awareness among financial intermediaries about the need to comply with the eligibility criteria, and to further develop its own internal mechanisms for detecting ineligible loans. The action plan aims to address the irregularities we found and mitigate the residual regularity risks for SME Initiative OPs. 6.39. In financial instruments, the controls to be carried out by the financial intermediaries constitute an essential element in the assurance process. The Commission takes note of the comprehensive set of measures proposed by the EIF to address these issues. It will closely monitor their implementation in cooperation with the national authorities so that expenditure eligible in line with Article 39(8) CPR is declared and accepted at closure of the SME Initiative programme.

Recoverable VAT incorrectly certified as eligible expenditure

6.40. Value-added tax (VAT) is an indirect tax levied on the consumption of goods and services. A taxable person, generally a business, sole trader or other professional, is obliged to charge VAT on its sales and forward it to the national tax authority. The taxable person can request a refund of VAT already paid on its own purchases. This is the concept of ‘recoverability’. A non-taxable person is not obliged to charge VAT and cannot recover VAT on purchases. 6.40 and 6.41. The concept of VAT recoverability needs to be assessed on a case-by-case basis. The Commission has prepared guidance for the use of programme authorities and has made proposals to simplify eligibility of VAT for the 2021-2027 programming period.

6.41. According to the EU rules, VAT expenditure is eligible for co-financing only if it is not recoverable under national VAT legislation. It is irrelevant whether or not VAT expenditure is actually recovered. In the case of public bodies, VAT is recoverable whenever an operator charges fees with VAT to the ultimate users of infrastructure (41).

6.42. We found 10 projects in five OPs in which the Member State authorities had declared recoverable VAT as eligible expenditure. In those cases the audit authorities interpreted the eligibility rules incorrectly and did not report the situation as an irregularity. This contributed 0,3 percentage points to the estimated level of error. For three of these five OPs, the financial impact of the errors materially affected the reported residual error rate validated by the Commission (i.e. the rate exceeded 2 %). In our 2015 annual report we highlighted the controversy of including recoverable VAT in declarations of expenditure, especially in the case of public bodies (42). In this case, EU funds reimburse VAT as an eligible cost to Member State authorities and at the same time, the Member State collects this VAT via its normal taxation system. In some cases, the EU reimbursement may even exceed the actual costs incurred for the project, net of VAT. 6.42. Nine out of the 10 projects mentioned by the ECA relate to four 2014-2020 ESF OPs in one Member State. The Commission had already identified this problem in previous audits, but the adaptation of the Member State’s system did not eliminate the problem in its full extent. The Commission is currently working closely together with the Member State’s administration to ensure that this matter could be resolved in the near future through the introduction of a simplified cost option or the introduction of the obligation of VAT recovery for final recipients. The Commission will apply the necessary financial corrections.

Box 6.6 — Example of co-financing for ineligible VAT expenditure In Poland, the managing authority signed grant agreements to help small businesses purchase equipment, with the overall aim of reducing unemployment. Where recipients stated that they did not intend to recover VAT, the managing authority assessed that the VAT expenditure was eligible for co-financing. This was incorrect, since the recipients were registered for VAT and the VAT expenditure was actually recoverable.

Ineligible expenditure

6.43. We detected significant weaknesses in checks on the eligibility of participants in 11 projects. This particularly affected the YEI and the requirement concerning NEET status (43). We also found seven cases in which the audit authorities had not detected ineligible or overstated indirect costs. 6.43. For the cases detected with ineligible or overstated indirect costs, the Commission will do the necessary follow up and will propose action as it deems necessary. Concerning indirect costs (see box 6.7), the issues identified by the ECA were to a large extent identified by the audit authority, who applied a 10 % financial correction on this expenditure. The Commission will follow up with the relevant authorities and will consider the implementation of any additional financial correction that may be required.

Box 6.7 — Example of ineligible expenditure — overdeclaration of indirect costs The beneficiary, a large research institute in Germany, has its headquarters outside the eligible area for the OP. The audited project was financed by the ESF and implemented by a dedicated project group set up by the beneficiary in the eligible area. The project costs consisted mainly of direct spending on staff, materials and services, and a significant proportion of indirect spending under the same headings. To be eligible, indirect costs should be incurred in the eligible area, have a direct relationship with the project and they should be allocated based on their contribution to the project. We found, however, that most of the cost categories allocated to the project as indirect costs were in fact general costs incurred at the institute’s headquarters. Several costs relating to these categories were also directly charged to the project as actual expenditure incurred by the project group at the project location. During its audit, the audit authority was unable to obtain sufficient evidence about the composition of the indirect costs; as a result it applied a 10 % flat-rate correction to indirect costs. We found that the actual error was greater than 10 %, so we consider all costs not directly related to the project to be ineligible.

Our assessment of the work of audit authorities

6.44. The work of audit authorities is a critical part of the assurance and control framework of Cohesion spending (see paragraph 6.22). This was the first year audit authorities implemented closures for the 2007 2013 programming period and the new control and assurance framework for 2014-2020. Our review of their work is part of a process with a view to possibly making more use of the Commission’s assurance model in the future. In this year’s audit we assessed the work of seven out of 21 audit authorities for the 2014-2020 period from Member States which, in 2017, submitted assurance packages with expenditure. We found a number of shortcomings which affect the extent to which their work can currently be relied upon. 6.44. The Commission welcomes the ECA’s audit approach, which has the potential to overall reduce duplication and overlaps of audit activities for programme authorities and beneficiaries. As in the previous programming period, the Commission continues to work closely with the 122 audit authorities covering ERDF/CF/ESF/YEI/FEAD to prevent possible shortcomings and ensure fully reliable audit results. In addition, the Commission's single audit strategy for Cohesion policy foresees a strong focus on re-performance work on the results of audit authorities on a rolling and risk basis. See also common Commission reply to paragraphs 6.49 and 6.50.

6.45. In all of the assurance and closure packages we examined, the audit authorities had reported a residual error rate below 2 %. However, because of the additional errors we detected, our recalculated rate was above 2 % for six of the 12 assurance packages for the 2014-2020 period and three of the 10 closure packages for 2007-2013. 6.45. For three programmes in one Member State, without contesting the irregularities raised, the Commission notes that the additional errors identified leading to a residual error rate above 2 % are all linked to a misinterpretation on VAT eligibility by all the Member State’s authorities and not by the audit authority alone. Furthermore, in the case of the SME Initiative programme specific legal requirements were in place at the time of issuing the audit authority's report which did not allow the audit authority to carry out audits at the level of transactions. See also Commission replies to paragraphs 6.29 and 6.37.

Weaknesses in some audit authorities’ sampling affected the representativeness of samples

6.46. Due to the large number of operations co-financed by each OP, audit authorities have to use sampling to obtain an opinion on the eligibility of expenditure. To produce reliable results, samples must be representative of the audited population and, as a general rule, based on a statistically valid method (44). We checked the audit authorities’ sampling method for the 22 assurance/closure packages we examined.

6.47. In the case of the OP under the SME Initiative, there was no sampling because neither the audit authority nor the external auditor contracted by the EIF carried out any audits at financial intermediary level (see paragraph 6.37). 6.47. There is a clear regulatory restriction for the audit authority to carry out audits at the level of final recipients in relation to the SME Initiative programmes in Article 40(3) CPR. The Commission refers to its reply to paragraph 6.37.

6.48. We found weaknesses, such as insufficient sample size, overrepresentation of certain operations and incorrect use of sampling parameters, in three of the 22 packages we examined (all of them for 2014-2020). To some extent, these weaknesses affected the representativeness of the samples, and hence also the reliability of the reported residual error rates concerned. 6.48. See common Commission reply to paragraphs 6.64 and 6.65.

Other shortcomings in audit authorities’ work

6.49. Insufficient or inadequate checks increase the risk of not detecting ineligible expenditure. Furthermore, the international audit standards (45) require auditors to document their checks, including clear references to all documents that are most relevant to the audited expenditure. This enables them to be accountable for their work and helps internal or external reviewers to conclude on the extent and sufficiency of the checks. 6.49 and 6.50. Audit authorities should be able to demonstrate the scope and quality of the verifications they carried out. However, when it comes to documentation of the audit process, an appropriate balance should be found to ensure sound and efficient administrative procedures. The Commission will keep working closely with the audit authorities in order to address the issues raised by the ECA through sharing of best practices and exchange of audit tools and methodologies, including the Commission’s own checklists. Under its Single Audit Strategy for Cohesion policy, the Commission has planned to conduct 58 compliance audits and 19 fact-finding missions on a risk basis in 2018 and 2019 in order to ensure the reliability of the audit authorities’ work. It is envisaged that 21 of these compliance audits and eight fact-finding missions will be carried out in 2018 and the results of these will be available for the 2018 AARs. See also Commission reply to paragraph 6.26.

6.50. We were able to conclude for 96 out of the sample of 217 transactions (44 %) on the basis of our own review of the audit authorities’ work. We identified shortcomings with the scope, quality and documentation of the audit authorities’ work in 121 transactions (56 %), which required us to re-perform the audit work. The shortcomings related to 17 of the 22 assurance/closure packages we examined. They concerned, for example, checklists that did not cover all necessary points, or incorrect checks of the eligibility of expenditure. For 80 of these transactions (37 % of the total), this meant we had to visit the beneficiary. In 30 of the 121 transactions we re-performed, we found quantifiable errors that had not been previously identified by the audit authority. These related to 11 packages.

The Commission’s work and its reporting of the residual error rate in its annual activity reports

6.51. AARs are the Commission’s main tool for reporting whether it has reasonable assurance that the control procedures put in place ensure legality and regularity of expenditure (46). For ‘Economic, social and territorial cohesion’, the AARs give a number of different rates as a measure of the expenditure at risk. Those rates include the residual error rate for the accounting year 2015/2016, for which the Commission has accepted the accounts and on which it has concluded in terms of regularity (47). 6.51. The Commission provides different rates in the AARs in relation to expenditure under shared management to measure the specific risks linked to different types of payments in the reporting year (2007-2013 closure and different accounting years for 2014-2020 expenditure), as well as an aggregated error rate for the calendar year 2017.

6.52. In this sub-section, we examine the Commission’s work to obtain assurance that the Member States’ systems under the new control and assurance framework are functioning effectively and that the resulting residual error rate is a reliable estimate.

6.53. Our observations relate to the design of the assurance framework and the Commission’s work in accepting accounts, validating individual residual error rates and establishing an overall residual error rate.

The design of the 2014-2020 control and assurance framework

The AARs should be further streamlined and adapted to the new control and assurance framework

6.54. The new control and assurance framework means that it takes almost two years from the end of the relevant accounting period before the Commission can first report its conclusion on the reliability of audit authorities’ residual error rates for a given accounting year. This period includes eight months for the audit authorities to carry out their audit work. Subsequently the Commission carries out its desk reviews, complementary fact-finding missions and regularity audits. Box 6.8 presents the timeline for the new framework. 6.54. The two years referred to by the ECA start at the end of the relevant accounting period and therefore include the eight months foreseen by the regulation for the audit authorities to carry out their audit work. Subsequently, and in line with its Single Audit Strategy for Cohesion Policy, the Commission aims to assess on time for the subsequent annual activity report, i.e. within a maximum of 13 months, the assurance packages received for each of the 419 Cohesion programmes. Following such thorough desk checks and on the spot, risk based audits and fact-finding missions, the DG will report in the subsequent annual activity report whether it considers the residual error rates previously reported as reliable or if further verifications are ongoing or needed. See also the common Commission reply to paragraph 6.49 and 6.50.

Box 6.8 — Timeline for the new control and assurance framework Source: European Court of Auditors.

THE COURT’S OBSERVATIONS THE COMMISSION’S REPLIES

6.55. Last year we drew attention to the fact that the AAR reporting period is not the same as the period covered by the Member States’ assurance packages (48). The residual error rates reported by audit authorities relate to the accounting year. In the AARs, however, to reflect the budgetary principle of annuality, and following the guidance provided by DG BUDG and the Secretariat-General, the directors-general provide reasonable assurance for the calendar year. For this purpose, in the 2017 AARs, the directors-general of DGs REGIO and EMPL decided to estimate the risk by projecting a provisional residual error rate for 2017 calendar year expenditure, which they had not yet accepted and validated. 6.55. The Commission refers to its reply to paragraph 6.34 of the ECA 2016 Annual report. The Financial Regulation and the timeframe set for the assurance for 2014-2020 allow for a better alignment of reporting periods, with only a six months gap compared to one year in 2007-2013. The DGs have adapted accordingly their annual activity reports and the way they obtain assurance for the different types of payments made, which cover up to 3 different programme accounting years, as described in the 2016 AARs (section A.1.2). The residual error rates for the 2016-2017 accounting year, received by 1 March 2018, were considered to be the best and most recent estimates for both calculating the risk on the expenditure of the year and for entering reservations on individual programmes, where necessary. This was particularly the case in 2017 due to the limited number of accounts with expenditure received and therefore error rates reported for the 2015-2016 accounting year. Furthermore, the Commission underlines that, at the time an AAR is signed off, expenditure declared in the previous calendar year is covered by the 10 % retention on EU interim payments.

6.56. The reporting requirements for AARs have not been sufficiently adapted to the new control and assurance framework. In our view, given that the residual error rates for the 2015/2016 accounting year are the only ones for which the Commission could have the necessary assurance from audit authorities and its own regularity work, they should be the main indicator for regularity in the 2017 AARs. They should also be the main basis for the DGs’ declarations of assurance for the specific policy areas. However, neither DG gives them sufficient prominence or mentions them as a key performance indicator. 6.56 and 6.57. The DGs report in the AARs a key performance indicator on regularity (KPI 5) in line with central instructions to ensure a consistent and harmonised approach across all budget areas. Additional visibility was given to the residual risk rate in relation to the 2015-2016 accounts in the 2017 AARs (including in footnote to KPI 5 and in the executive summary), where a specific section is also devoted to the validation of reported residual error rates for that accounting year. The Commission is open to reflect on, and to discuss with the ECA, how to further improve and streamline its presentation of error rates in its future AARs.

6.57. Furthermore, in the AARs the Commission presents at least 13 different rates for the two programming periods as a measure of the expenditure at risk. Such a large number of rates leads to a lack of clarity and potential confusion as to their relevance and the assurance provided.

Advances to financial instruments should not be taken into account for the calculation of the residual error rate

6.58. The set-up for the 2014-2020 programming period aims to simplify the closure of OPs by introducing a procedure for the annual examination and acceptance of accounts that is similar to ‘partial closure’. This can only work if the residual error rates reported by audit authorities are reliable and information is available that refers exclusively to eligible expenditure at closure (i.e. without advances). 6.58. The Commission has indeed indicated that annual accounts aim at further legal certainty for Member States in a multiannual context under shared management by introducing the validation of ‘annual blocks of eligible expenditure’. Such partial closure does not by definition cover advance payments made under financial instruments or State aid that should be further transformed into eligible expenditure in subsequent accounting years (cumulative reporting is therefore foreseen in annual accounts). In the AARs the Commission services have provided such information on residual error rates that refer only to eligible expenditure at closure (i.e. without advances from financial instruments). See also common Commission reply to paragraphs 6.59 to 6.62 below.

6.59. Regulation (EU) No 1303/2013 does not specify how to calculate residual error rates. 6.59 to 6.62. In line with Article 137 of Regulation (EU) No 1303/2013, programme accounts are based on the amounts declared to the Commission, including advance payments to financial instruments. Audit authorities provide residual error rates based on the expenditure included in the accounts. To constructively follow up on the 2016 ECA recommendation, audit authorities agreed to provide to the Commission additional detailed information allowing it to calculate in the AARs residual error rates excluding the impact of advances paid into financial instruments. Thus, the Commission could calculate the impact of advances paid in the 2016-2017 accounts to financial instruments (and to SME Initiative programmes) to be less than 0,2 percentage points on average for ERDF (based on 67 financial instruments) and 0,02 for ESF (based on 15 financial instruments) (see page 50 of DG REGIO’s 2017 annual activity report and page 58 of DG EMPL’s 2017 annual activity report). The ECA’s recommendation therefore helped improving the disclosure of information for individual programmes in the AARs, without such advance payments to financial instruments having shown so far a material overall impact on the reported error rates.

6.60. The audit authorities’ current practice of including advances to financial instruments in the audit population is very likely to lead to an understatement of the residual error rates reported to the Commission (49).

6.61. Acting on our recommendation in last year’s annual report (50), in its AARs the Commission now provides additional residual error rates which exclude advances to financial instruments.

6.62. In previous programming period, the audit authorities, did not usually carry out audit work at financial intermediary level. For the 2014-2020 period, for the second and subsequent tranches of advances (51), audit authorities will need to verify that the eligibility conditions are met at the level of financial intermediaries. Partial closure cannot take place for OPs that include financial instruments unless the audit authorities carry out appropriate work at this level.

6.63. This problem may be more acute in future years for those programmes where the number of financial instruments increases significantly (52).

The Commission’s work on assurance in the AARs

Sampling weaknesses remained unaddressed in the review of assurance packages

6.64. Our work on the audit authorities’ sampling methodology showed that weaknesses in three of the 12 assurance packages we examined (25 %) affected the reliability of the reported residual error rates to some extent (see paragraphs 6.47 to 6.48). 6.64 and 6.65. The Commission identified some of the issues raised by the ECA, but did not consider that the conditions of Article 31 of Regulation (EU) No 480/2014 applied to these cases. For the SME Initiative programme, the Commission refers to its reply to paragraph 6.47.

6.65. In line with its internal procedures, the Commission checked the sampling method for all the packages submitted. However, it did not raise the issues referred to above. Where there are serious shortcomings, the regulation provides for a flat-rate correction (53).

6.66. In all the above cases, the Commission settled the balance and released the 10 % previously retained to protect the EU budget (see Box 6.8 ). 6.66. The Commission has to settle the outstanding amount for the accounting year in question in all cases when conditions foreseen in Article 139(2) CPR are fulfilled. Where this results in a payment to be made, the Commission has a possibility to interrupt it where the residual risk remains material. When the balance results in a recovery, the Commission must proceed to recover the amount without delay, in line with the principle of sound financial management.

The Commission’s conclusion on the regularity of expenditure in the 2015/2016 accounting year is not yet final

6.67. The Commission undertook in its 2016 AARs to carry out its in-depth assessment on regularity no later than nine months after the submission of assurance packages (54). This would have meant concluding on the residual error rates for the 2015/2016 accounting year in the 2017 AARs. Both DGs concluded that the residual error rate for the 2015/2016 accounting year was below the 2 % materiality threshold. DG EMPL did not finally conclude on some OPs for which it had not yet obtained the required assurance itself, and which were therefore still subject to its final assessment. 6.67. There is no formal obligation to conclude on the validity of reported error rates for all programmes each year but the DGs are clearly working with this objective, when all conditions are fulfilled and allow for such a conclusion to be made by the Commission services. The Single Audit Strategy for Cohesion Policy foresees to carry out audits, where necessary, up to the end of the regulatory period for beneficiaries to keep supporting documents. Therefore specific situations (like for the ‘running-in’ of this first year of implementation of the assurance system) or administrative efficiency may lead Commission compliance audits to cover more than one accounting year, with possible bearing on the revision of previously reported error rates.

6.68. The 2015/2016 accounting year was the first under the new control and assurance framework. For the 50 relevant assurance packages submitted by March 2017, DG EMPL carried out just two compliance audits (55) from June 2017 to April 2018 covering eight operations and two assurance packages in two Member States. DG REGIO carried out no such audits. We found that audit authorities underestimated the residual error rates which they reported to the Commission in six of the 12 assurance packages we examined. To protect the EU budget, the regulation gives the Commission the possibility to carry out further audits until the end of the retention period for supporting documents (see paragraph 6.8). These two elements could imply that other deficiencies detected in future Commission audits may further increase the residual error rates for the 2015/2016 accounting years. Consequently, the residual error rates on which the Commission concludes in the 2017 AARs are not yet final. 6.68. In order to protect the EU budget, the Commission has the possibility to carry out further audits taking into account the legal requirement for the availability of documents and audit evidence. Any irregularities identified, as a result of such audits need to be appropriately corrected with possible bearing on previously validated error rates. See also Commission reply to 6.9. In 2017, DGs REGIO and EMPL accompanied ECA on a significant number of its audits. The low number of assurance packages actually submitted in this first accounting year with limited expenditure declared and therefore limited possibilities to re-perform audits carried out by audit authorities, coupled with the extent of ECA's audit coverage of these programmes, caused the Commission's decision to revise its planned compliance audits in 2017. The purpose of this decision was to avoid duplicating the audits on the same programmes and beneficiaries. Other factors, including the possibility for future Commission compliance audits to look back over 3 accounting years (for administrative efficiency in case of low numbers of audits to be reviewed) and the overlap with the REGIO/EMPL audit work on closure of the 2007-13 period, were taken into account by the Commission, from both an efficiency and effectiveness perspective, when planning (and subsequently reviewing) its 2017 audit work. See also common Commission reply to paragraphs 6.56 and 6.57.

There is no overall residual error rate for Cohesion

6.69. For both programming periods, the Commission’s AARs each give a separate residual error rate (56). However, the Commission does not calculate and provide an overall residual error rate covering the ‘Economic, social and territorial cohesion’ area (MFF sub-heading 1b) for the accounting year. 6.69 and 6.70. As required by the Financial Regulation, the Commission's reporting is based on the calendar year. The Commission’s corporate annual management and performance report (AMPR) provides an aggregate residual error rate for cohesion, migration and fisheries based on data and indicators reported by each DG in their respective AARs. This approach has been the same since the 2015 AMPR for comparability reasons.

6.70. The Commission could produce this information with little additional effort in the Annual Management and Performance Report for the EU budget (AMPR), which would help the discharge authorities. Instead the AMPR provides a combined error rate for ‘Cohesion, migration and fisheries’ for the calendar year.

6.71. In Box 6.9 we provide a summary of the information provided by the AARs on the 2014-2020 programming period.

Box 6.9 — Overview of information in the AARs for the 2014-2020 period (million euros) Activity Fund Total expenditure certified in the assurance package Number of assurance packages (72) OPs (72) Residual error rate (73) 2014-2020 programming period ERDF/CF 993,7 20 34 0,6 % ESF/YEI (74) 1 443,5 33 49 0,7 % FEAD 144,3 8 8 0,6 % Total without SME Initiative (75) 2 581,5 50 70 0,7 % Source: 2017 AARs of DG REGIO and DG EMPL.

THE COURT’S OBSERVATIONS THE COMMISSION’S REPLIES

6.72. The reported residual error rates in 2017 AARs for the 2015/2016 accounting year are below the materiality level (57). However, the results of our audit of a representative sample of operations audited by audit authorities indicate that: the level of error remains material (above 2 %) and that a number of residual error rates submitted by these audit authorities were understated. 6.72. The Commission has reported the issues raised by the ECA in the respective annual activity reports of DG REGIO and DG EMPL and taken them into account in its validated error rates, where it considered it possible. For the individual programmes concerned, where it is concluded that the residual error rates reported were understated, the Commission will take additional financial corrections.

CONCLUSION AND RECOMMENDATIONS

Conclusion

6.73. The overall audit evidence indicates that the level of error in spending on ‘Economic, social and territorial cohesion’ was material (see paragraphs 6.26 to 6.43). 6.73. The Commission notes with satisfaction that the ECA’s estimate of the level of error for Cohesion policy, this year, represents a significant improvement compared to previous years. In particular, the Commission notes that the single SME Initiative programme present in the 2015-2016 accounts, with a specific regulatory framework compared to mainstream programmes due to its innovative nature, contributes 1 percentage point to this estimate. Based on all information it has obtained, the Commission concludes that, for the vast majority of programmes, there is no remaining material level of error in the 2015/2016 accounts.

6.74. The number of weaknesses we found in the work of several audit authorities covered by our sample (see paragraphs 6.44 to 6.50) currently limits the reliance that can be placed on that work. 6.74 and 6.75. The Commission has already reported some of the issues raised by the ECA in the respective annual activity reports of DG REGIO and DG EMPL and taken them into account in its validated error rates, where it considered it possible. The DGs will apply the necessary additional financial corrections. The Commission will keep working closely with the audit authorities in order to address the issues raised by the ECA. In addition, the Commission’s Single Audit Strategy for cohesion foresees a strong focus on reperformance work on the results of audit authorities on a risk basis. See also Commission replies to paragraphs 6.26, 6.29, common Commission reply to 6.34 and 6.35, 6.37, 6.39 and common Commission reply to 6.49 and 6.50.

6.75. Because of this and shortcomings in the Commission’s own work (see paragraphs 6.51 to 6.72), the overall residual error rates presented for the accounting year 2015/2016 in the Commission’s AARs are underestimated, and we cannot currently rely on them. Due to the additional errors we detected, our recalculated rate was above 2 % for six of the 12 assurance packages for the 2014-2020 period and three of the 10 closure packages for 2007-2013.

6.76. The new control and assurance framework has been designed with a view to ensure that the residual error rates are below the materiality threshold of 2 % on an annual basis. However, our audit showed that further improvements are necessary in particular in terms of the framework’s implementation at both audit authority and Commission level. 6.76. Without prejudice to its common reply to paragraphs 6.74 and 6.75 above, the Commission also considers that further improvements are always welcome and will take the necessary steps to improve further the assurance framework and ensure that all irregularities are timely detected and corrected. As regards the Commission's own work, the Commission's audit plans must take into account the work planned by the ECA with a view to reduce overlaps and duplications of audits at the level of the responsible administrations and beneficiaries. This was particularly the case for the first 2014-2020 accounts with limited expenditure certified and therefore with limited possibilities to re-perform audits carried out by audit authorities. The Commission also refers to its reply to paragraph 6.68.

Recommendations

6.77. Annex 6.2 shows the findings of our follow-up review of the seven recommendations we made in our 2014 annual report (58). The Commission had implemented four recommendations in full, while two had been implemented in some respects and one had not been acted upon at all. 6.77. Regarding recommendation 3 from the 2014 ECA's annual report, the ECA had noted that the use of simplified cost options (SCOs) is limited to 36 % of the programme budget for ESF and to only 2 % for ERDF/CF. The Commission keeps promoting the use of simplified cost options during the 2014-2020 programming period. In this regard, the Commission has published on 27 March 2018 its final report on the use of SCOs in ESF, ERDF, CF and EAFRD) which showed that between 2014 and 2017 the large majority of ESIF managing authorities used SCOs (73 % and 95 % for respectively ERDF/CF and ESF programmes). In terms of projects, 65 % of ESF, 50 % of ERDF and 25 % of CF projects are using SCOs. Moreover, the Omnibus proposal, in its drafting resulting from the political agreement between the Council and Parliament, broadens the scope and applicability of SCOs. As for the recommendation deemed not to be acted upon at all, the Commission underlines that this recommendation was not accepted and therefore was not implemented.

6.78. Based on this review and our findings and conclusions for 2017, we recommend that the Commission:

— Recommendation 1: ensure that the audit arrangements for financial instruments managed by the EIF are adequate at the level of financial intermediaries. When the EIB/EIF uses agreed-upon procedures with external auditors, the Commission should define the minimum conditions of such contracts with a view to the need to provide assurance, in particular the obligation for sufficient audit work at the level of the Member State. Implementation date: immediate. The Commission accepts the recommendation and considers that with the entry into force of the Omnibus proposal, in its drafting resulting from the political agreement between the Council and Parliament, and with new provisions for audit of financial instruments managed by EIF, the recommendation will be implemented. Moreover the Member States will have to carry out verifications and audits at the level of financial intermediaries and, for audit authorities, at the level of final recipients in their jurisdiction, where relevant.

— Recommendation 2: propose legislative changes for the post-2020 financial framework which would exclude reimbursement of VAT to public bodies from EU funds. Implementation date: before approval of the post-2020 legislative framework. The Commission partially accepts the recommendation and has made a legislative proposal for the financial framework 2021-2027. The Commission proposal (COM (2018) 375 final) introduces a simple rule in relation to VAT, independently from the private/public status of beneficiaries: for projects below a total cost of EUR 5 million the VAT is considered eligible while above the threshold the VAT is ineligible.

— Recommendation 3: address the weaknesses we have identified in its verification of the audit authorities’ work in the context of the Commission’s regularity audits. Implementation date: immediate. The Commission accepts the recommendation. In line with their Single Audit Strategy and agreed audit plan, the Commission services will from 2018 on focus their audit work on verifying the quality and compliance of audit authorities' work with a view to obtaining fully reliable audit opinions and results.

— Recommendation 4: address the complexity of the information presented on the 2014-2020 control and assurance framework in the AARs of DGs REGIO and EMPL, by: (i) focusing on expenditure that has gone through the control cycle, i.e. assurance packages covering expenditure before 30 June ‘n-1’. For this purpose, the Commission should adjust its reporting to ensure that it does not present provisional estimates; (ii) clearly indicating which expenditure has not yet been examined in depth (including regularity checks). It should indicate which preventive actions protect the EU budget and whether these actions are sufficient to provide assurance, but refrain from calculating a residual rate for expenditure that has not yet been examined. (iii) disclosing an overall residual error rate for MFF sub-heading 1b for each accounting year. Implementation date: June 2019. The Commission partially accepts this recommendation. The Commission underlines that the annual activity reports of the concerned Directorates general already provide a clear focus on the validation of error rates for programmes and accounts in relation with accounting years ending 30 June n-1. This can be made even clearer in future reports. In line with the Financial Regulation, the assurance required from authorising officers in the annual activity reports refers to all types of payments executed in the reporting year n. DGs report separately on the level of assurance obtained for each of them. The amount at risk is therefore established on the basis of the relevant expenditure of the calendar year. The Commission reiterates furthermore that it is open to reflect on, and to discuss with the ECA, how to improve and simplify its system leading to the presentation of error rates in its AARs.

— Recommendation 5: ensure that audit arrangements are changed in accordance with the proposal made by the Commission for financial instruments in the post-2020 regulatory framework so that only the actual use of funds at final recipient level is used for the calculation of residual error rates. Implementation date: before implementation of the post-2020 legislative framework begins. The Commission accepts this recommendation to the extent the Commission’s proposal for the post-2020 regulatory framework is adopted by the co-legislators. Following this proposal adopted on 29 May 2018, the Member State will declare expenditure of financial instruments based on disbursed amount to final recipients. There will be only one advance payment of 25 % to financial instruments that should not be part of the audited population defined in the regulatory framework which will limit the expenditure which may be declared before payment is made to final recipients.

— Recommendation 6: carry out sufficient regularity checks to conclude on the effectiveness of audit authorities’ work and obtain reasonable assurance on the regularity of expenditure at the latest in the AARs it publishes following the year of accepting the accounts. Implementation date: immediate. The Commission partially accepts this recommendation which is in line with its Single Audit Strategy for the concerned funds. The Commission, has set as an objective to be able to report each year in the respective AARs on the reliability of the residual error rates reported by audit authorities in the previous year's assurance packages. It underlines at the same time that it has up to 3 years to carry out audits on the expenditure included in the accounts to conclude on the reliability of the error rates reported by audit authorities and to confirm the legality and regularity of the certified expenditure.

THE COURT’S OBSERVATIONS

PART 2 — ASSESSMENT OF PROJECT PERFORMANCE

6.79. The principle of sound financial management in the implementation of the EU budget presupposes a focus not only on regulatory compliance but also on achieving the stated objectives (59). For this reason, in addition to checking the regularity of transactions, we assess the performance of the operations to which those transactions belong.

6.80. Of the 217 transactions we examined this year, 113 came from operations that were physically complete at the time of our audit (56 for 2007-2013 and 57 for 2014-2020). For these operations, we assessed:

— the way the performance system was designed (in particular whether the OPs had output and result indicators that were relevant to their objectives, and whether the output and result objectives specified in project documents (60) corresponded to the OP objectives for each priority axis);

— The actual performance of projects, i.e. whether they had reported on whether the targets set for each indicator were met and whether the objectives were achieved.

6.81. For financial instruments, we examined the actual disbursement rate (the amount disbursed to final recipients as a share of total funds).

Assessment of performance system design

6.82. The EU regulations relevant to this spending area require beneficiaries to define and report on outputs (61). Member States also have the option of defining result indicators that link project results to the corresponding indicators for the OP priority axis. Setting result indicators, wherever possible and meaningful, is good practice as it allows the authorities to measure the specific contribution made by a project to objectives of the corresponding priority axis.

THE COURT’S OBSERVATIONS THE COMMISSION’S REPLIES

6.83. On examining the sample of 113 projects, we found a number of weaknesses in the way Member State authorities had designed their performance measurement systems at both OP and project level. These findings are summarised in Box 6.10 . We found that a performance measurement system was broadly in place to link project outputs and results to the OP objectives for only 74 of the 113 projects (65 %). In five of the 74 cases, however, the authorities had wrongly classified outputs as results in the OP and/or the project approval documents. 6.83. A well-designed intervention logic as described in the operational programme shall ensure that the outputs and results at project level contribute to achieving the expected results of the operational programmes, bearing in mind that these are also influenced by external factors.

6.84. In 34 cases the authorities had not defined result indicators at project level, and in four they had not set indicators or targets to measure performance at project level at all. In one further case, the corresponding OP itself had not defined any targets. In these circumstances, it is not possible to determine whether these projects contributed to the overall programme objectives. Around three quarters of these 39 cases relate to the 2007-2013 period. 6.84. Assessing the contribution of the co-financed intervention (reflected by the output and project level result indicators) to the OP objectives (reflected by the result indicators) is a task for impact evaluation. Indeed, the insufficient focus on results has been identified in the ex post evaluation of the CF/ERDF. This has been addressed in the legislation governing the programming period 2014-2020 through strengthening of the intervention logic and focus on results.

6.85. These findings complement our recent report on project selection and monitoring (62) in the 2014-2020 programming period. The report concluded that the selection criteria for ERDF and ESF projects in the 2014-2020 period do not systematically require the definition of quantified result indicators at project level. Even where such indicators exist, they do not necessarily correspond to the OP indicators. 6.85. The Commission considers that there is a difference between direct results at the level of projects financed and the results to be achieved at the level of the OP (reflected by the result indicators). Whereas direct results are influenced only by the action co-financed, results indicators are affected by different external factors outside the control of the managing authorities. A well-designed intervention logic shall ensure that the outputs and direct results of the selected projects contribute to achieving the expected results of the OP.

6.86. Our report on absorption (63) also shows that, for the 2007-2013 period, the Commission and Member States took action to tackle slow absorption and ensure legality, but failed to take due account of performance considerations, which led to insufficient focus on results. 6.86. The Commission proposed measures to Member States being at risk of under-absorption which were in line with the 2007-2013 legal framework, the programme objectives and intervention logic and were adapted to the specific circumstances for each programme concerned. Therefore, there appeared to be no need for any exceptional consideration for results only for safeguards that the proposed results are indeed delivered. It is standard programme management practice that when a programme shows indication of a lower absorption rate, the actions taken to address these issues focus on absorption within the existing targets.

Box 6.10 — Assessment of performance system design (113 projects) Source: European Court of Auditors.

THE COURT’S OBSERVATIONS THE COMMISSION’S REPLIES

Many projects do not fully achieve performance objectives

6.87. In most of the cases we examined, we did not conclude that all the performance objectives had been fully achieved. For the 74 projects where both result and output indicators had been set, we found that these had been fully achieved in 26 cases (35 %), partially achieved in 43 cases (58 %), could not be assessed in two cases (3 %) and were not achieved in three cases (4 %). One of the projects that did not achieve its objectives was ‘non-functioning’ according to the definition in the closure guidelines (64). 6.87 and 6.88. It is important to recall that the ECA sample concerned a mixture of 2007-2013 and 2014-2020 projects, with a significant change to the performance approach and programmes intervention logic under the 2014-2020 regulation. The reported results are therefore providing a limited view of the specific results for 2014-2020 so far. However, altogether the Commission notes that 93 % of projects with both result and output indicators had fully or partially met them, which is in line with previous years' results. The figure was even as high as 100 % in cases where only output indicators had been set.

6.88. Of the 34 cases where output indicators had been set (but not result indicators), we found that these had been fully achieved in 23 cases (68 %) and partially achieved in 11 cases (32 %) (see Box 6.11 ).

6.89. We were unable to assess the achievement of performance objectives in the remaining five cases because no indicators or targets had been set.

Box 6.11 — Achievement of performance objectives Source: European Court of Auditors.

6.90. In the 2014-2020 period, audit authorities are required to examine the reliability of performance data. We found that they generally carry out checks of project performance during their audits of operations. In 18 out of 57 cases the audit authority was unable to form an opinion on the achievement of objectives because the project was still ongoing. In 20 other cases the audit authority had limited its verifications to output objectives. 6.90. The Commission notes with satisfaction that audit authorities have included this additional element of performance data reliability into the scope of their audits on operations for 2014-2020, in line with the Commission guidance and recommendations and with the regulatory framework. These results are useful and complementary to the own thematic audits in this area carried out by the Commission.

THE COURT’S OBSERVATIONS THE COMMISSION’S REPLIES

Most Member States were unable to use their FISM contributions in full

6.91. Member States are required to report annually on the actual disbursement of funding from FISMs to final recipients (65). Box 6.12 shows how FISMs were used up to the last reporting period (66). Only four Member States fully used the available contributions, with substantial variation in disbursement rates. For the 2014-2020 period, 16 Member States reported no disbursements to final recipients as of the end of 2016 (67). 6.91. Regarding the 2014-2020 period, in most of the Member States the funding agreements were signed in 2016 and even in the second part of 2016. The fact that no disbursements were made to final recipients by end 2016 is thus in line with the timing of the set-up of those financial instruments.

Box 6.12 — Cumulative disbursement rates of financial instruments Note (1) Ireland and Luxembourg had not set up any financial instruments by end 2016. Note (2) Disbursement rates reported to the Commission in accordance with Article 67(2)(j) of Regulation (EC) No 1083/2006 and Article 46 of Regulation (EU) No 1303/2013. For 2014-2020, this covers the calendar year and not the accounting year. Note (3) In the 2014-2020 programming period Member States cannot usually request further OP contributions to financial instruments until they reach a certain level of disbursement to final recipients. Source: European Court of Auditors based on Commission information.

THE COURT’S OBSERVATIONS THE COMMISSION’S REPLIES

Conclusion