# EUR-Lex - 52014SC0293 - EN

*|*

# 52014SC0293

**COMMISSION STAFF WORKING DOCUMENT Summary of Executive summaries Internal audit engagements finalised by the IAS in 2013 Accompanying the document Report from the Commission to the European Parliament and the Council Annual report to the Discharge Authority on Internal audits carried out in 2013 (Art 99(5) of the Financial Regulation) /* SWD/2014/0293 final */**

			*

			*

Content
of this annex. 3

1.         Statistical
data. 3

1.1.          Implementation of the 2013 audit plan. 3

1.2.          Statistical data on IAS
recommendations. 4

2.         Horizontal
audits. 5

2.1.          Management letter on the delegation
of new tasks to Executive Agencies (multi-DG). 5

2.2.          Limited review - Performance of
Coordination with EU Decentralised Agencies (DGs SANCO, HOME, JUST, ENV, MOVE,
ENTR). 8

2.3.          Performance Audit on the
Effectiveness of HR management to support the financial crisis in DG ECFIN, DG
COMP, DG MARKT. 9

3.         Agriculture,
Natural Resources and Health. 16

3.1.          Limited review of residual error rate
calculations in DG AGRI (pillar 1 and 2). 16

3.2.          Audit on Control Strategy -
Implementation in DG AGRI 18

4.         Cohesion. 22

4.1.          Implementation of ESF 2007-13 (DG
EMPL). 22

4.2.          Performance Audit of DG REGIO
Performance Measurement Systems. 25

4.3.          Performance Audit of DG EMPL
Performance Measurement Systems. 28

5.         Research,
energy and transport 33

5.1.          Limited Review of the calculation and
the underlying methodology of DG RTD's residual error rate for the 2012
reporting year. 33

5.2.          Joint IAS-IAC Audit on Grant
Management in TEN-T EA.. 35

5.3.          European Economic Recovery Plan (DG
ENER). 36

5.4.          Audit on Control Strategy in EACI 39

5.5.          Implementation of FP7 Control Systems
in REA.. 42

6.         External
Aid, development and enlargement 45

6.1.          Limited Review of DG DEVCO's
Methodology and Calculation of the  Residual Error Rate (RER)  45

6.2.          DG DEVCO's state of preparedness for
the revised ECA DAS Methodology (Performance Audit)  48

6.3.          Audit on Financial Management
Humanitarian Aid (DG ECHO). 49

6.4.          Procurement – decentralised (DG
DEVCO). 52

7.         Education
and citizenship. 54

7.1.          Performance audit of National
Agencies (DG EAC). 54

7.2.          Performance Audit on Risk Management
in DG COMM... 57

8.         Economic
and financial affairs. 59

8.1.          Performance Audit of GMES/Copernicus
Programme (DG ENTR). 59

9.         General
services and HR.. 62

9.1.          DG ESTAT's preparedness to fulfil its
role in the Economic Governance Framework (Performance Audit)  62

9.2.          Performance audit on the efficiency
and effectiveness of the design and implementation of the financial circuits in
OIB. 66

10.      IT
audits. 68

10.1.       Performance Audit on Management of Local
IT in DG ESTAT. 68

10.2.       Agrex 3 (DG AGRI). 72

10.3.       Performance Audit on the management of
the security of EU ETS IT system (DG CLIMA, DG DIGIT and DG HR Security
Directorate). 72

10.4.       SYGMA - Phase 1 - audit on development
process (DGs RTD and CNECT) (Performance Audit)  76

11.      Follow-up
engagements (summarised) 79

11.1.       2nd Follow-up on DG DEVCO audit -
Financial Management of main programmes in Latin America  79

11.2.       2nd Follow-up on DG DEVCO audit -
Financial Management of main programmes in Neighbourhood   79

11.3.       2nd Follow-up on DG DEVCO audit -
Implementation of the framework agreement with UN agencies  80

11.4.       2nd Follow up of AFIS (OLAF). 80

11.5.       2nd Follow up on the financial
management of the Common Foreign Security Policy budget (FPI)  80

11.6.       Follow-up on Management of Guarantees
(multi-DG). 80

11.7.       Payment deadlines (BUDG, ENER/MOVE,
DEVCO, ECHO) (including IT part). 81

11.8.       Follow-up of Joint Audit on
Implementation by the EIF of the High Growth and Innovative SME Facility (GIF)
under the Entrepreneurship and Innovation Programme (EIP) in DG ECFIN.. 82

11.9.       Follow-up on the Spot Controls/Fraud in
DG CNECT. 82

11.10.     2nd Follow-up on DG DEVCO audit -
Financial Management of Regional Programmes. 83

11.11.     2nd Follow-up on DG DEVCO audit -
Eligibility of Costs under the Financial and Administrative Framework Agreement
with the United Nations by DG AIDCO.. 83

11.12.     Follow-up on Risk Management Implementation
(SG, BUDG, RELEX, ENV). 83

11.13.     Follow-up on Business Continuity
Management (DG DIGIT). 84

11.14.     2nd Follow-up on Set up of Internal
Control and Financial Management Systems in REA.. 84

11.15.     Follow up of Control Strategy audit (TEN
T-EA). 84

11.16.     2nd Follow up of Operational Budget
management audit in the TEN-T EA (recommendation addressed to DG MOVE). 84

11.17.     2nd FU on Recoveries (multi-DG). 84

11.18.     Follow up on the Monitoring and
Supervision of DG EAC National Agencies. 85

11.19.     2nd Follow up of Enterprise Europe Network
Audit. 85

11.20.     2nd Follow up: Management of the
operational budget of EAHC. 85

11.21.     2nd Follow up on Ex-post control
activities and implementation of financial circuits in EACEA   85

11.22.     2nd Follow up: Procurement in EAHC. 86

11.23.     Management of local IT - the 2nd follow up
(SJ). 86

11.24.     2nd follow-up on Treasury and Accounting
System (DG ECFIN). 86

11.25.     2nd follow up audit on Grant Management of
Schengen Facility (DG HOME). 86

11.26.     Follow up audit on EFF Control Strategy
(DG MARE). 86

11.27.     2nd follow-up on the management of
procurement contracts in OIL. 86

11.28.     Management of local IT - the 2nd follow up
(PMO). 87

11.29.     Follow up on Development of IT systems
 to support Horizon 2020 programmes under DG RTD, Part I (URF/PDM &
SEP). 87

11.30.     Follow up of audit on Fraud Prevention and
Detection (DG HOME). 87

11.31.     Follow up of audit on Fraud Prevention and
Detection (DG JUST). 87

11.32.     2nd follow-up of Security of IT
environment in subcontracted projects (DG REGIO). 87

11.33.     Follow up Programme Estimates (DG DEVCO). 88

11.34.     Follow up EDF grants (DG DEVCO). 88

11.35.     Follow up of Competitive Activities audit
(JRC). 88

11.36.     2nd Follow up on HR management, part II
(DG HR). 88

11.37.     Follow up of EIP Performance audit (DG
ENTR). 89

11.38.     2nd Follow up of GNSS Audits (DG ENTR). 89

11.39.     HR IT Corporate Application - NAP - the
Follow-up (PMO). 89

11.40.     Follow up of Control Strategy audit (DG
ENER). 89

11.41.     Follow up: Procurement (OIB). 90

11.42.     Follow up of Local IT in DG EMPL. 90

11.43.     Follow-up on Thematic Budget Lines (DG
DEVCO). 91

11.44.     Follow-up on Fraud prevention and
detection in OLAF. 91

11.45.     2nd Follow-up on DG DEVCO audit -
Financial Management of main programmes in Asia. 91

11.46.     2nd Follow-up on DG DEVCO audit - Ex-Post
Control Activities. 91

11.47.     Follow-up of IPA Grants (DG ELARG). 92

11.48.     Follow-up on Monitoring of Public
Procurement in the Member States (DG MARKT). 92

Content of this annex

This annex contains the introduction, the objectives
and scopes, the strengths and the critical and very important recommendations
of the original executive summaries reflecting the state of play when the audit
engagements were finalised (cut-off date for the exercise 31/01/2014).

Each executive summary underwent the applicable
standard professional validation and contradictory procedures between auditor
and auditee at the time of the finalisation. It aims to provide a quick understanding
of the audit and its main results.

1. 
Statistical data
1.1. 
Implementation of the 2013 audit plan[1]

By the cut-off date of 31 January 2014, the IAS.B had
implemented[2]
100% of its planned engagements (target 100%).

Eighty-seven reports (including audits, follow-ups,
limited reviews and one management letter) were finalised, and can be broken
down as follows:

|| 2013 || 2012 || 2011

|| Engagements || Reports || Engagements || Reports || Engagements || Reports

Audit || 22[3] || 23 || 29 || 49 || 20 || 23

Follow-up || 48 || 59 || 32 || 37 || 41 || 50

Limited Review || 4 || 4 || 1 || 1 || 3 || 3

Management Letter || 1 || 1 || 1 || 1 || ||

Consulting || 0 || 0 || 1 || 1 || 1 || 1

Total[4] || 75 || 87 || 64 || 89 || 65 || 77

In order to ensure an
efficient and effective implementation of the audit plan, the IAS plans its
audit work on the basis of a risk assessment and a capacity analysis. The
implementation is then regularly monitored and adjustments are made as
necessary.

1.2. 
Statistical data on IAS recommendations[5]

The
IAS issued the following number of recommendations (including their acceptance
rate) in 2013:

|| New recommendations || Accepted recommendations || Non-accepted recommendations

Priority || || || % || || %

Critical || 2 || 2 || 100% || 0 || 0%

Very Important || 59 || 59 || 100% || 0 || 0%

Important || 67 || 67 || 100% || 0 || 0%

Desirable || 6 || 6 || 100% || 0 || 0%

Total || 134 || 134 || 100% || 0 || 0%

For
all accepted recommendations, the auditees drafted action plans, which were
submitted to and assessed as satisfactory by the IAS.[6]

With respect to
the period 2009-2013, the auditees reported that 79% of the accepted
recommendations were implemented in the beginning of 2014 as follows:

Year || Priority || Total || Implemented[7] || In progress (by number of months overdue)

Nr || % || Nr || % || No delay || 0 - 6 || 6 - 12 || 12+

2009 || Critical || 2 || 2 || || 0 || || 0 || 0 || 0 || 0

Very Important || 131 || 130 || || 1 || || 0 || 0 || 0 || 1

Important || 143 || 141 || || 2 || || 0 || 0 || 0 || 2

Desirable || 13 || 13 || || 0 || || 0 || 0 || 0 || 0

2009 Total || 289 || 286 || 99% || 3 || 1% || 0 || 0 || 0 || 3

2010 || Critical || 2 || 2 || || 0 || || 0 || 0 || 0 || 0

Very Important || 120 || 113 || || 7 || || 1 || 0 || 0 || 6

Important || 150 || 139 || || 11 || || 0 || 1 || 0 || 10

Desirable || 11 || 11 || || 0 || || 0 || 0 || 0 || 0

2010 Total || 283 || 265 || 94% || 18 || 6% || 1 || 1 || 0 || 16

2011 || Critical || 0 || 0 || || 0 || || 0 || 0 || 0 || 0

Very Important || 47 || 44 || || 3 || || 0 || 1 || 0 || 2

Important || 101 || 93 || || 8 || || 0 || 0 || 1 || 7

Desirable || 10 || 10 || || 0 || || 0 || 0 || 0 || 0

2011 Total || 158 || 147 || 93% || 11 || 7% || 0 || 1 || 1 || 9

Year || Priority || Total || Implemented[8] || In progress (by number of months overdue)

Nr || % || Nr || % || No delay || 0 - 6 || 6 - 12 || 12+

2012 || Critical || 0 || 0 || || 0 || || 0 || 0 || 0 || 0

Very Important || 81 || 49 || || 32 || || 24 || 7 || 1 || 0

Important || 110 || 77 || || 33 || || 18 || 10 || 3 || 2

Desirable || 0 || 0 || || 0 || || 0 || 0 || 0 || 0

2012 Total || 191 || 126 || 66% || 65 || 34% || 42 || 17 || 4 || 2

2013 || Critical || 2 || 1 || || 1 || || 1 || 0 || 0 || 0

Very Important || 59 || 4 || || 55 || || 55 || 0 || 0 || 0

Important || 67 || 5 || || 62 || || 61 || 1 || 0 || 0

Desirable || 6 || 0 || || 6 || || 5 || 1 || 0 || 0

2013 Total || 134 || 10 || 8% || 124 || 93% || 122 || 2 || 0 || 0

TOTAL 2009-2013 || 1055 || 834 || 79% || 221 || 21% || 165 || 21 || 5 || 30

Out
of all recommendations rated 'very important' or 'critical' and issued in the
period 2009-2013, ten very important recommendations (2.3%) were overdue by
more than six months with respect to the deadline set in the initial action
plan[9].
Three additional overdue very important recommendations were issued before 2009[10].
No critical recommendation is outstanding. The Audit Progress Committee (APC)
was regularly informed of critical or very important recommendations overdue
for more than six months.

The total number
of accepted recommendations issued during 2009-2013 and for which the IAS had
conducted follow-up audits by the end of 2013, amounts to 686 (65%).

The follow-up work
carried out by the IAS confirmed that overall recommendations are being
implemented in a satisfactory way thus contributing to the improvement of the
control systems in the audited services. The IAS closed 96% of the
recommendations followed-up during this period.

2. 
Horizontal audits

2.1. Management letter on the delegation of new tasks to Executive Agencies (multi-DG)

Introduction

In order to concentrate on its
institutional tasks, such as policy-making and strategic management, the
Commission has delegated responsibility for the implementation of some
Community programmes to third-party bodies (executive agencies). This has
proved to be successful in terms of cost savings, the provision of better
service delivery (reduced time for contracting, more rapid approval procedures
for technical and financial reports, lower payment delays) and simplification
of processes. It has also resulted in increased external communication and
dissemination of results, which contribute to enhance the visibility of the EU.

The
next Multiannual Financial Framework (MFF) will bring new challenges due to the
financial and economic crisis, constraints in public spending and a 5%
reduction of staff in each EU Institution, body and agency over the period
2013-2017. To address these challenges, the Commission has committed, amongst
others, not to create new executive agencies for the implementation of
programmes but to make use of the existing ones through, where appropriate, an
extension of their mandate.

Objectives
and scope

The purpose of this management letter
addressed the main issues of a horizontal nature raised in the 2010 IAS
overview report on executive agencies and subsequent audits conducted in this
field and to help those responsible for the design and implementation of the
process to delegate tasks to executive agencies to mitigate the attendant
risks. It was not intended to provide an assessment of the state of
implementation of the recommendations contained in these reports nor to provide
an opinion on the design or effective implementation of on-going actions or
processes not yet subject to an audit, such as the Cost Benefit Analysis
currently being performed.

Main
issues for consideration

Cost
benefit analysis

The cost benefit analysis used to assess
the various options should be based on a well-defined and consistent
methodology, including the use of appropriate tools and methodologies to
support the staff allocation process (data on workload, productivity). It
should take account of the risks/impact of the transfer of new programmes (e.g.
their complexity) to the agencies. Where accurate and/or reliable data cannot
be obtained, a sensitivity testing of all options should be performed with a
clear indication of the range of assumptions used to address the uncertainties.
In addition, the selected option should be regularly confirmed throughout the
life of the project.

Roles
and responsibilities

As part of the delegation of programme
management process, the current supervisory and support functions should be
reviewed to ensure the most efficient use of staff of both parent DGs and
agencies based on the maturity of the agencies' internal control systems. In
addition, appropriate indicators should be developed to enable an assessment of
the effectiveness of the supervision strategy. As executive agencies grow in
size in terms of staff and reach a critical mass of programmes to be managed,
consideration should be given to alternative scenarios to achieve further cost
savings and efficiency gains by pooling together various agencies’ horizontal
services such as IT, HR or communication .

Monitoring
performance

Greater focus should be placed on
improving their performance to meet the rising expectations of stakeholders.

Human
resource management

Given the scale of the delegation of
programme management exercise, appropriate steps should be taken to ensure the
continuous availability of sufficient contractual agents with the required
profile in the long run together with a policy on the retention of agency
staff. In addition, consideration should be given to ensuring that the
conditions for the secondment of Commission officials to executive agencies are
attractive enough. Central services should also further develop the framework
for the management and monitoring of staff allocation.

IT

Guidance to executive agencies regarding
the acquisition of hardware and the development of software should be further
developed to ensure compatibility with the Commission's infrastructure and
architecture. In addition, executive agencies should be more integrated in the
Commission's IT governance framework by, in particular, requiring them to
describe their IT projects in Business Cases, Vision documents and progress
reports which are submitted to the Information Systems Projects Management
Board (ISPMB) for approval.

Guidelines

Executive agencies
should be reminded of the need to comply with appropriate guidance such as the
development of a roadmap for the handover of programmes (including a readiness assessment
and update of Business Continuity Plans and control strategies).

2.2. Limited review - Performance of Coordination with EU Decentralised Agencies (DGs SANCO, HOME, JUST, ENV, MOVE, ENTR)

The
IAS limited review on the Performance of Coordination with EU Decentralised
Agencies was included, as a result of a risk assessment; in the 2012 IAS-IAC
coordinated Audit Work Programme. The engagement was announced in July 2012 and
was planned to be started in the latter part of 2012 but, due to other on-going
engagements, it commenced only in 2013.

The objective of the limited review was to
assess the performance of the Commission regarding its
relationship / supervision / coordination with EU
decentralised agencies, and any risks arising from the agencies' activities to
which the Commission may be exposed. The scope included all decentralised
agencies, their parent DGs and horizontal Commission services providing
administrative support to them.

The European Parliament, Council and
Commission adopted in July 2012 a comprehensive set of guiding principles
— a "common approach[11]",
in order to make the agencies more coherent, effective and accountable.
Moreover, in this respect, the Secretariat-General of the Commission prepared
in December 2012 a "roadmap on the follow-up to the Common Approach with
concrete timetables for the planned initiatives[12]",
including an action list with deadlines set for the implementation of remedial
measures tackling the risks identified to which the Commission may be exposed,
whether of an operational, reputational, safety, efficiency or governance
nature. All actions are expected to be implemented by the end of 2014.

During the fieldwork, the IAS held
interviews with both a sample of parent DGs[13] and
horizontal Commission services providing administrative support to
decentralised agencies[14], and
sent a survey to a sample of 16 agencies[15]. The
IAS also relied on previous IAS, Internal Audit Capability (IAC) and European
Court of Auditors (ECA) audit reports.

The fieldwork confirmed that almost all
risks in the above categories had already been satisfactorily addressed by the
Secretariat-General in its "roadmap".

2.3. Performance Audit on the Effectiveness of HR management to support the financial crisis in DG ECFIN, DG COMP, DG MARKT

(Three
audit reports: DG COMP, DG ECFIN, DG MARKT)

General background

Since 2008, the global economy has been
facing a financial crisis involving banking systems, stock markets and the flow
of credit. This has turned into a sovereign debt crisis for the EU with a subsequent
crisis of confidence in the Euro-zone. The Commission has been engaged in
designing, negotiating and implementing policy responses.  In this context, a
number of DGs are involved in identifying and managing solutions to handle and
mitigate the consequences of the financial crisis.

Audit Objectives

The objective of the audit was to assess
whether the Human Resources Management (hereafter referred to as HRM) of the DG
has been effective in responding to HR challenges resulting from the policy
responses to the financial crisis and enabled the DG to achieve its operational
objectives in dealing with the financial crisis situation.

In particular, the
audit focused on the following topics:

a)         The
adequacy of the HR planning, allocation and selection

b)         The adequacy of the monitoring,
reporting and follow-up of HR matters.

In addition, the IAS looked at the level
of coordination and dissemination of good management practices on HR matters
between DG COMP, DG ECFIN and DG MARKT.

Audit Scope

This audit engagement focused on the
procedures, methods and tools in place for HRM in relation to the DG's
contributions to tackle the financial crisis and related staff allocations.
However, where the DG had not implemented any distinct process for activities
related to financial crisis policy responses, the audit analysed the standard
process existing since the beginning of the financial crisis, regardless of the
type of resources/activities managed (whether or not related to the financial
crisis).

In 2013 IAS performed a comprehensive
audit on “Management and monitoring of staff allocation in the Commission
services" which also included DG COMP. Processes and controls analysed as
part of that audit have not been included in the scope of this audit.

The Task Forces for Cyprus and Greece, though administratively part of DG ECFIN, have not been included in the scope of
this audit.

Audit
report addressed to DG ECFIN:

Background

As one of the DGs that are involved in
identifying and managing solutions to handle and mitigate the consequences of
the financial crisis, DG ECFIN is responsible for:

- 
Program
country support based on instruments (such as EFSF[16]/EFSM[17] and
ESM[18]),
operating in five[19] euro
area Member States (MSs) and Balance-of-Payments assistance for two non-Euro
countries[20].

- 
New
economic governance instruments providing a stronger role for the
Commission to vet national draft budgets (e.g. "6-pack",
"2-pack" and fiscal compact treaty).[21]

- 
Financial
market regulation that creates measures to restructure
banks, reregulate the financial sector and prevent future crises.

To be able to deal with the new
challenging responsibilities deriving from the financial crisis and fully
assume its role in reinforcing EU economic governance, DG ECFIN has received
from the central services a substantial reinforcement of human resources[22] (in
the form of annual and complementary allocations) with a level of flexibility
to be able to recruit many new staff in a short period of time. The DG has also
adapted its structure and operations to this new environment, by implementing a
major reorganisation and creating three new Directorates[23].

From a human resource management point of
view, those changes have required the DG to increase its flexibility and
capacity in order to adapt to its new tasks by systematically redefining
priorities, redeploying staff[24] and
reducing staff allocated to the administrative tasks to concentrate the
resources on the DG’s core activities[25].

Overview of Major
Strengths

DG ECFIN has launched a number of
strategic HR initiatives aimed at addressing the important challenges it faces
in terms of HRM in the recent years:

- 
An
HR Management Board was established to define and coordinate DG ECFIN's overall
HR policy. It is composed of the DG, the two DDGs, the Resource Director and
the Head of Unit R1.

- 
Together
with DG MARKT and DG COMP, and with the support of DG HR, DG ECFIN launched a
Commission-wide call for expression of interest in November 2008 to reinforce
on a temporary basis existing teams in the three DGs. An additional call for
interest was launched in 2011 particularly for DG ECFIN.

- 
The
DG has recently launched a 360° feedback mechanism for middle and senior
managers to develop a feedback culture in DG ECFIN, and help managers improve
their HRM capacities.

- 
DG
ECFIN invested substantial resources in the specialised competition for
macroeconomics and financial economics specialists launched by EPSO to build a
more stable and qualified labour force for the next years. In particular, the
DG developed detailed competency profiles for economists and participates in
the selection board of the competition.

- 
ECFIN
has revamped its Learning & Development strategy, in particular by
improving the training offer (now open to many other DGs), developing ECFIN
training passports to provide guidance to ECFIN staff and by introducing new
delivery model (short courses, a summer school, course linked to the timing of
the surveillance cycle).

Major Findings

The IAS has identified the following three
very important issues:

HRM Strategy (Observation 1)

HR-related strategic issues are regularly
discussed at senior management level, in particular in the HR Management Board
meetings and formalised in different documents including the status of the
reflection, the identified challenges and the actions planned/taken. These
strategic documents do however not contain any qualitative and quantitative
(multi-annual) analysis of staffing needs[26] and do
not prioritise the actions identified to address the HR challenges. In
addition, there is neither a monitoring nor a follow-up mechanism to measure
their level of implementation and effectiveness (deadlines, targets and KPIs[27]).

Annual HR Planning (Observation 2)

DG ECFIN's HR unit coordinates the annual
resource planning based on discussions among the different levels of management
and within the HR Management board. The HR unit prepares the detailed
justification for requests for additional posts or credits. However, DG ECFIN
does not have at its disposal comprehensive data to support the (possible)
re-allocation of resources in the context of the annual resource planning.

Monitoring and Reporting on HRM
(Observation 3)

DG ECFIN provides management with a number
of HRM related statistics and reports (e.g. vacancy reports, screening of local
overhead, equal opportunity figures, HR budget consumption) both in the context
of the HR planning exercises and in response to ad-hoc requests. However, there
is no comprehensive HR report that would allow regular monitoring and measuring
of HR-related performance and exhaustive and periodic reporting to management
based on KPIs and targets agreed with senior management.

Audit
recommendations

In order to
address these issues, following very important recommendations were issued:

HRM Strategy – Very
important

DG ECFIN should further develop its
multi-annual HR strategy based on a qualitative and quantitative analysis of
staff needs, including the identification of possible options to address the
staff reduction. The implementation of the HR strategy should be supported by tools
to measure, monitor and report on its effectiveness.

Annual HR Planning – Very
important

DG ECFIN annual resource planning process
should be based on complete, reliable and up-to-date information on the
existing HR and their allocation to the DG's tasks and associated priorities
(so-called “task mapping”). In addition, the DG should implement tools to
assess regularly the staff workload in order to align the staff allocations
with tasks, priorities and workload identified.

Monitoring and Reporting on HRM –
Very important

DG ECFIN should enhance its monitoring and
reporting process on the HRM related activities by developing a comprehensive
HR report that covers the different aspects of HRM and measure and monitor the
related KPIs. This would enable the DG to measure HR management’s performance
and facilitate management decisions on HR.

DG
ECFIN established an action plan which the IAS considers adequate to address
the identified issues.

Audit
report addressed to DG COMP:

Background

As one of the DGs that are involved in
identifying and managing solutions to handle and mitigate the consequences of
the financial crisis, DG COMP plays a leading role in tackling the crisis, in
particular by using State aid rules as a key coordination instrument. In addition,
it participates in the coordination of the financial market regulation, and is
active in maintaining the stability of the financial system as a whole.

To be able to deal with the new
challenging responsibilities deriving from the financial crisis, DG COMP has
received a substantial reinforcement of human resources from the central
services (in the form of annual and complementary allocations[28]
as well as derogation to speed up the recruitment process) and has adapted its
structure and operations to this new environment, including the management of
human resources. In particular, the DG has created the "Task
Force Financial Crisis" to deal with the additional workload,
redefined its priorities refocusing its activities on files related to the
financial crisis and has systematically redeployed its staff (by regrouping
people from different units to work on priority files). In addition to the
matrix organisation already in place in DG COMP, these changes have helped the
DG to react/adapt quickly to new requests/more urgent tasks.

Overview of Major
Strengths

DG COMP has launched a number of strategic
HR initiatives aimed at addressing the important challenges it faces in terms
of HRM in the recent years, notably:

- 
Together
with DG ECFIN and DG MARKT, and with the support of DG HR, DG COMP launched a
Commission wide call for expression of interest in November 2008 to reinforce
on a temporary basis existing teams in the three DGs.

- 
Following
the request of DG COMP, EPSO launched in 2010 a specialist lawyers and economists
competition to provide a more stable and qualified labour force for the years
to come. DG COMP invested a substantial amount of resources by participating in
the selection panels and developing detailed competency profiles.

- 
In
2009 and 2012, DG COMP organised temporary agent AD5 and AD8 competitions for
the Chief Economist Team, which reserve lists were shared with DG MARKT.

- 
DG
COMP has developed a comprehensive workload assessment tool (called PETRA) which allows timely reporting on HR allocation and workload. The information
provided by this tool is an integrated part of DG COMP's HR planning processes.

- 
DG
COMP has developed a knowledge-sharing platform called Comp-Wiki to disseminate
information of an operational nature. Staff involved in case handling update it
on a daily basis. It is also used by newcomers to become quickly operational,
as it includes case summaries and sector-related information.

- 
DG
COMP has developed a comprehensive learning and development framework and a
rolling training programme based on an extensive internal consultation launched
in 2009.

Major Audit
Findings

The
audit in DG COMP did not identify any issues that gave rise to critical or very
important recommendations.

Audit
report addressed to DG MARKT:

Background

As one of the DGs that are involved in
identifying and managing solutions to handle and mitigate the consequences of
the financial crisis, DG MARKT is at the forefront of the activities to tackle
the financial crisis. It works on structural measures to address key reforms
necessary in the development and the completion of the Single Market, which is
a central element of the European growth agenda. In particular, the focus is on
rules for bank recovery and resolution and consumer protection in financial
services.

To be able to deal with the new
challenging responsibilities deriving from the financial crisis, DG MARKT has
received from the central services a substantial reinforcement of human
resources[29]
(in the form of annual and complementary allocations) as well as derogations
to speed up the recruitment process. The DG has also adapted its structure and
operations to this new environment. In particular, the DG implemented a major
reorganisation and created three new units and one principal advisor’s post.

From the HRM point of view, those changes
have required the DG to increase its flexibility by systematically redefining
priorities, redeploying staff[30] and
reducing the staff allocated to the administrative tasks to concentrate the
resources on the DG’s core activities[31].

Overview of Major Strengths

DG MARKT has launched a number of
strategic HR initiatives aimed at addressing the important challenges it faces
in terms of HRM in the recent years, notably:

- 
Together
with DG ECFIN and DG COMP, and with the support of DG HR, DG MARKT launched a
Commission wide call for expression of interest in November 2008 to reinforce
on a temporary basis existing teams in the three DGs. An additional call for
interest was launched in 2011 particularly for DG ECFIN.

- 
Following
the request of DG MARKT and DG ECFIN, EPSO launched in 2013 an AD5/AD7
specialist competition in macroeconomics and financial economics to provide a
more stable and qualified labour force for the years to come. DG MARKT invested
substantial resources by participating in the selection board of the
competition and developing detailed competency profiles for financial
economics.

- 
As
part of its learning and development framework DG MARKT has carried out an
in-depth analysis of learning needs in each Directorate in order to identify
key competencies and knowledge required for the future and related learning
solutions to fill the gaps.

- 
To
facilitate mobility between the DGs, DG COMP and DG MARKT agreed on an inter
mobility programme for Heads of Unit.

- 
DG
MARKT modified its structure to address the challenges deriving from the
financial crisis. In particular, it strengthened the financial services
coordination unit under the direct authority of the DDG for financial services,
and created a dedicated economic analysis unit (G1) to reinforce the economic
analysis capability in financial services, a unit for Financial stability (H4)
to deal with crisis management and resolution issues and another one on Banks
and Financial Conglomerates II (H2) to meet policy priorities pertaining to
structural bank reform.

- 
The
DG has recently launched a 360° feedback mechanism for managers to develop a
feedback culture in DG MARKT, and help managers improve their HRM skills.

Major Audit
Findings

The IAS has identified the following
three very important issues:

HRM Strategy (Observation 1)

HR-related strategic decisions are
described in the Personnel Action Plan and in some other additional documents
and are regularly discussed at senior management level. These strategic
documents do however not contain any quantitative (multi-annual) analysis
aiming at, among other things, identifying possible approaches to address the
5% staff reduction over the next 5 years and do not prioritise the actions
identified to address the HR challenges. In addition, there is no monitoring
and follow-up mechanism to measure their level of implementation and
effectiveness (deadlines, targets and KPIs).

Annual HR Planning (Observation 2)

DG MARKT implemented an annual HR resource
planning cycle, which involves both the HR team and the operational units. In
particular, each unit has to prepare a management plan (UMP) including, among
others, the list of tasks and an estimation of current and future staff
required to accomplish them. The IAS noted however that DG MARKT is not fully
equipped to assess the workload of each team/unit.

Monitoring and Reporting on HRM
(Observation 3)

DG MARKT provides management with a number
of HRM related statistics and reports (e.g. weekly vacancy reports, screening
of local overhead, equal opportunity figures) both in the context of the HR
planning exercises and in response to ad-hoc requests. However, there is no
comprehensive HR report that would allow regular monitoring and measuring of
the HR-related performance and an exhaustive and periodic reporting to
management based on KPIs and targets agreed with senior management.

Audit
recommendations

In order to address these issues,
following very important recommendations were issued:

HRM Strategy – Very
important

DG MARKT should further develop its
multi-annual HR strategy based on a qualitative and quantitative analysis of
staff needs, including the identification of possible options to address the
staff reduction. The implementation of the HR strategy should be supported by
tools to measure, monitor and report on its effectiveness.

Annual HR Planning – Very
important

DG MARKT should implement tools to
regularly assess staff workload and should set the priorities of its tasks. The
staff allocations should be done on the basis of tasks, priorities and the
workload identified.

Monitoring and Reporting on HRM – Very
important

DG MARKT should enhance its monitoring and
reporting process on the HRM related activities by developing a comprehensive
HR report that covers the different aspects of HRM and measure and monitor the
related KPIs. This would enable the DG to measure the HR management’s
performance and facilitate management decisions on HR.

DG MARKT established an action plan
which the IAS considers adequate to address the identified issues.

3. 
Agriculture, Natural Resources and Health

3.1. Limited review of residual error rate calculations in DG AGRI (pillar 1 and 2)

Background

The IAS strategic audit plan for 2013-2015
includes, for 2013, a limited review on the RERs reported by DG AGRI for the
2012 reporting year. Similar reviews were conducted in 2013 for DG DEVCO and DG
RTD.

Objectives

The objective was to review the
calculation and underlying methodology of the RERs reported by DG AGRI in its
(draft) AAR 2012 enabling DG AGRI to take appropriate actions, if any, before
their disclosure in the final AAR and Synthesis report.

Scope

The review covered the following aspects:

- 
methodology
for the calculation of the RERs for both EAGF and EARDF and the underlying
control statistics (including reliability of the MSs control statistics);

- 
testing
of the calculated RERs;

- 
presentation
of the RERs in the draft AAR (including compliance with the Standing
Instructions for the AAR 2012).

The IAS reviewed the draft AAR transmitted
to the central services (SG/BUDG) on 1/03/2013 and the draft part 2 of the AAR transmitted on 4/03/2013 and the preliminary RER calculations which were provided by DG
AGRI on 6/03/2013.

It should be noted that in its 2011 AAR and in the draft AAR 2012 DG AGRI has included a reservation on Rural Development
expenditure based on the RER exceeding the materiality threshold and/or
reputational grounds.

The fieldwork was finalised on 08/03/2013.
All observations and recommendations relate to the situation as of that date.

Major Findings

The review made findings on the
reliability of the MS control statistics used by DG AGRI to calculate its RERs
(Critical), DG AGRI's calculation of the RERs (Very important), the process for
making reservations in its AAR (Very important) and the presentation of the
RERs in the AAR (Very important).

Recommendations

The following critical and very
important recommendations were
issued in order to address the issues identified:

Reliability of MS control statistics used
by DG AGRI to calculate RERs – Critical

In calculating its RERs DG AGRI depends
very heavily on control statistics reported by MS, which the IAS found to be
unreliable, taking account the work performed by the ECA and MS Certifying
Bodies (CB).

DG AGRI should broaden the basis for
calculating its RERs, in particular by ensuring that it takes into account all
relevant, available information and making corrections where necessary. For the
2012 AAR, it should assess and quantify the impact of the weaknesses affecting
the reliability of the statistics and strengthen the reliability of its
assurances by taking into account the statistics which are actually validated
by the CB.

Also for the 2012 AAR, it should take
account of known/proposed corrections from previous years or on-going clearance
of accounts procedures when concluding on the reliability of the MS statistics
as a basis for its error rate calculations. To reinforce the assurances for
2012, it should try to validate certain of the highest spending aid/measures
and where necessary, correct the control statistics or apply flat rates.

Looking beyond the 2012 AAR, DG AGRI
should put in place a system to improve the reliability of the MS statistics.
Specifically, it should:

- 
ensure
that there is a proper process in place for assessing reliability, together
with appropriate internal guidance to staff;

- 
ensure
that its audit work specifically addresses reliability;

- 
reinforce
the work of the CB to include legality and regularity aspects, which should in
turn strengthen the assessment of the Paying Agency's on-the-spot checks and
control statistics;

- 
automate
the transmission of control statistics by MS to the Commission;

- 
issue
adequate instructions/guidance to the MS on the control statistics and
underlying sampling requirements, in particular on the random sample component;

- 
ensure
the CBs provide reliable opinions on the PA management declaration covering the
legality and regularity of the underlying transactions, based inter alia on the
re-performance of a representative sample of transactions checked by the PA
on-the-spot, as foreseen in the proposal for the post 2013 CAP regulations;

- 
following
the entry into force of the post 2013 CAP regulations DG AGRI should closely
monitor the reinforced work of the CB on the legality and regularity aspects of
the PA's controls. The results of that work should be assessed on a yearly
basis in the context of the AAR and be subject to a comprehensive review after
two years of application with a view to deciding whether the reinforced work of
the CB has delivered the expected results or whether DG AGRI should consider
other ways for obtaining assurance. This would be in line with the provision of
Art 32(5) of the Financial Regulation, which requires improvements/re-design of
systems if weaknesses in control systems are seen to persist. This could be
obtained through the CB performing independently on-the-spot checks on a
representative sample of transactions (out of the total population of payments
approved) and reporting the resulting error rates to DG AGRI.

Calculation RER –
Very important:

DG AGRI should ensure that the RERs are
calculated on the basis of correct and representative data for all parts of the
budget and takes into account residual errors after administrative checks. It
should address the inconsistencies noted and ensure that cross-compliance
errors are included in the calculation. If excluded, the impact should be
explained.

DG AGRI should ensure adequate back-up and
business continuity procedures, including full documentation of the calculation
basis and procedure. It should automate both the calculation process and the
process for compiling MS control statistics.

Reservations – Very
important:

DG AGRI should not apply the currently
used 25% margin to the RERs as a basis for the decision on reservations, unless
there is a clear and statistically sound rationale to support it. It should
consider adapting the procedure for making reservations, where supportable by
reliable MS control statistics. This would be more in line with the latest
materiality criteria used by other shared management DGs.

AAR presentation – Very
important:

DG AGRI should clearly explain in the AAR the reasons for differences between its own error rates and those of the ECA. It should
explain the basis on which it relies on the MS control statistics and make
better use of available information on financial corrections/recoveries to
corroborate the level of residual error in the population and assess the
residual amount at risk. In addition, it should disclose RERs by MS/ABB activity
where relevant.

DG AGRI established an action plan which the IAS considers adequate
to address
the identified issues. It was followed up in
the context of the AAR for the reporting year 2013 which revealed that
considerable progress had been made already for the 2012 but in particular for
the 2013 AAR.

3.2. Audit on Control Strategy - Implementation in DG AGRI

Background

Common Agricultural Policy (CAP)
expenditure is mainly accounted for under two shared management funds. The
European Agricultural Guarantee Fund (EAGF) finances direct aid and market
measures (€330 billion in total between 2007-13) and the European Agricultural
Fund for Rural Development (EAFRD) co-finances rural development programmes
(€96.3 billion in total between 2007-13). Expenditure under both funds is
managed through some 81 national Paying Agencies (PAs) across the MS.

Under shared management, the MS have
primary responsibility for implementing effective internal control systems to
prevent, detect and correct irregular expenditure, while the Commission
performs a supervisory role over national systems and assumes final
responsibility for the implementation of the budget.

Audit Objectives

This audit was a continuation of the IAS's
audit in 2012 on the design of the control strategy employed by DG AGRI's Audit
Directorate (Directorate J). The objective was to assess the execution of the
audit and control strategy in practice, in particular the effective
implementation of audit engagements, their supervision and corrective measures
to address weaknesses in MS management and control systems.

Audit Scope

The audit specifically covered the
following areas:

- 
The
audits and checks made by Units J2, J3 and J4 as part of the multi-annual
Conformity Clearance process, which is designed to obtain assurance on the
legality and regularity of underlying transactions.

- 
The
audits and checks made by Unit J5 as part of the annual Financial Clearance of
Accounts process, which is designed to provide assurance on the veracity,
completeness  and accuracy of the PAs' accounts.

- 
The
work of the Coordination Unit J1 as support to the audit process.

There were no scope limitations.

DG AGRI included in its 2012 AAR reservations on both EAGF and EAFRD. For EAGF the reservation concerned serious
deficiencies in direct payments in Portugal, Bulgaria and France, mainly due to deficiencies in the functioning of the Land Parcel Identification
Systems (LPIS). For EAFRD, two reservations were made. One concerned the
reliability of the MS control statistics used to derive the RER. The other
reservation concerned deficiencies in the supervision and control of organic
production.

The IAS fieldwork was finalised in
mid-June 2013. All observations and recommendations relate to the situation as
of that date.

Strengths

The auditors recognise the on-going
efforts made by DG AGRI to reduce error rates and protect the budget through a
range of detective and corrective measures. On the detective side, the DG is
working hard to understand and assess the root causes of errors in close cooperation
with the MS bodies with a view to identifying corrective measures. The DG
continues to build, refine and develop its audit procedures. Furthermore, as
part of this audit the IAS saw first-hand (by accompanying DG AGRI's auditors
on-the-spot) the challenges they face, the extreme time pressure they work
under and the professionalism and competency with which they conduct their
work. In addition, the IAS notes in particular the efforts made to strengthen
corrective measures through the implementation of the recent revision of
Commission Regulation n° 883/2006 to interrupt payments.

Major Audit Findings

Detective measures (report finding 1)

There is scope for improving the audit
process in a number of key areas. Currently, the planning and preparation stages
are hampered by the need to translate and analyse important MS documentation
before auditors leave on mission, which if not done well in advance puts
pressure on the time available on-the-spot for actual testing. In addition,
where sampling is used, this is not always clearly justified and the Clearance (of Accounts) Audit Trail System (CATS)
is not always used. Documents setting out the MS key and ancillary controls are
out of date. Regarding EAGF, although there are some attempts to involve the
operational units in the audit planning process to exploit their knowledge and
expertise on the actual implementation of policy, it is clear that this is
limited in practice.

There is a general lack of
trail/cross-referencing between the various audit documents, including from the
planning stage through to reporting, together with a lack of documented
supervision. Although there is an increasing use of standardised checklists and
audit programmes in certain areas, significant scope for improvement remains in
order to provide a more solid basis for the assurance process. Furthermore,
despite the increasing error rates and concern over the effectiveness of the
first level MS checks, very few re-performance checks of controls are made
on-the-spot. The focus is very much on the overall systems rather than
operations. The time taken to finalise the audit process is very lengthy and is
not helped by a lack of effective audit monitoring tools (see Report Finding
3). However, the IAS acknowledges that the contradictory and conciliation
procedure is itself very long and that it is in the interests of MSs to delay
this in order to lessen the impact of financial corrections.

Finally, the coordination and
communication arrangements both within the Audit Directorate and outside can be
improved in order to strengthen the assurance process overall. The Audit
Directorate has the challenge of dealing with many different responsibilities
in very demanding circumstances, but currently suffers from a lack of effective
sharing of key information, including potentially very valuable input from the
operational side.

Corrective measures (report finding 2)

Key to an effective control strategy is an
effective sanctions process, which in the first instance ensures that
weaknesses and deficiencies are subject to financial corrections, but in the
longer term ensures real and sustainable improvements in MS management and
control systems. Again, this is even more important in the light of increasing
concerns as regard the higher error rates and the effectiveness of the first
level MS controls. The IAS acknowledges the effectiveness of the financial
corrections process in protecting the budget, but notes that DG AGRI has not so
far extensively used the interruption and suspension measures, which come
earlier in the process, due to complex legal provisions in the current
legislative framework. However, the rules proposed for the period 2014-2020
introduced new interruption and suspension clauses in the legislation. These
measures are potentially very powerful tools in reinforcing the responsibility
of MS to implement robust systems of control.  In terms of ensuring that those
improvements to MS system are sustained in practice, there is a real need to
build capacity. However, DG AGRI has yet to put the monitoring systems in place
to help deliver this. There is no central overview of the audit findings and
recommendations, which would provide a solid basis to help both the audit and
operational units track improvements in the MS and to feed back into the audit
planning cycle, although the IAS notes that very preliminary work has been
undertaken on establishing a recommendations database.

In addition, the IAS noted that, for EAGF,
there is very little guidance concerning the processes for establishing, reviewing
and following up specific MS Action Plans, including the respective roles and
responsibilities. So far, such plans have been made for Greece and Portugal and one is currently being prepared for France.

Monitoring and reporting (report finding
3)

Adequate monitoring, measurement and
reporting mechanisms are essential for ensuring the effective management of an
audit function. Although very powerful, the existing COMBO database system
lacks the reporting functionality the Audit Directorate needs to monitor
overall progress and for the Units to monitor their own day to day work. As a
consequence, certain units have developed their own manual based systems.
Although progress on audits is in fact monitored in practice, this could be
done on a more regular basis. This is even more important in view of the length
of time it takes to close audits, as noted in Report Finding 1 and the large
number of audits still open. In addition, there is significant room for
improving current audit performance information. At the moment this is very
limited in both quantitative and qualitative terms and lacks the consistency
and depth necessary for effectively managing and monitoring the audit process.

Audit recommendations

The following recommendations were issued
to address above mentioned findings:

Detective measures (report finding 1: Very
important)

DG AGRI
should ensure that the key preparatory documents are obtained in advance and,
where necessary, translated and analysed. It should also try to better match
auditors' language skills. Sampling criteria need to be properly justified and
more use made of CATS. The key data on MS key and ancillary controls should be
updated.

It
should make more use of the operational units' knowledge and expertise at the
planning stage.

DG AGRI
should undertake tests of operations that are selected on a risk basis, in
order to strengthen assurance on the legality and regularity of underlying
transactions.

The DG
should ensure that the relevant completion and reporting deadlines are met. It
should ensure that the appropriate programmes and checklists are used, properly
signed off and evidenced for management review.

DG AGRI
should ensure that it takes advantage of key information provided by other
sources within the DG, notably in the context of the proposed strengthening of
the role of the CBs and the checks foreseen on legality and regularity of
transactions. It should ensure that it liaises effectively with other partners.

Overall,
in view of the measures needed to strengthen the audit process, DG AGRI should
develop a quality assurance and improvement programme, in line with
international audit standards, which could include provisions for internal and
external quality review.

Corrective measures (report finding 2: Very
important)

DG AGRI
should  apply payment reduction and suspension procedures where appropriate. It
should use its existing IT system (COMBO) and finalise the initial work
undertaken to establish a database to better track and follow up audit
recommendations.          
 DG AGRI should also clarify the roles, responsibilities and modalities as
regards MS action plans. This should include a clear statement and guidance on
the conditions under which MS are required to submit such plans, the respective
roles of DG AGRI's audit and operational units, together with review, reporting
and follow up arrangements.

Monitoring and reporting (report finding
3: Very important)

DG AGRI
should ensure that it regularly monitors and reports on audit progress and
achievement of targets and objectives. Bearing in mind the problems of data
reliability in the COMBO system, checks should be made to ensure the accuracy
and consistency of the data presented in the reports. It should ensure that the
monitoring system implemented is supported by clear reporting requirements for
the communication of performance data, including the regularity, content and
level, together with a clear statement of responsibilities.

DG AGRI established an action plan
which the IAS considers adequate to address the identified issues.

4. 
Cohesion

4.1. Implementation of ESF 2007-13 (DG EMPL)

Background

The
European Social Fund (ESF) is a key vehicle at the disposal of the European
Union (EU) to assist MSs in the field of employment. Over the programming
period 2007–13 ESF spending will amount to over EUR 75 billion, representing
around 8 % of the total EU budget. The ESF is put into action through 117
Operational Programmes (OPs) for the 2007–13 programming period (PP). In 2012,
DG EMPL made payments of EUR 11,28 billion.

Although
the management and control systems for the ESF, ERDF and CF are governed by
common rules, in practice most ESF operational programmes have their own
specific authorities and control systems. Under shared management, the MSs have
primary responsibility for implementing effective internal control systems to
prevent, detect and correct irregular expenditure, while the Commission
performs a supervisory role over national systems and assumes final
responsibility for the implementation of the budget.

Audit Objectives

Recognising
the persistently high error rates in the Cohesion area, the IAS conducted this
audit on the implementation of the 2007-13 PP for ESF programmes in order to
assess the extent to which DG EMPL has taken sufficient and adequate measures
to reduce the high error rates.

Audit Scope

The
audit specifically covered the following areas:

- 
The
checks made by DG EMPL in order to place reliance on national audit authorities
(AAs), based on a sample of files and by accompanying DG EMPL auditors during
audit missions to MS.

- 
Follow
up of the actions resulting from the Commission working paper on the analysis
of errors on Cohesion Policy and from other action plans.

- 
Preventive,
detective and corrective measures to tackle the problem of high error rates.

- 
Through
file examination, analysis of the key decision processes at the basis for
making/lifting reservations, interruptions, suspensions and financial
corrections.

There
were no scope limitations.

The
fieldwork was finalised in mid-October 2013. All observations and recommendations
relate to the situation as of that date.

The
following reservation was made in the 2012 AAR concerning specifically the
area/process covered by the scope of this engagement:

"Reservation
concerning the ESF Fund management and control systems for the period
2007-2013:

Significant
issues regarding the effective functioning of management and control systems
for 27 OPs in the following Member States: Belgium, Czech Republic, France,
Germany, Ireland, Italy, Poland, Romania, Slovakia, Spain, United Kingdom and
IPA programmes for Turkey and FYROM."

Strengths

The IAS
recognises the continuing efforts made by DG EMPL to reduce error rates through
a range of preventive, detective and corrective measures and in seeking to
report those error rates clearly and concisely. On the preventive side and
recognising the critical role the MS AAs have to play in the assurance building
process, the DG has worked very hard to build capacity and understanding by
developing, in cooperation with the other Structural Funds DGs, comprehensive
guidance to AAs on how to assess MS control systems and on the sampling
methodologies and by organising a range of seminars on specific topics. In
response to the Commission Staff Working Paper[32] and
previous ECA reports, DG EMPL has strongly encouraged the MS to use the
simplified cost options (SCOs), and provided supporting guidance, with the
result that one or more of these options have been used in nearly 60% of OPs.
ECA recognizes in its 2012 Annual Report the positive impact the simplification
rules have had on the level of errors in ESF[33].

From a
detective side, the DG continues to build, refine and develop its audit
procedures through the various Enquiry Planning Memorandums (EPMs),
particularly those covering the "Re-performing the audit work conducted by
a national audit authority" aimed at gaining assurance on the reliability
of AAs and "Review of management verification process in place",
which aims to get direct assurance on MS systems where the AA cannot be relied
on. These are robust and proactive responses to the assurance building
challenge. Furthermore, as part of this audit the IAS gained first-hand
knowledge (by accompanying DG EMPL's auditors on-the-spot) of the challenges
they face, the extreme time pressure they work under and the professionalism
and competency with which they conduct their work. However, the IAS also
recognises that the audit process is only one of a package of measures aimed at
reducing error rates. DG EMPL has devoted a significant effort, mainly through
the Audit Directorate, to discussing in detail the results of ECA findings and
explaining and clarifying specific cases in the framework of the contradictory
procedure.

In
addition, the IAS notes in particular the strong corrective measures implemented
by DG EMPL, through timely and increased levels of interruptions and
suspensions which are monitored by the Interruption Suspension Financial
Corrections Committee (ISFC Committee) and which is also strongly driven by the
work of the Audit Directorate.

Major Audit Findings

Detective measures to reduce error rates
(Report Finding 1)

As
noted in previous IAS reports, DG EMPL has reinforced its audit approach in all
areas. However, the IAS found that the preparation phase could be strengthened
by a better consideration of the Geographical Units' (GU) knowledge and, in
case of follow-up audits, the initial risks and the action plan to be tested.
Similarly, for re-performance audits of Audit Authority work, the specific
risks could be better reflected in the Mission Planning Memorandum (MPM).
Preparation is even more important in view of the time pressures brought about
by the overlapping of two programming periods from 2014 onwards and the
numerous other tasks borne by the audit units, such as supporting the ISFC
Committee work, contradictory procedures with ECA and continued capacity
building work with MSs, all of which are aimed at reducing the error rate. The
IAS’s first-hand experiences in accompanying the DG auditors on-the-spot found
that there were variations between missions in the depth of testing undertaken
and the extent of work at the beneficiaries’ premises. We acknowledge the need
to adapt the work to fit the circumstances, but consider there is room for more
consistency overall, particularly given the pressure to make the most effective
use of scarce audit resources. Finally, the IAS noted that the reporting phase
could be improved for follow-up audits. DG EMPL reports on an exception basis
only and does not provide a clear confirmation that recommendations and
corrective actions have been implemented in practice.

Audit recommendations

The
following recommendations were issued to address above mentioned findings:

Detective measures to reduce error rates
(Report Finding 1: Very important)

DG EMPL
should incorporate in the MPM the relevant key information received from the
GUs; the GUs should examine case-by-case the added value of their participation
in the audit mission and decide accordingly whether to participate in the whole
mission, the closing meeting, or not at all. Furthermore, DG EMPL should ensure
more consistency of approach for the same audit enquiry type through further
instruction/guidance and supervision on the extent of testing to be carried out
on the spot. It should ensure that appropriate checklists are used on missions
and that tailor-made checklists are developed for each mission that involves
the follow up of an action plan or with specific risks. For follow-up audits,
it should develop a specific template to record the auditor's conclusion on
whether recommendations and corrective actions have been implemented.

DG EMPL established an
action plan which the IAS considers adequate to address the identified issues.

4.2. Performance Audit of DG REGIO Performance Measurement Systems

Background

Knowing
how well an organisation is doing is essential in developing strategy and
policies to meet the organisation's aims. Performance information shows how
well an organisation is performing against its stated objectives. Clear
objectives, with well-selected performance measures, are the foundation of a
good performance measurement system. This not only determines what needs to be
done, but also sets the conditions for success.

Following
the entry into force of the Lisbon Treaty, the European Commission is required
to submit to the European Parliament and the Council a yearly evaluation report
on the Union’s finances based on the results achieved (Article 318 report).
Moreover, the European Parliament considers that performance audits measuring
the extent to which spending has achieved its objectives pursued have become
ever more important and the Commission should develop a new culture of
performance. In June 2011, the central services of the Commission reminded
operational services that the current financial and economic crisis and severe
constraints in public spending has shifted the focus to achieving more with
less and that post-2013 spending programmes should place a greater focus on
performance measurement. Other initiatives have been taken at Commission level
on this move towards a performance culture, e.g. the setting up of a working
group of Directors-General of the Commission on “Move to a performance culture”
in February 2013.

In
order to meet stakeholders' expectations, one of the objectives of the IAS in
its Strategic Audit Plan for the period 2013-2015 is to carry out performance
audits focusing on the economy, efficiency and effectiveness of the use of
resources. A number of audits have been planned in 2013 to assess the
performance measurement system of DGs, including not only DG REGIO, but also
DGs EMPL and EAC.

For DG
REGIO, the issue of performance management is particularly relevant. Operating
in a multi-annual shared management framework, it faces the challenge of
supervising closely the MSs, but being ultimately accountable for implementing
the budget. It has the dual responsibility of ensuring legality and regularity
of the expenditure, but at the same time trying to promote sound financial
management and effective overall implementation of policy. The focus on policy
performance will be even greater in the new 2014-20 programming period with the
stronger overall results orientation and the introduction of the Performance
Framework and Performance Reserve.

Audit Objectives

The
main objective of the audit was to assess the extent to which DG REGIO has
adequate performance measurement systems in place for monitoring, reporting and
evaluating the performance of activities both in terms of its day to day,
operational and administrative activities (internal) and in terms of measuring
the delivery of policy objectives (external). The audit addressed the following
key questions:

- 
Is
the performance measurement framework comprehensive?

- 
Does
the DG adequately monitor and measure internal performance including the use of
human resources?

- 
Does
the DG adequately monitor and evaluate its policies?

Audit Scope

The
audit focused on:

- 
A
horizontal analysis of DG REGIO's internal processes for setting objectives and
KPIs and related reporting and monitoring systems.

- 
A
detailed review of the processes for setting objectives, indicators,
monitoring, evaluation and performance reporting concerning the ERDF and
Cohesion Fund.

There
are no observations/reservations in the AAR that relate to the area/process
audited.

The
fieldwork was finalised on 11/10/2013. All observations and recommendations
relate to the situation as of that date.

Strengths

The IAS
recognises the on-going efforts made by DG REGIO to strengthen the results and
performance aspects not just of its management of the ERDF and CF, but also of
its own internal activities and resources.

This
focus is very evident in the draft regulations for the 2014-20 programming
period, whereby key pillars for this new results and performance based focus
include a clear articulation of objectives of programmes with a strong
intervention logic, the definition of a set of mandatory common output
indicators, the introduction of a performance framework for each programme
linked to a performance reserve and strengthened provisions for the evaluation
of programmes. Although the preparations for the new programming period (e.g.
development of guidance, negotiation of partnership agreements and operational
programmes) are still on going, the IAS acknowledges the efforts devoted to
bringing the results and performance orientation into practice.

The IAS
also recognises the challenge the DG faces to monitor and evaluate the
2007-2013 programmes and to report on their performance, for example because of
the lack of mandatory common indicators. In particular, the IAS notes the
efforts already made by the DG to improve the quality of the data reported by
the MS on core indicators and also the continued efforts to improve the quality
of evaluations conducted by the MS and the DG itself.

DG
REGIO considerably strengthened its internal performance measurement systems by
setting out in the Management Plan (MP) a set of priorities and reporting
progress towards their achievement on a bi-monthly basis to the board of
directors. In addition, units' reporting on progress towards these priorities
is reported to the board on a quarterly basis.

Also, a
major effort has been devoted to mapping human resources to activities and
tasks and assessing the workload linked to these activities and tasks. The IAS
considers this workload assessment can provide a good basis for helping the DG
to make important HR allocation decisions and further developing its
performance measurement systems.

Major Audit
Findings

The IAS
identified three very important issues which should be the main priority:

Performance measurement framework

- 
The
MP and the AAR constitute the main elements of the performance measurement
framework. One sets out the performance information to be reported (objectives,
priorities and indicators), whilst the other is used to report what was
actually achieved in practice. In general, priorities are well defined and
accompanied by indicators, but the MP should explain more clearly how
priorities relate to objectives. More fundamentally, the way in which
performance indicators are set and reported on needs to be improved in terms of
presentation, content and coverage. Currently, they do not generally meet the
widely accepted RACER[34]
criteria and the number of indicators is not always proportionate to the
importance or size of the activity measured. Also, there are no
efficiency/economy indicators in the MP. Policy achievements are reported
mainly in terms of the degree of financial implementation and according to a
limited number of core indicators. Although presented as if they are results or
impacts-based, they are in fact more outputs-based.

Monitoring and evaluating Regional Policy

- 
The
quality and reliability of MS performance information is a major issue for the
DG. It carries out desk checks on the indicators reported, but these are very
limited in nature and there is currently no audit work undertaken to
specifically check reliability. Although the draft regulations for 2014-20 set
out certain quality expectations for managing the Performance Framework as a
process, there is nothing foreseen aimed specifically at ensuring the quality
and reliability of MS performance information. The MS Audit Authorities are not
obliged to audit the reliability of performance information, but equally there is
nothing to stop them doing so in practice. In addition, there are concerns
about the quality and coverage of the MS evaluation work. The lack of reliable
MS performance information also presents an obstacle to the effectiveness of
the Commission's own evaluation work.

- 
The
new regulations for 2014-20 foresee a much greater emphasis on performance
aspects, with the introduction of requirements for programme specific result
indicators with baselines and obligatory common output indicators as well as a
Performance Framework and a Performance Reserve. Currently however, the
monitoring of policy implementation, as exercised by the Geographical Units
(GUs), mainly focuses on financial implementation rather than assessing
performance against objectives. Operational progress on the ground is
monitored, but without clear objectives, methodology and procedures on what
criteria/information should be taken into account. The move to a new
performance oriented environment brings with it new expectations, which need to
be prepared for. Internal guidance on the Performance Framework has been
developed, but as current monitoring arrangements are not
performance-orientated, such guidance will need to be made very practical and
concrete.  More specifically, the introduction of the Performance Reserve poses
the risk that MS will set un-ambitious milestones and targets and calls for
properly targeted checks. The extent to which monitoring, evaluation and audit
can play a part in responding to this overall challenge is still very much
under reflection in the DG, particularly given the continued pressure on audit
resources to supervising MS management and control systems.

Audit
recommendations

To
address the issues, DG REGIO should:

- 
Better
integrate the DG's priorities into the MP, and explain how they contribute to
the specific objectives. Ensure that its indicators are RACER and targets are
well defined and accompanied by appropriate milestones. If indicators/targets
are unlikely to be met this should be clearly explained. The number of
indicators should be proportionate and less important ones reported only in
lower level documents. Efficiency indicators should be developed for internal
activities.  For policy implementation in 2014-20, indicators should be
developed based on an assessment of the overall performance of each OP. Policy
achievements should be better explained in terms of targets and objectives met
and the resources used. The AAR should report more clearly also on
general/specific objectives, in addition to reporting on priorities.  Looking
forward and working in conjunction with the other European Structural and
Investment Funds (ESIF) DGs, it should ensure that the evaluation strategy for
2014-20 draws upon the lessons learnt from the current period, in particular on
how the DG plans to obtain evaluation evidence over time and how this can
contribute to the AAR and art. 318 report.

- 
Together
with DG EMPL, develop a strategy to improve the reliability of performance
information reported by the MS. Given that performance measurement impacts
across a wide range of different aspects of programme management (e.g.
monitoring, evaluation, reporting, financial impact for 2014-20 period), DG
REGIO should consider a multi-disciplinary approach, involving the Geographical
Units, Evaluation Units and the Audit Directorate. Although the key challenge
will be to develop a comprehensive strategy for the next programming period, DG
REGIO should continue to do as much as possible to improve further the quality
of data reported for the 2007-13 programming period and continue to strengthen
the capacity and contribution of MS evaluation work.

- 
Strengthen
the role of the GUs in monitoring the performance of the OPs and ensure that
results are properly reflected in the AOSD management opinion. Although the
focus should be on the 2014-20 period, it should also ensure that the
performance of OPs is sufficiently assessed for the remainder of the 2007-2013
period, taking into account the limitations of the current legal base. This
could be done, for example, by adapting the checklists used to review the
Annual Implementation Report (AIR). It should develop practical guidance for
the GUs to assess milestones and targets and ensure that assumptions are
properly documented. It should develop a clear approach for building up
assurance on the performance of Cohesion policy, particularly for the 2014-20
programming period. It should assess how this assurance can be delivered,
including through monitoring, evaluation and audit.

DG REGIO
established an action plan which the IAS considers adequate to address the
identified issues.

4.3. Performance Audit of DG EMPL Performance Measurement Systems

Background

Knowing
how well an organisation is doing is essential in developing strategy and
policies to meet the organisation's aims. Performance information shows how
well an organisation is performing against its stated objectives. Clear
objectives, with well-selected performance measures, are the foundation of a
good performance measurement system. This not only determines what needs to be
done but also sets the conditions for success.

Following
the entry into force of the Lisbon Treaty, the European Commission is required
to submit to the European Parliament and the Council a yearly evaluation report
on the Union’s finances based on the results achieved (Article 318 report).
Moreover, the European Parliament considers that performance audits measuring
the extent to which spending has achieved its objectives pursued have become
ever more important and the Commission should develop a new culture of
performance. In June 2011, the central services of the Commission reminded
operational services that the current financial and economic crisis and severe
constraints in public spending has shifted the focus to achieving more with
less and that post-2013 spending programmes should place a greater focus on
performance measurement. Other initiatives have been taken at Commission level
on this move towards a performance culture, e.g. the setting up of a working
group of Directors-General of the Commission on “Move to a performance culture”
in February 2013.

In
order to meet stakeholders' expectations, one of the objectives of the IAS in
its Strategic Audit Plan for the period 2013-2015 is to carry out performance
audits focusing on the economy, efficiency and effectiveness of the use of
resources. A number of audits have been planned in 2013 to assess the
performance measurement system of DGs, including DG EMPL, but also DGs REGIO
and EAC.

For DG
EMPL, the issue of performance management is particularly relevant. The DG has
a major role contributing to key objectives of the EU 2020 strategy on
employment, social inclusion and education. Regarding the ESF,  operating in a
multi-annual shared management framework, it faces the challenge of supervising
closely the MSs, but being ultimately accountable for implementing the budget.
It has the dual responsibility of ensuring legality and regularity of the
expenditure, but at the same time trying to promote sound financial management
and effective overall implementation of policy. The focus on policy performance
will be even greater in the new 2014-20 programming period with the
introduction of the Performance Framework and Performance Reserve.

Audit
Objectives

The
main objective of the audit was to assess the extent to which DG EMPL has
adequate performance measurement systems in place for monitoring, reporting and
evaluating the performance of activities both in terms of its day to day,
operational and administrative activities (internal) and in terms of measuring
the delivery of policy objectives (external). The audit addressed the following
key questions:

- 
Is
the performance measurement framework comprehensive?

- 
Does
the DG adequately monitor and measure internal performance including the use of
human resources?

- 
Does
the DG adequately monitor and evaluate its policies?

Audit Scope

The
audit focused on:

- 
A
horizontal analysis of DG EMPL's internal processes for setting objectives and
KPIs and related reporting and monitoring systems.

- 
A
detailed review of the processes for setting objectives, indicators,
monitoring, evaluation and performance reporting concerning the European Social
Fund (ESF)

There
are no observations/reservations in the AAR that relate to the area/process
audited.

The
fieldwork was finalised on 11/10/2013. All observations and recommendations
relate to the situation as of that date.

Strengths

The
IAS recognises the on-going efforts made by DG EMPL to strengthen the results
and performance aspects not just of its management of the ESF, but also of its
own internal activities.

This
focus is very evident in the draft regulations for the 2014-20 programming
period, whereby key pillars for this new results and performance based focus
include a clear articulation of objectives of programmes with a strong
intervention logic, the definition of a set of mandatory common output and
result indicators, the introduction of a performance framework for each
programme linked to a performance reserve and strengthened provisions for the
evaluation of programmes. Although the preparations for the new programming
period (e.g. development of guidance, negotiation of partnership agreements and
operational programmes) are still on going, the IAS acknowledges the efforts
devoted to bringing the results and performance orientation into practice.

The
IAS also recognises the challenge the DG faces to monitor and evaluate the
2007-2013 programmes and to report on their performance, for example because of
the lack of mandatory common indicators (other than data by participants). In
particular, the IAS notes the efforts already made by the DG to improve the
quality of the data reported by the MS on core indicators and the continued
efforts to improve the quality of evaluations conducted by the MS and the DG itself.

DG
EMPL considerably strengthened its internal performance measurement systems by
developing a bi-annual report to the Policy Management Board (PMB) on progress
towards objectives, based on a comprehensive set of operational objectives and
indicators for ABB activities and horizontal activities.

Major Audit
Findings

The
IAS identified four very important issues which should be the main priority:

Performance measurement framework

- 
The
MP and the AAR constitute the main elements of the performance measurement
framework. One sets out the performance information to be reported (objectives
and indicators), whilst the other is used to report what was actually achieved
in practice. Objectives are not always clearly defined, for example concerning
EMPL's operational activities contributing to the ESF's specific objectives,
and horizontal activities are not included in the MP itself.  Indicators are
not always well defined or classified. Currently, they do not always meet the
widely accepted RACER[35]
criteria and the number of indicators is not always proportionate to the
importance or size of the activity measured. There is room for improving
consistency between objectives, indicators and targets. In particular, main
outputs by specific objective are not presented in terms of objectives (or
description of outputs), indicators and targets and hence their usefulness as
performance information is limited. Also, there are no efficiency and/or
economy indicators included in the MP. Policy achievements of the ESF are reported
mainly in terms of the degree of financial implementation and the number of
participations to ESF programmes. Information reported on the DG's own
activities which contribute to the ESF specific objectives is limited. Also,
known problems concerning data quality are not reflected when reporting on ESF.

Monitoring and measuring internal
performance including the use of human resources

- 
The
DG has carried out workload assessments for certain activities when these were
reorganised. However, there is currently no system in place for assessing
workload across the full range of the DG's activities and tasks in order to
provide assurances on the efficient and effective allocation of staff
resources. As human resources are currently not mapped to objectives and activities
and workload assessments are not conducted, this means that, as noted under
finding N°2, the DG's internal performance measurement systems currently do not
cover the efficiency of the use of its resources. EMPL is in the process of
identifying its HR needs in terms of experience, knowledge and main skills and
competencies. However, the DG does not yet have a clear picture of current
profiles of staff in terms of previous experience, knowledge and main skills
and competencies in relation to future needs. It has not yet developed an HR
plan identifying the resources and staff's profiles needed to carry out its
current and future activities/tasks and meet its priorities, as well as the
planned measures to address any gaps (e.g. training, redeployment, recruitment,
staff reduction, etc.).

Monitoring and evaluating the ESF

- 
The
quality and reliability of MS performance information is a major issue for the
DG. It carries out desk checks on the indicators reported, but these are very
limited in nature and there is currently no audit work undertaken to
specifically check reliability. Although the draft regulations for 2014-20 set
out certain quality expectations for managing the Performance Framework as a
process, there is nothing foreseen aimed specifically at ensuring the quality
and reliability of MS performance information. The MS Audit Authorities are not
obliged to audit the reliability of performance information, but equally there
is nothing to stop them doing so in practice. In addition, there are concerns
about the quality and coverage of the MS evaluation work. The lack of reliable
MS performance information also presents an obstacle to the effectiveness of
the Commission's own evaluation work.

- 
The
new regulations for 2014-20 foresee a much greater emphasis on performance
aspects, with the introduction of a Performance Framework and a Performance
Reserve. Currently however, the monitoring of policy implementation, as
exercised by the Geographical Units (GUs), mainly focuses on financial
implementation rather than assessing performance against objectives.
Operational progress on the ground is monitored, but without clear objectives,
methodology and procedures on what criteria/information should be taken into
account. The move to a new performance oriented environment brings with it new
expectations, which need to be prepared for. Internal guidance on the
Performance Framework has been developed, but as current monitoring
arrangements are not performance-orientated, this will need to be complemented
by very practical and concrete guidance.  More specifically, the introduction
of the Performance Reserve poses the risk that MS will set un-ambitious
milestones and targets and calls for properly targeted checks. The extent to
which performance audit can play a key part in responding to this overall
challenge is still very much under reflection in the DG, particularly given the
continued pressure on audit resources to supervise MS management and control
systems. Since the completion of the fieldwork DG EMPL has informed the IAS
that the Policy Management Board (PMB) has adopted the audit strategy for
2014-16 which includes plans for performance audits and that the related audit
methodology is under development.

Audit
recommendations

To
address the above mentioned issues, DG EMPL should:

- 
Better
integrate into the MP the objectives/planned outputs for the DG's operational
activities which contribute to the ESF's specific objectives and the objectives
for the main horizontal and administrative activities. Further improve its
indicators to be RACER and targets to be well defined. Present main outputs in
support of the specific objectives in terms of objectives, indicators and
targets and consider transferring a number to lower level (e.g. unit) plans.
Efficiency indicators should be developed for internal activities.  For ESF
policy implementation in 2014-20, indicators should be developed based on
EMPL's assessment of the overall performance of each OP. Improve reporting on
the achievements of the ESF by including more qualitative analysis and
explanations in relation to achievements, by providing more information on the
context (target, inputs of resources) so that users can properly understand the
extent to which objectives are being met. Also, by reporting in the AAR on the DG's operational activities which contribute to the ESF's specific objectives.

- 
Building
on its work done so far in this area, it should further develop its approach to
workload assessments and extend the exercise to cover the full range of activities
across the DG in order to help ensure an optimal allocation of resources.  It
should use the information obtained through workload assessment, for example on
human resources linked to specific activities and workload indicators, in its
performance measurement system, to ensure it also covers the efficiency of the
use of its resources. It should ensure that it has up-to-date information on
the current experience, knowledge, skills and competencies of its staff and
further develop its HR plan accordingly to include the assessment of resource
needs and gaps and identify measures to address them.

- 
Together
with DG REGIO, develop a strategy to improve the reliability of performance
information reported by the MS. Given that performance measurement impacts across
a wide range of different aspects of programme management (e.g. monitoring,
evaluation, reporting, financial impact for 2014-20 period), DG EMPL should
consider a multi-disciplinary approach, involving the Geographical Units,
Evaluation Units and the Audit Directorate. Although the key challenge will be
to develop a comprehensive strategy for the next programming period, DG EMPL
should continue to do as much as possible to improve further the quality of
data reported for the 2007-13 programming period and continue to strengthen the
capacity and contribution of MS evaluation work.

- 
Strengthen
the role of the GUs in monitoring the performance of the OPs and ensure that
results are properly reflected in the AOSD management opinion. Although the
focus should be on the 2014-20 period, it should also ensure that the
performance of OPs is sufficiently assessed for the remainder of the 2007-2013
period, taking into account the limitations of the current legal base. This
could be done, for example, by adapting the checklists used to review the
Annual Implementation Report (AIR). It should develop practical guidance for
the GUs to assess milestones and targets and ensure that assumptions are
properly documented. It should develop a clear approach for building up assurance
on the performance of the ESF,  particularly for the 2014-20 programming
period. It should assess how this assurance can be delivered, including the
need for performance audits, and, in the light of the continued pressures on
the existing audit work, it should consider how to best develop the expertise
and resources needed, for example by making use of multi-disciplinary teams.

DG EMPL
established an action plan which the IAS considers adequate to address the
identified issues.

5. 
Research, energy and transport

5.1. Limited Review of the calculation and the underlying methodology of DG RTD's residual error rate for the 2012 reporting year

Background

The
IAS strategic audit plan for 2013-2015 includes, for 2013, a limited review on
the RERs reported by DG RTD for the 2012 reporting year.

Similar
reviews were conducted in 2013 for DG AGRI and DG DEVCO.

Objectives and scope of the review

The
objective was to review the calculation and underlying methodology of the RERs
reported by DG RTD in its (draft) AAR 2012 enabling DG RTD to take appropriate
actions, if any, before their disclosure in the final AAR and Synthesis report.

The
scope of the limited review was as follows:

- 
review
of the methodology for the calculation of the RERs that are based on the
results of the audits of the Common Representative audit Sample (CRaS);

- 
testing
of the calculated RERs;

- 
presentation
of the RERs in the draft AAR (including compliance with the Standing
Instructions for the AAR 2012).

In
its 2011 AAR the DG made two reservations concerning the rate of the residual
errors with regard to the accuracy of cost claims in the FP6 and FP7.

The
review was finalised on 06/03/2011. All observations and recommendations relate
to the situation as of that date.

Strengths

The
CRaS was selected for the first time in 2012 to identify the common errors
across the whole of FP7 operations for all research Commission services. This
reduced the audit burden on beneficiaries by reducing the number of repeat
audits whilst providing a representative view of the implementation of FP7.

The
IAS notes the co-ordination in place among Commission research services to
ensure a common approach for auditing the CRaS. In particular, the common
approach is ensured by:

- 
A
common audit manual;

- 
The
Audit Steering Committee for harmonizing audit results at DG level;

- 
The
Extrapolation Steering Committee, which decides on the need to proceed with
extrapolation cases, if an audit detects a systematic error in cost statements;

- 
The
CAR group (Coordination Group for External Audits in Research Family) meetings;

- 
The
Audits' Internal Supervision Committee – chaired by the Director of Directorate
RDT.M, which examines all cases with major errors (over 10% for the CRaS) to
ensure coherence.

Major findings

The
review raised the following very important finding:

Calculation of the RER
(Finding No 1):

The
accuracy of the RER calculated for AAR reporting purpose is affected by the
partial completion of audits of the Common Representative Audit Sample (84%).
Furthermore, the assumption made for the calculation of the RER that 100% of
the participations subject to extrapolation are clean from systematic errors is
found to be over-optimistic considering the past performance, i.e. 79% of FP6
extrapolation are implemented[36].

Recommendations

Calculation of the RER –
Very important:

While
some of the reasons for the delays of the audits are outside the DG's control,
DG RTD should nonetheless further improve its planning of the audits of the
CRaS in order to increase the completion rate of the audit sample at the
reporting date.

DG
RTD should also revise its assumptions and take a more conservative approach
when extrapolating ex-post results to reflect the possibility that part of the
extrapolated systematic errors will not be implemented. DG RTD accepted the
recommendation and corrected itfor the 2013 AAR.

DG RTD
established an action plan which the IAS considers adequate to address the
identified issues.

5.2. Joint IAS-IAC Audit on Grant Management in TEN-T EA

Background

The
IAS-IAC joint audit on Grant Management of the Trans-European Transport Network
Executive Agency (TEN-T EA) was included in the IAS 2010-2012 Strategic Audit
Plan and launched at the end of 2012. Following the rotation of audit
portfolios in Directorate IAS.B, a new audit team in the IAS took over as from
February 2013.

Audit Objectives

The
objective of the audit was to assess the adequacy and effective application of
the internal control system related to the grants under direct management of
TEN-T EA. Taking into account the audits already performed and planned by the
IAS and the IAC, this joint engagement focused on the grant proposals
evaluation process managed by the Agency.

Audit Scope

The
scope of the audit included the sub-processes for which the Agency is fully
responsible in the evaluation process for grants awarded following the call for
proposals.

There
are no observations or reservations in the 2012 AAR that relate to the process
audited.

The
fieldwork was finalised on 17/04/2013. The conclusions in this report relate to
the situation as of that date.

Strengths

The
auditors recognise the on-going efforts made by the TEN-T EA, and in particular
the following aspects:

- 
Guidance
and preparation in the external evaluation - Well
prepared external evaluation as well as training and guidance provided to all
actors involved in the evaluation, from experts to secretaries.

- 
Continuous
improvement of the evaluation process - Each
year, TEN-T EA appoints an 'independent observer' with a view to providing the
Agency with an independent assessment of the conduct and fairness of the
evaluation sessions, on the way the experts apply the evaluation criteria, and
on ways in which the procedures could be improved. In addition, TEN-T EA
launches lessons learned exercises after each call based on a feedback survey
from TEN-T EA staff and the external experts (evaluators for the call).

- 
Clear
and timely information published on the website for applicants.

- 
External
expertise - TEN-T EA uses external expertise from other
Agencies (i.e. ERA and EMSA) to participate in external evaluation panels.

No
significant issues that may adversely affect the achievement of the business
objectives for the process reviewed were identified.

5.3. European Economic Recovery Plan (DG ENER)

Background

The
origin of the European Energy Programme for Recovery (EEPR) lies in the
European Economic Recovery Plan, adopted by the Commission on 26 November 2008
as a response to the economic and financial crisis in Europe. The Plan called
for coordinated national action, complemented by direct EU action, aimed at
injecting purchasing power and boosting demand in the economy. The European
Council endorsed the Plan in December 2008 and invited the Commission to
present a list of concrete energy projects. Regulation (EC) No 663/2009 of
13/07/2009 established the financial envelope for the implementation of the
EEPR for 2009 and 2010 at 3,98 billion EUR.

The
IAS audit on the EEPR in DG ENER was included in the IAS 2012 Audit Work
Programme. This followed the strategic audit risk assessment carried out in
2011. Risks related to the limited time available for setting up and
implementing the programme were verified, as all the funds had to be committed
by the end of 2010 to help Europe recover from the economic crisis.

As
146 million EUR was not committed by the end of December 2010, the Budgetary
Authority authorised the creation of a financial facility supporting
initiatives for energy efficiency and renewable energy. This was done in 2011
by the creation of the European Energy Efficiency Fund (EEEF) investment
vehicle.

The
relative importance of the budget of the EEPR in the context of the IAS Overall
Opinion, justified its inclusion in the IAS' Strategic Audit Plan for
2010-2012.

Audit Objectives

The
objective of the audit was to assess the adequacy and effective application of
the internal control system (ICS), risk management and governance processes
related to the European Energy Programme for Recovery, managed by DG ENER.

In
particular, the audit aimed to assess whether the ICS is adequate and effective
in providing reasonable assurance regarding compliance with the relevant
legislation[37], the
reliability of financial and management information and the effectiveness and
efficiency of the audited processes.

Audit Scope

The
audit specifically covered the following areas:

- 
Implementation,
financial follow-up and closure of the EEPR programme relating to electricity
and gas infrastructure;

- 
Governance
and monitoring of the EEEF under joint management with the European Investment
Bank (EIB).

This
scope takes into account the work already performed by the IAS on the Control
Strategy in DG ENER, the IAC on the Offshore Wind Energy and Carbon Capture and
Storage part of the EEPR and the Court of Auditors on the DAS 2010 and 2011.

There
were no reservations in the 2011 AAR that relate to the audited processes.

The
2012 AAR includes a reservation on reputational grounds related to
beneficiaries of EEPR contracts not respecting public procurement rules when
subcontracting.

The
fieldwork was finalised on 15/02/2013. All observations and recommendations
relate to the situation as of that date.

Strengths

The
auditors recognise DG ENER's following on-going efforts to manage the EEPR in
an effective and efficient way.

EEPR Electricity
and Gas Infrastructure

- 
A
model Commission Decision for the granting of financial aid for actions in the
field of EEPR has been developed, which the operational units have to complete
according to the financed project. The model includes detailed annexes
explaining the financing rules and reporting requirements.

- 
Project
Officers make on the spot visits to verify progress made on the projects with
the assistance of an external technical expert. Further, Project Officers make
a formal appraisal of the reporting information submitted by the beneficiaries
at the time of the interim and the final payments.

- 
Consolidated
monitoring files exist for managing the projects' budget, reporting
requirements and delays. These are regularly crosschecked with the financial
management unit.

- 
Segregation
of duties between the operational and financial side is clearly defined. A
dedicated person in the financial management unit SRD3 is responsible for EEPR
payment files.

EEE F

- 
A
Delegation Agreement with the EIB sets out the rules and reporting requirements
for managing the fund, including the Commission representation in the decision
making process.

- 
Board
meetings are regularly held with detailed agendas and minutes. High-level
Commission staff participates in these meetings: an Advisor in the Management
Board meetings and a Director in the Supervisory Board meetings. The European
Commission is represented with 50% of the votes in the Supervisory Board and
may appoint the Chair. The chairperson has the decisive vote in case of a tie.

- 
A
dedicated person has been assigned in unit C3 for dealing with the technical
assistance approvals of the projects.

Major Audit Findings

EEPR project management and risk
management.

Many
projects are running late due to political and operational difficulties and
part of the funds reserved for some projects have not been used for a long time
since the commitment was made. The Commission report on the Implementation of
the EEPR[38] states
that when a viable solution to mitigate the delays is not found, the Commission
may terminate its financial support to the project, in which case the funds
flow back to the EU general budget. Further, for some projects, the risk
assessments foreseen in the Commission Decision granting financial aid are not
updated. Finally, the Project Monitoring Table is mainly limited to financial
figures and does not include the project's milestones with individual actions.

Public procurement monitoring and
implementation.

Beneficiaries
do not always follow the EC public procurement rules governed by the Directives
2004/18/EC and 2004/17/EC. Reimbursement of ineligible costs incurred by
beneficiaries having concluded contracts in order to carry out the action would
therefore constitute illegal and irregular expenditure.

Audit recommendations

EEPR project management and risk
management (Very Important)

The
IAS recommends DG ENER to reinforce its project management, in particular for
delayed projects, and the project's risk management process in view of the
Europe 2020 Connecting Europe Facility. It should also develop specific KPIs
for managing the EEPR's performance.

Public procurement monitoring and
implementation (Very Important)

The
IAS recommends DG ENER to reinforce and bring forward the timing of ex-ante and
ex-post controls on payment files, including the tendering procedure used by
the beneficiaries, to establish clear, user-friendly guidelines to
beneficiaries regarding the application of public procurement law and to
improve the Project Officers' advisory role to beneficiaries in following public
procurement law.

DG ENER
established an action plan which the IAS considers adequate to address the
identified issues.

5.4. Audit on Control Strategy in EACI

Background

The
EACI is responsible for the management of EU actions in the fields of energy,
entrepreneurship and innovation (including eco-innovation), and sustainable
freight transport under several EU programmes. The EACI is supervised by its
four parent DGs, i.e. DG ENTR, DG MOVE, DG ENER and DG ENV. For the period
2007-2013, the Agency manages four programmes for a total amount of € 1 700
million.

Audit Objectives

The
main objective of the audit was to assess whether the EACI's control strategy
is adequate, effectively implemented, regularly monitored and adequately
reported on, and is ensuring that corrective measures are taken promptly and
proportionately.

Audit Scope

The
audit covered the main building blocks to obtain assurance for the managed
funds, ex-ante and ex-post controls, and the disclosure of the assurance within
the AAR.

The
EACI's AAR does not state any observations/reservations related to the
areas/processes audited.

The
fieldwork was finalised on 31/05/2013. All observations and recommendations
relate to the situation as of that date.

Strengths

The
IAS recognises the EACI's significant efforts to put in place a sound internal
control system when implementing the EU programmes delegated by the European
Commission with the supervision of its four parent DGs.  Moreover, the most
recent EACI evaluation[39] has
pointed out a high overall level of satisfaction by the beneficiaries with the
service delivery (e.g. through simplifications and increased proximity to
beneficiaries).

Major Audit
Findings

The assurance building process and
disclosure of the assurance.

The
EACI has not developed a comprehensive and formalised overall control strategy
explaining how it expects to get reasonable assurance on the legality and
regularity of underlying transactions, with appropriate KPIs. According to its
AAR 2012, the declaration of assurance is mainly based on ex-post controls.
However, the RER provided is not fully representative for all underlying
transactions. An alternative assessment pattern to support the declaration of
assurance, as requested by the 2012 AAR Standing Instructions, should be further
developed. There is room for re-balancing the relative importance of ex-ante
and ex-post controls in terms of the allocation of resources.

Ex-ante checks.

The
ex-ante controls are based on desk reviews, which provide limited assurance
regarding the expenditure checked. The EACI has not established an ex-ante
control plan describing in detail its various constituting elements and how
they contribute to the AOSD's expected level of assurance (e.g. desk reviews,
reporting of error rate, audit certificates, project visits, etc.). The IAS has
found some weaknesses in the ex-ante verification of the eligibility of costs,
KPIs and formal empowerment of staff to sign "passed for payment"
endorsements for the validation of expenditure. Although project visits were disclosed
in the AAR 2012 as one of the elements in the building blocks towards
reasonable assurance, their actual contribution to the assurance is limited due
to the focus on the project deliverables and the very limited number of
financial controls on cost eligibility performed during these missions.

Ex-post checks.

The
target objectives of the ex-post Audit Strategy are not sufficiently clarified
and the actual coverage is far below the targets for some of the programmes
audited for the first time in 2012. The risk factors applied to support the
selection of projects are not sufficiently prioritised and their quality and
quantity can be improved. The methodology for extrapolating audit results to
non-audited projects is not fully developed. Finally, the ex-post control
sector is part of the Internal Audit Function with very limited resources.

Anti-fraud strategy.

As the
EACI has not yet developed an Anti-fraud Strategy and the existing fraud
prevention and detection measures are not fully embedded in an overall control
strategy, the capacity to detect and prevent fraud is very limited. The EACI
does not apply financial penalties, which are key deterrent measures to prevent
fraud.

Audit recommendations

The assurance building process and
disclosure of the assurance – Very important

The
EACI should formally and comprehensively describe its Control Strategy and
re-assess the internal control system, including the link between key controls
and their results and the specific assurance objectives regarding legality and
regularity. The Agency should update the procedures for the reporting by the
AOSDs by including additional control related KPIs and should consider
re-balancing the weight of ex-ante and ex-post controls according to the
assurance to be obtained.

Ex-ante checks – Very
important

The
EACI should strengthen the ex-ante checks by developing a risk-based approach
for all the levels of checks (desk reviews, audit certificates, project visits,
etc.), establish solid and consistent KPIs, reinforce the verification of
eligibility of costs and adopt formal decisions to empower technically
competent staff to provide the "passed for payment" endorsements.
Following the adoption of the new FR and in view of the next MFF, the EACI
should conclude its current internal reflexion regarding the restructuring and
strengthening of the financial function.

Ex-post checks – Very
important

The
EACI should better define and justify global and specific objectives of the
ex-post audit strategy, and improve the risk-based selection process of
projects by reassessing the quantity and the quality of the related risk
factors. The EACI should separate the ex-post control sector from the IAC and
provide it with adequate resources.

For the
next MFF, the EACI should explore the possibility to apply a common ex-post
Audit Strategy with the DGs/Agencies managing the same or similar programmes.

Anti-fraud strategy – Very
important

The
EACI should develop an anti-fraud strategy to ensure proper fraud awareness in
its overall control strategy.

EACI established an
action plan which the IAS considers adequate to address the identified issues.

5.5. Implementation of FP7 Control Systems in REA

Background

The
Research Executive Agency (REA) is responsible for managing research projects
funded under four research themes (People, SMEs, Security, Space) of the 7th
Framework Programme. The Agency carries out its tasks under the supervision of
its parent DGs (EAC, RTD and ENTR) and its Steering Committee.

The
total amount of FP7 funds managed by REA since 2009 amounts to € 6.4 billion
(12% of the entire FP7 budget) for the scientific projects funded under the
four above-mentioned research themes.

Audit Objectives

The
main objective of the audit is to assess whether the REA's FP7 control systems
were effectively and efficiently implemented, in particular as regards the
assurance process, and whether the Agency ensured that corrective measures were
taken promptly and proportionately in order to obtain reasonable assurance on
the legality and regularity of the transactions.

Audit Scope

The
audit assessed whether the REA's FP7 control strategy has produced the expected
results until the end of 2012, in particular:

- 
whether
the Agency has adequately and effectively demonstrated reasonable assurance for
the managed programmes over the years, including the calculation of RERs,

- 
whether
the action plans and other measures put in place to correct weaknesses reported
through the assurance process are adequate, effective and efficient, and
whether the Agency has properly and consistently applied corrective and
deterrent measures during the lifecycle of projects, such as early termination
of projects, financial penalties, liquidated damages and recoveries, and

- 
whether
the Agency implemented, through its Anti-Fraud Strategy, adequate measures to
mitigate the high risk of fraud identified in its own Risk Assessment Exercise.

The
following two reservations were made in the 2012 AAR concerning specifically
the process under the scope of this audit engagement:

- 
RER
above 2% in cost claims for research projects for Space and Security;

- 
RER
above 2% in cost claims for research projects for SMEs.

The
fieldwork was finalised on 7/11/2013. All observations and recommendations
relate to the situation as of that date.

Strengths

The IAS
recognise the REA's efforts to put in place a sound internal control system
when implementing the EU programmes delegated by the European Commission and
supervised by its three parent DGs.

In
particular, the IAS acknowledges the following strengths:

- 
The
REA's supervision campaigns concerning the review of project reports and the
grant agreement termination and recoveries proved to be good practice for
systematically improving its processes and following-up identified
recommendations through the Quarterly Reporting to the parent-DGs and in the
corresponding internal database (ASUR).

- 
The
External Evaluation of the REA covering the first three years of the Agency's
activities[40] comes
overall to a positive conclusion assessing that the REA "has been
efficient and effective" in managing the various programs under its
responsibility and providing administrative and logistical support to services
to all programme areas of the People, Capacities and Cooperation Programmes
(for example in respect of improving "Time to Grant" and "Time
to Pay", maintaining good relationships with parent-DGs and simplifying
its internal organisation). It recognised that the REA successfully addressed
the challenges related to the set-up phase of an Agency. Feedback from key stakeholders
and grant beneficiaries is also generally positive.

- 
The
REA participated proactively in the Research family working groups (e.g. close
cooperation with the ERCEA) and was the first service that developed a
centralised monitoring system for amending grant agreements. The REA also took
an active part in the working groups to prepare for H2020, e.g. on the new
grant agreement models.

Major Audit Findings

Research for the benefit of SMEs theme

Ex-post
audits carried out in 2010-2012 detected that the SMEs schemes were affected by
significant errors. The REA, based on an exception note decided, under certain
conditions, not to initiate any financial correction against the beneficiaries
concerning the detected lack of proper accounting records. Until 31 March 2013,
concerning the two related schemes, the REA had 582 running projects with a
budgetary open balance of €176 million, one 2012 ongoing call for proposal with
a planned budget of €234 million and it had closed 161 projects. Until the time
of the IAS audit (November 2013) REA had not assessed the possible impact of
these errors, which only ex-post controls can detect. However, the direct
ex-post audit coverage of the related schemes remains very low, at around 1%.

Calculation and disclosure of the
representative and RER in the AAR

The
REA, for the purpose of deciding on reservations in its AAR 2012, did not use
the Common Representative Error Rate across the whole FP7 operations as other
Research DGs did. Instead, it opted for an alternative approach. Furthermore,
the REA substituted the error rate relating to legality and regularity by the
financial budgetary impact indicator of the errors.

In the
absence of a representative error rate to rely on for assurance building
purposes, the final assessment on the declaration of assurance for the People
Programme, for which no reservation was made, did not follow the alternative
assessment pattern proposed by the AAR Standing Instructions.

Ex-post control planning, execution and
follow-up

The
REA's Audit Plan does not contain a specific approach to ex-post control
activities (representative sampling, audits on the main (top) beneficiaries and
risk-based audits), KPIs for monitoring the performance of these activities and
the quality of its deliverables, and the definition of the criteria to be used
for the risk-based selection of audits.

REA's corrective actions following the AAR Reservations

The REA
developed an action plan for the AAR reservations. However, it did not include
expected measurable/quantified results, the actions were insufficiently
monitored and their impact on reducing the error rate cannot be measured. The
very limited application of deterrent measures (only liquidated damages[41] as the
REA has never applied any financial penalties), no risk-driven ex-ante controls,
no systematic on-the spot visits and inefficient use of the Certificates of
Financial Statements have also impeded the reduction of the error rates.

Anti-fraud
measures

Although
the REA has identified a significant risk of fraudulent behaviour by
beneficiaries, only a very small number of actual cases of potential fraud have
been detected. In particular, the IAS noted that the risks of double funding
and plagiarism, the two most common fraud types in research grants, are not
sufficiently covered by the REA's current Anti-Fraud Strategy. The total amount
recovered in fraud cases is only € 7 031.

Audit recommendations

Research for the benefit of SMEs theme – Very
important

The REA
should assess the risk of irregularity and potential fraud in the SMEs schemes
and report it in the AAR 2013, clearly stating the reasons and justifications
for any exceptions made and the actions taken, formalise the conditions
established in the Communication Campaign regarding SMEs accounting practice,
fully apply the actions foreseen in the note and launch a specific intensified
audit exercise on these schemes.

Calculation and disclosure of the
representative and RER in the AAR – Very
important

The REA
should use CRaS as the basis for calculating and communicating the representative
and RERs for all managed funds. Any other error rate used to build up assurance
should be disclosed as "detected" and not representative. The
methodology to calculate its net financial impact rate should be aligned to the
one applied by DG RTD and this indicator should be disclosed separately.

Ex-post control planning, execution and
follow-up – Very important

The REA
should formalize a specific audit strategy, defining objectives, priorities,
KPIs for monitoring the performance and an improved risk assessment approach to
increase the number of risk-based audits. In formulating the audit strategy,
the ex-post audit resources should be allocated to the various programmes based
on their risk profile and the resulting cost/benefit ratio.

The REA's corrective actions following the
 AAR reservations – Very important

The REA
should further develop a risk-based approach for ex-ante controls, including
risk-based on-the-spot visits, and improve checklists and the use of
Certificates of Financial Statement. The REA should also apply systematically
the deterrent measures foreseen in the FP7 grant agreements and in particular
develop guidelines to apply financial penalties.

Anti-fraud
measures – Very important

The REA
should speed up the implementation of the actions included in its Anti-fraud
Approach document, initiate a reflection in the Research family on how to
perform a proper risk assessment on double funding, coordinate efforts with
other Research services on anti-plagiarism, and should take into account
specific identified risk cases when establishing its ex-post control strategy.

REA established an
action plan which the IAS considers adequate to address the identified issues.

6. 
External Aid,
development and enlargement

6.1. Limited Review of DG DEVCO's Methodology and Calculation of the Residual Error Rate (RER)

Background

The IAS
strategic audit plan for 2013-2015 included, for 2013, a limited review on the
RERs reported by DG DEVCO for the 2012 reporting year. Similar reviews were
conducted in 2013 for DG AGRI and DG RTD.

In its
annual report for the 2009 financial year, the ECA[42]
recommended that DG DEVCO develops a key indicator for the estimated financial
impact of residual errors after all compliance controls (including ex-post)
have been implemented. Since 2010, DG DEVCO has undertaken work to develop a
methodology for the calculation of its RER in consultation with DG BUDG and the
ECA. The RER methodology was developed by an external contractor (Moore
Stephens LLP) who subsequently carried out a study by applying this methodology
to all contracts closed by DG DEVCO in 2012 (covering all financial
instruments, the EU budget and EDF).Based on this study, a provisional RER
underpinning the Director General's conclusions and statement of assurance was established
for the first time and disclosed in DG DEVCO's 2012 AAR (RER of 3.63%).

Review Objectives

The
objective of the IAS limited review was to review the calculation and
underlying methodology of the RER reported by DG DEVCO in its (draft) 2012,
enabling DG DEVCO to take appropriate actions, if any, before its disclosure in
the final AAR and Synthesis report for the 2012 reporting year.

In
addition, the review also aimed to contribute to the 2012 Overall Opinion of
the IAS.

Review Scope

The
scope of the limited review included an assessment of:

- 
the
design of the RER methodology;

- 
the
accuracy and relevance of its  implementation and the actual calculation of the
RER (with the exception of the appropriateness of the choice, use and
implementation of the statistical method); and

- 
the
presentation of the RER in the draft AAR for the 2012 financial year, including
the underlying comments and explanations by DG DEVCO and compliance with the
Standing Instruction for the AAR 2012.

The
drafting of DG DEVCO's AAR for the 2012 financial year was on-going at the time
of the IAS limited review. The findings and conclusions highlighted in this
report are based on the draft version of DG DEVCO's 2012 AAR as officially
communicated to the IAS[43],
DG BUDG and SG on 28 February 2013. Given the limited time available to carry
out this limited review and the on-going nature of the process at the time of
its engagement, the IAS could not fully review the work of the external
contractor. The IAS carried out a limited analysis on a sample of 23
transactions (out of a total of 226 transactions) tested by the external
contractor.

The
fieldwork was finalised on 15 March 2013. All observations and recommendations
relate to the situation as of that date.

Strengths

The RER
study launched by DG DEVCO is an important step towards the improvement of its
assurance building process. It addresses the recommendation made by the ECA on
the implementation of a "key indicator for the estimated financial impact
of residual errors, after all ex-ante and ex-post controls have been
implemented"[44].

DG
DEVCO discussed the design of the methodology with the main stakeholders (DG
BUDG and the ECA) and the adopted methodology (issued in December 2011) took
into account their comments. DG BUDG was of the view that “the methodology,
pilot study and manual could be used as good practice examples even outside the
specific DEVCO management environment”[45], and
encouraged DG DEVCO to share the results with the External Relations family of
DGs.

The
development of the RER methodology was guided by cost-efficiency factors
related to this particular exercise (i.e. verifying transactions rather than
whole contracts and limiting the sample to 240 items[46]
covering the entire financial universe of DG DEVCO).

The
methodology was designed to provide a 95% confidence level and takes into
account a materiality level of 2%.

Major Findings

Compliance with Commission's AAR Standing Instructions – ABB Stratification (finding no. 1)

DG
DEVCO implements its programmes through a variety of instruments, management
modes and contract types, each with its own risk typology. The Standing
Instructions also require the materiality of weaknesses to be calculated for
each relevant ABB activity. Due to cost-efficiency considerations, the RER
study has been performed without allowing for any differentiation by ABB
activity with the use of one single sample. However, the 2012 draft AAR does not contain sufficient information to enable an assessment of the cost-efficiency
of the controls in place to be made. Moreover, no explanation was provided in
the draft AAR on the reasons for the different most likely error rates of the
EU Budget and EDF respectively identified by the ECA in its annual reports on
the 2009, 2010 and 2011 reporting years.

Implementation
of the methodology - Reliability of CRIS data (finding no.3)

Risks
related to data accuracy and reliability of CRIS had already been identified in
previous IAS audits and the ECA's Special report on CRIS (no. 5/2012). In
addition, the initial population of transactions extracted from CRIS contained
transactions that do not fall under the scope of the RER study. There is no
assurance that other transactions not falling under DG DEVCO's RER scope have
not remained within the final population of transactions.

Reliability of the RER study and its
incorporation in the draft AAR (findings no. 3 and 4)

The RER
study concludes that the RER is 3.63% (i.e. 2.67% based on actual errors found
and 0.96% based on an estimation of errors). According to the terms of reference
agreed between DG DEVCO and the external contractor, the RER study is
considered to be a measurement exercise, i.e. not an assurance engagement
intended to provide an opinion on the actual RER. The review by the IAS of the
work performed by the external contractor has revealed a number of weaknesses
such as the absence of a methodology that is consistently applied for
estimating errors in the absence of underlying documentation.

As a
result, the IAS considers that no reliance can be placed on the projected
estimated error of 0.96% and that the provisional statement included in the
draft 2012 AAR that "it is possible to give assurance regarding the 96.4%
of the budget implemented in 2012" has not been fully substantiated.

In
addition, the importance attached to the RER exercise in the draft AAR is not reflected as such in its narrative part. It is also not clear how the other
"building blocks" or the errors detected in the implementation of
other various internal controls contribute to the declaration of assurance and
taken into account when issuing the reservation.

Recommendations

Very important
recommendations for the next AAR (2013 reporting year)

DG
DEVCO should consider including in its 2013 AAR, as part of its obligation
stemming from the new Financial Regulation, an overview of all costs (both
internal and external) associated with existing controls in place. This
assessment of the overall cost of controls, together with the future
conclusions of the working group on cost of controls led by DG BUDG, may allow
to identify opportunities for cost efficiency and/or areas where controls could
be redesigned allowing the AOD to assess weaknesses by ABB activity (finding
no. 1).

DG
DEVCO should ensure that appropriate steps are taken to ensure the use, for the
next RER exercise, of a database of transactions extracted from CRIS where
consistency checks have already been performed (finding no. 3).

Very important
recommendations for the 2012 final AAR

DG
DEVCO should consider improving the narrative part of the AAR on the RER by:

- 
Making
a distinction between the actual errors identified and those estimated and
ensure that its statement on providing assurance on the implementation of the
2012 budget not being affected by errors is fully substantiated. (finding no.
3)

- 
Providing
a comprehensive explanation of its overall control strategy, the
inter-relationship between the various controls performed by DG DEVCO and a
detailed presentation of the methodology and the assumptions used for the
measurement exercise. (finding no. 4)

DG
DEVCO
established an action
plan which
the IAS considers adequate
to address
the identified issues.

6.2. DG DEVCO's state of preparedness for the revised ECA DAS Methodology (Performance Audit)

Background

In
2011, the ECA introduced methodological changes to its audit on the legality
and regularity of underlying transactions for the EU and European Development
Fund (EDF) budgets. The 2011 financial year served as a dry-run exercise for
the Court’s revised methodology and the new approach was applied by the Court
for the first time as of the 2012 reporting year.

The
methodological changes concern (a) the redefinition of the underlying
transactions in its audit sample (which now excludes pre-financings but
includes clearings), and (b) the method for quantifying serious irregularities
in public tendering in all policy areas, i.e. findings which would seriously
impair the application of the principles of “open, fair, transparent
competition” and “award to the best qualified bidder”. These serious procedural
procurement errors will be quantified by the Court at 100% for all management
modes.

Given
the Court's decision to apply the revised methodology ex-nunc (i.e. only for
new public procurement procedures which led to commitments and/or payments
based on contracts signed from 2011 onwards), the changes related to the
quantification of serious procedural procurement errors will gradually impact
the error rate in the following years, as each year more transactions (i.e.
clearings, payments etc.) based on contracts signed after 2011 will take place
and will thus be included in the Court's sample.

In view
of the Court's new approach, which is likely to increase the reported error
rates due to the exclusion of less error-prone pre-financing transactions from the
sample and the qualification of serious irregularities in public tendering at
100% for all management modes, the IAS undertook an audit on DG DEVCO's state
of preparedness for the revised ECA DAS (Statement of Assurance) methodology.

Audit Objectives and
Scope

The
objective of the IAS audit was to assess DG DEVCO's preparedness to deal with
the ECA's revised DAS methodology.

The
audit focused on the actions decided, planned and implemented by DG DEVCO in
order to address the ECA revised DAS methodology and ultimately to mitigate the
discharge risk (i.e. DG DEVCO's strategy and actions taken as from 2012 onwards
based on the conclusions related to errors identified by the Court in previous
DAS exercises, specific actions related to 2012 DAS exercise, other actions
envisaged as from 2013 onwards).

The
fieldwork was finalised on 10 June 2013. All observations and recommendations
relate to the situation as of that date, with the exception of the final ECA
error rates quoted which were made available on 1 July 2013.

Major Findings

The audit showed that DG DEVCO has made
adequate preparations in order to mitigate the discharge risk associated with
the revised ECA DAS Methodology.

DG DEVCO
has prepared a comprehensive action plan that deals with the revised DAS methodology
and also addresses other issues identified by its external auditors or in
audits carried out by the IAC of DG DEVCO or the IAS. It was approved by DG
DEVCO's senior management on 22 May 2013. Due to its recent issuance, the
effective implementation of the action plan by DG DEVCO was not assessed by the
IAS as part of this audit. The
IAS did not issue any recommendation.

6.3. Audit on Financial Management Humanitarian Aid (DG ECHO)

Background

The
Directorate General for Humanitarian Aid and Civil Protection (DG ECHO)
provides humanitarian aid to the most vulnerable victims of disasters in their
immediate aftermath and also in cases of complex and protracted crisis
situations, whenever needs arise that overwhelm the capacities of the
governments and local authorities of the third countries concerned or when
authorities in third countries affected are not willing to provide assistance.[47]

Humanitarian
aid represents over 60%[48] of DG
ECHO's budget (around EUR 650 million) implemented under direct centralised
management mode[49] mainly
through European non-governmental organisations (NGOs) and specialised agencies
in MSs and under joint management mode49
through United Nations (UN) agencies and international organisations such as
the International Committee of the Red Cross. In addition, DG ECHO maintains
also a network of offices in 37 countries[50] to
support its operations in the field. Most of these field offices manage an
imprest account. Controls in place to ensure a sound financial management of
humanitarian aid include ex-ante controls on projects and quality controls on
partners, audits and verification missions[51]
carried out by internal staff and by contracted auditors[52] and
ex-post controls.

Audit Objectives and Scope

The objective of the audit was to assess
the adequacy and effective application of the internal control system, risk
management and governance processes related to the financial management of
Humanitarian aid operations funded by DG ECHO, and in particular:

- 
Whether
final payments were made in accordance with the regulatory and contractual
requirements (including the imprest accounts),

- 
The
efficiency and effectiveness of the ex-post controls put in place for the
detection and correction of errors, and

- 
The
reporting of financial management in the DG's AAR.

- 
The
following areas were considered to be out of the scope of the audit:

- 
Compliance
with payment deadlines (as they were extensively covered in recent IAS and IAC
audits[53]);

- 
Disaster
preparedness (low impact on the budget) and food aid (already covered by the
IAS[54]).

There
are no observations/reservations in the 2011 AAR that relate to the
area/process audited.

The
fieldwork was finalised on 12 December 2012. All observations and
recommendations relate to the situation as of that date.

Strengths

The
auditors recognise the on-going efforts made by DG ECHO (Directorate C) to
strengthen the controls on the humanitarian aid operations. In particular:

- 
Payments:
DG ECHO ensures the correct application of the "certified correct"
procedure and improved the quality of its payment checklists in 2011;

- 
Imprest
account: A comprehensive methodology on ex-post controls has been developed
with a good coverage of the imprest account;

- 
Audits:
Methodology in place for the preparation of the annual audit plan; and

- 
AAR:
Methodology developed for the calculation of the RER.

Major Audit
Findings

Imprest account (Finding no 2)

Although
the procedures for the management of the imprest accounts have undergone
changes and administrative requirements for the submission of supporting
documents have been made lighter, no corresponding increased focus has been
given to ex-post checks. Moreover, the process for the preparation, submission
and verification of imprest expenses is managed inconsistently between ECHO
offices and within the same office.

Follow-up of audit recommendations
(Finding no 3)

The
current audit procedure does not require the auditee to submit an action plan
with implementation dates. Horizontal audit recommendations with a potential
financial impact addressed to partners are followed up a long time after the
recommendations are supposed to be implemented. A significant number of
recommendations related to horizontal issues are not closed but simply
re-issued.

Residual error rate (Finding no 6)

The
methodology developed by DG ECHO to calculate the RER that is reported in its AAR is based on the implementation of the audit strategy. Weaknesses in the methodology
(use of commitment rather than payment appropriations and methodology not
sufficiently explained and documented) and process inefficiencies in the
recovery procedure (long time lag from the time an audit report is finalised to
the potential issuance of a recovery order, non-formalisation of objective
criteria and responsibilities in determining amounts to be recovered) may have
an impact on the calculation of the annual RER.

Audit
recommendations

DG
ECHO should strengthen controls, adapted to the specific environment it
operates in, in order to address the findings:

- 
DG
ECHO should review the roles and responsibilities related to the imprest
account, taking into consideration the specific risks presented by the
different actors in the process. DG ECHO should also ensure that ex-post checks
comply with internal procedures and provide appropriate written guidance and
training to staff involved in the management of the imprest account. (very
important)

- 
DG
ECHO should ensure a timely and systematic follow-up of audit recommendations
addressed to partners, in particular those with a potential financial impact
for the Commission's finances. DG ECHO should consider introducing an
obligation for DG ECHO partners to submit an action plan with deadlines for the
implementation of recommendations of a potential financial impact, including
clarifying deterrents in case of non-implementation. This could be combined
with the development of an internal monitoring tool. (very important)

- 
DG
ECHO should review and streamline its recovery procedures bearing in mind the
cost-effectiveness of its controls. The calculation of the RER should be better
formalised and documented to ensure its representativeness in a given year.
(very important)

DG ECHO established an
action plan which the IAS considers adequate to address the identified issues.

6.4. Procurement – decentralised (DG DEVCO)

Background

DG
DEVCO is responsible, together with the European External Action Service
(EEAS), for the multi-annual programming of the external aid instruments that
come under its remit. It implements the European Union's external aid
instruments financed from the European budget and the European Development Fund
(EDF) mostly in a devolved manner, through the EU Delegations in countries
outside the European Union.

Procurement
contracts in DG DEVCO fall under both centralised direct and decentralised
management modes[55]. Under
decentralised management, the Commission may decide to entrust the management
of certain tasks (procurement award procedures and payments to final
beneficiaries) to the authorities of beneficiary third countries. As reported
by DG DEVCO in its 2012 AAR[56], the
cumulative amount of procurement under decentralised management mode and
programme estimates represent 14% of the total value of new legal commitments
of the DG.

The IAS
audit on Procurement under decentralised management mode (ref.
IAS.B5-2013-DEVCO-004) was included in the IAS 2013 Audit Work Programme
following the risk assessment carried out in 2013 by the IAS as part of the
preparation of the IAS Coordinated Strategic Audit Plan for 2013-2015. The main
risk related to procurement under decentralised management mode relates to the
lack of capacity of national administrations of the recipient countries to
manage EU funds, making delivery of aid less effective and more risky. In
addition, the ECA is applying a revised methodology on errors related to
procurement (with serious errors leading to a 100% error rate) in all
management modes as from the 2012 reporting year.

Audit Objectives

The
objective of the audit was to assess the adequacy and effective application of
the internal control system, risk management and governance processes related
to procurement award and contracting processes of the European Development Fund
(EDF) and the EU budget which are implemented under the decentralised
management mode, including procurement awarded within the imprest component or
as a specific commitment of Programme Estimates.

Audit Scope

The
following issues related to procurement under decentralised management were
included in the scope of the audit:

- 
Steps
of the procurement procedure from the needs analysis and planning to the
signature of the contract,

- 
General
coordination by the EU Delegations (EUDs)/DEVCO headquarters (HQ) units of the
procurement process, and

- 
Horizontal
issues: staff availability and resources, specific training, filing and
archiving, use of relevant IT systems.

DG
DEVCO made a global reservation in its 2012 AAR based on a RER of 3.6%, due to
the significant occurrence of errors in the underlying transactions (legality
and regularity). The "lack of documentation in tender procedures"
represented 15.2%[57] of the
overall error rate. However, there is no specific reference in this reservation
to issues related to procurement under decentralised management mode[58].

This
audit report summarizes the main findings following the IAS fieldwork in the
context of the audit, including missions to four EUDs (Ghana, Jordan,
Mauritania and Barbados and the Eastern Caribbean), as well as a review of the
tender files managed by DEVCO.E3 and E4 units with the Africa, Caribbean and
Pacific (ACP) Group of States Secretariat. The IAS reviewed only those steps of
the procurement procedures under the scope of our audit that had been finalised
as at the start date of each of our missions to EUDs/DEVCO HQ (i.e. 26 August -
 Ghana, 15 September – Jordan and 30 September 2013 – Mauritania and Barbados and Eastern Caribbean).

The
fieldwork was finalised on 8 November 2013. All observations and
recommendations relate to the situation as of that date.

Strengths

The
IAS noted the following strengths concerning the organisation and the
implementation of the tender procedures:

- 
The
calls for tender and contracts included in the audit sample were launched and
signed in accordance with the relevant Financing Decisions;

- 
The
applicable thresholds that determine the choice of the relevant procurement
procedures were adequately applied[59] and
there is no indication of calls for tender split into smaller amounts in order
to qualify for a different type of procurement procedure and/or qualify for a
different authorisation level (except for Programme Estimates);

- 
There
is a clear distinction between exclusion, selection and award criteria in all
contract notices reviewed and the award criteria were adequately set;

- 
In
all the calls for tender included in the audit sample, DG DEVCO staff attended
(part or all) the meetings of the Evaluation Committee, the composition
structure of which was adequate in all cases.

Major Audit Findings

Ethics and conflict of interest (finding
no. 1):

Two
cases raising questions related to ethics and prevention of conflict of
interest were identified by the IAS. In these two cases, at the time of the
signature of the contract, the key expert of the successful bidder was either
(i) still a member of staff of the EU Delegation (local agent), or (ii) an
official representative of the Contracting Authority.

Audit recommendations

Ethics and conflict of interest – Very
important

DG
DEVCO should clarify in its instructions (PRAG) the specific instances of
experts proposed by tenderers, to ensure that:

- 
members
of staff of the EU Delegations (local agents) must have officially terminated
their contract with the EUD before starting to work on an EU project under a
contract with an external organisation/company, and

- 
civil
servants or collaborators/other staff of other beneficiary entities, such as
international or regional organisations shall not be recruited as experts by
tenderers, unless the prior approval of the European Commission has been
obtained (as is the case for the civil servants or other staff of the public
administration of beneficiary countries).

DG DEVCO established an
action plan which the IAS considers adequate to address the identified issues.

7. 
Education and citizenship

7.1. Performance audit of National Agencies (DG EAC)

Background

Knowing
how well an organisation is doing is essential in developing strategy and
policies to meet the organisation's aims. Performance information shows how
well an organisation is performing against its stated objectives. Clear
objectives, with well-selected performance measures, are the foundation of a
good performance measurement system. This not only determines what needs to be
done but also sets the conditions for success.

Following
the entry into force of the Lisbon Treaty, the European Commission is required
to submit to the European Parliament and the Council a yearly evaluation report
on the Union’s finances based on the results achieved (Article 318 report).
Moreover, the European Parliament considers that performance audits measuring
the extent to which spending has achieved its objectives pursued have become
ever more important and the Commission should develop a new culture of
performance. In June 2011, the central services of the Commission reminded
operational services that the current financial and economic crisis and severe
constraints in public spending has shifted the focus to achieving more with
less and that post-2013 spending programmes should place a greater focus on
performance measurement. Other initiatives have been taken at Commission level
on this move towards a performance culture, e.g. the setting up of a working
group of Directors-General of the Commission on “Move to a performance culture”
in February 2013.

In
order to meet stakeholders' expectations, one of the objectives of the IAS in
its Strategic Audit Plan for the period 2013-2015 is to carry out performance
audits focusing on the economy, efficiency and effectiveness of the use of
resources. A number of audits have been planned in 2013 to assess the performance
measurement system of DGs, such as DGs REGIO, EMPL and EAC.

The
Directorate-General for Education and Culture (DG EAC) aims at reinforcing and
promoting lifelong learning, linguistic and cultural diversity, mobility and
the engagement of European citizens, in particular the young. In order to
accomplish its mission, the designated bodies, i.e. National Agencies (NAs) in
the MSs, implement a number of programmes (representing approximately 51% of
the total budget for the period 2007-2013 of DG EAC). The NAs are co-funded by
the Commission through operating grants (representing €354,9 m over the
2007-2013 period), covering approximately 50% of their running costs.
Externalisation, as a form of management, imposes additional risks in the
set-up of the supervisory system in DG EAC for the achievement of its policy
objectives.

The new
MFF for the period 2014-2020 establishes the “Erasmus+” programme, which will
bring together several activities currently covered by separate programmes. The
budgeted commitments are expected to increase by approximately 64%, with an
allocation of €19 billion for the 2014-2020 programming period.

Audit Objectives

The
main objective of the audit was to assess whether DG EAC has set up an
effective performance measurement system to monitor, report and evaluate the
performance of NAs.

In
particular, the audit assessed two specific dimensions:

- 
the
internal dimension of the process: i.e. how DG EAC measures the economic and
efficient use by NAs of their operating grant, and

- 
the
external dimension of the process: i.e. how DG EAC measures the efficient and
effective use by NAs of the decentralised grants.

Audit Scope

The NAs
implementing the Lifelong Learning (LLP) and Youth in Action (YiA) programmes
were included in the scope of the audit.

The
audit focused on the 2011 work programmes and annual reports of NAs as this is
the most recent year where a full implementation cycle of activities could be
assessed. In addition, the 2012 and 2013 work programmes and 2012 annual
reports were reviewed for the NAs visited.

There
are no observations/reservations in the AAR that relate to the area/process
audited.

The
fieldwork was finalised on 8 July 2013. All observations and recommendations
relate to the situation as of that date.

Strengths

The
proposal establishing the new "Erasmus+" programme places greater
emphasis on performance and results. It requires the Commission, in cooperation
with the MSs, to regularly monitor and report on the performance and results of
the programme against its stated objectives.

The IAS
recognises the on-going efforts that DG EAC is making in defining a performance
measurement system that will be implemented for the 2014 – 2020 programming
period. DG EAC started the preparations for the next programming period
relatively early at the beginning of 2012, by setting up a number of working
groups to deal with various aspects of the new programme, including one
specifically on the development of indicators, and by launching a number of
consultations, both within the DG and with the NAs. This has resulted in a high
level of commitment among the NAs to ensure the successful implementation of
the Erasmus+ programme.

The IAS
notes that the overall performance of NAs is assessed by DG EAC on the basis of
a set of elements, such as the NAs declarations of assurance, the yearly
reports from NAs, visits, etc. which provide assurance to the DG in terms of
legality and regularity. In addition, the performance of NA management of the
decentralised funds is measured on the basis of the budget executed, number of
final beneficiaries, respect of deadlines, number and type of complaints, etc.

Major Findings

The IAS has identified the following three
Very Important issues: Internal dimension of the process:

- 
Indicators
which explicitly measure the efficient or cost-effective use of the operating
grant by NAs (i.e. use of resources and link between performance and resources
available) have not been defined. An analysis performed by the audit team has
revealed wide variations in the economic and efficient use of the operating
grant (Finding 1).

External dimension of the process:

- 
Although
the Estonian YiA NA has successfully cascaded its high-level objectives, NAs in
general cannot clearly link their objectives with the higher-level objectives
of the Commission. In particular, the current objectives set at NA level are
too numerous (LLP), with the pre-defined actions not always relevant for each
country/action (LLP), or contain too many confusing priorities (permanent,
annual, national) (YiA). There may also be lost opportunities to achieve
synergies with programmes implemented by other services of the Commission. In
addition, the indicators (and the associated targets) used for the management
of the decentralised funds are mostly output-based (i.e. measurement of the efficiency
of the use of funds), with few or no indicators on outcome or impact
(measurement of the effectiveness of the use of the funds). Moreover, some of
the indicators used in DG EAC's Management Plan do not yet fulfil the RACER[60]
criteria (Finding 2).

- 
Weaknesses
in the definition and use of performance measures (indicators, targets and
milestones) at NA level (Finding 3).

Audit recommendations

To
mitigate the risks related to the issues noted above, DG EAC should:

- 
Establish
analytical techniques to monitor the economic and efficient use of the
operating grant by NAs and enable it to drive performance improvement
while ensuring the cost-effectiveness of the use of resources.

- 
Decompose
its high level political objectives into lower-level
SMART objectives in its MP and filtered down to NA level. The objectives set
for NAs should be specific and limited in number.

- 
Review
its indicators in its MP in order to ensure that they meet the
RACER criteria and cover both outputs and outcomes accompanied by targets and
milestones.

- 
Better
define performance measures (indicators, targets and milestones) for
NAs, which should be regularly assessed in order to identify unexplained
variances that may require a change in the implementation of the programme.

DG EAC established an
action plan which the IAS considers adequate to address the identified issues.

7.2. Performance Audit on Risk Management in DG COMM

Background

Communication
to the Commission SEC(2005)1327 “Towards an effective and coherent risk
management in the Commission services” introduced a risk management
framework in the Commission aimed at implementing a "continuous,
proactive and systematic process of identifying, assessing, and managing risks
in line with the accepted risk levels, carried out at every level of the
Commission to provide reasonable assurance as regards the achievement of the
objectives".

Risk
management in the Commission is embedded into the planning and decision-making
processes (SPP/ABM cycle) and each DG/Service should ensure, in that respect, that
it operates an effective and continuous Risk Management process.

In
2011, the IAS launched an audit to assess the Risk Management framework
implemented in DG COMM[61]. At
that time, the process was at an early stage of implementation and important
weaknesses were detected, in particular regarding the roles and
responsibilities in the Risk Management process, the methodology for
identifying and assessing risks, the reporting on risks[62] and the
implementation of the Risk Management process as a continuous exercise. For
this reason, the IAS issued an Interim report inviting the DG to address
properly the issues identified during the preliminary survey.

Audit Objectives and Scope

The
objective of this audit was to assess the progress made by DG COMM in the implementation
of the Risk Management process compared to the situation observed in 2011 when
the IAS launched the audit. Overall, the audit assessed:

- 
compliance
with the corporate framework and existing guidelines;

- 
the
adequate and effective implementation of Risk Management in DG COMM.

In
particular, the focus was on:

- 
roles
and responsibilities in the  risk management process;

- 
the
process in place for:

- 
identifying
and assessing risks (including cross-cutting ones);

- 
defining,
implementing and monitoring mitigating actions;

- 
reporting
on risks (risk register, Management Plan, AAR).

- 
knowledge
management (instructions, training) and awareness raising initiatives.

No
scope limitations were identified.

The
fieldwork was finalised in mid-November 2013. All observations and
recommendations relate to the situation as of that date.

Overview of the major strengths

DG
COMM has made significant efforts since 2011 to implement a Risk Management
process. The following strengths were observed:

- 
Strong
support from top management during the process of building DG COMM’s risk
management framework and regular top-level steer of the exercise (discussion of
significant risks and endorsement by the Senior management);

- 
Very
committed staff involved in the Risk Management process;

- 
Creation
of a Risk Management Group (RMG) in charge of monitoring the exercise and
ensuring consistent implementation of BUDG guidelines and quality review;

- 
Implementation
of the Risk Management as a continuous exercise, with a regular review of the
risks and update of the Risk Register.

Audit Findings

The IAS
did not identify any issues that gave rise to critical or very important
recommendations.

8. 
Economic and financial affairs

8.1. Performance Audit of GMES/Copernicus Programme (DG ENTR)

Background

The
Global Monitoring for Environment and Security (GMES) is an EU-led Earth
observation programme, carried out in partnership with the MSs and the European
Space Agency (ESA). Its objective is to provide, on a continuous basis,
reliable and timely services related to environmental and security issues.

The
programme is based on the development of three main components: the space
component consisting of five dedicated satellites; in situ observations (e.g.
airborne, seaborne); and service covering six areas (i.e. land monitoring,
marine monitoring, atmosphere monitoring, emergency response, security support
and climate change).

The
total EU contribution for GMES was approximately €1.2 billion over the period
2008-2013. Approximately 60% has been dedicated to the development of the space
component and implemented indirectly by entrusting budget implementation tasks
to the European Space Agency (ESA). The EU contribution to the space component
only covered approximately 30% of its total cost, the rest was paid by ESA's MSs.
The first satellite will be launched at the beginning of 2014.

The
service component is based on a series of research projects financed by the
successive EU framework programmes for research (i.e. FP5, FP6 and FP7). Two
services are operational since 2012.

This
audit takes place in a transition period between the current GMES Initial
Operations (GIO), implemented in 2011-2013, and the operational phase of the
programme, renamed Copernicus, in the next MFF, which will increase the total
contribution from the EU budget to approximately €4.3 billion over the period
2014-2020.

DG ENTR
is mainly responsible for the overall coordination of the programme's
activities and implementation of the EU dedicated Budget.

Audit Objectives and scope

The
overall objective of the audit was to assess the effectiveness of the
governance arrangements, risk management and internal control systems put in
place by DG ENTR to support the management of GMES. The audit addressed this
overall objective by examining mainly the following:

- 
whether
the relationship between the bodies involved in GMES is clear;

- 
whether
the management and oversight by the Commission is accountable and has clearly
defined objectives and responsibilities;

- 
whether
DG ENTR organisational structure is adapted to the goals of GMES;

- 
whether
DG ENTR plans its operations adequately to achieve its objectives;

- 
whether
DG ENTR identifies and manages its risks;

- 
whether
DG ENTR evaluates the results of the implementation of GMES/GIO, identifies and
implements the lessons learned;

- 
whether
DG ENTR has adequately prepared the transition towards Copernicus.

DG ENTR
included the following reservation in its 2012 AAR concerning specifically the
area under the scope of this audit engagement: Reservation concerning the
reliability of the annual financial reporting by the European Space Agency
(ESA) about the joint implementation of the space component of the Global
Monitoring for Environment and Security (GMES).

The
fieldwork was finalised on 18/10/2013. All observations and recommendations
relate to the situation as of that date.

Strengths

The
IAS recognizes the efforts made by DG ENTR to achieve the following objectives:

- 
Two
services – i.e. Land Monitoring and Emergency Management - are fully
operational. An external evaluation concluded that they are highly relevant to
the identified needs of users. As regards the space component, the first
satellites will be ready for launch in 2014.

- 
The
Commission adopted a Delegated Regulation[63] on
data policy and user access to GMES dedicated data and service information.
This will clarify for all stakeholders the principles of free, full and open
access to Copernicus data and information.

- 
EC-ESA
cooperated via a task force on financial reporting to work on actions following
the qualification of ESA 2011 accounts in order to have no qualification for
2012. This objective was achieved with the ESA independent auditor's report no
longer containing a qualified opinion on ESA’s annual accounts for 2012.

- 
The
administrative arrangements with REA, EEA and JRC as described in the related
Service Level Agreements are clear and appropriate with regard to the
distribution of roles and responsibilities.

Major Audit Findings

The IAS has identified the following three
Very Important issues: Governance

Whilst
the Commission should ensure the overall coordination of the EU Earth
observation programme including programme management, federation of user
requirements, and EU budget implementation, the current Delegation Agreement
(DA) signed with the European Space Agency (ESA) includes a number of
limitations in the Commission's control functions as regards ESA. In this
respect, DG ENTR's audits and checks are strictly limited to the financial
aspects, excluding any relevant technical and operational elements. Although
recognizing the context of the negotiation of the current DA (under which the
EU participation is only 30% to the space component), the IAS assesses that the
next DA covering the period 2014-2020 under Copernicus should enable an
increased EU control over the programme in line with the planned increase of
its contribution up to 70% of the space component and the Commission’s
responsibilities in the areas of programme, contract and asset management.

Management, Monitoring and Evaluation

DG ENTR
follows up and reports on the developments of the space component by means of
high level roadmaps showing the key milestones prior to the launch of
satellites set by ESA and attends key progress and acceptance reviews. Since
ESA was de facto programme manager for the development of the Space Component,
DG ENTR was not in a position to produce more detailed roadmaps (e.g. including
objectives/milestones) to follow closely the programme implementation. ESA's
current reporting does not allow a clear comparison between actual versus
budgeted expenditure at a detailed level. Although the IAS recognizes the
context mentioned above (see issue on Governance), it assesses that the current
arrangements to monitor and report on the progress of the space component from
a technical and financial point of view should be improved in line with the
Commission's overall responsibility over the programme in particular in the
context of increased EU participation and funding scheme under Copernicus.

The
external evaluation on GMES initial operations[64] (i.e.
covering the two operational services) recommended developing a formal
monitoring framework and indicator system.

The IAS
notes that the interim evaluations required in the FP7 legal base[65]
as well as the Regulation on GMES and its initial operations (GIO) were issued
with a significant delay and no formal specific and structured action plan was
established to address the weaknesses identified.

Transition to Copernicus

The
Commission’s proposal for a Regulation establishing the Copernicus programme
has been submitted to the Council and the Parliament. According to DG ENTR, its
entry into force may be delayed up to the second semester of 2015.

Such
delayed adoption of the Regulation may prevent the Commission to finance
certain key activities related to the manufacturing of the satellites or to the
implementation of the services. DG ENTR is investigating potential remedial
scenarios, in particular consulting DG BUDG on an exception that may allow the
use of 2014-2020 MFF funds before the Copernicus Regulation is adopted.

The IAS
was not provided with any clear and detailed calculation supporting the budget
set for the space and service components (€2944 million and €1328 million
respectively). Although recognizing the difficulty for DG ENTR to establish or
challenge the estimates from a technical or cost point of view, the IAS
considers that the current level of detail of the budget estimates for the
different components of Copernicus does not enable an appropriate monitoring of
the programme.

For
Copernicus, in line with the need for enhanced control stemming from the
planned increase of the EU contribution to 70% of the space component, DG ENTR
has proposed that the Commission will become the contracting authority for the
space component. This will entail additional responsibilities, which need to be
supported by appropriate organizational arrangements. The EU will also become
the owner of all tangible and intangible assets created or developed under
Copernicus, subject to the adoption of the final text of the Copernicus
Regulation in this regard. Despite these increased responsibilities, the IAS
was not provided with a detailed analysis of the impact of such decision on the
expected changes in the DG's organisation.

Audit recommendations

To
address the above mentioned issues, DG ENTR should:

- 
Ensure
that the new Delegation agreement with the European Space Agency is negotiated
with the objective to appropriately reflect the overall responsibilities of the
Commission for the programme, in line with the applicable rules.

- 
Negotiate
with ESA for the next delegation agreement that all necessary information for
an effective monitoring of the implementation of the delegated tasks are
provided, in particular for monitoring the delays and the budget consumption.

- 
Develop
an appropriate objectives and indicators framework to monitor and report on the
progress of the programme overall and for each individual service.

- 
In
the context of Copernicus, improve its monitoring of the actual versus budget
consumption, in particular for the space component, with due regard to the
change of funding schemes which implies that cost overruns may have to be
financed by the EU. In case a reserve is used to cover unexpected expenditure,
clear provisions should be in place so that this is duly justified by ESA and
agreed by DG ENTR.

- 
Draw
a structured action plan at operational level to remedy the weaknesses
identified in the interim evaluation report to enable continuous improvement in
the management of the GMES-Copernicus.

- 
Coordinate
with DG BUDG to prepare remedial plans ensuring continuous funding of the
Copernicus activities, in case of delayed adoption of the Regulation.

DG ENTR established an
action plan which the IAS considers adequate to address the identified issues.

9. 
General services and HR

9.1. DG ESTAT's preparedness to fulfil its role in the Economic Governance Framework (Performance Audit)

Background

The
mission of DG ESTAT is to be the leading provider of high quality statistics on
 Europe[66]. To
this effect, the statistical production of the DG covers the whole range of EU
policies and supports the functioning of key activities, such as the
verification of public finance statistics in Excessive Deficit Procedures
(EDP), the EU 2020 strategy, social Europe, energy and climate, the calculation
of own resources, etc.

Government
Finance Statistics (GFS) play a key role in EU economic monitoring. They
include in particular the data on government debt and deficit reported under
the EDP. On behalf of the Commission, DG ESTAT is responsible for assessing the
quality of the EDP statistics transmitted by MSs and for providing the
statistics to be used for EDP. It has the sole competence for the methodology
underlying these statistics.

The
European Union governance framework for fiscal statistics was in the light of
the recent economic crisis in MSs found to be insufficient to mitigate to the
extent necessary the risk of incorrect or inaccurate data being notified to the
Commission. Accordingly, to ensure that DG ESTAT's objective to publish deficit
and debt figures of the highest quality is met, the key legislation (Regulation
479/2009) was amended in 2010. This amendment[67]
extended the powers of DG ESTAT and the National Statistical Institutes (NSIs)
with respect to their assessment and control of upstream data.

The
economic and financial crisis also revealed a number of key structural
weaknesses in the economic governance of the EU's economic and monetary union.
The cornerstone of the EU response has been to approve the so-called economic
governance "six-pack" of new rules that entered into force on 13
December 2011. This was followed by the "two-pack" initiative aimed
at stepping up financial discipline within the Eurozone, which the European
Parliament approved on 12 March 2013.

Whilst
the overall implementation of both the six-pack and two-pack proposals is
ensured by DG ECFIN, DG ESTAT has been given the responsibility under one
Commission Delegated Decision for one aspect of the effective enforcement of
budgetary surveillance in the euro area. The Commission is empowered by EU
Regulation 1173/2011 to detect and expose manipulation of general government
deficit and debt data relevant for the application of the multilateral
surveillance system and the EDP. Regulation 1173/2011 sets out a system of
sanctions for enhancing the enforcement of the preventative and corrective
parts of the Stability and Growth Pack. Accordingly, under Article 8 of this
Regulation and Commission Delegated Decision 678/2012 "on investigations and
fines related to the manipulation of statistics", DG ESTAT has a
responsibility for undertaking the work to establish the detailed rules
concerning the procedures for investigations, the amounts, timing and
collection of fines, and rules establishing the rights of defence, etc.

Commission
Delegated Decision 678/2012 was adopted on 29/06/2012 (operational from
26/11/2012). A specialised Directorate (DG ESTAT.D) was put in place from
January 2012 to deal exclusively with these new and reinforced GFS responsibilities.

Audit Objectives

The
overall objective of the current audit engagement was to judge whether DG ESTAT
has appropriately prepared itself to fulfil its new role in the economic
governance framework. This overall objective was further broken down as follows:

- 
The
first objective was to evaluate whether the controls and procedures in place
for the planning, management, and implementation of the amended Protocol on the
EDP operate satisfactorily to address the objective of the Commission, to
ensure that deficit and debt figures are of the highest possible quality, and
that the risks of incorrect or inaccurate data being notified by MSs are
mitigated.

- 
The
second objective was to review and assess the work undertaken by DG ESTAT with
key partners in respect of the responsibilities and new investigative powers
placed upon it by Delegated Decision in respect of the requirements of EU
Regulations in place as part of the "six-pack".

Audit Scope

The
audit scope was on the activities of DG ESTAT that directly address Economic
Governance focussing firstly on the activities undertaken by the DG in respect
of the processes applied under Regulation 479/2009 and secondly on the DG's
responsibilities, including new investigative powers, arising from the Six-Pack
and Two-Pack legislation that reinforce the Stability and Growth Pack under the
new rules for economic and fiscal governance.

There
are no observations/reservations in the AAR that relate to the area/process
audited.

The
fieldwork was finalised on 27/08/2013. All observations and recommendations
relate to the situation as of that date.

Overview of Major Strengths

The
auditors recognise the considerable efforts being made by DG ESTAT and its
staff in the period of financial crisis to deal with its revised responsibilities
in the area of GFS to support and ensure the monitoring and provision of high
quality fiscal data.

The
Directorate has ensured that its core tasks of the publication of biannual
press releases on MSs Notifications of Deficit and Debt, and its quarterly
publications on "debt" and on "seasonally adjusted
surplus/deficits" have been delivered on time. Further, the IAS notes that
the new Directorate D has significantly strengthened the overall number of
visits being undertaken in MSs and that these visits and other tasks of the
Directorate are underpinned by a robust and well-developed risk model that is
subject to internal review and on-going enhancement. The Directorate has
developed a package of audit programmes that comply with international standards
to underpin not only its upstream dialogue visits (UDVs) but to also support
the standard dialogue visits (SDVs) that are increasingly focussing on the
assessment and control of upstream data in MSs.

The IAS
further notes the commitment of the Directorate to not only improving the
quality and provision of information from organisations upstream of the NSIs
but also to focus on the improvements of the GFS Quality Management covering
the whole chain of the provision and control of fiscal data. DG ESTAT is also
playing a lead role in driving improvement in the quality and reliability of
public accounting systems and is promoting the use of European public sector
accounting standards (EPSAS) for all sectors of General Government in EU MSs.

In
realising these objectives, the DG is working hard to further improve its
coordination and cooperation with both NSIs and with the national State Audit
Institutions.

Major Audit Findings

The IAS
identified the following one critical and three very important issues:

Preparedness of the DG to fulfil its role
in the Economic Governance Framework – New Investigation Powers (Critical)

In the
period since the agreement of the Delegated Decision 678/2012, no comprehensive
documentation or procedures, except for one summary document prepared in
parallel to the audit, have been prepared or discussed by DG ESTAT to deal with
the DG's new investigation responsibilities. The advice received from the DG's
Legal Unit on the necessary preparatory work has not been fully followed, no detail
work (manuals, guidelines, procedures, templates) has been undertaken in
respect of the different investigative steps, and no DG guidance has been
prepared explaining the criteria to be evaluated in terms of when to employ the
investigation procedure, including elements on the interpretation of key
terminology.

Resource management (Very important)

The
impact on Directorate D to maintain its delivery of its core tasks in the case
of a future methodological visit or investigation under Delegated Decision
678/2012, has not been sufficiently evaluated. The DG does not in its
Management Plan 2013 make any allowance or contingency for the occurrence of
methodological visits or investigations under the new powers. No guidance on
the required competences, selection or training of staff that possibly might be
required to undertake this new work has been prepared.

In
addition, there are competing pressures on the available resources to ensure
both the continued delivery of the DG's core tasks, principally the EDP Notification
procedures April and October, and also to recognise the impact and importance
of other activities.

Risk Analysis, visit and EDP verification
planning (Very important)

There
has not been an independent peer review of the Risk Model to evaluate its
compliance with the requirements of a standard Quality Management System (QMS).
The DG has only begun in 2013, on a pilot basis, to integrate elements of
detection risk (along with inherent and control risk) into the model. The
participation of key European Statistical System (ESS) partners in the risk
evaluation will not be fully integrated into the process, and the results of
the risk analysis are not shared with DG ESTAT's partners in the ESS.

The DG
has been unable to sustain and increase the number of UDVs and the reports of
these visits are not systematically separately published or distributed on
completion, in the same manner as the SDV reports. Further, there is no clear
and formal manner by which the organisations consulted and controlled upstream
of the NSIs, can have access to, or comment on, a UDV report and any
recommendations therein.

Methodological and ad-hoc
visits (Very important)

Since
the adoption of the revised Regulation 479/2009, one methodological visit has
taken place. DG ESTAT judges that it would prefer to conduct either a series of
more targeted ad-hoc (or technical) visits or UDVs and this has contributed to
the relatively low number of methodological visits so far conducted.

Audit recommendations

The IAS
issued therefore the following critical and very important recommendations:

Preparedness to implement New
Investigation Powers – Critical

The IAS
recommends DG ESTAT to take immediate steps to formalise its operational and
technical procedures, as appropriate taking cognisance of the expertise and
advice elsewhere in the Commission and prepare manuals and guidelines to
support the conduct of all phases of these investigations.

Resource Management – Very
important

The IAS
recommends that DG ESTAT should alert the Commission on any necessary resource
gaps/contingency required in the context of a risk and impact of event(s)
scenario for 2013 and 2014 and conduct an evaluation of the necessary
competences required, and appropriately train a core team for the new
investigation tasks.

The IAS
also recommends DG ESTAT to undertake a resource analysis, with a view to
confirming that the allocated level of resourcing and expertise to each of the
Directorate’s core tasks is sufficient. The DG should also amend its Management
Plan to take account of planning, training, and other possible issue related
work arising from the new investigation responsibilities.

Risk analysis, visit and EDP verification
planning – Very important

The IAS
recommends DG ESTAT to ensure that the Risk Model is subject to an independent
peer review that meets the requirements of a recognised QMS process and to take
action to ensure that the essential information for the detection risk
component of the model is achieved as soon as possible. The DG should further
review the added value that a more direct involvement of DG ECFIN and the NSIs
might have on the quality of the model and its output and give additional
qualitative information on the results of its risk exercise to key EDP
partners.

The DG
should routinely, and separately, publish all UDV Reports and forward them to
the Economic and Financial Committee (EFC) and with NSIs, take steps to
encourage the consultation of upstream bodies visited in the finalisation of
reports and implementation of actions.

Methodological and ad-hoc
visits – Very important

The IAS
recommends DG ESTAT to prepare a clear justification for the conduct of each
planned ad-hoc visit where this visit type is being used, and to submit a
summary of the ad-hoc visits conducted in each year to thekey stakeholders.

DG ESTAT established an
action plan which the IAS considers adequate to address the identified issues.
In a recent follow-up audit, the IAS noted the progress made. While not being
able to close the 'critical' recommendation, it was downgraded to 'very
important'.

9.2. Performance audit on the efficiency and effectiveness of the design and implementation of the financial circuits in OIB

Background

The
mission of OIB is to "ensure the implementation of all actions
connected with the accommodation of personnel, the management of social
infrastructure and the logistics of the institution in Brussels as well as
social infrastructure on the Ispra site."

Its
responsibilities include, among others, "to manage the purchase, rental
and maintenance of the movable and immovable property of the Commission".
This implies the management of a consequent number of contracts for goods and
services and a consequent processing of the related financial transactions,
from the budgetary commitment to the final payment (and the recovery of unduly
paid amounts, if necessary).

These
amount to around 10.000 transactions per year[68]
(including more than 8.000 payment orders) and both financial and operational
units intervene in the process[69]. If
the financial circuits are not optimally designed and implemented, they can
result in an inefficient use of resources if either too many staff is involved
in or too much time is spent on processing a financial transaction.

Taking
into account the high number of transactions processed by OIB, and the leverage
effect any improvements in the design and/or implementation of the circuits can
have in terms of resources used to process transactions, the IAS decided to
include a performance audit on the design and implementation of the financial
circuits in its audit plan for 2013.

Audit objectives and scope

The
main objective of the audit was to assess the efficiency of the circuits in
place in OIB to process financial transactions. In this context, a financial
circuit is efficient when it reaches the best cost/benefit relationship between
the resources employed (staff) and the outputs produced (financial transactions
processed)[70]. Given
its output (authorisation of the financial transactions), a financial circuit
is efficient if it is capable of minimising the resources involved in the
management of the transactions and/or reducing the time to process them.

When
reviewing the financial circuits, the auditors also looked at the control
environment in place in OIB to support the processing of transactions[71] and at
some effectiveness- related aspects[72] to the
extent that they can have an impact on the overall efficiency of the financial
circuits.

The
audit did not cover the legality and regularity of the transactions. However,
when assessing the design of the financial circuits and testing their
implementation, the IAS paid attention to elements that could have flagged
possible irregularities.

The
audit covered all the operational departments of OIB (RE, CPE and OS)[73], the
central financial unit DR.2 of DR department (Resources), and looked at the
circuits in place to process Budgetary commitments, Legal commitments and
Payments (including the 'certified correct' (Conforme aux faits) CAF
procedure).

No
scope limitations were identified.

The
fieldwork was finalised on 07/11/2013. All observations and recommendations
relate to the situation as of that date.

There
are no observations/reservations in the AAR that relate to the area/process
audited.

Strengths/Good practices

The IAS
recognises the efforts made by OIB to improve the efficiency of its financial
circuits.

At
central level, the Service has set up a Working Group on simplification that
has identified actions to streamline the financial procedures and ensure a
better use of resources.

At department
level, OIB has revised their internal organisation by setting-up central
coordination Units to centralise and better coordinate the operational
initiation and verification of the transactions and improve their efficiency.

Overall,
these actions have contributed to a significant improvement in OIB performance
in processing financial transactions, in particular in terms of the time to
pay.

Major Findings

The
audit did not reveal any issues that would give rise to a critical or very
important recommendation.

10. 
IT audits

10.1. Performance Audit on Management of Local IT in DG ESTAT

Background

The
role of the Directorate General for Statistics (DG ESTAT), as set up in the
Regulation 223/2009[74], is to
provide accurate European statistical data, as part of the European Statistical
System (ESS).

DG
ESTAT business strategy for the period 2010-2020 (so-called VISION) is defined
in Commission Communication COM(2009)404. Its main goal is to improve the
efficiency of statistical production, enabling the ESS to cope with the
conflict between increasing information needs on the one hand and resource
constraints on the other. This vision implies moving from the existing
"augmented stovepipe model" to an integrated model for producing
European statistics (labelled “European systems method to statistics”).

DG
ESTAT relies entirely on IT systems and IT services to support the cycle of
statistical data collection, development, production and dissemination. IT is
also essential for adequately replying to the challenges imposed by the new
business strategy, in particular in terms of use of common methodological and
ICT tools and knowledge sharing within ESS.

In this
context, DG ESTAT launched in 2012 the ESS.VIP programme, aiming at realising
economies of scale and productivity gains through sharing information, services
and costs and at developing a common ESS infrastructure and appropriate legal
framework and new administrative mechanisms that will allow for sharing of
information, services and costs among ESS partners.

The IT
activities in DG ESTAT are mainly under the responsibility of Directorate B
(Corporate Statistical and IT services), whose mission includes, among other
objectives, to "provide
the statistical and IT infrastructure, the developments and the support
required to assure efficient and secure administrative and statistical
production processes – from data collection to dissemination"[75]. DG ESTAT hosts and
manages a dedicated infrastructure (network, firewall and servers) for the
processing of confidential data sets, while non-confidential data is hosted in
the Data Centre, under the responsibility of DG DIGIT.

For
2012, the IT budget was 17,835 k€, financed at more than 98 % by operational
expenditures, and staff resources of 109 FTE for Directorate B - Corporate
statistical and IT services – complemented by 42 FTE from contractors and a
certain number of IT capabilities in the operational Directorates.

Audit Objectives

The
overall objective of the audit was to assess the internal control system put in
place by DG ESTAT to ensure an effective and efficient management of its local
IT activities, with a particular focus on the following areas:

- 
IT
Governance,

- 
physical
and logical security arrangements,

- 
management
of IT programmes and projects,

- 
organisation
and management of the IT operations.

Audit Scope

The
scope of the audit included the following processes:

- 
Plan
& Organise[76] (IT
organisation and IT governance, quality management, risk assessment and project
management);

- 
Acquire
& Implement[77]
(Change and release management);

- 
Deliver
& Support[78] (Services
continuity, logical and physical security, configuration management).

The audit
focused on the activities performed by Directorate B.

There
are no observations or reservations in the AAR 2012 relating to the processes
audited.

The
fieldwork was finalised on 12/05/2013. All observations and recommendations
relate to the situation as of that date.

Strengths

Overall,
the IAS noted a well-established IT department, with very competent staff
strongly committed to deliver high quality services to the end users. DG ESTAT
has set up an articulated IT organisation with the objective to respond quickly
and effectively to business needs in any of the phases of the statistical
production. Integrity and availability of data are in line with business
requirement and the costs of the IT activities, in particular those charged by
the external providers, are closely monitored.

Major Audit Findings

IT projects management (finding n° 1)

DG
ESTAT is managing a large portfolio of IT projects (mainly related to the
collection, production and dissemination of statistical data), some of which are
essential for the achievement of its strategic objectives (so-called ESS.VIP
projects).

So far,
DG ESTAT did not establish a fully-fledged IT project management office, to
ensure the proper implementation of the IT project management methodology
across the DG and to support and supervise the IT project managers. In the
transition to the new project management methodology (PM2[79]), the
DG did not put in place an effective mechanism to ensure that it is applied in
a standardised way when initiating, planning and controlling the execution of
the IT projects.

IT security (findings n° 4 and 5)

According
to the Commission IT security framework[80], DGs
are required to assess the security needs of their information systems and to
implement the IT security policies that enable them to protect information
against threats to its availability, integrity and confidentiality. DG ESTAT
implemented a series of security measures to protect its information systems,
in particular the confidential data collections. It did not classify its
information systems according to the levels defined in Commission Decision
C(2006)3602 and did not prepare and implement formal security plans for its IT
system in production or IT services.

Furthermore
DG ESTAT did not formally assess the risks to which the statistical data is
exposed to in order to identify the additional security requirements, if any.
There are mitigating actions currently in place, which should be improved.

Audit recommendations

IT projects management framework – Very
important

DG
ESTAT should set up a Project Management Office (PMO) in charge of supporting
the implementation of the project management methodology, coaching projects
teams and promoting internal coordination and best practices. The PMO should
identify a common set of practices, artefacts and processes to be used by DG
ESTAT for each category of IT projects. It should also define and implement a
Quality Management System to ensure that the deliverables are aligned with
business requirements.

IT security – Very important

DG
ESTAT should assess the criticality of its information systems (in line with
Commission Decision (2006)3602), revise the IT security needs and requirements
and identify the measures that meet the identified additional requirements, if
needed, (through security plans).

DG ESTAT established an
action plan which the IAS considers adequate to address the identified issues.

10.2. Agrex 3 (DG AGRI)

In
line with the IAS strategic planning, an audit on the IT project “AGREX 3” was
launched in December 2012 in DG AGRI.

The IAS
completed the in-depth preliminary review, which scope was to gain a better
understanding of the processes, to identify the specific risks and controls in
place and to more precisely define the objectives and scope of the audit
engagement.

During
the preliminary survey, the auditors interviewed the key actors in the project,
from both business and IT sides, and performed an in-depth review of the
documents provided by DG AGRI.

The
analysis of the available information indicated that DG AGRI has defined and
implemented adequate controls to reduce the inherent risks to an acceptable
level.  In particular, the IAS highlighted the involvement of both the business
and IT management in the different phases of the project and the methodologies
used to manage the project (in particular the development and the testing
methodologies). This assessment confirms the results of self-assessment made by
DG AGRI in the IT Master Plan 2013 (according to which, overall, the IT risks
have a relatively low likelihood and impact).

Therefore,
the IAS did not perform tests of detail and closed the engagement after the
preliminary survey.

10.3. Performance Audit on the management of the security of EU ETS IT system (DG CLIMA, DG DIGIT and DG HR Security Directorate)

Background

The
European Emission Trading System (EU ETS) is an international system for
trading greenhouse gas (GHG) emission allowances, covering more than 11.000
installations[81] in 31
countries[82],
including aircraft operators. It is a key tool for reducing industrial GHG
emissions in a cost-effective manner and is a cornerstone of the European
Union's policy to combat climate change.

The
EU ETS is supported by an IT platform (EU ETS IT System) that has two main
components:

- 
The
European Union Central Registry (EUCR)[83], an
online database that contains all the accounts of stationary installations and
aircraft operators and manages the various operations related to the ETS
processes (e.g. allocation of allowances, account management, transaction
management). Currently, more than 25.000 end-users have access to the EUCR.

- 
The
European Union Transaction Log (EUTL) that automatically checks, records
and authorises all transactions that take place between accounts.

The
value of the allowances held in the EU ETS Registry accounts is determined by
the market, according to the principle of “supply and demand”[84].
Currently this value is estimated at approximately 110 billion €, but it varied
significantly in the course of 2013[85].

DG
CLIMA is the system owner and system supplier of EU ETS system while an
external contractor is in charge of developing and maintaining ETS software[86]. DG
DIGIT provides hosting and specialised security services[87] and DG
HR.DS supports the process with advice and recommendations in the security
domain (in particular for the preparation of the IT Security plan, the GAP
analysis and the risk assessment).

As
“guardian and regulator of the carbon market” and responsible for managing the
EU ETS IT system, the Commission is exposed to high risks that vulnerabilities
of the system may be unlawfully exploited, resulting in its dysfunction and
that security flaws[88] may
ultimately lead to the carbon market distortion and to reputational and
financial consequences for the Institution. This risk materialised in the past
when the national registries (now replaced by the EUCR) were subject to
cyber-attacks that succeeded in stealing emission allowances. The EUCR is
facing the same inherent risks as previously and for this reason requires an
adequate level of protection against internal and external threats.

Since
2010, DG CLIMA has notified these risks in its AARs[89]. In
particular, in the 2012 AAR the DG reported that the security measures to cope
with significant and systemic weaknesses detected were only “partially
implemented” and that “measures from the 2010-2011 action plan are not fully
adequate anymore to mitigate current risks and new threats”. This risk was also
flagged as cross-cutting and an action plan was jointly prepared by DG CLIMA
and DG DIGIT.

Audit Objective
and scope

The
overall objective of the audit was to assess whether the control system put in
place by DG CLIMA, together with DG DIGIT and HR.DS, ensured that adequate
security measures were identified and that they are effectively implemented for
the EU ETS system.

Focusing
on the security aspects, the engagement covered in particular:

- 
The
governance set-up for the ETS project and the effectiveness of the strategic
decision-making process.

- 
The
management of the project (including the management of internal and external
providers and the implementation of the security measures).

- 
The
cooperation among key actors in the decision-making process and in the
subsequent implementation phase.

The
audit covered DG CLIMA, DG DIGIT and the Security Directorate of DG HR
according to their respective roles and responsibilities for managing the
security aspects of the EU ETS system.

The
fieldwork was finalised in mid-November 2013. All observations and
recommendations relate to the situation as of that date.

Strengths

The IAS
noted that unit B1 (the Registry team) in DG CLIMA, in charge of the ETS
systems, is made up of very competent staff, strongly committed to deliver high
quality services to the end-users despite its small size. In addition, they
have managed to meet very tight deadlines.

Over
recent years, the Registry team has acquired a solid competence in the security
domain and established a strong partnership with the Security Directorate of DG
HR to ensure that the Commission security framework is duly and properly
applied in the ETS project.

Major Audit
Findings

In the
course of the audit, the IAS identified the following very important issues:

IT Security

Finding
No. 1 - Implementation of the ETS's security controls (Lead DG: DG CLIMA in
cooperation with DG DIGIT and DG HR.DS)

Finding
No. 3 – DG DIGIT cooperation with DG CLIMA (Lead DG: DG DIGIT)

The
security measures defined as from 2010 have not yet been fully implemented.

In
addition, cooperation and the information flow between DG DIGIT and DG CLIMA
were not optimal and major disagreements on the appropriateness or feasibility
of certain key security measures were not settled. This further delayed the
identification and implementation of adequate security measures for the ETS
system.

ETS Project Governance

Finding
No.2 - (Lead DGs: DG CLIMA in cooperation with DG DIGIT and DG HR.DS for the
part on IT project governance structure, DG CLIMA for the part on
Organisational structure)

The
governance arrangements failed to resolve the participants' conflicting
positions and to achieve a common understanding on the appropriate security
measures for ETS and their timely and effective implementation.  In addition,
no evidence exists that the decisions of the ETS Project Steering Committee have
been properly followed up.

The
current organisation of the Registry team has not evolved since the beginning
of the project, despite it growing from two to twelve persons, and it is
currently not ensuring an adequate segregation of duties between key business
and IT actors and an optimal use of competences available in DG CLIMA.

ETS Project management

Finding
No.4 – Project Quality management  and Test management (Lead DG: DG CLIMA)

DG
CLIMA implemented a quality management system. However, the Quality Manager is
not involved in assessing all contractor deliverables and in the quality review
of the project and methodology applied by both the Registry team and the
contractors. In addition, the DG has not yet approved the Project Quality Plan
of the development contractor (which consequently is not yet implemented) and
the quality contractor is not performing all the duties foreseen in the
contract.

The
complex process for testing ETS IT system releases is not fully adequate and
not documented. The Test and Use cases are not complete or up-to-date and key
actors (Quality contractor, business manager and development contractor) do not
perform some of the key quality-related tasks. Also the complex and mostly
manual deployment process is not documented.

Audit
recommendations

The IAS
issued therefore following recommendations in order to address the above
issues:

IT Security – Very
important

DG
CLIMA, in cooperation with DG DIGIT, should ensure that the remaining necessary
security measures for ETS are duly implemented and in a reasonable time frame.

ETS Project Governance – Very
important

Liaising
with DG DIGIT and the Security Directorate of DG HR, DG CLIMA should enhance
the governance set-up and decision making process of the ETS project to ensure
that the implementation of the necessary security measures is no longer
delayed.  DG CLIMA should also revise and optimise the organisation of the
Registry team to ensure a clear attribution of roles, an adequate segregation
of duties and the best use of the available IT expertise.

ETS Project management – Very
important

DG
CLIMA should strengthen those processes to ensure that IT project-related
activities are controlled, measured and improved and that the tests are
organised in a more effective and efficient way.

The DGs audited
established action plans which the IAS considers adequate to address the
identified issues.

10.4. SYGMA - Phase 1 - audit on development process (DGs RTD and CNECT) (Performance Audit)

Background

In the
context of the Horizon 2020 Framework Program (H2020[90]),
the Research family DGs/Services are moving towards harmonising the grant
management business processes and streamlining the supporting IT systems, which
had already started during the FP7[91].
The aim is to standardise the way similar activities are performed within the
Research family members and to reduce the number of local applications
supporting the grant management sub-processes. This should also contribute to
improving data quality and the services offered to the public.

SYGMA
is an IT system supporting the grant management process for FP7/CIP[92],
H2020 and other non-research grants operated by the Research family members. It
will complement the existing common IT systems covering the process from
preparing to closing grants and it will gradually replace all legacy
applications (FP6 and FP7).

SYGMA
is an inter-DG project currently managed within the Research family, but with
the long-term objective to become the corporate grant management system. For
this reason, it is at the core of the Grant Management rationalisation exercise[93].

DG
CNECT (as System Owner) and DG RTD (as System supplier) are the main actors
involved in the development of SYGMA. They both report to the IT Programme
Steering Committee (ITPSC[94])
and to the Architecture Office Steering Committee[95].
Representatives from all Research DGs and Services are involved in defining the
business requirements. As from 1/1/2014 the responsibility for SYGMA will move
to the Common Support Centre (CSC), a newly created Directorate in DG RTD in
charge of providing high quality services to all Research DGs, executive
agencies and joint undertakings implementing the Horizon 2020 programme.

The
SYGMA project is financed from both the FP7 and Horizon 2020 budgets. The
estimated Total Cost of Ownership[96]
for the period 2011-2015 is about 23,6 M€[97].

A first
pilot of SYGMA was delivered during 2013 in order to cover FP7 grant process
support functionalities for about 40 projects in unit S2 (Space research) of
REA. However, the major functionalities of SYGMA should be operational for all
Services managing the H2020 programme as from March 2014.

Audit Objectives

The
objective of the audit was to assess the effectiveness and efficiency of the
management of the SYGMA project.

The
audit focuses on the following three main areas:

- 
IT
Project governance

- 
IT
Project Management

- 
Readiness
of IT Operations for the SYGMA system.

Audit Scope

The
present audit covered the management of the SYGMA project, both from the
business and IT perspectives, since its inception in March 2012 (when the
Business case and Vision document were produced), until mid-November 2013, when
the current audit fieldwork was completed. All observations and recommendations
relate to the situation as of that date.

The
audit covered both the system owner and the system supplier, in particular:

- 
DG
CNECT

– 
Directorate
E Net Futures

– 
Directorate
R.3 Informatics

- 
DG
RTD

– 
Directorate
M.2 Coordination of RTD framework programme operations

– 
Directorate
R.3 Informatics

The
IAS also conducted interviews with other SYGMA stakeholders, namely:

- 
Members
of the Architecture Office (DG DIGIT.B4).

- 
Business
representatives in the REA (REA.S.2).

Strengths

The IAS
recognises the Research DGs' efforts to move to a business process-oriented
organisation, with the H2020 operational activities being organised around nine
standard business processes covering the complete cycle of grant operations.

To
support this new organisation, the Research DGs have defined a new governance
model, operational since October 2013, and decided to create the Common Support
Centre (CSC) as from 1 January 2014, whose mandate is to provide high quality
services to the entities implementing the Horizon 2020 programme. This should
enhance their cooperation and facilitate the simplification and harmonisation
of the existing grant management business processes and convergence towards
common IT solutions.

In
addition, the IAS observed a constructive collaboration between business
stakeholders and the IT team, as well as the latter's high level of expertise
and commitment.

Finally,
the joint approach to the project, based on good practices and experiences in
project management of both DG RTD and DG CNECT, represents a key success
factor.

Major Audit Findings

During
the audit, the IAS identified the following very important issues:

Business Processes (Observation no 1)

The
definition of common and agreed business processes for H2020 has not been
completed yet. This is only partially due to the late adoption of the key
legislative acts by the legislative authority, which has additionally delayed
the delivery of other legal documents. Moreover, some elements (relationship
between the new IT governing bodies and the CSC not yet defined, lack of proper
monitoring on the progress of the Working Groups, lack of a unique and
up-to-date Business glossary, unclear roles and responsibilities for the
preparation of H2020 business documentation) could further delay (or even
impair) the Business Process definition, if not adequately addressed.

Project Management (Observation no 2)

The DGs
do not estimate and monitor the full cost of the project (internal/external,
IT/business). Only the cost for external IT staff is identified and reported.
In addition, there is neither an integrated project work plan nor business implementation
and transition to production plans for SYGMA. In terms of risk identification
and management, the IAS observed that the risk log of the SYGMA project does
not include any risks/threats linked to the on-going reorganisation of the
activities and its possible impact on the projects.

Audit recommendations

The IAS
issued therefore the following recommendations:

Business Processes – Very
important

DG RTD
and DG CNECT, in liaison with the Research Family members, should ensure the
definition of common and widely agreed Business Processes.

To
reach this goal, the DGs should ensure an effective interaction among the IT
governing bodies, the new CSC and the Working Groups and define common and
coherent indicators and mechanisms for monitoring and reporting progress. In
addition, a unique and complete Grant management Business Glossary should be
established to provide a common basis for identifying and agreeing requirements
and translating them in the IT systems.

Project Management– Very
important

DG
RTD and DG CNECT should improve the project management by performing an
appropriate cost-benefit analysis in case of future decisions with significant
impacts on the project and its resources, by ensuring that detailed integrated
Project Work Plan, Business Implementation Plan and Transition to Production
Plan are developed, and by identifying and managing the risks related to major
changes in the environment. In addition, the DGs should accurately plan and
monitor the use of internal and external resources (from both business and IT)
and identify the full cost of the project.

Both
DGs established action plans for the recommendations which the IAS considers
adequate to address the identified issues.

11. Follow-up engagements (summarised)

11.1. 2nd Follow-up on DG DEVCO audit - Financial Management of main programmes in Latin America

Based on the
results of our follow-up audit, IAS assessed that all three recommendations had
been adequately and effectively implemented and could be closed.

11.2. 2nd Follow-up on DG DEVCO audit - Financial Management of main programmes in Neighbourhood

Based on the
results of our second follow-up audit, IAS assessed that the design of the
controls to mitigate the risks identified are adequate. The residual risks have
been re-assessed as part of the preparation of the IAS 2013-2015 Strategic
Audit Plan and the effective implementation of these recommendations will be
assessed in future IAS audits. As a result, the three recommendations are now
closed[98].

11.3. 2nd Follow-up on DG DEVCO audit - Implementation of the framework agreement with UN agencies

Based on the
results of IAS' second follow-up audit, IAS assessed that the design of the
controls to mitigate the risks identified are adequate. The internal control
environment in DG DEVCO has significantly evolved since the original audit,
namely with the creation of the European External Action Service (EEAS) in 2011
and the implementation of DG DEVCO's revised internal control architecture
(control pyramid). The residual risks have been re-assessed as part of the
preparation of the IAS 2013-2015 Strategic Audit Plan and the effective
implementation of these recommendations will be assessed in future IAS audits.
As a result, the three recommendations are now closed.

11.4. 2nd Follow up of AFIS (OLAF)

Based on the
results of our follow-up audit, IAS assessed that all the recommendations
addressed to OLAF that resulted from the IAS Audit on the Anti-Fraud
Information System (AFIS) in OLAF, dated 21 January 2009, had been adequately
and effectively implemented.

11.5. 2nd Follow up on the financial management of the Common Foreign Security Policy budget (FPI)

Based on the
results of our follow-up audit, IAS assessed that the very important
recommendations “Overall monitoring and supervision of ESDP missions” and “Timely
closing procedure” could be closed.

The very important
recommendation “Set-up of CFSP actions” could also be closed, however, the IAS
will assess the effective application of the comprehensive guidance on the
financial management set-up during the next audit on the Control Strategy of
FPI and brought the inventory of IT applications and analysis of staff
potentially available from previous missions to the attention of DG IAS.A
for consideration in their future engagements of the EEAS.

11.6. Follow-up on Management of Guarantees (multi-DG)

(Four follow-up
reports: DG BUDG, DG ELARG, DG DIGIT, EACI)

Based on the
results of IAS' follow-up audit, IAS assessed that all the recommendations
which were addressed to the three DGs and one Executive Agency as a result of
the audit "Management of Guarantees" had been adequately and
effectively implemented.

11.7. Payment deadlines (BUDG, ENER/MOVE, DEVCO, ECHO) (including IT part)

(Four follow-up
reports: DG BUDG, DGs ENER and MOVE (ex-DG TREN), DG DEVCO, DG ECHO)

DGs
ENER, MOVE and BUDG specific findings:

Based on the
results of our follow-up audit, the IAS assessed that they have all been
adequately and effectively implemented.

DG
DEVCO specific findings:

Based on the
results of our follow-up audit, the IAS assessed that six recommendations had
been adequately and effectively implemented (and were closed). The five
remaining recommendations require further improvements to ensure that the
payments are correctly and timely processed.

In particular:

Recommendation
1 – Monitoring and reporting on the payment process (Very important)

The recommendation
requested DG DEVCO to improve its monitoring activities over the entire payment
process, both at central and directorates levels.

Monitoring
activities on the payment process are performed at central and Directorate
levels, and CRIS reports are made available to the DEVCO's Directorates for
further analysis.

However, IAS
observed that despite monitoring activities in place, DG DEVCO remained among
the 5 DGs with the highest number of late payments.

As a result, the
IAS recommended DG DEVCO to ensure that monitoring activities are regularly
performed at central and directorate levels, and that dedicated action plans
addressing the issues identified as a result of these monitoring activities are
systematically implemented.

Recommendation
5 – Time to encode invoices (Very important)

The recommendation
requested DG DEVCO to raise awareness of staff and to update the accounting
manuals regarding the timely encoding of invoices.

DG DEVCO
implemented several actions in order to improve the timely registration of
invoices (update of manuals, specific notes, etc.).

Consequently,, DG
DEVCO should ensure that the current and on-going analysis and reporting on
payment delays allow the DG to identify the necessary information (causes, main
directorates concerned by the non-timely registration, etc.) and/or take
additional actions in order to improve the situation with respect to the timely
registration of invoices.

Recommendation
6 – Management of suspensions (Very important)

The recommendation
requested DG DEVCO to clarify guidelines and manuals regarding suspensions,
adapt CRIS to ensure proper transfer of data in ABAC, implement regular
monitoring and perform spot-checks on suspensions.

The IAS found that
only part of the recommendation has been implemented.

Recommendation
7 - Payment checklists for operational and financial initiating and verifying
agents (Very important)

The recommendation
requested DG DEVCO to enhance quality of checks included in the checklists and
aimed at ensuring the effectiveness of the payment process.

The checklists
were updated in 2011 with some checks relating to the proper encoding of data
impacting payment delays. The follow-up audit highlighted that the checklists
should be further detailed as to focus on key dates relating to the calculation
of payment deadlines, in particular in those areas where systemic problems are
identified.

DG ECHO
specific findings:

The overall result
of this follow up audit is that DG ECHO implemented adequate solutions to
address the issues detected in the original audit and in particular adequately
and effectively implemented all the very important recommendations. However, in
some cases, they were not fully effective and hence did not allow for major
improvement of the timeliness of the payment process. Consequently, the IAS
identified areas were additional effort is required by DG ECHO to ensure that
the payments are correctly and timely processed.

11.8. Follow-up of Joint Audit on Implementation by the EIF of the High Growth and Innovative SME Facility (GIF) under the Entrepreneurship and Innovation Programme (EIP) in DG ECFIN

Based on the
results of our follow-up audit, IAS assessed that all the recommendations
addressed to DG ECFIN that resulted from the audit and had been sent for audit
review are adequately and effectively implemented. The two open
recommendations, of which one is classified as very important, were not yet due
for implementation and, therefore, were not covered by this follow-up
engagement. They will be covered by a second follow-up.

11.9. Follow-up on the Spot Controls/Fraud in DG CNECT

Based on the
results of our follow-up audit, IAS assessed that all the recommendations
addressed to DG INFSO/CNECT had been adequately and effectively implemented.

11.10. 2nd Follow-up on DG DEVCO audit - Financial Management of Regional Programmes

Based on the
results of our second follow-up audit, IAS assessed that three recommendations
had been adequately and effectively implemented or were no longer relevant as
the internal control environment in DG DEVCO had significantly evolved since
the original audit.

11.11. 2nd Follow-up on DG DEVCO audit - Eligibility of Costs under the Financial and Administrative Framework Agreement with the United Nations by DG AIDCO

Based on the
results of our second follow-up audit, IAS assessed that the design of the
controls to mitigate the risks identified for two very important
recommendations[99] was
adequate. Given that the internal control environment in DG DEVCO has
significantly evolved since the original audit, their effective implementation
will be assessed in future IAS audits. As a result, the two recommendations
were closed. The third outstanding recommendation, i.e. "Rec no.3 –
Nature of verification work – Verification missions" (Very important)
was assessed by the IAS as implemented.

11.12. Follow-up on Risk Management Implementation (SG, BUDG, RELEX, ENV)

(Three follow-up
reports: SG and DG BUDG, DG ENV, FPI)

SG and
DG BUDG specific findings:

In 2010, the IAS
finalised the audit on Risk Management process in the Commission. The overall
objective of the audit, conducted in the Secretariat-General and in DG BUDG (in
their central role) and in two operational DGs (DG ENV and DG RELEX, now FPI),
was to assess the adequacy of the Risk Management framework and its effective
implementation in the Commission. The scope of the audit covered the design of
the Commission's Risk Management framework and its implementation, and the
roles of the central services in providing guidance, training and support to
the operational DGs/Services.

Based on the
results of our follow-up audit, IAS assessed that thirteen out of fourteen
recommendations had been adequately and effectively implemented and were
closed.

Concerning the
remaining one on the “Management of cross cutting risk” (recommendation n. 13),
the IAS noted the progress made in particular regarding the mandatory
participation in the cross-cutting exercise, the clarification of the
definition of cross-cutting risks, the establishment of a dedicated peer-review
process for the cross cutting risks and the systematic feedback provided to
DGs/Services, and the implementation of a cross-cutting risk register.

However, as
regards the monitoring and following-up of the implementation of action plans
for mitigating cross-cutting risks, the IAS considered that further
improvements were still needed.

DG ENV
specific findings:

Based on the
results of our follow-up audit, IAS assessed that all recommendations had been
adequately and effectively implemented and were closed.

FPI
specific findings:

Based on the
results of our follow-up audit, IAS assessed that all very important
recommendations had been adequately and effectively implemented and were
closed.

11.13. Follow-up on Business Continuity Management (DG DIGIT)

Based on the
results of the follow-up audit, IAS assessed that all recommendation reported
as implemented by the DG had been adequately and effectively implemented.

11.14. 2nd Follow-up on Set up of Internal Control and Financial Management Systems in REA

Based on the
results of the follow-up audit, IAS assessed that overall, all the
recommendations addressed to the REA had been adequately and effectively
implemented.

11.15. Follow up of Control Strategy audit (TEN T-EA)

Based on the
results of the follow-up audit, IAS assessed that overall all the
recommendations addressed to the TEN-T EA had been adequately and effectively
implemented.

11.16. 2nd Follow up of Operational Budget management audit in the TEN-T EA (recommendation addressed to DG MOVE)

Based on the
results of the follow-up audit, IAS assessed that overall the recommendation
addressed to DG MOVE had been adequately and effectively implemented.

11.17. 2nd FU on Recoveries (multi-DG)

(Four follow-up
reports: DG BUDG, LS, DG DEVCO, DG EAC)

DG BUDG
specific findings:

Based on the
results of the 2nd follow-up audit, the IAS closed the two remaining
recommendations.

LS
specific findings:

IAS assessed that
the remaining four recommendations had been adequately and effectively
implemented and could be closed.

DG
DEVCO specific findings:

IAS assessed that
the one remaining recommendation had been adequately and effectively
implemented and could be closed.

DG EAC
specific findings:

Based
on the results of the second follow-up audit, IAS assessed that the four
remaining recommendations had been adequately and effectively implemented and
could be closed.

11.18. Follow up on the Monitoring and Supervision of DG EAC National Agencies

Based on the
results of the follow-up audit, IAS assessed that all the recommendations
addressed to DG EAC that resulted from the audit "Monitoring and
Supervision of DG EAC National Agencies" had been adequately and
effectively implemented.

11.19. 2nd Follow up of Enterprise Europe Network Audit

Based on the
results of the follow-up audit, IAS assessed that all the recommendations
addressed to the EACI that resulted from the IAS Audit on Enterprise Europe
Network IT Tools in EACI had been adequately and effectively implemented.

11.20. 2nd Follow up: Management of the operational budget of EAHC

Based on the
results of the follow-up audit, IAS assessed that all the recommendations
addressed to EAHC that resulted from the audit EAHC Management of the
operational budget had been adequately and effectively implemented.

11.21. 2nd Follow up on Ex-post control activities and implementation of financial circuits in EACEA

Based on the
results of our second follow-up audit, we assessed that two remaining
recommendations had been adequately and effectively implemented.

The third
remaining open recommendation was closed and its implementation will be
assessed during the IAS follow-up of the audit on "Lifelong Learning
Programme" in EACEA/DG EAC, where a similar recommendation was made.

11.22. 2nd Follow up: Procurement in EAHC

Based on the
results of the follow-up audit, IAS assessed that all the recommendations
addressed to the EAHC that resulted from the audit "Public Procurement in
the EAHC" had been adequately and effectively implemented.

11.23. Management of local IT - the 2nd follow up (SJ)

As a result of the
second follow-up, IAS assessed that all the recommendations issued in the
original audit were adequately implemented.

11.24. 2nd follow-up on Treasury and Accounting System (DG ECFIN)

Based on the
results of the work, IAS assessed that the one recommendation remaining open at
the time of the first follow-up had been adequately and effectively implemented
and could be closed.

11.25. 2nd follow up audit on Grant Management of Schengen Facility (DG HOME)

Based on the
results of the follow-up audit, IAS assessed that all three recommendations
remaining open after the first follow-up in 2012 had been adequately and
effectively implemented and could be closed.

11.26. Follow up audit on EFF Control Strategy (DG MARE)

Based on the
results of the follow-up audit, IAS assessed that all recommendations
considered "ready for review" by the DG had been adequately and
effectively implemented and were closed.

11.27. 2nd follow-up on the management of procurement contracts in OIL

Based on the
results of the follow-up audit, IAS assessed that all the recommendations
addressed to OIL that resulted from the audit "The management of
procurement contracts" had been adequately and effectively implemented.

11.28. Management of local IT - the 2nd follow up (PMO)

Based on the
results of the follow-up audit, IAS assessed that all remaining very important
recommendations had been adequately and effectively implemented and could be
closed.

11.29. Follow up on Development of IT systems to support Horizon 2020 programmes under DG RTD, Part I (URF/PDM & SEP)

Based on the results
of our follow-up audit, we assessed that all the recommendations addressed to
DG RTD that resulted from the above-mentioned audit had been adequately and
effectively implemented, except for recommendation No 3 on Project Work and
Resource Plan.

This
recommendation required to improve the project and resource planning by
identifying activities and interdependencies that are necessary to reach the
project objectives and by ensuring that both IT and business related resources
involved in the system development are included in the planning.

DG RTD made
progress in the project planning, but the current Resource Plan does not
include information on the resources involved on the business side. In
addition, the DG only estimates the cost of the external resources. For this
reason the IAS considered the recommendation not yet to be fully implemented
but the residual risk sufficiently mitigated to justify downgrading the rating
of the recommendation to 'Important'.

11.30. Follow up of audit on Fraud Prevention and Detection (DG HOME)

Based on the
results of the follow-up audit, IAS assessed that all eight recommendations
issued in the final report had been adequately and effectively implemented.

11.31. Follow up of audit on Fraud Prevention and Detection (DG JUST)

Based on the
results of the follow-up audit, IAS assessed that all six recommendations
issued in the final report had been adequately and effectively implemented.

11.32. 2nd follow-up of Security of IT environment in subcontracted projects (DG REGIO)

Based on the results
of the work, IAS assessed that the three recommendations proposed for closure
at the time of this second follow-up had been adequately and effectively
implemented and could be closed.

11.33. Follow up Programme Estimates (DG DEVCO)

Based on the
results of the follow-up audit, IAS assessed that all very important
recommendations had been adequately and effectively implemented and could be
closed.

11.34. Follow up EDF grants (DG DEVCO)

Based on the
results of the follow-up audit, IAS assessed that all very important
recommendations had been adequately and effectively implemented and could be
closed.

11.35. Follow up of Competitive Activities audit (JRC)

Based on the
results of the follow-up audit, IAS assessed that overall all the
recommendations addressed to DG JRC had been adequately and effectively
implemented.

11.36. 2nd Follow up on HR management, part II (DG HR)

At the time of the
first follow-up, the IAS found that all recommendations were adequately and
effectively implemented with the exception of the very important recommendation
n. 9 on "Management of absenteeism: day-to-day management" for which
not all the planned actions had been completed. The issues that were not fully
implemented related to initiatives and training to raise Heads' of Unit awareness
in the field of absence management, to the definition of a formal strategy for
absence management, and to improving the quality of the checks made in relation
to illness related absences and recurrent short term absences.

IAS' analysis
during the second follow-up highlighted that noticeable improvements had been
made..

Some actions still
needed to be implemented: The full training path will only be available in 2014
and the awareness raising initiatives envisaged was going to start in January
2014 only. In addition, the corporate strategy for absence management, in draft
version at the time of the second follow-up, had not been formally adopted yet.

Considering the
improvements already made and the state of play of the on-going actions, the
original risk has been partially mitigated, and IAS proposed to downgrade the
rating of the recommendation from very important to important.

11.37. Follow up of EIP Performance audit (DG ENTR)

Based on the
results of the follow-up audit, IAS assessed that all the recommendations
addressed to DG ENTR had been adequately and effectively implemented.

11.38. 2nd Follow up of GNSS Audits (DG ENTR)

Based on the
results of the follow-up audit, IAS assessed that all the recommendations
addressed to DG ENTR had been adequately and effectively implemented or were no
longer applicable and therefore closed

11.39. HR IT Corporate Application - NAP - the Follow-up (PMO)

Based on the
results of the follow-up audit, IAS assessed that all very important
recommendations reported by PMO as implemented had been adequately and
effectively implemented.

11.40. Follow up of Control Strategy audit (DG ENER)

Based on the
results of the follow-up audit, IAS assessed that overall all very important
recommendations addressed to DG ENER had been adequately and effectively
implemented, except for the recommendation listed below.

Recommendation
n° 3220 - Completeness and consistency of audit working papers (Very
important), points 3) and 4):

- 
Concerning
recommendation n° 3220, point 3), it was
found that the audit sector does not use during its audits audit
programme/checklist with the audit procedures to be performed. Therefore, the
audit sector will develop and use a standardised audit programme/checklist,
tailored to its needs, listing the audit procedures and tasks to be carried out.
The completion of the checks will be documented by signing off the appropriate
step in the audit programme/checklist.

- 
Concerning
recommendation n° 3220, point 4), it was
found that the audit sector during its audits does not address fraud prevention
and detection. Therefore, in line with DG MOVE & DG ENER Anti-Fraud
Strategy, adopted in November 2013, DG ENER will in particular develop ex-post
auditors' anti-fraud awareness and capacity (e.g. through training). In this
context, the IAS encourages DG ENER to take into account in the audit
programme/checklist the specific fraud risks factors and indicators ("red
flags") relevant for the programmes managed by the DG.

Considering that
the other points of this recommendation were implemented, IAS downgraded the
recommendation from 'Very important' to 'Important'.

11.41. Follow up: Procurement (OIB)

Based on the
result of the follow-up audit, IAS assessed that all recommendations considered
as 'ready for review' by OIB had been adequately and effectively implemented.

11.42. Follow up of Local IT in DG EMPL

Based on the
results of the follow-up audit, the IAS assessed that all very important
recommendations had been adequately implemented, except for two of them where
further improvements were needed. In particular:

Recommendation
no 1 on IT governance (Very important)

The
IT governance model was updated and the roles and responsibilities of IT
governance bodies, including the IT Steering Committee (ITSC), are now defined
in the Terms of Reference for the IT Governance at DG EMPL. However,
this document does not define any responsibility for horizontal IT security
issues. In addition, the current practice is not in line with what is formally
established (i.e. ITSC representative group is foreseen in the Terms of
Reference, however it is not conveyed and according to DG EMPL it is replaced
by the project steering committees which meet on a monthly basis with similar
functions).

Considering
the improvements already made, the IAS downgraded the rating of the
recommendation from Very important to Important.

Recommendation
no 2 on IT risk management (Very important)

The
follow-up showed that no specific IT risk management framework had been
implemented in DG EMPL. Its risk register mainly included risks related to IT
projects (which represent an important part of DG EMPL's IT activities), while
there was no evidence of identification and assessment of other IT risks (e.g.
related to IT security, infrastructure, operations or continuity). For this
reason, the IAS considered the recommendation open and downgraded the
criticality from Very important to Important.

11.43. Follow-up on Thematic Budget Lines (DG DEVCO)

Based on the
results of the follow-up audit, the IAS assessed that all six recommendations
had been adequately and effectively implemented.

11.44. Follow-up on Fraud prevention and detection in OLAF

Based on the
results of the follow-up audit, IAS assessed that all the recommendations
addressed to DG OLAF that resulted from the Fraud Prevention and Detection
audit and that had been declared as 'ready for review', had been adequately and
effectively implemented.

11.45. 2nd Follow-up on DG DEVCO audit - Financial Management of main programmes in Asia

Based on the
results of the second follow-up audit, the IAS assessed that adequate measures
had been taken to ensure that future Verification Missions were based on risk
criteria. The effective implementation of the recommendation will be assessed
in future IAS audits. As a result, the recommendation was closed following this
second follow-up.

11.46. 2nd Follow-up on DG DEVCO audit - Ex-Post Control Activities

Based on the
results of the second follow-up audit, the IAS assessed that the internal
control environment in DG DEVCO had significantly evolved since the original
audit (2006) and the subsequent follow-up (2010) with the creation of the
European External Action Service (EEAS) in 2011, the implementation of DG
DEVCO's revised internal control architecture (control pyramid) and the recent
suspension by DG DEVCO's management of ex-post controls, as stated in DG
DEVCO's 2012 AAR.

As a result, this
recommendation was closed and its effective implementation will be assessed in
future IAS audits.

11.47. Follow-up of IPA Grants (DG ELARG)

Based on the
results of the follow-up audit, the IAS assessed that all the recommendations
addressed to DG ELARG that resulted from the audit “Management of IPA Grants”
had been adequately and effectively implemented and could be closed.

11.48. Follow-up on Monitoring of Public Procurement in the Member States (DG MARKT)

Based on the results of the follow-up
audit, the IAS assessed that all the recommendations addressed to DG MARKT that
resulted from the audit "Monitoring by DG MARKT of the Application of
Public Procurement Rules in the Member States" had been adequately and effectively
implemented and could be closed.

[1] See also IAS's Annual Activity Report 2013.

[2] The Annex provides an overview of all
completed audit and follow-up audit engagements.

[3] The "Audit on Control Strategy - Implementation in DG
AGRI" is counted as two engagements conducted by two different audit teams
resulting in a single audit report.

[4]          The variation of number of engagements and
reports is not a suitable indicator on the workload as the individual
engagements may vary in length.

[5] A comprehensive overview on the IAS recommendations is provided in
the report addressed to the Audit Progress Committee dated 4 March 2014
(Ares(2014)573822).

[6] See section 3.4.4. for more details.

[7] As per the auditee.

[8] As per the auditee.

[9] Cut-off date is 31 January 2014.

[10] IAS - 2006 - DIGIT - 001 Data Center - Operations and Security,
IAS - 2007 - DIGIT  - 001 Corporate Data network infrastructures & services
and IAS.B - 2008 - ADMIN  - 004 Audit on Security.

[11]  http://europa.eu/agencies/documents/joint_statement_and_common_approach_2012_en.pdf.

[12]  http://europa.eu/agencies/documents/2012-12-18_roadmap_on_the_follow_up_to_the_common_approach_on_eu_decentralised_agencies_en.pdf.

[13]  DG SANCO, MOVE, ENER, ENV, HOME and JUST.

[14]  DG BUDG, HR, DIGIT and Secretariat-General.

[15]  ACER, CEPOL, EASA, EASO, ECDC, EEA, EFSA, EIGE, EMCDDA, EMEA,
EMSA, ERA, EUROJUST, EUROPOL, FRA, and FRONTEX.

[16]      The EFSF was established with a three-year tenure that expires
in June 2013.

[17]      Council Regulation (EU) No
407/2010 – establishing a European financial stabilisation mechanism (EFSM).

[18]      COM (2010) 713 final
"Communication from the Commission to the Council and the Economic and
Financial Committee on the European Stabilisation Mechanism".

[19]      Greece, Portugal, Ireland, Spain, Cyprus

[20]
    Latvia and Hungary. In Romania only BoP pre-cautionary assistance programme is implemented for the period 2011 –
2013.

[21]      The importance of these instruments is expected to increase
significantly in the future.

[22]      Staff growth of 35% (126 posts) since 2008 meant mainly an
increase of posts for the Administrators  specialised profiles

[23]      There are currently 11
Directorates in DG EFCFIN, out of which seven are currently involved in
financial crisis-related activities.

[24]      The 2013 redeployment exercise entailed 47 posts (9% of the
total establishment posts of the DG).

[25]      Local overheads decreased by 14% end of 2009 to 9.4% in
January 2013

[26]      Currently, the strategic documents
prepared by DG ECFIN do not include any specific analysis aiming at identifying
possible approaches to address the 5% staff reduction over the next 5 years.

[27]      KPI: Key Performance Indicator

[28]      Since 2008 the DG staff has increased
by 8% (66 posts).

[29]         Since 2008 staff has
increased by nearly 7% (53 posts).

[30]      The 2013 redeployment exercise entailed 47 posts (9% of the
total establishment posts of the DG).

[31]        DG MARKT has reduced the
amount of staff allocated to the administrative tasks. As of 1/2/2013 the
overheads represents the 8,7% of the posts.

[32]      Commission Staff Working Paper "Analysis of errors in
cohesion policy for the years 2006-2009_actions taken by the Commission and the
way forward" SEC(2011) 1179 final of 5.10.2011

[33]      ECA Annual Report 14/11/2013 – Chapter 6 ESF - § 6-23

[34] RACER: Relevant,
Accepted, Credible, Easy, Robust (Internal Control Standard N° 5)

[35] RACER: Relevant,
Accepted, Credible, Easy, Robust (Internal Control Standard N°5)

[36]  Source: draft 2012 AAR

[37] Especially Regulation (EC) no. 663/2009 and Regulation (EU) no.
1233/2010.

[38] See COM(2012)445 final of 08/08/2012.

[39] "Cost Benefit Analysis for the delegation of certain tasks
regarding the implementation of Union programmes 2014-2020 to the Executive
Agencies" – ICF GHK
Consulting Ltd - 8 February 2013 (page 72)

[40] The final report is
dated June 2013.

[41] Liquidated damages =
unjustified financial contribution x (overstated expenditure/total claimed).

[42]      ECA Annual Report on the Activities Funded By The Seventh,
Eighth, Ninth And Tenth European Development Funds (EDFs) for 2008 financial
year, OJ 2009/C 269/02 10.11.2009, page 274, § 29

[43]      Functional mailbox : IAS-AAR-REPORTS@ec.europa.eu

[44]      ECA 2008 Annual Report on EDF, 10.11.2009

[45]      Note DG BUDG to the Resource Director of DEVCO.R,  21/10/2011

[46]      The final sample contained 226 items (instead of 240) as some
transactions' values were greater than the sampling interval and therefore
counted more than once.

[47] Legal bases for DG ECHO actions are article 214 of the TFEU and
Council Regulation 1297/96 on Humanitarian aid.

[48] Ratio
of budget lines 23 02 01/23 02 applicable both to commitments and payments for
the years 2011, 2012, budget 2013.

[49] Under the Financial Regulation and Implementing Rules applicable
until 31 December 2012.

[50]
http://myintracomm.ec.europa.eu/echo/units/echo-c/Documents/echo-c4/List%20of%20ECHO%20offices.pdf

[51] Verification
missions are carried out only for projects implemented through UN agencies.

[52] Littlejohn since 2002 and KPMG since 2012

[53] IAS
audit on compliance with payment deadlines (2010), contained two very
important findings on the deadline for the
acceptance of the final reports and the management of suspensions; IAC Audit on
the legality and regularity on a sample of payments (2012) found weaknesses in
terms of late payment and lack of timeliness in the so called "stop the clock
procedure"; IAC Audit on the speediness of financial circuits (2012)
recommended to further improve the checklists in order solve inconsistencies
and establish a clear separation between units.

[54] IAS audit on Financial Management of
Food Assistance ("FA") in DG ECHO (2009, follow up performed in
2012): The audit concluded that the internal control system in place provided
reasonable assurance regarding the achievement of the business objectives on
the compliance with rules and the effectiveness and efficiency of the financial
management of FA projects.

[55]      As from 1 January 2014, following the entry into force of the
new Financial Regulation (ref. no. 966/2012 and Commission Delegated Regulation
C(2012)7507 on the rules of application) the decentralised management mode will
be classified as indirect management (art. 58 and 60 of the new Financial
Regulation). A similar classification is intended to be used for the new
Financial Regulation of the 11th EDF.

[56]      Procurement under decentralised management mode: €765.3 m,
Programme Estimates: €358.5 m, compared to the total value of €8258.8 m of new
legal commitments signed by DG DEVCO in 2012 (source: page 196, Annex 5 of
the 2012 AAR, DG DEVCO)

[57]      This figure relates to one single
transaction in Afghanistan for which the documentation was missing.

[58]      According to DG DEVCO (2012 AAR, section 3.1.1.2 Legality & regularity of transactions, §7. Retrospective checks
and recoveries), "errors resulting from contracts implemented directly
by the Commission such as service, supply and works contracts were not
particularly prevalent in the population by value, and the number of errors
detected were similarly small in number. However, the nature of the errors
detected in these contract types (e.g. access to procurement documentation)
means that a greater proportion of the resultant expenditure is affected by
error – as a consequence, the contribution made by these contract types to the
overall measured error rate is disproportionately high."

[59]      Except for one case in Mauritania, contract no. (CRIS)
268-445 (Marché de formation des professionnels de la justice)

[60] RACER: Relevant, Accepted, Credible, Easy and Robust.

[61]      In 2009 the IAS already performed an audit on Risk Management
in the Commission covering DG BUDG and the SG and a number of operational
Services. As result of this engagement, the central services revised the
Implementation Guide and produced the current version of October 2012 used as a
reference in the current engagement.

[62]      In Risk register, Management Plan and AAR.

[63] C(2013) 4311 of 12.07.2013

[64] Interim evaluation report on the GMES Initial Operations
(2011-2013) of January 2013

[65] Regulation 911/2010 on the Earth Monitoring
Programme (GMES) and its initial operations (2011 to 2013).

[66]      DG ESTAT's Annual Management Plan for 2013.

[67]      Regulation 679/2010.

[68]      In 2012 OIB processed 540 commitments/de-commitments, 8 385
payment orders and 2 014 recovery orders. Around the same number of
transactions were processed in 2011.

[69]      Only for Legal Commitments and for ISPRA the Service has
implemented a fully decentralised model with all the operations performed in
the operational units, including the authorisation of the transactions.

[70]      Efficiency can be measured in terms of quantity (number
of financial transactions processed), quality (legal and regular
financial transactions, complying with the Financial Regulation and the signed
legal agreement with the counterparties) and timing (financial
transactions processed timely)

[71]      The control environment comprises training of staff, provision
of guidelines/instructions/working tools and the implementation of a monitoring
system.

[72]      The audit looked at the implementation of the circuits by the
financial actors, the flow of information/documentation throughout the circuits
and the 'certified correct' (Conforme aux fait) procedure.

[73]      RE: Real Estate, CPE: Centre Petite Enfance, OS: Operations
and Services

[74]      Regulation (EC) n° 223/2009 of the European parliament and of
the Council of 11 March 2009

[75]      Extract of Dir. B mission statement (source: ESTAT Intranet)

[76]      Plan & Organise covers
strategy and tactics, and concerns the identification of the way IT can best
contribute to the achievements of the business objectives.

[77]      Acquire & Implement covers
identification, development, acquisition, implementation and integration of IT
solutions, including changes and maintenance of existing systems, to ensure
that solutions continue to meet business objectives.

[78]      Deliver & Support is
concerned with the actual delivery of required services, which include
management of security and continuity, service support to users, and management
of data and operational facilities.

[79]      DG DIGIT developed PM2 and promotes it as a possible standard
in IT project methodology at the Commission.

[80]      The Commission IT security framework is formed by Commission
Decision C(2006)3602 of 16.08.2006 (concerning the security of information
systems used by the European Commission), its implementing rules and a series
of guidelines and standards issued by DG HR.DS.

[81]      In particular, EU ETS covers CO2 emissions from installations such as power
stations, combustion plants, oil refineries and iron and steel works, as well
as factories making cement, glass, lime, bricks, ceramics, pulp, paper and
board.

[82]      28 EU Member States plus
Iceland, Liechtenstein and Norway. When
transactions involve other countries covered by the Kyoto protocol, they are
processed through the Independent Transaction Log (ITL) system of the United
Nations that holds the emission rights of these countries.

[83]      The new single
registry is established under Decision No 280/2004/EC of the European
Parliament and of the Council of 11 February 2004 concerning a mechanism for
monitoring Community greenhouse gas emissions and for implementing the Kyoto
Protocol. The EUCR has replaced, as from June 2012, all
the national registries previously managed at the level of individual MS.

[84]      According to this principle, too many allowances available
will result in a low carbon dioxide price, while too few allowances will lead
to an increase in the carbon dioxide price.

[85]      End of April 213 the carbon price decreased significantly due
to the refusal of the European Parliament to delay the auctioning of 900
million of pollution certificates. The current price is back to the levels of
the first quarter of 2013.

[86]      A second external contractor is in charge of the quality assessment of the deliverables prepared
by the contractor in charge of the development.

[87]      Hosting services are provided by Directorate C of DG DIGIT and
the security services are delivered by the Security Operation Centre (SOC) in DIGIT.01.

[88]      Unauthorised
disclosure, misuse, alteration or destruction of information, including carbon
allowances thefts and illegal transactions.

[89]      In 2010 and 2011 DG CLIMA issued a reservation due to significant security weaknesses in the national
registries of ETS while in 2012 the reservation covered the EU single Registry
ETS.

[90]    
 Horizon 2020 is the financial instrument implementing the Innovation Union, a
Europe 2020 flagship initiative aimed at securing Europe's global
competitiveness. It will cover the period 2014-2020 with a budget of around €70
billion.

[91]    
 In the context of FP7, the Research DGs developed common applications like the
Submission and Evaluation System (SEP) and the Unique Registration Facility and
Participant Data Management (URF/PDM). However, at the end of FP7, the Research
Family DGs were running 19 IT systems, each one having its own governance
structure.

[92]    
 Competitiveness and Innovation Framework Programme (CIP) target small and
medium-sized enterprises (SMEs). Its supports innovation activities (including
eco-innovation), provides better access to finance and delivers business
support services in the regions. The CIP runs from 2007 to 2013 with an overall
budget of € 3621 million.

[93]      The
grant Management Rationalisation exercise was launched in March 2011 following
the Communication "Getting the Best from IT in the Commission" from
Vice president Sefčovič. The aim is to achieve an efficient use of
the Commission's resources and investments and to ensure that efficient IT
tools support the real business needs.

[94]   The IT Programme Steering Committee is composed of the
Resources Directors of the Research DGs and it  is in charge of monitoring and
decision-making at operational level.

[95]   The Architecture Office Steering Committee (AOSC) coordinates
at IRM-level the IT activities between the IT teams. It supply its services
based on the agreement reached in the ITPSC meeting of 13/11/2008 on the
mission of the Architecture Office and its Service Level.

[96]    The Total Cost of Ownership (TCO) should include investments
and costs at IT and business side for the whole life of the project until its
final phase out. TCO is currently defined as all the costs (design,
construction and deployment) as well as the running costs (maintenance,
support, training, hosting, licences...) for the first five years of the project.
These costs should distinguish between outsourcing costs (in €) and
"internal ones" (to be expressed in FTE).

[97]      Considering
the cost of the phasing out to exiting IT systems in DG CNECT and RTD, the
total cost of the SYGMA Project goes up to 29M€.

[98]      A recommendation may be closed when partial actions have been
taken (e.g. design of the control) and/or where the internal control
environment has undergone significant change since the original audit, and
therefore either the residual risk needs to be reassessed or its effective
implementation in the context of the new internal control environment has to be
reassessed as part of a future audit. These recommendations, although closed,
are not considered by the IAS as implemented.

[99] Rec no1: "Ex-ante controls on contribution agreements"
and Rec no. 2: "Ex-ante controls on payments".