COMMISSION STAFF WORKING DOCUMENT Summary of Executive summaries Internal audit engagements finalised by the IAS in 2014 Accompanying the document Report from the Commission to the European Parliament and the Council Annual report to the Discharge Authority on Internal audits carried out in 2014 (Art 99(5) of the Financial Regulation)
Date: 2015-09-15
Institutions: European Commission, Secretariat-General
Subjects: administrative check, audit, budgetary discharge, financial year, fraud against the EU, management audit

EUROPEAN 
COMMISSION 

Brussels, 15.9.2015  
SWD(2015) 170 final 

COMMISSION STAFF WORKING DOCUMENT 

Summary of Executive summaries 

Internal audit engagements finalised by the IAS in 2014 

Accompanying the document 

Report from the Commission to the European Parliament and the Council 

Annual report to the Discharge Authority on Internal audits carried out in 2014 (Art 
99(5) of the Financial Regulation) 

{COM(2015) 441 final} 

EN 

   EN 

 
 
 
 
  
 
 
Content of this Staff working document (SWD)................................................................................. 5 

1. 

Statistical data ............................................................................................................................. 5 
Implementation of the 2014 audit plan ................................................................................ 5 
Statistical data on IAS recommendations ............................................................................. 6 

1.1. 
1.2. 

2. 

3. 

4. 

2.1. 

2.2. 

2.3. 

Horizontal audits......................................................................................................................... 8 
Audit on efficiency and effectiveness of the planning stage of the selection process - Multi 
DG (EPSO, DG HR, DG CNECT, DG SANCO – now DG SANTE, DG TAXUD)............................. 8 
Horizontal IT audit : Audit of management and supervision of outsourced IT services 
(contract management) – Multi DG (DG BUDG, DG DIGIT, DG HOME, OP, DG SANCO – now 
DG SANTE) ........................................................................................................................... 11 
Audit on the administrative processes supporting the European Semester – Multi DG (SG, 
SJ, DG COMM, DG COMP, DG ECFIN, DG EMPL, DG MARKT, DG TAXUD) .......................... 17 

3.1. 

Agriculture, natural resources and health.............................................................................. 19 
Gap Analysis Review of 2014-2020 Regulations for the Common Agricultural Policy, Phase 
1 - DG AGRI .......................................................................................................................... 19 

4.1. 

4.2. 

4.3. 
4.4. 
4.5. 

Cohesion..................................................................................................................................... 25 
Gap Analysis Review of Regulation 2014-2020 for European Structural and Investment 
Funds (ESI funds) Phase 1 – Multi DG (DG AGRI, DG EMPL, DG MARE, DG REGIO)............ 25 
Gap analysis of new legislation/design of 2014-20 Programming Period of European 
Structural and Investment Funds' (ESI funds) Phase 2 – Multi DG (DG EMPL, DG REGIO) . 30 
Audit on preparations for use of Financial Instruments in DG EMPL 2014-20 ................... 34 
Audit on preparations for use of Financial Instruments in DG REGIO 2014-20 .................. 36 
Limited Review of the calculation and the underlying methodology of DG REGIO's residual 
error rates for the 2013 Reporting Year.............................................................................. 38 

5. 

Research, energy and transport............................................................................................... 41 

5.1. 

5.2. 

5.3. 

5.4. 
5.5. 
5.6. 

Gap Analysis Review of the legislation regarding Horizon 2020 – Multi DG (DG CNECT, DG 
ENER, DG MOVE, DG RTD)................................................................................................... 41 
Audit on the implementation of FP7 control systems (including Supervision of External 
Bodies) in DG CNECT ........................................................................................................... 44 
Audit on the implementation of FP7 control systems (including Supervision of External 
Bodies) in DG RTD................................................................................................................ 47 
Audit on the implementation of FP7 control systems in ERCEA ......................................... 50 
Audit on procurement management in DG JRC .................................................................. 52 
Limited Review of the calculation and the underlying methodology of DG CNECT's residual 
error rate for the 2013 reporting year ................................................................................ 54 

6. 

Economic and financial affairs ................................................................................................ 56 
Audit on risk management and planning processes in DG ECFIN in the New Economic 
Governance context ............................................................................................................ 56 
Audit on DG MARKT's cooperation with the three Supervisory Bodies on Financial Services
............................................................................................................................................. 57 
Audit on performance measurement system in DG TAXUD Customs Activities................. 59 

6.1. 

6.2. 

6.3. 

7. 

External aid, development and enlargement .......................................................................... 63 

2 

 
 
7.1. 

7.2. 
7.3. 
7.4. 
7.5. 

Audit on contribution agreements with UN bodies and other International Organisations – 
DG DEVCO............................................................................................................................ 63 
Audit on contribution agreements with international organisations - DG ECHO ............... 66 
Audit on the assurance building process in EU Delegations – DG DEVCO .......................... 70 
Audit on budget support in DG DEVCO ............................................................................... 73 
Audit on the control strategy in FPI .................................................................................... 77 

8. 

IT audits ..................................................................................................................................... 80 
Audit on management of local IT in DG AGRI ..................................................................... 80 
Audit on IT governance in DG BUDG ................................................................................... 83 
Audit on the management of logical access to systems (ECAS/LDAP/windows) in DG DIGIT
............................................................................................................................................. 85 
Audit on the management of the IT projects in DG EAC (E4ALink and EVE)....................... 88 
Joint IAS/ IAC audit on the management of local IT in DG MARE ....................................... 90 
IT risk assessment in ERCEA ................................................................................................ 93 

8.1. 
8.2. 
8.3. 

8.4. 
8.5. 
8.6. 

9.  Management letter .................................................................................................................... 95 
Management letter on common features of performance measurement systems........... 95 

9.1. 

10.5. 

10.4. 

10.1. 
10.2. 
10.3. 

10.  Follow-up engagements (summarised).................................................................................. 100 
2nd Follow-up audit on risk management – Multi DG (SG, DG BUDG, FPI) ...................... 100 
Follow-up audit on the AAR process in the Commission - Multi DG (SG, DG BUDG)........ 100 
Follow-up audit on the charge-back process in the Commission - Multi DG (DG BUDG, DG 
DIGIT)................................................................................................................................. 101 
2nd Follow-up audit on compliance with payment deadlines - Multi DG (DG ECHO, DG 
DEVCO) .............................................................................................................................. 102 
Follow-up audit on the management and monitoring of staff allocation in the Commission 
services– Multi DG (SG, DG BUDG, DG HR, DG AGRI, DG COMP, DG DGT, DG RTD) ........ 102 
1st and 2nd Follow-up audit on the Overview Report on Executive Agencies – Multi DG 
(SG, DG BUDG, DG DIGIT, DG HR)...................................................................................... 103 
Follow-up audit on the Commission-wide audit on strategy and coordination of statistical 
data production, development and dissemination – Multi DG (DG AGRI, DG ESTAT, DG 
JRC, DG MARE, DG RTD) .................................................................................................... 103 
Follow-up audit of the internal control system for managing the Instrument for Pre-
accession Assistance for Rural Development (IPARD) in DG AGRI.................................... 104 
Follow-up audit on the control strategy implementation (Pillar 1 and 2) in DG AGRI...... 104 

10.9. 
10.10.  1st and 2nd Follow-up audit on the residual error rate calculations (Pillar 1 & 2) in DG 

10.8. 

10.7. 

10.6. 

AGRI................................................................................................................................... 104 
10.11.  Follow-up audit on fraud prevention and detection in DG AGRI ...................................... 106 
10.12.  Follow-up audit on the design and monitoring of DG AGRI Dir. J control strategy (Pillar 1 

and 2)................................................................................................................................. 106 
IAS Follow-up audit on SAM project management in DG BUDG....................................... 106 
10.13. 
10.14. 
IAS Follow-up audit on risk management in DG COMM ................................................... 106 
10.15.  1st and 2nd Follow-up audit on management of local IT in DG DEVCO ........................... 107 
10.16.  Follow-up audit on the limited review of the calculation of the residual error rate in DG 

DEVCO ............................................................................................................................... 107 

3 

 
 
10.17.  2nd Follow-up audit on EDF grants in DG DEVCO ............................................................. 107 
10.18. 

IAS Follow-up audit on long overdue recommendations from IAS audits on data centre 
operation and security (2006) and on corporate data network infrastructures and services 
(2008) in DG DIGIT............................................................................................................. 107 
10.19.  2nd Follow-up audit on business continuity management in DG DIGIT ........................... 108 
10.20.  Follow-up audit on the lifelong learning programme in EACEA / DG EAC ........................ 109 
10.21.  Follow-up audit on the control strategy in EACI (now EASME)......................................... 109 
10.22.  Follow-up audit on IT governance and performance in EAHC (now CHAFEA).................. 109 
10.23.  2nd Follow-up audit on the joint audit (IAC-IAS) on the implementation by the EIF of the 

competitiveness and innovation framework programme in DG ECFIN ............................ 109 
10.24.  Follow-up audit on off-budget operations: EFSM in DG ECFIN......................................... 109 
10.25.  Follow-up audit on performance of operational activities in DG ECHO ........................... 110 
10.26.  2nd Follow-up audit on IPA procurement in DG ELARG (now NEAR) ............................... 110 
10.27.  2nd Follow-up audit on the management of local IT in DG EMPL .................................... 110 
10.28.  Follow-up audit on the closure of the ESF 2000-2006 programming period in DG EMPL 110 
10.29.  Follow-up audit on the implementation of the ESF 2007-2013 programming period in DG 

EMPL.................................................................................................................................. 111 
10.30.  2nd Follow-up audit on the control strategy in DG EMPL................................................. 111 
10.31.  3th Follow-up audit on local IT systems supporting financial management in DG ENER . 111 
10.32.  Follow-up audit of the European Energy Programme for Recovery (EEPR) in DG ENER .. 112 
10.33.  Follow-up audit on the control strategy in DG ENTR (now GROW) .................................. 112 
10.34.  1st and 2nd Follow-up audit of DG ESTAT's preparedness to fulfil its role in the economic 

governance framework ..................................................................................................... 112 
10.35.  Follow-up of audit on the control strategy in shared management in DG HOME............ 112 
10.36.  2nd Follow-up audit on HR security in DG HR................................................................... 113 
10.37.  Follow-up audit on the monitoring of EU law implementation in DG JUST...................... 113 
10.38.  2nd Follow-up audit on management of procurement in OIB.......................................... 113 
10.39.  2nd Follow-up audit on fraud prevention and detection in OLAF .................................... 113 
10.40.  Follow-up audit of the joint Sickness Insurance Scheme managed by the PMO.............. 114 
10.41.  Follow-up audit on the implementation of FP7 control systems in REA .......................... 114 
10.42.  Follow-up audit on the closure of Cohesion Fund projects 2000-2006 in DG REGIO ....... 114 
10.43.  Follow-up audit on DG REGIO implementation of the 2007-2013 programming period . 114 
10.44.  Follow-up audit of the limited review on residual error rate in DG RTD .......................... 114 
10.45.  Follow-up audit on IT governance and performance in DG SANCO (now SANTE)............ 115 
10.46.  Follow-up audit on the control strategy in DG SANCO (now SANTE) ............................... 115 
10.47.  Follow-up audit on the handling of sensitive information in SJ........................................ 115 
10.48. 

IAS/IAC joint Follow-up audit on monitoring the implementation of EU law in DG TAXUD
........................................................................................................................................... 116 
10.49.  Follow-up audit on the management of local IT in DG TRADE.......................................... 116 

11.  List of acronyms ...................................................................................................................... 118 

4 

 
 
CONTENT OF THIS STAFF WORKING DOCUMENT (SWD) 

This  SWD  contains  the  introduction,  the  objectives  and  scopes,  the  strengths  and  the 
critical  and  very  important  recommendations  of  the  original  executive  summaries 
reflecting the state of play when the audit engagements were finalised (cut-off date for 
the exercise 31/01/2015). 

Each executive summary underwent the applicable standard professional validation and 
contradictory procedures between auditor and auditee at the time of the finalisation. It 
aims to provide a quick understanding of the audit and its main results. 

1.  STATISTICAL DATA 

1.1. 

Implementation of the 2014 audit plan1  

By  the  cut-off  date  of  31  January  2015,  the  IAS.B  had  implemented2  100%  of  its 
planned engagements (target 100%). 

One  hundred  and  five  reports  (including  audits,  follow-ups,  limited  reviews,  risk 
assessments and one management letter) were finalised, broken down as follows: 

2014  

2013 

2012 

Engagements 

Reports 

Engagements 

Reports 

Engagements 

Reports 

Audit 

Follow-up 

(Limited) Review 

Management 
Letter 

IT 
Assessment 

Risk 

Consulting 

Total6 

253 

53 

5 

1 

1 

0 

314 

67 

5 

1 

1 

0 

225 

48 

4 

1 

0 

0 

23 

59 

4 

1 

0 

0 

29 

32 

1 

1 

0 

1 

49 

37 

1 

1 

0 

1 

85 

105 

75 

87 

64 

89 

1 See also IAS's Annual Activity Report 2014. 
2 The Annex provides an overview of all completed audit and follow-up audit engagements. 
3 The  "Gap  analysis  of  new  legislation/design  of  2014-20  programming  period  of  European  Structural  and 
Investment Funds – Part 2" addressed to DG REGIO and DG EMPL is counted as two engagements. 

4 Some audits, in particular multi-DG audits, may give rise to more than one audit report. 
5 The "Audit on Control Strategy - Implementation in DG AGRI" is counted as two engagements conducted by 

two different audit teams resulting in a single audit report. 

6   The variation in the number of engagements and reports is not a suitable indicator on the workload as the 

individual engagements may vary in length.  

5 

 
 
 
 
 
  
 
                                                            
 
In order to ensure an efficient and effective implementation of the audit plan, the IAS 
plans  its  audit  work  on  the  basis  of  a  risk  assessment  and  a  capacity  analysis.  The 
implementation is then regularly monitored and adjustments are made as necessary. 

1.2. 

Statistical data on IAS recommendations  

The IAS issued the following number of recommendations (including their acceptance 
rate) in 2014: 

Priority 

Critical 

Very Important 

Important 

New 
recommendations 

Accepted 
recommendations 

Non-accepted 
recommendations 

0 

50 

77 

% 

N.A. 

100% 

97% 

0 

507 

75 

0 

0 

28 

% 

N.A. 

0% 

3% 

7 One  recommendation  rated  'very  important'  addressed  to  FPI  was  only  partially  accepted.  The  IAS 
recommended better documentation of the decision-making process for recoveries. FPI considered that they 
had  taken  several  initiatives  to  improve  this;  however,  the  IAS  audit  revealed  that  weaknesses  remained 
despite these initiatives. The IAS proposed this report for discussion at the APC. However, given the time 
that elapsed from finalising the audit and setting up the new APC, it was decided more practical to discuss 
this report in the context of a first follow-up, if significant issues arise. 

A second recommendation rated 'very important' addressed to DG REGIO was also only partially accepted. 
The IAS recommended that DG REGIO should develop guidance to the Member States on calculating the 
achieved  leverage  effect  of  financial  instruments  (i.e.  capacity  to  attract  additional  public  and  private 
funding)  to  show  the  net  effect  of  the  leverage  brought  about  by  the  use  of  financial  instruments  and  to 
ensure  that  this  is  properly  reported  in  the  annual  summary  reports  on  financial  instruments' 
implementation. DG REGIO considered that removing the national co-financing from the leverage concept 
would artificially lower the achieved leverage especially in cases when national co-financing is provided by 
private  investors.  According  to  the  IAS,  assessing  the  ability  of  financial  instruments  to  attract  new 
investment,  and  reporting on  this, would  be  entirely  in  line with  the  Commission's  move  towards  a more 
performance-based  culture,  particularly  given  the  increased  importance  of  this  funding  mechanism  in  the 
2014-20 period. This report was not proposed by the IAS for discussion at the APC due to the relatively 
early  stage  of  financial  instruments  in  their  life  cycle.  The  APC  may  decide  to  have  a  discussion,  in 
particular in the context of the first follow-up if significant issues arise. 

8 The  IAS  recommended  in  its  audit  on  contribution  agreements  with  international  organisations  that  DG 
DEVCO take appropriate steps to enable it to assess the achievement of project objectives and performance 
targets, by developing and disseminating best practices to Headquarters units and EU Delegations regarding 
project  monitoring  and  reporting,  such  as  reporting  examples  and  guidance  on  budget.  DG  DEVCO 
rejected this recommendation as they consider that it would not be in line with the financial rules applicable 
to  indirect  management,  by  which  the  international  organisations  decide  on  their  reporting  and  budget 
templates. The IAS drew the DG's attention to the fact that this recommendation did not relate to the legal 
framework but to a management issue. 
The IAS recommended in its audit on “Assurance Building Process in EU Delegations” that DG DEVCO 
should  require  EU  Delegations  (EUDs)  to  (i)  analyse  the  root  cause  of  errors  detected  during  the  DG's 
annual  Residual  Error  Rate  (RER)  study,  and  (ii)  if  and  when  systemic  errors  or  fraud-related  issues  are 
identified  during  this  study,  to  include  these  in  the  EUDs'  subsequent  External  Assistance  Management 
Reports  (EAMR)  (on  the  implementation  of  EU  development  and  cooperation  aid  including  the  Head  of 
Delegation's statement of assurance on the management of funds sub-delegated to him) together with any 
actions taken to mitigate the risks. DG DEVCO rejected the recommendation on the basis that the results of 
the RER study are not representative for individual Delegations and that the RER study is only available 

6 

 
 
 
  
  
  
  
                                                            
Desirable 

Total 

0 

127 

0 

125 

N.A. 

98% 

0 

2 

N.A. 

2% 

For  all  accepted  recommendations,  the  auditees  drafted  action  plans  9,  which  were 
submitted to and assessed as satisfactory by the IAS. 

The implementation of the accepted recommendations made during the period 2010-
2014, as assessed by auditees, as at 31 January 2015 was as follows10: 

Year 

Priority 

Total 

Implemented 

In progress (by number of months overdue) 

2010 

2011 

2012 

2013 

Critical 
Very Important 
Important 
Desirable 
2010 Total 
Critical 
Very Important 
Important 
Desirable 
2011 Total 
Critical 
Very Important 
Important 
Desirable 
2012 Total 
Critical 
Very Important 
Important 
Desirable 

% 

3,9%

1,9%

% 

2 
120 
151 
10 

No. 
2 
112 
148 
10 

No. 
0 
8 
3 
0 
283  272 96,1% 11
0 
0 
3 
0 
3
0 
12 
15 
0 

0 
47 
98 
10 
158  155 98,1%
0 
59 
105 
0 

0 
71 
120 
0 

0 
47 
101 
10 

191  164 85,9% 27 14,1%

0 
53 
68 
6 

0 
34 
44 
6 

0 
19 
24 
0 

No 
delay
0 
0 
0 
0 
0
0 
0 
0 
0 
0
0 
3 
5 
0 
8
0 
13 
6 
0 

0 - 
6 

 6 - 
12 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
2 
1 
0 
3 
0 
4 
13 
0 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
2 
2 
0 
4 
0 
2 
5 
0 

12+
0 
8 
3 
0 
11
0 
0 
3 
0 
3
0 
5 
7 
0 
12
0 
0 
0 
0 

after the completion of the EAMR and is just one of the tools through which fraud can be identified. The 
IAS  replied  to  the  DG  by  stressing  that  the  recommendation  focuses  on  the  mitigation  of  systemic  and 
fraud-related  risks,  if  any,  that  had  not  been  detected  by  all  previous  controls,  and  also  on  the  EUDs 
reporting on  this  in  the  EAMR,  and does  not  require  the  EUDs  to  use  the  RER  study  results  to  measure 
their weaknesses. 

9 Except in cases where the IAS made only general recommendations (e.g. where the engagements were only the 
first phase of a wider series of engagements) which DGs should take into account but which did not lead to 
action plans as such (for details, see the Gap Analysis reviews (phase 1) in sections 3.2.1, 3.3.1 and 3.3.2 
below). 

10 It should be noted that the table reflect the current priority rating of the recommendations and takes account of 
any  downgrading  of  recommendations  (for  instance  from  'very  important'  to  'important',  e.g.  following  a 
follow-up audit. This explains differences to previous years' reports.  

7 

 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
                                                                                                                                                                                          
2014 

2013 Total 
Critical 
Very Important 
Important 
Desirable 
2014 Total 

TOTAL 2010-2014  
Thereof Critical or Very 
Important  

84 66,1% 43 33,9%

12711 
19
0 
0 
0 
44 
48 
50 
56 
63 
75 
0 
0 
0 
125 
14 11,2% 111 88,8% 100
884  689 77,9% 195 22,1% 127

0 
2 
12 
0 

17 
0 
4 
7 
0 
11 
31 

7 
0 
0 
0 
0 
0 
11 

0
0 
0 
0 
0 
0
26

343 

256 

74,6%

87 

25,4% 

60 

10 

4 

13 

Out of all recommendations rated 'very important' or 'critical' and issued in the period 
2010-2014, 17 very important recommendations (2%) were overdue by more than six 
months with respect to the deadline set in the initial action plan12. No critical 
recommendation is outstanding. The Audit Progress Committee (APC) was regularly 
informed of critical or very important recommendations overdue for more than six 
months. 

The total number of accepted recommendations issued during 2010-2014 and for which 
the IAS had conducted follow-up audits by the end of 2014, amounts to 640 (72%). 

The follow-up work carried out by the IAS confirmed that overall recommendations are 
being implemented in a satisfactory way thus contributing to the improvement of the 
control systems in the audited services. The IAS closed 95% of the recommendations 
followed-up during this period. 

2.  HORIZONTAL AUDITS 

2.1.  Audit  on  efficiency  and  effectiveness  of  the  planning  stage  of  the 
selection process - Multi DG (EPSO, DG HR, DG CNECT, DG SANCO 
– now DG SANTE, DG TAXUD) 

Background 

11  The  difference  to  the  figures  presented  in  the  99.5-report  covering  the  year  2013  relates  to  seven  identical 
recommendations addressed to both DGs RTD and CNECT and stemming from the 2013 audit on SYGMA 
-  Phase  1.  These  seven  recommendations  were  inadvertently  counted  twice  in  the  report  and  is  now 
corrected, which explains the difference to last year's report.  

  Furthermore, it should be noted that the two DGs did not accept one part of a recommendation rated 'very 
important' in the end while at the time of preparation of the 99.5-report covering the year 2013, the DGs 
signalled  full  acceptance.  The  rejected  part  related  to  the  identification  of  the  full  cost  of  the  IT  project 
(SYGMA),  pending  a  decision  at  the  level  of  the  IT  governance  bodies  of  the  Commission  to  provide  a 
single  methodology  to  calculate  the  costs  of  an  IT  project.  The  IAS  considers  that  the  DGs  have  at  their 
disposal  basic  guidance  to  adequately  define  the  total  cost  of  a  project  and  therefore  kept  its 
recommendation. 

12 Cut-off date is 31 January 2014. 

8 

 
 
  
  
  
  
  
  
  
  
 
                                                            
 
The human resources selection process is defined as the process of differentiating 
between applicants in order to identify those with greater likelihood of success in 
the job. 

Within  the  European  Institutions  (EU  Institutions),  the  selection  process  should 
allow the recruitment of candidates with "the highest standard of ability, efficiency 
and integrity13". 

Two key players intervene in the EU Institutions' staff selection process, notably: 

(a) 

the  European  Personnel  Selection  Office  (EPSO),  that  since  2003  "organises 
open competitions with a view to securing the services of officials on optimal 
professional  and  financial  terms  for  the  institutions  of  the  European 
Communities14" and manages the reserve lists; 

(b) 

the EU Institutions, which are responsible for defining their staff needs and for 
recruiting staff from the pool of candidates placed by EPSO on reserve lists. 

Audit Objectives and scope 

The overall objective of the audit was to assess the effectiveness and efficiency of 
the  current  planning  stage  of  the  selection  process  in  replying  to  the  EU 
Institutions' needs of new staff. 

The  audit  covered  the  planning  processes  in  place  in  EPSO  and  in  the  European 
Commission. In particular, it focused on: 

•  At EPSO level: the review of the requests transmitted by the EU Institutions, 
the monitoring of the reserve lists and the planning for new competitions, as 
well as the coordination of the whole exercise; 

•  At  European Commission's level:  

–  DG  HR's  coordination  and  monitoring  activities,  the  identification  of  the 
Commission's needs based on the Workforce Simulator (WFS), the reliability 
of the data and criteria used for the assessment of needs; 

–  Operational DGs'/Services' methods to determine their needs of new staff, the 

reliability of data used and the adequacy of the local HR tools available. 

Concerning  the  operational  DGs/Services,  the  present  audit  analysed  the  procedures  in 
place  in  DG  CNECT,  DG  SANCO  and  DG  TAXUD.  The  results  of  other  IAS  audits 

13 Staff Regulations, Art.27, Chapter 1 Recruitment, Title III Career of Officials. 
14 Decision 2002/620/EC establishing a European Communities Personnel Selection Office (EPSO). 

9 

 
 
 
                                                            
related  to  HR  topics15,  performed  in  2012-2013,  have  been  taken  into  account  where 
relevant, in order to provide an overview of the situation at Commission level. 

There are no observations/reservations in the AAR of the DGs/Services covered by 
this engagement that relate to the area/process audited. 

fieldwork  was 

The 
recommendations relate to the situation as of that date. 

finalised  on  30/01/2014.  All  observations 

and 

Strengths 

a) At EPSO level 

•  implementation  of  a  three-year  rolling  plan  and  strategic  annual  planning  exercise  to 

predict and agree staffing needs with the EU Institutions; 

•  introduction of a regular cycle of competitions for the main generalist profiles, organised 

according to a fixed timetable; 

•  regular monitoring of the recruitment rate from the reserve lists and feedback provided to 
the  EU  Institutions  on  the  subject,  during  the  monthly  meetings  of  the  EPSO  Working 
Group. 

b) At Commission level 

•  implementation of the automated Workforce Simulator tool (WFS) to estimate the number 

and profiles of officials leaving the Commission in the coming years; 

•  close monitoring of the recruitment requests from DGs and of the use of reserve lists. 

Major Audit Findings 

The IAS has identified the following two very important issues: 

EPSO planning exercise (report finding N° 1) 

EPSO did not provide guidelines and instructions to foster a common understanding of 
the  requirements  which  should  result  in  a  common  method  for  assessing  the  need  for 
laureates across the EU Institutions, and did not ask the Institutions to provide sufficient 
details on the criteria taken into account in the assessment of their requests for laureates 
with  a  view  to  preparing  the  draft  planning  of  competitions  based  on  comparable 
information.  In  addition,  in  the  last  three  years  most  EU  Institutions  have  been  late  in 
contributing to the annual planning exercise, delaying the launch of open competitions. 

Planning exercise at Commission level – Role of the DGs (report finding N° 3) 

15 Audits on "Management and monitoring of staff allocation in the Commission Services" (carried out in 2012 in 
DG HR, RTD, COMP, AGRI and DGT), "HR management in response to the financial crisis" (DG ECFIN, 
DG MARKT and DG COMP), and "Performance measurement systems" (DG EMPL and DG REGIO). 
10 

 
 
 
                                                            
DGs/Services  do  not  implement  and  use  HR  management  (HRM)  tools  (strategic 
planning,  unit  management  plan,  task  mapping,  workload  assessment)  consistently  to 
assess their current and future staff needs. 

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  EPSO planning exercise:  

EPSO  should  provide  guidelines  and  instructions  to  the  EU  Institutions  to  increase 
the coherence and comparability of their requests for laureates and should ask them 
to provide sufficient details on the criteria taken into account with a view to correctly 
prioritising  the  requests  and  aligning  the  competitions  to  be  organised  with  the 
Institutions' real needs and recruiting capacities. Better scheduling of the exercise, in 
agreement with the Institutions, should also help EPSO reduce delays and finalise its 
plan in a timely manner. 

•  Planning exercise at Commission level – Role of the DGs: 

DGs/Services  should 
tools  (e.g.  strategic  planning,  unit 
management  plans,  workload  assessment,  competence/task  mapping)  consistently 
and use their output in the analysis of future recruitment needs. 

implement  HRM 

The audited services have established action plans which the IAS considers satisfactory 
to address the recommendations. 

2.2.  Horizontal  IT  audit  :  Audit  of  management  and  supervision  of 
outsourced IT services (contract management) – Multi DG (DG BUDG, 
DG DIGIT, DG HOME, OP, DG SANCO – now DG SANTE) 

Background 

The European Commission is continuously increasing the use of IT in most of its fields 
of activity. It therefore has to ensure that it is getting the best value for money from its 
investments in IT, which has consistently grown over the last years. 

A significant part of the IT expenditure of the Commission is devoted to the outsourcing 
of  IT  services  to  internal  or  external  providers.  The  benefits  of  outsourcing  include 
quality  improvements,  flexibility,  better  risk  management  and  freeing  up  internal 
resources to focus on core, value-adding activities. 

Outsourcing  should  be  viewed  not  just  as  a  procurement  management  exercise  (i.e. 
purchase of services) but also as a strategic management decision whereby third parties 
participate in the value chain through the provision of a service. 

Therefore, a key factor is that by outsourcing an IT service, an organisation transfers the 
operational  responsibility  to  the  supplier,  but  remains  accountable  and  hence  has  to 
ensure that the risks are being managed and there is continued delivery of value from the 
service provider. 

11 

 
 
If the outsourcing entity does not retain accountability for the outsourced IT services, its 
needs may not be clearly defined and/or not clearly communicated to the supplier or the 
contractors' underperformance may not be timely detected or not adequately addressed. 
These  may  negatively  affect  the activities and the targets of the outsourcing entity and 
lead  to  ineffective  use  of  financial  resources.  Consequently,  the  DGs/Services  have  to 
implement  an  appropriate  structure  (roles,  responsibilities,  procedures  and  controls) 
enabling a proper management and supervision of the outsourced IT services to ensure 
that the service providers deliver value for money. 

In the Commission, the responsibilities are set up at the corporate and local level. At the 
corporate level DG DIGIT and DG Budget provide guidance, instructions and templates, 
develop training and manage framework contracts. These contribute to the environment 
facilitating  the  management  of  IT  contracts  by  the  operational  DGs.  At  the  local 
(operational)  level,  DGs/Services  are  responsible  for  defining  needs,  implementing 
individual  contracts  and  ensuring  that  the  services  provided  meet  their  requirements  in 
quality,  quantity  and  timeliness.  This  can  be  achieved  through  an  effective  contract 
management which includes a correct definition of the needs, a clear communication of 
the  needs  to  the  IT  service  provider  and  a  continuous  monitoring  of  the  provider’s 
performance. 

Audit Objectives and Scope 

The  overall  objective  of  the  audit  was  to  assess  the  effectiveness  and  efficiency  of 
Commission's  processes  in  place  for  the  management  and  supervision  of  contracts  for 
outsourced IT services with a view to ensuring that value for money is obtained. 

The  scope  of  the  audit  covered  the  environment16  created  by  the  Central  Services  to 
facilitate  the  monitoring  and  supervision  of  IT  contracts  and  the  actions  taken  by  the 
operational Services for managing and supervising the IT contracts. The audit included 
DG  DIGIT  (in  its  central  and  operational  roles),  DG  Budget  (in  its  central  role)  and  a 
sample of operational Services (DG SANCO, DG HOME and the Publications Office). 

There  were  no  observations/reservations  in  the  AAR  of  the  DGs  covered  by  the 
engagement that relate to the area/process audited. 

The  fieldwork  was  finalised  in  mid-November.  All  observations  and  recommendations 
relate to the situation as of that date. 

Strengths 

•  Commission-level 

DG  Budget  has  developed  detailed  guidelines  on  procurement  and  templates  of 
contracts  for  IT  services  that  are  available  for  all  the  operational  Services  of  the 
Commission; 

For  the  calls  for  tender  for  Commission-wide  framework  contracts,  DG  DIGIT 
prepares  "Orientation  documents"  to  take  stock  of  the  lessons  learnt  from  previous 

16 The environment encompasses guidance, sharing of good practices, training, and involvement of DGs in the 

definition of needs. 

12 

 
 
                                                            
contracts  and  to  identify  the  risks  that  could  materialise  during  the  procurement 
process and the related mitigating measures. It also involves the potential users in the 
DGs/Services in the definition of the needs; 

DG DIGIT provides a comprehensive set of tools (i.e. e-Request, e-Ordering and e-
Invoicing)  facilitating  the  management  of  contracts  for  IT  services  and  the 
relationship between the users in DGs and the contractors; 

The  main  framework  contracts  recently  concluded  foresee  explicit  provisions  for 
checking, before the signature of the specific contract, the education and professional 
experience of "intra muros" consultants; 

All DG DIGIT framework contracts foresee the use of pre-defined KPIs to measure 
and monitor the performance of the contractors. 

•  Operational Services 

Some good practices were identified, in particular as regards: 

Clear  procedure  for  the  upstream  phase  (i.e.  before  launching  a  call  for  tender) 
enabling the proper identification of needs and of issues from previous contracts; 

DG's policy on type of working mode (i.e. time and means, quoted time and means, 
fixed price) for outsourced IT services; 

The use of an IT system for the management of intra muros staff; 

The  use  of  contractual  clauses  that  allow  immediate  action  in  case  of  contractor 
underperformance. 

Major Audit Findings for DG DIGIT 

The IAS has identified the following three very important issues: 

Estimation of needs before establishing a framework contract (report finding N° 1) 

At  Commission  level,  DG  DIGIT's  approach  to  estimate  the  needs  is  not  consistent 
across the different framework contracts it manages, in particular regarding the timing of 
the collection of the needs, the level of detail requested from the DGs/Services and the 
metrics  used  to  define  the  needs  (euro,  man-days,  man-year).  In  addition,  DG  DIGIT 
does not propose criteria to operational Services to estimate their needs so as to ensure 
comparability of data. The lack of detailed statistics on the use of framework contracts 
does  not  allow  DG  DIGIT  to  assess  adequately  the  reasonableness  of  the  needs  at 
Commission level. 

At the operational level, some DGs/Services do not have a structured process to estimate 
the needs for their own framework contracts encompassing the definition of criteria that  
should  be  considered  and  identifying  potential  issues  and/or  lessons  learnt  from  past 
contracts  that  should  be  addressed  in  the  tender  specifications.  In  some  cases,  this 
process  in  not  sufficiently  documented  to  allow  a  quality  review  of  the  tender 

13 

 
 
documentation  (e.g.  by  the  actors  involved  in  the  validation  of  the  tender  documents, 
such as the authorising officer). 

Quality management of services outsourced on a time and means basis (report finding 
N° 2) 

Although  they  had  the  possibility,  none  of  the  DGs/Services  using  IT  consultants 
working  on  a  time and means basis (i.e. intra muros) asked for supporting evidence of 
the  education  and  working  experience  of  the  intra  muros  consultants,  relying  on  the 
contractors' reliability for the information provided in the CVs. 

The  KPIs  for  measuring  the  contractors'  performance  under  the  Commission-wide 
framework contracts differ in terms of content and the way they have to be calculated. In 
addition,  they  do  not  allow  capturing  the  performances  of  the  contractors  at  the 
operational  DG  level  and  are  not  always  reported  to  the  end-users  in  the  operational 
in  case  of  contractor 
Services.  The  mechanism 
underperformance  is  not  the  same  for  all  framework  contracts  and  appears  to  be  too 
complex. As a result, it is rarely used. 

liquidated  damages 

to  apply 

For  their  own  framework  contracts,  the  operational  Services  have  different  levels  of 
maturity for defining and using KPIs. In some contracts no KPIs are defined, in others 
KPIs are pre-defined by the DGs. In one case, the DG also includes the possibility for the 
contractor to provide additional KPIs. 

Guideline on the choice of type of outsourcing (report finding N° 3) 

At Commission level, there is no guidance on how the operational Services should assess 
their  needs  for  intra  or  extra  muros  consultants  and  what  criteria  they  should  consider 
when  choosing  between  different  types  of  outsourcing,  in  order  to  have  a  coherent 
approach among Services. In addition, there is no analysis aiming at identifying the key 
factors  that  drive  the  choice  of  the  operational  Services,  the  advantages,  disadvantages 
and the risks of each working mode. 

The operational Services do not base their choice on the result of a cost-benefit analysis 
but  on  operational  reasons.  This  decision  is  normally  not  re-assessed  based  on 
experience with past contracts. The sampled  DGs  show  a  high  degree  of heterogeneity 
for similar tasks, with some using exclusively intra muros consultants and others using 
extra muros consultants with an exceptional use of intra muros ones. 

Recommendations 

To address these issues, the IAS formulated recommendations which can be summarised 
as follows: 

Estimation of needs before establishing a framework contract 

DG  DIGIT    should  strengthen  the  process  to  assess  the  needs  of  calls  for  tender  for 
which it is "chef de file" by harmonising the surveys used, providing a minimum set of 
instructions  to  ensure  DGs  use  similar  estimation  criteria  and  by  using  meaningful 
statistics to assess the reasonableness of the consolidated needs at Commission level. 

14 

 
 
DGs should implement a structured and traceable process based on pre-defined criteria, 
analysis of past issues and identification of risks related to the tender procedure. 

Quality management of work outsourced on a time and means basis 

DG  DIGIT,  in  its  central  role,  should  harmonise  and  enhance  the  performance 
monitoring mechanism and how it is translated in the contractual arrangements. The DGs 
should take stock of the good practices implemented at corporate level and improve the 
performance measurement system for their own framework contracts. 

Guideline on the choice of type of outsourcing 

DG  DIGIT,  in  its  central  role  should  identify  the  factors  that  drive  the  choice  of  the 
DGs/Services,  the  advantages,  disadvantages  and  the  risks  of  each  type  of  outsourcing 
and  issue  guidance  to  help  the  operational  DGs/Services  choose  the  most  appropriate 
working  mode.  At  the  operational  level,  the  choice  should  be  based  on  a  cost-benefit 
analysis  taking  into  consideration  the  particularities  and  constraints  of  the  outsourced 
services. 

Major Audit Findings for DG SANCO 

The IAS has identified the following two very important issues: 

Quality  of  tender  documentation  for  DG  SANCO's  own  framework  contracts  (report 
finding No.1-SANCO) 

Weaknesses were identified in three aspects of the DG's tender documentation for an in-
house IT framework contract: a) validation of tender documents: a lack of guidance for 
ex-ante financial verifiers has resulted in inconsistencies in tender documents; b) internal 
guidance  on  procurement:  there  is  currently  no  consolidated  version  of  the  DG's 
guidance  on  procurement  and  there  is  no  flowchart  covering  the  entire  procurement 
procedure  and  the  role  of  the  actors  intervening,  including  the  responsibilities  of  the 
Public  Procurement  Committee;  c)  contractual  provisions  defining  the  acceptable  level 
of  service:  DG  SANCO's  framework  contracts  do  not  define  indicators  and  targets  to 
accurately  measure  and  monitor  the  overall  performance  of  the  contractor.  In  addition, 
the  provisions  for  applying  liquidated  damages  in  cases  of  underperformance  are  not 
linked to an objective measurement of the contractor's performance. 

DG SANCO's outsourcing strategy (report finding No.2-SANCO) 

Contrary  to  DG  BUDG's  guidance  on  using  extra  muros  services  as  a  rule  and  intra 
muros  services  as  an  exception,  DG  SANCO  has  opted  for  the  opposite  solution.  The 
choice of working exclusively with Time & Means orders (i.e. intra muros services) is 
not based on a cost-benefit analysis (either at the DG level, or on a case by case basis, for 
each project) or on analysis of historical data, but mainly on the operational advantages 
of working with intra muros staff (e.g. adaptive planning, flexible and rapid response to 
changes). It should also be noted that an accurate cost-benefit analysis should factor in 
the  use  of  Commission  resources  by  intra  muros  staff.  For  these  reasons,  the  claimed 
savings  quantified  by  DG  SANCO  in  its  IT  Master  Plan,  resulting  from  using  intra 

15 

 
 
muros rather than extra muros development, are not justified and hence their correctness 
cannot be assessed. 

Recommendations 

To  address  these  issues  the  IAS  has  formulated  recommendations  which  can  be 
summarised as follows: 

Quality of tender documentation for DG SANCO's own framework contracts 

In  order  to  improve  effectiveness  of  ex-ante  controls  and  the  quality  of  tender 
specifications  DG  SANCO  should  improve  communication  on  tender  content  and 
context    between  operational  and  Finance  and Control  units  and  develop  checklists  for 
the  process  of  validating  tender  documents.  To  clarify  the  roles  and  responsibilities  of 
the  different  actors  involved  in  the  procurement  procedure,  including  the  Public 
Procurement  Committee,  DG  SANCO  should  consolidate  all  procurement  guidance  in 
one document. To monitor and obtain assurance that the services provided by contractors 
meet  the  expected  level,  DG  SANCO  should  include  a  minimum  set  of  performance 
indicators in the tender specifications of its future IT related calls for tender. 

DG SANCO's outsourcing strategy 

DG  SANCO  should  include  in  the  Business  Case  of  its  IT  projects  an  ex-ante  cost 
benefit analysis of the various outsourcing options (Time and Means, Quoted Time and 
Means, Fixed Price) to identify the best solution for each particular project. In addition, 
during the lifetime of an IT project, the need to continue working with Time and Means 
orders should be reassessed at key points, e.g. when passing from the development to the 
maintenance  phase,  to  ensure  the  best  value  for  money  when  choosing  the  type  of 
contract. 

Major Audit Findings for the Office of Publications (OP) 

The IAS has identified the following two very important issues: 

Evaluation of OP's own call for tenders prior to publication (report finding No.1-OP) 

Weaknesses were detected in the ex-ante evaluation procedure for OP calls for tender, in 
particular  the  use  of  an  overly-complex  questionnaire  for  evaluators.  This  has  led  in 
some cases to an incorrect estimation of needs. Consequently, the OP had to increase the 
maximum budgetary ceiling of the contract, where possible, or to launch a new call for 
tender earlier than foreseen, thus incurring additional costs for the Office. 

Provisions in the tender specifications of OP's own calls for tender (report finding No.2-
OP) 

The  Service  Level  Agreements  (SLAs)  incorporated  into  OP's  own  calls  for  tender  are 
not currently implemented as stated in the tender specifications, either in terms of their 
function or content. A separate issue was identified at the evaluation stage of OP calls for 
tender: contrary to  DG  BUDG guidance,  pre-defined KPIs are used as technical award 
criteria. There are also inconsistencies in evaluators' assessments of the additional KPIs 
submitted in tenderers' offers. 

16 

 
 
Recommendations 

To  address  these  issues  the  IAS  has  formulated  recommendations  which  can  be 
summarised as follows: 

Evaluation of OP's own call for tenders prior to publication 

To  improve  the  quality  of  the  evaluations  of  tenders  and  their  added  value  for  future 
tender  procedures,  OP  should  review  the  current  ex-ante  evaluation  questionnaire  to 
streamline and focus it on the most important points, e.g. needs estimation and lessons 
learned  from  previous  contracts.  Staff  awareness  of  the  importance  of  the  evaluation 
process  should  also  be  raised,  to  improve  the  quality  of  information  provided  by 
evaluators. 

Provisions in the tender specifications of OP's own calls for tender 

To  address  the  SLA  issue,  the  OP  should  assess  for  each  call  for  tender  if  an 
accompanying SLA is necessary and draft the tender specifications accordingly. In cases 
where  an  SLA  is  considered  necessary  for  the  execution  of  the  contract,  the  tender 
specifications should clearly define its content and it should be signed together with the 
contract. On the issue of KPIs, the OP should follow the guidance of DG BUDG on both 
pre-defined  and  additional  KPIs:  pre-defined  KPIs  should  no  longer  be  used  as  a 
criterion for the qualitative assessment of offers as tenderers have to comply with them; 
optional  elements  such  as  additional  KPIs  should  not  be  foreseen  in  the  tender 
specifications. 

The audited services have established action plans which the IAS considers satisfactory 
to address the recommendations. 

2.3.  Audit  on  the  administrative  processes  supporting  the  European 
Semester  –  Multi  DG  (SG,  SJ,  DG  COMM,  DG  COMP,  DG  ECFIN, 
DG EMPL, DG MARKT, DG TAXUD) 

Background 

The IAS audit on  the European  Semester (ES) process was scheduled in the IAS audit 
work programme for 2014. 

The  ES  has  been  designed  to  ensure  that  the  Member  States  (MSs)  discuss  and 
coordinate  their  budgetary,  macro-economic  and  structural  reform  plans  with  the  EU 
institutions and other MSs at specific times throughout the year. Several deliverables are 
produced  by  the  Commission,  which  involves  a  high  level  of  coordination  and 
cooperation between the DGs and Services. The ES process is relatively new as it was 
introduced in 2010 and executed for the first time in 2011. 

Audit Objectives 

17 

 
 
 
The  overall  objective  of  this  performance  audit  was  to  address  the  following  question: 
Are  the  administrative  processes  supporting  the  European  Semester  effective  and 
efficient across the Commission? The audit assessed the adequacy of the internal control 
system regarding the production and communication of the various ES deliverables. 

Audit Scope 

The  audit  focussed  on  the  following  main  areas  and  addressed  the  corresponding  sub-
questions: 

•  Has the Commission put in place effective and efficient processes and procedures for the 

organisation and management of the ES? 

•  Has the Commission put in place effective and efficient processes and procedures for the 

planning, implementation and monitoring of the ES? 

•  Have the DGs put in place an effective and efficient resource management? 

•  Have the DGs set up an effective and efficient quality assurance/management programme 

to follow-up on the continuous improvement of the ES process in all its aspects? 

•  Are communication/information channels effective and efficient? 

•  Have  the  DGs  put  in  place  an  effective  and  efficient  document  management  system  to 

ensure the security, including confidentiality and integrity of ES deliverables? 

The  following  DGs  were  selected  for  this  audit:  SG,  DG  ECFIN,  DG  EMPL,  DG 
TAXUD, DG COMM, DG COMP, DG MARKT and the Legal Service. 

There were no  observations in the  relevant  DGs'  Annual  Activity  Reports  (AARs)  that 
relate to the process audited. 

The fieldwork was finalised on 29/10/2014. All findings and recommendations relate to 
the situation as of that date. 

Strengths 

The  European  Semester  (ES)  coordinates  a  number  of  deliverables  with  different 
purposes emanating from the complex underlying legislation of the Stability and Growth 
Pact and the Macro-economic Imbalance Procedure. The scope of the ES covers the full 
range of the Europe 2020 strategy, leading to the involvement of several DGs and many 
people across the Commission. The ES calendar is constrained by dates fixed in the legal 
basis, the MS budgetary cycles, and requirements of the Council committees. 

Taking into account that the first cycle of the ES started in 2011 and that it had to evolve 
over  time,  the  auditors  recognise  the  good  results  that  the  DGs  involved  have  already 
achieved  and  that  the  Commission  has  consistently  delivered  its  ES  objectives  under 
extremely tight deadlines. 

The following major strengths in the ES process were identified. 

18 

 
 
•  In general, efficient internal DG coordination and timely delivery of inputs in a very tight 
ES timetable. In each DG involved in the ES process, a well-structured and strong Central 
Coordination unit has been created. 

•  The professionalism and dedication of the staff involved, including working overtime, also 

contributed significantly to the achievement of the ES objectives. 

•  Creation of cross DGs Country Teams, coordinated by SG's Country Team Leaders. 

•  Creation  of  European  Semester  Officers  (ESOs)  in  the  Representations  in  the  Member 

States and backup persons. 

•  In  the  course  of  the  ES,  several  meetings  are  held  to  coordinate  between  DGs  and  to 
inform  each  other  (e.g.  meetings  with  Europe  2020  coordinators  in  DGs,  country  team 
leaders, ESOs, and Country Team meetings, Core group meetings and Council committee 
meetings). 

•  Good communication, exchange of information and implementing a no surprise approach 

with the MSs via bilateral meetings. 

•  Internal  coordination  of  the  preparation  is  very  efficient  for  most  key  deliverables. 
Guidelines  and  templates  exist  in  the  SG  and  DG  ECFIN  and,  in  general,  sufficient 
guidance  is  given  for  the  units  involved  in  the  process.  Guidance  is  also  given  in  the 
periodic Country Team meetings, coordinated by the SG.  

•  After the closing of the ES, post-mortem exercises are held in several DGs involved in the 
ES  cycle,  which  shows  their  willingness  to  continuously  improve  the  whole  process  and 
working methods every year. 

•  Good quality training organised on the ES process open to Commission staff. 

Major Audit Findings 

The  audit  in  the  SG  and  the  sampled  DGs  did not  identify  any  issues  that  gave  rise  to 
critical or very important recommendations. 

Overall,  the  audit  showed  that  the  administrative  processes  in  the  SG  and  the  sampled 
DGs support the implementation of the European Semester across the Commission in an 
effective and efficient way. 

3.  AGRICULTURE, NATURAL RESOURCES AND HEALTH 

3.1.  Gap Analysis Review of 2014-2020 Regulations for the Common 

Agricultural Policy, Phase 1 - DG AGRI 

Background 

The Multi-annual Financial Framework provides for the new CAP period 2014-2020 for 
the  preservation  and  management  of  natural  resources  a  total  of  €  373  179  m, 

19 

 
 
representing  some  38%  of  the  total  Commission  expenditure  for  the  period.  Of  this,  € 
277  851  m  relates  to  Pillar  1,  which  includes  market  related  expenditure  and  direct 
payments.  The  management  of  the  European  Structural  and  Investment  Funds  (ESI 
Funds), which, like CAP expenditure, is shared between the Member States (MS) and the 
Commission, involves overall commitment appropriations of € 325 146 million in 2014-
20 under heading 1b, with € 84 936 million for the CAP - Pillar 2 under heading 2. 

As part of the IAS Strategic Audit Plan 2013-2015, the IAS is carrying out a gap analysis 
review of the 2014-2020 Regulation for the CAP in 2014-2015. This is being conducted 
in  two  phases.  Phase  1  consists  of  a  review  of  the  Common  Provisions  Regulation 
(CPR),  the  four  basic  regulations  for  the  CAP  -  Direct  Payments  (Pillar  1),  Single 
Common  Market  Organisation  (Pillar  1),  Rural  Development  (Pillar  2)  and  Horizontal 
Regulation  for  financing,  managing  and  monitoring  the  CAP  -  and  any  relevant 
Delegated and Implementing Acts (DAs and IAs), to the extent they are available at the 
time  of  the  audit.  Phase  2  will  be  a  more  in  depth  examination  of  the  design  and 
preparations being made by DG AGRI for dealing with the new programming period. 

There  are  six  layers  of  rules:  common  provisions,  general  provisions,  fund-specific 
provisions, DAs, IAs and Commission guidelines. The CPR was established to improve 
coordination  and  harmonised  implementation  of  the  Funds  providing  support  under 
Cohesion Policy (ERDF, ESF and CF), with the Funds for Rural Development (EAFRD) 
and for the Maritime and Fisheries Sector (EMFF), together the five ESI Funds. Part II 
sets  out  common  rules  for  the  ESI  Funds.  For  the  AGRI  area,  the  general  rules  on 
financing,  management  and  monitoring  have  been  set  up  in  the  regulation  on  the 
financing,  management  and  monitoring  of  the  Common  Agricultural  Policy  (the 
"Horizontal Regulation"),  covering the two funds. Specific rules applicable to each main 
area  (Rural  development,  Market  measures  and  Direct  payments)  were  specified  in 
separate regulations. 

The  CAP  regulations  empowers  the  Commission  to  adopt  Delegated  Acts  with 
supplementary rules to cover certain non-essential elements and also Implementing Acts, 
which  give  effect  to  the  rules  which  have  already  been  laid  down  for  the  situations  in 
which there is no need to establish any new rules or norms compared to the basic acts. 
The legislative package, comprising the CPR, the Horizontal Regulation for the CAP and 
the specific regulation for Cohesion and AGRI, was adopted on 17/12/2013 and for the 
EMFF on 20/05/2014. 

Objectives 

The main objective of the Phase 1 review was to highlight, for the most important areas, 
the  additional  risks  the  Commission  is  facing  as  a  result  of  the  new  CAP  regulations, 
taking  account  of  the  need  to  have  an  appropriate  balance  between  reducing  the 
administrative burden, but at the same time maintaining the necessary level of control for 
exercising its supervisory responsibilities under shared management. 

Scope 

The  review  covered  to  a  limited  extent  the  CPR  (part  II  applicable  to  Rural 
Development),  the  associated  secondary  legislation  where  appropriate,  the  Horizontal 

20 

 
 
Regulation for the CAP and the fund-specific regulations for Direct Payments, Common 
Market Measures and Rural Development respectively. 

There were no reservations relating to the new CAP period 2014-2020. 

However, the following reservations were made in the 2013 AAR related to the previous 
period: 

•  Reservation  1:  ABB02  –  Expenditure  on  Market  Measures:  7  aid  schemes  in  9  Member 
States  (11  elements  of  reservation):  Austria,  the  Czech  Republic,  France,  Italy, 
Netherlands, Poland, Spain, Sweden and the United Kingdom. 

•  Reservation  2:  ABB03  –  Direct  payments:  20  paying  agencies,  comprising  6  Member 
States:  Spain  (15  paying  agencies),  France,  UK  (RPA-  England),  Greece,  Hungary  and 
Portugal. 

•  Reservation 3: ABB04 – Rural development expenditure: 31 paying agencies, comprising 
19  Member  States:  Belgium,  Bulgaria,  Cyprus,  Germany  (2  paying  agencies),  Denmark, 
Spain (6 paying agencies), Finland, France (2 paying agencies), UK (2 paying agencies), 
Greece,  Ireland,  Italy  (5  paying  agencies),  Luxembourg,  Netherlands,  Poland,  Portugal, 
Romania and Sweden. 

The  fieldwork  was  finalised  on  15/10/2014.  All  observations  and  recommendations 
relate to the situation as of that date. 

Major Audit Findings 

This  engagement  constituted  a  very  specific review  of  the  legislation  underpinning  the 
new period 2014-2020 and it was only the first phase of a wider examination of the DG's 
preparations for 2014-20. The CPR and the CAP Horizontal Regulation bring together a 
number  of  key  improvements  aimed  at  harmonising  and  simplifying  the  arrangements 
governing  the  Structural  Funds  and  the  two  pillars  of  the  Agriculture  area.  The  IAS 
acknowledges  the  efforts  made  by  the  Commission's  services  during  the  negotiation 
phase to protect the Commission's interests in its supervisory role, particularly in the face 
of  very  strong  external  political  pressures.  However,  the  final  adopted  legislation  has 
resulted in significant additional risks, which will need to be addressed as part of the DG 
AGRI’s preparations for the design and implementation of controls in the new period. 

The main theme emerging as a result of this assessment, and which recurs across most of 
the  findings  is  the  sheer  complexity  and  volume  of  the  changes  brought  about  by  the 
legislative  process.  Across  the  board,  but  notably  in  key  areas  such  as  'greening',  a 
number of new measures were introduced, together with a large number of derogations, 
exceptions and supplementary rules which have offered greater flexibility to MS. With 
so many changes, the rules become complicated and therefore difficult to understand and 
apply in practice. The IAS notes the efforts made by DG AGRI, particularly towards and 
since  the  end  of  the  IAS  fieldwork,  to  address  these  concerns,  for  example  the 
preparation of vade-mecums and detailed guidelines to be adopted by the Commission. 
Nevertheless,  the  scope  for  interpretation  on  the  part  of  MS  has  been  significantly 
increased, which in turn can have an equally significantly impact on the error rates. 

21 

 
 
Responsibility of Managing Authorities, Paying Agencies and Certifying Bodies and 
Accreditation Process 

The  number  of  PAs  will  remain  the  same  (82)  for  the  new  period,  which  represents  a 
missed opportunity to strengthen the underlying systems by streamlining the organisation 
and having a narrower span of control. This could in turn mean that the existing control 
problems will persist in the new programming period. 

Suspensions of payments 

legal 

framework 

introduces  a  new 

The  CAP  Horizontal  Regulation 
for 
reducing/suspending payments and defines the conditions necessary for the identification 
of  serious  deficiencies  in  the  control  system  which  are  not  remedied  by  the  MS 
concerned in accordance with an action plan. It also imposes a limit (2 years) on the time 
the Commission can suspend payments in the case of systemic errors. The effect of this 
time limit could be that payments are resumed even before remedial actions to address 
the  reason  for  suspension  are  fully  implemented.  Also,  although  payments  can  be 
suspended if MS submit control statistics late, there is no penalty if the data is of poor 
quality  and  the  Commission's  powers  were  further  reduced  by  limiting  the  amounts 
which can be suspended. As a consequence, MS have little incentive to submit accurate 
data. This is another missed opportunity to tackle the long-standing problems as regards 
the reliability of information submitted by MS. 

Financial Corrections & Recovery of Irregular Payments 

An opportunity was missed to align more closely financial correction processes between 
CAP and Cohesion area. Under the CAP, this is expected to take twice as long compared 
to the Cohesion area - due to the additional conciliation step envisaged in the Horizontal 
Regulation.  Furthermore,  the  Delegated  Act  defines  or  categorizes  in  a  very  generic 
manner the key/ancillary controls which still needs to be complemented by more detailed 
guidelines. These controls are essential for assessing national systems, in order to decide 
on the seriousness of control weaknesses giving rise to corrections. 

As  is  the  case  with  Cohesion  area,  the  Commission  can  increase  the  level  of  flat  rate 
corrections in the case of persistent deficiencies under the CAP. However, the conditions 
for  doing  this  are  much  less  explicitly  stated  in  the  Delegated  Act  supplementing  the 
Horizontal Regulation than they are for the Cohesion area legislation and will therefore 
need to be supported by concrete guidance in order to be effectively applied in practice. 
Although  the  Commission  can  suspend/reduce  payments  if  MS  are  not  diligent  in 
recovering  irregular  payments,  the  financial  consequence  of  non-recovery  has  to  be 
shared between both the MS and the Commission (so called "fifty-fifty rule"), as was the 
case  previously  and  contrary  to  the  Commission's  proposal.  This  was  a  missed 
opportunity  to  incentivise  MS  to  improve  their  recovery  systems  and  simplify  its 
management. 

Monitoring performance (performance framework) 

The performance framework set out in the CPR is key to the results based focus of the 
ESI Funds in the 2014-2020 programming period and is complemented in the CAP area 
by  the  Horizontal  Regulation,  which  sets  out  monitoring  arrangements  covering  both 

22 

 
 
pillars.  However,  the  legislative  process  has  resulted  in  a  number  of  exceptions  and 
conditions  to  the  rules  being  accepted  which  could  effectively  prevent  the  use  of 
suspensions/financial corrections even where the priorities fail to achieve milestones or 
targets. This could in turn weaken the ultimate impact of the performance framework. 

The long-standing problems as regards the reliability of MS data may also impact on the 
ability  of  DGs  to  effectively  assess  progress  and  performance  and  there  is  also  an 
inherent  risk  that  MS  will  set  unambitious  milestones  and  targets  to  avoid  risking  the 
performance reserve. 

Eligibility Rules – First Pillar 

Eligibility rules have become significantly more complex overall. The legislative process 
has opened up a wide range of options and different possibilities for MS which can only 
serve  to  increase  the  risk  of  misinterpretation  and  ultimately  non-compliance.  In 
addition,  it  will  inevitably  place  significant  additional  pressure  on  the  Commission's 
resources  in  assisting  and  supporting  the  MS,  ensuring  consistency  and  controlling 
different practices across them. 

The  introduction  of  Ecological  Focus  Areas  (EFA)  is  one  of  the  key  new  measures. 
However,  the  definition  is  now  even  wider  than  originally  planned  and  MS  have 
considerable flexibility in choosing what elements to include, some of which are likely to 
be  more  difficult  or  costly  to  control3.  Also,  a  significant  number  of  exemptions  were 
introduced which again will complicate arrangements even further. 

The co-legislators also introduced a transitional period to allow MS ensure that the Land 
Parcel Identification System (LPIS) can cope with EFAs. The LPIS is a key part of the 
control  system  and  if  not  updated  on  time  and/or  the  underlying  data  is  not  of  good 
quality then it could have a significant impact on the error rate. During this transitional 
period, there is a higher risk of errors as it will not be possible to cross check between 
the EFA declared and the LPIS. Since the completion of the audit fieldwork DG AGRI 
has  since  told  the  IAS  that  it  plans  to  mitigate  this  risk  through  increased  on-the-spot 
checks  and  recording  the  EFA  declared  by  the  beneficiaries  in  the  LPIS,  after 
verification and before payment. 

As  regards  the  Common  Organisation  of  Markets  (CMO),  new  measures  such  as  the 
promotion of wine mean a greater overlap between the two Pillars, which could in turn 
increase the risk of double funding. In addition, the CAP now allows the Commission to 
take measures in crisis situations. However, as to what constitutes such a crisis and/or the 
underlying criteria which need to be met before triggering such measures, this has yet to 
be defined. This could present a reputational risk in so far as it could lead, through for 
example  political  pressure,  to  measures  which  distort  normal  price  movements  and 
impact  adversely  on  the  financial  possibilities  to  deal  with  genuine  cases  of 
emergency/crisis. 

Basic Payment Scheme and other direct payment schemes 

A number of new measures were introduced aimed at greening the CAP and ensuring a 
more equitable distribution of resources between and within MS. The most important of 
these  was  the  Basic  Payment  Scheme  (BPS),  but  a  range  of  other  measures  was  also 

23 

 
 
introduced, such as the young farmers scheme and also new voluntary measures such as 
redistributive  payments,  small  farmers  scheme  and  payments  for  areas  with  natural 
constraints and voluntary coupled support. These were all subject to significant changes 
during the negotiations aimed at essentially providing more flexibility to MS, but which 
at the time results in more complicated rules and greater scope for interpretation. 

As regards the BPS, the Commission originally proposed a uniform payment per hectare 
by 2019, but a significant number of exceptions and derogations were introduced during 
the negotiations relating to, for example, the allocation of payment entitlements, internal 
convergence rules and the possibility to maintain payment entitlements under the Single 
Payment  Scheme,  etc.  These  derogations  and  options  could  ultimately  lead  to  a  wrong 
allocation of payment entitlements with multi-annual implications. 

Greening 

'Greening' is probably the single most important change to the CAP, representing some 
30% of the national envelope for direct payments. It also constitutes one of the biggest 
overall  risks,  particularly  given  the  scale  of  amendments,  options  and  derogations 
introduced during the negotiations, which will make the rules very complicated and the 
measures difficult to control in practice. 

Greening is implemented through three main measures: crop diversification, maintaining 
of  existing  grassland  and  introduction  of  an  ecological  focus  area  (EFA).  However, 
under the 'equivalent practices' initiative, MS can replace these measures with measures 
which yield equal or greater environmental and climate benefits. These can be covered 
by  agri-environmental  climate  measures  under  rural  development  programs  or  by 
national  certification  schemes,  in  which  case  MS  can  define  the  certification 
arrangements. In addition to being inherently very complex schemes and prone to error, 
the possibility to deliver these measures under rural development programs means there 
is also the risk of double funding. 

Greening  will  also  present  a  major  challenge  for  the  Integrated  Administrative  and 
Control System (IACS), particularly as regards to the verification of crop diversification, 
the EFA, permanent grassland and equivalent practices. The complexity of the measures 
involved will mean that the IACS will need to be significantly adapted to cope with the 
new  rules  and  certain  requirements  can  be  checked  only  on-the-spot.  In  addition,  the 
sanctions  system  for  non-compliance  with  greening  requirements  is  likely  to  have  a 
lower deterrent effect than the normal sanctions system under IACS, particularly during 
the early years. The Commission proposed that penalties would apply from the first year 
of application (2015), but the co-legislators reduced the deterrent effect by deciding that 
these should apply only on a gradual basis. 

Recommendations 

Reflecting the fact that this is only Phase 1 of the gap assessment, only recommendations 
of  a  general  nature  have  been  formulated,  which  should  be  taken  into  account  going 
forward  in  preparing  for  the  2014-20  Programming  Period,  but  which  do  not  require 
specific  action  plans  as  such.  Where  appropriate,  these  recommendations  will  be  made 
more concrete, based on the detailed findings that will arise from the IAS Phase 2 work: 

24 

 
 
•  DG  AGRI  should  finalise  any  new  requirements  embedded  in  the  CPR,  the 
Horizontal Regulation for the CAP and the 3 other specific regulations as soon as 
possible  through  Delegated/Implementing  Acts  (if  empowered  in  the  legal  acts) 
and clarify the rules through detailed guidance to the MS. 

•  DG AGRI should formalise the risk assessment of the new schemes to support, 
assess and adapt their control and audit strategies accordingly to properly address 
the risks arising from the new legislation and the choices made by each MS. This 
should  also  help  DG  AGRI  management  to  monitor  and  supervise  the  required 
actions. 
In  view  of  the  new  elements  introduced  by  the  CPR  and  the  CAP  four  basic 
regulations,  appropriate  training  should  be  given  to  staff  so  as  to  ensure 
consistent application of the rules, both between and within DGs. 

• 

4.  COHESION 

4.1.  Gap Analysis Review of Regulation 2014-2020 for European Structural 
and Investment Funds (ESI funds) Phase 1 – Multi DG (DG AGRI, DG 
EMPL, DG MARE, DG REGIO) 

Background 

The management of the European Structural and Investment Funds (ESI Funds), which 
is shared between the Member States (MSs) and the Commission, involves commitment 
appropriations of EUR 325 146 million in 2014-20, representing some 45% of the total 
Commission expenditure for the period. 

As part of the IAS Strategic Audit Plan 2013-2015 the IAS is carrying out a gap analysis 
review  of  the  2014-2020  Regulation  for  ESI  Funds  in  2014-2015.  This  is  being 
conducted  in  two  phases.  Phase  1  consisted  of  a  review  of  the  Common  Provisions 
Regulation 
relevant  Delegated  and 
Implementing  Acts  (DAs  and  IAs),  to  the  extent  they  are  available  at  the  time  of  the 
audit. Phase 2 is a more in depth examination of the design and preparations being made 
by  the  specific  Directorates-General  (DGs)  concerned  for  dealing  with  the  new 
programming period. 

regulation  and  any 

fund-specific 

(CPR), 

There  are  six  layers  of  rules:  common  provisions,  general  provisions,  fund-specific 
provisions, DAs, IAs and Commission guidelines. The CPR was established to improve 
coordination  and  harmonised  implementation  of  the  Funds  providing  support  under 
Cohesion Policy (ERDF, ESF and CF), with the Funds for Rural Development (EAFRD) 
and for the Maritime and Fisheries Sector (EMFF), together the five ESI Funds. Part II 
sets  out  common  rules  for  the  ESI  Funds.  In  addition,  it  contains  in  Part  III  general 
provisions  which  apply  to  the  ERDF,  ESF  and  CF  (called  "Funds").  Part  IV  contains 
general provisions applicable to the Funds and the EMFF. The general provisions (part 
III and IV) as laid down in the CPR do not apply to the EAFRD. For the AGRI area, the 
general rules on financing, management and monitoring have been set up in the Common 
Agricultural Policy (CAP) regulation, which apply to both EAFRD and EAGF. 

The CPR empowers the Commission to adopt DAs of general application to supplement 
or amend certain non-essential elements of the legislative act and IAs, which lay down 

25 

 
 
uniform conditions for implementing legally binding Union acts. The first wave of DAs 
and IAs necessary for the programming phase has been adopted by the Commission and 
parts of the second wave necessary for programme implementation are close to adoption. 
In  addition,  some  70  draft  guidance  notes  have  been  produced  under  the  lead  of  DG 
REGIO  and  coordinated  with  the  other  DGs.  Reflecting  the  specificities  of  the  ESI 
Funds, specific rules applicable to each fund and to the European Territorial Cooperation 
under  the  ERDF  were  set  out  in  separate  regulations.  The  legislative  package, 
comprising  the  CPR  and  the  fund-specific  regulation  for  Cohesion  and  AGRI,  was 
adopted on 17/12/2013 and for the EMFF on 20/05/2014. 

Objectives 

The main objective of the Phase 1 review was to highlight, for the most important areas, 
the additional risks the Commission is facing as a result of the co-legislative process for 
the CPR, taking account of the need to have an appropriate balance between reducing the 
administrative burden, but at the same time maintaining the necessary level of control for 
exercising its supervisory responsibilities under shared management. 

Scope 

The review covered the CPR and associated secondary legislation where appropriate. It 
covered  the  ESI  Funds  and  the  DGs  responsible  (DG  REGIO,  DG  EMPL  and  DG 
MARE). Recognising that only part of the CPR applies to EAFRD, as managed by DG 
AGRI and that there is a separate horizontal CAP regulation, together with the specific 
regulations  for  Direct  Payments,  Common  Market  Measures  and  Rural  Development 
respectively,  the  findings  relating  specifically  to  DG  AGRI  were  consolidated  and 
communicated separately. 

There  were  no  observations/reservations  in  the  respective  AARs  that  relate  to  the 
area/processes reviewed. 

The fieldwork was finalised on 9/07/2014. All observations and recommendations relate 
to the situation as of that date. 

Major Audit Findings 

This  engagement  constituted  a  very  specific review  of  the  legislation  underpinning  the 
new Programming Period and it was only the first phase of a wider examination of the 
DGs’ preparations for 2014-20. The CPR brings together under one heading a number of 
key improvements aimed at harmonising and simplifying the arrangements governing the 
Structural Funds. The IAS welcomes this approach and acknowledges the efforts made 
by the Commission's services during the negotiation phase to protect the Commission's 
interests in its supervisory role, particularly in the face of very strong external political 
pressures.  However,  when  compared  to  the  Commission’s  original  proposals,  the  final 
adopted legislation has resulted in significant additional challenges which will need to be 
addressed as part of the DGs’ preparations for the design and implementation of controls 
in the new Programming Period. 

Legislation architecture and the need to harmonise/simplify 

26 

 
 
The  adopted  CPR  now  constitutes  one  overarching  set  of  rules  for  the  European 
Structural and Investment Funds (ESI Funds). However, when taken together with all the 
associated  secondary  legislation,  it  constitutes  a  complex  and  voluminous  overall 
package,  which  is  not  always  readily  understandable.  In  practice,  harmonisation  seems 
more  geared  to  the  beneficiaries  rather  than  towards  the  national  bodies  charged  with 
implementing the rules. The CPR was subject to a very heavy legislative process and the 
co-legislators  introduced  many  changes.  Although  the  legally  binding  secondary 
legislation can provide a safeguard in so far as it allows the Commission to impose more 
detailed mandatory requirements on Member States, it has been very extensive in nature. 
The associated definitions as to what should constitute derogations, supplementary rules 
or non-essential elements were not necessarily applied by the co-legislators in a coherent 
manner  to  the  cohesion  policy  legislative  package  as  a  whole,  which  meant  in  some 
cases  that  different  types  of  acts  were  adopted  for  essentially  the  same  areas.  Taken 
together,  this  could  lead  to  problems  of  interpretation  on  the  part  of  Member  States, 
which in turn could ultimately increase the risk of errors. 

Member State Management and Control Systems (roles of Managing Authorities, 
Certifying Authorities and Audit Authorities) 

The  Commission  originally  proposed  that  national  bodies  would  be  accredited  by  an 
accrediting body set up at the ministerial level, but following the legislative process this 
was  changed  to  a  designation  process,  in  line  with  the  provisions  of  the  Financial 
Regulation which was adopted by the co-legislator shortly before. Furthermore, whereas 
in  the  previous  programming  period  the  Commission  systematically  reviewed  at  a 
second stage the compliance assessment of Member State systems, the Commission will 
now  review  the  designation  process  on  a  risk  basis  only.  This  change  was  already 
included in the Commission's proposal and was adopted by the co-legislators. 

The 2014-20 legislation prevents the Commission from interrupting payments simply for 
the  reason  that  a  management  body  is  put  under  probation  and/or  its  designation  is 
ended.  However,  the  IAS  notes  that  it  can  use  Art  83  of  CPR  to  interrupt  payments 
where  there  is  evidence  to  suggest  significant  deficiencies  in  the  management  and 
control systems based on the results of national audit work. 

The Commission originally proposed that the Audit Authority (AA) could not be part of 
the same public body as the Managing Authority (MA) for high value OPs, but following 
the legislation process this was allowed, provided that either the Commission took an Art 
73 decision during the previous programming period (i.e. could formally rely on the AA) 
or where it is satisfied concerning the independence and reliability of the audit authority 
based on past experience. 

The DGs’ view is that designation risks are mitigated to the extent that in many cases the 
management and control systems will be similar to those under the previous period and 
that they have a detailed knowledge of these built up over many years. 

As  regards  the  Audit  Authorities,  the  Commission  originally  proposed  that  sampling 
should  be  appropriately  based,  without  specifying  whether  this  should  be  statistical  or 
nonstatistical. However, the final CPR allows the use of non-statistical sampling where 
appropriately  justified,  while  still  indicating  that  the  general  rule  is  that  statistical 
sampling should be used. Also, despite challenging the Council's proposals to reduce the 

27 

 
 
audit  coverage  parameters,  it  was  the  Council’s  proposals  which  were  finally  adopted, 
with a clear requirement in case of non-statistical sampling that "the size of the sample 
shall  be  sufficient  to  enable  the  AA  to  draw  up  a  valid  audit  opinion".  However,  in  a 
separate statement related to Art 127 of the CPR, the Commission highlights the risk that 
reducing the size of the sample of operations could make it insufficiently representative, 
which would in turn weaken the audit assurance. 

Concerning the single audit approach, the legislators went further than the Commission’s 
proposals  to  reduce  controls  where  it  can  rely  on  the  AA  or  the  national  systems  by 
specifically precluding Commission audits on-the-spot if the Commission has agreed so 
with  the  AA,  unless  there  was  evidence  of  deficiencies.  Also,  there  were  changes 
(reductions)  to  the  Commission's  original  proposals  as  regards  the  retention  of 
documents which in turn places additional pressures on both the Audit Authorities and 
the  Commission's  own  auditors,  as  it  reduces  in  practical  terms  the  time  window  for 
audits,  particularly  where  systemic  errors  are  involved  which  require  more  work  to 
estimate the population affected. 

In addition, the impact on resources could also be very important in so far as audits on 
the  2014-2020  period  will  overlap  with  those  on  the  closure  of  the  previous  period. 
Although the DGs expect to rely more and more on the work of the AAs under the new 
programing period, Commission audits will continue to play an important role as part of 
the  overall  control  strategy  but  audit  resources  are  scarce  and  require  very  careful 
planning  and  scheduling.  Any  knock-on  impact  on  the  Commission's  audit  coverage 
could  leave  it  open  to  challenge  in  the  case  where  there  are  financial  corrections 
involved. 

Suspensions of payments and Financial Corrections 

The application of financial corrections was intensively discussed during the legislative 
process  and  subject  to  many  modifications  from  the  co-legislators  before  being  finally 
adopted. These were aimed at seeking to restrict the circumstances and conditions under 
which  net  corrections  would  be  systematically  applied.  The  introduction  of  obligatory 
net financial corrections strengthens arrangements compared to the previous period and 
the  Commission  originally  proposed  that  net  corrections  would  apply  from  the  date  at 
which the Member States submit their accounts. However, the final CPR is less strict in 
so far as it allows MSs to continue to detect, report and correct irregularities and replace 
expenditure  up  to  the  date  at  which  EU  audits,  either  by  the  Commission  or  by  ECA, 
detect  a  serious  deficiency  not  previously  detected  nor  corrected  by  the  MS.  The  final 
outcome,  compared  with  the  original  Commission  proposal,  could  lead  to  a  reduced 
incentive  on  MS  to  correct  irregular  expenditure  at  an  early  stage  of  the  process  and 
could thus in turn delay the correction process overall. 

The  new  legislation  also  provides  for  the  suspensions  of  payments  and  financial 
corrections linked to the performance of the OPs themselves, for example, where there 
information,  ex-ante 
are  weaknesses  regarding 
conditionalities and the achievement of milestones. In the previous programming period, 
the  Commission  was  responsible  for  deciding  on  suspensions  and  for  2014-2020  its 
original  proposal  was  to  keep  the  whole  suspension  process  in  Commission  hands. 
However,  the  final  legislation  allows  the  Commission  to  propose  to  the  Council  the 

the  reliability  of  performance 

28 

 
 
suspension  of  part  or  all  of  the  payments  under  Art.  23  of  the  CPR  which  is  linked  to 
sound economic governance. 

Performance Framework 

The introduction of the performance framework is key to the results based focus of the 
ESI Funds in the 2014-2020 programming period and overall, constitutes a positive step 
the  move  towards  ensuring  more  effective  policy  delivery  in  practice.  However,  the 
legislative process resulted in agreement of a number of exceptions and conditions to the 
rules,  which  could  mean  that  non-performance  may  not  actually  be  extensively 
penalised,  which  could  in  turn  weaken  the  ultimate  impact  of  the  framework.  Inherent 
reliability problems with MS data may have a knock-on impact on the ability of DGs to 
effectively assess the progress of OPs. There is also an inherent risk that Member States 
will  set  unambitious  milestones  and  targets  to  avoid  risking  the  performance  reserve. 
From  the  DGs'  perspective,  this  needs  very  careful  monitoring  and  for  staff  to  be 
appropriately trained and guided on what to look for. The risk could be compounded by 
the short timescale available for assessing OPs. However, the IAS notes that the units in 
charge of evaluation are also involved in the negotiation phase for the OPs and that this 
partially addresses the risk. 

Eligibility Rules 

Reflecting the complex and extensive overall legislative package, the eligibility rules for 
the ESI Funds are equally complex and multi-layered in nature. They include the CPR, 
the Fund-specific regulations, two DAs and national level rules. Consequently, this may 
pose a challenge for both Commission and Member State bodies in terms of verification 
and control. There are still significant differences between the individual funds and as far 
as the "investments in infrastructure" or "productive investments" are concerned. Unclear 
legislative provisions can increase the risk of irregularities/ineligible expenditure. 

Recommendations 

Reflecting the fact that this is only Phase 1 of the gap assessment, only recommendations 
of  a  general  nature  have  been  formulated,  which  should  be  taken  into  account  going 
forward  in  preparing  for  the  2014-20  Programming  Period,  but  which  do  not  require 
specific  action  plans  as  such.  Where  appropriate,  these  recommendations  will  be  made 
more concrete, based on the detailed findings that will arise from the IAS Phase 2 work: 

•  The  DGs  should  finalise  any  new  requirements  embedded  in  the  CPR  and  the 
fund  specific  regulation  as  soon  as  possible  through  Delegated/Implementing 
Acts  (if  empowered  in  the  legal  acts)  and  clarify  the  rules  through  detailed 
guidance to the MSs. 

•  The DGs should assess and adapt their control and audit strategies accordingly to 

properly address the risks arising from the new legislation. 

• 

In view of the new elements introduced by the CPR, appropriate training should 
be given to staff so as to ensure consistent application of the rules, both between 
and  within  DGs.  The  IAS  notes  however  that  knowledge  is  shared  between  the 
DGs  through  the  Sharepoint  site  on  "interpretation",  the  common  use  of  the 

29 

 
 
workflow IT tool WAVE and common trainings courses delivered by DG REGIO 
to the other DGs' staff. 

4.2.  Gap analysis of new legislation/design of 2014-20 Programming Period 

of European Structural and Investment Funds' (ESI funds) Phase 2 – 
Multi DG (DG EMPL, DG REGIO) 

Background 

As part of the IAS Strategic Audit Plan 2013-2015, the IAS planned to carry out a gap 
analysis  of  the  new  legislation  and  design  of  the  2014-20  programming  period  of 
European Structural and Investment Funds (ESI Funds) to be conducted in two phases in 
2014. 

Phase 1 reviewed the adopted legislation in order to assess the extent to which it reflects 
the Commission's original proposals/objectives to have an appropriate balance between 
reducing the administrative burden, but at the same time maintaining the necessary level 
of  control  to  exercise  its  supervisory  responsibilities  under  shared  management.  This 
engagement concerned DG REGIO, DG EMPL, DG AGRI and DG MARE. 

Audit Objectives 

Phase 2 is a more in depth examination of the design of the systems for the management 
of the 2014-20 programming period of the ESI funds by DG REGIO (ERDF/CF) and DG 
EMPL (ESF), and to the extent possible in this early phase of the programming period, 
the implementation of these in practice. The outcome of the work on Phase 1 was taken 
into account for Phase 2. 

In  conducting  phase  2,  the  IAS  clearly  recognises  that  the  development  of  the  control 
architecture  is  very  much  an  on-going  process.  Therefore,  this  is  reflected  in  the  audit 
results, in so far as these present a snapshot at a particular point in time. It should also be 
emphasized  that  the  early  nature  of  the  audit  was  designed  to  be  able  to  capture  the 
approval process for Operational Programmes. 

Audit Scope 

The audit focused on the DGs' processes for: 

•  the negotiation, assessment and adoption of the Operational Programmes (OP), 

•  guiding and supervising the set-up of the Member States' (MS) Management and Control 

Systems (MCS). 

Particular emphasis was given to new elements of MCS as compared to the 2007-2013 
programming period as well as aspects related to the results orientation of the 2014-2020 
programming period. 

There  were  no  observations/reservations  in  the  AAR  that  relate  to  the  area/process 
audited. 

30 

 
 
The  fieldwork  was  finalised  on  30/09/2014.  All  observations  and  recommendations 
relate to the situation as of that date. 

Strengths 

The  auditors  recognise  the  ongoing  efforts  made  by  DGs  REGIO  and  EMPL  to  put  in 
place a solid basis for the new 2014-2020 programming period's operational programmes 
and  management  and  control  systems.  During  the  audit,  the  auditors  identified  the 
following strengths: 

(a) 

Negotiation and adoption of OPs 

Overall,  the  processes  for  the  negotiation  and  adoption  of  OPs  for  the  2014-20 
programming period were well prepared. The process started on time, mid-2012 with the 
preparation of the Position Papers17  and the informal dialogue phase. Detailed guidance 
was  prepared  jointly  by  the  ESIF  DGs  for  their  staff  and  for  the  MS.  The  negotiation 
phase  is  well  supervised  by  senior  management  in  both  DGs.  There  is  pro-active 
cooperation  between  the  ESIF  DGs  (e.g.  bi-lateral  monthly  meetings  of  the  Directors-
General,  Stock  Taking  Group,  Ex  ante  Conditionality  Suspension  Committee).  DG 
EMPL  has  in-house  policy  expertise  on  the  main  thematic  objectives  for  which  it  is 
responsible  and  the  role  of  the  Geographical  Units  (GU)  typically  comprises  both 
monitoring  of  policy  and  ESF  programme  management.  DG  REGIO  has  introduced 
Competence Centres (CC) to provide expertise on the thematic objectives to the GU. 

(b) 

Supervising Member State management and control systems 

The requirements for the MS' MCS largely build on the 2007-2013 programming period, 
but with important new elements, to further strengthen systems, such as a management 
declaration and annual accounts. DGs REGIO and EMPL have worked hard to develop a 
common  approach  to  supervising  the  MS'  management  and  control  systems  and  the 
development of common guidance to the MS is well advanced. The joint effort includes 
as well the development of a single audit strategy for the ESIF DGs and a common audit 
plan,  including  also  joint  audits,  based  on  a  common  risk  assessment.  Furthermore 
common Engagement Planning Memoranda (EPM) and checklists are being developed. 
Common  IT  systems  support  the  audit  (MAPAR)  and  data  mining  (ARACHNE) 
processes.  The  DGs  have  organised  anti-fraud/anti-corruption  seminars  attended  by  all 
MS, complemented by seminars for the most risky MS and targeted training to managing 
and audit authorities on management and control systems are on-going. 

Major Audit Findings 

The IAS has identified the following four very important issues: 

Supervising MS management and control systems (report finding N° 1) 

17 The purpose of the position papers is to set out the framework for dialogue between the Commission and the 
MS on the preparation of the Partnership Agreement and the Operational Programmes. The paper sets out 
the key country specific challenges and presents the Commission’s preliminary views on the main funding 
priorities. 

31 

 
 
                                                            
It is still relatively early in the programming period and the outline single audit strategy 
will still be updated, but as it stands, certain elements are not clear and will need to be 
addressed as the strategy matures. Currently, there is little explanation as to how the DGs 
will obtain assurance on the reliability of the Audit Authorities (AA), including their new 
tasks, even though their work will continue to be the main source of assurance. The IAS 
acknowledges  the  move  towards  a  more  annual  focus  in  the  2014-20  programming 
period, notably the review of the annual assurance package and that therefore the DGs' 
audit  strategy  focuses  mainly  on  the  assurance  driven  by  this  annual  review  (as  from 
February 2016 on the year 2015). However, the strategy does not specify how the DGs 
will in addition build up assurance on a multi-annual basis and how to ensure that any 
errors  can  be  detected  during  the  beneficiaries'  retention  period  for  documents18.  Few 
early  preventive  system  audits  are  initially  scheduled/budgeted,  although  these  are  the 
only  way  in  which  the  Commission  can  legally  implement  corrective  measures  in  the 
early phases of the programming period. However, the IAS notes that given the delays in 
the adoption of programmes no expenditure has been declared by MS for the moment. 

OP negotiation and adoption process (report finding N° 2) 

There are significant delays in the adoption of OPs in comparison to the initial forecasts. 
Whereas  DG  EMPL  did  not  have  a  specific  forecast  in  terms  of  number  of  OPs  to  be 
adopted,  DG  REGIO  aimed  to  adopt  all  mainstream  programmes  before  the  end  of 
October 2014. As of December 2nd, the forecast is that about half of the OPs for which 
REGIO  and  EMPL  are  chef  de  file,  will  be  adopted  by  the  end  of  the  year.  At  that 
moment,  only  30  OPs  for  which  REGIO  is  chef  de  file  and  24  OPs  for  EMPL  had 
actually been adopted. These delays are largely due to the timing and lack of quality of 
documents submitted by the MS. 

The  quality  issues  concern  most  of  the  OPs  submitted  by  the  MS,  but  these  were 
appropriately  identified  by  the  DGs  and  resulted  in  a  high  number  of  observations 
communicated  to  the  MS.  However,  the  OP  assessment/adoption  process  was 
documented in varying ways.  Most Desk Officers (DO) did not actually document their 
own  analysis  of  the  OP  and  some  checks  did  not  encompass  all  stages  of  the  process. 
Also, the follow up of observations communicated to MS varied considerably in practice. 
The Ex-Ante Conditionalities (EAC) assessment is mostly documented at the MS level, 
while only in some cases the checklist was filled-out for each OP. This is in particular 
true  for  general  EAC  No.7  "Statistical  systems  and  result  indicators",  which  has  to  be 
assessed mainly at the OP level, but this was not always done or documented, even when 
observations were already sent to the MS. 

Results orientation and performance framework (report finding N° 3) 

Although  the  observations  sent  to  MS  include  remarks  on  targets/milestones  and,  in 
some  cases,  such  issues  were  discussed  between  the  DO  and  the  MS  or  the  evaluation 
unit, there is limited concrete evidence of DOs actually trying to assess pro-actively their 
plausibility  in  order  to  address  possible  unambitious  targets  set  by  MS  and  any 
assessment made by them is generally not documented. 

18 For projects with a value above 1 million EUR, the retention period ends 2 years after the year in which the 
accounts including the final expenditure of the completed operation are submitted. For smaller projects the 
retention period ends 3 years after the year in which the accounts including the expenditure were submitted. 

32 

 
 
                                                            
IT systems supporting the management of the PP 2014-2020 processes (report finding 
N° 4) 

During the first months of the operation of WAVE, a high number of incidents/defects 
and last-minute change requests occurred, which, at times, have delayed the operational 
processes for managing the adoption of Partnership Agreements (PA) and OPs. This led 
DG EMPL to temporarily suspend the use of WAVE, while DG REGIO continued to use 
it. There have also been considerable delays in the development and implementation of 
WAVE.  These  are  due  to  a  combination  of  reasons,  including  changes  in  priorities 
agreed  by  the  Steering  Committee,  a  lack  of  stability  in  the  business  processes,  the 
priority attached to bug fixing, a lack of skilled IT developers, resource constraints in the 
IT unit and non-optimal working methods for development and defects resolution. As a 
result,  the  initial  project  planning  was  unreliable  and,  according  to  DG  REGIO,  the 
cumulative  budget  consumption  in  2014  for  the  project  was  some  27%  higher  than 
forecasted.  The  IAS  notes  that,  in  view  of  the  problems  experienced  in  implementing 
WAVE, DG EMPL is currently examining the possibility of switching to an alternative 
system. 

Recommendations 

To address these issues, the IAS formulated recommendations which can be summarised 
as follows: 

Supervising MS management and control systems: 

The IAS recognises that the audit strategy is in the process of being updated and it is still 
relatively early in the programming period. However, the gaps identified as regards the 
audit  strategy  need  to  be  addressed  in  good  time,  to  ensure  sound  assurance  building 
processes for the DGs from the start. 

DGs REGIO and EMPL should therefore further develop/clarify their audit strategy with 
respect to how they will obtain assurance on the reliability of the Audit Authority (AA), 
including on the new tasks of the AA. The annual focus of the audit strategy should be 
complemented  with  a  multi-annual  assurance  building  approach  and  audits  planned  in 
such a way as to optimise the use of the retention period for documents. The DGs should 
plan  more  early  preventive  system  audits  to  be  able  to  legally  implement  corrective 
actions in the early phases of the programming period. 

OP negotiation and adoption process: 

The  adoption  of  OPs  may  be  further  delayed  and  not  all  observations  sufficiently 
addressed  in  a  consistent  manner  before  adoption  of  the  OP.  There  may  also  be 
inconsistencies and a lack of transparency in the assessment of EACs. DGs REGIO and 
EMPL should therefore carefully monitor the final phases before OP adoption, including 
the follow-up given to the Commission's observations. The DGs should also clarify the 
minimum requirements for documenting the DO's work and ensure that DOs assess and 
document the EAC at the OP level where relevant. 

Results orientation and performance framework: 

33 

 
 
DGs  REGIO  and  EMPL  should  ensure  that  DOs  actually  question  the  plausibility  of 
milestones/targets,  including  through  their  own  research  and  estimation  work,  and 
document this assessment to allow consistency checks to be made and provide a basis for 
assessing  the  reasonableness  of  future  OP  modifications  and  the  achievement  of 
milestones and targets. 

IT systems supporting the management of the PP 2014-2020 processes: 

Additional  delays  in  the  delivery  of  stable  WAVE  functionalities  could  increasingly 
impact on the ESIF DGs' management of the 2014-20 programming period. DGs REGIO 
and EMPL should ensure that business processes are defined and agreed in time for the 
development, ensure a stable project team including the necessary development capacity, 
develop  a  reliable  project  planning  and  monitoring  and  improve  IT  development 
methods and defect resolution in order to ensure a stable platform. 

The audited service has established an action plan which the IAS considers satisfactory 
to address the recommendations. 

4.3.  Audit on preparations for use of Financial Instruments in DG EMPL 

2014-20 

Background 

The  legal  framework  for  the  2014-20  programming  period  significantly  expands  the 
scope for using financial instruments in the Cohesion area in comparison to the 2007-13 
period.  Consequently,  an  audit  on  the  preparations  being  made  by  DG  EMPL  for  their 
use in 2014-20 was included in the 2013-2015 Strategic Audit Plan of the IAS. 

Audit Objectives 

The main objective of the audit was to assess the readiness of DG EMPL to monitor and 
supervise  the  financial  instruments  under  the  new  legal  framework  and  to  highlight  in 
advance  any  weaknesses  in  the  DG's  control  system,  which  could  jeopardise  the 
achievement  of  objectives  of  the  increased  use  of  financial  instruments  in  the  new 
Multiannual Financial Framework (MFF). 

Audit Scope 

The audit was limited to the financial instruments set up under the European Structural 
and  Investment  Funds  (ESIF).  However,  as  the  new  legal  basis  foresees  the  possibility 
that  ESIF  Operational  Programmes  (OPs)  may  contribute  to  EU  level  financial 
instruments, the advantages and limitations of this option were also reviewed by IAS. 

The  fieldwork  was  finalised  on  21/03/2014.  All  observations  and  recommendations 
relate to the situation as of that date. 

There were no reservations in the 2013 AAR of DG EMPL that relate to the area/process 
audited. 

34 

 
 
Strengths 

The  IAS  recognises  the  joint  efforts  made  by  ESIF  DGs  in  strengthening  the  legal 
framework  for  managing  financial  instruments  in  the  2014-20  programming  period, 
particularly  as  regards  the  introduction  of  compulsory  ex-ante  assessments  before 
setting-up a financial instrument and a phased payments system19. Also, a strengthening 
of  the  provisions  for  management  costs  and  fees  together  with  improved  reporting 
requirements. 

Major Audit Findings 

The IAS has identified the following very important issue: 

Building financial instruments related capacity (report finding N°1) 

The complexity of financial instruments requires extensive capacity building both at the 
level of the Commission and at that of Member States (MS). The capacity problem was 
recognised as a key  factor for  the  low take-up/incorrect  use  of  financial instruments in 
the  previous  programming  period  and  for  2014-20  the  financial  instruments  technical 
advisory  platform  (FI-TAP)  is  expected  to  play  a  key  role  in  this  regard.  However, 
following  the  late  adoption  of  the  legislative  framework,  and  the  delays  in  the 
negotiation  of  a  Financial  and  Administrative  Framework  Agreement  (FAFA)  between 
the Commission and the EIB, this has been significantly delayed and there are continuing 
doubts  as  to  the  likely  take-up  under  ESF.  In  addition,  there  is  a  need  to  significantly 
improve the awareness raising effort. 

Recommendation 

To address this issue, the IAS formulated the following recommendation. 

DG EMPL should: 

•  Given  the  delay  to  the  launch  of  the  FI-TAP,  contribute  to  the  preparation  of  technical 
fiches necessary for effective monitoring and control over the implementation of financial 
instruments, taking into account ESF specificities. 

•  Provide  training  for  both  geographical  desk  officers  and  auditors,  either  on  an  in-house 

basis and/or by informing about trainings offered by DG REGIO. 

•  Ensure  that  ESF  needs  are  addressed  by  the  final  FI-TAP  work  programme,  e.g.  ESF-
specific  awareness  raising  activities  to  be  included  in  the  FI-TAP  work  programme  in 
order to ensure timely and effective communication with various stakeholders. 

•  Depending  on  the  take  up  in  the  2014-20  period,  further  develop  in-house  knowledge 
and/or  cooperation  arrangements  with  DG  REGIO  staff  to  ensure  appropriate  audit  and 
control coverage. 

19  Article  41  of  the  CPR,  which  stipulates  that  subsequent  payments  can  only  be  made  once  a  predefined  percentage  of 

programme contribution previously paid to the financial instruments has been disbursed to the final recipients. 

35 

 
 
                                                            
The audited service has established an action plan which the IAS considers satisfactory 
to address the recommendation. 

4.4.  Audit on preparations for use of Financial Instruments in DG REGIO 

2014-20 

Background 

The  legal  framework  for  the  2014-20  programming  period  significantly  expands  the 
scope for using financial instruments in the Cohesion area in comparison to the 2007-13 
period. Consequently, an audit on the preparations being made by DG REGIO for their 
use in 2014-20 was included in IAS Strategic Audit Plan 2013-2015. 

Audit Objectives 

The main objective of the audit was to assess the readiness of DG REGIO to monitor and 
supervise  financial  instruments  under  the  new  legal  framework  and  to  highlight  in 
advance  any  weaknesses  in  the  DG's  control  systems  which  could  jeopardise  the 
achievement  of  objectives  of  the  increased  use  of  financial  instruments  in  the  new 
Multiannual Financial Framework (MFF). 

Audit Scope 

The audit was limited to the financial instruments set up under the European Structural 
and  Investment  Funds  (ESIF).  However,  as  the  new  legal  basis  foresees  the  possibility 
that  ESIF  Operational  Programmes  (OPs)  may  also  contribute  to  EU  level  financial 
instruments, the advantages and limitations of this option were also reviewed by IAS. 

The  fieldwork  was  finalised  on  21/03/2014.  All  observations  and  recommendations 
relate to the situation as of that date. 

Two reservations were made in the 2013 AAR of DG REGIO concerning, inter alia, the 
financial  instruments.  One  of  them  is  a  full  reputational  reservation  related  to  non-
eligibility of expenditure of the financial instrument (CZ OP 2007CZ161PO004) and the 
other one is a partial reservation due to deficiencies related to financial instrument (ES 
OP 2007ES16UPO001). 

Strengths 

The IAS notes that DG REGIO has a lead DG role as regards financial instruments under 
all  ESIF  and  has  developed  a  key  competence  in  administering  this  complex  and 
technically  challenging  form  of  assistance.  Furthermore,  the  IAS  recognises  the  efforts 
made  by  the  DG  in  strengthening  the  legal  framework  for  managing  financial 
instruments in the 2014-20 programming period, particularly as regards the introduction 
of compulsory ex-ante assessments before setting-up a financial instrument and a phased 

36 

 
 
payments  system20.  Also,  a  strengthening  of  the  provisions  for  management  costs  and 
fees together with improved reporting requirements. 

Major Audit Findings 

The IAS has identified the following two very important issues: 

Legal framework for financial instruments in 2014-20 (report finding N° 1): 

The IAS identified the following issues arising from the new legal provisions, which are 
open to interpretation and which can pose risks to their practical implementation: 

•  Within  the  limitations  foreseen  by  Article  37(4)  of  the  CPR,  support  to  enterprises  may 
include, inter alia, financing of working capital, but as regards the general activities of an 
enterprise and/or realisation of new projects this could be misinterpreted by Member States 
(MS),  which  could  in  turn  raise  the  question  as  to  whether  financial  instruments  will 
actually generate real investment as opposed to acting simply as a quick disbursement tool. 

•  The  leverage  definition  as  per  the  Financial  Regulation  (FR),  to  which  the  Common 
Provisions  Regulation  (CPR)  refers,  does  not  take  into  account  the  specificities  of  ESIF 
(i.e.  the  MS  co-financing  element  which  is  generally  applicable  for  any  cohesion  policy 
action). This means that the additional funding attributable to financial instruments would 
not be specifically assessed and reported. However, although the data submitted by MS in 
their  Annual  Implementation  Reports  (AIRs)  could  allow  the  net  (i.e.  excluding  MS  co-
financing) leverage effect to be assessed, DG REGIO currently has no plans to make this 
assessment and report the results. 

•  Preferential  treatment  for  private  investors  might  be  needed  in  order  to  attract  private 
investors  to  invest  in  areas  of  market  failure,  which  is  important  for  the  achievement  of 
public  policy  objectives  and  this  should  in  turn  be  properly  justified  at  the  ex-ante 
assessment  stage.  However,  although  the  Commission  has  a  supervisory  role,  it  does  not 
actually  approve  these  assessments  and  therefore  has  limited  opportunity  at  this  stage  to 
address the associated risks.  

•  Despite  setting  stronger  rules  overall  and  defining  maximum  thresholds  for  management 
costs and fees, the Delegated Act (C(2014) 1207) allows an exception, for example where 
competitive tender arrangements apply. In addition, there are clear differences between the 
arrangements for ESIF and EU-Level financial instruments, which could have a bearing on 
the market for financial intermediaries. 

Building financial instruments related capacity (report finding N° 2): 

The complexity of financial instruments requires extensive capacity building both at the 
level  of  the  Commission  and  at  that  of  MS.  A  key  element  to  this  is  the  financial 
instruments technical advisory platform (FI-TAP). However, following late adoption of 
the  legislative  framework  and  the  delays  in  the  negotiation  of  a  Financial  and 
Administrative  Framework  Agreement  (FAFA)  between  the  Commission  and  the  EIB, 

20 Article 41 of CPR which stipulates that subsequent payments can only be made once a predefined percentage 
of  programme  contribution  previously  paid  to  the  financial  instrument  has  been  disbursed  to  the  final 
recipients. 

37 

 
 
                                                            
this  has  been  significantly  delayed.  Although  certain  capacity  building  measures  have 
been undertaken, actions such as training and guidance still need to be further developed. 

Recommendations 

To address these issues, the IAS formulated recommendations which can be summarised 
as follows: 

•  Ensure  that  the  risks  related  to  issues  identified  in  the  legal  framework  are  adequately 
mitigated and properly reflected in its guidance to DG REGIO staff and MS as well as its 
audit and control strategy, adapted for the 2014-2020 programming period. Specifically, it 
should  develop  guidance  internally  and  to  MS  on  the  eligibility  of  working  capital  and 
preferential  treatment  of  private  investors  and  ensure  that  its  staff  clearly  understand  the 
key  elements  of  the  ex-ante  assessment  reports  used  to  justify  the  set-up  of  financial 
instruments  with  ESIF  funding.  It  should  also  issue  guidance  to  help  ensure  that  the 
leverage effect is properly measured and reported in the summary reports to the Parliament 
and the Council21.  

•  Closely  monitor  the  work  and  the  timeliness  of  preparatory  works  to  speed  up  the 
launching of the FI-TAP. It should ensure that the FI-TAP work programme is sufficiently 
flexible  to  meet  all  stakeholders'  needs.  Pending  its  launch,  it  should  develop  a  schedule 
for  the  drafting  of  technical  fiches  and  further  develop  training  opportunities  for  both 
geographical desk officers and auditors. 

The audited service has established an action plan which the IAS considers satisfactory 
to address the recommendations. 

4.5.  Limited Review of the calculation and the underlying methodology of 
DG REGIO's residual error rates for the 2013 Reporting Year  

Background 

The Mutual Expectation Paper on the AAR Peer Review Process foresees that the IAS 
may  undertake  limited  reviews  of  the  calculations  and  underlying  methodologies  of 
residual error rates of up to three DGs each year selected on a risk basis. These limited 
reviews  take  place,  and  are  reported  on  to  the  DGs  concerned,  before  the  AARs  are 
finalised. The IAS strategic audit plan for 2013-2015 includes for 2014 a limited review 
on the residual error rates reported by DG REGIO for the 2013 reporting year. A similar 
review was conducted in 2014 for DG CNECT. 

21 DG REGIO partially accepted this Very Important recommendation, in particular on developing guidance for 
Member States on calculating the achieved leverage effect of financial instruments (i.e. capacity to attract 
additional  public  and  private  funding)  to  show  the  net  effect  of  the  leverage  brought  about  by  the  use  of 
financial instruments and to ensure that this is properly reported in the annual summary reports on financial 
instruments'  implementation.  DG  REGIO  considered  that  removing  the  national  co-financing  from  the 
leverage  concept  would  artificially  lower  the  achieved  leverage  especially  in  cases  when  national  co-
financing  is  provided  by  private  investors.  According  to  the  IAS,  assessing  the  ability  of  financial 
instruments  to  attract  new  investment,  and  reporting  on  this,  would  be  entirely  in  line  with  the 
Commission's move towards a more performance-based culture, particularly given the increased importance 
of this funding mechanism in the 2014-20 period. 

38 

 
 
                                                            
Objectives 

The  objective  was  to  review  the  calculation  and  underlying  methodology  of  the 
Cumulative Residual Risk/Error Rate (CRR) reported by DG REGIO in its (draft) AAR 
2013 and so contribute to help mitigate the discharge risk enabling DG REGIO to take 
appropriate actions, if any, before their disclosure in the final AAR and Synthesis report. 

Scope 

The review covered the following aspects: 

•  methodology  for  the  calculation  of  the  CRRs  for  the  ERDF/CF  operational  programmes 
(OPs) based on the error rates reported by the Audit Authorities (AAs) and statements of 
withdrawals  and  recoveries  reported  by  the  Certifying  Authorities  (CAs)  (including 
reliability of the data reported by the MS); 

•  testing of the calculated CRRs; 

•  presentation  of  the  CRRs  in  the  draft  AAR  (including  compliance  with  the  Standing 

Instructions for the AAR 2013). 

The  IAS  reviewed  the  draft  AAR  transmitted  to  the  central  services  (SG/BUDG)  on 
28/02/2014  and  the  preliminary  CRR  calculations  up  to  that  date.  As  REGIO's  CRR 
calculation  tables  were  updated  on  an  on-going  basis  until  the  AAR  was  finalised,  all 
data reported in the draft AAR and reviewed by the IAS were provisional at the time. 

In its 2012 AAR and in the draft AAR 2013, DG REGIO included a reservation for the 
2007-2013 programming period on significant issues regarding the effective functioning 
of  management  and  control  systems  for  ERDF/Cohesion  Fund/IPA  on  a  number  of 
programmes in several member states. The reservations were based on a range of factors, 
including the individual CRR figures for the OPs concerned. 

The audit fieldwork was finalised on 7/03/2014. All observations and recommendations 
relate to the situation as of that date, but certain figures included in the report have been 
updated to reflect the final AAR. 

Strengths 

The  IAS  recognises  the  on-going  efforts  made  by  DG  REGIO  to  strengthen  the 
assurances obtained from the AAs’ work and to demonstrate through the CRR the multi-
annual  corrective  capacity  of  the  management  and  control  systems  put  in  place  by 
Member States (MSs) and the Commission. The IAS found no significant errors in the 
CRR calculation process and adequate supporting evidence was provided for all data and 
changes to data used for the calculation process. 

The  IAS  notes  in  particular  that  the  process  for  reviewing  the  Annual  Control  Reports 
(ACRs) and assessing the reliability  of  AA  error rates is mature, well documented and 
supported  by  internal  guidance  and  guidance  to  MS.  This  process,  which  has  to  take 
place  within  a  very  narrow  timeframe,  is  planned  well  in  advance  and  monitored  very 
closely. In general, the IAS found that procedures were applied as intended. It also notes 

39 

 
 
more generally the importance of DG REGIO’s overall audit and control strategy in the 
decision  making  process  for  reservations.  The  assessment  of  MS  management  and 
control systems is a complex mix of assurance building blocks and is the key driver in 
the process. 

Major Audit Findings 

The  review  made  findings  related  to  the  way  in  which  DG  REGIO  presents  key 
information  in  its  AAR  on  reservations  and  the  amounts  at  risk  (Very  Important),  the 
reliability  of  validated  error  rates  (Very  Important),  the  reliability  of  MS  figures  for 
withdrawals  and  recoveries  (Very  Important)  and  the  calculation  of  the  CRR  (Very 
Important). However, given that DG REGIO implemented most of the recommendations 
concerning  the  reservations  and  amounts  at  risk  before  finalising  its  AAR  2013,  this 
recommendation  is  downgraded  from  Very  Important  to  Important  only.  The  IAS  also 
made a finding relating to the underlying business process (Important). 

Recommendations 

To address these issues, the IAS formulated recommendations which can be summarised 
as follows: 

Reliability of validated error rates: 

For 2014 onwards, DG REGIO should: 

•  take the necessary measures with AAs to ensure that the additional instructions provided in 

its note of December 2013 are applied in practice; 

•  clarify the requirement for AAs to report detailed error rate calculation tables, including a 
minimum level of information on the results of all audits of operations, currently included 
in the draft implementing acts for the 2014 period, for example by presenting the data in a 
standard template. 

Reliability of withdrawals and recoveries: 

Going forward, the DG needs to carry out more systematic checks on the MS figures and 
ensure that its audit work delivers the necessary coverage. In particular, REGIO should 
address the risk of double-counting corrections both taken into account in the estimation 
of the amount at risk and in the reported withdrawals and recoveries. 

Also going forward, REGIO should assess how long it takes to effect pending recoveries 
and the amounts which are actually cashed in practice, before taking them into account 
for  the  calculation.  In  addition,  the  DG  should  assess  whether  formal  agreements  are 
actually deducted from the next payment claim in practice in order to confirm whether it 
is  prudent  to  take  them  into  account  or  whether  to  consider  them  simply  as  relevant 
information for the reservation making process. Finally, REGIO should exercise caution 
in taking account of MS reports on withdrawals and recoveries submitted in advance of 
the 31 March deadline, unless it has obtained reasonable assurance on the reliability of 
the data. 

40 

 
 
 
The DG should ensure that the implementing acts for the 2014-20 programming period 
address the need to improve the way in which MSs report withdrawals and recoveries, 
including their origin, in order to specifically address the risk of double-counting. 

Calculation basis of the Cumulative Residual Risk/Error Rate (CRR) 

For the 2013 AAR, DG REGIO should analyse for each OP whether it is valid to use the 
error  rate  relating  to  last  year's  expenditure  as  a  best  estimate  for  this  year  when 
calculating the CRR and amount at risk. It should apply alternative approaches (e.g. flat 
rate  estimates)  if  this  is not  the  case.  For  the  2014  AAR  onwards,  negative  figures  for 
individual OPs should not be carried forward into subsequent year's calculations. 

The audited service has established an action plan which the IAS considers satisfactory 
to address the recommendations. A recent follow-up showed that all remaining actions of 
the very important recommendations have been implemented for the 2014 AAR. 

5.  RESEARCH, ENERGY AND TRANSPORT 

5.1.  Gap Analysis Review of the legislation regarding Horizon 2020 – Multi 

DG (DG CNECT, DG ENER, DG MOVE, DG RTD) 

Background 

Horizon  2020  is  the  Union's  new  funding  programme  that  brings  together  all  existing 
Union  research  and  innovation  funding,  including  the  Framework  Programme  for 
Research,  the  innovation  related  activities  of  the  Competitiveness  and  Innovation 
Framework Programme and the European Institute of Innovation and Technology (EIT). 
The Horizon 2020 commitment appropriations of EUR 74 316.9 million for 2014-20 are 
directly managed by Commission Directorate-Generals (DGs), Executive Agencies and 
other implementing bodies. 

As part of the Strategic Audit Plan 2013-2015 IAS has carried out a gap analysis review 
of the legal framework establishing and governing Horizon 2020. This constitutes part of 
a  wider  review  of  the  design  and  preparations  being  made  by  the  DGs  and  services 
responsible for its implementation. 

The "Innovation and Investment" legislative package establishes five public-private, four 
public-public  research  partnerships  and  an  initiative  to  pool  research  and  innovation 
investments in Air Traffic Management in support of the Single European Sky. 

These two overall packages are referred to as "Horizon 2020 legislation" in the report. 

Objectives 

The main objective of this review was to analyse the Horizon 2020 legislation as decided 
by the co-legislators, compare it to the initial Commission proposal and highlight, for the 
most important areas, the additional risks the Commission is facing as a result of the co-
legislative process. This review takes into account of the stated goal of the Commission 
to adopt a simplified programme architecture and a single set of rules for participation, to 

41 

 
 
achieve  an  appropriate  balance  between  trust  and  controls  and  to  reduce  the 
administrative burden on both beneficiaries and the Commission. 

Scope 

The  review  covered  the  Horizon  2020  legislation  and  the  DGs  responsible  (DG  RTD, 
CNECT,  EAC,  ENTR,  ENER,  JRC  and  MOVE)  for  the  implementation  of  Horizon 
2020.  However,  recognising  its  lead  role  in  the  Research  Area,  DG  RTD  is  the  prime 
focus of this report. Annex 2 of the report provides the exhaustive list of legislative acts 
covered by the review. 

There  were  no  observations/reservations  in  the  DGs'  past  AARs  that  relate  to  the 
area/process reviewed. 

The  fieldwork  was  finalised  on  22/10/2014.  All  observations  and  recommendations 
relate to the situation as of that date. 

Major Audit Findings 

This  engagement  constituted  a  very  specific  review  of  the  legislation  underpinning 
H2020 and makes up only the first phase of a wider examination of the preparations for 
its implementation. The IAS welcomes the efforts made through this legislation to bring 
together  a  number  of  improvements  aimed  at  harmonizing  and  simplifying  the 
arrangements  governing  the  research  framework  programme.  It  also  recognises  the 
efforts  made  by  the  Commission's  services  during  the  negotiation  phase  to  protect  the 
Commission's interest, particularly in the face of very strong external political pressure. 
When compared to the Commission's original proposal, the final legislation has resulted 
in a compromise text, which is not too far from what the Commission originally set out 
to achieve. 

Nonetheless, the changes have resulted in a number of additional risks that will need to 
be  addressed  as  part  of  the  preparations  for  the  design  and  implementation  of  controls 
going forward. The IAS findings were as follows: 

Rules for Participation 

A single set of rules governs Horizon 2020 participation and the dissemination of results. 
These  include  a  simplified  cost  reimbursement  model  aimed  at  reducing  complexities, 
the degree of paperwork involved and the potential for financial errors. It also includes a 
reduction in the time allowed to award a grant. 

However, the proposed regulation for the Rules for Participation was subject to a heavy 
legislative  process  and  the  co-legislators  proposed  a  number  of  changes.  Whilst  the 
overall  intention  of  the  legislation  was  to  strengthen  the  simplification  effort,  the  end 
result is that there will in fact be additional pressure on the Commission services and that 
H2020  will  be  characterised  by  some  of  the  complexities  for  which  FP7  was  often 
criticised. For example, a further reduction in the time-to-grant means the Commission 
services  will  need  to  be  well  prepared  organisationally  and  procedurally.  Also,  key 
changes  to  the  simplified  cost  reimbursement  model,  notably  as  regards  large 
infrastructure  beneficiaries  being  able  to  claim  parts  of  actual  indirect  costs  instead  of 

42 

 
 
flat rates, are likely to increase the scale of ex-ante checks needed by the Commission. In 
addition,  the  new  possibility  for  non-profit  organisations  under  innovation  actions  to 
claim 100% funding of direct costs could distort competition if these bodies also carry 
out economic activities and the increase in the single flat rate for the reimbursement of 
indirect  costs  from  20  to  25%  could  in  fact  simply  raise  H2020  costs  overall  without 
actually increasing the number of projects financed. 

Programme architecture 

The Commission originally proposed to establish three major priorities or pillars under 
Horizon  2020,  namely  generating  excellent  science,  fostering  industrial  leadership  and 
tackling  societal  challenges.  The  legislative  process  introduced  two  more  specific 
objectives,  which  reflect  the  political  priorities  of  the  co-legislators.  A  number  of 
changes were made to the Small and Medium Enterprise (SME) instrument that aim to 
make  this  scheme  more  appealing  to  its  target  audience  and  to  increase  participation 
rates. However, this could impact on the overall error rate in so far as SMEs traditionally 
have a higher risk profile. Also, by specifically earmarking funds in this way could limit 
the Commission in being able to select other good quality proposals, regardless the status 
of the beneficiary. 

Control framework 

The  Commission's  proposal  to  reduce  the  audit  burden  on  beneficiaries,  through  a 
revised control strategy, with a balance between trust and control, was not significantly 
modified  during  the  negotiation  process,  except  with  regard  to  a  limiting  of  the  time 
available to carry out ex-post audits. This was reduced from four years to two years, after 
the payment  of  the  balance for a project. This reduction in the time window for audits 
could impact on the potential reach and effectiveness of ex-post controls as it reduces the 
possibility to extrapolate and "clean" the non-audit population. In addition, the scope for 
additional risk-based control work is necessarily limited if irregularities are detected only 
after this window has expired. 

Monitoring and Evaluation 

Although  the  Commission's  original  proposal  envisaged  the  quantification  of  key 
indicators  for  assessing  results  and  impacts  for  H2020  general  and  specific  objectives, 
the  adopted  legislation  maintained  the  indicators,  but  did  not  quantify  the  associated 
targets.  Furthermore,  the  adopted  legislation  did  not  define  the  indicators  to  cover  the 
two additional special objectives. By not embedding the target value of indicators in the 
primary  legislation  represents  a  missed  opportunity  on  the  part  of  the  co-legislators  to 
reinforce the move towards a more performance-based culture. 

Innovation and Investment Package 

The number of public-private partnerships has been extended under H2020 through the 
increased  use  of  Joint  Undertakings  (JUs),  with  a  view  to  strengthening  links  to  the 
industry.  Whereas  the  Commission  originally  proposed  that  the  discharge  for  the 
budgetary implementation of the EU contribution to the JUs should be given indirectly 
through  the  Commission  as  part  of  the  simplification  effort,  the  co-legislators  decided 
that  the  JUs  should  have  separate  discharge  arrangements.  Thus,  the  extent  to  which 

43 

 
 
simplification  is  achieved  in  practice  might  be  limited.  Furthermore,  this  could  also 
impact on the Commission's effective oversight of budgetary implementation, given that 
it  remains  the  authorising  officer  for  payments  made  through  the  JUs.  H2020  also 
includes  public-public  partnerships  with  Member  States  in  order  to  help  implement 
national research programmes. As regards these bodies, the co-legislators set minimum 
and maximum levels for the EU contribution in relation to the MS share. In practice, this 
could have the effect of limiting the leverage effect of the EU element. 

Recommendations 

Reflecting  the  fact  that  this  is  only  the  first  part  of  a  wider  review  of  the  design  of 
controls  and  preparations  for  implementing  H2020,  only  one  recommendation  of  a 
general nature has been formulated, which should be taken into account going forward in 
preparing for the period 2014-20. Where appropriate, any further recommendations will 
be more concrete, based on the detailed findings that would arise from further IAS work. 

At  this  stage  the  IAS  recommends  only  that  DG  RTD  and  other  DGs  and  Agencies 
responsible  for  implementing  Horizon  2020  take  proper  account  of  the  changes  to  the 
original Commission proposal mentioned above in their preparations going forward and 
are  expected  to  adapt  their  internal  organisation/processes  in  order  to  address  the  risks 
identified. This should include preparing any guidance/instructions needed to clarify the 
rules. This recommendation does not require that a specific action plan be produced. 

5.2.  Audit on the implementation of FP7 control systems (including 

Supervision of External Bodies) in DG CNECT 

Background 

DG  CONNECT  implements  EU  research  policy  and  supports  the  development  of  the 
European  Research  Area  mainly  through  the  Research  Framework  Programmes.  The 
Seventh Framework Programme (2007-2013) is being phased out, but still a large portion 
of the payments appropriations will be spent against cost claims until 2016-2017. 

A wide-ranging audit on the implementation of FP7 control systems in DG CONNECT 
was included in the IAS Strategic audit plan, but was postponed until 2014 due to other 
audit  priorities.  In  the  meantime,  this  area  has  been  largely  covered  by  both  DG 
CONNECT  IAC  and  the  ECA.  Therefore,  the  audit  scope  was  defined  based  on  an 
analysis  of  the  previous  audits.  The  audit  focused  on  the  aspects  that  have  not  been 
sufficiently  covered  and  are  still  very  relevant  in  these  last  years  of  implementation  of 
FP7 (i.e. implementation of ex-post control results, anti-fraud measures and supervision 
of external bodies). 

The  ex-post  control  activity  for  FP7  was  transferred  from  the  research  DGs  to  the 
Common  Support  Centre  (CSC)  as  from  1.01.2014.  Although  the  creation  of  the  CSC, 
hosted by DG RTD, is a response to the criticised fragmented approach in the Research 
family,  the  transfer  of  activities  may  pose  specific  difficulties  and  risks  in  terms  of 
delays, business continuity and performance. 

44 

 
 
 
DG CONNECT supervises two Joint Undertakings (ENIAC and ARTEMIS which will 
be merged into a single one, i.e. ECSEL), one body established under art. 185 TFEU22 
[Ambient Assisted Learning (AAL)] and two EU Agencies (ENISA and BEREC). 

There was one reservation in the 2013 DG CONNECT AAR concerning the rate of the 
residual errors with regard to the accuracy of cost claims in FP7. 

Audit Objectives 

The  objective  of  the  audit  was  to  assess  the  adequacy  and  effective  application  of  the 
internal control systems for the following processes: 

•  Implementation  of  the  ex-post  control  results,  i.e.  leading  to  financial  corrections  and 

sanctions; 

•  The  current  anti-fraud  measures  including  identifying  potential  good  practices  and 
benchmarking within the Research family, given that prevention and detection of fraud is a 
growing concern for the Commission; 

•  Transfer of the ex-post control activity to the CSC; 

•  Supervision of the external bodies, i.e. ENIAC, ARTEMIS, AAL, ENISA and BEREC. 

Audit Scope 

The audit addressed the following main key questions: 

•  Whether  financial  corrections  and  sanctions  are  effectively  and  efficiently  applied,  and 

whether they comply with the current rules; 

•  Whether DG CONNECT's Anti-fraud Strategy is effectively applied and whether potential 
good practices are shared with other Research family Services (in particular DG RTD); 

•  Whether DG CONNECT has taken adequate measures to ensure a smooth transition of its 
ex-post control activity to the CSC and improved performance across the Research family 
(e.g. efficiency gains); 

•  Whether DG CONNECT supervision on the external bodies has ensured compliance with 
the applicable rules, sound financial management and adequate monitoring and control of 
the  achievement  of  the  bodies'  corresponding  policy  objectives.  In  this  respect,  the  IAS 
took  into  account  the  relevant  work  carried  out  by  the  ECA,  DG  CONNECT's  IAC  and 
IAS Directorate A. 

Strengths 

The  IAS  acknowledges  the  fact  that  DG  CONNECT  was  the  first  to  develop  a 
comprehensive anti-fraud strategy in 2011, which was also used as a reference point and 

22 Public-Public Partnership. 

45 

 
 
 
                                                            
helped  to  trigger  the  development  of  anti-fraud  strategies  in  some  of  the  other 
Commission Research Services. In particular, noted the actions undertaken in the areas 
of training courses, awareness raising campaigns and the preparation of ex-post controls 
with intelligent data gathering (i.e. on the legal form, staff and financial situation of the 
selected entities, using a data-mining tool). 

Major Audit Findings 

The IAS has identified the following very important issue: 

Anti-fraud strategy (deterrent measures, detection of plagiarism and double funding) 
(report finding N° 1) 

The existing measures put in place by DG CONNECT to deter and detect fraud are only 
partially effective due to the following weaknesses: 

•  Jointly with DG RTD, DG CONNECT developed internal guidelines laying down that the 
imposition of a financial penalty "is to be seen as ultima ratio […] and should be envisaged 
only  in  those  cases  where  serious  breach  of  contractual  obligations  did  not  trigger  the 
application  of  liquidated  damages".  The  IAS  considers  that  these  guidelines  are  not 
sufficiently developed to ensure their effective practical implementation, and notes that the 
REA has recently decided to apply penalties in the established cases of fraud. In practice, 
DG  CONNECT  has  not  yet  applied  any  financial  penalties  since  the  beginning  of  FP7. 
Over the same period, it has applied only one administrative penalty (exclusion from EU 
funding). 

•  The tools and approach for effectively detecting double funding and plagiarism need to be 

improved. 

Recommendations 

To address this issue, the IAS formulated the following recommendation: 

•  While recognizing that DG RTD is the lead DG in the area of fraud detection following the 
transfer  of  the  ex-post  control  activities  to  the  CSC,  DG  CONNECT  should  take  the 
initiative  and  collaborate  with  DG  RTD  to  further  develop  the  existing  guidance 
concerning  the  implementation  of  financial and  administrative  penalties.  DG  CONNECT 
should  ensure  their  systematic  application  (as  foreseen  by  the  current  FR,  the  CAFS  and 
the contractual framework for  FP7  and Horizon2020), at least in the established cases of 
fraud. 

•  DG CONNECT should collaborate with DG RTD to ensure the availability of an effective 
and integrated IT tool aimed at detecting double funding and plagiarism that can be used 
across all Commission Research Services, striking the right balance between coverage of 
the  riskiest  projects  and  cost  of  controls.  DG  CONNECT  should  develop  the  relevant 
internal procedures to integrate anti-plagiarism detection into current practices. 

The audited service has established an action plan which the IAS considers satisfactory 
to address the recommendation. 

46 

 
 
5.3.  Audit on the implementation of FP7 control systems (including 

Supervision of External Bodies) in DG RTD 

Background 

A wide-ranging audit on the implementation of FP7 control systems had been included 
in  the  IAS  Strategic  audit  plan,  but  was  postponed  until  this  year  due  to  other  audit 
priorities. In the meantime, this area has been largely covered by both DG RTD IAC and 
the ECA. The audit therefore focussed on the aspects that have not yet been sufficiently 
covered and are still very relevant in these last years of implementation of FP7. 

Even though the Seventh Framework Programme (2007-2013) is being phased out, DG 
RTD will still be spending a large portion of the payments appropriations (€ 8 609.38 m) 
still remains to be used against cost claims during the next years. 

Audit Objectives 

The  objective  of  the  audit  was  to  assess  the  adequacy  and  effective  application  of  the 
internal control systems for the following processes: 

•  Monitoring and supervision by DG RTD of: 

a.  two Executive Agencies (REA and ERCEA); 

b.  four European Joint Undertakings (JUs), of which three Joint Technology 
Initiatives  (JTIs)  (Innovative  Medicines  Initiative,  Clean  Sky  and  Fuel 
Cells and Hydrogen) and the JU for ITER (Fusion for Energy-F4E); 

c.  three  bodies  established  under  article  185  TFEU  [BONUS  (Baltic  Sea), 
EMRP (European Metrology Research Programme) and EUROSTARS]. 

•  Prevention  and  detection  of  fraud,  as  it  is  a  growing  concern  for  the  Commission. 
Following  the  Commission's  Communication  on  Anti-fraud  Strategy  and  the  Research 
family Anti-fraud Strategy (RAFS), DG RTD updated its anti-fraud Strategy in July 2012. 
In  2011  the  IAS  performed  an  audit  on  the  control  strategy  for  on-the-spot  control  and 
fraud  prevention  and  detection  in  DG  RTD.  However,  the  implementation  of  DG  RTD's 
updated anti-fraud Strategy has not been audited yet. 

•  The  transition  to  the  Common  Audit  Service  in  the  Common  Support  Centre  (CSC), 
which as from 1 January 2014 is responsible for implementing the ex-post audit strategy 
for the FP7 legacy managed in-house. 

Audit Scope 

The audit addressed the following main key questions: 

•  whether  DG  RTD's  supervision  of  the  external  bodies  has  ensured  compliance  with  the 
applicable rules, sound financial management and adequate monitoring and control of the 
achievement of the bodies' corresponding policy objectives; 

47 

 
 
•  whether  DG  RTD's  Antifraud  Strategy  is  effectively  applied  in  practice  and  whether 

potential good practices are shared with other Research family Services; 

•  whether DG RTD has taken adequate measures to ensure a smooth transition to the CSC to 
implement  effectively  and  efficiently  the  ex-post  audit  strategy  related  to  the  FP7  legacy 
managed in-house. 

In its 2013 Annual Activity Report, DG RTD made a reservation concerning the residual 
error rate with regard to the accuracy of cost claims in the FP7, the representative error 
rate from the Common Representative audit Sample being 4.14%. The residual error rate 
calculated was between 2.88% and 2.99%. 

Strengths 

The IAS acknowledges that DG RTD has put in place a number of mechanisms ensuring 
overall effectiveness in the operations of its internal control system as follows: 

•  Arrangements for operational cooperation between DG RTD, the Agencies and JTIs have 
been  established  (e.g.  Memorandum  of  Understanding  (MoU),  Compendia  of  procedures 
for EA and JTI JUs). 

•  The annual 'Stocktaking reports' prepared during the past four years are a good practice as 
an internal management and monitoring tool which analyses the governance, interrelations, 
procedures, documents, supervision etc. concerning the External Bodies. 

•  DG RTD has strengthened considerably its ex-post audit function during the last years and 

made significant efforts to improve its processes and procedures. 

Major Audit Findings 

While acknowledging the steps already taken, the IAS concludes that the internal control 
systems  in  DG  RTD  for  the  processes  audited  need  to  be  reinforced,  in  particular  to 
better address the challenges of supervising JUs and article 185 TFEU bodies, and more 
effectively prevent and detect potential fraud. The IAS' assessment takes into account the 
large amount of FP7 payments appropriations still to be used against cost claims during 
the next years, as well as the need to start implementing Horizon 2020 in the 2014-2020 
MFF  in  the  best  conditions,  since  the  budget  entrusted  to  JUs  and  article  185  TFEU 
bodies  will  be  increased  and  the  fight  against  fraud  will  remain  a  priority  for  the 
Commission. 

The IAS has identified the following three very important issues: 

Supervision of the Joint Undertakings set up to implement Joint Technology initiatives 
(report finding N° 1) 

•  Without prejudice to the supervision and monitoring activities, DG RTD's responsible staff 
were not fully aware of the Commission's overall accountability with respect to the Joint 
Undertakings. 

48 

 
 
•  The  calculation  and  presentation  of  the  error  rate  in  both  the  final  AARs  2012  and 
provisional AARs 2013 (issued mid-February 2014) submitted by the three JTI JUs are not 
consistent, resulting in a non-appropriate  disclosure of the error rates in DG RTD's AAR. 
The legal basis for the JTI JUs sets out that their AARs should be delivered by the end of 
June.  The  delivery  of  the  AAR  by  the  JTIs  JUs  and  the  preparation  of  the  Directorate's 
contribution to the AAR of DG RTD are not synchronised, which does not ensure that DG 
RTD has the most up-to-date information to support the assurance in its own AAR. 

The transition to the Common Audit Service (report finding N° 2) 

•  The CAS Audit Plan was based on the assumption that 15 posts would be transferred from 
the research DGs, but these were not yet available as of May 2014. Additionally, given the 
audit backlog of some services at 31/12/2013, the CAS will have to make up for this over 
the years to come. 

•  As of May 2014, the CAS has identified the procedures at the operational level that are to 
be  harmonised  given  that  different  services  are  involved.  These  were  discussed  by  the 
Executive  Committee  and  there  is  a  request  to  harmonise  these  procedures  by  October 
2014. 

Anti-fraud measures (report finding N° 3) 

•  The  Common  anti-fraud  strategy  in  the  research  family  (RAFS)  does  not  include  any 
concrete  objective  with  related  KPIs,  all  services  should  achieve  in  a  coordinated  way 
through a concrete action plan with related deadlines. 

•  Although DG RTD had planned to implement its revised Anti-fraud Strategy of 2012 by 
December 2013, its services assessed four out of the 18 actions as not fully implemented at 
the  end  of  March  2014,  with  five  more  requiring  subsequent  actions.  Whilst  DG  RTD 
made efforts to implement the strategy, areas for improvement mainly concern prevention 
and  detection  of  scientific  and  professional  misconduct,  including  double  funding  and 
plagiarism, the application of financial and administrative penalties, and the use of proper 
anti-fraud  KPIs  and  related  monitoring  tools.  DG  RTD's  quantification  of  the  amount  at 
risk for the identified fraud cases, i.e. between 0.15% and 0.25% of the annual payments, is 
far below the international benchmarks on estimated losses due to fraudulent activities. 

Recommendations 

To address these issues, the IAS formulated recommendations which can be summarised 
as follows: 

•  Taking into account the Commission's responsibilities for the implementation of the budget 
entrusted  to  the  JTI  JUs,  ensure  staff's  full  awareness  about  the  Commission's 
accountability  in  this  area,  obtaining  from  the  JTI  JUs  the  most  complete  and  up-to-date 
information  for  the  purpose  of  its  own  AAR,  and  ensuring  that  it  has  consistent 
information from across the different JTI JUs on the calculation of the residual error rate 
and materiality criteria. 

•  Seek internal agreement related to the creation of the Common Support Centre clarifying 

roles, responsibilities, tasks and procedures. 

49 

 
 
Given the challenges faced by the CSC, especially in obtaining from other DGs in 
the  research  family,  agreement  on  the  transfer  of  adequate  staff  or  posts,  the  IAS 
will bring the issue of the resource gaps to the attention of the central services by 
issuing a management letter on the subject. 

•  In  coordination  with  the  other  Research  family  Services,  update  the  Research  family 
common  anti-fraud  strategy,  including  concrete  actions  to  improve  fraud  prevention  and 
detection  activities,  and  in  particular  address  the  risks  of  scientific  and  professional 
misconduct,  double  funding  and  plagiarism;  further  develop  and  implement  clear 
guidelines  on  the  application  of  financial  and  non-financial  sanctions  in  both  FP7  and 
Horizon  2020;  develop  a  set  of  KPIs  to  be  able  to  measure  the  performance  of  the  anti-
fraud activity, and a proper monitoring and reporting tool for the potential fraud cases. 

The audited service has established an action plan which the IAS considers satisfactory 
to address the recommendations. 

5.4.  Audit on the implementation of FP7 control systems in ERCEA 

Background 

The  European  Research  Council  (ERC)  was  set  up  in  2007  to  implement  the  IDEAS 
programme  under  the  Seventh  Framework  Programme  (FP7)  for  the  benefit  of  the 
scientific community in Europe by financing frontier research projects. The ERC aims to 
provide researchers with the means to conduct their research independently, by selecting 
and  funding  investigator-driven  research  ideas  based  on  initiatives  from  the  scientific 
community. Another goal of the ERC is to offer career prospects to the best European 
researchers and also to attract top scientists to Europe. 

The  ERC  Executive  Agency  (ERCEA)  is  the  dedicated  structure  responsible  for  the 
implementation  and  execution  of  the  programme.  It  supports  the  activities  of  the 
Scientific Council, the main decision making body of the ERC. The Agency is managed 
by a Director and a Steering Committee. The Commission is the guarantor of the ERC's 
full  independence.  The  parent  DG  of  ERCEA  is  DG  RTD,  which  nominates  three 
members  of  the  Steering  Committee  –  the  other  two  are  nominated  by  the  Scientific 
Council. 

As of 1 January 2014, the ERC is part of the first pillar – "Excellent Science" – of the 
new Horizon 2020 framework programme with a total budget of EUR 12.8 billion for the 
period 2014 - 2020. The implementation of the ex-post strategy of the new programme 
will  be  under  the  responsibility  of  the  newly  created  Common  Audit  Service  for  the 
research family as a whole. However, ERCEA is still responsible for the ex-post controls 
of its transactions under FP7. Although the FP7 covering the period 2007-2013 is being 
phased out, a large portion (around 50%) of the budgeted amount still remains to be used 
over  the  next  few  years,  with  the  volume  and  value  of  payments  under  the  IDEAS 
programme expected to peak between 2014 and 2016, and the last final payments to be 
made in 2021. The RAL as at 31 December 2013 was EUR 3 919.64 million. 

The major inherent risks in implementing the IDEAS programme relate to the selection 
and  control  of  research  projects  (the  evaluation  of  the  progress  and  results  of  the 

50 

 
 
projects) due to their lifetime (five years on average) and complexity. The beneficiaries 
are usually large, public sector entities (universities and national research centres) with a 
previous history of participation in other FP7 programmes and well-established control 
systems. Together with the simplification measures inherent to the IDEAS programme, 
ERCEA considers that this results in a comparatively lower level of risk for its activities. 

Audit Objectives 

The  main  objective  of  the  audit  was  to  assess  whether  ERCEA's  FP7  control  strategy 
was efficiently and effectively implemented and reported in its Annual Activity Report. 
In  addition,  the  IAS  examined  whether  ERCEA  ensures  that  corrective  measures  are 
taken  promptly  and  proportionately  in  order  to  obtain  an  acceptable  level  of  error  as 
regards the legality and regularity of transactions. 

Audit Scope 

This audit focused on the implementation of procedures (planning, execution and follow-
up) in the area of ex-post controls, the implementation of financial corrections resulting 
from  ex-post  audits  and  the  building  up  of  assurance  for  the  implementation  of  the 
operational budget. 

The  audit  scope  did  not  include  the  following  areas:  calls  for  proposals  and  related 
evaluation, grants payment activities, and ERCEA's anti-fraud strategy. 

There  were  no  observations/reservations  in  the  AAR  that  relate  to  the  area/process 
audited. 

fieldwork  was 

The 
recommendations relate to the situation as of that date. 

finalised  on  4  November  2014.  All  observations  and 

Strengths 

The IAS recognises ERCEA's efforts to put in place a sound internal control system in 
implementing  the  IDEAS  programme  delegated  by  the  European  Commission  and 
supervised by its parent DG. 

In particular, the IAS acknowledges the following strengths: 

•  The External Evaluation of ERCEA covering the first three years of the Agency's 
activities23
  is  globally  positive,  concluding  that  ERCEA  "has  been  efficient  and 
effective  in  performing  its  tasks"  (for  example  in  respect  of  good  cooperation 
between ERCEA and Commission services, simplifying its own procedures, good 
indicators  for  "Time  to  Grant"  and  "Time  to  Pay",  and  its  significant  reputation 
within the scientific community); 

•  ERCEA participated in various Research family working groups, in the Research 
family  awareness  campaign  on  most  common  FP7  errors,  and  organised  grant 
management events with beneficiaries. 

23 The report covers the period: 16 July 2009 – 15 July 2012. The final report is dated June 2013. 

51 

 
 
                                                            
Major Audit Findings 

The IAS has identified the following two very important issues: 

Building up of assurance (report finding N°. 1) 

According  to  ERCEA,  the  CRaS  for  FP7  as  a  whole  cannot  be  used  to  disclose  a 
representative  error  rate  for  its  IDEAS  programme.  However,  the  ex-post  audits  of  its 
own  MUS250  sample  of  transactions  is  not  yet  complete  (63%  completion  rate).  As  a 
result, ERCEA is not yet in a position to calculate and disclose a representative error rate 
for  its  activities.  The  disclosure  of  the  errors  detected  through  the  risk-based  audits  as 
"representative"  in  ERCEA's  2013  AAR  may  not  provide  an  accurate  picture  to  the 
reader. 

Ex-post control planning and execution (report finding N°. 2) 

The audit revealed weaknesses in the planning and execution of the audit plans, notably 
the  absence  of  a  comprehensive  plan,  including  KPIs,  the  output  of  the  audit  risk 
assessments  not  fully  reflected  in  the  final  list  of  audits,  and  considerable  delays  in 
conducting audits. 

Audit recommendations 

To address these issues, the IAS formulated recommendations as follows: 

•  disclose a residual error rate based on a statistically representative sample or, if it uses 
an  alternative  assessment  pattern,  to  refer  to  it  as  a  "detected"  rather  than 
"representative" error rate. 

•  develop a comprehensive audit strategy and audit plan, including relevant KPIs, and 

regularly review its risk parameters to reflect the specificities of ERCEA. 

The audited service has established an action plan which the IAS considers satisfactory 
to address the recommendations. 

5.5.  Audit on procurement management in DG JRC 

Background 

The Joint Research Centre (JRC) is the European Commission's in-house science service 
and  the  only  Commission  service  in  charge  of  direct  research.  It  is  composed  of  ten 
directorates including seven scientific institutes located in six sites in five countries. In 
2013,  it  employed  3  023  staff  including  2  051  scientific  staff  and  972  administrative 
staff,  of  which  63  staff  are  directly  involved  in  the  procurement  activity.  In  financial 
terms, the DG managed in 2013 commitment appropriations of € 515 m, of which € 128 
m represented operational expenditure and € 387 m administrative expenditure. 

Procurement  is  vital  to  the  core  business  of  the  JRC  to  provide  EU  policies  with 
independent,  evidence-based  scientific  and  technical  support  throughout  the  whole 

52 

 
 
policy  cycle.  Over  75%  of  its  annual  budget  (excluding  staff  related  expenditure)  is 
implemented  through  a  large  number  of  procurement  procedures  and  the  signature  of 
numerous contracts. In 2013, the JRC signed 170 large value contracts (i.e. above € 60 
000) for a  total of  € 140 m  and 7 000 orders or contracts with a value below € 60  000 
representing  an  estimated  total  of  € 29 m.   The  use  of  a  large  number  of  procurement 
procedures for the supply of goods and services in six sites across five different countries 
exposes  the  JRC  to  a  high  risk  not  only  in  terms  of  legality  and  regularity  but  also  in 
terms of sound financial management. 

This area was last audited by the IAS in 2009. 

Audit Objectives 

The objective of the audit was to assess whether the procurement process at the JRC is 
compliant with the procurement rules and whether the controls in place are effective. 

Audit Scope 

The  scope  of  the  audit  was  on  the  procedural  aspects  of  the  procurement  process  as 
follows: 

•  Needs analysis and planning, 

•  Contract  preparation  (compliance  with  public  procurement  rules,  selection  criteria, 

contractual arrangements on monitoring performance and payment arrangements), 

•  Contract execution (efficiency and effectiveness of control system focusing on the certified 

correct procedure, compliance with payment delays), 

•  Ex-post control strategy. 

The fieldwork was finalised on 22 October 2014. All observations and recommendations 
relate to the situation as of that date. 

There  were  no  observations/reservations  in  the  AAR  that  relate  to  the  area/process 
audited. 

Strengths 

Since 2012, the JRC has taken steps which have led to improvements in the procurement 
process  and  ensure  compliance  with  the  procurement  rules.  In  particular,  it  centralised 
specific  competences  in  Unit  JRC.B5  (Finance  and  Procurement)  with  a  clear  purpose 
and remit for ensuring effective support and assistance to the decentralised Procurement 
Units  with  a  view  to  harmonising  financial  and  procurement  procedures  and  processes 
across the JRC. 

In  addition,  the  IAS  welcomes  the  role  played  by  the  Public  Procurement  Advisory 
Group  (PPAG)  over  major  procurement  procedures.  The  PPAG  is  the  JRC's  ex-ante 
control body for public procurement contracts. Although its opinions are not binding, the 
sub-delegated  authorising  officers  must  take  them  into  account  when  awarding  a 

53 

 
 
contract.  Another key control function  identified by the IAS is  the JRC's extensive ex-
post controls performed on a yearly basis on payment and procurement files. 

The  JRC  has  also  developed  and  is  using  a  pre-awarding  back  office  system  called 
Public  Procurement  Management  Tool  (PPMT),  which  has  been  designated  as  a 
corporate system for the Commission, in the frame of the IT rationalisation initiative. 

Major Audit Findings 

The IAS has identified the following very important issue: 

Management of low value procurement (report finding N°. 1) 

The  audit  identified  weaknesses  in  a  high  proportion  of  the  eleven  low  value 
procurement files reviewed: 

•  In  64%  of  the  procedures  (7  files),  the  value  of  the  procurement  was  very  close  to  the 
threshold  or  the  procedures  consisted  of  different  contracts  for  recurrent  or  similar 
services/delivery  of  goods  with  a  total  value  which  was  above  the  threshold.  The  IAS 
acknowledges that timing and geographic differences may not enable the JRC to know in 
advance  procedures  that  may,  in  aggregate,  reach  the  threshold  and  therefore  launch  a 
wider  procedure  in  the  first  place.  This  may  however  raise  concerns  if  the  same 
goods/services  are  delivered  every  year  and  therefore  give  rise  to  a  perceived  risk  of 
splitting of contracts. 

•  In 55% of the procedures, the justification for the use of the negotiated procedure was not 

sufficiently justified or the quantity ordered was not accurately estimated. 

•  In 73% of the procedures (8 files), the order form that formally establishes the contractual 

relationship was not returned signed by the suppliers. 

Recommendation 

To address this issue, the IAS formulated the following recommendations: 

•  Review  its  strategy  for  low  value  purchases  following  an  analysis  of  the  expenditure 
profile  of  each  site  accompanied  by  actions  such  as  awareness-raising  targeting  the 
operational units on the criteria used for determining the choice of procurement procedure. 

•  Implement  specific  control  measures  and/or  awareness  raising  actions  to  follow-up  the 

financial issues detected during the testing phase of the audit. 

The audited service has established an action plan which the IAS considers satisfactory 
to address the recommendations. 

5.6.  Limited Review of the calculation and the underlying methodology of 
DG CNECT's residual error rate for the 2013 reporting year 

Background 

54 

 
 
The Mutual Expectations Paper on the AAR Peer Review Process, introduced in 2013 in 
view of the preparation of the 2012 AARs, foresees that the IAS may undertake limited 
reviews of the calculations and underlying methodologies of residual error rates (RER) 
of up to three DGs each year selected on a risk basis. These limited reviews take place 
and are reported to the DGs concerned before their AARs are finalised. Hence, the IAS 
strategic  audit  plan  for  2013-2015  includes,  for  2014,  a  limited  review  on  the  residual 
error rates reported by DG CNECT for the 2013 reporting year. 

Objective and scope 

The objective was to review the calculation and underlying methodology of the residual 
error  rate  reported  by  DG  CNECT  in  its  (draft)  AAR  2013  and  so  contribute  to  help 
mitigate  the  discharge  risk,  enabling  DG  CNECT  to  take  appropriate  actions,  if  any, 
before their disclosure in the final AAR and Synthesis report. 

The scope of the limited review was as follows: 

•  review of the methodology for the calculation of the residual error rate; 

•  testing of the calculation of the residual error rates; 

•  presentation  of  the  residual  error  rates  in  the  draft  AAR  (including  compliance  with  the 

Standing Instructions for the AAR 2013). 

In  its  2012  AAR  the  DG  made  a  reservation  concerning  the  rate  of  the  residual  errors 
with  regard  to  the  accuracy  of  cost  claims  in  the  7th  Framework  Programme's  grant 
agreements. 

The review was finalised on 11/03/2014. All observations and recommendations relate to 
the situation as of that date. 

Strengths 

DG CNECT follows the FP7 Audit strategy (2009-2016) that was formally adopted by 
the  Commission  Research  Services  in  October  2009.  DG  CNECT  bases  its  assurance 
regarding legality and regularity of expenditure to FP7 final beneficiaries on an error rate 
that  is  calculated  from  a  Common  Representative  audit  Sample  (CRaS).  The  CRaS 
reduces the audit burden on FP7 beneficiaries, by reducing the number of repeat audits. 

The IAS also notes the co-ordination measures in place in the Research family to ensure 
a common approach for auditing the CRaS, in particular: 

•  A common audit manual; 

•  The Audit Steering Committee for harmonizing audit results at DG level; 

•  The  Extrapolation  Steering  Committee,  which  decides  on  the  need  to  proceed  with 

extrapolation cases, if an audit detects a systematic error in cost statements; 

•  The Coordination Group for External Audits in Research Family (CAR); 

55 

 
 
•  The  Audits  Internal  Supervision  Committee,  which  examines  all  cases  with  major  errors 

(over 10% for the CRaS). 

Major Audit Findings 

Based  on  the  objectives  and  scope  mentioned  above,  no  material  risks  were  identified 
that would give rise to recommendations rated 'critical' or 'very important'. 

6.  ECONOMIC AND FINANCIAL AFFAIRS  

6.1.  Audit on risk management and planning processes in DG ECFIN in the 

New Economic Governance context 

Background 

DG  ECFIN  plays  a  central  role  in  the  design,  negotiation  and  implementation  of  the 
policy  responses  of  the  Commission  in  addressing  the  impact  of  the  global  financial 
crisis involving banking systems, stock markets, and the flow of credit. This crisis turned 
into a sovereign debt crisis with a subsequent crisis of confidence in the euro zone. 

Since 2008, the DG has grown significantly in terms of the number of staff (from 614 to 
816), its responsibilities and the complexity of the regulatory framework within which it 
works. The DG has also undergone three reorganisations. 

Audit Objectives 

As with all DGs in the Commission, DG ECFIN is governed by the Strategic Planning 
and Programming cycle, which includes setting objectives (general policy objectives and 
specific objectives in the context of Activity Based Budgeting) in the Management Plan 
(MP), and reporting on policy achievements in the Annual Activity Report (AAR). Risk 
management feeds into the performance management process through the identification, 
assessment and addressing of risks to the achievement of the DG's objectives. 

The overall objective of the audit was to assess if DG ECFIN has based its management, 
monitoring and reporting of its new responsibilities in economic governance on effective 
risk management and planning procedures. 

Audit Scope 

The audit focussed on the design and conduct of risk management and annual planning 
and monitoring processes and the practical implementation of these processes, including 
roles and responsibilities, with a focus on the directorates and units in charge of the new 
economic governance. 

There were no reservations in the 2013  Annual  Activity Report related to the scope of 
the audit. 

The  fieldwork  was  finalised  on  30/06/2014.  All  observations  and  recommendations 
relate to the situation as of that date. 

56 

 
 
Strengths 

Since  2008,  DG  ECFIN  was  required  to  work  under  very  difficult  circumstances,  with 
extreme  and  unforeseen  events  taking  place  in  the  European  and  Member  State 
economies. The demands placed upon the DG in terms of workload and the often very 
tight deadlines within which it had to deliver in a very important and high profile area 
have been significant. 

Though being exposed to these challenges, ECFIN management has ensured that the DG 
delivered consistently while faced with significant demands. This is in large part due to 
dedicated  hands-on  management  coupled  with  the  noteworthy  field  expertise  and 
dedication  of  its  staff,  including  regular  periods  of  long  hours  and  weekend  working, 
which could be symptomatic of resource shortages. 

In this period of challenge and change, the DG has had to ensure delivering all required 
outputs  while  simultaneously  designing  and  launching  new  surveillance  and  support 
structures.  In  meeting  its  objectives  the  DG  was  required  to  work  in  what  might  be 
termed “a fire-fighting mode”, where the importance of meeting a very large number of 
often  very  high  priority  tasks  in  a  short  time  frame  did  not  always  permit  the  use  of 
“standard” planning, monitoring, and risk processes. 

The IAS recognises that the DG has taken significant steps to ensure that its management 
and staff have been aware of their key responsibilities, and that these were met on time 
and  to  a  satisfactory  standard.  The  use  of  central  planning  tools  to  ensure  consistency 
and  coordinate  the  often-complex  demands  placed  on  the  DG,  complemented  by  a 
system  of  regular  meetings  between  managers  in  the  different  Directorates  responsible 
has ensured that the DG has planned and delivered its work in a period of acute crisis. 

The professionalism and 'esprit de corps' of the staff has also contributed significantly to 
the DG’s achievement of its objectives. 

Major Audit Findings 

Based  on  the  objectives  and  scope  mentioned  above,  no  material  risks  were  identified 
that would give rise to recommendations rated 'critical' or 'very important'. 

Overall,  the  audit  confirms  that,  in  a  context  of  economic  crisis  and  challenging 
constraints,  DG  ECFIN's  management,  monitoring  and  reporting  of 
its  new 
responsibilities  in  economic  governance  are  based  on  effective  elements  of  risk 
management and planning generally in line with the central services' guidelines.  

6.2.  Audit on DG MARKT's cooperation with the three Supervisory Bodies 

on Financial Services 

Background 

Following  the  outbreak  of  the  financial  crisis  in  2008,  the  stabilisation  of  financial 
markets became a priority and financial sector reform a crucial instrument to achieve it. 

57 

 
 
 
The  financial  crisis  highlighted  the  need  for  better  regulation  and  supervision  of  the 
financial  sector.  Three  European  Supervisory  Authorities  (ESAs)  –  the  European 
Banking Authority (EBA), the European Securities and Markets Authority (ESMA) and 
the  European  Insurance  and  Occupational  Pensions  Authority  (EIOPA)  –  were 
established on 1 January 2011 to replace the former EU's supervisory architecture. 

As one of the main strands of the IAS' 2013-2015 Strategic Audit Plan is to respond to 
the increased emphasis given to all aspects of performance (effectiveness, efficiency and 
economy), the IAS audit of DG MARKT's cooperation with the three supervisory bodies 
on financial services was included in the IAS 2014 Audit Work Programme.. 

Audit Objectives 

The  overall  objective  of  the  audit  engagement  was  to  assess  DG  MARKT's  current 
performance management framework to follow up on and monitor cooperation with the 
three ESAs on financial services and for receiving information and reporting on progress 
towards the achievement of the policy objectives for European financial supervision. 

Audit Scope 

The audit reviewed strategy, planning and policy setting processes (including setting of 
objectives  and  key  performance  indicators  for  the  follow-up  on  and  monitoring  of  the 
ESAs), the internal organisation and processes, resources allocation, quality assurance of 
the  information  received,  communication,  and  the  evaluation  of  the  organisation  and 
operations of the ESAs. 

The ESAs themselves and the European Central Bank (ECB) were out of the audit scope. 

There were no reservations in the 2013  Annual Activity  Report related to the scope of 
the audit. 

The  fieldwork  was  finalised  on  14/05/2014.  All  observations  and  recommendations 
relate to the situation as of that date. 

Strengths 

Taking into account that the three ESAs started their operations in January 2011 and still 
are in their start-up phase until the end of 2014, the auditors recognise the good results 
DG MARKT already achieved regarding the cooperation with the Authorities. 

The following major strengths were identified. 

1. Cooperation of DG MARKT with the ESAs 

•  Both DG MARKT and the ESAs consider their working relationship to be very good. 

•  The  Commission/DG  MARKT  is  involved  in  all  stages  of  the  work  of  the  ESAs  and  is 
represented in all meetings of the Management Board, the Board of Supervisors, Steering 
Groups and in other relevant technical meetings of the ESAs. 

58 

 
 
•  The  deadlines  of  the  information/draft  technical  standards  (TS)  to  be  submitted  by  the 
ESAs  to  DG  MARKT  are  monitored  by  certain  units  in  DG  MARKT  via  monitoring 
tables. 

•  The draft evaluation report as well as the accompanying staff working document comply 
with  the  majority  of  the  points  of  the  requirements  laid  down  in  art.  81  of  the  Founding 
Regulations and go in certain areas even beyond the legal requirements. 

2. Internal Communication/Coordination in DG MARKT regarding the ESAs 

•  A central coordination unit in DG MARKT (unit 02) coordinates and monitors – amongst 
other tasks – all DG MARKT activities with the ESAs. Furthermore, for all budgetary and 
administrative  matters,  DG  MARKT  has  designated  the  advisor  in  Directorate  A  to 
monitor the administration of the agencies and authorities under the responsibility of DG 
MARKT. 

•  Unit  02  has  recently  developed  guidelines  on  the  preparation  and  adoption  of  delegated 
and implementing acts in the area of financial services in order to harmonise and improve 
DG MARKT's way of working with the ESAs. 

•  DG  MARKT  staff  attending  meetings  with  the  ESAs,  prepare  mission  reports  right  after 
each  mission  that  are  distributed  to  all  staff  and  management  concerned  to  ensure  they 
have up-to-date information regarding the on-going projects. 

Major Audit Findings 

Based  on  the  objectives  and  scope  mentioned  above,  no  material  risks  were  identified 
that would give rise to recommendations rated 'critical' or 'very important'. 

Overall, the audit showed that the design and implementation of DG MARKT's current 
performance  management  framework  regarding the cooperation with the three ESAs is 
adequate,  both  for  following  up  on  their  activities  and  for  receiving  information  and 
reports on progress towards achievement of the policy objectives for European financial 
supervision. 

6.3.  Audit on performance measurement system in DG TAXUD Customs 

Activities 

Background 

Following  the  entry  into  force  of  the  Lisbon  Treaty,  the  European  Commission  is 
required to submit to the European Parliament and the Council a yearly evaluation report 
on the Union’s finances based on the results achieved (Article 318 report). In June 2011, 
the  central  services  of  the  Commission  reminded  operational  services  that  the  current 
financial  and  economic  crisis  and  severe  constraints  in  public  spending  has  shifted  the 
focus to achieving more with less and that post-2013 spending programmes should place 
a greater focus on performance measurement. 

59 

 
 
In  order  to  meet  stakeholders'  expectations,  one  of  the  objectives  of  the  IAS  in  its 
Strategic  Audit  Plan  for  the  period  2013-2015  is  to  carry  out  performance  audits 
focusing on the economy, efficiency and effectiveness of the use of resources. A number 
of  audits  have  been  planned  to  assess  the  performance  measurement  system  of  DGs, 
including the present on customs activities in DG TAXUD. 

The  functioning  of  the  Customs  Union  relies  on  a  close  cooperation  between  DG 
the 
TAXUD  and  national  administrations.  The  main 
implementation of the customs policy was the Customs 2013 programme till 2013 and is 
the  new  Customs  2020  programme24  from  2014  onwards,  which  will  contribute  to  the 
EU  2020  strategy  by  boosting  the  effectiveness  of  the  Member  States’  customs 
administrations’  operational  work  and  provide  economies  of  scale.  A  priority  for  2014 
will be to further progress towards e-Customs, a modern and paperless environment for 
customs and trade based on the Union Customs Code (UCC) adopted on 9 October 2013. 

to  support 

instruments 

Audit Objectives 

The main objective of the audit was to assess the extent to which DG TAXUD had an 
adequate  performance  measurement  framework  in  place  for  customs  activities  both  in 
terms of its day to day operational and administrative activities (internal) and in terms of 
the delivery of policy objectives (external). 

Audit Scope 

The scope of the audit included: 

•  a  horizontal  analysis  of  DG  TAXUD's  internal  processes  for  setting  objectives 
and key performance indicators (KPI) as well as related reporting and monitoring 
systems, and 

•  a review of the processes for setting objectives, indicators, monitoring, evaluation 

and performance reporting concerning customs activities. 

The  monitoring  of  Customs  Legislation  implementation  was  out  of  scope  since  it  was 
covered by an IAS audit in 2012. In addition, pure IT activities were also out of scope 
since an IAS audit was scheduled later this year. Some HR aspects were the subject of 
the IAS Performance audit on the efficiency and effectiveness of the planning stage of 
the  selection  process  completed  in  April  2014  and  covering  DG  TAXUD  among  other 
sampled DGs. 

There  were  no  observations/reservations  in  DG  TAXUD's  Annual  Activity  Report 
(AAR) that related to the area/process audited. 

The  fieldwork  was  finalised  on  01/12/2013.  All  observations  and  recommendations 
relate to the situation as of that date with the exception of the 2014 Management Plan, 
which was also taken into account. 

24 REGULATION (EU) No 1294/2013 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 11 
December 2013 establishing an action programme for customs in the European Union for the period 2014-
2020 (Customs 2020) and repealing Decision No 624/2007/EC (Customs 2013 programme) 

60 

 
 
                                                            
Strengths 

The auditors recognise the ongoing efforts made by DG TAXUD to improve performance 
measurement  in  the  area  of  customs  activities  in  the  context  of  the  Commission's  on-
going move towards a performance culture. Beyond the implementation of such standard 
processes  as  strategic  planning  and  programming,  risk  management  and  evaluation,  the 
IAS identified the following strengths: 

•  A Performance Measurement System at the level of the EU Customs Union was developed 
with  Member  States'  support  and  endorsed  in  December  2013.    After  a  pilot  phase,  it  is 
becoming operational in 2014 and it will be progressively fine-tuned. 

•  A  Performance  Measurement  Framework  for  the  monitoring  of  the  Customs  2020 
programmes'  implementation  is  currently  under  preparation.  The  recent  regulation 
establishing the programme includes the list of indicators to measure the objectives. 

•  The  Electronic  Customs  Multi-Annual  Strategic  Plan  (MASP)  is  a  management  and 
planning  tool  for  the  development  and  implementation  of  e-Customs,  including  its 
regulatory and operational aspects. This document is reviewed and updated regularly. 

•  The  Business  Process  Modelling  Approach  is  a  powerful  instrument  to  support  and 
improve the functioning of the Customs Union and facilitate the use of many IT systems. 

•  Development  of  tools  for  efficient  collaboration  of  DG  TAXUD  with  MS  and  between 
MS.  DG  TAXUD  has  built  a  number  of  pilot  tools  and  projects  (PICS:  Programme 
Information and Communication Space) which modernise the way programme participants 
collaborate and cooperate. 

Major Audit Findings 

The IAS has identified the following two very important issues: 

Performance measurement of Customs Committees and Groups (report finding No.1) 

The current system of 13 Customs Code Committee Sections that assist the Commission 
in  implementing  EU  customs  legislation  and  over  150  Groups  providing  advice  and 
expertise is complex and resource consuming for  both national  administrations and the 
Commission  (e.g.  interpretation,  translation,  travelling,  logistics,  etc.).  An  effective 
performance measurement system is lacking, especially for the numerous Project Groups 
to  assess  needs,  optimise  meetings,  monitor  outputs,  milestones  and  priorities  and  to 
provide  a  comprehensive  overview  of  the  resources  employed  (human,  financial  and 
logistics). 

Performance  measurement  of  DG  TAXUD  internal  activities  in  the  customs  area 
(report finding No.2) 

DG TAXUD's current performance measurement system does not allow its management 
to optimally plan, coordinate, assess, follow up and supervise the diverse and technical 
internal  activities  the  DG  relies  on.  For  example,  the  DG's  Management  Plan  does  not 
clearly  distinguish  its  own  specific  mission  and  objectives  from  those  of  the  Customs 
Union.  Therefore,  its  implementation  depends  also  on  factors  beyond  DG  TAXUD's 

61 

 
 
control.  DG  TAXUD  plans,  measures  and  reports  on  performance  only  partially  at  the 
operational  level.  Risk  management  is  not  proactive  enough  to  increase  the  chances  of 
achieving the objectives. Operational information and knowledge are not shared enough 
across the DG. 

Recommendations 

To address these issues, the IAS formulated recommendations which can be summarised 
as follows: 

DG TAXUD should: 

•  set up a more effective performance measurement system for Committees and Groups with 

clearer responsibilities, improved coordination and resource monitoring. 

•  improve  its  own  performance  measurement  system  by  using  more  effectively  the 
Management  Plan  and  risk  management  as  management  tools  and  by  strengthening 
internal communication. 

The audited service has established an action plan which the IAS considers satisfactory 
to address the recommendations. 

62 

 
 
7.  EXTERNAL AID, DEVELOPMENT AND ENLARGEMENT  

7.1.  Audit on contribution agreements with UN bodies and other 

International Organisations – DG DEVCO 

Background 

The Financial Regulation (FR) applicable to the General Budget of the European Union 
(EU)  and  the  European  Development  Fund  (EDF)  sets  out  the  various  methods  of 
implementation of the budget. Under the indirect management mode25, the Commission 
can  entrust  budget  implementation  tasks  to,  inter  alia,  International  Organisations 
whereby  projects/actions  may  be  implemented  through  contribution  agreements  (either 
as a single donor or with other donors, i.e. multi-donor actions). 

DG  DEVCO  implements  development  and  cooperation  aid  under  the  indirect 
management  mode  through  a  various  range  of  international  organisations  (hereafter 
International Organisations – IOs): United Nations (UN) bodies and entities, World Bank 
Group  entities,  other  international  or  regional  Organisations.  The  financial  and 
contractual aspects of the cooperation with the UN and the World Bank are contained in 
framework  agreements:  the  Financial  and  Administrative  Framework  Agreement 
(FAFA) for UN bodies and the Trust Funds and Cofinancing Framework Agreement for 
the  World  Bank.  This  allows  UN  bodies  and  the  other  International  Organisations  to 
implement EU development aid projects on behalf of the EU through funding decisions 
that  are  translated  into  one  or  more  contribution  agreements  (or  indirect  management 
delegation  agreements  –  IMDA26  –  as  from  1  January  2014)  under  the  indirect 
management  mode27.  In  2013,  total  payments  made  by  DG  DEVCO  under  the  indirect 
management mode with IOs amounted to €1,423.48 m 28, representing 21% of the DG's 
total payments for the year. 

This management mode creates challenges and risks for DG DEVCO in the achievement 
of  its  policy  objectives.  In  addition,  there  are  high  stakeholders'  expectations  for  the 
Commission to move to a new performance-based culture and to demonstrate value for 
money in the implementation of the EU budget. 

As  a  result,  IOs  wishing  to  implement  projects  with  EU  funds  under  the  indirect 
management  mode  are  subject  to  an  ex-ante  assessment  under  the  so-called  "pillar" 

25  Under  the  previous  Financial  Regulation,  there  were  similar  requirements  under  "joint  management"  with 

International Organisations. 

26  NB:  IMDA  only  partially  replaces  the  previous  standard  contribution  agreements  since  it  only  applies  to 
indirect management. Depending on the nature of the tasks entrusted, cooperation with IOs may now also 
take  the  form  of  grants  (for  which  specific  provisions  have  been  developed)  or  indeed  procurement 
contracts.  Also,  cooperation  with  IOs  may  also  include  contracts  signed  directly  between  the  partner 
countries  and  IOs  for  the  implementation  of  EU  funds  (as  it  was  the  case  under  the  former  Financial 
Regulation, under “decentralised” management mode). 

27 Joint management mode under the Financial Regulation in force until 31 December 2012. 
28 Ref. DG DEVCO's 2013 AAR (page 127). 

63 

 
 
 
                                                            
system29.  The  aim  is  to  assess  whether  the  organisation's  procedures  are  suitable  and 
offer  sufficient  guarantees  regarding  the  management  of  EU  funds.  DG  ECHO  also 
performs a pillar assessment of its partners implementing humanitarian aid. 

Audit Objectives 

The objective of this audit was to assess the efficiency and effectiveness of the processes 
and procedures in place in DG DEVCO to implement the development and cooperation 
aid actions through contribution agreements with international organisations. 

In particular, the audit assessed the following: 

•  The robustness of the methodology in place for the pillar assessment; 

•  The alignment of the operational strategy in place with the strategic objectives of 

DG DEVCO; 

•  The effectiveness  of the  monitoring  and reporting  processes of the international 

organisations; 

•  The efficiency of the coordination with DG ECHO. 

The IAS conducted in parallel an audit on the same topic in DG ECHO. 

Audit Scope 

The  audit  focused  on  the  following  issues  related  to  the  management  of  contribution 
agreements with UN Agencies and International Organisations: 

•  Review of the design and implementation of the pillar assessment process, 

•  Assessment  of  DG  DEVCO's  strategic  and  operational  planning:  objective 
setting,  key  performance  indicators,  controls  set  up  by  DG  DEVCO  at 
identification/formulation  stage  (identification  fiche,  before  signature  of  the 
contract,  review  of  the  detailed  explanations  and  rationale  for  the  decision  to 
work  with  an  IO),  alignment  of  actions  of  IOs  with  strategic  objectives  of  DG 
DEVCO,  definition  of  SMART30  objectives  monitored  through  RACER31 
indicators on outputs32 and impacts33, 

reports  submitted  by 

•  Monitoring and reporting: (i) project monitoring, (ii) review of the financial and 
(iii)  design, 
narrative 
implementation  and  follow-up  of  the  audit  plan  related  to  verification  missions 
with  international  organisations,  (iv)  general  reporting  based  on  information  in 
CRIS, 

international  organisations, 

29  Article  60(2)  of  the  Financial  Regulation  applicable  to  the  general  budget  of  the  European  Union.  The 
assessment consists of six elements (i) internal control, (ii) accounting system, (iii) external audit, (iv) rules 
and  procedures  (for  grants,  procurement  and  financial  instruments)  for  providing  financing  from  Union 
funds, (v) ex-post publication of information on recipients, and (vi) protection of personal data. 

30 Specific, Measurable, Achievable, Relevant, Timely. 
31 Relevant, Accepted, Credible, Easy and Robust. 
32 What is produced or accomplished with the resources allocated to an intervention. 
33  Impacts  are  the  long-term  socio-economic  consequences  that  can  be  observed  a  certain  period  after  the 

completion of an intervention. 

64 

 
 
                                                            
•  Coordination between DG DEVCO and DG ECHO. 

The audit did not cover: 

•  The  legality  and  regularity  of  individual  transactions  (i.e.  payments  or  cost 
eligibility issues) related to the implementation of the contribution agreements34; 

•  The  contracts  awarded  to  International  Organisations  under  direct  management 
mode, i.e. grant or procurement contracts where the International Organisation is 
the beneficiary following a call for proposal or a call for tender or as an exception 
to the call for proposal (under article 190 of the Financial Regulation). 

The  following  IOs  were  included  in  the  scope  of  the  audit:  UN  bodies,  entities  and 
agencies, World Bank Group entities and other international/regional organisations. 

In  its  2012  and  2013  Annual  Activity  Reports  (AAR),  DG  DEVCO  made  a  global 
reservation  based  on  a  most  likely  estimate  of  the  representative  residual  error  rate  of 
3.63%  and  3.35%  respectively,  due  to  the  significant  occurrence  of  legality  and 
regularity  errors  in  the  underlying  transactions,  with  International  Organisations 
accounting for an important share of the errors identified "in two areas — budget support 
and  EU  contributions  to  multi-donor  projects  carried  out  by  international  organisations 
such as the UN — the nature of the instruments and payment conditions limit the extent 
to which transactions are prone to errors as defined in the Court’s audit of regularity"35. 
source:  ECA,  2012  Annual  Report  on  the  activities  funded  by  the  8th,  9th  and  10th 
European  Development  Funds  (EDFs),  paragraph  7  and  2012  Annual  Report  on  the 
implementation of the EU budget, paragraph 7.6. 

The fieldwork was finalised on 27 March 2014. All observations and recommendations 
relate to the situation as of that date. 

Major Audit Findings  

Based  on  the  objectives  and  scope  mentioned  above,  no  material  risks  were  identified 
that would give rise to recommendations rated 'critical' or 'very important'.36 

34 These are covered through the audit of the design and implementation of the different control layers set up by 
DG  DEVCO,  notably  the  verification  missions.  In  addition,  the  legality  and  regularity  of  transactions  is 
audited by the European Court of Auditors.  

35 Source:  ECA, 2012 Annual Report on the activities funded by the 8th, 9th and 10th European Development 
Funds (EDFs), paragraph 7 and 2012 Annual Report on the implementation of the EU budget, paragraph 
7.6. 

36 In a recommendation rated 'important' the IAS recommended DG DEVCO to take appropriate steps to enable 
it  to  assess  the  achievement  of  projects'  objectives  and  performance  targets  by  developing  and 
disseminating  best  practices  to  Headquarters  units  and  EU  Delegations  regarding  project  monitoring  and 
reporting, such as reporting examples and guidance on budget. DG DEVCO rejected this recommendation 
as they consider that it would not be in line with the financial rules applicable to indirect management, by 
which  the  international  organisations  decide  on  their  reporting  and  budget  templates.  The  IAS  drew  the 
DG's  attention  to  the  fact  that  this  recommendation  did  not  relate  to  the  legal  framework  but  to  a 
management issue. 

65 

 
 
                                                            
The IAS concluded that DG DEVCO has taken appropriate steps since the adoption of 
the  new  Financial  Regulation  in  order  to  adapt  its  internal  control  layers  to  these  new 
requirements.  

7.2.  Audit on contribution agreements with international organisations - 

DG ECHO  

Background 

The mandate of the Directorate-General for Humanitarian Aid and Civil Protection (DG 
ECHO)  encompasses  humanitarian  assistance  and  civil  protection,  the  two  main 
instruments  at  the  disposal  of  the  European  Union  to  ensure  a  rapid  and  effective 
delivery  of  EU  relief  assistance  to  people  faced  with  the  immediate  consequences  of 
disasters. DG ECHO does not implement assistance programmes itself37. DG ECHO is a 
donor and implements its mission by funding Community humanitarian actions through 
its partners. 

DG  ECHO  works  with  a  range  of  organisations  (UN  bodies,  other  International 
Organisations38, Non-Governmental Organisations (NGOs), and specialised agencies of 
Member  States)  to  support  its  humanitarian  objectives.  The  financial  and  contractual 
aspects of this cooperation are contained in the Financial and Administrative Framework 
Agreement  (FAFA)  for  UN  bodies  and  in  the  Framework  Partnership  Agreement  for 
other  International  Organisations  (FPA  IO).  This  allows  UN  bodies  and  the  other 
International Organisations to implement EC humanitarian aid projects on behalf of the 
EU  through  funding  decisions  that  are  translated  into  one  or  more  contribution 
agreements under the indirect management mode39. The use of these organisations offers 
a number of benefits, such as local knowledge and proximity and their ability to operate 
in  areas  prone  to  political,  security  or  access  issues.  This  management  mode  however 
creates challenges and risks for DG ECHO in the achievement of its policy objectives. In 
addition, there are high stakeholders' expectations for the Commission to move to a new 
performance-based culture and to demonstrate value for money in the implementation of 
the  EU  budget.  In  2012,  funds  for  humanitarian  operations  committed  by  DG  ECHO 
under  the  indirect  management  mode  amounted  to  €597,2m,  representing  46%  of  the 
DG's total commitments for the year, and implemented by its ten main partners40. 

DG  ECHO's  partners  are  subjected  to  an  ex-ante  assessment  of  their  project 
implementation  capacity  under  the  so-called  "pillar"  system41.  The  aim  is  to  assess 
whether  the  organisation's  procedures  are  suitable  and  offer  sufficient  guarantees 
regarding  the  management  of  EC  funds.  Following  the  entry  into  force  of  the  new 

37 With the exception of the provision of the 'ECHO flight' service which aims to open up remote areas to humanitarian aid 

agencies. 

38  The  three  International  Organisations  are  the  International  Committee  of  the  Red  Cross  (ICRC),  the  International 
Federation  of  the  Red  Cross  and  Red  Crescent  Societies  (IFRC),  and  the  International  Organisation  for  Migration 
(IOM). 

39 Joint management mode under the Financial Regulation in force until 31 December 2012. 
40 Seven UN bodies and three International Organisations. 
41 Article 60(2) of the Financial Regulation applicable to the general budget of the European Union. The assessment consists 
of  six  elements  (i)  internal  control,  (ii)  accounting  system,  (iii)  external  audit,  (iv)  rules  and  procedures  (for  grants, 
procurement  and  financial  instruments)  for  providing  financing  from  Union  funds,  (v)  ex-post  publication  of 
information on recipients, and (vi) protection of personal data. 

66 

 
 
                                                            
Financial  Regulation  (FR)  on  1  January  2013,  all  UN  bodies  and  International 
Organisations (hereafter International Organisations – IOs) must be re-assessed in order 
to be eligible for funding under the indirect management mode. 

In the Commission, DG DEVCO is the chef-de-file for all DGs in certain areas related to 
external actions. It provides guidance and instructions as well as leads the negotiations 
with  international  partners,  participates  in  the  FAFA  working  group,  and  negotiates 
framework agreements with UN bodies or other international organisations. DG DEVCO 
has developed the pillar assessment methodology and performs a pillar assessment of its 
partners  implementing  development  aid.  DG  ECHO  negotiates  its  own  framework 
partnership agreements with the ICRC, IFRC and IOM. 

Audit Objectives 

The objective of the audit was to assess the efficiency and effectiveness of the processes 
and  procedures  in  place  in  DG  ECHO  to  implement  humanitarian  aid  actions 
administered through contribution agreements with international organisations. 

In particular, the audit assessed the following: 

•  The robustness of the methodology in place for the pillar assessment; 

•  The alignment of the operational strategy in place with the strategic objectives of 

DG ECHO; 

•  The  effectiveness  of  the  monitoring  and  reporting  processes  of  the  actions  by 

international organisations; 

•  The efficiency of the coordination with DG DEVCO. 

Audit Scope 

The  audit  focused  on  the  following  issues  related  to  the  management  of  contribution 
agreements with UN Agencies and International Organisations: 

•  Review of the design and implementation of the pillar assessment process, 

indicators, 

•  Assessment of DG ECHO's strategic and operational planning: objective setting, 
key  performance 
at 
controls 
identification/formulation  stage  (identification  fiche,  before  signature  of  the 
contract,  review  of  the  detailed  explanations  and  rationale  for  the  decision  to 
work  with  an  IO),  alignment  of  actions  of  IOs  with  strategic  objectives  of  DG 
ECHO,  definition  of  SMART42  objectives  monitored  through  RACER43 
indicators on outputs44 and impacts45, 

set  up  by  DG  ECHO 

•  Monitoring and reporting: (i) project monitoring, (ii) review of the financial and 
(iii)  design, 

international  organisations, 

reports  submitted  by 

narrative 

42 Specific, Measurable, Achievable, Relevant, Timely. 
43 Relevant, Accepted, Credible, Easy and Robust. 
44 What is produced or accomplished with the resources allocated to an intervention. 
45 Impacts are the long-term socio-economic consequences that can be observed a certain period after the completion of an 

intervention. 

67 

 
 
                                                            
implementation  and  follow-up  of  the  audit  plan  related  to  verification  missions 
with  international  organisations,  (iv)  general  reporting  based  on  information  in 
HOPE, and actions funded through EDF as reported in CRIS (financial), 

•  Coordination between DG ECHO and DG DEVCO. 

The  following  international  organisations  were  included  in  the  scope  of  the  audit:  UN 
bodies,  entities  and  agencies,  and  Other  International  organisations.  The  audit  did  not 
cover  the  management  of  humanitarian  aid  implemented  under  the  Direct  Centralised 
Management mode with NGOs or the actions implemented by the specialised agencies of 
Member States. 

There were no observations/reservations in the 2013 AAR that relate to the area/process 
audited. 

The  fieldwork  was  finalised  on  7  April  2014.  All  observations  and  recommendations 
relate to the situation as of that date. 

Strengths 

UN bodies and International Organisations play a key role in implementing humanitarian 
aid  projects  and  thereby  contributing  to  the  achievement  of  DG  ECHO's  humanitarian 
objectives.  The  IAS  recognises  the  efforts  made  by  DG  ECHO  (together  with  DG 
DEVCO) in enhancing the earlier pillar assessment methodology of IOs by moving away 
from a desk review focused primarily on the design of the organisations' internal controls 
to  a  more  explicit  focus  on  their  effective  implementation  through  a  walkthrough  of  a 
limited  number  of  transactions  by  process.  This  has  enabled  the  DG  to  obtain  a  better 
understanding of its partners' administrative systems and procedures. 

In  terms  of  needs  analysis,  the  IAS  notes  that  DG  ECHO  has  a  worldwide  network  of 
field offices that is closely involved in the day-to-day follow-up of the DG's projects and 
provide up to date analysis of the humanitarian needs for a given country or region. This 
knowledge  is  effectively  used  by  DG  ECHO  in  developing  its  intervention  strategies, 
namely  through  its  Annual  Operational  Strategy  and  Humanitarian  Intervention  Plan 
(HIP). In recent years, DG ECHO has considerably improved the planning and decision-
making  processes  of  its  projects  by  introducing  a  series  of  internal  management  tools 
such as the HIP, an Integrated Analytical Framework (IAF) and a Project Dashboard. In 
addition,  as  from  2011,  DG  ECHO  has  streamlined  and  rationalised  its  financing 
decision-making  process  by  grouping  the  previous  country  decisions,  global  plans  and 
ad-hoc decisions into a single worldwide decision (WWD). 

The  IAS  also  notes  that  the  objectives  of  humanitarian  aid  as  set  out  in  the  HIP  are 
clearly  formulated,  with  a  clear  link  between  the  operational  objectives  and  those 
outlined in the individual projects. The use of the Single Form46 and logical framework47 
as project development tools enable DG ECHO's partners to better plan their projects as 

46  The  Single  Form  (Annex  1  of  the  Contribution  Agreement)  is  used  by  DG  ECHO's  partners  to  submit  their  Action 

proposal for funding. 

47  The  Single  Form  includes  a  logical  framework  template,  which  contains  the  project  objectives,  objectively  verifiable 

indicators, sources of verification as well as risk and assumptions at the project level. 

68 

 
 
                                                            
well  as  provide  a  good  framework  for  DG  ECHO  to  assess  the  proposals  and 
subsequently monitor the progress of the projects. 

Major Audit Findings 

The IAS has identified the following three very important issues: 

Monitoring framework (report finding N° 1) 

•  The IAS notes that DG ECHO implements its projects largely with the same partners. In 
accordance with its monitoring strategy, all projects are the subject of a monitoring visit at 
least once during their life cycle. As a result, pre-defined risk criteria which may render the 
process  more  cost-effective  are  not  used  for  the  selection  of  projects  to  be  visited.  In 
addition,  the  monitoring  visits  do  not  particularly  focus  on  an  assessment  of  the  level  of 
achievement of objectives against set benchmarks or targets. 

•  The pillar assessment methodology does not currently enable DG ECHO to assess upfront 
its partners' ability to monitor and report against set objectives, key performance indicators 
and  targets  and  to  provide  reliable  data  for  performance  monitoring  purposes.  While  the 
FR  requires  the  achievement  of  value  for  money  (principle  of  sound  financial 
management), which the Commission is placing a greater focus on, the pillar assessment 
methodology does not specifically address this point. The setting, monitoring and reporting 
against  well-selected  performance  measures  is  the  foundation  of  a  good  performance 
measurement  system  which  not  only  determines  what  needs  to  be  done  but  also  sets  the 
conditions for success. 

Reporting (report finding N°2) 

Although implementation issues were reported by the project teams in the majority of the 
projects included in the audit sample48, there is no clear documented audit trail justifying 
the approval of the final payment and corrective actions taken by the partner. 

Verification strategy (report finding N°3) 

The audit revealed that there is no consolidated verification strategy (i.e. HQ and Field 
verifications)  in  place  which  considers  how  other  assurance  and  monitoring  activities 
(e.g. pillar assessments and monitoring visits) complement this activity in order to strike 
an optimal balance between benefits and costs. 

Recommendations 

To address these issues, the IAS formulated recommendations which can be summarised 
as follows: 

DG ECHO should: 

•  Further develop its monitoring framework and take appropriate steps to evaluate the ability 
of its partners to monitor and report on the achievement of their objectives and results. 

48 Eleven projects (55%). 

69 

 
 
                                                            
•  Develop an overall monitoring framework that takes into account the most common reason 
for project failures and take due account when making the final payment of projects that 
have not fully or only partly achieved their objectives. 

•  Consolidate its verification strategy to include objectives and targets and address the cost 

benefit of controls. 

The audited service has established an action plan which the IAS considers satisfactory 
to address the recommendations. 

7.3.  Audit on the assurance building process in EU Delegations – DG 

DEVCO 

Background 

The  audit  on  DG  DEVCO  –  Assurance  Building  Process  in  EU  Delegations  was 
included  in  the  IAS  2014  Audit  Work  Programme  following  the  audit  risk  assessment 
carried out in 2013 as part of the preparation of the IAS's Strategic Audit Plan for 2013-
2015. 

DG  DEVCO  operates  in  a  highly  risky  and  complex  environment.  Its  activities  are 
geographically  dispersed,  with  a  high  level  of  risk  associated  with  the  "developing 
country"  context  and  consist  of  a  diversity  of  implementing  organisations  and  partner 
countries, aid delivery methods and management modes. The 2013 payments amounted 
to  €6,962  million  (€3,768  million  under  the  EU  budget  and  €3,194  million  under  the 
European Development Fund  -  EDF),  implemented  mostly  in  a  devolved  manner,  with 
around 75-80% of the resources managed in the EU delegations. 

In  2010,  DG  DEVCO  launched  a  major  revision  of  its  internal  control  architecture 
(Control  Pyramid  strategy).  This  initiative  resulted  in  a  number  of  specific  actions 
including  a  new  web-based  reporting  tool  (External  Assistance  Management  Reports  - 
EAMR) which is considered to be the foundation of DG DEVCO's control pyramid. The 
EAMR is both a management tool to report on progress made and issues related to the 
implementation  of  EU  development  and  cooperation  aid  and  an  accountability  tool, 
which  includes  a  statement  of  assurance  signed  by  the  Head  of  Delegation  on  the 
management of funds sub-delegated to him by DG DEVCO's Geographic Directors. 

The  HoDs  are  staff  members  of  the  European  External  Action  Service  (EEAS)  but, 
according  to  the  Financial  Regulation,  when  they  act  as  sub-delegated  authorising 
officers  (SDAOs)  of  the  Commission,  they  apply  the  Commission's  rules  for  the 
implementation  of  the  funds  and  are  subject  to  the  same  duties,  obligations  and 
accountability as any other SDAO of the Commission. 

Audit Objectives 

The  objective  of  the  audit  was  to  assess  the  adequacy  and  effective  application  of  the 
internal  control  system,  risk  management  and  governance  processes  related  to  the 
assurance building process within EU Delegations. 

70 

 
 
Audit Scope 

The audit focused on the following elements related to the assurance building process in 
EU Delegations: 

•  A review of the design and definition of DG DEVCO's KPIs used for the 2013 
EAMR  exercise  (notably  to  assess  to  which  extent  their  definition/calculation 
reflects the reality of operations and activities in EUDs); 

•  A  review  of  the  EUDs'  2013  results  compared  to  their  objectives  as  set  out  in 

their annual management plan; 

•  A  review  of  the  set-up  of  the  internal  organisation  within  the  EUD  for  the 
assurance  building  process,  i.e.  the  continuous  monitoring  and  reporting  of  the 
EUDs' internal control system and results of key controls(not only limited to the 
EAMR process); 

•  A review of the content of the EAMR (narrative vs. financial data); 

•  A review of the accuracy and reliability of the results of the KPIs as defined by 
DG DEVCO HQ and implemented by EUDs, based on a sample of KPIs (out of 
26). 

The audit did not cover: 

•  Transaction/payment checks (as these are already covered by the European Court 
of Auditors (ECA), and DG DEVCO's Residual Error Rate (RER) study, audits 
and verifications of expenditure). 

•  The EUDs' internal control environment outside the assurance building process. 

•  The  reliability  of  data  in  the  Common  Relex  Information  System  (CRIS)  as 
systemic issues and weaknesses related to the system have already been raised in 
previous reports (e.g. Special Report no. 5/2012 of the ECA). 

The fieldwork was finalised on 3 October 2014. All observations and recommendations 
relate to the situation as of that date. 

Strengths 

One of the fundamental elements of the assurance-building process in DG DEVCO is the 
assurance provided by Heads of Delegations in their EAMRs that the resources assigned 
to the activities of the EUD have been used for their intended purpose and in accordance 
with the principles of sound financial management. A good assurance confirms whether 
the internal control system in place is operating effectively by identifying and helping to 
mitigate any risks to an efficient and effective implementation of development aid. 

In 2013, DG DEVCO undertook a major revamp of the design of its EAMR (applicable 
for  the  first  time  to  the  2013  reporting  year).  It  resulted  in  a  streamlined  structure,  the 
definition and integration of Key Performance Indicators (KPIs) in the report (with the 

71 

 
 
calculation  of  the  actual  KPIs  fully  or  partially  automated),  the  use  of  DG-wide 
benchmarks in the EAMR and the use of a traffic lights system (based on an assessment 
of the EUD's results against the benchmarks), thus enabling a comparison among the EU 
Delegations and across geographical directorates in DG DEVCO. 

In addition, the IAS notes that the review of the EAMR template and of the design of the 
KPIs (definition and benchmarks) is an on-going process with lessons still being learnt. 
On  24  July  2014,  DG  DEVCO's  management  approved  a  set  of  partially  revised  KPIs 
and benchmarks and a new EAMR template for the 2014 reporting year. 

Major Audit Findings 

The IAS has identified the following very important issue: 

Management  representation  and  reservations,  Statement  of  Assurance  by  Heads  of 
Delegation (report finding no.1) 

There is no clear guidance given on where and how a reservation should be expressed by 
an EUD with DG DEVCO's instructions stating that a reservation should be included in 
the  text  of  the  declaration  of  assurance  and/or  the  dedicated  annex.  As  a  result,  it  is 
unclear if comments added after a positive declaration should be treated as a reservation 
and  therefore  be  accompanied  by  an  estimate  of  the  potential  financial/reputational 
impact  or  by  the  corrective  measures  taken  and/or  planned.  In  addition,  the  IAS  noted 
that in some cases management representations (i.e. comments included in the statement 
of assurance by the HoDs) were not complemented with sufficient information or with 
contextual  explanations  on  the  potential  impact  of  the  internal  control  weaknesses 
identified. 

Recommendations 

To address this issue, the IAS formulated recommendations which can be summarised as 
follows: 

DG DEVCO should improve its guidance on: 

•  The  definition  of  a  reservation,  including  elements  that  should  be  taken  into 
account when considering the issuance of a reservation by EUDs, notably its: 

o  potential financial or reputational impact at EUD level, and 

o  scope, i.e. weakness(es) identified should affect the payments of the year 
or, if reputational by nature, the risk should have materialised during the 
reporting year. 

•  The  consequences  of  a  reservation:  when  issuing  a  reservation,  an  EUD  should 
highlight  the  weaknesses  in  its  internal  control  system  or  the  specific 
circumstances  (security  issues,  political  environment,  etc.)  in  the  country  that 
prevented it from fully or only partially implementing the various internal control 
layers. It should also state the main actions defined, implemented or planned to 
be implemented in order to remedy the situation. 

72 

 
 
The audited services have established action plans which the IAS considers satisfactory 
to address the accepted recommendations49. 

7.4.  Audit on budget support in DG DEVCO  

Background 

The audit of Budget Support in DG DEVCO was included in the IAS 2014 Audit Work 
Programme  following  the  audit  risk  assessment  carried  out  in  2013  as  part  of  the 
preparation of its Strategic Audit Plan for 2013-2015. 

Budget  Support  (BS),  financed  both  by  the  EU  Budget  and  by  the  European 
Development  Fund  (EDF),  represents  around  20%  of  the  total  aid  to  third  countries50. 
The general objectives of this aid modality are to assist in eradicating poverty, promoting 
sustainable  and  inclusive  growth,  and  consolidating  and  improving  democratic  and 
economic  governance.  The  four  major  components  of  the  BS  aid  modality  consist  of 
policy dialogue, performance and result assessment, capacity development, and financial 
transfers. 

transferred 

The specificity of BS is that the use of the money contributed cannot be traced, as the 
funds  are 
treasury.  The  Commission's 
responsibilities when accounting for and auditing these resources are therefore limited to 
ensuring that the conditions for disbursement have been met and that the funds have been 
transferred in accordance with the agreement signed with the country. 

the  country's  national 

to 

At the end of 2013, there were 256 BS operations implemented or under preparation in 
84  countries,  with  ongoing  commitments  amounting  to  €10  779  million 51.  Africa  and 
European Neighbourhood Partnership countries are by far the largest recipients of budget 
support funds (44% and 31% of total ongoing commitments in 2013 respectively). 

The  use  of  certain  aspects  of  budget  support  by  the  Commission  has  been  challenged 
over the years by the European Parliament's Committees on Development (DEVE) and 
on Budgetary Control (CONT), as well as by Member States. In addition, in its Special 

49  In  one  recommendation  rated  'important',  the  IAS  recommended  that  DG  DEVCO  should  require  EU 
Delegations (EUD) to (i) analyse the root cause of errors detected during the DG's annual Residual Error 
Rate  (RER)  study,  and  (ii)  if  and  when  systemic  errors  or  fraud-related  issues  are  identified  during  this 
study, to include these in the EUDs' subsequent External Assistance Management Reports (EAMR) (on the 
implementation  of  EU  development  and  cooperation  aid  including  the  Head  of  Delegation's  statement  of 
assurance on the management of funds sub-delegated to him) together with any actions taken to mitigate the 
risks.  DG  DEVCO  rejected  the  recommendation  on  the  basis  that  the  results  of  the  RER  study  are  not 
representative  for  individual  Delegations,  the  RER  study  is  only  available  after  the  completion  of  the 
EAMR and is just one of the tools through which fraud can be identified. The IAS replied to the DG by 
stressing that the recommendation focuses on the mitigation of systemic and fraud-related risks, if any, that 
had not been detected by all previous controls, and the EUDs reporting on this in the EAMR, and does not 
require the EUDs to use the RER study results to measure their weaknesses. 

50 In 2013, BS disbursements amounted to €1.5 billion, representing 22.0% of  total disbursements (€6.8 billion) 
of  both  the  budget  and  the  EDF  (source:  DG  DEVCO,  Budget  Support  Financial  Implementation,  Risk 
Assessment and Selected Macroeconomic and Fiscal Country Results 2013). 

51 Budget Support Financial Implementation, Risk Assessment and Selected Macroeconomic and Fiscal Country 

Results 2013, page 5. 

73 

 
 
                                                            
Report  11/201052,  the  European  Court  of  Auditors  (ECA)  identified  weaknesses  in  the 
Commission's  management  of  budget  support.  In  response  to  the  concerns  of  Member 
States  for  BS  to  be  more  strongly  linked  to  the  fundamental  values  of  human  rights, 
democracy and the rule of law, the Commission adopted in 2011 a "Communication on 
the  future  approach  to  EU  budget  support  to  third  countries"53.  It  established  a  new 
approach to BS which is enshrined in the new "Budget Support Guidelines"54 issued in 
September 2012 and effective from 1 January 2013. 

Audit Objectives 

The objective of the audit was to assess DG DEVCO's approach to budget support and, 
in particular, whether DG DEVCO's processes to manage its budget support operations 
were efficient and effective. 

Audit Scope 

The audit focused on the following, based on a sample of budget support contracts: 

•  Review  of  the  design  and  implementation  of  the  policy  dialogue  process  which 
contributes  to  the  conclusion  as  to  whether  or  not  the  government  policy  is 
considered  sufficiently  relevant  and  credible  and  if  its  implementation  can  be 
supported with a BS programme; 

•  The  efficiency  and  effectiveness  of  the  decision  making  process  in  the 
identification  and  formulation  phases  leading  to  (i)  whether  or  not  to  grant  BS, 
(ii) setting the amount of BS to provide, (iii) establishing the fixed and variable 
tranches, and (iv) establishing the conditions for the release of variable tranches; 
•  The  effectiveness  and  consistency  of  the  risk  assessment  process  at  the 

identification and formulation stages; 

•  The  efficiency  of  coordination  between  DEVCO  HQ  and  EUDs  at  the 
identification  and  formulation  stages,  including  an  assessment  of  the  new 
structure following the decision not to create regional hubs. 

The following were not included in the scope of the audit: 

•  The intermediate results/indicators leading to payments, as the ECA analyses the 

specific conditions for payment in the context of its Annual Report55; 

•  The  evaluations/final  impact  of  budget  support  as  DG  DEVCO  has  recently 
conducted a study on "The uptake of strategic evaluations into EU development 

52 Special report 11/2010: "The Commission's management of general budget support in ACP, Latin American 

and Asian countries". 

53  COM  (2011)  638,  adopted  on  13/10/2011  and  proceeded  by  the  "Green  Paper  on  the  future  of  EU  Budget 

Support" COM(2010) 586, dated 19 October 2010. 

54 https://ec.europa.eu/europeaid/sites/devco/files/methodology-budget-support-guidelines-201209_en_2.pdf  
55  The  Court  examines  whether  the  Commission  has  complied  with  the  specific  conditions  for  making  budget 
support payments to the partner country concerned and has demonstrated that general eligibility conditions 
(such  as  progress  in  public  sector  financial  management)  have  been  complied  with  (ECA,  2013  Annual 
Report on the implementation of the EU budget, paragraph 7.7). 

74 

 
 
                                                            
cooperation" and issued a "Synthesis Analysis of the Findings, Conclusions and 
Recommendations of 7 Country Evaluations of Budget Support"; 

•  The  financial  transfers  to  the  national  treasury  account  of  the  partner  country 
which has been assessed to be less prone to errors as defined in the Court’s audit 
of regularity"56; 

•  The  reliability  of  the  Common  Relex  Information System (CRIS) data which is 

currently the subject of an extensive review by DG DEVCO57. 

In his Annual Activity Reports for the years 2012 and 2013, the Director-General of DG 
DEVCO included a global reservation on all ABB activities owing to: 

•  a  residual  error  rate  (RER)  of  3.63%  and  3.35%  respectively,  due  to  the 

significant occurrence of errors (legality and regularity); and 

• 

the findings of ECA during the 2011 and 2012 Declaration of Assurance (DAS) 
exercises58. 

The estimated amount at risk for the Community budget in 2013 was €228.55 million 59. 

The fieldwork was finalised on 22 October 2014. All observations relate to the situation 
as of that date. 

Strengths 

The  following  main  strengths  were  identified  during  the  fieldwork  for  the  audited 
processes: 

•  The Budget Support Guidelines60 provide appropriate guidance in the design and 
implementation  of  budget  support  operations,  in  line  with  the  relevant 
communication  issued  by  the  European  Commission  ("The  Future  Approach  to 
EU Budget Support to Third Countries"61); 

•  There  is  evidence  that  policy  dialogue  is  appropriately  coordinated  with  other 

donors at country level. 

•  There is appropriate coordination and division of tasks between the headquarters 

and the EUDs at the identification and formulation stages of the process. 

The IAS notes that a number of DG DEVCO initiatives were on-going at the time of the 
audit fieldwork: 

56 ibid., paragraph 7.6. 
57 Simplification of EuropeAid Business and Support Processes – Phase 2 report – page 157. 
58  For  2012  ECA  found  an  estimated  error  rate  for  transactions  of  3.3%  for  the  EU  budget  in  the  External 
Relations, Aid and Enlargement policy group and of 3% for the European Development Fund (EDF). 
59 Calculated by multiplying the most likely estimate of the residual error rate (3.35%) by the value of payments 

made by DG DEVCO in 2013. 

60 Issued in September 2012. 
61 Dated 13 October 2012. 

75 

 
 
                                                            
•  Drafting of the new/revised Project and Programme Cycle Management (PPCM) 
guidelines  (including  a  specific  chapter  on  Policy  Dialogue),  under  the 
supervision of Unit DG DEVCO.06; 

•  A  DG-wide  simplification  exercise  of  business  and  support  processes  under  the 

supervision of unit DG DEVCO.DG.02 General Coordination. 

The  IAS  took  into  account  these  latest  developments  when  drafting  its  findings  and 
recommendations. 

Major Audit Findings 

The  use  of  budget  support  offers  many  benefits  and  can  be  an  effective  component  of 
development  aid  by  supporting  discussion  with  the  governments  of  partner  countries 
over  policies  and  strengthening  their  public  financial  management  by  contributing  to 
capacity  development.  However,  it  also  carries  many  significant  risks  that  the  partner 
country  may  not  be  capable  of  using  the  funds  efficiently  and  effectively  and  that 
government  expenditures  will  consequently  not  achieve  the  objectives  agreed  between 
the government and development partners, which ultimately may affect the attainment of 
the  DG's  objectives.  The  audit  concludes  that  the  revised  Budget  Support  Guidelines 
constitute a significant improvement compared to the last one, with a stronger link to the 
fundamental values of human rights, democracy and the rule of law. The Budget Support 
Guidelines, together with the strengthened risk management framework, provide a good 
basis  for  informing  decision-making.  The  IAS  welcomes  the  upcoming  review  of  the 
Budget Support Guidelines in time to orient the implementation of the new Multiannual 
Indicative Programmes62 and which is intended to respond to specific areas of concern 
that the services expressed during the first two years of their implementation. However, 
some  weaknesses  in  BS  remain,  in  particular  in  the  framework  of  policy  dialogue 
(objectives, content, etc.) and in documenting roles and contributions to the process, in 
order to demonstrate a consistent and coherent implementation of policy dialogue and of 
the Budget Support Guidelines (report finding No 1). 

Recommendation 

To address this issue, the IAS formulated the following recommendation: 

DG DEVCO should improve the current guidance on policy dialogue (applicable also to 
budget support operations), notably as regards the relationship between political dialogue 
and  policy  dialogue,  the  ways  to  develop  a  strategic  approach  to  policy  dialogue;  the 
issues  and  content  of  policy  dialogue  to  consider  throughout  the  project/programme 
cycle and the implementation of policy dialogue. DG DEVCO should also include policy 
dialogue elements for a certain sector/subsector  in the Financing  Agreement63 to better 

62 The multiannual indicative programmes (MIPs) set out the Union's interests and priorities, specific objectives 
and expected results for the countries in question. MIPs under the 11th EDF for the period 2014-2020 are 
currently being signed. They will indicate,  as in previous programming cycles, the priority areas selected 
for Union financing and should indicate in general terms, for the relevant period, the indicative allocation of 
funds  for  each  priority  area  and  for  each  partner  or  group  of  partner  countries.  Unlike  previous 
programming (MIPs signed for the 2008-2013 period), they no longer indicate the implementation modality 
that will have to be used. They indicate general lines of action instead and refer to a decision on the exact 
modalities at the identification stage, closer to the start of implementation. 

63 Or in another document agreed with the national authorities. 

76 

 
 
                                                            
anticipate  the  main  orientations  of  the  policy  dialogue  (based  on  the  partner  country's 
own strategy and priorities, in line with the alignment strategy) and ultimately contribute 
to  the  achievement  of  the  targeted  results  for  the  specific  indicators  defined  in  the 
Technical and Administrative Provisions (TAPs). 

The audited service has established an action plan which the IAS considers satisfactory 
to address the recommendation. 

7.5.  Audit on the control strategy in FPI 

Background 

The Service for Foreign Policy Instruments (FPI) was created in 2011 to assist the then 
newly created European External Action Service (EEAS) in achieving the objectives of 
the EU's Common Foreign and Security Policy (CFSP). FPI manages an important part64 
of the foreign policy budget. It is responsible, inter alia, for the operational and financial 
management  of  CFSP  operations  and  of  the  crisis  component  of  the  Instrument  for 
Stability (IfS). The "Civilian Planning and Conduct Capability" (CPCC)65, which is part 
of the EEAS, provides strategic direction and is the permanent structure responsible for 
the conduct of civilian CSDP operations. 

FPI  implements  its  budget  in  complex  circumstances  which  create  specific  risks.  The 
environment in which it operates involves many geographically dispersed actors in order 
to respond to international crises and conflicts which require rapid and flexible actions. 
As a result, a particular feature of the CFSP operating environment is that each Common 
Security  and  Defence  Policy  (CSDP)  mission  is  created  from  scratch  without  prior 
assurance  that  they  fulfil  the  requirements  of  the  "pillar  assessment"66  as  stipulated  by 
Article 60 of the Financial Regulation (FR), which may increase the risk of fraud. 

The  CFSP  budget  (€353  million  for  new  contracts  in  2013) 67,  is  mainly  implemented 
(around 95%) under the indirect management mode through financing agreements with 
eleven  CSDP  missions68  with  specific  mandates  decided  by  the  Council  (e.g.  fostering 
rule of law, monitoring agreements, border assistance management, etc.) and twelve EU 
Special  Representatives69  (EUSRs)  with  a  mandate  to  promote  the  EU's  policies  and 
interests. 

64

 €742 million in 2013.

65 The Civilian Planning and Conduct Capability (CPCC) is  mandated to plan and conduct civilian Common Security and Defence Policy 

(CSDP) missions under the political control and strategic direction of the Political and Security Committee (PSC). 

66

  The  Commission  can  entrust  budget  implementation  tasks  to  entities  and  persons  that  guarantee  a  level  of  protection  of  the  financial 
interests of the EU equivalent to that required by the Financial Regulations with regard to the so called "pillars" i.e. internal control 
system, accounting system, independent external audit and rules and procedures for providing financing from EU funds through grants, 
procurement and financial instruments. For persons entrusted with the implementation of specific actions in the CFSP, these conditions 
shall be satisfied no later than six months after the start of their mandate (Art. 60 (2) last sub-paragraph of the Financial Regulation). 

67 Source: FPI AAR 2013. 
68 CSDP missions represent €303 million (source: 2013 AAR). Currently  civilian missions are present in Kosovo (EULEX, €110 million ); 
Afghanistan  (EUPOL,  €78);  Georgia  (EUMM,   €27);  Iraq  (EUJUST  Lex,  €15);  Horn  of  Africa  and  West  Indian  Ocean  (EUCAP 
Nestor, €12 million and EUCAP Sahel Niger, €7 million); Lybia (EUB AM, €7 million); Democratic Republic of Congo and Palestine 
(combines €25 million). 

69 Total budget €24 million in 2013 (Source: 2013 AAR). 

77 

 
 
                                                            
 
The  €243  million  devoted  to  IfS 70  complement  the  actions  of  DG  DEVCO  and  DG 
ECHO in countries facing a crisis or an emerging crisis by financing urgent short-term 
actions. IfS actions consist mainly of contracts implemented via the direct management 
mode through acts of subdelegation to 54 EU Delegations (EUDs)71 where DG DEVCO 
staff  performs  the  financial  management  under  an  SLA  signed  between  FPI  and  DG 
DEVCO, applying the DEVCO PRAG72 rules. 

Audit Objectives 

The  objective  of  this  comprehensive  engagement  was  to  assess  the  adequacy  and 
effectiveness of FPI's control strategy. 

The specific objectives included an assessment of: 

•  Compliance  with 

the 
implementation of the "Guidelines on monitoring missions by the Commission 
to CFSP missions"; 

the  control  strategy  defined 

including 

in  FPI 

•  Effectiveness  of  the  controls  and  anti-fraud  measures  in  place  for  CFSP  and 
IfS  operations  managed  through  EU  missions,  EUDs  and  EUSRs  to  provide 
reasonable assurance on the legality and regularity of underlying transactions; 

•  Timeliness and adequacy of corrective measures taken by FPI; and 

•  Compliance with the SLA with DEVCO with regard to internal controls. 

Audit Scope 

The audit focused on the control strategy over CFSP and IfS operations implemented by 
EUSRs and CSDP missions, and EU Delegations respectively. 

The audit covered: 

•  The design and implementation of the strategy put in place by FPI to control 

the CFSP and IfS instruments. 

•  The effectiveness of the controls underpinning the assurance building process 
related to CFSP and IfS (in particular ex-post checks, external audits, project 
monitoring, recoveries and reporting by EUDs, EUSRs, CSDP missions). 

•  The anti-fraud strategy put in place by FPI in the area of CFSP and IfS; 

•  The calculation and disclosure of the residual error rate in the 2013 AAR. 

The audit did not cover the following due to the low risk involved: 

•  Ex-ante  controls  performed  by  FPI.3  on  each  financing  agreement  with 
EUSRs/missions (with verification by FPI.1) or regularly performed by FPI.2 
on each act of subdelegation to the EUDs; 

70 Budgeted commitments in 2013. 
71 Ongoing contracts as of 31 December 2013, source: CRIS. 
72 http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:C:2010:083:0013:0046:en:PDF 

78 

 
 
                                                            
•  Prior  approval  by  FPI  of  procedures  related  to  CFSP  budget  implementation 

(procurement > €20,000 and grants > € 100,000); 

•  Direct management of CFSP and IfS budget at HQ. 

The fieldwork entailed a review of files and interviews at FPI HQ in Brussels and in four 
sampled  entities  -  EULEX  Kosovo,  EUMM  Georgia,  EUSR  Kosovo,  EUD  in  Georgia- 
for which mission reports have been already sent and validated. 

There  were  no  observations/reservations  in  the  AAR  that  relate  to  the  area/process 
audited. 

The  fieldwork  was  finalised  on  23  April  2014.  All  observations  and  recommendations 
relate to the situation as of that date. 

Strengths 

The auditors acknowledge FPI's efforts to: 

•  improve the CFSP control strategy, in particular the current reflection on how 
to  supervise  the  CSDP  missions  together  with  the  CPCC  through  a  shared 
resource centre; 

•  define  an  ex-post  control  methodology73  to  rank  missions,  delegations  or 
projects to be tested ex post, in view of determining the detected and residual 
error rate for each ABB activity. 

Major Audit Findings 

The IAS has identified the following three very important issues: 

Anti-fraud strategy for CSDP missions/EUSRs (report finding N° 1) 

To  date,  FPI  has  not  developed  and  implemented  an  anti-fraud  strategy  for  CSDP 
missions/EUSRs. Individual CSDP missions/EUSRs have taken ad-hoc actions in areas 
such  as  developing  a  whistleblowing  procedure,  policy  on  declaration  of  conflicts  of 
interest  or  guidelines  on  gifts  and  hospitality  but  these  initiatives  have  not  been 
coordinated or agreed with FPI. In addition, CSDP missions/EUSRs do not have access 
to a database to identify or alert others of potential fraud cases (red flags). 

Assurance from the current system of controls (report finding N° 2) 

The current system of controls has weaknesses caused, inter alia, by a lack of centralised 
guidance  and  deficiencies  in  the  design  of  some  controls  (mainly  the  effectiveness  of 
monitoring missions and ex-post controls for CSDP missions). 

Calculation of the residual error rate for CFSP and IfS (report finding N° 3) 

Although the activities of CSDP missions/EUSRs stretch over a number of years with a 
control  environment  that  is  stable,  FPI  uses  an  annual  rather  than  a  multi-annual 

73 Planning methodology for ex post controls and external audits- Version 2013. 

79 

 
 
                                                            
approach for the calculation of the detected and residual error rate. In addition, the audit 
revealed that the calculation of the error rate is based on the total population of payments 
instead of the payments actually audited. 

Recommendations 

To address these issues, the IAS formulated recommendations which can be summarised 
as follows: 

FPI should 

•  develop  and  implement  a  strategy  for  fraud  prevention  and  detection  in  CSDP 
missions/EUSRs and ensure that staff implementing CFSP budget are regularly trained 
on  anti-fraud  issues  and  ethics.  FPI  should  also  ensure  that  CSDP  missions/EUSRs 
have access to an appropriate anti-fraud database. 

•  provide  effective  and  centralised  guidance  to  missions  and  give  consideration  to 
creating  a  standard  package  of  tools  for  missions  from  their  very  start  in  all  major 
areas  of  activities.  FPI  should  also  re-assess  its  control  strategy  by  improving  its 
effectiveness  during  the  implementation  phase  to  minimise  the  amount  of  ineligible 
expenditure  identified  by  ex-post  controls and  review  its  audit  strategy  of  mandates. 
Finally, FPI should better document the decision-making process for recoveries. 

FPI  only  partially  accepted  this  recommendation  and  rejected  the  part  on  better 
documenting the decision-making process for recoveries. FPI considered that they had 
taken several initiatives to improve this. The IAS' audit, however, revealed that despite 
these initiatives weaknesses remained. 

•  Apply, in line with DG BUDG's AAR Standing Instructions, a multi-annual approach 
for the calculation of the error rate for activities that are multi-annual in nature based 
on payments actually audited, and take steps  to implement an alternative assessment 
pattern to complement the current methodology for the provision of the assurance. 

The audited service has established an action plan which the IAS considers satisfactory 
to address the accepted recommendations. 

8. 

IT AUDITS 

8.1.  Audit on management of local IT in DG AGRI 

Background 

The  mission  of  the  Directorate-General  for  Agriculture  and  Rural  Development  ("DG 
AGRI") is to promote the sustainable development of Europe’s agriculture and to ensure 
the well-being of its rural areas through the implementation of the Common Agricultural 
Policy  (CAP).  For  the  achievement  of  the  CAP's  strategic  objectives,  DG  AGRI  relies 
heavily on IT systems. The main ones support Agricultural market and Direct Payment 
management (ISAMM), Rural development (RDIS and RDIS II), Financial management 

80 

 
 
 
(AGREX, P-STO) and Audit management (CATS/COMBO).  Other systems support the 
quality  policy  (e-Ambrosia),  the  reporting  cluster  (RICA,  AGRIVIEW)  and  the 
legislative domain (APA, Ange Bleu). 

Most of the DG AGRI systems are managed by the IT unit but some systems/ projects 
are also managed outside the IT unit. 

The role of the IT unit in DG AGRI is to provide a high quality and secure Information 
and  Communication  Technology  environment,  to  deliver  and  maintain  up-to-date 
information  and  communication  systems  and  to  provide  services  in  support  of  the  DG 
AGRI activities. 

According  to  DG  AGRI's  Management  Plan,  the  Commitment  appropriations  for  IT 
investments in 2014 reach €7.8 million, exclus ively financed by the operational budget. 

Audit Objectives 

The overall objective of the audit was to assess the internal control system put in place 
by DG AGRI to ensure an adequate and effective management of its local IT activities. 

Audit Scope 

The audit focused in particular on the following areas: 

• 

• 

• 

• 

IT governance (including IT strategy and IT risk management), 

IT portfolio and project management (governance and methodology), 

IT operations (administration of information systems), 

physical and logical security arrangements. 

There are no observations/reservations in the AAR that relate to the area/process audited. 

The fieldwork was finalised in September 2014. All observations and recommendations 
relate to the situation as of that date. 

Strengths 

The IAS identified the following strengths: 

• 

• 

• 

• 

Very competent staff strongly committed to deliver high quality IT services to 

internal and external end-users; 

Constructive collaboration between business owners and the IT Unit; 

Co-operation with other DGs sharing the development of common IT systems 

(like SFC), in line with the rationalisation policy of the Commission. 

Establishment of a Common Architecture Team (AGRICAT) in DG AGRI's IT 
Unit (R3) to offer support for the development of all IT projects of AGRI R3 and to 
ensure project quality management. 

Major Audit Findings 

81 

 
 
The auditors have identified the following two very important issues: 

IT governance (report finding N° 1) 

The governance framework in place in DG AGRI, the functioning of the different bodies 
and the respective roles and responsibilities are not clearly defined. In this respect, the 
steering  function  of  the  IT  Steering  Committee  (ITSC)  is  currently  limited  to  the 
endorsement of the IT Master Plan (for which it meets once a year).  The committee does 
not  receive  regular  information  on  IT  issues  and  IT  activities,  hence  limiting  the 
possibility to adequately oversee the DG's IT activities and to take informed decisions on 
DG  AGRI  IT  priorities  and  investments.  At  the  operational  level,  information 
system/project  steering  committees  do  not  meet  frequently  enough  to  perform  their 
supervisory functions. In addition, the roles and responsibilities of key actors involved in 
project management are not consistently fulfilled. 

IT strategy (report finding N° 2) 

DG  AGRI  did  not  endorse  a  global  IT  strategy  document  describing  in  a  medium-  to 
long-term perspective how its core IT activities74 align with the business strategy for the 
policies under its responsibility. In addition, there is no formal assessment of the long-
term  financial  and  human  resources  required  by  IT  to  support  achievement  of  the 
business objectives. 

Recommendations 

To address these issues, the IAS formulated recommendations which can be summarised 
as follows: 

IT governance 

The DG should enhance the current governance set-up by ensuring that the roles of the 
Steering Committees (at DG, system and project level) are clearly defined and that they 
receive  adequate  information  to  effectively  exercise  their  decisional,  monitoring  and 
supervision responsibilities. 

IT strategy 

DG  AGRI  should  define  and  endorse  a  formal  long-term  IT  strategy  covering  all  IT-
related  activities  supporting  business  goals,  and  translate  it  into  a  more  operational 
medium and short-term IT strategy. This process should be complemented by a regular 
monitoring by the ITSC of the IT strategy alignment with the business objectives and of 
the adequacy of IT-related investments. 

The audited service has established an action plan which the IAS considers satisfactory 
to address the recommendations. 

74 IT core activities encompass IT systems, IT projects, IT services and IT infrastructure. 

82 

 
 
                                                            
8.2.  Audit on IT governance in DG BUDG  

Background 

When executing its tasks, DG Budget heavily  relies  on  IT  systems.  In  this  respect,  the 
Director-General  of  DG  Budget  is  the  System  Owner  of  the  Central  Financial 
Information Systems (ISs), which are largely developed and maintained by DG Budget. 
The  main  ones  are  ABAC75  (for  the  registration  of  the  budgetary  execution  and 
subsequent  accounting),  Badgebud  (for  the  budgetary  preparation)  and  RAD76  (for  the 
follow-up to the annual discharge). 

To get best value from IT and to ensure that it supports adequately the achievement of 
the business strategic goals, an entity has to implement an effective IT governance as a 
key  enabler  of  successful  strategic  alignment  between  business  and  IT,  value  delivery, 
risk management, resource and performance management. To this end, the entity has to 
define  governing  and  operational  structures  with  clear  roles  and  responsibilities, 
effective decision-making processes and communication and reporting lines within and 
between business and IT. 

Successful  IT  governance  ensures  an  effective  and  efficient  use  of  IT  in  terms  of 
satisfying business needs, timely delivery of high-quality IT solutions and services. 

DG Budget's IT Governance is based on different roles (System Ownership held by the 
Director-General,  Business  and  Data  Ownership  held  by  the  Commission  Accounting 
Officer  for  ABAC  and  RAD  and  by  the  Director  A  for  Badgebud)  and  different 
governing bodies (IT Steering Committee (ITSC), Enterprise Architecture Board (EAB) 
and ABAC Programme Management Meeting (PMM)). 

At the operational level, the IT-related activities are spread over four different units (03, 
D5,  R3  and  R4)  under  the  responsibility  of  three  senior  managers  (respectively  the 
Accounting  officer,  the  Director  for  the  Central  Financial  Service  and  the  Resource 
Director). 

Audit Objectives and scope 

The overall objective of the audit was to assess whether IT Governance in DG Budget 
ensures optimal alignment between business and IT, sound management of resources and 
effective IT solutions. 

The  audit  focused  on  DG  Budget's  current  framework  to  govern  and  oversee  its  IT 
activities.  In  particular,  it  looked  into  the  design  and  implementation  of  processes  and 
organizational  structures  in  place  to  ensure  that  IT  adequately  supports  the  DG's 
strategies and objectives. 

75  ABAC  (Accrual  Based  Accounting)  is  a  transversal,  transactional  information  system  allowing  for  the 
execution  and  monitoring  of  all  budgetary  and  accounting  operations  by  the  Commission,  an  Agency  or 
Institution.  ABAC  includes  a  comprehensive  set  of  features  to  ensure  compliance  with  the  Financial 
Regulation and the Rules of Application. 

76 RAD supports the tracking of the recommendations and observations formulated by external parties (Court of 

Auditors, Budgetary Authority) in relation to the discharge process. 

83 

 
 
                                                            
There  were  no  observations/reservations  in  the  AAR  that  relate  to  the  area/process 
audited. 

The fieldwork was finalised on 31 October 2014. All observations and recommendations 
relate to the situation as of that date. 

Strengths 

The  IAS  recognises  the  on-going  efforts  made  by  DG  Budget  to  improve  the  IT  value 
delivery. In particular, the IAS noted the following elements: 

•  DG Budget has defined a multi-layer governance structure encompassing the strategic (IT 

Steering Committee) and the tactical/operational (EAB and PMM) levels; 

•  It has a good track record of availability of  the financial systems, reliability of accounts, 

compliance with legal obligations and general user satisfaction (within DG Budget). 

•  A  monitoring  and  reporting  system  exists,  which  could  potentially  become  an  effective 

management tool to drive continuous improvement. 

•  Staff working in the IT units is competent and motivated. 

Major Audit Findings 

The IAS has identified the following three very important issues: 

IT governance organisation structure and key roles (report finding N° 1) 

DG  Budget  has  a  complex  IT  governance  structure  with  different  bodies  working  at 
different  levels.  Roles  and  responsibilities  related  to  its  IT  governance  are  not  always 
clearly  defined  or  consistently/fully 
the  roles  and 
responsibilities  of  key  actors  are  not  consistently  fulfilled.  In  addition,  the  steering 
function of the ITSC is currently limited endorsing the IT Master Plan. 

implemented.  In  particular, 

IT organisation (report finding N° 2) 

The  current  organisation  allocating  the  IT-related  activities  to  four  units  under  the 
responsibility  of  three  different  senior  managers  results  in  overly  complex  decision-
making and management processes (possibly resulting in a lack of clear leadership and 
accountability), improper separation of activities and ineffective communication between 
key actors. 

Priority setting and planning of IT activities (report finding N° 3) 

DG  Budget's  IT  planning  appears  over-ambitious  in  so  far  as  it  exceeds  the  available 
resources  (especially  for  some  expert  profiles),  which  in  turn  makes  it  difficult  to 
efficiently  prioritise  and  schedule  activities.  In  addition,  daily  operations  very  often 
consume  the  resources  which  were  originally  planned  for  project  management  related 
activities. This results in recurrent delays and postponing of deliverables, in particular in 
the projects belonging to the Review of the ABAC Architecture Programme. 

84 

 
 
 
Recommendations 

To address these issues, the IAS formulated recommendations which can be summarised 
as follows: 

IT Governance organisation structure and key roles 

DG  Budget  should  enhance  the  current  IT  Governance  set-up  by  revising  the 
configuration,  composition  and  mandate  of  the  governing  bodies  and  ensuring  their 
effective  functioning.  It  should  also  increase  the  frequency  of  the  ITSC  meetings  and 
clarify the roles of System Owner, Business Owner and Data Owner for the respective 
information systems. 

IT organisation 

DG Budget should reorganise its IT capacity in homogeneous areas and consolidate the 
IT-related tasks based on an inventory of the IT-related activities currently performed in 
the  DG  and  the  available  competencies.  In  particular,  it  should  separate  IT  supply 
(maître  d'oeuvre)  and  IT  demand  (maître  d'ouvrage)  related  tasks,  and  regulate  the 
relationship between those two components well, e.g. by formal service level agreements 
(SLAs). 

Priority setting and planning of IT activities 

DG Budget should implement a more realistic and, hence, achievable planning of its IT 
activities  and  projects,  assigning  clear  responsibilities  adequately  in  order  to  deliver  as 
planned,  timely  and  within  the  budget.  When  planning  its  IT  activities,  the  DG  should 
therefore take into consideration available resources and objective constraints so to avoid 
unrealistic  expectations  and  very  likely  failures/delays.  In  addition,  DG  Budget  should 
ensure that business requests are timely communicated to the IT function so that it can 
plan its activities more accurately. 

The audited service has established an action plan which the IAS considers satisfactory 
to address the recommendations. 

8.3.  Audit on the management of logical access to systems 

(ECAS/LDAP/windows) in DG DIGIT 

Background 

Identity and Access Management (IAM) is a set of policies, processes and technologies 
for managing the life cycle of digital identities and regulating the access to information 
assets77.  One  of  the  key  aspects  of  the  IAM  is  the  authentication  mechanism,  which 
allows confirming the identity of the users accessing information systems and data stored 
therein, and hence contributes to strengthening the control environment and to reducing 
information security risks. 

77 Information assets refer to information systems and the data treated therein. 

85 

 
 
                                                            
ECAS  (European  Commission  Authentication  Service)  is  the  primary  authentication 
system  used  in  the  European  Commission.  It  is  a  single  repository  of  the  credentials 
(login, password) which serves around 1.3 million users (internal and external) accessing 
corporate  and  local  IT  systems  that  support  administrative,  financial  and  policy-related 
activities. 

ECAS is used by a growing number of applications in different DGs and it is promoted 
by DG DIGIT as the most secure and preferred authentication mechanism to access the 
EC information systems. 

In 2013, DG DIGIT launched a major project (called EXODUS) for the upgrading of the 
ECAS IT infrastructure and the enhancement of ECAS security. 

Audit Objectives 

The overall objective of the audit was to assess whether the control system put in place 
by  DG  DIGIT  ensures  that  the  ECAS  authentication  service  adequately  supports  the 
needs for a secure access to the Commission’s information systems. 

The audit focused on the following main areas: 

•  Governance structure; 

•  Security management; 

•  Service management; 

•  Management of the EXODUS project. 

Audit Scope 

The  present  audit  covered  the  management  of  the  ECAS  system  and  in  particular  the 
activities performed by unit A4 (former 01), A3 and C4 (former C3). During the audit, 
the IAS also contacted DG HR.DS to understand its role in the context of the security-
related activities as well as a number of ECAS clients. 

There were no observations and reservations in the Annual Activity Report (AAR) that 
relate to the area/process audited. 

The  fieldwork  was  finalised  on  12/06/2014.  All  observations  and  recommendations 
relate to the situation as of that date. 

Strengths 

The audit identified the following strengths: 

•  A  well-established  organisation,  with  staff  committed  to  deliver  high  quality  services  to 

internal or external end users; 

86 

 
 
 
•  A  very  competent  development  team,  with  a  thorough  understanding  of  IAM  and 

authentication concepts and of the underpinning technologies; 

•  A  well  conceptualised,  robust  and  versatile  system  to  ensure  a  long  term  use  in  the 

changing EC environment; 

•  Good cooperation with DGs using ECAS services and a high level of users' satisfaction; 

•  Well  documented  instructions  for  systems  administrators  that  are  easily  available  to 

potential users of ECAS. 

Major Audit Findings 

The IAS has identified the following four very important issues: 

Vision and strategy for Identity and Access Management (IAM) (report finding N° 1) 

The vision for the IAM framework has not been properly updated since 2004 to reflect 
the  current  and  future  business  needs  for  the  IAM.  In  addition,  DG  DIGIT  has  not 
defined a proper long-term strategy to achieve this vision. 

Security requirements for ECAS (report finding N° 4) and Requirements management 
and planning of the Exodus project (report finding N° 7) 

There  is  no  proper  security  plan  for  ECAS.  In  addition,  the  requirements  identified  to 
improve  its  security  have  not  been  implemented  yet  and  there  is  currently  no 
plan/roadmap  defining  the  priorities,  timelines  and  resources  required  for  their 
implementation (as part of Phase 2 of the Exodus project). 

ECAS dependency on AD78, CED79 and CUD80 (report finding N° 5) 

In the current setup,  ECAS  security depends,  among  other  elements,  on  AD,  CED  and 
CUD  services.  However,  the  risks  related  to  existing  dependencies  are  not  fully 
addressed. 

Recommendations 

78 Windows Active Directory (AD) is a directory service that authenticates and authorizes all EC users (around 
30.000 in total) and computers in EC Windows domain networks, assigning and enforcing security policies 
for all computers and installing or updating software. AD also manages privileges to access MS Outlook and 
shared resources like network printers and network drives. It contains the credentials of all Windows network 
domain users. 

79 The  Commission  Enterprise  Directory  (CED)  is  a  repository  of  the  credentials  (login,  password)  of  around 
50.000 internal EC users to access less critical legacy applications and Europa web site, and to grant Internet 
access to EC users. It is an LDAP directory. LDAP (Lightweight Directory Access Protocol) is an application 
protocol  for  accessing  and  maintaining  distributed  directory  information  services  over  an  Internet  Protocol 
(IP) network and, for this reason, CED is sometimes referred as CED/LDAP. 

80 The Central User Directory (CUD) is an Oracle database in which user identities are stored and access rights 
are  managed.  It  is  a  sort  of  meta-directory  of  users  at  the  Commission  and  is  a  central  part  of  the  IAM 
infrastructure. It is a populated with user information from COMREF database and serves as an authoritative 
source  of  user  information  for  main  authentication  mechanisms  of  the  Commissions  (e.g.  ECAS,  Active 
Directory and CED).  

87 

 
 
                                                            
To address these issues, the IAS formulated recommendations which can be summarised 
as follows: 

Vision and strategy for Identity and Access Management (IAM) 

DG  DIGIT  should  update  the  vision  for  the  IAM  and  ensure  that  it  is  adequately 
translated  into  a  long-term  strategy  and  yearly  plans  with  clear  objectives  and 
deliverables. 

Security  requirements  for  ECAS  and  Requirements  management  and  planning  of 
the Exodus project 

DG  DIGIT  should  ensure  that  the  security  requirements  are  defined  involving  all  the 
stakeholders and documented in a proper security plan. A clear roadmap with resources, 
deadlines  and  deliverables  should  be  defined  in  the  context  of  Phase  2  of  the  Exodus 
project. 

ECAS dependency on AD, CED and CUD 

DG DIGIT should identify and assess unnecessary dependencies from other components 
and  implement  appropriate  security  measures  to  reduce  the  likelihood  of  security 
breaches and interruption of service. 

The audited service has established an action plan which the IAS considers satisfactory 
to address the recommendations. 

8.4.  Audit on the management of the IT projects in DG EAC (E4ALink and 

EVE) 

Background 

In the new Multiannual Financial framework for the period 2014-2020, the most relevant 
programme will be "Erasmus+". It will bring together several activities currently covered 
by separate programmes, for a total budget of € 19 billion for the 2014-2020 period. 

DG EAC is paying special attention to the planning of IT activities supporting the new 
generation  of  programmes,  in  particular  the  provision  of  IT  applications  to  support  the 
management  of  the  Erasmus+  programme  and  to  facilitate  the  collection  of  accurate, 
complete and consistent data on the programme’s execution. 

The  development  activities  are  mainly  outsourced  to  external  contractors  (intra-muros) 
who are supervised by DG EAC's staff. 

The  adequate  management  of  IT  projects  is  a  key  success  factor  to  ensure  that  IT 
systems  meet  the  users'  expectations  and  are  delivered  on  time  and  within  the  budget 
allocated. DG EAC has developed its own project management methodology and has set 
up  a  governance  and  operational  structure  to  ensure  an  effective  and  efficient 
development of IT systems. 

88 

 
 
Audit Objectives and Scope 

The  overall  objective  of  the  audit  was  to  assess  the  adequacy  of  the  IT  project 
management in DG EAC in terms of respect of the deadlines fixed to release the systems 
into  production,  respect  of  the  budget  allocated  to  the  projects  and  quality  of  the 
deliverables. 

In  particular,  the  audit  focused  on  IT  project  governance,  IT  project/programme 
management  (including  the  supervision  of  the  IT  software  development  related 
activities) and IT security arrangements (in order to ensure that the security elements are 
correctly and timely considered in the IT system development phase). 

During the fieldwork, the IAS looked at three IT systems currently under development, 
namely  E+link  (the  future  system  for  grant  management  for  the  2014-2020  period),  e-
Forms and EVE. 

There  were  no  observations/reservations  in  the  AAR  that  relate  to  the  area/process 
audited. 

The  fieldwork  was  finalised  on  02/12/2013.  All  observations  and  recommendations 
relate to the situation as of that date. 

Strengths 

The audit identified the following strengths: 

•  well-established  IT  department,  with  competent  staff  strongly  committed  to  deliver  high 

quality services to the end users; 

•  Definition of DG EAC's project management methodology; 

•  Effective IT governance framework and regular involvement of senior management in the 

discussion of issues related to IT core organisation and IT projects; 

•  Establishment  of  the  Project  Support  Office  in  charge  of  Analysis/Business  Support, 
Architecture, Quality Assurance, Communications, Project Support and Testing to support 
and ensure coherence of the IT project management in DG EAC; 

•  Physical security arrangements in line with the business requirements. 

Major Audit Findings 

The IAS has identified the following two very important issues: 

Project management framework (report finding N° 1) 

DG EAC project management methodology significantly simplifies the different phases 
of  the  project  management  life  cycle.  Some  key  artefacts,  decision  logs  and  checklists 
have not been prepared, resulting in fewer key controls being in place. 

89 

 
 
In  addition,  DG  EAC  has  not  formally  defined  a  programme  management  process  to 
coordinate  activities  and  the  interdependence  of  projects  contributing  to  the  same 
programme outcome. 

Information system (logical) security (report finding N° 4) 

IT  security  is  embedded  in  the  IT  project  management  methodology  over  the  different 
phases  of  the  project  life  cycle.  However,  DG  EAC  has  neither  conducted  a  formal 
business impact assessment nor finalised a security plan for its IT systems. In addition, 
the LISO and DPC are not systematically involved in the development of IT systems. 

Recommendations 

To address these issues, the IAS formulated recommendations which can be summarised 
as follows: 

Project management framework 

The DG should strengthen the control mechanisms in place by ensuring that the current 
and  future  projects  process,  artefacts  and  workflows  are  aligned  to  the  reference 
framework PM2 and by implementing a formal structure for the programme and portfolio 
management. 

IT logical security 

The DG should define and implement security plans based on the results of the business 
impact and risk assessments and the resulting criticality of the IT systems.  Both business 
side and security specialists should be involved in these exercises. 

The audited service has established an action plan which the IAS considers satisfactory 
to address the recommendations. 

8.5. 

Joint IAS/ IAC audit on the management of local IT in DG MARE 

Background 

The role of IT in DG MARE is to set up and maintain powerful and reliable information 
systems capable of satisfying DG MARE's business process and operational objectives. 
Responsibilities  in  IT  are  mainly  allocated  to  its  IT  sector  F3.002  (in  charge  of 
management  of  local  IT  infrastructure  and  of  IFDM-related  projects)  and  Unit  D4 
(responsible  for  the  European  integrated  environment  of  fisheries  data  management). 
Some operational units also manage their own IT projects without the involvement of the 
IT sector. 

The  budget  for  IT  investments  increased  significantly  from  2013  to  2014,  rising  from 
EUR 2.5 million to EUR 8.8 million. The additional budget received in 2014 has been 
allocated to systems supporting both the Fisheries and the Maritime policies. 

Audit Objectives 

90 

 
 
The overall objective of the audit was to assess the internal control system put in place 
by DG MARE to ensure an adequate and effective management of its local IT activities. 

Audit Scope 

The audit focused in particular on the following areas: 

• 

• 

• 

• 

IT governance (including IT strategy and IT risk management), 

IT project management, 

IT operation (administration of information systems), 

physical and logical security arrangements. 

There  were  no  observations/reservations  in  the  AAR  that  relate  to  the  area/process 
audited. 

The  fieldwork  was  finalised  in  August  2014.  All  observations  and  recommendations 
relate to the situation as of that date. 

Strengths 

The IAS identified the following strengths: 

•  Very competent staff strongly committed to delivering high quality services to 

internal and external end-users; 

•  Constructive collaboration between IFDM business owners and the IT team; 

•  Set-up (by the IT sector) of an agile project management methodology aligned 

with the EC framework; 

•  Enhanced co-operation with other DGs sharing the development of common IT 

systems, in line with the rationalisation policy of the Commission; 

•  Establishment  of  a  dedicated  business  unit  to  improve  the  data  quality 

management in DG MARE (IFDM programme); 

•  Enhanced  cooperation  with  Member  States  (MS)  and  other  stakeholders 
concerned to mutually share and exchange information through CISE and the 
IFDM programme. 

Major Audit Findings 

The auditors have identified the following five very important issues: 

IT strategy/governance (report finding N° 1 and 2) 

DG  MARE  did  not  endorse  a  global  IT  strategy  document  describing  in  a  long-term 
perspective how its core IT activities81 align with the business strategy for  the policies 
under  its  responsibility.  In  addition,  there  is  no  formalised  exercise  to  identify  and 

81 IT core activities encompass IT systems, IT projects, IT services and IT infrastructure. 

91 

 
 
                                                            
prioritise  business  needs  and  related  IT  projects  for  all  the  policies  in  DG  MARE's 
portfolio and to allocate to them the available resources. 

Concerning  the  governance  framework  in  place  at  DG  MARE,  the  functioning  of  the 
different  bodies,  the  relations  and  reporting  lines  among  them  and  the  respective  roles 
and  responsibilities  should  be  clarified.  In  this  respect,  the  IT  Steering  Committee 
(ITSC)  currently  performs  more  than  a  strategic  and  steering  role  as  it  is  regularly 
involved in detailed management/operational activities (thus limiting the time available 
for  overseeing  the  whole  IT  activity  in  DG  MARE).  At  a  more  operational  level,  the 
contribution  of  the  business  to  the  discussion  on  strategic  orientations  and  technical 
issues  related  to  the  IFDM  programme82  is  not  always  effective  and  the  relations  with 
other steering committees are not clear. Concerning the programme/project level, there is 
no  Programme  Steering  Committee  for  CISE  and  IFDM  and  Project  Steering 
Committees have not been formally defined for all the projects. 

IT operations (report finding N° 4) 

DG  MARE's  procedure  for  change  management  only  covers  change  requests  for  the 
IFDM-related systems. In addition, its current procedure requires further improvements 
to adequately cover emergency changes, software code review and formal acceptance of 
deliverables produced by external developers. 

IT project management (report finding N° 6 and 7) 

Concerning  the  portfolio  management  process,  there  is  no  central  function  at  business 
and IT level with a global view on the main business processes and related information 
systems  for  the  two  policies  managed  by  the  DG.  Furthermore,  DG  MARE  has  not 
defined  criteria  to  categorise  IT  projects  in  order  to  better  prioritise  them,  adapt 
management practices and assign appropriate competences according to their typology. 

There is  no  formalised  IT programmes management framework.  In terms of IT project 
management, key staff is playing several different roles implying simultaneous tasks and 
tight  deadlines,  which  results  in  limited  quality  controls  and  service  management.  In 
addition, there is no global quality management system in place. 

Recommendations 

To address these issues, the IAS formulated recommendations which can be summarised 
as follows: 

IT strategy/governance (Observations 1 and 2) 

DG MARE should define and endorse a formal IT strategy covering, for the long-term, 
all  the  IT-related  activities  supporting  the  business  goals.  This  process  should  be 
completed by a formal exercise to identify, assess and prioritise the IT-related needs for 
all the policies under DG MARE's responsibilities and to allocate the available resources 
to them. 

82 The business contribution to the IT decision-making process takes place in the Thematic Groups (TG), which 
represent the core business areas of the DG. The IFDM programme is under the responsibility of the TG5. 
92 

 
 
                                                            
The DG should enhance the current governance set-up by reviewing the functioning of 
the existing governing bodies (ITSC, Thematic groups) and establishing specific steering 
committees  for  programmes  and  projects  to  oversee  their  operational  aspects.  Roles, 
responsibilities and reporting modalities should be clearly defined and implemented for 
all the governing bodies. 

IT operations (Observation 4) 

DG  MARE  should  improve  its  change  management  framework  in  order  to  ensure  that 
common  procedures  cover  all  the  requests  for  change  for  all  the  IT  domains  (systems, 
applications,  projects,  documentation)  and  that  it  assesses  and  prioritises  the  change 
requests consistently. 

IT project management (Observations 6 and 7) 

DG  MARE  should  enhance  its  portfolio  and  programme  management  by  defining  an 
adequate framework encompassing organisation, roles and responsibilities, processes and 
tools both from the IT and business sides. 

DG  MARE  should  improve  the  support  given  to  the  business  and  project  managers, 
design  and  implement  a  quality  management  process  and  improve  the  service 
management function. 

The audited service has established an action plan which the IAS considers satisfactory 
to address the recommendations. 

8.6. 

IT risk assessment in ERCEA 

Background 

The  European  Research  Council  (ERC)  is  a  flagship  component  of  Horizon  2020 
programme.  Its  mission  is  to  encourage  the  research  in  Europe  and  to  support 
investigator-driven frontier research across all fields. 

To  support  the  ERC  in  the  achievement  of  its  goal,  ERCEA  (ERC  Executive  Agency) 
provides attractive, long term grants to excellent investigators and their research teams. 

The  Agency  work  is  supported  by  Information  Systems  serving  its  five  business 
processes (Evaluation, Grants, Administration, Policy and Communication). 

Currently, the IT unit manages 40 staff, out of which 11 intra-muros, and a total budget 
of  EUR  3,815  million.  With  the  move  of  some  IT  tasks  to  DG  DIGIT  (ITIC,  
management  and  hosting  of  main  IT  applications),  the  IT-related  activities  in  the 
Agencies  comprise  now  the  management  of  two  local  data  rooms,  the  development  of 
three  IT  systems  (Pecunia2,  SEP  and  PANEL  NOMINATION)  and  the  IT  project  and 
portfolio management. 

In line with its Strategic Audit Plan for 2015, the IAS has performed an in-depth IT risk 
assessment exercise to map the main risks the Agency is exposed to in the management 
of  IT-related  activities.  The  review  covered  five  key  processes  (IT  Governance,  IT 

93 

 
 
Security,  IT  Project  Management  and  IT  Operations)  and  the  activities  performed  by 
Unit D1  (IT  Tools Development and Management) and Unit B1 (Process Management 
and Review). 

This  IT  risk  assessment  exercise  looked  at  the  controls  in  place  in  terms  of  their 
reasonableness  (whether  they  appear  adequate  for  the  risk  they  intend  to  mitigate  and 
whether they are appropriately designed) but it did not test their effective implementation 
(which would be the scope of an audit engagement). Consequently, the results of the IT 
risk  assessment  represent  the  perception  of  the  extent  to  which  the  processes  reviewed 
are under control. 

The result of this in-depth IT risk assessment will feed into the planning for IAS' future 
audit work programme, notably the strategic Audit Plan 2016-2018. 

Overall conclusion of the IT risk assessment and ERCEA IT Risk Map 

As  a  result  of  the  IT  risk  assessment  exercise,  all  reviewed  processes  have  been 
evaluated according to the perceived risk (both inherent and residual risk) measured in 
terms of impact and vulnerability and positioned in the IT risk Map. 

Overall, the IT-related processes reviewed appear to be under control. Consequently they 
have been placed in the orange zone of the risk map ("Assurance"). 

However,  three  sub-processes  (IT  Project  Management,  IT  Project  Development  and 
Logical Security) have the highest residual risk and are close to the borderline between 
the orange and red zones of the risk map, which means that they may not be sufficiently 
well set up to mitigate the related risks. 

In  this  respect,  the  IAS  considers  that  ERCEA  should  pay  special  attention  to  the 
following areas for improvement: 

Concerning IT project management and IT development, the Agency should implement the 
selected methodologies (PM2 and RUP@EC) consistently across the organisation and for all 
the projects, independently from the project manager. It should therefore define baseline 
artefacts (preferably, depending on the complexity of the project) and the quality function 
should monitor their implementation. In addition, the Agency should develop a monitoring 
systems based on specific KPIs covering both the project management and the development 
activities in order to ensure the quality of the deliverables. 

In terms of logical security, the Agency should formally monitor the implementation of the 
security plans. In addition, it should proactively review the logs of users' activities (in 
particular those of the privileged users) and better manage the access rights of the service desk 
to the production environment of ERCEA's most critical application (ERC Evaluation). 

94 

 
 
9.  MANAGEMENT LETTER 

9.1.  Management letter on common features of performance measurement 

systems  

Introduction 

In  response  to  the  Commission's  move  towards  a  new  performance  culture  and  greater 
focus on value for money, the IAS is carrying out a number of performance audits as part 
of  its  2013-15  Strategic  Audit  Plan.  In  2013  it  addressed  a  number  of  key  themes, 
including  the  adequacy  of  DGs'  performance  measurement  systems  under  different 
management modes and how certain DGs manage their Human Resources83. The audits 
looked at the performance measurement frameworks for a number of DGs and the extent 
to  which  they  measure  performance  internally  and  whether  they  monitor  and  evaluate 
policy  achievements,  and  also  cover  HR  management  related  aspects  (planning, 
allocation and monitoring). 

A number of recurring issues emerged as a result of these audits, particularly as regards 
aspects of the annual planning and reporting cycles of the Management Plans (MPs) and 
Annual  Activity  Reports  (AARs),  but  also  as  regards  the  management  of  human 
resources, in particular concerning the matching of organisation structures to workloads, 
priorities and staff reduction. Whilst specific audit findings and recommendations have 
been  attributed  to  the  DGs  concerned,  the  IAS  considers  that  there  are  a  number  of 
common issues which should be brought to the attention of the Central Services with a 
view to helping the Commission respond effectively to these challenges. 

Objectives and Scope 

The purpose of this management letter is to bring to the attention of the Central Services 
the  common  issues  arising  from  the  IAS's  performance  audits  undertaken  in  2013  in 
order that they can take stock and assess what action needs to be taken centrally. It is not 
intended to summarise all the issues raised during these audits or to provide an overview 
state of implementation of the specific recommendations made to the DGs concerned. 

Detailed findings/Issues for consideration 

Management planning and annual reporting cycle (Lead DGs: SG, BUDG) 

Management Plans – Setting of objectives and indicators 

The IAS's findings need to be seen in the context of the Commission's move towards a 
more performance based culture, which should be underpinned by appropriately adapted 
underlying accountability and reporting structures (MP and AAR). The IAS recognises 
the  continuing  efforts  by  the  Central  Services  to  further  embed  the  principles  of 
performance management into these structures through guidance and instructions. In this 
regard,  the  IAS  identified  a  number  of  recurring  issues  which  should  be  considered  as 

95 

 
 
                                                            
 
part  of  a  package  of  measures  to  improve  the  measurement  and  reporting  of 
performance. 

As regards the setting of objectives and indicators in the Management Plans, our audits 
noted that in practice objectives are set at various levels, whether at the general level for 
the  policy  area  concerned  or  at  the  specific  level  for  operational  ABB  activities. 
However,  the  links  between  these  are  typically  not  well  explained  in  the  MP. 
Furthermore,  they  are  not  always  clearly  linked  to  the  more  strategic  objectives  of  the 
EU2020  targets/flagship  initiatives  and  clear  objectives  for  the  DG's  own  day-to-day 
activities which contribute to the achievement of specific objectives are often lacking. 

In addition, the accompanying performance indicators do not generally meet the widely 
accepted RACER criteria and in some cases can be more numerous than needed and/or 
insufficiently targeted according to need. As regards measuring policy achievements, the 
IAS  found  a  tendency  to  confuse  results  or  impact  indicators  with  output  indicators. 
Targets and indicators, which can be used as measures of outputs are sometimes referred 
to as measures of impacts or results. 

Furthermore,  the  IAS  noted  that  typically  there  is  no  use  of  efficiency/economy 
indicators, either related to policy or internal activities. 

Issue for consideration: 

The  MP  should  be  a  key  part  of  a  DG's'  tool  box  for  helping  to  shape,  direct  and 
ultimately  assess  whether  day  to  day  business  activities  are  ensuring  the  delivery  of 
specific  objectives.  More  specifically, 
the  Standing 
Instructions for the MPs, DGs should: 

improvements 

through 

to 

•  explain  performance  information  (operational  objectives,  indicators  and  targets)  for  their 

day to day activities which contribute to the achievement of specific objectives; 

•  explain how indicators should be interpreted; 

•  ensure  indicators  at  both  policy  level  and  those  covering  internal  activities  are  RACER 

compliant and targets well defined;  

•  ensure that the number of indicators used is proportionate to reflect the relative importance 

of the activity to be reported; 

•  include  more  information  in  the  MP  on  resource  inputs  for  internal  activities  and  relate 

these to outputs/results so as to develop meaningful efficiency indicators; 

•  where relevant, ensure that the appropriate information is set out in the MP on objectives, 
indicators and targets in relation to the 2007-2013 programming period in order to provide 
a  basis  for  measuring  and  reporting  in  the  AAR  and  ultimately  in  the  TFEU  Art.  318 
report. 

Reporting of performance information 

96 

 
 
The  main  vehicles  for  reporting  performance  information  are  the  DG  AARs  and  the 
TFEU Art 318 Evaluation Report. The AAR should of course be the mirror image of the 
MP  and  the  IAS  notes  the  continued  refinement  of  the  AAR  methodology.  Also,  the 
commitment  to address  the criticism of the Art 318 Report by ECA and the Discharge 
Authority,  in  particular  by  linking  it  more  closely  to  the  Part  1  of  the  DGs'  AARs.  In 
addition, policy delivery is also reported through sectorial reports to stakeholders. 

However,  whilst  there  have  been  many  improvements  in  the  way  the  AARs  are 
formulated, the tendency is to focus on reporting of outputs and financial implementation 
rather than on results and impacts. Furthermore, where outputs are reported, they are not 
typically linked to inputs and hence there is no measure of relative efficiency. 

As  regards  reporting  policy  results  and  impacts,  this  clearly  represents  the  major 
challenge  for  most  DGs,  particularly  in  the  main  spending  policy  areas  and  of  course 
these may only be really known after a number of years. This also raises the question of 
how the results of evaluation work are reported in the AAR. To date, evaluation work is 
mainly  reported  through  the  TFEU  Art  318  Report.  The  IAS  found  that  although 
evaluation represents a key part of the overall package of performance information it was 
not always clearly presented in the AAR. It notes the Commission's intention to integrate 
the Art 318 Report into the SPP cycle by enhancing the AAR standing instructions and 
for  the  AARs  to  contain  in  future  a  more  comprehensive  analysis  of  how  and  to  what 
extent EU spending has contributed to achieving policy objectives. 

Issue for consideration: 

Through improvements to the AAR standing instructions, DGs should; 

•  ensure that the AAR clearly reports on the objectives and indicators included in the MP; 

•  report on efficiency at policy and DG operational levels; 

•  ensure  that  their  evaluation  strategies  and  reporting  thereof  feed  into  the  TFEU  Art  318 

Report on a timely basis; 

•  ensure  that  where  they  are  obliged  to  report  separately  on  performance  information,  for 
example in policy area or sectorial reports, this is consistent with the information reported 
in the AAR. Where appropriate there should be a common, family of DGs/services based 
approach to ensure consistency and comparability. 

Internal performance measurement systems 

The  MP  and  AAR  are  the  basic  components  of  each  DG's  performance  measurement 
framework.  However,  they  are  not  the  only  elements  involved.  In  practice,  there  are  a 
range  of  other  tools  involved,  for  example  unit  management  plans,  reports  to  the 
management board etc. All these elements need to work together to ensure the clear and 
consistent  reporting  of  information  and  the  efficient  and  effective  use  of  resources. 
However, aside from the MP and AAR, there is no real guidance to the DGs on how to 
develop an internal performance measurement framework. 

97 

 
 
The objectives set in the MP should be properly cascaded down to the Unit level as far as 
possible.  The  ideal  vehicle  for  this  is  through  Directorate/Unit  level  plans  and 
corresponding reporting. However, the IAS found that the extent to which these are used 
varies in practice between and even within DGs. Furthermore, where they are used they 
also suffer from the problems noted above as regards the overall Management Plan, for 
example objectives not being clearly explained and indicators not RACER compliant. 

Reporting  to  senior  management  on  performance  is  also  a  key  part  of  the  internal 
performance  measurement  framework.  However,  these  reports  do  not  always  cover  all 
relevant  activities.  Also,  the  associated  indicators  are  not  always  appropriate  or 
adequately prioritised. 

Issue for consideration: 

Recognising  that  a  comprehensive  internal  performance  measurement  framework 
includes  more  than  the  MP  and  AAR,  there  needs  to  be  more  specific/comprehensive 
guidance  to  DGs  on  how  to  develop  this  in  practice.  This  should  be  in  addition  to  the 
existing standing instructions for the MP/AAR. 

Reliability of performance information reported by Member States 

A recurring theme is the quality and reliability of performance information reported by 
the DGs, both concerning their own day-to-day activities and the policies and spending 
programmes under their responsibility. This is particularly (but not exclusively) the case 
in  the  Shared  Management  and  Indirect  Management  areas  where  the  reporting  of  key 
performance  information  depends  on  Member  States  and  other  actors  responsible  for 
implementing the budget. The IAS notes that DGs face persistent problems in being able 
to  rely  on  this  and  also  that  what  they  receive  in  practice  is  often  limited  due  to  very 
strict legal obligations. 

Issue for consideration: 

DGs  should  be  encouraged  to  strengthen  controls/checks  on  the  reliability  of 
performance information, in particular where it is reported by Member States, including 
where  appropriate,  through  ex-post  audit  strategies.  In  addition,  the  Commission 
generally  should  ensure  that  any  new  legislative  proposals  (including  delegated  and 
implementing acts) clearly oblige MS to report the performance information necessary in 
order to be able to fulfil its overall reporting obligations. 

Human resource management (Lead DG: DG HR) 

Human resource planning and monitoring 

The identification of current and future HR needs is essential, particularly in view of the 
pressure  to  reduce  staff  numbers,  but  also  in  response  to  dealing  with  changes  to  the 
nature of the Commission's work and structural reorganisations. Human Resource plans 
at the DG level should act as the main vehicle for assessing quantitative and qualitative 
needs in relation to the current skills base and in turn outline any corrective measures, for 
example  redeployment,  recruitment,  training,  etc.  The  IAS  audit  found  that  although 
most DGs have prepared HR plans they are not as a rule sufficiently developed. DGs do 

98 

 
 
not  have  complete,  reliable  and  up-to-date  information  on  the  needs,  the  existing  HR, 
their allocation and the associated priorities.   Some attempts have been made to simulate 
changes  over a  period  of time, taking account of expected reductions, but these do not 
always take account of related workload assessments and/or skills surveys. In short, there 
is generally no common and comparable base to work to. 

However, the IAS notes that in 2013 DG HR launched an HR planning initiative, using 
pilot DGs, aimed at introducing rolling resource plans covering three years, based on a 
gap analysis between DGs' current resources and future needs. The IAS understands that 
this  approach  is  currently  further  developing,  based  on  the  experiences  of  the  pilot 
exercises. 

Issue for consideration: 

The  results  of  the  current  HR  planning  initiative  should  be  used  to  help  drive  a  wider 
package of HR initiatives in order to implement at the Commission level a coherent and 
joined-up response to dealing with the issue of staff reductions. 

Aligning organisational structure to workload and priorities 

Key to the effective  HR management  and also  to  the  measurement  of  DGs'  day-to-day 
business is a proper understanding of whether the organisational structure and allocation 
of  resources  is  appropriately  aligned  to  priorities  and  workload.  In  its  audits,  the  IAS 
found that the information available in operational services on staff allocation (workload 
assessment,    jobs/skills  mapping,  HR  plan)  is  heterogeneous  and  that  the  systems, 
methods and  tools  used to gather related information vary considerably in practice and 
are often locally developed. Assertions made by some DGs as regards the allocation of 
HR being aligned with business priorities is not always supported by the necessary tools, 
methods information. 

In some cases, significant work has been made on developing workload assessments to 
map  human  resources  to  activities  and  tasks,  whilst  in  others  such  assessments  were 
lacking  or  more  limited  and  had  not  been  extended  to  the  full  range  of  the  DG's 
activities. The IAS considers such assessments to be essential for providing assurances 
on the efficient and effective allocation of resources. Furthermore, they can provide very 
useful  information  for  further  developing  DGs'  performance  measurement  systems,  in 
particular  in  developing  efficiency  indicators  for  human  resource  inputs.  However, 
whilst  the  DGs  generally  recognise  the  need  for  such  assessment,  the  main  obstacles 
cited  include  the  time  needed  to  undertake  them,  together  with  the  lack  of  common 
definitions and underlying methodology. 

Issue for consideration: 

As part of a centrally led HR initiative (see point 2.1), workload assessments should be 
conducted  across  the  DGs  more  generally  and  according  to  common  definitions  and 
methodology. 

Inventories of skills and competencies 

99 

 
 
The IAS audits highlighted the issue of the degree to which DGs have visibility on the 
skill and competencies of their staff with a view to mapping these to the needs of their 
organisations. In this regard, the e-cv completion rate clearly varies between DGs, but it 
also  became  clear  that  the  DGs  have  differing  views  on  its  practical  usefulness,  with 
some  feeling  that  it  is  not  sufficiently  tailored  to  their  particular  circumstances.  Also, 
there  are  doubts  about  the  quality  (reliability  and  completeness)  of  the  information 
already contained in the database and the cost-effectiveness of its monitoring. The IAS 
notes  that  certain  DGs  are  already  working  with  DG  HR  to  improve  this.  Others  have 
launched or are planning to launch surveys to assess whether skills are matched to needs. 

Issue for consideration: 

Take-up rates for the e-CV and the quality of the underlying information contained need 
to be improved for it to be an effective HR management tool at the DG level. 

10.  FOLLOW-UP ENGAGEMENTS (SUMMARISED) 

10.1.  2nd Follow-up audit on risk management – Multi DG (SG, DG BUDG, 

FPI) 

Based  on  the  results  of  the  2nd  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations addressed to the SG, DG BUDG and FPI that resulted from the audit 
on risk management and that remained open after the 1st follow-up have been adequately 
and effectively implemented. All the recommendations have been closed. 

10.2.  Follow-up audit on the AAR process in the Commission - Multi DG 

(SG, DG BUDG) 

Based on the results of the first follow-up audit in 2014, the IAS assessed that two out of 
the  four  recommendations  addressed  to  the  SG  and  DG  BUDG  that  resulted  from  the 
audit  on  the  AAR  process  in  the  Commission  have  been  adequately  and  effectively 
implemented. These recommendations have been closed.  

For  one  important  recommendation  on  "Standing  Instructions",  the  IAS  noted  the 
progress  made  regarding  the  implementation  of  a  new  set  of  instructions  and  guidance 
and  in  terms  of  communication  at  the  time  of  the  first  follow-up.  However,  the  IAS 
considered this recommendation not to be fully implemented then. 

One  very  important  recommendation  on  "Reporting  on  sound  management"  was  not 
followed-up during the first follow-up and remains open since the original target date for 
implementation was not reached at the time of the first follow-up. 

In  July  2015,  the  IAS  finalised  a  second  follow-up  and  found  both  remaining 
recommendations to be adequately and effectively implemented. 

100 

 
 
 
10.3.  Follow-up audit on the charge-back process in the Commission - Multi 

DG (DG BUDG, DG DIGIT) 

DG BUDG 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  one  out  of  the  three 
recommendations  addressed  to  DG  BUDG  that  resulted  from  the  audit  on  the  charge-
back process in the Commission has been adequately and effectively implemented. This 
recommendation has been closed. 

For  two  very  important  recommendations,  the  IAS  assessed  that  not  all  the  planned 
actions have been implemented. 

Recommendation No. 1 on governance of the charge-back process  

In  March  2014,  the  ABM  Steering  Group  endorsed  the  ownership  of  the  chargeback 
process  as  well  as  the  guidance  for  the  charge-back  process  between  Commission 
Services.  It  also  "expressed  its  support  for  the  on-going  work  on  the  inter-institutional 
dimension of the Charge-back process". Pending the completion of the guidance for the 
charge-back process to other Institutions and bodies (see recommendation No. 2 below) 
which  has  to  be  endorsed  by  the  ABM  Steering  Group,  too,  the  IAS  considers 
recommendation  No.  1  not  to  be  fully  implemented  yet.  However,  considering  the 
progress made in the implementation of the recommendation on the "Governance of the 
charge-back process", the criticality of this recommendation has been downgraded from 
very important to important. 

Recommendation N° 2 on central guidance and instructions: 

DG BUDG established the framework of the process and published the guidance on the 
charge-back process within the Commission in March 2014. However, the guidance on 
the  charge-back  process  for  services  delivered  to  other  Institutions  and  bodies  has  not 
been published. 

DG DIGIT 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  one  out  of  the  three 
recommendations  addressed  to  DG  DIGIT  that  resulted  from  the  audit  on  the  charge-
back process in the Commission has been adequately and effectively implemented. This 
recommendation has been closed. 

Concerning one very important addressed to DG DIGIT, the IAS assessed that not all the 
planned actions have been fully implemented. 
In particular: 

Recommendation N° 1 on identification of IT services to be charged-back: 

Implementation was postponed to September 2015. This is due to the fact that the initial 
target to identify the baseline services delivered by Directorate C has been replaced by a 
more ambitious goal encompassing all DIGIT services. This project goes in parallel with 
the completion of the consolidated cost model for all the services expected for end of Q2 
2015  as  this  will  allow  defining  the  criteria  for  the  service  to  be  charged  back.  DG 

101 

 
 
DIGIT  expects  the  new  consolidated  cost  model  to  be  endorsed  by  the  ABM  Steering 
Group by September 2015. 

10.4.  2nd Follow-up audit on compliance with payment deadlines - Multi DG 

(DG ECHO, DG DEVCO) 

DG ECHO 

Based  on  the  results  of  the  2nd  follow-up  audit,  the  IAS  assessed  that  one 
recommendation  addressed  to  DG  ECHO  that  resulted  from  the  audit  on  compliance 
with  payment  deadlines  and  that  remained  open  after  the  1st  follow-up  has  been 
adequately and effectively implemented. In total, six out of the seven recommendations 
have been closed after follow-up. 

For  one  important  recommendation  on  "monitoring  and  reporting  on  the  payment 
process  cycle",  the  IAS  considered  the  measures  implemented  not  effective  and  the 
recommendation not to be fully implemented. 

DG DEVCO 

Based on the results of the follow-up audit, the IAS assessed that four recommendations 
addressed  to  DG  DEVCO  that  resulted  from  the  audit  on  compliance  with  payment 
deadlines  and  that  remained  open  after  the  1st  follow-up  have  been  adequately  and 
effectively  implemented.  In  total,  eight  out  of  the  nine  recommendations  have  been 
closed after follow-up. 

For one very important recommendation the IAS considered the measures implemented 
not effective and the recommendation not to be fully implemented: 

Recommendation N° 1 on monitoring and reporting on the payment process: 

The  IAS  observed  that  DG  DEVCO  has  implemented  some  measures  to  enhance  the 
monitoring  and  reporting  system  on  the  payment  process.  They  include  the  CRIS 
contract reviews performed by an external contractor in 2013 and 2014, the analysis on 
payment delays made by Unit DEVCO.R1 and the monitoring of Delegations (using the 
information  provided  in  the  EAMR)  and  Headquarters.  The  IAS  also  takes  note  of  the 
on-going  project  to  develop  harmonised  monitoring  tools  amongst  the  Directorates 
(which should  be  finalised early 2015).  These tools will enable DG DEVCO to have a 
view on the KPIs results at DG level, per directorate and per delegation. They will also 
enable the delegations to monitor more closely their performance. 

10.5.  Follow-up audit on the management and monitoring of staff allocation 

in the Commission services– Multi DG (SG, DG BUDG, DG HR, DG 
AGRI, DG COMP, DG DGT, DG RTD) 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations  addressed  to  DG  COMP,  DG  DGT  and  DG  RTD  that  resulted  from 
the  audit  on  the  management  and  monitoring  of  staff  allocation  in  the  Commission 

102 

 
 
services have been adequately and effectively implemented. All these recommendations 
have been closed. 

One  recommendation  addressed  to  DG  AGRI  that  resulted  from  the  audit  on  the 
management  and  monitoring  of  staff  allocation  in  the  Commission  services  has  been 
adequately and effectively implemented. This recommendation has been closed. 

Concerning  the  other  two  recommendations  addressed  to  DG  AGRI,  the  deadline  for 
recommendation N° 1 on "Mapping of Human Resources with activities and associated 
priorities"  has  been  postponed  to  31  December  2014,  while  recommendation  N°  2  on 
"Workload assessment" is due in March 2015. Based on the results of the follow-up, the 
IAS downgraded recommendation N° 1 from very important to important as DG AGRI 
has implemented most of the agreed actions and is now finalising the process to map HR 
with activities and priorities. 

Of the two recommendations that have been jointly addressed to the SG, DG BUDG and 
DG HR, the IAS assessed that one recommendation has been adequately and effectively 
implemented. This recommendation is closed. 

Considering  the  progress  made  in  the  implementation  of  the  recommendation  on 
"Reporting and accountability by Commission Services on the effective use of posts", 
the  criticality  of  this  recommendation  has  been  downgraded  from  very  important  to 
important. 

10.6.  1st and 2nd Follow-up audit on the Overview Report on Executive 
Agencies – Multi DG (SG, DG BUDG, DG DIGIT, DG HR)  

Based on the results of the follow-up audits performed in 2014, the IAS assessed that all 
the recommendations addressed to SG, DG BUDG, DG DIGIT and DG HR that resulted 
from the audit on the Overview Report on Executive Agencies have been adequately and 
effectively implemented. All the recommendations have been closed. 

10.7.  Follow-up audit on the Commission-wide audit on strategy and 

coordination of statistical data production, development and 
dissemination – Multi DG (DG AGRI, DG ESTAT, DG JRC, DG 
MARE, DG RTD)  

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations  addressed  to  DG  AGRI  and  DG  JRC,  seven  out  of  the  ten 
recommendations  addressed  to  DG  ESTAT,  four  out  of  the  five  recommendations 
addressed  to  DG  MARE  and  two  out  of  the  three  recommendations  addressed  to  DG 
RTD  that  resulted  from  the  Commission-wide  audit  on  strategy  and  coordination  of 
statistical  data  production,  development  and  dissemination  have  been  adequately  and 
effectively implemented. All these recommendations have been closed. 

One important recommendation on "Access to confidential data for scientific purposes" 
addressed  to  DG  ESTAT  has  been  assessed  as  partially  implemented.  Nevertheless, 

103 

 
 
 
based on the actions already taken that demonstrate an advanced state of implementation, 
the IAS decided to close this recommendation. 

One very important recommendation on "roles and responsibilities" and one important 
recommendation  on  "quality  checks"  addressed  to  DG  ESTAT  were  not  reported 
implemented  by  this  DG  and  remain  open.  One  important  recommendation  on  "roles 
and responsibilities" addressed to DG MARE was not reported implemented by this DG 
and remains open. 

For  one  important  recommendation  on  "quality"  addressed  to  DG  RTD,  the  IAS 
considered the recommendation as not implemented and remains open. 

10.8.  Follow-up audit of the internal control system for managing the 
Instrument for Pre-accession Assistance for Rural Development 
(IPARD) in DG AGRI 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  two  out  of  the  three 
recommendations  addressed  to  DG  AGRI  that  resulted  from  the  audit  of  the  internal 
control  system  for  managing  the  IPARD  have  been  adequately  and  effectively 
implemented. These recommendations have been closed. 

For  the  remaining  very  important  recommendation  the  IAS  acknowledged  the  actions 
already  implemented  by  DG  AGRI  and  considered  that  this  recommendation  can  be 
closed as well. 

10.9.  Follow-up audit on the control strategy implementation (Pillar 1 and 2) 

in DG AGRI 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  one  out  of  the  four 
recommendations  addressed  to  DG  AGRI  that  resulted  from  the  audit  on  the  control 
strategy implementation in DG AGRI has been adequately and effectively implemented. 
This recommendation has been closed. 

Concerning the very important recommendations N° 1 on "Detective Measures" and N° 
2  on  "Corrective  Measures",  the  IAS  noted  significant  progress  in  implementation  of 
the  action  plans.  Therefore,  those  recommendations  have  been  downgraded  from  very 
important to important. 

One  important  recommendation  on  fraud  prevention  and  detection  was  excluded  from 
the scope of the follow-up audit as it is being followed up through another multi-DG IAS 
audit on the adequacy and effective implementation of DG anti-fraud strategies. 

10.10.  1st and 2nd Follow-up audit on the residual error rate calculations 

(Pillar 1 & 2) in DG AGRI  

Based on the results of the follow-up audits performed in 2014, the IAS assessed that the 
four  recommendations  addressed  to  DG  AGRI  following  the  limited  review  on  the 

104 

 
 
residual  error  rate  calculations  remain  open.  The  criticality  of  some  of  the  open 
recommendations has been downgraded. 

In particular: 

Recommendation N° 1 on the "Reliability of Member States Control Statistics": 

The  IAS  assessed  that  the  relevant  parts  of  the  recommendation  dealing  with  the  2012 
AAR,  have  been  adequately  implemented.  Considerable  progress  has  been  made  in 
improving the methodology for checking Member States control statistics, including the 
development  of  internal  procedures  and  guidance  to  staff.  Some  more  long  ranging 
actions  are  still  in  progress  as  it  is  still  too  early  to  completely  assess  their 
implementation. These concern mainly the new tasks of the Certifying Body to provide 
an  opinion  on  the  legality  and  regularity  of  underlying  transactions  and  DG  AGRI’s 
related audit work. One key action, concerning the development of guidance to Member 
States on sampling, had not started. Therefore, this recommendation remained open. In 
the light of the progress made, the IAS decided to downgrade the status of the issue from 
critical to very important. 

Recommendation N°4 on the "AAR presentation": 

The IAS noted that considerable progress has  been made in  better explaining the basis 
for relying on MS control statistics and the overall presentation of key figures. The IAS 
also noted that DG AGRI is planning to provide in its AAR 2014 more information on 
the aggregate (Fund or ABB level) multi-annual impact of financial corrections and/or 
recoveries on the amount at risk. However, at the date of the follow-up, the details on the 
approach and presentation were to be defined. The IAS also noted that DG AGRI does 
not  plan  to  develop  a  multi-annual  or  cumulative  system  for  relating  error  rates  or 
amounts at risk with corrections at the Paying Agency level. 

The  IAS  considered  that  for  assurance  purposes,  the  approach  used  for  the  AAR  2013 
should continue to apply. In addition, DG AGRI should explain the inherent limitations 
of  comparing  amounts  at  risk  and  corrections, for example by referring to the time lag 
between  the  identification  of  errors  and  subsequent  corrections,  the  lack  of  data  on 
recoveries and the limited DG AGRI audit coverage at the level of the Paying Agency.  
It  should  then  illustrate  the  corrective  capacity  of  financial  corrections  over  time  by 
using an example based on rolling averages. 

Amounts  reported  in  note  6  of  the  EU  accounts  and  in  the  AAR  2013  concerning 
financial corrections and in note 6 concerning recoveries, include various reductions and 
sanctions charged to the MS (mainly related to cross compliance) that do not necessarily 
correspond  to  the  type  of  legality  and  regularity  errors  that  are  reported  on  in  DG 
AGRI’s  error  rate/amount  at  risk.  The  recoveries  reported  in  the  AAR  do  not  include 
these  amounts  and  therefore  do  not  correspond  to  the  amounts  reported  in  note  6.  DG 
AGRI should assess whether these amounts should be included in its assessment of the 
corresponding  corrective  capacity  and  disclose  any  differences  with  note  6.  Therefore, 
the  IAS  considered  that  the  implementation  of  this  very  important  recommendation 
remains in progress, pending the outcome of the AAR 2014. 

105 

 
 
It is important to address the relevant remaining issues in the AAR 2014 and ensure that 
sufficient  importance  is  dedicated  to  the  actions  to  strengthen  the  reliability  of  the  MS 
control statistics in the long run. 

Considering  the  progress  made  in  the  implementation  of  the  recommendations  on  the 
"Calculation of the RER" and "Reservations", the criticality of those recommendations 
has been downgraded from very important to important. 

10.11.  Follow-up audit on fraud prevention and detection in DG AGRI 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations  addressed  to  DG  AGRI  that  resulted  from  the  audit  on  fraud 
prevention  and  detection  in  DG  AGRI  have  been  adequately  and  effectively 
implemented. All the recommendations have been closed. 

10.12.  Follow-up audit on the design and monitoring of DG AGRI Dir. J 

control strategy (Pillar 1 and 2) 

Based on the results of the follow-up audit, the IAS assessed that three out of the four 
recommendations addressed to DG AGRI that resulted from the audit on the design and 
monitoring  of  DG  AGRI  Dir.  J  control  strategy  have  been  adequately  and  effectively 
implemented. All these recommendations have been closed. 

For one recommendation on "Audit strategy", the IAS considered the recommendation 
as  not  implemented.  Given  the  improvements  already  made,  the  IAS  decided 
nevertheless to downgrade the criticality of the recommendation from very important to 
important. 

10.13.  IAS Follow-up audit on SAM project management in DG BUDG 

Based on the results of the follow-up audit, the IAS assessed that recommendations N° 3, 
6  and  7  have  been  adequately  and  effectively  implemented.  Due  to  the  significant 
changes occurred since the initial audit in 2009, the remaining recommendations are no 
longer applicable. All the recommendations have been closed. 

10.14.  IAS Follow-up audit on risk management in DG COMM  

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations  addressed  to  DG  COMM  that  resulted  from  the  audit  on  risk 
management  have  been  adequately  and  effectively 
the 
recommendations have been closed. 

implemented.  All 

106 

 
 
10.15.  1st and 2nd Follow-up audit on management of local IT in DG DEVCO 

Based  on  the  results  of  the  two  follow-up  audits  performed  in  2014,  the  IAS  assessed 
that  eight  out  of  the  twelve  recommendations  have  been  adequately  and  effectively 
implemented and 3 recommendations became obsolete. All these recommendations have 
been closed. 

One very important recommendation was not reported implemented by DG DEVCO and 
requires further action. 

In particular, 

Recommendation N° 4 on "Local IT security plans": 

The  IAS  agreed  with  DG  DEVCO  that  not  all  the  planned  actions  have  been 
implemented.  DG  DEVCO  finalised  security  plans  for  PADOR  (September  2013), 
PROSPECT (November 2013) and CRIS Contract (August 2014). The security plan for 
PCM  ROM,  delayed  due  to  a  revision  of  the  system,  is  ready  to  be  approved  by  the 
system owner. No other individual or global84 security plan has been developed so far. 

10.16.  Follow-up audit on the limited review of the calculation of the residual 

error rate in DG DEVCO 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations addressed to DG DEVCO that resulted from the limited review of the 
calculation  of  the  residual  error  rate  in  DG  DEVCO  have  been  adequately  and 
effectively implemented. All the recommendations have been closed. 

10.17.  2nd Follow-up audit on EDF grants in DG DEVCO 

Based on the result of the 2nd follow-up audit, the IAS concluded that two out of the four 
recommendations addressed to DG DEVCO that resulted from the audit on EDF grants 
and that remained open after the 1st follow-up audit have been adequately and effectively 
implemented. These recommendations have been closed. 

Two important recommendations on "ex-post project evaluation" and on "verification 
missions to EU Delegations" remain open. 

10.18.  IAS Follow-up audit on long overdue recommendations from IAS 

audits on data centre operation and security (2006) and on corporate 
data network infrastructures and services (2008) in DG DIGIT  

The  original  audit  reports  included  respectively  30  and  13  recommendations,  out  of 
which 7 and 4 rated very important. The IAS has followed-up their implementation by 

84 For  systems  classified  as  STANDARD,  DG  DEVCO  could  envisage  having  a  global  security  plan 

encompassing the baseline security controls for all of them. 

107 

 
 
                                                            
regularly  reviewing  the  status  of  the  recommendations,  dedicated  follow-up  audit 
engagements and regular progress meetings with DG DIGIT and could close the majority 
of the recommendations. The IAS considered that three recommendations from the Data 
centre  audit  and  two  recommendations  from  the  Network  audit  are  not  (fully) 
implemented.  They  include  the  following  two  very  important  recommendations  on  IT 
security (one from each audit) and three important recommendations: 

Audit on Data Centre - Recommendation N° 5-05 - Ensure System Security: 

The  core  of  this  recommendation  requested  DG  DIGIT  to  "draw  up  a  comprehensive 
Data Centre Security plan, keep it up to date and be responsible for communication of 
the plan to all Data Centre staff". DG DIGIT has launched a project aiming at drafting 
and implementing a comprehensive Data Centre Security Plan under the supervision of 
the  ISSB  (Information  Security  Steering  Board)  and  of  the  CISO  (Chief  Information 
Security Officer). 

Audit  on  Corporate  Data  Network  -  Recommendation  N°  DS5  -  Ensure  System 
Security: 

The  IAS  considered  that  main  parts  of  the  recommendation  on  the  preparation  of  a 
network security plan and on logical access to the network have not been implemented. 

The  IAS  considered  that  for  the  very  important  recommendations  the  high  risks, 
affecting  the  Commission  as  a  whole  given  the  corporate  nature  of  the  IT  services 
audited,  have  still  not  been  fully  mitigated.  DG  DIGIT  is  progressing  with  their 
implementation,  in  some  cases  changing  the  focus  of  the  actions  originally  planned 
which  have  become  obsolete  in  the  meanwhile,  to  better  and  more  effectively  mitigate 
the  underlying  risks.  The  IAS  also  welcomes  the  supervisory  role  of  the  ISSB  in 
ensuring  that  timely  and  adequate  measures  are  taken  to  mitigate  the  Commission's 
exposure  to  security  risks  affecting  hosting  and  network  services  delivered  by  DG 
DIGIT. 

The  IAS  invited  DG  DIGIT  to  establish  a  revised  action  plan  for  the  outstanding 
recommendations with realistic but also ambitious deadlines to mitigate the outstanding 
risks.  In  the  meantime,  the  IAS  decided  to  close  the  remaining  outstanding 
recommendations and address the risks involved in a more comprehensive way through 
dedicated audits in its planning cycle 2016-2018. 

10.19.  2nd Follow-up audit on business continuity management in DG DIGIT 

Based  on  the  results  of  the  2nd  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations  addressed  to  DG  DIGIT  that  resulted  from  the  audit  on  business 
continuity  management  and  that  remained  open  after  the  1st  follow-up  have  been 
adequately and effectively implemented. All the recommendations have been closed. 

108 

 
 
10.20.  Follow-up audit on the lifelong learning programme in EACEA / DG 

EAC 

Based on the results of the follow-up audit, the IAS assessed that four out of the eight 
recommendations addressed to EACEA and DG EAC that resulted from the audit on the 
lifelong learning programme have been adequately and effectively implemented. These 
recommendations have been closed. 

Two  very  important  recommendations  on  "Supervisory  framework"  and  on  "Daily 
supervision by DG EAC" and two important recommendations on "EACEA reporting 
to  the  parent  DG"  and  "Manual  of  procedures"  have  not  been  reported  implemented 
by EACEA and DG EAC and remain open. 

10.21.  Follow-up audit on the control strategy in EACI (now EASME) 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  two  out  of  the  four 
recommendations addressed to EACI that resulted from the audit on the control strategy 
in EACI can be closed. 

Considering the progress made in the implementation of the recommendations on "Ex-
ante checks" and on "Anti-fraud strategy" the criticality of those recommendations have 
been downgraded from very important to important. 

10.22.  Follow-up audit on IT governance and performance in EAHC (now 

CHAFEA) 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations addressed to EAHC that resulted from the audit on IT governance and 
performance  in  EAHC  have  been  adequately  and  effectively  implemented.  All  the 
recommendations have been closed. 

10.23.  2nd Follow-up audit on the joint audit (IAC-IAS) on the 

implementation by the EIF of the competitiveness and innovation 
framework programme in DG ECFIN 

Based  on  the  results  of  the  2nd  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations  addressed  to  DG  ECFIN  that  resulted  from  the  audit  on  the 
implementation by the EIF of the competitiveness and innovation framework programme 
and  that  remained  open  after  the  1st  follow-up  have  been  adequately  and  effectively 
implemented. All the recommendations have been closed. 

10.24.  Follow-up audit on off-budget operations: EFSM in DG ECFIN 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations  addressed  to  DG  ECFIN  that  resulted  from  the  audit  on  off-budget 

109 

 
 
operations:  EFSM  have  been  adequately  and  effectively  implemented.  All  the 
recommendations have been closed. 

10.25.  Follow-up audit on performance of operational activities in DG ECHO 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations addressed to DG ECHO that resulted from the audit on performance of 
operational  activities  have  been  adequately  and  effectively  implemented.  All  the 
recommendations have been closed. 

10.26.  2nd Follow-up audit on IPA procurement in DG ELARG (now NEAR) 

Based  on  the  results  of  the  2nd  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations  addressed  to  DG  ELARG  that  resulted  from  the  audit  on  IPA 
procurement  and  that  remained  open  after  the  1st  follow-up  have  been  adequately  and 
effectively implemented. All the recommendations have been closed. 

10.27.  2nd Follow-up audit on the management of local IT in DG EMPL 

Based  on  the  results  of  the  2nd  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations  addressed  to  DG  EMPL  that  resulted  from  the  audit  on  the 
management  of  local  IT  in  DG  EMPL  and  that  remained  open  after  the  1st  follow-up 
have been adequately and effectively implemented. All the recommendations have been 
closed. 

10.28.  Follow-up audit on the closure of the ESF 2000-2006 programming 

period in DG EMPL 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  the  five 
recommendations addressed to DG EMPL following the audit on the closure of the ESF 
2000-2006 programming period in DG EMPL remain open. 

In particular, for the very important recommendations: 

Recommendation N° 1 on "Preparation for closure": 

The IAS noted the progress made on the 2000-2006 programming period, including DG 
EMPL's  contribution  as  regards  the  state  of  play  on  the  closure  process,  to  the 
Commission communication to the European Parliament on protection of the EU budget. 
For  the  2007-2013  period,  guidelines  were  issued  in  March  2013.  These  were  recently 
revised and, together with the strategy on closure, were planned to be adopted by the end 
of 2014. Additionally, seminars have been held with MS in 2013 and 2014 and are also 
planned  for  2015.  However,  due  to  the  further  revision  of  the  guidelines  a  number  of 
related actions aimed at training staff internally, including the finalisation of the Manual 
of  Procedure  and  holding  information  sessions  with  DG  EMPL  services,  were  also 

110 

 
 
delayed  and  will  only  be  implemented  in  2015.  Therefore  the  IAS  considered  this 
recommendation should remain in progress until the remaining actions are completed. 

Recommendation N°2 on "Checks on closure documents for the 2007-2013 period": 

The  IAS  noted  that  as  regards  supervisory  arrangements,  the  definition  of  the  business 
requirements  for  the  closure  process  in  relation  with  the  work-flow  system  RDIS  has 
been  achieved  and  meetings  with  the  IT  developers  are  on  track.  However,  the 
development  of  an  IT  tool  for  the  closure  process  is  in  the  preliminary  phase  only.  In 
addition,  the  IAS  noted  that  the  preparation  of  the  methodology  for  checks  on  key 
closure documents and the checklists is delayed due to the late revision of the guidelines 
on the closure process. The methodology together with the checklists will be part of the 
Manual of Procedure, which will be adopted only in 2015. Given the remaining actions 
still  to  be  completed,  DG  EMPL  revised  the  target  date  for  this  recommendation  to 
30/06/2015.  Therefore  the  IAS  considered  this  recommendation  should  remain  in 
progress until the remaining actions are completed. 

Three  important  recommendations  were  not  followed-up  and  remain  open  since  the 
original target date for implementation was not reached at the time of the follow-up. 

10.29.  Follow-up audit on the implementation of the ESF 2007-2013 

programming period in DG EMPL 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations  addressed  to  DG  EMPL  that  resulted  from  the  audit  on  the 
implementation  of  the  ESF  2007-2013  programming  period  in  DG  EMPL  have  been 
adequately and effectively implemented. All the recommendations have been closed. 

10.30.  2nd Follow-up audit on the control strategy in DG EMPL 

Based on the result of the 2nd follow-up audit, the IAS concluded that the last remaining 
open  recommendation  on  "Continuous  quality  improvement  of  the  audit  function" 
addressed to DG EMPL can be closed. 

10.31.  3th Follow-up audit on local IT systems supporting financial 

management in DG ENER 

Based  on  the  results  of  the  3th  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations  addressed  to  DG  ENER  that  resulted  from  the  audit  on  local  IT 
systems supporting financial management in DG ENER and that remained open after the 
2nd 
the 
recommendations have been closed. 

follow-up  have  been  adequately  and  effectively 

implemented.  All 

111 

 
 
10.32.  Follow-up audit of the European Energy Programme for Recovery 

(EEPR) in DG ENER  

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations addressed to DG ENER that resulted from the audit on the European 
Energy Programme for Recovery have been adequately and effectively implemented. All 
the recommendations have been closed. 

10.33.  Follow-up audit on the control strategy in DG ENTR (now GROW) 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations  addressed  to  DG  ENTR  that  resulted  from  the  audit  on  the  control 
strategy  have  been  adequately  and  effectively  implemented.  All  the  recommendations 
have been closed. 

10.34.  1st and 2nd Follow-up audit of DG ESTAT's preparedness to fulfil its 

role in the economic governance framework 

Based on the results of the follow-up audits performed in 2014, the IAS assessed that all 
the  recommendations  addressed  to  DG  ESTAT  that  resulted  from  the  audit  on  DG 
ESTAT's  preparedness  to  fulfil  its  role  in  the  economic  governance  framework  have 
been  adequately  and  effectively  implemented.  All  the  recommendations  have  been 
closed. 

10.35.  Follow-up of audit on the control strategy in shared management in DG 

HOME  

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  six  out  of  the  seven 
recommendations  addressed  to  DG  HOME  that  resulted  from  the  audit  on  the  control 
strategy  in  shared  management  have  been  adequately  and  effectively  implemented. 
These recommendations have been closed. 

For  one  very  important  recommendation  the  IAS  considered  that  it  was  not  fully 
implemented and that additional efforts are necessary. 

In particular: 

Recommendation N°2 on "Improvements of the ex-post strategy": 

The  IAS  noted  the  progress  made  in  drafting  an  audit  strategy  which  covers  both  the 
2007-2013  and  2014-2020  programming  periods.  However,  the  delays  in  adopting  the 
legislative framework for the 2014-2020 period, particularly as regards the clearance of 
accounts and the conformity clearance procedure, means that the audit strategy will need 
to be updated to reflect these provisions. It should also provide a clearer indication of the 
audit  resources  needed  for  both  periods,  particularly  for  contracted  out  work,  together 
with  a  clear  explanation  of  how  the  DG  intends  to  place  reliance  on  the  audit  work 
performed.  

112 

 
 
10.36.  2nd Follow-up audit on HR security in DG HR 

Based  on  the  results  of  the  2nd  follow-up  audit,  the  IAS  assessed  that  one 
recommendation  addressed  to  DG  HR  that  resulted  from  the  audit  on  HR  security  and 
that  remained  open  after  the  firstfollow-up  has  been  adequately  and  effectively 
implemented.  In  total,  eight  out  of  the  eleven  recommendations  have  been  closed  after 
follow-up. 

For two recommendations, initially rated as very important, the IAS assessed that some 
of the actions planned to mitigate the related risks have not been implemented. However, 
considering  the  progress  made  so  far  the  criticality  of  both  recommendations  has  been 
downgraded to important. 

One recommendation, initially rated as very important, was not reported implemented by 
DG  HR  and  requires  further  improvements.  However,  taking  into  consideration  the 
effective  implementation  of  some  parts  of  the  recommendation  and  the  mitigating 
measures  in  place,  the  IAS  decided  to  downgrade  also  the  criticality  of  this 
recommendation to important. 

10.37.  Follow-up audit on the monitoring of EU law implementation in DG 

JUST 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  three  out  of  the  five 
recommendations addressed to DG JUST that resulted from the audit on the monitoring 
of the EU law implementation have been adequately and effectively implemented. These 
recommendations have been closed. 

For  two  important  recommendations,  the  IAS  considered  that  they  were  not  fully 
implemented. 

10.38.  2nd Follow-up audit on management of procurement in OIB  

Based  on  the  results  of  the  2nd  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations  addressed  to  OIB  that  resulted  from  the  audit  on  the  management  of 
procurement  and  that  remained  open  after  the  1st  follow-up  have  been  adequately  and 
effectively implemented. All the recommendations have been closed. 

10.39.  2nd Follow-up audit on fraud prevention and detection in OLAF 

Based  on  the  results  of  the  2nd  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations  addressed  to  OLAF  that  resulted  from  the  audit  on  fraud  prevention 
and detection and that remained open after  the  1st follow-up have been adequately and 
effectively implemented. All the recommendations have been closed. 

113 

 
 
10.40.  Follow-up audit of the joint Sickness Insurance Scheme managed by 

the PMO 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations  addressed  to  PMO  that  resulted  from  the  audit  on  the  joint  Sickness 
Insurance  Scheme  have  been  adequately  and  effectively  implemented.  All  the 
recommendations have been closed. 

10.41.  Follow-up audit on the implementation of FP7 control systems in REA 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  two  out  of  the  five 
recommendations addressed to REA that resulted from the audit on the implementation 
of the FP7 control systems can be closed. 

Three very important recommendations on "Research for the benefit of SMEs theme", 
"REA's  corrective  actions  following  the  AAR  reservations"  and  on  "Anti-Fraud 
Measures" have not been reported as implemented by REA and remain open. 

10.42.  Follow-up audit on the closure of Cohesion Fund projects 2000-2006 in 

DG REGIO 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations addressed to DG REGIO that resulted from the audit on the closure of 
the  Cohesion  Fund  projects  2000-2006  in  DG  REGIO  have  been  adequately  and 
effectively implemented. All the recommendations have been closed. 

10.43.  Follow-up audit on DG REGIO implementation of the 2007-2013 

programming period 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  three  out  of  the  five 
recommendations  addressed  to  DG  REGIO  that  resulted  from  the  audit  on  the 
implementation of the 2007-2013 programming period can be closed. 

One very important recommendation was not reported implemented by DG REGIO and 
requires further action. Nevertheless, considering the partial implementation of the action 
plan the IAS decided to downgrade the criticality of the recommendation to important. 

For one important recommendation on "Corrective measures to reduce the error rate" 
the IAS considered that it was not fully implemented. 

10.44.  Follow-up audit of the limited review on residual error rate in DG RTD 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations  addressed  to  DG  RTD  that  resulted  from  the  limited  review  on  the 
residual  error  rate  in  DG  RTD  have  been  adequately  and  effectively  implemented.  All 
the recommendations have been closed. 

114 

 
 
10.45.  Follow-up audit on IT governance and performance in DG SANCO 

(now SANTE) 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations  addressed  to  DG  SANCO  that  resulted  from  the  audit  on  IT 
governance  and  performance  in  DG  SANCO  have  been  adequately  and  effectively 
implemented. All the recommendations have been closed. 

10.46.  Follow-up audit on the control strategy in DG SANCO (now SANTE) 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations  addressed  to  DG  SANCO  that  resulted  from  the  audit  on  the  control 
strategy  have  been  adequately  and  effectively  implemented.  All  the  recommendations 
have been closed. 

10.47.  Follow-up audit on the handling of sensitive information in SJ 

Based on the results of the follow-up audit, the IAS assessed that seven out of the twelve 
recommendations  addressed  to  SJ  that  resulted  from  the  audit  on  the  handling  of 
sensitive  information  have  been  adequately  and  effectively  implemented.  All  these 
recommendations have been closed. 

Concerning the remaining five very important recommendations, the IAS considered that 
some of the actions planned to mitigate the risks identified at the time of the audit have 
not been completed. 

In particular: 

Recommendation N° 1 on roles and responsibilities at central and team level: 

At the beginning of 2013 SJ set up a permanent working group created to supervise the 
management of  sensitive  information and  to ensure an effective coordination  at  central 
level.  The  IAS  notes  that  it  has  only  met  twice  so  far  and  there  is  no  evidence  of  an 
effective  steering  of  the  process.  Moreover,  there  is  no  evidence  of  a  security-oriented 
risk assessment and no security plan has yet been endorsed. At the legal team level, there 
is still no clear definition and recognition of the role of 'Documentalists' in the process of 
handling sensitive information. 

Recommendation N° 2 on policy and procedures for handling sensitive information: 

SJ  implemented  some  procedures  for  marking,  physical  security,  use  of  Information 
Systems and transmission and disposal of documents. However, this guidance has to be 
complemented  by  a  security  policy,  a  definition  of  "sensitive  information",  clear 
description  of  the  roles  and  responsibilities  of  the  actors  involved  in  the  process  and 
guidance to assess the sensitivity of the information handled in the context of a litigation 
or request for advice. 

Recommendation N° 3 on security incident reporting and management: 

115 

 
 
SJ  staff  has  been  instructed  about  whom  they  have  to  inform  in  case  of  threats  to  the 
operational  security  of  SJ  or  in  case  of  threat  to  the  computer  or  systems  security. 
However,  they  are  not  requested  to  analyse  the  causes  of  security  breaches  and  to 
identify the potential consequences or the type of response to be provided. According to 
SJ,  the  number  of  security  incidents  detected  so  far  (two)  does  not  justify  the 
development  of  a  complete  process  to  manage  and  report  on  security  incidents.  In  this 
respect, the IAS observed that the lack of a clear definition of "security information" and 
of roles and responsibilities of the actors in the process (recommendation No. 2), further 
guidance and monitoring on handling EU restricted information (recommendation No. 3) 
and  of  appropriate  training/awareness  raising  initiative  (recommendation  No.  10)  does 
not ensure that all security incidents have been identified and adequately communicated. 

Recommendation N° 6 on handling EU restricted documents: 

The  existing  guidance  does  not  provide  instructions  on  the  physical  handling  of 
classified  information.  In  addition,  there  is  no  monitoring  activity  in  place  to  ensure 
correct implementation of the procedure for handling EU restricted information. 

Recommendation N° 10 awareness and training on HSI and information security: 

There is no evidence of the provision of specific training in ethical issues or awareness 
raising initiatives. 

10.48.  IAS/IAC joint Follow-up audit on monitoring the implementation of 

EU law in DG TAXUD 

Based  on  the  results  of  the  follow-up  audit,  the  IAS  assessed  that  all  the 
recommendations addressed to DG TAXUD that resulted from the audit on monitoring 
the  implementation  of  EU  law  in  DG  TAXUD  have  been  adequately  and  effectively 
implemented. All the recommendations have been closed. 

10.49.  Follow-up audit on the management of local IT in DG TRADE 

Based on the results of the follow-up audit, the IAS assessed that eight out of the twelve 
recommendations  addressed  to  DG  TRADE  that  resulted  from  the  audit  on  the 
management  of  local  IT  in  DG  TRADE  have  been  adequately  and  effectively 
implemented. All these recommendations have been closed. 

For  two  very  important  recommendations  the  IAS  considered  that  they  were  not  fully 
implemented and that additional efforts are necessary for the effective implementation of 
the action plans. 

In particular: 

Recommendation N°1 on the role of the IT Steering Committee (ITSC): 

In  2013  DG  TRADE  approved  an  ITSC  charter  describing  the  objective,  tasks, 
composition,  meetings  frequency  and  agenda  of  this  Committee.  However,  in  terms  of 
implementation, the IAS observed that some key tasks are not effectively performed, in 

116 

 
 
particular  the  monitoring  of  the  status  of  IT  projects  (and  in  general  of  the  IT  services 
provided  to  DG  TRADE)  by  means  of  appropriate  reporting  and  key  performance 
indicators, and the monitoring of IT-related risks. In this respect, DG TRADE has not yet 
defined  the  nature  and  content  of  the  monitoring  function  (which  inputs/deliverables 
should  be  provided  to  the  ITSC,  the  kind  of  projects  or  Information  Systems  to  be 
monitored, the KPIs to be used, the IT Risks to be reported to the ITSC, etc.). 

Recommendation N° 2 on the management of IT related risks in DG TRADE:  

DG TRADE's risk management process includes a provision on the identification of IT 
risks and the IT Unit has provided assistance and guidelines to operational units for the 
assessment  of  security  risks.  However,  there  is  no  evidence  of  relevant  IT  risks  being 
assessed and consolidated and that the results of the assessment have been escalated to 
the ITSC for review and discussion. 

Concerning  recommendation  N°  10  on  "Review  of  Local  System  Administrator's 
activities logs" (rated important), DG TRADE has not found an acceptable cost/benefit 
mitigation  action  and  the  associated  risk  of  not  implementing  the  recommendation  has 
been  accepted  by 
this 
recommendation. 

the  Director-General.  Consequently, 

IAS  closed 

the 

One important recommendation on "Security plans" was not reported implemented by 
DG TRADE and remains open. 

117 

 
 
11.  LIST OF ACRONYMS 

Acronym 

AA 

AAL 

AAR 

ABAC 

ABB 

ABM 

ACRs 

AD 

AIRs 

APC 

BPS 

BS 

CAFS 

CAP 

CAs 

CAS 

CCs 

CED 

CF 

CFSP 

CISE 

CISO 

CMO 

CONT 

Description 

Audit Authority 

Ambient Assisted Learning 

Annual Activity Report 

Accrual Based Accounting 

Activity Based Budgeting 

Activity Based Management 

Annual Control Reports 

Windows Active Directive 

Annual Implementation Reports 

Audit Progress Committee 

Basic Payment Scheme 

Budget Support 

Commission Anti-Fraud Strategy 

Common Agricultural Policy 

Certifying Authorities 

Common Audit Service 

Competence Centres 

Commission Enterprise Directory 

Cohesion Fund 

Common Foreign and Security Policy 

Common Information Sharing Environment 

Chief Information Security Officer 

Common Organisation of Markets 

European Parliament's Budgetary Control 

118 

 
 
 
CPCC 

CPR 

CRaS 

CRIS 

CRR 

CSC 

CSDP 

CUD 

DAS 

DAs 

DEVE 

DGs 

DO 

EAB 

EAC 

EAFRD 

EAGF 

EAMR 

EAMRs 

EBA 

EC 

ECA 

ECAS 

ECB 

EDF 

Committee 

Civilian Planning and Conduct Capability 

Common Provisions Regulation 

Common Representative audit Sample 

Common Relex Information System 

Cumulative Residual Risk/Error Rate 

Common Support Centre 

Common Security and Defence Policy 

Central User Directory 

Declaration of Assurance 

Delegated Acts 

European Parliament's Committee on 
Development 

Directorates-General 

Desk Officer 

Enterprise Architecture Board 

Ex-Ante Conditionalities 

European Agricultural Fund for Rural 
Development 

European Agricultural Guarantee Fund 

External Assistance Management Report 

External Assistance Management Reports 

European Banking Authority 

European Commission 

European Court of Auditors 

European Commission Authentication Service 

European Central Bank 

European Development Fund 

119 

 
 
EFA 

EIB 

EIOPA 

EIT 

EMFF 

EPMs 

EPSO 

ERC 

ERDF 

ES 

ESA 

ESF 

ESIF 

ESMA 

ESOs 

EUD 

EUSR 

F4E 

FAFA 

FI-TAP 

FP7 

FPA IO 

FR 

GU 

Ecological Focus Areas 

European Investment Bank 

European Insurance and Occupational Pensions 
Authority 

European Institute of Innovation and Technology 

European Maritime and Fisheries Fund 

Engagement Planning Memoranda 

European Personnel Selection Office 

European Research Council 

European Regional Development Fund 

European Semester 

European Supervisory Authorities 

European Social Fund 

European Structural and Investment Funds 

European Securities and Markets Authority 

European Semester Officers 

European Union Delegation 

EU Special Representatives 

Fusion for Energy 

Financial and Administrative Framework 
Agreement 

Financial Instruments Technical Advisory 
Platform 

Seventh Framework Programme for Research and 
Technological Development 

Framework Partnership Agreement for other 
International Organisations 

Financial Regulation 

Geographical Units 

120 

 
 
HIP 

HoD 

HR 

HRM 

IACS 

IAF 

IAM 

IAs 

IAS 

ICRC 

IFDM 

IFRC 

IfS 

IMDA 

IO 

IOM 

IPARD 

ISSB 

ITSC 

JRC 

JTI 

JUs 

KPI 

LPIS 

MA 

MASP 

Humanitarian Intervention Plan 

Head of Delegation 

Human Resources 

Human Resources Management 

Integrated Administrative and Control System 

Integrated Analytical Framework 

Identity and Access Management 

Implementing Acts 

Internal Audit Service 

International Committee of the Red Cross  

Integrated Fisheries Data Management 

International Federation of the Red Cross and 
Red Crescent Societies 

Instrument for Stability 

Indirect Management Delegation Agreements 

International Organisations 

International Organisation for Migration 

Instrument for Pre-accession Assistance for Rural 
Development 

Information Security Steering Board 

IT Steering Committee 

Joint Research Centre 

Joint Technology Initiatives 

Joint Undertakings 

Key Performance Indicator 

Land Parcel Identification System 

Managing Authority 

Multi-Annual Strategic Plan 

121 

 
 
MCSs 

MFF 

MoU 

MP 

MS 

NGOs 

OP 

PA 

PAs 

PICS 

PMM 

PPAG 

PPMT 

PRAG 

Management and Control Systems 

Multiannual Financial Framework 

Memorandum of Understanding 

Management Plan 

Member States 

Non-Governmental Organisations 

Operational Programme 

Partnership Agreements 

Paying Agencies 

Programme Information and Communication 
Space 

Programme Management Meeting 

Public Procurement Advisory Group 

Public Procurement Management Tool 

RACER 

Relevant, Accepted, Credible, Easy and Robust 

RAFS 

RAL 

RER 

SAM 

SDAOs 

SG 

SLAs 

SMART 

SME 

SPP 

SWD 

Research family Anti-fraud Strategy 

Reste à Liquider 

Residual Error Rate 

State Administrative Manual 

Sub-Delegated Authorising Officers 

Secretariat General 

Service Level Agreements 

Specific, Measurable, Achievable, Relevant, 
Timely 

Small and Medium Enterprise 

Strategic Planning and Programming 

Staff Working Document 

122 

 
 
 
TAPs 

TFEU 

TG 

TS 

UCC 

WFS 

WWD 

Technical and Administrative Provisions 

Treaty on the Functioning of the European Union 

Thematic Groups 

Technical Standards 

Union Customs Code 

Workforce Simulator 

Worldwide Decision 

123