COMMISSION STAFF WORKING DOCUMENT Summary of Executive summaries Internal audit engagements finalised by the IAS in 2016 Accompanying the document Report from the Commission to the European Parliament and the Council Annual report to the Discharge Authority on Internal audits carried out in 2016 (Art 99(5) of the Financial Regulation)
Date: 2017-09-15
Institutions: European Commission, Secretariat-General
Subjects: administrative check, audit, budgetary discharge, financial year, fraud against the EU, management audit

EUROPEAN 
COMMISSION 

Brussels, 15.9.2017  
SWD(2017) 306 final 

COMMISSION STAFF WORKING DOCUMENT 

Summary of Executive summaries 

Internal audit engagements finalised by the IAS in 2016 

Accompanying the document 

Report from the Commission to the European Parliament and the Council 

Annual report to the Discharge Authority on Internal audits carried out in 2016  
(Art 99(5) of the Financial Regulation) 

{COM(2017) 497 final} 

EN 

   EN 

 
 
 
 
  
 
 
Contents 

CONTENT OF THIS STAFF WORKING DOCUMENT: ............................................................................... 8 

PART 1: FINAL REPORTS ...................................................................................................................... 9 

1. 

HORIZONTAL AUDITS .................................................................................................................. 9 
Audit on the management of intra-muros contractors .................................................... 9 

1.1. 

1.2. 

1.3. 

1.4. 

Audit on coordination and working arrangements with EU decentralised Agencies in 
DG SANTE and DG HOME ............................................................................................. 10 

Audit on performance and coordination of Anti-Fraud activities in the Traditional 
Own Resources area .......................................................................................................... 15 

Audit on the new Better Regulation agenda in the Commission - what is the state of 
play approximately one year after its adoption? ............................................................ 16 

1.5. 

Audit on financial management in the SG, LS, EPSC and DGT .................................. 19 

1.6. 

1.7. 

1.8. 

Audit on the early implementation of ESIF control strategy 2014-2020 in DGs 
REGIO, EMPL and MARE ............................................................................................. 20 

Audit on effectiveness of simplification measures under 2014-2020 ESI Funds in DG 
EMPL, REGIO and MARE ............................................................................................. 24 

Audit on the processes for managing and sharing data on agri-environmental-climate 
issues in DG AGRI, DG CLIMA and DG ENV .............................................................. 27 

1.9. 

Audit on the procurement process in OIB, OIL and DG BUDG .................................. 30 

1.10.  Audit on the procurement process in DG COMM, DG Interpretation (SCIC) and 

EPSO/EUSA ....................................................................................................................... 33 

2. 

2.1. 

AGRICULTURE, NATURAL RESOURCES AND HEALTH .............................................................. 34 
Audit on the design of DG AGRI's performance measurement system for the CAP 
2014-2020 ............................................................................................................................ 34 

2.2. 

Audit on DG AGRI's management and control system for Voluntary Coupled 
Support (VCS) ................................................................................................................... 36 

2.3. 

Audit on public procurement in DG CLIMA ................................................................. 39 

2.4. 

Audit on staff allocation and process management in response to staff reduction in 
DG ENV ............................................................................................................................. 40 

2.5. 

Audit on pilot projects and preparatory actions in DG SANTE ................................... 40 

3. 

RESEARCH, ENERGY AND TRANSPORT ..................................................................................... 41 
Audit on Human Resources management in DG CONNECT ....................................... 41 

3.1. 

3.2. 

Audit on closure of projects of legacy programmes in DG CONNECT ....................... 42 

2 

 
 
3.3. 

Audit on the management and functioning of Euratom Safeguards in DG ENER ..... 42 

3.4. 

Audit on the supervision of ITER in DG ENER ............................................................. 43 

3.5. 

Limited review of the calculation and the underlying methodology of the residual 
error rate for the 2015 reporting year in DG ENER ...................................................... 45 

3.6. 

Audit on Human Resources management in ERCEA .................................................... 45 

3.7. 

Audit on the coordination by INEA with its parent DGs during the key stages of the 
Strategic Planning and Programming cycle .................................................................... 46 

3.8. 

Audit on competitive activities in DG JRC ..................................................................... 46 

3.9. 

Audit on setting of objectives and measurement of performance in DG MOVE ........ 47 

3.10.  Audit on DG MOVE's monitoring of the aviation and maritime security policies, 

including related working arrangements with the EMSA Regulatory Agency ........... 49 

3.11.  Limited review of the calculation and the underlying methodology of the residual 

error rate for the 2015 reporting year in DG MOVE .................................................... 51 

3.12.  Audit on H2020 grant management in the REA: part a) from the preparation of the 

calls for proposals to the signature of the grant agreements part b) administrative 
logistical services provided for H2020 ............................................................................. 51 

3.13.  Limited review of the calculation and the underlying methodology of the residual 

error rate for the 2015 reporting year in the REA ......................................................... 53 

3.14.  Audit on Human Resources management in the REA ................................................... 54 

3.15.  Audit on procurement in DG RTD .................................................................................. 55 

3.16.  Audit on H2020 project management in DG RTD ......................................................... 56 

3.17.  Audit on the implementation of the FP7 ex-post audit strategy by the Common Audit 
Service in DG RTD ............................................................................................................ 57 

4. 

EXTERNAL ACTIONS .................................................................................................................. 58 
Audit on payment deadlines in DG DEVCO .................................................................. 58 

4.1. 

4.2. 

Audit on performance management system in DG DEVCO ......................................... 60 

4.3. 

Audit on direct management of grants in DG DEVCO (DCI and EDF) ...................... 61 

4.4. 

Audit on the instrument contributing to stability and peace in FPI ............................. 62 

4.5. 

Limited review of DG NEAR's residual error rate methodology and calculation for 
the 2015 reporting year ..................................................................................................... 63 

4.6. 

Audit on risk management in DG NEAR ........................................................................ 65 

3 

 
 
4.7. 

4.8. 

Review of the tender procedure EuropeAid/133797/DHL/SUP/XK, following the 
article 99(4) complaint received on 20 March 2016  (DG NEAR) ................................. 67 

Audit on procurement under the Instrument for Pre-Accession (direct management 
and indirect management with beneficiary countries) – phase I .................................. 67 

5. 

EDUCATION AND CITIZENSHIP .................................................................................................. 68 

5.1. 

5.2. 

5.3. 

Audit on performance management systems in DG EAC, including the contributions 
of executive agencies and national agencies to the achievement of policy objectives .. 68 

Audit on DG HOME's management of emergency assistance in the context of the 
migration crisis .................................................................................................................. 69 

Consulting engagement in DG HOME on the methodology for determining the 
'materiality level' and measuring the 'residual amount at risk' for the Annual Activity 
Report ................................................................................................................................. 72 

5.4. 

Audit on the management of grants under 2014-2020 Justice and Rights, Equality and 
Citizenship programmes in DG JUST ............................................................................. 73 

5.5. 

IAS review on mapping of EC refugee crisis interventions ........................................... 75 

6. 

ECONOMIC AND FINANCIAL AFFAIRS ....................................................................................... 75 
Audit on effectiveness of the management of the COSME Programme by EASME .. 75 

6.1. 

6.2. 

Audit on financial management, procurement and grant processes in DG ECFIN .... 76 

6.3. 

Audit on setting of objectives and measurement of performance in DG GROW ....... 77 

6.4. 

Audit on financial management and IT procurement in DG TAXUD ......................... 78 

6.5. 

Audit on ethics in DG TRADE ......................................................................................... 79 

7. 

GENERAL SERVICES ................................................................................................................... 79 
Audit on management of procurement under DG ESTAT's operational budget ....... 79 

7.1. 

7.2. 

Audit on procurement and grants in OLAF ................................................................... 80 

7.3. 

Audit on the governance, planning, monitoring and implementation of the budget line 
of the OLAF Supervisory Committee .............................................................................. 81 

7.4. 

Audit on the charge-back process in PMO ..................................................................... 81 

8. 

IT audits ..................................................................................................................................... 82 
Audit on effectiveness of measures to handle manual interventions in ABAC ............ 82 

8.1. 

8.2. 

Audit on management of EESSI project in DG EMPL ................................................. 84 

8.3. 

Audit on business continuity management at OP ........................................................... 86 

8.4. 

IT governance and portfolio management in DG GROW ............................................. 88 

4 

 
 
8.5. 

Audit on IT security in JRC ICT systems ....................................................................... 90 

PART 2: FOLLOW-UP ENGAGEMENTS (SUMMARISED) ...................................................................... 93 

1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

10. 

11. 

12. 

13. 

14. 

15. 

16. 

17. 

18. 

19. 

Follow-up audit on the design of DG AGRI's management and control system for 
greening .............................................................................................................................. 93 

Follow-up audit on payments suspensions and interruptions in the 2014-2020 CAP 
framework .......................................................................................................................... 94 

Follow-up audit on the management of the approval process of the 2014-2020 Rural 
Development Programmes (RDPs) .................................................................................. 95 

Follow-up audit on gap analysis of new legislation/design of 2014-2020 programming 
period of European Structural and Investment Funds Phase 2 in DG MARE ........... 96 

Follow-up audit of IAC recommendations in DG SANTE ............................................ 96 

Follow-up audit of management and supervision of contracts for the outsourced IT 
services in DG SANTE ...................................................................................................... 97 

Follow-up audit on preparations for use of financial instruments under 2014-2020 in 
DG EMPL .......................................................................................................................... 97 

Follow-up audit on preparations for use of financial instruments under 2014-2020 in 
DG REGIO ......................................................................................................................... 97 

Follow-up audit on gap analysis of new legislation/design of 2014-2020 programming 
period of European Structural and Investment Funds' (ESI Funds) Phase II ............ 97 

Follow-up audit on the governance and supervision of the nuclear decommissioning 
assistance programmes in DG ENER .............................................................................. 99 

Follow-up audit on the supervision of the implementation of CEF in DG ENER ....... 99 

Follow-up audit on the management and functioning of Euratom safeguards in DG 
ENER .................................................................................................................................. 99 

Follow-up audit on procurement management in JRC ............................................... 100 

Follow-up audit on strategic planning and programming / activity based management 
in JRC ............................................................................................................................... 100 

Follow-up audit of IAC recommendations in JRC ....................................................... 101 

Follow-up audit on the supervision of the implementation of CEF in DG MOVE ... 104 

Follow-up audit on the implementation of FP7 control systems (including supervision 
of external bodies) in DG RTD ....................................................................................... 104 

Follow-up audit of the set-up of the common support centre for H2020 ................... 104 

Follow-up audit on EDF grants in DG DEVCO ........................................................... 104 
5 

 
 
20. 

21. 

22. 

23. 

24. 

25. 

26. 

27. 

28. 

29. 

30. 

31. 

32. 

33. 

34. 

35. 

36. 

37. 

38. 

39. 

40. 

41. 

Follow-up audit on Budget Support in DG DEVCO .................................................... 105 

Follow-up audit of IAC recommendations in DG ECHO ............................................ 105 

Follow-up audit of IAC recommendations in DG NEAR ............................................ 105 

Follow-up audit on performance audit of National Agencies (DG EAC) ................... 106 

Follow-up audit on preparedness of DG HOME for 2014-2020 legislation in shared 
management (ISF and AMIF) ........................................................................................ 106 

Follow-up audit on knowledge management in DG COMP ........................................ 106 

Follow-up audit on the preparedness of the management and control systems for the 
SME instrument in EASME ........................................................................................... 106 

2nd Follow-up audit on HR management in response to the financial crisis in DG 
ECFIN .............................................................................................................................. 107 

Follow-up audit on risk management and planning processes in the new economic 
governance context in DG ECFIN ................................................................................. 107 

Follow-up audit of IAC recommendations in DG ECFIN ........................................... 107 

Follow-up audit of IAC recommendations in DG FISMA ........................................... 108 

Follow-up audit on the performance of DG GROW's supervision of ESA's 
implementation of Galileo ............................................................................................... 109 

Follow-up audit of IAC recommendations in DG GROW .......................................... 109 

Follow-up audit on the customs performance measurement system in DG TAXUD 109 

Follow-up audit of IAC recommendations in DG TAXUD ......................................... 110 

Follow-up audit on financial and procurement management in DG TRADE ........... 111 

Follow-up audit of IAC recommendations in OIB ....................................................... 111 

Follow-up audit of IAC recommendations in DG SCIC .............................................. 111 

Follow-up audit of IAC recommendations in DG ESTAT .......................................... 112 

Follow-up audit on management of local IT in DG AGRI .......................................... 113 

Follow-up audit on IT governance in DG Budget ........................................................ 114 

Follow-up audit on management of European Commission Authentication Service - 
ECAS ................................................................................................................................ 114 

42. 

Follow-up audit of IAC IT recommendations in DG DIGIT ....................................... 114 

6 

 
 
43. 

44. 

45. 

46. 

2nd Follow-up audit on management of local IT in DG ESTAT .................................. 115 

2nd Follow-up audit on management of local IT in DG MARE ................................... 116 

2nd Follow-up audit on management of local IT in DG TRADE ................................. 117 

Second follow-up to the performance audit on the Anti-fraud Information System 
(AFIS) by the former Internal Audit Capability at OLAF. ......................................... 117 

List of follow-up audits performed in 2016 for which all recommendations have been closed 

after the follow-up ........................................................................................................... 117 

PART 3: SUMMARY OF LONG OUTSTANDING RECOMMENDATIONS AS AT 31 JANUARY 2017 ...... 121 

7 

 
 
 
CONTENT OF THIS STAFF WORKING DOCUMENT: 

Part 1 of this  SWD contains the objective and scope, key findings  and the  critical and very 
important recommendations of the IAS engagements which were part of the 2016 IAS audit 
plan  (cut-off  date  for 
the  exercise:  31  January  2017).  Important  and  desirable 
recommendations  are  not  reproduced  in  this  SWD.  The  information  contained  in  this  SWD 
reflects the state of play when the audit engagements were finalised as stated in the executive 
summary  of  the  audit  report.  Each  executive  summary  underwent  the  applicable  standard 
professional validation and contradictory procedures between auditor and auditee at the time 
of  the  finalisation  and  aims  to  provide  a  quick  understanding  of  the  audits  and  their  main 
results. 

Part 2 of this SWD contains a summary of the IAS follow-up engagements in the period from 
1 February 2016 to 31 January 20171. 

Part  3  provides  a  summarised  overview  of  the  18  long  overdue  very  important 
recommendations as at 31 January 2017. 

1  

  The  summary  reflects  the  assessment  of  the  IAS  on  the  status  of  implementation  of  the  audit 
recommendations at the end of the follow-up assignment. It does not take into account any further actions 
that may have been undertaken by the auditee and reported to the IAS since the release of the IAS follow-up 
note or report, possibly having an impact on the status of the recommendation. 

8 

 
 
 
 
 
                                                            
PART 1: FINAL REPORTS 

1.  HORIZONTAL AUDITS 

1.1.   Audit on the management of intra-muros contractors 

Audit objectives and scope 

The overall objective of the audit was to assess if the Commission uses the external 
contractors working intra-muros in an effective and efficient way. The audit covered 
the arrangements both at the corporate level aimed at facilitating the management of 
intra-muros contractors by DGs and the way in which they are managed in practice at 
the individual DG level. 

At the corporate level, the audit scope included the overall framework put in place by 
DG  HR,  responsible  for  coordinating  the  personnel  and  administrative  policy  of  the 
Commission, as well as by DG BUDG, responsible for laying down the procurement 
procedures and contract templates. The audit also focussed on DG DIGIT, given that a 
large number of intra-muros contractors work in the IT domain. 

At  the  individual  DG  level,  the  audit  scope  included  the  process  of  monitoring  the 
work  performed  by  intra-muros  contractors  to  ensure  that  DGs  get  value  for  the 
money  spent  and  that  the  associated  risks  are  adequately  identified  and  appropriate 
mitigating measures put in place. 

There are no observations/reservations in the 2015 Annual Activity Reports of any of 
the DGs covered by this audit that relate to the area/process audited. 

The fieldwork was finalised on 25 May 2016. All observations and recommendations 
relate to the situation as of that date. 

Major audit findings 

The IAS identified two very important issues: 

•  Lack of a corporate framework for the use of intra-muros contractors  

Despite the significant number of intra-muros contractors in the Commission, there is 
no  corporate  framework  and  there  are  no  comprehensive  corporate  guidelines  to 
support DGs in their management. Certain DGs have developed guidelines at the local 
level, but these lack the corporate dimension needed to properly address not just the 
contractual issues, but also to allow them to take into account the HR implications of 
significant  dependency  levels.  The  need  for  such  corporate  guidance  was  clearly 
expressed by the audited DGs. 

•  Value for money in "time and means" contracts 

DGs/Services  can  use  two  major  categories  of  service  contracts.  Either  "result" 
contracts,  which  involve  buying  pre-defined  deliverables,  or  "time  and  means" 
contracts, which are used to purchase human resource capacity with certain skills for a 
given period of time, but without necessarily pre-defined deliverables.  

Although "time and means" contracts may be appropriate in certain circumstances, for 
example  in  the  early  stages  of  a  project  or  operational  service  in  which  an  output 

9 

 
 
cannot  be  clearly  defined  upfront,  they  tend  to  be  used  more  because  of  their 
flexibility  and  the  relatively  lighter  contract  preparation  work  needed  up  front. 
However,  they  generally  involve  much  less  stringent  reporting  requirements  on  the 
work  actually  performed  by  the  contractor  and,  unless  properly  monitored,  provide 
less  assurance  on  the  achievement  of  value  for  money.  Although  the  IAS  identified 
certain good practices  where framework contracts used  on a "time and  means" basis 
also included reporting requirements to allow implementation progress to be tracked, 
these  appear  to  be  exceptions  rather  than  the  rule.  Furthermore,  DG  DIGIT's  recent 
guidelines on outsourcing in the IT domain do not clearly advise DGs to use "result" 
contracts in preference over "time and means" contracts. 

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  A corporate framework for the use of intra-muros contractors  

The IAS recommends that the responsibility for defining the corporate framework for 
the  use  of  intra-muros  contractors  should  be  allocated  to  the  main  central  services 
involved  (DGs  BUDG,  HR  and  DIGIT),  under  the  general  oversight  of  the  ABM 
Steering  Group.  These  central  services  should  work  with  the  DGs  that  make  most 
significant  use  of  intra-muros  contractors  to  define  a  corporate  framework.  This 
should build upon the existing guidance at local level and be more specific in spelling 
out which types of contract are most applicable to different situations. 

•  Value for money in "time and means" contracts  

As  corporate  domain  leaders  in  IT  and  communication,  DGs  DIGIT  and  COMM 
should,  for  any  new  framework  contract  using  "time  and  means"  and,  to  the  extent 
possible, for existing ones, build in contractual safeguards aimed at ensuring value for 
money.  This  could  include  the  use  of  indicative  milestones  and/or  defining 
deliverables, together with reporting requirements on the activities performed by intra-
muros contractors.  

Finally, DG DIGIT should revise the recently finalised guidelines to clearly state that 
a reflection should be carried out prior to the launch of the procurement procedure and 
that "result oriented" contracts should be privileged over "time and means" contracts. 

1.2.   Audit on coordination and working arrangements with EU decentralised 
Agencies in DG SANTE and DG HOME 

Audit objectives and scope 

The overall objective of the audit was to assess the adequacy of the coordination and 
working arrangements of the partner DGs (HOME and SANTE) with their Agencies 
to  ensure  that  Agencies'  activities  contribute  efficiently  and  effectively  to  the  DGs' 
policy objectives. 

The  audit  focussed  on  the  following  three  areas:  (1)  the  clarity  of  the  role  and 
responsibility of the partner DG vis à vis its Agencies; (2) the adequacy of the overall 
strategy  of  the  partner  DG  vis  à  vis  its  Agencies  to  ensure  that  their  activities 
contribute efficiently and effectively to the achievement of the DG's policy objectives 

10 

 
 
and (3) the adequacy of the organisational structure of the partner DG in order to have 
efficient and effective interactions with its Agencies. 

As the areas under review are managed separately and under different organisational 
structures  in  the  partner  DGs  (HOME  and  SANTE),  two  separate  reports  have  been 
issued  by  the  IAS  for  clarity  purposes  and  to  facilitate  the  follow  up  of  the 
recommendations at partner DG level. 

There are no observations/reservations in the 2014 Annual Activity Reports (AAR) of 
the DGs that relate to the area/process audited. 

fieldwork  was 

The 
recommendations relate to the situation as of that date. 

finalised  on  15  January  2016.  All  observations  and 

Major audit findings 

The IAS identified one very important issue with regard to the overall strategy of each 
partner DG towards its Agencies in three key areas (i.e. programming, monitoring and 
control/reporting) as follows: 

DG HOME 

•  Partner DG's contribution to Agencies' programming and link to the DG's own 

programming activities 

The Commission (via DG HOME) has to provide an opinion on the work programmes 
of its Agencies as requested by the Agencies' Founding Acts. The IAS observed that 
the  work  programmes  of  certain  Agencies  lack  quality  in  terms  of  objective  setting 
(i.e. no SMART objectives) and the definition of Key Performance Indicators (KPIs) 
(i.e.  no  result  and  impact  indicators).  Furthermore,  some  issues  on  the  programming 
process have been noted: delays in providing the Commission's opinion, the need for 
closer  links  between  the  partner  DG  and  the  Agency  and  for  a  more  proactive 
Commission's  role  to  facilitate  the  timely  assessment  of  Agency's  budget  needs.  In 
addition,  despite  the  link  between  the  DG's  policy  objectives  and  the  outputs  of  the 
Agencies,  performance  indicators  are  not  systematically  included  in  the  DG's 
Management Plan which reflect the Agencies' contribution towards the achievement of 
DG  HOME's  policy  objectives.  Finally,  DG  HOME  does  not  explicitly  take  into 
account  the  extent  to  which  the  risks  reported  by  the  Agencies  may  hamper  the 
achievement of those policy objectives. 

•  Partner DG's monitoring of Agencies' activities 

The main mechanisms for monitoring the performance of decentralised Agencies are 
through  the  work  of  the  respective  Management  Boards  as  well  as  through  regular 
(informal) contacts. As is the case for other partner DGs, DG HOME is represented on 
the  Boards  of  its  Agencies.  Key  to  effective  monitoring  is  to  have  appropriate  KPIs 
and  adequate  performance  measurement  systems  and  reporting  at  Agency's  level. 
While this is the Agencies' responsibility, DG HOME has a vested interest that these 
arrangements  adequately  support  the  achievement  of  its  own  policy  objectives.  As 
noted above appropriate KPIs are not always in place and the quality and regularity of 
performance  measurement  and  reporting  in  the  Agencies  varies  considerably. 
Furthermore,  although  the  DG  has  supported  the  Agencies  in  implementing  certain 
actions under the 'Common approach' and associated 'Roadmap' aimed at making the 

11 

 
 
Agencies  more  coherent,  effective  and  accountable,  the  IAS  considers  that  it  could 
further strengthen its monitoring in this area. 

•  Partner  DG's  control  strategy  to  build  assurance  and  report  on  the  tasks 

entrusted to its Agencies in the framework of the Annual Activity Report 

DG HOME has not formalised yet its control strategy towards its Agencies, including 
a provision for adapting the intensity of controls to match the Agencies' respective risk 
profiles.  Furthermore,  no  'differentiated'  control  strategy  exists  for  the  two  Agencies 
which have been entrusted by DG HOME with budget implementation tasks through 
'Delegation  Agreements'.  The  risks  of  fraud  and  of  conflict  of  interest  (CoI)  are  not 
systematically monitored by the partner DG as a member of the Management Board. 
In addition, there is room for improvement in the way in which the DG builds up its 
assurance on the activities of the Agencies, due to the inefficient use of independent 
sources  of  assurance  such  as  evaluation  and  audit  work  but  also  due  to  the  limited 
nature  of  the  reporting  process  on  Agencies'  matters  by  the  AOSD  which  does  not 
systematically  report  on  the  results  of  the  Agencies'  activities  and  on  the  DG's 
monitoring arrangements towards its Agencies. 

DG SANTE 

•  Partner DG's contribution to Agencies' programming and link to the DG's own 

programming activities 

DG SANTE has a very limited role in the establishment of the work programmes of its 
Agencies. This is mainly due to the fact that under the Agencies' Founding Acts, there 
is no legal requirement for the DG to provide an opinion on the work programmes, but 
also  due  to  the  fact  that  in  some  cases  the  Agencies  allow  the  DG  only  a  limited 
opportunity  to  participate  in  the  discussions  at  an  early  stage  of  the  programming 
phase. The IAS observed that the work programmes of certain Agencies lack quality 
in terms of objective setting (i.e. no strategic objectives) and KPIs (i.e. no result and 
impact  indicators).  Furthermore,  despite  the  link  between  the  DG's  policy  objectives 
and  the  outputs  of  the  Agencies,  there  are  no  performance  indicators  in  the  DG's 
Management Plan which reflect the Agencies' contribution towards the achievement of 
DG SANTE's policy objectives. In addition, in its risk assessment, DG SANTE does 
not document the extent to which the risks reported by the Agencies may hamper the 
achievement of those policy objectives. 

•  Partner DG's monitoring of Agencies' activities 

The main mechanisms for monitoring the performance of decentralised Agencies are 
through the work of the respective Management Boards and Audit Committees (when 
applicable) as well as through regular (informal) contacts. To support these monitoring 
activities,  amongst  other  sources  of  information,  appropriate  performance  indicators 
should be in place. While this is the Agencies' responsibility, DG SANTE has a vested 
interest that these indicators adequately support the monitoring of the achievement of 
the DG's policy objectives. As noted above this is not always the case. Furthermore, 
although the DG has supported the Agencies in implementing certain actions under the 
'Common  approach'  and  associated  'Roadmap'  aimed  at  making  the  Agencies  more 
coherent, effective and accountable, the IAS considers that it could further strengthen 
its  monitoring  in  this  area.  Finally,  although  the  IAS  acknowledges  that  the  quality 
checks  performed  by  DG  SANTE  on  its  Agencies'  scientific  opinions  necessarily 

12 

 
 
follow  different  approaches  depending  on  the  particular  circumstances,  it  found  that 
neither the common principles nor the justification for the different approaches were 
documented.  

•  Partner  DG's  control  strategy  to  build  assurance  and  report  on  the  tasks 

entrusted to its Agencies in the framework of the Annual Activity Report 

The IAS notes that DG SANTE has formalised its overall control strategy towards its 
Agencies. Although this is risk-based, it does not describe the different Agencies' risk 
profiles and does not explain how the intensity of controls should be adapted to those 
risk profiles. In addition, there is room for improvement in the way in which the DG 
builds  up  its  assurance  and  reports  on  the  activities  of  the  Agencies.  Currently, 
independent  sources  of  assurance  such  as  evaluation  and  audit  work  are  not  used  as 
efficiently as they could be and the reporting made by the AOSD on Agencies matters 
focuses  more  on  budget  execution  than  on  results  linked  to  policy  achievements, 
operations and monitoring arrangements. 

Recommendations 

To address these issues, the IAS formulated the following recommendations for each 
partner DG: 

DG HOME 

•  Partner  DG's  contribution  to  Agencies'  programming  and  link  with  DG's 

programming  

DG HOME should reinforce its leverage effect on Agencies' programming. Firstly by 
being  involved  earlier  in  the  programming  phase  to  support  more  effectively  the 
Agencies for the setting of adequate objectives and the definition of appropriate KPIs 
and  secondly by  establishing closer links between Agencies  (i.e. field expertise)  and 
Commission (i.e. 'Policy') to ensure that lessons learnt can feed into all levels of policy 
development.  The  DG  should  also  reinforce  its  risk  assessment  process  by  taking 
account of the risks reported by the Agencies and strengthen its planning documents 
(i.e. Strategic Plan (2016/2020) and annual Management Plans), by explaining clearly 
how the Agencies activities contribute to the achievement of the policy objectives and 
how this is in turn supported/measured by appropriate indicators. 

•  Partner DG's monitoring of Agencies' activities  

Through  its  role  as  a  member  of  the  respective  Management  Boards,  DG  HOME 
should  strengthen  its  performance  monitoring  using  the  KPIs  established  by  the 
Agencies. It should further promote and support the implementation of the 'Common 
Approach' by its Agencies and follow up on the implementation of the 'Roadmap' in 
each individual Agency. 

•  Partner  DG's  control  strategy  for  building  assurance  and  report  on  the  tasks 

entrusted to its Agencies in the framework of the Annual Activity Report 

DG  HOME  should  strengthen  its  control  and  assurance  building  process  as  follows. 
Firstly, a control strategy should be formalised for the Agencies, allowing for different 
levels of control intensity in line with the Agencies' respective risk profiles. Secondly, 
a  separate  control  strategy  for  Agencies  with  delegated  budget  implementation  tasks 
(i.e.  Delegation  agreements)  should  be  established  as  the  discharge  in  respect  of  the 

13 

 
 
delegated  funds  is  given  to  the  Commission  (not  to  the  Agency)  and  the  Director 
General of DG HOME is the Authorising Officer by Delegation (not the Director of 
the  Agency).  Thirdly,  the  building  blocks  supporting  the  Authorising  Officer  by 
Delegation's declaration of assurance should be reinforced by a more efficient use of 
independent  sources  of  assurance  and  by  a  more  systematic  bottom-up  reporting 
process  aimed  at  ensuring  that  the  information  needed  for  the  DG's  AAR  on  the 
Agencies'  activities  is  reported  consistently  and  on  a  timely  basis  and  properly 
identifies issues which could have an impact on the declaration of assurance. Fourthly, 
the information included in the AAR on the Agencies' activities should be improved, 
particularly with regard to the main results and the contribution to DG HOME's policy 
objectives.  Finally,  the  DG  should  monitor  that  Agencies  establish  adequate  Anti-
Fraud  and  conflict  of  interest  policies  which  we  recommend  to  be  adopted  by  their 
respective Management Boards. 

DG SANTE 

•  Partner  DG's  contribution  to  Agencies'  programming  and  link  with  DG's 

programming  

DG SANTE should reinforce its leverage effect on Agencies' programming. Firstly by 
being  involved  earlier  in  the  programming  phase  to  support  more  effectively  the 
Agencies for the setting of adequate objectives and the definition of appropriate KPIs 
and  secondly  by  establishing  closer  links  between  Agencies  (i.e.  science/field 
expertise)  and  Commission  (i.e.  'Policy'),  while  respecting  both  the  independence  of 
the Agencies and the role of the Commission to ensure that lessons learnt can feed into 
all  levels  of  both  organisations.  The  DG  should  also  reinforce  its  risk  assessment 
process  by  documenting  properly  how  the  risks  reported  by  the  Agencies  are  taken 
into  account.  It  should  also  strengthen  its  planning  documents  (i.e.  Strategic  Plan 
(2016/2020) and annual Management Plans) by explaining clearly how the Agencies 
activities contribute to the achievement of the policy objectives and how this is in turn 
supported/measured by appropriate indicators. 

•  Partner DG's monitoring of Agencies' activities 

Through  its  role  as  a  member  of  the  respective  Management  Boards,  DG  SANTE 
should  strengthen  its  performance  monitoring  using  the  KPIs  established  by  the 
Agencies. It should further promote and support the implementation of the 'Common 
Approach'  by  its  Agencies  and  follow  up  on  the  implementation  of  the  Roadmap  in 
each individual Agency. The DG should ensure that the different approaches used as 
regards quality checks on Agencies' scientific opinions are properly documented and 
justified accordingly. 

•  Partner  DG's  control  strategy  for  building  assurance  and  report  on  the  tasks 

entrusted to its Agencies in the framework of the Annual Activity Report  

DG SANTE should strengthen its control and assurance building process as follows. 
Firstly,  the  control  strategy  should  describe  the  different  Agencies'  risk  profiles  and 
how the level of control intensity should be adapted to these risk profiles. Secondly, 
the building blocks supporting the AOD Declaration of assurance should be reinforced 
by a more efficient use of independent sources of assurance. Thirdly, there should be a 
more  systematic  bottom-up  reporting  process  aimed  at  ensuring  that  the  information 
needed for the DG's AAR on the Agencies' activities is reported consistently and on a 

14 

 
 
timely basis. Finally, the information included in the AAR on the Agencies' activities 
should  be  improved,  particularly  as  regards  the  main  results  and  the  contribution  to 
DG SANTE's policy objectives. 

1.3.   Audit on performance and coordination of Anti-Fraud activities in the 
Traditional Own Resources area 

Audit objectives and scope 

The overall objective of the audit engagement was to assess whether or not the Anti-
Fraud  activities  in  the  area  of  Traditional  Own  Resources  (TOR)  are  planned, 
managed and coordinated in an effective manner to ensure the best protection of the 
Commission's financial interests. 

The scope of this audit engagement covered the Commission’s Anti-Fraud activities in 
the TOR area with a particular focus on customs duties and cigarette smuggling.  

The audit covered: 

•   The  Commission  Anti-Fraud  Strategies  (CAFS)  and  high  level  coordination  and 

policy in the TOR area; 

•   Anti-Fraud Strategies (AFS) of the main DGs involved in TOR-related activities; 
•   Operational activities in the audited DGs to address the fraud risks at each stage of 

the Anti-Fraud cycle; 

•   Communication and information within the framework of Commission governance 
and  reporting  such  as  the  annual  risk  assessment  exercise,  the  Annual  Activity 
Reports (AARs), Management Plans (MPs), etc. 

The audit focused on the activities of OLAF, DG BUDG and DG TAXUD.  

There  are  no  observations  or  reservations  in  the  2015  AARs  of  the  audited  DGs, 
which relate to the audited process. 

The  fieldwork  was  finalised  on  30  September  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The IAS identified three very important issues: 

•  Anti-Fraud Strategies in own resources at Commission and DG level  

The  CAFS  and  the  individual  DGs'  AFSs  do  not  sufficiently  address  specific  fraud 
risks in the domain of EU own resources. In particular, the CAFS does not provide a 
clear  framework  for  fighting  fraud  in  the  own  resource  areas  as  a  whole  (including 
TOR), while concerning the AFSs, the TOR DGs do not coordinate their preparation 
to ensure that common fraud risks are adequately identified and addressed. 

•  OLAF support 

OLAF's  support  and  facilitation  activities  on  fraud  prevention  and  detection  in  the 
TOR  area  are  less  structured  and  comprehensive  than  in  the  expenditure  area.  In 
particular,  the  central  guidance,  support  and  coordination  provided  to  the  DGs,  the 

15 

 
 
 
training  programme  and  the  information  provided  in  the  Anti-Fraud  website  are 
mostly  focusing  on  the  expenditure  area  and  very  limitedly  on  TOR.  There  is 
moreover  no  working  group  or  forum  for  all  the  TOR  DGs  to  discuss  and  share 
common challenges and best practices in the TOR area. 

•  Roles and responsibilities in the TOR area 

There is no clear overview of how the TOR DGs share the Commission's competence 
for  fraud  prevention  and  detection  in  the  TOR  area,  and  how  they  ensure  effective 
cooperation and resolve strategic issues on fraud prevention and detection. In addition, 
the different committees with the Member States address Anti-Fraud aspects to a very 
limited extent, not all the TOR DGs attend them or are involved in the preparation of 
meetings to define a common EC position or propose issues for discussion. Lastly, the 
TOR DGs do not sufficiently coordinate their preparation and distribution of reports to 
the Member States on Anti-Fraud activities and performance. 

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  Anti-fraud Strategies in own resources at Commission and DG level 

OLAF  should  revise  the  CAFS  in  order  to  address  appropriately  issues  and  risks 
related to own resources, including TOR and facilitate a better coordination among the 
TOR  DGs  to  enable  identifying  common  risks  and  defining  coordinated  mitigating 
actions. 

•  OLAF support 

OLAF should strengthen its support to the TOR DGs by ensuring an enhanced service 
to TOR DGs as for expenditure DGs. This should include revising the AFS guidance 
in the  TOR  area, developing an appropriate range of awareness,  communication and 
training  tools  and  ensuring  that  the  Fraud,  Prevention  and  Detection  Network 
addresses TOR issues. 

•  Roles and responsibilities in the TOR area 

OLAF,  DG  BUDG  and  DG  TAXUD  should  better  cooperate  by  setting  up  a  strategic  steering 
function responsible for AFS in TOR, defining clearly the respective roles and responsibilities 
and  establishing  procedures  for  the  cooperation  among  them  (including  when  preparing 
reports  on  Anti-Fraud  activities).  The  DGs  should  also  review  and  formalise  the  different 
current practices for Member State committees and working groups. 

1.4.   Audit on the new Better Regulation agenda in the Commission - what is the state 
of play approximately one year after its adoption? 

Audit objectives and scope 

The  overall  objective  of  the  audit  was  to  assess  the  state  of  play  of  the  Better 
Regulation  (BR)  package  approximately  one  year  after  its  adoption.  Although  still 
relatively  early  for  such  an  important  and  wide  ranging  initiative,  with  expectation 
levels so high, it is important to assess the progress made so far, confirm or otherwise 

16 

 
 
that  it  is  on  track  and  to  highlight  as  early  as  possible  any  areas  for  possible 
improvement/corrective action. 

The audit scope included: 

• At the corporate level: the framework put in place to support the implementation of 
the BR package at Commission level and the measures taken by the SG so that DGs 
are  ready 
the  efficient  and  effective 
implementation of the package; 

to  manage,  monitor  and  report  on 

• At the DG level: the preparedness of a sample of DGs (EMPL, ENV and GROW) to 

efficiently and effectively implement the BR package in practice. 

• There  are  no  observations/reservations  in  the  2015  Annual  Activity  Reports  of  the 

audited services that relate to the area/process audited. 

The  fieldwork  was  finalised 
in  mid-June  2016  and  all  observations  and 
recommendations  relate  to  the  situation  as  of  that  date.  However,  the  situation  is 
continuously evolving and various factors and events have come to light since the end 
of  the  fieldwork.  These  have  been  taken  into  account  when  finalising  the  audit 
engagement. 

Major audit findings 

The IAS identified two very important issues: 

•  State of play of the main Better Regulation components 

Although  the  Commission  has  put  in  place  the  main  components  of  the  package 
(REFIT  -  Regulatory  Fitness  and  Performance  Programme  -  Platform,  Regulatory 
Scrutiny  Board,  feedback/consultation  mechanism  and  agenda  planning  etc.),  it  has 
yet  to  establish  proper  monitoring  and  measurement  arrangements  for  assessing 
whether these components are functioning adequately in practice. The IAS notes that 
the continual development of the supporting IT tools (the BR portal and Decide) will 
allow key data/statistics to be collected and indeed this is already underway. However, 
it  still  remains  to  be  decided  how  these  will  be  best  used  for  monitoring  and 
assessment purposes. 

As  regards  the  REFIT  platform,  the  IAS  found  that  the  working  arrangements  still 
need to be finalised, together with clearer explanations as to precisely what is expected 
from  the  platform.  At  the  fieldwork  date,  these  were  still  not  clear  to  the  platform's 
members. 

Concerning the Regulatory Scrutiny Board (RSB), there is a need for DGs to be better 
informed  about  the  quality  and  content  requirements  for  impact  assessments  and 
evaluations, as this would help them to prepare high-quality outputs from the outset. 
Furthermore,  the  RSB's  rules  of  procedure  and  working  arrangements,  which  were 
available only in draft at the time of fieldwork, still need to be finalised. 

Although the feedback and consultation mechanisms have been strengthened as part of 
an attempt to reach out to stakeholders, in practice this has proven to be a challenge as 
the response rate is, with a few notable exceptions, generally quite low. The language 
requirement appears to be a particular problem with less than 20% of the 2016 open 
public  consultations  being  made  in  all  EU  languages.  This  poses  a  natural  barrier  in 
the attempts to reach out to all EU citizens. In addition, the operational DGs audited 

17 

 
 
 
expressed concerns as to the proportionality of the feedback/consultation mechanism, 
although  the  IAS  acknowledges  that  it  may  be  too  early  to  draw  conclusions  in  this 
area in the absence of relevant performance information referred to above. 

As  regards  the  new  approach  for  planning  and  validating  major  initiatives,  the 
statistics available for 2016 at the time of the fieldwork show that the average time for 
the validation process is very encouraging overall, at ten working days, but about one 
quarter experienced considerable delays. In order to address this issue, the IAS notes 
that towards the end of the fieldwork the SG simplified the process and it expects the 
situation to improve.  

•  Fostering the Better Regulation culture 

The  new  impetus  that  the  BR  agenda  brought  also  requires  a  change  in  culture 
whereby the objectives and principles need to be deeply embedded in the regulatory 
activities  of  the  Commission.  To  this  end,  the  IAS  found  that  whilst  tools  and 
guidelines  have  been  made  available,  much  less  emphasis  was  given  in  practice  to 
helping foster the necessary change in culture. However, it notes that a communication 
strategy  was  developed  early  this  year  and  is  gradually  being  put  in  place  in  the 
context of the roll-out of the new BR portal. Furthermore, the audit identified a need 
to communicate more clearly on the workflow for policy development, the roles and 
responsibilities within the SG and the support that the SG is offering to the DGs.  

In  addition,  the  coordination  within  the  SG,  in  particular  between  the  Directorate 
responsible  for  the  BR  agenda  and  those  responsible  for  coordinating  the  policies 
throughout  the  Commission  needs  to  be  improved.  The  role  of  the  SG  is  pivotal  in 
fostering the BR culture and it follows therefore that the BR principles are understood, 
applied and communicated to the DGs in a coherent manner. The IAS also noted room 
for  improvement  with  regard  to  quality  review  by  the  SG.  In  particular,  supporting 
documents  to  guide  the  quality  review  are  not  used  consistently  and  there  is  no 
indicative timeline for the submission of documents. 

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  State of play of the main Better Regulation components 

SG and, where relevant, the RSB, should define appropriate performance measures for 
the main components of the BR package and monitor and evaluate these in practice. 
Furthermore,  the  SG  should  explain  to  the  REFIT  platform  members  more  precisely 
what is expected from them and from the process and should ensure that the working 
arrangements are agreed and properly understood amongst the platform's members. 

The  RSB,  in  collaboration  with  the  SG,  should  make  it  clear  to  the  DGs  what  is 
expected  of  them  in  terms  of  quality  and  content  for  impact  assessments  and 
evaluations and should also finalise its rules of procedure and working arrangements. 

Concerning  the  feedback/consultation  mechanisms,  the  SG  should  carefully  monitor 
progress, particularly as regards both the application of the language regime used for 
consultations,  and  the  proportionality  of  the  efforts  made  (inputs)  to  responses 
received  (outputs)  in  relation  to  both  consultation  and  feedback  mechanisms. 

18 

 
 
Furthermore, it should investigate the reasons for the low feedback rate and adapt the 
communication approach accordingly.  

Finally,  as  regards  the  planning  of  major  initiatives,  the  SG  should  monitor  the 
application of the new simplified process to assess whether the expected benefits are 
actually being achieved in practice and take any necessary remedial action. 

•  Fostering the Better Regulation culture 

To  better  foster  the  BR  culture  in  the  Commission  and  building  on  what  is  in  place 
already, the SG should further develop its communication strategy promoting the BR 
objectives. Particular emphasis should be placed on the importance of the 'tone at the 
top' and for Senior Management to be sending the right signals as to the importance of 
this initiative. This could be further complemented through stronger support from the 
political level. 

On  a  very  practical  level,  the  SG  should  clearly  set  out  (and  communicate 
accordingly) who does what and when, highlighting key review points for documents. 
Finally,  the  SG  should  strengthen  its  internal  coordination  and  quality  review 
arrangements to provide more consistent support to the DGs/services 

1.5.   Audit on financial management in the SG, LS, EPSC and DGT 

The  overall  objective  of  the  audit  was  to  assess  the  adequacy  of  the  financial 
management of the SG, LS, EPSC and DGT. In particular, it reviewed the design and 
the implementation of the controls in place to assess whether they ensure the legality 
and  regularity  of  the  financial  transactions.  Furthermore,  the  audit  assessed  the 
efficiency of the financial workflow. 

This  audit  covered  the  key  controls  designed  and  implemented  in  the  following 
processes: 

• The  procurement  process,  from  the  determination  of  the  needs  and  planning  to  the 

effective implementation of the contract; 

• The financial circuits of procurement, including commitments, payments (including 
payment deadlines) and recovery orders, to ensure proper segregation of duties and 
authorisation; 

• The recording of exceptions and ABAC access rights; 
• The risk register and Anti-Fraud strategy; 
• The reporting of the financial activity in the Annual Activity Report (AAR). 

The engagement covered the period 2015 and the first five months of 2016. There are 
no  reservations  in  the  2015  AAR  of  the  SG,  LS,  EPSC  and  DGT  that  relate  to  the 
area/process audited. 

The fieldwork was finalised during September and October 2016. All observations and 
recommendations relate to the situation as of that date. 

Major audit findings 

The SG, EPSC and DGT 

19 

 
 
 
The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

The LS 

The IAS identified one very important issue: 

•  Procurement process: Weaknesses in documentation 

The IAS found that as regards the procurement procedures relating to legal services, 
the documentation lacks a sufficient degree of formalisation to ensure a proper audit 
trail, as required by the Financial Regulation (FR). For example, there is no trail of the 
exclusion, selection and award criteria used for the procedure. In addition, exceptions 
to the FR and its rules of application (RAP) are not properly documented as required 
by the relevant internal control standard. Moreover, there is no formal evaluation and 
award  decision  and  the  relevant  manual  used  by  the  LS  is  very  brief  and  does  not 
explain  in  sufficient  detail  the  main  steps  to  be  followed  for  the  procurement 
procedure. 

The IAS also noted that the LS does not request a declaration on honour from the legal 
service contractors and is currently seeking the opinion of DG BUDG on the necessity 
to do so. 

Finally, the audit found that the requirements of the RAP regarding the publication of 
contracts awarded were not fully complied with. Their publication on the website of 
the LS was not exhaustive and did not meet the deadline as set in the RAP. 

Recommendation 

To address this issue, the IAS formulated the following recommendation: 

•  Procurement process: Weaknesses in documentation 

The LS should: 

• Develop a document (or further develop the existing template), which formalises the 
main steps of the procurement procedure for legal services, including exceptions to 
the FR and RAP; 

• Update its internal operational manual (Guide sur l'activité "contentieux" du service 
juridique  -  2016)  to  provide  more  detailed  guidance  to  the  legal  officers  in  this 
respect; 

• Clarify  with  DG  BUDG  whether  the  LS  has  to  request  the  ESPD/declaration  of 
honour  before  awarding  a  contract  for  legal  services  and  follow  the  position  as 
expressed by DG BUDG. For procurement procedures below EUR 15 000, carry out 
a risk assessment in order to assess whether or not to request these documents; 

• Publish a full list of all legal service contracts awarded in a given year respecting the 

deadline of 30 June year n+1. 

1.6.   Audit on the early implementation of ESIF control strategy 2014-2020 in DGs 
REGIO, EMPL and MARE 

Audit objectives and scope 

20 

 
 
The overall objective of the audit was to assess if the control strategy of DGs REGIO, 
EMPL  and  MARE  for  the  management  of  their  European  Structural  and  Investment 
Fund (ESIF) was properly designed, effectively implemented and well-coordinated in 
the early stages of the 2014-2020 programming period. 

The scope of the audit focussed on the following three main areas for the ESI funds 
managed by DGs REGIO (ERDF and CF), EMPL (ESF) and MARE (EMFF): 

• The appropriateness of the design of the control strategy for building up assurance on 

the management of the ESI funds for the 2014-2020 period; 

• The effective implementation of the control strategy in the early stages of the 2014-
2020  period  to  ensure  that  sufficient  assurance  is  available  before  reception  of  the 
first assurance packages with declared expenditure; 

• The  appropriateness  of  the  coordination  arrangements  between  the  three  DGs  (i.e. 
internal coordination) and with Member State authorities (i.e. external coordination) 
to  ensure  a  consistent  and  sound  control  approach  as  well  as  an  efficient  use  of 
resources in the early stages of the 2014-2020 period. 

The assessment of the IT systems used for the control/audit activities were excluded 
from the scope of the audit. 

There are no observations/reservations in the 2015 Annual Activity Reports (AAR) of 
DGs REGIO, EMPL and MARE that relate to the area/process audited. 

The fieldwork was finalised on 29 July 2016. All observations and recommendations 
relate to the situation as of that date. 

Major audit findings 

The IAS identified two very important issues: 

•  Design of the ESIF control strategy 2014-2020  

DGs' assurance process 

The  legal  basis  for  the  2014-2020  programming  period  introduced  a  number  of  new 
features  which  the  DGs  are  yet  to  fully  assess  in  terms  of  their  impact  from  a 
control/assurance  perspective.  In  particular,  this  concerns  the  impact  of  the  10% 
retention of interim payments on: (a) the procedure for interruption and suspension of 
payments, (b) the DG's decisions whether or not to issue a reservation in the AAR and 
its quantification and (c) the calculation of the 'amount at risk'. Also, it is not yet clear 
how  multi-fund  Operational  Programmes  (OPs)  will  be  treated  (a)  when  drawing 
conclusions  and  making  financial  corrections  based  on  audit  results  resulting  from 
common  samples  (i.e.  covering  both  funds)  and  (b)  when  defining  in  the  audit 
methodology the scope when covering those OPs. Finally, also not yet fully addressed 
is the control approach to be applied on the legality and regularity of payments under 
each stage of the control cycle. 

DGs' audit plans 

Delays in the start-up of the 2014-2020 programming period have resulted in the need 
for continuous adjustments of the DGs' audit plans. Although these are risk-based (in 
line with auditing standards) and properly supported by a workload analysis, there is a 
lack of consistency between the DGs on how to take into account resources shortages 

21 

 
 
 
when  developing  their  plans  and  it  is  not  clear  to  what  extent  those  shortages  might 
impact on the assurances needed for a given reporting period.  

'Control-related' simplification measures 

It  is  not  yet  sufficiently  clear  how  the  control  related  simplification  measures 
introduced in the 2014-20 programming period will deliver the expected results. Issues 
still  to  be  clarified  include:  (1)  article  148  of  the  Common  Provisions  Regulation 
(CPR)  on  'proportional  control  of  OPs',  setting  out  provisions  to  avoid  overlap  with 
Member States/ECA audits under certain conditions, (2) article 140(1) of the CPR on 
'shorter  retention  period  of  documents',  limiting  the  time  for  audit  activities  and 
financial corrections and (3) article 122(3) of the CPR on 'e-Cohesion', providing that 
by 31/12/2015 Member States have to exchange all information between beneficiaries 
and their national authorities by means of electronic data exchange systems. 

•  Implementation of the ESIF control strategy in the early stages  

Designation review  

The  Commission's  progress  in  reviewing  Member  States  designations  depends  very 
much on progress made by the Member States on the designation process itself, which 
is under their responsibility and has been subject to persistent delays. As at September 
2016, the Commission had received notice of complete designation for only 214 out of 
the 538 approved OPs (i.e. 40%). The DGs have provided guidance to help facilitate 
the designation process and some Audit Authorities found this to be useful. However, 
others  raised  concerns  about  the  feasibility  to  implement  the  so-called  'Light 
designation' for management and control systems, whereby the authorities concerned 
have essentially the same systems which existed in the previous period.  

The risk factors used by the Commission to select OPs subject to designation reviews 
(DR) are driven mainly by the amount of EU-co-financing at stake, rather than other 
factors such as the reliability of the Independent Audit Body (IAB). The IAS considers 
the latter to be more critical given the objective of the DR is to confirm the reliability 
of the IAB report. Also, although the DGs' methodology for the DR and guidelines to 
IABs  on  how  to  treat  IT  issues  at  the  designation  stage  are  clear  on  paper,  the  IAS 
found that in two out of the four sampled cases, either the DG's auditors (in one case) 
or  the  IAB  (in  the  second  case),  had  not  completely  followed  the  set  procedures  in 
practice. 

Early Preventive System Audits (EPSA) 

Concerning the EPSA methodology, the IAS notes that the impact of such audits on 
the  application  of  article  148  of  the  CPR  (which  limits  the  audits  that  can  be 
performed  on  the  same  beneficiary  by  the  DGs,  the  AA  and  the  ECA)  has  not  been 
properly reflected in the methodology. Also, the DGs have yet to update the checklist 
to verify compliance with the EU public procurement directive to cover the contracts 
published  from  April  2016.  Additionally,  the  risk  assessment  process  used  to  select 
OPs for such audits is lacking in so far as the decision-making is not always clearly 
documented and weightings are not adequately assigned to certain risk factors.   

Review of National Audit Strategies (AS) 

The  DGs'  review  of  the  AS  is  risk-based  and  includes  the  AS  of  the  OPs  which  are 
subject to DR. However, the DR automatically excludes lower-value OPs (i.e. below 
the thresholds in the legal basis) even though these may have a high-risk profile. The 
22 

 
 
IAS  notes  that  the  DGs  do  not  currently  plan  to  review  the  AS  for  certain  OPs 
identified  as  risky  according  to  the  EPSA  risk  assessment.  Also,  there  is  room  for 
improvement on the process and tools used for monitoring the reviews of AS. 

Thematic  audits  on:  (1)  Performance  Data  Reliability  (PDR)  and  (2)  Financial 
Instruments (FI)  

The  single  audit  strategy  attaches  priority  in  the  first  years  of  implementation,  in 
addition  to  the  compliance  audits,  to  audits  on  the  reliability  of  data  in  the  Member 
States and on Financial Instruments. However, at the end of the IAS audit fieldwork, 
in  REGIO  and  MARE  there  had  been  no  audits  on  assessing  data  reliability  and  a 
number  of  underlying  methodological  issues  have  yet  to  be  resolved  in  the  existing 
methodology  developed  jointly  by  the  DGs.  These  include  the  question  as  to  which 
performance  indicators  should  be  included  in  the  scope  of  the  audit,  the  risk  factors 
used for selecting OPs, the extent to which desk officers and evaluation experts can be 
used  and  finally  the  impact  on  suspension/interruption  of  payments  and  on  financial 
corrections. Concerning Financial Instruments, there is no audit methodology in place 
yet and no audits performed so far. However, the IAS notes that a working group has 
been set up to develop a methodology for mid-2017. 

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  Design of the ESIF control strategy 2014-2020 

DGs REGIO, EMPL and MARE should clarify: (1) the impact of the 10% retention 
from  interim  payments  on  the  interruption  and  suspension  of  payments,  on  the 
calculation of the 'amount at risk' and when deciding on the need to qualify the annual 
declaration of assurance by a reservation; (2) the impact of the 'multi-fund' OPs on the 
sampling method and the scope of the enquiry planning memorandum when covering 
those  OPs  and  (3)  the  control  approach  and  level  of  assurance  for  each  type  of 
payment and for each stage of the control cycle and clearly disclose this in the AAR. 

The DGs should revise their audit plans for the 2016-June2017 period to address any 
changes needed as a result of new events (e.g. new AA system audits reported). The 
plans  should  be  either  aligned  to  the  resources  available  or  alternatively  explain  the 
impact of any shortages in resources on the level of assurance in the reporting year.  

Finally,  the  DGs  should  address  some  points  resulting  from  the  control-related 
simplification  measures:  (1)  clarifying  the  sampling  implications  and  a  process  to 
exchange information on samples at beneficiary level, so as to avoid overlaps between 
audits on the same beneficiary by the DGs, the AAs and, under certain conditions, the 
ECA as per article 148 of the CPR; (2) consideration of the time limit for audit as per 
article 140.1 of the CPR in the risk assessment used for the selection of the OPs to be 
audited and (3) addressing the potential audit detection risk resulting from the use of 
'e-cohesion' through audit work on this topic. 

•  Implementation of the ESIF control strategy in the early stages 

DGs REGIO, EMPL and MARE should: 

23 

 
 
• 'Designations': in the short term, better facilitate the designation process through, for 
example, bilateral contacts with the Member States, giving priority to the risky OPs 
selected  for  designation  review.  In  the  long  term,  the  DGs  should  assess  the 
experiences  of  the  2014-2020  designation  process  to  draw  lessons  and  define  the 
control approach for the post-2020 legislative framework; 

• 'EPSAs':  strengthen  the  methodology  and  risk  assessment  process,  including 
improving the audit trail and attaching a higher weight to the reliability of the AA's 
work; 

• 'Review  of  national  audit  strategies':  include  as  part  of  their  review  the  additional 
high risk OPs identified in the EPSA risk assessment. DGs should also improve the 
existing tools and further develop the monitoring process; 

• 'PDR  audits':  strengthen  the  methodology  by  clarifying  the  scope,  the  role  of  desk 
officers and evaluation experts and the impact of any errors detected and ensure that 
such audits are carried out on the selected OPs as a matter of priority; 

• Audits  on  'Financial  Instruments':  develop  the  necessary  methodology  and  launch 
audits  as  soon  as  possible,  after  taking  due  consideration  of  any  audit  work  by  the 
AA on Financial Instruments to respect the "single audit principle" and based on the 
first substantial data on Financial Instruments reported by the Member States. 

1.7.   Audit on effectiveness of simplification measures under 2014-2020 ESI Funds in 
DG EMPL, REGIO and MARE 

Audit objectives and scope 

The overall objective of this audit was to assess whether or not DGs REGIO, EMPL 
and MARE have put in place the necessary processes to ensure that the simplification 
measures  introduced  in  the  2014-2020  regulatory  framework  are  effective  in 
achieving  the  objective  of  reducing  the  administrative  burden  (at  beneficiary  and 
Member State level), whilst at the same time obtaining the necessary assurances on 
legality  and  regularity  of  transactions  and  performance  of  programmes.  The  IAS 
audit focused on the following three areas: 

• The  appropriateness  of  the  design  of  the  processes  for  implementing  simplification 

measures; 

• The  activities  of  the  European  Structural  and  Investment  Funds  (ESIF)  DGs  in  the 
areas  of  promoting  the  use  and  monitoring  the  take-up  of  simplification  measures 
and in identifying any weaknesses in their implementation; 

• The  DGs'  efforts  to  identify  any  further  simplification  measures  as  well  as  take 

action in order to address the identified weaknesses in the existing measures. 

The scope of the Commission's simplification exercise for the multi-annual financial 
framework period 2014-2020 encompasses the Member States' national authorities as 
well as the beneficiaries of ESIF grants. It does not include simplification measures at 
the Commission level. The main simplification measures covered by this audit are: 

• Measures  related  to  simplifying  cost  reimbursement  rules,  notably  Simplified  Cost 
Options (SCO), simplifying eligibility rules and reducing "gold plating", and simpler 
rules for revenue generating projects; 

• Joint Action Plans (JAP); 

24 

 
 
 
• e-Cohesion. 

Another 2016 IAS audit on 'Early implementation of ESIF control strategy 2014-2020 
in DGs REGIO, EMPL and MARE' has covered the design aspects of several control 
related simplification measures. 

There are no observations/reservations in the DGs' 2015 Annual Activity Reports that 
relate to the area/process audited. 

The  fieldwork  was  finalised  on  28  September  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The IAS identified two very important issues: 

•  Uptake  and  impact  of  simplification  measures  and  the  DGs'  processes  to 

promote and monitor these measures  

The  provisional  results  of  a  DG  REGIO  study  show  that  the  reduction  in 
administrative costs is likely to be lower than expected, largely as a result of the lower 
than  expected  uptake  of  a  number  of  simplification  measures.  A  notable  exception 
concerns the ESF, where the expected uptake of SCO represents some 36% of the total 
ESF funding. This compares to some 2% for the ERDF/CF, and zero for the EMFF. 
For the ESF, this is a significant increase in comparison to the 7% uptake rate for the 
2007-2013  programming  period,  but  still  below  the  ambitious  target  of  50%  set  for 
this period. For the other Funds, the DGs have not set any targets for the 2014-2020 
programming  period  and  there  is  no  significant  increase  in  the  use  of  SCO  yet  in 
comparison  to  the  previous  period.  Furthermore,  the  e-Cohesion  requirements  were 
fulfilled  for  only  58%  of  the  ERDF/CF  Operational  Programmes  (OPs)  as  at  31 
December  2015.  No  information  was  available  on  the  uptake  of  this  measure  in  the 
case of the ESF. In addition, there has been no take up of JAP so far. 

Despite the ESIF DGs' efforts, they have not yet succeeded in overcoming a number 
of obstacles to further increase the uptake of simplification measures and to reduce the 
administrative  burden  of  beneficiaries  and  the  administrative  costs  of  the  Member 
States. 

The expected uptake of SCO is very much dependent on the type of projects funded. 
ESF  funded  actions  are  often  more  suitable  for  applying  SCO.  For  ERDF/CF  and 
EMFF, it is often not feasible to use flat rates, unit costs or lump sums. According to a 
survey  of  ESF  Management  Authorities  carried  out  by  DG  EMPL,  all  intend  to  use 
SCO  in  this  programming  period.  However,  other  surveys/studies  indicate  that 
respondents  have  certain  doubts  about  the  attractiveness  of  SCO  that  need  to  be 
addressed. 

Furthermore, SCO under article 67 Common Provisions Regulation cannot be used for 
operations  that  are  fully  publically  procured.  This  limits  the  potential  for  the  further 
uptake of SCO for ERDF/CF and EMFF. The different rules applicable to State Aid 
and simplification measures have not yet been sufficiently clarified and explained. 

The  lengthy  adoption  procedure  of  delegated  acts  under  article  14(1)  of  the  ESF 
Regulation is among the blocking factors for Member States opting for SCO. The lack 

25 

 
 
 
of  legal  certainty  on  the  Commission  accepting  the  Member  States  SCO  calculation 
methodologies  is  also  an  obstacle  the  DGs  need  to  overcome,  except  for  DG  EMPL 
when applying article 14(1) of the ESF Regulation. 

The  DGs  lack  a  comprehensive  analysis  of  the  Member  States'  rules  and  procedures 
implementing  the  ESI  Funds  at  the  local  level  to  be  able  to  help  them  reduce  gold-
plating in general and assess if the target for reducing gold-plating at the national level 
can  be  reached.  Furthermore,  overall,  corrective  actions  for  a  significant  number  of 
Management  Authorities  not  yet  complying  with  the  e-Cohesion  legal  requirements 
are not yet sufficiently clear. 

Finally,  whilst  acknowledging  that  it  is  early  in  the  programming  period,  the  IAS 
found  a  number  of  weaknesses  in  the  arrangements  the  DGs  have  put  in  place  for 
monitoring the uptake and impact of simplification measures. 

•  Mitigating risks associated with simplified cost options  

SCO  expose  the  ESIF  DGs  to  a  number  of  risks  they  will  need  to  address  in  the 
current  programming  period.  The  impact  assessment  supporting  the  legislative 
proposal for the 2014-2020 programming period did not sufficiently assess the impact 
of applying simplified rules on the level of assurance on legality and regularity to be 
obtained when using these new instruments. Furthermore, it is not certain if the new 
flat rates introduced in the Omnibus Regulation are a reliable proxy for real costs of 
certain types of funded operations,  as these have not been backed-up by an in-depth 
study into the various types of cost categories that comprise the financed operations of 
the ESI funds. 

Applying article 14(1) of the ESF Regulation does not guarantee simplification for the 
final beneficiaries. Firstly, national managing authorities can use a SCO to reimburse 
beneficiaries  which  is  different  to  the  one  approved  under  article  14(1)  or  can 
reimburse  them  based  on  actual  costs  incurred,  necessitating  a  double  accounting 
system adding administrative burden. Secondly, beneficiaries have to keep a full audit 
trail  when  the  operations  are  financed  by  more  than  one  ESI  fund  and  the  Member 
State has chosen a SCO provided under article 67 of the CPR for part of the financed 
operation  (i.e.  when  the  operations  are  'cross-financed').  The  IAS  has  also  found 
certain weaknesses in the procedures for applying article 14(1), mainly regarding key 
supporting  documentation.  Furthermore,  the  ESIF  DGs  have  yet  to  develop  their 
approach for assessing the continued relevance of the methodologies approved ex-ante 
under article 14(1) and possible over or under reimbursements to beneficiaries if they 
take another form than the SCO applied under article 14(1) for the reimbursement of 
Member States by the Commission. DG EMPL considers that the principle behind the 
use of article 14(1) ESF SCO means that this is not required. 

The use of SCO does not necessarily result in a stronger focus on results, as Member 
States  using  article  14(1)  have  chosen  to  be  reimbursed  based  on  process  or  output 
based  indicators  rather  than  results  in  several  cases.  Furthermore,  previous 
Commission audits have identified risks concerning the reliability of performance data 
collected  and  reported  by  the  Member  States,  but  it  is  unclear  how  the  DGs  and 
national  audit  authorities  intend  to  cover  the  reliability  of  performance  measurement 
indicators for SCO. 

It  is  also  not  yet  clear  how  the  DGs  will  keep  an  overview  of  all  the  findings 
concerning  SCO  resulting  from  audit  work  in  order  to  be  able  to  identify  systemic 

26 

 
 
issues and identify the need for a thematic approach to audit SCO. Finally, the role of 
the national audit authorities in providing assurance on the SCO calculation methods 
is yet unclear, as the applicable Regulations are silent about their precise role in this 
area. Certain national audit authorities are reluctant to get involved in assessing these 
methods because they fear that this would endanger their independence.  

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  Uptake  and  impact  of  simplification  measures  and  the  DGs'  processes  to 

promote and monitor these measures 

DGs EMPL, REGIO and MARE should, for the 2014-2020 period, further remove the 
above obstacles hindering the implementation of simplification measures, monitor the 
uptake and effectiveness of the simplification measures further along the programming 
period and take corrective measures where necessary. For the post 2020 period, they 
should gather up-to-date data on the Member States' progress regarding simplification 
before  submitting  their  legislative  proposals  for  the  post  2020  period.  They  should 
also set targets and indicators for the improved uptake of simplification measures and 
ensure  that  these  are  translated  into  the  different  funding  priorities  and  OPs  for  the 
post 2020 period to enable both the Member States and the Commission to monitor the 
Member States' actions in the area of simplification. 

•  Mitigating risks associated with Simplified Cost Options  

DG EMPL should further strengthen its procedures for approving the Member States' 
SCO  methodologies  under  article  14(1)  of  the  ESF  Regulation.  The  DG  should  also 
analyse  any  potential  instances  of  significant  differences  between  reimbursement  by 
the EC of the Member State and payments made by the Member State to beneficiaries 
to assess the underlying reasons and decide whether or not the approved SCOs need to 
be adjusted for future operations, where appropriate. 

DGs EMPL and REGIO should ensure that their own or the national audit authorities' 
audit work sufficiently covers the risks related to using an SCO throughout the 2014-
2020 programming period, if necessary through thematic audit work. They should also 
ensure that the SCO related data/indicators are output/results based where possible and 
their quality is sufficiently covered by audit work. For the post 2020 period the DGs 
should  properly  assess  the  effects  of  simplification  on  the  assurance  on  legality  and 
regularity  of  the  underlying  transactions  and  performance,  and  analyse  the  cost 
profiles  and  real  costs  incurred  by  publically  financed  projects  that  have  similar 
characteristics to those funded under the ERDF/CF, ESF and EMFF to provide a solid 
basis for calculating the flat rates proposed in the EU Regulations. 

1.8.   Audit on the processes for managing and sharing data on agri-environmental-
climate issues in DG AGRI, DG CLIMA and DG ENV 

Audit objectives and scope 

27 

 
 
The overall objective of the audit was to assess whether DG AGRI, DG CLIMA and 
DG ENV have put in place effective and efficient processes for managing and sharing 
agri-environmental-climate data. 

Agri-environmental-climate data was defined for the purpose of this audit as data and 
information related to the impact of agriculture on the environment and climate. 

The concept of knowledge management, which involves elements over and above the 
simple sharing of data, namely the use of skills and expertise needed  to analyse and 
interpret data, was not included in the scope of the audit. 

The  audit  covered  the  review  of  the  following  processes  in  DG  AGRI,  DG  CLIMA 
and DG ENV: 

• Processes  for  identifying  and  prioritising  agri-environmental-climate  data  needs  for 

policy support; 

• Processes for collecting agri-environmental-climate data, including the identification 

and mapping of available data; 

• Processes for storing, sharing and disseminating agri-environmental-climate data. 

The  audit  also  included  the  review  of  the  collaboration  between  DG  AGRI,  DG 
CLIMA and DG ENV and with other Commission services and European Union (EU) 
bodies  that  play  a  major  role  in  the  collection  and  dissemination  of  agri-
environmental-climate  data,  in  particular  Eurostat,  the  Joint  Research  Centre  (JRC), 
DG RTD and the European Environmental Agency (EEA). However, the audit did not 
cover the data management and sharing processes in these other Commission services 
and EU bodies. 

The  audit  work  took  into  account  the  rules  and  regulations  regarding  access  to 
documents, protection of personal data and confidential statistical data and protection 
of  intellectual  property  rights.  However,  its  primary  purpose  was  not  to  assess 
compliance with these rules and regulations. 

The  2015  Annual  Activity  Reports  of  DG  AGRI,  DG  CLIMA  and  DG  ENV  do  not 
contain any reservation/observation related to the processes audited. 

The  fieldwork  was  finalised  on  11  November  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The IAS identified two very important issues: 

•  Mapping  of 

information  needs  and  available  data  related 

to  agri-

environmental-climate issues  

Despite  certain  initiatives  undertaken  by  the  DGs  to  list  information  needs  and 
available  data,  there  is  no  comprehensive  and  coordinated  inventory  of  information 
needs, together with a list of already available data in the field of agri-environmental-
climate  issues.  Existing  inventories  are  limited  in  scope  and  not  always  shared 
amongst  the  three  DGs.  As  a  consequence,  the  IAS  noted  during  its  review  of  a 
sample of contracts for procuring studies in the agri-environmental-climate field, that 
it was often left to the contractors to make an inventory of available data/ information, 

28 

 
 
 
including  data/  information  produced  by  the  Commission  itself,  by  the  EEA  and 
through EU research projects. 

In the addition, current coordination mechanisms do not always function effectively. 
The  IAS  found  in  particular  that  there  is  insufficient  coordination  on  the  indicators 
related  to  agri-environmental-climate  data  and  that  DG  AGRI,  DG  ENV  as  well  as 
Eurostat  and  the  EEA  have  developed  indicators  which  either  address  the  same 
information  needs,  but  are  formulated  differently  or,  should  be  the  same,  but  in  fact 
are calculated using different sources and/or methodologies and hence lead to different 
results. 

•  Coordination of Member State reporting requirements and reuse of data 

There  is  insufficient  coordination  of  the  Member  State  reporting  requirements, 
including insufficient reuse of collected data. This results in overlaps in Member State 
reporting  requirements,  increase  in  the  workload  and  possible  inconsistencies.  The 
DGs informed the IAS that in some cases this was caused by insufficient coordination 
in  Member  States  themselves,  coupled  with  resistance  on  their  part  to  build  into  the 
underlying legislation the need for consistency between the different reports/data they 
are responsible for.  

In  addition,  the  IAS  noted  that  the  spatial  data  collected  by  Member  States  (and 
belonging  to  Member  States)  under  the  Common  Agricultural  Policy  (CAP)  control 
system  and  which  could  be  useful  for  environmental/climate  policy,  is  not  in  fact 
available  to  be  used  in  this  way  in  certain  Member  States  or  in  the  Commission.  In 
practice, this data is used essentially for controlling the CAP on the basis that under 
the  personal  data  protection  rules,  as  recalled  in  the  CAP  horizontal  regulation, 
personal data should not be used for another purpose than it was collected for. These 
restrictions  occur  in  spite  of  the  requirements  of  the  Inspire  Directive  for  sharing 
spatial data for environmental purposes. 

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  Mapping  of 

information  needs  and  available  data  related 

to  agri-

environmental-climate issues 

DG AGRI, DG CLIMA and DG ENV should (taking account of the role played by the 
main  EU  data  providers  and  building  on  existing  arrangements)  reinforce  the 
coordination of agri-environmental-climate data and related indicators and enhance its 
sharing.  They  should  also  establish  a  coordinated  inventory  of  agri-environmental-
climate information needs and available data. 

•  Coordination of Member State reporting requirements and reuse of data 

DG AGRI, DG ENV and DG CLIMA should:  

Actively  coordinate  between  themselves  and  with  the  EEA  and  Eurostat  to  ensure 
better consistency and, where possible, simplification through more effective re-use of 
collected  data  in  Member  States  reporting  requirements.  In  particular,  this  can  be 
included  in  the  European  Commission's  Regulatory  Fitness  and  Performance 

29 

 
 
Programme  (REFIT)  aimed  at  making  EU  law  simpler  and  reducing  the  regulatory 
costs. 

In addition, DG AGRI, DG CLIMA and DG ENV should: 

Clarify with the Legal Service what can be legally required from Member States under 
EU legislation, regarding the sharing of CAP spatial data between public authorities at 
national 
for 
environmental/climate purposes. 

the  European  Commission  and 

level  and  with 

the  EEA 

Upon  clarification  of  the  Legal  Service,  work  together  and  with  Member  States  to 
define  clear  arrangements/processes  for  the  sharing  of  the  CAP  spatial  data  for 
environmental-climate purposes. 

1.9.   Audit on the procurement process in OIB, OIL and DG BUDG 

Audit objectives and scope 

The  overall  objective  of  the  audit  was  to  assess  the  adequacy  of  the  design  and  the 
effective  implementation  of  DG  BUDG,  OIB  and  OIL's  internal  control  systems  for 
the  management  of  the  procurement  process  and  the  effectiveness  and  efficiency  of 
the related financial circuits. 

This  audit  tested  the  key  controls  as  well  as  management  and  monitoring  controls 
throughout the procurement process, from the identification and planning of the needs 
until  the  signature  of  the  contract,  including  amendments  and  price  revisions,  if 
applicable. 

The  audit  covered  the  controls  on  the  financial  transactions  in  the  period  1  January 
2015 – 31 May 2016 related to procurement procedures awarded in the same period as 
well as payments for procurement processed in the same period, which may however 
not  necessarily  be  linked  to  procurement  procedures  awarded  in  the  period  under 
review. 

There are no reservations in DG BUDG, OIB or OIL's 2015 Annual Activity Reports 
relating to the process audited. 

fieldwork  was 

The 
recommendations relate to the situation as of that date. 

finalised 

in  November  2016.  All  observations  and 

Major audit findings 

The IAS identified four very important issues: 

OIB 

•  Procurement procedures  

The  audit  identified  weaknesses  in  the  planning  phase  (i.e.  needs  analysis)  and  in 
relation to the transparency of public procurement procedures. 

During  testing  of  the  planning  phase,  the  IAS  found  weaknesses  in  OIB's  needs 
analysis in one of the eleven high value procedures  tested. In this case, OIB did not 
sufficiently consider local building regulations before the start of the procedure. As a 

30 

 
 
consequence, OIB could not sufficiently demonstrate the proper use of an exceptional 
procurement procedure. The IAS also noted that the real estate procedure currently in 
place (the 'Kallas procedure') has not been updated to reflect the changes made to the 
Financial  Regulation  that  requires  a  greater  degree  of  involvement  of  the  Budgetary 
Authorities.  The  'Kallas  procedure'  is  currently  being  revised  and  OIB  has  already 
prepared a first draft. 

Transparency is one of the fundamental principles in public procurement law. In this 
respect,  the  audit  found  that  in  the  real  estate  procedure  tested,  OIB  excluded  one 
tenderer  without  formally  explaining  the  grounds  on  which  the  decision  was  taken. 
However, a bilateral meeting was arranged at a later stage with the tenderer excluded. 
The IAS also noted that OIB did not take minutes of the meetings held with tenderers 
at the initial stages of the negotiation phase. 

•  Ex-post controls 

The audit also identified weaknesses in the methodology applied to ex-post controls. 
In particular, no pre-determined risk factors are defined and applied to the additional 
risk based sample of OIB. Furthermore, OIB does not make use of advanced sampling 
techniques, such as stratification of the population, which could increase the efficiency 
of the ex-post control function. 

In addition, the IAS found that the OIB had not tested the full number of transactions 
required for statistically meaningful results. This occurred because OIB made certain 
assumptions when selecting the sample during the year without assessing the need to 
adjust  it  at  year-end  in  order  to  take  account  of  significant  differences  between  the 
assumptions  made  and  the  actual  situation.  Also,  it  did  not  correctly  extrapolate  the 
errors  found  in  the  sample  across  the  entire  population.  The  IAS  does  acknowledge 
however, that this had no significant impact on the 2015 error rate. 

Furthermore,  although 
receive 
recommendations  from  the  ex-post  control  team,  they  do  not  establish  action  plans 
which could help foster the timely implementation of these recommendations. 

to  ex-post  control 

subjected 

services 

the 

Finally, the IAS noted that a high number of recommendations are still open some of 
which  were  classified  as  errors  of  importance  up  to  level  2  which  means  that  they 
could  have  a  financial  or  reputational  impact.  However,  OIB  has  not  carried  out  an 
assessment  of  the  actual  risks  that  the  office  is  facing  by  not  implementing  these 
recommendations and whether it would be cost-effective to do so. 

OIL 

•  Procurement procedures 

The  audit  identified  weaknesses  mainly  concerning  the  initial  steps  of  procurement 
procedures, namely the planning phase and the drafting of tender specifications.  

As  regards  the  planning,  the  IAS  found  that  that  OIL  does  not  provide  sufficient 
information  on  how  such  needs  have  been  quantified,  which  meant  in  practice  that 
OIL  could  not  sufficiently  justify  using  exceptional  procedures.  Furthermore,  in  one 
particular case, services vital for the implementation of a contract were not included at 
the needs assessment stage and consequently the tender specifications. They needed to 
be estimated by the Commission services at the evaluation stage. 

31 

 
 
Regarding the tender specifications, the audit found that in one case, these were overly 
specific and even included brands, which had the effect of limiting the competition. In 
one of the real estate procedures we tested, the award criteria were not clearly defined, 
although this constitutes a key element of the procurement procedure.  

•  Ex-post controls 

The audit also identified weaknesses in the methodology applied for ex-post control. 
In  particular,  OIL's  ex-post  controls  do  not  cover  procurement  procedures.  Hence,  a 
key  risk  is  not  covered,  even  though  this  constitutes  a  key  building  block  for  the 
assurance of the Authorising Officer by Delegation. In addition, the statistical method 
for  selecting  the  sample  and  the  extrapolation  of  the  error  rate  were  not  correctly 
applied in practice.  

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

OIB 

•  Procurement procedures  

OIB  should  improve  the  needs  analysis  by  including  compliance  with  specific 
building laws and regulations.  

Furthermore, OIB should take the necessary steps to launch a College decision on the 
revision of the 'Kallas procedure' by taking into account, inter alia, the changes to the 
Financial Regulation by including the new procedure as stipulated in Art. 203 of the 
Financial  Regulation  and  Art.  286  of  the  Rules  of  Application  to  the  Financial 
Regulation.  All  relevant  actors  in  the  field  of  real  estate  procurement  should  be 
consulted during the course of the revision. 

OIB  should  also  formally  justify  to  tenderers  when  the  decision  is  taken  to  exclude 
any one of them. 

•  Ex-post control  

OIB  should  comprehensively  document  its  sampling  methodology,  in  particular  the 
risk  factors  applied  for  the  additional  risk  based  sample.  Furthermore,  OIB  should 
consider  stratifying  the  sampled  population  to  increase  the  efficiency  of  the  sample 
testing.  

It should also assess the need to adjust the selection of the sample at year-end to take 
account  of  significant  differences  between  the  assumptions  made  and  the  actual 
situation as required by the Monetary Unit Sampling technique and correctly apply the 
statistical methodology when extrapolating the errors identified. 

Finally, the services that are subjected to the ex-post control should establish an action 
plan with target dates for the most significant recommendations made by the ex-post 
control  team.  Furthermore,  OIB  should  make  an  assessment  of  the  risks  associated 
with open recommendations for the lesser important errors and evaluate whether it is 
still cost-effective to implement these. 

OIL 

32 

 
 
•  Procurement procedures 

OIL  should  improve  the  needs  analysis  and,  in  particular,  document  how  it  has 
quantified  its  needs.  When  drafting  tender  specifications,  it  should  avoid  restrictive 
clauses  or  references  to  brands  or  trademarks,  except  for  duly  substantiated 
exceptions.  Furthermore,  OIL  should  also  set  clear  award  criteria  for  real  estate 
procurement procedures. 

•  Ex-post control  

OIL should include the key steps of the public procurement procedures in the scope of 
its  ex-post  controls  and  correctly  apply  the  statistical  method  when  sampling  and 
extrapolating the errors identified. 

1.10. 
(SCIC) and EPSO/EUSA 

Audit on the procurement process in DG COMM, DG Interpretation 

Audit objectives and scope 

The  overall  objective  of  the  audit  was  to  assess  the  adequacy  of  the  design  and  the 
effective  implementation  of  the  service's  internal  control  system  with  regard  to  the 
management  of  the  procurement  process  and,  in  particular,  its  compliance  with  the 
Financial Regulation and its Rules of Application. 

This  audit  tested  the  key  controls  related  to  the  procurement  process,  including  the 
management and monitoring controls. Testing covered the identification and planning 
of  the  needs  until  the  signature  of  the  contract,  including  amendments  and  price 
revisions, if applicable. It also included payments relating to procurements processed 
in 2015. 

The responsibilities of DG HR laid down in the Service Level Agreement concluded 
between  EPSO/EUSA  and  DG  HR  for  the  provision  of  financial  management  and 
procurement services were also included in the audit scope. 

There are no reservations in DG COMM, SCIC or EPSO/EUSA 2015 Annual Activity 
Reports that relate to the area/process audited. 

The fieldwork was finalised between 25 May and 13 June 2016. All observations and 
recommendations relate to the situation as of that date. 

Major audit findings 

DG COMM and EPSO/EUSA 

The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

SCIC 

The IAS identified one very important issue: 

•  Procurement process: weaknesses in tender documents, compliance issues 

33 

 
 
 
The  audit  revealed  non-compliance  issues  with  the  Financial  Regulation  (FR),  its 
Rules of Applications (RAP) or the case law of the European Court of Justice. More 
specifically,  the  IAS  found  weaknesses  in  the  tender  documents,  such  as  one 
procedure in which an award criterion which referred to the experience of the tenderer 
and  therefore  overlapped  with  a  similar  selection  criterion  and  two  procedures  in 
which the tender specifications were either not entirely clear for the tenderers or not 
fully defined. With regard to the evaluation of tenders, the IAS identified two cases in 
which  the  tender  specifications  were  not  strictly  followed.  Furthermore,  in  one 
procedure,  some  tenderers  were  contacted  without  ensuring  that  the  other  tenderers 
received the same level of information. Finally, the IAS noted one missing declaration 
of honour by a tenderer and four missing award decisions for low value procedures. 

DG  SCIC's  manual  on  public  procurement  provides for a formal visa by the finance 
unit  at  the  stage  of  the  draft  tender  documents.  However,  in  practice  this  control  is 
exercised only in an informal manner. 

The audit showed that, while the relevant ex-ante controls were carried out at the stage 
of the budgetary and legal commitment, i.e. after the evaluation and before awarding 
and  signing  of  the  contract,  they  neither  prevented  nor  detected  and  corrected  the 
weaknesses observed by the IAS. 

Recommendation 

To address this issue, the IAS formulated the following recommendation: 

•  Procurement Process: weaknesses in tender documents, compliance issues  

DG SCIC should: 

•  Remind  all  services  and  potential  members  of  evaluation  committees  to  ensure 
compliance with the FR, its RAP and the case law of the European Court of Justice 
for the cases identified through this audit; 

•  Update  its  procedures  on  ex-ante  controls  on  public  procurement  procedures  and 
formalise  the  control  ensuring  that  the  tender  specifications  meet  the  main 
requirements  as  set  out  by  the  FR  and  the  RAP  before  the  tender  documents  are 
published.  This  should  be  done  for  all  high  value  procedures,  together  with  a  risk-
based selection of low-value procedures; 

•  Revise  the  internal  checklists  used  for  commitments  and  payments  by  specifically 

including the main elements to be checked. 

2.  AGRICULTURE, NATURAL RESOURCES AND HEALTH 

2.1.   Audit on the design of DG AGRI's performance measurement system for the 
CAP 2014-2020 

Audit objectives and scope 

The  overall  objective  of  the  audit  was  to  assess  whether  DG  AGRI  has  adequately 
designed the Common Monitoring and Evaluation Framework (CMEF), including the 

34 

 
 
 
Common  Monitoring  and  Evaluation  System  (CMES),  in  order  to  monitor,  evaluate 
and report on the performance of the CAP 2014-2020.  

As  the  CMEF  is  still  in  an  early  stage  of  implementation,  the  audit  focused  on  its 
design and covered the following main steps:  

• A review of the design of the Common Agricultural Policy (CAP) intervention logic, 

including the CAP objectives and their related indicators; 

• A review of the design and preparedness of the processes put in place by DG AGRI 
for  ensuring  that  reliable  data  will  be  available  on  time  for  calculating  the  CMEF 
indicators values and reporting on them; 

• A  review  of  the  processes  put  in  place  by  DG  AGRI  for  providing  support  to  the 
Member States in the implementation of the CMES, including through the European 
Evaluation Helpdesk for Rural Development; 

• A review of the processes put in place by DG AGRI for planning evaluations of the 

CAP 2014-2020. 

The audit scope did not include the following: 

• Processes  related  to  the  detailed  monitoring  activities  performed  by  the  different 

units of DG AGRI for the policies they implement; 

• Processes  for  conducting  evaluations,  as  well  as  processes  for  managing  the 
contractual relationship with the contractors implementing the European Evaluation 
Helpdesk; 

• Processes  for  the  performance  review  linked  to  the  performance  reserve  that 
Regulation  (EU)  No  1303/2013  introduced  for  the  European  Structural  and 
Investment  Funds  (ESIF),  including  the  EAFRD.  This  is  a  specific  process,  which 
may be the subject of a separate audit at a later stage. 

DG  AGRI's  2014  Annual  Activity  Report  (AAR)  does  not  contain  any  reservation 
related to the performance measurement framework. 

fieldwork  was 

The 
recommendations relate to the situation as of that date. 

finalised  on  4  February  2016.  All  observations  and 

Major audit findings 

The IAS identified three very important issues: 

•  Quality of objectives, indicators and intervention logic  

The  specific  objectives  set  for  Pillar  I/Horizontal  Provisions  (and  related  indicators) 
do not always clearly define what the related policy is expected to achieve and, in a 
few cases, appear not to cover some essential aspects of the CAP's general objectives. 
In  addition,  the  intervention  logic  does  not  always  allow  to  identify  which  CAP 
instruments contribute to which objectives and how. 

•  Consistency and completeness of the CMEF 

The  CMEF  does  not  cover  all  the  various  CAP  instruments,  although  it  should  be 
noted  that  those  which  are  not  included  are  subject  to  performance  measurement 
provisions  laid  down  in  their  individual  legal  bases.  Furthermore,  while  the  CMEF 

35 

 
 
 
 
integrates Pillar I/Horizontal provisions and Pillar II at the level of impact indicators, 
this  is  not  the  case  for  result  indicators.  This  complicates  the  work  of  DG  AGRI  in 
demonstrating  the  combined  direct  effect  of  different  CAP  instruments  pursuing  the 
same specific objectives (for example for the payment for young farmers under Pillar I 
and the measures for young farmers under Pillar II).  

•  Reliability and availability of data  

Despite the fact that DG AGRI cooperates effectively with Eurostat, it faces continued 
problems  in  obtaining  reliable  data  for  calculating  the  values  of  certain  CMEF 
indicators, in particular for environmental indicators. This mainly concerns those for 
which  there  is  no  explicit  legal  basis  for  requiring  the  data  from  Member  States. 
According to DG AGRI, there is a strong resistance  from  Member  States  to  provide 
additional data due to the costs involved. 

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  Quality of objectives, indicators and intervention logic 

DG  AGRI  should  ensure  that  the  CAP  specific  objectives  for  Pillar  I/Horizontal 
Provisions  are  more  compliant  than  at  present  with  the  SMART  criteria  and  assess 
whether there is a need to include related additional indicators, based where possible 
on available data and taking into account cost effectiveness considerations in order to 
better demonstrate the achievement of policy objectives. The CAP intervention logic 
needs to be explained much more clearly.  

•  Consistency and completeness of the CMEF 

DG AGRI should develop additional indicators to cover the CAP instruments which 
are  not  adequately  addressed  through  the  current  set  of  CMEF  indicators  and  use  if 
possible  existing  data  to  integrate  Pillar  I  and  Pillar  II  aspects.  This  does  not 
necessarily  mean  including  additional  rural  development  indicators,  which  have 
already  been  established.  In  the  longer  term  and  for  the  next  multi-annual  financial 
framework programming period, DG AGRI should consider developing a set of result 
indicators aimed at showing the combined effects of both rural development and Pillar 
I/Horizontal Provisions.  

•  Reliability and availability of data  

For  the  data  which  is  currently  missing,  including  that  relating  to  environmental 
indicators,  it  should  follow  this  up  with  Eurostat  and  seek  to  obtain  additional  data 
through the mapping and cross-linking of available data, as well as through research 
projects. For the data needed to support the CMEF indicators, but for which there is no 
specific legal obligation on Member States to provide, the DG should assess whether 
this needs to be addressed through an implementing regulation. 

2.2.   Audit on DG AGRI's management and control system for Voluntary Coupled 
Support (VCS) 

Audit objectives and scope 

36 

 
 
The  overall  objective  of  the  audit  was  to  assess  the  design  and  as  far  as  possible, 
depending on their stage of implementation, the processes put in place by DG AGRI 
for  managing  and  controlling  VCS.  The  audit  assessed  in  particular  whether  these 
processes effectively contribute to the DG's assurance building process and ensure an 
effective monitoring of the VCS scheme.  

The audit assessed the management and control system put in place by DG AGRI for 
VCS,  including  performance  aspects.  It  covered  the  processes  put  in  place  by  DG 
AGRI  for  the  review  of  Member  State  notifications  on  their  VCS  decisions,  the 
guidance  provided  to  Member  States,  the  general  design  of  the  internal  monitoring 
process for the implementation of the scheme and the preparedness of the conformity 
clearance of accounts process regarding the VCS.  

The  audit  scope  did  not  include  certain  provisions  permitted  under  the  amended 
delegated act (i.e. modulated per unit amounts and transfers between measures), as the 
related processes were either only in progress during the fieldwork of the audit or only 
applicable to claim year 2016 to be paid under financial year 2017. 

The  first  claim  year  for  the  VCS  was  2015.  The  Commission  reimburses  only  since 
the beginning of 2016, the expenditure made by a Member State for 2015. Thus, the 
2015  Annual  Activity  Report  (AAR)  includes  no  reservations  relating  to  the  VCS. 
However,  it  includes  a  reservation  on  Direct  Payments  with  regard  to  10  Paying 
Agencies involving 6 Member States. Moreover, in annex 10 of the AAR concerning 
direct  payments  (including  VCS),  DG  AGRI  identifies  risks  linked  to  the  past 
implementation  of  art.  68  of  Regulation  (EC)  No  73/2009,  which  might  affect  the 
implementation  of  VCS.  It  also  identifies  risks  linked  to  the  implementation  of  the 
reformed system of Direct Payments, having as root cause the greater complexity of 
the  support  schemes,  the  flexibility  given  to  Member  States  and  their  diverging 
interpretations. In the same annex, DG AGRI defines actions to mitigate these risks. 

The  fieldwork  was  finalised  on  7  June  2016.  All  observations  and  recommendations 
relate to the situation as of that date. 

Major audit findings 

The IAS identified three very important issues: 

•  Follow-up of VCS notification assessments 

Through its assessment of VCS notifications, DG AGRI identified a number of issues, 
which  indicated  non-compliance  and/or  the  risk  of  non-compliance  and  in  certain 
cases  had  started  planning  EU  pilot  procedures.  However,  there  is  currently  no 
formalised  typology  used  to  categorise  the  detected  issues  according  to  their  nature, 
scope,  frequency  and  seriousness.  In  addition,  there  is  no  clear  approach  on  how  to 
follow up these issues through appropriate available tools (e.g. EU PILOT procedures, 
reduction or suspension as well as conformity clearance procedures). Finally, it is not 
clear which service in DG AGRI should primarily be responsible for following up the 
specific  cases  where  DG  AGRI  detected  a  risk  of  potential  cumulative/overlapping 
support with rural development.  

•  Monitoring and control of the 2015 financial ceilings 

37 

 
 
DG AGRI is required under the legislation to monitor and control that the amounts of 
support  per  measure  do  not  breach  the  measure-specific  ceilings.  However,  these 
ceilings  are  not  always  clearly  specified.  Furthermore,  for  the  claim  year  2015, 
although the general VCS ceiling per Member State is systematically controlled on a 
monthly basis by the EAGF financial unit, this is not the case for the measure-specific 
ceilings. The IAS analysed the draft working arrangements that were being developed 
in this regard at the time of the audit fieldwork and concluded that they needed to be 
further improved from an effectiveness and efficiency viewpoint. 

•  Monitoring of VCS performance 

The  objective  of  the  VCS  scheme  is  to  create  an  incentive  to  maintain  agricultural 
production  in  vulnerable  sectors  and  /or  regions.  In  monitoring  the  performance  of 
VCS,  DG  AGRI  compares  the  total  area  and  the  total  number  of  animals  for  which 
VCS has been paid with the area or number of animals notified by the Member State 
in  2014.  However,  the  data  on  which  this  analysis  is  based  is  not  always  correct  or 
clear  (cases  were  identified  where  data  were  missing  and/or  calculations  were 
erroneous). In addition, the IAS found only very limited evidence that DG AGRI had 
assessed  whether  the  amount  of  support  given  is  proportionate  to  the  difficulties 
described in relation to those sectors or regions concerned. Also, there is no structured 
monitoring of the impact of VCS in areas of high overall aggregated EU support or in 
those  areas  which  are  most  vulnerable  to  agricultural  market  crises.  Currently  the 
monitoring arrangements focus more on whether quantitative limits, as notified by the 
Member  States,  are  respected,  rather  than  at  assessing  the  effect  of  VCS  on  the 
corresponding sector and/or regions. 

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  Follow-up of VCS notification assessments  

DG  AGRI  should  send  letters  of  findings  in  the  context  of  the  conformity  clearance 
procedure  as  soon  as  possible  so  that  Member  States  can  resolve  the  deficiencies 
identified in time for the claim year 2017. It should also clarify the main conditions on 
how to follow up issues identified both in the 2014 notifications and for future years, 
for  example  through  triggering  EU  PILOT  procedures,  reduction  or  suspension 
provisions as well as conformity clearance procedures. DG AGRI should also develop 
internally  a  typology  to  support  the  comments  made  when  assessing  the  VCS 
notifications.  Finally,  where  DG  AGRI  detects 
risk  of  potential 
cumulative/overlapping support with rural development, it should ensure that they are 
properly  followed  up  on  a  timely  basis.  The  unit  primarily  responsible  should  be 
clearly designated. 

a 

•  Monitoring and control of the 2015 financial ceilings  

DG AGRI should ensure that, for all VCS measures, a fixed measure-specific ceiling 
is defined. It should also specifically check for the claim year 2015 that the respective 
ceilings are met and ensure that the staff responsible for this task are properly trained. 
As from claim year 2016, the checks should be automated. 

•  Monitoring of VCS performance  

38 

 
 
DG  AGRI  should  identify  those  VCS  measures  where  the  risks  of  not  meeting  the 
scheme's  objectives  are  highest  and  where  there  is  the  greatest  likelihood  of  market 
distortion.  For  these  measures,  DG  AGRI  should  strengthen  the  current  monitoring 
arrangements, for example by making more use of available complementary data and 
analysis. 

2.3.   Audit on public procurement in DG CLIMA 

Audit objectives and scope 

The  overall  objective  of  the  audit  was  to  assess  the  adequacy  of  the  design  and  the 
effective  implementation  of  the  service's  internal  control  system  with  regard  to  the 
management  of  the  procurement  process  and,  in  particular,  its  compliance  with  the 
Financial Regulation (FR) and its Rules of Application (RAP). 

This  audit  tested  the  key  controls  throughout  the  procurement  process,  from  the 
identification  and  planning  of  the  needs  through  to  the  signature  of  the  contract, 
including amendments and price revisions and related payments. 

The audit covered the procurement procedures awarded in 2015 and in 2016 as well as 
procurement  related  payments  processed  in  2015.  The  activities  of  the  Advisory 
Committee were also covered.  

Public procurement procedures relating to budget appropriations that CLIMA has sub-
delegated to other DGs or that DG CLIMA has received as sub-delegation from other 
DGs were excluded from the scope of the audit.  

There are no reservations in DG CLIMA's 2015 Annual Activity Report that relate to 
the area/process audited. 

The fieldwork was finalised on 24 June 2016. All observations and recommendations 
relate to the situation at that date.  

Major audit findings 

The IAS identified one very important issue: 

•  Justification of public procurement needs  

The  audit  revealed  weaknesses  in  relation  to  the  justification  of  public  procurement 
needs.  More  specifically,  it  was  not  always  possible  to  clearly  demonstrate  that  a 
needs  analysis  had  been  systematically  and  consistently  made.  Furthermore,  the 
documentary evidence to support the justification, verification and approval of certain 
initially  decided  (for  example 
significant  changes 
procurement procedure or budget line) was very limited in practice. 

the  procurement  plan 

to 

The  IAS  also  identified  additional  weaknesses  concerning  the  definition  and  the 
assessment of award criteria and the way in which studies are identified.  

Recommendation 

To address this issue, the IAS formulated the following recommendation: 

DG CLIMA should: 

39 

 
 
• Ensure  that  a  systematic  and  consistent  analysis  of  the  needs  for  procurement  is 
performed and documented and that any major modifications in planned procurement 
procedures  during  the  year  are  properly  justified  in  writing,  approved  and 
documented; 

• Intensify  awareness-raising  activities  (guidance,  training)  and  strengthen  ex-ante 
controls  (internal  supervision)  as  appropriate  to  ensure  full  compliance  with  the 
applicable rules and guidance (notably as regards award criteria and identification of 
studies). 

2.4.   Audit on staff allocation and process management in response to staff reduction 
in DG ENV  

Audit objectives and scope 

The overall objective of the audit was to assess whether, in the light of the challenges 
it  faces,  DG  ENV  has  adequate  systems  in  place  for  allocating  staff  and  is  ensuring 
that its processes are managed as efficiently as possible. 

The  audit  covered  DG  ENV's  HR  management  processes  and  in  particular,  the 
procedures,  systems,  methods  and  tools  used  to  allocate  staff  aligned  with  the  DG's 
key  priorities  and  objectives.  It  also  covered  the  DG's  overall  approach  (including 
methodologies  and  practices  in  place)  to  identify,  propose  and  implement  efficiency 
gains in its processes. However, the audit did not address the issue as to whether or not 
the DG has the right organisational structure in place. 

The 2015 Annual Activity Report (AAR) of DG ENV do not include any reservations 
related to the process audited. 

The fieldwork was finalised on 10 June 2016. All observations and recommendations 
relate to the situation as of that date. 

Major audit findings 

The IAS identified one very important issue: 

•  Workload assessment  

Currently, DG ENV does not have a structured monitoring framework, together with 
indicators),  providing  regular  and 
key  workload 
quantitative information on workload in the DG.  

indicators  (including  proxy 

Recommendation 

To address this issue, the IAS formulated the following recommendation: 

DG  ENV  should  develop  key  workload 
indicators,  supported  by  a  clear 
methodological base and ensure that these are periodically monitored and reported in 
order to optimise the efficient and effective allocation of its resources. 

2.5.   Audit on pilot projects and preparatory actions in DG SANTE 

Audit objectives and scope 

40 

 
 
The  overall  objective  of  the  audit  was  to  assess  the  effectiveness  of  the  controls 
covering  the  financial  management  of  pilot  projects  and  preparatory  actions  in  DG 
SANTE. 

In particular, the design and the implementation of the controls in place were reviewed 
to  assess  whether  they  ensure  the  legality  and  regularity  of  the  financial  procedures 
and the financial transactions and whether they are effective. 

The audit also assessed DG SANTE's internal organisation for the prior assessment of 
the  proposed  pilot  projects  and  preparatory  actions  as  well  the  design  and 
implementation of the financial circuits. 

This  audit  covered  the  key  controls  designed  and  implemented  in  the  following 
processes  of  the  financial  management  of  pilot  projects  and  preparatory  actions  for 
2014-2015: 

• The processes for ex-ante assessment of the proposed pilot projects and preparatory 
actions,  allocating  them  within  DG  SANTE  and  monitoring  and  reporting  on  their 
implementation; 

• The  procurement  process,  from  the  determination  of  the  needs  and  planning  to  the 

effective implementation of the contract;  

• The grants process, from the planning and preparation of the call for proposal to the 

closure of the grant; and 

• The financial circuits of the related grants and procurement, including commitments, 

payments, de-commitments and recovery orders. 

Major audit findings 

The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

3.  RESEARCH, ENERGY AND TRANSPORT 

3.1.   Audit on Human Resources management in DG CONNECT 

Audit objectives and scope 

The overall objective of the audit was to assess the effectiveness of DG CONNECT's 
Human  Resource  management  system  to  support  the  achievement  of  the  DG's 
priorities and core business. 

The  audit  aimed  to  answer  the  following  main  question:  "Has  DG  CONNECT 
designed  and  implemented  an  adequate  HR  management  process  to  deploy  a 
competent  and  engaged  workforce,  in  order  to  deliver  the  DG's  priorities  and  core 
business?" 

The  audit  covered  the  design  and  implementation  of  the  HR  strategy  and  the  HR 
planning  process,  including  workforce  planning  (in  the  light  of  potential  efficiency 
gains), staff allocation and change management. The audit also covered the activities 
performed in terms of learning & development, redeployment and career management. 

Major audit findings 

41 

 
 
 
The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

3.2.   Audit on closure of projects of legacy programmes in DG CONNECT 

Audit objective and scope 

The objective of the audit was to assess the effectiveness of the process for the closure 
of DG CONNECT’s projects belonging to the following legacy programmes:  

• The Sixth Research Framework Programme (FP6 2003-2006); 
• The Seventh Research Framework Programme (FP7 2007-2013); 
• The Competitiveness and Innovation Framework Programme (CIP 2007-2013); 
• Safer Internet (2009-2013). 

The  audit  covered  the  monitoring  and  reporting  on  the  closure  of  projects  and  the 
management  of 
the  financial  distribution  report,  decommitments,  archiving, 
amendments, complaints and of the implementation of ex-post audit results. 

There  are  no  observations/reservations  in  DG  CONNECT's  2015  Annual  Activity 
Report that relate to the area/process audited. 

The  following  reservations  were  however  made  in  the  2015  Annual  Activity  Report 
concerning legacy programmes: 

• For FP7, DG CONNECT estimated a residual error rate of 2.58%, which is above the 
2%  materiality  threshold,  and  therefore  issued  a  reservation  in  line  with  similar 
reservations expressed by the other DGs of the research family. 

• The residual error rate for CIP amounts to 4.42%. DG CONNECT estimated that the 
residual error rate will not decrease under the materiality threshold at the end of the 
programme  and  therefore  expressed  a  reservation  on  the  legality  and  regularity  of 
these payments.  

The  fieldwork  was  finalised  on  28  November  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

3.3.   Audit on the management and functioning of Euratom Safeguards in DG ENER 

Audit objective and scope 

The  overall  objective  of  the  audit  engagement  was  to  assess  the  efficiency  and 
effectiveness of the systems and procedures in place in DG ENER in ensuring that the 
EC  fulfils  its  obligations  stemming  from  the  Euratom  Treaty  and  international 
agreements. 

42 

 
 
 
 
The audit focused on (1) the EC governance framework associated with the Euratom 
Safeguards;  (2)  the  design  and  methodologies  of  the  safeguards  system;  (3)  the 
procurement of services and equipment supporting inspection activities and (4) human 
resources management. 

The  audit  did  not  cover  IT  systems  and  related  operations,  the  cooperation  and 
coordination  with  the  Euratom  Supply  Agency  and  with  ENER.D  -  Nuclear  energy, 
safety and ITER, as well as accompanying inspectors to on-site missions. 

There are no observations/reservations in the DG's 2014 Annual Activity Report that 
relate to the area/process audited. 

fieldwork  was 

The 
recommendations relate to the situation as of that date. 

finalised  on  16  March  2016.  All  observations  and 

Major audit findings 

The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

3.4.   Audit on the supervision of ITER in DG ENER 

Audit objective and scope 

The  overall  objective  of  the  audit  was  to  assess  whether  the  strategy  for  the 
supervision  of  the  ITER  project  has  been  adequately  designed  and  effectively 
implemented. 

The audit focused in particular on: 

• The legal/administrative arrangements of the supervision framework; 
• The Commission's supervision strategy on the ITER project management; 
• Participation in the work of the ITER IO/F4E governance bodies. 

There  have  been  no  observations/reservations  in  the  Annual  Activity  Reports  of  the 
respective  DGs  (RTD  until  2014,  ENER  in  2015)  that  relate  to  the  area/process 
audited.  

fieldwork  was 

The 
finalised  on  31  March  2016.  All  observations  and 
recommendations  relate  to  the  situation  on  that  date.  However,  during  the 
validation/reporting stage, the IAS also took note of the discussion of the evolution of 
the ITER project, which led to the agreement ad referendum of the long-term schedule 
by the ITER Council (IC) in June 2016 for the period until 2025. 

Major audit findings 

This IAS identified two very important issues: 

•  DG ENER's supervision strategy for the ITER organisation and project 

A  number  of  internal  notes  on  ITER  supervision  highlight  the  different  aspects  and 
weaknesses  of  the  ITER  project  set-up  (including  action  plans)  but  the  Commission 

43 

 
 
 
has  not  yet  defined  and  implemented  a  comprehensive  supervision  strategy  for  the 
ITER project. It is not yet defined what the DG and Euratom aim to achieve with their 
supervision activities (objectives), taking into account the available supervision tools 
and their effectiveness and how the effectiveness of the supervision activities will be 
assessed. 

In addition, the rules to provide relevant documents for the preparation of the line-to-
take at the ITER governing bodies (ITER Council, Management Advisory Committee 
and  Science  and  Technology  Advisory  Committee)  were  not  respected  (ITER 
Council)  or  were  non-existent  (Management  Advisory  Committee,  Science  and 
Technology Advisory Committee).  

•  Supervision and monitoring of F4E activities  

The  IAS  observed  that  the  Commission  is  currently  not  in  a  position  to  effectively 
monitor  F4E  and  use  this  knowledge  in  the  discussions  in  the  relevant  governance 
bodies. This is because it does not receive all the information that is essential to find 
the  best  way  to  address  the  delays  in  the  critical  and  highly  critical  components 
managed  by  F4E.  Furthermore,  the  latest  F4E  annual  report  and  quarterly  report  are 
not aligned with the structure of the work programme and  neither is the structure of 
the  work  programme  aligned  with  the  project  plan,  which  makes  it  very  difficult  to 
monitor the proposed activities and their level of achievement. 

In  addition,  "The  F4E  Administrative  Arrangement",  signed  in  2008,  has  not  been 
updated to take into account several legislative changes while "The working relations" 
with F4E have not yet been fully implemented. 

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  DG ENER's supervision strategy for the ITER organisation and project 

DG ENER should develop its ITER project supervision strategy, which should set out 
the supervision needs, the objectives for the supervision activities and the tools to be 
used.  DG  ENER  should  also  define  working  methods  and  procedures  needed  to 
achieve the supervision objectives. This supervision strategy should then be translated 
into short-term operational activities to mitigate the risks and should be accompanied 
by indicators to allow the monitoring of the performance of the strategy. 

DG  ENER  should  have  all  the  necessary  information  for  the  subsequent  decision-
making.  To  this  end,  it  has  to  ensure  that  ITER  submits  the  documents  for  the 
meetings  on  time  and  to  agree  formally  a  submission  deadline  for  the  Management 
Advisory Committee and Science and Technology Advisory Committee documents. It 
the  pieces  of 
the  availability  of/accessibility 
has  also 
information/results  of  analysis  necessary  to  take  a  considered  position  in  the  ITER 
Council. 

to  ensure 

to  all 

•  Supervision and monitoring of F4E activities 

DG ENER should reach an agreement with F4E on the type of information it needs on 
procurement/contract/technical  aspects  of  F4E  operations  and  the  results  of  related 
risk assessments and in which format this information should be shared to allow DG 
ENER  to  effectively  address  performance  issues.  Furthermore,  DG  ENER  should 

44 

 
 
update "the F4E administrative arrangement" to take into account the newly adopted 
legislation and assess the effectiveness of the existing "working relations" established 
between DG RTD and F4E. 

3.5.   Limited review of the calculation and the underlying methodology of the residual 
error rate for the 2015 reporting year in DG ENER 

Audit objective and scope 

The  overall  objective  of  this  limited  review  was  to  examine  the  calculation  and 
underlying  methodology  of  the  multi-annual  Residual  Error  Rate  (RER)  reported  by 
DG ENER in its (draft) 2015 Annual Activity Report (AAR), and in doing so, help the 
DG mitigate the discharge risk by enabling it to take appropriate actions, if any, before 
their disclosure in the final AAR and in the Synthesis Report. 

The review covered the following aspects: 

• The process and methodology for the calculation of the RER; 
• The calculated RER; 
• The presentation of the RERs in the draft AAR; 
• Compliance with the Standing Instructions for the 2015 AAR. 

The IAS reviewed the draft 2015 AAR and the preliminary RER calculations available 
on 29 February 2016. It also reviewed the final 2015 AAR to check whether the issues 
detected during the fieldwork were correctly addressed. 

The limited  review fieldwork was finalised on  15 March 2016. All observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

3.6.   Audit on Human Resources management in ERCEA 

Audit objectives and scope 

The overall objective of the audit was to answer the following question: 

Has  ERCEA  designed  and  implemented  an  adequate  HR  management  process  to 
deploy  a  competent  (knowledgeable)  and  engaged  workforce  in  order  to  deliver  its 
priorities and core business? 

The audit covered in particular: 

• The design and implementation of the HR strategy; 
• The HR planning process, including workload assessment and staff allocation; 
• Selection, recruitment and retention of staff; 
• Knowledge management (training, coaching, competence management); 
• Monitoring and reporting on HR. 

45 

 
 
 
There is no reservation in ERCEA's 2015 Annual Activity Report regarding the scope 
of this audit. 

The  fieldwork  was  finalised  on  2  June  2016.  All  observations  and  recommendations 
relate to the situation as of that date. 

Major audit findings 

The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

3.7.   Audit on the coordination by INEA with its parent DGs during the key stages of 
the Strategic Planning and Programming cycle 

Audit objective and scope 

The  overall  objective  of  the  audit  was  to  assess  whether  INEA  has  put  in  place 
appropriate coordination and working arrangements with its parent DGs to ensure the 
effective implementation of the key stages of the Strategic Planning and Programming 
(SPP) cycle. 

The  audit  focused  on  INEA's  coordination  with  its  parent  DGs  during  the  three  key 
stages of the SPP cycle: a) the planning phase (including the preparation of the CEF 
and H2020 work programmes and the Agency's Annual Work Programme (AWP)), b) 
the  implementation  phase  (implementation  of  the  AWP)  and  c)  the  reporting  phase. 
The parent DGs were not audited. 

There are no observations/reservations in the 2015 Annual Activity Report (AAR) of 
INEA that relate to the area/process audited. 

The  fieldwork  was  finalised  on  27  September  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

3.8.   Audit on competitive activities in DG JRC 

Audit objective and scope 

The  overall  objective  of  this  audit  was  to  assess  whether  the  Competitive  Activities 
(CA)  are:  (i)  effectively  planned,  monitored  and  reported  on,  (ii)  effectively  and 
efficiently implemented, and (iii) compliant with the applicable rules and guidance. 

The audit scope covered: 

• At DG level, the CA governance arrangements and administrative set-up as well as 

their strategic planning, monitoring and reporting; 

46 

 
 
 
• At  operational  level,  the  CA  contracts'  life  cycle,  namely  the  contracts'  proposals, 
preparation  and  implementation,  the  clients'  payments,  and  the  closure  of  the  CA 
contracts. 

The financial management of CA was not included in the scope of the present audit. 

There are no observations/reservations in the JRC's 2015 Annual Activity Report that 
relate to the area/process audited. 

The  fieldwork  was  finalised  on  3  June  2016.  All  observations  and  recommendations 
relate to the situation as of that date. 

Major audit findings 

The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

3.9.   Audit on setting of objectives and measurement of performance in DG MOVE 

Audit objective and scope 

The overall objective of this audit was to assess whether DG MOVE has an adequate 
performance  management  framework  in  place  for  its  day-to-day  operational  and 
administrative  activities  (internal)  and  for  the  delivery  of  its  policy  objectives 
(external).  The  audit  assessed  the  internal  processes  for  setting  objectives  and  key 
performance indicators as well as the related reporting and monitoring. 

The audit focused in particular on the following areas: 

• The process of setting high quality objectives and performance indicators (design and 

implementation of the process) in line with the policy; 

• The  performance  measurement  framework  for  monitoring,  evaluating and reporting 

the (internal and external) performance of activities. 

The  audit  covered  the  processes  related  to  the  preparation  of  the  Strategic  Plan  (SP) 
(2016-2020), the Management Plans (MP) (2014, 2015 and 2016), the Annual Activity 
Reports (AAR) (2014, 2015) and he Programme Statements (PS) for the Draft Budgets 
2016 and 2017. 

In  the  context  of  this  engagement,  the  IAS  also  performed  a  follow  up  audit  of  the 
2014  SIAC  Audit  of  Internal  Control  Standard  5  "objectives  and  indicators"  in  DG 
MOVE.  

There are no observations/reservations in the 2015 AAR of DG MOVE that relate to 
the area/process audited. 

fieldwork  was 

The 
recommendations relate to the situation as of that date. 

finalised  on  18  March  2016.  All  observations  and 

Major audit findings 

The IAS identified three very important issues: 

47 

 
 
 
 
•  DG MOVE performance framework  

The different tools DG MOVE currently uses to plan and monitor its activities, actions 
and  initiatives  are  not  complemented  by  an  overarching  strategic  vision  describing 
how  the  DG  organises  its  interventions  and  how  short-term  outputs  will  lead  to 
medium  and  long-term  results  and  impacts  and  contribute  to  the  achievement  of  its 
strategic objectives. Consequently, there is no overview, which demonstrates how the 
different DG's activities contribute to the achievement of its strategic and operational 
objectives without gaps or overlaps. Furthermore, there is no centralised approach to 
monitoring  and  reporting  on  longer-term  policy  achievements  (i.e.  results  and 
outcomes/impacts  of  transport  legislation  and  programmes).  Due  to  weaknesses 
identified in the SP and PS (in particular the quality of objectives and indicators), the 
DG does not have a complete picture of the progress made towards the achievement of 
its objectives. 

•  Quality of objectives and indicators in the 2016 SP/MP  

DG  MOVE’s  Specific  Objectives  (SOs)  are  not  sufficiently  specific  and  relevant.  In 
particular, they do not clearly specify the situation which needs to be changed and (if 
relevant)  the  target  group  concerned  and  do  not  address  the  needs  of  society 
/stakeholders and the wider political context. In addition, six result indicators are not 
relevant  since  they  measure  output  and  not  result.  Furthermore,  for  the  spending 
programmes  CEF  and  H2020,  objectives  and  indicators  in  the  SP  are  different  from 
the objectives and indicators in the PS and DG MOVE does not ensure the coherence 
between the two performance management tools. 

•  CEF PS  

There is no formal process to prepare the CEF PS and DG MOVE did not complement 
DG  BUDG's  Instructions  with  internal  guidance  defining  the  tasks  to  be  performed, 
the responsibilities and roles of each unit, the timing and workflow, the definition of 
the  indicators  with  the  source  of  information,  the  methodology  to  calculate  the 
indicators  and  the  unit  in  charge.  Due  to  the  lack  of  clear  responsibilities  and 
ownership  for  the  preparation  of  the  CEF  PS,  the  process  to  collect  relevant 
information was not launched on time, resulting in shortened deadlines and finally in 
reduced quality of the submitted document. 

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  DG MOVE performance framework 

level  priorities,  objectives  and  short 

DG MOVE should complete its performance framework by preparing a strategic view 
of the DG's activities that establishes a clear logical link (intervention logic) between 
its  high 
term  actions.  The  strategic 
view/intervention logic should show how the DG intends to prioritise and organise its 
actions in order to contribute to the SOs, assess whether or not the actions planned for 
a given year will contribute to achievement of its SOs and assess the overall progress 
made  towards  this  achievement.  DG  MOVE  should  also  develop  an  integrated 
approach to performance monitoring and reporting on policy achievements. 

•  Quality of objectives and indicators in the 2016 SP/MP 

48 

 
 
DG MOVE should ensure that its specific objectives meet the SMART criteria and are 
in  line  with  the  DG's  responsibilities  by  either  reformulating  them  or  by 
complementing them with a set of RACER result indicators. These latter should cover 
the most essential aspects of the DG's activities and focus on results in terms of added 
value to the EU stakeholders. Furthermore, DG MOVE should streamline the process 
to  set  objectives  and  indicators  (and  to  monitor  them)  by  re-using,  to  the  extent 
possible,  elements  included  in  different  performance  management  tools  (SP/MP, 
Programme Statement) or in the legal basis. 

•  CEF PS 

DG  MOVE  should  formally  attribute  the  responsibilities  for  the  preparation  of  CEF 
PS, and develop and document a procedure for its preparation and coordination. 

3.10.  Audit on DG MOVE's monitoring of the aviation and maritime security policies, 
including related working arrangements with the EMSA Regulatory Agency 

Audit objective and scope 

The overall objective of the audit was to assess the effectiveness and efficiency of DG 
MOVE's monitoring of aviation and maritime security policies. The audit reviewed the 
planning and execution of DG MOVE's inspection activities, the use of the  Member 
States annual reporting as well as the management of the necessary human resources 
for fulfilling the Commission’s obligations. The scope also included: i) DG MOVE’s 
reporting  to  the  main  stakeholders  on  the  assurance  obtained  from  its  monitoring 
activities; ii) DG MOVE's preventive and reactive measures in case of serious aviation 
and maritime incidents; and iii) cooperation with EMSA. 

The audit covered the activities conducted in the period 2011-2016. 

There are no observations/reservations in DG MOVE's 2015 Annual Activity Report 
that relate to the area/process audited. 

The  fieldwork  was  finalised  on  the  8  December  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The IAS identified three very important issues: 

Aviation security field 

•  Monitoring strategy for aviation security policy  

The  EU  legislation  on  aviation  security  does  not  provide  a  precise  indication  of  the 
level of assurance the Commission has to obtain with its monitoring activities, which, 
consequently, has to be defined by the Commission itself. DG MOVE's approach has 
not been formalised in a comprehensive strategy describing the level of assurance to 
be provided, the monitoring objectives, criteria and methodology, the timeframe, how 
to use the different monitoring tools and how many resources are needed for that. In 
addition,  DG  MOVE  has  not  defined  objectives  and  indicators  to  measure  the 
performance  of  the  monitoring  activity  and  to  evaluate  if  the  current  resources  are 
sufficient to achieve the required level of assurance. 

49 

 
 
•  Monitoring tools – aviation security policy  

The  information  provided  by  the  Member  States  in  the  annual  report  to  the 
Commission  on  the  measures  taken  to  fulfil  their  obligations  under  the  Regulation 
concerning their national quality control programmes is not always sufficient to allow 
DG MOVE to conclude on the effective implementation of these programmes. 

There is no documentation describing the exact use made by DG MOVE of the annual 
reports  when  monitoring  the  implementation  of  EU  rules  by  Member  States.  In  this 
respect,  DG  MOVE  does  not  send  a  formal  individual  comprehensive  evaluation  to 
the Member States emphasising points of reported non-compliance by the appropriate 
authority  (AA).  When  summarising  the  information  from  the  annual  Member  State 
reports and sharing it in the regulatory committee for civil aviation security (AVSEC), 
DG  MOVE  does  not  complement  it  with  relevant  conclusions  about  the  effective 
implementation of the national quality control programmes. 

Maritime security field 

•  Monitoring strategy for maritime security policy  

The EU legislation on maritime security does not provide a precise indication of the 
level  of  assurance  the  Commission  has  to  achieve  with  its  monitoring  activities  (i.e. 
the monitoring objective), which consequently has to be defined by the Commission 
itself. DG MOVE's monitoring approach has not been formalised in a comprehensive 
strategy  describing  the  level  of  assurance  to  be  provided,  the  monitoring  objectives, 
criteria and methodology, the timeframe, how to use the different monitoring tools and 
how  many  resources  are  needed  for  that.  In  addition,  DG  MOVE  has  not  defined 
objectives and indicators to measure the performance of the monitoring activity and it 
is not possible to evaluate if the current resources are sufficient to achieve the required 
level of assurance. 

Recommendations 

To address these issues the IAS formulated the following recommendations: 

•  Monitoring strategy for aviation security policy  

DG MOVE should formalise a comprehensive overall strategy for the EC monitoring 
of  the  implementation  of  the  EU  aviation  security  standards  by  the  Member  States. 
The  strategy  should  set  out  the  degree  of  assurance  to  be  obtained  through  the  EC 
monitoring  activities  and  from  the  different  monitoring  tools  (individually  and 
collectively), the indicators to be used to measure performance and progress towards 
the achievement of the monitoring objectives as well as the analysis of the resources 
needed to obtain the desired assurance.  

•  Monitoring tools – aviation security policy  

DG  MOVE  should  ensure  that  Member  States  provide  all  information  necessary  to 
conclude  on  the  effectiveness  of  the  implementation  of  the  national  quality  control 
programmes.  This  should  include,  among  others,  swiftly  following-up  with  the 
Member  States  cases  of  incomplete  reporting  and  revising  the  template,  if  structural 
weaknesses are noted. DG MOVE should document the methodology to be followed 
by its inspectors when analysing the annual reports in order to ensure that they draw 

50 

 
 
conclusions,  for  each  Member  State,  on  the  effective  implementation  of  national 
quality  control  programmes.  These  conclusions  should  be  shared  with  the  other 
Member States in the AVSEC committee meetings. 

•  Monitoring strategy for maritime security policy  

DG MOVE should formalise a comprehensive overall strategy for the EC monitoring 
of  the  implementation  of  the  EU  maritime  security  standards  by  the  Member  States. 
The  strategy  should  set  out  the  degree  of  assurance  to  be  obtained  through  the  EC 
monitoring  activities  and  from  the  different  monitoring  tools  (individually  and 
collectively),  the  indicators  needed  to  measure  performance  and  progress  towards 
achievement of the objective and the analysis of the resources needed to achieve DG 
MOVE's objective. 

3.11.  Limited review of the calculation and the underlying methodology of the residual 
error rate for the 2015 reporting year in DG MOVE 

Audit objective and scope 

The  overall  objective  of  this  limited  review  was  to  examine  the  calculation  and 
underlying  methodology  of  the  multi-annual  Residual  Error  Rate  (RER)  reported  by 
DG MOVE in its (draft) 2015 Annual Activity Report (AAR), and in doing so, help 
the DG mitigate the discharge risk  by  enabling it to take appropriate actions, if any, 
before their disclosure in the final AAR and in the Synthesis Report. 

The review covered the following aspects:  

• The process and methodology for the calculation of the RER; 
• The calculated RER; 
• The presentation of the RER in the draft AAR; 
• Compliance with the Standing Instructions for the 2015 AAR. 

The IAS reviewed the draft 2015 AAR and the preliminary RER calculations available 
on 01 March 2016. It also reviewed the final 2015 AAR to check whether the issues 
detected during the fieldwork were correctly addressed.  

The limited  review  fieldwork was  finalised on  15 March 2016. All observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

3.12.  Audit on H2020 grant management in the REA: part a) from the preparation of 
the calls for proposals to the signature of the grant agreements part b) administrative 
logistical services provided for H2020 

Audit objectives and scope 

51 

 
 
 
The overall objective of the audit was to assess the adequate design and effective and 
efficient implementation of REA's internal control system for:  

• The grant management process from the preparation of the calls for proposals phase 
to the signature of the grant agreements in order to ensure that the calls for proposals 
effectively support the achievement of the H2020 objectives, and that the processes 
in place ensure that the best research projects are selected and translated into grant 
agreements, in compliance with the applicable rules; 

• Administrative 

logistical  services  provided  for  H2020  programme  and 

its 

implementing entities. 

This audit follows the gap analysis review of the H2020 legislation performed by the 
IAS in 2015, which identified a number of risks faced by the Commission as a result 
of the co-legislative process. The current audit, as well as similar audits launched in 
other H2020 implementing bodies, also assessed whether the risks identified in the gap 
analysis audit are being addressed. 

The audit covered: 

• The  first  implementation  phases  of  H2020  from  the  planning  of  the  evaluation  of 

proposals to the signature of the grant agreements by REA in 2014 and in 2015; 

• Certain administrative and logistic support services provided for H2020 (planning for 
the calls for proposals and support for publication of calls, general logistical support 
for the evaluation of proposals including the management of the evaluation facility, 
and contracting of experts). 

The following areas were out of the scope of the audit: 

• The  services,  provided  for  the  other  EU  programmes,  including  the  validation 
process  (legal  validation  of  beneficiaries,  and  the  preparation  of  the  applicants' 
financial viability assessment); 

• The payment process for experts due to the changes and integration of the payment 

workflow in the COMPASS IT system as of 2016; 

• The  Research  Enquiry  Service  (RES)  through  which  REA  provides  replies  to  the 

broader public on EU research and innovation funding. 

There were no observations/reservations in REA’s 2015 Annual Activity Report that 
relate to the area/process audited. 

The  fieldwork  was  finalised  on  15  February  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The IAS identified one very important issue on the management of conflicts of interest 
(CoI). 

•  Managing conflicts of interest 

The  practices  applied  among  the  various  operational  units  regarding  the  extent  of 
checks  for  CoI  varies,  as  the  existing  corporate  guidance  does  not  describe  the 
minimum CoI-related checks to be performed and are not complemented by internal 
guidance. 

52 

 
 
 
 
 
Moreover,  in  some  cases  additional  keyword  checks  for  identifying  direct  CoI  of 
evaluation  experts  were  performed  after  the  signature  of  the  experts'  contracts,  and 
revealed some instances of CoI. However, as the experts were already carrying out the 
individual  evaluations,  their  work  needed  to  be  re-performed  and  travel  expenses 
reimbursed according to the contractual provisions. 

Furthermore, there are no clear procedures and guidance on the roles, responsibilities 
and  the  coordination  between  the  operational  and  the  contracting  units  regarding 
actions to be taken and procedure to be followed in case of breaches of confidentiality 
rules and unauthorised processing of personal data. 

Recommendation 

To address this issue, the IAS formulated the following recommendation: 

• The  Agency  should  ensure  that  sufficient  and  coherent  instructions  regarding  CoI 

checks are provided and their application is systematic and consistent; 

• The existing practices regarding implementation of the keywords matching controls 
should be harmonised and timely application of the controls – before experts being 
contracted – should be ensured by the Agency; 

The  Agency  should  issue  specific  guidance  for  staff  on  managing  CoI  discovered 
during the evaluation and establish the procedure to be followed in case of misuse of 
personal data in the context of the evaluation process where the role of the operational 
units  and  the  contracting  unit  should  be  described  with  the  timeline  and  steps  to  be 
followed. 

3.13.  Limited review of the calculation and the underlying methodology of the residual 
error rate for the 2015 reporting year in the REA 

Audit objective and scope 

The  overall  objective  of  this  limited  review  was  to  review  the  calculation  and 
underlying  methodology  of  the  multi-annual  Residual  Error  Rate  (RER)  reported  by 
the REA in its (draft) 2015 Annual Activity Report (AAR), and in doing so, to help 
the REA mitigate the discharge risk by enabling it to take appropriate actions, if any, 
before their disclosure in the final AAR and in the Synthesis Report. 

The review covered the following aspects: 

• The process and methodology for the calculation of the RERs; 
• The calculated RERs; 
• The presentation of the RERs in the draft AAR; 
• Compliance with the Standing Instructions for the 2015 AAR. 

The IAS reviewed the draft 2015 AAR and the preliminary RER calculations available 
on 09/02/2016 as well as the draft 2015 AAR provided to the SG. It also looked at the 
final  2015  AAR  to  check  whether  the  issues  detected  during  the  fieldwork  were 
correctly addressed. 

53 

 
 
 
The limited review also considered the results of the work done in 2014 by the IAS on 
the  audit  on  the  "Implementation  of  FP7  Control  Systems  in  REA-The  Assurance 
Process" and by the former REA's Internal Audit Capability on the audits on "Ex-post 
Audit Process" and on "Implementation of Ex-post audit Findings". 

The  audit  fieldwork  was  finalised  on  18  March  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

3.14.  Audit on Human Resources management in the REA 

Audit objectives and scope 

The overall objective of the audit was to answer the following question: 

Has  the  REA  designed  and  implemented  an  adequate  HR  management  process  to 
deploy  a  competent  (knowledgeable)  and  engaged  workforce  in  order  to  deliver  its 
priorities and core business? 

The audit covered in particular: 

• The design and implementation of the HR strategy; 
• The HR planning process, including workload assessment and staff allocation; 
• Selection, recruitment and retention of staff; 
• Knowledge management (competency management, training, coaching); 
• Monitoring and reporting on HR activities. 

There is no reservation in the REA's 2015 Annual Activity Report regarding the scope 
of this audit. 

The fieldwork was finalised on 15 July 2016. All observations and recommendations 
relate to the situation as of that date. 

Major audit findings 

The IAS identified one very important issue: 

•  Management of the selection process for contractual agents  

The procedure for the selection of contractual agents (which represent 76 % of current 
staff) does not clearly indicate where the original selection files should be kept, which 
documents should be part of the selection file, which documents should be registered 
in ARES and which ones should be kept only as a paper copies due to their sensitive 
nature.  Most  of  the  selection  files  examined  were  stored  in  different  places,  and 
additionally had key documents missing. 

Moreover, some of the panel members and staff committee observers had not received 
sufficient  guidance  on  the  selection  and  recruitment  procedures,  despite  not  having 
sufficient experience and knowledge of the process.  

54 

 
 
 
The current procedure does not clearly outline the controls over the extraction of data 
from  dedicated  databases  (EPSO,  SADB)  during  the  selection  process,  aimed  at 
ensuring  candidates'  compliance  with  the  selection  criteria.  The  testing  carried  out 
during the audit revealed one case where the approach of the selection panel regarding 
application of the essential selection criteria for the candidates was not compliant with 
the internal rules.  

Recommendation 

To address the issue, the IAS formulated the following recommendation: 

•  Management of the selection process for contractual agents 

The Agency should: 

• Update  and  revise 

its  selection  procedures  by  clearly  defining 

tasks  and 
the 

responsibilities  regarding 
documentation of the process, filing and archiving; 

the  checks 

to  be  performed,  and  rules  on 

• Provide training sessions on the roles and obligations of the selection panels for all 

panel members, secretaries and chairs; 

• Ensure that controls over the selection process are systematically implemented. 

3.15.  Audit on procurement in DG RTD 

Audit objectives and scope 

The overall objective of the audit was to assess whether the internal control system in 
place  in  DG  RTD  is  effective  in  ensuring  the  legality  and  regularity  of  the 
procurement management process. 

In  particular,  the  audit  assessed  whether  the  internal  control  system  provides 
reasonable assurance regarding: 

• Compliance with the Financial Regulation, its Rules of Application and the specific 

legal basis; 

• The effectiveness and efficiency of the processes, including management monitoring 
and  reporting,  and  the  need  for  simplifying  internal  administrative  rules  and 
procedures. 

The scope of the audit covered the procurement process, from the identification of the 
needs to the contract execution. 

There  are  no  observations/reservations  in  the  2015  Annual  Activity  Report  of  DG 
RTD that relate to the area audited. 

The  fieldwork  was  finalised  on  5  July  2016.  All  observations  and  recommendations 
relate to the situation as of that date. 

Major audit findings 

The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

55 

 
 
 
3.16.  Audit on H2020 project management in DG RTD 

Audit objective and scope 

The  overall  objective  of  the  audit  was  to  assess  the  effectiveness  of  the  project 
management process with a focus on: 

• The design of the guidance and procedures by the Common Support Center (CSC); 
• The implementation of the project management process in DG RTD. 

The audit covered the design by the CSC and the implementation in DG RTD of: 

• The  monitoring  approach  (desk  or  on  the  spot  checks  or  reviews,  using  internal  or 
external expertise, in terms of frequency of review) in line with the inherent risks of 
the projects; 

• Assessment of the activities of the projects based on deliverables and reports; 
• Selection of the appropriate course of action in the case of underperforming projects; 
• Amendments to the grant agreements. 

The  audit  assessed  how  DG  RTD  ensures  that  project  activities  were  carried  out  as 
agreed  and  that  the  project  deliverables  are  produced  as  envisaged.  The  monitoring 
and  assessment  of  the  scientific  content  of  the  funded  projects  during  project 
management was not included in the audit scope. 

Existing automated controls were considered as part of the audited process. However, 
the IT tools as such were not in the scope of the engagement.  

On  the  Strategy  for  an  effective  dissemination  and  exploitation  of  Horizon  2020 
research results, the audit fieldwork only covered the aspects of dissemination to be 
addressed during the assessment of the periodic reporting, i.e. mainly the assessment 
of  the  publishable  summary  and  the  review  of  the  progress  reached  in  the 
implementation of the project's Dissemination plan. 

There are no observations/reservations in the 2015 Annual Activity Report that relate 
to the area/process audited. 

The  fieldwork  was  finalised  on  13  December  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The IAS identified one very important issue: 

•  Determining the level of monitoring for projects 

H2020 implementing bodies have not reached a consensus on how project monitoring 
should  be  implemented.  At  the  level  of  the  CSC,  the  existing  guidance  that 
recommends to define the level of monitoring on the basis of a project's risk profile, is 
not mandatory and is only presented as a good practice. 

DG  RTD  does  not  systematically  apply  the  good  practice  proposed  by  the  CSC  to 
ensure that the level of monitoring is based on a sound analysis of the risks or on the 
specificities  of  the  projects.  In  principle,  the  project  officers  decide  on  the  specific 
monitoring  measures  they  want  to  apply,  resulting  in  different  practices  observed 
56 

 
 
 
 
between Directorates and, sometimes, to the use of a sub-optimal mix of monitoring 
tools as this would require a derogation from the rules or practices established at DG 
or Directorate level. 

Recommendations 

•  Determining the level of monitoring for projects  

The  CSC  should  adopt  rules  on  project  monitoring  to  ensure  that  the  implementing 
bodies  adapt  the  level  of  their  project  monitoring  based  on  a  sound  project  risk 
assessment  methodology.  These  rules  should  also  aim  at  harmonising  the  practices 
amongst the implementing bodies. 

DG RTD should cooperate with the CSC for the establishment of these rules. It should 
implement these new rules by ensuring that the existing internal rules and procedures 
on missions and experts do not hamper their application. 

3.17.  Audit on the implementation of the FP7 ex-post audit strategy by the Common 
Audit Service in DG RTD 

Audit objectives and scope 

The objective of this audit report was to conclude whether the objectives of the FP7 
ex-post audit strategy are achieved, by assessing the effectiveness of the key processes 
and internal controls designed and implemented by the Common Audit Service (CAS), 
with due consideration given to efficiency and economy principles. 

The audit focused on: 

• The control environment in the CAS; 
• The audit strategy and planning; 
• The execution of audit engagements; 
• The monitoring and reporting functions; 
• The supervision and quality assurance functions. 

The  IAS  finalised 
recommendations relate to the situation as of that date. 

the  fieldwork  on  19  July  2016.  All  observations  and 

Major audit findings 

The IAS identified two very important issues: 

•  Delivery of individual audit engagements 

There are significant delays in finalising an important number of audit engagements. 
The root causes for the delays in finalising the audit engagements relate to: a) a long 
decision-making  process for sensitive cases; b) the lack of procedures to finalise the 
engagements  under  special  circumstances  or  where  systemic  issues  are  identified;  c) 
ineffective  backup  procedures  in  cases  of  long-term  absences;  d)  ineffective 
prioritisation of long-outstanding engagements. 

•  Audit planning, monitoring and reporting  

57 

 
 
 
The review of the internal processes of the CAS highlighted that some activities (i.e. 
planning  of  audit  engagements,  staff  planning,  and  performance  monitoring  and 
reporting)  are  not  sufficiently  developed  to  support  the  level  of  activity  of  the  CAS. 
This is significant given that the implementation of the H2020 ex-post audit strategy 
will bring new challenges, thus requiring a more mature internal control system.  

Recommendations 

To address these issues the IAS formulated the following recommendations: 

•  Delivery of individual audit engagements  

The CAS should significantly reduce the average time needed to close the audit files 
by addressing the root causes of the delays. 

•  Audit planning, monitoring, and reporting  

For improved planning and monitoring of audit engagements, the CAS should develop 
a  list  with  the  audits  planned  for  the  year.  It  should  also  set  target  dates  and 
completion  dates  for  the  key  audit  milestones.  To  better  identify  resource  gaps  and 
establish  priorities,  the  CAS  should  reconcile  the  available  resources  with  those 
necessary  to  complete  the  plan.  It  should  also  set  budgets  for  audit  and  non-audit 
activities and make better use of the existing time-recording system. To enhance the 
monitoring and reporting activities, the CAS should make use of SMART objectives, 
indicators and targets. In addition, the H2020 annual targets should be reviewed on a 
regular basis against the pace that the beneficiaries are lodging cost statements. 

4.  EXTERNAL ACTIONS 

4.1.   Audit on payment deadlines in DG DEVCO 

Audit objectives and scope 

The overall objective of the audit was to assess the adequacy and effectiveness of the 
processes in place in DG DEVCO to comply with the rules and regulations, guidance 
and instructions related to the time limit to pay. 

The audit focused on the payment process put in place by DG DEVCO in direct and 
indirect management for the EU budget and the European Development Fund. 

The audit included an assessment of the following aspects: 

• Appropriateness of contractual conditions with external parties fixing the time limit 
to  pay  and  for  handling  and  transmitting  invoices  to  DEVCO  Headquarters/EU 
Delegations; 

• Effective  processing  of  payment  transactions,  starting  with  the  handling  of  the 

invoices; 

• Effective implementation of the encoding, registration and suspension procedures in 

DG DEVCO; 

• Adequacy  of  the  support  (procedures,  guidance,  training)  provided  on  payment 

processes; 

58 

 
 
• Adequacy and effectiveness of accounting, quality control, monitoring and reporting 

activities in place concerning payment deadlines. 

The  transaction  testing  covered  payments  processed  by  Headquarters  and  by  seven 
Delegations  during  2015.  Throughout  the  audit,  the  IAS  also  addressed  the  risks 
related to the open recommendation from the previous audit on payment deadlines in 
DG DEVCO, which was not yet sufficiently mitigated. 

The  scope  of  the  current  audit  did  not  include  IT  systems  supporting  the  audited 
process (CRIS and ABAC). 

There  are  no  observations/reservations  in  the  2015  Annual  Activity  Report  of  DG 
DEVCO that relate to the area/process audited. 

The fieldwork was finalised on 27 June 2016. All observations and recommendations 
relate to the situation as of that date. 

Major audit findings 

The IAS identified two very important issues: 

•  Encoding of payment requests in DG DEVCO) 

Despite  DG  DEVCO's  awareness  that  registration  of  payment  requests  in  the 
accounting system is one of the major causes contributing to its weak performance as 
regards respect of payment deadlines, it has not been able to find a workable solution 
to address the issue. For the period January-June 2016, the statistics from DG BUDG 
show an average time to register of 9 working days for DG DEVCO (compared to the 
corporate  reference  of  5  working  days).  This  long  registration  delay  is  due  to 
inefficiencies  in  the  physical  circulation  and  the  clerical  treatment  of  payment 
requests. 

•  Monitoring of the payment process in DG DEVCO 

In  2015,  DG  DEVCO  introduced  the  Portfolio  Management  Dashboard,  which 
provides  in  real  time  the  list  of  upcoming  and  already  late  payments,  based  on  data 
coming from ABAC. However, despite continuous refinement and improvement of the 
dashboard,  it  does  not  provide  for  active  monitoring  by  alerting  the  actors  in  the 
financial  circuits  of  possible  delays.  Moreover,  DG  DEVCO  is  not  using  and 
analysing the available data on the time spent in the different phases of the payment. 
Consequently, it is unable to detect and address appropriately the underlying reasons 
for delays. 

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  Encoding of payment requests in DG DEVCO 

DG  DEVCO  should  remind  staff  of  the  procedure  in  place  to  receive  and  register 
payment  requests  in  the  five  working-days  deadline.  It  has  also  to  ensure  that  all 
directorates monitor and manage their correct application in order to comply with the 
time to register. 

59 

 
 
 
•  Monitoring of the payment process in DG DEVCO 

DG  DEVCO  should  internally  set  deadlines  for  each  step  in  the  financial  circuit, 
monitor them and alert the responsible actors in the workflow of actual and potential 
delays.  Furthermore,  DG  DEVCO  should  also  monitor  the  use  of  suspensions  and 
signal payments with long suspension periods. 

4.2.   Audit on performance management system in DG DEVCO 

Audit objectives and scope 

The overall audit objective was to assess the adequacy of DG DEVCO's performance 
management system to plan, monitor and report on the achievement of its objectives. 

The audit covered in particular:  

• The  setting  of  objectives  and  the  related  indicators  in  the  different  performance 
management  tools:  Strategic  Plan  (SP)  and  Management  Plans  (MP),  and 
Programming  documents,  including  the  Multi-Annual/National/Regional  Indicative 
Plans (MIPs/NIPs/RIPs), the related Annual Action Programmes (AAPs), the Action 
Documents (ADs) and any other programme documents; 

• The monitoring of the objectives, performance indicators and related targets; 
• The  annual  reporting  in  the  External  Assistance  Management  Reports  (EAMRs), 
Sub-Delegated  Authoring  Officer  Reports  (SDAOs)  and  Annual  Activity  Report 
(AAR); 

• The set-up of the Result Framework (RF) and the first year of reporting; 
• Setting project-level indicators, monitoring and reporting, including project closure. 

The  scope  of  the  audit  included  planning  and  reporting  documents  prepared  in  the 
period 2014-2015, as well as the setting of objectives and indicators in the 2016-2020 
SP and 2016 MP (which were finalised during the fieldwork and the finding validation 
phase). 

The  scope  of  the  audit  did  not  include  the  evaluation  activities  and  the  (traditional) 
results-oriented monitoring (ROM), since they were included in the scope of a recent 
audit  performed  by  the  European  Court  of  Auditors  (Special  report  18/2014 
"EuropeAid's Evaluation and Results-Oriented Management Systems").  

The fieldwork was finalised on 30 June 2016 with the assessment of the final versions 
of  the  2016-2020  SP  and  2016MP.  The  observations  and  recommendations  relate  to 
the situation as of that date. 

Major audit findings 

The IAS has identified one very important issue: 

•  Monitoring of and reporting on DG DEVCO's performance towards achieving 

its objectives  

There is no systematic monitoring of progress towards the achievement of objectives 
and  targets  set  in  the  MP,  as  the  majority  of  the  result  and  output  indicators  in  DG 
DEVCO's  2015  MP  and  Directorates'  MPs  were  not  regularly  monitored  during  the 

60 

 
 
 
year  and  were  calculated  at  year-end  only  for  reporting  in  the  AAR.  For  MP  DEL, 
most of the EUDs sampled do not monitor the achievement of their objectives at all 
(not even at year-end). 

In addition, there is no central guidance on monitoring and reporting on the objectives 
and  targets  set  in  the  ADs.  Although  DG  DEVCO  monitors  the  performance  of 
individual  projects,  the  results  of  the  projects  belonging  to  the  same  AD  are  not 
consolidated to provide information on the achievement of the overall objectives. 

In terms of reporting, the type of information on DG DEVCO's performance provided 
by  the  different  Strategic  Planning  and  Programming-related  reports  (AAR,  SDAO 
reports,  EAMRs)  is  limited  and  does  not  give  an  actual  assessment  of  whether 
objectives have been achieved or not. At the level of programmes, there is no annual 
reporting  on  the  progress  made  toward  the  achievement  of  the  objectives  set  in  the 
programming  documents,  which  consolidate  the  results  measured  at  the  level  of  the 
projects. 

Recommendations 

To address this issue, the IAS formulated the following recommendation: 

•  Monitoring of and reporting on DG DEVCO's performance towards achieving 

its objectives  

DG DEVCO should significantly improve its monitoring and reporting arrangements 
to  ensure  that  key  indicators  established  in  the  different  performance  systems  are 
systematically  and  regularly  monitored  and  appropriate  information  is  provided  to 
senior  management  and  stakeholders  on  a  timely  basis.  The  frequency  of  the 
monitoring  and  reporting  should  be  defined  taking  into  account  the  nature  of  the 
objectives to monitor, the type of indicator and the collection methods as well as the 
monitoring  and  reporting  needs  and  expectations  expressed  by  management  and 
stakeholders. 

4.3.   Audit on direct management of grants in DG DEVCO (DCI and EDF) 

Audit objectives and scope 

The  overall  audit  objective  was  to  assess  the  control  systems  put  in  place  by  DG 
DEVCO  to  manage  grants  under  direct  management  in  order  to  achieve  the 
programme objectives and to ensure the legality and regularity of the expenditure. 

The  audit  covered  in  particular  DG  DEVCO's  processes  for  managing  grants  under 
direct management assessed to have the highest risks, namely: 

• Alignment  of  the  grant's  funded  activities  with  DG  DEVCO's  strategic  and 
the  programming  documents  (Annual  Action 

operational  objectives  set 
Programme and the related Action Documents); 

in 

• Assessment  if  the  grant  agreements  provide  an  effective  framework  for  the 
implementation of the projects involved (e.g. indication of the expected results and 
time  limits,  grant  amendments  not  changing  significantly  the  budget,  the  time  for 
implementation or the core of the action); 

61 

 
 
• Operational  monitoring  and  reporting  of  the  implementation  of  the  projects, 

including the assessment of their final results; 

• Review  of  the  process  to  check  that  the  payments  are  in  accordance  with  the 
contractual  provisions  (reports  submitted  by  the  grant  beneficiaries,  expenditure 
verification  reports  prepared  by  external  auditors  and  the  related  checks  performed 
by DG DEVCO). 

The scope of the audit did not include: a) the selection and award of grants and b) the 
evaluation activities (included in the scope of the European Court of Auditors' Special 
Report  18/2014  on  "EuropeAid's  Evaluation  and  Results-Oriented  Management 
Systems"). 

The  fieldwork  was  finalised  on  1  December  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

4.4.   Audit on the instrument contributing to stability and peace in FPI 

Audit objectives and scope 

The overall objective of the audit was to assess the effective and efficient management 
of the Instrument contributing to Stability and Peace (IcSP) by the FPI. 

The processes in the scope of the audit were:  

• Identification/formulation of crisis response actions; 
• Contracting; 
• Operational and financial monitoring; 
• Reporting from EU Delegations to FPI Headquarters. 

The  audit  scope  included  Art.  3  and  Art.  4  of  the  IcSP,  while  Art.  5  of  the  IcSP, 
managed by DG DEVCO, was outside the audit scope. In addition, the audit did not 
cover  the  IcSP  legal  basis,  programming  (Art.  4),  ex-post  controls  of  projects,  and 
evaluations. 

There  are  no  observations/reservations  in  FPI's  2015  Annual  Activity  Report  that 
relate to the area/process audited. 

The  fieldwork  was  finalised  on  22  November  2016.  The  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

62 

 
 
 
 
4.5.   Limited review of DG NEAR's residual error rate methodology and calculation 
for the 2015 reporting year 

Audit objectives and scope 

The  objective  of  this  limited  review  was  to  review  the  calculation  and  underlying 
methodology of the Residual Error Rates (RER) reported by DG NEAR in its (draft) 
2015  Annual  Activity  Report  (AAR),  and  in  doing  so,  to  help  the  DG  mitigate  the 
discharge risk by enabling it to take appropriate actions, if any, before the disclosure 
of the error rates in the final AAR and in the Synthesis Report. 

The review covered the following aspects: 

• The  process  and  the  methodology  for  the  calculation  of  the  RERs  for  the  different 

programmes and management modes of DG NEAR; 

• The calculated RERs; 
• The presentation of the RERs in the draft AAR; 
• Compliance with the Standing Instructions for the 2015 AAR. 

The  IAS  reviewed  the  draft  2015  AAR  and  the  RER  calculations  available  on  11 
March  2016.  It  also  analysed  the  final  AAR  dated  5  April  2016  to  verify  to  what 
extent the recommendations and comments of the Draft Audit Report were taken into 
account in the final AAR. 

The main part of the fieldwork was finalised on 18 March 2016. All observations and 
recommendations  relate  to  the  situation  as  of  that  date  except  for  the  points  arising 
from changes introduced in the methodology for the calculation of the amount at risk 
from the draft to the final AAR that were analysed by the IAS after the issuance of the 
final  AAR  on  5  April  2016.  The  results  of  that  additional  analysis  have  been 
incorporated into this report where applicable together with DG NEAR's comments as 
appropriate. 

As the result of the fieldwork, the IAS issued two sets of recommendations in order to 
a)  address  immediately  the  issues  detected  in  the  2015  draft  AAR,  and  b)  identify  a 
long term solution for future AARs. 

The  sub-recommendations  related  to  the  2015  AAR  have  been  either  immediately 
implemented by DG NEAR or rejected. They have been followed up in the course of 
the  engagement,  when  analysing  the  2015  final AAR. Therefore,  no  action plan  was 
requested to the DG in response to them.  

Major audit findings 

The IAS has identified two very important issues: 

•  IPA - Indirect management with beneficiary countries  

The 2015 RER for indirect management with beneficiary countries (IPA), representing 
approx.  13%  of  the  payments  executed  in  2015,  is  based  solely  on  the  error  rates 
reported  by  the  Audit  Authorities  of  the  three  beneficiary  countries  (Turkey,  the 
former  Yugoslav  Republic  of  Macedonia  (fYRoM)  and  Croatia).  As  stated  in  the 
individual  country  reports,  the  audit  work  is  based,  for  different  reasons,  on  non-
statistical  samples  and  in  some  cases  the  sample  selection  is  not  purely  random.  In 
63 

 
 
 
addition, no specific checks are performed by DG NEAR to obtain assurance on the 
reliability and representativeness of these results as a basis for the RER and no other 
sources  are  used  by  DG  NEAR  to  corroborate  them.  For  those  reasons,  the  residual 
error  rate  of  0.02%  included  in  the  2015  AAR  for  the  enlargement  programmes 
implemented through IMBC is neither representative nor reliable. 

Concerning  the  calculation  of  the  amount  at  risk,  for  2015  DG  NEAR  used  a  range 
which  is  based  on  figures  which  are  not  reliable.  In  particular,  DG  NEAR  used  a 
predicted  error  rate  based  on  certain  assumptions  rather  than  the  actual  RER  which 
was already known at the time of the calculation. 

•  Enlargement – Direct management  

A  number  of  detected  errors  included  in  the  ex-post  audit  reports  on  Enlargement  – 
direct  management  addressed  to  the  EU  Delegations  in  2015  are  reported  as  non-
quantifiable.  However,  the  approach  used  by  DG  NEAR  for  the  assessment  of 
procurement  errors  is  not  in  line  with  the  Standing  Instructions  and  some  of  these 
errors  may  be  quantified  if  the  methodology  prescribed  by  the  Standing  Instructions 
was  applied.  This  would  lead  to  a  higher  RER  for  this  category  of  expenditure  that 
represents approx. 16% of the payments executed in 2015. 

In addition, the calculation of the RER for Enlargement– direct management does not 
take into account the correct sampling interval for the establishment of the projected 
errors  for  the  period  2013  –  2015.  A  recalculation  leads  to  an  increase  of  the  RER 
from 1.48% to 1.62%. 

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  IPA - Indirect Management with Beneficiary Countries 

For the 2015 AAR, the IAS recommended DG NEAR to assess the reliability of the 
error  rate  information  reported  by  the  national  Audit  Authorities  of  the  beneficiary 
countries, taking into account their annual reports and other available assessments, in 
order to reach a conclusion whether the reported error rate for each country is reliable 
and  based  on  an  appropriate  and  robust  methodology.  In  the  2015  final  AAR,  DG 
NEAR  maintained  the  2015  RER  of  0.02%  despite  the  criticisms  raised  by  the  IAS. 
However, it disclosed the fact that "in 2016, DG NEAR intends to further fine-tune its 
approach  to  calculating  the  RER  in  this  control  environment,  as  well  as  offer 
continued support to the audit authorities."  

For  the  future  AARs,  DG  NEAR  should  define  a  sound  methodology  for  the 
calculation of the RER and the amount at risk which follow a multi-annual approach 
and  should  develop  guidance  on  the  assessment  of  the  reliability  of  the  error  rates 
reported  by  the  Audit  Authorities  and  the  calculation  method  for  the  RER  for  IPA- 
IMBC. 

•  Enlargement – Direct Management 

The  IAS  recommended  to  DG  NEAR  to  recalculate  the  RER  for  the  2015  AAR  by 
applying a sampling interval based on the entire 2013-2015 population and to assess if 
the  non-quantifiable  errors  reported  during  2015  can  be  quantified  based  on  the 

64 

 
 
 
methodology  prescribed  by  the  Standing  Instructions.  Both  points  were  taken  into 
account in the 2015 final AAR. 

Regarding  the  2016  AAR,  DG  NEAR  should  update  the  methodology  for  the 
treatment  of  errors  (quantifiable  versus  non-quantifiable)  based  on  the  Standing 
instructions for the AAR. 

4.6.   Audit on risk management in DG NEAR 

Audit objectives and scope 

The overall objective of the audit was to assess the effectiveness of DG NEAR's risk 
management  process  to  identify,  assess  and  manage  critical  and  significant  risks  in 
line with the accepted risk level. 

The  audit  covered  DG  NEAR's  risk  management  process,  from  the  identification  of 
objectives  until  the  monitoring  of  and  reporting  on  the  implementation  of  the  risk 
responses. 

The audit looked at the design of the risk management process and its implementation 
at DG NEAR Directorate-General and Directorate level. 

The following processes were out of the scope of the present engagement: 

• The  objective-setting  exercise:  the  audit  looked  at  the  choice  of  the  objectives  for 
which  the  risks  had  been  identified  and  assessed,  but  not  at  the  process  to  set  the 
objectives; 

• DG  NEAR's  specific  risk  assessment  exercises  (e.g.  in  the  context  of  IT  project 
management, external audit plan, Business Continuity Plan or IT security plans): the 
IAS  looked  at  whether  these  exercises  had  been  integrated  in  order  to  have  a 
complete  picture  of  the  risk  management  process  in  DG  NEAR,  but  did  not  audit 
them in detail. 

The fieldwork was finalised on 1 April 2016. All observations and recommendations 
relate to the situation as of that date. 

There  are  no  observations/reservations  in  DG  NEAR's  2015  Annual  Activity  Report 
related to the audited area/process. 

Major audit findings 

The IAS identified four very important issues: 

•  Risk management framework  

DG  NEAR  has  not  clearly  established  and  allocated  the  roles  and  responsibilities  of 
the various actors involved in the coordination of the risk management at central and 
Directorate  level  (Internal  Control  Coordinator  and  supporting  staff  and  risk 
management  coordinators  in  the  Directorates).  In  addition,  there  is  no  risk  steering 
committee  ensuring  a  high  level  coordination  of  the  risk  management.  Furthermore, 
there  is  no  integrated  management  of  risks,  encompassing  all  the  risk  assessment 
exercises performed in DG NEAR (e.g. coherent guidelines, unique methodology), to 
ensure cost-effectiveness and harmonisation. 

65 

 
 
 
•  Risk identification and assessment  

DG  NEAR  performs  its  risk  identification  and  assessment  mainly  through  desk 
reviews,  without  complementing  them  with  other  techniques  such  as  workshops, 
questionnaires, interviews or brainstorming sessions. In addition, the risk management 
exercise focuses solely on critical risks. Consequently, any risk not assessed as critical 
at DG level is  not formally  identified, assessed and addressed, and no action plan is 
prepared,  monitored  and  reported  upon.  Furthermore,  no  instructions  have  been 
developed  at  DG  level  to  ensure  a  consistent  implementation  of  risk  management 
across  the  Directorates.  Finally,  DG  NEAR  has  not  established  a  risk  register  to 
document risks and mitigating actions. 

•  Risk acceptance, risk response and implementation  

DG  NEAR  has  neither  identified  its  risk  appetite  nor  issued  guidance  to  support  the 
identification  of  the  most  appropriate  risk  response.  The  description  of  the  risk 
response is often vague and the action plans are frequently too generic, without stating 
the process owners, milestones, and deadlines. 

•  Monitoring and reporting  

DG  NEAR  has  not  established  the  modalities,  scope,  timing  and  allocation  of 
responsibilities for reporting progresses on the implementation of the action plans. In 
addition,  there  is  no  central  monitoring  of  and  reporting  on the  risk  identified  in  the 
context  of  the  annual  risk  management  exercise  and  the  related  action  plan.  In 
addition,  there  is  no  evidence  of  a  regular  reporting  to  the  Cabinet  of  DG  NEAR's 
critical risks and mitigating actions. Furthermore, sensitive information was found in 
documents related to the DG NEAR's risk management without an adequate protection 
against inappropriate disclosure. 

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  Risk management framework 

DG NEAR should clearly describe and formally attribute the roles, responsibilities and 
tasks  in  the  risk  management  process,  and  should  establish  a  steering  committee  to 
ensure that the risk management process is coordinated and consistent across the DG. 
It  should  also  improve  the  coordination  and  synergy  of  its  various  existing  risk 
assessment exercises. 

•  Risk identification and assessment 

DG NEAR should improve the methodology used for identifying and assessing risks, 
and  enlarge  the  scope  of  the  risk  management  exercise  to  significant  risks  at 
Directorate level. It should also provide internal guidelines to clarify key aspects for 
risk management, and establish a risk register at both DG and Directorate level. 

•  Risk acceptance, risk response and implementation 

DG NEAR should define its acceptable risk level, ensure that each identified risk has a 
clear  risk  response,  and  the  mitigating  actions  are  clearly  formulated  with  formally 
assigned process owners, milestones and deadlines. 

66 

 
 
•  Monitoring and reporting 

DG  NEAR  should  establish  proper  monitoring  and  reporting  arrangements,  with 
clearly established responsibilities. It should provide guidelines on data protection for 
the sensitive information included in the relevant risk management documents. 

4.7.   Review of the tender procedure EuropeAid/133797/DHL/SUP/XK, following the 
article 99(4) complaint received on 20 March 2016  (DG NEAR) 

Audit objectives and scope 

On 20 March 2016 the IAS received a complaint by e-mail under Article 99.4 of the 
Financial Regulation. The complaint contains allegations regarding the high price that 
was  due  to  be  paid  by  the  EU  Office  in  Pristina  for  the  supply  of  12  vehicles  and 
therefore  the  non-compliance  of  the  award  of  the  contract  with  the  principles  of 
economy,  efficiency  and  effectiveness.  The  complainer  asked  the  IAS  to  check  the 
information provided and, if there was a case to answer, to stop the procurement. 

While  the  IAS  has  no  management  responsibility  and  cannot  take  decisions 
concerning a tender procedure, it decided, in the context of its mandate, to perform a 
desk review of the tender procedure concerned. 

Major audit findings 

The  review  resulted  in  a  few  issues  for  consideration.  However,  the  IAS  did  not 
require  DG  NEAR  to  prepare  an  action  plan  and  will  not  follow-up  the  issues  for 
consideration. 

4.8.   Audit on procurement under the Instrument for Pre-Accession (direct 
management and indirect management with beneficiary countries) – phase I 

Audit objectives and scope 

The objective of the audit was to assess whether procurement under the Instrument for 
Pre-Accession  -  direct  management  and  indirect  management  with  beneficiary 
countries  is  implemented  effectively  and  in  compliance  with  the  applicable  rules  to 
ensure the legality and regularity of operations. 

The specific objectives included an assessment of: 

• NEAR Headquarters (HQ) guidance/procedures on the procurement process; 
• DG NEAR monitoring arrangements on the procurement process; 
• Coordination,  planning  and  monitoring  of  the  procurement  process  by  EU 

Delegations (EUDs); 

• Implementation of procurement under direct management; 
• Ex ante controls performed by DG NEAR staff in EUDs on procurement procedures 
managed  by  beneficiary  countries  under  indirect  management  with  beneficiary 
countries, except Turkey. 

For both management modes, the audit covered: 

67 

 
 
 
• NEAR HQ guidance, training and procedures on the procurement process; 
• NEAR HQ monitoring arrangements on the procurement process; 
• Coordination, planning and monitoring of the procurement process by EUDs; 
• The different phases of the procurement process from drafting the terms of reference 
/ technical specifications to the signature of the contract and its early amendments. 

There  are  no  observations/reservations  in  DG  NEAR's  2015  Annual  Activity  Report 
that relate to the area/process audited. 

The  fieldwork  was  finalised  on  30  November  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

5.  EDUCATION AND CITIZENSHIP 

5.1.   Audit on performance management systems in DG EAC, including the 
contributions of executive agencies and national agencies to the achievement of policy 
objectives 

Audit objectives and scope 

The overall objective of the audit was to assess the extent to which DG EAC has an 
adequate performance management framework/system in place both for its day-to-day 
operational and administrative activities (internal) and for the delivery of programme 
and policy objectives (external).  

The audit reviewed the internal processes for defining the DG's performance systems 
and  establishing  its  objectives  and  indicators,  as  well  as  the  related  reporting, 
evaluation,  monitoring,  and  supervision  systems.  The  scope  also  included  the 
processes within DG EAC to ensure appropriate contributions of EACEA, REA, and 
of  the  National  Agencies  (NAs)  to  the  monitoring  and  measurement  of  the  external 
performance. 

The audit covered the period 2014-2016. 

The supervision of EIT, the financial instruments delegated to EIF and the parts of the 
programmes whose implementation is delegated to other DGs, the performance of the 
NAs and DG EAC IT systems supporting the performance management systems were 
not included in the audit scope. 

There are no observations/reservations in DG EAC's 2015 Annual Activity Report that 
relate to the area/process audited. 

The  fieldwork  was  finalised  on  13  September  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

68 

 
 
 
The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations.  

5.2.   Audit on DG HOME's management of emergency assistance in the context of the 
migration crisis 

Audit objectives and scope 

The  overall  objective  of  the  audit  was  to  assess  if  DG  HOME  is  managing  the 
emergency assistance (EMAS) in the context of the migration crisis effectively, while 
still ensuring the legality and regularity of the underlying transactions.  

The  audit  covered  both  the  direct  management  of  EMAS  by  DG  HOME  and  the 
indirect  management  through  the  delegation  agreement  with  the  UNHCR.  For  the 
direct  management  part  of  EMAS  (representing  83%  (384.5  million  EUR)  of  the 
EMAS funds), we focused on: 1) the needs  assessment 2) the application  process 3) 
the  evaluation  process  4)  the  award  and  contracting  process  5)  monitoring  and 
reporting and 6) control systems for making payments (including ex-post controls).  

For  the  indirect  management  part  (representing  the  17%  (80  million  EUR  of  the 
EMAS  funds),  we  focused  on  assessing  the  processes  and  procedures  for  delegating 
and supervising the management of the EMAS actions by UNHCR. 

There are no reservations in the 2015 Annual Activity Report of the DG that relate to 
the area/process audited. 

The  fieldwork  was  finalised  on  21  September  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The IAS identified four very important issues: 

•  Direct grant management - Planning, evaluation and contracting  

Although  there  is  no  legal  obligation  for  DG  HOME  to  perform  a  needs  assessment 
and the situation on the ground is rapidly and constantly changing, the IAS has noted 
that  various  documents  exist,  which  assess  the  different  Member  States  (MS) 
underlying  funding  needs.  However,  currently  these  are  not  brought  together  in  an 
overall  analysis  to  clearly  contribute  to  and  support  the  EMAS  Annual  Work 
Programme  (AWP)  and  its  various  updates.  In  addition,  these  are  not  systematically 
taken into account for the evaluation process of applications for EMAS funding.  

Furthermore, the guidance to applicants lacks clear instructions on the information to 
be provided about the emergency situation concerned, the proposed actions and their 
impact. Consequently, the quality of a number of the applications reviewed by the IAS 
was insufficient, particularly regarding Key Performance Indicators and targets. Also, 
certain  applications  reviewed  by  the  IAS  lacked  sufficient  information  on  similar 
actions that may have been funded under previous programmes or from other sources. 

Internal guidance on evaluation does not clearly explain the timing and objectives of 
the first assessment of applications. As a result, there were different approaches on the 
timing,  depth  and  scope  of  this  assessment.  In  addition,  the  respective  roles  and 
69 

 
 
responsibilities of the country desk officer and the project officer were unclear and the 
audit trail not always complete.  

As  regards  the  evaluation  process,  DG  HOME's  requirement  that  applications  are 
evaluated  beforehand  by  three  individual  internal  evaluators  should  be  seen  as  good 
practice.  However,  the  guidance  and  training  provided  to  evaluators  is  lacking  with 
respect to assessing applications against the award criteria, the budget forecast and the 
complementarity with other actions funded. As a result, there are inconsistencies and 
gaps  in  the  way  the  individual  assessments  were  performed  and  documented.  In  a 
limited  number  of  applications,  individual  evaluations  were  not  made  as  the  process 
had to be conducted at very short notice in response to very urgent, high level political 
decisions.  The  IAS  acknowledges  that  in  such  situations  it  is  not  always  realistic  to 
follow the normal procedures. 

In addition, the minutes (reports) of the Evaluation Committee, which are the basis for 
the  award  decision  by  the  authorising  officer,  lacked  sufficient  justification  or 
explanation  for  the  decisions  taken  or  did  not  include  information  beyond  what  is 
already in the individual evaluation forms. However, the IAS notes that the minutes of 
recent Joint AMIF-ISF committee meetings are more detailed in this regard. 

For  a  number  of  award  decisions  taken  before  the  reorganisation  of  Directorate  E  at 
the  end  of  2015,  the  actual  grant  recipient  was  not  clearly  identified.  In  addition, 
certain  issues  which  required  clarification  prior  to  the  contracting  phase  were  not 
properly  documented and  therefore  it was  not  clear to the IAS that these had in fact 
been fully addressed in the grant agreement.  

Where grants have been amended, these sometimes lacked a documented justification. 
Furthermore,  the  approach  taken  was  not  always  consistent,  for  example  as  regards 
granting  or  refusing  an  extension  of  the  period  for  providing  the  final  reporting 
package  or  when  properly  justifying  the  amendment.  Internal  procedures  were  not 
always  respected,  for  example  on  consulting  policy  units  on  amendments  involving 
significant budget transfers or changes of the content of the actions. 

•  Direct grant management – Monitoring, payment and controls  

As  part  of  its  monitoring  arrangements  DG  HOME  carries  out  a  range  of  different 
missions to MS, but it has not yet defined the precise nature, purpose and timing for 
the  different  types  of  mission.  Consequently,  they  are  not  always  fully  effective  in 
helping  to  assess  the  actual  progress  made  towards  achieving  the  objectives  of  the 
actions  funded.  Furthermore,  the  grant  agreements  with  beneficiaries  did  not  always 
include provisions for reporting progress made to DG HOME. There was no evidence 
that the reports submitted by beneficiaries have actually been systematically reviewed 
and followed up by DG HOME. 

Concerning  payments,  the  pre-financing  for  EMAS  grants  is  typically  set  at  80%, 
although  some  grants  involve  a  higher  risk  (e.g.  resulting  from  the  urgent  nature  of 
actions or the retroactive funding of actions already started). It is usually good practice 
to use lower pre-financing rates or pay out in tranches in order to mitigate situations 
where specific grants present higher risks. This was done only for two of the EMAS 
grants reviewed by the IAS.  

Furthermore,  although  it  is  still  early  in  the  implementation  phase  and  few  final 
payments  have  been  made,  the  IAS  nevertheless  found  that  the  final  payment 
procedure  needs  to  be  improved  in  certain  areas.  Also,  due  to  the  heavy  reliance 
70 

 
 
placed by DG HOME on audit certificates, it will need to monitor very carefully the 
associated risks as regards their quality. Audit work performed by the IAS on the use 
of audit certificates in other policy areas highlights certain issues in this respect.  

Finally,  despite  the  fact  that  EMAS  grants  are  higher  risk  because  of  their  urgent 
nature and the significant increase in budgetary terms, the DG has not yet developed a 
fully-fledged control strategy comprising all control layers and procedures (i.e. ex-ante 
and ex-post; financial and operational). Also, there is not yet an audit strategy/plan in 
place for EMAS grants. In addition, the DG's overall Anti-fraud strategy does not yet 
fully take account of EMAS related risks. 

•  Indirect grant management - Delegation Agreement with UNHCR  

The basis for the Delegation Agreement (DA) with UNHCR lacks a solid documented 
analysis  of  needs  and  consultation  of  other  relevant  services.  Furthermore,  there  are 
gaps  in  the  design  of  the  monitoring  and  supervisory  arrangements.  In  particular, 
objectives  are  not  clear  or  specific  enough,  most  actions  lack  specific  targets,  key 
performance  indicators  and  milestones.  Although  the  Financial  and  Administrative 
Framework  Agreement  (FAFA)  between  the  EU  and  the  UN  provides  the  overall 
framework  for  Commission  controls  on  UN-led  projects  financed  by  the  EU,  DG 
HOME  has  not  yet  defined  its  own  specific  control  strategy  for  EMAS  projects 
implemented by UNHCR. Finally, the DG has not assessed the cost-efficiency of the 
actions included in the DA.  

•  Complementarity of EMAS with other DG HOME funding  

Whilst the funding of actions to address the migration crisis through EMAS is on the 
increase,  the  absorption  rate  under  the  shared  management  2014-20  National 
Programmes is very low for most MS. This is due to a combination of factors, such as 
the  late  adoption  of  the  legal  base  and  delays  in  the  designation  of  Responsible 
Authorities.  Consequently,  MS  have  found  it  difficult  to  mobilise  the  necessary 
funding from the National Programmes to address migration issues as quickly as has 
been possible under the EMAS mechanism. 

Although  it  is  too  early  to  assess  definitively  at  this  stage,  there  is  a  risk  that  the 
flexibility offered by the EMAS tool, coupled with the fact that this funding comes on 
top of the allocation to the MS under the National Programmes, may further contribute 
to the low take-up of the National Programmes. In certain cases, it would appear that 
more use could have been made of funding under the National Programmes to provide 
more  sustainable  and  longer-term  results.  This  is  very  clear  in  the  case  of  Greece, 
where the lack of budgetary and administrative capacity of the Greek government has 
led to EMAS being used to cover almost all funding and financial support needs for 
the management of the migration crisis. 

Recommendations 

•  Direct grant management - Planning, evaluation and contracting 

DG HOME should strengthen the needs assessment process, including the underlying 
analysis.  It  should  also  improve  its  guidance  on  the  key  steps  of  the  EMAS 
management process.  

•  Direct grant management – Monitoring, payment and controls 

71 

 
 
The  DG  should  finalise  and  establish  its  procedures  for  monitoring  EMAS  grants, 
clarifying  how  the  different  monitoring  tools  complement  each  other  in  order  to 
provide  sufficient  assurance  on  EMAS  grants  implementation.  DG  HOME  should 
continue to apply a more risk-based approach to pre-financing on an exception basis 
and plan to monitor the quality of audit certificates. The control strategy for the direct 
management  of  EMAS  should  be  defined,  comprising  all  control  layers  and 
procedures  (i.e.  ex-ante  and  ex-post;  financial  and  operational),  as  well  as  the  audit 
strategy and audit plan. 

•  Indirect grant management - Delegation Agreement with UNHCR 

DG HOME should ensure that the decision process, including needs analysis, for any 
future  modifications/amendments  or  extension  of  the  DA  is  sufficiently  justified, 
consulted  with  relevant  parties  and  adequately  documented.  The  DG  should  ensure 
that well-defined objectives and specific monitoring provisions for the funded actions 
in line with FAFA are established.  

•  Complementarity of EMAS to other DG HOME funding 

DG  HOME  should  perform,  in  the  context  of  its  preparations  for  the  post  2020 
programming period and the mid-term evaluation of the AMIF and ISF, an analysis of 
the  'lessons  learned'  from  the  first  years  of  implementation  of  the  National 
Programmes  and  EMAS  and  of  the  complementarity  between  the  National 
Programmes  and  the  EMAS.  This  should  be  used  to  feed  into  the  re-programming 
(amendments  of  National  Programmes)  or  re-orientation  of  the  different  funding 
tools/resources available to help in addressing the migration crisis. 

5.3.   Consulting engagement in DG HOME on the methodology for determining the 
'materiality level' and measuring the 'residual amount at risk' for the Annual Activity 
Report 

Audit objectives and scope 

The overall objective of the engagement was to review the processes put in place by 
DG  HOME  for  determining  the  'materiality  level'  and  for  measuring  the  'residual 
amount at risk' in the context of its reporting obligations in the Annual Activity Report 
(AAR) and to provide advice on potential improvements.  

The scope of the consulting engagement covered two areas, namely:  

• The process of determining the 'materiality level' (Part 1 of the engagement); 
• The process of measuring the 'residual amount at risk' (Part 2 of the engagement). 

According  to  DG  BUDG  Guidance,  in  order  to  come  to  a  sound  conclusion  on 
whether to qualify the Authorising Officer by Delegation's (AOD) declaration with a 
reservation and, if so, to estimate its impact in monetary terms the following approach 
(the "3+1 steps" approach) should be followed: 

• Step 1: calculating the representative detected error rate in a sample of transactions 
and taking account of any corrections made for the calculation of the residual error 
rate in the entire population; 

72 

 
 
 
• Step 2: estimating the financial exposure as (net) 'amount at risk' to the value of the 
relevant  payments  authorised  during  the  reporting  year,  based  on  those  error  rates 
calculated for a population of transactions mostly authorised in previous years; 

• Step  3:  relating  the  'amount  at  risk'  for  the  activity  considered  to  the  relevant 

aggregation level for determining whether a reservation would be due; 

• Step 4: "if" a reservation is entered, then assessing its relative impact on the AOD's 

overall assurance and Declaration. 

The  scope  of  our  consulting  engagement  as  regards  the  'materiality  level'  concerns 
'Step  3'  above,  in  particular  the  identification  of  the  most  appropriate  "relevant 
aggregation level" for determining whether a reservation would be due. The scope of 
our  consulting  engagement  as  regards  the  'residual  amount  at  risk'  concerns  'Step  1' 
and 'Step 2' above, in particular the assessment of the method used by DG HOME to 
calculate the 'residual error rate' (step 1) and the 'residual amount at risk' (step 2) for 
shared management. 

In the context of this consulting engagement, the IAS did not: 

• decide  on  the  'materiality'  level  to  be  used  in  the  AAR  or  on  the  method  used  for 

measuring the residual amount at risk. These are management (AOD) decisions; 

• perform substantive testing of the existing processes; 
• develop concrete templates to support the processes; 
• assess  the  clarity  and  completeness  of  the  information  provided  in  the  AAR  on 
'materiality level' and 'residual amount at risk' and drafting any input for the AAR. 

Major audit findings 

The consulting engagement resulted in a number of issues for consideration. As this is 
a consulting engagement and not an audit, the IAS does not follow-up these issues for 
consideration. 

5.4.   Audit on the management of grants under 2014-2020 Justice and Rights, Equality 
and Citizenship programmes in DG JUST 

Audit objectives and scope 

The overall objective of this compliance audit was to assess the design and effective 
application  of  the  internal  controls  for  managing  grants  under  the  2014-2020 
programming  period  by  DG  JUST.  In  particular,  the  audit  assessed  whether  the 
controls in place provide reasonable assurance regarding compliance with the relevant 
legislation  and  whether  they  ensure  sound  operational  management  of  the  grant 
management process. 

The audit took place at an early stage of the implementation of the Justice and Rights, 
Equality and Citizenship programmes. The grant management process could therefore 
only  be  audited  up  to  the  pre-financing  stage  of  the  2014  grant  procedures  and  the 
preparation of the 2015 calls/invitations for proposals. The audit also covered aspects 
of the implementation phase, to the extent possible in view of the early stages of the 
process overall. 

73 

 
 
 
 
 
The audit focused on: 

• Annual Work Programmes (AWPs) – preparation and publication; 
• Calls for proposals (CfP) – preparation, approval and publication; 
• Evaluation – selection of experts, evaluation of proposals, adjustment of proposals, 

award decision, and ex post publication of the list of awarded grants;  

• Contracting  –  formalisation  of  the  proposal  into  a  grant  agreement,  respect  of 

deadlines; 

• Payment – budgetary commitments, pre-financing;  
• Communication – provision of information to applicants; 
• Implementation phase – concerning mainly audit and control arrangements. 

There is a reservation in the 2015 Annual Activity Report of DG JUST concerning the 
high residual error rate (2,86%) in direct management grants.  

The fieldwork was finalised on 10 May 2016. All observations and recommendations 
relate to the situation as of that date. 

Major audit findings 

The IAS identified two very important issues: 

•  Evaluation process  

There  is  scope  for  considerable  improvement  in  the  overall  evaluation  process. 
Currently,  there  is  lack  of  guidance  to  support  evaluators  when  they  score  project 
proposals  and  inconsistencies  and  gaps  in  the  way  in  which  the  results  are 
documented, including the justification for third evaluations. Also, the process is less 
efficient because the irrelevant proposals could be eliminated at an early stage as part 
of  a  two-stage  procedure.  Currently,  this  is  not  done  and  all  the  proposals  are 
evaluated in depth as part of a single review. Moreover, only two evaluators evaluate 
all  the  proposals  and  an  additional  evaluator  could  help  eliminate  the  need  for 
reconciliations  and  lead  to  fewer  third  evaluations  being  needed.  Finally,  there  is  no 
overall panel review involving all the evaluators aimed at ensuring overall consistency 
and equal treatment of applicants. 

•  Contracting phase  

At  the  contracting  phase,  a  lack  of  guidelines  has  resulted  in  inconsistencies  in  the 
way in which proposed grant budgets are reviewed. Furthermore, this review process 
generally only starts after the award decision has been made, which is too late to be 
able  to  detect  potential  budgetary  problems.  Also,  whereas  adjustments  should  be 
flagged in the evaluation reports, in practice these are made only at the later, budget 
review stage.  

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  Evaluation process  

DG JUST should develop guidance on scoring, clearly justify in the evaluation reports 
the reasons for each third evaluation and perform a panel review with all the external 
74 

 
 
 
experts in order to compare and assess proposals. In order to increase the efficiency of 
the  process,  it  should  also  exclude  in  a  first  step  irrelevant  projects  from  further 
evaluation. 

•  Contracting phase  

DG JUST should ensure that the budget review starts as soon as there is a provisional 
merit  list  so  that  recommendations  for  any  adjustments  that  might  be  needed  can  be 
included  in  the  evaluation  report.  The  DG  should  also  ensure  a  consistent  approach 
between project officers for the budget review. 

5.5.  

IAS review on mapping of EC refugee crisis interventions 

The  mapping  exercise  was  included  in  the  2016  audit  plan  with  the  aim  to  gather 
sufficient, useful and relevant information to support and better focus the future IAS 
audit engagements on the key risks in the area. 

6.  ECONOMIC AND FINANCIAL AFFAIRS 

6.1.   Audit on effectiveness of the management of the COSME Programme by 
EASME 

Audit objective and scope 

The  overall  objective  of  the  audit  was  to  assess  the  effectiveness  of  EASME's 
management  and  control  systems  in  managing  the  delegated  parts  of  the  COSME 
programme.  

In  particular,  the  audit  assessed  the  effectiveness  of  the  division  of  roles  and 
responsibilities  between  EASME  and  DG  GROW  and  of  the  processes  in  place  in 
EASME to set the operational objectives and performance indicators in the context of 
the implementation of the COSME actions and to report to the parent DG. The scope 
also  included  the  adequacy  of  the  internal  control  system  to  manage  the  delegated 
COSME actions.  

The audit covered the period 2014-2016. 

There are no observations/reservations in the Agency's 2015 Annual Activity Report 
that relate to the area/process audited. 

The  fieldwork  was  finalised  on  15  October  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The IAS identified one very important issue: 

•  Cooperation between EASME and its parent DG for implementing COSME) 

The  Agency  has  had  most  of  the  time  no  robust  basis  for  preparing  the  COSME 
related part of its Annual Work Programme (AWP) and planning its work due to the 
late contribution by the parent DG and the significant changes in the COSME  Work 
75 

 
 
Programme  (WP)  during  the  mid-term  review.  This  had  resulted  in  certain  COSME 
related parts of the AWP already implemented during the first half of the year having 
become obsolete after the mid-year update of the COSME WP. Therefore, the related 
work  performed  until  this  modification  was  finally  wasted.  EASME  has  not 
sufficiently  assessed  the  impact  of  this  on  the  efficient  implementation  of  the 
delegated actions and has not established an up-to-date planning document that takes 
into account all the changes to the delegated actions during the year. 

Recommendations 

To address this issue, the IAS formulated the following recommendation: 

•  Co-operation  between  EASME  and  its  parent  DG  for  the  implementation  of 

COSME   

EASME should formally assess the impact of DG GROW's delays and of the changes 
to the WP, and identify possible measures to improve the cooperation with its parent 
DG, including a revision of the Memorandum of Understanding. For future COSME 
WPs,  EASME  should  formally  agree  with  DG  GROW  that  the  list  of  the  delegated 
actions  and  support  measures  is  provided  sufficiently  early  to  allow  for  preparing  a 
robust  AWP.  The  Agency  should  also  revise  its  Department  A  WP  to  take  into 
account any significant changes to the COSME WP. 

6.2.   Audit on financial management, procurement and grant processes in DG ECFIN  

Audit objectives and scope 

The audit assessed the adequacy of DG ECFIN's management of grants, procurement 
and  the  related  financial  transactions.  In  particular,  it  reviewed  the  design  and  the 
implementation of the controls in place to assess whether they ensure the legality and 
regularity of the financial procedures and the financial transactions and whether they 
are effective and efficient. 

This audit covered the key controls carried out on procurement and grant procedures 
completed  in  2015  and  on  financial  transactions  executed  in  2015.  The  engagement 
covered  the  controls  carried  out  on  the  financial  transactions  directly  and  entirely 
executed  by  DG  ECFIN  as  Authorising  Officer  by  Sub-Delegation  (AOSD),  i.e. 
excluding cross-delegations and sub-delegations.  

There  are  no  observations  or  reservations  in  DG  ECFIN's  2015  Annual  Activity 
Report related to the audited processes. 

The fieldwork was finalised on 31 May 2016. All observations and recommendations 
relate to the situation as of that date. 

Major audit findings  

The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

76 

 
 
6.3.   Audit on setting of objectives and measurement of performance in DG GROW 

Audit objectives and scope 

The  overall  objective  of  the  audit  engagement  was  to  assess  whether  the  DG  has  an 
adequate performance measurement framework in place for its day-to-day operational 
and  administrative  activities  (internal)  and  for  the  delivery  of  its  policy  objectives 
(external).  The  audit  assessed  the  internal  processes  for  setting  objectives  and  key 
performance indicators as well as the related reporting and monitoring. 

The audit focused in particular on the following areas: 

• The process of setting high quality objectives and performance indicators (design and 

implementation of the process) in line with the policy; 

• The  performance  measurement  framework  for  monitoring,  evaluating and reporting 

the (internal and external) performance of activities. 

The audit covered the processes related to the preparation of the Strategic Plan (2016-
2020),  the  Management  Plans  (2014,  2015  and  2016),  DG  ENTR  Annual  Activity 
Report (2014), DG GROW Annual Activity Report (2015) and Programme Statements 
(Draft Budget 2016 and 2017). 

There are no observations/reservations in the 2015 Annual Activity Report (AAR) of 
DG GROW that relate to the area/process audited. 

The fieldwork was finalised on 13 April 2016. All observations and recommendations 
relate to the situation as of that date. 

Major audit findings 

The IAS identified two very important issues: 

•  DG GROW performance framework  

The different tools DG GROW currently uses to plan and monitor its activities, actions 
and  initiatives  are  not  complemented  by  an  overarching  strategic  vision  describing 
how  the  DG  organises  its  interventions  and  how  short-term  outputs  will  lead  to 
medium  and  long-term  results  and  impacts  and  contribute  to  the  achievement  of  its 
strategic objectives. Consequently, there is no overview which demonstrates how the 
different DG's activities contribute to the achievement of its strategic and operational 
objectives  without  gaps  or  overlaps.  Furthermore,  the  different  performance 
management  tools  in  place  in  DG  GROW  are  insufficiently  coordinated  at  the 
planning  and  reporting  phase  and  their  respective  contents  are  not  aligned,  coherent 
and consistent. 

•  Monitoring of and reporting on performance in the context of the SPP cycle 

The DG's 2016-2020 Strategic Plan does not always provide sufficient information to 
understand  which  unit  is  in  charge  of  monitoring  the  different  indicators  and  which 
data  sources  will  be  used  for  this  purpose.  In  addition,  there  is  neither  a  formalised 
procedure  nor  internal  instruction/guidance  available  on  the  monitoring  of  the  result 
indicators included in the Strategic Plan and for reporting on progress made towards 
the achievement of the established targets. As regards reporting, IAS noted some cases 

77 

 
 
 
where different Strategic Planning and Programming documents provided inconsistent 
information about indicators. 

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  DG GROW performance framework   

DG  GROW  should  clearly  set  out  its  strategic  view  by  establishing  a  logical  link 
(intervention logic) between high level priorities, strategic and operational objectives 
and  short  term  actions  as  established  in  its  different  strategy  documents  and 
performance  tools  (Strategic  Plan,  Management  Plan,  Annual  Activity  Report, 
Programme  Statements,  Agenda  planning,  different tools at unit level). The strategic 
view should allow it to assess whether or not the actions planned for a given year will 
contribute  to  the  achievement  of  its  specific  objectives  and  of  the  Commission’s 
priorities.  

•  Monitoring of and reporting on performance in the context of the SPP cycle   

DG GROW should adopt a procedure for the measurement and monitoring of all result 
indicators included in the Strategic Plan and Programme Statements. In addition, the 
DG  should  document,  for  each  result  indicator,  key  information  such  as  the  data 
source, calculation method, person responsible for the calculation and the monitoring 
of  the  indicator  and  the  periodicity  for  the  reporting.  Furthermore,  the  DG  should 
perform  and  document  consistency  checks  among  the  indicators  included  in  the 
Management Plan, Annual Activity Report and Programme Statement. 

6.4.   Audit on financial management and IT procurement in DG TAXUD 

Audit objectives and scope 

The  overall  objective  of  this  audit  was  to  assess  the  adequacy  of  the  design  and  the 
effective  implementation  of  DG  TAXUD's  internal  control  systems  as  regards  its  IT 
procurement, contract and financial management processes as well as the effectiveness 
and efficiency of the related financial circuits.  

The  scope  included  the  2015  and  2016  IT  procurement  procedures,  framework 
contracts,  specific  contracts  and  requests  for  actions  as  well  as  the  related  financial 
transactions performed. 

There  are  no  observations  or  reservations  in  DG  TAXUD's  2015  Annual  Activity 
Report related to the audited processes. 

The  fieldwork  was  finalised  on  28  October  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

78 

 
 
6.5.   Audit on ethics in DG TRADE 

Audit objectives and scope 

The  overall  objective  of  the  audit  engagement  was  to  address  the  following  key 
question: 

Has  DG  TRADE  adequately  designed  and  effectively  implemented  an  ethics 
framework in compliance with the applicable values and rules to ensure that it serves 
the  EU  interest,  complies  with  ethics  standards  and  ensures  that  its  staff  behave 
ethically? 

The audit covered in particular: 

• The overall control environment for ethics  in DG TRADE (risk management, roles 
and responsibilities, compliance with Commission's rules and guidance, provision of 
specific guidance, support, training and awareness raising actions, reporting); 

• The compliance of DG TRADE's activities and its staff behaviour with ethics rules 

and standards. 

The audit did not cover the handling of sensitive information as it was included in the 
2013 audit of the former IAC on document management and in the IAS 2015 audit on 
the  efficiency  of  the  Trade  Defence  Instruments.  In  addition,  the  security  and 
confidentiality of information related to trade negotiations is part of the scope of the 
ongoing IAS audit on the administrative processes supporting trade policy negotiations 
and implementation. 

There  are  no  observations  or  reservations  in  DG  TRADE's  2015  Annual  Activity 
Report related to the audited process. 

The  fieldwork  was  finalised  on  15  September  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

7.  GENERAL SERVICES  

7.1.   Audit on management of procurement under DG ESTAT's operational budget 

Audit objectives and scope 

The  overall  objective  of  the  audit  was  to  assess  the  adequacy  of  the  design  and  the 
effective implementation of DG ESTAT's internal control systems for the management 
of the procurement process and the effectiveness and efficiency of the related financial 
circuits. 

In  particular,  it  reviewed  whether  the  internal  control  system  provides  reasonable 
assurance regarding the: 

79 

 
 
 
• Compliance  with  the  Financial  Regulation,  Rules  of  Application  and  specific 

legislation; 

• Prevention, detection and correction of errors, irregularities and fraud; 
• Effectiveness and efficiency of the procurement process and the need to simplify the 

internal administrative rules and procedures; 

• Reliability of reporting and monitoring; 
• Safeguarding of assets. 

This audit covered the key controls concerning: 

• The  procurement  process,  from  the  determination  of  the  needs  and  planning  to  the 

effective implementation of the contract, and;  

• The  financial  circuits  for  procurement,  including  commitments,  payments,  de-

commitments and recovery orders. 

The  audit  covered  procurement  procedures  launched  and  financial  transactions 
performed during 2015 and up to 31 July 2016. 

There  are  no  observations  or  reservations  in  DG  ESTAT's  2015  Annual  Activity 
Report related to the audited processes. 

The  fieldwork  was  finalised  on  14  November  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

7.2.   Audit on procurement and grants in OLAF 

Audit objectives and scope 

The audit assessed the adequacy of OLAF's management of grants, procurement and 
the  related  financial  transactions.  In  particular,  it  reviewed  the  design  and 
implementation of the controls in place to assess whether they ensure the legality and 
regularity of the financial procedures and transactions and whether they are effective 
and efficient. 

The  audit  covered  the  controls  carried  out  on  procurement  and  grants  procedures 
completed in 2015 and on the financial transactions executed in 2015. The audit did 
not cover procurement procedures under the budget line of the Supervisory Committee 
of OLAF, which is subject to a separate audit in 2016. 

There  are  no  observations  or  reservations  in  OLAF’s  2015  Annual  Activity  Report 
that relate to the audited process. 

The fieldwork was finalised on 23 May 2016. All observations and recommendations 
relate to the situation as of that date. 

Major audit findings 

80 

 
 
 
 
The  IAS  did  not  identify  any  material  risks  that  would  give  rise  to  critical  or  very 
important recommendations. 

7.3.   Audit on the governance, planning, monitoring and implementation of the budget 
line of the OLAF Supervisory Committee 

This  audit  has  been  classified  as  EU  restricted  and  as  such  was  disseminated  to  the 
interested parties in paper form only. 

7.4.   Audit on the charge-back process in PMO 

Audit objectives and scope 

The overall objective of the audit was to assess the effectiveness and efficiency of the 
design  and  implementation  of  the  charge-back  process  in  place  in  PMO  for  the 
services  provided  to  the  Commission's  internal  and  external  clients  (e.g.  other  EU 
Institutions, Agencies and Bodies) and its compliance with the fundamental principles 
laid down in the corporate guidance. 

The audit scope covered the roles and responsibilities of PMO related to the charge-
back  process  for  the  services  provided  to  the  Commission's  internal  and  external 
clients. It included all types of services provided by the Office that are subject to the 
charge-back of costs and the mechanisms used for charging-back costs to clients (i.e. 
recovery orders and delegations for specific budget lines). 

There  are  no  observations/reservations  in  PMO's  2015  Annual  Activity  Report  that 
relate to the area/process audited. 

The  fieldwork  was  finalised  on  29  November  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The IAS identified two very important issues: 

•  Service Level Agreements 

PMO  has  not  yet  updated  all  the  Service  Level  Agreements  (SLA)  with  its  clients 
signed  before  2014,  even  though  some  of  these  date  back  some  ten  years  and  no 
longer  reflect  the  actual  workload  and  costs  incurred  by  PMO.  Consequently,  some 
clients pay more and others less than the actual cost incurred by PMO.  

Moreover,  the  cost  of  certain  services  provided  to  EU  Institutions  and  other  EU 
Bodies is not calculated by using the new cost methodology, resulting in clients being 
charged different prices for similar services. 

Furthermore, PMO does not have a dedicated section on its website, or other readily 
available information that would allow (potential new) clients to understand precisely 
what services can be provided (its catalogue of services) and at what price. In addition, 
other  elements  which  make  up  the  charge-back  mechanism,  such  as  the  grouping  of 

81 

 
 
services into categories, the methodology and criteria used to calculate and to revise 
prices are not communicated to its clients.  

Finally,  although  certain  SLAs  include  provisions  on  the  evaluation  of  the  PMO 
services,  so  far  PMO  has  not  monitored  all  and  not  reported  on  any  of  the  Key 
Performance Indicators as stipulated in the SLAs to its clients. In other cases, no such 
provisions are included in the SLAs. 

•  Cost methodology 

The IAS found that PMO's current cost methodology is likely to have overestimated 
the cost of the services provided, as its overhead cost was accounted for twice in the 
calculation for the basic services offered for one year. 

In addition, the SLAs with the Agencies, as revised in 2015, include a clause granting 
a discount and this progressively decreases over a period of ten years. However, this 
discount is not related to any analysis of the associated workload of PMO.  

Finally, at present, PMO's cost methodology for charging back the cost of its services 
is  not  documented  in  a  comprehensive  manner.  Moreover,  the  knowledge  regarding 
the actual application of the cost methodology is currently limited to a few members 
of staff following the departure of some key staff involved in the cost calculation. 

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  Service Level Agreements 

PMO should initiate a revision of the SLAs signed with internal and external clients, 
which do not comply with the corporate guidelines on charge-back due to be finalised 
soon and, in particular, with the new cost methodology. Particular attention should be 
paid to those SLAs that no longer reflect the real costs incurred by the Office for the 
provision of the respective services; 

•  Cost methodology 

PMO  should  make  available  to  (potential)  clients  its  catalogue  of  services,  together 
with information on how the charged-back mechanism is applied and the details of the 
costing methodology used; it should incorporate in its SLAs harmonised provisions on 
monitoring and reporting to clients on the quantity and quality of the services provided 
(e.g. by means of relevant Key Performance Indicators with targets). 

8.  IT audits 

8.1.   Audit on effectiveness of measures to handle manual interventions in ABAC 

Audit objectives and scope 

The  overall  objective  of  the  audit  was  to  provide  re-assurance  on  the  controls  over 
Manual Interventions (MIs), specifically by reviewing and assessing the effectiveness 
of  DG  Budget's  processes  and  procedures  in  the  management  of  MIs  in  ABAC 
Accounting  (ABAC-ACC)  and  ABAC  Workflow  (ABAC-WF).  This  audit  aimed  to 
82 

 
 
 
complement  the  previous  work  of  the  European  Court  of  Auditors  by  testing  the 
implementation  of  the  new  procedure  for  MIs  and  performing  a  more  detailed 
substantive transaction testing. 

The audit focused on the following aspects: 

• The ABAC-ACC and ABAC-WF systems. These are the two main central financial 
information  systems  dealing  with  and  consolidating  information  on  payments, 
commitments, recovery orders, invoices, etc. and therefore carry a higher risk related 
to MIs; 

• The process for requesting and approving privileged user accounts with the necessary 

authorisations to perform MIs; 

• The process for requesting, performing and documenting MIs in production systems; 
• The process for reviewing the actual usage of privileged user accounts; 
• The  process  for  detecting  and  analysing  recurrent  MIs  and  for  the  identification  of 

measures aimed at reducing their frequency. 

The analysis of MIs was limited to DG Budget even though other DGs, such as DG 
ECFIN,  also  have  user  accounts  that  perform  MIs  in  ABAC-ACC.  This  audit  scope 
was  limited  because  the  main  control  mechanisms  within  the  scope  of  this  audit 
(privileged  user  creation,  monitoring  of  MIs  modifying  DG  Budget  data)  are 
applicable to all users.  

There  are  no  observations/reservations  in  the  2014  Annual  Activity  Report  of  DG 
Budget that relate to the area/process audited. 

fieldwork  was 

The 
recommendations relate to the situation as of that date. 

finalised  on  3  February  2016.  All  observations  and 

Major audit findings 

The IAS identified two very important issues: 

•  Extensive use of the Manual Intervention procedures  

MIs are extensively used to perform activities which could be performed according to 
more  appropriate  and  safe  arrangements,  such  as  standard  change  management 
procedures.  In  some  cases,  the  need  for  MIs  could  be  avoided  completely  as  the 
associated activities can be performed by users with much lower privilege rights such 
as a normal business user and not a privileged IT user. Furthermore, DG BUDG has 
produced  4  bi-annual  reports  on  the  use  of  MIs  since  2014,  and  while  there  were 
recommendations formulated in the reports, no formal action plans have been drawn 
up to address any issues identified. 

•  Too few controls over privileged user accounts  

User accounts used to perform MIs have extensive privileges, in some cases beyond 
the best practices recommended by the vendor of the software. Moreover, certain high 
privileged  user  accounts  are  not  linked  to  a  single  individual,  which  reduces 
traceability and accountability. In addition, there is no systematic and regular review 
of privileged user accounts, together with their access rights and insufficient resources 
are available for reviewing and controlling the MIs performed. 

83 

 
 
 
Recommendations 

To address these issues, the IAS formulated the following recommendations: 

DG  Budget  should  reduce  the  use  of  MIs  to  perform  changes  in  the  production 
environment  of  the  central  financial  IT  systems  by  identifying  activities  that  only 
require  limited  privileges  for  their  execution  and  by  performing  them  with  less 
privileged  users  as  well  as  by  implementing  specific  IT  developments  to  avoid  the 
need for MIs. In addition, the number of privileged accounts should be reduced to a 
strict minimum and their accountability and traceability enhanced. Moreover, the DG 
should  identify  activities  to  be  performed  by  business  users  and  implement  the 
functionalities  which  would  allow  them  to  be  performed  directly,  rather  than  having 
IT teams executing tasks on their behalf. This would improve the inherent security of 
the  operations,  facilitate  the  execution  of  detective  controls  and,  ultimately  would 
result in a more cost effective use of IT and user support resources. 

8.2.   Audit on management of EESSI project in DG EMPL 

Audit objectives and scope 

The overall objective of the audit was to review and assess the adequacy of the design 
and the effectiveness of the implementation of the internal controls put in place by DG 
EMPL for managing the Electronic Exchange of Social Security Information (EESSI) 
project,  with  a  specific  focus  on  its  execution  phase.  The  audit  aimed  to  identify 
weaknesses 
in  DG  EMPL's  processes  and  procedures  and  recommend  any 
improvements, where appropriate. 

The audit focussed on the following aspects: 

• The  Project  Plan,  including  past  performance  for  the  achieved  phases  and  future 

estimates for the remaining phases; 

• The adequacy of the project management artefacts, developed according to the PM2 

methodology; 

• The definition of the functional and technical specifications; 
• The process to implement, test and validate the implementation of the specifications; 
• The process for involving and receiving the validation of artefacts by DG EMPL and 

Member States stakeholders; 

• The accuracy of project reporting. 

There are no observations/reservations in the Annual Activity Report that relate to the 
area/process audited. 

The fieldwork was finalised on 31 March 2016. All observations and 
recommendations relate to the situation as of that date. 

Major audit findings 

The IAS identified three very important issues: 

84 

 
 
 
•  Incomplete  and  unstructured  procedures  for  final  acceptance  testing  and 

preparation for production release readiness  

Currently,  there  is  not  yet  a  detailed  and  comprehensive  agreed  list  of  acceptance 
criteria  for  each  feature  or  functionality.  The  test  procedures  (objectives,  test 
specifications,  ownership,  definitions  as  to  what  is  acceptable/not  acceptable  by  all 
stakeholders etc.) and the reporting of the results needed for the final project sign-off 
are still very much at the preliminary stage. The project is now in its final phase and 
the  specifications  tasks,  roles  and  responsibilities  need  to  be  defined  urgently. 
Furthermore,  Member  States  need  to  be  well-prepared  for  the  transition  period,  but 
this  is  currently  not  sufficiently  monitored  by  DG  EMPL.  Similar  weaknesses  in 
intermediary testing phases, among other reasons, have already resulted in a delay of 6 
months  in  the  project  timeline,  postponing  the  final  release  to  June  2017  instead  of 
December 2016. 

•  Insufficient integration of security requirements  

The  necessary  security  requirements  and  specifications  (such  as  approved  EESSI 
security  policies,  standards  and  guidelines  and  appropriate  business 
impact 
assessments), have not been fully built into the project architecture. Similarly, the IT 
Security Plan, that has to be finalised before production, is still in its very early stage 
of  development.  Although  this  is  already  part  of  on-going  discussion  in  the  EESSI 
Security  Expert  Forum,  its  finalisation  has  been  planned  for  the  last  phase  of  the 
development  and  it  may  prove  difficult  and  very  expensive  (in  terms  of  time  and/or 
skilled  resources)  to  achieve  in  practice.  Additionally,  the  security  tests  –  needed  to 
validate the actual implementation against the relevant requirements – as well as the 
IT security acceptance criteria and the strategy for what to do in the event of a failure 
to  meet  these  criteria  have  not  yet  been  defined  and  agreed  internally  (within  the 
Commission) and/or externally (with Member States). 

•  Gaps in the Project Plan update and limited reporting  

The Project Plan is currently incomplete as it does not define or integrate the transition 
tasks  and  ownership  required  to  test  and  validate  the  final  release  by  the  relevant 
stakeholders. Neither are there any provisions for monitoring the preparedness of the 
Member States. In addition, there is no process in place for reporting progress against 
agreed baselines. This is essential as the project is entering the critical finalisation and 
validation  phases.  Given  the  limited  time  available  to  undertake  final  testing, 
corrections  and  validation,  this  will  be  particularly  challenging  in  view  of  the  large 
number of stakeholders involved. 

In addition, the Total Cost of Ownership (TCO) of the project has not yet been fully 
estimated. In particular, there is no estimate of all IT investments and costs, internal or 
external  to  be  incurred  by  DG  EMPL,  foreseen  for  the  design,  construction  and 
operation  phases  for  the  first  five  years  (including  development,  deployment, 
maintenance, support, training and infrastructure, hosting and licences). 

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  Procedures for final acceptance testing and preparation for production release 

readiness  

85 

 
 
DG  EMPL  should  complete  and  finalise  quickly  the  necessary  elements  for  the 
acceptance  tasks  such  as  the  Deliverables  Acceptance  Management  Plan,  the 
traceability matrix and the transition plan. This should include the testing procedures 
and acceptance criteria for each solution specification, as well as all requirements and 
milestones needed by Member States to start the transition period. It is of paramount 
importance to properly identify ownership, assign responsibilities and set due dates for 
both internal and external stakeholders responsible for testing and acceptance. 

•  Integration of security requirements  

DG EMPL should ensure that the architectural specifications are finalised as soon as 
possible,  together  with  the  IT  Security  Plan.  In  parallel,  DG  EMPL  should  define  a 
timeline with tasks and ownership for security related tests to be performed in the last 
development  and  testing  phase.  It  should  allocate  sufficient  time  for  these  tests, 
including any incremental changes planned for the final production release. Finally, it 
should agree with all stakeholders the strategy and approach to take in the event that 
the security tests are unsuccessful. 

•  Project Plan update  

DG EMPL should identify missing tasks in the Project Plan for all key stakeholders, 
together  with  any  related  inter-dependencies,  and  update  and  complete  the  existing 
Project Plan accordingly. Specifically, the transition plan should be integrated into the 
overall  Project  Plan,  including  tasks,  owners  and  due  dates  for  activities  within  the 
transition workstream. DG EMPL should update the baseline project plan and report 
updates  and  deviations  for  the  remaining  phases.  Finally,  it  should  make  a  first 
estimate  of  the  TCO  for  DG  EMPL  for  the  transition  period  and  the  first  years  of 
solution in full production mode, as recommended by the IT Board instructions for the 
calculation of the TCO. 

8.3.   Audit on business continuity management at OP 

Audit objectives and scope 

The objective of the audit was to assess the adequacy of the design and the efficiency 
and  effectiveness  of  the  management  and  control  systems  put  in  place  by  the 
Publications  Office  (OP)  for  its  Business  Continuity  (BC)  management.  The  aim  of 
the  audit  was  to  help  identify  any  possible  weaknesses  in  OP's  business  continuity 
processes and to recommend improvements where needed. 

The audit focussed on the following aspects: 

• Completeness, relevance and consistency of OP's BC management documentation; 
• Effectiveness and consistency of contracts and service level agreements with non-OP 

service providers (including DG DIGIT); 

• Adequacy and effectiveness of OP's defined response plan to a major disruption; 
• Maturity of OP's BC awareness culture; 
• Adequacy and effectiveness of OP's technical arrangements including the testing of 

correct functioning. 

86 

 
 
 
The scope of the audit was limited to BC management at the DG/Service level in OP. 
BC arrangements at corporate level were not in the scope of this audit as they are SG's 
responsibility. 

There are no observations/reservations in the 2015 Annual Activity Repot of OP that 
relate to the area/process audited. 

The fieldwork was finalised on 20 May 2016. All observations and recommendations 
relate to the situation as of that date. 

Major audit findings 

The IAS identified three very important issues: 

•  Shortcomings of physical security in the alternate data centre 

The two data centres mirroring data for critical applications in real-time are a corner 
stone  of  OP's  BC  strategy.  However,  the  actual  power  density  in  the  alternate  data 
centre has significantly exceeded the levels assumed during the planning phase for a 
number  of  years,  a  problem  which  is  well  known  to  OP.  In  addition,  the  audit 
identified  shortcomings  in  the  physical  security  of  the  alternate  data  centre  which 
increases the risk of a fire. 

The IAS notes that OP has already initiated an action plan to address the weaknesses 
concerning the physical security of the alternate data centre, but stresses the need to 
ensure that the actions are implemented as soon as possible. 

•  Recovery Time Objectives for urgent applications not met by DG DIGIT  

Key  OP  business  processes  depend  very  much  on  services  provided  by  DG  DIGIT, 
which  are  subject  to  a  formal  Service  Level  Agreement  (SLA)  and  which  states  the 
recovery requirements in the event of a disruption. Even though DG DIGIT classifies 
key systems in OP as critical, the recovery times stated in the current SLA with DG 
DIGIT are significantly higher than the Recovery Time Objectives (RTO) which OP 
itself has defined for these very urgent key business processes. Consequently, there are 
no  formal  assurances  from  DG  DIGIT  that  it  would  be  able  to  meet  OP’s  recovery 
requirements in the event of a disruption.  

•  Business  Impact  Assessment  delivering  an  incomplete  picture  and  misleading 

results  

OP's assessment of the relative urgency of a situation may be distorted in so far as its 
Business Impact Assessment (BIA) wrongly confuses IT security and BC criteria. In 
addition,  the  BIA  does  not  provide  a  central  (big  picture)  overview  of  the  various 
interdependencies  between  the  various  functions  and  the  corresponding  impact  on 
services of a disruption to one or more of those functions. Furthermore, the BIA does 
not sufficiently document the nature and extent to which OP is dependent on external 
service providers.  

Recommendations 

To address these issues, the IAS formulated the following recommendations: 

•  Physical security in the alternate data centre  

87 

 
 
OP should reduce the power density by enhancing the space or increasing the power of 
the  air  conditioning  system  in  the  alternate  data  centre.  In  addition,  OP  should 
introduce regular checks by staff to ensure that the fire load is kept to the minimum 
possible. 

•  Recovery Time Objectives for urgent applications  

OP  should  re-assess  its  BC  requirements,  taking  into  account  the  constraints  of  its 
service providers. If a solution cannot be readily found, it should formally include this 
risk in its risk register and explore the possibility of alternative service providers more 
in line with its requirements. 

•  BIA delivery 

OP  should  revise  its  BIA  by  introducing  a  process-oriented  approach  and  using 
availability  as  the  sole  criterion  for  the  assessment  of  the  urgency  of  the  process 
concerned.  In  addition,  OP  should  clearly  document  all  dependencies  and  the 
resources needed to recover critical processes. 

8.4.  

IT governance and portfolio management in DG GROW 

Audit objectives and scope 

The  overall  objective  of  the  audit  was  to  assess  the  adequacy  of  the  design  and  the 
effective implementation of the management and control systems put in place by DG 
GROW for its IT governance, portfolio management and related domains. The aim of 
the  audit  was  to  help  identify  any  weaknesses  in  DG  GROW's  processes  and 
procedures to deliver effective and efficient results. 

The scope of the audit included the review of the following aspects in DG GROW: 

• IT governance and portfolio management related principles, policies, processes and 

procedures; 

• IT service management related principles, policies, processes and procedures; 
• Resources  and  capabilities  in  the  domains  of  business  process  management, 
data/information management, enterprise architecture and programme management; 

• IT sourcing related principles, policies, processes and procedures; 
• IT-related skills and competences principles, policies, processes and procedures. 

There are no observations/reservations in the DG’s 2015 Annual Activity Report that 
relate to the area/process audited. 

The  fieldwork  was  finalised  on  30  November  2016.  All  observations  and 
recommendations relate to the situation as of that date. 

Major audit findings 

The IAS identified two very important issues: 

•  IT  strategy,  IT-related  risk  management  and  functioning  of  the  IT  Steering 

Committee  

88 

 
 
Currently, the DG's IT strategy goals and related specific objectives are not linked to 
appropriate  key  performance  indicators  (KPIs)  in  a  way  that  makes  it  possible  to 
monitor  progress  towards  their  achievement  and  measure  the  achieved  benefits.  In 
addition, 
the  DG's  IT 
environment/landscape is expected to change in response to meeting business needs, 
going  forward.  The  possible  centralisation  option,  which  was  discussed  in  a  recent 
meeting  of  the  IT  Steering  Committee  (ITSC),  was  not  supported  by  a  sound  cost-
benefits and risk analysis. 

lacks  a  clear  vision  as 

the  IT  strategy 

to  how 

Furthermore,  there  is  no  overall  IT  risk  management  framework  to  ensure  that  IT-
related business risks at all levels (strategic, project or programme and operational) are 
properly identified and assessed, together with the establishment and implementation 
of  appropriate  action  plans.  Finally,  although  the  results  of  the  ITSC  meetings  are 
made available, there have been no concerted efforts aimed at informing the Directors 
who are currently not members of the ITSC of the wider aspects and implications of 
the issues discussed.  

•  IT portfolio and programme management  

For  new  projects  at  the  inception  phase,  the  DG's  standard  assessment  methodology 
does not cover costs and benefits. In addition, although DG GROW is currently in the 
process  of  undertaking  numerous  IT-related  or  IT-enabled business  initiatives,  it  has 
not put in place a formalised IT programme management approach. Moreover, there is 
no  multi-annual  roadmap,  which  links  the  initiatives/actions  mentioned  in  its  IT 
strategy  to  the  expected  deliverables,  the  resource  effort  and  costs  involved, 
milestones and any inter-related dependencies.  

Recommendations 

To address these issues, the IAS formulated the following recommendations:  

•  IT strategy, IT-related risk management and functioning of the ITSC 

DG GROW should improve its IT strategy, more specifically by linking its objectives 
to  appropriate  KPIs  and  strengthening  its  cost-benefits-risk  analysis  to  support  the 
choice of the preferred option for "The future IT Delivery Model of DG GROW".  In 
addition, it should approve an action plan to deliver the preferred option of the future 
IT Delivery model, taking due account of the need to align business and IT. 

Moreover, DG GROW should adopt a comprehensive IT risk management framework 
that  includes  managing  risks  at  the  strategic,  programme  or  project  and  operational 
level  and  develop  an  IT  risk  register.  In  addition,  it  should  strengthen  the  way  in 
which it involves and communicates key IT developments to the Directors who are not 
members of the ITSC. 

•  IT portfolio and programme management 

DG GROW should ensure that a formalised approach is in place to support IT-related 
or  IT-enabled  business  initiatives  by  a  comprehensive  (covering  cost,  benefits  and 
risks)  assessment  of  their  value,  both  at  an  early  stage,  at  the  project  inception,  and 
monitored  throughout  their  lifecycle.  Moreover,  DG  GROW  should  adopt  a 
programme management approach for IT-related and IT-enabled initiatives and devise 
a multiannual roadmap linking the actions with deliverables, corresponding allocated 

89 

 
 
resources, costs and milestones, and ongoing business initiatives and dependencies on 
other strategic actions. 

8.5.   Audit on IT security in JRC ICT systems 

Audit objectives and scope 

The  overall  objective  of  the  audit  was  to  assess  the  adequacy  of  the  design  and  the 
effectiveness  of  the  implementation  of the internal controls put in place by  DG  JRC 
for  protecting  electronic  information  and  assets,  and  regarding  connectivity  between 
JRC premises and the wider European Commission information network and systems.  

The audit focused on the following aspects: 

• IT  Security  governance  procedures  to  evaluate,  design  and  monitor  the  IT  security 
framework  in  the  JRC.  This  included  assessing  the  effectiveness  of  the  decision 
making  process  among  the  different  stakeholders  and  the  regular  execution  of 
awareness campaigns; 

• Security operations as regards applications, operating systems and network security 
devices.  This  included  reviewing  the  controls  currently  executed  by  the  Local 
Information  Security  Officer  (LISO)  and  the  connectivity  to  the  Commission 
networks; 

• Process to request, grant and provide privileged user accounts; 
• Process to request, perform and document changes in the security parameters; 
• Detection, communication and analysis of IT security incidents and identification of 

improvements in systems and processes to reduce the frequency. 

Specific nuclear IT systems were not included in the scope of the current audit. These 
are subject to specific German and Italian regulations and under strict supervision by 
the authorities. 

There are no observations/reservations in DG JRC's 2015 Annual Activity Report that 
relate to the area/process audited. 

The  fieldwork took place at  the JRC sites  of Ispra, Seville and Karlsruhe and it was 
finalised on 28 November 2016. All observations and recommendations relate to the 
situation as of that date. 

Major audit findings 

The IAS identified four very important issues: 

•  Management oversight of IT security  

Currently,  a  number  of  key  activities  are  not  assigned  to  defined  owners  and  are 
performed on an ad-hoc only basis and/or in an uncoordinated way. These include the 
evaluation  of  security  needs  and  requirements,  prioritisation  of  tasks  and  monitoring 
of according actions. In addition, there is no proven mechanism for ensuring that IT 
security related needs are properly heard and discussed at the appropriate level and by 
the right stakeholders. Furthermore, there are no clear corporate objectives for the JRC 
in terms of IT security or related Key Performance Indicators (KPIs) to measure  the 

90 

 
 
 
performance of IT security actions and controls at management level. Finally, mission 
statements  for 
the  split  of 
responsibility for operational tasks, monitoring and risk management are not formally 
defined and communicated.  

the  different  IT  security  stakeholders 

including 

•  IT security considerations built into the design of new IT systems and into the 

maintenance of existing systems  

Although  the  definition  of  security  requirements  is  recommended  at  early  stages  of 
every IT project, both by Commission Decision C(2006)3602 concerning the security 
of information systems used by the European Commission and the PM2 methodology, 
this  is  currently  not  done  in  JRC.  Security  requirements  are  not  systematically 
included in the project definitions or system change lifecycle. The existing procedures 
do not provide for security requirements to be defined upfront in a new project or for 
maintenance changes to include an impact analysis for IT security. What happens in 
practice currently depends very much on the developer or administrator of the system 
and is done on an ad-hoc basis, rather than as a result of a planned approach.  

•  Deployment  of  security  reference  configurations  and  monitoring  of  new 

vulnerabilities  

Reviews  of  actual  vulnerabilities  (scans)  are  useful  as  they  can  detect  and  prioritise 
weaknesses.  However,  currently  IT  security  reviews  of  existing  systems  are  only 
performed on a regular basis for some of the systems in DG JRC. As of today, among 
the  audited  JRC  sites  only  the  site  in  Seville  is  performing  regular  vulnerability 
assessments,  while  for  the  remaining  sites,  only  the  detection  of  vendor  published 
corrections  (patches)  is  made  on  a  regular  basis  and  even  then,  only  for  a  subset  of 
corporate  systems.  In  addition,  there  are  no  regular  reviews  performed  of  current 
security settings and privilege access to systems, etc. to detect possible unauthorised 
changes.  

•  Inventory of JRC IT systems and their security dependencies  

The  inventories  of  systems  and  software  connected  to  the  JRC  networks  are 
fragmented  and  maintained  by  different  groups,  depending  on  the  area  of 
responsibility.  The  JRC  lacks  a  complete  overview  of  all  the  systems  in  terms  of 
security  risks.  In  addition,  because  many  scientists  stay  for  periods  of  only  three  to 
five  years,  system  owners  are  not  always  timely  updated  in  the  registries,  with  the 
result that certain systems do not have a valid system owner. 

Recommendations 

•  Management oversight of IT security  

DG JRC should set up an IT security steering committee with members representing 
all relevant stakeholders (JRC management, Scientific Units, Support Units, LISO and 
ICT  Architecture).  The  DG  should  also  establish  a  set  of  objectives  and  measuring 
criteria (KPIs) for the IT security domain, based on a clear definition of the mission 
statements for the different teams involved in IT Security, with a clear split of scope 
and responsibilities. 

•  IT security considerations built into the design of new IT systems and into the 

maintenance of existing systems  

91 

 
 
DG  JRC  should  define  a  JRC-wide  standard  by  enforcing  new  developments  and 
relevant  system  changes  to  undergo  a  security  analysis  in  the  early  phases  of  their 
development.  In  particular,  management  should  enforce  mechanisms  that  facilitate 
integrating  security  practices  while  coding.  Once  in  production,  DG  JRC  needs  to 
include a set of criteria to identify the most relevant systems and establish a policy to 
perform regular security reviews on them. Lastly, the JRC should leverage its central 
IT  teams  and  IT  security  experts  in  the  organisation  when  creating,  deploying  or 
changing  new  IT  systems  for  scientific  projects,  to  re-use  existing  security  good 
practices  and  align  with  known  secure  configurations  and  software  development 
guidelines. 

•  Deployment  of  security  reference  configurations  and  monitoring  of  new 

vulnerabilities  

DG  JRC  should  establish  an  automated  procedure  to  identify  current  versions  and 
patches in the whole organisation and, depending on the constraints of each machine 
for their operational use, enforce automated updates or isolation of appropriate areas 
of  the  network.  DG  JRC  should  generalise  the  use  of  automated  vulnerability 
discovery  tools  to  report  on  current  known  vulnerabilities  and  possible  course  of 
action,  depending  on  the  scientific  activity  and  constraints.  Ideally,  the  JRC  should 
seek  to  leverage  internal  existing  expertise  in  virtualised  environments  for  scientific 
use  or  in  reference  configurations  for  the  different  types  of  needs  in  the  scientific 
domains.  As  a  minimum,  it  should  instigate  an  awareness  campaign  on  the 
recommended  settings  (to  be  selected  by  the  scientific  team  as  necessary).  Finally, 
according  to  the  residual  risk  identified,  DG  JRC  should  put  in  place  the  necessary 
compensating  controls,  for  example  network  segregation,  dedicated  monitoring,  etc., 
as needed. 

•  Inventory of JRC IT systems and their security dependencies  

DG  JRC  should  assess  the  feasibility  of  integrating  all  IT  systems  and  devices 
connected  to  the  JRC  networks  in  an  inventory  capable  of  identifying  installed 
software and versions. Once in place, a process should be set up to maintain an up-to-
date list of running systems and their owners, adapted to the nature of the work in the 
JRC and which takes into account the high rotation levels for researchers. To facilitate 
this, the JRC should leverage and coordinate the existing work observed separately at 
the  audited  sites  of  Ispra,  Seville  and  Karlsruhe  as  regards  the  network  segregation 
projects that allow for a better control on vulnerable devices. 

92 

 
 
PART 2: FOLLOW-UP ENGAGEMENTS (SUMMARISED) 

1.  Follow-up audit on the design of DG AGRI's management and control 

system for greening 

Based  on  the  results  of  our  follow-up  audits  of  the  accepted  recommendations,  we 
assess that: 

• 
Recommendation  N°  2  on  Assessment  of  the  notifications  of  equivalent 
practices  (rated  very  important),  Sub-recommendations  N°  1.1  on  Correcting  of  the 
ISAMM  template  for  EFA  notification  (rated  important),  N°  1.4  on  Cross  checks 
between  ISAMM  forms    (rated  important),  N°  5.2  on  Providing  Member  States  with 
records  of  Expert  Group  meetings  (rated  important)  and  recommendation  N°  6  on 
Improving and clarifying the greening requirements (rated very important) have been 
appropriately implemented and can be closed; 

The  following  (sub-)recommendations  have  not  yet  been  fully  implemented 

• 
and cannot be closed:  

o 
Sub-recommendation N° 1.2 on Compatibility of the Good agricultural 
and  environmental  condition  (GAEC)  notifications  with  the  notifications  of 
greening  choices  for  Ecological  Focus  Areas  (EFAs)  and  equivalent 
practices  (rated  important):  the  IAS  considers  that  the  current  set  of 
procedure  does  not  ensure  the  check  of  the  compatibility  of  GAEC 
notifications  with  EFA  notifications.  This  check  should  either  be  performed 
as  part  of  the  GAEC  assessment  or  as  part  of  a  second  check  of  the  EFAs 
once GAEC notifications are available, and the relevant procedure should be 
updated accordingly; 
o 
Sub-recommendations  N°  1.3  and  1.5  on  Enhancing  reporting 
functionalities  in  ISAMM  and  exploring  possibilities  of  cross-validation  in 
ISAMM  and  automated  interface  with  GAEC  database  (rated  important): 
work  on  the  enhancement  of  the  reporting  functionalities  in  ISAMM  is  still 
ongoing; 
Recommendation N° 3 on the Assessment of greening notifications that 
o 
are  not  related  to  equivalent  practices  (rated  important)  is  assessed  as 
partially  implemented  as  the  procedures  for  the  assessment  of  forest 
exemption notifications (sub-recommendation 3.1), as well for the monitoring 
of  permanent  grassland  ratios  (sub-recommendation  3.2),  still  remain  to  be 
drafted;  
o 
Recommendation  N°4  on  Risk  of  double-funding  between  greening 
measures  and  rural  development  programmes  with  regard  to  agroforestry 
and  afforested  areas  (rated  very  important):  the  sub-measure  fiches  for 
"afforested  areas  and  woodlands"  and  "establishment  of  agro-forestry 
systems"  as  well  as  the  explanatory  document  on  how  to  avoid  double 
funding have been modified and uploaded on CircaBC for information to the 
Member States. However, the IAS considers that these updated guidelines are 
confusing  and  do  not  provide  practical  details  and/or  examples  on  how  to 
93 

 
 
 
 
 
exclude  double-funding  in  the  specific  cases  of  agro-forestry  and  afforested 
areas  selected  as  EFAs  under  the  greening  payment.  The  IAS,  nevertheless, 
acknowledges that given the low take-up of agroforestry and afforested areas 
as  EFAs,  observed  after  the  first  year  of  implementation  of  the  greening 
payment, the risk of double funding appears to be limited. The rating of the 
recommendation is therefore downgraded from very important to important; 
o 
Sub-recommendation  N°  5.1  on  Establishing  a  written  procedure  for 
replies  to  Member  States,  bilateral  meetings  with  and  missions  to  Member 
States (rated important): the procedure remains to be drafted.  

2.  Follow-up audit on payments suspensions and interruptions in the 2014-

2020 CAP framework 

Based on the results of our follow-up audit, we assess that: 

• 
Recommendations N° 1 on the Legal basis (rated very important) and N° 4 on 
the Suspension Board (rated important) have been adequately implemented and will be 
closed; 
• 
Recommendation  N°  2  on  the  Internal  guidance  and  procedures  (rated  very 
important)  has  been  partially  implemented.  However,  in  the  light  of  the  overall 
progress made, the level of risk is assessed as lower and the recommendation has been 
downgraded  to  important.  The  following  sub-recommendations  remain  only  partially 
implemented: 

o 
Sub-recommendation  N°  2.a  on  The  application  of  Articles  41(1)  and 
41(2)  of  Regulation  1306/2013  as  further  clarifications  are  still  necessary, 
notably  on  the  triggering  conditions  for  applying  Article  41(2)(b)  for  the 
second Pillar of the CAP (hereafter Pillar 2). However, the IAS acknowledges 
that  the  envisaged  implementation  date  for  this  sub-recommendation  is  30 
June 2017; 
o 
for  proposing 
interruptions and suspensions/reductions including a de-minimis approach as 
regards applying the de-minimis approach also for the first Pillar of the CAP. 

Sub-recommendation  N°  2.c  on 

the  Criteria 

• 
Recommendation N° 3 on the Application of guidance and procedures (rated 
very important) has been partially implemented. The following sub-recommendations 
have been only partially implemented: 

Sub-recommendation  N°  3.a  on  Applying 

o 
the  "stop-the-clock" 
instructions  in  practice  and  compliance  with  the  45  days  payment  deadline, 
for which the IAS acknowledges that the instructions have been adapted but 
nonetheless needs a reasonable number of cases to have occurred to be able to 
judge whether the instructions have been effectively applied or not; 
o 
Sub-recommendation  N°  3.b  on  Clarifying  the  criteria  for  requesting 
action plans in the context of Article 41(2) as regards Pillar 2 and the link to 
corrective  action  plans  following  the  reservations  in  the  Annual  Activity 
Report, where further clarifications are still necessary; 

94 

 
 
 
 
o 
Sub-recommendation  N°  3.c  on  Ensuring a more consistent  approach 
to letters requesting action plans since the drafting of the letters is still under 
discussion; 
o 
Sub-recommendation  N°  3.d  on  Minimising  the  time  for  the  overall 
process  from  requesting  an  action  plan  to  the  Member  States  to  taking  the 
final  suspension  decision,  on  which  work  has  been  done  but  requires 
additional clarifications/consistency checks; 
o 
Sub-recommendation N° 3.e on Putting more emphasis on ensuring the 
timeliness  of  the  interruptions,  suspensions  and  reductions  procedures  as 
some  decisions  have  been  taken  and  work  is  in  progress,  but  the  IAS  will 
need  to  check  how  DG  AGRI  is  applying  this  concretely  during  a  second 
follow-up audit; 
o 
Sub-recommendation N° 3.f on Evaluating whether or not the approach 
has been effective in achieving the objectives set in the discharge procedure 
remains open as the deadline for implementation is at the end of 2018. 

3.  Follow-up audit on the management of the approval process of the 2014-

2020 Rural Development Programmes (RDPs) 

Based  on  the  results  of  our  follow-up  audits  of  the  accepted  recommendations,  we 
assess that: 

• 
Sub-recommendations N° 1.1 on Adjusting the process on the basis of 
the  experience  gained  (rated  very  important),  N°  1.2  on  Planning  and 
monitoring of the approval process in RDIS2 (rated very important), N° 1.3 
on Workload analysis and task allocation optimisation (rated very important), 
N°  2.1  on  Updating  the  master  checklist  with  checks  related  to  financial 
aspects  (rated  "Important"),  N°  2.2  on  Completeness  of  information  on 
transitional arrangements for RDPs to be approved (rated important), N° 2.3 
on  Documenting  the  RDP  approval  process  (rated  important),  N°  2.4  on 
Identifying  and  following  up  outstanding  points  on  adopted  RDPs  (rated 
important), N° 3.2 on Coordinating the follow-up of Ex-Ante conditionalities 
action  plans  with  other  ESIF  DGs  (rated  very  important),  N°  3.3  on 
Providing  support  to  the  Member  States  on  performance  framework  and 
indicators (rated very important), N° 3.4 on Back-up for experts on indicators 
(rated  very  important),  N°  4.1  on  Improving  the  structure  of  SharePoint 
collaborative  platform  (rated  important),  N°  4.2  on  Sharing  technical 
clarifications with Member States (rated important),  N°  4.3 on Mapping the 
expertise gained during the approval phase (rated important) and N° 4.4 on 
Expanding  the  role  of  the  Consistency  Board  to  amendment  process  (rated 
important) have been appropriately implemented and will be closed; 
• 
Sub-recommendation  N°  2.5  on  Correcting  inconsistencies  between 
RDPs  and  Partnership  Agreements  (rated  important)  has  been  partially 
implemented. However, the residual risk has been assessed as low; 
• 
Sub-recommendation  N°  3.1  on  Appropriate  assessment  of  ExAnte 
Conditionalities  (ExACs)  (rated  very  important)  is  considered  obsolete  and 
the remaining residual risk lies with the monitoring of the implementation of 

95 

 
 
the  programmes,  including  ExACs.  In  addition,  an  IAS  audit  on  the 
monitoring of RDPs is included in the IAS draft Plan for 2017; 
• 
Recommendation  N°  5  remains  open  for  the  following  reasons:  Sub-
recommendation  N° 5.1 on  Providing clear guidance to Member States  and 
identifying  legal  inconsistencies  (rated  very  important)  has  been  partially 
implemented. However, the residual risk has been assessed as low. For sub- 
recommendation  N°  5.2  on  Monitoring  the  implementation  of  RDPs  to 
prevent  double  funding  (rated  very  important)  three  of  the  four  planned 
actions have been completed. The outstanding action concerns double funding 
in  relation  to  the  carry-over  of  agro-environmental  commitments  signed 
before  2012.  For  sub-recommendation  N°  5.3  on  Reviewing  overlap  in 
coverage  of  the  two  pillars  of  the  CAP  in  the  long  term  (rated  very 
important),  the  actions  are  on-going  as  part  of  the  work  paving  the  way  for 
the adoption of a Communication on the modernisation and simplification of 
the  CAP  in  the  second  part  of  2017,  as  indicated  in  the  Commission  work 
Programme  for  2017.  While  sub-recommendation  N°  5.1  is  considered 
partially  implemented  but  the  related  residual  risk  is  now  assessed  as  low, 
sub-recommendations  N°  5.2  and  5.3  remain  open.  However,  as  significant 
progress  has  been  made  in  mitigating  the  related  risks,  the  IAS  downgrades 
the rating of recommendation N° 5 from very important to important. 

4.  Follow-up audit on gap analysis of new legislation/design of 2014-2020 

programming period of European Structural and Investment Funds Phase 
2 in DG MARE 

Based on the results of our follow-up audit, we assess that recommendations N° 2: OP 
negotiation and adoption process (rated very important) and N° 3 Results orientation 
and  performance  framework  have  been  adequately  and  effectively  implemented  and 
will be closed. 

5.  Follow-up audit of IAC recommendations in DG SANTE 

Follow-up  of  IAC  audit  on  the  management  of  funds  in  DG  SANTE  veterinary 
programmes 

forecast 

for  program's  adoption  (rated  very 

Based on the results of our follow-up audit, we assess that recommendation N° 1 on 
the  Financial 
important)  and 
recommendation  N°2  on  the  Reallocation  exercise  (rated  very  important)  have  been 
partially  implemented.  In  the  light  of  the  overall  progress  made,  the  level  of  risk  is 
assessed as lower and the recommendations have been downgraded to important. The 
remaining open actions concern sub-recommendations 2.2 and 3.2 on the availability 
of  an  audit  trail  by  using  the  IT  system  Qlikview.  The  IAS  considers  that  the  IT 
system needs to be stable and fully used for the allocation and re-allocation exercises 
to ensure the existence of an audit trail. 

96 

 
 
6.  Follow-up audit of management and supervision of contracts for the 

outsourced IT services in DG SANTE 

Based  on  the  results  of  our  follow-up  audit,  the  IAS  assessed  that  recommendations 
N° 1 on Quality of tender documentation for DG SANCO's own framework contracts 
(rated very important) and N°4 on Follow-up of memoranda of understanding between 
DG  SANCO  and  DG  DIGIT  (rated  important)  have  been  adequately  and  effectively 
implemented and will be closed. 
One  remaining  recommendation  N°  2  on  DG  SANCO's  outsourcing  strategy  (rated 
important)  cannot  be  considered  as  implemented.  The  recommendation  required  the 
DG  to  carry  out  a  cost  benefit  analysis  of  the  various  outsourcing  options  for  IT 
projects (time and means, quoted time and means, etc.). While DG SANTE provided 
evidence of a comparative analysis of the insourcing and outsourcing options for the 
main IT project it is currently managing, the various possibilities for outsourcing have 
not  been  assessed  yet  as  this  would  be  too  early  at  this  stage  of  the  project.  As  a 
consequence, the recommendation will not be closed. 

7.  Follow-up audit on preparations for use of financial instruments under 

2014-2020 in DG EMPL 

Based  on  the  results  of  our  follow-up  audit,  we  assess  that  recommendation  N°  1 
Building  financial  instruments  related  capacity  (rated  very  important),  has  been 
adequately and effectively implemented. 

8.  Follow-up audit on preparations for use of financial instruments under 

2014-2020 in DG REGIO 

Based  on  the  results  of  our  follow-up  audit,  we  assess  that  recommendation  N°  2 
Building  financial  instruments  related  capacity  (rated  very  important)  has  been 
adequately and effectively implemented.  

9.  Follow-up audit on gap analysis of new legislation/design of 2014-2020 

programming period of European Structural and Investment Funds' (ESI 
Funds) Phase II 

Based on the results of our follow-up audit, we assess that recommendation N° 2 OP 
negotiation  and  adoption  process  (rated  very  important),  and  N°  4  IT  systems 
supporting  the  management  of  the  2014-2020  programming  period  processes  (rated 
very  important)  addressed  to  DG  REGIO  and  DG  EMPL  have  been  adequately  and 
effectively implemented, and will be closed. 
Recommendation  No  3  (rated  very  important)  concerns  the  performance  framework 
and the checks performed by the DGs on the information provided by Member States 
(MS)  in  order  to  ensure  consistency  and  plausibility  of  milestones  and  targets.  The 
IAS  recommendation  aimed  at  addressing  the  "inherent  risk  of  unambitious  target 
setting by the MS" not being sufficiently mitigated by the DGs' checks.  

97 

 
 
As a result of this follow-up, the IAS notes that the first part of the recommendation, 
relating  to  the  timely  request  of  art.4  information  from  MS,  has  been  adequately 
implemented by both DG REGIO and EMPL. The second part of the recommendation 
which  again  concerns  both  DGs  and  relates  to  plausibility  checks  performed  on 
milestones/targets  by  Desk  Officers  (DO)  of  geographical  units  has  not  been 
adequately implemented. The IAS acknowledges that the DGs have developed internal 
guidance for the DO to assess the plausibility of milestones and targets at the time of 
operational  programme  negotiation  and  adoption.  Furthermore,  the  observations  on 
the draft operational programme sent to MS included comments on targets/milestones 
raised  by  the  DO  and  the  evaluation  unit.  Nevertheless,  the  IAS  has  found  that  the 
actual  assessment  by  the  DO  of  the  plausibility  of  targets/milestones  at  the  time  of 
operational programme negotiation/adoption was generally not documented by either 
DG. 

The  specific  part  of  the  recommendation  addressed  to  DG  REGIO  on  ensuring  that 
"reviews  of  operational  programmes  performed  by  the  evaluation  unit  are  supported 
by  a  clear  audit  trail",  has  not  been  fully  implemented.  The  IAS  found  that  the 
documentation  in  WAVE  of  the  evaluation  unit's  comments  on  the  draft  operational 
programmes  was  not  complete  as  a  number  of  comments  were  made  outside  of  the 
system and not documented in WAVE. 

The  last  part  of  the  recommendation  concerns  the  need  for  consistency  checks  on 
indicators to be further developed by DG EMPL. The IAS notes that this part of the 
recommendation  was  adequately  implemented  after  the  operational  programme 
adoption  and  negotiation  process  was  completed.  DG  EMPL  has  put  in  place  the 
"EMPL  strategy  for  a  performance-based  culture  for  the  ESF"  which  sets  out  clear 
objectives  for  data  reliability,  and  assessment  of  the  consistency  and  plausibility  of 
reported indicators. Consistent with the action plan, the evaluation unit has developed 
a template which allows the comparison of targets through a common dimension (e.g. 
cost  per  participant),  and  it  is  used  for  checking  the  consistency  and  plausibility  of 
targets.  

From the IAS analysis it can be concluded that recommendation N° 3 was overall only 
partially implemented at the time of operational programme adoption and negotiation. 
However,  given  that  all  operational  programmes  have  been  adopted,  the  parts  of  the 
recommendation that have been assessed as not implemented are no longer pertinent 
in  the  context  of  operational  programme  adoption  and  the  recommendation  will  be 
closed.  Nevertheless,  the  IAS  will  be  reviewing  whether  the  related  risks  have  been 
mitigated  for  the  adoption  of  operational  programme  amendments  and  the  review  of 
annual implementation reports in the context of the following audits: 

Audit  on  amendment  of  2014-2020  operational  programmes  in  DGs  REGIO, 

• 
EMPL, and MARE; 
• 
operational programmes by DGs REGIO, EMPL and MARE (scheduled for 2018). 

Audit  on  monitoring  the  implementation  and  performance  of  2014-2020 

98 

 
 
 
10.  Follow-up audit on the governance and supervision of the nuclear 

decommissioning assistance programmes in DG ENER 

Based on the results of our follow-up audit, we assess that recommendations N° 1 on 
Assessment of ex-ante conditionalities (rated critical) and N° 2 on Control strategy of 
DG ENER (rated very important) have been adequately and effectively implemented 
and will be closed.  

11.  Follow-up audit on the supervision of the implementation of CEF in DG 
ENER 

Based  on  the  results  of  our  follow-up  audit,  we  assess  that  sub-recommendation  N° 
1.3  has  been  fully  implemented  while  the  other  sub-recommendations  (rated  very 
important)  still  require  further  actions  to  address  satisfactorily  the  issues  detected 
during the audit. In particular: 
• 
Sub-recommendation  N°  1.1:  DG  ENER  adopted  in  September  2016  a 
"Supervision strategy on PCIs development" which describes the need to develop such 
a  strategy.  It  states  that  "The  Fora  in  which  supervision  has  to  take  place  are  the 
Regional Groups". However, the strategy does not indicate how and if an agreement 
on  this  common  supervision  strategy  has  been  or  will  be  reached  in  the  Regional 
Groups. In addition, the document lists five objectives for the supervision strategy and 
it describes the tools that shall be used for that purpose. However it does not identify 
key performance indicators for measuring the performance of the supervision activity 
or the resources necessary to allow DG ENER to reach the objective; 
• 
Sub-recommendation  N°  1.2:  The  document  describes  the  differences  in  the 
various  reports  on  PCIs  implementation  which  have  to  be  prepared  but  it  does  not 
explain  how  DG  ENER  will  ensure  i)  that  Regional  Groups  exploit  them  efficiently 
and  ii)  that  issues  and  recommendations  made  in  the  various  reports  will  be 
systematically followed-up and in a timely manner; 
• 
Sub-recommendation  N°  1.4:  DG  ENER  has  not  completed  the  development 
of  a  reliable  comprehensive  tool  to  monitor  the  implementation  of  the  PCIs 
development. 

Consequently, the recommendation will not be closed. 

12.  Follow-up audit on the management and functioning of Euratom 

safeguards in DG ENER 

Based on the results of our follow-up audit, we assess that recommendation N° 4 on 
Operational  objectives  and  performance  indicators  (rated  important)  has  been 
adequately and effectively implemented and will be closed.  

Recommendation  N°  1  on  Assessment  of  the  Euratom  safeguards  approach  (rated 
important):  DG  ENER  performed  an  assessment  of  the  need  to  update  the  current 
Euratom safeguards approach document. However, the IAS found that this assessment 
did  not  contain  information  on  the  human  and  financial  resources  needed  by  DG 
ENER  to  effectively  implement  the  Euratom  safeguards  tasks,  and  a  time  frame  for 
subsequent  re-assessments.  These  two  elements  will  be  included  in  the  Commission 

99 

 
 
Communication and Staff Working Document (SWD) on the principles and modalities 
of  the  implementation  of  Euratom  safeguard  tasks  under  article  77  of  the  Euratom 
treaty. Consequently, the IAS considers that, until the adoption of the proposal for this 
Communication and SWD, the recommendation is not fully implemented and will not 
be closed. 

13.  Follow-up audit on procurement management in JRC 

Based on the results of our follow-up audit, we assess that recommendations N° 1 on the 
Management  of  low  value  procurement  (rated  very  important)  and  N°  4  on  the  Ex-post 
controls (rated important) have been adequately and effectively implemented and will be 
closed. 

14.  Follow-up audit on strategic planning and programming / activity based 

management in JRC 

Based on the results of our follow-up audit, we assess that: 
• 
Recommendation  N°  2  on  Work  programme,  project  management  and 
management  plan  (rated  important):  The  IAS  recommended  JRC  a)  to  introduce 
results  indicators  in  the  management  plan  to  measure  and  then  report  on  the 
performance in implementing the work programme, and b) to adopt and implement for 
the JRC work programme a framework for planning, monitoring and reporting. In this 
respect, the JRC has introduced in its 2016-2020 strategic plan an indicator to measure 
the delivery rate of the policy deliverables but no indicators have been set to measure 
the implementation of the planned projects and the achievement of the objectives. In 
addition, the work programme planning process has been reviewed, but no procedures 
for  monitoring  and  reporting  at  the  level  of  project  or  work  package  have  been 
'partially 
implemented  yet.  Consequently,  the  recommendation  is  assessed  as 
implemented and will not be closed; 
• 
Recommendation  N°  3  on  Governance  of  the  strategic  planning  and 
programming  cycle  (rated  important):  The  IAS  invited  JRC  to  ensure  a  wider 
involvement of senior and middle management in the preparation of the management 
plan  and  the  annual  activity  report  and  to  reinforce  its  internal  communication  plan. 
JRC adopted in December 2016 an "internal Communication strategy on key aspects 
of SPP cycle" which has not yet been implemented. In addition, the updated strategy 
does not include actions to improve the involvement of or the dialogue with staff and 
management as input for the preparation of the strategic plan/management plan/annual 
activity report. Consequently, the recommendation is not considered implemented and 
will not be closed; 
• 
Recommendation N° 5 on Reporting of the activity based management (rated 
important):  At  the  time  of  the  audit,  the  IAS  found  that  the  information  on  the 
allocation  of  human  resources  in  the  management  plan  and  in  the  annual  activity 
report  needed  to  be  improved.  However  the  new  strategic  plan/management  plan 
introduced  in  2016  does  not  require  disclosing  the  human  resources  needed  to 
implement each activity based budgeting activity. Consequently, this recommendation 
has become obsolete and will be closed.  

100 

 
 
15.  Follow-up audit of IAC recommendations in JRC 

Follow-up of the IAC audit on security and safety in the JRC 

Based on the results of our follow-up audit, we assess that recommendations N° 4 on 
Language  of  the  safety  management  systems  documents  (rated  important)  and  N°  12 
on Infrastructure-related IT tools impacting health and safety (rated important) were 
adequately and effectively implemented. 

Concerning the remaining two recommendations, the IAS found that further progress 
is required to fully mitigate the underlying risks. In particular: 
• 
Recommendation  N°  14  on  Scientific  activities  out  of  the  JRC  sites  (rated 
important): JRC should establish an overall procedure on security and safety needs for 
scientific  activities  conducted outside  the JRC  sites.  According  to  JRC  management, 
'the full implementation of the action is expected to be finalised by the end of 2016'. 
• 
Recommendation  N°  19  on  Evaluation  of  radiation  risks  (rated  important): 
JRC  has  not  yet  completed  the  evaluation  of  the  non-ionising  radiation  risks  at  the 
Ispra Site, under the terms of the Italian law. According to JRC management, 'the full 
implementation of the action is expected to be finalised by September 2016'. 

Follow-up of the IAC audit on nuclear decommissioning and waste management 
programme – financial aspects - recommendations N° 1, 2, 3 and 5 

Based on the results of our follow-up audit, we confirm the JRC's assessment that, at 
this  moment  in  time,  the  four  recommendations  (rated  very  important)  have  not  yet 
been fully implemented. 

We also consider that the actions implemented so far do not substantially mitigate the 
risks  identified  in  the  original  audit  report,  which,  consequently,  remain  at  a  high 
level. This needs to be adequately reflected in the 2015 AAR.  

Finally,  the  IAS  observed  that,  on  the  basis  of  the  information  gathered  during  the 
follow-  up  engagement,  the  JRC  is  unlikely  to  respect  the  due  date  of  2  June  2016 
fixed in the action plan for the completion of the mitigating measures. 
2nd  Follow-up  of  the  IAC  audit  on  nuclear  decommissioning  and  waste 
management programme – financial aspects  

Based on the results of our follow-up audit, we assess that recommendations N° 2 on 
An urgent improvement plan addressing internal causes of procurement delays (rated 
very  important)  and  N°  4  on  Any  new  plans  proposing  investments  in  nuclear 
infrastructure,  installations  or  buildings  (rated  important)  have  been  adequately  and 
effectively  implemented  while  recommendation  N°  6  on  The  future  regulatory 
standard  11510  (rated  important)  has  become  obsolete  in  the  meantime.  These  three 
recommendations will be closed. 

For  recommendation  N°  5  on  A  mid-term  staffing  strategy  for  the  Decommissioning 
Programme  (rated  very  important),  the  former  IAC  highlighted  the  urgent  need  to 
define a mid-term staffing strategy for the Decommissioning Programme for the Ispra 
site, to be then extended (as soon as possible) to other nuclear sites. The IAS observed 
that  the  staffing  strategy  has  been  defined  for  Ispra,  but  not  yet  for  the  other  sites. 
Therefore, the IAS considers the recommendation as only partially  implemented and 
will reopen it. However, since the most urgent staffing strategy has been defined, the 
IAS will downgrade the recommendation to important. 

101 

 
 
Follow-up of the IAC audit on management of expert groups by the JRC 

Based on the results of our follow-up audit, we assess that the four recommendations 
(all  rated  important)  N°  1  on  Framework  rules,  N°  6  on  Reimbursement  of  experts 
travel  and  subsistence  expenses,  N°  7  on  Reimbursement  of  experts  travel  and 
subsistence  expenses  and  N°  8  on  Document  management  have  been  adequately  and 
effectively implemented and will be closed. 

Follow-up of the IAC audit on document management in JRC 

Based  on  the  results  of  our  follow-up  audit,  we  assess  that  the  six  recommendations 
(all rated important) N° 2 on IT tools for document management, N° 3 on The unfiled 
documents,  N°  5  on  Filing  plan  structure  for  scientific  project,  N°  7  on  Storing 
conditions in sites other than Ispra, N° 9 on  DMO function and N° 11 on Guidance 
and training have been adequately and effectively implemented and will be closed. 

Follow-up of the IAC audit on decommissioning: risk and project management at 
the Ispra site 

Based on the results of our follow-up audit, we assess that recommendations N° 1 on 
The full adaptation of intermediate and low-level documents in line with the Nuclear 
Decommissioning and Waste Management Programme (rated important); N° 2 on The 
compatibility between the operation of the nuclear installations and the development 
of  decommissioning  projects  (rated  important);  N°  3  on  The  minimisation  of 
technological waste (rated important); N° 4 on The identification of the needed legal 
advice  resources  and  internal  legal  advice  specialised  in  nuclear  law  (rated 
important);  N°  5  on  The  systematic  collection  and  evaluation  of  ‘lessons  learned’ 
(rated  important)  and  N°  6  on  An  increased  upstream  involvement  of  the  JRC 
Licensing  function  in  all  strategy  and  operational  decisions  regarding  the  nuclear 
decommissioning  and  waste  management  programme  (rated  very  important)  have 
been adequately and effectively implemented and will be closed.  

However, further progress is required to fully mitigate the underlying risks of the two 
remaining recommendations: 
• 
Recommendation  N°  7  (rated  very  important)  on  A  full  strategy  for 
guaranteeing a pool of qualified Project Leaders: this recommendation is assessed as 
'partially  implemented'.  Although  the  JRC  defined  the  competence  profile  of  the 
project  leaders  for  decommissioning  it  has  not  yet  developed  a  training  scheme  for 
them. In addition, the IAS has not found evidence of any formal back-up arrangements 
for the Project Leaders. This may lead to weaknesses in operational activities and in 
the monitoring of contractor activities. Therefore, the JRC should define a full strategy 
for  guaranteeing  a  pool  of  qualified  project  leaders  on  decommissioning,  which 
includes a plan for their training and back-up; 
• 
important)  on  Document  management 
requirements: this recommendation is assessed as 'not implemented'. According to the 
original IAC recommendation, "the JRC should define and present a proposal for the 
Italian  Safety  Authority  regarding  what  essential  documents  with  legal  implications 
should  be kept on paper  format". This recommendation was aimed at addressing the 
finding  according 
for 
to  which  "the  document  management 
decommissioning  are  extremely  sophisticated,  in  term  of  quantity  and  quality  of  the 
documentation and legal implications; therefore, an electronic document management 
is  necessary".  The  IAS  has  not  found  any  formal  opinion  of  the  Italian  public 

Recommendation  N°  8  (rated 

requirements 

102 

 
 
administration  on  this  issue  that  was  either  requested  or  received  by  the  JRC.  The 
auditors, observed however, that the relevant public counterpart for this issue may not 
be the national Safety Authority, as requested by the IAC recommendation. Recently, 
the JRC has issued a procedure stating that "the original of documents with legal effect 
are kept for 5 years at least", but this period has no legal reference to the national law. 
Moreover,  the  JRC  Unit  on  decommissioning  has  only  partially  followed  this 
procedure. The misalignment between the national law, the JRC procedure and current 
practices  may  lead  to  issues  of  irregularity  and/or  to  inefficient  use  of  resources  for 
document  management.  On  the  basis  of  the  work  done,  the  auditors  assess  that  the 
finding  is  still  relevant  although  the  original  recommendation  is  not  applicable  or  is 
obsolete as it stands. For these reasons, the IAS has reformulated the recommendation 
as  follows:  "JRC  should  obtain  legal  advice  concerning  the  dematerialisation  of  the 
paper  form  documents  on  decommissioning,  in  line  with  the  EU  and  national 
requirements on decommissioning. The internal procedure on document management 
(dematerialisation  after  5  years)  should  be  updated  to  match  the  legal  requirements, 
and applied". 

These two recommendations will not be closed. 

Follow-up of the IAC audit on intellectual property rights management 

Based on the results of our follow-up audit, we assess that recommendations N° 3 on 
Incentives  for  scientific  staff  to  engage  in  technology  transfer  activities  (rated  very 
important),  No.  4  on  The  rules  governing  the  innovative  project  competitions  (rated 
important), N° 5 on Corporate procedure for access to external scientific information 
resources  service  (rated  very  important),  N°  6  and  7  on  The  reporting  on  activities 
under administrative arrangements within the EU Commission (rated very important), 
N°  8  on  The  term  of  reference  of  the  license  agreement  provisions  (rated  important) 
and N° 9 on Exclusive licenses under the EU Treaties on European Union and Atomic 
Energy  Community  (rated  very 
important)  were  adequately  and  effectively 
implemented. 

infringement  of 

Concerning  the  remaining  very  important  recommendation  N°  2  on  The  controls  to 
intellectual  property  rights,  the  original  audit  report 
prevent 
recommended to the JRC to include in the publications process a “check on the non-
infringement  of  prior  existing  copyrights  or  other  intellectual  property  rights  from 
third  parties  before  final  approval  for  publication  is  given".  To  implement  this 
recommendation,  the  JRC  proposed  two  actions,  notably  a)  to  include  in  the 
publications process a self-declaration by the author of the articles concerning the non-
infringement  of  prior  existing  copyrights,  b)  to  provide  a  link  to  the  guidelines  for 
copyright for EC staff. The IAS found that, while a link to the guidelines for copyright 
was  provided,  the  first  action  was  not  implemented.  According  to  JRC,  this  was 
mainly due to the fact that the self-declaration by the author was considered to have 
little  value.  As  a  result,  the  high  risk  identified  at  the  time  of  the  IAC  audit  has  not 
been mitigated. 

The IAS invites the JRC to take the appropriate measures to enhance the procedure for 
the management of the intellectual property rights to avoid possible infringements, by, 
for instance, replacing the planned self-declaration of the author of the articles with a 
more robust check performed by independent officials and/or with an anti-plagiarism-
software. Recommendation N° 2 will not be closed. 

103 

 
 
16.  Follow-up audit on the supervision of the implementation of CEF in DG 

MOVE 

Based on the results of our follow-up audit, we assess that recommendation N° 1 on 
DG  MOVE's  Supervision  Strategy  on  Corridors  Development  (rated  very  important) 
has  not  been  fully  implemented.  In  July  2016,  DG  MOVE  adopted  the  "Supervision 
Strategy  on  Core  Network  Corridors  Development"  which  i)  puts  the  supervision  of 
the development of the individual Core Network Corridors (CNC) in the wider context 
of the "achievement of the Commission's policy goals set out in the 2011 White Paper 
on  Transport  Policy";  and  ii)  covers  also  the  monitoring  of  the  TEN-T  legislation's 
performance in meeting the EU policy objectives. The strategy for the supervision of 
the individual CNCs' development is mainly set out in chapter "4: Internal processes 
to  assess  progress  and  performance  in  a  coherent  manner",  which  describes  the 
objectives of the supervision as well as the tools/support that DG MOVE will provide. 
However,  the  objectives  are  not  timed,  specific  or  quantifiable  and  there  are  no  key 
performance indicators that would allow DG MOVE to measure the performance of its 
supervision activity. There is also no clear indication as to how detected issues will be 
addressed. Consequently, the recommendation will not be closed. However, in view of 
the  mitigating  actions  implemented  so  far,  the  IAS  has  decided  to  downgrade  the 
rating of the recommendation from very important to important. 

17.  Follow-up audit on the implementation of FP7 control systems (including 

supervision of external bodies) in DG RTD 

Based  on  the  results  of  our  follow-up  audit,  we  assess  that  recommendation  N°1 
Supervision of the Joint Undertakings (rated very important) has been adequately and 
effectively implemented and will be closed.  

18.  Follow-up audit of the set-up of the common support centre for H2020 

Based on the results of our follow-up audit, we assess that recommendations N° 1.1 on 
The  decision  making  process  at  governance  level  (rated  very  important),  N°  1.3  on 
The audit strategy for H2020 (rated important) and N° 3 on Risk management (rated 
important) have been adequately and effectively implemented and will be closed. 

Concerning  recommendation  N°  1.2  on  The  decision  making  process  at  operational 
level (rated important), the IAS did not observe substantial progress made to mitigate 
the  risk  of  non-harmonised  implementation  of  audit  results  and  follow  up  of  fraud 
cases.  Consequently  the  IAS  considers  the  recommendation  as  not  yet  implemented 
and will reopen it.  

19.  Follow-up audit on EDF grants in DG DEVCO  

Based on the results of our follow-up audit, we assess that recommendation N° 7 on 
Ex-post  project  evaluation  (rated  important)  has  been  adequately  and  effectively 
implemented and will be closed. 

104 

 
 
20.  Follow-up audit on Budget Support in DG DEVCO 

Based on the results of our follow-up audit, we assess that recommendation N° 2 on 
Human  resources  for  budget  support  (rated  important)  has  been  adequately  and 
effectively implemented and will be closed. 

21.  Follow-up audit of IAC recommendations in DG ECHO 

Follow-up  of  the  IAC  audit  on  contribution  agreements  with  UN  Bodies  and 
other international organisations 

Based on the results of our follow-up audit, we assess that recommendations N° 1 on 
Project monitoring (rated very important),  N°  2  on  Reporting (rated very important) 
and N° 5 on Project design and selection (rated important) have been adequately and 
effectively implemented.  

The IAS, however, considers recommendation N° 3 on Verifications of UN Agencies 
and international organisations (rated very important) as partially implemented. Some 
measures have been implemented so far (corresponding to sub-points 3.1, 3.2, 3.3 and 
3.6  of  the  recommendation).  However,  DG  ECHO  has  not  yet  updated  the  audit 
manual to take into account the new ECHO audit strategy 2016-2020 and has not yet 
approved  the  annual  audit  plan  for  2016.  Consequently,  points  3.4  and  3.5  of  the 
recommendation  are  still  open.  They  will  not  be  closed.  Based  on  the  measures 
implemented  so  far,  the  IAS  considers  that  the  original  risks  have  been  partially 
mitigated and will downgrade this recommendation to important. 

22.  Follow-up audit of IAC recommendations in DG NEAR 

Follow-up of the IAC audit on special approvals and derogations 

The IAS followed up all 17 recommendations issued by the IAC. As a result:  

• 
Recommendations N° 1 (rated very important), N° 2b (rated important), N° 3a 
(rated  important),  N°  3b  (rated  important),  N°  3c  (rated  important),  N°  3d  (rated 
important), N° 4 (rated important), N° 5a (rated important), N° 5b (rated important), 
N° 5c (rated important), N° 5d (rated important), N° 6 (rated very important), N° 7a 
(rated  very  important),  N°  7b  (rated  very  important),  N°  7d  (rated  very  important) 
have been assessed as implemented; 
• 
Recommendation  N°  2a  on  Processing  of  prior  approvals  and  deviations 
(rated  important)  has  been  assessed  as  not  fully  implemented.  However,  the  IAS 
assessed  the  underlying  risk  as  low  and  the  recommendation  as  desirable; 
consequently, the recommendation will be closed; 
• 
Recommendation  N°  7c  (rated  very  important)  on  Exception  reporting  has 
been  assessed  as  not  implemented.  The  original  audit  recommended  that  Director  E 
(predecessor  of  the  current  Director  R)  should  regularly  review  and  analyse  the 
information  on  exceptions  and  non-compliances,  and  if  similar  cases  across  the 
Directorate General are identified, undertake the necessary follow-up measures. While 
currently an analysis of exceptions and non-compliance events is required to be carried 
out  by  each  directorate,  no  analysis  at  DG  level  is  envisaged.  Consequently, 

105 

 
 
recommendation  N°  7c  will  not  be  closed.  Its  criticality  level  remains  at  very 
important, as originally rated by the IAC. 

23.  Follow-up audit on performance audit of National Agencies (DG EAC) 

Based on the results of our follow-up audit, we assess that recommendation N° 1 on 
Internal  performance  (rated  very  important)  has  been  adequately  and  effectively 
implemented and will be closed. 

Concerning  recommendation  N°  3  on  Performance  measure  (rated  very  important), 
the IAS found that some actions have been implemented. In particular, DG EAC has 
revised  the  template  for  the  National  Agency  Work  programme  and  updated  its 
assessment procedure of the National Agencies' annual report and annual management 
declaration  to  cover  the  National  Agencies'  performance  against  the  operational 
objectives set in their respective work programmes. Pending the implementation of the 
remaining action on the conduct of the mid-term evaluation of Erasmus+ as required 
by the legal base (due 30/06/2018), the IAS considers that the underlying original risk 
has  been  partially  mitigated  and  will  downgrade  the  recommendation  from  very 
important to important. 

24.  Follow-up audit on preparedness of DG HOME for 2014-2020 legislation 

in shared management (ISF and AMIF) 

Based  on  the  results  of  our  follow-up  audit,  we  assess  that  recommendations  N°  1 
Overall planning of activities (i.e. roadmap) (rated important) and recommendation N° 
3  Designation  of  responsible  authorities  (rated  very 
important)  have  been 
implemented. 

25.  Follow-up audit on knowledge management in DG COMP 

Based on the results of our follow-up audit, we assess that recommendations N° 1 on 
Contribution from users to COMPWiki (rated important), N° 3 on The search function 
of  COMPWIKI  (rated  important)  and  N°  4  on  Handover  file  (rated  important)  have 
been adequately and effectively implemented and will be closed.  

26.  Follow-up audit on the preparedness of the management and control 

systems for the SME instrument in EASME  

Based on the results of our follow-up audit, we assess that recommendation N° 2 on 
Guidance  to  evaluators  and  quality  of  evaluations  (rated  very  important)  has  been 
adequately and effectively implemented and will be closed. 

106 

 
 
27.  2nd Follow-up audit on HR management in response to the financial crisis 

in DG ECFIN  

Based on the results of our follow-up audit, we assess that recommendation N° 2 on 
HR  annual  planning  (rated  very  important)  has  been  adequately  and  effectively 
implemented and will be closed. 

28.  Follow-up audit on risk management and planning processes in the new 

economic governance context in DG ECFIN 

Based on the results of our follow-up audit, we assess that recommendation N° 1 on 
Risk management (rated important) has been adequately and effectively implemented 
and can be closed. 

Concerning  recommendations  N°  2  on  Management  plan  objectives  and  their 
alignment with operational planning and management (rated important) and N° 3 on 
Performance  monitoring  and  reporting  in  the  AAR  (rated  important),  improvements 
have  been  observed  in  terms  of  setting  objectives  and  indicators  in  the  Management 
Plan and their reporting in the Annual Activity Report. However, the elements related 
to  the  planning  and  monitoring  at  the  operational  level  have  not  yet  been  fully 
implemented. As a result, the IAS considers that further actions are deemed necessary 
to  adequately  mitigate  the  underlying  risks  identified  in  the  original  audit.  In 
particular: 
• 
Recommendation N° 2: DG ECFIN carried out a pilot exercise by mapping the 
2015  Management  Plan  with  the  work  plans  of  two  directorates.  This  exercise 
provides  a  starting  point  for  ensuring  a  coherent  structure  and  linkage  to  the 
operational  tasks.  However,  the  mapping  between  the  Management  Plan  and  the 
operational plans has still to be carried out for all the directorates. Additionally, DG 
ECFIN  has  to  define  a  standard  structure  and  minimum  requirements  for  the 
operational plans; 
• 
Recommendation  N°  3:  the  reporting  on  performance  against  the  key 
objectives  in  the  Management  Plan  has  been  improved  in  the  final  2015  Annual 
Activity  Report.  However,  there  is  still  no  consistent,  systematic  monitoring  of 
performance  at  the  operational  level  and  no  minimum  requirements  for  monitoring 
beyond the mid-year review of the Management Plan and the Annual Activity Report, 
e.g.  through  regular,  documented  status  updates  on  the  directorate  plans  in 
management meetings.  

29.  Follow-up audit of IAC recommendations in DG ECFIN 

Follow-up of the IAC audit on DG ECFIN's document management 

Based on the results of our follow-up audit, we assess that recommendations N° 1 on 
Strengthen the archiving process and increase awareness (rated important), N° 3 on 
Improving  quality  review  of  document  management  and  defining  document 
management objectives for staff (rated important) and N° 4 on Encouraging the use of 
E-signatory within DG ECFIN (rated important), have been adequately and effectively 
implemented and can be closed. 

107 

 
 
Establish  statistics  to  assess  the  performance  and  compliance  with  the 

Recommendation  N°  2  on  Public  requests  to  access  documents  (rated  important)  is 
assessed as not implemented. This recommendation required DG ECFIN to: 
• 
applicable legislation. However, no indicators have been developed yet; 
• 
Ensure  a  consistent  approach  and  raise  awareness  to  directorates  and 
operational units on handling public requests. The IAS found that a note on the state of 
play  was  issued  by  Unit  R4  on  23/06/2015,  but  since  then  no  other  action  has  been 
implemented. 

A training course was recently promoted by R3 (former R4) on how to manage access 
to document requests. However, out of the entire target population of 725 staff, only 
two  staff  members  attended  this  event.  No  other  evidence  of  raising  awareness  to 
directorates and operational units on handling public requests was provided. The IAS 
will reopen the recommendation.  

30.  Follow-up audit of IAC recommendations in DG FISMA  

2nd  Follow-up  of  IAC  audit  on  effectiveness  of  HR  management  to  support  the 
financial crisis 

Based on the results of our follow-up audit, we assess that for recommendation N° 3 
on Monitoring and reporting on HRM (rated very important) further improvements are 
needed  to  effectively  implement  the  recommendation.  The  IAS  will  reopen  the 
recommendation. 

The IAS recognises the progress made by DG FISMA in measuring several indicators 
on  HR  management  and  reporting  them  to  DG  FISMA's  senior  management.  They 
provide a picture of the staff structure (gender, nationality and category), recruitment 
(turnover  rate  and  staff  profile)  and  working  conditions  (sick  leave  rate,  parental 
leaves,  work  patterns  and  use  of  recuperation).  They  represent  a  positive  first  step 
towards  the  implementation  of  an  effective  monitoring  and  reporting  system. 
However,  the  current  indicators  mainly  focus  on  outputs  and  are  not  complemented 
with others focusing on results. For example, gender balance in middle management is 
measured via the percentage of middle management posts held by women but this is 
not  complemented  with  other  indicators  to  demonstrate  the  DG's  performance  in 
addressing  it  (which  could  be  measured,  for  instance,  via  the  participation  rate  of 
women  in  coaching  for  team  leaders).  In  addition,  these  indicators  are  not 
accompanied  by  complementary  information  explaining  whether  or  not  indicators 
highlight  possible  problems  or  identifying  possible  correlation  between  factors. 
Finally, the comparison of DG FISMA performance with the Commission's averages 
is  not  complemented  by  a  comparison  with  DG  FISMA  targets  and  their  evolution 
over time. These additional analyses would allow DG FISMA senior management to 
identify  potential  problems  and  their  causes  and  to  adequately  address  them.  In 
addition,  the  IAS  considers  that  DG  FISMA  should  better  align  the  HR  monitoring 
reports with the objectives expressed in its Strategic Plan 2016-2020 and Management 
Plan 2016 in order to demonstrate the progress made towards their achievement.  

108 

 
 
 
31.  Follow-up audit on the performance of DG GROW's supervision of ESA's 

implementation of Galileo  

Based on the results of our follow-up audit, we assess that recommendation N° 3 on 
DG  GROW's  Supervision  Strategy  (rated  very  important)  has  been  adequately  and 
effectively implemented and will be closed.  

Concerning recommendation N° 5 on Key Performance Indicators (rated important) to 
establish reporting by ESA that enables the DG to effectively monitor key elements of 
ESA's operational activities, the revised delegation agreement contains clearly defined 
Key  Performance  Indicators,  on  which  ESA  will  report  quarterly.  As  the  revised 
delegation agreement has not yet been signed, the recommendation has not been fully 
implemented. Therefore, the IAS will re-open recommendation N° 5. 

In  view  of  the  actions  implemented  so  far  regarding  the  two  open  recommendations 
N° 1 on Implementation of the procurement activities (rated very important) and N° 2 
on  Cooperation  between  DG  GROW  and  ESA  (rated  very  important)  the  IAS 
considers the related risks to be partially mitigated. Therefore, both recommendations 
can be downgraded from very important to important. 

32.  Follow-up audit of IAC recommendations in DG GROW 

Follow-up of IAC audit on the internal control strategy of GSA over the budget 
delegated by DG ENTR, focusing on procurement 

Based on the results of our follow-up audit, we assess that recommendations N° 1 on 
Manual  of  procedures  (rated  very  important),  N°  2  on  Checklists  (rated  very 
important), N° 4 on Conflict of interest policy (rated important) and N° 6 on Document 
management  policy  (rated 
important)  have  been  adequately  and  effectively 
implemented and will be closed. 

33.  Follow-up audit on the customs performance measurement system in DG 

TAXUD  

Based on the results of our follow-up audit, we assess that recommendation N° 1 on 
Performance measurement of committees and groups (rated very important) and N° 3 
on  Customs  programmes  evaluations  and  monitoring  (rated  important)  have  been 
adequately and effectively implemented. 

According  to  recommendation  N°  2  Performance  measurement  of  DG  TAXUD 
customs  activities  (rated  very  important),  DG  TAXUD  should  develop  its  planning, 
measurement  and  monitoring  processes  so  that  these  become  an  effective  tool  to 
manage,  supervise  and  improve  operational  activities  at  all  levels.  In  line  with  the 
recommendation, DG TAXUD implemented more controllable objectives and results 
reflecting  its  most  important  interventions  and  activities  and  introduced  Unit 
Management  Plans.  In  this  context,  the  DG  also  strengthened  the  risk  management 
assessment  process  by  linking  it  to  the  priorities  defined  at  unit's  level  and  by 
organising several specific workshops. Internal communication reporting, monitoring 
and  supervision  were  also  improved.  However,  one  of  the  sub-actions  for  this 
recommendation  envisages  that  the  Board  of  Directors  is  informed  at  least  twice  a 
109 

 
 
year about the results of a defined set of key performance indicators (scoreboard). This 
has  not  yet  taken  place  and  is  planned  to  be  implemented  in  autumn  2016.  For  this 
reason, the IAS considers that the recommendation is not yet fully implemented. As a 
consequence,  the  recommendation  will  not  be  closed.  However,  taking  into  account 
the improvements already made, we consider that the risk has been partially mitigated 
and therefore the recommendation is downgraded from very important to important.  

34.  Follow-up audit of IAC recommendations in DG TAXUD  

Follow-up of IAC audit on DG TAXUD's external communication strategy 

Based on the results of our follow-up audit, we assess that recommendations N° 3 on 
Unclear  definition  of  roles  and  responsibilities  (rated  very  important),  N°  4  on 
Internal  networking  and  work  coordinators  (rated  important),  N°  5  on  Capacity 
building and trainings on external communication (rated important), N° 6 on External 
communication strategy (rated important), N° 9 on Risk assessment in unit R3 (rated 
important), N° 10 on Contractors and contract management (rated important), N° 11 
on  Europa  Website  and  Social  Med  (rated  very  important)  and  N°  12  on  Use  of 
communication tools (other than Europa) (rated important) have been adequately and 
effectively implemented and can be closed.  

Recommendations  N°  1  on  Communication  as  core  business  (rated  very  important) 
and  N°  14  on  Monitoring  of  implementation  of  the  external  communication  strategy 
(rated very important) are partly implemented: 
• 
Recommendation  N°  1  calls  for  an  adequate  recognition  of  the  strategic 
importance  of  communication  in  the  taxation  and  customs  area.  It  is  the  subject  of 
public presentations to newcomers. HoUs' responsibility in external communication is 
formalised  in  most  but  not  all  job  descriptions.  Communication  activities  are 
addressed  in  the  Annual  Communication  Plan  (ACP)  for  2015  and  the  Unit 
Management Plans (UMP). However, neither the ACP nor the UMPs are aligned with 
the  recently  reviewed  and  approved  Multiannual  External  Communication  Strategy. 
Moreover,  UMPs  do  not  explicitly  plan  communication  actions  as  required  by  the 
applicable guidelines but rather some specific outputs (e.g. publications); 
• 
Recommendation N° 14 calls for performance measurement and management 
of external communication. Although DG TAXUD collects statistics on web site page 
views, the full implementation of this recommendation is pending the selection of the 
Key  Performance  Indicators,  the  online  availability  of  DG  COMM’s  tools  and 
benchmarks  and  the  on-going  centralisation  in  DG  COMM  of  some  core  horizontal 
communication activities. 
Considering  the  actions  already  taken  and  the  residual  risk  that  DG  TAXUD  is 
exposed to, we propose to downgrade the level of significance from very important to 
important for both recommendations. 

Recommendations  N°  7  on  Annual  planning  of  external  communication  actions  in 
units responsible for communications and policy units (rated very important) and N° 8 
on  Incomplete  audit  trail  for  budget  estimation  and  allocation  (rated  important)  are 
assessed as not implemented: 
• 
According to Recommendation N° 7, DG TAXUD should establish an ACP in 
line  with  the  Multiannual  External  Communication  Strategy.  The  IAS  observed  that 
for  2016  only  the  sectorial  communication  plan  for  the  Union  Customs  Code  is 

110 

 
 
available.  In  addition,  as  mentioned  previously  for  recommendation  N°  1,  the  2015 
ACP was not aligned with the recently reviewed and approved Multiannual External 
Communication  Strategy  and  the  communication  expenditure,  due  to  the  substantial 
modifications  to  the  original  budget  made  throughout  the  year  not  preceded  or 
followed by adequate and sufficient justifications; 
• 
Recommendation N° 8 recommends DG TAXUD to clearly establish the link 
between the ACP, the UMPs and the budget lines. However, the IAS did not find clear 
documented explanations for most of the budget revisions made in 2015. 

35.  Follow-up audit on financial and procurement management in DG 

TRADE 

Based on the results of our follow-up audit, we assess that recommendations N° 2 on 
Procurement  process,  needs  assessment  (rated  important)  and  N°  3  on  Reporting  on 
financial  data  (rated  important)  have  been  adequately  and  effectively  implemented 
and will be closed. 

Concerning  recommendation  N°  1  on  Procurement  procedure,  compliance  issues 
(rated  very  important),  for  which  your  service  has  requested  to  review  the  progress, 
the IAS notes that DG TRADE improved its internal guidance, training and support to 
streamline  and  enhance  the  procurement  procedures,  but  has  not  yet  re-assessed  its 
control model in place to increase the effectiveness (revised deadline: 30 June 2016) 
and decrease the risk of compliance issues with the applicable legal and administrative 
provisions. In view of the progress observed, the IAS considers that the original risk 
has  been  partially  mitigated  and  consequently  has  downgraded  the  criticality  of  the 
recommendation from very important to important. 

36.  Follow-up audit of IAC recommendations in OIB 

    Follow-up of IAC audit on concept and reproduction at the OIB 

Based on the results of our follow-up audit, we assess that recommendations N° 4 on 
Volume  and  production  cost  –  1  (rated  important),  N°  5  on  Volume  and  production 
cost – 2  (rated important), N° 9 on  Concept and reproduction-Rationalisation (rated 
very important), N° 12 on Resources Evolution (rated very important) and N° 13 on 
Industrial  strategy  –  1  (rated  important)  have  been  adequately  and  effectively 
implemented and will be closed.  

37.  Follow-up audit of IAC recommendations in DG SCIC 

Follow-up of IAC audit on the technical support provided to meetings and 
conferences 

Based on the results of our follow-up audit, we assess that recommendation N° 1 on 
The definition of a corporate governance framework (rated 'very important') has been 
adequately and effectively implemented and can be closed. 

111 

 
 
 
The  recent  Communication  on  Synergies  and  Efficiencies  explicitly  clarified  DG 
SCIC's mandate with respect to events and meeting room management, including the 
assignment of the ownership of the corporate process to DG SCIC. The IAS considers 
that in view of this, the main risks associated with the original IAC recommendation 
are  mitigated.  Furthermore,  given  that  DG  SCIC's  mandate  has  been  extended 
following the Synergies and Efficiencies review and following discussions with your 
services,  the  IAS  considers  that  the  four  remaining  recommendations,  together  with 
the related risks, remain valid even though the original audit report was drawn up at a 
time  when  DG  SCIC's  responsibilities  were  actually  more  limited.  For  example,  the 
DG still needs to establish a list of meeting rooms that will be managed by DG SCIC 
(recommendation N° 2, rated important) and will need to define and validate a service 
management plan (recommendation N° 3, rated very important). Furthermore, the use 
of  the  IT  tool  for  this  process  will  need  to  be  defined  (recommendation  N°  4,  rated 
important) and a quality assurance and improvement programme for the provision of 
the  technical  services  will  need  to  be  developed  (recommendation  N°  5,  rated 
important).  However,  we  acknowledge  that  the  original  action  plan  and  target  dates 
are now effectively superseded following the review. We therefore invite DG SCIC to 
draw  up  a  new  action  plan  addressing  the  four  remaining  recommendations  and 
provide us with new target dates. 

38.  Follow-up audit of IAC recommendations in DG ESTAT 

Follow-up of IAC audits on statistical processes I – GNI data, sensitive 
information, statistical process III – Agriculture statistics and ESTAT's business 
continuity 

Based on the results of our follow-up audit, we assess that recommendation N° 1 on 
The set-up of sensitive information in ESTAT (rated very important) from the audit on 
sensitive information; recommendations N° 1 on Organisational structure (rated very 
important),  N°  3  on  Annual  crop  statistics  production  -  collection,  validation, 
processing  and  dissemination  (rated  very  important)  and  N°  5  on  the  Compliance 
monitoring process in Unit E1 (rated important) from the audit on statistical process 
III  –  Agriculture  statistics  and  recommendations  N°  1  on  Business  continuity 
management  governance  and  setup  in  ESTAT  -  Roles  and  responsibilities  (rated 
important) and N° 2 on Business impact analysis and risk assessment (rated important) 
from the audit on ESTAT's business continuity have been adequately and effectively 
implemented. 

Follow-up of IAC audits on statistical processes I – GNI data, sensitive 
information, and ESTAT's business continuity 

Based on the results of our follow-up audit, we assess that recommendation N° 2 on 
Security  of  sensitive  information  in  the  dissemination  chain  (rated  very  important) 
from the audit on sensitive information and N° 5 on Business continuity management 
testing  from  the  audit  on  ESAT's  business  continuity  have  been  adequately  and 
effectively implemented. 

Follow-up of IAC audits on statistical process III – Agriculture statistics and 
ESTAT's business continuity 

112 

 
 
Based on the results of our follow-up audit, we assess that recommendation N° 3 on 
Business  continuity  plan  (rated  very  important)  from  the  audit  on  ESAT's  business 
continuity has been adequately and effectively implemented and will be closed. 

• 
Recommendations N° 2 on Farm Structure Survey (FSS) statistical production, 
collection,  validation,  processing  and  dissemination  (rated  very  important)  and  N°  4 
on  Treatment  of  confidential  data  in  the  agricultural  statistical  processes  (rated  very 
important) from the audit on statistical process III – Agriculture statistics are not fully 
implemented. However, considering the actions already taken and the residual risk that 
DG  ESTAT  is  exposed  to,  these  recommendations  are  downgraded  from  very 
important to important. 
• 
Recommendation  N°  2  requires  DG  ESTAT  to  improve  the  quality  and  the 
availability  of  data,  and  fix  several  weaknesses  related  to  outdated  and  incomplete 
documentation  about  the  production  process,  methodological  aspects  and  data 
validation  issues.  It  also  recommends  revising  the  structure  of  FSS  data  in  the 
dissemination data base, better following up issues about administrative data sources 
presented  in  the  Standing  Committee  for  Agricultural  Statistics  and  clarifying  the 
respective roles and responsibilities between DG ESTAT and DG AGRI. Most actions 
have been implemented in line with the action plan. However, DG ESTAT still needs 
to  improve  the  structure  of  FSS  data  in  the  dissemination  data  base.  In  particular,  it 
should  finalise  the  design  of  the  dissemination  tables,  prepare  them  in  the  ESTAT 
dissemination  database,  programme  the  table  structures  into  the  Eurofarm  database 
and produce the tables from the raw data; 
• 
Recommendation  N°  4  calls  for  securing  the  hosting  of  confidential  data 
outside  ESTAT’s  secure  environment,  updating  the  manual  on  the  protection  of 
confidential  data,  and  strengthening 
implementation  of  confidentiality 
requirements  for  the  encrypted  transmission  to  Member  States,  the  access  rights  for 
staff  and  the  filtering  processes  at  the  dissemination  stage.  All  actions  have  been 
implemented in line with the action plan except for the part related to the hosting of 
confidential data and the update of the manual on the protection of confidential data. 

the 

39.  Follow-up audit on management of local IT in DG AGRI  

Based on the results of our follow-up audit, we assess that recommendations N° 1 on 
IT governance (rated very important), N° 2 on IT strategy (rated very important), N° 3 
on  IT  risk  management  (rated  important),  on  N°  6  on  Project  management  (rated 
important), N° 8 on Management of firewalls (rated important) and N° 9 on Change 
management (rated important) have been adequately and effectively implemented and 
can be closed. 

Concerning recommendation N° 4 on Performance management (rated important), no 
performance indicators have been defined yet to cover the aspects of IT service design 
(service  level,  capacity,  availability),  transition  (change,  release,  testing,  and 
configuration management) and operations (incident and problem management). 

Concerning recommendation N° 10 on Configuration management (rated important), 
the current configuration management database is limited to the main IT systems and 
does not include the complete inventory of configuration items, with their attributes, 
baseline configuration and relationships.  

113 

 
 
Therefore, the IAS concludes that recommendations N° 4 and 10 have not been fully 
implemented and will not be closed.  

40.  Follow-up audit on IT governance in DG Budget  

Based on the results of our follow-up audit, we assess that recommendations N° 1 on 
IT Governance structure and key roles (rated very important), N° 4 on Performance 
measurement, monitoring and reporting of IT Activities (rated important) and N° 6 on 
IT  policy  and  strategy  (rated  important)  have  been  adequately  and  effectively 
implemented and will be closed. 

Concerning recommendations N°2 on IT organisation (rated very important) and N°3 
on Priority setting and planning of activities (rated very important) we have observed 
good progress in the implementation of the action plan,  but consider that the related 
risks have not yet been fully mitigated and consequently the recommendations cannot 
be  closed.  Nevertheless,  the  rating  for  both  recommendations  is  downgraded  from 
very important to important due to the progress made. 

Concerning recommendation N° 5 on HR Management (rated important), the IAS has 
not  found  sufficient  results  of  the  actions  implemented  and  therefore  concludes  that 
the recommendation cannot be closed. 

41.  Follow-up audit on management of European Commission Authentication 
Service - ECAS  

Based on the results of our follow-up audit, we assess that recommendations N° 1 on 
Vision and strategy for identity and access management (rated very important), N° 2 
on Definition of ECAS security roles and responsibilities (rated important), N° on 5 on 
ECAS dependency on AD, CED and CUD (rated very important), N° 6 on Involvement 
of  D  HR  DS  in  ECAS  security  management  (rated  important),  N°  7  on  Definition  of 
IAM  and  ECAS  services  in  the  service  catalogue  (rated  important)  and  N°  8  on 
Planning  of  the  EXODUS  project  (rated  very  important)  have  been  adequately  and 
effectively implemented and will be closed.  

Concerning  recommendation  N°  4  on  Security  requirements  for  ECAS  (rated  very 
important),  while  observing  good  progress  in  the  implementation  of  the  action  plan, 
the IAS considers that the related risks are not yet fully mitigated and consequently the 
recommendation  cannot  be  closed.  In  addition,  as  identifying  and  implementing  the 
missing security measures has not been finalised yet, ECAS is still vulnerable to the 
high risks identified at the time of the audit. 

42.  Follow-up audit of IAC IT recommendations in DG DIGIT  

Follow-up of IAC audit on external staff management 

Based  on  our  follow-up  results,  we  have  assessed  that  recommendations  N°  1  on 
Harmonise procedure for access request (rated important), N° 2 on Establish a central 

114 

 
 
 
local  point  in  Brussels  (rated  very  important),  N°  7  on  Clean  and  update  data  in 
ORIANA  (rated  very  important),  N°  8  on  Further  development  of  ORIANA  (rated 
important)  and  N°  11  on  Return  of  access  cards  (rated  important)  have  been 
adequately and effectively implemented and can be closed. 

Concerning the four other recommendations, the IAS considers that not all the planned 
actions have been implemented and the related recommendations can therefore not be 
closed:  
• 
Regarding  recommendation  N°  3  on  Harmonised  validity  of  access  cards 
(rated  important),  the  main  issue  is  that  DG  DIGIT  encodes  the  end  date  of  the 
framework  contracts  instead  of  the  specific  contracts  for  external  service  providers 
(ESP) in the tool ORIANA. This end date of the framework contract is then reported 
on  the  access  card  and  checked  by  guards  to  allow  entrance  to  the  EC  buildings. 
However,  specific  contracts  are  concluded  for  the  acquisition  of  services  for  a 
particular  profile,  corresponding  to  one  specific  individual,  for  a  period  (from  a  few 
days to a full year) generally shorter than the duration of the framework contracts. The 
current practice does not respect the instructions provided by the Security Directorate 
of DG HR and exposes the Commission to the risk that individuals may be allowed to 
enter the EC buildings despite they are no longer covered by a specific contract; 
• 
Regarding  recommendations  N°  6  on  Develop  guidelines  to  address  ethical 
aspects  (rated  important)  and  N°  9  on  Security  awareness  kit  for  external  service 
providers (rated important), the issue is that there is no evidence that units hiring ESPs 
other than DIGIT.R.1 use the template document called "Procedure interne à remettre 
au prestataire", which contains practical information on entry into service, and hand it 
to  ESPs.  Furthermore,  there  is  no  formal  acknowledgment  of  the  document  being 
received by the ESP. 
• 
Regarding recommendation N° 10 on Departure forms (rated very important), 
the  main  issue  is  that,  in  the  absence  of  an  automated  process,  which  might  not  be 
cost-effective  to  put  in  place  due  to  the  inherent  complexity,  DG  DIGIT  operational 
units  are  responsible  for  ensuring  that  access  and  parking  cards  are  returned  by  the 
ESPs upon their departure of the ESPs and taking the appropriate measures to collect 
their access and parking cards. However, figures provided by the Security Directorate 
of DG HR indicate that about 10% of the access and parking cards are not returned by 
the ESPs at the time their contract comes to an end. Therefore, the IAS considers that 
this  recommendation  is  not  effectively  implemented  and  will  not  be  closed. 
Nevertheless, as the operational units deactivate the ESP profile in Oriana and inform 
the  Security  Directorate  of  DG  HR  via  the "Formulaire  de  départ  d'un  prestataire  de 
services" that the access card should be deactivated, we consider the recommendation 
is  partially  implemented  and  thus  can  be  downgraded  from  very  important  to 
important. 

As a consequence, the IAS will not close recommendations N° 3, 6, 9 and 10. 

43.  2nd Follow-up audit on management of local IT in DG ESTAT  

Based  on  the  results  of  our  follow-up,  we  assess  that  recommendations  N°  3  on 
Project performance measurement, reporting and monitoring (rated important) and N° 
6  on  User  accounts  management  (rated  important)  have  been  adequately  and 
effectively implemented and will be closed.  

115 

 
 
The IAS considers that the planned actions for recommendations N° 4 on Information 
systems  security  and  N°  5  on  Security  requirements  for  managing  confidential  data 
(both rated very important), have not been fully implemented. As the other activities 
mentioned in the action plan for these two recommendations have been implemented, 
we  consider  the  risks  have  been  partially  mitigated  and,  thus,  both  issues  can  be 
downgraded from very important to important. 

• 
For  recommendation  N°  4,  the  main  outstanding  issue  is  linked  to  the  IT 
security  plans  not  being  in  line  with  Implementing  Rules  of  Commission  Decision 
C(2006)3602: 
o 
DG  ESTAT  classified  its  STANDARD  information  systems  in  three 
categories  (Information  Transmission,  Statistical  Applications  and  Data 
Management) and proposed to develop one IT security plan for each category. 
However, existing security plans cover only one single system in each category 
(resp. EBUS, EDIT and IS4STAT). We invite DG ESTAT to revise the scope 
of  each  IT  security  plan,  to  include  all  the  information  systems  under  each 
category; 
An  IT  security  plan  is  still  missing  for  the  SPECIFIC  system  EGR.  As  this 
system is planned to be migrated to a new secure environment currently under 
construction  by  DG  ESTAT  and  DG  DIGIT,  we understand  that this security 
plan will be developed and implemented in parallel with the new environment. 

• 
The  main  outstanding  issue  for  recommendation  N°  5  is  the  following:  to 
replace  the  process  of  mounting  its  secure  environment  on  user  workstations,  which 
does  not  provide  an  adequate  level  of  security  for  confidential  statistical  data,  DG 
ESTAT  launched  a  project  to  deploy  a  local  IT  infrastructure  containing  DMZ  and 
more secure rules for access to data. When, in 2015, DG ESTAT decided to move its 
secure  environment  to  the  corporate  data  centre  of  DG  DIGIT,  a  new  infrastructure 
with 3 DMZs was designed for the storage of sensitive statistical data. Access to data 
and  the  application  will  be  through  a  Windows  Terminal  Server,  which  will  add  a 
layer of security by preventing data transfer in clear through the corporate network and 
data storage on user workstation. The pilot for this project is planned to be finalised in 
February  2017,  before  a  phased  deployment  in  production  for  the  different 
applications. 

44.  2nd Follow-up audit on management of local IT in DG MARE  

   Based on the results of our follow-up audit, we assess that recommendations N° 1 on 
IT  strategy  and  IT  priorities  (rated  very  important),  N°  4  on  Change  management 
(rated  very  important)  and  7  on  Project  management,  quality  assurance  and  service 
management (rated very important) have been adequately and effectively implemented 
and will be closed.  

The  IAS  considers  that  not  all  the  planned  actions  have  been  implemented  for 
recommendation N° 5 on IT security management (rated important) for the following 
reasons: 

116 

 
 
 
 
There  is  no  evidence  that  compliance  of  DG  MARE's  IT  with  Commission 

A  number  of  controls  defined  in  the  IT  security  plans  have  not  yet  been 

• 
Not  all  IT  systems  under  DG  MARE's  responsibility  are  covered  by  an  IT 
security  plan  duly  approved  by  the  Director-General  under  in  accordance  with  the 
Implementing Rules of Commission Decision C(2006)3602; 
• 
implemented; 
• 
standards is regularly reviewed and reported by the LISO; 
• 
security incidents to the ITSC or immediate escalation to the senior management; 
• 
The  document  on  security  in  IT  project  management  specifying  IT  security-
oriented  deliverables  in  each  project  phase  has  not  been  approved  by  the  ITSC  and 
there is no evidence that it has been implemented. 

Absence  of  a  procedure  establishing  (as  a  minimum)  a  yearly  report  on  IT 

As a consequence, the IAS will not close recommendation this recommendation.  

45.  2nd Follow-up audit on management of local IT in DG TRADE 

Based on the results of our follow-up audit, we assess that recommendations N° 1 on 
Role of the IT Steering Committee (rated very important) and N° 2 on Management of 
IT  related  risks  in  DG  TRADE  (rated  very  important)  have  been  adequately  and 
effectively implemented and will be closed. 

46.  Second follow-up to the performance audit on the Anti-fraud Information 
System (AFIS) by the former Internal Audit Capability at OLAF. 

Based on the results of our follow-up audit, we assess that recommendations N °5 on 
System  improvements  -  reporting  and  ergonomy  (rated  important),  N°  8  on  Data 
integration  with  other  applications  (rated  important)  and  N°  12  on  AFIS  Steering 
Committee  (rated  important)  have  been  adequately  and  effectively  implemented  and 
will be closed. 

Regarding  recommendation  N°  9  on  User  account  management  (rated  important), 
despite  the  deployment  of  the  recommended  tool  already  in  February  2016,  its  first 
results,  in  particular  the  annual  user  validation  report,  will  be  available  in  February 
2017 only. Therefore, the IAS cannot yet assess if the implemented functionality duly 
mitigates  the  identified  risks.  As  a  consequence,  the  IAS  will  not  close  this 
recommendation.  

List of follow-up audits performed in 2016 for which all recommendations 
have been closed after the follow-up 

Based on the results of the follow-up audits performed in 2016, the IAS assessed that 
all the recommendations that resulted from the audits listed below and that remained 
open before the follow-up could be closed. 

Audit Title 

117 

 
 
47. Follow-up audit on Anti-Fraud strategy– Multi DG 
48. Follow-up audit on the objectives setting process in the context of the 
preparation of the management plans – Multi DG 
49. 3rd Follow-up  audit on the management and monitoring of staff 
allocation in DG AGRI 
50. Follow-up audit of IAC and IAS recommendations in DG AGRI (IAS 
audit on control strategy implementation and IAC audits on DG AGRI 
readiness for the implementation of the enhanced role of certification bodies 
in the new assurance model and the international dimension of the GI and 
organic policies) 
51. Follow-up audit of IAC recommendations in DG SANTE (IAC audits on 
external stakeholder consultations in DG SANTE, on costing practices on 
procurement in selected funding areas in DG SANTE, on the operations of 
Directorate F and the Food and Veterinary Office, on business continuity and 
on internal controls standards 5, 6, 7 and 8) 
52. Follow-up audit on DGs ENV and CLIMA's externalisation to EASME of 
the LIFE programmes 2014-2020 
53. Follow –up audit of IAC recommendations in DG ENV/DG CLIMA (DG 
ENV-CLIMA SIAC audits on IT governance and management in DG ENV 
and DG CLIMA and on Anti-Fraud strategy in DG ENV and DG CLIMA) 
54. Follow-up audit on the Limited review of the calculation and the 
underlying methodology of the residual error rates for the 2014 reporting year 
in DG EMPL 
55. 2nd Follow-up audit on DG EMPL's performance measurement systems 
56. Follow-up audit of IAC recommendations in DG EMPL (IAC audit on 
performance measurement) 
57. 2nd Follow-up audit on DG REGIO's performance measurement systems 
58. Follow-up audit of IAC recommendations in DG REGIO (IAC audits on 
major projects and on readiness assessment - ERDF 2000-2006 closure 
process) 
59. Follow-up audit of IAC recommendations in DG JRC (IAC audits on 
business continuity, third party liability, portfolio of buildings, asset 
management and management and sharing of scientific and technical 
knowledge) 
60. Follow-up audit of IAC recommendations in DG RTD (IAC audits on 
objectives, indicators and monitoring, fusion expenditure, communication, 
implementation of ex-post audit results, management of the risk sharing 
finance facility, Desk review on SEP evaluation, contribution to Joint 
Undertakings, management of project reports and dissemination of research 
Results (FP7) and the processes related to the closure of FP7 grants. 
61. Follow-up audit on the implementation of FP7 control systems in ERCEA  
62. Follow-up audit of IAC recommendations in REA (IAC audit on the 
implementation of ex-post audit findings) 
63. Follow-up audit on Implementation of the Anti-Fraud Strategy in REA  
64. Follow-up audit on H2020 grant management in DG CONNECT 
65. Follow-up audit on the implementation of FP7 control systems in DG 
CONNECT 
66. Follow-up audit of IAS and IAC recommendations in DG CONNECT 
(IAS audits on the implementation of FP7 control systems and on H2020 

118 

 
 
grant management, IAC audits on impact assessment and on delegated 
(externalised) research) 
67. Follow-up audit on the Limited review of the calculation and the 
underlying methodology of the residual error rate for the 2015 reporting year 
in DG MOVE 
68. Follow-up audit on the Limited review of the calculation and the 
underlying methodology of the residual error rate for the 2015 reporting year 
in DG ENER 
69. Follow-up audit of IAC recommendations in INEA (IAC audit on 
procurement) 
70. Follow-up audit on the adequacy and effective implementation of DG 
ECHO's Anti-Fraud strategy 
71. Follow-up audit on the assurance building process in EU Delegations (DG 
DEVCO) 
72. Follow-up audit on programme estimates financed by EU and EDF budget 
in DG DEVCO 
73. 2nd Follow-up audit on DG ECHO: financial management of 
humanitarian aid 
74. Follow-up audit of IAC recommendations in DG ECHO (IAC audits on 
the legality and regularity of payments for the year 2012 in DG ECHO and 
financial management of humanitarian aid) 
75. Follow-up audit of IAC recommendations in DG DEVCO (IAC audits on 
management of DEVCO's resources in EU Delegations, on communication 
flows between DEVCO's HQ and EU Delegations, on identification and 
management of recoveries) 
76. Follow-up audit of IAC recommendations in FPI (IAC audit on the 
management of the industrialised countries instrument by FPI HQ and the 
Tokyo and Washington EU Delegations) 
77. Follow-up audit of IAC recommendations in DG NEAR (IAC audits on 
year-end accounting closure and on cross-border-cooperation) 
78. Follow-up audit of IAC recommendations in DG EAC (IAC audit on 
organisation, processes and procedures of the HR function) 
79. Follow-up audit on Limited review of the calculation and the underlying 
methodology of the residual error rate for 2014 in DG EAC 
80. 3rd Follow-up audit of the lifelong learning programme in EACEA/DG 
EAC 
81. Follow-up audit of IAC recommendations in EACEA (IAC audit on the 
ERASMUS MUNDUS II programme and the intra-ACP academic mobility 
scheme) 

119 

 
 
82. Follow-up audit of IAC recommendations in DG TRADE (IAC audits on 
document management, planning and risk management and on enforcement of 
trade agreements) 
83. Follow-up audit of the IAC recommendations in DG COMP (IAC audit n 
the monitoring of state aid granted) 
84. Follow-up audit of IAC recommendations in DG GROW (ex DG 
MARKT IAC audit of the stakeholders consultation process) 
85. 2nd Follow-up audit on DG MARKT's (DG FISMA's) cooperation with 
the three supervisory bodies on financial services 
86. Follow-up audit of IAC recommendations in DG FISMA (IAC audits on 
the process of managing complaints / infringements in DG MARKT and on 
DG FISMA's staff learning and development). 
87. Follow-up audit of IAC recommendations in DG TAXUD (IAC audit on 
DG TAXUD's procurement and management of studies and databases) 
88. Follow-up audit of IAC recommendations in DG BUDG (IAC audit on the 
validation of local systems by unit C3) 
89. Follow-up audit of IAC recommendations in LS (IAC audit on the 
management of court cases in the Legal Service) 
90. 2nd Follow-up audit on the administrative processes supporting the 
European semester  
91. Follow-up audit of IAC recommendations in DG SCIC (IAC audits on 
2013 year-end, financial management and internal control system in DG 
Interpretation and on the professional support provided to the interpreters) 
92. Follow-up audit of IAC recommendations in DG ESTAT (IAC audit on 
the statistical process I – GNI data) 
93. Follow-up audit on outstanding IT recommendations in DG DEVCO 
94. Follow-up audit on management and supervision of contracts for the 
outsourced IT services in the Publications Office 
95. Follow-up audit of IAC IT recommendations (IAC audits on the 
reimbursement of expert's expenses managed by the PMO, information 
security in DGT, IT project management in ECFIN and business continuity in 
DG EMPL) 

120 

 
 
 
PART 3: SUMMARY OF LONG OUTSTANDING RECOMMENDATIONS AS AT 31 JANUARY 2017 

No. 

DG 

Audit title 

Recommendation 

Comments 

Design and 
effectiveness of the 
remedial/mitigating 
measures at contract 
level 

The IAS carried out a follow up audit in January 2017 which concluded that 
the recommendation is only partially implemented and will be re-opened. In 
order  to  fully  implement  this  recommendation,  DG  DEVCO  has  to  redesign 
the TA expert pool contract, which is still ongoing. 

Expected delay compared to the original target date of 6 months. 

I 

DEVCO 

II 

ENER 

IAS Audit on 
management of 
the African Peace 
Facility (APF) 

IAS Audit of the 
supervision of the 
implementation 
of Connecting 
Europe Facility 
(CEF) in DG ENER 
and MOVE 

DG ENER's 
supervision strategy 
on Projects of 
Common Interest 
(PCIs) development 

III 

ESTAT 

IAC Audit on 
ESTAT’s Business 
Continuity 
Management 

Disaster Recovery 
Planning and IT 
Business Continuity 

IV 

JRC 

IAC Audit on 
Nuclear 
Decommissioning 
and Waste 
Management at 
the JRC- Financial 
Aspects 

Delays in the Nuclear 
decommissioning and 
waste management 
programme and 
external uncertainties 

DG  ENER  adopted  in  September  2016  a  "Supervision  strategy  on  PCIs 
development" and declared the recommendation as implemented. In January 
2017 the IAS assessed  the  implemented mitigating measures as insufficient 
and re-opened the issue. 

The IAS follow-up will take place as soon as the recommendation is reported 
as ready for review by DG ENER. 

Expected delay compared to the original due date of 11 months. 

According  to  DG  ESTAT  all  but  one  action  points  of  this  recommendation 
have been addressed. The last remaining action shall be implemented within 
a few weeks. 

The  IAS  follow-up  will  take  place  in  the  course  of  2017  as  soon  as  the 
recommendation is reported as ready for review by DG ESTAT. 

Expected delay compared to the original due date of 1 year. 

According  to  the  JRC,  60%  of  actions  stemming  from  this  recommendation 
have  already  been 
for 
decommissioning  and  waste  management  is  ongoing  and  the  budget  is 
currently being reviewed for all four nuclear sites. 

implemented.  The  review  of 

the  Strategy 

The IAS follow-up will take place as soon as the recommendation is reported 
as ready for review by the JRC. 

Expected  delay  compared  to  the  original  due  date  of  1  year  and  1 
month. 

Final 
report date 

Original 
due date 

Revised due 
date 

21/01/2016 

30/06/2016 

15/12/2016 

(new updated 
target date to 
be confirmed 
by DG DEVCO) 

29/01/2016 

30/06/2016 

31/05/2017 

16/02/2015 

30/06/2016 

30/06/2017 

02/06/2015 

02/06/2016 

30/06/2017 

121 

 
 
 
 
 
No. 

DG 

Audit title 

Recommendation 

Comments 

V 

JRC 

IAC Audit on 
Nuclear 
Decommissioning 
and Waste 
Management at 
the JRC- Financial 
Aspects 

Relationship with 
authorities, licensing 
and insurance 

VI 

NEAR 

IAC Audit on 
Special approvals 
and derogations 

Exception reporting 

VII 

DEVCO 

IAS Audit on 
Budget Support in 
DG DEVCO 

Policy dialogue 
framework 

IAS Audit on 
Management of 
logical access to 
systems 
(ECAS/LDAP/Wind
ows) 

VIII 

DIGIT 

Security requirements 
for ECAS 

The communication strategy for improving the JRC's relationship with Italian 
authorities and other stakeholders has not been finalised yet although it has 
been under preparation since October 2015. The IAS also noted that despite 
efforts made by the JRC to improve the relations with the Italian Authorities 
and stakeholders, this part of the recommendation is outside the JRC's direct 
control.  

The IAS follow-up will take place as soon as the recommendation is reported 
as ready for review by the JRC. 

Expected  delay  compared  to  the  original  due  date  of  1  year  and  1 
month. 

In order to address this recommendation, all AOSDs were instructed to carry 
out  an  analysis  of  exceptions  and  non-compliance  events  per  directorate. 
However, no analysis has been envisaged at the central level. Therefore, the 
IAS  follow-up  in  December  2016  assessed  this  recommendation  as  not 
implemented. 

The  IAS  will  follow  up  this  recommendation  in  the  course  of  2017,  once 
reported as ready for review by DG NEAR. 

Expected  delay  compared  to  the  original  due  date  of  1  year  and  4 
months. 

In order to fully implement this recommendation DG DEVCO has to finalise 
the revised Budget Support guidelines. This has been postponed until the 
adoption of the new Consensus for Development (expected in May 2017). 

The IAS will perform a follow-up audit on this recommendation at the end of 
2017. 

Expected delay compared to the original due date of 1 year and 10 
months 

A first IAS follow-up performed in 2016 acknowledged the progress made in 
identifying the risks, drafting the security plan, prioritising the missing 
security controls and in the implementation of the action plan. However, the 
implementation of the missing security measures still has not been finalised. 
It is planned to be done by the end of June 2017. The IAS is planning a 
second follow-up of this recommendation in the second half of 2017. 

Expected delay compared to the original due date of 1 year and 9 
months. 

Final 
report date 

Original 
due date 

Revised due 
date 

02/06/2015 

02/06/2016 

30/06/2017 

29/09/2014 

31/12/2015 

30/04/2017 

11/12/2014 

31/12/2015 

31/10/2017 

24/07/2014 

30/09/2015 

30/06/2017 

122 

 
 
 
No. 

DG 

Audit title 

Recommendation 

Comments 

Final 
report date 

Original 
due date 

Revised due 
date 

IX 

FISMA 

IAS Audit on 
Effectiveness of 
HR management 
to support the 
financial crisis in 
DG ECFIN, DG 
COMP, DG MARKT  

Monitoring and 
reporting on HR 
management 

X 

GROW 

IAC Audit on the 
Internal Control 
Strategy of the 
GSA2 over the 
budget delegated 
by DG ENTR 

XI 

JRC 

IAC Audit on 
intellectual 
property rights 
management 

Prior existing 
intellectual propriety 
rights in the JRC 
publications 

(a  further  delay  of  6  months  compared  to  the  situation  in  the  previous  IAS 
report to the APC - October 2016) 

The IAS second follow-up audit carried out in September 2016 concluded 
that some improvements were made to enhance the monitoring and 
reporting process on the HR management related activities. However, these 
improvements were not sufficient to close or downgrade the rating of the 
recommendation. 

A third IAS follow-up audit is planned for the second quarter of 2017. 

Expected delay compared to the original due date of 2 years and 2 
months. 

GSA established a list of corporate risks and communicated it to the IAS in 
January 2017. Based on this risk register, GSA will now develop a risk 
management action plan and plans to follow it up by 31 December 2017. 

20/12/2013 

31/12/2014 

28/02/2017 

05/01/2015 

31/12/2015 

31/12/2017 

Expected delay compared to the original due date of 2 years. 

(a further delay of 12 months compared to the situation in the previous IAS 
report to the APC - October 2016) 

The IAS follow-up carried out in July 2016 concluded that the actions taken 
by the JRC were not in line with the original action plan or with the current 
practices applied by the scientific community. Therefore the JRC still faces 
the risk that its scientific publications may contain unauthorised intellectual 
propriety rights by third parties. 

The JRC made a self-declaration of non-infringement of intellectual propriety 
rights as an interim measure. An action plan established after the IAS follow-
up still remains to be implemented. 

The IAS will follow up this recommendation once reported as ready for 
review by the DG. 

Expected delay compared to the original due date of 3 years and 11 

30/01/2013 

30/07/2013 

30/06/2017 

Risk management 

The IAS will follow up this recommendation once reported as ready for 
review by the DG. 

2 European Global Navigation Satellite System (GNSS) Agency. 

123 

 
 
 
                                                            
No. 

DG 

Audit title 

Recommendation 

Comments 

Final 
report date 

Original 
due date 

Revised due 
date 

XII 

JRC 

IAC Audit on 
decommissioning: 
risk and project 
management at 
the ISPRA site 

Full strategy for 
guaranteeing a pool 
of qualified Project 
Leaders 

months. 

(a  further  delay  of  7  months  compared  to  the  situation  in  the  previous  IAS 
report to the APC - October 2016) 

According to the IAS follow-up carried out in September 2016, the original 
action plan was not fully implemented. Weaknesses still exist in the 
operational activities and in the monitoring of contractors related to the 
radioactive materials and assets decommissioning. The JRC accepted the risk 
until the full implementation of the recommendation and established a back-
up procedure for the project leaders on decommissioning as an interim 
measure. 

The IAS will follow up this recommendation once reported as ready for 
review by the DG. 
Expected delay compared to the original due date of 4 years. 

According to DG NEAR, the recommendation has been partially implemented. 
The outstanding actions – preparing a detailed workload assessment 
methodology and implementation for EU Delegations – will take place until 
mid-2017. 

13/06/2012 

13/03/2013 

31/03/2017 

07/05/2015 

31/12/2015 

30/06/2017 

XIII 

NEAR 

IAS Audit on 
Preparedness for 
IPA II in DG 
ELARG 

XIV 

XV 

OIB 

IAC Audit on 
Concept and 
reproduction 

HR planning for EU 
Delegations 
implementing IPA II 

The IAS will review the progress of this recommendation in the course of the 
current audit on preparedness for the mid-term review of ENI and IPA 
regulations in DG NEAR.  

A formal follow-up will take place in 2017 as soon as the recommendation is 
reported as ready for review by DG NEAR. 

Expected delay compared to the original due date of 1 year and 6 
months. 

Local information 
systems - 1 - Improve 
the Information 
Systems 

In  order  to  fully  implement  this  recommendation  OIB  needs  to  resolve 
certain difficulties in the implementation and complete installation of the new 
IT system to manage the printing requests. The new IT tool will be deployed 
in two phase by June 2017. 

Local information 
systems - 2 - Cost 
information for the 
client 

According to OIB, mitigating actions have been taken in the meantime – e.g. 
while the old IT tool is still partially being used, its reliability was improved. 
Furthermore, the new tool is already used for establishing the offers.  
The IAS will perform a follow-up in the course of 2017. 

11/11/2013 

31/12/2015 

30/06/2017 

31/12/2015 

30/06/2017 

124 

 
 
 
No. 

DG 

Audit title 

Recommendation 

Comments 

Final 
report date 

Original 
due date 

Revised due 
date 

XVI 

PMO 

IAC Audit on PMO 
management of 
accidents' 
insurance 

Reimbursement of 
accident costs 

XVII 

PMO 

IAC Audit on PMO 
contracts related 
to the 
management of 
missions 

CAF implementation 

XVIII 

SCIC 

IAC Audit on 
technical support 
provided to 
meetings and 
conferences 

Management tools 

Expected  delay  compared  to  the  original  due  date  of  1  year  and  6 
months 

(a  further  delay  of  7  months  compared  to  the  situation  in  the  previous  IAS 
report to the APC - October 2016) 

The  implementation  of  the  recommendation  is  dependent  on  several  IT 
modules (including ASMAL 2 being under development by DIGIT since Dec. 
2016),  which  shall  be  in  place  by  mid-2017.  Some  ad  hoc  measures  have 
been taken in the meantime. The IAS follow-up  will be performed once the 
recommendation has been assessed as implemented by PMO. 

Expected delay compared to the original due date of 4 years and 6 
months. 

A  first  follow-up  revealed  that  some  actions  have  been  implemented. 
However, the implementation of the remaining actions relates to an on-going 
Commission-wide  IT  development  and  for  which  PMO  is  dependent  on  DG 
DIGIT. The IAS will perform a follow up in the course of 2017. 

Expected delay compared to the original due date of 2 years and 9 
months. 

(a  further  delay  of  6  months  compared  to  the  situation  in  the  previous  IAS 
report to the APC - October 2016) 

A  follow-up  carried  out  by  the  IAS  in  the  summer  of  2016  confirmed  the 
validity of the original risks and the recommendation. However, the IAS also 
acknowledged  the  significant  changes  brought  about  by  the  Synergies  and 
Efficiency Review and their impact on the original action plan and the target 
dates, which had been superseded. Subsequently, DG SCIC prepared a new 
Action Plan and the new deadline was set at 31 December 2017. 

The  IAS  will  follow  up  this  recommendation  once  reported  as  ready  for 
review by the DG. 

Expected delay compared to the original due date of 3 years. 

21/02/2012 

31/12/2012 

30/06/2017 

25/10/2012 

30/09/2014 

30/06/2017 

25/06/2013 

31/12/2014 

31/12/2017 

125