COMMISSION STAFF WORKING DOCUMENT on the ex post evaluation of the " Prevention, Preparedness and Consequence Management of Terrorism and other Security related risks" 2007-2013 Programme (CIPS) Accompanying the document Report from the European Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions ex post evaluation for the period 2007 to 2013 of actions financed by the "Prevention and fight against crime" programme (ISEC) and the " Prevention, Preparedness and Consequence Management of Terrorism and other Security related risks" programme (CIPS)
Date: 2018-06-12
Institutions: Directorate-General for Migration and Home Affairs, European Commission
Subjects: EU programme, area of freedom, security and justice, cross-border cooperation, protection of freedoms, terrorism

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 12.6.2018  SWD(2018) 331 final  COMMISSION STAFF WORKING DOCUMENT on the ex post evaluation of the " Prevention, Preparedness and Consequence Management of Terrorism and other Security related risks" 2007-2013 Programme (CIPS) Accompanying the document Report from the European Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions  ex post evaluation for the period 2007 to 2013 of actions financed by the "Prevention and fight against crime" programme (ISEC) and the " Prevention, Preparedness and Consequence Management of Terrorism and other Security related risks" programme (CIPS)  {COM(2018) 455 final} - {SWD(2018) 332 final}  
 
 
Contents 

1 

Introduction ................................................................................................................................... 2 

2  Background to the initiative ................................................................................................ 3 

2.1 

Policy context .................................................................................................................. 3 

2.2  Baseline ........................................................................................................................... 6 

2.3 

The CIPS Programme as funding instrument .................................................................. 6 

3  Evaluation questions ......................................................................................................... 10 

4  Method .............................................................................................................................. 10 

5  Limitations ........................................................................................................................ 10 

6  Status of the Implementation ............................................................................................ 12 

7  Answers to the evaluation questions ................................................................................. 18 

7.1 Relevance ........................................................................................................................... 18 

7.2 Effectiveness ...................................................................................................................... 21 

7.3 Efficiency ........................................................................................................................... 27 

7.4 Coherence ........................................................................................................................... 32 

7.5 EU added value .................................................................................................................. 34 

8  CONCLUSIONS ............................................................................................................... 36 

ANNEX 1 — Procedural information ...................................................................................... 41 

ANNEX 2 — Stakeholders’ consultation ................................................................................ 42 

ANNEX 3 — Methodology ..................................................................................................... 48 

ANNEX 4 — List of evaluation questions ............................................................................... 51 

ANNEX 5 — List of abbreviations and country codes ............................................................ 53 

ANNEX 6 — The role of the Joint Research Centre within CIPS .......................................... 55 

ANNEX 7 — Sectoral analysis of CIPS projects .................................................................... 57 

ANNEX 8 — The different dimensions of CIPS ..................................................................... 60 

ANNEX 9 — Intervention logic of the CIPS Programme ....................................................... 62 

1 

 
 
 
 
 
 
1 

INTRODUCTION 

The  2007-2013  ‘Prevention,  Preparedness  and  Consequence  Management  of  Terrorism  and 
other  Security-Related  Risks’  programme  (CIPS)  was  established  by  Decision  No 
2007/124/EC1  and  is  one  of  the  two  financial  programmes2  falling  under  the  framework 
programme on ‘Security and Safeguarding Liberties’. Its total budget amounted to EUR 126.8 
million.  The  other  component  of  this  framework  programme  is  the  ‘Prevention  and  Fight 
against Crime,’ which is covered in a separate report. 

The  CIPS  programme  concerns  critical  infrastructure  and  other  security  issues,  including 
operational issues in areas such as crisis management, environment, public health, transport, 
research and technological development. Its general objective was to support Member States’ 
efforts  to  prevent,  prepare  for  and  to  protect  people  and  critical  infrastructure  from  terrorist 
attacks and other security-related incidents. 

The  Decision  establishing  the  CIPS  programme  requires  the  Commission  to  submit  to  the 
European Parliament and the Council three evaluations reports: 

  an annual presentation on the implementation of the programme3; 

  an interim evaluation report on the results obtained and the qualitative and quantitative 

aspects of the implementation of the programme by 31 March 20104; 

  A communication on the continuation of the programme by 31 December 20105; 

  An ex post evaluation report by 31 March 20156. 

This staff working document reports on the ex post evaluation for the evaluation of the CIPS 
programme  2007-2013  annual  work  programmes  implemented  in  the  participating  Member 

1   Council Decision of 12 February 2007 establishing for the period 2007 to 2013, as part of the Framework 
Programme on Security and Safeguarding Liberties, the Specific Programme ‘Prevention, Preparedness and 
Consequence Management of Terrorism and other Security-related risks’. 

2   Decision 2007/125/JHA established the Specific Programme ‘Prevention and Fight against Crime’ (ISEC) 
for  the  same  period,  namely  2007  to  2013.  The  ex-post  evaluation  of  this  programme  is  covered  in  a 
separate report. 

3   The state of play of the annual work programmes was presented at CIPS Committee meetings twice a year. 
4   See  European  Commission  (2010)  Evaluation  of  ‘Prevention  and  Fight  against  Crime’  and  ‘Prevention, 
Preparedness and Consequence Management of Terrorism and other Security Related Risks’ Programs  — 
JLS/2010/ISEC-CIPS/001-F4. 

5   Communication  from  the  Commission  to  the  European  Parliament  and  the  Council  on  the  mid-term 
evaluation  of  the  framework  programme  ‘Security  and  Safeguarding  Liberties’  (2007-2013),  COM(2011) 
318 final. 

6   Due to the internal reorganisation of DG HOME, the reporting on this ex post evaluation was postponed to 

2018. 

2 

 
 
 
 
                                                            
States,  based  on  the  findings  of  an  ex  post  evaluation  study  carried  out  by  an  external 
consultant and building on the previous evaluations mentioned above7. 

The ex post evaluation assessed the CIPS programme on the basis of the following criteria: 

 
relevance (whether its objectives met policy needs);  
  effectiveness (whether the objectives had been achieved);  
  efficiency (whether the costs were proportionate to the benefits gained);  
  coherence (between the actions financed by the instrument); and  
  added value (of intervening at EU level). 

The interim evaluation of the Internal Security Fund (‘the Fund’)8, its police component being 
the  successor  of  the  ISEC  and  CIPS  programmes  2014-2020,  takes  into  account  the 
conclusions  and  findings  of    this  evaluation,  assessing  what  has  already  been  implemented 
under the ISF-Police and what needs to be further addressed in the next Multiannual Financial 
Framework post-2020. The interim evaluation of the Fund is due in 20189. The Commission 
will  submit  an  interim  evaluation  report  on  the  Fund  by  30 June  2018  to  the  European 
Parliament, the Council, the European Economic and Social Committee and the Committee of 
the  Regions.  The  results  of  the  interim  evaluation  together  with  the  results  of  this  ex  post 
evaluation  will  contribute  to  shaping  future  EU  migration  and  home  affairs  policies,  and  in 
particular in preparing the new funding instruments for the Multiannual Financial Framework 
post-2020. 

2  BACKGROUND TO THE INITIATIVE 

2.1  POLICY CONTEXT 

This  section  introduces  the  EU  policy  context  of  the  two  main  focus  areas  under  the  CIPS 
programme, namely: (i) protection of critical infrastructure and (ii) terrorism. 

7   Austria, Belgium, Bulgaria, Cyprus, Croatia, Czech Republic, Denmark, Estonia, Finland, France, Germany, 
Greece,  Hungary,  Ireland,  Italy,  Latvia,  Lithuania,  Luxembourg,  Malta,  Netherlands,  Poland,  Portugal, 
Romania, Slovakia, Slovenia, Spain, Sweden, United Kingdom, and Croatia as of 2013. 

8     Regulation (EU) No 513/2014 of the European Parliament and of the Council of 16 April 2014 establishing, 
as part of the Internal Security Fund, the instrument for financial support for police cooperation, preventing 
and combating crime, and crisis management and repealing Council Decision No 2007/125/JHA. 

9   Regulation (EU) No 514/2014 of the European Parliament and of the Council of 16 April 2014 laying down 
general  provisions  on  the  Asylum,  Migration  and  Integration  Fund  and  on  the  instrument  for  financial 
support  for  police  cooperation,  preventing  and  combating  crime,  and  crisis  management,  OJ  L  150, 
20.5.2014, p. 112-142, Article 57. Part of the interim evaluation is the mid-term review of the ISF national 
programmes of the Member States (Regulation (EU) No 515/2014, Article 8) which will take place in 2017 
and 2018.  The  purpose  of  the  mid-term  review  exercise  is for  the  Commission  and  the  Member  States  to 
review their national programmes and assess the need for a possible revision of the programme, in the light 
of  developments  in  Union  and  national  policies  through  a  questionnaire  and  bilateral  dialogues  with  the 
Member States. In addition, if the need arises, the results of the mid-term review of the national programmes 
may support requests for additional funding made by the Commission to the budgetary authorities for the 
remaining implementation period. 

3 

 
 
 
 
 
                                                            
On  the  basis  of  the  conclusions  adopted  in  1999  by  the  Tampere  European  Council  in 
November  2004,  the  European  Council  reaffirmed  its  promotion  of  freedom,  security  and 
justice through the prevention of and fight against crime. This led to the Hague programme 
(2004-2009)10  which  focused  on  strengthening  security  in  the  EU’s  fight  against  terrorism, 
recruitment, financing, protecting critical infrastructures and developing a sound consequence 
the  2010-2015  Stockholm  programme11, 
management 
complemented by an action plan, led to the development of an EU Internal Security strategy, 
underlining the need to fight against terrorism.  

framework.  More 

recently, 

i) Protection of critical infrastructures 

Infrastructures  are  considered  ‘critical’  when  their  disruption  could  have  an  impact  on  the 
functioning  of  a  society  (in  terms  of  economy,  security  and  people’s  wellbeing)12.  Critical 
infrastructure protection is linked to terrorism as well as to chemical, biological, radiological, 
nuclear, and explosives (CBRN-E) threat and cyber security. However, critical infrastructures 
need to be protected also from man-made and natural disasters. Furthermore, these are highly 
interdependent:  the  disruption  of  one  of  them  might  cause  what  is  known  as  the  ‘domino 
effect’, with potentially broader consequences and multiplied impacts13. 

The  EU  has  been  active  in  critical  infrastructure  protection  since  2004,  although  protecting 
critical infrastructures is a Member State competence (while being highly interdependent). 

On the critical infrastructure protection legal framework, the main pieces of legislation are:  

i) the 2006 European Programme for Critical Infrastructure Protection (EPCIP)14;  

ii)  the  Council  Directive  2008/114/EC  of  8 December  2008  on  the  identification  and 
designation  of  European  critical  infrastructures  and  the  assessment  of  the  need  to  improve 
their protection15; and  

iii) Commission Staff Working Document on a new approach to the European Programme for 
Critical Infrastructure Protection16, which is the result of a review of the Critical Infrastructure 
Protection approach presented in 201317 and heavily focuses on interdependencies of critical 
infrastructure protection across sectors. 

The  framework  established  by  the  EPCIP  provides  procedures  to  identify  and  designate 
European critical infrastructures, and to carry out a threat assessment, introduced by Directive 

10   See https://ec.europa.eu/home-affairs/what-we-do/policies/crisis-and-terrorism/critical-infrastructure_en 
11   See  Liquefied  natural  gas  —  LNG  terminal  is  a  facility  for  regasifyng  of  LNG  (definition  from 

https://www.elengy.com/en/lng/what-is-an-lng-terminal.html). 

12   Council  Directive  2008/114/EC  of  8 December  2008  on  the  identification  and  designation  of  European 

critical infrastructures and the assessment of the need to improve their protection. 

13   Anne  van  Aaken,  Isabelle  Wildhaber,  Symposium  on  Critical  Infrastructures:  risk,  responsibility  and 

liability. Available at http://ejrr.lexxion.eu/data/article/7933/pdf/ejrr_2015_02-004.pdf 
14   http://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=URISERV:l33260&from=EN 
15   http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32008L0114&from=EN 
16

  See   

http://ec.europa.eu/energy/sites/ener/files/documents/20130828_epcip_commission_staff_working_docume
nt.pdf 

17   https://ec.europa.eu/home-affairs/sites/homeaffairs/files/what-we-do/policies/crisis-and-terrorism/critical-

infrastructure/docs/swd_2013_318_on_epcip_en.pdf 

4 

 
 
                                                            
2008/114/EC.  The  Critical  Infrastructure  Warning  Information  Network18  and  the  critical 
infrastructure protection expert groups at EU level19 are also included. The general objective 
of  this  framework  is  to  enable  the  implementation  of  EPCIP,  information  exchanges  and  to 
protect critical infrastructures. 
Furthermore,  the  Council20  identified  two  main  sectors  of  critical  infrastructure:  energy  and 
transport. 

On the energy sector, European critical infrastructures include those linked to subsectors like 
electricity (infrastructures and facilities to generate and supply electricity), oil (oil production, 
refining, treatment, storage and transmission by pipelines) and gas (gas production, refining, 
treatment, storage and transmission by pipelines, LNG21 terminals). 

Within the meaning of transport sector, road, rail, air, waterways as well as ocean and short-
sea shipping and ports are included. 

The EU has also been active in the protection of CBRN materials, in particular those already 
present  in  the  EU  soil,  e.g.  stored  in  national  plants  although  not  included  in  the  sectors 
regulated by Directive 2008/114/EC. For this reason, the EU adopted a CBRN action plan in 
2009, as a main outcome of the CBRN Task Force that was set up in 2008. The main goal of 
the action plan is to minimise the threat and damage of CBRN incidents, by promoting a risk-
based approach to security.  

ii) Terrorism 

Since 2007 Europol registered a decrease in the total number of completed, foiled and failed 
attacks.  That  said  as  from  201322  a  slight  increase  has  been  observed.  Over  the  evaluation 
period, the terrorism threat has significantly evolved and included, although in a smaller scale, 
also left-wing, separatist, anarchist and right wing extremism. More specifically, the EU has 
witnessed  a  number  of  religiously-inspired  terrorist  attacks  as  from  2011.  Indeed,  the  last 
years  of  the  period  under  evaluation  and  those  immediately  subsequent  saw  a  significant 
change  in  the  terrorism  phenomenon,  in  particular  with  the  continuation  of  the  civil  war  in 
Syria and the rise of the Islamic State in Iraq and Syria (ISIS). 

18   The CIWIN serves both as internet portal to exchange ideas, studies and good practices in the field and as a 
repository  for  CIP  information.  Indeed,  one  of  the  main  challenges  in  the  field  of  critical  infrastructure 
protection  is  the  non-optimal  partnership  between  public  and  private  sectors  in  a  field  where  private 
companies  play  a  key  role.  Please  see  https://ec.europa.eu/home-affairs/what-we-do/policies/crisis-and-
terrorism/critical-infrastructure_en 

19   http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2006:0786:FIN:EN:PDF 
20   Council Directive No 2008/114/EC. 
21   Liquefied  natural  gas  —  LNG  terminal  is  a  facility  for  regasifyng  of  LNG  (definition  from 

https://www.elengy.com/en/lng/what-is-an-lng-terminal.html). 

22   Europol, EU Terrorism Situation & Trend Report (Te-Sat), 2014, https://www.europol.europa.eu/activities-

services/main-reports/te-sat-2014-eu-terrorism-situation-and-trend-report 

5 

 
 
                                                            
At  the  EU  level,  a  Council  Framework  Decision  on  combating  terrorism23  was  adopted  in 
2002, followed three years later by the EU Counter-Terrorism strategy24 that was made up of 
four parts: (i) prevent, (ii) protect, (iii) pursue and (iv) respond.  

Furthermore, the EU internal security strategy in action (EU ISS) was adopted for the 2011-
201425 period. The EU internal security strategy identified five strategic objectives, including 
the objective of prevention of terrorism and prevention of radicalisation and recruitment. To 
prevent  terrorism,  the  EU  committed  to  cut  off  terrorists’  access  to  funding  and  non-
conventional materials (CBRN) and strengthen the protection of aviation, land and maritime 
transport systems.  

2.2  BASELINE 

In  order  to  fully  identify  the  contributions  the  funding  made  to  the  different  CIPS  areas  it 
would  have  been  desirable  to  establish  a  baseline,  reflected  in  a  set  of  indicators,  including 
data  on  national  spending  in  the  various  areas.  However,  as  explained  in  the  limitations 
section, such baseline is lacking for the CIPS Programme and it has not been possible within 
the  scope  of  this  evaluation  to  develop  a  detailed  baseline  covering  the  situation  before  the 
start of the Programme. 

2.3  THE CIPS PROGRAMME AS FUNDING INSTRUMENT 

As  mentioned  above,  one  of  the  general  objectives  of  the  CIPS  programme  was  to  support 
Member States’ effort to prevent, prepare for, and to protect people and critical infrastructure 
against  terrorist  attacks  and other  security-related incidents (prevention and  preparedness, 
general objective 1)26. 

Together  with  prevention  and  preparedness,  the  programme  also  sought  to  ensure 
consequence  management  (post-attack),  namely  protection  in  the  areas  such  as  the  crisis 
management, environment,  public health,  transport, research and technological development 
and economic and social cohesion, in terrorism and other security-related risks in the area of 
freedom, security and justice (consequence management, general objective 2)27.  

Specifically, the CIPS programme was particularly focused on critical infrastructures which is 
defined  as:  ‘those  physical  resources,  services,  information  technology  facilities,  networks 
and infrastructure assets which, if disrupted or destroyed, would have a serious impact on the 

23   http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32002F0475&from=en 
24   http://register.consilium.europa.eu/doc/srv?l=EN&f=ST%2014469 %202005 %20REV%204 
25   http://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=URISERV:jl0050&from=EN 
26   Council Decision 2007/124/EC, Euratom, February 2007, Art. 3.1. 
27   Council Decision 2007/124/EC, Euratom, February 2007, Art. 3.2. 

6 

 
 
 
 
 
 
                                                            
critical  societal  functions,  including  the  supply  chain,  health,  safety,  security,  economic  or 
social wellbeing of people or of the functioning of the Community or its Member States’28. 

To  achieve  its  overarching  objectives,  the  CIPS  programme  was  designed  to  contribute  to 
achieving seven specific objectives (the first five specific objectives are linked to the general 
objective 1 whereas the last two are linked to the general objective 2)29: 

1.  Stimulating,  promoting,  and  supporting  risk  assessments  on  critical 

infrastructure, in order to upgrade security. 

2.  Stimulating, promoting, and supporting the development of methodologies 
for  the  protection  of  critical  infrastructure,  in  particular  risk  assessment 
methodologies. 

3.  Promoting and supporting shared operational measures to improve security 
in  cross-border  supply  chains,  provided  that  single  market  competition 
rules are not distorted. 

4.  Promoting  and  supporting  the  development  of  security  standards,  and  an 
exchange  of  know-how  and  experience  of  protecting  people  and  critical 
infrastructure. 

5.  Promoting and supporting Community wide coordination and cooperation 

on protection of critical infrastructure. 

6.  Stimulating, promoting and supporting exchange of know-how to set best 
practice  in  order  to  coordinate  the  response  measures  and  to  achieve 
cooperation  between  actors  involved  in  crisis  management  and  security 
actions. 

7.  Promoting  joint  exercises  and  practical  scenarios  including  security  and 
safety  components,  in  order  to  enhance  coordination  and  cooperation 
between relevant actors at the European level. 

In terms of thematic areas, the CIPS programme covered the following: 

1.  crisis management, and 
2.  terrorism  and  other  security-related  risks  within  the  area  of  freedom, 
security  and  justice,  including  risks  relating  to  the  environment,  public 
health,  transport,  research  and  technological  development,  and  economic 
and social cohesion. 

The intervention logic of the CIPS Programme is presented in Annex 9.  

Implementation through Direct Management 

The CIPS programme has been implemented through actions under the 2007-2013 
annual work programmes30 (AWPs), which identified main sectors such as energy, 

28   Council Decision 2007/124/EC, Euratom, February 2007, Art.2 (c). 
29   Council Decision 2007/124/EC, Euratom, February 2007, Art.4. 

7 

 
 
 
 
 
 
 
                                                            
nuclear industry, ICT, water, food, health, financial, transport, chemical industry, 
space  and  research  facilities31.  The  projects  and  their  impact  according  to  this 
sectoral dimension are further discussed under the section on effectiveness and in 
Annex 7. 

EU  financial  support  for  CIPS  was  implemented  under  the  direct  management 
mode  through  projects  supported  by  Commission  grants,  contracts  for  services 
concluded  following  calls  for  tenders  published  by  the  Commission  or  through 
administrative arrangements with the Joint Research Centre (JRC)32. 

Through  direct  management,  the  Commission  retained  full  responsibility  for 
implementing  and  carrying  out  all  the  programming  and  operational  work. 
Political  priorities  and  objectives  for  projects  were  defined  in  the  annual  work 
programmes  adopted  each  year,  and  calls  for  proposals  were  launched  following 
the  approval  of  each  annual  work  programme.  Member  States  were  involved  in 
approving the annual work programmes and could provide advice on the selection 
of  projects  to  be  funded,  but  they  were  not  involved  in  the  management  of  the 
programme. 

Procurement contracts, as well as grants to standardisation bodies awarded without 
a  call  for  proposal,  i.e  to  the  European  Committee  for  Standardisation  (CEN), 
could  also  have  been  implemented  under  CIPS.  More  precisely,  the  funding 
available under the CIPS programme was awarded as follows: 

1.  Action grants through open calls for proposals (76.5 % of total 

funding). 

2.  Grants  to  standardisation  bodies  awarded  without  a  call  for  proposals 
— i.e. to the European Committee for Standardisation (CEN) (0.6 %). 

3.  Public procurement (8.5 %). 
4.  The CIPS programme specific actions to be carried out with the JRC 

(14.4 %). 

On the types of actors who could have applied for CIPS funding, according to the 
legal  basis33,  these  included  but  were  not  limited  to  public  bodies  (national, 
regional  and  local),  private  sector,  universities  and  research  institutes.  The  most 
common  coordinating  organisations  were  universities  or  research  organisations 
followed by private companies, which played a key role in CIP. 

Financial  support  under  the  CIPS  programme  was  provided  for  the  following 
types of actions34: 

30   Political  priorities  and  objectives  for  projects  are  defined  in  the  Commission’s  annual  work  programmes 

(AWPs) adopted each year and calls for proposals are launched following the approval of each AWP. 
31   https://ec.europa.eu/home-affairs/financing/fundings/security-and-safeguarding-liberties/terrorism-and-

other-risks_en 

32   More details on the role of the JRC within CIPS are reported in Annex 7. 
33   Article 6 of Decision No 2007/ 124/ EC, Euratom. 
34   Council Decision 2007/124/EC, Euratom, February 2007, Art. 5. 

8 

 
 
 
 
 
                                                                                                                                                                                          
1.  Operational  cooperation  and  coordination  (strengthening  networking, 
mutual  confidence  and understanding, development of contingency  plans, 
exchange and dissemination of information, experience and best practice). 

2.  Analytical, monitoring, evaluation and audit activities. 
3.  Development  and  transfer  of  technology  and  methodology,  particularly 

regarding information sharing and inter-operability. 

4.  Training, exchange of staff and experts. 
5.  Awareness and dissemination activities. 

As  part  of  the  evaluation,  categories  of  activities  were  defined  to  ensure  a 
coherent classification according to the objectives of the action under CIPS. 

All  action  grants  were  subsequently  allocated  to  one  of  the  categories.  More 
specifically,  a  grant  could  have  been  used  to  implement  the  activities  in  two  or 
more  categories.  However,  every  grant  was  classified  under  the  category  which 
represented the dominant element. 

The  categories  of  activities  are  the  following  (including  the  number  of  action 
grants implemented under each category): 

1.  Development  of  tools35    —  risk  assessment,  including  projects  aimed  at 
developing  tools,  instruments  and  methodologies  for  the  assessment  or 
risks and vulnerabilities of a critical infrastructure or of a security-related 
event (37). 

2.  Development of tools36   — Protection, including projects which aimed at 
developing  tools,  instruments  and  methodologies  to  better  protect  critical 
infrastructure or guarantee security at a particular sensitive event (33). 
3.  Cooperation  and  partnerships,  including  projects  aimed  at  improving 
cooperation and  partnerships  between  international  stakeholders,  Member 
States  authorities,  and  between  the  public  and  private  sectors.  This 
category  might  also  include  best  practice  manuals  targeting  authorities  in 
the Member States (27). 

4.  Response,  including  projects  aimed  at  improving  the  response  to  terrorist 
risks  and 
incidents 

thus  minimising 

security-related 

attacks  or 
consequences (14). 

5.  Training,  exercises  and  simulations  including  projects  which  envisaged  a 
‘live’ component or a live simulation of a security-related incident (15).   

35   On the basis of the scoping interview with a representative of the JRC, it was noted that ‘development’ does 
not  always  constitute  a  proper  development  of  something  new  but  rather  it  might  be  a  re-deployment  of 
existing tools to assess their feasibility and quality into the domain of the critical infrastructure protection. 
Therefore, projects in this category might refer to activities of ‘deployment and assessment’ and/or include a 
reference to ‘pilot projects’. 

36   Ibidem. 

9 

 
 
 
 
 
 
 
 
 
                                                            
3  EVALUATION QUESTIONS 

The  evaluation  was  planned  and  tendered  out  before  the  Better  Regulation  guidelines  were 
adopted on 19 May 2015, but still covered all the five key evaluation criteria required by the 
guidelines i.e. effectiveness, efficiency, relevance, coherence and EU added value. To assess 
these criteria, the evaluation reviewed the 10 questions that are in Annex 4. 

4  METHOD 

The  Commission’s  evaluation  relies  on  an  external  study  carried  out  between  August  2016 
and  August  2017  by  a  consultancy  specialised  in  carrying  out  evaluations.  The  study’s 
methodology  combined  desk  research,  and  both  qualitative  and  quantitative  analysis.  It 
required a systemic synthesis of the evidence on the implementation of the CIPS programme. 
Information was drawn mainly from various primary and secondary sources, such as: 

Primary  sources: 

three  online  surveys 

• 
(coordinating  organisations,  partner 
organisations  and  project  participants);  stakeholder  consultations  (37  face-to-face  and 
telephone interviews  were conducted with five stakeholders types; thematic case studies (2); 
analysis  of  public  consultation  responses  (6  responses);  expert  workshops  (2)  and  expert 
input. 

• 
Secondary  sources:  quantitative  data  on  all  CIPS  projects  provided  by  the 
Commission; in-depth analysis of a sample of projects (29 projects, representing 28 % of all 
104 finalised projects); a literature review of EU policy baseline and context and a literature 
review of national baseline and context. 

A more detailed overview of the methodology used for the evaluation study is in Annex 3. 

Information was triangulated when possible to ensure validity and robustness. Findings from 
the study are presented in this report together with the financial data extracted from ABAC37. 

5  LIMITATIONS 

The  evaluation  process  encountered  a  number  of  difficulties  that  induced  methodological 
limitations  and  hindered  the  extent  to  which  findings  and  conclusions  could  have  been 
formulated. A summary of these limitations is reported in this section and more details can be 
found in Annex 3. 

The lack of a baseline 

37   ABAC is the Commission’s accrual-based online accounting system. 

10 

 
 
 
 
                                                            
The  first  major  obstacle  encountered  in  the  evaluation  was  that  it  has  not  been  possible  to 
establish a baseline (i.e. the situation before the intervention of the programme) for measuring 
the  level  of  prevention,  preparedness  and  coordination  of  crisis  management  and  security 
actions  in  the  aftermath  of  a  terrorist  attack  or  of  security  incidents  across  the  EU  due  to  a 
lack of ex ante targets to measure results against.  

Furthermore, no methodologies were adopted for measuring in a comprehensive manner most 
of  the  activities  covered  by  the  actions  funded  through  CIPS.  Consequently,  no  robust 
conclusions could be drawn on the situation before the intervention started (i.e. the baseline) 
for the different thematic areas covered by the programme and the potential effect the funding 
has  had  on  these  compared  to  the  situation  after  the  programme  had  been  implemented.  In 
addition,  differences  in  the  definitions  and  content  of  critical  infrastructures  across  the  EU, 
including  their  interdependencies,  made  setting  a  common  EU  baseline  even  more 
problematic.  Therefore,  effectiveness  has  been  mostly  evaluated  in  terms  of  outputs  or 
perceived  impacts  or  results–  which  were  more  easily  measured  and  compared  across 
Member States — rather than real impacts or results.  

However,  to  partly  overcome  these  limitations,  where  this  was  relevant,  the  findings  and 
conclusions  in  the  ex  post  evaluation  have  been  compared  to  the  findings  and  conclusions 
formulated  during  the  interim  evaluation  of  ISEC  and  CIPS  and  where  it  concerned  similar 
evaluation  questions.  Given  the  lack  of  an  existing  baseline,  the  evaluation  team  had 
developed a set of indicators during the inception phase in order to map out where possible 
the national and EU context for the different thematic areas covered by CIPS and to identify 
trends  in  those  data  before,  during  and  after  the  programming  period.  However,  the  team 
encountered significant data issues when trying to populate the indicators and it was agreed 
with  the  Steering  Group  that  due  to  the  lack  of  (comparative)  data,  this  information  would 
only  be  used  for  illustrative  purposes  and  to  focus  on  presenting  the  EU  context  for  the 
different policy areas (see Section 2). 

Time lag between programme implementation and the ex post evaluation 

There was a time lag between the implementation of the Programme (2007-2013) and the ex 
post  evaluation  (August  2016-August  2017).  This  meant  that  the  relevant  beneficiaries  and 
national  and  EU  authorities  and  experts  were  often  no  longer  working  in  the  same  position 
and  were  therefore  difficult  to  reach.  This  is  reflected  in  a  low  response  rate  to  the  online 
survey for coordinating and partner organisations as  well  as to  the public consultations (see 
Annex 3). To try to mitigate these difficulties, the external consultancy took some measures, 
such  as  sending  several  email  reminders  to  the  people  surveyed,  searching  for  alternative 
contacts  when  possible,  raising  the  individual  partner  organisation  interviews  to  two  rather 
than  one,  and  extending  the  deadline  for  the  online  survey  carried  out  as  part  of  the  case 
studies in order to boost the number of responses. Overall, all evaluation questions have been 
answered to some extent, thanks to the use of triangulation of data where possible. However, 
where the evidence base has been deemed to be too low to be reliable, this has been indicated. 

11 

 
 
Gaps in financial data  

The calculations on the absorption  rate included all  amounts in  ABAC until  31 March 2017 
although  a  number  of  projects  were  not  fully  completed  by  that  date.  For  this  reason, 
absorption rates and trends, in particular for the last programming year, should be interpreted 
with caution since final amounts may be subject to variations on the basis of final payment for 
the non-completed projects. To try to mitigate this limitation, the costs have been estimated as 
accurately as possible, but have to be interpreted with caution. However, the absorption rate 
was not expected to change significantly.  As no estimates were available for JRC data, it has 
not been possible to analyse the absorption rate for the JRC part of the procurement sector.  

Limitations on monitoring the results and impacts of the programme 

Issues were noted when monitoring the progress of the project and the final results hindered 
the  assessment  of  effectiveness,  efficiency  and  EU  added  value.  The  absence  of  a  central 
repository to store the CIPS findings led to a variety of definitions of results and impacts of 
the  programme  depending  on  the  thematic  area.  Therefore  the  possibility  to  compare  the 
results across different thematic areas was restricted. However, to mitigate this limitation to 
the  extent  possible,  data  has  been  compiled  and  triangulated  from  multiple  sources 
(interviews, surveys, sampled projects).  

6  STATUS OF THE IMPLEMENTATION 

As  set  out  in  the  annual  work  programme  2007-2013,  a  total  budget  of  EUR 126.8  million 
was allocated to the CIPS programme, of which EUR 74.6 million was committed. 

The minimum EU co-financing increased from EUR 50 000 (2007-2009) to EUR 100 000 in 
2010  and  remained  at  EUR  100 000  from  2011  onwards.  The  maximum  rate  of  the EU  co-
financing increased consistently with the raise in minimum EU grant, from 70 % (2007-2009) 
to 80 % in 2010 and stayed at 90 % from 2011 onwards for the remainder of the programme 
implementation period. 

In  total,  126  CIPS  grants  were  implemented  across  the  programming  period,  of  which  121 
were  action  grants  to  open  calls  for  proposals  and  five  were  direct  grants  to  European 
Committee for Standardisation. The number of grants implemented per year remained largely 
the same throughout 2007-2013 (between 17 and 20), with a rise registered in 2009 (27). 

More  action  grants  were  funded  via  open  calls  (76.5 %),  followed  by  14.4 %  allocated  to 
administrative  arrangements  with  the  JRC.  Less  than  8.5 %  was  allocated  to  the  remaining 
public procurement and less than 1 % to grants to European Committee for Standardisation. 

12 

 
 
 
The number of projects implemented varied across the programme years38. For instance, at the 
start  of  the  programme,  in  2007,  it  was  quite  low  (17  projects)  but  increased  in  2009  to  27 
projects. 

The  demand-driven  design  of  the  annual  work  programme  (see  section  7.1)  resulted  in  a 
significant geographical imbalance in terms of the number of projects funded per coordinating 
organisation in each Member State. While organisations from Italy and Spain coordinated 48 
projects  and  23  projects  respectively,  organisations  from  15  Member  States  (UK,  BE,  NL, 
DE,  AT,  CZ,  PO,  HU,  RO,  BG,  EL,  CY,  SE,  EE,  SI)  coordinated  less  than  eight  projects, 
with  Slovenia,  Cyprus  and  Estonia  at  the  bottom  of  the  scale.  These  countries  only 
coordinated one project  each. Conversely, a slightly more balanced spread  was achieved by 
the  partner  organisations,  with  five  or  more  partner  organisations  located  in  21  Member 
States.  

The  majority  of  grants  were  cross-border  (73 %  or  88)  as  national  grants  represented  only 
27 %  (or  33).  Italy  coordinated  the  highest  number  of  national  projects  (15),  followed  by 
Spain  (5).  From  the  evaluation  analysis  it  emerged  that,  overall,  only  11  Member  States 
coordinated  at  least  one  national  project.  However,  this  irregular  geographical  spread  of 
coordinating organisations was counterbalanced by the location of the partner organisations. 
Indeed, a total of 86 national partner organisations were involved across these projects; while 
Italy  had  the  highest  number  of  partner  organisations  involved  overall  (47  organisations 
across 15 projects), Romania had the highest number of  partner organisations involved in one 
grant (9 organisations). 

Projects  were  considered  as  cross-border  if  they  included  at  least  one  partner  organisation 
based in a different Member States than the one where the leading organisation was based. In 
terms  of  the  geographical  distribution  of  cross-border  projects,  coordinating  organisations 
were based in 16 different Member States. The highest number of cross-border projects was 
hosted  by  Italy  (33  projects  or  36 %),  which  is  consistent  with  the  over-representation  of 
coordinating  organisations  located  in  that  Member  State  within  the  whole  programme, 
followed  by  Spain  (18  projects  or  20 %).  Only  in  Romania  the  number  of  national  projects 
was higher than the one of cross-border projects. Some organisations in Slovenia and the UK 
coordinated an equal number of national and cross-border partnerships. 

Overview of main characteristics of projects under the programme 

This  subsection  presents the descriptive analysis carried out on the sample projects, namely 
projects which were selected for in-depth evaluation analysis39. The documents accompanying 
the sample projects have been analysed and included the grant agreements, the final narrative 
reports, the mid-term reports (as from 2010) and the final evaluation reports prepared by the 
Commission. 

General description of the sample 

The  sample  of  29  projects  represents  28 %  of  all  104  finalised  projects  from  the  CIPS 
programme and was representative of all projects. Indeed, the list of finalised projects broadly 

38   Year related to the year of the AWP under which a certain call for proposals was issued and included in the 

project’s reference indicated in the Commission monitoring table. 

39   The  sample  of  individual  projects  included  only  ‘finalised’  project  since  the  implementation  phase  was 

ended and the necessary documentation was available.  

13 

 
 
                                                            
matched the characteristics of the complete list. The characteristics of the sample project are 
further  described  below.  However,  given  that  the  sample  only  contains  29  projects,  this  list 
should not be considered as exhaustive. 

98 % of CIPS grants were funded through open calls while only 2 % were funded via grants 
to standardisation bodies. In the sample of projects compiled the proportion is 97 % to 3 %. 
The sample is representative of the projects awarded per annual work programme per year. It 
contains  a  lower  proportion  of  projects  from  2013  as  at the  time  of  the  selection only  three 
projects were eligible and thus included. 

Partnership characteristics of the sample projects 

An  analysis  of  the  partnership  characteristics  of  the  sample  projects  was  carried  out  and 
included.  For  instance,  the  type  of  organisations  and  partnership  funded,  the  size  of  the 
partnership and the diversity of partnership. 

The  findings  on  possible  outcomes  and  results  in  light  of  the  above-mentioned  partnership 
characteristics are described in the EU Added value section. 

In terms of types of coordinating organisations funded, the most common were universities or 
research organisations (10 projects) followed by private sector organisations. The latter were 
often partners with public organisations to deliver CIPS projects and vice versa. 

On  the  size  of  partnerships,  it  varied  across  the  sample.  Overall,  the  average  number  of 
organisations participating in a CIPS project within the sample was four. On the diversity of 
partnership, a considerable variation in the number of different types of organisation included 
within a project partnership was shown by the sample. Among projects implemented by more 
than one organisation the average number of types of organisation involved was three. 

Project activities and outputs 

As explained in Section 2.3, financial support under the CIPS programme was awarded to five 
types of activities.  

i) Operational cooperation and coordination;  

ii) Analysis, monitoring and evaluation activities;  

iii)  Development:  development  and  transfer  of  technology  and  methodology  (including 
information sharing and inter-operability);  

iv) training: exchange of staff and experts; and 

v) Awareness: awareness and dissemination activities.  

Activity V ‘Awareness’ was the only mandatory component for projects (as from 2009) which 
explains the high number of projects implementing this activity (86 %). 

In terms of outputs, over a third (38 %) of all CIPS projects in the sample database involved 
an exchange of information or practice, and this was the most common output under activity 
1. 

For  activity  2,  over  half  (52 %)  of  all  CIPS  projects  in  the  sample  conducted  a  study  or 
evaluation. Other outputs were created by four projects within the sample and included: 

1.  Creation of a database, classification of information; 
2.  Identification of relevant geographical information systems; 

14 

 
 
3.  Analysis of technical standards and methods for resistance assessment and analysis of 

legislative and related documents; and 
4.  Policy analysis for creation of a handbook40. 

For  activity  3,  over  a  third  (38 %)  of  sample  projects  undertook  the  development  of  new 
equipment/software  while  one  third  (31 %)  were  related  to  the  development  of  new 
methodologies. Only 3 % of sample projects concerned a transfer of equipment or software. 
Three  projects  in  the  sample  produced  outputs  that  did  not  fit  into  the  categories  provided, 
namely: 

1.  Development of a new model for security in critical infrastructures; 
2.  Multi-level framework for the alignment of regional CIP/R strategies with national 

and EU strategies; and 

3.  Development of security standards and elaboration of methodologies for assessment of 

objects’ security. 

Only seven projects (representing  24 %  of the sample projects)  undertook training activities 
(activity 4). 

As  awareness  activities  (activity  5)  became  compulsory  after  2009,  the  vast  majority  of 
sample projects reported to have undertaken this activity. The most common output of activity 
5 was the organisation of dissemination events (83 % of projects) followed by dissemination 
materials (69 %). 

Analysis of fund absorption41 

As  stated  in  the  introduction  and  the  implementation  section,  a  total  budget  of  EUR 126.8 
million was allocated to the CIPS programme of which EUR 74.6 million was committed and 
EUR 51.7 was spent. Indeed, not all available funding was awarded due to the fact that some 
project  applications  did  not  meet  the  eligibility/  quality  criteria  for  being  awarded  funding. 
The  absorption  rates  are  calculated  using  the  financial  data  included  in  the  annual  work 
programme,  the  modifying  decisions  applied  on  these  amounts  and  the  DG  Budget  ABAC 
system. 

Absorption rate of grants 

During  the evaluation period,  a total of EUR 97.8  million  was allocated to  grants,  of which 
EUR 51.5  million was committed and EUR 43.0 million was spent by 31 March 2017. This 
translates  into  an  absorption  rate  of  83 %42.  The  table  below  presents  an  overview  of  the 
overall spending for the programme. 

Table 3: Overview of allocated, committed and spent funding, in millions of EUR (rounded) by 31 March 2017 

40   More specifically this concerned analysing the status of related Policy documents to provide a methodology 
to be applied to the selected use cases for the envisaged development of a European Smart Grid Operation 
Handbook and cyber-security aware Grid Codes for the Member States. 

41   The  analysis  of  fund  absorption  takes  into  account  budgetary  information  on  grants  as  received  up  to 

31/03/2017. 

42   With  a  total  of  19  projects  not  finalised  by  31/03/2017  of  which  16  for  2013  and  3  for  2012.  For  these 
projects, the final invoice amount, as recorded in ABAC, has been used for the calculation of the absorption 
rate. 

15 

 
 
 
 
                                                            
Year 

2007 

2008 

2009 

2010 

2011 

2012 

2013 

Total 

Allocated 

Committed 

Spent 

10.4 

12.1 

14.6 

16.1 

17.0 

18.0 

  9.5 

97.8 

6.6 

8.4 

9.3 

5.9 

6.2 

6.5 

8.6 

4.5 

6.9 

7.8 

4.6 

5.9 

5.7 

7.6 

51.5 

43.0 

Source: ICF analysis of data from the annual work programmes (allocated), and DG Budget 
ABAC data (committed and spent). 

Over  the  programming  period,  CIPS  had  a  higher  amount  of  committed  funding  (83 %) 
compared to ISEC (74 %) with regard to overall grants. The reasons of the higher absorption 
rate  under  CIPS  included  the  strongest  knowledge  of  beneficiaries  in  order  to  properly 
manage  the  calls,  and  the  changes  adopted  by  the  Commission  to  simplify  the  overall 
implementing process43. Furthermore, the enhanced capacity of CIPS stakeholders in planning 
project’s budget could also be considered as one of the reasons that contributed to this effect. 

More specifically, as reported in the Figure 1 below, an increase in the fund’s absorption was 
registered  for  CIPS  as from  2007  (68 %) to  2013  (88 %),  with  a  pick in  2011 (96 %).   The 
average  absorption  rate  after  2011  was  90 %,  and  this  can  be  explained  by  the  fact  that 
stakeholders became more familiar with the programme and as a consequence they were able 
to provide more realistic budget projections. 

Figure 1 Committed and spent budget for grants, and absorption rate, by year by 31 March 2017 

43   For more information, see the answer to the evaluation question B7.c. 

16 

 
 
 
 
                                                            
Source: ICF analysis of data on committed and spent budget. 

Absorption rate in the procurement sector 

A  total  of  106  procurement  items  were  funded  by  the  programme.  The  number  of  public 
procurement  items  in  this  sector  experienced  a  significant  increase  during  the  evaluation 
period, from only 6 in 2007 to over 20 in 2012 and 2013.  

On CIPS procurement amounts, EUR 10.80 million was set aside in line with the annual work 
programme,  of  which  EUR 10.3  million  was  committed  and  EUR 8.7  million  was  spent  by 
31 March 2017. Figure 2 below shows the absorption rate per year, an average absorption rate 
of 85 % was registered44. 

Figure 2 Initial procurement amount committed, paid and absorption rate across the period 2007-2013 by 
31 March 2017 

Source: ICF analysis of DG Budget ABAC data. 

Absorption rate of JRC actions/ administrative arrangements 

Seven  administrative  arrangements  or  actions  carried  out  with  JRC  were  identified  by  the 
evaluators based on the ABAC data received from the Commission. Originally, a budget of 
EUR 18.2 million was allocated to actions and administrative arrangements  with the JRC of 
which EUR 12.8 million was committed across the programming period. Final data on project 
expenditure was not available at the time of writing the report. 

44   Please note that the absorption rate for procurement items is based on ABAC data up to 31/03/2017. For all 
procurement entries used for the procurement sector as presented in this report there was both a committed 
and spent value which have been used to calculate the absorption rate. 

17 

 
 
 
 
 
 
 
 
                                                            
7  ANSWERS TO THE EVALUATION QUESTIONS 

7.1 RELEVANCE  

EQB1.a: To what extent did the objectives of the CIPS Programme correspond to the needs 
related to the prevention, preparedness and consequence management of terrorism and other 
security-related risks? 

for  critical 

infrastructures  protection 

The  objectives  of  the  CIPS  programme  were  designed  to  be  in  line  with  the  EU  policy 
framework in the area of critical infrastructure protection. The specific objectives were linked 
to the general objectives, which are preparedness and protection (Art.4 (2)) and consequence 
management (Art.4 (3)). Further details on the architecture of the fund are reported in Section 
2.3. 
The  European  programme 
(EPCIP),  whose 
implementation  was  supported  by  the  CIPS  programme,  recognises  that  many  critical 
infrastructures  have  a  cross-border  dimension,  which  would  affect  a  number  of  EU 
countries45.  The  Commission  considered  that  the  objectives  of  the  CIPS  Programme 
corresponded to the needs related to prevention, preparedness and consequence management 
of terrorism and other security related risks. Furthermore, the Programme responded to a real 
need  for  cross-border  cooperation  and  coordination  in  the  areas  of  prevention,  preparedness 
and consequence management of terrorism and other security-related risks given the increase 
of terrorist attacks in recent years. It should be pointed out that from the interviews with the 
stakeholders consulted it emerged that the CIPS programme was of continuous relevance to 
the  prevention,  preparedness  and  consequence  management  of  terrorism  and  other  security-
related risks throughout the evaluation period. 

The  continuous  relevance  of  the  Programme  was  also  ensured  by  the  process  of  setting  the 
CIPS  priorities  and  sub-priorities,  which  were  identified  by  Member  States  and  competent 
Directorate-Generals of the Commission46 on a yearly basis and included in the annual work 
programmes. 

The  majority  of  Member  State  authorities  representatives  interviewed  did  not  mention  any 
issues  related  to  the  process  of  priority-setting47.  Most  CIPS  coordinating  organisations  and 
partner  organisations  responding  to  the  online  survey  considered  the  priorities  set  by  the 
Commission  in  the  calls  for  proposals/annual  work  programmes  as  relevant  to  the  specific 
needs of the target group (9 out of 15), to the specific strategic needs at EU level (9 out of 15) 
and to the specific strategic need at national level (10 out of 15) (conclusions). 

Furthermore, around 50 % of coordinating organisation responding to the survey reported that 
they designed the project on the basis of a needs assessment whereas 20 %  highlighted that 

45   See also the EPCIP, p 8 on  

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2006:0786:FIN:EN:PDF 

46   The  process  mentioned  for  the  identification  of  priorities  and  sub-priorities  was  carried  out  in  regular 
consultation  with  the  Member  States,  through  a  comitology  procedure  and  an  inter  service  group  with 
relevant DGs competent in the area of protection of critical infrastructure in the Commission, including DG 
CNECT, DG ECHO, DG SANCO, DG MOVE, DG ENER and the JRC. 

47   Only  one  interviewee  commented  that  the  process  was  not  very  open  and  that  there  were  not  real 

possibilities to influence the content of the programmes. 

18 

 
 
 
 
                                                            
 
such  assessment  was  ‘sometimes’  carried  out  and  another  20 %  replied  negatively  to  this 
question48. Needs assessments were often carried out through meetings and discussions with 
relevant  stakeholders,  including  experts,  critical  infrastructure  operators  and  relevant 
Ministries and government agencies. 

EQB1.b:  To  what  extent  did  the  design  of  the  CIPS  programme  correspond  to  the  needs 
related to the prevention, preparedness and consequence management of terrorism and other 
security-related risks? 

The design of the programme relates to the main features of the CIPS programme, namely its 
scope, funding mechanisms, types of actions and stakeholders targeted under the programme. 
The evaluation study assessed to what extent this design corresponded to the needs related to 
the  prevention,  preparedness  and  consequence  management  of  terrorism  and  other  security 
related risks during the evaluation period. 

The rationale behind the CIPS programme was to provide funds where demand was highest in 
line with the priorities in the annual work programmes. For this reason, the CIPs programme 
was broadly demand-driven (based on open calls for proposals and allowed actors to apply 
for  funding  based  on  a  project  proposal)  rather  than  policy-driven.  Indeed,  if  no  project 
promoters  applied  for  funding  as  part  of  a  specific  priority  in  a  given  year,  that  particular 
priority would not be implemented. However, the Commission could add remaining funding 
from targeted calls to a subsequent general call. The logic behind the annual work programme 
was  to  provide  funds  where  demand  was  highest  in  line  with  the  priorities  outlined  in  the 
annual work programme. 

The  demand-driven  design  of  the  programme  contributed  to  a  significant  geographical 
imbalance in its implementation, especially for the location of the coordinating organisations 
(see  Section 7.1).  For  this  reason, the Commission  took  measures to  boost the geographical 
spread,  particularly  through  the  organisations  of  information  days  in  the  Member  States. 
However,  the  programme’s  implementation  depended  heavily  on  the  pro-activeness  of 
relevant government representatives and potential applicants from the Member States. 

On the thematic coverage of the programme, as provided in Section 2.3, this was defined in 
broad terms and covered: 

1.  Risks:  terrorism  and  other  security-related  risks  (this  could  include  other  criminal 

activities, natural disasters etc.); 

2.  Protection of people and critical infrastructure; 
3.  Pre-attack (preparedness and prevention) and post-attack (consequence management); 
4.  Unlimited range of sectors: energy, cyber, food, health, space, etc.; 
5.  Activities:  developing  tools,  methodologies  and  guidelines,  training,  exchange  of 

experience, best practice and know-how, etc. 

The  evidence  gathered  confirmed  that  this  extensive  coverage  was  perceived  as  positive  by 
consulted stakeholders as it allowed for flexibility throughout the funding period. Meanwhile, 

48   One coordinating organisation responded ‘I don’t know’. 

19 

 
 
 
 
 
                                                            
a  finding  emerging  from  the  various  consultation  rounds  and  expert  opinions  was  that  the 
broad coverage of the programme might have contributed to the lack of clarity of its scope, 
which  may  have  rendered  the  drafting  of  applications  challenging  for  a  number  of 
organisations.  For  this  reason,  the  difference  between  European  and  national  critical 
infrastructure could be made clearer, as CIPS has a strong link to the EPCIP. 

The  funding  mechanism  was  considered  appropriate  by  the  majority  of  stakeholders 
consulted49.  Moreover,  the  close  involvement  of  the  JRC  in  the  implementation  of  the 
programme was considered to be positive (further details on the role of the JRC are reported 
below). However, a minority of stakeholders considered that the programme should have been 
more flexible, as some of the eligibility criteria for obtaining CIPS funding discouraged some 
organisations from applying. 

49   See more details on the financial mechanism of the programme in Section 2.3. 

20 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                                            
7.2 EFFECTIVENESS 

To report on effectiveness of the Fund, the analysis presented below is structured around the 
two  general  objectives  and  the  seven  specific  objectives  set  out  in  the  legal  basis.  The 
assessment is based on nine evaluation questions (EQsB2 to B6). 

General  Objective  1-  Support  Member  States’  effort  to  prevent,  prepare  for  and  to  protect 
people and critical infrastructures 

General Objective 2 — Crisis Management 

Specific  Objective  1  —  Stimulating,  promoting,  and  supporting  risk  assessments  on  critical 

infrastructures:  

Specific  objective  2:  Stimulating,  promoting,  and  supporting 

the  development  of 
methodologies  for  the  protection  of  critical  infrastructures  (risk  assessment 
methodologies) 

Specific  objective  3:  Promoting  and  supporting  shared  operational  measures  to  improve 

security in cross-border supply chains 

Specific  objectives  4:  Promoting  and  supporting  the  development  of  security  standards,  and 
exchange of know-how and experience on protection of critical infrastructures:  

Specific Objectives 5: Promoting and supporting Union-wide coordination and cooperation on 

protection of critical infrastructures;  

Specific  Objective  6:  Promoting  and  supporting  exchange  of  know-how  and  experience 

related to the consequence management, in order to establish best practices:  

Specific  Objective  7:  Promoting  joint  exercises  and  practical  scenarios  in  consequence 
management,  including  security  and  safety  components,  in  order  to  enhance 
coordination and cooperation between relevant actors at the European level:  

EQ  B.2:  To  what  extent  did  the  CIPS  programme  contribute  to  supporting  Member  State 
efforts to prevent, prepare for and protect people and critical infrastructure against terrorist 
attacks and other security-related incidents? 
EQ B3: To what extent did  the CIPS programme  contribute to  protection in areas such as 
crisis  management,  environment,  public  health,  transport,  research  and  technological 
development in terrorism and other security-related risks?  

Overall, the evaluation findings suggest that the programme broadly achieved its general and 
specific  objectives.  This is  supported by most  coordinating organisations responded that the 
objectives were fully achieved (13 out of 14 respondents). Nevertheless, these findings should 
be seen in the light of the methodological limitations presented in Section 5, in particular the 

21 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
fact that the progress was assessed at output level mainly, on a limited sample of projects and 
based  on  qualitative  evidence.  Also,  when  looking  to  effectiveness,  it  should  be  taken  into 
account  that  the  programme’s  first  general  objective  was  only  to  support  Member  States 
efforts in protecting people and critical infrastructure without substituting to Member States in 
this  area.  In  addition,  the  relatively  small  allocated  budget  of  EUR 126.8  million  —  in 
comparison to other EU programmes as well as national budgets in CIP area — should be also 
taken  into  account  when  assessing  the  results  and  impacts  of  the  programme.  Finally,  the 
broad  scope  of  the  programme,  resulting  in  a  high  number  of  projects  of  relatively  small 
value, should also be taken into consideration when analysing its overall effects.  

Regarding  the  results  and  outcomes  of  the  sample  of  projects  analysed  (29),  from  the 
evaluation  study,  it  emerged  that  the  majority  of  these  projects  (62 %)  have  led  to  an 
improved  understanding  of  methods,  tools  and  practices,  59 %  to  an  increased  level  of 
knowledge and skills of practitioners and 52 % to enhanced networking and forms of sharing 
information and best practice. 

In  terms  of  impacts,  the  majority  of  projects  analysed  (16)  contributed  to  the  increase  of 
prevention  capacity  to  protect  people  and  critical  infrastructure  against  terrorism  and  other 
security-related  incidents.  A  high  level  of  projects  (15)  contributed  to  increase  the  level  of 
preparedness  whereas  only  7  projects  contributed  to  increase  risk  assessment  capacity  on 
critical  infrastructures.  Conversely,  no  projects  contributed  to  increase  response  capacity. 
Other  impacts  registered  were  the  exchange  of  good  practice,  between  Member  States  on 
technology and the improvement of participants’ skills and knowledge. 

On  the  overall  implementation  of  activities,  the  majority  of  coordinating  organisations 
involved in CIPS projects who participated in the survey reported that all project activities (12 
out of 14 respondents) or most of them (2 respondents) were delivered.  
From the analysis of the sample of projects for in-depth analysis it also emerged that 90 % of 
projects achieved all of their objectives whereas only 10 % of projects partially achieved their 
objectives.  The  final  assessment  carried  out  by  the  Commission  on  the  sample  projects 
reported 7 % of projects as being excellent, 59 % as having had positive results and impacts 
and 21 % were considered acceptable. Furthermore, 26 projects from the sample (out of 29) 
reached  their  initial  expectations.  On  the  other  hand,  some  of  them  also  implemented 
activities not originally envisaged such as additional external events and/or higher number of 
participants  to  activities.  However,  the  surveyed  also  reported  on  obstacles  encountered 
during  the  project’s  implementation  and  these  included,  for  instance,  (i)  the  low  level  of 
participation from project participants (6 projects) and the lack of cooperation from partners 
(1 project), (ii) technical difficulties, and (iii) language barriers. Moreover, based on the final 
evaluation  carried  out  by  the  Commission  it  emerged  that  some  outputs  could  have  been 
better designed to gain a better outreach. Indeed, considering that CIP is technically complex, 
the results were perceived in some cases as being too technical to promote wide use among 
policymakers. In certain other instances, the Commission highlighted that some projects had a 
too strong national focus which prevented the broader use of their results. 

In  terms  of  actions  funded  under  CIPS,  a  representative  sample  of  29  projects  have  been 
reviewed and showed that most of the projects were funded in the sub-area of protection of 
critical  infrastructure  and  people  (10  projects  or  35 %  of  the  sample),  followed  by  risk 
assessment (eight projects or 28 % of the sample). Most of the examined projects (20 projects 

22 

 
 
 
 
 
 
or 69 %) had the development of methodologies for the protection of critical infrastructures as 
an  objective.  More  precisely,  38 %  of  the  sampled  projects  aimed  to  contribute  to  the 
establishment of frameworks for the exchange of know-how and experience on the protection 
of critical infrastructures; 21 % aimed at establishing best practices and coordinating response 
included  coordination  and  cooperation  on  protection  of  critical 
measures;  17 % 
infrastructures;  14 %  focused  on  joint  exercises  and  practical  scenarios  and  7 %  on 
operational measures to improve security in cross-border supply chains. 

Evidence  collected  through  the  stakeholder  consultation  showed  that  results  based  on  the 
following  two  components  were  perceived  as  particularly  valuable  and  contributed  to  the 
increase of the overall effectiveness of the programme:  

a) potential for transferability to other sectors and other Member States; and  
b) readiness to be used by critical infrastructure owners and operators in contrast to prototypes 
and models that are not operable or market-ready.  

In consequence, for the future, similar programmes should be carried out in order to motivate 
coordinating organisations to work towards the development of demonstrators or prototypes 
(to  be  developed  as  market-ready  products)  which  are  based  on  the  results  of  previous 
projects. 

Furthermore,  given  the  relative  newness  of  the  Programme  and  the  lack  of  resources  of  the 
Commission at the time to closely follow all CIPS projects from start to end, the evaluation 
showed  that  a  dedicated  technical  assistance  budget  could  have  been  useful.  Such  a  budget 
could have better supported the development of a state of the art monitoring system to collect 
and  analyse  data  on  financial  progress,  outputs  and  results  of  projects,  as  well  as  provided 
technical  expertise  throughout  the  lifecycle  of  projects  and  increased  the  dissemination  of 
results. Therefore, the lack of a dedicated technical assistance budget has negatively affected 
the effectiveness of the programme.  

Sectoral analysis of CIPS projects 

Projects  were  categorised  per  specific  sectors  depending  on  their  aims  and  purposes.  The 
sectoral  dimension  has  enabled  the  coverage  of  sectors  with  critical  importance  such  as 
energy,  transport  and  telecommunication.  However,  some  sectors  were  covered  to  a  much 
greater  extent  than  others,  and  the  more  popular  were  the  information,  communication  and 
technology  (ICT,  23 %),  transports  (18 %)  and  energy  (12 %).  Most  of  the  projects  were 
‘multi-sector’  which  means  that  they  covered  more  than  one  sector.  Nearly  a  third  of  all 
projects were granted in 2009, half of them were led by a variety of bodies in Italy (including 
universities  and  research  institutes,  government  ministries  and  regional/city  councils).  The 
remaining projects were funded to organisations in Greece (2), Spain (5), France (1), Hungary 
(1),  the  Netherlands  (1),  Romania  (2)  and  UK  (2).  More  details  on  the  sectoral  analysis  of 
CIPS projects are reported in Annex 7. 

Dissemination 

Dissemination activities on the results were carried out by the coordinating organisations and, 
depending  on  the  nature  of  results  achieved,  took  place  via  dedicated  websites,  press 

23 

 
 
 
 
 
 
 
 
 
announcements, printed material and conferences. DG HOME, with the technical support of 
the JRC, developed the Critical Infrastructure Warning Information Network (CIWIN), which 
aimed  to  provide  a  platform  for  exchanging  CIP-related  information,  studies  and/or  good 
practices across all EU Member States and included also the results of CIPS-funded projects.  

The  CIWIN  became  operational  only  in  January  2013  and  the  stakeholders  involved 
appreciated  the  Commission’s  efforts  in  providing  information  on  the  outcome  of  CIP 
national contact points meetings and the CIWIN platform. 

It should be highlighted that despite the absence of a central repository for CIPS, the active 
involvement  of  a  number  of  Directorate-Generals  in  the  implementation  of  the  CIPS 
programme proved that the monitoring and dissemination activities were shared between the 
Directorate-Generals involved. 

The  results  produced  by  CIPS-funded  projects  implemented  by  JRC  are  stored  in  their 
publication repository: PUBSY50. 

EQ B4.a: To what extent did the CIPS programme contribute to the protection of people and 
infrastructure  by  promoting  and  supporting  risk  assessments  on  critical 
critical 
infrastructure, in order to upgrade security? 

EQ B4.b: To what extent did the CIPS programme contribute to the protection of people and 
critical  infrastructure  by  promoting  and  supporting  the  development  of  methodologies  for 
the protection of critical infrastructure, in particular risk assessment methodologies? 

Overall,  the  Commission  evaluated  the  results  of  the  CIPS  projects  on  protection 
methodologies  as  good  on  average,  and  the  transferability  of  the  outputs  was  particularly 
appreciated, stressing the importance of being able to adapt the outputs to other sectors and to 
implement them on a large scale. 

More  than  60 %  of  all  CIPS-funded  projects  focused  on  developing  frameworks, 
methodologies,  tools  and  approaches  for  risk  assessment  at  a  sectoral  level51.  Some  of  the 
tools and methodologies that resulted from CIPS projects were the following:  

i) A risk assessment toolbox for assessing the risks of terrorist attacks against specific critical 
infrastructures;  

ii)  Risk  assessment  conducted  on  online  social  media  through  the  analysis  of  open  source 
intelligence (SNAPSHOT project); and 

iii)  Methodologies  dedicated  exclusively  to  risk  assessment  of  interdependencies  and 
contingency plans of critical infrastructures. 

As an example of projects focusing on risk assessments at sectoral level, the project  Threat-
Vulnerability Path Identification for Critical Infrastructures Compilation (THREVI2) was led 
by  a  private  company  (NIER  INGEGNERIA  S.p.A.,  Italy)  and  is  a  compilation  of  a 
comprehensive all-hazard catalogue for critical infrastructures. A literature review of data and 

50   http://publications.jrc.ec.europa.eu/repository/ 
51   For more information, see Annex 7 ‘Sectoral analysis of CIPS projects’. 

24 

 
 
 
 
                                                            
historical events to develop a comprehensive catalogue of hazards and threats was produced 
and disseminated in conferences and workshops. 

Furthermore,  when  analysing  the  risk  assessment,  a  particular  attention  was  given  to  the 
‘cascading effect’  of common infrastructures. Indeed, to  ensure the effectiveness of the risk 
assessment, the cascading effect needs to be taken into account in order to prevent and reduce 
consequences of security incidents. Critical infrastructures are interdependent within different 
sectors, not only for the proximity of their geographical location but also because of resource 
exchange or cyber dependencies. Such interdependencies, if not well assessed and considered, 
may  lead  to  catastrophic  cascading  failures  and  domino  effects.  For  this  reason, 
approximately  20  projects  funded  by  CIPS  dealt  with  interdependencies  of  critical 
infrastructures,  predominantly  with  a  focus  on  risk  assessment.  The  focus  of  a  number  of 
CIPS  projects  on  interdependencies  and  ‘cascading  effects’  in  case  of  disruption  and 
destruction  of  critical  infrastructures  during  a  terroristic  attack  or  natural  disaster  was 
considered to have contributed to the programme’s objective related to critical infrastructure 
protection. 

The JRC also contributed to the promotion and support of risk assessment capacity on critical 
infrastructures  with  different  specific  projects.  For  instance,  one  project  on  studies  and 
support  related  to  threat  and  vulnerability  assessment  (2008)  and  another  one  on  Risk 
Assessment  methodologies  and  tools  for  CIP  entitled  ‘Increasing  resilience  against  all-
hazards’ (2013). 

EQ B4.c: To what extent did the CIPS programme contribute to the protection of people and 
critical infrastructures by supporting shared operational measures to improve security in 
cross-border supply chains, in compliance with EU regulations on competition within the 
internal market? 

It is not possible to draw clear conclusions on CIPS’s contribution in this area due to the small 
number of projects implemented in this sector. Only 6 out of 126 projects were  dedicated to 
the  security  of  cross-border  supply  chains,  and  focused  on  drinking  water,  food,  electricity 
and ports. Only one project did not focus on a specific type of supply chain but was dedicated 
to  the  improvement  of  the  management  of  cross-border  disruptions.  On  the  projects’ 
objectives, these varied and included: (i) identification and assessment of cascading effects in 
the  global  ports’  supply  chain,  and  (ii)  strengthening  the  resilience  capacities  required  to 
manage cross-border critical infrastructure disruptions. 

Some experts considered the wording ‘supply chain’ as misleading given that what needed to 
be  protected  was  the  cross-border  infrastructure  distributing  the  goods  and  not  the  ‘supply 
chain’, which refers to the whole distribution process.  

EQ B4.d: To what extent did the CIPS programme contribute to the protection of people and 
critical infrastructures by promoting and supporting the development of security standards, 
and  exchange  of  know-how  and  experience  on  protection  of  people  and  critical 
infrastructures? 

EQ  B5:  To  what  extent  did  the  CIPS  programme  contribute  to  promoting  and  supporting 

25 

 
 
 
 
exchange of know-how and experience related to the consequence management, in order to 
establish best practices so as to coordinate the response measures and to achieve cooperation 
between various actors in crisis management and security actions? 

On  average,  the  results  of  CIPS  projects  on  exchange  of  know-how  and  experience  were 
considered as good by the Commission, and the transferability of the outputs as particularly 
valuable. 

Overall,  17  projects  focused  on  the  exchange  of  know-how  and  experiences  related  to  both 
protection and crisis management. For instance, the ‘MIRACLE’ project aimed to support the 
coherent development of critical infrastructures protection and/or resilience (CIP/R) strategies 
at regional level. Its final objective was to improve existing capacities of EU Member States 
to  prevent,  prepare  and  protect  people  against  terrorist  attacks  and  security-related  risk  by 
setting up an international network of professionals and institutions for the exchange of know-
how  in  the  resilience  of  critical  infrastructures.  Following  selection  and  analysis  of  best 
practice, a manual dedicated to sharing good practice within Member States was produced. In 
term  of  results,  it  was  considered  as  user-friendly  and  well  adapted  to  the  target  audience, 
represented by policymakers. 

EQ B4.e: To what extent did the CIPS programme contribute to promoting and supporting 
Union-wide coordination and cooperation on protection of critical infrastructure? 

The results of CIPS projects on EU-wide cooperation and coordination on CIP were evaluated 
by the Commission as not very effective, with a few peaks of excellence.  

Several  projects  contributed  to  promoting  and  supporting  EU-wide  coordination  and 
cooperation among Member States. 

In prevention, EU-wide cooperation and coordination is linked to the exchange of information 
on  threats  among  public  authorities  and  stakeholders  involved  in  the  protection  of  critical 
infrastructures.  CIPS  funded  for  example  the  European  Cooperation  Network  on  Critical 
Infrastructure  Protection  (EUCONCIP)  which  brings  together  experts  on  CIP  from  across 
Europe  and  set  the  ground  for  the  creation  of  a  European  partnership  between  critical 
infrastructures  stakeholders,  with 
international 
cooperation, planning, networking and cross-border exchange of information.  

to  strengthen  public-private 

the  aim 

As another example of project, the ‘security liaison officer’ had the objective to support the 
creation  of  the  role  of  liaison  officer  in  charge  of  security  for  all  European  and  national 
critical infrastructures. Although every critical infrastructure has a security manager, the role 
of  security  liaison  officers  is  to  enable  the  protection  of  the  European  critical  infrastructure 
through  better  information  sharing  and  cooperation  as  security  managers  could  also  have 
other roles and responsibilities. 

EQ B6: To what extent did the CIPS programme contribute to promoting joint exercises and 

26 

 
 
 
 
 
practical scenarios in consequence management, including security and safety components, 
in order to enhance coordination and cooperation between relevant actors at the European 
level? 

Overall, a significant number of projects (19 projects) contributed to promoting joint exercises 
and practical scenarios between different actors at European level. 

An  example  of  interesting  CIPS  project  on  consequence  management  is  the  ‘Disaster  2.0’ 
project.  It  aimed  at  establishing  how  successful  the  Web  2.0  platforms52  were  in  mitigating 
damage  to  critical  infrastructure,  building  resilience  in  the  public,  and  what  were  the 
challenges encountered by their use. The project also explored the potential of next generation 
Semantic  Technologies  (Web  3.0)  in  disaster  response.  Furthermore,  it  collected  substantial 
data on behaviour, uptake and barriers to the use of social media in emergency management 
and  by  emergency  management  agencies.  As  envisaged,  it  also  developed  a  vision  for  the 
application  of  semantic  technologies  in  this  field  by  emergency  management  agencies,  for 
which it created a prototype software. 

7.3 EFFICIENCY 

EQ  B7.c:  What  kind  of  initiatives  or  approaches  were  adopted  to  simplify  access  to  and 
implementation of the actions funded by the programme (e.g. changed to shared management 
in ISF-P)? 

As  mentioned  in  Section  5,  due  to  the  lack  of  an  initial  baseline  and  to  the  difficulties 
encountered  in  comparing  project  costs,  the  assessment  made  of  efficiency  below  relies 
largely on the qualitative data collected through the stakeholder consultations and interviews. 
Furthermore, the low number of respondents to the surveys (19 %) should be also taken into 
account. 

In  order  to  improve  the  inefficiencies  related  to  accessibility,  the  interim  evaluation  of  the 
programme’s  implementation  completed  in  2010  identified  certain  areas  for  improvement. 
Consequently the Commission adopted the following measures: 

i) Changes related to the application and award process 

Overall,  the  changes  introduced  to  the  application  and  award  process  contributed  to  the 
decrease of ineligible project applications. However, some other factors such as the improved 
know-how of the application system and rules may have also contributed to this decrease. 

Application and awarding/grant agreement procedure — guidance for applicants 

The Commission organised some information days in 2009-2011 in Brussels which aimed to 
explain  the  application  procedures  to  potential  applicants.  Some  regional  information  days 
were also held in seven Member States (AT, IT, RO, HU, SE, SI and the UK) in 2010-2011. 
A significant increase in the total number of applications was registered from 2011. 

52   The  Web  2.0  platforms  are  applications  enabling  live  chatting  and  are  already  used  in  some  cases  by 

practitioners and the public in response to disasters. 

27 

 
 
                                                            
The Commission also improved the guidance documents for the calls for proposals. However, 
there was still room for improvements as the volume of accompanying documents could have 
been seen as burdensome and the information could have been compiled in a single package. 

Overall,  10  out  of  14  respondents  to  the  survey  for  CIPS  coordinating  organisations  were 
satisfied  with  the  management  of  the  application  process  applied  by  the  Commission. 
Furthermore,  9  out  of  13  respondents  also  reported  that  they  did  not  encounter  any  issue 
related  to  the  signing  of  grant  agreements  (conclusions).  However,  a  limited  number  of 
respondents  (4)  highlighted  that  they  experienced  certain  problems,  such  as  communication 
issues due to the numerous changes affecting the Commission’s staff and a lack of flexibility 
in the administrative process. 

ii) Revision to the process of budget verification in 2010 

Between 2007 and 2010, the Commission carried out a detailed analysis of the eligibility of 
the budgets outlined in the project applications. Consequently, a more efficient approach on 
the  negotiations  for  the  proposed  budget  with  the  beneficiaries  was  adopted  by  moving  the 
negotiations  before  the  award  decision  (whereas  originally  these  took  place  between  the 
award decision and the signing of the agreement). This change led to a sudden decrease of the 
number of days between the award decision and the grant agreement signature. Furthermore, 
from 2010 the concept of ‘value for money’ was considered as an award criterion hence cost-
effectiveness of projects increased. The ‘value for money’ principle was overall considered as 
a key driver across all activities53 by the majority of stakeholders therefore it is recommended 
to fund similar activities in future programmes. 

iii) Strengthening the rules of cross-border partnerships 

The Commission restricted the rules on the eligibility criteria on cross-border projects as, in 
the  first  years  of  the  programming  period,  the  Commission  was  not  in  a  position  to  easily 
check the degree of cross-border partnerships. For this reason, for the 2007 call for proposals, 
applicants had to ensure that cross-border projects involved at least one partner organisation 
from  another  Member  State  or  from  an  acceding/candidate  country.  However,  stricter  rules 
followed from 2008, such as the obligation to sign a partnership declaration. 

iv) Maximum length of projects 

In  2008,  the  maximum  length  of  a  project  was  decreased  from  three  years  to  two  years. 
Several  reasons  led  to  this  decision,  for  instance,  the  fact  that  reducing  the  project’s  length 
contributes  to  a  better  match  between  the  implementation  period  and  the  period  when  the 
policy formulation was laid down in the annual work plan. Indeed, the two years duration was 
considered as the best time period reflecting overall the timeline set to achieve national policy 
priorities. 

v) Minimum and maximum EU grant 

From 2010, the minimum EU co-funding amount to be requested increased from EUR 50 000 
(2007-2009)  to  EUR 100 000.  The  amount  was  increased  mainly  because  higher  amounts 
were needed for high technological investments. 

53   Whether the desired effects were achieved at reasonable costs. 

28 

 
 
                                                            
Agreements  on  budget  thresholds  were  discussed  between  Member  States  and  the 
Commission. However, increasing the minimum EU grant was perceived as most problematic 
in EU-12 Member States. 

The maximum rate of the EU grant share increased consistently with the increase in minimum 
EU grant, from 70 % (period 2007-2009) to 80 % in 2010 and remained at 90 % from 2011 
onwards for the remainder of the programme implementation period. 

Pre-financing rules 
To improve cost-effectiveness, the CIPS programme had generous pre-financing rules54, i.e. 
coordinating  organisations  received  60 %  (until  2010)  to  80 %  (in  2011)  at  pre-financing 
stage.  The  more  generous  advance  proved  to  be  particularly  helpful  for  the  smaller 
organisations to properly set up the partnership and undertake different actions. 

Overall,  coordinating  organisations  interviewed  considered  this  as  advantageous  and  no 
specific issues were mentioned. 

vi) Changes to reporting requirements and monitoring of results 

Financial reporting 

Project  partners  were  requested  to  provide  information  on  the  financial  aspects  of  project 
implementation.  The  2010  calls  onwards,  financial  reporting  became  more  detailed  on  cost 
claim requirements. Therefore, further guidance was provided in separate documents. A few 
grant  beneficiaries  stressed  the  fact  that  reporting  project  expenses  in  detail  was  too 
burdensome. 

Monitoring of results and technical reporting 

As  stated  above  under  the  effectiveness  section,  one  of  the  main  weaknesses  of  the 
management  of  the  programme  by  the  Commission  was  the  absence  of  a  centralised 
coordinated approach for monitoring project results.  

Furthermore,  as  other  Directorate-Generals  were 
implementation55, coordinating the monitoring of results across them was difficult to ensure. 

the  programme’s 

involved 

in 

For  this  reason,  interim  reporting  was  introduced  to  strengthen  monitoring  results  through 
data collection and to allow coordinating organisations to take stock of progress made to date 
and/or possible needs for adjustments. 

Although  projects  were  required  to  consider  EU  added  value  as  part  of  project 
implementation, there were no obligations to report on this. The reason behind this choice was 
to reduce the burdens on organisations, based on the fact that more detailed reporting alone 
does  not ensure  effective monitoring of results, which should  consequently be  accompanied 
by  the  formulation  of  indicators  and  quantification  of  results.  Nevertheless,  this  still 
represents  a  shortcoming  and  it  is  recommended  to  include  EU  added  value  and  a  self-
assessment of key results and overall impacts on the monitoring report. 

Administrative burden 

54   Compared to FP7 rules for instance. 
55   For more information see the Section on relevance. 

29 

 
 
                                                            
Despite  the  changes  introduced  during  the  CIPS  programming  period,  some  stakeholders 
raised  concerns  on  the  level  of  administrative  burden.  Five  respondents  to  the  survey  for 
coordinating  organisations  considered  the  level  of  administrative  burden  as  ‘significant’ 
whereas  three  of  them  as  ‘very  significant’.  Similar  concerns  were  raised  by  partner 
organisations. 

Private  companies  involved  in  the  CIPS  implementation  and  who  have  implemented  in  the 
past projects under Framework Programme 7 reported a higher level of administrative burden 
under CIPS. From the interviews it emerged that the main obstacles were a lack of flexibility 
and  excessive  bureaucracy  (e.g.  high  level  of  detail  required  conversely  to  some  other 
Programmes  such  as  Framework  Programme  7  and  H2020)  in  relation  to  administrative 
processes  and  the  need  for  simplification  of  reporting  in  terms  of  length  and  detail  (e.g. 
financial  information  requirements  which  is  further  explained  below  under  ‘monitoring  and 
reporting’). 

A shift towards shared management as a management mode for the successor fund 

Unlike CIPS, which  was  implemented 100 % under direct  management, the successor fund, 
ISF-Police,  is  being  implemented  on  the  basis  of  a  combination,  with  around  60 %  of  the 
available  funding  being  allocated  to  national  programmes  under  shared  management  and 
40 % allocated to EU actions under direct management. This change is expected to have an 
impact  on  the  efficiency  and  effectiveness  of  the  Commission’s  management  of  the  ISF 
programme. However, it was not in the scope of this ex post evaluation to determine to what 
extent the shift to shared management has provided efficiency and effectiveness gains in the 
context of the current ISF-Police programme56. 

EVQ  B7.a  &  B7.b:  To  what  extent  were  the  results  of  the  CIPS  programme  achieved  at  a 
reasonable cost in terms of financial and human resources deployed? 

On the cost-efficiency of the financial resources deployed a budget of EUR 126.8 million 
was  allocated  to  the  CIPS  programme  as  laid  down  in  the  annual  work  programme  2007-
2013, of which EUR 74.6 million was committed and EUR 51.7 million was spent. There was 
an increase in the fund’s absorption as from 2007 (68 %) to 2013 (88 %), with a peak in 2011 
(96 %).  The average absorption rate after 2011 was 90 %, and this can be explained by the 
fact  that  stakeholders  became  more  familiar  with  the  programme  and  in  consequence  they 
were  able  to  provide  more  realistic  budget  projections.  The  EU  funding  provided  was 
perceived  by  the  vast  majority  of  stakeholders  consulted  as  having  been  sufficient  and  as 
having been highly cost-efficient for all activities implemented under CIPS. 

Overall,  the  evaluation  analysis  showed  that  88 %  of  the  sample  projects  estimated  budget 
was  spent.  However,  some  differences  between  the  estimated  budget  included  in  the  grant 
agreement  and  what  was  effectively  spent  (as  declared  in  the  final  cost  statement)  were 
registered for certain costs categories. This led to a lower absorption of funding than what was 
expected.  It  should  be  highlighted  that  most  underspend  related  to  costs  for  travel  and  for 
conferences and seminars.  Variations in expenses from the initially forecasted budget to the 

56   Terms of Reference, Art. 2.3. 

30 

 
 
 
                                                            
final  declared  project  expenditure  were  due  to  different  reasons.  For  instance,  the  fact  that 
costs finally resulted higher or lower than expected or the fact that outputs were not delivered. 

Sufficiency of EU funding 

On  average,  88 %  of  projects  costs  were  funded  by  the  EU  contribution  (for  the  28  sample 
projects analysed). For these projects, on average 74 % of financing was made up by the EU 
contribution. The projects less dependent on the EU contribution registered 64 % of income 
made  up  by  the  EU  contribution,  and  only  10  out  of  28  projects  had  an  EU  contribution  of 
88 % or higher. 

Overall, coordinating organisations reported that EU funding was sufficient to implement the 
planned activities. Only a limited number of stakeholders highlighted the need for additional 
funding and the fact that certain costs were underestimated, i.e. staff’s costs. 

Value for money 
Overall,  for  four  out  of  five  activities  formulated  in  the  survey57,  at  least  half  of  all 
respondents to the coordinating organisations and partner organisations considered these to be 
high to very high value for money. On awareness and dissemination activities, just under half 
of the coordinating organisations responding to the survey considered these to be high to very 
high  value  for  money  (with  the  total  number  of  respondents  ranging  from  13  to  14  per 
activity).  

Efficiency of human resources deployed 

On the cost-efficiency of the human resources deployed, a few key performance indicators 
were introduced by the evaluation team to measure the efficiency of programme management 
by  the  Commission.  On  human  resources,  they  were  considered  sufficient  for  a  fairly  low 
number of projects across the whole programming period. 

However,  certain  coordinating  organisations  also  highlighted  few  weaknesses.  For  instance, 
the frequent staff changes and internal re-organisations (at their level and at the level of the 
Commission) led to more time-consuming communications, with valuable knowledge getting 
lost  in  the  process.  More  specifically,  five  coordinating  organisations  also  reported  the 
absence of follow-up/feedbacks from the financial operational officers. 

Several  key  performance  indicators  related  to  the  Commission’s  management  of  the 
implementation  of  the  programme  were  stipulated  in  the  Financial  Regulation  and  are 
reported below.  

1.  With  regards  to  the  time  to  award  (time  from  the  call  for  proposals’  application 
deadline  until  the  award  decision),  the  evaluation  showed  an  overall  negative  trend. 
However,  it  should  be  noted  that  it  remained  below  the  maximum  of  6  months 
anyway. 

2.  On  the  time  to  contract  (time  from  the  award  decision  until  the  grant  agreement 
signature) the average number of days was 122. A positive trend was registered during 

57   The  five  activities  were  formulated  in  the  survey  as  follows:  Operational  cooperation  and  coordination; 
Analytical,  monitoring,  evaluation  and  audit  activities;  Development  and  transfer  of  technology  and 
methodology; Training, exchange of staff and experts and Development and creation of network. 

31 

 
 
 
                                                            
the evaluation period, with the average time reduced by more than half (from 181 days 
in 2007 to 53 days in 2013). 

3.  On the time to pay (time to pay the pre-financing and the final invoice), the evaluation 
showed that, overall, 73 %  of payment requests were  paid on time, with  27 %  being 
late. Significant differences were identified between the pre-financing stage (87 % of 
requests were paid on time) and the final payment stage (only 59 % of invoices were 
paid on time). 

Analysis of project applications 

According to the award decisions,  a total  of 352 applications were  received over  the period 
2007-2013  and  129  grants  were  awarded  (37 %).  The  success  rate  increased  from  53 %  in 
2007  to  69 %  in  2008.  However,  from  2009,  the  success  rate  started  to  decrease  (24 %  in 
2013).  The  two  last  years  of  the  programme  the  number  of  applicants  almost  doubled, 
nevertheless, the number of projects awarded remained constant over the period. 

Between 2007 and 2013, 224 applications were not awarded a grant. 91 were ineligible and 
133 were unsuccessful. However, the ineligibility of projects started decreasing as from 2010. 
This  was  most  probably  due  to  the  fact  that  eligibility  criteria  were  better  followed  and 
understood  in  the  later  years  of  the  programme  and  that  the  changes  introduced  by  the 
Commission  to  the  guidance  and  the  application  procedure  helped  applicants  to  produce 
conform proposals. 

7.4 COHERENCE 

Coherence with other EU programmes 

To avoid overlaps between CIPS programme and other similar EU programmes, some internal 
mechanisms such as consultations and inter-service consultations were established. Moreover, 
a representative of DG Enterprise for the security research funding in Framework Programme 
7  was  involved  in  the  evaluation  of  proposals  for  the  first  two  calls  for  CIPS  proposals 
2008/2009  to  the  extent  needed  to  avoid  duplication  of  what  was  done  under  Security 
Research in Framework Programme 7. 

The Framework Programme 7 and the Civil Protection Financial instrument were selected for 
analysis and comparison with the CIPS programme. The coherence between different funding 
instruments can be multi-dimensional in terms of: (i) objectives and themes, (ii) programme 
management and eligible actions, and (iii) target groups. 

Coherence with Framework Programme 7 

Overall,  the  evaluation  study  concluded  that  the  activities  implemented  under  Framework 
Programme 7 in security were coherent with the CIPS programme. Indeed, this latter focused 
on prevention and protection measures against terrorist attacks as well as crisis management 
whereas projects under Framework Programme 7 were more specifically oriented to conduct 
research  on  development  of  new  technology  solutions  for  civil  protection,  including  risks 
arising from terrorist attacks. 

The JRC also ensured synergies between the two programmes. More specifically, the actions 
carried  out  with  the  JRC  under  the  CIPS  programme  and  the  activities  of  the  JRC  on  CIP 
carried out under FP7/H2020 were merged into two JRC actions (first CIP action, and as of 
2013 CIP and ERN-Sys actions). Under the administrative arrangements between DG HOME 
and  JRS,  some  actions  under  CIPS were  carried  out  by  the JRC  and  included  in  the  annual 

32 

 
 
work programmes. However, given that the JRC also had its own annual work programmes 
(based  on  the  Framework  Programme  7);  actions  funded  through  CIPS  were  designed  in 
coherence with the JRC’s own priorities. 

No overlaps were identified in terms of target groups of CIPS and Framework Programme 7. 
Furthermore,  the  mapping  of  CIPS  projects  revealed  that  a  number  of  other  groups  of 
beneficiaries  were  targeted,  including  the  private  sector,  NGOs,  the  general  public, 
researchers and universities. 

In  terms  of  eligible  actions,  the  evaluation  identified  possible  incoherencies  and  overlaps 
between the eligible actions of CIPS and Framework Programme 7. As stated above, actions 
funded under the Framework Programme 7 security area included, for instance, ensuring the 
security  of  citizens,  infrastructure  and  utilities  through  technology  solutions  for  civil 
protection  and/or  analysing  and  securing  critical  infrastructure.  There  was  scope  for 
complementarity  with  CIPS  activities  on:  ‘development  and  transfer  of  technology  and 
methodology’; ‘analytical, monitoring and evaluation activities’ and ‘operational cooperation 
and coordination’. 

Coherence with the Civil Protection Financial Instrument 

On  the  Civil  Protection  Financial  instrument,  its  actions  specifically  related  to  research  on 
prevention  measures  and  disaster  response  and  management,  hence  coherence  between  the 
two instruments was also ensured. 

No  overlaps  were  identified  in  terms  of  target  groups  of  CIPS  and  the  Civil  Protection 
Financial Instrument.  Furthermore, the mapping of CIPS projects revealed that a number of 
other  groups  of  beneficiaries  were  targeted,  including  the  private  sector,  NGOs,  the  general 
public, researchers and universities. 

On  actions  funded  under  the  Civil  Protection  Financial  instrument,  they  were  set  out  in 
Article 4  of  Council  Decision  2007/162/EC  Euratom  and  included  e.g.  studies;  training, 
exercises,  workshops,  exchange  of  staff  and  experts;  public  information,  education  and 
awareness  raising  and  associated  dissemination  actions  and  monitoring,  assessment  and 
evaluation activities. The scope for complementarity between CIPS and the Civil Protection 
Financial instrument was ensured in all eligible actions under CIPS. 

Overall,  little  to  no  evidence  of  overlap  was  observed  between  the  three  programmes 
compared, i.e. CIPS, Framework Programme 7 and the Civil Protection Financial instrument 
due to their differences in terms of thematic focus, eligible actions and eligible stakeholders 
and target groups. Furthermore, good coordination between different Directorate-Generals in 
the Commission (including DG CNECT, DG ECHO, DG MOVE, DG ENER, DG JRC, etc.) 
through  CIPS  inter-service  group  was  ensured.  Therefore  links  and  complementary  were 
enhanced. 

Synergies  between  the  results  of  actions  funded  under  the  CIPS  programme  and  the 
results of similar actions supported under other EU programmes 

The assessment of this evaluation question was limited by a lack of data collected. It is based 
on  the  stakeholders'  responses  to  an  online  survey.  A  total  number  of  20  replies  were 
received.  Bearing this limitation in mind, 14 out of 20 respondents reported that they knew 
about  other  activities  implemented  outside  the  project  and  covering  similar  objectives  only 

33 

 
 
 
partially.  Activities  identified  as  covering  similar  objectives  were  for  instance  other 
Framework  Programme  7  funded  projects  and/or  other  projects  under  ENISA58  and  were 
implemented at national, EU and international levels. Examples of links were provided only 
by  six  respondents  who  replied  to  this  question  and  included,  for  instance,  development  of 
networks and exchange of best practice and links related to the exchange of results and mutual 
update. 

Coherence with other related actions supported by other national resources 

The assessment of this evaluation question was limited by a lack of available data due to the 
fact that, according to  a number of  Commission  officials  interviewed, national actions were 
not  systematically  reviewed  on  their  potential  coherence/similarities  during  the  design  and 
implementation  of  CIPS.  From  the  interviews  with  some  Member  States  where  no 
coordinating organisations were presents, it emerged that the scope of the CIPS programme, 
maybe because of its broadness, was lacking clarity.  

Furthermore, certain of these Member States reported that there was no need for extra funding 
since  CIP  programmes  were  already  established  at  national  level,  hence  no  risks  of 
duplication  occurred.  Although  the  data  collected  for  this  evaluation  study  showed  little 
evidence  of  coherence  between  CIPS  and  the  national  frameworks,  overall,  it  can  be 
concluded that Member States were aware of the scope of the programme as they received a 
draft of the work programme and were invited to provide their comments. In a few cases, the 
evaluation  identified  connections  between  CIPS  and  national  actions.  For  instance,  the 
German federal cabinet in 2009 approved the national KRITIS-strategy, and this was in line 
with the CIPS programme59 thus coherences were ensured.  

7.5 EU ADDED VALUE 

EQ B9: To what extent would the beneficiaries under the CIPS programme (Member States, 
universities, institutes, associations, etc.) be able to carry out the activities necessary for the 
implementation of the EU policies in the prevention, preparedness and consequence 
management of terrorism and other security related risks without the support of the CIPS 
programme? 
Overall, stakeholders consultations showed that the added value of the programme is closely 
linked to the ‘importance’ of EU funding for the organisations involved and to its ability to 
foster  cross-border  cooperation  which  would  often  not  have  taken  place  if  projects  were 
supported  through  national  funding  alone.  Findings  suggest  that  organisations  struggled  to 
identify  and  access  national  funding  opportunities  to  implement  the  programme’s  activities 
hence it can be assumed that a significant part of them would have not been developed in the 
absence of the CIPS funding. 

58   ENISA is the European Network and Information Security Agency: https://www.enisa.europa.eu/ 
59   https://www.bmi.bund.de/SharedDocs/Downloads/DE/Broschueren/2009/kritis.pdf?__blob=publicationFile 

34 

 
 
 
 
                                                            
Coordinating organisations confirmed that, during the programming period, only few national 
funding opportunities were available to develop the activities implemented under the various 
policy areas of the Programme. 9 out of 13 respondents to a dedicated survey stated that the 
project activities would not have been implemented without EU funding. Only 4 respondents 
reported  that  the  project  could  have  been  fully  financed  through  other  funding  sources. 
Nevertheless, these respondents stressed the fact that they preferred to use the CIPS funding 
anyway,  as  the  latter  provided  them  with  the  opportunity  to  work  with  partners  in  other 
countries and to have an impact at EU level. Out of the 9 respondents who reported that they 
could not have funded their projects through other sources, 8 indicated that it was due to the 
unavailability or insufficiency of other funds whereas 3 explained that other potential funding 
sources typically did not support their project’s aims.  

The CIPS programme had a strong transnational dimension as reflected in its objectives which 
were  in  line  with  the  EU  objectives  on  the  development  of  EU  policies  in  the  prevention, 
preparedness and consequence management of terrorism and other security-related risks and 
on enhancing coordination and cooperation between relevant actors at EU level in protection 
of critical infrastructure. 

However, the EU added value may have been reduced by the fact that, as mentioned above60, 
CIPS  projects  were  mostly  led  by  coordinating  organisations  from  a  limited  number  of 
Member States (mostly Italy, Spain and to a lesser extent the United Kingdom and  France). 
Despite  the  Commission's  efforts  in  trying  to  involve  applicants  from  other  Member  States 
through  regional  information  days,  the  geographical  spread  remained  unchanged.  The 
evaluation found different reasons for the significantly varying degrees of uptake across the 
Member States and included, besides issues such as the ability of potential applicants to draft 
high-level  quality  proposals,  also  the  pro-activeness  of  Member  States'  Committee 
representatives promoting CIPS amongst relevant organisations. 

Conversely, a better geographical balance was reached with partner organisations where 30 or 
more of them located in 3 Member States (i.e. Italy, Poland and Spain) and 15 or more in 5 
Member  States  (i.e.  Czech  Republic,  France,  Germany,  Romania  and  the  UK).  Overall, 
partner organisations located in 21 of the 28 Member States participated to five or more CIPS 
projects,  achieving  a  better  geographical  spread  with  regard  to  partner  organisations  than 
coordinating ones.  

EU added value was enhanced by the fact that the vast majority of CIPS projects were cross-
border,  with  73 %  of  projects  including  at  least  one  organisation  from  a  Member  State 
different  than  the  Member  State  in  which  the  coordinating  organisation  was  located.  From 
2009 the eligibility criteria on cross-border projects became stricter to ensure that each cross-
border partner had an active role in the activities implemented. 

60   See Section 7.1. 

35 

 
 
                                                            
the  partnerships 
Overall,  coordinating  organisations 
established  through  projects.  From  the  survey  it  emerged  that  the  majority  of  these 
organisations considered the EU added value of the partnerships as positive a. For instance, it 
contributed to improved knowledge and expertise on the topic of the project (8 out of 14) or it 
helped to gain a better understanding on how neighbouring countries work (6 out of 14). 

interviewed  assessed  positively 

Furthermore,  coordinating  organisations  interviewed  assessed  positively  the  partnerships 
established  through  projects  and  considered  it  brought  EU  added  value.  Examples  of  EU 
added  value  were  the  possibility  to  collaborate  in  real-life  scenarios,  in  particular  when  it 
concerns  cross-border  cooperation  (i.e.  joint  patrols  to  reduce  terrorism  and  other  security 
related  risks)  and  the  exchanges  and  training  of  staff  from  different  countries  engaged  in 
protection of CI. The most important EU added value was found in the increase of knowledge 
and expertise related to a certain topic and the increased knowledge of EU policies related to 
this  topic,  according  to  13  respondents  (out  of  14  responding  to  the  online  survey  for 
coordinating organisations). 

Similarly, the analysis of the sample projects showed that, overall, the activities implemented 
and the outputs provided by both cross-border and national CIPS projects contributed to the 
EU dimension of the programme. 

69 % of the projects were dedicated to the development of common methodologies or tools, 
for  instance  on  risk  assessment.  The  activities,  implemented  by  national  and  cross-border 
partnership, contributed to the EU added value as they produced outputs to be used in other 
Member  States.  As  an  example  of  EU  added  value  projects,  the  Threat-Vulnerability  Path 
Identification for Critical Infrastructures (THREVI2) was a national project which contributed 
to  the  implementation  of  Directive  2008/114/EC  by  promoting  the  development  of  risk 
assessment  methodologies.  Another  national  project  which  developed  methodologies  to  be 
used at EU level was the ‘Interactive Risk Assessment in critical infrastructure based on the 
Integrated  Geographic  Information  System’.  It  developed  a  practical  tool  for  protection  of 
critical infrastructure and identified interdependencies between sectors in CIP. 

8  CONCLUSIONS 

This  section  summarises  the  evaluation  findings  and  provides  an  assessment  of  the  CIPS' 
overall role in the implementation of prevention, preparedness and consequence management 
of terrorism and other security-related risks. 

Relevance 

Overall, the evaluation showed that the CIPS programme was of continuous relevance to the 
prevention, preparedness and consequence management of terrorism and responded to a real 
need  for  cross-border  cooperation  and  coordination  in  the  above  mentioned  areas.  The 
stakeholders that were consulted reported that such cooperation would not have been financed 

36 

 
 
 
through  alternative  sources  of  funding,  such  as  national  budgets,  due  to  the  financial  crisis 
that occurred during the programme period. 

The  relevance  of  the  CIPS  programme  was  also  ensured  by  the  process  of  priority-settings: 
these were identified by the Member States and competent Directorate-Generals on a yearly 
basis and included in the annual work programmes. Therefore an updated and more detailed 
analysis  of  needs  was  ensured.  Most  CIPS  coordinating  organisations  and  partner 
organisations responding to the online survey considered the priorities set by the Commission 
in  the  calls  for  proposals/annual  work  programmes  as  relevant  to  the  specific  needs  of  the 
target group, at EU level and national level. However, the demand-driven design of the annual 
work programme resulted in a significant geographical imbalance in terms of the number of 
projects funded per coordinating organisation in each Member State, although a slightly more 
balanced spread was achieved for the partner organisations.  

In  order  to  allow  Member  States  more  equal  access  to  funding  and  to  improve  their 
participation, a shift towards shared management mode was introduced for the successor fund, 
ISF-Police. Unlike CIPS, which was implemented only under direct management, ISF-Police 
is being implemented on the basis of a combination, with around 60% of the available funding 
being  allocated  to  national  programmes  under  shared  management  and  40%  allocated  to 
Union  actions  under  direct  management.  This  change  is  expected  to  have  an  impact  on  the 
efficiency and effectiveness of the Commission’s management of the ISF Programme.  

The  evaluation  also  identified a  time  lag  between  the  formulation  of  priorities  under  AWPs 
and their actual implementation; this was due to the fact that the AWPs reflected the priorities 
set in the previous year, which meant that there was a one year difference between the setting 
of  priorities  and  their  implementation,  which  at  times  hindered  the  programme’s  relevance. 
This  was  partly  addressed  in  the  successor  fund,  ISF-Police,  through  the  inclusion  of 
emergency assistance “to address urgent and specific needs”.61 

Effectiveness 

Bearing  in  mind  the  limitations  of  the  evaluation  study,  it  can  still  be  said  that  the  CIPS 
programme broadly achieved its general and specific objectives. The in-depth analysis of the 
results, outcomes and impact (where this was evident) of the sample of CIPS projects and on 
the views gathered from the stakeholders consulted, showed that CIPS contributed positively 
on critical infrastructure protection. However, when assessing the impacts of the programme, 
its  relatively  small  budget  and  wide  scope  (resulting  in  a  high  number  of  relatively  small 
values of projects) should be taken into account. An important element of a number of CIPS 
projects  was  the  focus  on  interdependencies  and  preventing  ‘cascading  effects’  in  case  of 
disruption and destruction of critical infrastructures during a terroristic attack or other security 
related  risks,  thus  improving  its  overall  contribution  to  critical  infrastructure  protection. 
However, the results of CIPS projects on EU-wide cooperation and coordination on CIP were 
evaluated by the Commission as not very effective, with a few peaks of excellence. 

61 

 Article 10, Regulation No 513/2014. 

37 

 
 
 
 
                                                            
Certain issues which hindered the assessment of effectiveness (and consequently of efficiency 
and EU added value) were also identified. Given the relative newness of CIPS and the lack of 
capacity at the time within the Commission to closely follow all projects from start to end, the 
absence  of  a  dedicated  technical  assistance  budget  that  could  have  supported  a  monitoring 
system to collect and analyse data on (financial) progress, outputs and results, has negatively 
affected  the effectiveness of the Programme. Key elements of  this monitoring  system could 
have  been,  for  example,  the  use  of  monitoring  visits  and  the  use  of  an  IT  system  to 
systematically record data on financial progress, outputs and results. This has been addressed 
to  a  limited  extent  under  the  successor  fund,  ISF-Police,  through  the  possibility  for  the 
financial  instrument  to  contribute  annually  to  technical  assistance  activities,  which could  be 
used  for  the  development  of  a  monitoring  system.    However,  under  ISF-Police  a  problem 
remains  in  this  area,  given  that  the  technical  assistance  available  cannot  be  used  for  staff 
contracting  and  the  development  and  implementation  of  a  monitoring  system  requires  also 
additional human resources on both the Member State and the Commission side. This remains 
to be addressed in the future MFF. 

Furthermore, the evaluation found that the creation of a central repository containing detailed 
data  on  individual  CIPS  project  results  and  outputs  would  have  greatly  enhanced  the 
effectiveness of the programme. For example, methodologies developed to investigate critical 
infrastructures  protection,  best  practices,  training  materials,  etc.  could  have  been  compiled 
centrally and thus, their potential  multiplier effect and continuous usage for other interested 
stakeholders  in  other  Member  States  could  have  been  maximized.  In  consequence,  it  is 
recommended  that  for  projects  conducted  under  the  successor  funding  instrument,  a 
repository of the results and outputs of such projects could be created. This has not yet been 
put into practice under ISF but should be taken into consideration for the next MFF. 

Moreover, evidence gathered through the stakeholders' consultation showed that results based 
on potential for transferability to other sectors (and other Member States) and readiness to be 
used  by  critical  infrastructure  owners  and  operators,  in  contrast  to  prototypes  and  models, 
were  perceived  as  particularly  valuable  and  contributed  to  the  increase  of  the  overall 
effectiveness of the programme. 

In consequence, for the future, similar programmes should be carried out in order to motivate 
coordinating organisations to work towards the development of demonstrators or prototypes 
(to  be  developed  as  market-ready  products)  which  are  based  on  the  results  of  previous 
projects. 

Efficiency 

In  general, CIPS was found  to  be efficient  to a large extent. The vast majority of consulted 
stakeholders  perceived  EU  funding  to  have  been  sufficient  and  the  value  for  money  was 
perceived  as  good  to  very  high.  No  significant  differences  in  the  cost-effectiveness  of 
individual  projects  were  found  following  a  comparison  of  similar  financial  inputs  to  their 
outputs  and  results.  Some  private  stakeholders  mentioned  the  higher  level  of  administrative 
burden in comparison with the FP7 rules they were accounted to.  

A  significant  upward  trend  in  the  absorption  of  funds  spent  on  grants  was  observed,  rising 
from 68%  in 2007 to  88%  in  2013, with  83%  across  the 2007-2013 period. Most probably, 
this was due on the one hand, to the changes introduced by the Commission during the CIPS 

38 

 
 
 
 
programming period in order to increase the overall efficiency of the Programme and, on the 
other hand, thanks to the fact that stakeholders became more familiar with the Programme. 

It  should  also  be  pointed  out  that,  overall,  the  assessment  of  the  data  on  KPIs  showed  a 
satisfactory performance with regard to the average time needed to award, to contract and to 
make  pre-financing  payments.  However,  in  terms  of  timing  for  processing  final  payment 
requests, from the data collected it was evident that, in particular for the earlier years of the 
Programme, it resulted that the majority of these were not processed on time. 

Pertaining to the monitoring of project outputs and results, the evaluation found that a more in 
depth reporting from project beneficiaries would have strengthened the overall efficiency of 
the Programme. Indeed, in terms of improving the monitoring of project results on the basis of 
the required reporting, it was concluded also that a qualitative assessment of the results of the 
Programme was missing due to the absence of a peer review on project results conducted by 
external  experts.  Although  under  the  current  Fund  this  has  slightly  improved  as  Member 
States  have  to  submit  an  Annual  Implementation  Report  under  Shared  Management  and 
Interim  and  Final  Reports  under  Direct  Management,  the  absence  of  a  peer  review  still 
represents a shortcoming that should be addressed under the next generation of Funds.  

As  already  stated  under  relevance,  with  regards  to  dissemination  of  projects  outputs  and 
results,  consulted  experts  and  coordinating  organisations  recognised  that  the  absence  of  a 
central repository bringing together the findings of CIPS projects informing stakeholders on 
available results has decreased the effectiveness and efficiency of the Programme to a certain 
extent. It is therefore recommended to set up such a central repository bringing together the 
outputs/results of the projects in future similar programmes.  

Coherence 

The  CIPS  investments  made  under  the  2007-2013  annual  work  programme  were  coherent 
with activities funded under other EU Funds, namely Framework Programme 7 and the Civil 
Protection  Financial  instrument.  Indeed,  little  to  no  evidence  of  overlap  occurred  between 
funding instruments due to the differences of these Programmes in terms of thematic focus, 
eligible  actions  and  eligible  stakeholders  and  target  groups  whilst  demonstrating  the  good 
coordination  between  different  Directorate-Generals  in  the  Commission  (including  DG 
CNECT,  DG  ECHO,  DG  MOVE,  DG  ENER,  DG  JRC,  etc.)  through  CIPS  inter-service 
group. In this way, links and complementary were enhanced. 

The JRC acted as a ‘synergy vehicle’ between CIPS and Framework Programme 7. Actions 
funded through CIPS were designed in coherence with the JRC’s own priorities.  

On coherence with initiatives at national level, the data available was not sufficient to assess 
the extent to which this was achieved. 

EU added value 

The evaluation showed that due to a lack of national funding for cross-border cooperation in 
many Member States, a significant part of the activities developed under CIPS would not have 
been developed at all or would have not achieved the same results, in particular not at a cross-

39 

 
 
 
 
border level. Additionally, the high added value of CIPS was also linked to the high level of 
co-funding it provided (which could go up to 90 %). 

As reflected in the objectives of the Programme, CIPS had a strong cross-border dimension. 
However,  as  mentioned  above,  a  major  weakness  identified  by  the  evaluators  related  to 
funding  distribution  that  was  uneven  and  concentrated  in  a  few  Members  States  with 
coordinating  organisations  that  were  mainly  located  in  Italy  and  Spain.  Although  the 
Commission  took  measures  to  boost  the  geographical  spread,  particularly  through  the 
organisations of information days in the Member States, the submission of applications was 
still  very  dependent  on  factors  such  as  the  pro-activeness  of  Member  States’  Committee 
representative’s  knowledge  and  capacity  of  organisations  to  submit  high-quality  proposals 
and  their  compliance  with  the  eligibility  criteria.  This  geographical  imbalance  has  been 
addressed through the shift towards shared management. Regarding the national/transnational 
characteristics  of  projects,  the  shift  towards  shared  management  has,  however,  meant  that 
many  projects  implemented  by  Member  States  under  ISF-P  tend  to  have  less  of  a 
transnational  dimension  than  projects  previously  conducted  under  CIPS,  which  due  to  the 
Direct management mode naturally had a higher transnational dimension.  

40 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ANNEX 1 — PROCEDURAL INFORMATION 

Leading Directorate-General 
Participating Units of DG HOME 

Participating DGs  

DG HOME 
A2 — Legal Affairs 
B2 — Visa Policy and document security 
B3 — Information Systems for Borders and 
Security 
C2 — Border Management and Schengen 
E1 — Union Actions 
E2 — National programmes for South and East 
Europe, evaluation, AMF/ISF Committee 
E3 — National programmes for North and West 
Europe, budget, MFF, agencies 
Secretariat-General 
DG BUDG 
DG ENV 
DG ENERGY 
DG GROW 
DG JUST 
November 2015 
2016/HOME/055 
Contract signed in August 2016 with ICF 
Consulting Services Limited 

Roadmap approval 
Agenda Planning  
External consulting firm 
specialised in evaluation 
Number of steering group meetings  5 
Last deliverable handed in 
Approval of the final report by 
Steering Group 

January 2018 
January 2018  

41 

 
 
 
 
 
 
ANNEX 2 — STAKEHOLDERS’ CONSULTATION 

1. Overview 

The  ex  post  evaluation  of  the  prevention,  preparedness  and  consequence  management  of 
terrorism  and  other  security  related  risks  (CIPS)  2007-2013  programme 
included 
consultations with several stakeholders. The consultations were conducted by the Commission 
and  by  an  external  consultancy  specialised  in  evaluation  in  the  framework  of  their  contract 
with  the  Commission  to  provide  study  on  the  ex  post  evaluation  of  the  implementation  of 
CIPS  (‘the  external  evaluators’).  This  Annex  provides  an  overview  of  the  consultation 
processes  and  the  type  of  stakeholders  consulted.  Moreover,  it  presents  the  results  of  these 
consultations. The evaluation assessed the relevance, effectiveness, efficiency, coherence and 
EU added value (‘evaluation criteria’) of the CIPS programme. 

1.1 The public consultation conducted by the Commission 

From  14 July  2016  to  15 November  2016,  the  Commission  carried  out  an  Internet-based 
public  consultation  on  the  CIPS  2007-2013  in  the  form  of  an  online  questionnaire. 
Contributions  were  sought  from  individuals  (experts,  beneficiaries),  public  authorities, 
intergovernmental  and  non-governmental  organisations,  social  partners  and  civil  society, 
academic institutions, international organisations, EU Institutions and Agencies, based in EU 
Member States and non-EU countries.  

The aim of the consultation was to collect views and opinions on the results and impacts of 
actions  co-financed  by  the  CIPS  programme  in  2007-2013  and  to  assess  their  relevance, 
effectiveness,  efficiency,  coherence  and  EU  added  value.  The  final  number  of  respondents 
was  rather  low:  six  participants.  Two  respondents  responded  on  behalf  of  non-profit 
organisations (Italy), one responded as a beneficiary of the programme (United Kingdom) and 
three as EU citizens. The EU citizens were from Germany (two) and one from France. Given 
this  low  response  rate,  contributions  cannot  be  considered  as  being  representative  of  the 
targeted  stakeholders.  However,  the  results  may  provide  additional  insights  that  will  be 
presented  below  together  with  the  findings  of  the  other  consultations.  The  fact  that  the 
information available on the identities of the participants is based on self-reported values and 
cannot be verified should be considered as a further limitation. 

1.2 The consultations conducted by the external evaluators 

In  addition  to  the  consultation  conducted  by  the  Commission,  the  external  evaluators 
conducted targeted consultations with range of stakeholders using four kinds of interviews — 
online surveys, stakeholder consultations, thematic case studies, expert workshops and expert 
input. This section provides an overview of each type of interview and consultation process 
that  took  place  in  each  case.  The  unit  responsible  for  the  evaluation  (DG  HOME,  Unit  E2) 
supported the access to relevant EU level stakeholders. Stakeholders were contacted via email 
introducing  the  study  and  the  importance  of  their  contribution,  including  a  letter  of 
introduction from DG HOME. 

1.2.1 Online surveys 

42 

 
 
 
The external evaluators launched three online surveys at the end of December 2016  — with 
coordinating organisations, with partner organisations and with project participants. 

Online survey with coordinating organisations 

The  survey  ran  for  four  weeks  —  from  4 November  2016  to  5 December  2016  and  it  was 
made available in five languages: English, French, German, Italian and Spanish. The goal of 
this  survey  was  to  invite  all  coordinating  organisations  of  CIPS  projects  to  participate  by 
answering questions on the evaluation criteria (relevance, effectiveness, efficiency, coherence, 
EU  added  value).  The  evaluation  aimed  to  invite  all  coordinating  organisations  of  CIPS 
projects  to  participate  in  the  survey.  Out  of  the  131  total  CIPS  contacts,  108  contacts  were 
invited  to  complete  the  survey.  The  total  number  of  respondents  to  the  survey  was  20 
representing a response rate of 19 %. Out of these 20 responses, 15 were complete and five 
were incomplete responses. 

Online survey with partner organisations 

The  coordinating  organisations  were  asked  to  send  this  survey  to  the  relevant  partner 
organisations of CIPS projects. Similarly to the online survey with coordinating organisations, 
this one was open from 4 November 2016 until 5 December 2016 and it was available in the 
same  five  languages  as  mentioned  above.  A  total  number  of  18  respondents  replied  to  the 
survey of which 8 were complete responses and 10 incomplete responses. 

Online survey with project participants 

This online survey ran for 12 weeks — from 1 December 2016 until 28 February 2017. It was 
also  available  in  English,  French,  German,  Italian,  and  Spanish.  The  participants  were 
identified with the assistance of the coordinating organisations which forwarded the survey to 
the participants of their CIPS projects. The total number of replies of this online survey was 
15. However, 13 of the respondents were participants from the same project, so the results of 
this  survey  were  treated  with  caution  taking  into  account  the  strong  bias  to  this  particular 
project. 

1.2.2 Stakeholder consultation 

The  external  evaluators  carried  out  37  interviews  with  five  types  of  stakeholders.  The  five 
types  of  stakeholders  that  were  interviewed  for  the  purposes  of  the  evaluation  were  — 
scoping interviews with Commission officials (9), Commission officials responsible for other 
EU  programmes  (2),  European  and 
international  organisations  (1),  Member  State 
representatives (16), additional interviews with CIPS coordinators (9). The external evaluators 
carried  out  those  interviews  in  a  semi-structured  way  based  on  questionnaire  developed  for 
each  stakeholder  type. The  interviews  lasted  one  hour  and  were  conducted  via  telephone  or 
face-to-face. 

1.2.3 Case studies 

Following the request of DG HOME, the external evaluators carried out thematic case studies. 
The  aim  of  those  case  studies  was  to  examine  more  closely  the  result  sand  impacts  in  one 

43 

 
 
 
 
specific  thematic  area,  and  where  possible  linking  them  to  wider  context  at  EU  level.  Two 
case studies were carried out in which three Member States participated: 

Multi-country  case  study  (Germany  and  Hungary):  development  of  tools  and 

- 
protection; 

- 

Case study (Slovenia): risk assessment. 

In total, six interviews were carried out as part of the case studies’ data collection, including 
four  interviews  with  coordinating  organisations  of  CIPS  projects,  and  two  with  partner 
organisations. 

1.2.4 Expert workshop and expert input 

The external evaluators organised two expert workshops. Both were held in Brussels — one 
on  23 September  2016  and  one  on  6 March  2017.  Both  workshops  were  carried  out  with 
experts  involved  in  the  evaluation.  The  second  workshop  was  followed  by  written 
contributions from the experts on the findings presented in the draft final report.  

2. Results 

The most relevant results of the consultations carried out by the Commission and the external 
evaluators  are  grouped  in  this  section  and  presented  according  to  the  following  evaluation 
criteria: relevance, effectiveness, efficiency, coherence, EU added value. 

2.1 Relevance 

CIPS’s  relevance  assessment  aimed  to  find  out  the  extent  to  which  the  objectives  and  the 
design of the programme corresponded to the needs problems and issues in the protection of 
people and critical infrastructure from terrorism and other security related risks in 2007-2013. 
On the relevance of the objectives, stakeholders consulted consider that the CIPS programme 
was of continuous relevance to the prevention, preparedness and consequence management of 
terrorism and other security related risks throughout 2007-2013.  In addition, the Commission 
public  consultation  in  Question  7  asked  the  participants  to  what  extent  the  projects  and  the 
activities funded in their country of residence addressed the needs related to management of 
terrorism  and  other  security  related  risks.  Half  of  the  respondents  replied  with  ‘to  a  limited 
extent’,  one  EU  citizen  replied  ‘significantly’  and  the  beneficiary  of  the  programme 
mentioned that the projects and activities addressed the needs only ‘partially’. 

On the relevance of the design of the programme, the evaluation explored whether the design 
sufficiently  corresponded  to  the  needs  related  to  the  prevention,  preparedness  and 
consequence  management  of  terrorism  and  other  security  related  risks  over  2007-2013. 
Taking  into  account  the  design-driven  design  of  the  programme  that  led  to  a  significant 
imbalance  of  geographical  distribution  of  the  programme,  the  results  of  the  consultation 
showed that there are couple of reasons for the low level of participation: the administrative 
burden, the fact that CIPS was not relevant to some national context, unfamiliarity with the 
programme, lack of institutional capacity, lack of contacts and partners in other EU Member 
States.  

Moreover,  the  broad  thematic  coverage  of  the  programme  was  overall  welcomed  by  the 
consulted stakeholders as it allowed for flexibility throughout the funding period. The funding 

44 

 
 
mechanisms  of  CIPS  were  considered  appropriate  by  the  majority  of  the  stakeholders.  The 
involvement of JRC in the implementation of the programme was seen as positive. Those that 
expressed a desire for a more flexible programme design reported that some of the eligibility 
criteria for obtaining CIPS were a reason for some organisations not to apply. 

2.2 Effectiveness 

The  evaluation  questions  under  this  criterion  aim  to  assess  the  extent  to  which  CIPS  was 
effective  in  achieving  its  general  and  specific  objectives.  Question  4  of  the  Commission’s 
public consultations asked participant whether, based on their experience, the actions financed 
by  the  CIPS  in  their  country  were  consistent  with  the  objectives  of  the  Fund.  Four  of  the 
respondents replied that they ‘do not know’, one EU citizen replied with ‘to a limited extent’, 
while the beneficiary of the programme replied ‘partially’. 

The  evaluation  of  the  effectiveness  of  the  implementation  of  CIPS  projects/programme 
suggested that the majority of the CIPS-funded projects achieved their planned objectives and 
results.  Furthermore,  the  stakeholder  consultation  confirmed  that  JRC  played  an  important 
role in implementing the EPCIP. 

In terms of effectiveness in achieving the general and specific objectives of the programme, 
the evaluation showed that they were achieved and the programme contributed positively to 
the policy are of critical infrastructure protection. One of the limitations, however, is the fact 
that  the  programme  had  relatively  small  budget  and  a  wide  scope  which  led  to  a  higher 
number of small value projects.  

2.3 Efficiency 

The efficiency of CIPS aimed at measuring the extent to which the costs of the actions funded 
under  the  programme  were  justified  compared  to  the  outputs.  The  evaluation  focused  on 
examining the Commission’s management of the programme, the financial efficiency of the 
programme (the extent to which the resources were used efficiently), and the human resource 
efficiency. 

Overall,  the  majority  of  stakeholders  consulted,  reported  perceiving  some  degree  of 
administrative burden resulting from participation to the programme. However the number of 
respondents was very low. The majority of those that responded to the survey for coordinating 
organisations reported a significant (five) or very significant (three) administrative burden. A 
total of two respondents (out of 14) reported experiencing no administrative burden at all. On 
the  Commission’s  management  of  the  application  process  the  majority  (10  out  of  14)  of 
respondents to the survey for CIPS coordinating organisations were satisfied. 

The  financial  efficiency  of  the  programme  was  based  on  analysis  of  the  financial  resources 
allocated, committed and spent under the CIPS programme and also the extent to which the 
financial  resources  were  sufficient.  Overall,  EU  funding  was  found  to  be  sufficient  by  the 
coordinating  organisations  to  implement  the  planned  activities.  In  few  cases,  stakeholders 
highlighted  the  need  for  additional  funding  and  the  underestimation  of  certain  costs,  for 
example  in  relation  to  staff.  A  few  coordinating  organisations  indicated  that  forecasting 
research  and  development  costs  was  difficult  and  that  these  were  ultimately  higher  than 
planned. 

45 

 
 
On  human  resource  efficiency,  the  Commission  management  was  overall  considered 
positively. The results of the consultations noted the existence of some issues in the technical 
project  follow-up,  guidance  and  interpretation  of  rules.  Overall  there  was  sufficiency  in  the 
amount of human resources to deal with the number of projects, however frequent internal re-
organisations taking place across the programming period have impacted on the efficiency of 
the management, especially with regard to the monitoring aspect of the programme. 

2.4 Coherence 

The  evaluation  criterion  of  coherence  refers  to  the  extent  to  which  the  intervention  did  not 
contradict  or  create  duplications  with  other  interventions  with  similar  objectives.  The 
assessment targeted coherence with other EU programmes in the same field, coherence with 
national  programmes  and  initiatives  and  synergies  between  results  of  CIPS  and  results  of 
similar action funder under other EU programmes. 

The  external  evaluator’s  analysis  indicates  that  CIPS  had  ample  opportunity  for  coherence 
with other EU programmes, especially with Framework Programme 7 and the Civil Protection 
Financial  Instrument.  The  stakeholder  consultation  confirmed  that  there  was  little  risk  of 
overlap with either Framework Programme 7 or the Civil Protection Financial instrument due 
to the differences of these programmes with CIPS. 

The results from the Commission’s public consultation did not provide for enough support to 
such finding. Question 10 of the consultation asked participants about the extent to which they 
consider  that  the  EU  funding  for  CIPS  programme  was  coherent  with  other  actions  in  the 
same area funded by other EU financial instruments. Three of the respondents replied with ‘I 
do not know’. Two respondents said that the EU funding for CIPS programme was coherent 
with other actions to a limited extent and one respondent believed that it was coherent.  

The external evaluators found little evidence of coherence between CIPS and other national 
actions. The public authority representatives of some of the Member States commented that 
they already had working CIP programmes and frameworks in their countries and thus, there 
was no particular need for additional funding. 

On  the  links  between  results  of  CIPS  and  results  of  similar  action  funded  under  other  EU 
programmes, the CIPS coordinating organisations who responded to an online survey partially 
confirmed  that  they  were  aware  of  other  activities  implemented  outside  the  project  and 
covering  similar  objectives.  Only  six  respondents  to  the  online  survey  for  coordinating 
organisations  replied  to  the  question  relate  to  types  of  synergies  that  were  established.  As 
examples links in development of networks and exchange of best practice, knowledge-sharing 
and  links  related  to  the  exchange  of  results  and  mutual  update  were  given.  In  addition,  the 
results from the survey conducted among partner organisations showed similar results. 

2.5 EU added value 

The external evaluators assessed the EU added value of CIPS based on three criteria-  cross-
border dimension, partnership aspects, and importance of the EU funding. 

While  examining  the  cross-border  dimension  of  the  EU  added  value  of  CIPS,  the  external 
evaluator  concluded  that  eligible  actions  under  CIPS  also  included,  besides  cross-border 
projects, national projects to prepare or complement cross-border projects and/or EU actions 
or  contribute  to  developing  innovative  methods  and/or  technologies  with  a  potential  for 

46 

 
 
transferability to actions at EU level or to another Member State. In addition, the evaluation 
shows  that  CIPS  projects  were  mostly  led  by  coordinating  organisations  from  a  limited 
number of  Member States  (mostly Italy  and Spain). For the partner organisations, however, 
there was a better geographical spread. Thus, the evaluator concluded that the EU added value 
could  have  been  higher  if  organisations  in  all  Member  States  had  participated  more.  The 
evidence  gathered  throughout  the  evaluation,  show  that  some  of  the  reasons  for  the 
significantly varying degrees of uptake across the Member States were various and included, 
besides issues such as awareness of the programme and the ability of potential applicants to 
draft  high-level  quality  proposals,  also  the  pro-activeness  by  the  individual  Member  States’ 
Committee  representative  promoting  CIPS  among  relevant  organisations  continued  in  part 
following completion of the project. Only three partnerships did not continue after the project 
was completed. 

The  EU  added  value  was  significantly  strengthened  by  the  partnerships  established.  During 
the interviews with the coordinating organisations the following examples of EU added value 
were  provided  —  possibility  to  collaborate  in  real-life  scenarios,  in  particular  when  it 
concerns  cross-border  cooperation  (i.e.  joint  patrols  to  reduce  terrorism  and  other  security 
related  risks)  and  the  exchanges  and  training  of  staff  from  different  countries  engaged  in 
critical  infrastructure  protection.  In  addition  the  online  survey  for  partner  organisations 
repeats  the  views  of  the  coordinating  organisations.  The  majority  of  stakeholders  consulted 
confirmed that their partnership fully or at least partly had continued after the funding ended 
through for example continued collaboration of delivering training or continuing cooperation 
in general. 

The importance of EU funding to carry out the activities is linked to the added value of the 
programme.  The  findings  of  the  external  evaluators  suggest  that  organisations  struggled  to 
identify  and  access  national  or  other  funding  opportunities  to  implement  the  envisaged 
activities. Thus, it could be presumed that without the CIPS funding a significant part of the 
activities developed under the CIPS programme would have not been developed. In addition, 
the  results  of  the  survey  carried  out  with  the  CIPS  coordinating  organisations  suggest  that, 
during the programming period, there were few national funding opportunities to develop the 
activities  implemented  under  the  various  policy  areas  of  the  programme.  Nine  out  of  13 
respondents indicated that the project activities would not have been implemented without EU 
funding.  

Moreover,  only  four  stated  that  the  project  could  have  been  fully  financed  through  other 
funding sources. Out of the nine respondents who indicated that they could not have funded 
their  project  through  other  sources,  eight  indicated  that  other  funds  were  not  available  (or 
insufficient) and three explained that other potential funding sources typically did not support 
their  project’s  aims.  Following  the  interviews  with  the  coordinating  organisations  was 
concluded that one of the main benefits of the CIPS programme was indeed the possibility to 
implement  specific  activities  which  were  unlikely  to  be  funded  by  national  or  alternative 
funding  sources  (e.g.  exploratory  research  activities,  large  scale  projects).  The  four 
coordinating  organisations  which  could  have  made  use  of  other  funding  indicated  to  have 
chosen CIPS as the latter provided them with the opportunity to work with partners in other 
countries and to have an impact at EU level. 

47 

 
 
 
 
ANNEX 3 — METHODOLOGY 

Study conducted by external evaluators 

The preparatory ex post evaluation study conducted by the external evaluators was regarded 
as  providing  the  most  robust  and  impartial  overview  possible  of  the  Fund  because  it  was 
highly structured: 

•  A  regular  and  transparent  dialogue took  place  between the Commission  departments 

and the contractors; 

•  The parameters of the contract were clearly set out and respected; 

•  All data sources were assessed and presented data are clearly labelled. 

Communication between the Commission and the external evaluator 

The  study's  progress  was  followed  by  an  Inter-service  Steering  Group  (ISG)  comprised  of 
officials  from  the  following  services:  DG  for  Migration  and  Home  Affairs,  the  DG  for 
Environment,  the DG for Development  and Cooperation, the DG for Humanitarian Aid and 
Civil Protection, the DG for Energy, the DG for Justice and Consumers, the DG for Internal 
Market, Industry, Entrepreneurship and Small  and Medium  Enterprises, the DG for  Budget, 
the Secretariat-General, the Joint Research Centre, the European Anti-Fraud Office, as well as 
the external evaluators with their partners and experts. Regular meetings took place between 
the  contractors  and  the  ISG  and  structured  feedback  (in  both  directions)  was  provided 
throughout the contract. This two-way dialogue was enriched by the active participation in the 
ISG of policy and implementation units and shadowed by horizontal units and the Secretariat-
General. 

Evaluation stages 

A structuring feature of the external evaluation was the segmentation of the tasks into clearly 
defined stages which were closely observed by all parties. These stages had been determined 
in  the  Terms  of  Reference.  Complying  with  the  Terms  of  Reference  made  the  study  itself 
more efficient and transparent. 

1.  Inception  and  refinement  of  the  methodological  approach:  The  overall  purpose  of  the 
Inception Phase was to lay the groundwork and further elaborate and validate the method 
of approach to this evaluation. 

2.  Data collection and analysis. 

3.  Formulation of conclusions and recommendations: This phase aimed at bringing together 

the data collected and to produce findings, conclusions and recommendations. 

4.  Revision and finalisation: The aim of the final phase was to quality assure the deliverables 

through revision and finalisation based on the comments of the Steering Group. 

Data sources 

1.  Primary data 

  Three  online  surveys,  with  (i)  coordinating  organisations;  (ii)  partner  organisations; 

and (iii) project participants. 

  Stakeholder consultations — 37 face-to-face and telephone interviews. 

48 

 
 
 
  Two thematic case studies. 

- Development of tools and protection in Germany and Hungary, and; 

- Risk assessment in Slovenia, 

  Open public consultation carried out by the Commission. 

  Expert workshops and expert input. 

2.  Secondary Data 

  A quantitative analysis of all CIPS projects based on a monitoring table provided by 

the Commission was undertaken. 

 

In-depth  analysis  of  a  sample  of  CIPS  projects (29  projects  representing  28 %  of  all 
104 finalised projects). 

  A literature review of EU policy baseline and context. 

  A literature review of the national baseline and context. 

Methodological limitations 

A  number  of  methodological  obstacles  were  encountered  throughout  the  evaluation  which 
affected, for a number of evaluation questions, the extent to which findings and conclusions 
could be formulated. 

Lack of a baseline 

The first key obstacle encountered in the evaluation was the fact that establishing a baseline 
for  measuring  the  level  of  prevention,  preparedness  and  coordination  of  crisis  management 
and security actions in the aftermath of a terrorist attack or of security incidents across the EU 
is  difficult.  Indeed,  no  methodologies  were  established  for  measuring  in  a  comprehensive 
manner most of the activities covered by actions funded via CIPS.  It has not been possible to 
establish direct causal links between the CIPS programme’s intervention or identified needs 
on  the  one  hand  and  observed  impacts  or  results  on  the  other  hand;  therefore,  effectiveness 
has  been  mostly  evaluated  in  terms  of  outputs  or  perceived  impacts  or  results–  which  were 
more easily measured and compared across MSs — rather than actual impacts or results. By 
the  time  the  official  baseline  was  adopted  with  the  Better  Regulation  Guidelines,  the  CIPS 
programme was already under implementation. 

Time lag between programme implementation and the ex post evaluation 

There was a time lag between the implementation of the programme (2007-2013) and the ex 
post  evaluation  (August  2016-August  2017).  This  meant  that  the  relevant  beneficiaries  and 
national  and  EU  authorities  and  experts  were  often  no  longer  working  in  the  same  position 
and  were  thus  difficult  to  reach,  which  is  reflected  in  a  low  response  rate  to  some  of  the 
consultations  (see  Annex  3).  The  external  evaluator  took  some  initial  measures  to  mitigate 
these difficulties, such as sending several email reminders to the persons surveyed, searching 
for alternative contacts when possible, raising the individual  partner organisation interviews 

49 

 
 
 
 
 
 
to two rather than one, and extending the deadline for the online survey carried out as part of 
the case studied. Overall, all evaluation questions have been answered to some extent, thanks 
to the use of triangulation of data where possible. Where the evidence base has been deemed 
to be too low to be reliable, this has been indicated. 

Monitoring, accessibility and consistency of project data 

Issues have been noted on the monitoring of project progress and final results which hindered 
the assessment of effectiveness, efficiency and EU added value. Especially given the relative 
newness  of  the  programme  and  the  lack  of  capacity  at  the  time  within  the  Commission  to 
closely  follow  all  projects  from  start  to  end,  a  dedicated  technical  assistance  budget  could 
have  supported  a  monitoring  system  to  collect  and  analyse  data  on  (financial)  progress, 
outputs and results. The effect of these limitations on effectiveness, efficiency and EU added 
value is further explained in Sections 7.2 and 7.5. 

While relevant information could be collected for all evaluation questions, a full overview of 
final financial data and project data, especially on final absorption rate was not consistently 
available across all projects, given that data on final payments was not available for projects 
which  were  not  yet  complete.  In  addition,  data  provided  by  the  Commission  and  the  data 
drawn from other sources were not always consistent. This has limited the overall findings of 
the  evaluation  to  some  extent.  Where  possible,  estimates  have  been  used  to  calculate 
absorption  rates  and  implementation  trends.  However,  the  figures  on  the  last  programming 
year should be used with caution. Given that no estimates were available for the JRC data, no 
absorption rate has been calculated for the JRC element of the procurement sector. 

Another limitation which affects the assessment of effectiveness is that the outputs/results of 
the  projects  were  not  centrally  collected  and  the  final  assessments  of  projects  by  the 
Commission were not carried out comprehensively, often consisting only of a brief paragraph 
on the overall project,  without elaborating on particular elements or  activities of the project 
and  their  potential  future  use.  In  addition,  no  peer  review  of  the  products/outputs  was 
undertaken.  In  addition,  results  could  not  be  measured  due  to  the  lack  of  ex  ante  targets.  It 
was not feasible to compare the ‘cost of projects’’ under CIPS to other similar projects due to 
the  absence  of  comparable  projects.  This  hindered  the  assessment  of  efficiency  and 
effectiveness to a large extent, as it was difficult to compare project costs and to measure the 
extent to which the same objectives were met by the different types of projects in the various 
areas addressed by CIPS. 

To  mitigate  these  limitations,  the  assessment  of  effectiveness  and  efficiency  had  to  rely 
largely on the qualitative data collected through the stakeholder consultations. This, combined 
with the absence of a baseline, proved to be a limitation in terms of methodology. 

50 

 
 
 
 
 
ANNEX 4 — LIST OF EVALUATION QUESTIONS 

Relevance 

EQB1.a: To what extent did the objectives of the CIPS programme correspond to the needs 
related to the prevention, preparedness and consequence management of terrorism and  other 
security related risks? 

EQB1.b:  To  what  extent  did  the  design  of  the  CIPS  programme  correspond  to  the  needs 
related to the prevention, preparedness and consequence management of terrorism and other 
security related risks? 

Effectiveness 

EQB2:  To  what  extent  did  the  CIPS  programme  contribute  to  supporting  Member  State 
efforts  to  prevent,  prepare  for  and  protect  people  and  critical  infrastructure  against  terrorist 
attacks and other security related incidents? 

EQB3: To what extent did the CIPS programme contribute to protection in areas such as crisis 
management, environment,  public health,  transport, research and technological development 
in the field of terrorism and other security related risks? 

EQB4.a: To what extent did the CIPS programme contribute to the protection of people and 
critical  infrastructure  […]  by  promoting  and  supporting  risk  assessments  on  critical 
infrastructure, in order to upgrade security? 

EQB4.b:  […]  by  promoting  and  supporting  the  development  of  methodologies  for  the 
protection of critical infrastructure, in particular risk assessment methodologies? 

EQB4.c: […] by supporting shared operational measures to improve security in cross-border 
supply chains, in compliance with EU regulations on competition within the single market? 

EQB4.d: […] promoting and supporting the development of security standards, and exchange 
of know-how and experience on protection of people and critical infrastructure? 

EQB4.e: […] promoting and supporting EU-wide coordination and cooperation on protection 
of critical infrastructure? 

EQB5:  To  what  extent  did  the  CIPS  programme  contribute  to  promoting  and  supporting 
exchange  of  know-how  and  experience  related  to  the  consequence  management,  in  order  to 
establish best practices so as to coordinate the response measures and to achieve cooperation 
between various actors in crisis management and security actions? 

EQB6: To what extent did the CIPS programme contribute to promoting joint exercises and 
practical scenarios in consequence management, including security and safety components, in 
order to enhance coordination and cooperation between relevant actors at the EU level? 

51 

 
 
 
 
Efficiency 

EQB7.a:  To  what  extent  were  the  results  of  the  CIPS  programme  achieved  at  a  reasonable 
cost in terms of financial resources deployed? 

EQB7.b:  To  what  extent  were  the  results  of  the  CIPS  programme  achieved  at  a  reasonable 
cost in terms of human resources deployed? 

EQB7.c:  What  kind  of  initiatives  or  approaches  were  adopted  to  simplify  access  to  and 
implementation of the actions funded by the programme (e.g. changed to shared management 
in ISF-P)? 

Coherence 

EQB8.a:  To  what  extent  was  the  CIPS  programme  coherent  with  other  related  actions 
supported by other EU programmes? 

B8.b: To what extent was the CIPS programme coherent with other related actions supported 
by other national resources? 

EQ  B8.c:  To  what  extent  is  it  possible  to  identify  synergies  between  the  results  of  actions 
funded  under  the  CIPS  programme  and  the  results  of  similar  actions  supported  under  other 
EU programmes? 

EU added value 

EQB9: To what extent would the beneficiaries under the CIPS programme (Member States, 
universities, institutes, associations, etc.) be able to carry out the activities necessary for the 
implementation  of  the  EU  policies  in  the  prevention,  preparedness  and  consequence 
management  of  terrorism  and  other  security  related  risks  without  the  support  of  the  CIPS 
programme? 

52 

 
 
 
 
ANNEX 5 — LIST OF ABBREVIATIONS AND COUNTRY CODES 

CIPS 

SSL 
ISF 
MFF 
CBRN-E 
CIP 
EPCIP  
SWD 
ECI 
CIWIN 

ICT 
LNG 
JRC 
AWP 
CEN 
MS 
EU 
ISEC 
TOR 
NCPs 
ERN-CIP 
ERN-SyS 
GIS 
CIP/R   
SLO 
EMAs   
ISF 
KPIs 
ENISA  

Prevention,  Preparedness  and  Consequence  Management  of  Terrorism 
and other Security-Related risks 
Security and Safeguarding Liberties 
Internal Security Fund 
Multiannual Financial Framework 
Chemical, Biological, Radiological, Nuclear and Explosives 
Critical Infrastructure Protection 
European Programme for Critical Infrastructure Protection  
Staff Working Documents 
European Critical Infrastructures 
Critical Infrastructure Warning Information Network 

Information and Communication technologies 
Liquefied Natural Gas 
Joint Research Centre 
Annual Work Programme 
European Committee for Standardisation 
Member State 
European Union 
Prevention and Fight against Crime 
Terms of Reference 
National Contact Points 
European Reference Network for Critical Infrastructure Protection 
European Reference Networks for Security and Standards 
Geographic Information System 
Critical Infrastructures protection and/or Resilience 
Security Liaison Officer 
Emergency Management Agencies 
Internal Security Fund 
Key Performance Indicators 
European network and Information Security Agency 

List of country codes 

AT 
BE 
BG 
CH 
CY 
CZ 
DE 
DK 
EE 
EL 

Austria  
Belgium 
Bulgaria 
Switzerland 
Cyprus 
Czech Republic 
Germany 
Denmark 
Estonia 
Greece 

53 

 
 
 
 
 
 
 
 
ES 
FI 
FR 
HR 
HU 
IE 
IS 
IT 
LT 
LU 
LV 
MT 
NL 
NO 
PL 
PT 
RO 
SE 
SI 
SK 
UK 

Spain 
Finland 
France 
Croatia 
Hungary 
Ireland 
Iceland 
Italy 
Lithuania 
Luxembourg 
Latvia 
Malta 
Netherlands 
Norway  
Poland 
Portugal 
Romania 
Sweden 
Slovenia 
Slovakia 
United Kingdom  

54 

 
 
 
 
 
ANNEX 6 — THE ROLE OF THE JOINT RESEARCH CENTRE WITHIN CIPS 

implementation  and  actively  participated 

The  JRC,  the  Commission’s  science  and  knowledge  service,  played  an  active  role  in  the 
Programme’s 
inter-service 
consultations and groups. Furthermore, the JRC influenced the implementation process of EU 
policy  on  critical  infrastructure  protection.  On  the  other  hand,  CIPS  funding  supported  the 
JRC’s implementation of EU initiatives on critical infrastructures, including the development 
of  the  Critical  Infrastructure  Warning  Information  Network  (CIWIN)  and  the  European 
Reference  Network  for  Critical  Infrastructure  Protection  (ERN-CIP)  as  well  as  specific 
research  activities    for  instance  in  the  CBRNE  and  cybersecurity  sectors.  Additionally, 
through CIPS funding, the JRC provided support to DG HOME and the Member States on the 
preparation and review of Directive 2008/114/EC. 

the  CIPS’s 

in 

Due  to  the  JRC’s  expertise  in  security  and  CIP62,  it  contributed  to  the  support  and 
implementation of CIPS actions throughout the programming period. Actions carried out by 
JRC  were  indicated  in  the  annual  work  programme  as  administrative  arrangements  with 
JRC63.  The  JRC  has  its  own  annual  work  programme  (based  on  Framework  Programme  7) 
which  meant  that  actions  funded  through  CIPS  had  to  be  in  line  with  JRC’s  priorities.  To 
avoid gaps, an analysis on what actions could have been financed by Framework Programme 
7  and  what  could  have  been  financed  by  the  CIPS  programme  was  carried  out  by  the  JRC. 
Moreover, under Framework Programme 7 and H202064, the JRC started a scientific initiative 
in support of the implementation of the CIP Directive in 2010. This included organising and 
coordinating the meetings of all CIP national contact points. 

Two of the key actions implemented by the JRC under CIPS included ERN-CIP and CIWIN. 
The box below presents additional information on ERN-CIP and CIWIN. 

Overview of ERN-CIP and CIWIN developed by JRC under the CIPS programme 

European Reference Network for Critical Infrastructure Protection (ERN-CIP) 

In 2009, the administrative Agreement between JRC and DG HOME was dedicated to the 
establishment of the EU reference network for critical infrastructure (ERN-CIP); Following 
that, JRC support to ERN-CIP was reaffirmed in 2011 and 2014. 

The  ERN-CIP  experienced  two  phases  (preparatory  and  implementation  phases)  and  was 
based on needs and requirements of the Member States in relation to CIP identified during 
the  preparatory  phase.  During  the  12  months  preparatory  phase,  the  JRC  contributed  to 
identifying the needs of Member States and Commission; identifying the priorities for the 
activity of the ERN-CIP; the inventory of the existing laboratories and facilities in Europe 
relevant to the ERN-CIP. 

On the implementation, during the first sub-phase the project proposal was to be developed 
and delivered. Following approval, the proper implementation of the ERN-CIP started. In 
2011, following further administrative agreements, two additional phases were carried out 
by the JRC (until mid-2014): I) the inventory phase which delivered a database to be used 
remotely  of  European  facilities  and  a  thematic  studies  on  CIP-related  issues;  and  ii)  the 

62   https://ec.europa.eu/jrc/en/research-topic/cybersecurity 
63   A detailed description is provided in the technical annexes. 
64   https://ec.europa.eu/programmes/horizon2020/ 

55 

 
 
                                                            
operational  phase,  which  delivered  an  operational  demand-driven  network  of  innovative 
and competitive security solutions. 

Furthermore,  the  JRC  has  started  a  Framework  Programme  7/H2020  research  action  in 
January  2013  called  ERN-Sees  —  European  Reference  Networks  for  Security  and 
Standards. The support of the JRC to ERN-CIP was merged within the ERN-Sees action. 

Critical Infrastructure Warning Information Network (CIWIN) 

The  framework  designed  by  the  EPCIP  included  the  Critical  Infrastructure  Warning 
Information Network (CIWIN).  This network served as internet portal to exchange ideas, 
good practices and knowledge in the field and as a repository for CIP information. 

In 2008, the JRC was tasked with regards to the implementation of the CIWIN prototype. 
The  aim  was  to  demonstrate  and  validate  the  CIWIN  principles  in  scenarios  of 
interdependency, cross-sector and cross-border, as well as to investigate new technological 
solutions for an exchange of information. 

In  2009,  the  CIWIN  prototype  was  supported  and  hosted  by  the  JRC65.  It  included  all 
activities  related  to  installation,  configuration,  tuning,  testing  of  connectivity  and 
operability. The JRC also provided technical advice to DG HOME in evaluating the pilot 
phase  of  the  CIWIN.  In  2010,  after  the  pilot  phase,  the  support  role  of  the  JRC  was 
renewed.  The JRC supported DG HOME with the population of documents in the CIWIN 
platform. 

65   This  prototype  was  developed  by  an  external  service  provider  under  the  direct  and  solely  control  of  DG 

HOME (ex- DG JLS). The JRC supported only the hosting of CIWIN in its Data Centre. 

56 

 
 
 
 
 
                                                            
ANNEX 7 — SECTORAL ANALYSIS OF CIPS PROJECTS 

Transport sector 

24  projects  were  funded  under  this  sector  during  the  evaluation  period.  Over  two-fifths  of 
them were led by organisations in Italy. However, projects were also funded to organisations 
located in Germany (3), Greece (2), Spain (2), Poland (2), Belgium (1), Bulgaria (1), Romania 
(1), Slovenia (1) and the UK (1). 

As  an  example,  the  Risk  management  of  Transport  of  Hazardous  Substances  (RMTHS) 
project  was  awarded  funding  in  2012  and  coordinated  by  the  Urban  Planning  Institute  of 
Ljubljana.  Its  main  aim  was  to  develop  an  application  for  an  Integrated  Geographic 
Information  System  which  could  model  the  safest  route  for  trucks  carrying  hazardous 
materials.  The  activities  of  the  project  varied  and  included  different  domains  such  as 
legislation,  definition  of  the  main  road  network  and  preparation  of  final  documentation. 
However,  the  final  result  was  the  development  of  an  Integrated  Geographic  Information 
System  application  to  support  road  authorities  in  real-time  management  of  traffic  with  a 
particular focus on hazardous substances. The project delivered the following outputs: 

• Integrated Geographic Information System application for managing data on transport of 

hazardous substances; 

• Integrated Geographic Information System application for identification safest route for 

transport of hazardous substances; 

• Methodology for the monitoring of the transport of hazardous substances; 
• Integrated Geographic Information System expert model; and 
• A webpage, workshops, articles and final publication. 

Energy sector 

A  total  of  16  projects  across  organisations  in  different  Member  States66  were  granted  CIPS 
funding  in  this  sector.  The  organisations  leading  the  projects  included  police  bodies, 
universities  and  research  institutes,  private  companies  (i.e.  Deloitte)  and  state-sponsored 
companies (i.e. the Italian National Agency for New Technology, Energy and Environment). 
The project’s topics included:  

i) identifying risks and improving the security of energy smart grids (5 projects);  
ii) The coordination of EU-wide and Member States stakeholders (3 projects); and 
iii) prevention of malfunctions in energy infrastructures (3 projects). 

Three energy projects focused on the development of tools for protection or assessing risks. 
They  were  rated  as  ‘good’  by  the  Commission,  who  highlighted  the  importance  of  creating 
techniques, methods and tools to prevent threats to energy smart grids. 

The JRC also implemented actions in the energy section such as actions aiming to identify the 
key  vulnerabilities  of  the  European  Gas  Transmission  Network  in  order  to  determine  needs 
and actions and reduce its vulnerability. 

66   ES, IT, NL, HU, CZ, UK. 

57 

 
 
 
 
 
 
                                                            
As  an  example  of  implemented  project,  the  Italian  National  agency  for  new  technologies, 
energy and sustainable economic development (ENEA) led the MIA project  67 that aimed to 
develop a methodological framework to identify and measure interdependencies between ICT 
infrastructures and electricity generation infrastructures. 

Chemical sector 

In  2008,  2009  and  2013,  four  projects  focusing  on  the  chemical  sector  were  implemented. 
Two focused on developing workshops agreement for chemical, biological, radiological and 
nuclear  defence  and  were  led  by  the  European  Committee  for  Standardisation  (CEN)  in 
Belgium.  The  third  one  focused  on  building  protection  against  adversaries  using  high-risk 
chemicals  and  was  led  by  a  consulting  company  in  Czech Republic.  Finally,  the  fourth  one 
was led by the European Chemical Industry Council which aimed at improving knowledge on 
effective critical infrastructure protection. 

Finance sector 

Two organisations (based in Italy and Spain) coordinated a project each in this sector. Both 
projects aimed at improving coordination between different sectors hence had a cross-border 
focus.  More  specifically,  one  project  focused  on  the  analysis  of  the  domino  effects  due  to 
critical  infrastructure  failures  whereas  the  other  one  aimed  at  preventing  the  electronic 
payment fraud.  

Water sector 

Four  projects  were  implemented  under  CIPS  in  this  sector.  Two  were  led  by  police 
headquarters in Poland while the other two by the European Committee for Standardisation in 
Belgium. However, their common aim was the protection of drinking water supplies in crisis 
situations (terrorist attacks included). 

Food sector 

One project was implemented under the CIPS programme and was led by a university in Italy. 
It focused on identifying risks to prevent terrorist attacks on the food supply chain. 

Health sector 

One project focused on improving Member States response to terrorist attacks on hospitals. 

Space sector 

Three projects focused on this sector, two in 2007 and one in 2011. All were led by the same 
European spaceflight service company (Telespazio Spa) based in Italy. 

67   Definition  of  a  methodology  for  the  assessment  of  mutual  interdependencies  between  ICT  and  electricity 

generation / transmission infrastructures (MIA) JLS/2007/CIPS/019. 

58 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                                            
Research sector 

Four  projects  focusing  on  this  sector  were  funded  under  CIPS  and  were  led  by  different 
bodies  in  the  UK,  IT  and  ES.  Their  aims  varied  significantly  between  Member  State.  One 
project focused on changing the higher education curriculum to help students with the design 
and development of new or improved products and services aimed at the prevention, detection 
or  response  to  terrorism  activities,  whereas  another  one  focused  on  the  development  of  a 
methodology  and  tool  to  permit  the  mathematical  calculation  of  the  resilience  of  an 
infrastructure. 

Nuclear sector 

Two  projects  were  granted  CIPS  funding  under  this  sector.  One  was  led  by  a  private 
engineering  consultancy  firm  in  Germany  in  2010  and  aimed  at  identifying  critical 
infrastructures in nuclear power in order to strengthen its protection capacity. The second one 
was implemented by an institute in Bulgaria and focused on the development of tools needed 
to coordinate inter-sectoral power and transport critical infrastructures protection activities. 

59 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ANNEX 8 — THE DIFFERENT DIMENSIONS OF CIPS   

Dimension 

Explanation 

Risks included under 
CIPS 

  Terrorism 
  Other security-related risks  

Stages of CIPS in the CIP 
cycle 

  Preparedness and prevention (pre-attack) 
  Consequence management (post-attack) 

CIPS aims to protect 

  People 
  Society 
  Critical Infrastructure68  

Sectors  

Information communication technologies (ICT) 

  Energy 
  Nuclear industry 
 
  Water 
  Food 
  Health 
  Financial 
  Transport 
  Chemical industry 
  Space and research facilities 
  Other sectors (incl. government) 

Actions 

Preparedness and prevention (pre-attack) 

  Risk assessment of critical infrastructure and developing 

of risk assessment and methodologies 

  Promoting and supporting shared operational measures 
to improve security in cross-border supply chains 
  Promoting and supporting the development of security 

standards 

  Promoting and supporting Community wide 

coordination and cooperation on protection of critical 
infrastructure 

Consequence management (post-attack) 

  Exchange of know-how and experience and best 

practices to achieve cooperation between various actors 
of crisis management 

  Promoting joint exercises and practical scenarios  

Activities (based on 
categorisation performed 
during the evaluation) 

  Development of tools — Risk Assessment 
  Development of tools — Protection 
  Cooperation and partnerships 

68   CIPS aimed to protect the societal impact of the disruption/destruction of CI. 

60 

 
 
                                                            
Dimension 

Explanation 

  Response  to terrorist attacks or security-related 

incidents 

  Training, exercises and simulations  

61 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ANNEX 9 — INTERVENTION LOGIC OF THE CIPS PROGRAMME 

62