COMMISSION STAFF WORKING DOCUMENT EXECUTIVE SUMMARY OF THE COMMISSION STAFF WORKING DOCUMENT EU-LISA EVALUATION Accompanying the document Report from the Commission to the European Parliament and the Council on the functioning of the European Agency for the operational management of large-scale IT systems in the area of freedom, security and justice (eu-LISA)
Date: 2017-06-29
Institutions: Directorate-General for Migration and Home Affairs, European Commission
Subjects: EU office or agency, European security, Schengen Information System, area of freedom, security and justice, exchange of information, external border of the EU, information system, migration, personal data, rights of aliens

EN 

   EN 

   EUROPEAN COMMISSION  Brussels, 29.6.2017  SWD(2017) 250 final   COMMISSION STAFF WORKING DOCUMENT EXECUTIVE SUMMARY OF THE COMMISSION STAFF WORKING DOCUMENT  EU-LISA EVALUATION Accompanying the document Report from the Commission to the European Parliament and the Council on the functioning of the European Agency for the operational management of large-scale IT systems in the area of freedom, security and justice (eu-LISA) {COM(2017) 346 final} {SWD(2017) 249 final}  
 
 
1. Outcome of the external evaluation 

The  external  evaluation  of  the  European  Agency  for  the  operational  management  of  large-
scale  IT  systems  in  the  area  of  freedom,  security  and  justice  (eu-LISA)  confirmed  that  the 
Agency effectively ensures the operational management of large-scale IT systems in this area 
and fulfils the tasks laid down in the Regulations as well as new tasks entrusted to it. It also 
found  that  eu-LISA  effectively  contributed  to  the  establishment  of  a  more  coordinated, 
effective  and  coherent  IT  environment  for  the  management  of  large-scale  IT  systems  that 
support the implementation of Justice and Home Affairs (JHA) policies. 

The evaluation therefore provides the necessary reassurance about the continuing validity of 
the rationale behind creating eu-LISA and its further existence. 

However, the evaluation also identified certain shortcomings for all evaluation criteria at both 
the  operational  and  organisational  level,  and  recommended  a  number  of  changes  to  address 
them. In its actions, the Agency could (and should) be more efficient, effective and coherent 
and could also strengthen its relevance and added value. 
The  evaluation  also  identified  ways  in  which  the  Agency  could  improve,  in  particular  with 
regard to: 

 
 
 

its mission, tasks and working practices; 
its added value; and 
its contribution towards the EU and the Member States in the future. 

This  is  all  the  more  relevant  given  the  current  security  and  migration  challenges  facing  the 
EU.  This  highlights  the  importance  of  large-scale  IT  systems  under  eu-LISA’s  operational 
management for the smooth and secure functioning of the Schengen area. 

The evaluation outlined concrete findings and recommendations together with their expected 
legal, financial and organisational impact. The recommendations were classified as minor (7), 
important  (40),  very  important  (12)  and  critical  (7).  Most  of  the  shortcomings  can  be 
addressed without amending the applicable legal framework. 

The  evaluation  also  identified the  possible  need to  revise/increase  the  tasks  entrusted  to  eu-
LISA and introduced a limited set of recommendations for legislative amendments to the eu-
LISA establishing Regulation1and to other related instruments (e.g. the legislative instruments 
that  govern  the  large-scale  IT  systems  operated  by  the  Agency)  such  as  the  transfer  of  the 
Commission’s responsibilities relating to the communication infrastructure to eu-LISA. These 
amendments  are  required  to  improve  the  functioning  and  operational  effectiveness  of  the 
Agency. 

The  Commission  will  take  account  of  the  outcome  of  the  evaluation  when  proposing 
amendments to the applicable legal framework. 

1   Regulation  (EU)  No   1077/2011  of  the  European  Parliament  and  of  the  Council  of  25   October  2011 
establishing  a  European  Agency  for  the  operational  management  of  large-scale  IT  systems  in  the  area  of 
freedom, security and justice, 1.11.2011, L 286 p. 1. 

2 

 
 
 
 
 
 
                                                            
2. External evaluation of eu-LISA in a nutshell 

eu-LISA was established by Regulation (EU) No 1077/2011 in 2011 and became operational 
on 1 December 2012. 

eu-LISA  is  responsible  for  the  operational  management  at  central  level  of  the  Visa 
Information System (VIS), Schengen Information System (SIS II) and Eurodac. The Agency 
took up its core tasks on 1 December 2012 and operates the VIS since 1 December 2012, SIS 
II since 9 May 2013 and Eurodac since June 2013. It is also envisaged to entrust the Agency 
with  the  development  and  operational  management  of  new  systems  in  the  area  of  freedom, 
security  and  justice,  such  as  the  Entry/Exit  System,2  ETIAS3  or  the  automated  system  for 
registration,  monitoring  and  the  allocation  mechanism  for  applications  for  international 
protection,4 subject to the adoption of relevant legislative instruments. The seat of the Agency 
is Tallinn and the systems are operated from the technical site in Strasbourg. The backup site 
is located in St. Johann im Pongau, Austria. 

Established with 120 establishment plan posts, eu-LISA operated with an agreed EU budget 
of  EUR  80  022  000  in  2016.  In  general,  a  significant  reinforcement  of  the  Agency  is 
envisaged  in  the  Commission  proposals  on  the  Entry/Exit  system,  the  European  Travel 
Information  and  Authorisation  System  (ETIAS),  Eurodac  and  the  proposal  to  reform  the 
Dublin system.5 

Under  Article  31  of  the  establishing  Regulation,  the  evaluation  examined  the  way  and  the 
extent to which eu-LISA effectively contributes to the operational management of large-scale 
IT systems in the area of freedom, security and justice and fulfils the tasks laid down in the 
Regulation  and  new  tasks  entrusted  to  it.  The  evaluation  could  not  examine  eu-LISA’s 
contribution  to  the  Union  strategy  aimed  at  coordinated,  cost-effective  and  coherent  IT 
environment at EU level as this strategy has not yet been established. It therefore examined 
the  Agency’s  contribution  to  the  establishment  of  a  coordinated,  effective  and  coherent  IT 
environment for the management of large-scale IT systems that support the implementation of 
JHA policies. 

Article 31(2) of the establishing Regulation stipulates that, on the basis of the evaluation, the 
Commission, after consulting the Management Board, issues recommendations for changes to 
the Regulation and forwards them together with the opinion of the Management Board as well 
as  appropriate  proposals  to  the  European  Parliament,  the  Council  and  the  European  Data 
Protection Supervisor. 

2   Proposal for a Regulation of the European Parliament and of the Council establishing an Entry/Exit System 
(EES) to register entry and exit data and refusal of entry data of third country nationals crossing the external 
borders of the Member States of the European Union and determining the conditions for access to the EES for 
law enforcement purposes and amending Regulation (EC) No 767/2008 and Regulation (EU) No 1077/2011. 
COM(2016) 194 final, 6.4.2016. 

3   Proposal  for  a  Regulation  of  the  European  Parliament  and  of  the  Council  establishing  a  European  Travel 
Information  and  Authorisation  System  (ETIAS)  and  amending  Regulations  (EU)  No 515/2014,  (EU) 
2016/399, (EU) 2016/794 and (EU) 2016/1624, COM(2016) 731 final, 16.11.2016. 

4   Proposal  for  a  Regulation  of  the  European  Parliament  and  of  the  Council  establishing  the  criteria  and 
mechanisms  for  determining  the  Member  States  responsible  for  examining  an  application  for  international 
protection  lodged  in  one  of  the  Member  States  by  a  third-country  national  or  a  stateless  person  (recast), 
COM(2016) 270 final, 4.5.2016. 

5   Although  an  additional  budget  has  been  proposed  for  eu-LISA  in  light  of  the  Dublin  system  reform,  no 

additional staff has been envisaged in the proposal. 

3 

 
 
 
 
 
 
 
                                                            
The Commission commissioned an external evaluation while keeping the steering role as the 
chair  of  the  Steering  Committee.  In  accordance  with  the  establishing  Regulation,  the 
Management  Board  was  closely  consulted  throughout  the  evaluation.  eu-LISA  was  also 
heavily involved in this exercise and contributed to its success. Both the Management Board 
and  eu-LISA  were  also  represented  in  the  Steering  Committee.  In  addition,  there  was  a 
dedicated  Inter-Service  Steering  Group  consisting  of  the  representatives  of  relevant 
Commission departments, i.e. DG HOME, DG JUST (+SRD), DG HR, DG BUDG and DG 
DIGIT. 

The  external  evaluation  was  carried  out  from  March  to  September  2015.  Data  analysis, 
additional data collection where necessary and drafting and completing the final deliverables 
then  took  place  until  March  2016.  The  external  evaluation  report  and  its  Annexes  were 
validated by the Steering Committee in April 2016.6  

The  evaluation  methodology  was  based  on  the  four  main  assessment  criteria  below  and 
consisted of the inception phase, data collection and analysis phase and finally the judgment 
phase. 

This approach enabled an assessment of the following: 

 
 

 

 

the extent to which eu-LISA achieved its objectives (effectiveness); 
the costs (financial, human resources, time, expertise) directly incurred by the Agency 
to achieve the results and the way they are allocated and managed (efficiency); 
the  extent  to  which  eu-LISA  cooperation  activities  are  articulated  and  aligned  with 
those of the main stakeholders, the policies and strategies (coherence); and 
the extent to which the creation of the Agency induced benefits in comparison with the 
previous operational arrangements (relevance and added value). 

The intervention logic was developed based on a preliminary document review (in particular 
the regulatory framework), and strategic interviews were conducted. The evaluation questions 
presented in the terms of reference were also fine-tuned. A detailed analytical framework was 
then  developed:  for  each  evaluation  question,  precise  evaluation  criteria,  indicators, 
descriptors as well as the underlying sources of data were identified. 

Consultation  of  the  stakeholders  by  way  of  interviews  and  an  e-survey  together  with  an 
extensive documentation review were complemented by 5 case studies: VIS evolutions; Smart 
Borders pilot; Common shared service studies; Eurodac recast and TESTA-ng. 

6  http://bookshop.europa.eu/is-bin/INTERSHOP.enfinity/WFS/EU-Bookshop-Site/en_GB/-

/EUR/ViewPublication-Start?PublicationKey=DR0 116464. 

4