Special Report No 3/2014 ‘Lessons from the European Commission’s development of the second generation Schengen Information System (SIS II)’
Date: 2014-05-23
Institutions: European Court of Auditors
Subjects: EU police cooperation, Schengen Information System, biometrics, border control, disclosure of information, exchange of information, external border of the EU, personal data

EN 

2014 

NO 03

Special Report  Lessons from the 

European Commission’s 
development of the 
second generation 
Schengen Information 
System (SIS II)

EUROPEAN
COURT
OF AUDITORS

 
EUROPEAN COURT OF AUDITORS 
12, rue Alcide De Gasperi 
1615 Luxembourg 
LUXEMBOURG

Tel. +352 4398-1

E-mail: eca-info@eca.europa.eu 
Internet: http://eca.europa.eu

Twitter: @EUAuditorsECA 
YouTube: EUAuditorsECA

More information on the European Union is available on the Internet (http://europa.eu).

Luxembourg: Publications Office of the European Union, 2014

ISBN 978-92-9241-934-9
doi:10.2865/8113

© European Union, 2014
Reproduction is authorised provided the source is acknowledged.

Printed in Luxembourg

EN 

2014 

NO 03

Special Report

Lessons from the  
European Commission’s 
development of the second 
generation Schengen  
Information System (SIS II)

(pursuant to Article 287(4), second subparagraph, TFEU)

Contents

02

Paragraph

Glossary

I–VII 

Executive summary

1–3 

Introduction

4–7 

4–6 

7 

Audit scope and approach

Audit scope

Audit approach

8–69  Observations

8–32 

Section 1 — The Commission delivered SIS II albeit over 6 years late

8 

Audit criteria

9–12 

The initial project deadline was unrealistic

13–20 

Until 2007 the Commission did not allocate sufficient staff with the expertise to manage outsourced 
development work effectively

21–25  Working relations between some member countries and the Commission were initially poor, and, until the 

Commission created the Global Project Management Board, it did not manage to draw on the experience of 
all end-users

26–28 

It was not clear to all stakeholders who made key decisions

29–32 

System requirements were not sufficiently stable until the final project phase

33–49 

Section 2 — The cost of developing the central system and network increased from 23 million euro 
to 189 million euro

33 

Audit criteria

34–37 

The Commission did not update its 2003 global project budget until 2010

38 

The delays in delivering SIS II resulted in increased costs

39–41 

The main development contractor delivered an underperforming system in the first part of the project

42–43 

Changes to system requirements resulted in increased costs

 
Contents

03

44–49 

The Commission increased the value of the main development contract from 20 million euro to 82 million 
euro through negotiated amendments

50–63 

Section 3 — The Commission did not ensure that the business case for SIS II included a thorough 
reassessment of the costs and expected benefits, updated to take into account major changes

50 

Audit criteria

51–54 

After being made responsible for the project, the Commission’s updated business case for SIS II did not 
include a thorough reassessment of all costs, expected benefits and alternatives

55–63 

The Commission did not thoroughly reassess the business case during the project even though there were 
major changes to the costs and expected benefits

64–69 

Section 4 — The Commission learnt lessons from its experience during the first part of the project 
to complete SIS II and prepare future projects

64 

65 

Audit criteria

The Commission applied lessons from the first part of the project to the final phase

66–69 

The Commission has already applied lessons from SIS II to other projects

70–72  Conclusions and recommendations

Annex — Chronology of SIS II 

Reply of the Commission

 
 
Glossary

04

Alert: Information on persons or objects entered into the Schengen Information System by competent national 
authorities.

Business case: Explains the reasons for the project and examines costs, risks, benefits and alternatives. It is created 
in the initiating phase of the project and kept up to date during the project as costs, risks, benefits and alternatives 
change.

Business manager: Acts on behalf of the system owner in setting business objectives and priorities.

Decision log: Contains a summary of decisions taken. It brings visibility and accountability to how and by whom 
decisions are taken.

Deliverable: A document, hardware, software or other product delivered as agreed.

eu-LISA: European Agency for the operational management of large-scale IT systems in the area of freedom, 
security and justice.

Executing phase: The stage of a project in which the activities set out in project plans are carried out and project 
deliverables are produced.

Final project phase: Amendment to the SIS II contract agreed in December 2010 with the main development 
contractor to finalise the system by March 2013 incorporating new capacity, performance and testing requirements.

Governance: Concerns how decisions are made.

HPS: The main development contractor: a consortium of Hewlett-Packard and Steria.

DG Informatics: The Commission’s directorate-general responsible for IT.

Initiating phase: The first phase in a project to define the project objective and create the business case.

Interface control document: Document which describes the interface between the central Schengen Information 
System and national systems.

ISPMB: The Information Systems Project Management Board. It is chaired by DG Informatics and examines new IT 
project proposals.

Glossary

05

Member countries: The 26 countries of the Schengen Area. The UK and Ireland, though not part of the Schengen 
Area also participate in the Schengen Information System, with the exception of alerts relating to non-Schengen 
nationals.

Milestone tests: System tests introduced following the Council’s request of June 2009 which SIS II had to pass 
successfully in order to continue with the project.

Planning phase: Second stage of a project in which the various project plans are established.

Schengen Area: An area containing 26 European countries that have abolished passport and immigration controls 
at their common borders. It consists of 22 EU Member States and the four member countries of the European Free 
Trade Association (EFTA).

SISVIS Committee: Committee assisting the Commission in developing SIS II and the Visa Information System, 
composed of all member countries and chaired by the Commission.

Stakeholders: Those who can affect or be affected by the project. The business side includes the system owner, 
business manager and users. The IT side includes the system supplier, project manager and project team.

Steering committee: Sets out the main orientations and coordinates the project’s main tasks. It approves resources 
allocated to the project and main project deliverables.

System owner: Is responsible for setting the business objectives and priorities and typically chairs the steering 
committee.

System requirements: Description of the required behaviour of the system to be developed.

System supplier: Supplies the IT system in line with agreed requirements, timing and budget.

Unisys: The quality assurance contractor.

Visa Information System (VIS): Large-scale IT system developed by DG Home Affairs at the same time as SIS II and 
through the same main development contract.

Executive summary

06

I
The Schengen Information System (SIS) is used by 
border guards, police, customs, visa and judicial 
authorities throughout the Schengen Area. It contains 
information (alerts) on persons who may have been 
involved in a serious crime or may not have the right 
to enter or stay in the EU. It also contains alerts on 
missing persons and lost or stolen property, such as 
banknotes, vehicles, firearms and identity documents. 
Alerts are entered in the system by national authori-
ties (see paragraph 1). 

II
Developed and operated in an intergovernmental 
framework, the initial Schengen Information System 
(SIS 1) was in operation from 1995 until the launch of 
SIS II in 2013. In 2001, the Council decided to replace it 
with the second generation version (SIS II). The Council 
charged the Commission with its development and 
DG Home Affairs1 managed the project with a target 
launch date at the end of 2006. The main reason ini-
tially for developing SIS II was to connect more mem-
ber countries. In addition, SIS II was to benefit from 
the latest technological developments and added 
functions such as new categories of alerts, a facility to 
link alerts and the capacity to store documents associ-
ated with an alert (see paragraphs 2 and 3). 

III
The European Court of Auditors examined whether 
the Commission delivered SIS II on time and in line 
with initial cost estimates. It also examined whether 
there was a robust business case for SIS II throughout 
the project, which took into account major changes to 
the costs and expected benefits. In addition, the Court 
assessed whether the Commission had learnt and 
applied lessons from its management of the project 
(see paragraphs 4 to 7). 

IV
The audit found that the Commission delivered the 
central system, but over 6 years later than initially 
planned and at 8 times the initial budget estimate. 
The delay and overspending resulted partly from the 
challenging governance context which limited the 
Commission’s ability to address operational issues and 
partly from weaknesses in the Commission’s manage-
ment. This was particularly the case during the first 
part of the project up until 2009:

(a)  The initial project deadline was unrealistic (see 

paragraphs 9 to 12).

(b)  The Commission did not update its 2003 global  
project budget until 2010 (see paragraphs 34 
to 37).

(c)  System requirements evolved during the project 
to meet the needs of users and were not suf-
ficiently stable until the final project phase from 
2010. The system now in operation has a much 
larger capacity than the one foreseen in 2001 (see 
paragraphs 29 to 32 and 42 to 43).

(d)  Until 2007 the Commission did not allocate 

sufficient staff with the expertise to manage 
outsourced development work effectively (see 
paragraphs 13 to 20).

(e)  The main development contractor delivered an 
underperforming system in the first part of the 
project (see paragraphs 39 to 41).

(f)  Working relations between some member coun-
tries and the Commission were initially poor. 
Until the Commission created the Global Project 
Management Board in 2009, it did not manage to 
draw on the experience of all end-users (see para-
graphs 21 to 25). 

1  At the time, this was DG Justice, Freedom and Security.

Executive summary

07

(g)  It was not clear to all stakeholders who made key 

decisions (see paragraphs 26 to 28). 

(h)  The Commission increased the value of the main 
development contract from 20 million euro to 
82 million euro through negotiated amendments. 
Whilst this procedure is provided for in the Finan-
cial Regulation, there is an inherent risk with any 
negotiated procedure that the prices paid are not 
competitive (see paragraphs 44 to 49).

V
During the project there were major changes to the 
costs and expected benefits. The initial indicative 
estimates of the costs to the EU budget for the central 
system significantly underestimated the true scale 
of the investment necessary. The full cost of SIS II 
amounted to some 500 million euro by the end of 
the project: 189 million euro for the central system 
and an estimate of over 330 million euro for national 
systems. At the same time, the main benefit initially 
expected of SIS II was less relevant because SIS 1 had 
been successfully extended to new member countries 
in 2007 through SISone4all. However, despite these 
changes to the costs and benefits, the Commission 
did not thoroughly reassess the business case for the 
project in order to demonstrate that SIS II remained an 
organisational priority which provided a higher return 
on investment than other opportunities (see para-
graphs 50 to 63).

VI
The Commission did learn lessons from its experi-
ence during the first part of the project, enabling 
it to change its approach during the final project 
phase from 2010 and deliver SIS II in April 2013. In 
addition, although it has not carried out a formal 
evaluation, it has already applied some lessons 
from SIS II in preparing other large-scale IT projects 
(see paragraphs 64 to 69).

VII
On the basis of these observations, the Court’s main 
recommendations are that, when managing the devel-
opment of large-scale IT systems, the Commission 
should (see paragraphs 70 to 72): 

(a)  base the timetable on a technical analysis of the 

tasks to be performed;

(b)  ensure that all projects are integrated into corpo-
rate IT governance arrangements and make full 
use of in-house expertise to manage the work of 
contractors effectively;

(c)  ensure that business needs and the views of 

end-users are sufficiently taken into account in 
decision-making;

(d)  ensure the approval of the business case before 
progressing from project initiation to project 
planning and its reapproval in the event of major 
changes to project costs, expected benefits, risks 
or alternatives;

(e)  ensure that key project decisions are documented 
in a decision log so that they are easily traceable;

(f)  ensure that there is effective global coordina-
tion when a project requires the development 
of different but dependent systems by different 
stakeholders;

(g)  develop large-scale IT systems using interoper-

able building blocks which can easily be reused to 
prevent being locked in to a single contractor;

(h)  pass on the lessons learnt from the Court’s audit to 
directorates-general and EU institutions, agencies 
and other bodies. The Commission should evalu-
ate whether the expected benefits of SIS II were 
achieved.

Introduction

08

03 
The primary reason for developing 
SIS II was to connect the increased 
number of member countries after 
the 2004 enlargement of the European 
Union. In addition, SIS II was to benefit 
from the latest technological develop-
ments and added functions. 

2 

The 26 countries of the 
Schengen Area are Austria, 
Belgium, Czech Republic, 
Denmark, Estonia, Finland, 
France, Germany, Greece, 
Hungary, Iceland, Italy, 
Latvia, Liechtenstein, 
Lithuania, Luxembourg, 
Malta, Netherlands, Norway, 
Poland, Portugal, Slovakia, 
Slovenia, Spain, Sweden and 
Switzerland.

01 
The Schengen Information System 
is used by border guards, police, 
customs, visa and judicial authori-
ties throughout the Schengen Area2. 
It contains information (alerts) on 
persons who may have been involved 
in a serious crime or may not have the 
right to enter or stay in the EU. It also 
contains alerts on missing persons and 
lost and stolen property, such as bank-
notes, vehicles, firearms and identity 
documents. Alerts are entered into the 
system by national authorities. The 
initial Schengen Information System 
(SIS 1) was developed as an inter-
governmental project led by France 
and became operational in 1995. It 
remained in operation, managed by 
France, until May 2013, when it was 
replaced by SIS II. Both SIS 1 and SIS II 
consist of national systems, developed 
by member countries, linked to a cen-
tral system by a network (see Figure 1).

02 
The Schengen countries initially took 
the decision to create a second gen-
eration Schengen Information System 
(SIS II) in December 1996. In Decem-
ber 2001 the Council made the Com-
mission responsible for the develop-
ment of this system with a target date 
of the end of 2006. The Commission’s 
DG Home Affairs managed the project 
and outsourced the development work 
for the central system.

Introduction

09

1 Overview of the Schengen Information System
e
r
u
g
i
F

Users

national 
system

national 
system

Users

national 
interface

national 
interface

national 
interface

national 
interface

central 
system

Central domain

Source: European Commission, DG Home Affairs.

Users

national 
system

national 
system

Users

 
Audit scope and approach

10

Audit scope

04 
The audit examined whether the Com-
mission managed the development of 
SIS II well and addressed the following 
four questions:

(a)  Did the Commission deliver SIS II 

on time?

(b)  Did the Commission deliver SIS II in 
line with initial cost estimates?

(c)  Was there a robust business case 
for SIS II throughout the project 
which took into account major 
changes to the costs and expected 
benefits?

(d)  Did the Commission learn and ap-
ply lessons from its management 
of the SIS II development project?

05 
Within the Commission as a whole, 
there is considerable experience in the 
development of IT projects and guid-
ance to directorates-general on how to 
manage IT projects successfully3. The 
Commission also recognises the need 
to draw lessons from developing spe-
cific IT projects and to share these les-
sons across the organisation4. In 2011, 
for example, the Commission’s Internal 
Audit Service identified lessons from 
a review of the set-up of IT projects in 
various Commission directorates-gen-
eral5. The Court’s audit of SIS II provid-
ed an opportunity to identify, within 
a single project, the reasons for delays 
and overspending, and the changes 
which enabled the Commission finally 
to deliver the system.

06 
This was not an IT audit intended to 
provide assurance on the quality of 
SIS II and it did not examine how the 
national authorities managed the de-
velopment of their SIS II systems.

3 

The Rational Unified Process 
(RUP@EC) was the standard 
software development 
methodology within the 
Commission set out in the 
2004 Communication on the 
improvement of information 
technology governance 
in the Commission, 
SEC(2004) 1267. In 2011, 
this was complemented by 
the Project Management 
Methodology PM2.

4  Getting the best from 
IT in the Commission, 
SEC(2010) 1182 final of 
7 October 2010.

5 

Internal Audit Service 
management letter of 
1.2.2011, Set-up of IT Projects 
in the Commission.

Audit scope and approach

11

Audit approach

07 
The audit was based on:

(a)  structured interviews with SIS II 
stakeholders. The stakeholders 
interviewed included:

(i)  Commission staff in DG Home 
Affairs responsible for the 
development of SIS II; 

(ii)  Commission staff in DG 

Budget, DG Informatics, the 
Secretariat-General and DG 
Taxation and Customs Union 
regarding issues relating to IT 
governance at the Commission 
and the development of other 
large-scale IT systems;

(iii)  the main SIS II development 
contractor (a consortium of 
Hewlett-Packard and Steria) 
and the quality assurance con-
tractor (Unisys);

(iv)  staff from the Secretariat of the 
EU Council and the European 
Parliament who followed the 
development of SIS II;

(v)  stakeholder representatives 
from six member countries 
(Germany, Spain, France, Lux-
embourg, Hungary and Portu-
gal). The six member countries 
were selected in order to 
provide a broad cross-section 
of opinions. 

(b)  A survey of stakeholders involved 
in the development of SIS II from 
all member countries, the Secre-
tariat of the EU Council, the Com-
mission and contractors working 
for the Commission. The survey 
sought the views of stakeholders 
on various aspects of the develop-
ment of SIS II. 91 out of 144 se-
lected stakeholders replied to the 
survey, representing a response 
rate of 63 %. Of the respondents, 
81 were from member countries 
and the Council, whilst 10 were 
from the Commission and its 
contractors. Three quarters of 
respondents had been involved in 
the project since 2009 or earlier.

(c)  A review of documentation and 
procedures relating to SIS II.

Observations

12

6  Council Regulation 

(EC) No 2424/2001 of 
6 December 2001 on 
the development of 
the second generation 
Schengen Information 
System (SIS II) (OJ L 328, 
13.12.2001, p. 4) and Council 
Decision 2001/886/JHA of 
6 December 2001 on the 
development of the second 
generation Schengen 
Information System (SIS II) 
(OJ L 328, 13.12.2001, p. 1).

7 

Feasibility Study SIS II, Deloitte, 
7 April 2003.

8  COM(2003) 771 final of 
11 December 2003.

Section 1 — The Commis-
sion delivered SIS II albeit 
over 6 years late

The initial project deadline 
was unrealistic

Audit criteria

08 
The audit examined whether:

(a)  there had been a realistic project 

timetable;

(b)  the Commission had allocated suf-
ficient expertise to the project to 
monitor its execution and super-
vise contracts effectively;

(c)  the Commission had had good 

working relations with stakehold-
ers and had taken into account the 
views of end-users;

(d)  there had been clear decision-

making arrangements; 

(e)  the requirements of the system 

had been sufficiently stable to en-
able the system to be developed 
efficiently. 

09 
In December 2001, a Council regula-
tion6 set the initial project deadline 
as the end of December 2006. This 
was intended to enable the countries 
which were to join the EU in 2004 to 
become part of the Schengen Area at 
the end of 2006. Although the dead-
line was considered achievable by the 
Commission’s 2003 feasibility study, 
this assumed that the requirements 
would be stable and the specifications 
developed rapidly7. By the end of 2003, 
the Commission recognised that the 
initial deadline would be difficult to 
achieve8. However, it was not revised 
until the end of 2006. In the Court’s 
audit survey of key SIS II stakeholders, 
most respondents (69 %) considered 
that the initial deadline was unrealistic. 
A chronology of the development of 
SIS II is shown in the Annex.

10 
In the audit survey, respondents 
were asked to weight the reasons for 
the 75-month delay. Respondents con-
sidered that part of the delay was be-
cause the initial deadline had not been 
based on a realistic technical analysis. 
They gave a weighting of 16 % to this 
reason (see Figure 2). 

Observations

13

2 Reasons for the delays in SIS II
e
r
u
g
i
F

Lack of expertise to supervise 
contracts eﬀectively 30 %

Initial deadline not based on 
a realistic technical analysis 16 %

Other 4 %

Source: Audit survey of SIS II stakeholders.

11 
Because the Commission did not de-
liver SIS II on time, SIS 1 was adapted 
in December 2007 to connect the new 
member countries. This modification 
was a copy of the Portuguese national 
system and was called SISone4all. It 
provided a solution for 9 of the 10 
accession countries9, which was the 
primary objective of SIS II. The follow-
ing extensions to the initial project 
deadline were agreed, but were also 
not met, increasing the risk that the 
system requirements could change:

(a)  In December 2006, the deadline 

was extended to December 200810. 

(b)  In October 2008, the deadline was 

extended to June 201011.

Lack of commitment of some countries 
and poor working relations 17 %

Unsatisfactory decision-making 
arrangements 11 %

Instability of system 
requirements 22 %

9  Cyprus was unable to 

implement the Schengen 
rules and therefore did not 
join the Schengen Area.

10  Council Decision 
2006/1007/JHA of 
21 December 2006 amending 
Decision 2001/886/JHA on 
the development of the 
second generation Schengen 
Information System (SIS II) 
(OJ L 411, 30.12.2006, p. 78) 
and Council Regulation 
(EC) No 1988/2006 of 
21 December 2006 amending 
Regulation 2424/2001 on 
the development of the 
second generation Schengen 
Information System (SIS II) 
(OJ L 411, 30.12.2006, p. 1).

11  Council Regulation (EC) 

No 1104/2008 of 
24 October 2008 on 
migration from the Schengen 
Information System (SIS 1+) 
to the second generation 
Schengen Information 
System (SIS II) (OJ L 299, 
8.11.2008, p. 1) and Council 
Decision 2008/839/JHA 
of 24 October 2008 on 
migration from the Schengen 
Information System (SIS 1+) 
to the second generation 
Schengen Information System 
(SIS II) (OJ L 299, 8.11.2008, 
p. 43).

 
Observations

12 
In June 2010, the Commission pro-
duced a final schedule for the comple-
tion of SIS II, which was endorsed by 
the Council12. It met the deadline of 
March 201313 and SIS II was launched 
in April 2013, over 6 years later than 
originally planned. After a 1-month pe-
riod of intensive monitoring, it became 
fully operational and replaced SIS 1 in 
May 2013. 

Until 2007 the Commission 
did not allocate sufficient 
staff with the expertise to 
manage outsourced develop-
ment work effectively

13 
Survey respondents considered that 
part of the delay was due to the fact 
that the Commission lacked sufficient 
staff with the expertise necessary to 
supervise contracts effectively. They 
gave a weighting of 30 % to this rea-
son14 (see Figure 2). 

14 
In 2002, the SIS II project team in the 
Commission consisted of four staff 
supervised by a Head of Unit15. Four 
additional staff joined in 200316. At the 
start of the project a representative 
from DG Informatics participated in 

some meetings. Only 10 % of respond-
ents considered that, at the start of the 
project, the Commission had allocated 
sufficient staff with the necessary 
skills and experience to deliver SIS II 
successfully. 

15 
A 2005 request to senior manage-
ment17 for more resources described 
a situation of permanent crisis man-
agement and an inability to respond 
to issues raised by member countries. 
Staff on short-term contracts lacked 
relevant expertise and technical skills. 
The lack of Commission expertise was 
confirmed by the quality assurance 
contractor18. Almost half the project 
staff were on contracts which were 
due to expire in the coming year. Al-
though none of the Commission staff 
who responded to the audit survey 
considered that high staff turnover 
was an obstacle to good relation-
ships with stakeholders, 49 % of 
respondents from member countries 
considered that it was. The 2006 audit 
by the Commission’s Internal Audit 
Service (IAS)19 found that SIS II project 
staff were committed and competent, 
though constantly overloaded. Follow-
ing this audit, DG Home Affairs allo-
cated additional staff to the project. By 
the end of 2007, the SIS II sector in DG 
Home Affairs comprised 12 permanent 
staff, increasing to 17 by 201220.

14

12  Council Regulation (EU) 

No 541/2010 of 3 June 2010 
amending Regulation (EC) 
No 1104/2008 on migration 
from the Schengen 
Information System (SIS 1+) 
to the second generation 
Schengen Information System 
(SIS II) (OJ L 155, 22.6.2010, 
p. 19) and Council Regulation 
(EU) No 542/2010 of 
3 June 2010 amending Council 
Decision 2008/839/JHA on 
migration from the Schengen 
Information System (SIS 1+) 
to the second generation 
Schengen Information System 
(SIS II) (OJ L 155, 22.6.2010, 
p. 23) extended the project 
deadline to March 2013.

13  SEC(2010) 1138 final of 
21 September 2010.

14  This consisted of the following 

elements: 
(a)  Too few Commission 

staff with experience of 
large-scale IT projects and 
Schengen issues (9 %).
(b)  Underperformance of 
the main development 
contractor (9 %).

(c)  Contractual 

disputes (5 %).
(d)  The Commission did 

not monitor effectively 
the results of the 
main development 
contractor (4 %).
(e)  The Commission 
did not clearly 
communicate needs to 
the main development 
contractor (3 %) .

15  SEC(2003) 206 final of 
18 February 2003.

16  COM(2003) 771 final.

17  Note from the Programme 
Manager to the responsible 
Director in DG Justice, 
Freedom and Security of 
16 September 2005.

18 

19 

Issue follow-up report by the 
quality assurance contractor, 
August 2005.

Internal Audit Service report 
of IT Management in DG 
Justice, Freedom and Security 
(8 June 2006).

15

20  At the end of 2008 the SIS II 

sector still comprised 12 staff, 
increasing to 14 at the end 
of 2009. At the end of 2010 it 
comprised 13 staff, increasing 
to 16 at the end of 2011. In 
addition the team included 
contract staff and seconded 
national experts. For example 
the 2008 annual management 
plan of DG Justice, Freedom 
and Security indicated that 
there were 9 contract staff and 
6 seconded national experts.

21  The June 2006 Internal 

Audit Service report on IT 
Management in DG Justice, 
Freedom and Security found 
that there were insufficient 
staff to manage the increasing 
level of outsourcing.

22  Order of the President of 

the Court of First Instance, 
31 January 2005 in case 
T-447/04R.

Observations

16 
The lack of Commission staff with rel-
evant expertise before 2009 was high-
lighted by all six member countries 
interviewed during the audit. Two of 
them, as well as the main development 
contractor, suggested that the project 
would have benefited from more IT 
expertise available elsewhere in the 
Commission e.g. in DG Informatics. The 
Commission lacked sufficient expertise 
to manage the contract effectively and 
to ensure the quality of deliverables21.

17 
On 22 October 2004, HPS was awarded 
the contract to develop SIS II and the 
Visa Information System (VIS). How-
ever, on 18 November 2004, it had to 
suspend work on the contract until 
31 January 2005 whilst waiting for the 
outcome of an appeal by an unsuc-
cessful tenderer. This initial delay was 
a consequence of the method used 
by the Commission for evaluating 
the tenders which did not convinc-
ingly demonstrate that the best bid 
had been accepted (see Box 1). The 
Court of Justice in the framework of an 
injunction procedure considered that 
the Commission had committed errors 
in the assessment of the tenders but 
dismissed the unsuccessful tenderer’s 
application for interim measures 
because there was insufficient evi-
dence to demonstrate that the latter 
would suffer serious and irreparable 
damage22.

Observations

16

1
x
o
B

The Commission assessed the winning bid as cheaper, even though it was in reality 
more expensive

The bids were evaluated firstly on the basis of quality (the technical evaluation) and then on price (the finan-
cial evaluation). 

Out of the seven bids received, HPS and the unsuccessful tenderer who appealed performed best on the tech-
nical evaluation. Their scores were roughly equal. HPS’s score was 0,4 % higher than the unsuccessful tenderer. 

These two companies proceeded to the financial evaluation phase, the results of which were combined with 
the technical score to arrive at an overall score. 

In the financial evaluation HPS’s bid was assessed as 43 % cheaper. As a result it received a higher overall score 
and was awarded the contract. 

However, the amount of HPS’s bid was, in fact, 3,5 % more expensive than the unsuccessful tenderer’s bid 
(38,1 million euro compared with 36,8 million euro). 

The Commission assessed HPS’s bid as cheaper, even though it was in fact more expensive, because it divided 
the tender into 15 parts. The Commission calculated a separate score for each part based on the relationship 
between the bids of the two contractors. It then added these scores together (without weighting and without 
any distinction between fixed prices and items which would be ordered separately when required) to give 
a financial score which bore little relation to the overall bid.

18 
The Commission lacked sufficient staff 
to properly supervise the contract and 
to prevent the under-performance of 
the main development contractor23. 
The 2006 audit by the Commission’s 
Internal Audit Service (IAS) was criti-
cal of the management of the SIS II 
project for failing to ensure that the 
main development contractor deliv-
ered on time and according to the 
specifications. Five of the six member 
countries interviewed during the 

audit considered that the Commis-
sion lacked sufficient experience to 
assess the quality of deliverables and 
monitor the contractor’s performance. 
Only 20 % of respondents to the audit 
survey considered that, before 2009, 
the Commission had monitored the 
results of the main SIS II development 
contractor effectively. The figure in-
creased to 72 % from 2009, highlight-
ing an improvement in the Commis-
sion’s ability to monitor quality in the 
final phase of the project. 

23  UK House of Lords, European 
Union Committee, 9th 
Report of Session 2006–07, 
Schengen Information System 
II (SIS II), Report with Evidence, 
2 March 2007. The Commission 
regretted that it failed to 
exercise supervision as well as 
it could in its relations with the 
main development contractor. 
With the level of staffing at the 
time, the Commission stated 
that it could not supervise 
the contract properly in 
order to prevent the under-
performance of the main 
development contractor.

 
17

24  Council Conclusions, Justice 
and Home Affairs, 26 and 
27 February 2009 welcomed 
the review of the SIS II testing 
approach, guaranteeing an 
increased involvement of 
member countries in test 
design and management.

Observations

19 
The Commission approved test results 
based on feedback from the main de-
velopment contractor and the quality 
assurance contractor. Failed opera-
tional tests at the end of 2008 (which 
were the first tests of the central and 
national systems involving member 
countries) revealed problems in the 
system. In the final phase of the pro-
ject, the Commission’s ability to assess 
the quality of deliverables through 
system acceptance tests improved24 
(see Figure 3).

20 
All six member countries interviewed 
considered that, from 2009, the Com-
mission’s management and com-
munication had improved. The team 
which managed the final project phase 
included more staff with experience 
of developing large-scale IT projects 
and with knowledge of the Schengen 
Information System (e.g. from DG Taxa-
tion and Customs Union, from member 
countries or from the earlier phase of 
the SIS II project itself). A third of re-
spondents to the audit survey consid-
ered that, over the life of the project as 
a whole, the Commission had allocated 
sufficient staff with the necessary skills 
and experience to deliver SIS II suc-
cessfully. This was a significant im-
provement compared with a figure of 
just 10 % at the start of the project (see 
paragraph 14). 

3 Improvement in testing procedures from 2009
e
r
u
g
i
F

The criteria for assessing the results of tests
were clearly deﬁned

The division of tasks to be performed by the 
Commission and the member countries was 
clearly deﬁned

Test speciﬁcations were clearly deﬁned

0 %

20 %

40 %

60 %

80 %

respondents agreeing

Before 2009

From 2009

Source: Audit survey of SIS II stakeholders.

 
 
18

25  This consisted of the following 

elements:
(a)  Opposition or lack of 
commitment to the 
project by some member 
countries (8 %).
(b)  Poor working 

relations between 
stakeholders (5 %).

(c)  SIS II was not strategically 
important, particularly 
after the implementation 
of SISone4all (4 %).

Observations

Working relations between 
some member countries 
and the Commission were 
initially poor, and, until the 
Commission created the 
Global Project Management 
Board, it did not manage to 
draw on the experience of all 
end-users

21 
Survey respondents considered that 
poor working relations with stakehold-
ers and the lack of commitment of 
some member countries accounted 
for some of the delays. They gave 
a weighting of 17 % to this reason25 
(see Figure 2).

22 
Interviews with the member countries 
highlighted the difficult relations with 
the Commission. They described, for 
example, how the SISVIS Committee 
(see paragraph 27) became the arena 
for a series of arguments. In the survey, 
only 15 % of respondents, none from 
the Commission, considered that, be-
fore 2009, the meetings regarding the 
SIS II project took place in a construc-
tive atmosphere (see Figure 4). 

23 
The environment in which SIS II was 
developed before 2009 contrasts with 
the one in which DG Taxation and Cus-
toms Union developed another large-
scale IT system at that time (see Box 2).

4 Poor relations between some member countries and the Commission before 2009
e
r
u
g
i
F

Commission worked well with member countries
 in developing national systems

Commission used suggestions of stakeholders
 to drive improvements

Commission engaged eﬀectively with member
 countries on a basis of mutual trust

Constructive atmosphere in meetings

Stakeholders suﬃciently involved in decision-making

Source: Audit survey of SIS II stakeholders.

0 %

20 %

40 %
respondents agreeing

60 %

80 %

Before 2009

From 2009

 
Observations

19

2
x
o
B

Key success factors in developing IT systems in DG Taxation and Customs Union 

DG Taxation and Customs Union has successfully developed a number of large IT systems for participating 
countries using its TEMPO methodology26. The 2006 report by the Commission’s Internal Audit Service on 
large IT systems in DG Taxation and Customs Union considered the development of the New Computerised 
Transit System (NCTS) to be an example of best practice in project management. This system exchanges elec-
tronic messages between the customs offices of participating countries concerning imported goods, in order 
to help ensure that customs duties are paid at the final destination. The Commission considered that the NCTS 
was comparable in scale to the SIS II project 27. 

DG Taxation and Customs Union considered that the most important success factor in developing large-scale 
IT projects, such as the NCTS, was communicating and building both formal and informal relationships with 
participating countries in order to create a climate of confidence, cooperation and shared determination to 
achieve results. 

26  TAXUD (DG Taxation and Customs Union) electronic management for projects online (TEMPO): a project management methodology developed 

by DG Taxation and Customs Union.

27  COM(2001) 720 final of 18 December 2001, Annex 1 section 5.1.2 and COM(2003) 771, Annex 3 section 5.1.2.

Source: European Court of Auditors based on interviews with DG Taxation and Customs Union.

24 
A number of factors contributed to-
wards the lack of commitment of some 
member countries and poor working 
relations:

(a)  The Commission was not a user 

of the system and, even though it 
recruited some experts from mem-
ber countries, lacked the detailed 
knowledge of end-user practices 
necessary to understand business 
needs and user requirements. 

(b)  The Commission did not manage 

to draw on the experience of all 
member countries that were end- 
users and had knowledge of the 
existing system. Although there 
were various meetings in which 
member countries could com-
municate end-user views to the 
Commission (e.g. National Project 
Manager meetings, SIS II Commit-
tee, Friends of SIS II and the SIS II 
Task Force), before 2009, the Com-
mission did not engage effectively 
with member countries, which felt 
excluded from decision-making 
(see Figure 4). All six member 
countries interviewed considered 
that the views of end-users were 
not sufficiently taken into account.

 
20

28  The Difficult Road to the 

Schengen Information 
System II, Joanna Parkin, 
Centre for European Policy 
Studies, April 2011.

29  The SISVIS Committee 

assisted the Commission 
in the development of the 
second generation Schengen 
Information System (SIS II) 
and the Visa Information 
System (VIS). Council Decision 
2001/886/JHA set down 
two procedures for making 
decisions:
(i)  A regulatory procedure 
for major decisions 
regarding the SIS II 
architecture, data 
protection, issues with 
serious financial or 
technical implications for 
the member countries, 
security issues. If the 
Committee disagreed 
with measures proposed 
by the Commission, the 
Council could require 
the Commission to re-
examine its proposal.

(ii)  A management 

procedure for all other 
matters. If the Committee 
disagreed with measures 
proposed by the 
Commission, the Council 
could take a different 
decision.

Observations

(c)  The Commission provided a frame-
work for member countries to 
communicate with the main 
development contractor. However, 
five of the six member countries 
interviewed considered that 
this was not effective. In spite of 
numerous requests from member 
countries, the Commission did not 
provide them with a copy of the 
main development contract until 
advised to do so by its legal service 
in 2009. 

(d)  Some member countries were not 

convinced that SIS II was necessary. 

(e)  The poor quality of deliverables af-
fected the member countries’ con-
fidence in the Commission’s ability 
to deliver SIS II (see paragraph 16).

25 
The survey showed that relations 
with member countries improved 
after 2009 (see Figure 4). There were 
a number of reasons for this change:

(a)  The Commission created the 

Global Project Management Board 
(GPMB) in 2009 which enabled 
end-users from member countries 
to bring their vision and experi-
ence to the final phase of the pro-
ject. The weekly meetings of the 
GPMB included the Commission, 
experts from up to eight member 
countries (representing end-users 
rather than their own country) 
and the contractors. The GPMB 
meetings enabled more frequent 
interaction between the member 
countries and the main develop-
ment contractor. 

(b)  The Commission’s management 

and communication improved (see 
paragraph 20). 

(c)  The 2009 comparison with an 

alternative scenario provided some 
additional justification for the con-
tinuation of the SIS II project (see 
paragraph 63).

(d)  The main development contractor 
delivered better results during the 
final project phase from 2010 (see 
paragraph 41).

It was not clear to all  
stakeholders who made key 
decisions 

26 
In its 2011 review of the development 
of SIS II28, the Centre for European 
Policy Studies (CEPS) described the 
governance framework for the project 
(see Figure 5).

27 
According to the 2001 legislation, 
the Commission was responsible for 
developing SIS II, assisted by the SISVIS 
Committee. If the Committee disa-
greed with certain measures proposed 
by the Commission, the Council could 
require the Commission to re-examine 
its proposal29. The 2006 audit by the 
Commission’s Internal Audit Service 
found that the Commission’s ability to 
address operational issues was limited 
by the project’s steering mechanisms. 

Observations

21

5 SIS II governance framework from 2010
e
r
u
g
i
F

MDC

QAC

COMMISSION

DG Home
Aﬀairs

EUROPEAN
PARLIAMENT

COUNCIL
(COREPER)

CATS

FoSIS

GPMB

SIS
SIRENE

SIS
TECH

SIS II TF

MC

MC

MC

MC

Europol
Eurojust

SISVIS Committee

SIS II
TECH

SIRENE

NPM

CMB

TAG

Notes:

Arrows show reporting lines

National Project Managers meeting
Schengen member countries
Global Project Management Board
Friends of SIS II
Main development contractor
SIS II Task Force
Quality assurance contractor
Working party on Schengen matters (SIS TECH formation)
Test Advisory Group (advisory working group of the SISVIS Committee)
Working party on Schengen matters (SIS SIRENE formation)
Change Management Board (advisory working group of the SISVIS Committee)
Article 36 Committee

NPM 
MC 
GPMB 
FoSIS 
MDC 
SIS II TF 
QAC 
SIS TECH 
TAG 
SIS SIRENE 
CMB 
CATS 
SISVIS Committee  Committee assisting Commission to develop SIS II and the Visa Information System (VIS)
SIS II TECH 
SIRENE 

SIS II TECH formation of SISVIS Committee
SIRENE formation of SISVIS Committee

Source: Centre for European Policy Studies and DG Home Affairs.

 
Observations

22

System requirements were 
not sufficiently stable until 
the final project phase

29 
Respondents to the survey considered 
that the instability of system require-
ments accounted for some of the 
project delays. They gave a weighting 
of 22 % to this reason (see Figure 2).  

28 
It was not clear to all SIS II stakeholders 
who made key decisions in practice. 
Although the minutes of SISVIS Com-
mittee meetings were recorded, there 
was no decision log to enable the basis 
for all important decisions to be easily 
traced and understood. Half of the re-
spondents to the audit survey consid-
ered that it was the Commission who 
had made key decisions in the period 
to 2009 (see Figure 6). After the crea-
tion of the Global Project Management 
Board in 2009, this figure fell to 24 %. 
Although this was an advisory rather 
than a decision-making body, 34 % of 
respondents thought it made the key 
decisions. Respondents to the survey 
considered that complex and unclear 
decision-making accounted for some 
of the delays. They gave a weighting 
of 11 % to this reason (see Figure 2).

6 Stakeholder perceptions of who took key decisions in practice
e
r
u
g
i
F

60 %

y
e
v
r
u
s

t
i
d
u
a
o
t

s
t
n
e
d
n
o
p
s
e
r

:
y
b
n
e
k
a
t
e
r
e
w
s
n
o
i
s
i
c
e
d
y
e
k
d
e
r
e
d
i
s
n
o
c
o
h
w

50 %

40 %

30 %

20 %

10 %

0 %

Commission

Council

Commission and
Council together

SISVIS
Committee

Development
contractor

GPMB

Not clear

Other

Before 2009

From 2009

Source: Audit survey of SIS II stakeholders.

 
 
 
 
 
 
 
 
 
 
 
Observations

23

32 
Some changes, such as the increase 
in capacity, were partly linked to the 
longer than expected duration of the 
project. The initial 2004 contract was 
for a system to be launched in 2006 
which could handle 15 million alerts 
and be increased to 22 million alerts 
without technical changes. After 2007, 
there was a sharp increase in the rate 
of growth in the number of alerts, 
which led to changed requirements 
regarding system capacity. The 
amendment for the final project phase 
was for a system which could handle 
70 million alerts and could be scaled 
up to 100 million, based on the projec-
tions of a volumetric study carried out 
by the Commission in 2009. At migra-
tion in April 2013, there were 46 million 
alerts. Figure 7 shows the evolution 
of the number of records in the SIS 
database.

30  The contract was divided 

into two phases: Phase I was 
concerned with the technical 
design; Phase II concerned 
system development and 
delivery.

31  Article 54(1) of Regulation (EC) 
No 1987/2006 of the European 
Parliament and of the Council 
of 20 December 2006 on the 
establishment, operation and 
use of the second generation 
Schengen Information System 
(SIS II) (OJ L 381, 28.12.2006, 
p. 4) and Article 70(1) 
of Council Decision 
2007/533/JHA of 12 June 2007 
on the establishment, 
operation and use of the 
second generation Schengen 
Information System (SIS II) 
(OJ L 205, 7.8.2007, p. 63).

32  Amendment No 4 of 

30 April 2008 to contract 
JAI-C3-2003-01.

30 
The tender specifications, published in 
June 2003, provided an initial out-
line of the system architecture and 
functional, technical, performance 
and migration requirements. On the 
basis of these specifications, HPS was 
awarded the contract to develop the 
central system in October 2004. It then 
began work on Phase I30, the design of 
the detailed specifications. However, 
subsequent changes to the system re-
quirements necessitated amendments 
to the development contract, which 
contributed to delays. 

31 
For example, in December 2006, legis-
lation on the establishment, operation 
and use of SIS II included an addi-
tional capability, the ‘data amnesty’, to 
meet the needs of users. This allowed 
member countries a period of 3 years 
to ensure that alerts transferred from 
SIS 1 complied with the provisions of 
the legislation31. The data amnesty was 
not part of the tender specifications 
and was incorporated in an amend-
ment to the contract32.

7 Evolution of the number of records in the SIS database
e
r
u
g
i
F

s
d
r
o
c
e
r
n
o
i
l
l
i

M

140
120
100
80
60
40
20
0

2001

2003

2005

2007

2009

2011

2013

2015

2017

2019

2021

2023

2025

Actual SIS 1

Capacity planned for SIS II

Projections of 2009 volumetric study

Source: European Court of Auditors based on Council statistics, SIS II documentation and Commission 2009 volumetric study. 

 
 
Observations

24

Section 2 — The cost of 
developing the central 
system and network 
increased from  
23 million euro to 
189 million euro 

Audit criteria

33 
The audit examined whether:

(a)  the Commission had produced 

a realistic initial budget;

(b)  the Commission had avoided de-
lays resulting in increased costs;

(c)  the Commission had ensured the 
contractor delivered products on 
time to the required quality;

(d)  requirements had been stable;

33  COM(2001) 720 final.

(e)  any additional work had been 

34  COM(2003) 771 final.

good value for money. 

The Commission did not 
update its 2003 global pro-
ject budget until 2010

34 
In June 2001 the Council estimated 
that it would cost 14,6 million euro 
to develop SIS II. The Commission’s 
Communication of December 200133 
increased this figure to 23 mil-
lion euro. Two years later, the Com-
mission estimated that it would cost 
35,3 million euro34. By the end of the 
project in 2013, the estimated cost for 
developing the system amounted to 
188,6 million euro (see Figure 8). 

8 Evolution of the SIS II budget
e
r
u
g
i
F

2001 Council  
Regulation and 
Decision

Design, development  
and migration

Network

Total contracts

Commission staff and 
administrative expenses

Total

14,6

14,6

Not available

14,6

2001 Commission 
Communication

2003 Commission 
Communication

(million euro)

2010 Commission 
Staff Working  
Document

Commitments 
to end of project 
April 2013

17,2

1,0

18,2

4,8

23,0

31,3

31,3

4,0

35,3

124,5

124,5

20,5

145,0

122,8

45,3

168,1

20,5  
(based on 2010 
estimate)

188,6

Source: European Court of Auditors based on SIS II documentation.

 
25

35  COM(2005) 236 final of  
31 May 2005 and  
COM(2005) 236 final/2 
of 23 August 2005. The 
budgetary envelope of 
132 million euro in the 
financial statement covered 
the running costs of the 
central SIS II system and the 
network and also included 
40 million euro for the 
development and operation 
of a search engine based on 
a biometric identifier.

36  SEC(2010) 436 final of 

12 April 2010.

37 
In the survey, although most respond-
ents were not involved in the SIS II 
project in 2001, 57 % considered that 
the initial project budget of 23 mil-
lion euro was unrealistic, based on 
the information available at the 
time. It was, for example, based on 
an unrealistic deadline (see para-
graphs 9 and 10) as was the revised 
estimate in 2003. Respondents to the 
audit survey were asked to weight the 
reasons for the increase in the costs of 
developing the central system. They 
considered that part of the increase 
was because the initial budget was not 
based on a realistic technical analysis. 
They gave a weighting of 24 % to this 
reason (see Figure 9).

Observations

35 
The Commission emphasised that its 
initial cost estimate in 2001, based on 
the information available at the time, 
was provisional, and described it as 
a working hypothesis. It was updated 
in December 2003 to take into account 
more recent information, including the 
feasibility study. However, it remained 
a working hypothesis, which could 
increase because of decisions on user 
requirements or the need for external 
support to meet the tight schedule. 

36 
The Commission did not present 
a more realistic revised estimate when 
it had more complete information 
on the design of the system and the 
network and when it realised the dif-
ficulty and complexity of the project 
and its environment. For example, the 
cost of the network was estimated 
at 1 million euro in 2001 and was not 
included in the 2003 estimates on the 
basis that the cost was covered else-
where in the Community budget. In 
May 2005 the Commission presented 
a budget of 132 million euro for the 
running costs of SIS II in the peri-
od 2007 to 2013, following its planned 
launch in early 200735. This contained 
information that the network costs 
for a dedicated line to national SIS II 
access points would amount to 6 mil-
lion euro per year. However, there was 
no further presentation of the global 
project budget for the development 
of SIS II until 2010 when the Commis-
sion set out commitments to date 
and the costs likely to be incurred to 
complete SIS II36. 

Observations

26

9 Reasons for the increase in costs of central SIS II
e
r
u
g
i
F

Other 3 %

Initial budget not based on a realistic
 technical analysis 24 %

Changes to requirements 27 %

Powerful negotiating
 position of contractor 5 %

Poorly performing contractor 10 %

Source: Audit survey of SIS II stakeholders.

Project took longer than
 anticipated 31 %

The delays in delivering SIS II 
resulted in increased costs

38 
SIS II was initially planned to be com-
pleted by December 2006 to meet the 
needs of new member countries, but 
it was delivered over 6 years late in 
April 2013 (see paragraphs 9 to 32). In 
the survey, respondents considered 
that the long extensions to the project 
deadline accounted for part of the in-
crease in costs. They gave a weighting 

of 31 % to this reason (see Figure 9). 
A longer project duration resulted in 
an increase in certain costs such as 
quality assurance and the use of the 
network. Although the main develop-
ment contract was to develop SIS II for 
a fixed price, the delays resulted, for 
example, in the need for upgrades (see 
paragraph 43) and increased capac-
ity to handle the sharp increase in the 
number of alerts from 2007 (see para-
graph 32). The factors which contribut-
ed to the delays of SIS II therefore also 
contributed to the increase in costs.

 
27

37  Comments on the 

Commission’s progress 
report for SIS II, Council of the 
European Union note from 
SIS-TECH WG to SIS-SIRENE 
WG, 8861/05 of 13 May 2005.

38  For the VIS contract with HPS, 
which was combined with the 
SIS II contract, the Commission 
imposed liquidated damages 
of some 9 million euro, 
representing over 20 % of the 
VIS contract value.

39  The June 2006 Internal 

Audit Service report on IT 
Management in DG Justice, 
Freedom and Security noted 
that the DG perceived the 
quality of deliverables as 
insufficient. It recommended 
that payments made for 
deliverables should better 
reflect the work performed 
and that future contracts 
should have an effective 
penalty mechanism to 
ensure the timely delivery of 
deliverables that meet basic 
quality standards.

Observations

The main development  
contractor delivered an 
underperforming system in 
the first part of the project

39 
The Commission had insufficient 
resources to supervise the main 
development contractor effectively 
(see paragraph 18). In 2005, member 
countries highlighted the poor qual-
ity of the first documents delivered by 
HPS37. Five of the six member countries 
interviewed, as well as the quality 
assurance contractor, considered 
the initial specifications provided by 
HPS were poor quality. The failure of 
the operational system tests (OST) in 
December 2008 showed that HPS had 
delivered an underperforming system 
(see paragraph 19). Its failure to deliver 
on time to the required quality con-
tributed towards project delays, which 
resulted in increased costs (see para-
graph 38). Survey respondents consid-
ered that part of the cost increase was 
due to the poor performance of the 
main development contractor. They 
gave a weighting of 10 % to this reason 
(see Figure 9). 

40 
The Commission could only impose 
liquidated damages on HPS amounting 
to 390 000 euro (which represented 
less than 0,5 % of the final contract 
value of 82,4 million euro38 (see para-
graph 44)) for its specific failure to 
deliver the OST report on time. Other 
performance problems were not sub-
ject to financial penalties39.

41 
The main development contrac-
tor passed the first milestone test in 
March 2010 and successfully imple-
mented the final phase of the pro-
ject, including the second milestone 
test in May 2012 (see paragraph 63). 
Although only 11 % of respondents to 
the survey considered that the Com-
mission engaged effectively with the 
main development contractor in the 
period before 2009, this proportion 
rose to 64 % for the period from 2009. 
Four of the six member countries inter-
viewed during the audit commented 
on the positive impact of the change 
in the management team of the main 
development contractor to implement 
the final project phase from 2010. 

Changes to system  
requirements resulted in 
increased costs

42 
Survey respondents considered that 
part of the increase in costs was due to 
additional or changed requirements, 
such as the introduction of the data 
amnesty or the increase in capacity 
(see paragraphs 31 and 32). They gave 
a weighting of 27 % to this reason 
(see Figure 9). 

Observations

43 
Changes to requirements necessitated 
amendments to the main development 
contract and had cost implications. 
The data amnesty cost 505 000 euro. 
The most significant amendment was 
No 15 in December 2010 amounting 
to 35,0 million euro for the final pro-
ject phase. This amendment entailed 
significant redevelopment40 and incor-
porated new requirements regarding 
the system capacity, performance, 
testing strategy, interface control 
document, upgrades of software and 
system migration41.

The Commission increased 
the value of the main devel-
opment contract from  
20 million euro to  
82 million euro through 
negotiated amendments

44 
The value of the initial contract with 
HPS to develop SIS II in October 2004 
was 20,3 million euro42. The contract 
was for a fixed price and did not 
contain prices for smaller, carefully 
defined work packages which could be 
used as the basis for adjusting the pro-
ject without renegotiations. Amend-
ments to the contract quadrupled its 
value to 82,4 million euro by 201343. 
Figure 10 shows the evolution of the 
value of the main SIS II development 
contract. 

28

40  Conclusions of the first 

meeting of the evaluation 
committee on the negotiated 
procedure C2-2009-01 SIS II, 
17 September 2010.

41  SEC(2010) 1138 final.

42  3,3 million euro for Phase I 

(the detailed technical design) 
and 17,0 million euro for Phase 
II (system development and 
delivery).

43  There were a total of 

27 amendments to the main 
development contract for SIS II 
and VIS (the contracts for both 
systems were combined in 
a single agreement).

0 Value of the main SIS II development contract
1
e
r
u
g
i
F

Nos 12, 13, 15
(Final project phase)

Nos 24, 25, 27

Amendment No 2

Nos 9, 11

No 5

No 4

90
80
70
60
50
40
30
20
10
0

e
v
i
t
a
l
u
m
u
c
o
r
u
e
n
o
i
l
l
i

m

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

Source: European Court of Auditors based on SIS II contract amendments.

 
 
 
Observations

45 
Amendments agreed between 2006 
and 2008 increased the value of the 
contract with the main development 
contractor by 14,1 million euro (69 %) 
to 34,4 million euro (see Figure 10). Be-
fore the first increase of 4 million euro 
in December 2006 (amendment No 2), 
DG Home Affairs sought the advice 
of DG Budget on whether it could 
negotiate an increase in the contract 
value with the contractor. DG Budget 
advised that, under Article 126(1)(e) of 
the Rules implementing the Financial 
Regulation, DG Home Affairs could 
negotiate amendments for additional 
services which were not in the ini-
tial contract but which had become 
necessary. Under this article, the total 
value of amendments could not ex-
ceed 50 % of the amount of the initial 
contract44. The initial value of the SIS II 
contract was 20,3 million euro, so the 
total value of amendments agreed 
under Article 126(1)(e) could not ex-
ceed 10,2 million euro.

46 
In 2008 the total value of amend-
ments reached the 50 % limit. DG 
Home Affairs negotiated subsequent 
amendments under Article 126(1)(b) of 
the Rules implementing the Financial 
Regulation45. This article had no ceiling 
on the total value of amendments. 
However, it could only be applied on 
the condition that the contract could 
only be awarded to one particular 
company (referred to as a technical 
lock-in). 

47 
The tender specification for SIS II had 
not planned the system as a set of 
building blocks which could easily 
be reused by another contractor46. 
DG Home Affairs decided that, for 
technical reasons, the contract for 
the 35,0 million euro amendment 
in 2010 for the final project phase 
could only be awarded to the main 
development contractor, HPS. It con-
sidered that no other contractor would 
have been able to deliver SIS II to the 
required quality and within the time-
frame for the same price. In addition, 
a new tendering exercise would have 
required more time. Moreover, the 
Commission considered that because 
the SIS II contract was combined with 
the contract for the Visa Information 
System, terminating the contract for 
SIS II could have had a disruptive effect 
on the development of the latter.

48 
The Commission considered that, 
from a sound financial management 
point of view, it could not justify a new 
competitive tendering procedure. 
As a result, it increased the value of 
a contract, with a contractor who had 
delivered an underperforming sys-
tem in 2008, from 20,3 million euro 
to 82,4 million euro47. This was done 
through a series of negotiated amend-
ments. Whilst this procedure is pro-
vided for in the Financial Regulation, 
there is an inherent risk with any 
negotiated procedure that the prices 
paid are not competitive. 

29

44  Article 126(1)(e) of the 

Implementing Rules to 
the Financial Regulation 
states that contracting 
authorities may use the 
negotiated procedure 
(i.e. no competition) for 
additional services which 
were not in the initial 
contract but which, through 
unforeseen circumstances, 
have become necessary to 
complete the performance 
of the services and which 
cannot be separated from the 
main contract without serious 
inconvenience. The total 
value of additional contracts 
may not exceed 50 % of the 
amount of the initial contract.

45  Article 126(1)(b) of the 
Implementing Rules to 
the Financial Regulation 
states that contracting 
authorities may use the 
negotiated procedure 
(i.e. no competition), where, 
for technical or artistic reasons, 
or for reasons connected with 
the protection of exclusive 
rights, the contract can be 
awarded only to a particular 
company.

46  This component-based 
approach is described in 
the software development 
methodology used in the 
European Commission  
(RUP@EC).

47 

In 2012 amendments of 
4,7 million euro brought the 
total value to 82,4 million euro.

30

48  Mail from DG Home Affairs 
to the main development 
contractor, 7 September 2010.

49  SCH/OR.SIS(96) 165 of 

15 December 1996, Note 
from the Presidency of the 
SIS Steering Committee 
estimated the cost of the  
new central system to be  
50–60 million French francs 
(8–9 million euro) and the cost 
of each new national system 
to be 20–25 million French 
francs (3–4 million euro).

Observations

49 
Aware of this risk, the Commission 
took steps, in negotiating amend-
ment No 15 for the final project 
phase, to provide some assurance 
that the price was reasonable. The 
initial offer submitted by HPS was for 
49,9 million euro. Once consensus 
on the content of the offer had been 
achieved, the Commission made its 
own estimate of 25,8 million euro. It 
then compared the prices of a sample 
of 10 % of hardware items with those 
in a framework contract managed by 
DG Informatics48. It found that prices 
proposed were twice as high and was 
able to benefit from the best cus-
tomer clause guaranteeing the best 
price to the Commission. As a result of 
these measures it reduced HPS’s initial 
offer of 49,9 million euro by 30 % to 
35,0 million euro. Competitive tender-
ing might have saved even more.

Section 3 — The Commis-
sion did not ensure that 
the business case for SIS II 
included a thorough  
reassessment of the costs 
and expected benefits, 
updated to take into 
account major changes

Audit criteria

50 
The audit examined whether the Com-
mission reassessed the business case 
for SIS II when major changes occurred 
to the costs and the expected benefits.

After being made responsi-
ble for the project, the Com-
mission’s updated business 
case for SIS II did not include 
a thorough reassessment of 
all costs, expected benefits 
and alternatives 

51 
The Schengen countries took the 
decision to develop SIS II in Decem-
ber 1996 on the basis of a comparison 
with the advantages and disadvantag-
es of extending SIS 1. The cost esti-
mates for SIS II included an estimate 
of the cost of developing national sys-
tems and recognised that they would 
far exceed the costs of developing the 
central system49.

Observations

52 
In 2001 the Council described the costs 
and benefits of SIS II in the following 
terms50:

(a)  It would cost 14,6 million euro to 
develop the central system.

(b)  The primary reason for developing 
SIS II was to link up the increased 
number of member countries after 
enlargement. SIS II was also to ben-
efit from the latest technological 
developments and new functions.

53 
Most survey respondents (76 %) 
found this initial justification for SIS II 
convincing and considered that the 
benefits justified the costs. In 2001, 
there was no requirement for the SIS II 
project proposal to be submitted to 
DG Informatics for approval. Corpo-
rate IT governance and controls were 
strengthened within the Commission 
in 2004 through a Communication on 
IT governance51, which recommended 
that DG Informatics should provide 
an opinion on the development of 
any new information systems being 
planned by directorates-general inde-
pendent of budget source (whether 
administrative or operational). In prac-
tice, from 2004, this procedure was fol-
lowed only by projects financed from 
the administrative budget, but gener-
ally not by projects, like SIS II, financed 
from operational budgets52.

54 
After being made responsible for the 
development of SIS II, the Commission 
issued Communications in Decem-
ber 2001 and December 2003 which 
built on the justification in the 2001 
legislation and revised the estimate 
of costs (see Figure 8). However, the 
Commission’s updated business case 
for SIS II did not include a thorough 
examination of the costs, benefits and 
alternatives:

(a)  The initial indicative cost esti-
mates, not revised until 2010, 
significantly underestimated the 
true scale of the investment neces-
sary — some 500 million euro by 
the end of the project: 189 million 
euro for developing the central 
system (see paragraph 34), and an 
estimate of over 330 million euro 
for developing national systems53. 
The cost of developing national 
systems was met from national 
budgets, but included support 
of 95 million euro from the EU 
External Borders Fund. National 
systems are an integral part of the 
Schengen Information System as 
the main end-users of SIS II will 
have access to the central system 
via their national system (see Fig-
ure 1). Although the Commission 
was not responsible for delivering 
national SIS II projects, their coor-
dination was essential in order for 
the project to succeed54. This was 
recognised by the member coun-
tries in their initial 1996 proposal 
for SIS II (see paragraph 51). How-
ever, neither the initial, nor any 
subsequent, estimate presented by 
the Commission included the cost 
of their development. 

31

50  Council Regulation (EC) 
No 2424/2001, Council 
Decision 2001/886/JHA and 
initiative of the Kingdom 
of Belgium and of the 
Kingdom of Sweden with 
a view to the adoption of 
a Council Regulation on 
the development of the 
second generation Schengen 
Information System (SIS II) 
(9844/2001).

51  SEC(2004) 1267. Corporate IT 

governance and controls were 
strengthened through further 
Communications in 2010 and 
2011 partly as a response 
to the awareness of the 
reputational risk raised by the 
implementation of large-scale 
IT projects: SEC(2010) 1182 
and SEC(2011) 1500 of 
30 November 2011.

52  The Methodology, 

Architecture, Portfolio 
Management Working 
Group (MAP) chaired by DG 
Informatics should review 
project definition documents 
produced by DGs before 
the administrative budget is 
released. 

53  A 2009 report to the SIS-TECH 
Working Group (8708/09 
of 19 May 2009, note from 
Presidency on Alternative 
Scenario Assessment Paper 
SIS 1+RE) estimated that 
costs invested by member 
countries on national SIS II 
systems until May 2009 
amounted to 170 million euro. 
Estimates by the member 
countries in the context of 
the Commission’s mid-term 
evaluation of the External 
Borders Fund identified 
further investments in SIS II 
national systems of some 
160 million euro between 
2009 and 2012. The total 
investment is considerably 
higher, as figures for 
Denmark, Estonia, Germany, 
Greece, Luxembourg, the 
Netherlands, Portugal, 
Spain and Switzerland 
were not available. Out 
of this total investment in 
national systems of over 
330 million euro to the end 
of 2012, the Commission, 
through the External Borders 
Fund, provided 76 million euro 
and a further 19 million euro 
in 2013. 

32

54  The SIS II project plan, 

3 March 2008, describes how 
the SIS II project also included 
the coordination of the 
national preparation in each 
country participating in SIS II. 
It states that ‘the Commission 
is not in charge of delivering 
these projects but since 
national preparation impacts 
the SIS II project it is viewed as 
essential’.

55 

Initially the development 
contract covered Phase I only 
(design). The activation of 
Phase II (development and 
delivery) was at the discretion 
of the Commission based on 
the results of Phase I and the 
availability of the budget.

56  The Information Systems 

Project Management Board 
(ISPMB), chaired by the 
Director-General of DG 
Informatics, reviews the 
annual progress reports of 
major IT projects and provides 
early warning on difficulties 
encountered. DG Home Affairs 
agreed with the Commission’s 
IT governance bodies that the 
ISPMB was not applicable to 
the SIS II project.

57  COM(2010) 633 final 
of 5 November 2010, 
COM(2011) 391 final of  
29 June 2011. 

Observations

(b)  It did not set out the benefits of 
SIS II in terms of its contribution 
to fighting crime or strengthening 
external borders. It did not state 
the problems SIS II was designed 
to address and how its success 
would be measured.

(c)  It did not consider alternative sce-
narios. For example, there was no 
explanation of why it was not pos-
sible to extend SIS 1 to additional 
member countries. Five of the six 
member countries interviewed 
during the audit considered that 
SIS 1 could have been extended to 
the new member countries. When 
SIS II was not available as planned 
at the end of 2006, SIS 1 was in 
fact modified and SISone4all was 
launched in December 2007 to 
incorporate the new member 
countries (see paragraph 11).

The Commission did not 
thoroughly reassess the busi-
ness case during the project 
even though there were 
major changes to the costs 
and expected benefits

55 
Key decisions such as the 2004 de-
cision to begin detailed planning, 
the 2005 decision to activate Phase 
II of the project55, and the 2009–10 
decision to continue SIS II after 
the comparison with SIS 1+RE (see 

paragraphs 62 and 63) were made 
within the decision-making framework 
of the project. The corporate mecha-
nism introduced in the Commission 
from 2010 for deciding whether to 
continue with, or stop, an IT project 
was not applicable to the SIS II pro-
ject56. Decisions to continue the devel-
opment of SIS II were financed within 
the budget approved for the running 
costs of SIS II after its launch (see para-
graph 36)57. Decisions to continue the 
SIS II project did not involve a thor-
ough reassessment of the overall costs 
and expected benefits even though 
there were major changes to them.

56 
The costs of developing the central 
system were much higher than initially 
estimated (see paragraphs 34 to 49). 
The longer than expected duration of 
the project also led to additional costs 
for upgrading SIS 1 (see paragraph 59). 
Delays and changes to the design of 
SIS II also increased the cost of devel-
oping national systems. Four of the six 
member countries interviewed de-
scribed how they had purchased mate-
rial for their national system which had 
not been used because it had become 
obsolete. In the audit survey, 74 % of 
respondents from member countries 
considered that the need to find ad-
ditional funds to finance the repeated 
changes and extensions to the project 
deadline put at risk the development 
of their national systems. 

33

58  DG Home Affairs’ 2011 Annual 
Activity Report included 
a reservation concerning the 
reputational damage to the 
Commission from SIS II.

Observations

57 
Whilst the costs of SIS II increased, 
the main expected benefit (connect-
ing the increased number of member 
countries after enlargement) was 
less important because of the im-
plementation of SISone4all in 2007 
(see paragraph 11). In the audit survey, 
respondents were asked to weight the 
justifications for developing SIS II at 
the start and at the end of the project. 
Respondents considered that the most 
important justification at the start of 
the project was linking up more mem-
ber countries. However, by the end 
of the project the importance of this 
justification had fallen from 36 % to 
15 % (see Figure 11). Other reasons for 
developing SIS II were also different at 
the end of the project from the start. 
For example, the avoidance of harm to 
the EU’s reputation58 and the protec-
tion of investments already made were 
significant at the end of the project. 

1 Justification for developing SIS II
1
e
r
u
g
i
F

40 %

35 %

30 %

25 %

20 %

15 %

10 %

5 %

0 %

More member
countries

New functions

Capacity and
performance

Transfer SIS to
EU institutions

Avoid harm to
EU’s reputation

Protect 
investments made

Other

Start of project

End of project

Source: Audit survey of SIS II stakeholders.

 
Observations

34

58 
The second most important justifica-
tion for SIS II at the start of the project 
(providing new functions) became the 
most important justification by the 
end. SIS II provides end-users with new 
functions such as new categories of 
alert (stolen aircraft, boats, containers, 
means of payment), the facility to link 
alerts (such as an alert on a person and 
a vehicle) and the capacity to store 
documents associated with an alert 
(e.g. European arrest warrants) includ-
ing biometric information. 82 % of 
respondents considered that SIS II had 
significant additional functions which 
provided end-users with immedi-
ate visible practical advantages over 
SIS 1 (see Figure 12). 

59 
However, by the end of the project, 
apart from the new functions, re-
spondents were less certain about the 
benefits of SIS II compared with SIS 1. 
Respondents to the survey consid-
ered that the second most important 
justification for SIS II by the end of 
the project were the non-functional 
advantages including greater capacity 
and performance. However, just 52 % 
of respondents considered that SIS II 
has significant non-functional advan-
tages over SIS 1 (see Figure 12). This 
was because there had been a number 
of upgrades to SIS 1 due to the delays 
in developing SIS II (see the chronol-
ogy in the Annex): 

2 Advantages of SIS II over SIS 1
1
e
r
u
g
i
F

100 %

80 %

s
t
n
e
d
n
o
p
s
e
r

60 %

40 %

20 %

0 %

SIS II has significant
 additional functions
 which provide practical
 advantages over SIS 1

SIS II has non-functional
 advantages over
 SIS 1 (capacity and
 performance)

Benefits of SIS II
 justify the costs

There was an effective
 mechanism for deciding
 whether to stop SIS II,
 based on the project
 justification

Disagree

Neutral

Agree

Source: Audit survey of SIS II stakeholders.

 
35

59  Council document 6067/09, 

3.2.2009, note from Presidency 
on SIS II implementation 
measures.

60  Council 10005/09 of 

20 May 2009, Note from 
Presidency and Commission 
Services to Article 36 
Committee, Report on the 
further direction of SIS II and 
SEC(2010) 436 final.

61  The options explored, for 

example, in the Commission 
Staff Working Paper of 
31 August 2006 on global SIS II 
rescheduling all involved the 
continued development of 
SIS II. In November 2006 the 
Commission insisted that it 
was technically not possible 
to add on new functions to 
SIS 1. See House of Lords, 
European Union Committee, 
9th Report of Session 2006–07, 
Schengen Information 
System II (SIS II), report with 
evidence, 2 March 2007 reply 
to question 441.

62  Council Conclusions, Justice 
and Home Affairs, 26 and 
27 February 2009.

Observations

(a)  In 2006 when the hardware was 
renewed and functions added it 
was renamed SIS 1+R. 

(b)  In 2007 additional member 

countries were connected via 
SISone4all. 

(c) 

In 2011, due to expired main-
tenance contracts, SIS 1+R was 
redesigned and reprogrammed in 
Java (instead of the previous C++). 
Two of the six member countries 
interviewed during the audit con-
sidered that the performance of 
the new modular system, renamed 
SIS 1+R2, was equivalent to that of 
SIS II.

60 
By 2009 SIS II was already 2 years be-
yond the initial deadline, the costs had 
increased, the expected benefits had 
diminished, and the system had failed 
the operational system tests at the end 
of 2008. Despite the difficulties, delays 
and cost increases, most stakehold-
ers wanted to continue to build SIS II. 
Only 19 % of respondents to the audit 
survey considered that the devel-
opment of SIS II should have been 
stopped. Nevertheless, the project had 
arrived at a major decision point, and, 
in February 2009, the Council Presi-
dency called for careful consideration 
of alternative scenarios59. 

61 
There was, however, no decision on 
whether to continue with, or stop the 
project based on an analysis of costs 
and benefits. Only 30 % of respond-
ents to the survey considered that an 
effective mechanism for making such 
a decision existed (see Figure 12). 
The Commission did not reassess the 
overall costs (including the costs of 
developing national systems) and the 
expected benefits of SIS II (including 
its contribution to strengthening the 
security of the EU). It did not demon-
strate that the project remained an 
organisational priority which provided 
a higher return on investment than 
any other opportunities. In the au-
dit survey, only 38 % of respondents 
considered that the expected benefits 
of SIS II justified the final costs of its 
development (see Figure 12).

62 
Although it did not reassess the overall 
costs and benefits of SIS II, in May 2009 
the Commission contributed to a com-
parison with the alternative scenario 
of SIS 1+RE, which involved modifying 
SIS 1+R to include SIS II functions 60. 
This acknowledged, for the first time, 
that there was an alternative to SIS II 61. 
The Commission recognised that the 
financial implications for the member 
countries’ national SIS II projects were 
considerable, however the comparison 
did not include the costs of develop-
ing national systems62. Five of the six 
member countries interviewed during 
the audit perceived the comparison as 
unfairly weighted in favour of continu-
ing SIS II. They considered that the 
comparison underestimated the time 
and the cost necessary to complete 
SIS II and emphasised the difficulties of 
switching to SIS 1+RE. 

Observations

63 
In June 2009 the Council decided 
to continue with SIS II 63. However, it 
introduced two milestone tests and 
a mechanism to stop the development 
contract and to switch to the alterna-
tive scenario of SIS 1+RE if the system 
failed one of the tests64.

36

63  Council 10708/09 of 
5 June 2009, Council 
Conclusions on the further 
direction of SIS II.

64  The system passed the first 

milestone test in March 2010 
and the second milestone test 
in May 2012.

Section 4 — The Commis-
sion learnt lessons from 
its experience during the 
first part of the project to 
complete SIS II and  
prepare future projects

Audit criteria

64 
The audit examined whether the 
Commission had carried out an evalu-
ation of the SIS II project. It examined 
whether lessons learnt from the 
experience of managing SIS II were 
applied to later stages of the project, 
to other IT projects in the Commission 
and to the new Agency for large-scale 
IT systems (eu-LISA).

The Commission applied  
lessons from the first part of 
the project to the final phase

65 
The Commission applied knowledge, 
experience and understanding ac-
quired during the first part of the 
project to successfully deliver the 
final project phase. The audit sur-
vey highlighted a clear distinction 
between the first part of the project 
to 2009 and the final project phase 
from 2010. Figure 13 shows how the 
Commission changed its approach in 
the final phase of the project in order 
to address the reasons for the delays 
and cost increases and complete 
SIS II. Two of the six member countries 
interviewed commented that the final 
project phase was managed in a way 
that was appropriate to a large-scale 
IT project.

Observations

37

3 Lessons learnt from the unsuccessful first part of the project
1
e
r
u
g
i
F

Changed approach in final project phase 
from 2010

Reason for delays and increased costs in SIS II 

Unrealistic deadline

Lack of Commission expertise to manage  
outsourced development work effectively 

The relations between some member countries  
and the Commission were poor and it did not  
secure their commitment

Instability of the system requirements

The deadline of March 2013 to complete the project, 
valid from June 2010, was realistic and was met 

Commission allocated more resources to the project 
and was able to recruit staff with relevant  
experience (e.g. from SIS II itself, from DG Taxation 
and Customs Union and from member countries)

The 2009 comparison with SIS 1+RE provided  
some additional justification for the continued 
investment
The creation of the Global Project Management 
Board in 2009 enabled the Commission to draw 
more fully on the experience of end-users in  
member countries

System requirements were stable during the final 
project phase

Source: European Court of Auditors based on survey of SIS II stakeholders.

The Commission has already 
applied lessons from SIS II to 
other projects

66 
The Commission has not carried out 
a formal evaluation with representa-
tives of all stakeholders to identify 
lessons learnt from its management 
of the project. However, it has applied 
some lessons learnt from SIS II to other 
projects at the Commission in the fol-
lowing ways:

(a)  through project management 

guidelines;

(b)  through communications draw-
ing general lessons from SIS II to 
be applied to the development 
of future IT systems in the area of 
freedom, security and justice;

(c)  by already taking specific meas-
ures when planning future large-
scale IT systems.

 
38

65  PM2 is the project 

management methodology 
recommended to directorates-
general by DG Informatics 
since 2011.

66  COM(2011) 680 final of 
25 October 2011.

67  COM(2010) 385 final of 

20 July 2010.

68  Regulation (EU) No 1077/2011 

of 25 October 2011 
establishing a European 
Agency for the operational 
management of large-scale 
IT systems in the area of 
freedom, security and justice 
(OJ L 286, 1.11.2011, p. 1).

69  COM(2011) 680 final.

70  SEC(2012) 492 e-Commission 
2012–2015 states that all 
Commission information 
systems, including offices 
and executive agencies, 
whether financed under 
operational or administrative 
budgets, are subject to the IT 
governance arrangements of 
the Commission.

Observations

67 
The Commission’s latest project 
management methodology (PM2)65 
emphasises procedures which were 
lacking in SIS II such as:

69 
In planning the smart borders project 
the Commission has already taken the 
following concrete steps, reflecting 
lessons learnt from SIS II:

(a)  the important role of the business 
manager in ensuring user needs 
are met.

(b)  the approval by the project steer-
ing committee of the transition of 
a project through approval gates 
at the end of the initiating, plan-
ning and executing phases.

68 
Communication 2011(680) on smart 
borders66 insisted that lessons learnt 
from the development of large-scale 
IT projects like SIS II should be taken 
into account in the development of the 
entry/exit system (EES) and the regis-
tered traveller programme (RTP). Com-
munication 2010/385 concerning an 
overview of information management 
in the area of freedom, security and 
justice67 described how the Commis-
sion would develop future IT systems. 
It used the example of SIS II to high-
light the following two key lessons:

(a)  the need for requirements to be 

set out in legislation before devel-
opment work starts;

(b)  the importance of an effective 

governance structure.

(a)  The legislative financial statements 
for the entry/exit system and the 
registered traveller programme 
planned to meet the cost of devel-
oping national systems from the 
EU budget.

(b)  The new Agency for large-scale IT 
systems (eu-LISA)68, which became 
responsible for managing and 
maintaining SIS II after its launch, 
is intended to bring together the 
expertise required to develop 
future systems and so limit the risk 
of problems such as those which 
arose during the development of 
SIS II69. In the audit survey, 68 % 
of respondents considered that 
the creation of the new agency 
would facilitate the development 
of future large-scale IT systems. 
However, all member countries 
interviewed during the audit high-
lighted the fact that relations with 
member countries would be a key 
challenge for the new agency. As it 
is a regulatory agency, it does not 
fall under the Commission’s corpo-
rate IT governance arrangements70.

Conclusions and 
recommendations

39

70 
The Council charged the Commission 
with the development of SIS II and 
fixed the target date for completion 
as December 2006. This was to enable 
the new member countries, which 
were to join the EU in 2004, to become 
part of the Schengen Area. However, 
the Commission delivered the central 
system over 6 years later than planned, 
in April 2013, and at 8 times the initial 
budget cost. The delay and overspend-
ing resulted partly from the challeng-
ing governance context which limited 
the Commission’s ability to address op-
erational issues and partly from weak-
nesses in the Commission’s manage-
ment. This was particularly the case in 
the first part of the project to 2009:

(d)  The Commission did not at first 
allocate sufficient staff with the 
expertise to manage outsourced 
development work effectively (see 
paragraphs 13 to 20).

(e)  The main development contractor 
delivered an underperforming sys-
tem in the first part of the project 
(see paragraphs 39 to 41).

(f)  Working relations between some 
member countries and the Com-
mission were initially poor. Until 
the Commission created the 
Global Project Management Board 
in 2009, it did not manage to draw 
on the experience of all end-users 
(see paragraphs 21 to 25). 

(a)  The initial project deadline was 

(g)  It was not clear to all stakehold-

unrealistic (see paragraphs 9 to 12). 

ers who made key decisions (see 
paragraphs 26 to 28). 

(b)  The Commission did not update 

its 2003 global project budget un-
til 2010 (see paragraphs 34 to 37).

(c)  System requirements evolved dur-
ing the project to meet the needs 
of users and were not sufficiently 
stable until the final project phase 
from 2010. The system now in op-
eration has a much larger capacity 
than the one foreseen in 2001 (see 
paragraphs 29 to 32 and 42 to 43).

(h)  The Commission increased the 
value of the main development 
contract from 20 million euro to 
82 million euro through negoti-
ated amendments. Whilst this 
procedure is provided for in the 
Financial Regulation, there is an 
inherent risk with any negotiated 
procedure that the prices paid 
are not competitive (see para-
graphs 44 to 49).

Conclusions and recommendations

40

72 
The Commission learnt lessons from its 
experience during the first part of the 
project in order to change its approach 
during the final project phase and to 
complete SIS II. In addition, although 
the Commission has not carried out 
a formal evaluation of the project, it 
has already applied some lessons from 
SIS II in preparing other large-scale IT 
projects (see paragraphs 64 to 69).

71 
After being made responsible for the 
project in 2001, the Commission’s 
updated business case for SIS II did not 
include a thorough examination of all 
costs and expected benefits. During 
the project, there were major changes 
to the costs and expected benefits. 
The initial indicative estimates of the 
costs to the EU budget for the central 
system significantly underestimated 
the true scale of the investment neces-
sary (see Figure 8). Taking into account 
the national systems costs the overall 
costs amounted to some 500 mil-
lion euro by the end of the project. 
Although SIS II provides new functions 
and greater capacity, the main benefit 
initially expected (connecting the in-
creased number of member countries 
after enlargement) became less rel-
evant because of the implementation 
of SISone4all in 2007. However, despite 
these changes, decisions to continue 
the project were taken without thor-
oughly reassessing the business case 
for SIS II in order to demonstrate that 
it remained an organisational prior-
ity which provided a higher return on 
investment than other opportunities 
(see paragraphs 50 to 63).

Conclusions and recommendations

41

Recommendation 1

Recommendation 4

The Commission should base the 
project timetable for the development 
of large-scale IT systems on a technical 
analysis of the tasks to be performed, 
taking into account the risks and com-
plexity of the project.

Recommendation 2

The Commission should include all 
large-scale IT projects under its re-
sponsibility in its corporate IT govern-
ance arrangements. There should be 
no distinction between IT projects 
funded from operational budgets and 
those funded from the administra-
tive budget. The Commission should 
involve IT experts from DG Informatics 
or other directorates-general at the 
inception of large-scale IT projects. 
It should include a representative of 
DG Informatics in the project steer-
ing committee. When outsourcing 
development work, the Commission 
should ensure that it makes full use 
of in-house expertise to manage the 
contract and supervise the work of the 
development contractor effectively 
in order to minimise the risk of poor 
performance. 

Recommendation 3

The Commission should ensure 
that those representing users of the 
system understand and take into ac-
count business needs and end-user 
requirements. 

The project steering committee should 
approve the transition of a project 
through approval gates to the next key 
project phase. Before proceeding from 
project initiation to project planning, 
the project steering committee and 
the relevant Commission IT govern-
ance body should approve the busi-
ness case. 

The business case should include pos-
sible alternatives, a comprehensive 
cost-benefit analysis which identifies 
the full costs of the project and a re-
source plan which specifies the staff-
ing needed on the user and IT sides. 

The business case should be kept up 
to date and reapproved in the event 
of major changes to project costs, ex-
pected benefits, risks or alternatives.

Recommendation 5

The Commission should document key 
project decisions in a decision log so 
that they are easily traceable.

Recommendation 6

When a project requires the devel-
opment of different but dependent 
systems by various stakeholders, the 
Commission should ensure that there 
is an effective overall project manage-
ment to coordinate the whole project.

Conclusions and recommendations

42

Recommendation 7

Recommendation 8

The Commission should pass on the 
lessons learnt from the Court’s audit 
to directorates-general and EU institu-
tions, agencies and other bodies. The 
Commission should evaluate whether 
the expected benefits of SIS II were 
achieved.

The Commission should ensure that 
contracts to develop large-scale IT 
systems contain prices for smaller, 
carefully defined work packages which 
can be used as the basis for adjusting 
the project without renegotiations. 
Furthermore, the Commission should 
develop large-scale IT systems us-
ing interlinked building blocks which 
can easily be reused, for example 
to pass completed work from one 
contractor to another. Although it 
requires an investment in design, this 
component-based software develop-
ment approach minimises the impact 
of changes to requirements and is 
a means to benefit from advances in 
technology and avoid technical lock-in 
with contractors.

This report was adopted by Chamber IV, headed by Mr Louis GALEA, Member of 
the Court of Auditors, in Luxembourg at its meeting of 18 March 2014.

For the Court of Auditors

Vítor Manuel da SILVA CALDEIRA
President

Annex

43

Chronology of SIS II

x
e
n
n
A

BACKGROUND
1985
Signature of the
Schengen
Agreement

1990
Convention implementing the
Schengen Agreement –
provides for the Schengen
Information System (SIS)

1995
Schengen Area created and
SIS 1 becomes operational

Princess Marie-Astrid

Moselle

1
S
I
S

2001
SIS 1+ includes Nordic
countries

2006
Hardware renewed 
SIS 1+R

+
1
S
I
S

SIS 1+R

December 2007
SISone4all launched
to include 9 new
member countries

START OF SIS II
1996
Schengen countries
decide to develop
SIS II

2001
Regulation 2424/2001
makes Commission 
responsible for
development of SIS II

October 2004
contract signed
with Hewlett
Packard Steria

SIS 1

SIS II under construction

Dec 2006 
Initial
planned
end

Dec 2008
Revised 
planned
end

2006
Regulation (EC) No 1987/2006 
on establishment, operation 
and use of SIS II

Nov 2004 to Jan 2005 
Project on hold due to 
challenge of
unsuccessful tenderer

SIS II under construction

Dec 2008 
Failed operational system tests

2009
Commission creates Global
Project Management Board

R
+
1
S
I
S

June 2009 
Council Conclusions 
SIS 1+RE agreed 
as a fallback option

SIS I+RE

SIS II under construction

June 2010 
Council
endorses ﬁnal 
project phase

June 2010
Revised 
planned
end

March 2010 
First milestone test 
passed

Oct 2011 
Regulation (EU) No 1077/2011
establishing European Agency for
operational management of large-
scale IT systems

SIS II under construction

May 2012
Second milestone test
passed

SIS II under construction

2011 
reprogrammed 
in Java
SIS 1+R2

2
R
+
1
S
I
S

April 2013 
launch of 
SIS II

Source: ECA.

 
 
 
 
Reply of the Commission

44

Executive summary

II
The development of SIS II was started under a very 
particular legal and institutional framework apply-
ing at the time to this policy area. The Council took 
the decision to develop the SIS II on its own initia-
tive and without a Commission proposal.

Many requirements were not foreseen from the 
start but were added only later. They were therefore 
not adequately estimated by the Council in fixing 
the target date or the budget.

III
The fact that SIS II was not delivered in line with the 
target date and budget estimate set in 2001 is in no 
doubt and has been known for some time. However 
the system that was delivered was very different 
from the one foreseen. The Commission considers 
that SIS II is performing adequately and fulfils the 
needs of the users.

IV
The Commission has fully reported on many occa-
sions that the system would be delivered later and 
at a greater cost than initially planned and pro-
jected. It has also explained that the primary cause 
for this was the substantially changing system 
requirements.

IV (a)
The initial project deadline was set by the Council 
on the basis of its own analysis and without input 
from the Commission.

The Commission drew the attention of the Member 
States, in communications in 2001 and 2003, to the 
risks entailed with that deadline and the assump-
tions that would need to be met (and eventually 
were not) if the deadline was to be maintained.

IV (b) 
The Commission presented a realistic project 
budget in 2010 once it had complete information, 
notably on the system requirements.

IV (c)
The fact that the system requirements were con-
stantly evolving (mostly following MS requests) was 
a cause of the delays and the increase in costs. It 
also had a huge impact on the contract manage-
ment aspect of the project, since this implied sev-
eral revisions of the contract, which in turn did not 
facilitate the relationship with the contractor who 
was constantly faced with changing requirements.

IV (d) 
The lack of staff was experienced in the whole 
policy area which was growing very rapidly.

IV (f) 
The working relations between the Commission and 
some Member States was strained at times during 
the initial phase of the SIS II project. However DG 
Home Affairs, on the basis of the same contract 
and with the same contractors, delivered another 
large-scale IT system (VIS) in a constructive and 
harmonious working atmosphere with Member 
States, and with no ambiguities/challenges towards 
the decision-making process. The same can be said 
of the development of Eurodac.

The relationship with the SIS Community was less 
easy for a number of reasons such as a legacy 
system, a complex governance structure, policy 
considerations related to the overall approach of 
some member countries towards the enlargement 
of the Schengen area.

The governance structure was complex, reflect-
ing both legislative requirements and evolving 
stakeholder´s needs.

Reply of the Commission

45

IV (g)
Throughout the SIS II project, the Commission fully 
followed the Financial Regulation rules and negoti-
ated in the best interest of the Union budget.

V 
The initial business case had been analysed in the 
Council without a formal role of the Commission. 
During the life of the project, the project became 
subject to co-decision and the co-legislators con-
firmed that SIS II continued to be an absolute prior-
ity, notwithstanding the problems encountered.

The Commission systematically reassessed cost 
estimates at key milestones of the project when 
justified by new elements and always transparently 
communicated on budgetary matters, as spelled 
out under paragraph (70)(b).

There was no requirement on the Commission to 
estimate or track national expenditures for SIS II.

VI
The Commission confirms that it indeed has drawn 
and applied several key lessons from the devel-
opment of this system. Nevertheless, it has to be 
recognised that the development of SIS II and its 
implementation are a very specific undertaking dif-
ficult to replicate to other IT systems.

VII
Most of the Court’s recommendations reflect the 
Commission IT governance arrangements in place 
since 2010.

VII (h) 
The Commission can accept this recommenda-
tion. The lessons learnt from the Court’s audit will 
be shared with the other directorates-general at 
both technical and management levels, as well as 
disseminated to the relevant agencies, including 
eu-LISA, through the appropriate networks.

The impact of SIS II will be assessed in 2016 (3 years 
after SIS II entered operation), as required by 
the legal base (Article 50 paragraphs 4 and 5 of 
Regulation (EC) No 1987/2006 and Article 66 para-
graphs 4 and 5 of Decision 2007/533/JHA).

Introduction

01 
The initial Schengen Information System (SIS 1) was 
developed as an intergovernmental project led by 
France and became operational in 1995 (also sig-
nificantly later than initially planned). It remained 
in operation, managed by France, until May 2013, 
when it was replaced by SISII. Although both the 
architecture and functionalities of the systems dif-
fer substantially.

The Commission considers that the comparison 
between the original deadline and the actual deliv-
ery date should take account of the large difference 
between the nature of the system initially planned 
and the one that was delivered.

02
A year after SIS 1 began operations, the Schen-
gen countries decided in December 1996 to start 
work on defining the elements and requirements 
of an SIS II. The Council took its decision to build 
SIS II on the basis of a feasibility study conducted 
in 1998 and a series of consultations among Mem-
ber States culminating with the Swedish/Belgian 
initiative that was the basis for the first SIS II legal 
basis.

Reply of the Commission

46

The Member States worked within an intergovern-
mental framework and the Commission was not 
involved.

Once the Commission was entrusted with the 
development of SIS II, it launched a feasibility study 
to assess the business needs to be served by the 
new system, the technical choices and their impact, 
the timeline of the project and its budget, as well 
as the risks and constraints associated with the 
development. This feasibility study was carried out 
by a consultancy company in 2003. On this basis, 
the Council opted for one of the proposed imple-
mentation options (but not the one recommended 
by the feasibility study) in order to go ahead with 
the project.

03
While it is true that the primary reasons for develop-
ing SISII were the need to connect the new Schen-
gen member countries and benefit from latest tech-
nological developments, important political events, 
like the September 2001 attacks, changed the scope 
of the projects and the final system is largely differ-
ent than the one envisaged initially.

Audit scope and approach

Observations

09 
The Council took its decision to build SIS II on the 
basis of a feasibility study conducted in 1998 and 
a series of consultations among Member States 
culminating with the Swedish/Belgian initiative 
that was the basis for the first SIS II legal basis. 
The Council’s decision fixed the launch date of 
end 2006. The Commission had no formal role in 
the legislative process.

Apart from reporting in each AAR since 2001 on 
the advancement and the main difficulties of the 
project, the Commission drew the attention of the 
Member States, in communications in 2001 and 
in 2003, to the risks entailed with that deadline 
and the assumptions that would need to be met 
(and eventually were not) if the deadline was to be 
maintained.

Both the 2001 communication (COM(2001) 720) 
and then, in full detail, the subsequent feasibility 
study carried out by an external contractor, care-
fully assessed the business needs to be served by 
the new system, the technical choices and their 
impact, the timeline of the project and its budget, 
as well as the risks and constraints associated with 
its development.

05
The reasons for delays and spending more than ini-
tially planned as well as the difficulties encountered 
with the project have been reported by the Com-
mission including in each Annual Activity Report of 
DG Justice, Freedom and Security and subsequently 
Home Affairs since 2001.

On this basis, the Council opted for one of the 
proposed implementation options (but not the one 
recommended by the feasibility study) in order to 
go ahead with the project in 2003.

The SIS II legal framework was not finalised until 
late 2006.

10
The initial deadline set by the Council was indeed 
not realistic given that it was established primarily 
on the basis of political considerations, rather than 
on a stable set of system requirements and a sound 
technical analysis of the workload.

Reply of the Commission

47

11
Each SIS II schedule was based on the best available 
information at the time and was previously dis-
cussed with the Member States.

Following the adoption of the legal basis, the only 
parameter of the specifications that was subject to 
evolutions was the system’s sizing, which is linked 
to the Member States’ use of the system. This risk, 
which has been assessed throughout the project in 
the light of SIS1+ usage by the new Member States, 
indeed materialised due to a more intensive use 
than initially expected. However, even if the SIS II 
had been delivered according to the initial sched-
ule adopted by the Council in 2001, the system 
would have required an upgrade of its sizing of the 
same magnitude as the one implemented in the 
framework of the project’s extension carried out 
in 2010 (see figure 7) (final delta).

14
As of 16 December 2002, a new unit ‘Large-scale 
information system’ was created within DG Justice, 
Freedom and Security to exploit synergies between 
the major IT projects in the Justice and Home 
Affairs policy areas: SIS II, Eurodac and VIS (Visa 
Information System).

15
The Commission agrees that insufficient staffing 
was provided to the project in the early years and 
that SIS II project staff were committed and com-
petent, though constantly overloaded. Successive 
annual activity reports from DG Justice, Freedom 
and Security pointed to the general lack of staff 
faced in general by the DG and in particular on SISII.

However, the Commission does not consider that 
the turnover level differed from that in comparable 
Commission services. In fact, key actors such as 
the Programme Officer remained in place from the 
beginning to the end.

16
As regards the expertise in the team, the Commis-
sion would recall that the SIS II team’s expertise was 
mainly composed of staff either originating from 
Member States’ former SIS1 projects or persons 
who had worked for IT companies prior to joining 
the Commission. The same expertise was available 
in the VIS project. Directorates-general Informatics, 
Information Society, and Enterprise and Industry 
also provided DG Home Affairs with some sup-
port, notably during the first years of the project. 
In addition a dedicated contract with Unisys had 
been signed since a very early stage of the project 
(April 2002) to compensate for the lack of internal 
resources in the field of quality management.

The Commission maintains that the contract was 
managed effectively. As an example, from the 
beginning, the Commission used the weekly con-
tractor’s meeting to voice its disappointment with 
the quality of deliverables and require corrective 
actions. Already in October 2005, the Commission 
did require (and obtained) a change of the con-
tractor’s project manager. However, as noted in 
the evidence to the House of Lords referred to in 
footnote 22 the staffing level meant that it could 
not anticipate and prevent the underperformance 
of the contractor. As noted in footnote 21, the IAS 
found there were insufficient staff to manage the 
increasing level of outsourcing

17
The contract suspension of a few months was the 
consequence of a tenderer making use of his right 
to challenge the award decision in the Court of 
Justice. The evaluation of the tenders was done 
according to the published criteria and method, 
which were known and accepted by all tenderers 
choosing to apply.

Reply of the Commission

48

The conclusions of the President of the Court of 
Justice were done in an injunction procedure in 
which the Court of Justice, by definition, does not 
take a definite position until after a full adversarial 
procedure, but has to take a preliminary position in 
order to issue an injunction and must do so without 
a full hearing on the substance. Conclusions are 
thus only of relevance to the injunction procedure 
but do not allow one to deduce a final position of 
the Court of Justice; and in no way prejudge a final 
ruling. The evaluation method was thus never 
tested in the Court of Justice as the applicant with-
drew his application.

The evaluation of the tenders was done according 
to the published criteria and method.

There is no evidence that the computation of 
a weighted average (or not) for the scores of the 
fixed-price items would have given a different 
result. From a methodological point of view, there is 
no perfect solution for taking decisions on the basis 
of multi-criteria choices.

The Commission would like to firmly state the fact 
that it received two good offers which were very 
close; a decision was taken respecting the rules. 
Other rules might have led to another decision, but 
the Commission was bound to the chosen rules.

19
The operational system tests in 2008 (which failed) 
was only one phase of SIS II system testing. The 
structure of the SIS II system tests was conceived to 
allow for the progressive increase of their level of 
complexity. With such a testing approach it is inher-
ently possible to identify at a given phase issues 
that had not been discovered before (because of 
the mere nature of the tests performed so far) . 
The whole purpose is to make use of the different 
phases to identify (ideally all) potential problems 
before the system goes into production. This was 
the case in 2008, as well as during the final phase 
of the project after 2009 (as the testing approach 
remained unchanged throughout the project).

The additional posts received as from 2007 enabled 
the project team to recruit more staff with relevant 
experience.

22
The Commission notes that the SISVIS Commit-
tee also dealt with the VIS which reduced the 
time available to discuss SIS II. This was frustrating 
for SIS II experts. In fact, most of the issues that 
should normally have been discussed in the SISVIS 
committee were in fact discussed in the Council’s 
SIS-TECH working party which had responsibility 
for SIS 1 issues. Many technical issues linked to SIS II 
could possibly impact on SIS 1, which is why many 
Member States insisted that they would also be 
discussed in SIS-TECH.

Reply of the Commission

49

23
The environment in which the SIS II project was 
developed was less easy than the environment 
which DG Taxation and Customs Union experi-
enced. This was for a number of reasons, such as 
the fact that SIS II was based on a legacy system, 
had a complex governance structure and because 
of policy considerations related to the overall 
approach of some member countries towards the 
enlargement of the Schengen zone and ever-chang-
ing system requirements.

Box 2 
The prevailing contexts for the development of 
NCTS and SIS II substantially differed in terms of 
political and institutional challenges. Furthermore, 
the development of NCTS took place in an environ-
ment based on a long tradition of cooperation with 
Member States (which started in the early 1990s). 
Trust among all partners involved was supported by 
long-established networks.

24 (a)
The Commission accepts that it did not allocate 
sufficient staff to the project in the early stage. This 
was disclosed very clearly in the Annual Activity 
Report of the Directorate-General.

The Commission confirms that it was not an end-
user of the system, nor was it an end-user of VIS 
and Eurodac. Accordingly it had to rely on Member 
States’ input to be provided by Member States’ 
delegates in the National Project Managers’ meet-
ings and the ad hoc working groups as well as by 
states experts who were national experts seconded 
by Member States to the Commission.

24 (c)
The Commission encouraged and organised direct 
contact between the main development contractor 
and Member States but, as is recommended by IT 
project methodology and best practice, such con-
tact/interaction had to be placed in a framework. 
The contractors were present in the Committee and 
the National Project Managers’ meetings, which 
were held at least once a month. There was also 
a formal cycle of revision for documents, allowing 
national experts to assess the technical specifica-
tions that were then formally submitted to the 
SISVIS Committee for endorsement.

As regards the disclosure of the contract, one mem-
ber of the HP/S consortium opposed the disclosure 
of the contract.

24 (d)
The Council decided that SIS II was necessary 
in 2001 and maintained this position in successive 
Council conclusions, describing the implementation 
of SIS II consistently as ‘absolute priority’.

25 (a)
Following the suspension of the Operational Sys-
tems Test in 2008 because of a series of issues with 
the central system, the Commission put in place 
a global SIS II programme management approach 
from January 2009. This was done in order to ensure 
the necessary consistency between the devel-
opment of the central system and the national 
systems. (COM(2009)133). It included an informal 
body consisting of a limited number (eight) of 
Member States experts designated as the ‘Global 
Programme Management Board’ which was estab-
lished to enhance cooperation and to provide direct 
Member States support to the central SIS II project. 
The global approach was welcomed by the Council 
conclusions on SIS II of 26–27 February 2009.

Reply of the Commission

50

25 (b)
The increased staffing which the Commission 
allocated to the project after 2007 enabled it to 
improve the management of the project and its 
communication.

26
The Commission fully supports the Centre for 
European Policy Studies (CEPS) findings concern-
ing the complex governance structure for the SIS II. 
Indeed CEPS states that ‘…..Even after endowing the 
European Commission with the competence to man-
age the SIS II project, and following the expansion of 
the co-decision procedure which further strengthened 
the legislative roles of the Commission and European 
Parliament, (certain) Member States were not ready to 
relinquish control of a tool so central to security and 
migration management. Strategies to retain owner-
ship of the project emerged, including the prolifera-
tion of expert groups, and the SIS1+RE proposal for an 
“intergovernmental” alternative to SIS II.’

28
The Commission can only regret that a large share 
of respondents to the survey were ignorant of 
the decision-making structure which the Member 
States had agreed.

29
The Commission considers that the fact that system 
requirements were constantly evolving (mostly 
following Member States requests) was the primary 
cause of the delays and the increase in costs. It 
also had a major impact on contract management 
aspects of the project, since this implied several 
revisions of the contract, which in turn did not 
facilitate the relationship with the contractor who 
was constantly faced with changing requirements. 
It also increased the workload for the Commission 
in managing the contracts.

30
The Commission agrees that subsequent changes to 
the system requirements necessitated amendments 
to the development contract, which contributed 
to delays. The time taken to agree on the system 
requirements and to adopt the corresponding legal 
instruments was outside its control.

32
The resizing of the system’s capacity was not due to 
the delays but, on the contrary, it actually contrib-
uted to them. It arose mainly because of the higher 
number of transactions submitted by Member 
States than initially forecast by them (thereby indi-
rectly demonstrating the added value of the SIS). 
This seven-fold capacity increase (i.e. the current 
system’s capacity can be expanded to 100 mil-
lion records) necessarily also had an impact on the 
deadline and the costs. Again, it demonstrates that 
the instability of the system requirements was the 
primary cause of the delays and increased costs.

34
The Commission considers that the cost evolu-
tion should be placed in context. Each estimate 
was based on a changed set of requirements: The 
requirements and size of SIS II significantly changed 
in the course of the project, mostly following Mem-
ber States/end users’ requests :
•  SIS II was originally assumed to go live with a max-
imum of 15 million records and was extended to 
have a capacity of 70 (with a further possibility of 
extension to 100 million records, as needed);
•  The  test  approach  was  revisited  and  included 
a compliance test for each Member State system;

Reply of the Commission

51

•  The additional ‘Milestone 1’ and ‘Milestone 2’ tests 
imposed by the Council had to be added to the 
existing test plan, leading to a longer duration and 
additional tasks resulting in more expenses;

•  The new migration approach after 2010 requested 
by the Council included a fallback solution that 
required a converter able to, not only to convert 
SIS 1 data to SIS II, but also do so in the reverse 
direction — again leading to additional costs.

36
The Commission always sought to present a clear 
picture of the costs on the basis of the information 
available. This was notably done in the context of 
the Annual Budgetary Procedure as well as in each 
SIS II progress report. The Court gives the example 
of the network: moving, at the request of Member 
States, to a dedicated network infrastructure based 
on a virtual private network (VPN) on s-TESTA rather 
than using the already existing infrastructure on 
s-TESTA (euro domain) as initially envisaged indeed 
had a very significant impact on the budget. Using 
the euro domain as initially foreseen by the Com-
mission and as it is done for Eurodac would have 
cost much less. It was only in 2005, once the final 
design of the network was known that the Com-
mission was in a position to revaluate the network 
costs, which it duly did in COM(2005)236. Neverthe-
less, the network costs were disclosed every year 
in the draft budget proposed to the budgetary 
authority.

40
The amount of liquidated damages imposed on 
the contractor was calculated following a thorough 
evaluation of the contractual situation by both the 
legal service of the Commission and an external 
law firm: At that point in time, this amount was the 
maximum legally possible.

41
Before 2009, the programme and project manage-
ment structure from the main development con-
tractor had, on several occasions, failed to meet 
contractual commitments. The Commission did not 
hesitate to request several changes to the main con-
tractor’s project management composition.

42
The Commission agrees that changes to the system 
requirements resulted in increased costs. It also 
resulted in delays.

43
The Court’s example illustrates the impact of the 
instability of the system requirements.

44
The bulk of the cost increases entailed by these 
(amendments 12, 13, 15) corresponded to the major 
changes in the project’s requirements, namely: the 
size and capacity of the SIS II, performance, migra-
tion approach, testing approach and supporting 
services to Member States during testing, or the 
support to be given for the operational manage-
ment of the system. The Commission considers that 
the work package approach proposed can only 
work when all requirements are known from the 
beginning. Because of the instability of the system 
requirements, this was not the case here. A work 
package approach would not have prevented an 
increase of the costs linked to a different network 
design or to new capacity requirements.

Reply of the Commission

52

Common reply to paragraphs 45 
and 46
Already at the time of the first contract amend-
ment the Commission had identified the technical 
reasons (i.e. a technical lock-in situation with the 
winner of the initial contract) as a ground for the 
amendment of the contract by means of a negoti-
ated procedure. However, for the first amendment, 
the two options provided for in the Financial Regu-
lation, (the ‘additional services max 50% option’ and 
the ‘technical lock-in option’) were equally justified 
and relevant. In that context, the Commission chose 
to apply the first option for the first amendments. It 
is only when the second option remained the only 
one possible that the Commission opted for it.

47
The Commission considers that while a ‘building 
blocks’ approach is preferable where it can be 
applied and notably where the system require-
ments are stable, this was not the case for SIS II. In 
addition, SIS II is a highly complex, bespoke system 
that includes the migration of a legacy system. 
A ‘building block’ approach would have led to 
significant problems for handovers and ultimately 
a liability problem when a contractor for one build-
ing block could have avoided liability by blaming 
technical problems on the presumed shortcomings 
of other building blocks. The choice was therefore 
deliberately made (and in full transparency since 
the call for tender was also reviewed by Member 
State experts) to not use a building block approach. 
In addition, one must note that at the time the pro-
ject started the RUP methodology was not in place 
at the Commission.

48
The contractor underperformed for certain phases 
of the contract, for which it paid penalties accord-
ing to the contract.

49
The Commission considers that the negotiations of 
amendment 15 were conducted in line with best 
practice for negotiated procedures.

Whether prices initially proposed by the contrac-
tor were twice as high or not, is not relevant. What 
is relevant is the final price level obtained after 
negotiations, and the legal guarantee of best cus-
tomer prices in case any divergences would have 
appeared during implementation/delivery. The fact 
that the Commission verified the prices of a sample 
of items against those used by DG Informatics in 
their framework contracts is a further guarantee 
that the best possible price was obtained in the 
procedure.

The Commission reiterates that it believes that the 
solution used was indeed financially sound.

53
As the Court notes, given that SIS II was an IT pro-
ject developed for the Member States and not for 
the Commission, it did not fall within the remit of 
the Commission IT governance scheme put in place 
after 2004. Notwithstanding this, DG Informatics 
participated systematically in the monthly meetings 
of the Project Management Board (PMB) until the 
end of 2008 with only one interruption between 
May and October 2008. Furthermore, information 
was provided annually to DG Informatics about 
the development of SIS II in the framework of the 
‘Schema Directeur’ exercise.

54 
After being made responsible for the project, 
the Commission had to start the development 
of SIS II on the basis of a business case prepared 
by the Council, which did not include a thorough 
assessment of all costs, expected benefits and 
alternatives.

Reply of the Commission

53

54 (a)
The SIS II legal instruments state that the Member 
States are responsible for the development of the 
national systems. Two groups were created by 
the Council in the course of the project in order 
to monitor and report on national developments 
(which also include costs): the SIS II Task Force and 
Friends of SIS II. The Commission was a member of 
these two groups and consistently sought to obtain 
information about national projects, but without 
success.

The 2003 communication spelled out the basis 
for the distribution of costs for developing the 
SIS II were distributed between the Union and the 
Member States. It also spelled out the budget for 
developing the central systems but did not estimate 
the costs of developing national systems since such 
developments were outside its mandate. The Com-
mission had no control over, or information about 
these costs (except where they were co-financed 
from the External Borders Fund).

54 (b)
The Commission does not agree that it should have 
presented new analysis of the benefits of the pro-
ject once the Council had taken its decision. At the 
time, the Council, had exclusive competence in the 
adoption of the initial legal basis for SIS II.

54 (c)
The Council itself had ruled out the possibility of 
an extension of SIS 1 in its decision of 6 Decem-
ber 2001 on the development of the second  
generation Schengen Information System (SIS II)  
(2001/886/JHA). The Commission had no formal role 
in this legislative process.

56
As regards the funding of national developments, 
the Commission made significant efforts in 2011 to 
mitigate the risk of delay due to lack of financial 
resources for Member States by inviting national 
authorities to give higher priority to SIS II under 
their existing External Borders Fund (EBF) national 
allocations and via the organisation of a call for pro-
posals for emergency actions under the Community 
actions of the EBF under which an extra €7.5 million 
is being provided to the eight Member States most 
in need of additional resources (countries with very 
low EBF allocations and/or whose EBF resources 
were absorbed by other key priorities in the area of 
external borders). These projects started at the end 
of 2011.

57
Member States reported that SIS II has brought 
significant benefits in terms of performance and 
capacity.

The Commission confirms that the risk of reputa-
tional damage in case of a failure of the develop-
ment became more important towards the end 
of the project, as indicated by the reservations 
issued in DG Home Affairs Annual Activity Reports 
from 2008 to 2012. This reflects sound financial 
management considerations (not wasting invest-
ments already made).

58
Like the large majority of respondents, the Com-
mission also considers that SIS II had significant 
additional functions which provided end-users with 
immediate visible practical advantages over SIS 1.

Reply of the Commission

54

59
The Commission deems that, compared to the 
previous system, SIS II has brought significant func-
tional and non-functional benefits to the users.

61
The Council reaffirmed in February 2009 that the 
rapid entry into force of SIS II remained an absolute 
priority. (cf. footnote 63)

For instance, due to a much richer content (such 
as photographs, copies of European Arrest War-
rants …), the size of a SIS II alert is significantly 
higher on average than that under SIS1. Such alerts 
are however broadcast without any loss of perfor-
mance. SIS 1 was not built to handle such alerts.

Finally, SIS II provides the possibility to handle 
a high volume of direct queries on the central 
database (240 queries per second) while SIS 1 had 
no such functionality. There are five Member States 
that exclusively rely on this functionality for all of 
their SIS II transactions.

60
The magnitude of the issues faced by the SIS II pro-
ject became clear only at the stage of the failed OST 
test (end 2008), when the central system started 
interacting with a subset of national systems. The 
Commission made a reservation on SIS II in the 
DG Home Affairs 2008 Annual Activity Report and 
announced an action plan.

In an approach jointly agreed by all stakeholders, 
the Commission, together with the Member States 
and assisted by the GPMB, revisited the sizing speci-
fications, implemented technical changes aimed 
at improving the robustness and reliability of the 
system as well as implemented changes in the way 
that the SIS II would be tested and the migration 
itself conducted.

At the end of this process which indeed required 
time but was done as quickly as possible, the Com-
mission adopted a new, realistic schedule endorsed 
by all stakeholders and finalised the development 
of the SIS II project according to this schedule.

In June 2009, on the basis of a detailed Report of 
the Presidency and the Commission on the future 
direction of SIS II containing an analysis of the 
impact of two scenarios on, among other things, 
expenditure for the EU budget, technical feasibility 
and risks, Council decided to continue development 
of SIS II.

62
As the Commission explained in SEC(2010) 436 final, 
developing SIS II on the basis of SIS 1 + RE solu-
tion was likely to have a significant impact on the 
national projects. These costs could only be esti-
mated by Member States for each individual project 
and were therefore not part of the comparison 
report.

63
The Commission would like to point out that the 
inclusion of two milestone tests, as requested by 
the Council in its June 2009 Conclusions, increased 
both the duration and the costs of the project.

66
The Commission has drawn the lessons from 
the SIS II project in its 2010 Communication 
(COM(2010)385 page 27) and in the proposal to 
establish an agency dedicated to the management 
and development of large-scale IT systems in the 
policy area (COM(2010)93).

Reply of the Commission

55

67
The Commission would stress that when SIS II pro-
ject was launched the PM2 or similar methodology 
did not exist.

68 (a)
The Treaty revision means that any new legisla-
tion on the development of large-scale IT projects 
would be decided between the Council and the 
Parliament on the basis of a Commission proposal. 
This was not the case in 2001.

69 (a)
The Commission’s 2011 proposals for the multi-
annual framework 2014–20 (COM(2011) 749, p. 8) 
provides for the financing of the development and 
operation of the central and national systems.

Conclusions and recommendations

70 (a)
The Commission drew the attention of the 
Member States, in both communications 
from 2001 and 2003, to the risks entailed with that 
deadline and the assumptions that would need to 
be met (and eventually were not) if the deadline 
was to be maintained.

70 (b)
The Commission systematically reassessed cost 
estimates at key milestones of the project when 
justified by new elements and it communicated 
transparently on budgetary matters:

- 

- 

in 2005 when network costs were re-evaluated 
(COM(2005)236);
in May 2009 in the Commission/Presidency joint 
Report on the further direction of SIS II (Council 
document 10005/09);

- 

- 

in April 2010 in the Report on the global sched-
ule and budget for the entry into operation of the 
second generation Schengen Information System 
(SIS II) (SEC(2010) 436);
in September 2010 in the Report on the global 
schedule and budget for the entry into operation 
of the second generation Schengen Information 
System (SIS II) (SEC(2010)1138).

Furthermore, each progress report since 2003 con-
tained a section on budget execution.

70 (c)
The fact that the system requirements were con-
stantly evolving (mostly following MS requests) was 
a cause of the delays and the increase in costs. It 
also had a huge impact on the contract manage-
ment aspect of the project, since this implied sev-
eral revisions of the contract, which in turn did not 
facilitate the relationship with the contractor who 
was constantly faced with changing requirements.

70 (d)
The lack of staff was generalised to the whole policy 
area which was growing very rapidly.

70 (f)
The same DG Home Affairs unit that worked on 
the SIS II project, on the basis of the same contract 
and with the same contractors, delivered another 
large-scale IT system (VIS) in a constructive and 
harmonious working atmosphere with Member 
States, and with no ambiguities/challenges towards 
the decision-making process. The same can be said 
of the development of EURODAC.

There were underlying reasons for the poor rela-
tionship with the SIS Community (e.g. a legacy 
system, a complex governance structure, unrealistic 
political steering of the project, etc.).

Reply of the Commission

56

70 (g)
The Commission can only regret the overall poor 
recognition by many stakeholders of the decision-
making arrangement surrounding the SIS II project.

The complex governance structure generated dif-
ficulties for the management of the project.

70 (h)
Throughout the SIS II project, the Commission fully 
followed the Financial Regulation rules and negoti-
ated in the best interest of the Union budget.

71
The Commission recalls that the Member States 
built up the business case and carried out the 
necessary preparatory assessments within an 
intergovernmental framework The Council took its 
decision to build SIS II on the basis of a feasibility 
study conducted in 1998 and a series of consulta-
tions among Member States culminating with the 
Swedish/Belgian initiative that was the basis for the 
first SIS II legal basis.

The Commission had no competence to legally chal-
lenge or review such legislation emanating from 
the Council. The sole Commission obligation was 
to implement the Council’s decision. It should also 
be recalled that the Council reaffirmed on multiple 
occasions that the entry into operation of SIS II was 
an absolute priority.

Once the Commission was entrusted with the 
development of SIS II, it launched a feasibility study 
to assess the business needs to be served by the 
new system, the technical choices and their impact, 
the timeline of the project and its budget, as well 
as the risks and constraints associated with the 
development.

The Commission is not able to confirm the €500 mil-
lion figure quoted by the Court. National costs 
being outside the development’s perimeter funded 
centrally by the EU budget, there was no reason to 
include them in the financial statements attached 
to the legislative proposal. There was no require-
ment on the Commission to estimate or track 
national expenditures for SIS II.

The Commission systematically reassessed cost 
estimates at key milestones of the project when 
justified by new elements and always transparently 
communicated on budgetary matters, as spelled 
out under paragraph (70)(b).

72
The Commission confirms that it indeed has drawn 
several key lessons from the development of large-
scale IT systems, and has already implemented 
them, notably in the design of the Smart Borders 
package. However, this project must be regarded 
as ‘sui generis’ for the reasons set out in earlier 
paragraphs.

Recommendation 1
The Commission can accept this recommendation 
which reflects its IT governance arrangements in 
place since 2010. More precisely, during the analysis 
of IT projects by the Information Systems Project 
Management Board (ISPMB, created in end 2010) 
such considerations are carefully examined. The 
directorate-general in charge of IT (DG Informatics) 
is also consulted at the level of the impact assess-
ment phase when a new Commission initiative 
contains an IT component.

Reply of the Commission

57

Recommendation 2
The Commission can accept this recommendation 
which reflects its IT governance arrangements in 
place since 2010. The IT rationalisation process, 
launched in the Commission in 2010, already 
includes all IT projects managed by the Commis-
sion, irrespective of the source of their funding. The 
Communication SEC(2011) 1500 recalls that ‘in addi-
tion to the Infrastructure Consolidation project (ITIC), 
which will be rolled out to all services, all information 
systems of the Commission, including offices and 
executive agencies, financed under both operational 
and administrative budgets are subject to the 
rationalisation process’.

As far as large-scale IT projects belonging to the 
Commission are concerned, the governance mecha-
nisms put in place since 2010 ensure their close 
scrutiny not only by DG Informatics but also by the 
‘Information Systems Projects Management Board’ 
(ISPMB), of which not only the other horizontal DGs, 
but also five DGs representing different types of 
policies, are members. More precisely, the ‘tran-
seuropean systems’ domain (see also answer to 
recommendation 6) is supervised by the High level 
Committee on IT.

Recommendation 3
The Commission can accept this recommendation 
which reflects its IT governance arrangements in 
place since 2010. Before the start of a project, direc-
torates-general are obliged, under the IT govern-
ance rules put in place in 2010, to submit a business 
case or a vision document to the Information Sys-
tems Project Management Board (ISPMB) that spells 
out, amongst other things, how the governance 
structure of the project is set up. The Board pays 
particular attention to the proper representation of 
the users in the project governance. Moreover, all 
projects need to report yearly to the Board, thereby 
ensuring a close follow-up of the work carried out 
and of possible deviations from the initial scope, 
timetable, and budgets.

Recommendation 4
The Commission can accept this recommenda-
tion which reflects its IT governance arrange-
ments in place since 2010. The Commission fol-
lows the PM2 project management methodology 
and project artefacts already provide entries for 
alternatives, cost estimations, resources, etc. All 
projects with a total cost of ownership above 500K 
(over 5 years) needs to submit a business case/
vision document to the ISPMB and to report on an 
annual basis.

Recommendation 5
The Commission can accept this recommendation 
which reflects its IT governance arrangements in 
place since 2010. Indeed, the PM2 project artefacts 
already include four types of logs that can be used 
by project managers: risk log, issue log, decision log 
and change log. Moreover, the documents of each 
project (business case, vision document, progress 
reports …) are accessible through a central system, 
called GOVIS.

Recommendation 6
The Commission can accept this recommendation 
which reflects its IT governance arrangements in 
place since 2010. This is reflected in the ‘domain’ 
approach that has been followed in the IT ration-
alisation exercise since 2010. Systems have been 
grouped according to business domains (e.g. ‘grant 
management’, ‘financial management’, ‘procure-
ment’, etc.). There are currently 19 domains, each 
led by a domain leader, who is responsible for 
the convergence and the rationalisation in his/
her domain. New systems need to have the green 
light of the domain leader before being developed, 
thereby ensuring their compliance with the domain 
strategy and their coherence with the existing 
systems.

Reply of the Commission

58

Recommendation 7
The Commission can accept this recommendation 
which reflects its IT governance arrangements 
in place since 2010. The Commission services are 
now working on the definition of a Corporate 
Enterprise Architecture, based on a coherent set 
of reusable building blocks. Several such build-
ing blocks have already been identified and, once 
available, their use is compulsory for new systems 
or systems undergoing major revisions. Missing 
but necessary building blocks are currently being 
identified. DG Informatics has recently set up an 
Architecture Office. The ISA programme and, more 
recently, the Connecting Europe Facility (CEF) 
initiative are also recommending an increased use 
of cross-cutting common services to cover generic 
needs (e.g. secure transmissions, e-signatures, 
semantics tools)

Recommendation 8
The Commission can accept this recommenda-
tion. The lessons learnt from the Court’s audit will 
be shared with the other directorates-general at 
both technical and management levels, as well as 
disseminated to the relevant agencies through the 
appropriate networks.

The impact of SIS II will be assessed in 2016 (3 years 
after SIS II entered operation), as required by the 
legal base (Article 50 paragraphs 4 and 5 of Regula-
tion (EC) No 1987/2006 and Article 66  
paragraphs 4 and 5 of Decision 2007/533/JHA).

HOW TO OBTAIN EU PUBLICATIONS

Free publications:

• 

• 

 one copy: 
via EU Bookshop (http://bookshop.europa.eu);

 more than one copy or posters/maps: 
from the European Union’s representations (http://ec.europa.eu/represent_en.htm);  
from the delegations in non-EU countries (http://eeas.europa.eu/delegations/index_en.htm);  
by contacting the Europe Direct service (http://europa.eu/europedirect/index_en.htm) or 
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) (*).

(*) 

The information given is free, as are most calls (though some operators, phone boxes or hotels may charge you).

Priced publications:

• 

via EU Bookshop (http://bookshop.europa.eu).

Priced subscriptions:

• 

 via one of the sales agents of the Publications Office of the European Union 
(http://publications.europa.eu/others/agents/index_en.htm).nts/index_en.htm).

Q
J
-
A
B
-
1
4
-
0
0
3
-
E
N

-
C

I

S
S
N
1
8
3
1
-
0
8
3
4

The Schengen Information System contains 
information on people and property which is 
used by border guards and police throughout 
the Schengen Area. In this report the Court of 
Auditors examines the Commission’s 
management of the project to develop a new, 
second generation Schengen Information 
System (SIS II). It explains why the system was 
delivered 6 years later than planned and at  
8 times the initial budget estimate.

EUROPEAN
COURT
OF AUDITORS