ANNEX

‘SIS II TESTS

1. OBJECTIVES

The tests listed hereafter serve to demonstrate that the national system (N.SIS II), the communication infrastructure and the interactions between Central SIS II (C.SIS II) and N.SIS II work in accordance with the technical and functional requirements set out in Regulation (EC) No 1987/2006 and Decision 2007/533/JHA.

Moreover, those SIS II tests are also to demonstrate that N.SIS II, the communication infrastructure and the interactions between C.SIS II and the N.SIS II can work in accordance with non-functional requirements such as robustness, availability and performance set out in Regulation (EC) No 1987/2006 and Decision 2007/533/JHA.

2. PROCESS, DETAILED SCOPE AND ORGANISATION OF SIS II TESTS

The sequence of tests, their objective, scope (depending on the national implementation) and organisation is as follows.

2.1. Connectivity tests are the first phase of tests which address the testing of the connectivity and resilience of the SIS II communication infrastructure.

2.2. National compliance tests are the second phase of tests which address the testing of the compliance of N.SIS II to the specifications described in the reference version of the Interface Control Document (ICD).

2.3. Local National Interfaces (LNI)/Back-up Local National Interface (BLNI) tests are the third phase of the tests which address the connectivity and the resilience of the SIS II communication infrastructure with the aim of testing the correct functioning and resilience of the N.SIS II behind both the LNI and the BLNI, if a Member State has a BLNI.

2.4. Global tests are the fourth phase of the tests which address proper functioning of the N.SIS II against the valid ICD specification under similar conditions and with the involvement of other countries connected to SIS II, as required during the daily operations. They are split into two distinct phases:

2.4.1. Member State Tests

During this test phase all established Member States which are connected to SIS II already have to demonstrate their capabilities to consume messages from the acceding Member State. The acceding Member State's traffic may be generated by using simulators.

2.4.2. Comprehensive Tests

During this phase the system under test (SUT) will be exposed to traffic(nominal, stress and peak traffic) as can be expected during normal operations. Member States (except the SUT) are replaced by simulators. The aim of this phase is to ensure that the SUT can consume and process all incoming messages and perform normal operations, including resilience testing.

2.5. ITSM tests are the fifth phase of the tests which address the organisation of IT Service Management, including the operation and communication procedures via communication systems such as the SIS II SPoC mail, eOPM, SM7.

2.6. SIRENE connectivity tests are the sixth phase of the tests which address connectivity and resilience of the SIRENE Mail infrastructure and check the basic functioning of the mailboxes through simulating basic traffic.

2.7. SIRENE functional tests are the seventh phase of the tests which will address the functioning of the national SIRENE technical solution and exchange of information between SIRENE Bureaux through forms sent via the SIRENE Mail infrastructure according to the specifications provided for in the SIRENE Manual set out by Commission Implementing Decision (EU) 2015/219 (1), entering, modifying, flagging, deleting corresponding alerts in SIS II and attaching/detaching relevant additional information to SIS II alerts.

Other types of tests may be foreseen depending on the legal framework pertaining to the particular Member State intending to integrate to SIS II.

2.8. Coordination of the tests

The European Agency for the operational management of large-scale IT-systems in the area of freedom security and justice (eu-LISA) as the Management Authority coordinates all tests, eu-LISA draws up the test specifications, the test schedule and the final outcome of the tests. Moreover, it will identify, categorise and describe any issues it detects and propose options for solutions.

Member States will assist eu-LISA in the overall performance of all tasks related to the test executions.

2.9. Test documentation

eu-LISA will define the detailed test specifications. It will make available to the Member State involved the draft and final version of test specifications.

2.10. Running the tests

eu-LISA will execute the tests together with the Member State and other relevant stakeholders involved on the basis of the test specifications and in accordance with the schedule agreed together with Member State's experts, and demonstrate that the expected test results as foreseen in the test specifications are met. eu-LISA will provide the C.SIS II test environment for testing purposes, including the SIRENE functional tests, where SIRENE functional tests and corresponding alert modifications can be executed.

2.11. Acceptance of the tests

The verdict on a Member State test can be “passed”, “passed with remark”, “inconclusive”“failed” or “Not tested”/“Not applicable”.

(a) “Passed”: (i) Actual results are as expected in the “Expected result” description; (ii) All test conditions and the Test Plan were respected; (iii) No inconclusive criteria apply.

(b) “Passed with remark” The conditions listed in point (a) are fulfilled but specific conditions and/or well substantiated reasons caused a unexpected result or event during the test therefore the test passed with remark.

(c) “Inconclusive”: unexpected events independent of the SUT occurred during the test.

(d) “Failed”: one of the criteria for passing the test is not met.

(e) “Not tested”/“Not applicable”

eu-LISA will report on the results of the SIS II tests. It will identify, categorise and describe any issue it detects and propose options for solutions. The Member State's experts will provide all necessary information for the Test Coordination Group to perform its tasks.

Where the test documentation divides the tests into separate phases eu-LISA will inform the Member State of the results of each phase before the start of the following phase.

The acceptance of the tests will be based on reports containing a detailed analysis of the test results and conclusions as to the validation of the Member State's national system (N.SIS II or SIRENE application). If the Member State undergoing testing or eu-LISA considers that the tests could not be successfully completed, this is to be noted in the report. eu-LISA will provide an opinion on the successful completion of the SIS II tests, taking into account the views expressed by the Member State's experts and eu-LISA will submit the test result with its opinion to the relevant formations of the SISVIS Committee for the final approval.

3. INTERFACE CONTROL DOCUMENT (ICD) AND DETAILED TECHNICAL SPECIFICATION (DTS) FOR TESTING

The N.SIS II in each of the countries getting integrated to SIS II will be tested against the latest specifications.

The Detailed Technical Specifications (DTS) prepared by eu-LISA define the functional and non-functional specifications of the C.SIS II.

The ICD prepared by the eu-LISA will define the interface between the C.SIS II and N.SIS II. It contains the technical specifications of the system-to-system interactions in terms of data items and messages passed, protocols used as well as timing and sequencing of events.

Specifications, as provided in the ICD and DTS, will be fixed for a given period and the timing of the update of both systems will be laid down in a release plan that will define the reference version for a given test phase. Issues found during the test campaigns will be reported, analysed and solved in accordance with established operational procedures, taking into account the opinion of the experts of the Member State which is undergoing testing.

4. INTERIM AND FINAL REPORT ON THE RESULTS OF THE TEST PHASES

eu-LISA will draw up, on a regular basis, reports on the status of the tests. The reports will note which test phase is currently being dealt with and whether the Member State has completed, begun or not yet begun any of the phases. If any repercussions for the test timetable are noticeable, they and their cause should be recorded.

On the conclusion of each test phase eu-LISA will draw up a report on the results, any issues it detects and options for solutions. In cases where the Member State undergoing testing or eu-LISA considers that tests could not be successfully completed they will record this fact, stating the reasons, in a separate note.’