# Measures and Methods for the Evaluation of ATO Algorithms
## Literature Review
### Current State-of-the-Art ATO Test Methods  
One of the few resources of scientific research on ATO testing is based on the innovation initiative of _smartrail 4.0_ and in connection to the European program _Shift2 Rail_. The SBB and other Swiss railway companies and operators were testing ATO over ETCS on two different track sections. All specifications and results are explained in the reports [7; 8] although a closer insight into the technology used is not possible.  
Multiple test cases were executed testing the general functionality and in greater context the following characteristics:  
* driving behaviour,
* punctuality,
* stopping position,
* energy saving,
* comfort for passengers,
* advantages for rail operation and train drivers,
* compliance with the TSI norm drafts with Swiss railway operation.  
Figure 1: Visualization of the literature review process including examples for search terms and identified related terms.  
Already in 2019, ATO over ETCS was implemented in the core of London's rail network. The Thameslink program was aiming to enable a high-capacity passenger railway route by introducing ETCS Level 2 and ATO over ETCS functionalities. To introduce the system under real operational conditions several stages of testing were used. At first, a simulator was developed to showcase the technology and increase efficiency in the early development phase of the system. Furthermore, a system integration laboratory enabling a simulation of the Thameslink route was arranged. The next step was to conduct field tests at the ETCS National Integration Facility and finally operational tests on the Thameslink route itself [9].  
In [10], the authors are proposing an operational design domain (ODD) for a Chinese highspeed main line ATO system. The paper focuses on model validation and is including a case study of the linkage scenario between train doors and platform doors to extract system requirements. In this context, the ODD is used for mapping components relevant to the case study. The scenario model construction is based on an interaction process. Yet, it is not demonstrated how the ODD contributes to the extraction of the interaction process.  
Another example of improving interoperability and additionally maintainability is the initiative of defining an open-source version of the ETCS called _openETCS_. Between 2012 and 2015 a comprehensive toolchain for requirements identification, system modelling, code development, and verification as well as testing and further related development steps was established ([11]). Performance testing was not the main goal of this initiative rather than providing a reference system of the ETCS onboard unit (European Vital Computer) which allows the validation of commercially developed implementations.  
Throughout all of these projects, testing, both virtual and on the track, is necessary and is performed to prove the function of the system. On the other hand, comprehensive performance tests and the evaluation of different algorithms to determine variances in performance could not be identified during the literature research. This confirms the goal of this paper to provide an approach to evaluate and compare the performance of ATO algorithms in a virtual environment.  
While some standards for virtual testing in the railway sector have been identified, it could not be verified whether the described principles are used for testing ATO systems in current projects. Generally, the standardization of test methods and the transferability of the used data formats can contribute to an improvement of interoperability and comparability of technical systems in the railway sector.