# Measures and Methods for the Evaluation of ATO Algorithms
###### Abstract  
There is increasing interest in automating train operations of mainline services, e.g., to increase network capacity. Automatic train operation (ATO) is already achieved by several pilot projects, but is still not implemented on a large scale. Functional, interoperability and performance tests are necessary before ATO can be introduced generally. Virtual preliminary analysis will contribute to the validation process to ensure a safe and successful implementation. This paper aims to present an approach that applies to the performance testing of ATO systems. Therefore, methods and test standards for technologies enabling automatic operation in other transport sectors are reviewed. The main findings have been adapted, transformed and combined to be used as a general strategy for virtual performance testing in the railway sector. Specifically, universal performance indicators commonly used in the railway sector, namely punctuality, accuracy, energy consumption, safety and comfort, are presented. They are refined by adding sub-indicators specific to the performance evaluation of ATO algorithms. A layer model for scenario description is adapted from the automotive sector, as well as the definition of different scenario types. Lastly, factors that can influence the performance of an ATO algorithm are identified. For demonstration purposes, a simple case study is conducted. Thereby we exemplarily show-cased the approach for ATO performance testing using a microscopic train simulator in combination with an ATO algorithm. A 2022 12 4570 11  
The general scope of the approach is envisaged to allow a flexible performance analysis of ATO algorithms. The tests are specifically focused on determining whether an algorithm fulfils operational requirements. Runtime analysis and software specific questions are not addressed by the proposed approach since these requirements should be met in general. In Europe, ATO in combination with the European train control system (ETCS) is the preferred way. This is referred to as _ATO over ETCS_. An ATO system shall enable the train to regulate traction and braking automatically. The _ASTRail Consortium_, part of the European _Shift2 Rail_ project, was aimed at the identification of general functionalities of an ATO system. Within the project scope, [4] describes the tasks of an ATO system as follows:  
* retrieving necessary data for automatic operation,
* calculation of optimal speed profile,
* journey planning while including restrictions as well as provided trackside data and defining the resulting trajectory. Furthermore, based on the systems Grade of Automation (GoA) the following tasks need to be carried out by the ATO software components:  
* giving control to the human driver regularly or in case of an emergency or failure,
* opening and closing vehicle doors,
* establishing communication with trackside infrastructure, e.g., platform screen doors (PSD), passenger information systems, and send out commands. The use of ATO in different settings leads to the need to incorporate multiple train characteristics while simultaneously taking different track layouts into account. Challenging track sections can include gradient changes, varying speed limits, or complex routing. In combination with changing technical trackside equipment even within a country and differences in environmental conditions, all these characteristics contribute to a complex set of external factors [1; 5]  
The performance of ATO is connected to the combination of algorithms for the two main tasks of ATO, namely calculating the optimal speed profile and controlling the movement of the train. Different algorithms of each category exist with individual advantages regarding computational time or accuracy of calculations, or meeting other criteria like driving comfort and energy consumption in the case of vehicle control, respectively. One major condition must be met by all algorithms taking into account the real-world application. The computational time needs to be sufficiently short.