# WARM: Wearable AR and Tablet-Based Assistant Systems for Bus Maintenance
## 4 Description of the Experiments  
The evaluation of the WARM system has followed the EBSF2 EU project methodology [27], which assesses the performance variation in situations of \"do nothing\" and \"do something\" by measuring a sufficient set of Key Performance Indicators (KPIs) [28]. This evaluation framework has been applied to all the case studies of the EBSF2 project and other projects involving multisite demonstrators [29,30]. The methodology defines Validation Objectives (VO) that state general aspects in which the innovation tested is expected to be beneficial. The expected benefits of each VO are quantified through one or more performance targets (PT) that are actually measured through some KPIs. The evaluation method consists of analyzing whether the variation of the KPI values from \"do nothing\" to \"do something\" situations meet the expectations for each performance target. To assess whether a technological innovation is worth to retain or not, quantitative target values for each KPI have been set to serve as reference during the evaluation of test results. These improvement thresholds have been decided by the authors to assess the maturity of the innovations, i.e., to measure if the actual performance improvement is sufficient to continue the innovation development from current TRL 6/7 to TRL 9. For the evaluation of the WARM system, the \"do nothing\" situation corresponds to the current procedures, which is essentially paper-based handwritten data collection, and it is explained in Section 4.1. For the \"do something\" situation, two scenarios have been developed: one for the tablet-based solution, explained in Section 4.2, and another for the AR headset-based solution, detailed in Section 4.3. DBUS operations require that every morning at 6:00 am, a minimum of 103 vehicles of which 2218 m buses and 7312 m buses must be ready to run. The total fleet is composed of 130 buses. Maintenance tasks are organized in three periodical levels (M1 each 10,000 km, M2 each 30,000 km and M3 each 60,000 km), each level contains the previous one. In addition to these three actuations, the PRE_ITV assessment prepares the bus to succeed in the official technical inspection of the vehicle (known as ITV in Spain). In order to reduce the number of days that buses are out of service, the PRE_ITV assessment is made to coincide with one of the three periodical actuations. The garage (Figure 13) is able to service 10 buses simultaneously. Predictive maintenance work is planned with 7 to 10 days in advance. In addition to maintenance tasks, the daily work at the garage comprises eventually other reparations and actuations due to accidents, breakdowns and incidents occurred during daylight and nightly operations. To cope with this unpredictable workload, maintenance planning is revised every afternoon to guarantee that the needed vehicles to start operations the day after will be available. These four preventive maintenance tasks (M1, M2, M3 and PRE_ITV) were selected for testing the WARM system. The main purpose of the test was to compare the traditional reporting activity, i.e., handwritten reports, with the WARM-assisted reporting process using the tablet-based solution and the Holoens-based solution. In order to have a homogeneous data volume, the test plan has been oriented to the maintenance of MAN Lion's City bus models with numbers between 700 and 780. This bus model has the largest number of units in the fleet of DBUS. To compare the WARM system with the system currently used, a series of quantitative and qualitative Key Performance Indicators (KPIs) that represent the most relevant aspects of both maintenance management systems have been defined:  
* Dedicated effort in data management: time spent in planning and organizing work orders. * Dedicated effort in data processing: time spent completing the documentation of work orders. * Operator perception of workload: subjective perception of different methodologies by maintenance technicians. * Maintenance costs per vehicle.