# A State Validation System for Augmented Reality Based Maintenance Procedures
## 2 State of the Art  
Maintenance is one of the major applications for AR since its early adoption [10]. The first attempts of supporting technicians using AR tools can be dated back to the 1990's, as detailed in [6; 7], and the majority of these were based on special purpose hardware (e.g., HMDs). The maintenance domain poses some specific challenges to the adoption of AR technologies [11]: Tracking is one of the most important and has been deeply investigated through the years. Real objects' detection and tracking in an industrial environment is a challenging task based upon the progress of image processing algorithms. Whenever it is unavailable, marker-based solutions are adopted. Since the early experiments by [PERSON] et al. [12] on the KARMA project, with a head-worn AR prototype designed to support end-users in performing simple maintenance procedures on a laser printer, marker-based AR applications have been widely investigated. Other important projects based on maker solutions are the ARVIKA project [13], the STARMATE (SysTem using Augmented Reality for Maintenance, Assembly, Training and Education) project [14] and the ManuVAR [15] project. However, the use of markers is an impractical solution in industrial environments, thus the tracking of the target objects should rely on their natural characteristics, such as textures or edges, identified through image processing. In 2006, the ARTESAS project (Advanced Augmented Reality Technologies for Industrial Service Applications) continued the research activities started by the ARVIKA project, focusing on three themes: Markerless tracking systems in industrial contexts, interaction methodologies with AR devices and industrial AR applications [16]. Another challenge is provided by the industrial environment, since the real case scenario and target object could be far from the original representations of the procedure, maintainers may need help to correctly perform their task. The Etala project [17] proposed tele-assistance and AR to establish a communication channel between local maintainers and remote experts. A similar approach was adopted in [18], where AR and VR were used to remotely support both trainees and technicians. In [19], a collaborative framework solution is proposed to train technicians in assembly tasks of complex systems such as aircraft engines. Another example is provided by the EASE-R\({}^{3}\) project [20],which aimed to develop an integrated framework for an easy and cost-effective repair, renovation and reuse of machine tools within modern factories. As part of this project, an AR framework has been developed to provide: AR aids for maintenance procedures, remote assistance from an expert technician, and re-configurability of the maintenance procedures. However, industrial environments may prevent the deployment of such solutions, for example due to network limitations. Overall, to further increase the benefits of AR systems to the maintenance domain, it should be combined with other tools, such as real-time monitoring, fault-diagnosis and prevention systems. These tools may enhance the capabilities of technicians and thus improve maintenance performances. Automatic validation of the user actions could be one of these tools and, since markerless solutions are to be preferred in industrial environments, it could rely on the same image processing algorithms used to track the objects in the scene. The number of research projects on AR for maintenance clearly implies that this is a challenging area of investigation. However, the design and development of a system that could validate the tasks performed by the maintainers, independently of the environment, is still an open problem. This paper proposes a system that overcomes this problem using a combination of computer vision techniques.