# A State Validation System for Augmented Reality Based Maintenance Procedures
## 3 The Proposed System
### Tracking  
The first step to deploy an AR maintenance system is to determine how to perform the tracking of the machinery and how to identify the elements that are subject to changes during the procedure. In order to avoid the limitations provided by the context where the machinery is deployed, a markerless model-based tracking system has been adopted. Among the three available markerless algorithms provided by ViSP, the vpMbEdgeKltTracker has been chosen, since it is a hybrid solution that provides a more accurate tracking of the object; vpMbEdgeKltTracker relies on both edge detection [21; 22] and KLT (Kanade-Lucas-Tomasi) key points [23]. Edge detection algorithms usually perform well when objects without texture and with sharp and neat edges have to be tracked. On the other hand, the KLT (Kanade-Lucas-Tomasi) algorithm performs well when textured objects, with edges that could be difficult to recognize, have to be tracked.  
In order to adopt a model-based tracking technique, it is necessary to obtain a Computer-Aided Design (CAD) model of the object to be tracked for each step of the procedure, which could be the whole machinery or a part of it. Moreover, 3D CAD models suitable for ViSP have to be saved as Cao files. Cao is a proprietary format developed for ViSP, which defines a 3D object as a textual description of lines, circles and cylinders that represent the object. The open source modelling tool Blender ([[https://www.blender.org/](https://www.blender.org/)]([https://www.blender.org/](https://www.blender.org/))) has been used to model and to export CAD representations of the objects as Obj files. Since ViSP requires input geometries defined by Cao files, a Blender's Python script performing the conversion from Obj to Cao format has been implemented.  
Figure 3: Software layers of the proposed system. ViSP (Visual Servoing Platform), OpenCV (Open Source Computer Vision Library), Ogre3D (Open Source Graphics Rendering Engine).  
When the scene is framed by the camera, the system searches for the object to be tracked at the current step of the procedure, which could be the whole machinery or, if too big to be framed from a short distance, a part of it. However, all the three VISP's tracking algorithms consider all the feature key points for the framed object as a whole, thus, it is not possible to directly link a feature key point to a face of the CAD model. This limitation make it impossible to correlate feature points, provided by the algorithm, to a single element of the machinery that may change during the maintenance procedure. To overcome this issue, the original tracking algorithm has been modified in order to manage individually the different elements (the polygonal faces of the 3D CAD model) involved in the procedure. Since Cao files allow appointing each face of the object, it is possible to label the faces uniquely. Therefore, it is possible to define, for the chosen object, which are the elementary parts that are involved in the maintenance process, for each step of the procedure. The term 'component' is denoted as a part of the equipment under test, defined by a set of polygonal faces. If the maintenance operations have been correctly split into elementary steps, each step will usually affect only one component.