# A State Validation System for Augmented Reality Based Maintenance Procedures
## 4 Workflow
### Motion Detection  
When the user performs the procedure, false status changes could be recognized due to two possible reasons. Firstly, if a motion happens inside the framed scene, such as the movement of the user's hands. Secondly, if the distance and/or position between camera and tracking object changes, which may happen both if the camera or the object are moved or if the zoom factor of the camera is changed. In order to avoid these false positives in recognizing status changes, a mechanism of motion detection has been introduced. If at least one of the possible cause of motion is detected, then the validation system is halted till the motion end. Figure 7 shows all the operations performed by the motion detection modules. It is possible to define the motion inside the scene as a transitory state between two static poses, since usually the workflow of a maintenance procedure requires the user to:  
1. Frame the machinery, thus obtaining a static pose;
2. Learn through the assets which is the operation to be performed for the current step;
3. Interact with the machinery, thus generating a movement inside the scene. To avoid false recognitions of status changes, motion detection inside the scene is performed through an algorithm that adopts some functions provided by OpenCV. The proposed method is based on performing the difference between two subsequent frames and applying a thresholding on the result. First of all, the two frames are converted to grayscale and the difference is computed with the OpenCV's absdial function. Then, the thresholding is applied to the result of the difference. The final image will be completely black if the two images are equals, whereas white dots will identify the differences between the two images, thus implying that a movement occurred. In order to lower the number of false positive, such as the ones due to imperceptible movement of the camera, two different techniques have been adopted. Firstly, an erosion morphological operator has been applied to the result of the thresholding. Erosion allows deleting or reducing objects in a binary image, removing the smallest details. The resulting image is cleaner and presents less noise. Secondly, standard deviation has been computed for the image in order to obtain a value that actually describes motion distribution. Standard deviation will be equal or near zero if the motion affects only one pixel, such as the movement of a faraway object in the background. Otherwise, if most of the  
Figure 7: Motion detection schema. framed area is affected by motion, then the standard deviation value will be high. The value of the standard deviation is then taken into account to decide if a motion in the scene has occurred. The second issue was to identify camera motion, object motion or camera zooming. To this end, an algorithm has been developed to compare the coordinates of the bounding boxes that defines the distinct parts of the machinery at subsequent frame rates. When the tracking algorithm identifies the machinery, given its coordinates in the reference coordinate system, it is possible to compute the position of the bounding boxes which define its parts. If, at any next frame, one or more of these bounding boxes moves over at last one axis for more than a minimum threshold, as defined by Equation (2), then a movement is detected, as shown in Figure 8.  
\[\sum_{c=x}^{y}\sum_{j=min}^{max}\frac{(c_{ji}-c_{j(t-1)})\backslash th}{(c_{ min(t-1)}-c_{max(t-1)})\backslash th}=0 \tag{2}\]  
Each bounding box is defined by four values, \(x_{max}\), \(x_{min}\), \(y_{max}\) and \(y_{min}\). In the equation, \(c\) represents the \(x\) or \(y\) coordinate of the bounding box at the current frame \(t\) or at the previous frame \(t-1\). The variable \(th\) is the chosen threshold whereas the \(\backslash\) operator represents the integer division for \(th\). The threshold value \(th\) has been obtained experimentally performing numerous tests with the system and it defines the minimum movement along an axis that should be recognized to detect a motion.