## 2 Theory and Algorithm  
### Deviation Extraction in a Single Image  
Structural deviation variations should be extracted from video frames at the beginning of the analysis process. Consider a simple idealized edge profile in subimage \(I_{s}(x,y)\), where \((x,y)\) is the pixel coordinate and refers to the timeline; the edge profile at location \(x\) can be defined as:  
As can be seen from the derivations above, the average edge profile approximates the common edge profile up to a constant shift item. Given the observations of \(E(y)\), deviation signal \(f(x)\) can be obtained by using a least squares estimation:  
\[argmin\sum_{y}\left(I_{\mathrm{s}}(x,y)-E(y)-f(x)E^{\prime}(y)\right)^{2} \tag{7}\]  
which leads to the final approximate solution of the deviation variations in single subimage \(I(x,y)\):  
\[f(x)\approx\frac{\sum_{y}\left(I_{\mathrm{s}}(x,y)-E(y)\right)E^{\prime}(y)}{ \sum_{y}E^{\prime}(y)^{2}} \tag{8}\]  
We demonstrated the effectiveness of the deviation extraction by using a simple carbon tool steel saw blade experiment. A saw blade has a jagged edge on one side and a smooth edge on the other side. Figure 1a shows the tested saw blade fixed on a table vice. The image was captured with a Canon 70D SLR camera, and the shapes of the jagged edge and the smooth edge are difficult to identify with the naked human eye in the original image. Common edge detection algorithms will identify the jagged edge as an idealized straight-line element. These ideal geometries such as straight-line or curves on the surface of the test structure are considered to be the idealized edge profiles [25]. Our ultimate target is to extract subtle deviation variations along the jagged edge and the smooth edge. In this experiment, an analysis area was selected in Figure 1a for further operations. The selected analysis area is transformed from the RGB color space to grayscale because the deviation extraction algorithm only works on image intensities. From Figure 1b, two subimages within the yellow rectangles are selected as the regions of interest (ROIs). The size of the ROIs is \(480\times 50\) pixels, and the red and blue lines within the ROIs represent the idealized straight-line segments that are identified using the line segment detector. In reality, these detected line elements are not perfectly straight-lines and contain unnoticeable subtle vibrations. During the analysis, the detected smooth and jagged sides are considered as idealized straight-lines for the convenience of the following sampling process. We consider the direction of the idealized straight-line segments as the \(x\) axis, and the edge profiles are then sampled strictly perpendicular to it. The sampling direction (\(y\) axis) is  
Figure 1: Experimental setups of the saw blade experiment: (**a**) saw blade and the selected analysis area; (**b**,**c**) schematic and extraction results of the saw blade experiment. marked with green arrows in Figure 1b. Figure 1c shows the output deviation extraction signals of the jagged edge and the smooth edge without performing any denoising process. The unnoticeable cyclical jagged shape marked with a blue line is extracted successfully from the saw blade subimage. The amplitude of the deviation signal on the smooth side is quite small (less than 0.1) and can be observed as a smooth line. The simple saw blade experiment above explains the deviation extraction process from the idealized straight-line profiles. The sampling directions are programmed to be strictly perpendicular to the identified line to observe variations that are vertical to the idealized straight lines. For complicated edge profiles, such as circular or ellipse boundaries, the idealized fitted curves can be calculated by edge extraction methods, such as Hough transformation. Polar coordinates instead of rectangular coordinates are applied to locate the sampled pixel intensities.