# A Vision-Based Bolt Looseness Detection Method for a Multi-Bolt Connection
## 1 Introduction
Furthermore, a crucial prerequisite for implementing these methods is knowledge of the initial angle of the bolts before loosening, significantly constraining the scenarios in which they can be effectively used. In addition to estimating the angle of the bolt, researchers have also measured the vertical length (\(h\)) between the top of the bolt and the flat surface to detect loose bolts. [PERSON] et al. [21] achieved bolt looseness detection using only a smartphone. They combined several adaptive image processing algorithms to obtain various looseness-sensitive features. These features were used as the dataset of a linear SVM trained to recognize the loose bolts. But this method can only work well at a shooting distance between 78 mm and 122 mm and a perspective angle between 31\({}^{\circ}\) and 51\({}^{\circ}\); that is quite a small detection range. Similarly, [PERSON] et al. [22] located and recognized bolts using the Viola-Jones algorithm. They selected the exposed length of the bolt and the size of the bolt head as looseness features to train an SVM model for looseness detection. And [PERSON] et al. [23] built a classifier with a faster RCNN, similar to the classifier in [20]. They constructed an image dataset obtained from different perspective angles and distances to improve the robustness and utility of the model. But the detection accuracy of the method turned out to be affected by the perspective angle, which is a lack in the principle of machine vision. Compared to several previous methods that measured the \(h\), [PERSON] et al. [24] used readily available 2D images to reconstruct the three-dimensional (3D) map of multi-bolt connections from high-resolution images. They introduced a convolutional neural network (CNN) to recognize and segment the bolts in the 3D map. The vertical length between the bolt head and the reference plane was calculated to judge whether the bolt was loose. However, all these vision-based methods share common drawbacks: the need for prior information such as the initial angle and position of each bolt, requiring the direct visual observation of the bolts. The first issue is unavoidable in principle for those methods. Addressing the second issue could involve exploring the use of multiple cameras placed orthogonally within a plane though delving into this topic is a separate and complex matter, and it does not work well for some complex structures. In our previous work [25], we proposed a method to detect the looseness value of a single fastener (an elastic structure, which contains a deformable clip, just like a spring) in the track system. For elastic structure such as the fasteners in the track system, the previous method detects the loose fastener by detecting its looseness value, and it can only detect a single fastener at one time. In this paper, an automated vision-based method using digital shearing speckle pattern interferometry (DSSPI) and recurrent neural network (RNN) is proposed to detect multiple loose bolts in the multi-bolt connections (rigid structures) at one time. The main idea of this approach is to establish the correspondence between different kinds of fringe patterns and different bolt loosening states and then classify the speckle fringe patterns of the multi-bolt connection under different states to detect the loose bolts. The approach involves three key steps: (1) capturing the shearing speckle patterns of the multi-bolt connection under different states using the DSSPI system, (2) employing the DSSPI algorithm to obtain the speckle fringe patterns, and (3) inputting these fringe patterns to train the proposed RNN classifier. In actual detection, the pre-trained RNN classifier directly identifies the input fringe patterns to detect the loose bolts in multi-bolt connections.