# AR-Assisted Guidance for Assembly and Maintenance of Avionics Equipment
## 3 Materials and Methods
### Dynamic Assembly Pose Tracking
#### 3.2.1 Region Modality  
Our method integrates region, depth and texture modalities by solving the joint probability density function and subsequently optimizing it. Multi-modality fusion can improve the robustness of the algorithm and achieve stable and efficient assembly tracking.  
The region modality calculates the 3D pose transformation matrix between two frames by maximizing the probability that the sampling points on the contour of the tracking target in the previous frame are on the contour in the current frame.  
Our method calculates probabilities on the correspondence line. Firstly, the CAD model corresponding to the pose result of the previous frame is projected onto the imaging plane to form a mask. The sampling points on the contour of the mask are the midpoints of the correspondence lines. The correspondence line is obtained along the normal projection direction of the sampling point. The length of the correspondence line is 10 to 30 pixels. In order to reduce the amount of calculation, the length of the correspondence line will decrease with the increase in the number of iterative optimizations. The length is long enough in the first few iterations to ensure that the amount of change in the contour of the object between the two frames can be covered.  
The advantage of using sampling points is that the pixel values can be sparsely calculated along the correspondence lines. As shown in Figure 2, the blue points are the sampling points from the mask contour of the previous frame, the yellow lines are the correspondence lines, and the red line segment is the position of the contour edge with high probability. The length of the correspondence line decreases with the increase in the number of iterations, which reduces the amount of calculation and improves the operation speed.  
Figure 2: \(Cont.\)  
We calculate the probability that each pixel on the corresponding line belongs to the foreground and background. Based on Bayesian theory, we can calculate the posterior probability that each pixel of the correspondence line belongs to the mask contour of the current frame.  
First, we calculate the probability of each pixel belonging to the foreground or background along the current corresponding lines using the normalized color histogram method. Based on the assumption that each pixel is independent, the probability can be calculated using Bayesian theory.  
\[p_{si}(r_{s})=\frac{\prod_{y\in s}\ p(y\mid m_{i})}{\prod_{y\in s}\ p(y\mid m_{ f})+\prod_{y\in s}\ p(y\mid m_{b})},i\in\{f,b\} \tag{1}\]  
where \(y\) is the pixel color; \(f\) is the foreground; \(b\) is the background; \(p(y\mid m_{i})\) represents the probability that the pixel color belongs to the foreground or background; \(s\) is the step size, which is given by the user; and \(s\) pixels are divided into one segment to calculate the step size.  
Then, we calculate the distance (\(d_{s}\)) from the center (\(c\)) of the corresponding line to the estimated contour.  
\[d_{s}(\theta)=\left(n^{\top}(\pi(\ _C X(\theta))-c)\right)\frac{\tilde{n}}{s} \tag{2}\]  
where \(n\) is the normal vector and \(\tilde{n}=\max(\left|n_{x}\right|,\left|n_{y}\right|)\), \(\theta\) is the transform vector, \(\ _C X\) stands for the 3D points in the camera coordinate system, and \(\pi\) denotes the projection.  
Assuming that the above distribution conforms to the normal distribution after the pose transformation, the posterior probability density function corresponding to the pose transformation can be obtained.  
\[p(\theta\mid\omega_{s},l)\propto p(d_{s}(\theta)\mid\omega_{s},l)^{\frac{\pi^{ 2}}{\sigma^{2}\sigma^{2}}} \tag{3}\]  
where \(\omega_{s}\) is the area where the corresponding line \(l\) is located and \(\sigma_{c}\) is the user-defined standard deviation in the region modality.