## 2 Developed Methodology
### Supervised Pose Refiner  
In the following phase of TTool, the approximated estimation of the pose provided by the user needs to be refined to fit perfectly the corresponding physical tool's contour. To achieve this goal, we leveraged a modified version of a state-of-the-art pose refiner, SLET [20]. Necessary modifications were introduced to adapt the basic algorithm to our own study case and application scenario. The following is a description of our contributions to a modified version of the refiner and how human interaction is integrated into its functioning. It is worth noting that the designated refiner requires only the camera's RGB feed, making this phase particularly lightweight. In SLET [20], corresponding 2D-3D points are established by matching the contour points from the video frame (2D points) with the projected 3D model contour (3D points). SLET effectively addresses challenges such as occluding contour points, complex texture backgrounds, and other kinds of noise by leveraging probability computations. For each candidate contour point \(h_{ij}\), three probabilities are computed based on the color cue: the probability of \(h_{ij}\) belonging to the foreground (\(P(h_{ij}|F)\)), the probability of \(h_{ij}\) belonging to the background (\(P(h_{ij}|B)\)), and the probability of \(h_{ij}\) being the actual contour (\(P(h_{ij}|C)\)). To ensure the robustness of the selected contour points, SLET applies a filtering criterion to them. A candidate contour point is retained only when the probability of it being an actual contour point surpasses both the probability of it being in the foreground and it being in the background (Equation (3)). \[P(h_{ij}|C)>P(h_{ij}|F)\text{ and }P(h_{ij}|C)>P(h_{ij}|B) \tag{3}\]  
The refiner may lose track while attempting to update the pose on the current frame. In our setup, where the camera is fixed on the tool head (see Figure 10), a more effective strategy is either to maintain the current pose without updating it or to reset the pose to the best one previously specified by the user. This dual approach results in optimal performance due to the stationary nature of the tool heads relative to the camera. Hence, it is introduced in our version of the refiner. Our investigation reveals a direct correlation between the \(P(h_{ij}|C)\) of selected candidates and the tracking quality. When the tracker loses signal of the model, the average probability, denoted as \(\langle P(h_{ij}|C)\rangle\), Equation (4), of the representing contours is observed to be low. A correlation between the value of 0.00625 and tracking loss is also found via experimental tests. Therefore, the object's pose should reset to the initial pose. A value of 0.1 correlates with a low-quality update. Thus, the model should not update its pose to the new pose computed with these contours,  
\[\langle P(h_{ij}|C)\rangle=\frac{1}{|C|}\sum_{h_{ij}\in C}P(h_{ij}|C) \tag{4}\]  
where \(C\) is the filtered set of contour points. When tracking is lost, the negative feedback loop begins. In other words, an inaccurately oriented 3D model contour is employed for the subsequent pose computation. We use a semi-automatic adjustment to address this issue. We introduce Algorithm 1 using the tracking score described above. In the event of tracking loss, the object pose is reset to its initial position, as this initial pose (\(T_{i}\)) is expected to be near the current tool head's location. This facilitates the tracker in regaining its tracking capability. When the tracking achieves a higher score, indicating the presence of several quality candidates for the newly computed pose (\(T_{cm}\)), we proceed to update the pose. Alternatively, if the tracking score is lower, signifying a lack of robust candidates, we opt to freeze the tracking at the current pose. Moreover, we provide users with the option to specify whether the current mode pose is precise and should serve as \(T_{i}\) when tracking is lost.