## 5 Algorithm Description  
We can subdivide the developed algorithm into two main steps:  
* Vehicle segmentation from the equirectangular image;
* Vehicle pose detection with respect to the camera reference frame (6 DoF).  
To accomplish the first task, we made use of Yolact++ [55], a convolutional model for real-time instance segmentation. Yolact++ proved to be pretty accurate for the segmentation of trucks, also in equirectangular images, in which the distortion is significant. However, to make the vehicle segmentation more robust, we trained Yolact++ with 500 equirectangular images by manually labelling trucks in frames taken from 360\({}^{\circ}\) videos of open-pit mining operations. Figure 2 shows the result of the trained Yolact++ model.  
The only requirement for the vehicle pose detection is to have an accurate CAD model even without textures of the item, whose pose must be estimated. For the tests described in this paper, we used the CAD model of a Komatsu HD785 truck. We used the CAD model inside Unity 3D, a cross-platform game engine. In this case, we used Unity to create a simulation environment. We implemented a 360\({}^{\circ}\) camera simulator in this environment to capture equirectangular images of the CAD model.  
Using the output given by Yolact++, we can perform the vehicle pose detection. To accomplish this task, we developed the algorithm schematically described in Figure 3.  
Through a connection based on the ZMQ protocol, we allow data exchange between Unity and Matlab. We put the vehicle in a random pose inside Unity. Then we take a picture of the scene using the 360\({}^{\circ}\) camera simulation. We will call this picture the synthetic image. In Matlab, we compare this synthetic image with the output of Yolact++ (the segmented  
Figure 3: Scheme of the pose detection algorithm.  
Figure 2: The result of the trained convolutional model Yolact++ for an equirectangular image of a truck. (**a**) An example of an input image showing a truck in a mining environment. (**b**) Result of the segmentation in which the truck is correctly segmented.  
real-world image). A score is given for the similarity between the two pictures. A Particle Swarm (PS) optimizer is in charge of finding the optimal solution. At each new iteration, the algorithm sends a new pose to Unity. The 360\({}^{\circ}\) simulated camera takes a new picture, and we repeat the previous steps until convergence or the maximum number of iterations are reached.  
Hereafter, the expression of the Cost Function (CF):  
\[CF=S_{E}+S_{A}+S_{C}+S_{V}, \tag{1}\]  
where its terms depend on:  
* Edges (\(S_{E}\));
* Area (\(S_{A}\));
* Difference in the centroids of the edges (\(S_{C}\));
* Difference in the eigen vectors of the edges (\(S_{V}\));  
In the following subsections, we explain the various term of the cost function in detail. It is important to highlight that the terms of the cost function also work for texture-less objects because they mainly rely on shape features than on texture. To show the potential of the developed algorithm, we choose to apply it to a vehicle model whose CAD was at our disposal. The CAD model does not perfectly match the real object in every detail but we will show that the results are accurate enough.  
The real-world and synthetic images in Figure 4 are taken as an example to show the computations made for the different terms of the cost function.