## 4 Related Work  
6 DoF pose estimation using RGB images involves different fields such as bin picking problems [34], robot manipulation [35], autonomous vehicles [36], and MR applications [37]. Figure 1: Transitioning from 2D video to the 3D virtual object: (**a**) 2D video. (**b**) Object replacement after detection and localization. (**c**) Object rotating in front of the user viewpoint. (**d**) Digital information contextualized with the vehicle model. The corresponding videos can be found here ([[https://youtu.be/E1](https://youtu.be/E1) fwuexrCo0]([https://youtu.be/E1](https://youtu.be/E1) fwuexrCo0), accessed on 20 April 2022). Usually, to accomplish this task, deep learning methods are used. One of the main approaches to 6 DoF pose estimation, as described in [38], is to decouple the translation and the rotation estimation. The translation is estimated by localizing the centre of the object in the image and predicting its distance from the camera. After that, the rotation is estimated by regressing to a quaternion representation. A 6 DoF Object detection system with two stages is also proposed in [39]. A single Shot Multibox Detector (SSD) [40] extracts the object bounding boxes and an Augmented AutoEncoder (AAE) estimates the object rotation. Similar to the previous approach, DCS-PoseNet [41] uses a two-step process to estimate 6 DoF from 2D object bounding-boxes. First, the framework segments the object from the cropped image, followed by predicting 6 DoF pose using DSC-PoseNet, which employs a differential renderer. Some solutions try to regress rotation and translation simultaneously. For example, 6D-VNet [42] uses an end-to-end deep learning network to estimate the 6 DoF pose of vehicles. The network extends the Mask R-CNN object detector and takes its intermediate outputs and further regresses for rotation and translation of the object in 3D space. Other approaches instead try to solve a Perspective-n-Point problem [43]. For example, the pose estimation method Pix2 Pose [44] proposes a deep learning network to supplement a 2D detection pipeline to enable pose estimation. It regresses pixel-wise 3D coordinates from images using texture-less 3D models. The pixel-wise prediction is used to form 2D-3D correspondences. Finally, the PnP algorithm can be applied. In Ref. [45], the authors propose an extension of the EfficientDet architecture [46] used for 2D object detection to predict the rotation and the translation of the object in the 3D space. Most current works describe the problem statement and solution for regular RGB images. The application of the algorithms of these works to equirectangular images is tricky. The main reason is that equirectangular images present severe distortion and there is a lack of training data related to these types of images. To the best of our knowledge, some works try to perform 2D object detection in equirectangular images [47; 48], but none perform an estimate of the 6 DoF pose. For this reason, our work contributes moving toward objects pose estimation also for this kind of images. Indeed, 360\({}^{\circ}\) videos are becoming more popular in the educational context, as they provide students with a more engaging learning activity [49]. The method presented in this paper shows an example of how this media, together with VR technologies, can create contemporary virtual learning environments, which enable students to experience and interact with virtual content [50]. 360\({}^{\circ}\) videos, in contrast with 2D movies, offer an immersive experience not only by giving the perception of being physically present in a virtual environment but also allowing interaction with objects. This is only possible when knowing the pose of the object with which we want to interact with inside the scene. Our pipeline specifically solves the problem of 6 DoF pose estimation for objects in equirectangular images. Additionally, while other methods primarily rely on deep learning models to perform the task, ours uses deep learning only for segmentation, which is the first step.