## 3 Materials and Methods
### System Overview  
We solve the object SLAM problem for all ellipsoidal approximations of objects \(\mathcal{Q}=\{Q_{j}^{*}\}_{j=0}^{J}\) with \(J\) object landmarks and \(T\) poses of the camera \(\mathcal{X}=\{x_{t}|x_{t}\in\text{SE}(3)\}_{t=0}^{T}\). The front-end input of the system includes the monocular images, as well as odometry data from an external vision-based localization system. The object detector extracts bounding boxes from the RGB images. We extract \(K\) bounding box measurements of objects \(\mathcal{B}=\{B_{k}|B_{k}=[x_{min},y_{min},x_{max},y_{max}]\}_{k=0}^{K}\) along with the semantic class labels \(\mathcal{C}=\{c_{k}\}_{k=0}^{K}\)  
Figure 4: Estimation of orientation by minimizing the cost function. (**a**) The cost function value with respect to orientation angle. (**b**) The corresponding symmetry points under different angles. The top right is where the function reaches the minimum. The following data association method solves the correspondence between bounding boxes \(\mathcal{B}\) and landmarks \(\mathcal{Q}\). Although more sophisticated data association [35; 36] can be used instead, in this work, we use a minimal technique: each valid bounding box detection is associated to an existing ellipsoid or triggers a new landmark creation. Every landmark approximation ellipsoid is first back-projected into the frame as a rectangular landmark box using the measurement model proposed in [11]. Given a new bounding box detection \(B_{k}\), we synthesize a cost value for matching it with the existing object landmark \(Q_{j}\), which is defined as:  
\[cost\big{(}B_{k},Q_{j}\big{)}=\lambda*d_{k,j}+\mu*IoU_{k,j}+\xi*l_{k,j} \tag{8}\]  
where \(d_{k,j}\) is the distance between the centroids of the bounding box and the back-projected landmark box, \(IoU_{k,j}\) is the intersection over union (IoU) of the two boxes, and \(l_{k,j}\) measures the semantic discrepancy between the detection and the landmark. \(l_{k,j}\) is set to 0 if the detection and the landmark have the same semantic label, and 1 if their labels are different. Here, we let \(\lambda\) be the reciprocal of the image diagonal length and \(\mu=\xi\). Then, with the Hungarian matching algorithm [37], we associate the detection with the landmark when the cost reaches the lowest, and, if no match is found, we initialize a new landmark. With the data association problem solved, the problem in the back end of the object SLAM can be written as:  
\[ \begin{split}& P(\mathbf{\mathcal{X}},\mathbf{\mathcal{Q}}\mid\mathbf{\mathcal{ B}},\mathbf{\Theta}^{s},\mathbf{\mathcal{I}},\mathcal{C})\propto\underbrace{\prod_{k=0}^{K}P \Big{(}B_{k}\mid\mathbf{Q}_{j}^{*},\mathbf{x}_{k}\Big{)}}_{\text{Bounding Box \@@cite[cite]{[\@@bibref{}{Bow2019}{}{}]}}}\\ &\underbrace{\prod_{d=0}^{D}P\Big{(}\mathbf{\theta}_{s}^{s}\mid\mathbf{Q }_{j,t}^{*},\mathbf{x}_{t_{d}}\Big{)}}_{\text{Symmetry (Ours)}}\underbrace{\prod_{j=0}^{J}P\Big{(}\mathbf{Q}_{j}^{*}\mid c_{j}\Big{)}}_{ \text{Semantic Prior \@@cite[cite]{[\@@bibref{}{[PERSON]}{}{}]}}}\underbrace{\prod_{t=0}^{T}P (\mathbf{x}_{t}\mid\mathbf{I}_{0:t})}_{\text{Pose Prior}}.\end{split}  \tag{9}\]  
The problem can also be formed as a factor graph, including nodes composed of objects and camera poses and edges composed of observation constraints, as in Figure 5, where \(X\) is the camera poses and \(Q\) is the objects in the map. \(z\) is the camera-object observation constraint, \(o\) is the odometry constraint; both were described in detail in [11]. \(s\) is the newly added 3D symmetric constraints and will be emphasized in the next subsection. Finally, we can obtain the optimal estimation of camera poses \(\hat{\mathcal{X}}\) and objects \(\hat{\mathcal{Q}}\) by maximizing the posterior probability.