SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Wonhyeok Choi* Mingyu Shin* Sunghoon  $\mathbf{Im}^{\dagger}$
DGIST, Daegu, Korea
{smu06117,alsrb4446，sunghoonim}@dgist.ac.kr
Monocular 3D object detection poses a significant challenge due to the lack of depth information in RGB images. Many existing methods strive to enhance the object depth estimation performance by allocating additional parameters for object depth estimation, utilizing extra modules or data. In contrast, we introduce a novel metric learning scheme that encourages the model to extract depth-discriminative features regardless of the visual attributes without increasing inference time and model size. Our method employs the distance-preserving function to organize the feature space manifold in relation to ground-truth object depth. The proposed  $(K,B,\epsilon)$ -quasi-isometric loss leverages predetermined pairwise distance restriction as guidance for adjusting the distance among object descriptors without disrupting the non-linearity of the natural feature manifold. Moreover, we introduce an auxiliary head for object-wise depth estimation, which enhances depth quality while maintaining the inference time. The broad applicability of our method is demonstrated through experiments that show improvements in overall performance when integrated into various baselines. The results show that our method consistently improves the performance of various baselines by  $25.27\%$  and  $4.54\%$  on average across KITTI and Waymo, respectively.
Monocular 3D object detections [20, 22, 23, 27] has gained prominence as a cost-effective and easily deployable solution, playing a critical role in autonomous driving and robotic navigation systems. Typically, the frameworks consist of a feature extractor and lightweight multi-head architectures that predict the projected centers, depths, bounding box sizes, and heading directions of multiple objects from a single RGB image [18, 20, 22, 23, 26, 27, 41]. Among these tasks, object depth estimation, inferring the distance from a monocular camera to the center of an object, is the most challenging subtask due to depth ambiguity. Previous work [23] has highlighted this issue in ablation studies, revealing that substituting object depth predictions with ground-truth (GT) values significantly improves overall performance, while replacing other sub-tasks, such as heading direction and 3D size, does not notably enhance performance. Furthermore, several prior works [3, 10, 18, 22, 27, 28, 37, 41] have attempted to improve object depth estimation quality by adding additional modules or introducing new formulations. However, despite the subsequent increase in inference time and model size, the improvement in performance remains limited.
These observations suggest that monocular 3D object detection heavily relies on object depth quality; nevertheless, conventional methods yield unsatisfactory results due to the extraction of less-discriminative features for object depth inference. One reason is that the extracted feature involves visual attributes, such as the color, size, and heading direction of objects, resulting in the object
†Corresponding Author
depth head receiving feature with limited depth discernment. To improve object depth performance, the network should be capable of extracting purely depth-discriminative features that involve essential geometric information, irrespective of the visual attribute. A feasible method for extracting the depth-discriminative features involves using deep metric learning schemes such as contrastive learning [6, 7, 39] or representation learning [5, 34].
However, most existing deep metric learning schemes [6, 7, 16, 39] rely on aggressive two-view augmentation (i.e. Affine transform) to train the distance or similarity metric. This aggressive data augmentation method is hardly leveraged in current monocular 3D object detection frameworks due to the violation of geometric constraints, where horizontal flip and color distortion are the only two methods used in this field for a long time [19]. One alternative way is to learn a regression-aware representation by contrasting samples against each other based on their target distance using GT labels [39]. However, forcibly arranging the manifold of the feature space using this depth distance metric may negatively impact the performance of the other tasks, because the feature extractor for monocular 3D object detection inevitably produces complex shared representations across multiple sub-tasks. Note that the experiment of negative impact is conducted in Sec. 5.2.
To address these issues, we propose a  $(K,B,\epsilon)$ -quasi-isometric loss, a new metric learning approach that encourages the network to extract depth-discriminative features using object depth labels. Inspired by several manifold learning schemes [29, 33], we locally preserve the neighborhood distances to maintain the natural non-linear manifold of feature space in order to mitigate the negative transfer effect on other sub-tasks. Our approach utilizes the quasi-isometric properties providing a relaxed condition for the distance metric between the depth and feature metric spaces. This enables the model to arrange the feature space with respect to the object depth labels while maintaining the performance of other sub-tasks. Moreover, we introduce an auxiliary head for object-wise depth estimation to further improve object depth estimation. The head component is removed after the training process, ensuring that the inference time remains unaffected and does not experience an increase. Experimental results indicate that the proposed method consistently outperforms state-of-the-art baselines across a variety of 3D object detection datasets. Our method has been shown to be compatible with various monocular 3D object detection frameworks [15, 20, 22, 23, 27], demonstrating its broad applicability without compromising inference time or increasing model size. The effectiveness of each proposed module is further underscored through comprehensive ablation studies.
- We propose a simple yet effective metric learning scheme that preserves the geodesic distance of depth information to feature space.
- We present an auxiliary head for object-wise depth estimation, which enhances the depth quality without impacting the inference time, maintaining efficient performance.
- Our method significantly enhances the performance of various monocular 3D object detection methods without increasing inference time and model size.
Monocular 3D object detection. Monocular 3D object detection can be broadly categorized into two types. The first type [3, 18, 20, 22, 23, 25, 37, 41] predicts the localization of objects of interest around an ego vehicle using only RGB images, annotations, and camera calibrations. Most of them are based on CenterNet [43]. They are divided into several sub-tasks, with a primary focus on estimating accurate object depth. The works [22] and [41] propose the formulations of object depth estimation. MonoCon [20] leverages the abundant contexts in conventional 3D bounding box annotations to integrate various auxiliary tasks. The second category leverages additional data, such as pre-trained models [8, 10, 15, 27, 28, 35] or CAD models [24, 38], to compensate for the lack of 3D information in monocular images. Some prior works utilize pre-trained depth estimators to address the lack of depth information, either by converting the monocular setups to LiDAR/Stereo environments [8, 35] or by allocating more parameters to the depth estimation task [10, 15, 40]. Alternatively, certain methods employ completed depth maps from pre-trained depth completion models [27] or pre-trained LiDAR-based detectors [8, 14] to supervise their models. Notably, DID-M3D [27] divides object depth into visual and attribute depth using completed depth maps to address the ambiguity of the object depth.
Manifold geometry preservation. Deep learning networks, consisting of continuous and differentiable (precisely, almost everywhere differentiable) layers, ensure smooth mapping of networks  $f: \mathbb{R}^m \to \mathbb{R}^n$ . This guarantees that neighborhoods in  $\mathbb{R}^m$  will be mapped into neighborhoods in the embedding space  $\mathbb{R}^n$  with some amount of "stretching" and vice versa. Manifold learning has been studied to preserve most or all of the essential information by minimizing these stretches. Many works [1, 2, 17, 29, 42] aim to create meaningful representations when embedding high-dimensional data into lower dimensions. The traditional linear embedding algorithms commonly use the matrix decomposition to preserve the original variance[1] or pairwise distance [17]. More recent algorithms [2, 29, 33, 42] construct the neighborhood graphs by preprocessing the dataset using  $\epsilon$ -ball or  $k$ -nearest neighbors for each point, embedding the data while preserving the neighbor samples' distance. In particular, Isomap [33] estimates the shortest path in the neighborhood graph between every pair of data points and then employs the Euclidean Multidimensional Scaling (MDS) algorithm [17] to embed the points in  $d$  dimensions with minimum distance distortion all at once. These non-linear embedding algorithms can account for the non-linear nature of the manifold by preserving the distance between neighborhood samples.
Metric learning. Metric learning is one of the machine learning techniques that aims to learn an effective distance metric between data points by using training data. To achieve this, methods such as [6, 7, 16] attempt to minimize the distance between samples from the same class while maximizing the distance between samples from different classes. These methods demonstrate improved performance for self-supervised learning by learning in an end-to-end fashion, including two-view augmentation or depending on class labels. These deep metric learning methods typically target classification tasks because the positive/negative pairs are defined as belonging to the same or different classes. More recent works [31, 36, 39] extensively apply metric learning in the context of regression. These works either directly use the target distance to contrast samples against each other [36, 39] or leverage representation learning in a semi-supervised learning scenario [31].
Our final goal is to boost the object depth performance by extracting the depth-discriminative features without increasing inference time. Similar to common deep metric learning schemes [6, 7, 39], we propose a loss term that preserves a meaningful low-dimensional data structure in the feature space.
Metric space. A metric space is a mathematical concept that characterizes a set of points and a function that measures the distance between any two points in the set. Formally, a metric space can be defined as a pair  $(M,d)$ , where  $M$  represents a set and  $d$  is a distance function on  $M$ . The distance function  $d$  must satisfy the following axioms [13] for any three points  $x,y,z\in M$ :
1. Non-negativity:  $d(x,y) \geq 0$  and  $d(x,y) = 0$  if and only if  $x = y$ .
2. Symmetry:  $d(x,y) = d(y,x)$  for all  $x,y\in M$ .
3. Triangle inequality:  $d(x,y) \leq d(x,z) + d(z,y)$  for all  $x,y,z \in M$ .
Quasi-isometry. A quasi-isometry is a function between two metric spaces that preserves distances up to a constant factor, even though it may locally distort angles and distances. Let  $\mathcal{Q}$  be a function from one metric space  $(M_1,d_1)$  to another metric space  $(M_2,d_2)$ .  $\mathcal{Q}$  is considered a quasi-isometry from  $(M_1,d_1)$  to  $(M_2,d_2)$  if there exist constants  $K\geq 1$ ,  $B\geq 0$ , and  $\epsilon \geq 0$  such that both of the following properties hold:
1.  $\forall x_{1},x_{2}\in M_{1}:\frac{1}{K}\cdot d_{1}(x_{1},x_{2}) - B\leq d_{2}(\mathcal{Q}(x_{1}),\mathcal{Q}(x_{2}))\leq K\cdot d_{1}(x_{1},x_{2}) + B.$
2.  $\forall z\in M_2:\exists x\in M_1$  s.t.  $d_{2}(z,\mathcal{Q}(x))\leq \epsilon$
Quasi-isometry does not necessarily require continuity [4]. This property is advantageous because other distance-preserving transformations may not possess it, and most datasets are finite. Therefore, we use these conditions as constraints to ensure that the feature space retains the geometrical information of the depth metric space.
3.2 Problem Definition
The task of monocular 3D object detection aims to predict both the object class  $c$  and the 3D bounding box  $\mathbf{b}$  for multiple objects within an image  $\mathbf{I}$ . Recently, CenterNet [43] has become the best practice for monocular 3D object detection, with subsequent works [18, 20, 22, 23, 27, 41] adopting its pipeline. These methods decompose the bounding box  $\mathbf{b} = [X,Y,Z,h,w,l,\gamma]$  estimation problem into separate estimations of the coarse projected 3D center  $(u,v)$ , the depth of object center  $z$ , the center offset  $(\delta u,\delta v)$ , the 3D size dimensions  $(h,w,l)$ , and heading direction  $\gamma$  (i.e., the yaw angle). The 3D object center  $[X,Y,Z]$  is computed by back-projecting the projected center with the corresponding depth, given the intrinsic matrix of the camera  $\mathbf{K}$ , as follows:
$$
u _ {c} = u + \delta u, v _ {c} = v + \delta v,
$$
$$
[ X, Y, Z ] ^ {T} = \mathbf {K} ^ {- 1} [ u _ {c} \cdot z, v _ {c} \cdot z, z ] ^ {T}. \tag {1}
$$
The networks consist of feature extractor  $\mathcal{F}_{\theta}$  and task-specific heads  $\mathcal{G}_{\phi^t}$  that produce the feature maps  $\mathbf{h}$  and the per-pixel output maps  $\tilde{\mathbf{o}}_t$ , where  $t \in \mathbf{T} = \{t_c, t_u, t_v, t_{\delta u}, t_{\delta v}, t_z, t_h, t_w, t_l, t_\gamma\}$ , respectively, as follows:
$$
\mathbf {h} = \mathcal {F} _ {\theta} (\mathbf {I}), \mathcal {F} _ {\theta}: \mathbb {R} ^ {3 \times H \times W} \to \mathbb {R} ^ {C \times H ^ {\prime} \times W ^ {\prime}},
$$
$$
\tilde {\mathbf {o}} _ {t} = \mathcal {G} _ {\phi^ {t}} (\mathbf {h}), \mathcal {G} _ {\phi^ {t}}: \mathbb {R} ^ {C \times H ^ {\prime} \times W ^ {\prime}} \rightarrow \mathbb {R} ^ {H ^ {\prime} \times W ^ {\prime}}, \tag {2}
$$
where  $H, W$  represent the spatial resolution of the image, and  $C, H', W'$  denote the channel and spatial resolution of the feature maps, which is downsampled from the image resolution. The final object-wise results  $\mathbf{o}_{\mathbf{T}}$  are extracted from the non-learnable function  $\mathcal{H}$ , given the intermediate per-pixel outputs  $\tilde{\mathbf{o}}_{\mathbf{T}} = \{t_c, t_u, t_v, t_{\delta u}, t_{\delta v}, t_z, t_h, t_w, t_l, t_\gamma\}$ , as follows:
$$
\mathbf {o} _ {\mathbf {T}} = \mathcal {H} (\tilde {\mathbf {o}} _ {\mathbf {T}}), \mathcal {H}: \mathbb {R} ^ {| \mathbf {T} | \times H ^ {\prime} \times W ^ {\prime}} \rightarrow \mathbb {R} ^ {| \mathbf {T} | \times N}, N \geq 0, \tag {3}
$$
where the number of detected objects  $N$  is not fixed and can be zero if no object is detected.
3.3 Methodology
In this paper, our primary focus is on training the feature extractor  $\mathcal{F}_{\theta}$  with the objective of extracting features that enhance the discriminability of object depth. At the same time, we aim to preserve the discriminability of other sub-tasks, such as the projected 3D center, bounding box size, and heading directions. To achieve this, we propose a metric learning method that encourages the network to extract depth-discriminative features by leveraging object depth labels.
Given  $M$  training images  $\mathbf{I}^i$ , where  $i \in I = \{1, \dots, M\}$ , and  $N^i$  GT objects in the image  $\mathbf{I}^i$ , we extract object descriptors  $\rho^{(i,j)}$ , where  $j \in J = \{1, \dots, N^i\}$ , from feature maps  $\mathbf{h}^i$  using GT coarse projected 3D center  $u^{(i,j)}$ ,  $v^{(i,j)}$  and obtain the corresponding GT object depths  $z^{(i,j)}$ . We build a set of object descriptors  $\mathbf{P} = \bigcup_{\{i \in I, j \in J\}} \rho^{(i,j)}$  and a set of corresponding object depth  $\mathbf{Z} = \bigcup_{\{i \in I, j \in J\}} z^{(i,j)}$ . We then define two metric spaces  $(\mathbf{Z}, d_1)$  and  $(\mathbf{P}, d_2)$ , where  $d_1$  and  $d_2$  represent the Minkowski distance in Euclidean space (i.e., L1 distance). The finite sets  $\mathbf{Z}$  and  $\mathbf{P}$  consist of  $L$  elements, corresponding to objects in the dataset ( $|\mathbf{Z}| = |\mathbf{P}| = L = \sum_{i \in I} N^i$ ). Consequently, we can establish a one-to-one function  $\mathcal{Q}$  between these two metric spaces as follows:
$$
\mathcal {Q} \left(z ^ {l}\right) = \rho^ {l}, \mathcal {Q}: \mathbf {Z} \rightarrow \mathbf {P}, l \in \{1, 2, \dots , L \}. \tag {4}
$$
Our goal is for the function  $\mathcal{Q}$  to enforce a quasi-isometric between  $\mathbf{Z}$ -space and  $\mathbf{P}$ -space by using the properties of quasi-isometry in Sec. 3.1. This can encourage the network to extract depth-discriminative features in  $\mathbf{P}$ -space by utilizing object depth labels in  $\mathbf{Z}$ -space. However, enforcing a quasi-isometric between the low-dimensional  $\mathbf{Z}$ -space and the high-dimensional  $\mathbf{P}$ -space can damage the non-linearity of the natural manifold, potentially causing the negative transfer to other sub-tasks. Therefore, we adopt the local distance-preserving condition of the non-linear embedding methods such as Isomap [33] and LLE [29]. The revised version of the quasi-isometry condition is as follows:
(i)  $\mathbf{U}_{\hat{z}} = \{z\in \mathbf{Z}|z\in \mathcal{B}_{\hat{z},\epsilon}\}$
(ii)  $\forall z_{1}\in \mathbf{Z}$  , s.t.  $\forall z_{2}\in \mathbf{U}_{z_{1}}: \frac{1}{K}\cdot d_{1}(z_{1},z_{2}) - B\leq d_{2}(\mathcal{Q}(z_{1}),\mathcal{Q}(z_{2}))\leq K\cdot d_{1}(z_{1},z_{2}) + B,$
(iii)  $\forall \rho \in \mathbf{P}:\exists z\in \mathbf{Z}$  , s.t.  $d_{2}(\rho ,\mathcal{Q}(z))\leq \epsilon$
Figure 1: Illustration of our quasi-isometric loss  $\mathcal{L}_{qi}$ .
where  $\mathcal{B}_{\hat{z},\epsilon}$  is a ball with radius  $\epsilon$  centered around  $\hat{z}$ . By applying the quasi-isometric properties solely to neighboring samples within the  $\epsilon$ -ball, the non-linearity of the feature manifold is preserved, and the shortest curve distance between any two arbitrary samples on the manifold (a.k.a. geodesic) in  $\mathbf{P}$ -space is maintained instead of the Minkowski distance. By ensuring these three conditions (i), (ii), (iii) are met for a sufficiently small  $K \geq 1$  and  $B \geq 0$ , we can encourage object features  $\rho^l$  to be depth-discriminative using their corresponding object depth  $z^l$ . To achieve this, function  $\mathcal{Q}$  must satisfy these conditions, which involves training the parameter  $\theta$  of the feature extractor  $\mathcal{F}$ .
$(\mathbf{K}, \mathbf{B}, \epsilon)$ -Quasi-isometric loss. We propose a quasi-isometric loss term that enforces the quasi-isometric between the depth and feature metric spaces while preserving the distance among the neighbor data points as illustrated in Fig. 1. In particular, the proposed loss term is designed to arrange  $\mathbf{P}$ -space samples that do not satisfy these conditions. To ensure efficient training within the constraints of limited GPU memory, we use the samples within a mini-batch instead of using the entire input dataset as done in previous works [2, 29, 33]. We selectively choose the neighbor objects in a mini-batch with respect to their corresponding depth labels, where neighboring object depths are within  $\mathbf{U}_z$ .
Suppose that objects in mini-batch images  $\mathbf{I}_b$  correspond to the object depths  $\mathbf{Z}_b$  and object features  $\mathbf{P}_b$  on  $\mathbf{Z}$ -space and  $\mathbf{P}$ -space, respectively. We can then identify the object feature pair sets  $\mathbf{P}_b^+$ ,  $\mathbf{P}_b^-$  that violate the revised property (ii) by transposing it as follow:
$$
\forall \left(\rho_ {1}, \rho_ {2}\right) \in \mathbf {P} _ {b} ^ {+} \subset \mathbf {P} _ {b} \times \mathbf {P} _ {b}, \text {s . t .} | z _ {1}, z _ {2} | \leq \epsilon : d _ {2} \left(\rho_ {1}, \rho_ {2}\right) \not \leq K \cdot d _ {1} \left(z _ {1}, z _ {2}\right) + B, \tag {5a}
$$
$$
\forall \left(\rho_ {1}, \rho_ {2}\right) \in \mathbf {P} _ {b} ^ {-} \subset \mathbf {P} _ {b} \times \mathbf {P} _ {b}, \text {s . t .} | z _ {1}, z _ {2} | \leq \epsilon : \frac {1}{K} \cdot d _ {1} \left(z _ {1}, z _ {2}\right) - B \not \leq d _ {2} \left(\rho_ {1}, \rho_ {2}\right), \tag {5b}
$$
where  $(z_{1},z_{2})$  is the corresponding depth pair of the object features  $(\rho_{1},\rho_{2})$ . For an object feature pair with distance  $d_2(\rho_1,\rho_2)$  in  $\mathbf{P}$ -space, the distance is larger for  $\mathbf{P}_b^+$  in Eq. 5a or smaller for  $\mathbf{P}_b^-$  in Eq. 5b than the corresponding depth pair distance  $d_1(z_1,z_2)$  with a factor of  $K$  and the additive constant  $B$ . To ensure these property-violating object feature pairs  $\mathbf{P}_b^+$ ,  $\mathbf{P}_b^-$  satisfy the property (ii), we propose the  $(K,B,\epsilon)$ -quasi-isometry loss, which modifies the Normalized Temperature-Scaled Cross-Entropy loss (NT-Xent loss) from [6] as follows:
$$
\mathcal {L} _ {q i} = - \frac {1}{| \mathbf {P} _ {b} ^ {+} |} \sum_ {\left(\rho_ {i}, \rho_ {j}\right) \in \mathbf {P} _ {b} ^ {+}} \log \frac {\mathrm {S} ^ {+} \left(\rho_ {i} , \rho_ {j}\right)}{\mathrm {S} ^ {+} \left(\rho_ {i} , \rho_ {j}\right) + \sum_ {\left(\rho_ {k} , \rho_ {l}\right) \in \mathbf {P} _ {b} ^ {-}} \mathrm {S} ^ {-} \left(\rho_ {k} , \rho_ {l}\right)},
$$
$$
\mathrm {S} ^ {+} \left(\rho_ {i}, \rho_ {j}\right) = \exp \left(- \left(\| \rho_ {i}, \rho_ {j} \| _ {p} - K | z _ {i}, z _ {j} | - B\right) / \tau\right), \tag {6}
$$
$$
\mathrm {S} ^ {-} (\rho_ {i}, \rho_ {j}) = \exp (- (\frac {1}{K} | z _ {i}, z _ {j} | - B - \| \rho_ {i}, \rho_ {j} \| _ {p}) / \tau),
$$
Algorithm 1  $(K,B,\epsilon)$ -Quasi-isometric loss
Input:  $\mathbf{h},(u_i,v_i),z_i,$  where  $i\in \{1,2,\dots,n\}$  n is the number of objects in a batch. 1:  $\mathbf{h}_{avg}\gets \mathrm{avg\_pool\_5x5(h)}$  #feature map avg. pool 2: initialize  $\mathbf{P} = []$    
3: for  $\forall i$  do   
4:  $\mathbf{P}[i] = \rho_{i} = \mathbf{h}_{avg}[:,u_{i},v_{i}]$    
5: end for   
6:  $\mathbf{Z}\leftarrow \{z_1,z_2,\ldots ,z_n\}$  that correspond to  $\mathbf{P}$    
7:  $\mathcal{M}_{\mathbf{P}}\leftarrow \left( \begin{array}{cccc} <  \rho_1,\rho_1 > & <  \rho_1,\rho_2 > & \dots & <  \rho_1,\rho_n > \\ \vdots & \vdots & \ddots & \vdots \\ <  \rho_n,\rho_1 > & <  \rho_n,\rho_2 > & \dots & <  \rho_n,\rho_n > \end{array} \right)$  #where  $<  a,b>$  is the Minkowski metric  $\| a,b\| _p$    
8:  $\mathcal{M}_{\mathbf{Z}}\gets \left( \begin{array}{cccc}|z_{1},z_{1}| & |z_{1},z_{2}| & \dots & |z_{1},z_{n}|\\ \vdots & \vdots & \ddots & \vdots \\ |z_{n},z_{1}| & |z_{n},z_{2}| & \dots & |z_{n},z_{n}| \end{array} \right)$    
9:  $\mathcal{M}^{+}\gets \mathcal{M}_{\mathbf{P}} - K\mathcal{M}_{\mathbf{Z}} - B$  # To find property-violated object features. (Eq. 5a)   
10:  $\mathcal{M}^{-}\gets \frac{1}{K}\mathcal{M}_{\mathbf{Z}} - \mathcal{M}_{\mathbf{P}} - B$  # To find property-violated object features. (Eq. 5b)   
11:  $(\mathcal{M}^{+})_{ij}\gets 0$  , where  $|z_i,z_j| > \epsilon$  or  $i\geq j$  or  $(\mathcal{M}^{+})_{ij} <   0$    
12:  $(\mathcal{M}^{-})_{ij}\gets 0$  , where  $|z_i,z_j| > \epsilon$  or  $i\geq j$  or  $(\mathcal{M}^{-})_{ij} <   0$    
13: ancsc  $^+\gets \exp (-\mathcal{M}^{+} / \tau)$  #  $\tau$  is temperature term.   
14: ancsc  $^-\gets \sum (\exp (-\mathcal{M}^{-} / \tau))$    
15: Lqi= mean(- log(ancs+/ancs+ + anc5+ + d))) #  $\delta = 1e - 12$  : div. assert
where  $\tau$  is the temperature term, and  $\| \cdot ,\cdot \| _p$  is  $p$  -norm, and  $K\geq 1,B\geq 0$  denote the pre-defined hyperparameters that determine the hardness of quasi-isometric property. This loss aims to make the property-violated object features comply with the quasi-isometric property by assuming  $\mathbf{P}_b^{+ / - }$  as positive/negative anchors, respectively. The similarity metrics  $\mathrm{S}^{+ / - }$  are always positive values in (0, 1] that imply a negative distance gap between the distance metric on  $\mathbf{Z}$  -space and  $\mathbf{P}$  -space. These adjustments are necessary for proper alignment between the two spaces, noting that  $\mathbf{P}_b$  is derived from  $\mathcal{F}_{\theta}(\mathbf{I}_b)$ .