Original: Coded Residual Transform for Generalizable Deep Metric Learning
Revised: Coded Residual Transform for Generalizable Deep Metric Learning

Original: s
Revised: s

Original: Abstract
Revised: Abstract

Original: A fundamental challenge in deep metric learning is the generalization capabilityof the feature embedding network model since the embedding network learned ontraining classes need to be evaluated on new test classes.
Revised: A fundamental challenge in deep metric learning is the generalization capabilityof the feature embedding network model since the embedding network learned ontraining classes need to be evaluated on new test classes.

Original: To address this challenge,in this paper, we introduce a new method called coded residual transform (CRT) fordeep metric learning to signiﬁcantly improve its generalization capability.
Revised: To address this challenge,in this paper, we introduce a new method called coded residual transform (CRT) fordeep metric learning to signiﬁcantly improve its generalization capability.

Original: Specifically, we learn a set of diversiﬁed anchor features, project the feature map ontoeach anchor, and then encode its features using their projection residuals weightedby their correlation coefﬁcients with each anchor.
Revised: Specifically, we learn a set of diversiﬁed anchor features, project the feature map ontoeach anchor, and then encode its features using their projection residuals weightedby their correlation coefﬁcients with each anchor.

Original: The proposed CRT method hasthe following two unique characteristics.
Revised: The proposed CRT method hasthe following two unique characteristics.

Original: First, it represents and encodes the featuremap from a set of complimentary perspectives based on projections onto diversiﬁed anchors.
Revised: First, it represents and encodes the featuremap from a set of complimentary perspectives based on projections onto diversiﬁed anchors.

Original: Second, unlike existing transformer-based feature representationapproaches which encode the original values of features based on global correlationanalysis, the proposed coded residual transform encodes the relative differences between the original features and their projected anchors.
Revised: Second, unlike existing transformer-based feature representationapproaches which encode the original values of features based on global correlationanalysis, the proposed coded residual transform encodes the relative differences between the original features and their projected anchors.

Original: Embedding space density and spectral decay analysis show that this multi-perspective projection onto diversiﬁed anchors and coded residual representation are able to achieve signiﬁcantlyimproved generalization capability in metric learning.
Revised: Embedding space density and spectral decay analysis show that this multi-perspective projection onto diversiﬁed anchors and coded residual representation are able to achieve signiﬁcantlyimproved generalization capability in metric learning.

Original: Finally, to further enhancethe generalization performance, we propose to enforce the consistency on theirfeature similarity matrices between coded residual transforms with different sizesof projection anchors and embedding dimensions.
Revised: Finally, to further enhancethe generalization performance, we propose to enforce the consistency on theirfeature similarity matrices between coded residual transforms with different sizesof projection anchors and embedding dimensions.

Original: Our extensive experimentalresults and ablation studies demonstrate that the proposed CRT method outperformthe state-of-the-art deep metric learning methods by large margins and improvingupon the current best method by up to 4.28% on the CUB dataset.
Revised: Our extensive experimentalresults and ablation studies demonstrate that the proposed CRT method outperformthe state-of-the-art deep metric learning methods by large margins and improvingupon the current best method by up to 4.28% on the CUB dataset.

Original:  Introduction
Revised:  Introduction

Original: Deep metric learning (DML) aims to learn effective features to characterize or represent images, whichhas important applications in image retrieval [1, 2], image recognition [3], person re-identiﬁcation [4],image segmentation [5] and tracking [6].
Revised: Deep metric learning (DML) aims to learn effective features to characterize or represent images, whichhas important applications in image retrieval [1, 2], image recognition [3], person re-identiﬁcation [4],image segmentation [5] and tracking [6].

Original: Successful metric learning needs to achieve the followingtwo objectives: (1) Discriminative .
Revised: Successful metric learning needs to achieve the followingtwo objectives: (1) Discriminative .

Original: In the embedded feature space, image features with the samesemantic labels should be aggregated into compact clusters in the high-dimensional feature spacewhile those from different classes should be well separated from each other.
Revised: In the embedded feature space, image features with the samesemantic labels should be aggregated into compact clusters in the high-dimensional feature spacewhile those from different classes should be well separated from each other.

Original: (2) Generalizable .
Revised: (2) Generalizable .

Original: The learned features should be able to generalize well from the training images to test images ofnew classes which have not been seen before.
Revised: The learned features should be able to generalize well from the training images to test images ofnew classes which have not been seen before.

Original: During the past a few years, methods based ondeep neural networks, such as metric loss functions design [7, 8, 9, 10, 11], embedding transfer[12, 13, 14, 15, 16], structural matching [17], graph neural networks [2, 18], language guidance
Revised: During the past a few years, methods based ondeep neural networks, such as metric loss functions design [7, 8, 9, 10, 11], embedding transfer[12, 13, 14, 15, 16], structural matching [17], graph neural networks [2, 18], language guidance

Original: [19],and vision transformer [1, 20, 21], have achieved remarkable progress on learning discriminativefeatures.
Revised: [19],and vision transformer [1, 20, 21], have achieved remarkable progress on learning discriminativefeatures.

Original: However, the generalization onto unseen new classes remains a signiﬁcant challenge forexisting deep metric learning methods.
Revised: However, the generalization onto unseen new classes remains a signiﬁcant challenge forexisting deep metric learning methods.

Original: In the literature, to improve the deep metric learning performance and alleviate the generalizationproblem on unseen classes, regularization techniques [22, 15], language-guided DML [19], andfeature fusion [2, 8, 18] methods have been developed.
Revised: In the literature, to improve the deep metric learning performance and alleviate the generalizationproblem on unseen classes, regularization techniques [22, 15], language-guided DML [19], andfeature fusion [2, 8, 18] methods have been developed.

Original: Existing approaches to addressing thegeneralization challenge in metric learning focus on the robustness of linear or kernel-based distancemetrics [23, 24], analysis of error bounds of the generalization process [25], and correlation analysisbetween generalization and structure characteristics of the learned embedding space [22].
Revised: Existing approaches to addressing thegeneralization challenge in metric learning focus on the robustness of linear or kernel-based distancemetrics [23, 24], analysis of error bounds of the generalization process [25], and correlation analysisbetween generalization and structure characteristics of the learned embedding space [22].

Original: It shouldbe noted that in existing methods, the input image is analyzed and transformed as a whole into anembedded feature.
Revised: It shouldbe noted that in existing methods, the input image is analyzed and transformed as a whole into anembedded feature.

Original: In other words, the image is represented and projected globally from a singleperspective.
Revised: In other words, the image is represented and projected globally from a singleperspective.

Original: We recognize that this single-perspective projection is not able to represent and encodethe highly complex and dynamic correlation structures in the high-dimensional feature space sincethey are being collapsed and globally projected onto one single perspective and the local correlationdynamics have been suppressed.
Revised: We recognize that this single-perspective projection is not able to represent and encodethe highly complex and dynamic correlation structures in the high-dimensional feature space sincethey are being collapsed and globally projected onto one single perspective and the local correlationdynamics have been suppressed.

Original: According to our experiments, this single-perspective globalprojection will increase the marginal variance [26] and consequently degrade the generalizationcapability of the deep metric learning method.
Revised: According to our experiments, this single-perspective globalprojection will increase the marginal variance [26] and consequently degrade the generalizationcapability of the deep metric learning method.

Original: Furthermore, we observe that existing deep metric learning methods attempt to transform and encodethe original features.
Revised: Furthermore, we observe that existing deep metric learning methods attempt to transform and encodethe original features.

Original: From the generalization point of view, we ﬁnd that it is more effective to learnthe embedding based on relative difference between features since the absolute value of featuresmay vary signiﬁcantly from the training to new test classes, but the relative change patterns betweenfeatures may remain largely invariant.
Revised: From the generalization point of view, we ﬁnd that it is more effective to learnthe embedding based on relative difference between features since the absolute value of featuresmay vary signiﬁcantly from the training to new test classes, but the relative change patterns betweenfeatures may remain largely invariant.

Original: To further understand this idea, consider the following toyexample: a face in the daytime may appear much different from a face in the night time due tochanges in lighting conditions.
Revised: To further understand this idea, consider the following toyexample: a face in the daytime may appear much different from a face in the night time due tochanges in lighting conditions.

Original: However, an effective face detection with sufﬁcient generalizationpower will not focus on the absolute pixel values of the face image.
Revised: However, an effective face detection with sufﬁcient generalizationpower will not focus on the absolute pixel values of the face image.

Original: Instead, it detects the face basedon the relative change patterns between neighboring regions inside the face image.
Revised: Instead, it detects the face basedon the relative change patterns between neighboring regions inside the face image.

Original: Motivated by thisobservation, in this work, to address the generalization challenge, we propose to learn the embeddedfeature from the projection residuals of the feature map, instead of its absolute features.
Revised: Motivated by thisobservation, in this work, to address the generalization challenge, we propose to learn the embeddedfeature from the projection residuals of the feature map, instead of its absolute features.

Original: It shouldbe pointed out that the projection anchors used to produce the projection residuals may different fortraining classes and test classes, we learn those projection anchors based on the training classes andtransfer them into the test classes.

Original: In order to increase the generalization ability of the projectionresiduals, we compute adaptive weights between anchors and features to weight projection residualsof the test set.

Original: The above two ideas, namely, multi-perspective projection and residual encoding, lead to our proposedmethod of coded residual transform for deep metric learning.
Revised: The above two ideas, namely, multi-perspective projection and residual encoding, lead to our proposedmethod of coded residual transform for deep metric learning.

Original: Speciﬁcally, we propose to learn a setof diversiﬁed anchor features, project the features onto every anchor, and then encode the featuresusing the projection residuals weighted by their correlation coefﬁcients with the target anchor.
Revised: Speciﬁcally, we propose to learn a setof diversiﬁed anchor features, project the features onto every anchor, and then encode the featuresusing the projection residuals weighted by their correlation coefﬁcients with the target anchor.

Original: Unlikeexisting transformer-based feature representation approaches which encode the original values offeatures based on global correlation analysis [27, 28, 29], the proposed coded residual transformencodes the relative differences between original features and their projected anchors.
Revised: Unlikeexisting transformer-based feature representation approaches which encode the original values offeatures based on global correlation analysis [27, 28, 29], the proposed coded residual transformencodes the relative differences between original features and their projected anchors.

Original: Our extensiveexperimental results and ablation studies demonstrate that the proposed CRT method is able tosigniﬁcantly improve the generalization performance of deep metric learning, outperforming thestate-of-the-art methods by large margins and improving upon the current best method by up to4.28%.
Revised: Our extensiveexperimental results and ablation studies demonstrate that the proposed CRT method is able tosigniﬁcantly improve the generalization performance of deep metric learning, outperforming thestate-of-the-art methods by large margins and improving upon the current best method by up to4.28%.

Revised: We learn those projection anchors based on the training classes and transfer them into the test classes.

Original: Related Work and Unique Contributions
Revised: Although the training and test classes share the same set of anchors, the actual distributions of projectanchors during training and testing could be much different due to the distribution shift betweenthe training and testing classes. During our coded residual transform, we assign different weightsfor different project residuals based on the correlation between the feature and the correspondinganchor. Therefore, for training classes, the subset of anchors which are close to the training imageswill have larger weights. Similarly, for the testing classes, the subset of anchors which are close tothe test images will have larger weights.

Revised: This correlation-based weighting for the projection residualcontribute signiﬁcantly to the overall performance gain.

Revised:  Related Work and Unique Contributions

Original: This work is related to deep metric learning, transformer-based learning methods, and residualencoding.
Revised: This work is related to deep metric learning, transformer-based learning methods, and residualencoding.

Original: In this section, we review the existing methods on these topics and discuss the uniquenovelty of our approach.
Revised: In this section, we review the existing methods on these topics and discuss the uniquenovelty of our approach.

Original:  Deep metric learning
Revised: (1) Deep metric learning.

Original: Deep metric learning aims to learn discriminative features with the goal tominimize intra-class sample distance and maximize inter-class sample distance in a contrastive manner.
Revised: Deep metric learning aims to learn discriminative features with the goal tominimize intra-class sample distance and maximize inter-class sample distance in a contrastive manner.

Original: Contrastive loss [30] has been successfully used in early methods of deep metric learning, aimingto optimize pairwise distance of between samples.
Revised: Contrastive loss [30] has been successfully used in early methods of deep metric learning, aimingto optimize pairwise distance of between samples.

Original: By exploring more sophisticated relationshipbetween samples, a variety of metric loss functions, such as triplet loss [31], lifted structured loss [32],proxy-anchor loss [9], and multi-similarity (MS) loss [11], have been developed.
Revised: By exploring more sophisticated relationship between samples, a variety of metric loss functions, such as triplet loss [31], lifted structured loss [32],proxy-anchor loss [9], and multi-similarity (MS) loss [11], have been developed.

Original: According to thestudies in [22] and [33], the MS loss was veriﬁed to be one of the most efﬁcient metric loss functions.
Revised: According to the studies in [22] and [33], the MS loss was veriﬁed to be one of the most efﬁcient metric loss functions.

Original: Some recent methods explore how to use multiple features to learn robust feature embeddings.
Revised: Some recent methods explore how to use multiple features to learn robust feature embeddings.

Original: Kanet al.
Revised: Kanet al.

Original: [2] and Seidenschwarz et al.
Revised: [2] and Seidenschwarz et al.

Original: [18] adopt K-nearest neighbors (k-NN) of an anchor image tobuild local graph neural network (GNN) and reﬁne embedding vectors based on message exchanges between the graph nodes.
Revised: [18] adopt K-nearest neighbors (k-NN) of an anchor image tobuild local graph neural network (GNN) and reﬁne embedding vectors based on message exchangesbetween the graph nodes.

Original: Zhao et al.
Revised: Zhao et al.

Original: [17] proposed a structural matching method to learn a metricfunction between feature maps based on the optimal transport theory.
Revised: [17] proposed a structural matching method to learn a metricfunction between feature maps based on the optimal transport theory.

Original: Based on the form of softmax,margin-based softmax loss functions [34, 35, 36] were also proposed to learn discriminative features.
Revised: Based on the form of softmax,margin-based softmax loss functions [34, 35, 36] were also proposed to learn discriminative features.

Original: Sohn [37] improved the contrastive loss and triplet loss by introducing N negative examples andproposed the N-pair loss function to speed up model convergence during training.(2)
Revised: Sohn [37] improved the contrastive loss and triplet loss by introducing N negative examples andproposed the N-pair loss function to speed up model convergence during training.(2)

Original: Transformer-based learning methods.
Revised: Transformer-based learning methods.

Original: This work is related to transformer-based learningmethods since our method also analyze the correlation between features and uses this correlationinformation to aggregate features.
Revised: This work is related to transformer-based learningmethods since our method also analyze the correlation between features and uses this correlationinformation to aggregate features.

Original: The original work of transformer [27] aims to learn a self-attentionfunction and a feed forward transformation network for nature language processing.
Revised: The original work of transformer [27] aims to learn a self-attentionfunction and a feed forward transformation network for nature language processing.

Original: Recently, ithas been successfully applied to computer vision and image processing.
Revised: Recently, ithas been successfully applied to computer vision and image processing.

Original: ViT [38] demonstrates thatthat a pure transformer can achieve state-of-the-art performance in image classiﬁcation.
Revised: ViT [38] demonstrates thatthat a pure transformer can achieve state-of-the-art performance in image classiﬁcation.

Original: ViT treatseach image as a sequence of tokens and then feeds them to multiple transformer layers to performthe classiﬁcation.
Revised: ViT treatseach image as a sequence of tokens and then feeds them to multiple transformer layers to performthe classiﬁcation.

Original: Subsequently, DeiT [39] further explores a data-efﬁcient training strategy and adistillation approach for ViT. More recent methods such as T2T ViT [29], TNT [40], CrossViT [41]and LocalViT [42] further improve the ViT method for image classiﬁcation.
Revised: Subsequently, DeiT [39] further explores a data-efﬁcient training strategy and adistillation approach for ViT. More recent methods such as T2T ViT [29], TNT [40], CrossViT [41]and LocalViT [42] further improve the ViT method for image classiﬁcation.

Original: PVT [43] incorporates apyramid structure into the transformer for dense prediction tasks.
Revised: PVT [43] incorporates apyramid structure into the transformer for dense prediction tasks.

Original: After that, methods such as Swin[28], CvT [44], CoaT [45], LeViT [46], Twins [47] and MiT [48] enhance the local continuity offeatures and remove ﬁxed size position embedding to improve the performance of transformers fordense prediction tasks.
Revised: After that, methods such as Swin[28], CvT [44], CoaT [45], LeViT [46], Twins [47] and MiT [48] enhance the local continuity offeatures and remove ﬁxed size position embedding to improve the performance of transformers fordense prediction tasks.

Original: For deep metric learning, El-Nouby et al.
Revised: For deep metric learning, El-Nouby et al.

Original: [1] and Ermolov et al.
Revised: [1] and Ermolov et al.

Original: [20] adoptthe DeiT-S network[39] as a backbone to extract features, achieving impressive performance.(3)
Revised: [20] adoptthe DeiT-S network[39] as a backbone to extract features, achieving impressive performance.(3)

Original: Residual encoding.
Revised: Residual encoding.

Original: Residual encoding was ﬁrst proposed by Jégou et al.
Revised: Residual encoding was ﬁrst proposed by Jégou et al.

Original: [49], where the vectorof locally aggregated descriptors (VLAD) algorithm is used to aggregate the residuals betweenfeatures and their best-matching codewords.
Revised: [49], where the vectorof locally aggregated descriptors (VLAD) algorithm is used to aggregate the residuals betweenfeatures and their best-matching codewords.

Original: Based on the VLAD method, VLAD-CNN [50] hasdeveloped the residual encoders for visual recognition and understanding tasks.
Revised: Based on the VLAD method, VLAD-CNN [50] hasdeveloped the residual encoders for visual recognition and understanding tasks.

Original: NetVLAD
Revised: NetVLAD

Original: [51]and Deep-TEN [52] extend this idea and develop an end-to-end learnable residual encoder basedon soft-assignment.
Revised: [51]and Deep-TEN [52] extend this idea and develop an end-to-end learnable residual encoder basedon soft-assignment.

Original: It should be noted that features learned by these methods typically have verylarge sizes, for example, 16k, 32k and 4096 for AlexNet [53], VGG-16 [54] and ResNet-50 [55],respectively.(4)
Revised: It should be noted that features learned by these methods typically have verylarge sizes, for example, 16k, 32k and 4096 for AlexNet [53], VGG-16 [54] and ResNet-50 [55],respectively.(4)

Original: Unique Contributions.
Revised: Unique Contributions.

Original: Compared to the above existing methods, the unique contributions ofthis paper can be summarized as follows: (1) We introduce a new CRT method which learns a set ofanchor features, project the feature map onto each anchor, and then encode its features using theirprojection residuals weighted by their correlation coefﬁcients with each anchor.
Revised: Compared to the above existing methods, the unique contributions ofthis paper can be summarized as follows: (1) We introduce a new CRT method which learns a set ofanchor features, project the feature map onto each anchor, and then encode its features using theirprojection residuals weighted by their correlation coefﬁcients with each anchor.

Original: (2) We introduce adiversity constraint for the set of anchor features so that the CRT method can represent and encodethe feature map from a set of complimentary perspectives.
Revised: (2) We introduce adiversity constraint for the set of anchor features so that the CRT method can represent and encodethe feature map from a set of complimentary perspectives.

Original: Unlike existing transformer-based featurerepresentation approaches which encode the original values of features based on global correlationanalysis, the proposed coded residual transform encode the relative differences between originalfeatures and their projected anchors.
Revised: Unlike existing transformer-based featurerepresentation approaches which encode the original values of features based on global correlationanalysis, the proposed coded residual transform encode the relative differences between originalfeatures and their projected anchors.

Original: (3) To further enhance the generalization performance, wepropose to enforce the feature distribution consistency between coded residual transforms withdifferent sizes of projection anchors and embedding dimensions.
Revised: (3) To further enhance the generalization performance, wepropose to enforce the feature distribution consistency between coded residual transforms withdifferent sizes of projection anchors and embedding dimensions.

Original: (4) We demonstrate that this multiperspective projection with diversiﬁed anchors and coded residual representation based on relativedifferences are able to achieve signiﬁcantly improved generalization capability in metric learning.
Revised: (4) We demonstrate that this multiperspective projection with diversiﬁed anchors and coded residual representation based on relativedifferences are able to achieve signiﬁcantly improved generalization capability in metric learning.

Original: Our proposed CRT method outperforms the state-of-the-art methods by large margins and improvingupon the baseline method by up to 4.28%.
Revised: Our proposed CRT method outperforms the state-of-the-art methods by large margins and improvingupon the baseline method by up to 4.28%.

Original:  Method
Revised:  Method

Original: In the following sections, we present our proposed method of coded residual transform for deepmetric learning.
Revised: In the following sections, we present our proposed method of coded residual transform for deepmetric learning.

Original: 3.1 Method Overview
Revised: 3.1 Method Overview

Original: Figure 1 provides an overview of the proposed CRT method.
Revised: Figure 1 provides an overview of the proposed CRT method.

Original: We ﬁrst use a backbone network Φ toencode the input image and extract its feature map
Revised: We ﬁrst use a backbone network Φ toencode the input image and extract its feature map

Original: X ∈ R H
Revised: X ∈ R H

Original: × W × L , which consists of H × W featurevectors of size L .
Revised: × W × L , which consists of H × W feature vectors of size L .

Original: To improve the generalization capability of deep metric learning on feature map,we learn an anchor network to generate a set of diversiﬁed anchor features { c 1 , c 2 , · · · , c K } fromthe training set.
Revised: To improve the generalization capability of deep metric learning on feature map,we learn an anchor network to generate a set of diversiﬁed anchor features { c 1 , c 2 , · · · , c K } fromthe training set.

Original: Here, by "diversiﬁed" we mean these anchors are able to provide multi-perspectivecomplimentary representation of image features.
Revised: Here, by "diversiﬁed" we mean these anchors are able to provide multi-perspectivecomplimentary representation of image features.

Original: We then project all feature vectors in the feature map onto each anchor c k and encode them using the projection residuals weighted by their correlationcoefﬁcients with respect to this anchor.
Revised: We then project all feature vectors in the featuremap onto each anchor c k and encode them using the projection residuals weighted by their correlationcoefﬁcients with respect to this anchor.

Original: Each anchor c k will generate a coded residual representationr k of the feature map.
Revised: Each anchor c k will generate a coded residual representationr k of the feature map.

Original: After being further processed by a nonlinear embedding network Γ k , thesecoded residual representations will be added together to form the ﬁnal nonlinear feature embedding.
Revised: After being further processed by a nonlinear embedding network Γ k , thesecoded residual representations will be added together to form the ﬁnal nonlinear feature embedding.

Original: The above feature embedding process is referred to as coded residual transform .
Revised: The above feature embedding process is referred to as coded residual transform .

Original: To further enhancethe generalization performance, we introduce a consistency constraint between two different codedresidual transform with different values of K which is the size of the anchor set.
Revised: To further enhancethe generalization performance, we introduce a consistency constraint between two different codedresidual transform with different values of K which is the size of the anchor set.

Original: In the followingsections, we will explain the above major components of our CRT method in more details.
Revised: In the followingsections, we will explain the above major components of our CRT method in more details.

Original: 3.2 Multi-Perspective Projection and Correlation-Based Residual Encoding
Revised: 3.2 Multi-Perspective Projection and Correlation-Based Residual Encoding

Original: Given an input image I , a backbone network Φ is used to encode image I into a feature map
Revised: Given an input image I , a backbone network Φ is used to encode image I into a feature map

Original: X ∈ R
Revised: X ∈ R

Original: H
Revised: H

Original: × W × L , i.e., X =
Revised: × W × L , i.e., X =

Original: Φ( I ) .
Revised: Φ( I ) .

Original: First, we learn anchor network to generate a set of diversiﬁed anchorfeatures C
Revised: First, we learn anchor network to generate a set of diversiﬁed anchorfeatures C

Original: = { c 1 , c 2 , · · · , c K } , c k ∈ R L , whose learning process will be explained in the followingsection.
Revised: = { c 1 , c 2 , · · · , c K } , c k ∈ R L , whose learning process will be explained in the followingsection.

Original: For each anchor c k , we project the whole feature map X onto this anchor.
Revised: For each anchor c k , we project the whole feature map X onto this anchor.

Original: Speciﬁcally, foreach feature x j ∈ X inside the feature map, we ﬁnd the relative difference
Revised: Speciﬁcally, foreach feature x j ∈ X inside the feature map, we ﬁnd the relative difference

Original: d kj = x
Revised: d kj = x

Original: j − c
Revised: j − c

Original: k
Revised: k

Original: betweenthe feature x j and the anchor c k .
Revised: betweenthe feature x j and the anchor c k .

Original: Let w kj = x j · c
Revised: Let w kj = x j · c

Original: Tk be their correlation coefﬁcient.
Revised: Tk be their correlation coefﬁcient.

Original: Then, withanchor c k , the whole map is transformed into the following feature vector using weighted summationof d kj :
Revised: Then, withanchor c k , the whole map is transformed into the following feature vector using weighted summation [Figure] of d kj :

Original: [Equation]
Revised: [Equation]

Original: where the nonlinear function log[1+exp( w kj )] converts the correlation coefﬁcient w kj into a positivenumber.
Revised: where the nonlinear function log[1+exp( w kj )] converts the correlation coefﬁcient w kj into a positivenumber.

Original: In this way, each anchor c k will generate a separate coded residual representation r k ofthe feature map for the input image.
Revised: In this way, each anchor c k will generate a separate coded residual representation r k ofthe feature map for the input image.

Original: Finally, r k is further transformed by a nonlinear embeddingnetwork Γ k with multiple fully connected layers and one GELU nonlinear activation layer [56].
Revised: Finally, r k is further transformed by a nonlinear embeddingnetwork Γ k with multiple fully connected layers and one GELU nonlinear activation layer [56].

Original: Byaggregating those nonlinear embeddings, the ﬁnal embedded feature f is then given by
Revised: Byaggregating those nonlinear embeddings, the ﬁnal embedded feature f is then given by

Original: [Equation]
Revised: [Equation]

Original: The task of the anchor network is to generate an optimized set of anchors.
Revised: The task of the anchor network is to generate an optimized set of anchors.

Original: Its input is the featuresfrom the feature map.
Revised: Its input is the featuresfrom the feature map.

Original: Its weights are the anchors that need to be learned, and its output is thecorrelations between the input features and the weights, which are the correlations between thefeature map and anchor.
Revised: Its weights are the anchors that need to be learned, and its output is thecorrelations between the input features and the weights, which are the correlations between thefeature map and anchor.

Original: The anchor network will update these anchors using stochastic gradientdescent (SGD) algorithm in back propagation.
Revised: The anchor network will update these anchors using stochastic gradientdescent (SGD) algorithm in back propagation.

Original: In this work, we use an one-layer fully connectednetwork to implement this anchor network.
Revised: In this work, we use an one-layer fully connectednetwork to implement this anchor network.

Original: The regularization of anchors will be discussed in thefollowing section.
Revised: The regularization of anchors will be discussed in thefollowing section.

Original: Figure 2 shows two sets of examples of correlation map between different anchors and the featuremap.
Revised: Figure 2 shows two sets of examples of correlation map between different anchors and the featuremap.

Original: Each image corresponds to one anchor.
Revised: Each image corresponds to one anchor.

Original: The heat map shows the correlation between the anchorand the feature at the corresponding spatial location from the feature map.
Revised: The heat map shows the correlation between the anchor and the feature at the corresponding spatial location from the feature map.

Original: We can see that differentanchors are capturing different semantic components of the image and the feature map is projectedand encoded from a large set of different semantic perspectives.
Revised: We can see that differentanchors are capturing different semantic components of the image and the feature map is projectedand encoded from a large set of different semantic perspectives.

Original: Our generalization analysis andablation studies will demonstrate that this multi-perspective projection and encoding will providesigniﬁcantly enhanced generalization capability and improved metric learning performance.
Revised: Our generalization analysis andablation studies will demonstrate that this multi-perspective projection and encoding will providesigniﬁcantly enhanced generalization capability and improved metric learning performance.

Original: 3.3 Network Model Learning and Generalization Analysis
Revised: 3.3 Network Model Learning and Generalization Analysis

Original: As illustrated in Figure 1, our proposed CRT method involves the encoder network Φ , the anchornetwork A , and the multi-perspective feature embedding networks { Γ k } .
Revised: As illustrated in Figure 1, our proposed CRT method involves the encoder network Φ , the anchornetwork A , and the multi-perspective feature embedding networks { Γ k } .

Original: These networks are jointlytrained in an end-to-end manner.
Revised: These networks are jointlytrained in an end-to-end manner.

Original: The task of the anchor network A is to generate a set of anchorsfor efﬁcient projection of the input features.
Revised: The task of the anchor network A is to generate a set of anchorsfor efﬁcient projection of the input features.

Original: From our experiments, we ﬁnd that it is important toenforce diversity among these anchors so that they can provide a diversiﬁed and robust representationof the image features.
Revised: From our experiments, we ﬁnd that it is important toenforce diversity among these anchors so that they can provide a diversiﬁed and robust representationof the image features.

Original: Diversity means that the anchor features have small correlation with eachother.
Revised: Diversity means that the anchor features have small correlation with eachother.

Original: Let C
Revised: Let C

Original: = { c 1 , c 2 , · · · , c K } be set of anchors generated by the anchor network.
Revised: = { c 1 , c 2 , · · · , c K } be set of anchors generated by the anchor network.

Original: When trainingthe anchor network, we need to minimize the following average correlation between these anchorfeatures, which is deﬁned to be the anchor diversity loss
Revised: When trainingthe anchor network, we need to minimize the following average correlation between these anchorfeatures, which is deﬁned to be the anchor diversity loss

Original: [Equation]
Revised: [Equation]

Original: (3) where the cosine similarity is used to measure the correlation between two different anchor features.
Revised: (3) where the cosine similarity is used to measure the correlation between two different anchor features.

Original: To train the overall feature embedding network, we use the multi-similarity (MS) loss [11], denotedby L MS , as our metric loss function to minimize the similarity for negative pairs and maximize thesimilarity for positive pairs.
Revised: To train the overall feature embedding network, we use the multi-similarity (MS) loss [11], denotedby L MS , as our metric loss function to minimize the similarity for negative pairs and maximize thesimilarity for positive pairs.

Original: Here, negative pairs are samples from different classes while positivepairs are samples from the same class.
Revised: Here, negative pairs are samples from different classes while positivepairs are samples from the same class.

Original: The anchor diversity loss L DIV and the MS loss L MS arecombined to form the overall loss function L =
Revised: The anchor diversity loss L DIV and the MS loss L MS arecombined to form the overall loss function L =

Original: L DIV + λ 1 L MS for end-to-end training of our CRTnetwork.
Revised: L DIV + λ 1 L MS for end-to-end training of our CRTnetwork.

Original: Cross CRT consistency loss.
Revised: Cross CRT consistency loss.

Original: To further enhance the generalization performance, we propose toenforce the feature distribution consistency between coded residual transforms with different sizesof projection anchors.
Revised: To further enhance the generalization performance, we propose toenforce the feature distribution consistency between coded residual transforms with different sizesof projection anchors.

Original: As shown in Figure 3, we construct two CRT embedding branches withdifferent number of projection anchors.
Revised: As shown in Figure 3, we construct two CRT embedding branches withdifferent number of projection anchors.

Original: For example, in our experiment, the ﬁrst branch has ananchor set size K 1 = 48 with an embedded feature size of 128, while the second branch has ananchor set size K 2 = 64 with an embedded feature size of 1024.
Revised: For example, in our experiment, the ﬁrst branch has ananchor set size K 1 = 48 with an embedded feature size of 128, while the second branch has ananchor set size K 2 = 64 with an embedded feature size of 1024.

Original: Suppose that the current trainingbatch of images are I
Revised: Suppose that the current trainingbatch of images are I

Original: = { I 1 , I 2 , · · · , I N } .
Revised: = { I 1 , I 2 , · · · , I N } .

Original: Let the feature embeddings of the ﬁrst CRT branch be{ f (1)1 , f (1)2 , · · · , f (1) N } and the feature embeddings of the second branch be { f (2)1 , f (2)2 , · · · , f (2) N } .
Revised: Let the feature embeddings of the ﬁrst CRT branch be{ f (1)1 , f (1)2 , · · · , f (1) N } and the feature embeddings of the second branch be { f (2)1 , f (2)2 , · · · , f (2) N } .

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: Deﬁne the similarity matrix
Revised: Deﬁne the similarity matrix

Original: [Equation]
Revised: [Equation]

Original: Similarly, we can deﬁne the similarity matrix for the second branch as S (2) .
Revised: Similarly, we can deﬁne the similarity matrix for the second branch as S (2) .

Original: The consistency betweenthese two CRT branches are deﬁned to be the L 1 distance between these two similarity matrices
Revised: The consistency betweenthese two CRT branches are deﬁned to be the L 1 distance between these two similarity matrices

Original: [Equation]
Revised: [Equation]

Original: This consistency loss is then combined with the original loss function for the CRT network.
Revised: This consistency loss is then combined with the original loss function for the CRT network.

Original: Thus, theoverall loss is given by L =
Revised: Thus, theoverall loss is given by L =

Original: L DIV + λ 1 L MS + λ 2 · L CON .
Revised: L DIV + λ 1 L MS + λ 2 · L CON .

Original: Generalization capability analysis.
Revised: Generalization capability analysis.

Original: In this section, we analyze the generalization capability of ourproposed CRT method.
Revised: In this section, we analyze the generalization capability of ourproposed CRT method.

Original: Roth et al.
Revised: Roth et al.

Original: [22] established two metrics to measure the generalization capability of deep metric learning methods: (1) embedding space density and (2) spectral decay.
Revised: [22] established two metrics to measure the generalization capability of deep metric learning methods: (1) embedding space density and (2) spectral decay.

Original: Methodswith higher embedding space density and smaller spectral decay often have better generalizationcapabilities.
Revised: Methodswith higher embedding space density and smaller spectral decay often have better generalizationcapabilities.

Original: The embedding space density metric D ESD is deﬁned as the ratio between the average ofintra-class distance D Intra and the average of inter-class distance D Inter :
Revised: The embedding space density metric D ESD is deﬁned as the ratio between the average ofintra-class distance D Intra and the average of inter-class distance D Inter :

Original: [Equation]
Revised: [Equation]

Original: The spectral decay metric ρ SD is deﬁned to be the KL-divergence between the spectrum of d singularvalues V SV (obtained from Singular Value Decomposition, SVD) and a d -dimensional uniformdistribution µ d .
Revised: The spectral decay metric ρ SD is deﬁned to be the KL-divergence between the spectrum of d singularvalues V SV (obtained from Singular Value Decomposition, SVD) and a d -dimensional uniformdistribution µ d .

Original: It is inversely related to the entropy of the embedding space:
Revised: It is inversely related to the entropy of the embedding space:

Original: [Equation]
Revised: [Equation]

Original: Table 1 and Table 2 show the embedding space density and the spectral decay values of the featureembedding learned by the baseline method and our CRT method on the test datasets used in ourexperiments.
Revised: Table 1 and Table 2 show the embedding space density and the spectral decay values of the featureembedding learned by the baseline method and our CRT method on the test datasets used in ourexperiments.

Original: We can see that our method achieves signiﬁcantly higher embedding space density andmuch lower spectral decay when the model tested on unseen classes, which indicates signiﬁcantlyimproved generalization capability.
Revised: We can see that our method achieves signiﬁcantly higher embedding space density andmuch lower spectral decay when the model tested on unseen classes, which indicates signiﬁcantlyimproved generalization capability.

Original:  Experimental Results
Revised:  Experimental Results

Original: Following the same procedure of existing papers [2, 7, 13, 18, 20], we evaluate the performance ofour proposed CRT method on four benchmark datasets for image retrieval tasks.
Revised: Following the same procedure of existing papers [2, 7, 13, 18, 20], we evaluate the performance ofour proposed CRT method on four benchmark datasets for image retrieval tasks.

Original: 4.1 Datasets and Experimental Settings
Revised: 4.1 Datasets and Experimental Settings

Original: (1) Datasets.
Revised: (1) Datasets.

Original: In the following experiments, we use the same benchmark datasets as in existingpapers for direct performance comparison.
Revised: In the following experiments, we use the same benchmark datasets as in existingpapers for direct performance comparison.

Original: Four datasets, i.e., CUB-200-2011 [57], Cars-196 [58],
Revised: Four datasets, i.e., CUB-200-2011 [57], Cars-196 [58],

Original: Standford Online Products (SOP)
Revised: Standford Online Products (SOP)

Original: [32], and In-Shop Clothes Retrieval (In-Shop) [59], are used toour experiments.
Revised: [32], and In-Shop Clothes Retrieval (In-Shop) [59], are used toour experiments.

Original: We use the same training and test split as in existing papers.
Revised: We use the same training and test split as in existing papers.

Original: Hyperameters weredetermined previous to the result runs using a 80-20 training and validation split.(2) Performance metrics and experimental settings.
Revised: Hyperameters weredetermined previous to the result runs using a 80-20 training and validation split.(2) Performance metrics and experimental settings.

Original: Following the standard protocol in [2, 7, 13,18, 20, 60], the Recall@K [61] is used to evaluate the performance of our algorithm.
Revised: Following the standard protocol in [2, 7, 13,18, 20, 60], the Recall@K [61] is used to evaluate the performance of our algorithm.

Original: For all datasets,our method is evaluated with the image data only, without using the bounding box information.
Revised: For all datasets,our method is evaluated with the image data only, without using the bounding box information.

Original: During training, we randomly sample a set of images in each iteration to train the network.
Revised: During training, we randomly sample a set of images in each iteration to train the network.

Original: In eachiteration, we ﬁrst randomly choose the image classes, and then randomly sample 5 images fromeach class to form a batch.
Revised: In eachiteration, we ﬁrst randomly choose the image classes, and then randomly sample 5 images fromeach class to form a batch.

Original: For the CUB and Cars datasets, we sampled 16 classes in a batch.
Revised: For the CUB and Cars datasets, we sampled 16 classes in a batch.

Original: Forthe SOP and In-Shop datasets, we sampled 36 classes in a batch.
Revised: Forthe SOP and In-Shop datasets, we sampled 36 classes in a batch.

Original: We apply random cropping withrandom ﬂipping and resizing to 227 × 227 for all training images.
Revised: We apply random cropping withrandom ﬂipping and resizing to 227 × 227 for all training images.

Original: For testing, we only use thecenter-cropped image to compute the feature embedding.
Revised: For testing, we only use the center-cropped image to compute the feature embedding.

Original: Unless otherwise speciﬁed, we use the
Revised: Unless otherwise speciﬁed, we use the

Original: MixTransformer-B2 (MiT-B2) developed in [48] as our baseline encoder network with its pyramid
Revised: MixTransformer-B2 (MiT-B2) developed in [48] as our baseline encoder network with its pyramid

Original: MLP head being removed.
Revised: MLP head being removed.

Original: Our CRT algorithm operates on the feature map generated by the lastblock (the 4-th block) of the backbone encoder network.
Revised: Our CRT algorithm operates on the feature map generated by the lastblock (the 4-th block) of the backbone encoder network.

Original: The initial learning rate is 3 e − 5 .
Revised: The initial learning rate is 3 e − 5 .

Original: For all images, the MS loss weight λ 1 in the ﬁrst embedding branch is set as 1.0.
Revised: For allimages, the MS loss weight λ 1 in the ﬁrst embedding branch is set as 1.0.

Original: In second embeddingbranch, it is set as 0.1 for the CUB and Cars datasets, and 0.9 for the SOP and In-Shop datasets.
Revised: In second embeddingbranch, it is set as 0.1 for the CUB and Cars datasets, and 0.9 for the SOP and In-Shop datasets.

Original: Theconsistency loss weight λ 2 is set to 0.9.
Revised: Theconsistency loss weight λ 2 is set to 0.9.

Original: The backbone network is pre-trained on the ImageNet-1Kdataset.
Revised: The backbone network is pre-trained on the ImageNet-1Kdataset.

Original: 4.2 Performance Comparisons with the State-of-the-art Methods
Revised: 4.2 Performance Comparisons with the State-of-the-art Methods

Original: We compare the performance of our CRT method with the state-of-the-art deep metric learningmethods.
Revised: We compare the performance of our CRT method with the state-of-the-art deep metric learningmethods.

Original: Performance comparison results on the CUB and Cars datasets are shown in Table 3.
Revised: Performance comparison results on the CUB and Cars datasets are shown in Table 3.

Original: Table 4 shows the comparison results on the SOP and In-Shop datasets.
Revised: Table 4 shows the comparison results on the SOP and In-Shop datasets.

Original: It should be noted that thefeature dimension of our CRT method is set to be 128.
Revised: It should be noted that thefeature dimension of our CRT method is set to be 128.

Original: We can see that our CRT method signiﬁcantlyoutperforms the current best deep metric learning method Hyp-DeiT [20].
Revised: We can see that our CRT method signiﬁcantlyoutperforms the current best deep metric learning method Hyp-DeiT [20].

Original: The results of the HypDeiT method are cited under the same experimental settings where the embedding size is 128 withimage size of 227 × 227, and a pretrained model on the ImageNet-1K.
Revised: The results of the HypDeiT method are cited under the same experimental settings where the embedding size is 128 withimage size of 227 × 227, and a pretrained model on the ImageNet-1K.

Original: We choose this setting sothat we can compare our method with many recent papers on metric learning.
Revised: We choose this setting sothat we can compare our method with many recent papers on metric learning.

Original: For example, on the
Revised: For example, on the

Original: CUB dataset, the top-1 recall rate has been improved by 4.28%.
Revised: CUB dataset, the top-1 recall rate has been improved by 4.28%.

Original: The experimental results on the restdatasets suggest that our CRT method consistently outperforms other deep metric learning methodsby large margins.
Revised: The experimental results on the restdatasets suggest that our CRT method consistently outperforms other deep metric learning methodsby large margins.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: As analyzed in [22] and [33]
Revised: As analyzed in [22] and [33]

Original: [Table]
Revised: [Table]

Original: most of the deep metric learning methods use different experimentalconditions, such as different settings of batch size, data augmentation methods, embedding sizes,training strategies, backbone networks
Revised: most of the deep metric learning methods use different experimentalconditions, such as different settings of batch size, data augmentation methods, embedding sizes,training strategies, backbone networks

Original: [Table]
Revised: [Table]

Original: and input image sizes [Table] to obtain the state-of-the-art performance, which could lead to confusion on the effectiveness of those methods.
Revised: and input image sizes, to obtain the state-of-the-art performance, which could lead to confusion on the effectiveness of those methods.

Original: For example, Hyp-DeiT[20] and IRT R [1] used larger batch sizes and memory bank technique to obtain better performance
Revised: For example, Hyp-DeiT

Original: [Table]
Revised: [Table]

Revised: [20] and IRT R [1] used larger batch sizes and memory bank technique to obtain better performance.

Original: The method of XBM+RTT [21] used the re-rank technique to obtain the highest top-1 recall rateon the SOP dataset.
Revised: The method of XBM+RTT [21] used the re-rank technique to obtain the highest top-1 recall rateon the SOP dataset.

Original: The Group Loss++ [7] also used the re-rank method to improve the retrieval [Table]
Revised: The Group Loss++ [7] also used the re-rank method to improve the retrievalperformance.

Original: performance. Moreover, Proxy-Anchor [9], Group Loss++ [7] and ETLR [13] used mixed poolingmethods to improve embedding performance.
Revised: Moreover, Proxy-Anchor [9], Group Loss++ [7] and ETLR [13] used mixed poolingmethods to improve embedding performance.

Original: During our experiments, we ﬁnd that it is challengingto re-produce their original results with our available computing resources and to conduct fair comparisons, especially for those methods without publicly available source code.
Revised: During our experiments, we ﬁnd that it is challengingto re-produce their original results with our available computing resources and to conduct fair comparisons, especially for those methods without publicly available source code.

Original: Instead, we implementour CRT method with the MS method [11] which has been extensively used in existing deep metriclearning methods.
Revised: Instead, we implementour CRT method with the MS method [11] which has been extensively used in existing deep metriclearning methods.

Original: In the following, we provide a fair comparison between our method and thebaseline MS method with different backbone networks under the same experimental conditions.
Revised: In the following, we provide a fair comparison between our method and thebaseline MS method with different backbone networks under the same experimental conditions.

Original: 4.3 Performance Comparisons with Different Backbone Networks.
Revised: 4.3 Performance Comparisons with Different Backbone Networks.

Original: We conduct experiments with different backbone networks to verify the effectiveness of the proposed
Revised: We conduct experiments with different backbone networks to verify the effectiveness of the proposed

Original: CRT method.
Revised: CRT method.

Original: For fair comparisons [22, 33], we reproduce the MS method [11] and compare theperformance using the same experimental settings.
Revised: For fair comparisons [22, 33], we reproduce the MS method [11] and compare theperformance using the same experimental settings.

Original: The hyper-parameter settings are consistent withthe original paper [11].
Revised: The hyper-parameter settings are consistent withthe original paper [11].

Original: The batch size is set to 80 on the CUB and Cars datasets, and 180 on the SOPand In-Shop datasets.
Revised: The batch size is set to 80 on the CUB and Cars datasets, and 180 on the SOPand In-Shop datasets.

Original: The embedded feature size is set to 128.
Revised: The embedded feature size is set to 128.

Original: Results are shown in Table 5, wecan see that our proposed method is able to consistently improve the performance of the baselinemethod.
Revised: Results are shown in Table 5, wecan see that our proposed method is able to consistently improve the performance of the baselinemethod.

Original: It should be noted that the DeiT-S and MiT-B1 networks are both based on the transformer,a powerful method for exploiting the global correlation between features for very efﬁcient imagerepresentation.
Revised: It should be noted that the DeiT-S and MiT-B1 networks are both based on the transformer,a powerful method for exploiting the global correlation between features for very efﬁcient imagerepresentation.

Original: From Table 5, we can see that, even on top of these two baseline methods with thepowerful transformer backbone network, our coded residual transform is still able to improve themetric learning performance by up to 3.7%, which is quite impressive.
Revised: From Table 5, we can see that, even on top of these two baseline methods with thepowerful transformer backbone network, our coded residual transform is still able to improve themetric learning performance by up to 3.7%, which is quite impressive.

Original: This is because our CRTmethod introduces the new ideas of multi-perspective projection and residual encoding, which havesigniﬁcantly improved the generalization performance of the deep metric learning.
Revised: This is because our CRTmethod introduces the new ideas of multi-perspective projection and residual encoding, which havesigniﬁcantly improved the generalization performance of the deep metric learning.

Original: 4.4 Ablation Studies
Revised: 4.4 Ablation Studies

Original: In this section, we provide extensive ablation studies to further understand our proposed CRT methodand characterize the performance of major algorithm components.
Revised: In this section, we provide extensive ablation studies to further understand our proposed CRT methodand characterize the performance of major algorithm components.

Original: In the following experiments,unless otherwise speciﬁed, the batch size N is set to 80.
Revised: In the following experiments,unless otherwise speciﬁed, the batch size N is set to 80.

Original: The backbone network used in thisexperiment is the MiT-B1.(1)
Revised: The backbone network used in thisexperiment is the MiT-B1.(1)

Original: Contributions of major algorithm components.
Revised: Contributions of major algorithm components.

Original: From the performance evaluation perspective,our algorithm has two major components, the coded residual transform (CRT) and cross-CRTcorrelation consistency (CRT-Consistency).
Revised: From the performance evaluation perspective,our algorithm has two major components, the coded residual transform (CRT) and cross-CRTcorrelation consistency (CRT-Consistency).

Original: We ﬁx the MS loss weight λ 1 in the ﬁrst and secondembedding branches as 1.0 and 0.1, respectively.
Revised: We ﬁx the MS loss weight λ 1 in the ﬁrst and secondembedding branches as 1.0 and 0.1, respectively.

Original: The consistency loss weight λ 2 is set to be 0.9.
Revised: The consistency loss weight λ 2 is set to be 0.9.

Original: From
Revised: From

Original: Table 6, we can see that the performance dropped 1.98% for the top-1 recall rate without the CRT.
Revised: Table 6, we can see that the performance dropped 1.98% for the top-1 recall rate without the CRT.

Original: Theperformance future dropped 1.69% for the top-1 recall rate without the CRT-Consistency, which isthe baseline result that obtained based on the MS loss function.
Revised: Theperformance future dropped 1.69% for the top-1 recall rate without the CRT-Consistency, which isthe baseline result that obtained based on the MS loss function.

Original: Moreover, the multi-perspective CRTfeatures transformation plays an important role for the ﬁnal performance.
Revised: Moreover, the multi-perspective CRTfeatures transformation plays an important role for the ﬁnal performance.

Original: For different projectionanchors, the CRT features can be transformed using different embedding networks or the sameembedding network.
Revised: For different projectionanchors, the CRT features can be transformed using different embedding networks or the same embedding network.

Original: Here, we compare the performance of our CRT method with and without usingthe same embedding network for the transformation of CRT features.
Revised: Here, we compare the performance of our CRT method with and without usingthe same embedding network for the transformation of CRT features.

Original: Results are shown in Table 7.
Revised: Results are shown in Table 7.

Original: We can see that better performance can be obtained using different embedding networks for each CRTfeature corresponding to each projection anchor.
Revised: We can see that better performance can be obtained using different embedding networks for each CRTfeature corresponding to each projection anchor.

Original: This shows the effectiveness of our multi-perspectiveprojection.
Revised: This shows the effectiveness of our multi-perspectiveprojection.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: (2) Impact of the projection anchors.
Revised: (2) Impact of the projection anchors.

Original: In this experiment, we analyze the impact of different numbersof projection anchors.
Revised: In this experiment, we analyze the impact of different numbersof projection anchors.

Original: Table 8 and Table 9 show the results of different numbers of projection anchorsfor the ﬁrst embedding branch (low-dimensional embedding branch) and the second embeddingbranch (high-dimensional embedding branch), respectively.
Revised: Table 8 and Table 9 show the results of different numbers of projection anchorsfor the ﬁrst embedding branch (low-dimensional embedding branch) and the second embeddingbranch (high-dimensional embedding branch), respectively.

Original: In Table 8, the number of projectionanchors in the second embedding branch is ﬁxed at 64.
Revised: In Table 8, the number of projectionanchors in the second embedding branch is ﬁxed at 64.

Original: This table reports the results of embeddingcomputed from the ﬁrst embedding branch when the number of projection anchors changed from 1to 64.
Revised: This table reports the results of embeddingcomputed from the ﬁrst embedding branch when the number of projection anchors changed from 1to 64.

Original: The weights of the backbone networks are shared and the weights of the CRT heads are notshared for this experiment.
Revised: The weights of the backbone networks are shared and the weights of the CRT heads are notshared for this experiment.

Original: In Table 9, the number of project anchors are equal for the ﬁrst and thesecond CRT embedding branches, which are changed from 1 to 100, and the weights of both thebackbone network and the CRT branches are shared in this experiment.
Revised: In Table 9, the number of project anchors are equal for the ﬁrst and thesecond CRT embedding branches, which are changed from 1 to 100, and the weights of both thebackbone network and the CRT branches are shared in this experiment.

Original: We can see that when thenumber of anchors is set to be 64, the embedding performance reaches to the best.
Revised: We can see that when thenumber of anchors is set to be 64, the embedding performance reaches to the best.

Original: In Table 9, the network weight of the second embedding branch is shared from the ﬁrst embeddingbranch.
Revised: In Table 9, the network weight of the second embedding branch is shared from the ﬁrst embeddingbranch.

Original: Here, we compare the performance of our CRT method with and without shared weightsbetween these two embedding branches.
Revised: Here, we compare the performance of our CRT method with and without shared weightsbetween these two embedding branches.

Original: Results are shown in Table 10.
Revised: Results are shown in Table 10.

Original: We can see that theperformance remains almost the same between these two study cases.
Revised: We can see that theperformance remains almost the same between these two study cases.

Original: In other experiments, todecrease the memory consumption, we shared weights for all the experiments.
Revised: In other experiments, todecrease the memory consumption, we shared weights for all the experiments.

Original: anchors in the ﬁrst embedding branch.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original:  Conclusion
Revised:  Conclusion

Original: In this paper [Table] we have developed a coded residual transform for generalizable deep metric learning, which consists of a multi-perspective projection and coded residual transform encoder and across-CRT correlation consistency constraint.
Revised: In this paper, we have developed a coded residual transform for generalizable deep metric learning, which consists of a multi-perspective projection and coded residual transform encoder and across-CRT correlation consistency constraint.

Original: It has two unique characteristics.
Revised: It has two unique characteristics.

Original: First, it representsand encodes the feature map from a set of complimentary perspectives based on projections ontodiversiﬁed anchors.
Revised: First, it representsand encodes the feature map from a set of complimentary perspectives based on projections ontodiversiﬁed anchors.

Original: Second, unlike existing transformer-based feature representation approacheswhich encode the original values of features based on global correlation analysis, the proposed codedresidual transform encodes the relative differences between original features and their projectedanchors.
Revised: Second, unlike existing transformer-based feature representation approacheswhich encode the original values of features based on global correlation analysis, the proposed codedresidual transform encodes the relative differences between original features and their projectedanchors.

Original: The proposed CRT method has achieved new state-of-the-art metric learning performanceon benchmark datasets.
Revised: The proposed CRT method has achieved new state-of-the-art metric learning performanceon benchmark datasets.

Original: We hope our method can motivate further research.
Revised: We hope our method can motivate further research.

Original: One limitation is thatthe memory and compute usage will be increased during training for these two embedding branches,and we shared weights between them to solve this problem in our experiments.
Revised: One limitation is thatthe memory and compute usage will be increased during training for these two embedding branches,and we shared weights between them to solve this problem in our experiments.

Original: Another limitationis that the projection anchors were learned from the training set.
Revised: Another limitationis that the projection anchors were learned from the training set.

Original: It is unclear whether it is the bestprojection anchors for new test classes.
Revised: It is unclear whether it is the bestprojection anchors for new test classes.

Original: We leave it for the future work.
Revised: We leave it for the future work.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes] Abstract and Introduction
Revised: [Yes] Abstract and Introduction

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] Conclusion
Revised: [Yes] Conclusion

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes]
Revised: [Yes]

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [N/A]
Revised: [N/A]

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes]
Revised: [Yes]

Original: Supplemental
Revised: Supplemental

Original: material
Revised: material

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experi-
Revised: g., with respect to the random seed after running experi-

Original: [Table]

Original: ments multiple times)?
Revised: ments multiple times)?

Original: [Yes
Revised: [Yes]

Original: [Table]

Original: (d) Did you include the total amount of compute and the type of resources used (e
Revised: (d) Did you include the total amount of compute and the type of resources used (e

Original: g., typeof GPUs, internal cluster, or cloud provider)?
Revised: g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes]
Revised: [Yes]

Original: If you are using existing assets (e
Revised: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...
Revised: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No]
Revised: [No]

Original: [Table]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: [Table]

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: [Table]

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: [Table]

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: [Table]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: [Table]

Original: ) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: [Table]
Revised: [Table]

Original: applicable
Revised: applicable?

Original: [N/ [Table]
Revised: [N/A]

Original:  Did you describe any potential participant risks, with links to Institutional Review
Revised: ) Did you describe any potential participant risks, with links to Institutional Review

Original: [Table]

Original: Board IRB) approvals, if applicable
Revised: Board (IRB) approvals, if applicable?

Original: [N [Table]
Revised: [N/A]

Original: ) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: [Table]
Revised: [Table]

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: [Table]