Original: Coded Residual Transform for Generalizable Deep Metric Learnings
Revised: Coded Residual Transform for Generalizable Deep Metric Learning

Original: Abstract
Revised: Abstract

Original: A fundamental challenge in deep metric learning is the generalization capabilityof the feature embedding network model since the embedding network learned ontraining classes need to be evaluated on new test classes.
Revised: A fundamental challenge in deep metric learning is the generalization capability of the feature embedding network model since the embedding network learned on training classes need to be evaluated on new test classes.

Original: To address this challenge,in this paper, we introduce a new method called coded residual transform (CRT) fordeep metric learning to signiﬁcantly improve its generalization capability.
Revised: To address this challenge, in this paper, we introduce a new method called coded residual transform (CRT) for deep metric learning to signiﬁcantly improve its generalization capability.

Original: Specifically, we learn a set of diversiﬁed anchor features, project the feature map ontoeach anchor, and then encode its features using their projection residuals weightedby their correlation coefﬁcients with each anchor.
Revised: Speciﬁcally, we learn a set of diversiﬁed prototype features, project the feature map onto each prototype, and then encode its features using their projection residuals weighted by their correlation coefﬁcients with each prototype.

Original: The proposed CRT method hasthe following two unique characteristics.
Revised: The proposed CRT method has the following two unique characteristics.

Original: First, it represents and encodes the featuremap from a set of complimentary perspectives based on projections onto diversiﬁed anchors.
Revised: First, it represents and encodes the feature map from a set of complimentary perspectives based on projections onto diversiﬁed prototypes.

Original: Second, unlike existing transformer-based feature representationapproaches which encode the original values of features based on global correlationanalysis, the proposed coded residual transform encodes the relative differences between the original features and their projected anchors.
Revised: Second, unlike existing transformer-based feature representation approaches which encode the original values of features based on global correlation analysis, the proposed coded residual transform encodes the relative differences between the original features and their projected prototypes.

Original: Embedding space density and spectral decay analysis show that this multi-perspective projection onto diversiﬁed anchors and coded residual representation are able to achieve signiﬁcantlyimproved generalization capability in metric learning.
Revised: Embedding space density and spectral decay analysis show that this multi-perspective projection onto diversiﬁed prototypes and coded residual representation are able to achieve signiﬁcantly improved generalization capability in metric learning.

Original: Finally, to further enhancethe generalization performance, we propose to enforce the consistency on theirfeature similarity matrices between coded residual transforms with different sizesof projection anchors and embedding dimensions.
Revised: Finally, to further enhance the generalization performance, we propose to enforce the consistency on their feature similarity matrices between coded residual transforms with different sizes of projection prototypes and embedding dimensions.

Original: Our extensive experimentalresults and ablation studies demonstrate that the proposed CRT method outperformthe state-of-the-art deep metric learning methods by large margins and improvingupon the current best method by up to 4.28% on the CUB dataset.
Revised: Our extensive experimental results and ablation studies demonstrate that the proposed CRT method outperform the state-of-the-art deep metric learning methods by large margins and improving upon the current best method by up to 4.28% on the CUB dataset.

Original:  Introduction
Revised:  Introduction

Original: Deep metric learning (DML) aims to learn effective features to characterize or represent images, whichhas important applications in image retrieval [1, 2], image recognition [3], person re-identiﬁcation [4],image segmentation [5] and tracking [6].
Revised: Deep metric learning (DML) aims to learn effective features to characterize or represent images, which has important applications in image retrieval [1, 2], image recognition [3], person re-identiﬁcation [4], image segmentation [5], and tracking [6].

Original: Successful metric learning needs to achieve the followingtwo objectives: (1) Discriminative .
Revised: Successful metric learning needs to achieve the following two objectives: (1) Discriminative .

Original: In the embedded feature space, image features with the samesemantic labels should be aggregated into compact clusters in the high-dimensional feature spacewhile those from different classes should be well separated from each other.
Revised: In the embedded feature space, image features with the same semantic labels should be aggregated into compact clusters in the high-dimensional feature space while those from different classes should be well separated from each other.

Original: (2) Generalizable .
Revised: (2) Generalizable .

Original: The learned features should be able to generalize well from the training images to test images ofnew classes which have not been seen before.
Revised: The learned features should be able to generalize well from the training images to test images of new classes which have not been seen before.

Original: During the past a few years, methods based ondeep neural networks, such as metric loss functions design [7, 8, 9, 10, 11], embedding transfer[12, 13, 14, 15, 16], structural matching [17], graph neural networks [2, 18], language guidance [19],and vision transformer [1, 20, 21], have achieved remarkable progress on learning discriminativefeatures.
Revised: During the past a few years, methods based on deep neural networks, such as metric loss functions design [7, 8, 9, 10, 11], embedding transfer [12, 13, 14, 15, 16], structural matching [17], graph neural networks [2, 18], language guidance [19], and vision transformer [1, 20, 21], have achieved remarkable progress on learning discriminative features.

Original: However, the generalization onto unseen new classes remains a signiﬁcant challenge forexisting deep metric learning methods.
Revised: However, the generalization onto unseen new classes remains a signiﬁcant challenge for existing deep metric learning methods.

Original: In the literature, to improve the deep metric learning performance and alleviate the generalizationproblem on unseen classes, regularization techniques [22, 15], language-guided DML [19], andfeature fusion [2, 8, 18] methods have been developed.
Revised: In the literature, to improve the deep metric learning performance and alleviate the generalization problem on unseen classes, regularization techniques [22, 15], language-guided DML [19], and feature fusion [2, 8, 18] methods have been developed.

Original: Existing approaches to addressing thegeneralization challenge in metric learning focus on the robustness of linear or kernel-based distancemetrics [23, 24], analysis of error bounds of the generalization process [25], and correlation analysisbetween generalization and structure characteristics of the learned embedding space [22].
Revised: Existing approaches to addressing the generalization challenge in metric learning focus on the robustness of linear or kernel-based distance metrics [23, 24], analysis of error bounds of the generalization process [25], and correlation analysis between generalization and structure characteristics of the learned embedding space [22].

Original: It shouldbe noted that in existing methods, the input image is analyzed and transformed as a whole into anembedded feature.
Revised: It should be noted that in existing methods, the input image is analyzed and transformed as a whole into an embedded feature.

Original: In other words, the image is represented and projected globally from a singleperspective.
Revised: In other words, the image is represented and projected globally from a single perspective.

Original: We recognize that this single-perspective projection is not able to represent and encodethe highly complex and dynamic correlation structures in the high-dimensional feature space sincethey are being collapsed and globally projected onto one single perspective and the local correlationdynamics have been suppressed.
Revised: We recognize that this single-perspective projection is not able to represent and encode the highly complex and dynamic correlation structures in the high-dimensional feature space since they are being collapsed and globally projected onto one single perspective and the local correlation dynamics have been suppressed.

Original: According to our experiments, this single-perspective globalprojection will increase the marginal variance [26] and consequently degrade the generalizationcapability of the deep metric learning method.
Revised: According to our experiments, this single-perspective global projection will increase the marginal variance [26] and consequently degrade the generalization capability of the deep metric learning method.

Original: Furthermore, we observe that existing deep metric learning methods attempt to transform and encodethe original features.
Revised: Furthermore, we observe that existing deep metric learning methods attempt to transform and encode the original features.

Original: From the generalization point of view, we ﬁnd that it is more effective to learnthe embedding based on relative difference between features since the absolute value of featuresmay vary signiﬁcantly from the training to new test classes, but the relative change patterns betweenfeatures may remain largely invariant.
Revised: From the generalization point of view, we ﬁnd that it is more effective to learn the embedding based on relative difference between features since the absolute value of features may vary signiﬁcantly from the training to new test classes, but the relative change patterns between features may remain largely invariant.

Original: To further understand this idea, consider the following toyexample: a face in the daytime may appear much different from a face in the night time due tochanges in lighting conditions.
Revised: To further understand this idea, consider the following toy example: a face in the daytime may appear much different from a face in the night time due to changes in lighting conditions.

Original: However, an effective face detection with sufﬁcient generalizationpower will not focus on the absolute pixel values of the face image.
Revised: However, an effective face detection with sufﬁcient generalization power will not focus on the absolute pixel values of the face image.

Original: Instead, it detects the face basedon the relative change patterns between neighboring regions inside the face image.
Revised: Instead, it detects the face based on the relative change patterns between neighboring regions inside the face image.

Original: Motivated by thisobservation, in this work, to address the generalization challenge, we propose to learn the embeddedfeature from the projection residuals of the feature map, instead of its absolute features.
Revised: Motivated by this observation, in this work, to address the generalization challenge, we propose to learn the embedded feature from the projection residuals of the feature map, instead of its absolute features.

Original: The above two ideas, namely, multi-perspective projection and residual encoding, lead to our proposedmethod of coded residual transform for deep metric learning.
Revised: The above two ideas, namely, multi-perspective projection and residual encoding, lead to our proposed method of coded residual transform for deep metric learning.

Original: Speciﬁcally, we propose to learn a setof diversiﬁed anchor features, project the features onto every anchor, and then encode the featuresusing the projection residuals weighted by their correlation coefﬁcients with the target anchor.
Revised: Speciﬁcally, we propose to learn a set of diversiﬁed prototype features, project the features onto every prototype, and then encode the features using the projection residuals weighted by their correlation coefﬁcients with the target prototype.

Original: Unlikeexisting transformer-based feature representation approaches which encode the original values offeatures based on global correlation analysis [27, 28, 29], the proposed coded residual transformencodes the relative differences between original features and their projected anchors.
Revised: Unlike existing transformer-based feature representation approaches which encode the original values of features based on global correlation analysis [27, 28, 29], the proposed coded residual transform encodes the relative differences between original features and their projected prototypes.

Original: Our extensiveexperimental results and ablation studies demonstrate that the proposed CRT method is able tosigniﬁcantly improve the generalization performance of deep metric learning, outperforming thestate-of-the-art methods by large margins and improving upon the current best method by up to4.28%.
Revised: Our extensive experimental results and ablation studies demonstrate that the proposed CRT method is able to improve the generalization performance of deep metric learning, outperforming the state-of-the-art methods by large margins and improving upon the current best method by up to 4.28%.

Original: We learn those projection anchors based on the training classes and transfer them into the test classes.
Revised: We learn those projection prototypes based on the training classes and transfer them into the test classes.

Original: Although the training and test classes share the same set of anchors, the actual distributions of projectanchors during training and testing could be much different due to the distribution shift betweenthe training and testing classes.
Revised: Although the training and test classes share the same set of prototypes, the actual distributions of project prototypes during training and testing could be much different due to the distribution shift between the training and testing classes.

Original: During our coded residual transform, we assign different weightsfor different project residuals based on the correlation between the feature and the correspondinganchor.
Revised: During our coded residual transform, we assign different weights for different project residuals based on the correlation between the feature and the corresponding prototype.

Original: Therefore, for training classes, the subset of anchors which are close to the training imageswill have larger weights.
Revised: Therefore, for training classes, the subset of prototypes which are close to the training images will have larger weights.

Original: Similarly, for the testing classes, the subset of anchors which are close tothe test images will have larger weights.
Revised: Similarly, for the testing classes, the subset of prototypes which are close to the test images will have larger weights.

Original: This correlation-based weighting for the projection residualcontribute signiﬁcantly to the overall performance gain.
Revised: This correlation-based weighting for the projection residual contribute signiﬁcantly to the overall performance gain.

Original:  Related Work and Unique Contributions
Revised:  Related Work and Unique Contributions

Original: This work is related to deep metric learning, transformer-based learning methods, and residualencoding.
Revised: This work is related to deep metric learning, transformer-based learning methods, and residual encoding.

Original: In this section, we review the existing methods on these topics and discuss the uniquenovelty of our approach.
Revised: In this section, we review the existing methods on these topics and discuss the unique novelty of our approach.

Original: (1) Deep metric learning.
Revised: (1) Deep metric learning.

Original: Deep metric learning aims to learn discriminative features with the goal tominimize intra-class sample distance and maximize inter-class sample distance in a contrastive manner.
Revised: Deep metric learning aims to learn discriminative features with the goal to minimize intra-class sample distance and maximize inter-class sample distance in a contrastive manner.

Original: Contrastive loss [30] has been successfully used in early methods of deep metric learning, aimingto optimize pairwise distance of between samples.
Revised: Contrastive loss [30] has been successfully used in early methods of deep metric learning, aiming to optimize pairwise distance of between samples.

Original: By exploring more sophisticated relationship 
Revised: By exploring more sophisticated relationship between samples, a variety of metric loss functions, such as triplet loss , lifted structured

Original: between samples, a variety of metric loss functions, such as triplet loss [31], lifted structured loss [32],proxy-anchor loss[9], and multi-similarity (MS) loss [11], have been developed.
Revised: loss [32], proxy-anchor loss and multi-similarity (MS) loss , have been developed.

Original: According to the studies in [22] and [33], the MS loss was veriﬁed to be one of the most efﬁcient metric loss functions.
Revised: According to the studies in [22] and [33], the MS loss was veriﬁed to be one of the most efﬁcient metric loss functions.

Original: Some recent methods explore how to use multiple features to learn robust feature embeddings.
Revised: Some recent methods explore how to use multiple features to learn robust feature embeddings.

Original: Kanet al.
Revised: Kan et al.

Original: [2] and Seidenschwarz et al.
Revised: [2] and Seidenschwarz et al.

Original: [18] adopt K-nearest neighbors (k-NN) of an anchor image tobuild local graph neural network (GNN) and reﬁne embedding vectors based on message exchangesbetween the graph nodes.
Revised: [18] adopt K-nearest neighbors (k-NN) of an anchor image to build local graph neural network (GNN) and reﬁne embedding vectors based on message exchanges between the graph nodes.

Original: Zhao et al.
Revised: Zhao et al.

Original: [17] proposed a structural matching method to learn a metricfunction between feature maps based on the optimal transport theory.
Revised: [17] proposed a structural matching method to learn a metric function between feature maps based on the optimal transport theory.

Original: Based on the form of softmax,margin-based softmax loss functions [34, 35, 36] were also proposed to learn discriminative features.
Revised: Based on the form of softmax, margin-based softmax loss functions [34, 35, 36] were also proposed to learn discriminative features.

Original: Sohn [37] improved the contrastive loss and triplet loss by introducing N negative examples andproposed the N-pair loss function to speed up model convergence during training.(2)
Revised: Sohn [37] improved the contrastive loss and triplet loss by introducing N negative examples and proposed the N-pair loss function to speed up model convergence during training.

Original: Transformer-based learning methods.
Revised: Transformer-based learning methods.

Original: This work is related to transformer-based learningmethods since our method also analyze the correlation between features and uses this correlationinformation to aggregate features.
Revised: This work is related to transformer- based learning methods since our method also analyze the correlation between features and uses this correlation information to aggregate features.

Revised: The original

Revised: work

Revised: of transformer [27]

Revised: aims to learn

Revised: a self-

Original: The original work of transformer [27] aims to learn a self-attentionfunction and a feed forward transformation network for nature language processing.
Revised: attention function and a feed forward transformation network for nature language processing

Original: Recently, ithas been successfully applied to computer vision and image processing.
Revised: Recently, it has been successfully applied to computer vision and image processing.

Original: ViT [38] demonstrates thatthat a pure transformer can achieve state-of-the-art performance in image classiﬁcation.
Revised: ViT [38] demonstrates that a pure transformer can achieve state-of-the art performance in image classiﬁcation.

Original: ViT treatseach image as a sequence of tokens and then feeds them to multiple transformer layers to performthe classiﬁcation.
Revised: ViT treats each image as a sequence of tokens and then feeds them to multiple transformer layers to perform the classiﬁcation.

Original: Subsequently, DeiT [39] further explores a data-efﬁcient training strategy and adistillation approach for ViT. More recent methods such as T2T ViT [29], TNT [40], CrossViT [41]and LocalViT [42] further improve the ViT method for image classiﬁcation.
Revised: Subsequently, DeiT [39] further explores a data-efﬁcient training strategy and a distillation approach for ViT. More recent methods such as T2T ViT [29], TNT [40], CrossViT [41] and LocalViT [42] further improve the ViT method for image classiﬁcation.

Original: PVT [43] incorporates apyramid structure into the transformer for dense prediction tasks.
Revised: PVT [43] incorporates a pyramid structure into the transformer for dense prediction tasks.

Original: After that, methods such as Swin[28], CvT [44], CoaT [45], LeViT [46], Twins [47] and MiT [48] enhance the local continuity offeatures and remove ﬁxed size position embedding to improve the performance of transformers fordense prediction tasks.
Revised: After that, methods such as Swin [28], CvT [44], CoaT [45], LeViT [46], Twins [47] and MiT [48] enhance the local continuity of features and remove ﬁxed size position embedding to improve the performance of transformers for dense prediction tasks.

Original: For deep metric learning, El-Nouby et al.
Revised: For deep metric learning, El-Nouby et al.

Original: [1] and Ermolov et al.
Revised: [1] and Ermolov et al.

Original: [20] adoptthe DeiT-S network[39] as a backbone to extract features, achieving impressive performance.(3)
Revised: [20] adopt the DeiT-S network[39] as a backbone to extract features, achieving impressive performance.

Original:  Residual encoding.
Revised: (3) Residual encoding.

Original: Residual encoding was ﬁrst proposed by Jégou et al.
Revised: Residual encoding was ﬁrst proposed by Jégou et al.

Original: [49], where the vectorof locally aggregated descriptors (VLAD) algorithm is used to aggregate the residuals betweenfeatures and their best-matching codewords.
Revised: [49], where the vector of locally aggregated descriptors (VLAD) algorithm is used to aggregate the residuals between features and their best-matching codewords.

Original: Based on the VLAD method, VLAD-CNN [50] hasdeveloped the residual encoders for visual recognition and understanding tasks.
Revised: Based on the VLAD method, VLAD-CNN [50] has developed the residual encoders for visual recognition and understanding tasks.

Original: NetVLAD
Revised: NetVLAD

Original: [51]and Deep-TEN [52] extend this idea and develop an end-to-end learnable residual encoder basedon soft-assignment.
Revised: [51] and Deep-TEN [52] extend this idea and develop an end-to-end learnable residual encoder based on soft-assignment.

Original: It should be noted that features learned by these methods typically have verylarge sizes, for example, 16k, 32k and 4096 for AlexNet [53], VGG-16 [54] and ResNet-50 [55],respectively.(4)
Revised: It should be noted that features learned by these methods typically have very large sizes, for example, 16k, 32k and 4096 for AlexNet [53], VGG-16 [54] and ResNet-50 [55], respectively.

Original:  Unique Contributions.
Revised: (4) Unique Contributions.

Original: Compared to the above existing methods, the unique contributions ofthis paper can be summarized as follows: (1) We introduce a new CRT method which learns a set ofanchor features, project the feature map onto each anchor, and then encode its features using theirprojection residuals weighted by their correlation coefﬁcients with each anchor.
Revised: Compared to the above existing methods, the unique contributions of this paper can be summarized as follows: (1) We introduce a new CRT method which learns a set of prototype features, project the feature map onto each prototype, and then encode its features using their projection residuals weighted by their correlation coefﬁcients with each prototype.

Original: (2) We introduce adiversity constraint for the set of anchor features so that the CRT method can represent and encodethe feature map from a set of complimentary perspectives.
Revised: We introduce a diversity constraint for the set of prototype features so that the CRT method can represent and encode the feature map from a set of complimentary perspectives.

Original: Unlike existing transformer-based featurerepresentation approaches which encode the original values of features based on global correlationanalysis, the proposed coded residual transform encode the relative differences between originalfeatures and their projected anchors.
Revised: Unlike existing transformer-based feature representation approaches which encode the original values of features based on global correlation analysis, the proposed coded residual transform encode the relative differences between original features and their projected prototypes.

Original: (3) To further enhance the generalization performance, wepropose to enforce the feature distribution consistency between coded residual transforms withdifferent sizes of projection anchors and embedding dimensions.
Revised: (3) To further enhance the generalization performance, we propose to enforce the feature distribution consistency between coded residual transforms with different sizes of projection prototypes and embedding dimensions.

Original: (4) We demonstrate that this multiperspective projection with diversiﬁed anchors and coded residual representation based on relativedifferences are able to achieve signiﬁcantly improved generalization capability in metric learning.
Revised: (4) We demonstrate that this multi-perspective projection with diversiﬁed prototypes and coded residual representation based on relative differences are able to achieve signiﬁcantly improved generalization capability in metric learning.

Original: Our proposed CRT method outperforms the state-of-the-art methods by large margins and improvingupon the baseline method by up to 4.28%.
Revised: Our proposed CRT method outperforms the state-of-the-art methods by large margins and improving upon the baseline method by up to 4.28%.

Original:  Method
Revised:  Method

Original: In the following sections, we present our proposed method of coded residual transform for deepmetric learning.
Revised: In the following sections, we present our proposed method of coded residual transform for deep metric learning.

Original: 3.1 Method Overview
Revised: 3.1 Method Overview

Original: Figure 1 provides an overview of the proposed CRT method.
Revised: Figure 1 provides an overview of the proposed CRT method.

Original: We ﬁrst use a backbone network Φ toencode the input image and extract its feature map
Revised: We ﬁrst use a backbone network Φ to encode the input image and extract its feature map

Original: X ∈ R H
Revised: X ∈ R H

Original: × W × L , which consists of H × W feature vectors of size L .
Revised: × W × L , which consists of H × W feature vectors of size L .

Original: To improve the generalization capability of deep metric learning on feature map,we learn an anchor network to generate a set of diversiﬁed anchor features { c 1 , c 2 , · · · , c K } fromthe training set.
Revised: To improve the generalization capability of deep metric learning on feature map, we learn a prototype network to generate a set of diversiﬁed prototype features { c 1 , c 2 , · · · , c K } from the training set.

Original: Here, by "diversiﬁed" we mean these anchors are able to provide multi-perspectivecomplimentary representation of image features.
Revised: Here, by "diversiﬁed" we mean these prototypes are able to provide multiperspective complimentary representation of image features.

Original: We then project all feature vectors in the featuremap onto each anchor c k and encode them using the projection residuals weighted by their correlationcoefﬁcients with respect to this anchor.
Revised: We then project all feature vectors in the feature map onto each prototype c k and encode them using the projection residuals weighted by their correlation coefﬁcients with respect to this prototype.

Original: Each anchor c k will generate a coded residual representationr k of the feature map.
Revised: Each prototype c k will generate a coded residual representation r k of the feature map.

Original: After being further processed by a nonlinear embedding network Γ k , thesecoded residual representations will be added together to form the ﬁnal nonlinear feature embedding.
Revised: After being further processed by a nonlinear embedding network Γ k , these coded residual representations will be added together to form the ﬁnal nonlinear feature embedding.

Original: The above feature embedding process is referred to as coded residual transform .
Revised: The above feature embedding process is referred to as coded residual transform .

Original: To further enhancethe generalization performance, we introduce a consistency constraint between two different codedresidual transform with different values of K which is the size of the anchor set.
Revised: To further enhance the generalization performance, we introduce a consistency constraint between two different coded residual transform with different values of K which is the size of the prototype set.

Original: In the followingsections, we will explain the above major components of our CRT method in more details.
Revised: In the following sections, we will explain the above major components of our CRT method in more details.

Original: 3.2 Multi-Perspective Projection and Correlation-Based Residual Encoding
Revised: 3.2 Multi-Perspective Projection and Correlation-Based Residual Encoding

Original: Given an input image I , a backbone network Φ is used to encode image I into a feature map
Revised: Given an input image I , a backbone network Φ is used to encode image I into a feature map

Original: X ∈ R
Revised: X ∈ R

Original: H
Revised: H

Original: × W × L , i.e., X =
Revised: × W × L , i.e., X =

Original: Φ( I ) .
Revised: Φ( I ) .

Original: First, we learn anchor network to generate a set of diversiﬁed anchorfeatures C
Revised: First, we learn prototype network to generate a set of diversiﬁed prototype features C

Original: = { c 1 , c 2 , · · · , c K } , c k ∈ R L , whose learning process will be explained in the followingsection.
Revised: = { c 1 , c 2 , · · · , c K } , c k ∈ R L , whose learning process will be explained in the following section.

Original: For each anchor c k , we project the whole feature map X onto this anchor.
Revised: For each prototype c k , we project the whole feature map X onto this prototype.

Original: Speciﬁcally, foreach feature x j ∈ X inside the feature map, we ﬁnd the relative difference d kj = x
Revised: Speciﬁcally, for each feature x j ∈ X inside the feature map, we ﬁnd the relative difference d kj = x

Original: j − c k betweenthe feature x j and the anchor c k .
Revised: j − c k between the feature x j and the prototype c k .

Original: Let w kj = x j · c
Revised: Let w kj = x j · c

Original: Tk be their correlation coefﬁcient.
Revised: Tk be their correlation coefﬁcient.

Original: Then, withanchor c k , the whole map is transformed into the following feature vector using weighted summation[Figure] of d kj :
Revised: Then, with prototype c k , the whole map is transformed into the following feature vector using weighted summation of d kj :

Original: [Equation]
Revised: [Equation]

Original: where the nonlinear function log[1+exp( w kj )] converts the correlation coefﬁcient w kj into a positivenumber.
Revised: where the nonlinear function log[1+exp( w kj )] converts the correlation coefﬁcient w kj into a positive number.

Original: In this way, each anchor c k will generate a separate coded residual representation r k ofthe feature map for the input image.
Revised: In this way, each prototype c k will generate a separate coded residual representation r k of the feature map for the input image.

Original: Finally, r k is further transformed by a nonlinear embeddingnetwork Γ k with multiple fully connected layers and one GELU nonlinear activation layer [56].
Revised: Finally, r k is further transformed by a nonlinear embedding network Γ k with multiple fully connected layers and one GELU nonlinear activation layer [56].

Original: Byaggregating those nonlinear embeddings, the ﬁnal embedded feature f is then given by
Revised: By aggregating those nonlinear embeddings, the ﬁnal embedded feature f is then given by

Original: [Equation]
Revised: [Equation]

Original: The task of the anchor network is to generate an optimized set of anchors.
Revised: The task of the prototype network is to generate an optimized set of prototypes.

Original: Its input is the featuresfrom the feature map.
Revised: Its input is the features from the feature map.

Original: Its weights are the anchors that need to be learned, and its output is thecorrelations between the input features and the weights, which are the correlations between thefeature map and anchor.
Revised: Its weights are the prototypes that need to be learned, and its output is the correlations between the input features and the weights, which are the correlations between the feature map and prototype.

Original: The anchor network will update these anchors using stochastic gradientdescent (SGD) algorithm in back propagation.
Revised: The prototype network will update these prototypes using stochastic gradient descent (SGD) algorithm in back propagation.

Original: In this work, we use an one-layer fully connectednetwork to implement this anchor network.
Revised: In this work, we use an one-layer fully connected network to implement this prototype network.

Original: The regularization of anchors will be discussed in thefollowing section.
Revised: The regularization of prototypes will be discussed in the following section.

Original: Figure 2 shows two sets of examples of correlation map between different anchors and the featuremap.
Revised: Figure 2 shows two sets of examples of correlation map between different prototypes and the feature map.

Original: Each image corresponds to one anchor.
Revised: Each image corresponds to one prototype.

Original: The heat map shows the correlation between the anchor and the feature at the corresponding spatial location from the feature map.
Revised: The heat map shows the correlation between the prototype and the feature at the corresponding spatial location from the feature map.

Original: We can see that differentanchors are capturing different semantic components of the image and the feature map is projectedand encoded from a large set of different semantic perspectives.
Revised: We can see that different prototypes are capturing different semantic components of the image and the feature map is projected and encoded from a large set of different semantic perspectives.

Original: Our generalization analysis andablation studies will demonstrate that this multi-perspective projection and encoding will providesigniﬁcantly enhanced generalization capability and improved metric learning performance.
Revised: Our generalization analysis and ablation studies will demonstrate that this multi-perspective projection and encoding will provide signiﬁcantly enhanced generalization capability and improved metric learning performance.

Original: 3.3 Network Model Learning and Generalization Analysis
Revised: 3.3 Network Model Learning and Generalization Analysis

Original: As illustrated in Figure 1, our proposed CRT method involves the encoder network Φ , the anchornetwork A , and the multi-perspective feature embedding networks { Γ k } .
Revised: As illustrated in Figure 1, our proposed CRT method involves the encoder network Φ , the prototype network A , and the multi-perspective feature embedding networks { Γ k } .

Original: These networks are jointlytrained in an end-to-end manner.
Revised: These networks are jointly trained in an end-to-end manner.

Original: The task of the anchor network A is to generate a set of anchorsfor efﬁcient projection of the input features.
Revised: The task of the prototype network A is to generate a set of prototypes for efﬁcient projection of the input features.

Original: From our experiments, we ﬁnd that it is important toenforce diversity among these anchors so that they can provide a diversiﬁed and robust representationof the image features.
Revised: From our experiments, we ﬁnd that it is important to enforce diversity among these prototypes so that they can provide a diversiﬁed and robust representation of the image features.

Original: Diversity means that the anchor features have small correlation with eachother.
Revised: Diversity means that the prototype features have small correlation with each other.

Original: Let C
Revised: Let C

Original: = { c 1 , c 2 , · · · , c K } be  set of anchors generated by the anchor network.
Revised: = { c 1 , c 2 , · · · , c K } be a set of prototypes generated by the prototype network.

Original: When trainingthe anchor network, we need to minimize the following average correlation between these anchorfeatures, which is deﬁned to be the anchor diversity loss
Revised: When training the prototype network, we need to minimize the following average correlation between these prototype features, which is deﬁned to be the prototype diversity loss

Original: [Equation]
Revised: [Equation]

Original: (3) where the cosine similarity is used to measure the correlation between two different anchor features.
Revised: where the cosine similarity is used to measure the correlation between two different prototype features.

Original: To train the overall feature embedding network, we use the multi-similarity (MS) loss [11], denotedby L MS , as our metric loss function to minimize the similarity for negative pairs and maximize thesimilarity for positive pairs.
Revised: To train the overall feature embedding network, we use the multi-similarity (MS) loss [11], denoted by L MS , as our metric loss function to minimize the similarity for negative pairs and maximize the similarity for positive pairs.

Original: Here, negative pairs are samples from different classes while positivepairs are samples from the same class.
Revised: Here, negative pairs are samples from different classes while positive pairs are samples from the same class.

Original: The anchor diversity loss L DIV and the MS loss L MS arecombined to form the overall loss function L =
Revised: The prototype diversity loss L DIV and the MS loss L MS are combined to form the overall loss function L =

Original: L DIV + λ 1 L MS for end-to-end training of our CRTnetwork.
Revised: L DIV + λ 1 L MS for end-to-end training of our CRT network. [Table] [Table]

Original: Cross CRT consistency loss.
Revised: Cross CRT consistency loss.

Original: To further enhance the generalization performance, we propose toenforce the feature distribution consistency between coded residual transforms with different sizesof projection anchors.
Revised: To further enhance the generalization performance, we propose to enforce the feature distribution consistency between coded residual transforms with different sizes of projection prototypes.

Original: As shown in Figure 3, we construct two CRT embedding branches withdifferent number of projection anchors.
Revised: As shown in Figure 3, we construct two CRT embedding branches with different number of projection prototypes.

Original: For example, in our experiment, the ﬁrst branch has ananchor set size K 1 = 48 with an embedded feature size of 128, while the second branch has ananchor set size K 2 = 64 with an embedded feature size of 1024.
Revised: For example, in our experiment, the ﬁrst branch has an prototype set size K 1 = 49 with an embedded feature size of 128, while the second branch has an prototype set size K 2 = 64 with an embedded feature size of 1024.

Original: Suppose that the current trainingbatch of images are I
Revised: Suppose that the current training batch of images are I

Original: = { I 1 , I 2 , · · · , I N } .
Revised: = { I 1 , I 2 , · · · , I N } .

Original: Let the feature embeddings of the ﬁrst CRT branch be{ f (1)1 , f (1)2 , · · · , f (1) N } and the feature embeddings of the second branch be { f (2)1 , f (2)2 , · · · , f (2) N } .[Table] [Table]
Revised: Let the feature embeddings of the ﬁrst CRT branch be { f (1)1 , f (1)2 , · · · , f (1) N } and the feature embeddings of the second branch be { f (2)1 , f (2)2 , · · · , f (2) N } .

Original: Deﬁne the similarity matrix
Revised: Deﬁne the similarity matrix

Original: [Equation]
Revised: [Equation]

Original: Similarly, we can deﬁne the similarity matrix for the second branch as S (2) .
Revised: Similarly, we can deﬁne the similarity matrix for the second branch as S (2) .

Original: The consistency betweenthese two CRT branches are deﬁned to be the L 1 distance between these two similarity matrices
Revised: The consistency between these two CRT branches are deﬁned to be the L 1 distance between these two similarity matrices

Original: [Equation]
Revised: [Equation]

Original: This consistency loss is then combined with the original loss function for the CRT network.
Revised: This consistency loss is then combined with the original loss function for the CRT network.

Original: Thus, theoverall loss is given by L =
Revised: Thus, the overall loss is given by L =

Original: L DIV + λ 1 L MS + λ 2 · L CON .
Revised: L DIV + λ 1 L MS + λ 2 · L CON .

Original: Generalization capability analysis.
Revised: Generalization capability analysis.

Original: In this section, we analyze the generalization capability of ourproposed CRT method.
Revised: In this section, we analyze the generalization capability of our proposed CRT method.

Original: Roth et al.
Revised: Roth et al.

Original: [22] established two metrics to measure the generalization capability of deep metric learning methods: (1) embedding space density and (2) spectral decay.
Revised: [22] established two metrics to measure the generalization capability of deep metric learning methods: (1) embedding space density and (2) spectral decay.

Original: Methodswith higher embedding space density and smaller spectral decay often have better generalizationcapabilities.
Revised: Methods with higher embedding space density and smaller spectral decay often have better generalization capabilities.

Original: The embedding space density metric D ESD is deﬁned as the ratio between the average ofintra-class distance D Intra and the average of inter-class distance D Inter :
Revised: The embedding space density metric D ESD is deﬁned as the ratio between the average of intra-class distance D Intra and the average of inter-class distance D Inter :

Original: [Equation]
Revised: [Equation]

Original: The spectral decay metric ρ SD is deﬁned to be the KL-divergence between the spectrum of d singularvalues V SV (obtained from Singular Value Decomposition, SVD) and a d -dimensional uniformdistribution µ d .
Revised: The spectral decay metric ρ SD is deﬁned to be the KL-divergence between the spectrum of d singular values V SV (obtained from Singular Value Decomposition, SVD) and a d -dimensional uniform distribution µ d .

Original: It is inversely related to the entropy of the embedding space:
Revised: It is inversely related to the entropy of the embedding space:

Original: [Equation]
Revised: [Equation]

Original: Table 1 and Table 2 show the embedding space density and the spectral decay values of the featureembedding learned by the baseline method and our CRT method on the test datasets used in ourexperiments.
Revised: Table 1 and Table 2 show the embedding space density and the spectral decay values of the feature embedding learned by the baseline method and our CRT method on the test datasets used in our experiments.

Original: We can see that our method achieves signiﬁcantly higher embedding space density andmuch lower spectral decay when the model tested on unseen classes, which indicates signiﬁcantlyimproved generalization capability.
Revised: We can see that our method achieves signiﬁcantly higher embedding space density and much lower spectral decay when the model tested on unseen classes, which indicates signiﬁcantly improved generalization capability.

Original:  Experimental Results
Revised:  Experimental Results

Original: Following the same procedure of existing papers [2, 7, 13, 18, 20], we evaluate the performance ofour proposed CRT method on four benchmark datasets for image retrieval tasks.
Revised: Following the same procedure of existing papers [2, 7, 13, 18, 20], we evaluate the performance of our proposed CRT method on four benchmark datasets for image retrieval tasks.

Original: 4.1 Datasets and Experimental Settings
Revised: 4.1 Datasets and Experimental Settings

Original: (1) Datasets.
Revised: (1) Datasets.

Original: In the following experiments, we use the same benchmark datasets as in existingpapers for direct performance comparison.
Revised: In the following experiments, we use the same benchmark datasets as in existing papers for direct performance comparison.

Original: Four datasets, i.e., CUB-200-2011 [57], Cars-196 [58],
Revised: Four datasets, i.e., CUB-200-2011 [57], Cars-196 [58],

Original: Standford Online Products (SOP)
Revised: Standford Online Products (SOP)

Original: [32], and In-Shop Clothes Retrieval (In-Shop) [59], are used toour experiments.
Revised: [32], and In-Shop Clothes Retrieval (In-Shop) [59], are used to our experiments.

Original: We use the same training and test split as in existing papers.
Revised: We use the same training and test split as in existing papers.

Original: Hyperameters weredetermined previous to the result runs using a 80-20 training and validation split.(2) Performance metrics and experimental settings.
Revised: Hyperameters were determined previous to the result runs using a 80-20 training and validation split. (2) Performance metrics and experimental settings.

Original: Following the standard protocol in [2, 7, 13,18, 20, 60], the Recall@K [61] is used to evaluate the performance of our algorithm.
Revised: Following the standard protocol in [2, 7, 13, 18, 20, 60], the Recall@K [61] is used to evaluate the performance of our algorithm.

Original: For all datasets,our method is evaluated with the image data only, without using the bounding box information.
Revised: For all datasets, our method is evaluated with the image data only, without using the bounding box information.

Original: During training, we randomly sample a set of images in each iteration to train the network.
Revised: During training, we randomly sample a set of images in each iteration to train the network.

Original: In eachiteration, we ﬁrst randomly choose the image classes, and then randomly sample 5 images fromeach class to form a batch.
Revised: In each iteration, we ﬁrst randomly choose the image classes, and then randomly sample 5 images from each class to form a batch.

Original: For the CUB and Cars datasets, we sampled 16 classes in a batch.
Revised: For the CUB and Cars datasets, we sampled 16 classes in a batch.

Original: Forthe SOP and In-Shop datasets, we sampled 36 classes in a batch.
Revised: For the SOP and In-Shop datasets, we sampled 36 classes in a batch.

Original: We apply random cropping withrandom ﬂipping and resizing to 227 × 227 for all training images.
Revised: We apply random cropping with random ﬂipping and resizing to 227 × 227 for all training images.

Original: For testing, we only use the center-cropped image to compute the feature embedding.
Revised: For testing, we only use the center-cropped image to compute the feature embedding.

Original: Unless otherwise speciﬁed, we use the
Revised: Unless otherwise speciﬁed, we use the

Original: MixTransformer-B2 (MiT-B2) developed in [48] as our baseline encoder network with its pyramid
Revised: MixTransformer-B2 (MiT-B2) developed in [48] as our baseline encoder network with its pyramid

Original: MLP head being removed.
Revised: MLP head being removed.

Original: Our CRT algorithm operates on the feature map generated by the lastblock (the 4-th block) of the backbone encoder network.
Revised: Our CRT algorithm operates on the feature map generated by the last block (the 4-th block) of the backbone encoder network.

Original: The initial learning rate is 3 e − 5 .
Revised: The initial learning rate is 3 e − 5 .

Original: For allimages, the MS loss weight λ 1 in the ﬁrst embedding branch is set as 1.0.
Revised: For all images, the MS loss weight λ 1 in the ﬁrst embedding branch is set as 1.0.

Original: In second embeddingbranch, it isset as 0.1 for the CUB and Cars datasets, and 0.9 for the SOP and In-Shop datasets. Theconsistency loss weight λ 2 is set to 0.9. The backbone network is pre-trained on the ImageNet-1Kdataset.
Revised: In second embedding branch, it is

Original: 4.2 Performance Comparisons with the State-of-the-art Methods
Revised: 4.2 Performance Comparisons with the State-of-the-art Methods

Original: We compare the performance of our CRT method with the state-of-the-art deep metric learningmethods.
Revised: We compare the performance of our CRT method with the state-of-the-art deep metric learning methods.

Original: Performance comparison results on the CUB and Cars datasets are shown in Table 3.
Revised: Performance comparison results on the CUB and Cars datasets are shown in Table 3.

Original: Table 4 shows the comparison results on the SOP and In-Shop datasets.
Revised: Table 4 shows the comparison results on the SOP and In-Shop datasets.

Original: It should be noted that thefeature dimension of our CRT method is set to be 128.
Revised: It should be noted that the feature dimension of our CRT method is set to be 128.

Original: We can see that our CRT method signiﬁcantlyoutperforms the current best deep metric learning method Hyp-DeiT [20].
Revised: We can see that our CRT method signiﬁcantly outperforms the current best deep metric learning method Hyp-DeiT [20].

Original: The results of the HypDeiT method are cited under the same experimental settings where the embedding size is 128 withimage size of 227 × 227, and a pretrained model on the ImageNet-1K.
Revised: The results of the HypDeiT method are cited under the same experimental settings where the embedding size is 128 with image size of 227 × 227, and a pretrained model on the ImageNet-1K.

Original: We choose this setting sothat we can compare our method with many recent papers on metric learning.
Revised: We choose this setting so that we can compare our method with many recent papers on metric learning.

Original: For example, on the
Revised: For example, on the

Original: CUB dataset, the top-1 recall rate has been improved by 4.28%.
Revised: CUB dataset, the top-1 recall rate has been improved by 4.28%.

Original: The experimental results on the restdatasets suggest that our CRT method consistently outperforms other deep metric learning methodsby large margins.[Table] [Table] As analyzed in [22] and [33]
Revised: The experimental results on the rest datasets suggest that our CRT method consistently outperforms other deep metric learning methods by large margins.

Original: [Table] most of the deep metric learning methods use different experimentalconditions, such as different settings of batch size, data augmentation methods, embedding sizes,training strategies, backbone networks [Table] and input image sizes, to obtain the state-of-the-art performance, which could lead to confusion on the effectiveness of those methods.
Revised: As analyzed in [22] and [33], most of the deep metric learning methods use different experimental conditions, such as different settings of batch size, data augmentation methods, embedding sizes, training strategies, backbone networks, and input image sizes, to obtain the state-of-the-art performance, which could lead to confusion on the effectiveness of those methods.

Original: For example, Hyp-DeiT[Table] [20] and IRT R [1] used larger batch sizes and memory bank technique to obtain better performance.
Revised: For example, Hyp-DeiT [20] and IRT R [1] used larger batch sizes and memory bank technique to obtain better performance.

Original: The method of XBM+RTT [21] used the re-rank technique to obtain the highest top-1 recall rateon the SOP dataset.
Revised: The method of XBM+RTT [21] used the re-rank technique to obtain the highest top-1 recall rate on the SOP dataset.

Original: The Group Loss++ [7] also used the re-rank method to improve the retrievalperformance.
Revised: The Group Loss++ [7] also used the re-rank method to improve the retrieval performance.

Original: Moreover, Proxy-Anchor [9], Group Loss++ [7] and ETLR [13] used mixed poolingmethods to improve embedding performance.
Revised: Moreover, Proxy-Anchor [9], Group Loss++ [7] and ETLR [13] used mixed pooling methods to improve embedding performance.

Original: During our experiments, we ﬁnd that it is challengingto re-produce their original results with our available computing resources and to conduct fair comparisons, especially for those methods without publicly available source code.
Revised: During our experiments, we ﬁnd that it is challenging to re-produce their original results with our available computing resources and to conduct fair comparisons, especially for those methods without publicly available source code.

Original: Instead, we implementour CRT method with the MS method [11] which has been extensively used in existing deep metriclearning methods.
Revised: Instead, we implement our CRT method with the MS method [11] which has been extensively used in existing deep metric learning methods.

Original: In the following, we provide a fair comparison between our method and thebaseline MS method with different backbone networks under the same experimental conditions. 
Revised: In the following, we provide a fair comparison between our method and the baseline MS method with different backbone networks under the same experimental conditions. [Table] [Table]

Original: 4.3 Performance Comparisons with Different Backbone Networks.
Revised: 4.3 Performance Comparisons with Different Backbone Networks.

Original: We conduct experiments with different backbone networks to verify the effectiveness of the proposed
Revised: We conduct experiments with different backbone networks to verify the effectiveness of the proposed

Original: CRT method.
Revised: CRT method.

Original: For fair comparisons [22, 33], we reproduce the MS method [11] and compare theperformance using the same experimental settings.
Revised: For fair comparisons [22, 33], we reproduce the MS method [11] and compare the performance using the same experimental settings.

Original: The hyper-parameter settings are consistent withthe original paper [11].
Revised: The hyper-parameter settings are consistent with the original paper [11].

Original: The batch size is set to 80 on the CUB and Cars datasets, and 180 on the SOPand In-Shop datasets.
Revised: The batch size is set to 80 on the CUB and Cars datasets [Table] and 180 on the SOP and In-Shop datasets.

Original: The embedded feature size is set to 128.
Revised: The embedded feature size is set to 128.

Original: Results are shown in Table 5, wecan see that our proposed method is able to consistently improve the performance of the baselinemethod.
Revised: Results are shown in Table 5, we can see that our proposed method is able to consistently improve the performance of the baseline method.

Original: It should be noted that the DeiT-S and MiT-B1 networks are both based on the transformer,a powerful method for exploiting the global correlation between features for very efﬁcient imagerepresentation.
Revised: It should be noted that the DeiT-S and MiT-B1 networks are both based on the transformer, a powerful method for exploiting the global correlation between features for very efﬁcient image representation.

Original: From Table 5, we can see that, even on top of these two baseline methods with thepowerful transformer backbone network, our coded residual transform is still able to improve themetric learning performance by up to 3.7%, which is quite impressive.
Revised: From Table 5, we can see that, even on top of these two baseline methods with the powerful transformer backbone network, our coded residual transform is still able to improve the metric learning performance by up to 3.7%, which is quite impressive.

Original: This is because our CRTmethod introduces the new ideas of multi-perspective projection and residual encoding, which havesigniﬁcantly improved the generalization performance of the deep metric learning.
Revised: This is because our CRT method introduces the new ideas of multi-perspective projection and residual encoding, which have signiﬁcantly improved the generalization performance of the deep metric learning.

Original: 4.4 Ablation Studies
Revised: 4.4 Ablation Studies

Original: In this section, we provide extensive ablation studies to further understand our proposed CRT methodand characterize the performance of major algorithm components.
Revised: In this section, we provide extensive ablation studies to further understand our proposed CRT method and characterize the performance of major algorithm components.

Original: In the following experiments,unless otherwise speciﬁed, the batch size N is set to 80.
Revised: In the following experiments, unless otherwise speciﬁed, the batch size N is set to 80.

Original: The backbone network used in thisexperiment is the MiT-B1.(1)
Revised: The backbone network used in this experiment is the MiT-B1. [Table] [Table] [Table]

Original:  Contributions of major algorithm components.
Revised: (1) Contributions of major algorithm components.

Original: From the performance evaluation perspective,our algorithm has two major components, the coded residual transform (CRT) and cross-CRTcorrelation consistency (CRT-Consistency).
Revised: From the performance evaluation perspective, our algorithm has two major components, the coded residual transform (CRT) and cross-CRT correlation consistency (CRT-Consistency).

Original: We ﬁx the MS loss weight λ 1 in the ﬁrst and secondembedding branches as 1.0 and 0.1, respectively.
Revised: We ﬁx the MS loss weight λ 1 in the ﬁrst and second embedding branches as 1.0 and 0.1, respectively.

Original: The consistency loss weight λ 2 is set to be 0.9.
Revised: The consistency loss weight λ 2 is set to be 0.9.

Original: From
Revised: From

Original: Table 6, we can see that the performance dropped 1.98% for the top-1 recall rate without the CRT.
Revised: Table 6, we can see that the performance dropped 1.98% for the top-1 recall rate without the CRT.

Original: Theperformance future dropped 1.69% for the top-1 recall rate without the CRT-Consistency, which isthe baseline result that obtained based on the MS loss function.
Revised: The performance future dropped 1.69% for the top-1 recall rate without the CRT-Consistency, which is the baseline result that obtained based on the MS loss function.

Original: Moreover, the multi-perspective CRTfeatures transformation plays an important role for the ﬁnal performance.
Revised: Moreover, the multi-perspective CRT features transformation plays an important role for the ﬁnal performance.

Original: For different projectionanchors, the CRT features can be transformed using different embedding networks or the same embedding network.
Revised: For different projection prototypes, the CRT features can be transformed using different embedding networks or the same embedding network.

Original: Here, we compare the performance of our CRT method with and without usingthe same embedding network for the transformation of CRT features.
Revised: Here, we compare the performance of our CRT method with and without using the same embedding network for the transformation of CRT features.

Original: Results are shown in Table 7.
Revised: Results are shown in Table 7.

Original: We can see that better performance can be obtained using different embedding networks for each CRTfeature corresponding to each projection anchor.
Revised: We can see that better performance can be obtained using different embedding networks for each CRT feature corresponding to each projection prototype.

Original: This shows the effectiveness of our multi-perspectiveprojection.
Revised: This shows the effectiveness of our multi-perspective projection.

Original: [Table]

Original: [Table]

Original: (2) Impact of the projection anchors.
Revised: (2) Impact of the projection prototypes.

Original: In this experiment, we analyze the impact of different numbersof projection anchors.
Revised: In this experiment, we analyze the impact of different numbers of projection prototypes.

Original: Table 8 and Table 9 show the results of different numbers of projection anchorsfor the ﬁrst embedding branch (low-dimensional embedding branch) and the second embeddingbranch (high-dimensional embedding branch), respectively.
Revised: Table 8 and Table 9 show the results of different numbers of projection prototypes for the ﬁrst embedding branch (low-dimensional embedding branch) and the second embedding branch (high-dimensional embedding branch), respectively.

Original: In Table 8, the number of projectionanchors in the second embedding branch is ﬁxed at 64.
Revised: In Table 8, the number of projection prototypes in the second embedding branch is ﬁxed at 64.

Original: This table reports the results of embeddingcomputed from the ﬁrst embedding branch when the number of projection anchors changed from 1to 64.
Revised: This table reports the results of embedding computed from the ﬁrst embedding branch when the number of projection prototypes changed from 1 to 64.

Original: The weights of the backbone networks are shared and the weights of the CRT heads are notshared for this experiment.
Revised: The weights of the backbone networks are shared and the weights of the CRT heads are not shared for this experiment.

Original: In Table 9, the number of project anchors are equal for the ﬁrst and thesecond CRT embedding branches, which are changed from 1 to 100, and the weights of both thebackbone network and the CRT branches are shared in this experiment.
Revised: In Table 9, the number of project prototypes are equal for the ﬁrst and the second CRT embedding branches, which are changed from 1 to 100, and the weights of both the backbone network and the CRT branches are shared in this experiment.

Original: We can see that when thenumber of anchors is set to be 64, the embedding performance reaches to the best.
Revised: We can see that when the number of prototypes is set to be 64, the embedding performance reaches to the best.

Original: In Table 9, the network weight of the second embedding branch is shared from the ﬁrst embeddingbranch.
Revised: In Table 9, the network weight of the second embedding branch is shared from the ﬁrst embedding branch.

Original: Here, we compare the performance of our CRT method with and without shared weightsbetween these two embedding branches.
Revised: Here, we compare the performance of our CRT method with and without shared weights between these two embedding branches.

Original: Results are shown in Table 10.
Revised: Results are shown in Table 10.

Original: We can see that theperformance remains almost the same between these two study cases.
Revised: We can see that the performance remains almost the same between these two study cases.

Original: In other experiments, todecrease the memory consumption, we shared weights for all the experiments.
Revised: In other experiments, to decrease the memory consumption, we shared weights for all the experiments.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we have developed a coded residual transform for generalizable deep metric learning, which consists of a multi-perspective projection and coded residual transform encoder and across-CRT correlation consistency constraint.
Revised: In this paper [Table] we have developed a coded residual transform for generalizable deep metric learning, which consists of a multi-perspective projection and coded residual transform encoder and a cross-CRT correlation consistency constraint.

Original: It has two unique characteristics.
Revised: It has two unique characteristics.

Original: First, it representsand encodes the feature map from a set of complimentary perspectives based on projections ontodiversiﬁed anchors.
Revised: First, it represents and encodes the feature map from a set of complimentary perspectives based on projections onto diversiﬁed prototypes.

Original: Second, unlike existing transformer-based feature representation approacheswhich encode the original values of features based on global correlation analysis, the proposed codedresidual transform encodes the relative differences between original features and their projectedanchors.
Revised: Second, unlike existing transformer-based feature representation approaches which encode the original values of features based on global correlation analysis, the proposed coded residual transform encodes the relative differences between original features and their projected prototypes.

Original: The proposed CRT method has achieved new state-of-the-art metric learning performanceon benchmark datasets.
Revised: The proposed CRT method has achieved new state-of-the-art metric learning performance on benchmark datasets.

Original: We hope our method can motivate further research.
Revised: We hope our method can motivate further research.

Original: One limitation is thatthe memory and compute usage will be increased during training for these two embedding branches,and we shared weights between them to solve this problem in our experiments.
Revised: One limitation is that the memory and compute usage will be increased during training for these two embedding branches, and we shared weights between them to solve this problem in our experiments.

Original: Another limitationis that the projection anchors were learned from the training set.
Revised: Another limitation is that the projection prototypes were learned from the training set.

Original: It is unclear whether it is the bestprojection anchors for new test classes.
Revised: It is unclear whether it is the best projection prototypes for new test classes.

Original: We leave it for the future work.
Revised: We leave it for the future work.

Revised: Acknowledgments

Revised: This work was supported in part by the National Key R&D Program of China 2021YFE0110500, in part by the National Natural Science Foundation of China under Grant 62202499, 61872034, 62062021 and 62011530042, in part by the Hunan Provincial Natural Science Foundation of China under Grant 2022JJ40632, in part by the Beijing Municipal Natural Science Foundation under Grant

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
Revised: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes] Abstract and Introduction (b) Did you describe the limitations of your work?
Revised: [Yes] Abstract and Introduction (b) Did you describe the limitations of your work?

Original: [Yes] Conclusion (c) Did you discuss any potential negative societal impacts of your work?[Yes]
Revised: [Yes] Conclusion (c) Did you discuss any potential negative societal impacts of your work?

Original:  (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?[Yes]
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original:  Supplemental material (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?[Yes]
Revised: [Yes] Supplemental material (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original:  (c) Did you report error bars (e g., with respect to the random seed after running experi- ments multiple times)?[Yes]
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original:  (d) Did you include the total amount of compute and the type of resources used (e g., type of GPUs, internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 
Revised: [Yes] 4.

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: [No] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?

Original: [N/A] 
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if[Table] applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] ) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount[Table] spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]