DenoiseRep: Denoising Model for Representation Learning
Authors: zhengrui Xu
Guan'an Wang
Xiaowen Huang
Jitao Sang
Venue: NeurIPS 2024 oral (2024)
Keywords: Diffusion Model
Representation Learning
Generative Model
Discriminative Models
Primary area: diffusion_based_models
TL;DR: DenoiseRep is a computation-free, label-optional and model-irrelevant algorithm to incrementally improve representation learning.

Abstract: The denoising model has been proven a powerful generative model but has little exploration of discriminative tasks. Representation learning is important in discriminative tasks, which is defined as *"learning representations (or features) of the data that make it easier to extract useful information when building classifiers or other predictors"*. In this paper, we propose a novel Denoising Model for Representation Learning (*DenoiseRep*) to improve feature discrimination with joint feature extraction and denoising. *DenoiseRep* views each embedding layer in a backbone as a denoising layer, processing the cascaded embedding layers as if we are recursively denoise features step-by-step. This unifies the frameworks of feature extraction and denoising, where the former progressively embeds features from low-level to high-level, and the latter recursively denoises features step-by-step. After that, *DenoiseRep* fuses the parameters of feature extraction and denoising layers, and *theoretically demonstrates* its equivalence before and after the fusion, thus making feature denoising computation-free. *DenoiseRep* is a label-free algorithm that incrementally improves features but also complementary to the label if available. Experimental results on various discriminative vision tasks, including re-identification (Market-1501, DukeMTMC-reID, MSMT17, CUHK-03, vehicleID), image classification (ImageNet, UB200, Oxford-Pet, Flowers), object detection (COCO), image segmentation (ADE20K) show stability and impressive improvements. We also validate its effectiveness on the CNN (ResNet) and Transformer (ViT, Swin, Vmamda) architectures.

DenoiseRep: Denoising Model for Representation Learning
Zhengrui Xu
Guan'a Laan'an Wang f3 guan. wwng07060gmal..comm
Xiaowen Huang 1,2,3 xabuangubjtu.. edu.ccm
Jitao Sang 1,2,3
zrxu23dijtu. edd.cc.cc
jtsangubjtu.
Schooo of Computer Science and Technology, Beijing Jiaotong University Beijjig Key Lab of Traffic Data Analysis and Mining, Beijing Jiaotong University KKey Laboratory of Big Data & Artificial Intelligence TransportaaionnBeiiing Jiaotong University), Ministry of Education
Abstract
The denoising model has been proven powerful generative model but has little exploration discriminative tasks. Representation learning important in dis- criminative tasks, which is defined as "learning representations (or features) of the data that make easier to extract useful information when building classifiers other predictors" [4] In this paper, we propose novel Denoising Model for Representation Learning (DenoiseRep) to improve feature discrimination with joint feature extraction and denoising. DenoiseRep views each embedding layer in a backbone as a denoising layer, processing the cascaded embedding layers as if recursively denoise features step-by-step. This unifies the frameworks of feature extraction and demoising, where the former progressively embeds fea- tures from low-level to high-level, and the latter recursively denoises features step-by-step. After that, DenoiseRep fus es the parameters feature extraction and denoising layers, and theoretically demonstrates its equivalence before and after the fusion, thus making feature denoising computation-free DenoiseRep is a label-free algorithm that incrementally improves features but also complementary to the label if available. Experimental results on various discriminative vision tasks, including re-identification sharket-1501,, DukeMTMC-reIDD. MSMT17, CUHK-03, vehicleID), image classification (ImageNet, UB200, Oxford-Pet, Flow- ers). object detection (COCO), image segmentation (ADE20K) show stability and impressive improvements. We also validate its effectiveness on the CNN (ResNet) and Transformer (ViT, Swin, Vmamda) architectures. Code available at https///gghabccommaaaaaaaaa//aen
1 Introduction
Denoising Diffusion Probabilistic Models (DDPM) or Diffusion Model for short have been proven to be a powerful generative model (5). Generative models can generate vivid samples (such as images, audio and video) by modeling the joint distribution of the data P(X,Y) where X the sample and Y is the condition. Diffusion models achieve this goal by adding Gaussian noise to the data and training denoising model of inversion to predict the noise. Diffusion models can generate multi-formity and rich samples, such as Stable diffusion [50), DALL [47] series and Midjourney, these powerful image generation models, which are essentially diffusion models.
Equal Contribution.
Project Lead.
Corresponding Author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Trainale
[b)
Layer
Denoise Procass (Dencis Lsnse)
Fentues
Dencise Layer
Breiskinng Loye
Images
Feeture Extraction Pracess of Backbone
Runves
Hobe
Process fionabe lsnaa)
Ranvess
Clcan
(Fogresshey peooed features with Mlyyrr!
D 10-0-
I 1
O
100-
(c)
omages
Images
Clea
United Processes Feature Extraction and Denotong with extra N denatse ttyers
United Pracenses Feature Extraction Denolning with denotte tayers hited (Pogrestieel enteed H Wayerr descite tines, ao compacation)
Paotove
(Paggeshhey embed N lyers and dionalse dicleee require move (omputation)
brief description of our idea. (a) A typical denoising model for generative tasks recur- sively applies denoising layer. (b) A naive idea that applies denoising strategy to discriminative model is applying a recursive denoise layer on the feature of a backbone and taking extra inference latency. (c.d) Our DenoiseRep first unifies the frameworks of feature extraction and denoising in backbone pipeline, then merges parameters of the denoising layers into embedding layers, making the feature more discriminative without extra latency cost.
Figure 1:
However, its application discriminative models has not been extensively explored. Different from generative models, discriminative models predict data labels by modeling the marginal distribution of the data P(Y]X). Y can be various labels, such as image tags for classification, object boxes for detection, and pixel tags for segmentation. Currently, there are several methods based on diffusion models implemented in specific fields. For example, DiffusionDet ( is a new object detection framework that models object detection as denoising diffusion process from noise boxes to object boxes. It describes object detection generative denoising process and performs well compared to previous mature object detectors. DiffSeg 52 for image segmentation, which is a method of unsupervised zero-shot sample segmentation using pre-trained models (stable diffusion). introduces simple and effective iterative merging process to measure the attention maps between KL divergence and merge them into an effective segmentation mask. The proposed method does not require any training or language dependency extract the quality segmentation of any image.
The methods above are carefully designed for specific tasks and require particular data structure. For example, DiffusionDet [7] uses noise boxes and DiffSeg [52] uses noise segmentation. In this paper, we explore more general conception of how the denoising model can improve representation learning, i.e. "learning representations (or features) of the data that make it easier to extract useful information when building classifiers or other predictors"
[4, and contribute discriminative models. We take person Re- ee-Ideniification (ReID) [66, 33 as a benchmark task. RelD aims to match images of a pedestrian under disjoint cameras, and suffered by pose, lighting, occlusion and so on, thus requiring more
feature.
straightforward approach is applying the denoising process a backbone's final feature 26 14
reducing noise in the final output and making the feature more discriminative, as Fig. [b) shows. However, this way can be computationally intensive. Because the denoising layer needs to be proceeded on the output of the previous one in recursive and step-by-step manner. Considering that backbone typically consists of cascaded embedding layers (e.g., convolution layer, multi-head attention layer), we propose novel perspective: treating each embedding layer as a denoising layer. As shown in Fig. [T.), allows to process the cascaded layers as we are recursively proceeding through the denoising layer step-by-step This method transforms the backbone into series of denoising layers, each working on different feature extraction level. While this idea intuitive and simple, its practical implementation presents significant challenge. The main issue arises from the requirement of the denoising layer for the input and output features exist the same feature space. However, in typical backbone (e.g. ResNet (26), VIT [4)). the layers progressively map features

---

from a low level to high level. It means that the feature space changes from layer to layer, which contradicts the requirement of the denoising layer.
To resolve all the difficulties above and efficiently apply the denoising process to improve discrimina- tive tasks, our proposed Denoising Model for Representation Learning (DenoiseRep) is below:
Firstly, we utilize a well-trained backbone and keep it fixed throughout all subsequent procedures. This step is a free launch as we can easily use any publicly available backbone without requiring additional training time. This approach allows us to preserve the backbone's 's inherent characteristics of semantic feature extraction. Given the backbone and an image, we can get list of features. Next, we train denoising layers on those features. The weights of denoising layers are randomly initialized and their weights are not shared. The training process is the same as that in DDPM (21), where the only difference is that the denoising layer in DDPM takes dynamic t E [1. T. and our denoising denoise times and N is backbone layer
layers take fixed n E [1, N. where n is the layer index, number as shown in Fig. c). Finally, considering that the N denoising layers consume additional execution latency, we propose novel feature extraction and feature denoising fusion algorithm. As shown in Fig. []) the algorithm merges parameters of extra denoising layers into weights of the existing embedding layers, thus enabling joint feature extraction and denoising without any extra computation cost. We also theoretically demonstrate the total equivalence before and after parameter fusion. Please see Section 3.3 and Eq for more details.
Our contributions can be summarized as follows:
(1) We propose novel Denoising Model for Representation Learning (DenoiseRep), which innova- tively integrates the denoising process, originating from generative tasks, into the discriminative tasks. It treats N cascaded embedding layers of a backbone T times recursively proceeded denoising layers. This idea enables joint feature extraction and denoising is a backbone, thus making features more discriminative.
(2) The proposed DenoiseRep fuses the parameters the denoising layers into the parameters of the corresponding embedding layers and theoretically demonstrates their equivalence. This contributes to a computation efficcient algorithm, which takes no extra latency.
(3) Extensive experiments on 4 ReID datasets verified that our proposed DenoiseRep can effec- tively improve feature performance in label-free manner and performs better in the case of label- argumented supervised training or introduction of additional training data. We also extend DenoiseRep to large-scale (ImageNet), fine-grained (CUB200, Oxford-Pet, Flowers) image classifications, object detection (COCO) and image segmentation (ADE20K), showing its scalability.
2
Related Work
Generative models learn the distribution of inputs before estimating class probabilities. generative model learns the data generation process by learning the probability distribution of the input data and generating new data samples. The generative models first estimate the conditional density of categories P(æly = k) and prior category probabilities P(y = k) from the training data. The P(x) obtained by the full probability formula. So as to model the probability distribution of each type of data. Generative models can generate new samples by modelling data distribution. For example, Generative Adversarial Networks (GANs) I7 43 23 and Variational Autoencoders
(VAEs) [4. 53 69 64 are both classic generative models that generate real samples by learning po- tential representations of data distributions, demonstrating excellent performance in data distribution modeling. Recent research has focused on using diffusion models for generative tasks. The diffusion model was first proposed by the article (51) in 2015, with the aim of eliminating Gaussian noise from continuous application to training images. The DDPM (21) proposed in 2020 have made the use of diffusion models for image generation mainstream. In addition to its powerful generation ability, the diffusion model also has good denoising ability through noise sampling, which can denoise noisy data and restore its original data distribution.
Discriminative models learn condition distribution, f.e. P(y]z), where x is data and y is task-specific features. For example, classification tasks 2 map data to tags, retrieval tasks (36 62 map data to feature space where similar data should be near otherwise faraway, detection task 449 27
map data to space position and size. Person Re-Identification (ReID) a fine-grained retrieval task which identifies individuals among disjoint camera views. Considering its challenge to feature
3

---

discrimination, we take RelD as the major benchmark task and the others as auxiliary benchmarks.
Existing ReID methods can be grouped into hand-crafted descriptors 335
42
65
incorporated with
metric learning [25
34 71 and deep learning algorithms 58 57
56
44
16
10]
State-of-ther
art Rell models often leverage convolutional neural networks (CNNs)
28
to capture intricate
spatial relationships and hierarchical features within person images. Attention mechanisms 14],
spatial-tempont modeling [31 [0], and domain adaptation techniques n have further enhanced the adaptability of ReID models to diverse and challenging real-world scenarios.
3 DenoiseRep: Denoising Model for Representation Learning
3.1 Review Representation Learning
Representation learning plays a pivotal role in discriminative tasks, which is defined "learning representations features) of the data that make it easier to extract useful information when building classifiers or other predictors" backbone to extract discriminative features (e.g., ResNet [88), VIT [4] and a task-specific head that operates on these features (e.g. MLP [26] for classification, RCNN for object detection, FCN [40] for segmentation). It is evident that the vision backbone is central to representation learning. In this paper, we introduce a novel Denoising Model for Representation Learning (DenoiseRep), which integrates feature extraction and feature denoising within single vision backbone. This
A common architecture of discriminative tasks consists of vision
approach aims to enhance the discriminative power of the features extracted.
3.2 Joint Feature Extraction and Feature Denoising
We refer to the diffusion modeling approach to denoise the noisy features through T-steps to obtain clean features. At the beginning, we use the features output from the backbone network data samples for diffusion training. and get the noisy samples by continuously adding noise and learning through the network order simulate the data distribution of its features.
(Xx:: (0) = II (x-1)
t=1
q(xe(xx-1) = (x1; / Bix,-1,
where Xo represents the feature vector output by the backbone, represents the diffusion step size, 3 a set of pre-set parameters, and X, represents the noise sample obtained through diffusion process.
In the inference stage, as shown in Fig. [b), we perform T-step denoising on the output features, to obtain cleaner features and improve the expressiveness of the features.
T
Po (X == p(xr) II (x+- (x+-
(3)
po po(xx-1(x)) = Pi (), Eag (xxxx)))
(4)
where X, represents the feature vector output by the backbone in the inference stage. T' is the denoising step size, representing the magnitude of the noise. We adjust t appropriately based on different datasets and backbones to obtain the optimal denoising amplitude. According to po (x,-1/x)) denoise it step by step, and finally obtains Xg. which represents the clean feature after denoising.
3.3 Fuse Feature Extraction and Feature Denoising
As described Section 3.2 the proposed method above could effectively improve the discriminability of features. Still, extra inference latency is introduced caused by recursive calling of the denoising layers. To solve the problem, we propose to fuse parameters of feature denoising layers into parameters of existing embedding layers of the backbone. The core idea is to expand the linear layer each transformer encoder block into two branches, one for its original embedding layer and the other for extra denoising layer. As shown in Fig. during the training phase, we freeze the original embedding layers and only train the denoising layers. The training method consistent with section
3.2 and the features diffused and fed into the denoising layers. Please refer to Algorithm I for

---

Transformer Encoder
Denoise Layer
ReLU
e FeedForward
Phedictedd (eow
Re-Paramed Linear (r=W3+b)
Norm
Naive Linear (V-W00mm))
Gaussian Noise
Denoise (X-Wiu))
Diffusion
Novee
ReLU
Multi Head
Attention
RoLU
Re: Paramaarddd Linear
Linear
Norm
Infer Only
Train Only
Denoise Layer
T
Embedded
④ Petches
Freeze
df Trainable
[ IMGS
Figure 2: Pipeline our proposed DenoiseRep. VIT consists of N cascaded transformer encoder layers. During the training phase (see the right side "Train Only" process), we freeze the backbone parameters and only train the extra denoising layers. In the inference stage (see the left side "Infer Only" process), merge the parameters of denoising layers corresponding encoder layers. So there no extra inference latency cost. Please find definitions of W. b. WD. W', and // in Algorithm
more details. In the inference stage, we fuse the pre-trained parameters of embedding and denoising layers, merging the branches into single branch without additional inference time. Please note that, here we take the transformer architecture as an example, but DenoiseRep is suitable for CNN architecture. We demonstrate its scalibity on CNN experiments. The derivation of parameter merging is as follows:
1- at VI di
X,-
(X;
(X((,,)) ozz
(5)
Va
where at = 1 - Bt, Do are the parameters of the prediction noise network.
Y = W +
02 a,
X1 X =
va DoX;
022
V
(6)
Y, Vaz
Y
/ D&Y, O+ž
We make simple transformation of Eq. ( and multiply both sides simultancously by W. The simplified equation can be obtained by bringing Y, in terms of W X, + b:
Y
IW (() + CC2 + a+
Cz(() 1 1 0i-1 3,
(7)
(t) = vav1I
1- 1
C3 = 2 ~ N(0,1)
where WD denotes the parameters of (D t), Xx denotes the input of this linear layer, Y, denotes the output of this linear layer, and Y.- denotes the result after denoising in one step of Y. Due the cascading relationship of blocks, as detailed in Algorithm. 2 different f values are set according to the order between levels, and the one-step denoising of one layer is combined achieve the denoising process of Y + Yo. ensuring the continuity of denoising and ultimately obtaining clean features. We split the original single branch into a dual branch structure. During the training phase, the backbone maintains its original parameters and needs to train the denoising module parameters. In the inference stage, as shown on the left side of Fig. 2 we use the method of reparameterization, to replace the original W parameter with W where W [W
Ci (t) in Eq. (7). which has the same
5

---

Algorithm Training
Input:
The number of feature layers the backbone N, features extracted from each layer {Fi L.V the denoising module that needs be trained (Di())=r
1: repeat
2: for each i E [NV, 1] do
3:
f == i: Specify the diffusion step for the current layer based on the order of layers.
4:
~ N (0, I): Randomly sample a Gaussian noise.
5:
X; Va + v1 a,c: Forward diffusion process in Eq.
6:
Take gradient descent step on V ()(,,
7: end for
8:
until converged
number of parameters as W. thus achieving the combination of FC operation and denoising without additional time cost. a Computation-free method.
In Eq. we achieve one-step denoising Y; - Y,-1. we need to increase the denoising amplitude, extend it two-step multi-step denoising. The following is the derivation formula for two-step denoising:
1 Y, - Y C())DC( 0+2
(8)
Vat
Yi-1 Y = C1(t 1 1)Da 01-12
(9)
a
We can obtain this by eliminating Yi-1 from Eq. 8 and Eq. and replacing Y7 with W WX,, + b:
Yı- = - X
W
w"
Va+
Va:
(10)
- -)CL
C"
Note that a single module completes two steps of denoising. To ensure the continuity of denoising, the & value should be sequentially reduced by 2.
Our proposed DenoiseRep based on feature-Ievel denoising and can be migrated various downstream tasks. It denoises the features on each layer for better removal of noise at each stage, as the noise in the inference stage comes from multiple sources, which could noise in the input image noise generated while passing through the network. Denoising each layer avoids noise accumulation and gives better quality output. And according to the noise challenges brought by data in different scenarios, the denoising intensity can be adjusted by controlling I, 3,, and the number of denoising times, which has good generalization ability.
Algorithm 2 Inference
Input:
The number of feature layers the backbone N, features extracted from each layer {F& (F trained demoising module parameters [N in Algorithm(1}, after obtaining the initial feature FN through parameters [W.).Y, and [4 for the backbone.
is necessary to remove N-step noise from it, the pre-trained
Output: Feature FOG after denoising.
I: for each i [N, 1] do
2: = i: Set the denoising amplitude based on the depth of the current layer.
3: W' [W; C₁ (t),,,,,, = W, (t) b,: Parameter fusion according to Eq
4: F = W" Ft. + Fuse feature extraction and feature denoising.
5: end for
6: return F

---

3.4 Unsupervised Learning Manner
Our proposed RenoiseRep is label-free because its essence is generative model that models data by learning its distribution. Thus the training loss contains only the Lossp of denoising layers:
N
Lossp
6 (X 4i)
(11)
where denotes the sampled noise, N denotes the number of denoising layers, X, denotes the noise sample, denotes the diffusion step, and Do (X,,t) denotes the noise predicted by the denoising layer.
However, it is worth noting that method is complementary label if the label is available. Losst is the task-specific supervised loss with label, À is the trade-off parameter between two losses. The subel-argumented learning is defined as:
Loss = (1 - Xooss + XLossp
(12)
Results in experiments Section 4.1 shows the improvement from label.
4 Experiments
Table Experimental results on various discriminative tasks.
Task
Model
Backbone
Dataset
Metric
Baseline
*DnooseRep
Classification
Swin'T
39
V2-n
ImageNet-Ik
acc@
81.82%
82.13%
Person-ReID
TransReID-SSL
VIT-S
MSMTI7
mAP
66.30%
67.33%
Detection
Mask-RCNI
19
Swin-T
COCO
AP
42. 42.80%
44.30%
Segmentation
FCN
40
ResNet-50
ADE20K
BloU
28.70%
29.90%
Our proposed DenoiseRep a versatile method that can be incrementally applied various discrimi- that DenoiseRep yields stable and substantial improvements across
native tasks. image classification, object detection, image segmentation, and person re-identification. Given that person re-identification is nuanced image retrieval task that poses greater challenge to feature discriminability, we take it as our benchmark for model analysis. Details of the experimental set- tings provided in Appendix A Additional experimental results on various tasks are presented in Appendices
D and
4.1 Analysis of Label Informations
Table DenoiseRep label-free method that can also be effectively complemented with labels when they are available. The table below analyzes the effectiveness of using labels. The baseline method, TranSRel is based on a ViT-small backbone. "Label-free" indicates training without labels, "'abel-augmented refers to the use of labels, and "merged dataset" denotes the use of combined datasets without labels.
Method
DukeMTMC(S0)
MSMTI7(%)
Market1501(9))
CUHK-03(%%
TIDSS
81.20
66.30
91.20
83.50
+Dnooiseeep (label-free)
81.72 (1 0.52)
66.87 (1 0.57)
91.82 (1 0.62)
83.72 (1 0.22)
+DenoiseRep (label-aug)
82.12 (1 0.92)
67.33 (f 1.03)
92.05 (1 0.85)
84.11 (1 0.61)
+DenoiseRep (merged ds)
81.78 (1 0.58)
66.99 (1 0.69)
91.80 (1 0.60)
83.86 (1 0.36)
DenoiseRep is unsupervised denoising module, and its training does not require the assistance label information. We conducte the following experiments to identify three key issues.
As mentioned in Section 3.3
7

---

(1) Is this label-free and unsupervised training denoising plugin effective? As shown in Table 2 (line2), compared with baseline method (linel), the baseline method performs better after adding our label-free plugin, which shows that our method does have denoising capability for features.
(2) Could introducing label information for supervised training further improve performance? Introducing label information is actually adding Loss: as mentioned in Section as a supervised signal. As shown Table 2 (line3), baseline method with label-argumented DenoiseRep achieve improvements of 0.32% 0. 0.70% on the mAP metric, indicating that our denoising plugin has label compatibility, in other words, the plug-in effective for feature denoising regardless of label- argumented supervised or lable- free unsupervised training.
(3) Since our plugin can perform unsupervised denoising of features, it is natural to think about whether adding more data for training the plugin could further improve its performance? We merge four datasets for training and then test on each dataset using mAP to evaluate. Comparing the results of training on sigle dataset (line2) with on merged datasets (line4), we found that adopting other datasets for unsupervised learning can further improve the performance of DenoiseRep, which also proves that DenoiseRep has good generalization ability.
To demonstrate that our method can perform unsupervised learning and has good generalization, we merged four datasets and rearranged the sequence IDs to ensure the reliability of the experiment. The model is tested on the entire dataset. During the training process, we freeze the baseline parameters and only train the DenoiseRep module, without the need for labels, for unsupervised learning. Then test on single dataset and compare the results of training on a single dataset. As shown in Table 2 it can be observed that adding unlabeled training data from different datasets can improve the model's performance single dataset, proving that this module has certain degree of generalization.
4.2 Comparison with State-of-the-Art ReID Methods
We compare several state-of-the-art RelD methods four datasets. One of the best performing comparison methods is TransRelD-SSLL. which is a series of RelD methods based on the ViT backbones. Other methods are based on structures such as CNNs. We add our method to TransRelD- SSL series and observe their performance. As shown in Table 3 we have the following findings:
Table 3: Comparison with state-of-the-art Rell methods.
MSMT17
Mar
DMK
CUHK03-L
Method
Backbone
mAP
RI
mAP
R1
mAP
RI
mAP
RI
MGN 59
ResNet-50
86.90
95.70
78.40
88.70
67.40
68.00
OSNet F4
OSNet
52.90
78.70
84.90
94.80
73.50
88.60
BAT-net 15
GoogLeNet
56.80
79.50
87 40
95.10
77.30
87.70
76.10
78.60
ABD-Net 8
ResNet-50
60.80
82.30
88.30
95.60
78.60
89.00
RGA-SC
[68
ResNet-50
57,50
80.30
88.40
96.10
77.40
81.10
ISP [76
HRNet-W33
88.60
95.30
80.00
89 60
74.10
76.50
CDNet 29
CDNet
54.70
78.90
86.00
95.10
76.80
88.66 60
Nforer [00
ResNet-50
59.80
77.30
91.10
94.70
83.50
89.40
78.00
77.20
TransReID 20
ViT-base-ics
67.70
85.30
89.00
95.10
82,20
90.70
84.10
86.40
TransRelD
ViT-base
61.80
81.80
87.10
94.60
79.60
89.00
82.30
84.60
TransRcID-SSL_
ViT-small
66.30
84.80
91.20
95.80
81.20
87.80
83.50
85,90
TransReID-SSL
ViT-base
75.00
89.50
93.10
96.52
84.10
92.60
87.80
89.20
CLIP-REID 32
ViT-base
75.80
89.70
90.50
95.40
8
90.80
TransRelD + RenoiseRep
ViT-base-ics
68.10
85.72
89.56
95.50
82.35
90.87
84.15
86.39
TransRelD + DenoiseRep
VIT-hase
62.23
82.02
87.25
94.63
80.
89.33
82.44
84.61
TransReID-SSL. DDnoiseeee
ViT-small
67.33
85.50
92.05
96.68
82.12
88.72
84.11
86.47
MausReID-SSL- Den
biTe
75.35
89.62
93.26
96.55
84.31
92.90
88.08
89.29
CLIP-REID DenoiseRep
ViT-hase
76.30
90.60
91.10
95.80
83.70
91.60
(1) Our method stands out on four datasets on ViT-base backbone with a large number of parameters, achieving almost the best performance two evaluation metrics.
(2) The methods using our plugin outperforms the original methods with the same backbone on all datasets. In addition, the performance improvement of small-scale backbones with the addition of Rep more significant than the large-scale backbones approach due to the fact that DenoiseRep is essentially denoising module that removes the noise contained in the features during
8

---

the inference stage. For large-scale backbones, the extracted features already have good performance, so the denoising amplitude is limited. has already fitted the dataset well. For small-scale backbones with poor performance, due to their limited fitting ability, there is certain amount of noise in the extracted features during the inference stage. Denoising them can obtain better feedback.
(3) In fact, our method can be applied to any other backbone, just add it to each layer. In particular, the performance improvement of adding the denoising plugin to poorly performing backbone might be more significant. This needs to be further verified in subsequent work. However, it is undeniable that we have verified the denoising ability of the DenoiseRep in the currently optimal RelD method.
In this section, a comparative analysis was conducted on four datasets to assess various existing ReID methods. These methods represent current mainstream ReID approaches, employing ResNet101, VIT- S, ViT-B, and ResNet50 as backbone architectures for feature extraction, respectively. Experimental results indicate that our proposed method outperforms other approaches in terms of both mAP and Rank-1 metrics.
4.3 Analysis of Parameter Fusion
The proposed DenoiseRep is computation-free In section 3.3 we proved by theoretical derivation that inserting our denoising layer into each feature layer and fusion it does introduce additional computation. In this section, we also conduct related validation experiments, the results of which are shown in Table 4
Table 4: Parameter Fusion Performance Analysis. The DenoiseRep denoises based on the features of the final layer, while the RenoiseRep denoises based on the features of each layer. The baseline method -rSSS is based on ViT-small backbone.
Method -SSSS
DukeMTMC
MSMT17
Market 501
CUHK-03
Inference Time
81.20%
66.30%
91.20%
83.50%
0.34s
+DenoiseRep
81.56%
66.81%
91.07%
83.59%
0.39s (+15%)
ReenoiseRep 82.12%
67.33%
92.05%
84.11%
0.34s (+0%)
Compare the baseline method TransReID-SSL, adding DenoiseRep" is able to improve the the performance, proving that feature based denoising is effective. However, also brings extra inference latency (about 15%) because is adding extra parameter-independee denoising module the end of the model.
Adopting DenoiseRep achieves a greater increase, it denoise the features on each layer, which can better remove noise at each stage because the noise in the inference stage comes from multiple aspects, which may be the noise in the input image generated when passing through the network. Denoising each layer avoids noise accumulation and obtains better quality output. Most importantly, since the operation of fusion can merge the parameters of the denoising module with the original parameters, the adoption DenoiseRep does not take extra inference latency cost, which is a computation-free efficient approach.
4.4 Experiments on Classification Tasks
The DenoiseRep is based on denoising at the feature level and demonstrates strong generalization capabilities. To validate this generalization ability, we conduct experiments on other vision tasks to test the effectiivnnes of the DenoiseRep. We validate the generalization ability of DenoiseRep in image classification tasks on ImageNet-Ik [12] datasets and three fine-gru mne-grained image classification datasets (CUB200 [55], Oxford-Poc [46], and Flowers 45)) The accuracy index is chosen as the evaluation metric to assess model performance.
As shown in T we compare multiple classic backbones for representation learning on ImageNet- 1k, and after adding the DenoiseRep, the accuracy of both top-1 1 and top-5 metrics improve without adding model parameters. Our method shows significant improvement accuracy metrics compared to baseline on three fine-grained classification datasets. Prove that the DenoiseRep can improve the model's 's ability in image classification for different classification tasks. Additionally, our method proves to enhance the model's representation learning ability and extract more effective features

---

Table 5: The effectiveness of method in image classification tasks validated on three fine-
grained classification datasets (CUB200, Oxford-Peed Pettt, Flowers) and ImageNet-lk.
acc @
acc acc@5 @5
Method
Datasets
Param
Baseline
+DenoiseRep
Baseline
+DenoiseRep
SwinV2-T
39
ImageNet- lk
28M
81.82%
82.13%
95.88%
96.06%
Vmanba-T
38
ImageNet-
30M
82.38%
82.51%
95.89% 95.80%
ResNet50 18
ImageNet-1K
26M
76.13%
76.28%
92.86%
92.95%
ViT-B
14
CUB200
87M
91.78%
91.99%
ViT-B
Oxford-Pet
87M
94.37%
94.58%
ViT-B
Flowers
87M
99.12%
99.30%
through denoising without incurring additional time costs. More experimental analysis can be found in Table in Section of the appendix.
Conclusion
In this work, we demonstrate that the diffusion model paradigm is effective for feature level denoising in discriminative model, and propose computation-free and label-free method: DenoiseRep. It utilizes the denoising ability of diffusion models denoise the features in the feature extraction layer, and fuses the parameters of the denoising layer and the feature extraction layer, further improving retrieval accuracy without incurring additional computational costs. We validate the effectiveness of the DenoiseRep method on multiple common image discrimination task datasets.
Acknowledgement
This work supported by the National Natural Science Foundation of China (62202041), National Key Research and Development Program of China under Grant (F033YFC3310700) and Fundamental Research Funds for the Central Universities (202JJBMC057).
References
[1] Abien Fred Agarap. An architecture combining convolutional neural network (cnn) and support vector machine (svm) for image classification. arXiv preprint arXiv: 17Xiv:1712.03541, 2017.
[2] Saad Albawi, Tareq Abed Mohammed, and Saad Al-Zawi. Understanding of a convolutional neural network. In 2017 international conference on engineering and technology (ICET), pages
1 1-6. Icee, 2017.
[3] Apurva Bedagkar- and Shishir Shah. survey of approaches and trends in person re-identification. Image and vision computing, 32(4):270-286, 2014.
[4] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8): 1798-1828, 2013, doi: 10.1109//AMM..2013.5555
[5] Hanqun Cao, Cheng Tan, Zhangyang Gao, Yilun Xu, Guangyong Chen, Pheng-Ann Heng, and Stan Z. Li. A survey on generative diffusion models. IEEE Transactions Knowledge and Data Engineering, pages 1-20, 2024. doi: 10. 109/TKKDE.002..33447777
[6] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision, pages 213-229. Springer, 2020.
[7] Shoufa Chen, Peize Sun, Yibing Song, and Ping Luo. Diffusiondet: Diffusion model for object detection. Proceedings of the IEFF/CVF International Conference on Computer Vision, pages 19830-199 19843, 2023.
10

---

[8] Tianlong Chen, Shaojin Ding, Jingyi Xie, Ye Yuan, Wuyang Chen, Yang Yang, Zhou Ren, and Zhangyang Wang. Abd-net: Attentive but diverse person re-identification. In Proceedings of the IEEE/CVF international conference on computer vision, pages 8351 8351-8361, 2019.
[9] Yuhua Chen, Wen Li, Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Domain adaptive faster r-cnn for object detection in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3339-3348, 2018.
[10] Yoonki Cho, Woo Jae Kim, Seunghoon Hong, and Sung- Bui Yoon. Part-based pseudo label re- finement for unsupervised person re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7308-7318, 2022.
[11] Ruihang Chu, Yifan Sun, Yadong Li, Zheng Liu, Chi Zhang, and Yichen Wei. Vehicle re- identification with viewpoint-aware metric learning. In Proceedings of the IEEE/CVF interna- tional conference on computer vision, pages 8282-8291, 2019.
[12] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition, pages 248-255, 2009.
[13] Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE signal processing magazine, 29(6):141-144 141-142, 2012.
[14] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11922 2020.
[15] Pengfei Fang, Jieming Zhou, Soumava Kumar Roy, Lars Petersson, and Mehrtash Harandi. Bilinear attention networks for person retrieval. In Proceedings of the IEEE/CVF international conference on computer vision, pages 8030-8039 2019.
[16] Dengpan Fu, Dongdong Chen, Jianmin Bao, Hao Yang, Lu Yuan, Lei Zhang, Houqiang Li, and Dong Chen. Unsupervised pre-training for person re-identification. In InPPro Proceedings of the IEEE/CVF conference on computer vision and paltern recognition, pages 14750-14759, 2021.
[17] Ian Goodfellow, Jean Pouget-Abadis Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
[18) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[19] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961-2969, 2017.
[20] Shuting He, Hao Luo, Pichao Wang, Fan Wang, Hao Li, and Wei Jiang. Transreid: Transformer- based object re- identification. In Proceedings the IEEE/CVF International Conference Computer Vision (ICCV), pages 15013-15022, October 2021.
[21] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
[22] Andrej Karpathy and Li Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3128-3137, 2015.
[23] Tero Karras, Samuli Laine, and Timo Aila. style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4401 1401-4410, 2019.
[24] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint 1rXiv:1312.6114, 2013.

---

[25] Martin Koestinger, Martin Hirzer, Paul Wohlhart, Peter M Roth, and Horst Bischof. Large scale metric learning from equivalence constraints. In Computer Vision and Pattern Recognition. pages 2288 2295, 2012.
[26] Alex Krizhevsky, Ily Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012.
[27] Hei Law and Jia Deng. Cornemet: Detecting objects paired keypoints. In Proceedings of the European conference on computer vision (ECCV), pages 734-750, 2018.
[28] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 (6:11)) -3224, 1998.
[29] Hanjun Li, Gaojie Wu, and Wei-Shi Zheng. Combined depth space based architecture search for person re-identification. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 6729 6738, 2021.
[30] Jianxin Li, Shuai Zhang, Hui Xiong, and Haoyi Zhou. Autost: Towards the universal modeling of spatio-tempore sequences. Advances in Neural Information Processing Systems, 35:20498- 20510, 2022.
[31] Jing Li, Yu Liu, and Lei Zou. Dyngen: A dynamic graph convolutional network based on spatial- temporal modeling. In Web Information Systems Cngineering-WISS 2020: 21 International Conference, Amsterlam, The Netherlands, October 20-24, 2020, Proceedings, Part I 21, pages 83-95. Springer, 2020.
[32] Siyuan Li, Li Sun, and Qingli Li. Clip-reid: exploiting vision-language model for image re- identification without concrete text labels. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 1405-1413. 2023.
[33] Wei Li, Rui Zhao, Tong Xiao, and Xiaogang Wang. Deepreid: Deep filter pairing neural network for person re- re-identification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 152-159, 2014.
[34] Shengcai Liao and Stan Z Li. Efficient psd constrained asymmetric metric learning for person re-identification. In International Conference on Computer Vision, pages 3685-3693, 2015.
[35] Shengcai Liao, Yang Hu, Xiangyu Zhu, and Stan Li. Person re-identification by local maximal occurrence representation and metric learning. In Computer Vision and Pattern Recognition, pages 2197-2206, 2015.
[36] Kevin Lin, Huei-Fang Yang. Jen-Hao Hsiao, and Chu-Song Chen. Deep learning of binary hash codes for fast image retrieval. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 27-35, 2015.
[37] Hongye Liu, Yonghong Tian, Yaowei Yang, Lu Pang, and Tiejun Huang. Deep relative distance learning: Tell the difference between similar vehicles. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2167-2175, 2016.
[38] Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang, Qixiang Ye, and Yunfan Liu. Vmamba: Visual state space model. arXiv preprint arXiv:2401.10166 2024.
[39] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012-10022, 2021.
[40] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for se- mantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3431-3440, 2015.
[41] Hao Luo, Pichao Wang, Yi Xu, Feng Ding, Yanxin Zhou, Fan Wang, Hao Li, and Rong Jin. Self-supervised pre-training for transformer-basee person re-identification. arXiv preprint 122, 2021.
12

---

[42] Bingpeng Ma, Yu Su, and Frederic Jurie. Covariance descriptor based on bio-inspired features for person re-identification and face verification. Image and Vision Computing, 32(6-7):379-390 2014.
[43] Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint 1rXiv:1411.17888 2014.
[44] Hao Ni. Yuke Li, Lianli Gao, Heng Tao Shen, and Jingkuan Song. Part-aware transformer for generalizable person re-identification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11280-11289, 2023.
[45] Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing, pages 722-729, IEEE, 2008
[46] Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE conference on computer vision and patlern recognition, pages 3498-3505. IEEE, 2012.
[47] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International Conference Machine Learning, pages 8821-8831. PMLR, 2021.
[48] Joseph Redmon. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.0277
2018.
[49] Shaoqing Ren. Faster r-cnn: Towards real-time object detection with region proposal networks. arXiv preprint arXiv: 1506.01497 2015.
[50] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[51] Jascha vised learning using nonequilibrium thermodynamics. In International conference on machine learning, pages 2256-2266 PMLR, 2015.
Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper-
[52] Junjiao Tian, Lavisha Aggarwal, Andrea Colaco, Zsolt Kira, and Mar Gonzalez-France Diffuse, attend, and segment: Unsupervised zero-shot segmentation using stable diffusion. arXiv preprint arXiv::308.. 12469, 2023.
[53] Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning, Advances in neural information processing systems, 30, 2017.
[54] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
[55] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds-200-2011 dataset. 2011.
[56] Benzhi Wang, Yang Yang, Jinlin Wu, Guo-jun Qi, and Zhen Lei. Self-similarity driven scale- invariant learning for weakly supervised person search. arXiv preprint arXiv: 12986, 2023.
[57] Guan'a an Wang, Yang Yang, Jian Cheng, Jinqiao Wang, and Zengguang Hou. Color-sensitive person re-identification. In International Joint Conference on Artificial Intelligence, pages 933-939. 2019.
[58] Guan' an Wang, Shuo Yang, Huanyu Liu, Zhicheng Wang, Yang Yang, Shuliang Wang, Gang Yu, Erjin Zhou, and Jian Sun. High-order information matters: Learning relation and topology for occluded person re-identification. 2020.
[59] Guanshuo Wang, Yufeng Yuan, Xiong Chen, Jiwei Li, and Xi Zhou. Learning discriminative features with multiple granularities for person re-identification. In Proceedings of the 26th ACM international conference Multimedia, pages 274-282, 2018.
13

---

[60] Haochen Wang. Jiayi Shen, Yongtuo Liu, Yan Gao, and Efstratios Gavves. Nformer: Robust person re-identification with neighbor transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7297-7307, 2022.
[61] Longhui Wei, Shiliang Zhang, Wen Gao, and Qi Tian. Person transfer gan to bridge domain gap for person re-identification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 79-88, 2018.
[62] Tobias Weyand, Andre Araujo, Bingyi Cao, and Jack Sim. Google landmarks dataset v2-a large- scale benchmark for instance-level recognition and retrieval. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 2575-2584, 2020.
[63] Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo. Segformer: Simple and efficient design for semantic segmentation with transformers. Advances in neural information processing systems, 34:1 2021.
[64] Pei Xu, using timewise latents. In European Conference on Computer Vision, pages 511-528. Springer, 2022.
Hayet, and Ioannis Karamouzas. Socialvae: Human trajectory prediction
[65] Yang Yang, Jimei Yang, Junjie Yan, Shengeai Liao, Dong Yi, and Stan Z Li. Salient color names for person re-identification. In European conference on computer vision, pages 536-551, 2014.
[66] Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, and Steven CH Hoi. Deep learning for person re-identification: A survey and outlook. IEEE transactions on pattern analysis and machine intelligence, 44(6):2872-2888 2893,, 2021.
[67] Shifeng Zhang, Cheng Chi, Yongqiang Yao, Zhen Lei, and Stan Li. Bridging the gap between anchor-based and free detection via adaptive training sample selection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9759-9768, 2020.
[68] Zhizheng Zhang, Cuiling Lan, Wenjun Zeng, Xin Jin, and Zhibo Chen. Relation aware global attention for person re-identification. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition, pages 3186-3195, 2020.
[69] Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. Learning discourse- diversity for neural dialog models using conditional variational autoencoders. arXiv preprint arXiv: /703. 10960, 2017.
[70] Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang, and Qi Tian. Scalable person re-identification: A benchmark. In International Conference on Computer Vision, pages 1116-1124, 2015.
[71] Wei-Shi Zheng, Shaogang Gong, and Tao Xiang. Reidentification by relative distance com- parison. IEEE transactions on pattern analysis and machine intelligence, 35(3):653-666. 2013.
[72] Zhedong Zheng, Liang Zheng, and Yi Yang. Unlabeled samples generated by gan improve the person re-identification baseline in vitro. arXiv preprint arXiv:1701.07717 2017.
[73] Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 633-641, 2017.
[74] Kaiyang Zhou, Yongxin Yang, Andrea Cavallaro, and Tao Xiang. Omni-scale feature learning for person re-identification. In Proceedings of the IEEF/CVF international conference on computer vision, pages 3702-3712, 2019.
[75] Xingyi Zhou, Dequan Wang, and Philipp Krähenbühl. Objects as points. arXiv preprint /004:0/90000785/ 2019.
[76] Kuan Zhu, Haiyunn Guo, Zhiwei Liu, Ming Tang, and Jinqiao Wang. Identity dlentity-guided human semantic parsing for person re-identification. European Conference on Computer Vision, pages 346-363. Springer, 2020.
14

---

A Experimental settings
Datasets and Evaluation metries. We conduct training and evaluation on four datasets: DukeMTMC- rell [72), Market-1501 [70], MSMT17 [61], and CUHK-03 33 These datasets encompass wide range of scenarios for person re-identification. For accuracy. we use standard metries including Rank-1 curves (The probability that the image with the highest confidence in the search results is the correct result.) and mean average precision (MAP). All the results are from a single query setting.
Implementation Details. We implement our method using Python on a server equipped with a 2. 10GGz Intel Core Xeon (R) Gold 5218R processor and two NVIDIA RTX 3090 GPUs. The epochs we trained are set to 120, the learning rate is set to 0.0004, the batch size during training is 64, the inference stage 256, and the diffusion step size é is set to 1000.
Training and evaluation. To better constrain the performance of the denoised features of the DenoiseRep for downstream tasks, we employ alternating fine-tune methods. The parameters of the DenoiseRep and baseline are trained alternately, and when training a part of the parameters, the rest of the parameters 120. When evaluating, we average the results of the experiments under the same settings for times, thus ensuring the reliability of the data.
frozen and fine-tuned for 10 epochs time, with total number of epochs of
B Experiment on Vehicle Identification
In the image retrieval task, we also conduct experiments to verify the effectiveness of our method on the vehicle recognition task. Vehicle recognition in practical scenarios often results in images containing large amount of noise due to environmental factors such as lighting or occlusion, which increases the difficulty of detection. Our method based on denoising to obtain features with better representation ability. Therefore, we want to experimentally verify whether the DenoiseRep plays role in vehicle recognition tasks with higher noise levels. We select vehicleID [37] as the dataset, vehicle-ReID as the baseline, and ResNet-50 as the feature extractor for the experiment.
Table 6: The performance of the DenoiseRep on vehicle recognition tasks.
Method
Backbone ResNet-50
Datasets
mAP
Rank-1 1
vehicle-ReID
vehicleID
76.4%
69.1%
vehicle-Reile RelleRReD++ + DenoiseRep
ResNet-50
vehicleID
77.3%
70.2%
From the results in Table 6 it can be seen that DenoiseRep demonstrates excellent performance in vehicle detection tasks. Compared to the baseline, adding the DenoiseRep significantly improves both mAP and Rank-1 1 metrics without incurring additional detection time costs. It verifies the denoising ability of the DenoiseRep in noisy environments.
C Experiment on Large Scale Image Classification Tasks
In this section, we aim to test the generalization ability of DenoiseRep in other tasks. We conduct experiments on two image classification datasets, ImageNet-lk and Cifar-10. These two datasets are both classic image classification datasets, rich in common images in daily life, and belong to large-scale image databases. ImageNet- 1k is a subset of the ImageNet dataset, containing images from 1000 categories. Each category typically has hundreds to thousands of images, totaling over one million images. The Cfiar- 10 contains 60000 32x32 pixel color images, divided into 10 categories. Each category contains 6000 images. To evaluate the effectiveness of our method, we use standard metrics, including Top-1 accuracy and Top-5 accuracy, which are commonly used to evaluate model performance in image classification tasks and are widely used on various image datasets, and we conduct detailed comparative experiments on multiple backbones and models with different parameter versions to verify the reliability of our method.
As shown in Table 7 we compare multiple classic backbones for representation learning on two datasets, and after adding the DenoiseRep, the accuracy metrics improves without adding model parameters. Our method demonstrates the capability to enhance the model's I' representation learning
15

---

Table 7: The effectiveness of our method in image classification tasks validated on Cifar- 10 and ImageNet-1k.
acc @
acc 5
Method
Datasets
Param
Baseline
+DenoiseRep
Baseline
+DenoiseRep
SwinV2-T
39
ImageNet-1k
28M
81.82%
82.13%
95.88%
96.06%
SwinV2-S
39
ImageNet-IK lk
50M
83.73%
83,97%
96.62%
96.86%
SwinV2-B
39
ImageNet-Ik
88M
84.20%
84.31%
96.93%
97.06%
Vmanba-1
38
ImageNet-Ik
30M
82.38%
82.31%
95.80%
95.89%
Vmanba-S
38
ImageNet-Ik
50M
83.12%
83.27%
96.04%
96.22%
Vmanba-B
38
ImageNet-1k
89M
83.83%
83.91%
96.55%
96.70%
ViT-S
ImageNet-Ik
22M
83.87%
84.02%
96.73%
96.86%
ViT-B 14
ImageNet- 1k
86M
84.53%
84.64%
97.15%
97.23%
ResNet50 18
ImageNet-Il lk
26M
73.13%
76.28%
92.86%
92.95%
VIT-S [14
Cifar- 10
22M
96.13%
96.20%
ViT-B [4]
Cifar-10
87M
98.02%
98.31%
ability and extract more effective features through denoising, all while maintaining the same time costs. Moreover, DenoiseRep generalizes effectively to image classification tasks.
D Experiment on Image Detection Task
In this section, aim test the generalization ability of DenoiseRep in image detection tasks. We conduct experiments on the COCO (22 dataset. The COCO (Common Objects in Context) dataset is widely used dataset for large-scale image recognition, object detection, and image segmentation, particularly in computer vision tasks. It contains 80 types of objects, such as people, animals, daily necessities, etc., covering various common items in daily life. To verify that our method is model independent, we conduct experiments using different models including Mask-RCNN [19), Faster- RCNN 44), ATSS (67), YOLO 48), DETR 6 and CenterNet [75]. as well as diverse backbones. To evaluate the effectiveness of our method, we used standard metrics including AP, AP30. and APTs, which are commonly used metries in object detection tasks evaluate model performance. particularly for evaluating the effectiveness of bounding box detection. It is also an important part of the COCO dataset evaluation criteria, which can measure the detection ability of the model in multiple categories and scales.
Table 8: The effectiveness of our method in image detection tasks was validated on COCO.
APso
AP7s
Methods
Backbones
Baseline
+DenoiseRep
Baseline
+DenoiseRep
Baseline
+DenoiseRep
Swin-T
42.8%
44.3%
65.1%
67.1%
47.0%
48.6%
Mask-RCNN
Swin-S
48.2%
49.0%
69.9%
70.9%
52.8%
53.8%
ResNet-50
42.6%
43.2%
63.7%
65.0%
46.4%
46.8%
Fuster-RCNN
ResNet-50
37. 37.4%
38.3%
58.1%
58.8%
40.4%
41.0%
ATSS
ResNet-50
39.4%
39.9%
57.6%
58.2%
42.8%
43.2%
YOLO
DarkNet-53
27.9%
28.4%
49.2%
50.3%
28.3%
27.8%
DETR
ResNet-50 39.9%
40.8%
60.4%
59.9%
41.7%
42.9%
CenterNet
ResNet-50
40.2%
40.6%
58.3%
59.1%
43.9%
44.0%
As shown in Table 8 we compare multiple classic backbone networks across different methods. After adding DenoiseRep. the accuracy index shows improvement without the need for additional model parameters. This indicates that our method enhances the representation learning capability of the model and extracts more effective features through denoising, all while maintaining the same time costs. In addition, DenoiseRep well to generalized to image detection tasks.
16

---

E Experiment on Image Segmentation Task
In this section, our objective to assess the generalization capability of DenoiseRep in image segmentation tasks. The image segmentation task aims to divide an image into multiple regions in order to identify and understand objects or areas within them. The main challenges faces include: complex and varied backgrounds that can easily interfere with segmentation results, objects obstruct each other, making segmentation difficult, objects have diverse shapes and may undergo deformations, etc. We conduct experiments the ADE20K [73] dataset using the current mainstream image segmentation models. ADE20K is a widely used scene segmentation dataset, mainly used for image segmentation tasks. It contains approximately 20000 images and over 150 different object and region categories. We choose mloU and B-IOU as evaluation metrics to comprehensively evaluate the performance of image segmentation models. MIoU is the average of IoUS for all categories, which can effectively reflect the segmentation ability of the model on different categories. It provides quantitative model for the accuracy of handling complex scenes by calculating the degree of overlap between the predicted area and the real area. A higher mloU value means that the model can better identify and segment the target object. B-IoU focuses evaluating the accuracy of segmentation boundaries and is particularly suitable for object edge segmentation tasks. It provides sensitivity to boundary details by measuring the degree of overlap between predicted boundaries and real boundaries.
Table 9: The effectiveness of our method in image segmentation tasks validated on ADE20K.
aAce Baseline
B-IoU
mloU
Methods
Backhones
+eenoiseRep
Baseline
+DenoiseRep
Baseline
+DenoiseRep
FCN
40
ResNet-50
0.774
0.779
0.287
0.299
0.359
0.365
FCN
ResNet- 101
0.793
0.796
0.306
0.316
0.396
0.404
SegFormer 63
mit b0
0.782
0.788
0.292
0.297
0.374
0,381
SegFormer
mit
0.812
0.816
0.341
0.348
0.422
0.425
As shown in Table 9 we compare two classical backbone networks with different methods. After
adding the DenoiseRep, both IoU metrics improve without adding model parameters. Practice proves that our method can improve the representation learning ability of the model and obtain more effective features through denoising without increasing additional time costs. In addition, DenoiseRep well to generalized to image segmentation tasks.
F Fairness Experiment
To ensure the fairness of the experiment, we compared the performance of the baseline method with our proposed method under the same conditions. Specifically, during the training process, we strictly controlled the experimental variables so that both the baseline method and our method ran under the same number of training epochs and hyperparameter settings.
Table 10:
Comparison of performance between baseline method and our proposed method in the same additional training epochs. The baseline method TransRelD-Se RSLL is based on ViT-small backbone.
Epoch
120
160
200
240
Baseline 81.18%
81.19%
81.18%
81.16%
+DenoiseRep
81.18%
81.64%
82.00%
82.12%
The experimental results in Table 10 indicate that the baseline method did not show significant performance improvement under the same number training epochs. This observation indicates that the performance improvement obtained is attributed to our proposed method, rather than increase training time or number of epochs, which validates the effectiveness of our method.
G Limitations
Our proposed method DenoiseRep improves the accuracy of current mainstream backbones while ensuring label-free and no additional computational costs, and it has been experimentally verified to
17

---

be generalizable in multiple image tasks. However, from the experimental results, it can be found that our method has limited improvement model accuracy when generalized general tasks, and in order to fuse the parameters of the denoising layer and the feature extraction layer, only one or two steps of denoise for each denoising layer, and the number of denoising layers is not more than that of the feature extraction layer, which limits the denoising intensity. We will continue to explore how to further improve the accuracy of the model without adding additional inference time costs or only adding small number of additional parameters.
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In the abstract and introduction, have elaborated on the starting point and research direction of the paper, and summarized the contributions of the paper in detail.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We have explained the limitations of our method and future improvement directions in the appendix.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiitiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
19

---

Answer: [Yes]
Justification: In the section "Denoising Representation for Person Re Identification", we validated our proposed theory through detailed formula derivation.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: In the "Implementation Details" section, we introduced the training techniques and parameter settings, and the experimental code will be released on GitHub, so that readers can reproduce our method well.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
20

---

reproducible, they 5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [NA]
Justification: We do not include the code in this submission, but in the future, we will organize the experimental code and documentation, and released GitHub.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https::/niis.cc.
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
/nips. wwtpcc/pcccccpcccceooooooop httpa//gguiieee/deeuubbbiiiiiiii sereiisssooPoPoi
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We have provided detailed explanation of the model parameters and training methods in the "Experimental settings" section of the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We took the average of the results of five independent repeated experiments as the main experiment, but did not provide a detailed analysis and discussion of their variance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
21

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: In the "Experimental settings" section of the appendix, we provide sufficient information the computer resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttp:///eerrrp.....pppbb///t/t
Answer: [Yes]
Justification: aspects.
The research in this paper complies with NeurolPS ethical standards in all
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., s if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no societal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
22

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have added appropriate citation explanations to the existing papers, models, and datasets mentioned in this paper.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
23

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "2WucxEHY7d", "reviewer": "Reviewer_w7Th", "summary": "This paper proposes a novel denoising model called DenoiseReID, designed to enhance representation learning in person re-identification (ReID) tasks. This approach combines traditional denoising processes with feature extraction through a feature extraction and denoising fusion algorithm (FEFDFA) that incurs no additional computational cost. It incrementally improves the discriminability of features without the need for labels.", "questions": "I am particularly concerned about the performance comparison in this paper. Is it necessary to use diffusion models for conducting ReID tasks, and what advantages do they offer compared to CNNs or ViTs? Currently, it seems that their performance improvement is rather modest.", "limitations": "yes, the authors adequately addressed the limitations.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The experimental results demonstrate that DenoiseReID achieves stable performance improvements across multiple ReID datasets. Furthermore, it can be extended to large-scale image classification tasks such as ImageNet, CUB200, Oxford-Pet, and Flowers datasets.", "weaknesses": "1. In the comparative experiments, the authors missed an opportunity to benchmark their method against the latest advancements like CLIP-ReID, restricting comparisons solely to TransReID.\n   \n2. When contrasted with TransReID, this approach shows a moderate enhancement in performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1: **The authors missed an opportunity to benchmark their method against the latest advancements like CLIP-ReID.**\n\nA1: Thanks for your valuable feedback. CLIP is a very strong vision-text encoder, which is trained with 400 million data. Beyond CLIP, CLIP-ReID pioneeringly adapts CLIP, a zero-shot classifier, to ReID, a fine-grained image retrieval task. Our proposed method is model-free, thus it can be easily applied to CLIP-ReID without any modification. The experimental results of CLIP-ReID show stable improvement on the three datasets(DukeMTMC, MSMT, Market-1501). The code will be released if the manuscript could be accepted. We will add the experimental results to the revised manuscript. The experimental results are as follows, with mAP as the evaluation metric:\n\n|   Dataset    |  DukeMTMC |  MSMT | Market-1501 | \n|:-------|:------------:|:------------:|:------------:|\n| CLIP-REID |  83.1%    | 75.8%    | 90.5%    | \n| CLIP-REID+FEFDFA   |  83.9%(↑0.8%) |   76.5%(↑0.7%) | 91.1% |\n\nQ2: **When contrasted with TransReID, this approach shows a moderate enhancement in performance.** \n\nA2: Thanks for your kind comment.  Our proposed method is test-time computation-free (FREE LAUNCH!) and scalable to various baseline and downstream tasks, which is proven to be effective in CNN series (e.g. ResNet50), Transformer series (e.g. ViT, Vmanba), Person Re-ID (MSMT, Duke), Vehicle Re-ID (vehicleID), and Image Classification (ImageNet, CUB). For its scalability, we DO NOT hack it for backbones or tasks. With the same hyper-parameters, ours achieves consistent and stable improvement on ALL backbones and tasks above. For example, based on previous state-of-the-art CLIP-ReID, our proposed method achieves extra improvement from 0.8%-0.6% on Duke, MSMT and Market.\n  \nQ3: **Is it necessary to use diffusion models for conducting ReID tasks, and what advantages do they offer compared to CNNs or ViTs?** \n\nA3: Thanks for your kind comment. In this work, we use the ViT series for conducting ReID tasks. Our proposed Feature Extraction and Feature Denoising Fusion Algorithm (FEFDFA) fuse the idea of diffusion into the backbone (e.g. ResNet, ViT). We treat each block of the backbone as a denoising layer and utilize the denoising ability of the diffusion model to denoise on a different feature extraction level. During the inference phase, the denoising layer parameters are fused with the backbone network parameters without incurring additional inference time cost.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8PHmgj0gV4", "reviewer": "Reviewer_4MBF", "summary": "This paper proposes a new method, Feature Extraction and Feature Denoising Fusion Algorithm (FEFDFA), which utilizes the denoising ability of diffusion models to denoise the features in the feature extraction layer, and fuses the parameters of the denoising layer with those of the feature extraction layer through parameter fusion, further improving retrieval accuracy without incurring additional computational costs. The effectiveness of the FEFDFA method has been validated on multiple common image discrimination task datasets.", "questions": "same to the weaknesses.", "limitations": "The authors have addressed the limitations.", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.The article structure is complete and writing is generally clear.\n\n2.Some experimental results seem to good.", "weaknesses": "1.In fact, the intermediate layer features of the pre-trained diffusion model can be used directly for the downstream task such as the discrimination task [1][2][3].\nThe authors need to enrich the Related Work.\n\n2.Line144-146, no evidence provided to support the proposed hypothesis.\n\n3.The authors treat the backbone as a series of denoising layers, so the training loss in Eq. (11) should be the sum of the MSE losses of each denoising layer.\n\n4.The statement is inconsistent.\n\nLine172-173，\"we freeze the original parameters and only trained the FEFDFA.\"\n\nLine509-511，\"the parameters of the FEFDFA and baseline were trained alternately.\"\n\nIn addition, unless the baseline and FEFDFA are trained together, Eq. (12) is incorrect.\n\n\n[1] Mukhopadhyay, Soumik, et al. \"Diffusion models beat gans on image classification.\" arXiv preprint arXiv:2307.08702 (2023).\n\n[2] Li, Alexander C., et al. \"Your diffusion model is secretly a zero-shot classifier.\" ICCV 2023.\n\n[3] Baranchuk, Dmitry, et al. \"Label-Efficient Semantic Segmentation with Diffusion Models.\" ICLR 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1: **The authors need to enrich the Related Work.**\n\nA1: We thank the valuable suggestions. Our proposed DenoiseReID is different from the related works [1-3]. The related works [1-3] apply the itermediate layer features of an existing pre-trained diffusion model to improve downstream task. Ours applies the denoise algorithm to any existing pre-trained downstream model (e.g. Person ReID of Table 2, Vehicle ReID of table 5, Image Classification of Table 6). Compared with the related works [1-3], ours is more scalable (suitable to more downstream tasks) and light-weight (computation-free). We will carefully review and add them to the revised manuscript.\n  &emsp;\n\nQ2: **Line144-146, no evidence provided to support the proposed hypothesis.**\n\nA2: Thanks for the kind comment. Experimental results of Table 3 (FEFDUF) could empirically demonstrate the hypothesis of \"the features obtained by backbone extraction are noisy\". FEFDUF includes a well-trained person ReID model, and a denoise model which takes the feature of the ReID model as input and predicts its noise. During the training stage, the ReID model is always frozen, and the denoise model is trained like DDPM (i.e. adding a gaussian noise to feature, taking the nosied feature as input and predicting the original gaussian noise). During the inference stage, given images, we first extract feature A with the well-trained ReID model, then put the feature into the denoise module to predict its noise A'. Finally, we found that A-A' performs better than A. The training process DO NOT use any label because DDPM is an unsupervised manner. This experiment shows that simple denoising features contribute to improvement, partially supporting the view that \"the features obtained by backbone extraction are noisy\". We will add the analysis above to the revised manuscript.\n&emsp;\n\nQ3: **The training loss in Eq. (11) should be the sum of the MSE losses of each denoising layer.**\n\nA3: Thanks for the valuable reminder. This is a writing error. In the code implementation, the loss during the training of the denoising module is the sum of the MSE losses of each denoising layer. The code will be released if the manuscript could be accepted. We will polish the Eq. (12) to be:\n$$\nLoss_p = \\sum_{i=1}^{N} \\left\\| \\epsilon_i - D_{\\theta_i} (X_{t_i}, t_i) \\right\\|\n$$\n\nQ4: **The statement is inconsistent.**\n\nA4: Thanks for your comments. There are THREE questions. Let's discuss them ONE BY ONE: \n  - (1) \"we freeze the original parameters and only train the FEFDFA.\" This is our basic training setting. It only trains the parameters of FEFDFA with denoising and (optional) ReID loss. In all unsupervised settings, we use this training set (without ReID loss). It makes significant improvements. Specifically, in the experimental results between the second and first lines of Table 1, it can be observed that the performance of the unsupervised FEFDFA method has significantly improved compared to the baseline.\n  &emsp;\n  - (2) \"the parameters of the FEFDFA and baseline were trained alternately.\"  Alternately training is a TRAINING TRICK for the supervised setting. Specifically, we (a) freeze the original parameters, train FEFDFA with denoising and reid losses, then (b) merge parameters of FEFDFA into the original parameters, train latest original parameters with ReID loss only, (c) repeat (a) and (b). In all supervised settings, we use this trick. It carries more improvement. Specifically, the experimental results in the second and third lines of Table 1. Note that step (a) still carries improvements in the supervised setting.\n  &emsp;\n  - (3) \"In addition, unless the baseline and FEFDFA are trained together, Eq. (12) is incorrect.\" Eq.(12) is CORRECT even if parameters of FEFDFA are trained and the original parameters are frozen. Both ReID loss and denoising losses can be used to optimize parameters of FEFDFA. This equals to (2)-(a) above. Similar ideas of \"using task-related loss to supervise denoising module\" can be found in DiffDet[4] and DiffSeg[5]. \n\n  \n[1] Mukhopadhyay, Soumik, et al. \"Diffusion models beat gans on image classification.\" arXiv preprint arXiv:2307.08702 (2023).\n\n[2] Li, Alexander C., et al. \"Your diffusion model is secretly a zero-shot classifier.\" ICCV 2023.\n\n[3] Baranchuk, Dmitry, et al. \"Label-Efficient Semantic Segmentation with Diffusion Models.\" ICLR 2022.\n\n[4] Chen, Shoufa, et al. \"Diffusiondet: Diffusion model for object detection.\" Proceedings of the IEEE/CVF international conference on computer vision. 2023.\n\n[5] Tian, Junjiao, et al. \"Diffuse Attend and Segment: Unsupervised Zero-Shot Segmentation using Stable Diffusion.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cgXjpVBrtr", "reviewer": "Reviewer_DSeM", "summary": "This mauscript proposes a novel denosing model for representaetion learning and take person re-identification as a benchmark. It unifies the frameworks of feature extraction and feature denoising, where the former progressively embeds features from lowlevel to high-level, and the latter recursively denoises features step-by-step. Besides, a FEFDA is proposed to fuse feature extraction and denoising in a single backbone without changing its structure and taking extra runtime latency. Experiments on ReID, large-scale  and fine-grained classification tasks show its effectivenss.", "questions": "For questions, please see the weakness in the above section.", "limitations": "The authors have discussed the limitations of the proposed method.", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The idea of unifying feature extraction and feature denoising in a single backbone without changing its structure is interesting. As far as I know, its first time to see the idea.\n2. The characteristic computation-free and label-free is promising. The thoerical analysis seems precies and right.\n3. Experiments on 3 typical tasks and 9 datasets of representation learning (retrieval, classfication, fine-grained classification) are sufficient and extensive.", "weaknesses": "1. the proposed \"Feature Extraction and Feature Denoising Fusion Algorithm\" is little similar to reparameterization, please clarify their difference.\n2. its application to transformer series are well analyzed, it will be better if peformance on CNN series are shown.\n3. representation learning is a wide and fundational conception, its applications on more downstream tasks, such as detection, segmentation even generation, could be analyzed,  in future.\n4. The hyper-parameter analysis is missed.\n5. This method seems need extra training steps, what if the baseline methods are trained under the same steps?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1: **Clarify the difference between the proposed FEFDFA and reparameterization.**\n\nA1: Thank you for your insightful comment. We appreciate the opportunity to clarify the differences between our \"Feature Extraction and Feature Denoising Fusion Algorithm\" (FEFDFA) and reparameterization.\n   - Reparameterization: Reparameterization typically involves restructuring the parameters of a model to facilitate more efficient training. It is widely used in variational autoencoders (VAEs) to allow for gradient-based optimization of stochastic variables.\n   - Feature Extraction and Feature Denoising Fusion Algorithm (FEFDFA): Our FEFDFA is designed to integrate feature extraction and denoising within the same backbone. Each embedding layer in the backbone serves a dual purpose: extracting features and performing denoising simultaneously. By deriving the formula for diffusion model sampling, the denoising layer parameters are fused with the feature layer parameters, so that the model has no additional time cost in the inference stage.\n&emsp;\n\nQ2: **It will be better if peformance on CNN series are shown.**\n\nA2: Thanks for your valuable feedback. Our proposed method is scalable to various baseline (including CNN, see Table 5 and line seven of Table 6 for details)  and downstream tasks, including CNN series (e.g. ResNet50), Transformer series (e.g. ViT, Vmanba), Person Re-ID (MSMT, Duke), Vehicle Re-ID (vehicReID), Image Classification (ImageNet, CUB). The experimental results in Section 4.4 of the main text and Appendix C demonstrate that our method significantly improves these tasks.\n&emsp;\n\nQ3: **Representation learning applications on more downstream tasks.**\n\nA3: Thanks for your valuable suggestion. Representation learning is a broad and foundational concept with potential applications across various downstream tasks. Due to the limited time, we extend our proposed FEFDFA to the object detection task with Mask-RCNN[2] as a baseline and COCO[1] as a benchmark. Experimental results show that ours carries stable improvements of 1.6%-1.1%. Please see the table below for details. We will extend ours to more downstream tasks including segmentation and generation, in future.\n\n | **MaskRCNN** | **bbox_mAP** | **bbox_mAP_50** | **bbox_mAP_75** | **mask AP** |\n |--------------|:--------------:|:-----------------:|:-----------------:|:-------------:|\n | Swin-T       | 0.427        | 0.652           | 0.468           | 0.393       |\n | Swin-T+FEFDFA | 0.443(↑1.6%)       | 0.671           | 0.486           | 0.405       |\n | Swin-S       | 0.482        | 0.698           | 0.528           | 0.432       |\n | Swin-S+FEFDFA | 0.493(↑1.1%)       | 0.709           | 0.540           | 0.439       |\n\nQ4: **The hyper-parameter analysis is missed.**\n\nA4: Thanks for your reminder. We analyze hyperparameters in the model, including important parameters such as $\\beta_t$ (Line178), and diffusion step size T (Line73). The results of the hyper-parameter analysis will be included in the revised manuscript, along with discussions on how different settings affect the model's performance. We use ViT-small as the backbone and conducted experiments on the DukeMTMC dataset with mAP as the evaluation metric. The experimental results are as follows:\n | **$\\beta_t$** | **[1e-3, 0.02]** | **[1e-4, 0.02]** | **[1e-5, 0.02]** | **[1e-6, 0.02]** |\n |:--------------:|:--------------:|:-----------------:|:-----------------:|:-------------:|\n | mAP       | 81.15        | 81.22           | 81.21           | 81.13      |\n\n | **T** |  **100** | **500** | **1000** | **2000** | **5000** |\n |:------:|:------:|:-------:|:-------:|:-------:|:-------:|\n | mAP   |  80.92  | 81.15   | 81.22   | 81.19   | 80.98   |\n\nQ5:**What if the baseline methods are trained under the same steps?**\n\nA5: Thank you for your question. We conduct experiments to address this concern. We train the baseline methods using the same number of steps as our proposed method. However, we do not observe any significant performance improvements in the baseline methods. We will include these experimental results and detailed analyses in the revised manuscript to support this explanation. Thank you for your valuable feedback. The experimental results are as follows：\n\n |   epoch    |  120 |  160 |  200 |  240 |\n |:-------|:------------:|:------------:|:------------:|:----------:|\n | Baseline | 80.38%    | 80.39%    | 80.38%    | 80.36%   |\n | Ours   | 80.38% |  80.84% | 81.20% | 81.22% |\n    \n   where the original baseline trained for 120 epochs. Therefore, when the epoch is 120, our method did not participate in fine-tuning. When the epoch is 160, our method fine-tunes for another 40 epochs based on the pre-trained parameters of the baseline, and so on.\n  \n[1] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick. Microsoft COCO: Common objects in context. In ECCV, 2014. 2, 5\n\n [2] He, Kaiming, et al. \"Mask r-cnn.\" Proceedings of the IEEE international conference on computer vision. 2017.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uEITwFgYTu", "reviewer": "Reviewer_Ckqo", "summary": "This paper proposes DenoiseReID to improve feature discriminative with joint feature extraction and denoising, in which FEFDFA is developed to merge parameters of the denoising layers into embedding layers. Experimental results show the proposed DenoiseReID improves performance.", "questions": "N/A", "limitations": "Yes", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1.\tThe proposed joint representation learning and denoising process.\n2.\tThe proposed FEFDFA is a computation-efficient algorithm.", "weaknesses": "The author solves all my quentions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1: **What constitutes this noise?**\n\nA1: We appreciate your detailed review. Experimental results of Table 3 (FEFDUF) could empirically demonstrated the hypothesis of \"the features obtained by backbone extraction are noisy\". FEFDUF includes a well-trained person ReID model, and a denoise model which take feature of the ReID model as input and predict its noise. During the training stage, the ReID model is always frozen, and denoise model is trained under normal process in DDPM (i.e. adding a gaussian noise to feature, taking the nosied feature as input and predicting the original gaussian noise). During the inference stage, given images, we first extract features A with the well-trained ReID model, then put the features into the denoise module to predict its noise A'. Finally, we found that A-A' performs stably better than A. We will add the analysis above to the revised manuscript.\n&emsp;\n\nQ2: **The authors should evaluate the proposed method on these datasets instead of conventional ones.**\n\nA2: Thank you for the valuable suggestion. We validate our proposed FEFDFA on Occluded-Duke [1], Occluded-ReID [2] and Partial-ReID [3]. These datasets include person images occluded by misc obstacles such as vehicles, trees and so on, making them more challenging. We will add the experimental results to the revised manuscript. The experimental results are as follows, using mAP as the evaluation metric:\n |   Dataset    |  P-ReID |  OCC_Duke | OCC-ReID | \n |:-------|:------------:|:------------:|:------------:|\n | TransReID |  72.3%    | 59.5%    | 71.2%    | \n | TransReID+FEFDFA   |  73.5% (↑1.2%) |   60.4%  | 72.0% |\n\nQ3: **The authors lack an analysis of the model's complexity, including the number of parameters and training time.**\n\nA3: Thanks for your valuable feedback. Our proposed method takes LITTLE extra parameters and training time. The reason is the denoising layers employ simple linear layers, they are very light compared with heavy vision backbone (e.g. ViT, ResNet). For example, the parameter of ViT-Small is 22,313,320, while the sum of the parameters of the denoising module is only 3,538,944, an additional increase of about 15.8%. The detailed experimental results are as follows. Please note that these are training-time parameters, our proposed method takes no extra latency during the test stage.\n |   Backbone    |  Baseline |  Ours | Increase | \n |:-------|:------------:|:------------:|:------------:|\n | ViT-Small |  22,313,320    | 25,852,264    | 15.8%    | \n | ViT-Base   |  89,781,544 |   103,937,320 | 11.6% |\n\n\nQ4: **The performance gain are minimal.**\n\nA4: Thanks for your valuable feedback. Our proposed method is test-time computation-free (FREE LAUNCH!) and scalable to various baseline and downstream tasks, which is proven to be effective in CNN series (e.g. ResNet50), Transformer series (e.g. ViT, Vmanba), Person Re-ID (MSMT, Duke), Vehicle Re-ID (vehicReID), Image Classification (ImageNet, CUB). The latest experimental results on CLIP-REID show better improvement on MSMT (CLIP-REID: 75.8% v.s. CLIP-REID+OURS: 76.5% for mAP, see the table below). The code will be released after the manuscript is accepted. We will add the experimental results to the revised manuscript.\n&emsp;\n\nQ5: **The authors should verify the effectiveness of the proposed method using the visual encoder in CLIP as the backbone.**\n\nA5: Thanks for your insightful suggestion. CLIP is a very strong vision-text encoder, which is trained with 400 million data. Beyond CLIP, CLIP-ReID pioneeringly adapts CLIP, a zero-shot classifier, to ReID, a fine-grained image retrieval task. Our proposed FEFDFA is model-free, thus it can be easily applied to CLIP-ReID without any modification. The experimental results on CLIP-REID show stable improvement on MSMT (CLIP-REID: 75.8% v.s. CLIP-REID+OURS: 76.5% for mAP). The code will be released if the manuscript could be accepted. We will add the experimental results to the revised manuscript. The experimental results are as follows, with mAP as the evaluation metric:\n\n |   Dataset    |  DukeMTMC |  MSMT | Market-1501 | \n |:-------|:------------:|:------------:|:------------:|\n | CLIP-REID |  83.1%    | 75.8%    | 90.5%    | \n | CLIP-REID+FEFDFA   |  83.9%(↑0.8%) |   76.5%(↑0.7%) | 91.1% |\n\n\n [1] Jiaxu Miao, Yu Wu, Ping Liu, Yuhang Ding, and Yi Yang. Pose-guided feature alignment for occluded person re-identification. In ICCV, pages 542–551, 2019. 1, 2, 6\n\n [2] Jiaxuan Zhuo, Zeyu Chen, Jianhuang Lai, and Guangcong Wang. Occluded person re-identification. In ICME, pages 1–6. IEEE, 2018. 2, 6\n\n[3] Wei-Shi Zheng, Xiang Li, Tao Xiang, Shengcai Liao, Jianhuang Lai, and Shaogang Gong. Partial person reidentification. In ICCV, pages 4678–4686, 2015. 6", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper received all accept recommendations. Reviewers found the proposed method novel and interesting. AC agrees with reviewers and is happy to accept this paper for publication.
Decision: Accept (oral)