SF(DA)$^2$: Source-free Domain Adaptation Through the Lens of Data Augmentation
Authors: Uiwon Hwang
Jonghyun Lee
Juhyeon Shin
Sungroh Yoon
Venue: ICLR 2024 poster (2024)
Keywords: Source-free domain adaptation
Data augmentation
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We propose a novel source-free domain adaptation method that leverages intuitions derived from data augmentation

Abstract: In the face of the deep learning model's vulnerability to domain shift, source-free domain adaptation (SFDA) methods have been proposed to adapt models to new, unseen target domains without requiring access to source domain data. Although the potential benefits of applying data augmentation to SFDA are attractive, several challenges arise such as the dependence on prior knowledge of class-preserving transformations and the increase in memory and computational requirements. In this paper, we propose Source-free Domain Adaptation Through the Lens of Data Augmentation (SF(DA)$^2$), a novel approach that leverages the benefits of data augmentation without suffering from these challenges. We construct an augmentation graph in the feature space of the pretrained model using the neighbor relationships between target features and propose spectral neighborhood clustering to identify partitions in the prediction space. Furthermore, we propose implicit feature augmentation and feature disentanglement as regularization loss functions that effectively utilize class semantic information within the feature space. These regularizers simulate the inclusion of an unlimited number of augmented target features into the augmentation graph while minimizing computational and memory demands. Our method shows superior adaptation performance in SFDA scenarios, including 2D image and 3D point cloud datasets and a highly imbalanced dataset.

Published as a conference paper at ICLR 2024
(F SOURCE-FREE DOMAIN ADAPTATION THROUGH THE LENS OF DATA AUGMENTATION
Uiwon Hwang' Jonghyun Lee²
Jubyeon Shin
Sungroh 2.3+
1 Division of Digital Healthcare, Yonsei University
2 Department of Electrical and Computer Engineering, Seoul National University
3 Interdisciplinary Program in Artificial Intelligence, Scoul National University
u won. hwa hwangeyosse .ac.kr, {leejh9611, newjh12, sryoon) (rrcon)@sru..o @sru. ac. kr
ABSTRACT
In the face of the deep learning model's vulnerability domain shift, source-free domain adaptation (SFDA) methods have been proposed to adapt models to new, unseen target domains without requiring access source domain data. Although the potential benefits of applying data augmentation to SFDA are attractive, several challenges arise such as the dependence on prior knowledge of class- glass-preseriin transformations and the increase in memory and computational requirements. In this paper, we propose fource-free Domain Adaptation Through the Lens of Data Augmentation (SF(DA))). novel approach that leverages the benefits of data augmentation without suffering from these challenges. We construct an augmentation graph in the feature space of the pretrained model using the neighbor relationships between target features and propose spectral neighborhood clustering propose implicit identify partitions in the prediction space. Furthermore, feature augmentation and feature disentanglement as regularization loss functions that effectively utilize class semantic information within the feature space. These regularizers simulate the inclusion of an unlimited number of augmented target features into the augmentation graph while minimizing computational and memory demands. Our method shows superior adaptation performance in SFDA scenarios, including 2D image and 3D point cloud datasets and a highly imbalanced dataset.
INTRODUCTION
In recent years, deep learning has achieved significant advancements and widely explored for real- world applications. However, the performance of deep learning models can significantly deteriorate when deployed on unlabeled target domains, which differ from the source domain where the training data was collected. This domain shift poses challenge for applying deep learning models in practical scenarios. To address this, various domain adaptation (DA) methods have been proposed to adapt the
model to new, unseen target domains (Chen et al. 2019 Xu et al., 2019 Jin et al. 2020 Na et al.
2021 Long et al.] 2013) Ganin & Lempitsky 2015 Tzeng et al. 2017]QQn et al. 2019). Traditional domain adaptation techniques require both source domain and target domain data. However, practical limitations arise when source domain data is inaccessible or difficult to obtain due to cost or privacy concerns. Source-free domain adaptation (SFDA) overcomes the need for direct access to the source domain data by using only model pretrained on the source domain data. SFDA methods focus on adapting the model the target domain via unsupervised or self-supervised learning, which leverages the class semantic information learned from the source domain data (Liang et al. 2020 Yang et al.
2021a Lee et al. 2022 Yang et al. 2022; Zhang et al. 2022).
Recent studies (Chen et al. 2022 Zhang et al. 2022) have explored using data augmentation to enhance adaptation performance by enriching target domain information with transformations such as flipping or rotating images. These methods, however, require prior knowledge to ensure the preservation of class semantic information within augmented data. For instance, applying a 180- degree rotation to image of the digit 6 would result in the digit 9, which compromises the class semantic information. Moreover, increasing the number of augmented samples can lead to higher
"Corresponding author

---

Published as a conference paper at ICLR 2024
Target domain data
Augmentation graph G(Z,e) feature space
Spectral neighhor clustering (Agsc) prediction space
(Sec. 3.2)
3.3)
3/1 24 (z,, AEy))
Implicit feature augmentation (firA)
dist(Ex.
Feature disentanglement (fym) 3.4).
Overview of SF(DA) Here, I and g indicate the feature extractor and the classifier, denotes target feature, Σ is covariance matrix for class, and dist(.,) .) denotes distance measure.
Figure I:
memory usage and computational load. In response to these challenges, we aim to devise an SFDA method that effectively harnesses the advantages of data augmentation while mitigating its drawbacks.
In this paper, we propose a novel approach named Source-free Domain Adaptation Through the Lens of Data Augmentation (SF(DA))). We present unique perspective of SFDA inspired by data augmentation, which leads us to the introduction of an augmentation graph in the feature space of the pretrained model. The augmentation graph depicts relationships among target features using class semantic information learned by the pretrained model. Building upon the augmentation graph, the proposed method consists of spectral neighborhood clustering (SNC), designed for maximal utilization of the information from target domain data samples, and implicit feature augmentation (IFA) along with feature disentanglement (FD), designed to leverage additional information from the estimated distribution of the target domain data.
More specifically, we propose SNC within the prediction space to partition the augmentation graph via spectral clustering. The SNC loss promotes clustering in the prediction space and guides the classifier in assigning clusters, which facilitates effective adaptation. Furthermore, we explore the inclusion of augmented target features (vertices) into the augmentation graph. This allows us to identify better clusters, which might be challenging to discover using only the given target domain data samples. To this end, propose two regularization strategies: IFA and FD. We derive the IFA loss as a flosed-form upper bound for the expected loss over an infinite number of augmented features. This formulation of IFA minimizes computational and memory overhead. The FD loss serves as a crucial component in preserving class semantic information within the augmented features by disentangling the feature space.
We performed experiments on challenging benchmark datasets, including VisDA Peng et al. 2017) DomainNet (Peng et al. 2019). PointDA (Qin et al. 2019), and VisDA-RSUT (Li et al. 2021) )). We verified that our method outperforms existing state-of-the-art methods on 2D image, 3D point cloud, and highly imbalanced datasets. Furthermore, we observed that IFA and FD effectively boost the performances of existing methods. The contributions of this work can be summarized as follows:
We provide a fresh perspective on SFDA by interpreting it through the lens of data augmen- tation. Then, propose SF(DA)2 that thoroughly harnesses intuitions derived from data augmentation without explicit augmentation of target domain data.
We propose the spectral neighborhood clustering (SNC) loss for identifying partitions in the augmentation graph via spectral clustering. This approach effectively clusters the target domain data in the prediction space.
We derive the implicit feature augmentation (IFA) loss, which enables us to simulate the effects of an unlimited number of augmented features while maintaining minimal computational and memory overhead. To supplement the effectiveness of IFA, we propose the feature disentanglement (FD) loss. This further regularizes the feature space to learn distinct class semantic information along different directions.
We validate the effectiveness of the proposed method through experiments performed on challenging 2D image and 3D point cloud datasets. Our method significantly outperforms existing methods in the SFDA settings.
2

---

Published as a conference paper at ICLR 2024
2 RELATED WORK
Source-free Domain Adaptation SHOT Liang et al. 2020 utilizes information maximization and pseudo-labeling with frozen classifier weights. COWA-JMDS (Lee et al., 2022) estimates Gaussian mixture in the feature space to obtain target domain information. NRC (Yang et al. 2021a) and AaD YYng et al. 2022) are rooted in local consistency between neighbors in the feature space. Contrastive learning- based methods such as DaC (Zhang et al.. 2022) and AdaContrast (Chen et al. 2022) involve data augmentation, which requires not only prior knowledge to preserve class semantic information within augmented data but also computational and memory overhead. Our method circumvents these issues, providing efficient adaptation without explicit data augmentation.
Data Augmentation Data augmentation improves the generalization performance of the model by applying class- praseerngg transformations to training data and incorporating the augmented data in model training. To minimize the reliance prior knowledge, several studies propose optimization techniques to find combinations of transformations within constrained search spaces (Cubuk et al., 2019) Lee al. 2020; ZZeng et al.) 2022). ISDA (Wang al., (219)) implicitly augments data by optimizing an upper bound of the expected cross-entropy loss of augmented features. We propose implicit feature augmentation tailored for SFDA settings.
Spectral Contrastive Learning HaoChen et al. (2021) propose contrastive loss with the augmenta- tion graph, connecting augmentations of the same data sample and performing spectral decomposition. They prove that minimizing this loss achieves linearly separable features. Our work is motivated by spectral contrastive learning but with two key differences. First, we define the augmentation graph in the feature space, leveraging class semantic information learned by the pretrained model to directly enhance predictive performance on target domain data. Second, our method does not rely on explicit data augmentation for positive pairs; instead, we utilize neighbors in the feature space as positive pairs, considering them as highly nonlinear transformation relationships. We further discuss differences between existing work and the proposed method, well additional related work in Appendix D
3 PROPOSED METHOD
In the following sections, we propose Source-free Domain Adaptation Through the Lens of Data Augmentation (SF(DA)2). We first formulate the scenario of SFDA 3.11 From the intuitions of data augmentation, we define the augmentation graph in the feature space of the pretrained model (Section 3.2 find partitions of the augmentation graph (Section 3.3). and implicitly augment features (Section 3.4 An overview of SF(DA)2 is illustrated in Figure
3.1
SOURCE-FREE DOMAIN ADAPTATION SCENARIO
We consider source domain data D, = (xx),)))) where the class labels y belong to C classes. We also consider unlabeled target domain data D. = {xx ) sampled from target domain data distribution P(X*). Target domain data have the same C classes DD in this paper (closed-set setting). In SFDA scenarios, we have access to a source pretrained model consisting of a feature extractor and classifier g. The feature extractor ff takes a target domain data sample as input and generates target features z; = fxx).. The classifier g consists of a fully connected layer and predicts classes from the target features p; = ((g(2,) where 8 denotes the softmax function.
3.2 AUGMENTATION GRAPH ON FEATURE SPACE
In the proposed method, we focus the relationships between target domain data samples within the feature space of the pretrained model. We then build an augmentation graph grounded in our intuitions on the SFDA scenario.
Deep neural networks trained on source domain data encode class semantic information into distinct directions in their feature space (Wang et al.) 2019: Bengio al.) 2013). For example, different hairstyles for the 'person' class or wing shapes for the "airplane" class can be represented along specific directions. This property enables the pretrained model to map target domain data samples
3

---

Published as a conference paper at ICLR 2024
with similar class semantic information close to another. From this motivation, we make the following assumption:
Intuition 1 (Clustering assumption of source model). Target domain data that share the same semantic information are mapped their neighbors the feature space of the pretrained model. In this contex,, neighbors refer larget features with small cosine distances to a given larget, feature.
We also hypothesize that target domain data samples with shared class semantic information can be connected through highly nonlinear functions, representing potential augmentation relationships. Intuition 2 (Augmentation assumption of target domain data). Target domain data sharing class semantic information may have highly nonlinear functions to transform each other.
To formalize our intuitions of the relationships between target domain data, we introduce the concept of augmentation graph (HaoChen al., (02T). The augmentation graph consists of vertices representing target domain data and edge weights representing the probability of augmentation relationships between the target domain data. From our intuitions, samples with shared semantic information are neighbors in the feature space. These samples have also augmentation relation- ships, enabling their connection in the augmentation graph. Consequently, we can construct the augmentation graph on the feature space, which provides a structured representation of target domain data.
Given set of all target features in the population distribution of the target domain 2 {x fxx')lx ~ P. we define the population augmentation graph G(Z,e) e) with edges egj between two target features and zj follows:
Bij = (Zi k3) = Pr(z, E N.)
where N; is the set of neighbors of z;.
3.3 FINDING PARTITION ON PREDICTION SPACE
Using the augmentation graph, our next objective is to find meaningful partitions or clusters of the graph that represent different classes. To achieve this, we employ spectral clustering the graph
(Shi & Malik 2000 Ng et al. 200T), which reveals the underlying structure of the data. In particular,
HaoChen et al. (2021) proposed loss function that performs spectral clustering on the augmentation
graph:
Lemma 1 (HaoChen et al. 2021)). Let L be the normalized Laplacian matrix of the augmentation graph G(X,I). The matrix H. which has the eigenvectors corresponding to the largest eigenvalues of L as its columns, can be learned as function h by minimizing the following matrix factorizaiii
loss:
min C(h) - -L) -
(2)
La Ma)
const -2
+ +I (Maa))(aa))
(3)
where 24 and I2 vertices of the augmentation graph, lay,, is edge between 2; and Zz. L. = =E, denotes the degree of vertex Iú, and const is a constant term.
Using Lemma [ describe approach for finding partitions in the augmentation graph, focusing on the prediction space. Since we have the target domain data D, sampled from P(X'), ), we now build an instance of the population augmentation graph denoted by G. Each target feature is represented as a vertex. We consider edges only between the K --nearest neighbors for each vertex, assuming that these neighbors share the same class semantic information. The edge weight eij is set to 1 if 2, is among the K-nearest neighbors of z; in the feature space, denoted by N To efficiently retrieve the nearest neighbors, we utilize two memory banks J and S to retain all target features and their predictions as used in the previous studies (Yang et al. 2022, 2021a Saito et al. 2220). Since all samples are i.i.d. and each target feature has the same number of neighbors, the degree of all target features is equal (i.e., e; = Ej K for Vi,j [0, M.)). Given mini-batch B {x{) 16 of target domain data samples, we can derive the spectral neighborhood clustering (SNC) loss on G:
Lsnc(p.) = K
2
Σ JENK
PTP;
11 + Σ (p. pk)
(4)
REB
4

---

Published as a conference paper at ICLR 2024
The first term attracts the predictions of neighborhood features, leading to tight clusters in the prediction space, similar to previous approaches YYngg et al., [022 (02laab). In contrast, the second term encourages distinct separation between different clusters by driving the predictions of all target features apart from each other. We further incorporate a decay factor into the second term to make predictions more certain adaptation proceeds. As suggested in (Yang et al. (022), multiply the second term by (1 + 10 X
max_iter ) where we set 3 to 5 in our experiments. Overall, the SNC loss facilitates clustering in the prediction space and supports cluster assignment by the classifier g.
Her
3.4 IMPLICIT FEATURE AUGMENTATION
In this section, we aim obtain improved partition by further approximating the population augmentation graph G. We achieve this by augmenting target features connected to individual target features. The key focus of this section is perform feature augmentation without relying on prior knowledge.
To preserve class semantic information while augmenting target features, we revisit the property of deep learning models <Wang et al., 2019 Bengio et al. 2013). which suggests that class semantic information is represented by specific directions and magnitudes in the feature space. We discern the direction and magnitude for each class by estimating the class-wise covariance matrix of target features (denoted as Σ {EeJe=)) online during adaptation using mini-batch statistics. The detailed method for online covariance estimation, adopted from ISDA (Wang et al. 2019). is described in Appendix C.3 As target features lack class labels, we use their pseudo-labels y/: arg max, Pt to update the covariance matrices accordingly.
Using the estimated covariance matrices, we can preserve class semantic information in augmented features by sampling features from Gaussian distribution. Since an edge (Equation I connects target feature with its neighbors, we set the mean of the Gaussian distribution to the target feature itself, and we the estimated covariance matrix corresponding to the pseudo-label the variance:
ÿ.. 2g.
N(2 AEg.)
(5)
At the beginning of adaptation, the estimation of covariance matrices may be inaccurate. Therefore, following Wang et al. (2019), we set A = Ao x max_Rer and Ao 5 in experiments. This gradually increases the impact of the estimated covariance matrices on the model as the adaptation proceeds.
iter
Augmented features can be used in calculating the first term of the SNC loss to promote similar predictions. However, directly sampling features from the Gaussian distribution during adaptation can increase training time and memory usage, undesirable in the SFDA context. Instead, aim to derive an upper bound for the expected SNC loss. Since obtaining a closed-form upper bound for the first term of the SNC loss is intractable, we consider its logarithm. This simplifies the solution while preserving the goal of adaptation. The explicit feature augmentation (EFA) loss of two augmented features, zy and ž₁. sampled from the estimated neighbor distribution of z expressed as follows:
LEFA (žj, %x) = log 55 Pk:
where أÚ o(g(),)) and pk )()(2)) are the predictions of the augmented features.
We then derive an upper bound for the expected EFA loss and propose the implicit feature augmenta- tion (IFA) loss as follows:
Proposition 1. Suppose that ž. 2τ ~ NZ,,, XEg.). ), then we have an upper bound of expected LEFA for an infinite number of augmented features, which we call implicit feature augmentation loss:
CEPA (zn f.g) = B2,
(e [- (-log p3 pa]
exp(g(zi)c)
2
log
LIFA(Ze Eg..
(8)
g(z.)e + A (ww wo) I (we we)
c=1
exp
where ge denotes the classifier outpul (logit) for the c-th class, and w. is the weight vector for the c-th class of the classifier g.
Proof of Proposition]2 is provided in Appendix B Optimizing the IFA loss allows us to achieve the effect of sampling infinite number of features with minimal extra computation and memory usage.
5

---

distributed in distinet Published as a conference paper at ICLR 2024
In the denominator of the IFA loss, the first term is proportional to the prediction for the c'-th class of the target feature. The second term relates to the square of the cosine similarity between the normal vector of the decision boundary separating the e-th and c'-th classes, and the direction of feature augmentation. Namely, the denominator can be interpreted as a regularization term that aligns the decision boundary between the predicted and other classes with the principal direction of the covariance matrix of the predicted class. This is in accordance with the findings by Balestriero et al. (2022), which suggest that the explicit regularizer of data augmentation encourages the kernel space of the model's Jacobian matrix to be aligned with the principal direction of the tangent space of the augmented data manifold. Recalling that the target feature is augmented to the direction of variance of target features with the same pseudo-label, the IFA loss promotes the decision boundaries for each class to align with the principal direction of the variance of target features with the same pseudo-label.
Maintaining class semantics in augmented features relies heavily on a well-disentangled feature space. similar classes, such as cars and trucks, have similar covariances, suggests that the feature space may not capture their semantic differences sufficiently. This could lead to non-class-presevv transformations during augmentation, possibly undermining the model's performance when the IFA loss is incorporated. To mitigate this, more disentangled feature space is needed, ensuring distinct representation for each class.
To encourage each direction in the feature space to represent different semanties (i.c., to promote disentangled space), we maximize the cosine distance between covariance matrices corresponding to similar classes. This indicates that features with different pseudo-labed directions. Consequently, the semantic information of different classes is encoded in separate directions within the feature space, resulting in a disentangled feature space. We propose the feature disentanglement (FD) loss as follows:
tr EHE ES}
LFD Σ dij 1
where Σ, and Σy denote the covariance matrices for the i-th and j-th classes, respectively, and the weight aij serves dot product of the mean prediction vector (i.e., aij- o Pj. where P. [[ Eie(se me) Pc). Therefore, the FD loss places greater emphasis class pairs that exhibit similar predictions on the target domain data. The weights are calculated at the beginning of each epoch.
measure of similarity between i-th and j-th classes, and it is calculated the
The final objective for adaptation can be expressed as follows:
min LsNc a CICIFA 02LFD
(10)
f.g
where dn and 22 denote hyperparameters.. The procedure SF(DA)" is presented Algorithm
EXPERIMENTS
In this section, evaluate the performance of SF(DA)² on several benchmark datasets: Office-
Algorithm 1 Adaptation procedure of SF(DA)2
31 (Saenko et al. 2010). VisDA (Peng al.)
Require: f and g (trained on D.). D; = {x{]
2017). DomainNet (Peng et al. 2011)) PointDA-
1: while training loss is not converged do
10 (Qin al. 2019), and ViSDA-RSUT (Li al. 2021). The results on Office-31 and details of the datasets are provided in Appendix and
2: if epoch start then
Update asj for FD loss 4: end if'
3:
respectively.
5: Sample batch B from Dj and update F,
6: Retrieve neighbors A²K for each z; in B
Implementation details For fair compari-
7: Update of and g using SGD
son, we adopt identical network architectures, optimizers. and batch sizes as benchmark meth-
8: V LsNc + an LIFA + a2LFD
9: end while
ods (Liang al. 2020 Yang et al. 202 1 2022).
We run our methods with three ditferent random seeds and report the average accuracies. SF in the tables denotes source-free. More implementation details presented in Appendix C.I
Most hyperparameters of our method do not require heavy tuning. We set K to 5 on VisDA, PointDA-100, and VisDA-RSUT, and 2 on DomainNet. We set on to le-4 on VisDA, DomainNet, and

---

Published as a conference paper at ICLR 2024
Table Accuracy (%) on the VisDA dataset (ResNet-101).
SF
plane
bicycle
meyel
person
plant
Per-class
BSP Chen et ml
2019
X
92.4
61.0
81.0
57.5
89.0
80.6
90.1
77.0
$4.2
77.9
82.1
38.4
75.9
61.3
79.0
76.1
MCC
20.0
X
88.7
80.3
80.5
71.5
90.1
93.2
85.0
71.6
89.4
73.8
85.0
36.9
78.8
FixBi
x
96.1
87.8
90.5
90.3
96.8
95.3
92.8
88.7
97.2
94.2
90.9
25.7
87.2
Source only et
2016
60.9
21.6
50.9
67.6
65.8
6.3
82.2
23.2
57.3
30.6
84.6
8.0
46.6
3C GAN (li'e eli
94.8
73.4
68.8
74.8
91.1
95.4
88.6
84.7
89.1
84.7
83.5
48.1
81.6
SHOT [Lang
2020
87.5
80.4
92.9
95.1
83.1
89.2
85.8
56.9
NRC Yang ai 2021a
96.1
90.8
83,9
61.5
95.7
95.7
84.4
$0.7
94.0
91.9
89.0
59.5
85.3
COWA
2022
AaD YYang et al, 2022
96.8
89.3
83.8
82.8
96.5
95.2
90.0
81.0
95.7
92.9
88.9
54.6
$7.3
SF(DA)
89.3
82.9
95.7
90.4
88.5
64.7
88.1
Table 2: Accuracy (%) on 7 domain shifts of the DomainNet-122 126 dataset (ResNet-50).
Mothod
SF
S-+P
C-8
P--C
P-R
R-+S
R-CC
R-P
Avg.
2020
35.3
Source
only [Ile al
2016
50.1
46.9
52.0
75.0
46.3
55.5
62.7
55.6
TENT
SHOT
bany el 2020
66.1
60.1
66.9
80.8
59.9
67.7
68.4
67.1
NRC
Yang
2021a
65.7
58.6
64.5
82.3
58.4
65.2
68.2
66.1
Yang Brat
SF(DAY
67.7
89.6
67.8
83,5
60.2
68.8
70.5
68.3
Table 3: Comparison of SF(DA) 2 and other two-stage methods on VisDA (ResNet-101).
Methed
SHOT++ (Linng al. 2021
feat-mixup SHOT++
Kundu al. 2022
SF(DA)
Per-class
87.3
87.8
89.6
Table 4: Accuracy (%) on the PointDA-10 dataset (PointNet).
Method
SFF Model Shape Model Scan Shape MMdel Shape->Scir Scan-MMode Scan Scan-+Shape
Avg.
MMD
Long et al. 2013
57.5
27.9
40.7
26.7
47.3
54.8
42.5
DANN
Lamin & Lempitky 2015
58.7
29.4
42.3
30.5
48.1
56.7
44,2
17eng
30.5
51.1
MCD
et 208
X
62.0
31.0
41.4
31.3
46.8
59.3
45,3
PoinDANN (Qi
33.0
Source only
Oi
2017
43.1
17.3
40.0
15.0
33.9
47.1
32.7
VDM-DA Than el al
20/1
58.4
30.9
61.0
40.8
45.3
61.8
49.7
NRC
Yany
64.8
25.8
59.8
26.9
70.1
68.1
52.6
AaD
Yang 2t 2022
69.6
34.6
67.7
28.8
68.0
66.6
55.9
SF(DXF
70.3
35.5
68.3
29.0
70.4
69.2
57.1
PointDA-10. and le-3 on VisDA-RSUT. We 02 10 on VisDA, PointNet-10, and VISDA-RSUT, and 1 on DomainNet.
4.1 EVALUATION RESULTS
2D and 3D Datasets We compare SF(DA)2 with the source-present and source-free DA methods on 2D image datasets. For VisDA, we reproduce SHOT, NRC, CoWA-IMDS, AaD using their official codes and the pretrained models released by SHOT (Liang et al.) 2020). As shown in Table our method outperforms all other methods on VisDA in terms average accuracy. We find similar observations on the results on DomainNet in Table
Additionally, we extend SF(DA)2 into two-stage version called SF(DA) To ensure a fair comparison, utilize the second training stage, which follows the same approach as SHOT+ (Liang et al. 2021). The results presented Table demonstrate that SF(DA) "++ outperforms other two-stage methods on the VisDA dataset.
We also conduct comparisons on a 3D point cloud dataset, PoinDDA-10. In Table 4 our method significantly outperforms not only PointDAN (Qin et al. 2019] especially designed for source-present domain adaptation on point cloud data, but also cource-fres methods by a large margin (about 4.5%p).
These results clearly demonstrate the effectiveness of SF(DA)2 in domain adaptation.

---

Published as conference paper at ICLR 2024
Table Accuracy (%) on the VisDA- dataset (ResNett1101).
SF
plane
bicycle
meyel
person
plant
DANN (Gaiin & Lempitsky. 2015
X
71.7
35.7
58.5
21.0
80.9
71.0
45.7
23.7
12.2
4.3
1.5
0.9
35.8
(Chen
MCD sant al. 2018
x
63.0
41.4
84.0
67.3
86.6
93.9
85.6
76.3
84.1
11.3
5.0
3.0
58.5
only
2016
15.7
SHOT [Linng el mil 201200
86.2
48.1
77.0
62.8
92.
66.2
90.7
61.3
76.9
73.5
67.2
9.1
67.6
CoWA JMDDS
2022
63.8
32.9
69.5
59.9
93.2
95.4
92.3
69.4
85.1
68.4
64.9
32.3
68.9
NRC Yang et 2023
86.2
47.6
66.7
68.1
94.7
76.6
93.7
63.6
87.3
89.0
83.6
20.5
73.1
AaD Yang
73.9
33.3
56.6
714
90.1
97.0
91.9
70.8
88.1
87.2
81.2
39.4
73.4
SF(DA)
43.3
73.6
92.8
98.3
93.4
81.1
Vanilla
+IBA F
88:1
Parclas (%)
6910
$7.3
86.9
87.6
75.0
74.3
75.2
85.6
86.5
X2.0
68.9
04.0
88.0
$3.0 92.8
62.5
660
SNC (ous)
AaD
CoWA-JMDS
NRC
SHOT
SNC (ours)
Aali
CoWA-JMIDS
NRC
SHOT
(a) VisDA
(b) VisDA-RSUT
Figure 2: Effectiveness of implicit feature augmentation.
Imbalanced Dataset In real-world domain adaptation scenarios, the classes of data can often be imbalanced. In order evaluate our method in these scenarios, we perform comparison of our method with competitive benchmarks on VisDA-RSUT, a dataset with highly imbalanced classes. For VisDA-RSUT we reproduce SHOT, NRC, CoWA-JMDS, AaD using their official codes. The results presented in Table demonstrate that our method surpasses the baseline methods by significant margin (about 4%p). The results show the robustness of our method on extreme class imbalance. We also present results using additional metries in Appendix A.2
4.2 ANALYSIS
Effectiveness of Implicit Feature Augmentation FigureZiilussratees the performance improvement of existing SFDA methods and our SNC after incorporating [FA and FD losses. The results highlight that IFA and FD improve most existing SFDA methods. Remarkably, their effectiveness is prominent in VisDA-RSUT, as shown in Figure (b). leading to a 3%p performance gain on CoWA-IMDS and SNC. The significant improvement from incorporating IFA and FD into SNC can be attributed to their effective feature augmentation on the augmentation graph. The substantial performance gain in CoWA-JMDS can stem from its use of Gaussian distribution estimation the feature space. For AaD and NRC, their use of neighbor information in the feature space aligns well with the assumptions of IFA, leading to performance enhancement. However, the assumptions of SHOT do not align well with IFA, resulting in slight performance decrease when IFA loss is incorporated.
Loss Functions for Implicit Feature Augmentation We conduct ablation studies on the VisDA- RSUT dataset to understand the impact of IFA and FD loss functions. In Figure 3 we measure (a) average accuracy and (b) the cosine similarity between the weight vector and the largest eigenvector of the estimated covariance matrix the corresponding class. During adaptation with SNC, we observe a consistent increment in the cosine similarity.
We then analyze the impact adding IFA or FD loss alone to SNC loss. Firstly, adding IFA alone decreases the cosine similarity, aligning the Jacobian matrix of the classifier (w) with the principal direction the tangent space of the augmented data manifold (orthogonal with v), as discussed in Section 3.4 However, the sole addition of IFA results in performance decrease. If different directions in the feature space do not clearly correspond to distinct class semantics, performing feature augmentation in such entangled feature space may not preserve class information, causing
8

---

Published as a conference paper at ICLR 2024
SNC
SNC+FD
SNCHIFA
SF(D
77.3
74.3
Cosine similaity
Par-dlus
72.1
64
SNC
SNC-ro
SNCHBA
SF(DA)
Epochs
(a) Average accuracy
(b) Average cosine similarity
Figure Ablation study on loss functions for implicit feature augmentation.
Table 6: Runtime analysis on the VisDA dataset.
Method
Runtime (s/epoch)
Per-class
SHOT
353.53
83.0
AaD
274.36
87.3
SF(DA) (Ours)
284.49
88.1
Figure 4: tSNE visualization of the feature space before and after adaptation.
AaD (5% of F,S) SF(DA) (5% F)
266.76
85.9
273.52
87.4
a degradation in performance. Secondly, applying only FD loss introduces additional constraints to the model by disentangling the feature space. Therefore, the absence of term utilizing the disentangled space could result performance drop.
While the use of single loss leads to performance decrease, a synergistic effect arises when IFA loss and FD loss are jointly employed. The disentangled feature space obtained via FD loss allows the estimated covariance matrix to provide directions for class class-preserving feature augmentation, and IFA loss performs the feature augmentation. This leads to notable performance improvement and further decrease in cosine similarity (compared to SNC+IFA).
We also present the ablation study the hyperparameters used for the SNC loss in Appendix A.3
Runtime and tSNE Analysis We assess our method's efficiency by measuring runtime the VisDA dataset. The first three rows of Table 6 show our method outperforming SHOT and AaD with less runtime or marginal increase of runtime. The last two rows of Table limit the memory banks F and S to 5% of target domain data. Compared to AaD, our method shows more robust performance against the reduction of memory bank size with similar runtime. Figure visualizes visuaisess the feature space before and after adaptation on the VisDA dataset, with distinct colors indicating different classes. It clearly shows that target features are effectively clustered after adaptation.
CONCLUSION
In this work, we propose novel method for source-free domain adaptation (SFDA), called SF(DA) which leverages the intuitions of data augmentation. We propose the spectral neighbor clustering (SNC) loss to find meaningful partitions the augmentation graph defined on the feature space of the pretrained model. We also propose the implicit feature augmentation (IFA) and feature disentanglement (FD) loss functions to efficiently augment target features in the augmentation graph without linearly increasing computation and memory consumption. Our experiments demonstrate the effectiveness of SF(DA)2 in the SFDA scenario and its superior performance compared to state- of-the-art methods. In the future, plan to explore the potential of our method for other domain adaptation scenarios and investigate its applicability to other tasks beyond classification.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
This work was supported by the National Research Foundation of Korea (NRF) grants funded by the Korea government (Ministry of Science and ICT, MSIT)
and Institute of Information & Communications Technology Planning Evalu- ation (ITPP) grants funded by the Korea government (MSIT) (2021-0-0134) Artificial Intelligence Graduate School Program (Seoul National University) and 2022-0-00959), the BK21 FOUR program of the Education and Research Program for Future ICT Pioneers, Seoul National University in 2024, and 'Regional Innovation Strategy (RIS)' through the National Research Foundation of Korea(NRF) funded by the Ministry of (duCation(MOEE) (2022RIS-005).
REFERENCES
Randall Balestriero, Ishan Misra, and Yann LeCun. A data-augmentation is worth thousand samples: Analytical moments and sampling-free training. In Advances in Neural Information Processing Systems, 2022.
Yoshua Bengio, Grégoire Mesnil, Yann Dauphin, and Salah Rifai. Better mixing via deep representa- tions. In International conference on machine learning, pp. 552-560. PMLR, 2013.
Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 295-305, 2022.
Xinyang Chen, Sinan Wang, Mingsheng Long, and Jianmin Wang. Transferability vs. discriminability: Batch spectral penalization for adversarial domain adaptation. In International conference machine learning, pp. 1081-1090. PMLR, 2019.
Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation strategies from data. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 113-123, 2019.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by hackpropagation. In International conference on machine learning, pp. 1180-1189, PMLR, 2015.
Jeff Z HaoChen, Colin Wei, Adrien Gaidon, and Tengyu Ma. Provable guarantees for self-supervised deep learning with spectral contrastive loss. Advances in Neural Information Processing Systems, 34:5000-5011, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770-778, 2016.
Ying Jin, Ximei Wang, Mingsheng Long, and Jianmin Wang. Minimum class confusion for versatile domain adaptation. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28 2020, Proceedings, Part XXI 16, pp. 464 480. Springer. 2020.
Jogendra Nath Kundu, Akshay Kulkarni, Suvaansh Bhambri, Deepesh Mehta, Shreyas Anand Kulkarni, Varun Jampani, and Venkatesh Babu Radhakrishnan. Balancing discriminability and transferability for source-free domain adaptation. In International Conference on Machine Learn- ing. pp. 11710-11728. PMLR, 2022.
Donghoon Lee, Hyunsin Park, Trung Pham, and Chang D) Yoo. Learning augmentation network via influence functions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10961-10970, 2020.
Jonghyun Lee, Dahuin Jung, Junho Yim, and Sungroh Yoon. Confidence score for source-free unsupervised domain adaptation. In International Conference on Machine Learning, pp. 12365- 12377. PMLR, 2022.
Rui Li, Qianfen Jiao, Wenming Cao, Hau-San Wong, and Si Wu. Model adaptation: Unsupervised domain adaptation without source data. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9641-9650, 2020.
10

---

Published as a conference paper at ICLR 2024
Xinhao Li, Jingjing Li, Lei Zhu, Guoqing Wang, and Zi Huang. Imbalanced source-free domain adaptation. Proceedings of the 29th ACM International Conference on Multimedia, pp. 3330- 3339, 2021.
Jian Liang, Dapeng Hu, and Jiashi Feng. Do really need access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning, pp. 6028-6039. PMLR, 2020.
Jian Liang, Dapeng Hu, Yunbo Wang, Ran He, and Jiashi Feng. Source data-absent unsupervised domain adaptation through hypothesis transfer and labeling transfer. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11):8602- 0118666221177,- 2021.
Mingsheng Long, Jianmin Wang, Guiguang Ding, Jiaguang Sun, and Philip Yu. Transfer feature learning with joint distribution adaptation. In Proceedings of the IEEE international conference on computer vision, pp. 2200 2207, 2013.
Jaemin Na, Heechul Jung, Hyung Jin Chang, and Wonjun Hwang. Fixbi: Bridging domain spaces for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1094-1103, 2021.
Andrew Ng, Michael Jordan, and Yair Weiss. On spectral clustering: Analysis and an algorithm. Advances in neural information processing systems, 14, 2001.
Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. Visda: The visual domain adaptation challenge. arXiv preprint arXiv: 2017.
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 1406-1415, 2019.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652-660, 2017.
Can Qin, Haoxuan You, Lichen Wang, C-C Jay Kuo, and Yun Fu. Pointdan: A multi-scale 3d domain adaption network for point cloud representation. Advances in Neural Information Processing Systems, 32, 2019.
Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In Computer Vision-ECCE 2010: 11th European Conference on Computer Vision, Heraklion, Crete, Greece, September 5-11, 2010, Proceedings, Part IV 11, pp. 213-226. Springer, 2010.
Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3723-3732, 2018.
Kuniaki Saito, Donghyun Kim, Stan Sclaroff, and Kate Saenko. Universal domain adaptation through self supervision. Advances in neural information processing systems, 33:16282-16229 16292, 2020.
Jianbo Shi and Jitendra Malik. Normalized cuts and image segmentation. IEEE Transactions on pattern analysis and machine intelligence, 22(8):888-900 9088,995,2211 2000.
Jiayi Tian, Jing Zhang, Wen Li, and Dong Xu. Vdm-da: Virtual domain modeling for source data-free domain adaptation. IEEE Transactions on Circuits and Systems for Video Technology. 32(6):
3749-3760, 2021.
Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7167-7176. 2017.
Dequan Wang. Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test- time adaptation by entropy minimization. International Conference on Learning Representations, 2021.
11

---

Published as a conference paper at ICLR 2024
Yulin Wang, Xuran Pan, Shiji Song, Hong Zhang, Gao Huang, and Cheng Wu. Implicit semantic data augmentation for deep networks. Advances in Neural Information Processing Systems, 32, 2019.
Rujjia Xu, Guanbin Li, Jihan Yang, and Liang Lin. Larger norm more transferable: An adaptive feature norm approach for unsupervised domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1426-1435, 2019.
Shiqi Yang, Joost van de Weijer, Luis Herranz, Shangling Jui, et al. Exploiting the intrinsic neigh- borhood structure for source-free domain adaptation. Advances neural information processing systems, 34:29393-29405 2021a.
Shiqi Yang, Yaxing Wang, Joost Van De Weijer, Luis Herranz, and Shangling Jui. Generalized source-free domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8978 898788987444 2021b.
Shiqi Yang, Yaxing Wang, Kai Wang, Shangling Jui, et al. Attracting and dispersing: simple approach for source-free domain adaptation. In Advances in Neural Information Processing Systems, 2022.
Ziyi Zhang, Weikai Chen, Hui Cheng, Zhen Li, Siyuan Li, Liang Lin, and Guanbin Li. Divide and contrast: Source-free domain adaptation via adaptive contrastive learning. arXiv preprint arXiv:2211.0661,, 2022.
Yu Zheng, Zhi Zhang, Shen Yan, and Mi Zhang. Deep autoaugment. In International Conference on Learning Representations, 2022.
12

---

Published as a conference paper at ICLR 2024
A
ADDITIONAL RESULTS
A.1
ADDITIONAL. DATASET
For Office-31, reproduce SHOT, NRC, CoWA-JMDS, AaD using their official codes. Table presents comparison between SF(DA) and various DA methods, considering both source-present and source-free approaches. Notably, method exhibits the highest average accuracy among all source-free DA methods.
Table Accuracy (%) on the Office-31 dataset (ResNet-50).
Method
SF
A-D
A-W
D-A
D--
W-+A
W-+
Avg.
CDAN Long ml 2018
92.9
94.1
71.0
98.6
69.3
100.0
87.7
MCC
St 20201
95.6
72.6
98.6
73.9
100.0
89.4
SRDC
2020
95.8
95.7
76.7
99.2
77.1
100.0
90.8
Source only (He al.
2
68.9
68.4
62.6
96.7
60.7
99.3
76.
SHOT
Liang et 2020
93.8
98.9
75.3
99.9
88.7
NRC
Yang
202
92.9
93.5
76.0
98.1
75.8
99.9
89.4
3C 3C-GAN
LL
ef 2020
92.7
93.7
75,3
98.5
77.8
99.8
89.6
AnD (ang
94.5
94.5
75.6
98.2
75.4
99.9
89.7
SF(DAy
95.8
92.1
75.7
99.0
76.8
99.8
89.9
A.2 ADDITIONAL METRICS
SFDA methods have used accuracy as the performance metric for evaluation. However, accuracy can sometimes be misleading when used with imbalanced datasets. To mitigate this and provide more comprehensive evaluation, we add our analysis on the VisDA-RSUT dataset with the harmonic mean of accuracy and the Fi-score. In Tables and The results highlight the capability of our method in dealing with imbalanced datasets.
Table Harmonic mean of accuracies (%) on the ViSDA-RSUT dataset (ResNet-101)
Method
Source only
SHOT CoWA-IMDS NRC
AaD
SF(DA)
Harmonic
16.7
45.2
61.1
61.3
65.9
70.1
Table 9: Fi-score (%)
the VisDA-RSUT dataset (ResNet-101)..
Method
planc
bicycle
bus
car
horse
knife
meycl
person
plant
sktbrd
train
truck
Per-class
Source only
6.2
17.0
9.8
5.3
53.6
17.2
38.1
18.9
55.3
23.7
69.3
6.3
26.7
SHOT
9.0
8.7
9.5
72.9
35.1
83.1
85.1
82.9
79.7
14.4
45.8
CoWA-JMDS
17.7
12.1
75.4
42.2
76.7
62.3
87.9
78.1
50.1
NRC
36.4
4.2
15.5
11.4
67.5
48.4
78.6
61.2
90.3
92.8
90.6
36.9
52.8
AaD
63.9
23.7
20.4
12.1
76.6
46.3
81.8
61.9
87.2
90.1
88.6
56.0
59.0
SF(DA)
77.2
53.3
12.3
13.3
79.8
49.3
85.5
67.7
91.0
90.6
87.0
49.6
63.0
A.3 HYPERPARAMETERS
We present the ablation study of the hyperparameters used in SNC on the VisDA dataset. The second term of the SNC loss ensures diverse predictions for different target features, preventing them from collapsing into one class. During adaptation, weakening the second term can promote a more clustered prediction space and improve performance. Figure 5 shows the ratio of target features that share the same and correct prediction with the 5-nearest neighbors, and Figure (b) shows the average confidence for the predicted labels of the target domain data. As the decaying factor increases from to Figures demonsttteee that the target domain data are better clustered in (a) the feature space and (b) the prediction space. Figure 5 (c) shows that method is robust to the choice of hyperparameters 3 and K.
We also present the ablation study of the hyperparameters (a1 and a:2) used for IFA and FD losses on the VisDA dataset. As shown Tables 10 and the model's performance not sensitive to hyperparameters on and 022
13

---

Published as a conference paper at ICLR 2024
R=
f
-8-5
R K
0.40
Meen coufideess
Pen cloces (9)
Rotbo (%))
86
70
Itorations
Iterations
Value
@a) Ratio of correct shared prediction
(b) Mean confidence
e A blation study on B and
Figure 5: Ablation study on hyperparameters for spectral neighbor clustering.
Table 10: Ablation study on the hyperpa- rameter for spectral neighbor clustering.
Table 11: Ablation study on the hyperpa- rameter for feature disentanglement.
Per-class (%)
0r
Per-class (%)
le-3
87.3
1
88.0
lc-4
88.1
10
88.1
le-5
88.0
20
87.4
B PROOFS
Proposition 2. Suppose that 2,. 2y N(z,, ALg.). ). then we have an upper bound of expected CEFA for infinite number of augmented features, which we call implicit feature augmentation loss:
EFA (ze: f.g] = ((z,AA (ELndV(eLAE.)) [-log pk.
(11)
C
exp(e(Zi)o)
< <-
log
LIFA (Zi, Ege, g) (12)
cml
Σ exp
gizi)e
(wwe wo) Dx, (wo w.)
where g( denotes the classifier output (logit) for the e-th class, and we is the weight vector for the c-th class of the classifier g.
Proof. Let the classifier consists of the weight matrix W [ws ... wel" and the bias b bb...bbl bel. Now we will derive a closed-form upper bound of expected LEFA.
C (2i S.9) = (Eg
BEn [- 1o pM]]
(13)
exp(w? 2n + bc)
Rz
Ezg
logř; log
(14)
E exp(wx byr)
Ez
log Fi + log
exp (wwe w.) 2s + (be be))
(15)
c==l
E2
loggr + log Σ
Rz,
T 2 + (b2 be))l (exp (wwe w.)
(16)
E2
loo 99 + +og Σ
- -b) (10 w Ev, (w)
(17)
d=
14

---

Published as a conference paper at ICLR 2024
exp(w, 2i +
Eg
log Pj
log
(18)
c
2 exp u.:2 be 2 (wwe wel Ei Ewee wel)
exp(w? z; bo)
2 log
(19)
e
Σ exp d=
u.Z by 2/1r wo) (we we)
exp(g(zi)c)
log
(20)
a(zi)e A (wr xx (w/
==1 exp
(w/ we)
e-11
LIFA (Zi, -g,99)
(21)
where P denotes the prediction of 2, for the c-th class. Inequality iolis obtained by applying Jensen's incquality for concave functions (i.e., E|log X < log Σ B[X]) given that the sum of log functions
Σ Elo(()) is concave. To derive Equation 17 we require the lemma of the moment-generating function
for Gaussian random variable:
Lemma 2. For a Caussian random variable X ~ NN,, of, its moment moment-ggneratingg generating functiooo is:
E[c2X| e 2022
(22)
In the context of our proof. (w w.)' b + (b - ba) a random variable that follows the Gaussian distribution:
(we- w.) w.) 24 + (be bo) ~ N (mw - w.)"2, w.) 2, + (be be), A(wer - w.)" " Eg. (w> w.)) (23)
Lemma 2 then allows to compute the expectation of this random variable, resulting in:
Eg, [e((mrr (") 2 + (b. (b.. ba)) -b))) = w ww) tbb bb)b -w) (war-wa)
(24)
Since 2, and 7n are drawn from the same distribution, we use the same process to derive Equation concluding the proof.
C DETAILS
C.I IMPLEMENTATION DETAILS
We adopt the backbone of a ResNet-50 for DomainNet, ResNet-101 for VisDA and VisDA-RSUT, and PointNet for PointDA-10. the final part the network, we append a fully connected layer, batch normalization (Toffe & Szegedy| 2015). and the classifier y comprised of fully connected layer with weight normalization (Salimans & Kingma 2016). We adopt SGD with momentum 0.9 and train 15 epochs for VisDA, DomainNet, and RSUT. We adopt Adam (Kingma & Ba 2014) and train 100 epochs for PointDA-10 10. We set batch size to 64 except for DomainNet, where set it to 128 for fair comparison. We set the learning rate for VisDA and VisDA-RSUT to le-4, 5e-5 for DomainNet, and le-6 for PointDA- 10, except for the last two layers. Learning rates for the last two layers are increased by a factor of 10, except for PointNet-10 where they are increased by factor of 2 following NRC (Yang et al. 2021a). Experiments are conducted on a NVIDIA A40.
C.2 DATASET DETAILS
We use five benchmark datasets for 2D image and 3D point cloud recognition. These include Office-31 (Saenko et al., 2010), with 3 domains (Amazon, Webcam, DSLR), 31 classes, and total of 15,500 images; VisDA (Png et al. 2017). with 152K synthetic and 55K real object images across 12 classes: VisDA-RSUT (Li et al. 2021). subset of VisDA with highly imbalanced classes; DomainNet (Peng ct al. 2019), a large-scale benchmark with 6 domains and 345 classes (we select 4 domains (Real, Sketch, Clipart, Painting) with 126 classes, and evaluate SFDA methods 7 domain shifts following
15

---

Published as a conference paper at ICLR 2024
AdaContrast (Chen al. 2022 )): and PointDA-10 QQin et al. 2019), 3D point cloud recognition dataset with 3 domains (namely ModelNet-INe 10, ShapeNet-10 10, and ScanNet-100 10), 10 classes, and a total of around 27,700 training and 5,100 test images.
C.3 ONLINE COVARIANCE ESTIMATION
We follow the method for online covariance estimation proposed in ISDA (Wang et al. 2019) as
follows:
(1-1)
(1) + me pe
(1)
no
uii
(25)
ne (f-1)
+ m
[t-1) [t-1) (1-1))111))))) + me x(())
(t- (2) (HO
(t-1)
1(1) )
(1-1)
(()))T
ne
me
+ no
p60
(26)
(1-1)
+ mi.
(t)
(nc (t-1)
mc )2
(1)
[( ne
(1-1) mm
(27)
where u and >)()) represent the mean and covariance matrix, respectively, for the e-th class at time step t. me and E((t) denote the mean and covariance matrix, respectively, of target features which predicted as the e-th class in t-th minibatch. ne is the total number of target features that are (f) is the number of target features that are predicted as the e-th class in all / minibatches, and me predicted as the e-th class in 1-th minibatch.
D ADDITIONAL RELATED WORK
D.1 SPECTRAL CONTRASTIVE Loss
While we were inspired by the spectral contrastive loss (Spectral CL) (HaoChen et al. 2021 to find partitions our augmentation graph, there are distinct differences between Spectral CL. and our SNC
loss.
Spectral CL utilizes augmented data as positive pairs, and data augmentation requires prior knowledge about the dataset. For instance, transforming the color of a lemon to green would turn it into lime, thus changing the class of the data. Such augmentation that fails to preserve class information can introduce bias to the model, potentially bringing harm to the model's performance (Balestriero et al., 2022). Hence, while Spectral CL requires class-preserve preserving augmentations based on prior knowledge, our SNC loss determines positive pairs without needing explicit augmentation and prior knowledge.
D.2 IMPLICIT SEMANTIC DATA AUGMENTATION
The IFA loss is motivated by the ISDA (Wang et al., 2019), and we adopted the online estimation of the class-wise covariance matrix from ISDA. However, there are two notable differences between the IFA loss and ISDA.
Firstly, ISDA is designed for supervised learning, where data has labels and no domain shift. In contrast, IFA loss is computed using target domain data without labels and considers SFDA setting with domain shift. To perform implicit feature augmentation under these conditions, IFA loss utilizes pseudo-labels of target domain data which dynamically change during the adaptation process to estimate the wlssswiise covariance matrix.
Secondly, ISDA derives upper bound for the expectation of the cross entropy loss for supervised learning. On the other hand, IFA loss is designed to approximate the population augmentation graph, which allows SNC loss find improved partition, and it is achieved by deriving an upper bound for the expectation of (the logarithm of) our SNC loss tailored for SFDA.
D.3 OTHER DOMAIN ADAPTATION METHODS
To address the domain shift problem, domain adaptation has been actively studied, and numerous methods have been proposed. Yuu et al. 2023 Zhao al. 2020
16

---

Published as a conference paper at ICLR 2024
NRC (Yang et al. 2021a) and "Connect, not collapse" (CNC) (Shnn et al. 2022)
graph-based
domain adaptation methods and TSA (Li et al. (2221a) utilizes ISDA for domain adaptation. Our paper presents clear methodological differences and performance advantages compared to existing
methods.
NRC (Yang al. 2021a) is graph-based source-free method that encourages prediction consistency among neighbors in the feature space by utilizing neighborhood affinity, which is discussed in Section In contrast, our method constructs the augmentation graph in the feature space based the insights We then propose SNC loss to identify partitions of this augmentation graph. These methodological differences result in significant performance gaps (e.g., the results on VisDA,
2
from Intuitions T and 2
PoimDA-10, and VisDA-RSUT in Tables 4 and 5
CNC (Shen al., 2022) pretrains models using contrastive loss. While CNC depends on data augmentation that needs prior knowledge, our method defines positive pairs without explicit data augmentation. In Table [2 we present comparative experiment with CNC on 12 domain shifts in the DomainNet dataset (40 classes) and ResNet-50 architecture. The results demonstrate that our approach outperforms the direct application of contrastive loss for pretraining the model domain adaptation.
Table 12: Accuracy (%) 12 domain shifts of the DomainNet dataset (40 classes, ResNet-50). For CNC, we brought the performances SwAV+extra, which was the best- bert-perf contrastive learning method.
Method
S -C
>P
S
S-R
C-S
C-
C-R
P-S
P--C
P
R
-8
R-P
Avg.
SwaV-exttr Shen et al. 2022 SF(DA)2
53.5
46.8
58.1
46.2
41.7
59.4
48.7
41.3
69.0
44.6
54.2
$7.3
51.7
63.8
57.2
58.6
60.3
58.9
61.4
58.2
59.5
65.2
65.4
60.4
TSA (Li al. 2021a) employs ISDA for training the source model. Similar ISDA, it derives an upper bound for the expectation of the cross entropy loss and requires labeled (source domain) data. Conversely, IFA loss designed to approximate the population augmentation graph, which allows SNC loss to find an improved partition. IFA loss utilizes pseudo-labels of unlabeled target domain data, which dynamically change during the adaptation process, to estimate the class-wing wise covariance matrix. Using the covariance matrix, IFA loss is derived as closed-fore upper bound for the expectation (the logarithm of) our SNC loss. The difference in method results a notable performance gap (e.g., . the results in Table 13).
Table 13: Additional performance comparison the VisDA dataset.
Method
BSP + TSA
3C-GAN + HCL SF(DA)
Per-class
82.0
84,2
88.1
HCL (Huang et al. 2021) contrasts embeddings from the current model and the historical models. In the paper, 3C- the best-performing method on the VisDA dataset, showing large performance gap compared to our method (e.g., the results in Table 13).
Feat-mixup (Kundu et al. 2022) improves domain adaptation performance via mixup the feature space with augmented samples. Since this method utilizes data augmentation, it requires prior knowledge for class-preserving image augmentation and multiple forward passes for augmented samples. In contrast, our SNC loss, grounded in Intuitions I and in the manuscript, determines positive pairs without explicit augmentation. This difference results in large performance gap (e.g..
the results in Table 3)
Additionally, ProxyMix (Ding et al. 2023 utilizes classifier weights as the class prototypes and proxy features and employs mixup regularization to align the proxy and target domain. PGL (Luo et al. 2023) leverages graph neural networks for open-set domain adaptation.
E CODE AVAILABILITY
Code is available in Supplementary Material,
17

---

Published as a conference paper at ICLR 2024
REFERENCES IN APPENDIX
Randall Balestriero, Leon Bottou, and Yann LeCun. The effects of regularization and data augmenta- tion are class dependent. Advances in Neural Information Processing Systems, 35:37778-3789) 2022.
Dian Chen, Dequan Wang. Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In Proceedings the IEEE/CVE Conference on Computer Vision and Puttern Recognition, pp- 295-305, 2022.
Yuhe Ding, Lijun Sheng, Jian Liang, Aihua Zheng, and Ran He. Proxymix: Proxy-based mixup training with label refinery for source-free domain adaptation. Neural Networks, 167:92-103, 2023.
Jeff Z HaoChen, Colin Wei, Adrien Gaidon, and Tengyu Ma. Provable guarantees for self-super deep learning with spectral contrastive loss. Advances in Neural Information Processing Systems, 34:5000-5011, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Jiaxing Huang. Dayan Guan, Aoran Xiao, and Shijian Lu. Model adaptation: Historical contrastive learning for unsupervised domain adaptation without source data. Advances in Neural Information Processing Systems, 34:3635-3649, 2021.
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pp. 448-456. pmlr, 2015.
Ying Jin, Ximei Wang, Mingsheng Long, and Jianmin Wang. Minimum class confusion for versatile domain adaptation. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXI 16, pp. 464-480. Springer, 2020.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.69800 2014.
Jogendra Nath Kundu, Akshay Kulkarni, Suvaansh Bhambri, Deepesh Mehta, Shreyas Anand Kulkarni, Varun Jampani, and Venkatesh Babu Radhakrishnan. Balancing discriminability and transferability for source-free domain adaptation. In International Conference on Machine Learn- ing. pp. 11710-11723 11728. PMLR, 2022.
Rui Li, Qianfen Jiao, Wenming Cao, Hau-San Wong, and Si Wu. Model adaptation: Unsupervised domain adaptation without source data. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9641-9650. 2020.
Shuang Li, Mixue Xie, Kaixiong Gong, Chi Harold Liu, Yulin Wang, and Wei Li. Transferable semantic augmentation for domain adaptation. In Proceedings of the IEEE/CVF conference on computer vision and pallern recognition, pp. 11516-11522 2021a.
Xinhao Li, Jingjing Li, Lei Zhu, Guoqing Wang, and Zi Huang. Imbalanced source-free domain adaptation. In Proceedings of the 29th ACM International Conference on Multimedia, pp. 3330- 3339, 2021b.
Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning, pp. 6028-6039. PMLR, 2020.
Jian Liang, Dapeng Hu, Yunbo Wang, Ran He, and Jiashi Feng. Source data-absent unsupervised domain adaptation through hypothesis transfer and labeling transfer. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11):86028861 2021.
Mingsheng Long, Zhangjic Cao, Jianmin Wang. and Michael Jordan. Conditional adversarial domain adaptation. Advances in neural information processing systems, 31, 2018
18

---

Published as a conference paper at ICLR 2024
Yadan Luo, Zijian Wang, Zhuoxiao Chen, Zi Huang, and Mahsa Baktashmotlagh. Source-frees progressive graph learning for open-set domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. Visda: 2017.
The visual domain adaptation challenge. arXiv preprini
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 1406-1415, 2019.
Can Qin, Haoxuan You, Lichen Wang, C-C Jay Kuo, and Yun Fu. Pointdan: A multi-scale 3d domain adaption network for point cloud representation. Advances in Neural Information Processing Systems, 32, 2019.
Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains, In Computer Vision-ECCV 2010: 11th European Conference on Computer Vision, Heraklion, Crete, Greece, September 5-11, 2010, Proceedings, Part 11, pp. 213 113-226. Springer, 2010.
Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.
2018.
Tim Salimans and Durk Kingma. Weight normalization: A simple reparameterization accelerate training of deep neural networks. Advances in neural information processing systems, 29, 2016.
Kendrick Shen, Robbie M Jones, Ananya Kumar, Sang Michael Xie, Jeff Z HaoChen, Tengyu Ma, and Percy Liang. Connect, not collapse: Explaining contrastive learning for unsupervised domain adaptation. In International Conference Machine Learning, pp. 19847-19878. PMLR, 2022.
Hui Tang, Ke Chen, and Kui Jia. Unsupervised domain adaptation via structurally regularized deep clustering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8725-8735, 2020.
Yulin Wang, Xuran Pan, Shiji Song, Hong Zhang, Gao Huang, and Cheng Wu. Implicit semantic data augmentation for deep networks. Advances in Neural Information Processing Systems, 32, 2019.
Shiqi Yang, Joost van de Weijer, Luis Herranz, Shangling Jui, et al. Exploiting the intrinsic neigh- borhood structure for source-free domain adaptation. Advances in neural information processing systems, 34:29393-29405, 2021.
Shiqi Yang, Yaxing Wang, Kai Wang, Shangling Jui, al. Attracting and dispersing: A simple approach for source-free domain adaptation. In Advances in Neural Information Processing Systems, 2022.
Zhiqi Yu, Jingjing Li, Zhekai Du, Lei Zhu, and Heng Tao Shen. A comprehensive survey on source-free domain adaptation. arXiv preprint arXiv:2302.1 11003 2023.
Sicheng Zhao, Xiangyu Yue, Shanghang Zhang, Bo Li, Han Zhao, Bichen Wu, Ravi Krishna, Joseph E Gonzalez, Alberto Sangiovanni- Vincentelli, Sanjit Seshia, et al. A review of single-source deep unsupervised visual domain adaptation. IEEE Transactions on Neural Networks and Learning Systems, 33(2):473-4995 2020.
19


Reviews:
[{"review_id": "Py1RHl4LYC", "reviewer": "Reviewer_7eAE", "summary": "This work presents a novel approach to address the source-free domain adaptation task, which aims to adapt the pre-trained model to suit the unlabelled target domain with distribution shifts. The work looks at a novel perspective – from data augmentation in latent feature space instead of applying transformation of raw data to save computational costs while improving model robustness. The SNC loss is proposed to learn discriminative features and form tighter clusters; To augment target features, a class-wise covariance matrix is estimated in an online manner with the help of pseudo labels. IFA loss promotes the decision boundaries for each class to align with the principal direction of the variance of target features. To encourage each direction to be orthogonal, FD loss is further added. Comprehensive experiments are provided on both 2D and 3D datasets and evidence the validity of the proposed approach.\n\n---- Post Rebuttal ----\nThanks for the detailed explanation the authors provided. I would like to maintain the original positive rating for this work.", "questions": "My question mainly lies in (1) the ablation study: why do two regularizations need to be applied simultaneously? (2) the confusion matrix is estimated in an online way. How to guarantee its quality/accuracy?\n\nIf the questions are properly addressed, I will consider raising my rating.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ The paper looks at an interesting way of exploring augmentation in SFDA, where the feature diversity is guaranteed while the resulting computational costs are not significantly increased compared to conventional augmentation methods. The main idea is straightforward and effective, and the authors also bring in-depth discussions and theoretical analysis to provide insights into why the idea works. This makes the proposed well-motivated and inspiring.\n+ The paper presents a large number of experimental comparisons on different benchmark datasets and demonstrates a significant boost over previous methods. The running time analysis gives clear support to what is claimed in the introduction section. \n+ The paper is well structured and notations are used properly.", "weaknesses": "- As shown in Figure 3, the ablation study reveals that solely optimizing SNC performs better than with either FD or IFA loss. It remains unclear why SNC+IFA is inferior as augmented views are provided. More discussions can be shared to better help readers understand the proposed approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "g5Q8EKb3Dt", "reviewer": "Reviewer_44EQ", "summary": "This work proposed a novel perspective of solving source-free domain adaptation (SFDA) problems through implicit feature augmentation on augmentation graphs. Motivated by the two assumptions, the authors naturally constructed an augmentation graph within the feature space. Initially, the augmentation graph is formed based on neighboring features. Subsequently, it is harnessed to identify clusters within the feature space using the SNC loss. With high-quality clusters, target features are implicitly augmented by an EFA loss as an upper bound of the first term of SNC loss instead of directly sampling from Gaussian distribution. Considering similar categories, feature space is further disentangled by maximizing cosine distances, leading to preserved class semantics. Experiments and analysis were conducted to prove the effectiveness of the proposed method.", "questions": "Please see the weaknesses above.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality: This submission presents an innovative perspective on SFDA, introducing implicit augmentation without the need for prior knowledge. This novel approach has been thoughtfully considered and significantly broadens the scope of research in this area.\n\nQuality: The proposed method's quality is supported by a comprehensive body of theoretical and experimental evidence.\n\nClarity: The definitions and mathematical terms are highly connected with the corresponding elements of the proposed method. The presentation and logical flow are well-executed.\n\nSignificance: The application of data augmentation to SFDA from a novel standpoint is of great significance. While the theoretical contributions to the SFDA field may be limited, the method holds practical and theoretical value, as it is built upon sound mathematical foundations and addresses practical issues, such as handling similar categories.", "weaknesses": "A more detailed description of the relationship between the augmentation graph and section 3.4 is warranted, as there appears to be no explicit utilization of G_hat in section 3.4.\n\nHow does equation (4) influence G_hat, and what is the impact of equation (4) and G_hat on the overall optimization process, given that equation (4) only considers cosine-similarity neighborhoods and mini-batch statistics?\n\nWithin spectral clustering methods, could the scenario wherein two groups of features belonging to the same ground truth category are erroneously assigned to separate partitions ultimately lead to a degradation in model performance, as they may be pushed apart within the feature space?\n\nIt would be much appreciated if some DA surveys and reviews are included to better show the background and related work, such as \"A Comprehensive Survey on Source-free Domain Adaptation\", \"A Review of Single-Source Deep Unsupervised Visual Domain Adaptation\".\n\nThe compared baselines are obviously insufficient. For source-free DA, some typical and latest methods are not compared, such as SHOT++. For source-available DA, the compared baselines are too old. Some more baselines published in 2022 and 2023 are required but missing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lni4ttp0G7", "reviewer": "Reviewer_s4wJ", "summary": "This paper proposes a novel approach, namely Source-Free Domain Adaptation through the lens of Data Augmentation (SF(DA)^2), to address the challenge of source-free domain adaptation. The proposed method aims to harness the advantages of data augmentation while mitigating the drawbacks associated with relying on prior knowledge of class-preserving transformations and the increased memory and computational demands. SF(DA)^2 comprises three key components: Spectral Neighborhood Clustering (SNC) loss, Implicit Feature Augmentation (IFA) loss, and Features Disentanglement (FD) loss. The SNC loss is designed to cluster the target data in the prediction space, the IFA loss emulates the effects of augmented features without imposing additional computation and memory overhead, and the FD loss captures distinct class semantic information along diverse directions. Rigorous experimentation on various benchmarks verifies that the proposed method attains state-of-the-art performance in source-free domain adaptation.", "questions": "Please see the weaknesses.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "(1) The writing of this paper is good.\n\n(2) The method proposed is straightforward yet effective, and the results obtained are at the forefront of current research in the field.", "weaknesses": "(1)\tWhat are the differences between the proposed SNC loss and the loss function in the existing AaD [1]? As far as I know, the purpose of the loss in AaD is to encourage similar neighbors to be close to each other and dissimilar neighbors to be far apart within a batch. This is similar to the SNC loss proposed in this paper. The author needs to clarify the differences between these two loss functions.\n\n(2)\tAre the estimated covariance matrices accurate in the IFA and FD losses, given the domain shift between the source and target domains? The estimation of these matrices relies on the output probabilities of the target data. However, this estimation process may introduce biases, especially when there are significant dissimilarities between the classes in the target domain and those in the source domain. As a result, the probabilities associated with these classes in the target domain, as generated by the source domain model, may be greatly reduced. Consequently, there is a potential risk of losing the capability to effectively learn these specific classes during the subsequent learning process, presenting a challenge to rectify the situation.\n\n(3)\tIFA operations have shown promise as a universally applicable data augmentation technique in unsupervised tasks. Can IFA be applied to other existing unsupervised tasks? Can you provide examples to validate its effectiveness?\n\n(4)\tTwo questions regarding the experiment:\n(a) In VisDA, Domain-Net, and PointDA-10 datasets, IFA loss has a weight of 1e-4, while FD loss has a weight of 10, resulting in a limited impact of IFA loss on the training process compared to FD loss. Can the inclusion of the other two datasets, apart from the VisDA-Rust task, demonstrate the significance of this new data augmentation technique? \n(b) Why are there no comparisons of NRC and AaD methods on the DomainNet dataset? Similarly, why are the results of AaD not compared on the PointDA-10 dataset? \n\n(5)\tIn the Introduction, the authors mention that recent studies have used data augmentation techniques to improve adaptation performance. However, there is a lack of more relevant work cited, such as [2] and [3].\n\n[1] Yang, S., Jui, S., & van de Weijer, J. (2022). Attracting and dispersing: A simple approach for source-free domain adaptation. Advances in Neural Information Processing Systems, 35, 5802-5815.\n\n[2] Ding, Y., Sheng, L., Liang, J., Zheng, A., & He, R. (2023). Proxymix: Proxy-based mixup training with label refinery for source-free domain adaptation. Neural Networks, 167, 92-103.\n\n[3] Luo, Y., Wang, Z., Chen, Z., Huang, Z., & Baktashmotlagh, M. (2023). Source-free progressive graph learning for open-set domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tWGW2IEoRr", "reviewer": "Reviewer_jaw9", "summary": "This paper addresses source-free domain adaptation task. The submission investigates the problem from the view of data augmentation, along with neighborhood clustering.", "questions": "- The introductions of IFA and FD are the main contribution of the paper. In my understanding, IFA loss plays similar role as the first term in SNC loss, while FD loss plays the similar role as the second term of SNC loss (here FD loss may be better as it quantifies the similarity degrees between different categories). I am just wondering what if only using FD and IFA losses for adaptation.\n\n\n\n- In Fig. 2, the results show that the proposed method can not improve SHOT, while in AaD it indicates that SHOT is actually conducting similar operation as NRC and AaD. Do authors have clear explanation about the phenomenon, is it due to the pseudo labeling used in SHOT?\n\n- In Fig. 3, with only FD or IFA on SNC will deteriorate the performance. The authors posit some hypothesis for explanation, while I think there may need more results on several other datasets to prove the assumption. By the way, I think there is no information we could get from Fig. 3(b), since it is neither not consistent with the corresponding accuracy, nor cable to guide the hyperparameter tuning.\n\n- There are two extra hyperparameters for FD and IFA losses, and there seems no ablation study for those hyperparameters. In the paper, it mentioned that $\\alpha_1$ and $\\alpha_2$ are set to different values for different dataset. I conjecture they may be sensitive on some datasets and hard to tune in the unsupervised way.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The motivation is clear and sound, by using neighborhood (spectral) clustering to address SFDA, which is already proved by the previous related works.\n\n- Instead of utilizing heavy explicit data augmentation, the submission resorts to implicit augmentation via local neighbors. It decreases the complexity.\n\n- Experimental results on several methods are better compared to existing methods.", "weaknesses": "- The proposal of SNC is not new. SFDA method AaD already introduced quite similar method (in both motivation and final objective form). More specifically, the proposed SNC just takes [1] into the SFDA task.\n\n- Several results for other methods in the main tables are not identical to the results in their original papers. It is not good to report the reproduced results of own without mentioning it in the paper (though there are always software environment difference).\n\n\n\n***Reference***\n\n[1] Provable guarantees for self-supervised deep learning with spectral contrastive loss. NeurIPS 2021", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)