Learning Frequency-Adapted Vision Foundation Model for Domain Generalized Semantic Segmentation
Authors: Qi Bi
Jingjun Yi
Hao Zheng
Haolan Zhan
Yawen Huang
Wei Ji
Yuexiang Li
Yefeng Zheng
Venue: NeurIPS 2024 poster (2024)
Keywords: Semantic Segmentation
Domain Generalization
Vision Foundation Model
Haar Wavelets
Primary area: machine_vision
TL;DR: Learning style-invariant property from vision fundation model from the frequency space by Haar wavelet transform.

Abstract: The emerging vision foundation model (VFM) has inherited the ability to generalize to unseen images.
Nevertheless, the key challenge of domain-generalized semantic segmentation (DGSS) lies in the domain gap attributed to the cross-domain styles, i.e., the variance of urban landscape and environment dependencies.
Hence, maintaining the style-invariant property with varying domain styles becomes the key bottleneck in harnessing VFM for DGSS. 
The frequency space after Haar wavelet transformation provides a feasible way to decouple the style information from the domain-invariant content, since the content and style information are retained in the low- and high- frequency components of the space, respectively. 
To this end, we propose a novel Frequency-Adapted (FADA) learning scheme to advance the frontier.
Its overall idea is to separately tackle the content and style information by frequency tokens throughout the learning process.
Particularly, the proposed FADA consists of two branches, i.e., low- and high- frequency branches. The former one is able to stabilize the scene content, while the latter one learns the scene styles and eliminates its impact to DGSS. 
Experiments conducted on various DGSS settings show the state-of-the-art performance of our FADA and its versatility to a variety of VFMs.
Source code is available at \url{https://github.com/BiQiWHU/FADA}.

Learning Frequency-Adapted Vision Foundation Model for Domain Generalized Semantic Segmentation
Bi Jingjun Yi, Hao Zheng 204 Haolan Zhan Yawen Huang",
Wei Ji Yuexiang Li Yefeng Zheng! IX<
Westlake University, China, JJaviss Research Center, Tencent Youtu Lab, China, *Monash University, Australia, Yale University, United States,
University of Macau, Macau
howzheng@tencent. com, yuexiang- wwwiiiee.ee..... org zhengyefeng@vest..ee edu. cn
Abstract
The emerging vision foundation model (VFM) has inherited the ability to generalize to unseen images. Nevertheless, the key challenge of domain-generaliin semantic segmentation (DGSS) lies the domain gap attributed to the cross-domain styles, e.g., the variance of urban landscape and environment dependencies. Hence, maintaining the style-invariant property with varying domain styles becomes the key bottleneck in harnessing VFM for DGSS. The frequency space after Haar wavelet transform provides feasible way to decouple the style information from the Nomain--nvvariin content, since the content and style information retained the low- and high-frequency components of the space, respectively. To this end, we propose novel Frequency-Adaptee (FADA) learning scheme advance the frontier. Its overall idea is to separately tackle the content and style information by frequency tokens throughout the learning process. Particularly, the proposed FADA consists of two branches, i.e., low- and high- high-frequency branches. The former is able stabilize the scene content, while the latter learns the scene styles and eliminates its impact to DGSS. Experiments conducted on various DGSS settings show the state-of-the-ee performance our FADA and its versatility to a variety of VFMs. Source code is available at https:://// //gi thub. com WHUFAAA
Introduction
Most existing semantic segmentation tasks assume that the training and inference images follow the independent and identical distribution (i.i.d.) 12
74, which is far Domain-generalizn generalized semantic segmentation (DGSS) aims to infer robust pixel-wise from reality. Domain
81
semantic predictions on arbitrary unseen target domains when segmentation model is trained on the source domain (as illustrated in Fig. [b). Compared with general domain generalization tasks, the feature distribution discrepancy between the source domain and unseen target domains in context of DGSS holds some unique factors. Specifically, the cross-domain images in DGSS usually share the same content information (i.e., common semantic categories in driving scenes), while the cross-domain styles (i.e., urban landscape. weather, lighting conditions, etc.) mainly account for the feature distribution difference [65] 49
72
(4))
Existing DGSS methods can be summarized into three categories. The first category intends to
decouple the style information from the scene representation [55] 30 14 56
58, but does not
"Qi Bi is affiliated with University of Amsterdam. This research was conducted with Westlake University and Tencent Youtu Lab.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Mapillary
SYNTHIA
LL
0.4
crOSS domain style vanation
generalization
domains
LH
0.6-
M 6
HL
M
Similar Content
V Varied Styles
0.4
pedestrian
trees
night
dawaa
miny
sunny
HH
0.6
M
(a) Domain Generalized Semantic Sogmentation
(b) Domain Distribution after Haar Transformation
Figure I: (a) The key challenge of domain generalized semantic segmentation (DGSS) lies in the stability of the scene content, while the domain gap is caused by the style variation. (b) Analysis of frozen VFM features after Haar wavelet transform. We compute the correlation coefficient between the CityScapes source domain (C) and BDD, Mapillary, SYHTHIA, GTA5 unseen target domains (B, M, G, S). The low-frequency component exhibits higher correlation and smaller domain gap. In contrast, the high-frequency components exhibit a lower correlation and larger domain gap.
ensure strong content representation ability. The second category, on the contrary, directly focuses on the content representation ability regardless of the cross-domain style variations [19] 4.77 7 Such methods were recently developed much owing to the stronger pixel- mixel-wise representation ability of mask attention 12 The third category focuses on enriching the styles as much as possible during However, the content representation ability rarely taken into consideration.
training 82
The emerging vision foundation model (VFM) 62
20 37], with strong generalization ability inherited from large quantity pre-trained images. provides a new possible paradigm for DGSS. Under the realm of parameter-efficien fine-tuning, the hypothesis, i.e., a foundation model relies on the low intrinsic dimension 42
adapt to the downstream tasks, exemplified by low-rank adaptation (LoRA) (28). has shown great success, where the key idea is to inject trainable rank decomposition matrices into each layer. More recently. the low-rank adaptation paradigm has demonstrated to be feasible to fine-tune the VFM for DGSS [69]. Unfortunately. the style-invariant properties of VFM, which is fundamental problem for the extraction of domain generalized semantics, remain unexplored.
An ideal style invariant low-rank adaptation is supposed discern the subtle low intrinsic dimension while does not pose shift on the fragile and frozen VFM features. The prior DGSS methods learned style invariance from the fully trained image features (e.g., instance normalization [55] 30 and instance whitening [14] (66, 7 (88),, which are more likely to collapse and less feasible. Therefore. we adapt the low-rank adaptation to the frequency space, where the style and content have been
separated to figh-frequency and low-frequency components 229 46
40
77 67], respectively.
In this paper, present novel Prcquency-Adaplee learning scheme, dubbed FADA, to push this frontier. Its conceptual idea is to adapt the style and content representation from the VFM features separately in the frequency space. After transforming the frozen VFM features the frequency space by the Haar wavelet transform (60], the low-frequency branch exploits the scene content from frozen VFM features by the learnable low-frequency tokens. contrast, in the high-frequenee branch, the high-free high-ffequency token features are implemented using the instance normalization operation, so that the representation becomes invariant the scene style.
Notably, the proposed FADA introduces two new research lines. Firstly, the possibility of learning low- rank adaptation in the frequency space is explored, which can further benefit other VFM downstream tasks strongly related to the style and content representation. Secondly, it demonstrates the potential of harnessing the Haar wavelet transform for DGSS, which also inspire advancements in general visual domain generalization.
Concretely, contributions can be summarized as follows:

---

We propose Frequency- Adapted learning scheme, dubbed FADA, to fine-tune VFMs for domain generalieeee semantic segmentation.
The proposed FADA, aided by the Haar wavelet guidance to mine the style-invariant property of VFM, is versatile to a variety of VFMs.
Experimentally, the proposed FADA significantly outperforms the state-of-the-arl DGSS methods, and yields an improvement up 2.9% mIoU over the contemporary REIN [69
2 Related Work
Domain Generalization handles the challenging setting where the feature distribution of an arbitrary target domain is not identical that of the source domain. has been extensively studied the past few years. Multiple techniques. to namely a few, optimal transport 22 79), batch normalization [66],
causal inference [88 47 25], discrepancy regularization [80, 68 5] and uncertainty modeling
[59 70), have been proposed. Furthermore, domain generalization via unsupervised learning [26] 27
or from a single source domain [6]] 59
84
75
has also been recently studied.
Domain Generalization by Frequency Decoupling has drawn increasing attention. Its general idea rests in that the style and content have been demonstrated on high-frequenc and wwwfrreuency components [29] 46 [40] (77 [67 [7, respectively. Most of these methods implement Fast Fourier Transform (FFT) to transfer the image the frequency space, and then represent the style and content by the amplitude (high-frequency) and phase (lw-frequency) components, respectively. However, to the best of our knowledge, 1) leveraging Haar wavelet for domain generalization; and 2) enhancing the generalization ability of VFM features via frequency space have been rarely explored. Compared with other frequency analysis methods such as FFT, the orthogonal property of Haar wavelet basis leads to a stronger decorrelation (50). In the context of domain generalization, it indicates better separation between low- and high-frequency components and deserves exploration.
Domain Generalized Semantic Segmentation (DGSS) in the CNN era either decouple the style information (55] 30 56
[4 58
78 or enrich the style diversity [1 82
83
45 52).
With the
rapid development of Vision Transformer (ViT), recent DGSS methods usually leverage the masked Later, the masked
attention mechanism
12
to enhance the content representation [9
attention is used to decode the frozen contrastive image-text pre-trained features 33 and REIN [69]
fine-tunes the VFM under the low-rank adaptation paradigm. However, the style invariant properties of VFM, which are the key of the DGSS representation, remain unexplored.
3 Preliminary
3.1 Low-Rank Adapted VFM
To fine-tune a VFM with parameter efficiency, the low intrinsic dimension 442 assumes that VFM relies on the intrinsic low-dimension in the frozen VFM features to adapt to the downstream tasks. Inspired by this, the low-rank adaptation (LoRA) paradigm 28 is devised to inject trainable rank decomposition matrices into each layer. Given VFM with N sequential layers (denoted as L1, L2. ...,............................ LN), each layer corresponds a pre-trained weight matrix of W1, W2...., ... WN, i.e. W: E Rxx The frozen features from layer L; are denoted as Ji E Rexn, Particularly, for the first layer L,, fi is generated by fi Wi.. Here x denotes the image embedding, n denotes the patch number, and e denotes the channel size.
Denoting the learnable weight matrix as AW, E Rexe and the input of layer L, as fú. the feature propagation from the layer L; to Li-1 can be formulated as fi+1 = W.f. + AW, Then, we assume that the learnable weight matrix AW; can be formulated as low-rank decomposition, i.e.. AW,- B.A, where B E Rex and B E Rexe (r & c). More recently, REIN [69] specifies this paradigm in DGSS by transferring the learnable matrix term AW, into a learnable token T, followed by MLP (.()), denoted as fi+1 W.f. + (T. ff)), where T, E mme and m is the token length. This modification allows significance reduction of the token length (from a thousand magnitude to hundred or even ten magnitude m)) which can alleviate Curse of Dimensionality and allow each token to be better connected to the instances in an image 69]. Our low-rank adaptation implemented on Tú, i.e., T, A, where A, E gmxr and B, E grxe xe (r < min (m, c)).
3

---

Vision Foundation Model
Segmentation Decoder
S
Source Domain
Predicton
DLLL THEE
Instance Norm
FIL fi 1
inverso Witraeet Trarsforn
1 12 1
MLP
MLP
Wirreee Traeeforr
Hight High-rruurryy takon ??
1 f AM
Jowe Erequezcy taken T
MLP
MLP
S
Tameet Domain
training
Inference
point- wise product
adding
Prediction
Figure 2: Overview of the proposed ceqquency-adapter Vision Foundation Model (FADA) learning scheme. It innovatively incorporates the low-rank adaptation of VFM models the frequency space, where the low-thigh-frequency component contains more content/style, respectively. It consists of three key steps, namely, low-lhigh-frequencyy decomposition (in Sec. 4.1 low-frequency adaptation (in Sec. 4.2) and high-frequency adaptation (in Sec. 4.3)
3.2 Haar Wavelet Transform
The Haar transform 600 cross-multiplies a function with various shifts and stretches, which has been demonstrated to be effective in applications, such as signal and image processing.
Definition 1. Haar Scaling Function.
Given input signal the Haar scaling function is
mathematically defined as
f 0
051<1
(tt)
(I)
otherwise
Let Vo denote the space of all functions of the form Σtez ago(x k), where h an arbitrary integer, and a E R. As each element of Vo is zero outside bounded set, such function a 0z k) has finite or compact support.
Definition 2. Basis of the Step Function Space. Given arbitrary nonnegative integer j Z
Let Vj denote the step function space at the level j, which is spanned by the set
(.256222 1), 0(2) r)., 6(2) 1),
(2)
Definition 3. Haar Wavelet Function. The Haar wavelet is the function vb(x) ((2x) ) 6(2x: 1).
For more details of the properties of the Haar transform, please refer the supplementary material.
Methodology
Fig. gives overview of the proposed FADA. After each frozen VFM layer, it consists of three key steps, namely, low-/tigh-frequency decomposition (in (.1I low-frequency adaptation (in Sec. 4.2). and high-frequency adaptation (in Sec. 4.3p. Finally, the frequency components fused together and transferred from the frequency space back to the spatial space, and then fed to the next
VFM layer.
4.1 Low-/High-Frequene Decomposition
Orthogonal property (in Sec. 3.2 leads to the result of a strong decorrelation [50] This property after the Haar wavelet function allows better separation between low- and high-frequency components of input signal than other frequency analysis methods such Fourier transform. In the context of DGSS, the domain gap caused by the cross-domain style variation, while the cross-domain content is stable [4. 78 71 58 As has been well documented that the style and content are predominate

---

in the -ehhffrquency and low-free uwwfrequency components, respectively (29 46 40 77 67 feasible
way explore the style-invariant properties mitigate the variation of in the VFM features. Therefore, the first step is to decompose the frozen VFM feature f; into the low- and high-frequenc components, respectively.
components
In our work, we exploit the Haar wavelet transform to decouple the low- and high-frequeney compo- nents, where four kernels, namely, LL"" LHT", HHL.. HH are given by
L
1 [1
11,
- 1/.
(3)
v2
As discussed in prior works 600 3. the LL" kernel captures the average of the pixel responses, which more robust to the scene content and therefore preserves more low-frequency components. In contrast, by taking the differences between adjacent pixels into account, the L.H., H and HHT kernels tend to preserve the details from the horizontal, vertical and diagonal directions, respectively. These details are more related to the structures, edges, etc, which attribute to the style.
For layer Li, we implement the Haar wavelet transform on the frozen VFM feature Wifi by the above four kernels LL LII", H L and H H respectively. The f! component filtered by LL" captures more scene content, and in contrast the f. J. and f. components filtered by LHT, HL and HH² capture more style information. This decomposition is computed as f/ = (Wifi) & L.I." f = @Wif)) LHT jHh = (Waf)) @ HT"", HH = (W.f) HH" (4)
4.2 Low-Frequency Adaptation
In the context of DGSS, a stable scene content representation despite the style variance is important to predict the scene semanties. For each layer Li, the scene content from the frozen VFM features rests more in the low-frequency component J.L2 (in Eq. 4). which is learned in the low-frequency adaptation branch.
Assume we have a cow-freqenncy token T. E Rmxe where m is the sequence length of T. and c is the dimension of the frozen VFM feature defined in Sec. Sec. The low-frequency token T/E used to exploit the scene content from the low-frequency component f.L., while the same time following the low-rank adaptation paradigm (in Sec. 3.1
First, compute similarity map S, E R between the token T. and low- wwwffreuueccy component J!L from the frozen VFM feature, which measures the correlation between each element in T. and each patch embedding represented in f. given by
s = Softmax 1LL x T.LT
)
(5)
Vc
where Softmax denotes the softmax activation function.
Then, we project the token feature T. into the feature space of fLL by multilayer perceptron (MLP) parameterized by weight parameters W and bias parameters 67. followed by the point-wise product with the similarity map S.. The product with St allows the token features T. to better align to J!L,, where the scene content is highlighted. Assume the output is denoted as Jec Briefly, this process can be mathematically expressed as
- x (T. x W + +
(6)
Then, we fuse the projected token features J.L with the low-frequency features J! by another MLP parameterized by weight parameters W? and bias parameters b?, followed by the skip connection. Assuming the output is JLL this process can be mathematically computed
Tt- + J + fLL) W?? +
4.3 High-Frequency Adaptation
For DGSS, the robustness to the cross-domain style variance is particularly important. Such style difference is usually reflected on the high-frregh frequency components. The Haar wavelet transform
5

---

enables the separation of these high-freet frequeeyyency components ft fi and f:" (in Eq. 4 Directly eliminating all the high-frequency components seems to be a simple and straight-forward solution, but it also leads to the loss of other information such as structure and object boundary. It may degrade a scene representation and decline the segmentation performance. Therefore, the objective of the high-frequency adaptation branch is to mitigate the impact of cross cross-domain style variation, not directly removing all the high-frequency components.
As the decoupling of styles does not differentiate whether the high-frequency components are from the horizontal, vertical or diagonal directions, for simplicity, we concatenate them together for processing in this branch. Specifically, still in layer L;, assume we have high-frequency token T:" IR xmx The token size of T:" is tripled compared with the token size of T.", as three high- frequency components are involved. Similar to the low- low-frequency branch, the high-frequency token TH is used exploit the style information from the nigh-frequene components LLH H and JHH while at the same time following the low-rank adaptation paradigm (in Sec. 3.1
Same as the low-frequency adaptation branch, we compute similarity map S." R between the token 7." and high-frequenn components f! f! and f.llH given by
FHL pHH . 5
S x T:!
S Softmax
where [] denotes the concatenation operation.
Then, the highlighted positions in S reveal the predominant style responses from the source domain images. The high responses, which reflect more domain-specific styles, are supposed to be suppressed during training. It allows the fine-tuned VFM features to less impacted by the domain-specine styles. Instance normalization and standard deviation, effective to eliminate the styles. To this end, an instance normalization implemented on the feature-token similarity map S. given by
[55 31, which computes the channel-wise mean
SH
3m
3m
SH
11
SH
Σ S 24 1=1
11
3m
3m
1=1
Then, we project the token feature T. into the niihh-frequency feature space by a multilayer perceptron (MLP) parameterized by weight parameters W and bias parameters bü, followed by the point-wise product with the similarity map S.". The product with S. allows the token features T. to better align to the decoupled high-frequency features, which is less relevant to the source domain. Assume the output is denoted as J Briefly, this process can be mathematically expressed
T = 5 [T W + b,]-
(10)
Afterwards, we fuse the projected token features " with the high-frequency features f. by another MLP parameterized by weight parameters Wt and bias parameters bξ, followed by the skip connec- tion. Assuming the outputs of these three components are J.B.B. JHL and f.H... this process can be mathematically computed as
|LL 7
= ! JHL J S + + S! SH x W + +b.
(1))
Finally, the low- low-frequency component F.- and high- figh-frequency components HHH. J.H that have been processed by both branches are fused and transferred back by the inverse Haar wavelet transform. The output, denoted as fi+1. is the input of the next frozen layer Li+1.
4.4 Implementation Details
Same as the REIN (69 baseline, the loss function of FADA directly inherits the losses from the Former decoder [2]. given by
L Aoe Adice + AoteColer
(12)

---

where Lee, Ldice and Cete denote the cross-entropy loss, dice loss and classification loss. Here the prper-parameter X Adion and Lrts are 5.0, 5.0 and 2.0, respectively.
By default we use DINO-V2 53 as the frozen VFM, but the proposed FADA is also feasible to other VFMs. For fair evaluation, the Mask2Former segmentation decoder [ is used to generate the pixel-wise prediction as REIN does. Same as the existing paradigm [69], the images are re-sized 512 512 pixels before input the models. The Adam optimizer with an initial learning rate of 1 x 10 is used to train the model. The training process terminates after 20 epochs.
5
Experiments
5.1 Datasets & Evaluation Protocols
Five driving-scene semantic segmentation datasets that share 19 common scene categories are used for validation. Specifically, CityScapes (C) [66] consists of 2,975 and 500 images for training and validation, respectively. The images are captured under the clear conditions in tens of Germany cities. BDD-100K (B) [76] has 7,000 and 1,000 images for training and validation, respectively. The images are captured under diverse conditions from variety of global cities. Mapillary (M) [5T] is another large- scale semantic segmentation dataset, which consists of 25,000 images from diverse conditions. SYNTHIA (S) 64 a synthetic driving-scene segmentation dataset, which has 9,400 images. GTAS (G) 63) is another synthetic dataset, which has 24,966 simulated images from the American street landscape.
Following the evaluation protocol of existing DGSS methods [55] 56 14 38 a certain dataset is used as the source domain for training and the rest four are used as unseen target domains for validation. evaluation settings are: 1) G + C. B. M. S; 2) S - C, B, M, G; and 3) C - B, M. G, S. The evaluation metric is mean Intersection of Union (mloU, in percentage %). All of our experiments are implemented and averaged by three independent repetitions, starting from different random seeds.
Three commonly-
5.2 Comparison with State-of-the-art DGSS Methods
Existing DGSS methods are involved for comparison: 1) ResNet based methods, namely, IBN [5),
IW (56), Iternorm (3I. DRPC (78). ISW [15), GTR (57). DIRL 77T. SHADE [82, SAW [58],
WildNet [4), AdvStyle (83) and SPC 321 2) Mask2Former based methods, namely, HGFormer
19 and CMFormer [8 3) VFM based methods, namely, DIDEX 52 and REIN 69). By default,
the performance is directly cited from prior works (55] 56 14 58 denotes that the authors did
not reported the results nor provided source code. *** denotes re- implementation with official source code under all default settings.
GTA5 Source Domain. From left to right, the third column of Table L reports the performance. Compared with the VFM based REIN [69], the proposed FADA shows an mloU improvement of 1.83%, 1,54%, 1.99% and 1.50% on the C, B, M and S target domains, respectively. In addition, the proposed FADA shows average mloU improvement of 20% and 10% when compared with ResNet and Mask2Former based DGSS methods, respectively.
SYNTHIA Source Domain. The fourth column of Table shows that the proposed FADA achieves the offte-of-the-ar performance, outperforming the REIN by 1.45%, 1.41%. %, 1.22% and 1. 1.29% mloU on the C, B, M and G unseen target domains, respectively. In addition, the proposed FADA shows an average mloU improvement of 15% and 6% over ResNet and Mask2Former based DGSS methods.
CityScapes Source Domain. The last column of Table I shows that the proposed FADA shows an mloU improvement of 1.58%, 1.83%, 1.37% and 1.19% the B, M, G and S unseen target domains, respectively. In addition, the proposed FADA shows an average mIoU improvement of 15% and 5% over existing ResNet and Mask2Former based methods, respectively.
5.3 Ablation Studies
On Each Haar Component. Five settings are involved for experiments: (1) No wavelet components are used. The model fine-tunes on the frozen VFM, which is a simplified version of REIN [69] removing the instance link module; (2) Only fine-tuning on the low-frequency component J.L. and
7

---

Table 1: Performance comparison between the proposed FADA and existing DGSS methods. C: CityScapes [6];; B: BDD-100K [76]; M: Mapillary [3:; S: SYNTHIA 644; : G: GTA5 63 '-':
results were not reported and official source code is not available; :w: only reported one decimal official results; 't": re-implementation with official source code under all default settings, Evaluation
metric is mloU in %. Top three results are highlighted as best second and third, respectively.
Method
Proc: Year
Trained GTAS
(S) M
Trained Ciyssaaes (C)
3 M
TS
Resherr buved: IBN 55
ECCV2018
33.85
37.75
30.57
32.16
38.56
57.04
IW
CVPR2019
29.91
27.48
29.71
27.61
28,
27.12
26,31
26,51
48.45
55.
44.87
26. 10
hemorr
S1
CVPR2019
31.81
32.70
35.88
27.07
49.23
56.26
45,73
25.98
DRPC
ICCV2019
34.12
31.53
32.74
56.34
45.62
ISW
CVPR2021
36.58
35.20
40.33
28.30
35.8 83
31.62
30.84
27.68
50.73
58.64
45.00
26,20
GTR
TIP2021
37.53
33.75
34.52
28.17
36.84
32.02
32.89
28.02
50.75
57.16
45.79
26.47
DIRL
AAA12022
41.04
39.15
41.60
51.
46.52
26.50
SHADE H2
ECCV2022
44.65
39.28
43.34
50.95
60.67
48.61
27.62
SAW
CVPR2022
39.75
37.34
41,86
30.79
38.92
35.24
34.52
29.16
52.95
59.81
47.28
28.32
WikdNet
41
CVPR2022
44.62
38.42
46.09
31,34
50.94
58.79
47.01
27.95
AdvStyle
83
NcurIPS2022
39.62
35.54
37.00
37.59
27.45
31.76
SPC
CVPR2023
44,10
40.46
45.51
BlindNet
CVPR2002
45.72
47.08
31.39
51.84
60.18
47.97
28.51
Former
#GFomme" 19
CVPR2023
53.4
66.9
51.3
33.6
CMFormer
AAA12024
55.31
49.91
60.09
43,80
44.59
33.44
43:25
40.65
59.27
71.10
58.11
40.43
VFM bused.
DIDEX 52
WACV2024
62.0
63.0
REIN 69
CVPR2024
66.4
60.4
66.1
48. 86
49.591
44.42|
48.64
46.97
63.541
74.031
62.411
48.561
FADA (Ours)
68.23
61.94
68.09
50.36
50.04
45.83
49.86
48.26
65.12
75.86
63.78
49.75
11.83
11.54
11.99
1,50
11.45
11.41
11.22
11.29
11.58
11.83
11.37
11.19
Table 2: Ablation studies on each component of the proposed FADA. LL, LH, H and H denote the J.LL J J.LL and J!l components, respectively. v refers to that fine- tuning is implemented. Evaluation metric is mloU in %.
Table 3: Ablation studies of the rank r on generalization performance. Evalua- tion metric mloU in %.
Trained on Cityscapes (C)
Method
+
B
+ M
+ GG
+ S
Treqeecy Componenis
CitySrapes
on SYNTHIA
4
64.21
74.96
62.79
48.68
-?
XI
x
62.45
73.05
61.29
47.61
48.03
43:27
47.85
46.02
8
64.73
75.18
63.06
49.03
x
X
63.85
74.16
62.04
48.68
48.79
44.81
45.96
47.35
16
65.12
75.86
63.78
49.75
X
X
64.04
64.69
74.89
75.16
62.95
63:20
49.15
48.92
49.62
49.18
45.07
45.37
49.50
19:13
46.16
48.07
32
65,28
75.34
63.56
49.42
65.12
75.86
63.78
49.75
50.04
45.83
49.86
48.26
64
64.85
75.12
62.38
49.64
Table 4: Generalization ability test of the proposed FADA on different VFM models. One decimal result is reported and compared following prior references.
Table 5: Generalization performance compari- son on the four adverse condition domains from ACDC dataset (65). CityScapes the source domain. Top three results are highlighted as
Backbons
Fine tune
Method
Trainable
Params*
Ciys
BDD
mloU
Map
Avg.
Full
304.15M
51.3
47.6
54.3
51.1
best second and third respectively.
CLIP
Freeze
REIN (69
2.99M
57.1
54.7
48.7
60.5
57.4
52.4
Method
Fog Nigh:
Traind on C Cityscapox (C.)
-Rain
-SSow
FADA
11.65M
58.7
55.8
62.1
58.9
KeaNer Beved
Full
632.18M
57.6
51.7
61.5
56.9
E5
63.8
21.2
5014
43.7
SAM 39
Preeze
0.00M
57.0
47.1
58.4
54.2
30
23.8
51..
FADA
16.59M
61.0
39.6
52.0
53.2
63.4
60.0
$7.9
Trangfomerr Baned:
64.3
56.0
49.8
48.1
Full
304.24M
62.1
56.2
60.9
ISSA
33
67.5
33.2
55.9
53.2
52.5
EVA02 30
REIN 6
Freeze
2.99M
0.00M
65.3
56.5
60.5
53.6
64.9
58.6
63.6
56.2
Mask2Famms
HCiFomer [
[
69.9
73.4
52.7
37.1
63.6
72.0
62.5
68.5
67.2
58,0
FADA
11.65M
61.9
CMfFormer S
77.8
33.7
67.6
64.3
60.9
Fall
304,20M
63.7
37.4
64.2
61.7
Wild based:
DINOV2
Freeze
REIN (0
0.00M
2.99M
63.3
66.4
60.4
56.1
63.9
66.1
64.3
61.1
REIN' (3)
Ours
80.2
79.5
55.9
57.4
75.0
73.5
71.5
FADA
11.65M
68.2
62.0
68.1
66.1
10.7
215
12.
12.9
11.9
directly fusing with three high-frequency components without fine-tuning; (3) fine-tuning on f&L and f. (4) fine-tuning on f. and f: (5) fine-tuning on all the four frequency components (Ours). Table reports the results. Harnessing the low-frequency features leads to an average of 1% mIoU improvement component leads to a slight improvement on unseen target domains. It demonstrates the necessity to decouple the impact of cross-domane styles.
most experimental settings. In addition, fine-tuning on each high- high-frequency
On Low-Rank Dimension r. By default we set the low-rank dimension r as 16. We further test the generalization performance when is 4, 8, 32 and 64, respectively. Table rreporsss the performance when SityScapes is used as the source domain. When r 16 32, the performance on unseen target domains shows the most stable performance. However, when r is too small (e.g., 4 or 8) or too large
8

---

Basdlnn
Outs
Beeine
Figure 3: Channel-wist correlation matrix of the features from last VFM layer between source do- main (C) and unseen domain (B). The brighter cell is, the higher response.
BOOKOCK
Mapilay
GTAS
SYNTHIA
Figure 4: t-SNE visualization. Feature embed- ding is extracted from the last VFM layer. Left: baseline; Right: ours.
(e.g., 64), the performance on unseen target domains demonstrates clear decline, which can be explained by the under-fitting and over- ever-fitting, respectively.
Understanding from Cross-Domain Feature Correlation. We display the channel-wise correlation matrix of the last-layer feature embeddings from C source domain and target domain. The results
displayed in Fig. 3 Brighter indicates higher response. FADA allows both low- and figh-frequency token features from the source domain and unseen target domains to show similar channel-wise activation response, which allows the model to be better generalized to unseen target domains.
T-SNE Visualization. We display the features before the segmentation decoder by t-SNE visualiza- tion. The experiments are conducted under the C B, M. G. S setting. The feature space of the original REIN and the proposed FADA is visualized in Fig. 4 The samples from different unseen target domains are more uniformly distributed by the proposed FADA, narrowing the domain gap.
Understanding the Benefit of Instance Normalization to Mitigate Domain-specific Information. HH
We extract the three high-frequency components from the last VFM layer, and display them by I-SNE visualization. The feature space without (denoted as w.o.) and with (denoted as with) implementing the instance normalization fune- tion is visualized in the first and second row of Fig. respectively. The experiments are con- ducted under the C + B, M, G, S setting. It observed that the implementation normalization allows the samples from different unseen target domains to be more uniformly dis- tributed, indicating its effectiveness mitigate the domain-specific information containing in the high-freyuency components.
HI.
instance
Figure 5: Impact of instance normalization on the domain generalization property of gigh-rreueecy components. w.o./with: without/with implement- ing instance normalization.
5.4 Generalization Other Settings
To Different VFMs. We test the translation ability of the proposed FADA to other VFMs, namely, CLIP [62). SAM 33 and EVA02 (20). For comprehensive evaluation, each VFM is validated under full-training, fine-tuning (REIN), or frozen scheme [69], respectively. The reported one decimal results are directly cited from [69]. Experiments are conducted under the G - (C. B, M) setting, Table 4 shows the superiority of FADA when embedded into these VFMs. For comparison with parameter-efficient fine-tuning (PEFT) methods, please refer to Table 7 in the supplementary material.
To Adverse Domains Adverse Conditions Dataset with Correspondence (ACDC) 655 is semantic segmentation dataset that consists of samples from four types of adverse conditions, namely, rain, fog, night and snow. Table 5 shows that the proposed FADA outperforms existing DGSS methods by up to 0.7%, 1.5%, 2.5% and 2.9% the fog, night, rain and snow domains. respectively.

---

Unseent Images
ISW
SAW
WildNet
SPC
REIN
Ours
Figure 6: Exemplar segmentation results of existing DGSS methods (ISW 14], SAW (58). WildNet
41. SPC 332. CMFormer [ and REIN [69]) and FADA under the C (B, G, M. S) setting.
Unseen images
ISW
SAW
WildNet
SPC
REIN
Ows
Figure 7: Exemplar segmentation results of existing DGSS methods (ISW [44, SAW (58) WildNet
41.
SPC 322
CMFormer g and REIN 669 and FADA under the C four ACDC setting.
5.5
Quantitative Segmentation Results
Some exemplar segmentation results are compared under the C - B, M, G, S and C - adverse domains are provided in Fig. 66 and Fig. 7 respectively. The proposed FADA shows better pixel-wise prediction than not only ResNet based methods (i.e., ISW [14], SAW (58), WildNet [41), and SPC and Mask2Former based methods (i.e., CMFormer 8), but also VFM based REIN 69
32
Conclusion
In this paper, we focused on adapting VFM for DGSS by exploiting the style-invariant properties from the VFMs, and presented novel Frequency-A -ADpudd learning scheme to push this frontier. Concisely, Haar wavelet transform was introduced to decouple the frozen VFM features into low- and high-frequency components, which contain more scene content and style information, respectively. We innovatively modified the low-rank adaptation paradigm to both frequency features, and alleviated the impact of cross-domain variation high-frequency features. Consequently, the model achieved a better generalization on unseen target domains. Extensive experiments and ablation studies on variety of settings showed the effectiveness of the proposed FADA.
Limitation Discussion & Broader Societal Impact. The proposed FADA handles the low- and high- frequency features separately, which increases the trainable parameters compared with prior work (Table However, the increase of about 6M parameters is acceptable. The proposed FADA advances the reliability and safety of autonomous driving and alleviates human involvement, benefiting the human being. We do not envision any negative social impact.
Acknowledgments and Disclosure of Funding, This work was supported by the Science and Technology Major Project of Guangxi (AA22096030 and A AA22096032), and National Key R&D Program of China under Grant (20000000009000 and 2020AAA01095011,
10

---

References
[1] Armen Aghajanyan, Sonal Gupta, and Luke Zettlemoyer. Intrinsic dimensionality explains the effective- ness of language model fine-tuning. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, pages 7319-7328, 2021.
[2] Woo-Jin Ahn, Geun-Yeong Yang. Hyun-Duck Choi, and Myo-Taeg Lim. Style hlind domain general- ized semantic segmentation via covariance alignment and semantic consistence contrastive learning. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition. 2024.
[3] Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. Surf: Speeded up robust features. In Computer Vision- ECCV 2006: 9th European Conference on Computer Vision, Graz, Austria, May 7-13, 2006. Proceedings, Part I 9, pages 404 417, 2006.
[4] Qi Bi, Shaodi You, and Theo Gevers. Interactive learning of intrinsic and extrinsic propertics for all-day semantic segmentation. IEEE Transactions on Image Processing, 32:3821-3888 2023.
[5] Qi Bi, Jingjun Yi, Hao 7heng, Wei Ji, Yawen Huang, Yuexiang Li, and Yefeng 7heng. Learning generalized medical image segmentation from decoupled feature queries. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 810-818. 2024.
[6] Qi Bi, Shaodi You, and Theo Gevers. Generalized foggy-scene semantic segmentation by frequency decoupling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1389-1391113 1399, 2024.
[7] Qi Bi, Shaodi You, and Thco Gevers. Learning generalized segmentation for foggy-scenes by bi-directional wavelet guidance. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 801-809, 2024.
[8] Qi Bi, Shaodi You, and Theo Gevers. Learning content-enhanced mask transformer for domain generalized urban-scene segmentation. In Proceedings the AAAI Conference on Artificial Intelligence, volume 38, pages 819-827, 2024.
[9] Prithvijit Chattopadhyay, Kartik Sarangmath, Vivek Vijaykumar, and Judy Hoffiman. Pasta: Proportional amplitude spectrum training augmentation for syn-to-real domain generalization. Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 19288-19300, 2023.
[10] Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, and Ping Luo. Adapt- former: Adapting vision transformers for scalable visual recognition. Advances Neural Information Processing Systems, 35:1 16664-16678, 2022.
[11] Wei-Ting Chen. Kai-Kai Huang, Cheng-Clas -he Tsai, Hao-Hsiang Yang, Jian-Jiun Ding, and Sy-Yen Yen Kuo. Learning multiple adverse weather removal via two-stage knowledge learning and multi-contrastive regularization: Toward unified model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2022.
[12] Bowen Cheng, Alex Schwing, and Alexander Kirillov. Per-pixel classification is not all you need for semantic segmentation. Advances in Neural Information Processing Systems, 7817777847775 2021.
[13] Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. Masked- attention mask transformer for universal image segmentation. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 1290-1299, 2022.
[14] S. Choi, S. Jung, H. Yun, J. Kim, S. Kim, and J. Choo. Robustnet: Improving domain generalization in urban-scene segmentation via instance selective whitening. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11580 11590, 2021.
[15] Sungha Choi, Sanghun Jung, Huiwon Yun, Joanne T Kim, Scungryong Kim, and Jacgul Choo. Robustnet: Improving domain gencralization urban-scene segmentation via instance selective whitening. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11580- 11590, 2021.
[16] Marius Cordis, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference computer vision and puttern recognition, pages 3213-3223, 2016.
[17] Rui Dai, Yonggang Zhang, Zhen Fang, Bo Han, and Xinmei Tian. Moderately distributional exploration for domain generalization. In International Conference on Machine Learning, pages 6786-688 6817, 2023.
11

---

[18] Carlos A Diaz-Ruiz, Youya Xia, Yurong You, Jose Nino, Junan Chen, Josephine Monica, Xiangyu Chen, Katie Luo, Yan Wang, Mare Emond, et al. Ithaca365: Dataset and driving perception under repeated and challenging weather conditions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1333221392, 2022.
[19] Jian Ding, Nan Xue, Gui-Sonr Gui-Song Xia, Bernt Schiele, and Dengxin Dai. Hgformer: Hierarchical grouping transformer for domain generalized semantic segmentation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 1413-11423, 2023.
[20] Yuxin Fang, Wen Wang. Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang. and Yue Cao. Eva: Exploring the limits masked visual representation learning at scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19358-19369, 2023.
[21] Ziqi Gao, Qichao Wang. Aochuan Chen, Zijing Liu, Bingzhe Wu, Liang Chen, and Jia Li. Parameter- efficient fine-tuning with discrete fourier transform. In International Conference on Machine Learning, 2024.
[22] Milena Gazdieva, Alexander Korotin, Daniil Selikhanovych, and Evgeny Burnaev. Extremal domain translation with neural optimal transport. Advances in Neural Information Processing Systems, 36, 2023,
[23] Ziyang Gong, Fuhao Li, Yupeng Deng. Wenjun Shen, Xianzheng Ma. Zhenming and Nan Xia. Train one, generalize to all: Generalizable semantic segmentation from single-scene to all adverse scenes. In Proceedings of the st ACM International Conference on Multimedia, pages 2275-2284, 2023.
[24] Ziyang Gong, Fuhao Li, Yupeng Deng, Deblina Bhattacharjee, Xiangwei Zhu, and Zhenming Ji. Coda: Instructive chain-of-domail adaptation with severity-aware visual prompt tuning, arXiv preprint arXiv:2403. 17369, 2024.
[25] Shurui Gui, Meng Liu, Xiner Li, Youzhi Luo, and Shuiwang Ji. Joint learning of label and environ- ment causal independence for graph out-of-distribution generalization. Advances in Neural Information Processing Systems, 36, 2023.
[26] Sivan Harary, Eli Schwartz, Assaf Arbelle, Peter Staar, Shady Abu-Husscin, Elad Amrani, Roci Herzig, Amit Alfassy, Raja Giryes. Hilde Kuehne, et al. Unsupervised domain generalization by learning bridge across domains. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5280-5290, 2022.
[27] Conghui Hu and Gim Hee Lec. Feature representation learning for unsupervised cross-domain image retrieval. In European Conference on Computer Vision, pages 529-544. Springer, 2022.
[28] Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2021.
[29] Jiaxing Huang, Dayan Guan, Aoran Xiao, and Shijian Lu. Fsdr: Frequency space domain randomization for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6891-6902, 2021.
[30] L. Huang, Y. Zhou, F. Zhu, L. Liu, and L. Shao. Iterative normalization: Beyond standardization towards efficient whitening. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 4874-4883, 2019.
[31] Lei Huang. Yi Zhou, Fan Zhu, Li Liu, and Ling Shao. Iterative normalization: Beyond standardization towards efficient whitening. Proceedings of the ieeelcvf conference on computer vision and pattern recognition, pages 4874-4883, 2019.
[32] Wei Huang. Chang Chen, Yong Li, Jiacheng Li, Cheng Li, Fenglong Song, Youliang Yan, and Zhiwei Xiong. Style projected clustering for domain generalized semantic segmentation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 3061-33100 107111 2023.
[33] Christoph Hümmer, Manuel Schwonberg, Liangwei Zhong, Hu Cao. Alois Knoll, and Hanno Gottschalk. Vltseg: Simple transfer clip-based vision-language representations for domain generalized semantic segmentation. arXiv preprint arXiv:2312.0202) 2023.
[34] Wei Ji, Jingjing Li, Shuang Yu, Miao Zhang, Yongri Piao, Shunyu Yao, Qi Bi, Kai Ma. Yefeng Zheng, Huchuan Lu, et al. Calibrated rgb-d salient object detection. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 9471 198811 2021.
12

---

[35] Wei Ji, Shuang Yu, Junde Wu, Kai Ma, Cheng Bian, Qi Bi, Jingjing Li. Hanruo Liu, Li Cheng, and Yefeng Zheng. Learning calibrated medical image segmentation via multi-rater agreement modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12341- 12351, 2021.
[36] Wei Ji. Jingjing Li, Cheng Bian, Zongwei Zhou, Jiaying Zhao, Alan L Yuille, and Li Cheng. Multispectral video semantic segmentation: A benchmark dataset and baseline. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1094-1104, 2023.
[37] Wei Ji, Jingjing Li, Qi Bi, Tingwei Liu, Wenbo Li, and Li Cheng. Segment anything is not always perfect: investigation of sam on different real-world applications. Machine Intelligence Research, 21:617-630,
2024.
[38] Menglin Jia, Luming Tang. Bor-Chun Chen, Claire Cardie, Serge Belongie. Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In European Conference on Computer Vision, pages 709-727, 2022.
[39] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloc Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. arXiv preprint arXiv:2304.00 02644333,0, 2023.
[40] Sangrok Lee, Jongseong Bae. and Young Kim. Decompose, adjust, compose: Effective normalization by playing with frequency for domain generalization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11776-11 1785, 2023.
[41] Suhycon Lec, Hongje Scong, Scongwon Lec, and Euntai Kim. Wildnet: Learning domain generalized semantic segmentation from the wild. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 9936-9946, 2022.
[42] Chunyuan Li, Heerad Farkhoor, Rosanne Liu, and Jason Yosinski. Measuring the intrinsic dimension of objective landscapes. In International Conference on Learning Representations, 2018.
[43] Fuhao Li, Ziyang Gong, Yupeng Deng, Xianzheng Ma, Renrui Zhang, Zhenming Ji, Xiangwei Zhu, and Hong Zhang. Parsing all adverse scenes: Severity-awar aware semantic segmentation with mask-enhanced cross-domain consistency. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 13483-13491, 2024.
[44] Jingjing Li, Wei Ji, Qi Bi, Cheng Yan, Miao Zhang, Yongri Piao, Huchuan Lu,, et al. Joint semantic mining for weakly supervised rgh-d salient object detection. Advances in Neural Information Processing Systems, 34:11945-11959 2021.
[45] Yumeng Li, Dan Zhang. Margret Keuper, and Anna Khoreva. Intra-source style augmentation for improved domain generalization. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pages 509-519, 2023.
[46] Shiqi Lin, Zhizheng Zhang, Zhipeng Huang, Yan Lu, Cuiling Lan, Peng Chu, Quanzeng You, Jiang Wang, Zicheng Liu, Amey Parulkar, et al. Deep frequency filtering for domain generalization. Proceedings of the IEÉE/CVF Conference on Computer Vision and Puttern Recognition, pages 11797-11807, 2023.
[47] Fangrui Lv, Jian Liang, Shuang Li, Bin Zang, Chi Harold Liu, Ziteng Wang, and Di Liu. Causality inspired representation learning for domain generalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8046 805680066620 2022.
[48] Divyat Mahajan, Shruti Tople, and Amit Sharma. Domain generalization using causal matching. In International Conference on Machine Learning, pages 7313-7324. PMLR, 2021.
[49] M Jehanzeb Mirza, Mare Masana, Horst Possegger, and Horst Bischof. An efficient domain-incremental learning approach to drive in all weather conditions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2022.
[50] Vincent W Neo, Soydan Redif, John G McWhirter, Jennifer Pestana. Ian K Proudler, Stephan Weiss, and Patrick A Naylor. Polynomial eigenvalue decomposition for multichannel broadband signal processing: mathematical technique offering new insights and solutions. IEEE Signal Processing Magazine, 40(7): 18-37, 2023.
[51] Gerhard Neuhold, Tobias Ollmann, Samuel Rota Bulo, and Peter Kontschieder. The mapillary vistas dataset for semantic understanding of street scenes. In Proceedings of the IEEE international conference computer vision, pages 4999, 2017.
13

---

[52] Joshua Niemeijer, Manuel Schwonberg, Jan-Aike Termöhlen, Nico M Schmidt, and Tim Fingscheidt Generalization by adaptation: Diffusion-based domain extension for domain -enerrlleeee semantic segmen- tation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2830-2840, 2024.
[53] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby. et al. Dinov2: Learning robust visual features without supervision. arXiv preprint 2rXivv2304.07199, 2023.
[54] Junwen Pan, Qi Bi. Yanzhan Yang, Pengfei Zhu, and Cheng Bian. Label-efficient hybrid-supervised learning for medical image segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 2026- 2034, 2022.
[55] X. Pan, P. Luo, J. Shi, and X. Tang. Two at once: Enhancing learning and generalization capacities via ien-net. In Proceedings of the European Conference on Computer Vision, pages (6444799.40 2018.
[56] X. Pan. X. Zhan, J. Shi, X. Tang, and P. Luo. Switchable whitening for deep representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1863-1871, 2019.
[57] Duo Peng, Yinjie Lei, Lingqiao Liu, Pingping Zhang, and Jun Liu. Global and local texture randomization for synthetic-to-real semantic segmentation. IEEE Transactions on Image Processing, 30:6594-666 6608, 2021.
[58] Duo Peng, Yinjie Lei, Munawar Hayat, Yulan Guo, and Wen Li. Semantic-aware domain generalized segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 2594-2605, 2022.
[59] Xi Peng, Fengchun Qiao, and Long Zhao. Out-of-domain generalization from a single source: An uncertainty quantification approach. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022.
[60] Piotr Porwik and Agnieszka Lisowska. The haar-wavelet transform in digital image processing: its status and achievements. Machine graphics and vision, 13(1/2)77--9,, 2004.
[61] Fengehun Qiao, Long Zhao, and Xi Peng. Learning to learn single domain generalization. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 12556-12565, 2020.
[62] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabricl Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pages 8748-8763 1021.
[63] Stephan R Richter, Vibhav Vineet, Stefan Roth, and Vladlen Koltun. Playing for data: Ground truth from computer games. In European conference on computer vision, pages 102-118. Springer, 2016.
[64] German Ros, Laura Sellart, Joanna Materzynska, David Vazquez, and Antonio M Lopez. The synthia dataset: large collection of synthetic images for semantic segmentation of urban scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3234-3243, 2016.
[65] Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Aede: The adverse conditions dataset with corre- spondences for semantic driving scene understanding. In Proceedings of the IEFF/CVF International Conference on Computer Vision, pages 10765-10775, 2021.
[66] Mattia Segu, Alessio Tonioni, and Federico Tombari. Batch normalization embeddings for deep domain gencralization. Pattern Recognition, 135: 109115, 2023.
[67] Hao Shen, Zhong- Lrong-Qiu Zhao, Yulun Zhang, and Zhao Zhang. Mutual information-drinn triple interaction network for efficient image dehazing. In Proceedings of the 31st ACM International Conference on Multimedia, pages 7-16, 2023.
[68] Peifeng Tong, Wu Su. He Li. Jialin Ding, Zhan Haoxiang, and Song Xi Chen. Distribution free domain generalization. In International Conference on Machine Learning, pages 14369-34378, 2023.
[69] Zhixiang Wei. Lin Chen, Yi Jin, Xiaoxiao Ma, Tianle Liu, Pengyang Lin, Ben Wang, Huaian Chen, and Jinjin Zheng. Stronger, fewer, & superior: Harnessing vision foundation models for domain generalized semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024.
14

---

[70] Mixue Xie. Shuang Li, Rui Zhang, and Chi Harold Liu. Dirichlet-based uncertainty calibration for active domain adaptation. In The Eleventh International Conference Learning Reprexentations, 2023.
[71] Qi Xu, Liang Yao, Zhengkai Jiang, Guannan Jiang, Wenqing Chu, Wenhui Han, Wei Zhang, Chengjie Wang. and Ying Tai. Dirl: Domain-invariant representation learning for generalizable semantic segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36. pages 2884-2892, 2022.
[72] Xin Yang. Wending Yan, Yuan Yuan, Michael Bi Mi, and Robby T Tan. Semantic segmentation in multiple adverse weather conditions with domain knowledge retention. Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 6558 6566, 2024.
[73] Yuedong Yang. Hung- Yueh Chiang, Guihong Li, Diana Marculescu, and Radu Marculescu. Efficient low-rank backpropagation for vision transformer adaptation. Advances in Neural Information Processing Systems, 36, 2024.
[74] Qinghao Ye, Xiyue Shen, Yuan Gao, Zirui Wang, Qi Bi, Ping Li, and Guang Yang. Temporal cue guided video highlight detection with low-rank vusiol fusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7950-7959, 2021.
[75] Jingjun Yi, Qi Bi, Hao Zheng, Haolan Zhan, Wei Ji, Yawen Huang, Shaoxin Li, Yuexiang Li, Yefeng Zheng. and Feiyue Huang. Hallueinated style distillation for single domain generalization in medical image segmentation. International Conference on Medical Image Computing and Computer-Assine Intervention, pages 438-448, 2024.
[76] Fisher Yu, Wenqi Xian, Yingying Chen, Fangehen Liu, Mike Liao, Vashisht Madhavan, and Trevor Darrell, Bdd100k: diverse driving video database with scalable annotation tooling. arXiv preprint 1eeXiv:800..om 0.667, 2(5)66, 2018.
[77] Hu Yu, Jie Huang, Yajing Liu, Qi 7hu, Man Zhou, and Feng 7hao. Source-free domain adaptation for real-world image dehazing. In Proceedings of the 30th ACM International Conference on Multimedia, pages
2022.
[78] X. Yue, Y. Zhang, S. Zhao, A. Langiovanni-Vincentllli K. Keutzer, and B. Gong. Domain randomization and pyramid consistency: Simulation-to-real generalization without accessing target domain data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2100-21 10, 2019.
[79] Zhongqi Yuc, Qianru Sun, and Hanwang Zhang, Make the in uda matter: Invariant consistency learning for unsupervised domain adaptation. Advances in Neural Information Processing Systems, 36, 2023.
[80] Shanshan Zhao, Mingming Gong, Tongliang Liu, Huan Fu, and Dacheng Tao. Domain generalization via entropy regularization. Advances in Neural Information Processing Systems, 33:16096116107, 2020.
[81] Xiaoqi Zhao, Youwei Pang. Wei Ji, Baicheng Sheng, Jiaming Zuo, Lihe Zhang, and Huchuan Lu. Spider: A unified framework for context-dependent concept segmentation. In Proceedings of the 41st International Conference on Machine Learning, volume 235, pages 60906-60926, 2024.
[82] Yuyang Zhao, Zhun Zhong, Na Zhao, Nicu Sebe, and Gim Hec Lec. htyle-hallucinated dual consistency learning for domain generalized semantic segmentation. In Proceedings of the European Conference on Computer Vision, pages 535-552. Springer, 2022.
[83] Zhun Zhong, Yuyang Zhao, Gim Hee Lee, and Nicu Sehe. Adversarial style augmentation for domain generalized urban-scene segmentation. In Advances in Newral Information Processing Systems, 2022.
[84] Ronghang Zhu, Xiang Yu, and Sheng Li. Supi-super single domain generalization with label-free adversarial data augmentation. Transactions on Machine Learning Research, 2023.
15

---

A Appendix/ supplemental material
A.1 Theoretical Analysis on Haar Wavelets
The Haar transform [60] cross-multiplies a function with various shifts and stretches, which has been demonstrated to be effective in applications, such as signal and image processing. In other words, it is able to analyze the local aspects of a signal.
Definition 1. Haar Scaling Function. Given an input signal the Haar scaling function is mathematically defined
x(1) {
0st<1 otherwise
(13)
Given the space of all functions of the form ΣREE agd(I= - k) as Vo. where k E Z is an arbitrary integer, and ag E R. As each element of Vo is zero outside a bounded set, such a function 0x6(x - k) has finite or compact support.
Definition 2. Basis of the Step Function Space. Given an arbitrary nonnegative integer j E Z
Let , denote the step function space at the level j. which is spanned by the set
(.., + 1), ((2 d622x 1),.....
(14)
Theorem 1. (1) A function (x) belongs to Vo = f(2'x) belongs Vj. (2) A finction f(x)
belongs Vj f(2- belongs to Vo.
Proof. (1) If f(x) E Vo. then we have f(x) = ΣkEZ and(z k). where E R. Then, f(2/z) =
ΣKEZ 02 k). It means f(2'x) E Vg. (2) The proof of (2) is similar.
Theorem 2. The set of functions (21/2 k), E Z is an orthonormal basis Vf
Proof. The norm of certain basis 2// k)) is 2 =
21 2572
=1. (2) For any two basis m and (m A n), 2m 222"x k), 22) k) 2m/22 2n/2
< 6(2">xr k), ((2"x k) = 0. Thus, the set of functions (2)/2 k), k E Z} is an orthonormal basis of V,-
Definition 3. Haar Wavelet Function. The Haar wavelet is the function vxxx) ((xx) 612x 1).
Theorem 3. The Haar wavelet function (xx) E V1, and orthogonal to Vo-
Lemma 1. function f1 = EkEZ 02 k) E V1, iee, orthogonal to each 6(x - 1), E Z if and only if a, -ag. 03 022
Given (bxx) E Vo. if [x()) = Evez 02 k) Vo. then EvEz agd(2r - k) M(x). Therefore, fz( 1 Vo if and only if < E bez a220(2x k), 0(x) >= 0. As 6(z) = 6(2x) (22x
Proof.
1), we have
()
>= ao < ((2x), > ta1 < 1(2x-1), 0(2x >= 0.
kgz
(15)
Thus, 00 aj. Similarly, by inspecting < Ebez axdl2x - k), 6(x - -1) >= 0, we have
02 + 0g = 0,
Proof of Theorem 3. Based on Lemma 1, we have
(x) = Σ (02( - k) (22x - k - 1)) = Σ
a2 - k))
(16)
KEZ
kEZ
A.2 Impact on Token Length m
By default we set the token length m in the proposed FADA as 100. To study its impact on the unseen target domain, the scenarios when m are set 25, 50, 75, 125, 150 and 175 are tested. CityScapes is used as the source domain. The results on B, M, and S unseen target domains are displayed in Fig. 8 a, b, and d, respectively. It is observed that, the generalization performance when m ranges from 75 to 125 is relatively stable, while the performance when m is too small (i.e., 25, 50) or too large (i.e., 150, 175) shows a slight decline.
16

---

5d.
223
114
245
413
435
455
Ma: 15
412
413
275
02
25
2
06
25 413
5
120
25
(a) C to
(b) C to M
(c) C to
(d) C to S
Figure 8: Ablation studies on the token length m. Experiments are conducted under the -+
76 M [1. G 633. S (4 settings, which are displayed from left to right.
Table 6: Ablation study on the positions the frequency adapters. GTAS as the source domain. CityScapes, BDD and Mappilary unseen target domains. Evaluation metric is mloU in %. Top three results are highlighted best
second and third respectively.
Method
Citys
BDD
Map
Avg.
Full
63.7
57.4
64.2
61.7
Freeze
63.3
56.1
63.9
61.1
REIN [69
66.4
60.4
66.1
64.3
Shallow
67.6
61.5
67.4
65.5
Deep
67.3
61.2
67.0
65.2
FADA
66.7
61.9
66.1
64.9
A.3 Impact the Position of the Frequency Adapter
The high-level idea of our intuition focuses on the frequency space. As existing LoRA based and side-adapter based methods implement the adaptation each of the transformer layer, therefore treat the frequency space as whole, and embed it into each transformer layer.
Nevertheless, it would be meaningful have an in-depth analysis the learning behaviour from shallow to deep. To this end, apart from the REIN baseline [69] and the proposed FADA, we further provide two experiments, where the frequency adapter is attached in the first half seven layers (denoted as shallow) and the second half seven layers (denoted as deep). respectively. Results in show that:
Using the frequency adapter on the first half layers (shallow) shows a slightly better perfor- mance than the second half layers (deep). may be explained that the shallower features contain more cross-domain styles, such as illumination, landscape and etc.
Using the frequency adapter on all layers (ours) achieves the best performance, indicating its effectiveness on all layers.
A.4 Comparison with Token Fine- Fine-Tuning Methods
We further compare the proposed method with some existing parameter-efficient fine-tuning (PEFT) methods, namely, AdvStyle 833 PASTA M. GTR-LTR (571, LoRA (28), AdaptFormer (10), VPT
38] and REIN [69]. Following the setting in REIN 669 GTAV is used as the source domain. BDD, CityScapes and Map used as unseen target domains. Results in Table show that the proposed
FADA outperforms these methods on all unseen target domains.
A.5 More Visual Results
Fig. Mand Fig. show more results under C B, M. G, and C ACDC setting. On both settings, the segmentation results show that the proposed FADA shows better pixel-wise prediction than the compared DGSS methods, especially in terms of the completeness of objects.
17

---

Table 7: Performance Comparison of the proposed FADA against other DGSS and PEFT methods under the G-> C, B, M setting. The best results are highlighted. * denotes trainable parameters in
backbones. Top three results are highlighted as best second and third respectively.
Fine-tune
Trainable
mloU
Backbone
Method
Params*
Citys
BDD
Map
Avg.
Full
304.24M
62.1
56.2
64.6
60.9
+AdvStyle 83
304.24M
63.1
56.4
64.0
61.2
EVA02
20
+PASTA 9
304.24M
61.8
57.1
63.6
60.8
Freeze
0.00M
56.5
53.6
58.6
56.2
+AdyStyle
83
0.00M
51.4
51.6
56.5
53.2
+PASTA 9
0.00M
57.8
52.3
58.5
56.2
+GTR-LTR
0.00M
52.5
52.8
57.1
54.1
+LoRA 28
1.18M
55.5
52.7
58.3
55.5
+AdaptFormer 10
3.17M
63.7
59.9
64.2
62.6
+VPT 38
3.69M
62.2
57.7
62.5
60.8
+Rein (ours)
2.99M
65.3
60.5
64.9
63.6
+FADA
11.65M
66.7
61.9
66.1
64.9
Full
304.20M
63.7
57.4
64.2
61.7
+AdvStyle
X3
304.20 20M
60.8
58.0
62.5
60.4
+PASTA 9
304.20M
62.5
57.2
64.7
61.5
DINOv2
+GTR-LTR
304.20M
62.7
57.4
64.5
61.6
(Large)
53
Freeze
0.00M
63.3
56.1
63.9
61.1
+AdvStyle S3
0.00M
61.5
55.1
63.9
60.1
+PASTA 9
0.00M
62.1
57.2
64.5
61.3
+GTR-LTR
0.00M
60.2
57.7
62.2
60.0
+LoRA 28
0.79M
65.2
58.3
64.6
62.7
+AdaptFormer
10
3.17M
64.9
59.0
64.2
62.7
+VPT 38
3.69M
65.2
59.4
65.5
63.3
WHT 73
3.51M
65.8
58.9
65.3
63.3
*FourieFFT
21
0.67M
66.1
59.2
65.8
63.7
+REIN (69
2.99M
66.4
60.4
66.1
64.3
+FADA
11.65M
68.2
62.0
68.1
66.1
18

---

Unseen images
SAW
WildNet
SPC
REIN
Ours
Figure 9: Visual segmentation results on unseen target domains under the C B, M, G, setting.
The proposed FADA is compared with ISW (15), SAW (58), WildNet [41), SPC [32 and Rein [? J.
19

---

Unseen images
SAW
WildNet
SPC
REIN
Ours
Figure 10: Visual segmentation results on unseen target domains under the C + ACDC setting. The proposed FADA is compared with ISW (15), SAW (58), WildNet [41], SPC [32 and Rein [? ].
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: This paper proposes to learn style-invariant property from vision foundation model from the frequency space by Haar wavelet transformation.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: At the end of the conclusion section, the limitation of the proposed method has been discussed.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiitiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
21

---

Answer: [Yes]
Justification:
The theory assumptions are in the methodology section.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: The model realization and implementation details are provided in the experi- mental section.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
22

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
The datasets are public. The source code will be publicly available up on publication.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ https/cccppllligguddeeeeeeeeeeee Cooee//ddeeeebmimcia ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The experimental settings and details are provided in the experiment section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: The evaluation protocols of these segmentation datasets do not require report the error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
23

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The computation resources and details are discussed in experiment section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't I make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The authors have read the code of ethics. The experiments are all on public datasets without obeying the code of ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
At the end of the conclusion section, the broader impacts have been discussed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This work does not have such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The asserts used in this paper are all public available for academic researches.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
25

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
26


Reviews:
[{"review_id": "UVu3CFnnuA", "reviewer": "Reviewer_esrP", "summary": "This paper leverages the vision foundation model (VFM) for domain-generalized semantic segmentation (DGSS). It proposes to adapt the VFM to the downstream task in the frequency domain. The proposed method decouples the low-frequency and high-frequency components of the VFM features and separately applies low-rank adaptation to them. This aims to stabilize the domain-invariant content and, on the other hand, eliminate the impact of domain-specific styles on DGSS.", "questions": "VFMs are known for their strong generalization ability across different domains. What specific style-invariant properties of VFM are the authors attempting to exploit in this paper?", "limitations": "Please provide the necessary evidence and adequate clarification of the problems in the weaknesses.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* The paper is well-written and organized. \n* The idea of applying low-rank adaptation of VFM in the frequency domain has some certain novelty. \n* The proposed method achieves competitive performance on several datasets across different settings compared to previous methods.", "weaknesses": "* **Insufficient justification of key concepts:** The paper's claim that domain-specific style and domain-invariant content can be demonstrated by high-frequency and low-frequency components is not well-justified in this context. And, the explanation of how the proposed method stabilizes domain-invariant content while mitigating domain-specific style is insufficiently supported.\n\n* **Lack of detailed analysis on style elimination:** Given the stated importance of style-invariant properties in VFM for DGSS representations, eliminating the impact of domain-specific styles is crucial. While the paper uses instance normalization to address this, it lacks a detailed analysis of the effectiveness of this approach.\n\n* **Unclear representation of Haar components:** Although the paper suggests that the three high-frequency components represent domain-specific style information that should be eliminated for better DGSS, Table 2 indicates that the best results are achieved by preserving all components. The discussion around this in Line 250 is not sufficiently informative.\n\n* **Inadequate comparison with previous methods:** The paper also mentions previous works that have utilized FFT for frequency analysis and claims that it may be the first to use Haar wavelet for domain generalization. However, it does not adequately discuss the advantages of Haar wavelet over other methods or provide comparative analysis in this context.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1**: Justify: 1) domain-specific style and domain-invariant content are demonstrated by high- and low-frequency components 2) how the proposed method stabilizes domain-invariant content while mitigating domain-specific style.\n\n**R**: Thanks for your valuable comment. To clarify:\n\n1) In visual domain generalization, an important research line is to learn a domain-invariant content representation from the frequency feature space. It has been observed and well-documented that the low- and high- frequency components rest more domain-invariant content and domain-specific style properties [25,36,32,62,56].\n\n2) To validate this point in our context, we extract the frozen VFM feature from driving scene samples containing the same semantic categories (i.e., content) but from four unseen domains (i.e., style), and implement the Haar wavelet transformation on them.\nAccording to the t-SNE feature visualization in Fig.R1 Row 1 (attached in the 1-pg PDF file), the high-frequency features ($HH$, $HL$, $LH$) exhibit some domain-specific clusters, where driving scenes from the same domain (points of the same color) lean to cluster together.\nIn contrast, the low-frequency features ($LL$) allow the driving scenes from different domains (point of different color) to be more uniformly mixed and distributed.\n\n3) To inspect the effectiveness of the proposed method on mitigating the domain-specific features, we implement the instance normalization transformation on the high-frequency features ($HH$, $HL$, $LH$) from the VFM.\nAfter that, the t-SNE feature visualization is conducted and the results are shown in Fig.R1 Row 2.\nIt can be seen that, compared with the original high-frequency features in Fig.R1 Row 1, the domain-specific clusters are alleviated, and the samples from different domains (points of different colors) are more uniformly distributed.\n\n**Q2**: More analysis on instance normalization for style invariance.\n\n**R**: Thanks for your insightful comments. Please refer to the 2) and 3) response to Comment\\#1 for the detailed analysis.\n\nIn addition, we would like to kindly raise the reviewer's attention that, Fig.3 and Fig.4 in the main text provide analysis on how the proposed method eliminates the domain-specific styles in both feature space and channel-wise activation patterns. \n\n**Q3**: L250 \\& Table 2. Eliminate high-frequency components or preserving all components. \n\n**R**: To clarify, our objective is not *to eliminate the high-frequency components that rest in domain-specific styles*, but *to be invariant to cross-domain style variation* (L172) \\& *to decouple the high-frequency component from the impact of cross-domain styles* (L250).\nThe reason behind is that the high-frequency components contain not only styles, but also other information such as structure and object boundary.\nA naive removal of all high-frequency components loses such information, which degrades a scene representation and declines the segmentation performance. \nTherefore, our objective is to allow the high-frequency representation to be invariant to the cross-domain styles by instance normalization, not to entirely eliminate all the high-frequency representation.\nIt aligns with the ablation study in Table 2, where the low-rank adaptation with instance normalization on high-frequency components contribute positively to the performance. \nWe will clarify this point accordingly when revising.\n\n**Q4**: Discuss/compare the advantages of Haar wavelet over FFT methods.\n\n**R**: \n1) Both FFT and Haar wavelet decouple a scene representation into low- and high- frequency components.\nHowever, compared with FFT, the orthonormal basis of Haar wavelet warrants the orthonormality between the low- and high- frequency components.\nOrthonormality is essentially a type of de-correlation, which provides a better separation between the low- and high- frequency components.\nIn the context of DGSS, the orthonormality of Haar wavelet transformation allows a better separation between the scene content and the style. \n\n2) From the experimental side,  please refer to Table R1 in the general response for the comparison.\nIt is observed that, the proposed method, with the aid of Haar wavelet, shows a predominant performance improvement than FFT based [a] on all unseen target domains. \nMeanwhile, we respectively ask for the reviewer's understanding that, as most frequency decoupling based domain generalization methods are devised for classification task , it would be difficult and impractical to adapt all these methods for VFM based DGSS.\n\n[a] Gao, Z., Wang, Q., Chen, A., Liu, Z., Wu, B., Chen, L., \\& Li, J. (2024). Parameter-Efficient Fine-Tuning with Discrete Fourier Transform. ICML 2024.\n\n**Q5**: What specific style-invariant properties?\n\n**R**: Indeed, VFM has an inherited ability to generalize to out-of-distribution.\nHowever, the performance of current large segmentation models still lacks in specific down-stream scenarios, so additional training is needed.\nConsequently, fine-tuning VFM on DGSS encounters the unique challenge of DGSS, that is, an ideal representation for the task of DGSS is to learn stable pixel-wise content representation despite the cross-domain style shift [44,26,13,45,63,60,47].\nHere the style shift can be posed by different types of weather, illumination, urban landscape and etc [67].\nHowever, the fine-tuning of VFM is highly dependent on the low-dimensional intrinsic embeddings [58], which is subtle and fragile to the distribution shift, which is posed by the style shift in the context of DGSS.\nTherefore, an ideal VFM representation for DGSS is supposed to demonstrate robust style-invariant property to generalize well on different types of weather, illumination, urban landscape and etc [67].\n\nOn the other hand, as shown in Table 1, 2, 4 and 5, the modified VFM representation from the proposed method shows a significantly better generalization on unseen target domains, in different weather and day-night conditions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VezXxgkioi", "reviewer": "Reviewer_eWsS", "summary": "This paper focuses on adapting the vision foundation model for domain-generalized segmentation via frequency-aware adaptation. Specifically, the intermediate features are decomposed into low-frequency (content) and high-frequency (style) components, which are then being processed with separate low- and high- frequency branches. To this end, sufficient experiments on extensive datasets validate the effectiveness of the proposed method.", "questions": "See the weakness section.", "limitations": "Limitations have been discussed.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The problem setting is important. To adapt the vision foundation model in a domain-generalizable manner, parameter-efficient fine-tuning such as LoRA has been extensively used, this paper proposed a solution in an orthogonal direction -- adapting in the frequency domain. The core intuition makes sense to me. This paper decomposes the features into low-frequency and high-frequency domains and is adapted separately. The whole solution is simple and elegant. Sufficient comparison on extensive datasets and difference foundation model architectures validates the effectiveness of the proposed method.", "weaknesses": "1. When conducting the adaptation, it is intuitive that the low-frequency/ high-frequency components correspond to the content/style. However, can the author comment on the intuition on adapting in all the transformer laters? It may not be that clear whether the low-frequency and high-frequency features in the later layers have such correspondence anymore. From that sense, it would be nice to ablate the layer location where the frequency adapter is added (as opposed to being added to all the layers) \n\n\n2. When comparing with the REIN, REIN only has a trainable parameter of 2.99M (compared to FADA 11.65), the author also shows in Table 3 that FADA performance is influenced by the number of Rank and tunable parameters. It would be nice to add an experiment that compares these two methods with similar tunable parameters. \n\n\n3. The core intuition behind the decomposition is to prioritize the domain-invariant low-frequency features (content features) over the high-frequency features. However, as indicated by the ablation results in Table 2, better outcomes were achieved by adapting to all features. The current model design does not differentiate between high-frequency and low-frequency features, raising questions about the alignment between the model design and the initial intuition. A potential experimental setup that might clarify this involves training the model on multiple domain datasets with a shared low-frequency branch and distinct high-frequency branches. However, this would alter the experimental conditions. I would appreciate the authors' comments on this matter.\n\n4. Conducing Haar transform can cause additional inference time, it would be nice to include the inference time comparision.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1**: 1) Intuition on adapting in all the transformer layers. 2) Ablate the layer location.\n\n**R**: Thanks for the constructive comment.\n\n1) The high-level idea of our intuition focuses on low-rank adaptation (LoRA) in the frequency space.\nAs existing LoRA methods [24,58] implements the adaptation on each of the transformer layer, and we further introduce the frequency space.\nTherefore, we treat the frequency space as a whole, and embed it into each of the transformer layer.\n\n2) But we definitely agree with the reviewer that, it would be meaningful to have an in-depth analysis on the learning behaviour from shallow to deep.\nTo this end, apart from the baseline (REIN [58]) and the proposed method, we further provide two experiments, where the frequency adapter is attached in the first half seven layers (denoted as shallow) and the second half seven layers (denoted as deep), respectively.\nResults in Table R1 show that:\n- using the frequency adapter on the first half layers (shallow) shows a slightly better performance than on the second half layers (deep). It may be explained that the shallower features contain more cross-domain styles, such as illumination, landscape and etc.\n- using the frequency adapter on all layers (ours) achieves the best performance, indicating its effectiveness on all layers.\n\nTable R1: Ablation study on the positions of the frequency adapters. GTA5 as the source domain. CityScapes, BDD and Mappilary are unseen target domains. Evaluation metric mIoU in \\%.\n| Method | Citys | BDD | Map | Avg.  |\n|----------|----------|----------|----------|----------|\n| Full | 63.7 | 57.4 | 64.2 | 61.7 |\n| Freeze | 63.3 | 56.1 | 63.9 | 61.1 |\n| REIN [58] | 66.4 | 60.4 | 66.1 | 64.3 |\n| Shallow | 67.6 | 61.5 | 67.4 | 65.5 |\n| Deep | 67.3 | 61.2 | 67.0 | 65.2 |\n| FADA | **68.2** | **62.0** | **68.1** | **66.1** |\n|||||\n\n**Q2**: Compare FADA with REIN under similar tunable parameters.\n\n**R**: Thanks for your constructive suggestion.\nThe feature dimension of MLPs in LoRA impacts the parameter number. We therefore minimize the feature dimension in FADA, from 1024 to 256, so that the parameters are reduced significantly.\nResults in Table R2 show that, even if the feature dimension is only one fourth of REIN [58], FADA still shows a 1.1\\% mIoU improvement on three unseen domains.\n\nTable R2: Comparison between FADA and REIN under similar tunable parameters. GTA5 as source domain. \n| Method | Rank | Dimension | #para. | Citys | BDD | Map | Avg.  |\n|----------|----------|----------|----------|----------|----------|----------|----------|\n| REIN | 16 | 1024 | 2.99M | 66.4 | 60.4 | 66.1 | 64.3 |\n| **FADA** | 16 | 256 | **2.92M** | **67.6** | **61.2** | **67.5** | **65.4** |\n||||||||\n\n**Q3**: 1) prioritize low- over high-frequency features v.s. Table 2 adapting to all features. 2) suggest an experiment design.\n\n**R**: Thanks for your constructive comments, so that we could have a chance to have a more in-depth analysis between the low- and high- frequency representation.\n\n1) In general, the intuition of the proposed method is to allow the high-frequency components to be invariant to the style shift, instead of eliminating the entire high-frequency components.\nThe reasonable behind is that, high-frequency components of an image representation consist of not only styles, but also other types of information such as the scene structure, object boundaries and etc.\nScene structure and object boundary is also important for scene segmentation.\nTherefore, our objective is to adapt the high-frequency components so that they are invariant to the cross-domain shift, not to directly eliminate all the high-frequency components.\n\n2) We definitely agree with the reviewer that, it would be beneficial to specify the high-frequency features from multiple domains.\nFollowing your suggestion, we follow the evaluation setting where GTA5 and Synthia are both used as the source domains, while CityScapes, BDD and Mapillary are used as unseen target domains, and REIN [58] as the baseline.\nAfter Haar wavelet transformation on the per-batch image from each domain, their low-frequency components are fused into a single low-rank adaptation module. \nInstead, two low-rank adaptation modules are used to process the high-frequency components from the two source domains, respectively (denoted as $HH_{G+S}$).\nBoth high-frequency components are implemented with an instance normalization after the adaptation. \nIt is compared with the scenario where one high-frequency component is used, denoted as $HH_G$ and $HH_S$.\nThe results in Table R3 show that, handling the high-frequency component from each source domain indeed improves the performance on unseen target domains.\nHandling the high-frequency components from both source domains shows a further improvement.\n\nTable R3: Impact on the learning paradigm of high-frequency components.\nGTA5 and SYNTHIA are both source domains. CityScapes, BDD and Mapilary are unseen target domains.\n| Method | Citys | BDD | Map | Avg.  |\n|----------|----------|----------|----------|----------|\n| Freeze | 64.8 | 60.2 | 65.2 | 63.4 |\n| REIN [58] | 68.1 | 60.5 | 67.1 | 65.2 |\n| $HH_{G}$ | 68.9 | 61.4 | 68.0 | 66.1 |\n| $HH_{S}$ | 68.7 | 61.2 | 67.6 | 65.8 |\n| $HH_{G+S}$ | 69.4 | 61.8 | 68.4 | 66.5 |\n| **FADA  (ours)** | **70.2** | **62.4** | **68.9** | **67.2** |\n|||||\n\n**Q4**: Compare the inference time.\n\n**R**: We compare the inference time of the proposed FADA along with REIN [58] and a vanilla VFM. All the experiments are conducted on a single RTX 3090Ti GPU.\nAs reported in Table R4, the proposed FADA, implementing Haar wavelet transformation with additional parameters, only leads to a slight decrease on the inference time than the previous REIN [58].\n\nTable R4: Inference Time Comparison. Evaluation Metric: Frame Per Second (FPS).\n| Method | Vanilla VFM | REIN | FADA (Ours)  | \n|----------|----------|----------|----------|\n| FPS | 2.50 | 2.49 | 2.46 |\n||||", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eQimcS6kEe", "reviewer": "Reviewer_3Mxh", "summary": "This paper introduces a novel FADA learning scheme to improve domain-generalized semantic segmentation. The proposed method leverages the Haar wavelet transformation to separate style and content information into high- and low-frequency components, respectively, allowing for better handling of domain variations. Experimental results demonstrate that FADA outperforms existing DGSS methods, showcasing its effectiveness and versatility across various vision foundation models (VFMs).", "questions": "1. Can the proposed low rank adaptation method outperforms the other recent low rank adaptation methods as mentioned in weakness 1?\n\n\n2. The authors are encouraged to add more insights towards the proposed method in the experiment analysis section.\n\n3. The ground truth figures are suggested to be added in Figure 5 and Figure 6-\n\n4. The implementation details are encouraged to be enriched.", "limitations": "yes it is mentioned be the authors.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed method is interesting and show good DGSS performances on various DG settings, which will be beneficial to the community.\n\n2. Comprehensive ablation studies are done by the authors. The ablation on different VFMs is very important to showcase the model generalizability of the proposed low rank adaptation method.\n\n3. The method is clearly described and great visualizations are provided.\n\n\n----------------------------\n\nScore has been updated. Thanks the authors for the rebuttal.", "weaknesses": "1. Lack of comparison with other recent low rank adaptation methods during the experiments, e.g., the works as follows,\n\n[1] Gao, Z., Wang, Q., Chen, A., Liu, Z., Wu, B., Chen, L., & Li, J. (2024). Parameter-Efficient Fine-Tuning with Discrete Fourier Transform. arXiv preprint arXiv:2405.03003.\n[2] Yang, Y., Chiang, H. Y., Li, G., Marculescu, D., & Marculescu, R. (2024). Efficient low-rank backpropagation for vision transformer adaptation. Advances in Neural Information Processing Systems, 36.\n\n\n2. Lack of insights in the experimental analysis, e.g., in section 5.2, the authors simply listed out the performance gains brought by the proposed method compared with the state-of-art methods without mentioning why it can achieve these superior performances.\n\n3. The ground truth images are not listed on Figure 5 and Figure 6, which hinders the comparison regarding the qualitative performances.\n\n\n4.Lack of description of the implementation details. What losses are used during the experiments? Which lr scheduler is used?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1**: Compare recent LoRA methods [a,b].\n\n[a] Gao, Z., et al. (2024). Parameter-Efficient Fine-Tuning with Discrete Fourier Transform. ICML.\n\n[b] Yang, Y., et al. (2024). Efficient low-rank backpropagation for vision transformer adaptation. NeurIPS.\n\n**R**: We've compared both methods, namely, FourierFT [a] and WHT [b], with the baseline, REIN [58] and the proposed FADA under the G$\\rightarrow$ C, B, M setting.\nFor fair evaluation, both [a] and [b] are attached to process the VFM features of each layer as the common low-rank adaptation paradigm does.\nThe results in Table R1 show that the proposed FADA shows its superiority than the recent methods [a, b] on all unseen target domains with a $\\textgreater$ 2\\% mIoU improvement. \n\nTable R1: Performance Comparison of the proposed FADA with recent PEFT methods [a,b]. Evaluation metric mIoU in \\%.\n| Method | Citys | BDD | Map | Avg.  |\n|----------|----------|----------|----------|----------|\n| Full | 63.7 | 57.4 | 64.2 | 61.7 |\n| Freeze | 63.3 | 56.1 | 63.9 | 61.1 |\n| LoRA [24] | 65.2 | 58.3 | 64.6 | 62.7 |\n| REIN [58] | 66.4 | 60.4 | 66.1 | 64.3 |\n| FourierFT [a] | 66.1 | 59.2 | 65.8 | 63.7 |\n| WHT [b] | 65.8 | 58.9 | 65.3 | 63.3 |\n| FADA | **68.2** | **62.0** | **68.1** | **66.1** |\n|||||\n\n**Q2**: Sec.5.2, the authors are encouraged to add more insights towards the proposed method in the experiment analysis section.\n\n**R**:\nThanks for your constructive suggestions, so that we could have the chance to provide more insight in the experimental section. We will enrich the following analysis accordingly.\n\n(1) From the perspective of representation. Vision Transformer based encoder is more capable to capture the scene content owing to the long-range dependencies than the CNN based encoder, which explains the significant improvement of Transformer based methods than CNN based methods.\nOn top of it, both the proposed method and REIN [58] use Transformer based foundation model, which not only possesses the inherited representation ability from the Transformer, but also occupies the inherited out-of-distribution generalization ability from the large-scale data pre-training.\nMost importantly, compared with REIN [58], the proposed method further advances low-rank adaptation in the frequency space, which handles the scene content and the style variation in a divide-and-conquer manner, therefore further improving the generalization ability on unseen target domains.\n\n(2) From the perspective of domain gap. The first two experiments use GTA5 and SYNTHIA as source domains, respectively.\nBoth source domains are synthetic data.\nEven if trained on such synthetic source domain, the proposed method still shows a generalization performance of 68.2\\% on real-world CityScapes target domain.\nThis outcome is very close to some modern CNN and Transformer segmentation models when full-trained on CityScapes, which further indicates the generalization ability of proposed method.\nIn contrast, the compared DGSS methods still shows a significantly inferior performance when the domain gap is huge. \n\n(3) From the perspective of low-rank adaptation in frequency space. In visual domain generalization and DGSS, an ideal representation is to be stable to the scene content when the cross-domain styles shift greatly. \nAs many prior works [25,36,32,62,56] reflect, the decoupling of high- from low-frequency component is an important research line to decouple the style from content. \nThe proposed method takes advantage of this aspect, and implements Haar wavelet transformation to realize this objective.\nThe individual low-rank adaptation for low- and high- frequency components provides a divide-and-conquer path for VFM features to handle the style and content, respectively, and therefore significantly improves the performance on unseen target domains when compared with REIN [58].\n\nFinally, we would like to kindly raise the reviewer's attention that, in Sec.5.3 we have provided the ablation studies from four different aspects to analyze why the proposed method achieves a better performance than the VFM baseline and REIN [58].\n\n**Q3**: The ground truth images are not listed on Figure 5 and Figure 6, which hinders the comparison regarding the qualitative performances. \n\n**R**: Thanks for your constructive suggestions.\nWe have accordingly provided the ground truth map for each sample in Fig.5 and Fig.6.\nThe updated Fig.5 and Fig.6 have been provided in the attached PDF file in the general response, named as Fig.R2 and Fig.R3, respectively. \n\n**Q4**: Lack of description of the implementation details. What losses are used during the experiments? Which lr scheduler is used? \n\n**R**:\nWe are sorry that the implementation details of the proposed method is extensive than intended.\nIn general, all the loss functions, learning scheduler and other hyper-parameters keep the same as REIN for fair comparison.\nSpecifically, the optimizer uses AdamW.\nThe initial learning rate for the backbone is $1\\times10^{-5}$.\nThe initial learning rate on the decoder and the proposed FADA's parameter is set $1\\times10^{-4}$.\nThe final loss function $\\mathcal{L}$ directly inherit the losses from the Mask2Former decoder [12], given by\n\n$\\mathcal{L} = \\lambda _ {ce} \\mathcal{L} _ {ce} + \\lambda _ {dice} \\mathcal{L} _ {dice} + \\lambda _ {cls} \\mathcal{L} _ {cls}$\n\nwhere $\\mathcal{L} _ {ce}$, $\\mathcal{L} _ {dice}$ and $\\mathcal{L} _ {cls}$ denote the cross-entropy loss, dice loss and classification loss. \nHere the hyper-parameters $\\lambda _ {ce}$, $\\lambda _ {dice}$ and $\\mathcal{L} _ {cls}$ are 5.0, 5.0 and 2.0, respectively.\nFor more details of the classification loss, please refer to [12].\nWe will enrich these details accordingly. \n\nFinally, should you have further questions and suggestions, we are glad to address during the discussion stage.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ptYB7eBBH7", "reviewer": "Reviewer_nB2o", "summary": "This paper proposes a Frequency Adaptive (FADA) learning approach. Its core idea is to process content and style information separately, by using frequency tokens. Specifically, the FADA comprises two branches for low-frequency and high-frequency components. The high-frequency components learn scene styles and eliminate their impact on DGSS, while the lower-frequency components contribute to stabilizing scene contents. Experiments in various DGSS settings demonstrate FADA's performance and versatility across different VFMs.", "questions": "See the weaknesses.", "limitations": "Yes.", "rating": 5, "confidence": 1, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The overall presentation is pretty good.\n2. The idea of using frequence token is impressive.\n3. The achieved performance seems meaningful.", "weaknesses": "Honestly, I do not know well about this topic.\nSo I would like to ask some questions from the general perspective.\n\n1. Is the assumption of \"similar content yet varied style\" practical? It seems that the main target of this paper is autonomous driving. Then it is rational to assume almost the same categorical distribution would be observed over various data, including the unseen test one. However, how about the joint distribution of content and style? Can we assume that they are independent across the dataset?\n\n2. Can I see the results on the other VFMs? I understand that DINO-V2 is one of the most popular choices, but I am personally interested in whether the proposed method functions well on a more segmentation-oriented model, such as SAM. I think replacing DINO-V2 with the SAM image encoder can be an interesting experiment.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1**: Is the assumption of \"similar content yet varied style\" practical? The joint distribution of content and style? Can we assume that they are independent across the dataset?\n\n**R**: Thanks for your insightful comment so that we could have a chance to further clarify the basic assumption in domain generalized semantic segmentation (DGSS).\nIn the context of DGSS and autonomous driving, it has been well established and recognized by the prior DGSS works that the key challenge is 'similar content yet varied style' [13,28,33,47,66,67].\n\nJust as the reviewer acknowledged, despite the style shifts greatly in driving scenes, it is reasonable that in driving scenes the categorical distribution is nearly the same.\n\nOn the other hand, we definitely agree with the reviewer that, it is a great perspective to discuss if the joint style and content distribution (i.e., the driving scenes) is independent or not.\nIt should be highlighted that, the joint style and content distribution in DGSS, same as the assumption in generic visual domain generalization, is not (rigorously) independent.\nAs extensively discussed in [5,67], the style in driving scenes and DGSS is impacted and jointly determined by multiple extrinsic factors such as weather, lighting, urban landscape and etc.\nAn example is that, two datasets collected from two cities with different landscapes are under the same weather, which makes the style vary but not rigorously independent between each other. \n\nTo sum up, given that domains in DGSS are not rigorously independent and generally have the distribution shift, which in lines with the generic visual domain generalization, we would like to raise the reviewer's attention that, the established assumption 'similar content yet varied style' [13,28,33,47,66,67] is practical in DGSS.\n\n**Q2**: Results on the other VFMs (e.g. SAM).\n\n**R**:\nWe would like to kindly raise the reviewer's attention that, the results on other VFMs have been provided in Table 4 and we attach it as follows.\n\nTable R1: Performance Comparison of the proposed FADA with baseline on other VFMs. Evaluation metric mIoU in \\%.\n| Backbone | Method | Citys | BDD | Map | Avg.  |\n|----------|----------|----------|----------|----------|----------|\n| SAM | Full | 57.6 | 51.7 | 61.5 | 56.9 |\n| SAM | Freeze | 57.0 | 47.1 | 58.4 | 54.2 |\n| SAM | REIN [58] | 59.6 | 52.0 | 62.1 | 57.9 |\n| SAM | **FADA** | **61.0** | **53.2** | **63.4** | **60.0** |\n|----------|----------|----------|----------|----------|----------|\n| CLIP | Full | 51.3 | 47.6 | 54.3 | 51.1 |\n| CLIP | Freeze | 53.7 | 48.7 | 55.0 | 52.4 |\n| CLIP | REIN [58] | 57.1 | 54.7 | 60.5 | 57.4 |\n| CLIP | **FADA** | **58.7** | **55.8** | **62.1** | **58.9** |\n|----------|----------|----------|----------|----------|----------|\n| EVA02 | Full | 62.1 | 56.2 | 64.6 | 60.9 |\n| EVA02 | Freeze | 56.5 | 53.6 | 58.6 | 56.2 |\n| EVA02 | REIN [58] | 65.3 | 60.5 | 64.9 | 63.6 |\n| EVA02 | **FADA** | **66.7** | **61.9** | **66.1** | **64.9** |\n|----------|----------|----------|----------|----------|----------|\n| DINOv2 | Full | 63.7 | 57.4 | 64.2 | 61.7 |\n| DINOv2 | Freeze | 63.3 | 56.1 | 63.9 | 61.1 |\n| DINOv2 | REIN [58] | 66.4 | 60.4 | 66.1 | 64.3 |\n| DINOv2 | **FADA** | **68.2** | **62.0** | **68.1** | **66.1** |\n|||||\n\nIn short, when using SAM as the image encoder, the proposed method outperforms REIN [58] by 1.4\\%, 1.2\\% and 1.3\\% mIoU on C, B and M unseen target domains;\nwhen using CLIP as the image encoder, the proposed method outperforms REIN [58] by 1.6\\%, 1.1\\% and 1.6\\% mIoU on C, B and M unseen target domains;\nwhen using EVA02 as the image encoder, the proposed method outperforms REIN [58] by 1.4\\%, 1.4\\% and 1.3\\% mIoU on C, B and M unseen target domains.\nThese outcomes further demonstrate that the proposed method is robust and generalized on a variety of VFM image encoders.\n\nFinally, should you have further suggestions, we are glad to address during the discussion stage.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NrY8AQeqXb", "reviewer": "Reviewer_4xQk", "summary": "The paper proposes a Frequency-Adapted (FADA) learning scheme, where Haar wavelet transformation is introduced to decouple the frozen VFM features into low- and high-frequency components. Experiments demonstrate the proposed method achieves a better generalization on unseen target domains.", "questions": "I'm more concerned about [1] also focusing on mitigating the effects of style variations on the DGSS. What is the difference between the paper with [1]? Besides, why is exploring the style-invariant properties of VFM important and urgent? What are the advantages of 1) leveraging Haar wavelet for domain generalization and 2) enhancing the generalization ability of VFM features via frequency space in Domain Generalization by Frequency Decoupling over existing related works?\n[1] Style Blind Domain Generalized Semantic Segmentation via Covariance Alignment and Semantic Consistence Contrastive Learning. CVPR 2024", "limitations": "The proposed method heavily relies on Low-/High-Frequency Decomposition. However, the manner is fixed and not adjusted dynamically.", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "The proposed method achieves good performance.", "weaknesses": "1. The writing and logic are poor. For example, the relationship among the existing three DGSS category methods is missing. Why explore the style-invariant properties of VFM is important and urgent? The introduction is not accompanied by an explanation of Figure 1.\n2. What are the advantages of 1) leveraging Haar wavelet for domain generalization and 2) enhancing the generalization ability of VFM features via frequency space in Domain Generalization by Frequency Decoupling over existing related works?\n3. [1] also focuses on mitigating the effects of style variations on the DGSS. What is the difference between the paper with [1]?\n[1] Style Blind Domain Generalized Semantic Segmentation via Covariance Alignment and Semantic Consistence Contrastive Learning. CVPR 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1**: Writing \\& logic. 1) the relationship among three DGSS category methods; 2) Why the style-invariant properties of VFM is important and urgent? 3) The introduction is not accompanied by an explanation of Fig. 1.\n\n**R**: Thanks for your valuable feedback so that we could have the chance to introduce a native speaker to thoroughly polish the writing and logic. The mentioned problems are clarified as follows and will be incorporated accordingly.   \n\n1) Existing three types of DGSS methods, either decouple the style, augment the style or stabilize the content, are all full-training paradigm (on a CNN or Vision Transformer encoder).\nIn contrast, the realm of VFM provides a new paradigm for DGSS, which only fine-tunes on a minor part of model parameters instead of full-training.\nThe motivation of this work is to allow the VFM features to stabilize the content while at the same time be invariant to the style shift, which aligns the advantages of different types of existing DGSS methods.\n\n2) An ideal representation for DGSS is to learn stable content representation despite the cross-domain style shift [44,26,13,45,63,60,47].\nHowever, the fine-tuning of VFM is highly dependent on the low-dimensional intrinsic embeddings [58], which is subtle and fragile to the distribution shift, which is posed by the style shift in the context of DGSS.\nTherefore, an ideal VFM representation for DGSS is supposed to demonstrate robust style-invariant property, so that it is able to demonstrate robust scene representation from different domains. \n\n3) We would like to kindly raise the reviewer's attention that, the explanation of Fig.1 is in the third line of the introduction; while the caption of Fig.1 further details the the domain generalization property of the high-/low- frequency components of Haar wavelet transformation.\n\n**Q2**: Advantages of 1) Haar wavelet for domain generalization, 2) enhancing generalization of VFM features by Frequency Decoupling over existing related works?\n\n**R**:\nThanks for your constructive comments. To clarify:\n\n1) Both FFT based methods and Haar wavelet can decouple a scene representation into low- and high- frequency components.\nHowever, compared with FFT, Haar wavelet transformation consists of orthonormal basis, and warrants the orthonormality between the low- and high- frequency components.\nOrthonormality is essentially a type of de-correlation, which provides a better separation between the low- and high- frequency components.\nIn the context of DGSS, the orthonormality of Haar wavelet transformation allows a better seperation between the scene content and the style. \nMeanwhile, from the experimental side, Table R1 (in general response) shows that our Haar wavelet based method shows a superior performance on all unseen target domains than the FFT based FourierFT [a].\nThis outcome further demonstrates the advantage of Haar wavelet over existing Fourier transformation based methods.\n\n2) We would like to kindly raise the reviewer's attention that, harnessing VFM for DGSS is an emerging research line.\nBy the time of submission, the closest work is REIN [58].\nREIN [58] directly implements low-rank adaptation to learn the entire scene representation. It does not decouple the content and style, which is necessary in visual domain generalization.\nIn contrast, the advantage of the proposed method is that, it provides a feasible path to decouple the style from the content by the separation between high- and low- frequency components. \nThis decoupling allows the representation learning to handle the style and content in a divide-and-conquer manner, where the style representation is supposed to be invariant to the domain, while the scene content is supposed to be stable.\n\n3) The research line of frequency decoupling based methods is focused on generic domain generalization (classification task), not for DGSS or VFM based DGSS.\n\n**Q3**: What is the difference between the paper with [1]? \n\n[1] Style Blind Domain Generalized Semantic Segmentation via Covariance Alignment and Semantic Consistence Contrastive Learning. CVPR 2024.\n\n**R**:\nThanks for your reference suggestion, so that we could have a chance to discuss the difference between the recent CVPR 2024 work BlindNet. \nWe would like to kindly raise the reviewer's attention that, this paper is essentially different from the proposed method in many critical aspects.\n\n1) Completely different methodology design. BlindNet first augments the style diversity from the image, and constraints the scene representation by designing style similarity and content consistency based losses.\nIn contrast, the proposed method decouples VFM features in the frequency space by Haar wavelet to decouple the style and content separately.\nIt does not reply on any types of style augmentation or augmented domains, which is more universal and more generalized than BlindNet.\n\n2) Completely different path to eliminate the style. BlindNet\nfirst augments the styles from the original image, and then\nmitigates the style variation by computing and reducing the style similarity between the per- and post- augmented styles.\nIn contrast, the proposed method decouples the style and content from the frozen VFM features by high- and low- frequency components, respectively.\nAfterwards, the low-rank adaptation on the high-frequency representation is implemented with an instance normalization transformation, so that the high-frequency representation is invariant to the style shift and better generalized to unseen target domains.\n\n3) Significant performance improvement. The official report of BlindNet on $C \\rightarrow \\{C, B, M, S\\}$ is 38.56\\%, 34.51\\%, 40.11\\% and 25.64\\% mIoU, while this paper achieves 68.23\\%, 61.94\\%, 68.09\\% and 50.36\\% mIoU.\nThe official report of BlindNet on $C \\rightarrow \\{B, M, G, S\\}$ is 59.27\\%, 71.10\\%, 58.11\\% and 40.43\\% mIoU, while this paper achieves 65.12\\%, 75.86\\%, 63.78\\% and 49.75\\% mIoU.\n\nWe will discuss this work accordingly.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4fGQ3KfZod", "reviewer": "Reviewer_ojsk", "summary": "This paper deals with adapting a computer vision foundational model to a new domain.  They propose to use the frequency based on Haar wavelets to  to decouple the style and content information and then to address separately  content and style  domain adaptation. Experiments show state of the art results.", "questions": "see above", "limitations": "Yes", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is relatively well-written. Adapting high and low frequency components separately seems like a good idea although it could be more motivated.  \n\nFigure 1 is compelling. \n\nExperimental results show consistent improvement. \n\nTested across various foundational models.", "weaknesses": "The adaptation for low-frequency and high-frequency components are different. The paper could provide more intuition to explain the particular adaptations chosen for low and high frequency components.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1**: The adaptation for low-frequency and high-frequency components are different. The paper could provide more intuition to explain the particular adaptations chosen for low and high frequency components.\n\n**R**:\nThanks for your general positive comments on our work and your insightful comment, so that we could have the chance to provide a more in-depth analysis on the particular adaptations.\n\nIn general, the adaptation on high-frequency components is further implemented by an instance normalization when compared to the low-frequency components.\nInstance normalization, which computes the channel-wise mean and standard deviation, is effective to eliminate the domain-specific styles [44,27].\n\nTo further validate this point, in the attached file in general response, we visualize the t-SNE feature space of the samples from four unseen domains.\nThe results without and with implementing the instance normalization on the high-frequency components are displayed in the first and second row of Fig.R1, respectively.\nFig.R1 is attached in the 1-pg PDF in the general response.\nNotice that, when without instance normalization, the method is the same as using the same adaptation on the low-frequency component. \n\nIt can be seen that, the use of instance normalization alleviates the existence of the domain-specific clusters (in the same color) and allows samples from different domains to be more uniformly distributed (in different color).\nThe domain-specific clusters indicate that samples from the same domain have less distance and higher similarity in the feature space.\nIt indicates that the different adaptation for high-frequency component is more effective to develop the style-invariant property.\n\nFinally, should you have further suggestions and questions, we are glad to address during the discussion stage.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for their time and constructive suggestions, and are glad that the reviewers unanimously give appreciation in a few points:

- Technique Contribution \& Innovation (**esrP**: low-rank adaptation in the frequency domain has novelty; **eWsS**: the whole solution is simple and elegant; **3Mxh**: the proposed method is interesting; **nB2o**: the idea of using frequency token is impressive; **ojsk**: compelling.)

- Writing \& Presentation (**esrP**: well-written and organized; **eWsS**: good presentation; **3Mxh**: The method is clearly described
and great visualizations are provided; **nB2o**: The overall presentation is pretty good; **ojsk**: well-written.)

- Good Performance (**esrP**: competitive performance; **eWsS**: Sufficient comparison; **3Mxh**: how good DGSS performances; **nB2o**: The achieved performance seems meaningful; **4xQk**: good performance; **ojsk**: consistent improvement.)

However, there are also some major concerns as follows.

- Compare with frequency based / parameter-efficient fine-tuning methods (**esrP** comment\#4; **3Mxh** comment\#1; **4xQk** comment\#2).

**R**: A recent frequency based foundation model fine-tuning method FourierFT [a] and a low-rank adaptation method WHT [b] are involved for comparison.
While some improvement over baseline can be observed, the proposed FADA still demonstrates the optimal performance in DGSS.
Meanwhile, we respectively ask for reviewer's understanding that, most existing frequency decoupling methods are devised for cross-domain classification [25,36,32,62,56], and can be difficult to be incorporated into the foundation model fine-tuning pipelines.

Table R1: Performance Comparison of the proposed FADA with recent PEFT methods [a,b]. Evaluation metric mIoU in \%.
| Method | Citys | BDD | Map | Avg.  |
|----------|----------|----------|----------|----------|
| Full | 63.7 | 57.4 | 64.2 | 61.7 |
| Freeze | 63.3 | 56.1 | 63.9 | 61.1 |
| LoRA [24] | 65.2 | 58.3 | 64.6 | 62.7 |
| REIN [58] | 66.4 | 60.4 | 66.1 | 64.3 |
| FourierFT [a] | 66.1 | 59.2 | 65.8 | 63.7 |
| WHT [b] | 65.8 | 58.9 | 65.3 | 63.3 |
| FADA | **68.2** | **62.0** | **68.1** | **66.1** |
|||||

- Analyze how handling low- and high- frequency component help stabilize content and invariant to style (**esrP** comment\#1, \#2; **ojsk** comment\#1).

**R**: 
We extract the frozen VFM feature from driving scene samples containing the same semantic categories (i.e., content) but from four unseen domains (i.e., style), and implement the Haar wavelet transformation on them.
According to the t-SNE feature visualization in Fig.R1 (attached in the 1-pg PDF file), the high-frequency features ($HH$, $HL$, $LH$) exhibit some domain-specific clusters, where driving scenes from the same domain (points of the same color) lean to cluster together.
In contrast, the low-frequency features ($LL$) allow the driving scenes from different domains (point of different color) to be more uniformly mixed and distributed.

To inspect the effectiveness of the proposed method on mitigating the domain-specific features, we implement the instance normalization transformation on the high-frequency features ($HH$, $HL$, $LH$) from the VFM.
After that, the t-SNE feature visualization is conducted and the results are shown in Fig.R1 Row 2.
It can be seen that, compared with the original high-frequency features in Fig.R1 Row 1, the domain-specific clusters are alleviated, and the samples from different domains (points of different colors) are more uniformly distributed.

- Eliminating styles \& high-frequency features v.s. preserving all features (**b4sx** Comment\#5; **hFq3**: comment\#3).

**R**: We humbly suggest there may existing some misunderstanding in the methodology objective.
Eliminating *the high-frequency components* and *be invariant to the domain-specific styles* is two different things.
The high-frequency components of a scene not only contain styles, but also contain other types of information such as structure, object boundary and etc. A naive removal of all the
high-frequency components also leads to the loss of such information, which can degrade a scene representation and therefore declines the segmentation performance.
Therefore, our objective is to allow
the high-frequency representation to be invariant to the cross-domain styles by instance normalization,
not to entirely eliminate all the high-frequency representation. It aligns with the ablation study in
Table 2, where the low-rank adaptation with instance normalization on high-frequency components
can contribute positively to the performance. We will clarify this point accordingly when revising

We hope our clarification could help to make a more informed evaluation to our work.
In the following individual response, we provide answers to each raised weakness/question.

Best regards,

Authors

Meta-review: Dear authors, 
Overall this draft got positive reviews (R:5 / C:5, R:4 / C:4,  R:6 / C:4, R: 7 / C:3,  R:5 / C:2,  R:5 / C:1 ). Two reviewers updated their rating to positive side. 
After careful review of the reviewer comments, answers by the authors, and draft itself, we are accepting the draft. However, authors are strongly encouraged to improve the readability of the paper, incorporate suggestions by the reviewers and handle the comments. 
For example,  
1) "advantages of Haar wavelet over other methods or provide comparative analysis in this context."  --- Needs to be  added with clarity and given due importance in the. text.
2) Role of Instance Normalization for style invariance needs more explanation and study (even if added to supplementary material and referred in the text). 
3) Fig.1's. caption should include basic information to read the figure. LL, HH, etc.. are defined much below in the text rather than before. Therefore, add this information in the caption.  At the same time  statements like "Domain Generalized Semantic Segmentation (DGSS) learns to generalize to arbitrary unseen target domains when only trained on the source domain; The key challenge lies in the stability
of the scene content, while the domain gap is caused by the style ..."  should be part of the text rather than caption. 
4) Table-1 either explain what is C, B and M are or explicitly refer to the text where these are properly defined. 
5) incorporate experiments using to reply to the author's questions.
etc..
Decision: Accept (poster)