Lightweight Frequency Masker for Cross-Domain  Few-Shot Semantic Segmentation
Authors: Jintao Tong
Yixiong Zou
Yuhua Li
Ruixuan Li
Venue: NeurIPS 2024 poster (2024)
Keywords: cross-domain few-shot segmentation
frequency component
feature disentanglement，channel attention
Primary area: machine_vision
TL;DR: We find simply filtering frequency components significantly improves CD-FSS performance. We delve into this for an interpretation, and further propose a lightweight frequency masker, importing only 2.5% parameters but averagely improving over 11%.

Abstract: Cross-domain few-shot segmentation (CD-FSS) is proposed to first pre-train the model on a large-scale source-domain dataset, and then transfer the model to data-scarce target-domain datasets for pixel-level segmentation. The significant domain gap between the source and target datasets leads to a sharp decline in the performance of existing few-shot segmentation (FSS) methods in cross-domain scenarios. In this work, we discover an intriguing phenomenon: simply filtering different frequency components for target domains can lead to a significant performance improvement, sometimes even as high as 14% mIoU. Then, we delve into this phenomenon for an interpretation, and find such improvements stem from the reduced inter-channel correlation in feature maps, which benefits CD-FSS with enhanced robustness against domain gaps and larger activated regions for segmentation. Based on this, we propose a lightweight frequency masker, which further reduces channel correlations by an Amplitude-Phase Masker (APM) module and an Adaptive Channel Phase Attention (ACPA) module. Notably, APM introduces only 0.01% additional parameters but improves the average performance by over 10%, and ACPA imports only 2.5% parameters but further improves the performance by over 1.5%, which significantly surpasses the state-of-the-art CD-FSS methods.

Lightweight Frequency Masker for Cross-Domain Few-Shot Semantic Segmentation
Jintao Tong
Yixiong Zou"
Yuhua Li
Ruixuan Li
School of Computer Science and Technology, Huazhong University of Science and Technology (jintaotong, yixiongz, idcliyuhua, rxli)Ohust.edu..
Abstract
Cross-domain few-shot segmentation (CD-FSS) is proposed to first pre-train the model on a large-scale source-domain dataset, and then transfer the model to data-scarce target-domain datasets for pixel-level segmentation. The significant domain gap between the source and target datasets leads to a sharp decline in the performance of existing few-shot segmentation (FSS) methods in cross-domain scenarios. In this work, we discover an intriguing phenomenon: simply filtering different frequency components for target domains can lead to a significant perfor- mance improvement, sometimes even as high as 14% mloU. Then, we delve into this phenomenon for an interpretation. and find such improvements stem from the reduced inter-channel correlation in feature maps, which benefits CD-FSS with enhanced robustness against domain gaps and larger activated regions for segmen- tation. Based on this, we propose a lightweight frequency masker, which further reduces channel correlations by an Amplitude- Phase Masker (APM) module and an Adaptive Channel Phase Attention (ACPA) module. Notably, APM introduces only 0.01 % additional parameters but improves the average performance by over 10%, and ACPA imports only 2.5% parameters but further improves the performance by over 1.5%, which significantly surpasses the state-of-the-art CD-FSS methods.
Introduction
Recent advancements in semantic segmentation have been driven by large-scale annotated datasets and developments in deep neural networks 8 50) 44 Nevertheless, the requirement for extensive labeled data remains significant challenge, particularly for dense prediction tasks like semantic segmentation. Hence, few-shot semantic segmentation (FSS) [35] 49] has been proposed to meet this challenge, aiming to produce predictions for the unseen categories with only limited annotated data. However, these FSS methods perform poorly when confronted with domain shifts, particularly when there is significant gap between the novel class (target domain) and the base class (source domain). This issue has spurred the development of the cross-domain few-shot semantic segmentation (CD-FSS) task (24). Despite various efforts CD-FSS, the outcomes remain sub-optimal.
To handle the domain shift problem, efforts have been made to study the generalization of neu- ral networks. Recently, some works 42 40 have explored this from the perspective of the frequency domain, achieving theoretical breakthroughs. Compared to humans, neural networks exhibit heightened sensitivity to different frequency components. Additionally, amplitude and phase exhibit distinct properties and effects on neural network performance. Inspired by these works, we study the domain shift problem from the perspective of the frequency domain and discover an intriguing phenomenon shown in Figure I for model already trained on the source domain, simply filtering frequency components of images during testing can lead to significant performance improvements, sometimes even as high as 14% mloU.
Corresponding author. Code is available http::////hthhbbbbom///gggaaaagp
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

A
P2
P
P5
P
P7
1441
RMA
JAP
X124
18/6
2223
413
659
BSS 935-3800
x
6d5
M:1
3:40
XM.4
4M4
M441
21N6
IMH
4714
605
11
643
1336
11:44
17:0
1011
2114
40
Figure 1: For a model already trained on the source domain, we simply filter out different frequency components and plot mloU against the maintained ones of images. P denotes Phase, A denotes Amplitude, H denotes High Frequency, and denotes Low Frequency. We can see the performance is significantly improved in most cases compared with the baseline (An,. Pa), even as high as 14% on the Chest X-ray dataset (A4, P.). In this paper, we delve into this phenomenon for interpretation, and propose lightweight frequency masker for efficient cross-domain few-shot segmentation.
In this paper, we delve into this phenomenon for an interpretation. Through experiments and mathematical derivations, we find the filtering of the phase and amplitude effectively disentangles feature channels, which lowers the channel correlations and helps the model capture larger range of semantic patterns. This benefits the model with improved robustness against large domain gaps, and helps to discover the whole object for segmentation.
Based on the above interpretations, task. This masker does not need to be trained on the source domain, and can be directly inserted into intermediate feature maps during target-domain fine-tuning. It includes an Amplitude-Phase Masker (APM) module and an Adaptive Channel Phase Attention (ACPA) module. APM adaptively learns on target domains to filter out harmful amplitude and phase components at a finer granularity, which improves the effectiveness of channel disentanglemennt ACPA learns the attention over channels through phase information. Notably, the APM module only introduces 0.01 % additional parameters, but can effectively improve the mIoU by over 10% average, and ACPA further improves the performance by 1.5% on average with only 2.5% additional parameters introduced.
propose lightweight frequency masker for the CD-FSS
In summary, our contributions can be listed as
We find a phenomenon that simply filtering frequency components on target domains can signifi- cantly improve performance, with the highest improvement reaching nearly 14%.
We delve into this phenomenon for interpretation. We find the frequency filtering operation can effectively disentangle feature map channels, benefiting the model with improved robustness against large domain gaps and a larger range of discovered regions of interest.
Based our interpretations, which significantly improves the mloU by 1 11% on average with only 2.5% parameters introduced.
propose lightweight frequency masker for the CD-FSS task,
Extensive experiments on four target datasets show that our work, through a simple and effective design, significantly outperforms the state-of-the-art CD-FSS method.
2
Interpreting Enhanced Performance from Frequency Filtering
In this section, we delve into why filtering certain frequency components can significantly improve CD-FSS performance in certain target domains for interpretation.
2.1 Preliminaries
Cross-domain few-shot semantic segmentation (CD-FSS) aims to generalize knowledge acquired from source domains with ample training labels to unseen target domains. Given source domain D. = (X,, C.K,,. va) and a target domain D. = (X,, }1), where X' represents input data distribution and ) represents label space. The model will be trained the training set from the D.. then applied to perform segmentation on novel classes in the D,. Notably, D, and D, exhibit distinct input data distribution, with their respective label spaces having no intersection, i.e., X, A X. Vd, O Di 0.
In this work, we adopt the episodic training manner. Specifically, both the training set sampled from D. and the testing set sampled from D, are composed of several episodes, each episode is constructed

---

Table Mutual information between feature channels for the best and the worst cases in Fig. find that mutual information (MI) consistently decreases when the performance is improved.
We
FSS-1000
Deepglobe
ISIC
Chest X-ray
Dataset
baseline
best
woest
huseline
best
weest
baseline
best
woest
haseline
best
wOuSE
I-sbot mloU
77.54
64,311
48.051)
33.19
40.121
33,441
33,161
27.124|
47.34
61.58
31.391
support MI
1.3736
1.37911
1
1,3679
1.39224)
1335841
1,3889
136974L
1.39511
1,3952
1,39304,
1.4315
quary MI
1.3739
1.3805
1882011
1.3667
1.34354)
1.35844]
1.3992
1.36944)
1.3890
1.3921
1.38774.
1.4368
235-1003
Deogdabe
ESIC
Cless X-rr
zoqgimpe
Figure 2: Mean Magnitude of Channels (MMC) for the best case in Fig. D on four target datasets. For domains with improved performance, their curves are lower than the baseline after masking.
of K support samples S = {I, M and a query Q = (bg (I is the image and M is the label). Within each episode, the model is expected to use {I,, M,3 and Ig to predict the query label.
2.2 Enhanced Performance Stem from Reduced Inter-Channa Correlation
Existing research indicates inter-channel relationships are crucial for performance, as different feature channels can represent distinct features 30). Therefore, we study the change of channel correlations
brought by the frequency filtering. Specifically, we measured the mean mutual information (MI) [4] between channels from the last layer of the backbone network. The measured cases include the combination of phase and amplitude for the highest and lowest performance in Fig. [ We report the
1-shot mIoU and MI in Tab. [ where we can see that for frequency combinations with improved mloU, their MI consistently decreases, indicating reduced inter-channel correlation in the feature maps. Conversely, for frequency combinations with decreased mloU, their MI consistently increases.
The higher the mutual information value, the larger the correlation between channels, while a low MI indicates more independent semantic information captured by different channels. Therefore, the experimental results demonstrate that improved performance is associated with decreased inter- channel correlation in the feature map.
2.3 Why Lower Inter-Channel Correlation is Better?
The reduced inter-channel correlation benefits our model in two aspects:
(1) Cross-domain generalization. Previous works 2 51 indicate that lower correlation between features implics reduced redundancy and enhanced generalizability. Intuitively, a lower inter-channel correlation demonstrates that channels capture patterns more independently, therefore each channel will capture patterns in the input image more uniformly, which means the mean magnitude of each channel across all images will be more uniform. Consistent with intuition, [30] shows the channel bias problem affects the generalizability of few-shot methods, and utilizes the Mean Magnitude of Channels (MMC) to visualize and measure the channel response in features, where effective few-shot methods might have a more uniform MMC curve in the testing set. Therefore, this means the reduced correlation also benefits our model by addressing the channel bias problem, as studied in (30
Inspired by this, we visualize the MMC before and after applying the mask to filter frequency components on four target datasets. As shown in Figure for FSS-1000, performance degrades after masking, with the curve steeper than the baseline. Conversely, for the other three target datasets, performance improves with the curve more uniform than the baseline after masking. This indicates the channel bias problem is also handled by frequency filtering, which benefits the model with more independent and diverse semantic patterns to represent target domains.
(2) Exploring larger activated regions for segmentation. To study why reducing inter-channel correlation through frequency filtering benefits the segmentation task, we visualize the heatmap of feature maps before and after filtering out specific frequency components. As shown Figure Ja), after filtering certain frequency components, the heatmap demonstrates expanded activation regions,
3

---

which results from more recognition patterns captured by independent channels. Since the segmenta- tion task requires the model to pixel-wisely detect the whole semantic object, an expanded activation region means the model can better detect the entire object, instead of only focusing on the most discriminative parts.
2.4
Why Feature Disentanglement in the Frequency Domain?
In this subsection, we illustrate why the feature disentanglement is carried in the frequency domain.
Fourier Transform (FT) FT transforms finite signals into complex-valued functions frequency. For single channel E Rxx w of the feature map, the Fourier transform is formulated as:
w- A-1
u(u,u) = wh Σ
Σ 2=0 y=0
f(x y) 12x (#+
(1)
where i is imaginary unit, h and the height and the width of f. f(x, y) an element of f at spatial pixel (zz y). and Fu, v) represents the Fourier coefficient at frequency component (u, v).
The process in which the spatial feature f is decomposed into the amplitude and phase p is called spectral decomposition. The corresponding frequency feature F can be reassembled from amplitude and phase p (|F) is the modulus of F):
= a (p) + ia sin(p) = a: ctP
(2)
F V a + sin) Va² =
Mathematical Derivation. To intuitively prove the correlation between phase differences and channel correlation within a feature map. For different channels of feature map, representing distinct features, Fi (immm) n)) = 01 t and (2 n.) atee are defined as the frequency domain representations of the same location in different channels. The correlation coefficient calculated based on the following formula in the frequency domain:
h-1 10ml
n)
f- 10-1
Σ
r(m, n)
(4)
(lFimm,n) F2 n)
m==0
m=0 n=0
The F2 (m, n) is the complex conjugate of F2 n), can be computed as:
(m, n) 0t2 cos(p2) iag sin(p2) ) 0:26 - ip2
(5)
Substituting equations (2), (3), and (5) into equation (4) yields:
Fi Fi n) F5, n) = a1aye"(01-pp)
(6)
[iim, (11m,
=fa?
(7)
from which we can further derive:
(pi-pa)) ((P1-pp), P1 P2 = Ap E [0,
r(m, n) = Vataz
(8)
According to Euler's formula, we know that e- = - and e = 1. From this derivation, we have proved that the correlation between features in the spatial domain can be translated into phase differences and amplitudes in the frequency domain. When the frequency components are identical, the following can be inferred: 1) Ap O, r(m, n) 1 indicates perfect positive correlation; 2) Ap π,, r(m, n) = -1 indicates a perfect negative correlation. Therefore, when the phase differences of more corresponding points from different channels the frequency domain aggregate around ( or n, there is higher correlation between the channels. For amplitude, when the phase differences are the same, the closer the amplitudes, the more similar the waveforms are, thus indicating higher correlation between the channels.

---

TSS-1080
Chest
degress
64
Neas
Image
travelirr
mask
Place Serncce (aadaar)
Prose Noraro (ediooa)
Map Visuslization
(b) Diflerence Histograms
Figure 3: (a) After masking certain frequency components, the model's attention regions are enlarged with more patterns encompassed. (b) A higher concentration of phase differences at and indicates a higher correlation, so that on FSS-1000 the performance drops but on Chest X-ray it increases.
Experiments for derivation. To validate our derivations, we measured the phase difference his- tograms between channels, using amplitude as weights, to observe the relationship between phase differences among feature map channels. The phase difference and weight are as follows:
01
Ap |p P2l, weight
01 a22
The results shown in Figure Bib), indicate that for FSS-1000, after masking certain frequency components, the inter-channel correlation increases. Correspondingly, phase differences between channels tend to cluster more around ( and Conversely, for Chest X-ray, masking certain frequency components reduces inter-channel correlation, resulting in fewer phase differences clustering around 0 and æ. The experimental results have validated the accuracy of our derived conclusions.
2.5 Conclusion and Discussion
Through mathematical derivation and experiments, we demonstrate that manipulating the frequency can reduce inter-channel correlation in feature maps. Each channel of the feature map represents distinct pattern, and lower inter-channel correlation implies higher degree of channel disentangle- ment, leading to more independent and diverse semantic patterns for each feature. This benefits the model with 1) alleviation channel bias, boosting model robustness on target domains; and 2) larger activated regions for segmentation, therefore simple frequency filtering operation can significantly improve performance for the CD-FSS task.
Based on the above analysis, we can draw the following insights: 1) operates at the input level, but fundamentally affects the feature map's channel correlation. Therefore, we can directly apply mask operations to the frequency domain of each channel in the feature map; 2) Different domains require filtering different components. The aforementioned mask manually filters different frequencies based on the target domain, but the mask can be made adaptive; 3) The aforementioned mask does not perform well on FSS-1000. We believe this may be due to the overly coarse high-low frequency division. A finer frequency division can be designed, dividing the frequency into h wr parts (where and w are the spatial dimensions of the feature map).
The aforementioned mask
Method
Our method consists of two major steps, i.e., 1) amplitude-phase masker is proposed to reduce feature correlation, and obtain more accurate and generalized feature maps; 2) adaptive channel phase attention is proposed to select features that benefit the current instance and align the feature spaces of support and query. Our modules do not require source-domain training and can be directly integrated into during target-domain fine-tuning The overall framework of our approach is shown in Figure
3.1 Amplitude-Phant Phase Masker
Amplitude-Phane Phase Masker(APM) is a model-agnostic module that filters out negative frequency components at the feature level within feature maps. Through mathematical derivation, it is shown that APM accomplishes feature disentanglement. Consequently. this leads to a feature map that is more robust, generalizable, and provides broader and more accurate representations.
In our work, we utilize a fixed encoder, trained on the source domain, to extract feature maps F gexhxw where c, h and represent channels, height, and width. We then apply the Fast
5

---

Plas
Seature
sipwoiid
fealure
Pluir Jucsion Wriget
Jeature
IFFT
igess
sigmoid
Adaptive Channel Phase Attention
Aupeltul Spectann
support
Affinityy Maps
Encoder
APM
Compartioo Module
Decoder
Figure 4: Overview of our method in a I-shot example. After obtaining the feature map, APM is introduced to adaptively filter certain frequency components based on different domains, facilitating feature disentanglement to achieve more generalizable representations. Additionally, we propose ACPA encourage the model to focus on more effective features while aligning the feature space of the support and query images. The internal structure of APM and ACPA is highlighted in green.
Fourier Transformation (FFT) to convert these feature maps from the spatial domain into the frequency domain, further decomposing them into phase spectrum P and amplitude spectrum A:
A FF)(F)
(10)
Then, the phase and amplitude obtained from the FFT are each subjected to Hadamard product with their respective sigmoid-activated phase mask (PM) Mp and amplitude mask (AM) Ma- This operation effectively filters out negative components from both the phase and amplitude:
Penk Sigmoid( (M))) & P
(11)
Aenh Sigmoid(MM.) & A
where & indicates the element-wise multiplication, Penk and Aenh denotes the enhanced phase and amplitude, respectively. For each task, the APM initialized with all ones, where Sigmoid(MM [0. 1]. Here, I allows complete passage of frequency components, while 0 results in their total filtration. The original APM (APM-S) is lightweight module, configured as an h x w matrix that matches the height and width of the feature map. We also offer a variant APM (APM-M) that expands the dimensions to e h x w, in alignment with the feature map's 's dimensions.
The filtered phase and amplitude components are recombined and transformed back into the spatial domain using the Inverse Fast Fourier Transform (IFFT) to produce the enhanced feature map:
Fens = (.FEThAnnhee PPnn))
(12)
We iterate and optimize the APM using the support and its corresponding labels. After the APM process, the model generates feature map that is accurate and generalizable. This feature map then fed into the subsequent Adaptive Channel Phase Attention module for further optimization.
3.2 Adaptive Channel Phase Attention
Adaptive Channel Phase AitentionAACPA) the APM-optimized feature map, ACPA encourages the model to focus on more effective channels (features) while aligning the feature spaces of the support and query. Its underlying insight is that amplitude and phase are considered as style and content, respectively. Therefore, the phase can be seen an invariant representation, with consistent phase elements across both support and query.
be seen process of feature selection. Building
For the enhanced support feature maps Fsup support mask M E HxW is applied to discard irrelevant activations:
mask = Fesp & M
(13)
Subsequent operations input to ACPA are consistent with those applied to the query feature maps. Therefore, we uniformly refer to the feature map fed into ACPA as Fenh- We adopt a SE block following SENet [19] as our channel attention module, denoted as SF:
W Wphase SE((Pnn)
(14)

---

Table 2: Mean-IoU of I-shot and 5-shot results on the CD-FSS benchmark. The best and second-best results are in bold and underlined, respectively. denotes the model implemented by ourselves.
APM-S is an x w matrix, while APM-M (more parameters) expands to e x w.
FSS-1000
Deepglobe
ISIC
Chest X-ray
Average
Method
1-shot
5-shot
1-shot
5-shot
I-shot
5-shot
1-shot
5-shot
1-shot
5-shot
PGNNet 45
62.42
62.74
10.73
12.36
21.86
21.25
33.95
27.96
32.24
31.08
PANet
41
69.15
71.68
36.55
45.43
25.29
33,99
57.75
69.31
47.19
55.10
CaNer
70.67
72.03
22.32
23.07
25.16
28.22
28.35
28.62
36.63
37.99
RPMMs
43
65.12
67.06
12.99
13.47
18.02
20.04
30.11
30.82
31,56
32.85
PFENet
37
70.87
70.52
16.88
18.01
23.50
23,83
27.22
27.57
34.62
34.98
RePRI
70.96
74.23
25.03
27.41
23.27
26.23
65.08
65.48
46.09
48.34
HSNet
31
77.53
80.99
29.65
35.08
31.20
35.10
51.88
54.36
47.57
51.38
HSNet"
31
77.54
80.21
33.19
36.46
32.65
35.09
47.34
48.63
47.68
50.10
PATNet
24
78.59
81.23
37.89
42.97
41.16
53.58
66.61
70.20
56.06
61.99
Ours (APM-S)
78.25
80.29
40.77
44.85
41.48
49.39
75.22
76.89
58.93
62.86
Ours (APM-M)
79.29
81.83
40.86
44.92
41.71
$1.16
78.25
82.81
60.03
65.18
where Wphase E R 1x
is phase attention weight, Penh is the phase of Fenh. Then apply the phase attention weights to the feature map Fenh to obtain the final feature map F Fina!:
Ffinal Gi(Whasse @ Fend
(15)
where @ is the Hardmard product and (+*) extends the weight to match the dimension of the feature map by expanding along the spatial dimension, i.e., Ç Rex -> Roxxxxx
Finally, pair of query feature maps Fery final and support feature maps finat are fed into comparison module forms affinity maps C' IR hxxxxxxx using cosine similarity:
Fimas F (Ti)
C(m,n) R
(16)
1 finat
finat (n)ll
where m, n denote 2D spatial positions of feature maps Fary C(m, n) is fed into the decoder to obtain segmentation results, as shown in Figure
final and Foup
final respectively. Then, the
Experiments
4.1 Datasets
We utilize the benchmark established by PATNet (24] and adopt the same data preprocessing methods. For training, our source domain is the PASCAL-5* dataset [35], an extended version of PASCAL VOC 2012
[3] enhanced with additional annotations from the SDS dataset. For evaluation, target
domains include FSS-1000 (26). Deepglobe [TO, ISIC2018 g 38 and the Chest X-ray datasets
6 20
See Appendix A.1 for more details about datasets.
4.2
Implementation Details
our encoder, initialized with weights pre-trained on ImageNet [34] The training manner is consistent with our baseline model HSNet (3I). To optimize memory usage and speed up training, the spatial sizes of both support and query images are set to 400 x 400. The model is trained using the Adam [2 optimizer with a learning rate of 1e-3.
We employ ResNet-50 [18
During the adaptation stage, the model initially predicts the support mask and then uses the corre- sponding label to optimize the APM and ACPA through CE loss. The adaption stage of the APM and the ACPA leverage feature maps from conv5_x x whose channel dimensions are 2048 and spatial size x3x 13, is performed using the Adam optimizer, with learning rates at 0.1 for Chest X-ray, 0.01 for 1000 and ISIC, and le-5 for Deepglobe. Each task undergoes a total of 60 iterations.
4.3 Comparison with State-of-the-Art Works
we compare our method with several state-of-the-art few-shot semantic segmentation approaches on the benchmark introduced by PATNet (24). Our results show significant improvement cross-domain semantic segmentation for both 1-shot and 5-shot tasks. Specifically, APM-S exceeds the performance of the state-of-the-an ahe PATNet, based on ResNet-50, by 2.87% and 0.87% in average
In Table 2
7

---

Table 3: Ablation study various designs
FXN T101
APM-S
APM-M
ACPA
I-shol
S-shol
47.68
50.10
Deegghtn
57.77
61.39
58.93
62.86
59.13 13
63.53
60.03
65.18
ESIC
Table 4: APM-S implemented in the trans- former architecture.
ESS
Deepglohe
ISIC
Chest
Average
PerSAM
support
query
FPTrans
47
78.9
38.29
47.60
78.92
60.33
Figure 5: Qualitative results our model.
FPTranssours
79.84
39.78
50.37
79.29
62.32
mloU for the 1-shot and 5-shot settings, respectively. APM-M outperforms the state-of-the-art by 3.97% and 3.19%. Additionally, Pireee/Spreeentss qualitative results of our method in 1-way 1-shot segmentation, highlighting a substantial enhancement in generalization across large domain gaps while maintaining comparable accuracy with similar domain shifts.
4.4 Ablation Study
Effectiveness of each module. We evaluated each proposed module in both 1-shot and 5-shot settings to assess their effectiveness. As detailed in Table introducing APM-S increased the average mloU by 10.09% for 1-shot and 11.29% for 5-shot. Adding ACPA further enhanced the mloU by 1.16% and 1.47%, respectively. Additionally, APM-M, variant of APM with more parameters, when combined with ACPA, increased the average mloU by 12.35% for 1-shot and 15.08% for 5-shot.
Model-agmostic method. We implemented our method for FPTrans 477. which employs the Vision Transformer (VIT) [12] as its encoder. As shown in Table our approach can also effectively enhance performance of models based on the transformer architecture. Moreover, the new large-scale SAM (22 model has significantly advanced image segmentation, showcasing impressive zero-shot capabilities. However, SAM is not suited for cross-domain few-shot segmentation. Thus, we evaluate method with the SAM-baser approach. The result shows that our method performs much better than PerSAM in cross-domain few-shot segmentation.
PerSAM 48 to compare
Comparison with other method. To demonstrate the effectiveness of our method, we compared with full parameter the-ttunig and feature disentangling method, as shown in Table Compared to full parameter fine-tuning, our method uses fewer parameters and achieves better performance. For spatial domain feature disentangling, we added mutual information loss to the baseline model during training to encourage each channel of the feature map learn independent representations. Our method significantly outperforms this approach. Intuitively, feature disentanglement in the frequency domain offers finer granularity and global representation, which is more beneficial for segmentation tasks compared to the local representation in the spatial domain.
4.5
APM: Feature Disentanglement via Frequency Operations
Reduce inter-channel correlation. As shown in Table we validated APM's ability to reduce inter- feature correlation and improve generalization performance by calculating the mutual information between feature map channels. Compared to Table T this result demonstrates that the adaptive feature-level approach is more effective than the input-level masking, further reducing inter-featur correlation. Furthermore, we plotted the cumulative distribution function (CDF) of inter-channel correlations in the feature maps, as shown in Figure 6 APM, the CDF curve shifts to the left, indicating a decrease in inter-channel correlations.
be observed that with the inclusion of
Table 5: Compare our method with fine-tu tne-tuning and spatial domain feature disentangle method.
Table 6: Verify the effectiveness of the APM by the mean mutual information (MI). (w/o ACPA)
FSS
Despglobe
ISIC
Chest
Average
APM-S
Deepglobe
baseline
77.54
32.65
1-shot mloU
77.98t
40.741
38.79*
73.55t
haseline ft
78.06
33.54
33.28
74.16
54.76
support MI
1.3501
1.2761
1.3139]
1,2610
haseline-Mi
78.90
32.28
32.53
52.
Purs(AMM-MM)
79.29
40.86
41.71
78.25
60.03
query MI
1.3509.
1.2794]
1.3121]
1.2620L

---

BSIC
Chart
Figure 6: Cumulative distribution function (CDF) of inter-channel correlations. After passing through APM, the CDF curve shifts to the left, indicating a decrease inter-channel correlations.
More independent semantic representations. As shown in Table 7 (Left), visualized the heatmaps of feature maps processed by APM and those not processed by APM. It is evident that after applying APM, the model focuses on more different features. For example, in the first column, the baseline only highlights the swan't ' head, whereas APM makes each feature representation more independent, allowing the model attend various features of the swan.
4.6 ACPA: Aligning Task-Relevant Features and Feature Spaces
feature selection, enabling the model to focus on features that more effective for the current task while aligning the feature spaces of the support and query feature maps. As shown in Table 7 (Left), after APM disentangles the features and produces more broadly represented feature map, ACPA selects features that are more effective and discriminative example, in the first row, ACPA selects the swan's wings and head. In the second column, it selects the bird's 's head, tail, and feet. Furthermore, we measured the CKA (Centered Kernel Alignment) to calculate the distance between the support feature map and the query feature map, validating that ACPA aligns the support and query feature spaces. CKA is proposed to measure both intra-domain and inter-domain distances [22; the smaller the CKA value, the closer the feature spaces. As shown in Table 7 (Right), after applying APM, the distance between support and query is reduced, and with the addition of ACPA, the support and query feature spaces further aligned.
ACPA can be
the current task. For
Image
ori
APM
ACPA
APM-S
АPM-M
ACPA
FSS
Deepglobe
ISIC
Chest
0.3591
0.2691
0.2494
0.5848
0.2678
0.2433
0.5025
0.2907
0.2676
0.2032
0.3628
0.3293
0.2675
0.2310
0.4635
0.2883
0.2674
0.1811
0.3986
Table 7: (Left) Feature map visualizations show: 1) APM achieves feature disentanglement. 2) ACPA encourages the model focus on more effective features. (Right) Verify the effectiveness of the ACPA in aligning the support and query feature spaces by CKA measure.
4.7 Comparison with Domain Transfer Methods
We compare our method against traditional frequency-based and correlation- approaches to validate our method's effectiveness. For fair comparison, all methods are implemented on the baseline HSNet 31 and then evaluated under the 1-shot setting on the CD-FSS benchmark.
Frequency-based method DFF (27] preserves frequency information beneficial for generalization. GFNet [32 replaces self-attention layer with global frequency filter layer. ARP [7] introduces Amplitude- Phase Recombination via amplitude transformation. DAC 23 proposes a normalization method that removes style (amplitude) while preserving content (phase) through spectral decompo- sition. Although these methods improve generalization, they fall short in addressing large domain gaps. Our method requires no source domain training. adaptively masks harmful components for the target domain at the feature level. By treating amplitude and phase separately, exploit phase invariance to design a channel attention module that handles intra-class variations. shown in Table & our method outperforms existing trequency-based approaches on the CD-FSS task.

---

Table Compare our method previous frequency-basee methods under 1-shot setting.
Table 9: Compare our method other reducing
correlation approaches under 1 1-shot setting.
FSS
Deepglobe
ISIC
Chest
Average
FSS
Deepglobe
ISIC
Chest
Average
baseline 31
77.54
33.19
32.65
47.34
47.68
baseline
31
77.54
33.19
32.65
47.34
47.68
DFF 27
78.18
32.16
35.71
$1.39
MMCCSSmpple
30
34.32
48.74
GFNet
32
76.86
32.23
33.95
53,12
49.04
MMC(Oracle)
3D
77.45
35.12
34.59
50.27
49 19.36
ARP 7
78.83
35.06
35.61
59.83
52.33
SRIP 0
78.13
34.61
34.05
50.58
49 14
DAC
24
78.83
35,98
36.02
$7.66
51,98
wuuline-wwiitenngg
77.92
33.22
32.98
50.89
ours(APMM.M.
79.29
40.86
41.71
78.25
60.03
ours(APMM-M)
79.29
40.86
41.71
78.25
60.03
Correlation- method For methods that directly constrain the model (orthogonality, whiten- ing): the few-shot setting means limited sample size, and existing models have a large number of parameters. Directly adjusting the model with constraints using such small datasets is effective and even can lead to negative optimization. As seen in Table 9 the performance of orthogonality ConstraintsSSRIP (T) and whitening is not satisfactory. For feature transformation/augmeenttn methods like MMC [30]: the stability is not guaranteed because they use specific feature transforma- tion functions. Due to the domain gap. a transformation method effective for one domain may not be effective for others. In contrast, our method has the advantages of being 1) lightweight (allowing for quick adaptation in the few-shot setting) and stable and robust (with adaptive adjustments for different target domains). These benefits are well reflected in the performance results.
Related Work
Few-shot learning Few-shot learning aims to build robust representations for new concepts with limited annotated examples. Existing approaches typically fall into three categories: metric learning optimization- methods [14 333 and graph-base based methods [I5] 28 Recently, cross- domain few-shot learning has gained attention due to disparities in both data distribution and label space between meta-testing and meta-training stages. BSCD-FSL [16] introduces challenging benchmark for cross-domain few-shot learning, featuring substantial domain gap between the source and target domains. It covers several target domains with varying similarities natural images.
36 39
Few-shot semantic segmentation Few-shot semantic segmentation aims to segment unseen classes in query images with only few annotated samples. OSLSM [35] is the first two-branch FSS model. Following this, PL [IT] introduces a prototype learning paradigm utilizing cosine similarity between pixels and prototypes. SG-One [49] adopts masked average pooling (MAP) to optimize the extraction of support features. Recently, many FSS methods have emerged in the research community, such as RPMMs [43, PFENet (37), ASGNet (25), and HSNet (3I. HSNet employs efficient 4D convolutions on multi-level feature correlations, serving as the baseline for our work. However, these methods primarily address segmenting novel classes within the same domain and struggle with generalization across disparate domains due to significant feature distribution disparities. Bridging this substantial domain gap, particularly with limited labeled data, remains a formidable challenge.
Conclusion
In this paper, we delve into the phenomenon that filtering specific frequency components based on different domains significantly improves performance, providing an interpretation through experiment and mathematical derivation. Building our interpretation, we propose the APM, feature-level frequency component mask designed to enhance the generalization of feature map representations. Further, we introduced ACPA. Based on the APM-optimized feature map, the ACPA encourages the model to focus on more effective features while aligning the feature spaces the support and query. Experimental results demonstrate the approach's 's effectiveness in reducing domain gaps.
Acknowledgments
This work is supported by the National Natural Science Foundation of China under grants 62206 12206102. 102, 62436003, 62376103 and 62302184, the Science and Technology Support Program of Hubei Province under grant 2022BAA046; Hubei Science and Technology Talent Service Project under grant 2024DJC078; and Ant Group through CCF-Ant Research Fund.
10

---

References
[1] N Bansal, X Chen, and Z. Wang. Can we gain more from orthogonality regularizations in training deep cnns? arxiv 2018. arXiv preprint arXiv: 180.0010...
[2] Adrien Bardes, Jean Ponce, and Yann LeCun. Viereg: Variance-ivarrine--cvaaaaa regular- ization for self-supervised learning. arXiv preprint arXiv:2 2021.
[3] David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissec- tion: Quantifying interpretability of deep visual representations. In Proceedings of the IEFE conference on computer vision and pattern recognition, pages 6541-6549, 2017.
[4] Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information neural estimation. In International conference on machine learning, pages 531-540. PMLR, 2018.
[5] Malik Boudiaf, Hoel Kervadec, Ziko Imtiaz Masud, Pablo Piantamida, Ismail Ben Ayed, and Jose Dolz. Few-shot segmentation without meta-learning A good transductive inference is all you need? In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13979-13988, 2021.
[6] Sema Candemir, Stefan Jaeger. Kannappan Palaniappan, Jonathan P Musco, Rahul K Singh, Zhiyun Xue, Alexandros Karargyris, Sameer Antani, George Thoma, and Clement J McDonald. Lung segmentation in chest radiographs using anatomical atlases with nonrigid registration. IEEE transactions medical imaging, 33(2):577-590, 2013.
[7] Guangyao Chen, Peixi Peng, Li Ma, Jia Li, Lin Du, and Yonghong Tian. Amplitude-phas recombination: Rethinking robustness of convolutional neural networks in frequency domain. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 458-467, 2021.
[8] Liang- Ching Chen, George Papandreou. Iasonas Kokkinos, Kevin Murphy, and Alan Yuille. Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv preprint 1412. 7062, 2014.
[9] Noel Codella, Veronica Rotemberg, Philipp Tschandl, M Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Marchetti, et al. Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic). arXiv preprint arXiv: X 002.03368. 2019.
[10] like Demir, Krzysztof Koperski, David Lindenbaum, Guan Pang, Jing Huang, Saikat Basu, Forest Hughes, Devis Tuia, and Ramesh Raskar. Deepglobe 2018: A challenge to parse the earth through satellite images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 172-181, 2018.
[11] Nanqing Dong and Eric P Xing. Few-shot semantic segmentation with prototype learning. In BMVC, volume 3, 2018.
[12] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. /929, 2020.
[13] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88:303-338, 2010.
[14] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adap- tation of deep networks. In International conference on machine learning, pages 1126-111 1335. PMLR, 2017.
[15] Victor Garcia and Joan Bruna. Few-shot learning with graph neural networks. arXiv preprint 11Xiv:171..04043, 2017.

---

[16] Yunhui Guo, Noel CF Codella, Leonid Karlinsky, John R Smith, Tajana Rosing, and Rogerio Feris. A new benchmark for evaluation of cross-domain few-shot learning. arXiv preprint 1IXiv:/9/20072000 2019.
[17] Bharath Hariharan, Pablo Arbeláez, Lubomir Bourdev, Subhransu Maji, and Jitendra Malik. Semantic contours from inverse detectors. In 20/1 international conference on computer vision, pages 991-998. IEFE, 2011.
[18] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pultern recognition, pages 770-778, 2016.
[19] Jie Hu, Li Shen, and Gang Sun. Squeers-and-eecitatiion networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7132-7141, 2018.
[20] Stefan Jaeger, Alexandros Karargyris, Sema Candemir, Les Folio, Jenifer Siegelman, Fiona Callaghan, Zhiyun Xue, Kannappan Palaniappan, Rahul K Singh, Sameer Antani, et al. Auto- matic tuberculosis screening using chest radiographs. IEEE transactions on medical imaging, 33(2):233-245, 2013.
[21] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[22] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 4015-4026, 2023.
[23] Sangrok Lee, Jongseong Bac, and Ha Young Kim. Decompose, adjust, compose: Effective normalization by playing with frequency for domain generalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11776-11786 2023.
[24] Shuo Lei, Xuchao Zhang, Jianfeng He, Fanglan Chen, Bowen Du, and Chang- Tien Lu. Cross- domain few-shot semantic segmentation. In European Conference on Computer Vision, pages 73 90. Springer, 2022.
[25] Gen Li, Varun Jampani, Laura Sevilla-Lara, Deqing Sun, Jonghyun Kim, and Joongkyu Kim. Adaptive prototype learning and allocation for few-shot segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8334 8334-8343, 2021.
[26] Xiang Li, Tianhan Wei, Yau Pun Chen, Yu-Wing Tai, and Chi-Keung Tang. Fss-1000: A 1000-class dataset for few-shot segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2869-2878, 2020.
[27] Shiqi Lin, Zhizheng Zhang, Zhipeng Huang. Yan Lu, Cuiling Lan, Peng Chu, Quanzeng You, Jiang Wang, Zicheng Liu, Amey Parulkar, et al. Deep frequency filtering for domain generalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11797-11807, 2023.
[28) Yanbin Liu, Juho Lee, Minseop Park, Saehoon Kim, Eunho Yang, Sung Ju Hwang, and Yi Yang. Learning propagate labels: Transductive propagation network for few-shot learning. arXiv preprint arXiv: /805. 10002, 2018
[29] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for se- mantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3431-3440, 2015.
[30] Xu Luo, Jing Xu, and Zenglin Xu. Channel importance matters in few-shot image classification. In International conference on machine learning, pages 14542-14559. PMLR, 2022.
[31] Juhong Min, Dahyun Kang, and Minsu Cho. Hypercorrelation squeeze for few-shot segmen- tation. In Proceedings of the IEFF/CVF international conference on computer vision, pages 6941-6952, 2021.
12

---

[32] Yongming Rao, Wenliang Zhao, Zheng Zhu, Jiwen Lu, and Jie Zhou. Global filter networks for image classification. Advances in neural information processing systems, 34:980-999 99000099399000 2021.
[33] Sachin Ravi and Hugo Larochelle. Optimization as model for few-shot learning, In Interna- lional conference on learning representations, 2016.
[34] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 1 115:211-22 2015.
[35] Amirreza Shaban, Shray Bansal, Zhen Liu, Irfan Essa, and Byron Boots. One-shot learning for semantic segmentation. arXiv preprint arXiv: (XXiv:770.00344: 2017.
[36] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Advances in neural information processing systems, 30, 2017.
[37] Zhuotao Tian, Hengshuang Zhao, Michelle Shu, Zhicheng Yang. Ruiyu Li, and Jiaya Jia. Prior guided feature enrichment network for few-shot segmentation. IEEE transactions pattern analysis and machine intelligence, 44(2):1050-106, 2020.
[38] Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The of multi-source dermatoscopic images of common pigmented skin lesions. Scientific data, 5(1):1-9, 2018.
dataset, large collection
[39] Oriol Vinyals, Charles Blundell, Timothy Lillicrap. Daan Wierstra, et al. Matching networks for one shot learning. Advances in neural information processing systems, 29, 2016.
[40] Haohan Wang, Xindi Wu, Zeyi Huang, and Eric P Xing. High-frequency component helps explain the generalization of convolutional neural networks. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 8684 8694, 2020.
[41] Kaixin Wang, Jun Hao Liew, Yingtian Zou, Daquan Zhou, and Jiashi Feng. Panet: Few-shot image semantic segmentation with prototype alignment. In proceedings of the IEEE/CVF international conference computer vision, pages 9197-9206. 2019.
[42] Zhi-Qin John Xu, Yaoyu Zhang, Tao Luo, Yanyang Xiao, and Zheng Ma. Frequency principle: Fourier analysis sheds light on deep neural networks. arXiv preprint arXiv: 101.023901.06523, 2019.
[43] Boyu Yang, Chang Liu, Bohao Li, Jianbin Jiao, and Qixiang Ye. Prototype mixture models for few-shot semantic segmentation. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part VIII 16, pages 763-778. Springer, 2020.
[44] Yuhui Yuan, Xilin Chen, and Jingdong Wang. Object-contextual representations for semantic segmentation. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23 2020, Proceedings, Part VI 16, pages 173-190. Springer, 2020.
[45] Chi Zhang, Guosheng Lin, Fayao Liu, Jiushuang Guo, Qingyao Wu, and Rui Yao. Pyramid graph networks with connection attentions for region-baee one-shot semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9587-9595, 2019.
[46] Chi Zhang, Guosheng Lin, Fayao Liu, Rui Yao, and Chunhua Shen. Canet: Class-agnostic segmentation networks with iterative refinement and attentive few-shot leamiin. In Proceedings of the IEFF/CVF conference computer vision and pattern recognition, pages 5217-5226,
2019.
[47] Jian-We Wei Zhang, Yifan Sun, Yi Yang, and Wei Chen. Feature-proxy transformer for few-shot segmentation. Advances in neural information processing systems, 35:6575-6588 2022.
[48] Renrui Zhang, Zhengkai Jiang, Ziyu Guo, Shilin Yan, Junting Pan, Xianzheng Ma, Hao Dong, Peng Gao, and Hongsheng Li. Personalize segment anything model with one shot. arXiv preprint arXiv: 30300308,, 2023.
13

---

[49] Xiaolin Zhang, Yunchao Wei, Yi Yang. and Thomas S Huang. Sg-one: Similarity guidance network for one-shot semantic segmentation. IEEE transactions on cybernetics, 50(9):3855- 3865, 2020.
[50] Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2881-2890, 2017.
[51] Haichen Zhou, Yixiong Zou, Ruixuan Li, Yuhua Li, and Kui Xiao. Delve into base-novel confusion: Redundancy exploration for few-shot class-incremental learning. arXiv preprint arXiv:2405.04918, 2024.
[52] Yixiong Zou, Shanghang Zhang, Yuhua Li, and Ruixuan Li. Margin-based few-shot class- incremental learning with class-level overfitting mitigation. Advances in neural information processing systems, 35:27267-277 -72799, 2022.
14

---

presents an A Appendix supplemental material
A.1 More Dataset Details
support
support
query
support
FSS-1000
Deepglobe
ISIC2018
Chest X-ray
Figure 7: Examples of segmentation for four target datasets.
Our experimental setup is grounded in the benchmark established by PATNet 24 Fig. example segmentation for four target datasets. Further details follow:
335 extends the PASCAL VOC 2012 [[3] by integrating additional annotations from the SDS dataset [17]. Utilizing PASCAL-5' as source domain for training, we then evaluate the models' performance across four target datasets.
PASCAL-5'
FSS-1000 26 is few-shot segmentation dataset comprising 1000 natural image categories, with each category containing 10 samples. In our experiment, we adhere to the official split for semantic segmentation and report results on the designated testing set, which encompasses 240 classes and 2,400 images. We consider FSS- 1000 as our designated target domain testing.
Deepglobe 10) consists of satellite images annotated densely at the pixel level across 7 categories: urban, agriculture, rangeland, forest, water, barren, and unknown. Since ground- -ruth labels are available only in the training set, we utilize the official training dataset comprising 803 images to showcase our results. We designate our testing target domain and follow the same processing approach as PATNet.
ISIC2018 9 38 is designed specifically for skin cancer screening and comprises images of lesions, with each image depicting exactly one primary lesion. Adhering to the guidelines established by PATNet, we process and utilize the dataset, considering ISIC2018 as target domain for testing.
Chest X-ray 6 20 is tailored for Tuberculosis diagnosis, comprising 566 images with a resolution of 4020 x 4892 pixels. These images depict cases from 58 Tubereulosis patients and 80 individuals with normal conditions. common approach to handling large image sizes involves resizing them to 1024 1024 pixels.
A.2 APM Reduces Inter-Channel Correlation by Frequency Domain Mask
615
Figure 8: Histogram of phase differences (weighted by amplitude) between channels the feature maps before and after APM.
As shown in Figure we present histograms of the phase differences between channels in the feature maps (weighted by amplitude) before and after adding the APM module. After applying APM, the phase differences between channels concentrated around and are reduced, which aligns with our mathematical derivation in the main text. The APM decreases correlation between feature map channels by altering phase and amplitude, thereby enhancing the independence of their semantic representations. Additionally, due to the use of finer-grained frequency domain partitioning, APM performs well FSS- 1000 compared to the simple high-low frequency partitioning at the input level.
15

---

FSS-1000
Deepglobe
ISIC
ChesX
Amplitude
Phuse
Figure The visualization of the frequency components filtered by the masker.
A.3 Analyze the Frequency Components Filtered by the APM
The visualization results in Figure 9 show the average frequency components filtered by the amplitude masker and phase masker across different domains. The center represents low frequencies, while the periphery represents high frequencies. White indicates a value of I, meaning the frequency component passes through, and black indicates value of O, meaning the frequency component is filtered out. For FSS, the amplitude masker (AM) primarily filters mid-to-high ho--high-rreuency components, while the phase masker (PM) mainly retains mid- nid-frequency components. For DeepGlobe, both AM and PM retain more mid-to-high frequencies. For ISIC, AM filters out more mid-to-high frequencies, retaining low frequencies, whereas PM retains relatively more mid frequencies. For ChestX, AM mainly retains fow-frequency components, while PM filters out frequencies across the spectrum, retaining relatively more low-to-mid frequencies. These results align well with the patterns observed in Figure of our main text. is evident that for different targets, AM and PM dynamically and adaptively filter different frequency components, selecting those more beneficial for the current domain. Additionally, the advantageous amplitude and phase frequency components vary across different target domains, underscoring the necessity of considering amplitude and phase separately.
A.4 Detailed Ablation Study Results
In the main text, we demonstrate the effectiveness of our various designs by presenting the average mloU. Here, shown in Table 10 we provide detailed results on each target dataset.
Table 10: Detailed ablation study results of various designs (Backbone: ResNet-50).
FSS-1000
Deepglobe
X-ray
Average
APM-S
APM-M
ACPA
I-shot
5-shot
I-shot
S-shot
I-shot
S-shot
1-shot
5-shot
I-shot
5-shot
77.54
80.21
33.19
36,46
48.63
47.68
50.10
77.98
79.85
40.74
44.80
38.79
44.16
73.55
76.73
57.77
61.39
78.25
80.29
40.77
44.85
41.48
49.39
75.22
76.89
58.93
62.86
78.98
81.21
40.81
44.82
38.99
45.49
77.73
82.60
59.13
63.53
79.29
81.83
40.86
44.92
41.71
51.16
78.25
82.81
60.03
63.18
A.5 Broader Impact
Our research delves into the phenomenon that filtering specific frequency components based different domains significantly improves performance, providing an interpretation. Furthermore, we demonstrated the relationship between frequency components and inter-channel correlation through mathematical derivation. Building on our interpretation and derivation, we propose the APM, feature-level frequency component mask designed enhance the generalization of feature map representations. Further, we introduced Adaptive Channel Phase Attention (ACPA). Based on the APM-optimized feature map, the ACPA encourages the model focus on more effective features while aligning the feature spaces of the support and query. Experimental results demonstrate the effectiveness our approach significantly enhances the model's cross-domain transferability. This work is applicable not only to CDFSS but also to other areas like domain generalization and domain adaptation. Future research will aim broaden our evaluations to encompass wider range of target domains, enhancing understanding of their performance in various real-world scenarios.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our main claims made in the abstract and introduction accurately reflect the paper's 's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the work in the conclusion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those
discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution low images taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers grounds for rejection, worse outcome might be that reviewers discover limitations that aren't tt acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
17

---

We have provided the full set of assumptions and a complete (and correct) proof for each theoretical result.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have reported the detailed settings and hyper-parameters. We will release our codes.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of model checkpoint, other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-souree dataset or instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g.. to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
18

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We have reported the detailed settings and hyper-parameters. We will release our codes.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this central to contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed reproduce the results. See the NeurIPS code and data submission guidelines (https
to
onlolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized ver- sions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We have reported the detailed settings and hyper--arannn parameters. We will release our codes.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: Experiments are conducted based on the code released by HSNet (ICCV'21) and benchmark proposed by PATNet (ECCV'22) which specified the random seed for re-implementation and comparison. Therefore, we follow the same random seed to conduct experiments and compare with other methods without the error bar.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
19

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report I-sigma error bars, but one should state it. The authors should preferably report 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have reported the computer resources the implementation details.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http:://wwwuppp..cccpppbb///hiaa
Answer:
[Yes]
Justification:
We conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consideration due laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have discussed the broader impact in the appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
20

---

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairnsss considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to partieular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out, For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition to attacks, mecha- nisms for monitoring misuse, mechanisms to monitor how system learns from feedback time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Our work does not have the risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We have respected the license of assets used in this paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include URL..
The name of the license (e.g.. CC-BY 4.0) should be included for each asset.
For scraped data from particular source (e.g., website), the copyright and terms service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
21

---

For existing datasets that are packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not have new assets in this paper.
Guidelines:
The answer NA means that the paper does not release new assets. Researchers should communicate the details of the dataset/code/mooel as part of their sub- missions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not include human subjects information.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This paper does not include human subjects information.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country which research conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
22


Reviews:
[{"review_id": "wdAavjyCp4", "reviewer": "Reviewer_jSMF", "summary": "This paper makes several notable contributions to the field of cross-domain few-shot segmentation (CD-FSS). The authors discover that filtering different frequency components for target domains can lead to significant performance improvements, attributing this to reduced inter-channel correlation in feature maps, which enhances robustness against domain gaps and expands activated regions for segmentation. Building on this insight, they propose a lightweight frequency masker comprising an Amplitude-Phase-Masker (APM) module and an Adaptive Channel Phase Attention (ACPA) module. These components effectively reduce channel correlations and further enhance segmentation performance. The proposed method demonstrates significant advancements over current state-of-the-art CD-FSS approaches, highlighting its potential impact on the field.", "questions": "1. The authors claim that filtering certain frequency components can lead to significant performance improvements. How sensitive is this improvement to the specific frequency components chosen? Is there a systematic way to determine the optimal frequency components for a given domain?\n2. The paper introduces the Amplitude-Phase Masker (APM) module. How does the initialization of the APM affect the final performance? Have the authors explored different initialization strategies?\n3. The Adaptive Channel Phase Attention (ACPA) module uses phase information for attention weights. What is the rationale behind using only phase information rather than both phase and amplitude? How would the results change if amplitude information were incorporated?\n4. The paper claims that the proposed method reduces inter-channel correlation in feature maps. How does this reduction in correlation compare to other feature decorrelation methods in the literature, such as those based on orthogonality constraints or whitening?\n5. The authors use mutual information to measure inter-channel correlation. Are there other metrics that could provide additional insights into the nature of the feature disentanglement achieved by this method?", "limitations": "1. Computational overhead: The paper does not adequately address the computational cost of the proposed method. Frequency domain operations and the additional modules (APM and ACPA) likely introduce significant computational overhead, which should be quantified and compared to existing methods.\n2. Ablation studies: The paper would benefit from more comprehensive ablation studies. For instance, the individual contributions of APM and ACPA are not clearly delineated, and the impact of different design choices within these modules is not thoroughly explored.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This paper presents several notable advantages in the field of cross-domain few-shot segmentation (CD-FSS). It identifies a significant performance improvement by filtering different frequency components for target domains, which reduces inter-channel correlation in feature maps and enhances robustness against domain gaps. The proposed lightweight frequency masker, consisting of the Amplitude-Phase-Masker (APM) and Adaptive Channel Phase Attention (ACPA) modules, effectively reduces channel correlations and improves segmentation performance with minimal additional parameters. The authors also provide relevant mathematical derivations to support their findings. The method demonstrates substantial improvements over state-of-the-art CD-FSS methods, making it a significant contribution to the field.", "weaknesses": "- Performing frequency domain filtering on features is likely to result in some loss of information, potentially damaging the original structure of the features. Moreover, the mask weights required for different domains should vary. Are the authors training the APM on the source domain and then directly testing it on different target domains?\n- The novelty of the method is limited. The idea proposed by the authors is very similar to [1] and seems to merely apply cross-domain techniques to cross-domain few-shot segmentation.\n- The method proposed by the authors shows very limited improvement on some datasets and even performs worse than the existing state-of-the-art (SOTA) methods.\n\n[1] Deep Frequency Filtering for Domain Generalization  CVPR2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**1. Filtering on features damages the original feature structure?**\nFiltering certain frequency components does not damage the original feature structure; instead, it is beneficial. Since not all frequencies are advantageous for the current domain, we dynamically adjust the mask and take its inverse (1-mask). We find that performance decreases compared to the baseline, indicating that the frequencies we filter out are indeed detrimental components.\n|                     |  FSS  | Deep  | ISIC  | Chest | Ave.  |\n| :-----------------: | :---: | :---: | :---: | :---: | :---: |\n|      baseline       | 77.54 | 33.19 | 32.65 | 47.34 | 47.68 |\n|   APM (w/o ACPA)    | 78.98 | 40.81 | 38.99 | 77.73 | 58.86 |\n| Inv. APM (w/o ACPA) | 77.25 | 30.26 | 31.23 | 47.07 | 46.45 |\n\n**2. Sensitivity analysis on the choice of frequency components to filter**\nSince different domains require different weights, our method adapts directly to the target domain without the need for source-domain training. We visualized the average masker results for each domain to observe the filtered frequency components, as shown in the global response PDF. We found that the masker effectively adjusts to filter different frequency components according to different domains. \n\n**3. Differences between our approach and Deep Frequency Filtering for Domain Generalization (DFF):**\nWe compare with DFF in global response answer 1. Here we provide a more detailed explanation.\n\n1) Motivation: DFF aims to explore and retain frequency information beneficial for generalization during training, while filtering out frequencies that are not. However, we found that useful frequency information varies across different domains; frequencies beneficial to one domain may be harmful to others. Therefore, we focus on adaptively selecting beneficial information for different domains. \n\n2) Amplitude and Phase: DFF does not distinguish between amplitude and phase, using attention mechanisms to filter out non-generalizable frequency components during training. However, amplitude and phase play different roles: amplitude contains domain-specific information, while phase contains domain-invariant information. Our APM independently adjusts amplitude and phase, filtering out detrimental frequency information separately. ACPA leverages the domain-invariant characteristic of phase to reduce intra-class variance between support and query.\n\n3) Effectiveness: DFF performs well when input distributions differ but the label space remains the same. However, its effectiveness is limited in our task, where both input distributions and label spaces differ. We implemented DFF in our task, and our method demonstrated superior performance (see Q1 in global response).\n\n\n**4. Performance could be further improved after segmentation refinement**\nTo highlight the effectiveness of our method, we did not employ methods such as data augmentation, or segmentation refinement. We used the segmentation refinement of PANet [37], which led to further performance improvements.  Our method already surpasses the existing SOTA and shows significant improvement over the baseline, even without using any additional methods.\n|                    |  FSS   |        |  Deep  |        |  ISIC  |        | Chest  |        |  Avg   |        |\n| :----------------: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: |\n|       Method       | 1-shot | 5-shot | 1-shot | 5-shot | 1-shot | 5-shot | 1-shot | 5-shot | 1-shot | 5-shot |\n|      baseline      | 77.54  | 80.21  | 33.19  | 36.46  | 32.65  | 35.09  | 47.34  | 48.63  | 47.68  | 50.10  |\n| PATNet [22] (SOTA) | 78.59  | 81.23  | 37.89  | 42.97  | 41.16  | 53.58  | 66.61  | 70.20  | 56.06  | 61.99  |\n|       APM-M        | 79.29  | 81.83  | 40.86  | 44.92  | 41.71  | 51.16  | 78.25  | 82.81  | 60.03  | 65.18  |\n|    APM-M refine    | 80.02  | 82.35  | 41.23  | 45.57  | 42.56  | 53.69  | 78.76  | 83.22  | 60.64  | 66.21  |\n\n**5. The different initialization strategies of APM**\nWe presented this experiment in our response to question 2 for reviewer H2xr. Please refer to the AMP initialization experiments in question 2 for reviewer H2xr. We sincerely hope this could resolve your concerns. \n\n**6. Why does ACPA only use the phase information**\nPrevious interpretability studies have shown that phase is an invariant representation, while the amplitude varies between samples and contains specific information. To alleviate intra-class variations (such as viewing angles, transparency, and distances, which hinder the model's ability to recognize the same features accurately), we leverage the invariant nature of phase to align the feature spaces of support and query.\n|               | Ave. 1-shot | Ave. 5-shot |\n| :-----------: | :---------: | :---------: |\n|  + amplitude  |    58.32    |    63.25    |\n| w/o amplitude |    60.03    |    65.18    |\n\n**7. Compare with other methods for reducing correlation/Compare with other frequency-based methods/More detailed about the individual contributions of APM and ACPA**\nWe answered these questions in the global response. We sincerely hope this could resolve your concerns.\n\n**8. Other metrics validate that our method achieves feature disentanglement**\nWe normalize the feature map channels with L2 normalization and then compute the L1 norm to measure their sparsity. A smaller value indicates higher sparsity. After masking certain frequency components, the sparsity value decreases, indicating sparser features. Sparse features imply lower feature redundancy, which benefits feature disentanglement and thereby enhances the model’s generalization capability. \n| sparsity |  FSS  | Deep  | ISIC  | ChestX |\n| -------- | :---: | :---: | :---: | :----: |\n| baseline | 31.85 | 32.41 | 32.4  | 31.86  |\n| APM      | 31.12 | 31.79 | 31.08 |  30.5  |\n\n**9. The complexity analysis**\nWe answer this question in the reviewer upoB's question 2. We sincerely hope this could resolve your concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZK4qRE2RzQ", "reviewer": "Reviewer_79M1", "summary": "This paper presents a novel approach to cross-domain few-shot semantic segmentation (CD-FSS) by introducing a lightweight frequency masker. This masker aims to enhance the robustness of models against domain gaps by filtering different frequency components during the testing phase. The authors claim that their method significantly improves performance, sometimes by as much as 14%, without the need for extensive retraining or parameter tuning.", "questions": "Could other frequency-based works also achieve the correlation reduction?", "limitations": "An important part of the analysis is the correlation reduction. However, the paper only includes a comparison with the reduction method by mutual information. Many other methods can also achieve this goal. In my opinion, the mathematical deduction only proves that the frequency operation is able to reduce the correlation, but does not show its advantages in such reduction. Therefore, I would like to see the author provide more comparison and analysis regarding this problem, such as directly comparing with [27].", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This paper introduces a novel frequency masker that is lightweight and does not require training on the source domain.\nThe authors provide a clear explanation of the phenomenon where frequency filtering improves performance, supported by mathematical derivations and experiments.\nThis paper demonstrates significant performance improvements on multiple target datasets, which is a strong empirical contribution.\nThe proposed APM and ACPA modules are innovative and show promise in addressing the domain gap problem in few-shot segmentation.", "weaknesses": "While the paper claims to reduce inter-channel correlation, why not directly constrain the model to reduce such correlation? How is this work compared with [27]?\nThe paper could benefit from a more thorough comparison with state-of-the-art methods, particularly those that also employ frequency domain techniques.\nThe authors might consider providing more details on the experimental setup, including data preprocessing and model training procedures, to ensure reproducibility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**1. Compare with directly constraining the model**\n\nWe answered this question in the global response. We sincerely hope this could resolve your concerns.\n\n**2. Comparing with [27]Channel Importance Matters in Few-Shot Image Classification**\n\nWe compared with [27] in the global response, and here we provide a more detailed explanation.\n\nWe implemented the two transformation methods from [27] under our task setting. The performance slightly declines on the FSS dataset, which is similar to the source domain. However, on the other three datasets that are more distant from the source domain, there is a slight performance improvement. Nonetheless, our method has an advantage in terms of performance, with improvements significantly surpassing those of [27].\n\nWe further elaborate on the differences between our work and [27]:\n\n1)[27] found that different channels recognize different patterns, and the channel bias present between channels can affect the model's recognition ability. They improved performance by eliminating this channel bias through feature transformation functions. In contrast, our work posits that different frequency components play different roles in different domains. We dynamically filter out detrimental frequency components based on the domain, thereby reducing channel correlation and improving performance.\n\n2)Compared to [27]'s spatial operations, our frequency operations have the advantage of better representing global information. A spatial position in the frequency domain represents information from the entire spatial domain, giving frequency domain operations a natural advantage in capturing long-term dependencies and maintaining a large receptive field. Additionally, compared to feature transformation and convolution operations in the spatial domain, frequency domain operations remove inductive biases.\n\n**4. Compare with other frequency-based methods**\n\nWe answered this question in the global response. We hope this could resolve your concerns.\n\n**5. Could other frequency-based works also achieve the correlation reduction**\n\nWe tested the MI of the aforementioned frequency-based methods, and not all of them were able to achieve correlation reduction. They achieved correlation reduction in certain domains because they are used during training to enhance the model's generalization. However, due to the domain gap, feature extraction patterns that perform well on the source domain may not benefit the target domain. For example, DFF explores frequency components beneficial for generalization during training but its results show it filters out a lot of high frequencies and retains low frequencies. Therefore, its performance improvement might be due to filtering out noise in high frequencies. We visualized how our masker filters frequency components across different domains (the global rebuttal PDF displays these results). The frequency components to be filtered differ among target domains. Additionally, phase and amplitude need to be considered separately rather than being treated as a single entity. Hence, not all frequency-based methods can achieve correlation reduction.\n\n| MI       |  FSS   |  Deep  |  ISIC  | ChestX |\n| -------- | :----: | :----: | :----: | :----: |\n| baseline | 1.3736 | 1.3679 | 1.3789 | 1.3952 |\n| DFF      | 1.3742 | 1.3701 | 1.3702 | 1.3429 |\n| GFN      | 1.384  | 1.3682 | 1.3781 | 1.3605 |\n| ARP-SP   | 1.3705 | 1.3568 | 1.3713 | 1.3488 |\n| DAC-SC   | 1.3722 | 1.3557 | 1.3676 | 1.3526 |\n| ours     | 1.3501 | 1.2761 | 1.3139 | 1.2610 |\n\n**6. More details on the experimental setup**\n\nHere, we provide a more detailed explanation of our data processing method. We adopt the same setup and data processing as PATNet[22]. For FSS-1000, the official split for semantic segmentation is used in our experiment. We report the results on the official testing set, which contains 240 classes and 2,400 testing images. For Deepglobe, the images have a fixed resolution of 2448 × 2448 pixels. To increase the number of testing images and reduce their size, each image was cut into 6 pieces. This cutting has minimal effect on segmentation due to the irregular shapes of the categories. After filtering out single-class images and the 'unknown' class, we obtained 5,666 images, each with a resolution of 408 × 408 pixels, for reporting the results. For ISIC, the images have a spatial resolution around 1022 × 767. We down-size the images to 512 × 512 pixels. For Chest X-ray,  due to the large size of the image, we down-size the images to 1024 × 1024 pixels.\n\n**7. Why is frequency operation more advantageous in reducing correlation**\n\nThe aforementioned (global response, answer2) orthogonality constraints, whitening, MMC, and MI Loss (discussed in the main text) all use spatial operations to reduce correlation. Here, we elaborate on the advantages of frequency operations compared to spatial operations.\n\n1)The frequency domain inherently offers finer granularity compared to the spatial domain, facilitating more precise feature disentanglement. When a spatial domain channel (feature) is transformed into the frequency domain, each point in the frequency domain represents the global information of the feature. This results in a finer granularity transformation from 1 to hw points.\n\n2)The frequency domain inherently provides a more lightweight operation compared to the spatial domain. A simple multiplication in the frequency domain can be equivalent to multiple convolutions in the spatial domain. This makes modules operating in the frequency domain more lightweight, easier to adapt to different domains, and more advantageous when data is scarce.\n\n3)The frequency domain inherently has a larger receptive field and better helps capture long-term dependencies, making it more effective for learning global information. This enables operations in the frequency domain to capture more independent channel patterns, leading to expanded activation regions and more generalized representations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lIlCewZk4N", "reviewer": "Reviewer_H2xr", "summary": "This paper discover a phenomenon that simpy filtering different frequency components for target domains can lead to a significant performance improvements. Then the paper delve into this phenomenon for an interpretation, and propose an approach based on this phenomenon, which achieves futher performance improvements. The proposed method includes an amplitude-phase-masker (APM) module and an Adaptive Channel Phase Attention (ACPA) module, which are lightweight but effective as validated by experiments.", "questions": "1. How this method can help other tasks such as cross domain few-shot learning, domain generalization, or few-shot object detection?\n2. How is this work compared to other works applying frequency operation?", "limitations": "This paper addresses the CDFSS task from the aspect of frequency analysis, however, solely the frequecy analysis is not an novel aspect. This paper lacks discussion about the difference with previous works regarding the frequency analysis. But in all, I still recognize the novelty and contribution of this paper.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper identifies an intriguing phenomenon where frequency filtering leads to performance gains in CD-FSS, which is a novel contribution to the field.\n2. The proposed lightweight frequency masker introduces minimal additional parameters (0.01%) yet achieves significant performance improvements (over 10% on average), which is a strong practical contribution.\n3. The paper includes extensive experiments on four target datasets, demonstrating the effectiveness of the proposed method.", "weaknesses": "1. Can this method be applied to other domains or tasks such as Cross domain few shot learning, domain generalization?\n2. It would be helpful to include a sensitivity analysis on the choice of frequency components to filter, to understand the robustness of the method to different filtering strategies.\n3. Some existing methods such as GFNet also applied filtering on the frequency domain, it would be better to compare with these methods, both in the related work and in the experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**1. Our method can be applied to other tasks**\n\nOur method can also be applied to cross-domain few-shot learning (CDFSL). Following BSCD-FSL[1]  we implemented our method under this task setting (5-way 1-shot), and experimental results show that our method is effective in CDFSL as well.\n\n|                 | CropDisease | EuroSAT | ISIC  | ChestX | Ave.  |\n| --------------- | :---------: | :-----: | :---: | :----: | :---: |\n| baseline [1]    |    73.39    |  66.12  | 35.07 | 21.98  | 49.14 |\n| baseline + ours |    82.01    |  68.95  | 38.86 | 24.07  | 53.47 |\n\n[1] A Broader Study of Cross-Domain Few-Shot Learning\n\n**2. Sensitivity analysis on the choice of frequency components to filter**\n\nFirst, we visualized the average masker results for each domain to observe the filtered frequency components, as shown in the global rebuttal PDF. We found that the masker effectively adjusts to filter different frequency components according to different domains. \n\nThen, we validated the robustness of APM by adding gaussian noise during its adaptive process. Even with the added noise, APM could still dynamically adjust and filter out the frequency components detrimental to the current domain, demonstrating its robustness.\n\n|             |  FSS  | Deep  | ISIC  | Chest | Ave.  |\n| :---------: | :---: | :---: | :---: | :---: | :---: |\n|  baseline   | 77.54 | 33.19 | 32.65 | 47.34 | 47.68 |\n|     APM     | 79.29 | 40.86 | 41.71 | 78.25 | 60.03 |\n| APM + noise | 79.03 | 40.06 | 40.82 | 77.92 | 59.46 |\n\n**APM's initialization** We also explored different initialization strategies for APM. A value of 0 means no frequency components pass through, while a value of 1 means all frequency components pass through. \"Rand\" indicates random values uniformly distributed in [0,1], \"gauss\" indicates values drawn from a normal distribution, \"clamp\" indicates values clipped to [0,1], and \"line\" indicates values scaled linearly to [0,1]. The experimental results show that our APM is robust, quickly adjusting and adapting even with an initial value of all zeros. Our default initialization strategy is all ones, meaning all frequency components pass through initially, which also facilitates the dynamic adjustment and learning of APM.\n\n|                  |  FSS  | Deep  | ISIC  | Chest | Ave.  |\n| :--------------: | :---: | :---: | :---: | :---: | :---: |\n|     baseline     | 77.54 | 33.19 | 32.65 | 47.34 | 47.68 |\n| **one (choose)** | 79.29 | 40.86 | 41.71 | 78.25 | 60.03 |\n|       zero       | 76.7  | 35.32 | 40.63 | 76.09 | 57.19 |\n|       rand       | 78.93 | 40.74 | 41.49 | 77.56 | 59.68 |\n|  gauss (clamp)   | 78.26 | 39.43 | 41.38 | 76.89 | 58.99 |\n|   gauss (line)   | 78.82 | 40.46 | 41.54 | 77.85 | 59.67 |\n\n\n**3. Compare with other frequency-based methods**\n\nWe answered this question in the global response. We sincerely hope this could resolve your concerns.\n\nHere, we provide a more detailed explanation of the differences between our work and GFNet (experimental results are in the global rebuttal table):\n\n1) The motivation of GFNet is to use global frequency filters to replace self-attention or MLPs, reducing computational overhead while removing inductive biases and maintaining a large receptive field (which helps capture long-term dependencies). This is reasonable because a spatial location in the frequency domain represents global information. In contrast, our work is motivated by the observation that different frequency components play different roles in different domains; a frequency component beneficial in domain A might be harmful in domain B. Therefore, we designed an adaptive masker to dynamically filter different frequency components according to different domains. We also explored and validated the relationship between frequency and feature correlation.\n\n2) GFNet's \"filter\" refers to a convolutional filter, which can be seen as a stack of multiple convolution operations (a multiplication operation in the frequency domain can be replaced by multiple convolution operations in the spatial domain), with values in the range (-∞, +∞). In contrast, our masker is used to filter frequency components, with values in the range [0,1].", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Y1kWcgtqcN", "reviewer": "Reviewer_upoB", "summary": "The paper presents a novel approach to cross-domain few-shot semantic segmentation (CD-FSS) by introducing a lightweight frequency masker that aims to improve performance by filtering different frequency components for target domains. The authors propose an amplitude-phase-masker (APM) module and an adaptive channel phase attention (ACPA) module to reduce inter-channel correlations and enhance feature robustness against domain gaps.", "questions": "NA", "limitations": "NA", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper addresses a relevant and challenging problem in the field of few-shot semantic segmentation, particularly in cross-domain scenarios where performance typically suffers due to domain shifts.\n2. The proposed lightweight frequency masker, including the APM and ACPA modules, introduces a novel perspective on feature disentanglement in the frequency domain, which is a promising direction for improving generalization across domains.\n3. The authors provide a thorough interpretation of the phenomenon of frequency filtering and its impact on feature channel correlations, which is well-supported by mathematical derivations and empirical evidence.\n4. The paper includes extensive experiments on four target datasets, demonstrating the effectiveness of the proposed method in reducing domain gaps and improving segmentation performance.", "weaknesses": "1. The novelty of the approach among existing work could be better established. It would be better to give a more detailed comparison with state-of-the-art methods that also attempt to address domain shifts in few-shot segmentation, especially those through the frequency operations.\n2. The paper does not discuss the computational efficiency of the proposed method, which is an important consideration for practical applications. It would be beneficial to include details on the runtime and resource requirements of the approach.\n3. Since overfitting is an important issue in the few-shot finetuning, I wonder how this method could benefit the model in reducing the overfitting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**1. Compare with other frequency-based methods**\n\nWe answered this question in the global response.  We hope this could resolves your concerns.\n\n**2. The complexity analysis**\n\nWe present the results of the complexity analysis, showing that our APM and ACPA are extremely lightweight, modules with minimal parameters and computational overhead. Our experiments were conducted on a single 4090 GPU.  We also compared it with a lightweight frequency-based method (DFF [1]), which further highlights our advantages in terms of computational overhead and parameters.\n\n|           | baseline (encoder + decoder) | APM-S | APM-M | ACPA  | DFF [1] |\n| :-------: | :--------------------------: | :---: | :---: | :---: | :-----: |\n| Params(K) |     26174 (23600 + 2574)     | 0.338 |  692  | 65.54 |  2100   |\n\n|           | baseline | ours (APM-S) | ours (APM-M) | DFF [1] |\n| :-------: | :------: | :----------: | :----------: | :-----: |\n| FLOPs (G) |  20.11   |    20.17     |    20.26     |  22.07  |\n\n[1] Deep Frequency Filtering for Domain Generalization, CVPR2023\n\n**3. Our method could benefit the model in reducing the overfitting**\n\nWe believe that overfitting in few-shot fine-tuning arises from two main reasons: 1) The extremely limited samples (few-shot) prevent the model from fully learning each feature, making it prone to fitting extreme features (noise) and overly relying on feature correlations (e.g., if the samples are red apples, the model binds red color and round shape, and fails if the test sample is a green apple). 2) Intra-class variations (such as viewing angles, transparency, and distances) hinder the model's ability to recognize the same features accurately.\n\nOur APM addresses the first issue by reducing the correlation between features and eliminating channel bias (eliminating extreme features). ACPA tackles the second issue by leveraging the phase's invariant information to minimize intra-class variations. Consequently, our approach effectively mitigates overfitting in few-shot fine-tuning.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
**1. Compare with other frequency-based methods**

Here, we elaborate on the differences between our work and previous frequency-based methods.

DFF [1] explores and retains frequency information beneficial for generalization during training while filtering out frequencies that are not. GFNet [2] uses global frequency filters to replace self-attention or MLPs, reducing computational overhead while maintaining a large receptive field. ARP [3] proposes that a robust CNN should be resilient to amplitude variance and focus on the phase spectrum, thus introducing the Amplitude-Phase Recombination data augmentation method. DAC [4] proposes a novel normalization method, which eliminates style (amplitude) only as the preserving content (phase) through spectral decomposition. Although all these methods enhance the model's generalization ability, they do not effectively bridge large domain gaps.

Our motivation stems from the observation that filtering certain frequency components can significantly improve performance, while different frequency components have varying effects on different domains due to domain gaps. We delved into this phenomenon and discovered that operations in the frequency domain can reduce the correlation between channels,  achieving feature disentangling. Therefore, our method does not require training on the source domain. Instead, it adaptively masks components that are detrimental to the current target domain (feature level). Additionally, we independently consider amplitude and phase rather than treating them as a whole, and we leverage the invariant characteristics of phase to design a channel attention module that addresses intra-class variations. Experimental results demonstrate that our method outperforms existing frequency-based methods in the CDFSS task.
|            |    FSS    |   Deep    |   ISIC    |   Chest   |   Ave.    |
| ---------- | :-------: | :-------: | :-------: | :-------: | :-------: |
| baseline   |   77.54   |   33.19   |   32.65   |   47.34   |   47.68   |
| DFF [1]    |   78.18   |   32.16   |   35.71   |   60.29   |   51.59   |
| GFNet [2]  |   76.86   |   32.23   |   33.95   |   53.12   |   49.04   |
| ARP-SP [3] |   78.83   |   35.06   |   35.61   |   59.83   |   52.33   |
| DAC-SC [4] |   78.27   |   35.98   |   36.02   |   57.66   |   51.98   |
| ours       | **79.29** | **40.86** | **41.71** | **78.25** | **60.03** |

[1] Deep Frequency Filtering for Domain Generalization, CVPR2023

[2] Global Filter Networks for Image Classification, NeruIPS2021

[3] Amplitude-Phase Recombination: Rethinking Robustness of Convolutional Neural Networks in Frequency Domain, ICCV2021

[4] Decompose, Adjust, Compose: Effective Normalization by Playing with Frequency for Domain Generalizatio, CVPR2023

**2. Compare with other methods of reducing correlation**

In the main text, we compared our method with MI Loss. Here, we further provide comparisons with orthogonality constraints, whitening, and MMC [27]. 

For methods that directly constrain the model (orthogonality, whitening): the few-shot setting means limited sample size, and existing models have a large number of parameters. Directly adjusting the model with constraints using such small datasets is not effective and, without careful tuning of hyperparameters, can lead to negative optimization. As seen, the performance of orthogonality constraints and whitening methods is not satisfactory.

For feature transformation/augmentation methods like MMC: the stability is not guaranteed because they use specific feature transformation functions. Due to the domain gap, a transformation method effective for one domain may not be effective for others. For example, MMC's performance on the FSS dataset not only failed to improve but declined. The MMC paper also mentioned that this method might experience performance degradation on certain datasets.

In contrast, our method has the advantages of being 1) lightweight (allowing for quick adaptation in a few-shot setting) and 2) stable and robust (with adaptive adjustments for different domains). These benefits are well reflected in the performance results.

| MIoU                 |  FSS  | Deep  | ISIC  | Chest |  Avg  |
| -------------------- | :---: | :---: | :---: | :---: | :---: |
| baseline             | 77.54 | 33.19 | 32.65 | 47.34 | 47.68 |
| MMC (Simple) [27]    | 77.48 | 34.70 | 34.32 | 48.74 | 48.81 |
| MMC (Oracle) [27]    | 77.45 | 35.12 | 34.59 | 50.27 | 49.36 |
| baseline + orthogonality [1]  | 78.13 | 34.61 | 34.05 | 50.58 | 49.34 |
| baseline + whitening | 77.92 | 33.22 | 32.98 | 50.89 | 48.75 |
| ours                 | **79.29** | **40.86** | **41.71** | **78.25** | **60.03** |

| MI                   |  FSS   |  Deep  |  ISIC  | Chest  |
| -------------------- | :----: | :----: | :----: | :----: |
| baseline             | 1.3736 | 1.3679 | 1.3789 | 1.3952 |
| MMC (Simple) [27]    | 1.3742 | 1.3601 | 1.3782 | 1.3629 |
| MMC (Oracle) [27]    | 1.3740 | 1.3582 | 1.3751 | 1.3605 |
| baseline + orth [1]  | 1.3695 | 1.3611 | 1.3758 | 1.3590 |
| baseline + whitening | 1.3702 | 1.3668 | 1.3783 | 1.3577 |
| ours                 | **1.3501** | **1.2761** | **1.3139** | **1.2610** |

[1] Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?

[27] Channel Importance Matters in Few-Shot Image Classification

**3. More details about the individual contributions of APM and ACPA**

APM filters out negative frequency components at the feature level within feature maps. It leads to a feature map that is more robust, generalizable, and provides broader and more accurate representations. Adaptive Channel Phase Attention (ACPA) can be seen as a process of feature selection. Building on the APM-optimized feature map, ACPA encourages the model to focus on more effective channels (features) while aligning the feature spaces of the support and query samples.

Meta-review: In this paper, the authors presented a method for cross-domain few-shot semantic segmentation. Based on an observation that filtering different frequency components in target domains can lead to performance gain, two modules -- amplitude-phase-masker (APM) and adaptive channel phase attention (ACPA), were proposed to reduce channel correlations. The key contributions of this paper are the observed phenomenon of frequency filtering with its impact on feature channel correlations and the proposed lightweight frequency masker, which could be of interest to the audience of NeurIPS and potentially inspire follow-up research.

The paper received consistent positive recommendations from four expert reviewers. All the reviewers acknowledged the above-mentioned contributions and the corresponding significant performance improvement, and believe it could be a good contribution to the field.
There were some concerns around the detailed design of the filtering method, novelty (compared to related works), and experiments. Through the rebuttal, these major concerns were addressed with some additional provided evidence.

Considering the above and the novelty and soundness requirements for NeurIPS papers, the AC recommends Accept for this paper, but the authors are reminded to include the further evidence provided in the rebuttal (to address those concerns) into their final version.
Decision: Accept (poster)