Random Registers for Cross-Domain Few-Shot Learning
Authors: Shuai Yi
Yixiong Zou
Yuhua Li
Ruixuan Li
Venue: ICML 2025 poster (2025)
Keywords: Cross-Domain Few-Shot Learning
Primary area: general_machine_learning->transfer_multitask_and_metalearning
TL;DR: We find prompt tuning, as a common way to train ViT, could consistently decrease the performance on target domains, while random noises can lead to an increase; we delve into this phenomenon for an interpretation, and propose a method based on it.

Abstract: Cross-domain few-shot learning (CDFSL) aims to transfer knowledge from a data-sufficient source domain to data-scarce target domains. Although Vision Transformer (ViT) has shown superior capability in many vision tasks, its transferability against huge domain gaps in CDFSL is still under-explored. In this paper, we find an intriguing phenomenon: during the source-domain training, prompt tuning, as a common way to train ViT, could be harmful for the generalization of ViT in target domains, but setting them to random noises (i.e., random registers) could consistently improve target-domain performance. We then delve into this phenomenon for an interpretation. We find that learnable prompts capture domain information during the training on the source dataset, which views irrelevant visual patterns as vital cues for recognition. This can be viewed as a kind of overfitting and increases the sharpness of the loss landscapes. In contrast, random registers are essentially a novel way of perturbing attention for the sharpness-aware minimization, which helps the model find a flattened minimum in loss landscapes, increasing the transferability. Based on this phenomenon and interpretation, we further propose a simple but effective approach for CDFSL to enhance the perturbation on attention maps by adding random registers on the semantic regions of image tokens, improving the effectiveness and efficiency of random registers. Extensive experiments on four benchmarks validate our rationale and state-of-the-art performance. Codes and models are available at https://github.com/shuaiyi308/REAP.

Random Registers for Cross-Domain Few-Shot Learning
Shuai !
Yixiong Zou
Yuhua Li
Ruixuan Li
Abstract
VIT Backbome
Cross-domain few-shot learning (CDFSL) aims to transfer knowledge from a data-sufficient source domain to data-scarce target domains. Although Vision Transformer (ViT) has shown superior ca- pability in many vision tasks, its transferability against huge domain gaps in CDFSL is still under- explored. In this paper, we find an intriguing phenomenon: during the source-domain train- ing, prompt tuning, as a common way to train VIT, could be harmful for the generalization of VIT in target domains, but setting them ran- dom noises (i.e., random registers) could consis- tently improve target-domain performance. We then delve into this phenomenon for interpre- tation. We find that learnable prompts capture domain information during the training on the source dataset, which views irrelevant visual pat- terns as vital cues for recognition. This can be viewed as kind of overfitting and increases the sharpness of the loss landscapes. In contrast, dom registers essentially novel way of per- turbing attention for the sharpness-aware mini- mization, which helps the model find a flattened minimum in loss landscapes, increasing the trans- ferability. Based on this phenomenon and inter- pretation, we further propose simple but effec- tive approach for CDFSL to enhance the perturba- tion attention maps by adding random registers the semantic regions of image tokens, improv- ing the effectiveness and efficiency of random registers. Extensive experiments on four bench- marks validate our rationale and state-of-the-art performance. Codes and models are available https:/ggitub..co///huaiiig
(source-domian phase torget-domatnn phase)
Prompti
(a) Adding Learnalle Prompts
VIT Backbone
(sowree-domianiphase only)
Frompts
(b) Adding Random Prompts
65.2%
64.6%
Ave. Accurncy
64.0%
Learrable (Staliow)
Lasrablee Prompts (Derp)
Randon Prompis (Shallow)
Prompts (Deep)
16
128 256 512 1024
Prompt Nurbber
(c) Prompt Number vs. TarTaa-ttDDmaiii Performance
Figure f. (a) Vision Transformer (ViT) takes the CLS token, image tokens, and learnable prompts as input for both the source-domain training and the target-domaii testing. (b) We replace the Iearnable prompts with random noises (i.e., random prompts) on source domain, which are then dropped on the target-domain phase. (c) We find an intriguing phenomenon: unlike other tasks, prompts learned on the source domain harm the target-domain performance, while random prompts (both deep and shallow types) improve
1. Introduction
Large models achieve great success in many tasks due to their power learning from large-scale datasets (Zou et al., 2022: Chen al., 2023b), with large vision models majorly taking the VIT-based architectures (Zhang et al., 2022a; Noori al., 2024; Zou et al., 2024b). However, the general- ization of ViT under the extreme cross-domain data-scares scenarios still needs to be explored since collecting suffi- cient data for every domain does not always hold in the real world (Tseng et al., 2020; Naseer et al., 2021). I). To this end, Coos--Domain Few-Shot Learning (CDFSL) has been proposed to transfer general knowledge from the source
"Scooo Computer Science and Technology, Huazhong Uni- versity of Science and Technology. Wuhan, China School of Artifi- cial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China. Correspondence to: Yixiong Zou <yixiongz @ hust. edu.en>>, Yuhua <ideliyuhua @ thust.edu.ccn.. Ruixuan Li <rxli @ www.t.ddu.c.>...........
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Random Registers for
Few-Shot Learning
domain (e.g., ImageNet (Krizhevsky al., 2017)) with numerous nature images to target domains (e.g., medical datasets (Codella et al., 2019)) with only a few labeled ex- amples (Guo et al., 2020; Zou et al., 2021). Huge domain gaps between source and target domains make it difficult to transfer source-domain-tranl -rainned-tt VIT to target domains for few-shot learning (Zou et al., 2024a).
replacing clustered image patches with random noises. This operation increases the ratio of perturbed information in the attention maps, which increases the efficiency of random registers. During the target-domain phase, we maintain all tokens as input and append learnable registers as prompts for finetuning to take advantage of their absorption of do- main information. Experiments on four datasets validate our rationale for the interpretations and prove that we can outperform state-of-the-an works.
To handle this task, we focus on a common way of training the VIT model: the Visual Prompt Tuning (Jia et al., 2022) method (Fig. la). which concatenates additional learnable prompts to the input sequence of ViT. We train the ViT model on the source-domain dataset with learnable prompts and evaluate the target-domain performance through the prototype-based method (Snell et al., 2017). However, we find such a common way of ViT training causes signifi- cant performance degradation on the target domain datasets (Fig. lc). Instead, interestingly, we directly abandon the learning of prompts and randomize these prompts with Gaus- sian noises (Fig. lb), and find this operation would consis- tently improve target-domain performance when increas- ing the number of prompts, for both the shallow and deep prompts. Surprisingly, the highest performance is achieved when the prompt number reaches the maximum that the GPU memory can hold.
In summary, our contributions can be listed as follows.
To the best of our knowledge, we are the first to find that prompt learning on the source domain harms the transferability target domains, but utilizing random registers would consistently improve it.
We delve into this phenomenon for an interpretation: the learnable registers absorb domain information, rep- resented as the focus on regions irrelevant to recogni- tion, while random registers novelly perturb attention maps for sharpness-awan minimization.
Based on the interpretation, we propose a novel method to enhance perturbations on attention maps, which adds random registers to semantic regions of image tokens, therefore improving the effectiveness and efficiency of random registers and enhancing model transferability.
In this paper, we delve into this phenomenon for an interpre- tation. As shallow prompts show higher performance and are simpler, we mainly target this kind of prompt, and we call it Register following (Darcet al., 2024). We first find attention maps on target domains always fail to find seman- tic objects, indicating a poorly transferred attention network of ViT. Then, we measure the transferability of attentions through the sharpness of loss landscapes (Foret et al., 2021; Zou et al., 2024a) against the perturbation on attentions, and quantitatively verify learnable registers decrease the transferability while random registers increase it. Inspired by this, we interpret random registers as a novel way of perturbing attention to conducting the sharpness-aware minimization (Foret et al., 2021), therefore improving the transferred attention on target domains. In contrast, we find learnable registers capture domain information during the training on source datasets, represented as viewing irrele- vant visual patterns (e.g., background) as important cues for recognition. This can be viewed as kind of overfitting to the source dataset, thereby increasing the sharpness.
Extensive experiments on four benchmark datasets val- idate our rationale and state-of-the-ar performance.
2. Delve into the Registers in ViT-based Cross-Domain Few-Shot Learning
2.1. Preliminaries
Cross-Domain Few-Shot Learning (CDFSL) requires the model to learn from a source-domain dataset with sufficient training samples, and then transfer to the downstream tasks, aiming to recognize the target-domain datasets by only few training data (Wang & Deng, 2021; Li et al., 2022). Specifi- cally, the source-domain and the target-domain datasets are denoted D = [1 $ N and DJ = -17.0771 1 with class labels y E CT respec- tively. Source and target classes disjoint: C NcT 0. and large domain gaps exist between them. During the train- ing on source domain DS, the goal is to train the whole model by minimizing the cross-entropy loss
J==1 with class labels y C.S.
Based on the above interpretations, we further propose simple but effective method to boost ViT's transferability to target domains, by improving the effectiveness and effi- ciency of random registers (so that the number of appended random registers can be small). During the source-domain phase, since the core of the random registers is perturbing at- tention maps by prompts (tokens), we add random registers to semantic regions of the input image tokens, by randomly
N Σ Lote (.
L
(I)
where f()) denotes a feature extractor (c.g., ViT) and d(.) denotes fully connected (FC) layer (i.e., classifter). Then
2

---

Random Registers for Cross-Domain Few-Shot Learning
Shumpe
24 TEAP
Starpooes
Stageess?
Weight
Portushation Level
Model weights Represcutation
(b) Perturbation vs. Shargness
Figure 3. (a) When the data is shifted from the training data, the originally effective weights or representations (i.e., minima in the loss landscape) may not be in low loss (blue point), whereas the increase of loss (i.c., sharpness) measures the vulnerability to domain shifts. (b) The model with Icarned registers consistently shows higher sharpness than others, indicating lower robustness to domain shifts. In contrast, the lower sharpness random registers verifies that the attention is more transferable across domains.
Image
Baseline
Learnable Registers
Random Registers
Figure 2. Visualization of the model's 's attention in the last block, indicating learnable registers make the model unable to recognize semantic regions on target domains. In contrast, random registers effectively guide the model's attention to the object.
2.2. How do registers affect the attention?
As ViTs are prevailing for their self- attenttention mechanism, we first visualize the attention map of the model with learn- able or random registers on the target domain Fig. 2. As the CLS token feature from the last VIT block utilized as the final output of the backbone network, we visualize the attention maps of the CLS token against image tokens in the last block. In visualizations, the red color indicates large attention scores while blue indicates small ones.
f(-) is transferred to target-domain datasets DT, where only 1 or 5 samples are available for each class. Following current works (Chen et al., 2021; Fu et al., 2021), n-way k-shot episodes are sampled for target-domain training and evaluation. It means each episode consists of support /==1f==1 with classes and ie samples in each class for training and query set { for evaluation. The classification is conducted by the distance between class prototypes and samples
set (Th, Uhy yn.k
From Fig. 2, we can see the baseline model struggles to cap- ture semantic regions when transferred to target domains. Adding learnable registers further intensifies the model's focus on irrelevant regions, while random registers help the model focus semantic objects in target domains. There- fore, we hypothesize the attention network, as the core of ViT, may not be well transferred to target domains.
T = arg min dl EOSDNOO
(2)
where -(:.-) denotes the Euclidean distance function. We follow StyAdv (Fu et al., 2023) use the Vision Trans- former (ViT) with the DINO pretrained network. Since in Fig. 1 the shallow prompt shows more significant changes in performance, below we mainly target to study this kind of prompt learning', which adds learnable tokens to the input token sequence (similar to the [CLS] token). Following (Darcet et al., 2024), we term registers. Then, the VIT takes the [CLS] token, image tokens, and learnable registers inputs P fed into feature extractor f()) later, written as
To verify our hypothesis, we quantitatively measure the transferability of the attention network by the sharpness of loss landscapes (Foret et al., 2021; Zou et al., 2024a). Specif- ically, given well-trained model from the source domain, each weight (Foret et al., 2021) representation (Zou et al., 2024a) for the source-domain input data can be viewed as a point associated with a loss value, where each minima point represents a good weight or representation (Fig. 3a). Sharpness measures how the loss value changes when the data shifts from the source domain, with lower loss change indicating better robustness domain shifts.
backbone
To evaluate the sharpness of the attention network, we add perturbations to the attention map by Gaussian noises
f(P)= = F(C(TT T
,T,))),
where the [CLS] token is denoted as T E Rd, image tokens as T())= E R=" xd and registers as T E Red ñ the register number, C(-.-) means the concatenation of different tokens (Fig. la). In the following, we will explore why random registers improve target-domain performance.
Sharpness = mar, [L(A +) L(A)), < e~ c~N(00 (4)
where A denotes the attention map, controls the perturba- tion level. As can be seen in Fig. 3b, all models experience rise in loss when subjected to these perturbations. How- ever, the model with learnable registers consistently shows a much higher sharpness than others, indicating they push the model to be less robust to domain shifts. The model with
For its deep version, please refer the Appendix, which shows similar results to the shallow one.
3

---

Random Registers for Cross-Domain Few-Shot Learning
0.12
0.08
0.04
CropDiseases
EuroSAT
ISIC2018
ChestX
Ave.
Registers Domain Similarity measured by the CKA
Leumable Registeess
Figure Adding learnable registers consistently decreases do- main similarity, indicating registers contain domain information only valid on the source dataset, increasing sharpness. Mean- while, random registers consistently improve the domain similarity, demonstrating the model pushed to learn domain-and domain-agnostic infor- mation, is interpreted sharpness-award minimization.
Image
Baseline
Learnable Registers Random Rugisters
Figure 5. The attention the source dataset. Learnable registers make the model concentrate on regions irrelevant to the object, which can be viewed as a kind of overfitting. In contrast, random registers effectively guide the model's attention the object.
random registers exhibits a smaller increase than the base- line model, quantitatively verifying that random registers improve the transferability of attention.
2.4. Why do registers influence sharpness?
Then, we delve into why registers influence sharpness and transferability. Since the domain gap is the most important challenge for transferability in CDFSL, we follow (Oh et al., 2022; Davari al., 2022) to take the CKA (Kornblith et al., 2019) similarity
2.3. Why do registers contribute to transferability?
Based on the above analysis, we interpret random registers as a kind of sharpness-aware minimization (SAM (Foret al., 2021), please refer to the appendix for more details) as follows. Following (Foret al., 2021), SAM can be formulated
tool to measure the domain similar- ity between source and target datasets. Specifically, after the backbone network is trained the source datasets, extract features of images from different domains and then compute the CKA similarity. A higher value means a higher similarity between the source and target domains, repre- senting more domain aagain-agnostic information the backbone network. As shown in Fig. 4, we can see:
LSAM = min max L(w + te) + A((Ww1B3)
(5)
where w refers the components vulnerable data shifts. In contrast, adding random registers to the input sequence can be reflected in the attention maps as
(1) Adding learnable registers trained the source domain can significantly drop the CKA similarity, indicating the reg- isters contain domain information learned from the source datasets. With the increase of the register number in Fig. 1, more domain information is absorbed by registers, guiding the model learn more domain-speciftt information that only valid on source- domain classification.
E Q.KY
Aij
(6)
ER=1
Q.KE ER=1e Q.KE
where n is the total patch number in an image and ñ is the number of registers we add to the input sequence fed into VIT. For each image token or CLS token, its query Q will be multiplied with randomized key K T W from random registers R Therefore, the term Eie e will also random noise. Denote such a noise as R we can rewrite SAM Eq. 5
(2) Random registers, added as random Gaussian noises, consistently increase the CKA similarity, and more random registers in Fig. 1 are more beneficial to the model. Since random registers are abandoned during the target-domain phase, this result means random registers push the model to learn domain-agnostic information valid across domains, as is explained sharpness-awen minimization.
LSAM = min [mas, L(A + e)) + +(I(w)))
As the attention is verified be vulnerable in Fig. 2, it is reasonable to add noise to the attention map. In all, adding random registers can be regarded as novel way to add per- turbations to the attention maps and can be viewed as a kind of SAM, which helps the model to find a more flattened minimum and be well transferred target domains.
Based it, we interpret the increased sharpness result of absorbed domain information, which is only effective for the source domain. Therefore, when the data is shifted from the source domain, such model cannot extract effective features from the shifted data, leading to a significant increase in the loss, i.e., larger sharpness.

---

Random Registers for Cross-Domain Few-Shot Learning
To further understand how the domain information is re- flected in the pattern learning, we visualize the attention map of the model with learnable and random registers the source domain in Fig. 5. We can see that by adding learnable registers, the model captures more regions irrel- evant to the recognition of the object (c.g., patterns in the background), while random registers help the model focus more on the object. For learnable registers, this result means the model takes these irrelevant regions crucial clues for classification. This phenomenon essentially represents the model's overfitting to the source domain since these re- gions are class-irrelevant and can be pource-omain--seciine ie., they may be useful for the source domain but invalid for other domains. For random registers, this result means the model is pushed focus less on regions irrelevant to the object, e.g., patterns outside the object, which are more likely to be class-relevant and transferable across domains. That is, the model is pushed to learn nomain-agnostic infor- mation, consistent with the analysis above.
VIT Buckbome
L
Additional Registers
Rundon Rajeen
Random Registers Enhanced Attention Perturbation
Caunc
Regiters
Image
Rundom Anchoes
Clusters
Rundom Registers
Source-oomain Stage
VIT Backhone
Additional Registors
Tokns
Leirnable Regiters
(b) Target- Target-Domain Stage
Figure 6. Overview of our framework, which consists of two stages. (a) In the source-domain stage, we randomly drop clustered image tokens and replace dropped tokens with random registers (vectors of Gaussian noises), which improves the effectiveness and effi- ciency of random registers, thereby encouraging the model to learn domain-agnostic information. (b) In the target-domain stage, we activate the additional registers to the learnable state to help the model adapt to target domains with only scaree data.
2.5. Conclusion and Discussion
Based on these, we interpret as follows: By applying learn able registers, the model focuses more on the regions irrele- vant to the object recognition. This can be viewed as a kind of overfitting to the source domain, which contains more domain information and increases the sharpness of the loss landscapes, i.e., harms the generalization to target domains. Applying random registers is an efficient and novel way to perturb the attention maps fed into the model and can be viewed as a kind of sharpness-awee -arppe minimization. There- fore, the model is trained to find a flattened minimum in loss landscapes and of good transferability to target domains. As a result, the model focuses less on the regions outside the object and generalizes better to target domains.
to target domains with scarce data.
3.1. Random Registers Enhanced Attention Perturbation (REAP)
In Fig. 1, the random register can be effective only when the number of it is very large, reaching the limit that the GPU memory can hold, which is ineffective and inefficient.
3. Method
Based on the above analysis and interpretation, we further develop a simple but effective approach called Random Registers Enhanced Attention Perturbation (REAP) for the CDFSL task by improving the effectiveness and efficiency of random registers, as to boost the model's transferability. Meanwhile, according the characteristies of these two types of registers, we further propose a two-stage training strategy for the model, as shown in Fig. 6.
To handle this problem, we think about the ratio of perturbed information that random registers can bring to the attention map of the model. Specifically, the reason why we need a large number of random registers is that the atlention maps has already captured rich information from input images, therefore we need a strong perturbation from random reg- isters to mitigate the influence of already-captured patterns on attention maps.
During the source-domain stage, we design an improved random-register-base method to improve both the effective- ness and efficiency of random registers, which encourages the model learn flattened minimum in loss landscapes on the source domain. During the target-domain stage, in- stead of random registers that perturb domain information, we come back to the regular learnable registers for their absorption of domain information, to help the model adapt
An intuitive way to strengthen the perturbation is to add more random registers, making random registers inefficient. What if we add random registers to hinder the original atten-
tion maps Σi eQ.K7 in Eq. 6? This would perturb the already captured information and intensify the perturbation that random registers bring. Based on this insight, we then
design a kind of perturbation method on the input image tokens that contain rich semantic regions and contribute
5

---

Random Registers for Cross-Domain Few-Shot Learning
most to attention maps, reflected in the attention maps as
above analysis and interpretation. We utilize REAP fully dig out the potential of random registers, helping the model learn more domain-agnostic information and find the flat- tened minimum in the loss landscape. Therefore, later the model can be well transferred to the target domain. The goal of our model in the source-domain stage is
As,
(8)
Σ E
e
Maintaiand Iouage
Image Perturbation
Register Perturhation
where m is the remaining number of original image patches, and n - m is the number of perturbed image patches. Fol- lowing (Naseer et al., 2021), ViT is highly robust to severe occlusions (e.g., random patch perturbations). Adding per- turbations to random patches seems ill-advised. However, as the core mechanism of ViT, the attention map tends to rely on capturing patterns from continuous regions in the image (Wei et al., 2024). This inspires us to first cluster image tokens, then replace clusters with random registers to increase the perturbation to the attention maps.
N
L
1 N
Σ
Lase (Ot .,,T"),, (13)
3
3.3. Target-domain stage
In this stage, our goal to help the model adapt to target domains with only scarce data. Since the random register reduces the learning of domain information, it is not suitable at this stage. Therefore, we revisit the learnable register for its tendency to learn domain information, which will help the model in the few-shot adaptation to target domains. So we activate the additional registers to the learnable state (e.g., Th), as shown in Fig. 6b, and finetune the model on the support set with a classifier for each episode as
Specifically, as shown in Fig. 6a, given an input image I and its patches X Raxd where X, is the average of pixels in the ith patch (i nt), we randomly select large portion of patches a (60%n I a<'n) for every image as the anchors A (c.g., the jth patch is selected as the anchor A, = Xy, j E a). Then, we compute the anchors' distance with image patches by leveraging the cosine similarity
N
L= N
Σ
Las (TT
(14)
X, A, ||,,|| ||A,||
cos(X A,)
(9)
4. Experiments
cluster is defined as consisting of an anchor patch A, and corresponding patches within the similarity threshold sim
4.1. Implementation Details
Cluster = {X: cos((; A,) 2 sim, X, E X]: (10)
Following current works (Oh al., 2022), our model is trained the minilmageNet dataset (Vinyals et al., 2016) as the source domain and then transferred four target- domain datasets, including CropDiseases (Mohanty et al., 2016), EuroSAT (Helber et al., 2019), ISIC2018 (Codella et al., 2019), and ChestX (Wang et al., 2017), using the k-way n-shot classification.
where the sim is automatically computed to guarantee the drop ratio is stable. Then, we replace the clusters with ran- dom registers. The random registers are randomly sampled from Gaussian distribution N00, 1) -T where 5 is a learnable parameter, written as
Xi, & Cluster T. TR.,X, X2 E Cluster
,T N(O, (11)
During the training on the source domain, we take VIT-S as the backbone network and DINO pretraining ImageNet as the initialization following (Zhang et al., 2022b; Fu et al., 2023). We set the ratio of anchor number and minimum drop ratio to 70% for cluster-dropping to the images, and replace them with Random Registers. We use the Icarnable standard deviation to generate random Gaussian noise and the initial value we set is 0.1. In addition, we also concate- nate additional 16 random registers the reconstructed patches as the input sequence of the model. Our model has trained with the Adam (Kingma & Ba, 2017) optimizer for 50 epochs with a learning rate of 10-5 for the backbone network and 10-3 for the classifier respectively. During the few-shot evaluation on target domains, we provide the image with the same number(16) of learnable registers as the input of the VIT and set learning rate of 10-3 for regis- ters especially for absorbing target-domain information.
We add additional random registers the end of the input token sequence (Fig, 6). but the number of additionally added registers is small (verified in Fig. 10b). So the input of the model fed into the feature extractor is written as Th. ,
= -(C(TC
(12)
By multiplied with randomized keys from random registers, the term Σi=r eQ.XR in Eq. will also be the noise. So it can also be regarded as SAM.
3.2. Source-domain stage
In this stage, our goal is to help the model learn domain- agnostic information on the source domain. Based on the
6

---

Random Registers for Cross-Domain Few-Shot Learning
Table 1. Comparison with state-of-the the-----tthhheee- art works based on ViT-S on target domains.
Method
Shot
FT
Mark
ChestX
ISIC2018
EuroAAT
CropDiseases
Average
MEM-FS (Walsh et al., 2023)
X
TIP-23
22.76
32.97
68,11
81.11
51.24
StyleAdv (Fu al., 2023)
X
CVPR-23
22.92
33.05
72.15
$2.34
FLoR (Zou et al., 2024a)
I
X
CVPR-22 24
22.78
34,20
72.39
81.81
52.80
DAMIM (Ma al., 2024)
X
AAAI-25
22.97
34.66
72.87
82.34
53.21
CD-CLS (Zou et al., b)
X
NeurlPS-22 24
22.93
34.21
74.08
83.51
53.68
AttnTemp (Zou et al., a)
X
NeurlPS-24
23.19
34,92
74.35
84.02
54.12
REAP
X
Ours
23.62
37.21
74.69
84.04
54.89
PMF (Shell Xu, 2022)
CVPR-22
21.73
30.36
70.74
80.79
50,91
FLoR (Zou et al., 2024a)
CVPR-24
23.26
35.49
73.09
83.55
53.85
StyleAdv (Fu et al., 2023)
CVPR-22
22.92
33.99
74,93
84.11
53.99
DAMIM (Ma al., 2024)
AAAI-25
23.38
36.35
73.61
83.90
$4.31
CD-CLS (Zou et al., b)
NeurlPS-24
23.39
35.56
74.97
84.54
54.62
AttTTemp (Zou et al., a)
NeurlPS-24
23.63
38,05
75.09
84.78
55,39
REAP
Ours
24.17
38.67
75.97
85.33
56.04
MEM-FS RDA" (Walsh et al., 2023)
TIP-23
23,85
37.07
75.91
83.74
55.14
DAMIM (Ma et al., 2024)
AAAI-25
23.91
38.07
77.23
86.74
56.49
CD-CLS (Zou et al., b)
NeurIPS-22 24
23.88
37,20
78,41
87.39
56.72
AttnTemp (Zou et al., a)
NeurIPS-24
23.96
40.13
77.40
87.58
$7.23
REAP
Ours
24.49
39,53
79.13
89.33
58.12
MEM-FS (Walsh et al., 2023)
X
TIP-23
26.67
47.38
86.49
93.74
63.57
StyleAdv (Fu et al., 2023)
5
x
CVPR-23
26.97
47.73
88.57
94.85
64.53
FLoR (Zou al., 2024a)
5
CVPR-22 -24
26.71
49.52
90.41
95.28
65.48
DAMIM (Ma et al., 2024)
X
AAAI-25
27.28
50.76
89.50
95.52
65.77
CD-CLS (Zou et al., b)
5
X
NeurlPS-22 24
27.23
50.46
91.04
95.68
66.10
AttnTemp (Zou et al., a)
5
NeurIPS-24
27.72
53.09
90.13
95.53
66.62
REAP
5
X
Ours
27.98
52.80
90.53
95.68
66.75
PMF (Shell Xu, 2022)
CVPR
27.27
50.12
85,98
92.96
64.08
StyleAdv (Fu et al., 2023)
5
CVPR-23
26.97
51.23
90.12
95.99
66.08
FLoR (Zou al., 2024a)
5
CVPR -24
27.02
53,06
90.75
96.47
66.83
DAMIM (Ma et al., 2024)
AAAI-25
27.82
54.86
91.18
96.34
67.55
CD-CLS (Zou et al., b)
5
NeurIPS-24
27.66
54.69
91.53
96.27
67.54
AttnTemp (Zou et al., a)
5
V
NeurIPS-24
28,03
54,91
90.82
96.66
67.61
REAP
Ours
28.34
55.28
91.79
96.71
68.03
MEM-FS + RDA' (Walsh et al., 2023)
TIP-23
27.98
$1.02
88.77
95.04
65.70
DAMIM" (Ma et al., 2024)
5
AAAI-25
28.10
55.44
91.08
96.49
67.78
CD-CLS' (Zou et al., b)
5
/
NeurIPS-24
28.25
55.66
91.68
96.62
68.05
AttTTemp' (Zou a)
NeurIPS-2 24
28.41
55.22
91.34
96.74
67.93
REAP
5
Ours
28.80
56.07
91.92
96.74
68.38
4.2. Comparison with State-of-the-Art Works
Table Ablation study of source-domain training by 5-shot.
Method
CropDisasee
FunSAT
TSIC2018
ChestX
Comparisons with the state-of-the-art works listed in
Baseline
94.61am
29.29
46. 16
26.21
64.07
Tab. 1, taking the ViT-S backbone for both 1-shot and 5-shot
Random Registers
95. 14g0
89 89.44
48. 28.92
26 18
65.05
settings on four target-domain datasets. We group work by whether finetuning (FT) or utilizing the transductive (*) setting for fairness, following (Zou et al., a;b), As shown Tab. 1, our works achieve the top average performance in settings, demonstrating that the approach could consistently outperform current works. Please refer to the appendix for more comparisons (e.g., CNN-based SOTAs).
REAP
95.68
90.53
52.80.c3
27.98:a1a
66.75
(a) Random-mask
a3.3
24.42amm
43. anc
14.06
Cluster-mask
94.61atm
59
47.33
26.38
64.29
Eq. 8 contribute to the performance of target domains. We further make comparison between our approaches with other similar works to validate the rationale of our designs.
4.3. Ablation Study
4.3. IMAGE PERTURBATION BY CLUSTERING
The ablation study of each module reported in Tab. 2. We both the random and image perturbation in
To study the contribution of clustering, we randomly select the anchors the image and mask these anchors in Tab. 2a.

---

Random Registers for Cross-Domain Few-Shot Learning
Table Ablation study of target-domain finetuning by 1-shot.
Mothod
CropDisease
BureSAT
ISIC2018
ChestX
Ave.
Regulur l'inetuning
14.5
36.52g325
54.50
Random Registers
83. 13eu
74.01
35.53
23.34amn
54,00
0.6
Learnable Registers
85.33
35297
38.67.119
24.17
56.04
0.4
For fair comparison, we keep the same masking ratio in both clustering and random approaches. We can see the performance is even much lower than the baseline, which means the random type would harm the model, and verifying the image perturbation by clusters is crucial.
0.2
CrepDleessee
EuroSAT
ISIC2018
CheetX
Ave.
Reselline
Random Ragisters
RFAP (Ouns)
Domain Similarity measured by the CKA
Figure 7. Utilizing our approaches significantly increases the do- main similarity, proving that utilizing our approach makes the model more domain-agnose
4.3.2. REPLACING WITH RANDOM REGISTERS
We study the contribution of replacing the clusters with ran- dom registers. We directly remove the clusters (Tab. 2b). The performance lower than ours, indicating that replac- ing the clusters with random registers is an effective way to help the model be more transferable.
4.3.3. TARGET-DOMAN FINETUNING
Image
Bessiine Oura Tarpet denalar
To study the contribution to target-domain finetuning, we compare the regular finetuning operation, finetuning with random registers, and finetuning with learnable registers. As shown in Tab. 3, finetuning with learnable registers could improve the model's performance on the target domains while random registers decrease it, which further verifies the absorption and perturbation of domain information in learnable and random registers, respectively.
Insage
Image
(l) Brecline vs. Oars the Soarce demais
Figure 8. (a) The heatmap on target domains proves that our ap- proaches effectively improve target-domain performance. (b) The heatmap on source domain demonstrates that our approach effectively avoids overfitting on the source domain.
4.4. Verification of model generalization
4.4.1. QUANTITATIVE STUDY
proving that the information in the image the most repre- sentable and influence the whole attention maps than others.
As shown in Fig. 7, we calculate the domain similarity of the features extracted from the trained backbone network between source and target domains by the CKA similarity. We can see model significantly increases the domain similarity, indicating our method encourages the model to learn agnosi-i information of well transferability.
(2) A high anchor ratio is better, but too high harm. From Fig. 9b, the ratio of anchor number sets between 40% and 80% significantly improves performance. A rising anchor number means that more continual areas tend to be replaced, but too much will destroy important information.
4.4.2. QUALITATIVE STUDY
(3) A high replaced ratio is better, but cannot be high. As we can see in Fig. 10a, setting the replaced ratio higher could see a steady rise in the performance target domains before 70%. After that, the performance drops sharply. The same situation happens with the anchor number ratio, indicating that a higher replaced ratio could help perturb more on the attention maps, while a ratio that too high may likely throw away too much information for learning.
The visualization of the attention maps our model on both the source and target domain is shown in Fig. 8. Com- pared to the dispersed attention observed in the baseline, model focuses more valid and concentrated regions within the image, verifying that our approach improves the generalization to target domains.
4.5. Sensitivity Study of Hyper-parametes
(4) Additional random registers beneficial. In Fig. 10b, after replacing the clusters with random registers, concate- nating some random registers further improves the perfor- mance on the target domain. Setting the extra register num- ber to 16 is better than less and more. Notably, this number of registers is much smaller than that Fig. 1b.
We study the hyper-parameters in Fig. 9, 10, 11 and see that:
(1) The input layer effective. In Fig. 9a, only applying our approach in the first block (input layer) could signifi- cantly improve the model's performance on target domains,
8

---

Random Registers for Cross-Domand Few-Shot Learning
Ave. Accuner
66.3%
Block ID 1s. Scbat Perooweree
66 6%
67.2%
Asdacc nito 5-bal Perboriazes
Ave. Accuncy
66.85%
06 0%
66 66.65%
05.3%
65.1%
64.%
60.8%
64.3%
64.2%
66.45%
0.2
VIT Block ID
Anchor nañoo
Noise vs. 5-shot Performance
Block vs. S-duoi Performanes
(b) Anchor roto vs, S-dtot Performance
66.25%
0,0001
0.001
Figure 9. (a) Applying our approach only on the first block (input layer) can improve the performance. (b) A high anchor ratio can effectively improve performance.
Noise std
Noise sid vs. S-shot performance
Figure I1. Setting the random registers with standard deviation of intermediate magnitude is optimal, demonstrating that the distur- bance attention maps should neither be too weak nor too strong.
56.5%
4500
raiio Sobeet Paformeen
Jokikedd Rundon Regieen Nas
Table 4. Ablation study of our method with more backbones.
90.%
Repkaed Replaced satl
16 64
128
Method
CropDiseases
EuroSAT
ISIC2018
ChestX
Ave.
Addidiai Randoen Regiiter
CLIP
93.08
74.34.125
41.29.oz:
23,98 sais
58.17
S-thet Pertermance
CLIP+ Ours
93.93.c1
81.10.33
44.90.03
23.78.ate
60.93
Figure 10. (a) A high replaced ratio can effectively improve perfor- mance, but too high would harm. (b) Additional random registers are beneficial, and the number much smaller than that in Fig. 1b.
IBOT
94
80.0
48.8
25. 32.31
63.00
IBOT Ours
94,88
89.18
46.71
26.36
64.28
DINO
95.39.
89.29..11
47.98.ma:
26.29
64.74
DINO-VIT-Base + Ours
96.11
89.77.111
50.88 Lo2
26.70..17
65.87
(5) Moderate perturbation helps. In Fig. 11, when we set consistent number of random registers (e.g., 16) and grad- ually increase the standard deviation of their normal dis- tribution, the model's performance initially rises and then declines, attesting to the necessity of a moderate level of perturbation to the attention map, neither too slight nor excessive.
performance of ViT in extreme cross-domain scenarios.
Prompts are widely used in natural language process- ing (Yao et al.. 2023), which refers to a given one or se- ries of text inputs intended to guide or trigger the model to produce specific output or response (Liu et al., 2023). As large language models progress, Prompt has played an important role (White et al., 2023). In computer vision, ViT has been widely used as backbone. The CLS token of ViT (Liu et al., 2021) is also a special kind of prompt of vital importance for the classification (Zhang et al., 2022a). Recently, prompt-based approaches (Yao et al., 2023; Wang et al., 2023; Sohn et al., 2023) are developed for various downstream vision tasks, like (Darcet al., 2024) extend the input sequence with prompts without adding any infor- mation and drop before the output, successfully avoiding collateral side effects. ProD (Ma et al., 2023) uses two paral- lel prompts to disentangle the Nomainnggencee and domain- specific knowledge to alleviate the domain gap. However, it merely utilizes prompts without considering or explaining the overfitting problem in prompt tuning within the source domain. In all, prompt learning under large domain gaps is still under-explored.
4.6. Applying Our Method to Other Backbones
We also implement our approach on different backbones, like ViT pretrained by iBOT(Zhou et al., 2021), ViT- Basc(Zhang et al., 2022b) pretrained by DINO, and VIT- Base pretrained by CLIP(Radford et al., 2021). The results can be seen in Tab. 4. Specifically, iBOT represents the iBOT-pretrrained ViT baseline, DINO-VTT-Base corresponds to the ViT-Base pretrained by DINO baseline and CLIP cor- responds to the ViT-Base pretrained by CLIP baseline. It is clarified from the average performance of four target do- mains that our approach shows considerable improvement among all backbones in 5-way 5-shot setting.
5. Related Work
Few-Shot Learning (CDFSL) was pro- posed in FWT (Tseng et al., 2020) and has a new benchmark in BSCD-FSL (Guo et al., 2020). Recently, it has been stud- ied by several works (Oh et al., 2022; Zhang et al., 2022c; Xu et al., 2023; 2024), which focuses on training a model on the source domain that can generalize well target domain with limited examples. Current works can be grouped into two types: meta-learning based approaches (Fu et al., 2022; Hu& & Ma, 2022), learning task-agnostic knowledge to learn new tasks efficiently (Guo et al., 2020), and transfer learn- ing based approaches (Zhou et al., 2023; Zou et al., 2024a), reusing the model trained on the base classes dataset. How- ever, there has been insufficient in-depth research into the
6. Conclusion
In this paper, we find a phenomenon that providing addi- tional learning registers is detrimental to the CDFSL per- formance while adding random noise to registers improves it. We delve into this phenomenon for an interpretation and find that the learnable registers naturally absorb domain information, while random registers perturb it and help the model find a flattened minimum in the loss landscapes of well transferability. Based on these, we further propose a method to make random registers more effective and effi- cient. Experiments validate our rationale and effectiveness.

---

Random Registers for Cross-Domain Few-Shot Learning
Acknowledgments
Das, D., Yun, S., and Porikli, F. Confess: A framework for single source cross-domain few-shot learning. In International Conference on Learning Representations, 2022.
This work is supported by the National Key search and Development Program of China under grant 2024YFC3307900; the National Natural Science Founda- tion of China under grants 62206102, 62436003, 62376103 and 62302184; Major Science and Technology Project of Hubei Province under grant 2024BAA008; Hubei Sci- ence and Technology Talent Service Project under grant 2024DJC078; and Ant Group through Cnt-Ant Research Fund. The computation is completed in the HPC Platform of Huazhong University of Science and Technology.
Re-
Davari, M., Horoi, S., Natik, A., Lajoie, G., Wolf, G., and Belilovsky, E. Reliability of cka as a similarity measure in deep learning, 2022.
Deng. J., Dong, W., Socher, R., I...J,, Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 248-255. Ieee, 2009.
Impact Statement
We introduce CD-FSL method that enhances perturbations in attention maps by adding random registers to the seman- tic regions of image tokens in the VIT. This helps mitigate the domain gap and improves generalization the target domain. Our approach is also applicable other areas, such as domain generalization, domain adaptation, and few-shot class-incremental learning, where improving model transfer- ability is a common challenge. While our evaluations focus on four distinet target domains, these may not encompass all potential real-world scenarios. Therefore, further evaluation across a wider range of target domains is needed to validate the approach in more realistic settings.
Foret, P., Kleiner, A., Mobahi, H., and Neyshabur, B. Sharpness-awan awwre minimization for efficiently improving generalization, 2021.
Fu, Y., Fu, Y., and Jiang, Y.GG. Meta-fdmixup: Cross- domain few-shot learning guided by labeled target data. In Proceedings of the 29th ACM international conference multimedia, pp. 5326 5334, 2021.
Fu, Y., Xie, Y., Fu, Y., Chen, J., and Jiang, Y.-G. Wave- Wavelet based style augmentation network for cross- domain few-shot learning, 2022.
Fu, Y., Xie, Y., Fu, Y., and Jiang, Y-G., Styleadv: Meta style adversarial training for cross-domain few-shot learning, 2023.
References
Chen, X., Hu, Q., K.,
K., Zhong, C., and Wang. G. Accumu- lated trivial attention matters in vision transformers small datasets. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 3984-3992, January 2023a.
Guo, Y., Codella, N. C.,, Karlinsky, L., Codella, V., Smith, J. R., Saenko, K., Rosing, T., and Feris, R. A broader study of cross-domain few-shot learning. In Proceedings of the IEEE/CVF European Conference on Computer Vision, pp. 124-141. Springer, 2020.
Chen, X., Wang, Y., Du, Y., Hassoun, S., and Liu, L.-P. On separate normalization in self-supervised transformers, 2023b.
Helber, P., Bischke, B., Dengel, A., and Borth, D. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification, 2019.
Chen, Y., Liu, Z., Xu, H., Darrell, T., and Wang, X. Meta- baseline: Exploring simple meta-learning for few-shot learning, 2021.
Hu, Y. and Ma. A. J. Adversarial feature augmentation for cross cross-domain few-shot classification, 2022.
Codella, N., Rotemberg, V., Tschandl, P., Celebi, M. E., Dusza, S., Gutman, D., Helba, B.. Kalloo, A., Liopyris, K., Marchetti, M., Kittler, H., and Halpern, A. Skin lesion analysis toward melanoma detection 2018: challenge hosted by the international skin imaging collaboration (isic), 2019.
Jia, M., Tang, L., Chen, B.-C., Cardie, C., Belongie, S., Hariharan, B., and Lim, S.-N. Visual prompt tuning. In European Conference on Computer Vision (ECCV), 2022.
Kim, D. and Han, B. On the stability-plasticity dilemma Mlas-incremental learning. In Proceedings of the IEEE/VV Conference on Computer Vision and Pattern Recognition, pp. 20196 20196-20204, 2023.
Darcet, T., Oquab, M., Mairal, J., and Bojanowski, P. Vi- sion transformers need registers. In The Twelfth In- ternational Conference on Learning Representations, 2024. URL https::///// ttttpp:///penrevicce /aet/forumt 1112
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization, 2017.
10

---

Random Registers for Cross-Domaie Few-Shot Learning
Kornblith, S., Norouzi, M., Lee, H., and Hinton, G. Similar- ity of neural network representations revisited. In Interna- tional Conference on Machine Learning, pp. 3519-3529. PMLR. 2019.
cc cc//appe__illeepppprr//////// cc00efaabff990096561188811133333 pdf.
Noori, M., Cheraghalikhani,, M., Bahri, A., Vargas Hakim,
Krizhevsky, A., Sutskever, I., and Hinton, G. E. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84-90 2017.
A., Osowiechi, D., Ayed, I. B., and Desrosiers, C. Tfs- vit: Token-level feature stylization for domain generaliza- tion. Pattern Recognition, 149:1 10213, 2024. ISSN 0031-
3203. doi: URL
Li, K., Zhang. Y., Li, K., and Fu, Y. Adversarial feature hallucination networks for few-shot learning. In Proceed- ings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 13470-13479, 2020.
Oh, J., Kim, S., Ho, N., Kim, J.-H., Song, H., and Yun, S.-Y. Understanding cross-domain few-shot learning based on domain similarity and few-shot difficulty, 2022.
Li, P., Gong, S., Wang. C., and Fu, Y. Ranking distance calibration for cross dons-domain few-shot learning, 2022.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G.,
Liang, M.. Huang, S., Pan, S., Gong, M.. and Liu, W. Learn- ing multi-level weight-centric features for few-shot learn- ing. Pattern Recognition, 128: 2022. ISSN 0031- 3203. doi:
Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
URL sci science/artiin httienee//rrcceee/p////031122222
Shell Xu, Da Li, J. Pushing the limits of simple pipelines for few-shot learning: External data and fine-tuning make difference, 2022.
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., and Neubig, G. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1-35, 2023.
Snell, J.. Swersky, K., and Zemel, R. Prototypical networks for few-shot learning. In Proceedings of the International Conference Neural Information Processing Systems, pp. 4080 2017.
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021.
Sohn, K., Chang, H., Lezama, J., Polania, L., Zhang, H., Hao, Y., Essa, I., and Jiang, L. Visual prompt tuning for generative transfer learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19840-19851, 2023.
Ma, R., Zou, Y., Li, Y., and Li, R. Reconstruction target matters in masked image modeling for cross-domain few- shot learning. arXiv preprint arXiv:2412.19110 2024.
Tseng, H.-Y., Lee, H.-Y., Huang, J.-B., and Yang, M.-H. Cross-domain few-shot classification via learned feature- wise transformation. In Proceedings of the International Conference on Learning Representations, 2020.
Ma, T., Sun, Y., Yang, Z., and Yang, Y. Prod: Prompting-to- disentangle domain knowledge for cross-domain few-shot image classification. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, 2023.
Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., and Wierstra, D. Matching networks for one shot learning. In Proceedings of the International Conference Neural Information Processing Systems, pp. 3637-3645, 2016.
pp.
Mohanty, S., Hughes, D., and Salathé, M. Using deep learn- ing for image-basee based plant disease detection. Frontiers in Plant Science, 7(September), September 2016. ISSN 1664-462X. doi: 10. 0t099//pls.211.0144114 Publisher Copyright: c 2016 Mohanty, Hughes and Salathé.
Walsh, R., Osman, I., and Shehata, M. S. Masked embed- ding modeling with rapid domain adjustment for few-shot image classification. IEEE Transactions Image Pro- cessing, 32:4907-4920, 2023, doi: 10. 10.11090 109//1/./,..., 3306916.
Naseer, M. M., Ranasinghe, K., Khan, H., Hayat, M., Shahbaz Khan, F., and Yang. M.-H. Intriguing properties of vision transformers. In Ranzato, M., Beygelzimer, A.. Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 23296-23308. Curran Associates, Inc., 2021. URL https: / //t:://rrrr pprocedding igss neurips.
Wang. H. and Deng. Z.-H. Cross-domain few-shot classifi- cation via adversarial task augmentation, 2021.
Wang. J., Zhou, P., Shou, M. Z., and Yan, S. Position- guided text prompt for vision-language pre-training. In

---

Random Registers for Cross-Domain Few-Shot Learning
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23242-23251, 2023.
Zhou, J., Wei, C., Wang, H., Shen, W., Xie, C., Yuille, A., and Kong, T. ibot: Image bert pre-training with online tokenizer. arXiv preprint urXiv:2111.07833 2021.
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., and Summers, R. M. Chestx-ray8: Hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax dis- eases. In 2017 IEEE Conference on Computer Vision and Puttern Recognition (CVPR). IEEE, July 2017. doi: 10. 201.0. URL http://dde doi. org/ 10 10.1109/CVPR..217. 369.
Zou, Y., Ma, R., Li, Y., and Li, Attention temperature matters in vit-based cross-domain few-shot learning. In The Thirty-eighth Annual Conference on Neural Informa- tion Processing Systems, a.
Zou, Y., Yi, S., Li, Y., and Li, R. closer look at the cls token for cross-domain few-shot learning. In The Thirty-eightt Annual Conference on Neural Information Processing Systems, b.
Wei, Z., Pan, Z., and Owens, A. Efficient vision-language pre-training by cluster masking. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 26815-26825, 2024.
Zou, Y., Zhang. S., Yu, J., Tian, Y., and Moura, J. M. Revisit- ing mid-level patterns for cross-domain few-shot recogni- tion. In Proceedings of the ACM International Conference Multimedia. pp. 741-749 2021.
White, J., Fu, Q.. Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smithh J., and Schmidt, D. C. prompt pattern catalog enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302. 1182, 2023.
Zou, Y., Zhang, S., Li, Y., and Li, R. Margin- few-shot class-inerementll learning with class-level overfitting mit- igation. Advances in neural information processing sys- tems, 35:27267-27235 2022.
Xu, H., Zhi, S., Sun, S., Patel, V. M., and Liu, L. Deep learning for cross-domain few-shot visual recognition: A survey. arXiv preprint 110.i::2033.85525. 208857222223 2023.
Zou, Y., Liu, Y., Hu, Y., Li, Y., and Li, R. Flatten long- range loss landscapes for eross- rross-domain few-shot learning, 2024a.
Xu, H., Liu, L., Zhi, S., Fu, S., Su, Z., Cheng, M.-M., and Liu, Y. Enhancing information maximization with distance-aware contrastive learning for source-free cross- domain few-shot learning. IEEE Transactions on Image Processing, 2024.
Zou, Y., Zhang, S., Zhou, H., Li, Y., and Li, R. Composi- tional few-shot daas-innremental learning. arXiv preprint arXiv:2405. /7022, 2024b.
Yao, H., Zhang, R., and Xu, C. Visual-language prompt tuning with knowledge guidedd context optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6757-6767, 2023.
Zhang, C., Zhang, M., Zhang, S., Jin, D., Zhou, Q., Cai, Z., Zhao, H., Liu, X., and Liu, Z. Delving deep into the generalization of vision transformers under distribution shifts. In 2022 IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pp. 7267-7276, 2022a. doi: 10.1 (199/CVPR5268.2222..000
Zhang, H., Li, F., Liu, S., Zhang, I.., Su, H., Zhu, J., Ni, L. M., and Shum, H.-Y. Dino: Detr with improved denois- ing anchor boxes for end-to-end object detection, 2022b.
Zhang, J., Song. J., Gao, L., and Shen, H. Free-lunch for cross-domain few-shot learning: Style-aware episodic training with robust contrastive learning. In Proceedings of the 30th ACM International Conference on Multimedia, pp. 1586-2594, 2022c.
Zhou, F., Wang, P., Zhang, L.,, Wei, W., and Zhang. Y. Re- visiting prototypical network for cross domain few-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 20061 20061-20070, June 2023.
12

---

Appendix for Random Registers for Cross-Domain Few-Shot Learning
A. Detailed Dataset Description
CropDiseases
EuroSAT
ISIC2018
ChestX
minilmageNet
nat
the Source-domain Dataset
Target-domain Dutasets
Figure Samples source-domain minilmageNet dataset.
Figure 13. Samples of target-domain datasets: CropDiseases, Fu- roSAT, ISIC2018, and ChestX.
magelma (Vinyals et al., 2016) is a widely used dataset in the field meta-learning and few-shot learning. It de- rived from ImageNet (Deng al., 2009) dataset but with smaller scale. The minilmageNet dataset contains total of 60,000 natural images in 100 categories, each with 600 sam- ples and each image size 84 x 84 pixels. As shown in Fig. 12, the images in the minilmageNet dataset come from various scenes, including but not limited to social scenes, some of which include human objects, while others are photos taken in real scenes, displaying diverse content and features. Following the current works (Oh al., 2022: Tseng al., 2020), split it into 64 base classes the source-domain dataset for training. Additionally, as depicted in Fig. 13, utilize datasets from four distinct domains as target-domain datasets, including plant disease, surface satellite imagery, skin disease, and chest X-ray images respectively. We'll introduce them sequentially below.
ISIC2018 (Codella et al., 2019) is an important skin disease image dataset for the classification of dermoscopic images, encompassing 10,015 medical images for skin lesion clas- sification across 7 different classes. It is even less similar to the minilmageNet as it could not even represent natural scenes.
ChestX (Wang et al., 2017) is medical imaging dataset containing 25,847 frontal X-ray images distributed across distinct classes. The dataset is the most dissimilar to the minilmageNet in three orthogonal criteria. Apart from the two factors mentioned above, losing color channels appears in the ChestX.
B. Detailed Descriptions of the CKA
Following current works (e.g.. (Oh et al., 2022)), the domain similarity is measured by comparing the distance between two batches of images, where each batch is sampled from single domain. We follow (Kornblith et al., 2019) take it as the similarity function.
CropDiseases (Mohanty et al., 2016) is essential source for agricultural disease identification. It is composed of high-resolution and highly similar original images of similar crop diseases. Specifically, it consists of 38 distinct classes and a total of 43,456 images, which are natural im- ages but are very specialized, including various infected crops, healthy plants, and their corresponding disease cate- gory labels.
The CKA is the abbreviation of the Centered Kernel Align- ment, measuring the similarity between feature representa- tions. Based on the idea of kernel function, CKA measures the similarity between samples in two feature spaces by calculating their kernel matrix. However, unlike traditional kernel functions, CKA also introduces centralized opera- tions to ensure that the contribution of each sample is taken into account when aligned. The detailed process CKA (Central-core Alignment) analysis consists of the following steps and formulas:
EuroSAT (Helber al., 2019) is remote sensing image dataset for land use and cover classification, containing total of 27,000 satellite images of the Earth categorized into 10 distinct classes. The images the AT are less sim- ilar to images in minilmageNet since they lack perspective distortion, but still color images of natural scenes.
13

---

Random Registers for Cross-Domain Few-Shot Learning
Table
Comparison with more state-of of ofeth--rrt works based by 5-way I-shot accuracy.
Method
Buckbone
FT
Mark
ChestX
ISIC2018
EuroSAT
CropDiseases
Average
GNN + FT (Tseng et al., 2020)
ResNet10
ICLR-20
22.00
30.22
55.53
60.74
42.12
MN + AFA (Hu & Ma, 2022)
ResNet10
ECCV-22
22.11
32.32
61.28
60.71
44.10
GNN ATA (Wang Deng, 2021)
ResNet10
IICAI-21
22.10
33.21
61.35
67.47
46.53
GNN + AFA (Hu Ma, 2022)
ResNet10
ECCV-22
22.92
33.21
63.12
67.61
46.97
LDP-net (Zhou et al., 2023)
ResNet10
X
CVPR-23
23.01
33.97
65.11
69.64
47.18
FLoR (Zou et al., 2024a)
ResNet10
CVPR-24
23.11
38,11
62.90
73.64
49.69
MEM-FS (Walsh al., 2023)
VIT-S
TIP-23
22.76
32.97
68.11
81.11
51.24
StyleAdv (Fu et al., 2023)
VIT-5
CVPR-23
22.92
33.05
72.15
81.22
52.34
FLOR (Zou et al., 2024a)
VIT-S
CVPR-24
22.78
34,20
72.39
81.81
52.80
DAMIM (Ma et al.. 2024)
VIT-S
AAAI-25
22.97
34,66
72.87
82.34
53,21
CD-CLS (Zou al.. b)
VIT-S
NeurlPS-24
22.93
34.21
74.08
83.51
53.68
AtmnTemp (Zou et al.,
VIT-S
NeurlPS-24
23.19
34.92
84.02
54.12
REAP
VIT-S
Ours
23.62
37.21
74.69
84.04
54.89
PMF (Shell Xu, 2022)
VIT-S
CVPR-24
21.73
30.36
70.74
80.79
50.91
FLoR (Zou et al., 2024a)
ViT-S
CVPR-24
23.26
35.49
73.09
83.55
53.85
StyleAdy (Fu ct al., 2023)
VIT-S
CVPR-23
22.92
33.99
74.93
84.11
53.99
DAMIM (Ma et al., 2024)
VIT-S
AAAI-25
23.38
36.35
73.61
83.90
54.31
CD-CLS (Zou
b)
VIT-S
NeurIPS-24
23.39
35.56
74.97
84.54
54.62
Att (Zou et al.,
ViT-S
NeurlPS-24
23.63
38.05
75.09
84.78
55.39
REAP
VIT-S
Ours
24.17
38.67
75.97
85.33
56.04
LDP-net" (Zhou al., 2023)
ResNet10
CVPR-23
22.21
33.44
73.25
81.24
52.54
TPN + ATA" (Wang & Deng, 2021)
ResNet10
IICAI-21
22.45
35.55
70.84
82.47
52.83
RDC (Li et al., 2022)
ResNer10
CVPR-22
22.32
36.28
70.51
85.79
53.73
MEM-FS RDA (Walsh al., 2023)
VIT-S
TIP-23
23.85
37.07
75.91
83.74
55.14
DAMIM"
al., 2024)
VIT-S
AAAI-25
23.91
38.07
77.23
86.74
56.49
CD-CLS (Zou et al., b)
VIT-S
NeurlPS-22 24
23.88
37.20
78.41
87.39
56.72
AttnTemp (Zou et al. a)
VIT-S
NeurIPS-24
23.96
40.13
77.40
87.58
57.23
REAP
VIT-S
Ours
24.49
39.53
79.13
89.33
58.12
Create Gram matrix: First, for the given two sets of fea- tures representing X and Y, compute their Gram matrices K and L. These Gram matrices are obtained by calculating the inner product between feature representations, i.e. K = X x and = Y Y Centralized Gram matrix: Next, the Gram matrices K and are centralized to eliminate the impact of the average value of the data the results. Cal- culate -Shhmidt-Schmidt independence criterion (HSIC): Use vectorization operation (vec(-)) to process the centralized Gram matrix, and then calculate the HSIC value. HSIC is calculated as follows
CKA be interpreted the similarity of the relationship between the features in X and Y from different domains.
Therefore, we quantitatively measure the domain distance between source and target datasets by the CKA similarity following (Davari et al., 2022). Specifically, given a back- bone network, we extract features from images in different domains and then calculate the CKA similarity by align- ing the channel dimension. Suppose model is completely overfitted to one domain, given other domains' images, the extracted features could be just random noises, therefore the domain similarity will be downgraded to 0. Therefore, following current works (c.g., (Kim & Han, 2023)), we hold that the larger domain similarity indicates the less domain information.
HSIC(K,L) vec(K) (K) vec(L) (N 1)2
(15)
Where vec(.) represents the vectorization operation and N the number input samples. Final calculation CKA: Finally, CKA calculates the similarity between two sets of feature representations by standardizing the HSIC. The calculation formula of CKA as follows
C. Sharpness-Aware Minimization
Due to the domain shift between source and target domains, the training loss value on source-domain datasets offers lim- ited assurances regarding the model's generalization ability. Therefore, merely optimizing the training loss value can steadily result in suboptimal model quality. Theoretically, from the sharpness of the loss landscapes, the sharper the minimum is the more vulnerable against the domain gaps
CKA(XY)= VHSICIXXT XX XXIICI
HSIC(XXT YYT)
(16) CKA is a standardized metric that represents the similarity of two Gram matrices and L. Since the Gram matrix reflects the feature relationship between the sample pairs,
14

---

Random Registers for Cross-Domain Few-Shot Learning
Table Comparison with more state-of the-art art works by 5-way 5-shot accuracy.
Method
Buckbone
FT
Mark
ChestX
ISIC2018
EuroSAT
CropDiseases
Average
MN AFA (Hu & Ma, 2022)
ResNet10
ECCV-22
23.18
39.88
69.63
80.07
53.19
GNN + FT (Tseng al., 2020)
ResNet10
ICLR-20
24.28
40.87
78.02
87.07
57.06
GNN ATA (Wang & Deng, 2021)
ResNet10
IICAI-21
24.32
44.91
83.75
90.59
60.39
LDP-net (Zhou et al., 2023)
ResNet10
CVPR-23
26.67
48.06
82.01
89.40
61.29
GNN + AFA (Hu & Ma, 2022)
ResNet10
X
ECCV-22
25.02
46.01
85,58
88.06
61.67
FLoR (Zou et al., 2024a)
ResNet10
CVPR-24
26.70
51.44
80,87
91.25
62.32
MEM-FS (Walsh al., 2023)
VIT-S
TIP-23
26.67
47.38
86.49
93.74
63.57
StyleAdv (Fu et al., 2023)
VIT-5
CVPR-23
26.97
47.73
88.57
94.85
64.53
FLoR (Zou et al., 2024a)
VIT-S
CVPR-24
26.71
49.52
90.41
95.28
65,48
DAMIM (Ma et al.. 2024)
VIT-S
AAAI-25
27.28
50.76
89.50
95.52
65.77
CD-CLS (Zou al.. b)
VIT-S
NeurIPS-24
27.23
50.46
91.04
95.68
66.10
AttnTemm (Zou et al.,
VIT-S
NeurlPS-24
27.72
53.09
90.13
95.53
REAP
VIT-S
Ours
27.98
52.80
90.53
95.68
66.75
PMF (Shell Xu, 2022)
VIT-S
CVPR-22
27.27
50.12
85.98
92.96
64.08
StyleAdv (Fu et al., 2023)
ViT-S
CVPR-23
26.97
51.23
90.12
95.99
66.08
FLoR (Zou et al., 2024a)
VIT-S
CVPR-24
27.02
53.06
90.75
96.47
66.83
DAMIM (Ma et al., 2024)
VIT-S
AAAI-25
27.82
54.86
91.18
96.34
67.55
CD-CLS (Zou
b)
VIT-S
NeurIPS-24
27.66
54.69
91.53
96.27
67.54
Attn (Zou et al., a)
ViT-S
NeurlPS-24
28.03
54.91
90.82
96.66
67.61
REAP
VIT-S
Ours
28.34
55.28
91.79
96.71
68.03
ConFeSS" (Das al., 2022)
ResNer10
ICLR-2022
27.09
48.85
84.65
88.88
62.37
LDP-net" (Zhou et al., 2023)
ResNet10
CVPR-23
26.88
48.44
84.05
91.89
62.82
RDC (Li et al., 2022)
ResNet10
CVPR-22
25.07
49.91
84,29
93.30
63.14
TPN ATA" (Wang & Deng. 2021)
ResNet10
IJCAI-21
24.74
49.83
85.47
93.56
63.40
MEM-FS RDA" (Walsh al., 2023)
VIT-S
TIP-23
$1.02
95.04
65.70
DAMIM" (Ma et al., 2024)
VIT-S
AAAI-25
28.10
55.44
91.08
96.49
67.78
CD-CLS' (Zou et al., b)
VIT-S
NeurlPS-22 24
28.25
$5.66
91.68
96.62
68.05
AttnTTmmp' (Zou al.,
VIT-S
NeurIPS-24
28.41
55.22
91.34
96.74
67.93
REAP"
VIT-S
Ours
56.07
96.74
68.38
Table Comparison study method with the haseline, image- perturbation, feature perturbation, weight perturbation, and atten- tion maps direct perturbation by the 5-way 5-shot accuracy.
= (8), is the number of parameters and assumed LD(w) < Ee-(0.p) [D + +)]. Following this criterion, measure the sharpness given perturbations model weights to compare the optimal loss achieved by those methods. the meantime, Sharpness Aware Mini- mization (SAM (Foret et al., 2021)) is introduced to improve the model's generalization ability, simultaneously minimiz- ing loss value and loss sharpness
different
Method
CropDiseases
FuroSAT
ISIC2018
ChestX
Ave.
Baseline
94.61
89.29
46.161em
26.21
64.07
imng.
14
89.73
46.30
26.542m17
64.43
feap
94,78
88.76
47.68+023
26.80=a.7
weight,
95. 12
89.73
46.42.oom
26.361m17
64.41
attn,
94.90
89.03=au7
46.09+02
25.90=au7
63.98
LSAM maz [L( + L(w)]
L(w)
(19)
Ours
95.68
90.53
52.80 10m
27.98 2n
66.75
Losr Valuee
The term enclosed in square brackets quantifies the sharp- ness by assessing how rapidly the training loss escalates when transitioning from w to a neighboring parameter value, The whole term aims at seeking parameters that reside in neighborhoods with uniformly low loss values rather than solely focusing parameters with low loss values them- selves, written
model will be. Specifically, the sharpness is measured as
Sharpness mazidd [L(w + L(w)). (17)
where refers to the model weights, refers the perturba- tion with the radius With this criterion, the generalization can be bounded as follows
(w) max 1el. Ls(Ww
min = min [maz]]l,5 + c)]
(20)
& logg(1 +
[|rl11 0² (1 log(n) )2) log n Ô(1)
To minimize LSAM, various perturbation approaches have been proposed and divided into three types: image perturba- tion (Foret et al., 2021), feature perturbation (Li et al., 2020), and weight perturbation (Liang et al., 2022). Image pertur- bation works by adding perturbation to the image ignoring
+
n
(18)
For any P 0 and any distribution D. with probability over the choice of the training
where
15

---

Random Registers for Few-Shot Learning
ChestX
ISIC2018
EuroSAT
CropDiseases
Image
Baseline
Ours
Image
Baseline
Ours
Image
Baseline
Ours
Figure The heatmap on larget domains proves that our approaches effectively improve the model's 's larget-domain performance.
the model's structure. Feature perturbation is not limited to the first layer, but can be flexibly used every layer of the model, while weight perturbation directly adding pertur- bations to the model parameters. We compare our methods with these three classical methods in Tab. 7. Due ignoring the idiosyncrasies of ViT, which highly robust to severe occlusions (Naseer et al., 2021) (e.g., random patch pertur- bations), all these perturbations are a little useful but not effective enough when compared with the baseline. The core structure of the ViT is the attention mechanism, which plays an essential role in the whole architecture (Chen et al., 2023a). Instead of perturbing the whole weight parame- ters, mainly focusing the attention maps may be good idea. However, directly adding perturbations to the attention maps harms the performance. Our method, working as new perturbation to attention maps indirectly, is proved to alleviate the domain shift problem in Tab. 7.
tion from source domains to target domains.
D.3. Applying the Learnable Registers to different
layers of the model
Ave. Accusicy
52.20%
52.10%
52.00%
$1.00%
Blook ID vs. shet peribomence
51.80%
VII's Block ID
Block ID of Adding Leamable Registers vs. I-shot performance
Figure 15. The model applied the learnable registers in the first block have the poorest performance target domains, validating that to registers' location, model tends to view the registers in the shallow layer (especially the input layer) a kind of domain-relevant pattern.
D. More Experiments
D.1. Comparison with more SOTAs
In Fig. 15, when we keep the registers number equal to 4, the model applied the learnable registers in the first block (input layer) performs poorest on target domains, proving that due the registers' location, the model tends to view the registers in the shallow layer (especially the input layer) as a kind of domain-relevant pattern.
As illustrated in Tab. 5 and Tab. 6, we conduct thorough comparison of various ViT and CNN-based approaches on CDFSL tasks. Our proposed methods consistently surpass all other approaches, achieving optimal performance. These findings emphasize the efficacy of our approach.
D.2. More Visualization of the attention maps
D.4. Similar Impact on deep prompts
The visualization of the attention maps in our model the target domain is shown in Fig. 14. Compared to the dispersed attention observed the baseline, our model fo- cuses on more valid and concentrated regions within the image, verifying that our approach improves the generaliza-
As shown in the Fig. 16, the model with deep prompts shows similar trends towards sharpness with the shallow types, quantitatively verifying that random registers improve the transferability of attention in both deep types or shallow types.
16

---

Random Registers for Cross-Domain Few-Shot Learning
proeuhh
poonpes
Figure The model that applies deep registers has similar pact sharpness.
66.00%
deep prompts
shallow prompts
Avc. Accuracy
65.40%
shallow prompts
64,80%
64.20%
63.60%
VII's Block
Block ID vs. 5-shot performance
Figure 17. The model applies shallow random registers from the imput layer and retains them until the final layer has the best per- formance target domains.
As depicted in the Fig. 17, uniformly the number of random registers to 1024 and add them the end of the token sequence using both deep and shallow approaches before feeding them into the VIT blocks. The shallow regis- ters without an asterisk (*) denote adding random registers starting from the nth layer, while those with an asterisk (*) indicate removal after the nth layer. Among these, the approach of adding shallow random registers from the in- put layer and retaining them until the final layer yields the optimal performance.
Table 8. Validations of REAP in both deep and shallow prompts source-domain training by 5-shot.
Method
CropDisease
BuroSAT
1SIC22018
ChestX
Aree
Baseline
46.
26.23
64.77
deep mindmm pronetts
23.40
27. ao
REAP w/ shallow randous prompts
05.68....
90.5
22.80
27. 20
66.75
As illustrated Tab. our method effectively enhances performance regardless of whether prompts are utilized in deep shallow mode.
17


Reviews:
[{"review_id": "tX6tvp3tqg", "reviewer": "Reviewer_zHqd", "summary": "Based on an intriguing observation that prompt tuning could be harmful for the generalization of ViT and the related analysis, this paper develops a novel solution for cross-domain few-shot learning method by replacing some clustered patches with random registers. Extensive experiments on four datasets demontrate the effecitiveness and superiority of the proposed method.\n\n## Update After Rebuttal\nI have checked the authors' rebuttal, and found most of my concerns have been solved, so I choose to keep my score as Weak accept.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer’s constructive feedback. Below are detailed responses to your questions:\n\n## **1. Clarification on Learnable Registers (Fig.5)**\n\n##### (1) Source-domain performance and visualization\n\n| Model              | Source-domain | Target-domain |\n| ------------------ | ------------- | ------------- |\n| Baseline           | 97.83         | 64.07         |\n| Learnable register | 97.87         | 63.17         |\n\nLearnable registers achieve slightly **higher source-domain accuracy** by **exploiting patterns (over)fitting the source domain**, so they can suffer in cross-domain generalization. For example, in Fig.5, birds are often seen with branches in the source domain, and the learnable register focuses on the bird and the branch (domain-specific contextual cues), which is the pattern that is only useful in the source domain (i.e., **overfitting to the source domain**) but may be useless in target domains. In contrast, random registers force attention to the bird itself, with less overfitting to the source domain, and thus benefit the target-domain generalization. \n\nOur visualization is designed to verify whether the model captures such domain-specific (object-irrelevant) patterns, the generalization is also quantitatively verified by the domain similarity in Fig.4 and sharpness in Fig.3.\n\n##### (2) Relation to [1]\n\n**[1] focuses on the in-domain training,** utilizing registers to capture the global information in the in-domain data, thereby reducing the outlier value in attention maps. In contrast, **our method focuses on the generalization to target domains**, by resisting the overfitting to in-domain data. Our observation is not contrary to the effect of learnable registers in resisting the outlier value in attention maps and is consistent with [1] in finding learnable registers can absorb in-domain information. We take a step further to identify such in-domain information as domain-specific information, which is further handled by our random registers.\n\n## **2. Sharpness Evaluation on Attention Maps**\n\nWe choose the attention map for the sharpness experiments because registers majorly interact with other tokens through the self-attention mechanism. In other parts of ViT, such as FFN, each token is processed separately and, therefore can hardly reflect the influence of registers. To verify it, we use LayerNorm and FFN perturbations to report the sharpness, and we can see **no significant influences** (Δ < 0.01) compared with the attention map.\n\n| Component | Baseline | Learnable register | Random register(Ours) |\n| :-------- | :------- | :----------------- | :-------------------- |\n| **Att.**  | 1.6      | 2.3                | 0.6                   |\n| **LN**    | 0.04     | 0.04               | 0.03                  |\n| **FFN**   | 0.06     | 0.07               | 0.06                  |\n\n*(Lower values indicate flatter minima and better generalization)*\n\n## **3. Source-domain performance comparison**\n\n REAP balances **moderate source-domain accuracy** with **significant target-domain gains** as below. A **1.5% source-domain drop** (which is acceptable) enables **+2.68% target-domain gain**, which is a favorable tradeoff for CDFSL.\n\n| Method             | Source-domain | Target-domain |\n| ------------------ | ------------- | ------------- |\n| Baseline           | 97.83         | 64.07         |\n| Learnable register | 97.87         | 63.17         |\n| Random register    | 97.50         | 65.05         |\n| **REAP(ours)**     | 96.33         | 66.75         |\n| Random-mask        | 96.28         | 60.90         |\n| Cluster-mask       | 94.23         | 64.29         |\n\n## **4. Generalization to Meta-Dataset**\n\nDue to time and resource constraints, we first pretrain on our datasets(miniImagenet), and then validate on parts of the Meta-Dataset under the 5-way 5-shot protocol below.\n\n| Dataset          | Baseline | REAP  | Δ         |\n| :--------------- | :------- | :---- | :-------- |\n| **Birds**        | 94.23    | 96.82 | **+2.59** |\n| **Fungi**        | 61.03    | 64.39 | **+3.36** |\n| **VGG Flower**   | 89.64    | 90.03 | **+0.39** |\n| **Traffic Sign** | 60.46    | 62.37 | **+1.91** |\n| **Ave***         | 76.34    | 78.40 | **+2.06** |\n\n## **5. Efficiency analysis**\n\n1. **Training Time**: REAP introduces **<7% additional training time** compared to the baseline (127.08s vs. 118.98s per epoch), attributable to the lightweight random register sampling.\n2. **Inference Cost**: Inference time **remains identical** to the baseline with no additional overhead. Architectural parity ensures seamless deployment in real-world applications.\n3. **Parameter Efficiency**: **Only one learnable standard deviation** is added to control register initialization.\n\nThis demonstrates REAP’s **lightweight design** achieves significant cross-domain gains with near-zero parameter and time penalties.\n\n## **6. Figure Readability Improvement**\n\nWe promise we will polish our paper in the final version.", "claims_and_evidence": "The claims in this paper are well supported by the related analysis and experimental results.\n\nHowever, I have the following questions:\n\n1. In Figure 5, we can see that adopting learnable registers can make model concentrate on regions irrelevant to the object, and random registers will guide the model's attention to the object. Does this mean that the learnable registers are useless or even have negative effects on the classification in the **source domain**? Is this observation conflict with the observation in the previous work [1]? \n\n2. Additionally, based on my own knowledge, the object-focused attention obtained in the **source domain** may be not a good indicator of better generalization ability, since it means that the model may capture more object related high-level semantic information, which can be hardly trasferred to the **target domain**. So could the authors provide more discussions of this observation to solve my confusion?\n\n[1] Darcet T, Oquab M, Mairal J, et al. Vision Transformers Need Registers[C]//The Twelfth International Conference on Learning Representations.", "methods_and_evaluation_criteria": "The proposed method is reasonable and easy to understand. The effects of all core components are studied by the experiments and ablation studies through well defined evaluation criteria.", "theoretical_claims": "The theoretical claims are inspired by the previous works and I have confirmed the correctness.\n\nI just have one small question:\n\n1. The authors evaluate the loss sharpness of different methods by adding Gaussian noises perturbations to the attention map as in Eq. (4). My question is why only choose the attention map for this study? In my opinion, the registers introduced at the input layer will influence all parts (e.g., layer normalization or feed-forward network) of the subsequent Transformer blocks, not just the attention map. Could the author give some explanations for such choice?", "experimental_designs_or_analyses": "The experiments and ablation studies are adequate, the related discussions and analyses are reasonable.\n\nDespite this, I have some concerns about the experimental results:\n\n1. As a cross-domain method, it is better to achieve higher classification accuracy for the target domain samples **without**  severely sacrificing performance on the source domain. Moreover, Figure 5 shows the random registers may lead to better results on the source domain. So could the authors compare the results of the source miniImageNet dataset of different methods for validating the claim of Figure 5?\n\n2. Could the proposed method be generalized to a more complex and diverse dataset, namely Meta-dataset which is a widely used benchmark for cross-domain fow-shot, under a more difficult varied-way varied-shot setting?\n\n3. It is better to perform comparison about the training/inference time and parameter size to show the effciency of the introduced random registers.", "relation_to_broader_scientific_literature": "The key contributions of this paper may be related to many research areas, including domain generalization, domain adaptation, transfer learning, etc, and many practical applications, such as the few-shot classification for medical, remote sensing, or agriculture images.", "essential_references_not_discussed": "The related works are appropriately cited.", "other_strengths_and_weaknesses": "Strengths:\n\n1. This paper is clearly presented with a good organization, the method is well motivated and easy to understand.\n\n2. The observation is interesting and the discussions seem to be reasonable.\n\n3. The contributions provide a new insight for understanding the domain generalization of ViT model.\n\nWeaknesses: \n\nPlease refer to the questions in \"Claims And Evidence\", \"Theoretical Claims\" and \"Experimental Designs Or Analyses\".", "other_comments_or_suggestions": "It may be better to enlarge the figures to make the texts in them more readbale.", "questions_for_authors": "Please refer to the questions in \"Claims And Evidence\", \"Theoretical Claims\" and \"Experimental Designs Or Analyses\".", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tpcgvVDslK", "reviewer": "Reviewer_UKT5", "summary": "This paper deals with the Cross-domain few-shot learning (CDFSL) problem, which needs to tackle the huge domain gaps. Existing methods utilizing learnable prompts might learn domain specific information of the source domain, while fail to generalize to the distant target domains. This paper proposed to leverage random prompts as an effective method to solve this issues. Furthermore, the random prompts are well interpreted with the Sharpness-Aware Minimisation (SAM) and analysis. \n\nBased on the random prompt idea, Registers Enhanced Attention Perturbation (REAP) is proposed to pertube both the image tokens and adding random noisy tokens at source-domain stage, which further increases the generalization to target domains. \n\nExperiments are conducted on the standard CDFSL benchmarks. The propposed method consistently outperforms state-of-the-art methods. Comprehensive ablation study and parameter analysis are conducted. \n\n## update after rebuttal\nThe random prompt idea is new and novel in the context of CDFSL. The explanation using Sharpness-Aware Minimisation (SAM) is reasonable and insightful.   \nNo further concern remains after rebuttal.   \nTherefore, I will keep my score as Accept.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your thorough and insightful review of our submission.   Your recognition of our work’s **innovative integration of random prompts with SAM principles** and its **practical value in addressing extreme domain gaps** is deeply encouraging.\n\nWe will keep on polishing our paper in the final version. Thank you again for your appreciation!", "claims_and_evidence": "Yes. The main claim is random prompts can improve the generalization to target domains, while the learnable prompts impede the generalization. This claim is well interpreted and verified experimentally and analytically with reasonable formulation of Sharpness-Aware Minimisation.", "methods_and_evaluation_criteria": "The proposed method is innovative, which consists of perturb the attention of image tokens, and additional random register tokens. The proposed method and perspective is new in the CDFSL generalization viewpoint. \n\nEvaluation benchmark and criteria are standard.", "theoretical_claims": "No proof in this paper.", "experimental_designs_or_analyses": "The experiments are sound and comprehensive, which involves SOTA comparison, ablation study of the proposed components, and analysis of important parameters of the model such as number of tokens, ratio of pertub, etc. \n\nMoreover, visualization comparison is also presented for random vs. learnable tokens.", "relation_to_broader_scientific_literature": "Prior findings leverage prompt tuning for downstream task finetuing, i.e. learnable tokens. This usually works for near domain generalization or relevant tasks. However, CDFSL tackles the distant domain with huge domai gap. This paper challenged the learnable token method and proposed a new random register method for CDFSL. \n\nWhat's more, the interpretation perspective is novel. It attributes the effectiveness of random register to Sharpness Aware Minimisation, building theoretical support for random register. \n\nThe proposed idea is an extension and further exploration of Registers in (Darcet et al., 2024).", "essential_references_not_discussed": "Yes.", "other_strengths_and_weaknesses": "Details are provided in above section.", "other_comments_or_suggestions": "None.", "questions_for_authors": "None.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hWmu7tBiaI", "reviewer": "Reviewer_cSmv", "summary": "In this work, the authors propose a two-stage learning framework namely REAP to tackle CDFSL problem. During the source domain training, REAP randomly masks the most discriminative region and fill the erased region by random prompts, then optimize the pretrained ViT on source data. And during target domain fine-tuning, REAP optimizes the learnable prompt to adapt to target data. The proposed REAP achieves comparable performance among all methods on multiple benchmarks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your suggestion.\n\n## **1. Claims**\n\n[1] studies the effectiveness of **prompt learning in in-domain data**, while we specifically target **extreme cross-domain shifts** (e.g., natural images to satellite images), where standard prompt tuning fails. As shown below, REAP outperforms [1] by **+30%** under 5-shot source domain pretraining settings and by **+9%** under target domain finetuning. To further validate REAP’s effectiveness, we conducted extensive comparisons with **prominent prompt tuning methods** (e.g., VPT, CoOp, MaPLe) across extreme cross-domain benchmarks.\n\n| Source-domain training | Cropdiseases | EuroSAT   | ISIC      | ChestX    | Ave.      |\n| ---------------------- | ------------ | --------- | --------- | --------- | --------- |\n| [1]CoOp                | 38.48        | 54.15     | 25.74     | 21.36     | 34.93     |\n| COCOOP                 | 40.36        | 60.93     | 27.38     | 22.47     | 37.79     |\n| MaPLe                  | 35.28        | 50.83     | 23.65     | 19.63     | 32.35     |\n| VPT                    | 77.92        | 75.93     | 49.89     | 24.30     | 57.01     |\n| **REAP**               | **96.68**    | **90.76** | **55.76** | **26.84** | **67.51** |\n\n| Target-domain finetuning | Cropdiseases | EuroSAT   | ISIC      | ChestX    | Ave.      |\n| ------------------------ | ------------ | --------- | --------- | --------- | --------- |\n| [1]CoOp                  | 92.82        | 84.92     | 42.93     | 22.83     | 60.88     |\n| COCOOP                   | 90.57        | 81.32     | 42.15     | 21.89     | 58.98     |\n| MaPLe                    | 93.07        | 89.35     | 46.56     | 23.16     | 63.04     |\n| VPT                      | 94.80        | 89.48     | 46.35     | 26.40     | 64.26     |\n| **REAP**                 | **98.35**    | **92.64** | **58.28** | **29.21** | **69.62** |\n\nWe can see existing prompts optimize for *in-domain* adaptation and harm the performance when facing the huge domain gap, while REAP’s **random register perturbation** explicitly **alleviates the domain gap**. Similar results in Fig.1, Fig.3, and Fig.4 also verify that the regular prompts are not enough for handling the CDFSL task.\n\n## **2. Methodological Novelty**\n\nWhile prior works (e.g., VPT) naively add learnable prompts, we are the **first to find that it harms the transferability** to target domains, and we **theoretically design the random register** and prove it **suppresses domain-specific attention patterns**. Based on it, we propose a novel method, REAP, to enhance perturbations on attention maps. This mechanism is novel, as noted by Reviewer zHqd and UKT5: *\"The proposed method is innovative... new in the CDFSL generalization viewpoint.\"*\n\n## **3. Sharpness-aware minimization**\n\n**(1) Performance Validation**\nThe experimental results (Tab. 3) demonstrate significant performance improvements (e.g., **+4.6%** on ISIC), which empirically confirm that the introduced randomness is **well-calibrated and beneficial** rather than \"excessive\" or detrimental. \n\n**(2) Sharpness Validation**\nThe sharpness analysis in **Fig. 3b** explicitly validates that our method reduces loss sharpness by **75%** compared to vanilla VPT, indicating that the random registers **enhance flat minima discovery** without violating SAM’s core principles. This demonstrates that our design **extends** SAM’s hypothesis to cross-domain scenarios rather than conflicting with it.\n\n**(3) Perturbation magnitude**\nRandom registers are initialized with a near-zero magnitude (σ=0.01) and adaptively scaled during training. Visualization in **Fig. 5** further confirms that their impact on attention maps is **subtle**, primarily acting as \"regularized perturbations\" to suppress domain-specific biases rather than overwhelming the original features. This design is consistent with SAM’s small-perturbation premise while addressing cross-domain challenges.\n\nIn all, our design *extends* SAM to cross-domain settings without violating its core hypothesis.\n\n## **4. Experimental Completeness**\n\nThe requested analyses **are fully provided in the paper**:\n\n**Ablation: Simple Masking vs. Random Registers**\n\n- **Table 3a**: Naive masking (no registers) drops accuracy by **5.8%**.\n\n**VPT Comparison**\n\n- **Fig. 1** has already shown that VPT harms performance during pretraining while random registers improve model transferability.\n\n**Initialization Analysis**\n\n- **Supp. Fig. 17**: Gaussian initialization of the random registers.\n\n## **5. Clarification on VPT Similarity**\n\nWhile the reviewer notes \"similarity with VPT\", we **explicitly differentiate**:\n\n- **Motivation (Fig.1)**:  We are the **first to find** that **VPT fails under extreme domain gaps** due to *absorbing domain information*, and **propose random registers** to solve it.\n- **Solution**: We come up with **REAP** to replace *fixed learnable prompts* with **random registers** during source training.\n- **Result**: REAP outperforms VPT by **4%** on distant domains.", "claims_and_evidence": "The claim is relative weak, previous work [1] manages to discuss some similar experimental results regarding learning prompts on source dataset and direct evaluation on target datasets, but prompt tuning is enough to tackle this issue. \n\n[1] Learning to Prompt for Vision-Language Models, IJCV", "methods_and_evaluation_criteria": "Though the authors discuss the problem from sharpness aware minimization aspect, the proposed method is still a vanilla pretraining on source and prompt tuning on target paradigm. The reviewer still doubt the novelty of the proposed method.", "theoretical_claims": "The theoretical claim is also weak. The perturbation ϵ is relative small and ensures that ω+ ϵ is nearby ω in the loss landscape. However, the proposed method introduce too many random registers to construct learn more domain-agnostic information, which is conflict with original SAM hypothesis.", "experimental_designs_or_analyses": "The experimental results are not sufficient, some important analysis is lacked. For example, the ablation study of random erasing strategy (i.e., simply masking but not using random registers to fill), initialization methods of random registry, and comparison between the proposed method and VPT.", "relation_to_broader_scientific_literature": "The proposed method largely related to few-shot learning and prompt tuning, but has limited contribution to future work of these topics.", "essential_references_not_discussed": "[1] Visual Prompt Tuning, ECCV 2022", "other_strengths_and_weaknesses": "N/A.", "other_comments_or_suggestions": "See all above weakness for details.", "questions_for_authors": "We recommend the authors to clarify the different between VPT (with backbone pretraining) and the proposed method. The proposed method has large similarity with VPT, which may limit the novelty of proposed method, though this work try to investigate the performance issue from SAM aspect.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This work is aimed at cross-domain few shot learning, where a vision transformer is first trained on a source domain, and then fine-tuned on the target domain with few-shot data. 

This work finds that adding learnable registers during source-domain learning harms the transfer to target domain, while "random" (noise) tokens do not suffer from this. Based on this, they propose a technique where during source-domain training, clusters of visual tokens are replaced with random "registers" (effectively a regularization / reduces overfitting), and learnable registers/tokens are added during target fine-tuning. This is demonstrated to help in cross-domain transfer, and is competitive with existing methods.

The analysis of the phenomena was found interesting by the reviewers who recommend this work for publication. Hence, I recommend acceptance of this work.
Decision: Accept (poster)