Dual-Kernel Adapter: Expanding Spatial Horizons for Data-Constrained Medical Image Analysis
Authors: Ziquan Zhu
Hanruo Zhu
Si-Yuan Lu
Xiang Li
Yanda Meng
Yunxiao Zhang
Gaojie Jin
Lu Yin
Lijie Hu
Di Wang
Lu Liu
Tianjin Huang
Venue: ICLR 2026 Poster (2026)
Keywords: Adapter; Medical Image Analysis; Data-Limited Training;
Primary area: transfer learning, meta learning, and lifelong learning
TL;DR: We propose the DKA, a lightweight module that effectively expands the receptive field while preserving local detail, achieving state-of-the-art performance in data-limited medical imaging tasks across diverse benchmarks.

Abstract: Adapters have become a widely adopted strategy for efficient fine-tuning of foundation models, particularly in resource-constrained settings. However, their performance under extreme data scarcity—common in medical imaging due to high annotation costs, privacy regulations, and fragmented datasets—remains underexplored. In this work, we present the first comprehensive study of adapter-based fine-tuning for vision foundation models in low-data medical imaging scenarios. We find that, contrary to their promise, conventional Adapters can degrade performance under severe data constraints, performing even worse than simple linear probing when trained on less than 1\% of the corresponding training data. Through systematic analysis, we identify a sharp reduction in Effective Receptive Field (ERF) as a key factor behind this degradation. Motivated by these findings, we propose the Dual-Kernel Adapter (DKA), a lightweight module that expands spatial context via large-kernel convolutions while preserving local detail with small-kernel counterparts. Extensive experiments across diverse classification and segmentation benchmarks show that DKA significantly outperforms existing Adapter methods, establishing new leading results in both data-constrained and data-rich regimes.

Published as a conference paper at ICLR 2026
DUAL-KERNEI ADAPTER: EXPANDING SPATIAL HORIZONS FOR DATA-CONSTRAINED MEDICAL IMAGE ANALYSIS
Ziquan Zhu'* Hanruo Zhu ,Si Si-YYunn Lu Xiang Li,, Yanda Meng", Gaojie Jin Lu Yin Lijie Hu", Di Wang", , Lu Liu Tianjin Huang
Lu
School Computing and Mathematical Sciences, University of Leicester, Leicester, UK School Communications and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing, China
3 Department of Computer Science, University of Bristol, Bristol, UK Department of Bioengineering, King Abdullah University of Science and Technology, Thuwal,
Saudi Arabia
Department of Computer Science, University of Exeter, Exeter, UK
Computer Science Research Centre, University of Surrey, Guildford, UK Machine Learning Department, Mohamed bin Zayed University of Artificial Intelligence, Abu
Dhabi, UAE
8 Division of Computer, Electrical and Mathematical Sciences and Engineering, King Abdullah University of Science and Technology, Thuwal, Saudi Arabia 9 Department of Mathematics and Computer Science, Eindhoven University of Technology, Eind- hoven, NL
ABSTRACT
Adapters have become widely adopted strategy for efficient fine-tuning of large pretrained models, particularly in resource-constrainggg performance under extreme data scarcity comeity-common in medical imaging due high annotation costs, privacy regulations, and fragmented datasets remains underex- plored. In this work, we present the first comprehensive study of adapter- based fine-tuning for large pretrained models in low-data medical imaging scenarios. We find that, contrary to their promise, conventional Adapters can degrade performance under severe data constraints, performing even worse than simple linear probing when trained on less than 1% of the corresponding training data. Through sys- tematic analysis, we identify sharp reduction in Effective Receptive Field (ERF) as a key factor behind this degradation. Motivated by these findings, we propose the Dual-Kernel Adapter (DKA), lightweight module that expands spatial con- text via large-kernel convolutions while preserving local detail with small-kernel counterparts. Extensive experiments across diverse classification and segmentation benchmarks show that DKA significantly outperforms existing Adapter methods, establishing new leading results both data-constrained and data-rich regimes. Code is available at /tltts://gitain com/ mi http///isswaaggy/gg....
settings. However, their
INTRODUCTION
The rapid proliferation of large pretrained models has significantly advanced various fields such as natural language processing (Chowdhary Chowdhary, 2020) and computer vision (Szeliski
2022). yet it has also amplified challenges related to computational overhead (Thompson et al. 2020).
memory consumption (Mahendran) 2021), and the complexity of downstream adaptation (Jiang et al.
2024). especially when deploying these models in specialized domains like medical imaging (Strubell
et al.
2020 Ji et al. 2021 Wang et al.] 2025). To address these issues, adapter-based fine-tuning
(Wang et al. 2020b
Wu
al. 2025 Gong et al. 2023 Hu et al. 2024 Chen et al. 2024) has
"Equal contribution.
"ooresponning author: T.Huang2 [.Huug2@exxteraac.tt

---

Published as a conference paper at ICLR 2026
emerged popular strategy, enabling efficient adaptation by adjusting only small subset of model parameters rather than performing full fine-tuning.
In medical imaging, assembling large, well-annotasted datasets is notoriously costly: expert radiolo- gists must painstakingly delineate structures in nigh-resolution 2-D and 3-D scans, and inter-observer variability further inflates the annotation burden (Suzuki 2017 Ritter et al. 2011). Strict privacy regulations such as HIPAA (U.S. Department of Health & Human Services 2003) and the GDPR (Fu ropean Union 2016), coupled with heterogeneous institutional policies, severely limit data sharing, fragmenting what little data exist (Schäter et al. 2023 Wang et al. 2021). As a result, many clinically important downstream tasks still operate in a pronounced low-data regime. This scenario naturally
leads to the critical question:
Can standard Adapter perform effectively in medical imaging tasks under constrained data?
In this paper, we first provide comprehensive evaluation and analysis of applying conventional Adapter (Houlsby et al. 2019) under constrained-data scenarios across various datasets and backbone architectures. Our study reveals several key insights:
VIT.B
Swin T
Swin BB
10
10
ACC (96)
5
-10
-5
0.63 1.25 12.5
25
30
75
100
0.63 1.25 12.5 25 50
75
100
0.63 1.25 12.5
25 50 75 100
Training Set (%)
Training Sot (%)
Training Set (%)
Tiny ImageNet
CIFAR-100
COVID
BUSI
2019-2019
Figure
Performance of Adapter across Various Training Data Sizes.
AACC
ACC Linear Probing + Adapteer ACC LinearProbinng Experiments are conducted on the pretrained ViT- B, Swin-T, and Swin-B backbones for Tiny ImageNet and CIFAR-100 (In-Domain), COVID, BUSI,
and 1S19-2019
Data Size Significantly Impacts Adapter Performance. We observe that the benefits of using Adapters diminish substantially as the size of the training dataset decreases. This effect is particu- larly pronounced for medical imaging (see Figure D.
Adapters Can Harm Performance Under Severe Low Data Setriigs. When the training data size is reduced to 1% or less, specifically in the context of adapting large pretrained models to medical imaging, Adapters perform worse than linear probing. This indicates that, under extreme data constraints, Adapters may negatively affect model adaptation (see Figure
Effective Receptive Field (ERF) Decreases with Reduced Training Data. Visualization of the ERF demonstrates that smaller training datasets result in reduced ERF, offering a plausible explanation for the observed performance degradation (see Figure 2
Inspired by these insights, we propose Dual-Kernel Adapter (DKA) that explicitly enlarges the ERF of standard Adapter modules. Each DKA. module introduces dual-branch convolution design: one branch leverages a depthwise large-kernel convolution to broaden the ERF, while the other employs a depthwise small-kernel convolution to preserve fine- fine-grained local details. We evaluate DKA on a range of medical imaging tasks, including both classification and segmentation, across diverse datasets and large pretrained models. Experimental results demonstrate that DKA consistently outperforms existing methods in both data-constrained and data-rich settings.
Summary of Contributions
We present the first systematic study of adapter- based fine-tuning for large pretrained models in low-data medical-ing -maeinnggiiinggggg scenarios, showing that the conventional Adapter can actually degrade performance under severe data searcity.
2

---

Published as a conference paper at ICLR 2026
We introduce the Dual-Kerne Adapter (DKA), lightweight module that pairs large- and amall-receptive-fiie convolutions in parallel, simultancously broadening spatial context and pre- serving fine-grained detail.
* Extensive experiments on multiple segmentation and classification benchmarks demonstrate that DKA sets new state of the art, and ablation studies reveal that using asynchronous learning rates between adapters and linear head is critical to its gains.
UNDERSTANDING STANDARD ADAPTER IN CONSTRAINED-DATA SETTINGS
Adapters have gained popularity in medical image analysis due to their parameter efficiency and adaptability for fine-tuning large pretrained models. However, their performance under constrained data conditions remains underexplored. In this section, we investigate this critical aspect using a diverse set of datasets, including Tiny ImageNet (Le & Yang 2013), CIFAR-100 (Sharma al. 2018). COVID (Chowdhury et al. 2020), BUSI (Zhang et al., (022). and ISIC-2019 (Gessert et al. 2020 Our experiments utilize three backbones- VVT-B (Dosovitskiy et al. 202T). Swin (Liu et al. 202T), and Swin-B (Liu et al. 2021)-al all pretrained on the ImageNet (Deng et al. 2009), to evaluate the impact of Adapters across varying data sizes, ranging from 0.63% to 100% of the training data. We report the difference in accuracy between the application of Adapters and the non-application of Adapters, which is denoted by AACC = observations are summarized as follows.
Adopen ACC Probing. Our key
① Degraded Adapter Performance with Less Training Data. In Figure | we consistently observe that the performance gains provided by Adapters diminish across multiple tasks and pretrained models. Notably, this decline is significantly more pronounced in medical datasets (COVID, BUSI, ISIC-2019) compared to natural-vision datasets (TinylmageNet, CIFAR- A plausible explana- tion is that medical tasks represent out-of-domain scenarios, requiring feature representations that diverge considerably from those Iearned by the original pretrained models. This challenge is further exacerbated in low-data settings, where learning domain-specific features becomes more difficult. In contrast, natural-vision tasks remain largely in-domain, aligning more closely with the pretraining distribution, and thus benefiting more from adapter -based fine-tuning.
Negative Effects of Adapters in Extremely Low Training Data in Medical Imaging. We further observe that in medical imaging tasks, when training data is limited to 1% or less, the performance gain from applying Adapters becomes negative, indicating detrimental impact on model performance. Unlike natural images, medical images typically exhibit low contrast, ambiguous boundaries, and small or irregular pathological structures (Zhang et al. 2024), which usually demand large effective receptive field (ERF) to capture long-range contextual dependencies. However, standard Adapter do not possess a strong inductive bias toward expanding the ERF. We hypothesize that under limited supervision, this limitation restricts the Adapter's 's capacity learn spatially dispersed features and long-range contextual dependence, thereby contributing to the observed degradation in performance.
Reduced Effective Receptive Field Under Constrained Training Data. To validate whether reduced supervision limits the Adapter's ability to capture spatially dispersed features and long- range contextual dependencies, we visualize the ERF of Adapters trained on varying proportions of the training set, ranging from 0.63% 100%, using the COVID dataset and the pretrained ViT-B model. Following the definition in (Araujo et al. 2019). the ERF of neural network layer refers to the region encompassing all input pixels that exert a negliggli influence on a given output unit. As shown in Figure 2 the ERF becomes progressively smaller as the amount of training data decreases. This observation supports our hypothesis that limited supervision restricts the Adapter's ability to learn spatially diverse patterns and long-range contextual relationships, which are particularly crucial in medical imaging tasks. We provide complementary ERF analysis on an alternative backbone (Swin-T) in Appendix F4 which leads to consistent conclusions, reinforeing the generality of our observations.
These findings indicate that standard Adapter fail to enlarge the ERF in low-data settings, which in turn degrades model performance. Consequently, it is crucial to develop new Adapter architectures with built-in inductive bias toward expanding the ERF.
3

---

Published as a conference paper at ICLR 2026
0.63%
12.5%
25%
50%
100%
0.6
0.6
0.6
0.6
04
0.4
0.4
0.4
02
0.2
022
0.2
02
Figure 2: Effective Receptive Field of Standard Adapters Across Varying Training Set Ratios. Experiments are conducted on the COVID dataset using the pretrained ViT-B.
3 DUAL-KERNEL ADAPTER
To mitigate the adverse effects of standard Adapter in low-data regimes, we propose the Dual-Kernel Adapter (DKA), which explicitly integrates large-kernel convolution ex- pand the ERF and small-kernel convolution to preserve local spatial details. Prior studies have demonstrated that large kernels introduce a strong inductive bias toward capturing broader contextual
information by expanding the ERF (Huang et al. 2023 Liu et al. 2022; Ding et al. 2022).
As illustrated Figure DKA first reduces the dimensionality of the input features through linear down-projection. After this step, the patch tokens are reshaped back to their 2D spatial layout so that depthwise convolutions can be applied in the image domain. The reduced features are then processed in parallel by two depthwise convolution branches- computationally efficient form of convolution that applies distinct filter to each input channel (Chollet) 2017). One branch employs a large kernel to significantly enlarge the receptive field, facilitating the modeling of long-range dependencies. The other branch uses smaller kernel to retain fine-grained spatial features essential for capturing localized structures. The outputs from the two branches are aggregated via element-wiss summation, followed by GELU activation, a linear up-projection, and a residual connection to the input.
Formally, the DKA operation can be expressed as:
fuxa( (x)
(DoC(((D (Down(z:) DWConw small (Down(z)))) (Down()
where Down(-) and Up(-) denote linear projection layers, DWConViarge and DW DWConV depth- wise convolutions with kernel sizes 51 and 5 respectively, and a is the GELU activation.
4 EXPERIMENTS
Trainable
Head
Frozen
Nx
To comprehensively evaluate the performance of DKA, we conduct extensive experiments on medical imaging tasks, spanning classification and segmentation benchmarks across diverse datasets, and additionally, models pretrained both natural and medical domains.
Depthwise Convolution
TKA
Add & Norm
Feed Forwird
Upsamppe
Pretrained Models. We consider two rep- resentative categories of pretrained models. Natural--retraaneed Models. For classification, we conduct experiments using ViT-B (Dosovit- skiy et al. 2021) and Swin-B (Liu al.) (021).
Activation
DKA
Upsample
Add Norn
51x51
5-5 DW
Activation
DW
Multi-Head Attention
both pretraned on the ImageNet. For segmenta- tion, we adopt Segmenter-B (Strudel et al. 2021) in the Open M.SSegmentation (Con- tributors 2020). preiical-rettaanedd Models. To assess domain adaptability, we Turther eval- uate DKA on medical backbones, including RadImageNet-prette pretrriind ResNet-50 (Mei et al. 2022) for classification and MedSAM (Ma et al.
Downsmmple
Dowasample
Embedding
(a) Standard Adapter
(b) DKA
(c) DKA Fine-tuning
Figure 3: Overview of the DKA Module. (a) Stan- dard Adapter. (b) The proposed DKA module. (c) DKA Fine-tuning.
2024 for segmentation.
4

---

Published as a conference paper at ICLR 2026
Datasets and Metrics. We evaluate the proposed DKA across both medical classification and segmentation tasks to ensure broad applicability. Medical Image Classification. We utilize three widely-adopted medical image classification datasets: COVID (Chowdhury et al., 2020p, BUSI (Zhang et al. 2022), and ISIC-2019 (Gessertt et al. 2020). The COVID dataset comprises chest X-ray images for COVID-19 diagnosis. The BUST dataset includes breast ultrasound images categorized as benign, malignant, or normal. The ISIC-2019 dataset is a large-scale dermoscopic dataset designed for multi-class skin lesion classification. For each dataset, we report Top-I Accuracy (ACC), FI Score (FI); and Sensitivity (SEN) under varying proportions of training data. Medical Image Segmentation. We further assess DKA on three representative segmentation datasets: BRATS (Menze al.[[201)). BUSI (Zhang et al.) 2022), and ISIC-2018 8 (Codella et al., 2019). BRATS focuses on brain tumor segmentation using multi-modal MRI scans. BUSI provides breast ultrasound images with annotated tumor regions. ISIC-2018 contains dermoscopic images with pixel-level annotations for skin lesion segmentation. We evaluate segmentation performance using standard metrics: mean Intersection over Union (mloU) and Dice coefficient (Dice). More details are provided in Appendix B.1
Baselines. We compare DKA against a broad range of baselines, categorized into three groups: Standard Fine-tuning Methods: (1) Linear Probing: freezing the backbone and tuning only the head; (2) Full Fine-tuning: updating all model parameters during downstream training. Adapter-tuning Methods: (1) Adapter (Houlsby et al. 2 adding adapters within each trans-
former block; (2) AdapterFormer (Chnn et al. 2022]: adding parallel adapters with learnable scaling to each MLP layer; (3) CIAT (Zhu et al.] 2027]: adding adapters before transformer blocks and parallel adapters within blocks; (4) Comppuss (Jie et al. 2024): enhancing adapters with parallel x 3
convolution branches; (5) AIM (Yang el al.) 2023): combining sequential and parallel adapters across spatial and temporal attention pathways. Other Farameter-efficien Fine-tuning (PEFT) Methods: (1) Prompt Tuning (Jia et al., 2022): fine-tuning extra learnable tokens; (2) LoRA (Hu et al. 2022):
injecting low-rank trainable matrices into attention modules; (3) BitFit (Zaken et al. 2027 tine-tuning only the bias terms of pretrained models.
Implementation Details. Following the common training protocol, we freeze all pretrained model weights and update only the parameters of DKA and head during fine-tuning. Specifically, DKA modules are inserted within transformer blocks following the placement strategy described in (Yin) al. 2024). For the DKA module, the middle dimension d is set 16 for classification tasks and 192 for segmentation tasks, as discussed in Section 4.5 The learning rates are set to le-4 for the task head and le-3 for the DKA modules, as shown in Section 4.4 Classification models are trained for 100 epochs, while segmentation models are trained for 300 epochs, balancing the need for convergence across task complexities. For experiments under constrained supervision (i.e., training with less than 100% of the training set), we perform a 5-fold cross-validattin on the training set while keeping the test fixed. All reported results are averaged across the prossvvliidation folds. More details can be found in Appendix B.2
Table 1: Comparison of Baselines and DKA on Three Classification Datasets Across Varying Data Sizes. Results are reported in terms of ACC (%). Experiments are based on the pretrained VIT-B. The best results are highlighted in bold.
COVID
BUSI
ISIC-2019
Methods
0.63%
1.25%
100%
0.63%
1.25%
100%
0.63%
1.25%
100%
Full Fine-tuning
87.43
88.00
98.43
71.17
76.73
94.62
60.04
61.21
82.05
Linear Probing
86.84
87.50
94.85
73.48
77.64
89.78
59.15
59.44
71.83
BitFit (Zaeen et al.
2021
73.91
79.65
96.95
57.19
60.20
88.27
50.80
53.22
79.84
Prompi Jia et al.
2022
77.91
83.75
98.45
61.34
64.30
93.07
52.55
53.59
81.02
LoRA Hu et al.
2022
80.43
85.91
98.73
63.64
67.41
94.75
51.08
53.96
81.75
Adapter Houllby
2019)
83.29
86.26
98.33
63.18
73.68
93.33
52.77
54.88
79.54
Adapterformer (Chen
al.
2022
82.46
84.32
98.18
63.42
72.75
92.65
51.62
52.71
78.19
Convpass (Jie et all
2024
84.72
86,94
98.45
64.83
74.63
93.97
54.72
56.25
80.45
CIAT (Zhu ct al.
2021
77.34
82.85
96.54
60.28
65.07
89.86
48.35
49.96
72.18
AIM (Yang et al.
2023
80.92
83.55
97.23
62.72
70.34
90.12
50.12
52.72
77.39
DKA
89.01
91.06
99.21
74.23
79.46
95.89
60.52
62.32
83.09
5

---

Published as a conference paper at ICLR 2026
Table 2: Comparison of Baselines and DKA on Three Segmentation Datasets Under Varying
Data Sizes. Experiments are based on the pretrained Segmenter-B B. mloU is reported as percentages. The best results are highlighted in bold.
BRATS
BUSI
ISIC-2018
Methods
0.63%
1.25%
100%
0.63%
1.25%
100%
0.63%
1.25%
100%
Full Fine-tuning
9.25
22.39
73.08
26.67
32.31
57.41
62.27
73.63
77.58
Linear Probing
7.95
20.20
69.86
25.53
31.96
54.07
60.90
71.06
74.10
BitFit Zaken et al.
2021
1.20
14.33
63.52
7.13
17.08
52.56
53.21
65.84
73.10
Prompt Jia et
2022
1.22
15.21
64.53
9.19
18.77
53.57
56.56
67.40
73.61
LoRA Hu et al.
2022
3.84
16.19
68.48
14.10
22.39
53,96
58.70
69.03
73.84
Adapter [Houlsby cl a
2019
6.16
18.95
72.02
18.18
25.90
55.01
59.78
72.80
76.71
Adapterformer (Chen c a.
2022
5.99
18.77
72.54
17.41
25.68
55.14
59.67
72.85
76.58
Convpass [Tie
2024
7.13
19.64
73.32
19.84
28.52
56.09
60.19
73.56
77.54
CIAT (Zhu 202T
3.80
16.81
70.41
13.40
20.20
54.76
58.26
69.52
75.09
AIM Yang el al. 2023
4.58
17.61
71.98
15.40
23.54
54.78
59.24
72.05
76.65
DKA
9.47
23.02
74.96
26.85
34.52
58.90
63.13
74.27
78.53
4.1 SUPERIOR PERFORMANCE ON CONSTRAINED DATA
We evaluate DKA under constrained-dar settings (0.63% and 1.25%) and the full-data setting. Experiments are organized by the type of pretrained backbone; natural-image models (VIT-B, Segmenter-B) and medical-image models RRadImagNNet-ReSNeett clear examination of DKA's generalization across different pretraining sources.
MedSAM), This design enables
Natural-prett pretraidde Models. We first evaluate DKA using natural-pretrained backbones, as sum- marized in Tables D and 2 DKA consistently outperforms all baselines across classification and segmentation tasks. Notably, under low-data regimes (0.63% and 1.25%), DKA even surpasses full fine-tuning and linear probing, while other PEFT methods exhibit clear performance degradation. These observations confirm the effectiveness of DKA when adapting natural- pretraiid models to medical domains. Additional results the natural- pretrained ViT-B and Segmenter-B - are provided in Appendix a Furthermore, similar performance improvements are observed when DKA is applied to other natural-prest pretrannde models such Swin-B (see Appendix F.T for details).
Medical-pretrrained Models. We further evaluate DKA on medical-ppetraiine backbones, including
RadImageNNetpretraiine ResNet-50 (Mei al. 2022) for classification and MedSAM (Ma al. 2024) for segmentation. As reported in Table 3 DKA consistently surpasses full fine-tuning, linear probing, and other PEFT baselines across ISIC ISI9-2019 and BUSI under all data scales. These im- provements demonstrate that the gains of DKA are not limited to natural image pretraining, but also generalize effectively to comain-specinfe medical large pretrained models. More results are reported in Appendix
Table 3: Performance of DKA with -retraind models. (a) Classification results on the ISIC-2019 using preeeanee ResNet-50 by reporting ACC (%). (b) Segmentation results on the BUSI using MedSAM by reporting mloU (%). The best results highlighted in bold.
(a) Classification Results.
(b) Segmentation Results.
Methods
0.63%
1.25%
100%
Methods
0.63%
1.25%
100%
Full Fine-tuning
52.70
55.17
76.63
Full Fine-tuning
36.21
45.04
70.62
Linear Probing
51.27
53.71
67.39
Linear Probing
34.72
42.76
66.54
BitFit
48.84
51.62
70.38
BitFit
27.85
36.92
64.43
Prompt
50.12
53.29
73.31
Prompt
29.57
38.71
64.62
LoRA
50.03
52.51
72.92
LoRA
32.39
40.35
67.04
Adapter
51.32
54.04
74.26
Adapter
35.40
43.62
68.06
DKA
53.69
56.58
78.56
DKA
37.13
46.27
72.53

---

Published as a conference paper at ICLR 2026
4.2 ERF VISCALIZATION
To whether DKA effectively expands the effective receptive field (ERF), visualize the ERFs of DKA alongside those of other adapter-tuning baselines, including Adapter (Houlsby et al. 2019), AdapterFormer (Chen al. )022), Convpass (Jie al. 2024). CIAT (Zhu et al. [02T]. and AIM (Yang et al. 2023), under both the constrained (0.63% and the full (100%) training data settings. As shown in Figure DKA consistently exhibits the broadest ERF across both settings, demonstrating its superior ability to capture extensive spatial context even under constrained conditions. contrast, other baselines yield more localized ERFs, partieularly in the constrained-data setting, which likely contributes to their comparatively weaker performance.
Adapterformer
Convpass
CIAT
Alm
Ours
0.8
0.5
0.8
0.6
0.8
0.63%
0.6
0.6
0.6
0.6
64
QU
0.8
4.8
bB
100%
08
0.55
-.4
0.2
422
02
Figure 4: Effective Receptive Field of DKA and Other Adapter-based Methods. Experiments are conducted on the COVID dataset based on the pretrained ViT-B under both constrained (0.63% and full (100%) training settings.
4.3 LARGE KERNEL MATTERS INSTEAD TRAINABLE PARAMETERS
Given that DKA introduces a slightly higher number of trainable parameters, a natural question arises: Are the observed performance gains primarily attributable the increased parameter count the effect of the large kernel? To investigate this, we perform a controlled comparison where the increase in trainable parameters arises either from enlarging the kernel size from expanding the intermediate dimension d. Specifically, adjust the middle dimension d of other adapter-based baselines that their total trainable parameter counts align with that of DKA with increasing kernel sizes. We fix the middle dimension d = 16 in DKA throughout all classification experiments, ensuring that any parameter increase stems solely from the enlarged kernel. As shown Figure the results reveal that: DKA consistently outperforms all baselines across the 0.63%, 1.25%. and 100% training data settings under similar parameter constraints; the performance improvements resulting from increasing the kernel size (from 11 to 51) are significantly steeper than those obtained by merely enlarging the hidden dimension, highlighting the effectiveness large kernels in enhancing DKA's representational capacity.
4.4 ASYNCHRONOUS LEARNING RATES MATTER
In standard adapter-based fine-tuning, it common practice to use the same learning rate for both the adapter and the head. However, given their different roles and characteristics, this one-size-fits all strategy may not be optimal. To investigate the effect of asynchronous learning rates for the adapter and head, we conduct experiments on the COVID dataset using a pretrained ViT-B, We systematically vary the learning rates assigned to the adapter and head, and assess the results under both the 0.63% and 1.25% training data. As shown Figure asynchronous learning rate schedules- where the adapter and head use different learning rates outperform symmetric configurations. We observe that the best results are not achieved when both components share the same learning rate, suggesting that the adapter and head benefit from distinct optimization dynamics. This trend holds across data scales, highlighting the importance of tuning these components independently. More results available in Appendix F2
7

---

Published as a conference paper at ICLR 2026
0.63%
1.25%
100%
80
96
ACC (%)
67
60
65
0.5
1.5
1.0
1.5
0.5
1.0
Trainable Parameters (M)
Trainable Parameters (M)
Trainable Parameters (M)
Adapter (á 16 :44)
Convpass (á >6)
AIM (á 229)
Adapterformer (å 16 87)
CIAT (6=16-29)
DKA (11 x1 51)
Figure 5: Comparison of Baselines and DKA with Comparable Numbers of Trainable Parameters. Experiments are conducted on 0.63%, 1.25%, and 100% subsets of the BUSI dataset using the pretrained ViT-B, The symbol d denotes the intermediate dimensionality in adapter-base based methods.
0.63%, ACC
0.63%, SEN
1.25%, ACC
1.25%, SEN
Head L.R
1n-4
#8.53
89.01
$4.33
H8.
16-4
81 23
24
81 2.
83 15
16-4
80 31
91.00
17 50
960 46
16-4 M4.37
07.76
83 03
06
1e-3
83.37
86
16:3
16:3
16:3
83.34
5e-2
05.31
76.38
24
100.05
71.31
16-3
16.5
5e-2
16-5
5e-2
16-5
5e-2
16-5
Adapter LR
Adapter LR
Adapter LR
Adapter LR
Figure 6: Performance Comparison Across Varying Learning Rates for DKA and Classification Head. Experiments are conducted on the COVID dataset using the pretrained ViT-B under 0.63% and 1. 1.25% training data. Results are reported in terms of ACC (%) and SEN (%)
4.5 ABLATION STUDY
Kernel Size Selection. We investigate the impact of different kernel size combinations in DKA's dual-branch convolution design. Specifically, we sweep over five candidate sizes (3x3, 5x5, 7x7, 9x9, 11 11x11, 1 31 51x51, 71) for both small- and large-depthwise convolution branches. As shown in Figure 7 the combination of 5x5 (small) and 51 x51 (large) consistently yields the best accuracy both the data-constrained (0.63% and 1.25%) setting and the full data setting (100%). Notably, kernel sizes that are too small or too large result in performance degradation, particularly under low-data conditions. These results support dual-branch design choice, balancing fine- grained detail extraction and large receptive field expanding. Extra experimental results are included in Appendix F5
0.63%
1.25%
100%
9x9
73.09
9x9
9x9
05.01
03.84
Small Kernel
68.27
70.01
73.06
76.04
77.00
20.40
77.
01.52
m
BK5
65.m1
71 65
76.23
72.41
5X5
75 R8
77.32
79.66
70.45
5x5
94.17
95.15
95. I
95. 05
3x3
72.01
3x3
75.
76.
3x3
93.75
31x31
51M51
11x11
31x31
71x71
51x51 71x7I Large Kernel
31x31
Large Kernel
Large Kernel
Figure 7: Performance of Different Kernel Size Combination. Experiments are conducted on the BUSI dataset using the pretrained VIT-B across three training setting (0.63%, 1.25%, and 100%). ACC (%) reported.
8

---

Published as a conference paper at ICLR 2026
Table 4: Performance Comparison of Different Middle Dimensions d in DKA. (a) Classification performance using the pretrained ViT-B on the BUSI dataset by reporting ACC (%). (b) Segmentation performance using the pretrained Segmenter-B on the ISIC-2018 dataset by reporting mioU (%). (a) Middle Dimension for Classification.
(b) Middle Dimension for Segmentation.
d
0.63%
1.25%
100%
0.63%
1.25%
100%
1
65.02
70.69
87,86
64
57.93
68.68
74.10
4
69.65
74.38
91,42
96
60.25
71.95
76.18
72.18
76.68
93.61
128
62.30
73.56
77.48
16
74.23
79.64
95.89
192
63.13
74.27
78.06
32
73.75
79.23
95.29
256
62.72
73.88
77.95
Single vs. Dual Convolutions. To evaluate the effectiveness of using both large- and -dept-dee convolutions in DKA, we compare the full dual-branch design (5x5 51 x51) against Linge-brrnch variants that use only one of the two kernels. Experiments are conducted on the COVID, BUSI, and ISIC-2019 datasets across varying training set sizes, As shown in Figure 8 the dual-branch design consistently outperforms both single-brame branch variants, particularly in low-data regimes. While the 5x5 branch better captures localized detail and the 51 X51 branch improves global coverage, neither alone matches the full-dual structure. See Appendix for additional results.
COVID
BUSI
ISIC-2019
100
ACC (9)
95
90
75
80
90
65
65
0.63 1 25 12.5 Traming Set (%)
2'5
50
75
100
0
0.631.25
12.5
25
50
75
100
0.63 1.25 12.5 25 50 73 100
Training Set (%)
Training Set (%)
Single x51)
Single (5x5)
Dual (5x5 + 51x51)
Figure 8: Ablation for Dual-Convolution Design. Experiments are based on the pretrained ViT-B across three classification datasets. ACC (%) is reported.
Middle Dimension. We investigate the impact of the middle dimension d in the DKA module for both classification and segmentation tasks. As shown in Table 4. increasing generally leads to improved performance. For classification on the BUSI dataset, performance steadily improves from = 1 to d = 16, reaching peak at d = 16, after which performance slightly declines or saturates, indicating potential overfitting or redundancy. For segmentation on the ISIC-2018 dataset, the optimal middle dimension is also moderate, with d = 192 offering the best trade-off between parameter efficiency and accuracy. This observation aligns with the findings in (Chen et al. (2022), which also highlight the lask-specific nature of optimal middle dimensions, emphasizing that different tasks require different parameter settings for optimal performance.
Learnable Kernel Sizes. To further investigate the learnable kernel sizes, following the design of Selective Kernel Networks Li et al. (2019), we implemented a learnable kernel-selection variant of our DKA. Specifically, we constructed two depthwise branches with different kernel sizes and fused them using a lightweight attention gate (global average pooling - two-layer MLP -> sofumax weights) that dynamically selects between the local and global branches for each input. This enables input-adaptive kernel aggregation. The results are reported in Table 5 While the learnable-kernel variant achieves competitive performance, our fixed 51 x 51 + 5 x 5 design remains slightly more robust in low-data regimes (0.63% and 1. 1.25%). likely because the learned gating requires additional data to generalize reliably. Under full data (100%), both approaches perform similarly. Overall, adaptive kernels are feasible, but our chosen configuration offers greater robustness and reliability in the lat-data medical scenarios targeted in this work.

---

Published as a conference paper at ICLR 2026
Table Comparison of Learnable Kernel Sizes and Our dDDsignn Experiments are conducted on the COVID dataset using the pretrained ViT-B.
Methods
0.63%
1.25%
100%
Learnable kernel sizes 51 51 x 5 (Ours)
88.98
91.05
99.21
89.01
91.06
99.21
5 RELATED WORK
Adapter- based Fine-tuning adapts large pretrained models to downstream tasks by inserting and
training lightweight modules while keeping the original model parameters frozen, offering significant
computational and memory advantages over full fine-tuning (Xu et al. [023] Ding et al.
2023 Han
al. 2024b). Early Adapter methods introduced task-specific bottleneck layers between transformer
blocks (Houlsby et 2019).. with subsequent innovations improving architectural designs (Wang
al.) 2020b] Zhang al. [021] and multi-modal applications (Sung al., 2022 Pan al.
2022
Gao et al. 2024). Recent medical imaging adaptations demonstrate how Adapter modules can
effectively transfer pretrained knowledge to diagnostic tasks while maintaining model integrity (Chen
et al. 2023 Dutt et al., 2023b, Lian et al., 2024). Recent large-kernel designs such as LKA (Zhu
cet al. 2025) enhance global receptive fields within convolutional backbones However, adapter-based methods typically assume access to a moderate amount of labeled data (Dutt al.) [023a] Liu cct al. 2024). Their effectiivness in data-constrained settings remains underexplored, raising critical questions about their performance in such scenarios.
Limited Data in Medical Imaging remains a major challenge, as labeled samples are scarce due to privacy, high annotation cost, and limited experts (Shaikhina & Khovanova] 2017:Chlap et al., 202I). This constraint is especially critical in clinical practice, where collecting large and diverse
datasets difficult (Ewing 2017 [ee & Yoon 2017 Li al. 2010). To address scarcity, strategies
include data augmentation (Grreaa et al.) [2233 Goceri 2023 Zhao et al. 2019 Islam et al. 2024).
transfer learning (Raghu et al. 2019 Kim et al. 2022 Kora et al. [022) semi-supervised learning
(Huynh et al. 2022] Chebli et 20188 Wang et al. 2020aa Han et al. [2024a] Zhou et al.)
2019)
and self-supervised representation learning (Ericsson al. (002]]Jaa et al.) 2020 Ye al. 2024
Krishnan et al. (0022) These approaches leverage unlabeled data or external sources to improve
generalization under low-resource settings (Zheng et al., 2024 Lin et al. 202 I), and recent work highlights large pretrained models to further reduce annotation needs (Moor et al. 2023 Zhang & Metaxas 2024; Khan et al. 2025). Nevertheless, most rely on full fine-tuning or moderately sized
datasets (Davila al. [2224] Khan & Fang 2023), while their integration with adapter- based methods
under severe scarcity remains underexplored but highly relevant in practice.
6 CONCLUSION
In this paper, we revisit adapter-based fine-tuning for medical image analysis and uncovered key limitations low-data settings: existing Adapters often struggle to capture relevant features under data scarcity, partly due to their constrained receptive field. To address this, we introduced DKA, dual-branch adapter module that integrates large- and small-kernel depthwise convolutions to enhance the receptive field while preserving local detail. Experiments and segmentation tasks, evaluated across various datasets and backbones show that DKA consistently outperform both full fine-tuning and other PEFT baselines by good margin, particularly in the constrained-date setting, without significantly increasing parameter count.
both medical image classification
7 ACKNOWLEDGMENTS
The authors acknowledge the use of resources provided by the UKRI SLAIDER project, the MRC SLAIDER- -QA project, the Isambard-Al National AI Research Resource (AIRR), and the Dutch national e-infrastructure, supported by the SURF Cooperative (Project 170911. Isambard-Al is operated by the University of Bristol and funded by the UK Government's ' Department for Science, Innovation and Technology (DSIT) via UK Research and Innovation and the Science and Technology
10

---

Published as a conference paper at ICLR 2026
Facilities Council [ST/AIRR/I-A-V1022 Finally, thank the anonymous reviewers for their insightful comments, which significantly improved the quality of this paper.
8 ETHICS STATEMENT
We confirm that this study adheres to the ICLR Code of Ethics, in every respect. All datasets used are publicly available and have been widely adopted in prior research.
REPRODUCIBILITY STATEMENT
We provide complete implementation details, including dataset splits, training settings, and evaluation protocols in Section Appendix [B.] and Appendix B.2 Code and data will be released facilitate reproducibility and further research.
REFERENCES
André Araujo, Wade Norris, and Jack Sim. Computing receptive fields of convolutional neural networks. Distill, 4(11)==21, 2019.
Asma Chebli, Akila Djebbar, and Hayet Farida Marouani. Semi Semi-supervised learning for medical application: survey. 2018 international conference on applied smart systems (ICASS), pp- 1-9. IEEE, 2018.
Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang. and Ping Luo. Adaptformer: Adapting vision transformers for scalable visual recognition. Advances in Neural Information Processing Systems, 35: 6664-16678, 2022.
Tianrun Chen, Lanyun Zhu, Chaotao Deng, Runlong Cao. Yan Wang, Shangzhan Zhang, Zejian Li, Lingyun Sun, Ying Zang, and Papa Mao. Sam-adapter: Adapting segment anything in underperformed scenes. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 3367-3375, 2023.
Tianrun Chen, Ankang Lu, Lanyun Zhu, Chaotao Ding, Chunan Yu, Deyi Ji, Zejian Li, Lingyun Sun, Papa Mao, and Ying Zang. Sam2-adapter: Evaluating adapting segment anything 2 in downstream tasks: Camouflage, shadow, medical image segmentation, and more. arXiv preprint MrXiv:2408.04577 2024.
Phillip Chlap, Hang Min, Nym Vandenberg, Jason Dowling, Lois Holloway, and Annette Haworth. A review medical image data augmentation techniques for deep learning applications. Journal of medical imaging and radiation oncology, 65(5):545-563. 2021.
François Chollet. Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1251-1258, 2017.
KR 1444 Chowdhary and KR Chowdhary. Natural language processing. Fundamentals of artificial intelligence, pp. 603 629,6449220 2020.
Muhammad EH Chowdhury, Tawsifur Rahman, Amith Khandakar, Rashid Mazhar, Muhammad Ab- dul Kadir, Zaid Bin Mahbub, Khandakar Reajul Islam, Muhammad Salman Khan, Atif Iqbal, Nasser Al Emadi, et al. Can ai help in screening viral and covid-19 pneumonia? leee Access,
132665-132676 2020.
Noel Codella, Veronica Rotemberg, Philipp Tschandl, Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo. Konstantinos Liopyris, Michael Marchetti, et al. Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic). arXiv preprint arXiv: 03366. 2019.
MMSegmentation Contributors. MMSegmeenattinn Openmmlab semantic segmentation toolbox and 2020.
benchmark. htttps://q http://///ppennmmmmmaaaa//eeeeee

---

Published as a conference paper at ICLR 2026
Ana Davila, Jacinto Colan, and Yasuhisa Hasegawa. Comparison of fine-tuning strategies for transfer learning in medical image classification. Image and Vision Computing, 146:105012, 2024.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248-255. Icee, 2009,
Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang. Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al. Parameter-efficien fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence, 5(3):220-235, 2023.
Xiaohan Ding, Xiangyu Zhang, Jungong Han, and Guiguang Ding. Scaling up your kemelss to 31x31: Revisiting large kernel design in cnns. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 19975, 2022.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.1 1929, 2021.
Raman Dutt, Ondrej Bohdal, Sotirios A Tsaftaris, and Timothy Hospedales. Fairtune: Opti- mizing parameter efficient fine tuning for faimess in medical image analysis. arXiv preprint 10005023 2023a.
Raman Dutt, Linus Ericsson, Pedro Sanchez, Sotirios A Tsaftaris, and Timothy Hospedales. Parameter-efficient fine-tuning for medical image analysis: The missed opportunity. arXiv preprint 0rX1v:23055082252 2023b.
Linus Ericsson, Henry Gouk, Chen Change Loy, and Timothy M Hospedales. Self-supervised representation learning: Introduction, advances, and challenges. IEEE Signal Processing Magazine. 39(3):42-62, 2022.
European Union General data protection regulation (gdpr) regulation (eu) 2016/679, 2016/ URL https: / eeeeeeereeeee -oor-lex..o europa et//eli/reg//16//11177 /017///////1/6///
GW Ewing. The limitations of big data in healthcare. MOJ Proteomics Bioinform, 5(2)00115, 2017.
Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters. International Journal of Computer Vision, 132(2)5581-995 2024.
Fabio Garcea, Alessio Serra, Fabrizio Lamberti, and Lia Morra. Data augmentation for medical imaging: A systematic literature review. Computers in Biology and Medicine, 152: 152:10 2023.
Nils Gessert, Maximilian Nielsen, Mohsin Shaikh, René Werner, and Alexander Schlaefer. Skin lesion classification using ensembles of multi-resolution efficientnets with meta data. MethodsX, 7: 100864, 2020.
Evgin Goceri. Medical image data augmentation: techniques, comparisons and interpretations. Artificial Intelligence Review, 56(1
2023.
Shizhan Gong, Yuan Zhong, Wenao Ma, Jinpeng Li, Zhao Wang. Jingyang Zhang, Pheng-Ann Heng, and Qi Dou. adaater: Holistic adaptation of sam from 2d to 3d for promptable medical image segmentation. arXiv e-prints, pp. arXiv-2306, 2023.
Kai Han, Victor S Sheng, Yuqing Song, Yi Liu, Chengjian Qiu, Siqi Ma, and Zhe Liu. Deep semi- supervised learning for medical image segmentation: A review. Expert Systems with Applications, 1230123 2024a.
Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403. 14608, 2024b.
Neil Houlsby, Andrei Giurgiu. Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan. and Sylvain Gelly. Parameter-efficien transfer learning for nlp. In International conference machine learning, pp. 2790-2799. PMLR, 2019.
12

---

Published as a conference paper at ICLR 2026
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3 2022.
Jihong Hu, Yinhao Li, Hao Sun, Yu Song, Chujie Zhang, Lanfen Lin, and Yen- Wei Chen. Lga: A language guide adapter for advancing the sam model's 's capabilities in medical image segmentation. In International Conference Medical Image Computing and Computer-Assiste Intervention, pp. 610-620. Springer, 2024.
Tianjin Huang, Lu Yin, Zhenyu Zhang, Li Shen, Meng Fang, Mykola Pechenizkiy, Zhangyang Wang, and Shiwei Liu. Are large kernels better teachers than transformers for convnets? International Conference Machine Learning, pp. 44023-14038 PMLR, 2023.
Tri Huynh, Aiden Nibali, and Zhen He. Semi-supervised learning for medical image classification using imbalanced training data. Computer methods and programs in biomedicine, 11::06662, 2022.
Tauhidul Islam, Md Sadman Hafiz, Jamin Rahman Jim, Md Mohsin Kabir, and MF Mridha. A systematic review of deep learning data augmentation in medical imaging: Recent advances and future research directions. Healthcare Analytics, pp. 100340, 2024.
Cheng Ji, Fan Wu, Zongwei Zhu, Li-Pin Chang. Huanghe Liu, and Wenjie Zhai Memory-efficient deep learning inference with incremental weight loading and data layout reorganization on edge systems. Journal of Systems Architecture, 1 118: 10210022,,3 2021.
Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In European conference on computer vision, pp. 709-727. Springer, 2022.
Wenyu Jiang, Zhenlong Liu, Zejian Xie, Songxin Zhang, Bingyi Jing, and Hongxin Wei. Exploring learning complexity for efficient downstream dataset pruning. arXiv preprint arXiv:240..05356 2024.
Jianbo Jiao, Richard Droste, Lior Drukker, Aris Papageorghiou, and Alison Noble. Self Self-supervised representation learning for ultrasound video. In 2020 IEEE 17th international symposium biomedical imaging (ISBI), pp. 1847-1850. IEEE, 2020.
Shibo Jie, Zhi-Hong Deng, Shixuan Chen, and Zhijuan Jin. Convolutional bypasses are better vision transformer adapters. ECAI 2024, pp. 202-209, IOS Press, 2024.
Muhammad Osama Khan and Yi Fang. Revisiting fine-tuning strategies for self-supervised medical imaging analysis. arXiv preprini 1001: 2023.
Wasif Khan, Seowung Leem, Kyle B See, Joshua K Wong, Shaoting Zhang, and Ruogu Fang. A comprehensive survey foundation models in medicine. IEEE Reviews Biomedical Engineering, 2025.
Hee Kim, Alejandro Cosa-Linan, Nandhini Santhanam, Mahboubeh Jannesari, Mate Maros, and Thomas Ganslandt. Transfer learning for medical image classification: a literature review. BMC medical imaging, 22(1):69, 2022.
Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint 2014.
Padmavathi Kora, Chui Ping Ooi, Oliver Faust, U Raghavendra, Anjan Gudigar, Wai Yee Chan, K Meenakshi, K Swaraja, Pawel Plawiak, and Rajendra Acharya. Transfer learning techniques for medical image analysis: A review. Biocybernetics and biomedical engineering, 42(1):79-107 2022.
Rayan Krishnan, Pranav Rajpurkar, and Eric J Topoo. Self-supervised learning in medicine and healthcare. Nature Biomedical Engineering, 6(12):1346-1352 2022.
Ya Le and Xuan S. Yang. Tiny imagenet visual recognition challenge. CS23In, 2015.
13

---

Published as a conference paper at ICLR 2026
Choong Ho Lee and Hyung-Jin Yoon. Medical big data: promise and challenges. Kidney research and clinical practice, 36(1):3, 2017.
Der-Chiame Li, Chiao- Wen Liu, and Susan C Hu. A learning method for the class imbalance problem with medical data sets. Computers in biology and medicine, 40(5):509-53 518, 2010.
Xiang Li, Wenhai Wang, Xiaolin Hu, and Jian Yang. Selective kernel networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 510-519, 2019.
Chenyu Lian, Hong- Yu Zhou, Yizhou Yu, and Liansheng Wang. Less could be better: Parameter- efficient fine-tuning advances medical vision foundation models. arXiv preprint arXiv:2401.122 2024.
Shuai Lin, Pan Zhou, Xiaodan Liang, Jianheng Tang, Ruihui Zhao, Ziliang Chen, and Liang Lin. Graph-evolving meta-leaming for low-resource medical dialogue generation. In proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 13362-13370, 2021.
Gang Liu, Jinlong He, Pengfei Li, Genrong He, Zhaolin Chen, and Shenjun Zhong. Pefomed: Parameter efficient fine-tuning of multimodal large language models for medical imaging. arXiv preprint wxXiv2241.002797, 2024.
Shiwei Liu, Tianlong Chen, Xiaohan Chen, Xuxi Chen, Qiao Xiao, Boqian Wu, Tommi Kärkkäinen, Mykola Pechenizkiy, Decebal Mocanu, and Zhangyang Wang. More convnets in the 2020s: Scaling up kernels beyond 51x51 using sparsity. arXiv preprint arXiv:2207.03620 2022.
Ze Liu, Yutong Lin, Yue Cao. Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 10012-10022, 2021.
Jun Ma, Yuting He, Feifci Li, Lin Han, Chenyu You, and Bo Wang. Segment anything in medical images. Nature Communications, 15(1):654, 2024.
N Mahendran. Analysis of memory consumption by neural networks based on hyperparameters. arXiv, 2021.
Xueyan Mei, Zelong Liu, Philip M Robson, Brett Marinelli, Minggian Huang, Amish Doshi, Adam Jacobi, Chendi Cao, Katherine E Link, Thomas Yang, al. Radimagenet: an open radiologic deep learning research dataset for effective transfer learning. Radiology: Artificial Intelligence, 4(5): e210315, 2022.
Bjoern H Menze, Andras Jakab, Stefan Bauer, Jayashree Kalpathy-Crane Cramer, Keyvan Farahani, Justin Kirby, Yuliya Burren, Nicole Porz, Johannes Slotboom, Roland Wiest, et al. The multimodal brain tumor image segmentation benchmark (brats). IEEE transactions medical imaging, 34(10); 1993-2024, 2014.
Michael Moor, Oishi Banerjee, Zahra Shakeri Hossein Abad, Harlan M Krumholz, Jure Leskovec, Eric Topol, and Pranav Rajpurkar. Foundation models for generalist medical artificial intelligence. Nature, 616(7956):259-2666 2023.
Junting Pan, Ziyi Lin, Xiatian Zhu, Jing Shao, and Hongsheng Li. St-adapter: Parameter-efficient video transfer learning, Advances in Neural Information Processing Systems, 35:26462- 26477, 2022.
A Paszke. Pytorch: An imperative style, high performmcce deep learning library. arXiv preprint 11X1v:19/2.01703. 2019.
Maithra Raghu, Chiyuan Zhang, Jon Kleinberg, and Samy Bengio. Transfusion: Understanding transfer learning for medical imaging. Advances in neural information processing systems, 32, 2019.
Felix Ritter, Tobias Boskamp. André Homeyer, Hendrik Laue, Michael Schwier, Florian Link, and H-O Peitgen. Medical image analysis. IEEE pulse, 2(6):60-70, 2011.
14

---

Published as a conference paper at ICLR 2026
Raphael Schäfer, Till Nicke, et al. Overcoming data scarcity in biomedical imaging with founda- tional multi-task task model. Nature Machine Intelligence, 5:1-10, 2023.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEEE international conference on computer vision, pp. 6186626, 2017.
Torgyn Shaikhina and Natalia A Khovanova. Handling limited datasets with neural networks in medical applications: A small-data approach. Artificial intelligence medicine, 75:51-63, 2017.
Fereshteh Shakeri, Yunshi Huang, Julio Mila-Rodrígguz, Houda Bahig, An Tang, Jose Dolz, and Ismail Ben Ayed. Few-shot adaptation of medical vision-language models. International Conference on Medical Image Computing and Computer-Assister Intervention, pp. 553-563. Springer, 2024.
Neha Sharma, Vibhor Jain, and Anju Mishra. An analysis of convolutional neural networks for image classification. Procedia computer science, 132:377-384, 2018.
Julio Maiva-Rodrigue,, Sina Hajimiri, Ismail Ben Ayed, and Jose Dolz. A closer look the few-shot adaptation of large vision-languee language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 23681-23699 2024.
Emma Strubell. Ananya Ganesh, and Andrew McCallum. Energy and policy considerations for modern deep learning research. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 33693-13696 2020.
Robin Strudel, Ricardo Garcia, Ivan Laptev, and Cordelia Schmid. Segmenter: Transformer for semantic segmentation. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 7262-7272, 2021.
Yi-Lin Sung, Jaemin Cho, and Mohit Bansal. Vl-adapter: Parameter-efficient transfer learning for vision-andd-anguuage tasks. In Proceedings of the IEEE/CVF conference on computer vision and paltern recognition, pp. 5227-5237, 2022.
Kenji Suzuki. Overview of deep learning medical imaging. Radiological physics and technology, 10(3):257-27,, 2017.
Richard Szeliski. Computer vision: algorithms and applications. Springer Nature, 2022.
Neil C Thompson, Kristjan Greenewald, Keeheon Lee, Gabriel F Manso, al. The computational limits of deep learning, arXiv preprint arXiv:2007.05558, 10. 2020.
U.S. Department of Health Human Services.
Summary of the hipaa privacy rule, 2003. URL https: //t:://wwww www. hhs www.goo//iiiggg por.eeesssso.ttss.aaaalllllle wat.unstttoooooioooooooooooo indx. html
Dong Wang, Yuan Zhang, Kexin Zhang, and Liwei Wang. Focalmix: Semi-supervised learning for 3d medical image detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3951-3960, 2020a.
Jajji Wang. Shuihua Wang, and Yudong Zhang. Deep learning medical image analysis. CAAI Transactions on Intelligence Technology, 10(1):1-35, 2025.
Rongguang Wang, Pratik Chaudhari, and Christos Davatzikos. Embracing the disharmony in medical imaging: A simple and effective framework for domain adaptation. Medical Image Analysis, 72: 102-110, 2021.
Ruize Wang. Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Guihong Cao. Daxin Jiang, Ming Zhou, et al. K-adapter: Infusing knowledge into pre-trained models with adapters. arXiv preprint arXiv:2002.01 8088 2020b.
15

---

Published as a conference paper at ICLR 2026
Zifeng Wang, Zhenbang Wu, Dinesh Agarwal, and Jimeng Sun. Medelip: Contrastive learning from unpaired medical images and text. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference Empirical Methods in Natural Language Processing, volume 2022, pp. 3876, 2022.
Junde Wu, Ziyue Wang, Mingxuan Hong, Wei Ji, Huazhu Fu, Yanwu Xu, Min Xu, and Yueming Jin. Medical sam adapter: Adapting segment anything model for medical image segmentation. Medical Image Analysis, pp. 103547, 2025.
Lingling Xu et al. Parameter-efffcienn fine-tuning methods for pretrained language models: A critical review and assessment. arXiv preprini 2rXiv:2312.121453 2023.
Taojiannan Yang, Yi Zhu, Yusheng Xie, Aston Zhang, Chen Chen, and Mu Li. Aim: Adapting image models for efficient video action recognition. arXiv preprint arXiv:2302.03024 2023.
Yiwen Ye, Yutong Xie, Jianpeng Zhang, Ziyang Chen, Qi Wu, and Yong Xia. Continual self- supervised learning: Towards universal multi-modal medical data representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 11114-11124, 2024.
Dongshuo Yin, Leiyi Hu, Bin Li, Youqun Zhang, and Xue Yang. 5%; 100%: Breaking performance shackles of full fine-tuning on visual recognition tasks. arXiv preprint arXiv:2408.066 08345, 2024.
Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg. Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models, arXiv preprini arXiv: 106. 10/99, 2021.
Renrui Zhang. Rongyao Fang, Wei Zhang, Peng Gao, Kunchang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. Tip-adapter: Training-free frre clip-adapter for better langualan modeling, arXiv preprint arXiv:211 /. 03930, 2021.
Shaoting Zhang and Dimitris Metaxas. On the challenges and perspectives of foundation models for medical image analysis. Medical image analysis, 91: 91:102996, 2024.
Yichi Zhang, Zhenrong Shen, and Rushi Jiao. Segment anything model for medical image segmenta- tion: Current applications and future directions. Computers in Biology and Medicine, pp. 108238, 2024.
Yingtao Zhang, Min Xian, Heng-Da Cheng, Bryar Shareef, Jianrui Ding, Fei Xu, Kuan Huang, Boyu Zhang, Chunping Ning, and Ying Wang. Busis: benchmark for breast ultrasound image segmentation. In Healthcare, volume 10, pp. 729. MDPI, 2022.
Amy Zhao, Guha Balakrishnan, Fredo Durand, John V Guttag, and Adrian Dalca. Data augmenta- tion using learned transformations for one-shot medical image segmentation. In Proceedings of the JEEE/CVF conference on computer vision and pattern recognition, pp. 3543-8553, 2019.
Fudan Zheng, Jindong Cao, Weijiang Yu. Zhiguang Chen, Nong Xiao, and Yutong Lu. Exploring Jow-resource medical image classification with weakly supervised prompt learning. Pattern Recognition, 149: 10250, 2024.
Yi Zhou, Xiaodong He, Lei Huang, Li Liu, Fan Zhu, Shanshan Cui, and Ling Shao. Collaborative learning of semi-super segmentation and classification for medical images. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 2079-2088, 2019.
Yaoming Zhu, Jiangtao Feng, Chengqi Zhao, Mingxuan Wang, and Lei Li. Counter-interference adapter for multilingual machine translation. arXiv preprint arXiv:2104.088
2021.
Ziquan Zhu, Si- Yuan Lu, Tianjin Huang, Lu Liu, and Zhe Liu. Lka: L arge k emell dapter for enhanced medical image classification. International Conference on Medical Image Computing and Computer-Assee Assisted Intervention, pp. 394-404. Springer, 2025.
16

---

Published as a conference paper at ICLR 2026
APPENDIX
The Use of Large Language Models (LLMs)
18
Experiment Settings
18
B. Datasets
18
B.2 Implementation Details
18
Additional Segmentation Results on Natural-pretrained Models
19
Segmentation Results with Additional Metrics
19
C.2
Segmentation Results with More Data Scales
19
Additional Classification Results on Natural-pretrained Models
21
D.I Classification Results with Additional Metries
21
D.2
Classification Results with More Data Scales
22
Additional Results on Medical-pretrained Models
23
Additional Ablations
24
DKA Different Backbone
24
F2
Performance of Dual-Kann Kernel Convolution and Asynchronous Learning Rates
24
DKA Position
25
ERF of Adapter in Different Backbone
25
Effect of Dilated vs. Standard Kernels
26
F6
Effect of Diverse Kernel Combinations
27
Effect under Extreme Low-data Regimes
28
F88
Effect on Medical
Medical- pretrained Vision- LanguuLan Models,
29
F.9
Frequency-domain Analysis of Kernel Sizes
29
10
Inference Latency and Memory Usage
29
Effects of Larger Batch Size
30
12
Parameter Efficiency Analysis
30
F
13
Complementary CAM Visualization
30
Pseudocode
32
17

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS (LLMS)
Large language models (LLMs) were not used as part of the core methodology. experiments, or original research contributions. They were only used as an assistive tool for language editing and formatting.
B EXPERIMENT SETTINGS
B.1 DATASETS
B.1.1 CLASSIFICATION DATASETS
COVID (howddhury et al., 2020 The COVID-19 Radiography Database, developed through a collaborative effort involving researchers from Qatar University, the University of Dhaka, and medical experts from Pakistan and Malaysia, includes approximately 3616 COVID-19 cases, 10,192 normal cases, 6012 lung opacity cases (representing non-COVID lung infections), and 1345 viral pneumonia cases. This dataset provides a comprehensive collection of chest X-ray (CXR) images for the diagnosis of COVID-19 19 and other lung conditions.
BUSI (Zang et al. 2022); The BUSI (Breast Ultrasound Images) dataset, collected in 2018, comprises approximately 780 ultrasound images from 600 female patients aged 25 to 75. It includes around 437 normal, 210 benign, and 133 malignant breast lesion images, each with an average resolution of approximately 500x500 pixels. The dataset is organized into three primary categories based on the clinical classification of breast lesions: normal, benign, and malignant.
ISIC-2019 Gessert et al. 2020): The ISIC-2019 dataset, part of the annual ISIC (International Skin Imaging Collaboration) challenges, includes 25,331 dermoscopic images compiled from previous ISIC challenges (2017 and 2018). It spans nine diagnostic categories: melanoma, melanocytic nevus, basal cell carcinoma, actinic keratosis, benign keratosis (including solar lentigo, seborrheic keratosis, and lichen planus-like keratosis), dermatofibroma, vascular lesion, squamous cell carcinoma, and category for images that do not fit into any of the other classes, providing comprehensive benchmark for multi-class skin lesion classification.
B. 1.1.2 SEGMENTATION DATASETS
BRATS (Menze et al. 2014); The BRATS (Brain Tumor Segmentation) dataset provides multi- institutional, clinically acquired multi-parametric MRI (mpMRI) scans of gliomas, including T1, post-contrast Te-weighted (TIGd), T2, and T2-FLAIR volumes. It includes pathologically confirmed cases with expert-annotated tumor sub-regions, including the enhancing tumor (ET), tumor core (TC), and whole tumor (WT), providing a comprehensive benchmark for brain tumor segmentation.
BUSI (Zhang al., 2022): The BUSI (Breast Ultrasound Images) dataset includes ground truth masks for precise lesion segmentation, facilitating the evaluation of automated lesion detection and boundary delineation methods. These masks correspond to the original ultrasound images, capturing the exact regions of interest for each lesion type. The dataset primarily supports the segmentation of benign and malignant breast lesions, providing detailed representation of lesion morphology.
1SIC-2018 (Codella et al., 2019): The ISIC-2018 dataset, released as part of the ISIC 2018 Task 1 challenge, contains a total of 3,694 dermoscopic images, each paired with binary segmentation mask outlining the precise lesion boundaries. This dataset serves as critical benchmark for evaluating automated skin lesion segmentation methods.
B.2 IMPLEMENTATION DETAILS
Our experiments are conducted on NVIDIA RTX 3090 GPUs. The code is based on PyTorch (Paszke 2019). Due to hardware memory constraints, we use different batch sizes for classification and segmentation tasks. Specifically, classification tasks use a batch size of 8, while segmentation tasks restricted to batch size of 1, larger batch sizes lead out-of-mentt of-momory errors. The test set is fixed at 20% of the total dataset, ensuring consistent evaluation across all experimental settings. We employ Adam optimizer (Kingma 2014) for both the head and DKA modules, albeit with different
18

---

Published as a conference paper at ICLR 2026
learning rates for each. When reducing the training data sizes, proportionally decrease samples per class to maintain balance. Even in extreme low-data scenarios, we ensure that each class retains some images to prevent complete absence of any category. The weights of the down-projection layers and the biases and weights of the up-projection layers are initialized to zero, providing stable starting point for training.
C ADDITIONAL SEGMENTATION RESULTS ON NATURAL-PRETRAINED MODELS
C.I. SEGMENTATION RESULTS WITH ADDITIONAL METRICS
To complement the segmentation results in the main text, we provide additional evaluations in Table where performance is reported in terms of Dice scores across three datasets (BRATS, BUSI, ISIC-2018) under different training ratios (0.63%, 1 1.25%, 100%). The results confirm that DKA consistently achieves higher Dice scores than all baseline methods across datasets and data scales, further demonstrating its robustness in capturing both local details and global context for medical image segmentation.
Table 6: Comparison of Baselines and DKA on Three Segmentation Datasets Under Varying Data Sizes. Experiments are based on the pretrained Segmenter-B B. Results are reported in terms of
Dice (%). The best results are highlighted bold.
BRATS
BUSI
ISIC-2018
Methods
0.63%
1.25%
100%
0.63%
1.25%
100%
0.63%
1.25%
100%
Full Fine-tuning
16.94
36.59
84.45
42.11
48.83
72.94
76.75
84.81
87.37
Linear Prohing
14.72
33.62
82.26
40.67
48.44
70.19
75.70
83.08
85.12
BitFit 7aken et al.
2021
2.40
25.06
77.69
13.31
29.18
68.90
69.46
79.41
84.46
Prompt Jia et al
2022
2.42
26.41
78.44
16.84
31.61
69.76
72.25
80.52
84.80
LoRA Hu et al.
2022
28.86
81.29
24.72
36.59
70.09
73.97
81.68
84.95
Adapter [Houlsby el a.
2019
11.60
31.86
83.73
30.77
41.14
70.98
74.83
84.26
86.82
Adapterformer (Chen Cf al.
2022
11,30
31.61
84.08
29.66
40.86
71.08
74.74
84.29
86.73
Convpass (Tie
2024
13.31
32.83
84.60
33.11
44.39
71.87
75.14
84.77
87.35
CIAT (Zhu 202T)
7.33
28.72
82.64
23.63
33.62
70.77
73.63
82.02
85.77
AIM Yang ct al.
2023
8.75
30.49
83.71
26.69
38.11
70.79
74.41
83.75
86.78
DKA
17.29
37.43
85.69
42.34
51.33
74.13
77.39
85.24
87.97
C.2 SEGMENTATION RESULTS WITH MORE DATA SCALES
To further validate the performance of DKA in segmentation tasks, we present additional results across a wider range of data scales (12.5% 75%) in Table These results, covering three segmentation medical imaging datasets, provide a comprehensive assessment of our model's performance in mid- scale training regimes, complementing the extreme low-data (0.63% and 1.25%) and full-data (100%) findings discussed in Section 4.1. Consistent with our previous observations, DKA outperforms all baselines across all datasets and training set ratios, achieving the highest mloU and Dice metrics. This demonstrates the effectiveness of DKA, highlighting its ability to effectively capture both local detail and global context across varying supervision levels, providing a reliable solution for medical image segmentation.
19

---

Published as a conference paper at ICLR 2026
Table Additional Segmentation Results on Three Datasets with Varying Training Set Ratios. Experiments are based on the pretrained Segmenter-B B. Results are reported as percentages for mloU and Dice. The best results are highlighted in bold.
Methods
12.5%
50%
50%
50%
milu (%)
Full Fine-tuning
55.25
61.35
70.43
71.50
51.44
53.70
55.35
56.8
74.60
75.36
75.50
77.10
Prabung
67.75
Zakenettl
Prompt
48.58
$4.08
62.73
63.01
38.49
43.38
46.91
50.30
68.54
69.16
69.63
72.31
LoRA
01 AIP
50.77
57.19 19
64. 50
67.75
41.22
47.73
50.42
52.97
69.64
70.83
71.61
72.95
Adapertirm:
74.15
Compues li [t 377
$4.08
61.08
69.67
71.26
49.30
51.77
$3.61
55.27
74.31
74.08
75.13
76.94
CIAT
7Tu 2
A14
51.70
53.23
67.99
68.45
40.74
47.72
51.71
53.85
70.83
71.42
72.94
74.57
DXA
71.95
76.02
78.06
Full Fine-tuninng
76.05
82.65
Linear Probine
68.
RiFi
Zakenetal ml
5021
64.96
69:43
76.
76.14
53.57
58.20
61.45
66.42
78.20
79.21
80.13
81.78
Prampt
65.21
70.20
77.10
77.51
55.59
66.51
63.86
67.07
81.33
81.77
82.10
83.93
Adapt
85.37
Caaperrfrritrr [ Teneet 41 1003
68.53
74.49
81.73
82.61
62.26
66.00
68.76
70.50
84.69
84.92
85.16
86.15
Compues
eie
3114
70.20
75
82.13
83.22
66.04
68.22
69.80
71.19
85.26
85.70
85.
86.97
Yang 2lal
73.76
81:00
81,90
60.56
67.73
70.10
84.20
84.76
DXA
72.34
77.24
83.69
84.34
68.68
70.77
72.15
73.48
85.91
86.25
86.38
87.68
20

---

Published as a conference paper at ICLR 2026
D ADDITIONAL CLASSIFICATION RESULTS ON NATURAL-PRETRAINED MODELS
D.1 CLASSIFICATION RESULTS WITH ADDITIONAL. METRICS
In addition to the accuracy results reported in the Section 4.1 we further provide a comprehensive
evaluation of DKA and baselines using Fl and sensitivity (SEN) scores on three classification datasets:
COVID, BUSI, and ISIC-2019. As summarized in Table g DKA consistently achieves the best or near-best performance across all data scales (0.63%, 1.25%, and 100%). In particular, under low-data regimes, DKA exhibits notable improvements over existing PEFT approaches such BitFit, LoRA, and Adapter-based methods, highlighting its robustness in capturing discriminative features when supervision is scarce. These results further validate the effectiveness of DKA beyond standard accuracy and confirm its ability to improve both predictive balance (F1) and clinical reliability (SEN) in medical image classification.
Table 8: Comparison of Baselines and DKA on Three Classification Datasets Across Varying
Data Sizes. Experiments are based the pretrained ViT-B. SEN is reported as percentages, while Fl is presented as raw value. The best results are highlighted in bold.
COVID
BUSI
ISIC-2019
Methods
0.63%
1.25%
100%
0.63%
1.25%
100%
0.63%
1.25%
100%
F1
Full Fine-tuning
0.845
0.855
0.970
0.697
0.752
0.939
0.293
0.317
0.539
Linear Probing
0.831
0.843
0.932
0.710
0.756
0.897
0.287
0.307
0.442
BitFit Zaken et al.
2021
0.718
0.762
0.950
0.542
0.573
0.879
0.184
0.235
0.501
Prompt Jia et
2022
0.753
0.819
0.972
0.588
0.618
0.930
0.218
0.235
0.534
LoRA Hu et al.
2022
0.786
0.814
0.973
0.603
0.641
0.945
0,214
0.235
0.528
Adapter [Houlsby el a
2019
0.802
0.833
0.978
0.607
0.704
0.931
0.231
0,248
0.501
Adapterformer (Chen C a.
2022
0.793
0.807
0.974
0.614
0.693
0.918
0.228
0.231
0.495
Convpass [Tie al.
2024
0.807
0.833
0.969
0.622
0.718
0.938
0.253
0.275
0.512
CIAT (Zhu 202T
0.748
0.794
0.947
0.574
0.635
0.899
0.162
0.195
0.429
AIM Yang el
0.776
0.804
0.953
0.602
0.682
0.907
0.182
0.227
0.433
DKA
0.865
0.881
0.981
0.720
0.774
0.951
0.296
0.326
0.542
SEN (%)
Full Fine-tuning
83.57
84.15
97.24
69.27
74.26
93.46
27.76
30.64
52.35
Linear Probing
82.64
83.49
92.74
70.25
74.43
87.74
26.87
28.68
42.71
BitFit (Zaken et al
2021
70,36
75.34
94.45
51.33
54.41
85.52
18.50
21.74
47.55
Prompt Jia et al. 2022
74.31
81.76
96.57
56.71
59.35
91.34
20.65
23.92
49.28
LoRA Hu et al.
2022
77.42
80.76
96.76
59.37
61.41
92.07
20.11
24.13
50.75
Adapter HHuulsy et
2019
81.38
82.40
97.41
59.40
66.69
92.58
21.84
24.93
49.50
Adapterformer (Chen al.
2022)
79.34
81.51
97.40
60.96
65.44
90.72
20.36
22.78
48.31
Convpass (Jie et 2024
81.23
82.71
95.76
61.79
67.90
92.74
24.63
27.36
$1.12
CIAT (Zhu et 2021)
74.93
78.34
94.35
54.52
61.57
87.85
16.24
20.17
42.18
AIM Yang
76.04
80.76
94.50
58.75
65.46
89.45
18.06
21.07
47.16
DKA
84.41
87.76
98.12
71.82
76.78
95.46
28.20
31.50
53.69
21

---

Published as a conference paper at ICLR 2026
D.2 CLASSIFICATION RESULTS WITH MORE DATA SCALES
To further substantiate the advantages of DKA identified in the main text, we present additional classification results across wider range of data scales (12.5% to 75%) in Table These results reinforce the key findings, demonstrating that DXA consistently outperforms other baselines across all datasets (COVID, BUSI, ISIC-2019). This superior performance holds not only in extreme low-data regimes (0.63% and 1.25%) and full-data regimes (100%) but also across mid-scale settings, confirming that DKA maintains its advantage regardless data availability. This performance reflects the effectiveness of its qual-branch design, which simultaneously captures local detail and broader context, providing critical edge in medical image classification.
Table 9: Additional Classification Results on Three Datasets with Varying Training Set Ratios. Experiments are based on the pretrained ViT-B, ACC and SEN are reported as percentages, while F! is presented as raw value. The best results are highlighted in bold.
Methods
COVID
BUST
ISIC-2019
12.5%
25%
50%
7548
12.59
25%
SORS
75%
12.598
25%
50%
75%
ACC((5)
Fine-tuning
97.07
Linear Prabing
92.84
93.43
94.
94.33
79.25
84.03
86.34
87.22
63.666
65.75
66.66
68.28
BiFRi
Zalen ml 2021
83.02
90.51
93.57
93.50
69.01
75.00
84.12
90.98
60.96
63.17
66.33
72.10
87.91
90.77
73.47
Adupter
Hoillhh el
2010
94.32
16.04
97. 14
07.67
80.51
85. 18
89.78
90.75
65.54
68.51
70.55
73.41
Adayterhimg hermenrerearrr
0023
90.00
92.15
94.67
97.30
78.00
85.19
87.75
89.37
63.94
69.45
69.73
71.88
Compass lic
94.23
72.47
83.56
87.55
67.55
70.25
AIM
4124
89.38
93.88
95.
96.17
76.61
82.00
85.63
86.92
65.62
69.24
70.82
73.58
DXA
96.86
97.34
98.29
98.55
87.26
91.10
93.73
95.01
70.47
74.04
77.42
80.06
F1
Full Fine-tuning
0.930
0.951
0.956
0.968
0.833
0.874
0.902
0.925
0.402
0.435
0.481
0.502
Lincar Prohing
0.898
0.916
0.922
0.929
0.782
01.828
0.855
0.861
0.335
0.351
0.379
0.392
Prompctaeel
0.938
0.953
0.397
LoRA of NIM
0
0.944
0.956
0.965
0.747
0.847
0.886
0.932
0.331
0.367
0.439
0.462
Adapter Honlthh el
2819
0.920
0.946
0.953
0.967
0.783
0.543
0.871
0.897
0.354
0.352
0.423
0.457
Cumpuee: [lc
0.941
0.958
0.962
0.794
0.878
0.907
0.372
0,409
0.421
0.458
CIAT
7hi ar XM.
0.843
0.905
0.911
0.925
0.708
0.738
0.815
0.844
0.293
0.364
0.375
0.417
AIM
Yang
2025
0.869
0.917
0.923
0.944
0.751
0.813
0.839
0.842
0.354
0.404
0.415
0.425
SEN (%)
Full Fine-tuning
93.24
95.11
95.52
96.68
82.24
87.00
89.92
91.00
38.84
40.01
41.55
49..6
Pruhing
Zaken
91.15
80.77
Promp
MAC 2.72
83.56
91.55
93.24
94.95
68.27
77.25
85.47
90.54
31.54
37.63
43.10
46.75
LoRA
01 NIM
85.15
92.90
95.65
96.22
71.12
81.34
87.93
90.23
31.88
37.31
42.61
46.90
Adaater
Adaptatforintt
87.35
39.80
91.01
95.54
73.68
82.74
85.79
87.16
32.52
39.10
39.75
41.56
Compuses
[he St 377
92.00
94.51
94.92
95.36
76.78
80.73
85.03
88.21
35.36
39.84
41.55
43.99
CIAT
7mm 2
AM4
84.52
89.49
91.77 73
93.61
69.64
76.49
80.83
84.21
21.79
24.31
36.25
36.50
DXA
95.24
97.18
97.59
83.88
93.46
93.89
39.72
44.07
47.50
30.39
22

---

Published as a conference paper at ICLR 2026
E ADDITIONAL RESULTS ON MEDICAL-PRETRAINEE MODELS
To provide more comprehensive evidence beyond the main paper, we further report detailed results using pertccal-rettraneed models. Table 10 presents classification performance on the ISIC-2019 dataset with RadImageNNt-ppetraiiee ResNet-50 (Mei et al. 2022), where DKA consistently improves over linear probing and standard adapter tuning across different training ratios in terms of ACC, F1, and SEN. Table I1 T reports segmentation results on the BUSI dataset with MedSAM (Ma et al. 2024), showing that DKA achieves clear gains over both baselines in terms mloU and Dice. These extended results corroborate our main findings in Section 4.5 demonstrating that the advantages of DKA generalize robustly to medical-pre perttca--ee models and are not restricted natural-pretrained backbones.
Table 10: Additional Classification Results based on the Radlmaggeee--peerriinee ResNet-50 backbone the ISIC-2019 dataset under varying training ratios. SEN is reported as percentages, while Fl is presented as raw value. The best results are bighlighted in bold.
Methods
0.63%
1.25%
100%
F1
Full Fine-tuning
0.223
0.262
0.483
Linear Probing
0.216
0.236
0,387
BitFit
0.172
0.208
0.411
Prompt
0.200
0.232
0.439
LoRA
0.203
0,230
0.431
Adapter
0.218
0.242
0.440
DKA
0.230
0.278
0.498
SEN (%)
Full Fine tannttunnng
22.57
26.86
43.14
Linear Probing
20.28
24.52
37.61
BitFit
17.69
22.86
40.20
Prompt
19.83
24.42
44.05
LoRA
19.79
25.77
44.38
Adapter
20.34
25.13
44.57
DKA
23.40
27.89
45.39
Table 11: Additional Segmentation Results based on MedSAM backbone on the BUSI segmenta-
tion task under varying training ratios. Results are reported in terms of Dice (%). The best results are highlighted in bold.
Methods
0.63%
1.25 %
100%
Dice (%)
Full Fine-tuning
54.54
63.08
81.15
Linear Probing
50.09
60.90
79.68
BitFit
23.46
47.31
74.15
Prompt
25.19
50.65
78.82
LoRA
32.58
55.03
80.42
Adapter
52.28
61.81
80.35
DKA
55.39
65.24
83.97
23

---

Published as a conference paper at ICLR 2026
F ADDITIONAL ABLATIONS
F.1 DKA IN DIFFERENT BACKBONE
To evaluate whether the advantages of DKA transfer to other architectures, we repeat classification experiments on the pretrained Swin-B B using the same datasets: COVID, BUSI, and ISIC-2019. As shown in Figure 9 DKA consistently outperforms full fine-tuning and all PEFT baselines across different data scales. Critically, while other PEFT methods struggle to match linear probing, and often degrade substantially in constrained-data regimes, DKA maintains strong performance both settings.
COVID
BUSI
ISIC-2019 2019
6
10
10
AACC
0
-3
-5
-6
10
0.63
1.25 12.5 25
50
75
100
0.63
1.25 12.5
25
50
75
100
0.63 1.25
12.5
25
50 75 100
Training Set (%)
Training Set (%)
Training Set (%)
Full Fine-tuning
Lora
Prompt
Adapter
Ours
Figure 9: Performance of Baselines and DKA across Various Training Data Sizes. AACC ACC Baslines ACC LCCLinearProbing Probing. Experiments are based on the pretrained Swin-B for COVID, BUSI, and ISIC-2019.
F.2 PERFORMANCE OF DUAL-KERNEE CONVOLUTION AND ASYNCHRONOUS LEARNING RATES
To further evaluate the impact of our proposed enhancements, including the introduction of dual- convolution design and learning rate split strategy within the DKA module, we present additional results in T2] This table compares DKA against two baselines (Adapter + Dual-Conv and Adapter + LR Split) across multiple datasets (COVID, BUSI, ISIC-2019) and training set ratios. Consistently, DKA outperforms both baselines, achieving the highest overall accuracy in each dataset, with notable gains observed even in low-data regimes. This confirms the effectiveness of our combined strategy in capturing richer local-global features and improving learning stability.
Table 12: Performance Comparison of Enhanced Adapter Designs. Experiments conducted on the pretrained ViT-B across three medical imaging classification datasets and varying training set ratios by reporting ACC (%).
Training Set Size
Datasets
Methods
0.63%
1.25%
12.5%
25%
50%
75%
100%
Adapter + Dual-Conv
86.78
88.89
96.01
96.98
98.13
97.95
98.61
COVID
Adapter + LR Split
88.04
88,95
96.20
97.04
97.64
98.07
98.79
Ours
89.01
91.06
96.86
97.34
98.29
98.55
99.21
Adapter +
67.73
77.32
83.39
89.14
90.73
91.05
94.25
BUSI
Adapter + LR Split
70.37
77.64
84.71
89.46
91.10
93.37
94.93
Ours
74.23
79.64
87.26
91.10
93.73
95.01
95.89
Adapter + Dual-Conv
59.49
60.51
70.12
73.27
76.60
78.52
82.25
ISIC-2019
Adapter + LR Split
59.06
60.09
69.99
72.24
76.07
78.52
81.67
Ours
60.52
62.32
70.47
74.04
77.42
80.06
83.09
24

---

Published as a conference paper at ICLR 2026
F.3
DKA POSITION
To understand how the position of inserted DKA influences performance, we explore three placement strategies using the pretrained ViT-B with 12 transformer blocks: inserting DKA into the bottom 4 blocks (Blocks 0-3), middle blocks (Blocks 4-7), and top 4 blocks (Blocks 8-11). We also include a reference setting where DKA is inserted into all layers. We extend our analysis three medical imaging classification datasets: COVID, BUSI, and ISIC-2019. As shown in Table [3] the position of DKA significantly affects model performance across various training set sizes. Among partial configurations, placing DKA in the middle layers consistently outperforms the top and bottom placements across datasets. Notably, under low-data regimes (e.g., 0.63%), placing DKA in the top layers offers stronger performance than bottom middle placement, highlighting the value of adapting higher-level representations when supervision is scarce. This trend consistent with observations from prior work (Yang al. 2023), which reported that inserting adapters in bottom blocks yields limited performance.
Table 13: Effect of Position. Experiments are conducted on the pretrained VIT-B across three medical imaging classification datasets and varying training set ratios by reporting ACC (%) with standard deviations.
Training Set Size
Datasets
Positions
0.63%
1.25%
Bottom
86,04
1.61
88.68
1.38
93.69
1: 0.89
94.29
1: 0.75
95.52
0.60
96.64
0.45
97.34
0.35
COVID
Middle
86.80
1.45
89.18
1.25
95.43
1 0.53
96.80
1: 0.41
97.76
1 0.29
98.43
0.25
98.79
0,21
Top
87.78
1.21
90.50
1.13
94.72
0.69
96.12
055
97.22
0.38
98.23
0.32
98.53
0.26
ALL
89,01
0.90
0.84 91.06
96.86
0.40
97.34
0.32
0.26 98.29
0.24 98.55
99.21
0.13
Bottom
2.06
BUSI
Middle
73.48 0:48::11 1.79
78.22
+ 1.28
86.62 + 0.75
90.43 + 0.44
93.43
0.25
94.94
14.94-2000
0.19
95.46
0.11
'Top
73.86
ALL
1.53 74.23
79.64 + 1.17
87.26 + 0.64
91.10 = 0.39
93.73 0.23
95.01 = 0.12
95.89
0.09
1.05
ISIC-2019
Middle
57.64 4- 2.33
60.01
2.05
69.88
1.34
73.95 + 1.06
76.74
0.74
79.43
0.43
82.59
0.20
Top
69.32
72.59
AL..
60.52
2,02
62.32 L8S
70.47 1.26
74.04
0.99
77.42
0.67
80.06
0.36
83,19
0.14
F.4 ERF OF ADAPTER IN DIFFERENT BACKBONE
To further validate our findings on the impact of data scarcity on Adapter performance, we extend the effective receptive field (ERF) analysis to the pretrained Swin-T model (Liu et al., [2021) as shown in Figure 10 This complementary analysis reinforces our observations from the pretrained ViT-B model (Figure revealing a similar contraction of ERFs as the training set size decreases. Specifically, under extreme data scarcity (e.g., 0.63% and 1. 1.25% training data), the pretrained Swin-T exhibits sharply reduced ERF, aligning with the earlier findings (see Section [ that Adapters can disrupt pretrained feature representations under severe data constraints. This result suggests that the negative impacts observed in the main text not limited to single architecture but are likely more general
phenomenon affecting a wide range of vision backbones.
0.63%
12.5%
25%
50%
100%
1.0
1.0
1.0
1.0
0.4
0.4
0.1
02
0.2
02
0.2
02
Figure 10: Effective Receptive Field Adapter under Different Training Set Ratios. Experiments are conducted on the COVID dataset using the pretrained Swin-T.
25

---

Published as a conference paper at ICLR 2026
F.5 EFFECT OF DILATED VS. STANDARD KERNELS
We further compare dilated convolutional kernels with our standard large-kernel design. In particular, evaluate several dilated kernel settings, including 3 3 with dilation rates of 25, 11 11 with d 5, and 26 x 26 with d = 2, against the dual-kernel configuration of 5 x + 51 x 51. The classification results on the BUSI dataset are shown Table 14
and the segmentation results on the
ISIC-2018 dataset are reported in Table 15
Across both datasets, the x 51 51 design consistently achieves the best performance in terms of ACC, F1, and SEN for classification, well as mloU and Dice for segmentation. Although dilated kernels provide larger effective receptive field, they underperform compared to explicitly using large standard kernel. This suggests that our dual-kernel design captures both local and global information more effectively than dilated alternatives, highlighting the efficiency of standard large kernels in the DKA framework.
Table 14: Comparison of different kernel designs on the BUSI dataset for classification under varying training ratios. Experiments based the pretrained ViT-B, ACC and SEN are reported as percentages, while Fl is presented as raw value. The best results are highlighted in bold. "d" represents the dilation rate dilated convolution.
Kernel Designs
0.63%
1.25 %
100%
ACC (%)
3x3 (d=3) 3x3 (d=25)
66.77
77.32
93.61
3x3 (d=3) 1Ixll (d=5)
67.41
77.64
94.25
3x3 (dd3))+22xx)) (d=2)
68.69
77.96
94.37
5x5 + 51x51
74.23
79.46
95.89
F1
3x3 (d=3) +333 (d=25)
0.641
0.751
0.931
3x3 (d=3) +1111 (d=5)
0.650
0.756
0.936
3x3 (d=3) + 26x26 (d=2)
0.663
0.758
0.942
5x5 + 51x51
0.720
0.774
0.951
SEN (%)
3x3 (d=3) 333 (d=25)
62.81
74.35
92.36
3x3 (d=3) + 11x11 (d=5)
63.99
74.43
93.44
3x3 (d=3) + 26x26 (d=2)
64.72
74.71
94.03
5x5 + 51x51
71.82
76.78
95.46
Table 15: Comparison of different kernel designs the ISIC-2018 dataset for segmentation under varying training ratios. Experiments are based on the pretrained Segmenter-B. mloU and Dice are reported percentages. The best results highlighted in bold. "d" represents the dilation rate in dilated convolution.
Kernel Designs
0.63%
1.25 %
100 %
mloU (%)
3x3 (d=3) + 3x3 (d=25)
60.25
72.71
76.43
3x3 (d=3) + 11x (d=5)
61.46
73.11
76.97
3x3 (d=3) 26x26 (d=2)
62.27
73.43
77.48
5x5 51x51
63.13
74.27
78.53
Dice (%)
3x3 (d=3) 3x3 (d=25)
75.19
84.20
86.64
3x3 (d=3) + 11x (d=5)
76.13
84.46
86.99
3x3 (d=3) + 26x26 (d=2)
76.75
84.68
87.31
5x5 + 51x51
77.39
85.24
87.97
26

---

Published as a conference paper at ICLR 2026
F.6 EFFECT OF DIVERSE KERNEL COMBINATIONS
To further analyze the contribution of different kernel configurations in DKA, we evaluate multiple kernel combinations for both classification and segmentation tasks. Specifically, we compare three settings: (7) 5 x + 11 x 11 + 51 x 51, (ii) 5 x 5 + 31 x 31 + 51 x 51, and (iii) x 5 51 x 51. Table 16 reports results on the BUSI classification dataset, while Table 17 presents results on the ISIC-2018 segmentation dataset.
Across both tasks, we observe that the dual-kernel configuration x 5 + 51) consistently achieves the best trade-off, outperforming the three-branch alternatives in terms of ACC, F1, and SEN for classification, as well mloU and Dice for segmentation. These results indicate that adding intermediate kernels (e.g., 11 11 or 31 31) does not provide additional benefits, and a simpler dual-kernel design is sufficient to capture both local and global dependencies. This further validates the efficiency of our proposed kernel selection strategy within DKA.
Table 16: Comparison of diverse kernel combinations on the BUSI dataset for classification under varying training ratios. Experiments are based on the pretrained VTT-B. ACC and SEN are reported as percentages, while F1 presented raw value. The best results are highlighted bold.
Kernel Combinations
0.63%
1.25%
100%
ACC (%)
1x1+11x11 x51
72.68
77.00
94.89
5x5 +31x31 51x51
71.04
76.68
94.57
5x5 + 51x51
74.23
79.46
95.89
F1
5x5 +
0.709
0.743
0.942
0.694
0.738
0.936
5x5+51x51
0.720
0.774
0.951
SEN (%)
51
69.17
73.56
94.11
5x5 +31x31 51x51
68.39
72.85
93.62
5x5 + 51x51
71.82
76.78
95.46
Table 17: Comparison of diverse kernel combinations on the ISIC-2018 dataset for segmentation under varying training ratios. Experiments based the pretrained Segmenter-B. mloU and Dice are reported as percentages. The best results are highlighted bold.
Kernel Combinations
0.63%
1.25%
100%
mloU (%)
5x5 +15x11111+ S1x51
62.72
73.76
78.06
5x5 +31331 51x51
62.40
73.63
77.54
5x5 + 51x51
63.13
74.27
78.53
Dice (%)
5x5 +11x111
77.09
84,90
87.68
5x5 +31x31 51x51
76.84
84.81
87.35
5x5 + 51x51
77.39
85.24
87.97
27

---

Published as a conference paper at ICLR 2026
F.7 EFFECT UNDER EXTREME LOW-DATA REGIMES
To further investigate the limits of DKA, we conduct experiments under an extreme low-data regime with only 0.125% of the training available. This setup approximately corresponds 5-shot setting for COVID classification and genuine 1-shot settings for BUSI and ISIC-2019 (classification), as well as BRATS, BUSI, and ISIC-2018 (segmentation). The results are reported in Table 18 and Table 19
Across all datasets, DKA consistently outperforms both Linear Probing and standard Adapter, even under such highly constrained supervision. For example, on ISIC-2019 classification, DKA improves ACC from 52.54% (Linear Probing) 55.95% and F1 from 0.227 0.263. Similarly, on BUSI segmentation, DKA achieves Dice of 37.43%, substantially higher than Linear Probing (23.02%) and Adapter (24.93%). These results confirm that the proposed large-kernel kernel design remains robust and effective even in the most challenging few-shot scenarios, highlighting its practicality for data-scarce medical applications.
Table 18: Comparison of Linear Probing, Adapter, and DKA on three classification datasets under 0.125% training data. Experiments are based on the pretrained VYT-B. ACC and SEN are reported as percentages, while Fl is presented raw value. The best results are highlighted in bold.
Methods
COVID
BUST
ISIC-2019
ACC (%)
Linear Probing
76.51
37.06
52.54
Adapter 71.38
35.46
47.25
DKA
78.74
39.62
55.95
F1
Linear Probing
0.735
0.273
0.227
Adapter
0.704
0.247
0.153
DKA
0.787
0.340
0.263
SEN (%)
Linear Probing
72.53
33.87
21.58
Adapter 68.40
30.41
15.07
DKA
75.64
37.13
24.39
Table 19: Comparison of Linear Probing, Adapter, and DKA three segmentation datasets under 0.125% training data. Experiments are based on the pretrained Segmenter-B, mloU and Dice reported as percentages. The best results are highlighted in bold.
Methods
BRATS
BUSI ISIC-2018
mloU (%)
Linear Probing
3.25
19.64
53.21
Adapter
1.20
14.24
47.05
DKA
6.36
32.83
60.25
Dice (%)
Lincar Probing
6,94
23.02
69.46
Adapter
2.40
24.93
63.99
DKA
11.96
37.43
75.19
28

---

Published as a conference paper at ICLR 2026
F.8 EFFECT ON MEDICAL-PREETRINEE VISION-LANGUAGE MODELS.
To further validate the generalization ability of DKA, we extend our evaluation to medical vision- language models. Specifically, adopt MedCLIP (Wang et al. 2022) the backbone and follow the few-shot image classification protocol commonly used in prior works on vision-language adapta- tion (Shakeri et al. 2024 [Silva-Rorrigezz al.) 2024). We report results the BUSI dataset under 1-shot, 4-shot, and 8-shot settings, where each configuration is averaged over five random seeds. As summarized in Table 20 DKA consistently surpasses linear probing and standard adapter tuning across all support sizes. These results demonstrate that the benefits of DKA are not limited to vision- onlly large pretrained models, but also extend multimodal -anguala models, highlighting its robustness in broader medical Al scenarios.
Table 20: Comparison different methods under few-shot settings on the BUSI dataset based on MedCLIP. Results are reported in terms of ACC, FI, and SEN.
Tuning Strategies
1-shot
4-shot
8-shot
ACC
Lincar Probing
66.25
74.21
78.41
Adapter
65, 86
74.08
79.32
DKA
71.20
79.82
82.27
F1
Linear Probing
0.631
0.712
0.776
Adapter
0.625
0.706
0.782
DKA
0.697
0.762
0.815
SEN (%
Linear Probing
62.55
69.72
77.45
Adapter
61.87
69.46
78.96
DKA
68.17
75.85
81.87
F.9 FREQUENCY-DOMAIN ANALYSIS OF KERNEL SIZES
To better understand the complementary roles of small and large kernels in DKA, analyze their frequency responses using the radial power spectral density (PSD). Specifically, we compute the spectral centroid and the normalized frequency foo that captures 90% of the cumulative energy. TT shows shoows the results for 5 x 5 and 51 x 51 kernels. The smaller kernel exhibits a higher spectral centroid (Se 0.618 vs. 0.503), indicating stronger sensitivity to high-frequency details. In contrast, the larger kernel shifts energy towards lower frequencies, facilitating global context modeling. This analysis provides further evidence for the effectiveness of combining diverse kernel sizes in DKA.
Table 21: -requency-domain analysis of kernel sizes. Results are reported as mean over different trained models.
Kernel Size
f9o
5
0.618 0.018
0.924 + 0.011
51 x 51
0.503 + 0.004
0.904 + 0.003
F. INFERENCE LATENCY AND MEMORY USAGE
We also report the inference efficiency of different methods in terms of latency and memory consump- tion the BUSI dataset using the pretrained ViT-B. As summarized in Table 22 DKA introduces only marginal overhead compared to the standard adapter framework, with inference latency increasing by less than 0.5 ms and memory usage by less than 6 MB. These differences are negligible in practice, indicating that the proposed large-kernel design achieves substantial performance gains with minimal computational cost.
29

---

Published as a conference paper at ICLR 2026
Table 23: Comparison of methods on the BUSI segmentation dataset with batch size 4 based on the pretrained Segmenter-B.
Methods
0.63%
1.25%
100%
Full Fine-tuning
27.02
33.11
57.91
Linear Probing
25.81
32.47
54.60
Adapter
18.67
26.50
55.46
DKA
27.26
35.21
59.35
Table 22: Comparison of inference latency and memory usage the BUSI dataset based on the prefrained ViT-B, DKA introduces only negligible overhead compared to standard adapter variants.
Methods
Inference Latency (ms)
Memory (MB)
Linear Probing
6.78
352.81
Adapter
11.54
433.70
Adapter + 5x5 Cony
11.66
433.90
Adapter + x51 Conv
11.69
439.56
DKA
11.97
439.63
F.11 EFFECTS OF LARGER BATCH SIZE
We further conducted additional segmentation experiments using larger batch size of under the same configuration. As shown in Table 23 increasing the batch size does not change the relative ranking: DKA consistently outperforms Full Fine-tuning, Linear Probing, and Adapter across all label ratios. This confirms that our improvements are robust to batch-size variations and are not tied to specific optimization setting.
F.12 PARAMETER EFFICIENCY ANALYSIS
The DKA module integrates two depthwise convolution branches with kernel sizes kn and k2 within each adapter. These convolutions are applied independently on each the d channels, contributing d(A: + k-)) parameters per module. Since DKA inserted twice in every Transformer block (after the attention and feedforward layers), the total number of additional parameters grows linearly with the number of blocks. Even with two kernels (c.g., k 51, l- = 5), the total number of trainable parameters introduced by all DKA modules remains less than 2% of the pretrained backbone.
maintaining strong parameter efficiency while providing strong performance gains, especially in low-data regimes.
F.13 COMPLEMENTARY CAM VISUALIZATION
To complement the ERF analysis, we additionally compare Grad-CAM (Selvaraju et al. 2017) responses of the standard Adapter and our DKA on BUSI. As shown in Figure [1] the standard Adapter exhibits diffuse and background-driven activations, whereas DKA produces coherent, lesion- centered responses. This confirms that enlarged ERF of DKA captures meaningful contextual information rather than merely expanding activation range.
30

---

Published as a conference paper at ICLR 2026
Figure 11: Grad-CAM comparison on BUSI. Left: standard Adapter; right: DKA with more focused, lesion-related activations.
31

---

Published as a conference paper at ICLR 2026
G PSEUDOCODE
Algorithm Pseudo-code of Transformer block with DKA class DKA:
def
_iiiiit (self, dim, middle dim, kernel_large kernel_small
self. downsampl Linear (dim, middle_dimi
self.conellarge Depthwi seConv (mi (middle_dim, kernel_larg
celi.conv_smal small DepthwiseConv (middle dim, kernel small)
self.activation - GELU ()
self.upsample Linear (mi middle dim, dim)
def forward
store the input for the residual connection
residual x self welff.dwnsse downsamplseeee (x)
Dual-Path Convolutions (Large + Small)
l_large self.conv. colf.convllaree (x)
small self.conv wtt.vossnallllllaceeccee swrvsssaallle (x)
x x_large x_smal
X self activation (x)
X self. upssamlee (x)
x residual
return
class TransformerBlock with DKA:
def
Enit_((elf,, dim, num_heads, mip_ratio, middle_dim, kernel_large ternel_small
Original ViT components
self Multtheaddttentiinn.coc (dim, num_heads)
self LayerNorm (dim)
self mlp MLP (dim, mip_raae: ratio)
nerm.norm2 LayerNorm(dim) (dim)
DKA Adaptor
self.dka = DKA (dim, middle_dim, kernel_large kernel_small]
def forward x)
residual
selm.norml (x)
x self.attn
x x residual
x sclf.de dka (x)
residual
self norm2 (x)
x self melf.miphe (x)
X x residual
x self, dka (x)
return
32


Reviews:
[{"review_id": "vUtGKCU2V1", "reviewer": "Reviewer_dSfe", "summary": "This paper studies adapter-based fine-tuning of large pretrained models under extreme data scarcity common in medical imaging. It reveals a surprising degradation of conventional adapters below 1% training data, where they perform worse than simple linear probing. The authors attribute this to a sharp reduction in Effective Receptive Field (ERF) under limited supervision. To overcome this, they propose the Dual-Kernel Adapter (DKA), a lightweight module that combines large-kernel (51×51) and small-kernel (5×5) depthwise convolutions in parallel to expand spatial context while preserving local detail. A comprehensive experimental evaluation across six medical imaging datasets (classification and segmentation), multiple backbones, and data regimes confirms that DKA consistently outperforms standard adapters and other parameter-efficient fine-tuning (PEFT) methods, especially in low-data settings.", "questions": "* Dataset Size Normalization: Can you provide a table showing the absolute number of training samples at each percentage for each dataset? How do results change when comparing at equal absolute sample sizes rather than percentages?\n* LoRA Configuration: What rank was used for LoRA in all experiments? Have you tried increasing LoRA rank to match DKA's parameter count?\n* Fair Comparison: Can you provide a comprehensive parameter-performance trade-off analysis where all methods are compared at multiple parameter budgets?\n* ERF Causality: How can you demonstrate that ERF reduction is the causal factor rather than just correlated with poor performance?\n* Optimization Dynamics: Given the batch size constraints (especially batch=1 for segmentation), how much of the performance difference could be attributed to optimization difficulties?\n* Alternative Architectures: Have you explored other ways to increase receptive fields, such as self-attention modules within adapters or hierarchical designs?\n* Theoretical Understanding: Can you provide more theoretical insight into why standard adapters fail in low-data regimes beyond the ERF hypothesis?\n* Hyperparameter Sensitivity: How sensitive is DKA to the specific kernel size choices? Would adaptive or learnable kernel sizes be beneficial?\n* Computational Cost: What is the actual inference time and memory overhead for 51×51 convolutions on high-resolution medical images?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- Important Problem: Addresses the critical challenge of limited labeled data in medical imaging\n- Comprehensive Evaluation: Thorough experiments across 6 datasets, multiple backbones, and various data scales\n- Surprising Finding: The observation that adapters can hurt performance under extreme data scarcity is counter-intuitive and valuable\n- Strong Empirical Results: DKA shows consistent improvements, particularly in low-data settings\n- Extensive Ablations: Thorough analysis of design choices including kernel sizes, learning rates, and architectural variations\n- Practical Impact: The method is parameter-efficient and shows minimal computational overhead", "weaknesses": "* Limited Technical Novelty: The solution essentially adds large-kernel convolutions to adapters - this is a straightforward extension rather than a fundamental innovation\n* ERF Analysis Concerns:\n   * The causal relationship between ERF and performance is assumed but not proven\n   * ERF computation methodology needs clarification\n   * Alternative explanations (e.g., optimization difficulties, overfitting) are not thoroughly explored\n* Kernel Size Selection:\n   * The choice of 51×51 kernels seems arbitrary and extreme\n   * Limited theoretical justification for why such large kernels are necessary\n   * Computational implications of such large kernels in 3D medical imaging are not discussed\n* Missing Comparisons:\n   * No comparison with recent vision-specific PEFT methods (e.g., Visual Prompt Tuning variants)\n   * Limited exploration of other architectural modifications that could expand receptive fields\n* Experimental Concerns:\n   * Batch size of 1 for segmentation might affect optimization dynamics\n   * The asynchronous learning rate finding (1e-3 for adapter, 1e-4 for head) seems important but is relegated to ablations\n* Dataset Size Discrepancy Makes Comparisons Misleading:\n   * Critical Issue: Figure 1 compares performance across datasets using percentages (0.63%, 1.25%, etc.) but these datasets have vastly different absolute sizes. For example, Tiny ImageNet has 100,000 training images while COVID has only 3,600. This means 0.63% of Tiny ImageNet (630 images) could be larger than 25% of COVID (~900 images).\n   * This makes the comparison fundamentally flawed - the paper is comparing different absolute data quantities while claiming to study \"low-data\" regimes\n   * The authors should report absolute sample numbers and normalize comparisons appropriately\n* Missing Critical Implementation Details:\n   * LoRA Rank: The paper doesn't specify what rank was used for LoRA, which is crucial as rank fundamentally determines LoRA's capacity and parameter count. A rank-1 LoRA vs rank-64 LoRA are completely different methods.\n   * Without this information, the comparison with LoRA is meaningless\n* Unfair Parameter Comparison:\n   * While the paper claims DKA adds <2% parameters, there's no detailed comparison of parameter efficiency\n   * The paper should provide:\n      * Exact parameter counts for each method\n      * Performance vs. parameter trade-off curves\n      * Comparison at iso-parameter budgets (e.g., what if LoRA used higher rank to match DKA's parameters?)\n   * Table in Figure 5 attempts this but only varies hidden dimensions, not comprehensively\n* Limited Applicability to Real Medical Imaging:\n   * The 51×51 kernel design is completely impractical for 3D medical imaging (MRI, CT scans)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TBkjl7kS2p", "reviewer": "Reviewer_MhXB", "summary": "This paper introduces Dual-Kernel Adapter (DKA) — a lightweight, parameter-efficient fine-tuning module designed for medical imaging tasks under extreme data scarcity. The authors first identify that conventional Adapters unexpectedly degrade performance when trained with < 1% of data, performing worse than linear probing due to reduced Effective Receptive Field (ERF).\n\nComprehensive experiments across multiple classification (COVID, BUSI, ISIC-2019) and segmentation (BRATS, BUSI, ISIC-2018) datasets show DKA consistently outperforms full fine-tuning and other PEFT baselines (AdapterFormer, LoRA, Convpass, AIM).\n\nThe paper also presents detailed ERF visualizations, ablations (kernel size, branch design, middle dimension, learning rates), and evidence that performance improvements stem from large-kernel receptive-field expansion rather than parameter count", "questions": "How does DKA scale computationally for high-resolution 3-D MRI or CT volumes?\n\nCan large kernels be replaced by dilated or deformable convolutions to reduce cost?\n\nHave you tested DKA in few-shot fine-tuning (< 10 samples/class) or cross-institutional domain adaptation?\n\nCould self-supervised initialization (e.g., SimCLR) further amplify DKA gains under extreme scarcity?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The empirical findings are compelling and well-supported: quantitative metrics (ACC, mIoU, Dice) improve across all data scales, especially ≤ 1.25%.\n\nThe ERF analysis provides strong evidence linking reduced receptive field to Adapter degradation and DKA’s advantage.\n\nThe controlled parameter-count experiment convincingly isolates kernel size as the key factor.", "weaknesses": "The theoretical reasoning behind ERF–generalization linkage could be formalized further.\n\nComputational overhead of large-kernel depthwise convolutions is not fully quantified.\n\nLimited theoretical depth: lacks analytical characterization of why ERF → generalization scaling behaves linearly with data size.\n\nCompute trade-off: large-kernel (51×51) convolutions increase FLOPs; energy/memory costs are not discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5lOkDVtPvu", "reviewer": "Reviewer_reej", "summary": "The authors conduct a thorough study on the effect of adapters in fine tuning large-scale pretrained models in the low data regime prevalent in the medical image segmentation and classification. The authors notice a sizable reduction in performance in limited data scenarios and attribute it to a reduced effective receptive field (ERF). Accordingly, they use known methods from literature to build an adapter based on large kernels convolutions to counter this reduction in ERF. They demonstrate that by doing so, performance improves on multiple pretrained networks against 10 baselines in tasks in classification as well as segmentation. They confirm that the improvement is owing to kernel size and not parameter count and include useful ablation experiments.\n\nThe method proposed in the paper is well motivated with a clear analytical section in the first part of the paper with a clear takeaway on ERF reduction in data sparsity. They propose a simple albeit effective method to address this problem and demonstrate that the problem is addressed on a number of pretrained models, datasets and tasks. Their ablation experiments are also useful takeaways in terms of training strategies and architecture design. Overall, the paper forms a positive impression and should be accepted.", "questions": "1. Does the Dual-Kernel Adapter require a step where the features move from token space into the spatial domain and back again?\n2. Can the authors offer any insight into whether their findings will extend into the 3D medical image segmentation space? Are there any restrictions towards the methods being used in such a setting as datasets like BraTS (Table 2) for example are also popular in the 3D segmentation space?\n3. Are the authors restricted to the token space in their adaptation? As in [1], should the authors not also be able to target ConvNeXt blocks?\n\nReferences:\n\n[1] Zhu, Ziquan, et al. \"LKA: Large Kernel Adapter for Enhanced Medical Image Classification.\" International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2025.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The work is well motivated with the analysis demonstrating the reduction in performance and tying that to a reduction in the effective receptive field with low data settings. To address this problem a simple large kernel convolution based dual kernel adapter is proposed.\n2. The results use multiple pretrained models, a large number of baseline methods and clearly demonstrates that the proposed method works well.\n3. The proposed method itself is extremely simple but the novelty lies in it uniquely solving the problem as motivated by the analysis in the first part of the paper.\n4. The work is well written and well illustrated with a very efficient use of space to convey the key messages of the paper.", "weaknesses": "1. There is seemingly the idea of reordering the tokens back into the spatial domain that is a part of the method design. However, the authors make no mention of this in the methods section.\n2. The work bears similarities to another large kernel adapter method [1] and seems to differ methodologically owing solely to a dual path convolution and the analysis in the first part of the paper. In fact, there also is an extremely similar analysis in [1] titled \"Large Kernel Matters Instead of #Trainable Parameters\" bearing similarities to the section 4.3 LARGE KERNEL MATTERS INSTEAD OF TRAINABLE PARAMETERS. While the similarities are unfortunate, it does significantly dampen the novelty of this work and positions it more as a derivative of existing work - while being aided by significantly denser analysis compared to [1]. However, this paper should be cited in the section for the single branch comparisons for proper referencing.\n\nReferences:\n\n[1] Zhu, Ziquan, et al. \"LKA: Large Kernel Adapter for Enhanced Medical Image Classification.\" International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nKkWFuwvqm", "reviewer": "Reviewer_MiK3", "summary": "1.The paper addresses the challenge of applying parameter-efficient fine-tuning (PEFT) methods, such as Adapters, in medical imaging where extreme data scarcity is common. The authors argue that conventional Adapters underperform in these low-data settings. To mitigate this, they propose the Dual-Kernel Adapter (DKA), a novel module that incorporates dual-path depthwise convolutions—one with a large kernel and one with a small kernel. This design is intended to allow the adapter to effectively capture both local fine-grained details and global contextual information. The results show that DKA significantly outperforms conventional Adapters, LoRA, and even full fine-tuning, especially under extreme data scarcity (1% of training data), achieving state-of-the-art results on multiple 2D and 3D medical image segmentation benchmarks.", "questions": "1.The evaluation focuses exclusively on 2D and 3D segmentation. Does the DKA demonstrate similar compelling performance improvements when applied to other common medical imaging tasks, such as disease classification or object detection, under the same low-data constraints?\n2.Is the 51×51 large kernel in DKA feasible for 3D medical imaging (e.g., BRATS with 3D MRI volumes)? A direct extension to a 51×51×51 kernel would lead to prohibitive parameter and computational costs.\n3.While Appendix F.10 shows minimal overhead, could you explain on how DKA scales with image resolution? \n4.The Effective Receptive Field (ERF) is computed based on the magnitude of input-output gradients, which reflects spatial influence but not semantic relevance. Have the authors considered complementary analyses to verify that the expanded ERF indeed captures meaningful contextual information rather than just broader but irrelevant activations?\n5.Are there any datasets or settings where DKA underperforms standard adapters?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.This work is presented as the first comprehensive study of adapter-based fine-tuning specifically tailored for low-data medical imaging scenarios.\n2.The Dual-Kernel Adapter (DKA) is a novel architectural design within the PEFT space. Explicitly integrating multi-scale spatial feature extraction (via dual-path convolutions with different kernel sizes) into the adapter structure is a unique and effective contribution to improving PEFT performance in a difficult data regime.\n3.The technical claims are robustly supported by strong experimental evidence, showing DKA's significant outperformance against well-established baselines like conventional Adapters, LoRA, and full fine-tuning.\n4.The results are highly significant for the medical imaging and broader computer vision communities. A PEFT method that achieves state-of-the-art results with as little as 1% of the training data is an immense breakthrough for high-cost, high-stakes domains like healthcare.", "weaknesses": "1. The choice of 51×51 as the large kernel seems somewhat arbitrary. While ablations justify it empirically, a more principled analysis (e.g., based on image resolution or pathology size) would be valuable. And is this optimal across all medical imaging modalities?\n2.Given the added complexity of the dual-kernel design, please provide a comprehensive analysis of the latency (inference time) and FLOPs of DKA relative to LoRA and the conventional Adapter, across the same backbone and hardware.\n3.Although the paper demonstrates DKA’s superiority across multiple datasets and settings, it does not systematically investigate scenarios where DKA might fail. For example: Could the large kernel introduce noise in tasks where pathological structures are extremely small or sparsely distributed? Is DKA still robust under extreme class imbalance (e.g., in ISIC-2019, where some classes have very few samples)? Are there specific backbone architectures (e.g., pure CNNs) for which DKA provides no benefit or even degrades performance?\n4.As noted in Appendix B.2, segmentation experiments use a batch size of 1 due to GPU memory constraints. This may lead to highly noisy gradient estimates, especially under low-data regimes and introduce optimization instability that confounds fair comparison across methods. In real-world deployment scenarios, larger batch sizes are typically used.\n5.While the paper uses ERF visualizations and frequency-domain analysis to argue that large kernels expand the receptive field, the authors fails to establish a theoretical or information-theoretic link between large kernels, improved generalization, and robustness under data scarcity;", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)