GaRA-SAM: Robustifying Segment Anything Model with Gated-Rank Adaptation
Authors: Sohyun Lee
Yeho Gwon
Lukas Hoyer
Suha Kwak
Venue: NeurIPS 2025 poster ()
Keywords: Robustness
Segment Anything Model
LoRA
Primary area: applications

Abstract: Improving robustness of the Segment Anything Model (SAM) to input degradations is critical for its deployment in high-stakes applications such as autonomous driving and robotics. Our approach to this challenge prioritizes three key aspects: first, parameter efficiency to maintain the inherent generalization capability of SAM; second, fine-grained and input-aware robustification to precisely address the input corruption; and third, adherence to standard training protocols for ease of training. To this end, we propose gated-rank adaptation (GaRA). 
GaRA introduces lightweight adapters into intermediate layers of the frozen SAM, where each adapter dynamically adjusts the effective rank of its weight matrix based on the input by selectively activating (rank-1) components of the matrix using a learned gating module. This adjustment enables fine-grained and input-aware robustification without compromising the generalization capability of SAM. Our model, GaRA-SAM, significantly outperforms prior work on all robust segmentation benchmarks. In particular, it surpasses the previous best IoU score by up to 21.3%p on ACDC, a challenging real corrupted image dataset.

GaRA-SAM: Robustifying Segment Anything Model with Gated-Rand Adaptation
Sohyun Leel
Yeho Gwon'
Lukas Hoyer
Suha Kwak
POSTECH
2 Google
Abstract
Improving robustness of the Segment Anything Model (SAM) to input degrada- tions is critical for its deployment in high-stakes applications such as autonomous driving and robotics. Our approach this challenge prioritizes three key aspects: first, parameter efficiency to maintain the inherent generalization capability of SAM; second, fine-gres -iaiieg and imput-aware robustification to precisely address the input corruption; and third, adherence standard training protocols for ease training. To this end, we propose gated-rank adaptation (GaRA). GaRA in- troduces lightweight adapters into intermediate layers of the frozen SAM, where each adapter dynamically adjusts the effective rank of its weight matrix based the input by selectively activating (rank-1) components of the matrix using a learned gating module. This adjustment enables line-grained and input-aware robustification without compromising the generalization capability of SAM. Our model, AARA-SAM, significantly outperforms prior work on all robust segmenta- tion benchmarks. In particular, it surpasses the previous best IoU score by up 21.3%p on ACDC, a challenging real corrupted image dataset.
Introduction
The Segment Anything Model (SAM) (8 has proven to be powerful tool for zero-shot image seg- mentation, exhibiting impressive generalization across unseen objects and images without additional training. Nevertheless, its performance deteriorates considerably when faced with degraded input due to noise, blur, low illumination, and adverse weather [8], as shown in Figure [b). This limitation significantly restricts its use in high-stakes applications such autonomous driving and robotics.
seemingly straightforward approach improving the robustness SAM attach an existing image restoration module to the front of SAM. However, this typically introduces significant compu- tational overhead, and often yields suboptimal segmentation performance since image restoration is optimized to enhance the perceptual quality of images, rather than to improve performance of segmentation models like SAM (7 30 33 52 57]. An alternative strategy involves fine-tuning SAM entirely degraded inputs, which, however, demands substantial computational resources and diminishes its inherent zero-shot generalization capability [8].
Chen et al. 8 addressed these limitations by introducing anti-degradation modules into SAM; these modules are trained refine degraded features by promoting feature-level consistency between pair of clean and corrupted images of the same content. Despite effectively enhancing robustness, their method, dubbed RobustSAM, has couple of limitations. First, its requirement for paired clean and degraded images, which cannot be captured by typical cameras, necessitates the use of synthetic degradations training, hindering generalization real-world degradations. Second, since RobustS AM aims at learning representations invariant to various degradations, it struggles to produce representations adapted to the specific degradation affecting the input at hand, restricting
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Sepmersatin
(b)
Inss
Acthvate [rank-1] comporents
composents
Mask
Mask
X
X
X
SAM
Foq
ronk=2
X
X
X
Mask Decoder
Input
Restored Input
X
X
X
X
Goussian
rank=5
X
X
X
X
GoRA
Prompt Encoder
Encoder
Unseen
X
X
SAM
Robus:SAM
Corruption
X
X
X
ronk-3
Point Prompt
Box
Input
Prompt
Image
X accive
Inactive
GaRA-SAM
Ground-Truth
Figure I: Overview and example results of GaRA-SAM. (a) Conceptual illustration of GaRA-SAM. (b) Example results on a real low-light image [5]. GaRA-SAM produces an accurate mask while the original SAM fails. Note that the restored input provided for illustrative purposes only; GaRA-SAM does not perform image restoration.
further performance improvement. These limitations reduce the effectiveness of RobustSAM in real-world scenarios, where inputs can be degraded by real and previously unseen corruptions.
We found that low-rank adaptation (LoRA) [20] offers a promising foundation for overcoming many of the previously discussed limitations. By introducing and training lightweight adapters within the frozen SAM, LoRA improves the robustness parameter-efficiently while preserving the generalization ability of SAM. Our empirical analysis confirms that SAM incorporating LoRA achieves impressive robustness without using paired clean and degraded images for training, thus establishing strong baseline as is. Nevertheless, our analysis also reveals a key limitation: the fixed rank of the adapters hinders its effective adaptation to a wide range of corruptions. We found significant variation in the optimal rank of the adapters between different corruption types and inputs, underscoring the need for input-adaptive rank modulation.
Based on this observation, we introduce gated-rank adaptation (GaRA), which is illustrated in Figure [[a,, a novel method designed to enhance the robustness of SAM while addressing the aforementioned limitations. GaRA dynamically adjusts the effective rank of the weight matrix of each adapter. while being curffmter-efffcientt and not demanding clean -egraded image pairs for training. Specifically, GaRA decomposes the weight matrix of an adapter into (rank-1) components and chooses a proper subset of them dynamically according to the input. To achieve this, we introduce a gating module that predicts a binary vector selectively activating the most appropriate components for the input. This mechanism allows GaRA to flexibly control both the number and combination of active components based on the input without any test-time optimization. Consequently, our zero-shot segmentation model integrating GaRA with SAM, named GaRA-SAM achieves fine- tne-grained and input-aware robustification while remaining parameter-efficient
GaRA-SAM achieves the state of the art on multiple robust segmentation benchmarks 5 47 51 55 6I), including both synthetic and real-world corruption datasets. Importantly, and in contrast to prior work, the design of GaRA-SAN enables training using real-world degraded images lacking clean counterparts, leading to notable performance improvement on real-world corruption benchmarks. The main contribution of this work is three-fold:
Our extensive analysis reveals the surprising effectiveness of LoRA in robustifying SAM, with its optimal rank varying significantly across different corruption types and individual images. These findings suggest research avenue for improving robustness of vision foundation models.
We propose GaRA, novel method to achieve robust SAM. At its core lies a lightweight and input-dependent adapter that enables fine-grained and parameter-efficient robustification without compromising the generalization capability of SAM. Also, GaRA does not require paired clean and degraded images for training and thus, unlike previous work g8) it can be learned using real degraded images without their clean references.

---

.
Our final model, GaRA-SAM, achieves the state of the art across multiple robust segmentation benchmarks. Notably, it significantly surpasses the previous best IoU score by up to 21.3%pp on ACDC, a challenging real-world corrupted image dataset.
2 Related Work
Robust Segment Anything. SAM [28] accepts free-form prompts along with image to produce relevant masks, showing superior zero-shor shot generalizability. Despite its success, its robustness against visual corruptions is questionable 8 48). To improve the robustness of SAM, RobustSA. [ adopts anti-degradation modules to approximate features of clean images. However, it requires clean- degraded image pairs for training, which are often unavailable in real-world scenarios. Improving the robustness of vision models with image restoration 32, 43 46 [52] and degradation
techniques 3 5 30 44 has also been investigated. AirNet [32 and URIE [52] are universal image restoration models, but they introduce heavy computational overhead and AirNet targets a better image quality, not improving performance of visual perception models. LORA-IR M. DA-CLIP 4 and PromptIR [46] tackle this by encoding degradation-spedde specific information into prompts. However, they rely on additional information such as degradation types and textual descriptions. and suffer from their complex modules and training schemes. Meanwhile, FIFO [30] and FreD [3] focus on extracting fog-invariant features, making their models robust to only foggy scenes. In contrast to these prior arts, GaRA is an efficient yet effective approach to fine-tuning SAM for improving its robustness without demanding clean references of degraded images for training.
Low-rank Adaptation. Fine-tuning large-scale pre-trained models introduces intensive overheads in space and time, To overcome this, various parameter-efficien learning schemes such as prompt tuning [33 [1]35]]] (50] and adapter-based fine-tuning 6 I 16 [8]
26 [36 45 60) have emerged. Among these, LoRA 19 leverages trainable low-rank matrices to catch up with full fine-tumi fine-tuning while introducing marginal extra learnable parameters. However, fixed ranks of LoRA yield limitations
19
such as poor generalization in certain tasks [56] 59 and inefficient parameter allocation 4 63
Despite the efforts of previous work to update the ranks during training dynamically [39, 56 633, they
still use fixed ranks at test time and require handcrafted rank selection. In contrast, GaRA learns a small module that selects and combines appropriate (rank-1) 1) components of LoRA block, adapting to individual samples with various visual corruptions even during inference.
Mixture of Experts. While model scaling has been known as an effective and promising way of
constructing powerful model, training such model on large-scale datasets 2
29 54 62 is
challenging due to high computational requirements [4 49]. To this end, mixture of experts (MoE). which adopts multiple submodules and considers each as an expert, has gained prominence [4 21
38
58 64). These submodules are active or inactive using a learnable gating module at both training and test time, resulting in efficient use of resources and improved training stability. Switch Transformer [1] sparsely simplifies the MoE paradigm and proposes to select experts sparsely. AdaMix 58 injects an MoE adapter consisting of various up- and down-sampling layers into each and performance. Motivated by the prior work, GaRA first decomposes the low-rank matrices of LoRA into (rank-l) components and considers each as expert. Then, a learnable gating module sparsely selects combination of (rank-1) components in an adput-adaptive manner, enabling adaptation on a per-input basis.
49
transformer layer, both improving parameter-effiiientt
3 Proposed Method
We first describe LoRA as a strong baseline for robustifying SAM in Sec. 3.I. and analyze the impact of the rank of its adapters on the robustness of SAM to verify our motivation of input-adaptive rank modulation in Sec. 3.2 Finally, Sec. 3.3 details GaRA that we integrate into SAM as GaRA-SAM.
3.1 Foundation: Low-rank Adaptation for Robustifying SAM
LoRA 20 is parameter-efficient adaptation method that imbues frozen pretrained model with
adaptability using a handful of trainable parameters. It freezes a pretrained weight matrix Wa E
D and introduces trainable low-rank update AW = BA, where B E RDR and A E R RxX
with rank R & mi) K). The adapted forward pass becomes:
h = Wox + BAx,
(I)
3

---

RobustSAM
LoR4-Rank2
0.82
LoRA Rank18
Lan4-28
IoU
0.78
0.7
0.76
0.66
0.74
0.62
Gaussian
Compression
Motion
Snow
Eleatic
Zoom
Resampling
Rain
IS0
Frosted
Contrast
Fog
impluss
Color
Brihhrrasp
0.72
RobustSAM
LoRA-Rank2
LoRk-Rank16
LanA-Rank128
Oraele Cracle-Corupt Corrunt
Oracle-inttnnce instance
(8)
Figure 2: Impact of the rank in LoRA integrated with SAM. The models were evaluated on LVIS [5]
using point prompts. (a) Performance versus rank under various corruption types. The best rank varies depending on the corruption type. (b) Comparisons of rank selection strategies. Oracle-Corrupt chooses the best rank per corruption type, while Dracle-Instance selects the best rank per image. The outstanding performance of Oracle-Instange suggests the need for input-adaptive rank manipulation.
where x E RK is the input and h E IR is the output.
Given its original purpose of parameter-efficient adaptation without compromising generalization, we argue that LoRA offers a reasonable baseline for improving robustness of SAM against input degradation. To validate this potential empirically, evaluated SAM integrated with LoRA for robust segmentation using point prompts following an evaluation protocol of RobustS [8]. Specifically, froze the original weights of SAM, attached low-rank adapters AW the key, query, and value projection layers of its image encoder, and trained the adapters with the standard segmentation loss degraded images. As demonstrated in Figure 2 SAM with LoRA clearly outperformed RobustSAM 8 on the LVIS dataset (15), although it does not require clean reference images and the auxiliary loss the RobustSAM demands.
3.2 Analysis on the Impact of Rank on Robustness
Since different degradations distort different aspects of the input, the level of representation capacity needed for robustifying SAM varies accordingly, motivating the investigation into the role of LoRA's rank. To analyze this, we first evaluated its performance across various input corruptions while varying the rank. The results in Figure 2 ()) suggest that the optimal rank varies depending on the corruption type. We conjecture that more pronounced corruptions lead to more substantial contamination of the semantic content of the input, and lower-rank adapters are more effective in such conditions since their restricted capacity forces them to prioritize more essential features for segmentation 42
To further verify this conclusion, evaluated SAM with LoRA under two oracle rank modulation scenarios: selecting the best rank per corruption type (Oracle-Corrupi and per image (Oracle- Instance) using ground-truth. Specifically, selected only the top-perte perfoffrrning rank for each corruption type or each image. While Oracle-Corrupi yielded moderate gains over fixed ranks, Oracle-Instance achieved significantly higher performance. highlighting the substantial potential of fine-grained, input-adaptive rank selection. These findings motivate our design of GaRA, which dynamically adjusts the rank per input.
first computed IoU scores for all candidate ranks and
3.3 Gated-rank Adaptation
Our solution for robustifying SAM, termed gated-rank adaptation (GaRA). is designed with three key properties in mind: parameter efficiency to preserve the generalization ability of SAM,, fine-grained and input-aware robustification, and adherence to standard segmentation learning protocols for ease of training. These properties are realized by a novel, lightweight adapter that dynamically adjusts itself based on the input corruption; the architecture of the adapter is depicted in Figure [ Our final model, GaRA-SAM, constructed by integrating these adapters into the key, query, and value projection layers of the image encoder of SAM, keeping the original SAM weights frozen. Also, GaRA-SAM trained solely with the standard segmentation loss computed from degraded images, without auxiliary learning objectives paired clean references.

---

:Active
Inortive
LoRA
GaRA
Higher-onk space
Higher
Lower
X
X
X
by (af)) by (
by 3"
bi (at)"
bg (4))
by (4A,))
2"
Linear
Linear
Lower-rane rank spore
s+
Ospace
bj (
(a
bj (at)
by (4.))))
Figure 3: Adapter architecture of GaRA and comparison to LoRA. GaRA leverages hierarchical gating for both coarse and fine control over the adaptation process. First, Ixpace selects between the higher-rank and lower-rank spaces based on the input. Then, the corresponding gating module g or g" predicts binary vector z" or zz to activate subset of (rank-1) components tailored to the input. These active components are composed to form the final update matrix for adaptation.
GaRA introduces a gating strategy to modulate the adapter's rank based on the input. To this end, reinterprete the update matrix of LoRA, Le., AW BA,, a composition of R (rank-1) components: W = E b,a," where b; E 2 and a, R are the i-th column and row of B and A, respectively. We assume that R is large enough so that the model has access to a rich set of (rank-1) components, enabling GaRA to flexibly choose a relevant subset depending on the input.
GaRA employs a hierarchical gating strategy that provides both coarse and fine control over the adaptation process. The adapter first coarsely selects between lower- and higher-rank spaces based on the input, and then, within the selected rank space, the gating module finely determines the rank of the adapter by activating subset of the (rank-l) components appropriate for the input. The coarse rank space selection separates the adapter's components into two exclusive sets, tailored to different degraded inputs demanding different representation capacities, mitigate potential conflicts between the sets and further improve performance. Mcanwhile, the fine- mne-grained gating enables flexible and input-specific composition of the update matrix, supporting efficient and expressive adaptation to diverse corruptions. Below we elaborate on this gating process.
Gating 1: Rank Space Selection. We explicitly divide the adapter into two distinct rank spaces, trw a lower-rank set (a b ) and higher-rank set {a b Jj=1 with r, < ThH & K, where K denotes the input feature dimension, and rr. and T'H indicate the maximal ranks of the lower-rank and higher-rank spaces, respectively To choose between these rank spaces, we employ binary gating module Skpace which takes as input the intermediate feature fxx computed from x and outputs a binary gating variable Zspace E {0. 1}, where O and indicate the lower- and higher-rant space, respectively. This module consists of a two-layer MLP followed by the Gumbel-San Gumbel-Sigmoid, allowing differentiable binary decisions during training. The gating process is formally expressed as:
MLP space (x)),
Oispace
where O'space is the gating logit computed from the input feature. To enable backpropagation despite the binary nature of the binary gating variable Espuce. we apply the Gumbel-Sigmoid relaxation 22
( + G)
space
where G ~ Gu, 1) is a noise sampled during training, o is the sigmoid function, and r is temperature parameter controlling the sharpness of the sigmoid. During training, we apply hard thresholding in the forward pass as &space = [[2space > 0.3] while using the continuous value &space to compute gradients in the backward pass. At test time, we discard the noise and compute the gate: Espace ((isace))) 0.5].
'We use the terms 'Jower-rank" and 'higher-rank' in relative sense. Both rz. and ra are substantially smaller than the full feature dimension K, and thus still lie in the low-rank regime.
5

---

(rank-1) Component Selection. Within the selected rank space, the associated gating module predicts a binary vector identifying (rank-1) 1) components suitable for the input:
Gating 2:
= = (f(x)) E 10, z" =" (f(x)) E 10.
where (*) and g g"(1 denote the gating modules for the lower-rank and higher-rank spaces, respec- tively. Each gating module consists of a three-layer MLP whose final output is a real-valued logit vector, followed by Gumbel-Sigmoid operations obtain binary masks:
at (f)), a MLPP
(S)
To enable gradient-based based learning, we apply the Gumbel-Sigmoid relaxation to each element of the logit vectors 22
2, = (-
i= 1 ................................ TL TH.
(6)
where o; the i-th logit from a² or H. Gi ~ Gumbel(0, 1) is the corresponding noise sample, 7 is a temperature parameter, and a is the sigmoid function. During training. hard thresholding is applied in the forward pass to produce binary decisions (z; = T[ > 0.5)) while the continuous 2, used for backpropagation. At inference time, the gating becomes deterministic: 2; I[o((a)) > 0.5]. We apply this relaxation to the gating functions 9b and gd to enable learnable and input-adaptive binary decisions for the selection of (rank-1) 1) components. This gating mechanism results in a dynamic, input-dependent adapter update:
FL Σ
TH Σ
AW- (1 2space)
2 b/"
+ Zspace
"anb"(aa)
This design of GaRA enables flexible and conflict-free adaptation to a wide range of corruptions by coarse-to-fine rank modulation.
4 Experiments
4.1 Experimental Setting
Dataset. For training and validation, we utilize the Robust-Seg dataset [ which is constructed by applying 15 types of synthetic corruptions to three semantic segmentation benchmarks: LVIS [5].
MSRA-10K (9. and ThinObjects-5K [37. comprising total of 26,000 masks. For evaluation,
we use five clear-condition image segmentation benchmarks: LVIS, MSRA-IOK, STREETS (5]), NDD20 [55) COCO 40).. Also, we test on real-world corrupted benchmark including BDD- 100K [61 and LIS S For training GaRA-SAN on real-world data, we use BDD-100K and LIS for training, and evaluate on BDD-100K and LIS, and ACDC (4).. Note that all these datasets are free from licensing issues. For brevity, we refer to the union of BDD-I0OK and LIS as BDD+LIS, and the union of STREETS and NDD20 STREETS+NDD. More details are given in the appendix.
Experimental Details. We adopt the VIT-B and VIT-L variants of SAM 13), and freeze their parameters during training the GaRA modules. The models are optimized by Adam [27] with learning rate of 10-4 for ViT-B and 1 x 10-5 for ViT-L,, a weight decay of x 10-b, and input batches of size 8, using both point and box prompts. The gating modules are trained with the same learning rate. We set the lower- and higher-rank dimensions as ry. = 16 and Iи = 256, respectively, and use Gumbel-Signe Sigmoid temperature of 0.5. Since no official evaluation code is provided by previous work, we reproduce and evaluate its models using the same protocol. All training and evaluation experiments were conducted at POSTECH.
4.2 Comparison on Seen Dataset
To assess the robustness of GaRA-SAM we first evaluate its performance on synthetic corruptions applied to seen datasets such as LVIS and MSRA-10K, The results in Table T show that GaRA- SAM outperforms prior methods, including RobustSAM R) HQ-SAM 25), and restoration-baser methods (e.g., AirNet 32) and URIE (52). when using both point and box prompts in both datasets. Notably, our model demonstrates significant improvements in the degraded setting, e.g., surpassing the previous best by up to 4.3%p on LVIS without compromising performance on clean images.

---

Table 1: Segmentation results on LVIS and MSRA using point and box prompts.
LVIS
MSRA
Backbone Method
Point Prompts
Box Prompts
Point Prompts
Box Prompts
Degrade
Clear
Degrade
Clear
Degradc
Clear
Degrade
Clear
IoU
Dice
IoU
Dice
IoU
Dice
IoU
Dice
IoU
Dice
IoU
Dice
IoU
Dice
IoU
Dice
SAM
65.0 76.3 70.1
80.1
75.5
84.5
78.4
86.1
75.9
84,5
79.1
86.6
86.6
92.2
88.7
93.4
HQ-SAM
69.5 80.1
76.0
84.8
79.0
87.1
83.1
89.7
82.8
89.6
86.6
92.0
89.7
94.3
92.4
95.9
AM
64.8
76.1 70.1
80.1
75.4
84.4
78.3
86.0
75.7
84.3
79.1
86.6
86.4
92.1
88.8
93.4
VT--B
URIE+SAM
64.8
76.2
70.0
80.0
74.5
83.7
78.0
85.8
74.6
83.5
77.6
85.6
85.9
91.8
88.8
93.5
RobustSAM
72.7
82.6
77.2
85.8
81.5
89.0
84.3
90.7
86.6
92.3
89.6
94.2
89.5
94.2
92.1
95.7
GaRA-SAM
77.0 85.7
81.3
88.7
83.7
90.5
86.1
91.9
89.4
94.0
91.4
95.2
92.3
95.8
93,9
96.8
SAM
66.2 76.0
75.0
83.0
79.9
87.7
82.8
89.4
77.3
84.6
82.0
88.1
87.6
92.9
88.9
93.5
HQ-SAM
72.6 82.1
79.0
86.7
81.1
88.6
84.7
90.8
85.0
91.0
87.6
92.6
89.4
94.1
91.4
95.2
AM
66.0
75.7
74.8
82.8
79.7
87.6
82.7
89.4
76.9
84.3
81.7
87.8
87.5
92.8
89.0
93.6
VI--
URIE+SAM
66.4
76.3
74.8
83.0
87.4
82.7
89.4
78.1
85.4
83.0
88.9
87.6
92.8
89.6
94.0
RobustSAM
75.6
84.5
80.0
87.5
83.6
90.3
85.8
91.6
87.6
92.9
90.1
94.4
91.6
95.4
93.6
96.5
GaRA-SAM
78.5
86.6
82.6
89.4
84.4
90.8
86.7
92.2
89.6
94.1
91.6
95.3
92.6
96.0
94.0
96.8
Table 2: Zero-shot segmentation results on COCO and STREETS using point and box prompts.
COCO
STREETS+NDD
Backbone Method
Point Prompts
Box Prompts
Point Prompts
Box Prompts
Degrade
Clear
Degrade
Clear
Degrade
Clear
Degrade
Clear
ToU
Dice
ToUl
Dice
ToU
Dice
ToU
Dice
ToU
Dice
ToU
Dice
ToU
Dice
ToU
Dice
SAM
65.2 76.3 69.6
79.6
76.0
84,8
78.9
86.4
74.4
83.5
81.8
89.1
80,2
88.3
85.4
91.8
HQ-SAM
70.2 80.6 76.0
84.8
79.6
87.5
83.5
90.1
75.4
84.5
82.2
89.5
80.8
88.7
86.2
92.3
AM
65.0
76.1 69.5
79.5
75.9
84.7
78.8
86.3
74.3
83.4
81.8
89.1
80.1
88.2
85.4 91.8
VT--B
URIE+SAM
65.0
76.2 69.6
79.6
75.0
84.0
78.2
85.8
74.1
83.4
81.0
88.6
79.8
88.0
84.9 91.5
RobustSAM
72.7
82.6 77.1
85.7
81.6
89.1
84.5
90.9
74.5
84.1
81.0
88.8
81.8
89.4
86.2
92.4
GaRA-SAM
77.2
85.8
81.0
88.4
84,2
90.8
86.4
92.1
77.6
86.4
82.9
90.1
84.1
91.0
87.7
93.3
SAM
66.9 76.5
74.6
82.5
80.4
88.0
82.9
89.4
72.9
81.3
82.1
89.0
81.5
89.2
86.4
92.5
HQ-SAM
73.2 82.5
79.0
86.7
81.7
89.0
85.1
91.1
76.8
85.4
83.9
90.6
81.7
89.3
86.7
92.6
Ai
66.6
76.3
74.4
82.4
80.3
87.9
82.8
89.3
72.5
81.0
81.9
88.
81.4
89.1
86.4
92.5
VT--
URIE+SAM
66.6
76.4
74.3
82,4
79.9
87.7
82.8
89.4
72.6
81.4
81.2
88
81.1
88.9
86.1 92.2
RobustSAM
75.3
84.3
79.9
87.6
83.8
90.5
86.2
91.9
75.7
84.8
82.7
89.9
83.0
90.2
87.5
93.1
GaRA-SAM
78.9
86.9
82.3
89.1
84,9
91.2
86.9
92.4
80.0
88.0
85.3
91.7
85.3
91.7
88.7
93.9
4.3 Zero-shot Segmentation Comparison
To further assess the generalization capability of GaRA-SAM, we evaluate it under both synthetic and real corruptions on datasets not seen during training. Specifically, we include COCO and STREETS+NDD with synthetic degradations, and BDD+LIS, which exhibits real corruptions such as fog, rain, motion blur, and low-light. As reported in Table GaRA-SAM consistently achieves the best across all prompt types and metrics in both COCO and STREETS+NDD Table 3 shows that also outperforms previous work on real corrupted images, improving over RobustS by more than 3.4%p IoU with point prompts ViT-L, despite being trained only on unpaired corrupted images. It demonstrates that GaRA-SAM generalizes well to both synthetic and real-world corruptions, highlighting the effectiveness of its adput-ad design.
4.4 Training on Real Corruption Datasets
GaRA-SAM enables training on real corruption datasets without requiring access clean references. To demonstrate this, we train GaRA- GaRA-SAN solely on the real-world dataset, BDD+LIS, and evaluate its performance on both the seen dataset (BDD+LIS) and an unseen real-world dataset, ACDC. As shown in Table 4 training directly on real corrupted images significantly improves the performance on all benchmarks, outperforming the previous best by up to 21.3%p IoU on ACDC. addition, we compare GaRA-SAN trained on BDD+LIS Robust-Seg dataset (TaRA-SAM-Syn) using box prompts and ViT-I. in Table 3, the results suggest the clear benefit of training directly real corrupted images.
and that trained on the synthetic
7

---

Table 3: Evaluation results real degraded image dataset, BDD+LIS, using point and box prompls.
Table 4: Evaluation results on BDD+LIS
and ACDC, using box prompt with VIT-
L. GaRA-SAM-Syn and -Rcal are trained on
Robust-Seg and BDD+LIS, respectively.
Point Prompts
Box Prompts
Backbone
Method
BDD+LIS
ACDC
IoU
Dice
Dice
Method
IoU
Dice
loU
Dice
SAM
65.2
75.4
74.2
82.7
SAM
74.3
82.0
65.9
72.6
HQ-SAM
67.6
77.9
69.4
78.0
RobustSAM
78.1
86.1
67.5
77.0
AirNet+SAM
64.1
74.5
GaRA-SAM-SyI
80.0
87.4
71.6
80.4
VTT-B
URIE+SAM
65.7
76.1
74.1
82.8
GaRA-SAM-Rea
89.7
93.9
88.8
92.9
RobusISAM
69.5
79.5
75.7
84.1
Table 5: Comparison of our GaRA and LoRA
GaRA-SAM
71.3
80.9
76.8
85.3
on LVIS with ViT-B, using point prompts.
SAM
68.9
77.8
74.3
82.0
Degraded
Clear
HQ-SAM
72.7
81.7
76.3
84.3
Method
IoU
Dice
loU
Dice
AirNe1+SAM
67.2
76.4
73.1
81.3
VIT-L
URIE+SAM
69.7
78.8
74,4
82.1
Rank 16
75.0
84.3
79.4
87.3
LoRA
Rank 128
75.6
84.6
80.6
88.
RobustSAM
71.4
80.9
78.1
Rank 256
73.4
83.1
78.5
86.6
GaRA-SAM
74,8
83.4
80.0
87.4
GaRA-SAM
77.0
85.7
81.3
88.7
HO-SAM
RobustSAM
GORA-SAM (ours)
Synthetic
Real
Figure 4: Results on synthetic (COCO, STREETS+NDD) and real corruption (BDD+LIS) datasets.
4.5 Qualitative results
Figure 4 presents qualitative results under synthetic and real-world corruptions. GaRA-SAM delivers more accurate and complete masks than SAM, HQ-SAM, and RobustSAM, especially under severe degradations: it better preserves object boundaries in synthetic cases and is more reliable in low-light and adverse weather conditions.
4.6 In-depth analysis
We conduct comprehensive ablation study assess the contribution of each design choice in GaRA- SAM. All experiments are performed on the LVIS dataset, using point prompts for consistency.
LoRA vs. GaRA. We first compare GaRA with the standard LoRA with fixed ranks. As shown in T GaRA-SAM consistently outperforms fixed-rank LoRA. While LoRA establishes a strong baseline, its fixed-rank design limits flexibility. In contrast, GaRA-SAM dynamically composes (rank-1) components based on the input, resulting in improved robustness. These results validate the benefit of input-adaptive rank modulation over static alternatives.

---

Table 6: Comparison of GaRA and MoE LoRA.
Table 7: Effect of the rank space separation.
Degraded IoU
Clear
Degraded
Clear
Method
Method
Dice
ToU
Dice
IoU
Dice
ToU
Dice
MоË LoRA (2, 16, 128, 256)
76.4 85.3 79.7 87.5
Rank 16
76.2 85.2 79.9 87.7
w/o Gating
MoE LoRA (2, 16, 128. 1024)
75.4 84.5
78.6 86.7
Rank 256
73.3
82.8
77.3 85.5
GaRA-SAM
77.0 85.7
81.3 88.7
Gating
77.0
85.7
81.3
88.7
80
Table 8: Comparison of computational efficiency.
75
IoU
70
Training
Inference
60
Method
Learnable Params
# GPU
GPU Memory
FPS
0.3
0.5
0.7
0.9
Temperature
SAM
1250M
256
3.36GB
2.9
Ours
SAM
RobustSAM
403M
8
5,41GB
2.8
Figure 5: Effect of Gum- bel temperature.
GaRA-SAM
343M
8
5,39GB
2.6
Active Inactive
H
Clean Reference
Snow, rank: 155
Rain,
Elastic, rank:
z
zz
Clean Reference
Snow, rank: 151
Rain, rank: 11
Elastic, rank: 159
Figure 6: Visualization of the binary gating vectors and ranks under various images under corruptions.
MoE LoRA vs. GaRA. We also investigate an alternative approach to rank selection using mixture- of-experts (MoE) 64 variant of LoRA, where multiple fixed rank LoRA blocks are instantiated, and a shared gating module selects one for each input. As reported in Table [ both MoE variants perform reasonably well, but fall short GaRA-SAM both clean and degraded conditions. This gap not trivial regarding the fewer number of parameters of G-RA-SAM.
Impact of Rank Space Selection. To evaluate the effectiveness of rank space selection (Gating 1), compare GaRA-SAM with a variant that removes the separation between the lower- and higher-rank spaces, using single unified (rank- 1) components with fixed maximum rank (e.g., 16 256). As shown in Table climinating rank space selection results in performance degradation, suggesting that separating the low- and high-rank components allows more effective specialization.
Impact of Temperature in Gumbel-Sigmoid. We investigate the effect of the temperature of the Gumbel-Sigmoid by varying its value. Figure summarizes the performance of variants of GaRA-SAM trained using different values of temperature; the optimal value for 7 is 0.5. The results suggest that GaRA-SAM remains stable across wide range of temperature values, suggesting that GaRA-SAM insensitive to temperature settings.
Computational Cost Analysis. We compare the computational efficiency of GaRA-SAM in terms of learnable parameters, GPU resources, memory consumption, and inference speed (FPS), As summarized in Table g GaRA-SAM achieves the lowest number of learnable parameters and GPUs, owing to its parameter-efficien design. At inference time, requires less GPU memory than RobustSAM, while maintaining comparable FPS both SAM and RobustSAM.
Impact of Maximum Rank Configuration. We demonstrate the effect of our chosen maximum lower-rank and higher-rank configuration. Yable@Jsummard the performance of variants of GaRA- SAM using different maximum rank configurations. The optimal rank combination is (16, 256), achieving the best performance.

---

Table 9: Effect of rank configuration.
Table 10: Effect of gat-
Table 11:
Effect of rank
Table 12: Effect of hierar- chy depth.
ing strategy.
space selection.
Rank Config.
IoU
Gating
Gating 2 loU
Hierarchy Depth
IoU
Gating Strategy
IoU
[8, 256)
76.3
73.4
2-1evel (Ours)
77.0
16, 256]
16. 128)
77.0
76.9
Random
73.9
76.0
3-level
76.4
[16, 512)
76.6
Ours
77.0
77.0
4-level
74.0
Analysis on Gating Vectors. Figure demonstrrats how the gating module responds to different combinations of corruption type and image. For each corrupted imput, present the corresponding binary activation vector z" or z according to their selected rank space, along with the resulting number of active (rank-1) 1) components. We observe that GaRA activates different (rank-1) components depending on the image contents as well as the corruption types. Even when the selected rank is similar, the constituent components often vary, highlighting the input-adaptive and fine-grained gating mechanism of GaRA.
Analysis on Gating in GaRA-SAM. To empirically demonstrate that the gating modules are learned to provide desirable gating paths even with such indirect supervision, we compared GaRA-SAM and its variant with random gating. Table I0 indicates that GaRA-SAM significantly outperforms the random gating variant, supporting the effectiveness our training strategy.
Isolating Effect of Rank Space Selection. To isolate the effect of rank space selection, we ex- perimented with rank space selection alone with fixed rank settings, disabling (rank-1) component selection. Table IT demonstrate that rank space separation alone significantly improves performance. We believe this because it enables each rank space to specialize in handling different degraded inputs, allowing the gating to select the most suitable representational capacity for each. Further- more, enabling (rank-1) component selection on top further boosts the performance, highlighting its complementary benefit.
Impact of Depth of Hierachical Gating. We conducted additional experiments assess deeper hierarchical gating structures (i.e., 3-level and 4-1evel). As shown in Table. [2] increasing the depth introduces optimization challenges due to more complex gating decisions, ultimately degrading performance. Notably, our current 2-Ievel hierarchy yields the best performance and offers the most favorable capacity and trainability.
Conclusion
In this paper, we introduce GaRA-SAM, novel approach for robustifying Segment Anything Model (SAM) under diverse image degradations. Through extensive empirical analysis, we observed that the optimal LoRA rank varies significantly across corruption types and individual inputs, motivating design of Gated-Rank Adaptation (GaRA), a lightweight and aaput-adaptive module that dynamically modulates the effective rank of LoRA adapters. GaRA operates without requiring paired clean and degraded images, enabling fine-grained and parameter-effice inherent zero-shot generalization capabilities. GaRA-SAM achieves state-of-the performance across all robustness benchmarks and, notably, supports training directly on real-world corruption datasets without clean references. This leads to substantial gains in real-world scenarios, highlighting the practical utility and broad applicability of our method.
adaptation while preserving SAM's
Limitations. While GaRA adaptively determines the appropriate rank for each rank space, several design choices, such as the split between lower- and higher-rank spaces, the predefined maximum ranks, and the fixed 2-level gating hierarchy, generalized automated designs are conceptually desirable but practically challenging due to unstable optimization. Developing automated methods remains important future direction.
manually and thus limit flexibility. More
Acknowledgement. This work was supported Samsung Research Funding Incuhation Center of Samsung Electronics (SRFC-IT1801-52) AI Graduate School Program at POSTECH (RS-2019-I119999 and the IITP grants (RS 2044-0045788,, 0---22220000225 funded by Ministry Science and ICT, Korea.
10

---

References
[1] Yuang Ai, Huaibo Huang, and Ran He. Lora-ir: Taming low-rank experts for efficient all-in-one image restoration. arXiv preprint arXiv:2410.15385, 2024.
[2] Ibrahim Alabdulmohsin, Behnam Neyshabur, and Xiaohua Zhai. Revisiting neural scaling laws in language and vision. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https:// openreview
[3] Qi Bi, Shaodi You, and Theo Gevers. Generalized toggy-scene semantic segmentation by frequency decoupling. In Proceedings of the IEEF/CVÉ Conference on Computer Vision and Puttern Recognition (CVPR) Workshops, pages 1389-1399, June 2024.
[4] Huandong Chang, Zicheng Ma, Mingyuan Ma, Zhenting Qi. Andrew Sabot, Hong Jiang, and HT Kung. Elalora: Elastic & learnable low-rank adaptation for efficient model fine-tuning. arXiv preprini arXiv:2504.00254 2025.
[5] Linwei Chen, Ying Fu, Kaixuan Wei, Dezhi Zheng, and Felix Heide. Instance segmentation in the dark. International Journal of Computer Vision, 2023. URL https://github. com/
Linwei-Chan Caaas--henLLSS
[6] Shoufa Chen, Chongjian Ge. Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, and Ping Luo. Adaptformer: Adapting vision transformers for scalable visual recognition. In Proc. Neural Information Processing Systems (NeurIPS), 2022.
[7] Wei-Ting Chen, I-Hsiang Chen, Chih- Yeh, Hao-Hsiang Yang, Hua-En Chang, Jian-Jiun Ding, and Sy-Yen Yen Kuo. Rysl: Robust vehicle similarity learning in real hazy scenes based on sepi-super learning. In Proc. European Conference on Computer Vision (ECCV). Springer, 2022.
[8] Wei-Ting Chen, Yu-Jiet Vong, Sy-Yen Yen Kuo, Sizhou Ma, and Jian Wang. RobustS segment anything robustly degraded images. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 4081-409) 2024.
[9] Ming-Ming Cheng, Niloy J. Mitra, Xiaolei Huang, Philip H. S. Torr, and Shi-Min Hu. Global contrast based salient region detection. IEEE Transactions Pattern Analysis and Machine Intelligence (TPAMI), 2015. URL atttps://mmcheet www.nntttmmsseee mmttmt//////////////
[10] Noel CF Codella, David Gutman, M Emre Celebi, Brian Helba, Michael A Marchetti, Stephen W Dusza, Aadi Kalloo. Konstantinos Liopyris, Nabin Mishra, Harald Kittler, et al. Skin lesion analysis toward melanoma detection: challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic). In 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), 2018.
[11] Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv:2305. 14314, 2023.
[12] Steven Diamond, Vincent Sitzmann, Frank Julca-Aguilar, Stephen Boyd, Gordon Wetzstein, and Felix Heide. Dirty pixels: Towards end-to-end image processing and perception. ACM Transactions on Graphics (TOG), 2021.
[13] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image worth 16x16 words: Transformers for image recognition at scale. In Proc. International Conference on Learning Representations (ICLR), 2021.
[14) William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23 (120):1-39, 2022.
[15] Agrim Gupta, Piotr Dollar, and Ross Girshick. Lvis: A dataset for large vocabulary instance segmentation. In Proc. IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. URL URLhttps: https:////// h/tp::://wwww lvisdataset. .rrg/
11

---

[16] Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg- Kirkpatrikaakkk and Graham Neubig. To- wards unified view of pffcmeter-efficient transfer learning, arXiv preprint arXiv:2110.04366, 2021.
[17] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[18] Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly, Parameter-efficien transfer learning for nip. In Proc. International Conference on Machine Learning (ICML). PMLR, 2019.
[19] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2/06.09666 2021.
[20] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022.
[21] Changho Hwang, Wei Cui, Yifan Xiong. Ziyue Yang. Ze Liu, Han Hu, Zilong Wang. Rafael Salas, Jithin Jose, Prabhat Ram, Yuen Chau, Peng Cheng, Fan Yang, Mao Yang, and Yongqiang Xiong. Tutel: Adaptive mixture-of-ewt of-experts at scale. D. Song, M. Carbin, and T. Chen, editors, Proceedings of Machine Learning and Systems, 2023.
[22] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint 1et.:111111444 2016.
[23] Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In European Conference on Computer Vision (ECCV), 2022.
[24] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 2020.
[25] Lei Ke, Mingqiao Ye, Martin Danelljan, Yifan Liu, Yu- Yu-Wing Tai, Chi-Keung Tang, and Fisher Yu. Segment anything in high quality. In NeurIPS, 2023.
[26] Sungyeon Kim, Boseung Jeong, Donghyun Kim, and Suha Kwak. Efficient and versatile robust fine-tuning of zero-shot models. In European Conference on Computer Vision (ECCV), 2024.
[27] Diederík P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980) 2014.
[28] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4026-4026. 2023.
[29] Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, and Neil Houlsby. Big transfer (bit): General visual representation learning. In Computer Vision- ECCV 2020: 16th European Conference, Glasgow, UK, Augus! 23-28, 2020, Proceedings, Part V 16, pages 491-507. Springer, 2020.
[30] Sohyun Lee, Taeyoung Son, and Suha Kwak. Fifo: Learning fog-invariant features for foggy scene segmentation. In Proc. IEEF/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR), 2022.
[31] Brian Lester, Rami A1-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021.
12

---

[32] Boyun Li, Xiao Liu, Peng Hu, Zhongqin Wu, Jiancheng Lv, and Xi Peng. All-In-One Image Restoration for Unknown Corruption. In IEEE Conference on Computer Vision and Pattern Recognition, New Orleans, LA,, June 2022.
[33] Siyuan Li, Iago Breno Araujo, Wenqi Ren, Zhangyang Wang, Eric K Tokuda, Roberto Hirata Junior, Roberto Cesar-Junior, Jiawan Zhang, Xiaojie Guo, and Xiaochun Cao. Single image deraining: A comprehensive benchmark analysis. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[34] Xiang Li, Tianhan Wei, Yau Pun Chen, Yu-Wing Tai, and Chi-Keung Tang. Fss-1000: A 1000-class dataset for few-shot segmentation. Proceedings of the IEEF/CVE conference on computer vision and pattern recognition, 2020.
[35] Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprini urXiv:2101.000
2021.
[36] Dongze Lian, Daquan Zhou, Jiashi Feng, and Xinchao Wang. Scaling & shifting your features: new baseline for efficient model tuning. Advances in Neural Information Processing Systems, 35:109-123, 2022.
[37] Jun Hao Liew, Scott Cohen, Brian Price, Long Mai, and Jiashi Feng. Deep interactive thin object selection. In Winter Conference on Applications of Computer Vision (WACV), 2021. URL
htt.hb/..cco/iieeeuaaaa///aaaaaa -ttpaa//ttttooootee-t-e---eeeeee selection
[38] Bin Lin, Zhenyu Tang, Yang Ye, Jiaxi Cui, Bin Zhu, Peng Jin, Junwu Zhang, Munan Ning, and Li Yuan. Moe-llava: Mixture of experts for large vision-language models. arXiv preprint arXiv:2401. 15947, 2024.
[39] Cheng Lin, Lujun Li, Dezhi Li, Jie Zou, Wei Xue, and Yike Guo. Nora: Nested low-rank adaptation for efficient fine-tuning large models. arXiv preprint arXiv:2408. 2024.
[40] Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pictro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Proc. European Conference on Computer Vision (ECCV). Springer, 2014. URL http:tt///cc:oeeees
org
[41] Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, 2017.
[42] Jiaming Liu, Senqiao Yang, Peidong Jia, Ming Lu, Yandong Guo, Wei Xue, and Shanghang Zhang. Vida: Homeostatic visual domain adapter for continual test time adaptation. arXiv preprint wxXiv23306.04444 2023.
[43] Ziwei Luo, Fredrik K. Gustafsson, Zheng Zhao. Jens Sjölund, and Thomas B. Schön. Con- trolling vision- language models for multi-task image restoration. In The Twelfth International Conference on Learning Representations, 2024. URL httpps://ppeerrvieeeeee www.eew.weeeeooeeeeeeeeeeeeeeeee net/forum?
id=t3vanLeajU
[44] Xianzheng Ma, Zhixiang Wang, Yacheng Zhan, Yingiang Zheng, Zheng Wang, Dengxin Dai, and Chia-Wen Lin. Both style and fog matter: Cumulative domain adaptation for semantic foggy scene understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11922-19931, 2022.
[45] Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. Adapterfusion: Non-destructive task composition for transfer learning. arXiv preprint arXiv:2005.00247, 2020.
[46] Vaishnav Potlapalli, Syed Waqas Zamir, Salman Khan, and Fahad Khan. PromptIR: Prompting for all-in-one image restoration. In Thirty-seventhh Conference on Neural Information Processing Systems, 2023.
[47] Christos Sakaridis, Dengxin Dai, and Luc Van Gool. ACDC: The adverse conditions dataset with correspondences for semantic driving scene understanding. Proc. IEEF/CVE International Conference Computer Vision (ICCV) 2021. UtL /ccc. vision. ee. we.ethz.ccmm ch/
13

---

[48] Madeline Chantry Schiappa, Shehreen Azad, Sachidanand Vs, Yunhao Ge, Ondrej Miksik, Yogesh S Rawat, and Vibhav Vineet. Robustness analysis on foundational segmentation models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition,
pages 1786-1799 1796, 2024.
[49] Noam Shazeer, * *Azalia Mirhoseini, *Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture- of-experts layer. In International Conference Learning Representations, 2017. URL
https://openrevla htt.:ee..etttttrrmmmm/ccctttaooo
[50] Sheng Shen, Shijia Yang. Tianjun Zhang, Bohan Zhai, Joseph E Gonzalez, Kurt Keutzer, and Trevor Darrell. Multitask vision-language prompt tuning. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pages 5656-5661 2024.
[51] Corey Snyder and Minh Do. Streets: A novel camera network dataset for traffic flow. Advances in Neural Information Processing Systems, 2019. URL https://dattbank, illinois edu/ datasets Ienettt/IIBB-361751551
[52] Taeyoung Son, Juwon Kang, Namyup Kim, Sunghyun Cho, and Suha Kwak. Urie: Universal image enhancement for visual recognition in the wild. In Proc. European Conference on Computer Vision (ECCV), 2020.
[53] Carole H Sudre, Wenqi Li, Tom Vercauteren, Sebastien Ourselin, and M Jorge Cardoso. Gen- eralised dice overlap as deep learning loss function for highly unbalanced segmentations. In Proc. Medical Image Computing and Computer-Assisted Intervention (MICCAI). Springer, 2017.
[54] Mingxing Tan and Quoc Le. EfficientNet: Rethinking model scaling for convolutional neural networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 6105-6114. PMLR, 09-15 Jun 2019. URL httpps:/pproceeddnnt/ mlr
press/ /vt/ tan19a. html
[55] Cameron Trotter, Georgia Atkinson, Matt Sharpe, Kirsten Richardson, A Stephen Mc- Gough, Nick Wright, Ben Burville, and Per Berggren. Ndd20: A large-scale few-shot dol- phin dataset for coarse and fine-grained categorisation. arXiv preprint 2xXi: 2020. URL httpt:tt .nc. ac horolletttoo///ooooooottbrooooot Nor.huumberraadd_oolpp.pii Aitaaot_2000044822422
[56] Mojtaba Valipour, Mehdi Rezagholizadeh, Ivan Kobyzev, and Ali Ghodsi. DyLoRA: Parameter- efficient tuning of pre-trained models using dynamic search-free low-rank adaptation. In Andreas Vlachos and Isabelle Augenstein, editors, Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3274-3287, Dubrovník, Croatia, May 2023. Association for Computational Linguistics. doi: 10. 18653/v/ www.aclllmain.... 239. URL https: http:///aaaaanoooaaaaoo org/2023. eacl-main 239/
[57] Rosaura G Vidal, Sreya Banerjee, Klemen Grm, Vitomir Struc, and Walter Scheirer. Ug" 2: A video benchmark for assessing the impact of image restoration and enhancement automatic visual recognition. In Proc. IEEE Winter Conference on Applications of Computer Vision (WACV), 2018.
[58] Yaqing Wang, Sahaj Agarwal, Subhabrata Mukherjee, Xiaodong Liu, Jing Gao, Ahmed Hassan Awadallah, and Jianfeng Gao. AdaMix: Mixture-of-adaptations for parameter-efficient model tuning. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7744-7760, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguis- tics. doi: 10.18665/w1/2222.com emn emnlp 300-main.3888
388. URL htttp:://aclatttoloooo org/2022 2022
[59] Wenhan Xia, Chengwei Qin, and Elad Hazan. Chain of lora: Efficient fine-tuning of language models via residual learning, arXiv preprint w0Xiv22401.04151, 2024.
14

---

[60] Dongshuo Yin. Leiyi Hu, Bin Li. Youqun Zhang, and Xue Yang. 5%>100%: Breaking performance shackles of full fine-tuning on visual recognition tasks, 2024. URL https
7/arxiv. /arxxv.org//as// wwtp:////gabb/.0.....mmm/ 08345
[61] Fisher Yu, Wenqi Xian, Yingying Chen, Fangchen Liu, Mike Liao, Vashisht Madhavan, Trevor Darrell, et al. Bdd100k: A diverse driving video database with scalable annotation tooling. arXiv preprint arXi: 004677700477 2018.
:/bdd data. berkeley edu/
[62] Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transform- ers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12104-12113, 2022.
[63] Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adaptive budget allocation for sreameter-effcieen fine-tuning. In The Eleventh International Conference on Learning Representations, 2023. URL htttps://operrevie http/forummid-1qquuiis
[64] Yanqi Zhou, Tao Lei, Hanxiao Liu, Nan Du Yanping Huang, Vincent Zhao, Andrew M Dai, Quoe Le, James Laudon, et al. Mixture-of-expert with expert choice routing, Advances in Neural Information Processing Systems, 35:7103-7114, 2022.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction of our paper sufficiently reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer:
[Yes]
Justification:
We discuss the limitations of the work in Section 5
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
16

---

Justification:
The work introduces no theoretical assumptions and results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We disclose all the training and evaluation recipes in Section
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

Answer: [No]
Justification: We do not disclose the code and data at the moment. But we explain the model architecture and experimental settings in detail in Section 4.1
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurIPS code and data submission guidelines (https
wtt.:g/dddeeeeeSSuuurrtr ssionPolicy) for more details. hetpoolliigguddde/oooooee
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We explain the detailed experimental settings in Section 4.1 For the composi- tion of the benchmarks used in the paper, please refer to the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification:
Due to the computational tractability, we conduct each experiment once.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
18

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The number of trainable parameters and GPU memory footprint for inference
are reported Table 8
We also explain the computational environments the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http:///wwurr....pppoop/cc/cctaa
Answer: [Yes]
Justification:
We follow the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
The work poses no societal impacts.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper releases no findings with such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All the datasets in this paper are free from license issues and are cited properly. We also cite the models used in the paper. Please refer to Section 4.1
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the batasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not introduce new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper contains no experiments involving human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This work contains user study.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
21

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification:
We used LLMs for editing purposes only.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy for what should or should not be described.
22

---

Appendix
This appendix presents additional experimental details and results that are omitted from the main paper due to space limit.
A Experimental details
Implementation details. Each GaRA module consists of branch selector, implemented two- layer MLP with hidden dimension 64 and ReLU activation, which outputs a binary gating decision and two rank-wise gating networks (one for low-rank,
(low vs. high rank) using Gumbel-Sigmoll one for high-rank), each composed of three-layer MLP with hidden dimension 16 and ReLU activation, and produces binary masks over rank-1 components via Gumbel-Sigmoid sampling. The selected components modulate separate LoRA paths applied to the QKV projections. Following
RobustSAM &
combined segmentation loss, Laeg Laice 53 + Liocal 41.
Resource. We use 8 A6000 GPUs for training each method and A6000 GPU for evaluation.
Dataset configuration. Following the dataset configuration introduced in RobustSAM 8 we utilize diverse collection of segmentation datasets for training and evaluation. For training. we adopt three synthetic datasets: MSRA 10K [9] (10,000 images) and LVIS (15] (20,252 images) provide large-scale pixel-wise annotations for salient and instance segmentation, while ThinObject-5K [37 (4.748 images) contains fine-gra une-grained masks for thin structures to enhance structural sensitivity, For evaluation, we consider both synthetic and real-world scenarios. The synthetic benchmark includes STREETS [51] (1,000 annotated traffic scenes) and NDD20 [55] (1,000 dolphin segmentation images captured underwater and above-water environments). We exclude FSS- 1000 34 due to licensing restrictions. For real-world evaluation, we adopt BDD-100K [6]], driving video dataset with 100,000 videos (frame-level annotations at the 10th second) covering diverse conditions, and LIS [5], low-light instance segmentation dataset with 2,230 paired short/long exposure images (8,920 total images) containing over 10,000 labeled instances.
B Empirical analysis
B.1 Rank distribution analysis
We analyzed the gated rank distribution across all blocks and corruption types on the MSRAIOF dataset. As shown in Figure al] the activated rank values vary significantly not only across corruption types but also across blocks, indicating that GaRA dynamically adapts its rank selection.
B.2 Corruption type gated rank
To further investigate whether our gating mechanism responds differently depending on the corruption type. we conduct an F-statisties analysis on the number of activated ranks across transformer blocks. In this setting, each corruption type is treated as group, and each sample's 's gated rank serves as an observation. We compute the F-statistic, which measures the ratio between-group within- groupp variance, and the p-value, which indicates whether the observed variation could arise by chance.
As shown in Figure a2] most blocks exhibit statistically significant dependency between corruption types and the number of activated ranks (i.e., high F-statistic and low p-value). Interestingly, blocks 2, 5. 8, and 11 do not show such significance. These blocks use standard attention, whereas the others adopt window attention, according to the original structure of SAM (28). In the window-basee setting, different windows within the same image may capture varied content under the corruption type, leading the shared gating network to attend more to the global corruption style rather than content. This behavior highlights the corruption-awa -wware design of our gating module.
B.3 Justification for rank space separation
The rank space separation is motivated by our empirical observations that, in unified rank space, the model exhibits limited diversity in (rank-1) component activation across different degraded inputs. Specifically, shown in Table al the model tends activate components within narrow
23

---

150_
100.2
125
133.77
dear
110.22
108.68
300.29
134.00
98.07
130.38
108.15
03
124 86
116.77
97.15
115
iiter
109.95
108.88
302.20
115.41
57.75
109.90
108.52
106
57
124,46
315.70
97.24
130.37
compressian
111.35
107.46
115
Mastic transform
110.65
108.48
201.85
114.02
97.63
1C9.91
108.
100
124.84
116
97.72
23
fng
111-78
168.69
$7.03
132.50
00.41
107.08
108.57
1:0
044
126.00
134.88
96.48
136611
brstedd _ glane blu
110.76
50.31
134.33
111
106
125.03
117.25
110
noise_
200.34
117.87
105
metion
210.49
I09.35
38.46
112.46
96.90
111.09
109
106 51
125,48
136 94
96.76
116.54
rain 110.94
109.45
19.16
114.91
97.43
131.69
30
130
123.83
117.20
95.36
114.46
resampling.eee
100
115.02
130.51
107.14
117.03
200m, bur- 110.11 LU8.M
200144
115.4r
97.b3
112.24
1016
125.33
117.20
39.40
110.37
Figure al: Block-wits and corruption-ver average activated ranks on MSRA10K. Each cell repre- sents the mean number of activated rank components for given corruption type and transformer block. The clear variance across both axes demonstrates that GaRA performs input- and corruption- aware rank selection, rather than using fixed configuration.
Table al: Effect rank space separation.
Variant
Rank Space Separation (rank-1) Selection
IoU
# of Active (rank-1) Components
Runge
Unified Rank Space GaRA-SAM
73.3
161.8 + 7.7
133-190 5-181
77.0
85.7 75.8
range (133-190; std: 7.7), limiting its ability to adjust representational capacity according each degraded input. This is potentially suboptimal because different degraded inputs require different representational capacities. By separating the rank space into lower-rank and higher-rank regions, we allow the model to activate a more diverse range of components (5-181; std: 75.8), yielding significant performance improvement. We guess this is because the rank space separation into two exclusive sets, each specialized for different degraded inputs demanding different representation capacities, helps mitigate potential conflicts during (rank-1) component selection and promotes more diverse usage of components.
C Additional experimental results
C.1
Generalization ability beyond driving scenarios
To further evaluate the generalization capability of GaRA-SAM beyond driving scenarios, additionally tested on the ISIC-2016 medical segmentation dataset [IO]. As shown Table a2 using box prompts, GaRA-SAM showed notable improvements over SAM. These results indicate that GaRA-SAM maintains strong performance even domains with very different visual characteristics.
C.2 Isolating the effect of dynamic (rank-1) component selection
In Table a3 we tested variants using only dynamic (rank-1) component selection within unified rank space. Enabling the dynamic selection improves performance when using unified rank space with the small maximum rank (i.c., 16), highlighting the benefit of adaptively choosing components

---

16
F-stanistics pa
14
including evaluations using the p 0.055 threaho
12
30
10
4op 1010 (0lovvluu))
F-tatitie
20
6-
4
0
Block
Figure a2: Statistical analysis of gated rank selection under different corruption types on MSRA 10K. For each block, we compute the F-statistic and p-value (shown as (p)) higher F-statistic and p-value below 0.05 (dashed line) indicate that the variation in activated ranks across corruption types is statistically significant.
Table a2: Generalization performance the ISIC-2016 medical segmentation dataset.
Model
loU
Dice
AP
SAM
70.6
80.7
76.8
GaRA-SAM
78.7
86.2
81.7
input. However, using unified rank space with the large maximum rank (i.e., 256) did not lead better results. We attribute this to the limited diversity in active components when using a unified space, which can restrict the ability to adjust representational capacities according to each degraded input, thus hindering input-adaptive modulation. motivated our design to split the rank space, which led to improved performance.
C.3 Further finetuning GaRA-SAM real-world dataset
GaRA-SAM supports training on real-world corrupted datasets without requiring clean counterparts. To validate this capability, we further fine-tune GaRA-SAM on real-world degradation datasets, namely BDD-100K and LIS. In contrast, RobustSAM is not designed for unpaired training and cannot be directly applied this setting. Nevertheless, for a fair comparison, we fine-tune RobustS on the same datasets using only segmentation loss, despite the lack of paired supervision. We evaluate both models on BDD+LIS (seen during fine-tuning) and ACDC, which serves as a real-world dataset not seen during fine-tuning. As shown in Table a4 this additional finetuning leads significant performance improvements across all benchmarks. This result highlights that GaRA-SAM not only generalizes well in zero-shot setting, but also scales effectively with real-world corrupted inputs, whereas RobustSAM lacks inherent support for this setting and only included for comparative purposes.
C.4 Quantitative results
We report additional experimental results in Tables pixel accuracy (PA) metric. values are consistently reported across all benchmark settings using both point and box prompts.
a5
a6
7
a8
a9
25

---

Table a3: Ablation on dynamic (rank-1) component selection and rank-space gating.
Variant
(rank-1) Component Selection
Rank space gating
IoU
w/o (rank-1) component selection (max rank 16)
75.0
w/ (rank-1] I) component selection (max rank 16)
76.2
w/o (rank-1) I) component selection (max rank 256)
73.4
w/ (rank-1) I) component selection (max rank 256)
73.3
GaRA-SAM (max rank 256)
77.0
Table a4: Fine-tuning results with ViT-L. Models are further fine-tuned on BDD+LIS and evaluated on BDD+LIS and ACDC using box prompt.
BDD+LIS
ACDC
Method
Fine-tuning
IoU
Dice
IoU
Dice
SAM
74.3
82.0
65.9
72.6
RobustSAM
78.1
86.1
67.5
77.0
RobustSAM
85.9
91.5
83.9
89.5
GaRA-SAM
80.0
87.4
71.6
80.4
GaRA-SAM
v
89.5
93.8
88.3
92.6
C.5
Qualitative results
We present additional qualitative comparisons in Fig. a3 showing the segmentation results of SAM, HQ-SAM, RobustSAM, and our GaRA-SAM under variety of degraded inputs. The top section illustrates results under synthetic corruptions (e.g., motion blur, snow, noise), while the bottom section includes real-world challenges such as low-light and adverse weather conditions. Compared to other baselines, GaRA-SAM consistently demonstrates more robust and accurate segmentation masks, especially in cases where the standard SAM or HQ-SAM fails to localize object boundaries. These results highlight GaRA-SAM's strong generalization capabilities across both synthetic and naturally occurring corruptions.
26

---

Table a5: Additional evaluation results on LVIS including pixel accuracy (PA) using point and box
prompts.
Point Prompts
Box Prompts
Backbone Method
Degrade
Clear
Degrade
Clear
loU
PA
Dice
IoU
Dice
IoU
PA
Dice
IoU
PA
Dice
SAM
65.0
89.8
76.3
70.1
91.5
80.1
75.6
93.5
84.5
78.4
93.7
86.1
HQ-SAM
69.5
91.8
80.1
76.0
93.3
84.8
79.0
94.8
87.1
83.1
95.3
89.7
AiM
64.8
89.8
76.1
70.1
91.5
80.1
75.4
93.4
84.4
78.3
93.6
86.0
VIT-B
URIE+SAM
64.8
89.9
76.2
70.0
91.4
80.0
74.5
93.2
83.7
77.9
93.5
85.8
RobustSAM
72.7
93.0
82.6
77.2
93.9
85.8
81.5
95.0
89.0
84.3
95.5
90.8
GaRA-SAM
77.0
94.4
85.7
81.3
95.3
88.7
83.7
96.0
90.5
86.1
96.3
91.9
SAM
66.2
87.1
76.0
75.0
91.5
83.0
79.9
94.8
87.7
82.8
95.1
89.4
HQ-SAM
72.6
92.5
82.1
79.0
94.1
86.7
81.1
95.4
88.6
84.7
95.8
90.8
SiM
66.0
86.9
75.7
74.8
91.4
82.8
79.7
94.8
87.6
82.7
95.0
89.4
VIT-L
URIE+SAM
66.4
87.6
76.3
74.8
91.5
83.0
79.4
94.7
87.4
82.7
95.0
89.4
RobustSAM
75.6
94.1
84.5
80.0
95.1
87.5
83.6
96.0
90.3
85.8
96.3
91.6
GaRA-SAM
78.5
94.8
86.6
82.6
95.6
89.4
84.4
96.0
90.8
86.7
96.5
92.2
Table a6: Evaluation results MSRA using point and box prompts.
Point Prompts
Box Prompts
Backbone
Method
Degrade
Clear
Degrade
Clear
loU
PA
Dice
loU
PA
Dice
IoU
PA
Dice
IoU
PA
Dice
SAM
75.9
93.5
84.5
79.1
94.8
86.6
86.6
97.0
92.2
88.8
97.5
93.4
HQ-SAM
82.8
95.7
89.6
86.6
96.7
92.0
89.7
97.7
94.3
92.5
98.3
95.9
AinNet+SAM
75.7
93.5
84.3
79.1
94.8
86.6
86.4
96.9
92.1
88.8
97.5
93.5
VIT-B
URIE+SAM
74.6
93.2
83.5
77.6
94.5
85.6
86.0
96.8
91.8
88.8
97.5
93.5
RobustSAM
86.6
96.6
92.3
89.6
97.4
94.2
89.5
97.6
94.2
92.1
98.2
95.7
GaRA-SAM
89.4
97.5
94.0
91.4
98.0
95.2
92.3
98.3
95.8
93.9
98.7
96.8
SAM
77.3
89.0
84.6
82.0
92.1
88.1
87.6
97.2
92.9
88.9
97.5
93.5
HQ-SAM
85.0
95.9
91.0
87.6
96.6
92.6
89.4
97.7
94.1
91.4
98.1
95.2
AinNet+SAM
76.9
88.7
84.3
81.7
91.8
87.8
87.5
97.2
92.8
89.0
97.5
93.6
VIT-L
URIE+SAM
78.1
90.0
85.4
83.0
93.0
88.9
87.6
97.2
92.8
89.6
97.7
94.0
RobustSAM
87.6
96.8
92.9
90.1
97.4
94.4
91.6
98.1
95.4
93.6
98.6
96.5
GaRA-SAM
89.6
97.5
94.1
91.6
98.0
95.3
92.6
98.4
96.0
94.0
98.7
96.8
27

---

Table a7: Evaluation results on COCO using point and box prompts.
Point Prompts
Box Prompts
Enc.
Method
Degrade
Clear
Degrade
Clear
IoU
PA
Dice
IoU
PA
Dice
IoU
PA
Dice
IoU
PA
Dice
SAM
65.2
90.2
76.3 69.6
91.7
79.6
76.0
93.6
84.8
78.9
93,9
86.4
HQ-SAM
70.2
92.2
80.6
76.0
93.6
84.8
79.6
95.1
87.5
83.6
95.6
90.1
SiM
65.0
90.1
76.1
69.5
91.7
79.5
75.9
93.6
84.7
78.8
93.9
86.3
ViT-B
URIE+SAM
65.0
90.1
76.2
69.6
91.6
79.6
75.0
93.4
84.0
78.2
93.7
85.8
RobustSAM
72.7
93.0
82.6
77.1
93.9
85.7
81.6
95.2
89.1
84.5
95.8
90.9
GaRA-SAM
77.2
94.5
85.8
81.0
95.3
88.4
84.2
96.3
90.8
86.4
96.5
92.1
SAM
66.9
87.8
76.5
74.6
91.4
82.6
80.4
95.1
88.0
82.9
95.3
89.4
HQ-SAM
73.2
92.8
82.5
79.0
94.3
86.7
81.7
95.7
89.0
85.1
96.1
91.1
AirNet+SAM
66.6
87.6
76.3
74.4
91.3
82.5
80.3
95.1
87.9
82.8
95.3
89.3
ViT-L
URIE+SAM
66.6
87.8
76.4
74.3
91.4
82.4
79.9
95.0
87.7
82.8
95.3
89.4
RobustSAM
75.3
94.0
84.3
79.9
95.2
87.6
83.8
96.1
90.5
86.2
96.5
91.9
GaRA-SAM
78.9
94.8
86.9
82.3
95.5
89.1
84.9
96.3
91.2
86.9
96.7
92.4
Table a8: Evaluation results on STREETS+NDD using point and box prompts.
Point Prompts
Box Prompts
Enc.
Method
Degrade
Clear
Degrade
Clear
IoU
PA
Dice
loU
PA
Dice
loU
PA
Dice
IoU
PA
Dice
SAM
74.4
98.9
83.5
81.8
99.5
89.1
80.2
99.7
88.3
85.4
99.8
91.8
HQ-SAM
75.4
99.4
84.5
82.2
99.6
89.5
80.8
99.7
88.7
86.2
99.8
92.3
AirNet+SAM
74.3
98.9
83.4
81.8
99.5
89.1
80.1
99.7
88.2
85.4
99.8
91.8
VIT-B
URIE+SAM
74.1
99.1
83.4
81.0
99.5
88.6
79.8
99.7
88.0
84.9
99.7
91.5
RobustSAM
74.5
99.4
84.1
81.0
99.5
88.8
81.8
99.7
89,4
86.2
99.8
92.4
GaRA-SAM
77.6
99.5
86.4
82.9
99.7
90.1
84.1
99.8
91.0
87.7
99.8
93.3
SAM
72.9
96.7
81.3
82.1
98.8
89.0
81.5
99.7
89.2
86.4
99.8
92.5
HQ-SAM
76.8
99.3
85.4
83.9
99.6
90.6
81.7
99.7
89.3
86.7
99.8
92.6
AirNet+SAM
72.5
96.6
81.0
81.9
98.7
88.7
81.4
99.7
89.1
86.4
99.8
92.5
VIT-L
URIE+SAM
72.6
97.3
81.4
81.2
98.9
88.4
81.1
99.7
88.9
86.1
99.8
92.2
RobustSAM
75.7
99.4
84.8
82.7
99.6
89.9
83.0
99.7
90.2
87.5
99.8
93.1
GaRA-SAM
80.0
99.6
88.0
85.3
99.7
91.7
85.3
99.8
91.7
88.7
99.8
93.9
28

---

Table a9: Evaluation results on BDD+LIS using point and box prompts.
Point Prompts
Box Prompts
Enc.
Method
IoU
PA
Dice
IoU
PA
Dice
SAM
65.2
87.8
75.4
74.2
92.3
82.7
HQ-SAM
67.6
90.0
77.9
69.4
91.7
78.0
AiM
64.1
87.0
74.5
73.4
92.1
82.1
ViT-B
URIE+SAM
65.7
89.0
76.1
74.1
92.5
82.8
RobustSAM
69.5
91.1
79.5
75.7
92.4
84.1
GaRA-SAM
71.3
91.0
80.9
76.8
91.8
85.3
SAM
68.9
86.4
77.8
74.3
91.9
82.0
HQ-SAM
72.7
91.6
81.7
76.3
93.4
84.3
AirNet+SAM
67.2
84.8
76.4
73.1
91.6
81.3
ViT-L
URIE+SAM
69.7
87.8
78.8
74.4
92.0
82.1
RobustSAM
71.4
91.1
80.9
78.1
92.5
86.1
GaRA-SAM
74.8
92.2
83.4
80.0
93.5
87.4
29

---

Input
SAM
HQ-SAM
RAM
GaRA-SAN (Ours)
GT
Synthetic
Real
Figure a3: Qualitative results under both synthetic (top) and real-world (bottom) corruptions. From left to right: input image. SAM, HQ-SAM, RobustSAM. our method GaRA-SAM. and the ground truth (GT). GaRA-SAM consistently produces more accurate and complete segmentations, particularly under challenging conditions such as noise, blur, and low-light scenarios.
30


Reviews:
[{"review_id": "cOnDk95vpJ", "reviewer": "Reviewer_feAp", "summary": "The paper \"GaRA-SAM: Robustifying Segment Anything Model with Gated-Rank Adaptation\" proposes GaRA to enhance SAM's robustness against input degradations. GaRA introduces lightweight adapters into SAM's intermediate layers, dynamically adjusting the effective rank of weight matrices based on input via a learned gating module. GaRA-SAM achieves state-of-the-art performance on robust segmentation benchmarks, significantly surpassing previous methods.", "questions": "1.GaRA needs an extra 60% GPU Memory (from 3.36 GB of original SAM to 5.39GB of GaRA)， and 343M extra LoRA params, which may be a heavy load for resource-constrained devices. I wonder if GaRA will outperform a bigger SAM with 1250M+343M params co-trained by clean pairs and degraded pairs, although I understand that this strategy holds significant practical value due to its flexibility.\n2. How can we supervise the training process to ensure GaRA chooses the best gating path?\n3. Impact of Rank Space Selection ablation in the Table.7, why choose rank 16 and 256 only? It is hoped that more experimental results on w/o gating ranks can be provided to demonstrate the necessity of space selection fully. Similarly, why are only these two levels selected for hierarchical gating, and why not go deeper into more levels?", "limitations": "The article mentions the potential drawbacks of the maximum rank number", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "1.Strengths：\n(1) Innovative methodology: The introduction of GaRA, which dynamically adjusts the rank of adapters based on input, is a novel approach to enhancing model robustness. This method effectively addresses the limitations of fixed-rank LoRA and other existing robustification techniques, like attaching an existing image restoration module to the front of SAM.\n(2) Sota Results: GaRA-SAM achieves significant performance improvements on multiple robust segmentation benchmarks, particularly on real-world corrupted datasets like ACDC. This demonstrates its practical effectiveness and superiority over previous methods.\n(3) Practicality and Efficiency: GaRA-SAM maintains parameter efficiency and adheres to standard training protocols, making it easy to integrate into existing workflows. It also enables training on real-world degraded images without requiring clean references, which is a major advantage for practical applications.\n(4) GaRA is easy to understand and follow.\n\n2.Weaknesses: \n(1) GaRA, as a robust image encoder, while achieving good performance, needs an extra 60% GPU Memory (from 3.36 GB of original SAM to 5.39GB of GaRA), which may be a heavy load for resource-constrained devices.", "was_revised": true, "final_justification": "Considering the novelty and conciseness of the GaRA-SAM method, along with the relatively comprehensive experimental validation, its excellent performance, and the additional key experiments provided in the subsequent rebuttal, it is evident that GaRA is a high-quality article. Therefore, I am inclined to accept this paper for formal publication.", "rebuttal": "We appreciate your insightful feedback and constructive suggestions, which help improve our paper substantially. We will address all the comments and include additional experiments in the revision. Please find our responses to the comments below. All ablation studies were conducted using the ViT-B backbone and evaluated on LVIS with point prompts.\n___\n### **Weakness 1 & Question 1-1. Memory overhead of GaRA-SAM**\n\n\nThank you for highlighting this important point. Memory efficiency is indeed an important direction for future research, particularly for deployment on edge devices. Although GaRA-SAM introduces 2 GB of additional memory for inference, we note that its total memory usage (5.39 GB) is comparable to RobustSAM (5.41 GB). Additionally, this footprint remains within the capability of widely used modern edge devices such as Jetson Xavier NX and AGX Orin, making GaRA-SAM practical for real-world applications.\n___\n### **Question 1-2. Comparison with a larger SAM**\n\nThank you for the constructive suggestion. To assess this, we designed a larger variant of SAM by inserting two additional 4-layer MLP blocks immediately after the image encoder and before the mask decoder, matching GaRA-SAM’s parameter count. Despite its increased capacity and full fine-tuning, this larger model showed inferior performance compared to GaRA-SAM. We attribute this to the loss of the strong generalization capability of the pretrained SAM due to the full fine-tuning. On the other hand, GaRA-SAM effectively preserves the capability by learning the lightweight, input-adaptive modules only while freezing the pretrained SAM.\n| Model               | IoU |\n|--------------------|:------:|\n| Fine-tuned Larger SAM | 72.9 |\n| GaRA-SAM           | **77.0** |\n\nThis comparative analysis will be included in the revision.\n___\n### **Question 2. How is the gating path supervised?**\n\nThe gating modules of GaRA-SAM are not directly supervised, but they are trained indirectly through the segmentation loss alone, following established practices in prior work [1,2]. In other words, no direct supervision is provided to explicitly guide which gating paths the modules should take.\nTo empirically demonstrate that the gating modules are learned to provide desirable gating paths even with such indirect supervision, we compared GaRA-SAM and its variant with random gating. The results reported below indicate that GaRA-SAM significantly outperforms the random gating variant, supporting the effectiveness of our training strategy.\n\n| Gating Strategy     | IoU |\n|---------------------|:------:|\n| GaRA-SAM w/ random gating    | 73.9 |\n| GaRA-SAM            | **77.0** |\n\nThese findings will be elaborated on in the revised manuscript.\n\n[1] Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer, ICLR 2017\n\n[2] Categorical Reparameterization with Gumbel-Softmax, ICLR 2017\n___\n### **Question 3-1. Why are ranks 16 and 256 used?**\n\n\nWe conducted comprehensive ablation studies on various rank configurations. As summarized below, the combination of {16, 256} consistently achieves the best performance.\n| Rank Configuration | IoU |\n|--------------------|:------:|\n| {8, 256}           | 76.3 |\n| {16, 128}          | 76.9 |\n| {16, 256} (Ours)          | **77.0** |\n| {16, 512}          | 76.6 |\n| {16, 1024}         | 76.0 |\n\nThank you for the comment. We will include these results in the revised manuscript.\n\n___\n### **Question 3-2. Isolating the effect of rank space selection**\n\nThank you for the valuable suggestion. To isolate the effect of rank space selection, we experimented with rank space selection alone with fixed rank settings (i.e., rank 16 or rank 128), disabling (rank-1) component selection. The results demonstrate that rank space separation alone significantly improves performance. We believe this is because it enables each rank space to specialize in handling different degraded inputs, allowing the gating to select the most suitable representational capacity for each. Furthermore, enabling (rank-1) component selection on top further boosts the performance, highlighting its complementary benefit.\n| Variant                            | Rank Space Selection | (rank-1) Component Selection | IoU |\n|----------------------------------------------------------|:----------------:|:---------------------------:|:------:|\n| w/o Rank Space Selection                                 |                  |                             | 73.4 |\n| w/ Rank Space Selection                             |        ✔         |                             | 76.0 |\n| w/ Rank Space Selection + (rank-1) Component Selection (GaRA-SAM)  |     ✔     |      ✔      | **77.0** |\n\nThese results will be included in the revision.\n___\n### **Question 3-3. Impact of the depth of hierarchical gating**\n\nThank you for the suggestion. We conducted additional experiments to assess deeper hierarchical gating structures (i.e., 3-level and 4-level). Our findings reveal that increasing the depth introduces optimization challenges due to more complex gating decisions, ultimately degrading performance. Hence, we conclude that the proposed 2-level hierarchy offers both strong performance and architectural simplicity.\n| Hierarchy Depth | IoU |\n|-----------------|:------:|\n| 2-level (Ours)  | **77.0** |\n| 3-level         | 76.4 |\n| 4-level         | 74.0 |"}, {"review_id": "7Rkyb4GKYo", "reviewer": "Reviewer_uBVp", "summary": "The paper proposes to use gated-rank adaptation to robustify SAM (segment anything model) against image corruptions, while maintaining the inherent generalization capabilities of such models. The core technique proposed is to freeze the parameters of a pre-trained SAM model and to introduce gated-rank adaptation modules that learn to adjust the effective rank of its weight matrix based on the current input. The authors show that this technique maintains the generalization capabilities of the model while significantly improving robustness to several synthetic and real-world image corruptions — especially when trained directly on real-world corrupted data.", "questions": "- Why do you use two GaRA gates for high- and low-dimensional spaces — instead of a single one that can freely adapt any rank?\n- L217: why r_L = 16 and r_H = 256? How were this values determined?\n- L222: can you please clarify what you mean by “seen datasets”? I assume this means performance of a held-out test set from the same data distribution as the training data — as opposed to testing on a distribution from a different dataset?\n- Table 8: Can you please provide more details on the performance evaluation? What types of GPUs do you refer to? How many are used for inference? How many fixed (I.e. non-learnable) parameters are there? And how much training time is required for fine-tuning with GaRA-SAM vs. RobustSAM in total (in e.g. GPU-hours)?", "limitations": "The paper does not properly address its limitations. L294 has two sentences on it, but only raises a single point — the manual engineering of the GaRAs. It would be helpful to go into more details on why the authors chose this design compared to more general and flexible options. Additionally, other limitations should be addressed such as no comparison to training a SAM model with corrupted data from scratch, only computing single runs without std or statistical significance values, etc.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n- Robustifying SAM models is crucial for real-world applications, as unexpected shifts in the data distribution due to image corruptions from manifold sources can significantly degrade segmentation performance. For example, for self-driving vehicles, adverse weather conditions, low light, sensor degradation, dirt, or lighting changes can significantly impact segmentation performance. The paper addresses this crucial problem.\n- The proposed solution is lightweight, allowing to reuse existing pre-trained SAM models. This saves significant cost and allows reusing existing models without retraining from scratch.\n- The proposed method can be trained on real-world corrupted data without paired clean examples. Compared to prior work, this allows avoiding the synthetic-to-real domain shift, leading to significantly improved robustness.\n- The paper is well presented and follows a logical structure. It includes ablations and experimental evidence for assumptions and intuitions presented.\n\n**Weaknesses**\n- A discussion of the reasons for this particular design of the GaRA modules would improve the paper. Why was this split between low and high rank adapters chosen? Why not just one of them? Or a more general approach that allows the gate to choose from any dimension? While the technical implementation is clear from section 3.3, the reasoning, as well as the advantages and drawbacks remain elusive.\n- The paper does not properly address its limitations. L294 has two sentences on it, but only raises a single point — the manual engineering of the GaRAs. It would be helpful to go into more details on why the authors chose this design compared to more general and flexible options. Additionally, other limitations should be addressed such as no comparison to training a SAM model with corrupted data from scratch, only computing single runs without std or statistical significance values, etc.\n- Some of the tables and figures are very crammed, which makes reading them difficult. For example, Table 6, 7, and 8, and Figures 5 and 6 barely have any separation.\n- The authors provide no code to evaluate the implementation or build upon their work.", "was_revised": true, "final_justification": "The authors addressed most of my concerns. The only remaining point is that the method's limitations could be addressed more thoroughly. However, this is a minor concern, outweighed by the method's good performance and evaluation. The other reviewers share my positive evaluation and do not raise any additional concerns. I therefore keep my recommendation to accept.", "rebuttal": "We appreciate your insightful feedback and constructive suggestions, which help improve our paper substantially. We will address all the comments and include additional experiments in the revision. Please find our responses to the comments below. All ablation studies were conducted using the ViT-B backbone and evaluated on LVIS with point prompts.\n___\n\n### **Weakness 1 & Question 1. Justification for rank space separation**\n\nWe sincerely appreciate this insightful comment. The rank space separation is motivated by our empirical observations that, in a unified rank space, the model exhibits limited diversity in (rank-1) component activation across different degraded inputs. Specifically, the model tends to activate components within a narrow range (133-190; std: 7.7), limiting its ability to adjust representational capacity according to each degraded input. This is potentially suboptimal because different degraded inputs require different representational capacities (Figure 2(b) of the paper).\n\nBy separating the rank space into lower-rank and higher-rank regions, we allow the model to activate a more diverse range of components (5-181; std: 75.8), yielding significant performance improvement. \n| Variant                         | Rank Space Separation | (rank-1) Component Selection | IoU  | # of Active (rank-1) Components (Mean ± Std) | Range     |\n|----------------------------------|:----------------------:|:-----------------------------:|:----:|:------------------------------------:|:---------:|\n| Unified Rank Space              |                        | ✔                             | 73.3 | 161.8 ± 7.7                         | 133 ~ 190 |\n| Separated Rank Space (GaRA-SAM) | ✔                      | ✔                             | 77.0 | 85.7 ± 75.8                         | 5 ~ 181   |\n\n\nWe guess this is because the rank space separation into two exclusive sets, each specialized for different degraded inputs demanding different representation capacities (Figure 2(b) of the paper), helps mitigate potential conflicts during (rank-1) component selection and promotes more diverse usage of components.\n\nWe will clarify this motivation and supporting evidence in the revised manuscript.\n___\n\n### **Weakness 2-1. Expanded discussion on limitations**\n\nWe sincerely appreciate the thoughtful comment. We will expand our limitations section, clearly detailing the manual design choices and their potential constraints. We plan to revise the section as follows:\n\n> **Limitations.** While GaRA adaptively determines the appropriate rank for each rank space, several design choices, such as the split between lower- and higher-rank spaces, the predefined maximum ranks, and the fixed 2-level gating hierarchy, are set manually and thus limit flexibility. More generalized automated designs are conceptually desirable but practically challenging due to unstable optimization. Developing automated methods remains an important future direction.\n___\n\n### **Weakness 2-2. Addressing experimental limitations**\n\nThank you for raising these important concerns. We conducted additional experiments to address the experimental limitations.\n1. Training SAM from scratch: We trained SAM from scratch using corrupted data, which resulted in significantly lower performance compared to GaRA-SAM. This performance gap highlights the importance of leveraging the strong generalization ability of SAM, which originates from large-scale pretraining. We will include these results in the revised manuscript.\n| Model                    | IoU |\n|-------------------------|:------------:|\n| SAM trained from scratch   | 57.3 |\n| GaRA-SAM            | **77.0** |\n2. Statistical significance of results: We repeated the main experiments three times with different random seeds using the ViT-B backbone and evaluated the models using point prompts. As shown in the table below, our method consistently achieves strong performance with low standard deviation across four datasets. We will revise all tables to reflect the statistical results.\n| Dataset | Degraded IoU | Degraded PA | Degraded Dice | Clear IoU | Clear PA | Clear Dice |\n|---------|:--------------:|:-------------:|:----------------:|:------------:|:-----------:|:-------------:|\n| LVIS    | 76.7 ± 0.3   | 94.2 ± 0.2  | 85.5 ± 0.2     | 82.0 ± 0.8 | 95.4 ± 0.1| 89.1 ± 0.6  |\n| MSRA    | 89.3 ± 0.2   | 97.4 ± 0.1  | 93.9 ± 0.2     | 91.3 ± 0.1 | 97.9 ± 0.1| 95.1 ± 0.1  |\n| COCO    | 76.9 ± 0.5   | 94.4 ± 0.2  | 85.6 ± 0.3     | 80.5 ± 1.0 | 95.2 ± 0.3| 88.0 ± 0.7  |\n| SN      | 77.3 ± 0.5   | 99.5 ± 0.0  | 86.2 ± 0.4     | 82.6 ± 0.4 | 99.7 ± 0.0| 89.9 ± 0.3  |\n\n___\n\n### **Weakness 3. Figure and table readability**\n\n\nThank you for your helpful comment. We will improve readability in the revision by adjusting figure spacing, table layouts, and font sizes.\n___\n\n### **Weakness 4. Code availability**\n\nWe fully agree that open-sourcing is essential for reproducibility and future research. We will release the full implementation, including model weights and training/evaluation scripts.\n\n___\n### **Question 2. Why $r_L = 16$ and $r_H = 256$?**\n\n\nThese values were selected based on ablation studies across various rank configurations. The combination of {16, 256} consistently shows the best performance:\n| Rank Configuration | IoU |\n|--------------------|:------:|\n| {8, 256}           | 76.3 |\n| {16, 128}          | 76.9 |\n| {16, 256} (Ours)          | **77.0** |\n| {16, 512}          | 76.6 |\n| {16, 1024}         | 76.0 |\n\nWe will include these results in the revised version. \n\n___\n### **Question 3. Clarification on “seen datasets”**\n\n\nThank you for pointing this out. The term “seen datasets“ refers to test splits from the same distributions as training data. We will clarify this explanation in the revision.\n___\n\n\n### **Question 4. Details on performance evaluation**\n\n\nAll experiments were conducted using four NVIDIA RTX A6000 GPUs for training and one A6000 GPU for inference. The number of non-learnable parameters was consistent across methods and amounted to 1,250M parameters. Regarding training time, GaRA-SAM required 25 hours, while RobustSAM required 33 hours under identical settings."}, {"review_id": "KIQ6LtA3hR", "reviewer": "Reviewer_N9R9", "summary": "The paper addresses the robustness problem of the Segment Anything Model (SAM) under various image degradations. The authors propose Gated-Rank Adaptation (GaRA), an adaptive and lightweight modification based on Low-Rank Adaptation (LoRA). GaRA dynamically adjusts the effective rank of the adapters integrated into the frozen layers of SAM according to the specific degradation type present in the input image. This gating mechanism adaptively selects between lower-rank and higher-rank components, enabling fine-grained, input-aware robustification without requiring paired clean-degraded training images. Experimental evaluations demonstrate state-of-the-art results on multiple robust segmentation benchmarks.", "questions": "1. Would fine-tuning the query, key, and value parameters directly with a low learning rate or replacing LoRA with traditional adapters [a] also enhance the model’s robustness similarly?\n\n> [a] Houlsby et al. *Parameter-efficient transfer learning for NLP*. ICML 2019.\n\n2. If LoRA were applied to fine-tune the decoder instead of the encoder, would this also result in improved robustness?\n\n3. Can GaRA be merged into the SAM weights to reduce model size further? The paper mentions (line 100) that GaRA selects and combines appropriate (rank-1) components dynamically during inference, suggesting it may not be mergeable into the original SAM model weights like other static LoRA methods.\n\n4. Will the code and trained models be fully open-sourced?", "limitations": "See Weaknesses and Questions", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n\n1. The writing is clear and the presentation is well-structured.\n\n2. The proposed method is intuitive and straightforward.\n\n3. Experimental evaluation is comprehensive, and the method surpasses baseline approaches across multiple datasets.\n\n**Weaknesses:**\n\n1. My primary concern relates to methodological innovation. The proposed GaRA method employs a gating mechanism to dynamically activate low-rank or high-rank LoRA adapters. However, similar adaptive rank-selection concepts have already been extensively explored in related work such as NoRA, AdaLoRA, and ElaLoRA, raising questions regarding the novelty of this approach.\n\n2. The sequence of tables is somewhat confusing: Table 2 is presented first in Section 4.2, while Table 1 is referenced later in Section 4.3. This inconsistency affects readability and clarity.\n\n3. There seems to be an issue with the third row of Figure 4, as the input image appears missing or unclear. Clarification or correction is needed.", "was_revised": true, "final_justification": "All of my concerns have been addressed during the rebuttal period. Considering the paper’s comprehensive experimental evaluation, I am assigning a final score of 4 Borderline accept.", "rebuttal": "We appreciate your insightful feedback and constructive suggestions, which help improve our paper substantially. We will address all the comments and include additional experiments in the revision. Please find our responses to the comments below. All ablation studies were conducted using the ViT-B backbone and evaluated on LVIS with point prompts.\n___\n\n### **Weakness 1. Methodological innovation**\n\nThank you for the comment. We would like to clarify that GaRA introduces a key difference from existing methods such as NoRA [1], AdaLoRA [2], and ElaLoRA [3] by enabling input-adaptive rank modulation at inference time.\n\nSpecifically, NoRA employs a nested LoRA structure to improve parameter efficiency but uses a fixed rank throughout both training and inference. AdaLoRA and ElaLoRA adjust ranks during training to optimize parameter-budget allocation, but their rank configurations remain fixed at inference time and do not adapt according to the input.\n\nIn contrast, GaRA-SAM is motivated by the observation in Figure 2(b) of the paper that optimal LoRA ranks vary significantly across inputs. GaRA is specifically designed to leverage this variability by dynamically selecting and combining (rank-1) components per input. This enables effective input-adaptive rank modulation even at inference time. \n\nWe will revise the paper to more clearly highlight this distinction.\n\n[1] Nora: Nested low-rank adaptation for efficient fine-tuning large models, arXiv 2024.\n\n[2] Adaptive budget allocation for parameter-efficient fine-tuning, ICLR 2023.\n\n[3] Elalora: Elastic & learnable low-rank adaptation for efficient model fine-tuning, arXiv 2025.\n\n___\n\n### **Weakness 2. Table order confusion**\n\n\nWe apologize for the confusion. We will reorganize the tables in the revision to match the flow of sections, improving overall readability.\n___\n### **Weakness 3. Visibility of Figure 4**\n\n\nThe third row in Figure 4 shows a real nighttime image under extreme low-light conditions. The RGB pixel intensities range from 0 to 34 with a mean of 4.49, indicating severely limited illumination in the scene. To enhance visibility, we will include a brightness-enhanced version and clarify this explanation in the revision.\n___\n### **Question 1-1. Fine-tuning query, key, and value projections**\n\n\nWe conducted additional experiments directly fine-tuning query, key, and value projection layers. Although this fine-tuning strategy improved robustness compared to the pretrained SAM, it significantly underperformed compared to GaRA-SAM, as shown in the table below.\n\n| Method               | IoU  |\n|----------------------|:-------:|\n| QKV Fine-tuning    | 73.1  |\n| GaRA-SAM            | **77.0** |\n\n\n___\n### **Question 1-2. Fine-tuning the traditional adapter**\n\n\nWe fine-tuned the traditional adapter [1] as an alternative to LoRA. The results below showed limited robustness improvement compared to LoRA. We suspect this is due to the nonlinear bottleneck of the traditional adapter, which can distort features and lead to unstable adaptation under corruption. In contrast, LoRA applies linear updates directly to the weights, enabling more stable adaptation.\n\n| Method               | IoU  |\n|----------------------|:-------:|\n| Traditional adapter [1]    | 63.0  |\n| LoRA            | **73.4** |\n\n[1] Parameter-efficient transfer learning for NLP, PMLR 2019.\n___\n### **Question 2.  Applying LoRA to the decoder**\n\n\nWe applied LoRA to the decoder instead of the encoder, but this resulted in a significant performance drop. We attribute this to the encoder continuing to generate degraded representations under corrupted inputs, which causes a mismatch with the adapted decoder parameters. In contrast, applying LoRA to the encoder directly mitigates feature degradation, leading to substantially better performance:\n\n| Method               | IoU  |\n|----------------------|:-------:|\n| LoRA on decoder   | 29.6  |\n| LoRA on encoder           | **73.4** |\n\n___\n### **Question 3. Mergeability of GaRA into the weights of SAM**\n\n\nGaRA cannot be merged statically into SAM since its gating decisions are dynamic and input-dependent, i.e., different (rank-1) components are activated for different inputs during inference. Nevertheless, GaRA remains efficient, as it introduces only minimal overhead while keeping the backbone frozen. We will clarify this point in the revision.\n___\n### **Question 4.  Code availability**\n\nWe fully agree that open-sourcing is essential for reproducibility and future research. We will release the full implementation, including model weights and training/evaluation scripts."}, {"review_id": "MwfuHSZNNA", "reviewer": "Reviewer_Ljhg", "summary": "The paper GaRA-SAM introduces a novel approach to enhance the robustness of the Segment Anything Model (SAM) under diverse input degradations, such as noise, blur, and adverse weather conditions. The core innovation is the Gated-Rank Adaptation (GaRA) module, which integrates lightweight, input-adaptive adapters into SAM’s frozen architecture. Unlike prior methods like RobustSAM, which require paired clean-corrupted images for training, GaRA dynamically adjusts the effective rank of its adapters via a hierarchical gating mechanism. This enables fine-grained adaptation to specific corruptions while preserving SAM’s zero-shot generalization capabilities.", "questions": "1. While GaRA-SAM demonstrates clear improvements over RobustSAM across synthetic and real-world benchmarks, the paper does not address whether the performance gaps stem solely from input-adaptive rank modulation or other architectural differences (e.g., hierarchical gating vs. RobustSAM’s anti-degradation modules). A controlled ablation isolating the impact of dynamic rank selection versus gating mechanisms would strengthen causal claims.\n\n2. [minor] GaRA-SAM’s training on BDD+LIS (autonomous driving data) raises questions about its applicability to non-driving scenarios (e.g., medical or aerial imagery). The paper does not validate cross-domain robustness beyond adverse-weather use cases. The reviewer understands the difficulty of the experiments.", "limitations": "See Questions.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\nComprehensive benchmarking: GaRA-SAM is evaluated across six datasets, including synthetic (LVIS, MSRA-10K) and real-world (BDD+LIS, ACDC) corruptions. It consistently outperforms baselines like RobustSAM, HQ-SAM, and restoration-based methods (AirNet, URIE) by 3.4–21.3% IoU.\n\nMechanistic validation: Ablation studies dissect the impact of hierarchical gating (removing it reduces performance by 3.6% IoU) and rank-space selection (unified rank spaces underperform by 1.4–4.3% IoU)\n\nReal-world applicability: Training on unpaired real data (BDD+LIS) closes the sim-to-real gap, achieving 89.7% IoU on BDD+LIS and 88.8% IoU on ACDC\n\nWeaknesses:\n\nStatistical reporting: Results are reported from single runs without error bars or significance tests, limiting insights into variance", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your insightful feedback and constructive suggestions, which help improve our paper substantially. We will address all the comments and include additional experiments in the revision. Please find our responses to the comments below. All ablation studies were conducted using the ViT-B backbone and evaluated on LVIS with point prompts.\n___\n\n### **Weakness 1. Statistical reporting**\nThank you for the comment. We agree that reporting statistical variation is important for ensuring reliability. Accordingly, we repeated the main experiments three times with different random seeds using the ViT-B backbone and evaluated the models using point prompts. As shown in the table below, our method consistently achieves strong performance with low standard deviation across four datasets.\n\n| Dataset | Degraded IoU | Degraded PA | Degraded Dice | Clear IoU | Clear PA | Clear Dice |\n|---------|:--------------:|:-------------:|:----------------:|:------------:|:-----------:|:-------------:|\n| LVIS    | 76.7 ± 0.3   | 94.2 ± 0.2  | 85.5 ± 0.2     | 82.0 ± 0.8 | 95.4 ± 0.1| 89.1 ± 0.6  |\n| MSRA    | 89.3 ± 0.2   | 97.4 ± 0.1  | 93.9 ± 0.2     | 91.3 ± 0.1 | 97.9 ± 0.1| 95.1 ± 0.1  |\n| COCO    | 76.9 ± 0.5   | 94.4 ± 0.2  | 85.6 ± 0.3     | 80.5 ± 1.0 | 95.2 ± 0.3| 88.0 ± 0.7  |\n| SN      | 77.3 ± 0.5   | 99.5 ± 0.0  | 86.2 ± 0.4     | 82.6 ± 0.4 | 99.7 ± 0.0| 89.9 ± 0.3  |\n\nWe will revise all tables to reflect the statistical results.\n___\n### **Question 1. Impact of dynamic (rank-1) selection and rank space gating**\n\n\nThank you for the insightful suggestion. To isolate the contributions of dynamic (rank-1) component selection and rank space gating, we conducted the following ablation studies.\n\n**1. Isolating the effect of dynamic (rank-1) component selection**\n\nWe tested variants using only dynamic (rank-1) component selection within a unified rank space. Enabling the dynamic selection improves performance when using a unified rank space with the small maximum rank (i.e., 16), highlighting the benefit of adaptively choosing components per input. However, using a unified rank space with the large maximum rank (i.e., 256) did not lead to better results. We attribute this to the limited diversity in active components when using a unified space, which can restrict the ability to adjust representational capacities according to each degraded input, thus hindering input-adaptive modulation. It motivated our design to split the rank space, which led to improved performance.\n| Variant                       | (Rank-1) Component Selection | Rank space gating | IoU  |\n|-----------------------------------|:-------------------------------:|:------------:|:-------:|\n| w/o (rank-1) component selection (max rank 16) |                |    |           75.0  |\n| w/ (rank-1) component selection (max rank 16)  | ✔              |   |            76.2  |\n| w/o (rank-1) component selection (max rank 256) |             |    |          73.4  |\n| w/ (rank-1) component selection (max rank 256)  | ✔             |    |          73.3  |\n| GaRA-SAM (max rank 256) | ✔             |  ✔  |         **77.0**  |\n\n**2. Isolating the effect of rank space gating**\n\nWe also experimented with rank space gating alone with fixed rank settings (e.g., rank 16 or rank 128), disabling dynamic (rank-1) component selection. The results demonstrate that rank space gating alone significantly improves performance. We believe this is because it enables each rank space to specialize in handling different degraded inputs, allowing the gating to select the most suitable representational capacity for each. Furthermore, enabling (rank-1) component selection on top further boosts the performance, highlighting its complementary benefit.\n| Variant                       | (Rank-1) Component Selection | Rank space gating | IoU  |\n|-----------------------------------|:-------------------------------:|:------------:|:-------:|\n| w/o rank space gating |             |   |           73.4  |\n| w/o rank space gating |            | ✔   |           76.0  |\n| GaRA-SAM | ✔             |  ✔  |            **77.0**  |\n\nThese findings suggest that both dynamic (rank-1) component selection and rank space gating mechanisms are crucial and complement each other. We will include these analyses in our revised manuscript.\n___\n### **Question 2. Generalization beyond driving scenarios**\n\n\nWe appreciate your important question regarding cross-domain applicability. While BDD consists of autonomous driving scenes, LIS contains diverse low-light scenes across both indoor and outdoor environments. We will clarify this in the revision.\n\nTo further evaluate the generalization capability of GaRA-SAM beyond driving scenarios, we additionally tested it on the ISIC-2016 medical segmentation dataset [1]. Using box prompts, GaRA-SAM showed notable improvements over SAM:\n| Model      | IoU    | Dice   | AP     |\n|------------|:--------:|:--------:|:--------:|\n| SAM        | 70.6 | 80.7 | 76.8 |\n| GaRA-SAM | **78.7** | **86.2** | **81.7** |\n\nThese results indicate that GaRA-SAM maintains strong performance even in domains with very different visual characteristics. We will include this result in the revision.\n\n[1] Skin lesion analysis toward melanoma detection: A challenge at the 2017 International symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC), arXiv 2016"}]

Author rebuttal:

Meta-review: The paper proposes GaRA-SAM, a method to improve the robustness of SAM (Segment Anything) when images are degraded. GaRA-SAM introduces a Gated Rank Adaptation module that adjusts the rank of an adapter dynamically based on the input. The reviewers initially praised the paper's strong results and practical relevance, but raised concerns about memory overhead, the rationale for the gating design, limited discussion of limitations, and unclear statistical reporting. 

During the rebuttal period, the authors reported new ablations (e.g., rank configurations, gating depth), comparisons to larger SAMs and alternative adaptations, repeated runs with variance reporting, and promised to extend their discussion of limitations in the final version. They also committed to open-sourcing the code.

After rebuttal and discussion, the reviewers agree that most concerns were convincingly addressed, with only minor limitations left. Three reviewers (feAp, uBVp, Ljhg) recommend accept, highlighting GaRA-SAM’s strong empirical results, practicality, and significance despite some minor weaknesses (memory overhead, limitations discussion, statistical reporting). One reviewer (N9R9) remains a bit more cautious, rating it borderline accept, mainly due to concerns about novelty and presentation, but still acknowledging solid experiments and overall contribution.

Given the constructive interactions during the rebuttal phase and the generally very positive view after discussion, I think this paper makes a valuable contribution to the NeurIPS community and am in favor of accepting it.
Decision: Accept (poster)