Title: GaRA-SAM: Robustifying Segment Anything Model with Gated-Rank Adaptation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Robustness

Segment Anything Model

LoRA
Abstract: Improving robustness of the Segment Anything Model (SAM) to input degradations is critical for its deployment in high-stakes applications such as autonomous driving and robotics. Our approach to this challenge prioritizes three key aspects: first, parameter efficiency to maintain the inherent generalization capability of SAM; second, fine-grained and input-aware robustification to precisely address the input corruption; and third, adherence to standard training protocols for ease of training. To this end, we propose gated-rank adaptation (GaRA). 
GaRA introduces lightweight adapters into intermediate layers of the frozen SAM, where each adapter dynamically adjusts the effective rank of its weight matrix based on the input by selectively activating (rank-1) components of the matrix using a learned gating module. This adjustment enables fine-grained and input-aware robustification without compromising the generalization capability of SAM. Our model, GaRA-SAM, significantly outperforms prior work on all robust segmentation benchmarks. In particular, it surpasses the previous best IoU score by up to 21.3%p on ACDC, a challenging real corrupted image dataset.

GaRA-SAM: Robustifying Segment Anything Model with Gated-Rank Adaptation

Abstract

Improving robustness of the Segment Anything Model (SAM) to input degradations is critical for its deployment in high-stakes applications such as autonomous driving and robotics. Our approach to this challenge prioritizes three key aspects: first, parameter efficiency to maintain the inherent generalization capability of SAM; second, fine-grained and input-aware robustification to precisely address the input corruption; and third, adherence to standard training protocols for ease of training. To this end, we propose gated-rank adaptation (GaRA). GaRA introduces lightweight adapters into intermediate layers of the frozen SAM, where each adapter dynamically adjusts the effective rank of its weight matrix based on the input by selectively activating (rank-1) components of the matrix using a learned gating module. This adjustment enables fine-grained and input-aware robustification without compromising the generalization capability of SAM. Our model, GaRA-SAM, significantly outperforms prior work on all robust segmentation benchmarks. In particular, it surpasses the previous best IoU score by up to 21.3%p on ACDC, a challenging real corrupted image dataset.

1 Introduction

The Segment Anything Model (SAM) [28] has proven to be a powerful tool for zero-shot image segmentation, exhibiting impressive generalization across unseen objects and images without additional training. Nevertheless, its performance deteriorates considerably when faced with degraded input due to noise, blur, low illumination, and adverse weather [8], as shown in Figure 1(b). This limitation significantly restricts its use in high-stakes applications such as autonomous driving and robotics.

A seemingly straightforward approach to improving the robustness of SAM is to attach an existing image restoration module to the front of SAM. However, this typically introduces significant computational overhead, and often yields suboptimal segmentation performance since image restoration is optimized to enhance the perceptual quality of images, rather than to improve performance of segmentation models like SAM [7, 12, 30, 33, 52, 57]. An alternative strategy involves fine-tuning SAM entirely on degraded inputs, which, however, demands substantial computational resources and diminishes its inherent zero-shot generalization capability [8].

Chen et al. [8] addressed these limitations by introducing anti-degradation modules into SAM; these modules are trained to refine degraded features by promoting feature-level consistency between a pair of clean and corrupted images of the same content. Despite effectively enhancing robustness, their method, dubbed RobustSAM, has a couple of limitations. First, its requirement for paired clean and degraded images, which cannot be captured by typical cameras, necessitates the use of synthetic degradations in training, hindering its generalization to real-world degradations. Second, since RobustSAM aims at learning representations invariant to various degradations, it struggles to produce representations adapted to the specific degradation affecting the input at hand, restricting

Image

Encoder

Point

Prompt

Pred.

Mask

GT

Mask

Box

Prompt

Input

Image

Segmentation

loss

(1) Activate (rank-1) components

(2) Combine components

Fog

Noise

Unseen

Corruption

rank=5

rank=2

rank=3

(a)

Input

SAM

RobustSAM

GaRA-SAM

Ground-Truth

(b)

SAM

GaRA

active

inactive

🔥

further performance improvement. These limitations reduce the effectiveness of RobustSAM in real-world scenarios, where inputs can be degraded by real and previously unseen corruptions.

We found that low-rank adaptation (LoRA) [20] offers a promising foundation for overcoming many of the previously discussed limitations. By introducing and training lightweight adapters within the frozen SAM, LoRA improves the robustness parameter-efficiently while preserving the generalization ability of SAM. Our empirical analysis confirms that SAM incorporating LoRA achieves impressive robustness without using paired clean and degraded images for training, thus establishing a strong baseline as is. Nevertheless, our analysis also reveals a key limitation: the fixed rank of the adapters hinders its effective adaptation to a wide range of corruptions. We found a significant variation in the optimal rank of the adapters between different corruption types and inputs, underscoring the need for input-adaptive rank modulation.

Based on this observation, we introduce gated-rank adaptation (GaRA), which is illustrated in Figure 1(a), a novel method designed to enhance the robustness of SAM while addressing the aforementioned limitations. GaRA dynamically adjusts the effective rank of the weight matrix of each adapter, while being parameter-efficient and not demanding clean-degraded image pairs for training. Specifically, GaRA decomposes the weight matrix of an adapter into (rank-1) components and chooses a proper subset of them dynamically according to the input. To achieve this, we introduce a gating module that predicts a binary vector selectively activating the most appropriate components for the input. This mechanism allows GaRA to flexibly control both the number and combination of active components based on the input without any test-time optimization. Consequently, our zero-shot segmentation model integrating GaRA with SAM, named GaRA-SAM, achieves fine-grained and input-aware robustification while remaining parameter-efficient.

GaRA-SAM achieves the state of the art on multiple robust segmentation benchmarks [5, 9, 15, 40, 47, 51, 55, 61], including both synthetic and real-world corruption datasets. Importantly, and in contrast to prior work, the design of GaRA-SAM enables training using real-world degraded images lacking clean counterparts, leading to notable performance improvement on real-world corruption benchmarks. The main contribution of this work is three-fold:

Our extensive analysis reveals the surprising effectiveness of LoRA in robustifying SAM, with its optimal rank varying significantly across different corruption types and individual images. These findings suggest a new research avenue for improving robustness of vision foundation models.

We propose GaRA, a novel method to achieve robust SAM. At its core lies a lightweight and input-dependent adapter that enables fine-grained and parameter-efficient robustification without compromising the generalization capability of SAM. Also, GaRA does not require paired clean and degraded images for training and thus, unlike previous work [8], it can be learned using real degraded images without their clean references.

Our final model, GaRA-SAM, achieves the state of the art across multiple robust segmentation benchmarks. Notably, it significantly surpasses the previous best IoU score by up to 21.3%p on ACDC, a challenging real-world corrupted image dataset.

2 Related Work

Robust Segment Anything. SAM [28] accepts free-form prompts along with an image to produce relevant masks, showing superior zero-shot generalizability. Despite its success, its robustness against visual corruptions is questionable [8, 48]. To improve the robustness of SAM, RobustSAM [8] adopts anti-degradation modules to approximate features of clean images. However, it requires cleandegraded image pairs for training, which are often unavailable in real-world scenarios. Improving the robustness of vision models with image restoration [1, 32, 43, 46, 52] and degradation-specific techniques [3, 5, 30, 44] has also been investigated. AirNet [32] and URIE [52] are universal image restoration models, but they introduce heavy computational overhead and AirNet targets a better image quality, not improving performance of visual perception models. LoRA-IR [1], DA-CLIP [43], and PromptIR [46] tackle this by encoding degradation-specific information into prompts. However, they rely on additional information such as degradation types and textual descriptions, and suffer from their complex modules and training schemes. Meanwhile, FIFO [30] and FreD [3] focus on extracting fog-invariant features, making their models robust to only foggy scenes. In contrast to these prior arts, GaRA is an efficient yet effective approach to fine-tuning SAM for improving its robustness without demanding clean references of degraded images for training.

Low-rank Adaptation. Fine-tuning large-scale pre-trained models introduces intensive overheads in space and time. To overcome this, various parameter-efficient learning schemes such as prompt tuning [23, 31, 35, 50] and adapter-based fine-tuning [6, 11, 16, 18, 19, 26, 36, 45, 60] have emerged. Among these, LoRA [19] leverages trainable low-rank matrices to catch up with full fine-tuning while introducing marginal extra learnable parameters. However, fixed ranks of LoRA yield limitations such as poor generalization in certain tasks [56, 59] and inefficient parameter allocation [4, 63]. Despite the efforts of previous work to update the ranks during training dynamically [39, 56, 63], they still use fixed ranks at test time and require handcrafted rank selection. In contrast, GaRA learns a small module that selects and combines appropriate (rank-1) components of a LoRA block, adapting to individual samples with various visual corruptions even during inference.

Mixture of Experts. While model scaling has been known as an effective and promising way of constructing a powerful model, training such a model on large-scale datasets [2, 17, 24, 29, 54, 62] is challenging due to high computational requirements [14, 49]. To this end, mixture of experts (MoE), which adopts multiple submodules and considers each as an expert, has gained prominence [14, 21, 38, 49, 58, 64]. These submodules are active or inactive using a learnable gating module at both training and test time, resulting in efficient use of resources and improved training stability. Switch Transformer [14] sparsely simplifies the MoE paradigm and proposes to select experts sparsely. AdaMix [58] injects an MoE adapter consisting of various up- and down-sampling layers into each transformer layer, both improving parameter-efficiency and performance. Motivated by the prior work, GaRA first decomposes the low-rank matrices of LoRA into (rank-1) components and considers each as an expert. Then, a learnable gating module sparsely selects a combination of (rank-1) components in an input-adaptive manner, enabling adaptation on a per-input basis.

3 Proposed Method

We first describe LoRA as a strong baseline for robustifying SAM in Sec. 3.1, and analyze the impact of the rank of its adapters on the robustness of SAM to verify our motivation of input-adaptive rank modulation in Sec. 3.2. Finally, Sec. 3.3 details GaRA that we integrate into SAM as GaRA-SAM.

3.1 Foundation: Low-rank Adaptation for Robustifying SAM

LoRA [20] is a parameter-efficient adaptation method that imbues a frozen pretrained model with adaptability using a handful of trainable parameters. It freezes a pretrained weight matrix W 0 ∈ R D × K and introduces trainable low-rank update ∆ W = BA , where B ∈ R D × R and A ∈ R R × K with rank R ≪ min( D,K ) . The adapted forward pass becomes:

0.62

0.66

0.7

0.74

0.78

0.82

RobustSAM

LoRA-Rank2

LoRA-Rank16

LoRA-Rank128

(a)

IoU

(b)

IoU

0.72

0.74

0.76

0.78

0.8

0.82

where x ∈ R K is the input and h ∈ R D is the output.

Given its original purpose of parameter-efficient adaptation without compromising generalization, we argue that LoRA offers a reasonable baseline for improving robustness of SAM against input degradation. To validate this potential empirically, we evaluated SAM integrated with LoRA for robust segmentation using point prompts following an evaluation protocol of RobustSAM [8]. Specifically, we froze the original weights of SAM, attached low-rank adapters ∆ W to the key, query, and value projection layers of its image encoder, and trained the adapters with the standard segmentation loss on degraded images. As demonstrated in Figure 2, SAM with LoRA clearly outperformed RobustSAM [8] on the LVIS dataset [15], although it does not require clean reference images and the auxiliary loss the RobustSAM demands.

3.2 Analysis on the Impact of Rank on Robustness

Since different degradations distort different aspects of the input, the level of representation capacity needed for robustifying SAM varies accordingly, motivating the investigation into the role of LoRA's rank. To analyze this, we first evaluated its performance across various input corruptions while varying the rank. The results in Figure 2(a) suggest that the optimal rank varies depending on the corruption type. We conjecture that more pronounced corruptions lead to more substantial contamination of the semantic content of the input, and lower-rank adapters are more effective in such conditions since their restricted capacity forces them to prioritize more essential features for segmentation [42].

To further verify this conclusion, we evaluated SAM with LoRA under two oracle rank modulation scenarios: selecting the best rank per corruption type (Oracle-Corrupt) and per image (OracleInstance) using ground-truth. Specifically, we first computed IoU scores for all candidate ranks and selected only the top-performing rank for each corruption type or each image. While Oracle-Corrupt yielded moderate gains over fixed ranks, Oracle-Instance achieved significantly higher performance, highlighting the substantial potential of fine-grained, input-adaptive rank selection. These findings motivate our design of GaRA, which dynamically adjusts the rank per input.

3.3 Gated-rank Adaptation

Our solution for robustifying SAM, termed gated-rank adaptation (GaRA), is designed with three key properties in mind: parameter efficiency to preserve the generalization ability of SAM, fine-grained and input-aware robustification, and adherence to standard segmentation learning protocols for ease of training. These properties are realized by a novel, lightweight adapter that dynamically adjusts itself based on the input corruption; the architecture of the adapter is depicted in Figure 3. Our final model, GaRA-SAM, is constructed by integrating these adapters into the key, query, and value projection layers of the image encoder of SAM, keeping the original SAM weights frozen. Also, GaRA-SAM is trained solely with the standard segmentation loss computed from degraded images, without auxiliary learning objectives or paired clean references.

GaRA

LoRA

Linear

𝑨

𝑩

Linear

𝐛

!

"

𝐚

!

" #

…

𝐛

$

"

𝐚

$

" #

𝐛

%

"

𝐚

%

" #

𝐛

&

!

"

𝐚

&

!

"

#

…

𝐛

!

'

𝐚

!

'

#

𝐛

$

'

𝐚

$

'

#

𝐛

%

'

𝐚

%

'

#

𝐛

&

"

'

𝐚

&

"

'

#

Lower-rank space

𝐛

(

"

𝐚

(

" #

𝐛

)

"

𝐚

)

"

#

𝒛

!

…

∈ ℝ

&

"

Higher-rank space

𝑔

!

…

:

Active

:

Inactive

𝒛

"

𝑔

"

𝑔

#$%&'

∈ ℝ

&

!

Higher

Lower

文

X

↑

十

↑

△

↑

GaRA introduces a gating strategy to modulate the adapter's rank based on the input. To this end, we reinterprete the update matrix of LoRA, i.e. , ∆ W = BA , as a composition of R (rank-1) components: ∆ W = ∑ R i =1 b i a ⊤ i , where b i ∈ R D and a i ∈ R K are the i -th column and row of B and A , respectively. We assume that R is large enough so that the model has access to a rich set of (rank-1) components, enabling GaRA to flexibly choose a relevant subset depending on the input.

GaRA employs a hierarchical gating strategy that provides both coarse and fine control over the adaptation process. The adapter first coarsely selects between lower- and higher-rank spaces based on the input, and then, within the selected rank space, the gating module finely determines the rank of the adapter by activating a subset of the (rank-1) components appropriate for the input. The coarse rank space selection separates the adapter's components into two exclusive sets, tailored to different degraded inputs demanding different representation capacities, to mitigate potential conflicts between the sets and further improve performance. Meanwhile, the fine-grained gating enables flexible and input-specific composition of the update matrix, supporting efficient and expressive adaptation to diverse corruptions. Below we elaborate on this gating process.

Gating 1: Rank Space Selection. We explicitly divide the adapter into two distinct rank spaces, a lower-rank set { a L i , b L i } r L i =1 and a higher-rank set { a H j , b H j } r H j =1 with r L < r H ≪ K , where K denotes the input feature dimension, and r L and r H indicate the maximal ranks of the lower-rank and higher-rank spaces, respectively. 1 To choose between these rank spaces, we employ a binary gating module g space, which takes as input the intermediate feature f ( x ) computed from x and outputs a binary gating variable z space ∈ { 0 , 1 } , where 0 and 1 indicate the lower- and higher-rank space, respectively. This module consists of a two-layer MLP followed by the Gumbel-Sigmoid, allowing differentiable binary decisions during training. The gating process is formally expressed as:

where α space is the gating logit computed from the input feature. To enable backpropagation despite the binary nature of the binary gating variable z space, we apply the Gumbel-Sigmoid relaxation [22]:

where G ∼ Gumbel (0 , 1) is a noise sampled during training, σ is the sigmoid function, and τ is a temperature parameter controlling the sharpness of the sigmoid. During training, we apply hard thresholding in the forward pass as z space = I [˜ z space > 0 . 5] while using the continuous value ˜ z space to compute gradients in the backward pass. At test time, we discard the noise and compute the gate: z space = I [ σ ( α space ) > 0 . 5] .

Gating 2: (rank-1) Component Selection. Within the selected rank space, the associated gating module predicts a binary vector identifying (rank-1) components suitable for the input:

where g L ( · ) and g H ( · ) denote the gating modules for the lower-rank and higher-rank spaces, respectively. Each gating module consists of a three-layer MLP whose final output is a real-valued logit vector, followed by Gumbel-Sigmoid operations to obtain binary masks:

To enable gradient-based learning, we apply the Gumbel-Sigmoid relaxation to each element of the logit vectors [22]:

where α i is the i -th logit from α L or α H , G i ∼ Gumbel (0 , 1) is the corresponding noise sample, τ is a temperature parameter, and σ is the sigmoid function. During training, hard thresholding is applied in the forward pass to produce binary decisions ( z i = I [˜ z i > 0 . 5] ), while the continuous ˜ z i is used for backpropagation. At inference time, the gating becomes deterministic: z i = I [ σ ( α i ) > 0 . 5] . We apply this relaxation to the gating functions g L and g H to enable learnable and input-adaptive binary decisions for the selection of (rank-1) components. This gating mechanism results in a dynamic, input-dependent adapter update:

This design of GaRA enables flexible and conflict-free adaptation to a wide range of corruptions by coarse-to-fine rank modulation.

4 Experiments

4.1 Experimental Setting

Dataset. For training and validation, we utilize the Robust-Seg dataset [8], which is constructed by applying 15 types of synthetic corruptions to three semantic segmentation benchmarks: LVIS [15], MSRA-10K [9], and ThinObjects-5K [37], comprising a total of 26,000 masks. For evaluation, we use five clear-condition image segmentation benchmarks: LVIS, MSRA-10K, STREETS [51], NDD20 [55], COCO [40]. Also, we test on a real-world corrupted benchmark including BDD100K [61] and LIS [5]. For training GaRA-SAM on real-world data, we use BDD-100K and LIS for training, and evaluate on BDD-100K and LIS, and ACDC [47]. Note that all these datasets are free from licensing issues. For brevity, we refer to the union of BDD-100K and LIS as BDD+LIS, and the union of STREETS and NDD20 as STREETS+NDD. More details are given in the appendix.

Experimental Details. We adopt the ViT-B and ViT-L variants of SAM [13], and freeze their parameters during training the GaRA modules. The models are optimized by Adam [27] with a learning rate of 1 × 10 -4 for ViT-B and 1 × 10 -5 for ViT-L, a weight decay of 1 × 10 -5 , and input batches of size 8, using both point and box prompts. The gating modules are trained with the same learning rate. We set the lower- and higher-rank dimensions as r L = 16 and r H = 256 , respectively, and use a Gumbel-Sigmoid temperature of 0.5. Since no official evaluation code is provided by previous work, we reproduce and evaluate its models using the same protocol. All training and evaluation experiments were conducted at POSTECH.

4.2 Comparison on Seen Dataset

To assess the robustness of GaRA-SAM, we first evaluate its performance on synthetic corruptions applied to seen datasets such as LVIS and MSRA-10K. The results in Table 1 show that GaRASAM outperforms prior methods, including RobustSAM [8], HQ-SAM [25], and restoration-based methods ( e.g. , AirNet [32] and URIE [52]), when using both point and box prompts in both datasets. Notably, our model demonstrates significant improvements in the degraded setting, e.g. , surpassing the previous best by up to 4.3%p on LVIS without compromising performance on clean images.

4.3 Zero-shot Segmentation Comparison

To further assess the generalization capability of GaRA-SAM, we evaluate it under both synthetic and real corruptions on datasets not seen during training. Specifically, we include COCO and STREETS+NDD with synthetic degradations, and BDD+LIS, which exhibits real corruptions such as fog, rain, motion blur, and low-light. As reported in Table 2, GaRA-SAM consistently achieves the best across all prompt types and metrics in both COCO and STREETS+NDD. Table 3 shows that it also outperforms previous work on real corrupted images, improving over RobustSAM by more than 3.4%p IoU with point prompts in ViT-L, despite being trained only on unpaired corrupted images. It demonstrates that GaRA-SAM generalizes well to both synthetic and real-world corruptions, highlighting the effectiveness of its input-adaptive design.

4.4 Training on Real Corruption Datasets

GaRA-SAM enables training on real corruption datasets without requiring access to clean references. To demonstrate this, we train GaRA-SAM solely on the real-world dataset, BDD+LIS, and evaluate its performance on both the seen dataset (BDD+LIS) and an unseen real-world dataset, ACDC. As shown in Table 4, training directly on real corrupted images significantly improves the performance on all benchmarks, outperforming the previous best by up to 21.3%p IoU on ACDC. In addition, we compare GaRA-SAM trained on BDD+LIS (GaRA-SAM-Real) and that trained on the synthetic Robust-Seg dataset (GaRA-SAM-Syn), using box prompts and ViT-L in Table 3; the results suggest the clear benefit of training directly on real corrupted images.

Input

SAM

HQ-SAM

RobustSAM

GaRA-SAM (ours)

GT

Synthetic

Real

4.5 Qualitative results

Figure 4 presents qualitative results under synthetic and real-world corruptions. GaRA-SAM delivers more accurate and complete masks than SAM, HQ-SAM, and RobustSAM, especially under severe degradations: it better preserves object boundaries in synthetic cases and is more reliable in low-light and adverse weather conditions.

4.6 In-depth analysis

We conduct a comprehensive ablation study to assess the contribution of each design choice in GaRASAM. All experiments are performed on the LVIS dataset, using point prompts for consistency.

LoRA vs. GaRA. We first compare GaRA with the standard LoRA with fixed ranks. As shown in Table 5, GaRA-SAM consistently outperforms all fixed-rank LoRA. While LoRA establishes a strong baseline, its fixed-rank design limits flexibility. In contrast, GaRA-SAM dynamically composes (rank-1) components based on the input, resulting in improved robustness. These results validate the benefit of input-adaptive rank modulation over static alternatives.

60

65

70

75

80

0.3

0.5

0.7

0.9

IoU

Temperature

Ours

SAM

Elastic, rank: 12

Snow, rank: 155

𝐳 𝐻

𝐳 𝐿

𝐳 𝐿

Rain, rank: 9

Active

Inactive

Clean Reference

Clean Reference

𝐳 𝐻

Snow, rank: 151

𝐳 𝐻

Elastic, rank: 159

Rain, rank: 11

𝐳 𝐿

MoE LoRA vs. GaRA. We also investigate an alternative approach to rank selection using a mixtureof-experts (MoE) [64] variant of LoRA, where multiple fixed rank LoRA blocks are instantiated, and a shared gating module selects one for each input. As reported in Table 6, both MoE variants perform reasonably well, but fall short of GaRA-SAM in both clean and degraded conditions. This gap is not trivial regarding the fewer number of parameters of GaRA-SAM.

Impact of Rank Space Selection. To evaluate the effectiveness of rank space selection (Gating 1), we compare GaRA-SAM with a variant that removes the separation between the lower- and higher-rank spaces, using a single unified set of (rank-1) components with a fixed maximum rank ( e.g. , 16 or 256). As shown in Table 7, eliminating rank space selection results in performance degradation, suggesting that separating the low- and high-rank components allows more effective specialization.

Impact of Temperature in Gumbel-Sigmoid. We investigate the effect of the temperature of the Gumbel-Sigmoid by varying its value. Figure 5 summarizes the performance of variants of GaRA-SAM trained using different values of temperature; the optimal value for τ is 0.5. The results suggest that GaRA-SAM remains stable across a wide range of temperature values, suggesting that GaRA-SAM is insensitive to temperature settings.

Computational Cost Analysis. We compare the computational efficiency of GaRA-SAM in terms of learnable parameters, GPU resources, memory consumption, and inference speed (FPS). As summarized in Table 8, GaRA-SAM achieves the lowest number of learnable parameters and GPUs, owing to its parameter-efficient design. At inference time, it requires less GPU memory than RobustSAM, while maintaining comparable FPS to both SAM and RobustSAM.

Impact of Maximum Rank Configuration. We demonstrate the effect of our chosen maximum lower-rank and higher-rank configuration. Table 9 summarizes the performance of variants of GaRASAM using different maximum rank configurations. The optimal rank combination is {16, 256}, achieving the best performance.

Analysis on Gating Vectors. Figure 6 demonstrates how the gating module responds to different combinations of corruption type and image. For each corrupted input, we present the corresponding binary activation vector z H or z L according to their selected rank space, along with the resulting number of active (rank-1) components. We observe that GaRA activates different (rank-1) components depending on the image contents as well as the corruption types. Even when the selected rank is similar, the constituent components often vary, highlighting the input-adaptive and fine-grained gating mechanism of GaRA.

Analysis on Gating in GaRA-SAM. To empirically demonstrate that the gating modules are learned to provide desirable gating paths even with such indirect supervision, we compared GaRA-SAM and its variant with random gating. Table 10 indicates that GaRA-SAM significantly outperforms the random gating variant, supporting the effectiveness of our training strategy.

Isolating Effect of Rank Space Selection. To isolate the effect of rank space selection, we experimented with rank space selection alone with fixed rank settings, disabling (rank-1) component selection. Table 11 demonstrate that rank space separation alone significantly improves performance. We believe this is because it enables each rank space to specialize in handling different degraded inputs, allowing the gating to select the most suitable representational capacity for each. Furthermore, enabling (rank-1) component selection on top further boosts the performance, highlighting its complementary benefit.

Impact of Depth of Hierachical Gating. We conducted additional experiments to assess deeper hierarchical gating structures ( i.e. , 3-level and 4-level). As shown in Table. 12, increasing the depth introduces optimization challenges due to more complex gating decisions, ultimately degrading performance. Notably, our current 2-level hierarchy yields the best performance and offers the most favorable capacity and trainability.

5 Conclusion

In this paper, we introduce GaRA-SAM, a novel approach for robustifying Segment Anything Model (SAM) under diverse image degradations. Through extensive empirical analysis, we observed that the optimal LoRA rank varies significantly across corruption types and individual inputs, motivating our design of Gated-Rank Adaptation (GaRA), a lightweight and input-adaptive module that dynamically modulates the effective rank of LoRA adapters. GaRA operates without requiring paired clean and degraded images, enabling fine-grained and parameter-efficient adaptation while preserving SAM's inherent zero-shot generalization capabilities. GaRA-SAM achieves state-of-the-art performance across all robustness benchmarks and, notably, supports training directly on real-world corruption datasets without clean references. This leads to substantial gains in real-world scenarios, highlighting the practical utility and broad applicability of our method.

Limitations. While GaRA adaptively determines the appropriate rank for each rank space, several design choices, such as the split between lower- and higher-rank spaces, the predefined maximum ranks, and the fixed 2-level gating hierarchy, are set manually and thus limit flexibility. More generalized automated designs are conceptually desirable but practically challenging due to unstable optimization. Developing automated methods remains an important future direction.

Acknowledgement. This work was supported by Samsung Research Funding & Incubation Center of Samsung Electronics (SRFC-IT1801-52), AI Graduate School Program at POSTECH (RS-2019-II191906), and the IITP grants (RS-2024-00457882, RS-2024-00509258) funded by Ministry of Science and ICT, Korea.

References

Yuang Ai, Huaibo Huang, and Ran He. Lora-ir: Taming low-rank experts for efficient all-in-one image restoration. arXiv preprint arXiv:2410.15385 , 2024.

Ibrahim Alabdulmohsin, Behnam Neyshabur, and Xiaohua Zhai. Revisiting neural scaling laws in language and vision. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems , 2022. URL https:// openreview.net/forum?id=h3RYh6IBBS .

Qi Bi, Shaodi You, and Theo Gevers. Generalized foggy-scene semantic segmentation by frequency decoupling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops , pages 1389-1399, June 2024.

Huandong Chang, Zicheng Ma, Mingyuan Ma, Zhenting Qi, Andrew Sabot, Hong Jiang, and HT Kung. Elalora: Elastic & learnable low-rank adaptation for efficient model fine-tuning. arXiv preprint arXiv:2504.00254 , 2025.

Linwei Chen, Ying Fu, Kaixuan Wei, Dezhi Zheng, and Felix Heide. Instance segmentation in the dark. International Journal of Computer Vision , 2023. URL https://github.com/ Linwei-Chen/LIS .

Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, and Ping Luo. Adaptformer: Adapting vision transformers for scalable visual recognition. In Proc. Neural Information Processing Systems (NeurIPS) , 2022.

Wei-Ting Chen, I-Hsiang Chen, Chih-Yuan Yeh, Hao-Hsiang Yang, Hua-En Chang, Jian-Jiun Ding, and Sy-Yen Kuo. Rvsl: Robust vehicle similarity learning in real hazy scenes based on semi-supervised learning. In Proc. European Conference on Computer Vision (ECCV) . Springer, 2022.

Wei-Ting Chen, Yu-Jiet Vong, Sy-Yen Kuo, Sizhou Ma, and Jian Wang. RobustSAM: segment anything robustly on degraded images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4081-4091, 2024.

Ming-Ming Cheng, Niloy J. Mitra, Xiaolei Huang, Philip H. S. Torr, and Shi-Min Hu. Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) , 2015. URL https://mmcheng.net/msra10k/ .

Noel CF Codella, David Gutman, M Emre Celebi, Brian Helba, Michael A Marchetti, Stephen W Dusza, Aadi Kalloo, Konstantinos Liopyris, Nabin Mishra, Harald Kittler, et al. Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic). In 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018) , 2018.

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv:2305.14314 , 2023.

Steven Diamond, Vincent Sitzmann, Frank Julca-Aguilar, Stephen Boyd, Gordon Wetzstein, and Felix Heide. Dirty pixels: Towards end-to-end image processing and perception. ACM Transactions on Graphics (TOG) , 2021.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In Proc. International Conference on Learning Representations (ICLR) , 2021.

William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research , 23 (120):1-39, 2022.

Agrim Gupta, Piotr Dollar, and Ross Girshick. Lvis: A dataset for large vocabulary instance segmentation. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2019. URL https://www.lvisdataset.org/ .

Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig. Towards a unified view of parameter-efficient transfer learning. arXiv preprint arXiv:2110.04366 , 2021.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2016.

Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In Proc. International Conference on Machine Learning (ICML) . PMLR, 2019.

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 , 2021.

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations , 2022.

Changho Hwang, Wei Cui, Yifan Xiong, Ziyue Yang, Ze Liu, Han Hu, Zilong Wang, Rafael Salas, Jithin Jose, Prabhat Ram, HoYuen Chau, Peng Cheng, Fan Yang, Mao Yang, and Yongqiang Xiong. Tutel: Adaptive mixture-of-experts at scale. In D. Song, M. Carbin, and T. Chen, editors, Proceedings of Machine Learning and Systems , 2023.

Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144 , 2016.

Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In European Conference on Computer Vision (ECCV) , 2022.

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 , 2020.

Lei Ke, Mingqiao Ye, Martin Danelljan, Yifan Liu, Yu-Wing Tai, Chi-Keung Tang, and Fisher Yu. Segment anything in high quality. In NeurIPS , 2023.

Sungyeon Kim, Boseung Jeong, Donghyun Kim, and Suha Kwak. Efficient and versatile robust fine-tuning of zero-shot models. In European Conference on Computer Vision (ECCV) , 2024.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 4015-4026, 2023.

Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, and Neil Houlsby. Big transfer (bit): General visual representation learning. In Computer VisionECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part V 16 , pages 491-507. Springer, 2020.

Sohyun Lee, Taeyoung Son, and Suha Kwak. Fifo: Learning fog-invariant features for foggy scene segmentation. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2022.

Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , 2021.

Boyun Li, Xiao Liu, Peng Hu, Zhongqin Wu, Jiancheng Lv, and Xi Peng. All-In-One Image Restoration for Unknown Corruption. In IEEE Conference on Computer Vision and Pattern Recognition , New Orleans, LA, June 2022.

Siyuan Li, Iago Breno Araujo, Wenqi Ren, Zhangyang Wang, Eric K Tokuda, Roberto Hirata Junior, Roberto Cesar-Junior, Jiawan Zhang, Xiaojie Guo, and Xiaochun Cao. Single image deraining: A comprehensive benchmark analysis. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2019.

Xiang Li, Tianhan Wei, Yau Pun Chen, Yu-Wing Tai, and Chi-Keung Tang. Fss-1000: A 1000-class dataset for few-shot segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020.

Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190 , 2021.

Dongze Lian, Daquan Zhou, Jiashi Feng, and Xinchao Wang. Scaling & shifting your features: A new baseline for efficient model tuning. Advances in Neural Information Processing Systems , 35:109-123, 2022.

Jun Hao Liew, Scott Cohen, Brian Price, Long Mai, and Jiashi Feng. Deep interactive thin object selection. In Winter Conference on Applications of Computer Vision (WACV) , 2021. URL https://github.com/liewjunhao/thin-object-selection .

Bin Lin, Zhenyu Tang, Yang Ye, Jiaxi Cui, Bin Zhu, Peng Jin, Junwu Zhang, Munan Ning, and Li Yuan. Moe-llava: Mixture of experts for large vision-language models. arXiv preprint arXiv:2401.15947 , 2024.

Cheng Lin, Lujun Li, Dezhi Li, Jie Zou, Wei Xue, and Yike Guo. Nora: Nested low-rank adaptation for efficient fine-tuning large models. arXiv preprint arXiv:2408.10280 , 2024.

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Proc. European Conference on Computer Vision (ECCV) . Springer, 2014. URL https://cocodataset.org .

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision , 2017.

Jiaming Liu, Senqiao Yang, Peidong Jia, Ming Lu, Yandong Guo, Wei Xue, and Shanghang Zhang. Vida: Homeostatic visual domain adapter for continual test time adaptation. arXiv preprint arXiv:2306.04344 , 2023.

Ziwei Luo, Fredrik K. Gustafsson, Zheng Zhao, Jens Sjölund, and Thomas B. Schön. Controlling vision-language models for multi-task image restoration. In The Twelfth International Conference on Learning Representations , 2024. URL https://openreview.net/forum? id=t3vnnLeajU .

Xianzheng Ma, Zhixiang Wang, Yacheng Zhan, Yinqiang Zheng, Zheng Wang, Dengxin Dai, and Chia-Wen Lin. Both style and fog matter: Cumulative domain adaptation for semantic foggy scene understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 18922-18931, 2022.

Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. Adapterfusion: Non-destructive task composition for transfer learning. arXiv preprint arXiv:2005.00247 , 2020.

Vaishnav Potlapalli, Syed Waqas Zamir, Salman Khan, and Fahad Khan. PromptIR: Prompting for all-in-one image restoration. In Thirty-seventh Conference on Neural Information Processing Systems , 2023. URL https://openreview.net/forum?id=KAlSIL4tXU .

Christos Sakaridis, Dengxin Dai, and Luc Van Gool. ACDC: The adverse conditions dataset with correspondences for semantic driving scene understanding. In Proc. IEEE/CVF International Conference on Computer Vision (ICCV) , 2021. URL https://acdc.vision.ee.ethz.ch/ .

Madeline Chantry Schiappa, Shehreen Azad, Sachidanand Vs, Yunhao Ge, Ondrej Miksik, Yogesh S Rawat, and Vibhav Vineet. Robustness analysis on foundational segmentation models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1786-1796, 2024.

Noam Shazeer, *Azalia Mirhoseini, *Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixtureof-experts layer. In International Conference on Learning Representations , 2017. URL https://openreview.net/forum?id=B1ckMDqlg .

Sheng Shen, Shijia Yang, Tianjun Zhang, Bohan Zhai, Joseph E Gonzalez, Kurt Keutzer, and Trevor Darrell. Multitask vision-language prompt tuning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 5656-5667, 2024.

Corey Snyder and Minh Do. Streets: A novel camera network dataset for traffic flow. Advances in Neural Information Processing Systems , 2019. URL https://databank.illinois.edu/ datasets/IDB-3671567 .

Taeyoung Son, Juwon Kang, Namyup Kim, Sunghyun Cho, and Suha Kwak. Urie: Universal image enhancement for visual recognition in the wild. In Proc. European Conference on Computer Vision (ECCV) , 2020.

Carole H Sudre, Wenqi Li, Tom Vercauteren, Sebastien Ourselin, and M Jorge Cardoso. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In Proc. Medical Image Computing and Computer-Assisted Intervention (MICCAI) . Springer, 2017.

Mingxing Tan and Quoc Le. EfficientNet: Rethinking model scaling for convolutional neural networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning , volume 97 of Proceedings of Machine Learning Research , pages 6105-6114. PMLR, 09-15 Jun 2019. URL https://proceedings.mlr. press/v97/tan19a.html .

Cameron Trotter, Georgia Atkinson, Matt Sharpe, Kirsten Richardson, A Stephen McGough, Nick Wright, Ben Burville, and Per Berggren. Ndd20: A large-scale few-shot dolphin dataset for coarse and fine-grained categorisation. arXiv preprint arXiv:2005.13359 , 2020. URL https://data.ncl.ac.uk/collections/The_Northumberland_Dolphin_ Dataset_2020/4982342/1 .

Mojtaba Valipour, Mehdi Rezagholizadeh, Ivan Kobyzev, and Ali Ghodsi. DyLoRA: Parameterefficient tuning of pre-trained models using dynamic search-free low-rank adaptation. In Andreas Vlachos and Isabelle Augenstein, editors, Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics , pages 3274-3287, Dubrovnik, Croatia, May 2023. Association for Computational Linguistics. doi: 10.18653/v1/ 2023.eacl-main.239. URL https://aclanthology.org/2023.eacl-main.239/ .

Rosaura G Vidal, Sreya Banerjee, Klemen Grm, Vitomir Struc, and Walter J Scheirer. Ugˆ 2: A video benchmark for assessing the impact of image restoration and enhancement on automatic visual recognition. In Proc. IEEE Winter Conference on Applications of Computer Vision (WACV) , 2018.

Yaqing Wang, Sahaj Agarwal, Subhabrata Mukherjee, Xiaodong Liu, Jing Gao, Ahmed Hassan Awadallah, and Jianfeng Gao. AdaMix: Mixture-of-adaptations for parameter-efficient model tuning. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages 5744-5760, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.emnlp-main.388. URL https://aclanthology.org/2022. emnlp-main.388/ .

Wenhan Xia, Chengwei Qin, and Elad Hazan. Chain of lora: Efficient fine-tuning of language models via residual learning. arXiv preprint arXiv:2401.04151 , 2024.

Dongshuo Yin, Leiyi Hu, Bin Li, Youqun Zhang, and Xue Yang. 5%>100%: Breaking performance shackles of full fine-tuning on visual recognition tasks, 2024. URL https: //arxiv.org/abs/2408.08345 .

Fisher Yu, Wenqi Xian, Yingying Chen, Fangchen Liu, Mike Liao, Vashisht Madhavan, Trevor Darrell, et al. Bdd100k: A diverse driving video database with scalable annotation tooling. arXiv preprint arXiv:1805.04687 , 2018. URL https://bdd-data.berkeley.edu/ .

Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 12104-12113, 2022.

Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adaptive budget allocation for parameter-efficient fine-tuning. In The Eleventh International Conference on Learning Representations , 2023. URL https://openreview. net/forum?id=lq62uWRJjiY .

Yanqi Zhou, Tao Lei, Hanxiao Liu, Nan Du, Yanping Huang, Vincent Zhao, Andrew M Dai, Quoc V Le, James Laudon, et al. Mixture-of-experts with expert choice routing. Advances in Neural Information Processing Systems , 35:7103-7114, 2022.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction of our paper sufficiently reflect the paper's contributions and scope.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitations of the work in Section 5.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: The work introduces no theoretical assumptions and results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We disclose all the training and evaluation recipes in Section 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: We do not disclose the code and data at the moment. But we explain the model architecture and experimental settings in detail in Section 4.1.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We explain the detailed experimental settings in Section 4.1. For the composition of the benchmarks used in the paper, please refer to the appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: Due to the computational tractability, we conduct each experiment once.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The number of trainable parameters and GPU memory footprint for inference are reported in Table 8. We also explain the computational environments in the appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We follow the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: The work poses no societal impacts.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This paper releases no findings with such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All the datasets in this paper are free from license issues and are cited properly. We also cite the models used in the paper. Please refer to Section 4.1.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: This paper does not introduce new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper contains no experiments involving human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This work contains no user study.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: We used LLMs for editing purposes only.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Appendix

This appendix presents additional experimental details and results that are omitted from the main paper due to space limit.

A Experimental details

Implementation details. Each GaRA module consists of a branch selector, implemented as a twolayer MLP with hidden dimension 64 and ReLU activation, which outputs a binary gating decision (low vs. high rank) using a Gumbel-Sigmoid; and two rank-wise gating networks (one for low-rank, one for high-rank), each composed of a three-layer MLP with hidden dimension 16 and ReLU activation, and produces binary masks over rank-1 components via Gumbel-Sigmoid sampling. The selected components modulate separate LoRA paths applied to the QKV projections. Following RobustSAM [8], we use a combined segmentation loss, L seg = L dice [ 53 ] + L focal [ 41 ] .

Resource. We use 8 A6000 GPUs for training each method and 1 A6000 GPU for evaluation.

Dataset configuration. Following the dataset configuration introduced in RobustSAM [8], we utilize a diverse collection of segmentation datasets for training and evaluation. For training, we adopt three synthetic datasets: MSRA10K [9] (10,000 images) and LVIS [15] (20,252 images) provide large-scale pixel-wise annotations for salient and instance segmentation, while ThinObject-5K [37] (4,748 images) contains fine-grained masks for thin structures to enhance structural sensitivity. For evaluation, we consider both synthetic and real-world scenarios. The synthetic benchmark includes STREETS [51] (1,000 annotated traffic scenes) and NDD20 [55] (1,000 dolphin segmentation images captured in underwater and above-water environments). We exclude FSS-1000 [34] due to licensing restrictions. For real-world evaluation, we adopt BDD-100K [61], a driving video dataset with 100,000 videos (frame-level annotations at the 10th second) covering diverse conditions, and LIS [5], a low-light instance segmentation dataset with 2,230 paired short/long exposure images (8,920 total images) containing over 10,000 labeled instances.

B Empirical analysis

B.1 Rank distribution analysis

We analyzed the gated rank distribution across all blocks and corruption types on the MSRA10K dataset. As shown in Figure a1, the activated rank values vary significantly not only across corruption types but also across blocks, indicating that GaRA dynamically adapts its rank selection.

B.2 Corruption type vs. gated rank

To further investigate whether our gating mechanism responds differently depending on the corruption type, we conduct an F-statistics analysis on the number of activated ranks across transformer blocks. In this setting, each corruption type is treated as a group, and each sample's gated rank serves as an observation. We compute the F-statistic, which measures the ratio of between-group to within-group variance, and the p-value, which indicates whether the observed variation could arise by chance.

As shown in Figure a2, most blocks exhibit a statistically significant dependency between corruption types and the number of activated ranks (i.e., high F-statistic and low p-value). Interestingly, blocks 2, 5, 8, and 11 do not show such significance. These blocks use standard attention, whereas the others adopt window attention, according to the original structure of SAM [28]. In the window-based setting, different windows within the same image may capture varied content under the same corruption type, leading the shared gating network to attend more to the global corruption style rather than content. This behavior highlights the corruption-aware design of our gating module.

B.3 Justification for rank space separation

The rank space separation is motivated by our empirical observations that, in a unified rank space, the model exhibits limited diversity in (rank-1) component activation across different degraded inputs. Specifically, as shown in Table a1, the model tends to activate components within a narrow

ISO_noise-

100.35

114.13100.52 107.53 104.86108.19

121.82114.50

97.68

116.87

125

brightness-110.25 108.9699.71

113.77

96.56

113.51 108.48 109.45123.71117.04

97.25

118.50

clear-110.22 108.69 100.28114.1098.07

110.38108.18 110.03124.86116.77

97.15

115.96

color jitter-109.95 108.88 102.20 113.4197.75 109.90 108.52 108.97124.46115.7697.24

116.37

120

compression -110.16 109.24 102.08 114.38 97.54 111.35107.46 109.10121.03116.16 98.28

115.44

contrast-110.74 108.32100.96

5115.1498.43

108.27 107.57108.54124.55116.45

98.03

116.78

- 115

elastic_transform-110.65 108.48 101.85 114.0297.63

109.91108.67 109.60124.84116.85

97.72

116.23

corruption

fog -111.78 108.6997.85

96.48

116.18

frosted_glass_blur -110.76 109.4798.31

114.3398.48111.95106.38108.65125.03116.39

97.84

117.25

110

gauss_noise-110.52 109.0199.74

114.86100.34109.74105.56108.30123.17114.97

97.40

118.52

impulse_noise-110.69108.7698.71

114.73

97.98

113.39107.79109.82125.45116.22

96.80

117.67

105

motion_blur-110.49109.3598.46

112.4696.90

111.09 109.44 109.51125.48116.94

96.76

116.54

rain-110.94 109.4599.16

114.9197.43

3111.68107.10110.49123.83117.2095.36

114.46

resampling_blur-110.45 108.9499.24 113.01 96.51 108.40 109.80 108.39124.77116.1197.19

117.18

100

snow -110.51 108.4597.90

115.92

99.34

117.03

zoom_blur-

110.11 108.92 100.94

113.37

97.60

112.22 108.56 109.76123.55117.80

96.40

116.97

0

1

2

3

4

5

6

8

6

10

11

block

range (133-190; std: 7.7), limiting its ability to adjust representational capacity according to each degraded input. This is potentially suboptimal because different degraded inputs require different representational capacities. By separating the rank space into lower-rank and higher-rank regions, we allow the model to activate a more diverse range of components (5-181; std: 75.8), yielding significant performance improvement. We guess this is because the rank space separation into two exclusive sets, each specialized for different degraded inputs demanding different representation capacities, helps mitigate potential conflicts during (rank-1) component selection and promotes more diverse usage of components.

C Additional experimental results

C.1 Generalization ability beyond driving scenarios

To further evaluate the generalization capability of GaRA-SAM beyond driving scenarios, we additionally tested it on the ISIC-2016 medical segmentation dataset [10]. As shown in Table a2, using box prompts, GaRA-SAM showed notable improvements over SAM. These results indicate that GaRA-SAM maintains strong performance even in domains with very different visual characteristics.

C.2 Isolating the effect of dynamic (rank-1) component selection

In Table a3, we tested variants using only dynamic (rank-1) component selection within a unified rank space. Enabling the dynamic selection improves performance when using a unified rank space with the small maximum rank (i.e., 16), highlighting the benefit of adaptively choosing components

16

F-statistics

40

14

-log10(p-value)

p=0.05 threshold

12

30

-log10(p-value)

F-statistics

10

8

20

6

4

10

0

4

6

8

10

Block

per input. However, using a unified rank space with the large maximum rank (i.e., 256) did not lead to better results. We attribute this to the limited diversity in active components when using a unified space, which can restrict the ability to adjust representational capacities according to each degraded input, thus hindering input-adaptive modulation. It motivated our design to split the rank space, which led to improved performance.

C.3 Further finetuning GaRA-SAM on a real-world dataset

GaRA-SAM supports training on real-world corrupted datasets without requiring clean counterparts. To validate this capability, we further fine-tune GaRA-SAM on real-world degradation datasets, namely BDD-100K and LIS. In contrast, RobustSAM is not designed for unpaired training and cannot be directly applied in this setting. Nevertheless, for a fair comparison, we fine-tune RobustSAM on the same datasets using only segmentation loss, despite the lack of paired supervision. We evaluate both models on BDD+LIS (seen during fine-tuning) and ACDC, which serves as a real-world dataset not seen during fine-tuning. As shown in Table a4, this additional finetuning leads to significant performance improvements across all benchmarks. This result highlights that GaRA-SAM not only generalizes well in a zero-shot setting, but also scales effectively with real-world corrupted inputs, whereas RobustSAM lacks inherent support for this setting and is only included for comparative purposes.

C.4 Quantitative results

We report additional experimental results in Tables a5,a6,a7,a8,a9, including evaluations using the pixel accuracy (PA) metric. PA values are consistently reported across all benchmark settings using both point and box prompts.

C.5 Qualitative results

We present additional qualitative comparisons in Fig. a3, showing the segmentation results of SAM, HQ-SAM, RobustSAM, and our GaRA-SAM under a variety of degraded inputs. The top section illustrates results under synthetic corruptions ( e.g. , motion blur, snow, noise), while the bottom section includes real-world challenges such as low-light and adverse weather conditions. Compared to other baselines, GaRA-SAM consistently demonstrates more robust and accurate segmentation masks, especially in cases where the standard SAM or HQ-SAM fails to localize object boundaries. These results highlight GaRA-SAM's strong generalization capabilities across both synthetic and naturally occurring corruptions.

Input

SAM

HQ-SAM

RobustSAM

GaRA-SAM (Ours)

GT

Synthetic

Real

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper GaRA-SAM introduces a novel approach to enhance the robustness of the Segment Anything Model (SAM) under diverse input degradations, such as noise, blur, and adverse weather conditions. The core innovation is the Gated-Rank Adaptation (GaRA) module, which integrates lightweight, input-adaptive adapters into SAM’s frozen architecture. Unlike prior methods like RobustSAM, which require paired clean-corrupted images for training, GaRA dynamically adjusts the effective rank of its adapters via a hierarchical gating mechanism. This enables fine-grained adaptation to specific corruptions while preserving SAM’s zero-shot generalization capabilities.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nComprehensive benchmarking: GaRA-SAM is evaluated across six datasets, including synthetic (LVIS, MSRA-10K) and real-world (BDD+LIS, ACDC) corruptions. It consistently outperforms baselines like RobustSAM, HQ-SAM, and restoration-based methods (AirNet, URIE) by 3.4–21.3% IoU.\n\nMechanistic validation: Ablation studies dissect the impact of hierarchical gating (removing it reduces performance by 3.6% IoU) and rank-space selection (unified rank spaces underperform by 1.4–4.3% IoU)\n\nReal-world applicability: Training on unpaired real data (BDD+LIS) closes the sim-to-real gap, achieving 89.7% IoU on BDD+LIS and 88.8% IoU on ACDC\n\nWeaknesses:\n\nStatistical reporting: Results are reported from single runs without error bars or significance tests, limiting insights into variance\n\nQUESTIONS:\n1. While GaRA-SAM demonstrates clear improvements over RobustSAM across synthetic and real-world benchmarks, the paper does not address whether the performance gaps stem solely from input-adaptive rank modulation or other architectural differences (e.g., hierarchical gating vs. RobustSAM’s anti-degradation modules). A controlled ablation isolating the impact of dynamic rank selection versus gating mechanisms would strengthen causal claims.\n\n2. [minor] GaRA-SAM’s training on BDD+LIS (autonomous driving data) raises questions about its applicability to non-driving scenarios (e.g., medical or aerial imagery). The paper does not validate cross-domain robustness beyond adverse-weather use cases. The reviewer understands the difficulty of the experiments.\n\nLIMITATIONS:\nSee Questions.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the robustness problem of the Segment Anything Model (SAM) under various image degradations. The authors propose Gated-Rank Adaptation (GaRA), an adaptive and lightweight modification based on Low-Rank Adaptation (LoRA). GaRA dynamically adjusts the effective rank of the adapters integrated into the frozen layers of SAM according to the specific degradation type present in the input image. This gating mechanism adaptively selects between lower-rank and higher-rank components, enabling fine-grained, input-aware robustification without requiring paired clean-degraded training images. Experimental evaluations demonstrate state-of-the-art results on multiple robust segmentation benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1. The writing is clear and the presentation is well-structured.\n\n2. The proposed method is intuitive and straightforward.\n\n3. Experimental evaluation is comprehensive, and the method surpasses baseline approaches across multiple datasets.\n\n**Weaknesses:**\n\n1. My primary concern relates to methodological innovation. The proposed GaRA method employs a gating mechanism to dynamically activate low-rank or high-rank LoRA adapters. However, similar adaptive rank-selection concepts have already been extensively explored in related work such as NoRA, AdaLoRA, and ElaLoRA, raising questions regarding the novelty of this approach.\n\n2. The sequence of tables is somewhat confusing: Table 2 is presented first in Section 4.2, while Table 1 is referenced later in Section 4.3. This inconsistency affects readability and clarity.\n\n3. There seems to be an issue with the third row of Figure 4, as the input image appears missing or unclear. Clarification or correction is needed.\n\nQUESTIONS:\n1. Would fine-tuning the query, key, and value parameters directly with a low learning rate or replacing LoRA with traditional adapters [a] also enhance the model’s robustness similarly?\n\n> [a] Houlsby et al. *Parameter-efficient transfer learning for NLP*. ICML 2019.\n\n2. If LoRA were applied to fine-tune the decoder instead of the encoder, would this also result in improved robustness?\n\n3. Can GaRA be merged into the SAM weights to reduce model size further? The paper mentions (line 100) that GaRA selects and combines appropriate (rank-1) components dynamically during inference, suggesting it may not be mergeable into the original SAM model weights like other static LoRA methods.\n\n4. Will the code and trained models be fully open-sourced?\n\nLIMITATIONS:\nSee Weaknesses and Questions", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes to use gated-rank adaptation to robustify SAM (segment anything model) against image corruptions, while maintaining the inherent generalization capabilities of such models. The core technique proposed is to freeze the parameters of a pre-trained SAM model and to introduce gated-rank adaptation modules that learn to adjust the effective rank of its weight matrix based on the current input. The authors show that this technique maintains the generalization capabilities of the model while significantly improving robustness to several synthetic and real-world image corruptions — especially when trained directly on real-world corrupted data.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- Robustifying SAM models is crucial for real-world applications, as unexpected shifts in the data distribution due to image corruptions from manifold sources can significantly degrade segmentation performance. For example, for self-driving vehicles, adverse weather conditions, low light, sensor degradation, dirt, or lighting changes can significantly impact segmentation performance. The paper addresses this crucial problem.\n- The proposed solution is lightweight, allowing to reuse existing pre-trained SAM models. This saves significant cost and allows reusing existing models without retraining from scratch.\n- The proposed method can be trained on real-world corrupted data without paired clean examples. Compared to prior work, this allows avoiding the synthetic-to-real domain shift, leading to significantly improved robustness.\n- The paper is well presented and follows a logical structure. It includes ablations and experimental evidence for assumptions and intuitions presented.\n\n**Weaknesses**\n- A discussion of the reasons for this particular design of the GaRA modules would improve the paper. Why was this split between low and high rank adapters chosen? Why not just one of them? Or a more general approach that allows the gate to choose from any dimension? While the technical implementation is clear from section 3.3, the reasoning, as well as the advantages and drawbacks remain elusive.\n- The paper does not properly address its limitations. L294 has two sentences on it, but only raises a single point — the manual engineering of the GaRAs. It would be helpful to go into more details on why the authors chose this design compared to more general and flexible options. Additionally, other limitations should be addressed such as no comparison to training a SAM model with corrupted data from scratch, only computing single runs without std or statistical significance values, etc.\n- Some of the tables and figures are very crammed, which makes reading them difficult. For example, Table 6, 7, and 8, and Figures 5 and 6 barely have any separation.\n- The authors provide no code to evaluate the implementation or build upon their work.\n\nQUESTIONS:\n- Why do you use two GaRA gates for high- and low-dimensional spaces — instead of a single one that can freely adapt any rank?\n- L217: why r_L = 16 and r_H = 256? How were this values determined?\n- L222: can you please clarify what you mean by “seen datasets”? I assume this means performance of a held-out test set from the same data distribution as the training data — as opposed to testing on a distribution from a different dataset?\n- Table 8: Can you please provide more details on the performance evaluation? What types of GPUs do you refer to? How many are used for inference? How many fixed (I.e. non-learnable) parameters are there? And how much training time is required for fine-tuning with GaRA-SAM vs. RobustSAM in total (in e.g. GPU-hours)?\n\nLIMITATIONS:\nThe paper does not properly address its limitations. L294 has two sentences on it, but only raises a single point — the manual engineering of the GaRAs. It would be helpful to go into more details on why the authors chose this design compared to more general and flexible options. Additionally, other limitations should be addressed such as no comparison to training a SAM model with corrupted data from scratch, only computing single runs without std or statistical significance values, etc.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper \"GaRA-SAM: Robustifying Segment Anything Model with Gated-Rank Adaptation\" proposes GaRA to enhance SAM's robustness against input degradations. GaRA introduces lightweight adapters into SAM's intermediate layers, dynamically adjusting the effective rank of weight matrices based on input via a learned gating module. GaRA-SAM achieves state-of-the-art performance on robust segmentation benchmarks, significantly surpassing previous methods.\n\nSTRENGTHS AND WEAKNESSES:\n1.Strengths：\n(1) Innovative methodology: The introduction of GaRA, which dynamically adjusts the rank of adapters based on input, is a novel approach to enhancing model robustness. This method effectively addresses the limitations of fixed-rank LoRA and other existing robustification techniques, like attaching an existing image restoration module to the front of SAM.\n(2) Sota Results: GaRA-SAM achieves significant performance improvements on multiple robust segmentation benchmarks, particularly on real-world corrupted datasets like ACDC. This demonstrates its practical effectiveness and superiority over previous methods.\n(3) Practicality and Efficiency: GaRA-SAM maintains parameter efficiency and adheres to standard training protocols, making it easy to integrate into existing workflows. It also enables training on real-world degraded images without requiring clean references, which is a major advantage for practical applications.\n(4) GaRA is easy to understand and follow.\n\n2.Weaknesses: \n(1) GaRA, as a robust image encoder, while achieving good performance, needs an extra 60% GPU Memory (from 3.36 GB of original SAM to 5.39GB of GaRA), which may be a heavy load for resource-constrained devices.\n\nQUESTIONS:\n1.GaRA needs an extra 60% GPU Memory (from 3.36 GB of original SAM to 5.39GB of GaRA)， and 343M extra LoRA params, which may be a heavy load for resource-constrained devices. I wonder if GaRA will outperform a bigger SAM with 1250M+343M params co-trained by clean pairs and degraded pairs, although I understand that this strategy holds significant practical value due to its flexibility.\n2. How can we supervise the training process to ensure GaRA chooses the best gating path?\n3. Impact of Rank Space Selection ablation in the Table.7, why choose rank 16 and 256 only? It is hoped that more experimental results on w/o gating ranks can be provided to demonstrate the necessity of space selection fully. Similarly, why are only these two levels selected for hierarchical gating, and why not go deeper into more levels?\n\nLIMITATIONS:\nThe article mentions the potential drawbacks of the maximum rank number", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}