Singular Value Fine-tuning: Few-shot Segmentation requires Few-parameters Fine-tuning
Anonymous Author(s)
AffiliationAddress email
Abstract
Freezing the pre-trained backbone has become a standard paradigm to avoid overfitting in few-shot segmentation.
In this paper, we rethink the paradigm and explorea new regime: fine-tuning a small part of parameters in the backbone .
We presenta solution to overcome the overfitting problem, leading to better model generalization on learning novel classes.
Our method decomposes backbone parameters intothree successive matrices via the Singular Value Decomposition (SVD), then onlyfine-tunes the singular values and keeps others frozen.
The above design allowsthe model to adjust feature representations on novel classes while maintainingsemantic clues within the pre-trained backbone.
We evaluate our Singular Value
Fine-tuning (SVF) approach on various few-shot segmentation methods with different backbones.
We achieve state-of-the-art results on both Pascal-5 i and COCO-20 iacross 1-shot and 5-shot settings.
Hopefully, this simple baseline will encourageresearchers to rethink the role of backbone fine-tuning in few-shot settings.
Introduction
Benefiting from the large amounts of annotated data, deep learning has achieved noticeable improvements in the field of semantic segmentation [44, 17, 34].
In contrast, their performancesdramatically degrade when novel classes arrive or label data is insufficient.
Thus, few-shot segmentation (FSS) [22, 49] was proposed to address these challengs.
In FSS, one needs to segment novelclass objects in query images given only a few densely-annotated samples ( i e. , support images andsupport masks).
1 Due to the extremely limited data in FSS, over-fitting has become a critical problemthat needs to be carefully handled.
One feasible solution is to restrict the model’s learning capacity so that it can not overfit the smalldataset.
Most recent works [43, 47, 30, 33] follow this idea by freezing the pre-trained backbone.
Then, different feature fusion methods and prototypes are introduced to enhance the generalizationability.
Although this paradigm has achieved promising results, it is still suboptimal to directly adoptparameters pre-trained on image classification to image segmentation.
The semantic clues containedin the pre-trained backbone can be irrelated to objects shown in support images, bringing unexpectedobstacles to segmenting novel class objects in FSS.
In this paper, we rethink the paradigm of freezing the pre-trained backbone and show that finetuning a small part of parameters in the backbone is free from overfitting, leading to better modelgeneralization in learning novel classes.
Our method is illustrated in Figure 1(b).
First, to find such asmall part of parameters for fine-tuning, we decompose pre-trained parameters into three successive
[Figure]
QueryImage
SegmentationHead
Support Mask
F s
F q
SegmentationHead
Support Mask
F s
F q
SupportImage
QueryImage
Pre-trainedBackbone
U S V W
Freezing weight finetuning weight
SVD matrices via the Singular Value Decomposition (SVD).
Second, we then fine-tune the singular valuematrices and keep others frozen.
The above design, called Singular Value Fine-tuning (SVF) , followstwo principles: (i) maintaining rich semantic clues in the pre-trained backbone and (ii) adjustingfeature map representations when learning to segment novel classes.
We evaluate our SVF on two few-shot segmentation benchmarks, Pascal-5 i and COCO-20 i .
Extensiveexperiments show that SVF is invulnerable to overfitting and works well with various FSS methodsusing different backbones.
It is significantly better than the freezing backbone counterpart, leading tonew state-of-the-art results on both Pascal-
i
and COCO-
i .
Moreover, we provide quantitative andqualitative analyses on how singular values change during fine-tuning.
Results show that SVF helpsmodels focus more on the objects to be segmented instead of the noisy background.
Our experimentshighlight that proper backbone fine-tuning consistently outperforms backbone freezing on severalleading methods.
We hope our simple method will encourage researchers to rethink the role ofbackbone fine-tuning for few-shot segmentation.
Related Work2.1 Few-shot Segmentation
The purpose of few-shot segmentation is to segment the unseen class in query image with a fewdensely-annotated samples.
In this task, a semantically rich representation and a nice matchingapproach have a particularly large impact on the results.
Therefore, mainstream methods [43,40, 32, 5] focus on obtaining excellent prototypes from support images, and obtaining accuratesegmentation results by improving the quality of prototype features.
CANet [45], PFENet [35] and
PANet [37] filters class irrelevant information by global average pooling to obtain foreground andbackground prototypes.
ASGNet [16] pointed out that increasing the number of prototypes canfurther improve the segmentation results.
CyCTR
[47] believes that pixel-level features in supportimage are important for segmentation tasks, and proposes to use pixel-level prototypes to predictquery images.
On the other hand, some methods focus on designing better matching methods toimprove segmentation performance.
SG-One [49] uses cosine similarity to match prototype and queryfeature for segmentation results.
CANet [45] proposes an additive alignment module to iterativelyrefine the network output.
HSNet [23] exploits neighborhood consensus to disambiguate semanticsby analyzing patterns of local neighborhoods in matching tensors.
In addition to the above work,
BAM [15] utilizes the segmentation results of the base class to guide the generation of unseen classes,and achieves SOTA results.
However, the above methods are all based on backbone freeze, andfreezing backbone not only reduces the representational ability of the model, but also does not fitdistribution to data better.
Unlike previous work, in this paper we focus on the prospect of fine-tuningbackbone in FSS.
Therefore, instead of proposing a new model, we adopt the classic PFENet [35]
[Figure]
, but also and BAM [15] as our baselines.
Our SVF enables these methods to further improve segmentationresults.
2.2 Backbone Fine-tuning
Fine-tuning backbone in downstream tasks has become a common approach in deep learning.
Theinitial breakthroughs in vision tasks [10, 13, 18] were achieved by fine-tuning classification networksbased ImageNet pre-trained weight, such as R-CNN [7] for detection and FCN [20] for segmentation.
However, the direct application of fine-tuing the entire backbone in few-shot scenarios will lead toover-fitting of the model [30].
Therefore, fine-tuning part parameters of the backbone in few-shotlearning may avoid model over-fitting.
P-Transfer [30] utilizes NAS to search parameters of backbonethat require fine-tune in few-shot classification tasks.
However, this method is very complicated andcannot be directly applied to small sample segmentation.
And some works [14] borrow the idea ofprompt [36, 31, 39] in NLP to fine-tuning part parameters in the visual transformer [6].
The abovemethods are proposed in a transformer-based model, but most few-shot segmentation models use
CNN-based backbones.
Applying prompt-based methods to various few-shot segmentation methodsmay need further adjustments.
Different from the above methods, our SVF borrows the commonlyused SVD [1] in model compression and constructs [4, 50] a novel part fine-tune method for few-shotsegmentation task.
In addition, some approaches [28, 26, 25] introduce highly constrained subset ofparameters to fine-tuning.
However, these methods are not applied on few-shot segmentation task.
From Freezing Backbone to Singular Value Fine-tuning
In this section, we start with the preliminaries on the few-shot segmentation (FSS) setting.
Then, werevisit the overfitting problem in FSS when fine-tuning the backbone in Section 3.1.
In Section 3.2, wepropose a novel Singular Value Fine-tuning (SVF) method for FSS instead of freezing the pre-trainedbackbone as proposed in previous methods.
Section 3.3 provides a discussion on the differencesbetween SVF and other fine-tuning methods.
FSS Setup.
Few-shot segmentation (FSS) aims to segment novel class objects given only a fewdensely-annotated samples.
In this task, datasets are split into the training set ( D train ) with baseclasses ( C train ) and the testing set ( D test ) with novel classes ( C test ), where C train ∩ C test = ∅ .
Following previous works, we adopt episode training.
Each episode consists of k support imagesand one query image to construct a k -shot segmentation task ( k = 1 or k = 5 in this paper).
Then,
FSS methods are trained with episodes to segment novel class objects in the query image given theknowledge of k support images and support masks.
3.1 Revisiting Model Overfitting in FSS
As presented in Section 1, model overfitting is a critical problem in extremely limited data scenarios(1-shot and 5-shot), especially when the model has large amounts of learnable parameters.
Wevalidate this problem and design experiments with a typical FSS method, PFENet [35].
Figure 2shows that as model training moves on, fine-tuning backbone leads to better performance on thetraining dataset while it does not improve results on the validation set.
It is a typical overfittingphenomenon.
In contrast, freezing backbone can achieve steady improvements on the validation set during training.
Therefore, existing methods in FSS turn to freezing the pre-trained backbone to avoidthe overfitting problem.
Although this strategy has achieved promising results, it is clear that directly adopting an ImageNetpre-trained backbone to image segmentation can be suboptimal.
One need to extract the most relatedsemantic clues within the backbone instead of involving too much noise coming from the irrelevantcategories learned from upstream tasks.
In light of this, we rethink the paradigm of freezing thepre-trained backbone and try to find a new solution to the overfitting problem.
3.2 Singular Value Fine-tuning
According to the analysis above, fine-tuning all parameters in the backbone can be unsatisfactory.
Onefeasible solution is to restrict the backbone’s learning capacity so that it is suitable for the few-shotcircumstance.
Instead of freezing the whole pre-trained backbone as in previous works, we considerexploring a new regime, which is only fine-tuning a small part of the parameter in the backbone .
[Table]
However, it is nontrivial to find such a smallpart of parameters to be fine-tuned in thebackbone.
Simply splitting the backbone’sparameters into learnable and freezing onesresults in negative results.
Table 5 and Table 6 show the inferior performances no matter splitted by layers or convolution types.
We attribute this to the adjustment of thebackbone, making the model biased towardsbase class objects shown in the training set,yet leads to worse model generalization insegmenting novel classes.
Figure 3 also provides evidences of the overfitting problem.
Given that the backbone is pre-trained on a large-scale dataset with a classification task, it containsrich semantic clues but it is suboptimal to adopt for a segmentation task directly.
We deliver twoprinciples for finding a small part of fine-tuning parameters in the backbone: (i) maintaining richsemantic clues in the pre-trained backbone and (ii) adjusting feature map representations whenlearning to segment novel classes.
To fulfill the above goal, we resort to model compression methods in this paper.
They are designed toapproximate the original pre-trained model with fewer parameters, and also follow the above twoprinciples.
Among these methods, low-rank decomposition is a common technique to achieve modelcompression.
It first splits the model weights into multiple subspaces and then compresses eachsubspace by shrinking its rank.
We follow this direction and decompose the backbone parameters intosubspaces via the Singular Value Decomposition (SVD).
However, we do not shrink subspaces’ rankssince our target is to find a small part of parameters to be fine-tuned instead of model compression.
In detail, for a convolution layer with C i input channels, C o output channels, and a kernel size of
K × K in the pre-trained backbone, we first fold its weight tensor W ∈ R C o × C
i × K × K into a matrix
W ′ ∈
R C o × C
i K 2 , then decompose the obtained matrix by applying SVD with full-rank in subspaces(rank R = min( C o , C
i K 2 ) ).
Thus,
[Equation]
T , where U ∈
[Equation]
R
[Equation]
S
[Equation]
R R
[Equation]
R
[Equation]
and V T
[Equation]
The obtained pair of matrices V T and U construct two new convolution layers, and S is a diagonalmatrix with singular values on the diagonal.
Then back to convolutions, the results of the Equation 1corresponds to three successive layers: a R × C
i × K × K convolution layer, a scaling layer, andfollowed by a C o × R × 1 × 1 convolution layer
(Algorithm 1 further provides a pseudo-code for
SVF).
We thus split each convolution layer in the pre-trained backbone into three functionalities:(i) decouple the semantic clues into a subspace with rank R , (ii) re-weight the semantic clues withsingular values for the given task, and (iii) project the re-weighted clues back to the original space.
Based on the interpretation above, we propose to fine-tune the scaling layer, which is Singular Value
[Table]
Fine-tuning (SVF).
SVF does not erase the semantic clues contained in the pre-trained backbone,but it re-weights the representations to help adjust the model for new segmentation tasks.
As SVFrestricts the learnable parameters to only singular values, which are extremely few ( 0 . 25% ) comparedwith the parameters in the whole backbone, SVF is less vulnerable to overfitting and shows bettergeneralization capacity in learning novel classes (shown in Table 5, Table 6, Figure 2, and Figure 3).
3.3 Discussion on Fine-tuning Methods
Fine-tuning pre-trained backbones is a promising way to achieve state-of-the-art results on downstream vision tasks.
Many fine-tuning methods have been introduced to transfer pre-trained backbone’s knowledge, such as full-model fine-tuning [10, 46], task-specific fine-tuning (freezing thebackbone)[9, 41], residual adapter [12, 2, 38], and bias tuning [48, 8].
We compare our SVF withthese methods.
As presented in previous sections, methods like full-model fine-tuning may not besuitable with extremely limited data, and task-specific fine-tuning can not provide adjustment tothe representations in the backbone for downstream tasks.
Moreover, methods like residual adapterand bias tuning need prior knowledge to model structure or weights.
SVF does not have the abovelimitations in the few-shot segmentation task.
Recently, a new fine-tuning method named Vision Prompt Tuning (VPT) [14] has been proposed tofine-tune vision transformers.
It introduces a small number of trainable parameters in the input spacewhile keeping the backbone frozen.
From this perspective, our SVF also introduces a small numberof trainable parameters but in the singular value space.
In SVF, the learned singular value diagonalmatrix S can be formulated as a product of a frozen matrix S frozen and a trainable matrix S trainable ,which is S = S frozen S trainable .
We give a detailed explanation of this perspective in the Appendix.
Experiments
We conduct experiments on Pascal-5 i [29] and COCO-20 i [24] to discuss fine-tuning approach in
FSS.
In this section, we first introduce the used representative method and implementation details.
Then we discuss the impact of different fine-tune methods on the FSS model, and finally verify theeffectiveness and versatility of the proposed fine-tune method.
Datasets.
Experiments are conducted on Pascal-5 i [29] and COCO-20 i [24].
Following the previouswork [23, 35, 29], we separate all classes in both datasets into 4 folds.
For each fold, Pascal-5 i [29]has 15 classes used for training and 5 classes for test, COCO-20 i [24] has 60 classes used for training [Table] [Table] [Table]
and 20 classes for test.
To verify the performance of the model, we randomly sample 1000 querysupport pairs in each fold.
Following the BAM [15], we remove images from training set containingnovel classes of test to prevent potential information leakage.
We give a detailed explanation of thissetting about train sets in the Appendix.
Methods.
To quickly verify the effectiveness of SVF, we propose a simple baseline method.
Then,two representative methods are used to verify the generality of SVF.
Next, we briefly introduce thesethree methods.
Meanwhile, for fair comparison, we unify the three methods into the same framework.
• Baseline : We replace FEM module on PFENet [35] with ASPP module to get the baseline
method.
The baseline only sets main loss, without adding auxiliary loss.
• PFENet [35] : As a classic method in FSS, it proposed the Prior Guided Feature Enrichment
Network, which has a huge impact on the subsequent FSS methods.
• BAM [15] : As the state-of-the-art method in FSS, it represents the most cutting-edge results
in FSS.
Dataset Tricks: The previously methods annotated novel classes in the training set as backgroundduring training step.
It become a common paradigm in few-shot segmentation.
However, based on
BAM [15], we found a novel dataset trick to improve the performance of FSS models.
It simplyremoves images from the training set that contain the novel classes.
For fair comparison with BAM,we use this trick in our experiments. However, we know that previously methods does not use thistrick. Therefore, we present the experimental results with and without dataset trick in Table 1, andmore detailed fair comparison results in Appendix. Here, we hope that researchers can make faircomparison under the same setting.
20406080 1 21 41 61 81 101 121 141 161m I
o
U % epoch
Val Train
Trend
[Figure]
(Val) 20406080 1 21 41 61 81 101 121 141 161m I
o
U % epoch
ValTrain Trend line (Val) 20406080 1 21 41 61 81 101 121 141 161m I o U % epoch
Val Train
Trend line(Val)
[Figure]
[Figure]
) (c) (d)
[Figure]
[Figure]
[Table]
[Table] , 3,- - 67.14 (+2 . 07) Implementation details. In our experiments, we use VGG-16 [3] and ResNet-50 [11] as thebackbone network and initialize it with ImageNet [27] pre-trained weights.
Due to the particularityof BAM, we use the initialization weights provided by author, and SVF does not finetuning basebranch in BAM.
We use SGD optimizer with cosine Learning rate decay [21], the learning rate0 .
015 and the random seed 321 when fine-tuning backbone.
We keep original settings for the modelwithout fine-tuning.
All models are trained 200 epochs on Pascal-5 i with batch size 8 and trained 50epochs on COCO-20 i with batch size 8.
Image is resized to 473 × 473 on Pascal-5 i and 641 × 641on COCO-20 i .
Following [19, 41, 42] , we adopt the mean intersection over union (mIoU) andforeground-background IoU (FB-IoU) as our evaluation metric.
Since the middle-level features andhigh-level features are used in all FSS model, we set the SVF to fine-tuning the parameters of layers2, 3, and 4 only.
All model runs on four NVIDIA A100 GPUs.
4.2 Comparison with State-of-the-Art
In this section, the effectiveness of SVF is validated on three most representative methods.
For faircomparison, we rerun all methods with unified framework.
Then, we compare different methods withsingular value fine-tuning and freeze backbone.
The experimental results on Pascal-5 i are in Table 1.
It can be seen that the performance of different methods has been significantly improved after SVF.
[-) (d) 20406080 1 21 41 61 81 101 121 141 
-]
When the backbone is VGG-16, the SOTA method BAM improves miou by 0 .
46 and 0 .
35 in 1-shotand 5-shot respectively after singular value fine-tuning.
However, when the backbone is ResNet-50,
SVF improves BAM by 1 . 14 and 1 .
37 mIoU on 1-shot and 5-shot, respectively.
It shows that SVFbring better performance in deeper backbone.
Meanwhile, Table 2 shows the effectiveness of SVF onmore complex dataset COCO-20 i .
Exspecially, SVF improves the performance of BAM by 2 . 24 and2 .
71 mIoU on 1-shot and 5-shot.
Furthermore, Table 3 shows the comparison results with FB-IoU on Pascal-5 i .
Our SVF can also improve the performance of model.
This experiment proves that SVFnot only achieve state-of-the-art results, but also is a general method in FSS.
In addition, the results(without † ) in Table 1 prove that the dataset trick can indeed improve the performance of the model.
It also shows that whether or not the dataset tricks is used does not affect the effectiveness of SVF.
And we give more comparative results in the Appendix.
4.3 Ablation Study
To verify the effectiveness of SVF, we conduct a series of ablation study in this section.
We usethe baseline method to conduct ablation study on Pascal-5 i 1-shot setting with ResNet-50 as thebackbone network.
Furthermore, we give the ablation study about hyperparameter in the Appendix.
Batch Normalization (BN): In Table 4, we test the effect of BN on SVF.
In the case of only finetuning the BN layer, the baseline will greatly reduce the performance.
Next, we test SVF (fine-tunesubspace S ) on the baseline without fine-tuning BN.
The results show that SVF achieves the bestperformance.
Finally, we test the performance of the baseline method when fine-tuning subspace
S and BN simultaneously.
The results show that fine-tuning parameters of BN layer can causeperformance of baseline to degrade.
Therefore, we freeze the parameters of BN layer when using
SVF.
Traditional fine-tune methods: In this part, we conduct experiments to verify the impact oftraditional fine-tuning methods on the FSS model.
Traditional fine-tuning methods can be dividedinto fully fine-tune and part fine-tune.
The fully fine-tune method means to fine-tuning all theparameters in the backbone.
The part fine-tune methods means to fine-tuning part parameters inthe backbone, which includes layer-based and convolution-based fine-tune methods.
In table 5, weconduct quantitative experiments with fully and layer-based fine-tune on baseline method.
Theresults show that fully fine-tune brings negative results to baseline method.
Meanwhile, we find thatthe negative results of fully fine-tuning method are mitigated as the number of fine-tuning layersis reduced.
However, these methods do not have a positive impact on the baseline.
In table 6, weconduct quantitative experiments on convolution-based fine-tune methods.
For fair comparison, weonly fine-tuning convolutions of 2, 3 and 4 layers.
The results show that only fine-tuning 3 × 3convolution or 1 × 1 convolution can further improve the performance of layer-based fine-tune study of SVF fine-tuning different subspace on Pascal-5 i .
[Table] [Table]
methods.
It show that traditional fine-tune methods cannot bring positive results.
However, SVFbrings positive results to the baseline method.
The success of SVF proves that traditional fine-tunemethod destroys the rich semantic clues in the pre-trained backbone.
In Figure 3, we compare themIoU curves of the training and test sets in different fine-tune methods.
The results show that partfine-tune method also produces the over-fitting problem.
It proves that disrupting the rich semanticcues in pre-trained backbone will lead to model over-fitting, reducing model generalization.
However,
SVF solves the over-fitting problem without destroying semantic clues in pre-trained weight.
And, itbrings a new perspective for fine-tuning backbone.
Fine-tuning which subspace: To verify the influence of different subspaces on SVF, we conductexperiments on the subspace after SVD decomposition.
The results are shown in Table 7.
We findthat only fine-tuning S subspace brings positive results.
Either fine-tuning the U or V subspacereturns negative results.
It shows that U and V contain rich semantic information in pre-trainedweight after SVF.
In other words, directly changing the feature distribution of the U or V subspacereduces the generalization ability of the model.
To verify the above point, we test the performanceof fine-tuning different subspace combinations.
The results confirm that changing the distributionof U or V spaces brings negative results.
The subspace S represents the weight distribution ofdifferent semantic cues.
Therefore, fine-tuning the subspace S does not change the semantic cues ofpre-trained weights.
Meanwhile, adjusting the weights of different semantic cues enables model tobetter perform downstream tasks.
Fine-tuning which layers: Since the FSS model directly uses feature maps of layers 2, 3, and 4, weinitially fine-tuning the subspace S of layers 2, 3 and 4.
However, this setting is unreasonable.
Toverify which layer S have a greater impact on baseline, we conducted experiments on SVF underdifferent layer combinations.
The results are shown in Table 8.
It can be seen that fine-tuninglayers 3 and 4 achieves the best performance, while only fine-tuning the layer 4 achieves the lowestperformance.
It shows that semantic clues in layer3 are the most important for FSS.
Next we discussthe reasons why SVF can achieve better performance by visualizing semantic cues in layer3.
4.4 Discussion
on Why SVF Works
[Figure]
The larger singular value in subspace S , themore important semantic cues in subspace
U and V .
We first focus on the changesof singular values during fine-tuning basedon the initial distribution of S .
In Figure 4,we visualize the variation of Top-30 singularvalues in pre-trained weights.
It can be seenthat the singular values of either 1 × 1 or3 × 3 convolution change dramatically afterfine-tuning.
Next, we visualize the semanticcues of subspace U with the largest variation in singular values.
The results are shown in Figure 5 and
Figure 6.
We only visualize the semantic cues where the singular value grows and decreases the mostfor a better view.
Notice that semantic cues of decreasing singular values tend to focus on backgroundregions.
The semantic cues of increasing singular values always focus on foreground regions.
Thebackground-focused semantic cues in pre-trained backbone will damage the performance of FSSmodel.
Since the original distribution of semantic cues in pre-trained backbone is not suitable fordownstream tasks, SVF brings positive results to FSS model by increasing the weight of foregroundcues and reducing the weight of background cues.
It is also important to keep the semantic cuesunchanged during fine-tuning.
Overall, dynamically adjusting the weight of each cue withoutchanging the semantic representation is the key to the success of SVF.
4.5 Broader Impact
In this paper, we prove that freeze backbone is not the only paradigm in few-shot segmentation,fine-tune backbone is feasible.
Meanwhile, we explore a new mechanism to redistribute the weightsof different semantic cues without changing the semantic cues.
As a new perspective of few-shotsegmentation, it exposes the influence of pre-trained backbone on few-shot segmentation.
Moreover,this mechanism not only works on few-shot, but also may be effective when fine-tune very largepre-trained models.
This greatly reduces the cost of fine-tuning large models on downstream tasks.
4.6 Limitations
Although the above experiments demonstrate the power of SVF, it still has some limitations.
Forinstance, SVF introduces a small number of learning parameters, but the occupancy rate of memoryresources is high during training process.
Using SVF in ResNet-50 will occupy 16G video memoryper image in COCO-20 i 5-shot setting.
Furthermore, SVF increase a small amount of training timecompared with freeze backbone.
Conclusion
In this paper, we rethink the paradigm of freezing backbone in FSS and propose a new paradigm
Singular Value Fine-tuning (SVF) for fine-tuning backbone.
Firstly, SVF decompose pre-trainedparameters into three subspaces by SVD, and then only fine-tune the singular value.
Our SVFdynamically adjusts the weights of different semantic cues without changing the rich semantic cuesin pre-trained backbone.
We evaluate the effectiveness of SVF on two commonly used benchmarks,
Pascal-5 i and COCO-20 i .
Extensive experiments prove that SVF as a new perspective to avoidoverfitting and significantly improve the performance of various FSS methods.
As a new paradigm offinetune, we will extend it to a variety of vision tasks in the future. [Figure] [Figure]
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .
• Did you include the license to the code and datasets?
[No] The code and the data are proprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section 4.
(c) Did you discuss any potential negative societal impacts of your work?
[No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[N/A]
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] See Section 4.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] See Section 4.
(d) Did you include the total amount of compute and the type of resources used (e g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Section 4.
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[No]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[No]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[Yes]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
[Table]
you
[Table]
wage [Table] participants and
[Table]
total
amount spent on participant compensation
[N/A