IODA: Instance-Guided One-shot Domain Adaptation for Super-Resolution
Authors: Zaizuo Tang
Yu-Bin Yang
Venue: NeurIPS 2024 poster (2024)
Keywords: one-shot domain adaptation
super resolution
domain adaptation
Primary area: machine_vision
TL;DR: We propose an Instance-guided One-shot Domain Adaptation for Super-Resolution (IODA) to enable efficient domain adaptation with only a single unlabeled target domain LR image.

Abstract: The domain adaptation method effectively mitigates the negative impact of domain gaps on the performance of super-resolution (SR) networks through the guidance of numerous target domain low-resolution (LR) images. However, in real-world scenarios, the availability of target domain LR images is often limited, sometimes even to just one, which inevitably impairs the domain adaptation performance of SR networks. We propose Instance-guided One-shot Domain Adaptation for Super-Resolution (IODA) to enable efficient domain adaptation with only a single unlabeled target domain LR image. To address the limited diversity of the target domain distribution caused by a single target domain LR image, we propose an instance-guided target domain distribution expansion strategy. This strategy effectively expands the diversity of the target domain distribution by generating instance-specific features focused on different instances within the image. For SR tasks emphasizing texture details, we propose an image-guided domain adaptation method. Compared to existing methods that use text representation for domain difference, this method utilizes pixel-level representation with higher granularity, enabling efficient domain adaptation guidance for SR networks. Finally, we validate the effectiveness of IODA on multiple datasets and various network architectures, achieving satisfactory one-shot domain adaptation for SR networks. Our code is available at https://github.com/ZaizuoTang/IODA.

IODA: Instance-Guided One-shot Domain Adaptation for Super-Resolution
Zai-Zuo Tang, Yu-Bin Yang"
State Key Laboratory for Novel Software Technology Nanjing University
Nanjing, China
tangzz@smaill .ww..nu...c...m....m en, yangyubin(nju. edu. cn
Abstract
The domain adaptation method effectively mitigates the negative impact of domain gaps the performance of super-resolution (SR) networks through the guidance numerous target domain low-resolution (LR) images. However, in real-world scenarios, the availability of target domain LR images is often limited, sometimes even to just one, which inevitably impairs the domain adaptation performance of SR networks. We propose Instance-guu guidedd One-shot Domain Adaptation for Super-Resolution (IODA) to enable efficient domain adaptation with only a single unlabeled target domain LR image. To address the limited diversity of the target domain distribution caused by a single target domain LR image, we propose instance-guider target domain distribution expansion strategy. This strategy effectively expands the diversity of the target domain distribution by generating instance-specific features focused different instances within the image. For SR tasks emphasizing texture details, we propose an image-guided domain adaptation method. Compared to existing methods that text representation for domain difference, this method utilizes pixel-level representation with higher granularity, enabling efficient domain adaptation guidance for SR networks. Finally, we validate the effectiveness of IODA on multiple datasets and various network architectures, achieving satisfactory one-shot domain adaptation for SR networks. Our code available at https://ithub.com//aiin hotp://...oom/aa////////0/O000 Tang/IODA.
Introduction
Performance improvement by IODA
Original network
Acd IODA
Super-resolution (SR) has significant practical value in various fields, including remote sens- ing imagery
27.822
27.45 27.461
27-477
4 5 the restoration of old films [6] [7] and even as a preprocessing strat- egy enhance downstream tasks such as ob-
3
25.916
26
24.824
24
ject detection [
[ [1Q] and image segmentation However, SR also faces domain gap issues, where the performance of networks significantly declines when the distribution of test low-resolution (LR) images differs from that of the training LR images [14]. Therefore, do- main adaptation methods for SR [14 15] have emerged. These methods represent training LR images the source domain and test LR images
SAFMIN
SRFormer
Figure Performance improvement by IODA. The performance of the real-world pre-trained weights (from DF2K dataset (0) provided in the original paper serves the baseline on the real- world dataset R (2].
"corresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

as the target domain, then use generative adversarial networks to align the two domains during adaptation training, mitigating the negative impact of domain gaps on network performance. (Refer to Appendix A.2 for related work on SR network.)
However, the methods mentioned above require large number of target domain LR images for training the generative adversarial network alignment. In some special scenarios, such as photos taken by users or uploaded to the internet, users often provide only a small amount even a single LR image for SR, rendering the aforementioned methods inapplicable.
In other tasks such as image generation [16
18 19], semantic segmentation 20 and object
detection [2I], one/zero-sho domain adaptation methods have emerged (collectively referred to as text-guided domain adaptation, Appendix [B) These methods use text descriptions to guide domain adaptation, inputting the text descriptions of the source and target domains into the CLIP model [22] compute the text domain difference, namely the text cross-domain direction vector. Subsequently, the predicted target domain images and source domain images are input into the CLIP model to generate the image cross-domain direction vector. By constraining the target domain generation model to align the image cross-domain direction vector with the text cross-domain direction vector, these methods achieve domain adaptation with one/zero-shot learning. (Refer Appendix A. for related work on one/zero-shot domain adaptation.)
However, when it comes to the SR task, onc/zero-shol domain adaptation methods have not yet been explored. If existing one/zero-shot domain adaptation methods are directly applied to the SR task, they will encounter the following issues:
Text descriptions lack fine granularity, which prevents accurate and comprehensive represen- tation of domain differences in image texture details, thereby failing to provide effective guidance for domain adaptation in the SR task.
The diversity of the target domain distribution is limited. Existing one/zero-shot domain adaptation methods typically sample latent vectors from random noise for subsequent image generation, enriching the target domain distribution through multiple samplings. However, for the SR task, the target domain consists of only single LR image, resulting in a limited diversity of target domain distribution, which increases the risk of pattern collapse during training.
To address the first issue, this paper proposes an image-guided domain adaptation method that transforms the existing text-level description of domain differences between source and target domains into a pixel-level description of domain differences in LR images. This approach achieves a high level of granularity in representing image texture details, effectively guiding domain adaptation for SR networks. Regarding the second issue, we propose an restance--uider target domain distribution expansion strategy. By enabling Alpha-CLIP 23 to generate instance-specific features, the diversity of the target domain feature distribution is effectively expanded through the diversity of instances in the image.
To the best of our knowledge, this is the first work applying a one-shot domain adaptation method in the SR task. Our main contributions can be summarized follows:
We propose an image-guided domain adaptation method to address the issue of incomplete image texture detail representation in existing text-guidee ext--guied domain adaptation methods.
We propose an iettacce-gudeed target domain distribution expansion strategy to address the issue of limited target domain distribution diversity caused by one-shot scenarios.
We introduce novel one-shot domain adaptation method for the SR task. This method utilizes a single LR image from the target domain, effectively alleviating the negative impact of domain gaps the performance of SR networks.
We propose a plug-and-play domain adaptation method for SR task, which demonstrates good robustness for different data distributions and network architectures.

---

2 Proposed method
2.1 Problem setting
Ji=l+ target domain dataset D +LR get H [. Ii=1 ===1 a source domain SR network Mg trained solely on the source domain dataset, and a target domain SR network Mr initialized by Ms. The problem addressed in this paper is to adjust the network Mr (one-shot domain adaptation) using only single LR image LR Ligearget from the target domain dataset Dl'arget (without a corresponding HR image H as label), aiming achieve better performance of the adjusted network Mr the target domain dataset DTarget compared to the source domain network Ms. Notably, the target domain HR images H Rt arget are used only for performance testing after the domain adaptation training is complete and are not visible to the network during domain adaptation training.
Given source domain dataset DSurcee
{ (Luree HR RS N,
2.2 Overview
LR Source
+
Loss
Ms
Stage one
SRSuucce
HR
LR Source
SR
Ms
Train
Weight
LR
L.trection
cross-domain direction vector
direction vector
Frozen
Weight
Target
SR
SRh
Target
My
Stage
Figure 2: Overall framework
As shown in Figure IODA is primarily divided into two stages. In the first stage, the SR network Ms is pre-trained using the source domain dataset Souree, In the second stage, domain adaptation performed the SR network. Firstly, the weight of the pre-trained SR network Ms is frozen and used to initialize the target domain SR network Mr. Subsequently. using LR images from the source domain Source and target domain LR as well as SR images from the source domain S Source and target domain SR.Target we compute and L.R cross-domain direction vectors in the Alpha-CLIP space. The Ldirection loss is calculated align these two cross-domain direction vectors, enabling domain adaptation training for the target domain SR network My. The focus of this paper is the second stage: domain adaptation of the SR network, corresponding to Sections Subsequent sections will not delve into the details of the first stage, which aligns with the
2.3 2.4
traming methods employed in the original papers [24 25 26
2.3
Image-guided domain adaptation
Existing text-guided domain adaptation methods seek the text cross-domain direction vector by comparing the domain difference between the source domain text description and the target domain text description "cct""and"dog" the domain differences in the texture details between the source and target domains, such as the orange short fur of cat versus the black long fur of dog, the pink nose of cat versus the black nose of a dog. In the SR task, texture details are extremely critical. The SR network needs to finely reconstruct texture details such as fur, eyes, performance metric is also calculated at the pixel level. Therefore, existing text-guided domain adaptation methods are not suitable for SR tasks (Table We aim develop a domain difference representation method with higher granularity.
shown in Appendix However, text description cannot capture
nose. Moreover, the commonly used PSNR
3

---

LR Source
Ms
SR SR
E
E
LR
Sounce
Clip Space
ALR
ASR
LR Targer
SR
Srgge
E
E
LR
L
Target 1-th
M
SR i
Target
Figure 3: Image-guided domain adaptation method. This figure depicts the second stage of domain adaptation. My and Mr represent the source domain SR network and the target domain SR network, respectively.
When it comes to representing texture details, images superior to text because they can depict texture detail features at the pixel level. Compared to the network's 's perception of differences between two words, the differences between images more fine-grained and contain more detailed features. Additionally, the SR processing does not alter the image's content information. The cross-domain direction from the source domain the target domain in LR images should also be followed in SR images (aligned with it). Specifically, in the source domain to a dog image in the target domain, then after image SR processing, this transformation rule should also be followed: transitioning from cat image in the source domain dog image in the target domain. Therefore, we propose image-guided domain adaptation method (Figure 3j, which utilizes the LR images from the source and target domains to determine the domain differences. This method has pixel-level domain difference representation, providing higher granularity and more efficient domain adaptation guidance for SR network.
the LR image space, if transitioning from a cat image
As shown in Figure 3] image-guided domain adaptation directly uses the CLIP model to learn the LR image cross-domain direction vector ALR between the source domain LR image Souree and the target domain LR image LIR Simultancously. computes the SR image cross-domain direction vector ASR between the source domain SR image S SRooeeee and the target domain SR image SR.'arget By imposing collinearity constraints on these two cross-domain direction vectors, enables domain adaptation training for the target domain SR network,
ALR (L Target (L 7-th
ASR (LR Targett
E3 (Ms (MRSSSoorree)
(2)
ALR ASR
Ldirection
ALR [ASR[
E, represents the image encoder of CLIP 22
2.4 Instance-guided target domain distribution expansion strategy
The onc/zero-shot domain adaptation method of the generation network can construct the diversity distribution of the target domain (latent vectors) by multiple sampling from random noise (Appendix B}. However, for the SR task, it is not possible to sample from random noise, as there is only one target domain LR image available. Although randomly cropping subregions from single target domain LR image can slightly increase the diversity the target domain distribution (as shown in Figure 4a). the diversity of the target domain distribution is still limited. These limitations increase the risk of pattern collapse during domain adaptation training, negatively impacting the efficiency of domain adaptation.

---

-10
Certer (CL,2
Eod. SanAdor Cun e MALJA (CLA
Bluec Rardorm Celo Occkedon (CLIF)
110
(a)
(b)
(c)
Figure 4: Feature distribution comparison. Inputting a single target domain LR image into the image encoders of the CLIP model (a. b) and the Alpha-CLIP model (c), visualizing the output features after dimension reduction using T-SNE (27] (repeated 1000 times). It's worth noting that the more dispersed the scatter plot distribution, the more diverse the target domain feature distribution.
Region- range Mask
Region- range Mask
Full one mask
Full one mask
SR Source
Soureeee
Alpha
Alpha
Occlusion Mask
Add
CLIP
CLIP
Randem, Occlusion Mask
directdion
Alpha
Alpha
CLIP
CLIP
Masksalecse
Occlusion Mask
Up sampling
Occlusion Mask
LR Terect
concatenate
Perget
SAM
Maskuu
Figure 5: Instance-guided guided target domain distribution expansion strategy. The source domain and target domain SR images Souveee SR T'arget are generated from the corresponding LR images (b-th)
SRuoee LR through their respective domain-specific SR networks My and MT.
we intend to introduce occlusion masks to randomly mask the cropped target domain LR images and use the masked LR images for domain adaptation training the target domain SR network. This method enhances the diversity of the target domain distribution through the randomness of the occlusion masks (as shown Figure 4b).
Inspired by MAE 288
Although occlusion masks increase the diversity of target domain distribution to some extent, we aim further expand the extent of this increasement. Recently, Alpha-CLa CLIP 23 (as shown in Appendix C has received widespread attention for its ability to enhance CLIP [2 by incorporating specified attention regions. By providing Alpha-CLIP with region-range mask, network can focus more on the specified area. Motivated by this, we intend to provide Alpha-CI.IP with different region-range masks each time, enabling to generate Alpha-CLIP spatial features focused on different areas the image, thereby expanding the diversity of the target domain distribution.

---

For the generation strategy of region- rangee masks, benefiting from the powerful performance of the SAM model (2). which has good generalization and can accurately segment each instance in arbitrary scene images. SAM is used to generate region-range masks, corresponding to each instance in the image. Specifically, we use region-range masks generated by SAM to enable Alpha-CI.IP to focus on different instances when generating multiple Upha-CLIP spatial features (instance-specin features). For example, in Figure features could focus on the leaves, the first bird, the second bird, and so on. In addition, compared to naively randomly generating rectangular region masks, instance-specific masks implicitly convey semantic information, exhibiting better domain adaptation efficiency (Table 3). As shown in Figure 4c the introduction of Alpha- CLIP equipped with region-rang mask further enhances the diversity of the target domain distribution.
As shown in Figure 5] we replace the CLIP image encoder with the Alpha- image encoder. Firstly, we use the SAM model to perform instance segmentation on the target domain LR image, obtaining a collection of instance masks, denoted as MaskALL E {mask;) N. Randomly select a certain number of masks from the mask collection to concatenate and generate region-range mask M Maskselect- The target domain LR images with added occlusion mask and region-range mask M Maskseleet are input into the image encoder of Alpha-CLIP to generate Alpha-CLIP spatial features. For the region-range mask the target domain SR image, to maintain consistency with the focused region of the target domain LR image, the upsampled region-range mask is used for SR image. For the source domain HR and LR images H RSouree SSurce, due to their abundant quantity, there is no need to focus on individual instances in the image with Alpha-CLIP Therefore, the region-range mask covering the entire image is used as input.
3 Experiments
3.1 Experiment details
The experiments validate the effectiveness of the proposed IODA method using the RealSR (2]. DF2K [. GTA 30, Cityscapes [1, and ACDC [32 datasets. Specifically, the GTA, Cityscapes, and ACDC datasets utilize downsampling methods from BasicSR (63 to generate paired LR images. For pre-trained weights, the original papers provided the real-world pre-trained weights of the SR networks (HAT [24]. SRFormer (25) SAFMN (26)) on the DF2K dataset. For other datasets like GTA and Cityscapes, we conduct pre-training on these datasets using the provided source code (as explained in the table notes). For the experimental devices, 2080ti GPU is utilized for domain adaptation training of the SFAMN and SRFormer networks. Due to the larger memory demands of the HAT network, a V 100 GPU is employed as the training device. For experimental data selection, we select a single LR image from the target domain dataset for network one-shot domain adaptation training. The entire target domain dataset is used the test set for evaluating network performance. is worth noting that to mitigate the uncertainty in network performance caused by random sample selection, repeat the IODA-related experiments five times. Each time, different single sample is selected from the target domain dataset for adaptation training, and the results are presented as mean and variance.
Table 1: Effectiveness validation of IODA. Source domain dataset: DF2K M. Target domain dataset:
RealSR Canon 2 Network architecture: SAFMN 26).
Method
PSNR
SSIM
Baseline
24.824
0.744
Image-guided
27.014 + 0.185
0.773 + 2.379c-04
Image-guided Diversity strategy (IODA)
27.461 + 0.008
0.790 + 9.865e-06
3.2
Ablation experiment
3.2.1
Effectiveness ablation experiments for each module of IODA
To validate the effectiveness of the proposed modules, ablation experiments were conducted for each module. The network pre-trained on the source domain dataset was tested on the target domain dataset, and its test results were used as the baseline.

---

As shown in Table 1 with the introduction of image-guided domain adaptation (onage-guided)) guiding the SR network through the cross-domain direction vector between LR images in the source and target domains resulted in improvements of 2. in PSNR and 0.029 in SSIM performance metrics. Furthermore, with the introduction of the instance-guided target domain distribution expansion strategy (Image-guided & Diversity strategy), the diversity of the target domain distribution was further expanded, improving network performance with an increase in PSNR by 0.447 and SSIM by 0.017. Additionally, as shown in Appendix Figure 8 and Figure g our proposed method effectively alleviates the negative impact domain gaps on network performance.
Table 2: Ablation experiments on image-guided domain adaptation, using the DF2K [ dataset as the source domain dataset and RealSR ___n_n the target domain dataset, with the SAFMN network architecture [26]. The performance the pre-trained network (pre-trained on the source domain) on the target domain is represented as the baseline. Indicates that the corresponding domain adaptive method is ineffective for SR task.
Method
PSNR
SSIM
Baseline
24.824
0.744
Text Guided
Real HR
27.157 + 0.152
0.787 + 1.988 x 10
Image-Guided & SR (IODA)
27.461 + 0.008
0.790 + 9.865 x 10
3.2.2 Ablation experiments on image-guided domain adaptation
As shown in Table 2 we validated a text-guided domain adaptation method (Text Guided) similar to [2 In this experiments, for a clearer and more consistent representation domain differences, we employed domain adaptation from GTA (game scenes) to Cityscapes (real scenes), with text descriptions "driving in a game" " and "driving" (same as (20). However, the collinearity constraints from text not suitable for tasks like SR that require high-detail fidelity. They are ineffective in expressing cross-domain directions, thus unable to achieve domain adaptation for SR.
Furthermore, during domain adaptation, replacing the SR images SR generated by the source domain SR network with real HR images H Source was performed to verify whether more accurate HR images would provide greater performance improvements (Real HR). However, compared to using predicted generated SR images (Image-Guides
SR), the domain adaptation network using real HR images exhibited a decrease in PSNR performance by 0.304. This indicates that using real HR images causes the target domain network lose information from the source domain network during domain adaptation, lacking the explicit constraints of the source domain network. Besides retaining some source domain information through target domain weight initialization, no information about the source domain network is acquired during domain adaptation training. Using SRorre generated by source domain network can implement explicit constraints the source domain network on the target domain network, demonstrating better domain adaptation efficiency.
Table 3: Ablation experiments diversity strategies, using the DF2K ①1 dataset the source domain dataset and RealSR ___noon 2 as the target domain dataset, with the SAFMN network architecture 266. The baseline refers to the domain adaptation performance after incorporating image-guided domain adaptation without any diversity enhancement.
Method
PSNR
SSIM
Baseline
27.014 + 0.185
0.773 + 2.379 x 10
Occlusion Mask
27.270 0.049
0.790 + 1.865 x 10-5
Region-range Mask -G
27.259 + 0.084
0.788 + 4.396 x 10-5
Region-range Mask -B
27.321 + 0.021
0.791 + 1.450 x 10
Region-range Mask -I
27.384
0.008
0.788 + 1.897 x 10
Occlusion Mask Region-range Mask (IODA)
27.461
+ 0.008
0.790 + 9.865 10-6
7

---

3.2.3 Ablation experiments on diversity strategies
As shown in Table 3 we also conducted extensive ablation experiments in the diversity strategy. Inspired by MAE [28] we directly added random occlusion masks to the images (Occlusion Mask). Furthermore, we conducted ablation experiments on the shape of region-range mask in the Alpha- CLIP network, including random grid region-range mask (Region-raie rangee Mask -G), single random- sized regie mask (Region-range Mask -B), and instance region-range masks (Region-range Mask -I). From Table 3 it can be observed that instance region-rang, masks implicitly convey semantic information, leading to better domain adaptation performance compared randomly generated region-range masks. Finally, we combined the instance region-range masks with random occlusion masking (Occlution Mask Region-range Mask -I). From Figure 4 and Table 3] can be observed that the proposed method effectively improved the diversity of target domain distribution and significantly enhanced the performance of the network the target domain dataset, with a PSNR improvement of 0.447.
Table Robustness experiments on different source and target domain datasets with the pre-trained network SAFMN [26]. The performance of the pre-trained network (pre-trained on the source domain) on the target domain is represented the baseline. - signifies the domain adaptation from the source domain to the target domain.
Method
Task
PSNR
SSIM
DF2K + RealSR_Canon
Baseline
24.824
0.744
Add IODA
27.461 + 0.008
0.790 9.865 x 10-6
DF2K + RealSR_Niker
Baseline
23.980
0.707
Add IODA
26.616
0.042
0.757 9.852 10-6
GTA + Cityscapes
Baseline
38.390
0,966
Add IODA
38.938
0.002
0.962 7.012
10-8
GTA + ACDC_rain
Baseline
31.854
0.897
Add IODA
31.919
= 9.891
10
0.894 + 8.560
10
GTA ACDC_ snow
Baseline
31.698
0.869
Add IODA
31.751
= 2.605
10
0.867 + 5.339 10-7
Cityscapes ACDC_rain
Baseline
28.550
0.879
Add IODA
30.396 + 0.112
0.879 + 1.279 x 10-
Cityscapes ACDC_ snow
Baseline
29.110
0.848
Add IODA
30.521 + 0.016
0.848 + 1.834 x 10-5
3.2.4 Ablation experiments on different datasets
To validate the effectiveness of the proposed method across different data domains, conducted experiments on various source and target domain datasets. From Table it can be observed that the proposed method exhibits good robustness different data distributions, showing varying degrees of performance improvement across multiple datasets.
3.2.5 Ablation experiments on different network architectures
To validate the effectiveness of the proposed method on different network architectures, as shown in T conducted experiments SAFMN (26. HAT [24), and SRFormer [25) network archi- tectures, respectively. was observed that IODA exhibited good robustness, showing performance improvements with one-shot domain adaptation across different network architectures.
3.3 Comparative experiment
In this section, we compared IODA with other methods (As shown in Table including Zero-shot SR [4 domain adaptation SR network (DASR [4]), self-implemented variant one-shot DASR, self-implemented text-guided domain adaptation methods for SR tasks (Style-GAN-NADA [19].

---

Table Robustness experiments on different networks architectures, with the source domain dataset: DF2K and target domain dataset: RealSR Canon. The performance of the pre- pre-trained network (pre- trained on the source domain) on the target domain is represented as the baseline. Fine-tune refers to the network utilizing all samples from the target domain H R prgget to fine-tune the pre-trained weights from the souce domain (Same as the original papers).
Method
Task
PSNR
SSIM
SAFMN
Baseline
24.824
0.744
Fine-tune
29.875
0.845
Add IODA
27.461
0.008
0.790 + 9.865 x 10-6
HAT 24
Baseline
27,477
0,8035
Fine-tune
29.989
0.851
Add IODA
27.821 + 0.008
0.812 + 1.118 x 10-6
Former
25
Baseline
25.946
0.7893
Fine-tune
29.899
0.848
Add IODA
27.657 0.045
0.810 + 1.663 10-5
Table Performance comparison on the RealSR_Canon 2 real-world dataset. Indicates that the corresponding domain adaptive method is ineffective for SR task. None indicates that the results temporarily unavailable. Time represents the duration of domain adaptive training.
Method
PSNR
SSIM
TIME(MIN)
DASR(Domain) [4]
26.23
0.766
None
DASR oonesho((Domaalle [4]
23.22
0.645
19.33
ZSSR 34]
26.01
0.748
641.17
StyleGGAN-NADAID
None
P0DA
None
BSRGAN
26.91
None
None
Ceal-ESRGANBA
26.14
None
None
SwinIR[37]
26.64
None
None
DASR 38
27.40
None
None
HAT
27.48
0.804
None
SAFMN 26 +IDDA
27.46
0.790
15.3
SRFormer 25 +IODA
27.66
0.810
11.23
27.82
0.812
9.01
classic real-world SR networks (BSRGAN 35 networks (Swinlr (37), DASR [38), HAT (4).. It can be observed that proposed method achieves satisfactory performance.
PODA 20
Real-ESRGAN 36)). and blind SR
4
Conclusion
In this paper, we propose Instance-guided One-shot Domain Adaptation for SR (IODA), which enables efficient domain adaptation with only single unlabeled target domain LR image. IODA introduces an image-gus gugged domain adaptation method that is more effective in enforcing constraints texture details compared to existing text-guided domain adaptation methods. Additionally, propose the instance- instance-guided target domain distribution expansion strategy. This strategy utilizes the diversity of instances in images expand the diversity of domain distribution, effectively enhancing the performance of one-shot domain adaptation.
Although the proposed method effectively mitigates the negative impact of the domain gap on SR network performance, domain adaptation training still requires a relatively long time (over ten minutes), which is not user-friendly for individuals who only need to super resolution of single photo. In the future, we plan further optimize the speed of domain adaptation, for example, by refining the selection of parameters related network domain adaptation.
Limitation:

---

Acknowledgments
This work was supported by the Natural Science Foundation of China (Grant 62176119), and the Jiangsu Graduate Research Innovation Program (Grant KYCX24_0255)
References
[1] Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. Enhanced deep residual networks for single image super-resolution. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1132-1140, 2017.
[2] Jianrui Cai, Hui Zeng, Hongwei Yong, Zisheng Cao, and Lei Zhang. Toward real-world single image super-resolution A new benchmark and a new model. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 3086-3095, 2019.
[3] Yi Xiao, Qiangqiang Yuan, Kui Jiang, Jiang He, Chia- Wen Lin, and Liangpei Zhang. Tist: A top-k token selective transformer for remote sensing image super-resolution. IEEE Transactions on Image Processing, 33:738-752, 2024.
[4] Yuchun Miao, Lefei Zhang, Liangpei Zhang, and Dacheng Tao. Dds2m: Self-super de- noising diffusion spatio-spectral model for hyperspectral image restoration. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 10022-12062 2023.
[5] Yuchun Miao, Lefei Zhang, Liangpei Zhang, and Dacheng Tao. Dds2m: Self Sepevvupe de- noising diffusion spatio-spectral model for hyperspectral image restoration. In 2023 JEEE/CVF International Conference on Computer Vision (ICCV), pages 12052-12062, 2023.
[6] Ziyu Wan, Zhang, Dongdong Chen, and Jing Liao. Bringing old films back to life. In 2022 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 17673-17682, 2022.
[7] Shan Lin and Edgar Simo-Serra. Restoring degraded old films with recursive recurrent trans- former networks. In 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 6704-6715 14, 2024.
[8] Jiaqing Zhang. Jie Lei, Weiying Xie, Zhenman Fang. Yunsong Li, and Qian Du. Supery- olo: Super resolution assisted object detection in multimodal remote sensing imagery. IEEE Transactions Geoscience and Remote Sensing, 61:1-15, 2023.
[9] Wei- Yen Hsu and Pei-Ci Chen. Pedestrian detection using stationary wavelet dilated residual super-resolution. IEEE Transactions on Instrumentation and Measurement, 71:1-11, 2022.
[10] Jinze Yang, Kun Fu, Youming Wu, Wenhui Diao, Wei Dai, and Xian Sun. Mutual-feed learning for super-resolution and object detection in degraded aerial imagery. IEEE Transactions on Geoscience and Remote Sensing, 60:1-16 2022.
[11] Zhongxi Qiu, Yan Hu, Xiaoshan Chen, Dan Zeng, Qingyong Hu, and Jiang Liu. Rethinking dual- stream super-resolution semantic learning in medical image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(1):451 1644,,444222 2024.
[12] Lin Wang, Tae- Kyyn Kim, and Kuk-Jin Yoon. Joint framework for single image reconstruction and super-resolution with an event camera. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(1 (::11)))7677777 2022.
[13] Shaohua Guo, Liang Liu, Zhenye Gan, Yabiao Wang. Wuhao Zhang, Chengjie Wang, Guannan Jiang, Wei Zhang, Ran Yi, Lizhuang Ma, and Ke Xu. Isdnet: Integrating shallow and deep networks for efficient uigh resolution segmentation. In 2022 IEEF/CV Conference Computer Vision and Pattern Recognition (CVPR), pages 43660, 2022.
[14] Yunxuan Wei, Shuhang Gu, Yawei Li, Radu Timofte, Longcun Jin, and Hengjie Song. Unsuper- vised real-world image super resolution via domain-distance aware training. 2021 IEEF/VF Conference Computer Vision and Pattern Recognition (CVPR), pages 13380- 13389, 2021.
10

---

[15] Wei Wang, Haochen Zhang. Zehuan Yuan, and Changhu Wang. Unsupervised real-world super- resolution: domain adaptation perspective. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 4298 4307, 2021.
[16] Jiayi Guo, Chaofei Wang. You Wu, Eric Zhang, Kai Wang, Xingqian Xu, Shiji Song, Humphrey Shi, and Gao Huang. Zero-shot generative model adaptation via image-specific prompt learning. In 2023 IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 11494-1 1503, 2023.
[17] Seogkyu Jeon, Bei Liu, Pilhyeon Lee, Kibeom Hong. Jianlong Fu, and Hyeran Byun. Improving diversity in zero-shot gan adaptation with semantic variations. In 2023 IEEE/CVF International Conference Computer Vision (ICCV), pages 7224-7233, 2023.
[18] Ceyuan Yang. Yujun Shen, Zhiyi Zhang. Yinghao Xu, Jiapeng Zhu, Zhirong Wu, and Bolei Zhou. One-shot generative domain adaptation. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 7699-7708, 2023.
[19] Rinon Gal, Or Patashnik, Haggai Maron, Amit H. Bermano, Gal Chechik, and Daniel Cohen-Or. Stylegan-nada. ACM Trunsactions on Graphics (TOG), 41:1 - 13, 2021.
[20] Mohammad Fahes, Tuan-Hung Vu, Andrei Bursuc, Patrick Pérez, and Raoul De Charette. PØda: Prompt-driven zero-shot domain adaptation. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 18577-18587, 2023.
[21] Rundong Luo, Wenjing Wang, Wenhan Yang, and Jiaying Liu. Similarity min-max: Zero-shot day-night domain adaptation. In 2023 IEEE/CVF International Conference Computer Vision (ICCV), pages 8070-8080, 2023.
[22] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Interna- tional Conference on Machine Learning, 2021.
[23] Zeyi Sun, Ye Fang, Tong Wu, Pan Zhang, Yuhang Zang, Shu Kong, Yuanjun Xiong, Dahua Lin, and Jiaqi Wang. Alpha-clip: A clip model focusing wherever you want. ArXiv, abs/2312.0381s 2023.
[24] Xiangyu Chen, Xintao Wang, Jiantao Zhou, Yu Qiao, and Chao Dong. Activating more pixels image super-resolution transformer. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 22367 22377,2237,222 2023.
[25) Yupeng Zhou, Zhen Li, Chun-Le Guo, Song Bai, Ming-Ming Cheng, and Qibin Hou. Srformer: Permuted self-attention for single image super-resolution. In 2023 IEEE/CVF International Conference Computer Vision (ICCV), pages 17734-12745, 2023.
[26] Long Sun, Jiangxin Dong, Jinhui Tang, and Jinshan Pan. Spatially-adaptive feature modulation for efficient image super-resolution. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 13144-13153, 2023.
[27] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, (886) 2008.
[28) Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll'ar, ar, and Ross B. Girshick. Masked autoencoders are scalable vision learners. 2022 IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15979-15988, 2021.
[29] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yes Lo, Piotr Dollár, and Ross B. Girshick. Segment anything. 2023 IEFE/CVF International Conference on Computer Vision (ICCV), pages 3992-4003, 2023.
[30] Stephan R. Richter, Vibhav Vineet, Stefan Roth, and Vladlen Koltun. Playing for data: Ground truth from computer games. ArXiv, htts/1608.02122... 2016.

---

[31] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for seman- tic urban scene understanding. In 2016 IEEE Conference on Computer Vision and Puttern Recognition (CVPR). pages 3213-3223, 2016.
[32] Christos Sakaridis, Dengxin Dai, and Lue Van Gool. Acde: The adverse conditions dataset with correspondences for semantic driving scene understanding. In 2021 IEEF/CVF International Conference on Computer Vision (ICCV), pages 10745-10755 2021.
[33] Xintao Wang. Liangbin Xie, Ke Yu, Kelvin C.K. Chan, Chen Change Loy, and Chao Dong. BasicSR: Open source image and video restoration toolbox. https://gittub: com/ XP xPixelGroup/ Baa.llorru//BasiSS.t./ Basic
2022.
[34] Assaf Shocher, Nadav Cohen, and Michal Irani. Zero-shot super-resolution using deep internal learning. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3118-3126, 2017.
[35] Kai Zhang, Jingyun Liang. Luc Van Gool, and Radu Timofte. Designing practical degradation model for deep blind image super-resolution. In IEEE International Conference on Computer Vision, pages 4791 4791-4800, 2021.
[36] Xintao Wang, Liaggin Xie, Chao Dong, and Ying Shan. Real-esrgan: Training real-world blind super-resolution with pure synthetic data. 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pages 1905-1914 2021.
[37] Jingyun Liang. Jie Cao, Guolei Sun, K. Zhang, Luc Van Gool, and Radu Timofte. Swinir: Image restoration using swin transformer. 2021 IEEE/CVE International Conference on Computer Vision Workshops (ICCVW), pages 1833-1844 2021.
[38] Jie Liang, Huiyu Zeng, and Lei Zhang. Efficient and degradation-adaptive network for real- world image super-resolution. In European Conference on Computer Vision, 2022.
[39] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for genera- tive adversarial networks. IEEE Transactions on Pattern Analysis and Machine Intelligence. (112))2221222222 2021.
[40] Jonathan Ho. Ajay Jain, and P. Abbeel. Denoising diffusion probabilistic models. ArXiv, abs/2006.1 11239, 2020.
[41] Gwanghyun Kim, Taesung Kwon, and long-Chul Ye. Diffusionclip: Text-guided diffusion models for robust image manipulation. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages
2021.
[42] James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, An- Demis
drei A. Rusu, Kieran Milan, John Quan. Tiago Ramalho, Agnieszka Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114:3521 3526, 2016
[43] Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2):295-377, 2016.
[44] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang. Stephen Lin, and Baining Guo. Swin wranffrmer: Hierarchical vision transformer using shifted windows. 202/ IEEE/CVF International Conference Computer Vision (ICCV), pages 9992-10002, 2021.
[45] Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Neural Information Processing Systems, 2017.
[46] Xindong Zhang, Huiyu Zeng, Shi Guo, and Lei Zhang. Efficient long-range attention network for image super-resolution. In European Conference Computer Vision, 2022.
12

---

[47] Jac Woong Soh, Sunwoo Cho, and Nam Ik Cho. Meta- Meta-transfer learning for zero-shot super- resolution. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3513-3522, 2020.
[48] Zeshuai Deng, Zhuokun Chen, Shuaicheng Niu, Thomas Li, Bohan Zhuang, and Mingkui Tan. Efficient test-time adaptation for super-resolution with second-order degradation and reconstruction. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 74671-74701 Curran Associates, Inc.. 2023.
[49] Mohammad Saeed Rad, Thomas Yu, Behzad Bozorgtabar, and Jean-Philippe Thiran. Test- time adaptation for super-resolution: You only need to overfit on few more images. In 202/ IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pages 1845-1854, 2021.
[50] Lei Zhang, Jiangtao Nie, Wei Wei, and Yanning Zhang. Unsupervised test-time adaptation learning for effective hyperspectral image super-resolution with unknown degeneration. IEEE Transuctions on Puitern Analysis and Machine Intelligence, 46(7):5008-5025, 2024.
[51] Zhen Cheng, Zhiwei Xiong, Chang Chen, Dong Liu, and Zheng-Jun Zha. Light field super- resolution with zero-shot learning. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10005-100144 2021.
[52] Wei Sun, Dong Gong, Qinfeng Shi, Anton van den Hengel, and Yanning Zhang. Learning to zoom-in via learning to zoom-out: Real-world super-resolution by generating and adapting degradation. IEEE Transactions on Image Processing, 30:2947-2962. 2021.
[53] Manuel Fritsche, Shuhang Gu, and Radu Timofte. Frequency separation for real-world super- resolution. 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCWW), pages 3599-3660 -3099 2019.
[54] Xiaozhong Ji, Guangpin Tao, Yun Cao,, Ying Tai, Tong Lu, Chengjie Wang, Jilin Li, and Feiyue Huang. Frequency consistent adaptation for real world super resolution. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 1664-1672, 2021.
[55] Zhicun Yin, Ming Liu, Xiaoming Li, Hui Yang, Longan Xiao, and Wangmeng Zuo. Metaf2n: Blind image super-resolution by learning efficient model adaptation from faces. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 12987-12998, 2023.
[56] Xiaoqian Xu, Pengxu Wei, Weikai Chen, Yang Liu, Mingzhi Mao, Liang Lin, and Guanbin Li. Dual adversarial adaptation for cross-device real-world image super-resolution. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5657-5666, 2022.
[57] Paul Albert, Mohamed Saadeldin, Badri Narayanan, Brian Mac Namee, Deirdre Hennessy, Noel F. O'Connor, and Kevin McGuinness. Unsupervised domain adaptation and super resolution on drone images for autonomous dry herbage biomass estimation. In 2022 IEEF/VF Conference Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1635- 1645, 2022.
13

---

A Related Work
A.1 One/Zeer-shot domain adaptation
Luo al. 21 conducted research on zero-shot domain adaptation of object detection networks from daytime to nighttime scenes. They utilized data augmentation to maximize the distance between daytime and nighttime input images while minimizing the differences in features within the backbone network, thereby enhancing the network's robustness to variations in image brightness. GAL et al. [9] were the first integrate the CLIP network (22] into the Style-GAN [39] image generation model for zero-shot domain adaptation. They proposed a text-guided image generation network domain adaptation method, leveraging the robustness of the CLIP network open-world scenarios to guide network adaptation from the source domain to the target domain using text descriptions. Furthermore, GAL et [9] proposed a strategy for selecting fine-tuning layers. During the network adaptation towards the target domain, specific layers are adjusted based on the magnitude of weight changes to enhance the stability of network domain adaptation. Kim al. replaced the StyleGAN network in the StyleGAN-NADA method with the diffusion network [4),, proposing DiffusionCLIP [41] which achieved better performance. Guo et al. [66] argued that StyleGAN-NADA and DiffusionCLIP identify all images using single text description, leading to overly uniform directions in domain adaptation and potential pattern collapse issues. Therefore, they proposed a specific text descriptor prediction module that individually predicts specific text description for each source domain image, effectively reducing the risk of pattern collapse. To increase the diversity of domain transfer directions, Jeon et al. [17] proposed perturbation strategy where noise is artificially added during text feature extraction, along with an orthogonal constraint to reduce the redundancy of the noise. Additionally, Jeon et al. 117 argued that specifying certain layers for fine-tuning in StyleGAN-NAIAA is unreasonable, hecause the remaining layers also contribute to the network's 's predictions in the target domain. Therefore, they introduced an EWC 42] (Elastic Weight Consolidation) regularization term to fine-tune all layers of the network, but with constraints on the range of weight changes to avoid excessive adjustments. Fahes al. 220] proposed the zero-shot domain adaptation method PODA for semantic segmentation. Its core was a text-guided feature augmentation module. This allowed the augmentation module to augment source domain features according to the text description of the target domain, obtaining features that approximated the target domain distribution. As a result, during training, the network gained the ability to handle target domain feature distributions.
A.2 Super resolution network
was the first to introduce neural networks into the SR task, achieving end-to-end SR. Liang et al. 38 constructed multi-expert system where each expert was intended to handle image information within different degraded subspace. Through cooperation among multiple experts, the system was capable of processing image information across the entire degradation space. Liang et al. (37) introduced such as patch boundary artifacts and limited receptive fields with patches, using a sliding window approach. HAT [24] further enhanced the feature correlation between windows in SwinIR by introducing Overlapping Cross-Attention Blocks (OCAB) to enhance feature interactions between adjacent windows. Additionally, they combined channel attention with self-attention methods to activate more pixels for SR reconstruction, further improving its SR performance.
Dong et al.
43
Tran-Trranfrrmmrr [44) into the SR network, which effectively alleviated issues
Although Transformer [55 could bring significant performance improvements to SR networks, the computational burden it imposed was considerable. Therefore, Zhang al. (46] first employed shift convolution to extract local structural information from images while maintaining complexity similar to 1x1 convolutions. Then, they introduced group-wise multi-scale self-attention (GMSA) modules, which used different window sizes to compute self-attention on non-overlapping feature groups By cascading shift convolution with GMSA modules, ELAN effectively enhanced the processing speed of existing Transformer- based SR networks. Zhou et al. [25] aimed to reduce the computational burden of networks using large-scale window self-attention. Its core concept was permuted self attentiooo (PSA), which reduced the channel dimension of the key and value matrices and then incorporated spatial information into the channel dimension. This approach effectively prevented the loss of spatial information while reducing the number of channels. Sun et al. [26] split features along the channel dimension and extracted these split features at different downsampling ratios, enabling the network to have larger receptive field while effectively reducing computational cost.
14

---

A.3 Domain Adaptation Methods in SR
The feature distribution differences between the training and test sets led to significant performance degradation of SR networks that performed well on the training set when evaluated the test set. To address this issue, test-time domain-adaptive SR networks 34 treating the training on the target domain, the network simulated the degradation experienced by target domain LR images to generate additional training samples. Training the network with these simulated samples, which approximated the target domain's 's degradation, effectively reduced the negative impact of distribution discrepancies on network performance.
48) 49 [50] were introduced, the source domain and the test set as the target domain. During inference
Shocher et al. 34] can be seen as a test-time domain- Komain--daptiiv SR network. During inference, it performed Bicubic downsampling on target domain LR images to generate pseudo-LR images, simulating the degradation of the target domain LR images. It then used paired target domain LR and pseudo-LR images for supervised training, achieving super-resolution without requiring labels for target domain HR images. Soh et al. [47] suggested that the ZSSR (4) network repeatedly performed domain adaptation training from the random initial weights, leading to long training times. Therefore, they attempted to find a universal initial weight parameter to reduce the duration of domain adaptation training. Deng al. 48 argued that the ZSSR network's consideration of only the Bicubic downsampling degradation model was insufficient to represent the more complex degradation models encountered by LR images real-world scenarios. Therefore, they proposed the SRTTA network. which considered various degradation factors such as GaussianBlur, DefocusBlur, GlassBlur, and GaussianNoise. They used a pre- pre-trained degradation classification network identify the degradation category of target domain LR images and generated corresponding pseudo-LR images based on this classification. This more accurate degradation modeling enabled SRTTA to achieve better SR performance. Rad et al. [49] constrained fine-tuning samples by actively selecting additional reference samples that optimize fine-tuning efficiency, thereby improving network performance. Additionally, Zhang et al. (50) and Cheng et al. [51] applied the concept of test-time domain adaptation to propose Light Field Super-Resolution and Hyperspectral Image Super-Resolutie respectively.
Although test-time domain domain-adaptive SR networks had considered various degradation models, real- world scenarios involved highly complex degradation due to factors such as lighting and imaging devices, which manual degradation models could not fully represent. To address this, adversarial gen- crated domain adaptation methods emerged
[4. [52] (53) [54] (55 (56] [57], using generative adversarial networks for implicit modeling of degradation, thereby avoiding the need for complex manual modeling. Fritsche et al. (53) separated high-frequency and low-frequency information for domain adaptation training. They considered that texture details correspond to
information, which crucial for SR tasks. Therefore, they applied high-frequency filtering before feeding the features into the discriminator, using the discriminator to constrain the high-frequency information, effectively improving SR performance in reconstructing texture details. et al. [54] similarly constrained generated images at the frequency level, using the discriminator for adversarial training on uighffrequency information and introducing a Frequency Density Comparator to enable the network to perceive firequency differences at varying sampling rates, further improving SR performance. Subsequently, Wang et al. [5] considered the impact of domain distance between the target domain and the source domain
network domain adaptation training. They optimized the network adaptation process based on the domain distance mapped by a discriminator, assigning higher learning weights samples with higher domain similarity. further enhancing the network's fit to the target domain. Yin et al. (5 also adopted the concept of distance awareness from [52 and achieved good performance in facial SR tasks. Xu et al. [56) introduced two adversarial adaptation modules to align source domain features with target domain features, achieving effective cross-device domain adaptive super-resolution performance.
While adversarial generated domain adaptation networks achieved good performance, they required a large number of target domain samples for network adaptation, making deployment challenging in real-world scenarios. Testing-time domain adaptation methods could perform inference on individual test samples from the target domain, but they required complex manual modeling of the target domain's 's degradation model and separate degradation modeling and training for each test LR image, which was time-consuming.
The Alpha-CLIP model (23,, trained on millions of data samples, covers a wide range of scenarios, including various lighting conditions and degradation models, and possesses rich prior knowledge
15

---

and strong generalization. Therefore, we proposed the IODA method, which leverages Alpha-CLIP's extensive prior knowledge to guide domain adaptation for SR networks. IODA performs domain adaptation using only single LR image from the target domain without requiring HR image labels. Additionally, when performing SR inference on batch of data, domain adaptation training on single LR image suffices to achieve efficient super-resolution for all LR images in the target domain.
Supervised LOSS
Sampling
Gsource
Latent vector
Stage one
Image
GT GT
"Cat"
Sampling
Sourree
Image
Text Siance
Random
Latent vector
noise
Image
Clip Space
Almage
Text
Sampling
Garget
Text
Dog
Stage two
Image Tareet
Figure 6:
Text-guided domain adaptation. Existing generative networks typically sample from random noise to generate latent vectors, which are then input into the generator network to produce images. Red indicates adjustable weights, while blue indicates frozen weight.
B Text-guided domain adaptation
As shown in Figure 6, the existing text-guided domain adaptation is primarily used in the field of image generation. In the first stage, after pretraining the source domain generation network Gourree using the source domain dataset, it is duplicated to serve as the target domain generation network GTorget- In the second stage, sample from random noise generate latent vectors. Then, the source domain generation network Gourcee and the target domain generation network GTarget are used to respectively predict the source domain generated image I ImageSSurce and the target domain generated image Image""arget from latent vectors. Corresponding source domain text description T Teert Source ("Cat") and target domain text description T l'arget ("Dog") are provided. Convert the source domain generated image, target domain generated image, source domain text description, and target domain text description into CLIP space using CLIP image encoder and text encoder, respectively. Existing methods aim to determine the text cross-domain direction vector AText from source domain text features to target domain text features in CLIP space, as well to determine the image cross-domain direction vector Image from source domain generated image features to target domain generated image features in CLIP space. By aligning these two cross-domain direction vectors, training constraints are imposed on the target domain generation network. For instance, the text changes from "cat" to "dog," the generated image should also transition from cat image to dog image.
C Alpha-CLIP
As shown in Figure 7 Alpha CLIP
23 introduces an additional region-range masking branch
(Alpha Conv) compared CLIP 22 During network training, this method concatenates the region-range mask with RGB images into the CLIP image encoder E, and feeds the corresponding text descriptions of masked regions into the CLIP text encoder ET. Contrastive loss is used to constrain the outputs of both encoders, where paired text-image samples are positive and unpaired samples are negative. The introduction of region-range masks allows the network to focus more on target features corresponding to the masked regions during inference.
16

---

RGB Conv
Alpha-CLIP Image Encoder
Image
Alpha Conv
CLIP Image Encoder
CLIP Text Encoder
A bird
similarity matrix
Figure 7: Alpha-CI.IP
LR image
Add IODA
Baseline
Figure 8: Visual comparisons. The source domain dataset is the GTA 30 daytime scene dataset, and
the target domain dataset includes various scene branches from the ACDC [32] dataset, such as rain, night, and snow. The network architecture used is the SAFMN (26 network.
Visualization results
The network pre-trained the source domain dataset tested on the target domain dataset, and its test results were used as the baseline. From Figure (8, it can be observed that domain gap causes significant noises in the baseline model under rainy, night, and snowy scenes. After the SR network undergoes IODA domain adaptation, the noise eliminated, resulting much improved visual appearance. In addition, shown in Figure IODA demonstrates superior performance compared to other domain adaptation methods for SR.
17

---

Fish ball
Beef ball
Fish ball
Fish ball
Fish ball
Fish ball
Crab most hall
Beef ball
Beef ball
Beef ball
Beef ball
Cuttlefas ball
Meat ball
Crab meat ball
Crab meat ball
Crab meat ball
Crab meat ball
and vegetal
Cuttlefish ball
Cuttlefis ball
Cuttlefis ball
Cuttlefis ball
Sausago
Dumpling
Beef brisket
Fish ball
Fish ball
Fish ball
Fish ball
Beef ball
Beef ball
Beef ball
Beef ball
Soyed smasage
Marieated belly
Crab meat ball
Crab meat ball
Crab meat ball
Crab meat ball
Marinated pork tripe
Cuttlefis ball
Cuttlefis ball
Cuttlefis ball
Cuttlefish ball
BE VEGETABLES ES BOB
4708
4708
4708
4708
AYS
MENT
Z 5500 4708
WE APOLOGIZE FOR THE INCONVENIECE CAUSED F 12 =
4708
4708
4708
4708
88t 1/ &
Figure 9: The large image on the left is the LR image, and the sub-images on the right DADA (56. DASR [1)., ZSSR 34. SRTTA 48 (first row), SAFMN 26 +IODA, SRFormer 25 +IODA,
HAT 2
+IODA, and GT images (second row). Please zoom-in screen.
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We describe the contributions and scope of the paper in the abstract and introduction sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We describe the limitations of the proposed method the conclusion section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
19

---

Justification: [TODO]
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibiliyy
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have provided a detailed description of the proposed method in the paper, and will release the source code to enhance the reproducibility of the method.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
20

---

Answer: [Yes]
Justification:
We will release the source code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/
wwtsSubbmiiiioooiiioooooooio for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We will release source code and provide detailed description in the experi- mental details section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We repeat the relevant experiments five times, using the mean and variance to represent the results and reduce result uncertainty.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
21

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We describe the computational requirements the Experimental Details
section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: [TODO]
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: [TODO]
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
22

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
23

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We will release the source code and provide detailed explanation.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mooel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
24


Reviews:
[{"review_id": "VSA7fErCmF", "reviewer": "Reviewer_wDY8", "summary": "This paper addresses one-shot domain adaptation (OSDA) in the field of super-resolution (SR). It leverages the fact that the content remains unchanged during super-resolution to propose an image-guided domain adaptation method, ensuring consistency by aligning the direction between the source domain and the target domain. The authors highlight the difficulty in learning the target domain's distribution during OSDA in SR and propose increasing data samples using random masking, similar to MAE. They also utilize SAM and Alpha Clip to obtain instance-aware representations. The method trains to align the direction between the source and target domains using the representation obtained from Alpha Clip. Various experiments demonstrate the effectiveness of the proposed method.", "questions": "Please refer to the Weaknesses section", "limitations": "They've addressed the limitations of the proposed method.", "rating": 3, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- They propose an OSDA method for real-world applications of super-resolution (SR).\n- They effectively utilize foundation models to address the domain adaptation (DA) challenges in the SR task.", "weaknesses": "There seems to be an insufficient survey of domain adaptation methods in the SR task, specifically those outlined in references [1-4]. Major revision is needed to emphasize the necessity and originality of the proposed method, based on an analysis of these existing methods. Particularly, since [4] addresses test-time adaptation, it is essential to highlight the advantages of one-shot domain adaptation over test-time adaptation. Additionally, the experimental tables predominantly report performance improvements over the baseline, but a comparison with existing domain adaptation methods is also required.\n\n[1] Wang, Wei, et al. \"Unsupervised real-world super-resolution: A domain adaptation perspective.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.\n\n[2] Wei, Yunxuan, et al. \"Unsupervised real-world image super resolution via domain-distance aware training.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021.\n\n[3] Xu, Xiaoqian, et al. \"Dual adversarial adaptation for cross-device real-world image super-resolution.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.\n\n[4] Deng, Zeshuai, et al. \"Efficient test-time adaptation for super-resolution with second-order degradation and reconstruction.\" Advances in Neural Information Processing Systems 36 (2023): 74671-74701.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Question1**: There seems to be an insufficient survey of domain adaptation methods in the SR task, specifically those outlined in references [1-4]. Major revision is needed to emphasize the necessity and originality of the proposed method, based on an analysis of these existing methods. Particularly, since [4] addresses test-time adaptation, it is essential to highlight the advantages of one-shot domain adaptation over test-time adaptation.\n\n**Response:** In the paper, we describe domain adaptation for SR tasks in Appendix A1, \"One/Zero-shot Domain Adaptation\", where references 1 and 2 cited by the Reviewer correspond to references 12 and 13 in the paper. To provide a clearer and more comprehensive overview of domain adaptation methods in the SR task, we will add a section 'Domain Adaptation Methods in SR' in the related work. In this section, we added descriptions of references [3, 4], provided a detailed analysis of test-time domain adaptation networks, and highlighted the advantages of one-shot domain adaptation.\n\n**Domain Adaptation Methods in SR**\n\nThe feature distribution differences between the training and test sets led to significant performance degradation of SR networks that performed well on the training set when evaluated on the test set. To address this issue, test-time domain-adaptive SR networks were introduced, treating the training set as the source domain and the test set as the target domain. During inference on the target domain, the network simulated the degradation experienced by target domain LR images to generate additional training samples. Training the network with these simulated samples, which approximated the target domain's degradation, effectively reduced the negative impact of distribution discrepancies on network performance.\n\nShocher et al. [28] can be seen as a test-time domain-adaptive SR network. During inference, it performed Bicubic downsampling on target domain LR images to generate pseudo-LR images, simulating the degradation of the target domain LR images. It then used paired target domain LR and pseudo-LR images for supervised training, achieving super-resolution without requiring labels for target domain HR images. Deng et al. [4] argued that the ZSSR network's consideration of only the Bicubic downsampling degradation model was insufficient to represent the more complex degradation models encountered by LR images in real-world scenarios. Therefore, they proposed the SRTTA network, which considered various degradation factors such as GaussianBlur, DefocusBlur, GlassBlur, and GaussianNoise. They used a pre-trained degradation classification network to identify the degradation category of target domain LR images and generated corresponding pseudo-LR images based on this classification. This more accurate degradation modeling enabled SRTTA to achieve better SR performance.\n\nAlthough test-time domain-adaptive SR networks had considered various degradation models, real-world scenarios involved highly complex degradation due to factors such as lighting and imaging devices, which manual degradation models could not fully represent. To address this issue, adversarial generated domain adaptation methods emerged, using generative adversarial networks for implicit modeling of degradation, thus avoiding complex manual modeling.\n\nWang et al. [13] employed a generative network to generate fake LR images paired with high-resolution (HR) images and used a discriminator to constrain the generated LR images to align with the target domain distribution. Subsequently, Wei et al. [12] considered the impact of domain distance between the target domain and the source domain on network domain adaptation training. They optimized the network adaptation process based on the domain distance mapped by a discriminator, assigning higher learning weights to samples with higher domain similarity, further enhancing the network’s fit to the target domain. Xu et al. [3] introduced two adversarial adaptation modules to align source domain features with target domain features, achieving effective cross-device domain adaptive super-resolution performance.\n\nWhile adversarial generated domain adaptation networks achieved good performance, they required a large number of target domain samples for network adaptation, making deployment challenging in real-world scenarios. Testing-time domain adaptation methods could perform inference on individual test samples from the target domain, but they required complex manual modeling of the target domain’s degradation model and separate degradation modeling and training for each test image, which was time-consuming.\n\nThe CLIP model, trained on millions of data samples, covers a wide range of scenarios, including various lighting conditions and degradation models, and possesses rich prior knowledge and strong generalization. Therefore, we proposed the IODA method, which leverages CLIP’s extensive prior knowledge to guide domain adaptation for SR networks. IODA performs domain adaptation using only a single LR image from the target domain without requiring HR image labels. Furthermore, when performing SR inference on a batch of data, domain adaptation training is required only for the first image, enabling efficient inference for subsequent images.\n\n**Question2**: Additionally, the experimental tables predominantly report performance improvements over the baseline, but a comparison with existing domain adaptation methods is also required\n\n**Response:**  As shown in Table 1 of the Rebuttal materials, we have additionally included domain adaptive methods for super-resolution, DADA and SRTTA, in our comparative experiments. Currently, we compared super-resolution domain adaptive methods DASR, DADA, SRTTA, and ZSSR. Notably, we compared the performance of DASR with single target domain samples and multiple data samples. Additionally, we also evaluated the performance of general domain adaptive methods (StyleGAN-NAN, P0DA).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jOQI9Zjnta", "reviewer": "Reviewer_V6qe", "summary": "The paper presents a novel approach to one-shot domain adaptation for super resolution. The key idea is to use the CLIP directional vector between low resoultion source and target domain images to guide the SR image generation in the target domain. They further use occlusion masks to further increase the performance of the model by providing pre-trained Alpha-CLIP with different region-range masks, enabling it to generate Alpha-CLIP spatial features focused on different areas of the image, thereby expanding the diversity of the target domain distribution.", "questions": "- Can you compare the performance of your models against the other domain adaptation-based SR approaches like [12] and [13]?\n- Can you provide a more comprehensive visualization analysis to better support you performance claims?\n- Can you provide a comparison of the efficiency between this method and other methods? A comparison that considers both efficiency and PSNR/SSIM would be more reasonable.\n\n\n\n\n\n[12] Yunxuan Wei, Shuhang Gu, Yawei Li, Radu Timofte, Longcun Jin, and Hengjie Song. Unsuper304 vised real-world image super resolution via domain-distance aware training. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13380–13389, 2021.\n13] Wei Wang, Haochen Zhang, Zehuan Yuan, and Changhu Wang. Unsupervised real-world super307 resolution: A domain adaptation perspective. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 4298–4307, 2021.", "limitations": "The main limitations of this paper is the relatively low training efficiency that would impose certain limitations in practical scenarios. Although the idea looks nice, but as mentioned in the paper, it takes approx 10 mins to generate a SR version of a single image. Therefore, I'm not completely convinced about its complexity.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper has a well-organized structure and expresses its ideas clearly.\n- Ablation analysis is provided in the paper to study the effect of various components of the pipeline on the downstream performance.\n- The idea is pretty interesting and new and the motivation is sound.", "weaknesses": "- Given the high time complexity of the approach, the performance gains over the baseline seems to be small.\n- The SR performance of the model needs to be compared against stronger SR baselines to better analyze the performance gains.\n- More visualization analysis is needed in the paper to better support the performance gains.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Question1:** Can you compare the performance of your models against the other domain adaptation-based SR approaches like [12] and [13]?\n\n**Response:** As shown in Table 1 of the Rebuttal materials  (Rebuttal.pdf), we have additionally included domain adaptive methods for super-resolution, DADA [1] and SRTTA [2], in our comparative experiments. Currently, we compared super-resolution domain adaptive methods DASR [3], DADA [1], SRTTA [2], and ZSSR [4]. Notably, we compared the performance of DASR with single target domain LR image and multiple LR images. Additionally, we also evaluated the performance of general domain adaptive methods on SR tasks (Style-GAN-NAN [5], P0DA [6]). Reference [13] did not release its source code, so it was not included in the comparison experiments.\n\n**Question2:** Can you provide a more comprehensive visualization analysis to better support you performance claims?\n\n**Response:** As shown in Figure 1 of the Rebuttal materials  (Rebuttal.pdf), we provide additional visualizations to demonstrate the effectiveness of the proposed method.\n\n**Question3:** Can you provide a comparison of the efficiency between this method and other methods? A comparison that considers both efficiency and PSNR/SSIM would be more reasonable.\n\n**Response:** As shown in Table 1 of the Rebuttal materials  (Rebuttal.pdf), we have additionally included inference efficiency metrics to demonstrate the effectiveness of the method.\n\n**Question4:** Explanation and optimization of network adaptation training time consumption\n\n**Response:** Existing domain adaptation methods for SR can be broadly classified into test-time domain adaptation methods and adversarial generated domain adaptation methods. The test-time adaptation methods need to individually manual explicit model degradation and perform domain adaptation training for each image, which can be time-consuming for relatively larger target domain datasets (see Table 2 of the Rebuttal materials) and complex to manual model.\n\nAs summarized in Section 2.1 of the paper, the proposed IODA method utilizes a single LR image from the target domain for adaptation training, enabling the network to adapt from the source domain to the target domain. IODA requires only one adaptation training session on a single LR image from the target domain to perform super-resolution inference on the entire  target domain dataset, eliminating the need for repeated domain adaptation training on other target domain LR images. Additionally, IODA method leverages the rich prior semantic information from CLIP, resulting in lower training time compared to adversarial generated domain adaptation methods for SR task (DASR_oneshot and DASR (Domain)).\n\nAdditionally, the domain adaptation training time of IODA can be optimized. When the source and target domains have similar distributions in high-level semantic information, the adaptation training time can be significantly reduced. For example, as shown in Table 2 of the Rebuttal materials, when the source domain is the daytime driving dataset Cityscapes and the target domain is the rainy driving dataset ACDC_rain, the training time is reduced to 1.16 minutes. In addition, selective fine-tuning of network parameters is also an effective strategy for speed improvement, which will be the focus of our future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gi5bZ1G4tR", "reviewer": "Reviewer_G4LY", "summary": "This paper proposes a framework of efficient instance-guided one-shot domain adaptation (abbr. IODA) with only one unlabeled target domain LR image for addressing image super-resolution (SR) issues. On top of that, it designs an instance-guided target domain distribution expansion strategy to expand the diversity of domain distribution, thus enhancing one-shot DA performance. Extensive ablation studies across multiple datasets and networks have shown the effectiveness of IODA.", "questions": "1. During model training, is the selection of LR images for unlabeled target domains is random? Or does it need to be based on some criteria that are not stated in this manuscript?\n\n2. In practice, is it also necessary to add some screening criteria or conditional restrictions to the distribution expansion strategy to limit its scope? If the selection is random, how to ensure its representativeness?", "limitations": "While IODA provides inspiration for subsequent solutions to the task of domain adaption-based SR, it may encompass too many existing approaches such as CLIP, Alpha-CLIP, SAM, and it would further enhance the significance and value of this work if some specialized designs or optimizations could be proposed for the current research.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper provides a new perspective for solving the problem of image super-resolution based on the domain adaptation method, especially in resource-constrained situations.\n\n2. IODA achieves efficient domain adaptation by using a single unlabeled low-resolution (LR) image of the target domain. Moreover, the instance-guided target domain distribution expansion strategy prevents the pattern collapse by expanding the diversity of domain distribution. These facilitate IODA to carry out practical applications in the real world.", "weaknesses": "1. In order to clearly illustrate the methodology of this paper, the description of the specific implementation of the image-guided domain adaptation and instance-guided target domain distribution extension strategies needs to be further supplemented and refined, in particular with respect to the associated constraints.\n\n2. The evaluation of SR results cannot be limited to pixel-oriented PSNR and SSIM, it is equally important for the evaluation of visual perceptual effects, especially in real-world applications. In addition, visual demonstrations should be provided in the main text, not just in an appendix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Question1:** During model training, is the selection of LR images for unlabeled target domains is random? Or does it need to be based on some criteria that are not stated in this manuscript?\n\n**Response:** To ensure the reliability of the experimental results, we repeated each experiment 5 times and took the average result, selecting different images from the dataset each time as the single LR image for training in the target domain. In the ablation studies, to avoid the impact of different target domain images on module performance, we kept the target domain images consistent across all ablation experiments, using the first 5 images from the target domain dataset.\n\n\n\n**Question2:** In practice, is it also necessary to add some screening criteria or conditional restrictions to the distribution expansion strategy to limit its scope? If the selection is random, how to ensure its representativeness?\n\n**Response:** The proposed target domain distribution expansion strategy primarily extends the CLIP spatial features by introducing Alpha-CLIP, which generates features focusing on different instances. This method does not introduce new objects into an image, unlike other data augmentation methods such as Cut-mix, which might place a cat into an image full of dogs. Instead, it ensures that the network, when generating multiple feature maps, has each feature map focus on different dogs within the image. For example, one feature map focuses on a standing dog, while the next focuses on a lying dog. This approach avoids disrupting the original feature distribution with newly introduced feature distributions.\n\nThis method is similar to the human visual sensory mechanism, where one repeatedly focuses on different objects within a scene to achieve a comprehensive understanding of the scene.\n\n\n\n**Question3:** The evaluation of SR results cannot be limited to pixel-oriented PSNR and SSIM, it is equally important for the evaluation of visual perceptual effects, especially in real-world applications. In addition, visual demonstrations should be provided in the main text, not just in an appendix.\n\n**Response:** As shown in Figure 1 of the Rebuttal materials, we provide additional visualizations to demonstrate the effectiveness of the proposed method.\n\n \n\n**Question4:** Regarding the use of CLIP, Alpha-CLIP, and SAM.\n\n**Response:** In this paper, we address the specificity challenges of domain adaptation for low-level tasks like super-resolution by proposing an image-guided domain adaptation method. This method leverages the focus on detailed textures in super-resolution tasks, using images to guide the adaptation process and overcoming the limitations of text-guided approaches in representing fine details. Additionally, to address the limited diversity in target domain distribution caused by single target domain sample scenarios, we introduce an instance-guided target domain distribution expansion strategy. Our innovative approach efficiently enhances distribution diversity by focusing on different instances.\n\nIn the first innovation, we specifically optimize domain adaptation networks for super-resolution by proposing image-guided domain adaptation instead of text-guided approaches. In the second innovation, to address the limited diversity in target domain distribution, we introduce an instance-guided diversity expansion strategy. This is the first use of Alpha-CLIP for enhancing target domain feature distribution, with a designed instance region partitioning scheme rather than random mask shapes.\n\nMany excellent works utilize large models like CLIP, Llama, and SAM as the foundation for further exploration. For example, current domain adaptation networks for object detection use the CLIP network for guidance, but we do not deny the contributions of their work. Similarly, this paper employs powerful models like Alpha-CLIP and SAM as tools to achieve our objectives.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "s0RVY5TuQb", "reviewer": "Reviewer_ohtL", "summary": "The author uses CLIP to assist in extracting the representation of target domain samples and implements a one-shot domain adaptation framework.", "questions": "Why can Alpha-CLIP enhance the performance of SR when other versions of CLIP cannot?", "limitations": "The proposed method exhibits slow inference speed.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The introduction of CLIP improves the performance of one-shot domain adaptation.", "weaknesses": "Although the method is effective, it does not appear to be the first time that CLIP has been used for domain adaptation. Moreover, this work seems to merely combine Alpha-CLIP with SR.In addition, the training scenario for CLIP may involve cross-domain scenarios of super-resolution (SR). However, the author did not explain this issue.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Question1:** Why can Alpha-CLIP enhance the performance of SR when other versions of CLIP cannot?\n\n**Response:** Conventional CLIP-based SR network domain adaptation methods face limited target domain diversity when using a single target domain LR image. To address this problem, we propose an instance-guided target domain distribution expansion method. Similar to human perception of the environment, which achieves comprehensive understanding by repeatedly focusing on different targets within a scene, we generate multiple image features from a single LR image, each focusing on different regions. We argue that focusing on instances within the image, rather than randomly selected regions, can implicitly introduce high-level semantic information. Thus, we optimize the focus areas to target different instances.\n\nAlpha-CLIP, a new model introduced at CVPR24, enhances the standard CLIP by allowing specified focus areas. This makes it an ideal feature extractor for our approach. We first use the SAM model to construct an instance mask pool, then select a certain number of instance masks to guide Alpha-CLIP to focus on different instances in the image, ultimately expanding the target domain feature distribution.\n\nTherefore, alpha-CLIP's unique ability to specify focus areas makes it indispensable for our proposed method.\n\n \n\n**Question2:** Although the method is effective, it does not appear to be the first time that CLIP has been used for domain adaptation. Moreover, this work seems to merely combine Alpha-CLIP with SR. In addition, the training scenario for CLIP may involve cross-domain scenarios of super-resolution (SR). However, the author did not explain this issue.\n\n**Response:** The CLIP model, trained on millions of data samples, covers a wide range of scenarios, including various lighting conditions and degradation models, and possesses rich prior knowledge and strong generalization. This makes it widely used in downstream tasks such as domain adaptation for object detection and image generation, leveraging CLIP's extensive prior knowledge and strong generalization capabilities for efficient domain adaptation. Therefore, this paper introduces CLIP for the first time in the domain adaptation for low-level tasks like super-resolution, using its rich prior information for SR domain adaptation.\n\nHowever, existing CLIP-based domain adaptation methods cannot be directly applied to SR tasks, as SR focuses more on restoring low-level features like texture details, while existing tasks emphasize high-level semantic information. To address this, we propose an image-guided domain adaptation method for SR tasks. Additionally, to address the limited diversity in target domain distribution caused by single target domain sample scenarios, we introduce an instance-guided target domain distribution expansion strategy. Our innovative approach efficiently enhances distribution diversity by focusing on different instances.\n\nIn the first innovation, we specifically optimize domain adaptation networks for super-resolution by proposing image-guided domain adaptation instead of text-guided approaches. In the second innovation, to address the limited diversity in target domain distribution, we introduce an instance-guided diversity expansion strategy. This is the first use of Alpha-CLIP for enhancing target domain feature distribution, with a designed instance region partitioning scheme rather than random mask shapes.\n\n\n\n**Question3:** Explanation and optimization of network adaptation training time consumption\n\n**Response:** Existing domain adaptation methods for SR can be broadly classified into test-time domain adaptation methods and adversarial generated domain adaptation methods. The test-time adaptation methods need to individually manual explicit model degradation and perform domain adaptation training for each image, which can be time-consuming for relatively larger target domain datasets (see Table 2 of the Rebuttal materials) and complex to manual model.\n\nAs summarized in Section 2.1 of the paper, the proposed IODA method utilizes a single LR image from the target domain for adaptation training, enabling the network to adapt from the source domain to the target domain. IODA requires only one adaptation training session on a single LR image from the target domain to perform super-resolution inference on the entire  target domain dataset, eliminating the need for repeated domain adaptation training on other target domain LR images. Additionally, IODA method leverages the rich prior semantic information from CLIP, resulting in lower training time compared to adversarial generated domain adaptation methods for SR task (DASR_oneshot and DASR (Domain)).\n\nAdditionally, the domain adaptation training time of IODA can be optimized. When the source and target domains have similar distributions in high-level semantic information, the adaptation training time can be significantly reduced. For example, as shown in Table 2 of the Rebuttal materials, when the source domain is the daytime driving dataset Cityscapes and the target domain is the rainy driving dataset ACDC_rain, the training time is reduced to 1.16 minutes. In addition, selective fine-tuning of network parameters is also an effective strategy for speed improvement, which will be the focus of our future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
1. ### **Additional visual demonstrations**

As shown in Figure 1 of the Rebuttal materials (Rebuttal.pdf), we provide additional visualizations to demonstrate the effectiveness of the proposed method.

2. ### **Additional comparisons with domain adaptive methods for Super-resolution**

As shown in Table 1 of the Rebuttal materials  (Rebuttal.pdf), we have additionally included domain adaptive methods for super-resolution, DADA [1] and SRTTA [2], in our comparative experiments. Currently, we compared super-resolution domain adaptive methods DASR [3], DADA [1], SRTTA [2], and ZSSR [4]. Notably, we compared the performance of DASR with single target domain LR image and multiple LR images. Additionally, we also evaluated the performance of general domain adaptive methods on SR tasks (Style-GAN-NAN [5], P0DA [6]).

3. ### **Explanation and optimization of network adaptation training time consumption**

Existing domain adaptation methods for SR can be broadly classified into test-time domain adaptation methods and adversarial generated domain adaptation methods. The test-time adaptation methods need to individually manual explicit model degradation and perform domain adaptation training for each image, which can be time-consuming for relatively larger target domain datasets (see Table 2 of the Rebuttal materials) and complex to manual model.

As summarized in Section 2.1 of the paper, the proposed IODA method utilizes a single LR image from the target domain for adaptation training, enabling the network to adapt from the source domain to the target domain. IODA requires only one adaptation training session on a single LR image from the target domain to perform super-resolution inference on the entire  target domain dataset, eliminating the need for repeated domain adaptation training on other target domain LR images. Additionally, IODA method leverages the rich prior semantic information from CLIP, resulting in lower training time compared to adversarial generated domain adaptation methods for SR task (DASR_oneshot and DASR (Domain)).

Additionally, the domain adaptation training time of IODA can be optimized. When the source and target domains have similar distributions in high-level semantic information, the adaptation training time can be significantly reduced. For example, as shown in Table 2 of the Rebuttal materials, when the source domain is the daytime driving dataset Cityscapes and the target domain is the rainy driving dataset ACDC_rain, the training time is reduced to 1.16 minutes. In addition, selective fine-tuning of network parameters is also an effective strategy for speed improvement, which will be the focus of our future work.

 

###  **Reference**

[1] Xu, Xiaoqian, et al. "Dual adversarial adaptation for cross-device real-world image super-resolution." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.

[2] Deng, Zeshuai, et al. "Efficient test-time adaptation for super-resolution with second-order degradation and reconstruction." Advances in Neural Information Processing Systems 36 (2023): 74671-74701.

[3] Wang, Wei, et al. "Unsupervised real-world super-resolution: A domain adaptation perspective." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.

[4] A. Shocher, N. Cohen and M. Irani, "Zero-Shot Super-Resolution Using Deep Internal Learning," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 3118-3126, doi: 10.1109/CVPR.2018.00329.

[5] Gal R, Patashnik O, Maron H, et al. Stylegan-nada: Clip-guided domain adaptation of image generators[J]. ACM Transactions on Graphics (TOG), 2022, 41(4): 1-13.

[6] M. Fahes, T. -H. Vu, A. Bursuc, P. Pérez and R. De Charette, "PØDA: Prompt-driven Zero-shot Domain Adaptation," *2023 IEEE/CVF International Conference on Computer Vision (ICCV)*, Paris, France, 2023, pp. 18577-18587, doi: 10.1109/ICCV51070.2023.01707.

Meta-review: This paper presents an approach to using the CLIP directional vector to guide the SR image generation in the target domain. The method is interesting and recognized by most reviewers. The authors have addressed most reviewer concerns, receiving three borderline accept recomendations. The remaining concerns mainly from wDY8, who believes the authors should provide a more convincing survey of existing methods. In the rebuttal, the authors have updated the survey, which make clear the difference between the new method and domain adaption methods in SR.
Decision: Accept (poster)