Time-Varying LoRA: Towards Effective Cross-Domain Fine-Tuning of Diffusion Models
Authors: Zhan Zhuang
Yulong Zhang
Xuehao Wang
Jiangang Lu
Ying Wei
Yu Zhang
Venue: NeurIPS 2024 poster (2024)
Keywords: Low-rank adaptation
diffusion models
cross-domain learning
Primary area: other
TL;DR: This paper introduces a novel time-varying low-rank adapter that offers an effective fine-tuning framework for domain flow generation, and generation-based cross-domain learning methods to mitigate domain shifts.

Abstract: Large-scale diffusion models are adept at generating high-fidelity images and facilitating image editing and interpolation. However, they have limitations when tasked with generating images in dynamic, evolving domains. In this paper, we introduce Terra, a novel Time-varying low-rank adapter that offers a fine-tuning framework specifically tailored for domain flow generation. The key innovation of Terra lies in its construction of a continuous parameter manifold through a time variable, with its expressive power analyzed theoretically. This framework not only enables interpolation of image content and style but also offers a generation-based approach to address the domain shift problems in unsupervised domain adaptation and domain generalization. Specifically, Terra transforms images from the source domain to the target domain and generates interpolated domains with various styles to bridge the gap between domains and enhance the model generalization, respectively. We conduct extensive experiments on various benchmark datasets, empirically demonstrate the effectiveness of Terra. Our source code is publicly available on https://github.com/zwebzone/terra.

Time- Varying LoRA: Towards Effective Cross-Domain Fine-Tuning of Diffusion Models
Zhan Zhuang! Yulong Zhang
Xuehao Wang'
Jiangang Lu
Ying Wei Yu Zhang
Southern University of Science and Technology
City University of Hong Kong *Zhejiang University
12250063email sustech. edu. {zhangylcse, lujg, ying.wiinnii...... wei]@zju. edu. (xuehaowangfi, yu. zhang, ust com
Abstract
Large-scale diffusion models are adept generating high-fidelity images and facilitating image editing and interpolation. However, they have limitations when tasked with generating images in dynamic, evolving domains. In this paper, we introduce Terra, a novel Time-varying low-rank adapter that offers a fine-tuning framework specifically tailored for domain flow generation. The key innovation of Terra lies in its construction of a continuous parameter manifold through a time variable, with its expressive power analyzed theoretically. This framework only enables interpolation of image content and style but also offers generation- based approach address the domain shift problems unsupervised domain adaptation and domain generalization. Specifically. Terra transforms images from the source domain the target domain and generates interpolated domains with various styles to bridge the gap between domains and enhance the model generalization, respectively. We conduct extensive experiments on various benchmark datasets, empirically demonstrate the effectiveness of Terra. Our source code is publicly available on http://gittuhhcom/wweboonnneeer
Introduction
Recently, text-to-image diffusion models [38, 47, 48, 45] have revolutionized computer vision by synthesizing high-quality, creative images. Those models provide a user-friendly method for generating images through text prompts. Furthermore, with advancements in fine-tuning techniques of diffusion models [4], users can casily customize [83]. edit [27], and interpolate [88, 80, 7] images. common approach involves using low-rank adapter (LoRA) [25] to fine-tune diffusion models with a few images to generate customized images. This inspires a generation- approach to address fundamental and classical problem in machine learning known domain shift.
Domain shift is commonly studied the cross-domain learning [70, 82, 61] with two settings: unsupervised domain adaptation (UDA) [40, 12, 94], which aims to transfer knowledge from source domain to target domain, and domain generalization (DG) [89, 64], which focuses on training model on source domains and then generalizing unseen target domains. Prior methods [91, 15, 68, 90, 73] have demonstrated the effectiveness image translation and interpolation on the learning paradigms based mixup [79, 60], generative adversarial networks [16, 92], and diffusion models [24, 36]. Considering the impressive capabilities of diffusion models and the efficiency of fine-tuning techniques like LoRA, it is natural extend them to generate domain flow, which generates intermediate domains and bridges the source and target domains, as illustrated in Fig. 1(b).
"Equal contribution.
Conresponding authors.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

LoRA
generated
W
W/o
AW
W
t=
Terra manifold
Terro
Target domain
mid
down
up
Source domain
(a) From LoRA Terra
(b) Terra for domain flow generation
Figure I:
Illustration of the proposed Terra.
However, previous methods [36, 80] require multiple LoRAs to customize multiple domains, since single LoRA cannot effectively express knowledge of multiple domains with plugin [77]. To address this limitation, as illustrated in Fig. ((a), we propose a Time-varying low-rank adapter (Terra), which offers framework for gradual domain transferring by constructing continuous parameter manifold. Instead training multiple LoRAs for different domains, Terra maintains the parameter efficiency. To this end, inspired by the perspective of dynamic flows [75], Terra introduces a time variable for each domain and incorporates a square matrix that varies with time t within the original low-rank structure.
As depicted in Fig. 1(b), Terra enables use of different time values for various intermediate domains. Consequently, Terra can generate intermediate images that are natural and smooth when morphing image pairs, subjects, and styles. For UDA tasks, we generate target samples and transform the source samples into the target domain to form expanded source domain. Due to the smaller domain shifts, transferring from the expanded source domain to the target domain can improve the performance of existing UDA methods. For DG tasks, we interpolate among all source domains to generate images in various styles. Then, the generated samples are combined with the source domain images improve the performance of existing DG methods.
In summary, our contributions are four-fold:
We introduce Terra, a novel framework that integrates a square matrix with a time variable into the original low-rank structure, facilitating effective and flexible knowledge sharing across different domains while maintaining parameter efficiency.
We provide a theoretical analysis of the expressive power of Terra, comparing it to LoRA.
We demonstrate the application of Terra in image transformation and generation for UDA tasks and image interpolation for DG tasks via Terra, respectively.
Extensive experiments validate the effectiveness of Terra across various tasks, including generative interpolation, unsupervised domain adaptation, and domain generalization.
2 Related Work
Fine-Tuning of Text-to-Image Diffusion Models. The impressive performance diffusion mod- els [24, 55] has sparked surge of interest in text-to-image generation tasks. As the demand for personalized content synthesis grows [83], pioneer works such Textual Inversion [1]] and Dream- Booth [49] have proposed optimized text embedding and full fine-tuning frameworks to generate subject images with limited reference samples. Recently, several parfmeter-efficient methods for fine-tuning diffusion modules have been proposed, including adapters [54], LoRA [17, 50, 52], singular value decomposition on weight matrices [20], subsets of cross-attention [56, 32], and image prompt adapter [81, 76, 37, 66). Among those methods, several have been developed to address the challenges of multi-concept generation [32, 20, 17] and natural image interpolation [62, 30, 80, 88). Different from those methods, Terra focuses generation and interpolation within domain flows.

---

Domain Adaptation and Generalization. UDA [74, 34, 13, 67, 87, 63] is designed to address the challenge of adapting models trained on labeled source domains unlabeled target domains. The central premise of UDA methods is to learn domain-invarian features that minimize the domain gap. UDA approaches primarily fall into two branches: discrepancy-
methods [34, 72, 93, 19] and adversarial-ban based methods [13, 46, 85]. Conversely, DG [64, 89] seeks to train models that could generalize well to unseen target domains using multiple source domains. Effective DG methods, such SWAD [5] and SAGM [65] enhance the generalization by identifying and leveraging flatter minima of training losses landscapes. However, the performance of UDA and DG methods can be constrained by the availability of training data. To address this limitation, recent data augmentation techniques [73, 71, 84, 36] have been developed to improve the transfer effects of UDA and DG methods. Those methods can be categorized into feature-level [71, 95, 42] and image- level methods [73, 84, 36, 22], which enhance transfer performance through the transformation or generation of auxiliary samples at the feature and image levels. For instance, MSGD) [71] and GGF [95] use intermediate domains to gradually reduce the domain shift between the source and target domains, while BDG [73] employs pairs of cross-domain generators to synthesize domain- specific data based on the other domains. Additionally, CDGA [22] leverages the latent diffusion model to generate synthetic samples across domains and Domaindiff [36] trains LoRAs for each source domain to conduct domain fusion.
3 Methodology
3.1 Preliminary
LoRA [25] uses two low-rank matrices, W down E Rrxn and W up E Rmxr, where r < min(m: n)) to compute the weight matrix updates AW W up
W down E Rmxn, The forward pass of the new
weights changes from h
Wo Qz
h = Woz OW x= = Wox + a up W down
where a is scaling factor for the magnitude of the changes applied to the original weights. Although LoRA is primarily used for fine-tuning large language models, it is also employed in diffusion models for personalizing image generators with limited training samples, targeting specific styles or subjects [49, 52, 80J. The objective function in previous studies is expressed as noise matching:
((20)= = F...
[|ee - tao+ T,
T, e(c)))
(2)
where Og and AO denote the parameters of the text-to im-ge diffusion model and LoRA, respectively. The function e denotes the text encoder, and c corresponds to the text prompt. During the forward diffusion process. the variable 2, is obtained by gradually adding noise to the initial image Vy using the equation x, V + V1- Here o,, follows a decreasing schedule, and a is calculated as the cumulative product of values up timestep т. In the objection function, the timestep is sampled from a uniform distribution (111, 7), where T denotes the total number of timesteps. And the model is utilized to predict the noise 60.+-0 estimate the true noise e. After training, the well-trained denoiser Oo- + AO can denoise noises and generate images within a few sampling steps.
3.2 Terra: Time- Varyin Low-Rank Adapter
To address the need for fine-tuning diffusion models across multiple domains while maintaining the parameter efficiency, we propose the Terra, as depicted in Fig. 1(a). Terra involves constructing a LoRA flow that provides a parameter manifold by incorporating time-varying updates as
htt)= W AW (t)= Wa + W up KC() W down K(t) = F(W (Wiid.)) t)
(3)
where W mid E Rexx, t is one-parameter variable, and F is time- deeendeenddnnt function. This formulation enables the differentiable evolution of the parameters AW(t) (t) based on a middle time- varying matrix K(tt). A simple form of F(W, t) is tW + I, where I represents an identity matrix. Since r & min(m, nt), the parameter difference between Terra and LoRA with the same rank is negligible. Furthermore, by setting the parameter / to O, Terra will degenerate to LoRA. A. It is worth noting that the form J(W, 1) here just one of the possible variations. More forms can be found in Table 5 of Appendix B and comparison with MoE-based LoRA [69] is provided in Appendix E.
3

---

Here, we present a theoretical analysis of the expression power of the proposed Terra We define I, as a diagonal matrix with its first r diagonal entries as I and the remaining entries O. In the following theorem, we prove that Terra can effectively implement two LoRAs for specific downstream tasks by constructing a parameter manifold with reduced parameters.
Theorem 1. (The Equivariance between Terra and Multiple LoRAs) Assume there exist two LoRAs AWA. AWB E Rmxx with ranks of p and respectively. that effectively solve two specific down- stream tasks. Let k = max maxxrank([A [[WAA AW Bl), rank([A A AWBll Bl) where rank(-) denotes
the rank of a matrix. Then, there exists a Terra with W up E m W E R W mid E IR & and K((t) tW mid r, such that the updated matrix (t W up ((t down can simultane- ously solve the two downstream task, that is, we have AW(O) (0)= = AWA and A (1(1) = AWB B.
In Theorem 1, the number of trainable parameters of Terra is governed by |+| (m + +n)n k2 contrasting with that of two LoRAs |6| (m + n)(p + q). Note that & represents the maximum rank of the matrices obtained by concatenating the row and column spaces the two LoRA matrices, which is not greater than the sum of the ranks of the two LoRA matrices, ie., k S p p+ q.
Drawing inspiration from prior research the expressive power of LoRA [78], we further demon- strate the expressive power of Terra. Here, we focus on the multi-layer feedforward neural network with identity activation functions, and the analysis can be extended to fully connected neural networks and transformer networks [78]. Assuming that the target models fA and fB for two specific tasks, as well as the frozen model fo, are linear, they can be represented as:
[a(w) W aæ; In(#) W næ, fo(æ)) W L. Wix= (IL' Wi x.
where the frozen model has I. layers with consistent dimensions. We define the error matrices EA := W A I Wt, and EB == W B I Wt, and their ranks as Rg rank (Ea) and Rg, = rank By utilizing Terra ÁW (t), we can modify the pre-trained frozen model to closely approximate the two target models W and W We denote the d-th largest singular value of W by (a(W), and the best rank-r approximation [8] of W by LR,(W) The following theorem presents an upper bound for the approximation error with rank-k Terra.
Theorem 2.
(The Expressive Power of Terra) For each layer I, the rank-k Terra has updated matrix (t), and the function of time-varying matrix is K({f) = tW mid,1 + I. Assume that all weight matrices of the frozen model (i) I Wr + (R,((E)), and I Wi (En) are non-singular for all r & k([. 1).
1). Then the approximation error satisfie
L (
4 + AW()) WT < 20 (4)
min AWO
- WA
where the ditt as the (k:L + 1)-th largest singular values obtained by merging the singular values of EA a and En. Moreover, when k >
Rg R + the upproximation error zero.
We compare the approximation errors of Terra and multiple LoRAs with consistent parameter sizes for the above target models. We consider a rank of 2k for Terra and two k-rank LoRA in both tasks. Prior work [78] establishes an upper bound on LoRA's approximation error as o (Ex) + TbL+1 (EB), B). In Theorem 2, we demonstrate that Terra's 's approximation error bound is 20 Considering the definition of a², it is evident that our Terra's error bound is not greater than LoRÅ's.
Terra is capable of cross-domain generative tasks, where samples from different domains possess different f's. In the following sections, show the use of Terra in three different learning problems.
3.3 Warm Up: Constructing Evolving Visual Domains via Terra
In this section, we show the first application of Terra to construct evolving visual domains for generative interpolation between two image domains Ds and Dr characterized by the differences in the style subject, which is the key to apply Terra to UDA and DG.

---

Terra
%
Text-to-lmage
Xg
Diffusion Model
6oo (-)
Loss
A0
x
x
11(,T)
x T
For source images For torget images
Forward process
c "An ell painting of a porson"
Figure 2: The illustration of the training process of constructing evolving visual domains via Terra.
Our method is different from existing methods [53, 55, 88, 80] that employ direct interpolation between two images on embedding using spherical linear interpolation (a.ka slerp). To accomplish this, Terra incorporates a continuous time variable t. Training on the source images involves setting t to O, yielding the formulation AW(0)
W W down- Similarly, for the target images, t set to 1, leading to A (1) = W up (1) W down- In the context of fine-tuning text-to-image diffusion models, we employ image descriptions construct prompts for diffusion models, where the corresponding class label is denoted by "A [class]", , where "[class]" denotes the placeholder for the class label. Finally, the training objective, depicted in Fig. 2, is formulated follows
((20)= = Bew.N [Ba (De t= ||e 60o+A0 (w ),t)
(5)
+B DD t=1 ||e cot+50 (x. (((e)), (I
where AO represents the parameters of the Terra, e and c" denote text prompts for the source and target, and v and z/f represent the source and target samples. Formally, we construct evolving visual domains by the following two stages: (1) Fine-tune the parameters of Terra (i.e., AO uses source samples Ds and the second part with = 1 uses target samples Dr. (2) Generate an intermediate domain by uniformly sampling from [0, 1] and inputting the text prompt and random noise into the fine-tuned diffusion model corresponding to domain for the backward process.
Wap UW Wiid mid U W Wownn) using Eq. (5), where the first part with t
3.4
Generation-basee Unsupervised Domain Adaptation via Terra
Built on the first application introduced in the previous section, we introduce the second application of Terra in UDA. Under the UDA setting, we have labeled source domain Ds and an unlabeled target domain Dr. To alleviate domain shifts, we propose two-stage framework utilizing generation- based approach to augment the source domain.
Similar to the construction of evolving domains discussed in Section 3.3, the first stage sets out to train the parameters of Terra that accommodate source domain generation with and target domain generation with / = 1. This enables the generation of target images according to the class labels and transitive source images into the target domain. However, due to the polysemous words on the class labels, directly generating images with the text prompt may cause unexpected results. For example, "mouse" usually refers to a rodent, but in some datasets, it refers to computer mouse. Therefore, we leverage the source samples conduct semantic alignment between images and class labels while the unlabeled target domain samples contribute to learning style information for fine-tuning the diffusion model. To achieve this, adopt the same objective function Eq. (5), where set 0 for source training with the prompt "A [class]" and f. for target training with the prompt "An image".
The second stage involves synthesizing transitive source domain that can benefit the learning of UDA methods, as depicted in Fig. 3(a). We employ two approaches to achieve this. First, we set t= to synthesize target samples from Gaussian noises for each category with the corresponding prompt, i.e., "A [class]". Those synthesized samples constitute a generated target domain denoted by Dr. Second, transform the source samples into the target domain while preserving semantic
5

---

Contrastive learning loss
Stager
t=0
t=1 Denoise
D3
Recon loss
Invension
g
Text to Image Diffusion Modei
*A [eless]"
predictor
Terr
Denoise from
U
Stages
UDA loss
Genotee from
2, 10X
DF
The second stage of UDA method
(h) The two-stage framework of our DG method
Figure 3: The illustration of the proposed generation-based UDA and DG frameworks via Terra.
information. This is achieved by first setting the source images into noise. Then, with setting t = 1, we use the diffusion model equipped with Terra denoise, resulting in the adapted source domain gg. After generating images, we combine the adapted source domain and the generated target domain to form a transitive source domain DE D Dr. Here the transitive source domain could have smaller domain gap to the target domain than the original source domain due to the generation process, which could facilitate the knowledge transfer from the transitive source domain the target domain.
and applying DDIM inversion [55] convert
Finally, we conduct transfer learning from the transitive source domain to the target domain by using existing UDA method. The objective function is formulated
1
fuda = arg min
Σ
(ce(f(x). Bludo (DE, Dr),
(6)
DE
(
where - denotes the cross-entropp loss, 3 is a trade-off parameter, and Luda (:.)) is
transfer loss (e.g., domain discrepancy loss [34, 72, 93] and domain discrimination loss [13, 46, 85]) used alleviate the domain shift. In this manner, our method can be integrated with any off off-the-shelf UDA methods to enhance the transfer performance.
3.5 Generation-based Domain Generalization via Terra
In this section, we study the application of Terra to DG problems. Under the DG setting, we have K source domains [Dx = {(a] ) where nı denotes the number samples in Dr. To enhance the generalization capability.
shown in Fig. 3(b) and detailed as follows, Terra is adopted to synthesize new source domains by interpolating among existing source domains. Consequently, we expect more generalized learner that well adapts to both existing and synthesized source domains.
In the first stage, accommodate the various styles exhibited by multiple source domains, we utilize a network g(-) to predict sample-level for the Terra. The t-predictor g(-) aims to generate similar t values for images from the same domain. Moreover, due the diverse range of styles in the training set, each t = g(x) is represented as vector instead of a scalar value used in previous settings. This allows us to better capture various styles and intra-domain differences. Specifically, we train the network g(-) via contrastive learning and the loss function to be minimized is formulated as
Th
Lcon (g)
[o( Σ
o(x))))e
ma [|g(x;)) ((xxm))l))
(7)
where 8 a predefined positive margin and ||:1|2 denotes the Euclidean distance. In Eq. (7), the first term in the sum is enforce samples from the same domain yield similar outputs, while the second term is to encourage the distance between the outputs corresponding to samples from two domains to be larger than the margin via the hinge loss.

---

"A oil palnting of porson"
"A high- Speed train"
"A pet on the Lawx"
Figure 4: Qualitative evaluation. The three rows illustrate examples of morphing in image pairs, subjects, and styles, respectively. The text the left side represents the training prompts, with the red text indicating detailed descriptions used during inference. Additional examples and comparisons with other methods can be found in Appendix C.I.
Upon learning of the g(-), we can obtain t's for all the samples in all the source domains. Then based on t's, we fine-tune the diffusion model using Terra with the prompt "A [class]", and the training objective is formulated as
K
((20) = Be
Egg D Do.t=((00)) ||e - coot00 (a e(c) t)l2
(8)
After fine-tuning, in the second stage, we set t to various values to generate diverse samples for each category with the corresponding prompt. The generated samples could originate from various domains which may be beyond the original source domains {Dx R but we do not need to identify their specific domains. We combine these generated samples with the original source domain samples to form expanded domains DE. which can improve the generalization capability of models. The objective function of DG based on Terra is formulated
Sag = arg min
(ool((x))))) + (ldg((D)) (B.))E(DE)
Σ
DE
where 0 trade-off parameter, and Edg (.) is a domain generalization loss (e.g. Sharpness- Aware Minimization (SAM)-based loss [10, 5, 65] and representation learning- bbsed loss [1, 13, 2]) used improve the generalization capabilities. any off-the-shelf DG methods to enhance their performance.
this manner, our method can be integrated with
4 Experiments
4.1 Experimental Setups
For the UDA experiments, we utilize three benchmark datasets, including Office31 [51], which consists of 4, 10 images from 31 categories across three domains: Amazon (A), Webcam (W), and Dsir (D); Office-Home [59], containing 15,588 images from 65 categories across four domains: Art (Rw); and VisDA [43], featuring 207.785 images Synthetic and Real. For the DG experiments, we employ PACS [33], Office-Homme Home, and VLCS [9] datasets. The PACS dataset contains 9,991 images from seven categories across four domains: Art painting (A), Cartoon (C), Photo (P), and Sketch (S), and VLCCS contains 10,729 images from five categories across four domains: VOC2007 (V), LabelMe (I.), Caltech101 (C), and SUN09 (S). The basclines and implementation details put in Appendix B.
(Ar), Clipart (CI), Product (Pr), and
from 12 categories across two domains:

---

4.2 Experiments on Generative Interpolation Tasks
For generative interpolation tasks, we conduct qualitative and quantitative evaluations of our method, focusing on morphing in image pairs, subjects, and styles.
For morphing in image pairs, we train Terra by setting t 0 for the first image and t = 1 for the second one with a text prompt "An oil painting of a person". After training, we produce intermediate images by uniformly transitioning from O to with the same text prompt. The experimental results can be found in the first row of Fig. 4. We also provide qualitative comparisons with other baselines in Fig. 8. As can be seen, Terra produces natural and smooth interpolation between two images.
In addition to its ability to perform image morphing, Terra can perform style and subject morphing, capability that DiffMorpher [80] lacks. Due page limit, implementation details put in Appendix B. As shown in the second and third rows of Fig. 4, Terra is capable of generating a sequence of intermediate images seamless transition in styles and subjects.
To quantitatively evaluate the quality of the interme- diate images and the smoothness of the transition, we utilize the Frechet Inception Distance (FID) [23] and Perceptual Path Length (PPL) [29] metrics, following the setting in DiffMorpher [80]. As shown in Table I, the quantitative results demonstrate that Terra achieves comparable performance to DiffMorpher and outper- forms DGP, DDIM, and LoRA Interpolation. Note that DiffMorpher is specifically designed for morphing by customized techniques such attention interpolation, adaptive normalization, and a new sampling sched- ule. Equipped with the customized techniques used in DiffMorpher, Terra is even better than DiffMorhper.
Table 1: Quantitative evaluation of gener- ative interpolation tasks. We evaluate the fidelity and smoothness of the generated in- termediate images in terms of FID (4) and PPL (L).
image paizs
styles
subjects
FID
PPI.
FID
DXPP (GBAN-hased) [+1]
223,82
1.98
DDIM [5S]
176.34
1.35
LoRA Interp. [80)
89.37
0.91
256.64
1.02
194.17
1.44
DidfMorpher [80]
78.26
0.77
(curs)
62.25
18185
Tem+DiiTM
44.80
0.72
Table 2: Transfer accuracies (%) on the Office- Home and VisDA datasets under UDA setting. The best performance is highlighted in bold.
Home Pr-AAr
VirDM
Method
Ar-CI
Ar-PPr
Ar-RR
Cl-tAr
CI-Pt
CI-Rw
Pr-+CI
Rw->Ar
Rw-=C1
Rw-+Pr
Avg
mean
ERM 1581
44.06
67.12
74.26
53.26
61.96
64.54
51.91
38.90
72.94
64.51
43.84
75.39
59.39
$1.47
DANN
52.53
62.57
73.20
56.89
67.02
68.34
58.37
54.14
78.31
70.78
60.76
80.57
65,29
79.02
AFN 121
52.58
72.42
76.96
64.90
71.14
72.91
64.08
$1.29
77,83
72.21
57.46
82.09
67.99
74.64
CDAN [35]
54.21
72.18
78.29
61.97
71.43
72.39
62,96
55.68
80.68
74.71
61.22
83.68
69.12
80.74
MDD 1661
56.37
75.53
79.17
62.95
73.21
73.55
62.56
54,86
79.49
73.84
61.45
84.06
69.75
81.10
SDAT [4]]
58,20
77.46
81.35
66.06
76.45
76.41
63.70
56.69
82.49
76.02
62.09
85.24
71.85
83.23
MSGD [7]]
58.70
76.90
78.90
10
76.20
76.60
69.00
57.20
82.30
74.90
62,70
84.50
72.40
84.60
MCC [28]
56.83
82.66
77.02
77.82
72.24
83.32
MCC-Tam
63.49
81.51
83,46
72.52
82.89
81.25
73.20
61.66
83.16
74.36
63:45
84.41
75.45
85,
[85]
$7.79
77.65
81.62
66.59
76.74
76.43
62.69
56.69
82.12
75.63
62.85
85.35
71.84
83.40
ELS+Tema
64.62
82.33
83,60
71.19
84.25
80.31
73.00
63.57
83.81
76.20
66.56
85.70
76.26
86.86
4.3
Experiments on Unsupervised Domain Adaptation
In this section, we evaluate the proposed generation- based UDA method via Terra as introduced in Section 3.4. The comparison results against of-te-of-the-nne UDA methods the Office-Home and VisDA datasets are shown in Tables 2. Due to page limit, detailed results for VisDA and Office31 I are shown in Tables 7 and of Appendix C.2 and more results with CoVi and PMTrran are shown in Table 10. The standard deviations from three experiments are presented in Appendix D.
As can be seen, method has achieved significant performance improvements of 4.42%, 3.46%, and 1.07% for ELS on the Office-Home, VisDA, and Office31 datasets, respectively, surpassing all the baseline methods. Thus, Terra can serve as a good plugin for existing UDA methods.
The effectiveness of our method can be further verified through the t-SNE [57] visualizations, as depicted in Fig. 5. The adapted source domain D3 and the generated target domain Dr exhibit a smaller domain discrepancy to the target domain Dy than the original source domain, thereby reducing the domain gaps. Additionally, Fig. 6 presents example images illustrating the transformation from the source domain to the target domain. It can be observed that the style transfer is achieved while preserving the semantic information and subject shapes.
8

---

on Curae Soina Piad boraet Car
bencrta lrgar Deneen
Sours our
(a) Backpack
(b) Bike
(c) Drill
(d) Sneakers
Figure T-SNE visualization of the source domain, target domain, adapted source domain, and generated target domain in four classes of the Pr-CCI task on Office-Home under UDA setting.
"An alarm clock" bodd "A desk Lamp""
Art Cliport
Art Product
Art Real World
"A bottle" drill" "A backpack"
HELP!
Clipart - Art
Clipart Product
Clipart Real World
"A snoaker" "A computter" "A fax"
Product >> Art
Product Clipart
Product > Real World
"A bucket"
"A mouse"
"A shel??"
Real World -> Art
Real World >> Clipart
Real World > Product
Figure 6: Examples of the source images from Ds and corresponding adapted images from D3 for Office-Home tasks under UDA setting. The text prompts are shown on the left. For instance, first image pair showcases image from the Art domain and its corresponding generated image to Clipart domain based on the text prompt "An alarm clock".
Table 3: Ablation studies on the Office-Home Home dataset under UDA setting. The best in bold.
Method
Ar-+Cl
Ar-Prr Ar-Rw Rw
Cl-AA Cl->Pr Cl-Rw Pr-AA Pr-ICI Pr-Rw Rw-tAi Rw-ICI
Rw Avg
Ds + Dr
57.79
77.65
81.62
66.59
76.74
76.43
62.69
$6.69
82.12
75.63
62.85
85.35
71.84
D3 + Dr
61.25
78.89
80.71
79.03
75.59
66.50
60.84
80.55
73.30
65.10
84.97
72.92
D Dr
58.66
80.71
80.94
69.74
80.68
78.95
69.70
$4.20
81.68
71.92
56.98
82.03
72.18
DE Dr
64.62
82.33
83.60
71.19
84.25
80.31
73.00
63.57
83.81
76.20
66.56
85.70
76.26
The ablation studies of ELS+Terra presented in Table 3 show that the best performance is achieved when transferring from the expanded domain DD to the target domain Dy, validating the necessity and effectiveness of combining the adapted source domain with the generated target domain. To highlight the design advantages, we conduct comparison with SDXL's prior knowledge in Appendix C.5.
4.4 Experiments Domain Generalization
In this section, conduct experiments the PACS, Office-Home, and VLCS datasets to evaluate the effectiveness of our DG method proposed Section 3.5. The results presented in Table clearly reveal that our method achieves notable performance improvements across all tasks based three

---

Table 4: Accuracies (%) on the PACS and OfficeHome datasets under DG setting. The best is in bold.
PACS
OfficeHomme
Method
C
Avg
Ar
Rw
Avg
MIRO [6]
87.25
76.95
97.83
77.65
84.92
67.01
55.58
78.82
81.02
70.61
CDGA [22]
87.30
80.90
96.60
82.50
86.80
60.50
56.50
77.10
80.60
68.
ERM [58]
87.00
78.23
98.05
74.35
84,41
63.41
52.61
77.20
77.63
67.71
DRM+DomainDiff [36]
84,90
82.90
95.50
79.00
85.60
$7.60
49.20
73.00
75.20
63.70
ERM+Terra
89.51
79.66
98.20
78.64
86.50
65.43
53.79
78.99
80.30
69.63
SAGM [65]
85.72
81.13
96.59
77.46
85,23
65.55
55.09
78.68
79.39
69.68
SAGM+Terra
91.34
82.28
96.78
80.80
87.80
66.70
56.53
79.64
81.91
71.19
SWAD [5]
89.67
83.13
97.48
82.78
88.27
66.08
57.37
79.58
80.49
70.88
SWAD+Tene
91.07
83.50
18
84.62
89.34
68.02
58.31
82.03
72.23
09
641
04
-A1--ct---2aal cas
(a) P
(b) A
(c) C
(d) S
Figure 7: Visualization of learned time variables on the PACS dataset under the DG setting.
state-of-the-an art DG methods (i.e., ERM, SWAD, and SAGM). Furthermore, our method outperforms the stable diffusion generation-baset method, DomainDiff, which requires training separate LoRA for each source domain while our method maintains the parameter efficiency with only one single low-rank structure and different t capture diverse styles.
Additionally, Fig. 7 shows the learned values of The t predictor assigns distinct values each domain, enabling Terra to generate different interpolated images among the source domains based on varying values. Moreover, the random sampling clearer understanding of the rationale behind our approach that the generated samples may bring useful information for the target domain. We also show some generated images of the expanded domains on the PACS dataset in Fig. 10 of Appendix C. As can be seen, using Terra can generate diverse styles of images that are different from the source domains. With the expanded domains, the generalization capability of the source model can be improved.
effectively covers the target domain, offering
Besides, conduct an ablation study on the form of Terra and the dimensionality of in Appendix B, demonstrating that refining Terra's s form can further enhance its expressive power. We also compare Terra with other domain generalization morphing techniques, as shown Appendix C.4, to verify its effectiveness in expanding source domains for improved generalization.
Conclusion and Future Works
In this paper, introduce Terra, framework that facilitates effective cross-domain modeling through the construction of continuous parameter manifold. Terra incorporates a time-varying parameter within the manifold domains, enabling flexible and smooth interpolations. This approach facilitates effective knowledge sharing across different domains by training only single low-rank adaptor. Additionally, based on the designed generation-base strategies, Terra can serve as a plugin for existing UDA and DG methods to enhance performance. We also theoretically analyze the expressive capabilities of Terra. Extensive experiments demonstrate the superior performance of Terra in a range of tasks. For future works, we aim to extend Terra to cover more settings, including different modalities, larger datasets, and more complex tasks.
Acknowledgements
This work was supported by NSFC key grant 63136005 and NSFC general grant 62076118.
10

---

References
[1] M. Arjovsky, L. Bottou, I. Gulrajani, and D. Lopez-Paz. Invariant risk minimization. arXiv preprint 1907. 02893, 2019.
[2] G. Blanchard, A. A. Deshmukh, U. Dogan, G. Lee, and C. Scott. Domain generalization by marginal transfer learning. Journal of Machine Learning Research, 22(2):1-55, 2021.
[3] A. Brock, J. Donahue, and K. Simonyan. Large scale gan training for high fidelity natural image synthesis. In International Conference Learning Representations, 2018.
[4] H. Cao, C. Tan, Z. Gao, Y. Xu, G. Chen, P.-A. Heng, and S. Z. Li. A survey on generative diffusion models. IEEE Transactions on Knowledge and Data Engineering, 2024.
[5] J. Cha, S. Chun, K. Lee, H.-C. Cho, S. Park, Y. Lec, and S. Park. Swad: Domain generalization by seeking flat minima. Advances Neural Information Processing Systems, 34:22405-22418, 2021.
[6] J. Cha, K. Lee, S. Park, and S. Chun. Domain generalization by mutual-informattion regulariza- tion with pre-trained models. European Conference on Computer Vision, pages 440-457. Springer, 2022.
[7] Z. Chen, H. Li, F. Wang, O. Zhang, H. Xu, X. Jiang, Z. Song, and E. H. Wang. Rethinking the diffusion models for missing data imputation: A gradient flow perspective. Advances in Neural Information Processing Systems, 38:1 38:1-50, 2024.
[8] C. Eckart and G. Young. The approximation of one matrix by another of lower rank. Psychome- trika, 1(3):211-218, 1936.
[9] C. Fang, Xu, and D. N. Rockmore. Unbiased metric learning: On the utilization of multiple datasets and web images for softening bias. In Proceedings of the IEEE International Conference on Computer Vision, pages 1657-1664, 2013.
[10] P. Forct, A. Kleiner, H. Mobahi, and B. Neyshabur. Sharpness- awaree minimization for efficiently improving generalization. In International Conference on Learning Representations, 2020.
[11] R. Gal, Y. Alaluf, Y. Atzmon, O. Patashnik. A. H. Bermano, G. Chechik, and D. Cohen-Or, An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208. 01618, 2022.
[12] Y. Ganin and Lempitsky. Unsupervised domain adaptation by backpropagation. In Interna- lional Conference on Machine Learning, pages 1180-1189, PMLR, 2015.
[13] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. March, and Lempitsky. aovaariadversariall training of neural networks. Journal of Machine Learning Research, 17(59) 2016.
[14] R. H. Goldman. Transformations exponentials. Graphics Gems II, 2:332, 1991.
[15] R. Gong, W. Li, Y. Chen, and L. V. Gool. Dlow: Domain flow for adaptation and generalization. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 2477-2486, 2019.
[16] I. Goodfellow, J. Pouget-Abadie M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems, volume 27, pages 2672-2680, Jan. 2014.
[17] Y. Gu, X. Wang, Z. Wu, Y. Shi, Y. Chen, Z. Fan, W. Xiao, R. Zhao, S. Chang, W. Wu, et al. Mix- sh-w: Decentralized low-rank adaptation for multi-concept customization of diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
[18] Gulrajani and Lopez-Paz. search of lost domain generalization. International Conference Learning Representations, 2021.
[19] P. Guo, J. Zhu, and Y. Zhang. Selective partial domain adaptation. In BMVC, page 420, 2022.

---

[20] L. Han, Y. Li. H. Zhang, P. Milanfar, D. Metaxas, and F. Yang. Svdiff: Compact parameter space for diffusion fine-tuning. arXiv preprint arXiv:2303. 2023.
[21] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770-778, 2016.
[22] S. Hemati, M. Beitollahi, A. H. Estiri, B. A. Omari, X. Chen, and G. Zhang. Cross domain generative augmentation: Domain generalization with latent diffusion models. arXiv preprint arXiv:2312.05387, 2023.
[23] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in Neural Information Processing Systems, 30, 2017.
[24] J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-683 2020.
[25] E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang. L. Wang, and W. Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022.
[26] T. Huang, J. Liu, S. You, and C. Xu. Active generation for image classification. arXiv preprint arXiv:2403.06517. 2024.
[27] Y. Huang, J. Huang, Y. Liu, M. Yan, Lv, J. Liu, W. Xiong, H. Zhang, S. Chen, and L. Cao. Diffusion model-based image editing: survey. arXiv preprint arXiv:240..17525, 2024.
[28] Y. Jin, X. Wang, M. Long, and J. Wang. Minimum class confusion for versatile domain adaptation. In European Conference on Computer Vision, pages 464-480. Springer, 2020.
[29] T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila, Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 81 $110-8119, 2020.
[30] B. Kawar, S. Zada, O. Lang. O. Tov, H. Chang. T. Dekel, I. Mosseri, and M. Irani. Imagic: Text- based real image editing with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6007-6017, 2023.
[31] A. Kumar, T. Ma, and P. Liang. Understanding self-training for gradual domain adaptation. In International conference on machine learning, pages 5468-5479, PMLR, 2020.
[32] N. Kumari, B. Zhang. R. Zhang, E. Shechtman, and J.-Y. Zhu. Multi-concept customization of text-to-image diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1931-1941, 2023.
[33] D. Li, Y. Yang, Y.-Z. Song, and T. M. Hospedales. Deeper, broader and artier domain general- ization. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 5542-5550, 2017.
[34] M. Long, Y. Cao, Wang, and M. Jordan. Learning transferable features with deep adaptation networks. International Conference on Machine Learning, pages 97-105. PMLR, 2015.
[35] M. Long, Z.. Cao, J. Wang, and M. I. Jordan. Conditional adversarial domain adaptation. Advances in Neural Information Processing Systems, 31, 2018.
[36] Q. Miao, Yuan, S. Zhang, F. Wu, and K. Kuang. Domaindiff: Boost out-of-distribution generalization with synthetic data. In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 5640-5644. IEFE, 2024.
[37] C. Mou, X. Wang, L. Xie, Y. Wu, J. Zhang, Z. Qi, and Y. Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 4296-4304 2024.
12

---

[38] A. Q. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. Mcgrew, I. Sutskever, and M. Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. In International Conference on Machine Learning, pages 16784-16804. PMLR, 2022.
[39] C. C. Paige and M. A. Saunders. Towards generalized singular value decomposition. SIAM Journal on Numerical Analysis, 18(3):398 405, 1981.
[40] S. J. Pan, I. W. Tsang, J. T. Kwok, and Q. Yang. Domain adaptation via transfer component analysis. IEEE transactions on Neural Networks, 22(2):99--11, 2010.
[41] X. Pan, X. Zhan, B. Dai, D. Lin, C. C. Loy, and P. Luo. Exploiting deep generative prior for versatile image restoration and manipulation. In European Conference on Computer Vision, 2020.
[42] D. Peng, Q. Ke, Y. Lei, and J. Liu. Unsupervised domain adaptation via domain-adaptive diffusion. arXiv preprint arXiv:2308. 13893, 2023.
[43] X. Peng, B. Usman, N. Kaushik, J. Hoffman, D. Wang. and K. Saenko. Visda; The visual domain adaptation challenge. arXiv preprint arXiv:1/10.06924, 20111
[44] R. Penrose. A generalized inverse for matrices. In Mathematical proceedings of the Cambridge philosophical sociery, volume 51, pages 406 406-413. Cambridge University Press, 1955.
[45] D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. Müller, J. Penna, and R. Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXivv2307.019522 2023.
[46] H. Rangwani, S. K. Aithal, M. Mishra, A. Jain, and V. B. Radhakrishnan. A closer look at smoothness in domain adversarial training. In International Conference on Machine Learning, pages 18378-183888 18399. PMLR, 2022.
[47] M. D. M. Reddy, M. S. M. Basha, M. M. C. Hari, and M. N. Penchalaiah. Dall-e: Creating images from text. UGC Care Group I Journal, 8(14):71-75, 2021.
[48] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10695, 2022.
[49] N. Ruiz, Li, V. Jampani, Y. Pritch, M. Rubinstein, and K. Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages -2500-22511 2023.
[50] N. Ruiz, Y. Li, V. Jampani, W. Wei, T. Hou, Y. Pritch, N. Wadhwa, M. Rubinstein, and K. Aberman. Hyperdreambooth: Hypernetworks for fast personalization of text-to-image models. arXiv preprint arXiv:2307.066 00990.999999.6 2023.
[51] K. Saenko, Kulis, M. Fritz, and T. Darrell. Adapting visual category models to new domains. In European Conference Computer Vision, volume 6314, pages 213-226. Springer, 2010.
[52) V. Shah, N. Ruiz, F. Cole, E. Lu, S. Lazebnik, Y. Li, and V. Jampani. Ziplora: Any subject in any style by effectively merging loras. In European Conference on Computer Vision, pages 422-438 Springer, 2025.
[53] K. Shoemake. Animating rotation with quaternion curves. In Proceedings of the 12th annual conference on Computer graphics and interactive techniques, pages 245-254, 1985.
[54] K. Sohn, N. Ruiz, K. Lee, D. C. Chin, I. Blok, H. Chang, J. Barber, L. Jiang, G. Entis, Y. Li, et al. Styledrop: Text-to-ine toagee generation in any style. arXiv preprint 2tt.:23306000983.. 2023.
[55] J. Song, C. Meng, and S. Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2020.
[56) Y. Tewel, R. Gal, G. Chechik, and Y. Atzmon. Key-locked rank one editing for text-to-image personalization. In ACM SIGGRAPH 2023 Conference Proceedings, pages 1-11, 2023.
13

---

[57] L. Van der Maaten and G. Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(1),, 2008.
[58] V. Vapnik. The nature of statistical learning theory. Springer science business media, 1999.
[59] H. Venkateswara, J. Eusebio, S. Chakraborty, and S. Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5018-5027, 2017.
[60] V. Verma, A. Lamb, C. Beckham, A. Najafi, I. Milliagkas, D. Lopez-Paz, and Y. Bengio. Mani- fold mixup: Better representations by interpolating hidden states. In International Conference on Machine Learming, pages 6438-6447. PMLR, 2019.
[61] C. Wang. J. Gao, Y. Hua, and H. Wang. Cross-domain learning with normalizing flow. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5, IEEE, 2023.
[62] C. J. Wang and P. Golland. Interpolating between images with diffusion models. arXiv preprint arXiv:2307.125600 2023.
[63] H. Wang. J. Fan, Z. Chen, H. Li, W. Liu, T. Liu, Q. Dai, Y. Wang, Z. Dong, and R. Tang. Optimal transport for treatment effect estimation. Advances in Neural Information Processing Systems, 36:1-1:: 2023.
[64] J. Wang. C. Lan, C. Liu, Y. Ouyang, T. Qin, W. Lu, Y. Chen, W. Zeng, and S. Y. Philip. Generalizing to unseen domains: A survey on domain generalization. IEEE Transactions on Knowledge and Data Engineering, 35(8):8052-8072, 2022.
[65] P. Wang, Z. Zhang. Z. Lei, and L. Zhang. Sharpness-award gradient matching for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3769-3778 2023.
[66] Q. Wang, X. Bai, H. Wang, Z. Qin, and A. Chen. Instantid: Zero-shot identity-preserving generation in seconds. arXiv preprint arXiv:2401.07519 2024.
[67] X. Wang, P. Guo, and Y. Zhang. Unsupervised domain adaptation via bidirectional cross- attention transformer. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 309-325. Springer, 2023.
[68] X. Wang, K. Yu, C. Dong. X. Tang, and C. C. Loy. Deep network interpolation for continuous imagery effect transition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1692-1701, 2019.
[69] X. Wu, S. Huang, and F. Wei. Mixture of loRA experts. In International Conference on Learning Representations, 2024.
[70] H. Xia and Z. Ding. Structure preserving generative cross-domain learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4364 4373, 2020.
[71] H. Xia, T. Jing, and Z. Ding. Maximum structural generation discrepancy for unsupervised do- main adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):3434- 3445, 2023.
[72] R. Xu, G. J. Yang, and L.. Lin. Larger more transferable: An adaptive feature norm approach for unsupervised domain adaptation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 1426-1435, 2019.
[73] G. Yang. H. Xia, M. Ding, and Ding. Bi-directional generation for unsupervised domain adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 6615-6622. 2020.
[74] Q. Yang, Zhang, W. Dai, and Pan. Transfer learning. Cambridge, U.K.: Cambridge Univ. Press, 2020.
14

---

[75] F. Ye, X. Wang, Y. Zhang, and I. W. Tsang. Multi-tass learning via time-aware neural ode. In International Joint Conference on Artificial Intelligence, pages 4495-4503, 2023.
[76] H. Ye, J. Zhang, Liu, X. Han, and W. Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.06722 2023.
[77] Zadouri, A. Üstün, A. Ahmadian, B. Ermis, Locatelli, and Hooker. Pushing mixture of experts to the limit: Extremely parameter efficient moe for instruction tuning, In The Twelfth International Conference on Learning Representations, 2023.
[78] Y. Zeng and K. Lee. The expressive power of low-rank adaptation. In International Conference on Learning Representations, 2023.
[79] H. Zhang, M. Cisse, N. Dauphin, and D. Lopez-Paz. mixup: Beyond empirical risk mini- mization. In International Conference on Learning Representations, 2018.
[80] K. Zhang, Y. Zhou, X. Xu, X. Pan, and B. Dai. Diffmorpher: Unlcashing the capability of diffusion models for image morphing. arXiv preprint arXiv:2312.07409, 2023.
[81] L. Zhang, A. Rao, and M. Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 3836-3847 2023.
[82] P. Zhang, H. Dou, Y. Yu, and X. Li. Adaptive cross-domain learning for generalizable person re-identification. In European Conference on Computer Vision, pages 215-232. Springer, 2022.
[83] X. Zhang, X.-Y. Wei, W. Zhang, J. Wu, Z. Zhang, Z. Lei, and Q. Li. A survey on personalized content synthesis with diffusion models. arXiv preprint 2rX5::2405.05538 2024.
[84] Y. Zhang, S. Chen, W. Jiang, Y. Zhang, J. Lu, and J. T. Kwok. Domain gaaiddd conditional diffusion model for unsupervised domain adaptation. arXiv preprint arXiv:2309, /4360, 2023.
[85] Y. Zhang, J. Liang, Z. Zhang, L. Wang, R. Jin, T. Tan, et al. Free lunch for domain ad- versarial training: Environment label smoothing. In International Conference on Learning Representations, 2023.
[86] Y. Zhang, T. Liu, M. Long, and M. Jordan. Bridging theory and algorithm for domain adaptation. In International Conference on Machine Learning, pages 7404-7413, PMLR, 2019.
[87] Y. Zhang, Y. Yao, S. Chen, P. Jin, Y. Zhang, J. Jin, and J. Lu. Rethinking guidance information utilize unlabeled samples: A label encoding perspective. In International Conference on Machine Learning, 2024.
[88] P. Zheng, Y. Zhang, Z. Fang. T. Liu, D. Lian, and B. Han. Noisediffusion: Correcting noise for image interpolation with diffusion models beyond spherical linear interpolation. In International Conference Leurning Representations, 2024.
[89] K. Zhou, Z. Liu, Y. Qiao, T. Xiang, and C. C. Loy. Domain generalization: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45:4396 4415, 2023.
[90] K. Zhou, Y. Yang, Y. Qiao, and T. Xiang. Domain generalization with mixstyle. In International Conference on Learning Representations, 2020.
[91] J.-Y. Zhu, Krähenbühl, E. Shechtman, and A. A. Efros. Generative visual manipulation on the natural image manifold. In European Conference Computer Vision, pages 597-613, Springer, 2016.
[92] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision, pages 2223-2232, 2017.
[93] Y. Zhu, F. Zhuang. J. Wang. G. Ke, J. Chen, J. Bian, H. Xiong, and Q. He. Deep subdomain adaptation network for image classification. IEEE Transactions on Neural Networks and Learning Systems, 32(4): 2020.
15

---

[94] F. Zhuang, Z. Qi, K. Duan, D. Xi, Y. Zhu, H. Zhu, H. Xiong, and Q. He. A comprehensive survey on transfer learning. Proceedings of the IEEE, 109(3))3-, 2020.
[95] Z. Zhuang, Y. Zhang, and Y. Wei. Gradual domain adaptation via gradient flow, International Conference Learning Representations, 2024.
16

---

A Proofs
Lemma 1. Given two matrices A, E Rmxn and rank(-) denotes the rank of a matrix. Then, rank([A BI) S. rran(()) + rank B) and rank([A" B²I) S rank (A) + +rank(B)
Lemma 2. Given two matrices E RR B E Rnxx and rank(.) denotes the rank of a matrix. Then, rank([AB)) S min (rank(A), rBn)
Theorem 3. (Generalized Singular Value Decomposition (GSVD) (39] For given matrices A, B E Rmxn, let CT [A] B²] and denote its rank by rank( (C), there exist orthogonal matrices
UA UB E Im Q Raax and W E &Rx so that
UA AQ EA ( R. 0].
UB BQ ΣB
[W
R, 0].
(10)
IA
OB
ΣA- =
SA
ΣB =
SB
(11)
OA
lB
where R is real diagonal contains the nonzero singular values of C in decreasing order, EA, EB E mmak real non-negative block-diagonal matrices, where IA E "R" and IB E RR are identity matrices, O E Rm - and OB E ER" m- m- are zero matrices with possibly no
columns, and S ar++1.......... 0r and SB 13,41,..., Br+ss And we have
1 > a,+1 2- ... 2 ar+s 0, 0 < Brt1 S ... Br+s 1, af + B =1 i be
Indeed, GSVD is a powerful tool in numerical linear algebra and data analysis. It can be seen as an extension of the singular value decomposition (SVD). Notably, when the matrix B the identity matrix, the GSVD of matrix A and B simplifies to the SVD of matrix A.
Theorem 1.
(The Equivariance between Terra and Multiple LoR As) Assume there exist two LoRAs AWA, AWB E Rmxn with ranks of p and respectively, that effectively solve two specific down- stream tasks. Let k max [rax(ra[[[AAW) A AW B)), rank([AW') AW (l)), where rank(-) denotes the runk of a matrix. Then, there exists a Terra with W up E Rm W down E BRxxn W mid E IR Beokk, and KC(t) tW mid + I,, such that the updated matrix (t W Kt) W down. can simultane- ously solve the two downstream task, that is, we have AW (0) = AW A and (1 = AW B.
Proof. Our goal is find matrices W W down. and W mid to satisfy AW W W down AW A. and A W (1) = Wup (W mid + I)W down = AW B-
From Theorem 3, since k 2 rank([A T. AWT)). we know GSVD can decompose the two LoRA adapters with a common right generalized singular vectors X E kxn,
AWA UAΣAX, AW UR
(12)
Similarly, we can transpose the matrices of the two LoRA adapters, since l: 2 rank[[AW A AW, n)), and use GSVD again, then we have common left generalized singular vectors Y E Rmak
AWA A YZAVA. AW YZB
(13)
For each matrix W, there exists pscudo-inverse (a.ka. the Moore-Penrose inverse [44]) W such that WW+ W = W. Then we have a special decomposition of the LoRA adapters:
AWA = UAΣAX
UXEX(XX
(14)
A
AY YY X
where KA E Rixk Similarly, we have:
AW = Y A YKRX.
(15)
Assume the SVD of KA is of the following form: KA UKAVK.
(16)
17

---

where UK, VK E R are orthogonal matrices.
We represent the diagonal matrix A A diag(01, 02...., "25....90r,00 O,....... first r rows have 1/0 1/02,..., 1/0, on the diagonal, and the product of and A A a square matrix whose first diagonal entries are and whose others are 0, i.e. I.- Then, can get
0). Define A whose
(W
Y (I(KB KA) + KA)X
Y((KBB KA) + UKAVK)
-YUKKK T (U(BB - -AA) + UKAVK) VK VK VRX
= =YUK (iUK T (KB KK T +A) VKX
(17)
YUK (UUK' T (KB KKVVK"A++H, AVKX
Wup up ((W I,) W down-
Finally, we can construct the following matrices to prove the theorem:
W... up YUK,
W mid UK (KB KA) W down AVRX.
(18)
Theorem 2.
(The Expressive Power of Terra) For each layer I, the rank-k Terra has updated matrix ( and the function of time-varying matrix is K((1; = IW mid,1 + I. Assume that all weight matrices of the frozen model (Wi) I W, + (R,), and I Wi IR, non-singular for all & k(I R(I-1). Then the approximation error satisfies
IL + AW())
W AW())) AW())) - WA +
L w. AW()) - Wn n
min
S 20
(4)
where the ok441 as the (kL + 1)-th largest singular values obtained by merging the singular values of EA and En. Moreover, when k > RRE +RE the approximation error is zero.
Proof. We first adopt similar construction consistently with the prior work [78]:
SA = II (W - II W. SB = II (Wi + (W())) -II W
(19)
E=1
1=1
1=1
Then, approximate error can be represented as:
1 +
I w. + AW())) WA
min (W(
AW())) WT
[=1
(20)
min
SSA EAll S EB
AW())
Following the prior work, we can also decompose SA into an accumulation of SA, as follows:
T-1
L-2
SA (0) II
+ W/
II
(AW(O), + Wi)
1=
II [[ W.
(AW(0), + W.)
II W
(21)
V=2
1=1
w.
AW(
(W, AW (0).)
:
18

---

Similarly, we have SB = SB SB:
(Ti W:) AW()), (T ( (L).).
We select the largest &L largest terms of the singular values of E and EB, and we denote there are p values from E and q values from EB. To prove the theorem, need show the following:
min (IIS - Ealla + S| Enll2) S 20++1 AW())
(22)
Define EAA = LR A), FBB LR, (EB).
BB.). based on the Eckart- Young Theorem [8], then we have:
|| E Eal12 + E --- -Enll2 I (pti(EA) + a (EB) S 20
(23)
Based on (22) and (23), if we can construct Terra parameter A AW(P (t) to make SA E'A and SB EB then will finish the proof. We refer to the SVD of E'A and E
E = UAAAVA.
EB = UBAB VB.
(24)
We introduce QA and Q Bd to divide E'A and E'D into L parts:
Σ EAQAAI E'A 1=1
Σ E E I=1
(25)
We define Lab as diagonal matrix whose diagonal entries from the a-th to b-th position are 1 and others are O. Here define the matrices (QA,))=. and (QB,)k= by:
A. = 1A V QB. =0,
for Rl < p.
Q =
V Qn. = I VB
for p $ Rl <pt4,
(26)
QA.- O, QB.1 VBIRR 1) VB
for + Rl.
easy to find that rank A.)) + rank (Qp.) S R. Based on Lemma 1, we have
rank (( (AA F 2B $ k, rank ([(E'QAA)) (EBQB,)) k
(27)
Now, we show a feasible solution to make S E'A and SB = E'B follows these conditions:
AW = III w -
A.
(W, AW(O))) for all / [L],
(28)
L
AW II W:) E E B,
(W; N.W:AAWON AW(1))) for all E [4].
(29)
(=++1
1=1
rank ( AW (0)) rank (W: ( D, for Borall E [T. ---1] 1].
(30)
Based on the assumptions of (Wi) DE I' W + LR, (EA), and Ih Wi + LR, are non-singular for all S k(L 1) and the Eq. (28) and Eq. (29). it's easy prove that Eq. (30) is satisfied [78].
Using the Lemma 2 and Eq. (27), show rank (/AW(O) AW(i))) s by
rank ([a(()) AW(1).)
1-
(w AW()))- 1)
(I AW
rank [EAQAA EBQBI]]
rank ([EAAQA, (BB)..)))))))) <
Similarly, also get rank (lA)()) AW(1)) & k. Then, based on Theorem 1, for each layer I, we can prove that there exists Terra can satisfies AW(O); AW(O), and AW(1,, =
AW thereby completing the proof.
19

---

B Baselines and Implementation Details
Baselines. For the generative interpolation tasks, we use DGP [41], DDIM [55], DiffMorpher [80],
and LoRA Interpolation for comparison. DGP leverages large-scale pre-trained GAN [3] for im-
age morphing. DDIM means the DDIM inversion and latent interpolation as discussed in [55].
DiftMorpher performs image morphing between two images by interpolating corresponding two
LoRAs and latent noises. LoRA Interpolation represents directly training two LoRAs and performing interpolation. For the UDA tasks, we compare with ERM [58] and various UDA methods, including
AFN [72], MDD [86], MCC [28], DANN [13], CDAN [35], SDAT [46], ELS [85], and MSGD [71].
We integrate the proposed Terra with the state-of- fi--the-art art UDA methods, i.e. MCC, and ELS. For
the DG tasks, we compare with ERM [58], MIRO [6], CDGA [22], SWAD [5]. SAGM [65], and
DomainDiff' [36]. Note that previous works [18, 5J have found that ERM is effective in DG and
outperforms previous DG methods. Thus, we integrate Terra with ERM and the state-of-the-rr DG methods, i.e., SWAD, and SAGM.
Implementation Details. The text-to-image diffusion model used in this paper is the Stable Diffusion XL (SDXL) model [45]. The default resolutions of the generated images are 1024 1024. The rank of LoRA is set as 16 for generative interpolation tasks and 32 for generation-base UDA NVIDIA A100 GPU with three random trials.
and DG tasks. All experiments are conducted
For generative interpolation tasks, the training data utilized is sourced from the repository of Diff- morpher' and the LoRAs from Hugging Face space "LoRA the Explorer" The training images can be found in the supplementary materials provided. For morphing in styles, given images crayon and watercolor styles for training, we set = 0 for training on the crayon images and L 1 for training on the watercolor images, with the prompt being "An image". During the inference phase, by uniformly transitioning t from 0 to 1 and using the text prompt "A high-speck train", the generated results are shown in the second row of Fig. 4. For morphing in subject, given five images of cats and eight images of dogs, we set f = 0 for training
the cat images and t 1 for training the dog images, with the prompt being "A pet". During the inference phase, by uniformly transitioning t from O to 1 and using the text prompt "A pet on the lawn", the generated results shown in the last row of Fig. 4.
For UDA tasks, We generate 50 images per category for the Office31 and Office-Home, and 1000 images per category for the VisDA datasets. For images translated from the source domain to the target domain, we scale the long side of each source image to 1024 pixels, adjusting the short side proportionally. Following [46], the ResNet-50 is used as the backbone on the Office31 and Office-Home datasets, and the ResNer-101 [21] is used as the backbone on the VisDA-2017 dataset.
The learning rate scheduler follows [13]. For MCC+Terra and ELS+Terra, we follow the settings as the original papers [28, 85].
For DG tasks, we generate 400, 160, and 400 images per category for the PACS, Office-Home and VLCS datasets, respectively. The dimension of parameter ł is set as two, with each dimension sampled from -2 to 2 at intervals of 0.1 to generate diverse samples. We employ ReyNel-50 as the backbone and adopt the same training, evaluation protocols, and hyperparameter search results as outlined in [5, 65, 6]. R0 also used as the backbone for prediction network g(-).
Table 5: Possible forms of Terra and corresponding differentiable functions.
General
Diagonal
Linear
Exponential
Cosine
F(W,t) th 4F(W,t) W
IW + I
exp(tW) W exp(tt)
cos((W)
- -ii(tW) I
F(W.0)
Jr
Phaps://gituu..com/KKc http:://hhftccmmm/eeeeeeet//ee/e
20

---

We provide three possible forms of Terra listed in Table 5, L.e., Linear, Exponential, and Cosine. We apply the Linear form of Terra for generative interpolation and UDA tasks and the "Cosine-Sine" form, variant of "Cosine", for DG tasks. Specifically, the form of "Cosine-Sine" is cos(IW) on the diagonal of K (t), and sin( at other positions. provide more insights, we elaborate on the guiding principles behind the choice of these forms:
The #W is the simplest form, related to straight and steady flow, which sufficient for two domains according to Theorem I and 2. Its constant velocity of weight changes ensures smooth morphing and is suitable for simple interpolating between two domains under the UDA setting.
Linear:
Cosine-Sine: This form is adopted because of the bounded range and non con-linearity of trigonometric functions, preventing image collapse during generation and enabling complex parameter manifold capture relationships between multiple domains. We recommend using this form in complex scenarios, such as interpolating multiple domains in DG.
Exponential: e I+ Ek=1 E W implemented using "torch exp", also defines smooth curve in high-dimensional manifold. This form is more expressive and suitable for handling multiple domains. Notably, related to three types of transformations: scalings, rotations, and shears [14].
Table 6: Evaluation on the dimension of time variable and Linear form (dim2) of Terra on the PACS dataset under the DG setting. The best is in bold.
PACS
Method
A
C
S
Avg
ERM
87.00 77.00=a..o
78. 78.23
98.0540.00
74.35
84,41
TRM+Terra (diml)
88.29 1.33
82.36
97.53
73.31
85.37
ERM+Terra (dim2)
89,51 20.67
79.66=0.03
98.20 1:000
78.6412.08
86.50
ERM+Tema (dim3)
09.26=11700
81.72=2.22
97.94
+1.11
86.26
ERM+Terra (Linear)
87.47
80.17 20.46
97.85
77.1641.50
85.66
Empirically, the "Cosine-Sine" form of Terra brings better performance for DG compared with the Linear form according to the results shown in Table 6. As can be seen, ERM+Terra with dimension achieves the best average performance, thus we use 2 as the default dimension for DG tasks.
C More Experimental Results
C.1 Results on Generative Interpolation Tasks
DCP (GANbeal)
LoRA Inters
DiTMuppher
Terms DHIM.
Figure 8: Qualitative results of image morphing using various methods. "Terra DiffM." integrates Terra with DiffMorpher. As shown, our method generates smooth and natural intermediate images.
The qualitative comparisons of image morphing using various methods are shown in Fig. 8. We perform more qualitative samples of our Terra in Fig. 9. These samples further demonstrate Terra's ability to handle morphing under various scenarios.
21

---

Figure 9: Supplementary samples of qualitative evaluation. The four rows display examples of morphing in image pairs, styles, and objects (purple-to-dog bags, colorful-to-shing sneakers).
Cartoon
hoto
Sketch
Generated samples
Figure 10: Example images of the expanded domains on the PACS dataset under the DG setting.
C.2 Results on More Datasets
Table 7: Transfer accuracies (%) on the VisDA dataset under UDA setting. The best is in bold.
Method
acro
bicycle
bus
horse
knife
motor
plant
skate
train
truck
mean
ERM [S8]
81.71
22.46
54.08
76.21
10.69
83.81
80.88
28.66
79.66
5.98
$1.47
DANN [13]
94.75
73.47
83.46
47.91
87.00
88.30
88.47
77.18
88.16
90.05
87.21
42.26
79.02
AFN [72]
93.13
54.76
81.03
69.74
92.36
75.88
92.11
73.83
93.16
55.55
90.48
23.63
74.64
CDAN [35]
94.55
74.41
82.22
58.92
90.56
96.22
89.71
78.90
86.11
89.06
84.81
43.42
80.74
MDD [6]]
92.68
65.26
82.29
66.78
91.68
92.09
93.18
79.67
92.12
84.95
83.85
48.66
81.10
SDAT [46]
94.51
83.56
74.28
65.78
93.00
95.83
89.61
80.04
90.86
91.47
84,95
54.93
83,23
MSGD [71]
97.50
83.40
84.40
69.40
95,90
94. 10
90.90
75.50
95.50
94.60
88.10 10
44,90
84.60
MCC [28]
95.26
86.14
69.98
92.83
94.84
86.52
90.26
90.98
85.68
52.52
83.32
MCC+Terra
96.20
87.27
78.77
70.59
94.18
95.49
85.08
85,48
92.24
93.20
86.26
59.88
85,39
ELS [85]
94.76
83.38
75.44
66.45
93.16
95.14
89.09
80.13
90.77
91.06
84.09
57.36
83,40
ELS+Terra
95.98
87.12
81.60
70.84
95.14
96.29
88.47
87.78
94.75
94.06
86.47
63,83
86.86
22

---

Table Transfer accuracies (%) on the Office31 dataset under the UDA setting.
The best in bold.
Method
A-*W
D
W-+D
A->D
D-A
W-+A
Avg
ERM [58]
17.07
96.6010.00
99.2010.00
81.08
64.11
64.01
80.35
DANN [13]
89.8511.34
97.9510.08
99.9010.08
83.26
73.28
73.75
86.33
AFN [72]
91.82+0.63
98.77
100.00.0.00
95.12+0.68
72.43++0.00
70.71
88.14
CDAN [35]
92.42.
98. 18.12
100.00.0.00
91.44411.19 1.11
74.61 1.0.79
72.80
88.32
MDD [86]
93.5541.00
98. 08.66
100.00 2..00
93.92.10.10
75.29 4.0.68
73.95
89.23
SDAT [46]
91.32:1.88
98. 1.13
100.00
95.25
76.97=20.67
73.1940.34
89.26
MSGD [71]
95,50
99.20+0.30
100,00=0.00
95.60
77.30
77.00
90.80
MCC [28]
94.09 ss
98. 2..3220..08
99.67.10.09
94,25
75.89
75.4610.20
89.61
MCC+Terr
94,55
99.03
100,00
96,46 4:0
78. 78.64
79.37
91.34
ELS [85]
93.8440.61
98. +8.78
100.00
#0.00
95.78+0.22
77.72+t.64
75.13
90.21
ELS+Terra
94.09
99.21 +0.06
100.00 20.00
96.25
78.67 +0.28
79.45 40.11
91.28
Table 9: Testing accuracies (%) the VLCS dataset under the DG setting, The best is in bold.
VLCS
Method
C
Avg
MIRO
98. 10
64.05
73.3 1.77
76.36 78
77.95
ERM
97.76
63.11
72.17 72.17
76.56
77.40
ERM+Terra
98.79 65.5441.07
71.04
77.66+0.43
78.25
SAGM
98.35 65.29
75.22
79.13 79.13
79,50
SAGM+Terra
99.21
66.5220.31
73.95=00.00
80,80 40.31
80.12
SWAD
98.74
62.70.0.44
74.09
75.64
77.79
Terr+Terra
98.94
63.98
73.91
80.19 0.226
79.26
The complete results on the VisDA dataset under the UDA setting are shown in Table 7. The results on the Office31 dataset under the UDA setting are shown in Table and the results on the VLCS dataset under the DG setting are shown in Table 9, As can be seen, Terra is still effective in the two datasets.
C.3 Results with More Baselines
Moreover, the results with CoVi and PMTrans are shown Table 10. Notably, Terra consistently improves performance in all tasks with those UDA methods, further verifying the effectiveness of our method.
Table 10: Comparative analysis with two baseline methods Office-Home under UDA setting.
Methed
Ar-+CI
Ar-Pr Ar-Rw
Cl-+Ar Cl-+Pr Cl-RR
Pr-AAr Pr-+C
Pr-RW Rw-AAr Rw-+CI Rw-+Pr
Avg
CoVi
58.50
77.00
60.20
63.60
86.50
73.10
CoVi-Tera
64.56
80.65
83.36
71.45
81.03
80.77
70.83
64.86
84,07
76.76
64.19
87.18
75.81
PMTrans
82.17
91.55
92.36
89.40
92.48
92.49
87.92
80.57
92.88
88.94
82,34
94.45
88.96
PMTransTTerra
83.57
94.50
89.78
C.4 Comparison of Morphing Works
In addition, for fair comparison of Terra's effectiveness in expanding source domains that generalize better, we include the comparison against off-the-shelf DG + morphing works on Office-Home. That is, we train a A for each domain and adopt LoRA Infero/DDiffMorpherr to interpolate. The results shown in Table 11 verify the effectiveness of Terra, since Terra interpolates between domains instead of images and thus better models the distributions in two domains.
23

---

Table 11: Comparison morphing works on Office-Home using the off- aff-the-shelf method (SWAD)
under DG setting. Note that DiffMorpher exhibits lower performance due to the large gap between image pairs, even within the same class.
Method
Ar
C1
Pr
Rw
Avg
SWAD
66.08
57.37
79.58
80.49
70.88
+DiffMorpher
64.06
57.43
77.91
81.04
70.11
+LoRA Interp.
67.23
58.06
80.09
81.33
71.68
+Terra
68.02
58.31
80.56
82.03
72.23
C.5 SDXL Prior
To further highlight the design advantages of Terra, we conduct comparison with data augmentation with SDXL's prior knowledge. Specifically, we design several methods to synthesize data based on the SDXL model and evaluate their effectiveness on UDA tasks:
(i) SDXL (random): We use the prompt "A [CLASS]" generate samples for each class, where [CLASS] denotes the placcholder for the label.
(ii) SDXL (styles): We first use the prompt "Generate 50 prompts describing diverse styles for image generation" to ask GPT-4, and then the prompt "A [CLASS), everyday object in office and home, in the style of [STYLE]" to generate samples, where [STYLE] denotes the placeholder for style prompts generated by GPT-4 (e.g. "Classic". "Modern").
(iii) SDXL (target): Based on (ii), we use the name of the target domain (e.g. "Clipart") to replace the [STYLE] as the new placeholder for exploring the SDXL prior on the target domain.
(iv) SDXL (target styles): We the prompt "Generate 50 prompts describing [TARGET] style for image generation" to ask GPT-4 and obtain more detailed style prompts for synthesis.
(v) SDXL (selected): Inspired by [26], we use a confidence-biac based activate learning method to filter out poor-quality and misclassified samples generated in (iv) and select valid samples.
The comparison results on Office-Home for UDA are shown in Table 12. Terra outperforms the comparison methods, indicating that despite the boost in accuracy from target style design and active learning, the prior knowledge is insufficient to align with the downstream tasks. This issue can be further mitigated through finetuning with Terra, which demonstrates the design advantages of Terra.
Table 12: Comparison of target-like samples generation by SDXL prior on Office-Home based on ELS under UDA setting.
Method
Ar-CI
Ar-Pr
Ar- Rw
Cl-Ar
CI+Prr
Cl-RR
Pr-tAr
Pr-+CI
Pr-tR
Rw-tAt Rw-+CI
Rw-Pt
Avg
(random)
56.88
SDXL (styles)
55.23
77.09
80.26
68.11
77.09
80.26
68.11
55.23
80.45
68.11
55.23
77.04
70.18
SDXL (target)
66.67
SDXL (torgot styles)
60.76
79.
81.68
70.95
79.52
81.68
70.95
60.76
81.68
70.95
60.76
79.52
73.23
SDXL (selecte))))
61.63
79.81
82.19
71.98
79.73
81.82
71.69 .9
61.58
82.07
72.76
62.15
80.42
73.99
Terra
64.62
82.33
83.60
71.19
84.25
80.31
73.00
63.57
83.81
76,20
66.56
85.70
76.26
D Standard Deviations of Experiments
The standard deviations of three random experiments the Office-Home, VisDA, and ablation studies under UDA setting are shown in Tables 13, 14, and 15, respectively. Table 16 presents the standard deviations on the PACS and OfficeHome datasets under DG setting.
E
Comparison with Other LoRA Variants
cross-domain learning based on MoLE [69], the process can be viewed as first training LoRAs on different domains separately, followed by training gating function to integrate the trained LoRAs. Although both MoL. and Terra are designed for diffusion model customization, they differ several
key aspects:
24

---

Table 13: The standard deviation of three random experiments Office- under UDA setting.
Method
Ar-CI
Ar-PPr
Ar-Rw
Cl-+Ar
CI-+r
CI-RRw
Pr-Ar
Pr-CI
Pt-Rw Rw-Ar
Rw-CI
Rw-Pr
ERM [58]
0.25
0.26
0.42
0.17
0.20
0.15
0.07
0.17
0.05
0,34
0.33
0.01
DANN [13]
0.72
0.38
0.02
0.30
0.39
0.58
0.47
0.14
0.51
[72]
0.06
0.14
0.15
CDAN [35]
0.25
0.62
0.22
0.37
0.58
0.30
0.57
0.36
0.16
0.33
0.23
0.35
MDD [86]
0.51
0.32
0.06
0.24
0.73
0.41
0.36
0.53
0.24
0.03
0.09
0.11
SDAT [46]
0.51
0.44
0.24
0.13
0.41
0.01
1.46
0.40
0.11
0.46
0.19
0.29
MCC [28]
0.59
0.22
0.16
0.27
0.52
0.16
0.16
0.38
0.25
0.35
0.35
0.23
MCC+Tera
0.21
0.11
0.14
0.25
0.28
0.18
0.18
0.29
0.25
0.15
0.06
0.11
[85[
0.38
0.19
ELS+Tera
0.06
0.30
0.14
0.30
0.37
0.21
0.10
0.18
0.13
0.68
0.24
0.16
Table 14: The standard deviation of three random experiments on VisDA under UDA setting.
Method
bicycle
hus
horse
knife
motor
person
plant
skate
train
truck
ERM [58]
9.90
2.64
3.26
2.20
1.35
3.60
1.41
1.03
1.80
3.97
0.79
0.67
DANN [13]
0.39
1.94
0.38
2.80
0.80
3.40
0.76
0.86
0.72
2.00
0.32
2.72
AFN [72]
0.69
3.84
1.80
2.55
1.48
2,51
0.48
2.08
2.47
3.93
1.11
1.27
CDAN [35]
0.38
3.72
2.55
1.36
0.53
0.52
0.14
2.58
0.67
0.49
2.61
2.43
MDD [86]
2.40
9.46
1.18
0.66
0.85
4.01
0.65
1.81
1.21
4.30
1.58
0.33
SDAT [46]
1.40
2.64
1.60
1.67
0.48
0.92
0.82
0.24
0.78
0.84
1.36
0.60
MCC [28]
0.12
0.92
2.91
0.39
0.28
0.54
0.80
0.87
0.15
0.77
0.55
2.25
MCC+Terra
0,21
0.59
0.12
0.69
0.60
0.56
0.35
0.40
0.35
0.47
ELS [85]
0.93
1.20
1.39
0.47
0.15
0.95
1.38
0.73
1.59
1.02
1.35
0.27
ELS+Terra 0.34
0.79
0.41
1.31
0.06
0.39
0.85
0.43
0.92
0.64
0.26
0.19
Table 15: The standard deviation of three random experiments of ablation studies of ELS+Terra on Office-Home under UDA setting.
Method
Ar Ar-CCI Ar Pr
Ar-RW CI-Ar CI-Pr CI-tRw Pr-AAr Pr--CI Rw-t& Pr-Rw
Rw Rw-+CI
Rw->Pr
Ds >> Df
0.83
0.45
0.38
0.08
0.46
0.19
0.39
0.39
0.08
0.02
0.44
0.05
D3 Dr
0.09
0.06
0.13
0.33
0.15
0.24
0.13
0.01
0.08
0.36
0.18
0.43
0.17
0.28
0.14
0.07
0.08
0.25
0.39
0.09
0.59
DR Dr
0.06
0.30
0.14
0.30
0.37
0.21
0.10
0.18
0.13
0.68
0.24
0.16
Table 16: The standard deviation on the PACS and OfficeHome datasets under DG setting.
PACS
OfficeHome
Method
A
C
P
Ar
CI
Pr
Rw
MIRO [6]
1.22
1.66
0.21
1.18
0.39
0.49
0.43
CDGA [22]
1.50
1.60
0.70
0.90
1.20
0.30
0.40
0.20
ERM [58]
0.46
1.16
0.06
3.43
0.72
0.63
0.34
0.49
ERM+-DomaiDDiff [36]
1.60
0.00
0.00
0.90
0.40
0.60
0.60
0.90
ERM+Terra
0.75
1.57
0.37
3.47
0.15
0.74
0.15
0.14
SAGM [65]
0.86
1.48
0.74
2.49
0.06
SAGM+Terra
0.12
0.61
0.30
1.86
0.67
0.63
0.58
0.36
SWAD [5]
0.08
0.73
0.04
0.38
0.17
0.17
0.10
0.65
SWAD+TerTa
0.10
0.03
0.28
0.83
0.28
0.21
0.45
0.37
Objective: MoLE focuses on combining multiple pre-trained LoRAs achieve multi-concept customization, whereas Terra aims to learn single adapter structure that can capture multiple domains and construct domain flow for generation.
Training: MoLE only optimizes the gating function to preserve the characteristics trained LoRAs on different domains, whereas Terra participates in the diffusion fine-tuning stage and aims to learn domain -eneral knowledge and domain-specific knowledge, allowing for control over different domains through time variable.
25

---

Expressiveness: MoLE uses separate gating function for each LoRA layer, which requires entropy- based balancing resolve conflicts when combining multiple LoRAs. In contrast, Terra achieves domain adaptation through a single time variable I, making it more stable. For two-domain interpola- tion, Terra and MoL. have similar expressiveness. Considering two domains with time variables 4 and (2, we have
(ad1 + (1 - a))2) = BK(ad +
(at, + a))) A + BA
(31)
(L)(()) + -1- (1-)))W((())
This is equivalent the linear arithmetic composition in MoLE.
Finally, the relation between MoLE and Terra similar to that between Gaussian Mixture Model (GMM) and Gaussian Process (GP). GMM composes complex distribution by multiple Gaussian distributions, and GP is distribution over functions within a continuous domain (such as time). Analogously, MoLE excels composition capabilities, while Terra excels at constructing manifold.
F Broader Impact and Ethics Statements
The ability to generate realistic images can be misused to create deepfakes or other deceptive content, potentially leading to misinformation and privacy violations. While our work has the potential to advance the field of PEFT and generation-bait associated risks, particularly in terms of ethical considerations.
cross-domain learning, it is crucial to address the
G Limitation and Failure Cases
Despite showing promising results in data-augmeentttion-bee based UDA and DG, Terra has some limitations. Generating images via Terra for data augmentation requires additional storage space. For UDA tasks, generate target domain samples and transform source domain samples into the target domain, without utilizing Terra's ability to generate intermediate domains. Note that the intermediate domain be leveraged by using methods in gradual domain adaptation [31], but have not explored this due to different settings. We leave it for future studies. Additionally, while we have adapted downstream domains through fine-tuning. of the foundation model to some extent.
model may still be influenced by the prior
We acknowledge that small number of generated images may exhibit poor quality due to the conflict between SD prior knowledge and the knowledge required for downstream tasks. We showcase some failure cases in Fig. 1. However, the number of those poor-quality images is small, and it does not affect the overall performance of the model.
Computer
Webcam
Chair
Table
Spoon
Bottle
Drill
Telephone
krelhasthoonel
Class Confusion
Incomplete Content
Noisy Backgrounds
Strong Prompt Influences
Figure 11: Ilsttrtaionn failure cases generated samples.
26

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: See Abstract and last two paragraphs in Introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: See Appendix G.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
27

---

Justification:
See Section 3.2 and Appendix A.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: See supplemental material for the code and Appendix for the experimental details.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
28

---

Answer: [Yes]
Justification:
The code and data can be found in the supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: /Codeeuomiisseeeeeeee
/nips. wwtpccccpppcccaecccccccccccccclt /tneeededdeeeeeeeeuiiiiii ssisssolicyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
The implementation details can be found in Appendix B.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The standard deviation of three random experiments with different seeds shown in Table 8, and 13-16.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
29

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Appendix B.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the csQuidelllllleeccc NeurIPS Code of Ethics https: //te://neessseeeeeeeeee http//uurrs.ccppplliiieeeeeeeeee wwcpppppcc/caaececccceeceeeceeet ie ines?
Answer: [Yes]
Justification:
We have read and complied with the Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
See Appendix F.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
30

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
There are no such risks in this paper.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: See the cited reference and supplemental material.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
31

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [NA]
Justification: This paper dos not release assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Not human subjects research.
Guidelines:
The answer NA means that the paper does involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Not human subjects research.
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
32


Reviews:
[{"review_id": "a1qwd3Hf1m", "reviewer": "Reviewer_Y8y3", "summary": "This paper proposes Terra, a time-varying low-rank adapter based on the Low-Rank Adapter (LoRA) method, designed to enable continuous domain shifts from one domain to another. The core idea is to incorporate a time-dependent function between the LoRA low-rank matrices, using time t to control the interpolation of generated samples between the source and target domains. Qualitative results demonstrate that Terra facilitates continuous changes when transferring images across domains. Quantitative experiments indicate that Terra can serve as a foundation for generalization-based unsupervised domain adaptation and domain generalization tasks, thereby improving performance.", "questions": "Please refer to the weaknesses.", "limitations": "Limitations are discussed in the paper. There are no potential negative societal impact.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The proposed method is well-motivated, straightforward and sounded. \n- The authors conduct systematically experiments to verify the usage of Terra in multiple domains (by combining with most of the off-the-shelf methods).\n- The code is provided.", "weaknesses": "- The presentation can be improved. I suggest elaborate and provide more details on the section 3.3 and to move Fig. 7 in the appendix to the main paper, since this part should be the most crucial part as how it constructs the evolving visual domains and serve as the basis to the following applications. Besides, the implementation details of morphing between style and subject should also be explained here.\n\n- The experimental comparisons should be improved. For Sec 4.2, instead of only simply stating that \"following the setting employed in DiffMorpher\", the authors should also provide qualitative comparisons. Terra also does not show performance improvement against DiffMorpher or LoRA Interp. in terms of PPL. As a result, for Sec 4.3 and 4.4, it is expected that off-the-shelf UDA/DG + Terra can improve against those without Terra, but the comparisons should include off-the-shelf UDA/DG + other similar morphing works (e.g. DiffMorpher) for a fair comparison with prior arts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "#### [W.1] The presentation can be improved. I suggest elaborate and provide more details on sec. 3.3 and to move Fig. 7 in appendix to the main paper. The implementation details of morphing between style and subject should be explained here.\n\n>We will follow your suggestion by (a) relocating Fig. 7 to Sec. 3.3 and (b) enhancing the organization of Sec. 3.3 to better illustrate how to construct evolving visual domains.\n> - **Stage 1: Fine-tune the parameters of Terra** (i.e., $\\Delta_\\theta=W_{up}\\cup W_{mid}\\cup W_{down}$) using the loss function defined in Eq. (5), where the first part with $t=0$ uses source samples $\\mathcal{D}_S$ and the second part with $t=1$ uses target samples $\\mathcal{D}_T$.\n> - **Stage 2: Generate an intermediate domain** by (a) uniformly sampling $t$ from [0,1] and (b) inputting the text prompt and a random noise into the fine-tuned diffusion model corresponding to domain $t$ (i.e., $\\theta_0 + \\Delta W(t)$ where $\\theta_0$ is the pre-trained diffusion model) for the backward process.\n>\n>Regarding the implementation details of generative interpolation tasks, including those involving morphing between image pairs, styles and objects: \n>  - We will follow the reviewer's suggestion to **introduce a new section** on Generative Interpolation via Terra between Secs. 3.3 and 3.4. This section will list **generative interpolation as one of the three concurrent applications** of the proposed domain flow generation framework of Terra. It will present the details originally provided in Line 241-256 and 604-614.\n>  - For convenience, we also provide the details here. \n>    - Morphing in image pairs: In Stage 1, we instantiate the loss function in Eq. (5) by setting (a) $\\mathcal{D}_S$ to include **one image of the pair**, (b) $\\mathcal{D}_T$ to include **the other image**, and (c\\) the text prompt to describe the images. In Stage 2, we generate intermediate images by uniformly transiting $t$ from 0 to 1 using the same text prompt as in Stage 1; each value of $t$ results in an interpolated image.\n>    - Morphing in styles/objects: This differs from morphing in image pairs only in $\\mathcal{D}_S$ and $\\mathcal{D}_T$ used in Stage 1, which are **a group of images in one style/object and a group of images in another style/object**, respectively.\n\n#### [W.2a] The authors should provide qualitative comparisons.\n\n>We provide qualitative comparisons of our method with all other baselines in Fig. r1 of the Rebuttal-PDF, which shows that Terra could generate intermediate images that are visually smooth and natural.\n\n#### [W.2b] Terra does not show performance improvement over DiffMorpher or LoRA Interp. in PPL. Comparisons should include off-the-shelf UDA/DG+other similar morphing works for a fair comparison.\n>We'd like to humbly clarify the potential misunderstandings.\n>\n>First, Terra is a **general framework that constructs a continuous parameter manifold** and thus generates a domain flow. To illustrate, in this paper, we show the three representative applications of Terra, including image morphing, UDA, and DG.\n>\n>Second, image morphing, UDA, and DG are three parallel applications of Terra, so that **the roles of Terra in them differ**.\n> - In image morphing, similar to DiffMorpher or LoRA Interpolation, Terra generates a continuously interpolative sequence of images.\n> - In UDA, as shown in Fig. 2(a), Terra bridges the gap between the source and target domains by generating more target-like samples from source samples and random noise. Thus,\n>   - in UDA, image morphing, whose output is an interpolative sequence of images, cannot be directly applied into off-the-shelf UDA methods.\n>   - for a fair comparison with prior arts in Terra's effectiveness in **generating more target-like samples**, we add a comparison against **off-the-shelf UDA + direct generation of target style samples** using prompts on Office-Home:\n>     - **SDXL (target)**: We use the prompt `A [CLASS] in the style of [TARGET]` to generate samples, where [CLASS] and [TARGET] denote the placeholders for the label name and target domain name, respectively. \n>     - **SDXL (target styles)**: We prompt GPT-4 with `Generate 50 prompts describing [TARGET] style for image generation` to obtain more detailed style descriptions for synthesis. \n>\n>     The results in Tab. r3 of the Rebuttal-PDF verify Terra's effectiveness in generating more target-like samples.\n>  - In DG, as shown in Fig. 2(b), Terra expands the source domains by (a) learning a $t$ predictor that maps each source domain to [-1,1] and (b) randomly sampling values of $t$ to generate more domains. \n>    - Image morphing can be adapted to off-the-shelf DG methods by including interpolative sequences of all pairs of images from different source domains, despite being computationally expensive.\n>    - For a fair comparison in Terra's effectiveness in **expanding source domains that generalize better**, we include the comparison against **off-the-shelf DG + morphing works** on Office-Home. That is, we train a LoRA for each domain and adopt **LoRA Interp./DiffMorpher** to interpolate. The results in Tab. r2 of the Rebuttal-PDF verify Terra's effectiveness, since Terra interpolates between domains instead of images and thus better models distributions in two domains.\n> \n>Third, even on the image morphing, Terra outperforms the DiffMorpher, which specifically designed for morphing by customized techniques such as attention interpolation, adaptive normalization, and a new sampling schedule.\n> - Terra supports morphing of styles and objects, which DiffMorpher seems incapable of.\n> - Terra enjoys better rationality and fidelity (measured by FID) than DiffMorpher.\n> - Terra is competitive in smoothness and consistency (**measured by PPL**) with DiffMorpher, despite being simple and general without any specific design. Moreover, **equipped with those customized techniques used in DiffMorpher, Terra is even better than DiffMorhper** (see Tab. r1 of the Rebuttal-PDF).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1hAe4u1wKZ", "reviewer": "Reviewer_76UZ", "summary": "This article introduces Terra, a simple time-varying low-rank adapter based on LoRA for domain flow generation. Terra efficiently bridges the source and target domains using a parameter-efficient method. By generating data with smaller domain shifts, Terra effectively improves performance in incorporation, UDA, and DG tasks.", "questions": "This article uses Terra-finetuned SD XL to generate more training data for UDA and DG, which is quite similar to other works that enhance classifier performance through data generation (e.g., [1,2]). Both approaches leverage the pre-trained SD's prior knowledge to generate images as a form of data augmentation. Hence, I recommend adding a comparison with these methods to demonstrate Terra's design advantages, given that both utilize the SD prior. If direct integration into the current evaluation framework is not feasible, a simple approach could be to generate corresponding data augmentations by changing prompts, showcasing the improvements brought by the SD prior itself. Alternatively, I encourage the authors to illustrate the respective contributions of Terra and the SD prior to the performance boost through other reasonable means.\n\nAdditionally, I noticed a minor issue in Table 1: DGP is based on a GAN model, while Terra uses SD XL. Therefore, a direct numerical comparison is unfair. I suggest the authors include the base model for clarity.\n\nReferences:\n\n[1] Synthetic data from diffusion models improves imagenet classification.\n\n[2] Active Generation for Image Classification", "limitations": "The authors didn't discuss the limitations in the main paper.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The writing is clear and easy to follow.\n- The method provides an intuitive and effective approach to enhancing LoRA for domain flow generation, with a theoretical analysis of its expressive power.\n- Terra shows promising results in interpolation tasks.\n- The idea of generating data with smaller domain shifts to the training set is innovative and enhances model performance in UDA and DG.", "weaknesses": "- The method essentially adopts the LoRA approach and constructs a low-rank parameter manifold through F(W,t)=tW+I. This can be seen as an interpolation version of LoRA with fewer parameters, which might limit the novelty of the model.\n- There is a lack of more direct comparative experiments, as mentioned in the Questions section.\n- As a method that fine-tunes SD, the paper would benefit from directly evaluating the quality of generated images using different fine-tuning methods (such as those mentioned in lines 58-61). \n- Lack of discussion of limitations and failure cases.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "#### [W.1] The method essentially adopts the LoRA approach and constructs a low-rank parameter manifold through F(W,t)=tW+I. This can be seen as an interpolation version of LoRA with fewer parameters.\n\n> Terra constructs a \"LoRA flow\" in the parameter space, and is NOT an interpolation version of LoRA with fewer parameters. Here are three key differences:\n> \n> - **Formulation**: $tW+I$ is just one instance of Terra. Other possible forms are listed in Table 6 of the Appendix, i.e., $\\exp(t{W})$ and $\\cos(t{W})$.\n> \n> - **Training**: Unlike LoRA interpolation, which needs training separate LoRAs for different domains, our method only needs to train one adapter for multiple domains. In our training, the middle time-varying matrix $\\mathcal{K}(t)$ is domain-specific, while the matrices $W_{up}$ and $W_{down}$ are shared across domains, enabling domain-general knowledge learning.\n> \n> - **Application**: The interpolation between two domain is just one application of our Terra. More importantly, based on Terra, we can design effective frameworks for UDA and DG:\n>   - For UDA, our framework can learn the domain-general subjects and domain-specific styles due to the Terra structure, enabling the generation of target-like samples from both source samples and random noise, thereby reducing the domain gap.\n>   - For DG, Terra constructs a domain manifold in the parameter space, facilitating random interpolation to generate diverse samples, which enhances the model's generalization ability.\n\n#### [W.2] There is a lack of more direct comparative experiments, as mentioned in the Questions.\n\n> Please refer to our following response to [Q.1a].\n\n\n#### [W.3] As a method that fine-tunes SD, the paper would benefit from directly evaluating the quality of generated images using different fine-tuning methods (such as those mentioned in lines 58-61).\n>- The methods mentioned involve customized image generation tasks, such as image editing and multi-concept generation, focusing primarily on single-domain image generation. In contrast, our approach enhances the LoRA structure to create a continuous parameter manifold, allowing for **image generation across a domain flow**.\n>\n>- A related work is Diffmorpher, which trains two LoRAs on image pairs and introduces techniques for **generating a continuously interpolative sequence of images**, referred to as image morphing. We compare Diffmorpher with our method in the manuscript.\n> \n>- To demonstrate that the paper's potential, we conduct an additional experiment by replacing the two LoRAs with our proposed Terra. The combined method \"Terra + Diffmorpher\" yields improved FID and PPL scores. The qualitative and quantitative results are presented in Fig. r1 and Tab. r1 of the Rebuttal-PDF.\n\n#### [W.4] Lack of discussion of limitations and failure cases.\n> - **Limitations**: We have thoroughly discussed the limitations in Appendix D. Additionally, while we have adapted to downstream domains through fine-tuning, our model may still be influenced by the prior of the foundation model to some extent.\n>\n> - **Failure cases**: We acknowledge that a small number of generated images may exhibit poor quality due to the conflict between SD prior knowledge and the knowledge required for downstream tasks. We showcase some failure cases in Fig. r3 of the Rebuttal-PDF. However, the number of those poor-quality images is small, and it does not affect the overall performance of the model.\n\n#### [Q.1a] I recommend adding a comparison with data augmentation with SD's prior knowledge to demonstrate Terra's design advantages. Alternatively, I encourage the authors to illustrate the respective contributions of Terra and the SD prior to the performance boost through other reasonable means.\n> We appreciate the reviewer's suggestion to explore the prior of foundation models. To address this concern, we design several methods to synthesize data based on the SDXL model and evaluate their effectiveness on UDA tasks:\n> - (1) **SDXL (random)**: We use the prompt `A [CLASS]` to generate samples for each class, where [CLASS] denotes the placeholder for the label.\n> - (2) **SDXL (styles)**: We first use the prompt `Generate 50 prompts describing diverse styles for image generation` to ask GPT-4, and then use the prompt `A [CLASS], an everyday object in office and home, in the style of [STYLE]` to generate samples, where [STYLE] denotes the placeholder for style prompts generated by GPT-4 (e.g. \"Classic\", \"Modern\").\n> - (3) **SDXL (target)**: Based on (2), we use the name of target domain (e.g. \"Clipart\") to replace the [STYLE] as the new placeholder for exploring the SD prior on the target domain.\n> - (4) **SDXL (target styles)**: We use the prompt `Generate 50 prompts describing [TARGET] style for image generation` to ask GPT-4 and obtain more detailed style prompts for synthesis.\n> - (5) **SDXL (selected)**: Inspired by [2], we use a confidence-based activate learning method to filter out poor-qulity and misclassified samples generated in (4) and select valid samples. \n> \n> The comparison results on Office-Home for UDA are shown in Tab. r3 of the Rebuttal-PDF. Terra outperforms the comparison methods, indicating that despite the boost in accuracy from target style design and active learning, the prior knowledge is insufficient to align with the downstream tasks. This issue can be furthur mitigated through finetuning with Terra, which demonstrates Terra's design advantages.\n\n#### [Q.1b]: I noticed a minor issue in Table 1: DGP is based on a GAN model, while Terra uses SD XL. Therefore, a direct numerical comparison is unfair. I suggest the authors include the base model for clarity.\n> Thank you for pointing out this issue. Here, we compare with GAN-based model DGP and the stable diffusion-based methods DDIM, LoRA Interpolation, and DiffMorpher. In our revision, we will include the base models for clarity. You can find the updated Tab. r1 and Fig. r1 of the Rebuttal-PDF.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KJFTIcjReQ", "reviewer": "Reviewer_LKTK", "summary": "The paper presents a variant of LoRA, with additional time variable conditioned low-rank square matrix, for fine-tuning a diffusion model for unsupervised domain adaptation and domain generalization. Besides, the paper also study how to better apply the proposed Terra on UDA and DG tasks. Compared to prior arts, the proposed method achieve better performance.", "questions": "Generally I feel the paper propose a simple yet interesting method for the domain generalization problem. Please see weakness for my questions.", "limitations": "No other limitation as far as I can see.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper presents an interesting idea on introducing a \"time\" condition on the LoRA matrix, which paves a path from source domain towards target domain.\n\n- The paper also presents in depth analysis, providing interesting insight\n\n\n- The paper is generally well-written and technically sound", "weaknesses": "- I feel the term \"time\" and \"t\" contradicts to the widely used time step t in diffusion model. Although I do see the authors used another symbol for diffusion models' timestep, it would be much better to use another term for the proposed \"time\" term to avoid confusion.\n\n-  I appreciate the provided possible forms of Terra in Tab 6. Yet, I am wondering if the authors can provide more insights/principles in terms of how to choose the function form. Besides, there seems missing an ablation on the different functions for Terra, to justify why \"Linear\" is used for generative interpolation/UDA and \"cosine-sine\" is used for DG?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "#### [W.1] I feel the term \"time\" and \"t\" contradicts to the widely used time step t in diffusion model. Although I do see the authors used another symbol for diffusion models' timestep, it would be much better to use another term for the proposed \"time\" term to avoid confusion.\n> We appreciate the reviewer's concern. As you mentioned, we have used the symbol $\\tau$ to differentiate our notation from the timestep in diffusion models. We intend to retain the term \"time\" because Terra draws inspiration from multiple research fields, including **Fluid Dynamics** and **Control Theory**, where \"time\" and \"t\" are also commonly used notations.\n>\n> - In the context of fluid dynamics, for each time variable $t$, the matrix updates of Terra is $\\Delta W(t)$, enabling a time-dependent velocity field $\\frac{d}{d t} \\Delta W(t)$. Terra constructs a \"LoRA flow\" in the parameter space based on the Lagrangian and Eulerian descriptions [r1]. Therefore, after the cross-domain diffusion fine-tuning, Terra demonstrates the ability to generate domain flow.\n>\n> - From a control theory perspective, Terra can be viewed as solutions to time-varying systems. For instance, consider a linear system $\\dot x(t) = A x(t)$, where the \"state vector\" $x(t)$ can be seen as the continuous image feature across different domains, and a closed-form solution is $x(t) = e^{A(t)} x_0$ [r2]. Here is the \"exponential\" form Terra with the time-varying matrix function.\n>\n> To clarify, we will provide a more detailed explanation of $\\tau$ used in the diffusion model. Specifically, \"since $t$ in this paper refers to the time variable in Terra, we use $\\tau$ here to represent the timestep $t$ in the diffusion model\".\n> \n> [r1] Villani C. Optimal transport: old and new. Berlin: springer, 2009, Pages 26.\n>\n> [r2] Williams R L, Lawrence D A. Linear state-space control systems. John Wiley & Sons, 2007, Pages 52-55.\n\n \n#### [W.2a] I appreciate the provided possible forms of Terra in Tab 6. Yet, I am wondering if the authors can provide more insights/principles in terms of how to choose the function form. \n> We appreciate the reviewer's interest in the possible forms of Terra presented in Table 6. To provide more insights, we elaborate on the guiding principles behind the choice of the three forms: \n> \n> - **Linear**: The $tW+I$ is the simplest form, related to a straight and steady flow, which is sufficient for two domains according to Theorem 1 and 2. Its constant velocity of weight changes ensures smooth morphing and is suitable for simple interpolating between two domains under the UDA setting. \n> \n> - **Cosine-Sine**: This form is adopted because of the bounded range and non-linearity of trigonometric functions, preventing image collapse during generation and enabling a complex parameter manifold to capture relationships between multiple domains. We recommend using this form in complex scenarios, such as interpolating multiple domains in DG.\n> \n> - **Exponential**: $e^{tW} = I + \\sum_{k=1}^{\\infty} \\frac{t^k}{k!} W^k$, implemented using `torch.matrix_exp`, also defines a smooth curve in a high-dimensional manifold. This form is more expressive and suitable for handling multiple domains, as it enables feature transformations as mentioned in our response to [W.1]. Notably, it related to three types of transformations: scalings, rotations, and shears [r3].\n> \n> [r3] Ronald N. Goldman, VII.3 - Transformations As Exponentials, Graphics Gems II, 1991, Pages 332-337.\n\n#### [W.2b] Besides, there seems missing an ablation on the different functions for Terra, to justify why \"Linear\" is used for generative interpolation/UDA and \"cosine-sine\" is used for DG?\n> We appreciate the reviewer's comment and would like to provide clarification on this point. \n> - For tasks involving only two domains, such as generative interpolation and UDA, the expressive abilities of the \"Linear\" and \"cosine-sine\" functions are equivalent. Specifically, when $t=0$, both terms of the middle matrix reduce to the identity matrix $I$, and when $t=1$, the learnable matrices are constrained equally.\n> - However, for tasks involving more domains, such as DG, we have conducted an empirical ablation study to investigate the effectiveness of the \"cosine-sine\" form compared to the \"linear\" form. The results, presented in Table 7 of the manuscript and reproduced below, demonstrate the superiority of the \"cosine-sine\" form on the PACS dataset in DG scenarios.\n> \\begin{array}{lcccccl}\n\\hline\n \\textbf{Method} & \\text{A} &   \\text{C} & \\text{P} & \\text{S} &   \\text{Average} \\newline\n\\hline\n\\text{Linear} & 87.47 & 80.17 & 97.85 & 77.16 & 85.66   \\newline\n\\text{Cosine-sine (dim=1)}  & 88.29 & \\textbf{82.36} & 97.53 & 73.31 & 85.37   \\newline\n\\text{Cosine-sine (dim=2)}  & \\textbf{89.51} & 79.66 & \\textbf{98.20} & \\textbf{78.64} & \\textbf{86.50}   \\newline\n\\hline\n\\end{array}", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Oh6JkwHK45", "reviewer": "Reviewer_STii", "summary": "This paper introduces a low-rank adapter, Terra, for effective cross-domain modelling through the construction of a continuous parameter manifold. This approach facilitates knowledge sharing across different domains by training only a single low-rank adaptor. The expressiveness of the model was analyzed theoretically and bounds for approximation error were given. Extensive experiments on various UDA and UG benchmarks demonstrate the effectiveness of the proposed method. Ablation studies verify the robustness of the proposed framework to variations of key components.", "questions": "In addition to the above weaknesses, I have the following questions regarding the theoretical part:\n\n**(Q1) Scalability to multiple domains:** To my understanding, Terra can express knowledge of multiple domains (e.g., in the DG setting). However, the paper only demonstrates its equivalence with two LoRAs in Theorem 1. I am curious about the scalability of Terra when applied to more domains or tasks (since this greatly enhances the potential of Terra under real-world scenarios (i.e., multiple environments)).\n**(Q2) Comparison with other LoRA variants**: Could the authors provide an analysis of how Terra differs from other LoRA variants, such as MoLE [3], particularly on the expressiveness of these models?\n\n[3] Mixture of LoRA Experts, ICLR 2024.", "limitations": "The authors discuss the limitations adequately in the Appendix.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "**(S1)** The proposed Terra is a simple yet effective PEFT method for diffusion models fine-tuning, which successfully generates various images in a customized domain flow.\n\n**(S2)** This paper is well-written and easy to follow. The proposed framework facilitate effective and flexible knowledge sharing across different domains while maintaining parameter efficiency.\n\n**(S3)** Terra involves constructing a continuous parameter manifold using a time variable, with its expressive power theoretically analyzed and smooth interpolation empirically verified. \n\n**(S4)** Terra can serve as a plugin for existing UDA and DG methods to help alleviate domain shifts, achieving state-of-the-art performance on various benchmarks.\n\n**(S5)** Codes are available for reproducibility. (Thanks for providing the codes, resolved some key questions on the implementations of the method)", "weaknesses": "**(W1) Qualitative Evaluation**: Although Figure 3 is interesting as it demonstrate how Terra can handle morphing under different scenarios, it is suggested that the authors perform more qualitative samples to evaluate the interpretability of Terra more comprehensively.\n\n**(W2) Comparative analysis**: Although the paper includes extensive comparisons of Terra with two baseline methods in the UDA experiments, it would be beneficial to extend comparisons with recent CNN-based and ViT-based UDA methods, such as CoVi [1] and PMTrans [2].\n\n**(W3) Clarity of Technical Details**: The rationale behind randomly sampling the value of \"t\" for the DG experiments to generate diverse domains is unclear. While Figure 6 shows that the learned time variables cluster within the existing source domains, the distribution of the target domain is not adequately explained.\n\n[1] Contrastive vicinal space for unsupervised domain adaptation, ECCV 2022.\n\n[2] Patch-mix transformer for unsupervised domain adaptation: A game perspective, CVPR 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "#### [W.1] Qualitative Evaluation\n > We appreciate the reviewer's suggestion. we have included additional qualitative samples in Fig. r2 of the Rebuttal-PDF. These samples further demonstrate Terra's ability to handle morphing under various scenarios.\n\n#### [W.2] Comparative Analysis\n> We thank the reviewer for suggesting additional comparisons. In response, we have conducted experiments with CoVi and PMTrans, and the results are presented in Tab. r4 of the Rebuttal-PDF. Notably, Terra consistently improves performance in all tasks with those UDA methods, further verifying the effectiveness of our method. \n\n#### [W.3] Clarity of Technical Details\n\n> Thank you for your comment. In the DG experiments, we first use contrastive learning loss to train a network $g(\\cdot)$ to predict sample-level $t$ for Terra. This approach allows us to better capture inter-domain and intra-domain styles differences. After training, randomly sampling the value of $t$ can generate more diverse samples between domains. Specifically, we use two-dimensional $t$, with each dimension sampled from -2 to 2 at intervals of 0.1 to generate diverse samples. We will include these details in the revision. \n> Moreover, we present the distribution of the **learned time variable** of the target domain in Fig. r4 of the Rebuttal-PDF. As illustrated, the random sampling of $t$ effectively covers the target domain, offering a clearer understanding of the rationale behind our approach that the generated samples may bring useful information for the target domain under the DG setting.\n\n#### [Q.1] Scalability to Multiple Domains\n\n> We appreciate the reviewer's interest in the scalability of Terra. Our theoretical analysis indeed extends to multiple domains or tasks.\n> \n> **Theorem 1 can be generalized to multiple matrices using higher-order generalized singular value decomposition (HO GSVD)** [r1]. Specifically, given K low-rank adapters $\\Delta W_k \\in \\mathbb{R}^{m \\times n}$ for K domains/tasks,  we can concatenate the matrices horizontally and vertically to form $H$ and $V$, and let $k = \\max\\\\{rank(H), rank(V)\\\\}$. Then, similar to Eqs. (14) and (15), based on HO GSVD, each matrix can be exactly factored as $\\Delta W_i = Y K_i X$, where $Y \\in \\mathbb{R}^{m \\times k}$, $K_i \\in \\mathbb{R}^{k \\times k}$, and $X \\in \\mathbb{R}^{k \\times n}$. We can then construct $\\mathcal{K}(t)$ to satisfy $\\mathcal{K}(t_i) = K_i$ for fixed $t_i$. In the case of multiple domains or tasks, we propose using a vector $\\symbfit{t}_i$ with a small increase in parameters. The matrix $K_i$ can be constructed using interpolation methods, such as polynomial interpolation or spline interpolation, or non-linear time-varying matrices as those in Table 6. \n> \n> In summary, this approach allows Terra to use one adapter structure to represent multiple LoRAs with fewer parameters. When the domains share knowledge—implying $k$ is small—the required parameters are further reduced. Additionally, for unknown tasks, we can determine the matrices $\\mathcal{K}(t_i)$ using least squares or other optimization algorithms [r2], enabling a meta-learning approach. \n> \n>\n>[r1] Ponnapalli SP, Saunders MA, Van Loan CF, Alter O. A Higher-Order Generalized Singular Value Decomposition for Comparison of Global mRNA Expression from Multiple Organisms. PLOS ONE, 2011. \n>\n>[r2] Friedland S, Torokhti A. Generalized rank-constrained matrix approximations. SIAM Journal on Matrix Analysis and Applications, 2007.\n\n#### [Q.2] Comparison with Other LoRA Variants\n> For cross-domain learning based on MoLE, it can be seen as respectively training LoRAs on different domains first and then training a gating function to combine the trained LoRAs. While both MoLE and Terra are designed for customization of diffusion models, they differ in several key aspects:\n> - **Objective**: MoLE focuses on combining multiple pre-trained LoRAs to achieve multi-concept customization, whereas Terra aims to learn a single adapter structure that can capture multiple domains and construct a domain flow for generation.\n> - **Training**: MoLE only optimizes the gating function to preserve the characteristics of trained LoRAs on different domains, whereas **Terra participates in the diffusion fine-tuning stage** and aims to learn domain-general knowledge and domain-specific knowledge, allowing for control over different domains through a time variable.\n> - **Expressiveness**: MoLE uses a separate gating function for each LoRA layer, which requires entropy-based balancing to resolve conflicts when combining multiple LoRAs. In contrast, Terra achieves domain adaptation through a single time variable $t$, making it more stable. For two-domain interpolation, Terra and MoLE have similar expressiveness. Considering two domains with time variables $t_1$ and $t_2$, we have\n> $$\n> \\begin{equation}\n> \\Delta W(\\alpha t_1 + (1-\\alpha)t_2) =  B\\mathcal{K}(\\alpha t_1 + (1-\\alpha)t_2)A =(\\alpha t_1 + (1-\\alpha)t_2)BWA+ BA = \\alpha \\Delta W(t_1) + (1-\\alpha) \\Delta W(t_2).\n> \\end{equation}\n> $$\n> This is equivalent to the linear arithmetic composition in MoLE. As shown in the response to [Q.1], this conclusion can be extended to interpolation with three or more LoRAs.\n> \n> Finally, the relation between MoLE and Terra is similar to that between **Gaussian Mixture Model (GMM)** and **Gaussian Process (GP)**. GMM composes a complex distribution by multiple Gaussian distributions, and GP is a distribution over functions within a continuous domain (such as time). Analogously, MoLE excels at the composition capabilities, while Terra excels at constructing a manifold.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers and ACs,

We sincerely thank all the reviewers and ACs for your diligent efforts and high-quality reviews. If you have any additional question or require further clarification, please feel free to let us know. Your insights are highly valued.

We are delighted to note that reviewers find that:

- our method is innovative (`Reviewer 76UZ`), well-motivated, straightforward, and sounded (`Reviewers Y8y3 and LKTK`), with clear and easy-to-follow writing (`Reviewers 76UZ, LKTK, and STii`).
- our paper is providing an intuitive and effective approach to construct a continuous parameter manifold for domain flow generation (`Reviewers 76UZ, LKTK, and STii`) and synthesize data to help alleviate domain shifts (`Reviewers 76UZ, LKTK, and STii`).
- our method includes a theoretical analysis of its expressive power (`Reviewers 76UZ, LKTK, and STii`), achieving promising results on various benchmarks (`Reviewers Y8y3, 76UZ, and STii`), and is supported by reproducible results with provided code (`Reviewers Y8y3 and STii`).

In responce to your valuable suggestions, we have conducted additional experiments and included the new results in the supplementary Rebuttal-PDF for your convenience:

- **Figure r1 & Table r1**: We have added the **qualitative** and **quantitative** comparison results of image morphing (suggested by `Reviewer y8Y3`) and introduced a combined method "Terra + Diffmorpher" (suggested by `Reviewers y8Y3 and 76UZ`).
- **Figure r2**: Supplementary samples for qualitative evaluation of the image morphing tasks (suggested by `Reviewer STii`). 
- **Figure r3**: Some failure cases in generated samples (suggested by `Reviewer 76UZ`). 
- **Figure r4**: We present the learned time variables to provide rationale behind our generation approach (suggested by `Reviewer STii`).
- **Table r2**: A comparison with **morphing works** (LoRA Interp. and DiffMorpher) that expand source domains under the DG setting (suggested by `Reviewer y8Y3`).
- **Table r3**: A comparison with **target-like samples augmentation methods** using the SDXL prior under the UDA setting (suggested by `Reviewers 76UZ and y8Y3`). 
- **Table r4**: A **comparative analysis** with two state-of-the-art baseline methods (suggested by `Reviewer STii`). 

Finally, due to character limits, we have condensed some reviews from `Reviewers y8Y3, 76UZ, and STii` in our responses.  

Best regards,

The Authors

Meta-review: This paper has received consistent feedback from four reviewers, all of whom are in favor of acceptance. It introduces Terra, a time-varying low-rank adapter based on the Low-Rank Adapter  method, which enhances LoRA for domain flow generation. The approach is simple yet effective, and the related experiments are comprehensive. Therefore,  the AC  has decided to accept this paper.
Decision: Accept (poster)