D UAL D IFFUSION I MPLICIT B RIDGES FOR I MAGE - TO -I MAGE T RANSLATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Common image-to-image translation methods rely on joint training over data from both source and target domains.
The training process requires concurrent access to both datasets, which hinders data separation and privacy protection; and existing models cannot be easily adapted to translation of new domain pairs.
We present
Dual Diffusion Implicit Bridges (DDIBs), an image translation method based on diffusion models, that circumvents training on domain pairs.
Image translation with DDIBs relies on two diffusion models trained independently on each domain, and is a two-step process: DDIBs ﬁrst obtain latent encodings for source images with the source diffusion model, and then decode such encodings using the target model to construct target images.
Both steps are deﬁned via ordinary differential equations (ODEs), thus the process is cycle consistent only up to discretization errors of the ODE solvers.
Theoretically, we interpret DDIBs as concatenation of source to latent, and latent to target Schr¨odinger Bridges, a form of entropy-regularized optimal transport, to explain the efﬁcacy of the method.
Experimentally, we apply DDIBs on synthetic and high-resolution image datasets, to demonstrate their utility in a wide variety of translation tasks and their inherent optimal transport properties.
I NTRODUCTION
Transferring images from one domain to another while preserving the content representation is an important problem in computer vision, with wide applications that span style transfer (Xu et al., 2021) and semantic segmentation (Li et al., 2020).
In tasks such as style transfer, it is usually difﬁcult to obtain paired images of realistic scenes and their artistic renditions. Consequently, unpaired translation methods are particularly relevant, since only the datasets, and not the one-to-one correspondence between image translation pairs, are required.
Common methods on unpaired translation are based on generative adversarial networks (GANs, Goodfellow et al.
(2014); Zhu et al.
or normalizing ﬂows (Grover et al., 2020).
Training such models typically involves minimizing an adversarial loss between a speciﬁc pair of source and target datasets.
While capable of producing high-quality images, these methods suffer from a severe drawback in their adaptability to alternative domains.
Concretely, a translation model on a source-target pair is trained speciﬁcally for this pair of datasets.
Given a different pair, existing models cannot be easily adapted for translation.
If we were to do pairwise translation among a set of domains, the total number of models needed is quadratic in the number of domains – an unacceptable computational cost in practice.
One alternative is to ﬁnd a shared domain that connects to each source / target domains as in
StarGANs (Choi et al., 2018).
However, the shared domain needs to be carefully chosen a priori ; if the shared domain contains less information than the target domain ( e.g. sketches v.s. photos), then it creates an unwanted information bottleneck between the source and target domains.
An additional disadvantage of existing models resides in their lack of privacy protection of the datasets: training a translation model requires access to both datasets simultaneously.
Such setting may be inconvenient or impossible, when data providers are reluctant about giving away their data; or for certain privacy-sensitive applications such as medical imaging.
For example, quotidian hospital usage may require translation of patients’ X-ray and MRI images taken from machines in other hospitals.
Most existing methods will fail in such scenarios, as joint training requires aggregating conﬁdential imaging data across hospitals, which may violate patients’ privacy.
In this paper, we seek to mitigate both problems of existing image translation methods.
We present
Dual Diffusion Implicit Bridges (DDIBs), an image-to-image translation method inspired by recent advances in diffusion models (Song et al., 2020a;b), that decouples paired training, and empowers the domain-speciﬁc diffusion models to stay applicable in other pairs wherever the domain appears again as the source or the target.
Since the training process now concentrates on one dataset at a time, DDIBs can also be applied in federated settings, and not assume access to both datasets during model training. As a result, owners of domain data can effectively preserve their data privacy.
Speciﬁcally, DDIBs are developed based on the method known as denoising diffusion implicit models (DDIMs, Song et al.
(2020a)).
DDIMs invent a particular parameterization of the diffusion process, that creates a smooth, deterministic and reversible mapping between images and their latent representations.
This mapping is captured using the solution to a so-called probability ﬂow (PF) ordinary differential equation (ODE) that forms the cornerstone of DDIBs.
Translation with DDIBs on a source-target pair requires two different PF ODEs: the source PF ODE converts input images to a latent space; while the target ODE then synthesizes images in the target domain.
Importantly, trained diffusion models are speciﬁc to the individual domains, and rely on no domain pairing information.
Effectively, DDIBs make it possible to save a trained model of a certain domain for future use, when it arises as the source or target in a new pair.
Pairwise translation with DDIBs requires only a linear number of diffusion models (which can be further reduced with conditional models (Dhariwal & Nichol, 2021)), and training does not require scanning both datasets concurrently.
Theoretically, we analyze the DDIBs translation process to highlight two important theoretical properties.
First, the probability ﬂow ODEs in DDIBs, in essence, comprise the solution of a special Schr¨odinger Bridge Problem (SBP) with linear or degenerate drift (Chen et al., 2021a), between the data and the latent distributions.
This justiﬁcation of DDIBs from an optimal transport viewpoint that alternative translation methods lack serves as a theoretical advantage of our method, as DDIBs are the most OT-effective translation procedure while alternate methods may not be.
Second, DDIBs guarantee exact cycle consistency: translating an image to and from the target space reinstates the original image, only up to discretization errors introduced in the ODE solvers.
Experimentally, we ﬁrst present synthetic experiments on two-dimensional datasets to demonstrate
DDIBs’ cycle-consistency property.
We then evaluate our method on a variety of image modalities, with qualitative and quantitative results: we validate its usage in example-guided color transfer, paired translation, and conditional ImageNet translation.
These results establish DDIBs as a scalable, theoretically rigorous addition to the family of unpaired image translation methods.
P RELIMINARIES 2. S CORE - BASED G ENERATIVE M ODELS (SGM S )
While our actual implementation utilizes DDIMs, we ﬁrst brieﬂy introduce the broader family of models known as score-based generative models.
Two representative models of this family are score matching with Langevin dynamics (SMLD) (Song & Ermon, 2019) and denoising diffusion probabilistic models (DDPMs)
(Ho et al., 2020).
Both methods are contained within the framework of Stochastic Differential Equations (SDEs) proposed in Song et al.
(2020b).
Stochastic Differential Equation (SDE)
Representation Song et al.
(2020b);
Anderson (1982) use a forward and a corresponding backward SDE to describe general diffusion and the reversed, generative processes:
[Equation]
where w is the standard Wiener process, f ( x , t ) is the vector-valued drift coefﬁcient, g ( t ) is the scalar diffusion coefﬁcient, and r x log p t ( x ) is the score function of the noise perturbed data distribution (as deﬁned by the forward SDE with initial condition p
( x ) being the data distribution).
At the endpoints t = { 0 , 1 } , the forward Eq.
(1) admits the data distribution p 0 and the easy-to-sample prior p 1 as the boundary distributions.
Within this framework, the SMLD method can be described using a VarianceExploding (VE) SDE with increasing noise scales  ( t ) :
d x = p d[  2 ( t )] / d t d w .
In comparison,
DDPMs are endowed with a Variance-Preserving (VP) SDE: d x =  [  ( t ) / 2] x d t + p  ( t ) d w with  ( t ) being another noise sequence.
Notably, the VP SDE can be reparameterized into an equivalent
VE SDE (Song et al., 2020a).
Probability Flow ODE Any diffusion process can be represented by a deterministic ODE that carries the same marginal densities as the diffusion process throughout its evolution.
This ODE is termed the probability ﬂow (PF) ODE (Song et al., 2020b).
PF ODEs enable uniquely identiﬁable encodings (Song et al., 2020b) of data, and are central to DDIBs as we solve these ODEs for forward and reverse conversion between data and their latents.
For the forward SDE introduced in Eq.
(1), the equivalent PF ODE holds the following form:
[Equation]
which follows immediately from the SDEs given the score function.
In practice, we use ✓ - parameterized score networks s t, ✓ ⇡ r x log p t ( x ) to approximate the score function.
Training such networks uses a variational lower bound, described in Ho et al.
(2020) and in Appendix B.
We may then use numerical ODE solvers to construct x at different times.
Empirically, it has been shown that SGMs have relatively low discretization errors when reconstructing x at t = 0 via ODE solvers (Song et al., 2020a).
For conciseness, we use v ✓ =
d x / d t to denote the ✓ -parameterized velocity ﬁeld (as deﬁned from Eq.
(2), where we replace r x log p t ( x ) with s t, ✓ ), and use the symbol ODESolve to denote the mapping from x ( t 0 ) to x ( t 1 ) :
[Equation]
which allows us to abstract away the exact model (be it a score-based or a diffusion model), or the integrator used.
In our experiments, we implement the ODE solver in DDIMs (Song et al., 2020a)
(Appendix B); while we acknowledge other available ODE solvers that are usable within our framework, such as the DPM-solver (Lu et al., 2022), the Exponential Integrator (Zhang & Chen, 2022), and the second-order Heun solver (Karras et al., 2022).
2. S CHR ¨ ODINGER B RIDGE P ROBLEM
Our analysis shows that DDIBs are Schr¨odinger Bridges (Chen et al., 2016; L´eonard, 2013) between distributions.
Let ⌦ = C ([0 , 1]; R n ) be the path space of R n -valued continuous functions over the time interval [0 , 1] ; and D ( p 0 , p 1 ) be the set of distributions over ⌦ , with marginals p 0 , p 1 at time t = 0 , t = 1 , respectively.
Given a prior reference measure W , the well-known Schr¨odinger Bridge Problem (SBP) seeks the most likely evolution across time t between the marginals p 0 and p 1 :
Problem 1 (Schr¨odinger Bridge Problem) .
With prescribed distributions p 0 , p 1 and a reference measure W as the prior, the SBP ﬁnds a distribution from D ( p 0 , p 1 ) that minimizes its KL-divergence to W : P SBP :=
arg min { D KL ( P k W ) | P 2 D ( p 0 , p 1 ) } .
The minimizer, P SBP , is referred to as the Schr¨odinger Bridge between p 0 and p 1 over prior W .
The SBP has connections to the Monge-Kantorovich (MK) optimal transport problem (Chen et al., 2021b).
While the basic MK problem seeks the cost-minimizing plan to transport masses between distributions, the SBP incorporates an additional entropy term (for details, see Page 61 of Peyr´e et al. Input
Relationship Between SBPs and SGMs Chen et al.
(2021a) establishes connections between
SGMs and SBPs.
In summary, SGMs are implicit optimal transport models, corresponding to SBPs with linear or degenerate drifts.
General SBPs additionally accept fully nonlinear diffusion.
To formalize this observation, the authors ﬁrst establish similar forward and backward SDEs for SBPs:
[Equation]
where  , ˆ  are the Schr¨odinger factors that satisfy density factorization: p t ( x )
=
t ( x )ˆ  t ( x ) .
The vector-valued quantities z t = g ( t ) r x log  t ( x ) , ˆ z t = g ( t ) r x log ˆ  t ( x ) fully characterize dynamics of the SBP, thus can be considered as the forward, backward “policies”, analogous to policy-based methods described in Schulman et al.
; Pereira et al.
To draw a link between SBPs and SGMs, the data log-likelihood objective for SBPs is computed and shown to be equal to that of
SGMs with special choices of z t , ˆ z t (details in Chen et al.
(2021a)).
Importantly, likelihood equality occurs with the following policies:
[Equation]
When the marginal p 1 at time t = 1 is equal to the prior distribution, it is known that such ( z t , ˆ z t ) are achieved. Since in SGMs, the end marginal p 1 is indeed the standard Gaussian prior, their log-likelihood is equivalent to that of SBPs.
This suggests that SGMs are a special case of SBPs with degenerate forward policy z t and a multiple of the score function as its backward ˆ z t .
Probability Flow ODE
In a similar vein to the SGM SDEs, a deterministic PF ODE can be derived for SBPs with identical marginal densities across t 2 [0 , 1] .
The following PF ODE characterizes the probability ﬂow of the optimal processes of the SBP deﬁned in Eq.
(4) (Chen et al., 2021a):
[Equation]
where z depends on x .
We shall show that the PF ODEs for SGMs and SBPs are equivalent.
Thus, ﬂowing through the PF ODE in DDIBs is equivalent to ﬂowing through a Schr¨odinger Bridge, with one of the marginals being Gaussian.
D UAL D IFFUSION I MPLICIT B RIDGES
DDIBs leverage the connections between SGMs and SBPs to perform image-to-image translation, with two diffusion models trained separately on the two domains.
DDIBs contain two steps, described in Alg. 1 and illustrated in Fig.
At the core of the algorithm is the ODE solver ODESolve from
Eq.
Given a source model represented as a vector ﬁeld, i.e. , v ( s ) ✓ deﬁned from Eq.
(2), DDIBs ﬁrst apply ODESolve in the source domain to obtain the encoding x ( s ) of the image at the end time t = 1 ; we refer to this as the latent code (associated with the diffusion model for the domain).
Then, the source latent code is fed as the initial condition (target latent code at t = 1 ) to ODESolve with the target model v ( t ) ✓ to obtain the target image x ( t ) .
As discussed earlier, we implement ODESolve with
DDIMs (Song et al., 2020a), which are known to have reasonably small discretization errors.
While recent developments in higher order ODE solvers (Zhang & Chen, 2022; Lu et al., 2022; Karras et al., 2022) that generalize DDIMs can also be used here, we leave this investigation in future work.
Despite the simplicity of the method, DDIBs have several advantages over prior methods, which we discuss below.
Exact Cycle Consistency A desirable property of image translation algorithms is the cycle consistency property: transforming a data point from the source domain to the target domain, and then back to source, will recover the original data point in the source domain.
The following proposition validates the cycle consistency of DDIBs.
Proposition 3.1 (DDIBs Enforce Exact Cycle Consistency) .
Given a sample from source domain x ( s ) , a source diffusion model v ( s ) ✓ , and a target model v ( t ) ✓ , deﬁne:
[Equation]
Assume zero discretization error.
Then, x ( s ) = x 0 ( s ) .
Since PF ODEs are used, the cycle consistency property is guaranteed.
In practice, even with discretization error, DDIBs incur almost negligible cycle inconsistency (Section 4.1). In contrast,
GAN-based methods are not guaranteed the cycle consistency property by default, and have to incorporate additional training terms to optimize for cycle consistency over two domains.
Data Privacy in Both Domains In the DDIBs translation process, only the source and target diffusion models are required, whose training processes do not depend on knowledge of the domain pair a priori .
In fact, this process can even be performed in a privacy sensitive manner (graphic illustration in Appendix A).
Let Alice and Bob be the data owners of the source and target domains, respectively.
Suppose Alice intends to translate images to the target domain.
However, Alice does not want to share the data with Bob (and vice versa, Bob does not want to reveal their data either).
Then,
Alice can simply train a diffusion model with the source data, encode the data to the latent space, transmit the latent codes to Bob, and then ask Bob to run their trained diffusion model and send the results back.
In this procedure, only the latent code and the target results are transmitted between the two data vendors, and both parties have naturally ensured that their data are not directly revealed.
DDIBs are Two Concatenated Schr¨odinger Bridges DDIBs link the source data distribution to the latent space, and then to the target distribution.
What is the nature of such connections between distributions?
We offer an answer from an optimal transport perspective: these connections are special
Schr¨odinger Bridges between distributions.
This, in turn, explicates the name of our method: dual diffusion implicit bridges are based on denoising diffusion implicit models (Song et al., 2020a), and contain two separate Schr¨odinger Bridges that connect the data and latent distributions.
Speciﬁcally, as considered earlier, when conditions about the policies z t , ˆ z t in Eq.
(5) and the density p 1 ( x ) being a Gaussian prior are met, the data likelihoods (at t = 0 ) for SGMs and SBPs are identical.
Indeed, these conditions are fulﬁlled in SGMs and particularly in DDIMs.
This veriﬁes SGMs as special linear or degenerate SBPs.
Forward and reverse solving the PF ODE for SGMs, as done in DDIBs, is equivalent to ﬂowing through the optimal processes of particular SBPs.
Proposition 3.2 (PF ODE Equivalence 1 ) .
Eq.
(2) is equivalent to Eq.
(6) with forward, backward policies ( z t , ˆ z t ) =
(0 , g r x log p t ( x )) as attained in SGMs and particularly in DDIMs.
Thus, DDIBs are intrinsically entropy-regularized optimal transport: they are Schr¨odinger Bridges between the source and the latent, and between the latent and the target distributions.
The translation process can then be understood as traversing through two concatenated Schr¨odinger Bridges, one forward and one reversed.
The mapping is unique and minimizes an (regularized) optimal transport objective, which probably elucidates the superior performance of DDIBs.
In contrast, if we train the source and target models separately with normalizing ﬂow models that do not have such a connection, there are many viable invertible mappings, and the resulting image translation algorithm may not necessarily have good performance.
This is probably the reason why AlignFlow (Grover et al., 2020) still have to incorporate an adversarial loss even when cycle-consistency is guaranteed.
E XPERIMENTS
We present a series of experiments to demonstrate the effectiveness of DDIBs.
First, we describe synthetic experiments on two-dimensional datasets, to demonstrate DDIBs’ cycle-consistent and 1 Proof in Appendix D.
[Table]
optimal transport properties.
Next, we validate DDIBs on a variety of image translation tasks, including color transfer, paired and unpaired translation, and conditional ImageNet translation 2 .
4. 2D S YNTHETIC E XPERIMENTS
We ﬁrst perform domain translation on synthetic datasets drawn from complex two-dimensional distributions, with various shapes and conﬁgurations, in Fig. 2a.
In total, we consider six 2D datasets:
Moons (M); Checkerboards (CB); Concentric Rings (CR); Concentric Squares (CS); Parallel Rings (PR); and Parallel Squares (PS).
The datasets are all normalized to have zero mean, and identity covariance.
We assign colors to points based on the point identities ( i.e. , if a point in the source domain is red, its corresponding point in the target domain is also colored red).
Clearly, the transformation is smooth between columns.
For example, on the top-right corner, red points in the CR dataset are mapped to similar coordinates, both in the latent and in the target domain.
Cycle Consistency Fig.
2b illustrates the cycle consistency property guaranteed by DDIBs.
It concerns 2D datasets: Moons, and Checkerboards.
Starting from the Moons dataset, DDIBs ﬁrst obtain the latents and construct the Checkerboards points.
Next, DDIBs do translations in the reverse direction, transforming the points back to the latent and the Moons space.
After this round trip, points are approximately mapped to their original positions.
A similar, smooth color topology is observed in this experiment.
Table 1 reports quantitative evaluation results on cycle-consistent translation among multiple datasets.
As the datasets are normalized to unit standard deviation, the reported values are negligibly small and validate the cycle consistent property of DDIBs.
4. E XAMPLE -G UIDED C OLOR T RANSFER
DDIBs can be used on an interesting application: example-guided color transfer.
This refers to the task of modifying the colors of an input image, conditioned on the color palette of a reference image.
To use DDIBs for color transfer, we train one diffusion model per image, on its normalized RGB space.
During translation, DDIBs obtain encodings of the original colors, and apply the diffusion model of the reference image to attain the desired color palette.
Fig.
3 visualizes our color experiments.
[Table]
[Table]
Comparison to Alternative OT Methods As DDIBs are related to regularized OT, we compare the pixel-wise MSEs between color-transferred images generated by DDIBs
[Table]
and images produced by alternate methods, in Table 2.
We compare DDIBs against four OT methods: Earth Mover’s
Distance; Sinkhorn distance (Cuturi, 2013); linear and Gaussian mapping estimation (Perrot et al., 2016).
Results of DDIBs are very close to those of OT methods.
Appendix E.2 includes full color translation results.
4. Q UANTITATIVE T RANSLATION E VALUATION
Quantitatively, we demonstrate that DDIBs achieve competitive results on paired domain tests.
Such numerical evaluation is despite that DDIBs are formulated with a weaker setting: diffusion models are trained independently, on separate datasets.
In comparison, methods such as CycleGAN and
AlignFlow assume access to both datasets during training and jointly optimize for the translation loss.
Paired Domain Translation As in similar works, we evaluate DDIBs on benchmark paired datasets (Zhu et al., 2017): Facades and Maps.
Both are image segmentation tasks.
In the pairs of datasets, one dataset contains real photos taken via a camera or a satellite; while the other comprises the corresponding segmentation images.
These datasets provide one-to-one image alignment, which allows quantitative evaluation via a distance metric such as mean-squared error (MSE) between generated samples and the corresponding ground truth.
To facilitate the workings of DDIBs, we additionally employ a color conversion heuristic motivated by optimal transport on image colors (Appendix E.1).
Table 3 reports the evaluation results.
Surprisingly, DDIBs are able to produce segmentation images that surpass alternative methods in MSE terms; while reverse translations also achieve decent performance.
4. C LASS -C ONDITIONAL I MAGE N ET T RANSLATION
In this experiment, we apply DDIBs to translation among ImageNet classes.
To this end, we leverage the pretrained diffusion models from Dhariwal & Nichol (2021).
The authors optimized performance of diffusion models, and end up with a “UNet” (Ho et al., 2020) architecture with particular width, attention and residual conﬁgurations.
The models are learned on 1000 ImageNet classes, each with around 1000 training images, and at a variety of resolutions.
Our experiments use the model with resolution 256 ⇥ 256 .
Moreover, these models incorporate a technique known as classiﬁer guidance (Dhariwal & Nichol, 2021), that leverage classiﬁer gradients to steer the sampling process towards arbitrary class labels during image generation.
The learned models combined with classiﬁer guidance can be effectively considered as 1000 different models.
Fig.
4a exhibits select translation samples, where the source images are from ImageNet validation sets.
DDIBs are able to create faithful target images that maintain much of the original content such as animal poses, complexions and emotions, while accounting for differences in animal species.
Multi-Domain Translation Given conditional models on the individual domains, DDIBs can be applied to translate between arbitrary pairs of source-target domains, while requiring no additional ﬁne-tuning or adaptation.
Fig.
4b displays results of translating a common image of a roaring lion (with class label 291), to various other ImageNet classes.
Interestingly, some animals roar, while others stick their tongues out.
DDIBs successfully internalize characteristics of distinct animal species, and produce closest animal postures in OT distances to the original shouting lion.
[Figure]
R ELATED W ORKS
Diffusion Models With roots in thermodynamics (Sohl-Dickstein et al., 2015), diffusion models reverse the dynamics of a noising process to create data samples.
These models excel in tasks ranging from stroke-based image editing and composition (Meng et al., 2021), image density estimation (Kingma et al., 2021), to linear-inverse based image restoration (Kawar et al., 2022).
Seminal diffusion models are denoising diffusion probabilistic models (DDPMs, Ho et al.
(2020)), that proposed Gaussian parameterizations in the ELBO objective; while ILVR (Choi et al., 2021) invents a conditional method to guide DDPM generation towards reference images.
Denoising diffusion implicit models (DDIMs, Song et al. (2020a)) improve the inference speed of DDPMs via a nonMarkovian sampling process.
DDIMs invent an ODE to describe its generative process, that sits at the core of DDIBs.
While GANs (Goodfellow et al., 2014) have been dominating deep generative leaderboards, a recent publication (Dhariwal & Nichol, 2021) optimizes the architecture of diffusion models to outperform
GANs in image synthesis; whereas Nichol & Dhariwal (2021) improves diffusion models to win log-likelihood competitions.
In our experiments, we leverage their industry-scale models to produce high-quality translation results.
Diffusion Models for Unpaired Image Translation While the widely-received GANs (Goodfellow et al., 2014; Zhu et al., 2017; Almahairi et al., 2018; Zhao et al., 2020) achieve remarkable performance in unpaired image translation, an array of recent publications instead employ diffusion models.
For instance, DiffusionCLIP (Kim & Ye, 2021) performs image manipulation by incorporating text guidance into DDPMs; and UNIT-DDPM (Sasaki et al., 2021) proposes a novel coupling between the domain pairs, to train joint DDPMs for translation.
Unlike their joint training, DDIBs directly apply pretrained diffusion models and leverage the geometry of the shared space for image translation.
Schr¨odinger Bridge for Generative Modeling Solutions to the SBP (Section 2) are commonly used to design generative models.
For instance, De Bortoli et al.
(2021) proposes a novel approximation of the Iterative Proportional Fitting scheme to solve the SBP for image generation in continuous spaces, analogous to the Sinkhorn algorithm in discrete domain; while Wang et al.
(2021) presents a new generative formulation via entropic interpolation with an SBP.
Strong connections between
SBPs and diffusion models have been established in Chen et al.
(2021a) where diffusion models are shown to be special cases of SBPs.
While the initial design of DDIBs was not to explicitly mimic a
Schr¨odinger Bridge, our theoretical analysis reveals its true characterization as the solution to a SBP.
C ONCLUSIONS
We present Dual Diffusion Implicit Bridges (DDIBs), a new, simplistic image translation method that stems from latest progresses in diffusion models, and is theoretically grounded as Schr¨odinger
Bridges in the image space.
DDIBs solve two key problems.
First, DDIBs avoid optimization on a coupled loss speciﬁc to the given domain pair only.
Second, DDIBs better safeguard dataset privacy as they no longer require presence of both datasets during training.
Powerful pretrained diffusion models are then integrated into our DDIBs framework, to perform a comprehensive series of experiments that prove DDIBs’ practical values in domain translation.
Our method is limited in its application to color transfer, as one model is required for each image, which demands signiﬁcant compute for mass experiments.
Rooted in optimal transport, DDIBs translation mimics the mass-moving process which may be problematic at times (Appendix C).
Future work may remedy these issues, or extend DDIBs to applications with different dimensions in the source and target domains.