EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equations
Abstract
Score-based diffusion generative models (SDGMs) have achieved the SOTA FIDresults in unpaired image-to-image translation (I2I).
However, we notice that existing methods totally ignore the training data in the source domain, leading tosub-optimal solutions for unpaired I2I. To this end, we propose energy-guidedstochastic differential equations (EGSDE) that employs an energy function pretrained on both the source and target domains to guide the inference process of apretrained SDE for realistic and faithful unpaired I2I. Building upon two featureextractors, we carefully design the energy function such that it encourages thetransferred image to preserve the domain-independent features and discard domainspecific ones.
Further, we provide an alternative explanation of the EGSDE as aproduct of experts, where each of the three experts (corresponding to the SDE andtwo feature extractors) solely contributes to faithfulness or realism.
Empirically, wecompare EGSDE to a large family of baselines on three widely-adopted unpaired
I2I tasks under four metrics.
EGSDE not only consistently outperforms existing
SDGMs-based methods in almost all settings but also achieves the SOTA realismresults (e.g., FID of 65.82 in Cat → Dog and FID of 59.75 in Wild → Dog on
AFHQ) without harming the faithful performance.
Introduction
Unpaired image-to-image translation (I2I) aims to transfer an image from a source domain to arelated target domain, which involves a wide range of computer vision tasks such as style transfer,super-resolution and pose estimation [32].
In I2I, the translated image should be realistic to fit thestyle of the target domain by changing the domain-specific features accordingly and faithful topreserve the domain-independent features of the source image.
Over the past few years, generativeadversarial networks [12] (GANs)-based methods [10, 51, 47, 33, 3, 49, 39, 18, 17, 25, 10] dominatedthis field due to their ability to generate high-quality samples.
In contrast to GANs, score-based diffusion generative models (SDGMs) [43, 16, 31, 44, 2] perturbdata to a Gaussian noise by a diffusion process and learn the reverse process to transform thenoise back to the data distribution.
Recently, SDGMs achieved competitive or even superior imagegeneration performance to GANs [9] and thus were naturally applied to unpaired I2I [7, 29], whichhave achieved the state-of-the-art FID [13] and KID [4] results empirically.
However, we notice thatthese methods did not leverage the training data in the source domain at all .
Indeed, they trained adiffusion model solely on the target domain and exploited the test source image during inference (seedetails in Sec. 2.2).
Therefore, we argue that if the training data in the source domain can be exploitedtogether with those in the target domain, one can learn domain-specific and domain-independentfeatures to improve both the realism and faithfulness of the SDGMs in unpaired I2I.
[Figure]
npu t
[Figure]
u t
[Figure]
t
[Figure]
Output realistic expert realistic
[Figure]
= 𝜆 !
=faithful 𝜆 " =𝜆 " =faithful expert
[Figure]
To this end, we propose energy-guided stochastic differential equations (EGSDE) that employs anenergy function pretrained across the two domains to guide the inference process of a pretrained SDEfor realistic and faithful unpaired
I2I. Formally, EGSDE defines a valid conditional distribution via areverse time SDE that composites the energy function and the pretrained SDE.
Ideally, the energyfunction should encourage the transferred image to preserve the domain-independent features anddiscard domain-specific ones.
To achieve this, we introduce two feature extractors that learn domainindependent features and domain-specific ones respectively, and define the energy function uponthe similarities between the features extracted from the transferred image and the test source image.
Further, we provide an alternative explanation of the discretization of EGSDE in the formulation ofproduct of experts [15].
In particular, the pretrained SDE and the two feature extractors in the energyfunction correspond to three experts and each solely contributes to faithfulness or realism.
Empirically, we validate our method on the widely-adopted AFHQ [8] and CeleA-HQ [19] datasetsincluding Cat → Dog, Wild → Dog and Male → Female tasks.
We compare to a large familyof baselines, including the GANs-based ones [33, 51, 17, 25, 3, 10, 49, 50] and SDGMs-basedones [7, 29] under four metrics (e.g., FID).
EGSDE not only consistently outperforms the directcompetitor [7, 29] in almost all settings but also achieves the SOTA realism results (FID of 65.82 in
Cat → Dog and FID of 59.75 in Wild → Dog on AFHQ) without harming the faithful performance.
Furthermore, our method can flexibly trade off realism and faithfulness by tuning weighting hyperparameters in the energy function and be extended to multi-domain translation easily.
Background2.1 Score-based Diffusion Generative Models
Score-based diffusion generative models (SDGMs) gradually perturb data by a forward diffusionprocess, and then reverse it to recover the data [44, 2, 42, 16, 9].
Let q ( y 0 ) be the unknown datadistribution on R D .
The forward diffusion process { y t } t ∈ [0 ,T ] , indexed by time t , can be representedby the following forward SDE:
[Equation]
where w ∈ R D is a standard Wiener process, f ( · , t ) :
R D → R D is the drift coefficient andg ( t ) ∈ R is the diffusion coefficient.
The f ( y , t ) , g ( t ) is related into the noise size and determines the perturbation kernel q t |
( y t | y 0 ) from time 0 to t . In practice, the f ( y , t ) is usually affine so thatthe the perturbation kernel is a linear Gaussian distribution and can be sampled in one step.
Let q t ( y ) be the marginal distribution of the SDE at time t in Eq.
Its time reversal can bedescribed by another SDE [44]:
[Equation]
where w is a reverse-time standard Wiener process, and d t is an infinitesimal negative timestep.
Wewish to solve the reverse SDE in Eq.
(2) to generate data, where we need to learn the score function∇
y log q t ( y ) . [44] adopts a score-based model s ( y , t ) to approximate the unknown ∇ y log q t ( y ) byscore matching, thus inducing a score-based diffusion generative model (SDGM), which is definedby a SDE:
[Equation]
There are numerous SDE solver to solve the Eq.
(3) to generate images. [44] discretizes it using the Euler-Maruyama solver.
Formally, adopting a step size of h , the iteration rule from s to t =
s − h is:
[Equation]
2.2 SDGMs in Unpaired Image to Image Translation
Given unpaired images from the source domain X ⊂ R D and the target domain Y ⊂ R D as thetraining data, the goal of unpaired I2I is to transfer an image from the source domain to the targetdomain.
Such a process can be formulated as designing a distribution p ( y 0 | x 0 ) on the target domain
Y conditioned on an image x 0 ∈ X to transfer.
The translated image should be realistic for the targetdomain by changing the domain-specific features and faithful for the source image by preserving thedomain-independent features.
ILVR [7] uses a diffusion model on the target domain for realism .
Formally, ILVR starts fromy T ∼ N ( 0 , I ) and samples from the diffusion model according to Eq.
(4) to obtain y t .
Forfaithfulness , it further refines y t by adding the residual between the sample y t and the perturbedsource image x t through a non-trainable low-pass filter
[Equation]
where Φ( · ) is a low-pass filter and q t | 0 ( ·|· ) is the perturbation kernel determined by the forward SDEin Eq.
Similarly, SDEdit [29] also adopts a SDGM on the target domain for realism , i.e., sampling from the
SDGM according to Eq.
For faithfulness , SDEdit starts the generation process from the noisysource image y M ∼ q M |
( y M | x 0 ) , where M is a middle time between 0 and T , and is chosen topreserve the original overall structure and discard local details.
We use p r 1 ( y 0 | x 0 ) to denote themarginal distribution defined by such SDE conditioned on x 0 .
Notably, these methods did not leverage the training data in the source domain at all and thus can besub-optimal in terms of both the realism and faithfulness in unpaired I2I.
Method
To overcome the limitations of existing methods [7, 29] as highlighted in Sec. 2.2, we proposeenergy-guided stochastic differential equations (EGSDE) that employs an energy function pre-trainedacross the two domains to guide the inference process of a pretrained SDE for realistic and faithfulunpaired I2I
(see Fig. 2).
EGSDE defines a valid conditional distribution p ( y 0 | x 0 ) by compositing apretrained SDE and a pretrained energy function under mild regularity conditions 1 as follows:
[Equation]
6) where w is a reverse-time standard Wiener process, d t is an infinitesimal negative timestep, s ( · , · ) : 100 R D × R → R D is the score-based model in the pretrained SDE and E ( · , · , · ) : R D × R D × R → R is the [Figure] energy function.
The start point y M is sampled from the perturbation distribution q M |
( y M | x 0 ) [29],where M = 0 .
5 T typically.
We obtain the transferred images by taking the samples at endpoint t = 0following the SDE in Eq.
Similar to the prior work [7, 29], EGSDE employs an SDE trained solely in the target domain asin Eq.
(2), which defines a marginal distribution of the target images and mainly contributes tothe realism of the transferred samples.
In contrast, the energy function involves the training dataacross both the source and target domain, making EGSDE distinct from the prior work [7, 29].
Notably, although many other possibilities exist, we carefully design the energy function such that it(approximately) encourages the sample to retain the domain-independent features and discard thedomain-specific ones to improve both the faithfulness and realism of the transferred sample.
Below,we formally formulate the energy function.
3.1 Choice of Energy
In this section, we show how to design the energy function.
Intuitively, during the translation,the domain-independent features (pose, color, etc .
on Cat → Dog) should be preserved while thedomain-specific features (beard, nose, etc .
on Cat → Dog) should be changed accordingly.
Motivatedby this, we decompose the energy function E ( y , x , t ) as the sum of two log potential functions [5]:
[Equation]
where E i ( · , · , · ) : R D × R D
×
R
→
R
and
E s ( · , · , · ) : R D × R D × R → R are the log potential functions,x t is the perturbed source image in the forward SDE, q t | 0 ( ·|· ) is the perturbation kernel from time 0to time t in the forward SDE, S s ( · , · , · ) :
R D × R D × R → R and S i ( · , · , · ) : R D × R D × R → Rare two functions measuring the similarity between the sample and perturbed source image, andλ s ∈ R > 0 , λ i ∈ R > 0 are two weighting hyper-parameters.
Note that the expectation w.r.t.
q t | 0 ( x t | x )in Eq.
(7) guarantees that the energy function changes slowly over the trajectory to satisfy theregularity conditions in Appendix A.1.
To specify S s ( · , · , · ) , we introduce a time-dependent domain-specific feature extractor E s ( · , · ) : 125 R D × R → R C × H × W , where C is the channel-wise dimension, H and W are the dimension ofheight and width.
In particular, E s ( · , · ) is the all but the last layer of a classifier that is trained onboth domains to predict whether an image is from the source domain or the target domain.
Intuitively,
E s ( · , · ) will preserve the domain-specific features and discard the domain-independent features foraccurate predictions.
Building upon it, S s ( · , · , · ) is defined as the cosine similarity between the features extracted from the generated sample and the source image as follows:
[Equation]
where E hws ( · , · ) ∈ R C denote the channel-wise feature at spatial position ( h, w ) .
Here we employthe cosine similarity since it preserves the spatial information and helps to improve the FID scoreempirically (see Appendix C.1 for the ablation study).
Intuitively, reducing the energy value in Eq.
(7)encourages the transferred sample to discard the domain-specific features to improve realism.
To specify S i ( · , · , · ) , we introduce a domain-independent feature extractor E i ( · , · ) :
R D × R → R D ,which is a low-pass filter.
Intuitively, E i ( · , · ) will preserve the overall structures (i.e., domainindependent features) and discard local information like textures (i.e., domain-specific features).
Building upon it, S i ( · , · , · ) is defined as the negative squared L 2 distance between the featuresextracted from the generated sample and source image as follows:
[Equation]
Here, we choose negative squared L 2 distance as the similarity metric because it helps to preservemore domain-independent features empirically (see Appendix C.1 for the ablation study).
Intuitively,reducing the energy value in Eq.
(7) encourages the transferred sample to preserve the domainindependent features to improve faithfulness.
In this paper we employ a low-pass filter for itssimpleness and effectiveness while we can train more sophisticated E i , e.g., based on disentangledrepresentation learning methods [38, 6, 14, 22, 27], on the data in the two domains.
In our preliminary experiment, alternative to Eq.
(7), we consider a simpler energy function that onlyinvolves the original source image x as follows:
[Equation]
which does not require to take the expectation w.r.t.
x t .
We found that it did not perform well becauseit is not reasonable to measure the similarity between the noise-free source image and the transferredsample in a gradual denoising process.
See Appendix C.2 for empirical results.
3.2 Solving the Energy-guided Reverse-time SDE
Based on the pretrained score-based model s ( y , t ) and energy function E ( y , x , t ) , we can solve theproposed energy-guided SDE to generate samples from conditional distribution p ( y 0 | x 0 ) .
There arenumerical solvers to approximate trajectories from SDEs.
In this paper, we take the Euler-Maruyamasolver following [29] for a fair comparison.
Given the EGSDE as in Eq.
(6) and adopting a step sizeh , the iteration rule from s to t =
s − h is:
[Equation]
The expectation in E ( y s , x 0 , s ) is estimated by the Monte Carlo method of a single sample forefficiency.
For brevity, we present the general sampling procedure of our method in Algorithm 1.
Thesampling procedure for the specific variance preserve energy-guided SDE (VP-EGSDE)
[44, 16] isexplained in Appendix A.3 and is conducted in our experiments.
Following SDEdit [29], we furtherextend this by repeating the Algorithm 1 K times (see details in Algorithm 2 in Appendix A.2).
3.3 EGSDE as Product of Experts
Inspired by the posterior inference process in diffusion models [41], we present a product of experts [15] explanation for the discretized sampling process of EGSDE, which formalizes our motivation in an alternative perspective and provides insights on the role of each component in EGSDE.
We first define a conditional distribution ˜ p ( y t | x 0 ) at time t as a product of experts:
[Equation]
where Z t is the partition function, p e ( y t | x 0 ) ∝ exp( −E ( y t , x 0 , t )) and p r 1 ( y t | x 0 ) is the marginaldistribution at time t defined by SDEdit based on a pretrained SDE on the target domain.
Algorithm
Require: the source image x 0 , the initial time M , denoising steps N , weighting hyper-parameters λ s , λ i , the similarity function S s ( · , · , · ) , S i ( · , · , · ) , the score function s ( · , · ) y ∼ q M | 0 ( y | x 0 )
# the start point h
=
MN for i
= N to 1 do s ← ih x ∼ q s | 0 ( x | x 0 )
# sample perturbed source image from the perturbation kernel
E ( y , x , s ) ← λ s S
s
( y , x , s )
− λ
i S i
( y , x , s )
# compute energy with one Monte Carlo y ← y − [ f ( y , s ) − g
( s )
( s ( y , s ) − ∇ y E
( y , x , s ))]
h # the update rule in Eq.
(12) z ∼ N ( 0 , I ) if i > 1 , else z =y ← y + g ( s ) √ h z end for y 0 ← y return y
To sample from ˜ p
( y t | x 0 ) , we need to construct a transition kernel ˜ p ( y t | y s ) , where t = s − h and his small.
Following [41], using the desirable equilibrium ˜ p ( y t | x 0 ) =
(cid:82) ˜ p ( y t | y s )
˜ p ( y s | x 0 )
d y s , weconstruct the ˜ p ( y t | y s ) as follows:
[Equation]
(14) where ˜ Z t ( y s ) is the partition function and p ( y t | y s )
=
N ( µ ( y s , h ) , Σ( s, h ) I ) is the transitionkernel of the pretrained SDE in Eq.
(4), i.e., µ ( y s , h ) =
y s − [ f ( y s , s ) − g ( s ) 2 s ( y s , s )]
h andΣ( s, h ) =
g ( s ) 2 h .
Assuming that E ( y t , x 0 , t ) has low curvature relative to Σ( s, h ) − 1 , it can beapproximated using Taylor expansion around µ ( y s , h ) and further we can obtain
[Equation]
More details about derivation are available in Appendix A.4.
We can observe the transition kernel˜ p ( y t | y s ) in (15) is equal to the discretization of our EGSDE in Eq.
Therefore, solving theenergy-guided SDE in a discretization manner is approximately equivalent to drawing samples froma product of experts in Eq.
Note that E ( y t , x 0 , t ) =
λ s E s ( y t , x 0 , t )
+ λ i E i
( y t , x 0 , t ) , the˜ p ( y t | x 0 ) can be rewritten as:
[Equation]
where p r 2 ( y t | x 0 ) ∝ exp( − λ s E s ( y t , x 0 , t )) , p f ( y t | x 0 ) ∝ exp( − λ i E i ( y t , x 0 , t )) .
In Eq.
(16), by setting t = 0 , we can explain that the transferred samples approximately follow thedistribution defined by the product of three experts, where p r 1 ( y t | x 0 ) and p r 2 ( y t | x 0 ) are the realismexperts and p f ( y t | x 0 ) is the faithful expert , corresponding to the score function s ( y , t ) and the logpotential functions E s ( y , x , t ) and E i
( y , x , t ) respectively.
Such a formulation clearly explains therole of each expert in EGSDE and supports our empirical results.
Related work
Apart from the prior work mentioned before, we discuss other related work including GANs-basedmethods for unpaired I2I and SDGMs-based methods for image translation.
GANs-based methods for Unpaired I2I.
The methods in two-domain unpaired I2I are mainly dividedinto two classes: two-side and one-side mapping [32, 49].
In the two-side framework [51, 47, 24, 28,26, 23, 11, 1, 46, 48, 20], the cycle-consistency constraint is the most widely-used strategy such as in
CycleGAN [51], DualGAN [47] and DiscoGAN [24].
The key idea is that the translated image shouldbe able to be reconstructed by an inverse mapping.
More recently, there are numerical studies toimprove this such as SCAN [26] and U-GAT-IT [23].
Since such bijective projection is too restrictive,several studies are devoted to one-side mapping [33, 3, 10, 51, 49, 34, 18].
One representative [Figure] approach is to design some kind of geometry distance to preserve content [32].
For example,
DistanceGAN [3] keeps the distances between images within domains.
GCGAN [10] maintainsgeometry-consistency between input and output.
CUT [33] maximizes the mutual informationbetween the input and output using contrastive learning.
LSeSim [49] learns spatially-correlativerepresentation to preserve scene structure consistency via self-similarities.
SDGMs-based methods for Image Translation.
Several studies leveraged SDGMs for imagetranslation due to their powerful generative ability and achieved good results.
For example, DiffusionCLIP [21] fine-tune the score network with CLIP [35] loss and has achieved great performance in awide range of applications including text-driven image manipulation, zero-shot image manipulationbetween unseen domains and multi-attribute transfer. GLIDE [30] focus on text-to-image translation.
As for I2I, SR3 [37] and Palette [36] learn a conditional SDGM and outperform state-of-art GANs-based methods on super-resolution, colorization and so on tasks, which needs paired data.
Forunpaired I2I, ILVR [7] and SDEdit [29] utilize a SDGM on the target domain and exploited the testsource image to refine inference, which ignored the training data in the source domain.
Comparedwith these methods, our method employs an energy function pretrained across both the source andtarget domains to improve the realism and faithful of translated images.
Experiment
Datasets.
We validated the EGSDE on following datasets, where all images are resized to 256 × 256 :(1) CelebaA-HQ [19], where we perform Male → Female .
(2) AFHQ [8], where we perform Cat → Dogand Wild → Dog .
More details is available in Appendix B.
Implementation.
The time-dependent domain-specific extractor E s ( x , t ) is trained based on thebackbone in [9].
The resize function including downsampling and upsampling operation is usedas low-pass filter and is implemented by [40].
For generation process, the initial time M is set0 . 5 .
Denoising steps N is set 500.
By default, the weight parameter λ s and λ i are set 500 and 2respectively.
More details about implementation are available in Appendix B.
[Table]
65.82 ± 0.
47.22 ± 0.
19.31 ± 0.
[Table]
[Table]
EGSDE † 51.04 ± 0.
62.06 ± 0.
17.17 ± 0.
[Table]
[Table]
[Table]
[Table]
CUT [33] 92.
[Table]
[Table]
[Table]
ILVR [7] 75.
[Table]
[Table]
[Table]
[Table]
[Table]
[Table] [Table] [Table] SDEdit [29] 68.
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
EGSDE 59.
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
EGSDE † 50.43± 0.
66.52±
[Table]
[Table]
40± 0.
0.300±
[Table]
[Table]
CUT [33] 36.
[Table]
[Table]
ILVR [7] 46.
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
SDEdit [29] 49.
[Table]
43.70 ± 0.
20.03 ± 0.
[Table]
[Table]
[Table]
EGSDE 41.93 ± 0. 42.04 ± 0. 20.35 ± 0. 0.574 ± 0.
EGSDE † 30.61 ± 0.
53.44 ± 0.
18.32 ± 0.
0.510 ± 0. Evaluation Metrics. For realism, we report the widely-used Frechet Inception Score (FID) [13]between translated images and the target dataset.
To quantify faithfulness, we report the L 2 distance,
PSNR and SSIM [45] between each input-output pair.
To quantify both faithfulness and realism,we leverage Amazon Mechanical Turk(AMT) human evaluation to perform pairwise comparisonsbetween the baselines and EGSDE.
5.1 Two-Domain Unpaired Image Translation
In this section, we compare EGSDE with the following state-of-the-art I2I methods in three tasks:
SDGMs-based methods including ILVR [7] and SDEdit [29], and GANs-based methods including
CUT [33], which are reproduced using public code.
We also report the performance of other state-ofthe-art GANs-methods on Cat → Dog, which are public results from CUT[33] and ITTR [50].
Weprovide more details about reproductions and evaluation in Appendix C.
The quantitative comparisons and qualitative results are shown in Table 1 and Figure 3.
We canderive several observations.
First , our method outperforms the SDGMs-based methods significantlyin almost all realism and faithfulness metrics, suggesting the effectiveness of employing energy
[Figure]
K= K=
K=
S D E d it
K=
K= K=
[Figure]
function pretrained on both domains to guide the generation process. Especially, compared with themost direct competitor, i.e., SDEdit, with a lower L 2 distance at the same time, EGSDE improvesthe FID score by 8.35, 8.76 and 7.5 on Cat → Dog, Wild → Dog and Male → Female respectively. Second , EGSDE outperforms the current state-of-art GANs-based methods by a large margin on thechallenging AFHQ dataset including Cat → Dog and Wild → Dog tasks. In addition, the humanevaluation shows that EGSDE are preferred compared to all baselines (> 50 % ).
The qualitative resultsin Figure 3 agree with quantitative comparisons in Table 1, where our method achieved the resultswith the best visual quality for both realism and faithfulness .
We show more results in Appendix C.
5.2 Ablation Studies
Repeating K Times.
Following SDEdit [29], we show the results of repeating the Algorithm 1 Ktimes.
The quantitative and qualitative results are depicted in Table 2 and Figure 4.
The experimentalresults show the EGSDE outperforms SDEdit in each K step in all metrics.
With the increase of K ,the SDEdit generates more realism images but the faithful metrics decrease sharply, because it onlyutilizes the source image at the initial time M .
The function of each expert.
We validate the function of realistic expert E s ( y , x , t ) and faithfulexpert E i ( y , x , t ) by changing the weighting hyper-parameter λ s and λ i .
As shown in Table 3 and
Figure 1, larger λ s results in more realistic images and larger λ i results in more faithful images.
different K times on Male → Female.
The results on Cat → Dog and Wild → Dog are reported in
Appendix C.4.
[Table]
[Table]
[Table]
Conclusions and Discussions
In this paper
[Table]
we propose energy-guided stochastic differential equations (EGSDE) for realistic andfaithful unpaired I2I, which employs an energy function pretrained on both domains to guide thegeneration process of a pretrained SDE.
Building upon two feature extractors, we carefully designthe energy function to preserve the domain-independent features and discard domain-specific ones ofthe source image.
We demonstrate the EGSDE by outperforming state-of-art I2I methods on threewidely-adopted unpaired I2I tasks.
One limitation of this paper is we employ a low-pass filter as the domain-independent feature extractorfor its simpleness and effectiveness while we can train more sophisticated extractor, e.g. based ondisentangled representation learning methods [38, 6, 14, 22, 27], on the data in the two domains.
Weleave this issue in future work.
In addition, we must take care to exploit the method to avoid thepotential negative social impact (i.e., generating fake images to mislead people).
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section 6.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] SeeSection 6.(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See AppendixA.1.(b)
Did you include complete proofs of all theoretical results?
[Yes] See Appendix A.4.3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See Section 5and Appendix B
(b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Appendix B and Appendix C
(c) Did you report error bars (e g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] See Table
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Appendix B.4.
If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Section 5 andAppendix B(b) Did you mention the license of the assets?
[Yes] See Appendix B(c)
Did you include any new assets either in the supplemental material or as a URL?
[Yes]See Appendix B.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A] We use the public datasets.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
[Table]
information or offensive content?
[N/A] We use the public datasets. [Table]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if [Table] applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[N/A]