Spatially Sparse Inference for Generative Image Editing
Anonymous Author(s)
AffiliationAddress email
Abstract
Deep generative models excel at synthesizing photo-realistic images and enablevarious image synthesis and editing applications.
However, when editing a photo,existing methods tend to re-synthesize the entire output from scratch, includingthe unedited region, leading to a significant waste of computation, especially forminor editing operations.
In this work, we present Spatially Sparse Inference (SSI),a general-purpose speedup technique that selectively performs computation foredited regions and is compatible with different types of generative models.
Ourkey observation is that user editing is often incremental in the interactive setting.
This allows us to pre-compute the feature maps of the original image.
Given anedited image, we sparsely apply the filters to the edited regions while reusingthe pre-computed features for the unedited regions.
Based on our algorithm, wepropose Sparse Incremental Generative Engine (SIGE) to convert the theoreticalcomputation reduction to latency reduction on commonly-used hardware.
With1.2%-area edited regions, our method reduces the computation of DDIM by 7 . 5 ×and GauGAN by 18 × while preserving the visual fidelity.
With SIGE, we acceleratethe inference time of DDIM by 3 . {+
 ×+} on RTX3090 and 6 . 6 × on Apple M1 Pro,and GauGAN by 4 . 2 × on RTX3090 and 14 × on Apple M1 Pro.
[Figure]
of DDIM [1] by 4 ∼ 6 × and GauGAN [2] by 15 × for the examples shown in the figures, while preserving the image quality.
Our method could also be applied to existing model compression methods.
When applied to GAN Compression [3], we further accelerate GauGAN by 47 × .
Introduction
Deep generative models ( e.g ., GANs [4] and diffusion models [5, 6]) excel at synthesizing photo- realistic images, enabling many image synthesis and editing applications.
For example, users can edit
[Figure]
Pre
[Figure] an image by drawing sketches [7, 8], semantic maps [7, 2], or strokes [9].
All of these applicationsrequire users to frequently interact with generative models and therefore demand fast inference time.
In practice, content creators often edit images gradually and only update a small image region eachtime.
However, recent generative models often synthesize the entire image, including the uneditedregion, even though only a minor edit has been made, leading to a waste of computation.
As aconcrete example shown in Figure 2, the vanilla DDIM [1] generates an entire image to obtain the16.9% edited region, wasting the majority of the compute.
A naive approach to address this issuewould be to first segment the edited region, synthesize the corresponding output region, and blend theoutput back into the original image.
But this method often creates visible inconsistency between theedited and unedited regions.
How could we save the computation by only updating the edited regionswithout losing global coherence?
In this work, we propose Spatially Sparse Inference (SSI), a general method to accelerate generativeimage editing by utilizing the spatial sparsity of edited regions.
As shown in Figure 2, our key idea isto pre-compute the feature maps of the original image and then reuse these activations when updatingthe sparse edited regions.
Specifically, given a user input, we first compute the difference maskbetween the original image and edited image to locate the edited regions.
For each convolution layerin the model, we only apply the filters to the masked region sparsely while reusing the original imageactivations for the unedited region.
The sparse update can significantly reduce the computation withouthurting the image quality.
However, this process involves a gather-scatter process which has lowhardware efficiency with existing deep learning frameworks, leading to significant latency overheads.
To address the issue, we propose Sparse Incremental Generative Engine (SIGE) that translates thetheoretical computational reduction of our algorithm to latency reduction on various hardware.
To evaluate our method, we automatically create new image editing benchmark datasets on LSUN
Church [10] and Cityscapes [11].
Through extensive experiments, our method could reduce thecomputation on DDIM [1] by 7.5 × , Progressive Distillation [12] by 2.7 × , and GauGAN by 18 ×regarding MACs * , without loss of visual fidelity.
Compared to existing generative model accelerationmethods [3, 13, 14, 15, 16, 17, 18], our method directly uses the pre-trained weights and could beapplied to these methods as a plugin.
When applied to GAN Compression [3], our method reducesthe computation of GauGAN by ∼ 50 × .
See Figure 1 for some examples of our method.
With SIGE,we accelerate DDIM 3.0 × on 3090 and 6.6 × on Apple M1 Pro, and GauGAN 4 . 2 × on 3090 and38 × on Apple M1 Pro.
Our code and benchmark datasets will be released upon publication.
Related Work
Generative models.
Generative models such as GANs [4, 19, 20, 21], diffusion models [6, 5,22], and auto-regressive models [23, 24] have demonstrated impressive photorealistic synthesiscapability.
They have also been extended to conditional image synthesis tasks such as image-to-imagetranslation [25, 7, 26], controllable image generation [9, 27, 2], and real image editing [28, 27, 29, 30,31].
Unfortunately, recent generative models have become increasingly computationally intensive,compared to their recognition counterparts.
For example, GauGAN [2] consumes 281G MACs, 500 ×more than MobileNet [32, 33, 34].
Similarly, one key limitation of diffusion models [6] is theirslow inference time and huge computational cost—in order to generate one image, DDPM requireshundreds or thousands of forwarding steps [6, 22], which is often infeasible in real-world interactivesettings.
To improve the sampling efficiency of DDPMs, recent works [1, 35, 36] propose to interpretthe sampling process of DDPM from the perspective of ordinary differential equations. However,
[Figure] [Figure]
) is already pre computed. first compute a difference mask the original and edited images and the the indices. Then F f , we directly gather the small active the according to the reduced indices, stack the blocks along the batch dimension, and feed them into F l .
After getting the output blocks, we scatter them back into the F l ( A original l ) to get the edited output, which should be similar to F l ( A edited l ) .
this approach still requires hundreds of steps to generate high-quality samples.
To further reducethe sampling cost, DDGAN [37] uses a multimodal conditional GAN to model each denoising step.
Salimans et al .
[12] propose to progressively distills a pre-trained DDPM model into a new modelthat can be sampled in fewer steps.
Although this approach drastically reduces the sampling steps, thedistilled model itself still remains computationally prohibitive.
Unlike prior work, our work focuseson reducing the computation cost of a pre-trained model.
Our work is complimentary to recent effortson model compression and distillation.
Model acceleration.
People apply model compression techniques including pruning [38, 39, 40,41 , 42, 43] and quantization [38, 44, 45, 46, 47, 48] to reduce the computation and model size ofoff-the-shelf deep learning models.
Recent work applies Neural Architecture Search (NAS) [49, 50,51 , 52, 53, 54, 55] to automatically design efficient neural architectures.
The above ideas can besuccessfully applied to accelerate the inference of GANs [3, 56, 57, 58, 13, 59, 14, 15, 16, 17, 18, 60].
Although these methods have achieved prominent compression and speedup ratios, they all reduce thecomputation from the model dimension but failed to exploit the redundancy in the spatial dimensionduring image editing.
Besides, these methods require re-training the compressed model to maintainperformance while our method can be effortlessly applied to existing pre-trained models.
We showthat our method is complimentary to model compression [3] and can be combined to achieve ∼ 50 × MACs reduction in Section 4.1.
Sparse computation.
Sparse computation has been widely explored in the weight domain [61,62, 63, 64], input domain [65, 66], and activation domain [67, 68, 69, 70].
For activation sparsity, RRN [71] utilizes the redundancy in consecutive video frames to accelerate video models.
However,their difference sparsity is unstructured, which requires special hardware to reach its full speeduppotential.
SBNet [67] uses a spatial mask, either from a priori problem knowledge or from anauxiliary network, to sparsify the activations.
It adopts a tiling-based sparse convolution algorithmto handle the spatial sparsity.
More recent works further integrate the mask generation networkinto the sparse execution network in an end-to-end manner [72] and extend the idea to differenttasks [73, 74, 75, 76].
Our method refers to SBNet’s sparse kernel implementation.
Compared to SBNet [67], our mask is directly derived from the difference between the original image and the editedimage.
Additionally, our method reuses the pre-trained weights and does not require any auxiliarynetwork or extra model training.
We also introduce other optimizations such as normalization removaland kernel fusions to better adapt the kernels for image editing.
Method
We build our method based on the following observation: during interactive image editing, a useroften only changes the image content incrementally.
As a result, only a small subset of pixels in a local region are being updated at any moment.
Therefore, we should be able to reuse the activationsof the original input image for the unedited regions.
Figure 3 illustrates our method.
We firstpre-compute all activations of the original input image.
During the editing process, we locate theediting region by computing a difference mask between the original and edited image.
We then reusethe pre-computed original activations for the unedited region and only update the edited region byapplying convolutional filters on it.
In Section 3.1, we demonstrate the sparsity of the intermediateactivations and present our main algorithm.
In Section 3.2, we discuss the technical details on howour Sparse Incremental Generative Engine (SIGE) supports the sparse inference and converts thetheoretical computation reduction to measured latency reduction on hardware.
3.1 Activation Sparsity
Preliminary.
First, we closely study the computation within a single layer.
We denote A original l and
A edited l as the input tensor of the original image and edited image to the l -th convolution layer F l ,respectively.
W l and b l are the weight and bias of the F l .
The output of F l with input A edited l could becomputed in the following way due to the linearity of convolution: [Equation] sparse
[Equation]
where ∗ is the convolution operator and ∆ A l = A edited l − A
[Equation] l .
If we already pre-computed allthe F l ( A original l ) , we only need to compute W l ∗ ∆ A l .
Naïvely, computing W l ∗ ∆ A l has the samecomplexity as W l ∗ A edited l .
However, since the edited input shares similar features with the originalinput given a small edit, ∆ A l should be sparse.
Below, we discuss different strategies to leverage theactivation sparsity to accelerate model inference.
Our first attempt was to prune ∆ A l by zeroing out the smaller elements below a certain thresholdto achieve the target sparsity.
Unfortunately, it is quite difficult for this pruning method to achievemeasured latency reduction due to the overhead of on-the-fly pruning and irregular sparsity pattern.
Structured sparsity.
Fortunately, user edits are often highly structured and localized.
As a result,∆ A l should also share the structured spatial sparsity , where the non-zero values are mostly aggregatedwithin the edited regions.
We then directly use the original image and edited image to compute adifference mask and sparsify ∆ A l with this mask.
3.2 Sparse Engine SIGE
But how could we leverage the structured sparsity in our kernel design?
A naïve approach is to cropthe edited region out of the input feature map for each convolution, and only compute features forthe cropped regions.
Unfortunately, the naive cropping method does not work well for the irregularedited regions ( e.g ., the third row of Figure 5).
Tiling-based sparse convolution.
Instead, as shown in Figure 4(a), we use a tiling-based sparseconvolution algorithm.
We divide ∆ A l into multiple small blocks of the same size spatially and indexthe spatial difference mask.
Each block index refers to a single block with non-zero elements We thengather the non-zero blocks (we also call it active blocks) accordingly along the batch dimension andfeed them into the convolution F l .
Finally, we scatter the output blocks into a zero tensor accordingto the indices to recover the original spatial size and add the pre-computed residual F l ( A original ) back.
This pipeline in Figure 4(a) is equivalent to a simpler pipeline in Figure 4(b).
Instead of gathering∆ A l , we could directly gather A edited l .
The convolution needs be computed with bias b l .
Besides, weneed to scatter the output blocks into F l ( A original l ) instead of a zero tensor.
In this way, we do notneed to store A original l anymore, which saves more memory and reduce the overheads of addition andsubtraction.
Figure 3 shows the visualization of the pipeline.
However, the aforementioned pipeline still fails to produce a noticeable speedup because of kernelcalling and memory movement overheads in the Gather and Scatter .
For example, the originaldense 3 × 3 convolution with 128 channels and input resolution 256 × 256 would take 0.78ms on
[Figure]
⊖
Indices Indices
A edited l
A original l Δ A l
F l ( A original l )
Conv F l Gather
Scatter
Indices Indices
A edited l
F l ( A original l )
RTX3090.
The sparse convolution using pipeline Figure 4(b) on the example at the third row of
Figure 5 (15.5% editing size) needs 0.42ms in total, where the Gather and Scatter take 0.17ms(about 41%) overheads, which cannot be ignored.
To reduce the overhead, we further optimize our
SIGE by pre-computing normalization parameters and applying kernel fusion.
Pre-computing normalization parameters.
For batch normalization [77], it is easy to remove thenormalization layer during inference time, since we can use pre-computed mean and variance statisticsfrom model training.
However, recent deep generative models often use instance normalization [78,79] or group normalization [80, 81], which tends to compute the statistics on the fly during inference.
These normalization layers create overheads as we need to compute the statistics of the full-sizetensors during inference.
However, as the original image and the edited image are quite similar givena small user edit, we assume A original l ≈ A edited l , which allows us to reuse the statistics of A original lfor the normalization instead of recomputing them for A edited l .
Thus, normalization layers could bereplaced by simple Scale+Shift operations with pre-computed A original l statistics.
Kernel fusion.
Although the Gather and Scatter operations do not involve any arithmeticcomputations, their overheads can not be ignored.
To reduce them, we fuse several kernels with the
Gather and Scatter together [67, 82, 83].
Since Scale+Shift and Nonlinearity are all element-wise operations, we can fuse them into
Gather and Scatter .
In this way, these element-wise operations are only applied to the activeblocks ( i.e ., edited regions).
Furthermore, we perform inplace computation to reduce the kernelcalling and memory allocation overheads.
In Scatter , we need to copy the pre-computed activation.
This copying operation has lots ofredundancy, as the majority of elements from the pre-computed activation do not involve in anycomputation given a small edit and will be discarded by the next Gather .
To reduce the tensorcopying overheads, we fuse the Scatter and the following Gather together by directly gathering theedited blocks from the pre-computed activation and the input blocks to be scattered.
Sometimes, theresidual connection in the ResBlock [84] contains a shortcut 1 × 1 convolution to make the channelnumber of residual equal to the ResBlock output.
We also fuse the Scatter in the shortcut branch,main branch and the residual addition together to avoid the tensor copying overheads in shortcut Scatter .
Please refer to our appendix for more details.
Experiments
Below we first describe our datasets, models, and evaluation protocols.
We then discuss our mainqualitative and quantitative results.
Finally, we include a detailed ablation study regarding theimportance of each algorithmic design.
Models.
We conduct experiments on three models, including diffusion models and GAN-basedmodels, to explore the generality of our method.
• DDIM [1] is a fast sampling approach for diffusion models.
It proposes to interpret the sampling
process of diffusion models through the lens of ordinary differential equations.
• Progressive Distillation (PD) [12] adopts network distillation [85] to diffusion models.
It progres-
sively distills a pre-trained diffusion model into multiple models that can be sampled efficiently. • GauGAN [2] is a paired image-to-image translation model, which learns to generate a high-fidelityimage given a semantic label map.
Datasets.
We use the following two datasets in our experiments: [Table] [Table]
: Uniformly pruning 40% weights of the model without fine-tuning. Patch : Cropping the smallest image patch that covers all the edited region of input and scatter the model output back to the original output. 0.19 GauGAN : Uniformly reducing each layer of GauGAN to 19% channels and training from scratch.
For all models, our method consistently outperforms other baselines with less computation.
When applying our method to GAN Compression, we reduce the MACs of GauGAN by 40 × with minor performance degradation.
• LSUN Church .
We use the LSUN Church Outdoor dataset [10] and follow the same preprocessingsteps as prior works [6, 35].
To automatically generate a stroke editing benchmark, we first useDetic [86] to segment the images in the validation set.
For each segmented object, we use itssegmentation mask to inpaint the image by CoModGAN [87] and treat the inpainted image as theoriginal image.
We generate the corresponding user strokes by first blurring the masked regionwith the median filter and quantize it into 6 colors following SDEdit [9].
We collect 454 editingpairs in total (431 synthetic + 23 manual).
Please refer to our appendix for more details regardingthis benchmark.
We evaluate DDIM [1] and PD [12] on this dataset.• Cityscapes .
The dataset [11] contains the images of German street scenes.
The training set and thevalidation set consists of 2975 and 500 images, respectively.
Our editing dataset has 1505 editingpairs in total.
Please refer to our appendix for more details.
We evaluate GauGAN [2] on this dataset.
Metrics.
Following the previous work [9, 3, 2], we use the standard metrics Peak Signal Noise Ratio(PSNR, higher is better), LPIPS (lower is better) [88], and Fréchet Inception Distance (FID, loweris better) [89, 90] † to evaluate the image quality on both LSUN Church [10] and Cityscapes [11].For Cityscapes, we also adopt a semantic segmentation metric to evaluate the generated images.
Specifically, we run DRN-D -105 [91] on the generated images and compute the mean Intersectionover Union (mIoU) of the segmentation results.
Generally, if the generated images look more realisticand correspond to the input better, its mIOUs should be higher.
Implementation details.
We dilate the difference mask by 5, 2, 5, and 1 pixels for DDIM, PD withresolution 128, PD with resolution 256 and GauGAN, respectively.
Besides, we apply our sparsekernel to all convolutions whose input feature map resolutions are larger than 32 × 32 , 16 × 16 ,16 × 32 and 32 × 64 for DDIM, PD, original GauGAN and GAN Compression, respectively.
4.1 Main Results
Image quality.
We report the quantitative results of applying our method on DDIM [1], Progressive
Distillation (PD) [12] and GauGAN [2] in Table 1.
We also show some qualitative examples in
Figure 5.
We compare our method against the following baselines:
• Patch .
We crop the smallest patch that cover all the edited regions in the model input, and scatter
the model output to the original output.
• 40% Pruning .
We uniformly prune 40% weights of the models without further fine-tuning, as ourmethod directly uses the pre-trained weights.
Since the fine-grained pruning is unstructured, it isrequires special hardwares to achieve measured speedup, so we do not report MACs for this baseline.† We use clean-fid for FID calculation. [Table] [Figure]
• 0.19 GauGAN .
We uniformly reduces each layer of GauGAN to 19% channels and train it from scratch.
• GAN Compression [3].
A general purpose method to compress the conditional GANs.
For PSNR and LPIPS, with G.T. means computing the the metric with the ground-truth images.
With Orig. means computing the the metric with the samples generated by the original model.
On LSUN Church, we only use 431 synthetic images for the PSNR/LPIPS with G.T. metrics, as manualediting does not have ground truths.
For the other metrics, we use the whole LSUN Church dataset(431 synthetic + 23 manual).
On Cityscapes, we view the synthetic semantic maps as the originalinput and the ground-truth semantic maps as the edited input for the PSNR/LPIPS with G.T. metrics,which has 1505 samples.
For the other metrics, we additionally include symmetric editing (view theground-truth semantic maps as the original inputs and synthetic semantic maps as the edited inputs),which has 3010 sample in total.
For the models with method Patch and Ours whose computation isediting dependant, we measure the average MACs over the whole dataset.
For DDIM and Progressive Distillation, our method outperforms all these baselines consistentlyand can achieve results on par with the original model.
The Patch inference fails when the editedregion is small as the global context is not enough.
Although our method only applies convolutionalfilters to the local edited regions, we could reuse the global context stored in the original activations.
Therefore, our method could perform the same as the original model.
For GauGAN, our methodalso performs better than GAN Compression [3] with even larger MACs reduction.
When applyingour method to GAN Compression, we further achieve on average 40 × MACs reduction with minorperformance degradation, beating 0.19 GauGAN by a large margin.
Model efficiency.
For real world interactive image editing applications, inference acceleration onhardware is more critical than the theoretical computation reduction. To verify the effectiveness of [Table] 2080Ti
Intel
[Table]
the model to 50% channels. Crop : For each convolution, we find a smallest bounding box covering the masked elements and crop it out, feed it into the convolution and scatter the output patch in the original image activation. Our method could reduce at most 18 × computation regarding MACs, 4.1 × , 2 . 9 × , 6.0 × and 14 × latency reduction on NVIDIA RTX 3090, 2080Ti, Intel Core i9-10920X and M1 Pro.
With GAN Compression, we could further speedup GauGAN by 9 . 5 × on Intel Core-i9 and 38 × on Apple M1 Pro.
our proposed engine, we measure the speedup of the editing examples shown in Figure 5 on fourdifference devices, including NVIDIA RTX 3090, NVIDIA RTX 2080Ti, Intel Core i9-10920X
CPU, and Apple M1 Pro, with different computational powers.
We use batch size 1 to simulate thereal-world use.
For GPU devices, we first perform 200 warm-up runs and measure the average latencyof the next 200 runs.
For CPU devices, we perform 10 warm-up runs and 10 test runs, repeat thisprocess 5 times and report the average latency.
The latency is measured in PyTorch 1.7 ‡ .
The resultsare shown in Table 2.
The original Progressive Distillation [12] can only generates 128 × 128 images,which is too small for real use.
We add some extra layers to adapt the model to resolution 256 × 256 [-. 0.
-]{+.0.
+}
Original means linearly scales each layer to 50% channels, and we only use this to benchmarkour latency results.
Crop means for each convolution F l , we directly find a smallest bounding boxcovering all the masked elements of the input activations A edited l , crop it out, feed it into the F l , andscatter the output patch in F l ( A original l ) .
When the editing pattern is like a rectangle, this algorithmcould saves more computation than ours ( e.g ., the first two examples of DDIM in Figure 5).
However,the speedup is still limited on 3090, 2080Ti and Intel Core i9-10920X due to the large memory indexoverheads in native PyTorch.
When the editing region is far from a rectangle ( e.g ., the third exampleof DDIM), this cropped patch has much redundancy.
Therefore, even though only 15.5% region isedited, the MACs reduction is only 1.7 × .
With 1 . 2% editing size, our method achieves 7.5 × , 4.6 ×and 18 × MACs reduction for DDIM, Progressive Distillation and GauGAN, respectively.
With SIGE,we achieve at most 4.1 × , 2.9 × , 6.0 × and 14 × speedup on RTX 3090, 2080Ti, Intel Core i9-10920Xand Apple M1 Pro, respectively.
When applied on GAN Compression, we achieve 9.5 × and 38 ×latency reduction on Inter Core i9-10920X and Apple M1 Pro, respectively. MACs
Optimizations
Latency
[Table]
(a)
[Table]
[Figure]
4.2 Ablation Study
Below we perform several ablation studies to show the effectiveness of each design choice.
Memory usage.
The pre-computed the activations of the original image requires additional memorystorage.
We profile the peak memory usage of the original model and our method in PyTorch.
Ourmethod only increases the peak memory usage of a single forward for DDIM, PD, GauGAN and GAN
Compression by 0.1G, 0.1G, 0.8G and 0.3G, respectively.
Specifically, our method needs to storeadditional 169M, 56M, 275M and 120M parameters for DDIM [1], PD [12], original GauGAN [2]and GAN Compression [3], respectively for a single forward.
For the diffusion models, we need tostore activations for all iteration steps ( e.g ., 50 for DDIM and 5 for PD).
However, data movementand kernel computation are asynchronized on GPU, so we could store the activations in CPU memoryand load the on-demand ones on GPU to reduce the peak memory usage.
Speedup of each design.
Table 3a shows the effectiveness of each kernel optimizations we addto SIGE for DDIM [1] on 2080Ti.
Naïvely applying the tiling-based sparse convolution couldreduce the computation by 9.1 × , but the latency reduction is only 1.6 × due to the large memoryoverheads in Gather and Scatter .
Pre-computing the normalization parameters could removethe latency of normalization statistics calculation and reduce the overall latency to 29.6ms.
Fusingelement-wise operations into the Gather and Scatter could remove some redundant operationsthat are applied to the unedited region and also reduce the memory allocation overheads (about 9ms).
Finally, fusing the Scatter and Gather to Scatter-Gather and Scatter in shortcut branch andmain branch could further reduce about 1.6ms overheads of tensor copying in Scatter , and achieve2.9 × speedup finally.
Comparisons with TensorRT.
Real-world DL model deployment always resorts to some optimizedlibraries and runtimes as their backends to generate the fastest executable.
To demonstrate theeffectiveness and extensibility of our SIGE, we also implement our kernels in a widely-used DLbackend TensorRT § and benchmark the DDIM latency results on RTX2080Ti in Table 3b.
Specifically,our speedup ratio is not diluted but more prominent with TensorRT comparing to PyTorch, especiallywhen the editing size is small.
This is because TensorRT has better support for the small convolutionkernel and could make better use of GPU capacity than PyTorch.
Conclusion
In an interactive image editing setting, existing deep generative models often waste computation byre-synthesize the image regions that do not require modifications.
To solve this issue, we presenta general-purpose method, Spatially Sparse Inference (SSI), that selectively performs computationfor edited regions.
Based on SSI, we propose Sparse Incremental Generative Engine (SIGE) toconvert the theoretical computation reduction to latency reduction on commonly-used hardware.
Ourapproach demonstrates strong performance on various settings.
Specifically, our proposed SIGEallows us to accelerate the inference time of DDIM by 3 . [-0 ⇥ on RTX 3090 and 6 . 6 ⇥ on Apple M
-]{+
+}
× on RTX3090 and 6 . 6 × on Apple M1 Pro,and GauGAN by 4 . 2 × on RTX3090 and 14 × on Apple M1 Pro on various image editing settings.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .•
Did you include the license to the code and datasets?
[No] The code and the data areproprietary.•
Did you include the license to the code and datasets?
[N/A] Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes](b) Did you describe the limitations of your work?
[Yes] See Appendix.(c)
Did you discuss any potential negative societal impacts of your work?
[Yes] SeeAppendix.(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A](b)
Did you include complete proofs of all theoretical results?
[N/A]3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes](b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[No]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes] See Appendix.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[Yes] See Appendix.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[Yes] See Appendix.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]