Pluralistic Image Completion with Gaussian Mixture Models
Abstract
Pluralistic image completion focuses on generating both visually realistic and diverse results for image completion.
Prior methods enjoy the empirical successes of this task.
However, their used constraints for pluralistic image completion are argued to be not well interpretable and unsatisfactory from two aspects.
First, the constraints for visual reality can be weakly correlated to the objective of image completion or even redundant .
Second, the constraints for diversity are designed to be task-agnostic , which causes the constraints to not work well.
In this paper, to address the issues, we propose an end-to-end probabilistic method.
Speciﬁcally, we introduce a uniﬁed probabilistic graph model that represents the complex interactions in image completion.
The entire procedure of image completion is then mathematically divided into several sub-procedures, which helps efﬁcient enforcement of constraints.
The sub-procedure directly related to pluralistic results is identiﬁed, where the interaction is established by a Gaussian mixture model (GMM).
The inherent parameters of GMM are task-related , which are optimized adaptively during training, while the number of its primitives can control the diversity of results conveniently.
We formally establish the effectiveness of our method and demonstrate it with comprehensive experiments.
The implementation is available at https://github.com/tmllab/PICMM.
Introduction
Pluralistic image completion refers to the task of ﬁlling in the missing region of an incomplete image, so as to produce visually realistic and diverse image completion solutions [59–61].
Different from single image completion [39, 12, 55, 50] that learns a deterministic mapping from an incomplete image to a complete image, and produces a unique result, pluralistic image completion can generate various results with visually realistic contents.
Pluralistic image completion follows the fact that image completion is a highly subjective process and beneﬁts a series of applications such as photo restoration [27], object removal [13], and transmission error concealment [37].
Pluralistic image completion is complicated and challenging.
The state-of-the-art methods encode images to latent features.
To constrain the visual reality of results, heuristic constraints are introduced, e.g. , the perceptual loss [27] and attention module [59, 44].
To constrain the diversity of results, the distribution of latent features is assumed to be a unimodal Gaussian distribution with predeﬁned parameters [59].
Image completion is performed by sampling from the unimodal Gaussian distribution.
Although the above paradigm enjoys empirical successes of pluralistic image completion, it is not well interpretable and unsatisfactory in practice.
We detail the issues from two aspects.
First, the added constraints for the visual reality of results are based on general purposes .
These constraints may work well in some tasks.
Unfortunately, there is no clear understanding of what role they play in pluralistic image completion.
Inappropriate constraints are likely to have side effects.
It is not easy to ﬁnish the determination of added constraints as our desideratum, especially in complex image completion tasks.
Second, the diversity of image completion results is hard to be constrained in reason.
For a speciﬁc task, the parameters of the unimodal Gaussian distribution are directly related to the diversity.
Nevertheless, these parameters are designed to be prior knowledge and task-agnostic , which cause unreasonable pluralistic results .
[Figure]
In this paper, to address the above issues, we present an end-to-end probabilistic method.
The method is constructed with a uniﬁed probabilistic graph model (PGM) [17, 22] and translates the problem of pluralistic image completion into a structured mathematical representation.
Specifically, we suppose that pluralistic image completion can be represented with a directed acyclic graph that is designed reasonably.
The nodes of the graph denote different kinds of images and corresponding latent features.
The edges of the graph represent the interactions between nodes.
Based on this directed acyclic graph, the entire procedure of pluralistic image completion is divided into several sub-procedures.
Then, we perform different constraints for different subprocedures as our desideratum.
In particular, the diversity of image completion results is implemented by modeling the interaction between the latent features of the missing region and incomplete image with a Gaussian mixture model (GMM) [40, 64].
Compared with the unimodal Gaussian distribution, GMM has a larger capacity and is more competitive to meet the output diversity [56, 38].
For our method, the inherent parameters of GMM are task-related and optimized adaptively during training.
The different primitives of GMM represent the outputs with different patterns.
Additionally, the number of primitives can be chosen artiﬁcially with the needs for diversity in real-world applications.
Before delving into details, we clearly emphasize our contributions as follows:
• To our best knowledge, this paper is the ﬁrst one that uses a PGM for pluralistic image completion.
Besides, the PGM is specially designed with justiﬁcations, rather than borrowing existing models.
With the designed PGM, the added constraints for visual reality and diversity are explainable.
• We propose to use GMM to increase result diversity.
One remarkable advantage is that the inherent parameters for diversity are task-related, leading to more reasonable results.
• We conduct a series of experiments on benchmark datasets to support our claims.
In both qualitative and quantitative comparisons with the state-of-the-art methods, our method achieves superior performance.
The generated contents for image completion are both visually realistic and diverse, such as those shown in Figure 1.
The rest of the paper is organized as follows.
In Section 2, we brieﬂy review pluralistic image completion.
In Section 3, we discuss the proposed method step by step.
Experimental results are provided in Section 4.
Finally, we conclude the paper in Section 5.
Problem Setting.
We ﬁrst discuss image completion.
For the task of image completion, suppose we have an image, originally
I o , but degraded by some missing pixels to become a masked partial image I m .
The image that comprises the original missing pixels is called a complement partial image, which can be denoted by I c .
The goal of the image completion task is to reconstruct
I o by using I m .
The illustrations for different kinds of images are provided in Figure 2.
Traditional single image completion reconstructs the original image in a deterministic fashion, which results in only a single solution [12, 9].
Pluralistic image completion [59] is motivated by the fact that image completion is a highly subjective process.
Also, completed images by different experts may agree on high-level semantics, but have substantially different details [59].
Therefore, in pluralistic image completion, we need to generate multiple and diverse plausible results when presented with a masked partial image.
The problem of pluralistic image completion is challenging since we need to take care of the diversity and visual authenticity of results at the same time.
Prior Work.
Traditional methods on single image completion, e.g., diffusion-based methods [3, 1, 23] and patch-based methods [2, 4, 5], assume that image holes share similar content to visible regions.
Therefore, they choose to directly propagate the contextual appearances or realign the background patches to complete the image holes.
Beneﬁting from the strong power of deep neural networks, deep learning based single image completion methods achieve promising completion performance, e.g., employing shepard convolutional neural networks [39], restraining global and local consistency between image holes and visible regions [16], performing deep feature rearrangement [47], and learning a pyramid-context encoder network [53], etc .
In addition to these methods, multiple single image completion methods involve the generative adversarial networks [10] to learn the semantic of images [48, 49], e.g., [25, 51, 8, 6, 57, 12, 9].
Although these single-solution methods achieve outstanding performance in predicting deterministic result for image holes, they cannot generate various semantically meaningful results.
Existing works for pluralistic image completion usually rely on generative models [24], e.g. , CVAE [43, 13, 59], hierarchical VQ-VAE [37], PD-GAN [27], and BicycleGAN [63].
Generative pluralistic image completion methods can diversify meaningful completion results with different kinds of constraints.
However, such methods are not well interpretable as discussed.
The issues largely limit their applications in the real world.
Methodology
This section presents a probabilistic method for pluralistic image completion.
We ﬁrst design a probabilistic graph model for our task (Section 3.1).
Then, based on the probabilistic graph model, we show how to divide the entire procedure of pluralistic image completion into several sub-procedures (Section 3.2).
Afterward, we discuss how to use GMM to diversify image completion results (Section 3.3).
Finally, we add reconstruction and adversarial losses to strengthen image completion, and summarize all algorithm ﬂows (Section 3.4).
3.1 Probabilistic Graph Model Construction
Recall the notations in image completion, i.e. , I o , I m , and I c , they correspond to the latent features z o , z m , and z c respectively.
We suppose a probabilistic graph model for the image completion task, which is shown in Figure 3.
The probabilistic graph model is designed reasonably.
The graphical illustration of overall computational paths of the probabilistic graph model is presented in Figure 3(d).
Speciﬁcally, the images I m and I c form I o .
Their latent features z m
and z c can be exploited to form
I o .
Given z m , we can surmise z c .
We use the latent features here, since they are low-dimensional and informative [11, 21, 45, 46].
We further details Figure 3 as follows.
[Figure]
Inference.
We do not directly map I m
and I c to I o , since they are too high-dimensional and have information redundancy.
We exploit generative models.
The processes I m !
z m
and I c !
z c denote that we encode images into corresponding latent features.
During training, we learn the encoder f to ﬁnish the inference.
Dynamic Inference.
Given I m ( resp.
I c ), we can infer the latent feature z m ( resp.
z c ).
Afterward, z m 99K z c means that we use a variational distribution p ( z c | z m ) to approximate the distribution p ( z c | I c ) .
Here we denote this relationship using a dashed line, which is designed to be dynamic to diversify outputs.
Generation.
In image completion, the images I m
and I c can compose the image
I o .
Accordingly, for their latent features, we splice z m and z c to output I o .
During training, we learn the decoder g to ﬁnish the generation.
3.2 Image Completion Objective Decomposition
We cannot observe the underlying I c in the test procedure.
Based on the probabilistic graph model, we perform the following variational inference with the Kullback-Leibler (KL) divergence:
[Equation]
(1) where q ( ·|· ) and p  ( ·|· ) are two conditioned distributions, with and  being the parameters of their corresponding functions.
The divergence (1) is minimized with respect to all parameters.
It should be noted that, in fact, the overall objective should be KL [ q ( I o | I m )
k p  ( I o | I m , I c )] =
KL [Equation] z [Equation]
z
[Equation]
|
[Equation] [Equation] [Equation] z
[Equation]
z
[Equation]
| [Equation] m , I c )]] . As the expectation in this formula is intractable , we need to deal with the 1-step Monte Carlo sampling objective, which is why the divergence (1) is formed.
Another advantage of using the divergence (1) is that constraining each sampling point can make the overall constraint tighter.
We further decompose the divergence (1) based on the probabilistic graph model as follows.
Proposition 1 Regarding the KL divergence (1), we show that the divergence can be decomposed as:
[Equation]
where q ✓ is the variational distribution, and the parameters ✓ are involved in divergence minimization.
The proof can be found in Appendix A.1.
We analyze three terms in Eq.
(2) combining the probabilistic graph model and our purpose:
• For a  , as discussed, we can generate I o with I m
and I c , or with their latent features z m and z c .
We minimize the distance about the generation of I o in the two ways.
The minimization can ensure that the generation from the images and latent features is consistent .
• For b  , we do not limit the inference from I c to z c as a deterministic function.
Instead, to diversify outputs, we use the variational distribution to restrain q ✓ ( z c | z m ) .
The choice and discussion about the variational distribution will be provided in Section 3.3.
• For c  , it aims to keep that the inference using the variational distribution and the inference using the posterior is close , which also guarantees the reliability of sampling.
Besides, the variational distribution does not include I c to enable latent features to be used in test.
We employ the variational evidence lower bound (ELBO)
L ELBO [21, 15] to constrain c  .
It is because, suppose that p ( I m ) is a constant given I m , it is equivalent for maximizing the L ELBO and minimizing c  .
The details are provided in Appendix A.4.
3.3 Gaussian Mixture Model
As discussed, we need a suitable choice for the variational distribution q ✓ ( z c | z m ) to output pluralistic results.
In this work, we employ the Gaussian Mixture Model (GMM) [30, 31] for the dynamic mode q ✓ ( z c | z m ) , which is more ﬂexible than a unimodal and can diversify outputs.
We denote the latent features of I c that are inferred from z m by ˆ z c , which is distinguished from z c achieved by the encoder.
For the latent features z , we denote the mean and covariance by µ z and ⌃ z .
Mathematically, we have
[Equation]
where k denotes the number of primitives, ↵ i denotes the mixing coefﬁcient of the i -th primitive, and µ i , ⌃ i denotes the distribution parameters of the i -th primitive.
Note that the mixing coefﬁcient ↵ i is the parameter of the Categorical distribution [31].
Intuitively, in Eq.
(3), we model z m with GMM, where the i -th primitive can be used to represent ˆ z ( i ) c .
In this way, we can obtain different ˆ z c to meet diversity requirements.
Besides, among all the parameters of GMM, except for k , the other parameters are called the inherent parameters of GMM.
In GMM (3), the number of primitives k can be determined artiﬁcially according to the need for output diversity.
The inherent parameters are optimized adaptively during training.
Speciﬁcally, for the optimization of the mixing coefﬁcients ↵ =
[ ↵ 1 , . .
. , ↵ k ] , we exploit the the frequency loss [38].
The objective is
[Equation]
For our task, with the frequency loss (4), the accumulated frequency approximate gradient is an asymptotically unbiased estimation of the true gradient.
We provide the detailed derivation and analysis in Appendix A.2.
The reason we mention the frequency loss is that, the metric for evaluating the performance of each primitive is adapted to KL divergence in our task, which is different from the original metric in [38].
For the optimization of µ and ⌃ of z m , we utilize the back-propogatemax-operation to reserve the distinguishable property of GMM (3).
For our problem, the objective of the back-propogate-max-operation is shown in the following proposition.
Proposition 2 By modeling the variational distribution with GMM (3), we have
[Equation]
[Equation] where j = argmin i KL [Equation]
i , and tr ( · ) denotes the trace of a matrix.
The derivation of Proposition 2 can be found in Appendix A.3.
For Proposition 2, if we minimize
L BM , we optimize µ and ⌃ of z m , which reduces the distance between ˆ z ( j ) c and z c .
For the term b  in Eq.
(2), we then have L GMM =
[Equation]
Discussion.
Note that the diversity of results is directly related to z m .
When GMM for z m is dominated by the i -th primitive ( ↵ i ⇡ 1 ), the diversity of image completion results is reduced.
Although this phenomenon did not appear in our experiments, it may exist in practice.
Actually, this phenomenon is reasonable.
From human cognition, if I m shows only one pattern , the inference for
I c will be deterministic [3].
For example, if the size of the missing region of an incomplete image is very small, we can infer the missing region both deterministically and reasonably, where the diversity of results is reduced.
Prior methods with task-agnostic constraints do not consider this fact, which may create unreasonable results .
For our method, the constraints for diversity are task-related, which can take this phenomenon into account.
I ( j ) o by g ⇤ with ˆ I ( j ) o =
g ⇤ ( z m , ˆ z ( j ) c ) .
7: end for
Output: pluralistic image completion results { ˆ I ( j ) o } kj =1 .
3.4 Reconstruction and Adversarial Losses
To ensure the minimization of the term a  in Eq.
(2), we incorporate the use of reconstruction and adversarial losses [10].
The reconstruction loss is formulated as:
[Equation]
where ˆ I o is generated from z m and z c , and ˆ I ( j ) m is generated from ˆ z ( j ) c .
Here, ˆ z ( j ) c can be inferred from z m by using Eq.
(3) and Eq.
The image ˆ I ( j ) m is obtained from ˆ I ( j ) o with the image mask that can degenerate I o to I m .
The image ˆ I ( j ) o is generated from z m and ˆ z ( j ) c .
The reconstruction loss (6) controls the visual rationality of image completion results.
Also, we efﬁciently avoid the deterministic fashion with GMM.
Furthermore, we leverage adversarial training [10] to make the generated images more realistic.
The adversarial loss L A is formulated as
[Equation]
where D is the discriminator optimized by the discriminator loss based on LSGAN [29].
We jointly train our encoder f and decoder g through the following combined loss:
[Equation]
where the weight  A is set to 0.05 in all experiments.
Then, the ﬁnal objective of the loss function is the sum of three losses, i.e. , L FINAL =
L GMM + [Equation] ELBO [Equation]
The ﬁnal loss is to maximize the log-likelihood of the conditional data distribution.
Speciﬁcally, the adversarial loss uses the
Wasserstein distance to make generated images realistic given latent features.
The ELBO loss uses KL divergence and distribution log-likelihood to make variational posterior approximate real posterior.
Algorithm Flows.
For the convenience of following technical details, we provide the algorithm ﬂows.
The algorithm ﬂows of training and test stages can be found in Algorithm 1 and Algorithm 2.
The illustration is shown in Appendix B.
It should be noted that the number of primitives k does not mean that we are limited to generate only k diverse results given I m (in Steps 3 and 4 of Algorithm 2).
In fact, we can sample from the k primitives to obtain lots of image completion results.
Moreover, as GMM has a much larger capacity than the unimodal Gaussian distribution [31], we can achieve greater diversity.
We provide empirical observations in Section 4.3 and Appendix C.
Experiments
In this section, we conduct a series of experiments to justify our claims.
We ﬁrst introduce the implementation of our method (Section 4.1).
The comprehensive experimental results and comparison with advanced methods are then provided and discussed (Section 4.2).
Finally, we conduct an analysis study to present and discuss our method in more detail (Section 4.3).
4.1 Implementation Details
Datasets.
We evaluated our proposed model on ﬁve popularly used datasets, i.e. , CelebA-HQ [18, 28],
FFHQ [19], Paris StreetView [7], Places2 [62], and ImageNet [41].
We verify the effectivness of our method with different types of mask regions, including both center and random masks [59, 44].
Network and Optimization.
The proposed method can be implemented efﬁciently.
The encoder and decoder of our pipeline are inspired by PIC [59] for comparison.
We apply average pooling and interpolation with convolutional layers to implement downsampling and upsampling respectively.
During optimization, we use the Adam optimizer [20].
The learning rate is ﬁxed to 10  4 during the training procedure.
Baselines.
We compare the proposed method with the following state-of-the-art methods, which include (1) single image completion methods: DFv2 [52], EC [33], and MED [26]; (2) pluralistic image completion methods: PIC [59] and ICT [44].
We abbreviate our method ( P luralistic I mage
C ompletion with Gaussian M ixture M odels) as PICMM.
The methods are implemented by PyTorch and evaluated on NVIDIA Tesla A100 GPUs.
[Figure] [Figure]
[Figure] [Figure]
Measurement.
For measurement, we provide both qualitative and quantitative results.
For the metrics of quantitative results, we use common evaluation metrics such as the peak signal-to-noise ratio (PSNR), structural similarity (SSIM), mean absolute error (MAE), and fr ´ echet inception distance (FID) [14] to measure the similarity between the image completion result and ground truth.
Furthermore, pluralistic image completion is supposed to focus on generating diverse realistic results rather than merely approximating ground-truth ones [58].
To measure the result diversity , we add two perceptual quality metrics to quantitative results, which include LPIPS [59] and DIV-FID [44].
4.2 Comparison with Prior Methods 4.2.1 Qualitative Comparisons
The face images come from CelebA-
We provide extensive qualitative comparison results to justify our claims.
First, we show the results on CelebA-HQ and FFHQ in Figure 4, which are images related to human faces.
As can be seen, the image completion results achieved by our method are realistic, following high image quality.
Also, compared with the baseline
PIC, our completion results are more diverse, e.g. , see the images in the second and last rows.
Second, we provide the results on Paris
StreetView and Places in Figures 5 and 6 respectively.
For the results on Paris StreetView, we can see that image completion by the baselines MED and PIC are somewhat distorted.
By contrast, our method can ﬁnish the image completion task better.
The details can be checked in the complements to walls.
Then, we turn the attention to the image completion results on Places2.
Our method still achieves superior performance compared with baselines.
Lastly, we provide the results on ImageNet in Figure 7.
Note that the sources of the data in ImageNet are very complicated.
The image completion task is rather challenging on this dataset, even for single image completion [59].
Different from some prior work [36, 16] that were trained on a 100k subset of training images of ImageNet, we directly the network on the original ImageNet training dataset with all images.
The results mean that our method can infer the content effectively.
Reﬁned Result Analysis.
We provide some reﬁned results in Figure 8 to analyze the image completion details of our method.
For the face images, we can ﬁnd that the image completion results have very different facial expressions.
[Table]
4.2.2 Quantitative Comparisons
[Table]
Common Metrics.
Quantitative evaluation with common metrics is difﬁcult for pluralistic image completion, as our goal is to obtain diverse but reasonable solutions for a masked image [59].
To make a feasible comparison, as did in [59], we sample one completed image many times from k images to compare with the original image.
The results are provided in
Tables 1 and 2.
As can be seen, for
FFHQ and Places2, on both the settings of center and random masking, our method achieves superior performance.
On ImageNet, our method performs the best with the center masking consistently, while it is competitive with the random masking.
It should be noted that ICT is a very strong baseline and works better than our method in some cases.
It is because ICT is built with the image transformer [35], which is powerful in vision tasks.
Compared with ICT, the advantages of our method lies in not only better interpretability as discussed, but also faster inference speeds.
We will demonstrate this in inference time
[Table]
Diversity Metrics.
For pluralistic image completion, the diversity evaluations are signiﬁcant.
We generate ﬁve completion images in each task and report the mean of the diversity metric.
The results are shown in Table 3, which represents that our method can generate more diverse completion images than baselines.
The results with common and diversity metrics support our claim very well.
That is to say, the image completion results of our method are both high-quality and pluralistic.
[Table]
Inference Time.
We report the inference time of our method, compared with PIC and ICT.
All inference runs on one NVIDIA Tesla A100 GPU for fairness.
The size of images is 256 ⇥ 256 .
We perform pluralistic image completion fordifferent images.
For each image, we generatecompletion results for it.
We report the total inference time for these images.
The results are shown in Table 4.
Note that as ICT is a transformer-based method, the calculated consumption of its inference is much heavier.
What’s worse, ICT needs iterative Gibbs sampling during inference.
The two issues make ICT have to face heavy computational consumption, which is also mentioned in [44].
In contrast, our method is more inference-efﬁciency, which is more than 750 times faster than ICT .
The merit could make our method easier to use in the real world.
4.3 More Analyses and Justiﬁcations
GMM Primitive Visualization.
We further stress the diversity of the completion results achieved by our method.
Speciﬁcally, we visualize the ˆ z c which is inferred from z m as discussed.
The visualization results are obtained with t-SNE [42].
Different ˆ z c are presented by the 2D-vectors in different colors, which are shown in Figure 9.
As can be seen, the vectors are scattered , which clearly demonstrates that the completion results of our method are diverse.
Reﬁned Diversity Comparison.
We argue that, beneﬁting from that GMM has larger capacities than a unimodal Gaussian distribution, the completion results of our method would be more diverse.
To justify our claims, we provide the reﬁned diversity comparison of our method with PIC and ICT, as shown in Figure 10.
Note that the six image completion results of PIC are obtained by sampling from one unimodal Gaussian distribution.
The results of our method are obtained by sampling from six primitives.
Clearly, our results are more diverse than PIC’s.
Besides, for ICT, it does not explicitly control the output diversity.
The diversity of pluralistic image completion is thus less interpretable than our method.
In addition, we state that our method is not limited to only generating k completion results before.
In fact, we can sample from k primitives of GMM to obtain lots of completion results.
Due to the limited page, we present the results and more analyses in Appendix C. Moreover, the richer completion result comparison is presented in Appendix D.
Conclusion
In this paper, we focus on the complicated and challenging problem of pluralistic image completion.
We propose a novel end-to-end probabilistic method for this challenging problem.
Based on the probabilistic graph model, our method divides the entire procedure of pluralistic image completion into several sub-procedures, where GMM is used to diversify outputs.
Experiments on a variety of datasets show that the image completion results by our method are both high-quality and pluralistic.
In future work, investigating the feasibility of the proposed method for other diverse decision-making scenarios might prove important.
Acknowledgements
Yibing Zhan was partially supported by the Major Science and Technology Innovation 2030 “
New
Generation Artiﬁcial Intelligence” key project (No. 2021ZD0111700) and the National Natural
Science Foundation of China (Grant No. 62002090).
Xiaobo Xia was supported by Australian
Research Council Projects DE-190101473 and Google PhD Fellowship.
Bo Han was supported by the RGC Early Career Scheme No. 22200720, NSFC Young Scientists Fund No. 62006202, and
Guangdong Basic and Applied Basic Research Foundation No. 2022A1515011652.
Tongliang Liu was partially supported by Australian Research Council Projects DP180103424, DE-190101473, IC190100031, DP- and FT-220100318.
The authors would give special thanks to Mingrui
Zhu (Xidian University), Zihan Ding (Princeton University), and Chenlai Qian (Southeast University) for helpful discussions
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] , (b) Did you describe the limitations of your work?
[Yes] , (c) Did you discuss any potential negative societal impacts of your work?
[No] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] , 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
[N/A] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]