GAUDI: A Neural Architect for Immersive 3D Scene Generation
Abstract
We introduce GAUDI , a generative model capable of capturing the distribution of complex and realistic 3D scenes that can be rendered immersively from a moving camera.
We tackle this challenging problem with a scalable yet powerful approach, where we ﬁrst optimize a latent representation that disentangles radiance ﬁelds and camera poses.
This latent representation is then used to learn a generative model that enables both unconditional and conditional generation of 3D scenes.
Our model generalizes previous works that focus on single objects by removing the assumption that the camera pose distribution can be shared across samples.
We show that GAUDI obtains state-of-the-art performance in the unconditional generative setting across multiple datasets and allows for conditional generation of 3D scenes given conditioning variables like sparse image observations or text that describes the scene.
Introduction
In order for learning systems to be able to understand and create 3D spaces, progress in generative models for 3D is sorely needed.
The quote "The creation continues incessantly through the media of man." is often attributed to Antoni Gaudí who we pay homage to with our method’s name.
In this paper we ask the question: can creation continue through the media of learning machines?
We are interested in generative models that can capture the distribution of 3D scenes and then render views from scenes sampled from the learned distribution.
Extensions of such generative models to conditional inference problems could have tremendous impact in a wide range of tasks in machine learning and computer vision.
For example, one could sample plausible scene completions that are consistent with an image observation, or a text description (see Fig.
1 for 3D scenes sampled from
GAUDI).
In addition, such models would be of great practical use in model-based reinforcement learning and planning [14], SLAM [43] or 3D content creation.
Recent works on generative modeling for 3D objects or scenes [60, 6, 8] employ a Generative
Adversarial Network (GAN) where the generator explicitly encodes radiance ﬁelds — a parametric function that takes as input the coordinates of a point in 3D space and camera pose, and outputs a density scalar and RGB value for that 3D point.
Images can be rendered from the radiance ﬁeld generated by the model by passing the queried 3D points through the volume rendering equation [Figure] to project onto any 2D camera view.
While compelling on small or simple 3D datasets ( e . g .
single objects or a small number of indoor scenes), GANs suffer from training pathologies including mode collapse [58, 65] and are difﬁcult to train on data for which a canonical coordinate system does not exist, as is the case for 3D scenes [61].
In addition, one key difference between modeling distributions of 3D objects vs. scenes is that when modeling objects, the distribution of valid camera poses does not depend on each object and is deﬁned per dataset ( i .
e .
typically as SO (3) ), which is not true for scenes.
See Fig. 3(b) and note how the two scene layouts deﬁne different areas of navigable space (different dark grey shaded areas) where valid camera poses can be placed, revealing a strong dependency between scenes and their camera pose distributions.
GAUDI captures this dependency by modeling the joint distribution of scenes and camera poses.
In GAUDI, we map each trajectory ( i .
e .
a sequence of posed images from a 3D scene) into a latent representation that encodes a radiance ﬁeld ( e .
g .
the 3D scene) and camera path in a completely disentangled way.
We ﬁnd these latent representations by interpreting them as free parameters and formulating an optimization problem where the latent representation for each trajectory is optimized via a reconstruction objective.
This simple training process is scalable to thousands of trajectories.
Interpreting the latent representation of each trajectory as a free parameter also makes it simple to handle a large and variable number of views for each trajectory rather than requiring a sophisticated encoder architecture to pool across a large number of views.
After optimizing latent representations for an observed empirical distribution of trajectories, we learn a generative model over the set of latent representations.
In the unconditional case, the model can sample radiance ﬁelds entirely from the prior distribution learned by the model, allowing it to synthesize scenes by interpolating within the latent space.
In the conditional case, conditional variables available to the model at training time ( e . g . images, text prompts, etc.) can be used to generate radiance ﬁelds consistent with those variables.
Our contributions can be summarized as:
• We scale 3D scene generation to thousands of indoor scenes containing hundreds of thousands of images, without suffering from mode collapse or canonical orientation issues during training.
• We introduce a novel denoising optimization objective to ﬁnd latent representations that jointly model a radiance ﬁeld and the camera poses in a disentangled manner. • Our approach obtains state-of-the-art generation performance across multiple datasets.
• Our approach allows for various generative setups: unconditional generation as well as conditional on images or text.
Related Work
In recent years the ﬁeld has witnessed outstanding progress in generative modeling for the 2D image domain, with most approaches focusing either on adversarial [21, 22] or auto-regressive models [69, 46, 11].
More recently, score matching based approaches [18, 62] have gained popularity.
In particular, Denoising Diffusion Probabilistic Models (DDPMs) [17, 37, 52, 68] have emerged as strong contenders to both adversarial and auto-regressive approaches.
In DDPMs, the goal is to learn a step-by-step inversion of a ﬁxed diffusion Markov Chain that gradually transforms an empirical data distribution to a ﬁxed posterior, which typically takes the form of an isotropic Gaussian distribution.
In parallel, the last couple of years have seen a revolution in how 3D data is represented within neural networks.
By representing a 3D scene as a radiance ﬁeld, NeRF [33] introduces an approach to optimize the weights of a MLP to represent the radiance of 3D points that fall inside the ﬁeld-of-view of a given set of posed RGB images.
Given the radiance for a set of 3D points that lie on a ray shot from a given camera pose, NeRF [33] uses volumetric rendering to compute the color for the corresponding pixel and optimizes the MLP weights via a reconstruction loss in image space.
A few attempts have also been made at incorporating a radiance ﬁeld representation within generative models.
Most approaches have focused on the problem of single objects with known canonical orientations like faces or Shapenet objects with shared camera pose distributions across samples in a dataset [60, 6, 38, 25, 5, 12, 75, 47, 10].
Extending these approaches from single objects to completely unconstrained 3D scenes is an unsolved problem.
One paper worth mentioning in this space is GSN [8], which breaks the radiance ﬁeld into a grid of local radiance ﬁelds that collectively represent a scene.
While this decomposition of radiance ﬁelds endows the model with high representational capacity, GSN still suffers from the standard training pathologies of GANs, like mode collapse [65], which are exacerbated by the fact that unconstrained 3D scenes do not have a canonical orientation.
As we show in our experiments (cf.
Sect. 4), these issues become prominent as the training set size increases, impacting the capacity of the generative model to capture complex distributions.
Separately, a line of recent approaches have also studied the problem of learning generative models of scenes without employing radiance ﬁelds [40, 70, 51].
These works assume that the model has access to room layouts and a database of object CAD models during training, simplifying the problem of scene generation to a selection of objects from the database and pose predictions for each object.
Finally, approaches that learn to predict a target view given a single (or multiple) source view and relative pose transformation have been recently proposed [27, 74, 57, 9, 13].
The pure reconstruction objective employed by these approaches forces them to learn a deterministic conditional function that maps a source image and a relative camera transformation to a target image.
The ﬁrst is that this scene completion problem is ill-posed ( e .
g .
given a single source view of a scene there are multiple target completions that are equally likely).
Attempts at modeling the problem in a probabilistic manner have been proposed [53, 49].
However, these approaches suffer from inconsistency in predicted scenes because they do not explicitly model a 3D consistent representation like a radiance ﬁeld.
GAUDI
Our goal is to learn a generative model given an empirical distribution of trajectories over 3D scenes.
Let X
=
{ x i 2 { 0 ,...,n } } denote a collection of examples deﬁning an empirical distribution, where each example x i is a trajectory.
Every trajectory x i is deﬁned as a variable length sequence of corresponding RGB, depth images and 6 DOF camera poses (see Fig. 3).
We decompose the task of learning a generative model in two stages.
First, we obtain a latent representation z
=
[ z scene , z pose ] for each example x 2 X that represents the scene radiance ﬁeld and pose in separate disentangled vectors.
Second, given a set of latents Z = { z i 2 { 0 ,...,n } } we learn the distribution p ( Z ) .
3.1 Optimizing latent representations for radiance ﬁelds and camera poses
We now turn to the task of ﬁnding a latent representation z 2 Z for each example x 2 X ( i .
e .
for each trajectory in the empirical distribution).
To obtain this latent representation we take an encoder-less view and interpret z ’s as free parameters to be found via an optimization problem [2, 39].
To map latents z to trajectories x , we design a network architecture ( i .
e .
a decoder) that disentangles camera poses and radiance ﬁeld parameterization.
Our decoder architecture is composed of 3 networks (shown in Fig.
• The camera pose decoder network c (parameterized by ✓ c ), is responsible for predicting camera poses ˆ T s 2 SE (3) at the normalized temporal position s 2 [  1 , 1] in the trajectory, conditioned on z pose which represents the camera poses for the whole trajectory.
To ensure that the output of c is a valid camera pose ( e . g .
an element of SE (3) )
, we output a 3 D vector representing a normalized quaternion q s for the orientation and a 3 D translation vector t s . [Table] [Figure]
to • The scene decoder network d (parameterized by ✓ d ), is responsible for predicting a conditioning variable for the radiance ﬁeld network f .
This network takes as input a latent code that represents the scene z scene and predicts an axis-aligned tri-plane representation [41, 5] W 2 R 3 ⇥ S ⇥ S ⇥ F .
Which correspond to 3 feature maps [ W xy , W xz , W yz ] of spatial dimension S ⇥ S and F channels, one for each axis aligned plane: xy , xz and yz .
• The radiance ﬁeld decoder network f (parameterized by ✓ f ), is tasked with reconstructing image level targets using the volumetric rendering equation in Eq.
The input to f is p 2 R 3 and the tri-plane representation W =
[ W xy , W xz , W yz ] (we do not condition on the camera orientation to improve consistency [12]).
Given a 3 D point p
=
[ i, j, k ] for which radiance is to be predicted, we orthogonally project p into each plane in W and perform bi-linear sampling.
We concatenate thebi-linearly sampled vectors into w xyz =
[ W xy ( i, j ) , W xz ( j, k ) , W yz ( i, k )] 2 R 3 F , which is used to condition the radiance ﬁeld function f .
We implement f as a MLP that outputs a density value  and a signal a .
To predict the value v of a pixel, the volumetric rendering equation is used (cf.
Eq. 1) where a 3D point is expressed as ray direction r (corresponding with the pixel location) at particular depth u .
[Equation]
We formulate a reconstruction objective to jointly optimize for ✓ d , ✓ c , ✓ f and { z } i = { 0 ,...,n } , shown in Eq. 2.
Note that while latents z are optimized for each example x independently, the parameters of the networks ✓ d , ✓ c , ✓ f are amortized across all examples x 2 X .
As opposed to previous auto-decoding approaches [2, 39], each latent z is perturbed during training with additive noise that is proportional to the empirical standard deviation across all latents, z = z +  N (0 , std( Z )) , inducing a contractive representation [50].
In this setting,  controls the trade-off between the entropy of the distribution of latents z 2 Z and the reconstruction quality.
With  =
the distribution of z ’s becomes a set of indicator functions ( i .
e . similar as one would get from a vanilla auto-encoder).
For large  > 0 the structure in distribution of z ’s is destroyed, as latents are perturbed with large magnitudes of noise.
We use a small  > 0 value to enforce a latent space in which interpolated samples (or samples that contain small deviations from the empirical distribution, as the ones that one might get from sampling a subsequent generative model) are included in the support of the decoder function, sacriﬁcing a small cost in reconstruction ﬁdelity.
[Equation]
We optimize parameters ✓ d , ✓ f , ✓ c and latents z 2 Z with two different losses.
The ﬁrst loss function
L scene measures the reconstruction between the radiance ﬁeld encoded in z scene and the images in the trajectory x im s (where s denotes the normalized temporal position of the frame in the trajectory), given ground-truth camera poses T s required for rendering.
We use an l 2 loss for RGB and l 1 for depth 1 .
The second loss function L pose measures the camera pose reconstruction error between the poses ˆ T s encoded in z pose and the ground-truth poses.
We employ an l 2 loss on translation and l 1 loss for the normalized quaternion part of the camera pose.
Although theoretically normalized quaternions are not necessarily unique ( e .
g .
q and  q ) we do not observe any issues empirically during training.
3.2 Prior Learning
Given a set of latents z 2 Z resulting from minimizing the objective in Eq.
2, our goal is to learn a generative model p ( Z ) that captures their distribution ( i .
e .
after minimizing the objective in Eq.we interpret z 2 Z as examples from an empirical distribution in latent space).
In order to model p ( Z ) we employ a Denoising Diffusion Probabilistic Model (DDPM)
[17], a recent score-matching [18] based model that learns to reverse a diffusion Markov Chain with a large but ﬁnite number of timesteps.
In DDPMs [17] it is shown that this reverse process is equivalent to learning a sequence of denoising auto-encoders with tied weights.
The supervised denoising objective in DDPMs makes learning p ( Z ) simple and scalable.
This allows us to learn a powerful generative model that enables both unconditional and conditional generation of 3D scenes.
For training our prior p ✓ p ( Z ) we take the objective function in [17] deﬁned in Eq. 3.
In Eq.
3 t denotes the timestep, ✏ ⇠ N (0 , I ) is the noise and ¯ ↵ t is a noise magnitude parameter with a ﬁxed scheduling.
Finally, ✏ ✓ p denotes the denoising model.
[Equation]
At inference time, we sample z ⇠ p ✓ p ( Z ) by following the inference process in DDPMs.
We start by sampling z T ⇠ N (0 , I ) and iteratively apply ✏ ✓ p to gradually denoise z T , thus reversing the diffusion
Markov Chain to obtain z 0 .
We then feed z 0 as input to the decoder architecture (cf.
Fig. 2) and reconstruct a radiance ﬁeld and a camera path.
If the goal is to learn a conditional distribution of the latents p ( Z | Y ) , given paired data { z 2 Z, y 2 Y } , the denoising model ✏ ✓ is augmented with a conditioning variable y , resulting in ✏ ✓ p ( z , t, y ) , implementation details about how the conditioning variable is used in the denoising architecture can be found in the appendix C.
Experiments
In this section we show the applicability of GAUDI to multiple problems.
First, we evaluate reconstruction quality and performance of the reconstruction stage.
Then, we evaluate the performance of our model in generative tasks including unconditional and conditional inference, in which radiance ﬁelds are generated from conditioning variables corresponding to images or text prompts.
Full experimental settings and details can be found in the appendix B.
4.1 Data
We report results on 4 datasets: Vizdoom [23], Replica [64], VLN-CE [26] and ARKit Scenes [1], which vary in number of scenes and complexity (see Fig.
3 and Tab. 1).
Vizdoom [23]: Vizdoom is a synthetic simulated environment with simple texture and geometry.
We use the data provided by [8] to train our model.
It is the simplest dataset in terms of number of scenes and trajectories, as well as texture, serving as a test bed to examine GAUDI in the simplest setting.
Replica [64]: Replica is a dataset comprised of 18 realistic scenes from which trajectories are rendered via Habitat [59].
We used the data provided by [8] to train our model.
VLN-CE [26]: VLN-CE is a dataset originally designed for vision and language navigation in continuous environments.
This dataset is composed of 3 .
6 K trajectories of an agent navigating between two points in a 3D scene from the Matterport 3D dataset [7].
We render observations via
[Figure]
Habitat [59].
Notably, this dataset contains also textual descriptions of the trajectories taken by an agent.
In Sect.
4.5 we train GAUDI in a conditional manner to generate 3D scenes given a description.
ARKitScenes [1]: ARKitScenes is a dataset of scans of indoor spaces.
This dataset contains more than 5 K scans of about 1 .
6 K different indoor spaces.
As opposed to the previous datasets where
RGB, depth and camera poses are obtained via rendering in a simulation ( i .
e .
either Vizdoom [23] or
Habitat [59])
, ARKitScenes provides raw RGB and depth of the scans and camera poses estimated using ARKit SLAM.
In addition, whereas trajectories from the previous datasets are point-to-point, as typically done in navigation, the camera trajectories for ARKitScenes resembles a natural scan a of full indoor space.
In our experiments we use a subset of 1 K scans from ARKitScenes to train our models.
4.2 Reconstruction
We ﬁrst validate the hypothesis that the optimization problem described in Eq.
2 can ﬁnd latent codes z that are able reconstruct the trajectories in the empirical distribution in a satisfactory way.
In Tab.we report reconstruction performance of our model across all datasets.
Fig.
4 shows reconstructions of random trajectories for each dataset.
For all our experiments we set the dimension of z scene and z pose to 2048 and  = 0 . 1 unless otherwise stated.
During training, we normalize camera poses for each trajectory so that the middle frame in a trajectory becomes the origin of the coordinate system.
See appendix E for ablation experiments.
[Figure]
[Table]
4.3 Interpolation
In addition to evaluate the structure of the latent representation obtained from minimizing the optimization problem in Eq.
we show interpolation results between pairs of latents ( z i , z j ) in
[Figure]
jPu/Bsz7Sy09UDgcM693JMTJpxp47rfTmlldW19o7xZ2dre2d2r7h [Figure] 0tUwVoS0iuVTdEGvKmaAtwwyn3URRHIecdsLxTe53HqnSTIp7M0loEOOhYBEj2FjJ92NsRmGUPU37D [Figure]
1qza27M6Bl4hWkBgWa
[Figure]
eqXP5AkjakwhGOte56bmCDDyjDC6bTip5ommIzxkPYsFTimOshmmafoxCoDFEllnzBopv7eyHCs9SQO7WSeUS96ufif10tNdBVkTCSpoYLMD0UpR0aivAA0YIoSwyeWYKKYzYrICCtMjK2pYkvwFr
[Figure] 8TNpnde [Figure] ifn53XmtcF3WU4QiO4RQ8uIQG3EITWkAggWd4hTcndV6cd
[Figure]
djPlpyip1D [Figure] APn8weGwJIE [Figure]
latexit
[Figure]
4.4 Unconditional generative modeling
Given latent representations z 2 Z that can reconstruct samples x 2 X with high accuracy as shown in Sect.
4.2, we now evaluate the capacity of the prior p ✓ p ( Z ) to capture the empirical distribution x 2 X by learning the distribution of latents
z
i 2 Z .
To do so we sample z
⇠ p ✓ p ( Z ) by following the inference process in DDPMs, and then feed z through the decoder network, which results in trajectories of RGB images that are then used for evaluation.
We compare our approach with the following baselines: GRAF [60], ⇡ -GAN [6] and GSN [8], where all models have access to ground-truth depth information during training.
We sample 5 k images from predicted and target distributions for each model and dataset and report both FID [16] and SwAV-FID [35] scores.
We report quantitative results in Tab. 2, where we can see that GAUDI obtains state-of-the-art performance across all datasets and metrics.
We attribute this performance improvement to the fact that GAUDI learns disentangled yet corresponding latents for radiance ﬁelds and camera poses, which is key when modeling scenes (see ablations in the appendix E).
We note that to obtain these great empirical results GAUDI needs to simultaneously ﬁnd latents with high reconstruction ﬁdelity while also efﬁciently learning their distribution.
[Table]
In Fig.
6 we show samples from the unconditional distribution learnt by GAUDI for different datasets.
We observe that GAUDI is able to generate diverse and realistic 3D scenes from the empirical distribution which can be rendered from the sampled camera poses.
4.5 Conditional Generative Modeling
GAUDI can also tackle conditional generative problems of the form p ( Z | Y ) , where a conditioning variable y 2 Y is given to condition p ( Z ) .
For a given conditional inference problem we assume the existence of paired data [46, 11, 45].
As an example, for training a text-conditional model we assume the existence of pairs { z i , y i } , where z i is a latent scene representation and y i is its corresponding text prompt.
In this section we show both quantitative and qualitative results for conditional inference problems.
The ﬁrst conditioning variable we consider are textual descriptions of trajectories.
Second, we consider a model where randomly sampled RGB images in a trajectory act as conditioning.
Finally, we use a categorical variable that indicates the 3D environment ( i .
e .
the particular indoor space) from which each trajectory was obtained.
Tab.
3 shows quantitative results for the different conditional inference problems.
Details on the implementation of the conditional DDPM are given in Sect.
D
[Figure]
[Table]
4.5.1 Text Conditioning
We tackle the challenging task of training a text conditional model for 3D scene generation.
We use the navigation text descriptions provided in VLN-CE [26] to condition our model.
These text descriptions contain high level information about the scene as well as the navigation path ( i .
e . "Walk out of the bedroom and into the living room" , "Exit the room through the swinging doors and then enter the bedroom" ).
We employ a pre-trained RoBERTa-base [29] text encoder and use its intermediate representation to condition the diffusion model.
Fig.
7 shows qualitative results of
GAUDI for this task.
To the best of our knowledge, this is the ﬁrst model that allows for conditional 3D scene generation from text in an amortized manner ( i .
e .
without distilling CLIP [44] through a costly optimization problem [19, 32]).
[Figure]
4.5.2 Image Conditioning
We now analyze whether GAUDI is able to pick up information from the RGB images to predict a distribution over Z .
In this experiment we randomly pick images in a trajectory x 2 X and use it as a conditioning variable y .
For this experiment we use trajectories in the VLN-CE dataset [26].
During each training iteration we sample a random image for each trajectory x and use it as a conditioning variable.
We employ a pre-trained ResNet-18 [15] as an image encoder.
During inference, the resulting conditional GAUDI model is able to sample radiance ﬁelds where the given image is observed from a stochastic viewpoint.
In Fig.
8 we show samples from the model conditioned on different RGB images.
[Figure]
4.5.3 Categorical Conditioning
Finally, we analyze how GAUDI performs when conditioned on a categorical variable that indicates the underlying 3D indoor environment in which each trajectory was recorded.
We perform experiments in the VLN-CE [26] dataset, where we employ a trainable embedding layer to learn a representation for categorical variables indicating each environment.
We compare the per-environment
FID score of conditional model with its unconditional counterpart.
This per-enviroment FID score is computed only on real images of the same indoor environment that the model is conditioned on.
Our hypothesis is that if the model efﬁciently captures the information in the conditioning variable it should capture the environment speciﬁc distribution better than its unconditional counterpart trained on the same data.
In Tab. 3 the last column shows difference ( e . g .
the  ) on the average per-environment FID score between the conditional and unconditional model on VLN-CE dataset.
We observe that the conditional model consistently obtains a better FID score than the unconditional model across all indoor environments, resulting in a sharp reduction of average FID and SwAV-FID scores.
In addition, in Fig.
9 we show samples from the model conditioned on a given categorical variable.
Conclusion
We have introduced GAUDI, a generative model that captures distributions of complex and realistic 3D scenes.
GAUDI uses a scalable two-stage approach which ﬁrst involves learning a latent representation that disentangles radiance ﬁelds and camera poses.
The distribution of disentangled latent representations is then modeled with a powerful prior.
Our model obtains state-of-the-art performance when compared with recent baselines across multiple 3D datasets and metrics.
GAUDI can be used both for conditional and unconditional problems, and enabling new tasks like generating 3D scenes from text descriptions.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information on how to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or [N/A] .
You are strongly encouraged to include a justiﬁcation to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .
• Did you include the license to the code and datasets?
[No] The code and the data are proprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] These will be discussed in appendix A (c) Did you discuss any potential negative societal impacts of your work?
[Yes] These will be discussed in appendix A (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A] (b) Did you include complete proofs of all theoretical results?
[N/A] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] We will release code publicly upon acceptance.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] These details will be clariﬁed in B (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] These details will be clariﬁed in B 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] See Sect.(b)
Did you mention the license of the assets?
[Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
[N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]