Learning Neural Acoustic Fields
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Our environment is ﬁlled with rich and dynamic acoustic information.
Whenwe walk into a cathedral, the reverberations as much as appearance inform us ofthe sanctuary
’s wide open space.
Similarly, as an object moves around us, weexpect the sound emitted to also exhibit this movement.
While recent advances inlearned implicit functions have led to increasingly higher quality representations ofthe visual world, there have not been commensurate advances in learning spatialauditory representations.
To address this gap, we introduce Neural Acoustic Fields(NAFs), an implicit representation that captures how sounds propagate in a physicalscene.
By modeling acoustic propagation in a scene as a linear time-invariantsystem, NAFs learn to continuously map all emitter and listener location pairs to aneural impulse response function that can then be applied to arbitrary sounds.
Wedemonstrate NAFs on both synthetic and real data, and show that the continuousnature of NAFs enables us to render spatial acoustics for a listener at arbitrarylocations.
We further show that the representation learned by NAFs can helpimprove visual learning with sparse views.
Finally we show that a representationinformative of scene structure emerges during the learning of NAFs.
Project site:https://sites.google.com/view/nafs-neurips
Introduction
The sound of the ball leaving the bat, as much as its visible trajectory, tells us whether the hit is likely to be a home run or not.
Our experience of the world around us is rich and multimodal, depending onintegrated input from multiple sensory modalities.
In particular, spatial acoustic cues provide us witha sense of the direction and distance of a sound source without needing visual conﬁrmation, allowus to estimate the properties of a surrounding environment and are critical to subjective realism ingaming and virtual simulations.
Recent progress in implicit neural representations has enabled the construction of continuous, differentiable representations of the visual world directly from raw image observations [1, 2, 3, 4].
However, our perception of the physical world is informed not only by our visual observations, butalso by the spatial acoustic cues present in the environment.
As a preliminary step in learning theacoustic properties of scenes, we explore an implicit model that represents the underlying impulseresponse of audio reverberations.
As shown in Figure 1, our model can model the spatial propagationof sound in a physical scene.
Past work has explored capturing the underlying acoustics of a scene [5, 6, 7].
These models,however, require handcrafted parameterizations which, critically, prevent such approaches from beingapplied to arbitrary scenes.
In this work, we extend this approach by constructing an implicit neuralrepresentation which captures, in a generic manner , the underlying acoustics of a scene.
Learning a representation of scene acoustics poses several challenges compared to the visual setting.
First, how do we generate plausible audio impulse responses at each emitter-listener position?
While
[Figure]
we may represent the visual appearance of a scene with an underlying three-dimensional vector, anacoustic reverberation (represented as an impulse response) can consist of over 10,000 time-domainvalues and, thus, is signiﬁcantly harder to capture.
Second, how do we learn an acoustic neuralrepresentation that densely generalizes to novel emitter-listener locations?
In the visual setting,ray-tracing can enforce view consistency across large portions of a visual scene (modulo occlusions).
While in principle, in a similar manner, we may reﬂect acoustic "rays" in our scene to obtain animpulse response, an intractable number of rays are necessary to obtain the desired representation.
To address both challenges, we propose Neural Acoustic Fields (NAFs).
To capture the complexsignal representation of impulse responses in a compact and spatially continuous fashion, NAFsencode and represent an impulse-response in the Fourier frequency domain.
Motivated by the stronginﬂuence of nearby geometry on anisotropic reﬂections [6], we propose to condition NAFs onlocal geometric information present at both the listener and emitter locations when decoding theimpulse response.
In our framework, local geometric information is learned directly from impulseresponses.
Such a decomposition facilitates the transfer of local information captured from trainingemitter-listener pairs to novel combinations of emitters and listeners.
We show that NAFs are able to outperform baselines in modeling scene acoustics, and providedetailed analysis of the design choices in NAFs.
We further illustrate how the structure learned by
NAFs can improve cross-modal generation of novel visual views of a scene.
Finally, we illustrate howthe learned representation of NAFs enable the downstream application of inferring scene structure.
Related Work
Audio Field
Coding There is a rich history of encoding methods for 3D spatial audio. Theseapproaches largely fall into two categories.
The ﬁrst approach encodes the sound ﬁeld at a usercentric location by capturing the sound from spatially distributed sources [8, 9, 10, 11].
While theymay leverage perceptual cues to create the sense of spatial audio, they do not allow the listener tofreely traverse the scene and experience sound from different locations.
The second approach aims tomodel the sound heard as a listener moves in a scene [5, 12, 6, 7, 13].
Since the complete acousticﬁeld of a scene is computationally prohibitive to simulate in real time, and expensive to store infull ﬁdelity, these methods have relied on a handcrafted encoding of the acoustic ﬁeld, prioritizingefﬁciency above reproduction ﬁdelity.
Our work allows a listener to move and experience sounds thatcome from anywhere in a scene, and can represent the acoustic ﬁeld continuously at high ﬁdelity bydirectly learning from data.
Implicit representations Our approach towards modeling the underlying acoustics a scene relies onthe use of a neural implicit representations.
Implicit representations have emerged as a promisingrepresentation of 3D geometry [14, 15, 16, 17] and appearance [1, 2, 3, 4, 18] of a scene.
Comparedto traditional discrete representations, implicit representations are a continuous mapping capable ofcapturing data at an "inﬁnite resolution".
In [19] proposed a grid based representation for implicitscene reconstruction, while more recently [20] has adopted spatial conditioning for 3D imagesynthesis, where in both settings, the grid enables a higher-ﬁdelity encoding of the scene.
Our workalso leverages local grids to model acoustics, but as an inductive bias and way to generalize to novelinputs.
Audio-Visual Learning Our work is also closely related to joint modeling of vision and audio.
Byleveraging the correspondence between vision and audio, work has been done to learn unsupervisedvideo and audio representations [21, 22], localize objects that emit sound [23, 24], and jointly usevision and audio for navigation [25].
Recent work aims to propose plausible reverberations or sounds [Figure] Emitter [Figure]
Local
[Figure]
Implicit Decoder
[Figure]
from image input [26, 27], these approaches model the STFT using either convolution or implicitfunctions, which we also utilize.
Different from them, our work leverages the geometric featureslearned by modeling acoustic ﬁelds to improve the learning of 3D view generation.
Methods
We are interested in learning a generic acoustic representation of an arbitrary scene, which cancapture the underlying sound propagation of arbitrary sound sources across both seen and unseenlocations in a scene.
We ﬁrst review relevant background information towards modeling environmentreverberations.
We then describe Neural Acoustic Fields (NAFs), a neural ﬁeld which we showcan capture, in a generic manner, the acoustics of arbitrary scenes.
We further discuss how we canparameterize NAF so that it can capture acoustics property even at unseen sound sources and listenerpositions.
Finally, we discuss the implementation details of our model illustrated in Figure 2.
3.1 Background on the Propagation of Sound
The sound emitted by a sound source undergoes decay, occlusion, and scattering due to both thegeometric and material properties of a scene.
For a ﬁxed location pair ( q , q (cid:48) ) , we deﬁne the impulseresponse at a listener position q , as the sound pressure p ( t ; q , q (cid:48) ) induced by an impulse at q (cid:48) .
Suchbehavior can be concisely and elegantly modeled utilizing the linear wave equation [28]: [Equation] where c is the speed of sound, p is the sound pressure, ( q , q (cid:48) ) being the listener and emitter locationrespectively, and δ the Dirac delta representing the forcing function, where we refer to sound pressurep ( t ; q , q (cid:48) ) as the impulse-response observed at listener position q . Given an accurate model of the impulse-response p ( t ; q , q (cid:48) ) described in Eqn (1), we may modelaudio reverberation of any sound waveform s ( t ) emitted at q (cid:48) , by computing the response r ( t, q , q (cid:48) )at listener location q by querying the continuous ﬁeld and using temporal convolution:
[Equation]
3.2 Neural Acoustic Fields
We are interested in constructing a continuous representation of the underlying acoustics of a scene,which may specify the reverberation patterns of an arbitrary sound source.
The parameterization ofan impulse-response introduced in Section 3.1 provides us with a method to model audio propagationwhen given an omnidirectional listener and emitter.
To construct a model of a directional listener, weneed to further model the 3D head orientation θ ∈ R 2 , and ear k ∈ { 0 , 1 } (binary left or right) of alistener, in addition to the spatial position q ∈ R 3 of the listener and q (cid:48)
∈ R 3 of the emitter.
We may then model the time domain impulse response v using a neural ﬁeld Φ which takes as inputthe listener and emitter parameters:
[Equation]
Directly outputting the impulse-response v ∈ R T magnitude in the time domain with a neuralnetwork is difﬁcult due to its high dimensional (over 10,000 elements) and chaotic nature.
A naïve [Figure] solution would be to further add t as an additional argument to our neural ﬁeld, but we found thatsuch a solution worked poorly, due to the highly non-smooth representation of the waveform (seesupplementary).
We instead encode the impulse-response utilizing a short-time Fourier transform(STFT) denoted v STFT , which we ﬁnd to be signiﬁcantly more amenable to neural network predictiondue to the smooth nature of the frequency space.
In Figure 3 we show magnitude spectrograms forground truth impulse responses and those learned by our network.
As v STFT is a complex value, wefurther factorize v STFT into log-magnitude and phase components.
For phase, we use the instantaneousfrequency (IF) representation proposed in [29].
Thus, our parameterization of NAF is a neural ﬁeldΩ that is trained to estimate the impulse response function φ , and outputs [ v STFT_mag , v STFT_IF ] for agiven time and frequency coordinate:
[Equation]
We train our model using MSE loss between the generated and ground truth spectrograms v STFT :
[Equation]
(5) across spectrogram coordinates t and f .
Where α is a scaling value used to balance the two losses.
3.3 Generalization through Local Geometric Conditioning
We are interested in parameterizing the underlying acoustic ﬁeld, so that we may not only accuratelyrepresent impulse-response at emitter-listener pairs we see during training, but also at novel combinations of emitter and listener seen at test time.
Such generalization may be problematic when directlyparameterizing NAFs utilizing a MLP with inputs speciﬁed in Eqn (4), as the network may learn todirectly overﬁt and entangle the relation between emitter and listener impulse-responses.
What generic information may we extract from a given impulse-response between an emitter andlistener?
In principle, extracting the full dense geometric information in a scene would enable usto robustly generalize to new emitter and listener locations.
However, the amount of geometricinformation available in a particular impulse-response, especially for positions far away from eithercurrent emitter and listener is limited, since these positions have little impact on the underlyingimpulse-response.
In contrast, the local geometry near either emitter and listener positions willhave a strong inﬂuence in the impulse-response, as much of the anisotropic reﬂection comes fromsuch geometry [30].
Inspired by this observation, we aim to capture and utilize local geometricinformation, near either emitter or listener locations, as a means to predict impulse-responses acrossnovel combinations.
To parameterize and represent these local geometric features, we learn a 2D grid of spatial latentswhich we illustrate in Figure 2.
When predicting an impulse-response at a given emitter and offsetposition, we query the learned grid features at both emitter and listener positions, and provide itas additional context into our NAF network Ω .
Such features provide rich information on theimpulse-response, enabling NAF to generalize better to unseen combinations of both emitter andlistener locations.
In the rest of this work, we refer to the NAFs with local geometric features asΩ grid .
We learn grid latent features jointly with the underlying parameters of NAF.
Additional detailscan be found in the supplementary.
[Figure] Such a design choice, however, still requires us to consider how to further combine local geometricinformation captured separately from either listeners or emitters.
A naïve implementation would beto maintain separate feature grids for both listener and emitter positions.
Such an approach fails toaccount for the fact that the local geometric information captured by emitter may also inform the localgeometric information around a listener.
Examining Eqn (1), we note that it is in fact symmetric withrespect to exchanging either listener or emitter positions [7], indicating that the impulse-responsedoes not change when omnidirectional listener and emitters are swapped.
Such a result means that wemay in fact utilize the local geometric information captured near an emitter position interchangeablyfor either emitters and listeners.
Thus, we propose our local geometric information as a single latentgrid, which we show to outperform the naïve dual grid implementation.
Experiments
In this section, we demonstrate that our model can faithfully represent the acoustic impulse responseat seen and unseen locations.
Additional ablation studies verify the importance of utilizing localgeometric features to enable test time generation ﬁdelity.
Next, we demonstrate that learning acousticﬁelds could facilitate improved visual representations when training images are sparse.
Finally weshow that the learned NAF can be used to infer scene structure.
4.1 Setup
For each scene, we holdout 10% of the RIRs randomly as a test set.
Each scene is trained for 200epochs, which takes around 6 hours for the largest scenes on four Nvidia V100s .
In each batch,we sample 20 impulse responses, and randomly select 2 , 000 frequency & time pairs within eachspectrogram.
An initial learning rate of 5 × 10 − 4 is used for the network and the grid features.
Weadd a small amount of noise sampled from N (0 , 0 . 1) to each coordinate during training to preventdegenerate solutions.
For evaluating the learned acoustic ﬁelds, we use two different datasets:
Soundspaces.
Soundspaces [25] is a synthetic dataset generated via ray-tracing, and is auralizedwith an ambisonic head-related transfer function (HRTF).
This dataset consists of R i probe pointsfor each scene, with each probe capable of representing an emitter or listener location for up to R 2 iemitter and listener pairs.
The emitters are represented as omnidirectional, while the listener acts as astereo receiver that can have one of four different orientations.
The listeners and emitters are at ﬁxedheight.
Our NAFs are trained on 6 representative scenes. Where 2 consist of multi-room layouts; 2consist of a single room with a non-rectangular walls; and 2 consist of a single room with rectangularwalls as in Figure 4.
MeshRIR.
The MeshRIR dataset [31] is recorded from a real scene, and contains data collectedfrom a cuboidal room.
The listener locations are at ﬁxed height.
The emitters surround the listenersboth above and below the listener plane.
4.2 Architecture Details
The Soundspaces dataset lacks the full parameterization of an acoustic ﬁeld described in Equation 4,so we train NAF with a restricted parameterization that is available in the dataset.
This allows for [Figure] two degrees of freedom along the x − y plane for the listener locations q ∈ R 2 and the emitterlocation q (cid:48)
∈ R 2 .
The listener is binaural with k ∈ { 0 , 1 } , and can assume four possible orientationsθ ∈ { 0 , 90 , 180 , 270 } , while the emitter is omnidirectional.
In particular, we utilize a parameterizationof Ω grid which maps an input tuple [ x, y, x (cid:48) , y (cid:48) , f, t ] × { 0 , 90 , 180 , 270 } × { 0 , 1 } to two scalar valuesthat represents the magnitude and phase for a given time and frequency in the STFT:
[Equation]
To encode the rotation θ , as there are only 4 possible discrete rotations in the dataset, we directlyquery into a learnable embedding matrix of shape R 4 × n , returning a R 1 × n vector.
Similarily, toencode the left and right ear k , we similarly query into a learnable embedding matrix of shape R 2 × n ,returning a R 1 × n vector.
The f, t tuple representing the frequency and time respectively are scaled to( − 1 , 1) and processed with sinusoidal encoding using 10 frequencies of sin and cos .
For MeshRIR,we set q (cid:48)
∈ R 3 to account for the emitters that can vary in height.
To obtain local geometric features for either an emitter or listener in a scene, we assume that ourscene is contained within a set of pixels P
= { P 1 ...P k } which form a grid over the scene.
For agiven position tuple ( x, y ) as query location, we then interpolate the local features.
Where L ( · ) is theinterpolation function.
( p ∗ 1 . . .
p ∗ k ) are the set of all pixel that form the grid, and ˜ f ( · ) represents thefeatures stored at a given pixel:
[Equation]
w i is determined by a Nadaraya-Watson estimator with a Gaussian weighting kernel applied to thedistance between query and grid coordinates:
[Equation]
Because this interpolation function is differentiable, we jointly learn the grid features during training.
These queried features are combined with the coordinates processed with sinusoidal encoding using10 frequencies of sin and cos functions.
We process both the listener and emitter position tuples thisway.
We combine the grid based features with the sinusoidal embeddings and the discrete indexedembeddings as the input to our multilayer perceptron f φ .
Please refer to Figure 2 for a visualizationof our model, and supplementary for further details.
We compare using a shared local geometricfeature with the emitter and listener, as well as using have the emitter and listener query their ownindividual grids.
4.3 Evaluation of Neural Acoustic Fields
We ﬁrst validate that we can capture environmental acoustics at unseen emitter-listener positions.
Baselines.
We compare our model against two widely used high performance audio coding methods: Advanced Audio Coding (AAC) and Xiph Opus.
The bitrates are chosen to approach the storagecosts for our NAF.
For each method, we apply both linear and nearest neighbor interpolation to the
[Table] [Table]
[Table] [Table]
coded acoustic ﬁelds.
Both linear and nearest neighbor approaches are widely used [32, 33, 34] inmodeling of spatial audio.
We implement the binaural DSP baseline described by [35], which usesthe image source method and a KEMAR HRTF.
We also compare the listener and emitter eithersharing or using individual local geometric features in our NAFs.
Each method is provided with the same train-test split.
We visualize the acoustic ﬁelds produced bydifferent methods in Figure 5.
For additional details of our baselines please see the appendix.
Metrics.
We evaluate the results of our synthesis by measuring the spectral loss [36] between thegenerated and the ground truth log-spectrograms, as well as measuring the percentage error betweenthe T60 reverberation time in the time domain.
In this case, lower spectral loss and T60-error valuesindicates a better result.
We also perform a human evaluation where subjects are presented witha two-alternative forced-choice task.
Each trial requires selecting if the NAFs or the AAC-nearestauralized music samples best match with ground truth auralization.
[Table]
Results.
As shown in Table 1, our NAFs achieve signiﬁcantlyhigher quality on the modeling of unseen impulse responses compared to strong interpolation baselines across all six scenes.
Observing the qualitative results in Figure 4, we observe that NAFscan predict smoothly varying acoustic ﬁelds that are affected bythe physical surroundings.
Extending our model to MeshRIRwhich is captured from a real scene, we observe that our NAFscontinue to perform better on both spectral and T60 metrics.
Comparing our results against baselines in Figure 5 and Table 3, ourmethods are able to better approximate the ground truth at a fraction of the storage cost, and does notexhibit the sound energy leakage present in linear interpolation.
The size of a spatial acoustic ﬁeldis important for real life applications.
In our human evaluation with 21 subjects who were asked tojudge 10 test-time RIRs, 82.38% of responses indicates that our NAFs were higher quality comparedto AAC-Nearest.
A comparison of using shared and dual local geometric features indicates that despite having fewerlearnable parameters, we achieve better performance by sharing the local geometric features.
Examples of individual impulse responses generated by our model are shown in Figure 3.
Generalization through Geometric Conditioning.
We next assess the impact of utilizinglocal geometric conditioning as a means to generalize to novel combinations of emitterlistener positions.
On the "Large 1" room, in Figure 7 we evaluate test set spectral er- ror when NAF is trained with a limited percentage of the training data either with orwithout local geometric conditioning.
We ﬁnd that such geometric conditioning enablesbetter test set reconstruction error, with the performance gap increasing with less data. [Figure] G B on l y
[Figure]
G B + A c ou s ti c
[Figure]
[Figure]
4.4 Cross-modal learning
[Figure]
In this experiment, we explore the effect of jointly learning acoustics and visual information whenwe are given sparse visual information.
Recall that our NAF includes a local geometric feature grid
P that covers the entire scene.
For our cross-modal learning experiment, we jointly learn this featuregrid with a NeRF network modiﬁed to accept local features sampled from this grid along with thetraditional sinusoidal embedding.
In the acoustics branch, we query the grid using emitter and listenerpositions.
In the NeRF branch, we use point samples along the ray projected on the grid plane toquery the features.
In both cases, the process is fully differentiable.
We use a standard implementationof NeRF with a coarse and ﬁne network.
In the NeRF only setting, we minimize color C reconstruction loss for a ray r over a batch of rays R :
L RGB =
(cid:80) r ∈R
|| ˆ C ( r ) − C ( r )
|| 22 .
In contrast, in the NAF + NeRF experiment, we jointly minimize
L RGB + L NAF , where L NAF is deﬁned in equation 5.
We utilize 64 coarse samples and 128 ﬁnesamples for each ray, and sample 1024 rays per batch.
Results.
We train on the two large rooms in our training set.
For each room 75 , 100 , 150 images areused for training, while the same 50 images of novel views are used for all conﬁgurations duringtesting.
In Table 2 we observe that training with acoustic information helps improve the PSNR and
SSIM of the visual output.
This effect is more signiﬁcant when the training images are very sparse,the NAF network helps less when there is sufﬁcient visual information.
Qualitative results are shownin Figure 6, we see there is a reduction of ﬂoaters in free space.
4.5 Inferring scene structure
Given a reverberant sound, humans are able to build a mental representation of the surrounding roomand make a judgement about the distance of nearby obstacles [37].
We investigate the intermediaterepresentations constructed by our neural network in the process of learning an acoustic ﬁeld, andexamine if these representations can be used to decode the scene structure.
Setup.
The intermediate representation of the NAF depends on both listener locations q and emitterlocation q (cid:48) , the rotation angle θ , the ear k , the time t and frequency f .
For consistency, at a givenlocation ( x ∗ , y ∗ ) in the scene, we extract the NAF latent by setting the emitter location q (cid:48)
i =
( x ∗ , y ∗ ) i .
For the listener location, we iterate over ﬁve randomly selected points in the scene q ∈
[ q 1 , . . . , q 5 ] ,which we keep constant for all q (cid:48)
i .
The rotation angle is ﬁxed to θ = 0 , and we compute therepresentation average over all possible ( k, t, f ) , and concatenate latents for the selected q .
For our
NAFs, latents are extracted from the last layer prior to the output which includes 512 neurons.
As acomparison to our learned representation, we extract Mel-frequency cepstral coefﬁcients (MFCCs)from the ground truth impulse response provided by a nearest neighbor interpolator.
We use a similarsetup as above, for a given location we set this to be q (cid:48) i , and iterate over the same ﬁve
[Figure]
[Table]
We ﬁt a single linear layer to NAF and MFCC features respectively.
For testing and visualization ofthe linear decoding results, we sample a regular grid of points with 0 .
1 m distance between each point.
For ﬁtting the linear decoder, we randomly sample points within the scene such that the number oftraining points are 10% as many as the testing points.
For each location in the scene, we extract thedistance to the nearest wall as the decoding target.
Results.
We visualize the results of our linear decoding in Figure 8.
The intermediate representationof our NAFs reveals an underlying structure that is both smooth and semantically meaningful.
Inthe multiroom scenes, the latent is well separated for each room.
We are able to successfully decodethe scene structure with a linear layer when using our NAFs, but decoding fails when using MFCCfeatures.
In Table 4, we show the amount of explained variance of our decoding results on the testset.
Our learned features are able to consistently achieve much higher scores than those using MFCCfeatures.
Discussion
Limitations and Future Work.
Although our method achieves generalization and high qualityrepresentations of acoustic ﬁelds within a single scene, NAFs do not currently generalize to multiplescenes.
Future work may explore generalization to novel scenes.
One possible approaches may be toincorporate multi-modal inputs with the goal of synthesizing an acoustic ﬁeld with few-shot visual oracoustic input.
Conclusion.
In summary, this paper introduces Neural Acoustic Fields (NAFs), a compact, continuous, and differentiable acoustic representation which can represent the underlying reverberation ofdifferent audio sources in a scene.
By conditioning NAFs locally on the underlying scene geometry,we demonstrate that our approach enables the prediction of plausible environmental reverberationseven at unseen locations in the scene.
Furthermore, we demonstrate that the acoustic representations learned through NAFs are powerful, and may be utilized to facilitate audio-visual cross-modallearning, as well as to infer the structure of scenes.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] Will beincluded in the supplementary.(d)
Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A](b)
Did you include complete proofs of all theoretical results?
[N/A]3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See anonymousrepo provided on site.
(b) Did you specify all the training details (e
g., data splits, hyperparameters, how theywere chosen)?
[Yes] .(c)
Did you report error bars (e
g., with respect to the random seed after running experiments multiple times)?
[Yes] We use a single ﬁxed seed for all our experiments.(d)
Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[No]4.
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[No] We utilize public datasets only.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No] (d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[Yes]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[N/A] Does not apply to our data.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[Yes]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[Yes] Was not determined to create any risks.
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[No]