SCONE: Surface Coverage Optimization in Unknown Environments by Volumetric Integration
Anonymous Author(s)
AffiliationAddress email
Abstract
Next Best View computation (NBV) is a long-standing problem in robotics, andconsists in identifying the next most informative sensor position(s) for reconstructing a 3D object or scene efficiently and accurately.
Like most current methods, weconsider NBV prediction from a depth sensor.
Learning-based methods relyingon a volumetric representation of the scene are suitable for path planning but donot scale well with the size of the scene and have lower accuracy than methodsusing a surface-based representation.
However, the latter constrain the camera toa small number of poses.
To obtain the advantages of both representations, weshow that we can maximize surface metrics by Monte Carlo integration over avolumetric representation.
Our method scales to large scenes and handles freecamera motion: It takes as input an arbitrarily large point cloud gathered by a depthsensor like Lidar systems as well as camera poses to predict NBV.
We demonstrateour approach on a novel dataset made of large and complex 3D scenes.
Introduction
Next Best View computation (NBV) is a long-standing problem in robotics [6, 29], which consistsin identifying the next most informative sensor position(s) for reconstructing a 3D object or sceneefficiently and accurately.
Typically, a position is evaluated on how much it can increase the totalcoverage of the scene surface.
Few methods have relied on Deep Learning for the NBV problem,even though Deep Learning can provide useful geometric prior to obtain a better prediction of thesurface coverage [33, 16, 25].
Like most current methods, we consider NBV prediction from a depthsensor.
Existing methods based on a depth sensor rely either on a volumetric or on a surface-basedrepresentation of the scene geometry.
Volumetric mapping-based methods can compute collisionefficiently, which is practical for path planning in real case scenarios [20, 23, 24, 14, 1, 7].
However,they typically rely on voxels or a global embedding [12, 3, 21, 22, 27, 5] for the scene, whichresults in poor accuracy of the reconstructions and poor performance in NBV selection for complex3D objects.
On the contrary, surface mapping-based methods that process directly a dense pointcloud of the surface as gathered by the depth sensor are efficient for NBV prediction with highdetailed geometry.
They are however limited to very specific cases, generally a single, small-scale,isolated object floating in the air with the camera constrained to stay on a sphere centered on theobject [15, 24, 8, 4, 13, 14, 14, 33, 16, 25].
Thus, they cannot be applied to the exploration of 3Dscenes.
As shown in Figure 1, we introduce a volumetric Deep Learning method that efficiently identifies
Next Best Views for unknown large-scale 3D scenes in which the camera can move freely.
Insteadof representing the scene with a single global embedding, we choose to use a grid of local pointclouds, which scales much better to large and complex 3D scenes.
We show how to learn to predictthe visibility of new 3D points in all directions given an estimate of the 3D scene geometry.
We can then integrate these visibilities in the direction of any camera by using a Monte Carlo integrationapproach, which allows us to optimize the camera pose to find the next most informative views.
Wecall our method SCONE, for Surface Coverage Optimization in uNknown Environments.
In this respect, we introduce a theoretical framework to translate the optimization of surface coveragegain, a surface metric on manifolds that represents the ability of a camera to increase the visible areaof the surface, into an optimization problem on volumetric integrals.
Such a formalism allows us touse a volumetric mapping of geometry, which is convenient not only to scale the model to explorationtasks and scene reconstruction, but also to make probabilistic predictions about geometry.
In particular, given a partial point cloud gathered by a depth sensor, our model learns to makepredictions with virtually infinite resolution about the occupancy probability in the scene volume bylearning a deep implicit function [17, 28, 18, 30, 31, 19].
Such predictions scale to very large pointclouds since they depend only on neighborhood geometry.
Then, for each point, our model leveragesa self-attention mechanism [26] to predict occlusions and learn information score functions mappedon a continuous sphere that represent visibility likelihood in all directions.
The occupancy probabilityfield is finally used as a basis to sample points and compute Monte Carlo integrals of visibility scores.
Since NBV learning-based methods are mostly limited to single, small-scale, centered object reconstruction in literature, we first compare the performance of our model to the state of the art on the
ShapeNet dataset [2], following the protocol introduced in [33].
While our method was designed tohandle more general frameworks such as 3D scene reconstruction and continuous cameras poses inthe scene, it outperforms the state of the art for dense reconstruction of objects when the camera isconstrained to stay on a sphere centered on the object.
We then conduct experiments in large 3D environments using a simple planning algorithm that buildsa camera trajectory online by iteratively selecting NBVs with SCONE.
Since, to the best of ourknowledge, we propose the first supervised Deep Learning method for such free 6D motion of thecamera, we created a dataset made of several large-scale scenes under the CC License for quantitativeevaluation.
We will make this dataset available for allowing comparison of future methods with
SCONE.
Approach
As discussed in the introduction, our method relies on a volumetric representation of the objector scene to reconstruct because such a representation is useful for collision-free robot navigation.
However, volumetric methods are generally less efficient to predict views that maximize densesurface metrics, because they dilute information in the 3D space with a sparser representation, e.g. , avoxel-based representation or a global encoding, and thus suffer from a loss in resolution.
To overcome this problem, we first show that we can maximize a surface metric by integrating over avolumetric representation with virtually infinite resolution.
As we argue below, this integration over a volume is much more efficient for optimizing surface coverage gain than the alternative of identifyingthe 3D points lying on the surface, which is difficult in an unknown and occluded environment.
Moreexactly, we derive a volumetric integral that is asymptotically proportional to the surface metric,which is enough for our maximization problem.
We first present this derivation considering volumerepresented as a perfect binary occupancy map.
We finally explain how our method, which takes as input a point cloud gathered so far by a depthsensor as well as the history of camera poses, predicts all terms involved in the volumetric integrationby leveraging neural models and self-attention mechanisms to predict occupancy and occlusions.
2.1 Maximizing Surface Coverage Gain on a Binary Occupancy Map
Here, we consider a binary occupancy map σ : R 3 → { 0 , 1 } representing the volume of the targetobject or scene.
We will relax our derivations to a probabilistic occupancy map when looking for thenext best view in the next subsections.
From the binary map σ , we can define the set χ of occupiedpoints, i.e. , the set of points x verifying σ ( x ) = 1 , its surface as the boundary ∂χ , and the surfacecoverage C ( c ) achieved by a camera pose c
= ( c pos , c rot ) ∈ C :
= R 3 × SO (3) as the followingsurface integral:
[Equation]
[Equation]
where | ∂χ | S :
= (cid:82) ∂χ
d x is the area of surface ∂χ .
χ c ⊂ χ is the subset of occupied points containedin the field of view of camera c , and v c ( x ) is the visibility of point x from camera c , i.e. , v c ( x ) {+=
+}
χ c ( x ) · 1 ( σ ( { (1 − λ ) c pos + λx such that λ ∈ [0 , 1) } )
= { 0 } ) .
Since we want to maximize the total coverage of the surface by all cameras during reconstruction,we are actually interested in maximizing the coverage of previously unobserved points rather thanthe absolute coverage.
Given a set of previous camera poses, which we call the camera history
H ⊂ C , and a 3D point x , we introduce the knowledge indicator γ H : R 3 → { 0 , 1 } such thatγ H ( x ) = max { v c ( x ) : c ∈ H } .
We then define the coverage gain G H ( c ) of camera pose c as:
[Equation]
where ν Hc ( x ) =
(1 − γ H ( x )) · v c ( x ) is the visibility gain of x in χ c , for camera history H .
Thisfunction is equal to 1 iff x is visible at pose c but was not observed by any camera pose in H .
Givena camera history H , our goal is to identify a pose c that maximizes G H ( c ) .
Given an occupancy map σ , we could evaluate the integral in Eq.
(2) by simply sampling points pon surface ∂χ .
However, in practice we will estimate the occupancy map iteratively in an unknownenvironment, and we will only have access to an occupancy probability distribution.
Extractingsurface points from such a probabilistic occupancy map gives results that can differ a lot from the truesurface: Indeed, in 3D, a surface acts as a very concentrated set with zero-measure, and requires highconfidence to give meaningful results.
Instead of extracting surface points, we extend the propertiesof such points to a small spherical neighborhood of the surface.
This will allow us to replace themaximization of a surface metric by the maximization of a volumetric integral, which is much easierto compute from our volumetric representation.
More exactly, we suppose that there exists a small quantity µ 0 > 0 such that any volume pointin the spherical neighborhood T ( ∂χ, µ 0 ) :
= (cid:8) p ∈ R 3 | ∃ x ∈ ∂χ, ∥ x − p ∥ 2 < µ 0 (cid:9) keeps the samevisibility property as its neighboring surface points.
With such a hypothesis, we give a thickness tothe surface, which makes sense when working with discrete points sampled in space to approximate avolume.
To this end, we introduce a new visibility gain function g Hc to adapt the definition of the formervisibility gain ν Hc .
For any 0 < µ < µ 0 :
[Equation]
where N is the inward normal vector field.
With further regularity assumptions about the surface thatare detailed in the supplementary material, such quantities are well defined.
Assuming µ 0 is small enough, the following explicit formula translates the surface approach into a volume integral for anycamera pose c ∈ C and µ < µ 0 :
[Equation]
with W x 0 the Weingarten map at x 0 , that is, the Hessian of the signed distance function on theboundary of χ , which is continuous on the scene surface, assumed to be compact [10].
By developing the determinant, we find that det( I − λW x 0 )
=
1 + λb ( λ, x 0 ) where b is a boundedfunction on the compact space [ − µ, µ ] × ∂χ .
Moreover, for all x 0 ∈ ∂χ , we have by definitiong Hc [Equation] µ [Equation]
+ [Equation] x [Equation]
= g [Equation] µ
[Equation] x 0 ) =
ν Hc ( x 0 ) when 0 ≤ λ < µ , and g Hc ( µ ; x 0 + λN ( x 0 )) [-=
-]=
0when − µ < λ < 0 .
It follows that, for every 0 < µ < µ 0 :
[Equation]
The complete derivations are given in the supplementary material.
Since the volume is supposed to be opaque, function g Hc ( µ ; · ) is equal to 0 for every point outside
T ( ∂χ, µ ) .
Moreover, considering the regularity assumptions we made on the compact surface, if µ 0is chosen small enough then for all x 0 ∈ ∂χ, µ < µ 0 , the point x 0 + µN ( x 0 ) is located inside thevolume, such that (cid:82) T ( ∂χ,µ ) g
Hc ( µ ; x ) d x = (cid:82) χ g Hc ( µ ; x ) d x .
Since | g Hc ( µ ; · ) | ≤ 1 for all c ∈ C and µ > 0 , we deduce the following theorem by bounding | b | on[ − µ, µ ] × ∂χ :
Theorem 1 Under the previous regularity assumptions on the volume χ of the scene and its surface∂χ , there exist µ 0 > 0 and M > 0 such that for all µ < µ 0 , and any camera c ∈ C : [Equation]
[Equation]
where | χ | V is the volume of χ .
This theorem states that, asymptotically for small values of µ , the volume integral (cid:82) χ g Hc ( µ [Equation] x ) d xgets proportional to the surface coverage gain values G H ( c ) that we want to maximize.
This result isconvenient since a volume integral can be easily approximated with Monte-Carlo integration on thevolume and a uniform dense sampling based on the occupancy function σ .
Consequently, the morepoints we sample in the volume, the smaller µ we can choose, and the closer maximizing the volumeintegral of spherical neighborhood visibility gain gets to maximizing the surface coverage gain.
2.2 Architecture
To approximate the volumetric integral in Equation 6 for any camera pose c , we need to computeχ c as well as function g Hc .
In this regard, we need to compute both the occupancy map and thevisibility gains of points for any camera pose.
Since the environment is not perfectly known, wepredict each one of these functions with a dedicated neural module.
The first module takes as inputthe partial point cloud gathered by the depth sensor to predict the occupancy probability distribution. The second module takes as input a camera pose, a feature representing camera history as well as apredicted sequence of occupied points located in the camera field of view to predict visibility gains.
Predicting the occupancy probability field ˆ σ .
The occupancy function σ is not known perfectlyin practice.
We thus approximate σ with the first module of our model, which consists of a deepneural network ˆ σ : P ( R 3 ) × R 3 → [0 , 1] that takes as inputs a partial point cloud P H ⊂ R 3 as wellas query points x ∈ R 3 , and outputs the occupancy probability ˆ σ ( P H ; x ) .
P H is obtained by mergingtogether all depth maps previously captured from cameras in H .
As shown in Figure 2, rather than using a direct encoding of the global shape of P H as input to ˆ σ ,we take inspiration from [9] to achieve scalability and encode P H using features computed fromthe points’ neighborhoods.
The difference with [9] is that we rely on a multiscale approach: For agiven query 3D point x , these features are computed from the k nearest neighbors of x computedat different scales.
For each scale s , we downsample point cloud P H around x into a sparser pointcloud P ( s ) H before recomputing the nearest neighbors p ( s ) i ( x ) , i = 1 , ..., k of x : In this way, the size of the neighborhood increases with scale.
Next, for each value of s , we use small attention units [26, 11] on the sequence of centered neighborhood points ( p ( s ) 1 ( x ) − x, ..., p ( s ) k
( x ) − x ) and apply pooling operations to encode each sequence ofk neighbors into a single feature that describes the local geometry for the corresponding scale.
Wefinally concatenate these different scale features with another uncentered global feature as well as thequery point x , and feed them to an MLP to predict the occupancy probability.
The last global featureaims to provide really coarse information about the geometry and the location of x in the scene.
This model scales well to large scenes: Adding points from distant views to the current partial pointcloud does not change the local state of the point cloud.
To avoid computing neighborhoods on theentire point cloud when reconstructing large scenes, we partition the space into cells in which westore the points in P H .
Then, given a query point x , we only use the neighboring cells to computep ( s ) i ( x ) , i = 1 , ..., k .
Predicting the visibility gain g Hc .
To maximize the surface coverage gain, we need to compute thevolumetric integral of the visibility gain functions g Hc .
We do this again by Monte Carlo sampling,however, in unknown environments we cannot compute explicitly occlusions to derive visibility gainfunctions g Hc since the geometry, represented as a point cloud, is partially unknown and sparser thana true surface.
We thus train the second module of our model to predict the visibility gain functionsby leveraging a self-attention mechanism that implicitly helps to estimate occlusion effects in thediscrete point cloud P H .
In particular, for any camera pose c ∈ C and 3D point x ∈ χ c , the second module derives itsprediction of visibility gains from three core features: (i)
The predicted probability ˆ σ ( P ; x ) of x tobe occupied, (ii) the occlusions on x by the subvolume χ c and (iii) the camera history H .
To feedall this information to our model in an efficient way, we follow the pipeline presented in Figure 3.
The model starts by using the predicted occupancy probability function ˆ σ to sample 3D points in thevolume χ .
These samples will be used for Monte Carlo integration.
We refer to these points as proxypoints as we use them to encode the volume in the camera field of view, i.e. , in a pyramidal frustumview.
We write ˆ χ as the discrete set of sampled proxy points, and ˆ χ c as the set of proxy points locatedin the field of view of the camera c .
We first encode these proxy points individually by applying a small MLP on their 3D coordinates andtheir occupancy probability value concatenated together.
Then, our model processes the sequenceof these encodings with a self-attention unit to handle occlusion effects of subvolume χ c on everyindividual point.
Note there is no pooling operation on the output of this unit: The network predictsper-point features and does not aggregate predictions, since we do it ourselves with Monte Carlointegration.
Next, for each proxy point x ∈ ˆ χ , we compute an additional feature h H ( x ) that encodesthe history of camera positions H with respect to this point as a spherical mapping: It consists inthe projection on a sphere centered on x of all camera positions for which x was in the field of view.
These features are concatenated to the outputs of the self-attention unit.
Our model finally uses an MLP on these features to predict the entire visibility gain functions ofevery point x as a vector of coordinates in the orthonormal basis of spherical harmonics.
With such aformalism, the model is able to compute visibility gains for points inside a subvolume in all directionswith a single forward pass.
In this regard, we avoid unnecessary computation and are able to processa large number of cameras in the same time when they share the same proxy points in their field ofview ( e.g. , reconstruction of a single object centered in the scene, where ˆ χ c
= ˆ χ for all c , or whenseveral cameras observe the same part of the 3D scene, i.e. , ˆ χ c
= ˆ χ ′ ⊂ ˆ χ for several c ).
Formally, if we denote by Y ml : S 2 → R the real spherical harmonic of rank ( l, m ) andϕ ml (ˆ χ c ; x, h H ( x ))
the predicted coordinate of rank ( l, m ) for proxy point x ∈ ˆ χ c with attention tosubset ˆ χ c and camera history feature h H ( x ) , the visibility gain of point x in direction d ∈ S 2 isdefined as
[Equation]
so that the coverage gain G H ( c ) of a camera pose c ∈ C is proportional to
[Equation]
The next best view among several camera positions is finally defined as the camera pose c ∗ with thehighest value for I H ( c ∗ ) .
Equation 8 is a Monte Carlo approximation of the volumetric integral in
Equation 6, where the occupancy map and the visibility gains are predicted with neural networks.
We also use spherical harmonics to encode the camera history encoding h H ( x ) of each point x , whichmakes it a homogeneous input to the predicted output.
Consequently, this input comes at the end ofthe architecture, and aims to adapt the visibility gain according to previous camera positions.
Thisconvenient representation, inspired by [32], allows us to handle free camera motion; On the contrary,several models in the literature encode the camera directions on a discrete sphere [33, 16, 25].
Training.
We train the occupancy probability module alone with a Mean Squared Error loss withthe ground truth occupancy map.
We do not compute ground-truth visibility gains to train thesecond module since it would make computation more difficult and require further assumptions:
Wesupervise directly on I H ( c ) by comparing it to the ground-truth surface coverage gain for multiplecameras, with softmax normalization and Kullback-Leibler divergence loss.
Extensive details aboutthe training of our modules and the choices we made are given in the supplementary material.
[Table]
Experiments
As discussed in the introduction
[Table]
deep learning-based NBV models for dense reconstruction arecurrently limited to single, small-scale, centered object reconstruction.
To compare SCONE to theseprevious methods in this context
[Table]
we first constrain the camera pose to lie on a sphere centered onan object.
We then introduce our dataset made of large-scale 3D models that we created to evaluate
SCONE on free camera motions.
3.1 Next Best View for Single Object Reconstruction
We first compare the performance of our model to the state of the art on a subset of the ShapeNetdataset [2] introduced in [33] and following the protocol of [33]: We sample 4,000 training meshesfrom 8 specific categories of objects, 400 validation meshes and 400 test meshes from the samecategories, and 400 additional test meshes from 8 categories unseen during training.
The evaluation on the test datasets consists of 10-view reconstructions of single objects.
Given amesh in the dataset, camera positions are discretized on a sphere.
We start the reconstruction processby selecting a random camera pose, then we iterate NBV selection 9 times in order to maximizecoverage with a sequence of 10 views in total.
The evaluation metric is the area under the curve (AUC)of surface coverage throughout the reconstruction.
This criterion not only evaluates the quality offinal surface coverage, but also the convergence speed toward a satisfying coverage.
Results arepresented in Table 1.
Further details about the evaluation and the metric computation are available inthe supplementary material.
3.2 Active View Planning in a 3D Scene
To evaluate the scalability of our model to large environments as well as free camera motion in 3Dspace, we also conducted experiments using a naive planning algorithm that incrementally builds apath in the scene:
For this algorithm, we first discretize the camera poses in the scene on a 5D grid,corresponding to coordinates c pos = ( x c , y c , z c ) of the camera as well as the elevation and azimuthto encode rotation c rot . The number of poses depends on the dimensions of the bounding box of thescene.
This bounding box is an input to the algorithm, as a way for the user to tell which part of thescene should be reconstructed. In our experiments, the number of different poses is around 10,000. [Figure]
Liberty
Bridge
Castle
Castle
Mean
Random Walk 0.
SCONE-Entropy 0.
SCONE 0.
[Figure]
Note we did not retrain our model for such scenes and use the previous model trained on ShapeNet asexplained in Section 3.1.
The depth sensor starts from a random pose (from which, at least, a part of the main structure toreconstruct is visible).
Then, at each iteration, our method estimates the coverage gain of directneighboring poses in the 5D grid, and selects the one with the highest score.
The sensor moves to thisposition, captures a new partial point cloud and concatenates it to the previous one.
Since we focuson online path planning for dense reconstruction optimization and designed our model to be scalablewith local neighborhood features, the size of the global point cloud can become very large.
We iteratethe process either 100 times to build a full path around the object in around 5 minutes on a single Nvidia GTX1080 GPU, and recovered a partial point cloud with up to 100,000 points.
Contrary to single, small-scale object reconstruction where the object is always entirely includedin the field of view, we simulated a limited range in terms of field of view and depth for the depthsensor, so that the extent of the entire scene cannot be covered in a single view.
Thus, taking picturesat long range and going around the scene randomly is not an efficient strategy; the sensor must find apath around the surface to optimize the full coverage.
We use a ray-casting renderer to approximate areal Lidar.
More details about the experiments can be found in the supplementary material.
We compared the performance of SCONE with simple baselines:
First, a random walk, which choosesneighboring poses randomly with uniform probabilities.
Then, we evaluated an alternate versionof our model, which we call SCONE-Entropy, which leverages the first module of our full modelto compute occupancy probability in the scene, then selects the next best view as the position thatmaximizes the Shannon entropy in its field of view.
The comparison is interesting, since SCONEEntropy adapts classic NBV approaches based on information theory to a deep learning framework.
Quantitative results are available in Table 5(a).
Despite being trained only on ShapeNet 3D models, our method is able to compute meaningful pathsaround the structures and consistently reach satisfying global coverage.
Since we focused on buildinga metric for NBV computation, this experiment is simple and does not implement any further priorfrom common path planning strategies: For instance, we do not compute distant NBV, nor optimalpaths to move from a position to a distant NBV with respect to our volumetric mapping.
There is nodoubt the model could benefit from further strategies inspired by the path planning literature.
[Figure]
Kullback
[Figure]
Divergence
[Figure]
3.3 Ablation Study
We now provide an ablation study for both modules of our full model:
The prediction of occupancyprobability, and the computation of coverage gain from predictions about visibility gains.
To evaluateboth modules separately, we compared their performance on their training critera: Mean-Squared
Error for occupancy probability prediction, and softmax followed by Kullback-Leibler divergenceon a dense set of cameras for coverage gain.
The values are reported in Figure 6.
We also providefurther analysis in the supplementary material.
Occupancy probability.
Apart from the base architecture presented in Figure 2, we trained theoccupancy probability prediction module under two additional settings.
First, we removed the multiscale neighborhood features computed by downsampling the partial point cloud P H , and trained themodule to predict an occupancy probability value ˆ σ only from global feature g
( P H ) and encodingf ( x ) .
As anticipated, the lack of neighborhood features not only prevents the model from an efficientscaling to large scenes, but also causes a huge loss in performance.
We also trained a slightly morecomplex version of our occupancy probability prediction module by feeding it the spherical mappingsh H ( x ) of camera history H as an additional feature.
It appears this helps the model to increaseits confidence and make better predictions, but the gains are quite marginal.
Extensive details areprovided in the supplementary material.
Visibility gain.
We trained two variations of our visibility gain prediction module.
In the firstone, we completely removed the geometric prediction: We use directly the surface points as proxypoints, mapped with an occupancy probability equal to 1.
As a consequence, the model suffersfrom a significant loss in performance to compute coverage gain from its predicted visibility gainfunctions.
Thus, we can confirm that the volumetric framework improves the performance of SCONE.
This result is remarkable, since models that rely on surface point clouds usually make better NBVpredictions for dense reconstruction of detailed objects in literature.
On the contrary, we show thatour formalism is able to achieve higher performance by leveraging a volumetric representation witha high resolution deep implicit function ˆ σ .
We trained a second variation without the sphericalmappings h H of camera history.
We verified that such additional features increase performance,especially at the end of a 3D reconstruction.
Limitations and Conclusion
Beyond the prediction of the Next Best View, our method SCONE is able to evaluate the value of anycamera pose given a current estimate of the scene volume.
We demonstrated this ability with a simplepath planning algorithm that looks for the next pose around the current pose iteratively.
However:
• Like current methods, SCONE relies on a depth sensor and assumes that the captured depth maps
are perfect.
In practice, such a sensor is not necessarily available and can also be noisy.
It would
be very interesting to extend the approach to color cameras.
• Also like current methods, it is limited to the prediction of the camera pose for the next step.
This
is a greedy, non-necessarily optimal as multiple camera poses are required anyway for a complete
3D reconstruction.
We believe that, thanks to its scalability, our method could be extended to the
prediction of the “Next Best Path”, where the future camera poses would be predicted jointly for
their complementarity.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See the last section.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] Adiscussion can be found in the supplementary material.(d)
Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes]3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[No] Not yet, but we will release our source code and dataset. (b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?
[Yes]
In the supplementary material.
(c) Did you report error bars (e g., with respect to the random seed after running experi- ments multiple times)?
[No] (d) Did you include the total amount of compute and the type of resources used (e g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See supplementary material.4.
If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
(a)
If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No](d) Did you discuss whether
and how consent was obtained from people whose data you’re
using/curating?
[Yes]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[No] Our data do not have such issue.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]