FOF: Learning Fourier Occupancy Field for Monocular Real-time Human Reconstruction
Abstract
The advent of deep learning has led to significant progress in monocular humanreconstruction.
However, existing representations, such as parametric models,voxel grids, meshes and implicit neural representations, have difficulties achieving high-quality results and real-time speed at the same time.
In this paper, wepropose Fourier Occupancy Field (FOF) , a novel powerful, efficient and flexible3D geometry representation, for monocular real-time and accurate human reconstruction.
The FOF represents a 3D object with a 2D field orthogonal to the viewdirection where at each 2D position the occupancy field of the object along theview direction is compactly represented with the first few terms of Fourier series,which retains the topology and neighborhood relation in the 2D domain.
A FOFcan be stored as a multi-channel image, which is compatible with 2D convolutionalneural networks and can bridge the gap between 3D geometries and 2D images.
FOF is very flexible and extensible, e.g ., parametric models can be easily integratedinto a FOF as a prior to generate more robust results.
Based on FOF, we design thefirst 30+FPS high-fidelity real-time monocular human reconstruction framework.
We demonstrate the potential of FOF on both public dataset and real captured data.
The code will be released for research purposes.
[Table]
Introduction
3D human reconstruction from a single image is very popular in computer vision and graphics, whichcan be widely used in various applications, such as mixed reality, virtual try-on, and “metaverse”.
However, high-fidelity monocular human reconstruction in real-time remains a challenging task.
3Drepresentation is at the core of this problem and determines the design and performance of approach.3D geometry representation is also the key for 3DTV and Holographic Telepresence systems, whichneed to fulfill the requirements of accuracy, efficiency, compatibility and generality [1].
Parametric human models, e.g ., SMPL [17] and SMPL-X [20], represent a 3D human body withparameters.
Their results are robust but they are unable to reconstruct the clothes.
Many classicrepresentations, such as voxel grids [31] or meshes [33, 2, 14], have been exploited for monocularhuman reconstruction.
However, the representation of voxel grids has to store discrete spatial gridsamplings in memory.
It has a space complexity of O ( n 3 ) (where n is the number of grids in eachdimension), which limits the achievable spatial resolution.
Meshes on the other hand are a compactrepresentation for surface geometry.
However, it is challenging for meshes to cope with change ofconnectivity, and as a result, they may produce unstable or over-smoothed results when there arechange of topology and/or substantial deformations.
Recently, implicit neural representations have emerged and been widely used in monocular humanreconstruction [22, 23, 30, 9, 3, 27].
The 3D space is treated as a continuous field, such as anoccupancy field and signed distance field, which can be formulated as R 3 → R : F ( x, y, z ) .
Ratherthan sampling the field and store it as a voxel grid, a neural network is used to fit the field.
Thus, theresults can be produced at any resolution.
However, human geometry is not explicitly reconstructed,and the network has to infer values for a huge number of spatial grid sample points to extract thegeometry.
The number of sampled points in inference is limited by available GPU memory, and thesampled points have to be forwarded in several batches, which is very time-consuming.
Therefore,it is very difficult to implement high frame rate real-time reconstruction based on implicit neuralrepresentations.
Although Monoport [15], as a real-time method, renders the mesh-free implicit fieldat 15FPS with an efficient sampling scheme, it does not reconstruct the geometry explicitly.
Thus, itis not compatible with traditional graphics applications.
In this paper, we propose a new representation, Fourier Occupancy Field (FOF) , for monocularreal-time and accurate human reconstruction, which is an expressive, efficient and flexible 3D objectrepresentation in the form of a 2D map aligned with the input image.
The key idea of FOF is torepresent a 3D object with a 2D field by representing the occupancy field along the z -axis into Fourierseries and only keeping the first few terms, which is memory-efficient and can preserve most of thegeometry information.
With grid sampling on the x -axis and y -axis, the FOF can be stored explicitlyas a multi-channel image, which is compatible with the existing CNN frameworks and suitable for3DTV applications.
Unlike depth maps, our FOF can represent the complete geometry rather thanjust the visible part, and it is very efficient to produce high-fidelity human shapes.
Moreover, highresolution meshes and our FOF can be easily inter-converted.
Detailed comparisons with existingrepresentations are given in Table 1.
Based on our FOF, we design a simple and efficient pipeline for monocular real-time human reconstruction, which is end-to-end and easy to train.
Some live captured examples are shown in Fig.1.
Experimental results on both public dataset and real captured data show that our approach canreconstruct human meshes accurately and robustly in real-time.
[Figure]
To summarize, the main contributions of our work include:
• We propose Fourier Occupancy Field , a novel representation for 3D humans, which canrepresent a high-quality object geometry with a 2D map aligned with the image.
It can
bridge the gap between 2D images and 3D geometries.
• We present a simple and efficient pipeline for monocular real-time 3D human reconstruction,
which is end-to-end and easy to train.
Parametric models can be optionally transformed into
a Fourier occupancy field as a prior to produce more robust results.
• Compared with SOTA methods based on other representations, our approach can produce
more realistic results in real-time.
Moreover, it is the first 30+FPS pipeline, which is 2 times
faster than Monoport [15] with better results.
Related work
We briefly overview literature on monocular human reconstruction using a single color camera.
Surface-based reconstruction.
This kind of approaches focuses on the interface between geometryand empty space and can be broadly classified into three categories: parametric models, UV-mapbased methods, and graph-based methods.
Parametric models, such as SMPL [17], SMPL-X [20] and
SRAR [19] are popular representations for 3D human.
They collect a large amount of naked humanshapes, and get an analytical model on them with statistical methods, which can generate a humanbody mesh with dozens of parameters.
Based on this, a lot of works [12, 29, 13] estimate 3D humanshapes from an RGB image by predicting the parameters of the parametric model, but these methodscannot reconstruct the clothes or hair.
An overview of 3D human shape estimation from a singleimage can be found in Tian et al .
Other methods try to estimate clothed humans by adjustingvertices of the parametric models.
Alldieck et al .
[2] warp the input image to align with the UVmap, and estimate the displacements on the T-pose SMPL mesh.
Their results are always in T-poseand cannot match the input images well.
Li et al .
[14] reconstruct the geometry of a clothed humanbody with graph neural networks, but the recovered mesh tends to be smooth and lacks fine details.
Zhu et al .
[32, 33] use four stages constrained by joints, silhouettes and shading, but their resultsare sometimes inconsistent with the images.
It is very difficult to represent 2D manifold surfacesof different 3D objects with a fixed structure, and hence most methods above rely heavily on thetopology defined by SMPL and cannot produce detailed human geometries with complex topology.
Habermann et al .
[7, 8] propose a deep learning approach for monocular dense human performancecapture, but this method needs to scan the actor with a 3D scanner as the template mesh.
Graph-basedmethods cannot get exact features aligned with the results, and hence the results are unstable andover-smoothed.
Volume-based reconstruction.
Volume-based methods estimate the attributes, e.g ., occupancy andsigned distance, for each point in the 3D space and can represent 3D shapes with arbitrary topology.
Voxel girds and implicit neural networks are two main representations used in volume-based methods.
Zheng et al .
[31] regress the voxel grids of 3D human using convolutional neural networks, but thiskind of methods requires intensive memory and has limited resolution of the model.
Unlike voxel grids, implicit neural representation can represent detailed 3D shapes without resolution limitations.
Saito et al .
[22] use pixel-aligned function to reconstruct 3D human from a single RGB image.
Theyfurther use normal maps to significantly improve the 3D geometric details [23].
Zheng et al .
[30]use the voxelized SMPL mesh as a prior to make the results more robust.
Xiu et al .
[27] propose amethod to correct SMPL model estimated by other methods [29, 13, 6] according to the input image,and then regress occupancy field from the normal map and the signed distance field of the corrected
SMPL.
Their results are robust to human poses, but the geometric details do not match the inputimages and are noisy.
The common problem of the methods based on implicit neural representationis that they are computationally heavy and very time-consuming.
Although Li et al .
[15] propose anefficient sampling scheme to speed up the inference, they can only achieve mesh-free rendering at15FPS.
When generating a complete mesh, their method will be less than 10FPS.
In this paper, we contribute the first real-time high-quality human reconstruction system working at30FPS with a single RGB camera, benefiting from the proposed efficient and flexible 3D geometryrepresentation FOF.
Method
The goal of our work is to reconstruct a high-fidelity 3D human model from a single RGB imagein real-time.
To this end, we propose an efficient and flexible 2D representation of 3D geometry(Sec. 3.1), named Fourier Occupancy Field (FOF) , as shown in Fig.
In the proposed FOF-basedreconstruction framework, we first learn the FOF representation for the input image by an image-toFOF mapping (Sec. 3.2), and then reconstruct the 3D human model by the efficient Fourier inversionwith low complexity (Sec. 3.3).
3.1 Fourier Occupancy Field
Without loss of generality, 3D objects to be reconstructed are normalized into a [ − 1 , 1] 3 cube.
Denoted by ( x, y, z ) ∈ [ − 1 , 1] 3 the 3D coordinates, and by S the surface of a 3D object.
The 3Dobject can be described by a 3D occupancy field F : [ − 1 , 1] 3 (cid:55)→ { 0 , 0 .
5 , 1 } , where the occupancyfunction F ( x, y, z ) is defined as
[Equation]
the object
[Equation]
the object
[Equation]
Such a 3D representation is highly redundant, because only a small subset, i.e ., the iso-surface of
F ( x, y, z ) with value of 0.5, is sufficient to represent the surface of the object.
Direct inference ofthe 3D occupancy field F from a single image not only confronts with the curse of dimensionality,but also requires more computation and memory in handling high dimensional feature maps.
Notethat the occupancy field defined in the 3D cube can be regarded as the collection of a large numberof 1D signals defined on the lines along the view direction.
Without loss of generality, we assumethat the view direction is the same as the z -axis of the 3D cube.
The occupancy function, as a 1Dsignal, along the line passing through a particular point ( x ∗ , y ∗ ) ∈ [ − 1 , 1] 2 on the xy -plane can bewritten as f ( z ) :
[ − 1 , 1] (cid:55)→ F ( x ∗ , y ∗ , z ) .
We explore Fourier representation for such a family of 1Doccupancy signals, and propose a more compact 2D vector field orthogonal to the view direction,namely FOF, for more efficient representation of the 3D occupancy field, which is detailed in thefollowing subsections.
Fourier series on a single occupancy line.
1D occupancy signals { f ( z ) } are essentially on-offsignals switching at the boundaries of human bodies, which lie in a low-dimensional manifold in theambient signal space.
For accurate and fast 3D reconstruction, the compact representation shouldhave the following merits:
Sampling Scalability : The representation should be able to adapt to different samplingrates in the inference stage without introducing systematic mismatch besides the inherent
approximation error due to sampling.
Without retraining, systems with such a representation
would support diverse applications with different resolution, quality, and speed requirements.
Stable Reconstruction : The representation should provide stable reconstruction quality for
diverse humans with various parameters such as garments, heights, weight, and poses.
Low-complexity Reconstruction : The representation should allow low-complexity recon- struction to support emerging real-time applications such as Holographic Telepresence.
However, most popular representation do not enjoy all the above merits.
For instance, discrete Fouriertransform and discrete wavelet transform have stable and low-complexity reconstruction, but onlysupport the sampling rate used in preparation of the training set due to their discrete nature.
Learneddictionaries are able to provide sparse representation, but require computation-demanding iterativereconstruction, leaving alone the sampling scalability.
Without doubt, sampling scalability is the mostdifficult property to meet.
Pushing the concept of sampling scalability to the limit, one representationwould be scalable to arbitrary sampling rates in theory if it allows continuous reconstruction alongthe z -axis.
Note that the representation should stay discrete and compact for efficient inference.
Sucha bridge of discrete-representation and continuous-reconstruction motivates us to use Fourier seriesfor the representation of 1D occupancy signals { f ( z ) } .
By nature, Fourier series would have infinitenumber of discrete coefficients, which is impractical.
Note that most energy of 1D occupancy signalsconcentrate on only a few low-frequency terms, which yields a compact representation by subspaceapproximation.
Formally, let f p ( z ) be periodic extension of f ( z ) .
Note that f p ( z ) satisfies the Dirichlet conditions:1)
f p z ) is absolutely integrable over one period; 2) f p ( z ) has a finite number of discontinuitieswithin one period; 3) f p ( z ) has a finite number of extreme points within one period.
Thus, f p ( z ) canbe expanded as a convergent Fourier series:
[Equation]
where { a n } , { b n } ∈ R are coefficients of basis functions { cos( nx ) } and { sin( nx ) } , respectively.
Note that by defining the occupancy function as Eq.
(1), Eq.
(2) holds for discontinuity at z ∗ onthe surface S because f ( z ∗ ) =
( f ( z ∗ − )
+ f ( z ∗ +)) / 2 = 0 .
5 , where f ( z ∗ − ) and f ( z ∗ +) are theleft hand limit and right hand limit of f ( z ) at z ∗ .
For compact representation, we approximate 1Doccupancy signals by a subspace spanned by the first 2 N + 1 basis functions:
[Equation]
where b ( z ) =
[1 / 2 , cos( z ) , sin( z ) , . .
, cos( Nz ) , sin( Nz )] ⊤ is the vector of the first 2 N + 1 basisfunctions spanning the approximation subspace, and
c =
[ a 0 , a 1 , b 1 , . .
. , a N , b N ] ⊤ is the coefficientvector which provide a more compact representation of the 1D occupancy function f ( z ) .
In ourimplementation, N is chosen as 15, which is accurate enough for most 3D human geometries.
Fourier occupancy field.
Then, such a Fourier subspace approximation is applied to all 1Doccupancy signals over the xy -plane.
[Equation]
where C ( x, y ) is the ( 2 N + 1 )
D Fourier coefficient vectors for the 1D occupancy signals at ( x, y ) .
Inthis way, we obtain the Fourier Occupancy Field C : [ − 1 , 1] 2 (cid:55)→ R 2 N +1 for 3D occupancy field F .
Note that 3D coordinates ( x, y, z ) in Eq.
(4) are continuous, and can be sampled as needed.
Particularly, unlike the discrete Fourier transform or discrete wavelet transform, the subspace dimensionfor 1D occupancy signals { f ( z ) } is not coupled with the sampling grid of z -axis.
This would avoidpotential sampling mismatch between the sampling grid used in the training data and that in thetesting stage.
The reconstruction of ˆ F from the FOF C is sampling scalable in the sense that samplingrate along the z -axis can be chosen as necessary, e.g ., according to the requirement of reconstructionquality and speed.
3D models at any resolution can be readily obtained from the FOF representationby simply alter the sampling rate along the z -axis.
Moreover, with sampling on xy -plane, e.g ., alignedwith the sampling grid of the input image of size H × W , the FOF C is a multi-channel image ofsize H × W × (2 N + 1) , which is more compact than the 3D occupancy field and is friendly fornetwork prediction as detailed in the following subsection.
3.2 Learning FOFs with neural networks and variants
Network.
Thanks to the efficient dimensionality reduction with subspace approximation, the taskof FOF learning from an input image is essentially to learn an image-to-image network.
To this end,we exploit the network standing on the HR-Net [26] framework for its outstanding fitting capabilityin various vision tasks.
We use the weak perspective camera model in our implementation, in which
FOF (and thus the reconstructed geometry) is naturally aligned with the input RGB image.
Notethat perspective camera model can also be used by using the normalized device coordinate space(NDC space).
Note that the proposed FOF representation is scalable in terms of sampling along notonly the z -axis (discussed in Sec. 3.1), but also the x - and y -axes.
For example, for the input imageof size W × H , we learn the FOF of size W × H × (2 N + 1) .
To reconstruct a 3D shape of size
W ′ × H ′ × K , we simply resize the learned FOF to the size of W ′ ×
H ′ × (2 N +1) along the xy -plane,and specify the number of sampling points K for the z -axis at the Fourier inversion.
Such a scalablereconstruction of 3D shapes is a significant departure from scaling-after-reconstruction strategiesthat involve complex 3D manipulation, and is more analogous to efficient image resizing.
Withoutrequiring complex accommodation, our method can be readily deployed in various applications withheterogeneous computation resources.
Variants.
Our FOF is also highly extensible.
Here we introduce two variants of the reconstructionpipeline, named FOF-SMPL and FOF-Normal.
In FOF-SMPL, we use the SMPL parametric modelas a prior to achieve more robust reconstruction, inspired by PaMIR [30].
Specifically, we transforma SMPL mesh to a FOF with the algorithm in Sec. 3.3, and concatenate it with the RGB image as theinput together.
This makes the network more robust to different poses and improves reconstructionquality for cases with high ambiguity.
In FOF-Normal, we use normal maps to enhance the results.
Specifically, we infer the front and back normal maps with the pix2pix network [11] for the inputimage, and also concatenate them with the RGB image together as input.
Note that we use inferrednormal maps, instead of ground-truth, in the training stage.
The results of the two variants arereported and analyzed in Sec. 4.3.
Loss.
We use the L1 loss is to train our FOF baseline and variants.
To make the network morefocused on the human geometry, we only supervise the human foreground region of the image.
Givena training set { ( I i , C i ) } mi =1 , the loss function L is formulated as:
[Equation]
where M is the set of foreground pixels, and ˆ C ( x, y ) is the FOF predicted by the network.
3.3 Transformation between FOF and mesh
FOF to mesh.
Extracting meshes is necessary for applications such as mesh-based rendering andanimation.
Recent methods based on implicit neural representations need to run a network on a 3Dsampling grid.
Instead, as described in Eq.
(4), the reconstruction of 3D occupancy field from FOF issimply a multiplication of two tensors.
Then, the 3D mesh is extracted from the iso-surface of the3D occupancy at the threshold of 0.5 with the Marching Cubes algorithm
Both steps can beparallelized on GPUs for fast mesh generation.
Mesh to FOF.
To prepare training data, 3D meshes should be transformed into FOFs as traininglabels.
By definition Eq.
(6), we are to calculate the first 2 N + 1 coefficients of the Fourier seriesfor each 1D occupancy signal f ( z ) .
For generic signals, this would require numerical integrationover a sampled version of f ( z ) , which, however, may introduce numerical errors and demandsconsiderable computation.
Fortunately, thanks to the particular form of occupancy signals, the Fouriercoefficients can be derived analytically with exact solutions.
Note that the 1D occupancy signalf ( z ) is actually defined by discontinuities, where the line goes inside or outside the mesh.
Weextract the discontinuous points of f ( z ) by designing a rasterizer-like procedure (See details in thesupplementary material).
Formally, suppose the line associated with f ( z ) passes though the meshfor k times.
For the i th traversing, let z i and z ′ 1 be the locations going inside and outside the mesh,respectively.
Then, the set Z ≜ { ( z i , z ′ i ) } ki =1 collects those intervals of the line that are inside the mesh.
According to the definition of the occupancy signal, we have: f ( z ) = 1 , z ∈ Z , and f ( z ) = 0otherwise except for a finite number of discontinuities with measure zero.
Therefore, we have thefollowing analytical solution for the Fourier coefficients.
[Equation]
The FOF representation is obtained by calculating the Fourier coefficients for all the 1D occupancysignals.
Since the number of intervals k is generally small, e.g ., 1 ∼ 3 for human meshes, thecalculation of a n or b n is of O (1) complexity.
Thus, the transformation from discontinuities to FOFhas a computational complexity of O ( HWN ) , and can be parallelized on GPUs for acceleration.
Experiments
4.1 Datasets and metrics
Datasets.
We collect 2038 high-quality human scans from Twindom 1 and THuman2.0 [28] totrain and evaluate our method.
Twindom contains 1512 scans with various clothing, but their posesare simple and lack diversity.
THuman2.0 contains 526 scans with more variety in body poses.
Werandomly select 1059 from Twindom and 368 from THuman2.0 as the training set, and 302 from
Twindom and 105 from THuman2.0 as the test set.
The remaining subjects are used as the validationset.
We render 256 views images with 4 lights randomly selected from 240 lights for each scan.
All the images are rendered with a weak perspective camera and pre-computed radiance transfer(PRT[24]) renderer.
To obtain the corresponding SMPL model, which is used for FOF-SMPL and
ICON [27], we use OpenPose [5] to detect 2D keypoints and generate SMPL model with a multi-viewversion of SMPLify-X [20].
Metrics.
We use Chamfer distance, P2S
(point-to-surface) distance, and normal image error forevaluation.
Chamfer distance is defined as:
[Equation]
where S pr is a set of points on the predicted mesh surface, and S gt is a set of points on the groundtruth mesh surface.
We average the Chamfer distances of all test samples, and report the squareroot of half of it.
We also measure the average point-to-surface distance from the vertices on thereconstructed surface to the ground truth.
To measure the visual quality, we render the reconstructedgeometries as normal maps from the view direction, and compute L1 loss with the ground truth.
4.2 The real-time pipeline
To perform real-time monocular human reconstruction, we design a three-stage pipeline.
In the firststage, we get an image from the video stream with OpenCV [4], and use RVM [16] to get a humanmask to remove the background.
RVM is originally a matting method, and we use the thresholdof 0.5 to obtain a binary mask.
In the second stage, the FOF of the human is estimated by our
[Table]
[Figure]
network in Sec. 3.2.
In the final stage, the FOF is resized to a proper resolution ( 256 × 256 in our implementation), and transformed to a mesh as introduced in Sec. 3.3.
We also render the mesh intoimages with PyTorch3D [21] in the final stage to preview the reconstructed geometry in real time.
Our three stages are all implemented with PyTorch and running on a single RTX-3090 GPU.
Ourpipeline works at 30FPS which is limited by the camera.
With a high-frame-rate camera, TensorRTand more GPUs, our frame rate can be further improved.
Fig.
3 gives some real-time reconstructionresults.
Please see demo video for more details.
4.3 Comparison
We compare our method with three state-of-art approaches, PIFu [22], PIFuHD [23] and ICON [27],together with our two variants, FOF-Normal and FOF-SMPL.
We retrain PIFu on our dataset.
Othermethods are used with the pre-trained checkpoints, because they do not provide the training codes.
We use 512 × 512 × 512 resolution for all these methods.
We do not compare our method with [3]and [10] because they do not release their codes.
For simplicity, we also ignore comparisons withsome works that have already been compared like PaMIR [30].
We compare our method with twoversions of ICON.
In table 2, ICON is the original version proposed in their paper, and ICON filter isprovided in their code, which is visually better but may have slightly worse metrics.
Quantitative results.
Table 2 gives Chamfer distances, P2S distances, and normal image errorson the test set.
Ground-truth SMPL meshes are used in FOF-SMPL and ICON [27].
Because of theweak perspective camera, the results of all the methods may not be aligned with the ground-truthson the view direction.
Thus, we randomly sample 400000 points on the estimated mesh and theground-truth mesh, and align them by making their z -coordinates have the same mean.
Our resultshave the lowest reconstruction errors compared with others, is slightly worse than PIFuHD in termsof the normal image error, coming in the second place.
Qualitative results.
Fig.
4 and Fig.
5 show some visual results for challenging poses and variousshapes, respectively.
As shown in the figures, our method is robust to various poses and shapes, andachieves detailed and reasonable reconstruction.
4.4 Ablation study
We show the effect of different N on the accuracy of the approximation in Fig.
When N ≤ 12 ,obvious artifacts appear on the meshes.
While the setting of N = 15 preserves most geometricdetails, which is used in our implement.
[Figure] [Figure]
Conclusion and discussion
Conclusion.
We propose a novel efficient and flexible representation, Fourier Occupancy Field(FOF), for monocular real-time human reconstruction.
FOF is aligned with the input image andcan be estimated with a simple image-to-image network.
Moreover, FOF can represent a complex3D shape in the form of a multi-channel image, while preserving most geometric details, whichis computationally efficient to extract the mesh and suitable for 3DTV applications.
Experimentalresults indicate that our method can infer high-fidelity human models in real-time with state-of-the-artreconstruction quality.
Limitations.
The FOF representation is based on the Fourier series expansion of the square-wavelike function.
When the duty cycle of the square-wave function is small, the spectrum of the functionwill contain many high-frequency components.
In such cases, we need more terms in the Fourierseries to approximate the function.
The number of terms we need is roughly inversely proportionalto the duty cycle.
Therefore, FOF cannot represent the objects that are too thin.
In future work, wewill try wavelet transformation for better representation capability and apply our representation tocommon objects.
Broader Impact.
The proposed method will promote the development of avatar generation and3DTV, which is useful for applications that require accurate and efficient 3D reconstruction.
However,this efficient acquisition may cause privacy and ethical problems.
We suggest policymakers toestablish an efficient regulatory system and inform users about potential risks to avoid the disclosureof personal information.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Section4.1.
Some are in the supplemental material.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] In the supplemental material.4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a)
If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[Yes]
In the supplemental material.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[Yes]
In the supplemental material.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if [Table] applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
[Table]
spent on participant compensation?
[N/A]
[Table]