Neural Transmitted Radiance Fields
Anonymous Author(s)
AffiliationAddress email
Abstract
Neural radiance fields (NeRF) have brought tremendous progress to novel viewsynthesis.
Though NeRF enables the rendering of subtle details in a scene bylearning from a dense set of images, it also reconstructs the undesired reflectionswhen we capture images through glass.
As a commonly observed interference, thereflection would undermine the visibility of the desired transmitted scene behindglass by occluding the transmitted light rays.
In this paper, we aim at addressingthe problem of rendering novel transmitted views given a set of reflection-corruptedimages.
By introducing the transmission encoder and recurring edge constraints asguidance, our neural transmitted radiance fields can resist such reflection interference during rendering and reconstruct high-fidelity results even under sparse views.
The proposed method achieves superior performance from the experiments on anewly collected dataset compared with state-of-the-art methods
Introduction
Novel view synthesis plays a vital role in various computer vision applications.
Tremendous progresshas been witnessed recently with the introduction of neural radiance fields (NeRF) [1].
By modelingthe emitted radiance and density of a scene from a set of images with a multilayer perceptron (MLP),it demonstrates an unprecedented level of fidelity on a range of challenging scenes.
Despite its successin diverse scenarios, such a framework still faces challenges when working with glass, a mediumwhich is frequently observed in photography and daily life [2] to provide see-through protection fordesired scenes.
However, glass’s reflectivity may reflect undesired scenes on images captured throughit.
People always feel interested in the transmitted scenes behind the glass and want to remove theundesired reflection.
This traps neural radiance fields into a dilemma:
Its well-designed framework iscapable of modeling every detail in a scene, including those undesired ones.
One plausible way to resolve this dilemma is opting for a post-processing tool to remove the undesiredcomponents before the captured images are fed into NeRF [1].
For example, existing reflectionremoval methods [3, 4, 5] can be adopted to suppress the undesired components and provide bettervisibility for transmitted scenes.
However, the processed images from different views may violatethe assumption of NeRF that the scene is photometrically static.
Besides, the reflection cannot becompletely removed in most cases by existing single-image-based reflection removal approaches [6],and the remaining reflection may be modeled by NeRF as a part of the scenes, undermining thequality of rendered results.
An alternative solution is to render with undesired components like NeRF-W [7] by separating thewhole scene into static and transient elements with exclusive properties.
However, the reflectionand transmission usually present a highly “twisted” relationship since they may influence each otherduring light transportation and reflect such ill-posed essence in captured images.
From results shownin Figure 1(left), it may challenge the applicability of a purely unsupervised strategy in NeRF-W [7],thus showing degraded performance.
Besides, NeRF-W [7] also partially relies on a dense set ofmulti-view images to conduct the separation.
As an impractical demand inherited from NeRF [1],
NeRF, 12 views NeRF, 6 views
NeRF-W, 12 views NeRF-W, 6 views
Our result, 6 views
NeRF, 18 views
Warp
A ft e r w a r p
i
ng
R e c u
rr i ng edge s
S pa r s e pa tt e r n
Recurring edge estimation Examples obtained by NeRF, NeRF-W, and our method it contradicts the casual photographing experience, where only limited inputs with sparse viewsare available.
The results in Figure 1(left) illustrate that NeRF-W [7] cannot perform high-fidelitysynthesis with limited views, let alone effective separation between the transmitted scene and thereflected scene.
Thus, how to render novel transmitted scenes with sparse views poses uniquechallenges.
In contrast to the exclusive properties required by NeRF-W [7], we employ the recurring edges,a phenomenon verified by pioneering reflection removal methods [8, 9], to link the above issuesby warping images with different viewpoints into a predefined target view.
As shown in Figure 1(right), given an aligned image sequence, the transmitted scenes appear in different views with a largeoverlapping area while the reflected scenes only have a sparse presence [8].
Besides, the alignedimage features from the neighboring views can also be used for a more high-fidelity reconstruction ofthe target view.
We, therefore, design a method supported by two pillars: 1) Edges of the repeatedpatterns, denoted as the recurring edges shown in Figure 1(right), that could be estimated as a pilot toguide the differentiation during rendering; 2) features from reference views, which are incorporatedinto rendering for high-fidelity reconstruction under sparse views.
Several modifications have to be conducted to accommodate the above designs.
Since the transmission/reflection edges only show their significance in a certain area, the classical “pixel-wise”rendering framework cannot capture such internal physical properties.
We employ a patch-basedrendering framework, which aggregates the pixels from a certain area into a patch as the trainingbatch.
Besides, due to the interference caused by the reflection, the features extracted by a networkpre-trained for general tasks [10] may introduce interference from feature level, which deteriorates thetransmission/reflection separation described before.
We, therefore, propose a transmission encoder topenetrate the feature-level interference caused by the reflection and render the transmission structuremore accurately.
At last, since the strength of reflection tends to rely on the viewing direction,the weighting coefficients for the two layers are obtained from the viewing direction to combinetransmission and reflection layers.
In summary, the above considerations lead to our Neural Transmitted Radiance Fields as depictedin Figure 2, and its main contribution can be concluded as follows:
• An approach to render novel transmitted views, which is capable of penetrating the reflectioninterference.
• A unified consideration derived from recurring edgew for removing reflection/transmission
ambiguity and increasing rendering fidelity of transmission.
• A dataset collected from the real world for evaluation.
Reflection embedding
Glass
V i e w B
V i e w A
V i e w C …
A B
C …
R-MLP T-MLP
Transmission feature encoder
Transmission feature integration
Recurring edge estimation
A B
C …
Image projection
Patch-based rendering
Position Viewing direction
E x t r a c t ed r e c u rr i ng edge s Reflectionscene
Aligned images
Image edges
Virtual images
Transmission features
See-through scenarios
Captured images 𝐂 𝐓 𝐂 𝐑 𝐆($) …
ℒ #$%#
Next, a transmission feature encoder extracts the transmission features from the reference images (transmission feature integration).
The extracted transmission features are then passed into the transmission MLP (T-MLP) for transmission rendering ( C T ).
For the reflection rendering ( C R ), we feed the viewing direction and corresponding reflection embeddings into the reflection MLP (R-MLP).
The recurring edges repeatedly appearing in the aligned image sequence are extracted to further differentiate the transmission and reflection based on the edge loss L edge (recurring edges estimation).
Related work
Novel view synthesis with NeRF.
Novel view synthesis aims at generating images from another viewpoint using a set of established views.
The classical approaches rely on the explicit representationslike multi-plane images [11, 12], meshes [13], or point cloud [14] to render novel views.
Despitetheir rendering efficiency at test time, they usually suffer from the limited expressiveness within theirrepresentation.
In contrast to the explicit representations, Neural Radiance Fields (NeRF) [1] utilizethe implicit representation to represent 3D scenes using neural networks.
For example, NeRF [1]approximates a continuous 3D MLP by mapping an input 5D coordinate to its corresponding sceneproperties.
Given its promising representation ability, NeRF [1] has been extended to solving problems like relighting and scene editing.
Recently, NeRF-W [7] proposes to adapt the original NeRF [1]to wild scenes by introducing an additional MLP to model the undesired transient element, whichgrants the original NeRF [1] extended ability to model the scenes with photometric variations andtransient components.
Since NeRF requires a large number of images for training, several methods [10, 15] are proposed to make it feasible with sparse views by utilizing the feature informationfrom neighboring views.
In this work, we aim to extend NeRF to see-through scenarios by eliminatingnegative interference caused by reflection.
Transmission recovery from reflection-corrupted images.
How to recover the transmitted scenefrom reflection-corrupted images has been discussed for decades.
Early methods [16, 17] makeuse of the edge difference to reduce the ambiguity between the transmission and reflection throughoptimization.
Recent methods mainly adopt the deep learning framework to solve this problem.
Forexample, Fan et al.
[18] proposed a two-stage deep learning approach to learn the mapping betweenthe mixture images and the estimated clean images.
Wan et al.
[19, 20] proposed a cooperative modelto better preserve the background details.
Zhang et al.
[21] proposed a perceptual reflection removalmethod based on the generative adversarial network.
Wei et al.
[5] proposed another method to solvethe reflection removal problem with non-aligned image pairs.
Besides, some recently proposed [4]methods also try to focus on the reflection-corrupted regions by localizing the reflection first.
Therecently proposed method [22] also removed reflections under the guidance of an additional polarizer.
Instead of solely relying on single images, plenty of methods [8, 23, 24, 25] also leverage advantagesfrom the priors provided by multiple images to remove reflections based on recurring edges belongingto the transmission.
Such inconsistency observed from different views also allows our framework todifferentiate transmission and reflection during rendering.
Preliminaries about Neural Radiance Fields
We first briefly review the settings in NeRF [1] and NeRF-W [7].
A NeRF represents a scene as acontinuous volumetric radiance field f , which maps the position of any given point x
= ( x, y, z ) and a viewing direction
d = ( θ, ϕ ) into a color c = ( r, g, b ) and density σ as f ( x , d ) =
( c , σ ) .
Then,the expected color ˆ C corresponding to the camera ray r ( t ) =
o + t d emitted from the center ofprojection of a camera at position o can be obtained as
[Equation]
(1) where t n and t f denote the near and far bounds of rendering, and T ( t ) =
exp( − (cid:82) t t n σ ( r ( s )) ds )denotes the accumulated transmittance.
NeRF-W [7] extends NeRF [1] with two MLPS to model thedesired static ( f s ) and undesired transient ( f t ) components asf s ( x [Equation] d , [Equation]
a ) =
( c s
[Equation] σ [Equation] and f t ( x , d , ℓ t ) =
( c t , σ t ) .
Due to the introduction of the additional MLP for transient elements, as well as the embeddings ℓ a andℓ t adapting to the various appearances and transient objects, NeRF-W [7] is capable of decomposingthe desired static components during rendering.
For an image I captured in see-through scenarios, the relationship between the reflection R andtransmission T can be formulated as I = β T + α R
[6], where β and α denotes the weightingcoefficients.
Applying NeRF-W [7] to achieve novel transmitted view synthesis is not directlyapplicable due to the following issues: 1) Dense view requirement: Both NeRF-W [7] and NeRF [1]inherit the substantial demand for dense multi-view images, and their performances degenerate underreal-world scenarios where only limited input with sparse views are available.
Undesired reflectioninterference: For the objects on both sides of the glass, their emitted light rays may be absorbed,refracted, and reflected, leading the light energy attenuation or highly mixed phenomenon in thecaptured image [6].
This makes their separation present a ill-posed essence, which cannot be simplyresolved by using an unsupervised model.
We propose to construct Neural Transmitted Radiance Fields to address these issues.
Neural Transmitted Radience Fields
Our goal is to learn radiance fields as a representation for the transmitted scene behind glass.
Toavoid the high separation ambiguity for transmission/reflection and low reconstruction fidelity undersparse views, we build a unified approach by warping pixels to the desired target views.
By extractingthe recurring edges after warping, our approach can get rid of the interference from the reflection.
Meanwhile, a transmission feature integration scheme is introduced to merge the transmission featuresfrom the warped images, which enables higher reconstruction fidelity even under sparse views.
The pixel-wise rendering scheme in NeRF [1] and NeRF-W [7] lacks the ability to reflect thetransmission/reflection correlation, since some statistical properties can only be observed in a certainarea.
To better consider the spatial correlation among neighboring pixels, we employ the patch-basedrendering scheme.
Specifically, during training we sample a K × K patch P ( u , s ) in a given imagefollowing the way proposed in [26] as follows:
[Equation]
where u = ( u, v ) denotes the center position of the patch, x and y range from − K 2 to K 2 − 1 tocreate a sampling grid, and s denotes the scale of the image patch P .
In our setting, we apply acombined sampling of grid sampling with s = 1 over the entire image and random sampling withs > 1 , allowing image patches to capture spatial information of different scales.
The aggregatedpixels from the image patch are then used for training, during which the model can not only learn thecolors and densities through one ray, but also get a thorough understanding towards the context.
4.1 Transmission feature integration
In contrast to NeRF [1] and NeRF-W [7], which solely rely on point coordinates and viewingdirection as the input, we propose to address the fidelity issue by means of the additional guidancefrom reference images.
Given a scene only with sparse views, the feature guidance from neighboringviews can help to reconstruct novel target views with higher fidelity.
In general, such image featuresare extracted by using a 2D CNN network [10] pretrained for general tasks.
However, since the images under neighboring views are also captured through glass, a general CNN network for viewsynthesis like previous methods [15] may lead to feature-level interference during rendering.
From Figure 2, we first project the neighboring views to their target view.
To resist such interferenceduring feature integration, we propose to obtain the transmission features by means of a transmissionencoder η obtained by pre-training a reflection removal network, e.g. , ERRNet [5].
As a featureencoder pretrained for reflection removal task, η aims at extracting the transmission features as W = η ( I ) , where W denotes the extracted feature volume.
Instead of solely relying on the singlescale feature like pixel-NeRF [27], which suffers from view misalignment in our setting, we proposeto build a feature pyramid as W =
{ W g , W l } , where W g and W l denote the global and localfeatures with different scales, respectively.
In general, as CNNs naturally store local informationin shallow layers and encode global information in deeper layers with a large receptive field, weextract W g and W l from the bottleneck layer and second last layer of η , respectively.
Prior worksalso suggest combining features from different scales can lead to better convergence and geometricdetails [28].
The two feature maps are upsampled using Nearest-Neighbor Interpolation to the size ofthe image.
The extracted global and local transmission features for the i -th target view are then passed intothe MLP network for the transmission rendering (T-MLP)
f T , along with the position and viewingdirection, as
[Equation]
(4) where c ( i ) t and σ ( i ) t are the estimated color and density of the transmitted scene for the i -th view.
As for reflection, a learnable reflection embedding ℓ ( i ) r is introduced for each view i , and the colorc ( i ) r and density σ ( i ) r of the reflection scene are estimated using an additional MLP network for thereflection rendering (R-MLP) as
[Equation]
where the reflection embedding is optimized along with the weights of f R .
Similar to existing NeRFs [1, 7], we construct a self-consistent loss as follows:
[Equation]
where p ( · ) denotes the mapping from a ray to its corresponding pixel position, and ˆ C ( r ) denotes theobserved color and C denotes the composite of C T and C R physically defined as C = β C T + α C R .
Here α denotes the weighting coefficients to balance C T and C R .
For simplicity, we directly setβ = 1 − α like previous reflection removal methods [29].
Several factors during light transportation can influence the weighting coefficients α .
In our setting,since the strength of reflection largely depends on viewing direction [30] rather than the exact cameraposition, we propose to model such influence by a simple MLP that outputs the weighting coefficientsgiven corresponding viewing direction encoding as f α ( x , d )
= α .
Specifically, as d and x refer tothe viewing direction and the position of any given points, respectively, the weighting map of a givenview is rendered similarly to Equation (1) by considering the two factors.
It enables the network toincrease its robustness for general cases.
4.2 Recurring edge constraints
The settings described above ensure a high-fidelity novel view reconstruction even under sparseviews.
However, since no reflection removal model can reach a completely reflection-free status, thereflection residuals left by the transmission feature encoder may further deteriorate the ambiguityissue.
We then utilize the benefits provided by the aligned image sequence to further disentangle theirtwisted relationship.
As pointed by pioneer methods [9, 29], when multiple images with different viewpoints are capturedthrough glass, the transmission appears frequently across all samples, while the reflection may onlyhave sparse presence.
This property makes the recurring transmission edges become a “pilot” torelieve ambiguity between the transmission and reflection.
In our method, we first transform alignedimages into their corresponding edge domain and measure the frequency of edge occurrence as
[Equation]
where G i ( z ) denotes the gradient magnitude at pixel location z of view i , and k denotes the totalnumber of pixels.
It measures the sparsity of the vector which achieves its maximum value of 1(when only one non-zero item exists) and achieves its minimum value of 1 /k (when all items arenon-zero and have identical values).
Using this measurement, each edge pixel is assigned with twoprobabilities regarding whether it belongs to the transmitted scene or not.
Recognizing the transmission and reflection poses a great challenge for a vanilla NeRF networkaiming at multi-view consistency, we provide our network with a likelihood estimation of thetransmission and reflection, following the strategy proposed in [29]:
[Equation]
where s is a sigmoid function to facilitate the separation.
Then, the final likelihood for transmissionedges can be defined as follows:
[Equation]
From the results shown in Figure 1(right), the likelihood of target transmitted scene effectively labelsthe transmission edges, which provides a clear guidance during rendering.
We search from 0 to 1with a step of 0 . 1 and fix the threshold in Equation (9) as 0 .
6 in our experiments. Based on the binarized edge map obtained in Equation (9), we propose to constrain rendering using asimple edge loss as follows:
[Equation]
which means we constrain the rendered image more on wherever the transmission edges are present,encouraging a more faithful reconstruction towards the mixture image.
Besides, the transmission edges and those of the reflection tend to be different, as prior works pointedout [21].
We inherit the exclusion assumption between the gradient of transmission and reflection todifferentiate the two parts in the gradient domain as follows:
[Figure]
[Equation]
where λ T = (cid:113) ∥
G ( C R ) ∥
F ∥ G ( C T ) ∥
F and λ R = (cid:113)
∥ G ( C T )
∥
F ∥ G ( C R ) ∥ F .
The downsampling operation in its originalsetting is discarded since our patch has been small enough.
Ψ( · , · ) defines a pixel-wise correlationbetween the transmission and reflection, which helps to separate them in the gradient domain.
By considering the above settings and constraints, our loss functions for the whole optimizationcan be concluded as L = γ L sc + δ L edge + ω L excl ,
where γ = 2 , δ = 0 . 002 , and ω = 1 are theweighting coefficients to balance the influence of each term.
All the calculations and optimizationprocesses are based on the patch sampled from the image.
4.3 Implementation details
We implement our framework using PyTorch.
In the training and testing phase, two eight-layer MLPswith 256 channels are used to predict colors c and densities σ corresponding to the transmitted andreflection scenes.
We train a “coarse” network along with a “fine” network network for importancesampling.
We sample 64 points along each ray in the coarse model and 64 points in the fine model.
Abatch contains an image patch of 32 × 32 pixels, equivalent to 1024 rays.
Similar to the settings in
NeRF [1], positional encoding is applied to input location before they are passed into the MLPs.
Weuse the Adam optimizer with defaults values β 1 = 0 .
999 , β 2 = 0 .
9 , ϵ = 10 − 8 , and a learning rate10 − 4 that decays following the cosine scheduler during the optimization.
We optimize a single modelfor about 100 K iterations on two NVIDIA V100 GPUs.
Our transmission encoder can extract transmission features of multiple resolutions by leveragingadvantages from ERRNet [5] and U-Net
Target view Ours
NeRF NeRF-W MVSNeRF RR+NeRF
RR+MVSNeRF NeRFReN
Experiments
Dataset.
Our experiments are all based on a real-world dataset we collect.
This dataset contains8 different real-world scenes, each consisting of 20 to 30 mixture images with different poses.
Specifically, four scenes are with the ground truth for quantitative evaluations in the experiments.
Wefollow the setup proposed in [6] by first capturing the mixture image through transparent glass andthen its corresponding transmitted scene by removing the glass. We follow the previous settings in
NeRF [1] and its variants [10, 32] to estimate the pose via COLMAP [33, 34].
Like most reflectionremoval methods [6], we assume a piece of planar glass when capturing images.
Though someglasses in the real world are slightly curved, it does not obviously affect the robustness of our method.
Baselines.
We compare our method with five NeRF-based methods: 1) NeRF [1]: the original NeRFmethod; 2) NeRF-W [7]: an unofficial implementation of NeRF in the Wild 1 ; 3) MVSNeRF [10]: a NeRF-based multi-view stereo model for novel view synthesis (NVS) which also relies on referenceviews; 4) Reflection removal (RR) + NeRF: the most direct way for novel transmitted view synthesis,namely training a NeRF network based on the results of a reflection removal method (without losinggenerality, we use ERRNet [5], a state-of-the-art method inspiring our design); 5) NeRFReN
[32]:a NeRF-based method designed for see-through scenarios released recently 2 ; 6) RR + MVSNeRF:
NVS using MVSNeRF [10] with reflection removal applied as a pre-processing.
All the resultsin Section 5 are obtained using six views for training.
Due to the page size limitation, the results ofother methods trained under more views can be found in the supplementary material.
Metrics.
We report quantitative performance using PSNR, SSIM and LPIPS [35].
For PSNRand SSIM, higher value indicates better performance.
For LPIPS [35], lower value indicates betterperformance.
By measuring image fidelity using high-level features, LPIPS [35] can better matchhuman judgements of image similarity.
5.1 Qualitative results
The qualitative results of the rendered novel transmitted views are presented in Figure 3 and Figure 4.
Due to the effectiveness of the proposed transmission feature integration scheme, our method canreliably reconstruct novel transmitted views under only six views.
Besides, the feature integrationscheme and its cooperative recurring edge constraints further reduce the ambiguity between reflectionand transmission during rendering.
For other methods, though NeRF [1] can partially reconstruct thedesired transmitted views, it still suffers from the interference caused by the reflection; NeRF-W [7]and NeRFReN [32] output degenerated results as they cannot separate reflection from transmissionwith as few inputs as our method; MVSNeRF [10] can keep high fidelity of the rendering results likeour methods, but obvious reflection residuals still remain in their results since their feature extractorcannot resist the reflection interference.
We and it by ourselves.
[Table]
5.2 Quantitative results
The quantitative results of the rendered novel transmitted view in Table 1 also validate the observationin Figure 4.
Higher PSNR values show that our method can render novel transmitted views andrecover the color information with higher accuracy.
Higher SSIM values indicate that our methodcan preserve the structural information with high-frequency details.
Lower LPIPS values show thatrecovered images by our method better aligns with human perception.
Specifically, for the resultsobtained by other methods, their higher LPIPS values indicate that they cannot reconstruct realisticresults with high fidelity when only six views are provided.
Specifically, MVSNeRF [10] achievesthe second best LPIPS results among all methods, which shows that the feature integration schemecan indeed improve the rendering fidelity.
On the other hand, RR+NeRF setting achieves the secondbest PSNR and SSIM result, which shows the effectiveness of this simple strategy to some degrees.
5.3 Ablation study
Our network consists of two parts: the transmission encoder to disentangle the transmitted andreflected scenes, and the patch-rendering scheme to utilize the physical constraints.
We conduct
[Table]
[Table]
w/o) η & P
[Table]
[Figure]
[Figure]
several experiments to evaluate the benefits of these two parts.
We first remove the transmissionencoder and directly feed the MLP with the position and viewing direction.
From the results shownin Figure 5, without the support of reference views, the position and viewing directions cannoteffectively reconstruct the desired transmitted views.
The errors shown in Table 2 also become closerto those of NeRF in Table 1.
We then remove the recurring edge constraints, and more reflection isobserved in the rendered target view since the network attaches equal importance to every pixel inthe patch.
We further remove the patch-based rendering scheme entirely.
In this situation, reflectionevidently remains on the generated target view due to a lack of physical constraint, as we discussedbefore.
At last, both the patch-based rendering scheme and the transmission encoder are removed,and we achieve a degenerated result similar to that of NeRF-W [7] shown in Table 1.
Without lossof generality, we specifically compare with MVSNeRF [10] on non-reflective common examples toshow that our method can also adapt to more general scenarios in Figure 6.
Conclusions
[Figure] We solve the problem of novel view synthesis for see-through scenarios in this paper.
We introduce atransmission encoder to address the fidelity issue caused by sparse views and ambiguity issues led bythe reflection interference in a unified framework.
Specifically, to further disentangle the twisted relationship between transmission and reflection during the rendering process, we introduce a recurringedge constraint by counting the frequency of edge occurrence among the aligned image sequence.
Experimental evaluation on a newly collected dataset demonstrates the promising performance fornovel transmitted view synthesis our method could achieve.
Limitations and future work.
Our method still faces the challenge led by the occlusion issue,since we rely on features from neighboring views to complement the missing information given asparse set of input views.
However, when features are not consistent with a certain view, occlusionsmay negatively undermine the performance of our proposed method, due to the inaccurate featureinformation for synthesizing novel views in that area.
Besides, our method relies on COLMAP [33,34] for the rendering, while it may fail on low-transmitted reflections covering large areas or thetransmission distortion caused by non-planar glass.
In this situation, COLMAP [33, 34] cannotcorrectly extract the transmission features.
Investigating NeRF without the requirement for establishedposes like [37] may help to alleviate this issue.
We will also consider the non-planar glass for itsinfluence on the transmitted objects’ light in our future work. Checklist For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section 6.(c)
Did you discuss any potential negative societal impacts of your work?
[No] We do not
think this work will have any potential negative societal impacts.
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section 4.
(b) Did you include complete proofs of all theoretical results?
[Yes] See Section 4.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[No] (during peer
review).
The code and data will be released once the paper is accepted.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] See Section 5.
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[No] We think error bars are unnecessary in our work.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Section 5.
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[No] We only use the
dataset captured by ourselves for demonstration.
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(during peer review).
The data will be released once the paper is accepted.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]