MoGDE: Boosting Mobile Monocular 3D Object Detection with Ground Depth Estimation
Abstract
Monocular 3D object detection (Mono3D) in mobile settings ( e.g. , on a vehicle, a drone, or a robot) is an important yet challenging task.
Due to the near-far disparity phenomenon of monocular vision and the ever-changing camera pose, it is hard to acquire high detection accuracy, especially for far objects.
Inspired by the insight that the depth of an object can be well determined according to the depth of the ground where it stands, in this paper, we propose a novel Mono3D framework, called MoGDE , which constantly estimates the corresponding ground depth of an image and then utilizes the estimated ground depth information to guide Mono3D.
To this end, we utilize a pose detection network to estimate the pose of the camera and then construct a feature map portraying pixel-level ground depth according to the 3D-to-2D perspective geometry.
Moreover, to improve Mono3D with the estimated ground depth, we design an RGB-D feature fusion network based on the transformer structure, where the long-range self-attention mechanism is utilized to effectively identify ground-contacting points and pin the corresponding ground depth to the image feature map.
We conduct extensive experiments on the realworld KITTI dataset.
The results demonstrate that MoGDE can effectively improve the Mono3D accuracy and robustness for both near and far objects.
MoGDE yields the best performance compared with the state-of-the-art methods by a large margin and is ranked number one on the KITTI 3D benchmark.
Introduction
Building on the promising progress achieved in 2D object detection in recent years [41, 40], 3D object detection, particularly on moving agents, has received increasing attention from both industry and academia as an important component in many applications, ranging from autonomous vehicles [17] and drones, to robotic manipulation and augmented reality applications.
Compared to LiDAR-based [12, 36, 42, 43, 23, 63] and stereo-based [10, 11, 21, 35, 38, 54] methods, a much cheaper, more energy-efﬁcient, and easier-to-deploy alternative, i.e. , monocular 3D object detection (Mono3D), remains an open and challenging research ﬁeld.
A practical Mono3D detector for moving agents should meet the following two requirements: 1) the 3D bounding box produced by the Mono3D detector should be accurate enough, not only for near objects but also for very distant objects, to secure, for instance, high-priority driving safety applications; 2) the Mono3D detector should remain robust in mobile scenarios, where the camera pose inevitably changes along with the movement of a mobile agent.
In the literature, recent Mono3D methods with complex network structures [62, 33, 52, 37, 48] have achieved considerably high accuracy for near objects, but the predicted 3D bounding boxes for far objects are often ill-posed due to the lack of depth cues.
This huge disparity between near and far objects lies in the nature of monocular vision.
Speciﬁcally, as depicted in Figure 1 (a), equal distances of different depth from the camera ( e.g. , ∆ z 1 = ∆ z 2 ) have distinct number of pixels in image ( e.g. , ∆ u 1 > ∆ u 2 and ∆ v 1 > ∆ v 2 ), which makes the pixel rounding errors have non-negligible impact for detecting far objects (see Appendix ??
for detailed analysis).
Furthermore, as illustrated in Figure 1(b), the camera pose variance can eventually result in a large offset both in form of 3D boxes and in the bird’s eye view [15].
To the best of our knowledge, existing Mono3D methods such as geometric constraint based [9, 34, 6, 53], pseudo-LiDAR based [29, 30, 50, 54, 37, 16, 28], and pure image based [41, 45, 61, 3, 25, 49, 13, 22, 4, 62, 60, 27, 59], have not taken into account the issue of inevitable camera pose changes in mobile scenarios.
In this paper, we propose a novel Mono3D method, called MoGDE , which ﬁxates on improving the detection accuracy and robustness in mobile settings.
We have one key insight that the depth of an object in 3D space can be well determined according to the depth of the ground where it stands .
Given the pinhole model and the pose of a camera, the depth of each pixel corresponding to the ground can be accurately derived.
Based on this insight, the core idea of MoGDE is ﬁrst to constantly estimate the ground depth while moving and then to utilize the estimated ground depth information to guide a Mono3D detector.
There are two main challenges in designing MoGDE.
First, how to detect varying camera pose ( e.g. , the pitch and roll angles) from an image, which is dynamically changing in mobile scenes, and how to ultimately obtain an accurate ground depth information are non-trivial.
It is clear that different camera poses correspond to distinct ground depth estimates.
To tackle this challenge, we introduce a pose detection network to extract the vanishing point and horizon information in an image to estimate the instant camera pose corresponding to this image.
After the view direction of the camera is decided, we then construct a feature map portraying pixel-level depth clues.
Speciﬁcally, we envision a virtual 3D scene containing only the sky and the ground and project this virtual scene to an image where each pixel is associated with a depth uniquely derived according to the 3D-to-2D perspective geometry.
Therefore, MoGDE can obtain the dynamic ground depth information as prior knowledge for guiding
Mono3D.
Second, how to incorporate the estimated ground depth into the image features to enhance the detection accuracy is challenging.
Based on our aforementioned insight, it is essential for the
Mono3D detector to identify those ground-contacting points of an object on the image.
For example in Figure 1 (a), these blue dots denote the ground-contacting points of a vehicle.
To this end, we design an RGB-D feature fusion network based on the transformer structure to tie the ground depth feature to the image feature.
Speciﬁcally, as illustrated in Figure 1(c), the feature fusion network captures the feature of pixels close to the centroid of an object and identiﬁes those ground-contacting points using the attention mechanism.
It then attaches depth values with weights to compute a new feature map containing object location information.
As a result, accurate 3D detection results can be obtained via a conventional Mono3D detector using the fused feature map.
Experiments on KITTI dataset [17] demonstrate that our method outperforms the SOTA methods by a large margin.
Such a framework can be applied to existing detectors and is practical for industrial applications.
The proposed MoGDE is ranked number one on the KITTI 3D benchmark by submission.
The whole suite of the code base will be released and the experimental results will be posted to the public leaderboard.
We highlight the main contributions made in this paper as follows: 1) A novel Mono3D detector in a mobile setting is introduced, leveraging the dynamically estimated ground depth as prior knowledge to improve the detection accuracy and robustness for both near and far objects.
A transformer-based feature fusion network is designed, which utilizes the long-range attention mechanism to effectively identify ground-contacting points and pin the corresponding ground depth to the image feature map.
3) Extensive experiments on the real-world
KITTI dataset are conducted and the results demonstrate the efﬁcacy of MoGDE.
Related Work
Monocular 3D Object Detection.
The monocular 3D object detection aims to predict 3D bounding boxes from a single given image.
Existing Mono3D methods can be roughly divided into the following three categories.
1) Geometric constraint based methods: Extra information of prior 3D vehicle shapes is widely used, such as vehicle computer aided design (CAD) models [9, 34, 6, 53, 26] or key points [1].
By this means, extra labeling cost is inevitably required.
2) Depth assist methods: A standalone depth map of the monocular image is predicted at the ﬁrst stage.
Such prior knowledge can be derived in various ways, such as depth map generated by LiDAR point cloud (or Pseudo-LiDAR) [50, 29, 8, 39], monocular depth predictors [37, 28, 16], or disparity map generated by stereo cameras [54].
However, such external data is not easily available in all scenarios.
In addition, the inference time increases signiﬁcantly due to the prediction of these dense heatmaps.
Pure image-based methods:
Without requiring extra side-channel information, such methods [20, 18, 24, 44] take only a single image as input and adopt center-based pipelines following conventional 2D detectors [41, 61, 45].
M3D-RPN [3] reformulates the monocular 3D detection problem as a standalone 3D region proposal network.
With very few handcrafted modules, SMOKE [25] and FCOS3D [49] predict a 3D bounding box by combining a concise one-stage keypoint estimation with regressed 3D variables based on
CenterNet [61] and FCOS [45], respectively.
To further strengthen monocular detectors, current SOTA methods have introduced more effective but complicated geometric priors.
MonoPair [13] improves the modeling of occluded objects by considering the relationship of paired samples and parses their spatial relations with uncertainty.
Kinematic3D [4] proposes a novel method for monocular videobased 3D object detection, which uses kinematic motion to improve the accuracy of 3D localization.
MonoEF [62] ﬁrst proposes a novel method to capture the camera pose in order to formulate detectors that are not subject to camera extrinsic perturbations.
MonoFlex [60] conducts uncertainty-guided depth ensemble and categorizes different objects for distinctive processing.
GUPNet [27] solves the error ampliﬁcation problem by geometry-guided depth uncertainty and collocates a hierarchical learning strategy to reduce the training instability.
MonoDETR [59] introduces a simple monocular object detection framework that makes the vanilla transformer to be depth-aware and enforces the whole detection process guided by depth.
The above geometrically dependent designs largely promote the overall performance of center-based methods, but the underlying problem still exists, namely, the detection accuracy for distant objects is still not satisfactory.
Object Detection with Transformer.
2D object detectors [61, 45] have achieved excellent performance in recent years but are equipped with cumbersome post-processing, e.g. non-maximum suppression (NMS) [41].
To circumvent it, the pioneering work DETR [5] constructs a novel and simple framework by adapting the powerful transformer [47] to the ﬁeld of vision detection.
DETR detects objects on images by encoding-decoding paradigm, which improves the detection performance by using the long-range attention mechanism.
DETR is further enhanced by designing deformable attention [64], placing anchors [51], setting conditional attention [31], embedding dense prior [57], and so on [32].
Some recent works have tried to apply transformer to some other tasks related to monocular scene reconstruction, depth prediction, etc.
Transformerfusion [2] leverages the transformer architecture so that the network learns to focus on the most relevant image frames for each 3D location in the scene, supervised only by the scene reconstruction task.
MT-SfMLearner [46] ﬁrst demonstrates how to adapt vision transformers for self-supervised monocular depth estimation focusing on improving the robustness to natural corruptions.
While these methods have made a demonstration of how to apply transformer to a monocular camera model [14, 55, 56], they all rely on other branches (either the environment reconstruction or the depth map), which will not be available in a typical Mono3D task based on RGB images.
Our MoGDE inherits DETR’s superiority for non-local encoding and long-range attention.
Specifically, we endow the transformer to be ground-aware by pinning ground depth to image features leveraging the encoder-decoder architecture to improve the detection accuracy for far objects.
[Figure]
Design of MoGDE 3.1 Overview
The core idea of MoGDE is to utilize dynamically estimated ground depth information to improve
Mono3D so that two goals can be achieved: 1) superior ground-aware image features are obtained to increase Mono3D accuracy for both near and far objects; 2) the impact of camera pose variation is diminished to enhance Mono3D robustness in mobile settings.
Figure 2 depicts the architecture of our framework.
Speciﬁcally, MoGDE ﬁrst adopts the DLA-34 [58] as its backbone, which takes a monocular image of size ( W × H × 3 ) as input and outputs a feature map of size ( W s × H s × C ) after down-sampling with an s -factor.
Then, the feature map is fed into three components as follows:
Ground Depth Estimation (GDE).
GDF mainly integrates two functions, i.e. , camera pose detection (CPD) and virtual scene construction (VSC).
Speciﬁcally, CPD estimates the camera pose ( i.e. , the pitch and roll angles) based on the predicted vanishing point and ground plane extracted by a pose detection network.
VSC establishes a 2D ground depth feature map based on a pose-speciﬁc virtual 3D scene containing only the sky and the ground.
Ground Depth Fusion (GDF).
GDF leverages the attention mechanism of a transformer network to fuse the image features with the ground depth feature map, resulting a superior ground-aware fused feature map.
Monocular 3D Detection (M3D).
MoGDE employs GUPNet [27], a SOTA CenterNet [61] based
SOTA monocular 3D object detector as its underlying detection core.
3.2 Ground Depth Estimation 3.2.1 Camera Pose Detection
In order to generate a ground depth estimate, it is key to detect the camera pose given an image feature map.
We have the following proposition:
Proposition 1:
Given a benchmark camera coordinate system P 0 , which is aligned with the ground plane coordinate systems, and the current camera coordinate system P i , which is not aligned with
P 0 due to camera movement, there exists a transformation matrix
A between P i and P 0 that can be uniquely determined by pitch θ p and roll θ p angle changes of the camera. )
Therefore, we introduce the subsequent neural network to learn the pitch θ p and roll θ p angle changes of the camera when the camera coordinate system changes from P 0 to P i .
Speciﬁcally, in addition to the regular regression tasks in CenterNet [61] based network, we introduce a regression branch for pose detection following MonoEF [62].
Since the camera pose is a feature that is implicit for images, we chose two physical quantities with a clear meaning for detection: the ground plane (associated with roll angle) and the vanishing point (associated with pitch angle).
Following the state-of-the-art odometer framework in DeepVP [7], we represent a regression task with L1 loss as:
[Equation]
where H is the input image feature; f pose is the CNN architecture used for horizon and vanishing point detection in the work [19]; ˆ y gp and ˆ y vp are the predicted ground plane and vanishing point; g is a mapping function g : (cid:0) R 2 , R 2 (cid:1) (cid:55)→ A 3 × 3 which turns pitch and roll angles into a matrix A .
The regression network is supervised by L pose and can be trained jointly with other Mono3D branches.
3.2.2 Virtual Scene Construction
We envision such a virtual scene, where there is a vast and inﬁnite horizontal plane in the camera coordinate system P 0 , and have the following proposition:
Proposistion 2:
Given the camera coordinate system P i , the virtual horizontal plane can be projected on the image plane of the camera according to the ideal pinhole camera model and the depth corresponding to each pixel on the image is determined by the camera intrinsic parameter K and pose matrix A from P 0 to P i .
We ﬁrst construct the ground depth feature map in the camera coordinate system P 0 .
Speciﬁcally, as illustrated in Figure 3, for each pixel on the depth image locating at ( u 0 , v 0 ) with an estimated depth ˆ z 0 , it can be back-projected to a point ( x 03 d , y 03 d , ˆ z 0 ) in the 3D scene:
[Equation]
where f x and f y are the focal length represented in the units of pixels along the x - and y -axis of the image plane and c x and c y are the possible displacement between the image center and the foot point.
These are referred to as the camera intrinsic parameters K .
We omit the camera extrinsic T for the sake of simplicity, and the depth corresponding to each pixel on the image is solely determined by the camera intrinsic parameter K under P 0 .
Assume that the elevation of the camera from the ground, denoted as EL , is known (for instance, the mean height of all vehicles in the KITTI dataset, including ego vehicles, is 1.65m [17]), the depth of a point on the depth feature map ( u 0 , v 0 ) can be calculated as:
[Equation]
Note that (3) is not continuous when the point is near the vanishing point, i.e. , v 0 =
c y , and does not physically hold when v 0 ≤ c y .
To address this problem, similar to the KITTI stereo setup, we encode the depth gradient value as an associated feature map using a virtual stereo setup with baseline
B = 0 .
54 m. We represent the ground depth d in the following form:
[Equation]
where b is a constant to prevent the value of d from being too large.
The ReLU activation is applied to suppress ground depth values smaller than zero, which is not physically feasible for monocular cameras.
As a result, the ground depth feature map becomes spatially continuous and consistent.
Finally, to obtain the ground depth feature map in P i , the model need to convert the 3D coordinate system by ( ?? )
ﬁrst, and then just apply (4).
We omit the formula derivation due to page limitation.
[Figure]
Virtual image plane
[Figure]
Moreover, by estimating the transformation matrix A between P 0 and an arbitrary P i , the ground depth feature map in P i can be obtained.
To utilize the ground depth feature, it is key to locate ground-contacting points of an object ( e.g. , the dark point) to get an accurate depth ( e.g. , z 0 r ).
On the contrary, misuse of the depth in the ground depth feature corresponding to other points on the object ( e.g. , the bright point) leads to obvious depth estimation error ( e.g. , z 0 ).
[Figure]
Decoder
Output
[Figure]
3.3 Ground Depth Fusion
In real-world scenarios, as depicted in Figure 3, objects have height.
To fuse the image feature and the ground depth feature, it is key to locate ground-contacting points of an object ( e.g. , the dark point) to get an accurate depth ( e.g. , z 0 r ).
On the contrary, misuse of the depth in the ground depth feature corresponding to other points on the object ( e.g. , the bright point) leads to obvious depth estimation error ( e.g. , z 0 ).
Speciﬁcally, the relation between the estimated depth of an object ˆ z 0 and the pixel displacement in locating ground-contacting points, denoted as T y , can be calculated as,
[Equation]
It can be seen from (5) that T y can cause inaccurate ˆ z 0 .
However, how to acquire T y is non-trivial.
Inspired by the great success of transformer [5, 64, 51, 31, 57, 32] in adaptive long-range relational modeling, we propose a ground-aware feature fusion method based on a transformer structure as depicted in Figure 4, leveraging its attention mechanism to automatically locate ground-contacting points of an object and fuse the corresponding depth feature with the image feature of that object.
Encoder.
Our transformer encoder aims to encode the correlation between image features using a selfattention mechanism.
The input of transformer encoder is the ﬂatten image features H img ∈ R N × C with position encoding and the output is the embedding vectors H e ∈ R N × C to be sent to the decoder.
Following the self-attention pipeline, given the input matrix calculated from the image features: query
Q ∈ R N × C , key K ∈ R N × C , and value V ∈ R N × C with sequence length N = W × H , the output of l + 1 -th layer of self-attention can be brieﬂy formulated as:
[Equation] [Equation]
Here, M l is the mask used to constrain the visible range of attention.
The introduce of M l is to take advantage of the ground-aware property ( i.e. , the depth of each object should be related to the depth of the object’s location), so that each pixel will only consider information within a window around that location.
The encoded feature obtained through multi-head self-attention operation is then re-transformed into a 2D feature map format and fed into a convolution-based feed-forward network (FFN).
The 2D reshape as well as convolution-based FFN are necessary because image data is two-dimensional, unlike one-dimensional serialized data.
Decoder.
The proposed transformer decoder aims to determine for each location its depth information, using the cross-correlation between the ground depth and the image features.
We propose utilizing the ground depth as the location query of the decoder instead of learnable embedding (object query), which is different from the common usage in previous encoder-decoder vision transformer works [5, 64].
The main reason is that, the simple learnable embedding is hard to fully represent the object’s property and handle complex depth variant situations in Mono3D task.
In contrast, plentiful distance-aware cues are hidden in the ground depth features, which will give the transformer a baseline estimate of the expected depth at each location.
To this end, the decoder can leverage the power of cross-attention in transformer to efﬁciently model the correlation between the target pixel point and the point of interest ( i.e. , the grounded point), thus achieving the ground-awareness for higher performance.
Speciﬁcally, the input of decoder is the ﬂatten ground depth H
dep ∈
R N × 1 with position encoding and embedding vectors H e ∈ R N × C obtained from encoder.
In addition, the output is the aggregated feature map H d ∈ R N × C .
The ground depth is ﬁrst embedded upon the standard self-attention architecture following (6).
For the cross-attention module, its input Q is derived from the selfattention part upstream in the decoder, and its K is derived from the encoder.
The input V is a concatenation of two sources from both encoder and decoder.
The purpose of this concatenation is to make the decoder to take into account both the information from the image and the depth during decoding.
Performance Evaluation
We conduct experiments on the widely-adopted KITTI3D dataset and KITTI Odometry dataset [17].
We report the detection results with three-level difﬁculties, i.e. easy, moderate, and hard, in which the moderate scores are normally for ranking and the hard category is generally distant objects that is difﬁcult to distinguish.
4.1 Quantitative and Qualitative Results
We ﬁrst show the performance of our proposed MoGDE on KITTI 3D object detection benchmarkfor car.
Comparison results with other state-of-the-art (SOTA) monocular 3D detectors are shown in
Table 1.
For the ofﬁcial test set, we achieve the highest score for all kinds of samples and is ranked
[Table] [Table]
No.1 among all existing methods with different additional data inputs on all metrics.
Compared to the second-best models, MoGDE surpasses them under easy, moderate and hard levels respectively by +3.42, +1.96, and +2.67 in AP 3 D , especially achieving a signiﬁcant increase (17%) in hard level .
The comparison fully proves the effectiveness of the proposed oracle fusion for images with prior depth knowledge.
Figure 5 shows the qualitative results on KITTI Odometry dataset.
Compared with the baseline model without the aid of ground depth, the predictions from MoGDE are much closer to the ground truth, especially for distinct objects.
It shows that the consideration of sight-based supporting depth clues can help to locate the object precisely.
4.2 Ablation Study
Effectiveness of each proposed components.
In
Table 2, we conduct an ablation study to analyze the effectiveness of the proposed components: (a)
Baseline: only using image features for 3D object detection, i.e. , without concerning posed variance and proposed ground-aware modules.
(b) Considering the camera pose variations implied in the images, we use the method described in [62] to apply a "projection transform" to the input image to remove the perturbations.
(c) Considering the use of ground plane clues, we generate a depth oracle about the scene (assuming constant pose) and use a convolutional neural network in (c) with
[Figure]
[Table]
the proposed ground-aware transformer, which has the ability to model long-range relationships of pixels.
(e) Full model except that we use a convolutional neural network for oracle fusion.
(f) Full model (MoGDE).
First, we can observe from (a → b, c → e
[Table]
and d → f) that there is an implicit uncalibrated pose variation in the KITTI dataset, and considering it is necessary to improve the detection accuracy.
Besides
[Table]
by observing (b → e)
[Table]
we illustrate that leveraging ground depth brings an improvement in accuracy in hard level
[Table]
but the improvement is limited because fusion by convolution is clumsy.
[Table] [Table]
[Table]
In contrast, (e → f) indicates the effectiveness of the transformer, which helps the model to understand the long-range attention relationship between pixel points and the ground plane.
Visualization of attention.
To facilitate the understanding of our ground-aware transformer, we visualize the depth self-attention map in the encoder and paint the query points red and mask region yellow in the third row of Figure 5.
As shown in the ﬁgure, it can be seen that, within the relevant region of each query, areas that are interfacing the object and the ground have the highest attention scores.
In contrast, for non-ground pixels of the object, the lower attention values indicate that the query is not relevant to them, even if they have similar image features and are geographically adjacent.
This implies that under the transformer’s attention mechanism guidance, the query is able to borrow depth information from regions of interest ( i.e. , the ground plane), which helps the fused feature map produce more accurate prediction results.
Simulation experiments on robustness.
In order to verify the robustness of our proposed MoGDE against camera pose variance, we set three cases of variances (tiny, medium, and large) to compare the accuracy degradation of MoGDE with that of baseline.
In Table 3, it can be noticed that the baseline is quite sensitive to pose variance, with very severe performance degradation, while our model only has a slight performance drop.
Moreover, our model performs more robustly especially in the hard case, gaining higher performance improvement.
This demonstrates the effectiveness of our proposed pose-speciﬁc ground depth in handling camera pose variance for mobile scenes.
Plugging into existing methods.
Our proposed approach is ﬂexible to extend to existing image-only
Mono3D detectors.
We respectively plug the Ground Depth Estimation and the Ground Depth Fusion components to three popular Mono3D detectors, which is shown in Table 4.
It can be seen that, with the aid of our proposed ground depth fusion, these detectors can achieve further improvements on KITTI3D val set, demonstrating the effectiveness and ﬂexibility of our approach.
Particularly,
MoGDE-enabled models tend to achieve more performance gains on the hard category.
For example, for Kinematic3D, the AP 3 D / AP BEV gain is +1.83/+1.97 on the easy category and +2.30/+2.86 on the hard category.
Conclusion
In this paper, we have proposed a Mono3D framework, called MoGDE , which can effectively utilize the estimated ground depth as prior knowledge to improve Mono3D in mobile settings.
The advantages of MoGDE are two-fold: 1) it can signiﬁcantly improve the Mono3D accuracy especially for far objects, which is an open issue for Mono3D; 2) it can improve the robustness of Mono3D detectors when applied in more appealing mobile applications.
Nevertheless, MoGDE still have two main limitations as follows: 1) it heavily relies on the pose detection, which directly affects the accuracy of the ground depth estimation; 2) it also counts on the detection of ground-contacting points.
In cases when such points are uncertain or ambiguous due to occlusion and truncation, it is hard for the proposed ground-aware feature fusion method to obtain accurate results.
These limitations also direct our future work.
We have implemented Mono3D and conducted extensive experiments on the real-world KITTI dataset.
MoGDE yields the best performance compared with the state-of-the-art methods by a large margin and is ranked number one on the KITTI 3D benchmark. Acknowledgements
This research was supported in part by National Key RD Program of China (Grants No.
2018YFC1900700), National Natural Science Foundation of China (Grants No. 61872240 and 61972081), and the Natural Science Foundation of Shanghai (Grant No. 22ZR1400200)
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] (c) Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See appendix.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See appendix.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[No] (c) Did you include any new assets either in the supplemental material or as a URL?
[No] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[No] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[No] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]