OGC: Unsupervised 3D Object Segmentation from Rigid Dynamics of Point Clouds
Anonymous Author(s)
AffiliationAddress email
Abstract
In this paper, we study the problem of 3D object segmentation from raw pointclouds.
Unlike all existing methods which usually require a large amount of humanannotations for full supervision, we propose the first unsupervised method, called
OGC, to simultaneously identify multiple 3D objects in a single forward pass,without needing any type of human annotations.
The key to our approach is tofully leverage the dynamic motion patterns over sequential point clouds as thesupervision signals to automatically discover rigid objects.
Our method consistsof three major components, 1) the neural backbone to directly estimate multiobject masks, 2) the auxiliary self-supervised scene flow estimator, and 3) our coreobject geometry consistency component.
By carefully designing a series of lossfunctions, we effectively take into account the multi-object rigid consistency andthe object shape invariance in both temporal and spatial scales.
This allows ourmethod to truly discover the object geometry even in the absence of annotations.
We extensively evaluate our method on three datasets, demonstrating the superiorperformance for object part instance segmentation and general object segmentationin both indoor and the challenging outdoor scenarios.
Introduction
Identifying 3D objects from point clouds is vital for machines to tackle high-level tasks such asautonomous planning and manipulation in real-world scenarios.
Inspired by the seminal work
PointNet [53], a plethora of sophisticated models [66, 71, 34] have been developed to accuratelydetect and segment individual objects from the sparse and irregular point clouds.
Although thesemethods have achieved excellent performance on a wide range of public datasets, they primarily relyon a huge amount of human annotations for full supervision.
However, it is extremely costly to fullyannotate every objects in point clouds due to the irregularity of data format.
Very recently, a few works start to address 3D object segmentation in the absence of human annotations.
By analysing 3D scene flows from sequential point clouds, Jiang et al .
[25] apply theconventional subspace clustering optimization technique to identify moving objects from raw pointcloud sequences.
With the self-supervised learning of 3D scene flow, SLIM [3] is the first learningbased work to showcase that the set of moving points can be effectively learned as an object againstthe stationary background.
Fundamentally, their design principle shares the key spirit of Gestalttheory [68, 65] developed exactly 100 years ago: the raw sensory data with similar motion are likelyto be organized into an object.
This is indeed true in our real world where solid objects usuallyhave strong correlation in rigid motions.
However, these methods can either effectively separatethe foreground and background points, being unable to simultaneously learn to segment multipleinterested 3D objects in one go, essentially due to their incapability of grouping diverse motionpatterns into multiple objects.
[Figure]
Motivated by the potential of motion dynamics, this paper aims to design a general neural frameworkto simultaneously segment multiple 3D objects, without requiring any human annotations but theinherent object dynamics in training.
To achieve this, a naïve approach is to train a neural networkto directly cluster motion vectors into groups from sequential point clouds, which is widely knownas motion segmentation in 2D images [72, 73].
However, such design requires that the input datapoints are sequential in both training and testing phases, and the trained model cannot infer objectsfrom a single point cloud.
Fundamentally, this is because the learned motion segmentation strategiessimply cluster similar motion vectors instead of discriminating the diverse geometry of objects, andtherefore such design is not general enough for real-world applications.
In this regard, we design a new pipeline which takes a single point cloud as input and directlyestimates multiple object masks in a single forward pass.
Without needing any human annotations,our pipeline instead leverages the underlying dynamics of sequential point clouds as supervisionsignals.
In particular, as shown in Figure 1, our architecture consists of three major components: 1) aneural backbone to extract per-point features and estimate all object masks from a single point cloud,as indicated by the orange block; 2) an auxiliary self-supervised network to estimate per-point motionvectors from a pair of point clouds, as indicated by the green block; 3) a series of loss functions tofully utilize the motion dynamics to supervise the object segmentation backbone, as indicated by theblue block.
For the first two components, we are actually flexible to adopt any of existing neuralfeature extractors [54] and self-supervised motion estimators [31].
Nevertheless, the third componentis particularly challenging to design, primarily because we need to take into account not only theconsistency of diverse dynamics of multiple objects in a sequence, but also the invariance of objectgeometry irregardless of different moving patterns.
To tackle this challenge, we introduce three key losses: 1) a multi-object dynamic rigid consistencyloss, which aims to evaluate how coherently all estimated object masks (shapes) can fit the motionvia rigid transformations; 2) an object shape smoothness prior, which regularizes all points of eachestimated object to be spatially continual instead of fragmented; 3) an object shape invariance loss,which drives multiple estimated masks of a particular object to be invariant given different (augmented) rigid transformations.
These losses together force all o bjects’ g eometry to be c onsistent andrepresented by high-quality masks, purely from raw 3D point clouds without any human annotations.
Our method is called OGC and our contributions are:
• We introduce the first unsupervised multi-object segmentation pipeline on 3D point clouds, withoutneeding any human annotations in training.
• We design a set of geometry consistency based losses to fully leverage the object rigid dynamicsand shape invariance as effective supervision signals.•
We demonstrate promising object segmentation performance on three datasets, showing significantly better results than classical clustering and optimization baselines.
Scope:
This paper neither aims to design a new self-supervised 3D scene flow estimation method,nor intends to replace existing fully-supervised 3D object segmentation approaches because thenever-seen static objects are unlikely to be discovered due to the lack of supervision signals.
Inaddition, estimating object categories or non-rigid objects is also out of the scope of this paper.
Related Works
Fully-supervised 3D Object Segmentation: To identify 3D objects from point clouds, existing fullysupervised solutions can be divided as 1) bounding box based object detection methods [78, 34, 57] or2) mask based instance segmentation pipelines [66, 71, 64].
Thanks to the dense human annotationsand the well developed backbones including projection-based [35, 7, 34], point-based [54, 61, 23] PointNet++ Transformer 𝑷 𝑡 𝑶 𝑡 𝑷 𝑡 F l o w S t e p
D 𝑷 𝑡+ 𝑴 𝑡 𝑶 𝑡 𝑷 𝑡 𝑴 𝑡 Weighted Kabsch 𝓁 𝑑𝑦𝑛𝑎𝑚𝑖𝑐 𝓁 𝑠𝑚𝑜𝑜𝑡ℎ 𝑶 𝑡 ,𝑷 𝑡 𝑷 𝑣2 𝑡 𝑶 𝑣2 𝑡 Hungarian 𝑷 𝑣1𝑡 𝑶 𝑣1 𝑡 𝓁 𝑖𝑛𝑣𝑎𝑟𝑖𝑎𝑛𝑡
Neural backbone
[Figure]
and voxel-based [19, 10] feature extractors, these methods achieve impressive performance on bothindoor and outdoor datasets.
However, manually labelling every object in large-scale point clouds islaborious and costly.
To alleviate this burden, this paper aims to pioneer 3D object segmentation inthe absence of human labels.3D Scene Flow Estimation and Motion Segmentation:
Given sequential point clouds, per-point3D motion vectors, also known as scene flow, can be accurately estimated.
Early works focus onfully-supervised scene flow estimation [39, 4, 21, 40, 52, 50, 67], whereas recent methods start toexplore self-supervised motion estimation [47, 69, 31, 43, 76, 36].
Taking the scene flow as input,a number of works [74, 24, 60, 3] aim to group similar motion vectors, and then obtain boundingboxes or masks only for dynamic objects.
Although achieving encouraging results, they either rely onground truth segmentation for supervision or can only segment simple foreground and backgroundobjects, without being able to simultaneously segment multiple objects.
In this paper, we leverage thesuccessful self-supervised scene flow estimator as our auxiliary neural network to provide valuablesupervision signals, so that multiple objects can be identified in a single forward pass.
Unsupervised 2D Object Segmentation: Inspired by the early work AIR [14], a large numberof generative models have been proposed to discover objects from single images without needinghuman annotations, including MONet [5], IODINE [20], Slot-Att [41], etc.
These methods arefurther extended to segment objects from video frames [32, 22, 46, 26, 79, 13].
However, all theseapproaches can only process simple synthetic datasets, and cannot discover objects from complexreal-world images or videos yet.
It is still elusive to apply these ideas on 3D point clouds where 3Dobjects are far more complicated and diverse in terms of geometry.2D Scene Flow Estimation and Motion Segmentation:
Given image sequences, pixel-level 2Dscene flow, also known as optical flow, have been extensively studied in literature [16, 77].
Theestimated flow field can be further grouped as objects [58, 59, 9, 42, 72, 38, 29].
Drawing insightsfrom these works, this paper aims to segment multiple diverse objects in the complex 3D space.
OGC
3.1 Overview
As shown in Figure 2, given a single point cloud P t with N points as input, i .
e ., P t ∈ R N × 3 , whereeach point only has a location { x, y, z } without color for simplicity, the neural backbone extractsper-point features and directly reasons a set of object masks, denoted as O t ∈ R N × K , where K is apredefined number of objects that is large enough for a specific dataset.
In particular, we firstly adopt PointNet++ [54] to extract the per-point local features.
In order to infer object masks from the pointfeatures, we simply plug the recent MaskFormer [8] architecture which shows excellent performancein object segmentation in 2D images.
Thanks to the powerful Transformer module [63], each objectmask inferred by MaskFormer is effectively modeled over the entire point cloud.
In the meantime, we have the corresponding sequence of point clouds for supervision, denoted as{ P t , P t +1 , · · · } .
For simplicity, we only use the first two frames { P t , P t +1 } and feed them into theauxiliary self-supervised scene flow network , obtaining satisfactory motion vectors for every pointin the first point cloud frame, denoted as M t ∈ R N × 3 , where each motion vector represents pointdisplacement { ∆ x, ∆ y, ∆ z } .
Among the existing self-supervised scene flow methods, we choosethe recent FlowStep3D [31] which shows excellent scene flow estimation in multiple datasets.
Implementation details of above neural components are in appendix.
Having the estimated objectmasks and scene flow at hand, the key component is the supervision mechanism to drive predictedmasks to be true objectness, which is discussed in the following section.
3.2 Object Geometry Consistency Losses
Given the input cloud P t and its estimated object masks O t and motion M t , we introduce thefollowing objectives to satisfy the geometry consistency on both frames P t and ( P t + M t ).
(1) Geometry Consistency over Dynamic Object Transformations
From time t to t + 1 , the rigid objects in point cloud frame P t usually exhibit different dynamictransformations which can be described by matrices belonging to SE (3) group.
For the k th object,we firstly retrieve its (soft) binary mask
O tk , and then feed the tuple { P t , P t + M t , O tk } into thedifferentiable weighted-Kabsch algorithm [28, 18], estimating its transformation matrix T k ∈ R 4 × 4 .
In order to drive all object masks to fully explain the corresponding motion patterns within all masks,the following dynamic rigid loss is designed to minimize the discrepancy of every point scene flowbetween time t and t + 1 .
Formally, it is defined as:
[Equation]
where the operation ◦ represents the rigid transformation of point clouds.
Intuitively, if one inferredobject mask happens to include two sets of points with two different moving directions, the transformed point cloud can be only in favor of one moving direction, thereby resulting in higher errors.
Therefore, the above constraint can push all object masks to fit the dynamic and diverse motionpatterns.
However, here arises a critical issue: a single rigid object may be assigned to multiple masks,i .
e .
oversegmentation.
We alleviate this issue by a simple smoothness regularizer discussed below.
[Equation]
The primary reason why a single object may be oversegmented is the lack of spatial connectivitybetween individual points.
However, our common observation is that physically neighbouring pointsusually belong to a single object.
In this regard, we simply introduce a geometry smoothnessregularizer.
Particularly, for a specific n th 3D point p n in the point cloud P t , we firstly search Hpoints from its neighbourhood using either KNN or spherical querying methods, and then force theirmask assignments to be consistent with the center point p n .
Mathmatically, it is defined as:
[Equation]
where o p n ∈ R 1 × K represents the object assignment of center point p n , and o p hn ∈ R 1 × K representsthe object assignment of its h th neighbouring point.
The distance function d () is flexible to choosethe simple L 1 and L 2 or a more aggressive cross-entropy function.
(3) Geometry Invariance over Scene Transformations
With the above geometry constraints designed in (1)(2), the shapes of dynamic objects can bereasonably segmented.
However, the learned object geometry may not be general enough.
Forexample, a moving car can be well segmented, yet another similar parked car may not be discovered.
To this end, we introduce an object geometry invariance constraint as follows:
• Firstly, given P t , we apply two transformations to get augmented point clouds P tv 1 and P tv 2 .
• Secondly, we feed the two augmented point clouds into the neural backbone, obtaining two sets of object masks O tv 1 and O tv 2 .
Because the per-point locations in two point clouds are transformed
differently, the position sensitive MaskFormer [63] naturally generates two different sets of masks.
• Thirdly, we leverage the Hungarian algorithm [33] to one-one match the individual masks in O tv 1and O tv 2 according to the object pair-wise IoU scores.
Basically, this is to address the issue that
there is no fixed order for predicted object masks from the two augmented point clouds.
• At last, we reorder the masks in O tv 2 to align with O tv 1 , and design the invariance loss as follows.
[Equation]
where o nv 1 and o nv 2 are the object assignments of the two augmented point clouds for a specific n thpoint.
The distance function ˆ d () is flexible to use L 1 , L 2 or cross-entropy.
Ultimately, this loss drivesthe estimated object masks to be invariant with different views of input point clouds.
3.3 Iterative Optimization of Object Segmentation and Motion Estimation
With the designed geometry consistency loss functions, the neural backbone for object segmentationis optimized by the combined loss: ℓ seg =
ℓ dynamic + ℓ smooth + ℓ invariant .
The auxiliary selfsupervised scene flow network FlowStep3D [31] is independently trained by its own losses untilconvergence.
Intuitively, with better and better object masks estimated, the estimated scene flow isalso expected to be improved further if we use the masks properly.
An ideal solution is to directlypass the estimated object masks into the scene flow network.
However, this is non-trivial because weneed to either adapt the scene flow network layers or add extra loss functions, and thus we leave itfor our further exploration.
In this paper, we instead propose a following Algorithm 1 to iterativelyimprove object segmentation and motion estimation.
• Independently and fully train the self-supervised scene flow network on the whole training data split, and obtain reasonable scene flow estimations: { ( P t , P t +1 , M t ) 1 · · ·
P t , P t +1 , M t ) S } . R
optimization.
Train the object segmentation network using ℓ seg for a total E epochs on the whole training split: { ( P t , P t +1 , M t ) 1 · · · ( P t , P t +1 , M t ) S } .
• Estimate reasonable object masks: { ( P t , P t +1 , M t , O t ) 1 · · · ( P t , P t +1 , M t , O t ) S } .
Stage 2: Scene flow improvement.
• For each pair of data ( P t , P t +1 , M t , O t ) , by drawing insights from the classical ICP [2], we propose an object-aware ICP algorithm to estimate new scene flow ˆ M t for point cloud P t .
• Update the new scene flow for next round training: { ( P t , P t +1 , M t ) 1 · · · ( P t , P t +1 , M t ) S } ← { ( P t , P t +1 , ˆ M t ) 1 · · · ( P t , P t +1 , ˆ M t ) S }
Empirically, setting the total number of rounds R to be 2 or 3 has a good trade off between accuracy and efficiency.
Due to the limited space, details of object-aware ICP algorithm is in appendix.
3.4 Implementation
Our object segmentation network is implemented in PyTorch and Adam optimizer with a learningrate of 0.001 is adopted.
All experiments are conducted on a single RTX3090 GPU.
For the geometrysmoothness regularization, the choice of neighbouring points H is specified in appendix for differentdatasets.
For the geometry invariance loss, the augmentation method is also detailed in appendix.
Experiments
Our method is evaluated on three different scenarios: 1) part instance segmentation of articulatedobjects on SAPIEN dataset [70], 2) object segmentation of indoor scenes on our own synthetic dataset,and 3) object segmentation of real-world outdoor scenes on KITTI datasets [45].
For evaluationmetrics, we follow [48] and report the F1-score , Precision , and Recall with an IoU threshold of 0.5.
In addition, we also report the Average Precision ( AP ) score following COCO dataset [37] and the Panoptic Quality ( PQ ) score defined in [30].
The mean Intersection over Union ( mIoU ) score andthe Rand Index ( RI ) score implemented in [24] are also included.
Note that, all metrics are computedin a class-agnostic manner.
4.1 Evaluation on SAPIEN Dataset
The SAPIEN dataset [70] provides 720 simulated articulated objects with part instance level annotations.
Each object has 4 sequential scans.
The part instances have different articulating (moving)states.
We follow [24] to use the training data generated from [75].
In particular, there are 82092pairs of point clouds for training, 2880 single point cloud frames for testing.
Each point cloud isdownsampled to 512 points in both training and testing.
Since there is no existing unsupervised method for multi-object segmentation on 3D point clouds, wefirstly implement two classical clustering methods: WardLinkage [27] and DBSCAN [15] to directly [Table] group 3D points from single point clouds into objects.
Secondly, we implement two classical motionsegmentation methods: TrajAffn [49] and SSC [48].
Note that, these two methods take the sameestimated scene flows of FlowStep3D as input, while our method uses the estimated scene flowsduring training only but takes single point clouds during testing.
In addition, we also include theexcellent results of several fully-supervised methods (PointNet++ [54], MeteorNet [40], DeepPart[74]) reported in MBS [24].
Their experimental details can be found in MBS [24].
Lastly, we trainour object segmentation backbone using single point clouds with full annotations, denoted as OGC sup .
All implementation details are in appendix.
Analysis: As shown in Table 1, our OGC surpasses the classical clustering based and motionsegmentation methods by large margins on all metrics, showing the advantage of our method in fullyleveraging both the motion patterns and various types of geometry consistency.
Compared with thefully supervised baselines, our method is only inferior to the strong MBS [24] and OGC sup .
However,we observe that our OGC actually shows a higher precision score than OGC sup , primarily becauseour method tends to learn better objectness thanks to the additional motion pattern constraints andavoid dividing a single object into pieces.
Figure 3 shows qualitative results.
4.2 Evaluation on Indoor Scene Dataset
We further evaluate our method to segment objects in indoor 3D scenes.
Considering that the existingdataset FlyingThings3D [44] tends to have unrealistically cluttered scenes with severely fragmentedobjects and it is originally introduced for scene flow estimation, we turn to synthesize a new dynamicroom dataset, called OGC-DR , that suits both scene flow estimation and object segmentation.
Inparticular, we follow [51] to randomly place 4 ∼ 8 objects belonging to 7 classes of ShapeNet [6]{chair, table, lamp, sofa, cabinet, bench, display} into each room.
In total, we create 3750, 250, and1000 indoor rooms (scenes) for training/validation/test splits.
In each scene, we create rigid dynamicsby applying continuous random transformations to each object and record 4 sequential frames forevaluation.
Each point cloud frame is downsampled to 2048 points.
Note that, we follow [11] to splitdifferent object instances for train/val/test sets.
More details of the dataset are in appendix and thisdataset will be released.
Analysis: As shown in Table 2, our method outperforms all classical unsupervised methods includingthe clustering based and the motion segmentation based methods.
Since our synthetic rooms all havecomplete 3D objects, and the generated point cloud sequences are of high quality.
Therefore, theperformance gap between our OGC and the supervised OGC sup is nearly closed.
This shows that therigid dynamic motions can indeed provide sufficient supervision signals, if not more than the humanannotations, to identify objects.
Figure 3 shows qualitative results.
4.3 Evaluation on KITTI Benchmarks
We additionally evaluate our method on the challenging real-world outdoor dataset.
In particularly,we conduct the following two groups of experiments on two benchmarks of KITTI dataset.
• Group 1: Officially, KITTI-SF (Scene Flow) dataset [45] consists of 200 (training) pairs of point clouds from real-world traffic scenes and an online hidden test for scene flow estimation. In
[Table] our experiment, we train our pipeline on the first 100 pairs of point clouds, and then test on theremaining 100 pairs (200 single point clouds).
We observe that in the 100 training pairs, themoving objects are only cars and trucks.
Therefore, in the testing phase, we only keep the humanannotations [1] of cars and trucks in every single frame to compute the scores.
All other objects aretreated as part of background.
Note that, the whole background is not ignored, but counted as oneobject in our evaluation.
• Group 2:
Given our well trained model in Group 1, we directly test it on the popular single frame based KITTI 3D Object Detection benchmark [17].
In this benchmark, there are officially point clouds for training, 3769 for validation.
Again, we only keep the ground truth object masks obtained from the bounding boxes for the car category in each frame.
All other objects are treated as part of background.
For comparison, we download the official pretrained models of three fully-supervised methods PointRCNN
PV-RCNN [55] and Voxel-RCNN [12] to directly teston the validation split using the same settings as ours.
In addition, we also directly use the welltrained OGC sup in Group 1 to directly test for comparison, denoted as OGC* sup .
At last, we trainthe OGC sup on the training split (3712 frames) from scratch and test it on the remaining 3769frames using the same evaluation settings, denoted as OGC sup . [Table] Supervised Methods [Table]
Unsupervised
Motion Segmentation
[Table] UnsupervisedMethods [Table] [Table]
SupervisedMethods
PointRCNN [56]
[Table]
PV-RCNN [55]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
Voxel-RCNN [12] [Table] [Table] Analysis: As shown in Table 3, the classical unsupervised clustering based and motion segmentationbased baselines catastrophically fail on the challenging KITTI-SF dataset.
Primarily, this is because
SSC cannot handle the extreme imbalance of 3D points between objects and background.
WardLinkage and DBSCAN favor spatially separated objects, but the background and objects in KITTI-SFare always connected because of the Earth’s gravity.
By comparison, our method obtains superiorsegmentation scores, being very close to our fully-supervised counterpart OGC sup .
As shown in
Table 4, our method can directly generalize to the single frame based 3D object segmentation withsatisfactory results, also being close to the fully-supervised counterpart OGC* sup .
It is understandablethat the other three fully-supervised models have a clear advantage over ours, because they are fullytrained on the similar training split while ours does not.
We hope that our method can serve the first baseline and inspire more advanced unsupervised methods in the future to close the gap.
Figure 3shows qualitative results.
4.4 Ablation Study
[Table]
(1) Geometry Consistency Losses: Tovalidate the choice of our design, wefirstly conduct three groups of ablativeexperiments on the SAPIEN dataset [70]:1) only remove the dynamic rigid lossℓ dynamic , 2) only remove the smoothness loss ℓ smooth , and 3) only removethe invariance loss ℓ invariant .
As shownin Table 5, combining the proposed three losses together gives the highest segmentation scores.
Basically, the dynamic rigid loss serves to discriminate multiple objects from different motion patterns.
Without it, the network tends to assign all points to a single object as the shortcut to minimize theother two losses.
However, we observe in Table 5 that without ℓ dynamic , the network still works tosome extend.
This is because the synthetic SAPIEN dataset tends to have a number of point cloudframes with only 2 or 3 objects, thus assigning all points to a single object can still get plausiblescores.
This issue is further validated by conducting additional ablation experiments on curated
SAPIEN dataset.
More details are in appendix.
[Table]
[Table]
(2) Iterative Optimization Algorithm:
We also conduct ablative experiments tovalidate the effectiveness of our proposed
Algorithm 1.
We set the number of iterative rounds R as { 1 , 2 , 3 } .
As shownin Table 6, after 2 rounds, satisfactorysegmentation results can be achieved,although we expect better results aftermore rounds with longer training time.
This shows that our iterative optimization algorithm canindeed fully leverage the mutual benefits between object segmentation and motion estimation.
4.5 Pushing the Boundaries of Unsupervised Scene Flow Estimation and Segmentation
[Table]
In addition to the improvement of object segmentation from our iterative optimization algorithm, the scene flow estimation can be naturally further improvedfrom our estimated object masks as well.
Given our well trained model on the
KITTI-SF dataset in Section 4.3, we usethe estimated object masks to further improved the scene flow estimation.
As shown in Table 7, following the exact evaluation settingsof FlowStep3D [31], our method, not surprisingly, significantly boosts the scene flow accuracy,surpassing the state-of-the-art unsupervised FlowStep3D [31] and other baselines in all metrics.
[Table]
In fact, our object segmentation backbone is also flexible to take the sceneflow as input instead of point xyz to segment objects.
This is commonly calledmotion segmentation.
We replace thesingle point clouds by (estimated) sceneflow vectors as our network inputs, and train the network from scratch using the same settings on the
KITTI-SF dataset.
As shown in Table 8, we can see that our network can still achieve superior resultsregardless of the modality of inputs, demonstrating the generality of our framework. TrajAffn SSC WardLinkage
DBSCAN
OGC 𝑠𝑢𝑝
OGC(Ours) GT TrajAffn SSC
WardLinkage
DBSCAN
OGC 𝑠𝑢𝑝
OGC(Ours)
GT
S A P
I
E N
OG CDR
K I TT
I - S F
K I TT
I - D e t G T
OG C ( O u r s )  Conclusion In this paper, we demonstrate for the first time that 3D objects can be accurately segmented usingan unsupervised method from raw point clouds.
Unlike the existing approaches which usuallyrely on a large amount of human annotations of every 3D object for training networks, we insteadturn to leverage the diverse motion patterns over sequential point clouds as supervision signalsto automatically discover the objectness from single point clouds.
A series of loss functions aredesigned to preserve the object geometry consistency over spatial and temporal scales.
Extensiveexperiments over multiple datasets including the extremely challenging outdoor scenes demonstratethe effectiveness of our method.
One limitation of our method is the lack of capability to discoverunseen object types, and we leave it for our future exploration.
Broader Impact:
The proposed OGC learns 3D objects from raw point clouds without requiringhuman annotations for supervision.
We showcase the effectiveness for some basic applicationsincluding object part instance segmentation, indoor object segmentation and outdoor moving vehicleidentification.
We also believe that our method can be general for other domains such as AR/VR.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] SeeSection 5(d)
Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running exper-
iments multiple times)?
[No] Error bars are not reported because it would be too
computationally expensive
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Section 3.44.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]