  Despite rapid progress in scene segmentation in recent years, 3D segmentation
methods are still limited when there is severe occlusion. The key challenge is
estimating the segment boundaries of (partially) occluded objects, which are
inherently ambiguous when considering only a single frame. In this work, we
propose Multihypothesis Segmentation Tracking (MST), a novel method for
volumetric segmentation in changing scenes, which allows scene ambiguity to be
tracked and our estimates to be adjusted over time as we interact with the
scene. Two main innovations allow us to tackle this difficult problem: 1) A
novel way to sample possible segmentations from a segmentation tree; and 2) A
novel approach to fusing tracking results with multiple segmentation estimates.
These methods allow MST to track the segmentation state over time and
incorporate new information, such as new objects being revealed. We evaluate
our method on several cluttered tabletop environments in simulation and
reality. Our results show that MST outperforms baselines in all tested scenes.
