Title: Point-MoE: Large-Scale Multi-Dataset Training with Mixture-of-Experts for 3D Semantic Segmentation
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: 3D Semantic Segmentation, Mixture of Expert, Point Cloud Understanding

Abstract: Massively scaling data and models has transformed NLP and 2D vision, but the impact on 3D point clouds remains limited. We take a first step toward scalable 3D learning under a realistic regime: large-scale, multi-dataset joint training for 3D semantic segmentation without dataset labels at inference. Point clouds span diverse sensors (depth cameras, LiDAR) and scenes (indoor, outdoor), leading to heterogeneous sampling and semantic biases; naïvely mixing such data degrades standard backbones. We introduce Point-MoE, a Mixture-of-Experts design that adds capacity via sparsely activated expert MLPs and a lightweight top-k router, letting tokens select specialized experts without dataset supervision. Trained jointly on a diverse mix of indoor and outdoor datasets and evaluated on seen datasets and in zero-shot settings, Point-MoE outperforms prior methods without using dataset labels for training or inference. Token routing promotes expert specialization with efficient inference and modest training overhead. This suggests a scalable path for 3D perception: let the model discover structure in heterogeneous 3D data rather than enforce it through manual curation or dataset-specific heuristics.

TL;DR: Point-MoE scales 3D semantic segmentation by routing tokens to sparse experts during multi-dataset training, improving accuracy without dataset labels at train or test time.