Point-MoE: Towards Cross-Domain Generalization in 3D Semantic
  Segmentation via Mixture-of-Experts
Authors: name: Xuweiyi Chen; name: Wentao Zhou; name: Aruni RoyChowdhury; name: Zezhou Cheng
Published: 2025-05-29 18:21:47

Abstract: While scaling laws have transformed natural language processing and computer
vision, 3D point cloud understanding has yet to reach that stage. This can be
attributed to both the comparatively smaller scale of 3D datasets, as well as
the disparate sources of the data itself. Point clouds are captured by diverse
sensors (e.g., depth cameras, LiDAR) across varied domains (e.g., indoor,
outdoor), each introducing unique scanning patterns, sampling densities, and
semantic biases. Such domain heterogeneity poses a major barrier towards
training unified models at scale, especially under the realistic constraint
that domain labels are typically inaccessible at inference time. In this work,
we propose Point-MoE, a Mixture-of-Experts architecture designed to enable
large-scale, cross-domain generalization in 3D perception. We show that
standard point cloud backbones degrade significantly in performance when
trained on mixed-domain data, whereas Point-MoE with a simple top-k routing
strategy can automatically specialize experts, even without access to domain
labels. Our experiments demonstrate that Point-MoE not only outperforms strong
multi-domain baselines but also generalizes better to unseen domains. This work
highlights a scalable path forward for 3D understanding: letting the model
discover structure in diverse 3D data, rather than imposing it via manual
curation or domain supervision.