Point-MoE: Large-Scale Multi-Dataset Training with Mixture-of-Experts for 3D Semantic Segmentation
Authors: Xuweiyi Chen
Wentao Zhou
Aruni RoyChowdhury
Zezhou Cheng
Venue: ICLR 2026 Poster (2026)
Keywords: 3D Semantic Segmentation
Mixture of Expert
Point Cloud Understanding
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: Point-MoE scales 3D semantic segmentation by routing tokens to sparse experts during multi-dataset training, improving accuracy without dataset labels at train or test time.

Abstract: While massively both scaling data and models have become central in NLP and 2D vision, their benefits for 3D point cloud understanding remain limited. 
We study the initial step of 3D point cloud scaling under a realistic regime: large-scale multi-dataset joint training for 3D semantic segmentation, with no dataset labels available at inference time. 
Point clouds arise from a wide range of sensors (e.g., depth cameras, LiDAR) and scenes (e.g., indoor, outdoor), yielding heterogeneous scanning patterns, sampling densities, and semantic biases; naively mixing such datasets degrades standard models. 
We introduce **Point-MoE**, a Mixture-of-Experts design that expands capacity through sparsely activated expert MLPs and a lightweight top-$k$ router, allowing tokens to select specialized experts without requiring dataset supervision. 
Trained jointly on a diverse mix of indoor and outdoor datasets and evaluated on seen datasets and in zero-shot settings, Point-MoE outperforms prior methods without using dataset labels for either training or inference.
This outlines a scalable path for 3D perception: letting the model discover structure in heterogeneous 3D data rather than imposing it via manual curation or dataset-specific heuristics.

Published as a conference paper at ICLR 2026
POINT-MOE: LARGE-SCALE MULTI-DATASET TRAIN- ING WITH OFRTURE--F-EXPEER FOR 3D SEMANTIC SEGMENTATION
Xuweiyi Chen'
Wentao Zhou'
Aruni RoyChowdhury²
Zezhou Cheng
University of Virginia
2 *MatWorks
https://point. nopiiintmmet cs virginia.edu
ABSTRACT
While massively scaling both data and models have become central in NLP and 2D vision, their benefits for 3D point cloud understanding remain limited. We study the initial step of scaling 3D point cloud understanding under realistic regime: large-scale multi-dataset joint training for 3D semantic segmentation, with no dataset labels available at training or inference time. Point clouds arise from a wide range of sensors (e.g., depth cameras, LiDAR) and scenes (e.g., indoor, outdoor), yielding heterogeneous scanning patterns, sampling densities, and semantic biases; naively mixing such datasets degrades standard models. Therefore, we introduce Point-MoF, a Mixture-of of-pxertss design that expands model capacity through sparsely activated expert MLPs and lightweight top-k router, allowing tokens select specialized experts without requiring dataset supervision. Trained jointly on diverse mix of indoor and outdoor datasets, and evaluated seen datasets well as in zero-shot settings, Point-MoB outperforms prior methods without using dataset labels for either training or inference. This outlines scalable path for 3D perception: letting the model discover structure in heterogeneous 3D data rather than imposing it via manual curation dataset-specifie heuristics.
INTRODUCTION
In both language and vision, we have witnessed remarkable advances driven by two trends: the aggregation of massive, heterogeneous datasets [4, 8, 58, 61, 66, 97], and the deployment of ever- larger models trained on them in unified manner [7, 16, 21, 36, 51, 57]. These models succeed not because they are crafted for a specific dataset, but because they learn underlying regularities across multiple datasets. However, 3D point cloud understanding has not yet followed the same trajectory, despite its wide applications in robotics [11, 53, 90], autonomous systems [12, 433, augmented example, although diverse 3D datasets exist-such as ScanNet [19]. SemanticKITTT [5], nuScenes [9], and ASE each covers only a narrow slice of real- world variation. Point clouds arise from diverse pipelines (RGB-D, LiDAR, multi-view stereo), each with their own distinct densities, artifacts, and semantic biases (Fig. left). The result is rich yet siloed ecosystem of 3D models, where models trained on one domain often fail to generalize to others.
reality [6, 77], and embodied intelligence [72, 79]. Taking 3D) semantic segmentation
train across silos, presumably resulting single set of model parameters that is effective across the spectrum of datasets. One might, for example, naively train the state- of-the-art Point Transformer V3 (PTv3) [81] (Fig. la) on a mixture of datasets from multiple domains, e.g. indoor and outdoor. In practice, however, such models struggle to reconcile the heterogeneity of the data distribution even as capacity scales (Tab. 1). To mitigate this, recent methods have introduced dataset-aware components. Point Prompt Training (PPT) [82] (Fig. 1b) assigns dataset-specific normalization layers, while Ane-for-All [75] uses a lightweight dataset classifier with dataset-specific adapters. While effective, these methods rely dataset labels. identifying which specific dataset an input sample originated from, during training and inference. Moreover, adapting only the normalization parameters may be insufficient for large-scale multi-dataset training.
natural response

---

Published as a conference paper at ICLR 2026
Calegorical Aligument
Categorical Alignment
Categorical A Alignment
xM
XM
Dataset
MLP
MLP
MLP
LayerNorm
LayerNorm
LayerNorm
Expert
Projection
Projection
Softmax
Softmax
Rase
K
V
Sottmax
Projection
Projection
Prejection
Dataset C
LayerNorm
LayerNorm
LayerNocm
BatchNomm
BarchNorm
BatchNont
Grdd Foeting & Grid Unpasiing
Grdd Poeling & Gnd Lipoolin
Grd hailng &e Ginal Lnppoiinn
Senalizatin Embedding
Scrialiaztion Enbeaking
Domais
Seiialiation & Embelllng
Multi-domain
(a)
(b) PPT
(c) Point-MoE
Overview of multi-datasets training architectures. Point clouds exhibit diverse character- isties across datasets. (a) Naively training Point Transformer V3 (PTv3) [81] on multi-datasets data leads to degraded performance within each domain. (b) Point Prompt Training (PPT) [82] addresses this by adding dataset-aware normalization parameters. (c) Our proposed Point-MoE tackles this challenge with Mixture-of-Eaaee (MoE), enabling dynamic expert specialization across datasets.
Figure 1:
At deployment, point clouds arrive from mixed, unlabeled sources with unknown provenance, i.e. there is no oracle "dataset ID" available during inference. Accordingly, we study a realistic regime: large-seale multi-dataset joint training for semantic segmentation with no dataset labels at inference. We hypothesize that Mfxture-of-Fxppets (MoE) architectures [30, 34] are well-suited to this setting. We introduce Point-MoE, a sparse MoE model built upon PTv3 [8]]. As illustrated in Fig. lc. Point-MoE replaces each attention projection layer in each PTv3 block with a MoE module composed of expert MLPs and a lightweight router that selects a sparse top-k subset of experts per token. Our design is motivated by three insights. First, MoE encourages expert specialization and token-to-expert to-expert routing [15, 87], enabling a single model to learn from mixed datasets without access dataset labels training test time. Second, compared to normalization- based adaptations such as PPT [82] and One-for-All [75], MoE offers far more expressive capacity to model -ppascc- variations. Third, sparse MoE architectures maintain computational efficiency during training and inference, and are standard mechanism for scaling large models in NLP and 2D vision [20, 22, 24, 40, 59]. We extend these benefits, well-established in language and 2D) vision, to 3D point cloud understanding, focusing on 3D semantic segmentation. We conduct extensive experiments across a diverse collection of 3D datasets spanning indoor and outdoor scenes, synthetic and real datasets, and multiple sensor modalities. We evaluate both in-domain and zero-shot datasets, all without dataset labels. To comprehensively assess Point-MoE, we evaluate both in- domain (seen) datasets and out-of-domain (zero-shot) datasets, all without access dataset labels. Our goal is to demonstrate that Point-MoE not only achieves strong performance on seen datasets but also generalizes robustly to unseen distributions, without relying on dataset-specific augmentation supervision. Across both regimes, Point-MoE consistently outperforms competitive multi-dataset baselines. We further ablate the number of experts, sparsity level, and MoE placement (Tab. 3), and find that Point-MoE learns coherent routing patterns and exhibits organic expert specialization driven by the underlying data distribution (Fig. 3,
To the best of our knowledge, this work provides the first systematic study of MoB for 3D point cloud understanding under the large-scale multi-datased training regime. Our contributions are: (1) we focus on multi-datasel 3D semantic segmentation without domain labels, using a single model, and put forth Mixture-of -ffreerrt----fsse--eeee as natural solution to this task; (2) we carefully explore the MoE design space and provide extensive ablations that highlight the trade-offs and inform effective configurations; (3) we demonstrate that Point-MoE achieves state-of-ther oftt--rrtttttttt performance across seven datasets under this protocol while maintaining inference efficiency; and (4) we analyze token-level routing trajectories and expert usage, revealing encoder-decodd decoder specialization patterns consistent with 3D geometry. Together, these findings point toward a scalable path for 3D perception: rather than building separate models for each dataset or domain, a single unified system can adapt across the spectrum of 3D data sources, leveraging scaling laws for data and compute [68].
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
Point Cloud understanding. Deep learning backbones for 3D scene understanding can be broadly categorized by how they process point clouds: projection- based methods [13, 41, 64), voxel-based methods [17, 25, 49, 63], and point-based methods [54, 55, 69, 93]. More recently, transformer-basee
architectures operating on point clouds have emerged as strong contenders in this space [27, 80, 81, 94]. Among them, Point Transformer V3 (PTv3) [81] represents the current state of the art, introducing a novel space-filling curve based serialization scheme, that transforms the unstructured modeling. Recent works [32, 39, 82, 83, 96] scale point cloud understanding models to multi-dataset settings and show that large-scale point cloud data can improve performance on specific datasets: Sonata [83] explores multi-dataset self-supervised learning, but does not train across indoor and outdoor datasets. On the other hand, Point Prompt Training (PPT) [82] demonstrates that using dataset-specific normalization layers during pretraining improves 3D semantic segmentation performance across multiple datasets along with battast-speccfii fine-tuning. In 3D detection, One-for-All [75] adopts a sparse convolutional network with dataset-specific partitions and normalization layers to boost multi-datasets performance, and UniDet3D [37] achieves state-of-the-ar performance with multi-dataset joint training by merging label spaces across datasets into unified taxonomy and building upon vanilla transformer [71] architecture. Rather than relying on dataset-specific preprocess or architectural branches [75, 82], we propose a unified sparse expert framework that learns to specialize dynamically across various datasets, without requiring explicit dataset labels during training inference.
3D points into a ID sequence, enabling efficient transformer-basee
Texture-of-Experte Mixture-off toeeeefff---tettt- (MoE) [30, 34] increases model capacity without propor- tional compute by activating only a subset of experts per input [62]. This conditional computation has powered trillion-scale NLP models [20, 22, 24, 33, 40, 46, 47, 98] and demonstrated similar efficiency in vision [59]. Beyond scaling, MoE is naturally suited for heterogeneous multi-datasets data, as experts tend to self- self-orgamize by dataset or task. Prior work has shown improvements in multi- source sentiment analysis and POS tagging [26]. dataset-adaptive language modeling [28], automatic chart understanding [87], and multilingual translation [15]. Mod-Squad [14] introduced modular expert routing for multi-task learning, balancing cooperation and specialization. Similar trends are observed in vision and vasion-language applications: DAMEX [31], MoE-LLaVA [44] and DeepSeek- VL [48, 85J. To our knowledge, this is the first work to extend MoE to multi multi-dataset training in 3D point clouds. MoE3D [76] recently introduces MoE formulation for 3D reconstruction.
Domain generalization. Classical domain generalization (DG) trains on one or more labeled source domains and targets zero-shot performance on specified target domain, typically in sensor- and cownntii-onnoloy--ommgeeo
outdoor LiDAR settings [35, 60, 84, 86, 88, 92]. We instead study large-scale joint training heterogeneous datasets with naturally misaligned label spaces, evaluating a single model on both seen and zero-shot datasets. Unlike DG, our goal is a single unified model for heterogeneous multi- dataset learning rather than transfer fixed target domain, although DG methods remain complementary and could be layered onto Point-MoE as a natural extension.
POINT-MOE FOR MULTI-DATASET POINT CLOUD TRAINING
3.1 LARGE-SCALE MULTI-DATASET JOINT TRAINING
[p.) where each point p: E 3 denotes XYZ coordinates, semantic segmentation aims to predict a single class label ÿ E for each point p; from fixed label set C = {cI, c....., Cm). }. Predicted labels for the entire point cloud are compactly denoted as y. Similarly, we denote the corresponding ground- -ruth labels by
Task definition. Given a 3D scene represented as point cloud P= -=(p,)
y [y, U2. ... ysr) Y C, where y; is the true semantic label of point pi.
Let a dataset containing multiple point clouds be denoted as D =
The complete
training set across multiple datasets is then D {D,} p The goal of our multi-dataset joint training is to learn unified model (f.e. single set of model parameters, without resorting per-dataset fine- tuning) over all datasets that minimizes prediction error across ID, while also generalizing effectively to shoo datasets drawn from distributions unseen at training time.
3

---

Published as a conference paper at ICLR 2026
Let Dg denote segmentation model with parameters 6, that takes in a point cloud P and predicts per-point class labels y. Let L(y. 3) denote the loss, e.g. cross-entropy, on a point cloud between ground-truth segmentation labels y and predicted labels y. Formally, the training objective is ming EDD inference. In other words, do not know which dataset sample is drawn from.
[4(y, (p))) We do not assume oracle dataset labels at training or
We can formally contrast this to alternate multi-dataset objective formulations such as [75, 82], that learn an explicit set of dataset-specific parameters wj. along with 0:
ming,(wy ED,-D y)-D, [C(y, Lo,,, (P)) PPT [82] instantiates {w;},=) ,=1 as the learnable gain and bias terms of the normalization layers of PTv3, trained on a per-dataset basis. During inference, the source dataset Dj of an incoming point cloud P must be identified in order to apply the correct normalization Wj. usually obtained from a lightweight dataset classifier as in One-for-All [75].
Language-guided classification. To bridge label discrepancies across datasets (i.e. C₁ * Cj for datasets D, and D) we project per-point features text embeddings, following PPT [82]. This enables supervision via class names, without relying on dataset labels. For instance, the class "pillow" is missing in ScanNet (grouped as "other") but explicitly labeled in Structured3D; aligning to the CLIP embedding of "pillow" transfers knowledge across datasets even without direct supervision. All baselines and our Point-MoE are adapted to this language- guided protocol, ensuring fair comparison.
shared language space using CLIP [57]
3.2 POINT-MOE
Cixture-of-Expertt (MoE) layer. Mixture-of-Exper [30] is conditional computation architecture enabling significant capacity expansion while only modestly increasing computational cost. The core of MoE is to route input tokens dynamically to specialized subnetworks, referred to experts, via lightweight gating mechanism. Given an input feature vector x E RDD each MoE layer contains N experts {fr J. where ft: PP + R2 and gating network G RD RN For each input x, the MoE layer output is computed as a weighted sum over the top-k activated experts:
MoF(x)
Σ Gx(x) VES
where Sz is the top-k expert set and Gx(x) are normalized gate weights. In practice, is a linear projection followed by sparse softmax, activating only & experts. Importantly, MoE introduces a large design space, including the choice of which components to replace, the number of experts, the routing strategy, whether to do auxiliary balancing loss and whether use top-1 or top-2 gating. We systematically study how different MoE configurations affect performance on 3D point clouds through detailed ablations in the experiments.
Integration into PTv3. Given point cloud P with M points, PTv3 [81] serializes and embeds the input into token sequence Z [æ1..... 21 PTv3 then applies stacked multi-head self-attention (MHSA) blocks interleaved with up- and down- wown-ssmpling for multi-scale reasoning on these por-token embeddings. At any intermediate layer f, let x, E MxD denote the token sequence at layer f, where D is the feature dimension, H is the number of heads in multi-head attention, and FFN denotes feed-forward network. The computations of a self-attention block using standard QKV-formulatee and pre-norm residuals is as follows:
X = Norm(x--1).
Q = XW g: K = XW V = XW.
(2)
Softmax QK /vna)
Vd) V, dn = D/H, H,
(3)
O W.A,
(4)
X Xx-1 +
(5)
X X + FFN(Nor((())
(6)
The learnable query, key, and value projections are denoted as Wg, W. and W., while W. denotes the projection applied on the attention output A. We apply MoE only at the attention output projection (W.) in every block, rewriting Eq. (4) as O MoE(A); the query, key, and value projections (W WR. W.) remain dense. Empirically, placing MoE at W, yields better performance than placing it in the FFN under the same expansion (Tab. 3d). We hypothesize that the gain comes from adding
4

---

Published as a conference paper at ICLR 2026
nonlinearity and capacity the projection path: with purely linear projections. the composition of W. and Wo can be merged into a single projection [47].
Beyond MoE-layer placement, employ from both indoor and outdoor data (Tab. 4). Having tokens from multiple datasets within a single minibatch, and thus contributing to the gradients of one update step, is useful in facilitating cross- sample interaction and emergent expert specialization.
training: each minibatch jointly samples
To illustrate the general applicability of this framework, we further extend MoE to OA-CNNs [52], sparse convolution backbone. MoE--aggented OA-CNNs likewise outperform the dense version, further underscoring the effectiveness and general applicability of MoE as it tied to a specific backbone architecture. Detailed results are in the supplementary material under Sec. B.2.
EXPERIMENTS
We conduct comprehensive experiments to evaluate Point-MoF for 3D semantic segmentation. First, we describe baseline methods and experimental setups 411c...1). Then, we present detailed empirical results under two primary joint training scenarios: (1) indoor-only, and (2) combined indoor-outdoor training (Sec. 4.2). Finally, we provide ablation studies that highlight key design choices of the MoE architecture (Sec. 4.3) and detailed analysis of MoE behaviors (Sec. 4.4).
4.1 EXPERIMENTAL SETUP
Datasets. We conduct experiments on three indoor benchmarks: ScanNet [19], S3DIS [1]. and Structured3D [95], and additionally evaluate zero-shot on Matterport3D [10]. Specifically, we train a single model on ScanNet, S3DIS, and Structured3DD, and directly evaluate its performance on both seen and unseen datasets via the innuuage-giided head using the same checkpoint. We further test the best-performing Point-MoE architectures and baseline configurations in a more challenging indoor and outdoor joint setting that incorporates nuScenes [9] and SemanticKITTI [5] as training data, and Waymo [67] for zero-shot evaluation. The same evaluation protocol is used across both settings.
Baselines, Since Point-MoE is built on PTv3 [81], we adopt PTv3 and PPT [82] as our primary baselines. PPT, originally designed with sptaset-specific normalization and per-dataset fine-tuning, is adapted here to the joint training setting. Because PPT requires dataset lahels at inference, we follow One-for-All [75] and train lightweight dataset classifier (achieving 2955 accuracy) predict the source dataset for an input sample at test time (Sec. B.2).
Evaluation and Implementation Details. To ensure fair comparison, we re-benchmark PTv3 under our unified protocol by varying batch composition and normalization, which yields an average improvement of 22 mloU (Sec. B.3). We also compare against PPT combined with One-for-All which uses ground ground-truth truth dataset labels during training and predicted dataset labels during inference. Model capacity is denoted with -S (small) and -I. (large) suffixes for architectures, including PTv3, Point-MoF, and PPT. PTv3-S and PPT-S use 14 encoder and 8 decoder layers, while their -L. variants use 18 encoder and 12 decoder layers; in contrast, both Point-MoE-S and -L have 14 encoder and decoder layers, with -S using experts per MoE layer and -L using 8. The expert hidden dimension set to xx the input dimension in Point-MoE-S and 2x in Point-MoE--. For details, please refer to Tab. 10 in the supplementary material.
4.2 EXPERIMENTAL. RESULTS
Single-dataset training generalizes poorly. We begin by evaluating the standard setting where models are trained and evaluated on individual datasets. Note that in our setup PTv3 is trained and evaluated with CLIP- CLIP-guided classification in an open-set setting, consistent with the multi-dataset regime [82], whereas the original PTv3 paper reports results under standard closed-set softmax label classification [81]. This difference makes our scores not directly comparable to the numbers in the original PTv3 paper. As shown in Tab. 1a, PTv3 achieves strong results on seen datasets, but its performance drops significantly on zero-shot datasets. For instance, PTv3 model trained on S3DIS achieves only 3.5% on Structured3D and 4.9% on ScanNet. While dataset-specific models are not expected to generalize across datasets categories, these results underscore the brittleness of
5

---

Published as a conference paper at ICLR 2026
Table 1: Results seen datasets. Results are reported in mloU. Point-MoE achieves the best average performance without using domain labels. Methods marked with use dataset labels during training. Models trained on a single dataset but evaluated on unseen datasets are in gray. (a) Indoor-only joint training on ScanNet [19], Structured3D [95], and S3DIS [.]. (b) Joint training across indoor and outdoor datasets: nuScenes [9] and SemanticKITTI (5).
(a) Indoor-only joint training
ScanNet
Structured3D S3DIS
Methods
Params Activated
Avg.
Val
Val
Test
Area5
Single-dataset Training
PTy3-ScanNet [81]
46M
46M
75.0
19.8
20.9
8.8
31.1
DTv3-Structured3D [81]
46M
46M
21.2
81.5
79.4
7.1
47.3
PTv3-S3DIS [81]
46M
46M
4.9
3.5
5.3
67.6
20.3
Multi dataset Joint Training
OA-CNNs [52]
52M
52M
71.6
59.0
58.6
60.6
62.5
PTv3-S [81]
46M
46M
71.3
56.6
55.9
69.4
63.3
PTv3-L [81]
97M
97M
72.2
56.0
58.1
67.4
63.4
PPT-S* [75. 82]
47M
47M
74.7
64.2
65.5
64.7
67.3
PPT-L [75, 82]
98M
98M
74.8
64.7
65.9
65.0
67.6
Point-MoE-S
59M
52M
74.5
69.0
69.9
68.0
70.4
Poi-ItMoF-LL
100M
59M
76.0
69.5
70.8
69.5
71.5
(b) Indoor and Outdoor joint training
ScanNet
Structured3D
S3DIS nuScenes SemKITTI
Methods
Params
Activated
Avg.
Val
Test
Area5
Val
Val
PTv3-I. [81]
97M
97M
74.6
64.1
63.9
68.0
69.6
62.9
67.2
PPT-L" [75, 82] 98M
98M
76.7
67.5
66.0
61.9
70.8
66.9
68.3
Point-MoE-L
100M
59M
69.5
72,9
66.4
70.8
Table 2: Zero-shot results unseen datasets. Point-MoE consistently achieves the strongest generalization across unseen datasets in the zero-shot setting. Methods marked with use domain labels during training.
(a) Indoor-only joint training
(b) Indoor and Outdoor joint training
Matterport3D
Matterport3D
Waymo
Method
Average
Method
Average
Val
Test
Val
Test
Val
Test
PTv3-L
39.0
45.4
42.2
PTv3-L
39.1
45.9
21.9
23.2
32.5
PPT-L"
27.8
26.3
27.1
PPT-L"
24.9
23.1
16.3
16.7
20.3
Poi-t-MoF-L.
41.0
46.7
43.9
Point-MoE-L.
41.3
49.6
23.7
25.3
35.0
single-dataset training. This motivates the need for multi-dataset joint training strategies that expose the model to diverse data and promote broader generalization.
Point-MoE. surpasses all baselines on seen datasets, without dataset labels during training. Point-MoE dispenses with dataset identifiers during training yet yields consistent gains across in- door benchmarks (Tab. la); both Point-MoE-S and Point-MoE-L variants improve on ScanNet, Structured3D, and S3DIS, with Point-MoE-L attaining new state-of-the-art average of 71.5 mloU. Despite joint training over multiple datasets, Point-MoE matches or exceeds single-dataset specialists, indicating effective expert routing rather than overfitting to any one source, or significantly exhibiting the negative transfer effects noted in PPT [82]. In the more challenging indoor and outdoor joint train- ing setting (Tab. 1b), Point-MoE maintains its advantage across datasets and, on average, surpasses PTv3-L. by 3.55 mloU and PPT-L by 2.45 mloU, demonstrating that label-free expert routing scales to heterogeneous datasets while simplifying the training pipeline. Moreover, performance on the seen indoor datasets remains closely aligned between indoor-only training and indoor-outdon joint training. suggesting that Point-MoE preserves seen datasets accuracy even when subsequently trained jointly with outdoor data. This generalization is achieved implicitly in the model, without requiring dataset labels for incoming samples or maintaining explicit per-dataset normalization parameters. Note, since lightweight dataset classifier, PPT's performance differs from what is reported

---

Published as a conference paper at ICLR 2026
Table 3: Ablations of Point-MoE designs and hyperparameters. We conduct systematic ablation on key design and hyperparameter choices of Point-MoE across four indoor multi-domain datasets: ScanNet (Scan), Structured3D (S3D), S3DIS, and Matterport3D (Mat).
Scan
S3D
S3DIS Mat
top l Scan S3D
S3DIS
Mat
norm Scan S3D
S3DIS
Mat
74.5
66.9
68.6
40.1
top 74.4
64.8
65.5
39.0
BN.
74.5 66.9
68.6
40.1
10
72.3
64.1
67.9
39.5
top 2 74.5
66.9
68.6
40.1
LN. 70.8
56.9
67.3
37.2
10-3
71.6
58.8
67.4
37.4
top 3 73.8
62.4
67.8
39.7
RMS.
45.3
36.5
51.1 24.6
(a) Auxiliary load balancing loss. Removing the auxiliary loss con- sistently improves performance.
(b) Top-k expert selection. Best performance is achicved with k=2 experts across datasets.
(c) Normalization. BatchNorm
yields strongest performance; RM- SNorm degrades results.
case Scan S3D
S3DIS Mat
act.
Scan S3D S3DIS Mat
case
Scan S3D S3DIS Mat
FFN
72.6
66.1
66.9 39.2
SiLU 73.6 65.7 64.9
40.1
IH 73.0 60.3
66.3 39.5
Proj.
74.5
66.9 68.6 40.1
ReLU 74.5
66.9 68.6
40.1
2H
73.5
58.6
65.4 39.8
(d) MoE position. Replacing the projection layer performs better.
(e) Activation functions. ReLU performs the best.
(i) Expert width. H indicates orig- inal feature dimension.
shared Scan S3D
S3DIS Mat
num.
Scan S3D
S3DIS Mat
batch Scan S3D S3DIS Mat
0
74.5
66.9
68.6
40.1
4
74.5
66.9
68.5
40.1
4 74.0 64.9
38.1 68.6 40.1
66.6
73.6
62.3
66.6
39.1
8
73.6
68.7
68.5
40.4
6
74.3
66.9
(g) Shared experts. Not sharing experts yields better performance.
(h) Number experts. More ex- perts improve performance.
ii) Batch size. Larger batchsize improves performance.
in the original paper, where oracle dataset identifiers are provided at inference. Finally, because only a subset of experts is activated, Point-MoF. achieves a favorable accuracy-compute trade-off, cutting compute by 30.9% (265.7 vs. 384.4 GFLOPs) and peak VRAM by 19.0% (33.3 vs. 41.1 GiB) relative to PPT-L (Tab. 5). We analyze the emergent datasct-awareness and routing behavior in Sec. 4.4.
Point-MoE outperforms all baselines consistently across zero-shot datasets. As shown in Tab. 2a and Tab, 2b, Point-MoE-L achieves the strongest zero shot performance, including on Matterport3D under indoor only training and on both Matterport3D and Waymo under indoor and outdoor training. without using dataset labels during training. We believe a key reason is that label free expert routing encourages specialization based on the underlying semanties and geometry of the input rather than on dataset identity, resulting in more stable behavior under distribution shift. By contrast, PPT-L relies on explicit dataset labels during training. While this provides useful supervision on seen datasets, it can also encourage the model to depend too heavily on dataset specific cues. As result, when a dataset is held out and its label is unavailable or no longer aligned with the training distribution at test time, this dependence becomes brittle and can hurt generalization. Additional analysis of Point-MoE's zero shot adaptation is provided in Sec. 4.4.
4.3 MOE DESIGN ABLATION
We investigate the impact of MoE design choices and pappmmerrseeters choices in indoor multi- dataset 3D semantic segmentation. Results are presented in Tab. 3. Note not using precise evaluator so results are not directly comparable to Tab. 1. Please refer to details in Sec. B.3 in the supplementary material.
Effect of load-balancing loss. The auxiliary load- balancinnning loss is designed to encourage uniform expert usage and prevent expert collapse [42, 56, 78]. We ablate its effect by training models with varying strengths of this loss in Tab. 3a. Interestingly, we find that removing the auxiliary loss entirely leads to consistently better performance across datasets. In contrast, stronger auxiliary losses result in significant degradation. We hypothesize that this is due to the inherent imbalance in the distribution of samples across datasets in 3D point cloud datasets a phenomenon also observed by ChartMoE [87].
Effect of activated experts. Tab. 3b compares the performance of configurations with 1, 2 and 3 activated experts, finding diminishing return when increasing number of activated experts. We find that activating two experts yields the best performance across datasets. This analysis suggests that merely increasing the number of activated experts does not guarantee improved performance.
7

---

Published as a conference paper at ICLR 2026
Effect of MoE position. We compare placing MoE in the feed-forward network (FFN) versus in the attention output projection W.. holding all other components fixed. Poojection-Mof con- sistently outperforms FFN-MoE in Tab. 3d. We hypothesize that Wo is the fusion point where multi-head outputs are recombined before the next normalization, so experts placed there can route on richer, head-aggregated signals that pre- serve cross-dataset geometry cues, while leaving Wq: Wk, W, shared and stable; in contrast, FFN experts operate on post-normalized activations where such dataset or geometry cues are more attenuated. This empirical gain leads to adopt projection-MoE as the default. Similar observations also appear in recent work: e.g. UMoE [89] finds that using MoE for the output (and value) projections yields strong compute savings with high performance.
Table 4: Ablations for mixed-datase training. "Mix" indicates that each training batch combines samples from multiple datasets.
Methods
Mix
Scan
S3D
S3DIS
Mat
Average
PTv3 PTv3
48.4
57.8
44.1
27.8
44.5
67.7
45.9
62.5
34,9
52.8(+8.3)
PPT
74.2
58.2
61.0
24.3
54.4
PPT
V
74.4
61.4
62.1
31.1
(7.3(+2.9)
PT-
54.1
60.9
40.6
28.0
45.9
PT-MoR
74.5
66.9
68.6
40.1
(2.5(41166)
Effect of model scaling with MoE. Tab. 3h compares Point-MoE with 4 and experts with activated experts fixed at 2. Increasing the number of experts from 4 to 8 leads improved performance on seen and zero-shot datasets. Tab. 3f ablates the expert width by varying the intermediate feature dimension and the results show mixed trend: wider experts improve performance on ScanNet and Matterport3D. but degrade results on Structured3D and S3DIS. Finally, we investigate the effect of adding shared experts [20, 65, 73]. We find that disabling shared experts consistently leads to better accuracy and dataset specialization across all benchmarks, as reported in Tab. 3g.
Effect of hyper-parameter choices. We provide justification for our selected hyper- apee-paarmmttrs. In Tab. 3c, BatchNorm [29] consistently outperforms other normalization methods [3. 91] in the multi-datasets setting. Tab. 3e shows that ReLUU [50] slightly outperforms SiL.U [23]. Additionally, Tab. 3i demonstrates that importance of batch size in MoE. training. In general, larger batch sizes lead to improved performance across datasets.
Mixed dataset training. In joint multi-dataset training, forming each minibatch with samples drawn from multiple datasets is consistently beneficial. Although there is no explicit batch-level interaction module, samples from different datasets still influence statistics and, for Point-MoE, through competition over a common expert pool. As shown in Tab. 4, this mixed-dataset training strategy improves the average mloU of all backhones: PTv3 +-8.3, PPT +2.9, and Point-MoF: + 16.6 mloU on average. For dense backbones such as PTv3 and PPT, mixing exposes the model to more diverse domain statistics within each optimization step. which appears to regularize training and reduce overfitting to dataset-specific feature distributions. To make this comparison fair for PPT, we re-implemented its normalization module so that mixed-dataset batching is supported. The effect is most pronounced for MoEn where diverse minibatches promote expert specialization across datasets, stabilizing routing and improving generalization.
another through shared normalization
4.4 ANALYSIS OF MoE
Dataset clustering emerges in Point-MoE. Fig. 2 shows t-SNE visualizations [70] of encoder and decoder features from PTv3, PPT [82], and Point-MoE. PTv3 exhibits clear separation in either space, indicating limited capacity to form dataset-specific representations. PPT, by contrast, produces sharply separated clusters both encoder and decoder levels. reflecting its reliance on dataset labels during training. While this aids seen-datasets performance, such rigid partitioning may hinder generalization capabilities as highlighted Tab. 2. Point-MoEE displays a different pattern: encoder features remain mixed across datasets, supporting shared representation learning, while the decoder disentangles dataset-specific structures. This apparent division of labor suggests that Point-MoE performs implicit dataset inference, without requiring explicit labels. Crucially, on the zero-shot Matterport3D dataset, Point-MoE aligns samples with the most related seen-dataset clusters (e.g., ScanNet), which likely contributes to its strong generalization. Overall, these results show that Point-MoE can self oegf-organize its experts and adapt to novel environments by effectively transferring knowledge across diverse training datasets.
8

---

Published as a conference paper at ICLR 2026
Eucoder Features PTv3
Encoder
Enceder
Point-MoBi
Decoder Features of PTv3
Decoder Features of PPT
Decoder Features Poins-MolF
Decoder Features of Point-Maf:
Net
S3DIS
Structure33D
Matterpor3
Figure t-SNE visualization feature clustering. The first three columns show that the decoders of both PPT and PointMoE have better separation between datasets in their decoder representations, than vanilla PTv3. The rightmost column highlights how Point-MoE generalizes to zero-shot datasets, with Matterport3D features aligning closely with ScanNet, which is indeed semantically similar.
Encoder Layer
Decoder Layer
Decoder Layer
Decoder Layer
Figure 3: Expert choice visualization. Different colors within each image indicate different experts, showing that Point-MoF. self-organizes point routing based spatial and semantic cues across different layers. (a) shows one expert (green) focusing on edges, suggesting spatially aware mid-level routing in the encoder. (b) and (c) show that semantically related regions (e.g., chairs and desks) consistently assigned to the same expert, even across datasets. (d) shows meaningful routing in an outdoor scene with sparse LiDAR points despite lower point density and more irregular geometry.
Expert choice visualization. Fig. visualizes expert assignments for validation scenes at selected encoder and decoder layers. In (a), the encoder layer primarily routes points according local geometric structure. In particular, one expert (green) consistently activated around object boundaries and thin structural regions, such the edges of desks and chairs, while other experts dominate flatter surfaces, suggesting spatially aware mid-level routing in the encoder. In (b) and (c), the decoder layers exhibit more semantically coherent expert selection. Semantically related regions, such as chairs, desks, floors, and walls, are more consistently assigned to the same experts, even across different scenes and datasets, likely because decoder features are closer to the supervision signal and thus encode higher-level semantics. In (d), we examine outdoor scene with sparse LiDAR points. Despite the lower point density and more irregular geometry, the model still produces meaningful routing patterns rather than random assignments. Nearby foreground regions and farther background regions tend to activate different experts, suggesting that the learned routing generalizes beyond dense indoor scans to more challenging outdoor environments. Additional visualizations Sec. C.3. We also observe few isolated points assigned to different experts than their neighbors, possibly due to PTv3 design choices such point serialization and positional encoding.
provided in

---

Published as a conference paper at ICLR 2026
Table 5: Inference efliciency. FLOPs per step are measured with a profiler, and peak VRAM reports allocated memory. Relative deltas are computed with respect to PPT-L.
Model
FLOPs/step (GFLOPs)
Peak VRAM (GiB)
A FLOPs PPT-L
VRAM PPT-L
PPT-L
384,4
41.1
0%
0%
PTv3
375.8
37.4
9.0% 2.2%
Point-MoE-L
265.7
33.3
J. 30.9%
4. 19.0%
Expert idx
SXPIS
SananticKITTI
NuScetes
Other park
BL ro
Bla0a
E2 1880
R top
at ca
na case
rato
tese
et tota
es am
BI ccso
m M.zas
(189
teaa
nt, con
oon
IN cn
Du
Layer Blook
Figure Expert routing across datasets. Left: the most frequent expert paths through each encoder (E) and decoder (D) layer, with channel sizes in parentheses, showing that expert selection varies across datasets. The effect is especially pronounced in decoder MoE layers. Right: Jensen Shannon Divergence (JSD) between dataset specific expert selection distributions at each MoE layer.
Token pathways. To understand how Point-MoE. adapts diverse datasets, analyze expert routing at the token level. We track each token's 's top-1 expert across all MoE layers and reconstruct full routing trajectories from the final layer back to the first. We then rank expert paths by frequency and focus on the top 100 (Fig. 4). Encoder paths are notably less diverse than decoder paths, suggesting that encoders perform more dataset-agnostic processing. We also find sparse routing deeper encoder layers, likely due not to feature reuse but to the U-Net design: features grow deeper spatially while tokens become sparser, reducing routing variability. When examining dataset-level trends, we find that certain dataset pairs, such as SemanticKITTI and nuScenes or ScanNet and Structured3DD share similar expert pathways, suggesting that Point-Mof implicitly clusters datasets with related geometric semantic structures. To quantify these observations, we compute the Maasen-Shannon Divergence (ISD) [45] between expert selection distributions across datasets at each MoE layer. JSD [45] is an entropy-based measure of divergence between expert routing distributions across datasets, weighted by their token proportions: its formal definition is provided in the supplementary. As shown in Fig. 4 (right), decoder layers exhibit significantly higher JSD, indicating stronger dataset-specific specialization. Several encoder layers also display nontrivial JSD, underscoring the benefit of placing MoE throughout the network (see Sec. C.2 for more analysis).
Efficiency analysis. As shown in Tab. 5, Point-MoE-LE requires only 265.7 GFLOPs per step and 33.3 GiB peak VRAM, reducing compute by 30.9% and memory by 19.0% relative to PPT-L (384.4 GFLOPs and 41.1 GiB). It is also more efficient than PTv3 (375.8 GFLOPs and 37.4 GiB). Despite this lower cost, Point-MoE-LL achieves the best average mloU both settings, reaching 71.5 in indoor only joint training and 70.2 2 in indoor and outdoor joint training (Tab. 1).
DISCUSSIONS
This work embraces the "bitter lesson" [68] in AI: scalable generalization emerges from flexible architectures trained on diverse data, rather than hand-engineered domain priors. We introduce Point-MoF, a miiture-offexxperts architecture for 3D understanding that dynamically routes tokens to specialized experts across both indoor and outdoor domains. Our experiments demonstrate that Point-MoE not only achieves superior performance on seen datasets but also generalizes effectively to zero-shot datasets without explicit domain labels. Through careful analysis of expert routing behavior, we show that the model self-organizes around both geometric and semantic structures, adapting its computation pathways to the nature of the input. These results highlight the potential of sparse, modular computation for robust and scalable 3D scene understanding. We provide more analysis of Point-MoF. and discuss our limitations in the supplementary materials.
10

---

Published as a conference paper at ICLR 2026
6 ACKNOWLEDGEMENT
The authors acknowledge the MathWorks Research Award, Adobe Research Gift, the University of Virginia Research Computing and Data Analyties Center, Advanced Miero Devices Al and HPC Cluster Program, Advanced Cyberinfrastrrucure Coordination Ecosystem: Services & Support (ACCESS) program, and National Artificial Intelligence Research Resource (NAIRR) Pilot for computational resources, including the Anvil supercomputer (National Science Foundation award OAC 2005632) at Purdue University and the Delta and DeltaAl advanced computing resources (National Science Foundation award OAC 2005572).
REFERENCES
[1] Iro Armeni, Ozan Sener, Amir R. Zamir, Helen Jiang, Ioannis Brilakis, Martin Fischer, and Silvio Savarese. 3d semantic parsing of large-scale indoor spaces. In 2016 IEEE Conference on Computer Vision and Pattern Recognition ( (CVPR), pp. 1534-1543, 2016. doi: 10.1 109/CPR.. 2016.170. 6, 18, 19, 21
[2] Armen Avetisyan, Christopher Xie, Henry Howard-Jenking Tsun- Yi Yang. Samir Aroudj, Suvam Patra, Fuyang Zhang, Duncan Frost, Luke Holland, Campbell Orme, et al. Scene- script:
Reconstructing scenes with autoregressive structured language model. In European Conference on Computer Vision, pp. 247-263. Springer, 2024.
[3] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.064500 2016. 8, 22
[4] Max Bain, Arsha Nagrani, Gül Varol, and Andrew Zisserman. Frozen in time: A joint video and image encoder for end-to-end retrieval. In IEEE International Conference on Computer Vision, 2021. 1
[5] J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stachniss, and J. Gall. Se- manticKITTI: Dataset for Semantic Scene Understanding of LiDAR Sequences. In Proc. of the IEEE/CVF International Conf. on Computer Vision (ICCV), 2019. 5, 6, 18, 21, 30
[6] Bruce Blumberg et al. Precise workcell sketching from point clouds using an toolbox. arXiv preprint arXiv:2410.00479 2024. 1
[7] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 13:1877-1901, 2020.
[8] Minwoo Byeon, Beomhee Park, Haecheon Kim, Sungjun Lee, Woonhyuk Back, and Sae- hoon Kim. Coyo-700m: Image-text pair dataset. https:://e /gtgtt...b.ttg/gggg/ggggg...h www/kokaakkaeeeeooooo htt///aaakbbiiinn/ coyo-datased 2022.
[9] Holger Caesar, Varun Bankiti, Alex Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan. Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11621-11631, 2020. 1,5, 6, 18, 19, 21,
[10] Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. arXiv preprint arXiv: 100.://7090.5555 2017. 5, 18, 19, 30
[11] Shizhe Chen, Ricardo Garcia, Cordelia Schmid, and Ivan Laptev. Polarnet: 3d point clouds for language-guided robotic manipulation. arXiv preprint arXiv:2309.15500 2023. 1
[12] Siheng Chen, Baoan Liu, Chen Feng, Carlos Vallespi-Gonzales and Carl Wellington. 3d point cloud processing and learning for autonomous driving. arXiv preprint arXiv:2003.0060) 2020.
[13] Xiaozhi Chen, Huimin Ma, Ji Wan, Bo Li, and Tian Xia. Multi-view 3d object detection network for autonomous driving. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 1907-1915, 2017. 1,3

---

Published as a conference paper at ICLR 2026
[14] Zitian Chen, Yikang Shen, Mingyu Ding, Zhenfang Chen, Hengshuang Zhao, Erik G Learned- Miller, and Chuang Gan. Mod-squad: Designing mixtures of experts as modular multi-task learners. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recog- nition, pp. 1828-118 1837, 1128-11837, 2023.
[15] Nadezhda Chirkova, Vassilina Nikoulina, Jean-Luc Meunier, and Alexandre Bérard. Investi- gating the potential sparse mwxtures-offfeeeee of-experts for multi-domain neural machine translation. arXiv preprint arXiv:2407.01126, 2024. 2,
[16] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):
1-113, 2023.
[17] Christopher Choy, Jun YunYoung Gwak, and Silvio Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9075-3084, 2019. 3
[18] Pointcept Contributors. Pointcept: A codebase for point cloud perception research. https:
//github. www.aaa..co///oiiiiiii wwtp//ooinncceee/ooootteoooooooo 2023. 19
[19] Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5828-5839, 2017. 1, 5, 6, 18, 19, 21, 30
[20] Damai Dai, Chengqi Deng, Chenggang Zhao, RX Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Yu Wu, et al. Deepseekmoe: Towards ultimate expert special- ization mixture-of-experts language models. arXiv preprint arXiv:2401.0mm 100666, 2024. 2, 3,
8
[21] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenbomn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x 16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010.1 19299, 2020.
[22] Nan Du, Yanping Huang, Andrew M Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, al. Glam: Efficient scaling of language models with mixture-of-experts, In International conference machine learning, pp. 5547-5569. PMLR, 2022. 2,
[23] Stefan Elfwing, Eiji Uchibe, and Kenji Doya. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural networks, 107:3-11, 2018. 8
[24] William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23 (120):1-39) 2022. 2,
[25] Benjamin Graham, Martin Engelcke, and Laurens Van Der Maaten. 3d semantic segmentation with submanifold sparse convolutional networks. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 9224-9232, 2018. 3
[26] Jiang Guo, Darsh J Shah, and Regina Barzilay. Multi-source domain adaptation with mixture of experts. arXiv preprint arXiv: 180.00226, 2018. 3
[27] Meng-Hao Guo. Jun Lun-Xiong Cai, Zheng-Ning Liu, Tai- Tai-Jiang Mu, Ralph R. Martin, and Shi-Min Hu. Pet: Point cloud transformer. Computational Visual Media, 7(2):187-199) June 2021. ISSN 2096-0662. doi: 10.
URL http:htpt///c dx doi.oooiooooooooooo www.doo...oooooooooooooooooooooe
s41 141095-000 02 5.
[28] Suchin Gururangan, Mike Lewis. Ari Holtzman, Noah A Smith. and Luke Zettlemoyer. Demix layers: Disentangling domains for modular language modeling. arXiv preprint (XXv:2/08.050366 2021.
12

---

Published as a conference paper at ICLR 2026
[29] Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pp. 148-456. pMlr, 2015. 8
[30] Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan, and Geoffrey E. Hinton. Adaptive mixtures of local experts. Neural Computation, 3(1):79-87, 1991. doi: 00.1162/ncoo.1911....
79. 2, 3, 4
[31] Yash Jain, Harkirat Behl, 7solt Kira, and Vibhav Vineet. Damex: Dataset-aware mixture- of-experts for visual understanding of mixture-of-ddttseee Advances in Neural Information Processing Systems, 36:69625-69637 2023.
[32] Guangda Ji, Silvan Weder, Francis Engelmann, Mare Pollefeys, and Hermann Blum. Arkit labelmaker: A new scale for indoor 3d scene understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 4398 2025.
[33] Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024.
[34] M.I. Jordan and R.A. Jacobs. Hierarchical mixtures of experts and the em algorithm. In Proceedings of 1993 International Conference Neural Networks (IJCNN-95 93-Nooyy, Japan), volume 2, pp. 1339-1344 vol.2, 1993. doi: 10. (0.1109/JCCNN.199)) 1993.1676919997779911 2,
[35] Hyeonseong Kim, Yoonsu Kang, Changgyoon Oh, and Kuk-Jin Yoon. Single domain gen- eralization for lidar semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 17587-17598, 2023.
[36] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitchead, Alexander Berg, Wan Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference computer vision, pp. 4015-4026, 2023.1
[37] Maksim Kolodiazhnyi, Anna Vorontsova, Matvey Skripkin, Danila Rukhovich, and Anton Konushin. Unidet3d: Multi-dataset indoor 3d object detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 4373, 2025. 3
[38] Alex H Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. Pointpillars: Fast encoders for object detection from point clouds. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 12697-12705, 2019.
[39] Junha Lee, Chunghyun Park, Jaesung Choe, Yu-Chiang Frank Wang, Jan Kautz, Minsu Cho, and Chris Choy. Mosaic3d: Foundation dataset and model for opea-uocabulary 3d segmentation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 14089-14100 2025.
[40] Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. Gshard: Scaling giant models with con- ditional computation and automatic sharding. arXiv preprint arXiv:2006. 16668, 2020. 2, 3
[41] Bo Li, Tianlei Zhang, and Tian Xia. Vehicle detection from 3d lidar using fully convolutional network. arXiv preprint 160ivv1608.000 2016. 3
[42] Dengchun Li, Yingzi Ma, Naizheng Wang. Zhengmao Ye, Zhiyuan Cheng, Yinghao Tang. Yan Zhang, Lei Duan, Jie Zuo, Cal Yang, et al. Mixlora: Enhancing large language models fine-tuning with lora-based mixture of experts. arXiv preprint arXiv:2404. 15159, 2024. 7
[43] Ying Li, Lingfei Ma, Zilong Zhong, Fei Liu. Michael A Chapman, Dongpu Cao, and Jonathan Li. Deep learning for lidar point clouds autonomous driving: review. IEEE Transactions on Neural Networks and Learning Systems, 3218):3412-3432 2020.
13

---

Published as a conference paper at ICLR 2026
[44] Bin Lin, Zhenyu Tang, Yang Ye, Jiaxi Cui, Bin Zhu, Peng Jin, Jinfa Huang, Junwu Zhang, Yatian Pang, Munan Ning, et al. Moe-llava: Mixture of experts for large vision- langunla models. arXiv preprint arXi::2401.15947, 2024.
[45] Jianhua Lin. Divergence measures based on the shannon entropy. IEEE Trans. Inf. The- ory, 37:145-151, 1991. URL https:/ /api. semanticscholae wwtp///rp///.......oo
12121532. 10
[46] Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chenggi Dengr, Chong Ruan, Damai Dai, Daya Guo, al. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. arXiv preprint arXiv:2405.04434, 2024. 3
[47] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprini 11X4 2024. 3,5
[48] Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, al. Deepseek-vl: towards weal-world vision-language under- standing. arXiv preprint arXiv:2403.0552. 2024. 3
[49] Daniel Maturana and Sebastian Scherer. Voxnet: A 3d convolutional neural network for real- time object recognition. In 2015 JEEE/RS International Conference on Intelligent Robots and Systems (IROS), pp. 922-928, 2015. doi: 10.
[50] Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML-10), pp. 807- 814, 2010. 8
[51] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2334.077/94 2023.
[52] Bohao Peng, Xiaoyang Wu, Li Jiang, Yukang Chen, Hengshuang Zhao, Zhuotao Tian, and Jiaya Jia. Oa-enns: Omni-adaptive sparse cnns for 3d semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21305-21315, 2024. 5, 6,6,20 20
[53] Skand Peri, lain Lee, Chanho Kim, Li Fuxin, Tucker Hermans, and Stefan Lee. Point cloud models improve visual robustness in robotic learners. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 17529-17536. IEEE, 2024. 1
[54] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas Guibas. Pointnet: Deep learning point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652-660, 2017.
[55] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning point sets in a metric space. Advances in neural information processing systems, 30, 2017.
[56] Zihan Qiu, Zeyu Huang, Bo Zheng, Kaiyue Wen, Zekun Wang, Rui Men, Ivan Titov, Dayiheng Liu, Jingren Zhou, and Junyang Lin. Demons in the detail: On implementing load balancing loss for training specialized mixture-of-expprt models. arXiv preprint arXiv:2501. 11873, 2025.
7
[57] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PmLR, 2021. 1,
[58] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1-67, 2020.
14

---

Published as a conference paper at ICLR 2026
[59] Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, André Susano Pinto, Daniel Keysers, and Neil Houlsby. Scaling vision with sparse mixture of experts. Advances in Neural Information Processing Systems, 34:8583-8855 2021. 2,3
[60] Cristiano Saltori, Aljosa Osep, Elisa Ricci, and Laura Leal-Taixé. Walking your lidog: A journey through multiple domains for lidar semantic segmentation. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 196- 2023. 3
[61] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion- 5b: An open large-scale dataset for training next generation image-text models. Advances in neural information processing systems, 35:25278 -52294, 2022.
[62] Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoe Le, Geoffrey Hinton, and Jeff Dean. Outragcously large neural networks: The sparsely-gates mixture-of-cxperts layer. arXiv preprint arXiv: 707.06016888 2017. 3
[63] Shuran Song, Fisher Yu, Andy Zeng, Angel X Chang, Manolis Savva, and Thomas Funkhouser. Semantic scene completion from a single depth image. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1746-1754, 2017. 3
[64] Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik Learned- Millee.. Multi-view convo- lutional neural networks for 3d shape recognition. In Proceedings of the IEEE international conference on computer vision, pp. 945-953, 2015.
[65] Zhenpeng Su, Xing Wu, Zijia Lin, Yizhe Xiong, Minxuan Lv, Guangyuan Ma, Hui Chen, Songlin Hu, and Guiguang Ding. Cartesianmoe: Boosting knowledge sharing among experts via cartesian product routing in mixture-of-experts. arXiv preprint arXiv:2410. 2024.
[66] Chen Sun, Abhinav Shrivastava, Saurabh Singh, and Abhinav Gupta. Revisiting unreasonable effectiveness of data in deep learning era. In Proceedings of the IEEE international conference on computer vision, pp. 843 852, 2017.
[67] Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaík, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhang, Jonathon Shlens, Zhifeng Chen, and Dragomir Anguelov. Scalability in perception for autonomous driving: Waymo open dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020. 5, 18, 30
[68] Richard S. Sutton. The bitter lesson. http::/wwet www.onneeeeeeeee........ net/ Incldeas/BitterLessse html, 2019, Accessed: 2024-05-15. 2, 10
[69] Hugues Thomas, Charles R Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J Guibas. Kpconv: Flexible and deformable convolution for point clouds. In Proceedings of the IEEE/CVF international conference on computer vision, pp- 6411-6420, 2019.
[70] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(86))257-22055 2008. URL httttt:://mmr.org///eeeo V
html. 8
[71] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[72] Tai Wang, Xiaohan Mao, Chenming Zhu, Runsen Xu, Ruiyuan Lyu, Peisen Li, Xiao Chen, Wenwei Zhang, Kai Chen, Tianfan Xue, et al. Embodiedscan: A holistic multi-modal 3d perception suite towards embodied ai. In Proceedings of the IEEF/CV Conference on Computer Vision and Pattern Recognition, pp. 19757-19767. 2024.
[73] Xu Wang, Jiangxia Cao, Zhiyi Fu, Kun Gai, and Guorui Zhou. Home: Hierarchy of multi- gate experts for multi-tast learning at kuaishou. In Proceedings of the 31st ACM SIGKDD Conference Knowledge Discovery and Data Mining V. 1, pp. 2638 2647, 2025. 8
15

---

Published as a conference paper at ICLR 2026
[74] Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph enn for learning on point clouds. ACM Transactions on Graphics (tog), 38(5):1-12, 2019. 19
[75] Zhenyu Wang, Ya-Li Li, Hengshuang Zhao, and Shengjin Wang. One for all: Multi-domain joint training for point cloud based 3d object detection. Advances in Neural Information Processing Systems, 37:56859-56877
2024. 1, 2, 3, 4, 5, 6, 22
[76] Zichen Wang, Ang Cao, Liam J Wang, and Jeong Joon Park. Moe3d: A mixture-of-experts module for 3d reconstruction. arXiv preprint arXiv:2001.05208; 2026.
[77] Maximilian Weber, Daniel Wild, Jens Kleesiek, Jan Egger, and Christina Gsaxner. Deep learning-based point cloud registration for augmented reality-guided surgery. In 2024 IEEE International Symposium on Biomedical Imaging (ISBI), pp. 1-5, IEEE, 2024.
[78] Tianwen Wei, Bo Zhu, Liang Zhao, Cheng Cheng, Biye Li, Weiwei Lü, Peng Cheng, Jianhao Zhang, Xiaoyu Zhang, Liang Zeng, et al. Skyeork-moe: A deep dive into training techniques for mixture--o-eexprts language models. arXiv preprint arXiv:2406.06566 2024. 7
[79] Erik Wijmans, Samyak Datta, Oleksandr Maksymets, Abhishek Das, Georgia Gkioxari, Stefan Lee, Irfan Essa, Devi Parikh, and Dhruv Batra. Embodied question answering in photorealistic environments with point cloud perception. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6659-6668, 2019. 1
[80] Xiaoyang Wu, Yixing Lao, Li Jiang, Xihui Liu, and Hengshuang Zhao. Point transformer v2: Grouped vector attention and cautition-basee pooling. Advances in Neural Information Processing Systems, 35:33330-33344 2022. 3
[81] Xiaoyang Wu, Li Jiang, Peng-Shuai Wang, Zhijian Liu, Xihui Liu, Yu Qiao, Wanli Ouyang, Tong He, and Hengshuang Zhao. Point transformer v3: Simpler faster stronger. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4840-4851, 2024.
2, 3, 4, 5, 6, 22
[82] Xiaoyang Wu, Zhuotao Tian, Xin Wen, Bohao Peng, Xihui Liu, Kaicheng Yu, and Hengshuang Zhao. Towards large-scale 3d representation learning with multi-dataset point prompt training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pallern Recognition, pp.
19551-19562, 2024. 2, 3, 4, 5, 6, 8, 22
[83] Xiaoyang Wu, Daniel DeTone, Duncan Frost, Tianwei Shen, Chris Xie, Nan Yang, Jakob Engel, Richard Newcombe, Hengshuang Zhao, and Julian Straub. Sonata: Self-supervised learning of reliable point representations. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 22193-22204, 2025. 3
[84] Yao Wu, Mingwei Xing, Yachao Zhang, Xiaotong Luo, Yuan Xie, and Yanyun Qu. Unidseg: Unified cross-domain 3d semantic segmentation via visual foundation models prior. Advances in Neural Information Processing Systems, 37:101223-10122 2024. 3
[85] Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, et al. Deepseek-v12: Mixture-of-experts vision- language models for advanced multimodal understanding. arXiv preprint arXiv:241.. 2024. 3
[86] Aoran Xiao, Jiaxing Huang, Weihao Xuan, Ruijie Ren, Kangcheng Liu, Dayan Guan, Abdulmo- taleb El Saddik, Shijian Lu, and Eric P Xing. 3d semantic segmentation in the wild: Learning generalized models for adverse-condition point clouds. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9382-9392, 2023. 3
[87] Zhengzhuo Xu, Bowen Qu, Yiyan Qi, Sinan Du, Chengjin Xu, Chun Yuan, and Jian Guo. Chartmoe: Mixture of diversely aligned expert connector for chart understanding. arXiv preprint arXiv:2409.03227, 2024. 2, 3.7
16

---

Published as a conference paper at ICLR 2026
[88] Longyu Yang, Ping Hu, Shangbo Yuan, Lu Zhang, Jun Liu, Hengtao Shen, and Xiaofeng collaboration for generalized lidar segmentation in
Zhu. Towards explicit adverse weather. In Proceedings of the Computer Vision and Pattern Recognition Conference.
pp. 139-149, 2025. 3
[89] Yuanhang Yang, Chaozheng Wang, and Jing Li. Umoe: Unifying attention and ffn with shared experts. arXiv preprint arXiv:2505.07260, 2025. 8
[90] Yanjie Ze, Gu Zhang, Kangning Zhang, Chenyuan Hu, Muhan Wang, and Huazhe Xu. 3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations. arXiv preprint arXiv:2403. 03954, 2024.
[91] Biao Zhang and Rico Sennrich. Root mean square layer normalization. Advances in neural information processing systems, 32, 2019.
[92] Haimei Zhao, Jing Zhang, Zhuo Chen, Shanshan Zhao, and Dacheng Tao. Unimix: Towards do- main adaptive and generalizable lidar semantic segmentation in adverse weather. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14781-14791, 2024. 3
[93] Hengshuang Zhao, Li Jiang, Chi-Wing Fu, and Jiaya Jia. Pointweb: Enhancing local neighbor- hood features for point cloud processing. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp- 5566, 2019. doi: 10. 10.110/C/VP..019.000
3
[94] Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Tom, and Vladlen Koltun. Point transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 16259-16268, 2021.
[95] Jia Zheng, Junfei Zhang, Jing Li, Rui Tang. Shenghua Gao, and Zihan Zhou. Structured3d: A large photo-realistic dataset for structured 3d modeling. In Proceedings of The European Conference on Computer Vision (ECCV), 2020. 5, 6, 18, 19, 21, 30
[96] Junsheng Zhou, Jinsheng Wang, Baorui Ma, Yu-Shen Liu, Tiejun Huang, and Xinlong Wang. Uni3d: Exploring unified 3d representation at scale. In International Conference on Learning Representations (ICLR), 2024.
[97] Xingyi Zhou, Vladlen Koltun, and Philipp Krähenbühl. Simple multi dataset detection. In Proceedings of the JEEE/CVF conference on computer vision and pattern recognition, pp. 7571-7580, 2022.
[98] Barret Zoph, Irwan Bello, Sameer Kumar, Nan Du Yanping Huang, Jeff Dean, Noam Shazeer, and William Fedus. St-moe: Designing stable and transferable sparse expert models. arXiv preprint
2022. 3
17


Reviews:
[{"review_id": "cLyQVhCKAm", "reviewer": "Reviewer_tPuN", "summary": "The paper explores the shift from dense to sparse Transformers for point cloud processing and investigates mixed indoor-outdoor training with a MoE framework. The approach aims to leverage heterogeneous data for improved generalization and domain adaptation. The results demonstrate notable accuracy gains across datasets and suggest that the MoE design effectively mitigates cross-dataset interference.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "the authors present evidence that mixed-data training enhances performance on each constituent dataset. The MoE framework appears effective at isolating domain-specific patterns while suppressing cross-dataset interference, thereby amplifying the positive effects of data scaling. This insight is valuable.", "weaknesses": "1. The shift from dense to sparse Transformers is a significant architectural choice; however, the paper would benefit from a more thorough analysis of its practical implications on point cloud data — particularly regarding training/inference latency overhead and training stability. These aspects are critical for assessing whether the reported accuracy gains come with hidden trade-offs (e.g., convergence difficulty, increased wall-clock time, or sensitivity to hyperparameters).\n2. The experiments focus on heterogeneous indoor-outdoor training and inference. But several key questions remain underexplored:  \n- Do different sensor modalities (or sensor configurations) benefit equally from the proposed approach?  \n- Are the learned experts equally capable of capturing and adapting to sensor-specific data distributions, or do some sensors dominate or degrade performance?\n3. Indoor and outdoor scenes differ substantially in object geometry, scale, density, and context. Training across both domains clearly leverages additional data — but what specific capabilities does this impart to the model? Is it improved generalization, better feature disentanglement, or simply regularization? \n4. Mixed-dataset training yields a performance gap between indoor and outdoor evaluation: outdoor performance lags significantly behind indoor gains. This raises an important concern — could indoor data be introducing noise or domain bias that hampers scaling benefits for outdoor scenes, rather than helping them? \n5.  The observed zero-shot gains are markedly stronger in indoor settings. Is this truly due to better scene or acquisition-mode discrimination — or might it instead reflect the MoE’s improved ability to specialize on object-level distinctions that happen to be more prevalent or cleanly separable indoors?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cEjpt9Lxjm", "reviewer": "Reviewer_wdW6", "summary": "This paper presents Point-MoE, a multi-dataset training method for 3D point-cloud semantic segmentation. Multi-dataset training is challenging for this task because 3D point clouds datasets are heterogeneous due to the different ways in which the point clouds are captured, making naïve multi-dataset training ineffective. Point-MoE addresses this challenge by introducing a Mixture-of-Experts (MoE) block into existing 3D segmentation models. This block routes point-level tokens to different 'expert' MLPs, allowing the model to have specialized learned layers for tokens from different datasets. Notably, this approach allows for dataset-specific components without requiring that the model knows to which dataset an input sample belongs, unlike existing work. This makes it more suitable for real-world application and improves generalization. Through experiments, Point-MoE is shown to be more effective than other multi-dataset training methods on indoor datasets and a combination of indoor and outdoor datasets.", "questions": "This paper presents an idea that is original in the context of 3D segmentation, and presents a method that is effective and has clear benefits over existing approaches, but I mainly have some concerns about the low performance of the baselines. In the rebuttal, I would like to ask the authors to provide a compelling reason why the performance of the baselines is lower than what is originally reported. Additionally, I would like to ask the authors to explain why the number of activated model parameters is so low for Point-MoE, and to fix the issues with the mathematical notation. If the authors resolve these concerns, I am willing to upgrade my rating.\n\nIn addition to this, I have the following question:\n* Could Point-MoE also improve the performance for single-dataset training, or it only effective for multi-dataset training? I could imagine that adaptively using specialized model parameters for different data subsets would also be useful within a single dataset. While it is beyond the main scope of the paper, I think the paper would be even stronger if it it could give some more insights into the situations in which Point-MoE is effective or not.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. Overall, the paper is well written. This makes it easy to understand how the method works and what the significance is of the findings.\n\n2. The idea of applying an MoE block in 3D segmentation models to facilitate multi-dataset training is original and well-motivated. By allowing tokens to be dynamically routed to expert MLPs, the model can learn to adaptively handle different types of data, which is required for multi-dataset 3D segmentation due to the heterogeneity of 3D point cloud datasets. Moreover, these MoE blocks allow for multi-dataset optimization without requiring explicit labels about the dataset that an input sample belongs to. As a result, Point-MoE puts fewer requirements on the information that should be available during inference, and generalizes much better to unseen data for which no dataset label is available (as demonstrated in Tab. 2).\n\n3. Experimental results (Tab. 1) show that Point-MoE outperforms existing multi-dataset training approach PPT when training on indoor data only, as well as when training on both indoor and outdoor data. This demonstrates the effectiveness of the approach in the considered experimental setting.\n\n4. The paper carefully analyzes the impact of many design choices that play a role when inserting a mixture-of-experts block into a 3D segmentation model (see Tab. 3), such as the number of experts that can be selected, the position of the MoE blocks, and the operations used for each expert. These analyses provide actionable insights into the effectiveness of various configurations, and validate the design choices made for Point-MoE.", "weaknesses": "1. While Point-MoE obtains impressive results in Tab. 1, the performance of its baselines (PTv3 and PPT) is lower than reported in the original papers. For PTv3 single-dataset training, this paper reports scores of 75.0 mIoU on ScanNet val and 67.6 mIoU on S3DIS val (should this be Area5 instead of val?), while the original paper [a] reports scores of 77.5 mIoU on ScanNet val and 73.4 mIoU on S3DIS Area5. Additionally, for PPT-S multi-dataset training, this paper reports 74.7 mIoU on ScanNet and 64.7 mIoU on S3DIS while the original paper [b] reports 78.6 mIoU on ScanNet and 74.7 mIoU on S3DIS. Why are there such considerable gaps in performance? The paper (L256) mentions that PTv3 is re-benchmarked in this paper under a 'unified protocol', but claims that this increases the performance rather than decreasing it. Due to the differences between the baselines' reported performance in Tab. 1 and in the original papers, it is not clear if Point-MoE obtains such impressive results because it is truly so effective, or if it merely obtains these results because the baselines are weaker than they should be. This limits the value of the paper.\n\n2. While Point-MoE performs quite well on average across all datasets, training it on more data does not improve performance across the board, on each individual dataset. For instance, in Tab. 1, the Structured3D performance for single-dataset training is much better than for multi-dataset training. Similarly, training on outdoor data in addition to indoor data (Tab. 1b) does not greatly impact the performance on indoor datasets (compared to Tab. 1a), and even causes a performance drop for S3DIS. These results suggest that Point-MoE still cannot fully leverage the power of massive heterogeneous datasets, which is a limitation of the approach.\n\n3. It is not clearly explained why, in Tab. 1, the number of activated model parameters is so low for Point-MoE, compared the number of parameters of PTv3. If I understand the paper correctly, Point-MoE only replaces the projection layer after the self-attention operation with a router and experts, where each expert consists of an MLP (L075). Since two of these expert MLPs will actually be used (L347-L354), this means that the parameters from PTv3's original projection layer are replaced with the parameters from the router and the 2 MLPs, when considering only activated parameters. Since the original projection layer is quite lightweight, I assume that the router and 2 MLPs do not have fewer parameters than the projection layer. Beyond this change, all other model parameters remain the same for Point-MoE compared to PTv3. In this case, how is it possible that the total number of activated parameters for Point-MoE is so much lower than for PTv3? This should be explained better. The same applies to the drops in VRAM and FLOPs (L299).\n\n4. There are some inconsistencies in the mathematical notation, and some symbols are not explained. \n    * The symbol $N$ is used to represent three different things: the number of point clouds (L148), the number of experts (L168), and the number of points (L180). This is very confusing. The paper would be clearer if a different symbol was used to represent each of these three things.\n    * Similarly, $f$ represents both the overall model in L153 and the expert function in L168.\n    * In Sec. 3.2, it is not described what $\\mathbf{Y}$ (Eq. 7) represents, and what it is used for.\n\n[a] Wu et al., Point Transformer V3: Simpler, Faster, Stronger, CVPR 2024.\n\n[b] Wu et al., Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Training, CVPR 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hV25qKBaNS", "reviewer": "Reviewer_rFAr", "summary": "This paper tackles an underexplored but important problem—large-scale multi-dataset training for 3D semantic segmentation—using a Mixture-of-Experts (MoE) framework. The authors identify a clear limitation of existing 3D methods that rely on dataset-specific normalization or labels and convincingly argue for a unified, label-free approach. The proposed Point-MoE integrates a (router+ different F expert) layer into Point Transformer V3, enabling specialization across heterogeneous 3D datasets. The method is well-motivated and supported by strong experimental evidence: consistent improvements over PTv3 and PPT across multiple datasets (ScanNet, Structured3D, S3DIS, nuScenes, etc.), as well as solid zero-shot generalization to unseen domains.  Ablations are thorough—covering routing strategies, normalization, expert count, and position—and the visual clustering analysis (t-SNE, JSD) provides meaningful insights. The writing is mostly clear, and the results are strong and significant.", "questions": "1.Routing behavior – Could you clarify how the router behaves at inference when encountering unseen domains? Are the same gating logits used as during training, or is there any adaptation (e.g., temperature scaling, entropy regularization)?\n\n2.CLIP alignment – How exactly does the CLIP-based language supervision interact with the MoE gating? Does semantic alignment influence expert selection, or are these processes independent?\n\n3.Load balancing – You mention that removing the auxiliary balancing loss improves results (Tab. 3a). Could you elaborate on why this happens and whether it risks expert collapse in larger-scale settings? Is the hypothesis that this is due to the inherent imbalance in the distribution of samples across datasets in 3D point cloud datasets verified, or a pure heuristic claim?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper clearly defines a practical setting—multi-dataset 3D semantic segmentation without dataset labels at inference—and proposes Point-MoE, which integrates sparse Mixture-of-Experts into Point Transformer V3. Each attention output projection is replaced with lightweight expert routing (top-k), enabling automatic expert specialization across heterogeneous 3D domains. The method achieves consistent gains over PTv3 and PPT on both indoor and mixed indoor–outdoor benchmarks, while maintaining better zero-shot generalization on unseen datasets such as Matterport3D and Waymo. The ablations are thorough—covering balancing loss, routing k, normalization, expert count/placement, and batch mixing—and the t-SNE/JSD analyses nicely reveal encoder–decoder specialization behavior. Moreover, Point-MoE reduces computation and memory compared with PPT, which strengthens its practical value.", "weaknesses": "The innovation lies mainly in applying MoE mechanisms as a routing layer into Point-TransformerV3 to 3D multi-dataset training. \nThe interaction between CLIP-based language supervision(section 3.1 Language-guided classification) is somewhat unclear to me,  and the MoE router is under-explored, like the ambiguity about how semantic alignment influences expert selection?\n\nPerformance improvements, while steady (~2.5–3.6 mIoU in the large joint setup), are moderate relative to the engineering complexity. Discussion on scalability and convergence stability for larger-scale or longer training runs is limited. Overall, the contribution is more an engineering advance than a conceptual breakthrough.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pJzyWRIk9U", "reviewer": "Reviewer_9WmA", "summary": "The paper presents a novel evaluation scheme -- multi-dataset 3d semantic segmentation without domain labels -- for existing indoor and outdoor 3d datasets. To solve this novel task authors presents Point-MoE -- and updated version of PTv3 with MoE layers inside transformer blocks. This model is trained on mixtures of indoor / outdoor / indoor + outdoor 3d semantic segmentation datasets. Point-MoE demonstrate state-of-the-art results on 6 datasets in its formulation.", "questions": "- How is the performance of Point-MoE compared to other multi-dataset 3d segmentation methods including: ARKit LabelMaker, Sonata, Mosaic3D? Can multi-dataset + MoE scheme be adapted to show actual single-dataset state-of-the-art results?\n\n - How is the proposed training without *dataset label* compared to the one from UniDet3D? Why UniDet3D shows state-of-the-art results on all benchmarks, and Point-MoE not?\n\n - Why in Tab. 1 single-dataset training on Structured3D is significantly better than multi-dataset?\n\n - Can the proposed MoE scheme reduce the number of active parameters several times like for other state-of-the-art transformers?\n\n - As there is no difference between 8 and 4 experts in MoE in Tab. 3h, why not to check 1 or 2 experts? Why the performance is not always correlate with number of active experts in Tab. 3b? Isn't it contradictory to result in recent LLM papers, like mentioned DeepSeek one?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper presents novel formulation of multi-dataset 3d semantic segmentation. Namely, authors claim that previous 3d understanding works in multi-dataset settings use *dataset label* (information from which dataset it the current point cloud) for train and / or test stages. I agree that not using this *dataset label* during test is a useful formulation.\n\n - Authors improve existing PTv3 method with novel MoE layers. These experts are claimed to split information between datasets and dataset domains, improving overall accuracy and inference speed.\n\n - The model is trained in different scenarios: single-dataset, indoor multi-dataset, indoor+outdoor multi-dataset. After training it is tested on single datasets from train set, or even zero-shot ones.\n\n - Authors provide comprehensive ablations of their MoE architecture and training parameters.", "weaknesses": "- Main weakness is the absence of mentioning of some important 3d understanding papers that push to exactly the same multi-dataset direction. The only 2 multi-dataset methods mentioned in the paper are PPT [CVPR 2024] for semantic / instance 3d segmentation, and One-for-All [NeurIPS 2024] for 3d detection. I believe this topic is much broader than just 2 papers, including ARKit LabelMaker [CVPR 2025], UniDet3D [AAAI 2025], Sonata [CVPR 2025], Mosaic3D [CVPR 2025], and Uni3D [CVPR 2023].\n  - Both PPT and One-for-All are claimed to use *dataset label* of each scene during train and test. And the main novelty of proposed Point-MoE is the elimination of this *dataset label*. However, I think it was already explored in UniDet3D, which is trained on a mixture of 6 datasets (incl. ScanNet and S3DIS) without *dataset label* during train or inference, merging all class labels to the same set. Also UniDet3D demonstrates the importance of multi-dataset training by achieving state-of-the-art results on 6 datasets. Which contradicts with the results of Point-MoE, which are actually far from state-of-the-art on all single datasets. Batch data mixing from Tab. 4 was also used in UniDet3D. Finally, I think Uni3D can be mentioned here, as trained on 3 outdoor datasets simultaneously.\n  - Regarding multi-dataset 3d semantic segmentation, it is very much progressed in recent ARKit LabelMaker, Sonata, and Mosaic3D. These papers combine the same set of indoor ScanNet+S3DIS+Structured3D or outdoor KITTI+nuScene+Waymo as is explored in Point-MoE. Mosaic3D even goes further, unifying class labels with VLM annotation, resulting in annotation-free solution without *dataset label*.\n\n - Second major weakness is that the proposed multi-dataset pipeline requires more data compared to existing single-dataset methods, however is significantly inferior to them in accuracy.\n   - In Tab. 1 single-dataset training of PTv3-S on Structured3D achieves 81.5 mIoU and multi-dataset training - only 69.0\n   - On ScanNet multi-dataset Point-MoE (based on PTv3) achieves 76.0 mIoU, however original PTv3 with just a subset of this data already achieves 77.5 mIoU. And this disadvantage is even bigger on all other datasets: S3DIS, nuScenes, and SemanticKITTI.\n   - Recent multi-dataset methods achieves even much higher mIoU on ScanNet: 78.6 for PPT, 79.1 for ARKit LabelMaker, 79.2 for Sonata, and 80.5 for DITR [CVPRw 2025]. This gaps remain significant on all other datasets: S3DIS, nuScenes, and SemanticKITTI.\n\n - Performance gains of MoE look limited. E.g. in Tab. 1 MoE brings 59M parameters to 52M active ones for PTv3-S and 100M to 59M for PTv3-L. This sounds not very significant for the case of using as much as 8 experts on MoE. E.g. with help of MoE GPT-OSS transforms 117B to 5.1B active, or 21B to 3.6B, which sound significant to practical applications. Also Tab. 3h doesn't show the benefit of large number of expert as there is no difference between 4 and 8 experts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)