Mamba Goes HoME: Hierarchical Soft Mixture-of-Experts for 3D Medical Image Segmentation
Authors: Szymon Plotka
Gizem Mert
Maciej Chrabaszcz
Ewa Szczurek
Arkadiusz Sitek
Venue: NeurIPS 2025 poster ()
Keywords: mixture of experts
state-space models
segmentation
medical imaging
Primary area: deep_learning
TL;DR: We propose a novel hierarchical soft mixture-of-experts for 3D medical image segmentation

Abstract: In recent years, artificial intelligence has significantly advanced medical image segmentation. Nonetheless, challenges remain, including efficient 3D medical image processing across diverse modalities and handling data variability. In this work, we introduce Hierarchical Soft Mixture-of-Experts (HoME), a two-level token-routing layer for efficient long-context modeling, specifically designed for 3D medical image segmentation. Built on the Mamba Selective State Space Model (SSM) backbone, HoME enhances sequential modeling through adaptive expert routing. In the first level, a Soft Mixture-of-Experts (SMoE) layer partitions input sequences into local groups, routing tokens to specialized per-group experts for localized feature extraction. The second level aggregates these outputs through a global SMoE layer, enabling cross-group information fusion and global context refinement. This hierarchical design, combining local expert routing with global expert refinement, enhances generalizability and segmentation performance, surpassing state-of-the-art results across datasets from the three most widely used 3D medical imaging modalities and varying data qualities. The code is publicly available at https://github.com/gmum/MambaHoME.

Mamba Goes HoME:
S
Hierarchical Soft Mixture-of-Experts for 3D Medical Image Segmentation
Szymon Plotka Gizem Mert² Maciej Chrabaszez Ewa Szezurek!2 Arkadiusz Sitek
1 Faculty of Mathematics, Informatics, and Mechanics, University of Warsaw, Poland Faculty of Mathematics and Computer Science, Jagiellonian University, Poland Institute of AI for Health, Helmholtz Munich, Germany
4 Faculty of Electronics and Information Technology, Warsaw University of Technology, Poland NASK National Research Institute, Poland
6 Faculty of Radiology, Massachusetts General Hospital, USA
7 Department of Radiology, Harvard Medical School, USA
Abstract
In recent years, artificial intelligence has significantly advanced medical image segmentation. Nonetheless, challenges remain, including efficient 3D medical image processing across diverse modalities and handling data variability, In this work, introduce Hierarchical Soft Soatturr-of--xpeerr (HoME), two-level token-routing layer for efficient long-conter contect modeling, specifically designed for 3D medical image segmentation. Built on the Mamba Selective State-Space Model (SSM) backbone, HoME enhances sequential modeling through adaptive expert routing. the first level, Soft offture--f-EEperrts (SMoE) layer partitions input sequences into local groups, routing tokens to specialized per-group experts for localized feature extraction. The second level aggregates these outputs through a global SMoE layer, enabling cross-group information fusion and global context refinement. This hierarchical design, combining local expert routing with global expert refinement, enhances generalizability and segmentation performance, sur- passing state-of-the-ad results across datasets from the three most widely used 3D medical imaging modalities and varying data qualities. The code is publicly available at github.com/ Baaimn/gu////aaaaaaaaea
1 Introduction
hinree-iimensoonl (3D) medical image segmentation lies the core of computer-aided diagnosis, image-guided interventions, and treatment planning across modalities such Computed Tomography
(CT) [51, 12, 44], Magnetic Resonance Imaging (MRI) [33, 48, 21], and Ultrasound (US) [60, 24, 6]. A key characteristic of medical imaging data is its hierarchical structure: local patterns, such as tumor lesions, are embedded within larger anatomical structures like organs, which themselves follow consistent global arrangement [20]. We hypothesize that models capable of capturing these local-to- secal-to-global spatial hierarchies in 3D medical data can enhance segmentation performance and yield latent representations that generalize effectively across diverse imaging modalities.
Convolutional Neural Networks (CNNs) provide local feature extraction with linear complexity in the number of input pixels, but their limited receptive fields hinder their ability to capture global
Correspondence to: Szymon Plotka (SZYMON. PLOTKA U...DUPP.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

spatial patterns [42, 57, 22, 52, 45]. In contrast, Vision Transformers (ViT') [16, 5, 31] leverage global attention mechanisms model long-range dependencies. However, their quadratic complexity in the number of tokens makes them computationally costly for high-resolution 3D data, posing significant challenges for scalability. Moreover, while many ViT-based models [53, 18, 38, 30, 27, 47] incorporate multi-scale feature extraction and attention mechanisms, they still struggle to effectively aggregate fine-to-coarse semantic information, particularly in dense prediction tasks such as volumetric segmentation. Although these models demonstrate improved performance, they do not explicitly model global and local spatial patterns and their mutual arrangement. Modeling the transition from local to global patterns requires processing long sequences and capturing long-range dependencies, imposing prohibitive memory and computational demands on current architectures [9].
Recently, Selective State-Space Models (SSMs), such as Mamba [19. 15], have emerged as efficient alternatives to ViTs by offering linear complexity the number of tokens to capture long-range dependencies, including in 3D medical imaging [54, 46, 29, 11, 50J. Although SSMs effectively capture global context with lower computational cost than attention- methods, they are not inherently designed to adaptively handle diverse local patterns in medical data. Efficient management of such local patterns in complex, multi-scale data while maintaining scalability is achieved through Mixture-of-Expert (MoE) frameworks, which dynamically route features specialized subnetworks. MaEeb methods have gained prominence across domains such as language modeling [4, 56, 26], vision tasks [39, 59, 41, 14, 17], multimodal learning [34, 8, 55], and medical applications [25, 13. 49, 38.. However, combining the global efficiency of SSMs with the localized aduptability of MoE remains largely unexplored, particularly in 3D medical imaging, where balancing efficiency and generalization across multi-modal datasets under resource constraints is paramount.
To address these challenges, we introduce the first-in-class model that smoothly integrates Mamba with hierarchical Soft MoE multi-stage network for local-to-global pattern modeling and 3D image segmentation with competitive memory and compute efficiency, while maintaining state-of- the-art segmentation performance. Specifically, our contributions are as follows:
1. We introduce Hierarchical Soft Mixture-of -f-ppet-of-Experts (HoME), a two-level, token-routing MoE layer for efficient capture of socal-to-glober pattern hierarchies, where tokens are grouped and routed to local experts in the first SMoE level, then aggregated and passed via a global SMoE in the second level,
2. We design a unified architectural block that integrates Mamba's 's SSMs with HoME, combin- ing memory-efficient long- seeuence processing with hierarchical expert routing,
3. We embed the above novel solutions into a multi-stage U-shaped architecture, called Mamba- HOME, specifically designed for 3D medical image segmentation, where the integration of hierarchical memory and selective state-space modeling enhances contextual representation across spatial and depth dimensions.
Through comprehensive experiments four publicly available datasets, including PANORAMA [2), AMOS [23], FeTTA 2022 [37], and MVSeg [10] as well as one in-house CT dataset, we demonstrate that Mamba-HoME outperforms current sfate-of-the-art methods in both segmentation accuracy and computational efficiency, while generalizing effectively across three major 3D medical imaging modalities: CT, MRI, and US.
2 Methodology
2.1 Preliminaries
Selective State-Space Models (Mamba). Our network builds on the Mamba layer [19], designed for long-sequence modeling with linear computational complexity. Unlike Transformer based ar- chitectures, which exhibit quadratic complexity with respect to sequence length, Mamba achieves linear-time processing through continuous-time recurrent formulation with input-dependent pa- rameters. Given a feature sequence = = { E Bxxxx the hidden state ht and output y, are updated as follows:
h4 = A(z)), + B(z))
Ve = c
(I)
where A(-), B(-), C(-) are learnable input-dependent linear mappings. This formulation allows Mamba to capture both short- and long-range dependencies with memory and computational

---

complexity of ((N.). In our architecture, the Mamba layer serves as backbone for volumetric sequence modeling, efficiently capturing spatial dependencies across 3D data inputs.
Soft Mixture-of-Experts (SMoE). SMoE framework [39] introduces modular compu- tation strategy where each input token is dy- namically routed multiple experts, and their outputs are combined using soft routing weights. Given an input tensor IR BxN xd and ex- perts, a gating network computes routing prob- abilities pb.n.e for each expert and token Fb,
Pleduaps
Lx
FFN:
FFN
PFNE
HoME
yb.n Σ Ph.n.e (FNN(
Rd Mb.n
Router
Norm
(2)
B
Mamba
where FFN, denotes the e-th expert network and Eez1 2 Pb.n.e = 1 ensures a valid soft as- signment. Each expert is typically a small feed- forward network, and the gating network is learned per-token function (e.g., an MLP) that assigns weights dynamically. While SMoE im- proves model capacity and modularity, global routing over all tokens incurs a computational cost of O(NdE), which becomes prohibitive for high-resolution 3D inputs where N can reach millions. Moreover, SMoE lacks spatial locality and hierarchical aggregation, both of which are important for structured volumetric data.
FFN.
FFN
FFN
G 0g de
p-01
p-02
p-0.3
Flattan
Router,
GSC
C1
G3
C₁
(a) Hierarchical Soft Mixture-of-Experte (HoME)
(b) Mamba-HoME Block
(Mamba-HoMEB)
overview of the HoME layer and Mamba-HoME Block design. (a) The HoME layer processes G, groups of K; tokens through a hier- archical soft routing. Tokens are soft-assigned to slots and projected by E₁ local experts (FFN()) for intra-group feature extraction (1))). Subse- quently, Eg global experts (FFN(2)) refine these representations to enable inter-group group communi- cation and global context integration (2) (b) Mamba-HoMEB sequentially integrates Gated Spatial Convolution (GSC) for local spatial priors, which is flattened into ID sequence for long- range modeling within the Mamba layer. Follow- ing specialized refinement by the HoME layer, the output is reshaped to restore its original dimension- ality. Dynamic Tanh ensures stable normalization and efficient computation across the architecture
Figure
Notation. At encoder stage E {l..... I. the input feature sequence is x E IR Bx.., where B is the batch size, N; is the sequence length, and d is the feature dimension. The sequence is divided into G; / K2] groups of up to K; tokens each. We define E; as the number of experts per level and S, the number of slots per expert. The total number of expert-slot pairs (total slots) is M, E, S,, and the learnable slot embeddings are E(() E R Md Tokens are assigned to slots using weights As.g.k.. The first-level experts F,, process the grouped slots to produce outputs y which are then refined by of second-level experts Eg, to produce the final outputs (2)
2.2 Hierarchical Soft Mixture-of-Experte (HoME)
We introduce the HoME layer (see Figure (a)), which enhances feature processing through a hierarchical two-level structure. HoME extends the SMoF concept with a hierarchical two-level routing structure that processes grouped tokens locally and enables inter-group information exchange efficiently. It comprises three key steps: (1) grouped slot assignment for token processing, (2) first-level MoE processing for local feature extraction, and (3) second-level MoE refinement for global feature extraction, allowing inter-group communication.
Grouped Slot Assignment. hierarchical vision encoders, particularly for dense 3D inputs (e.g.. volumetric data), the sequence length is highest in early stages due to large spatial resolutions. Global expert routing on these long sequences causes high computational and memory costs, as each token is compared to all expert slots, yielding complexity of O(N; Mt). where N; is the sequence length at stage i and M, is the total number of expert slots. To address this scalability bottleneck, introduce Grouped Slot Assignment, locality aware routing mechanism that divides the input

---

sequence into groups and performs soft assignment independently per group. We define the group size as K; K1 p--1 (0 < 1), and zero-pad sequences to N G; Kü giving & E BB Njxd Each group Igg E R Bx Bxxxx is routed independently. Assignment logits are computed via a dot product between each token and learnable expert-specific slot embeddings stoos R Let e1 {l...., E1,5)
and & E (I,...,,5. S.}. The logits are as follows:
d
Sb.g.k.er.
Tb.g.k.
E E
Se SERRGGXKXR,SS
(3)
==1
An optional binary mask M E (0.1) indicating valid (unpadded) tokens, is extended to M E {0, 1) Bx after padding. To prevent padded tokens from affecting expert assignment, the logits for invalid tokens are masked by setting:
{ Sb.g.kee.
if Mo, gk,tk = otherwise.
Sb.g.k.e..
Expert-slot pairs are flattened (m = 1 1)5 + s, Mr = E. S.). and normalized with softmax: exp(St.g.k.m)
A RRBXGGXK
k..m
(5)
Em exp(St.g.k..m)
Each slot representation is computed as weighted aggregation of tokens within its group:
Bb.g.e1.5.j
Ab.g..m Ib.g.
I E BBBxxXXL.XXxx
(6)
By performing routing within groups, peak memory usage reduced and locality is preserved, enabling efficient hierarchical token-to-expert assignment. While the total computational complex- ity remains ONN,M,d) the smaller group-wise computations allow scalable processing of long sequences while being memory-efficiemt
Hierarchical Expert Processing. Let g Bx denote the grouped slot representations after flattening the (ei, s) dimensions of G. The first level routes slots within each group to subset of Ei,; experts, promoting local specialization and feature refinement. This produces group-specific outputs while preserving slot structure. For each group g (1, ..... Gi}, the gating network computes routing weights for the E,, experts (FFN,, FFN,,.FFN,.. FFN
Router (Zt ((ue)) softmax
MLP1
1 M₁
Σ
b.g.m
m=1
M where Sh ( b.g.m m=1 the expert dimension, yielding Router, R. Each expert, implemented as FFN,, R. Mixd R.M. xd processes the group's slots independently. The output for expert {1,..., F1,)) is
Mr E M xd (11) are gating parameters, and the softmax normalizes over gale
Yb.g.e:
Router (Zh.gi obee)) FFNN, (Fb.g):
(8)
and the aggregated output is:
E
vb.9
yb.g.e1
y e BBxxGxMixd
(9)
e1=1
Dynamic Slot Refinement. The second level routes group slots to Eg, experts for global fea- ture refinement. The tensor y (1) is transformed into ÿ MG..M. before being passed through the second-level experts. The gating network computes routing weights for E2, experts (FFN,, FFN,,... ... FFNE..):
Routerz (ū: o ((2))) softmax softmax(MMLP (MLP)
(10)

---

where G are the gating parameters, producing Routery E BBE Each second-level expert, implemented as FFNe, ((G. (G) processes y. The output for expert e2 E {1...., E2 is:
yb,02
(ou 0 0caee] e2
FFN.2 (i6);
(11)
and outputs are aggregated into:
Ba,
y
yb,e21
yl E
(12)
The final stage reconstructs structured output using an attention-based combination emphasize relevant slots and remove padding, yielding compact, task- task-aligned representation. The tensor y(2) is reshaped to IR Bxx, x M; x
For batch element 6, group and token k E
the output is:
gb.g.k
Wb.g.km Vb.g.m
(2)
(13)
where Vm.g.m E Rd, and W..g.km E [0, 1] are attention weights satisfying E Wb.ggk.. After undoing padding, the final output is e E BxNxxd
(2)
2.3
Mamba-HoME Block
Figure 1(b) provides an overview of the proposed Mamba-HoME Block (Mamba-HoMEB), The Mamba-HoMEB extends the Mamba layer by incorporating hierarchical downsampling, Gated Spatial Convolution (GSC) module [54], and a Hicrarchical Soft Mixture-off-xpert (HoME) layer (sce
Section 2.2). To improve gradient stability and computational efficiency in SSMs, Dynamic Tanh (DyT) [58] normalization, defined as:
foyr(x) = w - tanh(a x) + b,
(14)
where E Bxd is the input tensor, w, b E Rd are learnable per-channel vector parameters, and a E IR is shared scalar. DyT applies point-wise nonlinearity, leveraging the bounded nature of tanh to stabilize gradients. Unlike Layer Normalization [7], it avoids costly mean and variance computations, reducing overhead while maintaining ((B)) complexity. DyT accelerates training and validation with performance par with normalization-ban based alternatives.
Given a 3D input volume E I Bx Dx XW the initial feature map x E Bx X x is and its
extracted by a stem layer. This feature map 2f is then passed through each corresponding downsampling layers. For the I-th layer (1 E {0, 10......,,4. 1}) within stage the representations for the i-th Mamba-HoMMal HoMEB are given by:
a!!' =cs==Jos(xxr])
F MMambaa (J [Norm (1lmm((())) (x(()) + =!! 2.. 1+1 SHHME (JNom (i()) + z/. (15)
where fasc denotes Gated Spatial Convolution module, fMamba the Mamba layer, fNorm corresponds to DyT normalization (see Eq. 14), and SHOEE the HoME layer. After applying fasc. the feature map is flattened along the spatial dimensions into sequence (length N₂) before being processed by fambaa- The output of the HoME layer is then reshaped back into the original volumetric form to yield x; !+1
The HoME layer operates hierarchically at each encoder stage E {l..... 1}, with the number of first-level experts denoted by E, and the group size (i.e., the number of tokens processed jointly within each local group) denoted by K; The number of experts E, increases monotonically with stage depth, reflecting increased specialization (E, E2 < decreases, enabling progressively finer-grained processing (K1 > K₂ > ... K2). In addition to first-level experts, each stage employs a second-level expert set E2,,, which scales proportionally with F,, i.e., E2, 2F, This second level facilitates global context integration across groups, enhancing inter-group communication. The HoME operation at each stage thus combines local expert routing with global feature aggregation (see Section 2.2).
E,). while the group size K;
5

---

2.4 The Mamba-HoME Architecture
Building upon SegMamba [54], we introduce a U-shaped encoder-decoder network, called Mamba- HoME, designed for 3D medical image segmentation. It leverages Mamba-basee encoder backbone to efficiently capture both long-range dependencies and local features. The model processes an imput 3D volume x E IR Bx Dx Hx xW W and produces a segmentation mask E R. Bx BxCCx where B is the batch size, C is the number of input channels, C" is the number of output classes, and H, W are spatial dimensions.
The encoder begins with stem layer that produces the initial feature map 5f (see Section 2.3), followed by / hierarchical stages. Each encoder stage { .......]] / applies a Mamba-HoMBil producing intermediate representations o for 0, ..... L; 1, where L; denotes the number of ................................ layers in the i-th Mamba-HoMEB, and concludes with downsampling operation:
2 = Downsample,(r? (x."),
(16)
where downsampling halves the spatial dimensions and doubles the channel depth: H1+1 [ W ++1 [W//2 and C,+1 = 2C,. This process results sequence of encoder feature maps {a 22 27 }. with x serving as the bottleneck representation.
D++1 = [D/2]]
The decoder mirrors the encoder with / upsampling stages. From the bottleneck uo = x each decoder stage = -,..... I fuses the corresponding encoder skip connection with the upsampled decoder features:
U = UpBlock, (x e Up(ug-1));
(17)
where u E and @ denotes channel-wise concatenation. The upsampled spatial dimensions are D, [D/2 H, [H/2/-1J, and W, [W/2/=1.. Fach UpBlock, refines the combined features and maps them to the target resolution for the next decoding stage, and final prediction layer converts u 1 to the segmentation mask E BxC' xDxHxW
is the skip connection from the encoder stage j.
The computational complexity of Mamba-HoME scales as O(BN,)) for the Mamba layer and { G. (E, + F22,) ) L, d) for the HoME layer at stage i, where N; is the token sequence length, d is the hidden dimension, G; is the number of groups, and Eg + F2g, reflects the total number of experts. Compared to Transformer- based models (O(BN?d)) d)), this linear scaling in N, ensures efficiency for large 3D volumes (e.g., N₂ zu 10 tokens), with HoME adding a modest overhead proportional to the expert count.
Experiments
In this section, we evaluate the performance of the proposed Mamba-HoME against state-of-the-ar methods for 3D medical image segmentation (see Section 3.3). We perform an ablation study un- derstand the importance of different configurations and parameters of Mamba- HoME. Moreover, we compare the segmentation performance of Mamba- HoME trained from scratch with a version pre-trained using a supervised learning approach (see Section 3.4). Finally, in Section 3.5, we investigate how Mamba-Hon HoME generalizes to new modalities. Throughout this section, the best and second-best performing results are bolded and underlined, respectively.
Table 1: Segmentation and efficiency performance on the PANORAMA and in-house test sets. PDAC and P denote pancreatic ductal adeno- careinoma and panereas, respectively. Methods marked with (*) are initialized with pre-trained weights.
Method
DSC (%) f
mDSC
mHD95
Params (M)
GPU
IS())
PDAC
P
(%)
(mm)
(G) J
VoCo-B (*) [53)
40.5
86.3
71.6
15.9
53.0
17.1
1.5
Hermes [18]
48.2
87.8
75.1
10.4
44.5
17.4
1.9
Swin SMT [38]
49.4
87.0
75.0
10.1
170.8
15.8
1.3
VSmTrans [30]
50.3
87.2
75.4
9.5
47.6
11.1
1.7
uC 3DU-Net [22]
$2.0
88.2
76.6
8.4
21.7
13.6
1.0
Swin UNETR [47]
46.3
87.4
74.2
10.6
72.8
17.1
1.5
SegMambu [54]
49.7
88.5
76.0
10.1
1.2
SuPreM [28]
51.7
88.3
76.6
62.2
17.1
1.5
Mamba-HoME
54.8
88.3
77.5
170.1
11.1
Mamba-HoME (*)
56.7
88.5
78.2
4.3
170.1
11.1
1.5
Inference speed (IS) is standardized to 1770 ms.

---

Table 2: Segmentation performance and generalizability on AMOS-CT and AMOS- MRRI subsets for four training settings. (1) CT: aigle-modality CT training: (2) CT-:MRI: CT pre-training followed
by MRI fine-tuning; (3) MRI: single-modality MRI training: (4) CT+MRI: joint multi-moda training.
Methods marked with (*) are initialized with pre-trained weights.
mDSC (%) 1
mHD95 (mm) 1
Method
CT
CT +
MRI
MRI
CT
MRI
CT
CT -
MRI
MRI
CT
MRI
Hermes [18]
85.3
84,8
80.7
82.9
11.3
8.2
13.0
7.3
Swin SMT [38]
85.7
83.2
63.2
81.2
47.2
9.0
22.5
17.7
VSmTrans [30]
85.3
84.0
74.6
78.0
39.1
14.1
18.5
15.2
UC 3DU-Net [22]
82.7
84.5
68.6
84.1
19.1
9.1
16.9
12.2
Swin UNETR [47]
84.3
84.2
75.0
81.2
43.6
13.3
14.7
12.4
SegMambu [54]
86.0
84.4
80.2
84.7
26.3
8.2
SuPreM((") [28]
86.0
83.9
70.3
83.5
16.5
18.2
11.8
Mamba-HoME
86.3
84.8
81.0
85.1
18.7
Mamba-HoME (*)
87.3
85.0
82.3
86.4
12.2
8.0
11.0
7.2
SuPreM was pre-trained the AMOS- CT training set, while VoCo-B was pre-trained on both the training
and validation sets. Here, we removed VoCo-B from benchmark as may lead to an unfair comparison.
3.1 Datasets
Pre-training. We utilize large-scale, publicly available datasets across two imaging modalities: AbdomenAtlas 1.1 [40, 27] for CT scans and TotalSegmentttto MRI [1] for MRI scans. Both datasets provide high-quality, voxel-wise anatomical annotations of abdominal structures, ensuring a diverse and robust representation for multi-modal pre-training.
Training and fine-tuning. We use datasets covering three primary 3D medical imaging modalities, including publicly available PANORAMA (CT) [2], AMOS (CT and MRI) [23]. FeTA 2022 (fetal MRD) [37], MVSeg (3D US) [10], and an in-house CT dataset. detailed description the datasets can be found in Appendix A.
3.2 Implementation details
The experiments conducted workstation equipped with 8 x NVIDIA H100 GPUs. For implementation, we employ Python 3.11, PyTorch 2.4 [35], and MONAI 1.3.0 within a Distributed Data-Parallel (DDP) training setup.
All training is performed using the LDiECE loss function and the Adam optimizer, with an initial learning rate of le-4 controlled by a cosine annealing scheduler [32], weight decay of 1e-5, and a batch size 2. All models are trained with 32-bit floating-point precision to ensure numerical stability and to standardize the training process across all experiments. A detailed description of the implementation can be found in Appendix B.
3.3 Comparison with state-of-the-an methods
We compare our proposed Mamba- HoMM with eight state-of- the-art approaches for 3D medical image segmentation, includ- ing uC 3DU-Net [22], Swin SMT [38], VoCo-B [53], SuPreM
Table 3: Segmentation performance on FeTA via 5-fold cross-validation. Methods marked with (*) are initial- ized with pre-trained weights.
[28], Hermes [18), Swin UNETR [47], VSmTrans [30], and SegMamba (baseline) [54], across four publicly available and one in-house dataset, covering diverse anatomical structures and imaging modalities, such as CT, MRI, and 3D US. Notably, both VoCo-B and SuPreM pre-trained on large-scale CT scans using self-supervised and supervised learning approaches, respectively. Additionally, we evaluate Mamba-Hok. trained from scratch against Mamba- HoMME pre-trained with super- vised learning approach to assess the impact pre-training on segmentation performance. Detailed quantitative and more qualitative results for benchmarking datasets can be found in Appendix C, and Appendix E, respectively.
mDSC
mHD95
Method
(%)*
(mm) 1
VoCo-B (") [53]
86.0
4.0
Hermes [47]
86.5
4.0
Swin SMT [38]
85.9
2.4
VSmTrans [30]
86.1
2.3
3DU-Net [22]
85.9
3.5
Swin UNETR [47]
86.2
2.5
SegMamba [54]
85.9
3.5
SuPreM (*) [28]
85.3
3.6
Mamba-HoMMi
87.5
2.1
Mamba-HoMBB (")
87.7
2.0
7

---

ON irpur dios
Ground fruhh
Samberi HOME
(01 SUPraM
LC STUAMM
Segiaamba
1s1 VonTrone
[N] Hermes
Figure 2: Qualitative segmentation results from top to bottom: CT, MRI, and 3D US. From left to right, each column shows the input slice, ground truth, the proposed Mamba- Mamba-HoME, and the five next best-performing methods.
Quantitative results. Results for our proposed method, Mamba-HoME, on the PANORAMA and in- house datasets are shown in Table Results for other modalities, including AMOS (CT/MRI), FeTA 2022 (fetal MRI), and MVSeg (3D US), are presented in Table 2, Table 3, and Table 4, respectively.
Our proposed method, HoMe, demonstrates consis- tent performance improvements over state-of-the basclines across all benchmark datasets and three imaging modalities. Evaluated under two distinct configurations (scratch and pre- trained), Mamba-HoME achieves superior segmentation ac- curacy, obtaining the best results in terms of both DSC and HD95. Despite having a relatively large number of parameters (170.IM) compared to competing methods, exhibits low GPU memory usage during inference (see Table 1), a crucial advan- tage for processing high-resolution 3D medical data. Although inference approximately 30% slower than the baseline, the performance gains present compelling trade-off between curacy and efficiency. The Wilcoxon signed-ran rank test indicates significant difference between Mamba-HoME and other state- of-the-art art methods, with a significance threshold of p < 0.05.
Table 4: Segmentation performance the MVSeg test set. Methods marked with (*) are initialized with pre-trained weights.
mDSC (%)*
mHD95
Method
(mm) 1
VoCo-B (*) [53]
84.3
5.1
Hermes [18]
83.5
5.3
Swin SMT [38]
83.4
4.9
VSmTrans [30]
84.4
6.2
uC 3DU-Net [22]
83.9
4.7
Swin UNETR [47]
84.4
4,8
SegMamba [54]
83.8
5.8
SuPreM (*) [28]
84.3
5.0
Mamba-HoMF
84.8
4.3
Mamba-HoME
85.0
4.1
Qualitative results. Figure 2 presents qualitative comparison of our proposed Mamba HoME method against the five top-performing baselines across three primary 3D medical imaging modalities: CT, MRI, and US. These modalities exhibit different organ contrasts, noise levels, and resolutions. Mamba- Hamba-HoME demonstrates consistent improvements in segmentation quality across these scenar- ios. In the first row, effectively handles small and closely located structures, showing precise boundary delincation while reducing common artifacts seen in baseline predictions. The second row highlights its capability to accurately segment organs of various shapes and sizes, under low image quality conditions, with reduced susceptibility to over- or semeetteemmntttionn. The third row illustrates Mamba-HoME's robustness in handling noisy and low-resolution data, maintaining clear and anatomically accurate boundaries.
3.4 Ablation studies
In this section, we investigate the impact of several factors on the performance of Mamba-HoME: (1) the parameters of the HoME layer, including the number of experts in the first (E₃) and second (E2) levels, the group size (K), and the number of slots per expert (S); (2) the effect of Dynamic Tanh normalization compared validation speed in SSMs and overall performance; and (3) the impact of the pre-trained model in supervised learning approach. For each configuration, we evaluate the number of model parameters, GPU memory usage, and average DSC across three datasets?. Further details regarding these ablation studies are provided in Appendix D.
Layer Normalization, specifically its influence on training and
In these experiments, use PANORAMA (PANO), AMOS-CT (AMOS). and FeTA
8

---

Effect of the number of experts. We evaluate the impact of varying the number of experts at each encoder stage (i), where
Table 5: Quantitative segmentation performance of Mamba- HoME with varying numbers of experts at each encoder stage i.
21,2,3, 3, 4), in a two-level HoME layer. For a fair com- parison, we keep the group size con- stant (K E {2048, 1024, 512, 256)) and set the number of slots to 4. Table 5 shows that the configu- ration with E1 E {4,8, 8, 12, 16) ex- perts at the first level and Eg {8, 16, 24, 32] experts at the second level achieves the best trade-off be- tween segmentation performance and parameter efficiency. This setup also requires the fewest parameters and the lowest GPU memory usage.
Number Experts (K)
Params (M) 4
GPU
mDSC (3)) 1
(G)
PANO
AMOS
FeTA
Ei
8, 12, 16]
170.1
11.1
77.5
86.3
87.5
Fg
24, 32)
E
24,
2
E2
[8 11. 24, 48
277.8
12.2
76.3
86.2
87.2
E. [8. 26,6,,2, 24,
359.2
85.9
F2 = [16, 32, 48, 96]
E 116, 82, 48, 64
E [16, 32, 48, 64
367.0
13.0
77.5
86.1
87.4
Effect of the group size. We evaluate the impact of the group size at each en-
Table 6: Quantitative segmentation performance of Mamba- HoME with varying group sizes at each encoder stage i.
coder stage (i), where E 2,,2,3, 4). in two-level HoME layer. For fair comparison, we keep the number of experts constant (E) {4,8, 12, 16) and F2 E {8. 16, 24, )2)) and set the number of slots to shows that Mamba-HoME achieves optimal performance with group sizes K E {2048, 1024, 512, 256]. while also minimizing GPU memory usage.
Group size (K)
Params
GPU
mDSC (%)
(M) L
(G)
PANO
AMOS
FeTA
[1024, 512, 256, 128)
170.1
11.1
77.2
86.1
87.4
2
[2048, 1024, 512, 256]
170.1
11.1
77.5
86,3
87.5
3
[2048, 1024, 512, 256]
277.8
12.2
76.8
86.2
87.3
4. Table 6
4 2048, 1024. 512]
170.1
11.1
27.4
86.1
87.4
The number of experts in the first and second levels of the HoME layer are equal (E, = E2 = [8, 16, 21,48). 48).
Effect of the number of slots per expert. We examine the im- pact of the number of slots (S) per expert (E) at each encoder stage (i), where i E {1, 3, 4). in a two-level HoME layer. For a fair comparison, we keep the number of experts constant (E1 E (4, 8, 12, 16) and Fg E (8, 16, 24, )2]) and the number of groups fixed (K E (2048, 1024, 512, 256)). Table shows that Mamba-HoME achieves optimal performance at S = 4 across all evaluated datasets, representing a sweet spot for the number of slots per expert. Variations in the number of slots (S E (1,2,8) do not yield sigmificant performance improvements, with other slot counts resulting in suboptimal performance while keeping the number of parameters and GPU memory constant.
Table 7: Quantitative segmen- tation performance of Mamba- HoME with varying numbers of slots per expert.
Slots (8)
mDSC (%)
PANO
AMOS
FeTA
1
76.2
85.9
87.3
77.2
86.0
3
77.5
86.3
87.5
8
76.5
86.1
87.4
Effect of Dynamic Tanh normalization in SSMs. While DyT accelerates CNN- and Transformer-based architectures [58], we in- vestigate its effectiveness in SSM-based architectures compared to Layer Normalization [7]. As shown in Table 8, segmentation perfor- mance remains largely unchanged, but DyT improves both training and inference speed by approximately 6% based on experimental runtime measurements
Table 8: Quantitative segmen- tation performance of Mamba- HoME trained with Layer Nor- malization (LN) and Dynamic Tanh (DyT).
mDSC (56
#
Dataset
LN
DyT
Impact of supervised pre-training. We evaluate the efficacy of the proposed Mamba-HoME model. pre-trained under a supervised learning paradigm on publicly available datasets, including 8,788 CT and 616 MRI scans with voxel-wise annotations. The evaluation spans three primary 3D medical imaging modalities and various anatomical regions. Overall, Mamba-HoME outperforms state-of- the-art methods, surpassing existing approaches in both DSC and HD95 across all evaluated datasets. These quantitative results highlight the effectiveness of supervised pre-training in enhancing segmen- tation accuracy and robustness. A key feature of Mamba-HoME is its cross-modal generalization,
PANO
2
X
v
77.5
AMOS
V
X
86.2
X
V
86.3
/
x
87.5
FeTA
87.4
For each dataset in this experiment, adopt the same settings described in Appendix B.

---

enabled by modality-agnostic feature representations, Pre-trained on CT and MRI scans, the model demonstrates superior adaptability to specialized tasks, such as fetal brain MRI segmentation in the FeTA dataset and 3D ultrasound mitral valve leaflet segmentation in the MVSeg dataset.
This adaptability highlights Mamba-HoME's ability to mitigate challenges posed by variations in modality, resolution, and clinical context. Moreover, its consistently high performance across heterogeneous datasets underscores its potential for practical deployment, where robust, modality- agnostic feature representations and precise segmentation are essential for scalable, real-world medical imaging applications. As shown in Figure 3, supervised pre-training significantly improves Mamba-Ho.ME's s performance compared to training from scratch or baseline methods, reducing artifacts and enhancing boundary segmentation for objects of varying sizes across the three primary 3D medical imaging modalities.
3.5 Generalizability analysis
To evaluate generalizability, we com- pare the proposed Mamba-HoME with several state-of-the-ar networks. Specifically, we investigate four con- figurations on the AMOS dataset: (1) training solely on CT, (2) pre-training all models on CT and fine-tuning on MRI. (3) training solely on MRI, and (4) joint training on both CT and MRI. Table 2 shows that Mamba- HoME demonstrates superior generalizability across modalities compared other models. Trained from scratch and fur- ther pre-trained on large-scale CT and MRI datasets, Mamba- HoME exhibits strong cross-modal generalizability to 3D ultrasound data, modality with distinct challenges such as high noise and lower resolution. Leveraging robust, modality- feature representations, the pre-trained model adapts to 3D ultrasound via efficient fine-tuning, out- performing state-of-the-an methods in both DSC and boundary HD95 metrics, as shown in Table 4. Qualitative results in Figure 2 further illustrate its ability to handle ultrasound-ppecffec artifacts. This cross-modal transferability highlights the model's 's versatility across diverse imaging modalities. Moreover, Mamba-HoMi demonstrates strong generalizability to external datasets within the same modality, especially MSD Pancreas and in-house CT dataset for PDAC and pancreas segmenta- tion, outperforming several state-of-the-art methods in both DSC and HD95 metrics (see Table 11).
36 rost slos
Ci Ground tutt
IS Vanba HOME
30 Vamba HIVEE
DC Segtarita
Figure 3: Qualitative segmentation results from top bot- tom: CT. MRI, and 3D US. From left to right, each column shows the input slice, ground truth, our proposed pre-trained Mamba-HoME Mamba-HoME trained from scratch, and the baseline SegMamba.
Detailed results for the generalizability analysis can be found in Appendix F.
4 Conclusions
In this work, we introduce the Hierarchical Soft Mixture-of-Experts (HoME), a two-level token- routing MoF layer designed to efficiently capture local-to-global pattern hierarchies. We integrate HoME with Mamba in the Haamba-HoME architecture, cnabling efficient Long-sequence processing. Comprehensive experiments show that Mamba-HoME outperforms several state-of-the-an methods and generalizes well across the three primary 3D medical imaging modalities.
Limitations. Scalability to Jarge-scale medical datasets (e.g., > 10,000 scans) remains unexplored, limiting our understanding of Mamba-HoMMi HoME's generalization across diverse image distributions. Although the model is pre-trained on large multimodal dataset using supervised learning, its behavior under large-scale self-super learning (e.g.. 2000,000 scans) has not yet been studied. We identify this as promising direction for future work to enhance Mamba-HoME's ability to capture complex patterns in unlabeled medical images. Key challenges include variations image resolution, noise, contrast, field-of view, acquisition techniques, and popatil-ttemoraa dependencies.
10

---

Acknowledgments
We acknowledge the use of the HPC cluster Helmholtz Munich for the computational resources used in this study. Ewa Szczurek acknowledges the support from the Polish National Science Centre SONATA BIS grant no. 2020/38///772/000055
References
[1] Tugba Akinci D' Antonoli, Lucas K Berger, Ashraya Indrakanti, Nathan Vishwanathan, Jakob Weiss. Matthias Jung, Zeynep Berkarda, Alexander Rau, Marco Reisert, Thomas Küstner, et al. Totalsegmentator mri: Robust reguence-independent segmentation of multiple anatomic structures in mri. Radiology. 314(2):c246133, 2025.
[2] Nils Alves. Merlijn Schuurmans. Dominik Rutkowski, Derya Yakar. Ingfrid Haldorsen, Marjolein Lieden- baum, Anders Molven, Phillip Vendittelli, Geert Litjens, Jurgen Hermans, et al. The panorama study protocol: Pancreatic cancer diaglosissradiilogiitts meet ai. Zenodo, 2024.
[3] Michela Antonelli, Annika Reinke, Spyridon Bakas, Keyvan Farahani, Annette Kopp-Schneider, Bennett A Landman, Geert Litjens, Bjoem Menze, Olaf Ronneherger, Ronald M Summers, al. The medical segmentation decathlon. Nature Communications, 13(1):1228, 2022.
Szymon Antoniak, Michal Krutul, Maciej Pióro, Jakub Krajewski, Jan Ludziejewski, Kamil Ciebiera, Krystian Król, Tomasz Odrzygóčdł, Marek Cygan, and Sebastian Jaszezur. Mixture tokens: Continuous moe through cross cooss-c aggregation. Advances in Neural Information Processing Systems, 37:103873- 103896, 2024.
[5] Anurag Arna,, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lučié, and Cordelia Schmid. Vivit: vidco vision transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6836 6846, 2021.
[6] Reza Azad, Ehsan Khodapanah Aghdam, Amelie Rauland, Yiwei Jia, Atlas Haddadi Avval, Afshin Bozorgpour, Sanaz Karimjjafarbiglo, Joseph Paul Cohen, Ehsan Adeli, and Dorit Merhof. Medical image segmentation review: The success of u-net. IEFE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[7] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.0m 06450. 2016.
[8] Hangbo Bao, Wenhui Wang, Li Dong, Qiang Liu, Owais Khan Mohammed, Kriti Aggarwal, Subhojit Som, Songhao Piao, and Furu Wei. Vlmo: Unified vision-languaa pre-tra pre-training with mixture- of of-modality Advances in Neural Information Processing Systems, 35:32897-3299 2022.
Maximilian Beek, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp, Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. xlstm: Extended long short-term memory. Advances in Neural Information Processing Systems, 37:1 37:107547-107703, 2024.
[10] Patrick Carnahan, John Moore, Daniel Bainbridge, Mehdi Eskandari, Elvis CS Chen, and Terry M Peters. Deepmitral: Fully automatic 3d echocardiography segmentation for patient specific mitral valve modelling. In International Conference on Medical Image Computing and Computer-Assiste Intervention, pages 459 468. Springer, 2021.
[11] Ao Chang, Jiajun Zeng, Ruobing Huang, and Dong Ni. Em-net: Efficient channel and frequency learning with mamba for 3d medical image segmentation. In International Conference on Medical Image Computing and Computer Assisted Intervention, pages 266 275. Springer, 2024.
[12] Jieneng Chen, Yingda Xia, Jiawen Yao, Ke Yan, Jianpeng Zhang, Le Lu, Fakai Wang, Bo Zhou, Mingyan Qiu, Qihang Yu, et al. Cancerunit: Towards single unified model for effective detection, segmentation, and diagnosis eight major cancers using a large collection of et scans. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 21327-21333 2023.
[13] Qian Chen, Lei Zhu, Hangzhou He, Xinliang Zhang, Shuang Zeng. Qiushi Ren, Yanye Lu. Low-rank mixture-of-expeer for continual medical image segmentation. In International Conference on Medical Image Computing and Compuder-ssiitted Intervention, pages 382-392. Springer, 2024.
[14] Tianlong Chen, Xuxi Chen, Xianzhi Du Abdullah Rashwan, Fan Yang. Huizhong Chen, Zhangyang Wang, and Yeqing Li. Adamv-moe: Adaptive multi-task vision mixture-of expertss... In Proceedings of the IEEE/CVF International Conference Computer Vision, pages
2023.
11

---

[15] Tri Dao and Albert Gu. Transformers are ssms: generalized models and efficient algorithms through structured state space duality. In Proceedings the International Conference Machine Learning, pages 10041-10071, 2024.
[16] Alexey Dosovitskiy, Lucas Beyer. Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold. Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In The Ninth International Conference Learning Representations, 2021.
[17] Zhiwen Fan, Rishov Sarkar, Ziyu Jiang, Tianlong Chen, Kai Zou, Yu Cheng, Cong Hao, Zhangyang Wang, et al. vit Mixture-of- cetture-ff-exeerts vision transformer for efficient multi-task learning with -ocel-accol co-design. Advances in Neural Information Processing Systems, 01224411224455 2022.
[18] Yunhe Gao. Training like a medical resident: Context-pric pontext-prior learning toward universal medical image segmentation. In Proceedings of the IREFJCVF Conference on Computer Vision and Puttern Recognition, pages 11 1494111244, 2024.
[19] Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00 00752, 2023.
[20] Joy Hsu, Jeffrey Gu, Gong Wu, Wah Chiu, and Serena Yeung. Capturing implicit hierarchical structure 3d biomedical images with supe-super hyperbolic representations. Advances in Neural Information Processing Systems, 34:5112-5122 12-5123, 2021.
[21] Kai Hu, Jinhao Li, Yuan Zhang, Xiongjun Ye, and Xieping Gao. One-to-multipe: A progressive style transfer unsupervised domain- adaptive framework for kidney tumor segmentation. Advances in Neural Information Processing Systems, 37:24496-24522, 2024.
[22] Xingru Huang, Jian Huang, Tianyun Zhang, HE HONG, Shaowei Jiang, Yaoqi Sun, et al. Upping the game: How 2d u-net skip connections flip 3d segmentation. Advances in Neural Information Processing Systems, 37:87282-877355 -83999 2025.
[23] Yuanfeng Ji, Haotian Bai, Chongjian Ge, Jie Yang, Ye Zhu, Ruimao Zhang, Zhen Li, Lingyan Zhanng, Wanling Ma, Xiang Wan, et al. Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation. Advances in Neural Information Processing Systems, 35:36722-36732, 2022.
[24] Mingjie Jiang and Bernard Chiu. A dual-stream centerline-guided network for segmentation of the common and internal carotid arteries from 3d ultrasound images. IEEE Transactions on Medical Imaging, 42(9):2690-2705, 2023.
[25] Yufeng Jiang and Yiqing Shen. M40e: A foundation model for medical multimodal image segmentation with mixture of experts. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 621-631. Springer, 2024.
[26] Jiachen Li, Xinyao Wang, Sijie Zhu, Chia-Wen Kuo, Lu Xu, Fan Chen, Jitesh Jain, Humphrey Shi, and Longyin Wen. Cumo: Scaling multimodal IIM with upeycled mixture-of-experts. Advances in Neural Information Processing Systems, 37:131224-131244 2024.
[27] Wenxuan Li, Chongyu Qu, Xiaoxi Chen, Pedro RAS Bassi, Yijia Shi, Yuxiang Lai, Qian Yu, Huimin Xue, Yixiong Chen, Xiaorui Lin, et al. Abdomenatlas: A large-scale, detailed-annotated, & multi-center dataset for efficient transfer learning and open algorithmic benchmarking. Medical Image Analysis, 97:103285, 2024.
[28] Wenxuan Li, Alan Yuille, and Zongwei Zhou. How well do supervised models transfer to 3d image segmentation? In The Twelfth International Conference on Learning Representations, 2024.
[29] Jiarun Liu, Hao Yang, Hong- Yu Zhou, Lequan Yu, Yong Liang, Yizhou Yu, Shaoting Zhang, Hairong Zheng, and Shanshan Wang, Swin uoin-umambat:: Adapting mamba-ban vision foundation models for medical image segmentation. IEEE Transactions on Medical Imaging, 2024.
[30] Tiange Liu, Qingze Bai. Drew A Torigian, Yubing Tong, and Jayaram K Udupa. Vsmtrans: A hybrid paradigm integrating self-attention and convolution for medical image segmentation. Medical Image Analysis, 98:103295, 2024.
[31] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages
2021.
12

---

[32] Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. In The Fifth International Conference on Learning Representations, 2017.
[33] Jiacheng Lu, Hui Ding, Shiyu Zhang, and Guoping Huo. M-net: Mri brain tumor sequential segmentation network via mesh-cast. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 201 16-12-2, 2025.
[34] Basil Mustafa, Carlos Riquelme, Joan Puigcerver, Rodolphe Jenatton, and Neil Houlsby. Multimodal contrastive learning with limoe: the language-image mixture of experts. Advances in Neural Information Processing Systems, 35:9564 9576, 2022.
[35] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chaman, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances Neural Information Processing Systems, 32, 2019.
[36] Kelly Payette, Hongwei Bran Li, Priscille de Dumast, Roxane Licandro, Hui Ji, Md Mahfuzur Rahman Siddiquee, Daguang Xu, Andriy Myronenko, Hao Liu, Yuchen Pei, et al. Fetal brain tissue annotation and segmentation challenge results. Medical Image Analysis, 18:102,33, 2023.
[37] Kelly Payette, Céline Steger, Roxane Licandro, Priscille De Dumast, Hongwei Bran Li, Matthew Barkovich, Liu Li, Maik Dannecker, Chen Chen, Cheng Ouyang, al. Multi-center fetal brain tissuc annotation (feta) challenge 2022 results. IEEE Transactions on Medical Imaging, 2024.
[38] Szymon Plotka, Maciej Chrabaszez, and Przemyslaw Biecek. Swin smt: Global sequential modeling for enhancing 3d medical image segmentation. In International Conference on Medical Image Computing and Computer-Assise Assisste Intervention, pages 689-698. Springer, 2024.
[39] Joan Puigcerver, Carlos Riquelme Ruiz, Basil Mustafa, and Neil Houlsby. From sparse soft mixtures of experts. In The Twelfth International Conference on Learning Representations, 2024.
[40] Chongyu Qu, Tiezheng Zhang. Hualin Qiao, Yucheng Tang, Alan L Yuille, Zongwei Zhou, et al. Abdomenatlas-8K Annotating 8,000 volumes for multi-organ segmentation three weeks. Advances Neural Information Processing Systems, 36:36620-366 -66200,36666.--- 2023.
[41] Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, André Su- sano Pinto, Daniel Keysers, and Neil Houlsby. Scaling vision with sparse mixture of experts. Advances in Neural Information Processing Systems, -89558888555,,,, 2021.
[42] Olaf Ronneherger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assiste Intervention, pages 234-241. Springer, 2015.
[43] Holger R Roth, Le Lu, Amal Farag, Mro-Chang Shin, Jiamin Liu, Evrim B Turkbey, and Ronald M Summers. Deeporgan: Multi-level deep convolutional networks for automated pancreas segmentation. In International Conference on Medical Image Computing and Computer- -sssiste Intervention, pages 556-564. Springer, 2015.
[44] Tomasz Szczepański, Michal K Grzeszezyk, Szymon Plotka, Arleta Adamowicz, Piotr Fudalej. Przemyslaw Korzeniowski, Tomasz Trzciński, and Arkadiusz Sitck. Let me decode you: Decoder conditioning with tabular data. In International Conference on Medical Image Computing and Computer-Assister Intervention, pages 228 238. Springer, 2024.
[45] Tomasz Szczepański, Szymon Plotka, Michal K Grzeszczyk, Arleta Adamowicz, Piotr Fudalej, Przemyslaw Korzeniowski, Tomasz Trzciáski, and Arkadiusz Sitek. Gepar3d: Geometry prior-assisted learning for 3d tooth segmentation. International Conference on Medical Image Computing and Computer-Assistee Intervention, pages 218 228. Springer, 2025.
[46] Fenghe Tang, Bingkun Nian, Yingtai Li, Zihang Jiang, Jie Yang, Wei Liu, and S Kevin Zhou. Mambamim: Pre-training mamba with state space token interpolation and application to medical image segmentation. Medical Image Analysis, 103:103606, 2025.
[47] Yucheng Tang, Dong Yang. Wengi Li, Holger Roth, Bennett Landman, Daguang Xu, Vishwesh Nath, and Ali Hatamizadeh. Self-supervised pre-training of swin transformers for medical image analysis. In Proceedings of the IEFF//VF Conference on Computer Vision and Pattern Recognition, pages 20730- 20740, 2022.
[48] Arvind M Vepa, Zukang Yang. Andrew Choi, Jungseock Joo, Fabien Scalzo, and Yizhou Sun. Integrating deep metric learning with coreset for active learning in 3d segmentation. Advances in Neural Information Processing Systems, 37:7 1777766337771 71671, 2024.
13

---

[49] Guoan Wang, Jin Ye. Junlong Cheng. Tianbin Li. Zhaolin Chen, Jianfei Cai, Junjun He. and Bohan Zhuang. Sam-med3d-moe: Towards non-forgetting segment anything model via mixture of experts for 3d medical image segmentation. In International Conference on Medical Image Computing and Computer-Assise Intervention, pages 352-561. Springer, 2024.
[50] Hualiang Wang. Yiqun Lin, Xinpeng Ding, and Xiaomeng Li. Tri-plane mamba: Efficiently adapting segment anything model for 3d medical images. In International Conference on Medical Image Computing and Computer-Assiste Intervention, pages 636-646. Springer, 2024.
[51] Jakob Wasserthal, ChrrssChrsttinn Breit, Manfred T Meyer, Maurice Pradella, Daniel Hinck, Alexander W Sauter. Tobias Heye, Daniel T Boll, Joshy Cyriac, Shan Yang. et al. Totalsegmentator, robust segmentation of 104 anatomic structures in images. Radiology: Artificial Intelligence, 5(5):2300024 2023.
[52] Boqian Wu, Qiao Xiao, Shiwei Liu, Lu Yin, Mykola Pechenizkiy, Decebal Constantin Mocanu, Maurice Keulen, and Elena Mocanu. E2enet: Dynamic sparse feature fusion for accurate and efficient 3d medical image segmentation. Advances in Neural Information Processing Systems, 37:1 17:11 18483-118512, 2025.
[53] Linshan Wu, Jiaxin Zhuang, and Hao Chen. Voco: A simplc-yet-effeetvve volume contrastive learning framework for 3d medical image analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22873-22882, 2024.
[54] Zhaohu Xing, Tian Ye, Yijun Yang. Guang Liu, Lei Zhu. Segmamba: Long-range sequential modeling mamba for 3d medical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 578-588. Springer, 2024.
[55] Zeyue Xue, Guanglu Song, Qiushan Guo, Boxiao Liu, Zhuofan Zong, Yu Liu, and Ping Luo. Raphacl: Text-to-image generation via large mixture of diffusion paths. Advances in Neural Information Processing Systems, 36:41693-44706 2023.
[56] Yanqi Zhou, Tao Lci, Hanxiao Liu, Nan Du, Yanping Huang, Vincent Zhao, Andrew M Dai, Quoc V Lc, James Laudon, al. Mixture-of-exxerts with expert choice routing. Advances in Neural Information Processing Systems, 35:7103-7114, 2022.
[57] Zongwei Zhou, Md Mahfuzur Rahman Siddiquee, Nima Tajbakhsh, and Jianming Liang, Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE Transactions on Medical Imaging. 3966):185611867 2019.
[58] Jiachen Zhu, Xinlei Chen, Kaiming He, Yann LeCun, and Zhwang Liu. Transformers without normalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
[59] Xingkui Zhu, Yiran Guan, Dingkang Liang, Yuchao Chen, Yuliang Liu, and Xiang Bai. Moe jetpack: From dense checkpoints to adaptive mixture experts for vision tasks. Advances in Neural Information Processing Systems, 37: 37:12094-1211 2024.
[60] Veronika A Zimmer, Alberto Gomez, Emily Skelton, Robert Wright, Gavin Wheeler, Shujie Deng, Nooshin Ghavami, Karen Lloyd, Jacqueline Matthew, Bernhard Kainz, et al. Placenta segmentation in ultrasound imaging: Addressing sources of uncertainty and limited field-of-viee Medical Image Analysis, 83:102639, 2023.
14

---

Appendix
Table of Contents
A Datasets
16
A.1 Pre-training
16
A.2 Training, fine-tuning, and test
16
B Experimental setup
B.1 Pre-training
................................
18
B.2 Training and fine-tuning
18
B.3 Evaluation metries
19
C Additional experimental results
21
Ablation studies
22
Effect of the number of experts
22
D.2 Effect of the group size
23
D.3 Effect of the number of slots
24
D.4 Impact of the HoMB layer
25
E Qualitative results
26
Generalizability analysis
G Impact statement
32
15

---

A Datasets
core objective of this work is to evaluate the robustness and generalizability of Mamba-HoME across a wide range of clinical scenarios. To this end, we conduct a comprehensive evaluation using datasets from three primary 3D medical imaging modalities: CT, MRI, and US. This multi-modal cohort provides diverse spectrum of imaging characteristics, ranging from high-resolution structural details low-resolution volumes with varying levels of noise and artifacts. Table provides detailed overview of the datasets utilized for pre-training, supervised training. fine-tuning and testing.
Table An overview of the datasets used for pre-training, training, fine-tuning, and testing. These datasets, spanning three modalities (CT, MRI, and US), cover diverse anatomical structures and lesions. Please note that all datasets configured in training mode were utilized for fine-tuning.
No.
Dataset
Modality
Body part
Mode
Label type
Pre-training
Train
Test
1.
AldomenAAlas 1.1 [40, 27, 28]
CT
Ahdomen
Pre-training
Voxel-wise
8,788 (-474)*
TotalSegmentator MRI [1]
MRI
Whole-body
Pre-training
Voxel-wise
616
PANORAMA [2]
Abdomen
Training
Voxel-wise
1,964
334
4. AMOS [23]
CT-MRI
Abdomen
Training
Voxel-wise
240
120
FeTA 2022 [37]
MRI
Fetal brain
Training
Voxel-wise
120
MVSeg [10]
Heart
Training
Voxel-wise
7.
In-house CT
CT
Abdomen
Test
Voxel-wise
60
Total
9,404
2,434
514
A.1
Pre-training
For pre-training, utilize two manually voxel-wise annotated, publicly available large-scale datasets across two modalities: AbdomenAtlas 1.1 [27, 40, 28), comprising multi-phase CT scans, and TotalSegmentator MRI [1], consisting of diverse MRI volumes. The mapping of classes across both modalities used for model pre-training is detailed in Table 10.
AbdomenAtlas. This large-scale dataset comprises 9,262 CT scans featuring voxel-wise annotations for 25 anatomical structures. The collection encompasses diverse range acquisition phases, including non-contrast, arterial, portal-venous, and delayed phases. Each CT volume contains between 24 and 2,572 slices, with in-plane resolutions ranging from 188 x 79 to 971 x 651 pixels. The voxel spatial resolution ranges from (00.38 1.5] [0.38 3.0] x [0.3 8.0) mm3 with mean resolution of 0.84 x 0.84 x 2.4 mm3 To ensure the integrity of our evaluation and provide fair comparison, we excluded cases that overlap with training and test sets. Specifically, removed 194 cases from MSD Pancreas, 43 from NIH Pancreas, and 200 training cases from the curated pre- vre-training cohort.
AMOS-CT dataset, resulting
TotalSegmentator MRI. This dataset consists of 616 MRI volumes featuring 50 anatomical structures manually annotated at the voxel level. For the pre-training phase, we specifically select subset of 22 classes that align with the labels provided the AbdomenAtlas dataset. Each MRI scan contains between 5 and 1,915 slices, with a highly variable voxel spatial resolution ranging from ([0.17 20.0] x [0.17 ~ 25.0] x [0.17 ~ 28.0)) mm3 and a mean resolution of 1.28 x 1.86 x 2.81 mm3
A.2 Training, fine-tuning, and test
To validate the efficiency of the Mamba-HoMe we conduct comprehensive experiments on both publicly available and in-house datasets across three primary modalities. For CT, we use PANORAMA [2]. AMOS-COS CT [23]. and private dataset for segmentation and diagnosis of pancreatic ductal adenocareinoma (PDAC) in abdominal CT. For MRI, we use AMOS-MRI [23], and FeTA 2022, which includes fetal brain MRI [36, 37J. For US we use MVSeg [10].
PANORAMA. This dataset consists of 2,238 multi-center contrast-enhanced computed tomography (CECT) scans acquired in the portal-venous phase. It includes 1,964 newly acquired scans from five European centers (located in the Netherlands, Norway, and Sweden), as well as publicly available data from two medical centers in the United States: the NIH [43] and MSD Pancreas [3]. The cohort
*Peasse note that, for a fair comparison, we excluded 200 cases from AMOS, 194 from MSD Pancreas, and 80 from NIH, respectively. These scans correspond to the original AMOS dataset and partly to the PANORAMA dataset.
16

---

Table 10: Class mapping for anatomical structures in CT scans (AbdomenAtlas) and MRI scans (TotalSegmentator MRI).
Index
Class
Index
Class
Background
13
Celiac trunk
Aorta
14
Colon
Gall bladder
15
Duodenum
3
Kidney (left)
16 Esophagus
Kidney (right)
17
Femur (left)
5
Liver
18
Femur (right)
Pancreas
19
Hepatic vessel
7
Postcava
20
Intestine
Spleen
21
Lung (left)
Stomach
22
Lung (right)
10
Adrenal gland (left)
23
Portal vein & splenic vein
11
Adrenal gland (right)
24
Prostate
12 Bladder
25
Rectum
comprises 676 PDAC (pancreatic ductal adenocarcinoma) and 1,562 non-PDAC cases. While the original dataset contains six voxel-wise labels, including PDAC lesion, veins, arteries, pancreatic parenchyma, pancreatic duct, and common bile duet, we specifically utilize PDAC lesion label and merge the pancreatic parenchyma and pancreatic duct classes into single pancreas label. Each CT volume contains between 37 and 1,572 slices, with an in-plane resolution ranging from 512 x 512 to 1024 x 1024 pixels. The voxel spatial resolution ranges from ([0.31 ~ 1.03] [0.31 1.03] X [0.45 5.0)) mm3, with a mean resolution 0.75 x 0.75 X 2.0 mm3
AMOS. This dataset comprises a total of 600 multi-modal scans (500 CT and 100 MRI). Ground truth annotations are provided for the training and validation cohorts, which include 240 and 120 scans, respectively. Each scan features 15 anatomical structures, manually annotated at the voxel level. For our experiments on AMOS-CT, we utilize 200 scans for training and 100 for testing. For AMOS. MRI, we employ 40 scans for training and 20 for testing. The CT volumes consist of 68 ~ 353 slices, with a voxel spatial resolution ranging from ([0.45 1.07] x [0.45 1.07] x [1.25 5.0]) mm3 and mean of ().00 x 0.70 x 4.20 mm3 The MRI volumes contain 60 ~ 168 slices, with voxel
resolution ranging from ([0.70 1.95] x |0.70 1.95) [1.09 3.0] mm3 and mean of 1.10 X 1.10 x 2.46 mm3
FeTA 2022. This dataset consists of 120 fetal MRI scans for brain tissue segmentation, collected from two prominent medical institutions: the University Children's 's Hospital Zurich and the Medical University of Vienna. The cohort includes 80 cases from Zurich and 40 from Vienna. It features seven manually annotated tissue classes provided at the voxel level: external cerebrospinal fluid, gray matter, white matter, ventricles, cerebellum, deep gray matter, and the brainstem. Each fetal MRI volume contains 256 slices, with an isotropic voxel spatial resolution ranging from (J0.43 ~ 1.0] x [0.43 1.0) x [0.43 1.0]) mm3 and mean resolution of 0.67 0.67 0.67 mm3
MVSeg. This dataset consists of 175 transesophageal echocardiography (TEE) 3D ultrasound scans dedicated mitral valve segmentation. Voxel-wise ground truth annotations are provided for both the posterior and anterior leaflets. The data were acquired at King's College Hospital, London, UK. Each 3D ultrasound volume contains 208 slices, with a voxel spatial resolution ranging from (00.00 ~ 0.63] x [0.31 ~ 0.90] x [0.13 ~ 0.39]) mm3 and mean resolution of 0.38 0.56 x 0.23 mm3
In-house CT. This dataset consists of 60 CECT seans of hisopaathloog--confimee PDAC cases, acquired from 40 different medical centers. Each volume contains 68 ~ 875 slices with in-plane resolution of 512 x 512 pixels. The voxel spatial resolution ranges from ([0.47 ~ 0.98) x [0.47 ~ 0.98] x [0.5 5.0]) mm3 These scans include voxel-wise annotations for both the pancreas and the PDAC lesion. The diameter of the PDAC lesions ranges from 1.25 ~ 6.06 cm, with a mean of 3.45 + 1.20 em. To ensure high-quality ground truth, the data initially annotated by two junior radiologists using 3D Slicer and subsequently reviewed and refined by two domain experts, each with over 30 years of clinical experience.
17

---

B Experimental setup
B.1 Pre-training
AbdomenAtlas Allas and TotalSegmentator MRL. For both the AbdomenAtlas 1.1 and TotalSegmentator MRI datasets, all volumes are resampled to anisotropic resolution of 0.8 x 0.8 x 3.0 mm3 Prior to training, the data are partitioned into training and validation sets using a stratified 85:15 split to maintain balanced distribution of CT and MRI modalities. Intensity preprocessing for CT scans involves clipping Hounsfield Units (HU) to the range of [-175, 250]. while MRI intensities are clipped to [0, 1000]; both modalities are subsequently min-max scaled the interval [0, 1]. Mamba- HoME is trained with patch size of 96 x 96 x 96 for 800 epochs. To enhance model robustness, we employ on-the-fly stochastic augmentations, including random scaling, rotation, flipping, brightness and contrast adjustment, Gaussian smoothing, and additive Gaussian noise. The pre-training phase was completed approximately 7 days using 8x NVIDIA H100 80GB GPUs.
B.2 Training and fine-tuning
PANORAMA. All CT volumes are resampled to an anisotropic resolution of 0.8 x 0.8 x 3.0 mm3. Voxel intensities are clipped to Hounsfield Unit (HU) range of [-175, 250] and subsequently min- max scaled to the interval [0, 1]. We employ stratified 80:20 split, resulting in 1,571 training and 393 validation scans, thereby preserving the class distribution of PDAC versus non-PDAC cases. The model is fine-tuned for 500 epochs using a consistent training patch size of 192 x 192 x 48 voxels. To maximize robustness, we utilize on-the-fly stochastic augmentations, including random scaling, rotation, flipping, brightness/contrast adjustments, and the application of Gaussian smoothing and noise. During the inference stage, a Buddng-windooy strategy employed with a corresponding crop size of 192 x 192 x 48 and an overlap ratio of 0.5, incorporating a Gaussian importance weighting filter to ensure seamless patch aggregation and minimize boundary artifacts.
AMOS. For both modalities, all volumes are resampled to an isotropic resolution of 1.5 x 1.5 x 1.5 mm3 Intensity preprocessing follows the protocol described in the pre- pre-training phase: CT scans are clipped to a Hounsfield Unit (HU) range of [- [1775, 250), while MRI scans are clipped to [0, 1000]. Both modalities are subsequently min-max scaled to the interval [0. 1]. The original training set is partitioned into training and validation subsets using a randomized 80:20 split. Across the training, fine-tuning, and inference stages, we maintain consistent spatial context by utilizing a patch size of 128x 128x x 128. For both validation and final testing, Hiiding-windowg inference strategy is employed with default overlap ratio of 0.5, incorporating a Gaussian weighting filter to ensure smooth patch aggregation. Data robustness is further enhanced via on-the-fly fly augmentations, including random scaling, rotation, flipping, brightness/contrast adjustments, and the application of Gaussian smoothing and additive noise.
FeTA 2022. All MRI volumes are resampled to an isotropic resolution of 0.8 x 0.8 0.8 mm3 Intensities are clipped to the range of [0, 1000) and subsequently min-max scaled to the interval [o, 1]. We evaluate model performance using 5-fold cross volss-aaiiotionn strategy, with each fold trained for 300 epochs. Throughout the training and inference phases, we utilize consistent patch size of 128 x 128 128 voxels. To maximize generalizability the presence of fetal motion and anatomical variability, we employ comprehensive on-the-fly stochastic augmentations, including random scaling, rotation, flipping, brightness/contrast adjustments, Gaussian smoothing, additive noise, and affine transformations.
MVSeg. All 3D ultrasound volumes are resampled to an isotropic resolution of 0.5 x 0.5 x 0.5 mm3
Intensities are clipped to the range of [o, 255] and subsequently min-max scaled to the interval [0. 1]. We partition the dataset into independent training. validation, and testing sets consisting of 105, 30. and 40 scans, respectively. The model is trained for 500 epochs using consistent patch size of 128 x 128 x 128 for both training and inference. To address the inherent speckle noise and artifacts typical of ultrasound imaging, we employ extensive on-the-fly fly stochastic augmentations, including random scaling, rotation, flipping, brightness/contrast adjustments, Gaussian smoothing, additive noise, and affine transformations.
18

---

B.3
Evaluation metrics
Dice Similarity Coefficient (DSC). The isC commonly used metric to evaluate the segmentation performance of a model in multi-class settings, especially in medical imaging. It measures the overlap between predicted and ground truth segmentations, providing an aggregate assessment across multiple classes. Given a segmentation task with C' classes, let p: E RC and gi E RC be the one-hot encoded predicted and ground truth vectors at voxel respectively. The DSC for class is computed as:
2 DiPliii DSC = ExPic E.PictE.gie
(18)
where Pi.e and Ii.e represent the predicted and ground truth binary masks for class at voxel i, respectively. The mean DSC (mDSC) across all C' classes is computed as follows:
mDSC C Σ DSC.
(19)
This metric ensures that each class contributes equally to the final score, regardless of class imbalance.
95th percentile Hausdorff Distance (HD95). The 95th percentile Hausdorff Distance (HD95) is robust evaluation metric used to quantify the spatial similarity between predicted and ground truth segmentations. Unlike the standard Hausdorff Distance, which is highly sensitive to outliers by measuring the maximum distance, the HD95 considers the 95th percentile of distances to provide a more stable measure of boundary agreement.
For given class c, let S.. and S represent the sets of surface boundary points for the predicted and ground truth masks, respectively. These surface points extracted by identifying voxels that change during binary erosion.
Directed Distance Sets: We first compute the set minimum Euclidean distances from every point in one set to the nearest point in the other:
D(S S()
min lp -oll p S yES
(20)
D(S s.. min lgg pllp PE
I 1g S
(21)
2. 95th Percentile Calculation: The bi-directional HD95 for class c defined as the maximum of the 95th percentiles of these two distance distributions:
H H max (percentilegs (D(Se" S()), percentileys (D(S S
(22)
3. Mean HD95 (mHD95): To evaluate the overall performance across all C' classes, the mean HD95 is calculated as the average of class-wiss means:
C
N HD95
= 2 N Σ
mHD95
(23)
where No is the number of cases where class e was present.
Sensitivity and Specificity. The sensitivity and specificity performance of a model in detecting the presence of a condition at the patient level. In this setting, a segmentation model processes medical scans and outputs a binary classification for each patient: either positive (presence of the condition) or negative (absence of the condition).
crucial metrics for evaluating the
Given dataset of patients, let TP, FP, FN, and TN denote the number of true positives, false positives, false negatives, and true negatives, respectively. The sensitivity (also known as recall or true positive rate) is defined as:
TP
Sensitivity
(24)
TP+FN'
19

---

where sensitivity measures the proportion of correctly identified positive patients out of all actual positive patients. The specificity (true negative rate) is given by:
TN
Specificity
(25)
TN + FP
where specificity quantifies the proportion of correctly identified negative patients out all actual negative patients.
These metrics provide comprehensive assessment of the model's ability to detect the condition while avoiding false alarms, which is critical for clinical decision-making.
Number of parameters. The total number of trainable parameters in neural network can be computed by summing the parameters across all layers and levels, as follows:
L-I Parameters ||O|lo + ΣΣΣΣ
A 10
(26)
where θ represents the of parameters from the backbone, the total number of levels layers in the network, Cj and Cj represent the number of input and output channels at each level and k, respectively, A is binary matrix indicating the presence of a weight connection between input channel at level and output channel j at level k; [|0 llo represents the count of non-zero weights in the connection between input channel and output channel j.
This formulation takes into account the layer-wise parameters while incorporating the structured sparsity of weights within the network.
Inference speed. The inference speed measures the temporal latency required for the model to process volumetric input and generate segmented output. Given the use of sliding window approach for high-resolution medical volumes, the average inference speed 7 is calculated across N samples following GPU warm-up phase to ensure hardware state stabilization. This metric is defined as:
T
(lend,n (start,)) x 10
(27)
where Extant and fend.n. represent the timestamps in seconds recorded immediately before the input tensor moved to the computation device and immediately after the sliding window aggregator completes the reconstruction of the prediction volume, respectively. The constant 103 scales the result to milliseconds (ms).
20

---

C Additional experimental results
Table 11: Quantitative segmentation and detection results on the PANORAMA test sets. Performance evaluated across the NIH (healthy controls only), MSD Pancreas, and an in-house dataset. We report mDSC (%) and mHD95 (mm) for the pancreas and pancreatic ductal adenocarcinoma (PDAC), alongside patient-level detection performance via Sensitivity (%) and Specificity (%). The best and second-best results are bolded and underlined. respectively. Methods marked with (*) are initialized with pre-trained weights. Note that SuPreM utilized a subset of the test data (MSD Pancreas and NIH) during its pre-training phase, which may impact the fairness of the comparison.
DDSC (%)
mHD99 (mam)
PDAC Detection (%)
Method
NIH Pancreas
MSD Paucreas
In-house Pancreus
Overall
Overall Pancreas
Overall
PDAC
Pancreas
PDAC
PDAC
Pancreas
PDAC
Sensitivity
Specificity
VoCo-B (*)
38.7
43.3
91.6
Hermes
92.0
40.0
86.5
58.3
80.8
48.2
87.8
21.4
5.1
87.3
94.0
Swin SMT
87.2
58.6
79.8
87.0
19.8
5.8
88.6
92.8
VSmTrans
92.0
47.1
87.3
56.1
80.5
50.3
87.2
18.2
5.2
89.9
90.4
SegMamba
92.7
47.9
88.8
54.0
81.9
49.7
88.5
19.5
4.3
84.8
95.2
oC 3DU--Ner
92.6
88.9
56.8
80.2
88.2
16.9
4.5
80.7
89.2
Swin UNETR
91.9
38.8
87 4
58.7
81.3
46.3
87.4
24.1
6.2
87.3
90.4
SuPreM
92.3
46.0
88.0 6
61
82.0
51.7
88.3
9.0
2.4
87.3
89.2
Mamba-HoME (*)
92.6
53.6
88.5
61.7
82.3
56.7
88.5
6.2
1.9
92.8
95.2
Table 12: Quantitative segmentation results using 5-fold cross-validation the FeTA 2022 dataset. Performance is reported as DSC (%) for each tissue class and summarized via mDSC (%) and mHD99 (mm) for the aggregate average across all folds. Tissue classes include: external cerebrospinal fluid (eCSF), gray matter (GM), white matter (WM), ventricles (V), cerebellum (C), deep gray matter (dGM), and brainstem (B). The best and second- bestt results bolded and underlined. respectively. Methods designated with (*) are initialized with pre-trained weights.
DSC (%)
Method
mDSC f
mHDD95 (mm)
eCSF
GM
WM
C
dGM
VoCo-B (*)
78.9
73.4
90.2
87.5
87.0
87.6
83.8
86.0
4.0
Swin
86.6
87.9
Hermes
79.3
74.2
87.6
87.1
88.8
84.7
86.5
4.0
SegMamba
79.6
73.7
90.6
87.1
87.5
88.8
85.2
86.5
5.8
uC 3DU-Net
79.1
73.2
90.3
87.4
86.0
88.0
83.4
85.9
3.5
SuPreM (*)
78.0
71.7
89.8
86.8
85.9
87.1
83.5
85.3
3.6
VSmTTrans
79.0
73.1
90.6
87.2
87.2
88.3
86.1
2.3
Mamba-HoME
80.4
75.8
88.8
89.4
2.1
Namba-HoME. (*)
80.6
76.1
91.9
88.8
90.0
88.8
86.1
87.7
2.0
Table 13: Quantitative segmentation results on the MVSeg test set. Performance is reported as
DSC (%) for the posterior leaflet (PL. and anterior leaflet (AL.), with mDSSC (%) and mHD195 (mm) representing the aggregate average across both classes. The best and second-best results are bolded and underlined, respectively. Methods marked with (") initialized with pre-trained weights.
DSC (%)
Method
mDSC
mHD99 (mm)
PL
AL
(%)
Swin SMT
82.2
84.5
4.9
Hermes
82.9
84.1
5.3
uC 3DU-Net
82.9
85.0
83.9
4.7
SuPreM (*)
83.2
85.4
84.3
5.0
VoCo-B (*)
83.3
85.3
84.3
5.1
Swin UNETR
83.8
85.1
84.4
4.8
SegMamba
83.0
84.7
83,8
5.8
VSmTTrans
83.4
85.4
84.4
6.2
Mamba-HoME
84.0
85.7
84.8
4.3
Mamba-HoME, (*)
84.2
86.0
85.0
4.1
21

---

D Ablation studies
We present detailed quantitative results from our ablation studies, analyzing three key factors: (1) the number of experts per stage; (2) the group size; and (3) the number of slots assigned to each expert.
D.1 Effect of the number of experts
For fair comparison, keep the group size constant (K E (2048, 1024, 512, 256}) and set the number of slots to 5 =
Table 14:
Quantitative ablation study on the influence of the number of experts across the PANORAMA test sets. Performance reported as DSC (%) for each dataset and summarized via mDSC (%) and mHD95 (mm) for the aggregate average across all cohorts (NIH, MSD, and in-house). The experts denotes the number experts across the four encoder stages. The best and second-best results are bolded and underlined, respectively. Configurations marked with (*) indicate a doubled number of experts in the second hierarchical level.
DSC (%) 1
mDSC (%)
mHD95
Experts (E)
NIH
MSD
In-house
Overall
(mm)
Pancreas
PDAC
Panereas
PDAC
Pancreas
PDAC
Pancreas
[4, 8, 12, 16]*
92.9
51.2
88.4
60.8
81.7
54.8
88.3
77.5
4,8
18, 16, 24, 48]
92.6
49.4
88.2
59.6
80.6
53.3
87.9
76.3
5.4
[8. 16, 24, 48)"
92.1
47.9
88.0
57.4
80.5
52.2
87.5
75.8
5,9
[16, 32, 48, 64]
92.7
51.0
88.5
60.4
81.9
54.5
88.4
77.5
4.9
Table 15: Quantitative ablation study on the influence of hierarchical expert configurations on the AMOS-CT validation set. Performance reported as DSC (%) for each of the 15 abdominal organs, with mDSC (%) and mHD95 (mm) representing the aggregate average. Organs include: Spleen (Sp), Right Kidney (RK), Left Kidney (LK), Gallbladder (GB), Esophagus (Es), Liver (Li), Stomach (St), Aorta (Ao), Inferior Vena Cava (IVC), Pancreas (Pa), Right Adrenal Gland (RAG), Left Adrenal Gland (LAG), Duodenum (Du), Bladder (BI), and Prostate/Uterus (Pr/Ut). The experts E denotes the number of experts across the four encoder stages. The best and second-best results are bolded and underlined, respectively. Configurations marked with (*) indicate doubled number of experts in the
second hierarchical level.
DSC (%) f
Experts (E)
mDSC mHD95
Sp
RK
LK
IVC
Pa
RAG
LAG
Pr/Ut
(%)
(mm)
[4, 8, 12, 16]"
96.0
95.0
94.4
81.7
82.0
94.7
90.2
93.8
88.9
84.0
74.4
74.1
78.4
83.8
83.2
86.3
18.7
16, 48]
95.7
93.7
89.3
93.2
83.4
75.0
83.3
86.2
19.6
[8. 16, 24, 48]
94.7
95.4
95.0
81.6
81.4
95.4
87.0
93.4
87.8
83.4
73.8
74.4
77.0
86.0
82.2
85.9
20.2
[16. 32,
48, 64]
95.8
93.8
94.8
80.0
83.0
95.7
87.5
93.6
88.4
83.8
75.0
74.5
77.5
85.7
82.4
86.1
19.7
Table 16: Quantitative ablation study on the effect of the number of experts in the HoME layer across 5-fold cross- validation the FcTA 2022 dataset. Performance is reported as DSC (%) for each fetal brain tissue class and summarized via mDSC (%) and mHD99 (mm) for the aggregate average across all folds. Tissue classes include: external cerebrospinal fluid (eCSF), gray matter (GM), white matter (WM), ventricles (V), cerebellum (C), deep gray matter (dGM), and brainstem (B). The experts E denotes the number of experts across the four encoder stages. The best and second-best results bolded and underlined, respectively. Configurations marked with (*) indicate a doubled number of experts in the second hierarchical level.
DSC (%) f
Experts
mDSC (%)
mHD95
eCSF
GM
WM
V
C
dGM
B
(mm)
[4, 12, 16]
80.4
75.8
91.4
88.4
88.8
89.4
86.2
87.5
2.1
[8. 16, 24, 24,48)
80.4
91.1
88.3
88.5
88.6
86.6
87.2
2.4
[8. 4
80.7
75.5
91.1
88.4
88.5
86,9
2.3
[16, 32,
80.5
75.4
91.5
88.5
88.0
88.9
86.5
87.4
2.3
22

---

D.2 Effect of the group size
For a fair comparison, we keep the number of experts constant (E1 E {4,8, 12, 16) and Eg {8, 16, 24, 323) and set the number of slots to S =
Table 17: Quantitative ablation study on the effect of the group size (K) across the PANORAMA test sets. Performance is reported DSC (%) for individual datasets and summarized via mDSC (%) and mHD95 (mm) for the aggregate average across all cohorts. The group size K denotes the number of tokens per routing group across the four encoder stages. The best and second-bess results are bolded and underlined. respectively. Configurations marked with (*) indicate a doubled number of experts in the second hierarchical level.
DSC (%)
mDSC (%)
mHD95
Group size (K)
NIH
MSD
In-house
Overall
(mm) !
Pancreas
PDAC
Pancreas
PDAC
Pancreas
PDAC
Pancreas
[1024, 512, 256, 128]
92.5
50.4
88.2
60.5
81.5
54.2
88.1
77.2
5.9
[2048, 1024, 512, 256]
92.9
51.2
88.4
60.8
81.7
54.8
88.3
77.5
4.8
[2048. 1024, 512, 256]"
92.7
81.7
52.6
88.2
76.8
[4096, 2048, 1024, 512]
92.5
51.9
88.1
60.6
81.5
55.2
88.0
77.4
5.0
Table 18: Quantitative ablation study the effect of the group size (K) on the AMOS-CT validation set. Performance reported as DSC (%) for each of the 15 abdominal organs, with mDSC (%) and mHD95 (mm) representing the aggregate average. Organs include: Spleen (Sp), Right Kidney (RK), Left Kidney (LK), Gallbladder (GB), Esophagus (Es), Liver (Li), Stomach (St), Aorta (Ao), Inferior Vena Cava (IVC), Pancreas (Pa), Right Adrenal Gland (RAG), Left Adrenal Gland (LAG), Duodenum (Du), Bladder (BI), and Prostate/Uterus (Pr/Ut). The group size K denotes the number of tokens per routing group across the four encoder stages. The best and second-best results bolded and underlined, respectively. Configurations marked with (*) indicate a doubled number of experts in the second hierarchical level.
DSC (%)
Group size (K)
MDSC mHD95
Sp
(%)
(mm)
[1024, 256, 128]
86.4
87.6
86.1
19.2
[2048, 1024, 512, 256]
96.0
95.0
94.4
81.7
82.0
94.7
90.2
93.8
88.9
84.0
74.4
74.1
78.4
83.8
83.2
86.3
18.7
[2048. 1024. 512, 256]"
95.5
95.2
95.6
82.1
89
86.2
98.2
87.5
75.2
75.1
77.3
82.1
86.2
18.9
[4096. 2048, 1024. 512)
95.2
95.1
95.5
812
82.0
89.0
86.5
93.5
87.8
82.8
73.3
75.0
76.4
87.8
82.0
86.1
19.1
Table 19: Quantitative ablation study on the effect of the group size (K) across 5-fold cross-validation on the FeTA 2022 dataset. Performance reported as DSC (%) for each fetal brain tissue class and summarized via mDSC (%) and mHD95 (mm) for the aggregate average across all folds. Tissue classes include; external cerebrospinal fluid (eCSF), gray matter (GM), white matter (WM), ventricles (V), cerebellum (C), deep gray matter (dGM), and brainstem (B). The group size K denotes the number of tokens per routing group across the four encoder stages. The best and second-best results are bolded and underlined, respectively. Configurations marked with (*) indicate a doubled number of experts in the second hierarchical level.
DSC (%)
Group size (K)
mDSC (%)
mHD99 (mm)
eCSF
GM
WM
C
dGM
[1024. 512, 256, 128]
80.4
75.7
90.3
87.2
88.3
89.4
87.0
87.4
2.3
[2048, 1024, 512, 256]
91.4
87.5
[2048, 1024, 512. 256]"
80.4
75.3
91.0
88.0
88.5
89.3
86.3
87.3
2.3
[4096. 2048, 1024, 512]
80.5
75.1
91.1
88.1
88.2
89.6
86.8
87.4
2.2
23

---

D.3 Effect of the number of slots
For a fair comparison, we keep the number of experts constant (E1 E {4, 12, 16) and Eg {8, 16, 24, 32}) and the number of groups fixed (K E {2048, 1024, 512, 256}).
Table 20: Quantitative ablation study on the effect of the number of slots per expert (S) across the PANORAMA test sets. Performance is reported DSC (%) for individual datasets and summarized via mDSC (%) and mHD95 (mm) for the aggregate average across all cohorts. The parameter S represents the capacity of each expert process tokens across the four encoder stages. The best and second- best results are bolded and underlined, respectively.
mDSC (%)
Slots (S)
mDSC (%) 1
mHD99 (mm)
NIH
MSD
In-house
Overall
Pancreas
PDAC
Pancreas
PDAC
Pancreas
PDAC
Pancreas
93.0
46.6
88.6
55.9
82.2
50.1
88.5
76.2
5.4
92.5
49.7
88.0
61.5
82.0
54.2
88.0
77.2
5.0
92.9
51.2
88.4
60.8
81.7
54.8
88.3
77.5
4.8
92.8
46.4
88.3
60.6
81.3
51.8
88.1
76.5
5.3
Table 21: Quantitative ablation study on the effect of the number of slots per expert (S) on the AMOS- CT validation set. Performance reported as DSC (%) for each the 15 abdominal organs, with mDSC (%) and mHD95 (mm) representing the aggregate average. Organs include: Spleen (Sp). Right Kidney (RK), Left Kidney (LK), Gallbladder (GB). Esophagus (Es), Liver (Li), Stomach (St), Aorta (Ao), Inferior Vena Cava (IVC), Panereas (Pa), Right Adrenal Gland (RAG), Left Adrenal Gland (LAG), Duodenum (Du). Bladder (BI), and Prostate/Uterus (Pr/Ut). The parameter S represents the capacity allocated to each expert for token processing across the four encoder stages. The best and second-best results are bolded and underlined, respectively.
DSC (%) f
Slots (S)
mDSC mHD95
Sp
RK
LK
GB
Es
Li
St
Ao
IVC
Pa
RAG
LAG
Du
BI
Pr/Ut
(%)
(mm) J
95.0 93.0 95.1 80.6
81.4
95.7
87.4
93.5
88.8
84.4
75.0
74.8
77.8
83.8
82.4
85.9
19.6
95.5 95.3
95.1
80.3
82.3
91.3
87.7
93.7
88.9
84.0
74.6
74.5
78.6
86.2
81.3
86.0
19.4
96.0 95.0
94.4
81.7
82.0
94.7
90.2
93.8
88.9
84.0
74.4
74.1
78.4
83.8
83.2
86.3
18.7
95.3 95.8
95.3
79.5
82.3
96.1
87.6
93.3
88.7
82.2
74.9
75.4
76.2
87.6
81.6
86.1
19.1
Table 22: Quantitative ablation study on the effect of the number of slots per expert (S) across 5-fold cross-validation on the FeTA 2022 dataset. Performance reported as DSC (%) for each fetal brain tissue class and summarized via mDSC (%) and mHD95 (mm) for the aggregate average across all folds. Tissue classes include: external cerebrospinal fluid (eCSF), gray matter (GM), white matter (WM), ventricles (V), cerebellum (C), deep gray matter (dGM), and brainstem (B). The parameter S defines the token processing capacity of each expert across the four encoder stages. The best and second-best best results are bolded and underlined. respectively.
DSC (%)
Slots (S)
mDSC (%)
mHD95
eCSF
GM
WM
dGM
B
(mm)
80.6
75.1
91.0
87.9
88. .4
89.4
86.4
87.3
2.2
2
80.7
75.3
91.1
87.9
88.5
89.6
86.4
87.4
2.4
4
80.4
75.8
91.4
88.4
88.8
89.4
86.2
87.5
2.1
8
80.5
75.3
91.1
88.1
88.5
89.5
88.6
87.4
2.2
24

---

D.4 Impact of the HoME layer
(a) Input slice
(b) Ground truth
(c) Mamba-HoME
(d) SegMamba
Figure 4: Qualitative comparison Mamba-HoME and SegMamba on abdominal CT seans from the PANORAMA test set. These images demonstrate the efficacy of integrating the HoME layer into the baseline SegMamba architecture. The green (PDAC) and red (pancreas) annotations highlight key segmentation differences, showing that Mamba- Hoob is more robust in identifying anatomical structures across varying scales from small tumors to larger organ boundaries. Notably, the Mamba-HoMB results shown achieved through training from scratch.
25

---

E
Qualitative results
(a) Input sicee
(b) Ground truth
(c) Mamba-HoME
(d) SuProM
(6) uc 3DU-Net
(1) SegMamba
Figure 5: Qualitative segmentation results for PDAC (green) and the pancreas (red) provided by Kamba-HoME and the next three oop-prffrmiin methods. The first three rows display cases from the MSD Pancreas dataset, while the last two rows show cases from the in-house dataset. Please note, we show Mamba- results trained from scratch.
Oround trahh
ICI Maribu HAVE
08 SUPYON
=G SOU
Slgghammee
IgI VBnnnaaa
06 Hermes
Figure Qualitative segmentation results on the AMOS-CT validation set for Mamba-HoMel (trained from scratch). SuPreM, uC 3DU-Net. SegMamba, VSmTrans, and Hermes. All models are trained on both the AMOS-CA and AMOS-MR training datasets.
26

---

Figure 7: Qualitative comparison of segmentation performance on the AMOS-MRI validation set for six methods: Mamba-HoME (trained from scratch), SuPreM, uC 3DU-Net, SegMamba, VSmTrans, and Hermes. The models are trained on both training dataset of AMOS-CT and AMOS-MRI.
Yout
Ground
SO Marnhes
HII SuPrIM
3DU-Net
SegMMamber
IgI VSSniTranss
Figure 8: Qualitative segmentation results the AMOS- MRI validation set for Mamba- MoMba SuPreM, uC 3DU-Net, SegMamba, VSmTrans, and Hermes. Models are first pre-trained on the AMOS-CT scans and subsequently fine-tuned on the AMOS-MRI training data.
Grnnnd Insh
Jc Ment
DC
Segitebb
Ig)
(N Hertes
Figure 9: Qualitative segmentation results on the AMOS-CT validation set for Mamba-HoME. SuPreM, uC 3DU-Net. Segbamba, VSmTrans, and Hermes. Each model is trained only on the AMOS-CT training scans.
27

---

(N iqpu
Gioudd suth
ICI Mambue--okk
06 WD
10 SegiVerrba
[gl
P0 Hermes
Figure 10: Qualitative segmentation results the AMOS-MRI validation set for Mamba- Houba SuPreM, uC 3DU-Net, SegMamba, VSmTrans, and Hermes. These models are trained solely on the AMOS-MRI training
ISI (6
SegMoribs
10
90
Figure 11: Qualitative segmentation results on the MV test set for Mamba-HoME. SuPreM, uC 3DU-Net. SegMamba, VSmTrans, and Hermes.
28

---

Generalizability analysis
Figure 12 provides an overview of the datasets utilized in the Mamba-HoMEl framework, encompass- ing CT, MRI, fetal MRI, and 3D ultrasound modalities. The figure presents voxel-wise ground truth labels across modall and cross- anssommatt domains. The first and third columns, as well as the second and fourth columns, display independent imput slices and their corresponding ground truth seg- mentations for two representative cases. The framework, initially developed using CT and MRI scans, demonstrates consistent segmentation of abdominal organs such the liver, spleen, and kidneys. The inclusion of fetal MRI and 3D ultrasound for fine-tuning further highlights the model's capacity to generalize across modalities with distinct anatomical features and imaging characteristics. This cross-modal and cross-anatomics representation emphasizes the versatility of the network in capturing variability across both imaging techniques and anatomical structures.
CT
MAI
fetal MRI
3D US
Input slice
Ground truth
Input slice
Ground truth
Figure 12: This figure illustrates the diversity of modalities used across the training pipeline. The model is pre-trained on CT scans (Row 1) and MRI scans (Row 2), then fine-tuned on downstream tasks involving fetal MRI (Row 3) and 3D ultrasound (Row 4). For two separate cases, Column and Column present the input slices, while Column 2 and Column 4 display the corresponding ground truth. This arrangement highlights the significant domain shift in feature representation, moving from the high-resolution anatomical structures of CT and MRI to the distinct imaging characteristics and noise profiles found in fetal MRI and 3D ultrasound.
We perform comprehensive guantitative evaluation of the Mamba-HoME's generalizability across four distinet experimental protocols. First, we establish intra-modal baselines by training and evaluating the model independently the AMOS-CT (see Table 23) and AMOS-MRI (see Table 24) datasets. Second, we investigate cross-modal transfer learning by pre-training the architecture on AMOS-CT followed by fine-tuning AMOS-MRI, assessing the efficacy of knowledge transfer across disparate imaging physics (see Table 25). Finally, we conduct joint multi-modal training, where the network is trained simultaneously both AMOS-CT and AMOS-MRI and evaluated across their respective validation sets (see Table 26) to determine the model's 's capacity for learning domain invariinn feature representations.
29

---

Table 23: Quantitative segmentation results the AMOS-CT validation set. All models were trained exclusively on CT data. Performance is reported using DSC (%) for each of the 15 abdominal organs and mDSC (%) and mHD99 (mm) for the final average across all structures. Organs include: Spleen (Sp). Right Kidney (RK). Left Kidney (LK), Gallbladder (GB), Esophagus (Es), Liver (Li), Stomach (St), Aorta (Ao), Inferior Vena Cava (IVC), Panereas (Pa). Right Adrenal Gland (RAG), Left Adrenal Gland (LAG), Duodenum (Du), Bladder (BI), and Prostate/Uterus (Pr/Utt. The best results are bolded, while the second-best underlined. Methods marked with (*) initialized with pre-trained weights.
Method
DSC (%)
mDSC miIDe5 (%)
Sp
RK
I.K
GB
Es
Li
St
IVC
Pa
RAG
LAG
Du
BI
Pr/Ut
(mm)
Hermes
95.7 94.5
95.3
80.9
93.9
88.0
87.8
82.5
73.7
71.8
77.7
83.3
79.3
85.3
11.3
Swin SMT
95.4
95.6
95.1
76.3
81.3
92.2
87.4
93.7
88.6
84.1
74.4
73.4
78.3
86.6
82.6
85.7
47.2
VSm
95.6 94.7 94.5
82.7
76.4
95.2
86.9
92.9
83.1
84.0
72.9
74.7
77.8
86.8
81.6
85.3
39.1
SegMamha
95.7 95.5 95.2
81.7
81.1
94.9
87.7
93.2
88.8
84.0
74.7
74.0
77.3
83.9
82.6
86.0
26.3
uC 3DU-Net
93.0 94.6
73.6
93.4
82.5
85.5
75.4
73.3
71.1
69.8
83.0
79.1
82.7
19.1
Swin UNETR
95.2 95.1 94.5
72.2
80.4
90.2
81.0
93.2
87.9
81.6
74.4
74.5
78.1
85.0
80.8
84.3
43.6
SuPreM (*)
95.6 95.0 94.4
86.2
79.4
97.1
91.2
93.0
88.7
85.3
69.3
68.8
81.3
86.0
78.9
86.0
16.5
Mamba-HoMI
96.0 95.0
94.4
81.7
82.0
94.7
90.2
88,9
84.0
74.4
74.1
78.4
83.8
83.2
86.3
18.7
Mamba-HooME (*)
96.0 95.4
95.4
85.3
82.1
96.8
91.3
93.3
88.6
84.7
74.0
74.6
80.8
88.0
82.8
87.3
12.2
Table 24: Quantitative segmentation results the AMOS-MRI validation All models were trained exclusively on MRI data. Performance is reported using DSC (%) for each of the 15 abdominal organs, with mDSC (%) and mHD95 (mm) representing the final average across all available structures. Organs include: Spleen (Sp), Right Kidney (RK), Left Kidney (LK), Gallbladder (GB), Esophagus (Es), Liver (Li), Stomach (St), Aorta (Ao), Inferior Vena Cava (IVC), Pancreas (Pa), Right Adrenal Gland (RAG), Left Adrenal Gland (LAG), Duodenum (Du), Bladder (BI), and Prostate/Uterus (Pr/Ut). The best results are bolded, while the second-best are underlined. Methods marked with (*) are initialized with pre-trained weights, while NA indicates organs not present or labeled in this specific validation subset.
DSC (%)
Method
mDSC (%)
mHD99
Sp
RK
LK
GB
Es
St
Ao
IVC
Pa
RAG
LAG
Du
BI
Pr/Ut
(mm)
Hermes
95.5 95.4
95.3
68.0
72.3
96.7
83.4
90.2
87.2
79.2
61.9
60.8
63.1
NA
NA
80.7
13.0
Swin SMT
92.8 94.2 93.7
56.5
0.0
95.8
81.1
88.6
83.2
79.4
0.0
0.0
56.3
NA
NA
63.2
22.5
VSmTrans
94.8 94.3 94.9
65.3
69.7
96.6
83.8
90.6
80.3
52.4
61.5
74.6
SegMamba
95.6 95.1
64.3
70.2
87.1
91.
86.6
56.3
59.8
12.6
uC 3DU-Net
91.6 92.6 93.4
71.0
67.1
95.0
83.2
89.0
83.4
79.4
0.0
0.0
59.8
NA
NA
68.6
16.9
Swin UNETR
93.5 94.5 94.7
70.3
68
96.3
84.4
90.2
86.0
80.5
54.5
0.0
61.5
NA
NA
75.0
14.7
SuPreM (*)
93.6 94.6 94.0
71.2
67.3
95.9
83.7
89.3
83.8
79.8
0.0
60.6
70.3
18.2
Mamba-HoME Mo
95.2 94.6
67.6
90.0
Mamba-Ho ME. (*)
ME.
96.1 95.5 95.7
75.3
72.3
97.5
89.1
90.4
86.5
85.3
58.2
62.1
65.9
NA
NA
82.3
11.0
30

---

Table 25: Quantitative segmentation results on the AMOS- validation set via cross-modal transfer (CT - MRI). Models were first pre-trained on the AMOS-CT dataset and subsequently fine-tuned on the AMOS-MRI training set prior evaluation. Performance reported using DSC (%) for each of the 15 abdominal organs, with mDSC (%) and mHD995 (mm) representing the aggregate average across available structures. Organs include: Spleen (Sp), Right Kidney (RK), Left Kidney (LK), Gallbladder (GB), Esophagus (Es), Liver (Li), Stomach (St), Aorta (Ao), Inferior Vena Cava (IVC), Pancreas (Pa), Right Adrenal Gland (RAG), Left Adrenal Gland (LAG), Duodenum (Du), Bladder (BI), and Prostate/Uterus (Pr/Ut). The best results are bolded, while the second-best are underlined. Methods marked with (*) are initialized with pre-trained weights, while NA indicates organs not present or labeled in this specific validation subset.
DSC (%)
Method
mDSC mHD95
Sp
RK
Es
Ao
RAG
LAG
Du
Pr/Ut
(%) f (mm) 1
Hermes
96.6
95.8
96.0
77.7
78.3
97.7
90.4
92.5
89.9
86.0
65.2
63.5
72.0
NA NA
84.8
8.2
SMT
96.1 95.4
75.7
75.2
88.2
91.7
82.3
62.9
65.2
83.2
VSmTraas
96.2 95.8
79.9
74.1
87.5
90.5
91.2
89.5
85,4
64.2
66.8
70.8 NA
84.5
14.1
SegMamba
94.7 95.0 95.0
67,4
69.9
96.3
84.3
90.5
84.7
80.8
57.0
50.5
60.6
NA
NA
79.0
8.2
uC 3DU-Net
96.1 95.8 95.8
80.1
76.8
97.3
88.5
91.8
89.1
84,3
63.8
68.4
71.0 NA
NA
84.5
9.1
Swin UNETR
96.1 95.6 95.8
78.9
72.4
97.2
89
91.0
88.9
85.1
65.9
69.3
84.2
SuPreM (*)
96.1 95.3 95.8
76.7
73.9
88.9
91.8
88.8
85.2
63.5
66.5
71.0 NA
NA
83,9
Mlama--HMMI
96.4 95.5 95.9
77.3
76.2
97.5
90.9
914
89.4
86,4
64.5
66.7
72.2 NA
NA
84.8
8.1
Mamba- ME
96.5 95.8
96.1
79.5
77.0
97.8
91.2
22.4
90.4
85.4
63.9
66.5
72.0
NA
NA
85.0
8.0
Table 26: Quantitative segmentation results the joint AMOS-CT and AMOS-MRI validation sets. Models were trained concurrently on both CT and MRI training sets and evaluated across both modalities. Performance is reported using DSC (%) for each of the 15 abdominal organs, while mDSC (%) and mHD95 (mm) represent the aggregate average across all structures. Organs include: Spleen (Sp). Right Kidney (RK). Left Kidney (LK), Gallbladder (GB), Esophagus (Es), Liver (Li), Stomach (St), Aorta (Ao). Inferior Vena Cava (IVC), Pancreas (Pa), Right Adrenal Gland (RAG). Left Adrenal Gland (LAG), Duodenum (Du), Bladder (BI), and Prostate/Uterus (Pr/Ut). The best and second-best results are bolded and underlined, respectively. The best results are bolded, while the second- best are underlined. Methods marked with (") are initialized with pre-trained weights.
Method
DSC (%)
mDSC mHD95 (%)
Sp
RK
LK
GB
Es
Li
St
Ao
IVC
Pa
RAG
LAG
Du
BI
Pr/Ut
(mm)
Hermes
94.5
93.7
93.3
77.8
77.6
93.8
87.2
92.5
87.7
82.0
71.7
70.0
72.2
83.3
79.3
82.9
7.3
Swin SMT
91.1
94.0
94.2
75.1
76.7
94.0
79.2
90.8
85.6
80.3
68.1
70.3
68.8
70.3
76.8
81.2
17.7
VSmTrans
85.0
87.1
89.1
71.7
73.9
86.
74.5
87.5
79.2
74.5
64.3
67.2
67.3
82.6
80.8
78.0
SegMamba
94.9
94.0
93.5
76.7
79.7
94.5
89
93.1
88.3
83.1
68.5
72.5
76.0
83.4
82.3
84.7
7.5
uC 3DU-Net
95.0 94.5 93.8
76.3
78.8
95.3
86.9
92.7
87.6
81.9
71.7
70.7
74.0
83.2
76.7
84.1
12.2
Swin UNETR
92.1 94.8 94.6
78.1
76.8
95.1
86.5
91.8
84.7
82.8
69.1
71.0
74.6
83.1
81.5
83.8
12.4
SuPreM (*)
92.8
94,1
78.5
78.1
93.2
84.4
84,5
82.3
67.8
71.9
74.2
81.2
HoMb
95.2
95.1 94.6
79.3
79.9
95.4
89
92.8
88.2
82.7
71.9
72.7
75.5
82.7
80.5
85.1
7.4
Mamba-HoME, (*)
95.0
95.3
95.6
82.7
80.0
96.3
90.4
93.0
87.8
84.5
72.1
72.9
77.7
89.0
83,2
86.4
7.2
31

---

G
Impact statement
This work introduces Hierarchical Soft Mixture-of-Expert. (HoME), novel architecture for efficient and accurate 3D medical image segmentation. By integrating two-level mixture-of-expeer routing mechanism with Mamba-based Selective State-Space Models, our method significantly advances long-context modeling for volumetric data. HoME is designed to address key challenges in medical imaging, namely, modeling local-to-global spatial hierarchies, handling modality diversity (CT, MRI, US), and achieving scalability for high-resolution 3D inputs. Our proposed Mamba-HoMi architecture demonstrates strong generalization and outperforms state-of-the- models across public and in- ho-house datasets, while being memory and compute efficient.
Beyond medical imaging, the architectural principles introduced, specifically the hierarchical token routing and the integration of local and global context processing, are applicable to other domains deal- ing with structured, hierarchical data under resource constraints. These include scientific computing. robotics, and spatiotemporal analysis in environmental or geospatial datasets.
Ethically, this work supports equitable healthcare by enabling accurate segmentation with reduced computational requirements, which is crucial for deployment in low-resource settings. We use publicly available datasets and provide open-source code ensure reproducibility and accessibility for the broader community. No personally identifiable information or sensitive patient data is used. Future extensions could include further robustness to distributional shifts medical data and broader clinical evaluation.
32

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In both abstract and introduction, we accurately reflect the paper's contributions and scope. Specifically, we introduce Hierarchical Soft fixture-of-Fxperre (HoME), two- soaltto-global pattern hierarchies.
level, token-routing MoE. layer for efficient capture Additionally, we design unified architectural block that integrates Mamba's SSMs with HoME. We embed the above novel solutions into multi-stage U-shaped architecture, called Mamba-HoME, designed for 3D medical image segmentation.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: conclusion, we provided Limitations section, where we list the main limitations of the paper. First, scalability to large-scale medical datasets and self self-supervised pre-training is a promising direction for future work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
33

---

3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: This paper focuses empirical results, and we do not provide any theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper details the experimental setups, dataset descriptions, hyperparam- eters, and humba-HoMis architecture. The code and four datasets used for training and evaluation are publicly available the supplementary material, and an in-house dataset used for testing is available upon request.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g., in the of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
34

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification:
The code and four datasets used for evaluation publicly available. An in no-hose dataset used for generalizability analysis is available upon request.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ //nips. ctaiss.cc/ubliti /ww.//ui.se///ee./// CodeSubmi ssionpolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper provides descriptions of training and testing, including data splits, hyperparameter choices, and the optimizer type.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We use the Wilcoxon signed-rank test to evaluate the statistical significance of performance differences between our proposed method, Mamba-HoMF, and other state-of- the-art approaches.
35

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the eπror bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
The paper details the computational resources used for training. Specifically, used Python 3, PyTorch 2.4, and MONAT 1.3.0. For hardware, employed an NVIDIA DGX system equipped with 8 x NVIDIA H100 80 GB GPUs. We used all eight GPUs for pre-training, while training, fine-tuning, and evaluation were performed on single NVIDIA H100 80 GB GPU.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: http:://eeurruuu.ieeeeeee httpppblii/tthhhcuuuuueeeeeeee
Answer: [Yes]
Justification: The research presented in this paper conforms in every respect the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
36

---

10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: This paper describes the efficiency and accuracy of 3D medical image segmen- tation, which can enhance medical diagnostic precision and patient outcomes.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification:
This paper does not present any associated risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: In this paper, we acknowledge the original authors of utilized assets, including code, datasets, and models, by citing them appropriately within the text.
37

---

Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The proposed Hierarchical Soft Mixture-of of-rrerr--sxpets (HoME) and integration into Mamba-HoME are fully described in the methodology section, including architecture, implementation, and datasets used for validation.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooe/mmodll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
38

---

Answer: [Yes]
Justification:
We have obtained IRB approval for our in-house data used as part of the test
set.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: We do not LLM for the core methodology, scientific rigorousness, or originality of the research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy (https: htt///uurs.cc///ooee// http://s..cccooeeeeneee/e/////// /LLM) for what should or should not be described.
39


Reviews:
[{"review_id": "XfS43SDUQw", "reviewer": "Reviewer_SpNP", "summary": "This author proposed a Mamba-HoME, a novel 3D medical image segmentation architecture combining Mamba state-space models with a Hierarchical Soft Mixture-of-Experts (HoME) layer. HoME uses two-level token routing to capture local-to-global dependencies efficiently. Evaluations across CT, MRI, and ultrasound datasets show state-of-the-art segmentation performance with favorable computational efficiency and generalizability.", "questions": "1. Some sentences in Introduction are very long—could you break them up for clarity?\n2. Please  standardize font sizes across all tables for consistency?\n3. How much improvement does HoME provide over plain Mamba when controlling for parameter count?\n4. What is the runtime and GPU memory footprint on a standard high-resolution volume?\n5. Did you test cross-institution generalization or domain shift robustness?\n6. Is there evidence that the second-level global routing meaningfully improves performance?\n7. Are the observed improvements statistically significant on all datasets?", "limitations": "Yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\nThe approach innovatively integrates SSMs with hierarchical expert routing, addressing the scalability issues of transformers for large 3D volumes. Extensive experiments across five datasets convincingly demonstrate consistent improvements in Dice score and boundary accuracy. The ablations systematically explore design choices (number of experts, group sizes, slots). The inclusion of pretraining and cross-modal transfer further strengthens the claim of generalizability.\n\nWeakness:\nPerformance gains over prior SSM methods (e.g., SegMamba) are modest (~1–2% Dice). The complexity of HoME increases parameter count and inference time (~30% slower). No user studies or clinical validation are provided. Some claims about “unprecedented memory and compute efficiency” may be overstated since inference speed lags behind simpler baselines. Details about dataset preprocessing and training schedules are scattered across appendix sections.", "was_revised": true, "final_justification": "Thank the authors for addressing my comments. Most of my concerns are well addressed. I still keep my original positive assessment.", "rebuttal": "> W1: Performance gains over prior SSM methods (e.g., SegMamba) are modest (1–2% Dice). The complexity of HoME increases parameter count and inference time (30% slower).\n\nWe are grateful for this comment. In Figures 2 and 6–12 (see Supplementary Material), we illustrate examples of clear qualitative improvements, particularly in the delineation of small or hard-to-detect structures such as pancreatic tumors. These improvements are not fully captured by modest gains in aggregate metrics like DSC.\nAdditionally, Table 1 and Table 11 (see Supplementary Material) demonstrate a significant improvement in the segmentation of small objects, such as PDAC tumors.\nIndeed, as the Reviewer points out, Mamba-HoME has 30 percentage points lower inference speed than the baseline SegMamba. Still, in practical 3D image analysis, segmentation is performed after image acquisition, and there is no need for real-time inference. Therefore, improving the accuracy over competitor models is far more important and justifies a decrease in speed.\nOn top of that, our new comparison demonstrated that inference is much faster for Mamba-HoME (2.01 s/scan) than for SegMamba scaled to a similar parameter count (2.53 s/scan). Please refer to our answer to question **Q1 in reply to Reviewer PbeP** for full details and the relevant table. At the same time, scaled SegMamba did not obtain a significant improvement in accuracy compared to baseline SegMamba. It also required more GPU memory and achieved lower mDSC and higher mHD95 than Mamba-HoME.\n\n> W2: No user studies or clinical validation are provided.\n\nWhile we agree that user studies and clinical validation are important for eventual deployment, they are beyond the scope of this work. Our current work focuses on establishing the methodological foundation for reliable medical image segmentation and evaluating the proposed model under quantitative benchmarks, such as generalizability to cross-institutional (with 40 institutions for the in-house and 2 additional institutions for the remaining CT scans), multi-modality (CT, MRI, US) datasets and robustness to domain shift. We consider clinical validation an important direction for future work and are actively exploring clinical collaborations to that end.\n\n> W3: Some claims about “unprecedented memory and compute efficiency” may be overstated since inference speed lags behind simpler baselines.\n\nTo avoid any overstatements, we will revise those sentences in the final version of the paper.\n\n> W4: Details about dataset preprocessing and training schedules are scattered across appendix sections.\n\nWe appreciate the Reviewer’s feedback and agree that clarity in presenting dataset preprocessing and training details is essential. To improve readability, we have revised the Supplementary Material to consolidate all preprocessing steps and training schedules into a single dedicated section. We also added cross-references from the main paper for easier navigation. This improved Supplementary Material will be incorporated into the final version of the paper. We hope this improves the accessibility and reproducibility of our work.\n \n> Q1: Some sentences in Introduction are very long—could you break them up for clarity?\n\nWe made sure to break up the very long sentences in the introduction section for clarity. For example, the sentence: \n“However, combining the global efficiency of SSMs with the localized adaptability of MoE remains largely unexplored, particularly in 3D medical imaging, where balancing efficiency and generalization across multi-modal datasets under resource constraints is paramount.”, \nwas edited into: \n“However, combining the global efficiency of SSMs with the localized adaptability of MoE remains largely unexplored. Such a combination would particularly benefit 3D medical imaging, where balancing efficiency and generalization across multi-modal datasets under resource constraints is paramount.”\nThese and other changes will be incorporated in the final version of the paper.\n\n> Q2: Please standardize font sizes across all tables for consistency?\n\nThe final version will use standardized font sizes across all tables to ensure consistency.\n\n> Q3: How much improvement does HoME provide over plain Mamba when controlling for parameter count?\n\nThank you for this excellent suggestion. To demonstrate the improvement over plain Mamba while controlling the parameter count, we compared to the plain Mamba-based segmentation method, SegMamba [1], with the number of parameters increased to approximately 176.5M, similar to that of Mamba-HoME. To this end, we increased:\n- the number of heads from [2, 2, 2, 2] to [4, 8, 16, 32],  \n- the feature size from [48, 96, 192, 384] to [64, 128, 256, 512], and  \n- the hidden size from 768 to 1024.\n\nThis choice of SegMamba scaling was based on [2].\n\nDespite increasing the number of parameters, we did not observe improvements in segmentation performance by SegMamba (SegMamba-L in the Table below) compared to the baseline SegMamba (SegMamba-B). Notably, Mamba-HoME achieved a consistently superior mDSC and mHD95 compared to both baseline and scaled SegMamba. \n\n| Method | Parameters (M) | PANORAMA | MVSeg | AMOS-CT |\n|:--------------:|:--------------:|:------------:|:-------------:|:-------------:|\n|  SegMamba-B | 66.8  | 76.0 (14.1)  |  83.8 (15.6)  |  86.0 (98.9)  |\n|  SegMamba-L |  176.5  | 76.3 (11.8)  |  84.0 (14.8)  |  85.4 (96.2)  |\n|  Mamba-HoME |  170.5 | 77.5 (6.9)   |  84.8 (12.6)  |  86.3 (45.0)  |\n\nValues are shown as mDSC (%) and mHD95 (mm) in brackets.\n\n> Q4: What is the runtime and GPU memory footprint on a standard high-resolution volume?\n\nTable 1 presents the average runtime and GPU memory footprint on CT volumes, including high-resolution data, such as PANORAMA and our in-house datasets. Refer to Supplementary Material A.2 for dataset details.\n\nTo further demonstrate performance on high-resolution cases, we performed additional experiments and selected the 100 cases with the highest resolution from the AbdomenAtlas 1.1 [3] dataset to report their specific runtime and GPU memory usage. Each CT scan consists of between 786 and 1,440 slices, with a mean of 1,156. The mean voxel spatial resolution is 0.82 × 0.82 × 0.79 mm³.\nFor those very high-resolution data, Mamba-HoME achieves a good balance between inference efficiency and memory usage, with an inference time of 17.7 s and a moderate GPU memory footprint of 15.1 GB, outperforming other large-scale models in memory efficiency while maintaining competitive inference speed. Please note that the inference time includes data pre-processing, such as resampling the data.\n\n| Method | Parameters (M) | Inference time (s) | GPU memory (GB) |\n|------------------|:------------------:|:----------------------:|:------------------:|\n| uC 3DU-Net        |       21.7         |         13.4           |        17.5        |\n| SuPreM            |       62.2         |         19.1           |        21.0        |\n| Swin SMT          |      170.8         |         15.2           |        19.7        |\n| Swin UNETR        |       72.8         |         19.2           |        21.1        |\n| Hermes            |       44.5         |         21.5           |        17.1        |\n| VSmTrans          |       47.6         |         19.9           |        14.9        |\n| SegMamba-B        |       66.8         |         14.5           |        14.0        |\n| SegMamba-L        |      176.5         |         21.5           |        17.1        |\n| Mamba-HoME        |      170.5         |         17.7           |        15.1        |\n\n> Q5: Did you test cross-institution generalization or domain shift robustness?\n\nWe did evaluate cross-institution generalization and robustness to domain shift, particularly for pancreas and PDAC segmentation. Specifically, we test Mamba-HoME on three external test sets: MSD Pancreas, NIH, and an in-house dataset. The in-house data includes samples from 40 independent medical centers (see Table 11 in the Supplementary Material). We also evaluated the performance of our model on various data domains, including CT, MRI and US. These benchmarks demonstrated excellent generalizability of Mamba-HoME to datasets across institutions and robustness to different data domains (see Tables 1-4). \n\n> Q6: Is there evidence that the second-level global routing meaningfully improves performance?\n\nFollowing this insightful suggestion, we performed an ablation study with three setups: (#1) Baseline - SegMamba, (#2) Mamba-HoME with first-level (local), and (#3) full Mamba-HoME with both first- and second-level routing. In this experiment, we used the FeTA 2022 dataset. This experiment evidenced that both first- and second-level routing indeed meaningfully improved the performance. This new result will be incorporated in the final version.\n\n| Setup (#) | mDSC (%) | mHD95 (mm) |\n|---------------------------|:------------:|:--------------:|\n| 1| 85.9 | 3.5 |\n| 2| 86.8 | 2.7 |\n| 3| 87.5 | 2.1 |\n\n> Q7: Are the observed improvements statistically significant on all datasets?\n\nWe used the Wilcoxon signed-rank test to evaluate the statistical significance of performance differences between our proposed method, Mamba-HoME, and other state-of-the-art approaches. We observed statistically significant improvements across all methods and datasets (Wilcoxon rank-sum test; p < 0.05). \nWe regret that this information was omitted from the main paper due to oversight and thank the reviewer for bringing this to our attention. This information will be added in the final version of the paper.\n\n**References**\n\n[1] Xing et al. \"Segmamba: Long-range sequential modeling mamba for 3d medical image segmentation.\" MICCAI, 2024\n\n[2] Hatamizadeh et al. \"Mambavision: A hybrid mamba-transformer vision backbone.\" CVPR, 2025\n\n[3] Li et al. \"Abdomenatlas: A large-scale, detailed-annotated, & multi-center dataset for efficient transfer learning and open algorithmic benchmarking.\" Medical Image Analysis, 2024"}, {"review_id": "jmvyEGMgMP", "reviewer": "Reviewer_PbeP", "summary": "The paper introduces the Hierarchical Soft Mixture-of-Experts (HoME), a novel two-level token-routing mechanism designed for efficient 3D medical image segmentation. It builds on the Mamba state-space model (SSM) to combine local and global context modeling effectively. The proposed method demonstrates robust cross-modality generalization and significant advancements in segmentation performance, particularly in handling hierarchical structures in 3D medical data. These contributions aim to address the challenges of scalability and generalization in 3D medical image analysis.", "questions": "1.\tInclude more detailed analysis of memory usage and training/inference times for large datasets.\n\n2.\tProvide additional visual or textual explanations of the HoME layer for readers less familiar with the technical details.\n\n3.\tDiscuss potential latency concerns in real-world applications of the model.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n\n1.\tThe methodology is robust and well-executed. The integration of the Hierarchical Soft Mixture-of-Experts (HoME) with the Mamba state-space model (SSM) addresses the need for efficient local-to-global feature modeling in 3D medical image segmentation.\n\n2.\tExtensive experimental results validate the proposed model's performance across diverse datasets and imaging modalities (CT, MRI, Ultrasound), demonstrating both accuracy improvements and computational efficiency.\n\n3.\tThe ablation studies provide a thorough exploration of the impact of architectural and hyperparameter choices, strengthening the validity of the conclusions.\n\nWeaknesses\n\n1.\tThe model has 170.5M parameters, which is larger than the lightweight model (e.g. uC 3DU-Net is only 21.7M), and is about 30% slower than the baseline SegMamba. The accuracy improvement comes at the expense of speed. While the paper demonstrates improvements in computational efficiency, it lacks detailed discussions of memory constraints and training times for very large-scale datasets, which might impact its scalability. The significance of the computational trade-offs (e.g., slightly slower inference speeds compared to simpler baselines) is not fully analyzed in the context of real-world clinical workflows, where latency may be a critical factor. \n\n2.\tWhile the integration of state-space models and hierarchical MoE is novel, the core ideas build upon well-established frameworks (e.g., SSMs and MoEs). So the novelty is limited.", "was_revised": true, "final_justification": "After reading the rebuttal and other reviews, I think the paper does have some merits but also have some limitations. So I choose to keep my originial rating.", "rebuttal": "> W1: The model has 170.5M parameters, which is larger than the lightweight model (e.g. uC 3DU-Net is only 21.7M), and is about 30% slower than the baseline SegMamba. The accuracy improvement comes at the expense of speed. While the paper demonstrates improvements in computational efficiency, it lacks detailed discussions of memory constraints and training times for very large-scale datasets, which might impact its scalability. The significance of the computational trade-offs (e.g., slightly slower inference speeds compared to simpler baselines) is not fully analyzed in the context of real-world clinical workflows, where latency may be a critical factor.\n\n\nWe are grateful for Reviewer’s comments and address each of them below. \nRegarding accuracy improvement at the expense of speed. In real-world clinical 3D analysis applications, real-time inference speed is generally not critical, as segmentation is typically performed after image acquisition and reconstruction. In such settings, even several seconds of computation are considered acceptable.\n\nWe also performed additional experiments and demonstrated that when SegMamba is scaled to the same number of parameters (SegMamba-L), it is slower and requires more GPU memory compared to our proposed method, Mamba-HoME. We ran the model on 1,020 CT scans from the AbdomenAtlas 1.1 [1] validation set to analyze mean inference time per scan and memory usage.\n\n| Method      | Inference time (s) | GPU memory (GB) |\n|-------------|:------------------:|:--------------:|\n| SegMamba-B  |       1.50         |      9.0       |\n| SegMamba-L  |       2.53         |      12.0      |\n| Mamba-HoME  |       2.01         |      10.2      |\n\n\nRegarding training times and memory constraints,  following Reviewer's suggestion we performed additional experiments on large scans, which are outlined in the answer to Q1 below. In general, training and inference times for large datasets scale linearly with the number of CT scans; therefore, the method should be scalable to larger datasets.\n\nFollowing the Reviewer’s insightful comment about performance on large datasets, we have now added an additional analysis in which we evaluate inference time and GPU memory usage for a set of high-resolution CT scans. For this experiment, we selected the 100 scans with the highest resolution from the AbdomenAtlas 1.1 dataset [1].\n\nEach CT scan consists of between 786 and 1,440 slices with a mean of 1,156. The mean voxel spatial resolution is 0.82 × 0.82 × 0.79 mm³.\n\nThe inference times obtained by Mamba-HoME even for these exceptionally large scans are perfectly suitable for implementation in clinical settings, where, as mentioned, segmentation is performed not in real-time but after image acquisition.\n\n| Method        | Parameters (M) | Inference time (s) | GPU memory (GB) |\n|------------------|:------------------:|:----------------------:|:------------------:|\n| uC 3DU-Net        |       21.7         |         13.4           |        17.5        |\n| SuPreM            |       62.2         |         19.1           |        21.0        |\n| Swin SMT          |      170.8         |         15.2           |        19.7        |\n| Swin UNETR        |       72.8         |         19.2           |        21.1        |\n| Hermes            |       44.5         |         21.5           |        17.1        |\n| VSmTrans          |       47.6         |         19.9           |        14.9        |\n| SegMamba-B        |       66.8         |         14.5           |        14.0        |\n| SegMamba-L        |      176.5         |         21.5           |        17.1        |\n| Mamba-HoME        |      170.5         |         17.7           |        15.1        |\n\n\n> W2: While the integration of state-space models and hierarchical MoE is novel, the core ideas build upon well-established frameworks (e.g., SSMs and MoEs). So the novelty is limited.\n\nTo the best of our knowledge, Mamba-HoME is the first model to provide an architectural framework that combines SSMs and hierarchical MoEs for the task of efficient 3D image segmentation.\n\n\n> Q1: Include more detailed analysis of memory usage and training/inference times for large datasets.\n\n\nTo address this comment, we have now trained the model on the large AbdomenAtlas 1.1 dataset, which consists of 9,262 CT scans in total. Of these, we used 8,242 for training and 1,020 for inference time and memory analysis. For training, we used 128 × 128 × 128 patches and a batch size = 2.\nAccording to the Table below, Mamba-HoME has a training time of around 0.47 seconds per scan, comparable to other large models, but even shorter than the lightweight uC 3DU-Net. \n\nThe average inference time for Mamba-HoME is approximately 2 seconds per scan, with a GPU memory requirement of 10.2 GB, which represents a good trade-off between speed and memory usage compared to other state-of-the-art methods. \n\nThis new Table will be incorporated in the final version of the paper.\n\n\n|      Method      | Parameters (M) | Training time (s) | Inference time (s) | GPU memory (GB) |\n|:----------------:|:--------------:|:-----------------:|:------------------:|:---------------:|\n|   uC 3DU-Net     |      21.7      |       0.59        |        1.57        |      12.4       |\n|     SuPreM       |      62.2      |       0.40        |        2.32        |      15.8       |\n|    Swin SMT      |     170.8      |       0.42        |        2.09        |      14.6       |\n|   Swin UNETR     |      72.8      |       0.40        |        2.32        |      15.8       |\n|     Hermes       |      44.5      |       0.44        |        2.27        |      16.2       |\n|    VSmTrans      |      47.6      |       0.60        |        2.48        |       9.8       |\n|  SegMamba-B      |      66.8      |       0.38        |        1.52        |       9.0       |\n|  SegMamba-L      |     176.5      |       0.49        |        2.53        |      12.0       |\n|  Mamba-HoME      |     170.5      |       0.47        |        2.01        |      10.2       |\n\n\n> Q2: Provide additional visual or textual explanations of the HoME layer for readers less familiar with the technical details.\n\nThe visualization of the HoME layer will be included in the final version of the paper. \n\n> Q3: Discuss potential latency concerns in real-world applications of the model.\n\nWe appreciate the Reviewer’s attention to practical deployment considerations. \nIn real-world clinical 3D imaging workflows, latency is typically not a limiting factor, as segmentation is performed after image acquisition and reconstruction, not in real time. This is particularly true in applications such as radiology, oncology planning, and surgical navigation, where segmentation is part of the offline post-processing pipeline. Even when segmentation is triggered during live image review by a radiologist or oncologist, an inference time of approximately 2 seconds is negligible compared to the overall image examination process, which typically takes at least several minutes.\n\nFurthermore, the hardware requirements for inference are modest and can be satisfied by a standard workstation or a modern laptop equipped with a dedicated GPU. That said, we acknowledge that ultra-low-latency inference may be necessary for certain time-sensitive applications, such as intraoperative guidance or robotic-assisted procedures. We will include these clarifications in the final version of the paper.\n\n\n**References**\n\n[1] Li et al. \"Abdomenatlas: A large-scale, detailed-annotated, & multi-center dataset for efficient transfer learning and open algorithmic benchmarking.\" Medical Image Analysis, 2024"}, {"review_id": "YodJlm1JyT", "reviewer": "Reviewer_k1E5", "summary": "This paper proposes a new framework, Mamba-HoME, for 3D medical image segmentation. It integrates Hierarchical Soft Mixture-of-Experts (HoME) into the Mamba model, which efficiently handles long-range dependencies and local-to-global feature extraction. HoME is designed with two levels of token-routing experts: one for local feature extraction and the other for global context aggregation. The authors demonstrate that Mamba-HoME outperforms existing state-of-the-art methods in segmentation tasks across multiple 3D imaging modalities, exhibiting improved accuracy and efficiency.", "questions": "- Could the authors provide further insights into how the model performs with extremely noisy or low-resolution data, particularly in clinical environments where such data is more common?\n- How do the number of experts and the group sizes affect computational efficiency during inference, especially with higher-resolution datasets?", "limitations": "- Scalability: The paper acknowledges the limitation in terms of scalability for very large-scale datasets, which remains an unexplored area for future work.\n- The paper mentions that the model is pre-trained using supervised learning on multimodal datasets. However, its behavior under self-supervised learning, especially on large unannotated datasets, remains unexplored.\n- The model is not tested across a broader range of medical image types, limiting its potential for deployment across all clinical applications.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n- Quality: The paper presents a novel and well-defined methodology, Mamba-HoME, integrating advanced techniques such as Mixture-of-Experts (MoE) and state-space models, and addresses important challenges in 3D medical image segmentation.\n- Clarity: The paper clearly explains the architecture and methodology, providing adequate details on the hierarchical expert structure and token routing process.\n- Significance: The model offers significant improvements over existing models in segmentation accuracy across various modalities, making it relevant for practical medical applications.\n- Originality: The integration of hierarchical MoE layers within a state-space model framework for 3D medical segmentation is a unique and novel approach.\n\n## Weaknesses\n- Ablation study on the specific choices of parameters for the MoE layers (e.g., number of experts, group sizes) could benefit from further discussion on how they influence the model’s performance.\n- The paper does not explore the scalability of the method to very large datasets (e.g., >10,000 scans), which limits understanding of the model’s real-world applicability to massive medical image datasets.", "was_revised": true, "final_justification": "The motivation is good and the methods seem solid.", "rebuttal": "> W1: Ablation study on the specific choices of parameters for the MoE layers (e.g., number of experts, group sizes) could benefit from further discussion on how they influence the model’s performance.\n\nWe agree with the Reviewer that the ablation study should be accompanied by a more thorough discussion of the results. In response, we conducted additional experiments (see detailed results in response to Question 2 (Q2)) and will include these findings, along with an expanded analysis, in the final version of the paper. \n\n> W2: The paper does not explore the scalability of the method to very large datasets (e.g., >10,000 scans), which limits understanding of the model’s real-world applicability to massive medical image datasets.\n\nFollowing this and other Reviewer’s comments, we have now trained the model on the large AbdomenAtlas 1.1 [1] dataset, which consists of 9,262 CT scans in total. Of these, we used 8,242 for training and 1,020 for inference time and memory analysis. For training, we used 128 × 128 × 128 patches and a batch size = 2.\n\nAccording to the Table, Mamba-HoME has a training time of around 3,930 seconds (65.5 minutes), 0.47 s per scan, comparable to other large models, but even shorter than the lightweight uC 3DU-Net (0.59 s per scan). The average inference time for Mamba-HoME is approximately 2 seconds per scan, with a GPU memory requirement of 10.2 GB, which represents a good trade-off between speed and memory usage compared to other state-of-the-art methods. \n\nThis new Table will be incorporated in the final version of the paper.\n\n| Method | Parameters (M) | Training time (s) | Inference time (s) | GPU memory (GB) |\n|:----------------:|:--------------:|:-----------------:|:------------------:|:---------------:|\n| uC 3DU-Net |      21.7      |       0.59        |        1.57        |      12.4       |\n| SuPreM |     62.2      |       0.40        |        2.32        |      15.8       |\n| Swin SMT |    170.8      |       0.42        |        2.09        |      14.6       |\n| Swin UNETR |      72.8      |       0.40        |        2.32        |      15.8       |\n| Hermes |      44.5      |       0.44        |        2.27        |      16.2       |\n| VSmTrans |      47.6      |       0.60        |        2.48        |       9.8       |\n| SegMamba-B |      66.8      |       0.38        |        1.52        |       9.0       |\n| SegMamba-L |     176.5      |       0.49        |        2.53        |      12.0       |\n| Mamba-HoME |     170.5      |       0.47        |        2.01        |      10.2       |\n\nWe also note that the training time and inference times on large datasets generally scale linearly with the number of samples. Therefore, we do not anticipate that application of our model to even larger datasets would incur runtime issues. \n\nFinally, we note that if training on very large datasets (containing thousands of samples) would benefit from further scaling the model to larger parameter numbers, it is technically very easy to scale due to the scalability of MoE [2]. Specifically, to scale the model, we would only require to increase the number of experts.\n\n> Q1: Could the authors provide further insights into how the model performs with extremely noisy or low-resolution data, particularly in clinical environments where such data is more common?\n\nWe appreciate the reviewer’s thoughtful question regarding model performance under noisy or low-resolution conditions, which are indeed common in clinical environments. To address this, we note that our experiments already include evaluations on four diverse 3D clinical imaging modalities: CT, MRI, fetal brain MRI, and 3D ultrasound. Both fetal brain MRI (FeTA) and 3D ultrasound (MVSeg) are particularly challenging due to motion artifacts, low signal-to-noise ratio, and limited spatial resolution. Despite these challenges, our model demonstrated strong and consistent performance across all modalities, indicating robustness to both low-quality and heterogeneous input data, as shown in Tables 3 and 4.\n\n> Q2: How do the number of experts and the group sizes affect computational efficiency during inference, especially with higher-resolution datasets?\n\nWe are grateful for this inspiring suggestion! We performed additional experiments using different numbers of experts and group sizes to show how they affect computational efficiency during inference, especially with higher-resolution datasets. In this case, we use the AMOS-CT dataset (Each CT scan consists of between 74 and 353 slices with a mean of 153. The mean voxel spatial resolution is 0.68 × 0.68 × 3.86 mm³), and we also selected the 100 largest CT scans from the AbdomenAtlas 1.1 dataset (Each CT scan consists of between 786 and 1,440 slices with a mean of 1,156. The mean voxel spatial resolution is 0.82 × 0.82 × 0.79 mm³).\n\nWe varied the expert configurations and observed a gradual increase in both inference time and GPU memory usage as the number of experts increased. These results show that while more experts lead to higher specialization, the increase in inference time and memory usage remains moderate and predictable, confirming scalability. E1 and E2 indicate the number of experts in the first and the second level of HoME, respectively. In this experiment, we used K = [2048, 1024, 512, 256] groups as the default.\n\n\n| E1 | E2 | Inference time AMOS-CT (s) | Inference time 100 high-dimensional (s) | GPU memory AMOS-CT (GB) | GPU memory 100 high-dimensional (GB) |\n|:---------------:|:---------------:|:--------------------------:|:-----------------------:|:----------------------:|:-------------------:|\n| [4, 8, 12, 16] | [8, 16, 24, 32] | 4.70 | 17.10 | 11.2 | 15.1 |\n| [8, 16, 24, 32] | [8, 16, 24, 32] | 5.19 | 19.90 | 12.3 | 16.2 |\n| [8, 16, 24, 48] | [16, 32, 48, 96] | 5.53 | 22.67 | 13.0 | 16.9 |\n| [16, 32, 48, 64] | [16, 32, 48, 64] | 6.27 | 23.55 | 13.1 | 17.0 |\n\nWe also tested various group sizes to study how granularity affects efficiency.\nThis trade-off suggests that larger group sizes enhance efficiency, particularly for large-volume data, while smaller groups offer more flexibility with a slight cost.\n\nIn this experiment, we use E1 = E2 = [4, 8, 12, 16] as the default, except **, where E1 = [4, 8, 12, 16], and E2 = [8, 16, 24, 32].\n\n| Group size (K) | Inference time AMOS-CT (s) | Inference time 100 high-dimensional (s) | GPU memory AMOS-CT (GB) | GPU memory 100 high-dimensional (GB) |\n|:--------------------------:|:---------------------------:|:------------------------:|:-----------------------:|:--------------------:|\n| [1024, 512, 256, 128] | 4.85 | 17.60 | 11.0 | 14.9 |\n| [2048, 1024, 512, 256]** | 4.70 | 17.10 | 11.2 | 15.1 |\n| [2048, 1024, 512, 256] | 4.60 | 17.00 | 11.0 | 14.9  |\n| [4096, 2048, 1024, 512] | 4.48 | 15.90 | 11.0 | 14.9 |\n\nOur findings indicate that increasing the number of experts results in a moderate and predictable increase in inference time and GPU memory usage. Even at the most demanding configuration ([16, 32, 48, 64] experts), inference remains practical, 6.27 seconds for AMOS-CT and 23.55 seconds for the largest CT volumes. These are acceptable runtimes given the scale of the input data, particularly for volumetric scans comprising over 1,000 slices. Note that, in practical 3D image analysis, segmentation is performed after image acquisition, and there is no need for real-time inference.\n\nImportantly, the high-dimensional CT scans used here represent extremely large cases such as whole-body scans, which are rare in clinical practice. In contrast, datasets like AMOS-CT, with an average of ~150 slices, reflect typical diagnostic studies used in routine clinical workflows (e.g., abdominal or pelvic CTs). For such cases, inference times remain around 4.7–6.3 seconds, and memory consumption is well within the capabilities of standard high-end GPUs.\n\nAdditionally, varying the group sizes demonstrates that coarser groupings improve computational efficiency, especially for large-volume scans, with no detrimental impact on memory usage.\n\n> L1: Scalability: The paper acknowledges the limitation in terms of scalability for very large-scale datasets, which remains an unexplored area for future work.\n\nIndeed, as noted in the paper, our current experiments were conducted on datasets of moderate size to enable controlled evaluation and thorough analysis. Applying our method to very large-scale datasets (e.g., hundreds of millions of examples) remains an important direction for future work. \n\n> L2: The paper mentions that the model is pre-trained using supervised learning on multimodal datasets. However, its behavior under self-supervised learning, especially on large unannotated datasets, remains unexplored.\n\nWe agree that exploring self-supervised pre-training on large unannotated datasets is an important future direction. However, our choice to adopt supervised pre-training on a smaller multimodal dataset was motivated by the findings of Li et al. [1], who demonstrate that supervised 3D models trained on relatively small datasets can transfer more effectively to downstream medical imaging tasks than models pre-trained in a self-supervised manner on large-scale data. Our focus in this work was to leverage these insights to maximize downstream performance under limited compute and data conditions.\n\n> L3: The model is not tested across a broader range of medical image types, limiting its potential for deployment across all clinical applications.\n\nWe would like to clarify that we have evaluated our method on three of the most widely used and diverse 3D medical imaging modalities: CT, MRI, and ultrasound. These modalities cover a broad spectrum of imaging characteristics, including differences in contrast and textures, high and low resolutions, noise properties, and anatomical focus. \n\n**References**\n\n[1] Li et al. \"How Well Do Supervised 3D Models Transfer to Medical Imaging Tasks?.\" ICLR, 2024\n\n[2] Puigcerver et al. \"From Sparse to Soft Mixtures of Experts.\" ICLR, 2024"}]

Author rebuttal:

Meta-review: This paper presents a method for 3D medical image segmentation. Overall, the presentation is clear and easy to follow. The results show some improvement on 3D segmentation tasks. There are some concerns on the novelty, but the reviewers are more on the positive side of the work. Based on my reading of the rebuttals, the responses have addressed the concern of the comments, with confirmation from reviewers as well.
Decision: Accept (poster)