Title: Mamba Goes HoME: Hierarchical Soft Mixture-of-Experts for 3D Medical Image Segmentation
Venue: NeurIPS 2025
Accepted: yes
Keywords: mixture of experts

state-space models

segmentation

medical imaging
Abstract: In recent years, artificial intelligence has significantly advanced medical image segmentation. Nonetheless, challenges remain, including efficient 3D medical image processing across diverse modalities and handling data variability. In this work, we introduce Hierarchical Soft Mixture-of-Experts (HoME), a two-level token-routing layer for efficient long-context modeling, specifically designed for 3D medical image segmentation. Built on the Mamba Selective State Space Model (SSM) backbone, HoME enhances sequential modeling through adaptive expert routing. In the first level, a Soft Mixture-of-Experts (SMoE) layer partitions input sequences into local groups, routing tokens to specialized per-group experts for localized feature extraction. The second level aggregates these outputs through a global SMoE layer, enabling cross-group information fusion and global context refinement. This hierarchical design, combining local expert routing with global expert refinement, enhances generalizability and segmentation performance, surpassing state-of-the-art results across datasets from the three most widely used 3D medical imaging modalities and varying data qualities. The code is publicly available at https://github.com/gmum/MambaHoME.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new framework, Mamba-HoME, for 3D medical image segmentation. It integrates Hierarchical Soft Mixture-of-Experts (HoME) into the Mamba model, which efficiently handles long-range dependencies and local-to-global feature extraction. HoME is designed with two levels of token-routing experts: one for local feature extraction and the other for global context aggregation. The authors demonstrate that Mamba-HoME outperforms existing state-of-the-art methods in segmentation tasks across multiple 3D imaging modalities, exhibiting improved accuracy and efficiency.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- Quality: The paper presents a novel and well-defined methodology, Mamba-HoME, integrating advanced techniques such as Mixture-of-Experts (MoE) and state-space models, and addresses important challenges in 3D medical image segmentation.\n- Clarity: The paper clearly explains the architecture and methodology, providing adequate details on the hierarchical expert structure and token routing process.\n- Significance: The model offers significant improvements over existing models in segmentation accuracy across various modalities, making it relevant for practical medical applications.\n- Originality: The integration of hierarchical MoE layers within a state-space model framework for 3D medical segmentation is a unique and novel approach.\n\n## Weaknesses\n- Ablation study on the specific choices of parameters for the MoE layers (e.g., number of experts, group sizes) could benefit from further discussion on how they influence the model’s performance.\n- The paper does not explore the scalability of the method to very large datasets (e.g., >10,000 scans), which limits understanding of the model’s real-world applicability to massive medical image datasets.\n\nQUESTIONS:\n- Could the authors provide further insights into how the model performs with extremely noisy or low-resolution data, particularly in clinical environments where such data is more common?\n- How do the number of experts and the group sizes affect computational efficiency during inference, especially with higher-resolution datasets?\n\nLIMITATIONS:\n- Scalability: The paper acknowledges the limitation in terms of scalability for very large-scale datasets, which remains an unexplored area for future work.\n- The paper mentions that the model is pre-trained using supervised learning on multimodal datasets. However, its behavior under self-supervised learning, especially on large unannotated datasets, remains unexplored.\n- The model is not tested across a broader range of medical image types, limiting its potential for deployment across all clinical applications.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces the Hierarchical Soft Mixture-of-Experts (HoME), a novel two-level token-routing mechanism designed for efficient 3D medical image segmentation. It builds on the Mamba state-space model (SSM) to combine local and global context modeling effectively. The proposed method demonstrates robust cross-modality generalization and significant advancements in segmentation performance, particularly in handling hierarchical structures in 3D medical data. These contributions aim to address the challenges of scalability and generalization in 3D medical image analysis.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1.\tThe methodology is robust and well-executed. The integration of the Hierarchical Soft Mixture-of-Experts (HoME) with the Mamba state-space model (SSM) addresses the need for efficient local-to-global feature modeling in 3D medical image segmentation.\n\n2.\tExtensive experimental results validate the proposed model's performance across diverse datasets and imaging modalities (CT, MRI, Ultrasound), demonstrating both accuracy improvements and computational efficiency.\n\n3.\tThe ablation studies provide a thorough exploration of the impact of architectural and hyperparameter choices, strengthening the validity of the conclusions.\n\nWeaknesses\n\n1.\tThe model has 170.5M parameters, which is larger than the lightweight model (e.g. uC 3DU-Net is only 21.7M), and is about 30% slower than the baseline SegMamba. The accuracy improvement comes at the expense of speed. While the paper demonstrates improvements in computational efficiency, it lacks detailed discussions of memory constraints and training times for very large-scale datasets, which might impact its scalability. The significance of the computational trade-offs (e.g., slightly slower inference speeds compared to simpler baselines) is not fully analyzed in the context of real-world clinical workflows, where latency may be a critical factor. \n\n2.\tWhile the integration of state-space models and hierarchical MoE is novel, the core ideas build upon well-established frameworks (e.g., SSMs and MoEs). So the novelty is limited.\n\nQUESTIONS:\n1.\tInclude more detailed analysis of memory usage and training/inference times for large datasets.\n\n2.\tProvide additional visual or textual explanations of the HoME layer for readers less familiar with the technical details.\n\n3.\tDiscuss potential latency concerns in real-world applications of the model.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis author proposed a Mamba-HoME, a novel 3D medical image segmentation architecture combining Mamba state-space models with a Hierarchical Soft Mixture-of-Experts (HoME) layer. HoME uses two-level token routing to capture local-to-global dependencies efficiently. Evaluations across CT, MRI, and ultrasound datasets show state-of-the-art segmentation performance with favorable computational efficiency and generalizability.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThe approach innovatively integrates SSMs with hierarchical expert routing, addressing the scalability issues of transformers for large 3D volumes. Extensive experiments across five datasets convincingly demonstrate consistent improvements in Dice score and boundary accuracy. The ablations systematically explore design choices (number of experts, group sizes, slots). The inclusion of pretraining and cross-modal transfer further strengthens the claim of generalizability.\n\nWeakness:\nPerformance gains over prior SSM methods (e.g., SegMamba) are modest (~1–2% Dice). The complexity of HoME increases parameter count and inference time (~30% slower). No user studies or clinical validation are provided. Some claims about “unprecedented memory and compute efficiency” may be overstated since inference speed lags behind simpler baselines. Details about dataset preprocessing and training schedules are scattered across appendix sections.\n\nQUESTIONS:\n1. Some sentences in Introduction are very long—could you break them up for clarity?\n2. Please  standardize font sizes across all tables for consistency?\n3. How much improvement does HoME provide over plain Mamba when controlling for parameter count?\n4. What is the runtime and GPU memory footprint on a standard high-resolution volume?\n5. Did you test cross-institution generalization or domain shift robustness?\n6. Is there evidence that the second-level global routing meaningfully improves performance?\n7. Are the observed improvements statistically significant on all datasets?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}