bibtext: @InProceedings{ Lyu_MetaUNETR_MICCAI2024,
                   author = { Lyu, Pengju and Zhang, Jie and Zhang, Lei and Liu, Wenjian and Wang, Cheng and Zhu, Jianjun },
                   title = { { MetaUNETR: Rethinking Token Mixer Encoding for Efficient Multi-Organ Segmentation } },
                   booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
                   year = {2024},
                   publisher = {Springer Nature Switzerland},
                   volume = { LNCS 15009 },
                   month = {October},
                   pages = { pending },
                }
abstract: The Transformer architecture and versatile CNN backbones have led to advanced progress in sequence modeling and dense prediction tasks. A critical development is the incorporation of different token mixing modules such as ConvNeXt, Swin Transformer. However, findings within the MetaFormer framework suggest these token mixers have a lesser influence on representation learning than the architecture itself. Yet, their impact on 3D medical images remains unclear, motivating our investigation into different token mixers (self-attention, convolution, MLP, recurrence, global filter, and Mamba) in 3D medical image segmentation architectures, and further prompting a reevaluation of the backbone architecture’s role to achieve the trade off in accuracy and efficiency. In the paper, we propose a unified segmentation architecture—MetaUNETR featuring a novel TriCruci layer that decomposes the token mixing processes along each spatial direction while simultaneously preserving precise positional information on its orthogonal plane. By employing the Centered Kernel Alignment (CKA) analysis on feature learning capabilities among these token mixers, we find that the overall architecture of the model, rather than any specific token mixers, plays a more crucial role in determining the model’s performance. Our method is validated across multiple benchmarks varying in size and scale, including the BTCV, AMOS, and AbdomenCT-1K datasets, achieving the top segmentation performance while reducing the model’s parameters by about 80% compared to the state-of-the-art method. This study provides insights for future research on the design and optimization of backbone architecture, steering towards more efficient foundational segmentation models. The source code is available at https://github.com/lyupengju/MetaUNETR.
title: MetaUNETR: Rethinking Token Mixer Encoding for Efficient Multi-Organ Segmentation
authors: Lyu, Pengju Zhang, Jie Zhang, Lei Liu, Wenjian Wang, Cheng Zhu, Jianjun