  Fully Convolutional Neural Networks (FCNNs) with contracting and expansive
paths (e.g. encoder and decoder) have shown prominence in various medical image
segmentation applications during the recent years. In these architectures, the
encoder plays an integral role by learning global contextual representations
which will be further utilized for semantic output prediction by the decoder.
Despite their success, the locality of convolutional layers , as the main
building block of FCNNs limits the capability of learning long-range spatial
dependencies in such networks. Inspired by the recent success of transformers
in Natural Language Processing (NLP) in long-range sequence learning, we
reformulate the task of volumetric (3D) medical image segmentation as a
sequence-to-sequence prediction problem. In particular, we introduce a novel
architecture, dubbed as UNEt TRansformers (UNETR), that utilizes a pure
transformer as the encoder to learn sequence representations of the input
volume and effectively capture the global multi-scale information. The
transformer encoder is directly connected to a decoder via skip connections at
different resolutions to compute the final semantic segmentation output. We
have extensively validated the performance of our proposed model across
different imaging modalities(i.e. MR and CT) on volumetric brain tumour and
spleen segmentation tasks using the Medical Segmentation Decathlon (MSD)
dataset, and our results consistently demonstrate favorable benchmarks.
