Original: HUMUS-Net: Hybrid Unrolled Multi-Scale Network Architecture for Accelerated MRI Reconstruction
Revised: HUMUS-Net: Hybrid Unrolled Multi-Scale Network Architecture for Accelerated MRI Reconstruction

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: In accelerated MRI reconstruction, the anatomy of a patient is recovered from a setof under-sampled and noisy measurements.
Revised: In accelerated MRI reconstruction, the anatomy of a patient is recovered from a set of under-sampled and noisy measurements.

Original: Deep learning approaches have beenproven to be successful in solving this ill-posed inverse problem and are capable ofproducing very high quality reconstructions.
Revised: Deep learning approaches have been proven to be successful in solving this ill-posed inverse problem and are capable of producing very high quality reconstructions.

Original: However, current architectures heavilyrely on convolutions, that are content-independent and have difﬁculties modelinglong-range dependencies in images.
Revised: However, current architectures heavily rely on convolutions, that are content-independent and have difﬁculties modeling long-range dependencies in images.

Original: Recently, Transformers, the workhorse ofcontemporary natural language processing, have emerged as powerful buildingblocks for a multitude of vision tasks.
Revised: Recently, Transformers, the workhorse of contemporary natural language processing, have emerged as powerful building blocks for a multitude of vision tasks.

Original: These models split input images into nonoverlapping patches, embed the patches into lower-dimensional tokens and utilizea self-attention mechanism that does not suffer from the aforementioned weaknesses of convolutional architectures.
Revised: These models split input images into nonoverlapping patches, embed the patches into lower-dimensional tokens and utilize a self-attention mechanism that does not suffer from the aforementioned weaknesses of convolutional architectures.

Original: However, Transformers incur extremelyhigh compute and memory cost when 1) the input image resolution is high and 2)when the image needs to be split into a large number of patches to preserve ﬁnedetail information, both of which are typical in low-level vision problems suchas MRI reconstruction, having a compounding effect.
Revised: However, Transformers incur extremely high compute and memory cost when 1) the input image resolution is high and 2) when the image needs to be split into a large number of patches to preserve ﬁne detail information, both of which are typical in low-level vision problems such as MRI reconstruction, having a compounding effect.

Original: To tackle these challenges,we propose HUMUS-Net, a hybrid architecture that combines the beneﬁcial implicit bias and efﬁciency of convolutions with the power of Transformer blocksin an unrolled and multi-scale network.
Revised: To tackle these challenges, we propose HUMUS-Net, a hybrid architecture that combines the beneﬁcial implicit bias and efﬁciency of convolutions with the power of Transformer blocks in an unrolled and multi-scale network.

Original: HUMUS-Net extracts high-resolutionfeatures via convolutional blocks and reﬁnes low-resolution features via a novel
Revised: HUMUS-Net extracts high-resolution features via convolutional blocks and reﬁnes low-resolution features via a novel

Original: Transformer-based multi-scale feature extractor.
Revised: Transformer-based multi-scale feature extractor.

Original: Features from both levels are thensynthesized into a high-resolution output reconstruction.
Revised: Features from both levels are then synthesized into a high-resolution output reconstruction.

Original: Our network establishesnew state of the art on the largest publicly available MRI dataset, the fastMRIdataset.
Revised: Our network establishes new state of the art on the largest publicly available MRI dataset, the fastMRI dataset.

Original: We further demonstrate the performance of HUMUS-Net on two otherpopular MRI datasets and perform ﬁne-grained ablation studies to validate ourdesign.
Revised: We further demonstrate the performance of HUMUS-Net on two other popular MRI datasets and perform ﬁne-grained ablation studies to validate our design.

Original:  Introduction
Revised:  Introduction

Original: Magnetic resonance imaging (MRI) is a medical imaging technique that uses strong magnetic ﬁeldsto picture the anatomy and physiological processes of the patient.
Revised: Magnetic resonance imaging (MRI) is a medical imaging technique that uses strong magnetic ﬁelds to picture the anatomy and physiological processes of the patient.

Original: MRI is one of the most popularimaging modalities as it is non-invasive and doesn’t expose the patient to harmful ionizing radiation.
Revised: MRI is one of the most popular imaging modalities as it is non-invasive and doesn’t expose the patient to harmful ionizing radiation.

Original: The MRI scanner obtains measurements of the body in the spatial frequency domain also calledk-space .
Revised: The MRI scanner obtains measurements of the body in the spatial frequency domain also called k-space .

Original: The data acquisition process is MR is often time-consuming.
Revised: The data acquisition process is MR is often time-consuming.

Original: Accelerated MRI
Revised: Accelerated MRI

Original: [Lustiget al., 2008] addresses this challenge by undersampling in the k-space domain, thus reducing thetime patients need to spend in the scanner.
Revised: [Lustig et al., 2008] addresses this challenge by undersampling in the k-space domain, thus reducing the time patients need to spend in the scanner.

Original: However, recovering the underlying anatomy fromundersampled measurements is an ill-posed problem (less measurements than unknowns) and thusincorporating some form of prior knowledge is crucial in obtaining high quality reconstructions.
Revised: However, recovering the underlying anatomy from undersampled measurements is an ill-posed problem (less measurements than unknowns) and thus incorporating some form of prior knowledge is crucial in obtaining high quality reconstructions.

Original: Classical MRI reconstruction algorithms rely on the assumption that the underlying signal is sparsein some transform domain and attempt to recover a signal that best satisﬁes this assumption in a technique known as compressed sensing (CS)
Revised: Classical MRI reconstruction algorithms rely on the assumption that the underlying signal is sparse in some transform domain and attempt to recover a signal that best satisﬁes this assumption in a technique known as compressed sensing (CS)

Original: [Candes et al., 2006, Donoho, 2006].
Revised: [Candes et al., 2006, Donoho, 2006].

Original: These classical
Revised: These classical

Original: CS techniques have slow reconstruction speed and typically enforce limited forms of image priors.
Revised: CS techniques have slow reconstruction speed and typically enforce limited forms of image priors.

Original: With the emergence of deep learning (DL), data-driven reconstruction algorithms have far surpassed
Revised: With the emergence of deep learning (DL), data-driven reconstruction algorithms have far surpassed

Original: CS techniques (see Ongie et al.
Revised: CS techniques (see Ongie et al.

Original: [2020] for an overview).
Revised: [2020] for an overview).

Original: DL models utilize large training datasetsto extract ﬂexible, nuanced priors directly from data resulting in excellent reconstruction quality.
Revised: DL models utilize large training datasets to extract ﬂexible, nuanced priors directly from data resulting in excellent reconstruction quality.

Original: In recent years, there has been a ﬂurry of activity aimed at designing DL architectures tailoredto the MRI reconstruction problem.
Revised: In recent years, there has been a ﬂurry of activity aimed at designing DL architectures tailored to the MRI reconstruction problem.

Original: The most popular models are convolutional neural networks(CNNs) that typically incorporate the physics of the MRI reconstruction problem, and utilize toolsfrom mainstream deep learning (residual learning, data augmentation, self-supervised learning).
Revised: The most popular models are convolutional neural networks (CNNs) that typically incorporate the physics of the MRI reconstruction problem, and utilize tools from mainstream deep learning (residual learning, data augmentation, self-supervised learning).

Original: Comparing the performance of such models has been difﬁcult mainly due to two reasons.
Revised: Comparing the performance of such models has been difﬁcult mainly due to two reasons.

Original: First, therehas been a large variation in evaluation datasets spanning different scanners, anatomies, acquisitionmodels and undersampling patterns rendering direct comparison challenging.
Revised: First, there has been a large variation in evaluation datasets spanning different scanners, anatomies, acquisition models and undersampling patterns rendering direct comparison challenging.

Original: Second, medicalimaging datasets are often proprietary due to privacy concerns, hindering reproducibility.
Revised: Second, medical imaging datasets are often proprietary due to privacy concerns, hindering reproducibility.

Original: More recently, the fastMRI dataset [Zbontar et al., 2019], the largest publicly available MRI dataset,has been gaining ground as a standard benchmark to evaluate MRI reconstruction methods.
Revised: More recently, the fastMRI dataset [Zbontar et al., 2019], the largest publicly available MRI dataset, has been gaining ground as a standard benchmark to evaluate MRI reconstruction methods.

Original: An annualcompetition, the fastMRI Challenge [Muckley et al., 2021], attracts signiﬁcant attention from themachine learning community and acts a driver of innovation in MRI reconstruction.
Revised: An annual competition, the fastMRI Challenge [Muckley et al., 2021], attracts signiﬁcant attention from the machine learning community and acts a driver of innovation in MRI reconstruction.

Original: However, over thepast years the public leaderboard has been dominated by a single architecture, the End-to-End VarNet Sriram et al. [2020] 1 , with most models concentrating very closely around the same performancemetrics, hinting at the saturation of current architectural choices.
Revised: However, over the past years the public leaderboard has been dominated by a single architecture, the End-to-End VarNet [Sriram et al., 2020] 1 , with most models concentrating very closely around the same performance metrics, hinting at the saturation of current architectural choices.

Original: In this work, we propose HUMUS-Net: a Hybrid, UNrolled, Multi-Scale network architecturefor accelerated MRI reconstruction that combines the advantages of well-established architecturesin the ﬁeld with the power of contemporary Transformer-based models.
Revised: In this work, we propose HUMUS-Net: a Hybrid, Unrolled, MUlti-Scale network architecture for accelerated MRI reconstruction that combines the advantages of well-established architectures in the ﬁeld with the power of contemporary Transformer-based models.

Original: We utilize the strongimplicit bias of convolutions, but also address their weaknesses, such as content-independence andinability to model long-range dependencies, by incorporating a novel multi-scale feature extractorthat operates over embedded image patches via self-attention.
Revised: We utilize the strong implicit bias of convolutions, but also address their weaknesses, such as content-independence and inability to model long-range dependencies, by incorporating a novel multi-scale feature extractor that operates over embedded image patches via self-attention.

Original: Moreover, we tackle the challenge ofhigh input resolution typical in MRI by performing the computationally most expensive operationson extracted low-resolution features.
Revised: Moreover, we tackle the challenge of high input resolution typical in MRI by performing the computationally most expensive operations on extracted low-resolution features.

Original: HUMUS-Net establishes new state of the art in accelerated
Revised: HUMUS-Net establishes new state of the art in accelerated

Original: MRI reconstruction on the largest available MRI knee dataset.
Revised: MRI reconstruction on the largest available MRI knee dataset.

Original: At the time of writing this paper,
Revised: At the time of writing this paper,

Original: HUMUS-Net is the only Transformer-based architecture on the highly competitive fastMRI Public
Revised: HUMUS-Net is the only Transformer-based architecture on the highly competitive fastMRI Public

Original: Leaderboard.
Revised: Leaderboard.

Original: Our results are fully reproducible and the source code is available in the supplementary.
Revised: Our results are fully reproducible and the source code is available online

Original:  Background 
Revised:  Background 2.1 Inverse Problem Formulation of Accelerated MRI Reconstruction

Original: 2.1 Inverse Problem Formulation of Accelerated MRI Reconstruction

Original: An MR scanner obtains measurements of the patient anatomy in the frequency domain, also referredto as k-space .
Revised: An MR scanner obtains measurements of the patient anatomy in the frequency domain, also referred to as k-space .

Original: Data acquisition is performed via various receiver coils positioned around the anatomybeing imaged, each with different spatial sensitivity.
Revised: Data acquisition is performed via various receiver coils positioned around the anatomy being imaged, each with different spatial sensitivity.

Original: Given a total number of N receiver coils, themeasurements obtained by the i th coil can be written as
Revised: Given a total number of N receiver coils, the measurements obtained by the i th coil can be written as

Original: [Equation]
Revised: [Equation]

Original: where x ∗ ∈ C n is the underlying patient anatomy of interest, S i is a diagonal matrix that representsthe sensitivity map of the i th coil, F is a multi-dimensional Fourier-transform, and z i denotes themeasurement noise corrupting the observations obtained from coil i .
Revised: where x ∗ ∈ C n is the underlying patient anatomy of interest, S i is a diagonal matrix that represents the sensitivity map of the i th coil, F is a multi-dimensional Fourier-transform, and z i denotes the measurement noise corrupting the observations obtained from coil i .

Original: We use k = ( k 1 , ..., k N ) asa shorthand for the concatenation of individual coil measurements and x = ( x 1 , ..., x N ) as thecorresponding image domain representation after inverse Fourier transformation.
Revised: We use k = ( k 1 , ..., k N ) as a shorthand for the concatenation of individual coil measurements and x = ( x 1 , ..., x N ) as the corresponding image domain representation after inverse Fourier transformation.

Original: Since MR data acquisition time is proportional to the portion of k-space being scanned, obtainingfully-sampled data is time-consuming.
Revised: Since MR data acquisition time is proportional to the portion of k-space being scanned, obtaining fully-sampled data is time-consuming.

Original: Therefore, in accelerated MRI scan times are reduced byundersampling in k-space domain.
Revised: Therefore, in accelerated MRI scan times are reduced by undersampling in k-space domain.

Original: The undersampled k-space measurements from coil i take theform
Revised: The undersampled k-space measurements from coil i take the form

Original: [Equation]
Revised: [Equation]

Original: where M is a diagonal matrix representing the binary undersampling mask, that has 0 values for allmissing frequency components that have not been sampled during accelerated 
Revised: where M is a diagonal matrix representing the binary undersampling mask, that has 0 values for all missing frequency components that have not been sampled during accelerated acquisition.

Original: The forward model that maps the underlying anatomy to coil measurements can be written conciselyas ˜ k
Revised: The forward model that maps the underlying anatomy to coil measurements can be written concisely as ˜ k

Original: = A ( x ∗ ) , where A ( · ) is the linear forward mapping and ˜ k is the stacked vector of allundersampled coil measurements.
Revised: = A ( x ∗ ) , where A ( · ) is the linear forward mapping and ˜ k is the stacked vector of all undersampled coil measurements.

Original: Our target is to reconstruct the ground truth object x ∗ fromthe noisy, undersampled measurements ˜ k .
Revised: Our target is to reconstruct the ground truth object x ∗ from the noisy, undersampled measurements ˜ k .

Original: Since we have fewer observations than variables torecover, perfect reconstruction in general is not possible.
Revised: Since we have fewer observations than variables to recover, perfect reconstruction in general is not possible.

Original: In order to make the problem solvable,prior knowledge on the underlying object is typically incorporated in the form of sparsity in sometransform domain.
Revised: In order to make the problem solvable, prior knowledge on the underlying object is typically incorporated in the form of sparsity in some transform domain.

Original: This formulation, known as compressed sensing [Candes et al., 2006, Donoho,2006], provides a classical framework for accelerated MRI reconstruction [Lustig et al., 2008].
Revised: This formulation, known as compressed sensing [Candes et al., 2006, Donoho, 2006], provides a classical framework for accelerated MRI reconstruction [Lustig et al., 2008].

Original: Inparticular, the above recovery problem can be formulated as a regularized inverse problem
Revised: In particular, the above recovery problem can be formulated as a regularized inverse problem

Original: [Equation]
Revised: [Equation]

Original: where R ( · ) is a regularizer that encapsulates prior knowledge on the object, such as sparsity in somewavelet domain.
Revised: where R ( · ) is a regularizer that encapsulates prior knowledge on the object, such as sparsity in some wavelet domain.

Original: 2.2 Deep Learning-based Accelerated MRI Reconstruction
Revised: 2.2 Deep Learning-based Accelerated MRI Reconstruction

Original: More recently, data-driven deep learning-based algorithms tailored to the accelerated MRI reconstruction problem have surpassed the classical compressed sensing baselines.
Revised: More recently, data-driven deep learning-based algorithms tailored to the accelerated MRI reconstruction problem have surpassed the classical compressed sensing baselines.

Original: Convolutional neuralnetworks trained on large datasets have established new state of the art in many medical imaging tasks.
Revised: Convolutional neural networks trained on large datasets have established new state of the art in many medical imaging tasks.

Original: The highly popular U-Net [Ronneberger et al., 2015] and other similar encoder-decoder architectureshave proven to be successful in a range of medical image reconstruction [Hyun et al., 2018, Hanand Ye, 2018] andsegmentation Çiçek et al.
Revised: The highly popular U-Net [Ronneberger et al., 2015] and other similar encoder-decoder architectures have proven to be successful in a range of medical image reconstruction [Hyun et al., 2018, Han and

Original: [2016], Zhou et al. [2018] problems.
Revised: Ye, 2018] and segmentation [Çiçek et al., 2016, Zhou et al., 2018] problems.

Original: In the encoderpath, the network learns to extract a set of deep, low-dimensional features from images via a series ofconvolutional and downsampling operations.
Revised: In the encoder path, the network learns to extract a set of deep, low-dimensional features from images via a series of convolutional and downsampling operations.

Original: These concise feature representations are then graduallyupsampled and ﬁltered in the decoder to the original image dimensions.
Revised: These concise feature representations are then gradually upsampled and ﬁltered in the decoder to the original image dimensions.

Original: Thus the network learns ahierarchical representation over the input image distribution.
Revised: Thus the network learns a hierarchical representation over the input image distribution.

Original: Unrolled networks constitute another line of work that has been inspired by popular optimizationalgorithms used to solve compressed sensing reconstruction problems.
Revised: Unrolled networks constitute another line of work that has been inspired by popular optimization algorithms used to solve compressed sensing reconstruction problems.

Original: These deep learning modelsconsist of a series of sub-networks, also known as cascades, where each sub-network correspondsto an unrolled iteration of popular algorithms such as gradient descent Zhang and Ghanem [2018]or ADMM Sun et al.
Revised: These deep learning models consist of a series of sub-networks, also known as cascades, where each sub-network corresponds to an unrolled iteration of popular algorithms such as gradient descent [Zhang and Ghanem, 2018] or ADMM [Sun et al., 2016].

Original: In the context of MRI reconstruction, one can view network unrollingas solving a sequence of smaller denoising problems, instead of the complete recovery problem inone step.
Revised: In the context of MRI reconstruction, one can view network unrolling as solving a sequence of smaller denoising problems, instead of the complete recovery problem in one step.

Original: Various convolutional neural networks have been employed in the unrolling frameworkachieving excellent performance in accelerated MRI reconstruction Putzky et al. [2019], Hammerniket al.
Revised: Various convolutional neural networks have been employed in the unrolling framework achieving excellent performance in accelerated MRI reconstruction [Putzky et al., 2019, Hammernik et al., 2018, 2019].

Original: E2E-VarNet [Sriram et al., 2020] is the current state-of-the-art convolutionalmodel on the fastMRI dataset.
Revised: E2E-VarNet [Sriram et al., 2020] is the current state-of-the-art convolutional model on the fastMRI dataset.

Original: E2E-VarNet transforms the optimization problem in (2.1) to thek-space domain and unrolls the gradient descent iterations into T cascades, where the t th cascaderepresents the computation
Revised: E2E-VarNet transforms the optimization problem in (2.1) to the k-space domain and unrolls the gradient descent iterations into T cascades, where the t th cascade represents the computation

Original: [Equation]
Revised: [Equation]

Original: where ˆ k t is the estimated reconstruction in the k-space domain at cascade t , µ t is a learnable stepsize parameter and G ( · ) is a learned mapping representing the gradient of the regularization term in(2.1).
Revised: where ˆ k t is the estimated reconstruction in the k-space domain at cascade t , µ t is a learnable step size parameter and G ( · ) is a learned mapping representing the gradient of the regularization term in (2.1).

Original: The ﬁrst term is also known as data consistency (DC) term as itenforces the consistency of theestimate with the available measurements.
Revised: The ﬁrst term is also known as data consistency (DC it consistency estimate with the available measurements.

Original:  Related Work
Revised:  Related Work

Original: Transformers in Vision – Vision Transformer (ViT) [Dosovitskiy et al., 2020], a fully nonconvolutional vision architecture, has demonstrated state-of-the-art performance on image classiﬁcation problems when pre-trained on large-scale image datasets.
Revised: Transformers in Vision – Vision Transformer (ViT) [Dosovitskiy et al., 2020], a fully nonconvolutional vision architecture, has demonstrated state-of-the-art performance on image classiﬁcation problems when pre-trained on large-scale image datasets.

Original: The key idea of ViT is to split theinput image into non-overlapping patches, embed each patch via a learned linear mapping and processthe resulting tokens via stacked self-attention and multi-layer perceptron (MLP) blocks.
Revised: The key idea of ViT is to split the input image into non-overlapping patches, embed each patch via a learned linear mapping and process the resulting tokens via stacked self-attention and multi-layer perceptron (MLP) blocks.

Original: For moredetails we refer the reader to Appendix F and [Dosovitskiy et al., 2020].
Revised: For more details we refer the reader to Appendix G and [Dosovitskiy et al., 2020].

Original: The beneﬁt of Transformersover convolutional architectures in vision lies in their ability to capture long-range dependencies inimages via the self-attention mechanism.
Revised: The beneﬁt of Transformers over convolutional architectures in vision lies in their ability to capture long-range dependencies in images via the self-attention mechanism.

Original: Since the introduction of ViT, similar attention-based architectures have been proposed for many othervision tasks such as object detection [Carion et al., 2020], image segmentation [Wang et al., 2021b] and restoration [Cao et al., 2021b, Chen et al., 2021, Liang et al., 2021, Zamir et al., 2021, Wang et al. ,2021c].
Revised: Since the introduction of ViT, similar attention-based architectures have been proposed for many other vision tasks such as object detection [Carion et al., 2020], image segmentation [Wang et al., 2021b] and restoration [Cao et al., 2021b, Chen et al., 2021, Liang et al., 2021, Zamir et al., 2021, Wang et al., 2021c].

Original: A key challenge for Transformers in low-level vision problems is the quadratic computecomplexity of the global self-attention with respect to the input dimension.
Revised: A key challenge for Transformers in low-level vision problems is the quadratic compute complexity of the global self-attention with respect to the input dimension.

Original: In some works, this issuehas been mitigated by splitting the input image into ﬁxed size patches and processing the patchesindependently [Chen et al., 2021].
Revised: In some works, this issue has been mitigated by splitting the input image into ﬁxed size patches and processing the patches independently [Chen et al., 2021].

Original: Others focus on designing hierarchical Transformer architectures[Heo et al., 2021, Wang et al., 2021b] similar to popular ResNets
Revised: Others focus on designing hierarchical Transformer architectures [Heo et al., 2021, Wang et al., 2021b] similar to popular ResNets

Original: [He et al., 2015].
Revised: [He et al., 2015].

Original: Authors in Zamiret al.
Revised: Authors in Zamir et al.

Original: [2021] propose applying self-attention channel-wise rather than across the spatial dimensionthus reducing the compute overhead to linear complexity.
Revised: [2021] propose applying self-attention channel-wise rather than across the spatial dimension thus reducing the compute overhead to linear complexity.

Original: Another successful architecture, the Swin
Revised: Another successful architecture, the Swin

Original: Transformer [Liu et al., 2021], tackles the quadratic scaling issue by computing self-attention onsmaller local windows.
Revised: Transformer [Liu et al., 2021], tackles the quadratic scaling issue by computing self-attention on smaller local windows.

Original: To encourage cross-window interaction, windows in subsequent layers areshifted relative to each other.
Revised: To encourage cross-window interaction, windows in subsequent layers are shifted relative to each other.

Original: Transformers in Medical Imaging – Transformer architectures have been proposed recently to tacklevarious medical imaging problems.
Revised: Transformers in Medical Imaging – Transformer architectures have been proposed recently to tackle various medical imaging problems.

Original: Authors in Cao et al.
Revised: Authors in Cao et al.

Original: [2021a] design a U-Net-like architecturefor medical image segmentation where the traditional convolutional layers are replaced by Swin
Revised: [2021a] design a U-Net-like architecture for medical image segmentation where the traditional convolutional layers are replaced by Swin

Original: Transformer blocks.
Revised: Transformer blocks.

Original: They report strong results on multi-organ and cardiac image segmentation.
Revised: They report strong results on multi-organ and cardiac image segmentation.

Original: In
Revised: In

Original: Zhou et al.
Revised: Zhou et al.

Original: [2021] a hybrid convolutional and Transformer-based U-Net architecture is proposedtailored to volumetric medical image segmentation with excellent results on benchmark datasets.
Revised: [2021] a hybrid convolutional and Transformer-based U-Net architecture is proposed tailored to volumetric medical image segmentation with excellent results on benchmark datasets.

Original: Similar encoder-decoder architectures for various medical segmentation tasks have been investigatedin other works [Huang et al., 2021, Wu et al., 2022].
Revised: Similar encoder-decoder architectures for various medical segmentation tasks have been investigated in other works [Huang et al., 2021, Wu et al., 2022].

Original: However, these networks are tailored for imagesegmentation, a task less sensitive to ﬁne details in the input, and thus larger patch sizes are oftenused (for instance 4 in Cao et al.
Revised: However, these networks are tailored for image segmentation, a task less sensitive to ﬁne details in the input, and thus larger patch sizes are often used (for instance 4 in Cao et al.

Original: [2021a] ).
Revised: [2021a] ).

Original: This allows the network to process larger input images, asthe number of token embedding is greatly reduced, but as we demonstrate in Section 5.2 embeddingindividual pixels as 1 × 1 patches is crucial for MRI reconstruction.
Revised: This allows the network to process larger input images, as the number of token embedding is greatly reduced, but as we demonstrate in Section 5.2 embedding individual pixels as 1 × 1 patches is crucial for MRI reconstruction.

Original: Thus, compute and memorybarriers stemming from large input resolutions are more severe in the MRI reconstruction task andtherefore novel approaches are needed.
Revised: Thus, compute and memory barriers stemming from large input resolutions are more severe in the MRI reconstruction task and therefore novel approaches are needed.

Original: Promising results have been reported employing Transformers in medical image denoising problems,such as low-dose CT denoising [Wang et al., 2021a, Luthra et al., 2021] and low-count PET/MRIdenoising [Zhang et al., 2021].
Revised: Promising results have been reported employing Transformers in medical image denoising problems, such as low-dose CT denoising [Wang et al., 2021a, Luthra et al., 2021] and low-count PET/MRI denoising [Zhang et al., 2021].

Original: However, these studies fail to address the challenge of poor scalingto large input resolutions, and only work on small images via either downsampling the originaldataset [Luthra et al., 2021] or by slicing the large input images into smaller patches [Wang et al. ,2021a].
Revised: However, these studies fail to address the challenge of poor scaling to large input resolutions, and only work on small images via either downsampling the original dataset [Luthra et al., 2021] or by slicing the large input images into smaller patches [Wang et al., 2021a].

Original: In contrast, our proposed architecture works directly on the large resolution images that often arise in MRI reconstruction.
Revised: In contrast, our proposed architecture works directly on the large resolution images that often arise in MRI reconstruction.

Original: Even though some work exists on Transformer-based architectures forsupervised accelerated MRI reconstruction [Huang et al., 2022, Lin and Heckel, 2022, Feng et al. ,2021], and for unsupervised pre-trained reconstruction [Korkmaz et al., 2022], to the best of ourknowledge ours is the ﬁrst work to demonstrate state-of-the-art results on large-scale MRI datasetssuch as the fastMRI dataset.
Revised: Even though some work exists on Transformer-based architectures for supervised accelerated MRI reconstruction [Huang et al., 2022, Lin and Heckel, 2022, Feng et al., 2021], and for unsupervised pre-trained reconstruction [Korkmaz et al., 2022], to the best of our knowledge ours is the ﬁrst work to demonstrate state-of-the-art results on large-scale MRI datasets such as the fastMRI dataset.

Original:  Method
Revised:  Method

Original: HUMUS-Net combines the efﬁciency and beneﬁcial implicit bias of convolutional networks with thepowerful general representations of Transformers and their capability to capture long-range pixeldependencies.
Revised: HUMUS-Net combines the efﬁciency and beneﬁcial implicit bias of convolutional networks with the powerful general representations of Transformers and their capability to capture long-range pixel dependencies.

Original: The resulting hybrid network processes information both in image representation (viaconvolutions) and in patch-embedded token representation (via Transformer blocks).
Revised: The resulting hybrid network processes information both in image representation (via convolutions) and in patch-embedded token representation (via Transformer blocks).

Original: Our proposedarchitecture consists of a sequence of sub-networks, also called cascades.
Revised: Our proposed architecture consists of a sequence of sub-networks, also called cascades.

Original: Each cascade representsan unrolled iteration of an underlying optimization algorithm in k-space (see (2.2)), with an imagedomain denoiser, the HUMUS-Block.
Revised: Each cascade represents an unrolled iteration of an underlying optimization algorithm in k-space (see (2.2)), with an imagedomain denoiser, the HUMUS-Block.

Original: First, we describe the architecture of the HUMUS-Block, thecore component in the sub-network.
Revised: First, we describe the architecture of the HUMUS-Block, the core component in the sub-network.

Original: Then, we specify the high-level, k-space unrolling architectureof HUMUS-Net in Section 4.3.
Revised: Then, we specify the high-level, k-space unrolling architecture of HUMUS-Net in Section 4.3.

Original: 4.1 HUMUS-Block Architecture
Revised: 4.1 HUMUS-Block Architecture

Original: The HUMUS-Block acts as an image-space denoiser that receives an intermediate reconstruction fromthe previous cascade and performs a single step of denoising to produce an improved reconstructionfor the next cascade.
Revised: The HUMUS-Block acts as an image-space denoiser that receives an intermediate reconstruction from the previous cascade and performs a single step of denoising to produce an improved reconstruction for the next cascade.

Original: It extracts high-resolution, shallow features and low-resolution, deep featuresthrough a novel multi-scale transformer-based block, and synthesizes high-resolution features fromthose.
Revised: It extracts high-resolution, shallow features and low-resolution, deep features through a novel multi-scale transformer-based block, and synthesizes high-resolution features from those.

Original: The high-level overview of the HUMUS-Block is depicted in Fig.
Revised: The high-level overview of the HUMUS-Block is depicted in Fig.

Original: High-resolution Feature Extraction– The input to our network is a noisy complex-valued imagex in ∈ R h × w × c in derived from undersampled k-space data, where the real and imaginary partsof the image are concatenated along the channel dimension.
Revised: High-resolution Feature Extraction– The input to our network is a noisy complex-valued image x in ∈ R h × w × c in derived from undersampled k-space data, where the real and imaginary parts of the image are concatenated along the channel dimension.

Original: First, we extract high-resolutionfeatures F H ∈ R h × w × d H from the input noisy image through a convolution layer f H written as
Revised: First, we extract high-resolution features F H ∈ R h × w × d H from the input noisy image through a convolution layer f H written as

Original: F H = f H ( x in ) .
Revised: F H = f H ( x in ) .

Original: This initial 3 × 3 convolution layer provides early visual processing at a relativelylow cost and maps the input to a higher, d H dimensional feature space.
Revised: This initial 3 × 3 convolution layer provides early visual processing at a relatively low cost and maps the input to a higher, d H dimensional feature space.

Original: Important to note that theresolution of the extracted features is the same as the spatial resolution of the input image.
Revised: Important to note that the resolution of the extracted features is the same as the spatial resolution of the input image.

Original: Low-resolution Feature Extraction– In case of MRI reconstruction, this resolution is typicallysigniﬁcantly higher than commonly used image datasets ( 32 × 32 - 256 × 256 ), posing a signiﬁcantchallenge to contemporary Transformer-based models.
Revised: Low-resolution Feature Extraction– In case of MRI reconstruction, input resolution is typically signiﬁcantly higher than commonly used image datasets ( 32 × 32 - 256 × 256 ), posing a signiﬁcant challenge to contemporary Transformer-based models.

Original: Thus we apply a convolutional featureextractor f L to obtain low-resolution features F L = f L ( F H ) , with F L ∈ R
Revised: Thus we apply a convolutional feature extractor f L to obtain low-resolution features F L = f L ( F H ) , with F L ∈ R

Original: h L × w L × d L where f Lconsists of a sequence of convolutional blocks and spatial downsampling operations.
Revised: h L × w L × d L where f L consists of a sequence of convolutional blocks and spatial downsampling operations.

Original: The speciﬁcarchitecture is depicted in Figure 4.
Revised: The speciﬁc architecture is depicted in Figure 4.

Original: The purpose of this module is to perform deeper visual processingand to provide manageable input size to the subsequent computation and memory heavy hybridprocessing module.
Revised: The purpose of this module is to perform deeper visual processing and to provide manageable input size to the subsequent computation and memory heavy hybrid processing module.

Original: In this work, we choose h L = h 2 and w L = w 2 which strikes a balance betweenpreserving spatial information and resource demands.
Revised: In this work, we choose h L = h 2 and w L = w 2 which strikes a balance between preserving spatial information and resource demands.

Original: Furthermore, in order to compensate for thereduced resolution we increase the feature dimension to d L = 2 · d H :=
Revised: Furthermore, in order to compensate for the reduced resolution we increase the feature dimension to d L = 2 · d H :=

Original: d .
Revised: d .

Original: Deep Feature Extraction– The most important part of our model is MUST, a MU lti-scale residual
Revised: Deep Feature Extraction– The most important part of our model is MUST, a MU lti-scale residual

Original: S win T ransformer network.
Revised: S win T ransformer network.

Original: MUST is a multi-scale hybrid feature extractor that takes the lowresolution image representations F L and performs hierarchical Transformer-convolutional hybridprocessing in an encoder-decoder fashion, producing deep features F D = f D ( F L ) , where the speciﬁcarchitecture behind f D is detailed in Subsection 4.2.
Revised: MUST is a multi-scale hybrid feature extractor that takes the low- resolution image representations F L and performs hierarchical Transformer-convolutional hybrid processing in an encoder-decoder fashion, producing deep features F D = f D ( F L ) , where the speciﬁc architecture behind f D is detailed in Subsection 4.2.

Original: High-resolution Image Reconstruction– Finally, we combine information from shallow, highresolution features F H and deep, low-resolution features F D to reconstruct the high-resolutionresidual image via a convolutional reconstruction module f R .
Revised: High-resolution Image Reconstruction– Finally, we combine information from shallow, highresolution features F H and deep, low-resolution features F D to reconstruct the high-resolution residual image via a convolutional reconstruction module f R .

Original: The residual learning paradigmallows us to learn the difference between noisy and clean images and helps information ﬂow withinthe network [He et al., 2015].
Revised: The residual learning paradigm allows us to learn the difference between noisy and clean images and helps information ﬂow within the network [He et al., 2015].

Original: Thus the ﬁnal denoised image x out ∈ R h × w × c in is obtained asx out = x in + f R ( F H , F D ) .
Revised: Thus the ﬁnal denoised image x out ∈ R h × w × c in is obtained as x out = x in + f R ( F H , F D ) .

Original: The speciﬁc architecture of the reconstruction network is depicted in
Revised: The speciﬁc architecture of the reconstruction network is depicted in

Original: Figure 4.
Revised: Figure 4.

Original: 2 Multi-scale Hybrid Feature Extraction via 
Revised: 4.2 Multi-scale Hybrid Feature Extraction via MUST

Original: The key component to our architecture is MUST, a multi-scale hybrid encoder-decoder architecturethat performs deep feature extraction in both image and token representation (Figure 1, bottom).
Revised: The key component to our architecture is MUST, a multi-scale hybrid encoder-decoder architecture that performs deep feature extraction in both image and token representation (Figure 1, bottom).

Original: First,individual pixels of the input representation of shape h 2 × w 2 ×
Revised: First, individual pixels of the input representation of shape h 2 × w 2 ×

Original: d are ﬂattened and passed through alearned linear mapping to yield h 2 · w 2 tokens of dimension d .
Revised: d are ﬂattened and passed through a learned linear mapping to yield h 2 · w 2 tokens of dimension d .

Original: Tokens corresponding to different imagepatches are subsequently merged in the encoder path, resulting in a concise latent representation.
Revised: Tokens corresponding to different image patches are subsequently merged in the encoder path, resulting in a concise latent representation.

Original: Thishighly descriptive representation is passed through a bottleneck block and progressively expanded bycombining tokens from the encoder path via skip connections.
Revised: This highly descriptive representation is passed through a bottleneck block and progressively expanded by combining tokens from the encoder path via skip connections.

Original: The ﬁnal output is rearranged to matchthe exact shape of the input low-resolution features F L , yielding a deep feature representation F D .
Revised: The ﬁnal output is rearranged to match the exact shape of the input low-resolution features F L , yielding a deep feature representation F D .

Original: Our design is inspired by the success of Residual Swin Transformer Blocks (RSTB) in imagedenoising and super-resolution [Liang et al., 2021].
Revised: Our design is inspired by the success of Residual Swin Transformer Blocks (RSTB) in image denoising and super-resolution [Liang et al., 2021].

Original: RSTB features a stack of Swin Transformerlayers (STL) that operate on tokens via a windowed self-attention mechanism [Liu et al., 2021],followed by convolution in image representation.
Revised: RSTB features a stack of Swin Transformer layers (STL) that operate on tokens via a windowed self-attention mechanism [Liu et al., 2021], followed by convolution in image representation.

Original: However, RSTB blocks operate on a single scale,therefore they cannot be readily applied in a hierarchical encoder-decoder architecture.
Revised: However, RSTB blocks operate on a single scale, therefore they cannot be readily applied in a hierarchical encoder-decoder architecture.

Original: Therefore,we design three variations of RSTB to facilitate multi-scale processing as depicted in Figure 2.
Revised: Therefore, we design three variations of RSTB to facilitate multi-scale processing as depicted in Figure 2.

Original: RSTB-B is the bottleneck block responsible for processing the encoded latent representation whilemaintaining feature dimensions.
Revised: RSTB-B is the bottleneck block responsible for processing the encoded latent representation while maintaining feature dimensions.

Original: Thus, we keep the default RSTB architecture for our bottleneckblock, which already operates on a single scale.
Revised: Thus, we keep the default RSTB architecture for our bottleneck block, which already operates on a single scale.

Original: RSTB-D has a similar function to convolutional downsampling blocks in U-Nets, but it operates onembedded tokens.
Revised: RSTB-D has a similar function to convolutional downsampling blocks in U-Nets, but it operates on embedded tokens.

Original: Given an input with size h i · w i × d , we pass it through an RSTB-B block and apply
Revised: Given an input with size h i · w i × d , we pass it through an RSTB-B block and apply

Original: PatchMerge operation.
Revised: PatchMerge operation.

Original: PatchMerge linearly combines tokens corresponding to 2 × 2 non-overlappingimage patches, while simultaneously increasing the embedding dimension (see Figure 3, top and
Revised: PatchMerge linearly combines tokens corresponding to 2 × 2 non-overlapping image patches, while simultaneously increasing the embedding dimension (see Figure 3, top and

Original: Appendix H for more details) resulting in an output of size h i 2 · w i 2 × 2 · d .
Revised: Appendix I for more details) resulting in an output of size h i 2 · w i 2 × 2 · d .

Original: Furthermore, RSTB-Doutputs the higher dimensional representation before patch merging to be subsequently used in thedecoder path via skip connection.
Revised: Furthermore, RSTB-D outputs the higher dimensional representation before patch merging to be subsequently used in the decoder path via skip connection.

Original: RSTB-U used in the decoder path is analogous to convolutional upsampling blocks.
Revised: RSTB-U used in the decoder path is analogous to convolutional upsampling blocks.

Original: An input withsize h i · w i × d is ﬁrst expanded into a larger number of lower dimensional tokens through a linearmapping via PatchExpand (see Figure 3, bottom and Appendix H for more details).
Revised: An input with size h i · w i × d is ﬁrst expanded into a larger number of lower dimensional tokens through a linear mapping via PatchExpand (see Figure 3, bottom and Appendix I for more details).

Original: PatchExpandreverses the effect of PatchMerge on feature size, thus resulting in 2 h i · 2 w i tokens of dimension d 2 .
Revised: PatchExpand reverses the effect of PatchMerge on feature size, thus resulting in 2 h i · 2 w i tokens of dimension d 2 .

Original: Next, we mix information from the obtained expanded tokens with skip embeddings from higherscales via TokenMix .
Revised: Next, we mix information from the obtained expanded tokens with skip embeddings from higher scales via TokenMix .

Original: This operation linearly combines tokens from both paths and normalizes theresulting vectors.
Revised: This operation linearly combines tokens from both paths and normalizes the resulting vectors.

Original: Finally, the mixed tokens are processed by an RSTB-B block.
Revised: Finally, the mixed tokens are processed by an RSTB-B block.

Original: 4.3 Iterative Unrolling
Revised: 4.3 Iterative Unrolling

Original: Architectures derived from unrolling the iterations of various optimization algorithms have proven tobe successful in tackling various inverse problems including MRI reconstruction.
Revised: Architectures derived from unrolling the iterations of various optimization algorithms have proven to be successful in tackling various inverse problems including MRI reconstruction.

Original: These architecturecan be interpreted as a cascade of simpler denoisers, each of which progressively reﬁnes the estimatefrom the preceding unrolled iteration.
Revised: These architecture can be interpreted as a cascade of simpler denoisers, each of which progressively reﬁnes the estimate from the preceding unrolled iteration (see more details in Appendix F).

Original: Following Sriram et al.
Revised: Following Sriram et al.

Original: [2020], we unroll the gradient descent iterations of the inverse problem in (2.1)in k-space domain, yielding the iterative update scheme in (2.2).
Revised: [2020], we unroll the gradient descent iterations of the inverse problem in (2.1) in k-space domain, yielding the iterative update scheme in (2.2).

Original: We apply regularization in image 
Revised: We apply regularization in image domain via our proposed HUMUS-Block, that is we have G ( k ) = F (cid:0) E (cid:0) D (cid:0) R (cid:0) F − 1 ( k ) (cid:1)(cid:1)(cid:1)(cid:1) , where

Original: 4.4 Adjacent Slice Reconstruction (ASR)
Revised: D denotes the HUMUS-Block, R ( x 1 , ..., x N ) = (cid:80) Ni =1 S ∗ i x

Revised: i is the reduce operator that combines coil images via the corresponding sensitivity maps and E ( x ) =

Revised: ( S 1 x , ..., S N x ) is the expand operator that maps the combined image back to individual coil images.

Revised: The sensitivity maps can be estimated a priori using methods such as ESPIRiT [Uecker et al., 2014] or learned in an end-to-end fashion via a Sensitivity Map Estimator (SME) network proposed in Sriram et al.

Revised: In this work we aspire to design an end-to-end approach and thus we use the latter method and estimate the sensitivity maps the low-frequency (ACS) region of the undersampled input measurements a standard U- network.

Revised: 4.4 Adjacent Slice Reconstruction (ASR)

Original: We observe improvements in reconstruction quality when instead of processing the undersampleddata slice-by-slice, we jointly reconstruct a set of adjacent slices via HUMUS-Net
Revised: We observe improvements in reconstruction quality when instead of processing the undersampled data slice-by-slice, we jointly reconstruct a set of adjacent slices via HUMUS-Net

Original: (Figure 5).
Revised: (Figure 5).

Original: That is,if we have a volume of undersampled data ˜ k vol =
Revised: That is, if we have a volume of undersampled data ˜ k vol =

Original: (˜ k 1 , ..., ˜ k K ) with K slices, when reconstructingslice c , we instead reconstruct the volume (˜ k c − a , ..., ˜ k
Revised: (˜ k 1 , ..., ˜ k K ) with K slices, when reconstructing slice c , we instead reconstruct the volume (˜ k c − a , ..., ˜ k

Original: c − 1 , ˜ k c , ˜ k
Revised: c − 1 , ˜ k c , ˜ k

Original: c +1 , ..., ˜ k
Revised: c +1 , ..., ˜ k

Original: c + a ) by concatenating themalong the coil channel dimension, where a denotes the number of adjacent slices added on each side.
Revised: c + a ) by concatenating them along the coil channel dimension, where a denotes the number of adjacent slices added on each side.

Original: However, we only calculate and backpropagate the loss on the center slice c of the reconstructedvolume.
Revised: However, we only calculate and backpropagate the loss on the center slice c of the reconstructed volume.

Original: The beneﬁt of ASR is that the network can remove artifacts corrupting individual slices as itsees a larger context of the slice by observing its neighbors.
Revised: The beneﬁt of ASR is that the network can remove artifacts corrupting individual slices as it sees a larger context of the slice by observing its neighbors.

Original: Even though ASR increases computecost, it is important to note that it does not impact the number of token embeddings (spatial resolutionis unchanged) and thus can be combined favorably with Transformer-based methods.
Revised: Even though ASR increases compute cost, it is important to note that it does not impact the number of token embeddings (spatial resolution is unchanged) and thus can be combined favorably with Transformer-based methods.

Original:  Experiments
Revised:  Experiments

Original: In this section we provide experimental results on our proposed architecture, HUMUS-Net.
Revised: In this section we provide experimental results on our proposed architecture, HUMUS-Net.

Original: First,we demonstrate the reconstruction performance of our model on various datasets, including thelarge-scale fastMRI dataset.
Revised: First, we demonstrate the reconstruction performance of our model on various datasets, including the large-scale fastMRI dataset.

Original: Then, we justify our design choices through a set of ablation studies.
Revised: Then, we justify our design choices through a set of ablation studies.

Original: Our source code is available in the supplementary.

Original: 5.1 Benchmark Experiments
Revised: 5.1 Benchmark Experiments

Original: We investigate the performance of HUMUS-Net on three different datasets.
Revised: We investigate the performance of HUMUS-Net on three different datasets.

Original: We use the structuralsimilarity index measure (SSIM)
Revised: We use the structural similarity index measure (SSIM)

Original: [Wang et al., 2003] as a basis of our evaluation, which is the mostcommon evaluation metric in medical image reconstruction.
Revised: [Wang et al., 2003] as a basis of our evaluation, which is the most common evaluation metric in medical image reconstruction.

Original: In all of our experiments, we follow thesetup of the fastMRI multi-coil knee track with 8 × acceleration in order to provide comparison withstate-of-the-art networks of the Public Leaderboard.
Revised: In all of our experiments, we follow the setup of the fastMRI multi-coil knee track with 8 × acceleration in order to provide comparison with state-of-the-art networks of the Public Leaderboard.

Original: That is, we perform retrospective undersamplingof the fully-sampled k-space data by randomly subsampling 12 .
Revised: That is, we perform retrospective undersampling of the fully-sampled k-space data by randomly subsampling 12 .

Revised: 5% of whole k-space lines in the

Revised: [Table]

Original: 5% of whole k-space lines in thephase encoding direction, keeping 4% of lowest frequency adjacent k-space lines.
Revised: [Table] For datasets with multiple trainvalidation split runs we show the mean and standard error of the runs. [Table] phase encoding direction, keeping 4% of lowest frequency adjacent k-space lines.

Original:  During training, wegenerate random masks following the above method, whereas for the validation dataset we keep themasks ﬁxed for each k-space volume.
Revised: Experiments on other acceleration ratios can be found in Appendix C. During training, we generate random masks following the above method, whereas for the validation dataset we keep the masks ﬁxed for each k-space volume.

Original: For HUMUS-Net, we center crop and pad inputs to 384 × 384 .
Revised: For HUMUS-Net, we center crop and pad inputs to 384 × 384 .

Original: We compare the reconstruction quality of our proposed model with the current best performingnetwork, E2E-VarNet.
Revised: We compare the reconstruction quality of our proposed model with the current best performing network, E2E-VarNet.

Original: For details on HUMUS-Net hyperparameters and training, we refer the readerto the supplementary.
Revised: For details on HUMUS-Net hyperparameters and training, we refer the reader to Appendix A. For

Original: For E2E-VarNet, we use the hyperparameters speciﬁed in Sriram et al.
Revised: E2E-VarNet, we use the hyperparameters speciﬁed in Sriram et al.

Original: fastMRI 
Revised: fastMRI –

Original: – The fastMRI dataset [Zbontar et al., 2019] is the largest publicly available MRI datasetwith competitive baseline models and a public leaderboard, and thus provides an opportunity todirectly compare different algorithms.
Revised: The fastMRI dataset [Zbontar et al., 2019] is the largest publicly available MRI dataset with competitive baseline models and a public leaderboard, and thus provides an opportunity to directly compare different algorithms.

Original: Speciﬁcally, we run experiments on the multi-coil knee dataset,consisting of close to 35 k slices in 973 volumes.
Revised: Speciﬁcally, we run experiments on the multi-coil knee dataset, consisting of close to 35 k slices in 973 volumes.

Original: We use the default HUMUS-Net model deﬁned [Table] [Table]
Revised: We use the default HUMUS-Net model deﬁned above with 3 adjacent slices as input.

Original: [Table]
Revised: Furthermore, we design a large variant of our model,

Original: above with 3 adjacent slices as input.
Revised: HUMUS-Net-L , which has increased embedding dimension compared to the default model (see details in Appendix A. We train models both only on the training split, and also on the training and validation splits combined (additional ≈ 20% data) for the leaderboard.

Original: Table 1 demonstrates our results compared to the best publishedmodels from the fastMRI Leaderboard 2 evaluated on the test dataset.
Revised: Table 1 demonstrates our results compared to the best published models from the fastMRI Leaderboard 3 evaluated on the test dataset.

Original: Our model establishes new stateof the art in terms of SSIM on this dataset by a large margin, and achieves comparable or slightlybetter performance than other methods in terms of PSNR and NMSE.
Revised: Our model establishes new state of the art in terms of SSIM on this dataset by a large margin, and achieves comparable or better performance than other methods in terms of PSNR and NMSE.

Original: Moreover, as seen in the secondcolumn of Table 2, we evaluated our model on the fastMRI validation dataset as well and comparedour results to E2E-VarNet, the best performing model from the leaderboard.
Revised: Moreover, as seen in the second column of Table 2, we evaluated our model on the fastMRI validation dataset as well and compared our results to E2E-VarNet, the best performing model from the leaderboard.

Original: We observe similarimprovements in terms of the reconstruction SSIM metric to the test dataset.
Revised: We observe similar improvements in terms of the reconstruction SSIM metric to the test dataset.

Original: Visual inspection ofreconstructions shows that HUMUS-Net recovers very ﬁne details in images that may be missed byother state-of-the-art reconstruction algorithms (see attached ﬁgures in the supplementary).
Revised: Visual inspection of reconstructions shows that HUMUS-Net recovers very ﬁne details in images that may be missed by other state-of-the-art reconstruction algorithms (see attached ﬁgures in Appendix I). Comparison based on further image quality metrics can be found in Appendix

Revised: H. We point out that even though our model has more parameters than E2E-VarNet, our proposed image-domain denoiser is more efﬁcient than the U-Net deployed in E2E-VarNet, even when their number of parameters are matched as discussed in Section 5.3.

Revised: Overall, larger model size does not necessarily correlate with better performance, as seen in other competitive models in Table 1.

Revised: Finally, we note that the additional training data (in the form of the validation split) provides a consistent small boost to model performance.

Revised: We refer the reader to Klug and Heckel [2022] for an overview of scaling properties of reconstruction models.

Original: Stanford 2D – Next, we run experiments on the Stanford2D FSE [Cheng] dataset, a publicly available
Revised: Stanford 2D – Next, we run experiments on the Stanford2D FSE [Cheng] dataset, a publicly available

Original: MRI dataset consisting of scans from various anatomies (pelvis, lower extremity and more) in 89fully-sampled volumes.
Revised: MRI dataset consisting of scans from various anatomies (pelvis, lower extremity and more) infully-sampled volumes.

Original: We randomly sample 80% of volumes as train data and use the rest forvalidation.
Revised: We randomly sample 80% of volumes as train data and use the rest for validation.

Original: We randomly generate 3 different train-validation splits this way to reduce variations in thepresented metrics.
Revised: We randomly generate 3 different train-validation splits this way to reduce variations in the presented metrics.

Original: As slices in this dataset have widely varying shapes across volumes, we center cropthe target images to keep spatial resolution within 384 × 384 .
Revised: As slices in this dataset have widely varying shapes across volumes, we center crop the target images to keep spatial resolution within 384 × 384 .

Original: We use the default HUMUS-Net deﬁnedabove with single slices as input.
Revised: We use the default HUMUS-Net deﬁned above with single slices as input.

Original: Our results comparing the best performing MRI reconstructionmodel with HUMUS-Net is shown in the third column of Table 2.
Revised: Our results comparing the best performing MRI reconstruction model with HUMUS-Net is shown in the third column of Table 2.

Original: We present the mean SSIM ofall runs along with the standard error.
Revised: We present the mean SSIM of all runs along with the standard error.

Original: We achieve improvements of similar magnitude as on thefastMRI dataset.
Revised: We achieve improvements of similar magnitude as on the fastMRI dataset.

Original: These results demonstrate the effectiveness of HUMUS-Net on a more diversedataset featuring multiple anatomies.
Revised: These results demonstrate the effectiveness of HUMUS-Net on a more diverse dataset featuring multiple anatomies.

Original: Stanford 3D – Finally, we evaluate our model on the Stanford Fullysampled 3D FSE Knees dataset[Sawyer et al., 2013], a public MRI dataset including 20 volumes of knee MRI scans.
Revised: Stanford 3D – Finally, we evaluate our model on the Stanford Fullysampled 3D FSE Knees dataset [Sawyer et al., 2013], a public MRI dataset including 20 volumes of knee MRI scans.

Original: We generatetrain-validation splits using the method described for Stanford 2D and perform 3 runs.
Revised: We generate train-validation splits using the method described for Stanford 2D and perform 3 runs.

Original: We use thedefault HUMUS-Net network with single slices as input.
Revised: We use the default HUMUS-Net network with single slices as input.

Original: The last column of Table 2 comparesour results to E2E-VarNet, showing improvements of similar scales as on other datasets we haveinvestigated in this work.
Revised: The last column of Table 2 compares our results to E2E-VarNet, showing improvements of similar scales as on other datasets we have investigated in this work.

Original: This experiment demonstrates that HUMUS-Net performs well not only onlarge-scale MRI datasets, but also on smaller problems.
Revised: This experiment demonstrates that HUMUS-Net performs well not only on large-scale MRI datasets, but also on smaller problems.

Original: 5.2 Ablation Studies
Revised: 5.2 Ablation Studies

Original: In this section, we motivate our design choices through a set of ablation studies.
Revised: In this section, we motivate our design choices through a set of ablation studies.

Original: We start from
Revised: We start from

Original: SwinIR, a general image reconstruction network, and highlight its weaknesses for MRI.
Revised: SwinIR, a general image reconstruction network, and highlight its weaknesses for MRI.

Original: Then, wedemonstrate step-by-step how we addressed these shortcomings and arrived at the HUMUS-Netarchitecture.
Revised: Then, we demonstrate step-by-step how we addressed these shortcomings and arrived at the HUMUS-Net architecture.

Original: We train the models on the Stanford 3D dataset.
Revised: We train the models on the Stanford 3D dataset.

Original: More details can be found in thesupplementary.
Revised: More details can be found in Appendix

Original:  The results of our ablation studies are summarized in Table 3.
Revised: B. The results of our ablation studies are summarized in Table 3.

Original: First, we investigate SwinIR, a state-of-the-art image denoiser and super-resolution model thatfeatures a hybrid Transformer-convolutional architecture.
Revised: First, we investigate SwinIR, a state-of-the-art image denoiser and super-resolution model that features a hybrid Transformer-convolutional architecture.

Original: In order to handle the 10 × larger inputsizes ( 320 × 320 ) in our MRI dataset compared to the input images this network has been designedfor ( 128 × 128 ), we reduce the embedding dimension of SwinIR to ﬁt into GPU memory (16 GB).
Revised: In order to handle the 10 × larger input sizes ( 320 × 320 ) in our MRI dataset compared to the input images this network has been designed for ( 128 × 128 ), we reduce the embedding dimension of SwinIR to ﬁt into GPU memory (16 GB).

Original: We ﬁnd that compared to models designed for MRI reconstruction, such as E2E-VarNet (last row in
Revised: We ﬁnd that compared to models designed for MRI reconstruction, such as E2E-VarNet (last row in

Original: Table 3) SwinIR performs poorly.
Revised: Table 3) SwinIR performs poorly.

Original: This is not only due to the reduced network size, but also due tothe fact that SwinIR is not speciﬁcally designed to take the MRI forward model into consideration.
Revised: This is not only due to the reduced network size, but also due to the fact that SwinIR is not speciﬁcally designed to take the MRI forward model into consideration.

Original: Next, we unroll SwinIR and add a sensitivity map estimator.
Revised: Next, we unroll SwinIR and add a sensitivity map estimator.

Original: We refer to this model as Un-SS .
Revised: We refer to this model as Un-SS .

Original: Due tounrolling, we have to further reduce the embedding dimension of the denoiser and also decrease thedepth of the network in order to ﬁt into GPU memory.
Revised: Due to unrolling, we have to further reduce the embedding dimension of the denoiser and also decrease the depth of the network in order to ﬁt into GPU memory.

Original: Un-SS , due to its small size, performs slightlyworse than vanilla SwinIR and signiﬁcantly lags behind the E2E-VarNet architectures.
Revised: Un-SS , due to its small size, performs slightly worse than vanilla SwinIR and signiﬁcantly lags behind the E2E-VarNet architectures.

Original: We note that
Revised: We note that

Original: SwinIR operates over a single, full-resolution scale, whereas state-of-the-art MRI reconstructionmodels typically incorporate multi-scale processing in the form of U-Net-like architectures.
Revised: SwinIR operates over a single, full-resolution scale, whereas state-of-the-art MRI reconstruction models typically incorporate multi-scale processing in the form of U-Net-like architectures.

Original: Thus, we replace SwinIR by MUST, our proposed multi-scale hybrid processing unit, but keep theembedding dimension in the largest-resolution scale ﬁxed.
Revised: Thus, we replace SwinIR by MUST, our proposed multi-scale hybrid processing unit, but keep the embedding dimension in the largest-resolution scale ﬁxed.

Original: The obtained network, which we call
Revised: The obtained network, which we call

Original: Un-MS , has overall lower computational cost when compared with Un-SS , however as Table 3 shows
Revised: Un-MS , has overall lower computational cost when compared with Un-SS , however as Table 3 shows

Original: MRI reconstruction performance has signiﬁcantly improved compared to both Un-SS and vanilla
Revised: MRI reconstruction performance has signiﬁcantly improved compared to both Un-SS and vanilla

Original: SwinIR, which highlights the efﬁciency of our proposed multi-scale feature extractor.
Revised: SwinIR, which highlights the efﬁciency of our proposed multi-scale feature extractor.

Original: Reconstructionperformance is limited by the low dimension of patch embeddings, which we are unable to increasefurther due to our compute and memory constraints originating in the high-resolution inputs.
Revised: Reconstruction performance is limited by the low dimension of patch embeddings, which we are unable to increase further due to our compute and memory constraints originating in the high-resolution inputs.

Original: The most straightforward approach to tackle the challenge of high input resolution is to increase thepatch size.
Revised: The most straightforward approach to tackle the challenge of high input resolution is to increase the patch size.

Original: To test this idea, we take Un-MS and embed 2 × 2 patches of the inputs, thus reducing thenumber of tokens processed by the network by a factor of 4 .
Revised: To test this idea, we take Un-MS and embed 2 × 2 patches of the inputs, thus reducing the number of tokens processed by the network by a factor of 4 .

Original: We refer to this model as Un-MS-Patch2 .
Revised: We refer to this model as Un-MS-Patch2 .

Original: This reduction in compute and memory load allows us to increase network capacity by increasing theembedding dimension 3 -folds (to ﬁll GPU memory again).
Revised: This reduction in compute and memory load allows us to increase network capacity by increasing the embedding dimension 3 -folds (to ﬁll GPU memory again).

Original: However, Un-MS-Patch2 performs muchworse than previous models using patch size of 1 .
Revised: However, Un-MS-Patch2 performs much worse than previous models using patch size of 1 .

Original: For classiﬁcation problems, where the generalimage context is more important than small details, patches of 16 × 16 or 8 × 8 are considered typical[Dosovitskiy et al., 2020].
Revised: For classiﬁcation problems, where the general image context is more important than small details, patches of 16 × 16 or 8 × 8 are considered typical [Dosovitskiy et al., 2020].

Original: Even for more dense prediction tasks such as medical image segmentation,patch size of 4 × 4 have been used successfully Cao et al. [2021a].
Revised: Even for more dense prediction tasks such as medical image segmentation, patch size of 4 × 4 has been used successfully [Cao et al., 2021a].

Original: However, our experiments suggestthat in low-level vision tasks such as MRI reconstruction using patches larger than 1 × 1 may bedetrimental due to loss of crucial high-frequency detail information.
Revised: However, our experiments suggest that in low-level vision tasks such as MRI reconstruction using patches larger than 1 × 1 may be detrimental due to loss of crucial high-frequency detail information.

Original: Our approach to address the heavy computational load of Transformers for large input resolutionswhere increasing the patch size is not an option is to process lower resolution features extracted viaconvolutions.
Revised: Our approach to address the heavy computational load of Transformers for large input resolutions where increasing the patch size is not an option is to process lower resolution features extracted via convolutions.

Original: That is we replace MUST in Un-MS by a HUMUS-Block, resulting in our proposed
Revised: That is we replace MUST in Un-MS by a HUMUS-Block, resulting in our proposed

Original: HUMUS-Net architecture.
Revised: HUMUS-Net architecture.

Original: We train a smaller version of HUMUS-Net with the same embeddingdimension as Un-MS .
Revised: We train a smaller version of HUMUS-Net with the same embedding dimension as Un-MS .

Original: As seen in Table 3, our model achieves the best performance across all otherproposed solutions, even surpassing E2E-VarNet.
Revised: As seen in Table 3, our model achieves the best performance across all other proposed solutions, even surpassing E2E-VarNet.

Original: This series of incremental studies highlights theimportance of each architectural design choice leading to our proposed HUMUS-Net architecture.
Revised: This series of incremental studies highlights the importance of each architectural design choice leading to our proposed HUMUS-Net architecture.

Original: Conclusion
Revised: 5.3 Direct comparison of image-domain denoisers

Revised: In order to further demonstrate the advantage of HUMUS-Net over E2E-VarNet, we provide direct comparison between the image-domain denoisers used in the above methods. E2E-VarNet unrolls a fully convolutional U-Net, whereas we deploy our hybrid HUMUS-Block architecture as a denoiser (Fig. 1).

Original: In this paper, we introduce HUMUS-Net, an unrolled, Transformer-convolutional hybrid networkfor accelerated MRI reconstruction.
Revised: We scale down the HUMUS-Block used in HUMUS-Net to match the size of the U-Net in E2E-Varnet in terms of number of model parameters. To this end, we reduce the embedding dimension from 66 to 30 . We train both networks on magnitude images from the Stanford3D dataset.

Original: HUMUS-Net achieves state-of-the-art performance on thefastMRI dataset and greatly outperforms all previous published and reproducible methods. Wedemonstrate the performance of our proposed method on two other MRI datasets and perform ﬁnegrained ablation studies to motivate our design choices and emphasize the compute and memorychallenges of Transformer-based architectures on low-level and dense computer vision tasks suchas MRI reconstruction.
Revised: The results are summarized in Table 4. We observe that given a ﬁxed parameter budget, our proposed denoiser outperforms the widely used convolutional U-Net architecture in MRI reconstruction, further demonstarting the efﬁciency of HUMUS-Net. This experiment suggests that our HUMUS-Block could serve as an excellent denoiser, replacing convolutional U-Nets, in a broad range of image restoration applications outside of MRI, which we leave for future work.

Original: A limitation of our current architecture is that it requires ﬁxed-size inputs,which we intend to address with a more ﬂexible design in the future. This work opens the door for theadoption of a multitude of promising techniques introduced recently in the literature for Transformers,which we leave for future work.
Revised: [Table]

Original:  Checklist
Revised:  Conclusion

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes]

Original: (b) Did you describe the limitations of your work?

Original: [Yes] See Conclusions.

Original: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] We were

Original: unable to identify possible negative effects on society.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?

Original: [Yes]

Original: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results? [N/A] (b) Did you include complete proofs of all theoretical results? [N/A]
Revised: In this paper, we introduce HUMUS-Net, an unrolled, Transformer-convolutional hybrid network for accelerated MRI reconstruction.

Original: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes] Full code is available in the supplementary.
Revised: HUMUS-Net achieves state-of-the-art performance on the fastMRI dataset and greatly outperforms all previous published and reproducible methods.

Original: Source code will be published upon paper acceptance.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)? [Yes] See Supplementary (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)? [Yes] We report the standard error ofruns in our tables.
Revised: We demonstrate the performance of our proposed method on two other MRI datasets and perform ﬁnegrained ablation studies to motivate our design choices and emphasize the compute and memory challenges of Transformer-based architectures on low-level and dense computer vision tasks such as MRI reconstruction.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Supplementary.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators? [Yes] (b) Did you mention the license of  the assets?
Revised: A limitation of our current architecture is that it requires ﬁxed-size inputs, which we intend to address with a more ﬂexible design in the future.

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL? [Yes] We attach source code in the Supplementary.
Revised: This work opens the door for the adoption of a multitude of promising techniques introduced recently in the literature for Transformers, which we leave for future work.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating? [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable
Revised: Acknowledgments

Original: information or offensive content? [N/A] If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: M. Soltanolkotabi is supported by the Packard Fellowship in Science and Engineering, a Sloan

Original: applicable?

Original: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: Research Fellowship in Mathematics, an NSF-CAREER under award #1846369, DARPA Learning with Less Labels (LwLL) and FastNICS programs, and NSF-CIF awards #1813877 and #2008443.

Original: spent on participant compensation?

Original: [N/A]

Original: AppendixA HUMUS-Net baseline details
Revised: Appendix A HUMUS-Net baseline details

Original: Our default model has 3 RSTB-D downsampling blocks, 2 RSTB-B bottleneck blocks and 3 RSTB-Uupsampling blocks with 3 − 6 − 12 attention heads in the D/U blocks and 24 attention heads inthe bottleneck block.
Revised: Our default model has 3 RSTB-D downsampling blocks, 2 RSTB-B bottleneck blocks and 3 RSTB-U upsampling blocks with 3 − 6 − 12 attention heads in the D/U blocks and 24 attention heads in the bottleneck block.

Original: For Swin Transformers layers, the window size is 8 for all methods and MLPratio ( hidden _ dim/input _ dim ) of 2 is used.
Revised: For Swin Transformers layers, the window size is 8 for all methods and MLP ratio ( hidden _ dim/input _ dim ) of 2 is used.

Original: Each RSTB block consists of 2 STLs with embeddingdimension of 66 .
Revised: Each RSTB block consists of 2 STLs with embedding dimension of 66 .

Revised: For HUMUS-Net-L, we increase the embedding dimension to 96 .

Original: We use 8 cascades of unrolling with a U-Net as sensitivity map estimator (same asin E2E-VarNet) with 16 channels.
Revised: We use 8 cascades of unrolling with a U-Net as sensitivity map estimator (same as in E2E-VarNet) with 16 channels.

Original: We center crop and reﬂection pad the input images to 384 × 384 resolution for HUMUS-Net and use the complete images for VarNet.
Revised: We center crop and reﬂection pad the input images to 384 × 384 resolution for HUMUS-Net and use the complete images for VarNet.

Original: In all experiments, we minimize the SSIM loss between the target image x ∗ and the reconstruction ˆ x deﬁned as
Revised: In all experiments, we minimize the SSIM loss between the target image x ∗ and the reconstruction ˆ x deﬁned as

Original: [Equation]
Revised: [Equation]

Original: fastMRI experiments– We train HUMUS-Net using Adam for 50 epochs with a learning rate of0 . 0001 , dropped by a factor of 10 at epoch 40 and apply adjacent slice reconstruction with 3 slices.
Revised: fastMRI experiments– We train HUMUS-Net using Adam for 50 epochs with a learning rate of 0 . 0001 , dropped by a factor of 10 at epoch 40 and apply adjacent slice reconstruction with 3 slices.

Original: We run the experiments on 8 × Titan RTX 5000 24GB GPUs with a per-GPU batch size of 1 . 
Revised: We run the experiments on 8 × Titan RTX 5000 24GB GPUs with a per-GPU batch size of 1 . We train

Revised: HUMUS-Net-L using a learning rate of 0 .

Revised: 00007 (using inverse square root scaling with embedding dimension) for 45 epochs, dropped by a factor of 10 for further 5 epochs.

Revised: We trained HUMUS-Net-L on 4 × A100 40GB GPUs on Amazon SageMaker with per-GPU batch size of 1.

Original: Stanford 3D experiments– We train for 25 epochs with a learning rate of 0 . 0001 , dropped by afactor of 10 at epoch 20 and reconstruct single slices.
Revised: Stanford 3D experiments– We train for 25 epochs with a learning rate of 0 . 0001 , dropped by a factor of 10 at epoch 20 and reconstruct single slices.

Original: We run the experiments on 4 × Quadro RTX5000 16GB GPUs with a per-GPU batch size of 1 .
Revised: We run the experiments on 4 × Quadro RTX 5000 16GB GPUs with a per-GPU batch size of 1 .

Original: Stanford 2D experiments– We train for 50 epochs with a learning rate of 0 . 0001 , dropped by afactor of 10 at epoch 40 and reconstruct single slices.
Revised: Stanford 2D experiments– We train for 50 epochs with a learning rate of 0 . 0001 , dropped by a factor of 10 at epoch 40 and reconstruct single slices.

Original: Moreover, we crop reconstruction targets to ﬁtinto a 384 × 384 box.
Revised: Moreover, we crop reconstruction targets to ﬁt into a 384 × 384 box.

Original: We run the experiments on 8 × Quadro RTX 5000 16GB GPUs with a per-GPUbatch size of 1 .
Revised: We run the experiments on 8 × Quadro RTX 5000 16GB GPUs with a per-GPU batch size of 1 .

Original: B Ablation study experimental details
Revised: B Ablation study experimental details

Original: Here we discuss the experimental setting and hyperparameters used in our ablation study in Section5.2.
Revised: Here we discuss the experimental setting and hyperparameters used in our ablation study in Section 5.2.

Original: In all experiments, we train the models on the Stanford 3D dataset for 3 different train-validationsplits and report the mean and standard error of the results.
Revised: In all experiments, we train the models on the Stanford 3D dataset for 3 different train-validation splits and report the mean and standard error of the results.

Original: We use Adam optimizer with learningrate 0 . 0001 and train for 25 epochs, decaying the learning rate by a factor of 10 at 20 epochs.
Revised: We use Adam optimizer with learning rate 0 . 0001 and train for 25 epochs, decaying the learning rate by a factor of 10 at 20 epochs.

Original: Forall unrolled methods, we unroll 12 iterations in order to provide direct comparison with the bestperforming E2E-VarNet, which uses the same number of cascades.
Revised: For all unrolled methods, we unroll 12 iterations in order to provide direct comparison with the best performing E2E-VarNet, which uses the same number of cascades.

Original: We reconstruct single slices anddo not use the method of adjacent slice reconstruction discussed in 4.4.
Revised: We reconstruct single slices and do not use the method of adjacent slice reconstruction discussed in 4.4.

Original: In models with sensitivitymap estimator,we used the default U-Net with 8 channels (same as default E2E-VarNet).
Revised: In models with sensitivity map estimator,we used the default U-Net with 8 channels (same as default E2E-VarNet).

Original: For Swin
Revised: For Swin

Original: Transformers layers, the window size is 8 for all methods and MLP ratio ( hidden _ dim/input _ dim )of 2 is used.
Revised: Transformers layers, the window size is 8 for all methods and MLP ratio ( hidden _ dim/input _ dim ) of 2 is used.

Original: Further hyperparameters are summarized in Table4.
Revised: Further hyperparameters are summarized in Table

Original: [Table]
Revised: [Table]

Revised: networks

Revised: downsampling

Original:  We run the experiments on 4 × Quadro RTX 5000 16GB GPUs 
Revised: B upsampling separately We run the experiments on 4 × Quadro RTX 5000 16GB GPUs with a per-GPU batch size of 1 .

Revised: C Results on additional accelerations

Revised: We observe that HUMUS-Net achieves state-of-the-art performance across a wide range of acceleration ratios.

Revised: We perform experiments on the Stanford 3D dataset using a small HUMUS-Net model with only 6 cascades, embedding dimension of 66 , adjacent slice reconstruction with 3 slices and for the sake of simplicity we removed the residual path from the HUMUS-Block.

Original: [Figure]
Revised: We call this model HUMUS-Net-S . We set the learning rate to 0 . 0002 and train the model with Adam optimizer. We generate 3 random training-validation splits on the Stanford 3D dataset (same splits as in other Stanford 3D experiments in this paper), and show the mean and standard error of the results in Table 6.

Revised: In our experiments, HUMUS-Net achieved higher quality reconstructions in every metric over

Original: C Number of unrolled iterations: ablation study
Revised: E2E-VarNet. [Table] D Effect of the number of unrolled iterations

Original: The number of cascades in unrolled networks has a fundamental impact on their performance.
Revised: The number of cascades in unrolled networks has a fundamental impact on their performance.

Original: Deeper networks typically perform better, but also incur heavy computational and memory cost.
Revised: Deeper networks typically perform better, but also incur heavy computational and memory cost.

Original: Inthis experiment we investigate the scaling of HUMUS-Net with respect to the number of iterativeunrollings.
Revised: In this experiment we investigate the scaling of HUMUS-Net with respect to the number of iterative unrollings.

Original: We perform an ablation study on the Stanford 3D dataset with × 8 acceleration for a ﬁxed trainingvalidation split, where 20% of the training set has been set aside for validation.
Revised: We perform an ablation study on the Stanford 3D dataset with × 8 acceleration for a ﬁxed trainingvalidation split, where 20% of the training set has been set aside for validation.

Original: We reconstruct singleslices, without applying adjacent slice reconstruction.
Revised: We reconstruct single slices, without applying adjacent slice reconstruction.

Original: We plot the highest SSIM throughout trainingon the validation dataset for each network on Figure 6.
Revised: We plot the highest SSIM throughout training on the validation dataset for each network on Figure 6.

Original: We observe improvements in reconstruction performance with increasing number of cascades, andthis improvement has not saturated yet in the range of model sizes we have investigated.
Revised: We observe improvements in reconstruction performance with increasing number of cascades, and this improvement has not saturated yet in the range of model sizes we have investigated.

Original: In ourmain experiments, we select 8 cascades due to memory and compute limitations for deeper models.
Revised: In our main experiments, we select 8 cascades due to memory and compute limitations for deeper models.

Original: However, our experiment suggests that HUMUS-Net can potentially obtain even better reconstructionresults given enough computational resources.
Revised: However, our experiment suggests that HUMUS-Net can potentially obtain even better reconstruction results given enough computational resources.

Original: D Adjacent slice reconstruction: ablation study
Revised: E Effect of Adjacent Slice Reconstruction

Original: In this section, we demonstrate that even though adjacent slice reconstruction is in general helpful, itis not the main reason why HUMUS-Net performs better than E2E-VarNet.
Revised: In this section, we demonstrate that even though adjacent slice reconstruction is in general helpful, it is not the main reason why HUMUS-Net performs better than E2E-VarNet.

Original: To this end, we add adjacent slice reconstruction to E2E-VarNet and investigate its effect on modelperformance.
Revised: To this end, we add adjacent slice reconstruction to E2E-VarNet and investigate its effect on model performance.

Original: We run experiments on the Stanford 3D dataset for 3 different random train-validationsplits.
Revised: We run experiments on the Stanford 3D dataset for 3 different random train-validation splits.

Original: We use adjacent slice reconstruction with 3 slices.
Revised: We use adjacent slice reconstruction with 3 slices.

Original: We have found that the default learningrate for E2E-VarNet is not optimal with the increased input size, therefore we tune the learning rateusing grid search and set it to 0 . 0005 .
Revised: We have found that the default learning rate for E2E-VarNet is not optimal with the increased input size, therefore we tune the learning rate using grid search and set it to 0 . 0005 .

Original: We compare the results with HUMUS-Net, where we matchthe number of cascades in E2E-VarNet, and use the default embedding dimension of 66 .
Revised: We compare the results with HUMUS-Net, where we match the number of cascades in E2E-VarNet, and use the default embedding dimension of 66 .

Original: We do notuse adjacent slice reconstruction when training HUMUS-Net in order to ablate its effect.
Revised: We do not use adjacent slice reconstruction when training HUMUS-Net in order to ablate its effect.

Original: The results are summarized in Table 5.
Revised: The results are summarized in Table 7.

Original: We observe that ASR boosts the reconstruction quality of
Revised: We observe that ASR boosts the reconstruction quality of

Original: E2E-VarNet.
Revised: E2E-VarNet.

Original: However, ASR alone cannot close the gap between the two models, as HUMUS-Netwithout ASR still outperforms the best E2E-VarNet model with ASR. 
Revised: However, ASR alone cannot close the gap between the two models, as HUMUS-Net without ASR still outperforms the best E2E-VarNet model with ASR. [Figure]

Original: [Table]
Revised: [Table]

Original: E Iterative denoising visualization
Revised: F Iterative denoising visualization

Original: Here, we provide more discussion on the iterative denoising interpretation of unrolled networks,such as HUMUS-Net.
Revised: Here [Table] we provide more discussion on the iterative denoising interpretation of unrolled networks, such as HUMUS-Net.

Original: Consider the regularized inverse problem formulation of MRI reconstructionin (2.1) and design the architecture based on unrolling the gradient descent steps on the aboveproblem.
Revised: Consider the regularized inverse problem formulation of MRI reconstruction in (2.1) and design the architecture based on unrolling the gradient descent steps on the above problem.

Original: Applying data consistency corresponds to the gradient of the ﬁrst term in (2.2), whereas welearn the gradient of the regularizer parameterized by the HUMUS-Block architecture.
Revised: Applying data consistency corresponds to the gradient of the ﬁrst term in (2.2), whereas we learn the gradient of the regularizer parameterized by the HUMUS-Block architecture.

Original: This can beconceptualized as alternating between enforcing consistency with the measurements and applying adenoiser based on some prior, represented by the HUMUS-Block in our architecture.
Revised: This can be conceptualized as alternating between enforcing consistency with the measurements and applying a denoiser based on some prior, represented by the HUMUS-Block in our architecture.

Original: We show an example of a sequence of intermediate reconstructions in order to support the denoisingintuition in Figure 7.
Revised: We show an example of a sequence of intermediate reconstructions in order to support the denoising intuition in Figure 7.

Original: We plot the magnitude image of reconstructions at the outputs of consecutive
Revised: We plot the magnitude image of reconstructions at the outputs of consecutive

Original: HUMUS-Blocks, along with the model input zero-padded reconstruction.
Revised: HUMUS-Blocks, along with the model input zero-padded reconstruction.

Original: We point out that in generalthe intermediate reconstruction quality (for instance in terms of SSIM) is not necessarily an increasingfunction of cascades.
Revised: We point out that in general the intermediate reconstruction quality (for instance in terms of SSIM) is not necessarily an increasing function of cascades.

Original: This is due to the highly non-linear nature of the mapping represented by theneural network, which might not always be intuitive and interpretable.
Revised: This is due to the highly non-linear nature of the mapping represented by the neural network, which might not always be intuitive and interpretable.

Original: F Vision Transformer terminology overview
Revised: G Vision Transformer terminology overview

Original: Here we provide a brief overview of the terms related to the Transformer architecture, some of whichhas been carried over from the Natural Language Processing (NLP) literature.
Revised: Here we provide a brief overview of the terms related to the Transformer architecture, some of which has been carried over from the Natural Language Processing (NLP) literature.

Original: We introduce the keyconcepts through the Vision Transformer architecture, which has inspired most Transformer-basedarchitectures in computer vision.
Revised: We introduce the key concepts through the Vision Transformer architecture, which has inspired most Transformer-based architectures in computer vision.

Original: For more details, we refer the reader to [Dosovitskiy et al., 2020].
Revised: For more details, we refer the reader to [Dosovitskiy et al., 2020].

Original: Traditional Transformers for NLP receive a sequence of 1D token embeddings.
Revised: Traditional Transformers for NLP receive a sequence of 1D token embeddings.

Original: Each token may represent for instance a word in a sentence.
Revised: Each token may represent for instance a word in a sentence.

Original: To extend this idea to 2D images, the input x ∈ R H × W × C image ( H and W stand for spatial dimensions, C for channel dimension) is split into N patches of size P × P , and each patch is ﬂattened into a 1D vector to produce an input of ﬂattened patches x f with shape N × P 2 C :x f =
Revised: To extend this idea to 2D images, the input x ∈ R H × W × C image ( H and W stand for spatial dimensions, C for channel dimension) is split into N patches of size P × P , and each patch is ﬂattened into a 1D vector to produce an input of ﬂattened patches x f with shape N × P 2 C :

Original: PatchFlatten ( x ) .
Revised: [Equation]

Original: In order to set the latent dimension of the input entering the Vision Transformer, a trainable linear mapping E ∈ R P 2 C × D is applied to the ﬂattened patches, resulting in the so called patch embeddings z of shape N × D :z
Revised: In order to set the latent dimension of the input entering the Vision Transformer, a trainable linear mapping E ∈ R P 2 C × D is applied to the ﬂattened patches, resulting in the so called patch embeddings z of shape N × D :

Original: [Equation]
Revised: [Equation]

Original: f E. In order to encode information with respect to the position of image patches, a learnable positional embedding E pos ∈ R N × D is added to the patch embeddings:
Revised: In order to encode information with respect to the position of image patches, a learnable positional embedding E pos ∈ R N × D is added to the patch embeddings:

Original: [Equation]
Revised: [Equation]

Original: The input to the Transformer encoder is this N × D representation, which we also refer to in thepaper as token representation, as each row in the representation corresponds to a token (in our casean image patch) in the original input.
Revised: The input to the Transformer encoder is this N × D representation, which we also refer to in the paper as token representation, as each row in the representation corresponds to a token (in our case an image patch) in the original input.

Original: G Detailed validation results
Revised: H Detailed validation results

Original: [Table]
Revised: [Table]

Original: H Additional ﬁgures
Revised: I Additional ﬁgures

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: HUMUS-Net reconstructs ﬁne details on MRI images that other state-of-the-art methods may miss.

Original: I Results on additional accelerations

Original: We observe that HUMUS-Net achieves state-of-the-art performance across a wide range of acceleration ratios.

Original: We perform experiments on the Stanford 3D dataset using a small HUMUS-Net modelwith only 6 cascades, embedding dimension of 66 , adjacent slice reconstruction with 3 slices and for the sake of simplicity we removed the residual path from the HUMUS-Block.

Original: We call this model

Original: HUMUS-Net-small .

Original: We set the learning rate to 0 . 0002 and train the model with Adam optimizer.

Original: We generate 3 random training-validation splits on the Stanford 3D dataset (same splits as in other

Original: Stanford 3D experiments in this paper), and show the mean and standard error of the results in Table7.

Original: In our experiments, HUMUS-Net achieved higher quality reconstructions in every metric over

Original: E2E-VarNet.

Original: [Table]