MotionTTT: 2D Test-Time-Training Motion Estimation for 3D Motion Corrected MRI
Authors: Tobit Klug
Kun Wang
Stefan Ruschke
Reinhard Heckel
Venue: NeurIPS 2024 poster (2024)
Keywords: deep learning based motion estimation
3D imaging
MRI
motion artifacts
medical imaging
test-time-training
motion correction
Primary area: machine_learning_for_healthcare
TL;DR: We propose MotionTTT a deep learning based test-time-training approach for motion estimation for the problem of motion artifact correction in 3D MRI.

Abstract: A major challenge of the long measurement times in magnetic resonance imaging (MRI), an important medical imaging technology, is that patients may move during data acquisition. This leads to severe motion artifacts in the reconstructed images and volumes. In this paper, we propose MotionTTT a deep learning-based test-time-training (TTT) method for accurate motion estimation. The key idea is that a neural network trained for motion-free reconstruction has a small loss if there is no motion, thus optimizing over motion parameters passed through the reconstruction network enables accurate estimation of motion. The estimated motion parameters enable to correct for the motion and to reconstruct accurate motion-corrected images. Our method uses 2D reconstruction networks to estimate rigid motion in 3D, and constitutes the first deep learning based method for 3D rigid motion estimation towards 3D-motion-corrected MRI. We show that our method can provably reconstruct motion parameters for a simple signal and neural network model. We demonstrate the effectiveness of our method for both retrospectively simulated motion and prospectively collected real motion-corrupted data. Code is available at \url{https://github.com/MLI-lab/MRI_MotionTTT}.

MotionTTT: 2D Test-Time-Trainne Motion Estimation for 3D Motion Corrected MRI
Tobit Klug'." Kun Wang". Stefan Ruschke² Reinhard H
School of Computation, Information and Technology, Technical University of Munich 2 Department of Diagnostic and Interventional Radiology, Klinikum rechts der Isar, School of Medicine and Health, Technical University of Munich
Abstract
A major challenge of the long measurement times in magnetic resonance imaging (MRI), an important medical imaging technology, that patients may move during data acquisition. This leads to severe motion artifacts in the reconstructed images and volumes. In this paper, we propose MotionTTT deep learning- time-training (TTT) method for accurate motion estimation. The key idea is that neural network trained for motion- free reconstruction has a small loss if there is no motion, thus optimizing over motion parameters passed through the reconstruction network enables accurate estimation of motion. The estimated motion parameters enable to correct for the motion and to reconstruct accurate images. Our method uses 2D reconstruction networks to estimate rigid motion in 3D, and constitutes the first deep learning based method for 3D rigid motion estimation towards 3DD-motion-correcte MRI. We show that our method can provably reconstruct motion parameters for simple signal and neural network model. We demonstrate the effectiveness of our method for both retrospectively simulated motion and prospectively collected real motion-corrupted data. Code is available at https::///// http://gitttt.tt com www/moIIi//.. MiiLI-llabMMM
test-
Introduction
Magnetic resonance imaging (MRI) is one of the most important medical imaging technologies due to its non-invasiveness and ability to foster diagnosis of wide range of diseases. However, its inherently long scan times make MRI susceptible to motion artifacts caused by patient movement during the scan. Repeating scans corrupted by motion artifacts causes additional costs and reduces patient through-put, and unnoticed artifacts can lead to misdiagnosis [2 36).
We consider the problem of imaging under motion and propose algorithmic solution to correct for the motion based only on the measurements acquired during the scan, without requiring additional hardware changing the measurement process, interrupting the clinical workflow.
traditional approach to algorithmic motion reconstruction to jointly estimate the motion parame- ters and motion-corrected reconstruction 7 6 15] but those methods are slow and can be inaccurate in particular for severe motion.
Deep learning based approaches have been proposed to accelerate reconstruction and potentially allow account for more severe motion. However, most existing data driven methods correct for in-plane motion within 2D MRI (see the review Spieker et al. (38) ), as 3D data for training 3D models only scarcely available and computationally expensive to handle [18). practice, however, motion occurs in 3D and not in mlpaaee. Moreover, the duration of a scan in 3D is significantly longer than in 2D making motion more likely and thus motion reconstruction more important. Finally, data-driven
Shared first authors in alphabetic order. CCorresponding author: reinhard.hecker tum.de
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

approaches so far often rely on simulated motion artifacts and hence are specific to the type of motion
they have been trained on [14] 16 34).
In this work, we propose a novel approach for rigid motion estimation and reconstruction in 3D MRI that is based on first estimating the motion parameters that describe the map from the motion-free image to the motion-corrupted measurement and second reconstructing the image or volume with the estimated motion parameters. Estimating the motion parameters is the critical step, once we know the motion parameters, reconstruction essentially amounts to reconstructing from a motion-free measurement, and variety of approaches work well for that.
For motion estimation, we utilize a neural network trained to reconstruct motion freeen-free undersampled 2D MRI images. The network only requires 2D motion- free data for training, and does not require 3D or motion-corupteed data which is difficult to come by. The neural network for reconstruction depends on the forward model which in turn depends the motion parameters. We construct a data- consistency loss and optimize over the motion parameters at test-time. The data consistency loss is small only for the correct motion parameters, as the model was trained for motion-free reconstruction.
In each iteration the model reconstructs the 3D data slice-wise along a random axis. Since motion artifacts occur globally in the image domain it is sufficient to compute gradients only for small random subset of slices keeping the computational and memory cost manageable. The estimated motion parameters are then used to reconstruct clean volume. To summarize, contributions are:
We propose Moti the first deep learning-based method for 3D rigid motion estimation for 3D motion-corrected MRI. Motion exploits the prior knowledge of pre-trained neural network for motion-free 2D MRI reconstruction.
We theoretically justify our method by proving for a simple theoretical signal and neural network model that the loss function has global minimum at the correct motion parameters.
We use retrospectively simulated motion to demonstrate the ability of our method to ac- curately estimate motion over wide range of motion severities. Combined with L1- minimization reconstruction module we achieve effective 3D imaging under motion and outperform a classical alternating optimization baseline (7) in terms of estimation speed and estimation performance under severe motion and deep learning based end-to-end motion correction baseline Al-Masni et al. (1 in terms of reconstructed image quality.
We demonstrate the potential of our method on prospectively acquired real motion-corrupted data achieving significant improvements in terms of visual image quality.
2 Related work
Approaches for retrospective rigid-motion correction for MRI can be categorized into supervised deep learning-based approaches, model-based optimization approaches, and combinations thereof.
Supervised deep-learning based approaches learn a mapping from undersampled measurements corrupted with simulated motion to the motion-free reference image and have been proposed for
2D 21 42 35 and 3D MRI 118 10. However, as demonstrated in our work the images produced by those end-to-ens approaches are often blurry and the methods pertain to the type of motion simulated during training 23
16).
Alternating optimization 7 is classical model-based approach for joint motion estimation and correction in 3D MRI, where every iteration alternates between optimizing over the motion parameters while fixing the estimated reconstruction and vice versa. However, jointly optimizing over both unknowns without any prior information is a highly complex optimization problem resulting long run times and errors in the presence of more severe motion as demonstrated in our work.
The speed and robustness of alternating optimization can be improved through augmenting either the reconstruction step or the motion estimation step with deep learning [14] 16]. However, both methods have so far only been proposed for 2D in-plane motion correction and rely on synthetic motion simulation during training, which makes them specific to the type of simulated motion. In practice, motion always occurs in 3D.
For motion estimation, Singh et al. (34) pre-trains a neural network to predict a motion-free image from the motion-corrupted measurements conditioned on the true motion parameter. At inference

---

50
50
40
30
30
kg
10
10
2x
Ky
ky
C)
Figure 1: Panel a): magnitude of a 3D volume; panel b): the corresponding 3D k-space data. Panels c)/d) show examples of undersampling masks used for the simulated and real data. The color coding illustrates an interleaved c) and random d) sampling trajectory indicating which lines along the readout dimension k are sampled within the same out of 50 shots.
the data consistency loss is optimized only over the motion parameters. However, the motion estimation relies on learning the characteristic of motion-corruptions. Hence, the model operates in the measurement space in which the corruptions occur, which in multi-coil MRI is of much higher dimension than the corresponding image space, thus making an extension of this approach 3D very challenging, Our method relies on learning the characteristics of motion-fre data in the image space and thus is efficent for 3D imaging as we show.
Levac et al. (22) proposes a method based on diffusion models trained on generating motion-free images for 2D motion reconstruction, that also does not rely motion simulation during training. At inference, sampling based reconstruction 17. with motion trajectory estimation is performed.
While our approach well the aforementioned works correct motion artifacts solely based on the acquired MRI measurements, another line of research corrects for motion prospectively or retrospectively based additional data collected during the scan via, c.g., external detectors (30, [27] or navigator sequences (40] 433 IT]. However, those methods usually require interference with the standard clinical measurement process and are often tailored to specific measurement sequence setup limiting their broad applicability in practice.
Problem statement: 3D MRI imaging under motion
A 3D multi-coil accelerated MRI measurement y (CXk,xky is obtained by
y Ax+
(1)
where A MFE is the forward model, Xr.Xryxr the object of interest, and measurement noise. The measurement y consists of C-many k-space measurements collected by C' coils. The expand operator is defined as Ex [Six,..., SSx] with coil sensitivity maps Sy. The 3D Fourier transform F and undersampling mask M are applied coil-wise. We consider 3D Cartesian undersam- pling, where undersampling takes place in the plane of the phase encoding dimensions k. x hy and the frequency encoding dimension ky is fully sampled (see Figure
We focus rigid motion, where the i-th motion state is defined by three translation and three rotation parameters m, = [t, ty f3, of obg Φ5]. MRI acquisition under motion can be described as
y = A(T, m)x
where the forward model A is function of the unknown motion states [m...... ma). where b the number of motion states, and of the known MRI sequence specific sampling trajectory T that specifies which part of the k-space data y is acquired when.
The goal of this work is to reconstruct the volume x from the undersampled, motion-corrupted measurement y without knowledge of the motion states. We do so by estimating the motion states m from the measurement y and sampling trajectory T and use the estimated parameters to reconstruct a motion-corrected volume x
Parameterization of the forward model under motion. practice, a measurement acquired in batches of lines in the k-space along the frequency encoding dimension k.. A batch, referred to as

---

Reference image
Reference image
Reference K-sace
Rotation corrupted 7F k-space
Corrupted ZF k-space
MI
X
N(T,0)
L(T,t)
Motion corruption
Motion correction
Nar(T
L(T,
F-1
x
Corrupted ZI image
ZF image
Corrected ZF image
Corrected ZF k-space
Rotation corrupted
ZF k-space
Figure 2: Illustration of the MRI forward models and zero-filled (ZF) reconstructions without (left) and with (right) motion for the 2D single-coil setup. Rotations implemented with the NUFFT N(T, Φ) and adjoint NUFFT aaa &), and translations with linear phase shifts L(T, t). During acquisition under rotations areas of the k-space sampled multiple times while others are not sampled at all, resulting in additional undersampling artifacts in the corrected ZF image compared to the motion-free ZF image.
shot, is acquired within a short time window followed by pause before the next subsequent shot. It popular to assume that a subject's position constant during one shot and motion happens during
the pause between shots. This is known as inter-shot motion (7] 6 [8] 22 34 [6]. For inter-shot
motion, the number motions states b is equal the number of shots and the sampling trajectory T
maps the lines in the k-space acquired during the i-th shot to the i-th motion state. See Figure (c,d)
for examples of sampling trajectories used in practice.
However, in practice, motion can occur anytime, and thus the inter-shot motion introduces an approximation error. Motion during the acquisition of one shot is referred to as intra-shot motion (14). Then, each k-space line acquired during such shot can have a distinct motion state. In this work, we investigate the capabilities of our method under both inter- and intra-shot simulated motion.
Within the forward model AT, m), motion corruption can be applied in the image or in the k- space [25), since rotations and translations in the image space translate rotations and linear phase shifts the Fourier space. We apply motion corruption in the k-space, because it is computationally more efficient for our setup (see Appendix A). We use the non-uniform FFT (NUFFT) N(T, Φ) to sample fully-sampled k-space data at the rotated coordinates for each shot. Translations are applied via linear phase shifts L(T, t) obtain motion-corrupted k-space data.
As a starting point for reconstructing the volume x from an undersampled measurement with zero-filled (ZF) missing entries, it is common to compute ZF reconstruction xf Aty, where Aly Σ j=1 S If the measurement y motion corrupted, corrected ZF reconstruction based on motion parameters m is x = A' -m)y. First, translations are reverted with phase shift in the opposite direction L(T, -t). Then, rotations are corrected for via the adjoint NUFFT Nadi (T, -). See Figure for an illustration of the forward model and ZF reconstruction.
4 MotionTTT
The proposed MotionTTT consists of: 1) pre-training a neural network for 2D motion-free image reconstruction, 2) test-time-trainn training estimate motion parameters of the k-space data and 3) reconstructing the motion-corrected 3D image based on the estimated motion.
3D
Step 1: Pre-training, Given motion-free data ((x1.11), NN). consisting of pairs of 2D
reference images x Orx and undersampled k-space data y e Cxkkx Xk, train U-net 32

---

fo with weights 0 to map a ZF reconstruction A A to the image x; by minimizing the loss
N
Luain (0) - ((11)((A)y)) |x,I|| + FEfa(A))) - FE /FFxx/))
(3)
i=1
We use this combined training loss between magnitude images and k-space data since it leads to better performance for motion-free reconstruction than using one of the individual losses (see Appendix (
Step 2: Test-time-training for motion estimation. Given an undersampled and potentially motion corrupted 3D measurement, y and a sampling trajectory T we freeze the weights @ of the trained network fg and estimate the motion parameters m by minimizing the data consistency loss
(rrr(m)= = ||A(T, m)fo (A (T,
( -m)y) - y|l,/ /lylll
(4)
with Adam [19]. The idea behind minimizing this loss as follows. applied to motion-corrupted data, at initialization with m = 0 the motion correction A AT -m) has no effect and the motion- corrupted network input results in a large loss as the network was trained on reconstructing motion- free data. Contrary, when the motion parameters are chosen correctly the network input is a motion- corrected ZF image, which similar to a motion-free ZF image and results small loss. See Figure 2 for example images. In Section 5 below we study this loss theoretically.
The
We call this approach trantttiiitnnn........ since the adjoint A (T, (T. can be considered be part of the network, and by optimizing over the motion states m we are optimizing over part of the network's parameters. Methods that optimize network at inference are referred as test-time-training methods, and are successful at prediction under distribution shifts (39
In every iteration, the network reconstructs the entire 3D input volume slice- wise, where the slicing direction is sampled uniformly at random to be either in the Tz x Ty, Ta X Tz or Ty X rz image plane. We compute gradients only for subset (of size 5, limited by GPU memory) of slices sampled independently in every iteration. While motion has a local effect in the k-space, the artifacts spread globally in the image space hence computing gradients with respect to single slice contain signal about all motion parameters.
In order to minimize the loss 4 reliably over different levels of motion severity the optimization scheme is important. We take a -hase- optimization approach.
Phase I optimizes over one motion state per acquired shot. We start with a large initial learning rate in order explore the non-convex loss landscape. Especially for strong motion initializing all parameters with 0 can lead to a large distance to the true motion parameters. During phase the learning rate decayed twice in order to converge to stable first estimate of the motion parameters.
At the start of phase 2 we compute the DC loss 4 for every estimated motion state m
(5) where the mask Ma. keeps only the part of the k-space acquired during the i-th state. Motion states with loss larger than certain threshold are likely estimated poorly and reset them to the average between the previous and next motion state that fall below the threshold. Phase 2 then only optimizes over the motion states that have been reset. For intra-shot motion, a thresholded motion state is not only reset, but Naplis additional motion states up to the number of acquired k-space lines per shot can be introduced in order to estimate more resolved motion trajectory for this shot.
(m.) = AA m.:). (TTT.m))) - MM.
Phase again optimizes jointly over all motion states with a small learning rate to converge to a final estimate of the motion trajectory.
Step 3: Reconstruction. Using the estimated motion parameters m. we can obtain an estimate of the motion-corree image directly from the network output fg (A (T, -m)y), and apply a data-consistency step improve performance
DCLyerr, This step moves the frequencies of the reconstructed image closer to the given frequencies, and is used for example by Chen et al. Alternatively, we can use any reconstruction method for moteon--rre data, such as a classi- cal L1 --mmimmzzaiion-baaee based reconstruction [26]. We refer to L1 or U-net reconstruction based on motion parameters estimated by MotionTTT as MotionTTT-L. and DouiionnTT-U-ne--DCCC and their respective performance with the oracle known motion parameters as Knwon and KnownMotion- JJ-net-DCaayer. We refer to the reconstruction after DC loss thresholding that excludes motion states with large DC loss from the reconstruction as MotionITT+TTh-
5

---

0.8
a
loss [(m)
0.6
0.4
0.2
a
shift m
Figure 3: For example with 2k, k 1400, d 100, and 4 and m 0 plot the loss as function of m1, where a is the number values for m2, ma. my that are set to an integer that is non-equal to my, mÿ. m respectively. It can be seen that there is sharp minima around m". This minima turns out to be unique under certain conditions. In our theory we consider discrete shifts, indicated by crosses.
5 Theory for motion TTT
We consider the following model to illustrate the principle of our method. We consider a signal R" the lies in dimensional subspace described by the matrix U Rxxd i.e., there is coefficient vector E Rd so that x = Uc. We take the matrix U as a random Gaussian matrix with iid (0, 1/vn)) entries, that the columns of the matrix approximately orthonormal.
Let F be the Fourier matrix with rows chosen in the set T S {O,.... n n- 1}. We assume a measure- ment model where the signal is shifted by unknown discrete integer parameters m .... m E Z and for each shifted version the signal, a set of measurements is collected according to
ye= = Dm; T FT,,,
where Dm,T, is a diagonal matrix with /2amj// .j E Te on its diagonal. Note that this multiplication with complex exponentials in the frequency domain implements a circular shift in the time domain. In this measurement model, the signal assumed motion-free while the measurements in the set FT, are collected. The frequencies in the set T7 are chosen by sampling each frequency independently with probability k/n. So in expectation, A frequencies are included in the set Te.
We consider the network f(x) = be UU x for reconstructing a clean signal, where bk is the total number of measurements collected. This choice of network is motivated by the fact that if the measurement is not motion corrupted, then we have that (see Appendix B.3
f(Fry) ~ x,
where T = U... UT,.U is the set of all measurements collected and (.)" denotes the complex conjugate. We consider our test-time-training loss for this model, which is
((m) |D (FF - y11
(8)
Below, we show that for our model, under certain conditions and with high probability, the loss has unique minimum at L(m"). Before stating our result, visualize the loss in Figure It can be
seen that the loss is not convex in m which makes it difficult to optimize.
Theorem Consider the model introduced above, and assume that the signal Uc chosen randomly by drawing the entries of lid from zero-mean unit anit-variance Gaussian distribution. Let a(m) be the number of values of m...... my that are non-equal m mi, The following statement holds for all m E {0......
1} {m²) simultaneously with high probability: If c
b² log(n)) (b d) n²
(1 - (1ma(m)/b)²
+ +oV
(9)
n
k262
bk
then T.m) I(mm),, where e a numerical constant.
The theorem implies that if the subspace dimension, d, and the number of shifts, b, are sufficiently small relative the number of measurements, bk, then the loss has global minimum at the true shift m".

---

Experiments
We demonstrate quantitatively and qualitatively on simulated data the ability of our method to accurately reconstruct images for wide range of levels of motion severity in the presence of inter- and intra-shot motion. Moreover, on prospectively acquired real motion cortunne data, we demonstrate that our method achieves significant gains in terms of visual reconstruction quality,
6.1 Simulated inter-shot motion experiments
We start with experiments with simulated inter-shot motion as described Section S The model, data, and basclines considered for both inter- and intra-shot motion experiments are as follows.
Model. We use a 17.5M parameter U-net 32] fo. a common baseline with good performance on image-to-image tasks 46 13). training details are in Appendix D
Data. We train the U-net fe on motion-free data and evaluate Motion with simulated motion- corrupted data sourced from the Calgary Campinas Brain MRI Dataset (37] (license CC BY-ND) consisting of 3D scans of size k2 x ky X k2 218 x 170 x 256 and C = 12 receiver coils. We select 40 subjects for training from the training set, 4 subjects for validation and hyperparameter tuning, and 10 subjects for testing from the validation set. We train and test with an undersampling factor of 4 using the mask from Figure (c). For training, we slice each 3D zero-filled reconstruction A y and the corresponding 3D reference volume x along all three dimensions resulting in about 25k pairs of 2D network inputs and targets. We compute sensitivity maps from 24 x 24 x 24 auto-calibration lines of the originally salpyesampled motion-fre free k-space with ESPIRiT 41
Motion simulation. We set the number of shots to B 50 similar how our own real data was acquired in the upcoming Section 6.3 and we use interleaved sampling trajectory T (Figure (c)), where every 50-th line in the k-space acquired within one shot and the 3 center of the k-space containing the largest energy is sampled in the first shot. Without loss of generality, we assume the subject to be in zero-motion state at the first shot and hence do not simulate and estimate motion parameters for the first shot.
As the characteristics of patient motion vary widely, synthetic rigid body motion is often simulated as random motion with rotations and translations drawn uniformly from some range, or from Gaussian
We simulate random motion with different levels of severity by varying the number of motion events No E (1,5, 10} per scan and the maximum possible rotations/translations M E {2, 5, 10) in degrees/mm. The shots between which a motion event occurs are sampled uniformly at random, and for each event translation and rotation parameters are sampled uniformly from [-M max, Mmax]. After motion event the subject stays in this position until the next motion event occurs. This yields 10 different levels of motion severity including the motion free case.
Baselines. We compare to alternating optimization by Cordero- Grarro--Grnneeeco et al. (7). which is one of very few approaches for retrospective motion estimation in 3D MRI. The method alternates between two steps of Ll-minimization reconstruction with wavelet regularization while fixing the motion parameters and four steps of motion parameter estimation while fixing the reconstruction. For the final reconstruction perform Ll-minimization from scratch based on the estimated motion parameters (AliOpt-LI) and with additional DC loss thresholding based on the estimated motion parameters (AliOpt+Th-LL)), to ensure fair comparison our method. We also perform LI Ll-minimization without any motion estimation (L/). We further compare to the E2E stucked U-net with self-assisted priors [[] method for end-to-end (E2E) motion artifact reduction in 3D MRI. The model from Al- Masni et al. (T] is trained with pairs of motion notion-free images and undersampled images corrupted by motion simulated as described above to predict the motion-corrected image with a single pass trough the network at inference.
Hyperparameters for MotionTTT and the baselines are in Appendix D
MotionTTT accurately estimates motion over wide range of motion severities. The results in Figure 4 show the reconstruction performance in PSNR function of motion severity averaged
7

---

KnownMotion-Li
35
KnowaMotion-Unnet-DCLL...
PSNR (dB)
MotionTTT-LI
MotionITT+Thh-LI
30
AltOpt-L1
AllOpt+Th-LI
25
E2E Stacked-U-net
20
1
2
4
5
8 9
(0,0)
(1.2)
(3,2) (10,2) (1.5) (1.10) (5.5) (10,5) (5.10) (10,10)
Level of motion severity
Figure 4: Reconstruction performance in PSNR as a function of the level of simulated inter-shot motion severity defined by (number of motion events, maximum rotation/translation degrees/mm). We consider L1 --minimization or U-net based reconstruction combined with either known motion, no motion-correction or motion estimated with MotionTTT alternating optimization. Error bars the standard deviation over test examples and randomly sampled motion trajectories.
E2E Stacked U-net
AltOpt+ Th-L1
MotionTTT-
MotionTTT+ Th-L1
Known Motion-
KnownMotion- U-net-DCLayer
Reference Img.
L1
L1
POWN
PSHN
PANN
Figure 5:
Reconstructions and difference images for simulated motion of severity level for all methods in Figure
over the test set and over two independently sampled motion trajectories per example. Figure and and
Appendix E. 1.1 contain example reconstructions. For both small and large motion severities, the PSNRs for reconstruction with known and estimated motion parameters is the same for Motion indicating that the motion parameters estimated very well. Reconstruction results for the motion parameters itself are in Appendix E.I.I
For AltOpt this is only true for small motion severities, for large ones MotionTTT significantly outperforms AltOpt. In addition, MotionTTT is about 6x faster than Al1Opt for this problem, see Appendix 1.2
The performance of E2E stacked U-ner lies significantly below the performance of methods that obtain reconstruction based on explicitly estimating motion like MotionT7T or Al1Opt. With increasing motion severity E2E methods suffer from the severe degradations in their network inputs, which can not be resolved without knowledge of the motion resulting in overly smooth reconstructions.
For the results Figure we used the fixed undersampling mask from Figure I (c) with acceleration factor 4. In Appendix 1.3 we additionally ablate over acceleration factors 2 and 8 showing robust performance acceleration factors and levels of motion severity.
Reconstruction module. Figure 4 shows that for motion, U-net based reconstruction (KnowWMoion----eet-DDCC DCCaa-aaaee- outperforms Ll-based reconstruction (KnownMotion-LI) as ex- pected, but for larger levels of motion severity L.I-baee reconstruction performs slightly better. A likely reason for this is distribution shift, which known to result worse performance (8): Ac- quiring MRI measurements under motion typically leads to some parts of the k-space being sampled

---

more than once, while parts that would be sampled if there were no motion are not sampled. This results in a change of effective undersampling mask and a cootion-specifie increase of the effective undersampling factor, which suggests why the reconstruction quality decays for all methods with increasing levels of motion even motion parameters are known. This problem, illustrated in Figure [ is well known [4 [2]. However, L.--based reconstruction is not as sensitive to changes in the mask as the U-net based reconstruction, which is trained with a fixed mask. Due to this distribution shift, the performance of U-net might degrade stronger than for LI-based reconstruction.
Finally, we note that, comparing Motion]TT-LI and Motion strong motion, DC loss thresholding essentially closes the gap to KnownMotion- L7. This indicates that the thresholding does not unnecessarily exclude motion states for moderate motion, but reliably detects incorrectly estimated motion states for severe motion.
Figure shows that for
6.2 Simulated intra-shot motion experiments
Next, we study the performance of our method for intra-shot motion. As mentioned before, the model, data, and baselines are the same as for the inter-shot experiments in Section 6.1
Motion simulation. We again set the number of shots to B = 50. We use a random sampling trajectory (Figure [ (d), where the 3 x 3 center is acquired first and all other k-space lines are acquired at a random order, and again assume the subject to be zero-motion state at the first shot, We randomly select [Ne/2] of the motion events to take place during the acquisition of one shot. Intra-shot motion is simulated by assigning a distinct motion state to each of the 182 k-space lines acquired during this shot such that the intra-shot motion trajectory connects the motion parameters from the previous to the next shot, where start and end point as well as the presence of up to two peaks due over- and/or under-shooting within the intra-shot trajectory is randomized.
Choice of the parameter NNplits- As explained in Section 4 Motion estimates intra-shot motion by splitting motion states that exhibit a large DC loss after phase I of the optimization scheme into Naplis motion states. Since ground truth intra-shot motion is simulated with a distinct motion state for each of the 182 k-space lines acquired during one shot, choosing the parameter Noplis 182 results in a discretization error. On the other hand, choosing Naplits large increases the difficulty of the estimation problem because the number of k-space lines corresponding to one motion state, decreases. We found Naplins = 10, i.e., about 18 k-space lines per motion state, to be a good trade-off, see Appendix R.2. 1 for ablation study.
Sampling order:
The sampling order within a shot is crucial for the ability to estimate intra-shot batch consisting of only high-frequency and low-energy components is difficult. In Appendix [E.2.2 we ablate over different sampling orders and find that acquiring all k-space lines random order works well.
motion estimating a motion state corresponding
MotionTTT with intra-shot motion estimation improves performance over discarding mea- surements corrupted by intra-shot motion. Figure @gguresshoss shows the reconstruction performance as a function of motion severity, where half of the motion events exhibit intra-shot motion. Recall that after phase of the optimization scheme outlined in Section 4 MotionTTT converged to one estimated motion state per shot. MotionTTT-LI (Phase 1) is based on those motion states and hence does not estimate intra-shot motion. Consequently, even for the lowest level of motion considered here, MotionnT7T++ Th-IJ (Phase 1) improves the performance as shots corrupted by intra-shot motion are discarded during DC loss thresholding before reconstruction.
In contrast, Motion (Phase 3) achieves on par performance with MotionIIT+Thhe Th-LI (Phase 3) for moderate motion levels indicating that intra-shot motion has been estimated successfully that motion states are below the threshold we set for DC loss thresholding. Nevertheless, a performance gap remains relative to KnownMotion-LI, which results from the irreducible discretization error.
For severe motion the gap between MotionTTT-LI (Phase 3) and Motion]TT+Ther (Phase 3) increases as more motion states are estimated incorrectly and have to be discarded. However, MotionTTT with only Phase 1 continues to be outperformed indicating the benefit of performing intra-shot motion estimation over discarding measurements corrupted by intra-shot motion.

---

KnownMotion-LI
Motion (Phase 3)
PSNR (dB)
Motion]TT4ThhLL (Phase 3)
34
Motion (Phase 1)
MotionITT+Th-LL (Phase 1)
32
30
3
8
9 (10,10)
(5.2)
(10,2)
(5,5)
(10,5)
(5.10)
Level of motion severity
Figure 6: Reconstruction performance PSNR as a function of the simulated motion severity defined in Figure only that here half of the motion events exhibit intra-shot motion. We consider LI- minimization based on known motion, motion estimated with MotionTTT after phase 1 (no intra-shot motion estimation) and after phase (with intra-shot motion estimation) of the optimization scheme. Error bars are the standard deviation over test examples and randomly sampled motion trajectories.
Figure in Appendix [ contains a qualitative comparison, where especially in the case without DC loss thresholding the difference in reconstruction quality between phase and 3 is clearly visible. After thresholding the differences are less visible, however that depends on the amount of intra-shot motion that simulate with more intra-shot motion the difference in undersampling factor between discarding and estimating intra-shot motion becomes larger and differences more visible.
Experiments with real motion
We now apply MotionTTT to real motion data. We use pre-trained model fo from Section 6.1 and the implementation details are described Appendix D
Data. We acquired four scans from one subject. To obtain motion-free reference data, mildly motion-corrupted and strongly motion-corrupted data the subject was instructed to not move at or move 1-3 times at distinct time points during the acquisition. The performed motions include nodding, head rotations, and either with or without returning to the original position. Sequence parameters (see Appendix D.4) were chosen to match those in the Calgary Campinas Brain MRI Dataset (37) as close as possible. Data was acquired with an undersampling factor of 4.94 and a random sampling trajectory (similar to Figure (d)) with B 52 shots. The acquisition of one shot lasts 1.3s followed by a pause of 1.6s resulting in total scan duration of about 150s.
We find that MotionTTT achieves significantly improved visual reconstruction quality compared to no motion-correction. Applying Motion]TT results in significant reduction of motion artifacts for both mild and strong motion (reconstructed images are in Appendix E.3
Results.
7 Limitations and future work
In this paper, we proposed the first deep-learning-basee 3D rigid motion estimation method for 3D MRI and have demonstrated that it is effective and computationally managable estimating motion and correcting for it.
As discussed in Section 6.1 currently use L1- L.--minimization, but a deep-learning method, e.g., regularization with diffusion models should yield further improvements.
reconstruction module (Step 3) has room for improvement. We
Finally, the model used perform MotionTTT can be improved principle. Currently it trained on fully-sampled data, which is scarce especially for 3D at high resolutions. It might be possible to train this module well with self-supervised training losses that require only undersampled data [44] 28,20) 20). Moreover, mentioned, there is a distribution shift in the mask and networks that work well with such distribution shifts might yield improvements.
10

---

Acknowledgments and Disclosure of Funding
The authors are supported by the German Research Foundation (DFG) under grant numbers 466465471,,44123330 and 517586365.
References
[1] M. A. Al-Masni, S. Lee, J. Yi, S. Kim, S.-M. Gho, Y. H. Choi, and D.-H. Kim. "Stacked U-Nets with Self-Assisted Priors towards Robust Correction of Rigid Motion Artifact in Brain MRI". In: Neurolmage (2022).
[2] J. B. Andre, B. W. Bresnahan, M. Mossa-Basha, M. N. Hoff, C. P. Smith, Y. Anzai, and
W. A. Cohen. "Toward Quantifying the Prevalence, Severity, and Cost Associated With Patient
Motion During Clinical MR Examinations". In: Journal of the American College of Radiology:
JACR (2015).
[3] T. Brooks, B. Mildenhall, T. Xue, J. Chen, D. Sharlet, and J. T. Barron. "Unprocessing Images for Learned Raw Denoising". In: IEEE Conference on Computer Vision and Pattern Recognition. 2019.
[4] Z. Chen, Y. Chen, Y. Xie, D. Li. and A. G. Christodoulou. "Data-Consistenn Non-Cartenr Cartesian Deep Subspace Learning for Efficient Dynamic MR Image Reconstruction", In: IEEE Interna- tional Symposium
Biomedical Imaging (2022).
[5] H. Chung and J. C. Ye. "Score-Base Based Diffusion Models for Accelerated MRI". In: Medical
Image Analysis (2022).
[6] L. Cordero-Grande, E. J. Hughes, Hutter, A. N. Price, and J. V. Hajnal. "Three-Dimensiinn Motion Corrected Sensitivity Encoding Reconstruction for Multi-Shot Multi-Slice MRI: Ap- plication to Neonatal Brain Imaging". In: Magnetic Resonance in Medicine (2018).
[7] L. Gordero-Grande R. P. A. G. Teixeira, E. J. Hughes, J. Hutter, A. N. Price, and J. V. Hajnal. "Sensitivityy Encoding for Aligned Multishot Magnetic Resonance Reconstruction" In: IEEE Transactions on Computational Imaging (2016).
[8] M. Z. Darestani, A. S. Chaudhari, and R. Heckel. "Measuring Robustness in Deep Learning Based Compressive Sensing". In: International Conference Machine Learning. 2021. [9] M. Z. Darestani, J. Liu, and R. Heckel. "Test-Time Training Can Close the Natural Distribution Shift Performance Gap in Deep Learning Based Compressed Sensing". In: International Conference on Machine Learning. 2022.
[10] B. A. Duffy, L. Zhao, F. Sepehrband, J. Min, D. Wang, Y. Shi, A. W. Toga, and H. Kim. "Retrospective Motion Artifact Correction of Structural MRI Images Using Deep Learning Improves the Quality of Cortical Surface Reconstructions" In: Neurolmage (2021).
[11] Gallichan, J. P. Marques, and R. Gruetter. "Retrospective Correction Involuntary Miero- scopic Head Movement Using Highly Accelerated Fat Image Navigators (3D FatNavs) at 71". In: Magnetic Resonance Medicine (2016).
[12] F. Godenschweger, U. Kägebein, D. Stucht, U. Yarach, A. Sciarra, R. Yakupov, F. Lüsebrink, P. Schulze, and O. Speck. "Motion Correction in MRI of the Brain". In: Physics in Medicine and Biology (2016).
[13] J. Gurrola-Ramos O. Dalmau, and T. E. Alarcón. "A Residual Dense U-Net Neural Network for Image Denoising". In: JEEE Access (2021).
[14] M. W. Haskell, S. F. Cauley, B. Bilgic, J. Hossbach, D. N. Splitthoff, J. Pfeuffer, K. Setsom- pop, and I.. L. Wald. "Network Accelerated Motion Estimation and Reduction (NAMER): Convolutional Neural Network Guided Retrospective Motion Correction Using a Separable Motion Model", In: Magnetic Resonance in Medicine (2019).
[15] M. W. Haskell, S. F. Cauley, and L. L. Wald. "Targeted Motion Estimation and Reduction (TAMER): Data Consistency Based Motion Mitigation for MRI Using a Reduced Model Joint Optimization" In: IEEE transactions on medical imaging (2018).
[16] J. Hossbach, D. N. Splitthoff, S. Cauley, B. Clifford, D. Polak, W.-C. Lo, H. Meyer, and
A. Maier. "Deep Learning-Based Motion Quantification from k-Space for Fast Model-Based Magnetic Resonance Imaging Motion Correction". In: Medical Physics (2023).
[17] A. Jalal, M. Arvinte, G. Daras, E. Price, A. G. Dimakis, and J. Tamir. "Robust Compressed Sensing MRI with Deep Generative Priors".. In: Conference on Neural Information Processing Systems. 2021.
11

---

[18] P. M. Johnson and M. Drangova. "Conditional Generative Adversarial Network for 3D Rigid- Body Motion Correction in MRI". In: Magnetic Resonance Medicine (2019).
[19] D. P. Kingma and J. Ba. "Adam: A Method for Stochastic Optimization". In: International Conference on Learning Representations (2014).
[20] T. Klug. D. Atik, and R. Heckel. "Analyzing the Sample Complexity of Seff-Supervised Image Reconstruction Methods". In: Conference on Neural Information Processing Systems. 2023.
[21] T. Küstner, K. Armanious, J. Yang. B. Yang, F. Schick, and S. Gatidis. "Retrospective Cor- rection of Motion-Affected MR Images Using Deep Learning Frameworks". In: Magnetic Resonance in Medicine (2019).
[22] B. Levac, A. Jalal, and J. I. Tamir. "Accelerated Motion Correction for MRI Using Score- Based Generative Models". In: 2023 IEEE 20th International Symposium on Biomedical Imaging
(ISBI). 2023.
[23] B. Levac, S. Kumar, S. Kardonik, and J. I. Tamir. "FSE Compensated Motion Correction for MRI Using Data Driven Methods". In: Medical Image Computing and Computer Assisted Intervention MICCAI 2022. Cham, 2022.
[24] J. Liu, M. Kocak, M. Supanich, and J. Deng. "Motion Artifacts Reduction in Brain MRI by Means of Deep Residual Network with Densely Connected Multi-Resolution Blocks (DRN-DCMB)". In: Magnetic Resonance Imaging (2020).
[25] A. Loktyushin, H. Nickisch, R. Pohmann, and B. Schölkopf. "Blind Retrospective Motion
Correction of MR Images". In: Magnetic Resonance in Medicine (2013).
[26] M. Lustig, D. L.. Donoho, J. M. Santos, and J. M. Pauly. "Compressed Sensing MRI", In: IEEE Signal Processing Magazine (2008).
[27] J. Maclaren et al. "Measurement and Correction of Microscopic Head Motion during Magnetic Resonance Imaging of the Brain". In: PloS One (2012).
[28] C. Millard and M. Chiew. "A Theorctical Framework for Self- Self-Supervised MR Image Recon- struction Using Sub-Sampling via Variable Density Noisier2Noise". In: IEEE transactions on computational imaging (2023).
[29] M. J. Muckley, R. Stern, and F. Knoll. "TorchKbNuff: A High-Level, Hardware-Agnostic Non- Non-Uniform Fast Fourier Transform". In: ISMRM Workshop on Data Sampling and Image Reconstruction. 2020.
[30] M. B. Ooi, S. Krueger, W. J. Thomas, S. V. Swaminathan, and T. R. Brown. "Prospective Real- Time Correction for Arbitrary Head Motion Using Active Markers", In: Magnetic Resonance in Medicine (2009).
[31] J. G. Pipe and P. Menon. "Sampling Density Compensation in MRI: Rationale and Iterative Numerical Solution".. In: Magnetic Resonance in Medicine (1999).
[32] O. Ronneberger, P. Fischer, and T. Brox. "U-Net: Convolutional Networks for Biomedical Image Segmentation". In: Medical Image Computing and Computer-Assisted Intervention (2015).
[33] M. Rudelson and R. Vershynin. "Noo-Asyyptotic Theory of Random Matrices: Extreme Singular Values". In: Proceedings of the International Congress of Mathematicians. 2010.
[34] N. M. Singh, N. Dey, M. Hoffmann, B. Fischl, F. Adalsteinsson, R. Frost, A. V. Dalca, and
P. Golland. "Data Consistent Deep Rigid MRI Motion Correction". In: Medical Imaging with Deep Learning. 2023.
[35] N. M. Singh, J. E. Iglesias, E. Adalsteinsson, A. V. Dalca, and P. Golland. "Joint Frequency and Image Space Learning for MRI Reconstruction and Analysis". In: The journal of machine learning for biomedical imaging (2022).
[36] J. M. Slipsager et al. "Quantifying the Financial Savings of Motion Correction in Brain MRI: A Estimate of the Costs Arising From Patient Head Motion and Potential Savings From Implementation of Motion Correction". In: Journal of magnetic resonance imaging: JMRI (2020).
[37] R. Souza, O. Lucena, J. Garrafa, D. Gobbi, M. Saluzzi, S. Appenzeller, L. Rittner, R. Frayne, and R. Lotufo. "An Open, Multi-Vendor, Salli--Feld-Sttrenng Brain MR Dataset and Analysis of Publicly Available Skull Stripping Methods Agreement". In: Neurolmage (2018).
[38] V. Spieker, H. Eichhorn, K. Hammemik, D. Rueckert, C. Preibisch, D. C. Karampinos, and J. A. Schnabel. "Deep Learning for Retrospective Motion Correction in MRI: A Comprehensive Review". In: IEEE Transactions on Medical Imaging (2024).
12

---

[39] Y. Sun, X. Wang, Z. Liu, J. Miller, A. Efros, and M. Hardt. "Test- Time Training with Self- Supervision for Generalization under Distribution Shifts". In: International Conference on Machine Learning- 2020.
[40] M. D. Tisdall, A. T. Hess, M. Reuter, E. M. Meintjes, B. Fischl, and A. J. W. van der Kouwe. "Volumetric Navigators for Prospective Motion Correction and Selective Reacquisition in Neuroamatomical MRI". In: Magnetic Resonance in Medicine (2012).
[41] M. Uecker, P. Lai, M. Murphy, Virtue, M. Elad, J. M. Pauly, S. S. Vasanawala, and M.
Lustig. "ESPIRiT an Eigenvalue Approach to Autocalibrating Parallel MRI: Where SENSE Meets GRAPPA". In: Magnetic Resonance Medicine (2014).
[42] M. Usman, Latif, M. Asim, B.-D. Lee, and J. Qadir. "Retrospective Motion Correction in Multishot MRI Using Generative Adversarial Network". In: Scientific Reports (2020).
[43] N. White, C. Roddey, A. Shankaranarayanan, Han, D. Rettmann, J. Santos, Kuperman, and A. Dale. "PROMO: Real-time Prospective Motion Correction in MRI Using Image-Based Tracking". In: Magnetic Resonance in Medicine (2010).
[44] Yaman,
Hosseini, S. Moeller, J. Ellermann, Ugurbil, and M. Akçakaya. "Self- Supervised Learning of Cuidee Reconstruction Neural Networks without Fully Sam- pled Reference Data", In: Magnetic Resonance in Medicine (2020).
[45] M. Zaitsev, J. Maclaren, and M. Herbst. "Motion Artifacts in MRI: A Complex Problem with Many Partial Solutions". In: Journal of magnetic resonance imaging: JMRI (2015).
[46] J. Zbontar et al. "fastMRI: An Open Dataset and Benchmarks for Accelerated MRI", In:
arXiv:1811 10008999 [physics, stat] (2018).
13

---

A Computational aspects of simulating motion in the image or k-space
D simulate motion in the k-space rather than in the image domain because it is computationally more efficient the number of motion states is larger than the number of coils C. Here, we elaborate on this statement.
As mentioned in the Problem Statement Section
In our work the MRI forward model under motion is implemented with the NUFFT, which for each shot in the sampling trajectory first computes the rotated coordinates based on the k-space data and the motion parameters of this shot. Then the k-space values at those coordinates for all shots can be obtained from single application of the NUFFT. This requires to compute single interpolated version of the k-space data, which consists of C'-anny 3D coil volumes, thus C-many interpolated volumes need to be computed.
In contrast, simulating motion the image domain required computing a transformed 3D image volume for each motion state, which then is expanded to the coil dimension and transformed to the k-space with the forward model (T). Hence, many interpolated volumes need to be computed. As in our work the number of coils C' 12 is smaller than the number of motion states it is computationaly more efficient to simulate motion in the k-space than in the image space.
B Proof of Theorem
In this appendix, we prove Theorem from the theory Section 5 bound the loss for the correct motion parameters, L(m") and lower bound the loss L(m) for all other motion parameters m m".
To prove the result, we upper
Assume without loss of generality that the ground-truth shift is equal to m 0. We have that
L L(m)= = F -yll
2
D D F
UU D Di Dm F7x
Dm F7x
2
Dm
D
F
F7x
UUT F x D F F
bk
SU F, Uc D F, Ue 2
bk
where D is the Hermitian transpose of the matrix D. Here, equation follows from the assumption that the optimal motion parameters zero, thus D I, and equation ii follows from the entries of Dm having absolute value one, and D Dr m I.
We first upper bound L(m") L(0). We have that
L(0)-
72
F,
FUT
- F, Uc
bk
72 bk
F
UU
F+
F Ucl1
2
F U) -1)
Uc
max
bb:
2
bk
d bk:
1 + 2
1 1 2
2
(10)
bk
n
bk
S 3/ bk: 4.
(11)
For the last inequality, we used that d/bk: < 1/4 by assumption. According to Theorem 2.6 in Rudelson and Vershynin (33]. the second to last inequality holds on the events
bk
d
bk
d
E1
1
2 bk
< Jmin (F S max (F S $V
+
2
(12)
bk
14

---

and
B
&g = |||||| 1
(13)
vn
with 3 = Vn. Those events hold with the probabilities, for all [ > 0
2c 1 2 4/2
(14)
2 2 2 c0a
(15)
Next, we lower-bound L(m) for m # O. Let be the number of individual motion parameters in the vector m that are non-equal to the true motion parameters m O. We have
((m)=
=
F
U
U
F
F D m
c
F U
D
FAFUUU
F
bk
(1(
U
D FUFUUU
F-U
n
U
F
F 7 - -)
2
F
D
F.F)
||
||
n2
U
F7U -
)
2
U
D FTU)
c
2 - 1-
U
)
2
bk 1
7 bk
32 n²
|1
a/6/²
(1 + a))
(4b (bb+c d)
8 n
1:212
bk 1
7
8a
/6
(1
a)
(4b d),
n
8
n kb
where the last inequality holds with probability at least e ca + 2d(d + b)e - -e- which follows from the bounds
P |1
0k
2 - 2 (1 a)
F
32 n
(4b +
+d) d) S sea + +dd(d + b)e -0g
k2bz
(16)
and
[[[ D - UU)
///////// $ a/bly
bk 72
< 3e od
(17)
82
Thus, we have with probability at least 1 --a + 2d(d + -ee- -e- cdd 4e-od cdd that L(m) > L(m") = L()) if
bk n 2
bk
g2 n
d bk bk n
|1
a/bl
(1+ a
(46 + d) 12
> 0
(18)
n
1212
which is equivalent to
7
32 n
(4b + d) + 2 d 22
-
2/b2
>
(1 + a)
(19)
4
bk'
By a union bound over all motion parameter m E {O...... 1} \ {m²} (there are n² many), we have that
P
max L(m) 2 L(m") < m
Σ [[[(m) > L(m')l
m
coa + 2d(d + b)e -03² 7e
S nt"
< e - + - 7e -o-od-c od- log(n)b
provided that
blog(n) n² b log(n)
|| 11 a/6|/ >
12 (4b + d) + 24 bk
(20)
n
which concludes the proof. It remains to prove the intermediate results, which we do next.
15

---

B.1 Lower bounding the first ferm, proof of equation (17)
Since the entries of D have absolute value one, and a/b E [0, 1]. we have
U a D F.U)
2 a F, Ucll2
b
2 | a/blamin (Frr U>]||1l12
bk
bk: (1
B
2 |1 - a/bly
Vn
bik 72
2 | a //yl
82
where the second to last inequality holds with probability at least 1-11- 1- od according to equations and (15). and where we used that d < 1/64 and S _
14
B.2
Lover-bounding the second term, proof of equation (6
Define A
U F D F U
8I for notational convenience. We have that
32 n² ||c|| 2 2 (1 + a)
P
(4b + d)
k2n2
< P
| d
82 n²
(4b + d)
+P
|Acl 2 2 (1
1
1:212
P
|
8² n
(4b + +d)
+P |Ac1 2 1
-|Al|=~ N 11
| |
k2b
S + 2d( d + bie
=22a
where the last inequality follows from the Hrnson- iniquulllt as well as from
P
|A|| 2 n k2b (4b + d) s d2(1 + b)e-
2
-8² + 2220-c8 cB?
(21)
We next prove the bound 21p.
We start with upper bounding the Frobenius norm of the matrix A. We split the squared Frobenius norm into sum of the squared diagonal entries and squared off-diagonal entries according to
||
n kb
U F
D
F
1 F
51
m
d
DuFFDD 2
u; F
D x F
Fuu
kb
kb
It follows that
P
|
2 2
82 n
4bb + d)
SP
A
2 F
2 d
( 1+ n
kvb ))²
+ + -1) d((d-1)
12
2 922
n 22
Vn
kb
n
n
a 2
3
/
n
27
<
P
u. F
D m
Fru
u;
2
1+
kb
0
1/2
kvb
i=1
Σ
d
P
ui F D
)u,
2
132
kb
kb
n
1=1 jデl
< d2(1 + b)e cs2
-39
+ t -88
The last inequality follows from
P
n
u
F
D
Fuu
F
u
a b
>
B (
1
n )
- 2 + -B
(22)
kb
Vn
kvb//
16

---

and
P D F Fuuj 2
B vn
s 3e -8
(23)
Bounding diagonal entry, proof of inequality (2 : First, consider a diagonal element and let Ze E R be the mask that selects the frequencies in the set Te, and recall that F E x is the Fourier transform. With slight abuse of notation, we let D., be the diagonal matrix that contains the frequencies that matrix the Fourier matrix it is multiplied with, i.e., in D,, F, the matrix D., is the n X n diagonal matrix with entries e 22 . [= .,..., the [T] x [T] diagonal matrix with entries e/2mmt/n { E T. For convenience, we drop the index and write
1, and in D.. Fr, the matrix D is
us. With this we have
Y1 k-b
a
71
11 F D F
b
kb
11
F.
Te
mm FT,
t=1
u
"
Ze
Fu
kb
my
a
+
b
71
u
F'
Zi
-I
k
D my
Fu + u
F'
Fu
kbb
n
b
t=1
h I
D ū+
E
Σ u'F'D
Fu
(24)
kb
n
b"
Here, the entries of ú Fu E orthonormal columns.
RD are iid CN((0, 1/ /n) distributed, since the DFT matrix F has
Thus, by union bound
P
n
u
F
D
a Fru:
1+
n
kbb
b
Vn
kvb
n2
k
b,
B
P
Ze
I
D
+P
F D
Fu
>
kb 1=1
n1
kb
vn
b
Vn
$ 2e
cBa 008
+
62e
c0a
2(1 +
b)e
c3a
(25)
where we used that, for all
0,
Vb3
< 2c0
P
Zi
I
D m.
2
(26)
n
n
and
Σ
b
D m.
Fu
2
8
S
b2e-c82
b2e
(27)
n
This concludes the proof of inequality
22
remains proof equations (6) and
27
Proof of equation
26
Note that
k D
k (2b
Zo
I
Σ
it uee
2xemi/n
(28)
is sub- uub-Gaussian zero random variable concentration inequality for sub- Gaussians yields
Since 26,1
h:
B
cL8/vR? la 0iz
P i
Z,
D
2
< 2e
< 3e 3c
(29)
Vn
17

---

Here, we used that ü is the entrywise product, and the random variable | -ull - (ū, i4) concentrates around its expectation n3o = n3/n² 3/n.
Thus we have shown that the random variables 89 (Ze - 1) D (conditioned on u) are sub-Gause Gabssansn.. The random variables are also zero-mean and independent (if conditioned on û), and thus by concentration of Lub-Gaussian random variables we get
Σ
u
Ze
1 n
D me
ú
2 Vn
3vb
S 2o
This concludes the proof of equation 266
me Fu Fu tion (24) If my 0, i.e., there is shift, have Do and thus the random variable becomes ū'u which is a sum of sub-exponential random variables and concentrates around 1. In case m is an integer non-equal zero, we have that F D Fu u Umme where Um, vector circularly shifted by mg. We can write u Um, as two sums of independent Gaussians, see this consider the case my = I and note that for this case we have
Proof of equation (7 : Next, consider the random variable u'F'D"
in equa-
T Umme
(1ln2 + 43 + +...) (u243 + u445 + ...).
S
S2
By the union bound and by Hoeffding's 's inequality, we get that
P I Umee 2 23/vnl s P P 2 8/vn| + +
8/Vn
2e
(30)
Combining this via a union bound for all summands & = 1,...,b yields the bound in equation (27
Now consider off-diagonal element ü: D m ūj. Since ú, and ü, are independent and the entries have zero mean and variance 1 n, this is sum of Gaussian random variables with variance 1/n. Thus, by sub- Gaussian concentration or Hoeffding's inequality, we have that
Bounding an off-diagonal element, proof of inequality 2,3
c0
P
ù D
<
3e
1
2c -002
(31)
vn
where we used that |[D,, u 1/2 | concentrates around
B.3 Comment on Equation
In the main body, stated Equation i.e.
f(Fry) ~ x,
(32)
where T U...U is the of all measurements collected.
To see that this approximation is accurate, note that for the noiseless case with a known shift, where y =D. Uc the network approximately reconstructs the signal since
[(DnF ((F)Din)) UU F D.. UC
(33)
(34)
UU F, Uc
(35)
(36)
where the first equality follows because the matrix Dm F has orthonormal rows, and the approxima- tion holds since F concentrates around R5 if U random subspace, and if the number of measurements, T is sufficiently large relative to the dimension of the subspace, d.
18

---

C Ablation Study on Pre-training Loss
In Section we stated that using training loss that consists of the two losses, one computed in the image domain and one in the measurement domain, is beneficial over using only loss in the measurement domain. this section, we conduct the corresponding ablation study. We evaluate three distinct loss functions: image domain loss, k-space loss, and combined loss. The image domain loss is follows:
N
Cuain ( (0) Σ
(lo(AAy) - /x1x|| ==1
(37)
and the k-space loss is
N
Lirain (a)in(0)))))) = Σ i=1
|[Efo(Aly))
FEx; | /||
(38)
To compare which two loss functions or the combination of them is best, we trained three U-Net models with each loss function. All models were trained under identical settings, as detailed in Appendix D with the exception that the model trained with magnitude loss had a single output layer representing the magnitude of the MRI image.
We evaluate the performance of the models in terms of their reconstruction quality as well as as component of Motion First, we measure the reconstruction quality on motion free data. Second, we apply the U-Net within the MotionnIT1-Th-L/ framework on data with interleaved inter-shot motion at severity level 9, following the same setup as described in Section 6..] Due to the requirement for complex-valued U-Net in the Motion framework, the magnitude-only U-Net can not be used within MotionTTT. The results are presented in the table below and show that the combined loss function provides the best performance for both motion-free reconstructions and when used within the Motion framework for correcting motion-corrupted data.
U-Net Training Loss
Image Domain Loss
k-space Loss
Combined Loss
Mrteon-free PSNR
36.64 Not Applicable
36.59
36.73
MotionI71-7hh-I (Severity 9) PSNR
35.23261
35.23587
D Hyperparameter configurations and implementation details
In this section we provide additional information for the experiments presented in Section inclluding hyperparameter configurations and implementation details for our proposed Mo and the baselines alternating optimization and E2E stacked U-net and the MRI sequence parameters used to acquire the data for real motion experiments.
D.1 MotionTTT Implementation details
In this section we discuss implementation details for the three components MotionTTT pre-training, test-time-trainine for motion estimation and reconstruction together with some computational aspects.
D.1.1 Pre-training
We train the standard U-net from the fastMRI repository (46) (MIT license) with 48 channels in the first layer and 4 blocks in the nown-/up-sampling part resulting in 17.5M network parameters. The real and imaginary parts of the complex valued input and output images are processed in two network channels. We train for 240 step with the Adam optimizer with learning rate 0.001 which is decayed once by a factor of 10 after 200 steps. In every step one of the 40 training volumes is loaded and in each plane Tz Ty. Fz r= and ry r. 20 random slices are backpropagated in three separate batches, i.... 3 gradient steps per step. The model was trained for 17h on a Nvidia RTX A6000 GPU.
Throughout (pretraining and test-time-training the sensitivity maps for the experiments with simu- lated motion are compute from 24 x 24 x 24 auto-calibration lines of the originally fully-sampled motion-free free k-space using ESPIRIT 41 with the BART toolbox
19

---

D.1.2 Test- test-time-training for motion estimation
To perform motion parameter estimation as outlined in Section 4 we use gradient based optimization with the Adam [19] optimizer. Phase runs for 70 iterations with an initial learning rate of 4.0, which is decayed twice by a factor of 4 at iterations 40 and 60. The DC loss threshold used to determine incorrectly estimated motion states after phase is set to 0.575. no motion states fall above the threshold, the optimization continues for another 30 iterations with one additional learning rate decay after 10 iterations. If motion states fall above the threshold, phase runs for 30 iterations with a learning rate of 0.5. Finally, phase 3 runs for another 30 iterations with leraning rate of 0.05.
For severe motion we found that optimizing only over rotation parameters for the first few steps facili- tates their correct estimation. To avoid single motion parameters to get stuck at a large value early dur- ing the optimization we clamp the estimated motion parameters at [5,8, 10,12,11 degrees/millimeters for steps smaller than 45,,00,5,600, 150].
We use TorchKbNufft (MIT license), an implementation of the NUFFT from Muckley al. (2 and use the option for density compensation based on the method of Pipe (31] when applying the adjoint NUFFT. To allow differentiation with respect to the input coordinates we build on the extension from Alban Gossard (MIT license).
D.1.3 Computational aspects
We run MotionTTT on a Nvidia L40 GPU with 46GB memory or on Nvidia A100 GPU with 80GB memory. Within our implementation there are three hyperparameters that control the required GPU memory. Recall that in every iteration of MotionTTT the entire 3D volume is reconstructed slice-wise. However, gradients are only computed for subset of randomly selected slices of size 5 as described in Section 4 The size of this subset is the first hyperparameter that we can control.
The second parameter is the batch size of the NUFFT. As described in Appendix [A] the NUFFT is applied for each of the C coils. The TorchKbNufft package allows batch wise computation the cost of increased GPU memory utilization.
As in our 3D setup we have to deal with lot more motion states (B- 50) than previous work that studied the 2D sctup, we implemented a third option to reduce GPU memory consumption. To this end, we split the estimated motion states into two batches of size of, e.g., 25 each and backpropagate the gradients subsequently before performing a single optimizer step that updates the estimated motion states.
Note that hyperparameters batch size of NUFFT and of motion states per backpropagation only affect the run time but do not change the optimization problem, where the cost of the latter could be compensated by increasing the number of GPUs accordingly. Hence, at the cost of prolonged run times or larger GPU cluster MotionTTT can be applied with any number of motion states. With our hardware and for our = 12-many coils we could use a NUFFT batch size of 12 (4) and batch of 50 (25) motion states for backpropagation in case of the A100 (L40) GPU.
D.1.4 Reconstruction
We perform DC loss thresholding for excluding the k-space data acquired during the i-th shot from the reconstruction based on its estimated motion state m, and the DC loss (5) (m))(m:) > 8 with threshold of 0.575.
We perform L1 l-miiimization with wavelet regularization based on the estimated motion parameters. We run 50 steps with SGD and learning rate of x 10" and regularization weight À 10-3
The U-Net reconstruction given the true motion parameters KnownMotion Dei-nettDLLand from Figure 4 is obtained by fine-tuning DCLayer as proposed in Chen al. (4) If the U-Net reconstruction is XU.Net Sa (At (T, -m)y). then the reconstruction of the DCL.yer is:
ATT m)x - yll, A
x Xu-Nerll
x arg min
yyl,,
|x
In the experiments, we set À = 0.1. The choice of learning rate and number of steps is critical for optimizing the DCLayer. After grid searching, we identified the optimal learning rate and steps for various severity levels, which are summarized in the following table:
20

---

Severity Level 0,
2,3,4,
6.7
8,
Learning Rate Steps
x 10th
1x 10th
1 x 10
5 10
20
50
100
50
D.1.5 Hyperparameters for real motion experiments
For applying TT to the real motion-corrupted data in Section 6.3 we use the same hyper- parameter configuration as described above up to choosing a slightly smaller initial learning rate of 1.0 during phase to explore the space of motion parameters more slowly and set the threshold parameter to 0.70 due to a generally higher DC loss level of the scanner data compared to the data used for the simulation.
For obtaining the final reconstruction with Ll-minimization based on the motion parameters estimated with MotionTT we set the hyperparameters learning rate to 10 and regularization weight to X 10
D.2 Alternating optimization Implementation details
To perform alternating optimization as described in Section 6.1 we run SGD with a learning rate of 5 x 10" and regularization weight À = 10 during the reconstruction steps and a learning rate of x 10-11 during the motion estimation step. In both steps the loss is the MSE between predicted and given measurement. The optimization process is capped at 500 iterations, but it terminates early the reconstruction loss falls below the threshold of
After alternating optimization we perform L1 .l-minimization from scratch based on the estimated motion parameters. We run 50 steps with SGD and learning rate of 5 10? and regularization weight X = 10-3
D.3
E2E stacked U-net - Implementation details
We train the stacked U-net with self-assisted priors proposed in Al-Masni et al. [T] as an example from the class of methods for end-to-end motion correction for 3D MRI. Their method consists of stack two U-nets that are trained to map the absolute value of a zero-filled (ZF) reconstruction of the motion corrupted and undersampled measurement the absolute value of the motion-free image enabling end-to-end motion correction with a single forward pass through the network. The network operates slice-wise however neighboring slices provided context at the network input to account for the 3D nature of the problem.
We train the E2E stacked U-net the same dataset and in similar way as trained the U-net on motion-free data in Section D.I.I In every step one of the 40 3D training volumes loaded from which the fully sampled target volume is computed as well as three different input volumes the motion-free undersampled ZF reconstruction and two different motion-corrupted undersampling ZF reconstructions generated with the inter-shot motion simulation described in Section 6. with random level of motion severity and a fixed motion severity level respectively. For each training example and level of motion severity a fixed set of three different motion trajectories is considered during training.
We include the task of motion-free and motion severity of level reconstruction in every training step to ensure that the model does not loose too much performance when reconstructing motion -oreion-free data compared to a model trained only for motion free reconstruction and to bias the model towards performing well for very mild motion artifacts as it the regime in which end-to-end methods can still provide reasonably good results, whereas for severe motion the reconstruction results are far from a potential use for medical diagnosis. See Figure 7 for examples of mild and very severe motion.
For each of the three degraded volumes 20 random slices are selected in each plane X Tgg Th and ry X r= resulting
3 20 180 pairs of undersampled and potentially motion corrupted training inputs and motion-free fully-sampled training targets per training step. As we train with batch size of 20, network parameters are updated times per training step and 40 360 times per epoch.
21

---

We train the model for 230 epochs with the L1 loss from equation (37) and the Adam optimizer with a learning rate of 10-4 which decayed twice by a factor of 10 epochs 200 and 220 and pick the model from the epoch with the highest reconstruction PSNR for motion-free reconstruction on the validation set.
We adopt the network design from Al-Masni et al. [T] where we set the number of channels in first layer of both U-nets 64 resulting total of 15.9M network parameters. We used instance norm instead of batch norm in our network as we found it to give more stable results across levels of motion severity.
The final motion-corrected reconstructions are obtained by reconstructing the 3D volume slice-wise in the axial rN ry plane. Before the PSNR scores shown in Figure are computed the reconstruction aligned with the motion-free reference volume.
D.4 MRI sequence parameters for real motion experiments
In this section we provide additional information regarding the acquisition of our own data used in Section 6.3] This study was exempt from Institutional Review Board (IRB), but potential risks were disclosed the subject and experiments were conducted with informed consent. Data was acquired on Ingenia Elition 3.OT X scanner (Philips Healthcare, Best, The Netherlands) using the standard 16-channel dStream HeadSpine coil array, where = 13 channels were used during acquisition. We perform 3D T1 -e-weighted Ultra-fast Gradient-echo (TFE) imaging with a Imm isotropic resolution and matrix-size of k x ky x hig 222 x 236 x 512, an undersampling factor of 4.94 and linear sampling trajectory illustrated in Figure (d). We subsample the data by a factor of two along the sullessampled frequency encoding dimension k2 to obtain a similar field of view as in our training data with k- 256. In one shot 204 lines in the k-space are acquired resulting in total number of 52 shots. See Table for an overview of all sequence parameters.
Table I: Sequence parameters used in the real motion experiments.
Parameter
Value
Sequence
3D TI-THE
Flip angle (deg)
8
TR (ms)
6.7
TE (ms)
3.0 (shortest) non-selective invert 1060 ms
TFE prepulse/ / delay (ms)
Min. TI delay (ms)
707
TFE factor
204
TFE shots
52
TFE dur. shot / acq (ms)
1742 / 1347
Shot interval (ms)
3000
Sampling
Cartesian
Under-sampling factor
4.94
Half-scan factor Y /
1 / 0.85
Number of auto-calibration lines
Profile order
random
FOV (FH x AP x RL, mm)
256 221 170
Acquisition matrix
256 221
Fold-over direction
AP
Fat shift direction
F
Water-fal fat shift (pixels)
1.6
E
Additional experimental results
In this section we provide additional ablation studies and further qualitative examples to complement
the experimental results presented in Sections 6. and real-motion respectively.
6.2 and 6.3 on simulated inter-/intra-shot motion
22

---

E2E Stacked
AltOpt+ Th-L1
MotionTTT-
Mot Th-L1
KnownMotion L1
KnownMotion- U-net-DCLayer
Reference
11
U-net
img.
PONE
PAME
PAGE
PANN
PARE
PARN
PSNR
PANE
PSHD
PEND
PSNN
PSMB
Figure 7: Visual comparison with reconstructions and difference images for simulated motion of severity level (first two rows) and severity level 9 (last two rows) for all methods presented in Figure
E.1 Additional inter-shot results
We present additional results on inter-shot motion estimation, analysing reconstructions at different levels of motion severity. Qualitative comparisons for mild and severe cases highlight the strengths and limitations of the motion estimation methods used.
E.1.1 Additional qualitative results
Figure in Section 6.1 in the main body shows reconstructed images for a simulated motion severity level 9.
Figure 7 shows additional reconstruction results at a lower severity level (level 3); for this lower motion severity level both MotionITT+Thh-L and AlrOpt+Th-LI achieve results comparable to KnownMotion-LI.
Figure & shows an example of predicted motion and corresponding DC loss for simulated inter-shot motion scenario at severity level The DC loss effectively detects incorrectly estimated motion states, highlighting their locations. This capability is particularly useful for DC thresholding, which improves robustness, as discussed Section 6. In this scenario, MotionTTT failed in only one shot, whereas the AltOpt method failed in 12 shots, further demonstrating the effectiveness of the MotionTTT approach.
E.1.2 Comparison of Computational Time
Beyond reconstruction performance, computational time an important factor for real-world applica- tions. If the reconstruction process is too slow, the algorithm may be impractical for clinical use. The following table summarizes the average computational times for the AltOpt and MotionTTT methods, tested an Nvidia A A100 GPU. As shown in the table, our Motion method is approximately 6 times faster than AltOpt when early stopping is applied, Without early stopping, AltOpt even 10 times slower than MoL.
23

---

lgg trans.
by trans.
lgg trans.
Rg-Ky rot.
ky rot.
R2-ka rot.
10
0.8
Motion parameters( (wwiiiersmm//egreeeeee
0.6
30
0.4
10
550)
10
0.8
My
0.6
10
0.4
20
40
0
20
40
0
20
40
20
40
20
40
20
40
Motion states
GT motions
Estimation enror
DC threshold
Figure Example of simulated inter-shot motion trajectory (GT motions) for severity level corresponding the example in Figure 5 Our Motion]TT (first row) estimation fails only for single motion state, whereas alternating optimization (second row) fails at recovering several motion states. The corresponding DC losses and the DC threshold indicate which shots are excluded from the reconstruction.
Method
AliOpt(full run)
AltOpt (early stopping)
Motion 25 minutes
Average Running Time 4 hours 15 minutes
2 hours 39 minutes
E.1.3 Ablation studies on the acceleration factor
In this section we investigate the role of the undersampling factor on the ability of MotionTTT to estimate inter-shot motion. We re- re-train the U-net on two additional Cartesian undersampling masks with acceleration factors 2, addition the existing results with acceleration factor 9 b,c for the masks). Figure TO shows the reconstruction performance in PSNR based on motion parameters estimated by our Motion compared ground truth motion over three levels of motion severity and the three acceleration factors.
(see Figure
As expected, the overall performance decays with increasing acceleration factors and motion severities. For mild and moderate motion, MotionTTT achieves highly accurate motion estimation for all acceleration factors indicated by the vanishing performance gap relative to using ground truth motion.
For the most severe motion, small performance gap exists for all accelcration factors due to incorrectly estimated motion states that discarded from the final reconstruction via DC loss thresholding. In fact, under severe motion an average of 2.5/100, 2.0/50 and 0. 12/25 shots have to be discarded for acceleration factors 2,4 and 8.
We conclude that MotionTTT can achive highly accurate motion parameter estimation robustly across different acceleration factors. We attribute the slight increase in discarded motion states for smaller acceleration factors to the increased complexity the optimization problem as the number of unknown motion states to be estimated increases linearly in the number of acquired shots.
E.2 Ablation studies for intra-shot motion estimation
In this section present addition ablation studies for the choice of the number of motion states Noplis estimated per shot and the sampling order used in our experimental results on simulated intra-shot motion estimation in Section 6.2
We also show reconstruction results from the experiments in the
main body in Figure 11
F.2.1 Number motion states per shot
We start with an ablation study the choice of the hyperparameter Noplitc that determines the number of motion states that are introduced at the end of phase of MotionTTT's optimization scheme outlined in Section 4 for each shot that exhibits data consistency loss larger than certain threshold.

---

100
25
50
20
60
15
40
10
20
Ky
ky
Ky
a) Interleaved, R
2
b) Interleaved, R
8
c) Interleaved, R
50
50
50
30
30
20
20
20
ky d) Random, R 4 c) Linear, 4
Kv
ky f) Determinsite,
Figure 9: Undersampling masks used in the ablation studies in Appendix for different acceleration factors R {2, 4, 8} with corresponding number of shots f25, 50, 100) such that constant number of k-space lines is acquired per shot. The color coding illustrates the sampling trajectory (interleaved, random or linear) indicating which k-space lines are sampled within the same shots.
KnownMotion-LL1, R
38
PSNR (dB)
MotionITT+Th-LI.. R
KnownMotion-LL1,
36
MotionIIT+TTh-.II,
KnownMotion-LL1,, R
34
R=
32
3
7
9
(10.2)
(10.5)
(10,10)
Level of motion severity
Figure 10: Performance of LI -minimization with known motion versus with motion estimated by TTT over three different levels of motion severity (defined in Figure 4 for acceleration factors R = 2///8. Results are averaged over validation examples with 2 motion trajectories each.
MotionTTT+LI
MotionTTT+L1 (Phase3)
MotionTTT+Th -L1(Phasel)
MotionTTT+Th -L1(Phase))
KnownMotion-
Reference Img.
(Phasel)
18:35
PENR
PAHR
PANR
PSNR
POUR
Reconstruction
Severity Level 6
Difference
Figure 11: Reconstructions and difference images for simulated intra-shot motion severity level 6.
25

---

Table Reconstruction performance of MotionITT+Th--L PSNR averaged over four validation examples and five randomly generated motion trajectories per example together with the average number of discarded k-space lines due to DC loss thresholding before the final reconstruction. We compare the performance of four different sampling orders random, deterministic, interleaved and linear for intra-shot motion correction.
Sampling order
Random
Deterministic
Interleaved
Linear
PSNR (dB) # of discarded lines
36.05
36.06
35.67
33.52
197
177
516
3389
As discussed in Section 6.2 the choice of this hyperparameter trades off the irreducible discretization error versus the available signal per estimated motion state and the computational complexity.
The discretization error results from estimating only Spspiss -mayy motion states for a shot that when affected by intra-shot motion exhibits a distinct motion state for each k-space line (182 in our setup) acquired within this shot. The error decreases with increasing N and depends on severity of motion as fast movements with a large amplitude result into a large discretization error.
On the other hand, increasing Naplis decreases the amount of k-space signal per estimated motion state potentially leading more incorrectly estimated motion states. Additionally, doubling the number of motion states Naplas doubles the computational complexity of phase 2 if the available hardware does not allow for parallelization (see Appendix D.1.3 for discussion of computational aspects).
To find the value for Noplas that trades off those two effects, we conduct the following experiment. We simulate motion trajectories containing 5 intra-shot motion events following Section 6.2 with maximal motion Mmax 5. We assume that all motion parameters are known except the ones during the intra-shot events. We then apply Motion]TT+ Th-LL with a random sampling order to estimate the motion states during intra-shot motion for different levels of discretization defined by the number of motion states per shot Naplits E (5, 10, 20). This simulates the case where phase of the intra-shot motion estimation scheme perfectly estimates the inter-shot motion parameters and detects all shots corrupted by intra-shot motion for which then motion parameters are estimated during phase 2. This allows us to focus the evaluation solely on the ability of the method to perform intra-shot motion estimation independently from the performance in phase 1.
We obtain PSNRs averaged over the four examples in the validation set and five simulated motion trajectories per example as 35.84, 36.05 and 36.07 for Nepins = 5, 10 and 20. Further, the average numbers of k-space lines that are discarded due to DC loss thresholding before the final reconstruction are 289, 197 and 302 out of 5 * 182 910 lines affected by intra-shot motion.
For Nopins 5 we obtain the lowest performance, which we attribute to large discretization error, which even results into more motion states with a DC loss above the threshold than for Naplits = 10. For Nyplits 10, 20 we obtain the same performance, however due to different reasons. While for Nsslins = 20 a smaller discretization error can be achieved, more motion states are estimated incorrectly and are discarded due to more difficult optimization problem as more motion states are estimated with less signal per motion state. We use Noplis = 10 for our experiments in Section 6.2 as it corresponds to smaller computational costs for the same performance.
E.2.2 Sampling order
Next, we present ablation study on the role of the sampling order for the ability of MotionTTT to estimate intra-shot motion as in Section 6.2
In order to focus our evaluation on the performance for intra-shot motion estimation, we consider the same experimental setup as in Appendix [E.2.]] where we simulate motion trajectories containing 5 intra-shot events with maximal motion M 5, apply MotionTT only to the motion states affected by intra-shot motion and assume all other motion states to be known. We fix the number of motion states per shot to Nepliss = 10 and report the performance for different sampling orders including the interleaved and random sampling orders used for the inter- and intra-shot motion experiments, as well linear and deterministic order. See Figure 9 masks c)-c) for visualizations.
26

---

The linear order acquires k-space lines according to their ky index from low to high. Hence, all shots towards the start and end of the scan only contain high-frequency components. The interleaved components evenly across shots, but within one shot the order follows the 2i index from low to high. Hence, all k-space lines acquired towards the start and end of one shot are high-frequency components. The random order acquires all k-space lines at a random order. The deterministic order we designed such that the average distance to the center of the k-space of two k-space lines acquired after each other is approximately constant, which ensures that after the acquisition of a higu-fryy components a low-frequency component is acquired. The random, interleaved and deterministic orders all start by sampling the 3 x center of the k-space.
order distributes Segh-/low-frequene
Table 2 shows the average performance of Motion]TT+ Th-LL in PSNR for each sampling order as well as the average number of k-space lines that are discarded due to the DC loss thresholding before the final reconstruction.
While random and deterministic orders perform equally, the interleaved order results in more in- correctly estimated motion states, For the linear order we obtain a significant loss in performance, and over a third of the 9078 lines acquired in total are discarded including many lines that are not affected by intra-shot motion and for which we assume knowledge of the true motion state in this experiments. In fact, under the linear order entire shots at the start and end of acquisition are discarded due their large DC loss despite assuming knowledge of the motion states. We attribute this to the U-net used during MotionTTT reconstructing low-frequency components, and conclude that in order to reliably estimate the motion state of a batch of measurements the batch needs to contain low-frequency components.
components not as faithfully as
The deterministic order fulfills this requirement by design and the random order with high probability as for the number of motion states per shot Naplins = 10 the smallest batch of measurements we consider consists of about 18 k-space lines, which is large enough to contain both high- and low- frequency components especially as the center of the k-space is much more densely sampled than towards the edge. In our experiments we hence use a random sampling scheme due to its ease of implementation on top of any arbitrary design of the undersampling mask.
E.3 Results from Real Motion Experiments
We now discuss the real motion experiments from Section 6.3 in more detail.
Visual reconstruction analysis. Figure 12 shows the reconstruction of the volume in three orthog- onal planes (axial, sagittal, and coronal). For mild motion, artifacts are still noticeable, particularly in the ky-k; plane. However, the MotionI7TTLL method effectively mitigates most of these artifacts, resulting in much clearer reconstruction than the other methods. For strong motion, many details across all three planes are entirely occluded, yet the MotionITT-LI method significantly improves image quality, restoring visibility to numerous details that would otherwise be lost due to severe motion.
Estimated motion parameters and DC loss. Figure 13 shows the estimated motion parameters as well as the DC loss (5) per state for the strong motion example. The number of motion states is 69, which results from starting phase 1 of MotionITT with 52 motion states (one per shot), of which four exceeded the threshold after phase 1. As the number of additional motion states per corrupted shot is Nyplits = 10, phase 2 and 3 continue with 92 motion states from which 23 were discarded during the final DC loss thresholding resulting in the 69 motion states depicted in Figure [3] The areas shaded in gray correspond to the motion states that were added after phase 1, i.e., intra-shot motion states, whereas the white areas correspond to motion states each pertaining a single shot. Consequently. the predicted motion and the DC loss of Mot (phase 1) is constant within the gray areas as intra-shot motion estimation is performed starting from phase 2. As we can see intra-shot estimation, i.e. Motion (phase 3) significantly reduced the DC loss compared to phase and the estimated motion is reasonably smooth, indicating the benefit of estimating intra-shot shot motion in practice.
27

---

Mild Motion1
Mild Motion2
Mild Motion3
Strong Motion
L1 (rxff plane)
No motion
Mation TT- Th-L1
(Gery plane)
L1 trxtz plane)
MotionITT+Th-L (refz plane)
(rrfzz plane)
L1
MotionITT+Tho Th-L1 (rriz plane)
Figure 12: Visual comparison in three planes of reconstructed data corrupted with real mild/strong motion and no motion. Our MotionI'TT-L achieves significant gains in image quality compared to L/ without any motion correction.
28

---

ky trans.
By trans.
lg trans.
0.8
0.7
mm ww Vnn
MAy My
ww
Motion parameters (mm//egrees)
ww
hmm
0.6
0.5
kg K.
By kgg rot.
le kgg rot.
0.8
0.7
ww
Momn
ww
0.6
0.5
60
20
40
60
20
40
60
Motion States
Motion params. (Phase
Motion carams.(Plase 3)
DC loss(Phase 1)
DC loss(Plase 3)
DC Thresbold
Figure 13: Estimated motion parameters and DC loss of TTT phase and 3 for the data from Figure 12 (strong motion). Gray areas indicate intra-shot motion states and white areas motion states that correspond to single shot each.

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstracts claims demonstrate the capabilities of our new method with simulated and real motion data. Experimental results for both are described in Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Limitations regarding the performance of the reconstruction module well as the dependency on fully-sampled training data are discussed in Section 7
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
29

---

Answer: [Yes]
Justification:
The proof Theorem T contained in Section
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof's provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The different elements of our new method are outlined in Section 4 and hyperparameter configurations and implementation derails are carefully described in Sec- tions 66. 6.33 and Appendix [D] The training data we used is publicly available and we publish our code for the entire MotionTTT pipeline including training, tetetiig-rrainngg and reconstruction https: bocctps:/giinn /ttp:://gttttbbub www.MLI Lat/Maa_Mattionnntt Guidelines:
Justification:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
30

---

reproducible, they 5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: The data we used [37] for training and the experiments with simulated motion is publicly available and we publish our code for the entire MT pipeline includ-
ing training, test-time-traiime trainengriinng and reconstruction bttps://ggtthn
com/MLI- 1ab Lb/MRI_
MotionTTY Unfortunately, we can not release our data on real motion-corrupted MRI due to privacy issues.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
7 //nips.cc/publcc///
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Hyperparameter configurations and implementation derails are carefully de- scribed in Sections 6.3 and Appendix [ Dataset splits and simulated motion trajectories can be found in the code repository htttps://githute cow.MIII Waw.LI-aaaa/MIMMMSSSS- MotionnTT
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: For the main results in Figure 4 provide error bars in form of the standard deviation from the mean PSNR over the different examples in the test set and randomness in the generation of the simulated motion trajectories.
Guidelines:
31

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi-
dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots. The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Training costs stated in Appendix Daand the run times of our Motion and the baseline in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer:[Yen Yes]
Justification:
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
32

---

Justification: The proposed method is targeted at the specific problem of correcting motion artifacts in MRI, which potentially improves the quality of medical diagnosis and increases the effective patient throughput of the MRI machine as discussed in Section We can not see any negative societal impacts arising from the proposed approach.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The license of the used dataset 337 is mentioned in Section 6.1 software packages used are cited with license Appendix D
Specialized
Guidelines:
The answer NA means that the paper does not existing assets.
33

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi thcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
Justification: We release the code for our method /bttps://github. www/cmmm//////.... lab/ MRI MotionTTT The method itself is described in detail in Section|Jann and instruction on how to use the code can be found in the repository.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [Yes]
Justification: One subject participated in the real motions experiments; compensation was not provided due to the negligible effort and risk of harm; details about instructions given to the participant provided in manuscript Section 6.3
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
34

---

Answer: [Yes]
Justification: As stated in Section D.4 the study where we acquired real motion-corrupted data exempt from Institutional Review Board (IRB). Nevertheless, the authors ensured that all experiments were conducted with the utmost care and respect for human subjects, adhering to the principles of personal autonomy, informed consent and minimization of risk of harm. Precautions to minimize risks and ensure the integrity of the research were undertaken by the authors."
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
35


Reviews:
[{"review_id": "UOvYHmrpKn", "reviewer": "Reviewer_fV1o", "summary": "The paper presents MotionTTT, a deep learning-based method for estimating and correcting rigid motion in 3D MRI images. The approach leverages a neural network pre-trained for 2D motion-free image reconstruction and employs test-time-training (TTT) to estimate motion parameters from motion-corrupted 3D measurements. The effectiveness of the method is demonstrated through evaluations on both simulated and real datasets.", "questions": "1. The idea behind minimizing loss (4) is not intuitive and clear. How do you prove the effectiveness of using loss (4) to recover and remove the motion corruption? Will the effectiveness be affected by the undersampling factor and trajectory? What if you have a higher or lower factor? or maybe a spiral trajectory? Can you find the optimal factor for your method? \n2. Based on the assumption that rigid motion leads to rotation and phase shift in k-space, is the linear trajectory the optimal undersampling trajectory? \n3. Do you really need the fully sampled k-space data to learn $f_\\theta$?", "limitations": "The indirect TTT optimization method lacks theoretical justification. The authors should investigate both theoretically and experimentally why this approach works and under what conditions it is most effective.\n\nThe performance of MotionTTT may be heavily dependent on the under-sampling pattern of the k-space and the specific regions affected by motion. The method’s effectiveness might vary significantly with different sampling patterns and motion artifacts.\n\nThe method relies on a pre-trained network using fully-sampled data, which may not always be available in practice.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The paper proposed a novel approach for 3D rigid motion estimation using a neural network trained on 2D motion-free images and TTT for motion parameter estimation. The application of test-time-training for motion estimation is a novel contribution that effectively addresses motion artifacts in 3D MRI images, and outperformed classical alternating optimization methods in terms of speed and accuracy, especially under severe motion conditions.", "weaknesses": "The theoretical foundations of MotionTTT are generally robust, but there are notable gaps. The method builds on existing neural networks for 2D image reconstruction, extending them to handle 3D rigid motion estimation. \n\nDuring the pre-training step, $f_\\theta$ is trained to map under-sampled k-space data back to fully sampled k-space data. The authors show that under-sampled k-space data with motion (where certain regions of the k-space undergo rotation and phase shifting) results in higher reconstruction loss for $f_\\theta$. However, this indirect optimization lacks theoretical proof and needs more robust experimental validation. \n\nFor instance, the performance of this method may heavily depend on the under-sampling pattern of the k-space (such as the undersampling factor and the linear sampling trajectory) and the specific regions in k-space that are affected by motion. However, in the experiments, the setup of the undersampling of the k-space was fixed. \n\nA more thorough theoretical and experimental investigation into why this optimization works and under what conditions it is most effective would strengthen the soundness of the method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the feedback and for acknowledging the novelty of our work. In the following we address the weaknesses (W), questions (Q) and limitations in the order as raised by the reviewer.\n- **W 1, lack of theoretical investigation of why the optimization works:**  \nTo get an understanding on why the proposed optimization works, we developed the following theory for a (very) simplified setup that models our approach. This setup also helps to understand the necessity of defining a learning rate schedule that in the presence of motion explores the loss landscape with an initially large learning rate before gradually decaying it. We will include this result and discussion in the revised version of our paper.   \n&nbsp;  \nWe consider the signal $\\mathbf{x} \\in \\mathbb{R}^n$ that lies in a $d$ dimensional subspace, i.e., the signal is generated as $\\mathbf{x} = \\mathbf{U} \\mathbf{c}$ with orthonormal $\\mathbf{U} \\in \\mathbb{R}^{n \\times d}$ and Gaussian vector $\\mathbf{c}$.  \n&nbsp;  \nLet $\\mathbf{F}_\\mathcal{T}$ be the Fourier matrix with rows chosen in the set $\\mathcal{T}$. \nWe assume a measurement model, where the signal $\\mathbf{x}$ takes on $N_s$ motion states defined by the unknown translations $t _1^\\ast,\\ldots,t _ {N _ s}^\\ast$, and for each translated version of the signal, a set of measurements is collected according to  \n$$\\mathbf{y _ s} = \\mathbf{D} _ {t _ s^\\ast, \\mathcal{T} _ s} \\mathbf{F} _ {\\mathcal{T} _ s} \\mathbf{x}, \\tag{1}$$\nwhere $\\mathbf{D} _ {t _ s^\\ast, \\mathcal{T} _ s} $ is a diagonal matrix with $e^{i 2 \\pi t_s^\\ast l / n}, l\\in \\mathcal{T} _ s$ on its diagonal. \nHere, we use that a circular shift by $t_s^\\ast$ in the spatial domain is a multiplication with a complex exponential in the frequency domain.   \nWe write the acquisition of the entire measurements $\\mathbf{y} \\in \\mathbb{R}^k$ from all motion states as $ \\mathbf{y} = \\mathbf{D} _ \\mathbf{t^\\ast} \\mathbf{F} _  \\mathcal{T} \\mathbf{x}.$\nFor simplicity we assume the fully sampled case, i.e., $k=n$.  \n&nbsp;  \nNext, we define a network $f( \\mathbf{x}^\\dagger) = \\mathbf{U} \\mathbf{U}^T \\mathbf{x}^\\dagger$, for which it is straightforward to see that if all motion parameters $\\mathbf{t}^\\ast$ are known we have that \n$$f( (\\mathbf{D} _ {\\mathbf{t}^\\ast} \\mathbf{F})^\\dagger \\mathbf{y} ) = \\mathbf{x}, \\tag{2}$$\ni.e., the network reconstructs the signal perfectly.\nThe loss function used in the paper for this setting is\n$$\\mathcal{L} _ {\\text{TTT}} (\\mathbf{t}) = || \\mathbf{D} _ {\\mathbf{t}} \\mathbf{F} _ {\\mathcal{T}} f( (\\mathbf{D} _ {\\mathbf{t}} \\mathbf{F} _ \\mathcal{T})^\\dagger \\mathbf{y} ) - \\mathbf{y} || _ 2^2, \\tag{3}$$\non which we can perform test-time-training with gradient descent with respect to the motion parameters $\\mathbf{t}$.  \n&nbsp;  \nAssuming that $\\mathbf{U}$ is a random subspace and that $\\mathbf{c}$ is drawn from a Gaussian with identity covariance matrix, it can be shown that the objective function concentrates around\n$$\\mathcal{\\tilde L} (\\mathbf{t}) =  || \\mathbf{D} _ \\mathbf{t} - \\mathbf{D} _ {\\mathbf{t}^\\ast} \\tag{4} ||^2_F.$$\nThis expected objective function has a unique minimizer at $\\mathbf{t} = \\mathbf{t}^\\ast$ and is convex in a small region around $\\mathbf{t}^\\ast$, but not globally.  \n&nbsp;  \nFor setting the number of motion states $N_s=1$ we obtain a simple one-dimensional optimization problem and can inspect the behavior of the loss function graphically. Without loss of generality we set $t^\\ast = 0$ and plot the loss $\\mathcal{L} _ {\\text{TTT}} (t)$ in PDF Fig. 4. As we can see the loss exhibits a global minimum at $t=t^\\ast$, but also local minima for $t \\neq t^\\ast$.  \n&nbsp;  \nIn order to minimize such a function in practice, we defining a learning rate schedule (see Appendix B.2 in the paper), where from the start we explore the loss landscape with a large learning rate if the initial loss is large as this indicates the presence of strong motion and initializing all motion parameters with zero might constitute to a large distance to the true motion parameters. Then, the learning rate is reduced gradually. Also once estimated motion parameters approach the true parameters, we did not observe significant deviations from this solution indicating the existence of a global minimum (or a good local one) in practice. \n- **W 2, experimental investigation under what conditions the method is most effective in terms of undersampling factor and sampling trajectory:** See author rebuttal.\n- **Q 1, why and under what conditions does the method work:** See Weakness 1 and 2.\n- **Q 2, is the linear trajectory the optimal undersampling trajectory:** See author rebuttal. In short, a linear trajectory is not the best choice for motion estimation with our method, an interleaved or random order is better.\n- **Q 3, do you need the fully sampled k-space data to learn $f_\\theta$:** We do not, as discussed in Section 6, our method can leverage recent advances in self-supervised training of MRI reconstruction networks. This is a strength of our method, other proposed deep learning based method for (2D) motion correction in MRI rely on simulated motion artifacts during training which require fully sampled k-space data.\n- **Limitations, the method’s effectiveness might vary significantly with different sampling patterns and motion artifacts:** Regarding the role of the sampling pattern see author rebuttal. Regarding the role of motion artifacts we evaluated our method on motion events occurring at random time points with different amplitudes and frequencies resulting in different motion artifacts achieving good performance for all of them. In general, we expect our method to work well for different types of motion as it was not trained on a particular type of simulated motion. Remaining limitations have been discussed above.\n\nWe hope the provided theory as well as the additional simulations and clarifications address the concerns of the reviewer and if yes, that the reviewer considers raising their score.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bHqfwwikO1", "reviewer": "Reviewer_4RZt", "summary": "This paper proposes a method for estimating the motion of patients, such that accurate motion-corrected images could be reconstructed. The key idea is that a neural network trained for motion-free reconstruction has a small loss if there is no motion, thus optimizing over motion parameters passed through the reconstruction network enables estimation of the motion.", "questions": "See the weaknesses.", "limitations": "Yes, limitations are discussed.", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The estimation of motion of patients is important in MRI measurement. This paper is a good trial to address this problem.", "weaknesses": "- There is no technical novelty in this work. It introduces some existing modules to address the proposed problem.\n- The experiments are weak to demonstrate the efficacy of the proposed method. \n  1）There lacks strong baselines and there are no compared methods. Although according to the authors, there are no other methods designed for 3D rigid motion estimation for 3D motion-corrected MRI, there should be some methods that could reconstruct the images.\n  2) The data are little for evaluating the methods, and more averaged quantitative results are suggested to report.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback. In the following we address the concerns in the order as pointed out by the reviewer.\n- **Weakness 1, introduction of existing modules and lack of novelty:** We would like to point out that combining a 2D reconstruction network trained on motion-free data with test-time-training for motion estimation in 3D has not been proposed before, neither a closely related method. Our work is the first that enables efficient motion estimation in 3D based on a 2D network trained on motion-free measurements. As discussed in our related work section, previous work on deep learning based motion estimation has exclusively focused on the 2D case and methods that are required to simulate motion during training and hence are specific to the type of motion used during training. Thus the work contains significant technical novelty. \n- **Weakness 2 part 1, lack of further baselines in particular motion corrupted image reconstruction networks:** While we are not aware of any deep learning based baselines for 3D motion estimation, there are indeed methods for 3D end-to-end reconstruction from motion corrupted volumes as discussed in our related work section, second paragraph. As we note there, this class of methods is well known to perform poorly relative to methods that perform motion estimation and then reconstruct, like our method. See Fig. 3 in [1] and Fig. 7 in [2]. \n- **Weakness 2 part 2, average quantitative results over more test examples:** For a given method each result in Fig. 3 in the paper is averaged over 5 test examples with each 2 independent motion trajectories. As we consider 10 different levels of motion, we evaluate in total over 100 different motion trajectories each consisting of up to 10 randomly generated motion events. Regarding the required number of test examples, for image reconstructions tasks this number is typically lower than in other ML domains like computer vision as one test image contains already many different structures and details that the method needs to recover in order to achieve a high image quality score. Especially, in 3D the amount of data per test volume is large and comparatively small test sets are often used (e.g. 7 volumes used in [3]). \n\nWe hope this addresses the reviewer's concerns and if yes, that the reviewer considers raising their score.\n\n\n[1] Haskell et al. “Network Accelerated Motion Estimation and Reduction (NAMER): Convolutional Neural Network Guided Retrospective Motion Correction Using a Separable Motion Model”. In: Magnetic Resonance in Medicine (2019).  \n[2] Hossbach et al. “Deep Learning-Based Motion Quantification from k-Space for Fast Model-Based Magnetic Resonance Imaging Motion Correction”. In: Medical Physics (2023).  \n[3] Johnson and Drangova. “Conditional Generative Adversarial Network for 3D Rigid-Body Motion Correction in MRI”. In: Magnetic Resonance in Medicine (2019).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zxidd9xzHL", "reviewer": "Reviewer_H96H", "summary": "This paper proposes a motion correction MRI reconstruction algorithm for 3D brain MRI. The proposed technique consists in a deep learning-based estimation of rigid motion parameters, which allows to correct the k-space before a final reconstruction.\nEstimation of motion parameters are based on a single optimization step freezing the reconstruction UNet parameters, and assuming loss will only (or mainly) be influenced by motion.", "questions": "Could the authors find another database of 3D data in order to train a 3D reconstruction UNet directly?\nGiven the proposed technique aims also at suppressing corrupted k-spaces lines, would it be sensible to train a reconstruction technique with higher a acceleration factor?\nWould the proposed technique be applicable for other sampling strategies, especially would it be possible to apply on stack of stars acquisitions or other 3D sampling strategy.\nWhy did the authors not apply other reconstruction (standard L1) reconstruction for the real dataset for comparison? Especially since it was outperforming the UNet approach on simulated motion data.", "limitations": "The authors should discuss the risks of hallucinations and bias of the proposed technique.", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "The paper addresses an important problem, namely motion correction, using a fully data-driven approach. Moreover, the originality of the approach lies in tackling this problem directly on the 3D frequency (or k-)space , when most deep-learning techniques are focusing on 2D acquisitions.", "weaknesses": "The overall structure of the paper is quite confusing, which makes it hard to follow. Experiments and alternative techniques seem to be presented throughout the result section.\nA proper ablation study seems required, in order to assess the added-value of each step. Proper baseline also seems necessary especially regarding the reconstruction step or the removal of motion artifacted lines, as more recent techniques could have been used as SOTA.\nThe overall motion estimation relies on a first 2D reconstruction pipeline however the performance of the technique seems to be lower than a standard L1 reconstruction technique. If so, how could the authors be sure that motion parameters are not biased or affected by the low performance of the 2D reconstruction pipeline.\nWhy did the authors not try to directly reconstruct from the 3D k-space domain, in order to estimate motion parameters in 3D? Moreover, I believe the authors should also provide further details on they are switching back and forth from 3D k-space domains to 2D, as a simple slicing in a given direction of the 3D k-space is not appropriate.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the feedback. In the following we address the concerns and questions in the order as pointed out by the reviewer.\n- **Weakness 1, experiments and alternative techniques seem to be presented throughout the result section:** Thanks for the feedback. We will make sure to describe all methods that we compare in the main Figure 3 at the end of the setup Section 5.1 before the results section.\n- **Weakness 2, a proper ablation study seems required, in order to assess the added-value of each step:** In fact, Figure 3 already shows the performances during all steps of MotionTTT. L1 indicates the performance without any motion correction (before applying MotionTTT), MotionTTT-L1 the performance after motion estimation and MotionTTT-L1+Th after motion estimation plus data consistency loss thresholding.\n- **Weakness 3, baselines regarding the reconstruction step or the removal of motion artifacted lines:** Regarding the reconstruction step, our method is agnostic to the method used for reconstruction based on the estimated motion parameters. Hence we do not expect any new insights from trying additional reconstruction methods as a better reconstruction method would lead to a better performance for any motion estimation method.  \n&nbsp;  \nRegarding baselines for the removal of motion artifacted lines, we are not sure to which method the reviewer would like to see a comparison. Our method estimates motion parameters with a high accuracy and hence will outperform any method that only detects and then removes motion corrupted lines. Only in the presence of severe motion we remove lines that exhibit a large data consistency loss indicating inaccurate estimation of motion parameters, a feature that comes for free with the proposed method. However, also in the most severe cases typically not more than 5% have to be discarded. \n- **Weakness 4, effect of the low reconstruction performance of the U-net on motion estimation:** It is not clear if a better reconstruction performance of the U-net due to an increased amount of training data would further improve the ability to estimate motion parameters especially as our experimental results show, the current reconstruction performance of the U-net already achieves highly accurate motion parameter estimation across a wide range of simulated motion and significantly improved image quality in case of prospectively acquired real motion-corrupted data. Nevertheless, we will investigate if even further improvements can be achieved by training on additional data sources like e.g. 2D brain data.\n- **Weakness 5, why did the authors not try to directly reconstruct from the 3D k-space domain, in order to estimate motion parameters in 3D:** Training 3D models on entire 3D volumes is infeasible in terms of memory and compute requirements. However, we agree that instead of chunking the volume slice-by-slice as we do, one could also train a 3D model on 3D chunks and perform chunk-by-chunk reconstruction.  \nWe decided to build on the well-established 2D convolutional U-net as it enabled highly accurate motion estimation combined with fast reconstruction and a low network parameter count, which is important as the entire volume has to be reconstructed in every iteration. In the future, this also enables enlarging the training set with 2D data in order to overcome the limited availability of 3D dataset.\n- **Weakness 6, regarding switching between 2D and 3D:** As the reviewer pointed out correctly, slicing the k-space in arbitrary directions is not appropriate. Hence, we only slice the zero-filled reconstruction in the image domain at the network input and add slices together at the network output before applying the 3D fourier transform to obtain the reconstructed k-space data. We will emphasize this more clearly in the Section 4, step 2: Test-time-training for motion estimation.\n- **Question 1, existence of another database of 3D data to train a 3D reconstruction UNet directly:** To the best of our knowledge the used dataset is the largest 3D brain dataset containing the original k-space measurements publicly available. While other smaller ones exist, we do not believe that even if compute would not be a problem the amount of training examples would suffice to train a 3D model on entire volumes. \n- **Question 2, training a reconstruction technique with higher acceleration factor to compensate for thresholding:** We only threshold a relatively small number of lines, typically not more than 5% of the acquired lines even under the most severe motion. Nevertheless, there is a discrepancy between the undersampling mask the U-net was trained on in the absence of motion and the undersampling of the motion-corrected network inputs due to rotations in the k-space as discussed in the paper in Section 5.2. However, both changes in the mask due to thresholding and rotations are motion specific and hence difficult to train on without simulating motion.  \nHence, in the paper we use L1-minimization for reconstruction, which is mostly independent of changes in the mask.\n- **Question 3, application to other 3D sampling strategies (stack of stars):** See author rebuttal point 1.\n- **Question 4, missing L1 reconstruction for the real-motion data:** We do show the results for MotionTTT+Th-L1 in Figure 6.\n- **Limitation, risks of hallucinations and bias:** As currently the final reconstruction results of our method are based on L1-minimization no learning based hallucinations can occur. If the motion parameter estimation is off, the result will be a reconstruction with motion artifacts similar to when no motion correction is applied. We are not sure what type of bias the reviewer is referring to. \n\n\nWe hope those responses address the reviewer's concerns and the reviewer considered raising their score.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mQ2AhFJH6C", "reviewer": "Reviewer_pjnK", "summary": "This work proposes a novel deep learning-based framework for recovering high-quality 3D MR images from undersampled and motion-corrupted k-data. The proposed approach is well motivated and technically sound. The authors perform extensive experiments on simulated and real MR datasets, which confirm the effectiveness of the proposed framework.", "questions": "See the section of Strengths and Weaknesses, please.", "limitations": "The authors mention enough limitations of their work.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Motivation and significance**\n\nMotion correction is an important issue in the field of MR imaging. The proposed method removes the reliance on motion simulation by training a neural network that reconstructs high-quality MR image from under-sampled k-data in advance. This new paradigm significantly improves the robustness of the reconstructed images.\n\n**Technical solid**\n\nIn the steps 2 of test-time training for motion estimation, the MRI acquisition knowledge, such as forward model, sampling trajectory, are effectively integrated into the framework, which improve the reliability of the reconstruction.\n\n**Clarity and organization**\n\nThis submission is well-written and easy to follow.\n\n**Experimental evaluation**\n\nThe authors perform experimental evaluations on simulation and real-world datasets. I am pleased to see the experiments based on the real-world dataset. I think it greatly improves this paper.", "weaknesses": "For this submission, I have a few minor suggestions as follows.\n\n**Various types of rigid motion**\n\nIn line 205, the authors show that random rigid motions ($M_\\text{max}=[2,5,10]$) are simulated. However, the rigid motion in the real world could follow some patterns, such as involuntary motion and abrupt motion. I think it is better to test these different movements.\n\n**Compared methods**\n\nFor supervised methods, only U-net is used as a baseline. Advanced supervised models [1][2] are not discussed. Furthermore, motion correction methods [3] based on diffusion models are not compared. \n\n> [1] Han Y, Yoo J, Kim H H, et al. Deep learning with domain adaptation for accelerated projection‐reconstruction MR[J]. Magnetic resonance in medicine, 2018, 80(3): 1189-1205.\n\n> [2] Liu J, Kocak M, Supanich M, et al. Motion artifacts reduction in brain MRI by means of a deep residual network with densely connected multi-resolution blocks (DRN-DCMB)[J]. Magnetic resonance imaging, 2020, 71: 69-79.\n\n> [3] B. Levac, A. Jalal, and J. I. Tamir. “Accelerated Motion Correction for MRI Using Score-Based 388 Generative Models”. ISBI 2023.\n\n**Some typos**\n\nFor example, the symbol $\\mathcal{E}$ in Eq. 3 is not defined.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback and the positive evaluation of our work. In the following we address the weaknesses in the order as pointed out by the reviewer.\n- **Weakness 1, various types of rigid motion:** We agree that in practice motion can be categorized into different types. However, the range of different motions is large varying with patient population and condition. Also we are not aware of good available models of the different types of motion based on real measurements in the literature. Hence, it is common to evaluate with randomly simulated motion (see line 202 for examples). \nNote that for the experiments in Figure 3 we simulated a total of 100 different motion trajectories with up to 10 randomly generated motion events per trajectory covering a wide range of possible motions. \nIn general, we expect our method to work well for different types of motion as different to previous deep learning based approaches it was not trained on a particular type of simulated motion.  \n- **Weakness 2, compared methods:** [1] proposes a domain adaptation method for reconstructing radial MRI in a limited data regime, but does not discuss motion correction. [2] proposes a novel network architecture for end-to-end motion artifact reduction but only in 2D and in general it has been found that end-to-end methods, while being significantly faster, result in inferior reconstruction quality compared to reconstruction based on previously estimated motion parameters like ours. See e.g. Fig. 3 in [4] and Fig. 7 in [5].  \nWhile diffusion model based motion correction is an interesting direction, the reason why we do not compare to [3] is that the proposed method can only operate 2D motion, while we investigate motion estimation in 3D. Directly extending the method to 3D requires training a diffusion model on entire 3D volumes, which is computational difficult and would require much larger 3D datasets. At the same time it is unclear how to use a 2D diffusion model to estimate motion in 3D.\n- **Weakness 3, some typos:** Thanks, we fixed it. \n\n[4] Haskell et al. “Network Accelerated Motion Estimation and Reduction (NAMER): Convolutional Neural Network Guided Retrospective Motion Correction Using a Separable Motion Model”. In: Magnetic Resonance in Medicine (2019).  \n[5] Hossbach et al. “Deep Learning-Based Motion Quantification from k-Space for Fast Model-Based Magnetic Resonance Imaging Motion Correction”. In: Medical Physics (2023).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Thanks for the reviews!  
We would like to start by emphasizing that our work is the first that enables efficient motion estimation in 3D based on a 2D neural network trained on motion-free measurements. We show that our method can reliably predict motion and that this yields significant improvements in image quality. We provide extensive numerical results, including for simulated motion, as well based on real motion in an MRI scan that we specifically collected for this project. Most existing works for motion estimation and correction are for 2D and do not contain results on real motion in an MRI scanner. 

In the following, we address two concerns raised by reviewers fV1o and H96H regarding the role of the sampling trajectory and undersampling factor for the success of our proposed motion estimation method. To address those concerns, we provide additional experimental results in the attached single-page PDF.

**1. The sampling order is important as it is difficult to estimate motion parameters for a batch of k-space measurements that contain only high-frequency components.**

Reviewer fV1o and H96H asked if the effectiveness of MotionTTT is affected by the sampling trajectory (e.g. Cartesian vs. spiral or stack of stars). 
Up to trajectory specific additional sources of artifacts like B0 field inhomogeneities that needed to be modeled, the problem of motion estimation does not change with the sampling trajectory, and as our forward model is implemented via the non-linear FFT, our method can process any k-space geometry. 

However, the sampling order is important, as we show with an additional experiment. We investigate three different sampling orders for a Cartesian trajectory:  We fix the undersampling mask and change the order at which k-space lines are acquired between interleaved, linear (both have been used already in the paper) and random (see PDF, Fig. 3 c,d,e for a visualization of sampling orders). 

For motion severity level 5 we obtain the average (8 instances) reconstruction performance in PSNR for reconstruction based on motion parameters estimated by our MotionTTT vs. ground truth motion as 35.98vs.36.00, 36.27vs.36.28 and 33.16vs.36.99 for interleaved, random and linear sampling orders.
Interleaved and random orders achieve perfect motion estimation, while the linear order leads to a significant gap of almost 3dB. 

For a given test example PDF Fig. 1 a,b shows the data consistency (DC) loss of MotionTTT at the first and final iteration. 
For the random and interleaved order motion states are estimated accurately resulting in a final DC loss well below our defined DC loss threshold. For the linear order first and last motion states, pertaining to shots that contain only high-frequency, i.e., low-energy components, maintain a high final DC loss and the corresponding estimated motion parameters are off as exemplified with the estimated translation parameter in $k_y$ direction in PDF Fig. 1 c. 
We attribute this finding to the U-net used during MotionTTT reconstructing high-frequency components not as faithfully as low-frequency components. 

Finally, we would like to note that the sampling order of a Cartesian trajectory can be customized without affecting any other sequence parameters like the shape of the undersampling mask, and hence choosing a, e.g., random order does not limit the generality of our method in the context of Cartesian sampling, which is still most commonly used in practical 3D MRI. 

**2. The acceleration factor influences the performance of MotionTTT as a smaller factor leads to overall higher reconstruction performance but also to a more difficult optimization problem as more shots are acquired translating to more unknown motion states to be estimated.**

Reviewer fV1o requested a more robust experimental validation of the conditions in terms of sampling trajectory (see above) and undersampling factor under which MotionTTT works. 

To this end, we conducted the following experiment, where we re-train the U-net on two additional Cartesian undersampling masks with acceleration factors R=2,8 in addition to the existing results with R=4 (see PDF Fig. 3 a,b,c for the masks). PDF Fig. 2 shows the reconstruction performance in PSNR based on motion parameters estimated by our MotionTTT compared to ground truth motion over three levels of motion severity and the three acceleration factors.

As expected, the overall performance decays with increasing acceleration factors and motion severities. 
For mild and moderate motion, MotionTTT achieves highly accurate motion estimation for all acceleration factors indicated by the vanishing performance gap to using ground truth motion. For strong motion, the best performance is still achieved for the lowest acceleration factor, but an increasing performance gap exists for decreasing acceleration factors due to incorrectly estimated motion states that are then discarded from the final reconstruction via DC loss thresholding. In fact, under severe motion an average of 20.7/100, 2.6/50 and 0.6/25 shots have to be discarded for acceleration factors 2,4 and 8.
We attribute the increasing numbers of incorrectly estimated motion states to the increasing complexity of the optimization problem as the number of unknown motion states to be estimated increases linearly in the number of acquired shots. 

We will include those results in the revised version of our paper. We hope that you find the additional experiments helpful and are happy to discuss further.

Meta-review: The manuscript is on the verge of rejection, yet the authors have presented a compelling rebuttal. This AC acknowledges the complexity of the problem and appreciate this first work towards 3D motion-compensated MRI reconstruction. This is why the AC believes this could be an interesting paper to be discussed at the conference. While this AC is inclined to support weak acceptance, the AC also understands it may not meet the priority threshold given the overall acceptance criteria/ranking. If the paper is accepted, it is with the expectation that the key points from the rebuttal will be effectively integrated into the manuscript to enhance its overall impact.
Decision: Accept (poster)