Original: Near-Exact Recovery for Sparse-View CT via Data-Driven Methods
Revised: Near-Exact Recovery for Sparse-View CT via Data-Driven Methods

Original: Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: This work presents an empirical study on the design and training of iterativeneural networks for image reconstruction from tomographic measurements withunknown geometry.
Revised: This work presents an empirical study on the design and training of iterative neural networks for image reconstruction from tomographic measurements with unknown geometry.

Original: It is based on insights gained during our participation in therecent AAPM DL-Sparse-View CT challenge and a further analysis of our winningsubmission (team name: robust-and-stable ) subsequent to the competitionperiod.
Revised: It is based on insights gained during our participation in the recent AAPM DL-Sparse-View CT challenge and a further analysis of our winning submission (team name: robust-and-stable ) subsequent to the competition period.

Original: The goal of the challenge was to identify the state of the art in sparseview CT with data-driven techniques, thereby addressing a fundamental researchquestion: Can neural-network-based solvers produce near-perfect reconstructionsfor noise-free data?
Revised: The goal of the challenge was to identify the state of the art in sparseview CT with data-driven techniques, thereby addressing a fundamental research question: Can neural-network-based solvers produce near-perfect reconstructions for noise-free data?

Original: We answer this in the afﬁrmative by demonstrating that aniterative end-to-end scheme enables the computation of near-perfect solutions onthe test set.
Revised: We answer this in the affirmative by demonstrating that an iterative end-to-end scheme enables the computation of near-perfect solutions on the test set.

Original: Remarkably, the fanbeam geometry of the used forward model iscompletely inferred through a data-driven geometric calibration step.
Revised: Remarkably, the fanbeam geometry of the used forward model is completely inferred through a data-driven geometric calibration step.

Original:  Introduction
Revised:  Introduction

Original: In recent years, deep learning methods have been successfully applied to problems of the naturalsciences [11, 17, 6].
Revised: In recent years, deep learning methods have been successfully applied to problems of the natural sciences [11, 17, 6].

Original: A prominent example of such scientiﬁc machine learning is the developmentof efﬁcient solutions strategies for inverse problems [2, 13], such as those encountered in medicalimaging.
Revised: A prominent example of such scientific machine learning is the development of efficient solutions strategies for inverse problems [2, 13], such as those encountered in medical imaging.

Original: Despite unprecedented empirical performance in various practical scenarios, a soundtheoretical understanding of data-driven reconstruction methods seems to be out of reach to date.
Revised: Despite unprecedented empirical performance in various practical scenarios, a sound theoretical understanding of data-driven reconstruction methods seems to be out of reach to date.

Original: For this reason, more and more critical voices are heard, pointing out a lack of evidence for the reliability of deep-learning-based solution strategies.
Revised: For this reason more and more critical voices are heard pointing out a lack of evidence for the reliability of deep-learning-based solution strategies.

Original: For instance, Sidky et al.
Revised: For instance, Sidky et al.

Original: [18] have recently demonstrated that post-processing of ﬁltered backprojection images with the prominent UNet-architecturemay not yield satisfactory recovery precision in sparse-view computed tomography (CT).
Revised: [18] have recently demonstrated that post-processing of filtered backprojection images with the prominent UNet-architecture may not yield satisfactory recovery precision in sparse-view computed tomography (CT).

Original: This gaverise to the recent AAPM Grand Challenge
Revised: This gave rise to the recent AAPM Grand Challenge

Original: “Deep Learning for Inverse Problems: Sparse-View
Revised: “Deep Learning for Inverse Problems: Sparse-View

Original: Computed Tomography Image Reconstruction”.
Revised: Computed Tomography Image Reconstruction”.

Original: The goal of the challenge was “to identify thestate-of-the-art in solving the CT inverse problem with data-driven techniques” [19] and thereby toevaluate whether deep-learning-based schemes can achieve (near-)exact precision, similarly to themodel-based benchmark of total variation (TV) minimization.
Revised: The goal of the challenge was “to identify the state-of-the-art in solving the CT inverse problem with data-driven techniques” [19] and thereby to evaluate whether deep-learning-based schemes can achieve (near-)exact precision, similarly to the model-based benchmark of total variation (TV) minimization.

Original: This article presents an analysis of our winning submission to the AAPM challenge (team name:robust-and-stable ).
Revised: This article presents an analysis of our winning submission to the AAPM challenge (team name: robust-and-stable ).

Original: Besides a description of the underlying methodology, our main objective isto distill several key insights regarding the design and training of our iterative network that are ofbroader interest.
Revised: Besides a description of the underlying methodology, our main objective is to distill several key insights regarding the design and training of our iterative network that are of broader interest.

Original: In a nutshell, our approach is rooted in the following (debatable) observation:
Revised: In a nutshell, our approach is rooted in the following (debatable) observation:

Revised: One example of a triple of 128-view sinogram, filtered backprojection, and ground truth phantom image taken from the AAPM challenge training dataset.

Original: High reconstruction accuracy is only possible if the forward model is explicitly incorporated into the solution map, e g., by an iterative promotion of data consistency.
Revised: High reconstruction accuracy is only possible if the forward model is explicitly incorporated into the solution map, e.g., by an iterative promotion of data-consistency.

Original: One example of a triple of 128-view sinogram, ﬁltered backprojection, and ground truth phantom image taken from the AAPM challenge training dataset.

Original: We propose a conceptually simple, yet powerful deep learning pipeline which turns a post-processing
Revised: We propose a conceptually simple, yet powerful deep learning pipeline which turns a post-processing

Original: UNet [16] into an iterative reconstruction scheme.
Revised: UNet [16] into an iterative reconstruction scheme.

Original: While many of its technical ingredients havebeen previously reported in the literature, the overall strategy is novel.
Revised: While many of its technical ingredients have been previously reported in the literature, the overall strategy is novel.

Original: Our design differs frommore common unrolled networks in several aspects, most notably: (a) we make use of a pre-trained
Revised: Our design differs from more common unrolled networks in several aspects, most notably: (a) we make use of a pre-trained

Original: UNet as the computational backbone, and (b) data-consistency is inspired by an (cid:96) 2 -gradient step, butemploys the ﬁltered backprojection (FBP) instead of the regular adjoint.
Revised: UNet as the computational backbone, and (b) data-consistency is inspired by an ℓ 2 -gradient step, but employs the filtered backprojection (FBP) instead of the regular adjoint.

Original: A speciﬁc difﬁculty of the AAPM challenge setup was an unknown (fanbeam) forward model .
Revised: A specific difficulty of the AAPM challenge setup was an unknown (fanbeam) forward model .

Original: Therefore, a crucial ﬁrst step of our strategy consists in the data-driven estimation of the underlyingfanbeam geometry.
Revised: Therefore, a crucial first step of our strategy consists in the data-driven estimation of the underlying fanbeam geometry.

Original: We accomplish this by ﬁtting a generic, parameterized fanbeam operator to theprovided sinogram-image pairs.
Revised: We accomplish this by fitting a generic, parameterized fanbeam operator to the provided sinogram-image pairs.

Original: Regarding the actual inversion method, our main ﬁndings can be summarized as follows:
Revised: Regarding the actual inversion method, our main findings can be summarized as follows:

Original: (i) End-to-end neural networks can achieve near-perfect accuracy on the prescribed CT reconstruction task.(ii)
Revised: (i) End-to-end neural networks can achieve near-perfect accuracy on the prescribed CT reconstruction task.

Original:  Our iterative scheme invokes the (learned) forward operator only 5 times (FBP 6 times), instark contrast to classical model-based solvers with hundreds or thousands of iterations.
Revised: (ii) Our iterative scheme invokes the (learned) forward operator only 5 times (FBP 6 times), in stark contrast to classical model-based solvers with hundreds or thousands of iterations.

Original: (iii) A careful training procedure is more important than speciﬁc details of the network architecture.
Revised: (iii) A careful training procedure is more important than specific details of the network architecture.

Original: The proposed approach could be useful for other inverse problems as well, given that it outperformsstate-of-the-art approaches, such as the learned primal-dual algorithm [1], by an order of magnitude.
Revised: The proposed approach could be useful for other inverse problems as well, given that it outperforms state-of-the-art approaches, such as the learned primal-dual algorithm [1], by an order of magnitude.

Original: AAPM Challenge and Dataset.
Revised: AAPM Challenge and Dataset.

Original: The challenge data consists of synthetic 2D grayscale imagescomparable to real-world mid-plane breast CT scans, see [19] for details.
Revised: The challenge data consists of synthetic 2D grayscale images comparable to real-world mid-plane breast CT scans, see [19] for details.

Original: A fanbeam geometry with128 projections over 360 degrees was used to create sinograms and FBPs, see Fig.
Revised: A fanbeam geometry with 128 projections over 360 degrees was used to create sinograms and FBPs, see Fig.

Original: 1 for an example.
Revised: 1 for an example.

Original: The exact fanbeam geometry was unknown to the challenge participants, see Step 1 in Section 2.
Revised: The exact fanbeam geometry was unknown to the challenge participants, see Step 1 in Section 2.

Original: A training set of 4000 triples of 512 × 512 phantom images, 128-view sinograms, and FBPs, wasprovided.
Revised: A training set of 4000 triples of 512 × 512 phantom images, 128-view sinograms, and FBPs, was provided.

Original: A test set of 100 tuples of sinograms and FBPs without (publicly available) ground truthphantoms was used for the ﬁnal challenge evaluation.
Revised: A test set of 100 tuples of sinograms and FBPs without (publicly available) ground truth phantoms was used for the final challenge evaluation.

Original: Initially, about 50 international teams haveparticipated out of which 25 submitted their method for the ﬁnal evaluation.
Revised: Initially, about 50 international teams have participated out of which 25 submitted their method for the final evaluation.

Original:  Methodology
Revised:  Methodology

Original: In this section, we give an overview of our approach and motivate its design choices.
Revised: In this section, we give an overview of our approach and motivate its design choices.

Original: A public coderepository can be found under https://github.com/jmaces/aapm-ct-challenge .
Revised: A public code repository can be found under https://github.com/jmaces/aapm-ct-challenge .

Original: Step 1: Data-Driven Geometry Identiﬁcation.
Revised: Step 1: Data-Driven Geometry Identification.

Original: The ﬁrst step of our reconstruction pipeline learnsthe unknown forward operator from the provided training data.
Revised: The first step of our reconstruction pipeline learns the unknown forward operator from the provided training data.

Original: The continuous version of tomographicfanbeam measurements is based on computing line integrals
Revised: The continuous version of tomographic fanbeam measurements is based on computing line integrals

Original: [Equation]
Revised: [Equation]

Original: where x 0 is the unknown image and L ( s, ϕ ) denotes a line in fanbeam coordinates, i.e., ϕ is thefan rotation angle and s encodes the sensor position ; see [4] for more details.
Revised: where x 0 is the unknown image and L ( s, φ ) denotes a line in fanbeam coordinates, i.e., φ is the fan rotation angle and s encodes the sensor position ; see [4] for more details.

Original: In an idealized 1situation, the fanbeam model is speciﬁed by few geometric parameters (see Fig. 6).
Revised: In an idealizedsituation, the fanbeam model is specified by few geometric parameters (see Fig. 6).

Original: In the AAPMchallenge, the resulting forward operator is severely ill-posed , since only the measurements of afew fan rotation angles are acquired.
Revised: In the AAPM challenge, the resulting forward operator is severely ill-posed , since only the measurements of a We to AAPM few fan rotation angles are acquired.

Original: Furthermore, the exact geometric setup is not disclosed to thechallenge participants — it is only known that fanbeam measurements are taken.
Revised: Furthermore, the exact geometric setup is not disclosed to the challenge participants — it is only known that fanbeam measurements are taken.

Original: We have addressedthis lack of information by a data-driven estimation strategy that ﬁts the geometry parameters tothe given training data pairs of discrete images x 0 ∈ R  N and fanbeam measurements y 0 ∈ R 128 · 1024= m .
Revised: We have addressed this lack of information by a data-driven estimation strategy that fits the geometry parameters to the given training data pairs of discrete images x 0 ∈ R 512 · 512= N and fanbeam measurements y 0 ∈ R 128 · 1024= m .

Original: In fact, the parameters in Fig.
Revised: In fact, the parameters in Fig.

Original: 6 are redundant and it sufﬁces to estimate a smallersubset of parameters, see Appendix B for details.
Revised: 6 are redundant and it suffices to estimate a smaller subset of parameters, see Appendix B for details.

Original: To that end, we have implemented a discrete fanbeam transform (and corresponding ﬁltered backprojection) from scratch in PyTorch .
Revised: To that end, we have implemented a discrete fanbeam transform (and corresponding filtered backprojection) from scratch in PyTorch .

Original: A distinctive aspect of our implementation is the use of avectorized numerical integration that enables the efﬁcient computation of derivatives with respectto the geometric parameters by means of automatic differentiation .
Revised: A distinctive aspect of our implementation is the use of a vectorized numerical integration that enables the efficient computation of derivatives with respect to the geometric parameters by means of automatic differentiation .

Original: This enables a data-driven parameter identiﬁcation.
Revised: This enables a data-driven parameter identification.

Original: We estimate the free parameters θ fan of the implemented forward operator
Revised: We estimate the free parameters θ fan of the implemented forward operator

Original: F [ θ fan ] ∈ R m × N from the collection of M = 4000 sinogram-image pairs { ( y i 0 , x i 0 ) }Mi [Equation]
Revised: F [ θ fan ] ∈ R m × N from the collection of M = 4000 sinogram-image pairs { ( y i 0 , x i 0 ) }

Original: 1 by solving
Revised: Mi =1 by solving

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: with a variant of gradient descent.
Revised: with a variant of gradient descent.

Original: Subsequently, we determine additional free parameters θ fbp of theimplemented ﬁltered backprojection FBP
Revised: Subsequently, we determine additional free parameters θ fbp of the implemented filtered backprojection FBP

Original: [ θ fan , θ fbp ] ∈ R N × m by solving
Revised: [ θ fan , θ fbp ] ∈ R N × m by solving

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original:  while keeping the already identiﬁed parameters θ fan ﬁxed.
Revised: ) while keeping the already identified parameters θ fan fixed.

Original: Finally, we still recognized a systematicerror in our forward model, presumably caused by subtle differences in the numerical integration incomparison to the true forward model of the AAPM challenge.
Revised: Finally, we still recognized a systematic error in our forward model, presumably caused by subtle differences in the numerical integration in comparison to the true forward model of the AAPM challenge.

Original: In compensation, we compute the(pixelwise) mean error over the training set, as an additive correction of the model bias.
Revised: In compensation, we compute the (pixelwise) mean error over the training set, as an additive correction of the model bias.

Original: From nowon, we will use the short-hand notation F and FBP for the estimated operators, respectively.
Revised: From now on, we will use the short-hand notation F and FBP for the estimated operators, respectively.

Original: Step 2: Pre-Training a UNet as Computational Backbone.
Revised: Step 2: Pre-Training a UNet as Computational Backbone.

Original: The centerpiece of our reconstructionscheme is a UNet-architecture U
Revised: The centerpiece of our reconstruction scheme is a UNet-architecture U

Original: [ θ ]: R N → R N [16].
Revised: [ θ ]: R N → R N [16].

Original: It is ﬁrst employed as a residual network topost-process sparse-view ﬁltered backprojection images, resulting in the reconstruction mapping
Revised: It is first employed as a residual network to post-process sparse-view filtered backprojection images, resulting in the reconstruction mapping

Original: [Equation]
Revised: [Equation]

Original: The learnable parameters θ are trained by (approximately) solving
Revised: The learnable parameters θ are trained by (approximately) solving

Original: [Equation]
Revised: [Equation]

Original: where we choose µ = 10 − 3 .
Revised: where we choose µ = 10 − 3 .

Original: This is tackled by 400 epochs of mini-batch stochastic gradient descentand the Adam optimizer [9] with initial learning rate 0 . 0002 and batch size 4 .
Revised: This is tackled by 400 epochs of mini-batch stochastic gradient descent and the Adam optimizer [9] with initial learning rate 0 . 0002 and batch size 4 .

Original: Step 3:
Revised: Step 3:

Original: Constructing an Iterative Scheme.
Revised: Constructing an Iterative Scheme.

Original: Our main reconstruction method, called ItNet, incorporates the (approximate) forward model F from Step 1 via the following iterative procedure:
Revised: Our main reconstruction method, called ItNet, incorporates the (approximate) forward model F from Step 1 via the following iterative procedure:

Original: [Equation]
Revised: [Equation]

Original: with learnable parameters θ = { ˜ θ k , λ k } Kk =1 , and the k -th data-consistency layer
Revised: with learnable parameters θ = { ˜ θ k , λ k } Kk =1 , and the k -th data-consistency layer

Original: [Equation]
Revised: [Equation]

Original:  y 
Revised: k , y : R N → R N [Equation] x (cid:55)→ x [Equation]

Revised: FBP

Original: [Equation]
Revised: [Equation]

Revised: F x − y ) .

Original: ItNet is trained analogously to (4) with µ = 10 − 4 .
Revised: ItNet is trained analogously to (4) with µ = 10 − 4 .

Original: The UNet-parameters ˜ θ k are initialized by theweights obtained in Step 2.
Revised: The UNet-parameters ˜ θ k are initialized by the weights obtained in Step 2.

Original: In summary, the central aspects of the architecture in (5) and importantdesign choices are:
Revised: In summary, the central aspects of the architecture in (5) and important design choices are:

Original: (i)
Revised: (i)

Original: The computational centerpiece is the UNet -architecture.
Revised: The computational centerpiece is the UNet -architecture.

Original: This contrasts earlier generations of unrolled iterative schemes, which rely on basic convolutional blocks instead, e.g., see [1, 23].
Revised: This contrasts earlier generations of unrolled iterative schemes, which rely on basic convolutional blocks instead, e.g., see [1, 23].

Original: Other recent state-of-the-art architectures also rely on more advanced sub-networks, e g.,see [10, 12, 7, 15, 20].
Revised: Other recent state-of-the-art architectures also rely on more advanced sub-networks, e.g., see [10, 12, 7, 15, 20].

Original: (ii) The data-consistency layer is inspired by a gradient step x (cid:55)→ x − λ k · F T ( F x − y ) on the loss x
Revised: (ii) The data-consistency layer is inspired by a gradient step x (cid:55)→ x − λ k · F T ( F x − y ) on the loss x

Original: (cid:55)→ λ k 2 (cid:107) F x − y (cid:107) 22 .
Revised: (cid:55)→ λ k 2 ∥ F x − y ∥ 22 .

Original: However, we replace the unﬁltered backprojection F T by its ﬁltered counterpart FBP for two reasons: (a) counteracting the fact that the unﬁltered backprojection issmoothing, and (b) producing images with pixel values at the right intensity scale. 
Revised: However, we replace the unfiltered backprojection F T by its filtered counterpart FBP for two reasons: (a) counteracting the fact that the unfiltered backprojection is smoothing, and (b) producing images with pixel values at the right intensity scale. [Table]

Original: (iii)
Revised: (iii)

Original: It is crucial to initialize the UNet-parameters ˜ θ k by the weights from Step 2.
Revised: It is crucial to initialize the UNet-parameters ˜ θ k by the weights from Step 2.

Original: This increases both the speed of convergence and the ﬁnal accuracy (see Fig.
Revised: This increases both the speed of convergence and the final accuracy (see Fig.

Original: In other words, a careful initialization of the UNet-block leads to a better local minima.
Revised: In other words, a careful initialization of the UNet-block leads to a better local minima.

Original: To the best of our knowledge, such an effect has not been reported in the literature yet.
Revised: To the best of our knowledge, such an effect has not been reported in the literature yet.

Original: In our experiments, we witnessed only minor effects by computing more than K = 5 iterations in (5).
Revised: In our experiments, we witnessed only minor effects by computing more than K = 5 iterations in (5).

Original: We discuss the choice of K in more detail in Appendix A, in particular, see Fig. 4.[Table]
Revised: We discuss the choice of K in more detail in Appendix A, in particular, see Fig. 4.

Original:  Results and Analysis
Revised:  Results and Analysis

Original: In terms of quantitative similarity measures, we restrict ourselves to reporting the average root-meansquared-error (RMSE), which was the main evaluation metric for the AAPM challenge.
Revised: In terms of quantitative similarity measures, we restrict ourselves to reporting the average root-meansquared-error (RMSE), which was the main evaluation metric for the AAPM challenge.

Original: Winning the Challenge.
Revised: Winning the Challenge.

Original: With an ensembling of ten ItNet 5 (more precisely a variant thereofreferred to as ItNet-post, see Appendix C)
Revised: With an ensembling of ten ItNet 5 (more precisely a variant thereof referred to as ItNet-post, see Appendix C)

Original: we were able to achieve near-exact recovery and win the
Revised: we were able to achieve near-exact recovery and win the

Original: AAPM challenge with a margin of about an order of magnitude compared to the runners-up teams.
Revised: AAPM challenge with a margin of about an order of magnitude compared to the runners-up teams.

Original: Notably, four out of the ﬁve best performing teams estimated the forward fanbeam operator and madeuse of the sinogram data.
Revised: Notably, four out of the five best performing teams estimated the forward fanbeam operator and made use of the sinogram data.

Original: Two out of them obtain a TV minimization solution that is further processedby a learned network.
Revised: Two out of them obtain a TV minimization solution that is further processed by a learned network.

Original: This requires a much higher number of iterations compared to our ItNet-post.
Revised: This requires a much higher number of iterations compared to our ItNet-post.

Original: Comparison Evaluations.
Revised: Comparison Evaluations.

Original: We compare variants of ItNet with different baselines and other state-ofthe-art methods.
Revised: We compare variants of ItNet with different baselines and other state-ofthe-art methods.

Original: 2 More precisely, we consider a post-processing of the FBP by the (in comparison tothe UNet-architecture) more advanced Tiramisu-architecture [3, 5, 8] as well as the iterative learnedprimal-dual (LPD) scheme [1] (modiﬁed by replacing the unﬁltered backprojection with the FBP ).
Revised: 2 More precisely, we consider a post-processing of the FBP by the (in comparison to the UNet-architecture) more advanced Tiramisu-architecture [3, 5, 8] as well as the iterative learned primal-dual (LPD) scheme [1] (modified by replacing the unfiltered backprojection with the FBP ).

Original: The average results are reported in Table 1.
Revised: The average results are reported in Table 1.

Original: To give a visual impression as well, reconstructions of animage from the validation set can be found in Fig.
Revised: To give a visual impression as well, reconstructions of an image from the validation set can be found in Fig.

Original: 2 and 7.
Revised: 2 and 7.

Original: Additional experiments regarding the aspects of data-consistency and number K of iterations can befound in Appendix A.
Revised: Additional experiments regarding the aspects of data-consistency and number K of iterations can be found in Appendix A.

Original:  Discussion
Revised:  Discussion

Original: We have demonstrated that, using only few evaluations of the forward model, data-driven approachescan achieve (near-)exact recovery on sparse-view CT tasks.
Revised: We have demonstrated that, using only few evaluations of the forward model, data-driven approaches can achieve (near-)exact recovery on sparse-view CT tasks.

Original: While our approach provides ﬁrstevidence of feasibility, several aspects are beyond the scope of this work.
Revised: While our approach provides first evidence of feasibility, several aspects are beyond the scope of this work.

Original: The reconstruction error of
Revised: The reconstruction error of

Original: ItNet-post reported in Table 1 is not zero.
Revised: ItNet-post reported in Table 1 is not zero.

Original: However, it is close to numerical precision, and from apractical viewpoint, the inverse problem of the AAPM challenge can be considered ‘solved’ by ourmethod.
Revised: However, it is close to numerical precision, and from a practical viewpoint, the inverse problem of the AAPM challenge can be considered ‘solved’ by our method.

Original: The academic setup of the challenge has provided an ideal experimental area to test ourresearch hypothesis that (near-)exact recovery is possible.
Revised: The academic setup of the challenge has provided an ideal experimental area to test our research hypothesis that (near-)exact recovery is possible.

Original: Although this is an important reliabilitycheck, a foundational understanding of learned reconstruction methods is still in its infancy.
Revised: Although this is an important reliability check, a foundational understanding of learned reconstruction methods is still in its infancy.

Original: Similarcase studies for different inverse problems and more realistic data are natural steps for future research
Revised: Similar case studies for different inverse problems and more realistic data are natural steps for future research.

Revised: Acknowledgments and Disclosure of Funding

Revised: We would like to thank the organizers of the AAPM DL-Sparse-View CT challenge for hosting the event and providing the data and Emil Sidky in particular for interesting and valuable discussions after the end of the challenge period.

Revised: M. G. is supported by the Priority Programme DFG-SPPof the German Research Foundation (DFG).

Original:  A Additional Results and Analysis
Revised: I G. is supported from the Research Training Group DAEDALUS (GRK 2433) funded by the German Research Foundation (DFG). A Additional Results and Analysis

Original: We provide further experiments and results complementing Section 3.
Revised: We provide further experiments and results complementing Section 3.

Original: Data-Consistency.
Revised: Data-Consistency.

Original: We analyze the aspect of data-consistency, i.e., the term y 0 − F · ItNet ( y 0 ) , in
Revised: We analyze the aspect of data-consistency, i.e., the term y 0 − F · ItNet ( y 0 ) , in

Original: Fig.
Revised: Fig.

Original: We observe that the consistency error is dominated by the error from estimating the forwardmodel in Step 1 (see Section 2) indicating that the performance of ItNet can be improved if the exactforward model is available.
Revised: We observe that the consistency error is dominated by the error from estimating the forward model in Step 1 (see Section 2) indicating that the performance of ItNet can be improved if the exact forward model is available.

Original: To verify this hypothesis, we train an ItNet on sinogram data simulatedfrom the ground truth phantom images using our estimated forward model.
Revised: To verify this hypothesis, we train an ItNet on sinogram data simulated from the ground truth phantom images using our estimated forward model.

Original: As expected, the resulting
Revised: As expected, the resulting

Original: ItNetSim shows an improved reconstruction accuracy (about factor 2) and a much reduced dataconsistency error (see Fig.
Revised: ItNetSim shows an improved reconstruction accuracy (about factor 2) and a much reduced dataconsistency error (see Fig.

Original: 3 right image).
Revised: 3 right image).

Original: It is also interesting to note that y 0 − F · Tiramisu ( y 0 )is about a factor 20 larger than the consistency error of ItNet, revealing that the post-processing by
Revised: It is also interesting to note that y 0 − F · Tiramisu ( y 0 ) is about a factor 20 larger than the consistency error of ItNet, revealing that the post-processing by

Original: Tiramisu suffers from a lack of data-consistency.
Revised: Tiramisu suffers from a lack of data-consistency.

Original: The Deeper the Better?
Revised: The Deeper the Better?

Original: Incorporating the forward operator is an essential ingredient for highlyaccurate reconstruction schemes.
Revised: Incorporating the forward operator is an essential ingredient for highly accurate reconstruction schemes.

Original: A central question is how many forward/adjoint operator evaluationsare required.
Revised: A central question is how many forward/adjoint operator evaluations are required.

Original: We address this by training ItNet K for different numbers of iterations.
Revised: We address this by training ItNet K for different numbers of iterations.

Original: 3 Fig.
Revised: 3 Fig.

Original: 4 showsthat, in contrast to classical model-based methods like TV-minimization, only a few forward operatorevaluations sufﬁce to achieve near-exact recovery using ItNet.
Revised: 4 shows that, in contrast to classical model-based methods like TV-minimization, only a few forward operator evaluations suffice to achieve near-exact recovery using ItNet.

Original: Here, we observe a trade-off betweenincreasing the model capacity and the difﬁculty of optimizing the resulting network.
Revised: Here, we observe a trade-off between increasing the model capacity and the difficulty of optimizing the resulting network.

Original: One possibilityto overcome some of the difﬁculties is sharing the UNet parameters between iterations, i.e., setting˜ θ 1 = · · · = ˜ θ K .
Revised: One possibility to overcome some of the difficulties is sharing the UNet parameters between iterations, i.e., setting ˜ θ 1 = · · · = ˜ θ K .

Original: In Fig.
Revised: In Fig.

Original: 4, we observe that (a) not sharing the UNet weights consistently outperforms weight sharing bya small margin independent of the number of iterations; (b) there is a sweetspot at about K = 5 afterwhich the improvement in accuracy resulting from increasing K is negligable and only the trainingtime increases.
Revised: 4, we observe that (a) not sharing the UNet weights consistently outperforms weight sharing by a small margin independent of the number of iterations; (b) there is a sweetspot at about K = 5 after which the improvement in accuracy resulting from increasing K is negligable and only the training time increases.

Original: Fig.
Revised: Fig.

Original: 5 clearly demonstrates that weight sharing also changes the reconstructiondynamic within the ItNet.
Revised: 5 clearly demonstrates that weight sharing also changes the reconstruction dynamic within the ItNet.

Original: Earlier iteration steps of the ItNets trained with weight sharing are moreeffective while the non-weight-shared counterparts draw most of the performance from the later steps.
Revised: Earlier iteration steps of the ItNets trained with weight sharing are more effective while the non-weight-shared counterparts draw most of the performance from the later steps.

Original: We conjecture that an improved training strategy of networks without weight sharing might unlockthe potential of the earlier UNet-blocks.
Revised: We conjecture that an improved training strategy of networks without weight sharing might unlock the potential of the earlier UNet-blocks.

Original: This could lead to an even wider performance gap betweenthe ﬁnal reconstruction accuracy of ItNets with and without weight sharing.
Revised: This could lead to an even wider performance gap between the final reconstruction accuracy of ItNets with and without weight sharing.

Original: A systematic study ofthis aspect is left to future research.
Revised: A systematic study of this aspect is left to future research.

Original: B Details on Step 1: Data-Driven Geometry Identiﬁcation
Revised: B Details on Step 1: Data-Driven Geometry Identification

Original: The fanbeam model is speciﬁed by the following geometric parameters (see Fig. 
Revised: The fanbeam model is specified by the following geometric parameters (see Fig. 6):

Original: • d source – the distance of the X-ray source to the origin, • d detector – the distance of the detector array to the origin, • n detector – the number of detector elements,• s detector – the spacing of the detector elements along the array, • n angle – the number of fan rotation angles, 
Revised: • d source – the distance of the X-ray source to the origin, • d detector – the distance of the detector array to the origin, • n detector – the number of detector elements, • s detector – the spacing of the detector elements along the array, • n angle – the number of fan rotation angles, • φ ∈ [0 , 2 π ] n angle – a discrete list of rotation angles.

Original: • ϕ ∈ [0 , 2 π ] n angle – a discrete list of rotation angles.

Original: In order to ﬁt the above set of parameters to the given training data we ﬁrst observe that theparametrization is redundant and without of loss of generality, we may assume that s detector = 1 (byrescaling d detector appropriately).
Revised: In order to fit the above set of parameters to the given training data we first observe that the parametrization is redundant and without of loss of generality we may assume that s detector = 1 (by rescaling d detector appropriately).

Original: Further, if the ﬁeld-of-view angle γ is known then the relation
Revised: Further if the field-of-view angle γ is known then the relation

Original: [Equation]
Revised: [Equation]

Original: (6) can be used to eliminate another parameter.
Revised: (6) can be used to eliminate another parameter.

Original: Thus, the fanbeam geometry is effectively determinedby the reduced parameter set ( d source , n detector , n angle , ϕ ) , where n angle = 128 and n detector =[-
 [Figure] r o [Figure]-]
Revised: Thus the fanbeam geometry is effectively determined by the reduced parameter set ( d source , n detector , n angle , φ ) , where n angle = 128 and n detector =[-
 [Figure] r o [Figure]-]

Original: a

Original: [Figure]
Revised: [Figure]

Original: can be directly derived from the provided training data.
Revised: can be directly derived from the provided training data.

Original: We determine the ﬁeld of view angle asγ = arcsin(256 /d source ) , so that the maximum inscribed circle in the discrete image is exactlycontained within each fan of lines, which is a common choice for fanbeam CT.
Revised: We determine the field of view angle as γ = arcsin(256 /d source ) , so that the maximum inscribed circle in the discrete image is exactly contained within each fan of lines, which is a common choice for fanbeam CT.

Original: Hence, (6) leads to
Revised: Hence, (6) leads to

Original: [Equation]
Revised: [Equation]

Original: The main difﬁculty of Step 1 in Section 2 lies in the estimation of the remaining parameters ( d source , ϕ ) .
Revised: The main difficulty of Step 1 in Section 2 lies in the estimation of the remaining parameters ( d source , φ ) .

Original: To that end, we have implemented a discrete fanbeam transform from scratch in PyTorch (togetherwith corresponding ﬁltered backprojection).
Revised: To that end, we have implemented a discrete fanbeam transform from scratch in PyTorch (together with corresponding filtered backprojection).

Original: More precisely, we use a ray-driven numerical integrationfor the forward model and a pixel-driven and sinogram-reweighting-based ﬁltered backprojection(with a Hamming ﬁlter) [4, Sec. 3.9.2].
Revised: More precisely, we use a ray-driven numerical integration for the forward model and a pixel-driven and sinogram-reweighting-based filtered backprojection (with a Hamming filter) [4, Sec. 3.9.2].

Original: In addition to the parameters ( d source , ϕ ) , we introducelearnable scaling factors s fwd and s fbp for the forward and inverse transform, respectively.
Revised: In addition to the parameters ( d source , φ ) , we introduce learnable scaling factors s fwd and s fbp for the forward and inverse transform, respectively.

Original: Theyaccount for ambiguities in chosing discretization units of distance compared to actual physical units.
Revised: They account for ambiguities in chosing discretization units of distance compared to actual physical units.

Original: As indicated in Section 2, we estimate the free parameters θ fan =
Revised: As indicated in Section 2, we estimate the free parameters θ fan =

Original: ( s fwd , d source , ϕ
Revised: ( s fwd , d source , φ ) ∈ R 130 of the implemented forward operator F

Original: ) ∈ R 130 of theimplemented forward operator F [ θ fan ] ∈ R m × N in a deep-learning-like fashion: The ability to compute derivatives d F d θ fan allows us to make use of the M = 4000 sinogram-image pairs { ( y i 0 , x i 0 ) } Mi =1by solving, cf.
Revised: [ θ fan ] ∈ R m × N in a deep-learning-like fashion: The ability to compute derivatives d F d θ fan allows us to make use of the M = 4000 sinogram-image pairs { ( y i 0 , x i 0 ) } Mi = by solving, cf.

Original: [Equation]
Revised: [Equation]

Original: (7) with a variant of gradient descent (see remark (i) below).
Revised: (7) with a variant of gradient descent (see remark (i) below).

Original: Finally, we determine θ fbp =
Revised: Finally, we determine θ fbp =

Original: s fbp bysolving, cf.
Revised: s fbp by solving, cf.

Original: [Equation]
Revised: [Equation]

Original: while keeping the already identiﬁed parameters ﬁxed.
Revised: while keeping the already identified parameters fixed.

Original: Some ﬁnal remarks regarding the identiﬁcationof the forward model:
Revised: Some final remarks regarding the identification of the forward model:

Original: (i)
Revised: (i)

Original: The formulation (7) is non-convex and therefore it is not clear whether gradient descent enablesan accurate estimation of the underlying fanbeam geometry.
Revised: The formulation (7) is non-convex and therefore it is not clear whether gradient descent enables an accurate estimation of the underlying fanbeam geometry.

Original: Indeed, standard gradient descentwas found to be sensitive to the initialization of θ fan and got stuck in bad local minima.
Revised: Indeed, standard gradient descent was found to be sensitive to the initialization of θ fan and got stuck in bad local minima.

Original: Toovercome this, we solve (7) by a block coordinate descent instead, alternatingly optimizing overs fwd , d source , and ϕ with individual learning rates.
Revised: To overcome this, we solve (7) by a block coordinate descent instead, alternatingly optimizing over s fwd , d source , and φ with individual learning rates.

Original: This strategy was found to effectively accountfor large deviations of gradient magnitudes of the different parameters.
Revised: This strategy was found to effectively account for large deviations of gradient magnitudes of the different parameters.

Original: Indeed, we observed afast convergence and a reliable identiﬁcation of θ fan , independently of the initialization.(ii)
Revised: Indeed, we observed a fast convergence and a reliable identification of θ fan , independently of the initialization. (ii)

Original: In principle, the strategy of (7) requires only few training samples to be successful.
Revised: In principle, the strategy of (7) requires only few training samples to be successful.

Original: However,when verifying the robustness of the outlined strategy against measurement noise, we observedthat it is beneﬁcial to employ more training data.
Revised: However, when verifying the robustness of the outlined strategy against measurement noise, we observed that it is beneficial to employ more training data.

Original: An example reconstruction comparing the estimated FBP with the FBP provided by the challenge isshown in Fig. 7, which complements Fig.
Revised: An example reconstruction comparing the estimated FBP with the FBP provided by the challenge is shown in Fig. 7, which complements Fig.

Original: reconstructions

Original: We compare the challenge FBP with our own FBP obtained from Step 1.
Revised: We compare the challenge FBP with our own FBP obtained from Step 1.

Original: See Fig.
Revised: See Fig.

Original: 2 for the corresponding network reconstructions
Revised: 2 for the corresponding network reconstructions

Original: C Details on the Exact Challenge Setup
Revised: C Details on the Exact Challenge Setup

Original: To ensure reproducability, we give an exact account of how we trained our winning method ItNet-postfor the AAPM challenge.
Revised: To ensure reproducability, we give an exact account of how we trained our winning method ItNet-post for the AAPM challenge.

Original: Since the systematic investigation of Section 3 and Appendix A of severalaspects of ItNet was done after the competition period, it became clear that not all of the followingdetails have a signiﬁcant impact on the performance.
Revised: Since the systematic investigation of Section 3 and Appendix A of several aspects of ItNet was done after the competition period, it became clear that not all of the following details have a significant impact on the performance.

Original: We start by training an ItNet 4 (with weight sharing) for 500 epochs of mini-batch stochastic gradientdescent and Adam optimizer with an initial learning rate of 8 · 10 − 5 and a batch size of 2 (restarting
Revised: We start by training an ItNet 4 (with weight sharing) for 500 epochs of mini-batch stochastic gradient descent and Adam optimizer with an initial learning rate of 8 · 10 − 5 and a batch size of 2 (restarting

Original: Adam after 250 epochs).
Revised: Adam after 250 epochs).

Original: Then we improve the accuracy by the following post-training strategy:
Revised: Then we improve the accuracy by the following post-training strategy:

Original: First, the ItNet is extended by one more iteration to obtain an ItNet 5 and all ˜ θ k are initialized bythe optimized weights of the previously trained ItNet 4 .
Revised: First, the ItNet is extended by one more iteration to obtain an ItNet 5 and all ˜ θ k are initialized by the optimized weights of the previously trained ItNet 4 .

Original: Then, ItNet-post is ﬁne-tuned by keepingthe weights ˜ θ 1 = ˜ θ 2 =
Revised: Then, ItNet-post is fine-tuned by keeping the weights ˜ θ 1 = ˜ θ 2 =

Original: ˜
Revised: ˜

Original: θ 3 of the ﬁrst three UNets ﬁxed and training only the last two iterations(without weight sharing).
Revised: θ 3 of the first three UNets fixed and training only the last two iterations (without weight sharing).

Original: Hoping for an additional speed-up of the training, we use the initialization( λ 1 , λ 2 , λ 3 , λ 4 ) =
Revised: Hoping for an additional speed-up of the training, we use the initialization ( λ 1 , λ 2 , λ 3 , λ 4 ) =

Original: 08] for training ItNet 4 , which was found by pre-training.
Revised: 08] for training ItNet 4 , which was found by pre-training.

Original: Then,λ 1 , λ 2 , λ 3 of ItNet-post are kept from the ﬁnal values of ItNet 4 , while initializing λ 4 = 1 . [-
 andλ-] 5 = 0 .
Revised: Then, λ 1 , λ 2 , λ 3 of ItNet-post are kept from the final values of ItNet 4 , while initializing λ 4 = 1 . [-
 andλ-]0 and λ 5 = 0 .

Original: 1 for the post-training.
Revised: 1 for the post-training.

Original: To improve the overall performance of our networks, we have additionally applied the following“tricks” for ﬁne tuning, which are ordered by their importance:(
Revised: To improve the overall performance of our networks, we have additionally applied the following “tricks” for fine tuning, which are ordered by their importance:

Original: i)
Revised: (i)

Original: Due to statistical ﬂuctuations, the networks typically exhibit slightly different reconstructionerrors, despite using the same training pipeline.
Revised: Due to statistical fluctuations, the networks typically exhibit slightly different reconstruction errors, despite using the same training pipeline.

Original: For the computation of our ﬁnal reconstructions,we therefore ensemble ten networks, each trained on a different split of the training set.
Revised: For the computation of our final reconstructions, we therefore ensemble ten networks, each trained on a different split of the training set.

Original: (ii)
Revised: (ii)

Original: Due to the training with small batch sizes, we replace batch normalization of the UNetarchitecture by group normalization [21].
Revised: Due to the training with small batch sizes, we replace batch normalization of the UNetarchitecture by group normalization [21].

Original: (iii)
Revised: (iii)

Original: We equip the UNet-architecture with a few memory channels , i.e., one actually has thatU [ θ ]: R N × ( R N ) c mem → R N × ( R N ) c mem (cf.
Revised: We equip the UNet-architecture with a few memory channels , i e., one actually has that U [ θ ]: R N × ( R N ) c mem → R N × ( R N ) c mem (cf.

Original: While the original image-enhancementchannel is not altered, the output of the additional channels is propagated through ItNet, playingthe role of a hidden state (in the spirit of recurrent neural networks).
Revised: While the original image-enhancement channel is not altered, the output of the additional channels is propagated through ItNet, playing the role of a hidden state (in the spirit of recurrent neural networks).

Original: For our experiments, wehave selected c mem = 5 .
Revised: For our experiments, we have selected c mem = 5 .

Original: (iv) It was beneﬁcial to occasionally restart the training of the networks, e.g., see Fig. 8.
Revised: (iv) It was beneficial to occasionally restart the training of the networks, e.g., see Fig. 8.

Original: The following modiﬁcations did not lead to a gain in performance and were omitted:
Revised: The following modifications did not lead to a gain in performance and were omitted:

Original: (i)
Revised: (i)

Original: Improving the FBP in Step 1 by making some of it components learnable (e g., the ﬁlter),cf.
Revised: Improving the FBP in Step 1 by making some of it components learnable (e.g., the filter), cf.

Original: Although this is advantageous for the reconstruction quality of the FBP itself, it leads to worse results for UNet and ItNet.
Revised: Although this is advantageous for the reconstruction quality of the FBP itself, it leads to worse results for UNet and ItNet.

Original: This suggests that a combination of model- and data-based methods beneﬁts most from precise and unaltered physical models.
Revised: This suggests that a combination of model- and data-based methods benefits most from precise and unaltered physical models.

Original: (ii) Adding additional convolutional-blocks in the measurement domain of ItNet.(iii)
Revised: (ii) Adding additional convolutional-blocks in the measurement domain of ItNet.

Original:  Modifying the standard (cid:96) 2 -loss by incorporating the RMSE or the (cid:96) 1 -norm.(iv)
Revised: (iii) Modifying the standard ℓ 2 -loss by incorporating the RMSE or the ℓ 1 -norm.

Original:  Utilizing different optimizers such as RAdam, AdamW, SGD, or MADGRAD.
Revised: (iv) Utilizing different optimizers such as RAdam, AdamW, SGD, or MADGRAD.

Original: In Fig. 8, we visualize the RMSE loss curves of all phases of our complete training pipeline includingoptimizer restarts, i.e., UNet → ItNet 4 + restart → ItNet-post + 2 × restart.
Revised: In Fig. 8, we visualize the RMSE loss curves of all phases of our complete training pipeline including optimizer restarts, i.e., UNet → ItNet 4 + restart → ItNet-post + 2 × restart.