Original: A Closer Look at Reference Learning for Fourier Phase Retrieval
Revised: A Closer Look at Reference Learning for Fourier Phase Retrieval

Original: Author(s)

Original: AfﬁliationAddress

Original: Abstract
Revised: Abstract

Original: Reconstructing images from their Fourier magnitude measurements is a problemthat often arises in different research areas.
Revised: Reconstructing images from their Fourier magnitude measurements is a problem that often arises in different research areas.

Original: This process is also referred to asphase retrieval.
Revised: This process is also referred to as phase retrieval.

Original: In this work, we consider a modiﬁed version of the phase retrieval problem, which allows for a reference image to be added onto the imagebefore the Fourier magnitudes are measured.
Revised: In this work, we consider a modiﬁed version of the phase retrieval problem, which allows for a reference image to be added onto the image before the Fourier magnitudes are measured.

Original: We analyze an unrolled GerchbergSaxton (GS) algorithm that can be used to learn a good reference image froma dataset.
Revised: We analyze an unrolled Gerchberg-Saxton (GS) algorithm that can be used to learn a good reference image from a dataset.

Original: Furthermore, we take a closer look at the learned reference imagesand propose a simple and efﬁcient heuristic to construct reference images that, insome cases, yields reconstructions of comparable quality as approaches that learnreferences.
Revised: Furthermore, we take a closer look at the learned reference images and propose a simple and efﬁcient heuristic to construct reference images that, in some cases, yields reconstructions of comparable quality as approaches that learn references.

Original: Our code is available at https://anonymous.4open.science/r/reference-learning-51F8/ .
Revised: Our code is available at https://github.com/tuelwer/reference-learning .

Original:  Introduction
Revised:  Introduction

Original: In general, Fourier phase retrieval is the problem of reconstructing an image from its Fourier magnitude measurements.
Revised: In general, Fourier phase retrieval is the problem of reconstructing an image from its Fourier magnitude measurements.

Original: The problem arises in different research areas, e.g., in X-ray crystallography[11], astronomical imaging [3], optics [15], array imaging [1], or microscopy [17].
Revised: The problem arises in different research areas, e.g., in X-ray crystallography [11], astronomical imaging [3], optics [15], array imaging [1], or microscopy [17].

Original: In particular, fornon-oversampled measurements, phase retrieval has not been fully solved yet.
Revised: In particular, for non-oversampled measurements, phase retrieval has not been fully solved yet.

Original: In this work, a slightlymodiﬁed version of the problem is considered: instead of reconstructing the image from the plainmagnitude measurements, a reference image is added onto the original image before the Fouriermagnitudes are measured, i.e., we reconstruct the image x ∈ R d × d from the modiﬁed measurements
Revised: In this work, a slightly modiﬁed version of the problem is considered: instead of reconstructing the image from the plain magnitude measurements, a reference image is added onto the original image before the Fourier magnitudes are measured, i.e., we reconstruct the image x ∈ R d × d from the modiﬁed measurements

Original: [Equation]
Revised: [Equation]

Original: where F denotes the discrete two-dimensional Fourier transform and u ∈ R
Revised: where F denotes the discrete two-dimensional Fourier transform and u ∈ R

Original: d × d is a known referenceimage.
Revised: d × d is a known reference image.

Original: Furthermore, we assume that both the image and the reference are non-negative.
Revised: Furthermore, we assume that both the image and the reference are non-negative.

Original: Figure 1gives an overview of the measurement process, that can also be implemented in practice.
Revised: Figuregives an overview of the measurement process, that can also be implemented in practice.

Original:  Related Work and our Contributions
Revised:  Related Work and our Contributions

Original: The modiﬁed phase retrieval problem was ﬁrst analyzed by Kim and Hayes [7, 6].
Revised: The modiﬁed phase retrieval problem was ﬁrst analyzed by Kim and Hayes [7, 6].

Original: Recently, Hyderet al.
Revised: Recently, Hyder et al.

Original: [5] showed that such a reference can be learned from a rather small dataset using an unrolledgradient descent algorithm.
Revised: [5] showed that such a reference can be learned from a rather small dataset using an unrolled gradient descent algorithm.

Original: While it is interesting that this is possible, a couple of questions arisewhich we discuss in this work.
Revised: While it is interesting that this is possible, a couple of questions arise which we discuss in this work.

Original: [Equation]
Revised: [Equation]

Original: Our Contributions:
Revised: Our Contributions:

Original: We modify the well-established Gerchberg-Saxton (GS) algorithm [4] so that it utilizes areference.
Revised: We modify the well-established Gerchberg-Saxton (GS) algorithm [4] so that it utilizes a reference.

Original: Furthermore, we show that this modiﬁed GS algorithm can be unrolled to learn areference image (similar to the approach of Hyder et al.
Revised: Furthermore, we show that this modiﬁed GS algorithm can be unrolled to learn a reference image (similar to the approach of Hyder et al.

Original: We answer the question under which conditions learning a reference image for Fourier phase retrieval is really necessary.
Revised: We answer the question under which conditions learning a reference image for Fourier phase retrieval is really necessary.

Original: For this, we propose a simple baseline reference image that is easily constructed and compare its performance with the learned references in the oversampled and the non-oversampled case.
Revised: For this, we propose a simple baseline reference image that is easily constructed and compare its performance with the learned references in the oversampled and the non-oversampled case.

Original: The idea of unrolling algorithms has been considered before for other phase retrieval problems:
Revised: The idea of unrolling algorithms has been considered before for other phase retrieval problems:

Original: The performance of an unrolled GS algorithm for phase retrieval has been analyzed by Schlieder etal.
Revised: The performance of an unrolled GS algorithm for phase retrieval has been analyzed by Schlieder et al.

Original: Naimipour et al.
Revised: Naimipour et al.

Original: [12] unrolled the Wirtinger ﬂow algorithm for compressive phase retrieval.
Revised: [12] unrolled the Wirtinger ﬂow algorithm for compressive phase retrieval.

Original: Unlike our work, these papers augment existing phase retrieval algorithms with learnable parameterswhich is different from the reference based phase retrieval we consider in this work.
Revised: Unlike our work, these papers augment existing phase retrieval algorithms with learnable parameters which is different from the reference based phase retrieval we consider in this work.

Original:  Unrolling the Gerchberg-Saxton Algorithm to Learn a Reference
Revised:  Unrolling the Gerchberg-Saxton Algorithm to Learn a Reference

Original: The phase retrieval algorithm discussed in Hyder et al.
Revised: The phase retrieval algorithm discussed in Hyder et al.

Original: [5] is a gradient descent method that requiresextensive parameter tuning of its step size.
Revised: [5] is a gradient descent method that requires extensive parameter tuning of its step size.

Original: For this reason, we unroll the GS algorithm, whichtypically converges within a few iterations.
Revised: For this reason, we unroll the GS algorithm, which typically converges within a few iterations.

Original: The GS algorithm iteratively replaces the magnitude ofthe current estimate with the measured magnitude and enforces a positivity-constraint on the imageafter the reference has been subtracted.
Revised: The GS algorithm iteratively replaces the magnitude of the current estimate with the measured magnitude and enforces a positivity-constraint on the image after the reference has been subtracted.

Original: Algorithm 1 shows our modiﬁed version of GS that utilizesa reference image.
Revised: Algorithm 1 shows our modiﬁed version of GS that utilizes a reference image.

Original: We learn the reference by calculating the mean squared error (MSE) betweenoutputs of the GS algorithm after n iterations and the corresponding original images.
Revised: We learn the reference by calculating the mean squared error (MSE) between outputs of the GS algorithm after n iterations and the corresponding original images.

Original: Then weperform backpropagation to calculate the gradient with respect to the reference u and update it usinga gradient descent step.
Revised: Then we perform backpropagation to calculate the gradient with respect to the reference u and update it using a gradient descent step.

Original: These steps are repeated for multiple batches until convergence is achieved.
Revised: These steps are repeated for multiple batches until convergence is achieved.

Original: retrieval with a reference image

Original: Input: Fourier magnitude y ∈ R d × d , reference image u ∈ R d × d , initialization x 0 ∈ R d × d , number of iterations n
Revised: Input: Fourier magnitude [Equation] × d , initialization x 0 ∈ R d × d , number of iterations n

Original: Output: Reconstruction x n ∈ R d × d 1 for k = 1 , . . . , n − 1 do p k +1 ← F ( x k + u ) /
Revised: Output: Reconstruction x n ∈ R d × d 1 for k = 1 , . . . , n − 1 do p k +1 ← F ( x k + u ) /

Original: |F
Revised: |F

Original: ( x k + u ) | // Estimate phase ¯ x k +1 ← F − 1 ( p k +1 (cid:12) y )
Revised: ( x k + u ) | // Estimate phase ¯ x k +1 ← F − 1 ( p k +1 (cid:12) y )

Original: − u //
Revised: − u //

Original: Fourier constraints and subtract u x k +1 ← max(0 , ¯ x k +1 ) //
Revised: Fourier constraints and subtract u x k +1 ← max(0 , ¯ x k +1 ) //

Original: Image constraints (max is element-wise) 5 end 6 return x n
Revised: Image constraints (max is element-wise) 5 end 6 return x n

Original:  Constructing Simple References Mimicking the Learned Ones
Revised:  Constructing Simple References Mimicking the Learned Ones

Original: Learning the reference image is computationally expensive.
Revised: Learning the reference image is computationally expensive.

Original: The most obvious baseline reference is arandom image.
Revised: The most obvious baseline reference is a random image.

Original: For this, we sample the pixel values from a uniform distribution U (0 , 1) .
Revised: For this, we sample the pixel values from a uniform distribution U (0 , 1) .

Original: Looking atvarious learned reference images produced by the algorithm of Hyder et al.
Revised: Looking at various learned reference images produced by the algorithm of Hyder et al.

Original: [5] and our unrolled GS
Revised: [5] and our unrolled GS

Original: [Figure]
Revised: [Figure]

Original: algorithm (Figure 2), we see that most pixels are either zero or one.
Revised: algorithm (Figure 2), we see that most pixels are either zero or one.

Original: This suggests to also considerrandom binarized reference images.
Revised: This suggests to also consider random binarized reference images.

Original: However, looking more closely at those learned reference images (Figure 2), we see that all references exhibit ﬂat areas.
Revised: However, looking more closely at those learned reference images (Figure 2), we see that all references exhibit ﬂat areas.

Original: Furthermore, the learned references do not show any symmetries.
Revised: Furthermore, the learned references do not show any symmetries.

Original: Theseobservations suggest trying a simple reference image as a baseline mimicking those features.
Revised: These observations suggest trying a simple reference image as a baseline mimicking those features.

Original: Toconstruct such a reference, we start with a black image that has a white square in the bottom rightcorner.
Revised: To construct such a reference, we start with a black image that has a white square in the bottom right corner.

Original: After blurring this image with a Gaussian ﬁlter, we normalize and add weighted Poisson noiseto every pixel.
Revised: After blurring this image with a Gaussian ﬁlter, we normalize and add weighted Poisson noise to every pixel.

Original: Finally, we binarize the image appropriately.
Revised: Finally, we binarize the image appropriately.

Original: A resulting simple reference is shown onthe right in Figure 2.
Revised: A resulting simple reference is shown on the right in Figure 2.

Original: In the next section, we compare the performance of these different references for reconstructingimages from their oversampled and non-oversampled magnitude measurements.
Revised: In the next section, we compare the performance of these different references for reconstructing images from their oversampled and non-oversampled magnitude measurements.

Original:  Comparing Baseline and Learned References
Revised:  Comparing Baseline and Learned References

Original: [Figure]
Revised: [Figure]

Original: Experimental Setup.
Revised: Experimental Setup.

Original: To study whether references arenecessary at all, we also reconstruct images from measurements that were taken without adding a reference onto theimage.
Revised: To study whether references are necessary at all, we also reconstruct images from measurements that were taken without adding a reference onto the image.

Original: In that case, the trivial ambiguities (translation andﬂip) cannot be resolved by the reconstruction algorithm.
Revised: In that case, the trivial ambiguities (translation and ﬂip) cannot be resolved by the reconstruction algorithm.

Original: Therefore, we register the reconstructions appropriately before calculating the error.
Revised: Therefore, we register the reconstructions appropriately before calculating the error.

Original: For fair comparison, we registerthe reconstructions of the other methods before calculatingthe MSE as well.
Revised: For fair comparison, we register the reconstructions of the other methods before calculating the MSE as well.

Original: We evaluate the reconstructions of images from the
Revised: We evaluate the reconstructions of images from the

Original: MNIST [10], the EMNIST [2], the FMNIST [16], the
Revised: MNIST [10], the EMNIST [2], the FMNIST [16], the

Original: SVHN [13], and the CIFAR-10 [9] datasets.
Revised: SVHN [13], and the CIFAR-10 [9] datasets.

Original: We convertthe images from the SVHN and the CIFAR-10 dataset tograyscale.
Revised: We convert the images from the SVHN and the CIFAR-10 dataset to grayscale.

Original: We consider non-oversampled magnitude measurements as well as measurements that are oversampledby a factor of two in each dimension.
Revised: We consider non-oversampled magnitude measurements as well as measurements that are oversampled by a factor of two in each dimension.

Original: To solve the Fourierphase retrieval problem, we use a similar algorithm as
Revised: To solve the Fourier phase retrieval problem, we use a similar algorithm as

Original: Hyder et al.
Revised: Hyder et al.

Original: We run the algorithm for 500 steps andset the step size to α = 1 . 95 , which was chosen on avalidation dataset.
Revised: We run the algorithm for 500 steps and set the step size to α = 1 . 95 , which was chosen on a validation dataset.

Original: For each of the datasets, we use 1000images for testing and we train the references on 100 training images using Adam [8] with a learningrate of 0 .
Revised: For each of the datasets, we useimages for testing and we train the references on 100 training images using Adam [8] with a learning rate of 0 .

Original: We use batches of size 10 .
Revised: We use batches of size 10 .

Original: During training, we unroll the GS algorithm for 15 steps anduse 500 steps to reconstruct the images at test time.
Revised: During training, we unroll the GS algorithm for 15 steps and use 500 steps to reconstruct the images at test time.

Original: Similar to the intensities of the images, we restrict the entries of the reference image to lie between 0and 1 as using larger values makes the problem trivial to solve, i.e., even a random reference withvalues between 0 and 100 yields perfect reconstructions.
Revised: Similar to the intensities of the images, we restrict the entries of the reference image to lie betweenand 1 as using larger values makes the problem trivial to solve, i.e., even a random reference with values between 0 and 100 yields perfect reconstructions.

Original: Results.
Revised: Results.

Original: Table 1 shows that our unrolled GS algorithm is able to learn references of similar qualityas the method proposed by Hyder et al.
Revised: Table 1 shows that our unrolled GS algorithm is able to learn references of similar quality as the method proposed by Hyder et al.

Original: We see that the reconstruction errors of our simplereference are quite close to the errors of the learned methods in the oversampled case.
Revised: We see that the reconstruction errors of our simple reference are quite close to the errors of the learned methods in the oversampled case.

Original: Figure 3 showsthe training and the validation loss for a reference trained using our unrolled GS algorithm on the
Revised: Figure 3 shows the training and the validation loss for a reference trained using our unrolled GS algorithm on the

Original: FMNIST dataset.
Revised: FMNIST dataset.

Original: We can observe that only a few stochastic gradient descent steps are necessaryto drastically decrease training and validation error.
Revised: We can observe that only a few stochastic gradient descent steps are necessary to drastically decrease training and validation error.

Original: Also, our GS algorithm requires only a small
Revised: Also, our GS algorithm requires only a small

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: e r s a m p l e d
Revised: e r s a m p l e d

Original: O v e r s a m p l e d number of iterations in training to learn a good reference.
Revised: O v e r s a m p l e d number of iterations in training to learn a good reference.

Original: Figure 4 visualizes some reconstructions.
Revised: Figure 4 visualizes some reconstructions.

Original: In the non-oversampled case, the learned references produce better reconstructions than our simplereference.
Revised: In the non-oversampled case, the learned references produce better reconstructions than our simple reference.

Original: Surprisingly, the random references that we use as baseline perform worse on the MNISTlike datasets than using no baseline in the non-oversampled case.
Revised: Surprisingly, the random references that we use as baseline perform worse on the MNISTlike datasets than using no baseline in the non-oversampled case.

Original: In the oversampled case, the learnedreferences, as well as our simple reference, produce almost perfect reconstructions.
Revised: In the oversampled case, the learned references, as well as our simple reference, produce almost perfect reconstructions.

Original: Therefore, weconclude that learning references is not necessary when the Fourier magnitudes are oversampled.
Revised: Therefore, we conclude that learning references is not necessary when the Fourier magnitudes are oversampled.

Original:  Conclusion
Revised:  Conclusion

Original: [Figure] In this paper, we discuss how the GS algorithm can be unrolled in order to learn a reference image forphase retrieval.
Revised: In this paper, we discuss how the GS algorithm can be unrolled in order to learn a reference image for phase retrieval.

Original: We show that our unrolled GS algorithm performs comparably to an existing methodand requires only a few unrolling steps while having no hyperparameters that require extensivetuning.
Revised: We show that our unrolled GS algorithm performs comparably to an existing method and requires only a few unrolling steps while having no hyperparameters that require extensive tuning.

Original: Furthermore, we provide a simple reference which puts the beneﬁt of a learned reference intoperspective, especially when oversampled magnitude measurements are available.
Revised: Furthermore, we provide a simple reference which puts the beneﬁt of a learned reference into perspective, especially when oversampled magnitude measurements are available.

Original:  Broader Impact
Revised:  Broader Impact

Original: Fourier phase retrieval is a fundamental and relevant imaging problem in many areas of science,e.g., in X-ray crystallography or microscopy.
Revised: Fourier phase retrieval is a fundamental and relevant imaging problem in many areas of science, e.g., in X-ray crystallography or microscopy.

Original: Ethical or societal consequences depend on the exactapplication of the phase retrieval algorithm.
Revised: Ethical or societal consequences depend on the exact application of the phase retrieval algorithm.

Original: Our insights set the performance gain of learned referenceimages for phase retrieval into perspective.
Revised: Our insights set the performance gain of learned reference images for phase retrieval into perspective.

Original: We propose a simpler and more efﬁcient approach forreference construction which yields similar results and requires less computation and thus less energy.
Revised: We propose a simpler and more efﬁcient approach for reference construction which yields similar results and requires less computation and thus less energy