Original: Data-Driven Computational Imaging for Scientiﬁc Discovery
Revised: Data-Driven Computational Imaging for Scientiﬁc Discovery

Original: Author s

Original: Abstract
Revised: Abstract

Original: In computational imaging, hardware for signal sampling and software for object reconstruction are designed in tandem for improved capability.
Revised: In computational imaging, hardware for signal sampling and software for object reconstruction are designed in tandem for improved capability.

Original: Examples of such systems include computed tomography (CT), magnetic resonance imaging (MRI), and superresolution microscopy.
Revised: Examples of such systems include computed tomography (CT), magnetic resonance imaging (MRI), and superresolution microscopy.

Original: In contrast to more traditional cameras, in these devices, indirect measurements are taken and computational algorithms are used for reconstruction.
Revised: In contrast to more traditional cameras, in these devices, indirect measurements are taken and computational algorithms are used for reconstruction.

Original: This allows for advanced capabilities such as super-resolution or 3dimensional imaging, pushing forward the frontier of scientiﬁc discovery.
Revised: This allows for advanced capabilities such as super-resolution or 3dimensional imaging, pushing forward the frontier of scientiﬁc discovery.

Original: However, these techniques generally require a large number of measurements, causing low throughput, motion artifacts, and/or radiation damage, limiting applications.
Revised: However, these techniques generally require a large number of measurements, causing low throughput, motion artifacts, and/or radiation damage, limiting applications.

Original: Datadriven approaches to reducing the number of measurements needed have been proposed, but they predominately require a ground truth or reference dataset, which may be impossible to collect.
Revised: Datadriven approaches to reducing the number of measurements needed have been proposed, but they predominately require a ground truth or reference dataset, which may be impossible to collect.

Original: This work outlines a self-supervised approach and explores the future work that is necessary to make such a technique usable for real applications.
Revised: This work outlines a self-supervised approach and explores the future work that is necessary to make such a technique usable for real applications.

Original: Light-emitting diode (LED) array microscopy, a modality that allows visualization of transparent objects in two and three dimensions with high resolution andhigh ﬁeld-of-view, is used as an illustrative example.
Revised: Light-emitting diode (LED) array microscopy, a modality that allows visualization of transparent objects in two and three dimensions with high resolution and ﬁeld-of-view, is used as an illustrative example.

Original: We release our code at https://anonymous.4open.science/r/LED_PVAE-1290 and our experimental data at https://figshare.com/s/635499acfdcdf0893750 .
Revised: We release our code at https://github.com/vganapati/LED_PVAE and our experimental data at https://doi.org/10.6084/m9.figshare.21232088 .

Original:  Introduction
Revised:  Introduction

Original: Computational imaging systems, which reconstruct objects from indirect measurements, are ubiquitous in modern scientiﬁc research.
Revised: Computational imaging systems, which reconstruct objects from indirect measurements, are ubiquitous in modern scientiﬁc research.

Original: For example, computed tomography allows for 3-dimensional visualization by collecting x-ray projections through an object, and is used in a range of ﬁelds including medicine [1], biology [2], materials science [3] and geoscience [4].
Revised: For example, computed tomography allows for 3-dimensional visualization by collecting x-ray projections through an object, and is used in a range of ﬁelds including medicine [1], biology [2], materials science [3] and geoscience [4].

Original: Another computational imaging modality, light-emitting diode (LED) array microscopy (also known as Fourier ptychographic microscopy) has shown success in quantitative 2-dimensional and 3-dimensional phase imaging [5–11], with applications in pathology [12,13] and biology [14,15].
Revised: Another computational imaging modality, light-emitting diode (LED) array microscopy (also known as Fourier ptychographic microscopy), has shown success in quantitative 2-dimensional and 3-dimensional phase imaging [5–11], with applications in pathology [12,13] and biology [14,15].

Original: Though computational imaging methods have achieved a high degree of success in spatial resolution, the temporal resolution (imaging speed) remains low.
Revised: Though computational imaging methods have achieved a high degree of success in spatial resolution, the temporal resolution (imaging speed) remains low.

Original: Increasing speed remains an active area of research, as success will allow visualization in unprecedented regimes of spatial and temporal resolution.
Revised: Increasing speed remains an active area of research, as success will allow visualization in unprecedented regimes of spatial and temporal resolution.

Original: In x-ray computed tomography, for example, thousands of 2-dimensional images must be collected for eventual object reconstruction.
Revised: In x-ray computed tomography, for example, thousands of 2-dimensional images must be collected for eventual object reconstruction.

Original: In LED array microscopy, the light source of a conventional wide-ﬁeld microscope is replaced with a 2-dimensional LED array.
Revised: In LED array microscopy, the light source of a conventional wide-ﬁeld microscope is replaced with a 2-dimensional LED array.

Original: Each LED of the array is individually addressable with tunable brightness, allowing different patterns to be illuminated.
Revised: Each LED of the array is individually addressable with tunable brightness, allowing different patterns to be illuminated.

Original: Generally, one image is collected per LED, and arrays may consist of hundreds of LEDs.
Revised: Generally, one image is collected per LED, and arrays may consist of hundreds of LEDs.

Original: Data-driven deep learning methods, using a reference training dataset of measurement-reconstruction pairs, have been widely proposed to improve the temporal resolution of computational imaging.
Revised: Data-driven deep learning methods, using a reference training dataset of measurement-reconstruction pairs, have been widely proposed to improve the temporal resolution of computational imaging.

Original: However, the necessity of a training dataset makes the technique prohibitive in many applications of scientiﬁc discovery.
Revised: However, the necessity of a training dataset makes the technique prohibitive in many applications of scientiﬁc discovery.

Original: A chicken-and-egg problem arises in the case of fragile or live specimens: without a reference object dataset, we cannot create a faster imaging method, but without the faster imaging method the training object dataset cannot be obtained.
Revised: A chicken-and-egg problem arises in the case of fragile or live specimens: without a reference object dataset, we cannot create a faster imaging method, but without the faster imaging method the training object dataset cannot be obtained.

Original: In this work, we outline a reconstruction method that only requires a representative dataset of sparse or partial measurements on each object.
Revised: In this work, we outline a reconstruction method that only requires a representative dataset of sparse or partial measurements on each object.

Original: To circumvent the need for complete training dataset pairs, we look to jointly reconstruct a set of similar objects, each with a low number of measurements.
Revised: To circumvent the need for complete training dataset pairs, we look to jointly reconstruct a set of similar objects, each with a low number of measurements.

Original: By pooling information from measurements across the set and incorporating the known forward physics of imaging, we aim to jointly infer the prior distribution and posterior distributions.
Revised: By pooling information from measurements across the set and incorporating the known forward physics of imaging, we aim to jointly infer the prior distribution and posterior distributions.

Original: We aim to allow for improved reconstructions with fewer measurements per object by using information from multiple similar objects.
Revised: We aim to allow for improved reconstructions with fewer measurements per object by using information from multiple similar objects.

Original: More precisely, computational imaging aims to reconstruct some object O from a sequence of n noisy measurements
Revised: More precisely, computational imaging aims to reconstruct some object O from a sequence of n noisy measurements

Original: M = [ M 1 , M 2 , ..., M n ] .
Revised: M = [ M 1 , M 2 , ..., M n ] .

Original: We aim to lower the total number of measurements n to minimize data acquisition time.
Revised: We aim to lower the total number of measurements n to minimize data acquisition time.

Original: We assume that we have a set of m objects { O 1 , O 2 , ..., O m } , sampled from some distribution P ( O ) , and we aim to reconstruct all objects in the set.
Revised: We assume that we have a set of m objects { O 1 , O 2 , ..., O m } , sampled from some distribution P ( O ) , and we aim to reconstruct all objects in the set.

Original: For each of the m objects, we are allowed n measurements.
Revised: For each of the m objects, we are allowed n measurements.

Original: Each sequence of measurements for an object j ,
Revised: Each sequence of measurements for an object j ,

Original: M j = [ M j 1 , M j 2 , ..., M jn ] is obtained with chosen hardware parameters p
Revised: M j = [ M j 1 , M j 2 , ..., M jn ] is obtained with chosen hardware parameters p

Original: j
Revised: j

Original: =
Revised: =

Original: [ p j 1 , p j 2 , ..., p jn ]
Revised: [ p j 1 , p j 2 , ..., p jn ]

Original: (e.g. rotation angles in the case of computed tomography or the LED illumination patterns in the case of LED array microscopy).
Revised: (e.g. rotation angles in the case of computed tomography or the LED illumination patterns in the case of LED array microscopy).

Original: We assume that the forward model physics P ( M | O ; p ) =
Revised: We assume that the forward model physics P ( M | O ; p ) =

Original: P ( M | O ) is known.
Revised: P ( M | O ) is known.

Original: For every object O , we aim to ﬁnd the posterior distribution P ( O | M ) =
Revised: For every object O , we aim to ﬁnd the posterior distribution P ( O | M ) =

Original: P ( M | O ) P ( O ) P ( M ) .
Revised: P ( M | O ) P ( O ) P ( M ) .

Original: The following problems arise in ﬁnding the posterior: (1) construction of the prior P ( O ) with no directly observed O , only indirect measurements M on each object of the set, and (2) calculating
Revised: The following problems arise in ﬁnding the posterior: (1) construction of the prior P ( O ) with no directly observed O , only indirect measurements M on each object of the set, and (2) calculating

Original: P ( O | M ) in a tractable manner.
Revised: P ( O | M ) in a tractable manner.

Original: To efﬁciently solve this problem, we create a novel technique through a reformulation of variational autoencoders.
Revised: To efﬁciently solve this problem, we create a novel technique through a reformulation of variational autoencoders.

Original: The probabilistic formulation considered in this work permits uncertainty quantiﬁcation, in contrast to most reconstruction algorithms that only yield a point estimate.
Revised: The probabilistic formulation considered in this work permits uncertainty quantiﬁcation, in contrast to most reconstruction algorithms that only yield a point estimate.

Original:  Related Work
Revised:  Related Work

Original: Deep learning has been widely applied to reduce the data acquisition burden of computational imaging systems.
Revised: Deep learning has been widely applied to reduce the data acquisition burden of computational imaging systems.

Original: In one line of research, training pairs of sparse measurements and corresponding high quality reconstructions are used to train a deep convolutional neural network, and implicitly embed prior information [16–40].
Revised: In one line of research, training pairs of sparse measurements and corresponding high quality reconstructions are used to train a deep convolutional neural network, and implicitly embed prior information [16–40].

Original: Subsequent sparse measurements can be reconstructed with a forward pass of the trained neural network, with the beneﬁt of avoiding computationally costly iterative algorithms.
Revised: Subsequent sparse measurements can be reconstructed with a forward pass of the trained neural network, with the beneﬁt of avoiding computationally costly iterative algorithms.

Original: Deep neural network approaches for object reconstruction have the advantage of incorporating knowledge about prior data and fast inference, but more traditional iterative (model-based) methods have the advantage of utilizing the known forward physics model (i.e. how measurements are generated, given the object).
Revised: Deep neural network approaches for object reconstruction have the advantage of incorporating knowledge about prior data and fast inference, but more traditional iterative (model-based) methods have the advantage of utilizing the known forward physics model (i.e. how measurements are generated, given the object).

Original: The advantages of these two approaches are combined by unrolling an iterative method, with each iteration forming a layer of a neural network [41–53].
Revised: The advantages of these two approaches are combined by unrolling an iterative method, with each iteration forming a layer of a neural network [41–53].

Original: This unrolled deep neural network can be trained to optimize iterative algorithm hyperparameters for a given training dataset, effectively optimizing an optimizer.
Revised: This unrolled deep neural network can be trained to optimize iterative algorithm hyperparameters for a given training dataset, effectively optimizing an optimizer.

Original: The unrolled iterative methods have been shown to require less training data and time than a convolutional neural network approach, due to the incorporation of the forward model.
Revised: The unrolled iterative methods have been shown to require less training data and time than a convolutional neural network approach, due to the incorporation of the forward model.

Original: Building off of this literature, a second body of approaches attempts to include the measurement process in neural network training, to discern the optimal measurement parameters (e.g. the LED illumination patterns in LED array microscopy) for sparse sampling and subsequent reconstruction.
Revised: Building off of this literature, a second body of approaches attempts to include the measurement process in neural network training, to discern the optimal measurement parameters (e.g. the LED illumination patterns in LED array microscopy) for sparse sampling and subsequent reconstruction.

Original: In this work, high quality reconstructions are needed, and corresponding noisy measurements are emulated with the known forward physics.
Revised: In these works, high quality reconstructions are needed, and corresponding noisy measurements are emulated with the known forward physics.

Original: The measurement process is included as the encoder part of an autoencoder neural network, and co-optimized with the reconstruction algorithm, which forms the decoder.
Revised: The measurement process is included as the encoder part of an autoencoder neural network, and co-optimized with the reconstruction algorithm, which forms the decoder.

Original: Many works use a convolutional neural network as the decoder [54–75] and others use an unrolled iterative solver [76–78].
Revised: Many works use a convolutional neural network as the decoder [54–75] and others use an unrolled iterative solver [76–78].

Original: Co-optimizing the measurement parameters has the beneﬁt of reducing the measurements required for computational imaging further than keeping them ﬁxed during training.
Revised: Co-optimizing the measurement parameters has the beneﬁt of reducing the measurements required for computational imaging further than keeping them ﬁxed during training.

Original: However, this approach still requires a reference training dataset.
Revised: However, this approach still requires a reference training dataset.

Original: In this work, we look to remove the need for ground-truth or reference reconstructions.
Revised: In this work, we look to remove the need for ground-truth or reference reconstructions.

Original: We aim to create a reconstruction method that only requires a representative dataset of sparse measurements on each object.
Revised: We aim to create a reconstruction method that only requires a representative dataset of sparse measurements on each object.

Original: This task has been previously undertaken, usually with generative adversarial networks [79–87].
Revised: This task has been previously undertaken, usually with generative adversarial networks [79–87].

Original: The intuition is that by using different experimental measurement parameters for every object of the set, it is possible to build a general understanding of what an object should look like (i.e. the prior).
Revised: The intuition is that by using different experimental measurement parameters for every object of the set, it is possible to build a general understanding of what an object should look like (i.e. the prior).

Original: However, these methods all lack probabilistic outputs.
Revised: However, these methods all lack probabilistic outputs.

Original: In this work, we propose a method based on variational autoencoders that solves for the posterior distribution in a principled manner and outline some of the progress required to make this technique usable for scientiﬁc discovery.
Revised: In this work, we propose a method based on variational autoencoders that solves for the posterior distribution in a principled manner and outline some of the progress required to make this technique usable for scientiﬁc discovery.

Original:  Physics-Informed Variational Autoencoder
Revised:  Physics-Informed Variational Autoencoder

Original: We assume that we have a set of m objects { O 1 , O 2 , ..., O m } drawn from P ( O ) , where P ( O ) is unknown and we cannot directly measure O .
Revised: We assume that we have a set of m objects { O 1 , O 2 , ..., O m } drawn from P ( O ) , where P ( O ) is unknown and we cannot directly measure O .

Original: For each object O , we are allowed to take n indirect measurements M =
Revised: For each object O , we are allowed to take n indirect measurements M =

Original: [ M 1 , M 2 , ..., M n ] .
Revised: [ M 1 , M 2 , ..., M n ] .

Original: The measurement M on an object O is obtained with chosen hardware parameters p
Revised: The measurement M on an object O is obtained with chosen hardware parameters p

Original: = [ p 1 , p 2 , ..., p n ] , and the forward model P ( M | O ; p ) =
Revised: = [ p 1 , p 2 , ..., p n ] , and the forward model P ( M | O ; p ) =

Original: P ( M | O ) is known through the physics of image formation.
Revised: P ( M | O ) is known through the physics of image formation.

Original: We aim to determine the posterior distribution P ( O | M ) for every object in the set.
Revised: We aim to determine the posterior distribution P ( O | M ) for every object in the set.

Original: We propose a general framework for posterior estimation that is inspired by the mathematics of the variational autoencoder [88,89].
Revised: We propose a general framework for posterior estimation that is inspired by the mathematics of the variational autoencoder [88,89].

Original: In a variational autoencoder, the goal is to learn how to generate new examples, sampled from the same underlying probability distribution as a training dataset of objects.
Revised: In a variational autoencoder, the goal is to learn how to generate new examples, sampled from the same underlying probability distribution as a training dataset of objects.

Original: To accomplish this task, a latent random variable z is created that describes the space on a lower-dimensional manifold.
Revised: To accomplish this task, a latent random variable z is created that describes the space on a lower-dimensional manifold.

Original: A deep neural network deﬁnes a function (the “decoder”) from a sample of z to a probability distribution P ( O | z ) , see Fig.
Revised: A deep neural network deﬁnes a function (the “decoder”) from a sample of z to a probability distribution P ( O | z ) , see Fig.

Original: Deep neural networks are chosen for function approximation due to their theoretical ability to approximate any function [90–92] and practical success in approximating high-dimensional functions [93].
Revised: Deep neural networks are chosen for function approximation due to their theoretical ability to approximate any function [90–92] and practical success in approximating high-dimensional functions [93].

Original: The parameters of the deep neural network are optimized to maximize the probability of generating the objects in the training set.
Revised: The parameters of the deep neural network are optimized to maximize the probability of generating the objects in the training set.

Original: In this work, we aim to ﬁnd the posterior probability distribution P ( O | M ) , where O is the object being reconstructed and M is the measurement.
Revised: In this work, we aim to ﬁnd the posterior probability distribution P ( O | M ) , where O is the object being reconstructed and M is the measurement.

Original: In our case, we only have a dataset of noisy measurements M and no ground truth objects O , but a known forward model, P ( M | O ) .
Revised: In our case, we only have a dataset of noisy measurements M and no ground truth objects O , but a known forward model, P ( M | O ) .

Original: Thus, instead of maximizing the probability of generating O , we can maximize the probability of generating M , a formulation we call the “physics-informed variational autoencoder.”
Revised: Thus, instead of maximizing the probability of generating O , we can maximize the probability of generating M , a formulation we call the “physics-informed variational autoencoder.”

Original: We aim to maximize P ( M ) =
Revised: We aim to maximize P ( M ) =

Original: (cid:82) (cid:82) P ( M | O ) P
Revised: (cid:82) (cid:82) P ( M | O ) P

Original: ( O | z ) P ( z ) dOdz .
Revised: ( O | z ) P ( z ) dOdz .

Original: To compute this integral in a computationally tractable manner, we can approximate with sampled values.
Revised: To compute this integral in a computationally tractable manner, we can approximate with sampled values.

Original: However, for most values of z and O , the probability P ( M | O, z ) is close to zero, causing poor scaling of sampled estimates.
Revised: However, for most values of z and O , the probability P ( M | O, z ) is close to zero, causing poor scaling of sampled estimates.

Original: Similar to a variational autoencoder, our framework solves this problem by estimating the parameters of P ( z | M ) by processing the measurements M using a function with trainable parameters (called the “encoder,” usually a deep neural network).
Revised: Similar to a variational autoencoder, our framework solves this problem by estimating the parameters of P ( z | M ) by processing the measurements M using a function with trainable parameters (called the “encoder,” usually a deep neural network).

Original: The estimate of P ( z | M ) is denoted Q ( z | M ) .
Revised: The estimate of P ( z | M ) is denoted Q ( z | M ) .

Original: The Kullback–Leibler divergence between the distributions is given by D [ Q ( z | M )
Revised: The Kullback–Leibler divergence between the distributions is given by D [ Q ( z | M )

Original: ||
Revised: ||

Original: P ( z | M )] =
Revised: P ( z | M )] =

Original: E z ∼ Q [log Q ( z | M ) − log P ( z | M )] .
Revised: E z ∼ Q [log Q ( z | M ) − log P ( z | M )] .

Original: We also have, by Bayes’ Theorem, log P ( z | M )
Revised: We also have, by Bayes’ Theorem, log P ( z | M )

Original: =
Revised: =

Original: log P
Revised: log P

Original: ( M | z )+ log P ( z ) − log P ( M ) .
Revised: ( M | z )+ log P ( z ) − log P ( M ) .

Original: Combining the expressions yields:
Revised: Combining the expressions yields:

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst term on the right side of this expression can be estimated with sampled values.
Revised: The ﬁrst term on the right side of this expression can be estimated with sampled values.

Original: As
Revised: As

Original: Kullback–Leibler divergence is always ≥ 0 and reaches 0 when Q ( z | M ) =
Revised: Kullback–Leibler divergence is always ≥ 0 and reaches 0 when Q ( z | M ) =

Original: P ( z | M ) , maximizing the right side (deﬁned here as the loss) during training causes P ( M ) to be maximized while forcing
Revised: P ( z | M ) , maximizing the right side (deﬁned here as the loss) during training causes P ( M ) to be maximized while forcing

Original: Q ( z | M ) towards P ( z | M ) .
Revised: Q ( z | M ) towards P ( z | M ) .

Original: In contrast to a conventional variational autoencoder, we do not attempt to use this formulation to synthesize arbitrary objects O by sampling P ( z ) directly.
Revised: In contrast to a conventional variational autoencoder, we do not attempt to use this formulation to synthesize arbitrary objects O by sampling P ( z ) directly.

Original: This framework only attempts reconstruction on the training examples themselves.
Revised: This framework only attempts reconstruction on the training examples themselves.

Original: After training, P ( O | M ) for every object can be sampled by ﬁrst sampling Q ( z | M ) then P ( O | z ) , see Fig. 2.
Revised: After training, P ( O | M ) for every object can be sampled by ﬁrst sampling Q ( z | M ) then P ( O | z ) , see Fig. 2.

Original: Crucially, unlike most data-driven approaches to reconstruction in computational imaging, this framework assumes that no ground truth dataset of objects O is available.
Revised: Crucially, unlike most data-driven approaches to reconstruction in computational imaging, this framework assumes that no ground truth dataset of objects O is available.

Original:  Light-Emitting Diode (LED) Array Microscopy
Revised:  Light-Emitting Diode (LED) Array Microscopy

Original: We prototype the physics-informed variational autoencoder with LED array microscopy as the example system.
Revised: We prototype the physics-informed variational autoencoder with LED array microscopy as the example system.

Original: An LED array microscope is a device created by replacing the illumination source of a standard wide-ﬁeld microscope with a 2-dimensional LED array [5], see Fig.
Revised: An LED array microscope is a device created by replacing the illumination source of a standard wide-ﬁeld microscope with a 2-dimensional LED array [5], see Fig.

Original: This device has great potential for enabling scientiﬁc advances in biology due to its ability to quantitatively measure unstained samples with a simple hardware platform [94].
Revised: This device has great potential for enabling scientiﬁc advances in biology due to its ability to quantitatively measure unstained samples with a simple hardware platform [94].

Original: Every LED of the microscope’s array is individually addressable with tunable brightness, allowing different patterns to be illuminated.
Revised: Every LED of the microscope’s array is individually addressable with tunable brightness, allowing different patterns to be illuminated.

Original: For every object (e.g. specimen or ﬁeld of view) imaged, a set of n illumination patterns with parameters p
Revised: For every object (e.g. specimen or ﬁeld of view) imaged, a set of n illumination patterns with parameters p

Original: =
Revised: =

Original: [ p 1 , p 2 , ..., p n ] are chosen, resulting in an image stack
Revised: [ p 1 , p 2 , ..., p n ] are chosen, resulting in an image stack

Original: M =
Revised: M =

Original: [ M 1 , M 2 , ..., M n ] .
Revised: [ M 1 , M 2 , ..., M n ] .

Original: The resulting measurements M are conventionally post-processed with an iterative optimization algorithm to yield a 2- or 3dimensional amplitude and phase reconstruction of the object O .
Revised: The resulting measurements M are conventionally post-processed with an iterative optimization algorithm to yield a 2- or 3dimensional amplitude and phase reconstruction of the object O .

Original: We aim to increase the temporal resolution (i.e. reduce acquisition time) by reducing the number of images n needed per object.
Revised: We aim to increase the temporal resolution (i.e. reduce acquisition time) by reducing the number of images n needed per object.

Original: Results from LED array microscopy can be translated to other computational imaging modalities such as computed tomography by changing the forward physics model P ( M | O ; p ) in Fig.
Revised: Results from LED array microscopy can be translated to other computational imaging modalities such as computed tomography by changing the forward physics model P ( M | O ; p ) in Fig.

Original: 4.1 Neural Network Architecture
Revised: 4.1 Neural Network Architecture

Original: The overall deep neural network design for the physics-informed variational autoencoder is shown in Fig.
Revised: The overall deep neural network design for the physics-informed variational autoencoder is shown in Fig.

Original: A permutation-invariant design is created so that the reconstruction is agnostic to the order of the n input measurements.
Revised: A permutation-invariant design is created so that the reconstruction is agnostic to the order of the n input measurements.

Original: The network takes the same basic shape as a U-Net, a network design that has shown success in image processing tasks such as segmentation [95] and superresolution [96].
Revised: The network takes the same basic shape as a U-Net, a network design that has shown success in image processing tasks such as segmentation [95] and superresolution [96].

Original: Each measurement parameter p i and corresponding measurement M i (in the case of LED array microscopy, the illumination pattern used and the corresponding intensity image collected, respectively) are concatenated and used as input to this architecture.
Revised: Each measurement parameter p i and corresponding measurement M i (in the case of LED array microscopy, the illumination pattern used and the corresponding intensity image collected, respectively) are concatenated and used as input to this architecture.

Original: The skip connections are parametrized as Gaussian distributions with mean and variance determined through a weighted average from all inputs i = 1 , 2 , ..., n .
Revised: The skip connections are parametrized as Gaussian distributions with mean and variance determined through a weighted average from all inputs i = 1 , 2 , ..., n .

Original: 4.2 Datasets 4.2.1 Synthetic Foam
Revised: 4.2 Datasets 4.2.1 Synthetic Foam

Original: A synthetic dataset is created from foam images generated through the Python package XD ESIGN [97], and simulated as 2D complex objects, see Fig.
Revised: A synthetic dataset is created from foam images generated through the Python package XD ESIGN [97], and simulated as 2D complex objects, see Fig.

Original: The measurement acquisition procedure is simulated using the known forward physics of LED array microscopy and statistics of Poisson noise, as in [66].
Revised: The measurement acquisition procedure is simulated using the known forward physics of LED array microscopy and statistics of Poisson noise, as in [66].

Original: Each object of the dataset is assumed to be measured using a single random illumination pattern on a 2D array of 29 LEDs.
Revised: Each object of the dataset is assumed to be measured using a single random illumination pattern on a 2D array of 29 LEDs.

Original: We emulate measurements with different illumination patterns for each object (middle two rows) and the same illumination pattern for every object (bottom two rows).
Revised: We emulate measurements with different illumination patterns for each object (middle two rows) and the same illumination pattern for every object (bottom two rows).

Original: We model the distribution of possible multiplexed illumination patterns
Revised: We model the distribution of possible multiplexed illumination patterns

Original: P ( p ) with a multivariate Dirichlet distribution.
Revised: P ( p ) with a multivariate Dirichlet distribution.

Original: We sample this distribution n times for every object, where n = 1 for this synthetic dataset.
Revised: We sample this distribution n times for every object, where n = 1 for this synthetic dataset.

Original: The
Revised: The

Original: Dirichlet distribution is deﬁned over x 1 , . . .
Revised: Dirichlet distribution is deﬁned over x 1 , . . .

Original: , x l where (cid:80) x i = 1 .
Revised: , x l where (cid:80) x i = 1 .

Original: Here, l is the number of LEDs, with each
Revised: Here, l is the number of LEDs, with each

Original: LED i having brightness x i .
Revised: LED i having brightness x i .

Original: The constraint of (cid:80) x i = 1 prevents detector saturation if the exposure is set judiciously.
Revised: The constraint of (cid:80) x i = 1 prevents detector saturation if the exposure is set judiciously.

Original: The Dirichlet distribution is parameterized by α 1 , ..., α l , with α i > 0 .
Revised: The Dirichlet distribution is parameterized by α 1 , ..., α l , with α i > 0 .

Original: If all α i = 1 , then all patterns are equally likely.
Revised: If all α i = 1 , then all patterns are equally likely.

Original: In this synthetic dataset, all α i = 0 . 1 , favoring sparser patterns.
Revised: In this synthetic dataset, all α i = 0 . 1 , favoring sparser patterns.

Original: To probe the impact of measurement diversity, we create a corresponding set of measurements where a single illumination pattern is sampled from the Dirichlet distribution and used to measure every object (i.e. P ( p ) is deterministic), see Fig.
Revised: To probe the impact of measurement diversity, we create a corresponding set of measurements where a single illumination pattern is sampled from the Dirichlet distribution and used to measure every object (i.e. P ( p ) is deterministic), see Fig.

Original: In standard LED array microscopy, the LEDs are illuminated sequentially, one at a time.
Revised: In standard LED array microscopy, the LEDs are illuminated sequentially, one at a time.

Original: This entire stack of images is post-processed with an iterative algorithm for the ﬁnal reconstruction.
Revised: This entire stack of images is post-processed with an iterative algorithm for the ﬁnal reconstruction.

Original: This image stack is visualized in
Revised: This image stack is visualized in

Original: Fig.
Revised: Fig.

Original: 4.2.2 Synthetic 3-Dimensional MNIST
Revised: 4.2.2 Synthetic 3-Dimensional MNIST

Original: A second synthetic dataset is created utilizing MNIST digits [98].
Revised: A second synthetic dataset is created utilizing MNIST digits [98].

Original: Each example consists of two
Revised: Each example consists of two

Original: MNIST digits in axial planes 10 µ m apart.
Revised: MNIST digits in axial planes 10 µ m apart.

Original: Each of the two MNIST digits is approximated as a thin phase object and the resulting intensity image is modeled using the assumptions of [9].
Revised: Each of the two MNIST digits is approximated as a thin phase object and the resulting intensity image is modeled using the assumptions of [9].

Original: An example object is shown in Fig.
Revised: An example object is shown in Fig.

Original: 4.2.3 Experimental Frog Blood Smears
Revised: 4.2.3 Experimental Frog Blood Smears

Original: We collect an experimental dataset using an LED array microscope from slides of ﬁxed frog blood smears (Eisco Labs), see Fig. 8.
Revised: We collect an experimental dataset using an LED array microscope from slides of ﬁxed frog blood smears (Eisco Labs), see Fig. 8.

Original: A programmable LED array of 257 LEDs in a concentric circular pattern, 6.5 mm average pitch (Spectral Coded Illumination), is mounted on an inverted microscope (Nikon Eclipse TE300).
Revised: A programmable LED array of 257 LEDs in a concentric circular pattern, 6.5 mm average pitch (Spectral Coded Illumination), is mounted on an inverted microscope (Nikon Eclipse TE300).

Original: In this work, we utilize only the 85 centermost LEDs of the array.
Revised: In this work, we utilize only the 85 centermost LEDs of the array.

Original: The distance z from the LED array to the sample plane is 115 mm and the wavelength of light from the LED array is approximately 525 nm.
Revised: The distance z from the LED array to the sample plane is 115 mm and the wavelength of light from the LED array is approximately 525 nm.

Original: The microscope objective is 40 × magniﬁcation with a numerical aperture of 0.75 (Nikon CFI Plan Fluor).
Revised: The microscope objective is 40 × magniﬁcation with a numerical aperture of 0.75 (Nikon CFI Plan Fluor).

Original: The image sensor captures 16-bit images with
Revised: The image sensor captures 16-bit images with

Original: 2048 × 2048 pixels and a pixel size of 6.5 µ m (pco.edge 4.2 LT).
Revised: 2048 × 2048 pixels and a pixel size of 6.5 µ m (pco.edge 4.2 LT).

Original: The dataset collected consists of 87 unique ﬁelds-of-view (i.e. objects).
Revised: The dataset collected consists of 87 unique ﬁelds-of-view (i.e. objects).

Original: Each ﬁeld-of-view was illuminated with a single randomly chosen LED pattern as follows: a 2.5 mm radius circle was randomly placed on the LED array and 12 of the LEDs within that circle were randomly chosen to be illuminated.
Revised: Each ﬁeld-of-view was illuminated with a single randomly chosen LED pattern as follows: a 2.5 mm radius circle was randomly placed on the LED array and 12 of the LEDs within that circle were randomly chosen to be illuminated.

Original: For reference, a dataset with sequential, single LED illumination was collected on a ﬁeld-of-view, see Fig.
Revised: For reference, a dataset with sequential, single LED illumination was collected on a ﬁeld-of-view, see Fig.

Original: This dataset is available for download at https://figshare.com/s/635499acfdcdf0893750 .
Revised: This dataset is available for download at https://doi.org/10.6084/m9.figshare

Original: 4.3 Results & Discussion
Revised: 4.3 Results & Discussion

Original: Code and documentation to reproduce our results are available at https://anonymous.4open. science/r/LED_PVAE-1290 .
Revised: Code and documentation to reproduce our results are available at https://github.com/ vganapati/LED_PVAE .

Original: We ﬁrst train the physics-informed variational autoencoder (P-VAE) on the synthetic foam dataset, varying training dataset size.
Revised: We ﬁrst train the physics-informed variational autoencoder (P-VAE) on the synthetic foam dataset, varying training dataset size.

Original: We compare a reconstruction sampled from the P-VAE, using one collected image (i.e. n = 1 ), with conventional  iterative reconstruction using the same image.
Revised: We compare a reconstruction sampled from the P-VAE, using one collected image (i.e. n = 1 ), with conventional gradient-based iterative reconstruction using the same image (see https://github.com/vganapati/LED_PVAE for more details on implementation).

Original: We also evaluate iterative reconstruction with a stack of 29 single LED images (i.e. n = 29 ).
Revised: We also evaluate iterative reconstruction with a stack of 29 single LED images (i.e. n = 29 ).

Original: Table 1 compares the peak signal-to-noise ratio for the resulting reconstructions, averaged over 10 object examples, for both the cases of a Dirichlet P ( p ) (different illumination patterns for every example) and a deterministic P ( p ) (same illumination pattern for every example).
Revised: Table 1 compares the peak signal-to-noise ratio for the resulting reconstructions, averaged over 10 object examples, for both the cases of a Dirichlet P ( p ) (different illumination patterns for every example) and a deterministic P ( p ) (same illumination pattern for every example).

Original: Fig.
Revised: Fig.

Original: 10 visualizes the comparison of peak signal-to-noise ratio (PSNR) among the methods for a single object.
Revised: 10 visualizes the comparison of peak signal-to-noise ratio (PSNR) among the methods for a single object.

Original: The results for standard iterative methods do not vary with dataset size.
Revised: The results for standard iterative methods do not vary with dataset size.

Original: We note that the P-VAE with a Dirichlet P ( p ) outperforms the standard iterative method with n = 29 , starting with a dataset size of n = 1 , 000 , and outperforms the standard iterative method with n = 1 , starting with a dataset size of n = 100 .
Revised: We note that the P-VAE with a Dirichlet P ( p ) outperforms the standard iterative method with n = 29 , starting with a dataset size of n = 1 , 000 , and outperforms the standard iterative method with n = 1 , starting with a dataset size of n = 100 .

Original: A deterministic P ( p ) does not perform as well, aligning with the intuition that diversity of measurements allows knowledge of the prior to be inferred.P-VAE , n =m
Revised: A deterministic P ( p ) does not perform as well, aligning with the intuition that diversity of measurements allows knowledge of the prior to be inferred.

Original: Dirichlet
Revised: [Table]

Original: Deterministic -4. Standard Iterative n =n =10.
Revised: [Table] object examples.

Original: The results from the -VAEs shown are 
Revised: The results from the P-VAEs shown are with m = 1 , 000 .

Original: A constant p over the dataset (deterministic P ( p ) ) results in reconstruction artifacts.
Revised: A constant p over the dataset (deterministic P ( p ) ) results in reconstruction artifacts.

Original: We see that if the dataset size is large enough, and P ( p ) is stochastic, learning from a dataset of limited measurements on different objects can lead to improvements of object reconstruction over conventional iterative methods.
Revised: We see that if the dataset size is large enough, and P ( p ) is stochastic, learning from a dataset of limited measurements on different objects can lead to improvements of object reconstruction over conventional iterative methods.

Original: We emphasize the signiﬁcance of our method in improving over standard iterative methods using less than 4% of the data.
Revised: We emphasize the signiﬁcance of our method in improving over standard iterative methods using less than 4% of the data.

Original: We note that our procedure is similar to conventional object reconstruction through iterative optimization.
Revised: We note that our procedure is similar to conventional object reconstruction through iterative optimization.

Original: The crucial difference is the joint reconstruction of multiple similar objects, with the pooling of knowledge between objects to ﬁnd the prior distribution.
Revised: The crucial difference is the joint reconstruction of multiple similar objects, with the pooling of knowledge between objects to ﬁnd the prior distribution.

Original: We observe a slight decrease in performance of the P-VAE from a dataset size of 1 , 000 to a larger dataset of 10 , 000 .
Revised: We observe a slight decrease in performance of the P-VAE from a dataset size of 1 , 000 to a larger dataset of 10 , 000 .

Original: The expectation is that a larger dataset would contain more knowledge of the prior distribution.
Revised: The expectation is that a larger dataset would contain more knowledge of the prior distribution.

Original: However, we hypothesize that the limited capacity of the neural network may not allow for joint reconstruction on all the examples of a larger dataset.
Revised: However, we hypothesize that the limited capacity of the neural network may not allow for joint reconstruction on all the examples of a larger dataset.

Original: Future work aims to use hyperparameter optimization [99] to investigate this hypothesis.
Revised: Future work aims to use hyperparameter optimization [99] to investigate this hypothesis.

Original: [Table]
Revised: [Table]

Original:  Results on the foam dataset demonstrate single-shot ( n = 1 ) imaging for a small array of 29 LEDs under the assumption of a thin, approximately 2D specimen.
Revised: Standard Iterative , n = [Table] Results on the foam dataset demonstrate single-shot ( n = 1 ) imaging for a small array of 29 LEDs under the assumption of a thin, approximately 2D specimen.

Original: In the case of 3D imaging, more measurements may be needed.
Revised: In the case of 3D imaging, more measurements may be needed.

Original: We train the
Revised: We train the

Original: P-VAE on the synthetic 3D MNIST dataset, with 1 ≤ n ≤ 4 , and show improvement for increasing number of measurements, see Table 2 and Fig.
Revised: P-VAE on the synthetic 3D MNIST dataset, with 1 ≤ n ≤ 4 , and show improvement for increasing number of measurements, see Table 2 and Fig.

Original: We demonstrate improved reconstruction with the P-VAE over the standard iterative method using the same number of measurements n , though the the performance of the P-VAE doesn’t exceed that of the standard method with n = 29 .
Revised: We demonstrate improved reconstruction with the P-VAE over the standard iterative method using the same number of measurements n , though the the performance of the P-VAE doesn’t exceed that of the standard method with n = 29 .

Original: The P-VAE was tested on the experimental dataset and again compared against standard iterative methods.
Revised: The P-VAE was tested on the experimental dataset and again compared against standard iterative methods.

Original: In the case of experimental data, there is no ground truth, so we utilize as reference the reconstruction obtained with a standard iterative method on a stack of 85 single LED images.
Revised: In the case of experimental data, there is no ground truth, so we utilize as reference the reconstruction obtained with a standard iterative method on a stack of 85 single LED images.

Original: We note that this is an imperfect metric, as we have seen the P-VAE outperform this method in certain cases of synthetic data.
Revised: We note that this is an imperfect metric, as we have seen the P-VAE outperform this method in certain cases of synthetic data.

Original: The experimental data reconstructions were conducted on an NVIDIA A100 GPU, and took 21 hours for reconstruction using a standard iterative method with n = 85 , 18 hours with standard iterative methods and n = 1 , and 47 hours for training the P-VAE.
Revised: The experimental data reconstructions were conducted on an NVIDIA A100 GPU, and took 21 hours for reconstruction using a standard iterative method with n = 85 , 18 hours with standard iterative methods and n = 1 , and 47 hours for training the P-VAE.

Original: The standard iterative methods result in one object reconstruction, but the P-VAE results in 87 object reconstructions.
Revised: The standard iterative methods result in one object reconstruction, but the P-VAE results in 87 object reconstructions.

Original: The result from the P-VAE with n = 1 outperforms the result obtained from the standard iterative method with n = 1 , when comparing to the reference, see Fig.
Revised: The result from the P-VAE with n = 1 outperforms the result obtained from the standard iterative method with n = 1 , when comparing to the reference, see Fig.

Original:  Open Questions
Revised:  Open Questions

Original: Though initial results are promising, more progress must be made to apply this imaging technique for scientiﬁc discovery.
Revised: Though initial results are promising, more progress must be made to apply this imaging technique for scientiﬁc discovery.

Original: 5.1 General Requirements
Revised: 5.1 General Requirements

Original: In the case of real, experimental data, there is no ground truth to determine reconstruction accuracy (deﬁned as average ﬁdelity of samples of P ( O | M ) to the true object by a metric such as meansquared error, structural similarity, or peak signal-to-noise ratio).
Revised: In the case of real, experimental data, there is no ground truth to determine reconstruction accuracy (deﬁned as average ﬁdelity of samples of P ( O | M ) to the true object by a metric such as meansquared error, structural similarity, or peak signal-to-noise ratio).

Original: Correlation of these quantities to the training loss must be analyzed with synthetic data where the ground truth is known.
Revised: Correlation of these quantities to the training loss must be analyzed with synthetic data where the ground truth is known.

Original: Results must be shown to be robust, characterized by the worst-case loss of multiple training runs with the same hyperparameters.
Revised: Results must be shown to be robust, characterized by the worst-case loss of multiple training runs with the same hyperparameters.

Original: Results also should be stable, meaning small variance in loss among multiple training runs.
Revised: Results also should be stable, meaning small variance in loss among multiple training runs.

Original: Theoretical guarantees on the correctness of the calculated prior distribution need to be developed, given the form of the prior, the number of objects in the dataset, number of measurements per object, the distribution of parameters P ( p ) , and the forward model P ( M | O ) .
Revised: Theoretical guarantees on the correctness of the calculated prior distribution need to be developed, given the form of the prior, the number of objects in the dataset, number of measurements per object, the distribution of parameters P ( p ) , and the forward model P ( M | O ) .

Original: Additionally, in the case of experimental data, the problem of model mismatch arises, where the speciﬁed likelihood
Revised: Additionally, in the case of experimental data, the problem of model mismatch arises, where the speciﬁed likelihood

Original: P ( M | O ) may have inaccuracies.
Revised: P ( M | O ) may have inaccuracies.

Original: The consequences of model mismatch need to be investigated in simulation with synthetic data.
Revised: The consequences of model mismatch need to be investigated in simulation with synthetic data.

Original: 5.2 Choice of Measurement Parameters
Revised: 5.2 Choice of Measurement Parameters

Original: Each object of the dataset is measured with some parameters p .
Revised: Each object of the dataset is measured with some parameters p .

Original: We aim for posterior distributions P ( O | M ) with low entropy, for higher conﬁdence in sampled reconstructions.
Revised: We aim for posterior distributions P ( O | M ) with low entropy, for higher conﬁdence in sampled reconstructions.

Original: The dependence of the posterior distribution entropy on the choice of P ( p ) needs to be investigated.
Revised: The dependence of the posterior distribution entropy on the choice of P ( p ) needs to be investigated.

Original: It is likely that the optimal choice of P ( p ) depends on the prior distribution P ( O ) , which is unknown before measurement commences.
Revised: It is likely that the optimal choice of P ( p ) depends on the prior distribution P ( O ) , which is unknown before measurement commences.

Original: The beneﬁt of adaptively picking p , i.e. choosing p for an object based on data collected from previous objects, can be evaluated in scenarios with ground truth or reference data.
Revised: The beneﬁt of adaptively picking p , i.e. choosing p for an object based on data collected from previous objects, can be evaluated in scenarios with ground truth or reference data.

Original: 5.3 High Memory Management
Revised: 5.3 High Memory Management

Original: This work shows results from synthetic experiments using small-scale data.
Revised: This work shows results from synthetic experiments using small-scale data.

Original: We demonstrate single-shot ( n = 1 ) and few-shot ( n ≤ 4 ) imaging for an array of 29 LEDs with measurements of 128 × 128 pixels or less.
Revised: We demonstrate single-shot ( n = 1 ) and few-shot ( n ≤ 4 ) imaging for an array of 29 LEDs with measurements of 128 × 128 pixels or less.

Original: Results are also shown from real experimental data with singleshot imaging on an array of 85 LEDs with image size of 2048 × 2048 pixels.
Revised: Results are also shown from real experimental data with singleshot imaging on an array of 85 LEDs with image size of 2048 × 2048 pixels.

Original: In scientiﬁc studies, the goal may be to achieve object reconstruction with larger arrays of around 800 LEDs, as more
Revised: In scientiﬁc studies, the goal may be to achieve object reconstruction with larger arrays of around 800 LEDs, as more

Original: LEDs in the illumination pattern generally mean higher spatial resolution [5].
Revised: LEDs in the illumination pattern generally mean higher spatial resolution [5].

Original: For these largescale, real-world problems, the number of measurements n required with this framework may be (cid:29) 1 .
Revised: For these largescale, real-world problems, the number of measurements n required with this framework may be (cid:29) 1 .

Original: For other computational imaging modalities, such as computed tomography, thousands of measurements may be collected per object [100].
Revised: For other computational imaging modalities, such as computed tomography, thousands of measurements may be collected per object [100].

Original: A single set of measurements on an object in x-ray holographic nano-tomography can be on the order of 100s of gigabytes [101].
Revised: A single set of measurements on an object in x-ray holographic nano-tomography can be on the order of 100s of gigabytes [101].

Original: Solving jointly for the posteriors of an object set will require techniques for high memory management.
Revised: Solving jointly for the posteriors of an object set will require techniques for high memory management.

Original: 5.4 Video Reconstruction
Revised: 5.4 Video Reconstruction

Original: Fast acquisition times are critical to imaging dynamic objects, to avoid motion blur and instead capture movement and changes.
Revised: Fast acquisition times are critical to imaging dynamic objects, to avoid motion blur and instead capture movement and changes.

Original: Like other computational imaging systems, the LED array microscope allows for increased functionality at the cost of temporal resolution.
Revised: Like other computational imaging systems, the LED array microscope allows for increased functionality at the cost of temporal resolution.

Original: For many biological applications, imaging live, changing specimens is of key importance to scientiﬁc discovery.
Revised: For many biological applications, imaging live, changing specimens is of key importance to scientiﬁc discovery.

Original: The outlined framework can be utilized for video reconstruction by treating each frame as a separate reconstruction problem.
Revised: The outlined framework can be utilized for video reconstruction by treating each frame as a separate reconstruction problem.

Original: In particular, snapshots of the same object at different timestamps t = 1 , 2 , ..., m can be thought of as the objects of the training dataset { O 1 , O 2 , ..., O m } .
Revised: In particular, snapshots of the same object at different timestamps t = 1 , 2 , ..., m can be thought of as the objects of the training dataset { O 1 , O 2 , ..., O m } .

Original: However, this assumes that snapshots are independent and does not take into account rich temporal dependencies among frames that can lead to improved reconstruction and elimination of “jumpy” artifacts [102–104].
Revised: However, this assumes that snapshots are independent and does not take into account rich temporal dependencies among frames that can lead to improved reconstruction and elimination of “jumpy” artifacts [102–104].

Original: Approaches with recurrent neural network architectures may allow for improved video reconstruction.
Revised: Approaches with recurrent neural network architectures may allow for improved video reconstruction.

Original:  Conclusion
Revised:  Conclusion

Original: We present a novel self-supervised framework, based on the mathematics of variational autoencoders, for reconstruction in computational imaging.
Revised: We present a novel self-supervised framework, based on the mathematics of variational autoencoders, for reconstruction in computational imaging.

Original: The framework is prototyped for improving the temporal resolution of LED array microscopy, and validated with synthetic and experimental data.
Revised: The framework is prototyped for improving the temporal resolution of LED array microscopy, and validated with synthetic and experimental data.

Original: As the framework is self-supervised and does not require ground truth or reference reconstructions for training, it is amenable for use in applications of scientiﬁc discovery, where both temporal and spatial resolution are needed.
Revised: As the framework is self-supervised and does not require ground truth or reference reconstructions for training, it is amenable for use in applications of scientiﬁc discovery, where both temporal and spatial resolution are needed.

Original: We outline future directions of research, releasing our code at https://anonymous.4open.science/r/LED_PVAE-1290 and our experimental data at https: //figshare.com/s/635499acfdcdf0893750 for further development. 
Revised: We outline future directions of research, releasing our code at https://github.com/vganapati/LED_PVAE and our experimental data at https://doi.org/ 10.6084/m9.figshare.21232088.v1 for further development. Acknowledgments and Disclosure of Funding

Revised: This work was supported in part by the U.S. Department of Energy, Ofﬁce of Science, Ofﬁce of

Revised: Workforce Development for Teachers and Scientists (WDTS) under the Visiting Faculty Program (VFP) and the AAUW Research Publication Grant in Engineering, Medicine and Science.

Revised: The authors thank Srinivas Turaga and Jan Funke at Janelia Research Campus and Vincent Dumont at

Revised: Lawrence Berkeley National Laboratory for participating in helpful discussions.