Unsupervised Learning From Incomplete Measurements for Inverse Problems
Anonymous Author(s)
AffiliationAddress email
Abstract
In many real-world inverse problems, only incomplete measurement data are available for training which can pose a problem for learning a reconstruction function.
Indeed, unsupervised learning using a fixed incomplete measurement process is impossible in general, as there is no information in the nullspace of the measurementoperator.
This limitation can be overcome by using measurements from multipleoperators.
While this idea has been successfully applied in various applications, aprecise characterization of the conditions for learning is still lacking.
In this paper,we fill this gap by presenting necessary and sufficient conditions for learning theunderlying signal model needed for reconstruction which indicate the interplaybetween the number of distinct measurement operators, the number of measurements per operator, the dimension of the model and the dimension of the signals.
Furthermore, we propose a novel and conceptually simple unsupervised learningloss which only requires access to incomplete measurement data and achievesa performance on par with supervised learning when the sufficient condition isverified.
We validate our theoretical bounds and demonstrate the advantages ofthe proposed unsupervised loss compared to previous methods via a series ofexperiments on various imaging inverse problems, such as accelerated magneticresonance imaging, compressed sensing and image inpainting.
Introduction
In multiple sensing applications, we observe measurements y ∈ R m associated with a signalx ∈ X ⊂ R n , through the forward process
[Equation]
where A ∈ R n × n is a linear measurement operator and ϵ denotes the noise affecting the measurements.
This is the case of computed tomography [1], depth ranging [2] and non-line-of-sight imaging [3]to name a few.
Estimating x from y is generally an ill-posed inverse problem due to the incompleteoperator
A with m < n and the presence of noise.
Knowledge of the signal model is required tomake this problem well-posed.
In many cases, obtaining ground-truth reconstructions x to learn the reconstruction function y
(cid:55)→ xmight be very expensive or even impossible.
For example, in medical imaging, it is not alwayspossible to obtain fully sampled images of patients as they require long acquisition times.
Inastronomical imaging, it is impossible to obtain ground-truth references due to physical limitations.
In electron-microscopy imaging [4], we can only measure 2D projections of a molecule.
In thesesettings, we can only access measurements y for learning.
Moreover, if the measurement process Ais incomplete, it is fundamentally impossible to learn the model with only measurements y , as thereis no information about the model in the nullspace of A .
Thus, we end up with a chicken-and-egg problem: in order to reconstruct x we need the reconstruction function, but to learn this function werequire some reconstructed samples x .
This fundamental limitation can be overcome by using information from multiple incomplete sensingoperators
A 1 , . .
A G , the general principle being that each operator can provide additional information about the signal model if it has a different nullspace.
For example, in the image inpaintingproblem, Studer and Baraniuk [5] used the fact that the set of missing pixels may vary betweenobserved images to learn a sparse dictionary model and reconstruct the images.
Yang et al.
[6]used multiple operators to learn a Gaussian mixture model in the context of hyperspectral imagingand high-speed video.
Bora et al.
[7] exploited this idea for learning a generative model in variousimaging problems such as deblurring and compressed sensing.
Matrix completion methods [8] exploita similar principle, as the missing entries of each column (i.e., signal) are generally different.
Ideallywe would like to learn the reconstruction function and signal model from only a small number ofdifferent measurement operators.
We are thus motivated to determine typically how many suchoperators are required.
The problem can be formalized as follows.
We first focus on the noiseless case to study the intrinsicidentifiability problems associated to having only incomplete measurement data.
The effect of noisewill be discussed in Section 4.
We assume that we observe a set of N training samples y i , where thei th signal is observed via A g i ∈
R m × n , one of G linear operators , i.e.,y
[Equation]
(2) where g i ∈ { 1 , . .
, G } and i = 1 , . . .
, N .
While we assume that the measurement operator A g i isknown for all observed signals, it is important to note that we do not know a priori if two observations( y i , A g i ) and ( y i ′ ,
A g i ′ ) are related to the same signal 1 x i .
There are two natural questions regardingthis learning problem:
Q1.
Signal Recovery Is there a unique signal x ∈ X
which verifies the measurements y
=
A g x ?
Inother words, is the reconstruction function f : ( y, A g )
(cid:55)→ x one-to-one?
Q2.
Model Identification Can we uniquely identify the signal model from only measurementsobtained these via incomplete operators?
In general, there can be a unique solution for neither problem, just one or both.
There might be aunique solution for signal recovery if the model is known, but it might be impossible to learn themodel in the first place (e.g., blind compressed sensing [9]).
The converse is also possible, that is,uniquely identifying a model without having enough measurements per sample to uniquely identifythe associated signal (e.g., subspace learning from rank-1 projections [10]).
The answer to Q1 is well-known from generalized compressed sensing theory, see for example [11].
Unique signal recovery is possible if the signal model is low-dimensional, i.e., if A g has m > 2 kmeasurements, where k is the model dimension.
On the other hand, Q2 has been mostly studied inthe context of matrix completion, where the set of signals is assumed to lie in a low-dimensionalsubspace of R n .
Bora et al.
[7] presented some results in the general setting, but only for the casewhere G = ∞ which is quite restrictive.
In this paper, we provide sharp necessary and sufficientconditions which hold for any low-dimensional distribution (beyond linear subspaces) and onlyrequire a finite number of operators G .
If the conditions for signal recovery and model identification are satisfied, we can expect to learn thereconstruction function from measurement data alone.
We introduce a new unsupervised learningobjective which can be used to learn the reconstruction function f : ( y, A g )
(cid:55)→ x , and providesperformances on par with supervised learning when the sufficient conditions are met.
The maincontributions of this paper are as follows:
• We show that unsupervised learning from a finite number of incomplete measurementoperators is only possible if the model is low-dimensional.
More precisely, we show thatm ≥ n/G measurements per operator are necessary for learning, and that for almost everyset of G operators, m > k + n/G measurements per operator are sufficient.
• We propose a new unsupervised loss for learning the reconstruction function that only
requires incomplete measurement data, which empirically obtains a performance on par
with fully supervised methods when the sufficient condition m > k + n/G is met.
• A series of experiments demonstrate that our bounds accurately characterize the perfor-
mance of unsupervised approaches on synthetic and real datasets, and that the proposed
unsupervised approach outperforms previous methods in various inverse problems
1.1 Related Work
Blind Compressed Sensing
The fundamental limitation of failing to learn a signal model fromincomplete (compressed) measurements data goes back to blind compressed sensing [9] for thespecific case of models exploiting sparsity on an orthogonal dictionary.
In order to learn the dictionaryfrom incomplete data, [9] imposed additional constraints on the dictionary, while some subsequentpapers [12, 13] removed these assumptions by proposing to use multiple operators A g as studied here.
This paper can be seen as a generalization of such results to more general signal models.
Matrix Completion Matrix completion consists of inferring missing entries of a data matrix Y =[ y 1 , . . .
, y N ] , whose columns are generally inpainted samples from a low-dimensional distribution,i.e., y i
=
A g i x i where the operators A g i randomly select a subset of m entries of the signal x i .
Thisproblem can be viewed as the combination of model identification, i.e., identifying the low-ranksubspace that the columns of X =
[ x 1 , . .
, x N ] belong to, and signal recovery, i.e., reconstructing theindividual columns.
Assuming that the samples belong to a k -dimensional subspace can be imposedby recovering a rank- k signal matrix X from Y .
If the columns are sampled via G sufficientlydifferent patterns A g i with the same number of entries m , a sufficient condition [14] for uniquelyrecovering almost every subspace model is 2 m ≥ (1 − 1 /G )
k + n/G .
A similar necessary condition was shown in [15] for the case of high-rank matrix completion [16],which arises when the samples x i belong to a union of k -dimensional subspaces.
We show that modelidentification is possible for almost every set of G operators with m > k + n/G measurements,however the theory presented here goes beyond linear subspaces, being also valid for general lowdimensional models.
Deep Nets for Inverse Problems Despite providing very competitive results, most deep learningbased solvers require measurements and signal pairs ( x i , y i ) (or at least clean signals x i ) in order tolearn the reconstruction function y
(cid:55)→ x from incomplete measurements.
A first step to overcome thislimitation is due to Noise2Noise [17], where the authors show that it is possible to learn from onlynoisy samples.
However, their ideas only apply to denoising settings where there is a trivial nullspace,as the operator A is the identity matrix.
Yaman et al.
[18] and Artifact2Artifact [19] empiricallyshowed that it is possible to exploit different measurement operators to learn the reconstructionfunction in the context of magnetic resonance imaging (MRI).
AmbientGAN [7] proposed to learn asignal distribution from only incomplete measurements using multiple forward operators, howeverthey only provide reconstruction guarantees for the case where an infinite number of operators A g isavailable 3 , a condition that is not met in practice.
Signal Recovery Preliminaries
We denote the nullspace of A as N A .
Its complement, the range space of the pseudo-inverse A † , isdenoted as R A , where R A ⊕ N A
= R n and ⊕ denotes the direct sum.
Throughout the paper, weassume that the signals are sampled from a measure µ supported on the signal set X ⊂ R n .
Signalrecovery has a unique solution if and only if the forward operator x
(cid:55)→ y is one-to-one, i.e., if forevery pair of signals x 1 , x 2 ∈ X where x 1 ̸ = x 2 we have that [Equation]
[Equation]
In other words, there is no vector x 1 − x 2 ̸ = 0 in the nullspace of A .
It is well-known that this isonly possible if the signal set X is low-dimensional.
There are multiple ways to define the notion ofdimensionality of a set in R n .
In this paper, we focus on the upper box-counting dimension which is m = O ( k log n ) is required to guarantee a stable recovery when the number of patterns G is large [8]. Their result relies on the Cramér-Wold theorem, which is discussed in Section 3. defined for a compact subset S ⊂ R n as
[Equation]
where N ( S, ϵ ) is the number of closed balls of radius ϵ with respect to the norm ∥ · ∥ that are requiredto cover S .
This definition of dimension covers both well-behaved models such as compact manifoldsand more general low-dimensional sets.
The mapping x
(cid:55)→ y is one-to-one for almost every forwardoperator A ∈ R m × n if [20]
[Equation]
) where ∆ X denotes the normalized secant set which is defined as
[Equation]
The term almost every means that the complement has Lebesgue measure 0 in the space of linearmeasurement operators R m × n .
The normalized secant set of models of dimension k generally hasdimension 2 k , requiring m > 2 k measurements to ensure signal recovery.
For example, the union ofk -dimensional subspaces requires at least 2 k measurements 4 to guarantee one-to-oneness [21].
Thisincludes well-known models such as k -sparse models (e.g., convolutional sparse coding [22]) andco-sparse models (e.g., total variation [23]).
In the regime k < m ≤ 2 k , the subset of signals whereone-to-oneness fails is at most (2 k − m ) -dimensional
Uniqueness of Any Model?
A natural first question when considering uniqueness of the model is: can we recover any measure µobserved via forward operators
A 1 , . .
A G , even in the case where X is the full R n ?
We show that,in general, the answer is no.
Uniqueness can be analysed from the point of view of the characteristic function of µ , defined asφ ( w )
=
E { e i w ⊤ x } where the expectation is taken with respect to µ and i =
√− 1 is the imaginaryunit.
If two distributions have the same characteristic function, then they are necessarily the samealmost everywhere.
Each forward operator provides information about a subspace of the characteristicfunction as
[Equation]
where A † g A g is a linear projection onto the subspace R A g .
Given that m < n , the characteristicfunction is only observed in the subspaces R A g for all g ∈ { 1 , . . .
, G } .
For any finite number ofoperators, the union of these subspaces does not cover the whole R n , and hence there is loss ofinformation, i.e., the signal model cannot be uniquely identified.
In the case of an infinite number of operators G = ∞ , the Cramér-Wold theorem guarantees uniqueness of the signal distribution if all possible one dimensional projections ( m = 1 ) are available [24, 7].
However, in most practical settings we can only access a finite number of operators and manydistributions will be non-identifiable.
Uniqueness of Low-Dimensional Models
Most models appearing in signal processing and machine learning are assumed to be approximatelylow-dimensional, with a dimension k which is much lower than the ambient dimension n .
Asdiscussed in Section 2, the low-dimensional property is the key to obtain stable reconstructions, e.g.,in compressed sensing.
In the rest of the paper, we impose the following assumptions on the model:
A
The signal set X is either
(a) A bounded set with box-counting dimension k .
(b) An unbounded conic set whose intersection with the unit sphere has box-counting
dimension k − 1 .
This assumption has been widely adopted in the inverse problems literature, as it is a necessaryassumption to guarantee signal recovery.
Our definition of dimension covers most models used inpractice, such as union of subspaces (simple subspace models, convolutional sparse coding models,k -sparse models), low-rank matrices and compact manifolds.
It is worth noting that dimension is aproperty of the dataset and thus independent of the specific algorithm used for learning.
In the rest of the paper, we focus on conditions for the identification of the support X instead ofthe signal distribution µ , due to the following observation: if there is a one-to-one reconstructionfunction (which happens for almost every A with m > 2 k as explained in Section 2), uniqueness ofthe support implies uniqueness of µ .
If X is known and there is a measurable one-to-one mappingfrom each observed measurement y to X , then it is possible to obtain µ as the push-forward of themeasurement distribution.
Before delving into the main theorem, we present a simple example which provides intuition of howa low-dimensional model can be learned via multiple projections
A g :
Learning a one-dimensional subspace Consider a toy signal model with support X ⊂ R 3 whichconsists of a one-dimensional linear subspace spanned by ϕ =
[1 , 1 , 1] ⊤ , and G = 3 measurementoperators
A 1 , A 2 , A 3 ∈ R 2 × 3 which project the signals into the x (3) = 0 , x (2) =
and x (1) =
0planes respectively, where x ( i ) denotes the i th entry of the vector x .
The example is illustratedin Figure 1.
The first operator A 1 imposes a constraint on X , that is, every x ∈ X should verifyx (1) − x (2)
= 0 .
Without more operators providing additional information about X , this constraintyields a plane containing X , and there are infinitely many one-dimensional models that would fit thetraining data perfectly.
However, the additional operator A 2 adds the constraint x (2) − x (3) = 0 ,which is sufficient to uniquely identify X as
[Equation]
is the desired 1-dimensional subspace.
Finally, note that in this case the operator
A 3 does not restrictthe signal set further, as the constraint x (1) − x (3) = 0 is verified by the other two constraints.
The ideas from the one-dimensional subspace example can be generalized and formalized as follows:for each projection A g , we can constrain the model support X by considering the set
[Equation]
11) which has dimension at most n − ( m − k ) .
Note that the true signal model is a subset of ˆ X g .
Theinferred signal set belongs to the intersection of these sets
[Equation]
[Equation]
which can be expressed concisely as
[Equation]
Even though we have derived the set ˆ X from a purely geometrical argument, the constraints in (13)also offer a simple algebraic intuition: the inferred signal set consists of the points v ∈ R n whichverify the following system of equations
[Equation]
for all possible choices of G points x 1 , . . .
, x G in X .
In other words, given a dataset of N incompletemeasurements { A g i x i } Ni =1 , it is possible to build ˆ X by trying all the possible combinations of Gsamples 5 and keeping only the points v which are the solutions of (14).
It is trivial to see that X ⊆
ˆ X ,
but when can we guarantee X
= ˆ X ?
As in the previous toy example, ifthere are not enough constraints, e.g., if we have a single A and no additional measurement operators,the inferred set will have a dimension larger than k , containing undesired aliases.
In particular, wehave the following lower bound on the minimum number of measurements:
Proposition 4.1 (Theorem 1 in [25]) .
A necessary condition for model uniqueness is that m ≥ n/G .
Proof.
In order to have model uniqueness, the system in (14) should only admit a solution ifv = x 1 = · · · = x G .
If m < n/G there is more than one solution for any choice of x 1 , . . , x G ∈ Xas the matrix on the left hand side of (14) has rank smaller than n .
Note that this necessary condition does not take into account the dimension of the model.
As discussedin Section 3, a sufficient condition for model uniqueness must depend on the dimension of the signalset k .
Our main theorem shows that k additional measurements per operator are sufficient for modelidentification:
Theorem 4.2.
For almost every set of G mappings A 1 , . . .
, A G ∈ R m × n , the signal model X can beuniquely recovered if the number of measurements verify m > k + n/G .
The proof is included in the SM. If we have a large number of independent operators G ≥ n ,
Theorem 4.2 states that only m > k + 1 measurements are sufficient for model identification, whichis slightly smaller (if the model is not trivial, i.e., k > 1 ) than the number of measurements typicallyneeded for signal recovery m > 2 k .
In this case, it is possible to uniquely identify the model,without necessarily having a unique reconstruction of each observed signal.
However, as discussedin Section 2, for k < m ≤ 2 k , the subset of signals which cannot be uniquely recovered is at most(2 k − m ) -dimensional.
Operators with Different Number of Measurements
The results of the previous subsectionscan be easily extended to the setting where each measurement operator has a different number ofmeasurements, i.e. A 1 ∈ R m 1 × n , . . .
, A G ∈ R m G × n .
In this case, the necessary condition in
Proposition 4.1 is (cid:80) G g =1 m g ≥ n , and the sufficient condition in Theorem 4.2 is 1 G (cid:80) {+G g =
+} m g >k + n/G .
As the proofs mirror the ones of Proposition 4.1 and Theorem 4.2, we leave the details tothe reader.
Noisy measurement data Surprisingly, the results of this section are also theoretically valid if themeasurements are corrupted by independent additive noise ϵ , i.e., y =
A g x + ϵ , as long as the noisedistribution is known and has a nowhere zero characteristic function (e.g., Gaussian noise):
Proposition 4.3.
For a fixed noise distribution, if its characteristic function is nowhere zero, thenthere is a one-to-one mapping between the space of clean measurement distributions and noisemeasurement distributions.
The proof is included in the SM.
If the clean measurement distribution can be uniquely identifiedfrom the noisy distribution, the results in Theorem 4.2 and Proposition 4.1 also carry over to thenoisy setting.
Note that this only guarantees model identifiability and makes no claims on the samplecomplexity of any learning process.
Algorithms
Unsupervised algorithms mainly come in two flavours: we can first learn a model ˆ X to then reconstructby projecting measurements into this set, or we can attempt to directly learn the reconstructionfunction parameterized by a deep network.
5.1 Learn the Model and Reconstruct
Dictionary and Subspace Learning
If X is (approximately) a union of subspaces [9] or a singlesubspace [8], we can learn a model by
[Equation]
z [Equation] 15) where ρ 1 ( D ) and ρ 2 ( z ) are regularisation terms that promote low-dimensional solutions, e.g., sparsecodes z if X is a union of subspaces.
At test time, the dictionary is fixed and the optimization isperformed over the codes only.
AmbientGAN Complex datasets are often better modelled by a generative network f : R k
(cid:55)→ R nwhose input is a low-dimensional latent code z ∈ R k .
The generative model can be learned using anadversarial strategy, i.e.,
[Equation]
(16) where d : R n (cid:55)→ R n is the discriminator network which compares measurements in the imagedomain, z is usually sampled from a Gaussian distribution, and q ( t ) = log( t ) for standard GANs andq ( t )
=
t for Wasserstein GANs.
At test time, the reconstruction can be obtained by finding the latentcode that best fits the measurements ˆ x = f (arg min z ∥
y − A g f ( z ) ∥ 2 ) , as in [26].
5.2 Learn to Reconstruct
Another approach consists in learning directly the reconstruction mapping f : R m × R m × n (cid:55)→ R nwhose inputs are the noisy measurement y and the associated operator A g , and the output is thereconstructed signal x .
The reconstruction can have either a denoiser form, f ( y, A g ) =
˜ f
( A † g x )where ˜ f is independent of A g [1], or a more complex unrolled structure with many denoising andgradient steps [27].
Measurement Splitting Inspired by the Noise2Noise approaches [17], some self-supervised methods [18, 19] split each measurement into two parts, y ⊤ =
[ y ⊤ 1 , y ⊤ 2 ] , such that the input is y 1 and thetarget is y i, 2 .
These methods can be summarised as minimising the following loss
[Equation] [Equation]
where y
=
A g, 1 x + ϵ
and y
=
A g, 2 x + ϵ 2 .
This approach suffers from the fact that f does not useall the available information in a given measurement, as it attempts to solve a harder reconstructionproblem associated with y 1 =
A g, 1 x + ϵ 1 .
As reconstruction networks often fail to generalise tooperators with more measurements [28], this method can suffer from suboptimal reconstructions attest time.
Proposed Method Inspired by the analysis in Section 4, we propose an unsupervised loss thatensures consistency across all projections A g , that is
[Equation]
(18) where ˆ x = f ( y, A g ) .
The first term ensures measurement consistency y =
A g f
( y, A g ) , whereasthe second term enforces consistency across operators, i.e., f ( y, A g ) =
f ( A s f ( y, A g ) , A s ) for allg ̸ = s .
Crucially, the second term prevents the network from learning the trivial pseudo-inversef ( y, A g )
=
A † g y .
In practice, we choose an operator
A s uniformly at random per minibatch.
Thus, compared the supervised case, we only require an additional evaluation of f and A s per minibatch.
We coin this approach multi-operator imaging (MOI).
This loss overcomes the main disadvantages of previous approaches: it doesn’t require traininga discriminator network, and it learns to reconstruct using all the available information in eachmeasurement y .
Experiments
In this section, we present a series of experiments where the goal is to learn the reconstructionfunction with deep networks using real datasets.
Experiments on low-dimensional subspace learningusing synthetic datasets are presented in the SM.
All our experiments were performed using aninternal cluster of 4 NVIDIA RTX 3090 GPUs with a total compute time of approximately 48 hs.
Compressed Sensing and Inpainting with MNIST We first evaluate the theoretical bounds on the
MNIST dataset, whose box-counting dimension, k ≈ 12 , is well-known [29].
The dataset contains N = 60000 training samples, and these are partitioned such that N/G different samples are observedvia each operator.
The forward operators are compressed sensing (CS) matrices with entries sampledfrom a Gaussian distribution with zero mean and variance n − 1 .
The test set consists of 10000 samples,which are also randomly divided into G parts, one per operator.
In order to evaluate the theoreticalbounds, we attempt to minimize the impact of the inductive bias of the networks’ architecture [30, 31]by using a network with 5 fully connected layers and relu non-linearities.
Figure 2a shows the average test peak-signal-to-noise ratio (PSNR) achieved by the model trainedusing the proposed MOI loss for G = 1 , 10 , 20 , 30 , 40 and m = 50 , 100 , 200 , 300 , 400 .
The resultsfollow closely the bound presented in Section 4 which is indicated by the red dashed line, as thenetwork is only able to learn the reconstruction mapping when the sufficient condition m > k + n/Gis verified.
In sampling regimes below this condition, the performance is similar to simply applyingthe pseudo-inverse A † g .
We also evaluate the reconstruction for a different number G of random inpainting masks anddifferent rates m .
The inpainting operators have a diagonal structure which has zero measure in 303 R m × n , however our sufficient condition still provides a reasonable lower bound on predicting theperformance, as shown in Figure 2b.
It is likely that due to the coherence between measurementoperators and images (both operators and MNIST images are sparse), more measurements are requiredto obtain good reconstructions than in the CS case.
[Figure]
Inpainting with CelebA We evaluate the unsupervised methods in Section 5 on the CelebAdataset [32], which is split into 32556 images for training and 32556 images for testing.
We usethe DCGAN architecture [33] for AmbientGAN as in [7], and the same U-Net architecture (see SMfor details) for MOI and supervised learning.
Reconstructed test images are shown in Figure 3 andaverage test PSNR is presented in Table 1.
The proposed method obtains an improvement of morethan 4 dB with respect to AmbientGAN and falls only 2 . 1 dB behind fully-supervised learning. Accelerated MRI with FastMRI Finally, we consider the NYU FastMRI dataset [34], where theset of forward operators A g consist of different sets of single-coil k -space measurements, with 4 × mask_rate=(0.5) A y 33. MOI (epoch=200) x (GT) 7.
mask_rate=(0.5)
A y 29.
(dcgan_G epoch=299) AMBIENTGAN x (GT) 7.
mask_rate=(0.5)
A y 33.
MOI (epoch=299) x (GT) 7. mask_rate=(0.5)
A y 35.
Sup (epoch=299) x (GT) 7. mask_rate=(0.5) A y 33.
MOI (epoch=200) x (GT)
A
[Table]
AmbientGAN
[Table]
Supervised
[Table]
Supervised
[Table]
acceleration, i.e., m/n = 0 .
We used 900 images for training and 74 for testing, which we splitacross G = 40 operators.
We compare measurement splitting, MOI and supervised learning, all usingthe same U-Net architecture.
For the measurement splitting method, we follow the strategy in [18],and choose to assign a random subset representing 60% of the measurements in A g to A g 1 and theremaining to A g 2 .
We observed that a model trained with measurement splitting obtained less testerror using reduced measurements associated with A g, 1 instead of just using the full measurements A g(see SM), so we report the best results using A g 1 .
As observed in [28], the network fails to generaliseto operators with more measurements.
Average test PSNR is presented in Table 1.
Reconstructed testimages can be found in the SM.
The proposed method obtains .
8 dB improvement over measurementsplitting and is approximately 2 dB worse than the fully supervised setting.
Limitations
Theorem 4.2 does not cover cases where the operators A g present some problem specific constraints(e.g.
, they are inpainting matrices) as well as cases where the signal model is only approximately lowdimensional.
Note however that Proposition 4.1 does apply to constrained operators.
We leave thestudy of sufficient conditions for these particular cases for future work.
The proposed loss might not be effective in problems where learning the reconstruction function isimpossible, e.g., due to high noise affecting the measurements [4].
In this particular case, it mightstill be possible to learn a generative model as in AmbientGAN.
Conclusions
We have presented sampling theorems for the unsupervised learning of signal models from incompletemeasurements using multiple measurement operators.
Our bounds characterize the interplay betweenthe fundamental properties of the problem: the ambient dimension, the data dimension and the numberof measurement operators.
Moreover, the bounds are agnostic of the learning algorithms and provideuseful necessary and sufficient conditions for designing principled sensing strategies.
Furthermore,we presented a new practical unsupervised learning loss which learns to reconstruct incompletemeasurement data from multiple operators, outperforming previously proposed unsupervised methods.
The proposed strategy avoids the adversarial training in AmbientGAN, which can suffer from modecollapse [35], and, contrary to measurement splitting, is trained using full operators A g .
Our resultsshed light into the setting where access to ground truth data cannot be guaranteed which is of extremeimportance in various applications.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e
g., data splits, hyperparameters, how theywere chosen)?
[Yes]
(c) Did you report error bars (e
g., with respect to the random seed after running experiments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e
g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
We used existing models and datasets
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[Yes] All datasets considered in the paper are publicly available.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[Yes]
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A]
We did not use any crowdsourcing or conducted research withhuman subjects.
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]