Learning to reconstruct from saturated data: audio declipping and high-dynamic range imaging
Authors: Victor Sechaud
Laurent Jacques
Patrice Abry
Julián Tachella
Venue: Accepted by TMLR ()

Abstract: Learning based methods are now ubiquitous for solving inverse problems, but their deployment in real-world applications is often hindered by the lack of ground truth references for training. Recent self-supervised learning strategies offer a promising alternative, avoiding the need for ground truth. However, most existing methods are limited to linear inverse problems. This work extends self-supervised learning to the non-linear problem of recovering audio and images from clipped measurements, by assuming that the signal distribution is approximately invariant to changes in amplitude. We provide sufficient conditions for learning to reconstruct from saturated signals alone and a self-supervised loss that can be used to train reconstruction networks. Experiments on both audio and image data show that the proposed approach is almost as effective as fully supervised approaches, despite relying solely on clipped measurements for training.

Published in Transactions on Machine Learning Research (01/2026)
Learning to reconstruct from saturated data:
audio declipping and high-dynamic range imaging
Victor Sechaud
victor. cectorsechaud@een--ygo....
CNRS and ENS de Lyon, Laboratoire de physique,
Lyon, Fronce
Laurent Jacques
lourent.jacquusshucclvvvo.....
UCLowain, ICTEAM,
Lowvain-la-Ner Neuve, Belgium
Patrice Abry
vatrice.bbry@ens-konn..
CNRS and ENS de Lyon, Laboratoire de physique, Lyon, Pronce CIFAR Fellow
Julián Tachella
julian. julien.tachelll@eens-yyo
CNRS and ENS de Lyon, Laboratoire de physique,
Lyon, Prance
Reviewed on OpenReview: https: // openreview. net/ form?? id= AkJWggl
Abstract
Learning basod methods are now ubiquitous for solving inverse problems, but their deploy- ment in real-world applications is often hindered by the lack of ground truth references for training. Recent self-supervised learning strategies offer a promising alternative, avoiding the need for ground truth. However, most existing methods are limited to linear inverse problems. This work extends self-supervised learning to the non-lincar problem of recover- ing audio and images from clipped measurements, by assuming that the signal distribution is approximately invariant to changes in amplitude. We provide sufficient conditions for learning to reconstruct from saturated siguals alone and self-supervised loss that can be used to train reconstruction networks. Experiments on both audio and image data show that the proposed approach is almost as effective as fully supervised approaches, despite relying solely on clipped measurements for training.
1 Introduction
Inverse problems appear in various engineering and science applications, such tomography (Jin et al. 2016), MRI (Lustig et al. 2007) phase retrieval Shhehhtman et al. 2015). They are described by the following forward equation:
y = h((())))))))))()))) +
(1)
where the aim is to recover the ground-truth x X R² from measurements y y C Rm. The set X defines the support of the of the signal distribution p(x), which refer to the "signal set". The function h Ra + Tm represents the forward operator and Rm the noise. These problems ill-posed due to potential information loss induced by h (for example, if m < n) or the existence of an infinito number of solutions. One way of solving this is to impose prior information on æ, such as piecewise constant assumptions Rudin et al. 1992). However, choosing the right prior can be challenging and misspecified priors can introduce biases or give poor approximations of the underlying signals. This problem can be mitigated by learning the prior or reconstruction function directly from data. Most learning based methods learn inverse operator of h on X, f + from training dataset containing ground truth with their
often

---

Published in Transactions on Machine Learning Research (01/2026)
associated measurements {( yoll and neural network fo that parameterizes the inverse function. The standard approach computes the parameters 0 IRP by minimizing the mean square error (MSE):
i = arg min Σ (lo(())) =1,1?
(2)
6
Despite the good performance in some applications, this approach suffers from two main drawbacks: (i) ground-truth images can be partieularly difficult or impossible to obtain, e.g., in scientific imaging appli- cations (Belthangady & Royer 2019), and (tt) even when we have access to a ground truth dataset, there might be significant distribution shift between training and testing.
Self-supervised learning presents an alternative that circumvents some of these limitations (Belt Belthangady & Royer 2019). This approach operates without the need for ground-truth training data, making it especially advantageous scenarios where obtaining such datasets is expensive or impractical. Furthermore, self- supervisod learning can be trained on mcasurement data directly, thus mitigating the challenge of distribution shift.
Learning from measurement data only is particularly challenging when the forward operator is not invert- ible TTchella et al.) 2023). If the forward operator is linear, self-supervised learning is possible by assuming that the underlying set of clean signals is invariant to a set of transformations, such as rotations or trans- lations (Chen et al. 2021.. This also holds for the non-linear problem of reconstructing heavily quantized signals (1-bit measurements) (Tachella & Jacques 2023). Here we show that invariance to rotations or translations is not enough for learning from clipped measurement data alone, and instead prove that in- variance to amplitude is sufficient for fully self-supervised learning. We propose self-supervised loss that enforces equivariance to changes in amplitude, and show throughout series of experiments that our method can achieve performance comparable to fully supervised methods. A preliminary version of this work was presented in Sechaud et al. (2024), and here we generalize to a larger class of signals such as images, and propose a thcorctical framework showing sufficient conditions for learning to recover signals from saturated measurements alone. In summary, the contributions of this work are threefold:
We provide necessary conditions for signal recovery under known signal set, and also demonstrate, under an invariance assumption, that the signal set can be identified with sufficient number of observed measurements;
We propose self-supervised loss that can be used to train neural networks for audio declipping and High-Dynamic Range (HDR) imaging;
We show that this method can achieve comparable performance to fully supervised methods on both audio and image data.
The rest of the paper is structured follows: Section 2 reviews related work concerning self-supervised learning and state-of-the--rt declipping methods. Section
3 introduces our theoretical framework, focusing model identification-icc. determining underlying signal set from observations- and signal recovery. From there, in Section E we describe how one can practically define and minimize a self-supervised cost. function with scale invariance. Section presents experimental results that validate our approach.
2 Related work
2.1 Self-supervised learning for inverse problems
Self-supervised methods allow to train reconstruction networks using only measurement data (Chen et al. 2021). Existing methods can be separated into two categories: the ones that handle the noise of the problem, 2018 Krull et al. (2019), and the others that handle non-invertible forward operators (Tachella al.) 2023). The last set of methods mostly focuses on the case linear operator where the loss of information is associated to its nullspace. Tt. is possible to overcome this missing information
o.g., Noise2X methods (Lchtinen
2

---

Published in Transactions on Machine Learning Research (01/2026)
by (i) accessing measurements from several operators with different nullspaces Daras et al. et al. 2023). or ((i) by assuming that the signal distribution is invariant to a set of transformations such
2023 Tachella
translations or rotations, & method known as equivariant imaging (El) (Chen et al. 2021). The El framework
is well studied for lincar inverse problems, both in terms of applications (Scanvic et al., 2024 Chen ot al.
2021 and theory (Tachella et al. 2023). . To the best our knowledge, the only case nonlinear inverse problems analyzed using the invariance approach is one-bit compressod sensing (Tachella & Jacques 2023
2.2
Audio declipping
Several unsupervised techniques for audio declipping have been proposed (Zaviska et al. 2021 Kitic et al, 2013 Gaultier et al. 2021). many of which make use of variational methods with priors such as sparsity cerlain bases (e-g. Fourier domain). Both lo and l-mmiimizattin approaches are commonly employed, often solved using dedicated optimization algorithms, with social sparsity as one of the best performing (Sicden- burg et al. 2014). Learning-based techniques, such as dictionary learning, have also shown to be effective, where a dictionary is trained from time windows of the clipped signal to represent it as sparse vector. Additionally, methods incorporating prior knowledge based on human perception (Defraene et al.) 2013). or using multichannel data (Ozcrov et al.) 2016 Gaultier et al.) 2018), have been used to improve performance. Diffusion models trained on a supervised way (i.e., with ground-truth references) have also been studied in the context of declipping (Moliner et al. 2023
2.3 High Dynamic Range images
The dynamic range in images refers to the difference between the brightest and darkest values. Since camera sensors and displays have limitations in capturing and showing both dark and bright regions, this often results in lost details in these zones. To address this, various techniques employed to extend the dynamic range of images. One common method is exposure bracketing (Mertens et al. 2007). which combines multiple photos at different exposures for higher quality but requires long exposures and a stationary target to avoid misalignment. Another approach is modulo sensing (Contreras et al., 2024 a promising technique, though it is still limited by the unavailability of commercial hardware. Finally, supervised deep learning methods, such as the ones proposed by Eilertsen et al. (2017). utilize a single image to estimate a high dynamic range. However, these methods require a large dataset of ground truths to train the model effectively, which be significant. limitation. For example, in ultra-high-speed imaging (thousands of frames per second), sensors must trade off dynamic range for acquisition speed. As result, high-contrast and transient phenomena such as combustion in rocket launch site monitoring remain challenging to capture accurately, even with the most high-end cameras GGassi et al. 2015 Tao et al. 2025
2.4 Signal recovery guarantees for saturated measurements
Some theoretical works have analysed the feasibility of the signal recovery problem from saturated obser- vations. Following compressive sensing approach (Candes & Wakin 2008 Foucart & Needham (0016) present unique recovery guarantees for the case of Gaussian measurements and using an & minimization approach, showing that sparse vectors with low magnitude can be uniquely recovered with high probability.
3 Analysis for model identification and signal recovery
3.1
Problem formulation
We consider the forward formulation y = 17(x), where threshold levels pr: P2:
R + R is the element-wise clipping operator with
M4 if
u S M1.
(u)
u E (M1,)2), if
#2 if
u > u2.
For our examples with audio and synthetic signals, we use a symmetric threshold μl - = p. For images, we only consider an upper threshold, where μn 0 and μ 1, assuming positive signals æc, as camera sensors
3

---

Published in Transactions on Machine Learning Research (01/2026)
do not suffer saturation from below. Indeed, under low-light conditions, the sensor produces a weaker signal, which may lead to noise or reduced image quality, but not to saturation. We assume that μn1 and #22 are known and that we have a dataset of saturated signals {y: We aim to learn the reconstruction function fo using the measurement dataset alone. We will therefore consider the following two theorctical questions: (i) model identification: for a forward operalor y and measurement sels }/ does a unique signal set X exist satisfying some mild priors and y = =))()? In other words, we want to know if is possible to find the support of the signal distribution from measurement data {yr);= alone as N - oo, and some mild priors on X (e.g. invariance to transformations). If the set X' can be identified, we could then hope to recover the signal æ from single observation y via the following program:
& E arg min [1( yl1?.
(3)
mEX
(ii) Signal recovery: is there unique solution to the problem in (3? In other words, can we uniquely recover the signal from observations knowing the set X? There may be unique solution for one, both, or neither of the problems (Tachella ct al.) 2023). Blind compressed sensing is an example where we can have signal recovery if the dictionary is known, but we cannot identify the dictionary from measurement data alone (Gleichman & Eldar 2011]. On the contrary, we can identify the signal set from rank-one measurements without being able to recover the signals associated with each measurement (Chen et al, 2013). Solving these two theoretical problems allows us to assess the feasibility of learning to reconstruct from measurement data alone, using the self-supervised loss proposed in Section 4 In the following, we will use "identify" only for finding the set X' given measurement set V, and "recovery" for finding vector æ given a measurement y.
3.2 Model identification.
In this section, we will focus the case where the full set of measurements y is observed, although in practice we only observe {y:) We thus study the conditions for identifying X' from the set of saturated signals y = n2²).
It may be interesting to note that in some situations we can easily identify the signal set. This happens in the trivial case where the whole signal set is below the saturation threshold.
Example 1. In the symmetric case with threshold level μ and no additional priors on X, if llyll < p for all y E ,, then X y.
However, in general without any assumptions on X neither V we cannot hope to guarantee identification, as shown in Remark [ We thus need to consider some assumptions on X' to identify the signal set from y. As done in equivariant imaging (Chen et al., 2021), we can consider the group of transformations G under which the signal set is invariant. The reason is that the invariance to transformations {T,) beT gives us access to the sets V,- = XT for all G. Indeed, we can see y as a set of measurements associated with the set X' for different forward operators (Ty.), as we have that:
yg = -(()))))(()==)))) = if Tyx'=
We then need to identify which groups of transformations can help in idcntifying X. We cannot use the classical groups considered in the linear framework (shift or rotation) as these transformations commute with the operator 7). As shown in the following proposition, if the transformations commute with n, they do not help to identify the signal set.
Proposition 1. Let G be group, if cannot identify X from y (7()) and if for all g E G, the Irunsformations Tg commule with v(-), then we cannot identify X from Y, = (TTx)) for all G.
Proof. Let G such that for all g G. Tay(.) (TT.))) We have access to all measurements associated with the different transformations of the signals:
Uy, y, {ug (TTa)) Vg G, Va E X}. gEG
4

---

Published in Transactions on Machine Learning Research (01/2026)
As T3 and 7 commute, we have
v (ug- Ty(()) Vg G, Vo X) geG
U Tan(X),
geG
and so we cannot identify X' from that set since it doesn't bring us more information than r7(2;; the set of measurements is a function of ()))).
Example 1 shows that the norm of x has impact the feasibility of reconstructing the signal, which gets us to consider the group action of scaling.
Definition 1 (Scale invariance). We say that X is scale invariant if
gx X for all g E E.
(4)
A set that is coal-invariiat is called come [Boyd & Vandenberghe, 2004).
Proposition 2. For any two distinct conic sets X' + X', we have that m7()) * ()2)).
Proof. Let the open set B, = {u E ||// < <p} with radius μ > We recall that we note y the set of measurement 7))). Taking the set of unsaturated signals in X, that is X' n Ba the operator n has impact them and therefore
XO n(X n BB))
Moreover, if x is i then |n(x)] μ, therefore
7(x n (B,) cynB
But if y is in m((e) B,,, then never reaches the threshold or -4, and there is thus an æ in X XB, such that y = f7xx) = x, i.c.,
(v12)) B,) c (X)
showing finally that
y =
Defining & = {gy geRt, y yEупB, as the conic extension of y IB,, We will show that & = X.
Inclusion X c :: Let x Then, there exists > 0 such that cæ belongs to X n Bp, Therefore, y belongs to &, which implies X' &
Inclusion & Cx: Let HE &. By definition, there exist R4 and E JNBB such that = gy- Since yn B, = X n B,,, we have y E X. As X' is a cone, it follows that & = 99 E X. Thus, we conclude that x c C, completing the proof.
Remark For asymmetric clipping, the same result can be obtained for μi S 0 p2 and X R However, we can find 2-dimensional counter example for 0 < in < N2: we choose X1 (am E R+} and X = {axz RR} with I1 =12, PT) and 22 (#2, a TT Then Xn and X, have the same measuremend sel:
y = (t) :t) e (2)) U ((22,t) (mw1,
3.3 Signal recovery
Sufficient conditions for signal recovery generally depend on the dimension of (Sauer al. 1991 Following previous work compressed sensing (Sauer al. 1991 Falconer 2013). we use the box-counting dimension, measure of the complexity of set. which generalize lizes several notions of dimension.
5

---

Published in Transactions on Machine Learning Research (01/2026)
Definition 2. The upper bor-counting dimension of compact subset X' is
log [x((c]]
dim(X') = sup
c-0 - log(())
where (e) is the covering number, i.e., the minimum number of closed balls of radius (with respect to the norm || (12] with centres in X' needed to cover X.
It can be applied to various structures:
Vector spaces: The box-counting dimension of space R intersected with the unit ball is k corre- sponding to the classical geometrical dimension.
Manifolds: For a manifold of dimension k, the box-counting dimension k, consistent with its geometric dimension.
Bounded k-sparse sets: In high dimensional spaces, the box-counting dimension reflects the effective dimension rather than the ambicnt dimension n.
Thus, the box-counting dimension uniquely determines the dimension of these various objects, providing a flexible framework for measuring the dimension of objects that can appear in compressive sensing or inverse problems.
We approach the problem signal recovery by considering, for conic signal set X, its normalized version:
{ | : ZEE
Sx
and the open bounded set
XR fxr E X | < RJ.
When studying the signal recovery problem, one observes that pathological cases where 7(x1) = n(x2) for #1 * æ2 may arise even when dim(X') is small, as illustrated in Figure 1 As the operator v acts as a projection onto the cube B, the signal set is orthogonal to one of the cube's faces, becomes impossible to recover a signal once it reaches that face. However, we expect such cases be relatively rare, and therefore we study the problem by randomizing the orientation. To this end, as commonly employed in signal recovery analyses (Ahmed al. 2013), consider the operator n(A)), where A E Rmxn is random Gaussian matrix, which acts approximaly
random rolation of the signal set. An illustration is provided in Figure [ The following theorem provides sufficient conditions on the number of measurements and the maximum radius R to ensure, with high probability, that distances between points in the set X'R are approximately preserved after applying (AA)). Section 5.2.1 provides a simplified example demonstrating how this matrix effectively addresses degenerate cases.
Theorem Let us assume that the normalized set Sx has a finite upper boz- boz-counting dimension dim (Sx) < k k, such that Nx(e) - for all < with e* (0, 1/2). For m, n 0, given a random matriz pm with i..dd. entries Aij N (0,1), a threshold μ 0, and radius R 0, there erists absolute constant
C > 0 such that if
m 2 llo((1/e*) and R < m(} ktl),
then
(()(Ax)==( for some x + u E XR) < 12e 12e-Cm Cm
(5)
In other words, we measure the probability that the mapping n not injective (and therefore impossible to invert) over XR.
Theorem shows that the event of picking two vectors of XR with identical image through 17 o A has a probability that decays exponentially fast with the dimension if this dimension is large compared to the

---

Published in Transactions on Machine Learning Research (01/2026)
A.
(-p,, p4)
(-P (.
(p. 10)
(AAX)
(2())
X
(x)) AX (AAx)
Figure 1: Example in 3 dimensions illustrating Theorem The black set presents particularly challenging scenario in which all saturated signals are projected to the same point. In contrast, with high probability on A, the colored set enables recovery of more signals with moderate norms, as points in X' beyond a certain radius- which is connected to the signal norm when A Gaussian are all projected onto a single corner, showing the non-injectivity beyond this radius.
dimension & of the normalized set Sx, and the radius is not bigger than p( interesting effect considering that that the fraction of components of Ax Au that impacted by the saturation 7) is constant (in average) when increases. Indeed, for æ E XR, each entry of Ax is distributed as a Gaussian random variable with zero mean and standard deviation ||=|]. Therefore, for ~ NN 1) and if m large enough, the fraction of saturated projections m=11(11 S i S m (Ax)) | 2 p)ll is close to m m EE P(((Ax) 2 μ) = [[||||=||| 2 μ4), which does not depend on m.
kt1). This is quite an
To consider the initial problem, that is, the forward operator n7.), we may introduce an assumption on X of the form
by considering it
X= (Ax: E 2),
where is a cone with Sz of box-dimension & and A Rexa such that AG ~ N (0,1), We can apply Theorem in the case of a random rotation m = n, show that a signal æ can be recovered with high probability ||æ||| ||A|| where ||A|| denotes the norm of the operator A, and in the case of Gaussian square matrix of variance 1, it is close to 2/n (Vershynin 2018
The proof of the theorem is provided in the Section A and sketched in Figure 1 It consists in showing that the same measurement vector cannot be associated with two distinct signals, with high probability. The core idea of the proof is to decompose the probability that the mapping is not injective into two parts: Given two distinct signals, we control, the probability that the number of common unsaturated measurements is small (thus making it impossible to differentiate between signals). On the other hand, we control the probability that two signals have the same common unsaturated measurements given that the number of saturated measurements is low and so it is acceptable to ignore them.
Let us stress that in Theorem we cannot set an upper bound on the probability in (5) for all signals in X' and need to limit this space to bounded signals from XR. Some signals in X with an excessive norm cannot be recovered as they have too numerous saturated measurements. In Figure [ this is illustrated by the accumulation, after saturation, of signals in the corners of the cube, particularly when signals have high norms, i... far from the cube's center. When all measurements are saturated, or equivalently when j tends to 0, expect to recover the behavior of 1-bit compressive sensing (Tachella. & Jacques 2023) where

---

Published in Transactions on Machine Learning Research (01/2026)
only be estimated up to non-zero error. This is why we focus signals with low norms. Theorem can be seen generalization of Foucart & Needham (2016) Theorem 1), to more general low-dimensional sets, going beyond sets of k-sparse signals.
|
4 self-supervise learning approach
Given the positive thcoretical results regarding model identification and signal recovery the previous section, we now propose a method for learning from saturated signals {y; - n(x;))==r We first introduce the loss functions used train the network, then describe the network architecture and implementation details used for the reconstruction.
4.1 Loss functions
Mcasurement consistency (MC) loss: in the absence of noise, a good reconstruction method should provide an estimale that is consistent with the observations, i.e,, for all measurements y, we should have
n(f(()))= yy.
Measurement consistency means that the reconstructed signal produces the same measurements those that were originally observed. Ensuring MC is crucial, as it guarantees that estimates adhere to observations. However, it does not necessarily mean that the solution is unique since multiple solutions could be consistent with the same measurements. A straightforward loss function for imposing mcasurement consistency is
(NMC( = Σ 33 - (fo(v.))
(6)
which we will call the naive measurement consistency loss. If at the first training steps, the network predicts a value & above the threshold, the gradient will be zero due to the y operator and thus get stuck on this prediction, preventing the network from learning. This is particularly problematic if yy below the threshold, since we know that the true value æÿ - My- We therefore choose the MSE where the signal is not saturated according to the following loss:
(MC(O) Σ (P((o(()). vill
(7)
where p is applied element-wise as: p(a, b) (1] bb al + 1 (0==)) (b a)+ + (0=- b)+
where 1(-) the indicator function and (+)+ the positive function. See Figure 2 for comparison of these functions.
However, even with this new loss, we cannot hope to learn the correct reconstruction function. Minimizing CMC allows us to learn a measurement consistent solution fe, but this solution could the form:
foly); =f
y3 if v11))
|,, S yy =
(8)
if
with v being any function lying beyond the threshold level μ i.e., v(y;) belongs to the preimage 17 '1 (1-111,1 uT). For example, the function fo = y minimizer of
Equivariance loss: To effectielly learn in the saturated region, we use the amplitude invariance assumption of the signal set defined above, going beyond the limitations imposed by measurement consistency alone.
To enforce this invariance property on the reconstruction network, we note that since gæ E X', the function fo must be capable reconstructing both æ and gæ for all x in X' and g E TR,, that is:
(T(())) g.r and fo(n(2))
(9)

---

Published in Transactions on Machine Learning Research (01/2026)
(+) = 1 (
Lmc($) = ||(@)
L for y < μ
L for y = μ
2
non zero gradients
zero gradient
0
/
0.5g
1
2
/4
0.5g
/
Figure 2: LNMC compared to LMC- On the left: Curve of the LNMC in one dimension for two examples in both cases saturated and not. On the right: Curve of the new LMC for the same two examples.
Thus, can conclude that:
fo fe( = (n(())))) for all æ E X, and g E R,.
(10)
Hence, the composition of fo with 1 should be equivariant with respect to the multiplicative group (R4, To cnsure this, we propose a loss function that enforces this oquivariance property:
(EI(0) Σ Egups [losfo (v.) (ofo
(11)
We can observe connection with Proposition 2 for N - xo, after rescaling, the sum becomes an expectation over a distribution supported on y, and the expectation over g basically acts as (restricted) conic extension of that support.
The expectation is evaluated with respect specific distribution Pa over the multiplicative group, which, in principle, could be any unbounded distribution over R4. However, in practical scenarios, signals are typically limited in amplitude, making it sufficient to use distribution defined over a finite range [Amin: fmax] (not necessarily uniform). The parameters (Amin: Imax) were determined empirically: setting Imax too low prevents the network from lcarning effectively, while excessively large valuos cause instability.
We could also extend 11] to include transformations ions not limited to scale invariance, such as affine group transformations of the form æ ax + b1, where (a, b) R, R and 1 1I) R"
The final loss function is combination of the measurement consistency and equivariance losses:
( = Lmc (0) + (O).
(12)
The parameter X is a positive hyperparameter that controls the trade-off between the two losses.
4.2
Network architecture
4.2.1
Core architecture
The proposed self-supervised loss can be used with any network architecture fe- For most of the experiments reported here, we choose the well-known U-Net neural network for the reconstruction function fe (Ron- neberger al., 2215),, which is well-suited for capturing the relevant (temporal or spatial) correlations natural signals. The details of the architecture vary depending on the type of data, audio or image, and are explained in the experiment section.

---

Published in Transactions on Machine Learning Research (01/2026)
Reconstruction with bias
Reconstruction without bias
Ground truth
Measurement
x
t
85
Mv ///// // Many
10
t
Figure 3: An example where the reconstruction is learned through the bias (third column) which prevents the reconstruction for low amplitude measurements. We thus observe that removing the bias makes the network naturally homogeneous and improves the reconstruction. On the first row, we consider a signal æ, on the second row I0
4.2.2 Bias-free neural network
We choose a bias-free version of the U-Net, to preserve the scale invariance assumption (Mohan et al. 2020 This is because feedforward neural network with ReLU activation functions and without bias defines homogencous function:
= Vg R..
g->0 This gives fo(gy) - O, which is particularly necessary for unsaturated signals. Figure 3 shows an example
where a biased network cannot reconstruct an unsaturated signal.
4.3 Masking
A key aspect of the threshold operator is that, by knowing the threshold levels, we can identify the unsatu- rated, and therefore non-degraded, portion of the sigual. While the measurement consistency loss encourages the network to preserve the unsaturated part, it does not guarantee strict adherence to this roist. To address this limitation, can enforce measurement consistency more rigorously by employing the follow- ing masking method, comparable to more structured skip connection architecture. The estimated signals & are computed using an element-wise linear blending between the outputs of the network fo(y) and the measurements, that is
i (1_),,,,,,,+ +b,
(13)
By choosing an appropriate blend value by for all sample points or pixels, the blend works as a mask that modifies only saturated measurements, avoiding artifacts the unsaturated parts.
by {
y P 1
= p
An example of an image and its associated mask is shown in Figure
10

---

Published in Transactions on Machine Learning Research (01/2026)
The blending mask
Figure 4: An image with its associated blending mask. The mask is white where channel is saturated and black when none is saturated.
5 Experiments
We begin by presenting experiments toy datasets, where control the properties of the signal set. The first experiment uses dataset associated to one-dimensional vector space that is invariant to amplitude. The second experiment is performed using various datasets generated by sampling vectors from random k-dimensional subspaces. To generate these datasets, we first control the dimension of the vector space, followed by the saturation proportion for each sampled vector. The reconstruction is then evaluated based on these varying parameters. Next, we conduct experiments on real-world music data, and we compare our method several state-of-the-art techniques. Finally, we conclude with experiments on two-dimensional signals, where aim to reconstruct HDR images.
5.1 Metrics
Two common metrics for evaluating the quality of reconstructions are the Signal-to-Distortion Ratio (SDR) and Peak Signal-to-Noise Ratio (PSNR). PSNR is more suitable for image quality assessment, as it incorpo- rates maximum possible pixel value, allowing for a clearer comparison of signal fidelity. SDR, on the other hand, is generally applicd to audio signals because it does not require knowledge of the maximum possible value. Although we do not know this value for HDR images, we still use PSNR to provide a more relevant comparison with other works. The PSNR and SDR metrics are defined follows:
PSNR(æ, +) 20 (1 || & 2112
1
11bb)
(14)
||a|||
SDR(æ, .) =20logıo
(15)
|| - Ø||2
Both metrics are referred to as distortion metrics since they assess the fidelity of the reconstruction process. In addition distortion metrics, perception metrics could be used to evaluate the quality of the recon- struction, offering a more accurate measure of how changes In this work, employ the widely adopted metric Perceptual Evaluation of Speech Quality (PESQ) to evaluate audio signals. For image, use the Natural Image Quality Evaluator (NIQE), no-reference (no ground-truth needed) metric that estimates image quality. While perceptual metrics are particularly relevant for applications like music audio, where subjective experience plays a key role, they are less suitable for signals where preserving critical information is essential, as they tend to permit greater hallucination introducing elements that were not part of the original signal, potentially compromising the accuracy of the reconstruction. Blau & Michaeli (2018) study the impossibility of algorithms optimize both perception and distortion metrics, known the perception-distorioon tradeoff.
signal image affect human perception.
11

---

Published in Transactions on Machine Learning Research (01/2026)
5.2 Toy datasets
5.2.1 MNIST experiment
In this example, we use a toy signal set, generated from a single signal and augmented with scaled versions, Lee. X { ex e E ] for zo fixed. In practice, we take an image æ from MNIST and create the dataset D = (egmo);== N with 6 realization of a random variable with exponential distribution of mean 2, for i E ,I,..., J. This dataset has two interesting properties: it is scale invariant, and we cannot have unique signal recovery as shown in Figure
7(x1) )
77( ((Ax))
fo (17((11)
fo (n((AA)))
21
5
5
7T22)
(Ax))
So [0(11(22))
(v( (Ar2))
22
122
Figure 5: Example where two different signals æ1, 22 have the same measurement. r7(x1), 17222) and so the network fe fails to recover them both (the second and fourth columns). Adding randomness implies that the two measurements are not equal anymore, and so network fg can reconstruct the original signal (the third and fifth columns).
Measurements are obtained by applying the forward function (AA)) with an orthogonal matrix and thresh- old levels μl 0, μ2 = 0.4. The matrix A drawn randomly with respect the Haar measure (Mezzadri) 2007). While Figure shows that is impossible without A to recover the original signals, adding ran- domness in the forward operator solves this issue, as illustrated in Figure [ Figure B that the neural network trained using the loss function LMC + LET successfully reconstructs images with dynamic range similar to the original images, whereas the network trained with CNMC + LEi as the loss function does not achieve this. However, both fail when the dynamic range of the original signal becomes excessively high. For signals with high dynamic range, characterized by a large norm ||. [|2, the network is unable to recover them as indicated in Theorem [
5.2.2 Synthetic dataset
First, we generate a random subspace of R100 with dimension & E {1,..., 100) using basis vectors whose coordinates are drawn from a standard normal distribution. Next, we generate N 1000 vectors { within this subspace, rescaling each vector such that proportion (0,1) of their components are clipped with the threshold μ set to 1.
We evaluate the recovery performance as a function of the parameters k and v. Figure 7 illustrates the network's ability to reconstruct unsaturated signals as these parameters vary. The performance decreases as the subspace dimension grows or the proportion of saturated samples increases. Indcod, signals from nigher-dimensional subspaces are more challenging reconstruct, which consistent with the result of Theorem 1
12

---

Published in Transactions on Machine Learning Research (01/2026)
Dynamic range of fe(y)
Dynamic range of fe (
2
Dynamic range of a
Dynamic range of
Figure 6: Plot of the dynamic range of fo(y) depending on dynamic range of æ, where - 77 Axx). Left: The training is done with the losses CNMC + LEI. Right: The training done with loss CMC + CEI- A perfect plot should be the identity line.
A Average test SDR
A Average test SDR
0.5
14.25
12.01
10.17
9.15
8.51
8.17
7.74
7.53
0.5
12.07
8.84
7.28
6.89
6.64
6.39
6.24
6.17
0.4
14.62
13.04
11.27
10.45
9.89
9.44
9.28
9.13
0.4
13.
10.56
9.27
8.63
8.27
7.99
7.93
7.77
Saturation
Saturation
0.3
15.53
13.87
12.7
12.18
11.73
11.21
10.87
0.3
14.6
12.26
10.65
10.36
10.05
9.88
0.2
17.66
15.34
14.55
14.09
13.78
13.47
13.28
13.18
0.2
15.68
14.46
13.56
13.16
12.96
12.6
12.55
12:38
0.1
21.68
18.59
17.99
17.7 17.58 17.2 17.2 17.07
0.1
19.87
17.
96
17.6
17.35 17.21 16.95 16.86 16.81
5
10
15
20
25
30
35
40
10
15
20
25
30
35
40
Dimensions A
Dimensions
k
Figure 7: Average reconstruction performance as a function of saturated part v and signal set dimension k for both supervised (left) and self-supervised (right) methods. The value indicated corresponds to the mean SDR over the test dataset.
5.3 Audio
We evaluate the method real audio dataset. Specifically, use the GTZAN dataset, which contains 10 genres (rock, classical, jazz, etc.), each represented by 100 audio files, all 30 seconds long with sampling rate of 22,050 Hz. The audio signals are split into 30 segments of 1-second each and then saturated with a threshold level = 0.1. Mcasurements y without any saturated entries are discarded. This process results in training dataset of 21898 audio samples and testing dataset of 100 samples.
The distribution of Pg is set uniformly on the interval [0.1. 2]. Results are compared with state-of-the- art variational method called social sparsity (Zaviska et al. 2021 Siedenburg et al. 2014] and supervised methods. Both the supervised and self-supervised approaches We refer to the method trained using only 5% of the whole training datasct as "Supervised 5%", which is used to compare the performance of the both approaches under limited ground-truth data acquisition. We also compare our method with the following methods: self-supervised with loss Lmc alone, supervised with MSE + LEI- As shown Table [1, the proposed self-supervised method achieves performance comparable the fully supervised approach, although the latter requires significantly less training time: hour 46 minutes compared to 13 hours 52 minutes for the self-supervised method. The results further confirm that
the network architecture for training.
13

---

Published in Transactions on Machine Learning Research (01/2026)
training with the MC loss alone does not yield additional information beyond the measurements themselves, as anticipated in Section 4 Similarly, the proposed method with a biased network does not perform
well, illustrating the importance of using a bias-free architecture as seen in Section 4.2.2 In addition, the supervised method with equivariance does not demonstrate any significant improvement over the standard supervised approach.
Table 1: Reconstruction performance, SDR and PESQ are averaged over all music test dataset.
Methods
SDR f
PESQ
Identity
4.44 2.12
2.7
Social Sparsity Supervised
9.92 + 4.46
2.09
11.69 2.25
1.94
Supervised 5% Supervised + LEI Proposed self-supervised Unsupervised with LMC alone
9.79 1.59
2.28
11.72
1.72
10.48 + 2.20
2.20 0.80
4.84 2.12
2.68 + 0.65
Proposed superviid with bias
9.27 3.22
2.43 0.90
To demonstrate that supervised training dataset may not generalize well to a different test set, we conduct the following experiment: we create a supervised training dataset consisting of only music, while the test dataset includes both music and voice recordings (Vryzas et al.) 2018a b), with only measurement data available (no ground truth). The supervised method, trained using the MSE loss (2), learns solely from the training dataset, whereas the self-supervised method is trained on both the training and test datasets since it does not require ground truth. Both methods are then evaluated the test dataset. The results, shown in Table 2 demonstrate that the self-supervised method is more robust when the training and test datasets differ.
Table 2: Average SDR performance the test dataset, which includes both music and voice recordings. SDR (dB)
Methods
Identity
6.54 2.34 2.34 10.94 + 2.00
Supervised (trained on music)
Proposed supervupe (trained on music and voice)
11.92 + 2.46
Table 3: Average SDR and PESQ performance on the test. MAESTRO dataset.
Methods
SDR f
PESQ 4
Identity
5.00 + 0
2.9 1 0.73
Supervised
6.70 + 1.69
2.22 = 0.52
Proposed self-supervised
7.78 8.64
2.47 # 1.12
We also compare the self-supervised learning method to a supervised learning approach using diffusion mod- els (Moliner et al., (023).. The comparison is conducted on portion of the MAESTRO dataset (Hawthorne et al. 2019), used in Moliner's work, which consists of 200 hours of classical solo piano recordings. The measurement signals from Moliner al. (2023) are used benchmark; although the saturation threshold varies for each signal, it consistently corresponds to an SDR value of 5. Each signal is then normalized to yield saturation threshold of 0.1, while preserving constant SDR of 5. This approach enables comparison between the proposed method, based on a fixed saturation threshold, and Moliner's trained method. Results demonstrate that the sele-super method outperforms the supervised one Table 3 It is important to note that the proposed self-supervised method focuses on optimizing the distortion metric SDR, whercas this supervised method aims optimize a perceptual metric PESQ. Results are then consistent with the perception-distortion tradeoff seen in Section 5.1
14

---

applied see Published in Transactions on Machine Learning Research (01/2026)
y
Esupervised
#self-supervised
y
Esupervised
suplessuerrieedd
Figure Image reconstruction on the test dataset. An exposition correction details better.
(Wikipedia)
5.4 HDR
The HDR experiment is inspired by Filertsen et al. (2017). and its goal is compare supervised and self- supervised methods. Results are summarized in Table and in Figure g The dataset is composed of 1043 HDR images noted [Le et al. 2023), which are considered as real scenes. Photos are taken with virtual camera calibrated by camera curve and exposure. For each image, we choose the exposure time to saturate between 5% and 15% of the pixels. We divide the image by the quantile go where v is chosen uniformly in (0.85,, 0.95). Then we have defined
2 >> = -1-2 IP is the cumulative histogram of u. The camera curve is
uuu); = (1 + σ)
We these functions to fit the database of real camera curves collected by Grossberg & Nayar (2003). We choose B ~ (V0.9, 0.1) and ~ (/0666 The images = represent the ground truth dataset. They are quantized and clipped to create the measurement dataset. This step is performed with:
255 min x,x;)) + 0.5]
yj
(16)
255
We unfolded network with U-Net architecture. It combines traditional iterative optimization with neural network learning. For each iteration, we replace part of the optimization step with neural network. We choose the Half-Quadratic Splitting (HQS) (Aggarwal et al. 2018) algorithm which aims to minimize p<xx) + Xx(x), for two closed proper convex functions &p and v. The iteration step is given by:
Us pr (Ix),
1k+1 prox (ux).
In setting, corresponds to the measurement consistency term (lx) [| y)ll and prox is replaced by neural network. The proximal operator of the measurement consistency term is given by:
21478 1ty
if |2, S μ.
prox yyp (2:);
Ij otherwise
for n..
15

---

Published in Transactions on Machine Learning Research (01/2026)
Table Reconstruction performance, PSNR averaged Methods
over all test dataset images.
PSNR t
NTQE 4
Identity
29.54 + 6.95
4.21 + 1.96
PnP DPIR
32.11 = 5.80
4.06 + 1.93
Supervised
36.67 + 6.19
4.04 + 1.74
Proposed self-supervised
35.04 + 6.33
4.10 + 1.85
In Table 4 we compare our method against two reference approaches: the supervised method from Filertsen et al. (2017) and the Plug-and-Plas (PuP) method (Venkatakrishhan et al., 2013). The PnP method an iterative technique that integrates denoising module into the optimization process, replacing the proximal operator. We used the Deep Plug-and-Plas Image Restoration (DPIR) method (Zhang et al.) 2021 a PnP approach based HQS in which the denoising network is pre-trained DRUNet, and the noisc level for the denoiser is adapted at each iteration. For beller results, modified the inilialization of the algorithm which is generally set to by adding a white Gaussian noise of standard deviation = 0.18, which we found to better avoid local minima close to y. In this way, DPIR outperforms the other PnP methods tested. Results in Table demonstrate that the self-supervised method performs on par with the supervised method and significantly outperforms the DPIR approach. As illustrated Figure g both the supervised and supervise methods effectively recover details in the images.
6 Conclusion
In this study, we propose a self-supervised method that uses amplitude invariance to address the nonlinear declipping problem. We provide a theoretical framework with guarantees for model identification of scale invariant signal models and unique recovery of the unsaturated signals when the operator involves random matrices with coofficients following Gaussian distribution. Experimental results, carried out on audio and images, indicate that this method can perform on par with the supervised approach, and surpasses variational methods, This work opens up possibilities for learning-baseed methods in nonlinear inverse problems.
7
Acknowledgment
Victor Sechaud and Julian Tachella are supported by the ANR grant UNLIP (ANR-23-CE23-0013) The reserch of Laurent Jacques is partly funded by FRS-ENRS (QuadSense, T.0160.24). Part of this research carried out when LJ supported by the OCKHAM team at ENS Lyon (France), with INRIA Chair at the Institute for Advanced Study, Collegium of Lyon.
References
Hemant K Aggarwal, Merry P Mani, and Mathews Jacob. MoDL: Model-based deep learning architecture for inverse problems. IEEE transactions on medical imaging, 38(2))394-400) 2018.
Ali Ahmed, Benjamin Recht, and Justin Romberg. Blind deconvolution using convex programming. IEEE Transactions on Information Theory, 6013):17 2013.
Chinmay Belthangady and Loie A. Royer. Applications, Promises, and Pitfalls of Deep Learning for Fluo- rescence Image Reconstruction, February 2019.
Yochai Blau and Tomer Michaeli. The Perception- Disttorioo Tradeoff. In 2018 IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 6228-6237, June 2018.
Stephen Boyd and Lieven Vandenberghe. Conver optimization. Cambridge university press, 2004.
E.J. Candes and M.B. Wakin. An Introduction To Compressive Sampling. IEEE Signal Processing Magazine, 25(2):21-38 -00 March 2008. ISSN 1053-5888.
16

---

Published in Transactions on Machine Learning Research (01/2026)
Dongdong Chen, Julián Tachella, and Mike E. Davies. Equivariant Imaging: Learning Beyond the Range Space. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp-
August 2021.
Yuxin Chen, Yuejie Chi, and Andrea J Goldsmith. Exact and Stable Covariance Estimation from Quadratic Sampling via Convex Programming. IEEE Transactions on Information Theory, 61(7):4034 4059, 2015.
Kebin Contrcras, Brayan Monroy, and Jorge Luis Bacca. High dynamic range modulo imaging for robust object delection in autonomous driving. In ROAM ECCV 2024, 2024.
Giannis Daras, Kulin Shah, Yuval Dagan, Aravind Gollakota, Alex Dimakis, and Adam Klivans. Ambient diffuuson: Learning clean distributions from corrupted data. Advances in Neural Information Processing Systems, 36:288 313, 2023.
Bruno Defraene, Naim Mansour, Steven De Hertogh, Toon Van Waterschoot, Moritz Dichl, and Mare Moo- nen. Declipping of audio signals using perceptual compressed sensing. IEEE Transactions on Audio, Speech, and Language Processing, 21(12):2627-25 2637, 2013.
Gabriel Eilertsen, Joel Kronander, Gyorgy Denes, Rafal K. Mantiuk, and Jonas Unger. HDR image recon- struction from a single exposure using deep CNNs. ACM Transactions on Graphics, 36(6):1- 15, December 2017. ISSN 0730-0301, 1557-7368.
Kenneth Falconer. Practal geometry: mathematical foundations and applications. John Wiley & Sons, 2013.
Simon Foucart and Tom Needham. Sparse recovery from saturated mcasurements. Information and Inference, pp. iaw020, December 2016. ISSN 2049-8764, 2049-8772.
Clément Gaultier, Nancy Bertin, and Rémi Gribonval. Cascade: Channel-aware structured cosparse audio declipper. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 571-575. IEEE, 2018.
Clément Gaultier, Srdjan Kitić, Rémi Gribonval, and Nancy Bertin. Sparsity-based audio declipping meth- ods: Sclected overview, new algorithms, and large-scale evaluation. IEEE/ACM Trasaactins on Audio, Speech, and Language Processing, 29:1 19:1174-111 1187, 2021.
Davide Giassi, Bolun Liu, and Marshall B Long. Use of high dynamic range imaging for quantitative combustion diagnostics. Applied optics, 54(14):4580-4588 2015.
Sivan Gleichman and Yonina C Eldar, Blind compressed sensing. IEEE Transactions on Information Theory, 57(10):6958-699) 6975, 2011.
Michael Grossberg and S.K. Nayar. What is the Space of Camera Response Functions? In 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings., volume 2, pp. IT 602. IEFE, July 2003.
Curtis Hawthorne, Andriy Stasyuk, Adam Roberts, Ian Simon, Cheng Zhi Anna Huang, Sander Dieleman, Erich Elsen, Jesse Engel, and Douglas Eck. Enabling factorized piano music modeling and generation with the MAESTRO dataset. In International Conference on Learning Representations, 2019.
Kyong Hwan Jin, Michael McCann, and Michaol Unser. Bpconvnct for compressed sensing recovery in bioimaging. 2016.
Srdjan Kitic, Laurent Jacques, Nilesh Madhu, Michael Peter Hopwood, Ann Spriet, and Christophe De Vleeschouwer. Consistent Tterative Hard Thresholding for Signal Declipping. In Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. IEEE, 2013.
Alexander Krull, Tim-Oliver Buchholz, and Florian Jug. Noise2void-learning denoising from single noisy images. In Proceedings of the IEEE/CVF conference on computer vision and pottern recognition, pp. 2129 2137, 2019.
17

---

Published in Transactions on Machine Learning Research (01/2026)
Phuoc-Hieu Le, Quynh Le, Rang Nguyen, and Binh-Son Hua. Single-Image HDR Reconstruction by Multi- Exposure Generation. In Proceedings of the IERE/CVE Winter Conference on Applications of Computer Vision (WACV), January 2023.
Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, and Timo Aila. Noise2Noise: Learning image restoration without clean data. International Conference Machine Learning, 2018.
Michael Lustig, David Donoho, and John M Pauly. Sparse MRI: The application of compressed sensing for rapid MR imaging. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, (8(6):1182-11999) 2007.
Tom Mertens, Jan Kautz, and Frank Van Reeth. Exposure Fusion. In 15th Pacific Conference on Computer Graphics and Applications (PG'07) pp. 382-390, Maui, HI, October 2007. TEEE. ISBN 978-0-7695-3009-3.
Francesco Mezzadri. How to generate random matrices from the classical compact groups. volume 54, pp. 592 -044 February 2007.
Sreyas Mohan, Zahra Kadkhodaie, Eero P. Simoncelli, and Carlos Fernandez Brrmandez-Granda. Robust and inter- pretable blind image denoising via bias-free convolutional neural networks. In International Conference on Learning Representations, February 2020.
Eloi Moliner, Jaakko Lehtinen, and Vesa Välimäki. Solving Audio Inverse Problems with a Diffusion Model. In ICASSP 2023-2023 JEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, March 2023.
Alexey Ozerov, Çağdaş Bilen, and Patrick Pérez. Multichannel audio declipping. In 2016 TEFE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 659 663. IEEE, 2016.
Gilles Puy, Mike E. Davies, and Remi Gribonval. Recipes for Stable Linear Embeddings From Hilbert Spaces to RT. IEEE Transactions on Information Theory, 63(4)22171 2187, April 2017. ISSN 0018-9448, 1557-9654.
James C Robinson. Dimensions, embeddings, and attractors, volume 186. Cambridge University Press, 2010.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image compuling and compuler-assisees intervention MICCAI 2015: 18th inter- national conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pp. 234 241. Springer, 2015.
Leonid 1 Rudin, Stanley Osher, and Emad Fatemi. Nonlinear total variation based noise removal algorithms. Physica D: nontinear phenomena, (0(14))2259 268, 1992.
Tim Sauer, James A. Yorke, and Martin Casdagli. Embedology. Journal of Statistical Physics, (3-4:
579 616, November 1991.
Jérémy Scanvic, Mike Davies, Patrice Abry, and Julián Tachella. Self-Supervised Learning for Image Super- Resolution and Deblurring, March 2024.
Victor Sechaud, Laurent Jacques, Patrice Abry, and Julián Tachella. Equivariance- based self-supervised learning for audio signal recovery from clipped measurements. In 2024 32nd European Signal Processing Conference (EUSIPCO), pp. 852 856. IEEE, 2024.
Yoav Shechtman, Yonina C. Eldar, Oren Cohen, Henry Nicholas Chapman, Jianwei Miao, and Mordechai Segev. Phase Retrieval with Application Optical Imaging: A contemporary overview. IEEE Signal Processing Magazine, 3233)887-10,, May 2015. ISSN 1053-5888.
Kai Siedenburg, Matthicu Kowalski, and Monika Dorfler. Audio declipping with social sparsity. In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1577-1581,
Florence, Italy, May 2014. IFFE. ISBN 978-1--499--893-4
18

---

Published in Transactions on Machine Learning Research (01/2026)
Julián Tachella and Laurent Jacques. Learning to reconstruct signals from binary measurements. In Four- teenth International Conference on Sumpling Theory and Applications, 2023.
Julián Tachella, Dongdong Chen, and Mike Davies. Sensing theorems for unsupervised learning in linear inverse problems. Journal of Machine Learning Research, 24(39):1 45, 2023.
Jing Tao, Youghong Zong, Banglei Guan, Pengju Sun, Taihang Lei, Yang Shang, and Qifeng Yu. Perceptual region-driven infrared-visible co-fusion for extreme scene enhancement. Optics 8 Laser Technology, 192: 113870, 2025.
Singanallur V Venkatakrishnan, Charles A Bouman, and Brendt Wohlberg. Plug-and-play priors for model based reconstruction. In 2013 IEEE globol conference on signal and information processing, pp. 945-948. IEEE, 2013.
Roman Vershynin. High-dimensional probability: An introduction with applications in data science, vol- ume 47. Cambridge university press, 2018.
Nikolaos Vryzas, Rigas Kotsakis, Aikaterini Liatsou, Charalampos A Dimoulas, and George Kalliris. Speech emotion recognition for performance interaction. Journal of the Audio Engineering Society, 66(6):457-467, 2018a.
Nikolaos Vryzas, Maria Matsiola, Rigas Kotsakis, Charalampos Dimoulas, and George Kalliris. Subjective ovaluation of speech emotion recognition interaction framework. In Proceedings of the Audio Mostly 2018 on Sound in Immerssoo and Emotion, pp. 1 7. 2018b.
Wikipedia, Exposure value. https://en. http:/daiaa.rrriiiiii///pppppsus value
Pavel Zaviska, Pavel Rajmic, Alexey Ozerov, and Lucas Rencker. A Survey and an Extensive Evaluation Popular Audio Declipping Methods. IEEE Journal of Selected Topics in Signal Processing, 15(1):5 24, January 2021. ISSN 1932-4553, 1941-0484.
Kai Zhang, Yawei Li, Wangmeng Zuo, Lei Zhang, Lue Van Gool, and Radu Timofte. Plug-and-play image restoration with deep denoiser prior. IEEE Trunsactions on Pallern Analysis and Machine Intelligence, 44(10):6360-66 6376, 2021.
A Appendix: Proof of Theorem
We recall that the idea of the proof is to upper-bound two probabilities: (i) the probability that the number of common unsaturated measurements is small; (ii) the probability that two signals have the same common unsaturated measurements given that the number of saturated measurements is low. Before we begin, we need to introduce some tools of linear embedding. Theorem allows us to prove Corollary 2.1 used to control the first targeted probability. Theorem 3 is used control the second one. We also recall the Hoeffding incquality for sub-Gaussian random variables, which we will use verify an assumption for Thcorem D
Lemma 1. Hoeffding inequality. Let XI,..., Xm h be independent random variables, where X; has mean h4 and o sub- Gaussian. Then for all 2 we have
2
IP
Σ i=
X,
14
21 < 2exp
2 o
Σ
==1
Thcorem 2. This theorem is adapted from Puy al. 2017
Theorem II.2).
Let us assume that the normalized set Sx has a finite upper bor- counting dimension dim(Sx) which strictly bownded by t 2 1 f.e. dim(Sx) < k, and so there erists a signal set. dependent constant e" (0, 3)) such that
19

---

Published in Transactions on Machine Learning Research (01/2026)
Nsx (c) S c for all e ct. Let A R - mm such that As ~ N (O,
:.). Then there erists O such
Uhat, for any E, E (0, we have sup
|c|| Azll - 1 s with probobilily at least 1 provided that
=ESx
C1
1
6 log
)
m 2 max
k log
0,82
E
where Ci is on absolute constant.
The proof of Theorem 2 closely follows that of Puy ct al. (2017 Theorem II.2). consists in using a general chaining argument (see, e.g., Vershynin 20188 Chapter 8)) on the function æ E IK + h(x) == c|| - provided that, for any fixed vectors u, Sx {0) and with high probability, the absolute difference [h(u) h(v)l does not deviate too much from the distance ||u tta-ell e|I, fact that we prove the following lemma.
Lemma 2. Let A: + R such that Asj ~ N (0, m)), e> O, and h be defined as
h: Ra
c|| ||
Then there erist constant E R such that for any fixed u. Sx U (0),
P ((h(u) h(v)l 2 Al/u b|l) 2am E
Proof- We consider h + cll cll ||a||2 with c 0. Restricted to Sx, have h: + c|| 1.
F ((h(w)= - /(v)] 2 Allu - -vl))
(17)
P(c|||ul]) - -|A 1 2 AlIu - vll)
m
= P
a u
| ml こ - mll v1l)
(18)
We note X; = |a u/ -la v/. Let's show that X; is sub- Gaussian::: For all g E N and for all i (l...., m}.
E || E IlaT u - la [0]== s Ella, (u 1)
by the reverse triangle inequality. As
a (u v) ~ N((0. 1|u 111
a (u - v) is Gaussian and thus X, is also Gan-sanssian with parameter of < -u-ell We can therefore apply the Lemma] 1
P
X, >t < 2eepp
i=0
20
and thus give upper bound of 18 ((h(u)-A(()) 2 al|u-=11) -ll-)
(Allue w1))?
$ 2exp
2c m
Σ of
1=
(All 1u w1)
2eep
20
Σ
|| w113
1=
2
2eep
26
2eepp
me X2).
20

---

Published in Transactions on Machine Learning Research (01/2026)
1
We conclude the proof with C1
2mc
Corollary 2.1. Under assumplions of Theorem 2 we have for all m 2 k log
P(Vx E X: ||ax|| S /ml|z)12) 6e -Cm
Proof. By applying Theorem 2 for with probability c.
2_ deduce that with C1 2mc
g. for any E, E (0, 1), we have
7m
2 m
V2 E Sxx
A ||
S d
(19)
as long as
C1
6 log
)
m 2
max
h: log
6
6
For all x E X, obtain by replacing by | in 19
ym m
A
S
+
1)
|
(20)
provided that
8C1
1
)
6 E
2
max
k: k:oog
log
82
By setting C
8C1 = and & = 6 cxp ( - Cm), we can verify that the last incquality holds for all
1 -
2 klogg
and thus:
P(Vx X: ||Ax||1 S /ml|=zlla - 6c -m
Theorem 3. Adapted from Robinson 2010 Thm. 4.3) Let a compact set X with finite upper boz-counting dimension (x) < k and A E gaxx such that Ar ~ N (o, h)) If m > 2k then
P (AAx Au for some + u E x) =0
Note that Robinson's theorem states that almost every linear map L: R.N. RR (for m > 2k) embeds injectively with Hölder distortion || b1l S Cl Lyll", for some p (0, 1). By considering the definition say that the set of non-injective linear maps has zero
of "almost every" in the sense of measure, measure, and therefore zero probability.
For the rest of the proof, we will consider
=== B (0, Vmpa)
with R = Vmpa and a > 0 to be define latter. We define
Inn(y) {j |us 2 n1 Ieai = [j 1y < μ4]
as the of the index of respectively unsaturated and saturated entries of y. We note TT the matrix represents the row-submatrix of A where only the rows indexed by I are selected.
21

---

Published in Transactions on Machine Learning Research (01/2026)
Let I = (Au) be the intersection of the unsaturated index sets of Az and Au. Using the law of total probability we can decompose the probability of Theorem into the two target probabilities. By considering;
A 3x, u d', x + u: A x = A A
B = (Vx, u E NR. + u: 1, > (1 2a m
where A is the set of events where there exists two signals having the same common unsaturated mea- surements, B the set of events where the number of common unsaturated measurements is greater than (1 - 2a for all couple of signals. And taking it into the law of total probability:
((A) = P(An B) + P (AnB)
= PP (A] B) x +P B) (P
s (A4 B) +P
(21)
as we have that P (BB) < and P (A[B) < 1.
IP (AA B) and P are respectively the probability that two signals have the same common unsaturated measurements given that the number of saturated measurements is low, and the probability there exist two distinct signals whose the number of commnn unsaturated measurements is low. Next, we upper-bound separately the two terms P ( and PP (B). We start with the first one:
P(A B) = I (f30,u E Xn. x # u A A
Hf Vor E Nn. x + u IZA (1 2a)m) }
< <PIII [o: ml, I (1 2a.m, Ex, u E In, u A A=u)
And because of X is a cone, we have Robinson 2010
dim(XR) < dim(X' S) + = dim (Sx) + 1 + ++1
We thus have, as long as (1 2a)m > 2(k + 1), and thanks to Theorem 3
P(II c [0 ml, [Z| > (1 2a)m, Ex, u E XR. x Fu: a A A =
(22)
And so the first right-hand hand side term 21 is equals 0.
From m 2(k + 1) which imposes < 2 m we can fix a and therefore fix /mp (3 k+1) We also note a > 0, we necessarily have 2(k: 1) <..
m
We can now upper-bound the second term appearing in 21
P(3x, Xn, u
IA
I (1 )m.
By observing that for two set I and (l....,m, m} with ||n]| s (1 - 2a22m.m this means that [īu J > 2am, which implics that cither |7| > am or [J] > am (since otherwise the union has size < 20m), therefore
M < (1 a)m or |J| & (1 a),, we can deduce:
IA 22
S (1 2a)m
T Axx)
I (Au)
S (1 2a)
Isos (Ax) <
m+ (1-2a
(1 m
2
Tane (Au) <
m 22amm 2
< (1 a)mm
22

---

Published in Transactions on Machine Learning Research (01/2026)
have
P [Ex Xn, u [A,
(1 2a)m)
$ P (3x E AR
Teat (Ax)
Σ (1 - a)m} } {Bu E dR: [Tea (Au)l S (1 a)m})
- 2P (3xe E AR: Isat (Ax)
I (1 a)m)
and if we remark that
m
||
Σ
a a
Σ i (Aw)
la xol 2
Σ
1=1
VETnne (Az))
wc obtain
| 2 ITaae (Ax) nr.
Thanks to Corollary 2.1 we know that with probability at least 6e Cm -m
| I Vmllz=la & maj.
We deduce from these two last inequalities
Tsst (Ax)) 1r S map
Isat (Ax) S ma
(1 a)m S Imaat (Ax)
And thus
P (3x E XR [saul (Ax) S (1 a)m) s 6e Cm
long
m 2 log
Finally,
P (B) < 12c Cm
(23)
we conclude by adding the upper-bound obtain in
22 and 23
P P(ax, u E An, æ + u: (Ax)) (Au)) < P la, u E XR u: A A=c.uu)
< P (A B) FB(B
< 12c Cm
Finally, (5 follows from a simple rescaling of the radius and the vectors x and u by a factor / /m, and an opposite rescaling of the entries A, of A by 1//m so that Ag (0, 1).
23

---

Published in Transactions on Machine Learning Research (01/2026)
B Appendix: Hyperparameters, training details and images reconstruction
Table Hyperparameters for different experiments.
Section Section 5.2.1
Pg
Architecture
Learning rate
Epochs
Batch size
u(0.1,2)
Bias-free U-Net, down 4 up blocks
5e
300
50
Section 5.2.2
u(0.5, 1.5)
MLP, inpul size 100, hid- den size 100, depth 5
le-1
300
100
Section 5.3
0.1
2001
Bias-free U-Net, down +
5e
100
35
5 up blocks
Section 5.4
0.1
u(0.2, 1.5)
Bias-free U-Net, down +
5e
400
12
4 up blocks
12
11
10
SDR
Supervisod
sele-vuue (ours)
7
10
20
30
40
50
60
70
80
90
100
Epochs
Figure SDR performance on the evaluation dataset during training for supervised and self-supervised methods.
24

---

Published in Transactions on Machine Learning Research (01/2026)
y
@supervised
@gelf-supervised
dsupervised
@Pelf-supprriso
superviasd
sell-supervised
supervisse
raperisaper
@supervised
a sele--upervieed
Figure 10: Image reconstruction on the test datasct. Artifacts can appear in saturation zonc
25


Reviews:
[{"review_id": "gt1sUTjUUy", "reviewer": "Reviewer_9rV1", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This review is to reconsider the submission of the same paper as previous submitted to TMLR, with reference link [here](https://openreview.net/forum?id=gwDNM3b353).\n\nLooking at the current version, there is literally no change from the previously rejected version. In particular, the main issue:\n```\n(1) The main theoretical result presented in the manuscript is trivial after authors reformulated proposition 3. Theorem 1 is also within expectation, because when the variance of the random sampling/compression matrix A is small (i.e. when m>>1), the approximate isometry of Gaussian matrix has classic Johnson-Lindenstrauss lemma that serves as the prototype to the presented result, regardless of the inclusion of the thresholding operator.\n\n(2) As another reviewer pointed out, there is no essential connection between the presented theorem and the practical reconstruction algorithm (the application of the invariance penalty).\n\nThe measure consistency loss does not make much sense. It reads that the thresholded reconstructed signal should align with the observation. Then there are infinite many solutions to this problem.\n\n(3) Authors still fail to formulate the problem they try to solve as a clear optimisation problem.\n```\n\nNone of these points is addressed. As there is no change to the previous version, the conclusion should be the same.", "claims_and_evidence_yn": "No", "claims_explanation": "See summary section.", "audience_yn": "Yes", "audience_explanation": "Topic is relevant", "requested_changes": "N/A", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "WNlWZBPkGN", "reviewer": "Reviewer_UfYi", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a self-supervised framework for reconstructing signals from saturated measurements, addressing nonlinear inverse problems in audio declipping and high dynamic range (HDR) imaging. The authors identify and formalize amplitude invariance as a key property enabling self-supervised learning in such settings. The paper makes the following contributions:\n\nTheoretical Foundations: The authors provide sufficient conditions under which a signal model can be identified and signals can be recovered solely from saturated observations. These conditions rely on the assumption that the signal set is scale-invariant (i.e., conic) and has low box-counting dimension.\n\nSelf-supervised Loss Design: A novel self-supervised loss is proposed, combining:\n\nA measurement consistency loss tailored to handle non-zero gradients even in the presence of clipping;\n\nAn equivariance loss that enforces consistency under amplitude scaling, reflecting the scale invariance of the data.\n\nExperiments on Audio and Images: The method is validated on both synthetic and real-world data:\n\nIn audio declipping, the self-supervised method outperforms traditional methods and approaches the performance of fully supervised models.\n\nIn HDR imaging, it performs comparably to supervised baselines, despite not requiring ground-truth during training.\n\nOverall, the work significantly expands the scope of self-supervised learning in inverse problems beyond linear regimes by handling nonlinear saturations, a common real-world issue.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The submission provides sufficient experimental results and comparisons with relevant baselines to support its main claims. The methodology is described clearly, and the evidence is presented in a structured and convincing manner.", "audience_yn": "Yes", "audience_explanation": "The paper addresses a timely and relevant problem in machine learning, and its findings would be of interest to researchers and practitioners in the field. The proposed approach and results contribute to ongoing discussions in the community, making the paper suitable for TMLR’s readership.", "requested_changes": "The author has already responded well to my previous questions, and I have no further questions.", "broader_impact_concerns": "No significant ethical risks were identified in this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "GSkKF4GmYo", "reviewer": "Reviewer_SbLd", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work considers the nonlinear inverse problem with saturated measurements, under the assumption that the underlying signal set is conical. This manifests as a special instance of equivariant learning. Using the same network architecture, the authors demonstrate that the proposed self-supervised method is reasonably well performing compared to supervised training, with the expected additional computational burden.", "claims_and_evidence_yn": "Yes", "claims_explanation": "* The experimental section is sufficiently ablated and clearly demonstrates the effect of each of the authors' modifications. The imaging section also compares to a powerful pretrained method (DPIR), and shows that the different degradation operator needs to be separately addressed.", "audience_yn": "Yes", "audience_explanation": "* The authors consider an interesting nonlinear problem which is not entirely covered by existing theory. \n* Theorem 1 is interesting as it considers the interaction between random feature and saturation. It provides a vanishing bound on the probability that two data will result in the same measurement, based on the box-counting/intrinsic dimension of the underlying signal set.", "requested_changes": "Strengthening points\n\n* Minor formatting/clarity issues that can be fixed easily. I list some easily fixable ones in the following section.\n\t* Example 1: maybe state that no additional priors are required on $\\mathcal{X}$.\n\t* Prop 1 is strangely placed and may be bit confusing by itself. Maybe combine with Remark 1?\n* Prop 3 can be simplified even more. Inscribing a sphere of radius $\\mu$ into the hypercube, the Euclidean projection of the hypercube onto the sphere is bijective. Then you can use that the cone is uniquely defined by its intersection with the sphere of radius $\\mu$ (and therefore by its intersection with the hypercube).\n* \"proposed self-supervised with bias\": I could not find the definition of this, but I assume it is the UNet but without the modification in 4.2.2?\n* Are there assumptions on $m,n$ in Theorem 1? It would be useful to have this in the statement.\n* Section 4.1 could do with some relations to variational regularization, in this case the regularization corresponding to the scale-invariance prior is given by the equivariance loss (11).\n* Fig 3 needs more detail in the caption, state that removing the bias makes the network naturally 1-homogeneous (... logic here ... ) and improves the reconstruction.\n* How does $C$ in Theorem 1 depend on $R$? This would be nice to quantify the effect of clipping.\n\n**Typos/small changes**\n- (p.1) \"mispecified\" -> \"misspecified\", \"the inverse operator\" -> \"an inverse operator\"\n- (p.2) could you clarify how self supervised learning mitigates distribution shift?\n- (p.3) reference issue in (Moliner et al., 2023), it starts as a new sentence. Also it may be good to add some examples of HDR image artifacts, like bars in low brightness gradient areas, or overblown whites\n- (p.3) \"for the case **of** Gaussian measurements and **using** an $\\ell_1$ minimization approach\"\n- (p.4) when defining model identification: perhaps \"for a forward operator $\\eta$ and measurement sets $\\mathcal{Y}$, does a unique signal set $\\mathcal{X}$ exist satisfying some mild priors and $\\mathcal{Y} = \\eta(\\mathcal{X})$\", or something along those lines. Also should it be $\\mathcal{Y}$ instead of $\\{\\mathbf{y}_i\\}$? \n- (p.4) Reference issue (Chen et al.). \"group of transformations $G$ for which the ...\" -> \"...transformations $G$ under which the ...\"\n- (p.6) \"as it is common signal recovery analyses\" -> \"as commonly employed in signal recovery analyses\" . Also random Gaussian matrix is not a rotation, maybe approximate rotation.\n- (p.7) \"here $\\mathcal{Z}$ is a cone\" -> \"where ...\"\n- (p.13) \"mintutes\"\n- (p.15) \"for $\\varphi$ and $\\psi$ two generic functions\" -> \"for two closed proper convex functions $\\varphi$ and $\\psi$\"\n- Equations are sometimes referred to as Equation (xx) or sometimes just (xx). Please unify", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)