Cross-Domain Lossy Compression via Rate- and Classification-Constrained Optimal Transport
Authors: Nam Nguyen
Thinh Nguyen
Bella Bose
Venue: ICLR 2026 Oral (2026)
Keywords: Lossy Compression
Image Compression
Image Restoration
Image Inpainting
Optimal Transport
Multi-task Learning
Rate-Distortion-Perception Tradeoff
Rate-Distortion-Classification Tradeoff
Deep Learning
Unsupervised Learning
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We study cross-domain lossy compression via constrained optimal transport with rate and classification constraints, derive closed-form tradeoffs, extend to perception divergences, and validate with deep restoration and inpainting experiments.

Abstract: We study cross-domain lossy compression, where the encoder observes a degraded source while the decoder reconstructs samples from a distinct target distribution. The problem is formulated as constrained optimal transport with two constraints on compression rate and classification loss. With shared common randomness, the one-shot setting reduces to a deterministic transport plan, and we derive closed-form distortion-rate-classification (DRC) and rate-distortion-classification (RDC) tradeoffs for Bernoulli sources under Hamming distortion. In the asymptotic regime, we establish analytic DRC/RDC expressions for Gaussian models under mean-squared error. The framework is further extended to incorporate perception divergences (Kullback-Leibler and squared Wasserstein), yielding closed-form distortion-rate-perception-classification (DRPC) functions. To validate the theory, we develop deep end-to-end compression models for super-resolution (MNIST), denoising (SVHN, CIFAR-10, ImageNet, KODAK), and inpainting (SVHN) problems, demonstrating the consistency between the theoretical results and empirical performance.

Accepted to ICLR 2026.
CROSS-DOMAIN LOSSY COMPRESSION VIA RATE- AND
CLASSFFCAATION-CONTTTAAAA TRANSPORT
OPTIMAL
Nam Nguyen, Thinh Nguyen, Bella Bose School of Electrical Engineering and Computer Science Oregon State University Corvallis, OR 97331, USA (nguynam4, thinhg. bella waw.lla.bbeeeeeeo...oe..... boossJoorggonttate.edd
ABSTRACT
We study cross-domain lossy compression, where the encoder observes a de- graded source while the decoder reconstructs samples from distinct target dis- tribution. The problem is formulated as constrained optimal transport with two constraints compression rate and classification loss. With shared common randomness, the one-shot setting reduces to a deterministic transport plan, and we derive closed-form iittootioonaatt-llasiiiiaattttti classification (RDC) tradeoffs for Bernoulli sources under Hamming distortion. In the asymptotic regime, we establish analytic DRC/RDC expressions for Gaus- sian models under mean-square squared error. The framework is further extended to incorporate perception divergences (Kullback-Leible and squared Wasserstein), yielding closed-form distortion-rate-peee distorioonntte-preeppioonnoeess perceppion-classiicatiio (DRPC) functions. To validate the theory, we develop deep end-to-end compression models for super- resolution (MNIST), denoising (SVHN, CIFAR-10, ImageNet, KODAK), and in- painting (SVHN) problems, demonstrating the consistency between the theoretical results and empirical performance.
(DRC) and rate-distortion-
INTRODUCTION
Classical dittorisoo (RD) theory provides a single-letter characterization of the minimal distor- tion achievable when reproducing source under rate constraint (Cover & Thomas 1999). This foundation has guided decades of research in lossy compression and inspired the design of mod- ern learned codecs. However, standard RD formulations assume that reconstructions should remain close to the observed input distribution. many setting, this assumption misaligned: the encoder observes degraded sample X (c.g., noisy low-resolution), while the desired output is restored sample Y that lies in different, target distribution py (c.g., clean or nigh-resolution). Moreover, beyond fidelity, the compressed representation must remain informative for downstream tasks such as classification, introducing additional constraints that are not captured by classical RD.
Perception-awart RD extends the RD framework by incorporating divergence between the source and reconstruction distributions, highlighting an intrinsic and motivating generative compression approaches (Blau & Michaeli 2018 2019 Theis & Wag- ner, 2021). In restoration tasks, the target is not the degraded input px but the clean domain Py, so enforcing perceptual closeness between py and px is conceptually mismatched. Task-aware extensions such as sire-distortion-llasiii classsfficatiiin (RDC) or rate-distortion oiteeddiitrrtioopprrppppiooooo (RDPC) explicitly account for classification performance (Wang et al. 2025), but typically assume that reconstructions remain in single domain rather than supporting cross-domain mappings with distinct marginals. parallel, compression has also been studied as denoising mechanism. Weiss- man et al. (Weissman & Ordentlich 2005) showed that optimal lossy compression followed by post- processing can asymptotically achieve the fundamental denoising limit, while more recent work (Za- fari et al. 2025b) introduced neural compression based denoising, including zero-shot framework with theoretical guarantees and algorithmic instantiations. These results highlight the value of com- pression denoising prior but do not address cross-domain alignment task-aware constraints.
www.iitootooopeeeeetriiieee (RDP) tradeoff'

---

Accepted to ICLR 2026.
Finally, optimal transport (OT) provides principled tool for coupling distributions (Villani 2009 and has been leveraged in unsupervised restoration (Wang et al. 2023a). However, OT by itself ignores coding constraints and does not account for downstream requirements. Related to our work, et (2022) formulated cross-domain lossy compression as entropyycoontrriined OTT and showed that shared randomness can decouple coding from transport, but classification constraints were not included. Unsupervised image restoration has also been studied in (Zhang et al. [017]] Menon et al.) 2020 Pan al., 2021) with fixed reconstruction distribution; however, these approaches neither investigate compression constraints nor incorporate
setfffaation-awarenese for downstream tasks.
To that end, we formulate cross-domain restoration as compression, through a constrained lossy optimal transport framework. Given degraded samples from px and desired reconstructions from py. with distortion function, we optimize couplings px,y y that simultaneously (i) minimize expected distortion, (ii) satisfy rate constraint, and (iii) preserve task utility by constraining the uncertainty of the downstream label S given the reconstruction Y. Following [iu et al. (202); Theis Agusts- son (2021); Theis & Wagner (202I), we exploit shared common randommess between the encoder and decoder to show that, in the one-shot regime, the system reduces to selecting a deterministic transport plan with effective rate and classification constraints. In the asymptotic regime, this leads to a mutual-informatinn inffrmationnccontraiinn constrained transport problem with an additional classification constraint, yielding Shannon-style single-letter characterization. Our work makes the following contributions.
We introduce constrained lossy optimal transport, generalizing OT by incorporating both coding and classification constraints. With common randomness, transport (reconstructing Y to match py) and compression (coding Y) structurally decouple, extending Liu et al. (2022) to task-aware settings. Closed-form characterizations are provided for (i) one-shot Bernoulli sources under Hamming dis- tortion, where DRC and RDC functions admit piecewise-lined forms, and (ii) asymptotic Gaussian sources under mean- equare error (MSE). where analytic expressions are derived.
The framework is further extended to the distortion-rate-prrset ceattc-erreettoncllsssee classificatioo (DRPC) setting with two perception divergences: bullback-Leiblee (KL) and squared Wasserstein. For Gaussian sources, extremality results yield closed-form DRPC characterizations that, to our knowledge, are the first to explicitly incorporate classification.
We implement deep end-to-end compression frameworks, incorporating (i) universal quantization for shared randomness, (ii) an entropy model for rate estimation, (iii) a WGAN discriminator for aligning reconstructions with py. and (iv) a classifier head for controlling classification loss. Ex- periments on super-resolution (MNIST), denoising (SVHN, CIFAR-10, ImageNet, KODAK), and inpainting (SVHN), demonstrate strong agreement with the thcoretical results.
2 SYSTEM MODEL AND ONE-SHOT SETTING RESULTS
We study scenario where the encoder observes an input X ~ px, which rep- resents a degraded version (e.g., cor- rupted by noise, reduced resolution) of an underlying clean source. Associated with each sample is a classification la- bel S ps. with preseribed covari- ance between X and S as Cov(X, S) E[(X [xx)=x))S
Classification Loss H(SIP)
9
9
Encader
2
Decoder
j. Following the approach in Liu et al. (2022). we uti- lize shared randomness between the en- coder and decoder to enhance perfor- mance. Specifically, we introduce common random variable U, accessible at both sides, which is independent of the input X, i.e., U) = This assumption ensures that the decoder has no prior knowledge of X beyond the transmitted representation. In practice, U' can be realized by pre-agreeing on pseudo-random number generator with a shared seed, enabling both parties to generate identical randomness. The encoder must map X into compressed representation Z under rate constraint, namely U) < R. The decoder, given Z, produces a reconstruction Y that should follow a target distribution py-
Distortion Loss E(d(X,
Figure 1: System model: noisy input X ~ px is stored as Y ~ py, supporting classification with label S.
2

---

Accepted to ICLR 2026.
We consider px and py probability distributions over X C R". Formally, enforce similar- ity between X and Y through a fidelity criterion defined by a distortion function d' X' x y - R. We assume d(XX, Y) 0 if and only if Y. For instance, d(X,, Y) can be the Hamming distance or the MSE distortion. In addition, we require the reconstruction to remain useful for downstream classification, specifically. the uncertainty of 5 conditioned on Y is constrained as H(SIY) s C for some C' > 0 (Wang et al. 2025).
The main objective of source restoration (e.g., denoising or super-resolution) in our setting is three- fold: (i) Degradation removal: mitigate artifacts and imperfections present in the degraded input X: (ii) Information preservation: retain as much information as possible about the underlying clean source X' contained in X; (ii) Classification utility: ensure that the reconstructed Y yields high classification performance. As an example, consider X as a noisy image and Y as its clean re- construction. Figure D provides a schematic of the full system, where compression and restoration jointly yield a sample Y Py that serves both fidelity and classification purposes. We interpret this formulation through the lens of optimal transport: the problem reduces to identifying joint distribution px,y consistent with given marginals px and py, subject to a distortion cost d(-; -), rate constraint R. and classification constraint C. We next connect this framework to optimal transport and describe how our formulation of classification lossy compression naturally extends it.
2.1 ONE-SHOT CONSTRAINED OPTIMAL TRANSPORT
Definition 1 (Optimal Transport), Let [pxx, py) denote the set of all joint distributions px, with marginals px and py. The classical optimal transport problem identifies coupling in this set that minimizes the expected transportation cost:
D(px.Pr)= Dx.y
inf
Eld(X, Y)
(1)
where d(:,-) is a prescribed distortion (or cost) function and px,y I(px. Py) is transport plan.
The transport plan in D the average distortion between input and output, subject only to the marginal distributions px and py. Our goal is to extend this framework by requiring the transport plan to additionally satisfy a rate constraint and a classification constraint, as formalized in the following definition.
Definition (Constrained Optimal Transport). Let X ~ px denote the degraded source, Y ~ py be the reconstruction, and S ~ ps be the associated classification variable with covariance Cov(X, with S ++ X < Y. Define M (px,Py) as the set of joint distributions PU.X.2.Y with marginals Px, PY that factorize PU.X.Z,Y PU Px P y Py where U is the shared common ran- domness. The constrained optimal transport problem with rate constraint R. classification loss C, and shared randomness is given by
D(R, Cxx.C.)x.PY)- =
inf P... s.t. H(ZU) R, H(SYY)
E[d(X,Y) Y)]
(2)
Given an input X and shared randomness U, the encoder produces a compressed representation Z P Leveraging U, the representation can be further losslessly encoded at an average rate not exceeding R. By standard coding theorems, any discrete Z admits a variable-length code with expected length at most HZZUU) via iy,, where the reconstruction is required to satisfy (S)Y) < C. The optimization in ( is carried out jointly over the distribution of shared randomness pv and the stochastic mappings P (encoder) and l,, (decoder). Furthermore, the constrained cost satisfies D(R, C, py) 2 D(px,Py) where D(px, Py) is the classical optimal transport cost from Definition The next result provides a simplification of this architecture.
bits. The decoder, with access to (Z, U), reconstructs Y
Theorem 1. Define Q(px, )Y)) as the set of joint distributions Pu,X.Y with marginals px. Py that factorize as pU,x,Y = pu Px PYIX,U- Then, the constrained optimal transport cost in Definition admits the representation
D(R, C,Px,Py)=
inf PU.x. E(d(X,V))
(3)
s.t. H(Y = 0, T(X;U)= = 0,
U(Y < R, H(SY)) < C.
3

---

Accepted to ICLR 2026.
Proof. The result follows by adapting Theorem in Liu et al. (2022). For completeness, detailed
proof is provided in Appendix A.I.I.
Following (Liu et al. 2022). the problem can be equivalently expressed using only the conditional distribution Xy) which generates the recon- struction Y directly without an intermediate repre- sentation Z, similar to the classical optimal trans- port formulation in Definition [ The condition HY U)= = ensures that the transport plan deterministic once the shared randomness U is fixed, with U providing the sole source of stochas- ticity. In this architecture, the encoder maps (X, U) to Y (transport), then compresses Y losslessly at an average rate approaching HY)UU) (compression), while enforcing H (S)Y) < C to preserve classifi- cation accuracy. The decoder simply decompresses and outputs Y.
(SS)))
Paue
2
Prizu
Pypo
EEd(X. V
Figure 2: System architecture of Theorem |
2.2 BERNOULLI CASE EXPRESSIONS
We now investigate the constrained optimal transport framework for Bernoulli sources. Let X ~ Bern(qx) and Y Bern(qy) with < gy S Using @ for modulo-2 addition, note that X + Y = iff X + Y. The classification variable S is modeled as S = X 1 S1. where Si Bern(gs,) with 0 S qs, S 2- This yields the marginal distribution qs P(S 1)
qx + 4s, 24x
We adopt the Hamming distortion dx Y) 1fx + Y). For any coupling of X and Y with these marginals, let Pxy = P(X x, Y = y). The expected distortion is
E(d = P(X # Y) P01 P10 = qx + qr 2p11-
Thus, minimizing (resp. maximizing) Pr(X * Y is equivalent to maximizing (resp. minimizing)
P11 subject to the Fréchet-Hoeffding bounds (Nelsen 2006, Sec. 2.5): max(0. 9x + qr -1} I
P11 S min(gx, gr }- The minimum distortion is attained at P11 minfqx. qy realized by the monotone coupling. Specifically, U Unif(o, 1). with X = I/U S (x) and = 1(U S ).
minccuplings P(X + Y) 19x lr The maximum distortion is attained at P11 = max (0 qx + QY 1}, realized by the antimonotone coupling: X IU S qx) and
Then, D (
Y = IU 21 }. Since qx, 4r I 2. we obtain ( maxcouplings P(X + Y) qx + Qy
For the independent coupling, Pı1 = qx yielding D( = P(X Y = qx (1 - qr + (1 qx )gy qx qy 2y Building these extremal couplings and benchmark distortions, next derive the DRC tradeoff under common randomness.
Theorem 2. Consider a Bernoulli source X Bern(gx), Y ~ Bern(qy), and a classification variable S with the binary symmetric joint distribution given by S = X @ Si where S ~ Bern(gs) and S₁ ~ Bern(gs,) (0 S qx 1xx 9s, S 1). The problem (3) is feasible C 2 Holgs,). Assume the Hamming distortion measure. Under common randomness, have
2(1 9x)9x (Holm) + D(
(m) - Hp(gs,)
Holge) CC R(Holas, )- Hm(m))
+ H2(m)
((B) Cx
Hx(gx)
2(1 9x)qxR R + (BB C
R R(H Ho(m))
+ H.(m)
Holgx)
Hblgx)
D ((B) C> Ho(gs) and R > Holgx).
where m = (1) (s,) + 9s qs = qx + 9s, 2qx9s, and Hy(.) denotes the binary entropy function.
Proof. The proof is provided in Appendix A.1.2
4

---

Accepted to ICLR 2026.
Theorem reveals three regimes. With loose classification constraint C, distortion decreases linearly with rate R, approaching the independent coding distortion D In the intermediate regime, distortion depends jointly R and C, capturing the tradeoff between compression effi- ciency and task fidelity. When R exceeds the source entropy and sufficiently large, the min- imal distortion Dui becomes achievable. Similarly, the definition and closed-form expression of (((B) (D, C, (x.xxx)) are provided in Appendix A.2
SYSTEM MODEL AND ASYMPTOTIC SETTING RESULTS
012
3.1 ASYMPTOTIC CONSTRAINED OPTIMAL TRANSPORT
010
Classical rate-distortion theory is usually considered in the asymptotic block-length regime, where arbitrarily long i.id. sequences compressed and coding theorems yield single- letter characterizations. Motivated by this, we now extend our one-shot constrained optimal transport formulation asymptotic case, where large block lengths are allowed. This generalization allows us to connect to Shannon's 's original set- ting and to establish information-theerrtie characterizations that hold in the limit. Let [ { and { be i..d. processes with marginals px, PY, and ps, respectively.
0.09
0.07
the
0.05
R (bits)
Figure 3: D(B) (R, C, qx, qr ver- sus R with C 0.8, qy 0.25, qs, 0.2.
0.3,
Definition (Asymptotic Constrained Optimal Transport). Consider i..d. random variables X; px, Y, ~ PY, and S, ps- The asymptotic constrained optimal transport problem with rate constraint R, classification loss C, and shared randomness U in the asymptotic regime (n - 20) is defined
(o0) CR.CP)
inf PU.X" EM P PPY) 2=
E(d(x, Y))
S.t. H((ZU) S R,
H(S|))) SC
n
Theorem 3. In the asymptotic regime, the DRC function admits the single-letter characterization
(R, Px.xET(Px.PP)
inf [Id(X,Y))
(4)
s.t. I(XYY) S R. H(SIY) S.
Proof. The result follows from Saldi et al. (2015b Theorem 7). combined with the arguments in Wang et al. (2025) Appendix F).
Remark 1. As in the one-shot formulation, shared common randomness U can be leveraged in the
asymptotic regime for the constrained optimal transport problem. However, since coding theorems in the block-length limit already allow randomized mappings without rate penalty, the asymptotic characterization in Theorem 3 coincides with the one-shot formulation, and the role of U does not further tighten the bound.
3.2 GAUSSIAN CASE EXPRESSIONS
We now investigate the constrained optimal transport framework for Gaussian sources under MSE distortion. Let ~ NNxxx and Y ~ (Wy,, ) be Gaussian random variables, and let S ~ A ((us, o3) denote the associated classification variable with 0, A Cov(X, S). In the Gaussian case, derive single-letter characterization of the asymptotic DRC tradeoffs with shared randomness.
Theorem 4. Consider X ~ (px. oxx) and Y ~ NNuy, o&) with MSE distortion, and let ~ N Nlps o3) be classification variable with Cov(X, S) 0,. The problem (4) is feasible if C Z
5

---

Accepted to ICLR 2026.
log 1
Bi 0'
+ h(S). Under shared randomness, the asymptotic DRC tradeoff is
(ux (ux- 1 --y) 0x
+ o
2
- 2h(5)+2C,
81
g
67
+h(5)
C
log
(1(1--22))
+ h(S),
03
03
D(G) Cxx
(ux )1x)² +0x +oy 2xxxx 2-2H
C> 1 log 1111-2
+ h((),
0303
0, C> h(S) and R > h(X).
Proof. A detailed proof is given in Appendix A.1.3
We provide the definition and closed-form expression of (D, in Appendix A.3
3.3 ASYMPTOTIC DRPC FUNCTION FOR GAUSSIAN SOURCES
Classical asymptotic RD analysis yields single-letter charac- terizations in the block-length limit. We extend this perspec- tive to the DRPC setting, where reconstructions are required not only to satisfy fidelity and rate constraints but also to pre- serve classification performance (Wang et al. 2025) and align with perceptual target distribution (Blau & Michaeli 2019
06
D
02
Theis & Wagner 2021).
Definition (Asymptotic DRPC Function). For i.i.d. random variables X; Px, Y; Py, and S, Ps, the DRPC func- tion with common randomness in the asymptotic regime is de- fined as
(bits)
10
Figure 4: D(G) C,qx, Qγ) ver- sus R with C 2, X,Y,S N 1), 01 = 0.6.
inf 1
1 (R,
Eld(X, YJ)
s.l. 1 H((ZU) S RR H(SAY) sC, I (px,, S.P
where o(.,-) is nonnegative divergence capturing perceptual quality.
Theorem 5. In the asymptotic regime, the DRPC function admits the single-letter characterization
( (R, inf E[d(X,Y) PYix I(x:Y) R, H(SYY) ppx.
s.t.
Proof. The result follows the asymptotic analysis in Theis & Wagner 2021): Saldi et al. (2015a) and adapting the arguments in Wang al. (2025) Appendix F).
We investigate two perception divergences of particular interest. The first is the Kullback-
(Y Leibler divergence, defined as pPpx,Py) = (Kz (px) E Elog px (V) The second di-
vergence we consider the squared quadratic Wasserstein distance, defined as Wi(px.PY) =
inf infpxyeT(pxxe)) E[(X-Y)?)
Since the source variables {X,) are i.i.d. and so are the reconstructions (Yi), the divergence term ((px,, PY;) independent of i. Thus, DRPC coding can be viewed as output-constrained source coding, where the reconstruction distribution is restricted the set {py p(px.pr) P). Ac- cordingly, the DRPC function is given by
(00) RR P,C)
inf pr:o(px PSP
(R,

---

Accepted to ICLR 2026.
Unlike ((x)) (R. C.px, py), the reconstruction distribution in PR, is not fixed but only required to satisfy the perceptual constraint @(px. py) I P. Leveraging the jointly Gaussian struc- ture. we obtain closed-form characterizations of the DRPC tradeoff under MSE distortion, subject to both classification and perception constraints. In particular, explicit expressions are derived when the perception measure is chosen as either the KI. divergence or the quadratic Wasserstein distance.
Theorem 6. Let X Nuux, x be a Gaussian source and S ~ NNus, o's) classification variable jointly Gaussian with X, such that Cov(X, = f1. Consider Y with mean F = juy, variance Var(Y) = oy, and covariance Cov(X, Y) Oz- Define YG a Gaussian random variable such that (X, YG) is jointly Gaussian with the same first and second moments as (X, Y): oy, and Cov(X,Yo,
E[Ya] Ny, Var
02. Under the MSE distortion with constraints I();Y) < R, h(S C, and ó(qx. 9r) P. the function D (R, (R, P. C.) is attained by such jointly Gaussian Ya when the perception measure is either W2 (qx;
(2x,, qy) or ΦKL (av (lxx).
Proof. The proof is provided in Appendix A.I.4
Theorem 7. Let X N(ux, x and Y ~ NuJ,, of) be two Gaussian random variables. Let S ~ N (pss, o3) ) be an associated classification variable with a covariance of Cov(X,55 S) = 01 and be jointly Gaussian. For the case ((XY)) = (X -Y)² and opx. (r) = (Kx((rl)px)) have
D
D KL
P.C)
1 log 1- 69(1 1011-20 -
xx - 0x - 2-11-2221).0 ((P) S S0x 2-11--2220 2R 2- and C>
+ h(S)
2
o500
xx + (P) - 2xxx 1
o (P) > ox xxvI - 2-28 22 and C > 1/2 log [
0f 73
+ h(())
200' x (1 2-2 ((P) - x X
x
V1
2 2h(5)+22
8
11
1
of
1
Of(08 -2
and
log
1
+h(S) S C
log
1
+ +SS)
2
20
2
USO 2
xx + 0² (P)
V 1 2-2h(5)+2 (o())
v V1-3 2 2-2h 2h)+++2
0
01
and log (1
1
07
1
log (1- 09(0²
0 2 22)
+ h(s) < C
+ h(S)
2
2
05 x
0, C> h(S) and R > h(X).
where a (P) is the unique number a E [0, satisfying v0(0) P and (0v) log σx or + o7-07 203
Proof. A complete proof is given in Appendix A.1.5
A detailed derivation of the closed-form expression for ((0)) P, P,C) under the quadratic Wasser- stein distance can be found in Appendix A.4
4 RELATED WORKS
Classical rate-distortion theory characterizes the fundamental limits of lossy compression with the best achievable distortion at a given rate (Cover & Thomas 1999), while the information bottleneck
links compression with task relevance (Chechik et al.J 2003). Task-aware extensions, CDP (Liu
et al., 2019a b) and the RDC/RDPC formulations (Zhang) 2023 Wang et al.) (025). make explicit how accuracy constraints reshape the RD function, yet they typically operate within a single domain. Perception-aware RD augments RD tradeoff with a divergence between source and reconstructions [Bluu & Michaeli, 2018: 2019 Theis & Wagner) 2021), inspiring generative codecs based on ad-
versarial learning and Aettiibuttoo--reeevviin objectives (Goodfellow et al. 2014 Arjovsky et al.)
2017 Guirajani et al. 2017: Tschannen et al. 2018 Agustsson et al. 2019 Mentzer et al. 2020).
Modern learned codecs pair maalsis-ssnthhsii transforms with entropy models and tighter rate
timation (Ballé et al. 2017: 2018 Minnen et al., 2018 Theis et al. 2017; Williams et al. 2020
Johnston et al. 2018 Agustsson al. 2017 Mentzer et al. 2018 Wu al. 2020 Alemi et al.
7

---

Accepted to ICLR 2026.
2018 Brekelmans et al. 2019 Huang al. 2020 Park al. 2020). Beyond fidelity, compression
has been used as a denoising prior from asymptotic limits Weissman Ordentlich 2005) to recent neural and zero-shot frameworks with theoretical guarantees Zatari et al. 2025a bb. Optimal trans- port provides a principled way to couple marginals (Villani 2009) and has informed unsupervised restoration (Wang et al. 2023a). Most relevant our work, cross-domain lossy compression has been cast as entropy-constrainee OT with shared randomness that decouples coding and transport (Liu al., 2022). Unsupervised image restoration has also been explored in (Zhang et al. 2017 Menon et al., 2020; Pan et al. (021) with fixed reconstruction distribution, but these approaches neither impose compression constraints nor incorporate classification awaaeneess for downstream tasks. Common randomness and stochastic encoders, realized via universal/ditthred quantization, enable output constraints and "free" synthesis randomness (Saldi al. 2015a 2013 Schuchman 1964 Gray & Stockham 1993 Ziv 1985 Li & El Gamal [00 8a Theis & Agustsson, 2021). Our
work extends this line by introducing rate- and classification crnttiio-oonstaained optimal transport, provid- ing one-shot and asymptotic characterizations with closed-fore tradeoff's, and validating the theory through deep generative compression models.
EXPERIMENTAL RESULTS
5.1 TRAINING SETUP
We consider the setting where the encoder observes degraded samples X Px (e.g., noisy or low-resolution) and the goal is to reconstruct outputs from a distinct target distribution Y pY (e.g., clean high-resolution). The objective is to compress X while ensuring that reconstructions preserve semantic content, align with py, and remain predictive the downstream label S. Note that Y is drawn from the clean dataset distribution, but does not correspond to the exact clean counterpart of X. this unsupervised setting, only unpaired noisy and clean samples are available.
Discriminator
Wasserstein
(4)
x
Encader
Quantizer
Decoder (g)
Clessifler
Clessification
Loss
Lass
Figure 5: Experimental architecture: stochastic autoencoder with classifier and WGAN discrimi- nator, conditioned on shared randomness
Following Liu et al. (2022) Wang et al. 20255) we adopt a stochastic autoencoder consisting of an encoder f. quantizer Q2 decoder g. classifier and WGAN discriminator d. Distortion is measured by MSE, while classification is enforced via cross-entropy loss CE(S, 5), which upper bounds the conditional entropy H (S)Y) (Boudiaf et al.) 2021 Wang et al. 2025). The rate is upper bounded by logg L, where the encoder output dimension and the quantization level. Formally, for target rate R and shared randomness U, the system solves
min E[ILX ((Q((((XX))))))))) F.g.Q
s.t. PoIQU((UU)) = PY, H(QU((XU))) S R, H(SIs(QU((XUU)) SC.
Letting Ý (Q( U)), the WGAN discriminator aligns py with py via Wasserstein-1
penalty Arjovsky et al., 2017). Shared randomness is implemented through universal quantiza- tion (Ziy 1985 Theis Agustsson 202T). With trained encoder f and decoder 9, restoration is obtained Y (Q( U) U) where U is common randomness available both encoder and decoder. In practice we optimize the relaxed loss
E|| + Wi py) A. CE(S,
which balances fidelity, distributional alignment, and classification.
8

---

Accepted to ICLR 2026.
5.2 RESULTS
Figure 6 presents the tradeoff's between rate, distortion, and accuracy. As expected, higher rates yield lower MSE and improved classification performance. We report both quantitative curves and qualitative results for super-resolution (MNIST) and denoising (SVHN). At low rates (e.g.. R = ( reconstructions capture coarse structure but remain blurry, stylized. or even ambigu- ous. At higher rates (c.g., R = 32), both distortion and perceptual quality improve substantially, producing reconstructions that closely match the high-resolution targets. A similar trend holds for denoising, where noisy digits progressively sharpen and become recognizable as the rate increases.
HB
IB
Red
2
0
D
O
Cassification Accuracy
0.510
5
0.43
MSE Distortion LOGS
0
0
f
0
0.93
5
8
5
&
f
S
4
f
"J
5
0.30
0.655
7
7
0
7
0.59
0.050
1 0
/
0
0
b
0
I
0
b
0 b
0.445
5
8
5
5
Upper
Bound (b) MSE rate (MNIST).
9
(a) Accuracy Vs. rate (MNIST).
(c)
Reconstructions (MNIST).
Clssilication Accuracy
0.
455 Distantonn LOCK
0.6
0.026
18:3
0.5
0.024
0.4
0.000
0.
0.528
15 15
18
10
15
25
30
30
25
20
25
32
Rate Upper Bound
Rote Upper Bound
(d) Accuracy vs. rate (SVHN).
(c) MSE vs. rate (SVHN).
(f) Reconstructions (SVHN).
Figure 6: Experimental results: 4x super-resolution on MNIST and denoising on SVHN with Gaus- sian noise with 20. Higher rates yield clearer reconstructions and improved classification per- formance.
Additional results on entropy model-based rate estimation for super-resolution (MNIST) and denois- ing (SVHN, CIFAR-10, ImageNet, KODAK) are provided in Appendix [B We further examine the inpainting problem on SVHN under both supervised and unsupervised settings in Appendix [B22 Together, these experiments enrich the empirical study of the RDC tradeoff and provide additional evidence that the observed behaviors align closely with the thcoretical predictions our framework.
CONCLUSION
We studied cross-domain lossy compression, where the decoder reconstructs samples from target distribution distinct from the degraded source observed by the encoder. By casting the problem as constrained optimal transport, compression rate and classification loss were unified into a sin- gle information--heoreiic framework. In the one-shot regime with shared randomness, the problem reduces to a deterministic transport plan, and we derived closed-form DRC/RDC expressions for Bernoulli sources under Hamming distortion. In the asymptotic regime, analytic DRC/RDC trade- off's were obtained for Gaussian sources under MSE. The framework was further extended to include perception divergences, such as KL and quadratic Wasserstein, leading to closed-form DRPC func- tions. To bridge theory and practice, we implemented deep end-to-end compression frameworks incorporating universal quantization for shared common randomness, entropy modeling for rate estimation, adversarial distribution alignment, and a task-specific classifier. Experiments on super- resolution (MNIST), denoising (SVHN, CIFAR-10, ImageNet, KODAK), and inpainting (SVHN) confirmed that empirical performance closely matches the theoretical predictions.

---

Accepted to ICLR 2026.
ACKNOWLEDGMENT
This work was supported by the National Science Foundation Grant CCF-2417898.
REFERENCES
Abderrahim Abdelhamed, Stephen Lin, and Michael S. Brown. A high-quality denoising dataset for smartphone cameras. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 1692-1700, 2018.
Akshay Agrawal, Robin Verschueren, Steven Diamond, and Stephen Boyd. A rewriting system for convex optimization problems. Journal of Control and Decision, 5(1):42-60, 2018.
Eirikur Agustsson, Fabian Mentzer, Michael Tschannen, Lukas Cavigelli, Radu Timofte, Luca Benini, and Luc Van Gool. Soft-to-hard vector quantization for end-to-end learning compressible representations. pp. 1142-1152, 2017.
Eirikur Agustsson, Michael Tschannen, Fabian Mentzer, Radu Timofte, and Luc Van Gool. Gener- ative adversarial networks for extreme learned image compression. In Proceedings of the IEEE International Conference Computer Vision, pp. 221-231, 2019.
Alexander Alemi, Ben Poole, Ian Fischer, Joshua Dillon, Rif A Saurous, and Kevin Murphy, Fixing a broken elbo. In International Conference on Machine Learning, pp. 159-168, 2018.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International Conference on Machine Learning, pp. 214-223, 2017.
Johannes Ballé, Valero Laparra, and Eero Simoncelli. End-to-end optimized image compression. In International Conference on Learning Representations, 2017.
Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. Variational image compression with a scale hyperprior. In International Conference Learning Represen- tations, 2018.
Joshua Batson and Loie Royer. Noise2self: Blind denoising by self-supervision. In Proceedings of the 36th International Conference Machine Learning (ICMI),, volume 97 of Proceedings of Machine Learning Research, pp. 524-533. PMLR, 2019.
T. Berger and R. Zamir. semi-continuous version of the berger-yeung, problem. IEEE Transactions on Information Theory, 45(5):1520-1522 1999.
Yochai Blau and Tomer Michaeli. The perception- -istoootoon tradeoff. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 0228-6237, 2018.
Yochai Blau and Tomer Michaeli. Rethinking lossy compression: The titee-dittottionereepppe
tradeoff. In International Conference on Machine Learning, pp. 675-685, 2019.
Malik Boudiaf, Jérôme Rony, Imtiaz Masud Ziko, Eric Granger, Marco Pedersoli, Pablo Piantanida, and Ismail Ben Ayed. A unifying mutual information view metric learning: cross-entropy vs.
pairwise losses. arXiv preprint, 2021. arXiv 2003.08983.
Rob Brekelmans, Daniel Moyer, Aram Galstyan, and Greg Ver Steeg. Exact rate-distortion in toencoders via echo noise. In Advances in Neural Information Processing Systems, pp. 3889-
2019.
Tim-Oliver Buchholz, Mangal Prakash, Deborah Schmidt, Alexander Krull, and Florian Jug. De- noiseg: Joint denoising and segmentation. In European Conference on Computer Vision (ECCV), pp. 324-337. Springer, 2020.
Gal Chechik, Amir Globerson, Naftali Tishby, and Yair Weiss. Information bottleneck for gaussian variables. Advances in Neural Information Processing Systems, 16, 2003.
Eastman Kodak Company. Kodak photocd true color image suite (24 high resolution images).
Dataset, 1991.
10

---

Accepted to ICLR 2026.
Thomas M Cover and Joy A Thomas. Elements of information theory. John Wiley & Sons, 1999.
CVX Research, Inc. CVX: Matlab software for disciplined convex programming, version 2.0.
https:// cvxrr com cvx, August 2012.
Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian. Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16 (8):2080-2095, 2007.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-seale hierarchical image database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 248-255, 2009.
Steven Diamond and Stephen Boyd. CVXPY: A optimization. Journal of Machine Learning Research, 17(83)11-5, 2016.
modeling language for convex
Keyan Ding, Kede Ma, Shiqi Wang, and Eero P. Simoncelli. Image quality assessment: Unity- ing structure and texture similarity. In IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 12113-12122, 2020.
Abbas El Gamal and Young-Han Kim. Network Information Theory. Cambridge University Press, 2011.
C.R. Givens and R.M. Shortt. A class of wasserstein metrics for probability. Michigan Mathematical Journal, 31(2):211-44,, 1984.
Ian Goodfellow, Jean Pouget-Abadie, Mchdi Mirza, Bing Xu, David Warde-Farley. Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. volume 27, pp. 2672-2680, 2014.
Michael Grant and Stephen Boyd. Graph implementations for nonsmooth convex programs. In V. Blondel, S. Boyd, and H. Kimura (eds.), Recent Advances in Learning and Control, Lecture Notes in Control and Information Sciences, pp. 95-110. Springer, 2008.
Robert M Gray and Thomas G Stockham. Dithered quantizers. IEEE Transactions on Information Theory, 39(3):005-811 812, 1993.
Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Im- proved training of wasserstein gans. In Advances in neural information processing systems, pp- 5767-5777, 2017.
Sicong Huang, Alireza Makhzani, Yanshuai Cao, and Roger Grosse. Evaluating lossy compression rates of deep generative models. In International Conference on Machine Learning, pp. 4444- 4454, 2020.
Satoshi lizuka, Edgar Simo-Serra, and Hiroshi Ishikawa. Globally and locally consistent image completion. ACM Transactions on Graphics (TOG), 36(4):10711 2017. Proceedings of SIGGRAPH 2017.
Nick Johnston, Damien Vincent, David Minnen, Michele Covell, Saurabh Singh, Troy Chinen, Sung Jin Hwang, Joel Shor. and George Toderici. Improved lossy image compression with prim- ing and spatially adaptive bit rates for recurrent networks. Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 4385-4393, 2018.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1412v:1412.6900 2014.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Alexander Krull, Tim-Oliver Buchholz, and Florian Jug. Noise2void learning denoising from single noisy images. Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2137-2137. IEEF, 2019.

---

Accepted to ICLR 2026.
Samuli Laine, Tero Karras, Jaakko Lehtinen, and Timo Aila. High-quality self-supervised deep image denoising. In Advances in Neural Information Processing Systems (NeurIPS), volume 32, pp. 1970-6980. Curran Associates, Inc., 2019.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11)) 1998.
Cheuk Ting Li and Abbas El Gamal. Strong functional representation lemma and applications to coding theorems. IEEE Transactions on Information Theory, 64(11):6967-6978 2018a.
Cheuk Ting Li and Abbas El Gamal. Strong functional representation lemma and applications to coding theorems. IEEE Transactions on Information Theory, 64(11):6967-6978, 20188
Dong Liu, Haochen Zhang, and Zhiwei Xiong. On the cuissfffttioo-ddistottoo-prprree In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Bue E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019a.
tradeoff.
Dong Liu, Haochen Zhang, and Zhiwei Xiong. On the clasiffcation-diioorioo--rreeee In Advances Neural Information Processing Systems, pp. 1206-1215, 2019b.
tradeoff.
Huan Liu, George Zhang, Jun Chen, and Ashish Khisti. Cross-domain lossy compression as entropy constrained optimal transport. IEEE Journal Selected Areas in Information Theory, 3(3):513- 527, 2022.
Chao Ma, Chih-Hung Yang. Xiaokang Yang, and David Wipf. Learning a no-reference quality metric for single-image super-resolution. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 666-675, 2017.
Sachit Menon, Alexandru Damian, Shijia Hu, Nikhil Ravi, and Cynthia Rudin. Pulse: Self- supervised photo upsampling via latent space exploration of generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2437-2445.
IEEE, 2020.
Fabian Mentzer, Eirikur Agustsson, Michael Tschannen, Radu Timofte, and Luc Van Gool. Condi- tional probability models for deep image compression. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.
2018.
Fabian Mentzer, George Toderici, Michael Tschannen, and Eirikur Agustsson. High-fidelity gen- erative image compression. In Advances in Neural Information Processing Systems, volume 33, 2020.
David Minnen, Johannes Ballé, and George Toderici. Joint autoregressive and hierarchical priors for leamned image compression. In Advances in Neural Information Processing Systems, pp- 10771-10780, 2018
Roger B. Nelsen. An Introduction to Copulas. Springer, 2nd edition, 2006. Fréchet-Hoeffdine bounds and comonotone/atimonoton couplings (Sec. 2.5).
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learning. In Proceedings of the 24th Interna- tional Conference on Neural Information Processing Systems (NIPS) Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
Xingang Pan, Xiaohang Zhan, Bo Dai, Dahua Lin, Chen Change Loy, and Ping Luo. Exploiting deep generative prior for versatile image restoration and manipulation. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 14(12):870088725) 2021.
Seonho Park, George Adosogiou, and Panos M Pardalos. Interpreting rate-distortion of vari- ational autoencoder and using model uncertainty for anomaly detection. arXiv preprint arXiv:2005.01889, 2020.
Deepak Pathak, Philipp Krähenbühl, Jeff Donahue, Trevor Darrell, and Alexei A. Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2536-2544 IEEE Computer Society, 2016.
12

---

Accepted to ICLR 2026.
J. Rissanen and G. Langdon. Universal modeling and coding. IEEE Transactions Information Theory, 27(1):12-23, 1981.
Naci Saldi, Tamás Linder, and Serdar Yüksel. Randomized quantization and optimal design with a marginal constraint. In 2013 IEEE International Symposium Information Theory, pp. 2349- 2353. IEEE, 2013.
Naci Saldi, Tamás Linder, and Serdar Yüksel. Output constrained lossy source coding with limited common randomness. IEEE Transactions on Information Theory, 61(9):4984-4998, 2015a.
Naci Saldi, Serdar Yüksel, and Tamás Linder. Finite-state controllers for partially observable multi- agent systems. SIAM Journal on Control and Optimization, 5365):2765-201, 2015b.
Leonard Schuchman. Dither signals and their effect on quantization noise. IEEE Transactions on Communication Technology, 12(4):122-165, 1964.
David S. Taubman, Michael W. Marcellin, and Majid Rabbani. Jpeg2000: Image compression fundamentals, standards and practice. Journal of Electronic Imaging. 11(2):286-28) 2002.
Lucas Theis and Eirikur Agustsson. On the advantages of stochastic encoders. arXiv preprint
(xXiv:2022009270) 2021.
Lucas Theis and Aaron Wagner. A coding theorem for the wel-distortionperreppoo function. arXiv preprint urXiv:2104. 13662, 2021.
Lucas Theis, Wenzhe Shi, Andrew Cunningham, and Ferenc Huszár. Lossy image compression with compressive autoencoders. In International Conference on Learning Representations, 2017.
Michael Tschannen, Eirikur Agustsson, and Mario Lucic. Deep generative models for distribution- preserving lossy compression. In Advances in Neural Information Processing Systems, pp. 5929- 5940, 2018
Cédric Villani. Optimal Transport: Old and New, volume 338. Springer, 2009.
Wei Wang, Fei Wen, Zeyu Yan, and Peilin Liu. Optimal transport for unsupervised denoising learn- ing. IEEE Transactions Pattern Analysis and Machine Intelligence, 45(2):2104-2) 111-114-11111 2118, 2023a.
Wei Wang, Feng Wen, Zhen Yan, and Peng Liu. Optimal transport for unsupervised denoising learning. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 45(2):2104 2118, 2023b.
Yuhan Wang, Youlong Wu, Shuai Ma, and Ying-Jun Angela Zhang. Task-oriented lossy compres- sion with data, perception, and classification constraints. IEEE Journal on Selected Areas in Communications, 43(7):2635-2650, 2025.
Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600- 612, 2004.
Tsachy Weissman and Erik Ordentlich. The empirical distribution rate-constrained source codes. IEEE Transactions Information Theory, 2005.
Will Williams, Sam Ringer, Tom Ash, John Hughes, David MacLeod, and Jamie Dougherty. Hier- archical quantized autoencoders. arXiv preprint 20Xiv:2002.081 11, 2020.
Alan S. Willsky and Gregory Wornell. 6.432 lecture notes. https: ht.ppp://www... wwwp www.wrle..m. /4/ chapter3 pdf 2005.
Lirong Wu, Kejie Huang, and Haibin Shen. A gan-based tunable image compression system. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2334- 2342, 2020.
Li Xie, Liangyan Li, Jun Chen, and Zhongshan Zhang. Output--onstrainet lossy source coding with application to rece-distottionperrppttee theory. IEEE Transactions on Communications, 73(3): 1801-1815, 2025.
13

---

Accepted to ICLR 2026.
Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S. Huang. Generative image inpainting with contextual attention. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5505-5514. IEEE Computer Society, 2018.
Jiahui Yu, Shengyang Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S. Huang. Free-form im- age inpainting with gated convolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4471-4480. IEEE, 2019.
Ali Zafari, Xi Chen, and Shirin Jalali, Decompress: Denoising via neural compression. 2025 IEEE International Symposium Information Theory (ISIT), pp. 1-6, 2025a.
Ali Zafari, Xi Chen, and Shirin Jalali. Zero-shot denoising via neural compression: Theoretical and algorithmic framework. In Proceedings of the 2025 Conference on Neural Information Processing Systems (NeurIPS), 2025b.
George Zhang, Jingjing Qian, Jun Chen, and Ashish Khisti. Universal rute-distortionnperrcppioo representations for lossy compression. IEEE Transactions on Information Theory, pp. 1-1, 2025.
Kai Zhang, Wangmeng Zuo, Shuhang Gu, and Lei Zhang. Learning deep CNN denoiser prior for image restoration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1929-39388 IEEE, 2017.
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 586-595, 2018.
Yuefeng Zhang- A pee-ddiisortiooncaasffitttttt approach for lossy image compression. Digital Signal Processing, 141:104163, September 2023. ISSN 1051-2004.
Jacob Ziv. On universal quantization. IEEE Transactions Information Theory, 31(3):344-347, 1985.
14

---

Accepted to ICLR 2026.
Supplementary Material of "Cross-Domain Lossy Compression via Rate- and Classfication-Cnnttaainee Optimal Transport"
A THEORETICAL RESULTS
A.1 PROOFS OF THEORETICAL RESULTS IN THE MAIN PAPER
A.1.1 PROOF OF THEOREM
Theorem I Define QPx.Py) as the set joint distributions PU.X.Y with marginals px, PY that factorize as PU.X,Y = Px PY- Then, the constrained optimal transport cost in Definition admits the representation
inf PXx,)= PUX..
E(d(X,Y))
s.t. XX) 0, I(XUU) = =0, U R, H(S s C.
Proof. Recall the formulation from Definition 2
C(x. =
inf E s.t. H(ZUU) RR H(S|Y) S.
Fld(x, Y)]
where M(px,Py) = (PUXXYY PUXXZY = PU px PZIX,U PYIZ.U PrIZ.)) {pU,x,: : PU,XY- = PU PX PYJU,X H(YJU, X) = =0}-
Let Q(xxPP)
Upper bound. Fix any PU,XY E Q(xx.PP) PP) that satisfies HU) S Radd H(S)Y) S C, and set & Y. Then PU, = PU px Ōz=Y(X.U) ōx=z E MMxx, with H(Z = H(Y(U) I R and the same [[d(X,)) and H(S)Y). Hence
D(R,C,Px,PM S inf
Eld(X,Y))
P s.t. H(YUU,)) = O, T(X;U)= 0, H(YU)) < R. H(SIY) SC
Tightness. Take any feasible PU,X.Z,Y M(xx py) with H(Z U)- < and H(S S C. By the functional representation lemma (El Gamal & Kim 2011 Li & El Gamal 2018b), there exist: (i a random seed Vi independent of XU, and a measurable Φ1 such that
Φi (U, X, Vi) in distribution (for P ); (ii) random seed V/2 independent of (U, X, V2) and measurable d such that Y = (U, Z, V2) in distribution (for PY]Z,U).
Let U" (U, Vi, V2). Then, Y
d2, di (U, V2) is deterministic given (U", X), U X) 0 and (U', X,Y) E Q(px, pv). The marginal (X,Y) is preserved, hence Eld(X, Y)] and HSS are unchanged.
For the rate term, conditioning reduces entropy, and determinism gives
H(ZIU) 2 H(Z 1/2 H(Z(U') 2 H(YUU))
and H(YIU") S R. Therefore,
D(R,C.px.PY) N inf
Ew(xxe Eld(X, Y)]
s.t. H(YYU", x)= 0, T(X:U')== =0,
H(YIU") < R. H(S SC
Since the auxiliary alphabet is unrestricted, we can relabel U' as U inside the infimum. Combining the two bounds, the proof is completed.
15

---

Accepted to ICLR 2026.
A.1.2 PROOF OF THEOREM 2
Theorem 2 Consider Bernoulli source X Bern(gx), Y Bern(qy) and classification
variable 5 with the binary symmetric joint distribution given by S X @ Si where 5
Bern(gs) and S1 Bern(gs,) (0 S qx, 4s, S 1)) The problem
feasible if C' Z
Ho(gs.) Assume the Hamming distortion measure. Under common randomness, we have
2(1 9x)9x (x)) (m)mmmmm)) + D
(m H1(gs,)
Ho(gat SCS R(Holas,) )- Hm(mm) + Ho(m)
(R, cx.x,v))
Hslax)
2(1 9x)9xR R + ind
D C> R(Holls,)) Hu(m))
+ Hb(m)
Ho(gx)
Holax)
D D
C> Hy(gs) and R > Holax).
where m (1 q1) qs,) + qs qs qx 9s, 24xqs, Holgs) Hy(m), and Ho(.) denotes the binary entropy function.
Proof. Since HYYUU) = T(X + X) T(XYYU) and HY|UU = 0. we can cquivalently view Y as deterministic function of (X, i... Y = f(x U). The problem (3). under the Hamming distortion measure, reduces to finding a distribution pv such that
((B) (R, C, 1x.x.90 .r. = inf PU s.t. XY 0, I(;UU) 0, T(XYYU) < R H(S S C.
Because Shannon entropy is defined only for discrete random variables, the auxiliary variable U must be chosen such that YUU
discrete for each u. even in continuous (X, Y) settings (Liu al. 2022). Here, py supported on 21 ê 21, ..... | and {fu u E U} denotes the set of all distinct mappings fa X' + V. By the support lemma (Appendix C. 631 Ef Gamal & Kim 2011]. suffices to assign positive probability to at most [}]| 1 such functions.
The optimization can be expressed as follows
()B) (R, C,9x, qx) = min Σ (w) P(X # YUU = u)
uEN
Σ (u) I(X; = u) S R.
s.t.
WEW
Σ (u) P(f = v) = qy, Vy E y,
WEW
Σ (u) H(S),. SC wEW
Without loss of optimality, the size of the alphabet of U be restricted at most four. There
exactly four distinct mappings from (0, 1} to (0. 1): (x)) = x, (2 = 1- - x, fs(x) = 0, and
fa(2) = 1 with z E {0, 1}. Hence, Y X if 1: 1= = 4. Therefore, we obtain
x if 2: if and
P(X xY) = Σ (u) # Y = u) = p2 + (3) + +11 - (4),
NEW
I(X:YIU) Σ Pu (u) = u)= 100=>nn (f) = (o (1) + (2)),
wel
well
P(Y = 1) = qy
Σ (x) = (1) + (1 - qx (x) + pu
weu
By the data-processing inequality (Cover & Thomas 1999) and the Markov relation S X ++Y,
H(SlY) 2 H(S(X) H(X Si H($)) Ho(gs,).
16

---

Accepted to ICLR 2026.
The classification constraint is feasible only if C 2 (S:).
The evaluation of H(S))) for each mapping is derived as follows
For U 1: (S) U 1) (( Ho
For = 2: ( = 2) = (() = Hn(gs)).
For U = 3: P(S 0)= (1-2x)(1 -Qs,) + 4x481
+ HS = = H(SU = = Ho((1 91)(1 9s,) + qx (s;).).
For = 4: H(S =4) HSSU =4) Ho((1 9x)(1 -9s.) + 9x9s.)
Hence,
H(S|Y) Σ pu( (X)) weM
(1v(11) + pu (2)) (os,)) + (pv(3) + po (4)) 1x -(s)) + qxus))
Let m (1 111 (s,) + 4s, we have H(S (po (1) pv (o) (pr(3) pv ()pp
The final optimization problem is represented as ( (R, C,xx
min (o()), po (2), pz), (2) + (3P + -xx)p(44)
(5)
s.t. Holux) (pr (1) +c) < R
(6)
(xPo(1)++0 (1- (2) + Pu (4) = qr.
(7)
+ (p (2 + (pu (3) + (4) S SC,
(8)
pc((1) + pu + (3) + pp((4) =1 (r),), (2), po (3), pu (4) 2 0.
(9)
(10)
The activity of the nonnegative constraints. From 9 and 77. we have
(3(3) = (1) + pu (2) - po (4) and (4) = qy (xPu(1) (1 qx )ru (2).
The objective is: E(dn((X Y) = D 2(1 xx)xxxu (1) + 2(1 xxx Pt (2).
For 0 Σ qx I 2. the distortion E(d (X, y)j is strictly increasing in pt: (2) and decreasing in pu (1);
hence optimal to take pe (2) and pc: (1) > O. We can also write
po (4) qy 1 9x (P
(2)] (2gx (u))),
Pu (3) = (1 .r) - (1 [Pu (1) + pa (2)] (2qx 1) (2).
0, it follows that With pu
pu (4) qy -xPu (1) 0 = po - Y
9x
Qy
pu(3) (3) = 1 qr - (1 -1- qx (1) 2 0 + < (1) < 1- gx
We state a supporting fact that will be used repeatedly in the arguments as follows. Lemma A.1. We have,
(m(m) 2 Ho(as)).
with equality if only ix [0,1) gs.
Proof of Lemma
A. The identity for m is immediate:
m (1 qx (1 qs.) + qxqs, = = +9x 1) (2qs, - -1).
m 1 = (qx 1)) (2gs, - 1) + m = 9x - 111 19s. - 1 < ggs,
since 1ix 1 S for ax E [0, 1]. The binary entropy is maximized at and strictly decreases
with 1: 2. hence Ho(m) > Ho(gs,), with equality if only if [gx 1 (i.e., qx E 1)) or
lgs, I -0 (i.e., 4s, )
17

---

Accepted to ICLR 2026.
We will invoke Lemma A.1 in the analysis below to justify the sign of denominators of the form Ho(m) - (9s. ). Note that,
(2
9x - (4
11p -3) 9x).
Combining constraints and 9
obtain (3) (x11-x0)pu00 qxPu
qy
Since problem (5) is linear program, it can be solved efficiently using standard convex optimization tools such as CVX in MATLAB (CVX Research, Inc., [2012] Grant & Boyd 2008) CVXPY in PYTHON (Diamond & Boyd 2016: Agrawal et al. 2018). Alternatively, an analytical solution can be derived via the Karush- Kuhn-Tucker Tucker (KKT) conditions. In practice, approach systematically
explores all possible combinations of active and inactive rate and classification constraints to fully characterize the optimal solution.
Case 1. Constraint is active and constraint X is inactive.
Using pu (2) 2 0 and the fact that 66 is active, obtain
R R (1) 2 ( Ho(ax)
Moreover,
pu +3)+ (x)) -2(1 xx]ax (PU (1)+ 9x JaxPo (2) +
2-2(1 xx)x (Pu (P) (2)
-2(1 9x 9x) R
(B)
Hol9x)
Thus, (R, (R, -2
Ho(ox)
This lower bound is tight, achieved by
(1) = H
R
(2) =
(3) = -(1-ax J
+ gr,
(4) = -9x +
Holux)
Holux)
Constraint (8) is inactive if
(pu (1) + pu (2)) + (p) (4)) < C,
C> R(Ho(gs,) (m)) + H,(m).
Holgx)
Case 2. Constraint g is active and constraint (6) is inactive.
The constraint is active if
(1v(1) + pu (p)p =0
From the constraint
9.. we have
(pv(1) (2) -1- (2))) =C,
C. (s(m)
pu (1) po (2) Holgs,) Hs(m)
Since Pu (2) 2 O, which implies
EX, Y)] = 22(1 - xx (po (1) + pu (2)) + + 4x)xxPu (2)+ + D
-2(1 9x )ax()xx(ppx Ho(m)) + ((
Ho(gs, Ho(m)
This lower bound is tight, achieved by
pú (1) = Holgs, (m))
(m)
(2) = 0,
(3) -(1-ax 100 (m))) qY,
Pt((4) -4x (C- (m)))
H,(gs; (m))
H.(gs, H, qy-
18

---

Accepted to ICLR 2026.
The constraint 6 is inactive if
R(Ho(gs,) - Ho(m))
(I pr R C'
+ Ho(m))
Holgx)
Case 3. Both constraints (6 and 8 are active.
From case 2, the classification constraint 8 is active if (1(() + po (2 Ha(gs; (m) and D(B) C, gx, qr = -2(1-9x (1---xxxxxCC-H---- (C-Hs(m)) + D ind
C-Hu(m)
Holns, (
The rate constraint (6) is active if R(Ho(ge,) )- H.(m)) Ho
Hslax)
Case 4. Both constraints (6) and ( are inactive. When > Hb(gs), implying that the classification constraint (8) is inactive, and the rate R is sufficiently large such that R Holgx), meaning the
rate constraint (6) is also inactive, the minimum achievable distortion ((B)(R C, reaches its theoretical lower bound, i..,, (B) (R, Cx (
In summary, combining all of the cases, the closed-form expression for D Cx under Hamming distortion is given by Theorem
A.1.3 PROOF OF THEOREM
Theorem 4 Consider X ~ N(xx xx and Y ~ NNuy, of) with MSE distortion, and let S ~ (us, o be a classification variable with Cov(X,5 S) = 0,. The problem 4 is feasible
+h(5). Under shared randomness, the asymptotic DRC tradeoff is ifC 2 2 0303
log
89
(ux 2y)² + xx
+ +oy
20 oy
2h15)+2C
9,
log 1
8
+ h(S) S C log
h(S),
o30
0303
(R, (v)
(ux ux)² + + o. -2xxx 2-22
0f 2 2R)
C > y log 1
h(5),
0307
0, C h(S) and R h()).
Proof. We now consider problem 4 with the MSE distortion criterion. Formally,
(R,) (.
inf
E((X - [
ex.x<T(gx.av) s.t. I(x:Y) < R, M(S C.
Using the closed-form expression for the mutual information Gaussian variables (Cover & Thomas 1999), obtain
-2 log (1
1
0
I(X:Y) 2 20
(11)
where 02 = Co, E((x - px)(Y ). The equality holds if and only if X and Y are jointly Gaussian random variables.
Since (X,Y, S) are jointly Gaussian and satisfy the Markov chain + X + Y, we have Cov(S, Y = E E[S - xs| E[Y - pr| = 9.62, For the classification constraint,
h(S h(S) - I(S;Y) SC - log ( 1
0 ogo
02
2 h(S) C.
oy
The MSE between X and Y admits the decomposition (Zhang et al. 2025
F((x y) (ux =xr)² 0x of -00--
(12)
19

---

Accepted to ICLR 2026.
Hence, the optimization problem can be formulated as
(R,) C,9x, = =min
(ax - -ur) + 0x + oy 200
(13a)
02
03
s.t.
log
1
S R,
(13b)
2
0
0g
log
1
2 21(5)
C.
(13c)
2
0
2
4
X
0
2
To ensure (3bb is well-defined, it is necessary that condition, the mutual information between S and Y satisties
8
7
0
xx
Under this
1 I(S:Y) log
02
02
0
x
-
log
1
oy
2
Thus, constraint (13c is infcasible whenever C' <
2 log
+ h(S). To guarantee feasi-
03
bility, we assume throughout that C 2 2 log
1
0003 + h(S).
The optimization problem [13 can then be analyzed using the KKT conditions. By systematically considering all possible combinations of active and inactive constraints, we can fully characterize optimal solution.
Case 1. Constraint
(3b) is active, while constraint 13ec is inactive.
From the entropy inequality and the fact that (1bb) holds with equality, we obtain
I(X;Y) = h(X) M(Y) 1(X,))))))))))))
2 log log( (xT log(2 h(X,Y)
oz
22 1og(2 + 2 1og(2 log (2 ) = 2 log
oz
08
The equality holds if only if X and Y are two two jointly Gaussian random variables. This implies that
02 oxox oxox 11-2-21
(14)
Substituting (14 into 12) yields
D ( (R, C, 2 (ux - uv) ox + +oy -2oxoy V 2-22
The classification constraint is inactive if 1 log (1-
87 o oz
87
> ((S) - C, which reduces to
C> log
[1
(-22) + h(S).
Case 2. Constraint (3b] is inactive, while constraint
(13c is active.
When 13c) is tight, we have
1
07 0 h(S) C C 02 oso
log 1
- 2-2)))())222
(15)
a o o
01
Substituting 15 into the distortion expression gives
20sox
E((X Y) = (μxx 2x) + + + +o
V1 111-2220 2-2
01
The corresponding mutual information is ((X;)) - - - 1
0303 07
(1 2 2-20(9)+20)
Thus, the rate constraint is inactive whenever
1
0z00 x (1 1-2 2-22())+20)
1
(1
07 -2 -2-2R)
log
1
R C < 2 log
+ h(()).
2
0f
20

---

Accepted to ICLR 2026.
Case 3. Both constraints (3bb) and (33c) are active.
From Case 2, when the classification constraint is tight, we have 02 This yields the distortion
2 2h(3)+2C.
01
20
E[(x - -) (ux 2x) + ox + to
21-2226(8)22
-
01
The rate constraint holds with equality: 1(X; log
log
(1 -1- 2 2-22(1)420)
030
R.
8,
log (1
01 + h(S).
which implies
2
030
Case 4. Both constraints (3bb) and 13c are inactive.
When > (()), the classification constraint (I3c) is inactive, and if the rate satisfies > h(X), the rate constraint (T3b) is also inactive. this regime, the minimum distortion D(C) C. C.qx, achieves its theoretical bound of zero, realized by setting Y = X, which yields F((X - -)4 =0
All constraints are satisfied since ((;Y)) ) = h(X)= Rt and h(S()) = h(SIX) & h(S) <C
In summary, combining all of the cases, the closed-form formm expression for ( C,4x,qy) under MSE distortion is given by Theorem
A.1.4 PROOF OF THEOREM 6
Theorem 6 Let X ~ ~ (ux. xxx be Gaussian source and S ~ NNs. classification variable jointly Gaussian with X, such that Cov( X, X,S 5) = 0G. Consider Y with mean E μy, variance Var(Y) = =o², and covariance Cov(X,Y Y) = 0z. Define YG a Gaussian random variable such that (X, Y.) is jointly Gaussian with the same first and second moments as (X,Y) E[Ya] py, Var(YG) with constraints (X; < R, h(S)Y) < C, and ólax. qy) S P. the function ((x)) PR, is attained by such jointly Gaussian YG when the perception measure is either W (gx or @KI(qY )l
orz and Cov (X, Ya) 02. Under the MSE distortion
Proof: Consider the DRPC problem in Theorem J with MSE distortion as follows.
( (R, = inf E[(X- - Y drix
s.t. T(X; $ R M(SIV) S
<xxr,4x) P.
Distortion objective equality. For any Y. E[([ - -Y) = (ux - My )2 +0x + o -200, depends only first and second moments. Since Y and YG share (NY, or, 02), then E((X 1-xX-Y1 2 E[(X- 2.
200,. which
Rate constraint under Gaussian. We begin with lemma from estimation theory that compares the performance of Gaussian and non-Gaussian estimators with matched second-order statistics.
Lemma A.2. Willsky & Wornell 2005) Let Y be random variable with mean E[Y] μy, variance Var(Y) oy, and covariance Cov(X,Y) 02. Let YG be jointly Gaussian with X and share the same mean, variance, and covariance Then.
E[(X [[X] [( - E[X[X])}].
This result implics that the MMSE general (possibly non- Gaussian) estimator Y is always less than equal to that of Gaussian estimator with the same first and second-order moments.
21

---

Accepted to ICLR 2026.
Following the derivation approach in (Zhang et al. 2025 , show that the mutual information
((XY)) is minimized when Y is constrained to be jointly Gaussian with X. Specifically, we have:
1(x = h(X) ((X
2 h(X) h(X E(XIY])
(0) h(X) log(2ne E((X E(X(())))
h(X) - 1/ log (2 E E((I)))))
= 4(X) 1(X E(X(Va])
9 MXX h(XIYa)
Y();
where inequality (a) follows from the fact that the Gaussian distribution maximizes differential en- tropy for given variance; inequality (b) follows from Lemma A.2 and equality (c) holds because the estimation error independent of Yg. Hence, if T(X: Y) S R then T(X; YG) S R.
Perception divergence under Gaussian. From Xie et al. 2025), have the following proposition
Proposition 1. Xie et al. (2025) For X N(xx, xxx) and any distribution qr with E[Y < cc, we obtain
@xz(arllax) 2 ) (| ([xxxx))))))) llog ox (ux -uy) + of ox
σy
20
Therefore, byr 2 (KI(W
= OKI
Similarly, for the case of W2 ). we have
Proposition 2. Givens & Shorit 1984 proved that for distributions px and pv with E[X²] < oo and [[ 2 < co,
(2() 2
= -
- -1xx-ox)²
Note that by expanding out W2(gx, 9y), one can see that the optimal coupling is identified only through the cross-term between X and Y; since every coupling of qx and qy induces a Gaussian coupling of qx and Qy with the same covariance, it follows that W (ax (gx,9y) Z W2 (ya). Hence, if d(gy, 9x) P. then p(gya. qx) P.
Classification constraint under Gaussian. We now demonstrate that the jointly Gaussian estimator YG for X optimal. To formalize this result and establish the optimality of YG. we first propose the following lemma.
Lemma A.3. Given S + X + Y, we have M(SY) > ((S)Ya).
oxds denote the correlation coefficient between X and S. Since X and S are jointly Gaussian, can be expressed as S = ax + +N, where a = -xs ox and N is zero-mean Gaussian random variable with variance (1 -1pxs ) independent of X.
Proof of Lemma A.3 Let Pxs = Cov(X.S) 8,
dy
Applying the conditional entropy-power power inequality (Berger Zamir 1999), we obtain:
M(SIY) h(ax + NIY) 2 log (22h(aXIY) 22h())
with equality if and only if (X, Y) are jointly Gaussian. follows that h(S)Y) 2 h(S)Ya) log (22h(aX)Y6) 22M(N)), which completes the proof.
The covariance of (S, is fixed by (oz. of. (S af2). Among all (S,Y) with given covariance matrix, the Gaussian joint maximizes I(S;Y); equivalently, minimizes h(S))). Thus, if h(S)Y) & C then also ((S) s C.
22

---

Accepted to ICLR 2026.
Overall, it suffices to solve the following optimization problem:
(R, P)C)
inf E[(X - Ya) Praix s.t.
T(X;Y)) - R h(S)Ya) CC, @ly 9x) P.
Therefore, the DRPC function, D( P)C), achieved by Yo with the cases of é(xx, qY) VVx,,, (v) or (y) = ΦKL(DV,
A.1.5 PROOF OF THEOREM
Theorem 7 Let X ~ N(xx, xx) and Y NNuY, [ be two Gaussian random vari- ables. Let 5 N(us, be an associated classification variable with covariance
of C))(X, = 01 and be jointly Gaussian. For the case d(X,Y) (X -Y and o(px,Py) = (py (pxx), we have
D (R,,
of - xx - 2 2R) ), o()) & ox 1 and C log 1- (1(1-222R) + h(S)
2
ox O -20 -2xx0 2 2-2
(PP) oxv1 2 2- and C> log 1 09 28
h(())
03
Loot X
(1 - 22-2h(5)+26 (P)) 050 X
V1 1 2-2h(5)+2C
x
0
01
07
09 - - -
and
log
1-
+h(S) S CO
log
1 1
+ +
2
x
0
20 ((P) P)
050X 2
x + 0² (P)
V 2- 2h(5)+20
(o(P)
v 2 2h(5)+20
01
89
89(07
2x22-28
and
2 log
1
+ +h(() S SCS log
1
+ M(S)
T
2 4 X
0, C> h(S) and R
> h(X).
where (P) being the unique number a E xx) satisfying (b)) P and (b(o)) = llog ax +
ax
o2-0
2
Proof. Extending from the result of Xie et al. (2025), we establish the following lemma.
Lemma A.4. Consider X ~ Nuxx, with MSE distortion di (X,Y) = (X (X-Y)² and perception
measure ((px,Py) = DKL (px). Then
(G)) (R..). = inf (00) (R. C.9.
s.t. bY = px, oY oY Sxx, MSSYY ΦkI(grllax) S.
Proof of Lemmu A.4 We argue that restricting to py with ay S ox incurs no loss of optimality. Suppose σy ax and define Y' = 2x nY) + px.
Distortion objective. A direct calculation gives
20x E((X xx)(Y - uY)]
E[(X - Y') = 20
or < + 0 - 2E - xx)(Y pyy) E[(X - Y)
where the inequality holds since k22 0z - 2k E[(X - (X - -yy)] is increasing E 12,
Rate constraint. Since + Z + is a Markov chain, then so X + Z ++ Y', and we have T(Y'; 7) = 7)
23

---

Accepted to ICLR 2026.
Classification constraint. Let 0 Y and 02 Cov(X, Y). Then
0f
2
MSSI" = 1(5) + 2 log
030
2
62
M(S|Y) = h(S) + log
0 4
X
oy
2
Since ; : ex (Y- + pxx. then M.y = E oy
ax (Y My) + px
oy (ny
- = px-
or
xx (or) or = xx and 8 ax 02. Therefore,
2
We also have σy
h(S = h(S) I(S:
07 8²2
h(()) + log
2
0 x oy
(0)
89 0g
$ M(S) + log
2
dzot oz
h(8) 1 (5; Y) \(S
022 dy oy
6
where (a) is due to σy > ox, infer that
2
Perception constraint. Finally,
σkI(ςr)llxx) -1(Y') ] log( 1og(2Tox}) + (ux- - 22x:)2++0 o
1
2
20x
S <MV + 2 log 1 + (ux -1y))2 + oz
1
(KI( = (xrl(11))))
203
o ái- ay 2n
where the inequality follows from the convexity of (lay) == log ax tive for 2 ox
which nonnega-
Together, these arguments establish that replacing Y by Y' cannot increase distortion, rate, and perception divergence, nor violate the classification constraint. This proves the lemma.
From Proposition 1 together with the constraints jy = μxx ay I xx, and Φ ( llpx) < P, follows that σy E (P()), where a (P) is uniquely defined as the value E [0, satisfying (b)))= P.
In view of Lemma A.4 it suffices restrict to distributions py with py = px and σy S ox when evaluating (H,, Moreover, Theorem 6 allows us to further assume that Y is Gaussian, yielding
D DRL
( =
min ex)
DIG) C,4xx.9)))))
(16)
Case 1. If 1 log
1og((1 1- 01(1-22211) 0303
1)
+ h(S), then
G (R, C, Cx gv) = (ux 2y) + + + of 20x V1 2-22R
The term of - 2y v1 2- 2R decreases monotonically over oy E 100xx11-22200 and increases thereafter. Thus, the minimizing ay in (16) is
loxvI-2-22 2n
o(P) < 0x xx 1 o(P) Tx 2n
σy (((P).
Hence,
(R, = ( 03 - -2 2-1 10 (P)2 - x (P)V a(P) > σxvl - 22-2R
a(P) < ox <xx x0x 2-22
24

---

Accepted to ICLR 2026.
Case 2. If log 1
0f
+ h(S) S SC log 11 Bi00
- 2-22)
+ h((S), then
2
0gax
03"x x
D (R,
= (ux my)² + - 0z
2000 ay
1 - 2--2-2h()+220 (h)
91
Here, og 2o -1-2 2h{+222 decreases over oy E
0,
v1 2h(5)+20
and
8,
81
increases thereafter. Thus,
0308 ví - 2 2h(8)420
c(P) v 080
v - 2 2h(5)+22
01
81
(PP),
o(P) > ase
VI 2 2h())+
Therefore,
0
030 of
(1 2- 22)
o(P) S 0ook
V1 2 (1)++22
D (R,
of
01
xx + (P) 20 ( vI - 22
o(P) ag VI 2 (5)+22
Case 3. If MSS and R h(X), ), then DC P,C)= = 0. In summary, by combining the above cases, we obtain the closed-form expression for D (R, C), as stated in Theorem 7
A.2 RDC EXPRESSION FOR BERNOULLI CASE IN ONE-SHOT SETTING
In addition to Theorem ! we propose the following definition of the rateddisortioon-cassffatttioo function based on the constrained optimal transport in the one-shot setting.
px be the degraded source, ~ PY the reconstruction, and S ps the associated classification variable with covariance Cov(X, S). Define Q(px, PY) the of joint distributions PU.X.Y with marginals Px Py that factorize as Pu.X.Y = Pu px py) The rate- distortion- cisoottonnclassficatton function based the constrained optimal transport with distortion loss D, classification loss C, and shared randomness as follows
Definition 5. Let
R(D, J.D,C.px.PY)
inf H(Y(U)
(17)
s.t.
H(Y[X,U) 0, & H(S|Y) S C.
I(X:U) 0,
The closed-form solution of R(B) ((B)) C, gv,4x) qV) for the Bernoulli case is derived by Theorem A.2
Theorem A.2. Consider a Bernoulli source X Bern(gx), Y Bern(qy), and classifi- cation variable S with the binary symmetric joint distribution given by S = =x @ S1 where S Bern(gs) and S1 Bern(gs.) (0 < qx, qs, qs, S 1²). The problem is is feasible C Z Ho(gs,). Assume the Hamming distortion measure. Under common randomness, we have
Hn(gx) (D( und - D)
D (B) min
<D <
2(1-9x 14x (x- Hy(m) Ho(nss)
+ +D(B) ind
2(1 9x)9x
Ha(gx) [H)mm) C
-1 1ax (C- (mm]
D/B D D BB
Ho(m) Holgs,)
Ha(m) Holus,
C 2 (m) and D ((B) ind
< < D (B)
0.
Proof. Following the proof of Theorem
2 can formulate the problem 17
RBB) (D, po(1), (2),), (3), (4) Hp( +r))))))))))))
min
(18)
s.t. (2 qxPo + (1 D, (1-xx)) + = qy,
(19)
(20)
(p)()))) (pr pu (4)) S CC
(21)
Pu + (3) (4) =1.
(22)
(1),), (2), (3),,, (4) 0.
(23)
25

---

Accepted to ICLR 2026.
The activity of the nonnegative constraints. We analyze the activity of the non-negativity constraints in (23). Since the objective function in (1) depends only on pu (1) and (2) mini-
mization requires reducing their values whenever possible. Hence, examine the following cases:
For Pu (1) = Pu (2) = Substituting into
20 and (22 yields pu (3) = 1 - gy and pv = qy-
In this situation, the feasibility conditions pu hence inactive.
and pu (4) 2 are automatically satisfied and
For (1) 0, Pv(2) * O. Constraint 22 becomes pt (2) + (3) + Pt (4) 1. Minimizing the objective requires allocating as much probability mass as possible to pe (3) and po (4), rendering the non-negativity constraints pc (3) 0 and po (4) inactive.
For (2) = O, Po # O. This case is symmetric to Case 2 and leads to the same conclusion.
In all of the cases, the optimal solution enforces inactivity of the non-negative constraints on Pt
and pr (4). On the other hand, eliminating po (3). (4) via (20 (22) gives
(Xd D ind 2(1 - qx (x)xxx + 2(1 PU
For 0 S qx S 2. the distortion E(dæ [X, Y) is strictly increasing in pu pc and decreasing in pu (1); hence optimal to take po (2) and pe (1) O.
Similar to the proof of Theorem 2 our approach explores all possible combinations of active and inactive rate and classification constraints to characterize the optimal solution.
Case 1. Constraint 19 is active, while constraint
21 is inactive.
Since po (2) 2 and the rate constraint holds with equality, we have
D 2(1 qx (x)xx((xxx))))) (1) pu + 4(1 qx (2) + D
2 - xx0xxxxxxxxc (pu: (1) Pu + (T)
DI 2[1 vax Hence,
Combining with 19 gives pu (1) + (2) z
Ho(qx) ) (D B) D) Ind
R (D. (D,C.xxx))))
2(1 ((1-4x)9x
This lower bound is tight, achieved by
(
p (1)
pt = 0,
2(1-4x)ux"
(
D
p (3)
+ 1-11-9 -
(4) 2[1-9x] gY
24x
Constraint 21 is inactive if
(po (1) + PC (2) (3) Pu (6)) <C
D 2(1 -1 )x [C- HH(mm) +D(B)
Hp(m) Holds;)
Case 2. Constraint 21 is active, while constraint 19 is inactive.
21 holds with equality, then
(po (1) (2)) (3) pu p4)) = C.
Using 22
this yields
(Pu(1) + (c))
(1) pu (2))Hllmm C,
Ho (m)) C
PU (1) (2) Ho(m) Ho(us))
Consequently, (D,C, q Hslax) (m)
(a(m))Hallsa)
26

---

Accepted to ICLR 2026.
This bound is tight, attained by
(1) -m
(t((2 = 0,
H,(m) (gs,
p0 (3) -(1-ox (m)))) qY,
(4) He(m)) Mo(as,
-ax -ax)H
+ qy
Ho(mm)-Mooss))
The rate constraint 19 is inactive provided
2(1 - -x |C Hn(m)]
Po (2) + 4xpo (3) + <
Ho(m) - Ho((s.) + D ind
Case 3. Both constraints 19 and are active.
From Case 2, if the classification constraint is tight,
Hs (mm and
Halgx) (H) CJ
(1(1) + (2) =
Ha(m) (gs,)
Ho(m) - Hol9s;)
The rate constraint simultaneously active if 2(1 2(1-9x)9x (CHHmmm] + (
Ha(m)- Holas,)
Case 4. Neither constraint 19 nor 21 is active.
We observe that the rate achieves its theoretical minimum when R (D, Hs(xx) [pr (1) (2) O, which implies (1) p2) = 0. Substituting into the con- straints (20) and (22p. we obtain (3) = qy, (4) = qy. Then, using constraints (19) and 21. we find that the feasibility of this configuration requires D 2 D and C 2 Ho (m). There- fore, the minimum achievable rate is zero, i.e., (D, C, qx Qv 0, if and only if the distortion
C.gx.qy)
and classification loss exceed the respective thresholds: D < - ( and C' 2 Hy (m).
In summary, combining all of the cases, the closed-form expression for (D, C, gx Q under Hamming distortion is given by Theorem A.2
A.3 RDC EXPRESSION FOR GAUSSIAN CASE IN ASYMTOPIC SETTING
propose the following definition of the rneeddisortionnccassffcaaion function based on the constrained optimal transport in the asymptotic setting.
In addition to Theorem 3
Definition 6. We have
R( (D,,c.xx))
inf px. I(XY))
(24)
s.t. E)d( DD, H(SIY)
The pair of functions (D(=))(R, C,)x,P)), (D, C,px,Py)) are natural asymptotic analogues of their one-shot counterparts. They provide a Shannon-style single-letter characterization cross- domain lossy compression with classification constraints. In the next section, we apply these results for Gaussian sources, deriving closed-form expressions that reveal the explicit tradeoff's between rate, distortion, and classification accuracy.
Theorem A.3. Consider X ~ N x and ~ N(Jy,, o be two Gaussian random variables, and let d(-,.) be the MSE distortion measure. Let S ~N(ss, o3) be an associated classification variable, with covariance of Cov(X, = 0,. The problem (4) is feasible if
27

---

Accepted to ICLR 2026.
C 2 log
0 0
+ h(S). Under the common randomness, we have
1
[ax My 0 o DJ2
log
2
40 x
[uxx (ux))xr)+o+ +o 10 1-
2-10 1-99CC-M9))0000j00
<
(D, C,4x,
1 0 (1 - 2 22-2 (3)+2)
log
2
0
D [(xx + 103/0 -2 23/1122C--M00lagog Mollogokoy
11
0, C > h(S) and D > (ux - 2x)
+ + toy.
Proof. Following the proof of Theorem
4 the problem (2 can be formulated
02
R (D,
min
log 1
(25a)
02
x
s.t.
(ex - -vr)2
+ 0x + o. - 20, S D,
(25b)
87
02
log
1
2 h(S) C. x
(25c)
2
2
o
The optimization problem (25 can analyzed using the KKT conditions. By systematically exam- ining all possible combinations of active and inactive rate and classification constraints, obtain the following cases.
Case 1. Constraint
25b is active while constraint 22cc is inactive.
From the distortion constraint, we have
(μx )2) o of 20.- = 82 (ux 22)) 0 + o D
2
Substituting this into the rate expression gives
(1
1ux - (( 22)) 0x + oy - D|3
log The classification constraint is inactive provided
(D, Cx qr)
40 0
1 - 87
0
log
h(S) -C
2
o σy
D < [ux - -yyx xx
+
10 221 22C
01
Case 2. Constraint 25b is inactive while constraint
25c
is active.
From the classification constraint, we obtain
1
0f 03
h(S) C 02 x oy
log
VI 2h())+20
2
01
Substituting into the rate expression yields
( C, qx. (x) = - log 1 0
2 2M(3)++)
0f
The distortion constraint is inactive if
(yx (yy) + xx + o - -0g < D
+ [ex - - o 12
2- C2(C-M()
0
28

---

Accepted to ICLR 2026.
Case 3. Both constraints (25b) and (2cc) are active.
From Case 2, (25c) is active then 02 = os vI 201--22h111 2b(5)+2C Meanwhile, the distortion
81
constraint requires (px )2r) ox + 2202 D, which implies
D (
+0x + 0 - 22 22 hiS
01
In this case, the rate is
(DCCgx)))) = log 1 - -
03 + 0. - D13
40 xy
Case 4. Both constraints (5b) and (5c are inactive.
When C' > h(S). the classification constraint (25c) becomes inactive, and if the distortion threshold satisfies D (μux )² + x + oy. the distortion constraint (25b) is also inactive. In this regime, the minimum rate reduces to its theoretical bound, i.e., R (D, C,gx, qr) = 0. This occurs when X and Y are independent, yielding YY 0. All constraints are satisfied since E[(X y) = (μxx )² xx of < D and h(S]Y) = h(S(X) S h(S) <C
In summary, combining all of the cases, the closed-form expression for (D, under MSE distortion is given by Theorem A.3
A.4 DRPC EXPRESSION FOR GAUSSIAN CASE WITH WASSERSTEIN DIVERGENCE
Theorem A.4. Let X N and ~ NNuy,, ab) be two Gaussian random variables.
Let S ~ N o be associated classification variable with covariance of Cor S)
On and be jointly Gaussian. For the case d(X, Y) (X (X=Y²² and ((xxPY) = (Px)))
have
D (R, P.C)
x ox (1 2-13,
0x -VP exv1 - 2 2-28 and C> 1 log 1
(1-2-22))
+ ((S)
0300
2 + (ox - -Pp² - -2xx(ox VP) V1 2 2-2R
vP > ox
xx V 2 2-26 and C > / log
(1-22-2R 28
+ +(S)
0 X
(1 2 21(5)+2 0,0x - vP < 0 x
x
V
2 2h)++22
07
01
1
(1
of
Of - -22-222R)
and_
log
h(()) S C S
2 log
+ hL(S)
2
0
2030 X (ax - VP)
x
2 + (ax
vpp2
V1 2 2h(5)+20
81
ox <P> g v1 2- (h)++22 and
log (1
+ +(S) S CC log
(1
87 8103-003221 28
+ M())
o3
03
0, C> h(S) and R > l(X).
Proof. Based on the proof of Theorem
7 can show the following Lemma.
Lemma A.5. For the case X ~ N oxx) with distortion d(X,Y) - (X-Y) and perception
divergence 6(px, pY) = W (2xxx ,Y we have
( (R, P.C) = inf ((0)) (R, (x9.9))))
s.t. = σy Sox,
MSIV)S Wi(px,pY) <
29

---

Accepted to ICLR 2026.
Proof of Lemma A.S
The argument parallels that of Lemma A.4 It suffices to show
(Px. (Px,
which follows directly from F((X YO S E[(X -
By Lemma A.5 it suffices to restrict to py with μy = μa and oy ax when computing DW (R, P.C). Further restricting py to Gaussian distributions (Theorem 6 gives
(R,
min oy P
(R,
(26)
+ hh()), then 0f112 2R)
Case 1. 1fC 2 log 1
0303
D(C) (R,
(ux nr)² +ox + +oy
20x0y V 2-2R
The term o² -0xx0y v 2 2-2 is decreasing for E =0,0xv 1 2-2n and increasing for
oy loxv 2 2-2R x)). Thus, the minimizing σy in 26 is
MevI-22-21 ox - vP > oxvl-21222 2-211 Ox - VP.
VP S oxv1-22221 2-22
oy
Therefore,
0 0 (1 2--2R,
2 2
OX VP S ox 2-22
(G) (R,P,C)= W
+ (ox vP)² - 20 20x VP)
V1 21-2-21 2R,
0x vP > ox VI 2-21
Case 2. If 1 log log(1
_
+ +MS) S C log (1 -0
+ h(()), then
oz
a500
(R, C,gx,
(ux - )2)) ) + ox or
20 81
v1 1 2-2)))222
Here, oz 20ga3 ay v1 2 2A())+2 decreases for oy
0g [0,
v1 2-2h(5)+20)
and in-
01
81
creases thereafter. Thus, the minimizing σy in 26 is
J30 v1 - 2
OX VP
go v1 - 2-20(5)+2C
Ty
ox vP,
vP
v1 2-2MS)+2C
ox
d,
Therefore,
T
(1 2 2N(5)+20),
0
0
0x vP= < asoi
VI 2 2h +2
19 (R,
01
0% + (ax - vpp² 20
2 (ax
VP)
1 2--2-2h (K)++2
0
ox vP osd
v1 - 2- 2-2h(5)+2C
Case 3. C
> ASS) and R > h(X), then D (R, P.C)) = 0.
In summary, by combining the three cases, the closed-fome foreddfommmcco expression for ( (R, P,P) is estab- lished in Theorem A4
30

---

Accepted to ICLR 2026.
B EXPERIMENTAL RESULTS
B.1 ENTROPY MODEL-BASED RATE ESTIMATION
This experiment directly estimates the compression rate R by computing the entropy of the latent representation rather than relying on an upper bound. The rate is defined as the expected code length under efficient entropy coding: R Ex~px |- log FfQ U))) where ((Q(f(X, U))) the learned entropy model of the encoder outputs. Following Ballé et al. (2018), a factorized, non-pan con-paraeetric distribution. Since the achieved rates of entropy coding are typically close to the true entropy (Rissanen & Langdon directly in terms of entropy:
parameterize PP as
1981
Ballé et al. 2017), we define the training loss
rate = E [|x - 8|[] - AlogP(Q(((XX + Wp + Ae CE(S,
which jointly balances fidelity, rate regularization, distribution alignment, and classification aceu- racy.
The rate-aceuracy and rate-distortion tradeoff curves for super-resolution (MNIST) and denoising (SVHN) are shown in Figure As expected, increasing the rate yields reconstructions with sharper visual quality and higher classification accuracy. Qualitative samples in Figures) [[c) and 7 illus- trate this trend: at low rates, MNIST digits appear blurry ambiguous and SVHN digits remain heavily corrupted by noise, whereas at higher rates, reconstructions become clearer and more faithful to the target distribution.
Additional experiments on CIFAR-10, ImageNet, and KODAK further confirm that the empirical tradeoffss align closely with theoretical predictions (Figures 8
10 and
31

---

Accepted to ICLR 2026.
0.040
Classification Accuracy
0.9
MSE Distortion Loss
0.035
0.8
0.030
0.025
0.020
0.015
0.5
0.030
0.4
0.4
0.8
1.2
1.4
0.0
0.6
0.8
1.2
144
(bpp)
(bpp)
(a) Accuracy rate (MNIST).
(b) MSE (MNIST).
HR
0.03
7
2
6
7
2
O
4
9
4
1
Y
9
1
4
5
9
0
6
7
0
6
9
0
5
9
0
5
4
Q
2
7 2
/
O
7 2
O
4
1 9
4 0 1
9
4
4
9 6 5
5
9
0
6
5
6
Ç
9
0
9
0
5
9
0
5
9
0
1
(c) Reconstructions (MNIST).
Classification Accuracy
0.040
0.035
0.0
0.030
0.5
0.025
0.4
MSE
0.005
0.0
1.5
2.0
0.5
1.0
1.5
2.0
2.5
(bpp)
(bpp)
(d) Accuracy rate (SVHN).
(c) MSE rate (SVHN).
Clean
Noisy
9
18
18:3
8839
1
0.91
1=1666
R
1
9
18133
181333
18883
$
(f) Reconstructions (SVHN).
Figure 7: The experimental results of 4x image super-resolution on the MNIST dataset and image denoising on the SVHN dataset corrupted by Gaussian noise, (0,, o²) with 25.
32

---

Accepted to ICLR 2026.
0.10
Classification Accuracy
MSE Distortion
0.4
0.06
0.3
0.04
0.2
0.02
0.00
0.25
0.50
0.75
1.00
1.25
1.50
1.75
2.00
0.00
0.25
0.50
0.75
1.00
1.25
1.50
1.75
2.00
Rate (bpp)
Rate (bpp)
(a) Accuracy vs. rate (CIFAR).
(b) MSE vs. rate (CIFAR).
Clean
Noisy
R 0.50
R= 1.22
2.07
R 3.41
(c) Reconstructions (CIFAR).
Classification Accuracy
0.10
0.5
MSE Distortion
0.08
0.4
0.06
0.1
0.02
Rate (bpp)
Rate (bpp)
(d) Accuracy vs. rate (ImageNet).
(c) MSE vs. rate (ImageNet).
Clean
Noisy
R
R 0.50
R 2.43
R 5.45
(f) Reconstructions (ImageNet).
Figure 8: The experimental results of image denoising on the CIFAR-10 and ImageNet datasets corrupted by Gaussian noise, AO o²) with a = 25.
33

---

Accepted to ICLR 2026.
Noise Standard Deviation
Classification Accuracy
0.10
0
15
MSE Distortion Loss
25
0.08
0.08
0.4
0.3
Noise Standard Deviation
0.04
0.2
0
15
0.1
0 50
0.0
0.5
1.0
1.5
2.0
0.0
0.5
1.0
1.5
2.0
Rate (bpp)
Rate (bpp)
(a) Accuracy rate (CIFAR).
(b) MSE vs. rate (CIFAR)
Noise Deviation
Classification Accuracy
0.12
15
MSE Distortion Loss
0
0.6
0.10
50
0.5
0.08
0.4
0.3
Noise Standard Deviation
0.04
25
0.02
Rate (bpp)
Rate (bpp)
(c) Accuracy vs. rate (ImageNet).
(d) MSE vs. rate (ImageNet)
Figure 9: The experimental results image denoising on the CIFAR and ImageNet datasets cor- rupted by Gaussian noise, NN0,0 o2).
Figure
reports denoising results on CIFAR-10 and ImageNet under varying Gaussian noise lev- (15, 25, 50). Across both datasets, we observe clear rate-accuracy and rate-distortion
els tradeoffs.
improves monotonically with rate, saturating near 0.7 for a = 15 and lower levels for heavier noise. The corresponding MSE curves (Fig- ure 9bb exhibit steep distortion reduction at low rates, followed by plateau, with higher consis- tently yielding larger residual errors.
For CIFAR-10, classification accuracy (Figure (a)
ImageNet shows similar pattern (Figures ( (d)). though the impact of noise is more pro- nounced. For σ 15, accuracy rises sharply with rate and approaches 0.8, while for a = 50 remains below 0.4 even the highest rates. MSE again drops quickly at low rates before stabilizing at dataset- and desendent levels.
Overall, stronger noise corruption reduces both achievable accuracy and rate-distortion efficiency. These results align with theoretical predictions: increasing rate enhances reconstruction fidelity and downstream performance, but noise severity imposes fundamental limits.
34

---

Accepted to ICLR 2026.
MSE Distortion Loss
0.06
(dB)
0.05
0.04
0.03
0.02
Rate (bpp)
(bpp)
(a) MSE rate.
(b) PSNR rate.
Clean
Noisy
R 0.13
R 0.29
R 0.76
R = 5.59
(c) Reconstructions.
Figure 10: The experimental results of image denoising the KODAK dataset corrupted by Gaus- sian noise, NO,003) with a = 25.
Figure 10 reports denoising results on the KODAK dataset with Gaussian noise at 25. The rate-distortion and rate-PSNR curves (Figures TO(b) show sharp distortion reduction as the rate increases from very low values, with PSNR rising rapidly and stabilizing near 20 dB.
Qualitative reconstructions (Figure [ follow the same progression. At very low rates (R 0.13), outputs remain dominated by residual noise and lose fine details. Moderate rates (R = 0.29- 0.76) yield substantially cleaner images, with sharper edges and restored textures. At high rates (R. 5.59), reconstructions closely approximate the ground truth, effectively suppressing noise while preserving perceptual details such as wall textures and color consistency. Additional close-up comparisons in Figure 11 further highlight these improvements.
35

---

Accepted to ICLR 2026.
Clean
Noisy
R 0.74
R 1.35
R 1.72
3.23
Figure 11: Examples for reconstructions with different rates of image denoising of Kodim01 on the KODAK dataset corrupted by Gaussian noise, (00,, o²) with 25.
B.2 INPAINTING PROBLEM
We study the inpainting setting, where the goal is to recover missing or occluded regions of an image from partially observed inputs. Let X [0. 1| denote a clean image and E {0, 1 binary mask indicating missing pixels (M 1 for missing, M 0 for observed). The observed input is then given by
Xors (1 M) X Mc,
where e E [0, 1] is a fixed fill value (e.g., zero) and @ denotes element-wise multiplication. The model reconstructs Y such that missing regions are restored while consistency is preserved in ob- served areas.
B.2.1 SUPERVISED INPAINTING (WITH CLEAN SOURCES)
When paired clean images are available. training is based on complementary mean-squared eтror losses. The primary loss focuses on fidelity in the masked region, while the context loss ensures coherence in the visible region:
IIM O (Y 2
MSEmiss
max 1, M)
MSF = [1(1 II(1- - M) @ (Y - x)Il
max(1, Σ(1 M))
where = = Σ Mij counts the masked pixels, while
HW M
counts unmasked ones. This normalization ensures averaging over the respective region sizes:
H
W
MSE max (1, Σ M)
Mr(Yys - xz)3
1=1
H
W
MSE max(1,
Σ
1MM)) - x.
( M))
=1
36

---

Accepted to ICLR 2026.
The full supervised objective function augments these terms with additional constraints:
Saper MSE + a MSE log P(Q() + Ap + Ae CB(S,S), (27)
where the entropy controls rate, the Wasserstein penalty enforces distributional alignment with Py, and the classification term encourages task-aware reconstructions. This parallels classical in-
painting formulations (Pathak al., 2016 lizuka al. [0177] Yu al. 2018 2019). but extends them by integrating compression and classification constraints.
B.2.2 UNSUPERVISED INPAINTING (WITHOUT CLEAN SOURCES)
In the absence of paired clean images, adopt self-supervised masking strategy inspired by Noise2Self (Batson & Royer 2019) and related approaches (Krull et al. [209 Laine et al.) 2019).
Specifically, randomly drop subset of observed pixels Marop (1 M) and form the input
Xin (1 Maro)) 2 Xohs + Marp
requiring the model to predict the dropped entries. Losses are defined on both the dropped and retained pixels:
Mannop ④ (Y Xats )
MSEarpp =
max(1, Σ Mdrop)
|I(11 - M) @ (1 Marop) o (Y Xahx 11
MSF
max (1 M) o Marop))
where Σ Marop counts dropped pixels, while (( M) @ (1 Mdro)) counts visible ones not dropped. Explicitly,
MSEarop
Σ> Σ
ij (Yg Xobb,,i)
max(1, Σ Mdrop)
H w
MSE
D (1 M1,)(1 - Marop, (hopps))(K3 (Y - Xoba.is)
max(1,E( 1, M) 3 (1 Mdrop))
i= =1
The resulting unsupervised objective function is
Lussuee MSE + aS Alog P(Q(F(X,U)) Wp + A CE(S, $), (28)
where MSEj discourages trivial copying of visible pixels and promotes robust self-prediction.
Supervised inpainting leverages paired clean targets, while unsupervised inpainting relies redun- dancy within noisy observations. Both variants naturally integrate into our compression--esstration framework, ensuring rate efficiency, perceptual alignment, and classification fidelity.
B.2.3 INPAINTING IMAGE RESULTS
Figures 12 and 13 report quantitative and qualitative results on the SVHN dataset for supervised and unsupervised inpainting, respectively.
37

---

Accepted to ICLR 2026.
0.040
Classification Accuracy
0.7
0,035
MSE (full image)
0.6
0.030
0.4
0.015
0.010
0.2
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.8
1.0
Rate (bpp)
Rate (bpp)
(a) Accuracy vs. rate.
(b) MSE vs. rate.
21
0.85
PSNR (dB, full image)
20
0.80
SSIM (full image)
0.75
0.70
18
17
10
0.55
0.50
14
0.45
0.8
1.0
0.2
Rate (bpp)
Rate (bpp)
(c) PSNR vs. rate.
(d) SSIM vs. rate.
Clear
Masked
R= 0.21
5
2
9
9
18
183
3
18.99
R 0.37
R= 0.89
R 1.08
18899
18
88
1
18
18889
(c) Reconstructions.
Figure 12: Supervised inpainting results on SVHN using loss (27). Higher rates consistently im- prove accuracy, reduce distortion, and yield reconstructions with sharper details and higher percep- tual quality.
38

---

Accepted to ICLR 2026.
Classification Accuracy
0.8
0.030
0.7
MSE (full image)
0.025
0.020
0.015
0.4
0.010
0.3
0.0
0.5
1.0
1.5
2.0
2.5
0.0
0.5
1.0
1.5
2.0
2.5
Rate (bpp)
Rate (bpp)
(a) Accuracy vs. rate.
(b) MSE vs. rate.
26
0.9
PSNR (dB. full image)
24
SSIM (full image)
22
20
0,7
88
0.6
16
1.5 Rate (bpp)
0.0
1.0
Rate (bpp)
(c) PSNR vs. rate.
(d) SSIM vs. rate.
Clear
Masked
R 0.24
5
9
18
81833
3
1899
R = 0.3
R 1.02
R= 1.91
5
9
11
88
18:3
6
181833
(e) Reconstructions.
Figure 13: Unsupervised inpainting results SVHN using loss 28 Despite lacking clean targets, the self-supervised model exhibits similar trends to the supervised case: accuracy and perceptual quality improve with higher rates, while qualitative examples show more coherent digit structures.
39

---

Accepted to ICLR 2026.
Supervised inpainting. With access paired clean targets, the model is trained using region- specific reconstruction losses (MSEmiss and MSExu) alongside rate, adversarial, and classification constraints. As shown in Figure [2] this setup produces reconstructions that sharper and more faithful to ground-truth digits, with missing regions plausibly filled and contextual consistency well preserved.
Unsupervised inpainting. Without clean targets, the model adopts a self-supervised masking strat- egy, minimizing prediction error on randomly dropped subsets (MSEdrop) and regularizing against trivial copying through MSEja. As illustrated in Figure 13 the reconstructions remain semantically coherent but are generally noisier, with softer digit boundaries than in the supervised setting. This underscores the robustness of sulf-superviiion, albeit with tradeoff in fine-detail fidelity.
Both supervised and unsupervised settings exhibit the expected RDC tradeoff: low rates, recon- structions are blurry and task accuracy degrades, while higher rates yield substantial gains in both pixel-level metrics (MSE, PSNR. SSIM) and classification accuracy. Remarkably, the unsupervised variant achieves competitive performance without paired clean data, validating the effectiveness of self-masked training within our compression-restoaaiin framework. Taken together, supervised models capitalize on explicit ground truth to achieve sharper reconstructions, while unsupervised models demonstrate resilience by leveraging internal redundancy, highlighting the flexibility of our framework across supervision regimes.
B.3 EXPERIMENTAL RESULTS OF SECTION B.1
B.3.1 DATASET DETAILS
MNIST. For 4x super-resolution, we MNIST (LeCun et al. 1998). Each 28x28 digit is down- sampled to 7x7 via bilinear interpolation and then upsampled back to 28 x8 to form the low- resolution (LR) input. Reconstructions evaluated against the original high-resolution (HR) tar- gets.
SVHN. We train SVHN (Netzer al. 2011) and evaluate the full 32 RGB test split. Inputs are corrupted with additive Gaussian noise at E (15, 25, 50,, and reconstructions are compared to the clean targets.
To study small-scale natural images, we use CIFAR-10 (Krizhevsky et al. 2009), consisting of 32x32 images from 10 classes. Inputs are corrupted with Gaussian noise with a E {15, 25, 50). yielding a non-trivial denoising task while remaining computationally efficient.
CIFAR-10.
ImageNet. For large-scale evaluation, we adopt ImageNet (Deng al. 2009). comprising high- resolution images across 1000 categories. Inputs are degraded with Gaussian noise at a (15, 25, 50).
KODAK. Finally, we test on the KODAK dataset (Company 1991), a benchmark of 24 uncom- pressed high- reghruesolution images widely used for perceptual quality evaluation due its fine textures and details.
B.3.2 TRAINING DETAILS
We adopt WGAN framework for distributional alignment, jointly training encoder f. decoder g. and discriminator d. By Cantorovich-Rubinsee duality (Villani 2009). the Wasserstein-1 distance
W Wi = sup E(d(Y) -(d())) |
where g())) and the Lipschitz constraint enforced via a gradient penalty GGuraaann et al. 2017). Unless otherwise specified, WGAN-GP uses Ap 0.02, AGP 10, Vddiseriminator 5. and classification weight X = 0.01. Optimization employs Adam (Kingma & Ba 2014) with learning rates 5x10-5 (autoencoder). 10 4 (entropy bottleneck), and 10 (discriminator), with (3,, 82) = (0.5. 0.999). We apply gradient clipping (norm 2.0), mixed precision, and compute exact rate hpp during evaluation.
40

---

Accepted to ICLR 2026.
Super-resolution (MNIST), Models trained for 3000 steps with batch size 512. The classifier ResNer18-smaal (grayscale), trained with SGD (learning rate 0.05, cosine schedule with 2-epoch warm-up, momentum 0.9, weight decay x 10 4), batch size 512.
Denoising (SVHN), Models are trained for 3000 steps with a batch size 1024. The classifier is ResNet18-smal (RGB), trained with SGD (learning rate 0.2, cosine schedule with 5-epoch warm- up, momentum 0.9, weight decay 5 x 10-4), batch size 512.
Denoising (CIFAR-10). 10).. Models trained for 3000 steps with batch size 1024. The classifier is WRN-28- 10,8,,000000 trained with SGD (learning rate 0.1, cosine warm-up 5 epochs, momentum 0.9, weight decay x 10- batch size 256.
Denoising (ImageNet). Models are trained for 10,000 steps with batch size 128, codec learning rate 5x10-+ -4, and otherwise identical optimization settings. The classifier is ResNet-18, trained with SGD (learning rate 0.05, cosine warm-up 5 epochs, momentum 0.9, weight decay 10-4), batch size 256.
Denoising (BSDS500 + KODAK). Training is performed on BSDS500 with additive Gaussian noise and evaluation on the 24 KODAK images. We sweep with batch size 32, using a Conv+GDN autoencoder with an entropy bottleneck (EB). Adam is used for codec and EB parameters (learning rate 10-3), and the discriminator is trained with Adam (learning rate 10 10-4, (81, 32) = (0.5, 0.999))-
Here WGAN-GP uses A, 10-3 AGP 10, and dddiciiiinatoor 5.
B.3.3 DETAILED RESULTS
Tables 5 summarize the effect the rate-weight parameter across datasets. For MNIST super- resolution (Table D larger À values reduce the rate but incur higher distortion and lower accuracy. SVHN ( exhibits the same trend, with accuracy dropping sharply under strong rate penalties. 10FAR-10 and ImageNet (Tables 3 and 4 show consistent trade-offs: accuracy remains stable for small but collapses once rate heavily constrained. For KODAK (Table 5). PSNR decreases steadily À increases.
Table 1: Performance across À values for 4x super-resolution on MNIST (Fig.
((a)
Fig. 7(b)
A
10
50
100
200
1000
Rate (bpp)
1.3623
0.8483
0.7609
0.6394
0.5060
0.2262
MSE
0.0055
0.0053
0.0051
0.0058
0.0063
0.0064
Accuracy X
0.9670
0.9664
0.9693
0.9587
0.9613
0.9532
2000
5000
10000
20000
50000
Rate (bpp)
0.1755
0.1109
0.0791
0.0537
0.0246
MSE
0.0078
0.0129
0.0158
0.0220
0.0407
Accuracy
0.9399
0.9114
0.8389
0.7248
0.3477
Table 2: Performance across À values for denoising SVHN with Gaussian noise, /N oZ) with σ 25 (Fig.
7d) Fig. 7()).
50
100
200
500
1500
Rate (bpp)
2.7139
1.1143
0.7819
0.5237
0.3048
0.1748
MSE
0.0057
0.0053
0.0056
0.0048
0.0060
0.0096
Accuracy
0.7717
0.7676
0.7615
0.7702
0.7449
0.6144
X
2000
2500
5000
10000
20000
Rate (bpp)
0.1426
0.1165
0.0792
0.0456
0.0356
MSE
0.0126
0.0176
0.0231
0.0332
0.0441
Accuracy
0.5502
0.4841
0.3058
0.2362
0.2044
41

---

Accepted to ICLR 2026.
Table 3: Performance across À values for denoising on CIFAR-10 with Gaussian noise, No o(),02)
with = 25 (Fig. 8(a) Fig. 8(b)
À
10
20
50
100
500
Rate (bpp)
2.0058
1.8343
1.5225
1.2778
0.5238
MSE
0.0095
0,0088
0.0090
0.0098
0.0141
Accuracy
0.6507
0.6456
0.6336
0.6038
0.4617
1000
2000
5000
50000
Rate (bpp)
0.3248
0.1953
0.0997
0.0168
MSE
0.0187
0.0275
0.0419
0.1043
Accuracy
0.3146
0.2155
0.1769
0.1276
Table 4: Performance across values for denoising on ImageNet with Gaussian noise, NV(0, o²) with 25 (Fig. 8(d) Fig. 8(e)
X
0
5
20
50
100
200
500
Rate (bpp)
5.4522
3.6295
3,0546
2.4334
1.7890
1.1477
0.4967
MSE
0.0147
0.0139
0.0143
0.0141
0.0145
0.0179
0.0249
Accuracy
0.6737
0.6806
0.6717
0.6705
0.6630
0.6466
0.4959
X
1500
2500
5000
10000
20000
50000
Rate (bpp)
0.2103
0.1405
0.0842
0.0503
0.0317
0.0153
MSE
0.0357
0.0423
0.0524
0.0675
0.0829
0.1233
Accuracy
0.1876
0.1154
0.0528
0.0438
0.0336
0.0333
Table Performance across values for denoising on KODAK with Gaussian noise, .N(0, o?) with 25 (Fig.
10
A
0
5
10
20
50
100
500
Rate (bpp)
5.5969
3.7743
3.3253
2.8155
2.0936
1.5241
0.2865
MSE
0.0097
0.0097
0.0096
0.0094
0.0093
0.0083
0.0090
PSNR (dB)
20.1417
20.1227
20.1665
20.2633
20.3248
20.8120
20.4752
X
1000
2500
5000
10000
20000
50000
Rate (bpp)
0.1723
0.0929
0.0549
0.0367
0.0232
0.0088
MSE
0.0127
0.0216
0.0291
0.0395
0.0525
0.0858
PSNR (dB)
18.9560
16.6618
15,3659
14.0345
12.7972
10.6646
B.3.4 NEURAL NETWORK ARCHITECTURES
We employ the same set of architectures across all datasets, summarized in Table 6 consists of an encoder f with three convolutional layers followed by a learnable entropy bottle- neck, and decoder g with three deconvolutional layers and GDN activations. The adversarial branch is WGAN-GP discriminator d, implemented as three strided convolutional blocks with LeakyReLU activations, global average pooling, and linear output. For downstream evaluation, we use ResNet 8-selal classifiers: grayscale variant for MNIST and RGB variants for SVHN, CIFAR-10. 10. and ImageNet. Each classifier contains four residual stages (two BasicBlocks per stage) without an initial max-pooling layer, followed by global average pooling and a linear prediction head.
The codec
Computational complexity and scalability of the proposed framework. Our model uses standard components from learned compression and restoration: convolutional autoencoder with an entropy model, WGAN-GP discriminator, and classifier. Training was performed on two RTX 3090 GPUs under Ubuntu using the PyTorch framework. Both training and inference are dominated by single forward/backward pass through the autoencoder. The discriminator and classifier reuse the same feature maps and therefore introduce only modest computational overhead, while rate estimation via the entropy model and universal quantization is computationally negligible.
42

---

Accepted to ICLR 2026.
For very high-resolution images, follow the standard patch/tile strategy used in modern learned codecs, so the overall complexity scales approximately linearly with the number of pixels and re- mains comparable to existing adversarial denoising and compression methods.
Table 6: Network architectures for autoencoder, discriminator, and classifier of Section B.1
Encoder f
Decoder g
Conv2D, stride GDN
Deconv2D, stride GDN
Conv2D, stride 2, GDN
Deconv2D, stride 2. GDN
Conv2D, stride 2
Deconv2D, stride 2
Entropy Bottleneck (EB)
Classifier (ResNet18-small. gray)
Stem: 3x3 Conv, BN, ReLU
Discriminator d (WGAN-GP)
Residual Stage I: BasicBlock x2 (stride 1)
Conv2D, stride 2, ReUU Conv2D, stride LeakyRell ReLU
Residual Stage 2: BasicBlock x2 (stride 2)
Residual Stage 3: BasicBlock x2 (stride 2)
Conv2D, stride 2, LeakyReLU
Residual Stage 4: BasicBlock x2 (stride 2)
Global AvgPool
Global Average Pooling
B.4 EXPERIMENTAL RESULTS OF SECTION B.2
B.4.1 DATASET DETAILS
SVHN (inpainting). We evaluate the SVHN dataset Netzer et al. 2011) with full 32x32 RGB images. For each image, binary mask M E 1 1xx322332 is synthesized (M=1 indicates missing random square box with side length sampled uniformly from [4, 8] pixels, and the fill value is set to 0.0 (black). Reconstructions are evaluated both on the masked region and on the entire image.
pixels). Unless otherwise specified, the mask
B.4.2 TRAINING DETAILS
Hyperparameters & schedule. Unless noted otherwise, training uses a=0.1, Ap 0.02, AGP- 10,
and Vddieriminnto- The codec (encoder-decoder) is optimized with Adam (learning rate 5x 10 3), while the WGAN diseriminator is trained with Adam (learning rate 10-4 (3,, .32) (0.5, 0.999))- Training runs for 3000 steps with batch size 1024, gradient clipping (norm 2.0), and mixed precision. For downstream evaluation, ResNet18-small classifier (RGB) is trained with SGD (learning rate 0.2, cosine schedule with 5-epoch warm-up, momentum 0.9, weight decay 5x10-9) batch size 512.
B.4.3 DETAILED RESULTS
Tables and report the performance of supervised and unsupervised inpainting the SVHN dataset across varying rate-penalty weights X, corresponding to the curves shown in Figures 12
and 13
43

---

Accepted to ICLR 2026.
Table Supervised inpainting on SVHN across values (Figure 12
A
Rate (bpp)
Accuracy
MSE
PSNR
SSIM
1
1.0909
0.7700
0.008996
20.4594
0.8410
10
0.8038
0.7176
0.008990
20.4626
0.8296
20
1.0586
0.7871
0.008122
20.9033
0.8517
50
0.8885
0.7296
0.008372
20.7719
0.8455
100
0.7879
0.7750
0.008514
20.6986
0.8430
200
0.5836
0.7644
0.008375
20.7703
0.8326
500
0.3692
0.6924
0.010719
19.6986
0.8185
1000
0.2656
0.6866
0.011975
19.2171
0.7997
1500
0.2055
0.5860
0.013879
18.5765
0.7688
2000
0.2033
0.5895
0.014372
18.4248
0.7721
2500
0.
0.4519
0.017193
17.6466
0.7151
5000
0.0973
0.2639
0.027659
15,5816
0,5665
10000
0.0624
0.2223
0.040927
13.8799
0.4488
Table 8: Unsupervised inpainting SVHN across À values (Figure
Rate (bpp)
Accuracy
MSE
PSNR
SSIM
2.796
0,852
0.002910
25.362
0.951
1.914
0.849
0.002742
25.619
0.948
20
1,412
0.849
0.003483
24.580
0.947
50
1.019
0.840
0.003849
24.147
0.936
100
0.760
0.836
0.002928
25.334
0.938
200
0.596
0.821
0.003712
24.303
0.922
500
0.407
0.771
0.006137
22.120
0.883
1000
0.295
0.733
0.006586
21.814
0.866
1500
0.237
0.654
0.009616
20.170
0.818
2000
0.207
0.632
0.008653
20.628
0.808
2500
0.183
0.580
0,011040
19.570
0.771
5000
0.125
0.433
0.017845
17.485
0.672
10000
0.085
0.319
0.024026
16.193
0.566
20000
0.050
0,245
0.032689
14.856
0.469
B.4.4 NEURAL NETWORK ARCHITECTURES
The codec is lightweight convolutional autoencoder with entropy bottleneck. The encoder consumes the 4-channel tensor [Xiss MM. and the decoder outputs RGB reconstruction Y. A small WGAN-GP discriminator d is used for distribution alignment, and a ResNet18-smin classifier evaluates downstream task performance.
Table Architectures for SVHN inpainting with denotes the number of latent bottleneck chan- nels.
Encoder f (4 (4->nb)
Decoder g (nb +3) Deconv3x3, stride 2, GDN
Conv3x3, stride 2, GDN
Conv3x3, stride 2, GDN
Deconv3x3, stride 2, GDN Deconv3x3, stride 2
Conv3x3, stride
Entropy Bottleneck
Classifier NetsNet18smman
Discriminator d (WGAN-GP)
3x3 stem (stride 1), BN. ReLU residual stages (BasicBlock x2)
Cony, stride 2, LeakyRel.U
Cony, stride 2, LeakyReL.U
Strides: 21, 2)
Cony, stride 2, LeakyReLU
Global Average Pooling
44

---

Accepted to ICLR 2026.
B.5 EXPERIMENTAL RESULTS OF SECTION 5
For these experiments, we follow the setup Li et al. 2022) with the following modifications. For super-resolution, models trained for 100 epochs with penalty weight À 0.05 across all rates. The learning rate is initialized at 10-2, decayed by factor of 5 after 30 epochs, and optimized using Adam (Kingma & Ba, 2014). The denoising setup mirrors super-resolution, except with À 0.03 and learning rate decay applied after 40 epochs.
The classifier ResNet20 trained jointly with the GAN. is updated alongside the discriminator using reconstructed images and ground-truth labels, optimized with SGD (learning rate 10-2, mo- mentum 0.9, weight decay 10-4). Training runs for 100 epochs with batch size 64 using standard cross-entropy loss. The classifier architecture is summarized in Table 10
Table 10: Classifier architecture.
Classifier
Input
Conv2D (10 filters, kernel=5), ReLU
MaxPool2D (kernel=2)
Conv2D (10 filters, kemel=5), ReLU
MaxPool2D (kernel=2)
Flatten
Linear, ReLU
Linear, Softmax
B.5.1 DETAILED RESULTS
Tables [I and 12 summarize the empirical trade-off's between distortion and accuracy under differ- ent rate budgets. For 4x super-resolution on MNIST (Table II). higher rates consistently reduce MSE while improving classification accuracy. For SVHN denoising with additive Gaussian noise (0, 20) (Table 12). accuracy rises steadily with rate and distortion decreases correspondingly. re- flecting the transition from rate-limited limk-iimtte regimes. These observations closely match
the theoretical predictions shown in Figures 6(a) 6(b) 6(d) 6(e)
Table 11: MSE distortion and accuracy different rates for 4x image super-resolution on MNIST
(Fig. ((a) Fig. 6(b)
Rate Upper Bound
4
8
12
16
20
24
28
32 0.0451
MSE
0.0718
0.0603
0.0535
0.0494
0.0477
0,0471
0.0444
Accuracy
0.8756
0.9158
0.9234
0.9288
0.9309
0,9309
0.9291
0.9317
Table 12: MSE distortion and accuracy different rates for denoising on SVHN with Gaussian No0 WN(0, o²) with a = 20 (Fig. 6(d) Fig. 6(e)).
Rate Upper Bound MSE
4
10
20
30
40
50
60
0.0364
0.0300
0.0226
0.0196
0.0189
0.0164
0.0170
Accuracy
0.1959
0.2115
0.3585
0.4459
0.5001
0.5389
0.5043
Rate Upper Bound MSE
70
80
90
100
110
120
0.0147
0.0142
0.0133
0.0127
0,0133
0.0126
Accuracy
0.6328
0.6442
0.6863
0.7056
0.6965
0.7070
45

---

Accepted to ICLR 2026.
C ADDITIONAL EXPERIMENTAL RESULTS
In this section, we provide supplementary empirical results that further validate our theoretical anal- ysis and support the claims made in the main paper.
C.I HEATMAP VISUALIZATION OF THE EMPIRICAL RDC FUNCTION IN FIGURE 6
The heatmap visualizations of the empirical RDC/CDR functions for the 4x super-resolution task MNIST and the denoising task SVHN with Gaussian noise 20 are presented in Figure and Figure T5] respectively. These results clearly illustrate the qualitative tradeoffs among distortion, rate, and classification accuracy: enforcing tighter classification constraint (i.e., requiring higher accuracy) at fixed rate consistently increases the achievable distortion. This behavior aligns with the theoretical structure of the RDC tradeoff.
0.90
30
ClocsiFication Accuracy
Classiication Accuracy
Rabe Upper Bound
0.30
Rate Upper Bound
20-
23
0.70
15
0.30
0.01
0.00
11:03
0.04
0.05
0.01
0.02
0.03
0.04
0.05
Distortion
MSE Distortion Loss
(a) RDC function MNIST dataset.
(h) CDR function MNIST dataset.
Figure 14: Experimental results: 4x super-resolution on MNIST. Higher rates yield better recon- structions and improved classification performance.
0.7
Clsssifiattion Acturacy
Classification Accuracy
Rate Upper Bound
Rate Upper Bound
IND 20
0.5
0.5
@900 do
20-
0.4
15
03
0.013 0.044 0.017 0.018
0018 0.019 MSE Distortion Loss
MSE Distortion Loss
(a) RDC function on SVHN dataset.
(b) CDR function on SVHN dataset.
Figure 15:
Experimental results: denoising on SVHN with Gaussian noise with 20. Higher
rates yield better reconstructions and improved classification performance.
C.2 CONTOUR VISUALIZAATION OF THE EMPIRICAL RDC FUNCTION IN FIGURE
The contour visualizations of the generated samples for the MNIST dataset on the super-resolution task and for the SVHN dataset the denoising task are shown in Figure 16 The equi-rate lines plot-
46

---

Accepted to ICLR 2026.
ted on R(D, C)) highlight the inherent tradeoff between distortion and classification performance: for any fixed rate constraint, improving classification accuracy (i.c., achieving lower cross- (ross-enrropy) requires accepting higher distortion, and conversely, reducing distortion necessitates weaker clas- sification constraint.
Rala Upper Bound
Bule Upper
Rate
0.55
Cop
1 E auti 2.12 605 MSE Distortion Loss
12
MSE Distarionn Loss
Clacifiottinn Accuracy
Clscsiicatinn Accuracy
0g
Rag
0.00
Gold
6.3
b013
(a) RDC function on MNIST dataset.
(h) RDC function on SVHN dataset.
Figure 16: The RDC functions on the MNIST and SVHN datasets, together with the equi-rate lines plotted on R(D,C C), highlight the tradeoff between distortion and classification performance at any fixed rate constraint.
C.3 QUANTITATIVE COMPARISON BETWEEN THEORETICAL AND EMPIRICAL DRC CURVES FIGURES AND 4
We conducted experiments to quantitatively compare the closed-form DRC curves with empirical estimates on synthetic data. The results are summarized in Figure [7 In both the Bernoulli and Gaussian settings, the theoretically predicted DRC curves closely match the empirical estimates, confirming the correctness of the closed-form expressions.
0.12
Closedifom
Closed form
Monte Carlo estimate
12-
Monte Carlo estimate
0.11
0.8
0.0
0.07
0.08
02-
0.6
0.8
0.9
1.0
10
A (bib))
(bb4)
(R,, C, Cx
lx.xxx999 )
R with
0.8. (b) D (R, C. qr)
R with C
2.
4x 0.3, qy 0.25, 45;
X.Y.S (00.))((01)).000
Figure 17: Closed-form DRC curves versus empirical estimates for Bernoulli and Gaussian distri- butions.
Bernoulli case. We consider the one-shot setting with X ~ Bern(qx), Y Bern(qy), S X t Si, and S, Bern (gs.), ), under Hamming distortion and constraints (R, C). For each admissible
pair (R, C), Theorem specifies the optimal joint distribution of (X, Y). We sample from this
47

---

Accepted to ICLR 2026.
joint distribution, estimate the mutual information I(X;Y) Y) and conditional entropy (S)Y), and compute the empirical distortion D Demp R, C). As seen in Figure 17(a)) the empirical points align closely with the theoretical DRC curve DBB)(B (R, C. 9s.)).
Gaussian case. We similarly validate Theorem 4 for jointly Gaussian (X, S) with quadratic distortion. For each (R, C), we construct the optimal coupling prescribed by the theorem, draw i.i.d. samples, and estimate T(X; Y), (S)Y), and the empirical distortion. Figure (7 that the empirical distortions again match the theoretical curve D(C) (R, C, 9x.xx,99 Qy with high fidelity.
C.4 EXPERIMENTS USING THE EXACT CONDITIONAL ENTROPY ((SY) PLACE OF CROSS-ENTROPY
Training with HSS)Y) directly. We note that H(S)Y) measures the uncertainty of the true label after observing the reconstruction y, i..., H(S|Y) Ey~po E, (+(())) log poll))) which depends on the true posterior Po(sly).
We now jointly train the classifier with the encoder/decoder, Rather than using pretrained classifier, we alternate between (i) training the encoder-decoder with fixed classifier and (ii) training the classifier using Y)) as loss function with a fixed encoder-decoder. Theoretically, the classifier should converge toward the true posterior Po(sly) for the current reconstructions In the limit of an optimal classifier (infinite capacity and converged training), the cross-entropy becomes equal to H (S Y') (Wang et al. 2025).
The resulting tradeoff curves using the exact (SSY) Figure 18 for the super-resolution task on MNIST and Figure 19 for the denoising task SVHN consistent with those obtained using cross-cntropy and confirm that decreasing (S corresponds to improved classification accuracy. These experiments further demonstrate the upper-hound relationship between (SS(Y) and CF(s, Wang et al. 2025).
0.070
ng ecact NDJYI
0.92
Classiiication Accuracy
0.005
MSC Ditorrai Lon LoSS
Dibtort Dittrtton LOGS
HIS Y9)
0.060
0.055
0.86
0.000
0.84
Using creas entrogy
Using
0.62
0.045
35
Rate Upper Bound
Rate Upper Bound
(a) (SJ)) versus rate upper bound.
(b) Classification accuracy versus rate upper bound.
Figure 18: Experimental results: 4x super-resolution on MNIST. Minimizing the conditional en-
tropy H(SY) is equivalent to maximizing the classification accuracy.
48

---

Accepted to ICLR 2026.
Using cress entropy Using exect HiSj9)
0.020
ClocsiFcation Accuracy
0.025
ISSE Distor LOSS
MSE Distortion LOGS
0.004
16-
HIS
0.022
0.000
10-
Using
0.8
Happ
Rate Upper Bound
Rate Upper Bound
(a) V) versus rate upper bound.
(b) Classification accuracy versus rate upper bound.
Figure 19: Experimental results: denoising on SVHN with Gaussian noise with a = 20. Minimizing
the conditional entropy (SIY) is equivalent maximizing the classification accuracy.
C.5 EMPIRICAL VALIDATION ON REAL-WORLD IMAGING AND PHOTOGRAPHIC NOISE.
Fluorescence microscopy images. In addition standard benchmarks, include experiments on real-world fluorescence microscopy images (Mouse Nuclei) (Buchholz et al. 2020), which ex- hibit statistics and structures very different from natural photographic images, under Gaussian noise (0, o²). As shown in Table I3 our method achieves decent denoising performance under Gaus- sian noise across different noise levels, with consistently high peak signal-to- and structural similarity index measure (SSIM) (Wang et al. [2004), as well as low perceptual distor- tions such as learned perceptual image patch similarity (LPIPS) (Zhang al., 2018) and deep image structure and texture similarity (DISTS) (Ding et al. 2020), averaged over 67 test images.
ratio (PSNR)
PSNR (dB)
SSIM
LPIPS +
DISTS + 0.1401
10
33.0340
0.8052 0.8028
0.0443
20
30.5880
0.0734
0.1675
Table 13: Performance values for denoising on Mouse Nuclei dataset.
Real-world smartphone images. To further probe robustness under real-world photographic noise, include experiments on the SIDD smartphone image dataset (Abdelhamed et al. 2018), which contains complex, signal-dependent noise patterns arising from real camera pipelines, averaged over 10 test images. As shown in Table [4] these additional experiments demonstrate that the proposed framework remains effective beyond controlled synthetic settings and applies to challenging real- world compreesiion/restoraiio scenarios.
PSNR (dB) SSIM
LPIPS
DISTS
33.605
0.9038
0.3233
0.2366
Table 14: Performance values on the SIDD dataset.
C.6 COMPARISON WITH OTHER COMPRESSION AND DENOISING APPROACHES.
quantitative comparison on the KODAK dataset corrupted by Gaussian noise with σ = 25 is shown in Table 15 We compare our method against the non-learning baselines JPEG-2K (Taub- man et al. 2002 and BM3D (Dabov et al. 2007). as well as the recent unsupervised denoising approaches DeCompress (Zafari et al. (225aa) and OTDenoising (Wang et al. 2023b). While these methods differ in objectives and constraints, the comparison is informative regarding tradeoff's in distortion, perceptual quality, and rate. In particular, although BM3D achieves higher PSNR, which
49

---

Accepted to ICLR 2026.
unsurprising, it does not explicitly penalize the rate. On the other hand, model performs favorably on perceptual metrics such as LPIPS, DISTS, and perceptual index (PI) (Ma et al. 2017 thanks to the WGAN-GP discriminator. For example, our PI score is significantly better than BM3D and DeCompress and is close to the best unsupervised method, OTDenoising. using a smaller rep- resentation rate. Overall, this comparison demonstrates that our framework preserves competitive restoration quality and perception.
Method
PSNR (dB)
SSIM T
LPIPS
DISTS
PI
JPEG-2K
26.4408
0.7357
0.4018
0.2419
7.4794
BM3D
31.8757
0.8687
0.2235
0.1640
2.6503
DeCompress
27.8315
0.7519
0.2627
0.1967
2.7979
OTDenoising 31.2893
0.8677
0.1150
0.1032
2.0095
Ours
27.8961
0.8035
0.1987
0.1638
2.1670
Table 15: Comparison of denoising performance the KODAK dataset with Gaussian noise NO o²), a 25. Best values are bold and second-best values are underlined.
50


Reviews:
[{"review_id": "YJX9kPzwmh", "reviewer": "Reviewer_F3r6", "summary": "This paper presents a comprehensive theoretical and empirical framework for cross-domain lossy compression, where the encoder observes a degraded source while the decoder reconstructs samples from a distinct target distribution. The authors formulate this as a constrained optimal transport problem with dual constraints on compression rate and classification loss. The key theoretical contributions include closed-form characterizations of distortion-rate-classification (DRC) and rate-distortion-classification (RDC) tradeoffs for both Bernoulli sources under Hamming distortion and Gaussian models under mean-squared error. The framework is further extended to incorporate perception constraints using Kullback-Leibler and squared Wasserstein divergences. The theoretical findings are validated through extensive experiments on super-resolution (MNIST), denoising (SVHN, CIFAR-10, ImageNet, KODAK), and inpainting (SVHN) tasks, demonstrating strong consistency between theoretical predictions and empirical performance.", "questions": "​Scalability and Generalization:​​ The theoretical results are derived for Bernoulli and Gaussian sources. How well do these theoretical predictions hold for more complex, multi-modal distributions encountered in real-world data?\n​​Optimization Tradeoffs:​​ The paper presents multiple constraints (rate, distortion, perception, classification). In practical applications, how should practitioners balance these competing objectives, and are there guidelines for setting the constraint parameters?", "limitations": "", "rating": 10, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "​Theoretical Novelty and Rigor:​​ The paper makes significant theoretical contributions by unifying optimal transport with rate-distortion theory and classification constraints. The derivation of closed-form solutions for both one-shot and asymptotic regimes represents substantial mathematical advancement in the field.\n​​Comprehensive Framework:​​ The authors successfully integrate multiple constraints—compression rate, distortion, perception quality, and classification performance—into a unified framework. The extension to distortion-rate-perception-classification (DRPC) functions with explicit perception constraints is particularly noteworthy.\n​​Strong Experimental Validation:​​ The paper provides extensive experimental validation across multiple datasets and tasks. The implementation of deep end-to-end compression models with quantization, entropy modeling, adversarial training, and classification components demonstrates practical applicability.\n​​Practical Relevance:​​ The problem formulation addresses real-world scenarios where compressed representations must serve multiple purposes simultaneously—maintaining perceptual quality, supporting downstream tasks, and operating under rate constraints.", "weaknesses": "​​Limited Real-World Application Scope:​​ Although the experiments cover multiple datasets, they primarily focus on standard academic benchmarks. Validation on more practical, real-world compression scenarios would strengthen the paper's impact.\n​​Computational Complexity:​​ The paper does not sufficiently address the computational requirements of the proposed approach, particularly for the adversarial training components and their scalability to very high-resolution images.\n​​Comparison with State-of-the-Art:​​ While the paper includes comparisons with baseline methods, more extensive comparisons with recent state-of-the-art learned compression methods would provide better context for the proposed approach's advantages.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8jllpW3pss", "reviewer": "Reviewer_Av5a", "summary": "The paper introduces a theoretical and algorithmic framework for cross-domain lossy compression, where the encoder observes degraded inputs while the decoder reconstructs samples from a distinct target distribution. The authors formalize this setting as a rate- and classification-constrained optimal transport (OT) problem, generalizing classical rate–distortion (RD) theory to cross-domain and task-aware scenarios. Closed-form solutions for Bernoulli and Gaussian models are derived, establishing distortion–rate–classification (DRC) and distortion–rate–perception–classification (DRPC) tradeoffs. The paper further bridges theory and practice through deep learning–based implementations, validated on tasks such as super-resolution, and denoising.", "questions": "Could the authors create a heatmap version of Figure 6? The 2 constraints (rate and classification) could be the x and y axes, and distortion is the color of the pixel (or something similar to Figure 4a in [1]).\n\nCould the authors create a 2D grid of generated samples from Figure 6 (similar to Figure 6 in [1])?\n\nCould the authors provide quantitative comparisons between the theoretically predicted DRC curves and the empirical results on the Bernoulli or Gauss distribution?\n\nCan the framework be extended to perception metrics (e.g., LPIPS or FID) rather than Wasserstein divergences?\n\n[1] Blau, Yochai, and Tomer Michaeli. \"Rethinking lossy compression: The rate-distortion-perception tradeoff.\" International Conference on Machine Learning. PMLR, 2019.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "This paper makes a conceptually original and mathematically rigorous contribution by integrating optimal transport, rate-distortion theory, and classification constraints within a single unified framework. \n\nThe idea of cross-domain compression-where the reconstruction belongs to a different distribution than the source, is a clear and meaningful generalization beyond standard RD or perception-aware setups. \n\nThe theoretical development is of high quality: the paper provides single-letter characterizations, leverages shared randomness to achieve deterministic transport plans, and derives closed-form DRC expressions.\n\nThe mathematical derivations are supported by intuitive illustrations. \n\nThe experimental section, although concise, demonstrates consistency between theory and deep generative implementations, validating the theoretical tradeoffs with empirical data.", "weaknesses": "While they derived closed from solution of the RDC tradeoff, for Bernoulli and Gaussian sources, for practical distributions like images, their evaluation is short.\n\nIn terms of clarity, the derivations are mathematically thorough but dense, which may reduce accessibility for readers outside the information theory community. \n\nThe experimental validation, while diverse in datasets, remains relatively superficial compared to the theoretical derivations.\n\nDiscussion of related work could be more critical, especially regarding how prior task-aware generative compression frameworks differ in assumptions, experimental scope, or real-world feasibility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "F2t7TnDl0z", "reviewer": "Reviewer_tNsX", "summary": "The paper models \"cross-domain lossy compression\" as a Constrained Optimal Transport problem. It derives closed-form expressions for the tradeoffs between Distortion-Rate-Classification (DRC) and Rate-Distortion-Classification (RDC).", "questions": "see weakness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper provides detailed and rigorous theoretical derivations.\n\n2. The paper addresses \"cross-domain lossy compression,\" which is more aligned with real-world application scenarios.", "weaknesses": "1. The core theoretical contribution of the paper lies in deriving closed-form DRC/DRPC tradeoffs for Bernoulli and Gaussian sources. However, the distribution of real-world natural images is far more complex than simple Bernoulli or Gaussian distributions. This may create a gap between the theory and practical application.\n\n2. The paper does not provide a quantitative benchmark comparison against current state-of-the-art (SOTA) image restoration (e.g., denoising, super-resolution) or compression-de-compression (codec) methods. This makes it difficult for readers to judge whether the proposed method is superior in practical performance or merely more theoretically complete.\n\n3. The paper's \"one-shot\" theory and experimental implementation both rely on shared common randomness ($U$). Although the paper mentions this can be implemented via a shared random seed, guaranteeing such randomness may be difficult in certain compression scenarios (e.g., write-once systems, broadcasting, or systems without two-way communication).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ujzQLgWIIv", "reviewer": "Reviewer_AfGP", "summary": "## Summary\n* This paper studies the rate distortion characteristic of a problem named \"cross-domain lossy compression\". In this problem, the authors considers joint optimization of distortion loss, classification uncertainty and even perception divergence.\n* The authors derive the closed formed solution to their function using constrained pptimal transport for Bernouli and Gaussian source.\n* The authors validate their theory through a somewhat different experimental setup using CE loss instead of classification uncertainty.", "questions": "## Questions\n* Is that possible to derive the theory with proper classification loss, such as CE?\n* What is the experimental result like using the real $H(S|Y)$ loss instead of CE? This is really not difficult to implement and the authors should present the honest result using the real loss following their theory.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 1, "strengths": "## Strength\n* This paper is well written and the study of the problem is comprehensisve. Both one-shot and asymptotic cases are considered.\n* The authors provide detailed example for both discrete and continous source (Bernouli and Gaussian).\n* The experimental design covers different degradation and different image dataset.", "weaknesses": "## Weakness\n* First, the mathematical formulation of the proposed approach is deviated from its objective.\n  * More precisely, the authors claim that the formulation considers the \"Classification utility\" and can be useful in practical scenario. However, the constraint imposed for the \"classification\" is $H(S|Y)$, the entropy of posterior distribution of class label conditioned on reconstruction. This $H(S|Y)$ has really nothing to do with classification utility as the classification in rate-distortion-classification (RDC) or rate-distortion-perception-classification\n(RDPC) [Task-oriented lossy compression with data, perception, and classification constraints].  In general, the constraint $H(S|Y)$ seems to be close to a measure of \"Inception Score\" [Improved Techniques for Training GANs], which is only a measure of how the reconstruction is close to any label, instead of how the reconstruction is close to the label of the source. \n  * For a MNIST example, we can let the decoder to constantly output random images with certainly recognizable digits. In that case, $\\forall C, H(S|Y)=0\\le C$. However, the reconstruction really has nothing todo with the input. And the $H(S|Y)$ really has no correlation with classification accuracy.\n* Second, the mathematical formulation of the proposed approach is also deviated from experimental results. Despite the $H(S|Y)$ in their theory is upperbounded by the CE loss they use in experimental result, the nature of $H(S|Y)$ and CE is very different. As I have explained previously, $H(S|Y)$ only measures how certain the model is on any label, given the reconstruction. However, CE is really about the classification accuracy of given label. I doubt that the authors prefe CE loss instead of $H(S|Y)$ in their experimental results just to get a better result. I can not see why it is necessay to use CE over $H(S|Y)$, as $H(S|Y)$ is really not hard to implement.\n* Overall, I do appreciate the effort the authors have spend in solving the formulated problem. However, the current problem formulation using $H(S|Y)$ is too different from the CE loss they use in experiments. Besides, the current problem formulation can not achieve the \"joint restoration + classification\" target that the authors have claimed. I recommend to reject this paper, as its theory deviates too much from their claim, and is too different from their empirical results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)