Lossy Common Information in a Learnable Gray-Wyner Network
Authors: Anderson de Andrade
Alon Harell
Ivan V. Bajic
Venue: ICLR 2026 Poster (2026)
Keywords: information theory
learnable compression
disentanglement
learning theory
representation learning
computer vision
neural networks
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Many computer vision tasks share substantial overlapping information, yet conventional codecs tend to ignore this, leading to redundant and inefficient representations. The Gray-Wyner network, a classical concept from information theory, offers a principled framework for separating common and task-specific information. Inspired by this idea, we develop a learnable three-channel codec that disentangles shared information from task-specific details across multiple vision tasks. We characterize the limits of this approach through the notion of lossy common information, and propose an optimization objective that balances inherent tradeoffs in learning such representations. Through comparisons of three codec architectures on two-task scenarios spanning six vision benchmarks, we demonstrate that our approach substantially reduces redundancy and consistently outperforms independent coding. These results highlight the practical value of revisiting Gray-Wyner theory in modern machine learning contexts, bridging classic information theory with task-driven representation learning.

Published as a conference paper at ICLR 2026
LOSSY COMMON INFORMATION IN A LEARNABLE GRAY-W WYNER NETWORK
Anderson de Andrade, Alon Harell & Ivan V. Bajié School of Engineering Science
Simon Fraser University Burnaby, BC, Canada
Sanderson.de.anraaee. a alonnharell, ibajae/esu....jj
ABSTRACT
Many computer vision tasks share substantial overlapping information, yet con- ventional codecs tend ignore this, leading to redundant and inefficient represen- tations. The Gray-Wyy Wyner network, classical concept from information theory, of- fers a principled framework for separating common and task-specific information. Inspired by this idea, we develop learnable three-channel codec that disentan- gles shared information from task-specific details across multiple vision tasks. We characterize the limits of this approach through the notion of lossy common infor- mation, and propose an optimization objective that balances inherent tradeoffs in learning such representations. Through comparisons of three codec architectures on two-task scenarios spanning six vision benchmarks, we demonstrate that our approach substantially reduces redundancy and consistently outperforms indepen- dent coding. These results highlight the practical value of revisiting Gray-Wyner theory in modern machine learning contexts, bridging classic information theory with task duikenriven representation learning.
INTRODUCTION
It is often the case that machine task classification, recognition, etc. requires only subset of the information provided as input. We can interpret neural networks as processes that discard irrelevant information from signal that its predictions in probability space similar to that of the target (Tishby & Zaslavsky| 2013). In multi-task settings, the same input is used to perform different tasks. These tasks have semantically different targets and hence might require different subsets of the input information.
Whenever tasks to ensure that communication is efficient. For example, when only object detection is needed, would be efficient for camera only transmit the information required the receiving device can perform that task. If it is then decided that semantic segmentation is needed for the same input, it would be efficient for the camera only transmit the additional information necessary, considering that some relevant information has already been transmitted. On the receiver device, it would be efficient if the information from the first transmission, that also relevant for semantic segmentation, is isolated, so it can be readily used without overhead.
performed jointly, isolating the information required for each one is critical
The line of work in coding for humans and machines (Choi Bajic 2022) focuses on this problem. It considers an image reconstruction task together with computer vision task. It is commonly assumed that all the information used by the computer vision task relevant to the reconstruction task. Thus, only two separate channels (tepresentations) are often designed: common channel used by both tasks, and a private channel only used by the reconstruction task. In this work, we focus on pair of tasks that have some common information (CI) between them, but also private (dedicated) information for each task, establishing three channels: a common channel, and two private channels.
We establish in this work that a complete isolation of the common information needed between two tasks is often unattainable. This also occurs in lossy coding (compression), where we send less in- formation (rate, e.g. bitrate), which inevitably performs the tasks with higher error (distortion).

---

Published as a conference paper at ICLR 2026
A system must then decide between transmitting some of the non-common information the com- mon channel, or some of the common information on the private channels. In the former case, the transmit rate the amount of information transmitted when performing both tasks one device can be optimal, but the receive rate the amount of information transmitted by performing each task on different device will not be optimal. The converse is true in the latter case.
We propose a neural network architecture that is able to separate the common information between two tasks. We compare the proposed architecture against other more intuitive architectures we designed, and include a theoretical justification for their difference in performance. We also propose a loss function that can optimize codec for the transmit rate, the receive rate, or tradeoff between both. We show how our methods perform on different pairs of computer vision tasks.
PREVIOUS WORK
In information theory, source coding methods convert a sequence of symbols from an information source into sequence of bits, allowing data compression. Our work is closely related to the Gray- Wyner Network (GWN) (Gray & Wyner 1974). is a source coding problem connecting two sources with two receivers Via common channel and two private channels. The work defines the Gray- Wynner region as a set of achievable rates (c.g. bitrates) for the three channels, in both lossy and lossless imput reconstruction tasks.
There are multiple notions of common information in literature (Viswanatha et al. 2014), including mutual information. In this work, we discuss Wyner's common information (Wyner 1975) and (GK) common information (Gacs & Körner 1973). These quantities are located in the Gray-Wynee achievable region. They are related by tradeoff between the total transmit and receive rate (Viswanatha et al. 2014).
The focus of seminal work on learnable image coding TThii et al., 2017 Ballé et al. 2018 He
let al. (2022) has been image lossy coding (compression) as a single task. They consist of a lossy analysis transform (encoder) and synthesis transform (decoder) acting as an autoencoder (Ballé iet al. 2018) such that the latent representation is better suited for coding, achieving a lower rate. An entropy model uses various types of context predict the distribution of a target representation. When optimizing for rate distoritooiion performance (Theis et al. 2017). the entropy model effectively induces an information bottleneck (Tishby et al.. 1999) on the target representation. A higher penalty on the probability estimates of the entropy model for the target representation produces lower rates at the expense of a higher task distortion (error).
The work in coding for humans and machines uses similar architectures to those used in learnable image coding. In Choi & Bajic| (2022), the output representation of an analysis transform is split in two. The computer vision task uses one representation over a common channel. The image reconstruction task uses the other representation over a private channel in addition to the representa- tion from the common channel. Better rate-distoria distortion performance the computer vision task was achieved when using two separate analysis transforms, one for each of these channels (Foroutan) al.) 2023). However, the common channel had information that was not fully utilized by the construction task. An ad-hoc reconstruction task for the common channel was shown to increase the usability (compatiblity) of the corresponding representations (de Andrade Bajic 2024).
Multitask learnable codecs exist in the literature (Chamain et al. 2021 Feng et al. 2022 Guo
al. 2024). They propose one or more common channels to perform several tasks, without private channels. Their rate is optimal only when all the tasks involved are performed jointly.
In representation learning literature, several thfoomation-theoretiic approaches propose variational autoencoders (VAEs) that learn disentangled representations of a source (Chen et al., 2016 Hig. gins et al. 2017 Chen et al. 2018h). These approaches are unsupervised in nature and thus do not distill information for a specific task or isolate the common information between tasks. A more related approach (Dubois et al., 202I) proposes a channel that can achieve high performance a set of predictive tasks, as long as they are invariant under a set of transformations. Although their proposed methods are also unsupervised, the right set of transformations can be used to isolate com- mon information between tasks. However, finding these transformations for non-trivial tasks is an open question. Several variational methods have been proposed to measure the mutual information between two sources (Poole et al. 2019). Although they might seem useful training objectives,
2

---

Published as a conference paper at ICLR 2026
Rg
R, Rx,,,
R₁
A
Ro R. Rx,(D.)
Yi
130011000
11
21
Bi xx D1,
Rg
> Separable
000011011
3 Rg Di.
X₁
R2
22
1/2
001100111
R1 Re Ray(Da)
(a) Gray-Wyner Network
(b) Gray-Wyner region
Figure 1: Diagram of the Gray- Beay-Wyner Network and a lower bound on its achievable region. The lower bound is given by planes A, B, and C. Plane A corresponds to the Pangloss, the contour of the achievable region that achieves Xx,,Xx,DD D2). Points with Rs = C(X1, X2; D1, D2) are found on it. Planes B and C' achieve Ro Ri Rx, (Di) and Ro + R2 Rx, (D2). respectively. On the intersection of both of these planes, we can find a point with Ro ((X,, X2; D1, D2). A trace connecting both points offers tradeoff's between the transmit and receive rates. If the mutual information between tasks, at distortions (D1, D2), is separable, both points coincide at the now achievable Separable point. We show that points close it, whether achievable or not, serve as bounds for both lossy common information measures.
these methods have significant tradeoff's between bias and variance. Morcover, they do not naturally offer the means to code (compress) the resulting representations. Compared against these existing techniques, our work directly addresses the isolation of common information between tasks and its efficient transmission.
2.1 PRELIMINARIES
We present the Gray-Wyner Network (Gray Wyner 1974) and define the two notions of com- mon information that define the boundaries of the transmi--rceeive tradeoff on which we want to operate. Let X1 and X2 be two source random variables. In the Gray-Wyne Network, an analysis transform (encoder) (Y,, Y2) X2) produces one common and two private dis- crete representations with corresponding rates (Ro, R1, R2). Two synthesis transforms (decoders) 2, 91 (Y,, Yi) and 22 92 (Y.. Y2) predict the targets Zı and Zg of two dependent tasks, produc- ing distortions D = di (Z,, Z1) = djd] (2, Z and D2 = d2(22, Z2) = Eld2 (2 Z2)]. respec- tively. In context, the distortion functions d1 and d2 are task losses for targets Z1 and Zz. We assume that one source does not have exclusive information that could assist its nor-corresponding task, implying the Markov conditions:
Za + X2 ++ X1s
21 ++ X1 X2.
The of all achievable rate tuples (Ry, Rg R2) for distortions (D1, D2) is given by the Gray- Wyner region Row (D1, D2). The contour of this region fower-bounde all other points in this convex set and are considered optimal in that they optimize particular rate-distortion function.
A rate-distortion function determines the minimal rate that should be communicated over a channel, so that input signals (sources) can be reconstructed without exceeding expected distortions. The joint rate-distortion function (Cover & Thomas 2006) is given as Rx,,xx(D,, D2). In our context, we define it as the minimum rate required to encode (Ž,, 22) jointly as a function of X2) so that tasks (di, d2) can be performed with at most (D,, D2) distortion. Marginal rate-distortion functions are given as Rx, (D1) and Rx, (D2), with similar definitions applying only to one task and one corresponding source. The conditional rate- ante-distottion function Rx,xx, (D1) is the minimum rate required to encode a function of X,, such that the task dj can be performed with at most distortion Dj, when X2 is available both the encoder and the decoder.
3

---

Published as a conference paper at ICLR 2026
In this setting, using previous definitions, define the Wyner's 's lossy common information as:
C(X,, X2; DI, D2) = inf T(X1, X2; U); where the infimum is over all joint densities P(X1, X2, 21, 22, U), under these Markov conditions:
2, + U + 22.
(x) + (21. 22) + U.
(3)
and where P(Z, X1, 11, E X1. is any joint distribution that achieves the rate-distortion Di function at (2), D2), ie.:; 1 X2; Z2; 22) = Xx (D1, D2). The mutual information function
Di
((3-) measures the dependence between two random variables (Cover & Thomas 2006). For the conditions in to hold, U must have at least the mutual information between Z1 and Z2-
Gáes-Kömer lossy common information defined for our particular setting as:
1(11, X2; D1, D2) = sup / ((1,, V),
(4)
where the supremum is over all joint densities P(X1, X2, 21 2z, V), such that the following Markov conditions are met:
X2 ++ t
X1 X2 x
X1 + 21 +
x ++ V,
(5)
where (() E PD and X()) E PDD are optimal rate-distortion encoders at Dj and D
respectively, i.e: ((x;;2)) (Di) and 1(x;; 22) = Rx (D2). The first two conditions establish that V cannot have information about X1 X2 that is not mutual. The last two conditions in establish that all information in V is present in both 21 and 2zz.
Wyner's ' lossy common information, C(X1, X2; D1, D2). can be thought of as the least information that must be included in the common channel to avoid the total transmit rate exceeding the optimal Xx,,x, (D1, D2). Conversely, Körrnr common information, K K(X,, X2; D1. D2), is the most information that can be included in the common channel while maintaining the optimal receive rate Rx, (D,) + Rx,(2) (D2).
The total transmit rate is RS = Ro + R + R₂ 2 Xx (D1, 2n, + R₁ + R₂ 2 Rx (D)) Rx,(D, Wyner's and KärssrKörner common information can be related through these concepts (Viswanatha et al.) 2014]. When the transmit rate is optimal, such that R2 Rx,.x, x, (D1, D2), we have that the minimum Ri Xx,,X,, D22 + C(X,, X2; D1, D2). Such points in the contour of Row (D1, D2) are achieved when Ro C(X1, X2, Dr, D2). When the receive rate is optimal such that R2 Rx, (D1) + Rx,(D2) we have that the minimum R is Rx, (D,) Rx (D2) K(X,, X2; D1, D2). This point in the contour of Row (Di, D2) is achieved when RS = ((1, X2, D1, D2). This implies that minimizing one type of rate comes at the cost of increasing the other, establishing the transmit-receive tradeoff.
(DI.D D2) and the total receive rate is R2
Figure [ visually describes these preliminary concepts. See Appendix I for more definitions.
CONTRIBUTIONS
3.1 BOUNDS FOR LOSSY COMMON INFORMATION
We extend result from Wyner 1975) in the lossless setting to the lossy case, showing bounds that separate the two lossy common in information terms discussed. The bounds are expressed in terms of interaction information ((X,, X2; Zi; 22), which is generalization of the mutual information for more than two variables (Ting 1962 Yeung 1991). See Appendix I for its definition.
Theorem 1. Let 2(1) be the set of tuples
that achieve (x D2). and (r) be the set of tuples (21, 22), such that 2, achieves Rx, (Dz), and 2, achieves Rx, (D2). Then:
K(X1,X2; D1, D2) <
max
T(X1, X2; 21; 22)
(2, Dy (()
<
min
I(X1, 21; 22) ((X D2).
(7)
Dy (2)
We have equality everywhere iff the maximum and minimum coincide at .2,,25) and we can represent Zi as (2,, W) and 732 as (Z2, W) such that Conditions Jand 3 for 7i, Zz, and W.
4

---

Published as a conference paper at ICLR 2026
Proof. See Appendix
Theorem allows to interpret the lossy versions of
and Wyner's common information similarly to their lossless counterparts. Any set of tuples (Z1. Z2) that achieye the transmit rate will always have at least the amount of interaction information 1 1 X2; Z1: Z2) of any set of tuples that achieve the receive rate. If there is gap of interaction information between the transmit and receive sets of tuples, exploring the transmit-receive tradeoff will produce tuples outside of those
two sets that cover such gap.
The conditions for equality everywhere imply that the two common information terms are the same when the interaction information T(X1, X2; Zi: Zz) is fully separable from private information or other excess common information that does not help reach optimal rate-distortion values. Because of the latter reason, this separation can be even more difficult to attain than in the lossless case. To see this, note that Gács-Körner common information in the discrete case is the entropy of the probability distribution of a point in the sample space of (X1, X2) belonging to a partition in an ergodic decomposition of the stochastic matrix defining (X1, X2) (Gacs & Körner 1973). This
means that to achieve 1 X2; 2;; Zz). this information must be separable such that the stochastic
matrix P(21, Z2) can be written as:
A
0
Ak
where A1,-., are probability matrices defined as product of marginals, i.e., with no mutual information. This implies that these random variables can be represented as 2i (2i, W) and Z: (2g, W), which, with W being a function of X1 or X2, implies that it can be isolated from X1 or X2. If some of the required mutual information is not separable, must be left out of W and thus I(X,, 2i; Z5)) cannot be produced. The work of Gacs & Körner (1973) shows that Fáes-Körner common information is often very small. In fact, is zero for Gaussian sources with correlation 1 - /Viswanatha et al. (014), which is usually distribution producing refinable results information theory (Cover Thomas 2006). Because we can often expect practice to have noticeable gap between the two common information measures discussed, there is significant motivation to explore the transmit-receive tradeoff.
3.2 TRANSMIT-RECEIV TRADEOFF OPTIMIZATION
As previosly discussed, there is tradeoff between optimizing for the transmit rate, resulting in more information available on the common channel than the mutual information, which increases the receive rate, or optimizing for the receive rate, resulting in less than the mutual information on the common channel, which increases the transmit rate. We propose an objective that can optimize for this tradcoff.
The work of Gray & Wyner 1974) establishes the following objective for optimizing the Gray- Wyner Network:
T(a1, D1, D2) & inf (I(X,, X2; Yo) + Rx (Dt) a2Rxx,1 (D2))
(9)
and an 02 2 and the infimum is over all probability distributions Py, E Py, with sample space Vo. The arguments an and a2 specify the transmission cost for each of the three channels. The values of T(a), 02;; D1, D2) over the domain of a and a: define the contour of the Gray-Wyner region Row(D, D2).
where 0 S 01, a2
Under assumptions suitable our proposed method, optimization over families of functions:
express this objective in terms of an
Theorem 2. Assume that Yo fo(X1, X2); = f(x)): E Fr, fe(Xe):
F2 are all deterministic functions of their corresponding inputs, and that 91 Gn and 92 E G2, where F(0.1.23 and G11.2) are families of functions such that there exits a fo, S1, f2, 91, and g2 in
their respective families that achieve T(01, 022 D1, D2). Then:
T(m,, 02;; D1, D2) = inf (Ho) + an ((i)V)) + 022 H())
(10)
where H-)) is Shannon's 's entropy function, and H-I--) is the conditional entropy function Coovr & Thomas (006). The infunum is over all for fix f2. It, and 92 in their corresponding family of functions, such that we obtain, at most, distortions Dj and D2.
5

---

Published as a conference paper at ICLR 2026
Y
fi
concas
St
2.
Pk
110011000
X2
ha
Pv
0200180ER
hv2
Pvs
021300233
f
spli:
ComCas
S2
2
Y/3
Figure 2: Architecture overview. The parameters for the probability distribution of the target repre- sentations are represented as Pr... Grey binary blocks denote bitstreams with rates R[u,,,2) For the particular choice of analysis and synthesis transforms used in the experimental evaluation, the grey bars on the corresponding blocks indicate the number of downsample or upsample operations.
Proof. See Appendix
We make use of entropy models to estimate and also steer the probability distributions of their target representations. With Theorem we can replace the entropy terms with rate functions ((0,1,2) given by entropy models:
T(1.2) ((1.2) (1,,2),, n No log P (YQ.2) (oo(Y) -E log P P())
(11)
where P denotes probability function assumed for the quantized target representations ((0,1.))
implicitly established by the entropy models.
The work of Gray& & Wyner 19744) highlights that T(a1, az; D1, D2) is difficult to optimize due to the lack of concavity or convexity in Pv. As such, we propose the Lagrangian relaxation method Urrrtt & Lemaréchal 1993) for the optimization problem in Theorem 2
Relaxing the dis- tortion constraints can help find better solutions in this non-convex space. The Lagrangian relaxation method is used extensively for rate-distortion optimization (Tishby et al. 1999). Due the convex- ity of that problem, it is used for convenience, since the method of Lagrange multipliers could also be used. For this problem, however, we have stronger motivation for its usage.
Assuming that the private channels have the same cost, such that a1 az2 replacing the entropy terms in Equation [0 with rate functions, and with the distortion constraints d, (Z,, Z1) S D₃ and d2 (22 Z2) I D2, the Lagrangian is given as:
C = inf (To((Y)) (Y,, Yo) + (Y2, + Aıd1 + X2d2 Z2).
(12)
where 8 2/1 and the infimum over the same families of functions in Equation 10 with the addition of the families of functions defining the three entropy models. The hyper- parameters A1 and Ag control the rate-distortion tradeoff (Cover Thomas 2006).
When 3 1, we optimize for the transmit rate Rg. When 3 = 2, we optimize for the receive rate R, As explained in Sections 2. and 3.1 optimizing exclusively for the transmit or the receive rate does not guarantee that the common channel will produce Wyner's or Gács-Kömee Körner common information. Therefore, values of 3 outside of the range (1,2) could result in suboptimal configu- rations. If we optimize for combination of both rates, could attain points on the trace in the contour of the achievable region Rew D2), connecting the operational points corresponding to C(X, D1, D2) and ((X, D1, D2) (Viswanatha et al. 2014). When 8 3/2, equally optimize for both the transmit and receive rates. If Theoremm I hoolds with equality, an optimal codec optimized for 3 E (1,2) 2) achieves both common information measures.
3.3 LEARNABLE GRAY- WYNER NETWORK
We now formulate version of Gray-Wyner Network that is grounded the proposed objective function. It separates common and private information between two tasks, as it explores the transmit- receive rate tradeoff. We learnalee entropy models the rate functions 1,,1,2 in Eq. T2 These

---

Published as a conference paper at ICLR 2026
entropy models produce rates that, part of the objective function, induce the desired type of information in the representations Y,,1, Moreover, they allow us to efficiently code (compress) the resulting representations.
Figure provides an overview of the proposed architecture. Inputs X1 and X2 are both processed by two analysis transforms fi and f2. Because each branch of the proposed architecture has access to both sources X, and X2, all exclusive information from either source is available to assist in performing tasks Zi or Zz. This effectively removes the requirement for the conditions in
The output of each analysis transform is passed through a quantization function that discretizes the representation. It has differentiable training-tima approximation that allows gradient propagation. The representation is then split into two tensors, such that:
(Y(L2), Y v((1 - (g f(122)) (X(12)).
(13)
Then, y are combined into Yo so that:
- ((x 10 + [,,]] if 1 = [100]
(Yo), -(0.
(14)
otherwise,
where i E Z indexes the elements in the tensors. Using auto-differentiation, the 1/2 (Y,2)) + y-²²) expression ensures that gradients flow to both inputs wherever elements match. An auxiliary loss term encourages the two input tensors match. The augmented loss function is given as:
7 [Yo] -x
Lang - C
(15)
where y influences the impact of this additional loss. Small values of y might result in elements of Yo and y/ never matching. A large can result in degenerate distributions for (- and - In both cases, the common channel is underutilized. Thus, this auxiliary loss can discourage the use of the common channel. We overcome this obstacle in practice by setting = 1 and reducing the cost of usage of the common channel /8 when necessary, offering as the only aaper-parameter
Entropy models for each private channel. h1 and h2, propose parameters of probability distribution assumed for its target representations, conditioned on the common representation. This conditioning is prescribed by the proposed objective function. Hence, these entropy models use context the previously coded elements in Y(122), in addition to Yo. to predict the distribution parameters of the current elements in Y(1,2), respectively. An entropy model for the common channel, ho. only uses as context the previously coded elements in Yo. As we have seen, is often difficult to discard the information in the common channel from the private channels, such that [(2, Yzi Yu) = O. Hence, using the common representation to model the entropy of the private representations can handle the redundancies between the private and common channels and improve compression.
Each private representation concatenated with the common representation and processed by a synthesis transform for its corresponding task. Each synthesis transform contains task-specific model to produce reasonable outputs within sample spaces of the targets Z(1.2), respectively. Using only the private channel as input for the synthesis transform is another plausible architecture but forces the common information to be present in the private channel. The conditional entropy models must then be able to predict this common information very accurately to avoid rate increases due to redundancies. This is often difficult for learnable codecs.
Other architectures for the analysis transform are evaluated in Section chitecture provides flexible representations while reducing the learning complexity of making them compatible. A theoretical justification is presented in Appendix in which we introduce a measure of compatibility between representations based on the generalization error induced by the hypothe- ses (family of functions) used to generate them.
Intuitively, this ar-
4 EXPERIMENTAL EVALUATION
In our experiments, the proposed architecture specializes to a single source X, so that (X)
X. We use architecture inspired by He al. (2022) as analysis and synthesis transforms. It
7

---

Published as a conference paper at ICLR 2026
Mothod: BD-Rate (%)
Muthod: BD-Rale (%)
Mutual Information (Empaneal)
Mutunl Infermation (TheomticcaD -93
0.8
(Therrttcca))
(Empirteall
100SU Stmm
0.8
Shared (5 1.3:
10.8
Starod: 10.42
Sepursted:
Shared ($ 2): 60.88
Combined: 87.55
0.4
0.0
10
Common Channel Rate (BPP)
Iransmit Kate (BPP)
(a) Common channel rate for 3 E {1, 2)
(b) Transmit rate for 3
=
Method: BD Rute (%)
M Method: BD-Rate (8) Sadependent (Engirica)) Indlependent [Therrtiraa)):
0.8
(Empriccal) (Theeoll
0.8
H00SU Stmm
0.6
Stared [3 1.55 3.41
Shured 11:
RMMSE Sum
0.8
Staredd 1.5): 2:40
Situred 18.59
Siared (p 23 24 5m
Sitared (3 2) 21 29.79
9.4
0.4
0.2
0.0
0.0
10
0.0
2.5
5.0
7.5
10.0
12.5
15.0
17.5
T Transmit Rate (SPP)
Receiv Rate ISPP)
(c) Transmit rate for E 2/,
(d) Receive rate for 8 {1, 9/2, 23
Figure 3: Rate-distortion curves for different methods, optimized for transmit or receive rate, or a mixture of both. BD-rates (Bjontegaard (001) are computed with respect the method with no assigned score. Bits-per-pixel (BPP) is the bitrate scaled by 1/64x64.
consists of stacks of 3 ResNet blocks each, with the stacks interjecting 4 convolutional layers acting as dimensional bottlenecks. These convolutional layers scale their inputs. As such, the analysis transform progressively reduces the spatial dimensions by total factor of 16, while increasing the number of channels to 24, 48, 192 and E. The synthesis transform performs the opposite in tandem, decreasing the number of channels and increasing the spatial size to that of the original input.
As a quantization function q, we use the trriighttthouuge gradient function proposed in Theis et al. 2017). We use the relatively simple model of Ballé et al. (2018). The common representation is processed and used in place of the hyper-prior to establish the conditional entropy models. As such, for each private channel, the common representation is processed by 2 stacks of 2 ResNet blocks each, interjected by a convolutional layer that increases the number of channels from E/2 to E. Masked convolutions turn the entropy model auto-regressive, which is suitable for coding (Ballé cet al. 2018
See Appendix D for architecture diagrams, hyper- training settings, other results, and further discussions. Code is available at:
4.1 TRANSMIT-RECEIVE TRADEOFF AND ABLATION STUDY
We developed a synthetic dataset to study the proposed method. Let X, and X₂ index the first dimension of a random variable X. We created an X such that HXX,,Xx X2) = 3.3 bits per element, 4.62, and consequently. X2); transformed to generate targets Z, and Z2 of two linear regression tasks. We train to minimize the RMSE loss between the predictions and the targets of each task.
H(1,) + HX2))
1.32. X2 and X₂ are individually
In addition to the proposed Shared architecture, we evaluate the rate-distortion performance of two additional encoder architectures. The Separated architecture has an independent analysis transform for each channel. The Combined architecture uses single analysis transform and splits the output tensor into parts corresponding to the GWN channels. If both tasks share a single channel, a resulting Joint architecture optimizes the transmit rate. Using a private channel for each task without a common channel, results in an Independent method which optimizes the receive rate. The rates produced by these methods used compute empirical estimates of the joint and marginal rate-
8

---

Published as a conference paper at ICLR 2026
Acturacy 5om
Method: (%)
Method:
Depandent
Depandent
Independentt:
Iedependentt 3.00
Miturre: 79.78
Mixture: 43.83
Misture ilsdependdent: 138.76
Mixture il's Sependendt- 41.68
Transmit Rate 35
30
35
Recetre (Bits)
(a) Transmit rate
(b) Receive rale
Figure 4: Validation rate-accuracy curves for colored MNIST on 3 different PMFs. The top-1 clas- sification accuracies are added and shown on the vertical axis. The total bitrate shown as rate. BD-rates are computed with respect to the Dependent PMF.
distortion functions, and mutual information between tasks. We contrast them against theoretical values. See Appendix D for details on the dataset, architectures, and measurement calculation.
Figure 3a shows that optimizing the transmit rate produces rates for the common channel that are higher than the empirical mutual information between sources. The same figure also shows that optimizing the receive rate produces rates for the common channel that are lower than the empirical mutual information. Codecs trained with 3 = 3/2 explore the transmit-receive tradeoff. They have a lower rate on the common channel than the codecs optimized for the transmit rate, but more information than those optimized for the receive rate.
For all explored, the Shared architecture outperforms the Separated and Combined alternatives, as shown in Figure for 3 1. See Appendix D for results the other B. Note that the empirical estimates of the rate are considerably higher than the theoretical values, as often seen in practice (Bajic] 2025). Figures and 3dl show that 3 3/2 reasonable value for this problem, since it performs marginally better than , = and [3 = 2, in both transmit and receive rates, respectively.
4.2 EDGE-CASE EXPL.ORATION WITH IMAGE CLASSIFICATION
Classification problems have well-defined theoretical measurements of mutual information against which compare. We randomly colorize digit images from MNIST (LeCun al. 2010) according to three different PMFs: A Dependent PMF, where one color always corresponds to one digit; 2. An Independent PMF, in which for each sample, one of 10 colors is sampled uniformly; and 3. A Mixture PMF, where each digit has a subset of 10 colors assigned to it, with uniform probability.
One of the tasks in our proposed method predicts the digit, while the other task predicts the color, using the colorized images and corresponding targets. The Dependent PMF has joint entropy of log, 10 bits and a mutual information of the same amount. The Independent PMF has joint entropy of 2 log, 10 bits, and a mutual information of 0 bits. The Mixture PMF has a joint entropy of 5.12 bits, and a mutual information of 1.4 bits.
Figures and 4b show the transmit an receive rates, respectively, for the 3 PMFs. We operate within an order of magnitude of the theoretical bounds, which is comparable to other codecs (Bajic) 2025). More importantly, the method trained on the Dependent PMF produces lower transmit rate since it places most of its information being common. On the other hand, the Independent method produces the lowest receive rate, since it has very low rate the common channel, with the underlying PMF
the common channel, taking advantage of all information
The common information in the Mixture PMF not very separable, which results in our method producing lower rate-distortion performance compared to the other PMFs. It still performs better, in terms of transmit rate, than the Independent approach from Section 4.I] where there is no common channel. Appendix D shows the rate of each channel, produced by our method, for every PMF

---

Published as a conference paper at ICLR 2026
Dutectio Koypeinting mÀP
1.305
mAU ltverse RMAIU
9.850
1,300
9.048
1,885
Mathodi BD-Ratt
Mathod: BD-Rate (8)
0.845
Joint
Independent:
1,000
deppptddnt::
0.644
Proposed
Proposed iTranemith:
0.842
Uncompreood
Propssed (Recsive):
1,085
Unramprwwed
Propozed (Beceirck 42.7
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
Rate (BFF)
Rate (BPP)
(a) Semantic segmentation and depth estimation
(b) Object detection and keypoint detection
Figure 5: Rate-accuracy curves of the proposed method against the Joint and Independent baselines. The tasks in (a) are reported for the validation set of Cityscapes. The tasks in (b) are reported for the validation set of COCO 2017. The transmit and receive rates the proposed method are included. BD-rates are computed with respect to the Joint method. The task performances are added. The depth RMSE is sealed so its inverse is a similar scale as the segmentation mean intersection over union (mloU). The detection performances are measured by the mean average precision (mAP). The Uncompressed lines correspond to the original performances of the pre-trained task models.
4.3 RATE-DISTORTION PERFORMANCE IN COMPUTER VISION TASKS
We evaluate the proposed method on semantic segmentation and depth estimation for Cityscapes (Cordts al., 2016), and object detection and keypoint detection for COCO 2017 (Lin et al.) 2014]. As part of 91 and 92, we append pre- pre-trained task-specific models to the synthesis transforms. We keep their weights fixed as we train the rest of the codec. We use DeepLabV3+ with MobileNet for semantic segmentation (Chen et al., 201 Sa), LRASPP with MobileNetV3 for depth estimation (Howard et al.] 2019), Faster R-CNN with ResNet50 for object detection (Ren et al., (017). and Keypoint R-CNN with ResNet50 for keypoint detection (He et al. 2020).
compares the performance of the proposed method. able to outperform an Independent approach and it is relatively close to the Joint approach. The curves for the receive rate are higher than the Independent approach, suggesting that the rate of the common channel is lower than the empirical mutual information between these tasks. Some curves in the Cityscapes experiments have an increase in distortion with the lowest compression, which is often informally attributed to the lack of regularization, provided by stronger rate constraints.
SUMMARY AND CONCLUSION
We validated the ability of the proposed learnable Pray-Wyner Network to distill common infor- mation between tasks and compared it against other architectures. The performance is theoretically justified by analyzing the compatibility between intermediate representations. We provided bounds that relate two measures of lossy common information. The proposed optimization objective is de- rived from this theory and was able empirically explore the transmit-receive rate tradeoff. The proposed method is also able to handle edge-cases, including a case where there is no mutual infor- mation between tasks, and another where the tasks are fully dependent. Finally, between the three computer vision experiments, our codecs achieved, on average, BD-rate advantage of -81.58% in transmit rate, against single-task codecs.
Isolating and coding the common information between dependent tasks allows for the efficient dis- tributed inference of machine tasks. Generating representations that explore the tradeoff between the transmit and receive rates in the Gray-Wyner Network has additional practical implications in storage and selective retrieval, and dispersive information routing (Viswanatha et al., 2011). Know- ing the information requirements of learned representations assist in planning for the resources allocated to a neural network, its dimensionality, and quantization levels.
Extensions to three more tasks possible, but since the total number of channels scales expo- nentially, a more dynamic architecture might be required. Nevertheless, the theoretical contributions of this work should prove useful in deriving new methods.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was partially funded by Intel Labs and the Natural Sciences and Engineering Research Council of Canada (NSERC).
REFERENCES
Ivan V. Bajić. Rate-accuracy bounds in visual coding for machines. In IEEE MIPR, 2025.
Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. Variational image compression with scale hyperprior. In ICLR, 2018.
Peter L. Bartlett and Shahar Mendelson. Rademacher and gaussian complexities: Risk bounds and structural results. JMLR, 2002.
Jens Behrmann, Will Grathwohl, Ricky T. Q. Chen, David Duvenaud, and Jörn-Henria Jacobsen. Invertible residual networks. In ICML, 2019.
Gisle Bjontegaard. Calculation of average PSNR differences between RD-curves. ITUT SC16/Q6 VCEG-M33, 2001.
Lahiru D. Chamain, Fabien Racapé, Jean Bégaint, Akshay Pushparaja, and Simon Feltman. End-to- end optimized image compression for multiple machine tasks. CoRR, 2103.04 04178, 2021.
Liang-Chich Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder- decoder with atrous separable convolution for semantic image segmentation. In ECCV, 201 8a.
Tian Qi Chen, Xuechen Li, Roger B. Grosse, and David Duvenaud. Isolating sources of disentan- glement in variational autoencoders. In NeurIPS, 2018b.
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Πlya Sutskever, and Pieter Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In NewrlPS, 2016.
Hyomin Choi and Ivan V. Bajic. Scalable image coding for humans and machines. IEEE TIP, 2022.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The Cityscapes dataset for semantic urban scene understanding. In IEEE CVPR, 2016.
Thomas M. Cover and Joy A. Thomas. Elements of information theory. Wiley, second edition, 2006.
Anderson de Andrade and Ivan V. Bajic. Towards task-compatible compressible representations, In IEEE ICME Workshops, 2024.
Yann Dubois, Benjamin Bloem-Reddy, Karen Ullrich, and Chris Maddison. Lossy compression for lossless prediction. In NeurlPS, 2021.
Ruoyu Feng, Xin Jin, Zongyu Guo, Runsen Feng, Yixin Gao, Tianyu He, Zhizheng Zhang, Simeng Sun, and Zhibo Chen. Image coding for machines with omnipotent feature learning. In ECCV, 2022.
Yalda Foroutan, Alon Harell, Anderson de Andrade, and Ivan V. Bajic. Base layer efficiency in scalable human-machine coding. IEEE ICIP, 2023.
Peter Gaes and J. Körner. Common information is far less than mutual information. Problems of Control and Information Theory, 1973.
Robert M. Gray. A new class of lower bounds information rates of stationary sources via condi- tional rate- distoriito functions. IEEE TIT, 1973.
Robert M. Gray and Aaron D. Wyner. Source coding for a simple network. IEEE BSTJ, 1974.
Sha Guo, Lin Sui, Chen-Lin Zhang, Zhuo Chen, Wenhan Yang, and Lingyu Duan. A unified image compression method for human perception and multiple vision tasks. In ECCV, 2024.

---

Published as a conference paper at ICLR 2026
Dailan He, Ziming Yang, Weikun Peng. Rui Ma, Hongwei Qin, and Yan Wang. ELIC: Efficient learned image compression with unevenly grouped space-channe contextual adaptive coding. In CVPR, 2022.
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross B. Girshick. Mask R-CNN. IEEE TPAMI, 2020.
Irina Higgins, Loic Matthey, Arka Pal, Christopher P. Burgess, Xavier Glorot, Matthew M. Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. In ICLR, 2017.
Jean-Baptiste Hiriart-Uruuty and Claude Lemaréchal. Convex analysis and minimization algorithms I:: Advanced theory and bundle methods. Springer Berlin, Heidelberg, 1993.
Andrew Howard, Ruoming Pang, Hartwig Adam, Quoe V. Le, Mark Sandler, Bo Chen, Weijun Wang, Kaan-CChihh Chen, Mingxing Tan, Grace Chu, Vijay Vasudevan, and Yukun Zhu. Search- ing for MobileNetV3. In IEEE/CVF ICCV, 2019.
Sham Kakade and Ambuj Tewari. Lecture notes in Rademacher composition and linear prediction, 2008
Yann LeCun, Corinna Cortes, and CJ Burges. MNIST handwritten digit database. ATT Labs [On- 2010.
line Available:
Michel Ledoux and Michel Talagrand. Probability in banach spaces: isoperimetry and processes. Springer Berlin Heidelberg, 2013.
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: Common objects in context. In ECCV, 2014.
Ben Poole, Sherjil Ozair, Aäron van den Oord, Alexander A. Alemi, and George Tucker. On varia- tional bounds of mutual information. In JCML, 2019.
Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE TPAMI, 2017.
Shai Shalev-Shwartz and Shai Ben-David. Understanding machine learning -from theory algo- rithms. Cambridge University Press, 2014.
Wenzhe Shi, Jose Caballero, Ferenc Huszar, Johannes Totz, Andrew P. Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang- Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In IEEE CVPR, 2016.
Lucas Theis, Wenzhe Shi, Andrew Cunningham, and Ferenc Huszár. Lossy image compression with compressive autoencoders. In ICLR, 2017.
Hu Kuo Ting. On the amount of information. Theory of Probability and Its Applications, 1962.
Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In IEEE I7W, 2015.
Naftali Tishby, Fernando C. N. Pereira, and William Bialek. The information bottleneck method. In Allerton Conference, 1999.
Vision maintainers and contributors. Viscon: PyTorch's computer vision library. GitHub repository, 2016.
Lan V. Truong. On Rademacher complexity- generalization bounds for deep learning. CoRR, 2208.04284, 2022.
Kumar Viswanatha, Emrah Akyol, and Kenneth Rose. An optimal transmit-receive rate tradcoff in gray-wyner network and relation to common information. IEEE ITW, 2011.
Kumar Viswanatha, Emrah Akyol, and Kenneth Rose. The lossy common information correlated sources. IEEE TIT, 2014.
12

---

Published as a conference paper at ICLR 2026
Aaron D. Wyner. The common information of two dependent random variables. IEEE TIT, 1975.
Raymond W. Yeung. A new outlook of Shannon's 's information measures. IEEE TIT, 1991.
A BOUNDS FOR LOSSY COMMON INFORMATION
A.1 1 ADDITIONAL PRELIMINARIES
Recall that the concept of mutual information measures the mutual dependence between two random variables. For two discrete distributions, it can be given
Σ X (x) log Xx, (211
x(x1;x))=
E
(16)
Px, (2) Px, (x,
31EX1 2gE
The concept also applies to continuous random variables, where a summation can be replaced with an integral over the corresponding sample space. The variables Xı or X2 can be extended joint probability functions of two or more variables, supporting concepts such as:
Σ Px Yun (Tim 31:n) log Px... (Ti:n Mltmn)
1m Vim)= = Σ
(17)
Px (21:)) (y1sn)
Thn Exm 91. Elim
where X1m = X1,..,, Xn and Y1: = .i..... Ya are all random variables. The conditional mutual information measure is the expected value of the mutual information of two random variables given the value of a third. can be given as:
log (x,ri2(z,g12 y12) Pxiz(1l=))P
I(X; ΣXEPrx
(18)
=EZ ZE
where is any discrete random variable. Similarly to mutual information, conditional mutual in- formation is non-negative and can be extended continuous random variables and joint probability distributions of two of more random variables.
The joint rate-distortion is given
xx,,x((1, D2) = min I(X1, X2; 21, 22).
(19)
where the minimum is taken with respect to all tuples (21, 22) that achieve at least (D1, D2) dis- tortion. The marginal dist-disto functions are given by:
Rx, (Di) = min T(X1; 21),
Rx, (D2) = min 7(X2; 22),
(20)
with similar distortion restrictions on Ż and 22. The conditional rate-distortion function is:
Rx, ix, (D1) min / (X1; (x2),
Rx,,x, (D2) min (X2;
(21)
with the corresponding distortion restrictions the minimums.
Recall that the interaction information (II) Ting 1962 Yeung 1991
between random variables X,
Y, and Z is given as:
I(X; Y; 2) ((X;) I(X; Y) [(X; Y Y
(22)
and that the chain rule of mutual information (Cover & Thomas 2006 is defined
I(X: I(X;Y, I(X:Y\Z) Z|
(23)
We derive the following corollary:
YX;Y; 2) = I(X,Y; ((X; I(Y;
(24)
An outline of the proof is presented as follows:
I(X;Y: = 1/2 [Xx - ZIY) + 2) I(Y: Z|x)]
EEq. 22 twice)
(25)
-(X,Y; - [X: I(Y;
(Eq. 23 twice)
(26)
The chain rule of mutual information and the definition of interaction information can be extended to any finite amount of random variables. Interaction information can be negative when the variables involved are conditional dependent.
13

---

Published as a conference paper at ICLR 2026
A.2 SUPPORTING STATEMENTS
Lemma 1. Let 201) D. be the set of tuples (21, 22) that achieve Xx D2), and 2(r)
D., be
the set of tuples (21, (2) such that Zi achieves Rx, (Di), and Zo achieves Rx, (D2). We have that:
max (2 22) D2
1(X1, X2; 21; 22)= x (2 (B.22))E2)... Di
min
1(X1, X2; 21 22)
(27)
Proof. By definition, we have:
(x + 1x(xx: = (x1: 2)12) +1 X22: 2(:22)
(28)
A ((())) 22 -
( (z/n1) (
Dr. D2
(r)
(29)
The minimization objective for Rx, (D1) Rx (D2), over all possible tuples (21, 22), is given
1(X1: 2,) + (2);
(30)
This objective selects 2 and 2g independently and does not consider their mutual information. Thus, solutions with lower or higher mutual information are not penalized and are present in 2(r)
DD, D2
as long as they meet the objective. Moreover, the objective does allow Z1 or Ze to have excess information in T(X1; 21) or T T(x2; Zz) to achieve D1 and D2, respectively, since different tuple would exist that discards this information, setting new goal for the accepted solutions in 2(r)
D..
To simplify notation, let X = X2). By definition, we have:
1 x; 2 -2 ( (x: 2/421) A (2 21 (
2,..,12/4.. 2D (31)
The minimization objective in Rx,,X, X2 (D1, D2), all possible tuples (2,, 22), is broken down as:
1x;
(32)
=T(X; (1) + T(X; 22)21)
EEq. 23
(33)
=I(X;21
+I(X; 2z)
I(X; 21; 2z)
(Eq.
(34)
(A)
(b)
(0)
I[X1 21)
I(x2: x1)
I(X2; 22)
[(x:: Xx)
I(X; 21: 22)
(Eq. 23
(35)
(a.1)
(a.2)
(b
(b.2)
(0)
Remarks:
1. Because of term (c), this objective considers the common information between variables, at the expense of allowing tuples to have private information from the other variable in them.
2. Information in (a.1) and (b.l) contributes to achieving D, and D2, respectively.
3. Information in (a.2) and (b.2) does not contribute achieving D1 or D2, respectively.
4. Interaction information [Xx; 21; 22) is present in the same proportion (a) and (b), each.
5. If some interaction information [(X; 21; 22) is (a.1), it (a.2). it not (b.1), it must be in (b.2).
6. Interaction information T(X; (1: 22) is distributed among (a.1) and (b.1), (a.1) and (b.2), and (a.2) and (b.1).
7. Interaction information T(X; 21; 22) present in (a.2) and (b.2) has no purpose and the minimization objective would select tuples where it does not exist, since the penalty is double and is not sufficiently countered by (c), increasing the objective.
8. Since information in 2, of type (a.2) or information in Zg of type (b.2) does not contribute achieving D1 D2. respectively, the minimization objective would select tuples that have zero information of those types, unless is also present in the other variable (Z, or 21), that it is countered by (c) without affecting the objective.
14

---

Published as a conference paper at ICLR 2026
9. Interaction information I(;; Z1: Z2) that reduces the private information in (a.1) and (b.1), takes precedence over other information when it decreases the objective.
Since all elements in 2(r) Dr have type (a.2) or (b.2) information and all have the same amount of type (a. 1) and (b. 1) information, only the term (c) differentiates them under the Rx..x, objective. Hence, the tuples with the most common information will have the lowest objective:
(x; 2 (1masx)
S T (x;
(x:71 (mmin)) ".(rain))
(36)
where (rmax) and (rmin) are tuples in ((r) D. with the highest and lowest I(X; 21; 22), respec- tively.
Let (fmiiu) be the tuple in 200 Dr, with the lowest [[;; 21; 22). This tuple has no information of type (a.2) (b.2), since it does not contribute reducing the objective. Moreover, this property, in conjunction with the conditions in [ establishes that / and 2,5 can be represented as functions of 11 and X2, respectively. Since these functions are applied independently to different variables, the resulting variables cannot be conditionally dependent on X. Thus, the interaction information involving these variables is non-negative. Using these properties, we have:
2)
(37)
=1(X1; 21) + 1(X2; 211X1) + I(X2; 22) + [(X1; 22(X2) - 1(X; 21; 22)
(Eq. 35
(38)
1 (x1:2(tmin) -2(mma) 1 (22; 2/ )) - X: 2x (mmn))) _2(
(no a.2/b.2)
(39)
< / (11 3(ttmm)) +1
(22 (x222 2g 254min))
(II 2 0) (40)
Thus, tuples that result from optimizing Rx, x, (Di, D2) can produce lower values than tuples resulting from optimizing Rx (D1) + Rx (D2). such that:
[xx: ((mm)))
(x; ((mmar)
(41)
A minor increase in private information at the expense of greater common information in (a.1) and
(b. 1) can cause the objective decrease further than any tuple optimized for Rx, (D1) + Rx, (2)) This result conforms to the one shown by Gray (1973). stating that:
RxI.x, (X,, D2) $ Rx,(D)) (D:) + Rx,
(42)
A.3 MAIN THEOREM
Theorem 1. Let 2(t) D2 be the set of tuples (2,, (2) that achieve Rx,,x,(1,, D2), and ((r)
D
be the set of tuples (21, 22). such that 2i achieves Rx, (Di), and 22 achieves Rx (D2). Then:
K(X,,X2; D1, D2) <
max (2 (2)
I(X1,X2; 21; 2z)
<
min
I(X, X2: 2;; 22) XXXX,, D2).
(7)
(2, (2,,2))
We have equality everywhere iff the maximum and minimum coincide at (2i, 23), and we can represent Zi as (Z,, W) and Zž as (Z2, ()) such that Conditions 3 for Zi, 2;, and W.
Proof. We reinstate the conditions for Wyner's common information individually:
2 + U + Zzz
(43)
(X1, + (2,, 22) ++ U,
(44)
and do the same for Gåcs-Körer common information:
X2 V,
(45)
X2 V,
(46)
x +
(47)
X 2g +
(48)
15

---

Published as a conference paper at ICLR 2026
To simplify notation, let X (X1, X2), and 2 =Zz, Z2). We show for the RHS that:
(X;; 2 22; + ((x; 22
(Eq. 22
(49)
= =[[; 21:
(Cond. 43
(50)
=[X; - -
(Eq.
(51)
S <T(X; 2;U)
(T(X; W; Y 2) 2 0)
(52)
=));U) -1(XU112)
(Eq. 22
(53)
I(XUU).
(Cond. 44)
(54)
Thus:
I(: 2o) & I(XUU) = inf I(X; 21; 22) inf (X::U)
(55)
min (X;; 21; 22) S X2; (Lemma
(56)
where the infimum over the same set as in Eq. 2 and the minimum is over the same set as in Le
To prove the LHS, first we derive the following:
T(X; 2;; = +))
00
(Eq.
23
46
48
(57)
I(X; 21; - - v(21) 0,
(Bq.
45
(58)
T(X;
(Bq.
23
48
46
(59)
Now, we have:
I(X; I(X;21;22 22)
1(X; 21; V) + 1(X; 21;;21;2211
(Eq..
22
(60)
2 I(X; 21; V2
(I(X; W Y 2
(61)
I(X; I(X; 2,V) 1(x))-((;;2200) (1x
(Eq.
24
(62)
=I(X; 2VV)
(Eq. 57 58)
(63)
= =I(X:V) - -((:V112
(Eq.
22
(64)
V(.;V).
(Eq.
59
(65)
Finally, we have:
I(X:V) < ((X; 21: = sup I(X:V) S sup I(X; 21; 22)
(66)
K(X,,XX D1, D2) S max [X; 21; 2z), (Lemma
(67)
where the supremum is over the same set as in Eq. and the maximum is over the same set in L
To obtain equality the RHS, Equations 57 and 58 shows us under which conditions the last two terms Equation 51 are zero. Since the conditions hold in equality for 2* and W, we continue the same analysis with the corresponding terms set to zero, yielding:
W) = T(X; ((:;;:)
T(X; (x;2;;2) T(X;
(Cds. 45 48
(68)
inf (X; 22) = inf I(X:W)
(69)
I(X; 2( = C(Xi, X2; Dı, D2),
(Lemma I (70)
where the infimum over the same conditions as in Eq. 2
For the LHS, due to Condition 43
holding in equality for 2- and W, have equality in Equation
61 Hence:
[X: 21: 2x) I(X; 21 22:
I(X: I(X; 21; 22)
(Cond. 43
(71)
T(p = sup T(X; ; 21; 722) K(X1, X2; D1, D2) T(X; 21; 2;; (Lemma
(72)
(73)
16

---

Published as a conference paper at ICLR 2026
where the supremum is over the same conditions in Eq.
If either of Conditions 45
) is present in W and equality is not achieved in Equation 52 Condition is not met, not all mutual information between 2j and 2z,,
48 is not met, more than (X; 2i)
in common with (X X2), is placed on W and equality is not achieved in Equation 61 If all these
conditions are met, but we have that:
I(X1, Z1: Z2)
min
[2; 21: Zz).
(74)
2
(2, (2) D1.
then the optimization of the joint rate-distortion and the marginal rate-distortion objectives produce tuples with different amounts of mutual information, and the lossy common information measures will not match.
B DERIVATION OF THE OPTIMIZATION OBJECTIVE
Theorem 2. Assume that Yo = fo(X1, X2); E Fo,Y, fi(i); E = J2(2); e F2 are all deterministic functions of their corresponding inputs, and that gi G1 and g2 E Gz. where 1 and ((1,2) families of functions such that there exits a for fI, f2. 91. and g2 in
their respective families that achieve T(a1, Or2 D1, D2). Then:
T(a1, 02; D1, D2) inf (Yo) + (Y) ayH((Y2)))))
(10)
where H()) is Shannon's 's entropy function, and H(---) is the conditional entropy function (Cover & Thomas (006). The infinum is over all fur fI, f2, Ir, and 92 in their corresponding family of functions. such that we obtain, at most, distortions D2 and Dz-
Proof. Since Yo is deterministic function of (2), have:
1(X1, Yo) & HYY) HYYo X2) HYo).
(75)
Since Y1 is a deterministic function of X1, we have for the conditional rate-distortion function that: Rx,1%. (Di) e min (XX;Y min (V)))) H(Y[X,,, min H(Yi)Y)),
(76)
where the minimum is over all probability distributions P(Y][X,) E Prij,, such that the resulting Yi produces most distortion D,. The converse holds, where, since Y2 is deterministic function of X2, we have that:
RxaIYa (D2)
(D2) & min T(x2; Y2) min (Y2 H(22 X2, Y)) min H(Y2/Yo),
(77)
where the minimum is over all probability distributions X2) E Pyslx, such that the resulting Y2 produces at most distortion Dz-
An optimization over probability distributions of random variables can be swapped by an optimiza- tion over family of functions generating those random variables, as long as the function producing the probability distribution minimizing the first problem is in the family of functions. Under this assumption, replacing the terms in Eq.
T(a 02; D1, D2) & inf (I(X, Yo) + a, Rx, (2Rxx,x(DD))
(78)
with Equations
75
76
and 77 we arrive at the desired result.
C THEORY OF COMPATIBLE REPRESENTATIONS
We define the compatibility between two representations as the measurable difficulty of learning a task that tries to match them in Euclidean space. Since their information content can be different, it might not be possible for the two representations to closely match. However, we do not propose to measure the loss itself but the generalization error induced by the family of functions used to generate these representations. An increase in the upper-bound on this generalization error means that the corresponding model has the potential make the representations more difficult to bring back to a common manifold.
Within this section, override all previous established notation:
17

---

Published as a conference paper at ICLR 2026
Definition 1. Let : > - R be function representing a neural network layer such as dense,
convolutional, ResNet, or transformer layer. Let a be an element wise function, such as a ReLU, ELU, sigmoid, tanh, an identity function. Let F be family of neural network layers. We define a neural network family of I. E B,4 homogeneous layers as:
FL= {f= f(y) = (fr 00 f2 000 fi)(y), v(fi.....L) E F1x E
FL;
(79)
F, Vi (1..,,, b)}.
(80)
Let D = [[y1) (y2l)) Yi, Y2 be a of samples from the joint distribution Y1, Y2. We de- fine the compatibility between representations' Y1 and Y2 as the generalization error C (D; Y1, Y2) of regression task of 11 onto Y2:
CF (D;Y,,Y)) sup (f) ED(E)) BELFL
(81)
Lr, = (e: (l:((y1,ee) = (1f - -2ll, E FL),
(82)
(C) Eiyr, (2),,,y)),, ED(() = 1
Σ (YI.2)ED
(2y,,2)).
(83)
Lemma 2. Let the functions in the family S C Fr, be Lipschitz such that Lip(f) S Vf E S, where Lip(f) retrieves the Lipschitz constant of a function f. The Rademacher complexity Radp(Cs) is bounded by:
(Cd) S (RRad (Di) + Rad(D2),
(84)
where Dj = {yr : (2) E D}. D2 = {y2 (21, E D}, and Rad(D) is the Rademacher complexity of set D.
Proof. Let A be a random variable with sample space (-1,1)* following the Rademacher distri- bution. We have that:
N age (Ivil: [y21))
Radp(Ls) = EaNA
sup RELs
(85)
N
N
<
EanA
sup
(f ((va)))((xa))) - -Iv2ld
(86)
N
/ES
E EanA
sup
a ((y1),)
Σ ai [y2]
(87)
N
[ES
N
Ea-A
sup
Σ a.f ((yil))
ai [y2]
(88)
N
EES
N
EanA
sup fES
a (bx1)))
+
Ea~A
sup fes
03
[Y2],
(89)
N
1=1
Radp, (S) + (D2)
(90)
<Raad(D) (D1) + Rad(D2)
(91)
Eq. 86 uses ussess the Kakade & Tewari Lemma (Kakade & Tewari 2008) based on Talagrand's contraction principle (Ledoux & Talagrand 2013 Bartlett & Mendelson 22002. states that all vectors in set A are operated by a Lipschitz function, then (ad(A) is at most multiplied by the Lipschitz constant of the function. It is easy to show that the norm is 1-Lipschitz. Eq. 91 uses the same
lemma. Eq. 88 uses la
bl < |a| + (6].
L
We can bound the compatibility measure Cs(D; X2) using a well-known generalization error bound in terms of Rademacher complexity:
Theorem 3. Let the functions in the family S C Fr, be Lipschitz such that Lip(f) S SNVE E S, and let the loss function be bounded such that ((y1,y2) I-B B) VE E Ls.y1 E V2,.2 E Y2, where V1 and Y22 are the sample spaces of Y1 and Yz, respectively. Then, Vo E (0,1), with probability at
least 1 - 8, we have:
Cs(D; Y2) 2 (KRdd((( + Rad(D2)) + BV 2/N log 2/6
(92)
18

---

Published as a conference paper at ICLR 2026
Proof.
(i; Y2) < 2Rad (Rs) + B 2/ 2/22//log/// 1/4 2/4
(93)
$ 2 (2 + Ra)( + B 2/N log
(94)
Eq. 93 uses Theorem 26.5 in Shalev-Shwartz Ben-David 2014). Eq. uses Lemma
Theorem 4. Assume that the representations Y1 and Y2 are functions of common random variable X, such that:
= (i)ii E 9LiGH FL., =2 E 9Lai91a FL2,
(95)
where GL, and 9L2 subsets corresponding to ResNets with 1- 1-Lipschitz activations of L1 and L2 layers, respectively. Further assume that the kernels ( W3 E R&xK, xxxK for the convolutional layers 1 (1,-.., max max(T1, L1, } in G and Giz are bounded such that:
max L2) sup
[w:]]
sup
S4
(96)
1=1
j.k
where 7, 2 O. Let Dx = {x1 ~ X be a set of samples from X. Then, 4g E (0, 1), with probability least 1-6 8, we have:
Cs(D;Y,,,)))
(1+ Rad(Dx) + //2 log 8/6.
(97)
Proof.
Cs(D; Y2) s (uRad(D;) + Rad(D2)) + B // 2/4
(98)
[uRado (92,) + Raddy (9 / log 2/8
(99)
< 2 [uLip(9r.,) Radl) + B / 2/N log 2/6
Rad
(100)
2 [u(1 + +7) 7)+ Rad(Dx) + B B 2//N log 2/8.
(101)
Eq. 98 uses Theorem 3 Eq. 100 uses the Kakade & Tewari Lemma previously introduced. Eq. 101
uses Lemma 2 in Behrmann al. (2019), where for any functions and g. such that f z+g
with Lip(g)
7 Then, it holds that Lip(f) < + The Lipschitz constant for a convolutional
layer with I-Lipschitz activations (e.g. ReLU) defined in Truong (2022) and shown in Eq. 96
We
also use Lip(g f) Lip(g)Lip(f)) for any Lipschitz functions and J.
C.I ANALYSIS
Theorem shows that, when * 0, the compatibility between representations Yı and Yg decreases with the depth of the ResNets producing them. For representations that share neural network, such that the representations Y1 and Y2 are splits of its output, X corresponds to the output of the next to last layer and hence the neural network producing Y; and Y2 would only have one layer, increasing their compatibility.
In our problem space, it is desirable for the common representation be compatible with the pri- vate representations. The Separable approach has a dedicated ResNet for each representation, which produces less compatible representations compared to our approach, which each private represen- tation shares most of ResNet with copy of the common representation.
In the Combined approach, all representations share most a ResNet, which increases compatibility among them, but constraints the private channels to produce similar information since they share many of the intermediate representations. compensate for this limitation, but it would be more inefficient than the proposed method, since the kernels would have to be larger in order to accommodate for the larger dimensionality, and parts of the input for each layer might be irrelevant some of the target representations.
Combined approach with higher dimensionality could
19

---

Published as a conference paper at ICLR 2026
Analysis Transform
Cunv. (94, 1/95)
Resiiet (3A)
ResNet (3R)
Conv: (2ga, 2/250
ResSict (SAO
Coews a/ 2/25))
(ptt 4/a5)
(a) Analysis transform
Synthesis
Conv, (192, aS)
ResNet (3X)
Cony. (48, as)
ResSeet (2)
Conv. (2gp, 25)
ResSlet (2A)
Cocyy (M, 283)
Task Model
In
Out
(b) Synthesis transform
Figure 6: Architecture diagrams of the analysis and synthesis transforms. The number of channels in the input is denoted as M. The values assigned to S corresponds to multiplying change in the spatial dimensions. The values assigned to X represents the number of blocks. The values 24, 48, and 192 correspond to the number of channels in the output representation of the module.
Massed
(aF)
concat
Conv, (3X, r/aF]
Ny.
ResNet (aX)
ResNet (2X)
(ail)
Masked (283
Conv. (3X)
NW.
ResNet (aX)
Comw: (aE)
ResNet (2X)
Masked
Cony, 1/28]
A's
Conv. (213
Figure 7: Architecture overview of the three entropy models. Ny represents the multivariate normal parameters with diagonal covariances assigned to 100. They define By The values assigned X represents the number of blocks. The values assigned to correspond a multiplying change in the number of channels. For example, "Conv. (3X)" corresponds 3 convolutional layers, and "Conv. (2E)" corresponds to a convolutional layer that doubles the number channels.
D ADDITIONAL DETAILS AND RESULTS
D.1 ARCHITECTURE DETAILS
Figure 66 shows architecture diagrams of the analysis and synthesis transforms. Figure 7 shows an architecture diagram of the three entropy models used in the proposed method. All ResNets EL.U activation functions. The number of channels in the intermediate representation in the ResNet blocks have the same number of channels as the input (and output).
Figure g presents an overview of the codec architectures used as benchmarks. The entropy model architecture used for the three-channel codecs is the same one described for the proposed architec- ture. The two-channel and one-channel codecs have the same entropy model architecture as the one used for the Y, representation in the proposed architecture.
20

---

Published as a conference paper at ICLR 2026
Analois Trunsform
Y
Yi
concat
Symtheris Transform
2
Syuthesis Trunsfrm
Analyst Transform
Analoii Transform
split
Y
Syuthesis Transform
concat
Syuthesis Trunsform
2
concat
Y2
Analysis Trunsform
%
(a) Combined
(b) Separated
Transfform
Anslysis
Yi
Synthesis Transform
2=
Transform
Synthesis
2,
Analysis
Transform
X
Transform
Amalsii
Y5
Transform
Synthesis
L
Synthesis Transform
(c) Independent
(d) Joint
Figure 8: Architecture diagrams of the codees used benchmarks. The Combined and Separated approaches have three channels each, and are considered different versions of a Gray-Wyner Net- work. The Independent codec has two channels, and the Joint codec only has one. For simplicity, the entropy models that code (,0,1,2) are omitted.
Table Hyper-parameter and training settings
Parameter
Synthetic
MNIST
Cityscapes
COCO 2017
Precision
float32
float32
float32
float32
Input representation channels (M)
2
3
Target representation channels (E)
512
192
192/288
Auxiliary loss weight (7)
1
Reconstruction loss weight
0
O
Random horizontal flip
No
No
Yes
Yes
Color jittering
No
No
Yes
No
Batch size Accumulated gradient batches
100
100
1
Optimizer
Adam
Adam
Adam 0.9, 0.999 0.0001
Adam 0.9, 0.999
Optimizer parameters
0.9, 0.999 0.0001 0
0.9, 0.999 0.0001
Learning rate
0.0001
Weight decay
0
Gradient norm clipping
Patience
25
50
Maximum number of epochs
100
200
1,000
100
D.2 TRAINING DETAILS
Since the task losses (distortion functions) used are on a similar scale, we only consider rate- distortion points where X Ag. Thus, scale the optimization objective L by 7 !/Acaa to obtain a single
controlling the rate-distortion tradeoff.
Table shows the training details used in all experiments. Additional considerations for repro- ducibility are addressed in other related sections.
D.3 TRANSMIT-RECEIVE TRADEOFF AND ABLATION STUDY
Elements of random variable with sample space (-4..., 4] 4]2x64x64 are identically distributed according probability mass function (PMF) created from the normal distribution, A00 4), which
21

---

Published as a conference paper at ICLR 2026
BD-Rate Bopependens (Empiricat)
Joint (Emapzioal)
Joint (T.coreticall: 56,
Dedependent (Thoeroticall: 95.53
20M5U Sum
Supanated: 45.67
Shured: 7.64
Sum
0.8
Shared: 17.76
Suparuted: 68.59
Combined 74 M4
Comhined 101 552
0.4
0.4
0.2
0.2
0.0
0.0
10
12
0.0
2.5
5.0
7.5
10.0
12.5
15.0
17.5
Transmit (BPP)
Recetvve
(a) Transmit rate for 3/2
(b) Receive rate
3/2
Mathod: 8D-Rate (8)
Mathod: Rato 196)
0.8
-+- Independicat (Enpatical)
Mutual Information (E (Emppirical
Indegendeen (Teeoretical) 80,
Mutual Information (Theoretical:: 93.64
10A350 5tmm
0.6
Shared: 12.28
30ASE Sum
10.6
Shared: 40.01
Combined:
0.4
0.4
0.2
0.2
12.5
15.0
17.5
Rate (BPF)
(BPF)
(c) Receive rate for 3 =
(d) Common information for /3 =
(%)
Mutual Information (Empiitoal)
Matual Information (Empiricul)
Mutual Information (Thcoretical) 88.94
Mutual Information iTheoretical:: 33.64
20355 Samm
Slarred -17.88
ROMSE Sum
Shared -68.88
0.8
Sepurated: 40.27
8.8
Separuted: -57.38
Combtted -29.16
Combined, -58.44
0.4
0.4
0.2
6.2
0.9
0.0
Common Channel Rate (BPP)
Common Channel (BPP)
(e) Common channel rate for 3 3/2
(f) Common channel rate for
Figure 9: Additional rate-distortion curves for different methods, optimized for transmit, receive, and mixed rates. The common information axes correspond to the bitrates the common channel, the theoretical measurements of mutual information, and its empirical estimates. BD-rates (Bjon-) tegaard, (2001) are computed with respect to the method with no score. Bits-per-pixel (BPP) is the bitrate scaled by 1/64x64.
Table 2: Rate-distortion performance of baselines synthetic dataset
Codec
8
R1
R₂
R
R
R
D1
D2
Independent
N/A
0.001
8.699
8,913
0
17.612
17.612
0.040
0.039
Independent
N/A
0.01
5.047
5.424
10.471
10.471
0.046
0.048
Independent
N/A
0.1
2.077
2.059
4.136
4.136
0.152
0.151
Independent
N/A
0.5
0.656
0.655
0
1.311
1.311
0.381
0.381
Independent
N/A
1.0
0.287
0.334
0
0.621
0.621
0.459
0.460
Joint
N/A
0.001
9.745
9.745
19.49
0.042
0.043
Joint
N/A
0.01
0
0
8.096
8.096
16.192
0.048
0.048
Joint
N/A
0.1
4.230
4.230
8.46
0.131
0.132
Joint
N/A
0.5
0
0.983
0.983
1.966
0.402
0.406
Joint
N/A
L.O
0.429
0.429
0.858
0.452
0.451
first clipped to [-4, 4], and then quantized to 9 symbols using the integers in the range. Contiguous element pairs were either selected to be copies, with probability 4/s, independent, with probability 1/s. The random variable positions were then shuffled and the resulting dependency map was stored and used to generate all samples from X.
22

---

Published as a conference paper at ICLR 2026
Table 3: Rate-distortion performance of proposed methods on synthetic dataset for _
Codec
B
"
Ri
R2
Ro
R₂
R2
D1
D2
Shared
0.001
3.747
4.419
2.278
10.444
12.722
0.042
0.041
Shared
1
0.01
3.110
3.639
1.622
8.371
9.993
0.046
0.048
Shared
0.1
1.504
2.056
1.091
4.651
5.742
0.128
0.126
Shared
1
0.5
0.247
0.509
0,410
1.166
1.576
0.403
0.403
Shared
1.0
0.050
0.230
0.180
0.460
0.640
0.461
0.456
Separated
0.001
4.287
4.604
4.563
13.454
18.017
0.036
0.035
Separated
0.01
2.729
3.201
3.224
9.154
12.378
0.042
0.042
Separated
0.1
1.781
1.792
1.384
4.957
6.341
0.129
0.128
Separated
0.5
0.212
0,233
0.205
0.650
0.855
0.445
0.445
Separated
1.0
0.106
0.134
0.088
0.328
0,416
0.467
0.462
Combined
0.001
4.130
4.029
4.778
12.937
17.715
0.039
0.040
Combined
0.01
2.807
2.749
3.481
9.037
12.518
0.044
0.045
Combined
0.1
1.880
1.794
1.230
4.904
6.134
0.131
0.131
Combined
0.5
0.233
0.201
0.339
0.773
1.112
0.443
0.447
Combined
1.0
0.095
0.097
0.086
0.278
0.364
0.471
0.470
Table 4: Rate-distortion performance of proposed methods on synthetic dataset for B 3/2
Codec
B
R₁
R2
Ro
R₂
R,
D2
D22
Shared
%/2
0.001
3.662
4.363
2.116
10.141
12.257
0.043
0.043
Shared
4/
0.01
3.238
3.698
1.392
8.328
9.720
0.047
0.046
Shared
Y/2
0.1
2.007
2.125
0.373
4.505
4.878
0.137
0.136
Shared
4/2
0.5
0.295
0.468
0.215
0.978
1.193
0.413
0.409
Shared
3/2
1.0
0.066
0.222
0.104
0.392
0.496
0.471
0.455
Separated
4/2
0.001
4.518
4.746
4.395
13.659
18.054
0.034
0.035
Separated
3/2
0.01
3.382
3.295
2.465
9.142
11.607
0.044
0.045
Separated
4/2
0.1
2.052
2.118
0.604
4.774
5.378
0.143
0.145
Separated
4/2
0.5
0.277
0.269
0.157
0.703
0.860
0.435
0.441
Separated
$/2
1.0
0.140
0.132
0.049
0.321
0.370
0.463
0.464
Combined
4/2
0.001
4.254
4.071
4.788
13.113
17.901
0.039
0.038
Combined
3/2
0.01
3.618
3.591
1.799
9.009
10.809
0.044
0.044
Combined
$/2
0.1
1.996
1.957
0.826
4.779
5.605
0.137
0.137
Combined
4/2
0.5
0.215
0.216
0.126
0.557
0.683
0.451
0.451
Combined
4/2
1.0
0.083
0.078
0.041
0.202
0.243
0.476
0.477
Two bijective linear functions of determinant are arbitrarily generated for X1 and X2, respectively. The output of these transformations are the corresponding targets Z; and Zg for two linear regression tasks. We dynamically generate a dataset of infinite samples from Px, Px,z, on the fly.
The mutual information I(X1; : X2) increases with the number of elements in X, that are duplicated in X2, Using the Blahut-Arimoto algorithm (Cover & Thomas (2066), we generate theoretical rate- distortion curves for X. The resulting PMF of the input reconstruction X allows the computation of H(2, 22), (21) + H(2)), and [2;; 22), for the corresponding distortion values D, and Dz-
To obtain empirical estimates of mutual information, we use the rate of the single channel the Joint method as empirical measurement H (2,, 2z). We use the rates of the Independent method em- pirical values H (2,) + / (22). . To compute an empirical estimation the mutual information be- tween tasks, interpolate and extrapolate the points within the rate distoriit curves of these meth- ods, covering all distortion points. Then, we compute Í [(2;; 2z) = ((z)) + ((2)) H(2,, 2z) between points with the same distortion value.
To improve task performance, all scaling convolutional layers are replaced with pixel shuffle opera- tions (Shi et al. 2016), removing dimensionality bottlenecks in the architecture. Each pixel shuffle
23

---

Published as a conference paper at ICLR 2026
Table 5: Rate-distortion performance of proposed methods on synthetic dataset for _
Codec
B
"
R1
R2
Ro
R₂
R2
D1
D2
Shared
2
0.001
3.656
4.48
2.137
10.273
12.41
0.044
0.041
Shared
2
0.01
3.332
3.777
1.291
8.4
9.691
0.049
0.046
Shared
2
0.1
2.116
2.139
0.133
4.388
4.521
0.143
0.147
Shared
2
0.5
0.305
0,475
0.136
0.916
1.052
0.424
0.41
Shared
2
1.0
0.066
0.239
0.087
0.392
0.479
0.474
0.455
Separated
2
0.001
4.494
4,474
4.615
13.583
18.198
0.034
0.035
Separated
2
0.01
3.748
3.535
1.881
9.164
11.045
0.044
0.045
Separated
2
0.1
2.016
2.015
0.366
4.397
4.763
0.16
0.166
Separated
2
0.5
0.168
0.319
0.099
0.586
0.685
0.46
0.428
Separated
2
1.0
0.075
0.115
0.029
0.219
0.248
0.478
0.47
Combined
2
0.001
4.215
4.095
4.854
13.164
18.018
0.036
0.037
Combined
2
0.01
3.863
3.94
1.366
9.169
10.535
0.045
0.045
Combined
2
0.1
2.11
2.042
0.545
4.697
5.242
0.137
0.138
Combined
2
0.5
0.221
0.207
0.09
0.518
0.608
0.451
0.46
Combined
2
1.0
0.099
0.094
0.033
0.226
0.259
0.473
0.474
Table 6: Rate-distortion performance of the proposed method on colored MNIST
Codec
R1
R2
Ro
R₁
R,
I/D.
1/Dz
Dependent
2.908
5.779
15.84
24.526
40.366
0.999
0.999
Dependent
10
1.460
4.071
10.641
16.172
26.812
0.998
0.999
Dependent
25
1.571
2.637
10.524
14.731
25.255
0.996
0.998
Dependent
50
1.267
2.500
9.921
13.688
23.609
0.995
0.996
Dependent
75
2.186
0.826
10.366
13.378
23.743
0.992
0.989
Dependent
100
1.067
2.318
8.991
12.376
21.368
0.975
0.986
Independent
22.269
6,864
0.071
29.203
29.273
0.989
0.999
Independent
10
15.974
7.030
0.056
23.060
23.116
0.983
0.999
Independent
25
13.568
5,656
0.017
19.241
19.257
0.975
0.996
Independent
50
12.073
5.776
0.019
17.868
17.887
0.970
0.969
Mixture
19.23
8,425
5.149
32.804
37,952
0.993
Mixture
10
9.322
5.349
10.966
25.637
36.603
0.992
Mixture
25
9.965
4.684
10.025
24.674
34.699
0.989
0.999
Mixture
100
9.547
3.660
9.629
22.835
32.464
0.975
0.997
Mixture (Independent)
27.262
15.658
42,919
42.919
0.991
0.999
Mixture (Independent)
10
16.182
18.592
0
34.773
34.773
0.988
0.999
Mixture (Independent)
25
16.031
15.464
31,495
31.495
0.979
0.999
Mixture (Independent)
50
14.913
13,892
0
28.805
28.805
0.977
0.998
Mixture (Independent)
75
13.804
14.685
28.489
28.489
0.967
0.994
or unshuffle operation has factor of 2, which implies that the analysis transform progressively in- creases the number of channels from to 8, 32, 128. and finally E 512. The synthesis transform decreases the number of channel reverse order. We use 1,000 steps per training epoch and 100 per validation epoch.
Figure shows additional results. The transmit and receive rate plots show that the Shared (pro- posed) method outperforms the Separated and Combined baselines. The common channel rate plots show in detail how 3 controls the amount of information in the common channel. Tables 2 3 show the values used in these plots. All rate values bitrates scaled by 1/84x64.
24

---

Published as a conference paper at ICLR 2026
Original
g
8
7
7
5
2
3
6
8
2
8
6
/
Private digit
9
3
3
9
9
Common
8
8
1
6
7
7
5
2
3
5
0
/
8
2
8
Private color
3 2
2
(a) Dependent PMF
Original
3
1
8 8
0
9
2
O
2
3
0
9
Private digit
3
8
8
0
7
9
O
0
2
9
Common
Private color
(b) Independent PMF
Original
6
5
1
2
2 6 9
2
5
2
4
3
3
2
6
/
7
Private digit
5
O
2
2
6 9
1 2
5
2
3
3
2
6
/
8
7 7
Common
5
/
6
2
1
7
1
7
6
/
4
6
3
Z
6
8
Private culor
(c) Mixture PMF
Figure 10: MNIST image reconstructions from individual channels Y(u produced by codecs trained on one of the 3 different PMF's discussed. The Dependent PMF places most information on the common channel. The Independent PMF places the digit and color information on the corre- sponding private channels. In this codec, the common channel does not seem to carry any informa- tion. The Mixture PMF shows a combination of information across channels. The correct answer is usually found in combination of channels.
D.4 EDGE-CASE EXPLORATION WITH IMAGE CLASSIFICATION
We set the number of dimensions to E 6. Due the low rate of the channels, the auxiliary loss term in Eq. [5] which encourages the common representations Y, and X, (2) to match, prevents the common channel from carrying any significant amount of information. We overcome this ob- stacle by setting 8 = 1/10, encouraging the use of the common channel, and compensating for the restrictions of the auxiliary loss.
Table 6 shows the values obtained for the different PMFs and methods. Rates are shown as bitrates (using log functions with base 2). The inverse distortions I/D, and I/D, are the top-1 classification accuracies. The Independent method has very low rate on the common channel, as it would be optimal do so. The Dependent method places some information on the private channels, but is on average 29. 29.22% of the transmit rate.
To visualize the information available in these channels, we train multiple tasks to reconstruct the input image, each using the representations of channel as input. The model architecture is the same as the proposed synthesis transform. The kernel sizes of the first and last layers are adapted to dimensional requirements of the input and output, respectively. We train to minimize the RMSE between the predictions and the input image, using the same settings described for MNIST Sec- tion D.2 We generate reconstructed images for each PMF from corresponding codecs that produce the lowest rate.
Figure 10 the results from uniform samples of the validation set. For the Dependent and Inde- pendent PMFs, it is easily observed that the available information matches the quantitative results. In the results from the Mixture PMF, the color channel does not contain much digit information. The digit channel contains some color information but it often incorrect. Whenever a private channel is incorrect, the common channel seems to contain the correct answer.
25

---

Published as a conference paper at ICLR 2026
Table Rate-distortion performance the proposed method on Cityscapes
Codec
!
R1
R2
Ro
R₂
R2
1/Dz
D2
Joint
0.001
2.444
2.444
4.888
0.670
5.493
Joint
0.01
0
0
2.069
2.069
4.138
0.669
5.489
Joint
0.05
1.611
1.611
3.222
0.670
5.497
Joint
0.1
0
1.527
1.527
3.054
0.669
5,496
Joint
1.074
1.074
2.148
0.663
5.497
Independent
0.001
2.113
2.296
4.409
4.409
0.666
5,477
Independent
0.01
1.808
1.892
3.700
3.700
0.668
5.480
Independent
0.05
1.227
1.337
0
2.564
2.564
0.665
5,493
Independent
0.1
1.104
1.191
2.295
2.295
0.663
5,493
Independent
0.788
0.846
1.634
1.634
0.660
5.513
Proposed
0.001
1.384
1.482
0.330
3.196
3.526
0.666
5,482
Proposed
0.01
1.329
1.415
0.243
2.987
3,230
0.669
5.487
Proposed
0.05
1.217
1.285
0.329
2.831
3.160
0.669
5,488
Proposed
0.1
1.110
1.151
0.414
2.675
3.089
0.669
5.487
Proposed
1.0
0.568
0.568
0.409
1.545
1.954
0.666
5.488
Proposed
5.0
0.315
0.311
0.319
0.945
1.264
0.662
5,496
Table 8: Rate-distortion performance of the proposed method on COCO 2017
Codec
1
R1
R2
Ro
R
R
1/D.
1/D₂
Joint
0.001
0
3.513
3.513
7.026
0.453
0.646
Joint
0.05
3.198
3.198
6.396
0.454
0.643
Joint
0.1
0
2.997
2.997
5,994
0.453
0.642
Joint
1.980
1.980
3,960
0.449
0.638
Independent
0.001
3.676
3.543
7.219
7.219
0.453
0.645
Independent
0.01
3.498
3.264
6.762
6.762
0.453
0.644
Independent
0.05
3.113
2.973
0
6.086
6,086
0.454
0.643
Independent
0.1
2.855
2.768
5.623
5.623
0.453
0.643
Independent
1.615
1.605
3.220
3.220
0.448
0.635
Proposed
0.001
2.725
2.681
1.482
6,888
8,370
0.454
0.646
Proposed
0.5
1.532
1.559
1.002
4.093
5.095
0.455
0.646
Proposed
1.230
1.250
0.842
3.322
4.164
0.452
0.643
Proposed
2.5
0.822
0.845
0.659
2.326
2.985
0.449
0.637
D.5 RATE-DISTORTION PERFORMANCE COMPUTER VISION TASKS
For semantic segementation on Cityscapes, use pretrained weights provided by Chen al. (2018)). For the object detection and keypoint detection tasks on COCO 2017, we use pretrained weights provided by Torch Vision maintainers and contributors (2016). The depth estimation model is trained from scratch without rate constraints and then used frozen pretrained model in the proposed method. Since the size of the Cityscape images is quite large, for tractability, we learn downsampler of the input images and an upsampler of the output predictions of the task model, for the semantic segmentation and depth estimation task models. The task distortions are still computed against the original Cityscapes targets. The factor of the samplers is 2.
We set E = 192 for all models except those trained on COCO 2017 with the proposed method, which required E 288. Since the task models frozen, reconstruction loss between the input and the output the synthesis transforms, before the task models, improves rate-distortion performance. The weight of this reconstruction loss is listed in Table
We note that the rates produced by our methods are relatively large. This is attributed to the synthesis transform being connected the first layer of the task models and them remaining frozen, effec-
26

---

Published as a conference paper at ICLR 2026
tively encouraging the reconstruction of the input images. Since the images are highly compressed JPEG images, they contain many artifacts that might need to be reconstructed, requiring additional rate. Interestingly, the imput reconstruction loss seems to improve rate-distortion performance, even though intuitively one might think it would not.
Tables and show the values obtained by our proposed method against the Independent and Joint baselines. All results were collected from validation sets. 'he rates are reported in bits-per-pixel (BPP). In the Cityscapes experiments. 1/D, and D2 correspond the mloU and RMSE metrics, respectively. In the COCO 2017 experiments, I/D, and I/D, correspond to the object detection mAP and keypointing mAP metries, respectively.
All experiments are trained with / = 1, which, as previously discussed, due to the constraints of the auxiliary loss, does not necessarily optimize for the transmit rate. In fact, since the results show that the receive rate of the proposed method is better than the Independent method, the common channel must not contain all empirical mutual information possible.
27


Reviews:
[{"review_id": "zSRTJg77k4", "reviewer": "Reviewer_nk8B", "summary": "This paper extends the Gray–Wyner information theory framework into a learnable neural network for efficient multi-task and multi-view compression. It introduces a neural Gray–Wyner network that disentangles data into a common channel (shared information) and private channels (task-specific details), allowing controllable trade-offs between joint transmission efficiency and individual task performance. The authors derive new lossy common information bounds, connect them to Wyner’s and Gács–Körner’s classical measures, and design a trainable objective that optimizes this trade-off via a single parameter. THey experiment on synthetic data, a colorized version of MNIST,  and on real datasets including COCO and cityscapes. They show that the proposed approach achieves significant compression gains and better representation sharing compared to independent or naive joint codecs.", "questions": "It might be valuable for the community to have a better \"wrap-up\" of the paper. Its hard to see what this contributions enables in the space of deep learning practice that wasnt already possible with a less principled method. Does this enable predicting something fundamentally new or solving a new class of problems? Or is it just a theoretical refinement and crystalization of what practitioners have been heuristically doing.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Nice theoretical generalization of Grey Wyner coding to lossy compression and deep function approximators\n- Nice connection between multi-task representation learning and information theory\n- The authors Experiments on real world datasets like COCO and Cityscapes is appreciated and nicely complement the theoretical analysis\n- Thorough theoretical analysis and high quality proofs in a detailed appendix\n- Thorough reporting of architectures and parameters used", "weaknesses": "- A bit dense and difficult to follow, assumes alot of prereqs in info theory, perhaps consider adding an appendix of preliminaries to help readers from outside the direct field of GWC understand why this is meaningful and useful\n- Though they compare against some baseline codes (joint and independant) the work exists largely outside the realm of modern computer vision representation learning and multi-task benchmarks. Some of this can be forgiven because its largely a theory paper, but some connection or comparsion to other representation learning schemes out there in the literature might make this paper more relevant for the community.\n- Its unclear whether these experiments are dependant on the particulars of thenetwork architecure used, as its a bespoke set of networks and not a backbone that appears elsewhere, also not using modern tools like transformers hinders some of the relevance of the experiments. \n\n- nit: overlapping text in Theorem 1 like 175 + 176  D_1 overlaps with Z1 hat\n- Willing to improve my score", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eY7w5S5OHv", "reviewer": "Reviewer_czUz", "summary": "This work proposes a learnable three-channel encoder-decoder based on the Gray-Wyner network from information theory, designed to disentangle shared information and task-specific information in multi-task visual scenarios. The method achieves promising results across multiple benchmarks and settings.", "questions": "See the Weaknesses.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. By analyzing the rate bounds of joint and independent encoding, the method clarifies the feasible conditions for disentangling shared information.  \n2. A Lagrangian-relaxed optimization objective is designed, allowing the amount of information in the shared channel to be dynamically controlled by the hyperparameter β, supporting a continuous trade-off from transmission-rate optimal (β=1) to reception-rate optimal (β=2).  \n3. Shared encoding layers extract common information, which is then split into private channels, ensuring representation compatibility while reducing redundancy.  \n4. The method supports direct integration with pre-trained task models, such as DeepLabV3+ and Faster R-CNN, achieving efficient encoding without the need to fine-tune the task networks.", "weaknesses": "1. Theorem 1 provides upper and lower bounds on the amount of shared information, but it is not specified how these bounds can be approached in a practical learning framework. Does this require specific architectural constraints or training mechanisms to ensure tightness of the bounds?  \n2. The method relies on a Markov assumption that “each source contains no exclusive information that benefits non-corresponding tasks” (Section 2.1). In real-world multi-task learning, cross-task information transfer is often beneficial. This assumption may be overly strict, potentially limiting the applicability of the method.  \n3. The entropy model for private channels uses the shared channel as context (Section 3.3), which may lead to leakage of shared information into private channels. Are there mechanisms in place to ensure the purity of information separation?  \n4. The experimental tasks are relatively simple (classification, regression, segmentation). It remains unclear whether the method remains effective for tasks requiring fine-grained spatial reasoning (e.g., instance segmentation) tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "g1bd6BMHQK", "reviewer": "Reviewer_wYFu", "summary": "This paper provides a theoretically grounded and empirically validated framework that connects Gray–Wyner information theory with modern learnable architectures. While the architectural novelty is moderate and writing can be improved, the theoretical novelty, empirical consistency, and conceptual clarity make it a strong contribution.", "questions": "See Weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "1.\tThe theoretical analysis appears rigorous and well-presented. I did not identify mistakes or inconsistencies in the proofs of Theorem 1 and Theorem 2.\n2.\tAddressing the challenge of distilling shared information in multi-task learning is both important and underexplored. The proposed learnable Gray-Wyner Network provides a natural and conceptually clear solution, supported by solid theoretical justification and empirical evidence.\n3.\tI appreciate the authors’ effort to draw inspiration from classical information-theoretic principles (i.e., the Gray-Wyner framework) and adapt them to modern machine learning contexts.\n4.\tThe empirical results are consistent with the theoretical analysis and convincingly demonstrate the effectiveness of the proposed method.", "weaknesses": "1.\tMy main concern lies in the presentation of the paper. The notations are dense and difficult to follow, particularly in Sections 2.1 and 3.1. The motivation behind the proposed method is somewhat unclear. Reorganizing the theoretical sections with more intuitive explanations and a clearer logical flow would significantly improve readability.\n2.\tAlthough the method is designed to “disentangle” shared information, the paper lacks visualization or analysis to demonstrate that the learned representations indeed capture task-common semantics.\n3.\tIt would be beneficial to include error bars in the experimental results to reflect the variance and enhance the reliability of the reported performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)