Approaching Rate-Distortion Limits in Neural Compression with Lattice Transform Coding
Authors: Eric Lei
Hamed Hassani
Shirin Saeedi Bidokhti
Venue: ICLR 2025 Spotlight (2025)
Keywords: Neural compression
vector quantization
lattice quantization
nonlinear transform coding
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Neural compression has brought tremendous progress in designing lossy compressors with good rate-distortion (RD) performance at low complexity. Thus far, neural compression design involves transforming the source to a latent vector, which is then rounded to integers and entropy coded. While this approach has been shown to be optimal on a few specific sources, we show that it can be highly sub-optimal on synthetic sources whose intrinsic dimensionality is greater than one. With integer rounding in the latent space, the quantization regions induced by neural transformations, remain square-like and fail to match those of optimal vector quantization. We demonstrate that this phenomenon is due to the choice of scalar quantization in the latent space, and not the transform design. By employing lattice quantization instead, we propose  Lattice Transform Coding (LTC) and show that it approximately recovers optimal vector quantization at reasonable complexity. On real-world sources, LTC improves upon standard neural compressors. LTC also provides a framework that can integrate structurally (near) optimal information-theoretic designs into lossy compression; examples include block coding, which yields coding gain over optimal one-shot coding and approaches the asymptotically-achievable rate-distortion function, as well as nested lattice quantization for low complexity fixed-rate coding.

Published as a conference paper at ICLR 2025
APPROACHING RATE-DISTORTION LIMITS IN NEURAL COMPRESSION WITH LATTICE TRANSFORM CODING
Eric Lei, Hamed Hassani & Shirin Saeedi Bidokhti Department of Electrical and Systems Engineering, University of Pennsylvania (elei, hassani, saeedi @saas upenn. edu
ABSTRACT
Neural compression has brought tremendous progress in designing lossy com- pressors with good rate-distortion (RD) performance at low complexity. Thus far, neural compression design involves transforming the source to a latent vector, which is then rounded to integers and entropy coded. While this approach has been shown to be optimal on few specific sources, we show that it can be highly sub-optimal on synthetic sources whose intrinsic dimensionality is greater than one. With integer rounding in the latent space, the quantization regions induced by neural transformations, remain square-like and fail match those of optimal vector quantization. We demonstrate that this phenomenon is due to the choice of scalar quantization in the latent space, and not the transform design. By employing lattice quantization instead, we propose Lattice Transform Coding (LTC) and show that it approximately recovers optimal vector quantization at reasonable complexity. On real-world sources, LTC improves upon standard neural compressors. LTC also provides a framework that can integrate structurally (near) optimal information- theoretic designs into lossy compression; examples include block coding, which yields coding gain over optimal one-shot coding and approaches the asymptotically- achievable rate-distortion function, as well as nested lattice quantization for low complexity fixed-rate coding.
INTRODUCTION
Neural compression, especially nonlinear transform coding (NTC), has made significant progress in advancing practical data compression performance on real-world data at low complexity (Ballé et al., 2021; Yang et al., 2022). In this work, seek understand when and how neural compressors may be optimal in terms of the rate-distortion trade-off. Obtaining such an understanding can provide insights on if and how neural compressors can be further improved, well as why they perform well.
To investigate this, it is important to understand the underlying principles behind NTC. Shown in Pa by first transforming
Fig. NTC operates on a source realization an analysis transform 9a- The latent vector is then scalar quantized to ý via Q, which is typically implemented via entry-wise rounding to integers. Then, synthesis transform ys transforms into the reconstruction á. Additionally, an entropy model Py estimates likelihoods of the quantized latent vectors ÿ. providing a coding rate estimate via its entropy. Together, Sa- Ie: and Pa are learned to minimize a rate-distortion loss,
latent vector y via
min Em log (n)) + [d(x, ()), 9a:Be:Po
where y ga (x), ÿ Q()). & ga ()). and À > 0 controls the rate-distortion tradeoff. NTC is considered a one-shot coding scheme of x. because compresses one source realization at a time.
Optimal compression of æ be performed by vector quantization (VQ) Gersho & Gray (2012) albeit at high complexity which grows exponentially in rate and dimension. NTC, on the other hand, has low complexity. We are interested in understanding if NTC recover VQ, and/or how its performance can be improved. If so, this would provide a lowccompeextty, source agnostic scheme capable of achieving the fundamental limits of compression.
Wagner Ballé (2021); Bhadane et al. (2022) demonstrate that on certain synthetic sources, NTC successfully achieves VQ performance. These sources nominally high-dimensional, but have

---

Published as a conference paper at ICLR 2025
(cB)
1/
0.5
1.0 1.5 20
(bits sample]
Figure 1: Left: Nonlinear Transform Coding (NTC) recovers scalar quantization (ECSQ) Gaussian i..d. sequences of any dimension n. Lattice Transform Coding (LTC) recovers optimal vector quantization (VQ) in each dimension, and approachees the rate-distortion function R(D) without needing an exponential codebook search. Right: NTC architecture. LTC replaces scalar quantizer Q with lattice quantizer QA-
a low-dimensional representation, which we refer to as the latent source. The authors show that Gas Ss can successfully extract the latent source, optimally quantize it, and map back the original space, providing an optimal compression scheme as a whole. However, most of the sources they analyze have a one-dimensional uniform latent source U ~ Unif (la,b. to analyze the role of Ia and Ds. it does not provide insights on the role of the quantizer Q. This because uniform source U can be optimally quantized using uniform scalar quantization (SQ), which is exactly what NTC uses in the latent space. For sources with higher-dimensioaa. latent sources, where uniform SQ is not necessarily optimal, it is unclear whether NTC can still achieve optimality. Conceptually, given the high representation power of neural networks, one may hope that the transforms Ia and Js. in conjunction with the basic integer rounding in the latent space, can implement more complex quantization such as VQ.
6)). While these are sufficient
In this paper, we first provide evidence that on sources with higher latent dimensionality, NTC performs sub-optimally: the learned transforms are insufficient overcome the sub-optimal latent SQ. Shown in Fig. 3, NTC quantization regions remain square-like (following integer rounding in the latent space), and do match the hexagonal regions from VQ. Our observation is also aligned with the recent analytical approach of Shevchenko et al. (2023) for two-layer VAEs, showing fundamental incapability in recovering VQ for Gaussian sources. To improve performance, we propose to use lattice quantization (LQ), which provides improved space-packki complexity compared VQ. We refer to this framework (i.e.. LQ in the latent space) as Lattice Transform Coding (LTCC). On a variety of synthetic and real-world sources, we demonstrate how LTC improves upon NTC in terms of the rate-distortion trade-off, while avoiding the high complexity of a direct codebook search under VQ. For example, on isotropic Gaussian vectors, Fig. shows how LTC achieves VQ performance without a codebook search.
efficiency over SQ, but reduced
The proposed framework of LTC further enables the integration of structurally-opting solutions from information theory. In particular, we show how LTC enables design of (1) lower complexity solutions using nested lattices; and (2) near-optimal solutions using block coding from information theory. Overall, our experiments support L.T. as a framework to achieve rate-distortion limits while mitigating high complexity. Our contributions
follows!.
1) We first demonstrate the inability of current NTC architectures optimally compress sources with latent dimensionality larger than one; we conclude this is due to the choice of SQ in the latent space, and not the transform design.
2) We propose Lattice Transform Coding (LTC), framework that replaces the latent integer rounding of NTC with general lattice quantization. LTC requires novel solutions for training, such as efficient lattice quantization algorithms, dealing with the non-differentiable quantization operation, integration of the entropy model over lattice regions, and accurate likelihood modeling.
3) LTC additionally enables integration of a variety of information -toormattionthhoretiic concepts into neural compression. Namely, we demonstrate how LTC allows the use of nested lattices in the latent space which provides a scheme with low encoding complexity, as well as implementation of block coding (necessary to achieve R(D)) which has applications in dataset compression.
"Code he found https: //t thub.
2

---

Published as a conference paper at ICLR 2025
4) We show the LTC framework near-optimally compress interesting synthetic sequences for which we have a benchmark of optimality (e.g. i.idd sources, the Banana source), while maintain- ing reasonable complexity. Our results reveal the interplay between nonlinear transforms, entropy models and the choice of L.Q.
5) For real-world sources with moderate to high dimensions, we demonstrate LTC's ability to consistently improve upon the performance of NTC when the latent dimension is larger than one, and bridge up 30% of the gap to R(D).
RELATED WORK
Px. the rate-distortion function R(D) describes the fundamental limit of compressing X (Cover & Thomas, 2006). Compressing the source one sample at time known as one-shot coding. R(D), on the other hand, is achievable asymptotically in the limit of large block-lengths when the source is compressed block by block and the rate and distortion are measured per-sample, For a general vector source æ. its optimal one-shot coding scheme can be found via entropy-constrainee vector quantization (ECVQ). ECVQ directly enumerates quantization centers (codewords) in the space of æ, and minimizes an
Rate-Distortion Theory. Given source X
objective,
min Ex | log P(e(=))) + X {
(2)
where e(x) := arg min (c:) d(x,,,), Since the solution is unknown in closed form, ECVQ is solved numerically by a generalized Lloyd's algorithm (Chou et al., 1989). Although ECVQ is optimal, an issue that precludes its wider use is its high complexity; the number of codebook vectors required is exponential in the rate and dimension. This complexity issue is one reason why NTC is appealing, since scalar quantization the transformed space is used, whose complexity is linear in the dimension of y. However, it is unknown whether NTC achieves optimality; this is an important question, since if NTC was optimal it would immediately offer a cwwmopleyxtty solution that achieves ECVQ performance. Our work reveals that NTC is not optimal; however, LQ offers an optimal low-complexity solution for NTC.
Nonlinear Transform Coding. The principles of nonlinear transform coding (NTC) are described in Ballé et al. (2021). Although NTC is based on VAEs with SQ in the latent space, several works have attempted apply VQ in the latent space instead. Notable works include VQVAE (van den Oord et al., 2017), which performs a direct codebook search in the latent space. PQVAE (El-Nouby et al., 2023) uses a product quantization scheme to improve scalability. Additionally, NVTC (Feng et al., 2023) applies ECVQ in the latent space, which also performs a direct codebook search. We show that LTC is able to recover the performance of ECVQ while avoiding direct codebook search in the latent space, which is infeasible for large rates and dimensions.
Lattice Quantization. A lattice quantizer QA (Conway & Sloane, 1999) consists of a countably infinite set of codebook vectors A given by uG, where G E = is the generator matrix, and u are integer vectors. The Voronoi region of lattice V(0)= {w R" Qx(x) 0}. where (x(x)= == arg min || 0 finds the closest lattice vector to æ in A. In lattice theory, the objective find lattices that minimize LA, the normalized second moment (NSM), which measures the lattice's 's packing efficiency. Recent work describing lattices which have the lowest known NSM can be found in (Agrell& Allen, 2023; Lyu et al., 2022: Agrell et al., 2024). Another objective is to find efficient quantization (decoding) algorithms (i.e., computing (2)). which requires solving the closest-vector problem (CVP); this is ha-hard in general. For many of the best-known lattices in dimensions up to 24 (n 2: hexagonal; n 4: D;,; 8: Gosset Es;; 16: A16 Marnes-Wall and n = 24: A21 Leech) efficient CVP solvers exist (Conway & Sloane, 1999).
Classically, lattices used to implement both fixed and variable-rate quantizers (Zamir et al., 2014). For the former, nested lattices shape the used part of the lattice and create a finite codebook with index-based encoding. For the latter, the entire (infinite) lattice is used, but entropy coding is applied; this is known as entropy-constrained lattice quantization (ECL-Q). Fixed-rate quantization trades lower encoding complexity for rate-distortion performance. For both cases, in the large rate regime, LQ achieves optimality when the dimension is asymptotically large. For example, for ECLQ,
lim 1 ((FFCI Rax (D)) = 2 log(ZT)Lx),
(3)
D-50
3

---

Published as a conference paper at ICLR 2025
where Ra(D) is the rate-distortion function of z and HECLO(D) (D) is the rate of ECLQ at distortion D (Zamir et al., 2014, Ch. 5). If G2 is the minimal NSM of any lattice in n dimensions, lim, G, = 1/(2Te), and thus (3) converges to O. For the integer lattice (SQ), the NSM is 1/12, which yields gap of s 0.255 bits per dimension. Thus, I.Q offers the maximal benefit when the rate-per-dimension is relatively small. While this may seem limiting, many real-world sources operate at low rate-per- dimension. For example, in NTC-based image compression, the rate-per-laters dimension is around the same as the bits-per-pixe used, which is typically between 0 and While classical transform coding may not have benefited much from LQ since the transform domain of images (e.g, DCT) is typically sparse, this is not true in NTC where the rate spread more evenly across all dimensions (Cheng et al., 2020; Bhadane et al., 2021) resulting in a lower bii-per-dimennionn for the quantizer.
Companding. Companding defines a function f R + R" known as the compressor, which is applied to the source æ E IR". (ax is then uniform lattice quantized, and passed to f-1. Clearly, NTC can be thought of as an instantiation of a compander where Ia. Is serve the role of f, f- Companding has typically been analyzed in the (asymptotically) large rate, small distortion regime. Optimal scalar quantization can be implemented using scalar companding (Bennett, 1948). For the vector case, vector companders cannot implement optimal VQ (Gersho, 1979; Bucklew, 1981; 1983), even when the best lattices are used, except for a very small class of sources. Despite this, it is still possible optimal companding may perform near-optimal VQ. This is supported by (Linder et al., 1999) which has a similar asymptotic result as (3). Due to this, we seek to utilize them in our learning-based framework and learn the companding function as guided by the companding literature.
Lattice Quantization and Neural Compression. The first work to integrate a lattice quantizer with NTC was LVQAC (Zhang& & Wu, 2023), which proposed the use of the D; lattice in the latent space, and showed some performance improvement over scalar quantization. However, LVQAC does not generalize to other lattices, and the D known to perform poorly compared to many other known lattices. In contrast, our approach is generalizable to tigh-perfomming lattices, and we further show how the performance of learned compressors depend on the space-packing efficiency of the lattice quantizer. LVQ-VAE (Kudo et al., 2023), a contemporary work to ours, seeks to exploit correlations among latent features by extending the hyperprior entropy model (Ballé et al., 2018) with a covariance matrix along quantized vectors. To mitigate the costly search over vector codewords, Kudo et al. (2023) uses lattice quantization in the latent space in a manner similar to our work. However, their work does not shed light on the role of lattice quantization in NTC. Moreover, their approach suffers from high complexity. In contrast, we seek to understand the precise role that quantization plays in NTC, and lattice quantization emerges We additionally show how nested lattices can help achieve optimality at much lower complexity. Finally, Cao et al. (2024) is another contemporary paper that provides low-complexity lattice VQ solution for image compression; however, details are sparse sin their manuscript for comparison.
solution achieve optimal rate-distortion performance.
NTC IS SUB-OPTIMAL
Let x ~ Ps be the vector source. We train an NTC model following Ballé et al. (2021) using factorized entropy model for the distribution of the latent y. We the objective in (1), with d(ær, .) = | 3.. and compare with (optimal) entropy-constraing We first consider the case when the entries of are i.i.d. with marginal S (O 1). Shown in Fig. 1, NTC with n = 1 is able to recover ECVQ with n = 1, also known as entropy-constrained scalar quantization (ECSQ). This is perhaps expected, since the task of the transforms is to scale the 1-D Gaussian to be optimally quantized on an integer grid in I-D, and then scale the quantized latent to form the reconstruction. Since the source is intrinsically dineemnnsi,, scalar quantization appears to be sufficient in the latent space to implement optimal quantization.
vector quantization (ECVQ).
For n = 2, however, NTC is unable to recover ECVQ, and always achieves the same performance as that of ECSQ. Shown in Fig. 2, we see that the rate-distortion performance for ECVQ at n = 2 is superior to that of ECSQ. However, NTC at n = 2 merely recovers ECSQ performance, and fails to outperform it. Visualizing the quantization regions on the 2-D Gaussian source (Fig. 3) reveals why. NTC is merely implementing a scalar quantizer of the source, perhaps with a rotation, which is equivalent to applying ECSQ the entries of æ independently. In 2-D, this induces square quantization regions in the source space. In contrast, the optimal ECVQ in n = 2 has hexagonal regions, not square. A similar phenomenon oecurs for the Banana source (Ballé et al., 2021). It
4

---

Published as a conference paper at ICLR 2025
ECSQ
XFCWG
SNR (dB)
8-
Hox LTC
4-
0.75
1.00
1.25
1,30
1.75
2,00
(bits per dimension)
Figure 3: Quantizer regions. Left: NTC. Right ECVQ.
Figure 2: Rate-distortion, 2-D Gaussians.
Top: 2-D Gaussian source. Bottom: Banana source.
is well-known that hexagonal tessellations yield superior packing efficiency compared square tessellations (Conway & Sloane, 1999). Thus, in 2-D, for NTC to be optimal, the quantization regions (induced by latent SQ and learned transforms) in the source space need to be hexagonal.
We found that when the latent space is rounded to integers, varying the depth, width, nonlinearities, biases, and even increasing the latent dimension resulted in square regions in the source space. For the i.i.d. Gaussian case, setting > 2 did not help either; all models recovered the performance of ECSQ best. This surprising observation reveals that the learned transforms Oe and Os seem unable to overcome use of scalar quantization. Moreover, confirms that when the latent dimensionality of the source greater than one, using SQ in the latent space, which is optimal for uniform scalar sources, does not result in optimality.
nigher-dimensiooal space did not help; all
Several related works the literature appear to support this observation. Shevchenko et al. (2023) analyzes 2-layer VAEs with form of scalar quantization in the latent space, and show that the optimal performance is given by SQ, not VQ. Results the companding literature (Gersho, 1979; Linder et al., 1999) additionally suggest that compressor such NTC is (i) unable to perform exact VQ for most sources, and that (ii) the performance is determined by the choice of the lattice in the latent space, which for NTC, is equivalent to the (sub-optimal) integer lattice.
To achieve VQ performance, these observations suggest that learned transforms are insufficient in overcoming sub-optimal quantization in the latent space, and that improving said latent quantization is required to achieve optimality. As will be shown, the proposed LTC is able to recover ECVQ regions and performance, with increasing gains over NTC as dimensionality increases (Fig. 1).
LATTICE TRANSFORM CODING
We seek compressor that can provide better performance than scalar quantization yet avoid the high complexity of vector quantization. Using a lattice quantizer QA provides an intermediate solution that manages both complexity and prace-filling efficiency. generalizes scalar quantization, which is equivalent to the integer lattice. It is known that many lattices can provide far more space-filling the integer lattice. We first discuss how extend current NTC designs to integrate the use of lattices. We then discuss method to reduce the entropy coding complexity of LTC by using nested lattice quantization. Finally, we discuss how LTC enables one to approach the asymptotic dite-distotiin function R(D) for any source.
efficiency compared
4.1 FROM SCALAR TO LATTICE QUANTIZATION
The primary challenges when integrating lattice quantization with (1) include computing the quanti- zation itself and computing integrals over the lattice cell for likelihood modelling.
Performing lattice quantization. The forward-pass the lattice quantizer QA requires the CVP algorithm of the lattice; exact algorithms use for the lattices in this paper described in Sec. A.1.
To allow differentiation for the backwards pass, we follow standard practices in NTC and use either the traiigh-tthoouge estimator (STE) dithered quantization with uniform noise the lattice cell. The STE approach can use (1) directly as an objective during training, but may lead to training instabilities. The uniform noise proxy has stabler training, but requires rate-distortion proxy loss of
5

---

Published as a conference paper at ICLR 2025
(I) using connection to dithered quantization (Ballé et al., 2021). We also assume that the lattice's generator matrix G has been normalized to be unit volume, i.e. (o)( = det((GTT) 1.
We use STE for the distortion term E. (d 9- (ga(())))))) Specifically. we compute the quantized output Q9 (x)) during the forward pass via
y + ceop__gradeent/e (Qx y)-
The rate term, however, requires more care.
Estimating the rate term. For the rate term, we typically have some parameterized density model Py for the continuous latent y (more details on this in the next section). This induces a PMF over the quantized latents Pý. given by (0) = S Pu(y)dy, where V == {y Qx = 9)-
In NTC, where QA implements the integer lattice, Pt( is easy to integrate when Py is assumed factorized (Ballé al., 2018). In that case, Py (ÿ) can be written in terms of the CDF of py, which is one reason why the factorized Py is typically parameterized by its CDF in NTC. However, for lattice quantization, solving the integral in (t) is not as straightforward.
As result, we compute PÚj (ū) via Monte-Carle estimate,
I Sr Pu()) Pu Pu(y)dy = Eu Um + u²),,
Pu Py(y)
(v( =
(5)
where Pu is the uniform density over (g)). Vol(V(()) = 1 since we assumed unit-volume lattices, and the last equality holds because for lattice quantizer Qa + v) = Qx)x) u. Vo A. From Conway & Sloane (1984), we can sample u ~ (V))((())) by computing u = -- 2 where ū sG, and s (ni 1)). Thus, (5) can be estimated via sample mean. With the ability to reliably estimate Pg ( we can also use STE for backpropagation of the rate term Eg log Pe [g)] by using (4). Prior works have achieved benefits from using the dithering approach (Ballé et al., 2021). In this case, random dither u' ~ Uni((V(O)) is applied prior to quantization, with the resulting rate B. [- log Py (Qa (y- u') ))))]. This rate equivalent in the integer lattice to E,, 1- log Pu Py + u')] (Ballé et al., 2021). The density Py is convolved density with likelihoods given by
Py+ (v)) - [ Pul - -2:--z))) ''' (u') = EE' Euu'y (on) (on Py - =u'],
(6)
which can be estimated via Monte-Carlo. To apply this to LQ during training, rather than provide the STE of the quantized output ÿ to the flonte-Carly likelihood estimate, we provide the noise- added continuous latent y + u', where u' is uniformly distributed over the Voronoi region. In our experiments, we observed that STE was less stable than the dithering approach, but had better performance at low rates, as will be shown in Sec. 5.
We note that concurrent work (Kudo et al., 2023) utilizes similar approach to compute the rate term for the use of general lattices in image compression. Their goal was to improve coding performance by removing correlation among latent features, whereas we seek to understand the fundamental nature of quantization in nonlinear transform coding. Comparing to other related work such as LVQAC (Zhang & Wu, 2023), we show later how their use of D lattice results in fundamental suboptimality.
Entropy Models. The rate term, Exe [ (o)) which is the cross-entropy between the true PMF on the lattice vectors P == PI (x) ÿ) and (D). is an upper bound on the true entropy of P(j). [ log P(O).. with equality if the learned PMF matches the true PMF exactly, Pa ()) = P()). Using the cross-entropy as an estimate of the true entropy critical due to the limitations of entropy estimation. Whereas direct entropy estimation requires a number of samples exponential in the rate, cross-entropy upper bounds are able to provide accurate estimates at large rates without requiring exponential samples (McAllester & Stratos, 2020). We cannot always expect the bound to be tight, and we will show that the choice of entropy model does play role in whether optimality can be achieved in Sec. 5.3. Since we are now using a more general Monte-Carlo integration for the likelihoods of ÿ. we are no longer tied to a factorized density and instead propose to parameterize the PDF Py in multivariate fashion using a normalizing flow (Kobyzev et al., 2020).
4.2 TOWARD LOWER COMPLEXITY
However, as the dimension of the lattice increases, entropy coding becomes challenging due to the large alphabet size. Large alphabet entropy coding is indeed an active area of research. Another

---

Published as a conference paper at ICLR 2025
approach that circumvents the need for entropy coding altogether is by quantization using nested lattices. To implement a fixed-rate code in the latent space using lattices, we use a nested lattice in the latent space. The idea is to the Voronoi region of coarse lattice Ag to define bounded region of a fine lattice A The fixed number of codebook vectors of A contained encode the source via index coding (and thus fixed rate). This happens when the majority of the latent mass is placed in the Voronoi region of the coarse lattice. Efficient encoding/decoding is enabled by transmitting the index of the coset, yielding complexity of (n) for an alphabet of size O(2"), compared to entropy coding which will be O(2"). To quantize the latent y. we compute ÿ ys Qs, ( where ys Q2, (y). This operation
the said
region are the ones wish to
selects the coset leader corresponding y inside the Voronoi region of A.. The rate is thus given by loga A = rd,, where |A1/Mr /2 is the number of relative cosets, dy is the dimension of y. and r is the nesting ratio (Zamir et al., 2014).
The majority of the latent mass is placed in the Voronoi region of A. when QA (yr) = 0 occurs with high probability. Otherwise, we have an error event == {Qx, (yr) + 0). Let P. = Pr(E). The total distortion can be broken up into two terms (Zamir et al., 2014, Ch. 10),
Ex (xx, +)] = (1 - Pe) Ez d(æ,, a)}E*} + Pe Ez (d a)[8].
(7)
In order to train LTC with latent NLQ, need to be able to differentiate through these terms. We use a normalizing flow density Py to estimate the P. with Monte-Carlo integration.
( vv
Pa
Po(yddy = Vol(V.(0)) Bu--Va (w)).
(8)
where Vo(0) is the Voronoi region of A.. To enforce Py to model the true distribution of y 9a (xx), we maximize its log likelihood Ex [oolllggpp Py (x))]. Thus, our complete loss given by
min (1 - P1) Eg d(x a)) E0 Pe E.g (d)=,,))))) + Ege log (gu (m)))
In our experiments, we found that detaching the gradient from the Egg (d( 2 term helped to significantly improve training stability. For the lattices used, used self-similar nested lattices: Ge I'- G,, where Ge and G are the coarse and fine lattices' generator matrices respectively, and is the integer-valued nesting ratio, used to vary the rate of the compressor.
4.3 FROM ONE-SHOT TO BLOCK CODING
LTC can be directly applied to any vector source æ E Rd. In this case, LTC operates as one-shot code for compress- ing æ. In Sec 5, we discuss the improvements that LTC provides compared to NTC in the one-shot sense. Rate- distortion theory informs us that to go beyond one-shot performance and approach asymptotic optimality (given by R,(D)), need to encode i.i.d. copies #1,..., Fn P= simultaneously. This is highly relevant in settings such as dataset compression, when we wish not just to compress single sample, but an entire dataset where samples can be assumed i.i.d. Note that n remains the block-length, but d' now the dimension of
1/1
8/1
Figure 4: Block-LTC (BLTC) applies LTC in the latent space for vector i.i.d. sequences.
To realize such a compressor, we propose Block-LTC (BLTC), which applies LTC across the i.i.d. latent variables of æ, shown in Fig. 4. Specifically, the analysis transform Ga gd dRd maps each vector of the block to its latent variable, y: = 9a (æ;), which has dy dimensions. This produces
a latent i.idd. vector sequence y1,.... ......y. Yn- Then, d, separate LTC models {c(() (() (() dy
J are
applied to across the block, where (6) (6) are the analysis/synthesis transforms of LTC. The i-th LTC model is applied the latent slice fy1,is...,,34. Un... which has i.i.d. scalar entries. The resulting reconstructed latents ÿ,
passed to the synthesis transform 9. Rd + Rd such that o; 9a (..). We use STE to train the BLTC models. The BLTC architecture first transforms the i.i.d. vectors into i...d. latent vectors ye the space- space-packing packing ability of LTC compresses the i.i.d. latent vectors.
7

---

Published as a conference paper at ICLR 2025
S0PP (48)
SPW (40)
12
2. LFC
ECVO
LTC
0.75
(éia dimassio)
(3is per smmp))
Figure 5: LTC (Ay lattice) quantization regions. Figure LTC on synthetic sources. Left: Gaus- sians. Right: Banana.
Left: 2-D Gaussians. Right: Banana.
RESULTS
We first discuss LTC on synthetic sources, before moving to real-world sources. Then, we demonstrate how LTC performs with nested lattices, and finally present results on block coding using BLTC. In addition, we provide an ablation study demonstrating the effects of various components in LTC.
Experimental setup. For LTC's rate term, we found varying results for dithering (6) and STE (5) depending on the source; these are specified for each experiment individually. Additional implementation details can be found in Appendix A. We choose lattices best in their dimension (Agrell & Allen, 2023): A2 (hexagonal), D Fig and A24 (Leech) lattices. These lattices all have efficient algorithms to solve the closest vector problem (CVP), which computes QA(y). For the Euclidean vector sources, we choose the transforms 9a and ga to be MLPs with two hidden layers, hidden dimension of 100, and softplus nonlinearities after each layer (except the last). For tensor-valued sources, we use standard source-specific transforms (e.g., CNNs for images). Unless mentioned otherwise, model Py (used for variable-rate and fixed-rate LTC), with 5 coupling layers and isotropic Gaussian prior. The rates plotted (unless mentioned otherwise) are given by the cross-entropy of the density model. We use 4096 samples for the MonteeCCarl estimate of the PMF in (5) and (8) for all lattices. This was sufficient for most sources and lattices; see Appendix. A.2. In most experiments, we compare with ECVQ as well as the rate-distortion function R RDD) of the source. When not known in closed-form form, R(D) is estimated using Blahut-Arimoto (BA) (Blahut, 1972; Arimoto, 1972) for 1-D and 2-D sources, and NERD (Lei et al., 2022) for higher-dimensional sources.
the RealNVP (Dinh et al., 2017) normalizing flow (NF) for the density
5.1 LTC AS ONE-SHOT CODE
We first consider 2-D isotropic Gaussians and the 2-D Banana source, where we can easily visualize the quantization regions. Then, we consider n-length i.i.d. sequences of several sources, including Gaussian with Le distortion, Laplacian with L1 distortion, and 1-dimensional marginal of the Banana source (Ballé et al., 2021) with L2 distortion. The 1-d Banana marginal is used to test a skewed distribution, since both Gaussian and Laplacian sources are symmetric. In this setting, is the dimension of both æ and the latent space y. We found that using STE for the rate term worked better for the i.i.d. sources, whereas dithering was more stable for the Banana source.
2-D Gaussian and Banana. In 2-dimensions, we choose QA to be the hexagonal lattice, which is known to be optimal (Conway & Sloane, 1999), and use the NF density model. As shown in Figs. 5, LTC produces quantization regions in the original source space that closely match that of optimal ECVQ in Fig. 3. The latent space (Fig. 17b) effectively uses the hexagonal lattice compared to NTC's use of the integer lattice (Fig. 1a)). The analysis transform Ia determines which source realizations get quantized to a specific lattice point in the latent space (colored regions); the synthesis transform g. maps the lattice points to their corresponding reconstructions in the original space (black dots). Shown in Fig. 6, the LTC rate-distortion performance matches that of ECVQ for the Gaussian. On the Banana, NTC and LTC achieve ECVQ at low rates; at larger rates, LTC improves on NTC but does not fully match ECVQ; see Fig. 6. Inspecting the quantization regions (Fig. 19, 20, 21) reveals why. At low rates, all methods quantize along 1-D nonlinear component of the source, resulting in the same quantization regions no matter the tessellating structure. At higher rates, 2 dimensions are used for quantization, where the integer lattice of NTC becomes sub-optimal compared to the hexagonal regions of ECVQ/LTC. We see that the quantizer used solely determines LIC's quantization regions; whereas the transforms' purpose is to warp the latent quantizer onto the source statistics.
8

---

Published as a conference paper at ICLR 2025
SNN (40)
PSWR
10:07
NTC
LTC 12.4
MTC
LTC
64
8
R
bop
Figure 7: LTC on real-world sources. Left: Speech. Middle two: Physics. Right: Kodak.
nllength i..d. Sequences. Here, choose Q^ to be the best-known lattices in n-dimensions, including A2 (hexagonal), Dj, Eg, and A24- As n grows, ECVQ performance will approach the rate-distortion function of the source, which for the Gaussian (squared error distortion) is given by R(D) 1/2 log 5 ) < D S o² and the Laplacian (absolute error distortion) is given by R(D) - ()D, D...ODDS/// 1/A (Cover & Thomas, 2006; Equitz & Cover, 1991). On Gaussians, we see that Fig. LTC performs very close optimal ECVQ in dimensions n 2, 4, 8 using the hexagonal, Dr and Es lattice respectively. For larger n. ECVQ is too costly to compute, but we see that LTT with the Leech lattice for n = continues the trend, further improving the rate-distortion performance and approaching R(D). We observe similar performance gains for the Laplacian and Banana- ID in Fig. 15. Since each of the lattice quantizers used have efficient CVP algorithms, LTC succeeds in larger dimensions and rates, whereas ECVQ could not even be run due to its exponential complexity. This result provides evidence that companding (in the form of LTC) is optiroltimal in terms of VQ performance, even if it may not exactly implement VQ (Gersho, 1979).
Real-World Sources. We now consider sources a drawn from real-world data. We consider the Physics and Speech datasets (Yang & Mandt, 2022), of dimension d = 16 and 33 respectively, and the Kodak image dataset (Franzen), which has variable dimension depending on image size.
For Physics and Speech, we map x to latent space with dimension dy using MLP-based transforms. For these sources, we found training instabilities with STE, so dithering is used instead. At higher rates, we also observed stabler training with the factorized density model compared to the normalizing flow. For Speech, we observe coding gain for rates above R bits with the Dj and Es lattices, shown in Fig. Coding gain is observed on Physics for LTC different rate regimes. LTC with the hexagonal lattice performs best at lower rates, but the performance drops off as rate increases. At higher rates, LTC with latent dimensions performs the best with Dÿ lattice. These rates are measured in bits-per- pes-spee and not normalized by dimension. Generally speaking, different rate regimes may require a different number of latent dimensions to be used for NTC and LTC. the number of latent dimensions used by the transforms does not match the lattice quantizer dimension, this can result in suboptimal performance as the tessellation in source space may not be optimal. This is likely why LTC performance "peaks" for different latent dimensions.
For large-scale images, we focus on the NTC architecture in Cheng et al. (2020), and use product Es and Leech A21 lattices along the channel dimension of the latent. We use dither during training and STE at test time, and train on the Vimeo-90k (Xue et al., 2019) dataset. We use the identical hyperparameter settings for transforms and entropy models, corresponding to the first 4 quality levels for the "cheng2020-attn" model in CompressAl (Bégaint et al., 2020). Shown in Fig. 7, LTC with product Es and product Leech achieves -5.274% and -16. 708% BD-rate gain respectively over NTC when evaluated on Kodak. In Sec. B, we compare all methods with Cheng2020-NTC and VTM as anchor in the BD-rate table (Tab. 1). While NVTC achieves a -2.919 BD-rate gain over VTM. Cheng2020 with Leech is significantly better, with -11 1.895% BD-rate gain. Note that NVTC and Cheng2020 models differ significantly in terms of the transforms and entropy models, so multiple factors (e.g., transforms and quantization) may contribute to the achieved performance. Additionally, NVTC applies vector quantization to vectors of size 16, meaning NVTC is able to leverage the packing efficiency of 16-dimensional space compared to the Es and Leech with 8 and 24 respectively. To compare with LVQAC, we additionally plot the performance using the D lattice, which achieves a -2.337% BD-rate gain over Cheng2020-NTC. This compares solely the effect of quantization scheme, and not other components of LVQAC (Zhang & Wu, 2023). This result, along with how the Eg and A24 perform relative to each other, further supports evidence that the rate-distortion performance is

---

Published as a conference paper at ICLR 2025
17-
SAR (4R)
NiAs LITEE BLTC
10.51
BLTE.
Au, LTCARD
(iita
per sample)
(bita sample]
(Mi sample]
Figure 8: Block coding. Left: Sawbridge. Center: Banana. Right: Physics.
Figure 9: LTC-NLQ.
significantly determined by the quantization choice the latent space, and that the NTC model able to leverage the enhanced packing efficiency of higher-dimensional lattices. We see that simply adding lattice quantization to the latent space of Cheng2020 has brought its performance to approach the ResNet- rate-distortion upper bound Ry (D) from Yang & Mandt (2022) on Kodak. To compare just the speed of SQ vs LQ. we run a forward pass through the transforms, quantization, and entropy model. When on single-core CPU, Cheng2020-NTC takes 3.241 seconds per image averaged over Kodak, whereas Cheng2000-LTC-E8 takes 4.002 seconds per image. Additional comments on complexity can be found in Sec. A, B, with full runtime comparison in Tab 2.
As NTC and LTC operate here as one-shot compressors of æ; these results demonstrate that the gap to R(D) is partially due NTC being sub-optimal one-shot code. Fully approaching R(D), however, requires encoding across multiple source realizations, which we discuss in Sec. 5.2.
Nested Lattice LTC. Here, demonstrate the ability of fixed-rate LTC with nested lattices to provide coding gain with lattice dimension, with much lower complexity when coding to bits. On i..dd Gaussians (Fig. 9), demonstrate LTC with NLQ in the latent space. We use self-similar nested lattices for the integer, Es and A21 lattices, and set the nesting ratio I = 5, 7, to sweep the rate. The performance improves the lattice dimension grows, and approaches R(D) before, but does not perform as well as variable-rate LTC for particular dimension, as expected. We note that LTC with NL.Q has encoding complexity linear in n and able to outperform NTC.
5.2 BLOCK CODING
We evaluate BLTC in the block coding setup (i.e., compressing multiple samples from dataset simultaneously) on the Sawbridge, Banana, and Physies sources. For the Sawbridge, the optimal VQ (i.e., one-shot coding) performance is known in closed form as (D) (Wagner & Ballé, 2021, Thm. 2). In this case, BLTC uses latent dimension dy
and the nominal dimension is d 1024. In Fig. 8, we verify that BLTC bridges the gap between H (D) and R(D). Note that going beyond H(D) impossible without compressing multiple realizations time; this verifies the ability of BLTC to leverage space-packing of the Sawbridge's latent source. For the Banana and Physics sources, the optimal one-shot performance unknown, but we see that BLTC improves upon NTC and LTC, which are both one-shot codes. Thus, we show BLTC's ability approach R(D).
5.3 ABLATION STUDY
We first show how it is necessary for good lattice be used to achieve optimal performance. Only using the best-known lattices in each dimension yielded near-ECVQ performance. If sub-optimal lattices are used, this was not the case. Fig. 16a shows the A, and D; lattices used for 8; they perform about the same as ECVQ for n 4, but still yield coding gain over NTC. This implies that IVQAC, which uses D sub-optimal compared to other lattices. In the appendix (Sec. B. B.1), we provide several additional ablations further revealing the interplay of LTC components.
DISCUSSION AND LIMITATIONS
In this work, we use the best known lattices up to dimension 24. Going to even larger dimensions will improve performance further; designing efficient high dimensional lattices with low complexity CVP solvers is an active area of research building on advances in modern coding theory (c.g. low density parity codes and polar codes).
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
This work was supported by The Institute for Learning-cnabler Optimization at Scale (TILOS), under award number USF-CCF-2112665 The work of Hamed Hassani was further supported by NSF CAREER award CIF- 194309,,,4, and the work of Eric Lei was further supported by a NSF Graduate Research Fellowship.
REFERENCES
Erik Agrell and Bruce Allen. On the best lattice quantizers. IEEE Transactions on Information Theory, 69(12):7650-7658, 2023. doi: 10.1109/TT.222333291133
Erik Agrell, Daniel Pook-Kolb, and Bruce Allen. Gradient- based optimization of lattice quantizers. arXiv preprint arXiv:2401.01799 2024.
S. Arimoto. An algorithm for computing the capacity of arbitrary discrete memoryless channels. IEEE Transactions on Information Theory, 18(1):14-20, 1972. doi: 10.1109/TTT.992.1000 1054753. 10554454.
Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. Variational im- age compression with scale hyperprior. In International Conference on Learning Representations, 2018. URL https: wwwp//eeeeeeee.ee/e/ee/e. net/ CorummiG=rrcQFMZRR
Johannes Ballé, Philip A. Chou, David Minnen, Saurabh Singh, Nick Johnston, Eirikur Agustsson. Sung Jin Hwang, and George Toderici. Nonlinear transform coding. IEEE Journal of Selected Topics in Signal Processing, 15(2):339-33,, 2021. doi: 10.1109//STSP22200003444000
Johannes Ballé, Sung Jin Hwang, and Eirikur Agustsson. TensorFlow Compression: Learned data compression, 2024. URL http:///// thtbb
thub. www/tensorrll.eeccoocoo ow compress com//tenssorrlloo/corrrresssss on.
Jean Bégaint, Fabien Racapé, Simon Feltman, and Akshay Pushparaja. Compressai: a pytorch library and evaluation platform for end-to-end compression research. arXiv preprint arXiv:2011.03029, 2020.
W. R. Bennett. Spectra of quantized signals. The Bell System Technical Journal, 27(3):446472, 1948. doi: 10. 10.1002/j. 1538-7305.
Sourbh Bhadane, Aaron B Wagner, and Jayadev Acharya. Principal bit analysis: Autoencoding with schur-concave loss. International Conference on Machine Learning, pp. 852-862. PMLR, 2021.
Sourbh Bhadane, Aaron B. Wagner, and Johannes Ballé. Do neural networks compress manifolds optimally? In 2022 IEEE Information Theory Workshop (ITW), pp. 582-587, 2022. doi: 10.1 10.1/ ITW54588.2022 010-4558.2022.9966 9995999.
R. Blahut. Computation of channel capacity and rate-distortion functions. IEEE Transactions on Information Theory, 18(4):460-473 1972. doi: 19110999TT11992.155555
Benjamin Bross, Ye-Kui Wang, Yan Ye. Shan Liu, Jianle Chen, Gary J. Sullivan, and Jens-Rainer Ohm. Overview of the versatile video coding (vvc) standard and its applications. IEEE Transactions on Circuits and Systems for Video Technology, 31110)3733-3377 3764, 2021. doi: 10. 10.1109/T. 2021.3101953.
Bucklew. Companding and random quantization in several dimensions. IEEE Transactions on Information Theory, 27(2):0072211, 1981. doi: 10.1109/TTT..1.
J. Bucklew. A note on optimal multidimensional companders (corresp.). IEEE Transactions on Information Theory, 29(2):79-2779 1983. doi: 10. 1983.
Maida Cao, Wenrui Dai, Shaohui Li, Han Li, Chenglin Li, Junni Zou, and Hongkai Xiong. Entropy relaxed lattice vector quantization for learned image compression. In 2024 Data Compression Conference (DCC), pp. 548-548, 2024. doi: 10. 00.1009/DCC58796222240000
Zhengxue Cheng. Heming Sun. Masaru Takeuchi, and Jiro Katto. Learned image compression with discretized gaussian mixture likelihoods and attention modules. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 7939-7948, 2020.

---

Published as a conference paper at ICLR 2025
P.A. Chou, T. Lookabaugh, and R.M. Gray, Entropy-constrainet vector quantization. IEEE Transac- tions on Acoustics, Speech, and Signal Processing, 37(1):31-42 1989. doi: 10.1109/29.17499
J. Conway and N. Sloane. Soft decoding techniques for codes and lattices, including the golay code and the leech lattice. JEEE Transactions on Information Theory, 32(1):41-50, 1986. doi: 10.1 10.110/TTITIOT 1986. 1057135.
H. Conway and N. A. Sloane. On the voronoi regions of certain lattices. SIAM Journal Algebraic Discrete Methods, 5(3):294-305, 1984. doi: 10.1137/0605031. URL https: //doi. org/ 0 11..11377 111111177/00 37/000033.
J. H. Conway and N. A. Sloane. Sphere Packings, Lattices, and Groups. Grundlehren der mathematischen Wissenschaften. Springer, New York, NY, 1999. ISBN 978-0-387-98585-5.
Thomas M. Cover and Joy A. Thomas. Elements of Information Theory (Wiley Series Telecommu- nications and Signal Processing). Wiley- Filey-Interscience, USA, 2006. ISBN 0471241954.
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real NVP. In International Conference on Learning Representations, 2017. URL httpp:://ppeneeeee
net/foruu i HkpbbH911
Alaaeldin El-Nouby, Matthew J. Muckley, Karen Ullrich, Ivan Laptev, Jakob Verbeek, and Herve Jegou. Image compression with product quantized masked image modeling. Transactions on Machine Learming Research, 2023. ISSN 2835-8856. URL https: /wwp/eeveew..ecc/////// net/
Forum?id-2215dBe
W.H.R. Equitz and T.M. Cover. Successive refinement of information. Information Theory, 37(2):269-275, 1991. doi: 10.1109/18.7524.
IEEE Transactions on
Runsen Feng, Zongyu Guo, Weiping Li, and Zhibo Chen. Nvte: Nonlinear vector transform coding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6101-6110 10, 2023.
Richard W. Franzen. Kodak lossless true color image suite. URLnttpss//=OO.uuggaaapa kodak/.
A. Gersho. Asymptotically optimal block quantization. IEEE Transactions on Information Theory, 25(4):373-388 1979. doi: 10. 1999 10560677.
Allen Gersho and Robert M Gray. Vector quantization and signal compression, volume 159, Springer Science & Business Media, 2012.
Dailan He, Ziming Yang, Weikun Peng. Rui Ma, Hongwei Qin, and Yan Wang. Elic: Efficient learned image compression with unevenly grouped space-channel contextual adaptive coding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
5718-5727, 2022.
Ivan Kobyzev, Simon JD Prince, and Marcus Brubaker. Normalizing flows: An introduction and review of current methods. IEEE transactions pattern analysis and machine intelligence, 43 (11):3964-9979 2020.
Shinobu Kudo, Yukihiro Bandoh, Seishi Takamura, and Masaki Kitahara. LVQ-VAE:end-to-end hyperprior-based variational image compression with lattice vector quantization, 2023. URL https: http:://pppeeeeee http///ppeeeeeeeccce/e/e. .nt wooumoiiddiaaaao
Eric Lei, Hamed Hassani, and Shirin Saeedi Bidokhti. Neural estimation of the ritt-distottion function with applications to operational souree coding. JEEE Journal Selected Areas in Information Theory, 3(4):674-6) 686, 2022. doi: 10.1 10.1009//AAT..222333377777
T. Linder, R. Zamir, and K. Zeger. High-resolution source coding for non-difference distortion measures: multidimensional companding. IEEE Transactions on Information Theory, 45(2): 548-561, 1999. doi: 10.1
12

---

Published as a conference paper at ICLR 2025
Shanxiang Lyu, Zheng Wang, Cong Ling, and Hao Chen. Better lattice quantizers constructed from complex integers. JEEE Transactions on Communications, 70(12):7932-7944 7940, 2022. doi: 10.1009TCOOM..202.2115555
David McAllester and Karl Stratos. Formal limitations on the measurement of mutual information. In Silvia Chiappa and Roberto Calandra (eds.), Proceedings of the Twenty Third International Conference on Artificial Intelligence and Staristics, volume 108 of Proceedings of Machine Learning Research, pp. 775. PMLR. 26-28 Aug 2020. URL https://ppoceedding m1 r.l. mlr..pess// 0 /mcaal lester20a .ttll.
Aleksandr Shevchenko, Kevin Kögler, Hamed Hassani, and Marco Mondelli. Fundamental limits of two-layer autoencoders, and achieving them with gradient methods. In International Conference on Machine Learning, pp. 31151-31209 PMLR, 2023.
Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learn- ing. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS' 17, pp. 1309-6318, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964.
A. Vardy and Y. Be'ery. Maximum likelihood decoding of the leech lattice. IEEE Transactions on Information Theory, 39(4)::435-144 1993. doi:
Aaron B. Wagner and Johannes Ballé. Neural networks optimally compress the sawbridge. In 2021 Data Compression Conference (DCC), pp. 142-152. IEEE, 2021.
Tianfan Xue, Baian Chen, Jiajun Wu, Donglai Wei, and William T Freeman. Video enhancement with task-oriented flow. International Journal of Computer Vision, 127:1106-1125 2019.
Yibo Yang and Stephan Mandt. Towards empirical sandwich bounds on the rate-distortion function. In International Conference on Learning Representations, 2022. URL http:::tt///peeeeee net / Forum? i d #4PmOQSZDY.
Yibo Yang, Stephan Mandt, and Lucas Theis. An introduction to neural data compression. arXiv preprint arXiv:2202.06533, 2022.
Ram Zamir, Bobak Nazer, Yuval Kochman, and Ilai Bistritz. Lattice Coding for Signals and Networks: A Structured Coding Approach to Quantization, Modulation and Multiuser Information Theory. Cambridge University Press, 2014.
Xi Zhang and Xiaolin Wu. Lvqac: Lattice vector quantization coupled with spatially adaptive companding for efficient learned image compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10239-1020 10248, 2023.
13

---

Published as a conference paper at ICLR 2025
A EXPERIMENTAL DETAILS
The code for these experiments will be released upon publication. all experiments, we use a batch size of 64 and train until convergence using Adam. For the i.i.d. scalar sequences, found that using CELU nonlincarities with no biases in the transforms sometimes helped improve training stability. For synthetic sources, Speech and Physics, the rate-distortion Lagrange multiplier is swept over 0.5, 1, 1.5, 2, 4, 8. For images, we use the default À values in Bégaint et al. (2020). In addition, for images, a product lattice is applied along the channel dimension of the latent tensor, which has shape C' x H x W. There are C/n lattices applied H W times, where n is the lattice dimension.
A.1 CVP ALGORIHTM
In this paper, we use the CVP algorithms outlined in Conway & Sloane (1999, Ch. 20). For the Barnes- Wall lattice Ae dimension 16, we use the algorithm outlined Conway & Sloane (1984). Finally, the Leech lattice A21 in dimension 24 has the slower method described in Conway & Sloane (1984) and the improved version in Conway Sloane (1986). In many cases, the lattice A = U 1 (r: - Ao) is a union of T cosets of a sublattice Ao. and one can solve CVP for A by computing Q (x(x) (x) arg mill QAo (2c r:) ) rs. For example, the Eg lattice is equivalent to two cosets of Dg.
A.2 MONTE-CARLO SAMPLES
Regarding variable-rate LTC, for lattices of lower dimensions, we found that setting the number of Monte-Carlo samples Nint for approximation of (5) to be 4096 was sufficient to recover ECV performance. For the Leech lattice of dimension 24, however, better performance could be recovered by increasing Nin the cost of increased GPU memory requirements; see Fig. 10.
0.40
0.40
0.35
0.35
0.30
0.30
ECSQ
0.20
0.20
24 Aua 1684
-... n- 75 Aug ITC Now 1966
0.15
0.15
24 LTC 33000
0.10
6.10
0.05
0.05
0.2
0.4
11.5
11.8
0.5
1.0
1.5
2.0
Figure 10: Effect of Nm on the Leech lattice, for i.i.d. Gaussian sequences.
We see that setting Ni 1024 results in worse performance than ECVQ for 8 at low rates. Setting Nim = 4096 improves the performance, and nearly approaches that of N = 15000.
A.3 SOURCE GENERATION
The Gaussian and Laplacian sources are straightforward to simulate. The Banana source is taken from the Tensorflow Compression library (Ballé et al., 2024). To generate the Banana-ld marginal, we simply take the first dimension of the 2-d Banana realizations. The Sawbridge source (Wagner Ballé, 2021) is continuous random process
x(t) = - 1t2U, 0 0st5l,
(10)
where U ~ (n) 1]). Following Wagner & Ballé (2021), we uniformly sample 1024 time steps between 0 and 1, which creates d024-dime vectors. For all synthetic sources. sample 10" samples as our training dataset. Finally, the Physics and Speech sources are taken from https: / / thub. com/mandt. 1ab/ RD//RD-SS sandwich
14

---

Published as a conference paper at ICLR 2025
A.4 EFFICIENCY
Compared to NTC, LTC requires more memory/training time, roughly 50% more for the low/moderate dimension sources and 10% more for images. The main computational bottleneck that LTC has is the use of Monte-Carlo integration for (5). Sampling uniformly random vectors from the Voronoi region of a lattice requires a quantization step. and hence solving the CVP. Since we use 4096 sampled vectors per batch, this means CVP needs to be solved on all 4096 vectors. The lattices chosen above all have GGU-ffrienlly implementations that can parallelize across the input vectors. On NVIDIA RTX 5000 GPUs with 16 GB memory, LTC training until convergence took at most few hours for the Speech and Physics sources, and minutes for the i.i.d. scalar sequences. This was roughly twice the amount of time train NTC. However, large-scale images, the training speed is dominated by the backwards pass through the convolutional neural networks, and so LTC takes around the same time as NTC (around 10 days per rate point).
B ADDITIONAL RESULTS
BD-rate and runtime tables for Kodak. Here, we show the BD-rate tables (Tab. 1) and full runtime comparison (Tab. 2) on the Kodak benchmark. For the runtime comparison, it is time taken per-image on single-core CPU, averaged over the rate-distortion tradeoff. We see that Cheng2020-LLTC-Nzz
outperforms all other methods in rate-distortion performance, including ELIC (He et al., 2022). We note that the ELIC model is NTC-based, and thus even further gains can be expected by integrating LTC with ELIC transforms and entropy model. This would also inherit the efficient nature of the ELIC layers, as the quantization and entropy modelling are independent components of the neural compressor. We leave this for future work as it is out of the scope of the current study. Regarding runtime on the Cheng2020 architecture, using the Egg lattice incurs a 1.23x increase in runtime over NTC, whereas the Leech lattice incurs a 2.68x increase. We note that the Leech lattice CVP solver we used (Conway & Sloane, 1986) is not the fastest; one can expect faster runtimes with the Leech decoder of Vardy & Be'ery (1993).
Table 1: BD-rate comparison on Kodak.
Model
BD-rate (NTC anchor)
BD-rate (VTM anchor)
NhTng2020-NTCC
0.000
5.210
Cheng2020-LTC.
-2.337
2.753
Cheng2020-LTC-E,
-5.274
-0.335
200-L-0200
-16.708
-11.895
NVTC (Feng et al., 2023)
-4.953
-2.919
VTM (Bross et al., 2021)
-7.879
0.000
ELIC (He et al., 2022)
10.607
-5.656
Table 2: Inference encoding runtime on Kodak, single CPU. LTC models use 1024 samples for Monte- Canto estimation.
Model
Runtime (sec.)
Cheng2020- -TC
3.241
Cheng2020- LT0200-LT-D-;-
4.016
Cheng2020-1LTC-FF,
4.002
Cheng2020-1LTC-Aaa
8.634
ELIIC (He et al., 2022)
3.132
NTC with lattice quantization during inference. Here, we show the performance of LTC when the transforms and entropy model are loaded from a pre-trained NTC model. In other words, a trained NTC model's ss scalar quantizer is replaced by a lattice quantizer at inference time. The performance of this scheme is largely dependent on the source at hand. Fig. 11 shows that on the Gaussian source, this scheme performs the same training LTC from scratch; it may shift the point on the R-D tradeoff but overall lies on the same R-D curve. This is perhaps not unexpected, as the transforms learned by NTC and LTC for the i.i.d. Gaussian source effectively perform scaling of the source, and so interchanging them has no effect. We use the same Flow entropy model for fair comparison. In general, however, this is not expected be the case, as the analysis transform determines which
15

---

Published as a conference paper at ICLR 2025
region of the lattice gets used; for complex sources, there may be more sophisticated ways the transform determines how the lattice is used. For example, in Fig. 12, we demonstrate the same scheme Kodak images with the Cheng2020-LTC-A22 model. Here, Leech lattice quantizer with the trained NTC model improves the performance over NTC by a BD-rate of -f,99, training the model with the lattice quantizer helps boost the performance further to BD-rate of 116. 10.70... This result implies that one can avoid training from scratch when good set of transforms and entropy model already trained from NTC architecture, and instead potentially fine-tune the NTC transforms and entropy model with the chosen lattice for best performance. In the future, when better transforms and entropy models are developed, one can also train with the lattice from scratch to get the best performance, as the training times between NTC and LTC (from scratch) are on the same scale (weeks).
that while using
10
34
33
SNR (dB)
PSNR
-32
NTC
000
LTC
Chegg200 KTC
Aa LTC w/ NTC weghts
05
Dhepp000
1.0
155
2.0
25
0.1
05
(bits per sample)
bpp
Figure 11: Gaussian source, comparing LTC (trained) versus LTC loaded with pretrained NTC weights.
Figure 12: Kodak images, comparing LTC (trained) versus LTC loaded with pretrained NTC weights.
Learned quantizer cells regions of low probability. Figs. 13, 14, we visualize the quantizer cells of LTC on regions of low probability density, for the 2D Gaussian source and Banana source, respectively. That is, we train LTC on a source Pæ. and evaluate on the support of pa as well as regions outside the support of Pæ- Overall, the quantization cells successfully generalize outside the support of Pæ. meaning that if a sample to be compressed is perturbed slightly outside the support of Pa, its reconstruction will still be close to æ.
0-
-2-
(a) In-distribution samples.
(b) Out-of-distribation samples.
Figure 13: Quantizer cells of LTC trained 2-d i.i.d. Gaussians.
(a) In-distribution samples.
(b) Out-of distributii samples.
Figure 14: Quantizer cells of LTC trained on Banana source.
16

---

Published as a conference paper at ICLR 2025
Rate-redundancy. For certain figures in the ablation study, we define the normalized rate- redundancy of compressor c,
E(( = (D) R(D))))))))()
(11)
where (D) is the rate achieved by c at distortion D, and R(D) is the rate-distortion function of Intuitively, (D) measures the fraction of additional bits per sample required to code sequence relative to R(D), and makes some of the performance gains easier to see in the figures.
MTC
A LTC
Dile
S. REITE
033
(88)
0.5
SHR (B)
0.30
203
NTC
MTC
0.01
000
ITC
01
L.0
1.5
1.30
200
0,0
(bie
sample
(bix per sample]
(a) Laplace source (L1).
(b) Banana- ID (L2).
(a) LTC, sub-opt lattices.
(b) Dither vs. STE.
Figure 15: Rate-distortion performance scalar i..d. sequences.
Figure 16: Ablation study, i.dd Gaussians.
n- LTC CE (Flm)
0.4
& LTC H(s)
0.8
0.1
(a) NTC (integer lattice).
(b) LTC (hexagonal lattice).
1.0
2.0
Figure 17: Latent space quantization regions.
Figure 18: Flow provides tighter entropy estimate.
(a) R ~ 3.
(a) R 3.
(a) R R 3.
(b) R R 5.
(b) R3
(b) R R3 5.
Figure 19: ECVQ quantization regions, banana source.
Figure 20: NTC quantization re- gions, Banana source.
Figure 21: LTC quantization re- gions, Banana source.
17

---

Published as a conference paper at ICLR 2025
(a) n
(b) R R3 5.
Figure 22: Quantization regions of Banana source using ECLQ. Without the transforms, quantization regions of ECLQ fail to conform to the source manifold.
B.1 ABLATION STUDY
We first provide comparison with intropy-constrainee lattice quantization (ECLQ). which applies lattice quantization directly to the source without transforms. While ECLQ approaches VQ perfor- mance in the large rate regime, is highly sub-optimal in the low/moderate rate regime. LTC, on the other hand, performs much better at all rates. This is shown in Fig. 23 for Gaussians and Banana. While the quantization regions look similar, the primary benefit of LTC that the Is transform moves the reconstructions è according to the source statistics. Standard ECLQ keeps the reconstructions the center of the lattice cells, which is sub-optimal for non-unine con-uniform sources. Moreover, on the Banana source, without the transforms, ECLQ fails to match the quantizer cells to the source statistics (Fig. 22), unlike LTT (Fig. 21). We further analyze (i) the training objective used, and (ii) the effect of density models on the entropy estimate of ý. For the training objective used, compare using dithered quantization with STE for the rate term (i.e., using likelihoods from (6) and (5) respectively). Shown in Fig. 16b, for low rates, the reported cross-entropies from dithering much larger than that of STE, but converge to the STE performance as rate increases. One possible reason is that the convolved density in (6) over-smooths the true density of y in the low-rate regime, leading to sub-optimality.
For the entropy models, compare the factorized density with the normalizing flow density in Fig. 18. Both models use STE during training. Interestingly, when computing the true entropy of the quantized latent H (r). both models yielded near-optimal performance (curve labeled H (ÿ)). However, the cross-entropy upper bound of the factorized density was not as tight as that of the normalizing flow. The normalizing flow has near-zero gap between (i) and the cross entropy.
SAI (10)
Ru 6%
54t (10)
R.D)
NTC
DCVO
L7C
cimanioe) (a) i.i.d. Gaussian.
[ sumplk)
(b) Banana source.
Figure 23: LTC vs. ECLQ.
18


Reviews:
[{"review_id": "f0X6cOw2T2", "reviewer": "Reviewer_quJZ", "summary": "This paper proposes lattice transform coding (LTC), which uses lattice quantization beyond the conventional scalar quantization in neural lossy compression. The paper offers evidence that learned transforms in conventional nonlinear transform coding (NTC) are insufficient in overcoming sub-optimal quantization in the latent space, and shows that LTC with more efficient space-filling lattices indeed bring improvements in R-D performance. Besides new training and rate estimation techniques for LTC, the paper also proposes nested LTC to lower the practical computation cost of entropy coding, as well as block LTC to perform block coding in order to close the gap to the information-theoretic optimal performance (R(D)). \nExperiments on various synthetic and real-world data sources support the effectiveness of the proposed LTC approach and give insight to the suboptimality of current NTC approaches.", "questions": "1. I'm confused by how LQ is applied when the source dimension does not equal the lattice dimension. It seems the lattice dimension used in this paper only goes up to 24, but the source dimension can be 33 (Speech) or even higher (Kodak). Is it based on some kind of product lattice then?\n\n\n2. Related to above, in the block LTC scheme (Sec 4.3), what is the motivation for grouping columns of the transform coefficients matrix into length-n blocks? Given that the data $x$ and the resulting latent tensor $y$ is often high dimensional, can we do block coding on length-n blocks of $y$ in some other ways? I would expect more gains if the entries in each block are **not** i.i.d.\n\n3. Any idea why the R-D performance is sometimes worse when a bigger latent space ($d_y$) is used, all else being equal (e.g., the $d_y=4$ curve is worse than $d_y=2$ at low rates in the Physics Figure 7)? The usual observation in NTC is that the R-D performance can be improved by increasing the latent dimensionality when the latter is too small (esp. towards the low distortion regime), but further increasing the latent dimensionality beyond a sufficient capacity generally does not harm the R-D performance.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I consider the originality of the paper to be moderately high. Although the ideas of lattice coding, nested quantization, block coding, etc. have been known for a while, and lattice coding itself even explored in the contemporary work of Kudo et al. (2023), this paper brings these ideas together and gives novel insights for the performance gap between the current NTC methods and the theoretical optimum of R(D). \n\nThe paper is well-written and mostly easy to follow (although see issues below), with insightful explanations and experiments. \n\nI consider the contribution of this paper timely and significant. Unlike much of current research on NTC which is in architectural / entropy modeling improvements, this paper examines a fundamentally different component of NTC and makes a compelling case of the advantage of lattice quantization over scalar quantization.  Achieving the theoretical advantage of block coding / VQ has been a non-trivial goal of lossy compression, and this paper offers valuable perspectives and a potential path forward.", "weaknesses": "1. A significant difficulty of lattice coding seems to be the complexity of implementing entropy coding. Nested lattice quantization was proposed but appears to come with a significant penalty to R-D performance. Therefore most of the R-D curves for LTC in this paper are theoretical. I don't expect this problem to be solved in this paper, but I think it'll be useful to give insights for how to operationalize the reported R-D curves, in particular considerations for deriving the discrete entropy model from the normalizing flow prior in a way that is reproducible by the decoder. \n\n2. Some of the the explanations around nested LQ and block LQ were a bit hard to follow. For example, the first two sentences on lines 334 - 335 seemed to come out of nowhere ( \"The goal is for the majority of the latent mass to be placed in the Voronoi region of Λc. This happens when QΛc (yf ) = 0.\"). Similarly, line 375 introduces a new pair of \"analysis/synthesis transforms of LTC\", denoted $c_a^{(i)}, c_s^{(i)}$ --- why are they necessary, given that we have already operate in the transformed space using $g_a, g_s$? And how should we choose  $c_a^{(i)}, c_s^{(i)}$ if they are indeed necessary? \n\nAlso, a minor typographical error: there seems to be inconsistency between \"n\" and \"d\": line 368 explains \"Note that we we retain n as the block-length parameter, but use d as the dimension of x.\" So should line 458 write \"d = 16 and 32\"?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FfgZa678Jj", "reviewer": "Reviewer_YbH3", "summary": "This paper covers the application of lattice vector quantization to neural lossy compression.\n\nThe problem of how to address the suboptimality of scalar quantization for lossy compression has been studied for decades, and even today most practical methods use scalar quantization, and instead rely on other techniques to improve compression.\n\nOne important difference is that conventional methods normally rely on linear transform, nonadaptive methods, while this paper novelty is that it studies neural compression that uses nonlinear learned transformations. Numerical experiments demonstrate better compression results.", "questions": "It would be interesting to see more information about the practical aspects of using lattice quantizers during training and for actual coding.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The authors correctly provide good evidence that the suboptimality of scalar quantization remains even with learned neural transformations.\nShowing it on toy problems helps to make it clearer, but is not good at indicating how much worse on practical cases. \n\nThe paper presents examples of application to other data, and a series of experimental tests are used to answer those question on those data sources.", "weaknesses": "The experiments are well planned with good comparisons, but somewhat limited. Maybe too much space is used to cover the synthetic cases, and could be better used with real practical data.\n\nIt would also be better to have more detailed analysis of the results in those cases. Comments like “…with the Leech significantly outperforming VTM and NVTC,” are too vague and not accurate.\n\nIt looks like there are good gains with the modifications to the Cheng2020 method, which may be the most important conclusion of the paper, and that should be more carefully reported.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZcRjpNhOzZ", "reviewer": "Reviewer_mQL8", "summary": "1. This submission extends scalar-quantized non-linear transform coding of signals (Ballé et al., 2020) using lattice quantization and coding. A classical result from signal compression is that vector quantization can provide packing gains over scalar quantization especially when applied directly on correlated (i.e. non-decorrelated) signals, at the cost of higher encoding complexity. The authors propose to overcome the complexity of vector quantization using lattice quantization, which can be considered a constrained form of vector quantization. Experiments (on images and speech) were conducted to measure the gain from lattice quantization over the scalar quantization baseline.", "questions": "1. Clarification surrounding the choice of hyperparameter $\\lambda$ discussed in \"Weaknesses\" would be appreciated.\n2. Experimental results: LTC performs worse than NTC at extremely low bit rates. This is surprising since the benefits of vector/lattice quantization should be more pronounced at such rates.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The submission's attempt at marrying nonlinear transform coding (NTC) with lattice quantization (LQ) as a way of sidestepping computationally expensive vector quantization is highly relevant to the learned compression research community.", "weaknesses": "1. The fact that integer quantization of latent variables of NTC does not induce hexagonal quantization cells in the source domain—this is not a surprising or unexpected result, as non-linear transforms still define continuous / diffeomorphic mappings between the two domains. A transform that maps squares (scalar quantization cells in the latent domain) to hexagons (vector quantization cells in the source domain) would be highly discontinuous around cell boundaries, not a mapping that is readily learned by a neural transform in the first place.\n2. Eq. (1) is central to this work, yet, no description of the Lagrange multiplier / rate–distortion tradeoff $\\lambda$ appears in the manuscript. Wouldn't the choice of $\\lambda$ have an impact on the characteristics/performance of the learned non-linear transform? Was an effort made to reparameterize $\\lambda$ as in the work of Ballé et al. (2020)? Discussion around this is lacking in the current version of the manuscript.\n3. LTC (of latent variables) inducing near-hexagonal quantization cells in the source domain (in the 2D case) seems to be due more to the diffeomorphic / continuous nature of the learned non-linear transform itself. This would mean that one could take the non-linear transform of Ballé et al. (2020) and lattice-quantize the latent variables post-hoc, and achieve similar results to what this work is proposing. The benefits of incorporating the actual lattice structure in the rate term of the optimization should be discussed.\n4. Experimental results fail to clearly demonstrate the advantage of LQ over SQ. Could this be an optimization issue? For example, was $\\lambda$ was fixed to some value to hurt the performance of the non-linear transform itself?\n\n-------\n4. The current presentation seems somewhat unbalanced, with the authors' actual work only starting on page 6. While this is not inherently a weakness per se, this does mean that the authors were perhaps not able to fully explore / describe the actual contributions of their work in the remaining 4–5 pages.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NyZDr35iCt", "reviewer": "Reviewer_WNyg", "summary": "The authors show the suboptimality of existing neural compression models for simple multidimensional sources, such as 2-D Gaussians. They point out that the scalar quantization used in current neural compression models can lead to the suboptimality. They then propose a solution, *i.e.*, lattice transform coding, with lattice quantization in latent space. Extensive experiments verify the superiority of the proposed lattice transform coding.", "questions": "1. Is it possible to further reduce the computational complexity of LTC models for training and inference? Authors can try the following things to reduce concern.\n   - Provide more detailed computational complexity analysis, comparing LTC to state-of-the-art NTC methods beyond just Cheng2020-NTC.\n   - Discuss potential approaches to reduce training and inference time for LTC, such as whether fine-tuning techniques used for NTC could be adapted.\n   - Comment on how the complexity of LTC scales with different lattice dimensions and source dimensionality.\n   - Address whether there are any trade-offs between computational complexity and rate-distortion performance for LTC.\n2. Figure 5 does not show the learned lattice of LTC in the regions of lower probability density. How does LTC perform in these regions? Please provide visualizations or quantitative analysis of LTC's performance in lower probability density regions, and discuss any potential implications for overall compression performance.\n3. LTC provides a construction on the quantization that is quite different from previous work. Then, how do the nonlinear transforms affect the rate-distortion performance of the LTC models? Please provide an ablation study or analysis that isolates the effects of the nonlinear transforms on LTC's rate-distortion performance.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The idea of using lattice quantization in neural compression has potential whose benefits cannot be achieved by improving transforms and the entropy model.\n2. The theoretical analysis and experimental evaluation presented in this paper are convincing. The authors have carried out extensive experiments on both synthetic sources and on higher dimensional real sources.\n3. The paper is well written and easy to follow.", "weaknesses": "1. The real-world application of the proposed Lattice Transform Coding (LTC) may be limited by the computational complexity of training and inference. \n\n   - *On training.* The authors show that training an LTC model per rate point for image compression takes about 10 days. By comparison, training of nonlinear transform coding (NTC) models can be reduced by fine-tuning the lower rate model from a pre-trained higher rate model. I'm not sure if such tricks can also speed up the training process of LTC.\n   - *On inference.* The authors provide a comparison of the NTC method (Cheng2020-NTC) with the proposed LTC method (Cheng2020-LTC-E8), which shows that the computational complexity of LTC is comparable to NTC. However, Cheng2020-NTC is a nonlinear transform coding method using spatial auto-regression in entropy modeling, which is notoriously time consuming. The method is further refined in subsequent work, such as wavefront coding, checkerboard context, and channel-wise auto-regression. Notably, the proposed lattice transform coding can be even more computationally expensive than spatial auto-regression. This can be a fatal flaw in applications.\n\n2. Typos:\n\n   - Line 470. The citation of Vimeo-90k should be realized by a `\\citep` command instead of `\\citet`.\n   - Line 472. The citation of CompressAI is not included in the reference.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NUw5GUV0C9", "reviewer": "Reviewer_bzaC", "summary": "This paper highlights the limitations of scalar quantization in neural compression and introduces lattice transform coding as a solution. Given the challenges in rate estimation within this approach, the authors propose a Monte Carlo-based method to address it. To manage complexity, they further introduce a nested lattice structure. Experimental results are presented using both synthetic datasets and real-world datasets, including speech and the Kodak image dataset.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The observation that neural transforms alone cannot overcome scalar quantization limitations is a valuable insight.\nThe use of dithered noise and Monte Carlo estimation for rate estimation is also noteworthy.", "weaknesses": "- The presentation could be improved for clarity. All figures are too small, and Section 3 (on the suboptimality of scalar quantization) would benefit from additional detail beyond the currently limited Figure 2. \n- Additionally, the distinction in block LTC needs clarification.\n- Lattice coding encoding complexity remains an issue (except for $n=24$). More specifics on how this scheme was applied to the Kodak image dataset would be helpful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)