Causal Context Adjustment Loss for Learned Image Compression
Authors: Minghao Han
Shiyin Jiang
Shengxi Li
Xin Deng
Mai Xu
Ce Zhu
Shuhang Gu
Venue: NeurIPS 2024 poster (2024)
Keywords: image compression
entropy model
causal context
Primary area: machine_vision
TL;DR: A Causal Context Adjustment loss for better training the learned image compression network.

Abstract: In recent years, learned image compression (LIC) technologies have surpassed conventional methods notably in terms of rate-distortion (RD) performance. Most present learned techniques are VAE-based with an autoregressive entropy model, which obviously promotes the RD performance by utilizing the decoded causal context. However, extant methods are highly dependent on the fixed hand-crafted causal context. The question of how to guide the auto-encoder to generate a more effective causal context benefit for the autoregressive entropy models is worth exploring. In this paper, we make the first attempt in investigating the way to explicitly adjust the causal context with our proposed Causal Context Adjustment loss (CCA-loss). By imposing the CCA-loss, we enable the neural network to spontaneously adjust important information into the early stage of the autoregressive entropy model. Furthermore, as transformer technology develops remarkably, variants of which have been adopted by many state-of-the-art (SOTA) LIC techniques. The existing computing devices have not adapted the calculation of the attention mechanism well, which leads to a burden on computation quantity and inference latency. To overcome it, we establish a convolutional neural network (CNN) image compression model and adopt the unevenly channel-wise grouped strategy for high efficiency. Ultimately, the proposed CNN-based LIC network trained with our Causal Context Adjustment loss attains a great trade-off between inference latency and rate-distortion performance.

Causal Context Adjustment Loss for Learned Image Compression
Minghao Han', Shiyin Jiang', Shengxi Li?, Xin Deng", Mai Xu², Ce Zhu', Shuhang Gu University of Electronic Science and Technology of China "Beihang University {minghao hmh, Bauhanggu)Ogmail..c
Abstract
In recent years, learned image compression (LIC) technologies have surpassed conventional methods notably in terms of rate-distortion (RD) performance. Most present learned techniques are VAE-based with an autoregressive entropy model, which obviously promotes the RD performance by utilizing the decoded causal context. However, extant methods are highly dependent on the fixed hand-crafted causal context. The question of how to guide the auto-encoder to generate a more effective causal context benefit for the autoregressive entropy models is worth exploring. In this paper, we make the first attempt in investigating the way to explicitly adjust the causal context with our proposed Causal Context Adjustment loss (CCA-loss). By imposing the CCA-loss, we enable the neural network to spontaneously adjust important information into the early stage of the autoregressive entropy model. Furthermore, as transformer technology develops remarkably, variants of which have been adopted by many state-of-the-ar (SOTA) LIC techniques. The existing computing devices have not adapted the calculation the attention mechanism well, which leads to burden on computation quantity and inference latency. To overcome it, we establish convolutional neural network (CNN) image compression model and adopt the unevenly channel- wwie grouped strategy for high efficiency. Ultimately, the proposed CNN-based LIC network trained with our Causal Context Adjustment loss attains a great trade-off between inference latency and rate-distortion performance. The code is available here.
Introduction
The burgeoning quality of high-resolution photos is driving an increasing demand for advanced image storage and transmission technologies. Consequently, lossy image compression techniques have been growing extraordinarily fast in recent years. In parallel to conventional coding technologies such as JPEG [42], BPG [6], WebP [15], VVC [40], learned image compression (LICC methods [3, 4, 10, 11, 17, 18, 20, 30, 35, 36, 47] emerge, achieving high peak signal-to-noiss ratio (PSNR) and multiscale structural similarity (MS-SSIM) [44] while operating fairly fast. Their superior compression results over those of VVC demonstrate an enormous possibility that LIC technology would appear on par with the traditional ones in the near future.
Learned lossy image compression methods are built upon variational auto- (VAE) framework proposed by Ballé et al. [4]. The VAE based LIC framework mainly comprises an auto-encoder and an entropy model. The auto-encoder conducts nonlinear transforms between the image space and the latent representation space; while, the entropy model minimizes the code length by estimating the probability distribution of latent representations. In comparison to the auto-encoder, which could borrow ideas from recent advances in network architecture design, the entropy model is a unique important component to LIC and has vital influence on the final compression results.
Cooresponding Author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

In the literature on LIC. the entropy model generally refers to parameterized distribution model. In their seminal work [3]. Ballé et al. established the end-to-end rate-distortion minimization framework and showed that the smallest average code length of latent representation is given by the Shannon cross entropy between the actual marginal distribution and a learned entropy model. Since then, numerous entropy models have been investigated. One category of studies investigates advanced network architectures for accurately predicting the distribution of latent representations. Meanwhile, another line of research study more fundamental perspective of the entropy model, i.e. conditional distribution modeling, to pursue a better rate-distortion trade-off. Taking side information (also termed as hyperprior) and decoded latent (also termed causal context) as conditions has become prevailing strategy in state-of-the thr-rrrrttttttt LIC models.
In this paper, we advance conditional distribution modeling in the entropy model with our proposed causal context adjustment loss (CCA-loss). Existing works generally train LIC networks with a combination of the rate loss and the distortion loss. The conditional predictability of the representation indirectly optimized, and the performance of entropy model highly relies on the hand-crafted causal context model, e.g. channel-wise [36], checkerboard [18] and space-channel [17] context model, Our CCA-loss makes the first attempt on explicitly imposing loss to adjust the causal context, making the latter representation more accurately predicted by the previously decoded representations, To be more specific, considering a two stage autoregresive context model with hyperprior 2, denote the latent representation to be decoded in the first and second stage y/1 and y2i in addition to minimizing the cross entropy loss for reducing bitstream, we introduce an auxiliary entropy model and tailored context causal adjustment loss, which let y/2 be accurately estimated by y and z, while, at the same time, let y2 can not be accurately estimated by merely 2. In this vein, our CCA-loss explicitly guides the encoder to adjust important information into the early stage of the autoregressive entropy model, providing the LIC framework a more rational causal context sequence for entropy coding. As the codes in the early stages are enhanced with our CCA-loss, we further study the schedule of causal context transmission, and adopt an uneven channel dimension schedule for the pursuit of better rate-distortion trade-off. The uneven channel schedule is also beneficial for reducing computational burden in the coding and decoding process, enabling our model to achieve state-of-the th-the-art art compression performance with less running time. Our contributions are summarized as follows:
We introduce causal context adjustment loss to explicitly adjust the causal context informa- tion, forcing the network encode important information early and therefore improving the autoregressive prediction accuracy of the entropy model.
We adopt an uneven schedule of autoregressive causal context and a convolutional auto- encoder architecture, delivering an efficient compression network which is easy to be implemented on modern deep learning platforms.
We evaluate our proposed compression network on various benchmark datasets, in which our method achieves better dist-iitootion trade-offs towards the existing state-of sate-off-the-r methods, with more than 20% less compression latency.
2
Related Works
Recent LIC studies broadly follow the seminal work of Ballé et al. [3], which utilizes a VAE based framework for rate-distortion optimization. Generally, VAE-based LIC models comprise an auto-encoder and an entropy model. In this section, we review respective progresses in advanced auto-encoders and entropy models. Another considerable technique in LIC is the quantization method, however, as we did not dig into the details and simply followed the quantization method of [35], we omit the review of quantization methods in this section.
2.1 Auto-Encoder Architectures for Learned Image Compression
The auto-encoder plays the role of extracting latent representation apt to be compressed in LIC framework. their pioneering work, Ballé et al. [3] first proposed to use a generalized divisive normalization (GDN) [2] to transform the input image into latent space. The later works followed the same VAE framework for LIC but exploited the convolution neural network (CNN) architecture, which is casier to implement and train. Beyond the basic CNN auto-encoder, the introduction of more

---

Entropy Model
Encoder
Hyper
Encoder
y
F
Aux. Entropy
Cocar
Melo)
Coding
LD
Model
Auxiliary Entropy Model
Hr(s)
Hyper Decoder
Ha(
Decoder
Coding
Entropy Model
BAC
Bits
Figure 1: Left: A systematic overview of our method. We adopt the VAE-based framework [3] with hyperprior [4] and channel-wise autoregressive entropy model [35]; besides the original Rate- Distortion loss (CR, LD). introduce auxiliary entropy model and propose the causal context adjustment loss (CCCA) for better training the entropy model. Right: An illustration of the entropy model and the auxiliary entropy model. The auxiliary entropy model does be encoded to predict the following representations, our LCca encourage the predicting gap between to enhance the importance of causal context in early stages.
the information
the two models,
complex nonlinear transforms [9, 11, 33] and various architectures [14, 28, 45, 46] promotes RD performance. Recently, inspired by the great successes Transformers have made in other vision tasks, self-attention modules have been widely utilized for extracting latent representations. Embedding Transformer variants, for example ViT [12], Swin-Transffrme enhances the RD performance. Moreover, Liu et al. [30] proposed hybrid approach, combining conventional CNN and wwin-Transformer. Although these Transformer-bases auto-encoder could improve the RD performance by extracting better latent representations, the inference of transformer architecture has not been well optimized by the existing hardware, resulting in slow coding and decoding speed. In this paper, we borrow ideas from recent advances in image restoration [8] and adopt CNN- based auto-encoder architecture. Thanks to our improved entropy model as well as the powerful NAF-block [8], our LIC model could achieve state-of-the-an compression results with much less runtime than recent Transformer-based approaches.
[31] in the auto-encoder [32, 48, 49]
2.2 Entropy Models for Learned Image Compression
The entropy model plays a key role in LIC for minimizing the bitstream latent representation. In the original work [3], the probability distributions of the latent representation are modeled using non-parne porammrmmttiic, fully factorized density model. In order to improve the distribution predicting accuracy, Ballé et al. [4] introduced side information as hyperprior latent variable and ultimately established the basic VAE architecture of LIC in the past decade. Beyond the hyperprior transmitted in the VAB-based LIC framework, newly proposed extra side information transmitted from encoder to decoder promotes the compression performance as well [20, 43]. Moreover, Duan al. [13] explored the hierarchical VAE structure with multiple hyperpriors.
In addition to the improvement on the side information, the introduction of causal context autore- gression greatly promotes the RD performance of LIC, efficiently utilizing the information from decoded parts without supererogatory amount of bits per pixel (bpp). Minnen al. [35] proposed the first autoregressive structure, using the decoded spatial context to better estimate the current prob- ability distribution. Numerous works attempt to establish an effective causal context for assistance in distribution estimations, such as channel-wis segmentation [36], checkerboard [18], unevenly grouping [17]. very recent work [34] explored different strategies to selectively transmit tokens. However, fixed hand-crafted causal context may not work well in diverse image distributions. In
this work, we impose a loss to adjust the causal context in the training phase, allowing the network to achieve accurate probability estimation.

---

An efficient network structure of the entropy model remains critical for achieving high RD perfor- mance [26, 27, 29]. Apart from the basic causal context and hyperprior entered into the entropy model, more references benefit RD performance [11, 16, 17, 38). Just as how Transformer performs in auto-encoder, the advantages of integrating the entropy model with Transformer are unearthed quickly. Previous works applied various Transformer blocks [20, 23, 25, 30, 37] to enhance features before entropy estimation. Following our modified architecture in auto-encoder, we embed the NAF-block in the entropy model to improve estimation accuracy.
3 Preliminary: Learned Image Compression with Variational Auto-Encoder
Variational Auto-Encoder based Image Compression Framework. Ever since the VAE archi- tecture was established [21], learned lossy image compression techniques maintain the primary constituent structure [3], including an auto-encoder to extract the latent representation for com- pression and an entropy model to assist in entropy coding. Given source image vector æ, the auto-encoder contains a parametric analysis transform 9u to obtain the latent representation y from and parametric synthesis transform ga for reconstruction. y is then quantized to ÿ. the discrete coding symbol for storage. The probability distributions of ÿ are modeled using a factorized density ( (1b) x 2: 2 ():)) As quantization introduces error, which is tolerated in the context of lossy compression, the optimization target approximates the true posterior Pale jae (i) az) with neural network (( (a:) as the expectation of their Kullback-Leibler (KI.) divergence over the data distribution Px:
model 2b as Polv (g)
(b)
(I) The former term refers to the image reconstruction distortion (measured by PSNR or MS-SSIM), and the latter term represents the bit-rate (expected code length). A hyperparameter À is multiplied on the latter term, so that we can control the rate-distortion trade-off obtain various compression rates.
Ew-re D Pg.s|ol Em-p, Egg. Palg (xl log Pa v (ol+b)]
Entropy Model with Hyperprior. However, directly modeling ý with the factorized density model & is less than satisfactory, as the estimation of which is not accurate and out of correlation with the data distributions. To capture the spatial dependence among the elements of ÿ. the side information z is introduced [4]. z is generated by hyper analysis transform ha from y, transmitted as a hyperprior latent feature to help predict the distributions of ğ accurately. Similarly to y. z is quantized to & in the same manner. The probability distributions of 1 are calculated using factorized density model sb, to encode & as Ps|w (2 xか) = [I. (vb) * ( (2.) During the entropy coding process, would be entered into a hyper synthesis transform h. to acquire the estimations {14, σ }: in normal distribution of each element ÿ,. This course can be formulated as Pole (g) (1 = 1I: (N(H,, ())),,??))))) * (1- 3 } ():), with {H, o = (2). The KL. divergence in the
basic VAB structure (Eq. 1) can be expanded as follows:
Pols (ū]=) - log PSI ( (2/x))] (2) On the other side of VAE, the parametric synthesis transform g, recovers the reconstructed image a from the decoded ý. Fig. reveals the general basic structure.
Exrepe [ || Pg.s m=
Autoregressive Entropy Model. In addition, an advanced architecture of the entropy model is the joint autoregression [35], which soon develops into more efficient channel-wise autoregression [36]. In the channel-wise autoregressive structure, the latent representation ý is grouped in the channel dimension and decoded in order. Thus, the second term of the KL divergence in hyperprior structure (Eq. 2) is expanded as:
Eng.s-n log Pols (0) [0]=)] E.g.snal log Pa. (9n ( ( 1 an (ú2) Φ1)
Pús 2.6 .0 (g à Φ1 92) Py. 1". . (9n 2 91.... 1nn
Besides the prior of 2, the estimation of the autoregressive entropy model conditions more on the causal context, that is. the model utilizes the information from the decoded parts (causal context) without introducing additional redundancy in information transmission. Therefore, the more effective the causal context, the stronger the performance of the autoregressive entropy model. Existing methods adopt various hand-crafted causal contexts to enhance it. We expect to establish a way that enables the network to adaptively adjust the causal context. Imposing a loss to explicitly adjust the causal context is delicate way.

---

4 Causal Context Adjustment for Efficient Learned Image Compression
In this section, we introduce the details of our LIC method. We firstly introduce our causal context adjustment (CCA) loss, which is able to explicitly push the encoder to encode important (in terms of information gain) representation at an earlier stage for better predicting the remaining representations. Subsequently, we introduce the implementation details of our efficient LIC method, including our CNN-based encoder and decoder architecture, unevenly grouped autoregressive schedule, light-weight entropy model, and overall loss function.
4.1 Causal Context Adjustment
As introduced in the previous section, exploiting the causal context from the hyperprior and the autoregressive framework to establish conditional distribution task is the key to state-of-the of-the--rtt entropy model. While introducing conditions is undoubtedly beneficial for improving the accuracy of distribution estimation, existing works intuitively set up the context models, such as checkerboard context model and slice-based context model, and there still lack in-depth study on how to constitute the causal context rationally in the literature. More concretely, the rate and distortion loss reflect the prediction error given the causal context and the reconstruction error given the decoded representa- tions, respectively; neither of them could explicitly affect the organization of the causal context. In this section, we introduce the CCA-loss, which explicitly encourages important information of the image to be encoded into earlier causal context, as to enhance the predictability of the autoregres- sive entropy model. To the best of our knowledge, our work is the first attempt that introduces loss instead of intuitively adjusting the context architecture improve the context model.
To introduce our proposed Causal Context Adjustment (CCA) loss, we first revisit the hyperprior and the autoregressive entropy model. Without loss of generality, we consider two-stage autoregressive context model. To encode the same latent representation, the cross entropy of the hyperprior model and the hyperprior + autoregressive model can be written as follows:
HH.PP ((()))), = H (1( (2), P(D 2)) + ( () R)), H...+ARR (g(0)=), (())))(()))) ( P( =)) (4( E. ú1). 2 ú1))
where HHP and HAP.AR represent the cross entropy with hyperprior and with hyperprior + autore- gressive estimation, respectively; q and p denotes the real distribution and the learned entropy model. According to Shannon information theory [39], more information is incorporated in the estimation of .2. H..PAAR. is less than or equal to HH... Moreover, the gap between HH. and HH.PAAR is related to the amount of information ÿJ could provide for estimating ÿ2. Therefore, by calculating the following equation:
HAP. H..PARR. (1(())l)). ((g (l). 91)),
(6)
could obtain the information gain introduced by causal context ÿ1. The above analysis inspires us to explicitly optimize Eq. 6 to enhance V1, encouraging it to encode important information that helps to better estimate V2. Concretely, in addition the original entropy model F() 2) - 922 we introduce an auxiliary entropy model, which only takes 2 as input: G(2) + ÿ2. With the introduced auxiliary entropy model, our CCA-loss can be defined as follows:
I [12: 91) EgirPo E Benpaio [- log Pú, = (02(2) + log Po in (g2 2 91)]
(7)
where Po.ls is (ي and P 9, (52) ÿi) are the estimated distributions of auxiliary and the major entropy model, respectively. The auxiliary and major entropy models are parameterized by two networks, i.c G(&) and F(y1. 2). It should be noted that the auxiliary entropy model is only introduced in the training phase for better optimizing the causal context; in the testing phase, our model still uses F (3) &) to compress the latent representation, our CCA-loss will not introduce additional computational burden for image compression.
An illustration of the proposed CCA-loss can be found in Fig. 1. Besides the above analysis in Eq. 4 to Eq. 7, straightforward interpretation of our CCA-loss is enlarging the prediction gap between the major entropy model F(Di 2) and the auxiliary entropy model G(ž); so that the encoder is forced to adjust causal context and make it contain important information for conditional modeling. For autoregressive models with more than two stages, Eq. 7 can be extended easily. Denote the i-th stage entropy model as Pa. 2.4 ( 12, Te)), which is parameterized with network
5

---

(2 Uc::; ; we introduce the corresponding auxiliary entropy model Pú; P. (0.12, -<i-1), which
is parameterized with network ( (ci-)). The multi-stage CCA-loss can be defined as follows:
LCCa= Byg-rii Beneeapeeeeeeue Re-paro [-
log log.o.......... Po.l ( + 1a. (9:12 أzi)] (8)
With our proposed CCA-loss, the learned image compression model is able to spontaneously adjust the causal context, thereby promoting the rate-distortion performance.
4.2 Training Compression Network with CCA Loss
4.2.1 Auto-Encoder Architecture
Inspired by the recent work [8], which designed a CNN-based nonlinear activation free network to improve image restoration performance, we stack NAF-Blocks [8] in the analysis transform 9a and the synthesis transform 9s- Following the previous CNN-based model [9, 17], we adopt the stacking residual blocks [19] in the auto-encoder transform for better nonlinearity. Due to the simplicity of the information that hyperprior carries, there are only simple convolution layers for the hyper analyzer h and synthesizer h2 Thanks our convolutional architecture, our approach is much faster than recent LIC methods which generally adopt Transformer blocks to comprise the auto-encoder. Detailed architectures of our auto-encoder can be found in the Supplementary Materials.
4.2.2 Channel-wise Unevenly Grouped Entropy Model
To establish a robust causal context and efficiently exploiting it in the autoregressive entropy models the key reaching state-of-the-art The existing approaches generally constitute the causal context model intuitively. In this paper, we propose the causal context adjustment loss (CCA-loss), which compels the analysis transform to generate estimation gain of early-stage context towards the latter latent representation. Theoretically, our proposed CCA-loss is architecture-agnosiic and can be utilized train various encoders adjust the causal context according to the given conditional modeling architecture. However, compared to the checkerboard context model that leverages adjacent spatial information context, it is easier for our convolutional encoder to adjust information across feature channels. We therefore adopt channel-wise grouped autoregressive architecture design our entropy model. Furthermore, since our CCA-loss could explicitly adjust the sigmificant information into the carlier channels, we explore an unevenly grouped strategy take full advantage of the first several informative channels. On account of the accumulated contexts [g), U2, 1i-1] input to the autoregressive entropy model to predict Úi, the unevenly grouped strategy also brings us advantages in the number of parameters and run time. Following our auto-encoded structure, we also utilize NAF-blocks [8] for a superior trade-off between accuracy and speed. For detailed network architectures of our entropy model as well as auxiliary entropy model, please refer to our Supplementary Materials. The comprehensive analysis of the benefits of the evenly and unevenly grouped strategies brought by our CCA-loss will be presented in the ablation study section.
potent causal context, that is, the enhanced
4.2.3 Overall Loss Function
We follow the commonly used rate-distortion optimization framework to train our model. In addition to the rate losses R(ý), R(2) and the distortion loss D( x), our proposed CCA-loss is introduced to explicitly adjust the causal context. The implementation of our CCA requires a group of auxiliary entropy models. In order to obtain feasible auxiliary entropy models, we further introduce auxiliary losses Auu, which let the auxiliary model estimate the same latent representation ý as the major entropy model. Therefore, the overall losses used for training our models are listed as follows:
L = A (R( R()) D(2. x)) + LoCA + LAurs
we only use one parameter À to adjust the compression rate. Detailed ablation studies about the introduced CCA-loss will be presented in our experimental section.

---

5 Experiments
5.1 Experimental Settings
Datasets. We follow the previous work [49] and train our models on the Open Images [24] dataset. Open Images Dataset contains 300k images with short edge no less than 256 pixels. For evaluation, three benchmarks, i.e., Kodak image set [22], Tecnick test set [1], and CLIC professional validation dataset [41], are utilized to evaluate the proposed network.
Implementation details. We set the channel of latent representation y 320 and that of hyperprior # is set as 192. Following the previous works, we turn the quantization operation to [ | instead of [y] and restore ÿ f _4] + p4, which benefits the entropy models. We adopt the unevenly grouped strategy to segment the latent representation into 5 uneven slices. Our detailed unevenly grouped method and discussion on it can be found in the Supplementary Materials. Our experiments and evaluations are carried out on Intel Xeon Platinum 8375C and a single Nvidia RTX 4090 graphics card. We train our network with Adam optimizer. We randomly crop 256 256 sub-blocks from the Open Images dataset [24] with a batch size of 8. We optimize the network with the initial learning rate le 4 for 2M steps and then decrease the learning rate le for another 0.4M steps. The network is optimized with the MSE metric, which represents the distortion loss D in Eq. 9. For the MSE metric, the multipliers À before rate loss are {0.3, 0.85, 1.8,3., 15).
Comparison methods and metries. We compare our method with the hand-crafted coding stan- dards VVC [40], BPG [6] and WebP [15] and recent state-of-the-an methods [4, 11, 17, 30, 45, 491. The results of hand-crafted methods and Ballé2018 [4] are based on the implementation from Com- pressAl AI [5]. while, the results of other methods are provided by the method authors. We mainly use PSNR to evaluate the image quality of compression results and use bits per pixel (bpp) value to indicate the compression ratio. The BD-rate [7] and runtime of several methods are also reported to comprehensively evaluate our model. Following the commonly used setting, we also compare the MS-SSIM metric on the Kodak dataset, the MS-SSIM optimized results by different methods are shown in our Supplementary Materials.
5.2 Ablation Study
We firstly conduct ablation experiments to validate the effec- tiveness of the proposed CCA-loss. In order to facilitate the analysis, we establish tiny model to conduct our ablation ex- periments. We halve the channel number and stacking count of NAF-blocks [8] in our model and only adopt a three-stage autoregressive entropy model. We evaluate our CCA-loss on evenly grouped channel-wiss autoregressive model as well as unevenly grouped channel-wise autoregressive model. The BD- rates of different models are reported in Table 1, without any additional computation in the testing phase, our CCA-loss could
Information Distributed Ratio
Evenly Grouped ChARM w/ vs. w/o CCA-loss
0.6
Even-CCA loss
0.4-
0.3
0.1
improve the evenly grouped and unevenly grouped models by considerable margin. Especially for the case of unevenly grouped strategy, which adopts more aggressive strategy and decodes less number of channels in the early stage, the enhancement brought by our CCA-loss is quite large. The phenomena reveals that utilizing small amount of significant information as the ini- tial condition is beneficial for autoregressive entropy modeling. which is in line with the motivation of our paper.
0.0
Hypusprior
Second dice
Third slice
Information Distributed Ratio
Unevenly Grouped ChARM w/ w/o CCA-loss
0.40
Uneven
UCAsen+CCAless
0.30
0:25
0.20
To further investigate the impact on information distributions of our proposed CCA-loss, we extend a visualization of the quantities of information (code length) in the hyperprior and latent representation. The averaged information distributed ratios on the Kodak testing images by different models are shown in Fig. 2. As can be clearly found in the histogram, for entropy models with the same network architecture, our CCA-loss is able push the network to encode significant information at an earlier stage of the autoregressive model.
0:10
0.05
Hyperarerr
Figure 2: The comparison of av- eraged information distributed ra- tios of various models in Table 1.
7

---

Table 1: Experiments on Kodak dataset. The effects of our proposed Causal Context Adjustment loss (CCA-llos) are verified on various channel- wise autoregressive models. Note that the anchor BD-rate is set as the results of BPG evaluated on Kodak dataset (BD-rate = 0%).
Model
CCA Loss (proposed)
Inference Time(ms)
BD-rate
ChARM (even)
126
-13.31%
ChARM (even)
126
-14.22%
ChARM (uneven)
116
-14.56%
ChARM (uneven)
116
-17.17%
BPG
0%
Ours
LKJCVPR23)
UEICVPRESI
HeVESPR221
2PJICVPRE20
XedCCHNM013
CranglCves25)
ViCjem12.11
BPG
1300
0.2
2.5
0.8
0.2
0.4
0.5
0.2
0.4
Figure 3: Rate-Distortion performance evaluation of PSNR on Kodak dataset (left), CLIC Professional Validation dataset (middle), Tecnick dataset (right), respectively.
5.3 Comparison with State-of-the-art Methods
Rate-Distortion Comparison. We evaluate the rate-distortion performance of our proposed models by drawing the rate-distortion curves. The distortion is assessed by PSNR while the rate is calculated by the bits per pixel (bpp). We first compare our proposed network with hand-crafted codec meth- ods [6, 15, 40] and the LIC models that once reached state-of-the-a art (SOTA) [4, 11, 17, 30, 45, 49] on the Kodak dataset. The result the PSNR metric is presented in Fig. (left), which demonstrates that our proposed methods could outperform other SOTA methods. The middle sub-figure and the right fub-figure Fig. 3 are evaluated on the CLIC Professional Validation dataset and the Tecnick dataset, respectively. The SOTA results in various datasets show the generalization and robustness of our proposed model.
Compression Latency. As described in our introduction, we established convolutional compres- sion model for the pursuit of efficient compression. In Table 2, we present the coding latency, as well as the number of parameters and GFLOPs, by our proposed network and recent state-of-the-an methods [30, 48, 49]. The BD-rate values by different methods are also provided for reference, the anchor RD performance
which is set as the results of VVC (vtm-12.1) 1) on Kodak dataset (BD-rate =0%). As can be found in the table, our method achieves a better trade-off between compression performance and coding latency than the competing methods. With more than 20% less runtime, our model obtains about 2% BD-rate gain over [30].

---

Table 2: Comparison of coding complexity evaluated on Kodak dataset. All the models are evaluated on the same platform. The lower BD-rate better.
Inference Latency(ms)
Model
#Params
FLOPs(G)
BD-Rate
Tot.
Enc.
Dec.
Zou et al. [49]
424
248
176
99.83M
200.11
-4.01%
Zhu et al. [48]
272
129
143
56.93M
364.08
-3.00%
Liu et al. [30]
255
122
133
75.90M
700.65
-11.88%
Ours
201
109
92
64.89M
615.93
-13.87%
VVC
0%
Ground Truth
VTM [0.254 31.97)
Liu et al. [0.255] 32215)
Ours [0.246] 32.41)
Ground Truth
VTM [0.123 32.40]
Liu al. [0.099] 32.19]
Ours [0.104 32.88)
Ground Truth
VIM [0.1151 31.088
Liu et al. [0.091 330933
Ours [0.098] | 31.33]
Figure 4: Visualization of the reconstructed images (top: kodim19, middle: kodim10, bottom: kodim4) from Kodak dataset. The titles under the sub-figures are represented as [bpp PSNR(dB)).
Visualization Analysis. Our proposed learned image compression technology is capable of restor- ing the image details. Fig. 4 shows two sets comparisons with the reconstruction of VVC [40] and recent SOTA model [30]. The visualization results produced low bit-rates the Kodak dataset [22]. The comparison of the reconstructed images demonstrates that our model restores more detailed and complicated textures than other methods. For example, restore more sharp textures on the hat (kodim4), more details of the grassland (kodim19) and wrinkles on the sails (kodim
6 Conclusion
In this work, we explore the approach adjust the causal context, which enables superior channel- wise autoregressive model and more accurate estimation in probability distributions, By imposing the Causal Context Adjustment loss (CCA-loss) and the unevenly channel-wise grouped strategy on our proposed CNN- model, we achieve state-of-the- rate-distortion performance. Thanks the advantages of convolutional neural network, our discussed unevenly grouped schedule and the training method by the proposed CCA-loss, our learned image compression model maintains great trade-off between compression latency and RD performance. Furthermore, since we did not dive into the information redistributed phenomenon brought by the unevenly grouped strategy and CCA-loss training this paper, the issue of the laws about the information distributed among the latent representation to be compressed is still worth investigating in the future.

---

Acknowledgement
This work was supported by National Natural Science Foundation of China (No. 62250001, 6223 62020106011 I), Beijing Natural Science Foundation (No. L223021) and Sichuan Natural Science Foundation (No. 2024NSFTD0041).
References
[1] Asuni, N., Giachetti, A.: Testimages: a large-scale archive for testing visual devices and basic image processing algorithms. In: STAG. pp. 63-70 (2014)
[2] Ballé, J., Laparra, V., Simoncelli, E.P. Density modeling of images using a generalized
normalization transformation. In: International Conference on Learning Representations (2016)
2
[3] Ballé, J., Laparra, V., Simoncelli, E.P.: End-to-end optimized image compression. In: Interna- tional Conference on Learning Representations (2017),
id=rJxd[3jeg 1, 2, 4,
[4] Ballé, Minnen, D., Singh, S., Hwang, S.J., Johnston, N.: Variational image compression with a scale hyperprior. In: International Conference on Learning Representations (2018), http:/iww......oooo/dddooooro///
1, 4, 7,
[5] Bégaint, J., Racapé, F., Feltman, S., Pushparaja, A.: Compressai: pytorch library and
evaluation platform for end-to-end compression research. arXiv preprint arXiv:2011.000 1.03029
(2020)* 7
[6] Bellard, F:: Bpg image format (2015),
org/ 7.
[7] Bjontegaard, G.: Calculation of average psnr differences between rd-curves. ITU SG16 Doc. VCEG-M33 (2001)
[8] Chen, L.,, Chu, X., Zhang, X., Sun, J.: Simple baselines for image restoration. In: European conference on computer vision. pp. 17-33. Springer (2022) 3, 6, 13
[9] Chen, T., Liu, H., Ma, Z., Shen, Q.. Cao, X., Wang, Y.: End-to-end learnt image compression via non-local attention optimization and improved context modeling. IEEE Transactions on Image Processing 30, 3179-3191 (2021) 3, 6, 13
[10] Cheng, Z., Sun, H., Takeuchi, M., Katto, J.: Deep convolutional sisteencoder-bassee lossy image compression. In: 2018 Picture Coding Symposium (PCS). pp. 253-257. IEEE (2018)
[11] Cheng, Z., Sun, H., Takeuchi, M., Katto, J.: Learned image compression with discretized gaus- sian mixture likelihoods and attention modules. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 7939-79999999 7948 (2020) 1,3,4,7,
[12] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., al.: An image is worth 16x16 words: Transformers for image recognition at scale. In: International Conference on Learning Representations (2020)
3
[13] Duan, Z., Lu, M., Ma. J., Huang, Y., Ma, Z., Zhu, F. Qarv: Quantization-ane aware resnet for lossy image compression, IEEE Transactions on Pattern Analysis and Machine Intelligence (2023) 3
[14] Gao, G., You, P., Pan, R., Han, S., Zhang, Y., Dai, Y., Lee, H.: Neural image compression via attentional multi-scale back projection and frequency decomposition. In: Proceedings of the IEEE/CVF International Conference Computer Vision. pp. 14677-14686 (2021) 3
[15] Google: Web picture format (2010) 7, 15
[16] Guo, Z., Zhang, Z., Feng, R., Chen, Z.: Causal contextual prediction for learned image compression. IEEE Transactions on Circuits and Systems for Video Technology 32(4), 2329- 2341 (2021) 4
10

---

[17] He, D., Yang, Z., Peng, W., Ma, R.. Qin, H., Wang, Y:: Elic: Efficient learned image compression with unevenly grouped space-channel contextual adaptive coding, In: Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 5718-5777 (2022) 1,
2, 4, 6, 7, 8, 13, 14
[18] He, D., Zheng, Y., Sun, B., Wang, Y., Qin, H.: Checkerboard context model for efficient learned image compression. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 14771-14780 (2021) 2,
[19] He, K., Zhang. X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceed- ings of the IEEE conference computer vision and pattern recognition, pp. 770-778 (2016)
[20] Kim, J.H., Heo, B., Lee, J.S.: Joint global and local hierarchical priors for learned image compression. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5992-6001 (2022) 3,4
[21] Kingma, D.P., Welling, M.: Auto-encoding variational bayes (2022) 4
[22] Kodak, E.: Kodak lossless true color image suite (photoed pcd0992) (1993), http: //r0: /u//////kuu//////
graphics/koda
[23] Koyuncu, A.B., Gao, H., Boev, A., Gaikov, G., Alshina, E., Steinbach, E.: Contextformer: A transformer with spatio-channel attention for context modeling in learned image compression. In: European Conference on Computer Vision. pp. 447-463. Springer (2022)
[24] Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., et al.: The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. International journal of computer vision 128(7), 1996-19881 (2020) 7
[25] Li, H., Li, S., Dai, W., Li, C., Zou, J., Xiong, H.: Frequency-aware transformer for learned image compression. In: The Twelfth International Conference on Learning Representations (2023) 4
[26] Li, M., Ma, K.. You, Zhang, D., Zuo, W.: Efficient and effective context-based convolutional entropy modeling for image compression. IEEE Transactions on Image Processing 29, 5900- 5911 (2020) 4
[27] Li, M., Zuo, W., Gu, S., You, J., Zhang, D.: Learning weighet- deep image compression. IEEE transactions on pattern analysis and machine intelligence 43(10), 3461 (2020) 4
[28) Lin, C., Yao, J., Chen, F., Wang, L.: A spatial rnn codec for end-to-end image compression. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13269-13277 (2020) 3
[29] Lin, F., Sun. H., Liu, J., Katto, J.: Multistage spatial context models for learned image compres- sion. In: ICASSP 2023-2023 JEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 1-5. IEEE (2023) 4
[30] Liu, J., Sun, H., Katto, J.: Learned image compression with mixed transformer-enn architectures. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 14388-14397 (2023) 1, 4, 7, 8, 15
[31] Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y.. Zhang, Z., Lin, S., Guo, B.: Swin transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF
international conference on computer vision. pp. 10012-10022 (2021) 3
[32] Lu, M., Guo, P. Shi, H., Cao, C., Ma, Z.: Transformer-bassee image compression. arXiv preprint arXiv:2111.00 NMV:2111.06607 1.06707 (2021)² 3
[33] Ma, H., Liu, D., Yan, N., Li, H., Wu, F:: End-to-end optimized versatile image compression with wavelet-like transform. IEEE Transactions on Pattern Analysis and Machine Intelligence 44(3), 1247-1263 (2020) 3

---

[34] Mentzer, F., Agustson, E., Tschannen, M.: M2t: Masking transformers twice for faster decoding. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 5340-5349
(2023) 3, 14
[35] Minnen, D., Ballé, J., Toderici, G.D.: Joint autoregressive and hierarchical priors for learned image compression. Advances in neural information processing systems 31 (2018) 1, 2, 3, 4, 13
[36] Minnen, D., Singh, S.: Channel-wise autoregressive entropy models for learned image com- pression. In: 2020 IEEE International Conference Image Processing (ICIP). pp. 3339-3343.
IEEE (2020) 1, 2, 3, 4, 13
[37] Qian, Y., Lin, M., Sun, X., Tan, Z., Jin, R.: Entroformer: A transformer-based entropy model for learned image compression. In: International Conference on Learning Representations (May 2022) 4
(38) Qian, Y., Tan, Z., Sun, X., Lin, M., Li, D., Sun, Z., Hao, I.,, Jin, R.: Learning accurate entropy model with global reference for image compression. In: International Conference on Learning Representations (2021), https: hatpp:://pppnree/e wwww. seet/formmidd=cbl...
[39] Shannon, C.E.: A mathematical theory of communication. The Bell system technical journal 27(3), 379-423 (1948)
[40] Team, J.V.E.: Versatile video coding reference software version 12. (2. (202)) https: 1,
//vcgit. fraunhofrr http://e////aaaraaraaaaa////////
[41] Toderici, G., Shi, W., Timofte, R., Theis, L., Ballé, J., Agustsson, E., Johnston, N., Mentzer, F:: Workshop and challenge on learned image compression (clic2020). In: CVPR (2020) 7
[42] Wallace, G.K.; The jpeg still picture compression standard. Communications of the ACM 34(4),
30-44 (1991) 1, 15
[43] Wang, D., Yang. W., Hu, Y., Liu, J.: Neural data-dependent transform for learned image compression. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17379-17388 (2022)
[44] Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multiscale structural similarity for image quality assessment. In: The Thrith-Seventh Asilomar Conference on Signals, Systems & Computers, 2003. vol. 2, pp. 1388-1402. Icee (2003) 1
[45] Xie, Y., Cheng. K.L., Chen, Q.: Enhanced invertible encoding for learned image compression. In: Proceedings of the 29th ACM international conference on multimedia. pp. 162-170 (2021)
3, 7,
[46] Yang, F., Herranz, L., Cheng, Y., Mozerov, M.G.: Slimmable compressive autoencoders for practical neural image compression. In: Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 4998-5007 (2021) 3
[47] Yang, Y., Bamler, R., Mandt, S.: Improving inference for neural image compression. Advances in Neural Information Processing Systems 33, 573-584 (2020)
[48] Zhu, Y., Yang, Y., Cohen, T.: Transformer-based transform coding. In: International Conference on Learning Representations (2022), httpp::/openreview http:///ppmeeeeeeeeeeeeerrrrrraa wtt.:///ooruumiiiddddDodddd
3, 8, 9
[49] Zou, R., Song, C., Zhang, 7.: The devil is in the details: Window- basee attention for image compression. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 17492-17501 (2022) 3, 8,9
12

---

A Network Architecture
A.1 Architecture of transform networks
Table 3: Architecture of main transforms and hyper transforms.
Analyzer go
Synthesizer gs
Hyper Analyzer ha
Hyper Synthesizer his
Conv 5x5, dimo, s2
TConv 5x5, dimg,
Conv 5x5, dimg, s2
TConv 5x5, dimg,
ResidualBlockx3
NAF-Blockx4
GELU
GELU
NAF-Blockx4
ResidualBlockx3
Conv 5x5, dimg, s2
TCony 5x5, dimg, s2 GELU
Conv 5x5, dim1, s2
TConv 5x5, dim1, s2
GELU
ResidualBlockx3
NAF-Blockx4 x
Conv 5x5, 192, s2
TConv 5x5, 320, s2
NAF-Blockx4
ResidualBlockx3
Conv 5x5, dimg, s2
TConv 5x5, dimg.
ResidualBlockx3
NAF-Block
NAF-Blockx4
ResidualBllok x3
Conv 5x5, M, s2
TConv 5x5, 3, s2
As introduced in our main paper, our compression framework is adopt the VAE framework proposed by Ballé et al. [3], and the strategy of hyperprior [4] and autoregressive entropy model [35]. Generally, the transform network comprise an analyzer Io and a synthesizer 9s. which play the
role of feature extraction and image reconstruction. For extracting side information, another pair of hyper analyzer fa and hyper synthesizer fú is used to extracting and reconstructing the hyperprior variable z. The detailed network architecture of the above components can found in Table 3.
The 5x5 convolution and the 5x5 transposed convo- lution are utilized downsample and upsample the feature maps, respectively. Following the previous works [9. 17], we adopt commonly used resid- ual blocks and the newly proposed NAF-Blocks to establish the analyzer and synthesizer. While, due to
NAF-black
Entropy Model
ÿ
LayerNorm
the simplicity of the information that hyperprior carries, there are only simple convolution layers for the hyper analyzer ha and hyper synthesizer hy. The dimension numbers dimo, dim, and dimy in the table set 192, 224 and 256, respectively.
ixd,
NAF-block
SimpleGate
Cony 3x3,
SCA
A.2 Architecture of entropy model
Conv 1x1, sl
Our proposed entropy model utilizes the NAF-block well (see Fig. 5). The stacking NAF-blocks can en- hance the concatenated features input to the entropy model, in order obtain of the latent representation. The dimension of the la- tent representation NAF-block set as 224. Please note that we do not conduct the special training strat- egy like [8] for the simple channel attention (SCA) in the NAF-block, on account of no performance loss caused by this. Following the previous work [36], we append the latent residual prediction (LRP) to restore the error introduced by quantization operation. For our auxiliary entropy model, the only difference that the input removes the previous one slice, that is, replace ÿz with ÿzi- in Fig. 5.
LayerNorm
Latent Resiciual Prediction
accurate estimation
Conv 2xd, s1
Decoding
GELU
SimpleGate
Conv 3x3,
GELU
Conv 3x3, sl
Conv 2x1, sl
Figure 5: Architecture of NAF-block, Entropy
Model and Latent Residual Prediction (LRP).
13

---

B Adjustable Unevenly Grouped Strategy
To take full advantage of our proposed CCA-loss, we adopt the unevenly grouped strategy proposed by He et al. [17] in our method. In this part, We dive deeper into the specific grouped method and the advantages brings. For the channel-wise autoregressive entropy models, the input them is accumulated as the decoding progresses to the latter slices.
Efficiency. For the i-th stage entropy model, the causal context contains (ú1, y2, y3, -i-1 xx C; where H W is the spatial size of the latent representation and C₁ denotes the channel number of y; For the overall n-stage entropy model, the shape of the total causal context written as Σn #x C. which can be expanded as Σr- TR. x x x(n-i)C, From this expression, we could see that the former slices are reused more times, leading to more parameters and latency. Thus, the unevenly grouped strategy could benefit the model complexity, as Table shows in the ablation study section.
Rate-Distortion Trade-off. In this part, we analyze the effects of selecting different grouped schedules. Inspired by previous work [34], we parameterize the unevenly grouped strategy via power schedule, i.e., C(i) = N i²,, where & denotes the steepness of the increasing slices and N normalizes the n-stage autoregressive slices in sum of M. For selecting best schedule for our model, models with different grouping hyperparameters with the loss function in Eq. 9 (including We evaluate grouping strategies with different val- ues. The rate-distortion trade-offs by different models are shown in Fig. 6, the RD curve achieved by our selected schedule (i.e. & 1.7) is presented for ref- erence. As can be seen in the figure, the setting of 1.7 achieves the best rate-distortion performance, which we select for our ultimate SOTA model.
25.8
PSW
0.50
D61
0.62
0.53
train compression
CCA-loss).
1-1.0
D.
0.5
Figure 6: Compression results with different grouped schedules. Detailed experimental settings can be found in the main text.
C MS-SSIM Optimized Result
For higher MS-SSIM performance to adapt the real eyesight, also produce the model of the MS-SSIM optimized objective. The distortion loss is replaced by 1 --MSSSSIM(2) and the multiplier before the rate loss are {0.2, 0.65, 1.5, 3.2, 6, 15). The comparison of the rate-distortion curves with previous works is released in Fig. 7.
22
Ours
UuICVPR2
ZouICVPR22)
20
ChengiCVP120)
WCCvvm12.
18
PSNR
16
14
12
0.4
0.6
Figure 7: Aate-Distortion performance evaluation of MS-SSIM on Kodak dataset.
14

---

D Image Reconstruction Visualization
We compare the reconstruction results on kodim20 (Fig. 8) and kodim24 (Fig. 9) of our model with those of Liu et al. [30] and several hand-crafted methods, i.e., VVC [40], Webp [15], JPEG [42].
Ground Truth
VTM [0.081 31.118
Liu et al. [0.083 31.82]
JPEG [0.195| 30.07]
Ours [0.083 32.07]
Figure 8: Visual comparison on reconstructed propeller airplane (kodim20) image.
15

---

Ground Truth
VTM [0.492| 30.86]
Liu et al. [0.458 31.21]
WebP [0.513] | 99.78]
Ours [0.459 31.47]
Figure 9; Visual comparison on reconstructed pattern on the walls of the house (kodim24) image.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Our abstract and introduction accurately reflect our paper's contributions and
scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [NA]
Justification:
To our best knowledge, our proposed methods have no limitation.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
17

---

Justification:
All the theoretical results in our paper are provided with the full set of assumptions and a complete and correct proof.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Our paper provides the detailed network architecture and the settings of the hyperparameters.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
18

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification: The data used from the Open Images datasets are publicly available. Since our codes have not been sorted and filed well, to avoid bringing the burden on reading a messy code and the details are presented enough for reproducing in our paper, we will release them when the codes are sorted well.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurlPS code and data submission guidelines (https:
ctaips.cc/public// weeiii/gguieeee guwdee/ide//// CCoeeSumeeeeaeeeeeeee wwaibbmmsssion ssssnnollll. onsoiicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We set up a special experimental setting section and provide more details in the Supplementary Materials.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: For learning-based image processing methods, is difficult to assess error bars. However, massive dataset drawn from natural images is utilized to perform the experiments and evaluate the results. The large number experiments ensures the statistical significance of our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
19

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide the compute resources in our experimental setting section.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: / http:///eeweus.ccccpuubbba cc/public//Biccccccc ethuideeil ines?
Answer: [Yes]
Justification: We with the NeurIPS Code of Ethics.
that research conducted in the paper conform, in every respect,
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our paper has societal impact of the work performed.
20

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considcrations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We are sure that the creators or original owners of assets (c.g., code, data, models), used in the paper, are properly credited and are the license and terms of use explicitly mentioned and properly respected.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
21

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
22

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
23


Reviews:
[{"review_id": "JtvZJphJrc", "reviewer": "Reviewer_NpdM", "summary": "The paper proposes an auxiliary neural network in conjunction with a loss  function during training to improve the better contextual modeling of the data during training phase. The advantage of the proposed technique is that the auxiliary network is not required during inference and hence the complexity is minimal. The work conducts a series of experiments that supports the claims.", "questions": "* Could you elaborate on MAC operations versus FLOPS?", "limitations": "No discussion on limitations", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper is well written.\n* The approach to bring in context information into play is relatively novel.\n* Experiments are convincing and covers the necessary and sufficient datasets.\n* The ablation study supports the claims.\n* Gain versus FLOP calculation is good.", "weaknesses": "* There is no strong weakness", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive comments. \n\nMultiply–accumulate operations (MACs) and floating point operations (FLOPs) are two important measurements of model computation. More specifically, MACs calculate the number of multiply-accumulate operations, and each MAC operation consists of a multiplication followed by an addition. While, FLOPs calculate the number of floating-point operations, including all types of floating-point computations such as addition, subtraction, multiplication, and division. In the literature of learned image compression (LIC), the number of FLOPs is reported as the metric of the complexity of the model. In our submitted paper, we followed the commonly adopted setting and reported the FLOPs number to compare with other methods. In order to answer your question, we recalculated the number of MACs and FLOPs by different methods and reported in the following table. As can be found in the table, our proposed method could achieve state-of-the-art compression results with less computational footprint. \n\n|        Model        | Enc. Time | Dec. Time | #Params | BD-rate | FLOPs(G) | MACs(G) |\n| :----------------- | :-------: | :-------: | :-----: | :-----: | :------: | :-----: |\n| Zou et al. (CVPR2022) |   248ms   |   176ms   | 99.83M  | -4.01%  |  200.11  | 199.94  |\n| Zhu et al. (ICLR2022) |   129ms   |   143ms   | 56.93M  | -3.00%  |  364.08  | 209.40  |\n| Liu et al. (CVPR2023) |   122ms   |   133ms   | 75.90M  | -11.88% |  700.65  | 702.81  |\n|        Ours         |   109ms   |   92ms    | 64.89M  | -13.87% |  615.93  | 492.65  |\n|         VVC         |     -     |     -     |    -    |   0%    |    -     |    -    |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cmxpo9N7uv", "reviewer": "Reviewer_r1X1", "summary": "This work proposes a novel causal context adjustment loss to explicitly guide the encoder in prioritizing important information at the early stage of the autoregressive entropy model, which is both interesting and significant compared to the implicit modeling in ELIC. The loss is designed based on the entropy loss between different conditional channel-wise transformed latent slices. Extensive experiments have demonstrated the effectiveness of this design.", "questions": "Whether this loss can be combined with the spatial context modeling strategy like \"checkerboard\"?", "limitations": "No limitation analysis is provided in this work.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This work proposes a novel causal context adjustment loss to explicitly guide the encoder in prioritizing important information at the early stage of the autoregressive entropy model, which is both interesting and significant compared to the implicit modeling in ELIC. \n2. The loss is designed based on the entropy loss between different conditional channel-wise transformed latent slices. Extensive experiments have demonstrated the effectiveness of this design.", "weaknesses": "There are some weaknesses that need to be addressed:\n\n1. In Table 1, why is the performance only compared with anchor BPG instead of VVC? Whether this strategy is effective with stronger foundational compression codecs? It is suggested to validate your method with stronger codecs and compare the performance with VVC as shown in Table 2.\n2. The abstract could be reorganized for a better illustration of the contribution. For example, you should highlight the explicit guidance for the encoder to adjust important information in the early stage of the autoregressive entropy model.\n3. It would be better to compare with ELIC in lines 58-60.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive comments and insightful suggestions. We have conducted additional experiments, and our detailed feedback to your comments is listed below. \n\n**Q1: Ablation study on stronger codecs**\n\nIn our submitted paper, we conducted ablation study with small model to facilitate our experiments. To address your concern about whether our proposed CCA-loss is helpful for stronger compression models, we further conducted an ablation study on our large model. We retrained our model without the proposed CCA-loss and reported the BD-rate in the following table. Moreover, we follow your request and use VVC as the baseline in the table. As shown in the table, our proposed CCA-loss is beneficial for both small and large models. \n\n|    Model     |   CCA Loss   | Inf. Time(ms) |            BD-rate             |\n| :---------- | :----------: | :-----------: | :---------------------------- |\n| Ours (large) |              |      201      |            -12.39%             |\n| Ours (large) | $\\checkmark$ |      201      | -13.87% ( $\\downarrow$ 1.48% ) |\n| Ours (small) |              |      116      |             4.78%              |\n| Ours (small) | $\\checkmark$ |      116      |  1.24% ( $\\downarrow$ 3.54% )  |\n|     VVC      |      -       |       -       |               0%               |\n\n**Q2: Revision suggestions (Abstract, Compare with ELIC in lines 58-60)**\n\nWe concur with the reviewer’s observation that there exists some inadequacy in our abstract and introduction sections. we would reorganize the abstract section and add the comparison with ELIC to the introduction section in our revised manuscript.\n\n**Q3: Combining with spatial context modeling**\n\nAs we have stated in our paper (lines 234 - 236), although our proposed CCA-loss is able to guide the encoder to adjust the causal context, the convolutional encoder used in our work can only extract information in a spatially invariant manner. Therefore, for commonly used spatial context models, such as the checkerboard model, our current model is unable to adjust the causal context to improve the compression performance. In order to boost the spatial context model with our proposed CCA-loss, we need to have a tailored encoder which could adjust representation according to spatial position. In the future, we will explore whether we could propose a new spatial context model for Transformer-based encoder, which uses decoded tokens to estimate the remaining tokens; in that case, we believe that our proposed CCA-loss could play a positive role in learning better spatial context model.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wugoMf2585", "reviewer": "Reviewer_cfcU", "summary": "This paper proposed Causal Context Adjustment loss (CCA-loss) to explicitly adjust the causal context. However, this paper also proposes an efficient image compression model, which seems to be irrelevant to its main contribution, namely CCA loss. CCA loss and efficient architectures are orthogonal, and there is no obvious correlation between the two, which makes this paper rather scattered.", "questions": "Why use MS-SSIM instead of SSIM?", "limitations": "Please see the weaknesses and questions.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper proposed CCA loss to force the entropy model to learn better causal context organization.\n2. This paper achieved the SoTA performance in the image compression task. \n3. This paper is more efficient than previous methods, as reflected in its lower latency.", "weaknesses": "1. CCA Loss should assist more network architectures to prove its effectiveness. Using NAFNet alone is not enough.\n2. This paper lacks analysis. Is the causal context organization of the model better after using CCA Loss? Some visual analysis can be given, for example, after using CCA Loss, can the model notice areas that the model without CCA Loss did not notice?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments, our detailed feedback towards your questions is listed as follows.\n\n**Q1: Evaluate CCA-loss on more network architectures**\n\nWe follow the reviewer's suggestion and replace the NAF-block in our paper with the residual block and the Swin-Transformer block, respectively. The compression results by different network architectures are shown in the following table, which clearly demonstrates the effectiveness of our proposed CCA-loss in improving the compression performance by different network architectures.\n\n|         Model          |   CCA Loss   | Inf. Time(ms) |            BD-rate            |\n| :-------------------- | :----------: | :-----------: | :--------------------------- |\n|  residual-block based  |              |      113      |            -15.02%            |\n|  residual-block based  | $\\checkmark$ |      113      | -16.69% ( $\\downarrow$ 1.67%) |\n| swin-Transformer based |              |      169      |            -16.46%            |\n| swin-Transformer based | $\\checkmark$ |      169      | -17.67% ( $\\downarrow$ 1.21%) |\n|    NAF-block based     |              |      116      |            -14.56%            |\n|    NAF-block based     | $\\checkmark$ |      116      | -17.17% ( $\\downarrow$ 2.61%) |\n|          BPG           |      -       |       -       |              0%               |\n\n**Q2: The analysis on casual context distribution**\n\nThe motivation of our proposed CCA-Loss is to adjust the causal context, making the latter representation more accurately predicted by the previously decoded representations. In our ablation study section, we have provided the information distribution ratios of different slices with and without our proposed CCA-loss (Figure 2 in our paper), and analyzed that our CCA-loss is able to push the network to encode significant information at earlier stages of the autoregressive model (lines 301-309). In order to address your concern, we further show **the coding bit map of the first three slices of *kodim06*, *kodim08* and *kodim13*, in our rebuttal PDF file**. The visualization examples clearly show the advantage of our CCA-loss. In our channel-wise autoregressive entropy model, CCA-loss is able to push the network to encode significant information at earlier slices and use them to better recover the latter slices. Overall, our model trained with CCA-loss could obtain a better rate-distortion trade-off. \n\n**Q3: MS-SSIM instead of SSIM as the eyesight evaluation**\n\nMS-SSIM is an extended version of SSIM, which calculates SSIM on multi-scales. In the literature of learned image compression, most recent works [4, 11, 14, 17, 20, 23, 25, 30, 37, 45, 49] adopt MS-SSIM as a loss or evaluation metric to train or evaluate the compression network. Therefore, in our paper, we follow the commonly adopted setting and use MS-SSIM instead of SSIM.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IuPF6ITkQN", "reviewer": "Reviewer_YBcc", "summary": "The paper presents a novel approach to learned image compression (LIC) by introducing a Causal Context Adjustment loss (CCA-loss). This method aims to improve the rate-distortion (RD) performance of autoregressive entropy models used in LIC. The proposed approach allows the neural network to adjust the causal context dynamically, enhancing the accuracy of latent representation estimations. The paper also leverages a convolutional neural network (CNN) based model and an unevenly channel-wise grouped strategy to achieve a balance between inference latency and RD performance. Experimental results on benchmark datasets demonstrate that the proposed method outperforms existing state-of-the-art LIC techniques in both RD performance and computational efficiency.", "questions": "1) Further Exploration of Causal Context Models:\na) Question: The paper mentions that current methods for organizing causal contexts are somewhat intuitive. Can you elaborate on the potential for more structured approaches in organizing causal contexts? Are there any preliminary ideas or experiments that could guide future research in this direction?\nb) Suggestion: Consider conducting a more detailed analysis or ablation study on different causal context models. This would help in understanding the strengths and limitations of various approaches and could provide a more comprehensive justification for the chosen method.\n2) Effectiveness of CCA-loss Across Different Models:\na) Question: The paper demonstrates the effectiveness of CCA-loss within the proposed model. Have you tested the applicability of CCA-loss with other LIC architectures or models? How does it perform in different settings?\nb) Suggestion: Extending the evaluation of CCA-loss to other architectures could demonstrate its versatility and robustness. Including results from such experiments would show whether the benefits of CCA-loss are generalizable across different models.", "limitations": "1) The authors mention that current methods for organizing causal contexts are somewhat intuitive and that further exploration is needed. However, they do not delve deeply into the potential limitations this may impose on their results.\n2) The implementation complexity of the proposed method, particularly the uneven channel-wise grouped strategy and CCA-loss function, is not explicitly discussed as a limitation.", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) Innovative Loss Function: The introduction of the CCA-loss represents a novel contribution that explicitly adjusts the causal context, leading to significant improvements in the accuracy of autoregressive entropy models.\n2) Efficient Architecture: The proposed CNN-based model with uneven channel-wise grouping demonstrates significant improvements in both rate-distortion performance and computational efficiency.\n3) Comprehensive Evaluation: The paper provides thorough experimental validation on multiple benchmark datasets, demonstrating superior RD performance and efficiency over state-of-the-art methods.\n4) Clear Contributions: The paper clearly articulates its contributions, including the development of CCA-loss, the efficient CNN-based architecture, and the evaluation of these innovations through rigorous experiments.\n5) Practical Implications: The reduction in compression latency by over 20% compared to state-of-the-art methods highlights the practical applicability of the proposed method.", "weaknesses": "1) Limited Exploration of Context Models: The paper mentions the potential for further investigation into the organization of causal contexts, indicating that current methods are somewhat intuitive and may benefit from more structured approaches.\n2) Complexity in Implementation: While the proposed method is efficient, implementing the uneven channel-wise grouped strategy and the CCA-loss function may pose challenges for practitioners not well-versed in deep learning techniques.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive comments and insightful suggestions, which have significantly inspired us and enhanced our work. Our detailed feedback towards your comments is listed as follows.\n\n**Q1: Further Exploration of Causal Context Models (future research direction)**\n\nLeveraging decoded information to predict remaining information, i.e., conditional distribution modeling, plays a vital role in recent learned image compression (LIC) works. The previous works investigate different causal context architectures to make use of the naturally (indirectly optimized) dependency between learned representations, while, in this paper, we made the first attempt at imposing loss to adjust the causal context, which explicitly optimizes the predictability of the later information given the decoded information. \n\nIn this paper, we validate our idea on the most commonly used causal context model, i.e., channel-wise autoregressive model. We hope our work could inspire follow-up works which not only investigate appropriate architectures for better modeling causal context but also pay attention to advanced learning strategy for obtaining better causal context model. To be more specific, with an advanced causal context adjusting capability, we think the encoder and entropy model in LIC should be designed together. For example, entropy models that use spatial context models should work together with the spatial-variant encoder; appropriate token coding order can be optimized in the token predicting framework, etc. We will follow your suggestion and conduct a more detailed analysis or ablation study on different causal context models. \n\n**Q2: Evaluate CCA-loss on more network architectures**\n\nWe follow the reviewer's suggestion and replace the NAF-block in our paper with the residual block and the Swin-Transformer block, respectively. The compression results by different network architectures are shown in the following table, which clearly demonstrates the effectiveness of our proposed CCA-loss in improving the compression performance by different network architectures.\n\n|         Model          |   CCA Loss   | Inf. Time(ms) |            BD-rate            |\n| :-------------------- | :----------: | :-----------: | :--------------------------- |\n|  residual-block based  |              |      113      |            -15.02%            |\n|  residual-block based  | $\\checkmark$ |      113      | -16.69% ( $\\downarrow$ 1.67%) |\n| swin-Transformer based |              |      169      |            -16.46%            |\n| swin-Transformer based | $\\checkmark$ |      169      | -17.67% ( $\\downarrow$ 1.21%) |\n|    NAF-block based     |              |      116      |            -14.56%            |\n|    NAF-block based     | $\\checkmark$ |      116      | -17.17% ( $\\downarrow$ 2.61%) |\n|          BPG           |      -       |       -       |              0%               |\n\n**Q3: Complexity in Implementation**\n\nThanks for your suggestion. We have tried our best to introduce our model clearly, and the uneven grouping strategy and the CCA-loss can be implemented with PyTorch by changing channel numbers in the entropy model and introducing auxiliary networks. Furthermore, we will release our code for implementation with easy-to-understand annotation after we finish the code cleanup.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We are sincerely grateful to all the reviewers for their valuable time and expert insights; we truly appreciate their diligent work and constructive feedback.

Meta-review: While there were some reservations in the initial reviews, the rebuttal addressed most of the reviewers' concerns. All the reviewers unanimously voted for acceptance in their final rating. The AC agrees with the reviewers' recommendations and supports acceptance of the paper. The AC recommends including additional evaluations performed during the discussion of the paper (either in the appendix or main paper, space permitting.).
Decision: Accept (poster)