Taming Hierarchical Image Coding Optimization: A Spectral Regularization Perspective
Authors: Wuyang Cong
Junqi Shi
Ming Lu
Xu Zhang
Zhan Ma
Venue: ICLR 2026 Poster (2026)
Keywords: Learned Image Compression
Hierarchical Variational Autoencoders
Spectral Regularization
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Hierarchical coding offers distinct advantages for learned image compression by capturing multi-scale representations to support scale-wise modeling and enable flexible quality scalability, making it a promising alternative to single-scale models. However, its practical performance remains limited. Through spectral analysis of training dynamics, we reveal that existing hierarchical image coding approaches suffer from cross-scale energy dispersion and spectral aliasing, resulting in optimization inefficiency and performance bottlenecks. To address this, we propose explicit spectral regularization schemes for hierarchical image coding, consisting of (i) intra-scale frequency regularization, which encourages a smooth low‑to‑high frequency buildup as scales increase, and (ii) inter-scale similarity regularization, which suppresses spectral aliasing across scales. Both regularizers are applied only during training and impose no overhead at inference. Extensive experiments demonstrate that our method accelerates the training of the vanilla model by 2.3$\times$, delivers an average 20.65\% rate–distortion gain over the latest VTM-22.0 on public datasets, and outperforms existing single-scale approaches, thereby setting a new state of the art in learned image compression.

Published as a conference paper at ICLR 2026
TAMING HIERARCHICAL IMAGE CODING OPTIMIZA- TION: A SPECTRAL REGULARIZATION PERSPECTIVE
Wuyang Cong, Junqi Shi, Ming Lu", Xu Zhang, Zhan Ma School of Electronic Science and Engineering, Nanjing University (congwuyang) @mmai .nju.ee.. edu. en,
ABSTRACT
Hierarchical coding offers distinct advantages for learned image compression by capturing multi-scale representations support scale-wise modeling and enable flexible quality scalability, making it a promising alternative to single-scale mod- els. However, its practical performance remains limited. Through spectral anal- ysis of training dynamics, we reveal that existing hierarchical image coding ap- proaches suffer from cross-scale energy dispersion and spectral aliasing, resulting in optimization inefficiency and performance bottlenecks. To address this, we propose explicit spectral regularization schemes for hierarchical image coding, consisting of (i) intra-scale frequency regularization, which encourages smooth low-to-high frequency buildup as scales increase, and (ii) inter-scale similarity regularization, which suppresses spectral aliasing across scales. Both regularizers are applied only during training and impose no overhead at inference. Extensive experiments demonstrate that our method accelerates the training of the vanilla model by 2.3x, delivers an average 20.65% rate-distortion gain over the latest VTM-22.0 public datasets, and outperforms existing single-scale approaches, thereby setting new state of the art in learned image compression.
INTRODUCTION
Learned image compression (LIC) (Chen et al., 2021; Lu et al., 2021; He et al., 2022; Duan al., 2023a; Liu et al., 2023; Li et al., 2023; Qin et al., 2024; Fu et al., 2024; Li et al., 2025c; Zeng et al., 2025: Jiang et al., 2025) has recently surpassed traditional hand-crafted codecs (Wallace. 1991; Bellard, 2015; Bross et al., 2021) in compression performance, largely benefiting from statistical learning and end-to-end optimization. Most existing LIC frameworks are built upon single-scale variational autoencoder (VAE) architectures, where the reconstruction relies on a single-scale la- tent representation and auxiliary variables (e.g.. hyperpriors (Ballé et al., 2018)) are primarily used for entropy modeling. Within this paradigm, progress has been driven by powerful transformation networks (Lu et al., 2021; Liu et al., 2023; Qin et al., 2024; Zeng et al., 2025) and advanced con- text modeling (Ballé al., 2018; Minnen et al., 2018; Cheng et al., 2020). While this design has proven highly effective, its performance is approaching saturation, particularly in high-bitrate and high-resolution scenarios.
To overcome the performance plateau of single-scale architectures, recent efforts have turned to hi- erarchical VAE (HVAE) designs (Duan et al., 2023b;a; Lu et al., 2024; Zhang et al., 2025a), which extend the single-scale processing to multiple scales. Such hierarchical representations are in prin- ciple well-suited for compression: they provide multi-scale signal descriptions, enable scale-wise autoregressive modeling, and support flexible quality scalability (Wallace, 1991; Schwarz et al., 2007: Boyce al., 2015). However, their empirical performance has not yet matched these theoreti- cal advantages. For instance, QARV (Duan et al., 2023a), one of the most representative hierarchical schemes, requires nearly 10 days of training on a single NVIDIA RTX 3090 GPU and still under- performs lighter single-scale models such as ELIC (He et al., 2022) in certain bitrate range. These limitations suggest that the potential of hierarchical coding remains far from fully exploited.
We identify that the challenge lies in the naive optimization approach, which overlooks the intended information allocation across scales. In hierarchical architectures, higher scales are expected to
Corresponding authors.

---

Published as conference paper at ICLR 2026
Normalized Radial Freq.
1.0
Scale
Scale
4
Scale 1
Scale
94
High Frequency
Nolse
42
0.4
01
Spectrum Allasing
200
100
200
100
200
100
200
Epoch
Epoch
Epoch
Epoch
(a) Training dynamics w/ naive end-to-end optimization.
Normalizen Meimalizedd Radial Freq.
Scale
Scale 2
Scale
Scale 4
1.0
0.
0.8-
-33
02
Low Frequency Interference
0.-
-00
-43
(c) Illustration of spec- trum problem in (a) (zoomed- in views at scale and 2).
Epoch
Epoch
Epoch
Epoch
(b) Training dynamics w/ proposed spectral regularization.
Figure 1: Spectral energy dynamics during hierarchical compression training, (a) naive train- ing exh-bitibits spectral dispersion, noise, and aliasing issues; (b) regularized training caining-shows shows clear and decoupled coarse-to-fine structure; (c) zoomed-in illustration of these spectral issues. More im- plementation details and plotting scripts can be found in Appendix A.2
capture www-freuency global structures, whereas lower scales should represent high-frequency de- tails (Sønderby et al., 2016; Vahdat & Kautz, 2020). However, naive training fails to respect this information hierarchy: it optimizes all scales over the full frequency spectrum, leading to undesir- able outcomes as illustrated in Fig. 1a. Specifically, (i) the energy of each scale becomes dispersed across frequencies, hampering its compact representation and convergence speed; and (ii) different scales exhibit severe spectral aliasing, resulting in the encoding of redundant frequency components. Both factors contribute to the reduced compression performance observed.
This observation naturally raises question: Can hierarchical models be explicitly trained toward frequency-strattfeed representations to fully exploit their potential? In this context, prior works have suggested a frequency principle (Rahaman et al., 2019; Ronen et al., 2019; Xu & Zhou, 2021; et al., 2025), indicating that different network layers exhibit distinct sensitivities to different frequency bands. In well-trained model, each layer tends to concentrate of frequencies. Motivated by this insight, we analyze the training dynamics of hierarchical models and introduce two plug-an Baug-and-pil and-play regularization strategies:
characteristic subset
1. Intra-scale frequency regularization: progressive spectral truncation scheme that guides each scale to specialize in its target frequency band, enabling a natural low-to-high fre- quency transition.
2. Inter-scale latent regularization: similarity- bbsed penalty in latent space that mitigates spectral aliasing across scales.
These regularizers operate only during training and impose no extra complexity at inference. This design effectively alleviates spectral dispersion and aliasing, leading faster convergence and im- proved compression efficiency, i.e., 2.3x faster convergence and an additional 9,49% rate-distortion improvement over our baseline hierarchical model trained without regularization.
Our main contributions are threefold:
1.
We conduct spectral analysis of training dynamics in hierarchical image coding, reveal- ing frequency dispersion, interference, and aliasing as the primary obstacles that hinder optimization efficiency and compression performance.
2

---

Published as a conference paper at ICLR 2026
2. We propose two lightweight regularization strategies spectral truncation for intra-scale specialization and similarity penalties for inter-scale coordination that effectively miti- gate these spectral issues.
3. We develop a compact hierarchical architecture which, combined with the proposed train- ing scheme, achieves 20.65% bitrate savings over VTM-22.00 surpassing both learned and traditional codecs and establishing a new state of the art in learned image compression.
2 PRELIMINARIES
2.1 SINGLE-SCALE IMAGE CODING
The single-scale VAE-basee image codec can be generally divided into two key components. First, various nonlinear networks such as CNNs (Ballé et al., 2018; Minnen et al., 2018; Cheng et al., 2020; He al., 2022), Transformers (Lu al., 2021; Liu et al., 2023) Mamba (Qin al., 2024; Zeng et al., 2025), are employed to perform encoding and decoding transformations from input image latent y and from decoded to reconstruction x, namely y ga ga(x and x (ý). These networks exploit the spatial and channel-wise correlations of the image, aiming to extract the most compact latent representations. Second, deliberate probabilistic modeling processes are used to estimate the distribution of the latent variables, such as Hyperprior (Ballé et al., 2018) which further extracts abstract representations from y to estimate the probability distribution ý, expressed as 2 (y) and ý h (2), and various context models (Cheng et al., 2020; He et al., 2022; Liu et al., 2023), which enable autoregressively modeling the distribution of latent variables in sequence.
For this single-scale structure, constrained rate-distortion optimization with Lagrangian multiplier À is applied to train in an end-to-end way, i.e.,
Enc
Dec
Lsingle R(y) + R(z) + A D(x, 8),
(I)
X
where R is the bitrate consumption of encoding latent y or hyperprior z (if any), and D is the distortion term between the reconstruction & and the input image x. The final goal of such single-scale codecs is to eliminate redundant information that has less impact the loss function, ultimately extracting the most com- pact single-scale representation, under a preset rate-distortion trade-off. However, recent studies show that training single-scale coding frameworks is hampered by conflicting rate-distortion objec- tives and unstable parameter updates. These problems induce inefficient training and limited perfor- mance. Consequently, recent studies have begun to examine the training dynamics of single-scale image codecs and propose solutions such as gradient modulation (Zhang et al., 2025c), improved optimizers (Li et al., 2025a; Zhang et al., 2025b), and auxiliary training networks (Li al., 2025b).
Figure Diagram of single-scale codecs.
2.2 HIERARCHICAL IMAGE CODING
Hierarchical image codecs (Hu et al., 2020; 2021; Ryder et al., 2022; Duan al., 2023b;a; Lu al., 2024; Zhang et al., 2025a) extend single-scale models into multi-scale framework, in which image is represented through a hierarchy of latent variables at different resolutions. Each seale captures complementary information: higher-scale latents encode abstract, global structures, while lower-scale latents represent fine-graine grainer high- frequency details (Sonderby et al., 2016; Vahdat Kautz, 2020). Dur- ing compression, these latent variables are progressively predicted and entropy-coded. allowing for flexible rate allocation and efficient recon- struction. Formally, an L-scale hierarchy is optimized by minimizing the sum of scale-wise bitrate costs and the final reconstruction distortion, i.c.,
Biases
EncL + + Enc1
Dec7
-
= DeoL
Lrser R(zz) X D).
(2) Figure 3: Diagram of hierarchical codecs.
From theoretical perspective, hierarchical coding naturally aligns with the frequency principle (Ra- haman et al., 2019; Ronen et al., 2019; Xu Zhou, 2021; Xu et al., 2025). Specifically, global im- age content, which is more efficiently captured at higher scales, predominantly lies in low-frequency
3

---

Published as a conference paper at ICLR 2026
components, whereas local details at lower scales correspond to high-frequenn components. This coarse-to-fime decomposition mirrors the frequency-dependee convergence dynamics observed in neural networks: deeper layers tend to model wwwffreueeccy components faster, while shallower lay- ers capture high-frequency components gradually. Hence, hierarchical image coding can be viewed structural embodiment of the frequency principle, with each scale specializing in specific spectral range, enabling efficient multiscale representation and progressive reconstruction.
However, in practice, this ideal frequency-aligned decomposition is not always perfectly realized. Latent variables at different scales may become entangled, and competing gradients during opti- mization can lead to overlapping spectral representations across scales. As a result, some scales may partially encode information outside their intended frequency range, and the hierarchical allo- cation of coarse-to-fine information may be disrupted. Understanding this deviation is crucial for analyzing hierarchical training dynamics and explaining why naive optimization sometimes fails to fully exploit the theoretical advantages of multiscale latent structures.
METHODOLOGY
3.1 SPECTRAL ANALYSIS OF HIBRARCHICAL TRAINING DYNAMICS
To investigate hierarchical training dynamics through spectral analysis, quantify the scale-wise contributions to the final reconstruction (see implementation details in Appendix A.2) and compute their spectral overlap with the input image. Tracking the evolution of this overlap across training epochs with heatmap (Fig. 1) reveals a two-stage pattern:
Different scales converge their respective frequency bands at different rates. Higher scales are more sensitive to low-frequency content and converge faster, whereas lower scales focus on high-frequency details and converge more slowly.
Early Stage:
Later Stage: As training progresses, each scale stabilizes within certain spectral range. At this point, the scales separate their spectra from one another, forming decoupled low-to-high frequency distribution (Maaløe et al., 2019; Vahdat & Kautz, 2020).
Overall, the training process broadly follows the frequency principle. However, several localized issues still disrupt this progression. As illustrated in Fig. 1a and Ic:
Intra-scale interference: Spectral components become entangled with high-frequency noise and low-frequent wwwffreuency interference. Worse still, these artifacts propagate through the hierarchy, causing the spectrum of the last scale to exhibit severe dispersion.
Inter-scale aliasing: Overlapping frequency bands persist across scales-for for example, the second scale contains an abnormal low-frequency band that overlaps with the first, while the last seale almost entirely covers the third.
We argue that these violations the frequency principle lead to ill-structured information hierar- chies, resulting in training instability and limited performance (see Fig. 7 and 8). This observation naturally motivates the question: Can we design explicit guidance based on the frequency principle promote spectral convergence and decoupling, thereby mitigating interference and aliasing?
To address this, we propose an intra-scale regularization in the carly training (c.g., the first 100 epochs) to stabilize scale-wise frreueecyy band convergence. Then switch an inter-scale regu- larization in later stages to mitigate spectral aliasing, more details are described below.
3.2 INTRA-SCALE FREQUENCY REGULARIZATION
Input Preprocessing
Hierarchical Architecture
To ensure that each scale converges quickly and accurately to its assigned frequency band in early training without mixing in abnormal frequency components, we design Discrete Cosine Transform (DCT) based spectral truncation scheme. Specifi- cally, at first, only the low-frequency components of the input fed into the entire model, and then higher frequencies can gradually be added. This allows the topmost scale (with largest receptive field) 21 rapidly and fully capture wwwfrruuency in-
= 0.05
Scale
High
Low
2D-DCT
0.10
fruq
Low
Freq Truncation
2D-ICT
0.35
High
Ads
Freq
Figure 4: Pipeline of proposed intra-scale regularization.
4

---

Published as a conference paper at ICLR 2026
formation, avoiding delegating fow-ereqyyncy responsibility to subsequent scales. Later scales can then focus on their assigned high-freuency contents, avoiding interference of nigh-frequency noise across scales. The overall pipeline is illustrated in Fig. 4.
In implementation, the training data x E I BB is first transformed into frequency domain F E BBCXHXx through 2D-DCT as equation 3, where B, C, H, W are the batch size, channel numbers, height and width, respectively.
(PH)u, = (2) cos
m(2z+1ja
V K
0, k
2H
F PHXPW: where
ak(k)-
(3)
(Pw)yy = aw (v) cos
x(2y
k
1,
2W
where PH E "I and Pw E RWxW are the orthonormal bases along the vertical and horizontal dimensions of DCT, respectively. u E [0, H 1] and [0, W 1] are the frequency indices, and the normalization term ox (k) is employed to guarantee orthogonality, with K (H, W).
Then, tire-varying soft radial mask M(u, v; E [0, 1] used for spectral truncation, defined as:
M(U, 12 = max (o, =(t) / 77 + W
(4)
(t))
where the term V(u/H) + (v/W)² represents the normalized frequency radius, and r(t) scheduling function of epochs that controls the cutoff radius, it typically increases from small initial value (e.g., r(0) = 0.05 in our scheme) to in a linear manner during training. Then the original spectrum F can be truncated to F = F MM,,
Finally, the truncated spectrum F will be transformed back to the pixel domain via 2D-IDCT, yield- ing images that retain only subset of frequency components x for model training. More details about the implementation of 2D-DCT can be found in Appendix A.2.
PH F Pw.
(5)
This low-to-high frequency learning strategy enables an incremental optimization from high low scales: high scales fully encode the low-frequee now-frequency information without leaking it to subsequent scales, while high-frequency information is gradually incorporated in lower scales the basis of low-frequency representations. Such a schedule accounts for the varying convergence rates and sen- sitivities different frequency bands across scales, dictated by the frequency principle, thereby avoiding abnormal frequency within each scale.
3.3 INTER-SCALE LATENT REGULARIZATION
Then, as training processes, each scale's approximate spectral range is largely fixed. At this stage, our goal is to prevent spec- tral overlap between scales and to organize inter-scale informa- tion effectively. To achieve this, we introduce regularization based on latent variables' similarity across adjacent scales, which encourages the features of neighboring scales to remain as dis- tant as possible. This ensures that subsequent scales allocate bi- trate only to frequency components not represented by preceding scales, thereby saving bitrate. The pipeline is illustrated in Fig. 5.
Maximize L2 Dist.
Z1
Zz
Freq, Alignment
Figure 5: Pipeline of proposed inter-scale regularization.
In practice, we insert convolutional downsampling module based on the Discrete Wavelet Trans- form (DWT) between consecutive latent variables only during training (disabled when inference). The lower-scale latent variable z₁ is downsampled via DWT, which decomposes it into frequency sub-ban bab-bands. We then apply a Ixl convolution across channels to linearly map and recompose fre- quency sub-bands so they align with the frequency channels of higher scale latent Z)-1- The feature regularization distances are measured by L2 loss and weighted into the loss for end-to-end optimiza- tion. For the topmost scale, the latent variable will be compared the initial learnable bias prior obtained from training. In this way, the model's 's training loss in equation 2 is re- re-formulated as:
Lhier-regu
R(zz) + X D(x, x) - 8 1=1
La L2(2:- C
(6)
-=1
5

---

Published as a conference paper at ICLR 2026
where 8 is weight parameter, it is fixed to 0.1 in our implementation. More details such as the implementation of DWT and different choices of 6, can be found in Appendix A.2 and A.4.
By integrating such inter-scale regularization, inter-scale penalty mechanism is established to discourage the aligned lower latent from predicting the same low- cow-frequnncy content in the higher latent. Hence, the model can carefully avoid spending bitrate to encode redundant overlapping spectral information. This guides the model to explore a more decoupled and efficient information distribution across scales, thereby saving overall bitrate cost.
EXPERIMENTS
4.1
SETUP
Base Model Design: thoroughly investigate the training dynamics of hierarchical coding, we design lightweight 4-scale hierarchical image codec, DHIC' (Deep Hierarchical Image Coding). Unlike prior approaches (Duan et al., 2023b;a; et al., 2024; Zhang et al., 2025a), our base model, a.k.a. DHIC-Base, adopts only single latent block per scale and replaces heavy backbones (e.g.. Transformer or Mamba (Gu& Dao, 2023)) with simple CNNs. This design eliminates performance confounds introduced by complex architectures, allowing to focus purely on the effect of training with and without regularization as mentioned above. We refer to the DHIC model trained with the proposed regularization schemes the DegC-Regu. Network details provided in Appendix A.2.
Training Settings: We train our models, e.g., DHIC- Base, and DHIC-Regu, on the mixed dataset in- troduced in Jiang al. (2025), which comprises images selected from Flickr20K (Lim al., 2017) ()17,, DIV2K (Agustsson & Timofte, 2017), COCO2017 (Lin et al., 2014), and ImageNet (Deng et al., 2009). During pretraining, images are randomly cropped into 256 x 256 patches with a batch size of 32, while finetuning is performed on 512 x 512 crops with a batch size of 4. The training procedure conducted on single NVIDIA RTX 4090 GPU using PyTorch and the Adam optimizer. The learning rate is initialized at 1e-4 and gradually reduced to 1e-5 via a Reduce LROnP eau scheduler during pretraining, and further decreased to 1e- the finetuning stage. In addition, our codec supports variable bitrate, with the Lagrangian factor À ranging from 64 to 4096.
Test Settings: We conduct tests on three widely used test datasets: the Kodak dataset (Eastman Kodak Company, 1993), which consists of 24 images with a resolution of 512x (or x68x512) the CLIC professional Valid dataset (Toderici et al., 2020), which contains 41 high-quality images of varying resolutions; and the Tecnick dataset (Asuni al., 2014), which contains 100 images at a resolution of x200 1200. For rate distoritooioon evaluation, we report bitrate in BPP and distortion using either PSNR or MS-SSIM (Wang et al., 2004). In addition, we adopt the Bjøntegaard Delta Rate (BD-Rate) (Bjontegaard, 2001) to measure gains to the anchor codec. Tests using 4K or 1080p images provided in the Appendix A.4.
We also report the model's 's parameter size (M), computational complexity in terms of KMACs/pixel (kilo multiply-accmmultee operations per pixel), and the encoding/decoding time (ms) to assess the complexity of the codecs. All evaluations are performed on a platform equipped with a single NVIDIA RTX 3090 GPU and an Intel Xeon Gold 6430 CPU.
4.2 EVALUATION RESULTS
Compression Performance:
We comprehensively evaluate the proposed DHIC-Regu, DHIC-Base,
a series of state-of-the-art single-scale learned codecs (He et al., 2022; Liu al., 2023: Li et al., 2023: Qin et al., 2024; Fu et al., 2024; Li et al., 2025c; Zeng et al., 2025; Jiang et al., 2025), and the representative hierarchical image codec QARV (Duan al., 2023a). As summarized in Table 1, and using VTM-22.0 as the anchor baseline, DHIC-Regu achieves the best BD-Rate performance across all three datasets, i.c., -19.73% -18. -18.13% and -24.09%. Notably, our method demonstrates even greater advantages on high-resolution images compared to the latest single-scale codec HPCM- Large (Li et al., 2025c), as detailed in Appendix A.4. Moreover, compared to DHIC-Base without regularization, DHIC-Regu achieves additional 9.49% bitrate reduction over VTM-22.0, with- out introducing any extra testing complexity. This clearly demonstrates the effectiveness of our regularization design. We also validate its effectiveness on QARV, as reported in Appendix A.4.

---

Published as a conference paper at ICLR 2026
Table Compression performance and complexity comparison of learned image codecs across multiple datasets (Anchor: VTM-22.0).
Enc. Time
Dec. Time
KMACs
Params
Kodak
CLIC Pro
Tecnick
Avg.
Model
(ms)
(ms)
(/pixel)
(M)
(%)
(%)
(%)
(%)
ELIC (C)PR'22)
43.78
48.15
573.88
36.93
-3.22
-3,89
4.57
-3.89
TCM-Large (CVPR 23)
153.57
142.83
1823.58
76.57
-9.97
-9.65
-13.24
-10,95
MIJC++ (NCW ICMI''23)
191.46
186.08
1282.81
116.72
-11.83
-12.18
-17.25
-13.75
FLJC (ICLR'24)
> 1000
1000
1096.04
70.97
-12.97
-10.53
-15.82
-13.11
WeConvene (ECCV'24)
333.45
227.41
2343.13
107.15
-6.98
-8.54
-10.81
-8.78
Ma (Arxiv'24)
137.88
124.95
813.80
53.32
-8.72
-5.66
-8.63
-7.67
MambalC (CVPR'25)
156.82
113.07
1284.86
75.78
-15.12
-9.98
-13.65
-12.92
HPCM-Large (ICCV*25)
117.40
112.27
1261.29
89.71
-19.19
-18.37
22.20
-19.92
QARV (TPAMI'24
158.42
71.61
718.96
93.4
-5.81
-6.91
-8.88
-7.20
DHIC-Base (Ours)
102.46
68.48
977.73
106.93
-9.62
-10.79
-3.06
-11.16
DHIC-Regu (Ours)
102.46
68.48
977.73
106.93
-19.73
-18.13
-24.09
20.65
Complexity: We first examine the training complexity. With the proposed regularization, DHIC-
Regu converges within approximately 3.8 days, which is highly competitive compared to both single-scale codecs (e.g., TCM (Liu et al., 2023)) and hierarchical codecs (c.g., QARV), each of which typically requires nearly 10 days to complete training.
For testing complexity, we compute the time overhead, computational cost, and parameter counts for each codec, as shown Table 1. For faimesss these metries are re-evaluated on the same platform with open-source implementations whenever available, while reported data from original papers are adopted otherwise. As observed, our hierarchical codec achieves lower complexity than the recent performance-leading methods, such as MLIC+ (Jiang et al., 2025) and HPCM-Large (Li et al., 2025c), while also offering superior performance. This benefits from both our lightweight network design and training-only regularizers. Furthermore, the hierarchical codec naturally provides more efficient context both spatial and frequency domains, thereby reducing the overhead of complex autoregressive contexts and achieving the fastest decoding speed similar complexity.
40
120
160
200
240
280
40
80
120
180
200
240
280
S1
S1
53
S3
(a) The input.
(b) Vanilla naive training.
(c) Regularized training.
Figure 6: Visualization of scale-wise latent variables during two different training processes.
The source image is Kodak002. (Eastman Kodak Company, 1993). The horizontal axis denotes the training epochs, while the vertical axis corresponds to the hierarchical scales.
Qualitative Results: To intuitively validate the effectiveness of our proposed regularization, we visualize the hierarchical information structure under naive and regularized training via scale-wise latent variables (Fig. 6). The results show that: (a) Under naive training, latent variables across scales remain entangled and indistinct. Higher scales (e.g., S1) fail capture global semantics, while lower scales (e.g., S4) cannot effectively represent high-frequency structures, instead showing scattered, noisy patterns. The model also exhibits grid-like artifacts and central blur, and structured features emerge only after the 120th epoch, without achieving clear scale decoupling. (b) In con- trast, with regularized training, distinct latent variables emerge as early as the 40th epoch. These representations progressively refined and naturally decoupled, resulting in clear coarse-to-fine information hierarchy.
Overall, the visualizations confirm that the proposed regularization mitigates spectral issues such as dispersion, noise, and aliasing, thereby promoting the emergence of well-structured multiscale representations and fully unlocking the capacity of hierarchical coding architectures.

---

Published as a conference paper at ICLR 2026
4.3 DEEP DIVE
Intra-Scale Training Dynamics: According to the frequency principle, models tend to learn low-frequency information first during training, as it reduces loss more effectively at lower bi- trate cost. However, in the absence of explicit low-frequency constraints, each scale attempts capture information across the full frequency spectrum in a naive manner. This mixing of frequencies prevents scales from specializing in their most relevant bands. For example, higher scales often attempt to en- code high-frequency details that they are ill-suited for, leading to misrepresentation that propagates backward and ultimately Figure 7: Loss vs. training epochs.
Ohnesthol Treining
Loss
0.5
Epochs
degrades both training stability and reconstruction quality. To illustrate this effect, visualize the training loss curve in Fig. 7, which shows large fluctuations without substantial loss reduction during the first 100 epochs. By contrast, when explicit low-to-high frequency guidance is introduced, each scale quickly concentrates on its assigned frequency range, resulting in faster convergence (Fig. lb) and effectively mitigating the instability and inefficiency in naive training.
Scale
Scale
Scale
Scale
0.106 (4)
0.18 (b)
0.50 (0)
0.525 (d)
+0.033
Scale-Wise BPP
0.17
3,010
0.45
0,500
0,100
0.16
0.40
0.475
0.15
0.35
0.080
50
100
150
200
250
0.14
50
100
150
200
250
100
150
200
250
0.450
0
50
100
150
200
250
Epoch
Epoch
Epoch
Epoch
(a) Scale-wise bitrate (measured with Bits Per Pixel (BPP)) variation during training.
Scale 1
Scale 2
Scale 3
Scale
Scale- WWie Distortion (MSE)
0.0020
0.0014
0.0038
O. 0.00050
0.0018
0.0013
0.0012
0.0007
0.00045
0.0016
0.0011
0.00040
0.0010
0.00035
0.0012
0.0009
0.0009
0.0006
0.00030
0.0007
0.0034
0.00025
0.0008
0.0010
2220-04
0.0005
0.0006
20m
0.0008
p 00020
0.00015
0.0005
0
100
150
0.0003
100
150
50
100
0.00010
100
150
Epoch
Epoch
Epoch
Epoch
(b) Scale-wise distortion (measured with Mean Squared Error (MSE)) variation during training.
Figure 8: Scale-wise rate-distortion change during training. The light-colored curves represent the naive training, while the dark- colored curves show the regularized one. Differences in layer-wise bitrate and distortion are marked with dashed lines and text annotations in each subplot.
Inter-Scale Training Dynamics: Due to the inherent dependencies across scales in hierarchical architectures, latent variables at higher scales are conditioned on those from earlier ones, leading mutual interference in scale-wise rate-distortion optimization. As a result, naive optimization following equation 2 struggles to establish an efficient hierarchical information structure. From a spectral perspective, this manifests as spectral aliasing (Fig. 1), where multiple scales redundantly encode overlapping frequency components. Such redundancy wastes additional bitrate and ulti- mately limits the achievable rate rate- distortion performance.
To further examine the effect on training dynamics, we visualize the scale-wise rate and distortion curves in Fig. 8. The results reveal that, during training, the model persistently reallocates bitrate across scales, preventing the scale-wise bitrate from steadily increasing or stabilizing. Instead, it undergoes frequent fluctuations and abrupt shifts. In parallel, scale-wise distortion also exhibits pronounced oscillations. These instabilities intensify at deeper scales, ultimately causing excessive bitrate consumption and higher distortion. Such behavior validates that inter-scale spectral aliasing severely disrupts training and degrades performance. In contrast, when inter-scale regularization is
8

---

Published as a conference paper at ICLR 2026
introduced, this aliasing is largely mitigated (see Fig. 1b), resulting in markedly improved training stability and superior iate-distortion performance.
4.4 ABLATION STUDIES
The separate effects of two regularization methods: Although both of the proposed regularization methods aim to address the issues observed in spectral analysis, their effects differ. Table 2 presents the individual impact of each method on accelerating training and improving performance. It can be observed that applying intra-scale regularization solely in the early stages of training primarily accelerates model training but does not significantly improve the rate-distortion performance after full training. On the other hand, applying inter-scale regularization solely in the subsequent stages will somewhat slow down convergence but provide a substantial performance boost. Furthermore, essentially designed to
both of these regularizers guide hierarchical optimization to better follow the fre- quency principles,
Table 2:
Ablation study of the sepa- rate effects of two regularizers (Base- line: the naive optimized model).
they complement each other to some extent. When both regularization methods are com- bined, the model achieves synergistic effect, resulting in faster convergence and improved final performance.
Regularization Acceleration BD-Rate (%)
Intra-Scale Inter-Scale
1.84x
-1.07
In addition, we also examine the separate effects of the two regularizers on QARV (Duan et al., 2023a) and obtain similar conclusions. The detailed results are provided in Appendix A.4.
0.91
-7.66
Both
2.30x
-10.11
The efffec of different regularization implementation settings: We conduct ablation studies to compare the effects of different implementations of two regularization methods. Specifically:
The scheduler during the early-stage DCT truncation: In our implementation, the high- frequency components are linearly increased from 0.05 to 1.0 during the first 100 epochs. We further explore alternative scheduling strategies, including different initialization values and exponential growth modes, reported in Table 3a. Empirically, the proposed linear schedule yields the best trade-off' between training efficiency and final performance.
2. Inter-scale latent regularization during subsequent training: our approach, latent variables from lower scales downsampled using convolutional layer combined with wavelet transformation, followed by L2 alignment with the preceding scale. We also in- vestigate alternative strategies, including standard strided convolution. downsampling fol- lowed by convolution, and feature alignment using L1 loss or cosine similarity, as sum- marized in Table 3b. Empirically, the proposed scheme achieves the best performance. Theoretical analysis of such superiority can be found in Appendix A.4.
Table 3: Ablation studies of intra-scale and inter-scale regularization implementations (Base- line: the naive trained model, best implementation approaches are marked blue color).
(a) Intra-scale regularization (First 100 epochs).
(b) Inter-scale regularization (Remaining epochs).
Implementation
Acceleration BD-Rate
Implementation Acceleration BD-Rate (%)
0.025 (linear)
1.62x
-1.01
Conv w/ Stride
0.87x
-5.49
0.05->1.0 (linear)
1.84x
-1.07
Down Conv
0.90x
-5.22
0.1-+1.0 (lincar)
1.77x
-1.05
DWT Conv
0.91
-7.66
0.025 1.0 (cxp)
1.49x
-0.93
L1
0.82x
-7.07
0.05+11.0 (exp)
1.60x
-0.98
L2
0.91
-7.66
0.1-+1.0 (exp)
1.53x
-1.02
Cos Similarity
0.85x
-6.55
More ablation studies on regularization setups and modules design, are detailed in Appendix A.4.
5 CONCLUSION
In this work, we tackle the fundamental optimization challenges inherent in hierarchical image com- pression by diagnosing and countering spectral dispersion and aliasing phenomena that arise during

---

Published as a conference paper at ICLR 2026
training. Our spectral analysis reveals that these issues lead to inefficient training and degraded rate-distortion performance. To address these issues, we introduce two complementary regulariza- tion strategies: a low-to-high frequency-awain aware truncation mechanism that guides each intra-scale toward its intended spectral band, and an inter-scale similarity constraint that prevents the encoding of redundant frequency information across scales. These training-only techniques yield model that converges 2.3x faster and achieves an additional 9. 9.49% improvement in rate-distortion performance over VTM-22.0, culminating in an average 20.65% bitrate savings over VTM-22.0 D-ettinng new state-of-the-art in learned image compression. Our approach not only demonstrates the untapped po- tential of hierarchical architectures but also provides a principled spectral perspective for optimizing multi-scale latent models, opening avenues for future research in efficient neural compression.
ETHICS STATEMENT
This work focuses on hierarchical image compression with the goal of improving compression efficiency from spectral regularization perspective. All experiments performed using pub- liely available datasets-innludiig MLIC-Train-I00H UVG, CLIC professional Valid, Tecnick, and M1U4K-v2-whie consist of non-sensitive and openly accessible data. We acknowledge that com- pression algorithms inevitably introduce distortions, which could affect the reliability of image in- terpretation safety-critical applications such as medical imaging or autonomous systems. We therefore emphasize the importance of responsible deployment of the proposed method, particularly stah-stakes domains where inaccuracies may carry significant consequences.
REPRODUCIBILITY STATEMENT
To support reproducibility, include detailed descriptions of the model architectures, training configurations, and evaluation protocols. All experiments were conducted under controlled settings with fixed random seeds and hardware platform. Furthermore, complete source code publicly upon acceptance the paper to facilitate replication and encourage further research in this direction.
committed releasing the
ACKNOWLEDDGEMENT
This work was supported in part by Natural Science Foundation of China (Grant No. 62431011, 62401251) and Natural Science Foundation of Jiangsu Province (Grant No. BK20243038). The authors would like to express their sincere gratitude to the Interdisciplinary Research Center for Future Intelligent Chips (Chip-X) and Yachen Foundation for their invaluable support.
REFERENCES
Eirikur Agustsson and Radu Timofte. Ntire 2017 challenge on single image super-resolution: Dataset and study. In Proceedings of the IEEE conference on computer vision and pattern recog- nition workshops, pp. 126- 135, 2017.
Nicola Asuni, Andrea Giachetti, al. Testimages: a large-scale archive for testing visual devices and basic image processing algorithms. In STAG, pp. 63-70, 2014.
Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. Variational image compression with scale hyperprior. In 6th International Conference on Learning Rep- resentations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018.
Fabrice Bellard. Bpg image format. htttps:/bbelarrd.org 12.
2015. Accessed: 2025-09-
Gisle Bjontegaard. Calculation average psnr differences between rd-curves. ITU-T SG16, Doc. VCEG-M33, 2001.
10

---

Published as a conference paper at ICLR 2026
Jill M Boyce, Yan Ye, Jianle Chen, and Adarsh K Ramasubramonian. Overview of shve: Scalable extensions of the high efficiency video coding standard. IEEE Transactions on Circuits and Systems for Video Technology, 26(1):20-344 2015.
Benjamin Bross, Yu-Kui Wang. Yan Ye, Shan Liu, Jianle Chen, Gary Sullivan, and Cens-Rainer Ohm. Overview of the versatile video coding (vve) standard and its applications. IEEE Transac- lions on Circuits and Systems for Video Technology, (110):3336-3774, 2021.
Yuan Cao, Zhiying Fang, Yue Wu, Ding Xuan Zhou, and Quanquan Gu. Towards understanding the spectral bias of deep learning. International Joint Conferences Artificial Intelligence Organi- zation, 2021.
Tong Chen, Haojie Liu, Zhan Ma, Qiu Shen, Xun Cao, and Yao Wang. End-to-end learnt image compression via non-local attention optimization and improved context modeling. IEEE Trans- actions on Image Processing, 50:3179-3191 2021.
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, and Jiro Katto. Learned image compression with discretized gaussian mixture likelihoods and attention modules. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248-255, leee, 2009.
Zhihao Duan, Ming Lu, Jack Ma, Yuning Huang, Zhan Ma, and Fengqing Zhu. Qarv: Quantization- aware resnet vae for lossy image compression. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 46(1):436-450, 2023a.
Zhihao Duan, Ming Lu, Zhan Ma, and Fengqing Zhu. Lossy image compression with quantized hierarchical vaes. In Proceedings of the IEEE/CVF winter conference on applications computer vision, pp. 198-207, 2023b.
Eastman Kodak Company. Kodak lossless true color image suite. http::::///...uu/graaphhti kodak/, 1993.
Haisheng Fu, Jie Liang, Zhenman Fang, Jingning Han, Feng Liang, and Guohe Zhang. Weconvene: Learned image compression with wavelet-domain convolution and entropy model. European Conference on Computer Vision, pp. 37-53. Springer, 2024.
Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752,, 2023.
Dailan He, Ziming Yang. Weikun Peng, Rui Ma, Hongwei Qin, and Yan Wang. Elie: Efficient learned image compression with unevenly grouped space-channel contextual adaptive coding. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 5718- 5727, 2022.
Yueyu Hu, Wenhan Yang, and Jiaying Liu. Coarse-to-fine hyper-prior modeling for learned image compression. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 11013-11020, 2020.
Yueyu Hu, Wenhan Yang. Zhan Ma, and Jiaying Liu. Learning end-to-end lossy image compression: A benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(8):4194- 4211, 2021.
Wei Jiang, Jiayu Yang. Yongqi Zhai, Feng Gao, and Ronggang Wang. Mlic++: Linear complexity multi-reference entropy modeling for learned image compression. ACM Transactions on Multi- media Computing, Communications and Applications, 21(5):1-25, 2025.
Chao Li, Chuanmin Jia, Fanyang Meng, Hao Qi, Siwei Ma. and Yongsheng Liang. Dmso: A dynamic momentum- momentum-smoothing smoothing optimizer for learned image compression. In 2025 IEEE In- ternational Conference on Image Processing (ICIP), pp. 307-312, 2025a. doi: 100 110.109/ TCIP55913, 2025. 11084556.
11

---

Published as a conference paper at ICLR 2026
Han Li, Shaohui Li, Wenrui Dai, Chenglin Li, Junni Zou, and Hongkai Xiong. Frequency- aware transformer for learned image compression. arXiv preprint 1xX1123300.13887, 2023.
Han Li, Shaohui Li, Wenrui Dai, Maida Cao, Nuowen Kan, Chenglin Li, Junni Zou, and Hongkai Xiong. On disentangled training for nonlinear transform learned image compression. arXiv preprint arXiv:2501.1375// 2025b.
Yuqi Li, Haotian Zhang. Li Li, and Dong Liu. Learned image compression with hierarchical pro- gressive context modeling. arXiv preprint 1rXiv:2507.19125, 2025c.
Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. Enhanced deep resid- ual networks for single image super- resoutrcs...iion. In Proceedings of the IEEE conference com- puter vision and pattern recognition workshops, pp. 136-144, 2017.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Jiaying Liu, Dong Liu, Wenhan Yang, Sifeng Xia, Xiaoshuai Zhang, and Yuanying Dai. compre- hensive benchmark for single image compression artifact reduction. IEEE Transactions image processing, 29:7845-7860, 2020.
Jinming Liu, Heming Sun, and Jiro Katto. Learned image compression with mixed transformer- cnn architectures. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14388-14397 2023.
Zhen Liu, Hao Zhu, Qi Zhang, Jingde Fu, Weibing Deng. Zhan Ma, Yanwen Guo, and Xun Cao. Finer: Flexible spectral-bias tuning in implicit neural representation by variable-pericn pereeeeerooicc activa- tion functions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2713 27222, 2024.
Ming Lu, Peiyao Guo, Huiqing Shi, Chuntong Cao, and Zhan Ma. Transformer-baser image com- pression. arXiv preprint arXiv:2111.067000 2021.
Ming Lu, Zhihao Duan, Fengqing Zhu, and Zhan Ma. Deep hierarchical video compression. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 8859-8867, 2024.
Lars Maaløe, Marco Fraccaro, Valentin Liévin, and Ole Winther. Biva: very deep hierarchy of latent variables for generative modeling, Advances in neural information processing systems, 32, 2019.
David Minnen, Johannes Ballé, and George Toderici. Joint autoregressive and hierarchical priors for learned image compression. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kris- ten Grauman, Nicolò Cesa-Bianchi, and Roman Gamett (eds.), Advances in Neural Information Processing Systems 31: Anmual Conference on Neural Information Processing Systems 2018, NeurlPS 2018, 3-8 December 2018, Montréal, Canada, pp. 10794- 2018.
Shiyu Qin, Jinpeng Wang, Yimin Zhou, Bin Chen, Tianci Luo, Baoyi An, Tao Dai, Shutao Xia, and Yaowei Wang. Mambave: Learned visual compression with selective state spaces. arXiv preprint arXiv:2405. 15413, 2024.
Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville, On the spectral bias of neural networks. In International conference on machine learning, pp. 5301-5310. PMLR, 2019.
Basri Ronen, David Jacobs, Yoni Kasten, and Shira Kritchman. The convergence rate neural net- works for learned functions of different frequencies. Advances in Neural Information Processing Systems, 32, 2019.
Tom Ryder, Chen Zhang, Ning Kang, and Shifeng Zhang. Split hierarchical variational compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 386-395, 2022.
12

---

Published as a conference paper at ICLR 2026
Heiko Schwarz, Detlev Marpe, and Thomas Wiegand. Overview of the scalable video coding exten- sion of the h. 264/ave standard. IEEE Transactions on circuits and systems for video technology, (799:1103-11200 2007.
Junqi Shi, Mingyi Jiang, Ming Lu, Tong Chen, Xun Cao, and Zhan Ma. Hiner: Neural represen- tation for hyperspectral image. In Proceedings of the 32nd ACM International Conference on
Multimedia, pp. 9837-9846, 2024.
Junqi Shi, Qirui Zhang, Ming Lu, and Zhan Ma. Compression as restoration: A unified implicit approach self-supervised hyperspectral image representation. IEEE Journal of Selected Topics in Signal Processing, pp. 1-15, 2025. doi: -
2025. 3632533
Casper Kaae Sønderby, Tapani Raiko, Lars Maaløe, Søren Kaae Sønderby, and Ole Winther. Ladder variational autoencoders. Advances in neural information processing systems, 29, 2016.
Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, and Ren Ng. Fourier features let net- works learn high frequency functions in low dimensional domains. 2020.
George Toderici, Wenzhe Shi, Radu Timofte, Lucas Theis, Johannes Balle, Eirikur Agustsson, Nick Johnston, and Fabian Mentzer. Workshop and challenge learned image compression (clic2020). In CVPR, 2020.
Arash Vahdat and Jan Kautz. Nvae: A deep hierarchical variational autoencoder. Advances in neural information processing systems, 33:19667-19679, 2020.
Gregory K Wallace. The jpeg still picture compression standard. Communications of the ACM, 34 (4):30-44, 1991.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600- 612, 2004.
Shaowen Xie, Hao Zhu, Zhen Liu, Qi Zhang, You Zhou, Xun Cao, and Zhan Ma. Diner: Disorder- invariant implicit neural representation. In Proceedings of the IEEE/CVF Conference on Com- puler Vision and Pattern Recognition, pp. 6143-6152, 2023.
Zhi-Qin John Xu, Yaoyu Zhang, and Yanyang Xiao. Training behavior of deep neural network in frequency domain. In International Conference on Neural Information Processing, pp. 264-274. Springer, 2019.
Zhi-Qin John Xu, Yaoyu Zhang, and Tao Luo. Overview frequency principle/spectral bias in deep learning. Communications Applied Mathematics and Computation, 7(3):827-864 2025.
Zhiqin John Xu and Hanxu Zhou. Deep frequency principle towards understanding why deeper learning is faster. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 10541-10550, 2021.
Fanhu Zeng, Hao Tang, Yihua Shao, Siyu Chen, Ling Shao, and Yan Wang. Mambaic: State space models for terfhrperfor learned image compression. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 18041-18050, 2025.
Guangyi Zhang, Hanlei Li, Yunlong Cai, Qiyu Hu, Guanding Yu, and Runmin Zhang. Learned image transmission with hierarchical variational autoencoder. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 39, pp. 13215-13223, 2025a.
Yichi Zhang, Zhihao Duan, Yuning Huang, and Fengqing Zhu. Accelerating learned image com- pression through modeling neural training dynamics. Transactions on machine learning research, 2025:4249, 2025b.
Yichi Zhang, Zhihao Duan, Yuning Huang, and Fengqing Zhu. Balanced rate-distortion optimization in learned image compression. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 2428 2428-2438, 2025c.
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 PRELIMINARY OF FREQUENCY PRINCIPLE
In this section, we supplement more prior works on spectrum bias, frequency principle and hierar- chical architecture related to spectrum analysis and training dynamics to further help understand the theoretical basis of our work.
Frequency principle, also referred to as the spectral bias theory, denotes the empirically observed tendency of deep neural networks to learn low-frequency components of target function earlier and faster than high-froe freuurry components. This phenomenon was first systematically identified by (Rahaman et al., 2019), which demonstrated -ia Fourier-domain analyses synthetic sinusoidal targets and examinations of manifold geometry that low-frequency components not only converge more quickly but are also more robust to parameter perturbations, thereby providing empirical account of networks' implicit smoothing behavior. Subsequent works formalized and generalized this observation, including Xu et al. (2019); Xu & Zhou (2021); Xu al. (2025) proposed that the same cow-to-high frequency learning order appears across multiple architectures and standard vision benchmarks, arguing that the smoothness properties of common activations help explain the bias and its implications for generalization. From theoretical perspective, the Neural Tangent Kernel (NTK) regime (Cao et al., 2021) proves that, considering the infinite-width linearization of training, training dynamics admit spectral decomposition in which the kernel's eigenvalues determine convergence rates: directions associated with large eigenvalues (often low-frequency components) are learned rapidly, whereas directions associated with small eigenvalues (often high frequencies) are learned slowly.
Motivated by these insights, subsequent works began to regular the training process of deep neural network. For instance, Tancik et al. (2020) demonstrated that mapping low-dimensional inputs into a higher-dimensional sinusoidal feature space yields an effective kemel with controllable bandwidth, substantially improving MLPs' ability to fit high-frequency functions and underpinning the success of positional encodings in implicit-repreeentation systems (Xie al., 2023; Liu et al., 2024; Shi et al., 2024; 2025).
Our spectrum analysis and regularization methods also based on the above strong conclu- sion bion-Different scales converge to different frequency bands at different rates, which has been widely proven in prior works, specifically including:
1. Studies on neural network spectral bias or spectral principles Rahaman et al. (2019); Xu et al. (2019); Xu & Zhou (2021); Xu al. (2025) have shown that networks exhibit preference for low-frequency functions during training. Low frequencies are learned earlier and faster than high frequencies in deeper module of the network. Mapping this finding to our network structure can explain why different scales exhibit different convergence speeds for different frequency bands.
2. Existing HVAE Sønderby et al. (2016); Vahdat & Kautz (2020) and hierarchical coding works Duan et al. (2023b;a) both indicate that different hierarchical scales tend to capture information at different frequencies: low-resolution scales capture more abstract global in- formation (typically low-frequency structures), whereas high-resolution scales handle local details and residual information (typically high figh-frequeny).
Furthermore, our existing experiments can also support the conclusion. For instance, Figure I shows that scale has essentially converged to its corresponding spectral position (and no longer increases) at around epoch 30; subsequently, scale 2 converges at around epoch 50, scale 3 at around epoch 80, and scale 4 at around epoch 100. The original training loss curves in Fig. 8 also roughly follow this trend, which validates the above claim.
Taken together, these theoretical analysis and visualizations demonstrate, in our hierarchical struc- ture. different scales will converge different frequency bands at different rates: deeper, lower- resolution scales converge faster to low-frequens regions, and shallower, higher-resolution scales then gradually converge their corresponding high-frequency positions that basis.
14

---

Published as a conference paper at ICLR 2026
A.2 MORE IMPLEMENTATION DETAILS
In this section, we provide additional implementation details of our methods and experiments, in- cluding network design, benchmark implementation, and testing commands. The details are pre- sented as follows.
Network Design: Fig. 9 shows the Architecture of our hierarchical image codec used in this work. It consists of four scales (L = 4). specifically including bottom-up encoding pathway and a top-down entropy model, plus decoding pathway. Given an input x, we first apply Patchi fy operations to reorganize the feature channels and spatiality. Then the reorganized feature will be fed into four cascaded encoder stages that downsample and extract features gradually, producing encoded features 11:L at 1/8, 1/16, 1/32, and 1/64 resolution levels of input x. Each encoder stage is formed by two branches: a main cascaded convolutional branch and a single-scale, wavelet- based convolutional branch. The two branches are summed to better capture multi-scale and multi- frequency components. At the end of the encoding pathway, a convolutional block will produce two 1/64-resolution bias features, Em bias and Dec bias. These features seed the subsequent entropy models and decoders and are directly quantized and entropy- into the bitstream.
Starting from the Em bias and Dec bias, each level is processed by its entropy model branch in sequence first. The entropy model conditions the entropy-decodke level eJ- (for the first scale, it is Em bias) as prior Pt to estimate the posterior distribution qi of the current level's 'ss  encoded features rj- Then we compute the KI. divergence between the posterior and the prior z II Pt), yielding the seale-wise rate term RG. We then sample the latent 2 from posterior q:- Adding the sampled z₁ the previous level's entropy-decoded features e pro- duces the current level's entropy-decoded features et. Next, the decoder at each level concatenates the previous level's 's decoded features d (for level one this is Dec bias) with the current level's entropy -ecoded features e/ and fuses them to produce the level's decoding features dj. At the end of each level, the entropy-decoded features et and decoding features d, are upsampled and refined, then passed to the next level. After four levels, the final decoding features d, are Un-Pat ch f ed to produce the final reconstructed image.
features from the previous
To focus on the training dynamics of the hierarchical structure, we adopt simple CNN backbone and design the Basi cblock shown in Fig, 9, Each Basicblock consists of a re-parameterized convolution, a SiLU activation, and Feed-Forward Neural Network (FFN) module in cascade, with a residual shortcut. The block is lightweight and compact, which preserves basic feature-eppressive ability while keeping inference efficient.
Progressive Decoding: The hierarchical structure naturally supports a progressive decoding func- tion, which enables independent decoding with scale-wise latent. We can leverage this characteristic to quantify scale-wise contribution to the final reconstruction. Specifically, for scale I, we decode normally for previous scales < I. For subsequent scales > I, we replace their posteriors q=1 with the mean of priors P>1, so no information from the bitstream of those scales is leveraged. Then a scale- wise progressively decoded reconstruction xf can be obtained. In this case, we can define the residual xf xf-1 as the contribution of the latent variable at the 1-th scale, denoted as If. Next, we transform If and the input image x to the frequency domain with the discrete cosine transform (DCT). We then compute their spectral overlap and plot it as a heatmap, as illustrated in Fig.
Implementation Details of DCT and DWT: For the intra-scale regularization, we adopt the stan- dard orthogonal DCT-II basis with a full-image size. After computing the DCT coefficients, a fre- quency mask is applied that preserves only subset of low-frequency components while zeroing out the others. The mask is progressively expanded during training: in the lincar schedule, the retained frequency ratio grows linearly from 0.05 to 1.0 over the first 100 epochs; in the exponential sched- ule, the ratio follows r(t) ro + (1.0- ro) 1-exp(-0.02xx such that r(t) initializes to ro (i.e., 0.05) and approaches 1.0 at epoch 100.
100)
For the inter-scale regularization, we employ Haar wavelet downsampling block (only retain and concatenate LL. LH, and HL components), followed by an 1 x 1 convolution to align feature reso- lutions before computing similarity. We also experimented with alternative implementations (c.g., stride-2 convolution, downsample followed by convolution), and found that the combination of Haar wavelet and x 1 convolution provided the best performance gain.
15

---

Published as a conference paper at ICLR 2026
Freq Shortcut Pathway (FSP)
Optional (See Appendix A.3)
Dane
FSP
FSP
FSP
FSP
Encoder
Basicalook
Basicalock
Dons Wwn.Doon Down
Basichlock
Beoichlock
bios
Cone
Donu
Down
Ern olas
Entropy Model
Latton
Bestcbookk cook
PotehUpp
Besicbiock
Latter Block
Besicblock
Up Rellne
Basicbiook
Liteet Brock
Basicblock
PatehUjp
Lateet Black
Bestcblock
Decoder
Basichlock
Ub-Rafin
Patch Up
Mere
Besichlokk
Us Refree
Beacblook
Besicblock
Parc-Uj
Merg
Besicblook
Scales: 192
Seated: 3014
Scatez: 512
Scatel
Conv
SIUVI
Conc 13
DW
DW Convat
Latent
Block
Free
py (e((i))))))
Basic block
FFN
Batk slook
Coan
Figure 9: Our proposed lightweight hierarchical image codec architecture. The above the overall network framework, where the three rows from top to bottom are the encoding pathway, entropy model pathway, and decoding pathway. And the shaded area represents the FSP module, which has been proven to be unnecessary in Appendix A.4. The lower left corner shows the network structure of the latent block in the entropy model, while the bottom right corner shows the structure of the basic model employed in our whole architecture.
Further explanation of the inter-scale regularization: We perform explicit frequency-aware alignment by applying DWT-based transform to the lower latent, which decomposes into fre- quency sub-bands. We then apply a x1 convolution across channels to linearly map and recompose frequency sub-bands so they align with the higher-scale latent's s frequency channels. The inter-scale penalty then discourages the aligned coarser latent from predicting the tent the higher latent. Thus, the higher-scale latent retains the difficult-to- toeeedt high-free gigh-freueecc components while redundant fre-fregynccy parts are suppressed, alleviating spectral aliasing.
low-frequency
Implementation Details of Fig. 1: Figure is a heatmap that measures the degree of spectral over- lap between the scale-wise latent and the original image during training (in practice, this reflected through the progressive decoded reconstruction of each scale). The horizontal axis represents the number of training epochs, and the vertical axis represents the normalized frequency range. A point at (epoch, frequency position) indicates the peectral-overlan intensity between the latent of a certain scale and the original image that frequency when training reaches the current epoch. The specific implementation steps are as follows:
1. Using progressive decoding, obtain 21, which is normally decoded to layer 4, while the remaining L layers directly use the mean value of prior;
2. Define I àt - (To -n)) the mutual information between the reconstruction using only the latent of scale and the original image;
3. Apply 2D-DCT transform to the original image æ and each layer's mutual information 4 obtain their spectra F, and F;;
4. Perform radial binning the 2D spectrum and convert it into 1D form;
16

---

Published as a conference paper at ICLR 2026
5. Normalize the frequency range and compute the spectral-overlap degree between the mu- tual information It and the original image in each bin: = 14+4 F
6. Finally, use the matplotlib plotting script to generate the heatmap for each scale.
Besides, We also add the matple ot b plotting script for Figure as follow.
import numpy as np
import matplotlib ptppplttib pit from numpy. numpy.linas linalg import
data = np. np.load(( spectra P = data['p']
axiq datal'freqf
epochs, layers, bins P.shape
fraction = (P. sum (axis=1, keepdims=True) 1c-12)
centroiós np. (epochs, layers))
leakage np. zeros (epochs, layers))
r_cut 0.3 for in range (epochs):
for in range (layers):
Pk Ple, k, :]
centroids(e; (e, k] (freq_axis .k)m) suk.sum() 1e-12)
leakage k] Pk[freg_axis [_muu].sum/ / (Pk. sum 1c-12)
similarity np. zeros (epochs, layers, layers))
for e range
for i in range (1ayers): for in range (layers): vi Ple, i, vj Ple, i, :l
denom (norm (vi) nerm(vj) 1e-12) similarityle, i, jl - np. (vi, vj) denom
fig, plt. subplots (4, layers), figsize==12 (12, 4), sharey=Crue) for
layers))
data = fractionl:; k, :].f
im ax)s[k] imshow (data, aspect='auto', Soogin='lowerr' vmin=data. min(), vmax=data.max
extent=(0, epochs 1, freg_axis(0), freq_axis
[-1]]] axes [e]_see_xlabbl] (pochh', fentsize=10)
if -- 0:
axes kkllkk.se .xes|kl.set_yllaee (' Norraliiee Radial Freq.', fontsize=10)
exeslk] .etk/..et__ee. set.k.eee_tiieeeeeeee Scale (k+ll', fontsize=19) axes params (axis='both', labelsize=t)
cbar_ax fig. (10.92, 0.15, 0.02, 0.71)
fig.colorbar (im, cax-cbar_ax)
plt.tight_layout (rect- 10, 0, 0.9, 11) pad 0.44
bbox inches=/ transparent-False,
plt.close (fig)
For the traditional image codecs, VTM-22.0, We directly employ its open-source standard testing software https: //v//gttt.tt hhi fraunhofer de//veet/Vicsttward. VTM, and the testing commands are illustrated as follows:
Benchmarks:
# Convert RGB image to YUV444 format ffmpeg -i [input_file fill] \
17

---

Published as a conference paper at ICLR 2026
[width]x [height]
-pix_fmt yuv444p [output_file]
# Encode
VVCSoftware
www.ottwar.....//iiniiii. itW.bba/FEncddeepptttttctccc -i [input_file] file] \
[config_file]
-q [quality]
-o [output_path]
-b [bitstream_fii] [ile]
-wdt [image_xidth] \
-hpt [image_height]
-fr 1
-f 1 - 1nputHitllepthh@
conforrmnnce/innooooilel- \
Decode
http://www..///bbii/ee/ccceeaapp -b [bitstream_file] \
-o [output_filel
-d 8
For the other learned image codecs, whenever possible, we use the open-source code to conduct evaluation under the same setups.
A.3
MORE RESULTS
In this section, we present additional comparisons on both objective and subjective quality, including rate-distortion curves optimized with MSE and MS-SSIM, respectively, well as visual compar- isons of reconstructions from different codecs. We also provide progressive decoding results at each scale before and after applying the proposed regularization. The results are as follows.
18

---

Published as a conference paper at ICLR 2026
Kodak
38
37
36
35
PSNR
34
S2C-Hybrid
HPCM-Large
33
MLIC+
TCM-Large
ELIC
32
MambaVC
FTIC
31
VTM 22.0
QARV
DHIC
30
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
BPP
Figure 10: Rate-Distortion curves on Kodak dataset, all models optimized with MSE
19

---

Published as a conference paper at ICLR 2026
CLIC Pro Valid Set
39
38
37
36
35
PSNR
34
S2C-Hybrid
33
HPCM-Large
MLIC+
TCM-Large
32
ELIC
FTIC
VTM 22.0
31
QARV
DHIC
30
0.1
0.2
0.3
0.4
0.5
0.6
BPP
Figure 11: Cate-Distortion curves on CLIC Professional Valid dataset, all models are optimized with MSE.
20

---

Published as a conference paper at ICLR 2026
Tecnick 1200x1200
39
38
37
36
PSNR
35
34
S2C-Hybrid
HPCM-Large
MLIC+
33
TCM-Large
ELIC
FTIC
32
VTM 22.0
QARV
DHIC
31
0.1
0.2
0.3
0.4
0.5
0.6
BPP
Figure Pate-Distortion curves on Tecnick dataset, models are optimized with MSE.
21

---

Published as a conference paper at ICLR 2026
Kodak
22
20
18
MS-SSIM
HPCM-Large
16
MLIC+
TCM-Large
ELIC
MambaVC
14
MambaIC
FTIC
VTM 22.0
QARV
DHIC
12
0.1
0.2
0.3
0.4
0.5
0.6
BPP
Figure 13: Date-Distortion curves on Kodak dataset, models are optimized with MS-SSIM.
22

---

Published as a conference paper at ICLR 2026
CLIC Pro Valid Set
21
20
19
MS-SSIM
18
17
16
HPCM-Large
MLIC+
FTIC
15
QARV
DHIC
0.1
0.2
0.3
0.4
0.5
BPP
Figure 14: Rate-Distortion curves on CLIC Professional Valid dataset, all models are optimized with MS-SSIM.
23

---

Published as a conference paper at ICLR 2026
Tecnick 1200x1200
22
20
MS-SSIN M
18
16
HPCM-Large
MLIC+
FTIC
14
QARV
DHIC
0.1
0.2
0.3
0.4
0.5
BPP
Figure 15: Disttiistotioon curves on Tecnick dataset, models are optimized with MS-SSIM.
24

---

Published as a conference paper at ICLR 2026
Original Image
VTM-22.0
GARV
MambalC
Base Modal (Ours) 0.339
Regulariand (Ours] 0.331 $1.96
BPP PSNR
0.947 131.81
0.3421 31.54
(a) Kodak013.pmm
UPLA
TOL
lired
Freed
Fieed
lired Mambalt: 0.305 (32.87
lieed
Fieed Regularized (Dun) 0.299 32,80
Orginal image BPP PSNR
VIM-220
GARV 0,297 32.20
Base Miccial (Ourn) 0.300 32.01
0.250 19228
(b) Kodak008.prg.
Figure 16:
Subjective quality visualization of decoded reconstruction of various image codecs
(Zoom in for more details).
A.4 MORE ANALYSIS
In this section, we supplement more analysis and experiments of our regularization method to in- vestigate the principles and effectiveness of it. Besides, based on this, we further explore the fun- damental advantages and characteristics of hierarchical image codec, especially with regularized optimization, compared the vanilla hierarchical model with naive training approach and the best single-scale paradigm (e.g., HIPCM-Large (Li et al., 2025c)). The details are presented as follows.
Ablation study on the basic network design: Compared to the previous hierarchical coding archi- tectures (Duan et al., 2023b), our basic model has integrated some module improvements to further enhance its performance as shown Fig. 9, including introducing a skip connection like FSP mod- ule between scales and using reparameterization design in the iasick.ock. To further quantify their effects, we have supplemented relevant ablation studies here. Moreover, it is worth empha- sizing that these designs only aimed at improving the basic structure, which are not related to the main contribution of this architectures.
the spectral analysis and regularization training of hierarchical
The FSP module consist of DWT module, followed by a conv 1x1 operation and finally transformed back to feature domain via IDWT, serving skip-like component between scales in our hierarchical coding architecture. In practice, we integrate the FSP module only during the finetuning stage after the main regularization training, and get about 0.68% bitrate savings. However, further experiments reveal that this gain primarily resulted from more complete finetuning under an adjusted LR scheduler setup (i.e., ReduceLRon? eau patience increase from 2 to 5), not from the F SS itself. Equivalent finetuning schedules without FSP can still produce similar rate- dist-iitootion gains, and adding FSP do not produce noticeable training speedups, too. Therefore, FSP is not necessary for our hierarchical architecture. The ablation results are reported in Table
Table 4: Ablation study on the FSP module. (Baseline anchor for BD-Rate: VTM-22.0, Test dataset: Kodak)
Metrics
w/o F FSP+wrong setup w/ FSP+full training w/o FSP+full training
BD-Rate (%)
-19.05
-19.73
-19.70
Numbers finetuning epochs (%)
78
Besides, we re-parameterize the original DWconv 3x3 of BasicBlock into three-branch
structure, DWconv 3x3, DWconv 1x1, and ident ity during training. At test time these branches fused into single convolution. This training-time time multi-branch design effectively pands model capacity. To quantify its effect, we re-train a network without this re- paraamtttrriztionn and compare R-D performance and training speed; the results summarized in Table 5.
25

---

Published as a conference paper at ICLR 2026
Table 5: Ablation study on the oe-parameterization (abbreviated Rep.) design. (Baseline anchor for BD-Rate: VTM-22.0, Test dataset: Kodak)
Metries
w/o Rep. w/ Rep. w/Rep. but keep similar KMACs
BD-Rate (%)
-6,445
-19773
-18.10
Numbers of full training epochs (%)
350
380
500
Why L2 distance is preferable in inter-scale regularization: In general, the gaussian distribu- tion assumption used by the L2 loss aligns more closely with the conditional probability modeling process y== 2{-1) under the gaussian approximate prior in the hierarchical structure. Using the L2 loss can directly and effectively minimize the negative log-likelle likelihood between 2--1 and (i.e., maximize the log- sggliillihood), forming a more effective mechanism in which the preceding layer predicts the subsequent one.
The detailed derivation is as follows. We already know that in our hierarchical coding architecture, the distribution of the prior p(21.1) can be expressed as
p(=I.)) = pl =0) II I(al==)) 1=1
where p(zo) is the probability distribution of the initial learnable bias and can be temporarily ig- nored, while the conditional distribution p(2; 2=1) is typically regarded a gaussian distribution of the form NV (2)(2)),, 1) in hierarchical coding or HVAE.
Its negative log- lellliklliood can be written as:
f(=-1) -(a-))- C,
27 2
(8)
where the first term is exactly an L2 loss form. Therefore, the L2 loss is maximizing the negative log-likelihood between the two latent layers =1-1 and zt. meaning that zı-1 can better help predict zı- Note that our final objective is to maximize the L2 loss, namely to avoid excessive similarity between the two latent scales, which would lead to spectral aliasing and redundant bitrate. In addition, the ablation study in Table 3 also empirically demonstrates that the alignment effect of the L.2 loss is superior to other methods (such as L1 loss or cosine similarity).
Ablation study on different choices of the weight parameter 5: We conduct a series of training with different values to explore the optimal weight parameter. We find that setting too large or too small leads decline in rate-distortion performance; therefore, we ultimately set 8 = 0.1, as detailed in Table
Table 6: Ablation study on different choices of the weight parameter 8 (Baseline: Our proposed hierarchical architecture without integrating regularization).
Values of BD-Rate (%)
0.05
-7.15
0.1
-11.50
0.2
-8.68
Compression performance with respect to image resolution: Test images used in the main text have relatively small spatial resolutions. Yet, high-definition or even ultra-high-defnnition im- ages are increasingly prevalent in our daily lives, imaging devices have advanced significantly in recent years. We thus conduct ablation studies to evaluate the compression performance with high-definition images. We compared the performance and complexity of the proposed hierarchical coding model with regularization ("DHIC-Regu") the base model without regularization ("DHIC- Base"), as well as the latest single-scale model, HPCM-Large (Li al., 2025c).
In practice, we utilized the LIU4K-v2 valid dataset (Liu et al., 2020), which comprises many high- resolution, complex, and visually high-quality 4K-resolution images. Samples downsampled
26

---

Published as a conference paper at ICLR 2026
multiple times to testing dataset with images different resolutions. We then test the rate- distortion performance, decoding time changes, as shown Table
As resolution increases, the hierarchical coding structure yields larger performance gains while its decoding time grows more slowly than that the single-scale baseline. With additional regulariza- tion, decoding speed remains effectively constant and performance improves further. These results highlight the hierarchical codec's scaling efficiency, making it well-suited to emerging real-time, high-resolution applications.
Table 7: Compression performance and decoding complexity for images with various resolutions (Baseline anchor for BD-Rate: VTM-22.0). The best method at each resolution marked in blue.
Resolution
Method
BD-Rate (%)
Decoding Time (ms)
DHIC-Base
-3.27
56,59
480x270
DHIC-Regu
-3.98
56.59
HPCM-Large
-4.12
72.80
DHIC-Base
-6.01
88.
940
DHiC-Regu
-7.58
88.14 14
HPCM-Large
-7.50
189.92
DHIC-Base
10.65
341.18
1920x 1080
DHIC-Regu
-13.98
341.18
HPCM-Large
-10.81
648 .40
DHIC-Base
-13.66
1297.44
3860x2160
DHIC-Rep -17.19
1297.44
HPCM-Large
-12.98
2754.45
Effectiveness of our proposed regularization method at hierarchical codecs with different com- plexity levels: Under our current network architecture, by adjusting the network width, depth, and number of cascade modules, obtain models at different complexity levels; the performance com- parison is shown Table 8. can observed that our method consistently yields improvements across models of varying complexity.
Table 8: Effectiveness of our proposed regularization method hierarchical codecs with different complexity levels (Baseline: VTM-22.0).
Complexity (KMACs/pixx BD-Rate (%)
BD-Rate w/regularization (%)
Training Acceleration
356.41
3.38
-1.74
1.8x
683.89
-1.60
-11.22
2.0x
977.73
-9.62
-19.73
2.3x
Effectiveness of the proposed inter-scale regularization method in other HVAE- frame- work: To validate the generalizability of our proposed regularization method, implement the regularization on an additional, representative bicrarchical coding architecture QARV (Duan et al., 2023a) and conduct the same experiments. Compared to the hierarchical architccture proposed and used the main text, QARV shares a similar pipeline but utilizes more latent blocks in the entropy model pathway for each scale. As illustrated in Table 9, the results show that regularization strat- egy remains effective on QARV, yielding an approximately 1.65x training speed-up and an 8.20% bitrate savings.
Table 9; Effectiveness of using proposed regularization QARV (Duan et al., 2023a) (Baseline: vanilla QARV).
Regularization Setup
BD-Rate (%)
Training Speed
w/ intra scale
-0.42
1.53x
w/ inter scale
-6.06
1.02x
both
-8.20
1.65x
27

---

Published as a conference paper at ICLR 2026
Applicability to single-scale models with complex context modeling: Although the proposed regularization method is designed to address the spectral issues observed on multi-scale latents in hierarchical structures. But further, we also curious whether this approach is equally effective different slices in the context modeling process of a single-scale structure. To this end, we conduct experiments three representative single-scale codecs with different advanced contexts, ELIC (He et al.. 2022), MLIC+ (Jiang et al., 2025), and HPCM (Li et al., 2025c). The results are illustrated in Table 10 below.
Table 10: Further results of integrating the proposed regularization methods (abbreviated as Regu.) on three single-scale codec. (Baseline anchor for BD-Rate: VTM-22.0, Test dataset: Kodak)
Codecs
Metrics
w/o Regu.
w/ Regu.
BD-Rate (%)
-3.56
-3.88
ELIC
Number of epochs
220
205
BD-Rate (%)
-9.22
-2.56
MLIC+
Number of epochs
525
490
BD-Rate (%)
115.5
-7.67
HPCM
Number of epochs
475
490
It can be seen that integrating the proposed regularization ELIC (He al., 2022) can bring slight performance and training speed gains, but it is not as obvious as our hierarchical structure. In the context modeling processes of MLIC+ (Jiang et al., 2025) and HPCM (Li al., 2025c), which are more complex, performance degradation even occurs. This suggests that additional customized designs may be needed to adapt single-scal scale VAEs, which is also a promising direction for our future work.
In depth, We analyze the essential differences between hierarchical coding structures and single- scale ones with various context modeling process.
Hierarchical models perform explicit scale-by-scall transformations that naturally produce multi- scale latents, where each scale roughly corresponds to specific frequency band. Finally, it can conduct scale-wise coding process by using upper-scale latent as a condition to assist in modeling lower-scale one. Based on this, integrating the proposed intra-scale and inter-scale explicit regu- larization into this process, essentially corresponds scale-by-scale to the entire pipeline of model feature transformation, latent modeling, and decoding reconstruction, making casier to address issues such as spectral energy dissipation within scales and spectral aliasing between scales.
By contrast, for single-scale models with various context modeling, they rely on powerful condi- tional modeling capabilities to directly fit the single-scale latent conditional distribution. Essentially, it is still a conditional probability modeling of the single-scale latent obtained from single-scale transformation (the hierarchical design in HPCM Li et al. (2025c) is also a hierarchical conditional probability modeling of the single-scale latent). There no explicit multi-scale transformation of the input signal corresponding to the multi-scale latent design. so it cannot guarantee the natural frequency decomposition process. In other words, in the context modeling process, it is difficult to ensure that different slice components of single-scale latent can be effectively decomposed on the spectrum and modeled scale by scale. Blindly using such regularization may even disrupt the original context modeling design.
B THE USE OF LARGE LANGUAGE MODELS
We would like to acknowledge the assistance of AI tools in improving the expression of partial writing in this paper. All ideas, methods, code implementations, experiments, and the overall con- ceptualization of this work were independently developed by the authors, with no involvement of AL.
28


Reviews:
[{"review_id": "MN7InDNETR", "reviewer": "Reviewer_NjpX", "summary": "This paper investigates the optimization challenges in hierarchical learned image compression via spectral analysis. The authors identify two major issues—cross-scale energy dispersion and spectral aliasing—and introduce two regularization strategies to mitigate them: (1) intra-scale frequency regularization through progressive DCT truncation, and (2) inter-scale latent regularization using similarity penalties. Experimental results demonstrate strong performance, achieving a 20.65% BD-Rate improvement over VTM-22.0 and a 2.3× training speedup without adding inference complexity.", "questions": "Please refer to Weaknesses.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The spectral interpretation of hierarchical training dynamics provides an intuitive understanding of cross-scale interactions and training instability.\n2. Strong empirical results: 1) State-of-the-art compression performance across multiple datasets, 2) Significant 2.3× training acceleration without inference overhead, 3) Robust performance across diverse resolutions (480p–4K).\n3. The proposed regularizers are training-only and do not increase inference complexity, making the method easy to deploy.\n4. The identification of spectral dispersion and aliasing as key bottlenecks adds valuable understanding to hierarchical compression models.", "weaknesses": "1. The inter-scale regularization term minimizes $ L2(z_{l-1}, Conv(DWT(z_l)))$, which encourages similarity between adjacent scales. However, the text claims the objective is to make features “as distant as possible”. This appears contradictory and should be clarified.\nThe paper lacks a formal explanation of how minimizing latent similarity mitigates spectral aliasing. The relationship between spatial-domain similarity and frequency-domain decoupling needs stronger theoretical grounding.\n2. The model diagram includes the FSP component, but it is not sufficiently discussed in the text. Compared to QARV, it remains unclear how much FSP contributes to convergence speed and rate–distortion performance. FSP seems conceptually close to AuxT [1], yet no detailed comparison or ablation is provided, despite citing AuxT.\n3. The reparameterized block structure is mentioned but not thoroughly evaluated and its impact on performance should be quantified.\n4. Hierarchical VAE has been previously applied to image compression (e.g., by Yueyu Hu et al. [2-3] ); this line of work should be properly cited and discussed.\n5. It would be valuable to analyze whether the proposed regularization strategies could be extended to models such as HPCM or MLIC++, where complex context modeling divides single-scale features into multiple slices. Would regularizing these slices be similarly effective?\n\n[1] Li, Han, et al. \"On disentangled training for nonlinear transform in learned image compression.\" arXiv preprint arXiv:2501.13751 (2025).\n\n[2] Hu, Yueyu, et al. \"Learning end-to-end lossy image compression: A benchmark.\" IEEE Transactions on Pattern Analysis and Machine Intelligence 44.8 (2021): 4194-4211.\n\n[3] Hu, Yueyu, Wenhan Yang, and Jiaying Liu. \"Coarse-to-fine hyper-prior modeling for learned image compression.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 34. No. 07. 2020.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EvFxNxXq6s", "reviewer": "Reviewer_QW9U", "summary": "This paper tackles optimization bottlenecks in hierarchical image compression by identifying that standard training suffers from \"spectral aliasing\" and \"energy dispersion\" across scales. To resolve this, the authors introduce two training-only spectral regularizers: an intra-scale DCT-based curriculum for progressive low-to-high frequency learning, and an inter-scale similarity penalty to reduce redundancy. This approach accelerates training by 2.3x and sets a new state-of-the-art, achieving a 20.65% average BD-Rate saving over the VTM-22.0 video codec. I like the paper's insightful analysis and strong empirical results. However, its methodological soundness is weakened by a lack of theoretical justification connecting the proposed regularization schemes to their stated goal of spectral separation.", "questions": "1.Regarding Figure 1:\n Could you provide a more detailed technical explanation of how these heatmaps were generated? Specifically, how is the \"spectral overlap\" between a scale's contribution and the input image quantitatively defined and computed? The current description in the Appendix is a bit brief for full reproducibility.\n\n2.Regarding Figure 9:\nCould you elaborate on the specific design purpose and function of the FSP?\nWhat is the relationship between the FSP and the proposed Inter-Scale Latent Regularization? Do they work synergistically, or are they independent components?\nI noticed a similar shortcut structure in a recent related work, AuxT [R1]. Could you clarify the key differences and relationships between your FSP and the auxiliary structure in AuxT?\n\n[R1] Li, et. al, On Disentangled Training for Nonlinear Transform in Learned Image Compression. ICLR 2025.\n\n3.Can the proposed spectral regularization methods be adapted for non-hierarchical (single-scale) compression frameworks?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "1.The paper provides a novel and insightful spectral analysis, identifying \"energy dispersion\" and \"spectral aliasing\" as the root causes of training difficulties in hierarchical compression models. The compelling visualizations strongly support this diagnosis.\n2.The proposed regularization techniques are well-designed and directly target the diagnosed spectral issues. The approach is rigorous, logically sound, and adds no inference overhead, making it practical.\n3.The method achieves state-of-the-art performance with substantial gains, including a 20.65% average BD-Rate saving over VTM-22.0. The 2.3x training acceleration is a significant practical advantage.\n4.The paper is supported by thorough experiments, including extensive SOTA comparisons, detailed ablation studies, and proven generalizability to other architectures, which robustly validate the authors' claims.", "weaknesses": "1.The paper's design, guided by the \"frequency principle,\" assigns low-frequency content to semantically deeper scales and high-frequency content to shallower scales. This creates an apparent tension with the common understanding that more challenging, high-frequency details often benefit from the greater expressive power of deeper network layers. While the empirical results are strong, the paper's methodological contribution would be further strengthened by a clearer theoretical justification for this seemingly counter-traditional design, explaining why assigning the most complex information to relatively shallower structures is advantageous in this context.\n\n2.The paper lacks a theoretical basis for its claim that an L2 latent penalty (Eq. 6) enforces spectral separation between scales. The connection between latent distance and spectral orthogonality is not established. Furthermore, the implementation uses a 1x1 convolution, a feature-mixing operation that could potentially contradict the goal of suppressing spectral aliasing. The authors should clarify the true source of this regularizer's effectiveness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Df69jLUotd", "reviewer": "Reviewer_HdPe", "summary": "The authors proposed two techniques to address the inefficiency in training in hierarchical coding. The first is for intra-scale frequency regularization, which is to guide the training frequencies by progressive truncation. The second for inter-scale similarity regularization, which is to suppress the similarity between neighboring scales (?). Experimental results demonstrate the successfulness of the proposed method.\n\nFor the weakness, 1. The inter-scale similarity regularization part needs more explanation. It looks like that Eq.(6) encourages similarity instead of suppressing it. 2. Some figures are not helpful for explaining the method, e.g. Figs 4 and 5.", "questions": "1.\tThe inter-scale similarity regularization part needs more explanation. It looks like that Eq.(6) encourages similarity instead of suppressing it.\n2.\tSome figures are not helpful for explaining the method, e.g. Figs 4 and 5. \n3.\tWhat is the meaning of “time t”?\n4.\tThe inner minimum in Eq.(4) seems not necessary.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The authors proposed two techniques to address the inefficiency in training in hierarchical coding. The first is for intra-scale frequency regularization, which is to guide the training frequencies by progressive truncation. The second for inter-scale similarity regularization, which is to suppress the similarity between neighboring scales (?). Experimental results demonstrate the successfulness of the proposed method.", "weaknesses": "1. The inter-scale similarity regularization part needs more explanation. It looks like that Eq.(6) encourages similarity instead of suppressing it. \n2. Some figures are not helpful for explaining the method, e.g. Figs 4 and 5.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oQy8bdBSnx", "reviewer": "Reviewer_RfuU", "summary": "They propose explicit spectral regularization (intra-scale frequency regularization for smooth frequency buildup across scales, inter-scale similarity regularization to suppress cross-scale aliasing) — applied only in training with no inference overhead. Experiments show the method accelerates vanilla model training by 2.3x, achieves 20.65% average rate–distortion gain over VTM-22.0 on public datasets, outperforms single-scale approaches, and sets a new SOTA in learned image compression.", "questions": "See Weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. This manuscript tries to extract disentangled features from images using spectral regularization.\n\n2. The motivation is clear.\n\n3. The results about the performance of the proposed method are convincing in Kodak, CLIC datasets.", "weaknesses": "1. The related details of the proposed method are not described clearly. For example, how to obtain the y-axis of Figure 1 ? What does it mean about the lines with different color of Figure 8?\n\n2. The training of inter-scale regularization as shown in Figure 5 may not be stable since $z_1, z_2$ maybe the noise at the early stage and the corresponding regularization makes no sense.\n\n3. There exists no analysis or discussion about \"Different scales converge to their respective frequency bands at different rates\" at line 183. It could be better if this manuscript provides some clues.\n\n4. For visualization in Figure 6, it is hard to totally distinguish the frequencies of the details without any aliasing. Besides the demonstration of Figure 6(b) has little color shift compared with 6(c), can the regularization be more effective for color representation?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)