HiCache: A Plug-in Scaled-Hermite Upgrade for Taylor-Style Cache-then-Forecast Diffusion Acceleration

## Abstract

Abstract Diffusion models have achieved remarkable success in content generation but suffer from prohibitive computational costs due to iterative sampling. While recent feature caching methods tend to accelerate inference through temporal extrapolation, these methods still suffer from server quality loss due to the failure in modeling the complex dynamics of feature evolution. To solve this problem, this paper presents HiCache, a training-free acceleration framework that fundamentally improves feature prediction by aligning mathematical tools with empirical properties. Our key insight is that feature derivative approximations in Diffusion Transformers exhibit multivariate Gaussian characteristics, motivating the use of Hermite polynomials—the potentially theoretically optimal basis for Gaussian-correlated processes. Besides, We further introduce a dual-scaling mechanism that ensures numerical stability while preserving predictive accuracy. Extensive experiments demonstrate HiCache’s superiority: achieving 6.24× speedup on FLUX.1-dev while exceeding baseline quality, maintaining strong performance across text-to-image, video generation, and super-resolution tasks. Core implementation is provided in the appendix, with complete code to be released upon acceptance.

## 1. Introduction

Diffusion Models (DMs) (Ho, Jain, and Abbeel 2020; Nichol and Dhariwal 2021; Dhariwal and Nichol 2021), especially when implemented with the scalable Transformer architecture (DiT) (Peebles and Xie 2023), have set a new standard for high-fidelity content generation (Chen et al. 2023; Rombach et al. 2022; Karras et al. 2022; Saharia et al. 2022; Balaji et al. 2022). This success, however, is tempered by the substantial computational overhead inherent to the diffusion process. The iterative sampling, requiring hundreds of sequential forward passes through large models, results in high latency that critically hinders practical deployment. Consequently, developing efficient acceleration methods is a key research objective.

Among various acceleration strategies, feature caching has emerged as a particularly effective and training-free approach.
Motivated by the observation that diffusion models exhibit similar features in the adjacent timesteps,
the original feature caching seeks to directly reuse features in the adjacent timesteps, which
leads to significant generation quality loss when the acceleration ratios increase (Selvaraju et al. 2024; Ma et al. 2023; Zou et al. 2024). Recently, the paradigm has advanced to a “cache-then-forecast” mechanism as TaylorSeer (Liu et al. 2025), where Taylor series expansion is used to extrapolate features in the future timesteps, significantly reducing the error from feature caching. TaylorSeer significantly reduces the error introduced by feature reusing, but still faces fundamental theoretical limitations:
*The standard polynomial basis of a Taylor series is suboptimal for modeling the complex, non-monotonic trajectories of feature evolution in DMs.*

Figure [2](https://arxiv.org/html/2508.16984v1#Sx1.F2) visualize the features of the original diffusion models (*i.e., ground-truth*) and the predictive features from TaylorSeer. It is observed that Taylor-based extrapolation exhibits significant deviations from ground truth trajectories, particularly at turning points where its monotonic nature fails to capture the underlying dynamics.
This mismatch between the Taylor-based predictive model and the underlying feature distribution establishes a bottleneck for achieving much higher acceleration ratios without significant quality degradation. Besides, TaylorSeer is designed to employ high-order Taylor expansion to make use of more historical information, making it sensitive to the noise in the feature trajectory, further limiting its effectiveness.

Figure: Figure 2: Trajectory prediction comparison of Taylor and HiCache methods on FLUX. *“Full trajectory”* indicates the feature trajectory from the original Flux (Esser et al. 2024) in the 14th and 28th layer. The y-axis denotes the most principal component of the features in diffusion models. HiCache demonstrates more stable fitting at trajectory turning points, while Taylor expansion tends to over-extrapolate and deviate from original trajectories.
Refer to caption: figures/prediction_comparison/prediction_comparison_order3_last_patch_7_layer28_img.png

To overcome this limitation, in this paper, we propose HiCache (Hermite Polynomial-based Feature Cache), a framework that improves the feature forecasting by aligning the predictive basis with the intrinsic statistical properties of the feature dynamics. Our approach is based on a key empirical observation: *the derivative approximations of feature vectors in DiTs consistently exhibit a multivariate Gaussian distribution (details in Section 3.3.1)*. Motivated by this finding, HiCache replaces the standard polynomial basis with scaled Hermite polynomials. According to approximation theory (Szegö 1975), Hermite polynomials are the potentially optimal orthogonal basis for representing Gaussian-correlated processes, making them theoretically better suited for this task.
Besides, to address the numerical challenges of Hermite polynomials at large extrapolation steps, we further introduce a dual-scaling mechanism that simultaneously constrains predictions within the stable oscillatory regime and suppresses exponential coefficient growth in high-order terms through a single hyperparameter. This principled design enables both accurate and numerically stable feature prediction. As shown in Figure [2](https://arxiv.org/html/2508.16984v1#Sx1.F2), the feature trajectory predicted by HiCache closely matches the original FLUX, demonstrating its great effectiveness.

Extensive experiments have been conducted to demonstrate the effectiveness of HiCache in class-to-image generation with DiT, text-to-image generation with FLUX.1-dev, image-to-image generation with Inf-DiT for image super-resolution, and text-to-video generation with HunyuanVideo (Kong et al. 2024), showing consistent improvements over previous methods.
For instance, on FLUX.1-dev, HiCache achieves 5.55$\times$ acceleration with 4.25% improvement on ImageReward over the second-best method.

The main contributions of this work are as follows:

- •
We systematically validate the multivariate Gaussian nature of feature derivative approximations in Diffusion Transformers, offering a new statistical foundation for designing more efficient feature caching methods.
- •
We propose HiCache, which introduces Hermite polynomials into the feature caching of diffusion models, and propose a dual-scaling mechanism to simultaneously constrain predictions within the stable oscillatory regime and suppress exponential coefficient growth in high-order terms, achieving robust numerical stability.
- •
We conduct extensive experiments on four diffusion models and generative tasks, demonstrating HiCache’s universal superiority and broad applicability.

## 2. Related Work

##### Sampling Step Reduction

aims to reduce the total number of denoising iterations. DDIM (Song, Meng, and Ermon 2021) introduced deterministic sampling with larger steps, while DPM-Solver (Lu et al. 2022) formalized this as an ODE with high-order numerical methods. Recent advances include knowledge distillation (Meng et al. 2023) and Consistency Models (Song et al. 2023) for few-step generation. Other notable approaches include cascaded diffusion models (Ho et al. 2022a) and improved sampling techniques (Karras et al. 2022). Text-to-image diffusion models have seen remarkable progress (Saharia et al. 2022; Balaji et al. 2022; Ramesh et al. 2022; Podell et al. 2023; Zhang, Rao, and Agrawala 2023), while video generation (Singer et al. 2022; Ho et al. 2022b) presents additional temporal challenges.

##### Denoising Network Compression

focuses on reducing per-step computational cost. Model compression methods include neural network pruning (Fang et al. 2023) and quantization (Li et al. 2023) have applied the traditional model compression methods to diffusion models.
Besides, token merging (Bolya and Hoffman 2023) has been introduced to reduce the number of tokens computed in each timestep. However, these methods tend to lead clear loss in generation quality, and sometimes rely on retraining the diffusion model.

##### Feature Caching

has been recently proposed to offer a training-free alternative. Motivated by the great similarity of features in the adjacent timesteps, these methods aim to skip the computation in some timesteps based on the historical features, which can be roughly divided into two paradigms: (1) *Cache-then-Reuse* directly reuses features from adjacent timesteps (DeepCache (Ma et al. 2023), FORA (Selvaraju et al. 2024), ToCa (Zou et al. 2024)), but suffers quality degradation at large intervals; (2) *Cache-then-Forecast* predicts future features using historical data. TaylorSeer (Liu et al. 2025) pioneered this approach with Taylor series expansion, significantly improving quality at high acceleration ratios. However, its standard polynomial basis remains suboptimal for complex feature dynamics, motivating our Hermite polynomial-based approach.

## 3. Methodology

### 3.1 Preliminaries: Diffusion Transformer

The Diffusion Transformer (DiT) architecture is central to many state-of-the-art diffusion models. Building upon the self-attention mechanism (Vaswani et al. 2017), it operates by progressively denoising a noisy input $x_{t}$ through a series of $L$ stacked blocks. Within each block, an input feature map $\mathcal{F}_{\text{in}}$ is updated via sequential residual corrections:

$$ $\displaystyle\mathcal{F}^{\prime}$ $\displaystyle=\mathcal{F}_{\text{in}}+\text{MHSA}(\text{AdaLN}(\mathcal{F}_{\text{in}},t))$ (1) $\displaystyle\mathcal{F}_{\text{out}}$ $\displaystyle=\mathcal{F}^{\prime}+\text{MLP}(\text{AdaLN}(\mathcal{F}^{\prime},t))$ (2) $$

where AdaLN denotes adaptive layer normalization conditioned on timestep $t$. This temporal conditioning, combined with nonlinearities (e.g., Softmax and GeLU), causes feature trajectories to exhibit complex, non-monotonic dynamics—a significant challenge for acceleration techniques relying on smooth extrapolation. The importance of such architectural choices in generative models has been widely studied (Karras et al. 2022; Dhariwal and Nichol 2021).

### 3.2 Taylor Expansion-based Feature Prediction

To accelerate sampling, existing methods like TaylorSeer extrapolate future features using Taylor series. We first formalize the finite difference framework:

###### Definition 1 (Finite Difference Operator) .

The $i$-th order backward difference operator $\Delta^{(i)}$ on feature trajectory $\{\mathcal{F}_{t}\}$ is defined recursively:

$$ $\Delta^{i}\mathcal{F}_{t}=\frac{\Delta^{i-1}\mathcal{F}_{t}-\Delta^{i-1}\mathcal{F}_{t-N_{\text{interval}}}}{N_{\text{interval}}},\quad\text{with}\quad\Delta^{0}\mathcal{F}_{t}=\mathcal{F}_{t}$ (3) $$

Taylor expansion-based prediction then takes the form:

$$ $\hat{\mathcal{F}}_{t-k}^{\text{Taylor}}=\mathcal{F}_{t}+\sum_{i=1}^{m}\frac{\Delta^{i}\mathcal{F}_{t}}{i!}(-k)^{i}$ (4) $$

###### Proposition 1 (Limitation of Monomial Basis) .

For feature trajectories with bounded variation but containing turning points, the Taylor prediction error grows as:

$$ $\|\hat{\mathcal{F}}_{t-k}^{\text{Taylor}}-\mathcal{F}_{t-k}\|=O\left(\frac{k^{m+1}}{(m+1)!}\sup_{\xi\in[t-k,t]}\|\mathcal{F}^{(m+1)}(\xi)\|\right)$ (5) $$

where the supremum can be arbitrarily large at trajectory inflection points.

This limitation of the monomial basis $\{(-k)^{i}\}_{i=1}^{m}$ motivates our search for a more suitable basis function.

###### Definition 1 (Finite Difference Operator) .

###### Proposition 1 (Limitation of Monomial Basis) .

### 3.3 HiCache: Hermite-based Feature Caching

In this paper, we propose to leverage the Gaussian nature of feature dynamics for optimal basis selection.

###### Observation 1 (Gaussianity of Feature Derivatives) .

Let $\{\mathcal{F}_{t}\}_{t=1}^{T}$ be the feature trajectory in a Diffusion Transformer. The finite difference approximations $\{\Delta^{k}\mathcal{F}_{t}\}$ follow a multivariate Gaussian distribution:

$$ $\Delta^{k}\mathcal{F}_{t}\sim\mathcal{N}(\mu_{k},\Sigma_{k})$ (6) $$

Validated via Energy statistical tests with p-value = 1.0 across all configurations (see Section 5.1).

###### Corollary 1 (Optimal Basis Selection) .

By the Karhunen-Loève theorem, Hermite polynomials $\{H_{n}\}$ form the optimal orthogonal basis for Gaussian-correlated processes (Narcowich, Ward, and Wendland 2005), implying superior approximation properties over monomials.

###### Observation 1 (Gaussianity of Feature Derivatives) .

###### Corollary 1 (Optimal Basis Selection) .

#### 3.3.2 Hermite Polynomials as Basis Functions

Given the Gaussian characteristics, we adopt Hermite polynomials as our basis functions. The physicist’s Hermite polynomials $H_{n}(x)$ (Abramowitz and Stegun 1964) are defined via the Rodrigues formula:

$$ $H_{n}(x)=(-1)^{n}e^{x^{2}}\frac{d^{n}}{dx^{n}}e^{-x^{2}}$ (7) $$

and satisfy the recurrence relation:

$$ $H_{n+1}(x)=2xH_{n}(x)-2nH_{n-1}(x)$ (8) $$

Unlike monotonically growing Taylor basis functions, Hermite polynomials exhibit inherent oscillatory behavior (as shown in Figure [3](https://arxiv.org/html/2508.16984v1#Sx3.F3)). This oscillatory nature provides implicit regularization (van der Westhuizen and Lasenby 2018), enabling more accurate capture of non-monotonic dynamics in neural network optimization.
However, Hermite polynomials face numerical challenges: they grow exponentially for large $|x|$ and have rapidly increasing coefficients due to the factor of 2 in the recurrence. To address this, we introduce scaled Hermite polynomials:

###### Definition 2 (Scaled Hermite Basis) .

Scaled Hermite polynomials with contraction factor $\sigma\in(0,1)$ are defined as:

$$ $\tilde{H}_{n}(x)=\sigma^{n}H_{n}(\sigma x)$ (9) $$

This scaling provides dual stabilization: (i) input scaling $\sigma x$ constrains predictions within the stable oscillatory regime, and (ii) coefficient scaling $\sigma^{n}$ suppresses exponential growth in high-order terms.

Figure: Figure 3: Systematic comparison of Taylor and Hermite basis functions Hermite’s oscillatory behavior (e.g., $H_{2}(x)$ negative offset) captures non-monotonic evolution better than monotonic Taylor growth. (Y-axis: symmetric log scale).
Refer to caption: x2.png

###### Definition 2 (Scaled Hermite Basis) .

#### 3.3.3 HiCache Algorithm

###### Proposition 2 (HiCache Feature Prediction) .

Given cached derivative approximations $\{\Delta^{i}\mathcal{F}_{t}\}_{i=0}^{N_{\text{order}}}$, the optimal predictor under Gaussian feature dynamics is:

$$ $\hat{\mathcal{F}}_{t-k}^{\text{HiCache}}=\mathcal{F}_{t}+\sum_{i=1}^{N_{\text{order}}}\frac{\Delta^{i}\mathcal{F}_{t}}{i!}\tilde{H}_{i}(-k)$ (10) $$

where $k\in\{1,\ldots,N_{\text{interval}}-1\}$ is the prediction horizon and $\tilde{H}_{i}$ is the scaled Hermite polynomial from Definition [2](https://arxiv.org/html/2508.16984v1#Thmdefinition2).

Proof Intuition. Under the empirically validated Gaussian assumption (Observation [1](https://arxiv.org/html/2508.16984v1#Thmobservation1)), Corollary [1](https://arxiv.org/html/2508.16984v1#Thmcorollary1) suggests that Hermite polynomials provide a theoretically motivated basis for feature prediction. The factorial normalization ensures proper scaling of higher-order terms. See Appendix A.6 for detailed theoretical analysis and supporting arguments.

Figure: Algorithm 1 HiCache: Hermite-based Feature Caching

Computational Complexity. HiCache reduces computational cost by factor $(N_{\text{interval}}-1)/N_{\text{interval}}$ while maintaining $O(N_{\text{order}})$ prediction overhead per skipped step.

###### Proposition 3 (Error Bounds for HiCache) .

The total prediction error of HiCache decomposes as:

$$ $\mathbf{E}_{\text{total}}=\mathbf{E}_{\text{truncation}}+\mathbf{E}_{\text{approximation}}+\mathbf{E}_{\text{numerical}}$ (11) $$

with the bound:

$$ $\|\mathbf{E}_{\text{total}}\|\leq\underbrace{O\left(\frac{(\sigma\sqrt{2}|\Delta s|)^{N+1}}{\sqrt{(N+1)!}}\right)}_{\text{Hermite truncation}}+\underbrace{O(\Delta t_{\text{hist}}\sqrt{N})}_{\text{Finite difference}}+\underbrace{O(\epsilon_{\text{machine}})}_{\text{Floating point}}$ (12) $$

###### Corollary 2 (Superiority over Taylor Expansion) .

HiCache achieves:

- 1.
Faster convergence: $O(1/\sqrt{(N+1)!})$ vs. $O(1/(N+1)!)$
- 2.
Exponential suppression via $\sigma^{N+1}$
- 3.
Automatic adaptation to Gaussian feature statistics

Proof. See Appendix A.3 for complete derivation.

Figure: Figure 4: An overview of HiCache compared with TaylorSeer. TaylorSeer (orange) predicts the feature with a power basis. HiCache (green) uses the same functional form but replaces the basis with Hermite polynomials. Dashed curves show predicted trajectories against ground truth, highlighting error accumulation at skipped steps.
Refer to caption: x3.png

###### Proposition 2 (HiCache Feature Prediction) .

###### Proposition 3 (Error Bounds for HiCache) .

###### Corollary 2 (Superiority over Taylor Expansion) .

## 4. Experiments

### 4.1 Experiment Settings

We conduct comprehensive experiments across four representative generative tasks using state-of-the-art diffusion models: text-to-image generation with FLUX.1-dev (Esser et al. 2024), text-to-video generation with HunyuanVideo (Kong et al. 2024), class-conditional image generation with DiT-XL/2 (Peebles and Xie 2023), and image super-resolution with a modified Inf-DiT (Yang et al. 2024). Each task employs standard evaluation protocols and widely-adopted benchmarks to ensure fair comparison with existing methods. More detailed information is provided in Appendix A.4.

### 4.2 Text-to-Image Generation

##### Quantitative Analysis.

As shown in Table [1](https://arxiv.org/html/2508.16984v1#Sx4.T1), HiCache demonstrates a clear quantitative advantage. We evaluate using ImageReward (Xu et al. 2023) as the primary metric for perceptual quality assessment. At a  5.55x acceleration, HiCache not only surpasses its main competitor, TaylorSeer, but also slightly exceeds the unaccelerated baseline, suggesting near-lossless performance. This advantage becomes more pronounced at higher accelerations ( 6.24x), where HiCache maintains a robust score while competitors’ performance degrades more significantly.

**Table 1: Quantitative comparison on the text-to-image generation task with the FLUX.1-dev model.**
| Method | Latency(s) $\downarrow$ | Speed $\uparrow$ | FLOPs(T) $\downarrow$ | Speed $\uparrow$ | Image Reward $\uparrow$ | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | CLIP $\uparrow$ |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| DDIM-50 steps | 17.12 | 1.00$\times$ | 3719.50 | 1.00$\times$ | 0.9872 (+0.0000) | $\infty$ | 1.0000 | 0.0000 | 32.636 |
| DDIM-30 steps | 10.44 | 1.64$\times$ | 2231.70 | 1.67$\times$ | 0.9749 (-0.0123) | 30.308 | 0.7816 | 0.2452 | 32.656 |
| DDIM-25 steps | 8.77 | 1.95$\times$ | 1859.75 | 2.00$\times$ | 0.9691 (-0.0181) | 29.558 | 0.7327 | 0.3115 | 32.565 |
| DDIM-20 steps | 7.07 | 2.42$\times$ | 1487.80 | 2.62$\times$ | 0.9487 (-0.0385) | 29.122 | 0.6983 | 0.3599 | 32.326 |
| DDIM-17 steps | 6.10 | 2.81$\times$ | 1264.63 | 3.13$\times$ | 0.9147 (-0.0725) | 28.882 | 0.6767 | 0.3912 | 32.273 |
| FORA ($N=7$) | 4.22 | 4.08$\times$ | 670.44 | 5.55$\times$ | 0.7418 (-0.2454) | 28.315 | 0.5870 | 0.5409 | 31.519 |
| ToCa ($N=10,N=90\%$) | 7.93 | 2.17$\times$ | 714.66 | 5.20$\times$ | 0.8384 (-0.1488) | 28.761 | 0.6068 | 0.4887 | 31.834 |
| Taylor (N=7, O=2) | 4.93 | 3.47$\times$ | 670.44 | 5.55$\times$ | 0.9572 (-0.0300) | 28.634 | 0.6237 | 0.4520 | 32.189 |
| HiCache (N=7, O=2) | 4.84 | 3.54$\times$ | 670.44 | 5.55$\times$ | 0.9979 (+0.0107) | 28.937 | 0.6572 | 0.3982 | 32.454 |
| FORA ($N=9$) | 3.90 | 4.42$\times$ | 596.07 | 6.24$\times$ | 0.5457 (-0.4415) | 28.233 | 0.5613 | 0.5860 | 30.764 |
| ToCa ($N=12,N=90\%$) | 7.34 | 2.34$\times$ | 644.70 | 5.77$\times$ | 0.7155 (-0.2717) | 28.575 | 0.5677 | 0.5500 | 31.808 |
| Taylor (N=9, O=2) | 4.60 | 3.72$\times$ | 596.07 | 6.24$\times$ | 0.8562 (-0.1310) | 28.359 | 0.5882 | 0.5088 | 31.662 |
| HiCache (N=9, O=2) | 4.50 | 3.80$\times$ | 596.07 | 6.24$\times$ | 0.9113 (-0.0759) | 28.647 | 0.6443 | 0.4374 | 31.956 |

##### Qualitative Analysis.

As illustrated in Figure [5](https://arxiv.org/html/2508.16984v1#Sx4.F5), HiCache consistently produces high-fidelity results, avoiding common artifacts like facial distortions, conceptual errors, or text inaccuracies that affect competing methods. Furthermore, Figure [6](https://arxiv.org/html/2508.16984v1#Sx4.F6) highlights HiCache’s superior ability to preserve high-frequency details, such as intricate textures and subtle reflections, which are often lost in other accelerated outputs.

Figure: Figure 5: Qualitative comparison on the text-to-image task across nine diverse prompts.
Refer to caption: x4.png

Figure: Figure 6: Visualization of high-frequency detail preservation.
Refer to caption: x5.png

##### Robustness Under Increasing Acceleration.

Figure [7](https://arxiv.org/html/2508.16984v1#Sx4.F7) assesses stability under increasing acceleration factors (from 1.00x to 9.00x). TaylorSeer’s output shows significant style drift and structural degradation, consistent with the error accumulation discussed in Sec. 5.1. In contrast, HiCache maintains high stability and aesthetic consistency, demonstrating its robustness against such errors even at the highest acceleration levels.

Figure: Figure 7: Comparison between TaylorSeer (second-best method) and HiCache in different acceleration ratios.
Refer to caption: x6.png

### 4.3 Text-to-Video Generation

##### Quantitative Analysis.

We further evaluate HiCache on text-to-video generation using the HunyuanVideo model (Kong et al. 2024). As detailed in Table [3](https://arxiv.org/html/2508.16984v1#Sx4.T3), which benchmarks performance with VBench (Huang et al. 2024), HiCache consistently outperforms competitors. Notably, when comparing higher-order configurations (e.g., $N=7,O=2$), HiCache demonstrates even more significant advantages over TaylorSeer, which aligns with our theoretical analysis in Section 3.3.2 that Hermite polynomials’ oscillatory behavior better captures complex feature dynamics at higher orders, while Taylor expansion’s monotonic nature leads to increasing over-extrapolation.

**Table 2: Quantitative comparison on class-to-image generation on ImageNet with DiT-XL/2.**
| Method | Efficient Attention | Latency(s) $\downarrow$ | FLOPs(T) $\downarrow$ | Speed $\uparrow$ | FID $\downarrow$ | sFID $\downarrow$ | Inception Score $\uparrow$ |
| --- | --- | --- | --- | --- | --- | --- | --- |
| DDIM-50 steps | ✓ | 0.995 | 23.74 | 1.00$\times$ | 2.32 (+0.00) | 4.32 (-0.00) | 241.25 (+0.00) |
| DDIM-25 steps | ✓ | 0.537 | 11.87 | 2.00$\times$ | 3.18 (+0.86) | 4.74 (+0.42) | 232.01 (-9.24) |
| DDIM-20 steps | ✓ | 0.406 | 9.49 | 2.50$\times$ | 3.81 (+1.49) | 5.15 (+0.83) | 221.43 (-19.82) |
| DDIM-12 steps | ✓ | 0.128 | 5.70 | 4.17$\times$ | 7.80 (+5.48) | 8.03 (+3.71) | 184.50 (-56.75) |
| DDIM-10 steps | ✓ | 0.224 | 4.75 | 5.00$\times$ | 12.15 (+9.83) | 11.33 (+7.01) | 159.13 (-82.12) |
| DDIM-8 steps | ✓ | 0.189 | 3.80 | 6.25$\times$ | 23.13 (+20.81) | 19.23 (+14.91) | 120.58 (-120.67) |
| DDIM-7 steps | ✓ | 0.168 | 3.32 | 7.14$\times$ | 33.65 (+31.33) | 27.51 (+23.19) | 92.74 (-148.51) |
| FORA ($N=6$) | ✓ | 0.427 | 5.24 | 4.98$\times$ | 9.24 (+6.92) | 14.84 (+10.52) | 171.33 (-69.92) |
| ToCa ($N=9,N=95\%$) | ✗ | 1.016 | 6.34 | 3.75$\times$ | 6.55 (+4.23) | 7.10 (+2.78) | 189.53 (-51.72) |
| TaylorSeer ($N=6,O=4$) | ✓ | 0.210 | 4.76 | 4.98$\times$ | 3.11 (+0.79) | 6.35 (+2.03) | 223.85 (-17.4) |
| HiCache ($N=6,O=4$) | ✓ | 0.193 | 4.76 | 4.99$\times$ | 3.01 (+0.69) | 6.32 (+2.00) | 223.68 (-17.57) |
| FORA ($N=7$) | ✓ | 0.405 | 3.82 | 6.22$\times$ | 12.55 (+10.23) | 18.63 (+14.31) | 148.44 (-92.81) |
| ToCa ($N=13,N=95\%$) | ✗ | 1.051 | 4.03 | 5.90$\times$ | 21.24 (+18.92) | 19.93 (+15.61) | 116.08 (-125.17) |
| TaylorSeer ($N=8,O=4$) | ✓ | 0.190 | 3.82 | 6.22$\times$ | 4.40 (+2.08) | 7.34 (+3.02) | 205.00 (-36.25) |
| HiCache ($N=8,O=4$) | ✓ | 0.172 | 3.82 | 6.21$\times$ | 4.33 (+2.01) | 7.01 (+2.69) | 205.00 (-36.25) |
| FORA ($N=8,N=95\%$) | ✓ | 0.405 | 3.34 | 7.10$\times$ | 15.31 (+12.99) | 21.91 (+17.59) | 136.21 (-105.04) |
| ToCa ($N=13,N=98\%$) | ✗ | 1.033 | 3.66 | 6.48$\times$ | 22.18 (+19.86) | 20.68 (+16.36) | 110.91 (-130.34) |
| TaylorSeer ($N=9,O=4$) | ✓ | 0.179 | 3.34 | 7.10$\times$ | 4.58 (+2.26) | 7.30 (+2.98) | 201.12 (-40.13) |
| HiCache ($N=9,O=4$) | ✓ | 0.161 | 3.34 | 7.11$\times$ | 4.31 (+1.99) | 6.86 (+2.54) | 203.49 (-37.76) |

**Table 3: Quantitative comparison on text-to-video generation with HunyuanVideo on Vbench.**
| Method | Lat.(s) $\downarrow$ | Spd. $\uparrow$ | FLOPs(T) $\downarrow$ | Spd. $\uparrow$ | VBench(%) $\uparrow$ |
| --- | --- | --- | --- | --- | --- |
| Original (50 steps) | 145.00 | 1.00$\times$ | 29773 | 1.00$\times$ | 80.66 |
| DDIM (22% steps) | 31.87 | 4.55$\times$ | 6550 | 4.55$\times$ | 78.74 |
| TeaCache | 30.49 | 4.76$\times$ | 6550 | 4.55$\times$ | 79.36 |
| FORA | 34.39 | 4.22$\times$ | 5960 | 5.00$\times$ | 78.83 |
| ToCa | 38.52 | 3.76$\times$ | 7006 | 4.25$\times$ | 78.86 |
| DuCa | 31.69 | 4.58$\times$ | 6483 | 4.62$\times$ | 78.72 |
| TaylorSeer($N=6,O=1$) | 31.69 | 4.58$\times$ | 5359 | 5.56$\times$ | 79.78 |
| HiCache($N=6,O=1$) | 31.71 | 4.58$\times$ | 5359 | 5.56$\times$ | 79.89 |
| TeaCache | 26.61 | 5.45$\times$ | 5359 | 5.56$\times$ | 78.32 |
| TaylorSeer($N=7,O=1$) | 28.82 | 5.03$\times$ | 4795 | 6.21$\times$ | 79.40 |
| HiCache($N=7,O=1$) | 28.83 | 5.03$\times$ | 4795 | 6.21$\times$ | 79.51 |
| TaylorSeer($N=7,O=2$) | 28.83 | 5.03$\times$ | 4795 | 6.21$\times$ | 79.28 |
| HiCache($N=7,O=2$) | 28.84 | 5.03$\times$ | 4795 | 6.21$\times$ | 79.65 |

Qualitative Analysis.
The qualitative results in Figure [8](https://arxiv.org/html/2508.16984v1#Sx4.F8) reveal that competing methods often suffer from temporal inconsistencies, leading to artifacts such as distorted objects and incoherent movements, a common challenge in video generation (Ho et al. 2022b; Singer et al. 2022). In contrast, HiCache maintains high temporal stability and logical consistency, producing more realistic and coherent video sequences.

Figure: Figure 8: Qualitative comparison of text-to-video generation methods under significant acceleration (approx. 4–5.5$\times$). While competing methods often produce severe temporal artifacts such as distorted hands (“Hands distortion”), misshapen objects (“Fork distortion”), and vanishing tools, HiCache maintains high temporal coherence and object integrity.
Refer to caption: x7.png

### 4.4 Class-Conditional Image Generation

##### Quantitative Analysis.

On the task of class-conditional ImageNet generation (Dhariwal and Nichol 2021) with the DiT-XL/2 model (Peebles and Xie 2023), HiCache again demonstrates superior performance. As shown in Table [2](https://arxiv.org/html/2508.16984v1#Sx4.T2), it achieves a lower (better) FID score (Salimans et al. 2016) compared to competing methods at  5x acceleration. This performance gap widens at higher acceleration ratios ( 7.1x), where HiCache maintains a strong FID score while others degrade significantly, further confirming its robustness in high-acceleration scenarios.

### 4.5 Super-Resolution

On the image super-resolution task with a patch-based Inf-DiT model (Yang et al. 2024), HiCache achieves a significant theoretical ( 5.93×) and wall-clock ( 2.43×) speedup while maintaining near-lossless quality. This task follows the NTIRE 2025 challenge protocol (Chen et al. 2025). As shown in Figure [9](https://arxiv.org/html/2508.16984v1#Sx4.F9), both PSNR and SSIM consistently outperform TaylorSeer, with particularly notable improvements in SSIM. This demonstrates superior stability and quality preservation at high acceleration (detailed results in Appendix A.5).

Figure: Figure 9: Performance comparison of HiCache and TaylorSeer on image super-resolution across different acceleration intervals. HiCache consistently outperforms TaylorSeer in PSNR and SSIM, demonstrating superior stability and quality preservation at higher acceleration ratios.
Refer to caption: figures/ntire_results_comparison.png

### 4.6 Ablation Study

To validate the necessity of the contraction factor $\sigma$ introduced in Section 3.3.2, we conduct an ablation study *using the same experimental settings as Section 4.2*. As shown in Table [4](https://arxiv.org/html/2508.16984v1#Sx4.T4), the results confirm that the contraction mechanism is essential for the method’s viability, with the optimal range representing a careful balance between maintaining the oscillatory properties of Hermite basis functions and ensuring numerical stability, validating our theoretical design that addresses both prediction domain constraints and exponential coefficient growth suppression.

**Table 4: Ablation study demonstrating the necessity of the contraction factor $\sigma$ on FLUX for text-to-image generation.**
| Method | FLOPs Speed $\uparrow$ | Image Reward $\uparrow$ | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ |
| --- | --- | --- | --- | --- | --- |
| HiCache ($N=7,\sigma$=0.4) | 5.55$\times$ | 0.968 | 29.06 | 0.661 | 0.391 |
| HiCache ($N=7,\sigma$=0.5) | 5.55$\times$ | 0.998 | 28.94 | 0.657 | 0.398 |
| HiCache ($N=7,\sigma$=0.7) | 5.55$\times$ | 0.962 | 28.65 | 0.627 | 0.448 |
| HiCache ($N=7,\sigma$=1.0) | 5.55$\times$ | 0.759 | 28.09 | 0.368 | 0.721 |
| TaylorSeer ($N=7$) | 5.55$\times$ | 0.9572 | 28.63 | 0.6237 | 0.4520 |
| FORA ($N=7$) | 5.55$\times$ | 0.7418 | 28.32 | 0.5870 | 0.5409 |
| ToCa ($N=13,N=95\%$) | 5.77$\times$ | 0.7155 | 28.58 | 0.5677 | 0.5500 |
| DuCa ($N=12,N=95\%$) | 6.13$\times$ | 0.8382 | 28.95 | 0.5957 | 0.4935 |

## 5. Discussion

##### Empirical Validation of Observation 1

The foundation of HiCache rests on Observation [1](https://arxiv.org/html/2508.16984v1#Thmobservation1), which claims that feature derivative approximations follow a multivariate Gaussian distribution. We validated this using Energy tests (Székely and Rizzo 2005) across 1st to 5th order feature differences in five key FLUX model (Esser et al. 2024) modules. The tests confirmed Gaussianity with maximum confidence (p-value = 1.0) across all 25 configurations, providing strong empirical support for our theoretical framework and justifying Corollary [1](https://arxiv.org/html/2508.16984v1#Thmcorollary1) that Hermite polynomials form the optimal basis.

##### Prediction Simulation Framework

To fairly evaluate HiCache against Taylor expansion, we established a simulation framework using real feature trajectories from the FLUX model with activation interval $N_{\text{interval}}=6$, where derivative approximations are computed using sparsely sampled data from previous activation steps.
Our simulation eliminates cumulative error by initializing each prediction task with ground-truth historical data. This design isolates the analysis from cascading inaccuracies and enables meaningful quantitative comparison of the basis functions’ intrinsic approximation capabilities, ensuring direct evaluation of performance differences between Hermite and Taylor bases.

##### Empirical Verification of Oscillatory Advantages

Figure [10](https://arxiv.org/html/2508.16984v1#Sx5.F10) provides empirical evidence for the theoretical advantages of Hermite polynomials’ oscillatory properties (discussed in Section 3.3.2). Using the non-cumulative error framework on representative FLUX features, we observe that while both methods perform similarly at order 1, Taylor expansion increasingly over-extrapolates at higher orders, especially at trajectory turning points—precisely confirming Proposition [1](https://arxiv.org/html/2508.16984v1#Thmproposition1). HiCache’s oscillatory Hermite basis, stabilized by the contraction factor $\sigma$ (Definition [2](https://arxiv.org/html/2508.16984v1#Thmdefinition2)), maintains accurate fitting across all orders.

Figure: Figure 10: Visualization example of trajectory prediction for different polynomial orders under FLUX architecture. HiCache demonstrates more stable fitting at trajectory turning points, while Taylor expansion over-extrapolates.
Refer to caption: figures/prediction_comparison/prediction_comparison_unified_patch_7_layer28_img.png

##### Quantitative Validation of Proposition 3

To empirically verify the tighter error bounds promised by Proposition [3](https://arxiv.org/html/2508.16984v1#Thmproposition3), we quantitatively compared HiCache against Taylor expansion using the simulation framework. We define the relative error ratio as $R=\nicefrac{{\text{Taylor MSE}}}{{\text{Hermite MSE}}}$ (where $R>1$ indicates superior HiCache performance) and analyzed 25 configurations (5 FLUX modules × 5 polynomial orders).

Figure [11](https://arxiv.org/html/2508.16984v1#Sx5.F11) presents the cumulative error ratios:

Figure: Figure 11: Cumulative error ratios (Taylor MSE / HiCache MSE) across 5 FLUX modules as prediction step size varies. HiCache consistently outperforms TaylorSeer (all ratios $>1.0$), with the advantage becoming more pronounced at larger prediction steps and higher polynomial orders.
Refer to caption: figures/relative_error_simulation/timestep_analysis_all_layers_full_row.png

The results empirically confirm HiCache’s consistent advantage ($R>1.0$ across all modules), with the performance gap widening for larger prediction steps and higher polynomial orders. This validates our theoretical analysis: the $O(1/\sqrt{(N+1)!})$ convergence rate from Proposition [3](https://arxiv.org/html/2508.16984v1#Thmproposition3) indeed provides tighter bounds than Taylor’s $O(1/(N+1)!)$, with the advantage becoming more pronounced as $N$ increases—precisely as predicted by Corollary [2](https://arxiv.org/html/2508.16984v1#Thmcorollary2).

## 6. Conclusion

We presented HiCache, a training-free acceleration framework for diffusion models that fundamentally improves feature prediction by replacing Taylor series with scaled Hermite polynomials. Our approach is grounded in rigorous theory: Observation [1](https://arxiv.org/html/2508.16984v1#Thmobservation1) establishes the Gaussian nature of feature derivatives, Corollary [1](https://arxiv.org/html/2508.16984v1#Thmcorollary1) identifies Hermite polynomials as the optimal basis, and Proposition [3](https://arxiv.org/html/2508.16984v1#Thmproposition3) provides theoretical guarantees of superior convergence. The scaled Hermite basis (Definition [2](https://arxiv.org/html/2508.16984v1#Thmdefinition2)) with dual-scaling mechanism $\tilde{H}_{n}(x)=\sigma^{n}H_{n}(\sigma x)$ ensures numerical stability through a single hyperparameter, enabling HiCache to achieve superior performance across diverse tasks:  6.24× speedup on FLUX.1-dev while *exceeding* baseline quality, maintaining strong FID scores at high acceleration ratios, and pioneering cache-based acceleration for super-resolution. As a plug-and-play solution requiring no model retraining, HiCache establishes a new paradigm for diffusion acceleration by demonstrating that matching basis functions to intrinsic data statistics yields both theoretical and practical benefits.

## References

- Abramowitz and Stegun (1964)
Abramowitz, M.; and Stegun, I. A. 1964.
*Handbook of mathematical functions with formulas, graphs, and mathematical tables*, volume 55.
US Government printing office.
- Balaji et al. (2022)
Balaji, Y.; et al. 2022.
eDiff-I: Text-to-image diffusion models with an ensemble of expert denoisers.
*arXiv preprint arXiv:2211.01324*.
- Bolya and Hoffman (2023)
Bolya, D.; and Hoffman, J. 2023.
Token merging for fast stable diffusion.
In *CVPR Workshop on Efficient Deep Learning for Computer Vision*.
- Chen et al. (2023)
Chen, J.; et al. 2023.
Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis.
*arXiv preprint arXiv:2310.00426*.
- Chen et al. (2025)
Chen, Z.; Liu, K.; Gong, J.; Wang, J.; Sun, L.; Wu, Z.; Timofte, R.; Zhang, Y.; et al. 2025.
NTIRE 2025 Challenge on Image Super-Resolution (x4): Methods and Results.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops*.
- Dhariwal and Nichol (2021)
Dhariwal, P.; and Nichol, A. 2021.
Diffusion models beat gans on image synthesis.
*Advances in neural information processing systems*, 34: 8780–8794.
- Esser et al. (2024)
Esser, P.; et al. 2024.
Scaling rectified flow transformers for high-resolution image synthesis.
*arXiv preprint arXiv:2403.03206*.
- Fang et al. (2023)
Fang, G.; et al. 2023.
Structural pruning for diffusion models.
*Advances in Neural Information Processing Systems*, 36.
- Ho, Jain, and Abbeel (2020)
Ho, J.; Jain, A.; and Abbeel, P. 2020.
Denoising diffusion probabilistic models.
In *Advances in Neural Information Processing Systems*, volume 33, 6840–6851.
- Ho et al. (2022a)
Ho, J.; et al. 2022a.
Cascaded diffusion models for high fidelity image generation.
*The Journal of Machine Learning Research*, 23(1): 2249–2281.
- Ho et al. (2022b)
Ho, J.; et al. 2022b.
Video diffusion models.
*Advances in Neural Information Processing Systems*, 35: 8633–8646.
- Huang et al. (2024)
Huang, Z.; et al. 2024.
VBench: Comprehensive benchmark suite for video generative models.
*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 21807–21818.
- Karras et al. (2022)
Karras, T.; Aittala, M.; Aila, T.; and Laine, S. 2022.
Elucidating the design space of diffusion-based generative models.
*Advances in Neural Information Processing Systems*, 35: 26565–26577.
- Kong et al. (2024)
Kong, W.; et al. 2024.
HunyuanVideo: A Systematic Framework For Large Video Generative Models.
*arXiv preprint arXiv:2412.03603*.
- Li et al. (2023)
Li, X.; et al. 2023.
Q-Diffusion: Quantizing diffusion models.
*Proceedings of the IEEE/CVF International Conference on Computer Vision*, 17535–17545.
- Liu et al. (2025)
Liu, J.; et al. 2025.
TaylorSeer: Accelerating Diffusion Models via Taylor Expansion-based Feature Prediction.
*arXiv preprint arXiv:2501.00101*.
- Lu et al. (2022)
Lu, C.; et al. 2022.
DPM-Solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps.
*Advances in Neural Information Processing Systems*, 35: 5775–5787.
- Ma et al. (2023)
Ma, X.; et al. 2023.
DeepCache: Accelerating diffusion models for free.
*arXiv preprint arXiv:2312.00858*.
- Meng et al. (2023)
Meng, C.; et al. 2023.
On distillation of guided diffusion models.
*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 14297–14306.
- Narcowich, Ward, and Wendland (2005)
Narcowich, F. J.; Ward, J. D.; and Wendland, H. 2005.
Sobolev bounds on functions with scattered zeros, with applications to radial basis function surface fitting.
*Mathematics of computation*, 74(250): 743–763.
- Nichol and Dhariwal (2021)
Nichol, A. Q.; and Dhariwal, P. 2021.
Improved denoising diffusion probabilistic models.
In *International conference on machine learning*, 8162–8171. PMLR.
- Peebles and Xie (2023)
Peebles, W.; and Xie, S. 2023.
Scalable diffusion models with transformers.
In *Proceedings of the IEEE/CVF International Conference on Computer Vision*, 4195–4205.
- Podell et al. (2023)
Podell, D.; English, Z.; Lacey, K.; Blattmann, A.; Dockhorn, T.; Müller, J.; Penna, J.; and Rombach, R. 2023.
SDXL: Improving latent diffusion models for high-resolution image synthesis.
*arXiv preprint arXiv:2307.01952*.
- Ramesh et al. (2022)
Ramesh, A.; et al. 2022.
Hierarchical text-conditional image generation with clip latents.
*arXiv preprint arXiv:2204.06125*, 1(2): 3.
- Rombach et al. (2022)
Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; and Ommer, B. 2022.
High-resolution image synthesis with latent diffusion models.
10684–10695.
- Saharia et al. (2022)
Saharia, C.; et al. 2022.
Photorealistic text-to-image diffusion models with deep language understanding.
*Advances in neural information processing systems*, 35: 36479–36494.
- Salimans et al. (2016)
Salimans, T.; Goodfellow, I.; Zaremba, W.; Cheung, V.; Radford, A.; and Chen, X. 2016.
Improved techniques for training GANs.
*Advances in neural information processing systems*, 29.
- Selvaraju et al. (2024)
Selvaraju, R. R.; et al. 2024.
FORA: Fast-Forward Caching in Diffusion Transformer Acceleration.
*arXiv preprint arXiv:2407.01425*.
- Singer et al. (2022)
Singer, U.; et al. 2022.
Make-a-video: Text-to-video generation without text-video data.
*arXiv preprint arXiv:2209.14792*.
- Song, Meng, and Ermon (2021)
Song, J.; Meng, C.; and Ermon, S. 2021.
Denoising diffusion implicit models.
In *International Conference on Learning Representations*.
- Song et al. (2023)
Song, Y.; et al. 2023.
Consistency models.
*International Conference on Machine Learning*, 32211–32252.
- Szegö (1975)
Szegö, G. 1975.
*Orthogonal polynomials*, volume 23.
American Mathematical Society.
- Székely and Rizzo (2005)
Székely, G. J.; and Rizzo, M. L. 2005.
A new test for multivariate normality.
*Journal of Multivariate Analysis*, 93(1): 58–80.
- van der Westhuizen and Lasenby (2018)
van der Westhuizen, J.; and Lasenby, J. 2018.
The unreasonable effectiveness of the forget gate.
*arXiv preprint arXiv:1804.04849*.
- Vaswani et al. (2017)
Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, Ł.; and Polosukhin, I. 2017.
Attention is all you need.
In *Advances in neural information processing systems*, volume 30.
- Xu et al. (2023)
Xu, J.; et al. 2023.
ImageReward: Learning and evaluating human preferences for text-to-image generation.
*Advances in Neural Information Processing Systems*, 36.
- Yang et al. (2024)
Yang, Z.; et al. 2024.
Inf-DiT: Upsampling Any-Resolution Image with Memory-Efficient Diffusion Transformer.
*arXiv preprint arXiv:2405.04312*.
- Zhang, Rao, and Agrawala (2023)
Zhang, L.; Rao, A.; and Agrawala, M. 2023.
Adding conditional control to text-to-image diffusion models.
*Proceedings of the IEEE/CVF International Conference on Computer Vision*, 3836–3847.
- Zou et al. (2024)
Zou, H.; et al. 2024.
ToCa: Token-wise Feature Caching for Diffusion Model Acceleration.
*arXiv preprint arXiv:2410.05317*.

## A.1 Hermite Polynomials: Mathematical Foundations

We adopt the physicist’s Hermite polynomials $H_{n}(x)$, whose standard
differential form (Rodrigues formula) is:

$$ $H_{n}(x)=(-1)^{n}e^{x^{2}}\frac{d^{n}}{dx^{n}}e^{-x^{2}}$ (13) $$

The Gaussian kernel $e^{-x^{2}}$ in this formula directly embodies the
connection with Gaussian measure. From this differential form, we can derive
the recurrence relation:

$$ $H_{n+1}(x)=2xH_{n}(x)-2nH_{n-1}(x)$ (14) $$

where $H_{0}(x)=1$ and $H_{1}(x)=2x$. The first few polynomials are:

$$ $\displaystyle H_{0}(x)$ $\displaystyle=1$ $\displaystyle H_{1}(x)$ $\displaystyle=2x$ $\displaystyle H_{2}(x)$ $\displaystyle=4x^{2}-2$ (15) $\displaystyle H_{3}(x)$ $\displaystyle=8x^{3}-12x$ $\displaystyle H_{4}(x)$ $\displaystyle=16x^{4}-48x^{2}+12$ $$

## A.2 Detailed Algorithm Description

HiCache algorithm operates through two main phases. Central to this is the
concept of the derivative approximation, $\Delta^{k}\mathcal{F}$, which
approximates the $k$-th order derivative of a feature $\mathcal{F}$ at a given
timestep. As defined in Eq. ([3](https://arxiv.org/html/2508.16984v1#Sx3.E3)), it is computed
recursively from lower-order approximations at previous activation steps.

Phase 1: Cache Update (at activation steps $t_{i}$ where
$t_{i}\mod N_{\text{interval}}=0$)

This phase uses finite differences to compute high-order derivative
approximations and updates the cache. The interval between adjacent activation
steps is $\Delta t_{\text{hist}}=N_{\text{interval}}$.

Figure: Algorithm 2 Cache Update with Derivative Approximations

Phase 2: Feature Prediction (at non-activation steps $t$ where
$t\mod N_{\text{interval}}\neq 0$)

This phase performs efficient prediction based on cached derivative
approximations and scaled Hermite polynomials. The prediction step $\Delta t$
ranges from $[1,N_{\text{interval}}-1]$.

Figure: Algorithm 3 Feature Prediction

where Hermite polynomials are efficiently computed through the recurrence
relation:

- •
$H_{0}(x)=1$
- •
$H_{1}(x)=2x$
- •
$H_{k+1}(x)=2xH_{k}(x)-2kH_{k-1}(x)$ for $k\geq 1$

This algorithm design balances computational efficiency with prediction
accuracy. Key parameters include:

- •
Interval parameter $N_{\text{interval}}\in\mathbb{N}^{+}$: Controls
activation frequency, with larger values providing higher
acceleration ratio but potentially increased prediction error
- •
Contraction factor $\sigma\in(0,1)$: Stabilization parameter
that ensures numerical stability of Hermite polynomials within the prediction
interval
- •
Maximum order $N_{\text{order}}\in\mathbb{N}^{+}$: Expansion
order that balances approximation accuracy and computational complexity,
with practical implementations typically using
$N_{\text{order}}\leq 4$

## A.3 Complete Error Analysis and Theoretical Guarantees

This section provides rigorous proofs for the error bounds stated in Proposition 1 (Error Bounds for HiCache) in the main paper. We assume the feature evolution trajectory $f:[0,T]\to\mathcal{F}$ is sufficiently smooth with bounded derivatives up to order $N_{\text{order}}+1$.

The total prediction error decomposes into three components:

$$ $\mathbf{E}_{\text{total}}=\mathbf{E}_{\text{truncation}}+\mathbf{E}_{\text{approximation}}+\mathbf{E}_{\text{numerical}}$ (16) $$

### A.3.1 Truncation Error Analysis

Based on the Karhunen-Loève theorem and validated Gaussian properties, the feature function $f(s)$ can be expanded under the weight function $w(x)=e^{-(\sigma x)^{2}}$ as:

$$ $f(s_{\text{last}}+\Delta s)=\sum_{k=0}^{\infty}a_{k}\frac{\tilde{H}_{k}(\Delta s)}{\sqrt{2^{k}k!\sqrt{\pi}}}$ (17) $$

where the coefficients $a_{k}=\langle f,\tilde{H}_{k}\rangle_{w}$ possess rapid decay properties.

###### Lemma 1 (Truncation Error Bound) .

The truncation error from using order $N_{\text{order}}$ satisfies:

$$ $\mathbf{E}_{\text{truncation}}\leq C_{2}\frac{(\sigma\sqrt{2}|\Delta s|)^{N_{\text{order}}+1}}{\sqrt{(N_{\text{order}}+1)!}}\exp\left(\frac{(\sigma\Delta s)^{2}}{2}\right)$ (18) $$

###### Proof.

Using the asymptotic bound $|H_{k}(x)|\leq C_{1}\sqrt{2^{k}k!}e^{x^{2}/2}$ and the scaling properties of $\tilde{H}_{k}$:

$$ $\displaystyle\mathbf{E}_{\text{truncation}}$ $\displaystyle=\left|\sum_{k=N_{\text{order}}+1}^{\infty}\frac{1}{k!}\tilde{H}_{k}(\Delta s)\,\Delta^{k}f(s_{\text{last}})\right|$ (19) $\displaystyle\leq\sum_{k=N_{\text{order}}+1}^{\infty}\frac{\sigma^{k}|H_{k}(\sigma\Delta s)|}{k!}\cdot|\Delta^{k}f|$ (20) $\displaystyle\leq C_{2}\frac{(\sigma\sqrt{2}|\Delta s|)^{N_{\text{order}}+1}}{\sqrt{(N_{\text{order}}+1)!}}\exp\left(\frac{(\sigma\Delta s)^{2}}{2}\right)$ (21) $$

The contraction factor $\sigma<1$ ensures convergence.
∎

###### Remark 1 (Comparison with Taylor Expansion) .

The Taylor remainder $\mathbf{E}_{\text{truncation}}^{\text{Taylor}}=O((\Delta s)^{N+1}/(N+1)!)$ lacks the $\sqrt{(N+1)!}$ denominator and $\sigma^{N+1}$ suppression factor, resulting in looser bounds for non-monotonic trajectories.

###### Lemma 1 (Truncation Error Bound) .

###### Proof.

###### Remark 1 (Comparison with Taylor Expansion) .

### A.3.2 Finite Difference Approximation Error

###### Lemma 2 (Finite Difference Approximation Error) .

For a sufficiently smooth function $f\in C^{k+1}[0,T]$, the backward difference operator $\Delta^{(k)}$ satisfies:

$$ $\|\Delta^{k}f(t)-f^{(k)}(t)\|=\frac{\Delta t_{\text{hist}}}{k+1}\|f^{(k+1)}(\xi)\|+O(\Delta t_{\text{hist}}^{2})$ (22) $$

for some $\xi\in[t-k\Delta t_{\text{hist}},t]$.

###### Lemma 3 (Gaussian Decay in Feature Dynamics) .

Under the Gaussian feature dynamics (Observation [1](https://arxiv.org/html/2508.16984v1#Thmobservation1)), the expected norm of derivative approximations satisfies:

$$ $\mathbb{E}[\|\Delta^{k}\mathbf{F}\|^{2}]=\frac{\sigma_{\mathbf{F}}^{2}}{\Gamma(k/2+1)}\cdot(1+O(k^{-1}))$ (23) $$

where $\sigma_{\mathbf{F}}^{2}$ is the feature variance and $\Gamma$ is the gamma function.

###### Lemma 4 (Total Approximation Error) .

The cumulative finite difference error in HiCache satisfies:

$$ $\mathbf{E}_{\text{approximation}}\leq C_{4}\Delta t_{\text{hist}}\sqrt{N_{\text{order}}}$ (24) $$

###### Proof.

Combining Lemma [2](https://arxiv.org/html/2508.16984v1#Thmlemma2) and Lemma [3](https://arxiv.org/html/2508.16984v1#Thmlemma3):

$$ $\displaystyle\mathbf{E}_{\text{approximation}}$ $\displaystyle=\sum_{k=1}^{N_{\text{order}}}\mathbb{E}\left[\|\Delta^{k}f-f^{(k)}\|\cdot\frac{|\tilde{H}_{k}(\Delta s)|}{k!}\right]$ (25) $\displaystyle\leq\sum_{k=1}^{N_{\text{order}}}\frac{C_{3}\Delta t_{\text{hist}}}{k+1}\cdot\frac{\sigma^{k}}{\sqrt{\Gamma(k/2+1)}}$ (26) $\displaystyle\leq C_{4}\Delta t_{\text{hist}}\sum_{k=1}^{N_{\text{order}}}\frac{1}{k^{3/2}}$ (27) $\displaystyle\leq C_{4}\Delta t_{\text{hist}}\sqrt{N_{\text{order}}}$ (28) $$

The rapid decay from both the Gaussian property and factorial normalization ensures convergence.
∎

###### Lemma 2 (Finite Difference Approximation Error) .

###### Lemma 3 (Gaussian Decay in Feature Dynamics) .

###### Lemma 4 (Total Approximation Error) .

###### Proof.

### A.3.3 Numerical Stability Analysis

The recurrence relation $H_{k+1}(x)=2xH_{k}(x)-2kH_{k-1}(x)$ becomes numerically unstable when $|x|\gg 1$. The contraction factor $\sigma$ limits the effective computation domain to $|\sigma\Delta s|<2$, maintaining condition number $\kappa\approx O(1)$.

Due to the validated Gaussian properties of feature differences, Hermite coefficients possess optimal decay properties:

$$ $\text{Var}(c_{k})\propto\frac{1}{k!},\quad\text{Cov}(c_{i},c_{j})=0\text{ for }i\neq j$ (29) $$

###### Lemma 5 (Numerical Error Bound) .

The floating-point error in HiCache is bounded by:

$$ $\mathbf{E}_{\text{numerical}}\leq C_{5}\epsilon_{\text{machine}}\sqrt{\sum_{k=0}^{N_{\text{order}}}\frac{\sigma^{2k}}{k!}}\leq C_{5}\epsilon_{\text{machine}}e^{\sigma^{2}/2}$ (30) $$

###### Lemma 5 (Numerical Error Bound) .

### A.3.4 Total Error Bound and Performance Advantages

###### Theorem 1 (Main Error Theorem - Complete Proof of Proposition 1 in the main paper) .

Combining Lemma [1](https://arxiv.org/html/2508.16984v1#Thmlemma1), Lemma [4](https://arxiv.org/html/2508.16984v1#Thmlemma4), and Lemma [5](https://arxiv.org/html/2508.16984v1#Thmlemma5), the total HiCache prediction error satisfies:

$$ $\displaystyle\|\mathbf{E}_{\text{total}}\|$ $\displaystyle\leq\underbrace{C_{2}\frac{(\sigma\sqrt{2}|\Delta s|)^{N_{\text{order}}+1}}{\sqrt{(N_{\text{order}}+1)!}}e^{(\sigma\Delta s)^{2}/2}}_{\text{Truncation}}$ $\displaystyle\quad+\underbrace{C_{4}\Delta t_{\text{hist}}\sqrt{N_{\text{order}}}}_{\text{Approximation}}+\underbrace{C_{5}\epsilon_{\text{machine}}}_{\text{Numerical}}$ (31) $$

###### Corollary 3 (Comparative Advantage) .

HiCache achieves superior error bounds compared to Taylor expansion through:

- 1.
Faster convergence: $O(1/\sqrt{(N+1)!})$ vs. $O(1/(N+1)!)$
- 2.
Exponential suppression: Factor $\sigma^{N+1}$ with $\sigma<1$
- 3.
Adaptive basis: Hermite polynomials naturally match Gaussian feature statistics

###### Proof of Superiority.

The key advantage stems from the denominator $\sqrt{(N+1)!}$ in the Hermite truncation error, which grows much faster than $(N+1)!$ in the Taylor case. Combined with the $\sigma^{N+1}$ suppression factor, this yields exponentially tighter bounds for large $N$.
∎

###### Theorem 1 (Main Error Theorem - Complete Proof of Proposition 1 in the main paper) .

###### Corollary 3 (Comparative Advantage) .

###### Proof of Superiority.

## A.4 Detailed Experimental Setup

This section provides comprehensive technical details for all experimental configurations mentioned in Section 4.1.

### A.4.1 Model and Task Specifications

##### Text-to-Image Generation (FLUX.1-dev):

We employ the FLUX.1-dev model for high-resolution text-to-image synthesis. Images are generated at 1024×1024 resolution using 1,632 high-quality prompts sourced from the DrawBench benchmark. Quality assessment is performed using ImageReward, a robust perceptual metric for text-to-image alignment.

##### Text-to-Video Generation (Hunyuan-DiT):

The Hunyuan-DiT model is used for temporal content generation. We generate 4,730 video clips from 946 diverse prompts, evaluating performance using the comprehensive VBench framework which assesses multiple aspects including temporal consistency, object persistence, and text-video alignment.

##### Class-Conditional Image Generation (DiT-XL/2):

Following standard academic protocols, we conduct experiments on ImageNet using the DiT-XL/2 architecture. We generate 50,000 samples across all ImageNet classes and report FID-50k, sFID, and Inception Score for comprehensive quality assessment.

##### Image Super-Resolution (Inf-DiT):

We employ a modified patch-based Inf-DiT model for super-resolution tasks. Evaluation is conducted on the DIV8K dataset using standard PSNR and SSIM metrics to assess restoration quality and perceptual fidelity.

### A.4.2 Hardware and Computational Resources

All experiments are conducted on enterprise-grade GPU infrastructure:

- •
FLUX.1-dev experiments: NVIDIA H800 GPU
- •
DiT-XL/2 experiments: NVIDIA A800 GPU
- •
Hunyuan-DiT experiments: NVIDIA H100 GPU
- •
Inf-DiT super-resolution: NVIDIA A100 GPU

### A.4.3 Evaluation Protocols

Each task employs task-specific evaluation methodologies:

- •
Text-to-Image: ImageReward scores for semantic alignment and image quality
- •
Text-to-Video: VBench comprehensive evaluation including temporal consistency and motion quality
- •
Class-Conditional: Standard ImageNet metrics (FID-50k, sFID, Inception Score)
- •
Super-Resolution: Pixel-level metrics (PSNR, SSIM) for restoration accuracy

## A.5 Super-Resolution Task: Detailed Experimental Results

This section provides comprehensive experimental results for HiCache’s evaluation on image super-resolution using the patch-based Inf-DiT model. The experiments were conducted using standard evaluation metrics including PSNR and SSIM for restoration quality assessment.

Table [5](https://arxiv.org/html/2508.16984v1#Sx11.T5) presents detailed quantitative results across different acceleration intervals, comparing HiCache with TaylorSeer across multiple evaluation metrics including PSNR and SSIM.

**Table 5: Performance Comparison of Different Guiders on Image Super-Resolution Task**
| Method | Interval | Latency<br>(s) | Time<br>Speedup | FLOPs<br>(G) | FLOPs<br>Speedup | Restoration Track |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| PSNR$\uparrow$ | SSIM$\uparrow$ |  |  |  |  |  |  |
| HiCache | 1 | 256.3 | 1.00$\times$ | 12400 | 1.00$\times$ | 30.87 | 0.832 |
| 2 | 174.4 | 1.47$\times$ | 6639 | 1.87$\times$ | 30.92 | 0.835 |  |
| 3 | 141.6 | 1.81$\times$ | 4516 | 2.75$\times$ | 30.83 | 0.836 |  |
| 4 | 127.7 | 2.01$\times$ | 3606 | 3.44$\times$ | 30.80 | 0.838 |  |
| 5 | 118.6 | 2.16$\times$ | 3000 | 4.13$\times$ | 30.62 | 0.839 |  |
| 6 | 113.7 | 2.25$\times$ | 2696 | 4.60$\times$ | 30.32 | 0.831 |  |
| 7 | 108.9 | 2.35$\times$ | 2393 | 5.18$\times$ | 29.89 | 0.829 |  |
| 8 | 105.6 | 2.43$\times$ | 2090 | 5.93$\times$ | 30.18 (-0.69) | 0.820 (-0.012) |  |
| TaylorSeer | 1 | 259.5 | 1.00$\times$ | 12400 | 1.00$\times$ | 30.88 | 0.832 |
| 2 | 172.7 | 1.50$\times$ | 6639 | 1.87$\times$ | 30.90 | 0.833 |  |
| 3 | 139.3 | 1.86$\times$ | 4515 | 2.75$\times$ | 30.85 | 0.834 |  |
| 4 | 125.0 | 2.08$\times$ | 3606 | 3.44$\times$ | 30.80 | 0.831 |  |
| 5 | 115.9 | 2.24$\times$ | 2999 | 4.13$\times$ | 30.42 | 0.815 |  |
| 6 | 110.9 | 2.34$\times$ | 2696 | 4.60$\times$ | 29.95 | 0.796 |  |
| 7 | 105.9 | 2.45$\times$ | 2392 | 5.18$\times$ | 29.61 | 0.786 |  |
| 8 | 101.8 | 2.55$\times$ | 2089 | 5.94$\times$ | 29.71 (-1.17) | 0.799 (-0.033) |  |

The results demonstrate that HiCache maintains superior performance across all metrics, with particularly notable advantages at higher acceleration ratios. At interval=8, HiCache achieves a PSNR of 30.18 (only 0.69 drop from baseline), while TaylorSeer degrades to 29.71 (1.17 drop). This represents the first successful application of cache-based acceleration to super-resolution tasks, expanding the applicability of feature caching beyond traditional text-to-image and video generation.

## A.6 Theoretical Analysis of Gaussian Feature Dynamics and Hermite Optimality

This section provides theoretical analysis supporting the core claims in Proposition 2 (HiCache Feature Prediction) in the main paper. We examine both the empirical foundation and theoretical justification for using Hermite polynomials as the prediction basis.

### A.6.1 On the Gaussianity of Feature Derivatives (Observation 1 in the main paper)

While a complete theoretical proof of Gaussianity in neural network feature dynamics remains challenging, we provide several theoretical perspectives that support this empirical observation:

##### Central Limit Theorem Perspective

Consider the feature update through a transformer block:

$$ $\mathbf{F}_{t+1}=\mathbf{F}_{t}+\text{MHSA}(\mathbf{F}_{t})+\text{MLP}(\mathbf{F}_{t})$ (32) $$

The finite difference approximations aggregate many such updates:

$$ $\Delta^{k}\mathbf{F}_{t}=\frac{1}{\Delta t^{k}}\sum_{i=1}^{k}(-1)^{k-i}\binom{k}{i}\mathbf{F}_{t-i\Delta t}$ (33) $$

###### Proposition 4 (Asymptotic Gaussianity) .

Under mild regularity conditions, if the individual feature updates are weakly dependent with finite variance, the normalized finite differences converge in distribution to a Gaussian:

$$ $\frac{\Delta^{k}\mathbf{F}_{t}-\mathbb{E}[\Delta^{k}\mathbf{F}_{t}]}{\sqrt{\text{Var}[\Delta^{k}\mathbf{F}_{t}]}}\xrightarrow{d}\mathcal{N}(0,I)$ (34) $$

as the feature dimension $d\to\infty$.

Justification: This follows from the multivariate central limit theorem for weakly dependent random vectors. The transformer’s residual connections and normalization layers help maintain the required regularity conditions.

###### Proposition 4 (Asymptotic Gaussianity) .

##### Random Matrix Theory Perspective

The attention mechanism involves random-like weight matrices due to the softmax operation on query-key similarities:

$$ $\text{Attention}(Q,K,V)=\text{softmax}\left(\frac{QK^{T}}{\sqrt{d_{k}}}\right)V$ (35) $$

###### Lemma 6 (Gaussian Universality in Random Matrices) .

For large transformer models with random initialization and sufficient depth, the distribution of feature differences approaches a universal form that, under appropriate scaling, exhibits Gaussian characteristics in the bulk of the spectrum.

This is related to the Wigner semicircle law and its generalizations, which predict Gaussian-like behavior in high-dimensional random matrix ensembles.

###### Lemma 6 (Gaussian Universality in Random Matrices) .

##### Diffusion Process Approximation

In the continuous-time limit, the discrete diffusion process can be approximated by a stochastic differential equation:

$$ $d\mathbf{F}_{t}=\mu(\mathbf{F}_{t},t)dt+\sigma(\mathbf{F}_{t},t)dW_{t}$ (36) $$

where $W_{t}$ is a Wiener process. Under appropriate conditions on the drift $\mu$ and diffusion $\sigma$ coefficients, the marginal distributions of such processes exhibit Gaussian characteristics, especially for the increments.

### A.6.2 Hermite Polynomials as Optimal Basis (Corollary 1 in the main paper)

Given the empirically validated Gaussian nature, we now establish why Hermite polynomials form an optimal basis:

###### Theorem 2 (Karhunen-Loève Expansion for Gaussian Processes) .

Let $X(t)$ be a zero-mean Gaussian process with covariance kernel $K(s,t)$. If $K$ has the form:

$$ $K(s,t)=\exp\left(-\frac{(s-t)^{2}}{2\tau^{2}}\right)$ (37) $$

then the eigenfunctions of the covariance operator are Hermite functions, and the optimal $L^{2}$ expansion is:

$$ $X(t)=\sum_{n=0}^{\infty}\xi_{n}\psi_{n}(t)$ (38) $$

where $\psi_{n}$ are scaled Hermite functions and $\xi_{n}$ are uncorrelated Gaussian coefficients.

###### Proof Sketch.

The covariance operator $\mathcal{K}$ defined by:

$$ $(\mathcal{K}f)(t)=\int K(s,t)f(s)ds$ (39) $$

has Hermite functions as eigenfunctions when $K$ has Gaussian form. This follows from the fact that Hermite functions are eigenfunctions of the Fourier transform composed with multiplication by a Gaussian, which is essentially what the Gaussian kernel operator represents.
∎

###### Corollary 4 (Optimality for Feature Prediction) .

If feature differences $\Delta^{k}\mathbf{F}$ follow a multivariate Gaussian distribution, then the Hermite polynomial basis minimizes the expected squared prediction error:

$$ $\mathbb{E}\left[\|\mathbf{F}_{t-\Delta t}-\sum_{k=0}^{N}c_{k}\phi_{k}(\Delta t)\|^{2}\right]$ (40) $$

among all orthogonal polynomial bases $\{\phi_{k}\}$.

###### Proof Sketch.

By the Karhunen-Loève theorem, the optimal basis for representing a stochastic process is given by the eigenfunctions of its covariance operator. For Gaussian processes with appropriate covariance structure, these are Hermite functions. The optimality follows from the fact that this expansion minimizes the mean squared error for any finite truncation order $N$.
∎

###### Theorem 2 (Karhunen-Loève Expansion for Gaussian Processes) .

###### Proof Sketch.

###### Corollary 4 (Optimality for Feature Prediction) .

###### Proof Sketch.

### A.6.3 Robustness Under Approximate Gaussianity

Even if the Gaussianity assumption is only approximately satisfied, Hermite polynomials maintain advantages:

###### Proposition 5 (Robustness to Non-Gaussianity) .

Let $p(\mathbf{x})$ be a distribution with finite moments that can be expressed as:

$$ $p(\mathbf{x})=\phi(\mathbf{x})(1+\epsilon h(\mathbf{x}))$ (41) $$

where $\phi(\mathbf{x})$ is the Gaussian density, $|h(\mathbf{x})|\leq C$, and $\epsilon$ is small. Then the relative efficiency of Hermite basis compared to monomials degrades at most as $O(\epsilon)$.

This robustness property ensures that even under model misspecification, Hermite polynomials retain their advantages over standard polynomial bases.

###### Proposition 5 (Robustness to Non-Gaussianity) .

### A.6.4 Summary and Practical Implications

While we cannot provide a complete first-principles proof that neural network features must exhibit Gaussian dynamics, we have shown:

1. **Theoretical Support**: Multiple theoretical frameworks (CLT, random matrix theory, diffusion approximations) suggest Gaussianity is a reasonable expectation in high-dimensional neural networks.

2. **Conditional Optimality**: Given the empirically validated Gaussian nature, the optimality of Hermite polynomials follows rigorously from the Karhunen-Loève theorem.

3. **Robustness**: Even under approximate Gaussianity, Hermite bases maintain advantages over monomials due to their oscillatory nature and better numerical conditioning.

These theoretical insights, combined with our extensive empirical validation (Section 5.1), provide strong justification for the HiCache design choice of using Hermite polynomials as the prediction basis.

## A.7 Implementation Details

This section provides the complete Python implementation of the HiCache algorithm, including all core functions for derivative approximation, feature prediction, and cache management.

Figure: Listing 1: Core Implementation of HiCache Algorithm

The implementation includes several key components:

- •
Hermite Polynomial Computation (_hermite_polynomial): Recursive computation using the recurrence relation in Eq. ([14](https://arxiv.org/html/2508.16984v1#Sx7.E14))
- •
Feature Collection (_collect_trajectory_feature): Multi-layer, multi-module feature trajectory collection
- •
Derivative Approximation (derivative_approximation): Finite difference approximations as described in Algorithm 1
- •
Prediction Dispatcher (taylor_formula): Interface supporting both Taylor and Hermite prediction modes
- •
Hermite Prediction (_hermite_prediction_formula): Hermite polynomial-based feature prediction as described in Algorithm 2
- •
Taylor Prediction (_taylor_expansion_formula): Taylor expansion implementation

## Peer Reviews

confidence: 2
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. HiCache is evaluated only on diffusion-based language models. Could you clarify what aspects of the design make it incompatible with autoregressive LLMs? Would adapting it require major architectural changes?
2. The decay-based reuse predictor assumes gradual changes in attention importance across denoising steps. How well does it handle tasks where attention patterns change abruptly, such as in reasoning or code synthesis diffusion models?
3. Would it be possible to extend HiCache to multimodal diffusion transformers (e.g., text-to-image or video diffusion) where different modalities have separate attention structures?
soundness: 3
strengths: 1. The paper addresses a practical and underexplored efficiency bottleneck specific to diffusion-based language models, where repeated denoising iterations make KV caching more memory-intensive than in autoregressive models. This focus is well motivated and timely.
2. HiCache successfully adapts hierarchical caching concepts from systems design to the setting of diffusion language models. The integration into the diffusion pipeline is neat and minimally invasive, requiring no retraining or architecture modification.
3. The decay-based reuse predictor is computationally cheap and integrates naturally into diffusion processes, where temporal coherence of token or latent attention is strong. The simplicity makes it easy to deploy and tune. The plug-in nature of HiCache also makes it a practical engineering contribution. It could be incorporated into many diffusion-LM frameworks with minimal code changes.
summary: This paper presents HiCache, a hierarchical key-value (KV) caching framework aimed at accelerating large language model (LLM) inference, especially under long-context scenarios. Modern transformer-based models depend heavily on KV caches to store past hidden states, but as sequence length grows, the cache size quickly becomes a bottleneck for both memory and bandwidth.

HiCache introduces a two-level caching structure that separates tokens into “hot” and “cold” regions. The hot cache resides in fast GPU memory, while the cold cache is stored in slower memory (such as CPU DRAM or NVMe). A decay-based predictor estimates which tokens are likely to be reused soon, allowing the system to keep those in the hot cache and offload less active tokens. The method also includes an adaptive offloading policy that balances GPU memory usage with retrieval latency.

Experiments on models like LLaMA and OPT, evaluated on long-text generation and summarization tasks, show that HiCache can achieve up to a 2.3 times throughput speedup while cutting GPU memory usage roughly in half, with minimal loss in generation quality compared to full GPU caching.
weaknesses: Although HiCache is well designed and practically useful, its applicability is limited to diffusion-based language models. The caching and reuse patterns exploited here rely on the iterative refinement process of diffusion models, which differ substantially from autoregressive decoding. 

The evaluation focuses on throughput and memory reduction, but latency variance and system scalability are not thoroughly discussed. Diffusion inference involves synchronized denoising steps, so delayed cold-cache retrievals could lead to cumulative slowdowns. These potential risks are not well explored. Lastly, since diffusion-based LMs are still emerging, it would help to position HiCache more clearly within that research context and explain whether it generalizes to multimodal diffusion models (e.g., text-to-image diffusion transformers). 

As a reviewer, I personally am not very familiar with this part of LM. So if the weaknesses do not make sense, please explain it in your response.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 2
questions: The followings are the main questions:
1. Generality of Gaussian assumption: The core motivation for using Hermite polynomials relies on the assumption that feature derivatives in diffusion transformers follow Gaussian statistics. Have you tested this hypothesis across different architectures, such as U-Net–based diffusion models or non-transformer backbones? How consistent is the Gaussianity across layers, modalities, or timesteps?
2. Parameter sensitivity and scaling: How sensitive is HiCache’s performance to the choice of the contraction factor (σ) and expansion order (N)? Would an adaptive or learned scaling mechanism improve robustness across models and tasks?
3. Behavior under non-Gaussian or multimodal features: In real-world generative settings, feature distributions can become skewed or multimodal. How does HiCache perform when the Gaussian assumption breaks down? Are there fallback mechanisms or hybrid bases that could handle such cases?
4. Numerical stability in extreme accelerations: When acceleration ratios become very high (e.g., >8×), does the Hermite-based predictor remain stable, or do truncation errors accumulate? Could you quantify where the stability boundary occurs in practice?
soundness: 3
strengths: The key strengths of the paper lie in its strong theoretical foundation and practical effectiveness. HiCache introduces a principled improvement over Taylor-based caching by recognizing that diffusion transformer features evolve according to approximately Gaussian dynamics. By replacing Taylor’s monomial basis with scaled Hermite polynomials, which are theoretically optimal for Gaussian-correlated processes, the method provides a mathematically sound and statistically aligned framework for feature prediction. The addition of a dual-scaling mechanism further enhances numerical stability, allowing accurate high-order extrapolation without exploding coefficients or quality loss.

Another strength is the method’s generality and empirical robustness. HiCache is fully training-free and plug-and-play, requiring only a simple basis substitution in existing caching pipelines. It achieves consistent 5–6× acceleration across diverse tasks—text-to-image, text-to-video, class-conditional generation, and super-resolution—while often improving generation quality. These results demonstrate not only its theoretical elegance but also its strong practical value, combining mathematical rigor, stability, and real-world applicability in a single, lightweight framework.
summary: The paper introduces HiCache, a training-free acceleration method for Diffusion Transformers (DiTs) that replaces traditional Taylor-series extrapolation with a Hermite polynomial-based feature caching framework. Existing caching methods like TaylorSeer predict future diffusion features using monomial expansions, but they struggle with the non-monotonic and stochastic dynamics of diffusion features, leading to quality degradation at high acceleration ratios. HiCache addresses this by observing that feature derivatives in DiTs follow approximately Gaussian statistics and thus are better represented by scaled Hermite polynomials, which are the optimal orthogonal basis for Gaussian-correlated processes.

HiCache further introduces a dual-scaling mechanism that stabilizes numerical behavior by simultaneously contracting inputs and suppressing high-order coefficients. This enables accurate, stable extrapolation while preserving the plug-in simplicity of Taylor-based predictors. Experiments across text-to-image (FLUX.1-dev), text-to-video (HunyuanVideo), class-conditional (DiT-XL/2), and super-resolution (Inf-DiT) tasks show that HiCache achieves 5–6× acceleration and even slightly improves perceptual quality metrics such as ImageReward. The method integrates seamlessly into existing caching frameworks, offering a mathematically grounded, numerically stable, and empirically robust approach to accelerating diffusion models without retraining or architectural modifications.
weaknesses: The main weaknesses of the paper stem from its scope, assumptions, and evaluation coverage. HiCache is designed specifically for Diffusion Transformers (DiTs) and relies heavily on the assumption that feature derivatives follow Gaussian statistics. While this is empirically validated for certain architectures like FLUX, the assumption may not hold universally across other diffusion models, such as U-Net–based or multi-modal architectures. Likewise, the framework’s reliance on Hermite polynomials and the 2:4 Gaussian-correlated structure may limit its adaptability to more complex, non-Gaussian feature dynamics or models trained with nonstandard noise schedules.

Additionally, the experimental validation, though extensive, focuses primarily on large-scale diffusion models under controlled hardware and benchmark settings. The paper lacks detailed analysis of sensitivity to parameters such as the contraction factor σ and order N, which could affect stability in different model or task configurations. Finally, while the theoretical exposition is strong, the approach adds some conceptual and implementation complexity, and its practical impact beyond NVIDIA GPU environments or specific DiT variants remains to be demonstrated. Overall, the method is elegant and effective but somewhat specialized and assumption-dependent, leaving questions about its generality, robustness, and portability.

---

confidence: 5
contribution: 3
initial_rating: 6
presentation: 3
questions: It would be helpful if the authors could clarify whether HiCache is compatible with sparse- or efficient-attention variants of diffusion transformers.
soundness: 3
strengths: 1. The paper replaces Taylor’s monomial basis with Hermite polynomials derived from Gaussian feature correlations, leveraging Karhunen–Loeve optimality and a single scaling factor σ to improve stability and accuracy.
2. HiCache preserves almost the same implementation form as TaylorSeer, merely replacing the polynomial basis and adding a few scalar evaluations, thereby allowing direct integration into any feature caching–based acceleration framework with negligible computational overhead.
3. Extensive evaluations on text-to-image, text-to-video, and super-resolution tasks demonstrate the effectiveness of the proposed method in achieving substantial acceleration.
summary: This paper proposes HiCache, a training-free acceleration framework for diffusion transformers that replaces Taylor expansion with scaled Hermite polynomial-based feature caching, achieving more stable and accurate predictions across multiple generative tasks.
weaknesses: 1. The paper primarily relies on automated metrics such as PSNR, SSIM, LPIPS, and VBench. Incorporating human preference evaluations would make the assessment more convincing.
2. The paper heavily relies on the scaling factor σ to stabilize predictions, yet it lacks a principled rule or analysis on how to select or adapt σ across architectures, acceleration ratios, or polynomial orders.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: Please refer to the Weaknesses section.
soundness: 4
strengths: + Deep Insight into Trajectory Modeling: The paper identifies Taylor series’ limitation: its monotonic monomial basis fails to capture non-monotonic feature trajectories. In contrast, it links diffusion transformers’ Gaussian feature derivatives to Hermite polynomials and their oscillatory behavior, which accurately models non-monotonic dynamics.

+ Extensive Task Coverage: The paper evaluates HiCache across a broad range of generative tasks to demonstrate its generalizability. Extensive experiments are conducted on text-to-image, text-to-video, class-conditional image generation, and super-resolution tasks, demonstrating its superiority. It maintains strong performance across different acceleration ratios.

+ Effective Acceleration Design: The dual-scaling mechanism not only solves the numerical instability of Hermite polynomials for large extrapolation steps but also works standalone to improve TaylorSeer. HiCache’s plug-and-play nature—requiring only polynomial basis substitution without architectural changes—enables easy integration with existing cache-then-forecast pipelines, enhancing its practical value.
summary: This paper proposes HiCache, a training-free acceleration framework for diffusion transformers that addresses the quality loss issue of existing feature caching methods. This paper follows the "Cache-then-Forecast" paradigm. The core insight is that feature derivative approximations in diffusion transformers exhibit multivariate Gaussian characteristics, leading to the adoption of Hermite polynomials—optimal for Gaussian-correlated processes—as the prediction basis instead of the suboptimal monomial basis in Taylor series. HiCache also introduces a dual-scaling mechanism to ensure numerical stability and predictive accuracy.
weaknesses: - Modest Performance Advancement Over TaylorSeer: While the paper provides solid theoretical analysis for HiCache’s improvement over Taylor series-based methods, the practical performance gain over TaylorSeer is not prominently distinct in both qualitative visualizations and quantitative metrics. More compelling evidence—such or case studies is needed to further validate its superiority.

- Limited Variety of Experimental Models: The experimental evaluations focus on a relatively narrow set of diffusion transformers. For image generation, it primarily uses FLUX, with no tests on other widely adopted models like Stable Diffusion 3 or Qwen-Image. For video generation, only HunyuanVideo is included, without evaluations on other mainstream video diffusion models like Wan. Conducting experiments on these additional models would further strengthen the paper's validity.


Decision: Accept (Poster)