Model Already Knows the Best Noise: Bayesian Active Noise Selection via Attention in Video Diffusion Model

## Abstract

Abstract The choice of initial noise significantly affects the quality and prompt alignment of video diffusion models, where different noise seeds for the same prompt can lead to drastically different generations. While recent methods rely on externally designed priors such as frequency filters or inter-frame smoothing, they often overlook internal model signals that indicate which noise seeds are inherently preferable.
To address this, we propose ANSE ( A ctive N oise S election for G e neration), a model-aware framework that selects high-quality noise seeds by quantifying attention-based uncertainty. At its core is BANSA (Bayesian Active Noise Selection via Attention), an acquisition function that measures entropy disagreement across multiple stochastic attention samples to estimate model confidence and consistency.
For efficient inference-time deployment, we introduce a Bernoulli-masked approximation of BANSA that enables score estimation using a single diffusion step and a subset of attention layers. Experiments on CogVideoX-2B and 5B demonstrate that ANSE improves video quality and temporal coherence with only an 8% and 13% increase in inference time, respectively, providing a principled and generalizable approach to noise selection in video diffusion. See our project page: https://anse-project.github.io/anse-project/

# Model Already Knows the Best Noise: Bayesian Active Noise Selection via Attention in Video Diffusion Model

## 1 Introduction

Diffusion models have rapidly established themselves as a powerful class of generative models, demonstrating state-of-the-art performance across images and videos (3; 4; 5; 6; 7; 8; 9; 10; 11). In particular, Text-to-Video (T2V) diffusion models have received increasing attention for their ability to generate temporally coherent and visually rich video sequences. To achieve this, most T2V model architectures extend Text-to-Image (T2I) diffusion backbones by incorporating temporal modules or motion-aware attention layers (12; 13; 14; 15; 7; 16; 8). Furthermore, other works explore video generative structures, such as causal autoencoders or video autoencoder-based models, which aim to generate full video volumes rather than a sequence of independent frames (17; 9; 18; 19; 10; 11).

Beyond architectural design, another promising direction lies in improving noise initialization at inference time for T2I and T2V generation (20; 21; 22; 23). This aligns with the growing trend of inference-time scaling, observed not only in Large Language Models (24; 25) but also in diffusion-based generation systems (26). Due to the iterative nature of the diffusion process, the choice of initial noise profoundly influences video quality, temporal consistency, and prompt alignment (27; 1; 28; 2). As illustrated in Figure [1](https://arxiv.org/html/2505.17561v1#S0.F1), the same prompt can lead to drastically different videos depending solely on the noise seed, motivating the need for intelligent noise selection.

Several recent approaches attempt to address this by designing external noise priors. For example, PYoCo (27) introduces inter-frame dependent noise patterns to improve coherence, though it requires extensive fine-tuning. FreeNoise (28) reschedules noise across time using a fusion-based strategy, while FreeInit (1) applies frequency-domain filtering to preserve low-frequency components. FreqPrior (2) extends this idea via Gaussian-shaped frequency priors and partial sampling. While effective, these methods rely on externally designed priors and require multiple full diffusion passes to evaluate candidate seeds. More importantly, they fail to leverage internal signals within the model that indicate which noise seeds are inherently preferable.

To address this limitation, we propose a model-aware noise selection framework, ANSE (Active Noise Selection for Generation), grounded in Bayesian uncertainty. At the core of ANSE is BANSA (Bayesian Active Noise Selection via Attention), an acquisition function that identifies noise seeds inducing confident and consistent attention behaviors under stochastic perturbations. A conceptual comparison between our method and prior frequency-based approaches is illustrated in Figure [2](https://arxiv.org/html/2505.17561v1#S0.F2), highlighting the difference between external priors and model-informed uncertainty estimates.

Unlike BALD(29), which operates on classification logits, BANSA measures entropy in attention maps, arguably the most informative signals in generative diffusion. It compares the average entropy of individual maps to the entropy of their mean, capturing both uncertainty and disagreement across forward passes. A low BANSA score indicates that the model is more confident and certain in its attention, which empirically correlates with coherent video generation, as shown in Figure [1](https://arxiv.org/html/2505.17561v1#S0.F1).

To make this approach suitable for inference-time, we approximate BANSA using Bernoulli-masked attention, which enables multiple stochastic attention samples from a single forward pass. We further reduce computation by limiting BANSA evaluation to early denoising steps and a subset of informative attention layers, selected via correlation analysis. Our contributions are threefold:

- •
We present ANSE, the first active noise selection framework for video diffusion models, built on a principled Bayesian formulation of attention-based uncertainty.
- •
We introduce BANSA, a novel acquisition function that quantifies attention consistency under stochastic perturbations, enabling model-aware noise selection without retraining or external noise priors.
- •
Our method enhances both video quality and temporal consistency across various text-to-video architectures, with only a marginal increase in inference time of about 8% for CogVideoX-2B and 13% for CogVideoX-5B.

## 2 Preliminary

#### Video Diffusion Models

Diffusion models (30; 31) have achieved remarkable success across generative tasks. For T2V generation, directly operating in pixel space incurs high computational cost. To address this, video diffusion models (VDMs) typically adopt the latent diffusion model (LDM) framework, where the diffusion process is conducted in a compressed latent space.

A video autoencoder, composed of an encoder $\mathcal{E}$ and a decoder $\mathcal{D}$, is trained to reconstruct the original input video $\mathbf{x}$ such that $\mathbf{x}=\mathcal{D}(\mathcal{E}(\mathbf{x}))$. Denoting the latent code as $\mathbf{z}_{0}=\mathcal{E}(\mathbf{x})$, the forward diffusion process adds noise over time:

$$ $\mathbf{z}_{t}=\sqrt{\bar{\alpha}_{t}}\mathbf{z}_{0}+\sqrt{1-\bar{\alpha}_{t}} \,\boldsymbol{\epsilon},\quad\boldsymbol{\epsilon}\sim\mathcal{N}(0,\mathbf{I} ),\quad t=1,\dots,T,$ $$

where $\bar{\alpha}_{t}$ is a pre-defined variance schedule.
To learn the reverse process, a denoising network $\boldsymbol{\epsilon}_{\theta}$ is trained using the denoising score matching loss (32):

$$ $\mathcal{L}_{\theta}=\mathbb{E}_{\mathbf{z}_{t},\boldsymbol{\epsilon},t}\left[ \left\|\boldsymbol{\epsilon}_{\theta}(\mathbf{z}_{t},\mathbf{c},t)-\boldsymbol {\epsilon}\right\|^{2}\right],$ $$

where $\mathbf{c}$ denotes the conditioning text prompt. During sampling, the generation begins from Gaussian noise $\mathbf{z}_{T}\sim\mathcal{N}(0,\mathbf{I})$ and proceeds via a deterministic DDIM solver (33). The update at each step is computed as:

$$ $\mathbf{z}_{t-1}=\sqrt{\bar{\alpha}_{t}}\,\hat{\mathbf{z}}_{0}(t)+\sqrt{1-\bar {\alpha}_{t-1}}\,\boldsymbol{\epsilon}_{\theta}(\mathbf{z}_{t},\mathbf{c},t),$ $$

where the denoised latent estimate $\hat{\mathbf{z}}_{0}(t):=\frac{\mathbf{z}_{t}-\sqrt{1-\bar{\alpha}_{t}}\,
\boldsymbol{\epsilon}_{\theta}(\mathbf{z}_{t},\mathbf{c},t)}{\sqrt{\bar{\alpha
}_{t}}}$ is obtained using Tweedie’s formula (34; 35). This iterative process continues until $t=1$, yielding the final denoised latent $\mathbf{z}_{0}$, which is decoded into a video via $\mathcal{D}$.

#### Bayesian Active Learning by Disagreement (BALD)

Active Learning improves model performance by selecting the most informative samples from an unlabeled pool in training phase. Acquisition functions are typically categorized into uncertainty-based (36; 29; 37; 38) and distribution-based (39; 40; 41; 42) approaches, with some relying on external modules such as auxiliary predictors (38; 43; 44). While active learning has been predominantly applied to image classification tasks, in this work, we focus on adapting uncertainty-based methods to text-to-video generation, without requiring additional models.

Predictive entropy is a common uncertainty measure, but it captures only aleatoric uncertainty and fails to account for parameter uncertainty. BALD addresses this by quantifying epistemic uncertainty via the mutual information between predictions $\mathbf{y}$ and model parameters $\theta$:

$$ $\displaystyle\mathrm{BALD}(\mathbf{x})=\mathcal{H}[p(\mathbf{y}|\mathbf{x})]- \mathbb{E}_{p(\theta|\mathcal{D}_{U})}\left[\mathcal{H}[p(\mathbf{y}|\mathbf{x },\theta)]\right],$ (1) $$

where $\mathcal{H}[p]=-\sum_{y}p(y)\log p(y)$ is the Shannon entropy (45). The first term captures the entropy of the mean prediction, while the second term averages the entropy over stochastic forward passes. A high BALD score indicates confident but disagreeing predictions, revealing high epistemic uncertainty.

Since the posterior over $\theta$ is intractable, BALD is approximated using $K$ stochastic forward passes (e.g., Monte Carlo dropout):

$$ $\displaystyle\widehat{\mathrm{BALD}}(\mathbf{x})=\mathcal{H}\left[\frac{1}{K} \sum_{k=1}^{K}p^{(k)}(\mathbf{y}|\mathbf{x})\right]-\frac{1}{K}\sum_{k=1}^{K} \mathcal{H}\left[p^{(k)}(\mathbf{y}|\mathbf{x})\right].$ (2) $$

We reinterpret BALD for inference-time generative modeling. Rather than selecting samples for labeling, we apply BALD to rank noise seeds by their epistemic uncertainty. Selecting seeds with lower BALD scores results in more stable model behavior and leads to higher-quality generations.

Figure: Figure 3: Overview of our BANSA-based noise selection process. Given a text prompt $c$, we compute BANSA scores for multiple noise seeds $\{\mathbf{z}_{1},\dots,\mathbf{z}_{M}\}$ using Bernoulli-masked attention maps from selected layers at an early diffusion step. The seed with the lowest score, indicating confident and consistent attention, is selected for generation.
Refer to caption: extracted/6470240/main.jpg

## 3 Methods

We propose ANSE, a framework for selecting high-quality noise seeds in T2V diffusion models based on model uncertainty as shown in Figure [2](https://arxiv.org/html/2505.17561v1#S0.F2). ANSE is built upon an acquisition function called BANSA, which extends uncertainty-based criteria from classification tasks to the attention space of generative diffusion models (Section [3.1](https://arxiv.org/html/2505.17561v1#S3.SS1)). To enable efficient inference-time application, we approximate BANSA using Bernoulli-masked attention sampling (Section [3.2](https://arxiv.org/html/2505.17561v1#S3.SS2)). Furthermore, to reduce computational redundancy, we identify a representative attention layer using correlation-based linear probing (Section [3.3](https://arxiv.org/html/2505.17561v1#S3.SS3)). The overall pipeline is illustrated in Figure [3](https://arxiv.org/html/2505.17561v1#S2.F3).

### 3.1 BANSA: Bayesian Active Noise Selection via Attention

We introduce BANSA, an acquisition function for selecting optimal noise seeds in T2V diffusion models. Unlike classification tasks with explicit predictive distributions, diffusion models lack such outputs. We instead estimate uncertainty in the attention space, where alignment between text and visual tokens naturally emerges during generation. Here, attention maps are treated as stochastic predictions conditioned on the noise seed $\mathbf{z}$, prompt $c$, and diffusion timestep $t$. BANSA measures disagreement and confidence across multiple attention samples, capturing attention-level uncertainty analogous to BALD, but tailored to the generative setting.

###### Definition 1 (BANSA Score) .

Let $\mathbf{z}$ be a noise seed, $c$ a text prompt, and $t$ a diffusion timestep. Let $\mathbf{Q}(\mathbf{z},c,t),\mathbf{K}(\mathbf{z},c,t)\in\mathbb{R}^{N\times d}$ denote the query and key matrices from a denoising network $\epsilon_{\theta}$. The attention map is computed as:

$$ $A(\mathbf{z},c,t):=\texttt{Softmax}\left(\mathbf{Q}(\mathbf{z},c,t)\,\mathbf{K }(\mathbf{z},c,t)^{\top}\right)\in\mathbb{R}^{N\times N}.$ (3) $$

Let $\mathcal{A}(\mathbf{z},c,t)=\{A^{(1)},\dots,A^{(K)}\}$ denote a set of $K$ stochastic attention maps obtained via forward passes with random perturbations (e.g., Bernoulli masking). The BANSA score is defined as:

$$ $\text{BANSA}(\mathbf{z},c,t):=\frac{1}{K}\sum_{k=1}^{K}\mathcal{H}(A^{(k)})- \mathcal{H}\left(\frac{1}{K}\sum_{k=1}^{K}A^{(k)}\right),$ (4) $$

where the row-wise entropy of an attention map $A$ is given by:

$$ $\mathcal{H}(A):=\frac{1}{N}\sum_{i=1}^{N}\sum_{j=1}^{N}-A_{ij}\log A_{ij}.$ (5) $$

This formulation captures both the sharpness (confidence) and the consistency (agreement) of attention behavior. BANSA can be applied to various attention types (e.g., cross-, self-, or temporal) and allows layer-wise interpretability.

Given a noise pool $\mathcal{Z}=\{\mathbf{z}_{1},\dots,\mathbf{z}_{M}\}$, we select the optimal noise seed that minimizes the BANSA score:

$$ $\mathbf{z}^{*}:=\arg\min_{\mathbf{z}\in\mathcal{Z}}\text{BANSA}(\mathbf{z},c,t).$ (6) $$

A desirable property of BANSA is that its score becomes zero when all attention samples are identical, reflecting complete agreement and certainty. We formalize this as follows:

###### Proposition 1 (BANSA Zero Condition) .

Let $\mathcal{A}(\mathbf{z},c,t)=\{A^{(1)},\dots,A^{(K)}\}$ be a set of row-stochastic attention maps. Then:

$$ $\text{BANSA}(\mathbf{z},c,t)=0\quad\Leftrightarrow\quad A^{(1)}=\cdots=A^{(K)}.$ $$

The proof is deferred to the Appendix.
This condition implies that minimizing the BANSA score promotes attention behavior that is both confident and consistent under stochastic perturbations. Empirically, such attention patterns are associated with better prompt alignment, temporal coherence, and visual fidelity in generated videos. Therefore, BANSA serves as a principled criterion for model-aware noise selection in T2V.

###### Definition 1 (BANSA Score) .

###### Proposition 1 (BANSA Zero Condition) .

### 3.2 Stochastic Approximation of BANSA via Bernoulli-Masked Attention

While BANSA provides a principled objective for noise selection, its computation requires $K$ independent forward passes per noise seed $\mathbf{z}$, which is computationally expensive. To mitigate this cost, we propose a stochastic approximation using Bernoulli-masked attention, enabling multiple attention samples from a single pass.

Instead of computing stochastic attention maps from $K$ separate forward passes such as equipping dropout, we inject stochasticity directly into the attention computation by applying binary masks to the attention scores.
For each sample iteration $k=1,\cdots,K$, we generate a binary mask $m_{k}\in\{0,1\}^{N\times N}$ where each element is drawn i.i.d. from $\text{Bernoulli}(p)$. The masked attention map is computed as:

$$ $\hat{A}^{(k)}(\mathbf{z},c,t):=\texttt{Softmax}\left(\left(\mathbf{Q}(\mathbf{ z},c,t)\,\mathbf{K}(\mathbf{z},c,t)^{\top}\right)\odot m_{k}\right).$ (7) $$

where $\odot$ denotes element-wise multiplication. These masks simulate variability in attention patterns while keeping the input $(\mathbf{z},c,t)$ fixed. Using $K$ such samples, we define the approximate BANSA:

$$ ${\text{BANSA-E}}(\mathbf{z},c,t):=\frac{1}{K}\sum_{k=1}^{K}\mathcal{H}(\hat{A} ^{(k)}(\mathbf{z},c,t))-\mathcal{H}\left(\frac{1}{K}\sum_{k=1}^{K}\hat{A}^{(k) }(\mathbf{z},c,t)\right).$ (8) $$

Although BANSA-E may be biased due to the nonlinearity of entropy, it efficiently captures variation in attention and serves as a practical surrogate for uncertainty-based noise selection.(^††\dagger†^††\dagger†$\dagger$While BANSA is not derived from a formal Bayesian posterior, we use the term “Bayesian” in the spirit of epistemic uncertainty estimation, following the motivation behind BALD (36).) As shown in Table [1](https://arxiv.org/html/2505.17561v1#S4.T1), experimental validation confirms that this method is sufficient for selecting optimal noisy samples from the model’s perspective.

Figure: Algorithm 1 Active Noise Selection with BANSA Score for Video Generation

### 3.3 Layer Selection via Cumulative BANSA Correlation

BANSA can be computed at any attention layer, but attention behavior varies across depth. While using all layers provides a comprehensive uncertainty estimate, it is computationally heavy for deep T2V models. To address this, we propose a correlation-based truncation strategy that selects the smallest depth $d^{*}$ such that the averaged BANSA score over the first $d$ layers remains highly correlated with the full-layer score.

Given a noise seed $\mathbf{z}_{i}\in\mathcal{Z}=\{\mathbf{z}_{1},\dots,\mathbf{z}_{M}\}$ and $L$ attention layers, we compute per-layer scores ${\text{BANSA-E}}^{(l)}(\mathbf{z}_{i},c,t)$ and define the cumulative average up to layer $d$ as:

$$ $\displaystyle{\widehat{\text{BANSA-E}}}_{\leq d}(\mathbf{z}_{i},c,t):=\frac{1} {d}\sum_{l=1}^{d}{\text{BANSA-E}}^{(l)}(\mathbf{z}_{i},c,t).\vspace{-0.5em}$ (9) $$

To determine $d^{*}$, we compute the Pearson correlation (46) between ${\widehat{\text{BANSA-E}}}_{\leq d}$ and the full-layer average ${\widehat{\text{BANSA-E}}}_{\leq L}$, and select the smallest $d$ satisfying:

$$ $\text{Corr}\left({\widehat{\text{BANSA-E}}}_{\leq d},{\widehat{\text{BANSA-E}} }_{\leq L}\right)\geq\tau,$ (10) $$

with $\tau=0.7$ in our experiments. We validate this procedure using 100 prompts and 10 noise seeds across CogVideoX-2B, and CogVideoX-5B. As shown in Figure [5](https://arxiv.org/html/2505.17561v1#S5.F5), the correlation stabilizes at layer 14 in CogVideoX-2B, and 19 in CogVideoX-5B. We therefore set $d^{*}$ accordingly and define the BANSA score as ${\widehat{\text{BANSA-E}}}_{\leq d^{*}}$ to guide noise selection, as summarized in Algorithm [1](https://arxiv.org/html/2505.17561v1#algorithm1).

This layer selection procedure provides a lightweight and model-specific approximation of the full BANSA score. Since $d^{*}$ can be predefined for each model, it does not interfere with the noise sampling process and introduces no runtime cost during generation. As shown in the Appendix,
${\widehat{\text{BANSA-E}}}_{\leq d^{*}}$
closely approximates the full-layer score and achieves comparable generation quality across all models.

## 4 Experiments

Experimental Setting.
We evaluate ANSE on two representative text-to-video (T2V) diffusion models: CogVideoX-2B and CogVideoX-5B (9), chosen for their strong spatiotemporal modeling capabilities grounded in real-world dynamics. This setup enables rigorous evaluation of noise selection where attention and coherence are critical.
We follow the official DDIM sampling protocol with 50 denoising steps for both models.
Quantitative results for noise prior based approaches such as Freeinit (1) and FreqPrior (2) are omitted, as they are not officially supported on CogVideoX and incur 3$\times$ inference cost.
Nonetheless, ANSE is orthogonal and can be combined with these methods for further gains.
We use a noise pool of size $M{=}10$ with $K{=}10$ stochastic forward passes per noise, and apply Bernoulli-masked attention with a masking probability $p{=}0.2$.
Additional details are in the Appendix. All experiments are run on NVIDIA H100 GPUs.

Evaluation Metric.
To evaluate the impact of ANSE, we use VBench (47), a perceptually grounded benchmark for text-to-video generation.
VBench reports two high-level metrics—quality score and semantic score—which are combined into a total score via weighted averaging and normalized to a 0–100 scale.
Each score is a composite metric: the quality score is derived from 7 perceptual dimensions including *subject consistency*, *background consistency*, *temporal flickering*, *motion smoothness*, *dynamic degree*, *aesthetic quality*, and *imaging quality*;
the semantic score comprises 9 alignment-related criteria such as *object class*, *multiple objects*, *human action*, *color*, *spatial relationship*, *scene*, *temporal style*, *appearance style*, and *overall consistency*.
This decomposition ensures that VBench comprehensively assesses both visual fidelity and semantic alignment.
For each configuration (with and without BANSA), we generate 4,730 videos to ensure statistical reliability.

**Table 1: Quantitative results on VBench using CogVideoX-2B and -5B. ANSE consistently improves quality, semantic alignment, and total score. Scores for noise prior methods (1; 2) are omitted as they are not officially supported on CogVideoX and require costly re-implementation (3$\times$ inference), making large-scale evaluation impractical.**
| Model Backbone | Method | Quality Score | Semantic Score | Total Score | Inference Time |
| --- | --- | --- | --- | --- | --- |
| CogVideoX-2B (9) | Vanilla | 82.08 | 76.83 | 81.03 | 247.8 |
| + Ours | 82.56(+0.48) | 78.06(+1.23) | 81.66(+0.63) | 269.3(+8.68%) |  |
| CogVideoX-5B (9) | Vanilla | 82.53 | 77.50 | 81.52 | 1223.5 |
| + Ours | 82.70(+0.17) | 78.10(+0.60) | 81.71(+0.25) | 1392.1(+13.78%) |  |

Figure: Figure 4: Qualitative comparison of CogVideoX variants with and without ANSE. Results from CogVideoX-2B are shown in the first two rows; the rest show results from CogVideoX-5B. With ANSE, videos exhibit improved visual quality, better text alignment, and smoother motion transitions compared to the baseline.
Refer to caption: extracted/6470240/fig_second.png

Quantitative Comparison.
As shown in Table [1](https://arxiv.org/html/2505.17561v1#S4.T1), ANSE consistently improves performance across both CogVideoX models.
On CogVideoX-2B, the total VBench score increases from 81.03 (Vanilla) to 81.66 with ANSE, driven by gains in quality (+0.48) and semantic alignment (+1.23).
On the larger CogVideoX-5B, ANSE also improves all metrics: quality increases from 82.53 to 82.70 (+0.17), semantic score from 77.50 to 78.10 (+0.60), and total score from 81.52 to 81.72 (+0.25).
These results demonstrate that ANSE effectively enhances both perceptual fidelity and semantic alignment, even on large-scale, temporally grounded video diffusion models.
The consistent gains across both model scales highlight the robustness and generalizability of ANSE in selecting high-quality noise seeds under varying architectural complexities.

Qualitative Comparison.
Figure [4](https://arxiv.org/html/2505.17561v1#S4.F4) shows qualitative results on CogVideoX-2B and -5B with and without ANSE.
Our method improves semantic fidelity, motion portrayal, and visual clarity across diverse prompts.
For example, in "exploding" and ANSE captures key semantic transitions—generating visible explosions and preserving temporal continuity.
In "vacuuming", the subject remains static in the baseline but exhibits purposeful motion with ANSE.

On CogVideoX-5B, similar improvements are evident.
In "koala playing the piano" and "zebra running", ANSE generates anatomically coherent bodies with expressive motion.
These results demonstrate ANSE’s ability to enhance spatial-temporal fidelity and generalize to high-capacity video diffusion models.

Computational Cost.
As shown in Table [1](https://arxiv.org/html/2505.17561v1#S4.T1), ANSE increases inference time by only +8% on CogVideoX-2B and +13% on CogVideoX-5B, measured in denoising steps.
This overhead stems from noise seed evaluation but does not affect the sampling process, so memory usage remains unchanged.
In contrast, prior methods such as Freeinit and FreqPrior require three full sampling passes, resulting in a 200% increase in inference time.
While these methods have not been officially implemented, making direct comparisons challenging, ANSE reduces inference cost by approximately 64% while achieving comparable or superior generation quality.

## 5 Ablation Study and Analysis

**Table 2: Comparison of different acquisition functions for noise selection.**
| Method | Quality Score | Semantic Score | Total Score |
| --- | --- | --- | --- |
| Random | 82.08 | 76.83 | 81.03 |
| Entropy | 82.23 | 76.73 | 81.13 |
| BANSA (D) | 82.43 | 76.91 | 81.33 |
| BANSA (B) | 82.56 | 78.06 | 81.66 |

Table: Figure 5: Correlation analysis between cumulative BANSA score and full-layer scores. The 0.7 threshold is reached around layer 14 for CogVideoX-2B and layer 19 for CogVideoX-5B.

**Table 4: Quantitative comparison of reversed BANSA scoring on CogVideoX-2B. This presents results when selecting samples using the highest BANSA scores, compared to the default selection.**
| Method | Subject<br>Consistency | Background<br>Consistency | Temporal<br>Flickering | Motion<br>Smoothness | Aesthetic<br>Quality | Imaging<br>Quality | Dynamic<br>Degree | Quality<br>Score |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Vanilla | 0.9616 | 0.9788 | 0.9715 | 0.9743 | 0.6195 | 0.6267 | 0.6380 | 82.08 |
| + Ours (reverse) | 0.9626 | 0.9785 | 0.9700 | 0.9741 | 0.6181 | 0.6253 | 0.6328 | 81.93 |
| + Ours | 0.9641 | 0.9811 | 0.9775 | 0.9746 | 0.6202 | 0.6276 | 0.6511 | 82.56 |

Comparison of Acquisition Functions.
We compare BANSA with alternative acquisition strategies for noise seed selection using the CogVideoX-2B model.
As shown in Table [3](https://arxiv.org/html/2505.17561v1#S5.T3), we evaluate random sampling, entropy-based selection, and two BANSA variants: BANSA (B), which uses Bernoulli masking, and BANSA (D), which introduces Dropout-based stochasticity.
While all methods improve over the baseline, BANSA (B) consistently achieves the highest scores across quality, semantic, and total metrics.
This confirms that injecting uncertainty through Bernoulli masking is more effective than dropout, which is commonly used in Bayesian acquisition functions such as BALD.
The result highlights the importance of modeling attention-level uncertainty in a manner that reflects the structure of the underlying model.

Effect of Ensemble Size $K$.
We assess the impact of the number of stochastic forward passes $K$ on subject and background consistency, again using CogVideoX-2B.
As shown in Table [3](https://arxiv.org/html/2505.17561v1#S5.T3), both metrics improve steadily as $K$ increases from 1 to 10, suggesting that larger ensembles yield more robust and stable noise evaluations.
Performance saturates at $K=10$, which we adopt as the default throughout all experiments.

Effect of Noise Pool Size $M$.
We analyze the role of noise pool size $M$, which determines the diversity of candidate seeds assessed by BANSA.
While a larger $M$ increases the likelihood of discovering high-quality seeds, it also raises inference cost.
As shown in Figure [6](https://arxiv.org/html/2505.17561v1#S5.F6), performance saturates around $M=10$ for CogVideoX-2B and -5B.
We set these values as defaults for each model.

Reversing the BANSA Criterion.
To further validate BANSA, we conduct a control experiment where the noise seed with the *highest* BANSA score is selected—i.e., choosing the seed associated with the greatest model uncertainty.
As shown in Table [4](https://arxiv.org/html/2505.17561v1#S5.T4), this reversal results in degradation of quality-related metrics, confirming that lower BANSA scores are predictive of perceptually stronger generations and supporting the validity of our selection strategy.

Figure: Figure 7: Failure case and limitation of our method. Although the BANSA score indicates low uncertainty, the resulting video still contains unnatural content. This represent a limitation of ours: we select optimal seeds but do not alter the generation process itself.
Refer to caption: extracted/6470240/limit.jpg

## 6 Discussion and Limitations

Our method focuses on noise seed selection through model uncertainty estimation, yet it has notable limitations. As shown in Figure [7](https://arxiv.org/html/2505.17561v1#S5.F7), even seeds with low BANSA scores, which indicate high model confidence, can produce unnatural generations. This suggests that while ANSE effectively identifies promising initial seeds, it does not directly affect the generation process itself.
Moreover, there remains a gap between estimated uncertainty and perceptual quality. Although BANSA reliably captures attention-level uncertainty, it may not fully reflect semantic or aesthetic aspects. While generating multiple candidates per seed and selecting based on strong quality metrics would be ideal, this approach is computationally prohibitive. We thus view BANSA as a practical surrogate for such strategies.
Future work could further enhance performance by integrating it with information-theoretic refinement or active learning methods.

## 7 Conclusion

We present ANSE, a framework for active noise selection in video diffusion models. It is built around BANSA, an acquisition function that uses attention-derived uncertainty to identify noise seeds that promote confident and consistent attention, which are indicative of high-quality generation. BANSA adapts the BALD principle to the generative setting by operating in the attention space.
To enable efficient deployment, we introduce a stochastic approximation using Bernoulli-masked attention and a lightweight layer selection method. Experiments across multiple T2V backbones show that ANSE improves video quality and prompt alignment with little to no increase in inference time.
This work highlights the potential of inference-time noise selection guided by internal model signals. By extending active learning concepts beyond training-time data selection, ANSE enables model-aware decisions during inference and enhances generation without retraining. Our approach introduces a new inference-time scaling paradigm, where performance is improved not by modifying the model or increasing sampling steps, but through informed seed selection.

## References

- (1)
Tianxing Wu, Chenyang Si, Yuming Jiang, Ziqi Huang, and Ziwei Liu.
Freeinit: Bridging initialization gap in video diffusion models.
In European Conference on Computer Vision, pages 378–394. Springer, 2024.
- (2)
Yunlong Yuan, Yuanfan Guo, Chunwei Wang, Wei Zhang, Hang Xu, and Li Zhang.
Freqprior: Improving video diffusion models with frequency filtering gaussian noise.
arXiv preprint arXiv:2502.03496, 2025.
- (3)
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.
High-resolution image synthesis with latent diffusion models.
In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022.
- (4)
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al.
Scaling rectified flow transformers for high-resolution image synthesis.
In Forty-first international conference on machine learning, 2024.
- (5)
Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, et al.
Sana: Efficient high-resolution image synthesis with linear diffusion transformers.
arXiv preprint arXiv:2410.10629, 2024.
- (6)
Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li.
Pixart-$\sigma$: Weak-to-strong training of diffusion transformer for 4k text-to-image generation.
In European Conference on Computer Vision, pages 74–91. Springer, 2024.
- (7)
Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan.
Videocrafter2: Overcoming data limitations for high-quality video diffusion models.
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7310–7320, 2024.
- (8)
Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, et al.
Lavie: High-quality video generation with cascaded latent diffusion models.
International Journal of Computer Vision, 133(5):3059–3078, 2025.
- (9)
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al.
Cogvideox: Text-to-video diffusion models with an expert transformer.
arXiv preprint arXiv:2408.06072, 2024.
- (10)
Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You.
Open-sora: Democratizing efficient video production for all.
arXiv preprint arXiv:2412.20404, 2024.
- (11)
Wan Team.
Wan: Open and advanced large-scale video generative models.
2025.
- (12)
Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis.
Align your latents: High-resolution video synthesis with latent diffusion models.
In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22563–22575, 2023.
- (13)
Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen.
Latent video diffusion models for high-fidelity long video generation.
2022.
- (14)
Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang.
Modelscope text-to-video technical report.
arXiv preprint arXiv:2308.06571, 2023.
- (15)
Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, Chao Weng, and Ying Shan.
Videocrafter1: Open diffusion models for high-quality video generation, 2023.
- (16)
Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai.
Animatediff: Animate your personalized text-to-image diffusion models without specific tuning.
International Conference on Learning Representations, 2024.
- (17)
Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang.
Cogvideo: Large-scale pretraining for text-to-video generation via transformers.
arXiv preprint arXiv:2205.15868, 2022.
- (18)
Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, et al.
Ltx-video: Realtime video latent diffusion.
arXiv preprint arXiv:2501.00103, 2024.
- (19)
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al.
Hunyuanvideo: A systematic framework for large video generative models.
arXiv preprint arXiv:2412.03603, 2024.
- (20)
Xiefan Guo, Jinlin Liu, Miaomiao Cui, Jiankai Li, Hongyu Yang, and Di Huang.
Initno: Boosting text-to-image diffusion models via initial noise optimization.
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9380–9389, 2024.
- (21)
Luca Eyring, Shyamgopal Karthik, Karsten Roth, Alexey Dosovitskiy, and Zeynep Akata.
Reno: Enhancing one-step text-to-image models through reward-based noise optimization.
Advances in Neural Information Processing Systems, 37:125487–125519, 2024.
- (22)
Sherry X Chen, Yaron Vaxman, Elad Ben Baruch, David Asulin, Aviad Moreshet, Kuo-Chin Lien, Misha Sra, and Pradeep Sen.
Tino-edit: Timestep and noise optimization for robust diffusion-based image editing.
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6337–6346, 2024.
- (23)
Katherine Xu, Lingzhi Zhang, and Jianbo Shi.
Good seed makes a good crop: Discovering secret seeds in text-to-image diffusion models.
In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 3024–3034. IEEE, 2025.
- (24)
Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher Ré, and Azalia Mirhoseini.
Large language monkeys: Scaling inference compute with repeated sampling.
arXiv preprint arXiv:2407.21787, 2024.
- (25)
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar.
Scaling llm test-time compute optimally can be more effective than scaling model parameters.
arXiv preprint arXiv:2408.03314, 2024.
- (26)
Nanye Ma, Shangyuan Tong, Haolin Jia, Hexiang Hu, Yu-Chuan Su, Mingda Zhang, Xuan Yang, Yandong Li, Tommi Jaakkola, Xuhui Jia, et al.
Inference-time scaling for diffusion models beyond scaling denoising steps.
arXiv preprint arXiv:2501.09732, 2025.
- (27)
Songwei Ge, Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang, Ming-Yu Liu, and Yogesh Balaji.
Preserve your own correlation: A noise prior for video diffusion models.
In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22930–22941, 2023.
- (28)
Haonan Qiu, Menghan Xia, Yong Zhang, Yingqing He, Xintao Wang, Ying Shan, and Ziwei Liu.
Freenoise: Tuning-free longer video diffusion via noise rescheduling.
In The Twelfth International Conference on Learning Representations.
- (29)
Yarin Gal, Riashat Islam, and Zoubin Ghahramani.
Deep bayesian active learning with image data.
In International conference on machine learning, pages 1183–1192. PMLR, 2017.
- (30)
Jonathan Ho, Ajay Jain, and Pieter Abbeel.
Denoising diffusion probabilistic models.
Advances in neural information processing systems, 33:6840–6851, 2020.
- (31)
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole.
Score-based generative modeling through stochastic differential equations.
In ICLR, 2021.
- (32)
Pascal Vincent.
A connection between score matching and denoising autoencoders.
Neural computation, 23(7):1661–1674, 2011.
- (33)
Jiaming Song, Chenlin Meng, and Stefano Ermon.
Denoising diffusion implicit models.
In ICLR, 2021.
- (34)
Bradley Efron.
Tweedie’s formula and selection bias.
Journal of the American Statistical Association, 106(496):1602–1614, 2011.
- (35)
Kwanyoung Kim and Jong Chul Ye.
Noise2score: Tweedie’s approach to self-supervised image denoising without clean images.
Advances in Neural Information Processing Systems, 34:864–874, 2021.
- (36)
Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, and Máté Lengyel.
Bayesian active learning for classification and preference learning.
arXiv preprint arXiv:1112.5745, 2011.
- (37)
Andreas Kirsch, Joost Van Amersfoort, and Yarin Gal.
Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning.
Advances in neural information processing systems, 32, 2019.
- (38)
Donggeun Yoo and In So Kweon.
Learning loss for active learning.
In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 93–102, 2019.
- (39)
Oisin Mac Aodha, Neill DF Campbell, Jan Kautz, and Gabriel J Brostow.
Hierarchical subquery evaluation for active learning on a graph.
In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 564–571, 2014.
- (40)
Yi Yang, Zhigang Ma, Feiping Nie, Xiaojun Chang, and Alexander G Hauptmann.
Multi-class active learning by uncertainty sampling with diversity maximization.
International Journal of Computer Vision, 113:113–127, 2015.
- (41)
Ozan Sener and Silvio Savarese.
Active learning for convolutional neural networks: A core-set approach.
arXiv preprint arXiv:1708.00489, 2017.
- (42)
Samarth Sinha, Sayna Ebrahimi, and Trevor Darrell.
Variational adversarial active learning.
In Proceedings of the IEEE/CVF international conference on computer vision, pages 5972–5981, 2019.
- (43)
Toan Tran, Thanh-Toan Do, Ian Reid, and Gustavo Carneiro.
Bayesian generative active deep learning.
In International conference on machine learning, pages 6295–6304. PMLR, 2019.
- (44)
Kwanyoung Kim, Dongwon Park, Kwang In Kim, and Se Young Chun.
Task-aware variational adversarial active learning.
In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8166–8175, 2021.
- (45)
Claude E Shannon.
A mathematical theory of communication.
Bell system technical journal, 27(3):379–423, 1948.
- (46)
Karl Pearson.
Notes on regression and inheritance in the case of two parents.
Proceedings of the Royal Society of London, pages 337–344, 1895.
- (47)
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu.
VBench: Comprehensive benchmark suite for video generative models.
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.

## Appendix A Supplementary Section

In this supplementary document, we present the following:

- •
Implementation details of the BANSA score in Section [B](https://arxiv.org/html/2505.17561v1#A2).
- •
Proof of Proposition 1 from the main paper regarding the BANSA Zero condition in Section [C](https://arxiv.org/html/2505.17561v1#A3).
- •
Implementation details of ANSE and evaluation metrics in Section [D](https://arxiv.org/html/2505.17561v1#A4).
- •
Further explanation of layer selection through cumulative BANSA correlation in Section [E](https://arxiv.org/html/2505.17561v1#A5).
- •
Additional ablation studies including full-layer BANSA Score analysis in Section [F](https://arxiv.org/html/2505.17561v1#A6) and temporal scope effects in Section [G](https://arxiv.org/html/2505.17561v1#A7).
- •
Additional qualitative results demonstrating the impact of BANSA Score in Section [H](https://arxiv.org/html/2505.17561v1#A8).

## Appendix B Implementation Details of BANSA Score

#### Score Definition vs. Implementation.

In the main paper, we define the BANSA score as the difference between the mean of entropies and the entropy of the mean:

$$ $\text{BANSA}(\mathbf{z},c,t):=\frac{1}{K}\sum_{k=1}^{K}\mathcal{H}(A^{(k)})- \mathcal{H}\left(\frac{1}{K}\sum_{k=1}^{K}A^{(k)}\right),$ $$

This corresponds to the negative of the mutual information formulation used in BALD (29). But, for practical and interpretability reasons, we adopt the BALD-style computation in our implementation:

$$ $\text{BANSA}(\mathbf{z},c,t):=\mathcal{H}\left(\frac{1}{K}\sum_{k=1}^{K}A^{(k) }\right)-\frac{1}{K}\sum_{k=1}^{K}\mathcal{H}(A^{(k)}),$ $$

which yields a non-negative score. This implementation allows easier visualization and interpretation, where larger values correspond to higher disagreement (i.e., uncertainty) across stochastic attention maps.
We adopt this sign convention to align with standard BALD implementations, where mutual information is expressed as a non-negative measure of epistemic uncertainty.
All figures and tables in the paper (e.g., correlation plots and performance curves) reflect this convention, showing positive-valued BANSA scores.

#### Semantic consistency.

Although the mathematical sign differs between the definition and implementation, the semantic meaning and selection behavior are strictly preserved. In both cases, we minimize the BANSA score to select noise seeds with low attention uncertainty. The relative ordering of scores remains unchanged, and the interpretation of the score as a measure of model disagreement is fully retained.

For clarity, we note that the proof of Proposition [1](https://arxiv.org/html/2505.17561v1#Thmproposition1) in the following section is also presented under this BALD-style convention.

## Appendix C Proof of Proposition 1

###### Proposition 1 (BANSA Zero Condition) .

Let $\mathcal{A}(\mathbf{z},c,t)=\{A^{(1)},\dots,A^{(K)}\}$ be a set of row-stochastic attention maps. Then:

$$ $\text{BANSA}(\mathbf{z},c,t)=0\quad\Leftrightarrow\quad A^{(1)}=\cdots=A^{(K)}.$ $$

###### Proof.

BANSA is defined as the difference between the average entropy and the entropy of the average:

$$ $\text{BANSA}(\mathbf{z},c,t)=\mathcal{H}\left(\frac{1}{K}\sum_{k=1}^{K}A^{(k)} (\mathbf{z},c,t)\right)-\frac{1}{K}\sum_{k=1}^{K}\mathcal{H}(A^{(k)}(\mathbf{z },c,t)).$ $$

Since the Shannon entropy $\mathcal{H}(\cdot)$ is strictly concave over the probability simplex. Therefore, by Jensen’s inequality:

$$ $\mathcal{H}\left(\frac{1}{K}\sum_{k=1}^{K}A^{(k)}\right)\geq\frac{1}{K}\sum_{k =1}^{K}\mathcal{H}(A^{(k)}),$ $$

with equality if and only if $A^{(1)}=\cdots=A^{(K)}$. Thus, $\text{BANSA}(\mathbf{z},c,t)=0$ if and only if all attention maps are identical.
∎

###### Remark 1 .

(Interpretation)
This result confirms that the BANSA score quantifies disagreement among sampled attention maps. A BANSA score of zero occurs only when all stochastic attention realizations collapse to a single deterministic map—i.e., the model exhibits no epistemic uncertainty in its attention distribution. Higher BANSA values indicate greater variation across samples, and thus, higher uncertainty. In this sense, BANSA acts as a Jensen–Shannon-type divergence over attention maps, capturing their dispersion under stochastic masking.

###### Proposition 1 (BANSA Zero Condition) .

###### Proof.

###### Remark 1 .

## Appendix D Further Details on Evaluation Metrics and Implementation

#### Evaluation Metrics

To evaluate performance on Vbench, we use the Vbench-long version, where prompts are augmented using GPT-4o across all evaluation dimensions. This version is specifically designed for assessing videos longer than 4 seconds.

We rigorously evaluate our generated videos following the official evaluation protocol. The Quality Score is a weighted average of the following aspects: subject consistency, background consistency, temporal flickering, motion smoothness, aesthetic quality, imaging quality, and dynamic degree.

The Semantic Score is a weighted average of the following semantic dimensions: object class, multiple objects, human action, color, spatial relationship, scene, appearance style, temporal style, and overall consistency.

The Total Score is then computed as a weighted combination of the Quality Score and Semantic Score:

$$ $\text{Total Score}=\frac{w_{1}}{w_{1}+w_{2}}\cdot\text{Quality Score}+\frac{w_ {2}}{w_{1}+w_{2}}\cdot\text{Semantic Score}$ $$

where $w_{1}=4$ and $w_{2}=1$, following the default setting in the official implementation.

#### Implementation

As discussed in Section [B](https://arxiv.org/html/2505.17561v1#A2), we compute our BANSA score using the BALD-style formulation, which yields non-negative values. For clearer visualization, we normalize the BANSA scores from their original range (minimum: 0.45, maximum: 0.60) to the [0, 1] interval.
This normalization is used solely for visual clarity in figures and plots, and does not affect the noise selection process, which operates on the raw BANSA scores.

## Appendix E Further Detail of BANSA Layer-wise Correlation Analysis

#### Prompt construction.

We evenly sampled 100 prompts from the four official VBench categories: Subject Consistency, Overall Consistency, Temporal Flickering, and Scene. Each category contains 25 prompts, selected to ensure diversity in motion, structure, and semantics.

Below are representative examples:

- •
Subject Consistency (e.g., “A young man with long, flowing hair sits on a rustic wooden stool in a cozy room, strumming an acoustic guitar…”)
- •
Overall Consistency (e.g., “A mesmerizing splash of turquoise water erupts in extreme slow motion, each droplet suspended in mid-air…”)
- •
Temporal Flickering (e.g., “A cozy restaurant with flickering candles and soft music. Patrons dine peacefully as snow falls outside…”)
- •
Scene (e.g., “A university campus transitions from lively student life to a golden sunset behind the clock tower…”)

Prompt sampling was stratified to ensure coverage of diverse visual and temporal patterns. The full list of prompts will be made publicly available upon code release.

#### BANSA score computation and correlation analysis.

For each prompt, we generated 10 videos using different random noise seeds and computed BANSA scores at each attention layer. This yielded one full-layer BANSA score and a set of layer-wise scores per seed.

To obtain stable estimates, we averaged the per-layer and full-layer BANSA scores across the 10 seeds, reducing noise-specific variance and capturing consistent uncertainty patterns.

We then computed Pearson correlations between the cumulative BANSA scores (summed from layer 1 to $d$) and the official quality scores. The optimal depth $d^{*}$ was defined as the smallest $d$ at which the correlation exceeded 0.7. This procedure is visualized in Figure 5 of the main paper, and $d^{*}$ was applied consistently throughout all experiments.
This setup ensures that our correlation analysis reflects generalizable, noise-agnostic trends in attention-based uncertainty.

**Table 5: Comparison between full-layer and truncated BANSA score.**
| Backbone<br>Model | Method | Subject<br>Consistency $\uparrow$ | Background<br>Consistency $\uparrow$ | Temporal<br>Flickering $\uparrow$ | Motion<br>Smoothness $\uparrow$ | Aesthetic<br>Quality $\uparrow$ | Imaging<br>Quality $\uparrow$ | Dynamic<br>Degree $\uparrow$ | Quality<br>Score $\uparrow$ | Inference<br>Time $\downarrow$ |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| CogvideoX-2B | Full-layer | 0.9639 | 0.9810 | 0.9801 | 0.9743 | 0.6198 | 0.6244 | 0.6516 | 82.58 | 303.7 |
| Truncated | 0.9641 | 0.9811 | 0.9775 | 0.9746 | 0.6202 | 0.6276 | 0.6511 | 82.56 | 269.3 |  |
| CogvideoX-5B | Full-layer | 0.9660 | 0.9630 | 0.9863 | 0.9708 | 0.6168 | 0.6290 | 0.6979 | 82.71 | 1530.1 |
| Truncated | 0.9658 | 0̱.9639 | 0.9861 | 0.9711 | 0.6179 | 0.6290 | 0.6918 | 82.70 | 1392.3 |  |

## Appendix F Effectiveness of Truncated BANSA Score

To reduce the computational overhead of BANSA evaluation, we adopt a truncated score that aggregates attention uncertainty only up to a fixed depth $d^{*}$, rather than summing over all layers. To evaluate the effectiveness of this approximation, we compared the final generation quality when selecting noise seeds using either the full-layer or truncated BANSA scores.

As shown in Table [5](https://arxiv.org/html/2505.17561v1#A5.T5), both approaches yield highly similar results across all seven dimensions of the VBench evaluation protocol (subject consistency, background consistency, aesthetic quality, imaging quality, motion smoothness, dynamic degree, and temporal flickering). Importantly, the overall quality scores are preserved despite the substantial reduction in attention layers used.

This demonstrates that truncated BANSA is sufficient to capture the key uncertainty signals for reliable noise selection while reducing inference time. The strong alignment in quality stems from the fact that our method relies on relative ranking rather than absolute values, allowing for efficient yet robust selection with significantly lower computational cost. We attribute this effectiveness to the fact that most informative attention behaviors emerge early in the denoising process, allowing accurate uncertainty estimation without full-layer computation.

**Table 6: Effect of temporal scope in BANSA score on generation quality.**
| BANSA Scope | Subject<br>Consistency $\uparrow$ | Temporal<br>Flickering $\uparrow$ | Motion<br>Smoothness $\uparrow$ | Aesthetic<br>Quality $\uparrow$ | Imaging<br>Quality $\uparrow$ | Dynamic<br>Degree $\uparrow$ | Inference<br>Time $\downarrow$ |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 1-step | 0.9639 | 0.9801 | 0.9743 | 0.6198 | 0.6244 | 0.6516 | $\times$ 1 |
| 25-step avg | 0.9651 | 0.9798 | 0.9746 | 0.6202 | 0.6271 | 0.6511 | $\times$ 25 |
| 50-step avg | 0.9652 | 0.9799 | 0.9751 | 0.6203 | 0.6276 | 0.6514 | $\times$ 50 |

## Appendix G Effect of Temporal Scope in BANSA Score

While our method computes the BANSA score only at the first denoising step to minimize cost, it is natural to ask whether incorporating more timesteps improves its predictive power for noise selection. To investigate this, we compute the average BANSA score across the first 1, 25, and 50 denoising steps and compare their effectiveness in predicting video quality.

Table [6](https://arxiv.org/html/2505.17561v1#A6.T6) reports the VBench scores for subject consistency, aesthetic quality, imaging quality, motion smoothness, dynamic degree, and temporal flickering when using BANSA computed over different temporal scopes. Although using more timesteps results in slightly better quality, the gains are marginal. This indicates that most of the predictive signal for noise quality is embedded early in the generation trajectory.

More importantly, since BANSA is used solely to assess the uncertainty of the initial noise seed—not to track full-step generation behavior—our 1-step computation is sufficient to capture the core uncertainty signal. In contrast, computing BANSA over all steps requires running multiple attention forward passes across the full trajectory, resulting in substantial computational overhead that limits its practicality for real-world applications.

## Appendix H Additional Qualitative Comparison

#### More qualitative results.

Figures [8](https://arxiv.org/html/2505.17561v1#A8.F8) and [9](https://arxiv.org/html/2505.17561v1#A8.F9) present additional examples generated using our noise selection framework. Across diverse prompts, the selected seeds yield improved spatial detail, aesthetic quality, and semantic alignment, further validating the robustness of our approach. These examples complement our quantitative findings by illustrating the visual impact of BANSA-based noise selection.

#### Effect of BANSA score on generation quality.

Figures [10](https://arxiv.org/html/2505.17561v1#A8.F10) provide a qualitative comparison of outputs generated using three types of noise seeds: a randomly sampled seed, the seed with the highest BANSA score (lowest quality), and the seed with the lowest BANSA score (highest quality). All videos were generated using 50 denoising steps with the CogVideoX-5B backbone. The lowest-BANSA seed consistently produces sharper, more coherent, and semantically faithful videos, whereas the highest-BANSA seed often leads to structural artifacts or temporal instability. These results highlight the practical value of BANSA-guided noise selection.

Figure: Figure 8: Effect of ANSE on semantic fidelity and motion stability in CogVideoX outputs. Each block compares baseline generations with those using ANSE-selected noise. Across both CogVideoX-2B and 5B, ANSE improves semantic alignment to the prompt and reduces artifacts such as temporal flickering and object distortion.
Refer to caption: extracted/6470240/supple_1.png

Figure: Figure 9: Additional qualitative comparison of CogVideoX variants with and without ANSE. Results from CogVideoX-2B are shown in the first two rows; the rest show CogVideoX-5B. With ANSE, videos exhibit improved visual quality, better text alignment, and smoother motion transitions compared to the baseline.
Refer to caption: extracted/6470240/supple_2.png

Figure: Figure 10: Qualitative comparison of generations from different noise seeds. We compare outputs generated from a randomly sampled seed (top), the seed with the highest BANSA score (middle), and the seed with the lowest score (bottom), using the same prompt and model. BANSA-selected seeds produce more coherent structure, stable motion, and stronger semantic alignment than both random and high-uncertainty seeds.
Refer to caption: extracted/6470240/supple_3.png

## Peer Reviews

confidence: 2
contribution: 2
initial_rating: 6
presentation: 2
questions: 1. The paper mentions that BANSA can be applied to cross-, self-, or temporal attention. Did the authors investigate which type of attention map is most predictive of final video quality?
2. The Classifier-Free Guidance (CFG) scale is a crucial parameter for controlling prompt alignment in diffusion models. How does the choice of noise seed, as guided by BANSA, interact with different CFG values? Is it possible that a high BANSA score could be improved by using a higher CFG scale, or does the initial noise fundamentally constrain the quality regardless of the guidance strength?
soundness: 2
strengths: 1. The central idea of using attention-based uncertainty as an internal signal to guide noise selection is innovative. It shifts the paradigm from relying on external, often heuristic-based priors to a principled method that is aware of the model's own confidence.
2. Through approximations (Bernoulli masking, layer truncation), authors make the consistent quality gains with a reasonable inference overhead. Demonstrates robust generalization and real-world viability.
summary: The paper proposes ANSE (Active Noise Selection for Generation), an inference-time framework that improves text-to-video diffusion by selecting the initial noise seed that minimizes a new attention-based epistemic uncertainty measure, BANSA (Bayesian Active Noise Selection via Attention). This method improves video quality and temporal coherence across various T2V backbones and is approximated efficiently via Bernoulli-masked attention and layer truncation chosen by correlation analysis to keep overhead low.
weaknesses: 1. Lack of statistical confidence reporting. Improvements are modest in absolute terms; without standard deviations/CI or significance tests, it’s hard to assess robustness and effect sizes across prompts/seeds.
2. Lack of Sensitivity Analysis for Key Hyperparameters. The method relies on several key hyperparameters, notably the Bernoulli masking probability p (set to 0.2) . The paper does not provide a sensitivity analysis for these values. It is unclear how the optimal choices were determined and how robust the method's performance is to variations in these parameters.

---

confidence: 5
contribution: 3
initial_rating: 4
presentation: 4
questions: Please refer to the strengths and weaknesses
soundness: 2
strengths: + A novel metric is defined to guide noise selection.
+ The paper is well-organized and presents the idea clearly.
summary: This paper addresses noise selection in video diffusion models by introducing a metric called BANSA, which utilizes attention information to estimate model confidence and consistency. It also proposes a Bernoulli-masked approximation to efficiently compute scores from a single diffusion step.
weaknesses: - Would the method remain effective when applied to a powerful base video model (e.g., WAN 2.2/2.1 14B), or when the video model has undergone post-training procedures such as RL or DPO?
- There is a lack of visualizations showing the score distributions for different prompts and samples from different noise inputs.
- How does the Bernoulli masking affect the score distribution? Is there a significant difference between using and not using the mask?
- It would be helpful to show generated samples from the same prompt but with different scores, to help validate the effectiveness of the proposed metric.
- What is the underlying reason that using this score leads to improvements in motion, composition, and artifact reduction? More insight into the mechanism would be valuable.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 4
questions: Please see the [Weakness] section.
soundness: 3
strengths: * The paper is clearly written.

* The accompanying website includes excellent figures and animations, which are much appreciated.

* The experiments and ablation studies are detailed and thorough.
summary: The paper presents ANSE, a framework for active noise selection in video diffusion models using BANSA, an attention-based acquisition function that identifies noise seeds yielding consistent and confident generations. BANSA adapts the BALD principle to attention space with lightweight, efficient inference. Experiments across multiple text-to-video backbones show that ANSE may enhance video quality and prompt alignment with slight overhead.
weaknesses: 1. **Ensemble Size**  
   - In Table 7, the metrics consistently improve as the ensemble size $K$ increases.  
   - Could the authors provide results with even larger $K$ values?  

2. **Denoising Steps**  
   - What is the number of denoising steps used for each model?  
   - It appears to be 50 based on the appendix, but this is only explicitly mentioned for the CogVideoX-5B backbone (L839).  

3. **Computation Budget (NFE)**  
   - From a neural function evaluation (NFE) perspective, the total cost is effectively $M + 50$ (or the actual number of denoising steps).  
   - What would happen if the baseline methods were also given an equivalent $M + 50$ computation budget?  

4. **Ablation on Bernoulli-Masked Attention**  
   - Please include ablation results for the Bernoulli-masked attention mechanism, particularly analyzing the effect of the masking probability parameter $p = 0.2$.  

5. **Evaluation Metrics**  
   - Consider including additional motion-specific metrics, such as FVMD [1], to better assess the claimed improvements.  
   - The sample videos on the website look good and visibly outperform the baseline. A human evaluation could further validate these qualitative improvements, as VBench—while comprehensive—may have limitations in capturing perceptual quality.  

Ref:\
[1] Liu, J., Qu, Y., Yan, Q., Zeng, X., Wang, L. and Liao, R., 2024. Fr'echet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videos. arXiv preprint arXiv:2407.16124.


Decision: Accept (Poster)