Controllable Video Generation with Provable Disentanglement
Authors: Yifan Shen
Peiyuan Zhu
Zijian Li
Shaoan Xie
Namrata Deka
Zongfang Liu
Zeyu Tang
Guangyi Chen
Kun Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Video Generation
GAN
Disentanglement
nonlinear ICA
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Controllable video generation remains a significant challenge, despite recent advances in generating high-quality and consistent videos. Most existing methods for controlling video generation treat the video as a whole, neglecting intricate fine-grained spatiotemporal relationships, which limits both control precision and efficiency. In this paper, we propose \textbf{Co}ntrollable \textbf{V}ide\textbf{o} \textbf{G}enerative \textbf{A}dversarial \textbf{N}etworks (\ourmes) to disentangle the video concepts, thus facilitating efficient and independent control over individual concepts. Specifically, following the \textbf{minimal change principle}, we first disentangle static and dynamic latent variables. We then leverage the \textbf{sufficient change property} to achieve component-wise identifiability of dynamic latent variables, enabling independent control over motion and identity. To establish the theoretical foundation, we provide a rigorous analysis demonstrating the identifiability of our approach. Building on these theoretical insights, we design a \textbf{Temporal Transition Module} to disentangle latent dynamics. To enforce the minimal change principle and sufficient change property, we minimize the dimensionality of latent dynamic variables and impose temporal conditional independence. To validate our approach, we integrate this module as a plug-in for GANs. Extensive qualitative and quantitative experiments on various video generation benchmarks demonstrate that our method significantly improves generation quality and controllability across diverse real-world scenarios.

Published as a conference paper at ICLR 2026
CONTROLLABLE VIDEO GENERATION WITH PROVABLE DISENTANGLEMENT
Yifan Shen't ; Peiyyuan ; Li Shaoan Xie², Namrata Deka², Zongfang Liu', Zeyu Tang", Guangyi Chen 1:2 Kun Zhang 1.2f
Mohamed bin Zayed University of Artificial Intelligence 2 Carnegie Mellon University
ABSTRACT
Controllable video generation remains a significant challenge, despite recent ad- vances in generating high-quality and consistent videos. Most existing methods for controlling video generation treat the video as a whole, neglecting intricate fine-gres aine-graine spatiotemporal relationships, which limits both control precision and efficiency. In this paper, we propose Controllable Video Generative Adversarial Networks (Co to disentangle the video concepts, thus facilitating effi- cient and independent control over individual concepts. Specifically, following the minimal change principle, we first disentangle static and dynamic latent vari- ables. We then leverage the sufficient change property to achieve component wise identifiability of dynamic latent variables, enabling disentangled control of video generation. To establish the theoretical foundation, we provide rigorous analysis demonstrating the identifiability of our approach. Building on these theoretical insights, we design Temporal Transition Module to disentangle latent dynam- ics. To enforce the minimal change principle and sufficient change property, we minimize the dimensionality of latent dynamic variables and impose temporal conditional independence. To validate our approach, we integrate this module as a plug-in for GANs. Extensive qualitative and quantitative experiments on various video generation benchmarks demonstrate that our method significantly improves generation quality and controllability across diverse real-world scenarios.
INTRODUCTION
Video generation (Vondrick et al., 2016; Tulyakov et al., 2018; Wang et al., 2022) has become prominent research focus, driven by its wide-ranging applications in fields such world simulators (OpenAI, 2024), autonomous driving (Wen et al., 2024; Wang et al., 2023), and medical imaging (Li et al., 2024a; Cao et al., 2024). particular, controllable video generation (Zhang et al., 2025) is essential for advancing more reliable and efficient video generation models. Despite the impressive results achieved by recent commercial or large-scale models such as Kling (Kuaishou Technology. 2024) and Wan (Wan et al., 2024), precise control over specific aspects of generated video remains significant challenge, as illustrated in Figure 1. This issue may arise because these models typically represent the video prompts) directly to this global representation. Recent approaches (Ho et al., 2022; Zhou et al., 2022; Yang et al., 2024b; Zheng et al., 2024) follow similar strategy, with differences in modeling frameworks (e.g., diffusion or VAE) and shapes (e.g., vectors spatiotemporal blocks). However, these formulations often neglect the intricate spatio-temporal structure of videos, thereby limiting the ability to disentangle and control fine-grained factors, such as head movements and eye blinking.
unified 4D spatiotemporal block and apply conditioning signals (e.g., text
To address this issue, one intuitive solution to learn a disentangled representation of the video, within which the internal relationships are often not considered. Some models (Hyvärinen & Oja, 2000; Tulyakov et al., 2018; Yu et al., 2022; Skorokhodov et al., 2022: Wei et al., 2024) explicitly decompose video generation into two parts: motion and identity, representing dynamic and static
"quual contribution.
1 "ooresponndecee <kunzl @cmu.edu>.

---

Published as a conference paper at ICLR 2026
Wan Kling
Time stops
Figure 1: Videos are generated using Kling and Wan with the prompt: "while this person was speaking, the head gradually shifted from the middle to the right." " The first row shows that essential motion cues are partially omitted, while in the second row the head size changes undesirably.
information, respectively. This separation allows for more targeted control over each aspect, making it possible to modify the motion independently without affecting the identity. (Zhang et al., 2025; Shen et al., 2023) leverage attention mechanisms to further disentangle different concepts within the video, enhancing the ability to control specific features with greater precision. (Fei et al., 2024; Lin al., 2023) utilize Large Language Models find the intricate temporal dynamics within the video and then enrich the scene with reasonable details, enabling more transparent generative process. These methods are intuitive and effective, yet they lack solid guarantee of disentanglement, making the control less predictable and potentially leading to unintentional coupling of different aspects.
These limitations of previous approaches motivate us to rethink the paradigm of video generation. Inspired by recent advancements in nonlinear Independent Component Analysis (ICA) (Hyvarinen & Morioka, 2017: Khemakhem et al., 2020; Yao et al., 2022; Hyvarinen & Morioka, 2016) and the successful applications like video understanding (Chen et al., 2024), we propose the Controllable Video Generative Adversarial Network (CoVoGAN) with a Temporal Transition Module plugin. Building upon StyleGAN2-ADA (Karras al., 2020), we distinguish between two types of factors: the dynamic factors that evolve over time, referred to as style dynamics, and the static factors that remain unchanged, which call content elements. We also distinguish between different components within the dynamics, allowing for more precise control. By leveraging the minimal change principle, we demonstrate their block-wise identifiability (Von Kügelgen et al., 2021; Li et al., 2024b) and find the conditions under which motion and identity can be disentangled. explaining the effectiveness of the previous line of video generation methods that separate motion and identity. In addition, we employ sufficient change property to disentangle different concepts of motion, such as head movement or eye blinking. Specifically, we introduce a flow (Rezende & Mohamed, 2015) mechanism to ensure that the estimated style dynamics are mutually independent conditioned on the historical information. Furthermore, we prove the component dynamics and provide disentanglement guarantee for the motion in the video.
identifiability of the style
We conduct both quantitative and qualitative experiments on various video generation benchmarks. For quantitative evaluation, we use FVD (Fréchet Video Distance) (Unterthiner et al., 2019) to assess the quality of the generated videos. We also compare SAP (Kumar et al., 2017), modularity (Ridgeway & Mozer, 2018) and MCC to demonstrate the disentanglement capability. For qualitative analysis, we evaluate the degree of disentanglement by manipulating different dimensions of the latent variables and comparing the resulting video outputs. Experimental results demonstrate that our method significantly outperforms other video generation models of a scale similar to CoVoGAN.
Key insights and contributions of our research include:
We propose a Temporal Transition Module to achieve a disentangled representation, which lever- ages the minimal change principle and sufficient change property.
We implement the Module in GAN, i.e., CoVGGAN to learn the underlying generating process from video data with disentanglement guarantees, enabling more precise and interpretable control.
To the best of our knowledge, this is the first work to provide an identifiability theorem in the context of video generation. This helps to clarify previous intuitive yet unproven techniques and suggests potential directions for future exploration.
Extensive evaluations across multiple datasets demonstrate the effectiveness of CoVoGAN, achiev- ing superior results in terms of both generative quality and controllability.
The remainder of this paper is organized as follows. Section 2 formalizes the video generation process and its identifiability, and discusses how this enables disentangled control over video generation. Sec-
2

---

Published as a conference paper at ICLR 2026
tion 3 provides theoretical insights into when and how block-wise and component wise identifiability can be achieved. Section introduces the proposed CoVoGAN model and explains how its design is grounded in the theoretical results. Section 5 presents extensive experiments to comprehensively evaluate Co VoGAN. Finally, Section concludes the paper.
PROBLEM SETUP
2.1 GENERATING PROCESS
Consider a video sequence V= (x1, X2,..., XT] consisting of T consecutive frames. Each frame X₁ E IR* is generated via an arbitrary nonlinear mixing function g. which maps set of latent variables to the observed frame X,. The latent variables are decomposed into two distinct parts: zi _"n,, capturing the style dynamics that evolve over time, and D E IR"r, encoding the content video. Furthermore, these latent variables are
variables that remain consistent across all frames assumed to arise from stationary, non con-aaamettrc,, time-delayed causal process.
As shown in Figure and Equation 1, the generating process is formulated as:
zi
2Z
7
Zi
X = g(zi,2"), 2., f. (Pa(st.)) with { fúi ~ ~ Pey;
X
X
X3
..
X
= 19 (5),
in which 25 25 R refers to the i-th entry of zi and j-th entry of 2f, respectively. Pa(=f,) refers to the time- delayed parents of 24,;- All noise terms 6, and Ej are independently sampled from their respective distributions: Pt, for the i-th entry of the style dynamics, and pt, for the
Ze
Figure 2: The generating process. The gray shade of nodes indicates that the variable is observable.
j-th entry of the content elements. The components of z are mutually independent, conditioned on all historical variables Uن Pa(z;,). The non- non-parametric causal transition fi enables an arbitrarily nonlinear interaction between the noise term e,i and the set of parent variables Pa(z),), allowing for flexible modeling of the style dynamics.
2.2 IDENTIFICAIIN OF THE LATENT CAUSAL PROCESS
For simplicity, we denote I" the group of functions (f and similarly for f", p"
Definition 2.1 (Observational Equivalence). Let {xx, X2,,X2... x2. XT) represent the observed video generated by the true generating process specified by f. p", p"), as defined in Equation 1. learned model (g.f" if pp.p") is observationally equivalent to the true process if the model distribution P(g.3" F .a ..) (V) matches the data distribution P(g. p' .=) ( (V) for all values of V.
When observational equivalence is achieved, the distribution of videos generated by the model exactly matches that of the ground truth, i.c., the training In other words, the model produces video data that is indistinguishable from the actual observed data.
Illustration.
Definition 2.2 Bisc Identification of Generating Process). Let the true generating process be f. p") specified in Equation 1 and let its estimation be (g, f T.p". p.. p²). The generating process is identifiable up to the subspace of style dynamics and content elements, if the observa- tional equivalence ensures that the estimated (2), 2n) satisfies the condition that there exist bijective mappings from (2., 20) to (z;; 20) and from 2" to z Formally, there exists invertible functions h: RR. nD and he Re Te such that
pgr (V) Pl Merrrrri
[2]22]] 1((22,22)) h2.(2))
where [-] denotes concatenation.
Definition 2.3 (Component-wiis wise Identification of Style Dynamics). On top of Definition 2.2, when h" a combination of permutation π and a component-wis invertible transformation T. Formally,
Pi ge; Pla. p pn (V)) 2 ( () (x:)).
(3)
3

---

Published as a conference paper at ICLR 2026
2.3 FROM IDENTIHIABILITY CONTROLLABLE VIDEO GENERATION
Identifiability ensures the uniqueness of the latent representation, meaning that the learned latent variables correspond to true latent variables up to certain allowable transformations. Moreover, identifiability be defined at different levels, with higher levels indicating that the estimated variables align more closely with the true underlying factors, Stronger identifiability thus leads to efficient and precise control video generation.
more disentangled representations, enabling
When block-wise identifiability achieved, content-related components are effectively disentangled from motion. This allows motion control to be applied independently, enabling manipulation of motion without altering the underlying content of the video. For example, in sequence where a camera moves forward, can adjust the camera's direction without affecting the static scene.
In contrast, when component-wise identifiability is achieved, each learned component corresponds one-to-one with its true underlying factor. This property allows independent manipulation of each generative factor by adjusting the corresponding latent dimension, without undesired interference across factors. Such fine-grained control represents the ultimate goal of disentanglement, providing principled and theoretically grounded formulation. For instance, in generating a video of face, one can separately adjust head movements or eye blinks by modifying the associated latent variables.
The key remaining question, therefore, is under what conditions identifiability can be guaranteed, and how such guarantees can be established practice.
THEORETICAL ANALYSIS
In this section, we discuss the conditions under which the block-wise identification (Definition 2.2) and component-wise identification (Definition 2.3) hold. We consider only stationary latent process in this part for simplicity. However, our theorem can be extended to nonstationary scenarios when supervision is provided, discussed in Appendix A.4.
3.1 BLOCK-WISS WISE IDENTIFCATION
Without loss of generality, we first consider the case where Pa(=f,) = zi-1. meaning that the time-dependent effects governed by the dynamics of the previous time step.
Definition 3.1. (Linear Operator(Hu Schennach, 2008a; Dunford & Schwartz, 1988)) Consider two random variables and b with support A and B, the linear operator Laja is defined as mapping from a density function Pa in some function space F(AA) onto the density function Lbja o in some function space F(B),
F(A) + F(B): ps Loja Pa = -/: pojo(. a)pa(a)da.
To better illustrate Linear Operator, examples of linear operators are provided in the Appendix A.3.
Theorem 3.2 (Block-wise Identifiability). Consider video observation {x1, . generated by process (g.f .f -''p'''pp' with latent variables denoted as and z according to Equation I, where X2 E RR 26. E R."",""" Th. assumptton
BI (Positive Density) the probability density function of latent variables always positive and bounded:
B2 (Minimal Changes) the linear operators Txx-all), x and Texe-1lxess are injective for a bounded function space;
B3 (Weakly Monotonic) for any 2n, de E 2 2i (24 2.), the set (x, p(xx(()) # p(x+ has positive probability, and conditional densities are bounded and continuous;
are satisfied, then z, wise identifiable with regard to 2, from learned model (g. f .p" pp) under Observation Equivalence.
4

---

Published as a conference paper at ICLR 2026
Illustration of assumptions. The assumptions above are commonly used in the literature the identification of latent variables under measurement error (Hu & Schennach, 2008b). Firstly, Assumption BI requires a continuous distribution. Secondly, Assumption B2 imposes minimal requirement on the number of variables. The linear operator T ensures that there is sufficient variation in the density of b for different values of a, thereby guaranteeing injectivity. In a video, X₁ is of much higher dimensionality compared to the latent variables. As result, the injectivity assumption is easily satisfied. In practice, following the principle of minimal changes, if model with fewer latent variables can successfully achieve observational equivalence, it is more likely to learn the true distribution. Assumption B3 requires the distribution of x, changes when the value of latent variables changes. This assumption is much weaker compared to the widely used invertibility assumption adopted by previous works, such as (Yao al., 2022).
Overall, the three assumptions impose mild requirements on the underlying data generation process. When these assumptions are satisfied, the block-wise identifiability result established in our theorem holds. Importantly, real-world video scenarios naturally conform to these assumptions, and the structure of many existing generative models already aligned with the conditions specified in the theorem. A more detailed discussion of these assumptions and their practical relevance can be found in Appendix A.1.
Proof sketch. We separately prove the identifiability of all latent variables and z". For the first part, is built on (Fu et al., 2025), following the line of work from (Hu & Schennach, 2008b). Intuitively, required for
demonstrates that a minimum of different observations of latent variables identification under the given data generation process. For the second part, we use the contradiction show that the same z in different frames of video can be identified leveraging the invariance.
3.2 COMPONENT-WISS IDENTIFICATION
Theorem 3.3 (Component-wise Identifiability). Consider video observation V (xx,
generated by process [g. Jf .P ppp')) with latent variables denoted as z and Ze. according to Equation 1, where Xt E Rd z E R Rn,,2" Ene, Suppose assumptions in Theorem 3.2 hold. If assumptions
CI (Smooth and Positive Density) the probability density function of latent variables is always third-order differentiable and positive;
C2 (Sufficient Changes) let mei
log dog,(zz,, (2-1) and
pm.
8 8n.,.
8mm
8 amm..
Va.1 02
E 12
(4)
for l E (1,2, ... ,n]. For each value of 7ú there exists 2n, different values of 2t-1, such that the 2n, vectors Vi,1 E RR?.. are linearly independent;
C3 (Conditional Independence) the learned 2 is independent with 20, and all entries of 2 are mutually independent conditioned on أل
satisfied, then zł is component-wise identifiable with regard 2n from learned model (g.3 F.. -p".p") under Observation Equivalence.
Proof sketch. In summary, component-wiss identification relies on the changcability of style dynamics, i.c., sufficient changes. Starting from the results block-wise identification, we establish the connection between zt and 2 in terms of their distributions, i.e., p(2,) p(2.). HHz), where He the jacobian matrix. Leveraging the second-order derivative the log probability, we construct a
system of equations with terms of azū, 0=f. and coefficients specified in assumption C2. We dE,,, TE, leverage the third-order derivative of the previous latent variable 1, to eliminate |Hr), utilizing
the fact that the history does not influence the current mapping from estimation to truth. Solving this _ def Jet
system yields DEt
This indicates that zi,, is a function of most one E
Illustration of assumptions. The assumptions C1, C2 the data generating process are commonly adopted in existing identifiable results for Temporally Causal Representation Learning (Yao et al., 2022). Specifically, C1 implies that the latent variables evolve continuously over time, while C2
5

---

Published as a conference paper at ICLR 2026
Temporal Transition Module
Synthesis network
Const 4x4k512
z
Zi
zi
z
h
2
A
style
AdaiN
4x4
lt2
Z
2
Z
Upsample
Const 4x4w512
Z Z
style
AdaIN
Cony
style
Figure 3: Generator operates from left right, beginning with random noise input. The noise first passes through a Temporal Transition Module, which produces a disentangled representation of the underlying factors. This representation is then fed into the synthesis network to generate frames at the pixel level. the figure, the blue arrow illustrates the Deep Sigmoid Flow.
ensures that the variability in the data can be effectively captured. The assumption C3 constraints the learned model require it to separate different variables of style dynamics into independent parts.
4 APPROACH
Given our results identifiability. implement model, CoYGANN, Our architecture is based on StyleGAN2-ADA (Karras et al., 2020), incorporating a Temporal Transition Module in the Generator enforce the minimal change principle and sufficient change property. Additionally, we add Video Discriminator to ensure observational equivalence of the joint distribution p(V).
4.1 MODEL STRUCTURE
Noise sampling. The structure of the generator is shown in Figure 3. To generate a video with length T. we first independently sample random noise from normal distribution e (N( We then naively split it into several parts, i.e., [e"; é;; 65 ;
Temporal Transition Module. Following the generating process in Equation we handle z and zž separately. On the one hand, we employ an autoregressive model to capture historical information, followed by conditional flow to generate zł. Specifically, we implement Gated Recurrent Unit (GRU) (Chung al., 2014) and Deep Sigmoid Flow (DSF) (Huang et al., 201 8), formulated as
h GRU(h..1,._))
= DSF h.
(5)
where h, denotes the historical information until time step and zi,; will be mutually independent conditioned on h, On the other hand, since z not required to be mutually independent, an MLP to generate z i.c..
MLP((").
(6)
Concatenate zł and z and then obtain the disentangled representation z, z z for each frame at time step t of the video.
Synthesis network. The synthesis network is designed in the way StyleGAN2-ADA. The generated representation 2, is first fed into the mapping network to obtain a semantic vector w(z.) E W. and then the f-th frame of the video is generated by the convolutional network with
Discriminator structure. To ensure observational equivalence, we implement a video discriminator Dy separate from the image discriminator D), For the image discriminator, follow the design of

---

Published as a conference paper at ICLR 2026
the original StyleGAN2-ADA. For the video discriminator, adopt channel-wit concatenation of activations at different resolutions to model and manage the spatiotemporal output of the generator.
Loss. In addition to the original loss function of StyleGAN2-ADA, introduce two additional losses: (1) a video discriminator loss, and (2) a mutual information maximization term (Chen et al., 2016) between the latent dynamic variables z and the intermediate layer outputs of the video discriminator. This encourages the model to learn a more informative and structured representation.
4.2 RELATIONSIIIP BETWEEN MODEL AND THEOREM.
Block-wise identification. As discussed in Theorem 3.2, achieving block-wise identifiability benefits from minimizing the dimension n, of the style dynamics, especially when the true n, is unknown. In practice, this translates to a hyperparameter selection question. Therefore, we opted for relatively modest value of n, and observed that it suffices to attain a satisfactory level of disentanglement capability. Furthermore, as required by the assumptions, the learned variables 2 and 2 are block- wise independent, i.e., 2 11 2². This independence is necessary to achieve block-wise identifiability.
Component-wise identification. As outlined in Theorem 3.3, the sufficient change property is critical assumption for achieving identifiability. To enforce temporally conditional independence, we employ a component-wen flow model that transforms a of independent noise variables ef into the style dynamics, conditioned on historical information. Furthermore. the flow model is designed to maximally preserve the information from <; to zi.5, enabling the model to effectively capture sufficient variability in the data. Note that when computing the historical information ht, we utilize ef instead of 2 (as illustrated in the generating process in Equation 1) as the condition for the component- wise flow. This approach offers two key advantages. First, it simplifies the model architecture since the flow does not need to incorporate the output from another flow. Second, the noise terms already fully characterize the corresponding style dynamics, which remains consistent with the theoretical framework. Furthermore, given that the precise time lag of dynamic variables remains unspecified priori in the dataset, the GRU's gating mechanism selectively filter out irrelevant historical information that lies outside Pa(z:). This capability enables the model to demonstrate significantly superior performance compared to traditional non-gated architectures, such as vanilla RNNs. A detailed ablation study is presented in Section 5.4.
5 EXPERIMENTS
5.1 EXPERIMENTAL SETUP
Datasets. We evaluate our model on four different real-world datasets: FaceForensics (Rössler et al., 2018), SkyTimelapse (Xiong et al., 2018), RealEstate (Zhou al., 2018) and CelebV-HQ (Zhu et al., 2022). The first three datasets contain videos with resolution of 256 x 256 pixels, while the last dataset consists of videos at 512 512 resolution. We employ standard train-test splits for fair evaluation. Detailed information about the datasets are provided in Appendix C.
Evaluation metrics. To comprehensively evaluate the performance of CoVGANN, we employ both quantitative and qualitative assessment metrics. For quantitative evaluation, we adopt the Fréchet Vidco Distance (FVD) (Unterthiner et al., 2018), a widely-used metric for assessing video generation quality. We report FVD scores at two different temporal scales: FVD; and FVD1e. where the subseript denotes the number of frames in a video. To better assess the disentanglement capability of our method, we compare the widely used disentanglement metrics SAP (Kumar et al., 2017) and modularity (Ridgeway & Mozer, 2018). Additionally, we measure the Mean Correlation Coefficient (MCC), a standard metric for disentanglement. Details of the metrics can be found in Appendix D.
5.2 VIDEO QUALITY
We consider four GAN- models: MoCoGAN-HD, DIGAN, StyleGAN-V, and MoStGAN-V V. We also compare three diffusion-based models: VDM (Ho et al., 2022), LVDM (He et al., 2022) and Latte (Ma et al., 2024). For MoCoGAN-HD, we freeze the image generator pretrained by the original authors and train the motion generator and discriminator. We exclude MoCoGAN-HI HD from the comparisons on the specific datasets without pretrained image generator. For Latte, we directly use the released checkpoints. For the remaining baselines, we train the models from scratch using their official implementations to ensure comparability. The quantitative results in Table ?? show
7

---

Published as a conference paper at ICLR 2026
StyleGAN-V
StyleGAN-V
MoSIGAN-V
MoSIGAN-V
CoVoGNN
CoVoGAN
(a) Head adjustment on the FaceForensics dataset.
(b) Camera
Figure 4: Controllability in the latent space across datasets and methods. Each method is evaluated with three samples by varying a single latent dimension. Only CoVoGAN exhibits consistent control across identities: (a) head pose adjustment FaceForensics, (b) camera translation RealEstate.
FaceForensies
TkyTelllapee
RealEstate
CelebV-HQ
Method
FVDs
FVDre
FVDs
FVD16
FVDs
FVD16
FVD16
McCuGAN-IE HD
140.05
185.51
1214.13
1721.89
412.50
DIGAN
57.52
61.65
60.54
105.03
182.86
178.27
StylGGANV
49.24
52.70
45,30
62,35
199.66
201.95
147.81
MoSIGAN-V
47.67
49.85
40.97
55.36
247.77
265.54
127.62
VDM
1038.29
1046.60
1099.21
1104.80
1524.17
1526.04
307.22
40.21
CoVoGAN
48.80
35.58
46.51
174.87
97.16
Table 1: FVD; +. and FVD16 + results across different datasets.
that CoVoGAN consistently achieves top performance across all datasets, despite diffusion based models occasionally exhibiting higher visual fidelity. More visualization results are provided in the Appendix F.5.
5.3 CONTROLLABILITY
Block-wise disentanglement. Figure 4 demonstrates the video controllability CoVoGAN in comparison with baseline methods, highlighting our model's supe- rior capability for disentanglement between motion and content. The analysis follows sy systematic procedure: we first generate base video sequence, then apply con- trolled modification by adding value specific motion-rclated latent variables.
Example
Example
Original
Blink
Blink
+ Shake
Figure 5: Controllability visualization results the Face- Forensics dataset. Two distinct motion concepts are manip- ulated to illustrate component-wist disentanglement, Corre- sponding videos are provided in the supplementary materials for better visualization.
For CoVoGAN, we modify one dimension of the style dynamics zf. For StyleGAN- and MoStGAN-V, we manipulate one dimension of their (latent) motion code. We apply equivalent modifications to the corresponding latent dimensions each baseline model. To validate the consistency of our controllability analysis, we randomly sample three distinct video sequences and apply identical modifications to their respective
8

---

Published as a conference paper at ICLR 2026
latent representations. Other baselines not compared since there are specific variables with semantic information.
The results show that proposed CoVoGAN model Iearns disentangled representation that effectively separates style dynamics from content elements. (1) This disentanglement enables independent manipulation of motion characteristics while preserving content consistency. (2) A key advantage of this approach is that identical modifications to the style latent space consistently produce similar motion patterns across different content identities. Baseline models achieve only partial disentanglement, exhibiting two major limitations: (I) visual distortions of the modified videos and (2) inconsistent or misaligned motion patterns when applied different identities.
Component-we wise disentanglement. We also show the precise control over individual motion components. This capability is enabled by the component-wiss identifiability of our model, which ensures each latent di- mension corre- sponds to a spe- cific and inter- pretable motion attribute, c.g.. eye blinking or head shaking. Our experimental procedure begins with randomly sampling two distinct video se- quences, as illustrated in the first row of Figure 5. We then selectively modify the latent dimension corresponding eye blinking dynamics the second line. Subsequently, we modify second latent dimension controlling head shaking motion while maintaining the previously adjusted eye blinking pattern in the last line. The results show naturalistic head movements from left to right, synchronized with the preserved eye blinking, illustrating our model's 's capability for independent yet coordinated control of multiple motion components.
Table 2: Disentanglement Comparison across different models.
Metrics
StyleGAN-V
MoSIGAN-V
LVDM
Latte
CoVoGAN
MCC (%)
29.00
27.95
21.60
20.87
33.78
SAP (%)
4.25
5,90
0.72
0.75
8.48
Modularity (%)
7.66
13.48
7.25
7.44
17.37
Disentanglement metries. We first extract semantic annotations from the videos for disentanglement evaluation. For this purpose, we conduct experiments on the FaceForensics dataset, which facilitates the extraction of meaningful facial attributes. We utilize pretrained model from Dlib to extract facial landmarks and compute semantic annotations, including eye size, mouth size, head position, and head angle. We only compare method with two models that explicitly incorporate semantic representation layers, which we use to compute the metrics. Since diffusion- provide compact latent representation by design, we adopt the following procedure: first randomly sample videos, then extract frame-wise representations from the high-dimensional latent space. These representations are reduced 128 dimensions using PCA, and the disentanglement metrics are computed in this reduced space. For our method, we compute the metrics using the dynamic latent variables zf. As shown in Table 2, our method achieves the best performance.
models do not
5.4
ABLATION STUDY
We conduct ablation study to eval- uate the contributions of the proposed components within the Temporal Tran- sition Module, as shown in Table 3. Re- placing the GRU with a standard RNN leads to a noticeable performance drop. This is primarily due the loss of sparsity provided by the GRU's gating mechanism, which helps isolate time- delayed effects. Without this mecha- nism, the model faces a larger search space, making it harder to learn an effective transition function. Substituting the component-wiss flow with fully connected MLP results in an even more significant degradation. This decline can be attributed to two factors: (1) the mutual independence between style dynamics can longer be maintained, and (2) capturing sufficient changes becomes more difficult.
Table 3: Ablation studies on different GRU configurations and component-wise flow the FaceForensics dataset.
Metric
CoVoGAN
w/o GRU
w/o flow
FVDis
48,80
53.68
82.81
MCC (%) 1
33.78
26.59
8.22
SAP (%) f
8.48
7.25
0.55
Modularity (%)
17.37
12.40
10.24

---

Published as a conference paper at ICLR 2026
CONCLUSION
In this paper, proposed Temporal Transition Module and implemented it in GAN achieve CoVoGAN. By leveraging the principles of minimal and sufficient changes, we successfully disen- tangled (1) the motion and content, and (2) different concepts within the motion. We established an identifiability guarantee for both block-wise and component-wise disentanglement. Our proposed CoVoGAN model demonstrates high generative quality and controllability. We validated the perfor- mance on various datasets and conducted ablation experiments to further confirm the effectiveness of our model. Overall, our work provides principled and practical framework for disentangled video generation and opens new directions for fine-grained interpretable, and controllable visual synthesis. Limitations and Future works: focuses primarily on theoretical contributions and their empirical validation, while the integration with higher-fidelity generative architectures and application to open-domain scenarios remains an important direction for future work.
ACKNOWLEGEMENT
We would also like to acknowledge the support from NSF Award No. 2229881, Institute for Societal Decision Making (AI-SDM), the National Institutes of Health (NIH) under Contract RO1HL159800 and grants from Quris AI, Florin Court Capital, MBZUAI- WIS Joint Program, and the Al Deira Causal Education project.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
We have taken several steps to ensure the reproducibility of our results. A complete description of the datasets and preprocessing steps is included in Appendix C. The implementation details, including model architectures, training hyperparameters, and optimization settings, are provided in Appendix E. Extended results are reported in Appendix F, addition, we release full source code and configuration files in the supplementary materials.
REFERENCES
Max Bain, Arsha Nagrani, Güil Varol, and Andrew Zisserman. Frozen in time: joint video and image encoder for end-to-end retrieval. In IEEE International Conference on Computer Vision, 2021.
Xu Cao, Kaizhao Liang, Kuci-Da Liao, Tianren Gao. Wengian Ye. Jintai Chen, Zhiguang Ding, Jianguo Cao, James Rehg, and Jimeng Sun. Medical video generation for disease progression simulation. arXiv preprint arXiv:2411 1943, 2024.
Raymond Carroll, Xiaohong Chen, and Yingyao Hu. Identification and estimation of nonlinear models using two samples with nonclassical measurement errors. Journal of nonparametric statistics, 22(4):379-399, 2010.
Guangyi Chen, Yifan Shen, Zhenhao Chen, Xiangchen Song, Yuewen Sun, Weiran Yao, Xiao Liu, and Kun Zhang. Caring: Learning temporal causal representation under non-invertible generation process. arXiv preprint 24Xivv2401.1455 2024.
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Info- gan: Interpretable representation learning by information maximizing generative adversarial nets. Advances in neural information processing systems, 29, 2016.
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3553 2014.
Nelson Dunford and Jacob T Schwartz. Linear operators, part 1: general theory, volume 10. John Wiley & Sons, 1988.
Hao Fei, Shengqiong Wu, Wei Ji, Hanwang Zhang, and Tat-Seng Chua. Dysen-vdm: Empowering dynamics-aware text-to-video diffusion with 1lms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7641-7653, 2024.
Minghao Fu, Biwei Huang, Zijian Li, Yujia Zheng, Ignavier Ng, Yingyao Hu, and Kun Zhang. Identification of nonparametric dynamic causal structure and latent process in climate system. arXiv preprint arXiv:2501.12500, 2025.
Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for figel-iiyiity long video generation. arXiv preprint arXi::2211.13221, 2022.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633-8664 2022.
Yingyao Hu and Susanne M. Schennach. Instrumental variable treatment of nonclassical mea- surement eиror models. Econometrica, 76(1):195-2166 2008a. doi: https://doi.or/11011111 0012-9668..00080008333 00.223.................... URL https:://////....... onl inel bebarry..................ea. wiw wiley... com/ doi /bbs/10. 1/11/j.001200
2008 00823.*
Yingyao Hu and Susanne M Schennach. Instrumental variable treatment of nonclassical measurement error models. Econometrica, 76(1): 76(1):195-216, 2008b.
Yingyao Hu and Matthew Shum. Nonparametric identification of dynamic models with unobserved state variables. Journal of Econometrics, 171(1):32 2012.
11

---

Published as a conference paper at ICLR 2026
Chin-Wer Wei Huang, David Krueger, Alexandre Lacoste, and Aaron Courville. Neural autoregressive flows. In International conference on machine learning, pp. 2087.MMLR 2018
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang. Dahua Lin, Yu Qiao, and Ziwei Liu. Vbench: Comprehensive benchmark suite for video generative models, 2023. URL htwps:ttp/.o.m a arxiv org/abs 11911.1792..
Aapo Hyvarinen and Hiroshi Morioka. Unsupervised feature extraction by cineccontrastive learning and nonlinear ica. Advances in neural information processing systems, 29, 2016.
Aapo Hyvarinen and Hiroshi Morioka. Nonlinear ica of temporally dependent stationary sources, In Artificial Intelligence and Statistics, pp. 460-469, PMLR, 2017.
Aapo Hyvärinen and Erkki Oja. Independent component analysis: algorithms and applications. Neural networks, 134-5))411-430, 2000.
Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. In Proc. NewrIPS, 2020.
Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational autoencoders and nonlinear ica: A unifying framework. In International conference on artificial intelligence and statistics, pp. 2207-2217. PMLR, 2020.
Lingjing Kong, Shaoan Xie, Weiran Yao, Yujia Zheng, Guangyi Chen, Petar Stojanov, Victor Akinwande, and Kun Zhang. Partial identifiability for domain adaptation. arXiv preprint arXiv:2306.0655 10, 2023.
Kuaishou Technology. Kling: Proprietary video generation model by kuaishou, 2024. URL shou. http://wwsseelleeeeeeeeeeeeeeeee https: kuahssuuuu..
otw.:n/eeeeppppprraeeeeeeeeeeeee cessed: 2025-05-15.
Ac-
Abhishek Kumar, Prasanna Sattigeri, and Avinash Balakrishnan. Variational inference of disentangled latent concepts from unlabeled observations. arXiv preprint 17 2017.
Chenxin Li, Hengyu Liu, Yifan Liu, Brandon Y Feng, Wuyang Li, Xinyu Liu, Zhen Chen, Jing Shao, and Yixuan Yuan. Endora: Video generation models as endoscopy simulators. In International Conference on Medical Image Computing and Computer- Assisted Intervention, pp. 230-240. Springer, 2024a.
Zijian Li, Ruichu Cai, Guangyi Chen, Boyang Sun, Zhifeng Hao, and Kun Zhang. Subspace identification for multi- domain adaptation. Advances in Neural Information Processing Systems, 36, 2024b.
Zijian Li, Yifan Shen, Kaitao Zheng, Ruichu Cai, Xiangchen Song, Mingming Gong, Zhengmao Zhu, Guangyi Chen, and Kun Zhang. On the identification of temporally causal representation with instantaneous dependence. arXiv preprint arXiv:2405.1 /5325, 2024c.
Han Lin, Abhay Zala, Jaemin Cho, and Mohit Bansal. Videodirectorgpt: Consistent multi-scene video generation via llm-guided planning. arXiv preprint arXiv:2309.15001, 2023.
Juan Lin. Factorizing multivariate function classes. Advances in neural information processing systems, 10, 1997.
Phillip Lippe. Sara Magliacane, Sindy Löwe, Yuki M Asano, Taco Cohen, and Stratis Gavves. CITRIS: Causal identifiability from temporal intervened sequences. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.). Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 13557-13603. PMLR, 17-23 Jul 2022. URL https: : pproceedings. mlr. mlr..rees/vll66 62 httpp////2///pppp2/////ppp 2pe22a html.
12

---

Published as a conference paper at ICLR 2026
Phillip Lippe, Sara Magliacane, Sindy Löwe, Yuki M Asano, Taco Cohen, and Efstratios Gavves. Causal representation learning for instantaneous and temporal effects in interactive systems. In The Eleventh International Conference on Learning Representations, 2023. URL https: //openrevi ew net/ forum? i 1=
Chang Liu, Rui Li, Kaidong Zhang. Yunwei Lan, and Dong Liu. Stablev2v: Stablizing shape consistency in video-to-video editing. arXiv preprint 2rXiv:2411. 1005,, 2024.
Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. arXiv preprint T:Xiv:2401.03044 2024.
Lutz Mattner. Some incomplete but boundedly complete location families, The Annals of Statistics, pp. 2158-2162, 1993.
OpenAl. Video generation models as world simulators. Technical report, OpenAl, 2024. URL /ettss:/opeene wwt.coo//ooooooo sora/.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. URL https: //arxiv. org/ abs 2103. 00020.
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In International conference on machine learning. pp. 1530-1538. PMLR, 2015.
Karl Ridgeway and Michael C Mozer. Learning deep disentangled embeddings with the f-statistic loss. Advances in neural information processing systems, 31, 2018.
Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. Faceforensics: A large-scale video dataset for forgery detection in human faces. arXiv preprint arXiv:1803.091 17,, 2018.
Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Mostgan-v: Video generation with temporal motion styles. arXiv preprint arXiv:2304.0277) 2023.
Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209. 14792, 2022.
Ivan Skorokhodov, Sergey Tulyakov, and Mohamed Elhoseiny. Stylegan-v: A continuous video generator with the price, image quality and perks stylegan2. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3626-3636, 2022.
Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: dataset 101 human actions classes from videos in the wild, 2012. URL https: /awxiviivvvvv.////a org/ org/abs////ooooooooeo.../oaaeert 12111111111 0402.
Shan Sun, Feng Wang, Qi Liang, and Liang He. Taichi: A fine-grained action recognition dataset. In Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval, pp. 429-433, 2017.
Shuyuan Tu, Zhen Xing, Xintong Han, Zhi-Qi Cheng, Qi Dai, Chong Luo, and Zuxuan Wu. Stableani- mator: High-quality ddentity-preserving human image animation. arXiv preprint arXiv:2411.17697, 2024.
Sergey Tulyakov, Ming- Yu Liu, Xiaodong Yang, and Jan Kautz. Mocogan: Decomposing motion and content for video generation. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1526-1535, 2018
Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski. and Sylvain Gelly. Towards accurate generative models video: A new metric challenges. arXiv preprint arXiv: /8/2.0////2007//// 2018
13

---

Published as a conference paper at ICLR 2026
Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, and Sylvain Gelly. Fvd: A new metric for video generation. 2019.
Julius Von Küggelgen, Yash Sharma, Luigi Gresele, Wieland Brendel, Bernhard Schölkopf, Michel Besserve, and Francesco Locatello. Self-supervised learning with data augmentations provably isolates content from style. Advances in neural information processing systems, 34: 16451-16467, 2021.
Carl Vondrick, Hamed Pirsiavash, and Antonio Torralba. Generating videos with scene dynamics. Advances in neural information processing systems, 29, 2016.
Yufei Wan, Yijun Qiu, Yujing Lin, Yujie Xiao, Zheng Zhu, Zhe Wu, Yujing Huang, and Qixiang Ye. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.03314 2024. URL https::////... / arxiv. org. abs
Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jiagang Zhu, and Jiwen Lu. Drivedreamer: Towards real-world-driven world models for autonomous driving. arXiv preprinl arXiv:3309.09777, 2023.
Yi Wang. Kunchang Li, Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao, Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, et al. Internvideo: General video foundation models via generative and discriminative learning. arXiv preprint arXiv:2212.0311 2022.
Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. In ACM SIGGRAPH 2024 Conference Papers, pp. 1-11, 2024.
Yujie Wei, Shiwei Zhang, Zhiwu Qing. Hangjie Yuan, Zhiheng Liu, Yu Liu, Yingya Zhang, Jingren Zhou, and Hongming Shan. Dreamvideo: Composing your dream videos with customized subject and motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6537 6549, 2024.
Yuqing Wen, Yucheng Zhao. Yingfei Liu, Fan Jia, Yanhui Wang, Chong Luo, Chi Zhang. Tiancai Wang, Xiaoyan Sun, and Xiangyu Zhang. Panacea: Panoramic and controllable video generation for autonomous driving. In Proceedings of the IEEE/CVE Conference on Computer Vision and 2024.
Pattern Recognition, pp.
Shaoan Xie, Lingjin Kong, Yujia Zheng, Zeyu Tang, Eric P. Xing, Guangyi Chen, and Kun Zhang. Learning vision and language concepts for controllable image generation. In Fecty- International Conference on Machine Learning, 2025. URL https: opwnrevieeeeeeeeeeeeeeeeeeee www.ewewweee. n forum?id-bUHRTaTEE
Jinbo Xing, Menghan Xia, Yuxin Liu, Yuechen Zhang, Yong Zhang, Yingqing He, Hanyuan Liu, Haoxin Chen, Xiaodong Cun, Xintao Wang, et al. Make-your-videe video: Customized video generation using textual and structural guidance. IEEE Transactions on Visualization and Computer Graphics, 2024.
Wei Xiong, Wenhan Luo, Lin Ma. Wei Liu, and Jiebo Luo. Learning to generate time-lapse videos using multi-stage dynamic generative adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2364-2373 2018.
Shiyuan Yang. Liang Hou, Haibin Huang, Chongyang Ma, Pengfei Wan, Di Zhang, Xiaodong Chen, and Jing Liao. Direct-a-video. Customized video generation with user-directed camera movement and object motion. In ACM SIGGRAPH 2024 Conference Papers, pp. 1-12, 2024a.
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang. Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with expert transformer. arXiv preprint arXiv:2408 06072, 2024b.
Weiran Yao, Yuewen Sun, Alex Ho, Changyin Sun, and Kun Zhang. Learning temporally causal latent processes from general temporal data. arXiv preprint 1 2021.
Weiran Yao, Guangyi Chen, and Kun Zhang. Temporally disentangled representation learning. Advances in Neural Information Processing Systems, 012264922-6503,, 2022.
14

---

Published as a conference paper at ICLR 2026
Sihyun Yu, Jihoon Tack, Sangwoo Mo, Hyunsu Kim, Junho Kim, Jung- Woo Ha, and Jinwoo Shin. Generating videos with dynamics-awann aware implicit generative adversarial networks. arXiv preprint
arXiv:2202.10571, 2022.
David Junhao Zhang, Dongxu Li, Hung Le, Mike Zheng Shou, Caiming Xiong, and Doyen Sahoo. Moonshot: Towards controllable video generation and editing with motion-aware multimodal conditions. International Journal of Computer Vision, pp. 1-16, 2025.
Kun Zhang, Zhikun Wang. Jiji Zhang, and Bernhard Schölkopf. On estimation of functional causal models: general results and application to the post-nonlinear causal model. ACM Transactions on Intelligent Systems and Technology (TIST), 7(2):1-22, 2015.
Zangwei Zheng, Xiangyu Peng. Tianji Yang. Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all, March Sora.
2024. URL https:t///// /
com /tte:://gitttit github.thh....g.gg httpuu..co//pcccte www.///pcaattccc/eeoecoc
Daquan Zhou, Weimin Wang, Hanshu Yan, Weiwei Lv, Yizhe Zhu, and Jiashi Feng. Magicvideo: Efficient video generation with latent diffusion models. arXiv preprint arXiv:2211.11018, 2022.
Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learning view synthesis using multiplane images. arXiv preprint arXiv:1805.09817 2018.
Hao Zhu, Wayne Wu, Wentao Zhu, Liming Jiang, Siwei Tang, Li Zhang, Ziwei Liu, and Chen Change Loy. CelebV-HQ: A large-scale video facial attributes dataset. In ECCV, 2022.
Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Zilong Dong, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image animation with 3d parametric guidance. In European Conference on Computer Vision, pp. 145-162. Springer, 2025.
15

---

Published as a conference paper at ICLR 2026
Appendix for
"Controllable Video Generation with Provable Disentanglement"
A IDENTIFIABILITY THEORY
A. PROOF
Without loss of generality, we first consider the case where Pa(zī,;) z meaning that the time-dependent effects are governed by the dynamics of the previous time step.
Theorem A1 (Blockwise Identifiability). Consider video observation V = X2..X XT gener- ated by process (g.f" .f .p" ppp ,p') with latent variables denoted as z and Ze, according to Equation 1, where X2 E 1 Zi E R
z" "R If assumptions
Bl (Positive Density) the probability density function of latent variables is always positive and bounded;
B2 (Minimal Changes) the linear operators Txx 1/2 x and 1xx-118441 are injective for bounded function space;
11x041
B3 (Weakly Monotonic) for any 21, Zt Z x 2 (2. + 2.). the set {x, p(x (2,) + p(x,(Za)} has positive probability, and conditional densities are bounded and continuous;
are satisfied, then z, is blockwisely identifiable with regard to 2n from learned model (g.s" I '.p"... )0) under Observation Equivalence.
Proof. We first prove the monoblock identification of zi 2 h(22,, 2¹), then we prove the blockwise identification z' (2%)
Monoblock Identification. Following (Hu & Shum, 2012: Hu & Schennach, 2008b), when assump- tions B1, B2, B3 satisfied, the blockwise identifiability of [z], 2 assured, according to Theorem 3.2 (Monoblock identifiability) in (Fu et al., 2025). In short ,there exists invertible function g such that [zf, 2 h22,, &²), where [:] denotes concatenation.
Identification of c We prove this by contradiction. Suppose that for any 20, we have
z he (2, 2").
where there exist at least two distinct values of 2; such that z takes different values. When observational equivalence holds, the function remains the same for all values of t:
( , 20) = 9. (x,) (g_² (2), (27,2%) 20),
(8)
where g first demix x, then extract the content part, defined in Equation 2.
For any two distinct * 1', we have
= =2= (2,,,22")
(9)
which holds for all pairs (2), 27.) within the domain definition.
According to Assumption B1, the joint distribution p(2j, 27.) is always positive. Thus, to satisfy Equation 9, 2 must not contribute to z" through hg. In other words, we obtain
z (29).
(10)
Theorem A2 (Component-wine wise Identifiability). Consider video observation V = {x1, X2...., X1
generated by process (g.f f pp ppppp") with latent variables denoted z and z according to Equation where X, E R. z E R²,, R Suppose assumptions in Theorem Al hold. If
assumptions
16

---

Published as a conference paper at ICLR 2026
CI (Smooth and Positive Density) the probability density function of latent variables is always third-order differentiable and positive;
C2 (Sufficient Changes) let Mi,
- log p(2i,l/zz and
T..
T 0 12,1021-11
V1,1 -
(11)
@zz110z1111
for E (1,2,-.., n... For each value of lt. there exists 2n, different of values of 2t-1, such that the 2n, vector Vt,1 R2n, are linearly independent:
C3 (Conditional Independence) the learned 27 independent with V and all entries of 2 are mutually independent conditioned on 27-1i
are satisfied, then zi is component-wisely identifiable with regard to 2f from learned model (g.f fp'pppp p') under Observation Equivalence.
Proof. According to Theorem A1, we have
[ x h(2,, 2²)),
(12)
where [-] denotes the concatenation operation. The corresponding Jacobian matrix can be formulated
as
He
02
5
ix
(13)
82
82
Consider mapping from (x--1, 29) to (x--1, zi; 2%) and its Jacobian matrix
I
0 H
(14)
where stands for any matrix, and the absolute value of the determination of this Jacobian is He]- Therefore p(x+-1, zi, ) = x(xt-111 1, 27, / Dividing both side by p(x:-1) gives
pl 2 xt-1) ) = p(2,, (xxxx11)//
p(z 2 1g 2/ 1 2)) = =p(2,, 2 (2; //)
pl 2 zi-11 2²)
= 2 2:.2-111 2²)) //)//H,)
(15)
pzz 2 1²2)
= p(τ, 2, 1,, 1
pl p = p(2; 2 H),
For the first two implications, utilize the inversion of the mixing function replace the condition. For the third, since z is conditioned on itself, it remains fixed. For the last one, given that in the generating process, <t,, and Eș for all j, t are independently sampled, their disjoint successors also independent, i.e., z =,|2ţ_1 Thus, we can remove z" from the condition.
z Similarly, following assumption C3, have 2 22 1 2
(2),)) For simplicity, denote n7. - logg(ziizi_)) and Me, A log (z)_1) and we have
T/ + log Hr].
(16)
For any two different 27 2t, E z in partial derivative with regard to 2i, gives
On dzf,e
Oi, aog H.l
(17)
D2g 02,
d2g
Reorganize the left-hand side of Equation 17 with mutual independence of z 2 yields
On. dzi
dzi
0 T
0zi
=
OT., 0z
(18)
02
0z
027,
d2i
k=1
k=
17

---

Published as a conference paper at ICLR 2026
and we have
mmck
Ōzi,
die.
olog H4
(19)
Ozh
02g,
0
02g,
Further get the second-order derivative with regard to 27,j as
T
dzi,k
Ozi
82
0² log [ee]
(20)
J2 7
Ozy
02, O
80. Next, using the mutual independence of م in assumption C3, we have 322, 027 to the connection between conditional independence and cross derivatives (Lin, 1997). Thus we have
0 according
8 Pzf
dzi,k der,
The
02 log HPl
(21)
dzū
0
027, 027
Now we get the third-order derivative with regard any 21-1,1 as
Ozū
Om
0z
0,
(22)
02T-11
dzi
02,, 02,
where we use the property that the entries of He do not depend on 2/-1.1-
Given assumption C2, there exists 2n, different values of zi-1, such that the 2n, vectors Va, linearly independent. The only solution to Equation 22 is to set
Oz dzi 0zi
02 2
0. 0
(23)
02t,
According to Theorem A1, the blockwise identifiability is established. Thus,
H.
02
(24)
dz.
On
invertible, with 82 05,
and On! 0x;
has most one nonzero element in each row and each column.
Thus, we have
H.
(25)
Ox.
79
dz; and must have one and only one non-zero entry in each column and row. d2:
A.2 DISCUSSION OF ASSUMPTIONS
In this section, we give brief discussion of the assumptions in the real world scenarios.
Assumption B1 requires a smooth distribution, which is usually hold in the real world. Secondly, Assumption B2 imposes minimal requirement on the number of variables. The linear operator Taja ensures that there sufficient variation the density of for different values of a, thereby guaranteeing injectivity. In video, is of much higher dimensionality compared to the latent variables. As result, the injectivity assumption easily satisfied. In practice, following the principle of minimal changes, if model with fewer latent variables can successfully achieve observational equivalence, it more likely to leam the true distribution. Assumption B3 requires the distribution of x, changes when the value of latent variables changes. This assumption is much weaker compared to the widely used invertibility assumption adopted by previous works, such as (Yao et al., 2022). The aforementioned assumptions concern the underlying data generatnnn process. The aforementioned
18

---

Published as a conference paper at ICLR 2026
assumptions pertain the underlying data-generating process, and are often easily satisfied in real- world video scenarios. Given a model that can explicitly disentangle motion and identity while maintaining high generative fidelity, block-wise disentanglement becomes natural and attainable property under these conditions. This also explains why models such as MoCoGAN (Tulyakov et al., 2018) are able to effectively separate motion and identity.
Assumption C1 further requires the underlying distribution to be smooth, while Assumptions C2 and C3 impose mild conditions on the variability of latent factors. These assumptions are overly restrictive and are often satisfied in practice. Notably, even when the sufficiency conditions are satisfied only by subset of latent variables, or when mutual independence is partially violated, identifiability can still be achieved at the subspace level, as demonstrated by (Kong et al., 2023) in Section A. To further attain disentanglement at the component- wise level, the architectural design of our proposed Temporal Transition Module (TTM) is essential.
A.3 EXAMPLES OF INJECTIVE LINEAR OPERATORS
The assumption that linear operator is injective is commonly used in nonparametric identifica- tion (Hu & Schennach, 2008b; Carroll al., 2010; Hu Shum, 2012). Intuitively, this means that distinct input distributions of linear operator correspond to distinct output distributions. To clarify this assumption, we provide several examples illustrating the mapping from Pa + Pb. where a and random variables.
Example (Inverse Transformation). g(a), where g is an invertible function. Example (Additive Transformation), b = a + 6, where the distribution p(e) must not vanish entirely under the Fourier transform (Theorem 1 in (Mattner, 1993)).
Example b = g(a + e, requiring the same conditions as Examples 1 and 2.
Example 4 (Post-linear' Transformation). = 91 (g)(a) + e). a post-nonlincar model with invertible nonlinear functions 91 and 92. combining the assumptions in Examples 1-3.
Example (Nonlinear Transformation with Exponential Family). b = g(a e), where the joint distribution p(a, b) belongs to an exponential family.
Example 6 (General Nonlinear Transformation). b = g(a, e), representing general nonlinear mapping. Certain deviations from the nonlinear additive model (Example 3), such as polynomial perturbations, still be tractable.
A.4 EXTENSION TO NONSTATIONARY SCENARIOS
When the generative process is nonstationary, meaning that all latent variables conditioned on exogenous variable u, we obtain the nonstationary form Equation 1:
= S (Pa(2 ,), C,, u), 2/ = (c)
(26)
where u may represent weak supervision or textual guidance. As stated in Theorem of (Yao et al., 2022), the identifiability results easier establish because variations in help better satisfy Assumption C2. More precisely, Equation 11 can be rewritten as
Vt.1 A 02 (mc,1 (w) - (u'))
( (u) - Me,n, (,n)(")))
2z 02,1021-111
Jz,,, 0z-111
(27)
8 (m. (2)))
2 (M... (u) T7 Mnn, (u')
821,002-111
8 82 021-1,1
where (u) A pog(( zi-1. u), and u and w' denote two different exogenous conditions.
A.5 EXTENSION TO CONTENT AND STYLE ENTANGLEMENT
For example, the motion patterns of fish differ from those of human, suggesting that =f could affect z². In such cases, z can still reformulated as a function of df and an independent noise term (Zhang et al., 2015). Consequently, the noise term can be regarded as the new dynamic variable z² Therefore, the video can still be interpreted as being generated from a fixed zf and a varying, independent ~
19

---

Published as a conference paper at ICLR 2026
B RELATED WORKS
B.1
CONTROLLABLE VIDEO GENERATION
Recent advances in controllable video generation have led to significant progress, with text-to-video (T2V) models (Yang et al., 2024b; Singer et al., 2022; Ho et al., 2022; Zhou et al., 2022: Zheng al., 2024) achieving impressive results in generating videos from textual descriptions. However, effectiveness of the control is highly dependent on the quality of the input prompt, making it difficult achieve fine- -raiind control over the generated content. An alternative method for control involves leveraging side information such as pose (Tu et al., 2024; Zhu et al., 2025), camera motion (Yang al., 2024a; Wang et al., 2024), depth (Liu al., 2024; Xing al., 2024) and so on. While this approach allows for more precise control, it requires paired data, which can be challenging to collect. Besides, most of the aforementioned alignment-based techniques share a common issue: the control signals are directly aligned with the entire video. This issue not only reduces efficiency but also complicates the task of achieving independent control over different aspects of the vidco, which further motivates us to propose framework to find the disentanglement representation for conditional generation.
B.2 NONLINEAR INDEPENDENT COMPONENT ANALYSIS
Nonlinear independent component analysis offers potential approach to uncover latent causal variables in time series data. These methods typically utilize auxiliary information, such as class labels or domain-specific indices, and impose independence constraints enhance the identifiability of latent variables. Time-contrastive learning (TCL) (Hyvarinen & Morioka, 2016) builds on the assumption of independent sources and takes advantage of the variability in variance across different segments of data. Similar Permutation-bast 2017) introduces a learning framework that tell true independent sources from their permuted counterparts. Additionally, i-VAE (Khemakhem et al., 2020) employs deep neural networks and Variational Autoencoders (VAEs) to closely approximate the joint distribution of observed data and auxiliary non-stationary regimes. Recently, (Yao et al., 2021: 2022) extends the identifiability to linear and nonlinear non-Gaussian cases without auxiliary variables, respectively. CaRiNG (Chen al., 2024) further tackles the case when the mixing process is non-invertible Additionally, CITRIS (Lippe et al., 2022; 2023) emphasizes the use of intervention target data, and IDOL (Li et al., 2024c) incorporates sparsity into the latent transition process identify latent variables, even in the presence of instantaneous effects.
contrastive learning (PCL) (Hyvarinen & Morioka,
C DATASET DETAILS
We use the following datasets verify our model.
FaceForensics (Rössler et al., 2018): A forensics dataset consisting of 1,000 original video sequences. All videos contain a trackable, mostly frontal face without occlusions.
SkyTimelapse (Xiong et al., 2018): Typically consists of sequential images videos capturing the dynamic behavior of the sky over time. We use the 2,368 officially released videos.
RealEstate (Zhou et al., 2018): large dataset of camera poses corresponding to 10 million frames from about 80,000 video clips, gathered from around 10,000 YouTube videos. Each clip's 's poses form a trajectory specifying camera position and orientation, derived via SLAM and bundle adjustment. To our knowledge, our method is the first GAN- approach to leverage this dataset for unconditional video generation.
Celeb (Zhu et al., 2022): A large-scale, high-quality video dataset with diverse celebrity identities and actions. It contains 55,666 video clips with minimum resolution of 512x512, covering 15,653 identities.
D METRIC DETAILS
We the following metrics to verify model.
20

---

Published as a conference paper at ICLR 2026
FVD (Fréchet Video Distance (Unterthiner et al., 2019)) is metric for evaluating video generation quality. compares real and generated videos by extracting spatiotemporal features with pretrained model and computing the Fréchet distance between their feature distributions.
SAP (Separated Attribute Predictability (Kumar et al., 2017)) measures how well each latent dimension is associated with a single ground-truth factor. It's computed by training simple regressor to predict true factors from each latent dimension, then comparing the two best-performing dimensions. A higher score means better disentanglement.
Modularity (Ridgeway & Mozer, 2018) measures whether each latent dimension encodes information about at most one factor. It penalizes when single latent variable car- ries mixed information about multiple true factors. Perfect modularity means each latent dimension corresponds only to single factor.
MCC (Mean Correlation Coefficient) is a commonly used disentanglement metric in repre- sentation learning. Let Z Rº be the ground-truth latent vector and Ż R 5 the estimated vector. To calculate MCC, we first compute the Pearson correlations Rıj corr (2,, 2z), then select an injective matching ı {l...., D) + {I....,D) maximizing Σ R Finally, the MCC value is defined MCC 1 | R.
E REPRODUCTABILITY
All of our models are trained NVIDIA A100 40G GPU. Our model requires around throughput of 20 million frames for convergence. For the baseline models, we use the official implementation with the default hyperparameters. The configuration of hyperparameters for model training is as shown in Table 4.
Table 4: Configuration details for the model and training setup.
Name
Variable Name
Value/Description
Information regularization term weight Dimensionality
lambda_Ki KL.
512
Dimensionality GRU
Dimensionality æ
FaceForensics: 4. SkyTimelapse: 12, RealEstate:
Batch size (per GPU) Conditional mode Flow normalization Sparsity weight
batch
cond_mode
flow
flow_nomn
Lumbda_sparse
Number input channels Number mapping layers
channel
num_ num_layers
Label embedding features Intermediate layer features Activation function Learning multiplier
embed_ features
512
layer_ features
512
activation
Irelu
multiplier
0.01
Moving average decay Discriminator architecture
__avg_hee heta
0.995
resnet
Channel base Maximum number of channels
channel base
32768
channel
512
The architecture our proposed Temporal Transition Module is shown in Table
F MORE EXPERIMENTS
F. LONGER VIDEOS
We also verify methods generating longer videos with 32 frames the FaceForensics Dataset, as shown Table Our method achieves substantially better performance than MoStGAN-V on disentanglement metrics.
21

---

Published as a conference paper at ICLR 2026
Table 5: Architecture of Temporal Transition Module.
Component
Structure
MappingNettorkk.e.
x 256 x
Mapping.Network.h.co
FullyConnectedLayer (256 x 64)
Mapping.Network.cmbeee
FullyConnectedLayer (64 x 512)
DenseSigmoidFlow
DenseSigmoidFlow
MappingNeerwork.foww
FullyConnectedlayee (512 x 512)
FullyConnectedllayy Layer (512 x 512)
Layer
MappingNNttorrk.Ing fc2
FullyConnectedLayer (512 x 284)
FullyConnectedLayce
MullyConnectedLayer (512 x 512)
FullyConnectedLayer
FullyConnectedLayer (512 x 512)
Table 6: Performance on FaceForensics with length=32.
Metrics
MoSIGAN-V
CoVoGAN
FVD +
159.24
145.77
MCC (%) 1
14.23
29.99
SAP (%) t
0.87
4.20
Modularity (%)
8.15
10.14
F.2 CHOICE OF nss
In CoVoGAN, the dimension of n, is treated as tunable hyperparameter. While the choice of n. is important, it turns out to be not overly sensitive. From a theoretical standpoint, assuming a known latent dimensionality is standard in the nonlinear ICA literature (Hyvarinen & Morioka, 2016; Kong et al., 2023). Moreover, even when n, is not specified exactly, the model can still yield meaningful results: if nt is set too large, the model tends fit some noise, whereas it is set too small, certain components may be merged. This explains why our empirical results show robustness with respect to n, As presented in Table 7, varying it within a reasonable range does not lead to significant performance changes.
Table 7: Ablation n, the FaceForensics dataset.
71
4
FVD
48.8
47.9
F.3 SUPERVISED VIDEO GENERATION
In this paper, we propose method for controlling video generation without relying specific supervisory signals such as text trajectories. Our goal like text-to-video or trajectory-guider models, but rather to explore the feasibility of achieving disentangled video generation in the absence of explicit supervision. This setting considerably more challenging, as annotations are typically task-specific and costly to obtain. Moreover, when appropriate supervisory signals such as text or trajectories are available, our method can combined with them to enhance controllability through an identifiable generative process.
to surpass supervised approaches
To further strengthen our evaluation, we have integrated our TTM module into two additional models: the diffusion-based Text-to- Text-lo-Video generation model, Wan (Wan al., 2024), and the trajectory- supervised model, MotionCtrl (Wang et al., 2024).
For text-to-video generation, integrate TTM module into the Wan-1.3B model and finetune on a subset of WebVid-100 (Bain et al., 2021), using the first 10% of the data. We conduct a thorough comparison against the original Wan-1.3B model and CogVideoX-2B (Yang al.. 2024b). The results, presented in Table 8, were evaluated using the VBench (Huang et al., 2023) toolkit for the first six metrics, and three different implementations of CLIP similarity: hf _f_p b32,
22

---

Published as a conference paper at ICLR 2026
hf clip vit 114 (Radford et al., 2021), hf laion clip_ vit b32 (Rössler et al., 2018) for the controllabil- ity measures. While our model shows a slight decrease in some metrics compared to Cog VideoX-222 (e.g,, hf clip vit b32), it still outperforms the original Wan-1.3B model, especially to the aspect of controllability. Examples of generated videos are provided in Figure 9.
Table 8: Comparison of text-to-video generation across multiple metrics (larger is better).
Metric
VogVideuX-22 -B
Wan-1.3B
TTM + Wan-1.3B
boaging_qualit f
0,4870
0.6851
0.6877
motion_ssme smoioonsmmo
0,9886
0,9874
0.9935
dynamic_dege _eegreeeeeeeeeeeeeeeeee f
0,3465
0,3960
0.4059
Subject_consistenge
0,9347
0,9324
0.9529
background consistency besthetic_qualin
Eaigground_coossstennge
0,9585
0,9375
0,9628
0.3849
0.5370
0.4924
hf_ mf_clip_vit_b322
0.2428
0,2234
0,2288
hf hf_clip_wi hitp__it______ [iir_114+ 1
0.2007
0.1788
0.1892
hf laion clip vit b32
0,1859
0,1791
0.1980
We also integrate our TTM module into the MotionCtrl model and conducted a comparison on the RealEstate IKK dataset, with regard to video generation conditioned on trajectory of camera poses. As shown Table 9, our TTM module successfully improves performance of the MotionCtrl model.
Table 9: Comparison of CotionCtrl and TTM + MotionCtrl across multiple metrics (lower better).
Metric
MotionCtrl
TTM MotionCtrl
CamMC
0.0840
0.0776
FID
130.29
129.05
FVD
934.37
917.28
Since both baseline models are diffusion-baseee rather than GAN-based, we integrate the TTM module through pross-attention as a conditioning mechanism, following a similar strategy to that used in an identifiable image generation model (Xie et al., 2025). Although this design is not fully aligned with our theoretical framework, highlights promising direction for future investigation.
F.4 COMPUTATIONAL EFFICIENCY
We also compare the computational efficiency with that of the baselines. The size of our model's generator comparable that of StyleGAN2-ADA significantly smaller than the generators of the baselines. Additionally, the inference time of our model is much faster, as shown in Table 10.
F.5 MORE VISUALIZATION RESULTS
In this section, we provide more qualitative video results generated by approach. As shown in Figure 6, we compare the generative quality of the videos among all models. As can be seen in Figure our method can control different identities of sky scenes with consistent constructed motions.
F.6 EXPERIMENTS ON HUMAN MOTION DATASET
We also conduct experiments on more TaiChi (Sun et al., 2017), a more complex dataset with richer human motion dynamics. We compare FVD, MCC, SAP, and Modularity on the TaiChi 64 dataset. For the disentanglement metrics, we use the open-source package MediaPipe to extract human motion keypoints as ground-trui variables. However, for MoStGAN-V. MediaPipe fails to extract any keypoints from the generated videos due poor generation quality, making the disentanglement metrics inapplicable for this method. The quality results and quantity results can be found in Figure and Table 11 respectively.
23

---

Published as a conference paper at ICLR 2026
Table 10: Comparison of different models in terms of generator parameters (Params, millions) and the time (Time, in seconds) required to generate 16- (6-frame 2048 videos with resolution of 256 x 256.
Method
Params (M)
Time (s)
StyleGAN2-ADA
23.19
DIGAN
69.97
142.06
StyleGAN-V
32.11
181.45
MoStGAN-V
40,92
207.11
Latte
757.38
45340.74
CoVoGAN (ours)
24.98
98.93
Table 11: Comparison of StyleGAN-V, MoSIGAN-V, and CoVoGAN on Taichi.
Metrics Taichi
StyleGAN-V
MoSIGAN-V
CoVoGAN
FVD
68.52
60.39
55.3
MCC (%) f
18.2
42.3
SAP (%) f
1.5
3.2
Modularity (%)
0.2
1.9
F.7 EXPERIMENTS ON INDEPENDENT UTILITY OF 2" AND 2d
In this subsection, we verify that the =" and =² can be utilized independently to generate videos. We have added the following ablation experiments: we first randomly sample 2" and z" independently from the learned distribution, then for each z", we use different 2" to generate videos, and vice versa. We compare the FVD and disentanglement metrics and both show that the two representations can be used independently to generate videos with good performance. The results are shown in Table 12.
F.8 EXPERIMENTS WITH MULTI-DOMAINS
To further verify our model on a more complex dataset with multiple domains, we conduct experiments UCF-101 (Soomro al., 2012), which contains realistic videos of 101 human action classes. The dataset includes over 13k clips and a total of 27 hours video data. We evaluate our model with an increasing number of action categories to demonstrate its ability to generalize across multiple-domain scenarios.
We train and evaluate both our model and the baseline conditional generation models, following the generative process described in Equation 26. Experiments are conducted on three subsets of the UCF-101 dataset: classes (ApplyLipstick. Body WeightSquats,, PlayingGuitar)) 20 classes (BabyCrawling. BalanceBeam, Billiards, BlowingCandles, Body WeightSquatss, BoxingSpeedBag, Fencing, FieldHockeyPenalty, FloorGyymastics, FrontCrawl, Hammering, MoppingFloor, PlayingSi- tar, PullUps, PushUps, Rowing, ShavingBeard, Skijet, SumoWrestling, Typing), and all 101 classes. downsample the resolution to 6422 due to GPU shortage. Since is impractical to use keypoints as ground truth for controllability evaluation when multiple actions are involved, we instead assess controllability by measuring the classification accuracy of generated videos with respect to their target action classes. The classifier is implemented as simple CNN. The performance results summarized in Table 13. The baseline model crashed during training when class number is larger than so we list the results 3 classes only. Our model outperforms the baseline model for both fidelity and controllability. Randomly sampled videos trained on 3 classes setting are shown in Figure 10.
F,9 EXPERIMENTS ON BLOCKWISE IDENTIFICATIO
To evaluate the blockwise identification, we further assess the consistency of motion when only the content latent code is modified. Experiments are conducted the FaceForrensics dataset (256"). Specifically, we randomly sample a video and then alter its z to generate another video. We extract facial keypoints from both videos using dlib and compute the average keypoint distance (AKD) between them. lower AKD indicates stronger disentanglement between content and motion. The results summarized in Table 14.
24

---

Published as a conference paper at ICLR 2026
LVDM MoSIGAN-Y StyleGAN- DIGAN
Latte
CoVoGAN
LVDM MoSIGAN-V StyleGAN-V DIGAN
Latte
CoVoGAN
CoVoGAN LVDM MoSIGAN-V StyleGAN-V DIGAN
Figure 6: Random samples from the comparison baselines and our model on real-world datasets
FaceForensics 2562. SkyTimlapsse 256², RealEsstate 256 Start from t= 0 and report every second frame from 16-frame video clip.
25

---

Published as a conference paper at ICLR 2026
Table 12: Extrapolation ablation study on FaceForensics.
Metrics
Vanilla CoVGA
Varying =*
Varying 2
MCC (%) t SAP (%) f
33.78
36.89
31.76
8.48
8.71
10.39
Modularity (%
17.37
14.75
11.29
StyleGAN-V
MoSIGAN-V
CoVoGAN
Figure 7: Generate same motion of different identities on SkyTimelapse Dataset.
G GANS VERSUS DIFFUSION MODELS
In this section, we explain the reason that we adopt the GAN framework the primary implementation in our work. The key difference between our GAN-based framework and diffusion-based approaches lies mainly the training objective rather than the architecture. illustrate, consider an ideal scenario where both videos and their true latent factors are observable. In this fully supervised case, the model could be trained in a straightforward, sample-to-sample manner. However, under the unsupervised setting considered in this work. the latent variables are unobserved, and the model can
Class No.
Metric
FVD
Accuracy
MoSIGAN-V
897.86
0.35
3
CoVoGAN
178.03
0.64
20
CoVoGAN 790.23
0.10
101
CoVoGAN 1050.61
Table 13: Experiments on UCF-101 with different numbers action classes. FVD evaluates video quality, and classification accuracy measures controllability of the generated videos.
26

---

Published as a conference paper at ICLR 2026
StyleGAN-V
MoSIGANA
CoVoGAN
Figure 8: Generate same motion of different identities Taichi Dataset.
Prompt: explains surgery using a plaster mandible, hulding tools while speaking with an elderly patient.
Text Prompi: A chestah the ground, licking lips while scumning surnoundings
Figure Controllable text-to-vided generation results.
learn the mapping by instead ensuring observational equivalence, i.c., aligning the distributions of generated and real videos. This naturally motivates the use of GAN framework, which enforces such distributional alignment.
Moreover, theoretical guarantees established in framework are also compatible with diffusion- based generative processes. In practice, one potential solution is to adopt a GAN-style loss for matching the marginal video distributions while retaining ddiffusion-hase architecture. Alternatively,
27

---

Published as a conference paper at ICLR 2026
Figure 10: Random samples of generated videos with class: ['ispkyLipstick' 'ulayingGuitar', "Body "eughtWegstSsutss from top to bottom.
Method
AKD
StyleGAN-V
19.53
MoSIGAN-V
15.86
CoVoGAN (Ours)
6.45
Table 14: Comparison of AKD on the FaceForensics dataset. Lower means better content-motion
disentanglement.
it is possible to predict the independent noise component directly from video sequences, enabling sample-to-sample training that preserves extra supervision signals, as demonstrated (Xie et al., 2025). We implement the latter one in experiments in Appendix F.3.
H IMPACT STATEMENTS
This study introduces both theoretical framework and practical approach for extracting disen- tangled causal representations from videos. Such advancements enable the development of more transparent and interpretative models, enhancing grasp of causal dynamics in real-world settings. This approach may benefit many real-world applications, including healthcare, auto-driving, content generation, marketing and
28


Reviews:
[{"review_id": "pFFJm8j0OB", "reviewer": "Reviewer_9LG2", "summary": "This paper presents Controllable Video Generative Adversarial Networks , a novel framework designed to address the challenge of fine-grained control in video generation by leveraging provable disentanglement. The authors introduce a Temporal Transition Module that separates static content elements and dynamic style dynamics, grounded in theoretical guarantees of block-wise and component-wise identifiability. By integrating TTM into a GAN architecture, CoVoGAN achieves disentangled representations through the \"minimal change principle\" and \"sufficient change property,\" enabling independent manipulation of motion components like head movement and eye blinking. Extensive experiments on datasets including FaceForensics and RealEstate demonstrate superior performance in both generation quality  and controllability  compared to baselines.", "questions": "The method in this paper belongs to unconditional video generation. Can it be applied to text-to-video generation models? The latter has greater practical application value.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper provides a strong theoretical foundation by proving block-wise and component-wise identifiability under mild assumptions. This formalizes why disentanglement between motion and content is achievable, addressing gaps in prior heuristic-based methods. The TTM effectively combines GRU for temporal dependency modeling and Deep Sigmoid Flow for conditional independence, ensuring dynamic variables are disentangled and controllable. Ablation studies validate the critical role of GRU and flow components in maintaining performance.", "weaknesses": "While CoVoGAN excels in unsupervised disentanglement, it does not address how to align latent dimensions with semantic labels (e.g., explicitly mapping a dimension to \"eye blinking\"). Integrating weak supervision or text guidance could enhance interpretability and usability. \nAdditionally, the article lacks experiments on the UCF101 dataset, and it is recommended to supplement them.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wtivxGf2zS", "reviewer": "Reviewer_S4AE", "summary": "The authors propose Controllable Video Generative Adversarial Networks (CoVoGAN), an approach designed to disentangle video concepts to facilitate efficient and independent control over individual factors. CoVoGAN is built upon rigorous theoretical insights derived from nonlinear Independent Component Analysis (ICA), establishing conditions for identifiability. It used to achieve component-wise identifiability of the dynamic latent variables, allowing for disentangled control within the motion itself. It proposed the Temporal Transition Module (TTM), integrated as a plug-in into a GAN architecture (based on StyleGAN2-ADA). Extensive experiments across various datasets (including FaceForensics, SkyTimelapse, RealEstate, and CelebV-HQ) demonstrate that CoVoGAN improves both generative quality (measured by FVD) and controllability (measured by SAP, modularity, and MCC) compared to existing video generation models.", "questions": "see weaknesses", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The model successfully achieves both block-wise identifiability (separating content from motion) and component-wise identifiability (separating distinct factors within motion).\n2. CoVoGAN achieves the best performance across disentanglement metrics (MCC, SAP, Modularity) compared to strong baselines like StyleGAN-V, MoStGAN-V, LVDM, and Latte. \n3. CoVoGAN is computationally efficient, having a generator parameter count (24.98M) comparable to StyleGAN2-ADA and significantly smaller than other baselines.", "weaknesses": "1. The primary implementation and theoretical guarantees are grounded in the GAN framework, how to integrate the idea into current diffusion-based models? Compared to diffusion models, GAN-based approaches still have large visual fidelity gap.\n2. Experiments only focus on domain-specific datasets, how will be the performance of CoVoGAN on UCF101 or even larger dataset such as Kinetics? It would be better if the author could show the abilities of the model on more complex datasets.\n3. Can CoVoGAN be used in long-term video generation?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "p5vvICEYFz", "reviewer": "Reviewer_KQV9", "summary": "This paper introduces CoVoGAN, a video generation model that achieves fine-grained control through theoretically-grounded disentanglement. The approach separates videos into static content (identity) and dynamic factors (motion), with component-wise separation of individual motion concepts (e.g., head movement vs. eye blinking). Experiments across multiple datasets show superior video quality and controllability compared to baselines, enabling precise independent manipulation of different video aspects.", "questions": "None.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. This paper provides strong theoretical foundation. It provide rigorous identifiability theorems for video generation, offering mathematical guarantees for disentanglement at both block-wise and component-wise levels.\n\n2. This paper enables independent manipulation of specific motion concepts (e.g., eye blinking, head movement) without affecting content/identity, demonstrated across multiple datasets.\n\n3. The evaluation is thorough across different metrics and datasets.", "weaknesses": "1. The paper assumes a strong assumption: videos decompose cleanly into content (which is static) and dynamic motion. However, many real-world vidoes don't fit this dichotomy. For example, deformable objects have constant but moving parts. The identity and motion are entangled. Lighting changes, and shadow moving cannot fit into these two categories either. I am concerned about the limited generalizability of the proposed method to broader video domains.\n\n2. Performance on complex human motion (TaiChi dataset) shows room for improvement, suggesting challenges with highly dynamic scenes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cARgrI4AlE", "reviewer": "Reviewer_7TFq", "summary": "The paper introduces CoVoGAN, a GAN-based framework for controllable video generation with provable disentanglement between content (static) and motion (dynamic) factors. the authors propose to add a light Temporal Transition Module (GRU + component-wise flow) in front of a StyleGAN-like generator, plus a video discriminator and an MI term to tie dynamics to what is being produced.", "questions": "See above.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The authors provided solid theory to back their claims\n- The paper is well-written and easy to follow", "weaknesses": "- The authors provide a disentanglement comparisons between all method, yet it is likely favor methods with compact latents (i.e Diffusion models vs GANs)\n- while the author demonstrate independence between content and motion, I could not find any table quantifying this improvement/ \n- The method is based of GANs. However, due the popularity of diffusion models recently, the contribution on GANs might be undermind.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)