EasyTune: Efficient Step-Aware Fine-Tuning for Diffusion-Based Motion Generation
Authors: Xiaofeng Tan
Wanjiang Weng
Haodong Lei
Hongsong Wang
Venue: ICLR 2026 Poster (2026)
Keywords: RLHF
Motion Generation
Differentiable Reward
Primary area: generative models
TL;DR: We propose EasyTune, a fine-tuning framework for diffusion models that decouples recursive dependencies and enables (1) dense and effective optimization, (2) memory-efficient training, and (3) fine-grained alignment.

Abstract: In recent years, motion generative models have undergone significant advancement, yet pose challenges in aligning with downstream objectives. Recent studies have shown that using differentiable rewards to directly align the preference of diffusion models yields promising results. However, these methods suffer from (1) inefficient and coarse-grained optimization with (2) high memory consumption. In this work, we first theoretically and empirically identify the *key reason* of these limitations: the recursive dependence between different steps in the denoising trajectory. Inspired by this insight, we propose **EasyTune**, which fine-tunes diffusion at each denoising step rather than over the entire trajectory. This decouples the recursive dependence, allowing us to perform (1) a dense and fine-grained, and (2) memory-efficient optimization. Furthermore, the scarcity of preference motion pairs restricts the availability of motion reward model training. To this end, we further introduce a **S**elf-refinement **P**reference **L**earning (**SPL**) mechanism that dynamically identifies preference pairs and conducts preference learning. Extensive experiments demonstrate that EasyTune outperforms DRaFT-50 by 7.7% in alignment (MM-Dist) improvement while requiring only 31.16% of its additional memory overhead and achieving a **7.3$\times$** training speedup. The project page is available at this [link](https://xiaofeng-tan.github.io/projects/EasyTune/index.html).

Published as a conference paper at ICLR 2026
EASYTUNE: EFFICIENT STEP-AWARE FINE-TUNING FOR CNFFUSION-BASER MOTION GENERATION
Xiaofeng Tan" Wanjiang Weng" Haodong Lei
Hongsong Wang!
Department of Computer Science and Engineering, Southeast University, Nanjing. China
Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, Nanjing, China [xiao fengtan, wjweng, hongsong. wang ) @seu. edu. cn
ABSTRACT
In recent years, motion generative models have undergone significant advancement, yet pose challenges in aligning with downstream objectives. Recent studies have shown that using differentiable rewards to directly align the preference of diffusion models yields promising results. However, these methods suffer from (1) inefficient and coarse- coarse-grained optimization with (2) high memory consumption. In this work, we first theoretically and empirically identify the key reason of these limitations: the recursive dependence between different steps in the denoising trajectory. Inspired by this insight, we propose EasyTTnn which fine-tunes diffusion at each denoising step rather than over the entire trajectory. This decouples the recursive dependence, allowing us to perform (1) dense and fine-grained, and (2)
optimization. Furthermore, the scarcity of preference motion pairs restricts the availability of motion reward model training. To this end, we further introduce a Self-refinement Preference Learning (SPL) mechanism that dynamically identifies preference pairs and conducts preference learning. Extensive experiments demon- strate that EasyTune outperforms DRaFT-50 by 7.7% in alignment (MM-Dist) improvement while requiring only 31.16% of its additional memory overhead and achieving a 7.3x training speedup. The project page available at this link.
INTRODUCTION
Text-to-motion generation aims to synthesize realistic and coherent human motions from natural language (Shen et al., 2025), enabling applications in animation (Azadi et al., 2023), human-computer interaction (Peng et al., 2024), and virtual reality (Tashakori et al., 2025). Recent advances driven by diffusion models, which capture complex distributions and synthesize high-quality motions from text (Chen et al., 2023). However, their likelihood- training (Guo al., 2022a) often misaligns with downstream goals such as semantic understanding (Tan et al., 2025; Su et al., 2026), motion plausibility (Wang et al., 2024), and user preference (Tan et al., 2026).
To bridge this gap, reinforcement learning from human feedback (RLHF) (Kirstain et al., 2023) has been explored to fine-tune diffusion models toward human preferences and task-specific goals. Existing approaches include differentiable reward methods (Clark et al., 2024), reinforcement learning (Black et al., 2023), and direct preference optimization (DPO) (Wallace et al., 2024). Among these, DPO provides a effective way to align models using preference pairs. However, acquiring large-scale, high-quality preference pairs remains challenging due to the cost and difficulty of capturing nuanced semantic and preference signals. A more efficient alternative is to fine-tune models using a reward model that captures semantic alignment and task preference. Reinforcement learning methods, such as DDPO (Black et al., 2023) and DPOK (Fan et al., 2023a), treat the denoising trajectory Markov Decision Process, where intermediate motions are states and final motions are evaluated by a reward model. Differentiable reward methods, including DRaFT (Clark al., 2024) and DRTune (Wu et al., 2025a), directly backpropagate gradients from a differentiable reward R(x") ) to optimize the model 0.
Equual contribution. 'Corresponding author.

---

Published as a conference paper at ICLR 2026
0.300
Better quality
0.6
Better quality
Better Alignment
DRaFT-10
DRaF't T-50
0,5
0.250
AlipnProp
w/ Easy Tine
DRTune
0..4
PD
0.225
Easy Tume
FID
MLD MLD++
0.3
0.200
MotionLCMM
0.175
0.2
MM MDM
Easy Tune (Ours)
0.150
MotionDiftase
More Efficient
0.1
wi Resy
0.125
22.5
25.0
27.5
30.0
32.5
35.0
37.5
40.0
2.6
2.7
2.8
2.9
3.0
3.1
3.3
Memory (GB)
MM Dist
(a)
(b)
Figure 1: Comparison of the training costs and generation performance on HumanML3D (Guo al., 2022a). (a) Performance comparison of different fine-tuning methods (Clark et al., 2024; Prabhudesai et al., 2023; Wu et al., 2025a). (b) Generalization performance across six pre-trained diffusion-basee based models (Chen et al., 2023; Dai al., 2025; 2024; Tevet al., 2023).
However, these methods still face several primary limitations that hinder their application to diffusion- based motion generation: (1) Sparse and coarse-grained optimization: Most approaches only optimize model parameters 0 once after completing a multi-step denoising trajectory, resulting in sparse optimization signals and slowing down convergence. (2) Excessive memory consumption: These methods optimize the model 9 by backpropagating the gradients of the reward value (xR(x,), which is related the overall denoising trajectory. Notably, this requires storing a large computation graph of the entire trajectory, leading to excessive memory consumption. Beyond these computational challenges, existing methods rely intricate designs such early stopping partial gradient blocking, increasing implementation complexity and limiting applicability. Moreover, research on motion-specific reward models is limited, current approaches (Tan al., 2025) typically use general-purpose retrieval models, which may inadequately capture motion preferences.
Contributions, In this work, we first theoretically (Corollary 1) and empirically (Fig. 6) identify key factor of the significant computational and memory overhead: the optimization is recursively related the multi-ster denoising trajectory, causing the reward value generated motions, R(x"), to be recursively depended on each denoised step throughout the overall trajectory. Specifically, each denoised motion x generated from the diffusion model, x rs, and recursively dependent on previous steps x x Thus, computing the gradient Vax² requires solving for that of the prior step, Voxf, which in turn depends on those of subsequent steps, Vex., Vex..... Vox², leading to the large significant computational and memory overhead. Building on this, we then theoretically analyze and empirically validate (Fig.3) the primary limitation of existing methods: coarse- -aarse-r chain optimization leads to vanishing gradients, hindering optimization of early denoising steps. To address this, we introduce a simple and effective insight: perform optimization at each denoising step, thereby decoupling gradients from the full reverse trajectory. By decoupling gradients from the denoising trajectory, our EasyTune framework facilitates: (1) dense and fine-grain inn-grained optmization through clearing the computational graph after each denoising step, (2) avoiding storing them until denoising completes, and thus (3) obviation of the need for complex memory-saving techniques.
Nevertheless, two critical challenges remain: the lack of a reliable motion reward model and reward perception for intermediate denoising steps. The first issue stems from limited large-scale, high- quality preference data, making it difficult train a motion-specific reward model directly. To overcome this, we propose a Self-refinement Preference Learning (SPL., which adapts a pre-trained ext-to-motion retrieval model for preference evaluation without human annotations. We dynamically construct preference pairs from the retrieval datasets and fail-retrieved results, and fine-tune this retrieval model to capture implicit preferences. For noisy intermediate steps, we employ single-step prediction rewards for ODE-based models and noise-aware rewards for SDE-based models.
Finally, we evaluate EasyTune on HumanML33 (Guo et al., 2022a) and KIT-ML (Plappert et al., 2016) with six pre- tre-triined diffusion models. As shown in 1ig. EasyTune Tune achieves SoTA performance (FID = 0.132, 70.7 % better than the MLD base model), while cutting memory usage to 22.10 GB and achieving 7.3x training speedup over DRaFT. In summary, contributions are follows:
2

---

Published as a conference paper at ICLR 2026
1. We theoretically and empirically identify the cost and performance limitations of existing differentiable-
methods, and propose EasyTune, an effective step-aware fine-tuning method. 2. To the best of our knowledge, this work is the first to fine-tune diffusion- text-to-motion models with a differentiable reward. To achieve this, we introduce SPL to fine-tune pre-trained retrieval model for preference evaluation without human-annotated preference pairs.
3. Extensive experiments demonstrate that Easy Tune significantly outperforms existing methods in terms of performance, optimization efficiency, and storage requirements.
RELATED WORKS
Text-to-Motion Generation. Text-to-motion generation (Chen et al., 2023; Guo et al., 2023) produces human motion sequences from textual descriptions. Among these works, as a powerful generative model, diffusion models (Chen et al., 2023; Tevet et al., 2023) iteratively denoise latent motions under text guidance, offering higher quality and stability. Recent advances include transformer- diffusion with geometric losses (Tevet et al., 2023), few-step controllable inference (Dai et al., 2024), and hybrid discrete-continuous modeling (Meng et al., 2025). However, these methods primarily target the pretraining stage by aligning to fixed dataset distributions (Guo et al., 2022a), yet they remain misaligned with semantic coherence (Tan et al., 2025) and physical reality.
Post-training in Motion Generation. Post-training for motion generation improves semantic coher- ence (Tan et al., 2025; Pappa et al., 2024) and physical realism (Yuan et al., 2023; Han et al., 2025; Wang et al., 2025a). commonly uses motion discriminative models as reward signals (Petrovich et al., 2023; Weng et al., 2025a; Wang et al., 2025c;b; Weng et al., 2025b) and applies reinforcement learning to refine behaviors beyond supervised training. Representative methods include SoPo, which optimizes DPO objective from preference pairs (Tan et al., 2025), MotionCritic, which learns from human preferences with PPO (Wang et al., 2025a), ReinDiffuse with sule-based rewards (Han et al., 2025), and fake motion synthesis for improving motion detection (Tan et al., 2024). However, these methods often optimize within the pretraining reward domain, while the reward model is defined in a separate and opaque space (Janner et al., 2019; Yao al., 2022; 2024), which quality (Wang et al., 2025a) and increase dependence on preference data (Tan et al., 2025; Pappa et al., 2024). In contrast, we target semantic alignment in text-lo-motion generation with differentiable reward model and simple reinforcement learning algorithm, treating semantics as the primary objective.
limit feedback
Differentiable Reward Fine-Tuning for Diffusion Models. Fine-tuning pre-trained diffusion models with reward signals can be achieved via policy-gradient methods, DPO-based methods, or differentiable reward maximization. Policy-gradient methods (Wu et al., 2025b) such as DDPO and DPOK (Black al., 2023; Fan et al., 2023a) formulate denoising as an MDP and optimize expected rewards, while DPO-based methods such Diffusion-DPO (Wallace et al., 2024) and SoPo (Tan et al., 2025) optimize a preference objective from preference pairs. However, policy-gradient updates often rely on exactly computable likelihoods, which can be problematic for diffusion models (Zheng et al., 2025), and DPO introduces explicit dependence on preference data. We therefore adopt differentiable reward-based fine-tuning by directly maximizing reward values (Clark et al., 2024; Prabhudesai et al., 2023; Wu et al., 2025a), which is effective for aligning diffusion models to downstream objectives including semantics (Tan al., 2025) and safety (Su et al., 2025; Wang et al., 2026). However, as discussed in Sec.3, these approaches can require backpropagation through denoising, leading to sparse gradients, slow convergence, and high memory costs.
3 MOTIVATION: RETHINKING DIFFERENTIABLE REWARD-BASED METHODS
Preliminaries. As illustrated in Fig. 2, existing methods fine-tune pre-trained motion diffusion model by maximizing the reward value R. (xf, e) of the motion x generated via T -tep reverse pro- cess. Notably, this generated motion x requires retaining gradients throughout the entire denoising trajectory x and thus the model can be optimized via maximizing its reward value Vo c).
Given a pre-trained motion diffusion model parameterized by co, the optimization objective is to fine-tune 0 to maximize the reward value R2 (x6, c). with the loss defined as:
(()) -Be..D xf-0o (x0
(1)
3

---

Published as a conference paper at ICLR 2026
cyt Trainable diclusion model
Ry: Prozen reward mode!
x Sumpled modicn
Moricn with gradient from cy
Motion without gradient
Existing Methods (DRTune,, DRaFT, & AlignProp)
Easy Tune (Ours)
Fine-grained precise optimization
DRCTmm
skip
Stop gratien
Suar grilier
Sup gailire
Prenop
Ka
DiaFT
Esoup
AlianPop
Xin
Step1: Hyper-parameter Sampling
Step2: Process
Step3: Optimization
Only one step: Optimize durlnn denoising
Coarse optimization
Excessive Memory
Inefficient Optimization
Memory
Effcient Optimization
Figure 2: The framework of existing differentiable ewarrd-base methods (left) and our pro- posed EasyTune Tune (right). Existing methods backpropagate the gradients of the reward model through the overall denoising process, resulting in (1) excessive memory, (2) inefficient, and (3) coarse-grand gaaseegrained optimization. In contrast, EasyTune optimizes the diffusion model by directly backpropagating the gradients at each denoising step, overcoming these issues.
where is a text condition from the training set DT, and x is the motion generated from noise Xy AN(0, I) via a T Tseepp reverse process Te. The t-th step of the reverse process denoted as:
Be VI-
x
no (x)
x
Vai
where x is the denoised motion at step f and ats 3r are noise schedule parameters.
Gradient Analysis. To optimize the loss in Eq. (l), we further analyze the gradient computation, where the gradient [(0) w.r.t. the model parameters is computed via the chain rule:
OC(0) 00
OR c)
0xf
E Di Dro.Drs mo
0x
00
8R. (x).C) 0xg
represents the gradient of the reward model w.r.t. the generated motion, and 8x
Here,
captures the dependence of the generated motion xf on the model @ through the reverse trajectory.
Eq. (3) indicates that the gradient of loss function can divided into two terms: OR (xf e) /xx which can be directly computed from the reward model, and 0x// 00, which depends on the denoising trajectory no- Here, introduce Corollary 1 to analyze this gradient (See the proof in App. C.1). Corollary 1. Given the reverse process in Eq. (2), x no (x.. t, c). the gradient w.r.t diffusion
model 0, denoted 0x._1
can be expressed as:
50
dx.
(x t,)
Omo (x),t,c)
tx
(4)
00
00
0x!
08
Corollary 1 shows that the computation involves two parts: (1) a direct term (in blue) from the dependence of the diffusion model Ty 0, and (2) indirect term (in red) that depends the t-th step generated motion x²- However, the reverse process in diffusion models is inherently recursive, where the denoised motion x is relied on X4. which in turn depends X resulting in substantial computational complexity for time steps intermediate variables.
To compute the full gradient AC(6) unroll the 0x//00 using Corollary 1 and substitute it into Eq. (3) resulting in (see proof in App. C.3):
0C(0) - -E EEEDD xg (t(e)))
Ox,
Omo (xG.xx.
(ata (x²; L,
(5)
Oxf
ao optimizing step
send when larger
4

---

Published as a conference paper at ICLR 2026
Limitations. Eq. (5) reveals the core optimization mechanism of existing methods: the motions x are generated via the reverse process Tg. with the full computation graph preserved to enable the maximization of the reward Rx(x); c). However, shown in Fig. 2, this optimization incurs severe limitations:
(1) Menory-intensive and sparse optimization: Gradient computation over T reverse steps demands storing the entire trajectory {x and corresponding Jacobians, leading to high memory consumption and inefficient, sparse optimization compared to the sampling process.
(2) Vanishing gradient due to coarse-gres marse-grained optimization: Eq. (5) indicates that the optimization of 1-th noisy step relies on the gradient .t.e)
00 with a coefficient [[-1 (xp(x"...c) However,
8x
(x st.))
during optimization, the term
tends to converge to (see the blue line Fig. 3),
ox!
causing the coefficient I[==]
Oxalx" dxe
to also approach 0 (see the orange line in Fig. 3).
Consequently, the optimization process tends neglect the contribution of @xu(sx].t.c) More importantly, the ignored optimization at these early noise steps may be more crucial than at later steps (Xie & Gong, 2025).
Motivation. To address the aforementioned limitations, argue that the key issue lies in Corollary 1: the com- putation of öx. /09 recursively depends on 0x /00, making the computation of 0x// reliant on the entire T- step reverse process. This dependency necessitates storing a large computation graph, resulting in substantial memory consumption and delayed optimization. To overcome this, an intuitive insight is introduced: optimizing the gradient step-by-step during the reverse process. As illustrated in Fig. 2, step-by-step optimization offers several advantages: (I) Lower memory consumption and dense optimization: each update only requires the computation graph of the current step, allowing gradients to be computed and ap- plied immediately instead of waiting until the end of the
Gradient Norm
0.8
0.6-
II.
V Lanil cim & gralt
0.1
0.2
Step
Figure 3: Gradient norm with respect to denoising steps. Here, dim(-) de- notes the gradient dimension. Detailed settings are provided in App. B.1.
T' T'step reverse process. (2) Fine- fine-grained optimization: each step is optimized independently. so that
the update of the 1-th step does not depend on the vanishing product of coefficients II
0x ..c)
0x
However, in domains such as image generation, reward are predominantly output-level (Xu et al., 2023) rather than step- aware, since noised states with complex semantics are difficult interpret. In contrast, motion representations exhibit simpler and more interpretable semantics, thereby making step-aware motion reward viable (Fig. 4; see further details in App. A.8).
Modics Image
1.0
Similarity
0.6
0.4
Inspired by the above discussion, we propose EasyTune, step- aware differentiable reward-based fine-tuning framework for diffusion models, introduced in Sec.4.1. Specifically, EasyTune employs a step-aware differentiable reward model designed to evaluate noised, rather than clean, motion data, allowing us to perform optimization at each step without storing multi-steps computation graph. Nevertheless, due to the scarcity of human-annotated motion pairs, the primary challenge lies in training such reward model without any paired data. To address this issue, we present a self-refinement preference learning mechanism, in Sec.4.2, to identify preference data pairs specifically targeting the weaknesses of the pre-trained model, facilitating the acquisition of a reward model.
Denoising Step (0)
Figure 4: Similarity between 1-th
step noised and clean motion.
4 METHOD
4.1 EFFICIENT STEP-AWARE FINE-TUNING FOR MOTION DIFFUSION
Assuming the reward model for evaluating noisy motion, aim to propose step-aware fine-tuning method that reduces the excessive memory usage and performs efficient and fine- ime-grained optimization.
As discussed in Sec. limitations of existing methods (Clark al., 2024; Wu et al., 2025a) stem
5

---

Published as a conference paper at ICLR 2026
Computation graph
Using Chain Optimization
Using Step Optimization
Optimize
(t,)
Eq. (4)
T
Copy
Computation graph
Solve: d1
0x1
0
dMg
Arfen required
Eq. (7)
Opiamize 0'
88
00
3x
Solve: 0xf
x
x
or
Optimize
We (x)
Store the compulation graph
Clear the computation grapb
(a) Existing methods
(b) Easy (ours)
Figure 5: Core insight of EasyTune By replacing the recursive gradient in Eq.(4) with step-level ones in (7)( EasyTume removes recursive dependencies, enabling (1) step-wise graph storage, (2) efficiency, and (3) fine-grate unne-grined optimization. See App. for pseudocode and discussion.
from the recursive gradients computation. To address these issues, we introduce EasyTune, a simple yet effective method for fine-tuning motion diffusion models. The key idea is to maximize the reward value at each step, allowing the parameter to be optimized at each step without storing the full trajectory, shown in Fig. Specifically, the training objective function is defined as:
LunxyTune (0) =
-B Dr ( (0))
(6)
where Ro (xx,,, t) is the reward value of the stop gradient noised motion x at time step f, and 4(1,7) is a uniform distribution over the time steps. Here, the stop gradient noised motion x² and its gradient w.r.t the diffusion parameter are represents as:
(x(x
x To (s(g((x)), = sg(xx()
8.
(188((2))))))
where sg(-) denotes the stop gradient operations. Eq. (6) and Eq. (7) indicate that Easy Tune aims to optimize the diffusion model by maximizing the reward value of the noised motion x² at each step I. Corollary 2. Given the reverse process in Eq. (7), the gradient w.r.t. diffusion model 0 is denoted as:
0x Ong (sg(x)),
ae
00
Corollary 2 shows that EasyTume overcomes the recursive gradient issue, enabling efficient, fine- grained updates with substantially reduced memory. As Fig. illustrates, while prior methods incur O(T) memory by storing the multi-steps trajectory, EasyTune maintains constant O(1) memory. Guided by Corollary 2, we optimize the loss function LaasyTume as follows:
EuneyTunel (0) - De (100))))))000 OR.xx,,,
Oxo ( ( +1,11 1,
(9)
0x.
00
Discussion of Existing Methods. Unlike prior methods (Eq. (5)), EasyTune updates the diffusion model 0 using
15
DRaFT w/o
Eq. (6), computing the gradient 2=o(sg(x().t.o) each step 80
Memory (GB)
AlignProp
12
t without storing the full -st)- computation graph. Among related works, the closest DRTune (Wu et al., 2025a), which also uses stop-gradient operations sg(-) to solve the limitations of previous methods:
DRTune win BP
Ours Ourswe BP
Pt
to (eg((x)))
10
20
30
40
50
vI
Denoise step
8x
ix
A
den (sg(xx))
Figure 6: Memory usage comparison. Here, "w/o BP" indicates memory mea- sured without backpropagation. Com- prehensive analysis are in App. A.7.
Vai
& /1
However, recursive gradient computation remains issue in existing methods (Eq. (10)). As shown in Fig. 6, their memory usage grows linearly with the number of denois- ing steps (O()), while Easy Tune maintains a constant memory footprint (O(1). These analyses and experiments highlight the efficiency of method and details discussion are provided in App. B.

---

Published as a conference paper at ICLR 2026
4.2 SELF- REFINING PREFERENCE LEARNING FOR REWARD MODEL
Our goal is to develop a reward model without the requirement of human-labeled data. Existing works (Tan et al., 2025) often repurpose pre-trained texi-to-motion retrieval models (Petrovich et al., 2023) to score text-motion alignment. However, this is suboptimal: retrieval models focus on matching positive pairs in a shared embedding space, whereas reward models must distinguish between preferred and non-preferred motions. Given their shared architecture, retrieval models can be fine-tuned for preference learning. The challenge, however, lies in the scarcity of such preference data in the motion domain. To this end, we propose Self-refining Preference Learning (SPL), which leverages a retrieval-based auxiliary task to construct preference pairs for reward learning. SPL involves two steps: (1) Preference Pair Mining: retrieve motions for each text; treat the ground-truth as preferred and top incorrect retrieval as non-preferred if it's not retrieved; (2) Preference Fine-tuming: updating the encoders to assign higher scores to preferred motions.
Reward Model. Given a motion x and a text description c, the reward value is computed based on the similarity between the motion features and text features denoted as:
R (x, c) (x) (r))) -T,
(11)
where EM and ÊT are the motion and text encoders from the pre-traine pre-trained retrieval model (Weng et al., 2025a), and 7 is a trainable temperature parameter.
Additionally, dealing with noisy motions remains a key challenge in step-level optimization. Current diffusion- based models can be divided into based (Song et al., 2020) and ODE-based (Lu et al., 2022) models. For ODE-based settings (Dai et al., 2025), thanks to their deterministic sampling, we use the reward value of the coarse clean motion &o predicted by one-step prediction Xo T (x,, c) as the final reward value. For both SDE- and ODE-based settings (Tevet et al., 2023), we adopt noise-aware reward model to accurately calculate their reward values:
Rx, t, c) = Ro c), Only for ODE-based settings,
(12)
tx),
For SDE- and ODF-based settings.
Preference Data Mining. To identify non-pre pee--rrffered motions often incorrectly retrieved, we retrieve the top-k motions Dr from the training set or subset Dr given text condition c:
Dn = top; arg max (x, c) xEDT
arg max
Σ R,(x,,) c).
(13)
PR PRCPr.PPllk xEDR
where top, arg maxxeD, Rs(x, e) denotes the top-k motion with the largest reward value.
Given a ground-truth motion x5 and its text condition c, we retrieve the top-k motions Dn based on reward scores. If xx² & DR. we treat as the preferred motion and the highest-scoring retrieved motion the non-preferred one; this case provides an informative preference signal. Otherwise, the retrieval model already ranks the ground-truth among the top-k; so no informative negative exists; set x! = x8 and assign Q = (0.5, 0.5), which yields zero KI. gradient and effectively skips optimization for this sample:
arg maxxeDa Rx(x,)), c), xf $ DR.
(14)
otherwise.
Preference Fine-tuning, Given preference pair consisting of preferred motion x" and non- preferred motion x!, we compute their reward scores and convert them into softmax probabilities:
p= (P("". .c. P(xx .e)x'c) = Softmax (Ro(x", c). Rx(x),))
(15)
Following Pick-a-pic (Kirstain et al., 2023), we optimize model by aligning the predicted softmax distribution P with a target distribution Q, which reflects the ground-true preference between x" and x!, defined as:
e e= { (L.0, xf is preferred over x!, (0.5, 0.5), x = !!
(16)
Formally, the target distribution Q encodes the preference between preferred motion x" and non non-preferred motion x!. If x" is preferred, we set Q = (1.0, 0.0), encouraging P(x". .c) - 1 and P(x),c) O. If the two are identical (x" =x)), we set Q = (0.5, 0.5), indicating no preference.
7

---

Published as a conference paper at ICLR 2026
To optimize the reward model ø, we minimize the KL divergence between them:
((x, c) log Q(x, c)
(S) DRa(2
Σ
P(x,c)
(17)
where Q and are the target and reward distribution. By mining preference motion pairs by preference data mining, we can fine-tune praited retrieval models by Eq. (17), to obtain reward modesl without human-annotated data. Detailed process are described in App. B.4.
EXPERIMENT
5.1 EXPERIMENTAL SETUP
Datasets & Evaluation. We conduct experiments on HumanML3D (Guo et al., 2022a) and KIT- ML (Plappert et al., 2016). Following standard practice (Guo et al., 2023; Li et al., 2025), we report R- R-Precision @ k, Fréchet Inception Distance (FID), Multi-Modal Distance (MM Dist), and Diversity. We also measure peak memory usage to assess efficiency. Additionally, our SPL mechanism is evaluated using R-Precision@1l @ k: under the previous setup (Li et al., 2025).
Implementation. Our method consists two components: fine-tuning the diffusion model with EasyTune and fine-tuning a pretrained retrieval model obtain the reward model. For EasyTune, we evaluate pretrained backbones-Mo
(Chen et al., 2023), MLD++ (Dai et al., 2025), Mo- tionLCM (Dai et al., 2024), and MDM (Tevet et al., 2023) wwithh hyperparameters: learning rate of 1 x 10-5 and a batch size of 256. Easy Tune is benchmarked against differentiable reward-base baselines with their official hyperparameters, detailed in App.A.I. We fine-tune the reward model initialized with ReAlign (Weng et al., 2025a) using our SPL. with top- K samples (K" = 10). Then, the reward model is frozen and provides supervision for optimizing the diffusion model. The exper- imental results for the ODE- baEdd model, using both reward computation methods from Eq. (12), are provided. Results corresponding to the first and second terms are presented in Tab. and 2, and Tab. 3, respectively. Experiments are conducted on single NVIDIA RTX A6000 GPU with 48GB memory, detailed overhead is provided in App. A.7.
5.2 EVALUATION ON MOTION DIFFUSION FINE-TUNIAG
Comparison with SOTA Fine-Tuning Methods. To assess the effectiveness and efficiency of Easy- Tune, we compare it with recent state-of-the-art fine-tuning methods, including DRaFT, AlignProp, and DRTune, as shown Tab. 1. Easy consistently achieves the best overall performance across key metrics, including R-Precision, FID (0.132, +70.7%), MM Dist (2.637, +1 +13.6%), and Diversity, while also requiring least GPU memory (22. 10 GB). We attribute these gains to two core designs: optimizing rewards at each denoising step for finer supervision, and discarding redundant computation graphs to reduce memory usage.
Efficiency of the Optimization. To assess convergence efficiency, compare optimization curves of fine-tuning methods in Fig. S1 (in App. A). EasyTune converges faster and achieves consistently
Table 1: Comparison of fine-tuning methods on HumanML3D. Arrows 1. 4. and + indicate that higher, lower, and closer to real values are better. Bold and underline denote the best and second-best results. All methods adopt the noise-aware reward R. (x,, c).
Precision
Method
FID
Dist
Diversity
Memory (GB)
Top
Top 2
Top 3
Real
0.511
0.703
0.797
0.002
2,974
9.503
MLD (Base Model)
0.504
0.698
0.796
0.450
3.052 000
9.634
15.21
w/ RcFL-10 (Clark et al., 2024)
0.533.sam
0.720.
0.821
0.207
2.852.4.0
10.129...r
22.10.6a9
w/ ReFI--20 (Clark
2024)
0.528 4.5
0.718,29
0.813
0.241
46.4%
...8
189 asss
22. 10,69
w/ DRaFT-10 (Clark et al., 2024)
0.565.12.1
0.757.4.5
0.846...1
0.195.s6.5
2.703.11
9,85La.2
26.56..
w/ DRaFT:50 (Clark et al., 2024)
0.528
0.724.195.
0.819.1.u
0.197-4675
2.872 vs
9.641.om
37.32.12.1
w/ AliguProp (Prabbudessi et al., 2023)
0.560
0.753.
0.841.574
0,266
2.739.m89
9.877..24
30.40..s..v
w/ DKTune (Wu et al., 2025a)
0.549.x.4
0.746...s
0.836,1es
0.313.qp
2.795
9.930
27.01-:180
w/ EasyTTn Tune (Ours, Slep Optimization)
0.581
0.769,
0,855,744
0.132
2.637.1168
9.465
22.10.439
w/ FasyTune Tune (Ours, Chain Opimization)
0.574 13.09
0.766
0.854,138
0.172
2.614,449
9.348
24.21
8

---

Published as a conference paper at ICLR 2026
Table 2: Comparison of text-to-motion generation performance on the HumanMI.3D dataset.
Method
R Procision f
FID
MM Dist
Diversity
Top
Top 2
Top
Real
0.511
0.703
0.797
0.002
2.974
9.503
TM2T (Gno et 2022h)
0.424L8000
0.618 Loaoa
0.729 L0.000
[.50)
3,467 -0.00
8,589 -0.975
T2M (Guo 2022a)
0,455
0.636422000
0.736 oca
L0887:0.0c
3.347 oce
9.175
MDM (Tevet et al., 2023)
0.455=50.00
0.645=00000
0.749 00e
0.489=200047
3.330#9.026
9.920=0.00
T2M GPT (Zhang al,, 2023a)
0.492
0.679=00000
0.775+0.002
t500006
3.121 =00009
9. 9.722=0.062
ReMoDiffuse (Zhang et al., 2023b)
0,510$50.00
0.69g
0.795
0.795 004
0,103 11100000004
9.018#00000 2.974#9.016
Aur12M (Zhong al. 2023)
0.499
0.690)+0.00
0.786 +0.002
3.038=0.007
9. 9.700=00000
MotionDiffuse (Zhung et al, 2024a)
0.491 Au00u
0.681 4.0000
0.775
0.630-40.000
3.113
9.410
Motionl CM (Dai ct al., 2024)
0.502
0.698
1.000...om
0.798 im oon
0.304
3.012-6..00 2007
9. 9.607-a 265
MotiouMamba (Zhang al., 202-1b)
0.502#00000
0.693+50.00
0,792
0.281 #10001
3.060#0.000
9,871 0.871=0.004
CoMo (Huang et al., 2024)
0,502
0.602.49.0m
0.790 7790040-00m oon
0.262 0.262-49.004
3.032440.015
9.936 2.936-0.000 000
Puro (Zou al., 2024)
0.515*0.000
0,706+20.00
0 1801=#0.000
0,109
2.927#20000
9.576*0.008
SoPo (Tan et al., 20255
0.52g*0.00s
0.722
0.827
2.939=00001
9.584
MLD (Chen et al., 2023) (Base Model)
0.504
0.69g=00.00
0.796 +0.00
0,450=0.001
3.052#00000
9.634=0.064 00004
w/ EasyTuss (Ours)
0,581 413-35
0. 0,769
0,855 1000a
0.1 0.132720
2.637
9,465
MID++ (Dai et ol., 2025) (Base Model)
0,548 Socoud
0.738 500000
0.829
2000022
0.073300003
2.810=0.008
9.658
w/ BEEayyTue (Ours)
0,591
17.8
1.000s
0.777
15.3%
Lover
0.859
13.6
10
0.069
16.8%
Loueca
2.542
17.8%
L0L00w
9.705 0,06
10.345
lower loss, suggesting better local optima with higher reward values. This improvement stems from its fine-grained, step-wise optimization, in contrast to the sparser, trajectory-Ievel updates used in prior work (Clark et al., 2024), enabling more precise gradient signals and accelerated training. Furthermore, Easy Tune achieves a 7.3x training speedup over DRaFT to reach the same reward level, detailed computational overhead analysis is provided in App. A.7.
5.3 EVALUATION ON TEXT-TO- GENERATION
Comparison with SoTA Text-to-Motion Methods. We evaluate EasyTune on text-to-motion generation using MLD (Chen et al., 2023) and MLD++ (Dai et al., 2025) as base models, comparing with state-of-the-art methods on the HumanMI.31 (Guo et al., 2022a) and KIT-MI. (Plappert et al., 2016) datasets, as shown in Tab. 2 and S3 (in App. A.3). On HImanMI.3D, Easy Tune improves the R-P@ of MLD from 0.504 to 0.581 and MLD++ from 0.548 to 0.591, surpassing baselines like ParCo (Zou et al., 2024) (0.515) and ReMoDiffuse (Zhang et al., 2023b) (0.510). It also achieves the best MM Dist (2.637 and 2.592) and competitive FID (0.132 and 0.069).
Generalization across Different Pre- trained Models. To evaluate the gen- eralization of EasyTune across pre- trained text-to-motion models, we applied it to MLD (Chen al., 2023), MLD++ (Dai et al., 2025), 1L (Dai et al., 2024), and MDM (Tevet et al., 2023). As shown in Tab. 3, EasyTune consis- tently improved performance. For in- stance, MLD saw a 12.7% increase in R-P@1 1 (0.568) and a 56.9% reduction in FID (0. 194). MLD++ achieved a 6.0% gain in R-Precision @1 (0.581) and a 7.4% improvement in MM Dist (2.603). MLCM and 5DM also showed significant FID
Table 3: Performance enhancement of diffusion-based motion generation methods. For ODE samplings (MLD, MLD++, MLCMM,, adopt the one-step prediction reward.
Method
Precision 1
FID
MM Dist
Divensity
Top
Top
Top 3
Real
0.511
0.703
0.797
0,002
2.974
9,503
MLD
0.504
0.608
EasyTune
0.508
0.754
0.856
0.194
2.672
2025)
0.488
0738
FaayTine
6.
0.000
2,608 (400
MLCM (Duictal,, 2024)
0.502
0.701
0.503
3,05
0.71
0.766
9.692.3ux
MLCM (Dai 2024)
0.502
0.698
0.798
0,304
3,012
9,607
0.760
0,200.
2.691
9.812
MUN Clevet al 2023)
0.455
0.645
0.309
0,489
3,380
9,920
wi Faay Time
0.679
0.787
0.411
1.117
9.219
Mo. Difuue (Zhune et ol., 202443
D.491
0.651
0.775
0.630
3.113
9,410
of EanyTune
0.788
0.888.11
0
1008,
9 9.218.1.00
reductions of 59.7% and 16.0%, respectively. These results highlight the generalization of Easy Tune across various diffusion-based architectures.
5.4 ABLATION STUDY, USER STUDY& & VISUALIZATION
Self-refinement Preference Learning for Reward Model Training, We evaluate SPL on text- motion retrieval, comparing it with ReAlign and state-of-the-ar methods. As shown in Tab. 4, SPL boosts ReAlign (Weng et al., 2025a) on HumanML3D (R @ 1: 69.31, +2.5%; R@3: 88.66, +1.4%; motion-to-text R@1: 70.23, +1.9%), outperforming LaMP (Li et al., 2025) and TMR (Petrovich
9

---

Published as a conference paper at ICLR 2026
Table 4: Evaluation on text-motion retrieval benchmark, HumanML3D and KIT-ML. The
column "Noise" indicates whether the method can handle noisy motion from the denoised process.
Text-Mation Retrieval?
Motion-Tee Retrivval
Methods
Noise
Re5
R
R
R 10
R
R6F2
R
R
R 10
Humonn MI 1,30
TEMO (Petrowich al, 2022)
x
40.49
03.02
61.14
70.56
84.15
39.16
53.40
61.77
72.40
85.89
T2M (Giso et al,, 2022a)
x
52.48
71.05
80.65
89.66
96.58
52.00
71.21
81.11
89.87
96,78
(Perroich et al, 2012)
X
67.16
81.322
86.81
91.43
95.33
67:97
81:20
866.35
91.70
95.27
LaMP (Li al, 3025)
67.18
81,90
87 57.04
92.00
96.73
68,
82.10
87.50
122
96,90
ReAlign (Weng ol, 2025k) (Rase Model)
67.50
82:24
87.44
81.
96.28
68.
82,
857.01
1/2
110
w/ (PL(Our))
69.31
83.71
88.66
92.81
96.75
70.23
83.41
85.72
93.07
97.04
T2MOS (Petrovich et al,, 2022)
X
40.88
58,25
67:00
74.00
84.75
41.88
55.88
65.82
15.25
85.75
KIT-ML
T2M (Gio al., 2022a)
02.25
62.62
75.12
67.50
96.12
39.75
02.75
73:02
86.85
95.88
TMR (Petrovich et all,, 2023)
2023)
X
49.25
69.75
78.25
87.88
95.00
50.12
67.12
76.98
88.83
94.75
LaMP (Li al., 2025)
52.50
74.80
$4.70
92.20
97.60
54.00
75.30
81.40
92.20
97.60
ReAlignn (Weng al., 2025a) (Base Model)
52.84
71.86
82. 96
81.19
97.59
52. .9
72.87
84.18
92.61
96.87
w/ SPL(Ous))
53.27
73.58
S1.02
93.18
97.73
55.11
75.28
86.36
93,18
97.41
Baw Model Preferred
Same
RLJIF Model Preferred
wil SPL
wi SPI.
Motlon-Ter Text Aligumen
18%
13%
R-P FTD
32.55%
67.45%
44.87%
55.13%
Phydicl Fide
R-P
45.97%
54.03%
Cabereace and Fiernce
36%
41%
Tep
42.68%
57.32%
20
40
60
90
100
20
40
60
160
Preference (%))
Winning Rate (%)
Figure 7: User study on HumanML3D test set. We use MLD model as base model.
Figure 8: Comparison of models fine-tuned with and without SPL.
et al., 2023). On KIT-ML, SPL achieves R@5 of 93.18 (+2.2%), and notion-to-texx R@3 of 86.36 (+2.3%), consistently surpassing baselines.
Self-refinement Preference Learning for Fine-Tuning To evaluate the effect of SPI,, we fine-tune MLD (Chen et al., 2023) using reward models trained with and without SPI., and compare their win rates across epochs, as shown in Fig. 8. The model with SPL consistently outperforms the baseline, achieving win rates of 57.32%, 54.03%, 55. 13%, and 67. 67.45% on R-P Top Top 2, Top 3, and FID, respectively. This shows that SPL can improve motion generation by enhancing the reward model.
Human Evaluation. To assess whether our fine-tuned model exhibits reward hacking, we conducted user study and visualized the corresponding motions. These visualizations are presented in Fig. 7, with additional visualization in Fig. S3 (App. A.4). Results shows that our method enhance the alignment, fidelity, and coherence of generated motions. A more detailed discussion and further experimental results are provided App. A.2.
CONCLUSION
In this work, we theoretically identify recursive dependence in denoising trajectories as the key limitation in aligning motion generative models. Our proposed EasyTune method decouples this dependence, enabling denser, more memory-efficient, and fine-grained optimization. Combined with the SPL mechanism to dynamically generate preference pairs, experimental results demonstrate that Easy Tune significantly outperforms existing methods while requiring less memory overhead.
Limitation. In this work, focus improving semantic alignment in text-to-motion generation. Due to limited preference data, SPL relies on retrieval- based mining, which can introduce noisy or ambiguous pairs and provides limited physical grounding in reward design. Encouragingly, find that the semantic reward still implicitly distinguishes real motions from generated ones (App. A.9). Building unified reward model that explicitly captures both physical plausibility and semantic alignment remains an important direction for future work.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This research is supported by the National Natural Science Foundation of China 520s.52441503 and 62302093), the Natural Science Foundation of Jiangsu Province (Nos. (00.BKK0030833), and the Big Data Computing Center of Southeast University.
ETHICS STATEMENT
Our work on EasyTune, a method for fine-tuning motion generative models, introduces several ethical considerations that warrant careful discussion. our method designed align existing generative diffusion models, it inherits the potential biases and limitations of these foundational models. The large-scale motion datasets used to train these base models may contain demographic biases (e.g.. representation of age, gender, or physical ability) or may underrepresent certain types of human movement.
Furthermore, like other generative technologies, motion generation models could be misused by malicious actors. The ability to create realistic human motions could exploited to generate convincing deepfakes synthetic media for the purpose of disinformation, harassment, or creating non-consensual content. As EasyTune makes the process of aligning models to specific objectives more efficient, it could inadvertently lower the barrier for adapting these models to generate harmful or undesirable motions.
Finally, the advancement of motion generation technology may have significant socio-economic impact. On one hand, such tools could automate tasks traditionally performed by animators, choreog- raphers, and motion capture actors, potentially displacing jobs in creative industries. On the other hand, EasyTur Tune could also serve as powerful creative tool, democratizing animation and enabling new forms of artistic expression for independent creators and small studios. It also holds potential for positive applications in fields like robotics, virtual reality, and physical rehabilitation. We believe that continued research and community dialogue are essential mitigate the risks while harnessing the benefits of this technology.
REPRODUCIBILITY STATEMENT
We commit to releasing all code, model weights, and baseline implementations upon acceptance. To ensure the reproducibility of our experiments, we put the key parts in Appendix A.1. For datasets, we use open source datasets described in Sec. 5.1. For generated results, we upload generated videos to supplementary material.
LARGE LANGUAGE MODELS USAGE STATEMENT
We used Large Language Models (LLMs) as auxiliary tools during the preparation of this manuscript. In particular, L.MSs were employed to polish the language, improve grammar, and enhance readability of the text. All conceptual ideas, technical contributions, analyses, and conclusions presented in this work entirely were not used to generate novel scientific content, perform data analysis, or contribute to the design of experiments. We have carefully verified all statements and ensured that the final version of the manuscript accurately reflects our intended meaning and contributions.
own and were developed independently of LLM assistance. The models
11

---

Published as a conference paper at ICLR 2026
REFERENCES
Samanch Azadi, Akbar Shah, Thomas Hayes, Devi Parikh, and Sonal Gupta. Make-an-annmatte Large-scale text-conditional 3d human motion generation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 15039- 15048, 2023.
Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. arXiv preprint 330:::235.333000 2023. 1,3, 28
Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Executing your commands via motion diffusion in latent space. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 18000-18010, 2023. 1, 2, 3, 8, 9, 10, 18, 19, 20
Kevin Clark, Paul Vicol, Kevin Swersky, and David Fleet. Directly fine-tuning diffusion models differentiable rewards. In The Twelfth International Conference Learning Representations, 2024. URL https: wpwp/eeeeeeeeeeeeeeeeeeeeeeeeeee net/ Forum wew.vdv-vviill1111 1, 2, 3, 5, 8, 9, 20, 27, 29
Wenxun Dai, Ling- Ling-Hao Chen, Jingbo Wang, Jinpeng Liu, Bo Dai, and Yansong Tang. Motionlem: Real-time controllable motion generation via latent consistency model. In Aleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and Gül Varol (eds.), European Conference on Computer Vision, pp. 390-408, Cham, 2024. Springer Nature Switzerland. ISBN 978-3-031- 72640-8. 2, 3, 8,9
Wenxun Dai, Ling-Hao Chen, Yufei Huo, Jingbo Wang, Jinpeng Liu, Bo Dai, and Yansong Tang. Real-time controllable motion generation via latent consistency model. arXiv preprint, 2025. 2, 8,
Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Dpok: Rein- forcement learning for fine-tuning text-to-image diffusion models. In Oh. T. Nau- mann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 79858-79885. Curran Associates, Inc., 2023a. URL https: http://s///oroeeeisss...om neurips wttpippppppppppppppppppppp ftleeeeeeeeeeeeeeeeeeeeepeeeeeee wttes//appeeeee/22// hall:fffcSsaas11s344eccbboeeeeee
Paper CorrrrCooneeccecceeeee .df. 1,3
Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. Advances in Neural Information Processing Systems, 36:79858-79888 2023b. 28
Alessandro Flaborea, Luca Collorone, Guido Maria D' Amely Di Melendugno, Stefano D' DrArio,, Bardh Prenkaj, and Fabio Galasso. Multimodal motion conditioned diffusion model for skeleton- based video anomaly detection. In Proceedings of the IEEE/CVF international conference computer vision, pp. 10318-10322 2023. 27
Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 5152-5161, 2022a. 1, 2, 3, 8, 9, 10, 19
Chuan Guo, Xinxin Zuo, Sen Wang, and Li Cheng. Tm2t: Stochastic and tokenized modeling for the reciprocal generation of 3d human motions and texts. In European Conference on Computer Vision, pp. 580-597. Springer, 2022b. 9, 19
Chuan Guo, Yuxuan MU, Muhammad Gohar Javed, Sen Wang, and Li Cheng. Momask: Generative masked modeling of 3d human motions. 2023. 3,
Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma. Xiao Bi, et al. Deepseek-rl incentivizes reasoning in 1lms through reinforce- ment learning. Nature, 645(808) 1):6336633 2025. 28
12

---

Published as a conference paper at ICLR 2026
Gaoge Han, Mingjiang Liang, Jinglei Tang, Yongkang Cheng, Wei Liu, and Shaoli Huang. ReinDif- fuse: Crafting Physically Plausible Motions with Reinforced Diffusson Model. In 2025 IEEE/CVF Winter Conference Applications of Computer Vision (WACV), pp. 2218 1227222,7 Los Alamitos, CA, USA, March 2025. IEFE Computer Society. doi: 10.1 R..1009/AAV660411.00 2025.0022.. Utthttps: ety .rgg 10. 110 WACV61 .225 00222.
//doi, www.eeeeeeeeeeeeeee....
Yiming Huang, Weilin Wan, Yue Yang, Chris Callison-Bureh, Mark Yatskar, and Lingjie Liu. Controllable motion generation through language guided pose code editing. In Computer
Como:
Vision - ECCV 2024: 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part XXIX, pp. 180-196, Berlin, Heidelberg, 2024. Springer- Verlag. ISBN 978-3- 0111///97888--1-73333333 978- 000-3-0-1133399--11
031-73396-3, doi:
URL https: /tt///oi.o//// wwwpoooooo....oooooooo/ 1007/
9, Michael Janner, Justin Fu, Marvin Zhang, and Sergey Levine. When trust your model: Model-based policy optimization. Advances in neural information processing systems, 32, 2019. 3
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick- a-pic: An open dataset of user preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36:36652- 26663,, 2023.
Hanyang Kong, Kehong Gong, Dongze Lian, Michael Bi Mi, and Xinchao Wang. Priority-centric human motion generation in discrete latent space. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11806-1481, 2023. 19
Black Forest Labs. Flux. https: http//gittub com/black wwwest-
2024. 26
Zhe Li, Weihao Yuan, Yisheng HE, Lingteng Qiu, Shenhao Zhu, Xiaodong Gu, Weichao Shen, Yuan Dong, Zilong Dong, and Laurence Tianruo Yang. LaMP: Language-motion pretraining for motion generation, retrieval, and captioning. In The Thirteenth International Conference on Learning Representations, 2025. URL https::/ httpss///pppeevieeeeee net// fo Eorum? wwmmiddddaawwee BYka. 9, 10
Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-grpo: Training flow matching models via online rl. arXiv preprint arXiv:2505.05477 05570, 2025. 28
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in neural information processing systems, 35:5775-5787, 2022. 7
Zichong Meng, Yiming Xie, Xiaogang Peng, Zeyu Han, and Huaizu Jiang. Rethinking diffusion for text-driven human motion generation. 2025. 3, 19
Massimiliano Pappa, Luca Collorone, Giovanni Ficarra, Indro Spinelli, and Fabio Galasso. Modipo: text-to-motion alignment via ai-feedback-driver direct preference optimization. arXiv preprint
3xXi::2405.03803 2024. 3
Wenshuo Peng, Kaipeng Zhang, and Sai Qian Zhang. T3m: Text guided 3d human motion synthesis from speech. arXiv preprint arXiv:2408. /2885, 2024. 1
Mathis Petrovich, Michael J Black, and Gül Varol. Temos: Generating diverse human motions from textual descriptions. In European Conference on Computer Vision, pp. 480 497. Springer, 2022.
10
Mathis Petrovich, Michael Black, and Gül Varol. Tmr: Text-to-motion retrieval using contrastive 3d
human motion synthesis. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 9488-9497, 2023. 3, 7, 9, 10, 22
Matthias Plappert, Christian Mandery, and Tamim Asfour. The kit lation-l dataset. Big data, 4(4):236-25,, 2016. 2, 8,
Mihir Prabhudesai, Anirudh Goyal, Deepak Pathak, and Katerina Fragkiadaki. Aligning text-to-image diffusion models with reward backpropagation, 2023. 2, 8, 29
13

---

Published as a conference paper at ICLR 2026
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 9748-8763. PMLR, 2021. 26
Allen Z Ren, Justin Lidard, Lars L Ankile, Anthony Simeonov, Pulkit Agrawal, Anirudha Majumdar, Benjamin Burchfiel, Hongkai Dai, and Max Simchowitz. Diffusion policy policy optimization. arXiv preprint S0ti::240900058880 2024. 28
Keming Shen, Bizhu Wu, Junliang Chen, Xiaoqin Wang, and Linlin Shen. Finextrol: Controllable mo- 18927.
tion generation via fine-grained text, 2025. URL http:s///wawww.oo////////
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint 0rXiv:2011.134455 2020. 7
Zihan Su, Xuerui Qiu, Hongbin Xu, Tangyu Jiang, Junhao Zhuang, Chun Yuan, Ming Li, Shengfeng He, and Fei Richard Yu. Safe-sora: Safe text-to-video generation via graphical watermarking. arXiv preprint arXiv:2505. 2025. 3
Zihan Su, Hongyang Wei, Kangrui Cen, Yong Wang, Guanhua Chen, Chun Yuan, and Xiangxiang Chu. Generation enhances understanding unified multimodal models via multi-representation generation, 2026. URL https: : arxiv. org/ abs / 2 2611. 21406. 1
Xiaofeng Tan, Hongsong Wang, Xin Geng, and Liang Wang. Frequency-guided diffusion model with perturbation training for skeleton-based video anomaly detection. arXiv preprint arXiv:2412.03044, 2024. 3
Xiaofeng Tan, Hongsong Wang, Xin Geng, and Pan Zhou. Sopo: Text-to-motion generation using semi-online preference optimization. Advances in Neural Information Processing Systems, 2025.
2, 9, 21
Xiaofeng Tan, Jun Liu, Yuanting Fan, Bin-Bin Gao, Xi Jiang, Xiaochen Chen, Jinlong Peng. Chengjie Wang, Hongsong Wang, and Feng Zheng. Consistentrft: Reducing visual hallucinations in flow- based reinforcement fine-tuning, 2026. URL https:/t///// /ttp:://axvvvvvo org/ abs / 202. 03425.
Arvin Tashakori, Arash Tashakori, Gongbo Yang. 7. Jane Wang, and Peyman Servati. Flexmo- tion: Lightweight, physics-aware, and controllable human motion generation. arXiv preprint wwXi::2501.16778, 2025. 1
Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano. Human motion diffusion model. In The Eleventh International Conference on Learning Represen- tations, 2023. 2, 3, 8.
Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 8228 122888238,,28 June 2024. 1,3
Haoru Wang, Wentao Zhu, Luyi Miao, Yishu Xu, Feng Gao, Qi Tian, and Yizhou Wang. Aligning human motion generation with human perceptions. arXiv preprint arXiv:2407.02222, 2024. 1
Haoru Wang. Wentao Zhu, Luyi Miao, Yishu Xu, Feng Gao, Qi Tian, and Yizhou Wang. Aligning mo- tion generation with human perceptions. In International Conference on Learning Representations (ICLR), 2025a. 3
Hongsong Wang. Xiaoyan Ma, Jidong Kuang, and Jie Gui. Heterogencous skeleton-based action rep- resentation learning. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 19154-19164, 2025b.
Hongsong Wang. Wanjiang Weng, Junbo Wang, Fang Zhao, Guo-Sen Xie, Xin Geng, and Liang Wang. Foundation model for -keleton-based human action understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025c.
14

---

Published as a conference paper at ICLR 2026
Yiling Wang, Zeyu Zhang, Yiran Wang. and Hao Tang. Safemo: Linguistically grounded unlearning for trustworthy eext-to-motion generation. arXiv preprint orXiv:2601.00590, 2026. 3
Yin Wang, Zhiying Leng, Frederick WB Li, Shun-Cheng Wu, and Xiaohui Liang. Fg-12m: Fine- grained text-driven human motion generation via diffusion model. In Proceedings the IEEE/CVF International Conference on Computer Vision, pp. 22035-22044, 2023. 19
Wanjiang Weng, Xiaofeng Tan, Hongsong Wang, and Pan Zhou. Realign: Text-to-motinn generation via step-aware reward-guided alignment. arXiv preprint arXiv:2505.0497. 2025a. 3, 7, 8, 9, 10
Wanjiang Weng, Hongsong Wang, Junbo Wang, Lei He, and Guo-Sen Xie. Usdrl: Unified skeleton- multi-grained grained feature decorrelation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 8332-8340, 2025b. 3
based dense representation learning with multi
Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint urXiv:2306.09344 2023. 26
Xiaoshi Wu, Yiming Hao, Manyuan Zhang, Keqiang Sun, Zhaoyang Huang, Guanglu Song, Yu Liu, and Hongsheng Li. Deep reward supervisions for tuning text-to-image diffu- sion models. In Computer Vision and Pattern Recognition (ECCV), pp. 108-124, Cham, 2025a. Springer Nature Switzerland. URL https: //i///iin///./////////.......... wwinkk.spriggrr..... www.iinger.oooooeecoom chmp/hhaptr/////// 10. 10.1007/978-3-10 73010--773-----
1, 2, 5, 29
Yongliang Wu, Yizhou Zhou, Zhou Ziheng, Yingzhe Peng, Xinyu Ye, Xinting Hu, Wenbo Zhu, Lu Qi, Ming-Hsuan Yang, and Xu Yang. On the generalization of sft: A reinforcement learning perspective with reward rectification. arXiv preprint arXiv:2508. 0.6.99 2025b.
Xin Xie and Dong Gong. Dymo: Training-free diffusion model alignment with dynamic multi- objective scheduling. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 13220-13230 13230, 2025. 5, 19, 28
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to- text-to-image generation. Advances in Neural Information Processing Systems, 36:159033-5999 2023.
Zeyue Xue, Jie Wu, Yu Gao, Fangyuan Kong. Lingting Zhu, Mengzhao Chen, Zhiheng Liu, Wei Liu, Qiushan Guo, Weilin Huang, et al. Dancegrpo: Unleashing grpo on visual generation. arXiv preprint arXiv:2505.078. 2025. 28
Heyuan Yao, Zhenhua Song. Baoquan Chen, and Libin Liu. Controlvae: Model-based learning of generative controllers for physics-based characters. ACM Transactions on Graphics (TOG), 41()): 1-16, 2022.
Heyuan Yao, Zhenhua Song, Yuyang Zhou, Tenglong Ao, Baoquan Chen, and Libin Liu. Moconvq: Unified physics-based motion control via scalable discrete representations. ACM Transactions Graphics (TOG), -3(,):1-21, 2024.
Ye Yuan, Jiaming Song. Umar Iqbal, Arash Vahdat, and Jan Kautz. Physdiff: Physics-guider human motion diffusion model. In Proceedings of the IEEE/CVF international conference computer vision, pp. 16010-16021, 2023.
Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Yong Zhang, Hongwei Zhao, Hongtao Lu, Shen, and Ying Shan. Generating human motion from textual descriptions with discrete representations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
14730-147440 2023a. 9, 19
Mingyuan Zhang, Xinying Guo, Liang Pan, Zhongang Cai, Fangzhou Hong. Huirong Li, Lei Yang, and Ziwei Liu. Remodiffuse: Retrieval-auu auggmeetd motion diffusion model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 364-373, 2023b.
Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondiffuse: dext-driven human motion generation with diffusion model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(6):41 15-4128, 2024a. 9, 19
15

---

Published as a conference paper at ICLR 2026
Zeyu Zhang, Akide Liu, Ian Reid, Richard Hartley, Bohan Zhuang, and Hao Tang. Motion mamba: Efficient and long sequence motion generation. In European Conference on Computer Vision, pp. 265-282. Springer, 2024b. 9, 19
Kaiwen Zheng, Huayu Chen, Haotian Ye, Haoxiang Wang, Qinsheng Zhang, Kai Jiang, Hang Su, Stefano Ermon, Jun Zhu, and Ming-Yu Yu Liu. Diffusionnft: Online diffusion reinforcement with forward process. arXiv preprint arXiv:2509.16117 2025.
Chongyang Zhong. Lei Hu, Zihao Zhang, and Shihong Xia. Att22m: Text-driven human motion gen- eration with multi-perspective attention mechanism. In Proceedings of the IEEE/CVF International
Conference on Computer Vision, pp. 509-519, 2023. 9, 19
Qiran Zou, Shangyuan Yuan, Shian Du Yu Wang, Chang Liu, Yi Xu, Jie Chen, and Xiangyang Ji. Parco: Part-coordinating eext-to-motion synthesis. arXiv preprint 20X1v:2403.18512 2024. 9, 19
16

---

Published as a conference paper at ICLR 2026
Easy Tune: Efficient Step-Aware Fine-Tuning for Diffusion-Based Motion Generation
Supplementary Material
SUPPLEMENTARY CONTENTS
A More Experimental Results
17
A.1
Experimental Setting for Baseline.
17
A.2
Experimental Results and Discussion about Reward Hacking
18
A.3
Texi-to-Motion Generation Evaluation on KIT-ML dataset.
................................
18
A.4
Visualizations
19
A.5
Quantitative Results on Step- Reward Reweighting
................................
19
A.6
Sensitivity Analysis
21
A.7
Comprehensive Statistics on Overhead
............
24
A.8
Noise-Perception Analysis of Image and Motion
26
A.9
Evaluation on Physical Perception Ability of Reward Model.
................................
26
A.10 Failure Case Analysis: Reward Hacking.
27
B More Technical Discussions
................................
27
B.1 Gradient Analysis of Differentiable Reward Methods
27
........
B.2 Discussion Step Optimization and Policy Gradient Methods ........
28
B.3 Discussions on Existing Fine-Tuning Methods.
29
B.4 Details on Self-Refining Preference Learning
................................
29
B.5 Discussion on Noise-Aware and One-Step Reward
30
C Proof
.........
31
C.1 Proof of Corollary
31
C.2 Convergence Analysis.
32
C.3 Proof of Eq. (5).
34
This supplementary document provides additional experimental results, technical discussions, and theoretical analysis. It is organized as follows: Sec. A presents extended experimental results, including baseline settings, reward hacking analysis, KIT-MI,, qualitative qualitatvee visualizations, ablation studies step-level reweighting and sensitivity analyses, efficiency analyses, cross-domain noise-perception comparison, and failure case analysis. Sec. B offers in-depth discussions on gradient analysis, connections policy gradient methods, existing fine-tuning methods, SPL details, and noise-aware reward strategies. Sec. C contains theoretical proofs, including Corollary 1, convergence analysis, and the derivation of Eq. (5).
generation performance on
A MORE EXPERIMENTAL RESULTS
A.1 EXPERIMENTAL SETTING FOR BASELINE
The hyperparameter configurations for our Easy method and the baseline are provided, with the majority of the settings following the official settings, presented in Tab. S1.
17

---

Published as a conference paper at ICLR 2026
Table S1: Hyperparameters for Easy'Tune and baseline methods.
Hyperparameter
MIDD
AlignProp
ReFI.-10
ReF1.20
DRaFT-10
DRaFT-50
DRTTnee
Eacy
TacyTune Tune (Ouzs)
Description
Random Seed
1234
1254
1234
1234
1254
1234
1254
1234
Seed for reproduciilitt
Banch Size
256
256
256
256
256
256
256
256
Training examples per step
Gradient Clip Range
Maximum gradient norm
Checkpoint Interval
1000
1000
1000
1000
1000
1000
1000
Steps between chockpoints
Reward Weight
le-1
le-1
le-1
le-1
le-1
Ic-1
le-1
Weight for reward loss
Leaming Rate
1e-5
le-5
le-5
le-5
le-5
le:5
Te-5
le-5
Optimizer learning rale
Range
10
(40)50)
[0.50]
[40,50]
[0,50]
Damoising range fine-tuning
Total Steps
50
Desoising schedaler steps
Early-Stop Range M
[40,50]
Early stoly-stop timestep range
TBPTT Length
25
Truncated BPTT window window size
A.2 EXPERIMENTAL RESULTS AND DISCUSSION ABOUT REWARD HACKING
Discussion about Reward Hacking. Previous studies have discussed reward-based differentiable approaches to mitigating reward hacking. Specifically, the proposed strategies include early stopping on a validation set, fine-tuning with LoRA instead of full- parameerrt tuning, and incorporating KL- divergence regularization. Importantly, method can be seamlessly combined with these strategies. In our implementation, we provide support for early stopping on a validation set as well as fine-tuning using LoRA.
Additionally, EasyTune is compatible with established techniques like KL regularization and multi- aspect rewards, which effectively mitigate overfitting, as shown in prior work (D-RaFT, AlignProp, DTune). Following these methods, we omitted KL regularization in our main loss:
KL. EasyyTure Tune ( -Bow m (11,7))
(x t, c) (xL
(SI)
Experimental Results about Reward Hacking. Quantitative analysis (Fig.S3) confirms that our method is free from reward hacking. To further investigate this behavior, conducted a user study adn compare the baseline method with a variant of our approach that incorporates KL-divergence regularization. Specifically, we generated motions for the first 100 prompts in the HumanML33 test set using both the pre- and post-finetuned models. Each generated motion was independently evaluated by five participants. Using MDM and MLD as base models, the results prrseeteed in Fig. S2 and Fig. 7, respectively-shone that our method consistently outperforms the baseline models in human evaluations without exhibiting signs of reward hacking.
Additionally, Tab. S2 provides detailed comparison between the baseline and the KL-regularized variants. Results shows that KL regularization helps mitigate overfitting and improves diversity, although at slight cost in generation quality.
A.3 TEXT-TO-MOTION GENERATION EVALUATION ON KIT-ML DATASET
Tab. S3 presents the quantitative performance of text-to-motion generation models on the KIT- ML dataset, evaluated across multiple metrics: R-Precision (Top-1, Top-2, Top-3) for text-motion alignment, Frechet Inception Distance (FID) for motion quality, Multi-Modal Distance (MM Dist) for semantic relevance, and Diversity for motion variety. The analysis compares models enhanced with EasyTune against comprehensive set of baselines.
The results reveal significant improvements in models enhanced with Easy Tune. For MDM (Chen et al., 2023), Top-1 R-Precision increases from 0.403 0.442 (a 9.7% gain). Top-2 R-Precision from 0.606 to 0.655 (8. 1% gain), and Top-3 R- R-Precision from 0.731 to 0.773 (5.7% gain). FID of MDM decreases substantially from 0.497 to 0.284 (42.9% improvement), indicating enhanced motion quality. MM Dist of MDM improves from 3.096 to 2.755 (11.0% reduction), reflecting
Table S2: Performance comparison between Easy with and without KL-regularized.
Mothod
R@P1
R@P2
R@P3
FID
MM-Dist 1
Div.
Memory
Easy Tune
0.581
0.769
0.855
0.132 0.172
2.637 2.674
9.465 9.482
22.10
Easy EasTTuncKLL
0.575
0,763
0.846
29.70
18

---

Published as a conference paper at ICLR 2026
Table S3: Comparison of text-to-motion generation performance on the KIT-ML dataset.
R Precision 1
Method
FTD
MM Dist
Diversity
Top 1
Top
Top
Real
0.424
0.649
0.779
0.031
2.788
11.08
TM2T (Guo et al., 20226)
0.280
0.463
0.587
3.599
4.591
9.473
T2M (Guo al., 2022a)
0.361
0.559
0.681
3.022
2.052
10.72
M2DM (Kong et al., 2023)
0.416
0.628
0,743
0.515
3.015
11.42
T2M-GPT (Zhang al., 2023a)
0.416
0.627
0.745
0.514
3,007
10.86
Fg-12M (Wang et al., 2023)
0.418
0.626
0,745
0.571
3.114
10.93
ArrT2M (Zhong al., 2023)
0.413
0.632
0.751
0.870
3,039
10.96
MotionMamba (Zhang al.. 2024b)
0.419
0.645
0,765
0.307
3,021
11.02
CoMo (Huang ot al., 2024)
0.422
0.638
0.765
0.332
2.873
10.95
ParCo (Zou al., 2024)
0,430
0.649
0,772
0.453
2,820
10.95
SiT (Meng et al., 2025)
0.387
0.610
0.749
0.242
MDM (Chen
2023)
0.403
0.606
0.731
0.497
3.096
10.76
w/ EasyTum Tune (ours)
0.442.47%
0.655.x1%
0,773.x175
0.284
2.755.11.r5
11.27
MoDilfuse (Zhang 2024a)
0.621
w/ EasyTune (ours)
0.438
0.649
0.777
1.719 (1200
2,892,
10.63.043
-0.2
aptinizatios
sptinun
Bosc Model Preferred
Same
RL.F Mede Preferred
Loss value
-0.4
Matina-Ta Teat Alignment
-0.6
Rectest dacadsa
Physical
Fidelit
18%
41%
39%
-08
Coheretice and Flaency
28%
21%
65%
200
400
800
1000
1200
1400
Iterations
Preference Rate (%)
Figure $1: Loss curves for Easy Tune and exist- ing fine-tuning methods. Here, the x-axis rep- resents the number of generated motion batches.
Figure S2: User study on HumanML3D test set. We use MDM model as base model.
stronger semantic alignment, while Diversity of MDM slightly rises from 10.76 to 11.27, suggesting maintained motion variety. Similarly, for MoDiffuse (Zhang et al., 2024a), Top-1 R-Precision improves from 0.417 to 0.438 (5.0% gain). Top-2 R-Precision from 0.621 to 0.649 (4.5% gain), and Top-3 R-Precision from 0.739 to 0.777 (5.1% gain). FID of MoDiffuse decreases from 1.954 to 1.719 (12.0% improvement), and MM Dist of MoDiffuse reduces from 2.958 to 2.892 (2.2% improvement). However, Diversity of MoDiffuse slightly declines from 11.10 10.63, indicating minor trade-off in variety for improved alignment and quality. Compared to baselines, eesy'Tune-enhanee models achieve superior performance. Top-1 R- R-Precision of MDM with Easy'Tune (0.442) surpasses that of ParCo (0.430) and MotionMamba (0.419), while FID of MDM with Easy Tune (0.284) is competitive with SiT (0.242). MM Dist of MDM with EasyTune (2.755) outperforms most baselines, approaching the real data's ' 2.788. These results establish Easy'l BasyTTuneeent Tune-enhanced models as new state-of-the-ar -art on KIT-MI,, highlighting their ability to improve text-motion alignment, motion quality, and semantic relevance.
A.4 VISUALIZATIONS
We visualize motions generated by the original MLD (Chen et al., 2023) and by MLD fine-tuned with our EasyTune, as shown Fig. S3. Our proposed EasyTune substantially improves the capacity of text-to-motion models to comprehend textual semantics. For example, in Fig. S3(j,, the model fine-tuned with our proposed Easy Tune effectively generates a motion that accurately reflects the semantic intent of the description "The man is marching like a soldier," whereas the original model fails to capture this nuanced behavior.
A.5 QUANTITATIVE RESULTS ON STEP-LEVEL REWARD REWEIGHTING
Prior work has demonstrated that early demoising steps exert substantial influence the final generation quality (Xie & Gong, 2025). We further observe that existing trajectory-level optimization methods frequently under-optimize these early steps, as discussed in Sec. 3. To systematically
19

---

Published as a conference paper at ICLR 2026
EasyTume Easy
EasyTune Easy Tume
Easy EasyTune A person side steps left and then stands straight
(a) Person moves their body kicks with left foot forward
(b) Person sits down in chair sorts and then gets back
Easy Easy
EasyTune Tune (e) A person squats down
Easy
Fasy
EasyTTuee
(d) A person balances on one leg
(f) Person over
wlo EasyTune
w/ Easy
w/o Easy Tune person walks beam
Easy Tune
(g) Walking circle with hands
Easy Tune
wi EasyTune
w/o Easy
EnsyTune
(i) person walks forward then stops and left hand circles
(i) The man is marching like soldier
Figure S3: Visual results on HumanMI.3D dataset. "w/o EasyTune" refers motions generated by the original MLD model (Chen et al., 2023), while "w/ Easy'l EasyTTune" indicates motions generated by the MLD model fine-tuned using our proposed EasyTune.
examine the effect of fine-tuning different subsets of steps, we evaluate four alternative reweighting strategies: (1) optimizing only the final 20 steps; (2) optimizing only the initial 20 steps; (3) linear increasing reweighting with w, T- T + 0.5 at each step; and (4) linear decreasing reweighting with wy - + 1.5 at each step. Results are summarized in Tab. $4.
The experimental results Tab. highlight the critical role of appropriately weighting differ- ent denoising steps. Strategy (1), which optimizes only the final 20 steps. yields clearly inferior performance (R@I: I: 0.546, FID: 0.184) compared to Easy Tuny (Full), indicating that restricting
Table $4: Ablation study step-level reward reweighting strategies for Easy The baseline is MLD.
R Procision
Method
FID
MM Dist J
Diversity
Memory (GB)
Top
Top
Top
Real
0.511
0.703
0.797
0.002
2.974
9.503
DRaPT-50 (Clark 2024)
0.197
w/ Easy Tune (Full)
0.581
0.769
0.855
0.132
2.637
9.465
22.10
Easy Tune (()
0.546
0.735
0.184
2.815
22.10
Easy Tune (2)
0.567
0.759
0.842
0.158
2,673
9,430
22.10
Easy
Easy Tune (4)
0.584
0.773
0.859
0.136
2.631
9.521
22.10
20

---

Published as a conference paper at ICLR 2026
Table S5: Performance of SPL mechanism for fine-tuning TMR.
Text-Motion Retrievalf
Motion-Ten Retrievalf
Methods
R@
R@2
R@3
R@5
R@ 10
R@1
R@2
R@3
R@5
R@ 10
TMR
67.16 68.76
81.32
86.81 87.99
91.43
95.36 96.47
67.97 69.03
81.20
86.35
91.70
95.27
+SPI.
82.36
92.06
82.87
87.84
92.56
96.45
optimization to late steps is insufficient to fully exploit the potential the diffusion process. This result is also comparable to that of DRaFT-50 (R (R@1: 0.528, FID: 0. 09197), suggesting that DRaFT-50's suboptimal performance likely stems from gradient vanishing, which effectively causes the method to under-optimize early denoising steps. Strategy (2), which optimizes only the initial 20 steps, achieves somewhat better performance (R@1: I: 0.567, FID: 0,158) than Strategy (I), but still falls short of EasyTune (Full), showing that early-step optimization alone is also not sufficient.
For the linearly increasing reweighting scheme (Strategy (3), We = 7 + 0.5), which places larger weights later steps. In contrast, the linearly decreasing reweighting scheme (Strategy (4), Wy = - + 1.5), which emphasizes earlier steps, achieves the best overall performance among the reweighted variants (R@1 1: 0.584, FID: O. 0.136, MM Dist: 2.631). Notably, its performance is on with, or slightly better than, EusyTune (Full) in terms of alignment (R @ I: 0.584 vs. 0,581) while maintaining comparable generation quality. Taken together, these results provide strong empirical evidence that properly optimizing early denoising steps is crucial for downstream performance.
A.6 SENSITIVITY ANALYSIS
A.6.1 SENSITIVITY ANALYSIS OF RETRIEVAL MODEL SELECTION
Effect on Settings. We investigate the sensitivity of our SPL mechanism to different reward models and their impact on final fine-tuned generation results. A key observation is that weaker reward models, when trained using hyperparameter settings optimized for stronger models, can suffer from training collapse. This occurs because the core principle of SPI. is to mine motion pairs and maximize the gap between preferred and non-preferred motions. In other words, the model must learn to produce preferred motions while forgetting non-preferred ones. However, forgetting is inherently simpler than learning, and weaker retrieval models are more prone to mining incorrect pairs during online sampling. To address this, we employ more relaxed candidate number K to increase the probability of successful pair mining, thereby strengthening learning signals and reducing erroneous unlearning.
Effect on Text-Motion Retrieval Task. We demonstrate this using TMR, moderately weaker retrieval model, where SPL consistettyyy improves performance. As shown in Tab. S5, the motion-text retrieval R@ improves from 67. to 68. 76%, and motion-text retrieval @@ 1 improves from 67.97% 69.022 19.03%, confirming that our method generalizes effectively weaker reward models with appropriate hyperparameter adjustments.
Effect Text-to-Motion Generation Task. We further explore whether step-aware fine-tuning generalizes to weaker pre-trained reward models. As shown in Tab. S6, even when using TMR, a less discriminative retrieval model compared SPL, our step-aware optimization approach consistently improves generation quality. Specifically, when combined with Easy Tune using TMR as the reward model, both the step- and oofhnu- variants achieve substantial gains: the step variant reaches R @ 1 of 0.573 (vs. baseline 0.504), and the chain variant reaches 0.567. They still represent significant improvements over the baseline. This finding suggests that the effectiveness of step-aware optimization is not solely dependent insight is that even weaker but still discriminative reward models can provide effective supervision signals, their ranking capability, though inferior to stronger models, still exceeds that of the base generation model itself (Tan et al., 2025). This robustness to reward model choice broadens the applicability of our approach.
using the strongest available reward model. Rather, the key
21

---

Published as a conference paper at ICLR 2026
A.6.2 ANALYSIS OF CANDIDATE NUMBER K SELECTION & RETRIEVAL POOL
Mechanism. As discussed above, the core mechanism of our SPL is to mine preference motion pairs online and to maximize the learning signal by enlarging the gap between preferred and non-preferred pairs. In essence, SPL is designed to forget incorrectly generated motions while retaining correct ones. However, this task is inherently asymmetric: forgetting is much easier than remembering, making it crucial to carefully control the frequency at which negative samples are forgotten. In our implementation, both the candidate number K and the retrieval pool configuration substantially affect this behavior. Specifically, when retrieval fails, SPL jointly learns to memorize correct samples and forget incorrect ones; otherwise, the model simply optimizes the original pre-training objective. A larger K and smaller retrieval pool generally increase the retrieval success rate. Intuitively, we should therefore choose K and the retrieval pool such that successful retrievals occur substantially more often than failures.
Experimental Settings. Fortunately, Petrovich al. (2023) has discussed similar retrieval settings. Specifically, four retrieval pool settings are considered: (a) All: Using the entire test set without modification, though this can be problematic due to repetitive or subtly different text descriptions (e.g., "person" vs. "human", "walk" vs. "walking"). (b) All with threshold: Searching over the entire test set but accepting a retrieval as correct only if the text similarity exceeds a threshold (set to 0.95 [0, 1] scale). This approach is more principled, distinguishing between genuine matches and superficially similar pairs like "A human walks forward" vs. "Someone is walking forward". (c) Dissimilar subset: Sampling 100 motion-text pairs with maximally dissimilar texts (via quadratic knapsack approximation). This provides a cleaner but easier evaluation setting. (d) Small batches: Randomly sampling batches of 32 motion-text pairs and reporting average performance, providing a more realistic in- thl- scenario.
Among these four configurations, settings (a) and (b) yield low retrieval success rates and are computationally expensive. Their success rates are often even lower than the failure rates, which substantially hinders the practical deployment of SPL. By contrast, configurations (c) and (d) achieve much higher retrieval success rates. In our main experiments, we therefore adopt setting (d). With K == 10, configuration (d) attains a retrieval failure ratio of approximately 1:20. This ratio empirically leads to stable optimization. In comparison, configuration (c) exhibits a failure ratio of about 1 :66 at K = 10, which tends to result in less stable performance due to the higher frequency of retrieval failures.
Results & Discussion. Based on configuration (c) and (d), we systematically study how different choices of K and retrieval pool settings influence generation performance. Specifically, we evaluate K E {100 15, 20) for both settings, and report the results Tab. S7.
In practice, recommend conducting similar sensitivity analysis under limited computational resources (e.g.. 10 minutes on a single GPU) to determine appropriate values of and the retrieval pool for given application. A retrieval failure ratio of approximately 1:20 typically leads to stable and robust optimization across different scenarios.
A.6.3 AVAILABILITY OF NOISE-AWARD REWARD
Experimental Setting, In this experiment, we assess the noise-aware availability of our reward model, ie.,, how well the underlying retrieval models operate under noisy motion inputs. Specificallly, based on the HumanML33 dataset, we construct noisy test samples x, by running the forward diffusion process of MLD. For noise-aware reward models, directly evaluate on x,- For output- aware models (TMR (Petrovich et al., 2023)), we instead apply an ODE-based denoising process to x, and use the resulting predicted clean samples &n inputs.
Table S6: Fine-tuning performance using TMR as reward model. The baseline is MLD.
Method
Precision
FID 1
MM Dist 1
Diversity +
Top
Top
Top
Real
0.511
0.703
0.797
0.002
2.974
9.503
+TMR (w/ SPL, Step)
0.573
0.760
0.843
0,173
2.682
9.942
+TMR (w/ SPI., Chain)
0.567
0.753
0.836
0.158
2.698
9.874
22

---

Published as a conference paper at ICLR 2026
Table S7: Sensitivity analysis of the number candidate motions and retrieval pool settings. The baseline is MLD. Numbers in the Method column denote the value of K.
Precision
Method
FID
MM Dist
Diversity +
Top
Top
Top
Real
0.511
0.703
0.797
0.002
2.974
9.503
K 10
0.581
0.769
0.855
0.132
2.637
9.465
K 15 (d)
0.571
0.758
0.843
0.142
2,668
9.486
K (d)
0.564
0.747
0.184
2.704
9.629
K 10
K 15 (c)
0,585
0.773
0,859
0.155
2.626
9.428
K (c)
0.574
0.759
0.844
0.149
2.653
9.495
Table S8: Experimental results of Noise-Aware Text-Motion Retrieval.
Methods
Input Duta
Text-Motion Retrievalt R@2
Motion-Text Retrievalt
R@1
R@3
R@5
R@ 10
R@1
R@2
R@3
R@5
R 10
RcAlign
Clean Data
67.59
82.24
87.44
91.97
96.28
68.00
82.86
87.95
92.44
96.28
ReAlign
Noisy Data
67.20
81.46
87.11
91.39
95.67
68.02
81.84
87.56
91.39
95.69
ReAlign
Predicted Clean Data
67.80
82.38
87.86
92.27
96.61
68.04
82.47
87.97
92.10
96.34
SPL
Clcan Data
69.31
83.71
88.66
92.81
96.75
70.23
83.41
88.72
93.07
97.04
SPL.
Noisy Data
69.36
83.63
88.53
92.83
96.76
70.34
83.41
88.66
93.04
96.93
SPL
Predicted Clean Data
68.39
83.31
96.73
68.
82.35
88.06
92.79
96.53
TMR
Clean Data
67.16
81.32
86.81
91.43
95.36
67.97
81.20
86.35
91.70
95.27
TMR
Predicted Clean Data
66.98
81.04
87.09
92.11
95.74
68.32
80.69
86.43
92.13
95.84
Results & Discussion. As shown in Tab. S8, both noise-aware models (ReAlign and SPL) demon- strate remarkable stability across conditions, with SPL achieving virtually identical performance noisy data (69.36% @ 1) versus clean data (69.31% R@ 1), while output-aware models like TMR be effectively adapted via ODE-based denoising (66.98% recovery from clean baseline 67. 67.16%). collectively validating that modern retrieval models reliably perceive intermediate denoising states and can robust reward models for diffusion-based optimization.
A.6.4 SENSITIVITY ANALYSIS ON LEARNING RATE
To further assess the robustness of our approach, we examine the sensitivity EasyTune to the learning rate hyperparameter. Across reasonable range of learning rates (from 10-5 to x 10-1), our method exhibits only minor performance variation, indicating strong robustness to this critical hyperparameter. The corresponding results shown Fig. $4.
3.1
0.86
0.6
Ire2e:s Iowle
Ic-le 05
3.0
Ie=2w-t Us
Icll 05
MM Dist
0.84
2.9
R3
0.62
2.8
Im=1=05
1n-2-05
b-1:
0.3
2.7
2.6
250
500
750
250
500
750
250
750
Step
Step
Step
(a) R- Precision (Top-3)
(b) FID
(c) MM Distance
Figure S4: Learning rate sensitivity analysis on validation set. Performance metrics remain stable across the learning rate range (spanning from 2 10 to 10 10-5), demonstrating the robustness of EasyTune to this hyperparaneter. (a) R- R-Precision at Top-3; (b) Frechet Inception Distance; (c) Multi- Modal Distance.
23

---

Published as a conference paper at ICLR 2026
Aligz
Uvage Racigrguggntie
35000
DRaFT DRTuo
3306
DRaFT
Memnay Usage (MD0
Our
Menory Usage (MBB
50006
(Optinzz Desoizin Prices)
Dues (Optizz Otee fo Per Denocsing Preces))
20064
2000
15006
1900
Monury Aurreaoe
5066
1900
Loat Model o8
Faaode Proeugt
Dartin Drnnntiiog
5000
Lond Model
Ennoe Prcept
VnEl Deweling
Oytixatike: Aft Backwad
Key Stages
(a) Memory Usage for Key Stages
(b) Memory Growth per Denoising Step
Figure S5: Comprehensive memory analysis of Easy Tune and existing fine-tuning methods. We report the memory usage of key stages (model loading, prompt encoding, denoising, VAE- based motion decoding, and reward computation with backpropugation), trajectory during optimization. EasyTunn achieves lower peak memory while maintaining high utilization, benefiting from the (1) memory growth of the denoising process.
well the full memory
A.7 COMPREHENSIVE STATISTICS ON OVERHEAD
A.7.1 MEMORY OVERHEAD
In this part, we provide a detailed comparison of the memory consumption of existing fine-tuning methods and our EasyTTuee framework. Our pipeline consists of several key stages, including loading the model, encoding text prompts, denoising in the latent space, decoding motions via the VAE, and computing rewards followed by backpropagation. As illustrated Fig. S5, we report both (a) the memory usage of each individual stage and (b) the overall memory trajectory throughout the full optimization process. All experiments are conducted on a single NVIDIA RTX A6000 GPU, with Intel(R) Xeon(R) Silver 4316 CPU @ 2.30GHz.
worth noting that our method performs multiple optimization steps within single denoising trajectory, which leads to relatively high average memory utilization. However, the peak memory consumption of EasyTuun Tune is significantly lower than that of existing methods. In practice, higher utilization indicates more efficient use of available GPU resources, while a lower peak memory footprint reflects reduced hardware requirements. The results further demonstrate that the memory savings of our method mainly stem from the (1) memory growth of the denoising process.
A.7.2 COMPUTATIONAL OVERHEAD
In this part, we benchmark the training-time and computational overhead of Easy against existing fine-tuning methods. Following the setup used in our main experiments, we measure the training time and total TFLOPs required to reach convergence. Additionally, all methods share the same sampling procedure as their corresponding base diffusion models, and thus incur no additional overhead during inference.
As shown in Tab. S9, EasyTune is consistently more training-efficient than prior differentiable reward-based methods. It achieves the lowest per-step optimization cost (1.47 seconds per update vs roughly 4.7-5.6 seconds for other methods), and to reach a reward score of 0.70 needs only 263.36 seconds and 10191 TFLOPs, compared to 466.27 seconds and 18044 TFLOPs for DRaFT. For reward score 0.75, EasyTune converges in 358. 17 seconds (13861 TFLOPs), while DRaFT requires 2616.54 seconds (101260 TFLOPs), yielding 7.3x speedup under much smaller compute budget. Moreover, Easy Tune is the only method that successfully reaches reward scores of 0.80 and 0.85 within the given budget, highlighting its stronger optimization capacity. Together with the memory efficiency analysis in Fig. S5, these results show that Easy Tune offers a substantially more efficient and practical fine-tuning strategy than existing differentiable reward-basee approaches.
24

---

Published as a conference paper at ICLR 2026
Table S9: Computational overhead comparison. report the training time and TFLOPs required to reach different reward scores. Total time is measured in seconds single NVIDIA RTX A6000 GPU. s." indicates the method could not reach that reward level within a reasonable training budget.
DRaFT
AlignProp
DRTune
ReFL
EasyTune Tune (Ours)
Time per Opt. (8)
4,90
4.72
Reward Score
Time (6)
466.27
271.99
554.77
820.29 31745
263.36
TFLOPs
18044
10526
21469
10191
Reward Score
0.75
Time TELOPs
2616.54
971.55
2009.59
358.17
101260
37599
77771
13861
Reward Score
0.80
Time (s) TFLOPs
452.53
17513
Reward Score = 0.85
Time (6)
1025.17
Natte Pradiake
New Fradedas
Verve
Similarity
Similarity
Similaritt 6
0.0
0.6
10:11 14
10 12 12 25
10 15
Denoising Stop
Demoising Stop (t)
Donoising Stop
(a) Noised Image v.s. Predicted Image
(b) Noised Motion vss Predicted Motion
(c) Predicted Image vas. Predicted Motion
Figure S6: Noise-perception comparison between images and motions. We report the cosine similarity between noisy states and their ODE-based predictions etween ODE-based predictions and trajectory-level rewards, across denoising steps for both the image (FLUX.1 dev) and motion (MLD) domains.
1:14
t=12
t=10
t=8
1:6
Noised Images
1=15
1=14
1:13
1=12
1=10
t=6
(b)Predictdd Image 3; via ODE
Figure S7: Noisy states and ODE-based predictions for images denoising steps. Visualiza- tion of intermediate noisy images and their corresponding ODE-based predictions at different steps of the denoising process using FLUX.1 dev.
25

---

Published as a conference paper at ICLR 2026
toheed Skdeeny
512
h:1
hd0
1:P
1-3
11
16
OfPrcdeeed Make 2, CIT
Figure S8: Noisy states and ODE-based predictions for motions across denoising steps. Visual- ization of intermediate noisy motions and their corresponding ODE-based predictions at different steps of the denoising process using MLD.
A.8 NOISE-PERCEPTION ANALYSIS OF IMAGE AND MOTION
Experimental Settings. To quantify the perceptibility and sensitivity of noisy states, we perform a unified study on both image and motion generation. For the image domain, select 50 prompts from HPDv2 (Wu et al., 2023) and use FLUX.1 dev (Labs, 2024) as the base model. For each prompt, generate 12 images with 16 denoising steps, guidance scale of 3.5, and a resolution of 720 720. At each of the 16 steps, we store both the noisy image and its ODE-based prediction, and compute their cosine similarity using CLIP ViT-L/14 features (Radford et al., 2021). To ensure statistical significance and robustness, all experiments are repeated 50 times per prompt over 50 prompts, and we report the mean.
For the motion domain, we follow a similar protocol using MLD as the base model and TMR as the feature extractor. We randomly sample 50 text prompts from HumanML3D and, for each prompt, generate 12 motion samples along the full denoising trajectory. At each step, we compute the cosine similarity between the noisy motion and its ODE- based prediction in the TMR feature space.
Results and Discussion. As summarized in Fig. S6, both image and motion models exhibit increas- ing similarity between noisy states and ODE-based predictions as the denoising process proceeds. However, the image final results consistently shows weaker similarity with noised state, especially at high-noise steps. This indicates that image generation results have a poorer perception early noise states compared to motion generation, making it harder for directly fine-tuning models by step-level optimization. Fortunately, their ODE-based prediction results show comparable performance, indicat- ing that their step-aware optimization can still be performed by ODE-based prediction. Additionally, visual examples of noisy states and their ODE- based predictions are presented Fig. S7 and Fig. S8, showing the perceptual differences across denoising steps for both domains.
A.9 EVALUATION ON PHYSICAL PERCEPTION ABILITY OF REWARD MODEL.
To investigate the physical perception capabilities of our reward model, we conducted an experiment to assess its ability to distinguish between real and generated motions. We selected 50 prompts and their corresponding ground-truth motions (Xgz) from the HumanML3D test set. For each prompt, we also generated motion (Xgen) using the pretrained MLDD model. We then evaluated both the ground-truth and generated motions using our reward model with an empty text condition (c ")), obtaining reward scores r rxxgr. c) and r r(Xgen: c).
Table S10: Physical perception evaluation of the reward model.
Comparison Result
Count
r(Xgi, c) r r(Xgen, c) r(X,i, c) S r r(Xgen c) 2
48
Total
50
Our findings are summarized in Tab. S10. The reward for the ground-truth motion was higher than for the generated motion in 48 out of 50 cases (96%). This result strongly suggests that the reward model possesses significant degree of physical perception. This capability likely arises because the reward model is trained on real-world motion data, treating it as in in-distribution, while viewing synthetically generated data out-of-distribution. Such distinction enables the model to
26

---

Published as a conference paper at ICLR 2026
Prompt: person stands from the ground, lifts their right foot, and sets back down.
Prompt: A squats down then stands up and moves forward.
Reward Hacking
w/ KL
Reward Hacking
wi KL
Figure S9: Illustration of reward hacking in motion generation. Examples demonstrating that over-fitting to reward signals may lead to semantically aligned but physically unrealistic motions. For better visualization, corresponding videos are provided in the supplementary materials.
develop a sensitivity to physical plausibility, phenomenon similarly observed in anomaly detection literature (Flaborea al., 2023).
A.10 FAILURE CASE ANALYSIS: REWARD HACKING.
As well-known challenge in reinforcement learning (Clark et al., 2024), reward hacking can emerge during model fine-tuning, where continued optimization after convergence may degrade generation quality.
As illustrated in Fig. S9, models may over-fit semantic alignment while neglecting realistic motion dynamics. For example, given the prompt "A person stands up from the ground, lifts their right foot, and sets back down," a model suffering from reward hacking might generate person continuously lifting their foot to over-fit to the "lifts their right foot" action. Similarly, for "A person squats down, then stands up and moves forward," the model might misinterpret this as "A person squats down while moving forward." Fortunately, our method combined with KL-divergence regularization (as discussed in See. A.2) can effectively mitigate this phenomenon. We recommend early stopping after initial convergence, with the stopping point determined by validation performance, which provides an effective strategy to alleviate reward hacking issues.
B MORE TECHNICAL DISCUSSIONS
B. GRADIENT ANALYSIS DIFFERENTIABLE REWARD METHONS
Given a diffusion model cg. we first sample a latent variable Xy ~ AN I). Starting from xy, we apply T denoising steps obtain x,, while retaining the computational graph for gradient analysis. At step I, we consider the Jacobian of the diffusion prediction with respect to its input.
(S2)
0x
and its sequential product across denoising steps, 1
Omo (x²)
II
(S3)
0x!
This quantity can be interpreted as the gradient of the noisy sample xf, with respect to the predicted clean sample x i.e., the effective coefficient governing optimization step I. Consequently, the gradient of the training objective takes the form
OC(0) 00
O
OTa (x 3,)) Ox!
Omo (x
-EeDDy 28 (-)--))
(S4)
OxL
00
optimizing 1-th step
tends as increases
27

---

Published as a conference paper at ICLR 2026
Importantly, during optimization, especially when t is large, these coefficients rapidly decay toward 0 (as shown in Fig. 3), ie.,
aro (x II
0.
(S5)
Ox²
Thus, for large noise steps, which often determine the final generation quality (Xie & Gong, 2025), this vanishing effect implies that existing methods tend to under-optimize such steps, thereby leading to coarse or suboptimal results.
B.2 DISCUSSION ON STEP OPTIMIZATION AND POLICY GRADIENT METHODS
In the field of diffusion pusting-training beyond mentions differentiable-rewaat approaches, another major line of work is policy-gradient methods, such as GRPO (Xue et al., 2025; Liu et al., 2025), PPO (Ren al., 2024), and DDPO (Black et al., 2023). this section, we point out that, both our approach and these volicy-gradien methods truncate the gradient along the diffusion chain, and in practice, they have not been observed to suffer from global inconsistency issues. As result, our method inherits widely accepted and principled design, and does not suffer from undesirable behaviors caused by breaking the differentiable chain structure. Here, we briefly discuss the connections between our method and existing volicy-gradide methods.
GRPO. DanceGRPO (Xue et al., 2025) can be viewed as instantiating GRPO (Guo et al., 2025) in the diffusion setting, where each denoising trajectory sampled under a condition is treated rollout generated by the policy pa- For every prompt c, a group g = fx l: of K trajectories is drawn from the current policy, and a group-wise relative advantage is computed to compare their rewards. By estimating advantages A from this group of K rollouts, GRPO (Xue et al., 2025) optimizes a PPO-style clipped surrogate objective:
JCRPO (0, 9) BoWD x x (, (L. min kit A clip(pp
1+ A+))
(S6)
where advantages A and probability ratios Pg k, are computed as follows:
A r(xt, c) p(((x! .c) Pe" Posia (xx xx
(x xi_
(S7)
a(r(xl:K,c) c))
where r(xh,c) c) is the scalar reward assigned to the final sample in the k-th rollout, and u(r(x1: .c) and a (r(xx:: lik c)) denote the mean and standard deviation of rewards within the group x This group-wise normalization makes AA a relative score that measures how much better (or worse) a sample is compared its peers under the same condition which stabilizes training and mitigates scale mismatch across prompts.
DDPO. As classical RL method, DDPO (Black al., 2023; Fan al., 2023b) optimizes the generative diffusion model via policy gradients using K rollouts. Its objective is written as:
VeJDDPO (0, 9) E,,D, x (,,T)) (-)(),
[r(x]i c) V lo (x(" c]] (S8)
Here r r(xti e) again denotes the scalar reward assigned to the final denoised sample in the k-th rollout, while log Po (xf x =)) is the bog-probability at step I.
Discussion. By examining Eq. (S7) and Eq. (S8), we identify a key observation: both GRPO and DDPO treat this optimization as multiple pre-step optimization (in the form of log (o(f xx=_,,)) c)), rather than optimizing the entire denoising process in single optimization pass. Specifically, GRPO and DDPO first sample K trajectories of length T under each condition. Subsequently, they assign scalar reward to each trajectory and use this reward weight progressively imitate these trajectories step by step.
However, observe fundamental inconsistency in existing differentiable-reward methods: differentiable-reward methods recursively decompose the optimization process as the opti- mization of multiple sub-chains, rather than as the optimization of independent steps. This design choice introduces the challenges discussed in the Sec. 3, particularly the vanishing gradient and memory problem illustrated in Fig. and 6. In contrast, our EasyTune solve this issue by step-aware optimization, and its objective is:
LEasy (0) BenDD x (e). (x( c))],
(S9)
28

---

where (x,, c) denotes the reward directly assigned to the intermediate denoised sample at step f, rather than a single trajectory-level scalar. Crucially, our approach evaluates and optimizes at each timestep independently, with uniformly sampled from (o...., ,).. Therefore, compared to the chain optimization as shown in Eq. (S4), method is more aligned with the principled design of Published as a conference paper at ICLR 2026
Algorithm 1 Easy Tuns:Tun: Efficient Step-Aware Fine- Tuning
Input: Pre-trained diffusion model çø, reward model Rø-
Output: Fine-tuned diffusion model eg.
1: for each text condition
"rr and not converged do
2: Xp ~ (N,, I)
3: if Chain Optimization then
4:
Copy +
5: end if
6: for f = T.... do
7:
Denoise by 0: x (x{) by = =x Eq. (7) if Chain Optimization then
8:
9:
Optimize: update diffusion model eæ by gradient from Eg: @.euw (0)
in Eq. (6)
86
10:
else Optimize: update diffusion model co by &CEnny (8) in Eq. (6)
09
12:
end if
13:
Stop Gradient: x - sg(x(-1)
14:
end for
15: if Chain Optimization then
16: Assign 0 + 6'
17:
end
18:
end for
policy-gradicns methods, while maintaining a step-wise optimization structure that avoids long-chain backpropagation and achieves better results.
B.3 DISCUSSIONS EXISTING FINE-TUNING METHODS
We compare EasyTune with existing direct reward fine-tuning methods, including DRaFT-K (Clark et al., 2024), AlignProp (Prabhudesaí et al., 2023), ReFL. (Clark et al., 2024), and DRTune (Wu et al., 2025a). The pseudocode of EasyTune provided in Algorithm 1. We highlight four key advantages:
(1) Higher Optimization Efficiency: Existing methods update model parameters only after complet- ing or T' tstop reverse steps, yielding one update per trajectory. EasyTune optimizes at every denoising step. significantly increasing update frequency and enabling faster convergence.
(2) Lower Storage Requirements: DRaFT-K, AlignProp, ReFL, and DRTune rely on recursive gradient computations, requiring storage of intermediate states across multiple timesteps (Eq. (4) and Eq. (10).. EasyTTune computes gradients solely for the current timestep (Eq. (7)), eliminating recursive state storage and substantially reducing memory usage.
(3) Fine-grained Optimization: Existing methods optimize over coarse timestep ranges or rely on early stopping, limiting their ability to capture step-specific dynamics. EasyTune performs optimization at each denoising step (Algorithm I), enabling precise per-step adjustments to better align with motion generation objectives.
(4) Simpler Pipeline: Existing methods introduce complex designs such as variable timestep sampling early stopping mitigate optimization and storage challenges. EasyTune simplifies the process via step-wise optimization (Algorithm 1), making it more straightforward and broadly applicable.
B.4 DETAILS ON SELF-REFINING PREFERENCE LEARNING
The Self-Refining Preference Learning (SPL) mechanism constructs preference pairs for reward model fine-tuning without human annotations, using retrieval-basse auxiliary task. Algorithm
29

---

Published as a conference paper at ICLR 2026
Algorithm 2 Self-Refining Preference Learning
Input: Training subset Dr, text/motion encoders E &r, temperature parameter т, retrieval number
k.
Output: Fine-tuned reward model EM, &r, and T.
I: Initialize: Parameters @ f- {EM. &r,
2: for each data pair (x8t c) E Dr and not converged do
3: Step 1: Preference Data Mining
4: Compute reward scores for all x E Dr using Eq. (11)
5:
Retrieve top-k motions PR using Eq. (13)
6:
Set winning x" " and losing motions x' using Eq. (14)
7:
Step 2: Preference Fine-tuning
8:
Compute softmax probabilities P using Eq. (15)
9:
Define target distribution Q using Eq. (16)
10: Compute loss LsPr. (() by Q and P using Eq. (17)
Update parameters o by VoLsPL())
12:
end for
outlines the process, which iterates over a training subset DD of motion-text pairs to refine text and motion encoders εT. EM, and a temperature parameter т, collectively parameterized as Φ.
Algorithm 2 formalizes the process of SPL, which operates on a training subset DD containing motion-text pairs (x8*, c), utilizing pre-trained text and motion encoders &r, EM, and temperature parameter collectively parameterized as ø. Overall, at each optimization iteration, SPL attempts to mine a preference pair consisting of a winning motion x and losing motion x'. such a pair is found (i.e., when retrieval fails), the model is optimized based on this pair; otherwise (i.e., when retrieval succeeds), it falls back to the pretraining objective to reinforce the correct knowledge. The algorithm executes two core steps: Preference Data Identification and Preference Fine-tuning. In the first step (Lines 3-6), for each text condition c, reward scores are computed for all motions in DT based on the similarity between motion and text features scaled by r. The top-k motions are retrieved, and the ground-truu -ruth motion x8 is designated as the preferred motion x" x is not among the retrieved motions, the highest-scoring retrieved motion is set the non-preferred motion x otherwise, xl is set to xst, and optimization is skipped to avoid trivial updates. This retrieval- based approach effectively mines preference pairs by identifying motions that are incorrectly favored by the current model, thus providing a robust signal for refinement. In the second step (Lines 7-10), the reward scores of the preference pair (x" x)) are converted into softmax probabilities P. representing the model's predicted preference distribution. These are aligned with a target distribution Q, which assigns probability of 1.0 to x" and 0.0 to x' when a preference exists, or 0.5 to both when they are identical. The model is optimized by minimizing the KL divergence between Q and P, with the resulting loss used to update é via gradient descent. This fine-tuning process iteratively refines the encoders to assign higher scores to preferred motions, enhancing the reward model's 's ability to capture fine-gres tnn-grained preferences. The iteration continues until convergence, yiclding a reward model tailored for motion generation tasks.
The
B.5 DISCUSSION ON NOISE-AWARD AND ONE-STEP REWARD
In Sec. 4.1, we introduced both the Noise- reward for ODE and SDE sampling and the One-Step reward specifically for ODE sampling. Here, provide recommendations for selecting between these strategies and briefly compare their performance.
Perceptual Difference Between Noisy and Predicted Data. As analyzed in App. A.8, the pre- dictability of noisy data in the motion domain is relatively strong compared to the image domain (see Fig. S6). Fig. S8 and Fig. S7 demonstrates that ODE-based strategy further enhances this predictabil- ity. Consequently, both reward strategies can effectively perceive noisy data in motion generation. For image generation, where noisy data is harder to interpret, we recommend the One-Step reward strategy for more accurate perception.
Quantitative Analysis of Retrieval Results on Noisy Data. In App. A.6.3, we quantitatively analyze the performance difference between the two strategies on retrieval tasks using noisy data.
30

---

Published as a conference paper at ICLR 2026
The results in Tab. S8 demonstrate that both strategies achieve robust performance noisy data retrieval, comparable to results on clean data.
Quantitative Comparison Generation Results. For ODE-based models, both strategies are applicable. In Tab. 2, we provided performance metrics for MLD and MLD++ under both strategies. We revisit and consolidate those results in Tab. S11. The results indicate that the Noise-Awar reward generally yields better performance. Therefore, recommend using the Noise-Aware strategy if the reward model possesses noise perceetinnn capabilities. Otherwise, the -ne-Step reward can achieve comparable results.
Table S11: Comparison of Noise-Aware and One-Step Rewards on ODE-based models.
R Preccision
Model
Strategy
MM-Dist J
Top I
Top
Top
One-Step
0.568
0.754
0.846
0.194
2.672
MLD
Noise-Aware
0.581
0.769
0.855
0.132
2.637
One-Step
0.581
0.762
0.849
0.073
2.603
MI.++
Noise-Aware
0.591
0.777
0.859
0.069
2.592
C PROOF
C.I PROOF OF COROLLARY
Recall the Corollary
Corollary. Given the reverse process in Eq. (2), xf-1 = To(x} t, c). the gradient w.r diffusion dxL_, can be expressed as:
model 0, denoted as 06
dxf_1 @To((x(,L,))
(x)(xx,(,c) 0x?
00 +
(S10)
00
Ox? 00
Proof. Let u(0) x v(0) 0. and define F(0) F(u(O), v(0)) To(O) To(0) (u()),t,c). By the multivariate chain rule, the derivative of i' w.r.t. is:
aF(0) aF
du
(S11)
00 dv 00 8u 00
Oy can be expressed as: The first term ao
0u 00
I,
(S12)
a0
00
and the second term
du can be expressed as: B0
du 0x4
(S13)
00 80
Hence, we can rewrite the equation as:
(F
aF
0x
(S14)
00
Dv
du
00
Furthermore, we substitute F(u, v) with Na (x(, L, c), and thus the relationship described in Eq. (S10)
holds:
0xp_1
(x),
Om (x,, L.c)
Oio 1 (,
dx,
(S15)
09
Ox
00
The proof is completed.
31

---

Published as a conference paper at ICLR 2026
C.2 CONVERGENCE ANALYSIS
We now provide a convergence guarantee for EasyTune. For clarity, we write its update rule in the generic stochastic-gradien form
0k+1 02 7,9k;
(S16)
where ds is the stochastic gradient computed from a minibatch of noisy motions at randomly sampled denoising step, following the EasyTune objective in Eq. (6). Let (()) denote the corresponding expected training objective.
We make the following assumptions on the EasyTume update:
(A1) (Lower bounded and smooth objective) C(0) is lower bounded by some Lint > and has L-Lipschitz continuous gradient (i.e., L-smooth), meaning (VL(0)-VC(()) = L| l'l for all 0, 0'.
(A2) (Bounded second moment of stochastic gradient) The stochastic gradient gk satisfies |1 0%] < G² for some constant G >> 0.
(A3) (Controlled bias from stop-gradient) The bias induced by the stop-gradient operation is uniformly bounded and proportional to the step size, i.c., || 0x] VL((&)) S b) for some constant b 2 0.
These assumptions are standard the analysis of non-convex stochastic gradient methods and, in our iiffusion-basee motion tuning setting. (A2) captures the bounded variance of the minibatch gradient obtained by sampling noisy motions and timesteps, while (A3) models the O(ni) bias introduced by the stop- noo-ggadieett design.
Theorem S1 (Convergence of EasyTune). Under the above conditions, the sequence {0x} generated by EasyTune satisfies the following properties:
E[[(0++1)] & [B[C(0x)] - ca me E [[V + 02 n/:
(S17)
Proof. The proof follows the standard template for non-convex stochastic gradient descent, adapted the EasyTune update.
By L-smoothness of C (Assumption (A1)), for any is we have
[(0x+1) S [(0x)) + VZ(02)) (0&+1 0x) + |10k+1 - 0x11
(S18)
Substituting as+1 0₁ 14-K gives
[(0++1) [(0x) T7s VL(O)) 9h + 1 m 11
(S19)
Taking conditional expectation given 0% and using the tower property of expectation, obtain
E[[((++11 0k] S C(O+) 7 VL(A)) Elgn 0xl + nr El 11? 0x).
(S20)
By Assumption (A2), 11]l9>1' || 0s] < G2, so
E L((())+)) 0&] v [(0%) Th VL(O2) Elgk 0&] 1/2 ni G2.
(S21)
Next we control the inner product term using Assumption (A3). Let m. Elyn 0s] and write
((Ax) m+ VC(B))"
(Ox))) (m+ vL(Ok))
(S22)
2 | -1V_))/)--1(__-0.a00000_ ||m - VL(Oz>I
(S23)
2 /V - b [V(e))l
(S24)
where we used Cauchy-Schward and (A3) in the last inequality. Hence
-Th VL(A)) my < mms [V + bng [V(())]]
(S25)
Applying Young's 's inequality 2ab S a² + 1/2 with a = /mtk || [ and b
3/2
we get
b VL()))] < n VL(0,)I 2b
(S26)
32

---

Published as a conference paper at ICLR 2026
Therefore
7h V.(())) my S m
- IV + 2²2 me.
(S27)
Combining the above bounds yields
B[(((x++1) 0z) S ((2x) c v [ + C1 n
(S28)
for some positive constants c1, C1 depending only on L, G. and (we absorb the Onl) term into the (n term). Taking full expectation over 0%. then gives
E[[(Ox+1)] S [[[((=)] cı m [IV + c2 m
(S29)
where we set C2 :- CI. This proves the claimed one-step descent inequality.
Building on this classical descent inequality and following standard non-convex SGD theory, we can derive a global convergence consequence for Easy
Corollary S1 (Asymptotic stationarity of EasyTunc). Suppose Assumptions (AI)-(A3) hold and that the step sizes satisfy di > Ek The = oo and =k= The < oc. Then the EasyTune iterates satisfy
K=D Th E [2(0=) 0,
K-1
lim inf
(S30)
K -+0
Ek-1 7
and in particular
lim im
(S31)
k-000
That is, EasyTune converges to first-order critical points in the standard non-convex sense.
Proof. From Theorem $1 have, for all k: 2
E[C(0++1)] - (0:) c1 [[I + c2 n Summing this inequality over 1: = O....,A K
(S32)
1 and using telescoping of the left-hand side gives
K K-1
K
[[C(Ok)] < [[C()))
c1 LE V2(00)
Σ
02
Σ
T
(S33)
Rearranging the previous inequality to move the gradient term to the left-hand side, and then applying that L is bounded below by Lint (Assumption (A1)) together with B[[(Ox) 2 Lint and the monotonicity IK-1 v < k= nh obtain
20
The E V2(0)) < B[C(0)) [[[(0))] + 02
Σ n < B[C(0))] Line + C2
Σ
k=
k=
(S34)
The right-hand side is finite by the assumptions on {m7 so letting
E[[(6u)]
Line
C2
Co
m < 00,
(S35)
C1
C1
deduce
m E [[((0)1 S Co.
(S36)
k=U
Dividing both sides by Ek- 7 and letting K + xo yields
K= Mh E L(I
K-1
Co
<
0,
(S37)
Ck-1m T
Σ
7
K-000
where used Σx 77 oo in the last step. This proves the weighted- statement. The and Ko such that E[|[V[(0+)]]]) 2 E for all 2 Ko, the weighted average would be bounded below by e, contradicting the previous limit. Hence lim inf E[] [[(+))]]
liminf statement then follows: if there existed
33

---

Published as a conference paper at ICLR 2026
This corollary is direct application of classical convergence theory for non-convex stochastic gradient methods; we include the standard argument above for completeness.
Discussion. Theorem S1 provides one-step descent inequality showing that each EasyTune Tune update decreases the expected training objective up to small quadratic term the step size. The corollary then instantiates the standard non-convex SGD theory in Liffusion-basedd motion tuning setting, proving that, under mild step-size conditions, the Easy Tune iterates converge to first-order critical points expectation. In other words, despite the stop-gradient design and step-aware sampling in Eq. (S9), Easy enjoys the same asymptotic convergence guarantees as classical stochastic gradient methods.
C.3 PROOF EQ. (5)
Proof. Given diffusion model eø, and reward model R.g. the diffusion model is fine-tuned by maximizing the differentiable reward value:
OC(0) - EewDr xf (-(-ee) 00
(x)
0x
(S38)
Ox6
00
where Ty denotes the reverse process defined in Eq. (2).
Here, we introduce Theorem to compute
0x
and thus we have:
Oc(0)
(x,)
0xp
00
0x0
00
OR (x()
(TT((p))
(TT(x²)
Oxq
-EWDD mml
Ox!
00
Ox9
00
OR
(iT((x))
@To(x!)
(TT(
Ox0
+
+
0x
00
Oxc
00
Oxc
0x!
30
oRo(x
OTO (x?)
amo (x,))
- Dr
Oxy
OxL
00
(S39)
The proof is completed.
34


Reviews:
[{"review_id": "x77YVtDlFf", "reviewer": "Reviewer_yMBz", "summary": "This paper presents EasyTune, a step-aware fine-tuning method for diffusion-based motion generation. The approach aims to improve the efficiency of differentiable reward optimization by decoupling recursive dependencies across denoising steps. A Self-refinement Preference Learning (SPL) mechanism is also introduced to train reward models without human annotations. Experimental results on standard benchmarks indicate performance and memory advantages over existing methods, suggesting that the proposed framework is effective and practically relevant.", "questions": "1. Could the authors provide more details on how the step-wise optimization interacts with the diffusion time schedule? For instance, does the reward weighting or learning rate vary across steps, and how sensitive is the method to this design?\n\n2. In Eq. (7), the stop-gradient operation ensures local optimization per step. Does this affect the global consistency of the final denoised result? Have the authors observed any degradation in smoothness or temporal coherence of the generated motions?\n\n3. Regarding the SPL mechanism, how often are new preference pairs mined during training? Is the mining process iterative (i.e., reward model updated → new pairs mined), or is it performed only once before fine-tuning?\n\n4. The proposed method is demonstrated on motion generation tasks. Do the authors believe EasyTune can generalize to other diffusion domains (e.g., image or video generation), or are there task-specific assumptions that limit its applicability?\n\nLooking forward to hearing from the authors for more explanation and discussion.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper clearly explains the main problem in existing diffusion fine-tuning methods and gives a straightforward way to make the optimization more efficient.\n\n2. The experiments are thorough and show steady improvements on several benchmarks and models.\n\n3. The method can be applied in practice since it does not rely on human-labeled data for reward learning.\n\n4. The writing and presentation are clear and easy to follow.\n\n5. The video results presented in the supplementary materials look decent.", "weaknesses": "### Major Concerns\n1. In the proposed Self-refinement Preference Learning (SPL), a pre-trained text-to-motion retrieval model is used for preference evaluation. It would be important to clarify how critical the choice of this model is. For example, if a weaker retrieval model were used, would the overall performance of EasyTune be significantly affected? Some discussion or analysis on this point would strengthen the argument, especially since the retrieval model is further fine-tuned.\n\n2. While the paper emphasizes the large reduction in GPU memory, the efficiency analysis could be expanded. It would be helpful to report additional metrics such as GFLOPs, inference/training time, and the GPU type used, to give a more complete picture of the claimed efficiency gains.\n\n3. The theoretical analysis, although helpful for understanding the recursive dependency issue, remains somewhat limited. A more detailed examination of convergence behavior or optimization stability under step-wise updates would improve the technical depth.\n\n4. The SPL module depends on automatically mined preference pairs, which could introduce noise or bias. Some quantitative analysis on the reliability or noise level of these pseudo pairs would make the approach more convincing.\n\n### Minor Concerns\n- In Line 78, the symbol “θ” appears too close to the “p” in “depended”; it would be better to slightly adjust the spacing for readability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qKR7PgpoRT", "reviewer": "Reviewer_v4NY", "summary": "The paper addresses the challenge of efficiently fine-tuning diffusion-based motion generation models using differentiable rewards, which suffer from coarse-grained optimization and extremely high memory usage due to recursive gradient dependencies across denoising steps. The paper propose EasyTune, a step-aware fine-tuning framework that optimizes the diffusion model at each denoising step independently instead of over the full trajectory. In EasyTune, the model computes the reward of the current noised motion, applies a stop-gradient operation to decouple dependencies, updates parameters locally, and clears the computation graph before moving to the next step—achieving fine-grained supervision and constant memory cost. To overcome the lack of human preference data, the paper further introduces Self-refinement Preference Learning (SPL), which adapts a pre-trained text-motion retrieval model into a reward model by automatically mining motion preference pairs from retrieval errors and fine-tuning them using a KL-divergence objective. Experiments on HumanML3D and KIT-ML with six pre-trained motion diffusion models show that EasyTune significantly outperforms prior fine-tuning methods such as DRaFT and DRTune.", "questions": "Can the proposed method be applied to other domains such as image generation?", "limitations": "", "rating": 10, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "- The paper addresses an important and timely problem in diffusion model fine-tuning, providing a thorough analysis of the limitations in existing differentiable reward methods.\n- The proposed EasyTune framework is conceptually clear and mathematically well-formulated, offering a principled solution to recursive gradient and memory inefficiency issues.\n- The authors conduct extensive experiments and comparisons across multiple datasets and backbone models, demonstrating consistent and substantial performance gains with strong quantitative and qualitative evidence.", "weaknesses": "The paper’s presentation could be improved — some sections are densely formatted and may benefit from clearer visual structure (e.g., spacing, figure placement, and paragraph organization) to enhance overall readability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gdmBFJp3KK", "reviewer": "Reviewer_pyGX", "summary": "# Summary\nThe paper aims at aligning motion diffusion models with downstream objectives using differentiable rewards. The authors first diagnose a root cause behind prior methods’ inefficiency and memory use: recursive gradient dependence across denoising steps. They propose EasyTune, which fine‑tunes at each denoising step by stopping gradients through the current sample and backpropagating only through the local denoiser. This decouples steps, enabling dense, fine‑grained updates with O(1) step memory. To address the scarcity of motion preference data, they introduce Self‑refinement Preference Learning (SPL) that adapts a retrieval model into a reward model without human‑labeled pairs. \nExperiments on HumanML3D and KIT‑ML across multiple backbones (MLD/MLD++, MotionLCM, MDM, MotionDiffuse) report consistent improvements and lower peak memory than trajectory‑level fine‑tuning baselines.", "questions": "Besides the questions in the above section: \n\nThe objective uses uniform \\(t \\sim U(0, T)\\), yet the paper argues early, high‑noise steps are critical and under‑optimized by chain methods. Would non‑uniform sampling or up‑weighting early steps help?\n\nIn SPL, the authors set K=10. How do different K and different retrieval pools (subset vs. full) impact reward quality and final generation?\n\nAny failure cases (I mean figures/videos)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper cleanly shows how chain‑rule recursion creates sparse/vanishing gradients and large memory graphs and contrasts chain vs. step optimization. The motivation is clear and reasonable.\n- The proposed method seems to be a simple, general, and effective idea: The per‑step objective with `sg(·)` is easy to implement on standard motion diffusers.\n- Results span two datasets and several backbones; on HumanML3D, EasyTune improves R‑Precision/FID/MM‑Dist vs. DRaFT/AlignProp/DRTune while using less peak memory than those baselines. EasyTune also achieves faster convergence curves for optimization.\n- The authors promised to release the code and models.\n- Most details are provided in the paper and appendix. Video result (website) is well-made.", "weaknesses": "I am a bit worried about the technical novelty (but it seems ok: The fine-tuning input x_t can be obtained by denoising sampling - seems new). But Theorem 1 is a direct chain‑rule decomposition (can it be called a Theorem?); Theorem 2 is the local derivative after inserting `sg(·)`. Both are too straightforward to be called Theorem...  Besides, SPL improvements are modest... On HumanML3D retrieval, SPL improves ReAlign by R@1 +2.5% / R@3 +1.4%; on KIT‑ML, R@5 +2.2%. In the Limitation section, the authors note SPL’s reliance on retrieval mining may introduce noisy/ambiguous pairs.\n\nMinor:\n- The Implementation section states LR=2×10⁻⁴, batch=128, whereas Tab.S1 shows LR=1×10⁻⁵, batch=256 for all methods including EasyTune?\n- Please also end‑to‑end costs, EasyTune’s total compute/time…Fig.5’s “w/o BP” curves illustrate the theoretical O(1) vs. O(T) difference without backprop; training, however, always with backpropagates.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7qHcqwVqp3", "reviewer": "Reviewer_UAQn", "summary": "The paper proposes EasyTune, a step-aware fine-tuning scheme for diffusion-based motion generation that optimizes at each denoising step using stop-gradient to break recursive dependencies; this yields fine-grained updates and O(1) memory rather than O(T). It trains a reward model without human labels via Self-refinement Preference Learning (SPL), mining preference pairs from retrieval results and fine-tuning a retrieval backbone as a differentiable reward.", "questions": "- Are the metrics giving a distorted signal? R-Precision exceeds the “Real” upper bound (0.581 > 0.511), yet FID is still notably higher than Real (0.132 vs 0.002). How do you interpret this divergence, and does it indicate semantic alignment without true distributional realism?\n- Why must rewards be differentiable here? In RLHF for diffusion, rewards need not be differentiable. For motion, why is a differentiable reward essential—especially when non-differentiable motion metrics (e.g., foot-skate or slip measures) could be used as rewards? \n- What exactly is being aligned? In EasyTune + SPL, is the alignment primarily to text–motion retrieval similarity, to human preference proxies, or to motion-quality attributes (e.g., fluency/physicality)? Please specify the concrete targets the gradients optimize toward.\n- Missing citations ( PhysDiff, ReinDiffuse, Aligning Human Motion Generation with Human Perceptions ).\n- I'm glad to raise the score if the author can address my concerns.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- Clear motivation and theory; step-wise objective avoids vanishing gradients and heavy graph storage.\n- Strong empirical results on alignment metrics (R-Precision/MM-Dist) with reduced memory footprint.\n- Practical reward learning via SPL removes dependence on expensive human preference data.", "weaknesses": "- Limited discussion on when to use noise-aware vs. one-step (ODE) rewards and their sensitivity.\n- Reward model is retrieval-based, which may under-capture physics/realism and encourage metric chasing. (Authors note limited physical grounding.)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)