Title: PID-controlled Langevin Dynamics for Faster Sampling of Generative Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Langevin Dynamics

Generative Models

PID controller
Abstract: Langevin dynamics sampling suffers from extremely low generation speed, fundamentally limited by numerous fine-grained iterations to converge to the target distribution. We introduce PID-controlled Langevin Dynamics (PIDLD), a novel sampling acceleration algorithm that reinterprets the sampling process using control-theoretic principles. By treating energy gradients as feedback signals, PIDLD combines historical gradients (the integral term) and gradient trends (the derivative term) to efficiently traverse energy landscapes and adaptively stabilize, thereby significantly reducing the number of iterations required to produce high-quality samples. Our approach requires no additional training, datasets, or prior information, making it immediately integrable with any Langevin-based method. Extensive experiments across image generation and reasoning tasks demonstrate that PIDLD achieves higher quality with fewer steps, making Langevin-based generative models more practical for efficiency-critical applications. The implementation can be found at \href{https://github.com/tsinghua-fib-lab/PIDLD}{https://github.com/tsinghua-fib-lab/PIDLD}.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method for adaptive reward-guided sampling in diffusion models by modulating the guidance strength during Langevin sampling. The authors frame controllable generation as a feedback control problem, where the error between a current reward and a desired target is used to dynamically adjust the scale of reward gradients injected during sampling. The method is implemented as a training-free wrapper and is tested on several text-to-image generation tasks using Stable Diffusion. Empirical results show improved alignment with target rewards and sample quality compared to fixed or heuristic guidance strength baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1 Clear motivation: The paper identifies a common practical issue in reward-guided diffusion sampling — the need to manually tune or schedule the guidance strength α — and addresses it in a principled way.\n\n2 Conceptual novelty: Framing guidance as a control problem and applying a PID controller to reward error is novel in this context. To the best of my knowledge, this is the first use of classical control theory for online adaptation of guidance strength in diffusion models.\n\n3 Training-free and modular: The method can be applied without retraining the base diffusion model or the reward model. It is compatible with any differentiable reward and can be used as a drop-in wrapper around existing samplers.\n\nWeaknesses\n\n1 Lack of ablation on PID components: The benefit of using the full PID structure (Proportional + Integral + Derivative) over a simple proportional or heuristic controller is not clearly demonstrated. An ablation comparing PID vs. just P (or vs. α ∝ gradient norm) would help isolate the contribution of each term and justify the additional complexity.\n\n2 Heuristic alternative baselines are missing: While the paper compares against fixed α and some reward-guided variants, it does not benchmark against simple time-based or norm-based scaling heuristics (e.g., α increasing over time, or α ∝ 1 / ∥∇ℓ(x)∥). These baselines are common in the literature (e.g., FreeDoM, classifier guidance) and would provide a better sense of the gain from PID.\n\n3 Limited analysis of control behavior: The paper does not deeply analyze the PID controller's behavior over time — e.g., plots of α_t, error evolution, or control stability across prompts. This would strengthen the claim that the controller leads to smoother and more stable guidance.\n\n4 Conceptual novelty may be overstated: While the framing of PID-controlled sampling is novel, the underlying mechanism — adjusting guidance strength based on feedback — is not entirely new. Related ideas exist in norm-based scaling, adaptive weighting, and heuristic scheduling. The paper would benefit from more explicit comparison and positioning relative to these.\n\nQUESTIONS:\nHave you evaluated the relative contribution of the I and D terms in the PID controller? Would a proportional-only controller suffice?\n\nHow sensitive is the method to the choice of PID gains (k_p, k_i, k_d)? Are they robust across prompts and reward types?\n\nHave you considered combining reward error-based scaling with gradient-norm normalization for better numerical stability?\n\nLIMITATIONS:\nThe paper does not extensively discuss potential failure cases or unintended behavior (e.g., reward hacking, overshooting, or instability for ill-behaved reward functions). The PID controller is sensitive to hyperparameters (k_p, k_i, k_d), and though fixed values are used throughout, no systematic analysis of robustness is provided. Including such analysis would help assess broader applicability.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 1, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper tackles the challenge of accelerating Langevin sampling by reframing the problem as one of feedback control. Concretely, the authors observe that in most sampling schemes the instantaneous gradient of the log-density at the current state acts as a natural “error” signal: driving that gradient to zero corresponds to reaching high-probability regions. Building on this insight, they ask whether classical control-theoretic tools might be employed to attenuate this gradient error as quickly as possible.\n\nTheir key contribution is to augment the usual sampling dynamics with a PID controller, treating the gradient as the error variable and tuning proportional, integral, and derivative gains to shape the trajectory. Importantly, this approach does not need any re-training. The authors then compared sampling with a pre-trained model when tuning the different PID terms, and discussed the advantages of the approach.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The paper’s principal contribution is to recast Langevin sampling as a feedback‐control problem, treating the instantaneous gradient of the log-density as an “error” signal to be driven to zero. This novel framing is likely to inspire further cross‐pollination between control theory and stochastic sampling methods.\n2. By augmenting standard sampling dynamics with a PID controller, the authors demonstrate measurable speedups in convergence and improvements in sample quality across their test problems. These numerical results substantiate the high‐level intuition that classical control tools can benefit generative sampling.\n3. The use of a well-understood controller (PID) makes the method easy to implement and tune, and potentially compatible with a wide variety of existing samplers without algorithmic redesign or need for re-training. It is interesting because it is an instance of post-training hyper-parameter tuning.\n\nWeaknesses:\n\n1. Limited exploitation of the insights from control theory. From the experiments, it is not clear to me how the control-theoretic insights about proportional, integral, and derivative terms translate to langevin sampling (see also my questions, and the outlined limitations). For instance: Does the derivative term actually attenuate stochastic oscillations? Does the integral term actually correct for steady-state bias? Or is the control-theoretic motivation only useful to inspire another hyper-parameter to tune? The idea of using PID to drive optimization procedure is not novel, see e.g. https://proceedings.mlr.press/v139/farahmand21a.html. A discussion about these may be needed.\n2. The introduction of a decay factor on the integral term appears ad hoc. From a control standpoint this resembles a leak in an I-controller, but without the connection to control theory it seems a bit out of scope.\n3. The reported FID scores lag behind state-of-the-art generative models. See e.g. https://arxiv.org/abs/2504.10612 and references therein. It’s unclear whether this is due to underlying base model. The authors should clarify their baseline and test whether PID control yields comparable gains when used on top of a SOTA sampler.\n\nQUESTIONS:\n1. In control theory, the integral component is introduced to eliminate steady-state error in the presence of constant disturbances. By analogy, when sampling with noisy gradient estimates (e.g. minibatched or corrupted gradients), the I‐term should compensate for bias and drive the long-term average gradient toward zero. Can the authors perhaps design a simple experiment to validate if these insights from control theory translate in this setting and may be useful? Perhaps by eliminating an estimation bias in the gradient (e.g., due to training). Comparing the different sampling scheme would help elucidating these connections.\n2. The D‐component in PID is classically used to add damping and counteract overshoot or oscillatory behavior around the setpoint (you show this in an example in Figure 1, but I believe the paper could have been stronger by going back to this example in the context of Langevin sampling with experiments). To test this in sampling, one could intentionally mis-tune the integral/proportional gain (e.g. choose a P-gain large enough to induce oscillatory sampling trajectories in a simple 1D double-well potential). Then compare how P-only versus PD controllers traverse between modes, measuring quantities like autocorrelation time or the amplitude of oscillations in $x_t$. The hypothesis is that the D-term will attenuate high-frequency jitter—improving both mixing speed and robustness to P-gain mis-specification. I believe that this type of analyses would greatly improve the contribution of the paper, in explaining and motivating the connections between control theory and Langevin sampling. If these or other insights do not translate to the higher dimensional setup of sampling for images, it is also interesting.\n3. Can you run experiments with stronger baseline models? This would allow to understand if the gains of using your scheme are only due to a poor baseline or if they extend to stronger models as well.\n\nLIMITATIONS:\nThe paper does not address properly its limitations. These are deferred to the appendix and mixed with future work, but reading that paragraph I only appreciate directions for future work but I do not understand the limitations of the method. For instance, I instead identify the following ones by reading the paper:\n\n1. I believe the idea and insight holds great potential, but the authors did not capitalize on the control theory interpretation, or at least I was not able to grasp, reading the paper, how the insights from control actually translate to Langevin sampling. Of course they inspired adding the derivative term to momentum-based schemes, but given the premise of the paper I believe it should go more in depth into the control-theoretical interpretations of the proposed scheme. Similarly, when discussing the I-term I would have expected more emphasis on the control-theoretic interpretation (also mentioned in Figure 1) of constant disturbance rejection, rather than the momentum interpretation (less novel).\n2. Can you run your experiments on top of a state-of-the-art sampler? It looks like your FID scores are much higher than the state-of-the-art. It would be interesting to understand if the gains of your method depend on a week baseline of if they extend (and how) to stronger baselines.\n\nThe authros might want to discuss limitations more thoroughly in the main paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose to augment standard Langevin dynamics (LD), in the context of sampling from diffusion and energy-based models (EBMs), with PID control.  That is, they interpret the discrete-time approximation to LD as a dynamical system that is driven by the energy gradient (i.e. the \"score\" or force), interpreted as an error signal that needs to be driven toward zero.  Accordingly, they replace it with the classic PID scheme, i.e. a linear combination of the (discrete) derivative and integral of the error, as well as the error itself.  They show that this scheme (slightly modified to decay the integral term) outperforms standard Langevin dynamics in FID on CIFAR10 and CelebA at various choices for the total number of neural function evaluations (NFEs), both in score-based model and in EBMs.  Similar results hold for sampling-based reasoning in EBMs.  The authors also compare against a similar, recent proposal (MILD) from another group.\n\nSTRENGTHS AND WEAKNESSES:\nThe idea appears to be novel (this reviewer is surprised that it has never been tried before), the basic experiments appear sound, and the topic---reducing the required number of samples for LD---is important.  There are also weaknesses:\n\n1. The evaluations are on simple datasets (CIFAR-10, CelebA); it would be nice to see results on (say) ImageNet.  I am also not sure they are totally fair: I believe NCSN typically uses far (about an order of magnitude) more Langevin steps (T) than noise levels (L); whereas in the experiments reported in Table 1, the reverse is the case.  A fairer comparison would seem to be to fix a budget of NFEs, and then allocate them to L and T in the way that yields best performance for the sampling method under consideration.\n\n2. Although it's promising to see that this method can produce faster samples compared to standard Langevin dynamics, there has been significant progress in accelerating sampling for diffusion models using techniques such as deterministic sampling (e.g., DDIM) and ODE-based samplers.  These methods can generate high-quality samples in just a few steps. It would be interesting to see how the proposed approach compares to these alternatives.\n\n3. Along these lines: The manuscript explores the proposed technique in the context of sampling from a pre-trained EBM.  A more compelling application would be to evaluate how well the sampler performs when used for *training* EBMs. This would also provide better insight into the stability and robustness of the hyperparameters.\n\n4. The authors argue that the proposed sampler improves sampling from multi-modal distributions by traversing energy barriers.  To test whether the sampler actually mixes, a good experiment would be to run it beyond 25 steps and observe how the samples evolve: does each sample stay fixed at a particular image, or does it explore the manifold?  If the sampler does mix effectively, one would expect results similar to Figure 3 of the following paper: https://arxiv.org/pdf/2006.06897\n\n5. Is there any reason MILD was omitted from Table 2?  Here I would expect MILD to be more competitive with the PID controller, since apparently (Fig. 6) the integral term is doing most of the work here, and MILD is in a sense a pure integral-controlled system (i.e., the input is low-pass filtered).\n\nMinor:\n1. The authors cited a paper on biomolecular PID controllers when introducing the concept--was this intentional?\n\n2. Presentation: The MS is somewhat repetitive, with the basic rationale for PID controls repeated about once per page for the first four pages.  Also, there are some distracting grammatical mistakes: e.g, I think the title should read, \"...sampling *or* [or *from*] generative models\"; \"related works\" -> \"related work\" (\"work\" is a mass noun and shouldn't be pluralized); etc.\n\nQUESTIONS:\nCan the authors addresses the five major weaknesses listed above?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the problem of improving inference-time sampling from energy-based models (EBMs) and score-based generative models (SGMs) using Langevin Dynamics. To address this problem, the authors propose a novel control-based method, Proportional-Integral-Derivative controlled Langevin Dynamics (PIDLD). Their proposed approach uses the energy gradient $-U_\\theta(x)$, standard in Langevin Dynamics, as a feedback signal for the PID controller. Through empirical experiments on a variety of image generation tasks and reasoning tasks, the authors show the effectiveness of their proposed method relative to vanilla sampling approaches with Langevin Dynamics.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n- This work presents an interesting and novel idea of using a Proportional-Integral-Derivative (PID) controller for sampling with Langevin dynamics to improve inference-time sampling from energy-based models and score-based generative models. \n- The proposed method, PIDLD, is evaluated across several empirical experiments, on both image generation tasks and reasoning tasks.\n- The paper is generally well written, and relevant concepts are clearly explained.\n\n**Weaknesses:**\n\n- The central claim that PIDLD improves sampling speed is not sufficiently supported. Specifically, on lines 61-62, the authors state, \"PIDLD achieves at least 10x sampling speedup over baselines under SGM model\". However, no experiment explicitly reports these results/numbers. I see the authors report NFEs; however, I don't believe this is sufficient to support this claim. To support this claim, I recommend including an experiment that reports evaluation time throughout the sampling process, alongside a performance metric. This could be done for both image generation tasks as well as the reasoning tasks.  \n- For image generation tasks, only one metric (FID) is considered. This provides only a narrow view of the performance/comparison of PIDLD. Other relevant metrics to consider are Feature likelihood divergence (FLD) (Jiralerspong et al, 2023) and Inception Score. \n- Some relevant works in the realm of controllable generation/sampling, which I believe are related to PIDLD, are missed in the discussion of related works. For example, see (Domingo-Enrich et al, 2024, Havens et al, 2025).\n- With this, although to my understanding the idea of using a PID controller to improve sampling of Langevin dynamics, the notion of \"controllable\" sampling with Langevin dynamics is not new. I encourage the authors to include a more comprehensive and explicit related works section discussing the state of this area of research and literature.\n\nQUESTIONS:\n- How does this work relate to the recent works (references listed above) in controllable generation? \n- Proposition 1 assumes that $-U_\\theta(x)$ is \"_m-strongly convex_\". Is this a reasonable assumption given, to my understanding, most sampling applications of interest deal with highly non-convex $-U_\\theta(x)$?\n- An active area of research in sampling is application to molecules and proteins. How do you envision the performance of PIDLD on these domain-specific applications? It would be interesting to see an experiment in one of these areas (e.g. Tan et al, 2025). I also believe an additional experiment would strengthen the contributions of this work. \n\nReferences:\n\n1. Jiralerspong, et al. \"Feature likelihood divergence: evaluating the generalization of generative models using samples.\" NeurIPS (2023).\n2. Domingo-Enrich, et al. \"Adjoint matching: Fine-tuning flow and diffusion generative models with memoryless stochastic optimal control.\" (2024).\n3. Havens, et al. \"Adjoint sampling: Highly scalable diffusion samplers via adjoint matching.\" (2025).\n4. Tan, et al. \"Scalable equilibrium sampling with sequential boltzmann generators.\" (2025).\n\nLIMITATIONS:\nYes, but they are discussed at the end of the appendix. I recommend that the authors move this to the end of the main text (or to the conclusion).", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}