GenAI4ELab/papercli-papers huggingface dataset:
Improving Regret Approximation for Unsupervised Dynamic Environment Generation
Authors: Harry Mead; Bruno Lacerda; Jakob Nicolaus Foerster; Nick Hawes
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Unsupervised Environment Design (UED) seeks to automatically generate training curricula for reinforcement learning (RL) agents, with the goal of improving generalisation and zero-shot performance. However, designing effective curricula remains a difficult problem, particularly in settings where small subsets of environment parameterisations result in significant increases in the complexity of the required policy. Current methods struggle with a difficult credit assignment problem and rely on regret approximations that fail to identify challenging levels, both of which are compounded as the size of the environment grows. We propose Dynamic Environment Generation for UED (DEGen) to enable a denser level generator reward signal, reducing the difficulty of credit assignment and allowing for UED to scale to larger environment sizes. We also introduce a new regret approximation, Maximised Negative Advantage (MNA), as a significantly improved metric to optimise for, that better identifies more challenging levels. We show empirically that MNA outperforms current regret approximations and when combined with DEGen, consistently outperforms existing methods, especially as the size of the environment grows. We have made all our code available here: \url{https://github.com/HarryMJMead/Dynamic-Environment-Generation-for-UED}.
LaRes: Evolutionary Reinforcement Learning with LLM-based Adaptive Reward Search
Authors: Pengyi Li; Hongyao Tang; Jinbin Qiao; YAN ZHENG; Jianye HAO
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: The integration of evolutionary algorithms (EAs) with reinforcement learning (RL) has shown superior performance compared to standalone methods. However, previous research focuses on exploration in policy parameter space, while overlooking the reward function search.
To bridge this gap, we propose **LaRes**, a novel hybrid framework that achieves efficient policy learning through reward function search. LaRes leverages large language models (LLMs) to generate the reward function population, guiding RL in policy learning. The reward functions are evaluated by the policy performance and improved through LLMs.
To improve sample efficiency, LaRes employs a shared experience buffer that collects experiences from all policies, with each experience containing rewards from all reward functions.
Upon reward function updates, the rewards of experiences are relabeled, enabling efficient use of historical data. 
Furthermore, we introduce a Thompson sampling-based selection mechanism that enables more efficient elite interaction.
To prevent policy collapse when improving reward functions, we propose the reward scaling and parameter constraint mechanisms to efficiently coordinate reward search with policy learning.
Across both initialized and non-initialized settings, LaRes  consistently achieves state-of-the-art performance, outperforming strong baselines in both sample efficiency and final performance.
The code is available at https://github.com/yeshenpy/LaRes.
Uncovering a Universal Abstract Algorithm for Modular Addition in Neural Networks
Authors: Gavin McCracken; Gabriela Moisescu-Pareja; Vincent Létourneau; Doina Precup; Jonathan Love
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: We propose a testable universality hypothesis, asserting that seemingly disparate neural network solutions observed in the simple task of modular addition actually reflect a common abstract algorithm. While prior work interpreted variations in neuron-level representations as evidence for distinct algorithms, we demonstrate---through multi-level analyses spanning neurons, neuron clusters, and entire networks---that multilayer perceptrons and transformers universally implement the abstract algorithm we call the approximate Chinese Remainder Theorem. Crucially, we introduce approximate cosets and show that neurons activate exclusively on them. Furthermore, our theory works for deep neural networks (DNNs). It predicts that universally learned solutions in DNNs with trainable embeddings or more than one hidden layer require only $\mathcal{O}(\log n)$ features, a result we empirically confirm. This work thus provides the first theory‑backed interpretation of \textit{multilayer} networks solving modular addition. It advances generalizable interpretability and opens a testable universality hypothesis for group multiplication beyond modular addition.
Inference-time Alignment in Continuous Space
Authors: Yige Yuan; Teng Xiao; Li Yunfan; Bingbing Xu; Shuchang Tao; Yunqi Qiu; Huawei Shen; Xueqi Cheng
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Aligning large language models with human feedback at inference time has received increasing attention due to its flexibility. Existing methods rely on generating multiple responses from the base policy for search using a reward model, which can be considered as searching in a discrete response space. However, these methods struggle to explore informative candidates when the base policy is weak or the candidate set is small, resulting in limited effectiveness. In this paper, to address this problem, we propose Simple Energy Adaptation ($\textbf{SEA}$), a simple yet effective algorithm for inference-time alignment. In contrast to expensive search over the discrete space, SEA directly adapts original responses from the base policy toward the optimal one via gradient-based sampling in continuous latent space. Specifically, SEA formulates inference as an iterative optimization procedure on an energy function over actions in the continuous space defined by the optimal policy, enabling simple and effective alignment. For instance, despite its simplicity, SEA outperforms the second-best baseline with a relative improvement of up to $ \textbf{77.51\%}$ on AdvBench and $\textbf{16.36\%}$ on MATH. Code is publicly available at [this link](https://github.com/yuanyige/sea).
DexFlyWheel: A Scalable and Self-improving Data Generation Framework for Dexterous Manipulation
Authors: Kefei Zhu; Fengshuo Bai; YuanHao Xiang; Yishuai Cai; Xinglin Chen; Ruochong Li; Xingtao Wang; Hao Dong; Yaodong Yang; Xiaopeng Fan; Yuanpei Chen
Venue: NeurIPS 2025 (NeurIPS 2025 spotlight)

Abstract: Dexterous manipulation is critical for advancing robot capabilities in real-world applications, yet diverse and high-quality datasets remain scarce. Existing data collection methods either rely on human teleoperation or require significant human engineering, or generate data with limited diversity, which restricts their scalability and generalization. In this paper, we introduce DexFlyWheel, a scalable data generation framework that employs a self-improving cycle to continuously enrich data diversity. Starting from efficient seed demonstrations warmup, DexFlyWheel expands the dataset through iterative cycles. Each cycle follows a closed-loop pipeline that integrates Imitation Learning (IL), residual Reinforcement Learning (RL), rollout trajectory collection, and data augmentation. Specifically, IL extracts human-like behaviors from demonstrations, and residual RL enhances policy generalization. The learned policy is then used to generate trajectories in simulation, which are further augmented across diverse environments and spatial configurations before being fed back into the next cycle. Over successive iterations, a self-improving data flywheel effect emerges, producing datasets that cover diverse scenarios and thereby scaling policy performance. Experimental results demonstrate that DexFlyWheel generates over 2,000 diverse demonstrations across four challenging tasks. Policies trained on our dataset achieve an average success rate of 81.9\% on the challenge test sets and successfully transfer to the real world through digital twin, achieving a 78.3\% success rate on dual-arm lift tasks.
Towards Irreversible Attack: Fooling Scene Text Recognition via Multi-Population Coevolution Search
Authors: Jingyu Li; Pengwen Dai; Mingqing Zhu; Chengwei Wang; Haolong Liu; Xiaochun Cao
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Recent work has shown that scene text recognition (STR) models are vulnerable to adversarial examples.
Different from non-sequential vision tasks, the output sequence of STR models contains rich information. 
However, existing adversarial attacks against STR models can only lead to a few incorrect characters in the predicted text.
These attack results still carry partial information about the original prediction and could be easily corrected by an external dictionary or a language model.
Therefore, we propose the Multi-Population Coevolution Search (MPCS) method to attack each character in the image. 
We first decompose the global optimization objective into sub-objectives to solve the attack pixel concentration problem existing in previous attack methods.
While this distributed optimization paradigm brings a new joint perturbation shift problem, we propose a novel coevolution energy function to solve it.
Experiments on recent STR models show the superiority of our method.
The code is available at \url{https://github.com/Lee-Jingyu/MPCS}.
An Effective Levelling Paradigm for Unlabeled Scenarios
Authors: Fangming Cui; Zhou Yu; Di Yang; Yuqiang Ren; Liang Xiao; Xinmei Tian
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Advancements in direct-integration fine-tuning frameworks have underscored their potential to enhance the performance of labeled scenarios and tasks. To enhance the generalization of different categories in the same dataset, some methods have added visual loss to these frameworks for unlabeled scenarios. However, the performance of these methods through visual loss does not improve significantly in domain generalization and cross-dataset generalization tasks. This may be attributed to the uncoordinated learning of the two-modalities alignment and visual loss. To mitigate this issue of uncoordinated learning, we propose a novel method called Levelling Paradigm (LePa) to improve performance for unlabeled tasks or scenarios. The proposed LePa, designed as a plug-in module, dynamically constrains and coordinates multiple objective functions, thereby improving the generalization of these baseline methods. Comprehensive experiments have shown that our design can effectively address generalized scenarios and tasks.
Filter Like You Test: Data-Driven Data Filtering for CLIP Pretraining
Authors: Mikey Shechter; Yair Carmon
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: We introduce Filter Like You Test (FLYT), an algorithm for curating large-scale vision-language datasets that *learns* the usefulness of each data point as a pretraining example. FLYT trains a scoring model that learns to weigh each example's features using gradient signals from downstream tasks training sets. Based on FLYT, we implement Mixing-FLYT (M-FLYT), which takes the per-example scores generated by different scoring methods as features, and learns to unify them into a single score. FLYT naturally produces a distribution over the training examples, which we leverage through Soft Cap Sampling (SCS), a strategy for obtaining a filtered pretraining dataset from per-example probabilities that samples examples while preventing over-representation through a repetition penalty. Using these methods, we achieve 40.1\% ImageNet zero-shot accuracy on the DataComp medium scale filtering benchmark, a 2\% absolute accuracy increase over all previous results and a 5.5\% increase over results that---like us---use only public resources. Our approach also yields 37.7\% on the average of 38 DataComp evaluation tasks, outperforming previous public-resource approaches by 0.4\%.
A Gradient Guidance Perspective on Stepwise Preference Optimization for Diffusion Models
Authors: Joshua Tian Jin Tee; Hee Suk Yoon; Abu Hanif Muhammad Syarubany; Eunseop Yoon; Chang D. Yoo
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Direct Preference Optimization (DPO) is a key framework for aligning text-to-image models with human preferences, extended by Stepwise Preference Optimization (SPO) to leverage intermediate steps for preference learning, generating more aesthetically pleasing images with significantly less computational cost. While effective, SPO's underlying mechanisms remain underexplored. In light of this, we critically re-examine SPO by formalizing its mechanism as gradient guidance. This new lens shows that SPO uses biased temporal weighting, giving too little weight to later generative steps, and unlike likelihood centric views it reveals substantial noise in the gradient estimates. Leveraging these insights, our GradSPO algorithm introduces a simplified loss and a targeted, variance-informed noise reduction strategy, enhancing training stability. Evaluations on SD 1.5 and SDXL show GradSPO substantially outperforms leading baselines in human preference, yielding images with markedly improved aesthetics and semantic faithfulness, leading to more robust alignment. Code and models are available at https://github.com/JoshuaTTJ/GradSPO.
COLA: Towards Efficient Multi-Objective Reinforcement Learning with Conflict Objective Regularization in Latent Space
Authors: Pengyi Li; Hongyao Tang; Yifu Yuan; Jianye HAO; Zibin Dong; YAN ZHENG
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Many real-world control problems require continual policy adjustments to balance multiple objectives, which requires the acquisition of high-quality policies to cover diverse preferences. Multi-Objective Reinforcement Learning (MORL) provides a general framework to solve such problems. However, current MORL methods suffer from high sample complexity, primarily due to the neglect of efficient knowledge sharing and conflicts in optimization with different preferences.
To this end, this paper introduces a novel framework, Conflict
Objective Regularization in Latent Space (**COLA**).
To enable efficient knowledge sharing, COLA establishes a shared latent representation space for common knowledge, which can avoid redundant learning under different preferences. Besides, COLA introduces a regularization term for the value function to mitigate the negative effects of conflicting preferences on the value function approximation, thereby improving the accuracy of value estimation. The experimental results across  various multi-objective continuous control tasks demonstrate the significant superiority of COLA over the state-of-the-art MORL baselines. Code is available at https://github.com/yeshenpy/COLA.
Learning Robust Vision-Language Models from Natural Latent Spaces
Authors: Zhangyun Wang; Ni Ding; Aniket Mahanti
Venue: NeurIPS 2025 (NeurIPS 2025 spotlight)

Abstract: Pre-trained vision-language models (VLMs) exhibit significant vulnerability to imperceptible adversarial perturbations. Current advanced defense strategies typically employ adversarial prompt tuning to improve the adversarial robustness of VLMs, which struggle to simultaneously maintain generalization across both natural and adversarial examples under different benchmarks and downstream tasks. We propose a collaborative adversarial prompt tuning (CoAPT) approach from pre-trained VLMs to target robust VLMs. Inspired by the image mask modeling, we adopt an improved real-time total variation algorithm to suppress and eliminate high-frequency details from images while preserving edge structures, thereby disrupting the adversarial perturbation space. Subsequently, guided by the high-level image and text representations in the latent space of the pre-trained VLMs, the corrupted natural features are restored while inheriting the superior generalization capability. Experiments on four benchmarks demonstrate that CoAPT achieves an excellent trade-off among natural generalization, adversarial robustness, and task-specific adaptation compared to state-of-the-art methods.
Accelerating Diffusion LLMs via Adaptive Parallel Decoding
Authors: Daniel Mingyi Israel; Guy Van den Broeck; Aditya Grover
Venue: NeurIPS 2025 (NeurIPS 2025 spotlight)

Abstract: The generation speed of LLMs are bottlenecked by autoregressive decoding, where tokens are predicted sequentially one by one. Alternatively, diffusion large language models (dLLMs) theoretically allow for parallel token generation, but in practice struggle to achieve the speed of autoregressive models without significantly sacrificing quality. We therefore introduce adaptive parallel decoding (APD), a novel method that dynamically adjusts the number of tokens sampled in parallel. We achieve this by defining a multiplicative mixture between the dLLM marginal probabilities and the joint probability of sequences under a small auxiliary autoregressive model. This inverts the standard setup of speculative decoding, where the goal is to sample from a large autoregressive verifier by drafting from a smaller model. We further optimize APD by enabling KV caching and limiting the size of the masked input. Altogether, our method puts forward three tunable parameters to flexibly tradeoff throughput and quality. We show that APD provides markedly higher throughput with minimal quality degradations on downstream benchmarks.
CORE: Collaborative Optimization with Reinforcement Learning and Evolutionary Algorithm for Floorplanning
Authors: Pengyi Li; Shixiong Kai; Jianye HAO; Ruizhe Zhong; Hongyao Tang; Zhentao Tang; Mingxuan Yuan; Junchi Yan
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Floorplanning is the initial step in the physical design process of Electronic Design Automation (EDA), directly influencing subsequent placement, routing, and final power of the chip.
However, the solution space in floorplanning is vast, and current algorithms often struggle to explore it sufficiently, making them prone to getting trapped in local optima. To achieve efficient floorplanning, we propose **CORE**, a general and effective solution optimization framework that synergizes Evolutionary Algorithms (EAs) and Reinforcement Learning (RL) for high-quality layout search and optimization.
Specifically, we propose the Clustering-based Diversified Evolutionary Search that directly perturbs layouts and evolves them based on novelty and performance. Additionally, we model the floorplanning problem as a sequential decision problem with B*-Tree representation and employ RL for efficient learning. 
To efficiently coordinate EAs and RL, we propose the reinforcement-driven mechanism and evolution-guided mechanism. 
The former accelerates population evolution through RL, while the latter guides RL learning through EAs. The experimental results on the MCNC and GSRC benchmarks demonstrate that CORE outperforms other strong baselines in terms of wirelength and area utilization metrics, achieving a 12.9\% improvement in wirelength. CORE represents the first evolutionary reinforcement learning (ERL) algorithm for floorplanning, surpassing existing RL-based methods. The code is available at https://github.com/yeshenpy/CORE.
Volume Transmission Implements Context Factorization to Target Online Credit Assignment and Enable Compositional Generalization
Authors: Matthew Storm Bull; Po-Chen Kuo; Andrew Lawrence Smith; Michael A Buice
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: The modern connectivist framing of neural computation emphasizes the primacy of synaptic communication at the risk of neglecting the influence of the surrounding neuromodulatory environment --- a neuron's 'biophysical context.' Decades of experimental work has established two views of neuromodulatory (NMs) influence: 1) NMs significantly alter circuit dynamics and 2) NMs gate synaptic plasticity, acting as a 'third factor' in learning. Here, we unify these perspectives, proposing that neuromodulation via volume transmission implements a powerful computational principle: context factorization. We derive an endogenously neuromodulated Recurrent Neural Network (e-nmRNN) from a rate reduction of NM release, showing how NM concentrations dynamically factorize network connectivity. This framework reveals how multiplicative NM gating distinctly influences dynamical regimes compared to additive input. Crucially, this context factorization enables targeted online credit assignment: learning rules derived for the e-nmRNN are naturally gated by NM concentrations, localizing updates to relevant contexts. We demonstrate that e-nmRNN dynamics can learn to approximate gradient descent, facilitating rapid in-context adaptation akin to meta-learning. Empirically, e-nmRNNs achieve strong compositional generalization in sequence-to-sequence tasks, outperforming baselines and exhibiting greater hyperparameter robustness. Furthermore, when trained on complex multitasking benchmarks, e-nmRNNs develop emergent properties mirroring biological observations, including modularity, cell-type specialization based on NM release, and distinct neuromodulatory timescales encoding task context. The model's interpretability allows us to reverse engineer these emergent structures. Notably, in reinforcement learning tasks, the e-nmRNN learns to encode context and signals like Reward Prediction Error (RPE) within its neuromodulator dynamics, demonstrating a mechanism for RPE-gated online credit assignment essential for learning how to learn. By bridging biophysical mechanisms with computational principles and empirical validation, our work presents e-nmRNNs as a performant, interpretable model for understanding the computational role of neuromodulation in flexible and compositional learning.
DiffBreak: Is Diffusion-Based Purification Robust?
Authors: Andre Kassis; Urs Hengartner; Yaoliang Yu
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Diffusion-based purification (DBP) has become a cornerstone defense against adversarial examples (AEs), regarded as robust due to its use of diffusion models (DMs) that project AEs onto the natural data manifold. We refute this core claim, theoretically proving that gradient-based attacks effectively target the DM rather than the classifier, causing DBP's outputs to align with adversarial distributions. This prompts a reassessment of DBP's robustness, accrediting it two critical factors: inaccurate gradients and improper evaluation protocols that test only a single random purification of the AE. We show that when accounting for stochasticity and resubmission risk, DBP collapses. To support this, we introduce DiffBreak, the first reliable toolkit for differentiation through DBP, eliminating gradient mismatches that previously further inflated robustness estimates. We also analyze the current defense scheme used for DBP where classification relies on a single purification, pinpointing its inherent invalidity. We provide a statistically grounded majority-vote (MV) alternative that aggregates predictions across multiple purified copies, showing partial but meaningful robustness gain. We then propose a novel adaptation of an optimization method against deepfake watermarking, crafting systemic perturbations that defeat DBP even under MV, challenging DBP's viability.