GenAI4ELab/papercli-papers huggingface dataset:
Globally Optimal Policy Gradient Algorithms for Reinforcement Learning with PID Control Policies
Authors: Vipul Kumar Sharma; Wesley A. Suttle; S Sivaranjani
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: We develop policy gradient algorithms with global optimality and convergence guarantees for reinforcement learning (RL) with proportional-integral-derivative (PID) parameterized control policies. RL enables learning control policies through direct interaction with a system, without explicit model knowledge that is typically assumed in classical control. The PID policy architecture offers built-in structural advantages, such as superior tracking performance, elimination of steady-state errors, and robustness to model error that have made it a widely adopted paradigm in practice. Despite these advantages, the PID parameterization has received limited attention in the RL literature, and PID control designs continue to rely on heuristic tuning rules without theoretical guarantees. We address this gap by rigorously integrating PID control with RL, offering theoretical guarantees while maintaining the practical advantages that have made PID control ubiquitous in practice. Specifically, we first formulate PID control design as an optimization problem with a control policy that is parameterized by proportional, integral, and derivative components. We derive exact expressions for policy gradients in these parameters, and leverage them to develop both model-based and model-free policy gradient algorithms for PID policies. We then establish gradient dominance properties of the PID policy optimization problem, and provide theoretical guarantees on convergence and global optimality in this setting. Finally, we benchmark the performance of our algorithms on the controlgym suite of environments.
Conformal Prediction under Lévy-Prokhorov Distribution Shifts: Robustness to Local and Global Perturbations
Authors: Liviu Aolaritei; Qianyu Julie Zhu; Zheyu Oliver Wang; Michael I. Jordan; Youssef Marzouk
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Conformal prediction provides a powerful framework for constructing prediction intervals with finite-sample guarantees, yet its robustness under distribution shifts remains a significant challenge. This paper addresses this limitation by modeling distribution shifts using Lévy-Prokhorov (LP) ambiguity sets, which capture both local and global perturbations. We provide a self-contained overview of LP ambiguity sets and their connections to popular metrics such as Wasserstein and Total Variation. We show that the link between conformal prediction and LP ambiguity sets is a natural one: by propagating the LP ambiguity set through the scoring function, we reduce complex high-dimensional distribution shifts to manageable one-dimensional distribution shifts, enabling exact quantification of worst-case quantiles and coverage. Building on this analysis, we construct robust conformal prediction intervals that remain valid under distribution shifts, explicitly linking LP parameters to interval width and confidence levels. Experimental results on real-world datasets demonstrate the effectiveness of the proposed approach.
Nyström-Accelerated Primal LS-SVMs: Breaking the $O(an^3)$ Complexity Bottleneck for Scalable ODEs Learning
Authors: Weikuo Wang; Yue Liao; Huan Luo
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: A major problem of kernel-based methods (e.g., least squares support vector machines, LS-SVMs) for solving linear/nonlinear ordinary differential equations (ODEs) is the prohibitive $O(an^3)$ ($a=1$ for linear ODEs and 27 for nonlinear ODEs) part of their computational complexity with increasing temporal discretization points $n$. We propose a novel Nyström-accelerated LS-SVMs framework that breaks this bottleneck by reformulating ODEs as primal-space constraints. Specifically, we derive for the first time an explicit Nyström-based mapping and its derivatives from one-dimensional temporal discretization points to a higher $m$-dimensional feature space ($1< m\le n$), enabling the learning process to solve linear/nonlinear equation systems with $m$-dependent complexity. Numerical experiments on sixteen benchmark ODEs demonstrate: 1) $10-6000$ times faster computation than classical LS-SVMs and physics-informed neural networks (PINNs), 2) comparable accuracy to LS-SVMs ($<0.13\%$ relative MAE, RMSE, and $\left \| y-\hat{y}  \right \| _{\infty } $difference)  while maximum surpassing PINNs by 72\% in RMSE, and 3) scalability to $n=10^4$ time steps with $m=50$ features. This work establishes a new paradigm for efficient kernel-based ODEs learning without significantly sacrificing the accuracy of the solution.
Reasoning as an Adaptive Defense for Safety
Authors: Taeyoun Kim; Fahim Tajwar; Aditi Raghunathan; Aviral Kumar
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Reasoning methods that adaptively allocate test-time compute have advanced LLM performance on easy to verify domains such as math and code. In this work, we study how to utilize this approach to train models that exhibit a degree of robustness to safety vulnerabilities, and show that doing so can provide benefits. We build a recipe called $\textit{\textbf{TARS}}$ (Training Adaptive Reasoners for Safety), a reinforcement learning (RL) approach that trains models to reason about safety using chain-of-thought traces and a reward signal that balances safety with task completion. To build TARS, we identify three critical design choices: (1) a ``lightweight'' warmstart SFT stage, (2) a mix of harmful, harmless, and ambiguous prompts to prevent shortcut behaviors such as too many refusals, and (3) a reward function to prevent degeneration of reasoning capabilities during training. Models trained with TARS exhibit adaptive behaviors by spending more compute on ambiguous queries, leading to better safety-refusal trade-offs. They also internally learn to better distinguish between safe and unsafe prompts and attain greater robustness to both white-box (e.g., GCG) and black-box attacks (e.g., PAIR). Overall, our work provides an effective, open recipe for training LLMs against jailbreaks and harmful requests by reasoning per prompt.
Selective Omniprediction and Fair Abstention
Authors: Sílvia Casacuberta; Varun Kanade
Venue: NeurIPS 2025 (NeurIPS 2025 spotlight)

Abstract: We propose new learning algorithms for building selective classifiers, which are predictors that are allowed to abstain on some fraction of the domain. We study the model where a classifier may abstain from predicting at a fixed cost. Building on the recent framework on multigroup fairness and omniprediction, given a pre-specified class of loss functions, we provide an algorithm for building a single classifier that learns abstentions and predictions optimally for every loss in the entire class, where the abstentions are decided efficiently for each specific loss function by applying a fixed post-processing function. Our algorithm and theoretical guarantees generalize the previously-known algorithms for learning selective classifiers in formal learning-theoretic models.
    
We then extend the traditional multigroup fairness algorithms to the selective classification setting and show that we can use a calibrated and multiaccurate predictor to efficiently build selective classifiers that abstain optimally not only globally but also locally within each of the groups in any pre-specified collection of possibly intersecting subgroups of the domain, and are also accurate when they do not abstain. We show how our abstention algorithms can be used as conformal prediction methods in the binary classification setting to achieve both marginal and group-conditional coverage guarantees for an intersecting collection of groups. We provide empirical evaluations for all of our theoretical results, demonstrating the practicality of our learning algorithms for abstaining optimally and fairly.
Partner Modelling Emerges in Recurrent Agents (But Only When It Matters)
Authors: Ruaridh Mon-Williams; Max Taylor-Davies; Elizabeth Mieczkowski; Natalia Vélez; Neil R Bramley; Yanwei Wang; Thomas L. Griffiths; Christopher G. Lucas
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Humans are remarkably adept at collaboration, able to infer the strengths and weaknesses of new partners in order to work successfully towards shared goals. To build AI systems with this capability, we must first understand its building blocks: does such flexibility require explicit, dedicated mechanisms for modelling others—or can it emerge spontaneously from the pressures of open-ended cooperative interaction? To investigate this question, we train simple model-free RNN agents to collaborate with a population of diverse partners. Using the 'Overcooked-AI' environment, we collect data from thousands of collaborative teams, and analyse agents' internal hidden states. Despite a lack of additional architectural features, inductive biases, or auxiliary objectives, the agents nevertheless develop structured internal representations of their partners' task abilities, enabling rapid adaptation and generalisation to novel collaborators. We investigated these internal models through probing techniques, and large-scale behavioural analysis. Notably, we find that structured partner modelling emerges when agents can influence partner behaviour by controlling task allocation. Our results show that partner modelling can arise spontaneously in model-free agents—but only under environmental conditions that impose the right kind of social pressure.
Blending Complementary Memory Systems in Hybrid Quadratic-Linear Transformers
Authors: Kazuki Irie; Morris Yau; Samuel J. Gershman
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: We develop hybrid memory architectures for general-purpose sequence processing neural networks, that combine key-value memory using softmax attention (KV-memory) with fast weight memory through dynamic synaptic modulation (FW-memory)---the core principles of quadratic and linear transformers, respectively. These two memory systems have complementary but individually limited properties: KV-memory offers precise retrieval but is constrained by quadratic complexity in sequence length, while FW-memory supports arbitrarily long sequences and enables more expressive computation but sacrifices precise recall. We propose and compare three methods to blend these two systems into a single memory system, differing in how and when input information is delivered to each system, to leverage the strengths of both. We conduct experiments on general language modeling and retrieval tasks by training 340M- and 1.3B-parameter models from scratch, as well as on synthetic algorithmic tasks designed to precisely illustrate the benefits of certain hybrid methods over others. We also evaluate our hybrid memory systems on reinforcement learning in partially observable environments. Overall, we demonstrate how a well-designed hybrid can overcome the limitations of its individual components, offering new insights into the design principle of neural memory systems.
Graph Neural Network Based Action Ranking for Planning
Authors: Rajesh Devaraddi Mangannavar; Stefan Lee; Alan Fern; Prasad Tadepalli
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: We propose a novel approach to learn relational policies for classical planning based on learning to rank actions. We introduce a new graph representation that explicitly captures action information and propose a Graph Neural Network (GNN) architecture augmented with Gated Recurrent Units (GRUs) to learn action rankings. Unlike value-function based approaches that must learn a globally consistent function, our action ranking method only needs to learn locally consistent ranking. Our model is trained on data generated from small problem instances that are easily solved by planners and is applied to significantly larger instances where planning is computationally prohibitive. Experimental results across standard planning benchmarks demonstrate that our action-ranking approach not only achieves better generalization to larger problems than those used in training but also outperforms multiple baselines (value function and action ranking) methods in terms of success rate and plan quality.
Length Generalization via Auxiliary Tasks
Authors: Pranjal Awasthi; Anupam Gupta; Ravi Kumar
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: _Length generalization_, the ability of sequence models to
  generalize to sequences longer than those encountered during
  training, remains a key challenge for transformers,
  especially in tasks requiring algorithmic reasoning. Existing
  theoretical understanding of length generalization is limited, often providing
  only asymptotic results or focusing on specific problem classes or
  architectural variants, while empirical approaches frequently rely on
  ad hoc and often fragile techniques.

  In this work we introduce a novel framework for analyzing and
  proving length generalization bounds under specified, verifiable assumptions. A key
  outcome of the theory is the identification of a natural set of
  _auxiliary_ tasks, intricately related to the primary task structure,
  such that strong performance on these auxiliary tasks, alongside the
  primary task, provably guarantees length generalization within the framework.  This
  motivates a multi-task training procedure that explicitly optimizes
  performance on both the primary and the identified auxiliary tasks.

  Empirical evaluations on a variety of synthetic benchmarks
  known to be challenging for length generalization, including sequence
  sorting, and reversal, demonstrate that our proposed method yields
  significant improvements in generalization to substantially longer
  sequences.
Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptive
Authors: Tyler Farghly; Peter Potaptchik; Samuel Howard; George Deligiannidis; Jakiw Pidstrigach
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Diffusion models have achieved state-of-the-art performance, demonstrating remarkable generalisation capabilities across diverse domains. However, the mechanisms underpinning these strong capabilities remain only partially understood. A leading conjecture, based on the manifold hypothesis, attributes this success to their ability to adapt to low-dimensional geometric structure within the data. This work provides evidence for this conjecture, focusing on how such phenomena could result from the formulation of the learning problem through score matching. We inspect the role of implicit regularisation by investigating the effect of smoothing minimisers of the empirical score matching objective. Our theoretical and empirical results confirm that smoothing the score function—or equivalently, smoothing in the log-density domain—produces smoothing tangential to the data manifold. In addition, we show that the manifold along which the diffusion model generalises can be controlled by choosing an appropriate smoothing.
Mitigating Hallucination Through Theory-Consistent Symmetric Multimodal Preference Optimization
Authors: Wenqi Liu; Xuemeng Song; Jiaxi Li; Yinwei Wei; Na Zheng; Jianhua Yin; Liqiang Nie
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Direct Preference Optimization (DPO) has emerged as an effective approach for mitigating hallucination in Multimodal Large Language Models (MLLMs). Although existing methods have achieved significant progress by utilizing vision-oriented contrastive objectives for enhancing MLLMs' attention to visual inputs and hence reducing hallucination, they suffer from non-rigorous optimization objective function and indirect preference supervision. To address these limitations, we propose a Symmetric Multimodal Preference Optimization (SymMPO), which conducts symmetric preference learning with direct preference supervision (i.e., response pairs) for visual understanding enhancement, while maintaining rigorous theoretical alignment with standard DPO. In  addition to conventional ordinal preference learning, SymMPO introduces a preference margin consistency loss to quantitatively regulate the preference gap between symmetric preference pairs. Comprehensive evaluation across five benchmarks demonstrate SymMPO's superior performance, validating its effectiveness in hallucination mitigation of MLLMs.
Efficient Kernelized Learning in Polyhedral Games beyond Full Information: From Colonel Blotto to Congestion Games
Authors: Andreas Kontogiannis; Vasilis Pollatos; Gabriele Farina; Panayotis Mertikopoulos; Ioannis Panageas
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: We examine the problem of efficiently learning coarse correlated equilibria (CCE) in polyhedral games, that is, normal-form games with an exponentially large number of actions per player and an underlying combinatorial structure—such as the classic Colonel Blotto game or congestion games. Achieving computational efficiency in this setting requires learning algorithms whose regret and per-iteration complexity scale at most polylogarithmically with the size of the players’ action sets. This challenge has recently been addressed in the full-information setting, primarily through the use of kernelization; however, in the more realistic partial information setting, the situation is much more challenging, and existing approaches result in suboptimal and impractical runtime complexity to learn CCE. We address this gap via a novel kernelization-based framework for payoff-based learning in polyhedral games, which we then apply to certain key classes of polyhedral games—namely Colonel Blotto, graphic matroid and network congestion games. In so doing, we obtain a range of computationally efficient payoff-based learning algorithms which significantly improve upon prior work in terms of the runtime for learning CCE.
Adjusted Count Quantification Learning on Graphs
Authors: Clemens Damke; Eyke Hüllermeier
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: *Quantification learning* is the task of predicting the label distribution of a set of instances.
We study this problem in the context of graph-structured data, where the instances are vertices.
Previously, this problem has only been addressed via node clustering methods.
In this paper, we extend the popular *Adjusted Classify & Count* (ACC) method to graphs.
We show that the prior probability shift assumption upon which ACC relies is often not applicable to graph quantification problems.
To address this issue, we propose structural importance sampling (SIS), the first graph quantification method that is applicable under (structural) covariate shift.
Additionally, we propose Neighborhood-aware ACC, which improves quantification in the presence of non-homophilic edges.
We show the effectiveness of our techniques on multiple graph quantification tasks.
Controlling The Spread of Epidemics on Networks with Differential Privacy
Authors: Dung Nguyen; Aravind Srinivasan; Renata Valieva; Anil Vullikanti; Jiayi Wu
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Designing effective strategies for controlling epidemic spread by vaccination is an important question in epidemiology, especially in the early stages when vaccines are limited.
This is a challenging question when the contact network is very heterogeneous, and strategies based on controlling network properties, such as the degree and spectral radius, have been shown to be effective.
Implementation of such strategies requires detailed information on the contact structure, which might be sensitive in many applications.
Our focus here is on choosing effective vaccination strategies when the edges are sensitive and differential privacy guarantees are needed.
Our main contributions are $(\varepsilon,\delta)$-differentially private algorithms for designing vaccination strategies by reducing the maximum degree and spectral radius.
Our key technique is a private algorithm for the multi-set multi-cover problem, which we use for controlling network properties.
We evaluate privacy-utility tradeoffs of our algorithms on multiple synthetic and real-world networks, and show their effectiveness.
Deep RL Needs Deep Behavior Analysis: Exploring Implicit Planning by Model-Free Agents in Open-Ended Environments
Authors: Riley Simmons-Edler; Ryan Paul Badman; Felix Baastad Berg; Raymond Chua; John Vastola; Joshua Lunger; William Qian; Kanaka Rajan
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Understanding the behavior of deep reinforcement learning (DRL) agents—particularly as task and agent sophistication increase—requires more than simple comparison of reward curves, yet standard methods for behavioral analysis remain underdeveloped in DRL.
We apply tools from neuroscience and ethology to study DRL agents in a novel, complex, partially observable environment, ForageWorld, designed to capture key aspects of real-world animal foraging—including sparse, depleting resource patches, predator threats, and spatially extended arenas.
We use this environment as a platform for applying joint behavioral and neural analysis to agents, revealing detailed, quantitatively grounded insights into agent strategies, memory, and planning.
Contrary to common assumptions, we find that model-free RNN-based DRL agents can exhibit structured, planning-like behavior purely through emergent dynamics—without requiring explicit memory modules or world models.
Our results show that studying DRL agents like animals—analyzing them with neuroethology-inspired tools that reveal structure in both behavior and neural dynamics—uncovers rich structure in their learning dynamics that would otherwise remain invisible.
We distill these tools into a general analysis framework linking core behavioral and representational features to diagnostic methods, which can be reused for a wide range of tasks and agents.
As agents grow more complex and autonomous, bridging neuroscience, cognitive science, and AI will be essential—not just for understanding their behavior, but for ensuring safe alignment and maximizing desirable behaviors that are hard to measure via reward.
We show how this can be done by drawing on lessons from how biological intelligence is studied.