GenAI4ELab/papercli-papers huggingface dataset:
3DID: Direct 3D Inverse Design for Aerodynamics with Physics-Aware Optimization
Authors: Yuze Hao; Linchao Zhu; Yi Yang
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Inverse design aims to design the input variables of a physical system to optimize
a specified objective function, typically formulated as a search or optimization
problem. However, in 3D domains, the design space grows exponentially, rendering
exhaustive grid-based searches infeasible. Recent advances in deep learning have
accelerated inverse design by providing powerful generative priors and differentiable surrogate models. Nevertheless, current methods tend to approximate the
3D design space using 2D projections or fine-tune existing 3D shapes. These
approaches sacrifice volumetric detail and constrain design exploration, preventing
true 3D design from scratch. In this paper, we propose a 3D Inverse Design (3DID)
framework that directly navigates the 3D design space by coupling a continuous
latent representation with a physics-aware optimization strategy. We first learn a
unified physics–geometry embedding that compactly captures shape and physical
field data in a continuous latent space. Then, we introduce a two-stage strategy to
perform physics-aware optimization. In the first stage, a gradient-guided diffusion
sampler explores the global latent manifold. In the second stage, an objectivedriven, topology-preserving refinement further sculpts each candidate toward the
target objective. This enables 3DID to generate high-fidelity 3D geometries, outperforming existing methods in both solution quality and design versatility.
Accelerating Visual-Policy Learning through Parallel Differentiable Simulation
Authors: Haoxiang You; Yilang Liu; Ian Abraham
Venue: NeurIPS 2025 (NeurIPS 2025 spotlight)

Abstract: In this work, we propose a computationally efficient algorithm for visual policy learning that leverages differentiable simulation and first-order analytical policy gradients.
Our approach decouple the rendering process from the computation graph, enabling seamless integration with existing differentiable simulation ecosystems without the need for specialized differentiable rendering software.
This decoupling not only reduces computational and memory overhead but also effectively attenuates the policy gradient norm, leading to more stable and smoother optimization.  
We evaluate our method on standard visual control benchmarks using modern GPU-accelerated simulation. 
Experiments show that our approach significantly reduces wall-clock training time and consistently outperforms all baseline methods in terms of final returns.
Notably, on complex tasks such as humanoid locomotion, our method achieves a $4\times$ improvement in final return, and successfully learns a humanoid running policy within 4 hours on a single GPU.
Videos and code are available on https://haoxiangyou.github.io/Dva_website
Online Prediction with Limited Selectivity
Authors: Licheng Liu; Mingda Qiao
Venue: NeurIPS 2025 (NeurIPS 2025 spotlight)

Abstract: Selective prediction [Dru13, QV19] models the scenario where a forecaster freely decides on the prediction window that their forecast spans. Many data statistics can be predicted to a non-trivial error rate *without any* distributional assumptions or expert advice, yet these results rely on that the forecaster may predict at any time. We introduce a model of Prediction with Limited Selectivity (PLS) where the forecaster can start the prediction only on a subset of the time horizon. We study the optimal prediction error both on an instance-by-instance basis and via an average-case analysis. We introduce a complexity measure that gives instance-dependent bounds on the optimal error. For a randomly-generated PLS instance, these bounds match with high probability.
Do different prompting methods yield a common task representation in language models?
Authors: Guy Davidson; Todd M. Gureckis; Brenden Lake; Adina Williams
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Demonstrations and instructions are two primary approaches for prompting language models to perform in-context learning (ICL) tasks.
Do identical tasks elicited in different ways result in similar representations of the task? An improved understanding of task representation mechanisms would offer interpretability insights and may aid in steering models. We study this through function vectors (FVs), recently proposed as a mechanism to extract few-shot ICL task representations. We generalize FVs to alternative task presentations, focusing on short textual instruction prompts, and successfully extract instruction function vectors that promote zero-shot task accuracy. We find evidence that demonstration- and instruction-based function vectors leverage different model components, and offer several controls to dissociate their contributions to task performance. Our results suggest that different task prompting forms do not induce a common task representation through FVs but elicit different, partly overlapping mechanisms. Our findings offer principled support to the practice of combining instructions and task demonstrations, imply challenges in universally monitoring task inference across presentation forms, and encourage further examinations of LLM task inference mechanisms.
Aligning What Matters: Masked Latent Adaptation for Text-to-Audio-Video Generation
Authors: Jiyang Zheng; Siqi Pan; Yu Yao; Zhaoqing Wang; Dadong Wang; Tongliang Liu
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Text-to-Audio-Video (T2AV) generation aims to produce temporally and semantically aligned visual and auditory content from natural language descriptions. While recent progress in text-to-audio and text-to-video models has improved generation quality within each modality, jointly modeling them remains challenging due to incomplete and asymmetric correspondence: audio often reflects only a subset of the visual scene, and vice versa. Naively enforcing full alignment introduces semantic noise and temporal mismatches. To address this, we propose a novel framework that performs selective cross-modal alignment through a learnable masking mechanism, enabling the model to isolate and align only the shared latent components relevant to both modalities. This mechanism is integrated into an adaptation module that interfaces with pretrained encoders and decoders from latent video and audio diffusion models, preserving their generative capacity with reduced training overhead. Theoretically, we show that our masked objective provably recovers the minimal set of shared latent variables across modalities. Empirically, our method achieves state-of-the-art performance on standard T2AV benchmarks, demonstrating significant improvements in audiovisual synchronization and semantic consistency.
Let Them Talk: Audio-Driven Multi-Person Conversational Video Generation
Authors: Zhe Kong; Feng Gao; Yong Zhang; Zhuoliang Kang; Xiaoming Wei; Xunliang Cai; Guanying Chen; Wenhan Luo
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Audio-driven human animation methods, such as talking head and talking body generation, have made remarkable progress in generating synchronized facial movements and appealing visual quality videos. However, existing methods primarily focus on single human animation and struggle with multi-stream audio inputs, facing incorrect binding problems between audio and persons. Additionally, they exhibit limitations in instruction-following capabilities. To solve this problem, in this paper, we propose a novel task: Multi-Person Conversational Video Generation, and introduce a new framework, MultiTalk, to address the challenges during multi-person generation. Specifically, for audio injection, we investigate several schemes and propose the Label Rotary Position Embedding (L-RoPE) method to resolve the audio and person binding problem. Furthermore, during training, we observe that partial parameter training and multi-task training are crucial for preserving the instruction-following ability of the base model. MultiTalk achieves superior performance compared to other methods on several datasets, including talking head, talking body, and multi-person datasets, demonstrating the powerful generation capabilities of our approach.
Gemstones: A Model Suite for Multi-Faceted Scaling Laws
Authors: Sean Michael McLeish; John Kirchenbauer; David Yu Miller; Siddharth Singh; Abhinav Bhatele; Micah Goldblum; Ashwinee Panda; Tom Goldstein
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Scaling laws are typically fit using a family of models with a narrow range of frozen hyperparameter choices. 
In this work we study scaling laws using multiple architectural shapes and hyperparameter choices, highlighting their impact on resulting prescriptions.
As a primary artifact of our research, we release the Gemstones: an open-source scaling law dataset, consisting of over 4000 checkpoints from transformers with up to 2 billion parameters and diverse architectural shapes; including ablations over learning rate and cooldown.
Our checkpoints enable more complex studies of scaling, such as analyzing the relationship between width and depth.
By examining our model suite, we find that the prescriptions of scaling laws can be highly sensitive to the experimental design process and the specific model checkpoints used during fitting.
Metritocracy: Representative Metrics for Lite Benchmarks
Authors: Ariel D. Procaccia; Benjamin Schiffer; Serena Lutong Wang; Shirley Zhang
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: A common problem in LLM evaluation is how to choose a subset of metrics from a full suite of possible metrics. Subset selection is usually done for efficiency or interpretability reasons, and the goal is often to select a "representative" subset of metrics. However, "representative" is rarely clearly defined. In this work, we use ideas from social choice theory to formalize two notions of representation for the selection of a subset of evaluation metrics. We first introduce *positional representation*, which guarantees every alternative is sufficiently represented at every position cutoff. We then introduce *positional proportionality*, which guarantees no alternative is proportionally over- or under-represented by more than a small error at any position. We prove upper and lower bounds on the smallest number of metrics needed to guarantee either of these properties in the worst case. We also study a generalized form of each property that allows for additional input on groups of metrics that must be represented. Finally, we tie theory to practice through real-world case studies on both LLM evaluation and hospital quality evaluation.
BiggerGait: Unlocking Gait Recognition with Layer-wise Representations from Large Vision Models
Authors: Dingqiang Ye; Chao Fan; Zhanbo Huang; Chengwen Luo; Jianqiang Li; Shiqi Yu; Xiaoming Liu
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Large vision models (LVM) based gait recognition has achieved impressive performance.
However, existing LVM-based approaches may overemphasize gait priors while neglecting the intrinsic value of LVM itself, particularly the rich, distinct representations across its multi-layers. 
To adequately unlock LVM's potential, this work investigates the impact of layer-wise representations on downstream recognition tasks.
Our analysis reveals that LVM's intermediate layers offer complementary properties across tasks, integrating them yields an impressive improvement even without rich well-designed gait priors.
Building on this insight, we propose a simple and universal baseline for LVM-based gait recognition, termed BiggerGait.
Comprehensive evaluations on CCPG, CAISA-B*, SUSTech1K, and CCGR_MINI validate the superiority of BiggerGait across both within- and cross-domain tasks, establishing it as a simple yet practical baseline for gait representation learning.
All the models and code are available at https://github.com/ShiqiYu/OpenGait/.
Uncertainty Estimation on Graphs with Structure Informed Stochastic Partial Differential Equations
Authors: Fred Xu; Thomas Markovich
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Graph Neural Networks (GNNs) have achieved impressive results across diverse network modeling tasks, but accurately estimating uncertainty on graphs remains difficult—especially under  distributional shifts. Unlike traditional uncertainty estimation, graph-based uncertainty must account for randomness arising from both the graph’s structure and its label distribution, which adds complexity. In this paper, making an analogy between the evolution of a stochastic partial differential equation (SPDE) driven by Mat\'ern Gaussian Process and message passing using GNN layers, we present a principled way to design a novel message passing scheme that incorporates spatial-temporal noises motivated by the Gaussian Process approach to SPDE. Our method simultaneously captures uncertainty across space and time and allows explicit control over the covariance kernel’s smoothness, thereby enhancing uncertainty estimates on graphs with both low and high label informativeness. Our extensive experiments on Out-of-Distribution (OOD) detection on graph datasets with varying label informativeness demonstrate the soundness and superiority of our model to existing approaches.
Exploring Diffusion Transformer Designs via Grafting
Authors: Keshigeyan Chandrasegaran; Michael Poli; Daniel Y Fu; Dongjun Kim; Lea M. Hadzic; Manling Li; Agrim Gupta; Stefano Massaroli; Azalia Mirhoseini; Juan Carlos Niebles; Stefano Ermon; Li Fei-Fei
Venue: NeurIPS 2025 (NeurIPS 2025 oral)

Abstract: Designing model architectures requires decisions such as selecting operators (e.g., attention, convolution) and configurations (e.g., depth, width). However, evaluating the impact of these decisions on model quality requires costly pretraining, limiting architectural investigation.
Inspired by how new software is built on existing code, we ask: can new architecture designs be studied using pretrained models? To this end, we present *grafting*, a simple approach for editing pretrained diffusion transformers (DiTs) to materialize new architectures under small compute budgets. Informed by our analysis of activation behavior and attention locality, we construct a testbed based on the DiT-XL/2 design to study the impact of grafting on model quality. Using this testbed, we develop a family of hybrid designs via grafting: replacing softmax attention with gated convolution, local attention, and linear attention, and replacing MLPs with variable expansion ratio and convolutional variants. Notably, many hybrid designs achieve good quality (FID: 2.38–2.64 vs. 2.27 for DiT-XL/2)
using $<2$% pretraining compute. We then graft a text-to-image model (PixArt-$\Sigma$), achieving a 1.43$\times$ speedup with less than a 2% drop in GenEval score. Finally, we present a case study that restructures DiT-XL/2 by converting every pair of sequential transformer blocks into parallel blocks via grafting. This reduces model depth by 2$\times$ and yields better quality (FID: 2.77) than other models of comparable depth. Together, we show that new diffusion model designs can be explored by grafting pretrained DiTs, with edits ranging from operator replacement to architecture restructuring. Code and grafted models: https://grafting.stanford.edu.
Generative Trajectory Stitching through Diffusion Composition
Authors: Yunhao Luo; Utkarsh Aashu Mishra; Yilun Du; Danfei Xu
Venue: NeurIPS 2025 (NeurIPS 2025 spotlight)

Abstract: Effective trajectory stitching for long-horizon planning is a significant challenge in robotic decision-making. While diffusion models have shown promise in planning, they are limited to solving tasks similar to those seen in their training data. We propose CompDiffuser, a novel generative approach that can solve new tasks by learning to compositionally stitch together shorter trajectory chunks from previously seen tasks. Our key insight is modeling the trajectory distribution by subdividing it into overlapping chunks and learning their conditional relationships through a single bidirectional diffusion model. This allows information to propagate between segments during generation, ensuring physically consistent connections. We conduct experiments on benchmark tasks of various difficulties, covering different environment sizes, agent state dimension, trajectory types, training data quality, and show that CompDiffuser significantly outperforms existing methods.
Probabilistic Token Alignment for Large Language Model Fusion
Authors: Runjia Zeng; James Chenhao Liang; Cheng Han; Zhiwen Cao; Jiahao Liu; Xiaojun Quan; Yingjie Victor Chen; Lifu Huang; Tong Geng; Qifan Wang; Dongfang Liu
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Training large language models (LLMs) from scratch can yield models with unique functionalities and strengths, but it is costly and often leads to redundant capabilities. A more cost-effective alternative is to fuse existing pre-trained LLMs with different architectures into a more powerful model. However, a key challenge in existing model fusion is their dependence on manually predefined vocabulary alignment, which may not generalize well across diverse contexts, leading to performance degradation in several evaluation. To solve this, we draw inspiration from distribution learning and propose the probabilistic token alignment method as a general and soft mapping for alignment, named as PTA-LLM. Our approach innovatively reformulates token alignment into a classic mathematical problem: optimal transport, seamlessly leveraging distribution-aware learning to facilitate more coherent model fusion. Apart from its inherent generality, PTA-LLM exhibits interpretability from a distributional perspective, offering insights into the essence of the token alignment. Empirical results demonstrate that probabilistic token alignment enhances the target model's performance across multiple capabilities.
VLMs have Tunnel Vision: Evaluating Nonlocal Visual Reasoning in Leading VLMs
Authors: Shmuel Berman; Jia Deng
Venue: NeurIPS 2025 (NeurIPS 2025 spotlight)

Abstract: Vision Language Models (VLMs) excel at complex visual tasks such as VQA and chart understanding, yet recent work suggests they struggle with simple perceptual tests. We present an evaluation that tests vision-language models’ capacity for non-local visual reasoning— reasoning that requires chaining evidence collected from multiple, possibly distant, regions of an image.  We isolate three distinct forms of non‑local vision: comparative perception, which demands holding two images in working memory and comparing them; saccadic search, which requires making discrete, evidence‑driven jumps to locate successive targets; and smooth visual search, which involves searching smoothly along a continuous contour. Flagship models (e.g., GPT-5, Gemini 2.5 Pro, Claude Sonnet 4), even those that perform well on prior primitive‑vision benchmarks, fail these tests and barely exceed random accuracy on two variants of our tasks that are trivial for humans. Our structured evaluation suite allows us to test if VLMs can perform similar visual algorithms to humans.  Our findings show that despite gains in raw visual acuity, current models lack core visual reasoning capabilities.
A Generalized Binary Tree Mechanism for Private Approximation of All-Pair Shortest Distances
Authors: Zongrui Zou; Chenglin Fan; Michael Dinitz; Jingcheng Liu; Jalaj Upadhyay
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: We study the problem of approximating all-pair distances in a weighted undirected graph with differential privacy, introduced by Sealfon [Sea16]. Given a publicly known undirected graph, we treat the weights of edges as sensitive information, and two graphs are neighbors if their edge weights differ in one edge by at most one. We obtain efficient algorithms with significantly improved bounds on a broad class of graphs which we refer to as *recursively separable*. In particular, for any $n$-vertex $K_h$-minor-free graph, our algorithm achieve an additive error of $ \widetilde{O}(h(nW)^{1/3} ) $, where $ W $ represents the maximum edge weight; For grid graphs, the same algorithmic scheme achieve additive error of $ \widetilde{O}(n^{1/4}\sqrt{W}) $.

Our approach can be seen as a generalization of the celebrated binary tree mechanism for range queries, as releasing range queries is equivalent to computing all-pair distances on a path graph. In essence, our approach is based on generalizing the binary tree mechanism to graphs that are *recursively separable*.
Dynamic Diameter in High-Dimensions against Adaptive Adversary and Beyond
Authors: Kiarash Banihashem; Jeff Giliberti; Samira Goudarzi; MohammadTaghi Hajiaghayi; Peyman Jabbarzade; Morteza Monemizadeh
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: In this paper, we study the fundamental problems of maintaining the diameter and a $k$-center clustering of a dynamic point set $P \subset \mathbb{R}^d$, where points may be inserted or deleted over time and the ambient dimension $d$ is not constant and may be high. Our focus is on designing algorithms that remain effective even in the presence of an \emph{adaptive adversary}—an adversary that, at any time $t$, knows the entire history of the algorithm’s outputs as well as all the random bits used by the algorithm up to that point. We present a fully dynamic algorithm that maintains a $2$-approximate diameter with a \emph{worst-case} update time of $poly(d, \log n)$, where $n$ is the length of the stream. Our result is achieved by identifying a robust representative of the dataset that requires infrequent updates, combined with a careful deamortization. To the best of our knowledge, this is the first efficient fully-dynamic algorithm for diameter in high dimensions that \emph{simultaneously} achieves a $2$-approximation guarantee and robustness against an adaptive adversary. We also give an improved dynamic $(4+\epsilon)$-approximation algorithm for the $k$-center problem, also resilient to an adaptive adversary. Our clustering algorithm achieves an amortized update time of $k^{2.5} d \cdot poly(\epsilon^{-1}, \log n)$, improving upon the amortized update time of $k^6 d \cdot poly( \epsilon^{-1}, \log n)$ by Biabani et al. [NeurIPS'24].